Automate pulling of RSS statistics from the website

[ehelms]

Infrastructure SIG notes:

• Currently a manual process to login to web01 every 3 weeks and pull the data
• Blocks moving the website to the CDN as these stats would be lost (see osuosl migration: logging for fastly #1465)
• A dashboard would be nice
• We should automate the pulling process

While working on this, privacy of users should be respected. We only log a very minimal amount (just the Foreman version). The IPs are hashed during processing but they are needed to check for uniqueness of instances.

Action Items

• Add automation to pull RSS statistics
• Add graphing for a nice dashboard -- Moved here Publish stats based on RSS feed data in the newsletter #1699

[ekohl]

Blocks moving the website to the CDN as these stats would be lost

See #1465 for that.

[evgeni]

b125154 contains the automation of RSS data pulling,

the dashboard is technically a new feature, which is kinda outside of the "automate pulling" scope?

[ekohl]

It would be good to wait until the cronjob has ran at least once before closing this.

[evgeni]

I did run it once manually, if that helps ;)

[ekohl]

Looks like it did run once:

Dec 15 05:07:16 web01 run-parts(/etc/cron.weekly)[9250]: starting rss-stat
Dec 15 05:07:17 web01 run-parts(/etc/cron.weekly)[9257]: finished rss-stat

However, the timestamps in /var/log/rss-stat indicate no file was created yet since Dec 13. Probably because that manually processed all files already.

I'm a day early with checking.

[ekohl]

Looks like it was created:

-rw-r--r--. 1 root root  987204 Dec 22 06:44 rss-stat-20211219.gz

Given this was said:

A dashboard would be nice

I think we should scope this issue to just the pulling (which is done). If a dashboard or other processing is desired, this should be created as a new issue and I'd be in favor of closing this as resolved now.