Skip to content

webscraping fuel prices in QLD Australia, using github actions

License

Notifications You must be signed in to change notification settings

gsajko/QLD_fuel_scraping

Repository files navigation

QLD scrapping

Data is collected using the cron schedule:

'11 * * * *' Each hour at 11

Monthly data

Each month's fuel price changes are published on: Fuel Price Reporting.

Once downloaded, the file is saved to data/month folder.

Daily data

The daily scrape was made to fill the gap after the last published month (I assume less than 2 months).

Scraped records are saved to a .csv file inside data/week folder. Each week has a separate file in the format [year]_[week].csv.

For example, we have the 31st week of the year 2021, the file will be named 21_31.csv.

You need to request for access token from https://www.fuelpricesqld.com.au/ ("Signing-up as a publisher or data consumer") for weekly data.

Note

If you want to work on historic data older than 2 months, it's better to just use Fuel Price Reporting monthly data.

About

webscraping fuel prices in QLD Australia, using github actions

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published