Skip to content

Latest commit

 

History

History
51 lines (40 loc) · 1.98 KB

README.md

File metadata and controls

51 lines (40 loc) · 1.98 KB
Raw

Is It Getting Hotter or Is It Just Me?

A visualization of historical temperature data and trends

WORK IN PROGRESS

TODO

  • see if usaf values are unique
  • For each year in /data/gsod
    • list tar contents
      • check if file names match listings in database
      • If a significant number of file names don't match, be wary.
    • get tar contents
      • put tar contents into database

Export corrected isd-station and fips-to-country for sharing!

Notes & Caveats

Download GSOD Data

I tried using /scripts/get-gsod.js at first, however LFTP refused to connect when run on my VPS through node. Thus, I ran LFTP manually.

$ cd data/gsod
$ lftp
:~> open ftp://ftp.ncdc.noaa.gov/pub/data/gsod/
:~> mirror -I */*.tar
:~> bye

This downloads 3.7GB of archived Global Summary of the Day data. (according to du -sh data/gsod)

Be careful if you have a slow internet connection.

Mapping Country Ids (in isd station) to country names

fips-to-country has discrepancies. It doesn't map perfectly to the country ids in isd-station. I cross-referenced with fips-to-iso, and google to resolve the discrepancies as best as possible.

Originals can be found in /data/original. My updated versions will be in /data/updated.

isd-problems

There are problems with the data, listed in isd-problems.pdf. Here's a link to the original NOAA file

References

also available in csv