LINZ to OSM Street Address Import (2021 update)

Street address data from Land Information New Zealand (LINZ) was first imported into OpenStreetMap in 2017. Since then, the imported data has become out of date.

This project aims to update the address data, and set up a system to reguarly update addresses in OSM, by conflating them with the data from LINZ.

🚩 This repository also contains the code used to import topographic and hydrographic data from LINZ. For more info, see the the wiki page.

🌏 For more information, please see the wiki page for this project. The rest of this document contains only technical detail. Continue reading if you wish to contribute to the code

The fork of RapiD

This is modified version of the RapiD editor, which is a modified version of the iD editor.

The source code is in a separate repository (see here). It is available at osm-nz.github.io

Operation

If all five status badges at the top of this document are green, then the script is automatically running once a week (on Friday morning NZ time). The results can be viewed and actioned from osm-nz.github.io

Setup

If you want to use the code to manually run the process, follow these steps:

1. Clone this repository
2. Download nodejs v20.11 or later
3. Generate an API from https://data.linz.govt.nz/my/api with "Full access to Exports Access"
   1. Then create a file called .env in this folder, and add LINZ_API_KEY=XXXXX, where XXXXX is the token you just generated.
   2. Then repeat this step for https://datafinder.stats.govt.nz/my/api and save it as STATS_NZ_API_KEY. This is required for the rural urban boundaries
4. Run npm install
5. Run npm run request-linz-export to request an export from the LDS, and wait for it to be generated
6. Run npm run download-linz to download the requested export
7. Download the planet file (for just NZ) by running npm run download-planet. This will create ./data/osm.pbf
8. Start the preprocess script by running npm run preprocess. This will take ca. 2.5 minutes and create ./data/osm.json and ./data/linz.json
9. Start the confate script by running npm run conflate. This will take 30 seconds and create ./data/status.json. Some computationally expensive diagnostics are only generated if you run npm run conflate -- --full, which takes 20 times longer.
10. Start the action script by running npm run action. This will take 20 seconds and generate a ton of files in the ./out folder
11. Upload the contents of the ./out folder to the CDN by running npm run upload. This will take ca. 4 minutes

Tests

There are end-to-end tests than run based on a mock planet file and a mock linz CSV file.

To start the test, run npm test. If it changes the contents of the snapshot folder, commit those changes.

Process