Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Importing diff ends with a PostgreSQL error, retries and continues to fail #325

Open
ingarpedersen opened this issue Oct 1, 2020 · 0 comments

Comments

@ingarpedersen
Copy link

I have imported a country, and the initial generation works just fine.
The next day I'm updating using diffs, but now the update fails:

Starting map-update_postgres_1 ... done
docker-compose run --rm -u 1001:1001 openmaptiles-tools sh -c 'pgwait && import-update'
[Oct  1 10:05:14] [INFO] [  1m0s] C:   77000/s (4642703) N:     500/s (32009) W:       0/s (0) R:      0/s (0)
[Oct  1 10:06:07] [INFO] [diff] Parsing changes, updating cache and removing elements took: 1m53.168368631s
[Oct  1 10:06:07] [INFO] [ 1m53s] C:   78000/s (5165068) N:     500/s (39150) W:   15100/s (590221) R:   3020/s (25845)
[Oct  1 10:07:07] [INFO] [  1m0s] C:       0/s (0) N:       0/s (0) W:    3200/s (158359) R:   2580/s (7702)
[Oct  1 10:08:07] [INFO] [  2m0s] C:       0/s (0) N:       0/s (0) W:    3000/s (332858) R:   2580/s (7702)
[Oct  1 10:09:07] [INFO] [  3m0s] C:       0/s (0) N:    5800/s (73334) W:    2900/s (356748) R:   2580/s (7702)
[Oct  1 10:09:13] [INFO] [PostGIS] Updating generalized tables took: 43.070865217s
[Oct  1 10:09:13] [INFO] [diff] Processing /import/000/002/941.osc.gz took: 4m59.534247281s
[Oct  1 10:09:13] [INFO] importing #2941 till 2020-10-01 00:00:00 +0000 UTC took: 4m59.53537228s
[Oct  1 10:09:13] [ERR] pq: Could not complete operation in a failed transaction
[Oct  1 10:09:13] [INFO] retrying in 2s

After the retrying there seems to be 2 processes working. After a while the same error, with 4s delay, then there are 3 processes, and so on.

I have deleted everything (all docker containers, volumes etc), and imported again, the same thing happens with the first diff update.

I have checked and there is no issue with disk space or RAM.

Any clues where to look??

Best regards
Ingar F Pedersen

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant