Dedect address duplicates and warn

[hubgitti]

Hi,

would it be possible to detect identical addresses of the currently downloaded data in Josm and warn the user if he or she tries to add that address again using the Austrian address helper?

This issue is discussed in the Austrian forum:
https://forum.openstreetmap.org/viewtopic.php?id=61805

Thanks, Hubgitti

[thomaskonrad]

Hi Hubgitti,

I just commented on the forum. Yes, I think this is a brilliant idea and needs to be done. Duplicates are of course not what we want. We could use the Overpass Turbo API to detect duplicate tag combinations.

I can't find the time right now to implement this.

Willing to try it and send a pull request?

Thomas

[thomaskonrad]

Well... already working on it.

Give me a couple of days to finish it.

[natrius]

Brilliant idea, thanks for further developement. And thanks for already working on that issue :D

[mmd-osm]

BTW: please don't use dev.overpass-api.de/api_mmd endpoint for this purpose, as it is our development server, i.e. it could be unavailable or malfunctioning at any time without prior announcement.

Once the 0.7.55 release is available, you could switch to overpasss-api.de instead. I expect this to happen some time in April or May 2018. If you don't need any of the 0.7.55 features, you can start right away on the public production server of course.

[thomaskonrad]

I just finished implementing this: aaea664

@mmd-osm: I use the endpoint https://overpass-api.de/api/interpreter, is that ok?

I also created a release for it: https://github.com/JOSM/austriaaddresshelper/releases/tag/v0.6.0. Would someone be so kind as to test it and give me feedback?

[thomaskonrad]

I can't comment on the corresponding OpenStreetMap forum thread any more because it has been closed.

[mmd-osm]

JOSM has a central config setting for the Overpass server, and I’d recommend to this setting instead of hard coding a server name.

[natrius]

You can open a thread in the austrian forum where you can post news about the austriaadresshelper. (New version, changelog, etc.)
I asked and it would be okay and the mods would keep it clean.

[thomaskonrad]

@mmd-osm, I'll do.

@natrius, yes, please.

[natrius]

https://forum.openstreetmap.org/viewtopic.php?id=62304

[3mg4]

Nice Work, and thanks for the feature, it works smoothly! A Traum wie man bei uns sagt;-)

[Luzandro]

Seems to use a wrong id if the address is a node, e.g. for this address:
https://www.openstreetmap.org/node/4945804120
you get a link to this node:
https://www.openstreetmap.org/node/650836824

In case of ways the linked id is correct, but it still took some time for me to find a building in this area that leads to an error message, because many buildings don't have addr:city specified and thereby they aren't seen as duplicate. IMHO the check could be less restrictive and only check for street/place + housenumber - I guess this shouldn't result in wrong hits as long as the search radius is small enough (?)

Finally in my opinion it's more likely to get duplicates because 2 buildings in close proximity get the same result which isn't prevented by the overpass search for existing entries. So maybe it would make sense to prevent the AddressHelper from returning the same value two times in a row if that's easy to implement.

[Luzandro]

just wanted to notify you, that I'm working on a JOSM-validator test for duplicated addresses that covers this issues (including speed issues discussed in the forum) and isn't limited to the plugin

https://github.com/Luzandro/josm-address-validator

[thomaskonrad]

That‘s nice @Luzandro! I appreciate every help, I don‘t have much time right now.

I had another idea: I have the OSM AT data on my server anyway, so i could check for duplicates via SQL and send the info by in the response. That way you would probably not notice any difference.

[Luzandro]

That would still have the same problem, that it only checks for values that already exist in the database, but not the current working data.
But I just noticed a similar conceptual problem with my solution, as it only takes the selected data into account. So it works perfectly fine if you launch it manually and have nothing selected (which means everything gets selected), but the check that is performed before upload quite reasonably only has the changed objects selected and thereby only finds duplicates if both entries are new / changed - so somehow it's complementary to yours, but as it's supposed to be a general solution, I have to look into it, if I can fix this

edit: need to clean it up a bit, but basically my problems seems to be solved

[Luzandro]

the patch for the JOSM validator is now shipped with the current stable version, so there is no need anymore for a separate check
https://josm.openstreetmap.de/ticket/16310

[thomaskonrad]

That‘s nice! But I guess it only works for the area the user already downloaded, right?

[thomaskonrad]

I‘ll give it a try soon and update the address helper if I see that it works as described.

[Luzandro]

That's right, it only checks the local data, so in some cases e.g. at the edge of your download area it could miss duplicates that your check finds