Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for custom language sets #494

Open
deicidemilan opened this issue Sep 16, 2020 · 12 comments
Open

Support for custom language sets #494

deicidemilan opened this issue Sep 16, 2020 · 12 comments

Comments

@deicidemilan
Copy link

deicidemilan commented Sep 16, 2020

Hello,

Is there a way to get more details from API as namedetails? For example i need Belgrade on Serbian language (cyrilic).

http://photon.komoot.de/api/?q=beograd&limit=5&osm_tag=place:village&osm_tag=place:town&osm_tag=place:city

This returns only one name.

Nominatim have it, but it's not Autocomplete, Partial search or Wildcard searches.

https://search.meetserbia.rs/search.php?city=beograd&format=jsonv2&namedetails=1&type=administrative

I Install both on my server so I can change code or conf if it posible to get this.

@deicidemilan
Copy link
Author

deicidemilan commented Sep 16, 2020

I install both nominatim and photon.
Import nominatim database into photon, and everiting works perfect.
But I need all name spacec as in nominatim database.
I have only one country imported.

"namedetails": { "name": "Стара Пазова", "name:de": "Alt-Pasua", "name:en": "Stara Pazova", "name:hu": "Ópazova", "name:ru": "Стара-Пазова", "name:sr": "Стара Пазова", "name:sr-Latn": "Stara Pazova" }

If It's a way to get just name:sr and name:en will be perfect.
Is this posible or i need to chenge source code?
If I need to change where is file in source code to add this from nominatim database?
I can edit anything because all code is on my server, but no idea what :(

@deicidemilan deicidemilan changed the title Autocomplete, Partial search or Wildcard searches namedetails Sep 16, 2020
@deicidemilan deicidemilan changed the title namedetails Output details namedetails Sep 16, 2020
@mtmail
Copy link
Contributor

mtmail commented Sep 16, 2020

es/mappings.json lists the 5 languages imported by default. I think you just need to add 'sr' (and 'sr-Latn') there and reimport.

@deicidemilan
Copy link
Author

I will try and give you feedback.

@deicidemilan
Copy link
Author

I was explore importing and how photon works.
I import all supported language and only one to see how it works. So this is definitly what I need.
Now I need to edit something from source to see if it's work.
Problem is when I change es/mappings.json how to build photon-0.3.4.jar. Because es/mappings.json is in source and i don't know how to build photon-0.3.4.jar.
I need additional language only for "name" so if this is a good way for this or I need to add everywhere 'rs'?

{ "name": { "properties": { "alt": { "type": "text", "index": false, "fields": { "raw": { "type": "text", "analyzer": "index_raw" } }, "copy_to": [ "collector.default" ] }, "de": { "type": "text", "index": false, "fields": { "ngrams": { "type": "text", "analyzer": "index_ngram", "search_analyzer": "search_ngram" }, "raw": { "type": "text", "analyzer": "index_raw" } }, "copy_to": [ "collector.de" ] }, "default": { "type": "text", "index": false, "copy_to": [ "collector.default", "name.sr", "name.en", "name.de", "name.fr", "name.it" ] }, "en": { "type": "text", "index": false, "fields": { "ngrams": { "type": "text", "analyzer": "index_ngram", "search_analyzer": "search_ngram" }, "raw": { "type": "text", "analyzer": "index_raw" } }, "copy_to": [ "collector.en" ] }, "sr": { "type": "text", "index": false, "fields": { "ngrams": { "type": "text", "analyzer": "index_ngram", "search_analyzer": "search_ngram" }, "raw": { "type": "text", "analyzer": "index_raw" } }, "copy_to": [ "collector.sr" ] } } } }

@deicidemilan
Copy link
Author

deicidemilan commented Sep 17, 2020

I found this too in source code @Parameter(names = "-languages", description = "languages nominatim importer should import and use at run-time, comma separated (default is 'en,fr,de,it')") private String languages = "en,fr,de,it";
In file CommandLineArgs.java
I think here needs to add 'sr' language too.
Can you help me where to add 'sr' and build photon-0.3.4.jar again.

"Updated"

I was add 'sr' everywhere in es/mappings.json
https://pasteid.fledix.com/1h1g

"Updated"

Actuly I was successfully update new elasticsearch from nominatim with comand:
java -jar photon-0.3.4.jar -nominatim-import -host localhost -port 5432 -database nominatim -user nominatim -password mypass -languages sr
But whan i request API &lang=sr I got
{ "message": "language sr is not supported, supported languages are: default, en, fr, de, it" }
Need to add 'sr' in supported languages array.

@lonvia lonvia changed the title Output details namedetails Support for custom language sets Oct 6, 2020
@numair
Copy link

numair commented Nov 11, 2021

HI @lonvia @deicidemilan were you able to make this work with languages other than the defaults? I'm thinking of trying to make this work with Japanese. I am assuming that, in some manner, it would be possible to get this library to work with all of the options listed here:

https://wiki.openstreetmap.org/wiki/Nominatim/Country_Codes

I understand that Japanese might be particularly tricky given the difference in composition etc, so I appreciate any ideas you may have on how this could be accomplished.

@Lelelo1
Copy link

Lelelo1 commented Mar 26, 2022

I need further languages. I notice that when searching food i town in china I get results, but when searching for (google translated) I get non related results elsewhere in china, like motorways.

Is it as "simply" as indexing chinese osm data or are there more configuration like in Nomantim needed?

@lonvia
Copy link
Collaborator

lonvia commented Apr 6, 2022

@Lelelo1 Please note that Photon does not searching by category, so a search for 'food' will usually not yield the expected results.

As for integrating Chinese: it can be simply added as a language in the list. The problem is that Chinese is not well supported by standard analysers, so there is probably some effort necessary, to change the analysers that ES uses. See #563 for an attempt to port Photon to Japanese.

@Yuri-Chud
Copy link

Hello, @lonvia!
I try to use some custom languages tags, such as:

  1. "alt_name";
  2. "alt_name:ru".

How can we configurate es/mappings.json file to be able to search for information that may be contained in these tags? Thank you in advance.

@lonvia
Copy link
Collaborator

lonvia commented Oct 25, 2023

alt_name is already considered for search. The list of possible tags is hardcoded here.

@Yuri-Chud

This comment was marked as off-topic.

@lonvia

This comment was marked as off-topic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants