Looking for improvement ideas for Deepparse

[davebulaval]

We are looking for ideas to improve Deepparse. So, if you have any, you can provide them by replying to this issue.

See #136 for previous discussions.

[d4rkm0nkey]

It would be great to add a trainable tokenizer to deepparse. For example if you want to parse house number addition as an extra tag, an address can be written like this "my-street 12 a", but it could also be written like this: "my-street 12a", "my-street 12/a", "my-street 12A". In these cases it would not be possible to give the "12" the house number label and the "a" the house number addition label.
A trainable tokenizer could split the 12 and the A into separate tokens.
With doing so, I would suggest to support the label-studio labeling format for the training data.

[davebulaval]

In dev, a feature currently adds a user-define pre-processor to clean the address before parsing (kind of a tokenizer). It is mostly functions to apply cleaning on an address. It could be a trained tokenizer or else. It still needs work to do since it only applies on call and is not included in the test or train methods.

If you have time, it would be great if you could test this feature in dev and get us feedback if it is what you are looking for.

[HayatoSempai]

Hello Dave, I don't know it the tokenizer is still relevant but I am testing your algorithm (thx for your work 💯 ) on a large batch of addresses and I saw that it also replicates libpostal mistakes on french house number addition :
For instance : "72B rue Goncourt" will give 72B as house_number and "rue Goncourt" as street whereas "72 B rue Goncourt" will give 72 as house_number and "B rue Goncourt" as street.

If you need test on the tokenizer feature I could help !

Also do you think an address standardization feature as libpostal's is relevant ?

[davebulaval]

@HayatoSempai, you are right about this specific problem. We don't use a complex tokenizer. For now, we only split words using whitespace.

However, we have added a data processor that you can pass during parsing. A quick workaround would be to split using a regex data processor. Something like ([0-9]+)([A-Za-z]+) and separated the two groups with a whitespace between them.

If you have some data for this case (we don't need much), it would be nice if you could share it so we can see the performance, add a new data processor, and even retrain the based model. That way, it would be solved for everyone.

[HayatoSempai]

Yes I work in R&D in a french small company and I have gathered quite a lot of address in Spain, Scotland, France and GB for my use case. I could share some of it with you if it could help.
I added you on linkedin to talk further about it.

[davebulaval]

• Add comparison of AddressParser model params
• Add Apex support to use quantization and improve inference performance

[cyberbeat]

It would be usefull, if address could be
a) including a venue-name
b) could be extracted out of some longer text

[davebulaval]

I don't understand "a)". Can you elaborate, please?

For "b)", it has been proposed. The problem is that we need a dataset to train the model; thus, we cannot offer such a feature right now.

[cyberbeat]

This is from your docs:

Address parser to parse an address or a list of addresses using one of the seq2seq pretrained networks either with FastText or BPEmb. The default prediction tags are the following

"StreetNumber": for the street number,

"StreetName": for the name of the street,

"Unit": for the unit (such as an apartment),

"Municipality": for the municipality,

"Province": for the province or local region,

"PostalCode": for the postal code,

"Orientation": for the street orientation (e.g. west, east),

"GeneralDelivery": for other delivery information,

"EOS": (End Of Sequence) since we use an EOS during training, sometimes the models return an EOS tag.

I am missing a tag for location/venue-namem for example "venue":"Club 42"

[davebulaval]

Unfortunately, we don't have such data to train a model with this tag.

You can retrain a model with this new tag if you have some. See our documentation here for details.

[Tayssirbo]

Hello @davebulaval, great work!
I'm currently learning about deepparse and I think it would be a great idea to return the probability of all the tags together (the probability of the entire adress not tag by tag)

[davebulaval]

Hi @Tayssirbo, tks!

This is not possible since we do per-word classification. Why do you need an entire address probability?

[Tayssirbo]

I need to know which addresses have the lowest probability so I can reparse them separately.
I'm working on a project and the result of address parsing should be at least 90% correct (not always the case with French addresses).

[davebulaval]

In that case, you can compute a weighted mean of the individual tags. It is clearly not a probability, but it can give you the insight you need.
Why weighted? You could give weights to different tags based on your observations (who has the lowest score or more frequent error).

[Tayssirbo]

Hello again @davebulaval,

I'm still learning about deepparse and I was thinking that it would be very usefull feature if we can implement tags dictionaries.
For example we can implement unit dictionary :

{
    "unit": [
        "Appartement",
        "AP":,
        "App"
    ]
}

this way the program, if there is one of the words of the dictionary in the adress, will know automatically it's the unit!
In your opinion to what extent dictionaries can be added ?! Do you think this feature could increase the precision of the results ? (especially if we are working with a specific language)

[davebulaval]

Do you mean a set of already known words that can automatically tag it as a class X?

For example, using your example, if the words "AP" or "App" appear in the address, it would tag it as "unit".

@Tayssirbo

[Tayssirbo]

yes exactly @davebulaval

[davebulaval]

Yes, it would help. Do you have time to contribute to developing the idea?