Is it possible to add two NERs into one Pipeline?

[rsoika]

Hi,
I wonder if it is possible to add two different NER in one pipeline. My idea is to start with the existing en_core_web_sm model, which already provides some useful entities.

Now I want to add my own second NER and train some different entity (with the hope that my new NER will benefit form the existing).
I understand that I have to disable during training the pipes from the pretrained en_core_web_sm model to avoid the catastrophic forgetting.

 if "my_ner" not in nlp.pipe_names:
            logging.info("...adding new pipe 'ner'...")
            ner = nlp.add_pipe('ner',"my_ner")
            .....
            // define new lables for the new ner
            ner.add_label(_label1)
            ner.add_label(_label2)
            ....
....
    // disable other pipse....
    other_pipes =  [ pipe for pipe in nlp.pipe_names if pipe !="my_ner"]
    with nlp.disable_pipes(*other_pipes):
        if restartTraining: 
            logging.info("...begin new training!")
            optimizer = nlp.create_optimizer()
        else: 
            optimizer = nlp.resume_training()

        // load training data
        ....
       losses = nlp.update(examples, sgd=optimizer);
       // ! EXCEPTON

But when I try to call the update method I got the following error:

[E022] Could not find a transition with the name 'O' in the NER model.

I think the "O" is an entity from the NER in the "en_core_web_sm" model that I thought was disabled?

So my question is: is it possible to train different NERs in one pipeline?

[thomashacker]

Hello,
I would recommend looking at this spaCy project, which shows how to combine multiple NER components

[rsoika]

Thanks for this hint. I think I understand the problem now better. So in my case it makes no sense to add different NERs in one pipeline, as the entities of interests would overlap - DATE in the core model and INVOICE.DATE in my own model. I have had the hope that the 2nd NER will adjust entities found by the 1st NER. But as the project explained this is not the case.