AraELECTRA for NER

[Astudnew]

Hi
how can I use AraELECTRA for named entity recognition?

[WissamAntoun]

Use this notebook and just change the model name https://colab.research.google.com/drive/128wX6r_Nz-2LZ_lROeWjWS_yTi96KR7J?usp=sharing

[WissamAntoun]

for the model you need to use aubmindlab/araelectra-base-discriminator , and send aubmindlab/araelectra-base to the ArabertPreprocessor.

Also Araelectra doesn't need Farasa. ArabertPreprocessor will handle it internally

[WissamAntoun]

yes, I will probably change it soon to accept the whole model name from huggingface

[WissamAntoun]

yes, I will probably change it soon to accept the whole model name from huggingface

And by soon, I mean i just added the change, no you can provide the same model name from huggingface to the preprocessor

[Astudnew]

Thanks, it works but why get this message in training part:
"Some weights of the model checkpoint at aubmindlab/araelectra-base-discriminator were not used when initializing ElectraForTokenClassification: ['discriminator_predictions.dense.weight', 'discriminator_predictions.dense.bias', 'discriminator_predictions.dense_prediction.weight', 'discriminator_predictions.dense_prediction.bias']

• This IS expected if you are initializing ElectraForTokenClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
• This IS NOT expected if you are initializing ElectraForTokenClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
  Some weights of ElectraForTokenClassification were not initialized from the model checkpoint at aubmindlab/araelectra-base-discriminator and are newly initialized: ['classifier.weight', 'classifier.bias']
  You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference."

[WissamAntoun]

ignore it, these are some warnings that you are not re-using the final dense layers used fine-tuning which is expected