Reference code to fine-tune arabertv2 model for [MASK] prediction #116
-
|
Hi, Good day. I would like to fine-tune arabertv2 with new dataset for [MASK] prediction purpose, could you point me to the correct sample? Also is this sample relevant? Please kindly advise. Thank you. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
for Masked Language modeling I suggest you use https://github.com/huggingface/transformers/blob/master/examples/pytorch/language-modeling/run_mlm.py script from huggingface. You can apply AraBERT preprocessing, before sending the training sentences to the script |
Beta Was this translation helpful? Give feedback.
-
|
Hi Wissam Antoun, Thank you very much. |
Beta Was this translation helpful? Give feedback.
for Masked Language modeling I suggest you use https://github.com/huggingface/transformers/blob/master/examples/pytorch/language-modeling/run_mlm.py script from huggingface. You can apply AraBERT preprocessing, before sending the training sentences to the script