Way lower accuracy than example notebooks

[J0ekr]

Hey,
Thanks for the great repo! I am currently trying to train a NLU and tested out your using_bert_crf_nlu example notebook with your provided data.
I have tested both Bert and Albert embeddings. However, I always only get 14% accuracy (so random classification), while you have ~98% accuracy. I am currently using python 3.11 - the only major compatibility issue I had was with sentencepiece (dependency from transformers), where I now use the newest version instead of your specified version range.
I can't imagine this being the case for the model not working anymore. I will continue investigating this, but wanted to ask whether you or someone else already had this issue and has ideas!

Thanks!

[J0ekr]

You set the BERT Embedding trainable=True. I have now set this to False, and get accuracies of ~80% after 3 epochs .

I assume you hotloaded your model used in the example notebook?