Can a pertained model be fine-tuned, or not ?

[kgoderis]

Just a general question, as I am puzzled and unable to google the answer : when using a pretrained model as part of a pipeline, is it possible to finetune that model, or will the weights of the model be overwritten by any fit() called upon the pipeline?

In there is a paragraph that states: "It’s important to understand that the Spark pipeline concept relies on a full data set when you do retraining, not increments. We have now overwritten the pre-trained pipeline with a very badly trained pipeline (only 2 rows of training data), so this example is not particularly useful except to walk through how to code up the training process"

[maziyarpanahi]

Hi @kgoderis

When you are using Spark NLP pre-trained models inside a pipeline or a pre-trained pipeline, the weights are constant, they cannot be fine-tuned nor be overwritten. If the entire pipeline is pre-trained models and rule-based annotators, then the .fit(df) stage will be skipped and it goes directly to .transform().

In Sparrk NLP the trainable annotators have Approach in their names, and the pretrained models only will be accessible with annotators with Model in their names. (99% of the time). This means, if you have a pretrained model let's say in NerDLModel, it won't be fine-tuned nor overwritten since it's not trainable. The trainable annotator for NER is called NerDLApproach, and if that is in the pipeline it gets trained in .fit() stage.

The short answer, there is no fine-tuning nor zeroing any of the pre-trained models/pipelines inside Spark NLP.

[kgoderis]

Yet, do you still need to do a .fit() on a pretrained Pipeline in order to get a PipelineModel ?

[maziyarpanahi]

A pretrained pipeline you download for Spark NLP is already a PipelineModel. No need to be .fit(), not sure if it's even possible. However, if you are constructing your own Pipeline, even if all the annotators/stages are pretrained models, you still need a .fit() since it's mandatory by Apache Spark (Spark ML), but it doesn't cost anything since it will be immediately skipped.