Question: can we save all the evaluated pipelines as fitted model?

[kirane61]

I was looking for a way to extract fitted pipelines of all the pipelines or individuals evaluated by TPO. Is there any way that we can save all the evaluated pipelines as fitted models?

For example, if I set my generations to 2 and my population size to 2, then I want to save all six fitted pipelines evaluated by tpot for my further usage. Is there any way I can get the pipelines fitted so that I can use them directly without training them again?

[perib]

The short answer is no. TPOT only fits the pareto front models (including the best model) to the full training set. TPOT does not save the fitted models for each fold of the CV.

Here are the models that you are able to access.

1. The model with the best cv score fitted to the full training data.
2. The list of Pareto front models fitted to the full training data
3. With some work, you can extract all evaluated pipelines, but they will be unfitted. You can find more information here tpot.evaluated_individuals_ to pipeline #516

from tpot import TPOTRegressor, TPOTClassifier
from sklearn.model_selection import train_test_split
import sklearn
import sklearn.datasets
import tpot
import dill as pickle

X, y = sklearn.datasets.load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.80, test_size=0.20, random_state=42)

est = TPOTClassifier(generations=2, population_size=2, verbosity=2, random_state=42, n_jobs=-2 ,cv=10)

est.fit(X_train, y_train)


# 1 save the model with the best cv score fitted to the full training data.
pickle.dump(est.fitted_pipeline_, open('tpot_iris_pipeline.pkl', 'wb'))

# 2 save the list of unfitted Pareto front models
pickle.dump(list(est.pareto_front_fitted_pipelines_.values()), open('tpot_iris_pareto_front_models.pkl', 'wb'))

We are currently working on TPOT2 where you can more easily access all evaluated pipelines without workarounds. However, like in TPOT1, we do not train all pipelines on the full dataset so these pipelines are unfitted. Example here: