Custom Objective Function

[jrodenberg0]

Is it possible currently to integrate a custom objective function via MLForecast to LGBM or XGB?

I am attempting to use sample wieghts and I know that isn't currently supported. I was hoping to simply integrate my weights into my objective function (RMSE). Is there a better workaround for this?

(I know you can fit/predict with sample weights, but i have also noticed I can't pass exog features).

[jmoralez]

Hey. You can do something like here but to define the weights:

import numpy as np
import pandas as pd
from lightgbm import LGBMRegressor
from mlforecast import MLForecast
from mlforecast.utils import generate_daily_series

class LGBMWeights(LGBMRegressor):
    def fit(self, X, y):
        weights = X['weight']
        X = X.drop(columns='weight')
        return super().fit(X=X, y=y, sample_weight=weights)

    def predict(self, X):
        if 'weight' in X.columns:
            X = X.drop(columns='weight')
        return super().predict(X=X)

freq = 'D'
h = 10
series = generate_daily_series(1, freq=freq)
series['weight'] = np.random.default_rng(seed=0).random(series.shape[0])
train_end = series['ds'].max() - h * pd.tseries.frequencies.to_offset(freq)
train_mask = series['ds'] <= train_end
train = series[train_mask]
valid = series[~train_mask]
fcst = MLForecast(
    models=LGBMWeights(verbosity=-1),
    freq=freq,
    date_features=['dayofweek'],
)
fcst.fit(train, static_features=[]).predict(h=h, X_df=valid)

I've been thinking of adding a wrapper like that, which takes the model and the name of the weight argument (sample_weight in this case) and just does slight override of the fit and predict methods.

I've also been meaning to remove the custom training guide, since everything there can be achieved with scikit-learn estimators and/or pipelines and that allows you to also use it in cross validation.

Please let us know if this doesn't work as expected and if you have any thoughts regarding the previous points.

[jrodenberg0]

Wow, youre incredible thanks @jmoralez !!! I would say for me personally in Supply Chain Forecasting that would be incredibly useful... Just saved me lots of time...

[jrodenberg0]

@jmoralez revisiting this because when predicting out of sample I get an error... saying the sample weight column isn't found in X_df. Could I just provide the column of all zeros inside of X_df (implying to keep all samples weighted in my case), without messing with the forecast?

[jmoralez]

Hey, yeah I think it's because the weight gets interpreted as a feature. I'm already working on adding a weight_col argument to handle this better, but in the meantime adding the weight after preprocess should work:

series = generate_daily_series(1)
h = 10
train_end = series['ds'].max() - h * pd.offsets.Day()
train_mask = series['ds'] <= train_end
train = series[train_mask].copy()
valid = series[~train_mask]

fcst = MLForecast(
    models=LGBMWeights(verbosity=-1),
    freq='D',
    date_features=['dayofweek'],
)
X, y = fcst.preprocess(train, static_features=[], return_X_y=True)
X['weight'] = np.random.default_rng(seed=0).random(X.shape[0])  # weight isn't a feature
fcst.fit_models(X, y)
fcst.predict(5)

You can also use a column full of zeros, it'll get dropped before passing it to the model.

[jmoralez]

Hey @jrodenberg0, v0.14.0 adds the weight_col argument to fit and cross validation (guide). Please give it a go and let us know if something doesn't work as expected.