Question regarding “new_df” for future prediction with a pretrained model

[Garsdal]

Hi Nixtla / MLForecast,

I have been testing out the MLForecast library and have had good experience with training forecasting models on historical data. Now I want to deploy a pretrained model for inference on a schedule.

My problem is that my target variable 'y' is often delayed and inconsistent. When I trained my model I used no lagged feature and only depend on exogenous variable e.g. 'X_df' input. However, at inference time I have to provide a 'new_df' as well as my 'X_df' to the .predict() method.

I have been testing this locally and can only get inference to work on future data if I provide 'new_df' at t-1 when my exogenous variables start from 't' until 't+h'. Since my target will always be delayed there is no way for me to provide the model with 'new_df' like this, and I can never make predictions even though the model was only trained on the exogenous variables.

Can anyone help me how I would use a pretrained model for future predictions with only 'X_df'?

[jmoralez]

Hey. The new_df is used to get the last training times (to produce the future times) and the values of the target to compute the lag features and tqrget transformations. If you're not doing any target transformations you can just provide the times before your forecast horizon for each id and random values of the target.

[jmoralez]

Can you elaborate on your use case a bit? If you're just using exogenous I don't think mlforecast provides any extra value, you could just use the scikit learn models directly.

[Garsdal]

We would be training 1000s of forecasting models for various different signals, some have "y" available at almost realtime while others have much longer delays, and some no "y" at all. In these cases we would want to predict the same horizon always (lets say h=24), but for some of the models would want to utilize the available lagged "y" but others only depend on exogenous features.

I am therefor looking for a 'general purpose' model that can be used both in the case where "y" is available or not. We assume that we will always have exogenous variables available. The problem we are facing with using a scikit-learn model directly, is that if we want to predict h=24 ahead, we can at most use y_t-24 in the model. MLForecast seems to be able to handle the case of using both y_t-1 and predict up until h=24, which is why we are investigating this option.

[jmoralez]

Do you train the models accordingly? e.g. if you know you'll get the target delayed by 24 hours, do you train on at most the lag 24 (assuming hourly data)?

[Garsdal]

Yes, we would manage that accordingly with configurations. The idea would be that we could use MLForecast as the single model architecture that can handle both short-term forecasts and the long horizon. Which it seems to be able to, but it feels wrong to use incorrect "y" dummy data.

[jmoralez]

So these would be the same series the model saw during training, right? You're not doing transfer learning, just updating the targets with newer values, correct? If that's the case I think the best approach would be to use the update method instead (docs). That should allow you to just use predict with X_df when you don't have any newer targets and update + predict when you do.