Analyzing the trained models

Access and interpret the models after fitting

Data setup

from mlforecast.utils import generate_daily_series

series = generate_daily_series(10)
series.head()

	unique_id	ds	y
0	id_0	2000-01-01	0.322947
1	id_0	2000-01-02	1.218794
2	id_0	2000-01-03	2.445887
3	id_0	2000-01-04	3.481831
4	id_0	2000-01-05	4.191721

Training

Suppose that you want to train a linear regression model using the day of the week and lag1 as features.

from sklearn.linear_model import LinearRegression

from mlforecast import MLForecast

fcst = MLForecast(
    freq='D',
    models={'lr': LinearRegression()},
    lags=[1],
    date_features=['dayofweek'],
)

fcst.fit(series)

MLForecast(models=[lr], freq=<Day>, lag_features=['lag1'], date_features=['dayofweek'], num_threads=1)

What MLForecast.fit does is save the required data for the predict step and also train the models (in this case the linear regression). The trained models are available in the MLForecast.models_ attribute, which is a dictionary where the keys are the model names and the values are the model themselves.

fcst.models_

{'lr': LinearRegression()}

Inspect parameters

We can access the linear regression coefficients in the following way:

fcst.models_['lr'].intercept_, fcst.models_['lr'].coef_

(3.2476337167384415, array([ 0.19896416, -0.21441331]))

SHAP

import shap

Training set

If you need to generate the training data you can use MLForecast.preprocess.

prep = fcst.preprocess(series)
prep.head()

	unique_id	ds	y	lag1	dayofweek
1	id_0	2000-01-02	1.218794	0.322947	6
2	id_0	2000-01-03	2.445887	1.218794	0
3	id_0	2000-01-04	3.481831	2.445887	1
4	id_0	2000-01-05	4.191721	3.481831	2
5	id_0	2000-01-06	5.395863	4.191721	3

We extract the X, which involves dropping the info columns (id + times) and the target

X = prep.drop(columns=['unique_id', 'ds', 'y'])
X.head()

	lag1	dayofweek
1	0.322947	6
2	1.218794	0
3	2.445887	1
4	3.481831	2
5	4.191721	3

We can now compute the shap values

X100 = shap.utils.sample(X, 100)
explainer = shap.Explainer(fcst.models_['lr'].predict, X100)
shap_values = explainer(X)

And visualize them

shap.plots.beeswarm(shap_values)

Predictions

Sometimes you want to determine why the model gave a specific prediction. In order to do this you need the input features, which aren’t returned by default, but you can retrieve them using a callback.

from mlforecast.callbacks import SaveFeatures

save_feats = SaveFeatures()
preds = fcst.predict(1, before_predict_callback=save_feats)
preds.head()

	unique_id	ds	lr
0	id_0	2000-08-10	3.468643
1	id_1	2000-04-07	3.016877
2	id_2	2000-06-16	2.815249
3	id_3	2000-08-30	4.048894
4	id_4	2001-01-08	3.524532

You can now retrieve the features by using SaveFeatures.get_features

features = save_feats.get_features()
features.head()

	lag1	dayofweek
0	4.343744	3
1	3.150799	4
2	2.137412	4
3	6.182456	2
4	1.391698	0

And use those features to compute the shap values.

shap_values_predictions = explainer(features)

We can now analyze what influenced the prediction for 'id_4'.

round(preds.loc[4, 'lr'], 3)

3.525

shap.plots.waterfall(shap_values_predictions[4])