AutoMLSearch for time series problems — EvalML 0.82.0 documentation

But first, what is a time series?#

A time series is a series of measurements taken at different moments in time (Wikipedia). The main difference between a time series dataset and a normal dataset is that the rows of a time series dataset are ordered chronologically, where the relative time between rows is significant. This relationship between the rows does not exist in non-time series datasets. In a non-time-series dataset, you can shuffle the rows and the dataset still has the same meaning. If you shuffle the rows of a time series dataset, the relationship between the rows is completely different!

What does AutoMLSearch for time series do?#

In a machine learning setting, we are usually interested in using past values of the time series to predict future values. That is what EvalML’s time series functionality is built to do.

Loading the data#

In this guide, we work with daily minimum temperature recordings from Melbourne, Austrailia from the beginning of 1981 to end of 1990.

We start by loading the temperature data into two splits. The first split will be a training split consisting of data from 1981 to end of 1989. This is the data we’ll use to find the best pipeline with AutoML. The second split will be a testing split consisting of data from 1990. This is the split we’ll use to evaluate how well our pipeline generalizes on unseen data.

[2]:

import pandas as pd
from evalml.demos import load_weather

X, y = load_weather()

             Number of Features
Categorical                   1

Number of training examples: 3650
Targets
10.0    1.40%
11.0    1.40%
13.0    1.32%
12.5    1.21%
10.5    1.21%
        ...
0.2     0.03%
24.0    0.03%
25.2    0.03%
22.7    0.03%
21.6    0.03%
Name: count, Length: 229, dtype: object

The argument 'infer_datetime_format' is deprecated and will be removed in a future version. A strict version of it is now the default, see https://pandas.pydata.org/pdeps/0004-consistent-to-datetime-parsing.html. You can safely remove this argument.
The argument 'infer_datetime_format' is deprecated and will be removed in a future version. A strict version of it is now the default, see https://pandas.pydata.org/pdeps/0004-consistent-to-datetime-parsing.html. You can safely remove this argument.

[3]:

train_dates, test_dates = X.Date < "1990-01-01", X.Date >= "1990-01-01"
X_train, y_train = X.ww.loc[train_dates], y.ww.loc[train_dates]
X_test, y_test = X.ww.loc[test_dates], y.ww.loc[test_dates]

Visualizing the training set#

[4]:

import plotly.graph_objects as go

[5]:

data = [
    go.Scatter(
        x=X_train["Date"],
        y=y_train,
        mode="lines+markers",
        name="Temperature (C)",
        line=dict(color="#1f77b4"),
    )
]
# Let plotly pick the best date format.
layout = go.Layout(
    title={"text": "Min Daily Temperature, Melbourne 1980-1989"},
    xaxis={"title": "Time"},
    yaxis={"title": "Temperature (C)"},
)

go.Figure(data=data, layout=layout)

Fixing the data#

Sometimes, the datasets we work with do not have perfectly consistent DateTime columns. We can use the TimeSeriesRegularizer and TimeSeriesImputer to correct any discrepancies in our data in a time-series specific way.

To show an example of this, let’s create some discrepancies in our training data. We’ll also add a couple of extra columns in the X DataFrame to highlight more of the options with these tools.

[6]:

X["Categorical"] = [str(i % 4) for i in range(len(X))]
X["Categorical"] = X["Categorical"].astype("category")
X["Numeric"] = [i for i in range(len(X))]

# Re-split the data since we modified X
X_train, y_train = X.loc[train_dates], y.ww.loc[train_dates]
X_test, y_test = X.loc[test_dates], y.ww.loc[test_dates]

[7]:

X_train["Date"][500] = None
X_train["Date"][1042] = None
X_train["Date"][1043] = None
X_train["Date"][231] = pd.Timestamp("1981-08-19")

X_train.drop(1209, inplace=True)
X_train.drop(398, inplace=True)
y_train.drop(1209, inplace=True)
y_train.drop(398, inplace=True)

With these changes, there are now NaN values in the training data that our models won’t be able to recognize, and there is no longer a clear frequency between the dates.

[8]:

print(f"Inferred frequency: {pd.infer_freq(X_train['Date'])}")
print(f"NaNs in date column: {X_train['Date'].isna().any()}")
print(
    f"NaNs in other training data columns: {X_train['Categorical'].isna().any() or X_train['Numeric'].isna().any()}"
)
print(f"NaNs in target data: {y_train.isna().any()}")

Inferred frequency: None
NaNs in date column: True
NaNs in other training data columns: False
NaNs in target data: False

Time Series Regularizer#

We can use the TimeSeriesRegularizer component to restore the missing and NaN DateTime values we’ve created in our data. This component is designed to infer the proper frequency using Woodwork’s “infer_frequency” function and generate a new DataFrame that follows it. In order to maintain as much original information from the input data as possible, all rows with completely correct times are ported over into this new DataFrame. If there are any rows that have the same timestamp as another, these will be dropped. The first occurrence of a date or time maintains priority. If there are any values that don’t quite line up with the inferred frequency they will be shifted to any closely missing datetimes, or dropped if there are none nearby.

[9]:

from evalml.pipelines.components import TimeSeriesRegularizer

regularizer = TimeSeriesRegularizer(time_index="Date")
X_train, y_train = regularizer.fit_transform(X_train, y_train)

/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-evalml/envs/v0.82.0/lib/python3.8/site-packages/woodwork/type_sys/utils.py:40: UserWarning:

The argument 'infer_datetime_format' is deprecated and will be removed in a future version. A strict version of it is now the default, see https://pandas.pydata.org/pdeps/0004-consistent-to-datetime-parsing.html. You can safely remove this argument.

/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-evalml/envs/v0.82.0/lib/python3.8/site-packages/woodwork/type_sys/utils.py:40: UserWarning:

Could not infer format, so each element will be parsed individually, falling back to `dateutil`. To ensure parsing is consistent and as-expected, please specify a format.

/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-evalml/envs/v0.82.0/lib/python3.8/site-packages/woodwork/type_sys/utils.py:40: UserWarning:

The argument 'infer_datetime_format' is deprecated and will be removed in a future version. A strict version of it is now the default, see https://pandas.pydata.org/pdeps/0004-consistent-to-datetime-parsing.html. You can safely remove this argument.

/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-evalml/envs/v0.82.0/lib/python3.8/site-packages/woodwork/type_sys/utils.py:40: UserWarning:

Could not infer format, so each element will be parsed individually, falling back to `dateutil`. To ensure parsing is consistent and as-expected, please specify a format.

Now we can see that pandas has successfully inferred the frequency of the training data, and there are no more null values within X_train. However, by adding values that were dropped before, we have created NaN values within the target data, as well as the other columns in our training data.

[10]:

print(f"Inferred frequency: {pd.infer_freq(X_train['Date'])}")
print(f"NaNs in training data: {X_train['Date'].isna().any()}")
print(
    f"NaNs in other training data columns: {X_train['Categorical'].isna().any() or X_train['Numeric'].isna().any()}"
)
print(f"NaNs in target data: {y_train.isna().any()}")

Inferred frequency: D
NaNs in training data: False
NaNs in other training data columns: True
NaNs in target data: True

Time Series Imputer#

We could easily use the Imputer and TargetImputer components to fill in the missing gaps in our X and y data. However, these tools are not built for time series problems. Their supported imputation strategies of “mean”, “most_frequent”, or similar are all static. They don’t account for the passing of time, and how neighboring data points may have more predictive power than simply taking the average. The TimeSeriesImputer solves this problem by offering three different imputation strategies: - “forwards_fill”: fills in any NaN values with the same value as found in the previous non-NaN cell. - “backwards_fill”: fills in any NaN values with the same value as found in the next non-NaN cell. - “interpolate”: (numeric columns only) fills in any NaN values by linearly interpolating the values of the previous and next non-NaN cells.

[11]:

from evalml.pipelines.components import TimeSeriesImputer

ts_imputer = TimeSeriesImputer(
    categorical_impute_strategy="forwards_fill",
    numeric_impute_strategy="backwards_fill",
    target_impute_strategy="interpolate",
)
X_train, y_train = ts_imputer.fit_transform(X_train, y_train)

Now, finally, we have a DataFrame that’s back in order without flaws, which we can use for running AutoMLSearch and running models without issue.

[12]:

print(f"Inferred frequency: {pd.infer_freq(X_train['Date'])}")
print(f"NaNs in training data: {X_train['Date'].isna().any()}")
print(
    f"NaNs in other training data columns: {X_train['Categorical'].isna().any() or X_train['Numeric'].isna().any()}"
)
print(f"NaNs in target data: {y_train.isna().any()}")

Inferred frequency: D
NaNs in training data: False
NaNs in other training data columns: False
NaNs in target data: False

Trending and Seasonality Decomposition#

Decomposing a target signal into a trend and/or a cyclical signal is a common pre-processing step for time series modeling. Having an understanding of the presence or absence of these component signals can provide additional insight and decomposing the signal into these constituent components can enable non-time-series aware estimators to perform better while attempting to model this data. We have two unique decomposers, the PolynomialDecompser and the STLDecomposer.

Let’s first take a look at a year’s worth of the weather dataset.

[13]:

import matplotlib.pyplot as plt

length = 365
X_train_time = X_train.set_index("Date").asfreq(pd.infer_freq(X_train["Date"]))
y_train_time = y_train.set_axis(X_train["Date"]).asfreq(pd.infer_freq(X_train["Date"]))
plt.plot(y_train_time[0:length], "bo")
plt.show()

../_images/user_guide_timeseries_23_0.png

With the knowledge that this is a weather dataset and the data itself is daily weather data, we can assume that the seasonal data will have a period of approximately 365 data points. Let’s build and fit decomposers to detrend and deseasonalize this data.

Polynomial Decomposer#

[14]:

from evalml.pipelines.components.transformers.preprocessing.polynomial_decomposer import (
    PolynomialDecomposer,
)

pdc = PolynomialDecomposer(degree=1, period=365)
X_t, y_t = pdc.fit_transform(X_train_time, y_train_time)

plt.plot(y_train_time, "bo", label="Signal")
plt.plot(y_t, "rx", label="Detrended/Deseasonalized Signal")
plt.legend()
plt.show()

../_images/user_guide_timeseries_26_0.png

The result is the residual signal, with the trend and seasonality removed. If we want to look at what the component identified as the trend and seasonality, we can call the plot_decomposition() function.

[15]:

res = pdc.plot_decomposition(X_train_time, y_train_time)

/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-evalml/envs/v0.82.0/lib/python3.8/site-packages/woodwork/type_sys/utils.py:40: UserWarning:

The argument 'infer_datetime_format' is deprecated and will be removed in a future version. A strict version of it is now the default, see https://pandas.pydata.org/pdeps/0004-consistent-to-datetime-parsing.html. You can safely remove this argument.

/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-evalml/envs/v0.82.0/lib/python3.8/site-packages/woodwork/type_sys/utils.py:40: UserWarning:

Could not infer format, so each element will be parsed individually, falling back to `dateutil`. To ensure parsing is consistent and as-expected, please specify a format.

../_images/user_guide_timeseries_28_1.png

It is desirable to enhance the decomposer component with a guess at the period of the seasonal aspect of the signal before decomposing it. To do that, we can use the determine_periodicity(X, y) function of the Decomposer class.

[16]:

period = pdc.determine_periodicity(X_train_time, y_train_time)
print(period)

The PolynomialDecomposer class, if not explicitly set in the constructor, will set its period parameter based on a statsmodels function freq_to_period that considers the frequency of the datetime data. This will give a reasonable guess as to what the frequency could be. For example, if the PolynomialDecomposer object is fit with period not explicitly set, it will take on a default value of 7, which is good for daily data signals that have a known seasonal component period that is weekly.

In this case where the seasonal period is not known beforehand, the set_period() convenience function will look at the target data, determine a better guess for the period and set the Decomposer object appropriately.

[17]:

pdc = PolynomialDecomposer()
pdc.fit(X_train_time, y_train_time)
assert pdc.period == 7
pdc.set_period(X_train_time, y_train_time)
assert 350 < pdc.period < 370

STLDecomposer#

The STLDecomposer runs on statsmodels’ implementation of STL decomposition. Let’s take a look at how STL decomposes the weather dataset.

[18]:

from evalml.pipelines.components import STLDecomposer

stl = STLDecomposer()
X_t, y_t = stl.fit_transform(X_train_time, y_train_time)

res = stl.plot_decomposition(X_train_time, y_train_time)

/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-evalml/envs/v0.82.0/lib/python3.8/site-packages/woodwork/type_sys/utils.py:40: UserWarning:

The argument 'infer_datetime_format' is deprecated and will be removed in a future version. A strict version of it is now the default, see https://pandas.pydata.org/pdeps/0004-consistent-to-datetime-parsing.html. You can safely remove this argument.

/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-evalml/envs/v0.82.0/lib/python3.8/site-packages/woodwork/type_sys/utils.py:40: UserWarning:

Could not infer format, so each element will be parsed individually, falling back to `dateutil`. To ensure parsing is consistent and as-expected, please specify a format.

../_images/user_guide_timeseries_34_1.png

This doesn’t look nearly as good as the PolynomialDecomposer did. This is because STL decomposition performs best when the data has a small seasonal period, generally less than 14 time units. The weather dataset’s seasonal period of ~365 days does not work as well since STL extracted a shorter term seasonality for decomposition.

We can generate some synthetic data that better highlights where STL performs well. For this example, we’ll generate monthly data with an annual seasonal period.

[19]:

import random
import numpy as np
from datetime import datetime
from sklearn.preprocessing import minmax_scale


def generate_synthetic_data(
    period=12,
    num_periods=25,
    scale=10,
    seasonal_scale=2,
    trend_degree=1,
    freq_str="M",
):
    freq = 2 * np.pi / period
    x = np.arange(0, period * num_periods, 1)
    dts = pd.date_range(datetime.today(), periods=len(x), freq=freq_str)
    X = pd.DataFrame({"x": x})
    X = X.set_index(dts)

    if trend_degree == 1:
        y_trend = pd.Series(scale * minmax_scale(x + 2))
    elif trend_degree == 2:
        y_trend = pd.Series(scale * minmax_scale(x**2))
    elif trend_degree == 3:
        y_trend = pd.Series(scale * minmax_scale((x - 5) ** 3 + x**2))
    y_seasonal = pd.Series(seasonal_scale * np.sin(freq * x))
    y_random = pd.Series(np.random.normal(0, 1, len(X)))
    y = y_trend + y_seasonal + y_random
    return X, y


X_stl, y_stl = generate_synthetic_data()

Let’s see how the STLDecomposer does at decomposing this data.

[20]:

stl = STLDecomposer()
X_t_stl, y_t_stl = stl.fit_transform(X_stl, y_stl)

res = stl.plot_decomposition(X_stl, y_stl)

../_images/user_guide_timeseries_38_0.png

On top of decomposing this type of data well, the statsmodels implementation of STL automatically determines the seasonal period of the data, which is saved during fit time for this component.

[21]:

stl = STLDecomposer()
assert stl.period == None
stl.fit(X_stl, y_stl)
print(stl.period)

None

Running AutoMLSearch#

AutoMLSearch for time series problems works very similarly to the other problem types with the exception that users need to pass in a new parameter called problem_configuration.

The problem_configuration is a dictionary specifying the following values:

forecast_horizon: The number of time periods we are trying to forecast. In this example, we’re interested in predicting weather for the next 7 days, so the value is 7.
gap: The number of time periods between the end of the training set and the start of the test set. For example, in our case we are interested in predicting the weather for the next 7 days with the data as it is “today”, so the gap is 0. However, if we had to predict the weather for next Monday-Sunday with the data as it was on the previous Friday, the gap would be 2 (Saturday and Sunday separate Monday from Friday). It is important to select a value that matches the realistic delay between the forecast date and the most recently avaliable data that can be used to make that forecast.
max_delay: The maximum number of rows to look in the past from the current row in order to compute features. In our example, we’ll say we can use the previous week’s weather to predict the current week’s.
time_index: The column of the training dataset that contains the date corresponding to each observation. While only some of the models we run during time series searches require the time_index, we require it to be passed in to top level search so that the parameter can reach the models that need it.

Note that the values of these parameters must be in the same units as the training/testing data.

Visualization of forecast horizon and gap#

forecast and gap

[22]:

from evalml.automl import AutoMLSearch

problem_config = {"gap": 0, "max_delay": 7, "forecast_horizon": 7, "time_index": "Date"}

automl = AutoMLSearch(
    X_train,
    y_train,
    problem_type="time series regression",
    max_batches=1,
    problem_configuration=problem_config,
    automl_algorithm="iterative",
    allowed_model_families=[
        "xgboost",
        "random_forest",
        "linear_model",
        "extra_trees",
    ],
)

/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-evalml/envs/v0.82.0/lib/python3.8/site-packages/evalml/automl/automl_search.py:544: UserWarning:

Time series support in evalml is still in beta, which means we are still actively building its core features. Please be mindful of that when running search().

[23]:

automl.search()

/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-evalml/envs/v0.82.0/lib/python3.8/site-packages/woodwork/type_sys/utils.py:40: UserWarning:

The argument 'infer_datetime_format' is deprecated and will be removed in a future version. A strict version of it is now the default, see https://pandas.pydata.org/pdeps/0004-consistent-to-datetime-parsing.html. You can safely remove this argument.

/home/docs/checkouts/readthedocs.org/user_builds/feature-labs-inc-evalml/envs/v0.82.0/lib/python3.8/site-packages/woodwork/type_sys/utils.py:40: UserWarning:

Could not infer format, so each element will be parsed individually, falling back to `dateutil`. To ensure parsing is consistent and as-expected, please specify a format.