Neural networks for timeseries forecasting

[dberardo-com]

i am using SNARIMAX for timeseries forecasting, but i am not quite happy with it. so i would like to give neural networks a try, to build some more non linear logics.

my input series is a univariate timeseries that gets augmented with this pre-processing step:

def add_time_features(x):
    _x = dict(x)
    timestamp=_x.pop(time_column)
    feat = dict(
        **_x,
        #minute = timestamp.minute,
        hour= timestamp.hour,
        **{'monday': int(timestamp.weekday() == 1),
        'tuesday': int(timestamp.weekday() == 2),
        'wednesday':int( timestamp.weekday() == 3),
        'thursday': int(timestamp.weekday() == 4),
        'friday': int(timestamp.weekday() == 5),
        'saturday': int(timestamp.weekday() == 6)},
        day_of_week= timestamp.weekday(),
        day_of_month= timestamp.day,
        day_of_year = timestamp.timetuple().tm_yday,
        month= timestamp.month,
        year= timestamp.year,
        is_holiday= int(timestamp.date() in it_holidays),
        week_of_year = timestamp.isocalendar()[1],
        quarter = (timestamp.month - 1) // 3 + 1 
    )
    return feat

i am using the following NN to test:

model = (
    add_time_features |
    pp.MinMaxScaler() |
    nn.MLPRegressor(
        hidden_dims=(10,),
        activations=(
            #nn.activations.Identity,nn.activations.Identity,nn.activations.Identity
            #nn.activations.ReLU,nn.activations.Identity,nn.activations.Sigmoid,
            nn.activations.ReLU,nn.activations.ReLU,nn.activations.ReLU
        ),
        optimizer=optim.SGD(1e-3),
        seed=42
    )
)

but i am really not happy with the results (red line is the actual (hourly) signal, blue line is the week-ahead forecast) - notice that end of march / beg. april were holidays:

my questions are:

• is it possible to feedback the previous predicted values to the neural network, so as to add a sort of memory effect to it ?
• is it possible to avoid scaling of the extra features ? i have the feeling that the scalers generate information losses on the augmented features of the input timestamp. wouldn't it be better to change the augmentation function to a normalized version of itself and avoid using scaling altogether? any advice ?

i mean removing scaler from pipeline and using this instead:

def add_time_features(x):
    _x = dict(x)
    timestamp=_x.pop(time_column)
    feat = dict(
        **_x,
        #minute = timestamp.minute,
        hour= timestamp.hour / 23,
        **{'monday': int(timestamp.weekday() == 1),
        'tuesday': int(timestamp.weekday() == 2),
        'wednesday':int( timestamp.weekday() == 3),
        'thursday': int(timestamp.weekday() == 4),
        'friday': int(timestamp.weekday() == 5),
        'saturday': int(timestamp.weekday() == 6)},
        day_of_week= timestamp.weekday() / 6,
        day_of_month= timestamp.day / 31,
        day_of_year = timestamp.timetuple().tm_yday / 365,
        month= timestamp.month / 11,
        year= timestamp.year - 2024 ,
        is_holiday= int(timestamp.date() in it_holidays),
        week_of_year = timestamp.isocalendar()[1] / 52,
        quarter = ((timestamp.month - 1) // 3 + 1 ) / 4
    )
    return feat