Update neuralprophet model with new data

[miguelc3]

Hello, is it possible to update a fitted model with new data? So that it can be used in a incremental way with new data every 5 minutes? I think it is possible to do that with fbprophet, but I coudn't find a way with neuralprophet.

Thank you

[mmangione]

Yeah. We do that by saving & loading the model with pickle. Then, when you need to load more data, simply update your dataframe with new data and re-run the fit & predict proceedures. If you set your random seed to a fixed, then you can eliminate the fluctuations introduced by the refit.

[miguelc3]

Thank you for taking the time to answer.

The problem is that I won't have access to all the data because of memory restrictions ... I was hoping for the existence of a method like 'partial_fit(new_data)'

[mmangione]

Ah. That's a much larger challenge. Pickling a model brings its data with it so that these pickles can get really big. You're really looking for some model checkpoint that you can incrementally train from.

[miguelc3]

Yes, I'm going to try thay. It's a very big challenge ... I have a small amount of data for the initial training, and I wanto the model to keep improving with new data ...

[mmangione]

What we do to solve this problem right now is to save the hyperparameters and save the seed we use. We also use a database for the configuration and for the data. When we have more data, we just add it to the db data table for our model target. Then, we use all of the data for our target each time. The sequence looks like:

• Initialize the model with the hyperparameters
• Freeze the random seed to our chosen value
• Train again using all of our available data

That's how we update our models in production. NP is quite lightweight for training. I think it takes us something like 30s to retrain using all of our settings with GPU acceleration. So, the compute time is really minimal compared with other models we run.

The (much harder) alternative is that you can dig into the internals of NP and figure out how to get to each of the values for their time series model. Then, figure out how to load those as your initial state and run your training starting from there for your incremental addition of data. But, you're really going to have to dig deep into the internals and use some pretty fundamental torch functions. It would be much easier to rewrite the model from scratch in Torch and take control of the core training steps. You won't get their optimized training and evaluation insights, but you'll have low-level access to the model controls.

[ourownstory]

@mmangione Thank you for answering this! You are very right on most points. I just wanted to point out, that enabling this functionality is not as difficult as you write. Seems like three changes would be needed: 1) enable re-training of already trained model 2) add option for model-fine tuning type training 3) make sure that large datasets do not affect model pickling.
If someone would like to pick up this project, I would be glad to support them.