On device fine-tuning?

[Tejaswgupta]

I've fine-tuned a model for hindi which is giving pretty good accuracy. However , user's might have unique words which aren't in the model vocabulary. Is it possible to integrate some sort of active/continuous learning mechanism to incorporate each individuals's vocabulary when they use the model.
We can collect data when the user makes a change to the transcription marking the error and the correct alternative.

Any pointers or suggestions will be great. Thanks!