Simultaneously call inference method

[delgerdalai]

Last week I tried to host flask + tacotron2 + waveglow.
It was not possible to simultaneously call decoder.inference due to the

tacotron2/model.py

Line 437 in ece7d3f

self.initialize_decoder_states(memory, mask=None)

overwrites previous request's decoder_sate.

I've just sent PR. #176

[OswaldoBornemann]

Does Simultaneously access means that the model could process two requests at the same time ? If that is the case, what's the different between simultaneously access and batch prediction ?

[delgerdalai]

I mean just it wasn't possible to simultaneously call inference method.
I think there is no different between simultaneously access and batch prediction if batch prediction simultaneously call inference method.

[dambaquyen96]

I'm very appreciate your work about this problem! Can you share your deployment source code (Flask + tacotron2 + waveglow)? Thanks in advance

[maxbaluev]

@delgerdalai Yes, please share u source

[delgerdalai]

There is nothing new code. I used code of inference.ipynb (notebook). Only added little change on model.py. You can see pull request.