re-training n+1 speakers

Hello, 
I tarin 5 speakers, then I need add speaker number 6, but can not use the pre-trained network trained on 5 speaker because get error number of embedding 6 but expecting 5.

How can fix this issue?  dont want to train from scratch take long time. 