takeover value did not converge

[lihua12312]

"Dear authors,
Thank you for your excellent paper. I am writing to ask about an issue I encountered while trying to replicate your work. During my tests, the takeover value did not converge and reached as high as 500.
I was wondering if you have ever encountered a similar situation? If so, could you please share how you resolved it?
Thank you very much for your time and help!"

[pengzhenghao]

Could you describe what experiment you are working on? The takeover rate ~= 100% means the student policy learns to do weird behavior so it triggers expert take over all the time.