I am trying to run msckf on WHUVID dataset with stereo+IMU, when running on the fast-speed dataset, there is drift in the beginning ,however then running on the low-speed dataset there is drift in the end.
The upper is fast-speed using evo tool to evaluate:

The other is low-speed:

And I wounder why is like this? I'd appreciate it for your response.