Colour shift problem when using regional prompt feature

[mysticfall]

I find the regional prompt this extension provides to be essential in my toolbox since it's one of the very few ways I can compose a complex scene without destroying the overall consistency.

However, I found that it tends to heavily affect the tone and lighting of the scene, sometimes making it look strongly yellow-tinted regardless of whatever I do to counter it.

Is there any way I can mitigate the problem?

[Rkkss]

Use different vae