Seems like PMPro Vat Tax in not compatible with Stripe Checkout (or in genera with offsite payments maybe).
It works somehow, but not in a consistent way. There’s no way to remove the vat number once added and validated (I had to manually unset the $_SESSION variables).
Discovered after a while trying to complete a checkout w/o a vat number, but still getting the old value from the session.
Patch
/**
* Don't store the VAT number and other infos in the session, incompatible with off-site Stripe Checkout.
*/
remove_action( 'pmpro_checkout_before_processing', 'pmprovat_pmpro_checkout_before_processing' );