When a SLO URL is not set, the SAML backend does not offer the option to logout.
Providing a custom logout URL (or disabling it) is a user backend feature. As long as a user, that is provisioned through a different backend (e.g. LDAP) has the session after the SAML login, according to session data the SAML backend can enforce its logout URL policy.
After a browser restart however, this information is not present anymore, so the SAML backend does not kick into action and the default logout URL is being used.