MAS doesn't accept Synapse migrated macaroons

[benbz]

Describe the bug

MAS doesn't accept Synapse migrated macaroons. When one is used with Synapse MAS responds with e.g.2025-02-23T15:36:38.073459+00:00 lon2 synapse-main-mas[1032]: 2025-02-23T15:36:38.073059Z ERROR http.server.request{otel.kind="server" otel.name="POST /oauth2/introspect" network.protocol.name="http" network.protocol.version="1.1" http.re quest.method="POST" url.path="/oauth2/introspect" url.scheme="http" http.route="/oauth2/introspect" user_agent.original="curl/8.7.1"}:handlers.oauth2.introspection.post{client.id="01JKN8KGSHZ5JS8H183VTT9W42"}: mas_handlers::oauth2::intros pection: crates/handlers/src/oauth2/introspection.rs:160: error=invalid token format

I've replicated directly with MAS hitting the introspection API directly. If I reduce the amount of the Macaroon I pass across in the request I eventually get error=unknown compat access token instead. This Macaroon has - in it which isn't a standard base64 character, so I suspect #3797 needed to only try and decode a subset of the passed Macaroon.

I can provide the migrated Macaroon on request (and will rotate it afterwards)

To Reproduce

• Attempt to interact with Synapse with a Macaroon as the bearer token
• Request fails

Expected behavior

The request succeeds

Additional context

MAS 0.14.1. DB was migrated with syn2mas 0.14.0 or 0.14.1 I can't 100% recall sorry

[sandhose]

It does recognise the right kind of token (that's why it says 'unknown compatibility access token'), so #3797 did the right thing. Question is whether the token was imported in the database or not.

Can you check if:

It exists in the MAS database:

SELECT cs.* FROM compat_access_tokens ct
LEFT JOIN compat_sessions cs USING (compat_session_id)
WHERE access_token = '...';

It existed in the Synapse database:

SELECT id, user_id, device_id FROM access_tokens
WHERE token = '...';

In particular, it looks like we skipped access tokens that did not have a device associated with them (WHERE device_id IS NULL) when importing from Synapse

[benbz]

It does recognise the right kind of token (that's why it says 'unknown compatibility access token'), so #3797 did the right thing. Question is whether the token was imported in the database or not.

For clarity, I get error=invalid token format when using the full token (inc - in it), I only get error=unknown compat access token at specific points when I pass a subset of the token.

It exists in the MAS database:

SELECT cs.* FROM compat_access_tokens ct
LEFT JOIN compat_sessions cs USING (compat_session_id)
WHERE access_token = '...';

          compat_session_id           |               user_id                | device_id  |         created_at         | finished_at | is_synapse_admin | last_active_at | last_active_ip | user_session_id | user_agent | human_name 
--------------------------------------+--------------------------------------+------------+----------------------------+-------------+------------------+----------------+----------------+-----------------+------------+------------
 01772be9-dfd7-9b7a-eb8b-b50870df23bc | 01772be9-ddf8-995c-6233-b8267e8c8501 | VEHHBSGYAM | 2021-01-22 21:04:43.479+00 |             | f                |                |                |                 |            | 
(1 row)

It existed in the Synapse database:

SELECT id, user_id, device_id FROM access_tokens
WHERE token = '...';

 id |         user_id         | device_id  
----+-------------------------+------------
 26 | @alertmanager:banzan.uk | VEHHBSGYAM
(1 row)

[sandhose]

I think I figured it out: macaroons are encoded with base64url, without padding, but MAS tries to base64url decode with padding, and is quite strict about it. To confirm that, add one, two or three = at the end of the token, and the error message should change from 'invalid token format' (= it couldn't base64 decode it) to 'unknown compat access token' (= it did not find it in the database)

[benbz]

Running cargo test in crates/data-model after modifying test_is_likely_synapse_macaroon with my Macaroon (or another one that got migrated) fails

[benbz]

I think I figured it out: macaroons are encoded with base64url, without padding, but MAS tries to base64url decode with padding, and is quite strict about it. To confirm that, add one, two or three = at the end of the token, and the error message should change from 'invalid token format' (= it couldn't base64 decode it) to 'unknown compat access token' (= it did not find it in the database)

Confirmed. 1 Macaroon required == to pass test_is_likely_synapse_macaroon the other required =

[sandhose]

#4093 should fix it. It will be available on the :main docker tag in half an hour or so, time for CI to run

[benbz]

I can confirm that when using sha-ab31de7 as my container image I can successfully introspect both migrated Macaroons that are in my DB now. TYVM!