SCRAM-SHA-256 Authentication: Design Plan

[SeanTAllen]

SCRAM-SHA-256 Authentication: Design Plan

This document covers the design for adding SCRAM-SHA-256 authentication support to the postgres driver -- the second Tier 1 item in the roadmap. SCRAM-SHA-256 has been the default authentication method since PostgreSQL 10, and MD5 (currently the only method we support) is deprecated.

Change type: Added (new feature). PR label: changelog - added.

Scope

In scope: SCRAM-SHA-256 authentication (mechanism name SCRAM-SHA-256), without channel binding (GS2 flag n). MD5 authentication continues to work alongside SCRAM -- the server determines which method to use via pg_hba.conf, and the driver handles whichever auth request type the server sends (type 5 for MD5, type 10 for SASL/SCRAM).

Out of scope:

• SCRAM-SHA-256-PLUS (channel binding) -- follow-up work
• SASLprep password normalization -- PostgreSQL does not enforce normalization, and ASCII passwords work without it. Can be added later if needed.
• Other SASL mechanisms

Protocol Flow

SCRAM-SHA-256 replaces the single-round-trip MD5 exchange with a multi-step SASL handshake:

Client                                Server
  |                                     |
  |--- StartupMessage ----------------->|
  |                                     |
  |<-- AuthenticationSASL(10) ----------|  Mechanism list includes "SCRAM-SHA-256"
  |                                     |
  |--- SASLInitialResponse('p') ------->|  client-first-message: n,,n=,r=<nonce>
  |                                     |
  |<-- AuthenticationSASLContinue(11) --|  server-first-message: r=<nonce>,s=<salt>,i=<iterations>
  |                                     |
  |--- SASLResponse('p') ------------->|  client-final-message: c=biws,r=<nonce>,p=<proof>
  |                                     |
  |<-- AuthenticationSASLFinal(12) ----|  server-final-message: v=<signature>
  |                                     |
  |<-- AuthenticationOk(0) ------------|

The crypto operations:

1. PBKDF2(password, salt, iterations) -> SaltedPassword
2. HMAC(SaltedPassword, "Client Key") -> ClientKey; SHA-256(ClientKey) -> StoredKey
3. HMAC(StoredKey, AuthMessage) -> ClientSignature; ClientKey XOR ClientSignature -> ClientProof
4. HMAC(SaltedPassword, "Server Key") -> ServerKey; HMAC(ServerKey, AuthMessage) -> ServerSignature

The client sends the ClientProof; the server returns the ServerSignature. The client verifies the ServerSignature to authenticate the server (mutual authentication).

Wire Format

AuthenticationSASL (type 10): Backend message. Same 'R' byte as other auth messages. After the Int32(10) type code: null-terminated mechanism name strings, terminated by an empty string (single null byte).

SASLInitialResponse (frontend, type 'p'): String(mechanism) Int32(response_length) Bytes(client-first-message). The mechanism name is null-terminated. The response length is the byte length of the client-first-message. A length of -1 means no initial response.

AuthenticationSASLContinue (type 11): Backend message. After Int32(11): raw bytes of server-first-message to end of message (not null-terminated).

SASLResponse (frontend, type 'p'): Raw bytes of client-final-message. Not null-terminated.

AuthenticationSASLFinal (type 12): Backend message. After Int32(12): raw bytes of server-final-message to end of message.

SCRAM Message Content

• client-first-message: n,,n=,r=<client_nonce> -- GS2 header n,, (no channel binding, no authzid), empty username (following libpq convention; PostgreSQL ignores the SCRAM username and uses the StartupMessage username)
• server-first-message: r=<combined_nonce>,s=<base64_salt>,i=<iterations> -- the combined nonce is the client nonce with the server's nonce appended
• client-final-message: c=biws,r=<combined_nonce>,p=<base64_proof> -- biws is base64("n,,")
• server-final-message: v=<base64_server_signature> (success) or e=<error> (failure)

The AuthMessage used for HMAC computation is: client-first-message-bare + "," + server-first-message + "," + client-final-message-without-proof, where "bare" means without the GS2 header (n=,r=<nonce>), and "without-proof" means without the ,p=... suffix (c=biws,r=<combined_nonce>).

Implementation Approach

Crypto Primitives

The implementation requires HMAC-SHA-256, PBKDF2, and secure random byte generation, none of which are currently available in ponylang/ssl. SHA-256 (Digest.sha256()) and constant-time comparison (ConstantTimeCompare) are already available. Base64 encoding is in Pony stdlib. See Decision 1 for where the missing primitives should be implemented.

State Machine

Add a new _SessionSCRAMAuthenticating state to handle the multi-step SASL exchange. This follows the precedent of _SessionSSLNegotiating -- a separate state class for a multi-step protocol exchange with intermediate data.

_SessionConnected  --AuthSASL(10)-->  _SessionSCRAMAuthenticating
_SessionSCRAMAuthenticating  --SASLFinal(12) ok + AuthOk(0)-->  _SessionLoggedIn
_SessionSCRAMAuthenticating  --error/failure-->  _SessionClosed

On receiving AuthenticationSASL(10), _SessionConnected (via _AuthenticableState.on_authentication_sasl):

1. Validates that SCRAM-SHA-256 is in the mechanism list (if not, see Decision 3)
2. Generates a client nonce (base64-encoded output of 24 random bytes from RAND_bytes, which naturally avoids the forbidden , character)
3. Builds and sends the SASLInitialResponse with client-first-message
4. Transitions to _SessionSCRAMAuthenticating carrying: client nonce, client-first-message-bare, user credentials, password, notify reference, and the readbuf

_SessionSCRAMAuthenticating handles these callbacks:

• on_authentication_sasl_continue(msg) -- parses server-first-message, validates the combined nonce starts with the client nonce, computes SaltedPassword/ClientProof/ServerSignature, sends SASLResponse, stores expected ServerSignature
• on_authentication_sasl_final(msg) -- verifies server signature using ConstantTimeCompare on raw decoded bytes (not base64 strings); on mismatch, terminates the connection (see Decision 2)
• on_authentication_ok() -- transitions to _SessionLoggedIn
• on_authentication_failed() -- notifies client and shuts down (same behavior as _AuthenticableState; handles ErrorResponse with codes 28000/28P01 during the SCRAM exchange, which _ResponseMessageParser already intercepts and routes to this callback)

The state mixes in _ConnectedState (for write capability and readbuf handling) and implements auth callbacks directly -- it cannot use _AuthenticableState because it needs different callback combinations (no on_authentication_md5_password, but has SASL-specific callbacks). on_authentication_md5_password and on_authentication_sasl default to _IllegalState().

Trait hierarchy changes for the new callbacks: Three new methods are added to _SessionState: on_authentication_sasl, on_authentication_sasl_continue, on_authentication_sasl_final. Default _IllegalState() implementations for all three go into _NotAuthenticableState, which covers all states except _SessionConnected and _SessionSCRAMAuthenticating. _AuthenticableState (used by _SessionConnected) provides the real on_authentication_sasl implementation and _IllegalState() for on_authentication_sasl_continue / on_authentication_sasl_final (which should never arrive while still in _SessionConnected). _SessionSCRAMAuthenticating provides real implementations for on_authentication_sasl_continue and on_authentication_sasl_final and _IllegalState() for on_authentication_sasl and on_authentication_md5_password.

Parser Changes

_authentication_request_type.pony: Add sasl(): I32 => 10, sasl_continue(): I32 => 11, sasl_final(): I32 => 12.

_backend_messages.pony: Three new message classes:

• _AuthenticationSASLMessage -- carries mechanism list as Array[String] val
• _AuthenticationSASLContinueMessage -- carries raw server-first-message as Array[U8] val
• _AuthenticationSASLFinalMessage -- carries raw server-final-message as Array[U8] val

Add these to _AuthenticationMessages union type.

_response_parser.pony: Extend the auth type dispatch in the authentication_request() branch to handle types 10, 11, 12. For type 10, parse null-terminated strings until empty string. For 11 and 12, extract remaining bytes as raw data.

_response_message_parser.pony: Route the three new message types to session state callbacks.

Frontend Messages

_frontend_message.pony: Two new message builders:

• sasl_initial_response(mechanism: String, response: Array[U8] val): Array[U8] val -- type 'p', body is null-terminated mechanism name + Int32(response length) + response bytes
• sasl_response(response: Array[U8] val): Array[U8] val -- type 'p', body is raw response bytes

Testing

Crypto unit tests: HMAC-SHA-256 against RFC 4231 test vectors. PBKDF2 against RFC 6070 test vectors.

SCRAM logic unit tests: Full SCRAM computation against the RFC 5802 test vector (username "user", password "pencil", known nonce/salt/iterations -> expected proof and server signature).

Protocol unit tests (mock server): Following the _TestSSLNegotiation* pattern:

• Mock SCRAM server completes full handshake -> pg_session_authenticated fires
• Server offers no supported mechanisms -> connection fails (see Decision 3)
• Server returns bad server signature -> connection terminates (see Decision 2)
• Server sends ErrorResponse during SCRAM exchange -> pg_session_authentication_failed fires

Frontend message unit tests: Verify wire format of SASLInitialResponse and SASLResponse.

Parser unit tests: Verify parsing of AuthenticationSASL(10), AuthenticationSASLContinue(11), AuthenticationSASLFinal(12).

Integration tests: Connect to PostgreSQL configured with SCRAM-SHA-256: successful auth, failed auth (wrong password), query execution after SCRAM auth, SSL + SCRAM combined.

Build and run: make ssl=3.0.x (all tests), make unit-tests ssl=3.0.x (unit only).

Example

No new example is needed. SCRAM-SHA-256 is negotiated transparently -- the server determines the auth method, not the client. Existing examples work unchanged against a SCRAM-configured server.

Design Decisions

Decision 1: Where to implement HMAC-SHA-256, PBKDF2, and RAND_bytes

These are general-purpose cryptographic primitives that don't exist in ponylang/ssl today. All three have stable OpenSSL C APIs (HMAC(), PKCS5_PBKDF2_HMAC(), RAND_bytes()).

Option A -- Add to ponylang/ssl: Add FFI wrappers in the ssl/crypto package. Architecturally clean -- crypto primitives belong in the crypto package, and ponylang/ssl already has the OpenSSL FFI infrastructure. This would be a prerequisite: design, implement, test, review, release ponylang/ssl, then update the dependency in postgres.

Option B -- Implement in postgres via direct OpenSSL FFI: Add use @HMAC[...] and use @PKCS5_PBKDF2_HMAC[...] declarations directly in the postgres package. Faster to ship but puts crypto code in the wrong package and duplicates FFI infrastructure.

Decision 2: How to report server signature verification failure

If the server's signature in AuthenticationSASLFinal doesn't match our computed ServerSignature, the connection must be terminated. This indicates a potential MITM attack. How should this be surfaced to the user?

Option A -- New AuthenticationFailureReason variant: Add ServerVerificationFailed to AuthenticationFailureReason. Fire pg_session_authentication_failed with this reason. The application can distinguish "wrong password" (InvalidPassword) from "server couldn't verify itself" (ServerVerificationFailed).

Option B -- Use pg_session_connection_failed: Treat it as a connection-level failure rather than an authentication failure, since the issue is the server's identity, not the client's credentials.

Option C -- Reuse InvalidAuthenticationSpecification: Simple, but lumps together very different failure modes.

Decision 3: Unsupported mechanism handling

If the server sends AuthenticationSASL(10) but the mechanism list doesn't include SCRAM-SHA-256, how should we handle it? The current behavior for truly unsupported auth types (e.g., cleartext) is that _ResponseParser returns _UnsupportedMessage, which is silently consumed by _ResponseMessageParser, and the session hangs until timeout.

Option A -- Fire pg_session_authentication_failed with InvalidAuthenticationSpecification: Explicit failure. Consistent with how we handle auth errors.

Option B -- Fire pg_session_connection_failed: Treats it as a connection-level incompatibility. Consistent with how SSL refusal is handled.

Note: either option is an improvement over the current silent hang for unsupported auth types. Whichever we pick, we could also consider retroactively improving the _UnsupportedMessage handling.

Decision 4: Integration test infrastructure

The current CI uses PostgreSQL containers with MD5 auth (user: postgres, password: postgres). We need SCRAM-configured PostgreSQL for integration tests.

Option A -- Add SCRAM user to existing containers: Configure pg_hba.conf to use scram-sha-256 for a specific user while keeping md5 for the existing user. Least infrastructure change.

Option B -- Add new container pair (plain + SSL) with SCRAM auth: Separate containers on separate ports. Clean separation but more CI resources.

Option C -- Switch to SCRAM default, keep one MD5 user: Make SCRAM the default (matching modern PostgreSQL defaults), add a dedicated MD5 user for backward-compat tests.

Phasing

1. Prerequisite (if Decision 1 = A): Add HMAC-SHA-256, PBKDF2, and RAND_bytes to ponylang/ssl. Separate repo, separate PR cycle.
2. Core implementation: SCRAM state, parser, frontend messages, crypto logic, unit tests with mock servers.
3. Integration tests and CI: Container configuration changes, integration test suite.
4. Documentation: Release notes, CHANGELOG, CLAUDE.md updates.

[SeanTAllen]

decision 1: we want to do the work in ponylang/ssl
decision 2: option b, but ideally we can distinguish failure types.
decision 3: this feels very much like it should be option A, but that is making me question my feeling for decision 2 when it is considered by itself.
decision 4: i think option C is a reasonable option.

[SeanTAllen]

The ponylang/ssl prerequisite has been implemented and released in ponylang/ssl 1.0.3