SSL/TLS Negotiation: Research and Design Decision

[SeanTAllen]

Research and design decision for adding SSL/TLS negotiation to the ponylang/postgres driver (item 1 from the feature roadmap).

PostgreSQL SSL Negotiation Protocol

PostgreSQL uses a STARTTLS-style negotiation that happens before the StartupMessage:

1. Client opens a plain TCP connection
2. Client sends an 8-byte SSLRequest message: Int32(8) | Int32(80877103)
3. Server responds with a single byte: S (proceed with TLS) or N (no SSL)
4. If S: client performs a TLS handshake on the existing TCP socket
5. Client sends StartupMessage (now encrypted)
6. Normal authentication and query flow continues over TLS

The SSLRequest code (80877103 = 1234 << 16 | 5679) is intentionally distinct from any valid protocol version number. The server's response is a raw byte, not a standard protocol message.

sslmode Behaviors

sslmode	Sends SSLRequest?	Server says N	Server says S	Cert verification
disable	No	N/A	N/A	None
prefer (default)	Yes	Falls back to plaintext	TLS handshake	None
require	Yes	Aborts	TLS handshake	None
verify-ca	Yes	Aborts	TLS handshake	CA chain
verify-full	Yes	Aborts	TLS handshake	CA chain + hostname

PostgreSQL 17+ Direct SSL

PostgreSQL 17 added sslnegotiation=direct, where the client starts TLS immediately (no SSLRequest round-trip). This requires ALPN extension with protocol name "postgresql". Only works with sslmode=require or higher.

Security Considerations

• CVE-2021-23222: After receiving S, a MITM could inject bytes before the TLS handshake. The client must read exactly one byte, not buffer additional data.
• CVE-2024-10977: If the server responds with E (ErrorResponse) to SSLRequest, the server is unauthenticated — the client must not trust or display the error content.

Current Driver Architecture

The connection flow in session.pony:

Session.create() → TCPConnection.client(plain TCP)
on_connected()   → sends StartupMessage immediately
on_received()    → _ResponseParser → _ResponseMessageParser → auth flow

SSL negotiation needs to be inserted between on_connected() and sending the StartupMessage.

Approaches Considered

Option 1: Modify lori to support TLS upgrade (chosen)

Add a start_tls(sslctx) method to lori's TCPConnection that initiates a TLS handshake on an existing plain TCP connection. The postgres driver would:

1. Connect with plain TCP via TCPConnection.client()
2. Send SSLRequest (8 bytes, plaintext)
3. Read the single-byte response
4. If S: call connection.start_tls(sslctx) — lori handles the handshake
5. After handshake completes, send StartupMessage (encrypted)
6. If N: either send StartupMessage in plaintext or abort, depending on sslmode

Pros:

• TLS handling stays in lori where it belongs
• Lori already has all the SSL internals (_ssl_flush_sends, _ssl_poll, etc.) — this just allows initializing them post-connection
• Benefits any Pony project needing STARTTLS (SMTP, LDAP, XMPP, etc.)
• Clean separation: postgres handles protocol negotiation, lori handles TLS

Cons:

• Requires a lori release before postgres can use it
• Two-project coordination

Option 2: Manual SSL layer in the postgres driver (rejected)

Use lori for plain TCP, but handle TLS ourselves using the SSL class directly — intercept all reads/writes to encrypt/decrypt manually.

Rejected because: Duplicates what lori already does internally. Error-prone. Mixes transport concerns into the postgres driver.

Option 3: Direct SSL only via ssl_client() (rejected)

Use lori's existing TCPConnection.ssl_client() which starts TLS immediately.

Rejected because: Only works with PostgreSQL 17+ servers configured for direct SSL. Incompatible with the vast majority of PostgreSQL deployments. The integration test environment uses PostgreSQL 14.5.

Decision

Going with Option 1 — modify lori to support TLS upgrade. It's the correct, most maintainable path forward. TLS is a transport concern that belongs in the networking library, and the STARTTLS pattern is common across many protocols.

Tracking issue: ponylang/lori#170

[SeanTAllen]

Lori tracking issue for TLS upgrade support: ponylang/lori#170

[SeanTAllen]

this has been merged.

[SeanTAllen]

SSL/TLS Negotiation: Implementation Design

Implementation plan for adding SSL/TLS negotiation to the postgres driver, building on the start_tls() support merged in ponylang/lori#171.

Prerequisite: a lori release containing start_tls().

Scope Boundaries

In scope:

• SSLDisable mode (current behavior, the default)
• SSLRequire mode (negotiate SSL, abort if server refuses)
• SSLRequest wire protocol message
• Session state machine changes for SSL negotiation
• Unit tests (wire format, negotiation failure, negotiation success with mock TLS)
• Integration tests (full SSL connection to PostgreSQL)
• SSL-enabled test infrastructure (docker container with certificates)
• Example demonstrating SSL usage

Out of scope (future work):

• SSLPrefer mode (fallback to plaintext if server refuses)
• SSLVerifyCA / SSLVerifyFull modes (certificate verification)
• PostgreSQL 17+ direct SSL (sslnegotiation=direct)
• Client certificate authentication

Public API Changes

New types (new file: ssl_mode.pony)

use "ssl/net"

primitive SSLDisable
  """
  Do not use SSL. The connection is plaintext. This is the default.
  """

class val SSLRequire
  """
  Require SSL. The driver sends an SSLRequest during connection setup and
  aborts if the server refuses. The connection is encrypted before
  authentication begins.

  The `SSLContext` controls certificate and cipher configuration. Create one
  using the `ssl/net` package:

      let sslctx = recover val
        SSLContext
          .> set_client_verify(false)
          .> set_server_verify(false)
      end
      SSLRequire(sslctx)
  """
  let ctx: SSLContext val

  new val create(ctx': SSLContext val) =>
    ctx = ctx'

type SSLMode is (SSLDisable | SSLRequire)

Carrying the SSLContext inside SSLRequire makes it impossible to request SSL without providing a context — the illegal state is unrepresentable. SSLDisable is a primitive (no configuration needed).

Users must use "ssl/net" in their own code to create an SSLContext val. The ssl package is already a transitive dependency through lori, so no new library dependency is needed — just the import.

Session.create() signature change

new create(
  auth': lori.TCPConnectAuth,
  notify': SessionStatusNotify,
  host': String,
  service': String,
  user': String,
  password': String,
  database': String,
  ssl': SSLMode = SSLDisable)

The default SSLDisable preserves backwards compatibility. Existing code compiles without changes.

SessionStatusNotify: no new callbacks

SSL negotiation failure (server refused or TLS handshake failed) fires pg_session_connection_failed. From the user's perspective, the session failed to connect — they don't need to distinguish TCP failure from SSL failure to take action (both mean "could not connect").

pg_session_connected shifts semantics from "TCP connected" to "connection ready for authentication." For SSLDisable, this fires immediately after TCP connect (identical to today). For SSLRequire, this fires after the TLS handshake completes. Since SSL is a new feature, no existing code is affected by this semantic shift.

Wire Protocol

_FrontendMessage.ssl_request()

New method building the SSLRequest message. Format: Int32(8) | Int32(80877103) — 8 bytes total, no message type byte (same pattern as startup()). The magic number 80877103 equals 1234 << 16 | 5679.

Session State Machine Changes

New state: _SessionSSLNegotiating

Inserted between TCP connection and the existing _SessionConnected state:

SSLDisable path (unchanged):
  _SessionUnopened  --on_connected-->  _SessionConnected
    (fires pg_session_connected, sends StartupMessage)

SSLRequire path (new):
  _SessionUnopened  --on_connected-->  _SessionSSLNegotiating
    (sends SSLRequest, does NOT fire pg_session_connected)

  _SessionSSLNegotiating  --receive 'S'-->  calls start_tls(), waits
  _SessionSSLNegotiating  --on_tls_ready-->  _SessionConnected
    (fires pg_session_connected, sends StartupMessage)

  _SessionSSLNegotiating  --receive 'N'-->  pg_session_connection_failed, _SessionClosed
  _SessionSSLNegotiating  --on_tls_failure-->  pg_session_connection_failed, _SessionClosed
  _SessionSSLNegotiating  --start_tls() error-->  pg_session_connection_failed, _SessionClosed

_SessionSSLNegotiating details

Stored state:

• _notify: SessionStatusNotify, _user: String, _password: String, _database: String — carried forward to _SessionConnected on success
• _ssl_ctx: SSLContext val — for start_tls() call
• _host: String — for SNI in start_tls()
• _handshake_started: Bool — distinguishes "waiting for server byte" from "waiting for TLS handshake"

on_received: If _handshake_started is true, shuts down (should never happen — lori handles socket I/O during the handshake and doesn't deliver application data until _on_tls_ready). Otherwise, reads the server's single-byte response. On S, calls s._connection().start_tls(_ssl_ctx, _host), checks the return value (any StartTLSError triggers connection failure and shutdown), and sets _handshake_started = true. On N, fires pg_session_connection_failed and shuts down. Any other byte causes shutdown.

on_tls_ready: Resets expect(0) on the connection (critical — lori preserves the expect(1) value across start_tls() via _ssl_expect, so decrypted data would be delivered 1 byte at a time without this reset, breaking _ResponseParser). Then transitions to _SessionConnected, fires pg_session_connected, sends StartupMessage.

on_tls_failure: Fires pg_session_connection_failed, transitions to _SessionClosed. Lori follows _on_tls_failure() with _on_closed(). Since Session does not override _on_closed() (lori's default is a no-op), no additional handling is needed. If _on_closed() is ever added to Session, it must handle the _SessionClosed state gracefully — document this coupling on the Session actor with a comment noting that _on_tls_failure already completes cleanup.

Trait composition: Cannot reuse _ConnectedState because that trait routes on_received through _ResponseParser. Instead, _SessionSSLNegotiating mixes in _NotConnectableState (illegal-state defaults for on_connected/on_failure), _NotAuthenticableState (auth events), and _NotAuthenticated (query events). It implements close()/shutdown()/on_received()/process_responses() directly. process_responses is a no-op — _ResponseMessageParser is not involved during SSL negotiation, and _process_again() may fire but has nothing to process.

execute()/prepare()/close_statement(): Fail immediately with appropriate errors (SessionNotAuthenticated / silent no-op), same as _SessionConnected.

_SessionState interface additions

fun ref on_tls_ready(s: Session ref)
fun ref on_tls_failure(s: Session ref)

Default _IllegalState() implementations added to _UnconnectedState and _ConnectedState traits. These two traits cover all existing states. _SessionSSLNegotiating provides its own implementations.

_ConnectableState changes

trait _ConnectableState is _UnconnectedState
  fun on_connected(s: Session ref) =>
    match ssl_mode()
    | SSLDisable =>
      s.state = _SessionConnected(notify(), user(), password(), database())
      notify().pg_session_connected(s)
      _send_startup_message(s)
    | let req: SSLRequire =>
      // Set expect(1) BEFORE sending SSLRequest so lori delivers exactly
      // one byte per _on_received call. Any MITM-injected bytes stay in
      // lori's internal buffer, causing start_tls() to return
      // StartTLSNotReady (CVE-2021-23222 mitigation).
      try s._connection().expect(1)? end
      let st = _SessionSSLNegotiating(
        notify(), user(), password(), database(), req.ctx, host())
      s.state = st
      st.send_ssl_request(s)
    end

  fun ssl_mode(): SSLMode
  fun host(): String
  // existing: user(), password(), database(), notify()

_SessionUnopened gains _ssl_mode: SSLMode and _host: String fields to supply these. The host' parameter from Session.create() is currently passed only to lori.TCPConnection.client() and not stored. With SSL, Session.create() must also pass host' through to _SessionUnopened, which stores it and exposes it via the host() method. The value flows: Session.create(host') → _SessionUnopened._host → _ConnectableState.host() → _SessionSSLNegotiating._host → start_tls(ssl_ctx, _host) for SNI.

Session actor additions

fun ref _on_tls_ready() =>
  state.on_tls_ready(this)

fun ref _on_tls_failure() =>
  state.on_tls_failure(this)

These override the default None implementations from lori's ClientLifecycleEventReceiver.

CVE-2021-23222 Mitigation

A MITM could inject bytes between the server's S response and the TLS handshake. Lori's start_tls() precondition rejects the upgrade if there's unprocessed data in the read buffer (_bytes_in_read_buffer > 0), preventing injected data from crossing the plaintext/TLS boundary.

To make this precondition effective, set expect(1) on the connection before sending the SSLRequest. This ensures lori delivers exactly 1 byte per _on_received call and retains any extra bytes in its internal buffer. If a MITM injected bytes after S, they'd remain in lori's buffer, and start_tls() would return StartTLSNotReady. The driver treats any StartTLSError as a connection failure.

After _on_tls_ready(), reset to expect(0) (deliver all available bytes) before continuing to normal protocol handling.

Testing

Build and run commands

make ssl=3.0.x                    # build and run all tests
make unit-tests ssl=3.0.x         # unit tests only (no postgres needed)
make integration-tests ssl=3.0.x  # integration tests (needs SSL-enabled postgres)

Unit tests (no external dependencies)

1. _TestFrontendMessageSSLRequest — verify the 8-byte wire format.

2. _TestSSLNegotiationRefused — mock TCP server (same pattern as existing _TestHandlingJunkMessages) that accepts a connection, reads the SSLRequest, and responds with N. Verify that pg_session_connection_failed fires.

3. _TestSSLNegotiationJunkResponse — mock TCP server that responds with a byte that's neither S nor N. Verify session shuts down.

4. _TestSSLNegotiationSuccess — mock TCP server using lori's TLS infrastructure (same pattern as lori's _TestStartTLSPingPong). Server accepts the connection, reads the 8-byte SSLRequest, sends S, performs start_tls() on its end, then reads and validates the StartupMessage arrives over TLS. Requires self-signed test certificates in the repo. This tests the full negotiation-to-StartupMessage flow without requiring PostgreSQL.

Integration tests (require SSL-enabled PostgreSQL)

5. integration/SSL/Connect — connect with SSLRequire to PostgreSQL, verify pg_session_connected and pg_session_authenticated fire.

6. integration/SSL/Query — execute a query over an SSL connection, verify results.

7. integration/SSL/DisableStillWorks — connect with SSLDisable to the SSL-enabled PostgreSQL, verify plaintext still works. Confirms the server handles both modes.

Test infrastructure changes

• Generate self-signed test certificates and add to the repo (e.g., assets/test-cert.pem, assets/test-key.pem).
• Modify the docker container setup (make start-pg-container) to enable SSL: copy certificates into the container, set ssl = on in postgresql.conf.
• The same container serves both SSL and non-SSL connections on the same port (SSLRequest negotiation determines which), so existing integration tests are unaffected.

Example

New examples/ssl-query/ssl-query-example.pony demonstrating:

• Creating an SSLContext val with ssl/net
• Passing SSLRequire(sslctx) to Session.create()
• Executing a query over the encrypted connection
• Handling both pg_session_connected (SSL established) and pg_session_authenticated

Implementation Order

1. Lori release with start_tls() (prerequisite — Sean)
2. Bump lori dependency in corral.json
3. Wire protocol: _FrontendMessage.ssl_request() + unit test
4. Public types: SSLMode, SSLDisable, SSLRequire in ssl_mode.pony
5. State machine: _SessionSSLNegotiating, _SessionState interface additions, _ConnectableState/_SessionUnopened changes, Session actor TLS callbacks
6. Test certificates: generate self-signed certs, add to repo
7. Unit tests: wire format, negotiation refused, junk response, success with mock TLS
8. Docker changes: SSL-enabled PostgreSQL container
9. Integration tests: SSL connect, query, disable-still-works
10. Example: examples/ssl-query/
11. Update CLAUDE.md and package docstring

Files Changed

New files:

• postgres/ssl_mode.pony — SSLMode, SSLDisable, SSLRequire
• examples/ssl-query/ssl-query-example.pony
• assets/test-cert.pem, assets/test-key.pem (test certificates)

Modified files:

• corral.json — bump lori version
• postgres/session.pony — Session.create(), _on_tls_ready/_on_tls_failure, _SessionUnopened, _ConnectableState, _SessionState interface, _UnconnectedState, _ConnectedState, new _SessionSSLNegotiating class
• postgres/_frontend_message.pony — ssl_request() method
• postgres/_test.pony — test list additions
• postgres/_test_frontend_message.pony — SSLRequest wire format test
• postgres/_test.pony / new test file — SSL negotiation tests
• Makefile — docker container SSL configuration

[SeanTAllen]

Implemented in #79