Skip to content

Conversation

@corylanou
Copy link
Collaborator

@corylanou corylanou commented Nov 24, 2025

Summary

This PR adds VFS observability features for monitoring replica health and enhances time travel functionality. It provides both SQL functions and PRAGMAs for maximum flexibility:

SQL Functions (via extension auto-extension)

-- Get current transaction ID
SELECT litestream_txid();  -- Returns: 1

-- Get seconds since last successful poll
SELECT litestream_lag();  -- Returns: 2

-- Get current view time
SELECT litestream_time();  -- Returns: 2024-01-15T10:30:00.123456789Z

-- Set time travel with relative expressions
SELECT litestream_set_time('5 minutes ago');
SELECT litestream_set_time('yesterday');
SELECT litestream_set_time('LATEST');  -- Reset to current

PRAGMAs

-- Get current transaction ID
PRAGMA litestream_txid;  -- Returns: 1

-- Get seconds since last successful poll
PRAGMA litestream_lag;  -- Returns: 2

-- Get/set current view time (now returns actual timestamp instead of "latest")
PRAGMA litestream_time;  -- Returns: 2024-01-15T10:30:00.123456789Z

-- Set time travel with relative expressions
PRAGMA litestream_time = '5 minutes ago';
PRAGMA litestream_time = 'yesterday';
PRAGMA litestream_time = '2 hours ago';
PRAGMA litestream_time = LATEST;  -- Reset to current

Changes

  • vfs.go: Add new PRAGMA handlers, fields for tracking state, and time parsing helper
  • vfs_connections.go: Connection-to-VFSFile mapping for SQL function support
  • src/litestream-vfs.c: SQL function implementations and auto-extension registration
  • cmd/litestream-vfs/main.go: CGO exports for SQL functions
  • go.mod: Add github.com/markusmobius/go-dateparser dependency
  • cmd/litestream-vfs/time_travel_test.go: Add/update tests

Test Plan

  • Existing VFS tests pass
  • New PRAGMA tests for litestream_txid, litestream_lag, and relative time parsing
  • Pre-commit hooks pass

Documentation

Created issue benbjohnson/litestream.io#146 to document these features.

Closes #851, #852, #853, #854

Add VFS observability features for monitoring replica health:

- PRAGMA litestream_txid: Return current transaction ID
- PRAGMA litestream_lag: Return seconds since last successful poll
- Extend PRAGMA litestream_time to return actual LTX timestamp
  instead of "latest" when viewing current data
- Add relative time parsing for litestream_time (e.g., "5 minutes ago",
  "yesterday") using go-dateparser library

Fixes #851, #852, #853, #854

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
corylanou and others added 3 commits November 24, 2025 15:44
Add SQL functions (litestream_time(), litestream_txid(), litestream_lag(),
litestream_set_time()) as an alternative interface to the existing PRAGMAs.
This provides more flexibility for queries and monitoring.

SQL Functions (via extension auto-extension):
- SELECT litestream_time() - returns current timestamp
- SELECT litestream_txid() - returns current transaction ID
- SELECT litestream_lag() - returns seconds since last poll
- SELECT litestream_set_time('timestamp') - set time travel target
- SELECT litestream_set_time('LATEST') - reset to latest

PRAGMAs (unchanged):
- PRAGMA litestream_time
- PRAGMA litestream_time = 'timestamp'
- PRAGMA litestream_txid
- PRAGMA litestream_lag

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Keep latestLTXTime in sync when pollReplicaClient processes new LTX
files. Previously it was only set during rebuildIndex(), causing
PRAGMA litestream_time to return stale timestamps after the first poll.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
Move connection mapping code into vfs.go since there's no need for a
separate file - both have the same build tag and are part of the same
logical unit.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@benbjohnson
Copy link
Owner

@corylanou Can you have the litestream_txid PRAGMA & function return the TXID in hex format? There's an ltx.FormatTXID() that'll format it correctly. That's pretty much how every TXID is formatted. We could add a separate litestream_txid_int function in the future if someone wants the raw number but the formatted version is generally more useful.

Change litestream_txid PRAGMA and SQL function to return the TXID as a
16-character lowercase hex string using ltx.TXID.String() instead of
returning it as a decimal integer. This matches how TXIDs are formatted
throughout the rest of the codebase.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@corylanou
Copy link
Collaborator Author

@corylanou Can you have the litestream_txid PRAGMA & function return the TXID in hex format? There's an ltx.FormatTXID() that'll format it correctly. That's pretty much how every TXID is formatted. We could add a separate litestream_txid_int function in the future if someone wants the raw number but the formatted version is generally more useful.

Done!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(vfs): implement litestream_txid() SQLite function

3 participants