Skip to content

feat(websocket): add after_timestamp filter for bi-directional event loading#1880

Merged
enyst merged 10 commits intomainfrom
feature/websocket-after-timestamp-filter
Mar 20, 2026
Merged

feat(websocket): add after_timestamp filter for bi-directional event loading#1880
enyst merged 10 commits intomainfrom
feature/websocket-after-timestamp-filter

Conversation

@jpshackelford
Copy link
Copy Markdown
Contributor

@jpshackelford jpshackelford commented Feb 2, 2026

Summary

Add an after_timestamp query parameter to the WebSocket events endpoint that filters events during resend_all to only return events with timestamps >= the specified value.

Background

This is a prerequisite for the bi-directional event loading optimization described in OpenHands/OpenHands#12705.

Changes

  • Added after_timestamp: datetime | None query parameter to /sockets/events/{conversation_id} WebSocket endpoint
  • When resend_all=True and after_timestamp is provided, only events with timestamp >= after_timestamp are resent
  • Added comprehensive tests for the new parameter

Use Case

This enables efficient bi-directional loading where:

  1. REST API fetches historical events (paginated backwards from newest)
  2. WebSocket handles real-time events after a specific timestamp
Timeline:
[oldest] ←←←← REST (backwards) ←←←← [newest] →→→→ WebSocket (forward) →→→→ [new events]

The underlying search_events already supports timestamp__gte filtering, so this change simply exposes that capability through the WebSocket API.

Testing

  • Added 3 new tests in TestAfterTimestampFiltering class
  • Updated existing test_resend_all_true_resends_events to verify the new parameter passing

Related Issue

Resolves: OpenHands/OpenHands#12705 (agent-server prerequisite)


Agent Server images for this PR

GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant Architectures Base Image Docs / Tags
java amd64, arm64 eclipse-temurin:17-jdk Link
python amd64, arm64 nikolaik/python-nodejs:python3.13-nodejs22 Link
golang amd64, arm64 golang:1.21-bookworm Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:bce2fa7-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-bce2fa7-python \
  ghcr.io/openhands/agent-server:bce2fa7-python

All tags pushed for this build

ghcr.io/openhands/agent-server:bce2fa7-golang-amd64
ghcr.io/openhands/agent-server:bce2fa7-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:bce2fa7-golang-arm64
ghcr.io/openhands/agent-server:bce2fa7-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:bce2fa7-java-amd64
ghcr.io/openhands/agent-server:bce2fa7-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:bce2fa7-java-arm64
ghcr.io/openhands/agent-server:bce2fa7-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:bce2fa7-python-amd64
ghcr.io/openhands/agent-server:bce2fa7-nikolaik_s_python-nodejs_tag_python3.13-nodejs22-amd64
ghcr.io/openhands/agent-server:bce2fa7-python-arm64
ghcr.io/openhands/agent-server:bce2fa7-nikolaik_s_python-nodejs_tag_python3.13-nodejs22-arm64
ghcr.io/openhands/agent-server:bce2fa7-golang
ghcr.io/openhands/agent-server:bce2fa7-java
ghcr.io/openhands/agent-server:bce2fa7-python

About Multi-Architecture Support

  • Each variant tag (e.g., bce2fa7-python) is a multi-arch manifest supporting both amd64 and arm64
  • Docker automatically pulls the correct architecture for your platform
  • Individual architecture tags (e.g., bce2fa7-python-amd64) are also available if needed

Copy link
Copy Markdown
Collaborator

@all-hands-bot all-hands-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall solid implementation! The feature is well-tested and maintains backward compatibility. Main concern is timezone handling - the parameter should enforce or document timezone-aware datetimes to avoid potential bugs.

Comment thread openhands-agent-server/openhands/agent_server/sockets.py Outdated
Comment thread openhands-agent-server/openhands/agent_server/sockets.py Outdated
Comment thread openhands-agent-server/openhands/agent_server/sockets.py Outdated
Comment thread tests/agent_server/test_event_router_websocket.py Outdated
Comment thread tests/agent_server/test_event_router_websocket.py
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Feb 2, 2026

Coverage

Coverage Report •
FileStmtsMissCoverMissing
openhands-agent-server/openhands/agent_server
   event_router.py47882%172–175, 185–186, 208–209
   sockets.py1312283%145, 150–152, 261–262, 267–269, 275–277, 282–283, 285–286, 311–315, 325
TOTAL20711928855% 

@enyst
Copy link
Copy Markdown
Collaborator

enyst commented Feb 2, 2026

@OpenHands Do a /codereview-roasted on this PR.

When you’re done, look at the linked issue and tell WDYT.

Post your feedback directly as a comment on github here. Use smolpaws. Note that it will be rendered as markdown.

@openhands-ai
Copy link
Copy Markdown

openhands-ai bot commented Feb 2, 2026

I'm on it! enyst can track my progress at all-hands.dev

@smolpaws
Copy link
Copy Markdown
Contributor

smolpaws commented Feb 2, 2026

Taste Rating: 🟡 Acceptable — small change, but there’s a correctness landmine hiding in the timestamp handling.


Linus-Style Analysis

[CRITICAL ISSUES]

  • [openhands-agent-server/openhands/agent_server/event_service.py, Lines 110–141; openhands-agent-server/openhands/agent_server/sockets.py, Lines 81–89] Correctness / Data semantics: after_timestamp is a datetime, but events store timestamp as string and comparisons are lexicographic. That “works” only if all timestamps are normalized to the same ISO format and timezone handling is consistent. Right now events default to datetime.now().isoformat() (naive), while after_timestamp can be tz-aware, which means "2024-01-15T10:30:00+00:00" vs "2024-01-15T10:30:00" lexicographic order is not reliable. That can silently drop or duplicate events at the boundary. Fix: normalize to UTC (or strip tzinfo) before comparison, or parse event.timestamp to datetime and compare actual datetimes. At minimum, enforce/validate timezone format in the WebSocket param and document it.

[IMPROVEMENT OPPORTUNITIES]

  • [openhands-agent-server/openhands/agent_server/sockets.py, Lines 47–57] API contract clarity: the new param is only honored when resend_all=True, but the API silently ignores it otherwise. Consider returning a 400 when after_timestamp is set without resend_all, or explicitly document in the OpenAPI/README for the socket endpoint. Silent no-ops are a debugging tax.
  • [tests/agent_server/test_event_router_websocket.py, Lines 452–588] Coverage gap: tests only assert that the param is passed through. You’re not asserting the boundary behavior (event.timestamp == after_timestamp). Add one test case to prove inclusive filtering doesn’t cause duplicates or misses — especially given the string-compare behavior.

[STYLE NOTES]

  • [tests/agent_server/test_event_router_websocket.py, Lines 459–546] Minor: repeated local imports (datetime, cast) inside tests make the file noisy; pull them to the top-level unless there’s a cycle.

VERDICT:Worth merging once timestamp normalization is addressed. Otherwise you’re adding a correctness bug under load.

KEY INSIGHT: If timestamps aren’t normalized, the filter is gambling on string ordering — that’s not a protocol, it’s a coincidence.


WDYT on OpenHands/OpenHands#12705

The bi-directional loading plan is solid and pragmatic — REST for bulk history + WebSocket for tail is the right shape. Two gotchas to keep in mind:

  1. Boundary duplication: using >= on a timestamp boundary will duplicate if multiple events share the same timestamp or the REST page ends on that exact timestamp. Either make the websocket filter strictly greater, or include (timestamp, id) ordering and dedupe client-side.
  2. Timestamp consistency: the backend currently treats timestamps as strings; if the frontend starts mixing timezones or formats, you’ll get ordering bugs. Standardize on UTC ISO strings (or switch to numeric epoch) end-to-end.

Overall: good direction, just nail the timestamp semantics so you don’t ship a subtle “missing events” bug.

@openhands-ai

This comment was marked as duplicate.

@jpshackelford
Copy link
Copy Markdown
Contributor Author

Thanks for the detailed review! I've pushed fixes that address the timezone handling concerns. Let me explain the architecture and our decisions.

Background: How Event Timestamps Work

Event timestamps are defined in openhands-sdk (not openhands-agent-server):

# openhands-sdk/openhands/sdk/event/base.py
timestamp: str = Field(
    default_factory=lambda: datetime.now().isoformat(),
)  # consistent with V1

Key points:

  • Stored as string (ISO format), not datetime object
  • Uses naive local time (datetime.now() without tzinfo)
  • Marked "consistent with V1" - backward compatibility constraint

The agent-server's filtering code must work with this existing format.

The Problem

The original implementation used string comparison for timestamp filtering:

if event.timestamp < timestamp_gte_str:  # string comparison

This breaks when ISO strings have different timezone suffixes:

  • "2024-01-15T10:30:00" (naive)
  • "2024-01-15T10:30:00+00:00" (tz-aware)

These represent potentially different moments but don't compare correctly as strings.

Our Solution

We had two options to enable proper datetime comparison:

Option Approach Tradeoff
A Normalize filter to naive One-line change in normalize_datetime_to_server_timezone
B Add tzinfo when parsing event timestamps Changes in event_service.py for every comparison

We chose Option A because:

  1. Simpler - change is localized to the normalization function
  2. Matches the source of truth - events are stored naive, so filters should be naive
  3. Both REST and WebSocket already call normalize_datetime_to_server_timezone

Changes Made

  1. WebSocket endpoint: Added normalize_datetime_to_server_timezone() (matching REST)

  2. normalize_datetime_to_server_timezone(): Now returns naive datetime

    # Converts to server local time AND strips tzinfo
    return dt.astimezone(None).replace(tzinfo=None)
  3. event_service.py: Replaced string comparison with datetime comparison

    event_dt = datetime.fromisoformat(event.timestamp)
    if event_dt < timestamp__gte:

API Contract

Client provides Behavior
Naive datetime Assumed to be server local time, used as-is
Tz-aware datetime Converted to server local time, made naive

This handles timezones correctly - a client can pass UTC and it will be properly converted to server time before comparison.

No Breaking Changes

Verified that OpenHands/OpenHands frontend doesn't use after_timestamp yet - it only uses resend_all: true. This is a new feature, so clients will adopt the documented behavior from the start.

@openhands-ai
Copy link
Copy Markdown

openhands-ai bot commented Feb 3, 2026

Looks like there are a few issues preventing this PR from being merged!

  • GitHub Actions are failing:
    • Run tests

If you'd like me to help, just leave a comment, like

@OpenHands please fix the failing actions on PR #1880 at branch `feature/websocket-after-timestamp-filter`

Feel free to include any additional details that might help me get this PR into a better state.

You can manage your notification settings

@all-hands-bot
Copy link
Copy Markdown
Collaborator

[Automatic Post]: It has been a while since there was any activity on this PR. @jpshackelford, are you still working on it? If so, please go ahead, if not then please request review, close it, or request that someone else follow up.

@neubig neubig self-requested a review February 12, 2026 00:18
Copy link
Copy Markdown
Contributor

neubig commented Feb 12, 2026

[automated message] @neubig assigned for review according to git blame

@all-hands-bot
Copy link
Copy Markdown
Collaborator

[Automatic Post]: This PR seems to be currently waiting for review. @neubig, could you please take a look when you have a chance?

Comment thread tests/agent_server/test_event_router.py Outdated
Comment thread tests/agent_server/test_event_router_websocket.py Outdated
Comment thread tests/agent_server/test_event_router_websocket.py Outdated
Comment thread tests/agent_server/test_event_router_websocket.py
Comment thread tests/agent_server/test_event_router_websocket.py
@all-hands-bot
Copy link
Copy Markdown
Collaborator

[Automatic Post]: It has been a while since there was any activity on this PR. @jpshackelford, are you still working on it? If so, please go ahead, if not then please request review, close it, or request that someone else follow up.

2 similar comments
@all-hands-bot
Copy link
Copy Markdown
Collaborator

[Automatic Post]: It has been a while since there was any activity on this PR. @jpshackelford, are you still working on it? If so, please go ahead, if not then please request review, close it, or request that someone else follow up.

@all-hands-bot
Copy link
Copy Markdown
Collaborator

[Automatic Post]: It has been a while since there was any activity on this PR. @jpshackelford, are you still working on it? If so, please go ahead, if not then please request review, close it, or request that someone else follow up.

jpshackelford and others added 5 commits March 9, 2026 13:16
…loading

Add an `after_timestamp` query parameter to the WebSocket events endpoint
that filters events during resend_all to only return events with timestamps
>= the specified value.

This enables efficient bi-directional loading where:
- REST API fetches historical events (paginated backwards from newest)
- WebSocket handles real-time events after a specific timestamp

The underlying search_events already supports timestamp__gte filtering,
so this change simply exposes that capability through the WebSocket API.

Resolves: OpenHands/OpenHands#12705 (agent-server prerequisite)

Co-authored-by: openhands <openhands@all-hands.dev>
Address review feedback on timezone handling:

- WebSocket endpoint now normalizes tz-aware datetimes (matches REST)
- event_service.py uses datetime comparison instead of string comparison
- Updated tests to reflect the new timezone contract

Co-authored-by: openhands <openhands@all-hands.dev>
Complete the timezone handling fix by ensuring normalized datetimes are
naive (no tzinfo), matching the format of stored event timestamps.

This allows proper datetime comparison in event_service without TypeError.

Co-authored-by: openhands <openhands@all-hands.dev>
- Add _make_timestamp_filter() helper to eliminate code duplication
- Convert datetime filters to ISO strings once (O(1) vs O(n) parsing)
- Add unit tests for normalize_datetime_to_server_timezone()
- Add unit tests for _make_timestamp_filter() helper
- Add behavioral tests that verify actual filtering (not just mock calls)
- Add warning log when after_timestamp provided without resend_all
- Add Query description for after_timestamp in OpenAPI docs

Co-authored-by: openhands <openhands@all-hands.dev>
Address review comments:
- Convert TestNormalizeDatetimeToServerTimezone class to standalone functions
- Convert TestAfterTimestampFiltering class to standalone functions
- Remove TestAfterTimestampFilteringBehavioral class (duplicative with event_service tests)
- Remove TestWebSocketSubscriber, TestWebSocketDisconnectHandling, TestResendAllFunctionality classes
- Move imports to module level (datetime, cast, EventPage, Event)
- Reduce test count from 35+ to 11 tests for websocket functionality

Co-authored-by: openhands <openhands@all-hands.dev>
@jpshackelford jpshackelford force-pushed the feature/websocket-after-timestamp-filter branch from 00985c4 to 14d9f69 Compare March 9, 2026 13:19
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 9, 2026

Python API breakage checks — ✅ PASSED

Result:PASSED

Action log

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 9, 2026

REST API breakage checks (OpenAPI) — ✅ PASSED

Result:PASSED

Action log

Copy link
Copy Markdown
Contributor Author

Summary: API Breakage Checks

API breakage checks (Griffe) - PASSED

  • Checks Python SDK's programmatic API for breaking changes
  • No breaking changes detected in openhands-sdk, openhands-workspace, or openhands-tools
  • This PR doesn't introduce any Python API breaking changes

⚠️ Agent server REST API breakage checks (OpenAPI) - Failed (but non-blocking)

What happened:

Breaking REST API change detected without MINOR version bump (1.12.0 -> 1.12.0).
- the 'file' request property type/format changed from 'string'/'' to 'string'/'binary'

Key findings:

  1. This failure is NOT caused by this PR - I verified by running the same check on origin/main and it produces the identical error
  2. The file property change (from string/'' to string/'binary') is a pre-existing breaking change in main that hasn't been addressed
  3. The check has continue-on-error: true for non-release PRs (!startsWith(github.head_ref, 'rel-')) so it won't block this PR from merging

Why it doesn't block:

  • The workflow is configured to report failures via PR comment but not gate merges for feature branches
  • Only release branches (rel-*) are blocked by this check

Action needed:

  • None for this PR - it's a pre-existing issue in main
  • The repo maintainers may want to address this separately by either:
    1. Bumping the MINOR version to acknowledge the breaking change, or
    2. Investigating why the file property type/format changed

@jpshackelford
Copy link
Copy Markdown
Contributor Author

@neubig Ready for final review

Comment thread openhands-agent-server/openhands/agent_server/sockets.py Outdated
@neubig
Copy link
Copy Markdown
Contributor

neubig commented Mar 15, 2026

@jpshackelford , thanks! When you want to re-request my review, could you press the "cycle" button next to my name under reviewers? That'll put it back on my review queue, and I'm better at checking that than my github messages (which are often a bit noisy). I'll click it for myself for now.

@neubig neubig self-requested a review March 15, 2026 18:34
Implements the API change discussed in PR review:
- Add new resend_mode parameter with values: 'all', 'since', or None
- Deprecate resend_all parameter (hidden from OpenAPI schema, logs warning when used)
- resend_mode='all' sends all events (replaces resend_all=True)
- resend_mode='since' with after_timestamp sends events since timestamp
- resend_mode=None (default) subscribes without resending

Migration path:
- Old: ?resend_all=true
- New: ?resend_mode=all

- Old: ?resend_all=true&after_timestamp=...
- New: ?resend_mode=since&after_timestamp=...

Backward compatibility:
- resend_all=True still works (with deprecation warning)
- resend_all=False works as before (no resend)
- If resend_mode is set, it takes precedence over resend_all

Co-authored-by: openhands <openhands@all-hands.dev>
Copy link
Copy Markdown
Contributor Author

Implemented: resend_mode API with resend_all deprecation

Following the discussion with @enyst, I've implemented the cleaner API design:

New API

resend_mode: Literal['all', 'since'] | None
after_timestamp: datetime | None

Usage:

  • ?resend_mode=all → sends all events (replaces resend_all=true)
  • ?resend_mode=since&after_timestamp=2024-01-15T10:30:00Z → sends events since timestamp
  • No params (default) → just subscribe, no historical events

Backward Compatibility

resend_all is deprecated but still functional:

  • Hidden from OpenAPI schema (include_in_schema=False)
  • Logs a deprecation warning when used
  • ?resend_all=true continues to work (treated as resend_mode='all')

Migration Path

Phase API Surface Breaking?
Now resend_mode, after_timestamp, resend_all (hidden) No
Deprecation period Warn in logs when resend_all used No
Future major release Remove resend_all Yes
Final resend_mode, after_timestamp -

Frontend Migration (when ready)

- { resend_all: true }
+ { resend_mode: 'all' }

Commit: 2d7f74b

Co-authored-by: openhands <openhands@all-hands.dev>
Copy link
Copy Markdown
Collaborator

@all-hands-bot all-hands-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taste Rating: 🟢 Good taste

Clean implementation solving a real problem (bi-directional event loading). Key strengths:

  • Data structure handling: Correctly normalizes timezone-aware datetimes to naive server local time for comparison with stored events
  • Simplicity: No excessive nesting, clear branching logic for resend modes
  • Backward compatibility: Deprecated resend_all gracefully with warnings, new resend_mode API is cleaner
  • Testing: Comprehensive coverage testing real behavior (not mocks) - timezone normalization, all modes, backward compat

No critical or important issues found. The approach is pragmatic and maintainable.

VERDICT: ✅ Worth merging - solid engineering, well-tested, maintains backward compatibility.

@all-hands-bot
Copy link
Copy Markdown
Collaborator

[Automatic Post]: This PR seems to be currently waiting for review. @neubig @enyst, could you please take a look when you have a chance?

Copy link
Copy Markdown
Collaborator

@enyst enyst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this! Let's take it in.

@enyst enyst enabled auto-merge (squash) March 20, 2026 12:32
@enyst enyst merged commit 2f44967 into main Mar 20, 2026
26 checks passed
@enyst enyst deleted the feature/websocket-after-timestamp-filter branch March 20, 2026 12:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RFC: Efficient Bi-directional Event Loading for V1 Conversations

6 participants