Skip to content

Conversation

@rgarcia
Copy link
Contributor

@rgarcia rgarcia commented Oct 16, 2025

Note

Adds native input APIs (press key, scroll, drag mouse), serializes input ops, updates OpenAPI/client/server stubs, installs xdotool, and refines tests/deps.

  • API/Backend:
    • New Endpoints: POST /computer/press_key, POST /computer/scroll, POST /computer/drag_mouse with bounds/validation and xdotool-backed behavior.
    • Input Serialization: Add inputMu to serialize mouse/keyboard/screenshot actions.
    • Click Enum: Adjust ClickMouseRequestButton* enum names.
  • OpenAPI/Codegen:
    • Add schemas for PressKeyRequest, ScrollRequest, DragMouseRequest and wire routes into generated client/server.
    • Embedded swagger updated accordingly.
  • Docker:
    • Install xdotool in chromium-headless image.
  • Tests:
    • Add input smoke tests and convert assertions to require in e2e; minor test cleanups.
  • Dependencies:
    • Add github.com/samber/lo; bump stretchr/testify; update x/sync, add x/text.

Written by Cursor Bugbot for commit b12249d. This will update automatically on new commits. Configure here.

@rgarcia rgarcia requested a review from hiroTamada October 16, 2025 13:11
@mesa-dot-dev
Copy link

mesa-dot-dev bot commented Oct 16, 2025

Mesa Description

Note

Adds new input endpoints (press_key, scroll, drag_mouse), serializes input actions, updates OpenAPI/client, installs xdotool, and expands/refactors e2e tests with dependency bumps.

  • API/Backend:
    • New endpoints: POST /computer/press_key, POST /computer/scroll, POST /computer/drag_mouse with validation and xdotool integration.
    • Input serialization: add inputMu to serialize mouse/keyboard/screenshot actions; apply across MoveMouse, ClickMouse, TakeScreenshot, TypeText.
    • OpenAPI + Client: extend openapi.yaml; regenerate lib/oapi with new models/requests/responses and adjusted click button enums.
  • Infrastructure:
    • Dockerfile (headless): install xdotool.
  • Tests:
    • Add TestInputEndpointsSmoke; modernize e2e tests (use require, simplify flows) and minor cleanups.
  • Dependencies:
    • Bump testify, golang.org/x/sync, add samber/lo, golang.org/x/text.

Written by Cursor Bugbot for commit 2b85972. This will update automatically on new commits. Configure here.

Description generated by Mesa. Update settings

Copy link

@mesa-dot-dev mesa-dot-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Performed full review of 0260b79...cce4f45

Analysis

  1. The PR adds a dependency on xdotool for input automation, which may create platform compatibility issues or increase the Docker image size.

  2. While smoke tests are included, there's no mention of comprehensive unit tests or integration tests for these critical input simulation functions, potentially allowing edge cases to slip through.

  3. The PR refactors E2E tests by removing "flaky" tests, which might inadvertently reduce test coverage of important scenarios.

  4. The implementation of sophisticated features like configurable paths and delays for DragMouse could introduce performance overhead or timing-related issues in different environments.

Tip

⚡ Quick Actions

This review was generated by Mesa.

Actions:

Slash Commands:

  • /review - Request a full code review
  • /review latest - Review only changes since the last review
  • /describe - Generate PR description. This will update the PR body or issue comment depending on your configuration
  • /help - Get help with Mesa commands and configuration options

7 files reviewed | 0 comments | Review on Mesa | Edit Reviewer Settings

cursor[bot]

This comment was marked as outdated.

}
}

func TestChromiumHeadfulUserDataSaving(t *testing.T) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these flaky tests are not needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah getting rid of them since it tests a fairly complicated way of restoring user data that we don't actually use

Copy link
Contributor

@hiroTamada hiroTamada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

One questions I have is whether these mouse related endpoints should be protected by mutex?

Can we have scroll and dragging at the same time for example?

@rgarcia
Copy link
Contributor Author

rgarcia commented Oct 16, 2025

ahh good catch. Added mutex: 2b85972

@rgarcia rgarcia merged commit 4ef116e into main Oct 16, 2025
4 checks passed
@rgarcia rgarcia deleted the raf/kernel-149-os-level-screenshots branch October 16, 2025 14:55
_ = dumpContainerDiagnostics(ctx, name)
t.Fatalf("api not ready: %v", err)
}
require.NoError(t, waitHTTPOrExit(ctx, apiBaseURL+"/spec.yaml", exitCh), "api not ready: %v", err)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Misleading Error Messages in require.NoError Calls

Several require.NoError calls, particularly those checking waitHTTPOrExit, use an err variable from an earlier scope in their failure message. This can lead to misleading error output, as the message might report a previous error (e.g., docker lookup) instead of the actual waitHTTPOrExit failure. This occurs on lines 114, 214, 308, 359, and 404.

Fix in Cursor Fix in Web

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants