[DONT MERGE] PoC local code review #106360

giovanni-guidini · 2026-01-15T11:25:43Z

⚠️ DON'T MERGE

This is experimental code for an internal PoC

PR Description: Local Code Review API Endpoint

more info: https://linear.app/getsentry/project/expose-pr-review-via-cli-27948ef421c5/issues

Summary

Adds a new Sentry API endpoint that enables AI-code-review on local changes via sentry-cli (before creating a pull request).
This bridges sentry-cli and Seer's ML service to provide early feedback in the development cycle.

What it does

New endpoint: POST /api/0/organizations/{org}/code-review/local-review/
Sentry polls Seer internally (up to 10 min) and returns predictions synchronously
CLI displays bug predictions to developer

Architecture

sentry-cli → Sentry API → Seer ML Service
                ↑              ↓
                └── polls ─────┘

Add Django settings for CLI bug prediction: - CLI_BUG_PREDICTION_ENABLED: Enable/disable feature - CLI_BUG_PREDICTION_TIMEOUT: 10-minute max processing time - CLI_BUG_PREDICTION_POLL_INTERVAL: 2-second Seer polling interval - CLI_BUG_PREDICTION_USER_RATE_LIMIT: 10 requests per hour per user - CLI_BUG_PREDICTION_ORG_RATE_LIMIT: 100 requests per hour per org Add organizations:cli-bug-prediction feature flag for gradual rollout

Add CliBugPredictionRequestSerializer for validating CLI review requests: - Validates repository info (owner, name, provider, base_commit_sha) - Validates diff size (max 500KB) - Validates file count (max 50 files) - Validates commit SHA format (40-char hex string) Add RepositoryInfoSerializer for repository metadata validation

Add trigger_cli_bug_prediction() to initiate analysis: - POSTs diff and repository metadata to Seer - Returns run_id for tracking Add get_cli_bug_prediction_status() to poll for results: - Checks status of analysis by run_id - Returns status, predictions, and diagnostics when complete Both methods handle timeouts, retries, and error responses

Add comprehensive tests for trigger_cli_bug_prediction(): - Success case with and without commit message - Timeout and max retry error handling - Error status codes (500, etc.) - Invalid JSON response handling - Missing run_id in response Add comprehensive tests for get_cli_bug_prediction_status(): - Pending status - Completed status with predictions and diagnostics - Timeout and max retry error handling - Error status codes (404, etc.) - Invalid JSON response handling - Missing status field in response

Add OrganizationCliBugPredictionEndpoint with full implementation: - Feature flag check (organizations:cli-bug-prediction) - Rate limiting (10/hour per user, 100/hour per org) - Request validation using CliBugPredictionRequestSerializer - Repository resolution with proper scoping - Seer integration (trigger + polling) - Comprehensive error handling and mapping - Logging and metrics throughout The endpoint synchronously polls Seer for up to 10 minutes and returns predictions or appropriate error responses with user-friendly messages

Add URL route for CLI bug prediction endpoint: - Path: /api/0/organizations/{organization_slug}/bug-prediction/cli-review/ - Maps to OrganizationCliBugPredictionEndpoint - Named route: sentry-api-0-organization-cli-bug-prediction

Add unit tests covering all endpoint functionality: - Happy path with successful prediction - Feature flag disabled - Request validation (diff size, file count, empty diff) - Commit SHA validation (format, length) - Repository not found - Seer trigger timeout and errors - Seer polling timeout - Seer error mapping (base commit, diff size, file count, clone failed, unknown) - Rate limiting (user and organization) - Optional fields handling All tests use mocks to avoid external dependencies

Add integration tests for end-to-end flow: - Single poll cycle (immediate completion) - Multiple polling cycles with state transitions - Network error recovery during polling - Empty predictions response - Multiple predictions in single response - State transitions (pending -> in_progress -> completed) - Diagnostics data included in response These tests verify the full request-response cycle including the internal polling mechanism with mocked Seer responses

* Renaming things from `cli-bug-prediction*` to `code-review-local*` * The killswtch for the feature was created but was not being used. Added to the endpint. * Resolving some mypy complaints ! Still no local tests end-to-end. There are probably bugs lurking

github-actions · 2026-01-15T11:28:31Z

🚨 Warning: This pull request contains Frontend and Backend changes!

It's discouraged to make changes to Sentry's Frontend and Backend in a single pull request. The Frontend and Backend are not atomically deployed. If the changes are interdependent of each other, they must be separated into two pull requests and be made forward or backwards compatible, such that the Backend or Frontend can be safely deployed independently.

Have questions? Please ask in the #discuss-dev-infra channel.

giovanni-guidini · 2026-01-15T11:48:53Z

src/sentry/api/urls.py

        name="sentry-api-0-organization-autofix-automation-settings",
    ),
+    re_path(
+        r"^(?P<organization_id_or_slug>[^/]+)/code-review/local-review/$",


Should this be /internal endpoint?

giovanni-guidini · 2026-01-15T13:29:05Z

src/sentry/seer/code_review/endpoints/code_review_local.py

+    publish_status = {
+        "POST": ApiPublishStatus.PRIVATE,
+    }
+    permission_classes = (OrganizationIntegrationsPermission,)


Using this permission requires the org:integrations scope. For Sentry PATs the only level of permission that includes this seems to be the admin level (org:admin)

So if we keep with this permission level only org admins can use the endpoint. Might be OK for the PoC, but certainly not in general

Maybe TODO: reduce level to org:write ?

Should there be a new permission level for this sort of thing in the future?

giovanni-guidini · 2026-01-15T14:30:55Z

@sentry review

armenzg · 2026-01-15T14:53:08Z

src/sentry/seer/code_review/endpoints/code_review_local.py

We can place this under src/sentry/seer/code_review/

armenzg · 2026-01-15T14:57:19Z

src/sentry/seer/code_review/endpoints/code_review_local.py

+        Returns 200 with predictions on success, various error codes on failure.
+        """
+        # Check if feature is globally enabled
+        if not settings.CODE_REVIEW_LOCAL_ENABLED:


Killswitches are officially supported via this:

sentry/src/sentry/runner/commands/killswitches.py

Line 11 in 2dbe578

def killswitches():

One day we will have support for it via options-automator.

src/sentry/seer/code_review/endpoints/code_review_local.py

giovanni-guidini · 2026-01-15T16:25:17Z

TODO: Adjust the serializers to match what the CLI will send us (see getsentry/sentry-cli#3084)

Renames the "code-review-local" things from "cli-bug-prediction" or "cli-pr-review"

src/sentry/seer/code_review/endpoints/code_review_local.py

+            # Include the error message from Seer if available
+            error_msg = str(e)
+            if "Seer error" in error_msg:
+                return Response({"detail": error_msg}, status=502)


In general, to fix information exposure via exceptions, avoid returning raw exception messages to end users. Log the full error server-side (which the code already does with logger.exception) and replace user-facing responses with generic, non-sensitive messages. If you need to surface specific, safe information, use a controlled mapping (e.g., error codes to friendly messages) rather than passing through arbitrary exception text.

Here, the minimal, non-functional-breaking change is to stop returning error_msg directly in the response, while still allowing clients to distinguish the “Seer error” case via status code or a generic message. We can keep the existing logging (including error: str(e)) so developers retain full detail, but change the response body to a generic description such as "Failed to start code review analysis" or a slightly more specific but still generic "Code review service returned an error". Concretely, in src/sentry/seer/code_review/endpoints/code_review_local.py, in the except ValueError as e: block, replace:

174: error_msg = str(e) 175: if "Seer error" in error_msg: 176: return Response({"detail": error_msg}, status=502) 177: return Response({"detail": "Failed to start code review analysis"}, status=502)

with logic that does not expose error_msg to the client. No new imports or helpers are required.

giovanni-guidini added 9 commits January 14, 2026 18:04

feat(cli-bug-prediction): Add URL routing for endpoint

894ee53

Add URL route for CLI bug prediction endpoint: - Path: /api/0/organizations/{organization_slug}/bug-prediction/cli-review/ - Maps to OrganizationCliBugPredictionEndpoint - Named route: sentry-api-0-organization-cli-bug-prediction

1st round of human review

f117f34

* Renaming things from `cli-bug-prediction*` to `code-review-local*` * The killswtch for the feature was created but was not being used. Added to the endpint. * Resolving some mypy complaints ! Still no local tests end-to-end. There are probably bugs lurking

github-actions bot added Scope: Backend Automatically applied to PRs that change backend components Scope: Frontend Automatically applied to PRs that change frontend components labels Jan 15, 2026

vercel bot deployed to Preview January 15, 2026 11:30 View deployment

giovanni-guidini commented Jan 15, 2026

View reviewed changes

armenzg reviewed Jan 15, 2026

View reviewed changes

src/sentry/seer/code_review/endpoints/code_review_local.py

Copy link

Member

armenzg Jan 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can place this under src/sentry/seer/code_review/

armenzg reviewed Jan 15, 2026

View reviewed changes

some fixes from testing locally the sentry --> seer connection

70f0142

giovanni-guidini force-pushed the gio/poc-cli-review branch from c548f3d to 70f0142 Compare January 15, 2026 15:23

vercel bot deployed to Preview January 15, 2026 15:25 View deployment

vercel bot deployed to Preview January 15, 2026 15:29 View deployment

github-advanced-security bot found potential problems Jan 15, 2026

View reviewed changes

src/sentry/seer/code_review/endpoints/code_review_local.py Fixed Show fixed Hide fixed

refactor(code-review): Renames and moving files around

59d1dd5

Renames the "code-review-local" things from "cli-bug-prediction" or "cli-pr-review"

giovanni-guidini force-pushed the gio/poc-cli-review branch from e6540c6 to 59d1dd5 Compare January 16, 2026 10:46

🛠️ Sync API Urls to TypeScript

1315eb3

vercel bot deployed to Preview January 16, 2026 10:51 View deployment

github-advanced-security bot found potential problems Jan 16, 2026

View reviewed changes

szokeasaurusrex mentioned this pull request Jan 16, 2026

[PoC] Code Review in Sentry CLI #106434

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

[DONT MERGE] PoC local code review #106360

[DONT MERGE] PoC local code review #106360

giovanni-guidini commented Jan 15, 2026

Uh oh!

github-actions bot commented Jan 15, 2026

Uh oh!

giovanni-guidini Jan 15, 2026

Uh oh!

giovanni-guidini Jan 15, 2026

Uh oh!

giovanni-guidini Jan 15, 2026

Uh oh!

giovanni-guidini commented Jan 15, 2026

Uh oh!

armenzg Jan 15, 2026

Uh oh!

armenzg Jan 15, 2026

Uh oh!

Uh oh!

giovanni-guidini commented Jan 15, 2026

Uh oh!

Check warning

Copilot Autofix

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

@@ -170,10 +170,13 @@
                                 "error": str(e),
                             },
                         )
-                        # Include the error message from Seer if available
+                        # Return a generic error message to avoid exposing internal details
                         error_msg = str(e)
                         if "Seer error" in error_msg:
-                            return Response({"detail": error_msg}, status=502)
+                            return Response(
+                                {"detail": "Code review service returned an error"},
+                                status=502,
+                            )
                         return Response({"detail": "Failed to start code review analysis"}, status=502)
                     except Exception as e:
                         # Catch-all for unexpected errors

Uh oh!

[DONT MERGE] PoC local code review #106360

Are you sure you want to change the base?

[DONT MERGE] PoC local code review #106360

Conversation

giovanni-guidini commented Jan 15, 2026

⚠️ DON'T MERGE

PR Description: Local Code Review API Endpoint

Summary

What it does

Architecture

Uh oh!

github-actions bot commented Jan 15, 2026

Uh oh!

giovanni-guidini Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

giovanni-guidini Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

giovanni-guidini Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

giovanni-guidini commented Jan 15, 2026

Uh oh!

armenzg Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

armenzg Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

giovanni-guidini commented Jan 15, 2026

Uh oh!

Check warning

Uh oh!

Copilot Autofix

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants