Skip to content

Resolved all testing erors#6

Merged
Dumidu1212 merged 1 commit intomainfrom
feat/e001-s002-scoring
Oct 12, 2025
Merged

Resolved all testing erors#6
Dumidu1212 merged 1 commit intomainfrom
feat/e001-s002-scoring

Conversation

@Dumidu1212
Copy link
Owner

@Dumidu1212 Dumidu1212 commented Oct 12, 2025

Summary by CodeRabbit

  • Bug Fixes

    • Deterministic handling of overall execution timeouts, preventing hangs and ensuring clear timeout results.
    • Proper cleanup of timers and resources on success, timeout, or cancellation for improved reliability.
    • More robust fallback behavior when initial attempts fail without timing out.
  • Tests

    • Added unit test covering overall timeout behavior.
    • Updated end-to-end tests to align with the new execution contract.
    • Introduced a planner stub in tests to enable dependency injection and simplify setup.

@coderabbitai
Copy link

coderabbitai bot commented Oct 12, 2025

Walkthrough

Introduces an overall execution timeout in planner.ts with AbortSignal propagation and deterministic cleanup. Updates test executors to the new execute(tool, input, abort) signature. Adds a timeout-focused unit test and adapts existing tests for fallback behavior and planner injection via a stub.

Changes

Cohort / File(s) Summary
Planner timeout handling
src/planner/planner.ts
Adds overall timeout via setTimeout with abort reason 'overall-timeout', uses unref when available, clears timeout on success/timeout/abort, short-circuits on exec timeout, ensures cleanup after fallback loop.
E2E: executor signature update
tests/e2e/plan.e2e.test.ts
Updates StubExec.execute to execute(tool, input, overallAbort); imports JsonRecord; behavior unchanged aside from signature.
E2E: planner injection
tests/e2e/tools.e2e.test.ts
Adds PlannerStub implementing IPlanner; wires planner into app via DI during tests; returns fixed traceId and empty candidates.
Unit: fallback behavior/signature
tests/unit/planner.fallback.test.ts
Changes FailingExec.execute signature to (tool, input, abort); first call now non-timeout failure to trigger fallback; subsequent success remains.
Unit: new timeout test
tests/unit/planner.timeout.test.ts
Adds HangingExec awaiting overall AbortSignal; verifies planner surfaces 'timeout' result when timeout_ms is small; minimal registry/planner setup for test.

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant Planner
  participant Executor
  participant Timer as OverallTimeout(setTimeout)

  Client->>Planner: plan(ctx with timeout_ms)
  activate Planner
  Planner->>Timer: start(timeout_ms, abort 'overall-timeout')
  note over Planner,Timer: Planner stores handle and unrefs if supported

  Planner->>Executor: execute(tool, input, overallAbort)
  alt Executor completes before timeout
    Executor-->>Planner: ExecutionResult(status: success|failure)
    Planner->>Timer: clear()
    Planner-->>Client: PlanResult with exec result
  else Timeout fires
    Timer-->>Planner: abort signal ('overall-timeout')
    Executor-->>Planner: ExecutionResult(status: timeout)
    Planner->>Timer: clear()
    Planner-->>Client: PlanResult with status 'timeout'
  end

  opt Fallback iteration
    Planner->>Executor: execute(next candidate,...)
    alt overallAbort already signaled
      Planner->>Timer: clear()
      Planner-->>Client: Failure (ALL_CANDIDATES_FAILED or timeout)
    end
  end
  deactivate Planner
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Poem

A whisk of time, a tick, a tock—
I set a clock beside the rock. ⏱️
If tools should dawdle, I’ll gently nudge,
Abort with grace—no grudge, no fudge.
Clear the timers, hop along—
Plans now march both swift and strong. 🐇✨

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Title Check ⚠️ Warning The pull request title “Resolved all testing erors” is vague and includes a spelling mistake, failing to describe the main changes. The PR actually introduces an overall timeout mechanism in the planner and updates executor method signatures across tests, which should be reflected in a specific, concise title. Please update the title to clearly summarize the primary changes, for example “Add overall timeout handling to planner and update executor signatures in tests.”
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed No functions found in the changes. Docstring coverage check skipped.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feat/e001-s002-scoring

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e15296f and f234ce2.

📒 Files selected for processing (5)
  • src/planner/planner.ts (2 hunks)
  • tests/e2e/plan.e2e.test.ts (2 hunks)
  • tests/e2e/tools.e2e.test.ts (2 hunks)
  • tests/unit/planner.fallback.test.ts (1 hunks)
  • tests/unit/planner.timeout.test.ts (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (5)
tests/e2e/tools.e2e.test.ts (2)
src/planner/contracts.ts (3)
  • IPlanner (63-65)
  • PlanContext (10-23)
  • PlanResult (40-46)
src/app.ts (1)
  • buildApp (14-30)
tests/unit/planner.fallback.test.ts (1)
src/planner/contracts.ts (1)
  • ExecutionResult (32-37)
src/planner/planner.ts (1)
src/metrics/metrics.ts (2)
  • plannerSelectionTotal (21-25)
  • plannerFallbacksTotal (28-32)
tests/e2e/plan.e2e.test.ts (1)
src/planner/contracts.ts (2)
  • JsonRecord (7-7)
  • ExecutionResult (32-37)
tests/unit/planner.timeout.test.ts (5)
src/planner/contracts.ts (3)
  • IToolExecutor (54-60)
  • JsonRecord (7-7)
  • ExecutionResult (32-37)
src/registry/service.ts (1)
  • IRegistryService (4-4)
src/planner/planner.ts (1)
  • Planner (26-132)
src/planner/scoring.simple.ts (1)
  • SimpleScorer (40-54)
src/tracing/traceStore.ts (1)
  • TraceStore (16-34)
🔇 Additional comments (4)
tests/unit/planner.timeout.test.ts (1)

15-34: Hanging executor behaves perfectly under overall abort

Async wait on the planner’s AbortSignal and returning a timeout result keeps the test deterministic and mirrors the real executor contract nicely.

tests/e2e/tools.e2e.test.ts (1)

7-31: Solid planner stub wiring

Injecting a minimal IPlanner stub keeps the tools routes focused while satisfying the new DI requirement—clean approach.

tests/e2e/plan.e2e.test.ts (1)

24-34: Updated stub matches executor contract

The three-argument execute signature keeps the e2e flow aligned with the planner’s expectations—looks good.

tests/unit/planner.fallback.test.ts (1)

22-48: Fallback path test stays in step with executor API

Adjusting the stub to fail once (non-timeout) and then succeed keeps the fallback scenario realistic and aligned with the new signature. Nicely done.

Comment on lines +120 to 128
if (overallController.signal.aborted) {
if (overallTimeout) clearTimeout(overallTimeout);
break;
}
}

// No candidate succeeded
if (overallTimeout) clearTimeout(overallTimeout);
result.execution = { status: 'failure', error: 'ALL_CANDIDATES_FAILED' };
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Surface overall timeout instead of “ALL_CANDIDATES_FAILED”

When the overall abort fires during fallback (e.g., deadline expires right after a failure), we break out of the loop, but the final result remains { status: 'failure', error: 'ALL_CANDIDATES_FAILED' }. That masks the timeout as a generic failure. We should return { status: 'timeout', error: 'overall-timeout' } (and record a matching trace event) whenever overallController.signal.aborted is true so callers receive the correct cause.

🤖 Prompt for AI Agents
In src/planner/planner.ts around lines 120 to 128, the code breaks out of the
candidate loop when overallController.signal.aborted but still sets
result.execution = { status: 'failure', error: 'ALL_CANDIDATES_FAILED' }; update
that branch so when overallController.signal.aborted is true you clear the
overallTimeout (if set), set result.execution = { status: 'timeout', error:
'overall-timeout' }, and record a matching trace event for the overall timeout
before breaking/returning, ensuring callers see the actual timeout cause instead
of ALL_CANDIDATES_FAILED.

@Dumidu1212 Dumidu1212 merged commit 51f0010 into main Oct 12, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant