Skip to content

Fix duplicate task generation#5431

Merged
ndr-ds merged 1 commit intotestnet_conwayfrom
ndr-ds/fix-duplicate-task-generation
Feb 12, 2026
Merged

Fix duplicate task generation#5431
ndr-ds merged 1 commit intotestnet_conwayfrom
ndr-ds/fix-duplicate-task-generation

Conversation

@ndr-ds
Copy link
Copy Markdown
Contributor

@ndr-ds ndr-ds commented Feb 12, 2026

Motivation

The task processor can call process_actions multiple times while a previous batch's tasks are still in-flight.
When there's a backlog (schedule.start far in the past), the deadline timer fires
immediately because the next event is also in the past. The last_requested_callbacks guard passes, and
next_actions reads stale on-chain state (schedule.start hasn't advanced yet), generating overlapping timestamps.
When both batches are eventually submitted as blocks, batch 2's stale timestamps don't
match the now-advanced schedule.start, causing:

contract.rs:115: assert_eq!(timed_data.timestamp, next_timestamp, "unexpected timestamp in posted data")

Proposal

Introduce an in_flight_apps: BTreeSet<ApplicationId> guard in the task processor. When a batch of tasks is spawned
for an application, the app is marked as in-flight and subsequent process_actions calls skip it. A new
TaskMessage enum replaces the raw (ApplicationId, TaskOutcome) channel, adding a BatchComplete variant that the
spawned task sends after all outcomes. On BatchComplete, the guard is cleared and process_actions is
re-triggered for that app so it reads freshly-updated on-chain state.

Test Plan

  • CI
  • I had a stuck worker with a big backlog. After rebuilding pm-app with this fix and restarting the worker with the new package version, I see no more panics, and it seems that the worker is actually making progress through the backlog:

Screenshot 2026-02-12 at 13.44.18.png

@ndr-ds ndr-ds marked this pull request as ready for review February 12, 2026 19:21
@ndr-ds ndr-ds merged commit e56f3fe into testnet_conway Feb 12, 2026
33 checks passed
@ndr-ds ndr-ds deleted the ndr-ds/fix-duplicate-task-generation branch February 12, 2026 19:21
}
// Signal that this batch is done so the main loop can process
// the next batch for this application.
let _ = sender.send(TaskMessage::BatchComplete { application_id });
Copy link
Copy Markdown
Contributor

@ma2bd ma2bd Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we remove let _ = ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, my bad #5436

@ndr-ds ndr-ds mentioned this pull request Feb 13, 2026
ndr-ds added a commit that referenced this pull request Feb 13, 2026
## Motivation

Missed a `let _ = ` on
#5431

## Proposal

Remove it

## Test Plan

CI
github-merge-queue bot pushed a commit that referenced this pull request Feb 17, 2026
## Motivation

Frontport of #5431 from `testnet_conway` to `main`.

## Proposal

Cherry-pick of #5431 with conflict resolution (adapted `cursors` field
name used on `main`).

## Test Plan

CI.
ndr-ds added a commit that referenced this pull request Feb 17, 2026
## Motivation

Missed a `let _ = ` on
#5431

## Proposal

Remove it

## Test Plan

CI
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants