Skip to content

otelriver: Track inserts skipped as duplicate due to UniqueOpts#58

Merged
brandur merged 4 commits into
riverqueue:masterfrom
echatman:echatman/skipped-as-duplicate-tracking
Jun 1, 2026
Merged

otelriver: Track inserts skipped as duplicate due to UniqueOpts#58
brandur merged 4 commits into
riverqueue:masterfrom
echatman:echatman/skipped-as-duplicate-tracking

Conversation

@echatman
Copy link
Copy Markdown
Contributor

@echatman echatman commented Jun 1, 2026

What

Adds a skipped_as_duplicate boolean attribute to river.insert_count data points and a duplicate_skipped_count span attribute on the river.insert_many span, so callers can tell how many of the submitted jobs were dropped by UniqueOpts vs. actually enqueued.

After this change, each InsertMany call emits two river.insert_count data points:

  • river.insert_count{skipped_as_duplicate:false} — jobs that were actually enqueued
  • river.insert_count{skipped_as_duplicate:true} — jobs dropped because UniqueOpts matched an in-flight or pending duplicate

The sum across both data points still equals len(manyParams), so dashboards or monitors that sum the metric without filtering see no change. The span also carries the count directly:

span.SetAttributes(attribute.Int64("duplicate_skipped_count", skipped))

Why

Today every job submitted to InsertMany is counted in river.insert_count regardless of whether River accepted it or dropped it via UniqueOpts. That makes the insert metric unable to distinguish "we accepted 100 new jobs" from "we accepted 0 new jobs because all 100 were dedup'd against in-flight work" — both look identical on a chart.

I've been carrying a custom River middleware that emits a separate …insert_duplicate_skipped_count counter just to recover this information. Pushing this into otelriver removes the need for that workaround and gives every otelriver user duplicate-rate visibility out of the box.

Tests

Existing InsertMany, InsertManyError, and InsertManyPanic tests updated to assert against the new partitioned data points. Two new tests cover the new behaviour:

  • InsertManyAllDuplicates — 2 submitted, both flagged UniqueSkippedAsDuplicate: true
  • InsertManyMixedDuplicates — 5 submitted, 2 dedup'd / 3 enqueued; also asserts the grand-total invariant

A small test helper requireSumByAttrs was added (alongside the existing requireSum) for asserting against a specific subset of data points by attributes, since the partitioned metric now emits multiple data points per measurement.

make test and make lint both pass; tests also pass under -race.

Notes for reviewers


Made with Cursor on behalf of @echatman.

@echatman echatman marked this pull request as ready for review June 1, 2026 20:24
@echatman echatman changed the title otelriver: partition river.insert_count by skipped_as_duplicate otelriver: Track inserts skipped as duplicate due to UniqueOpts Jun 1, 2026
Comment thread otelriver/middleware.go Outdated
}

span.SetAttributes(attrs...) // set after finalizing status
span.SetAttributes(attribute.Int64("duplicate_skipped_count", skipped))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you modify this and the two instances below to prefix with unique_* like the row field name? I kind of like that unique_* is on there because for someone not familiar with the property already it's strongly suggestive that it's related to unique jobs.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. I was trying to keep it short but already failed at that, so may as well go with fully verbose for clarity. Fix incoming.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. I also added a CHANGELOG entry; let me know if you prefer to leave small changes like this out of the changelog and I'll remove it

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thx!

Nah, every change should ideally have a changelog entry. I didn't mention it yet because I knew it'd conflict with the addition in #59 ;)

With #59 in now, can you rebase and re-push?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

echatman and others added 4 commits June 1, 2026 14:48
Today every job submitted to InsertMany is counted in river.insert_count
even if River dropped it via UniqueOpts. That makes the insert metric
unable to distinguish "we accepted 100 new jobs" from "we accepted 0 new
jobs because all 100 were dedup'd against in-flight work" - both look
identical on a chart.

This adds a skipped_as_duplicate boolean attribute to river.insert_count
data points and a duplicate_skipped_count span attribute on the
river.insert_many span:

  - river.insert_count{skipped_as_duplicate:false} - jobs actually enqueued
  - river.insert_count{skipped_as_duplicate:true}  - jobs dropped by UniqueOpts
  - span.duplicate_skipped_count                   - how many of the batch were skipped

The sum across both insert_count data points still equals len(manyParams),
so existing dashboards that sum the metric without filtering see no
change. New dashboards / monitors can now compute a duplicate rate
directly from the otel metric rather than needing a separate counter in
each service that uses UniqueOpts.

Tests cover all-duplicates, no-duplicates, and mixed-batch shapes.

Co-authored-by: Cursor <cursoragent@cursor.com>
Match the existing requireSum helper's //nolint:unparam pattern;
the name parameter is constant for now but the helper is intended
to grow more callers as more partitioned-by-attribute metrics show up.

Co-authored-by: Cursor <cursoragent@cursor.com>
@echatman echatman force-pushed the echatman/skipped-as-duplicate-tracking branch from 34fe2ad to a0631e4 Compare June 1, 2026 21:49
@brandur brandur merged commit e41c037 into riverqueue:master Jun 1, 2026
2 checks passed
@brandur brandur mentioned this pull request Jun 1, 2026
brandur added a commit that referenced this pull request Jun 1, 2026
Prepare release v0.9.0, containing #58 and #59 for `otelriver`.

[skip ci]
brandur added a commit that referenced this pull request Jun 1, 2026
Prepare release v0.9.0, containing #58 and #59 for `otelriver`.

[skip ci]
@brandur
Copy link
Copy Markdown
Contributor

brandur commented Jun 1, 2026

Published v0.9.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants