docs: improve queueName docs to avoid high cardinality#558
Merged
benjie merged 2 commits intographile:mainfrom Aug 17, 2025
Merged
docs: improve queueName docs to avoid high cardinality#558benjie merged 2 commits intographile:mainfrom
benjie merged 2 commits intographile:mainfrom
Conversation
blimmer
commented
Aug 13, 2025
| * queue to run serially). (Default: null) | ||
| * queue to run serially). Avoid using high cardinality values (e.g., random | ||
| * strings, UUIDs, timestamps) as this degrades performance and requires | ||
| * periodic database cleanup. (Default: null) |
Contributor
Author
There was a problem hiding this comment.
Feedback is welcome on the phrasing here - I don't feel strongly!
benjie
previously approved these changes
Aug 15, 2025
website/docs/library/add-job.md
Outdated
Comment on lines
+34
to
+45
|
|
||
| :::warning | ||
|
|
||
| Avoid using high cardinality values (e.g., random strings, UUIDs, | ||
| timestamps) for queue names as this will create many dead queues that | ||
| degrade performance and require | ||
| [periodic database cleanup](../admin-functions.md#gc_job_queues). If you | ||
| find yourself needing to run the `GC_JOB_QUEUES` cleanup task regularly, | ||
| you're likely using queue names incorrectly. | ||
|
|
||
| ::: | ||
|
|
Member
There was a problem hiding this comment.
@jemgillam Please confirm this renders okay. If not, move it to below the list of options.
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Description
We recently ran into an issue where our serverless database had to dramatically scale up capacity. Upon reviewing those queries, we realized that the issue was caused by Graphile Worker query:
Specifically the subquery against the
_private_job_queuestable.We were creating lots of job queues using high-cardinality values (e.g., database row IDs), which caused many thousands of dead queues to build up in this table.
Because the graphql worker polling job necessarily does a table scan on
_private_job_queues, this caused a big performance issue for us, forcing the DB to scale up.When I reviewed the documentation, I found a warning about not using randomly-generated values in
queueNamesworker/website/docs/admin-functions.md
Lines 127 to 140 in ae40946
This was helpful. However, I think it makes sense to also warn about this on the
queueNameparameter itself, so people know before there's a problem. This PR adds those warnings.Performance impact
N/A - docs only
Security impact
N/A - docs only
Checklist
yarn lint:fixpasses.yarn testpasses.RELEASE_NOTES.mdfile (if one exists).