Skip to content

Conversation

@chain312
Copy link
Contributor

I added a joint index to solve the problem of slow mong queries
Mongodb slow query
"op" : "query", "ns" : "st2.action_execution_scheduling_queue_item_d_b", "command" : { "find" : "action_execution_scheduling_queue_item_d_b", "filter" : { "handling" : false, "scheduled_start_timestamp" : { "$lte" : NumberLong("1668782450676922") } }, "sort" : { "scheduled_start_timestamp" : 1, "original_start_timestamp" : 1 }, "limit" : 1, "singleBatch" : true, "lsid" : { "id" : UUID("34a962b6-09d5-4745-9cf0-7ba006881362") }, "$db" : "st2", "$readPreference" : { "mode" : "primaryPreferred" } }
This query takes 251ms. Through the sql query, it can be seen that the sql model structure is filter+sort+limit, and the index is built on the column of [sort]. That is, the joint index of scheduled_start_timestamp and original_start_timestamp is established. After the index is added, the query time is less than 20ms, and the cpu decreases by 11%,11 times faster action execution
image
image

@pull-request-size pull-request-size bot added the size/XS PR that changes 0-9 lines. Quick fix/merge. label Nov 24, 2022
@chain312 chain312 changed the title Mongo Add a federated index to resolve mongo slow queries Nov 24, 2022
@chain312 chain312 mentioned this pull request Nov 24, 2022
Copy link
Member

@arm4b arm4b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for submitting a PR to improve the performance case, appreciate the efforts 👍

Just left a few comments as looks like you're including changes from another PR.

@arm4b arm4b added this to the 3.9.0 milestone Nov 24, 2022
@chain312 chain312 requested a review from arm4b November 25, 2022 05:40
@arm4b
Copy link
Member

arm4b commented Nov 29, 2022

There are some python linting issues failing in CI that need fixing.
Check https://github.com/StackStorm/st2/actions/runs/3564946625/jobs/5989513576 for more details

@chain312
Copy link
Contributor Author

There are some python linting issues failing in CI that need fixing. Check https://github.com/StackStorm/st2/actions/runs/3564946625/jobs/5989513576 for more details

@armab Sorry, I reformatted the code, but I don't know why check failed

Copy link
Member

@arm4b arm4b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @chain312 !

I updated the PR for you.

@arm4b arm4b enabled auto-merge (squash) December 16, 2022 13:02
@arm4b arm4b requested a review from a team December 16, 2022 13:06
@arm4b arm4b merged commit ebaf307 into StackStorm:master Dec 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement performance size/XS PR that changes 0-9 lines. Quick fix/merge.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants