Skip to content

Add sequential task queue for processing algorithms#64262

Open
lanckmann wants to merge 12 commits intoqgis:masterfrom
lanckmann:feature-processing-task-queue-55564
Open

Add sequential task queue for processing algorithms#64262
lanckmann wants to merge 12 commits intoqgis:masterfrom
lanckmann:feature-processing-task-queue-55564

Conversation

@lanckmann
Copy link
Contributor

@lanckmann lanckmann commented Dec 11, 2025

Add sequential task queue for processing algorithms queue multiple
different algorithms for sequential execution to avoid system overload.
Implements #55564

@github-actions github-actions bot added this to the 4.0.0 milestone Dec 11, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Dec 11, 2025 •
edited
Loading

🪟 Windows Qt6 builds

Download Windows Qt6 builds of this PR for testing.
(Built from commit 08a6d0b)

🍎 MacOS Qt6 builds

Download MacOS Qt6 builds of this PR for testing.
This app is not notarized, run sudo xattr -d com.apple.quarantine /Applications/QGIS*.app to avoid the warning
(Built from commit 08a6d0b)

@nyalldawson
Copy link
Collaborator

nyalldawson commented Dec 11, 2025

@lanckmann the goal for processing is to minimise the python code. Can you move all this logic to the C++ classes?

@lanckmann
Copy link
Contributor Author

lanckmann commented Dec 11, 2025

@nyalldawson Thanks, that was actually my initial idea as well. I hesitated because I wasn’t sure how much we’re allowed to change in the QGIS core and create new C++ classes. But if that’s fine, then yes, I’ll move the logic into the C++ classes and minimise the Python part.

@nyalldawson
Copy link
Collaborator

nyalldawson commented Dec 12, 2025

@lanckmann yes please -- the python based code proved to be a maintenance nightmare.

Note that you'll also need to add appropriate unit tests covering this work too

@lanckmann lanckmann force-pushed the feature-processing-task-queue-55564 branch 2 times, most recently from d576741 to 1c497ad Compare December 17, 2025 09:08
@lanckmann lanckmann force-pushed the feature-processing-task-queue-55564 branch from 1c497ad to 5386265 Compare December 17, 2025 19:48
@github-actions
Copy link
Contributor

github-actions bot commented Jan 3, 2026

The QGIS project highly values your contribution and would love to see this work merged! Unfortunately this PR has not had any activity in the last 14 days and is being automatically marked as "stale". If you think this pull request should be merged, please check

  • that all unit tests are passing

  • that all comments by reviewers have been addressed

  • that there is enough information for reviewers, in particular

    • link to any issues which this pull request fixes

    • add a description of workflows which this pull request fixes

    • add screenshots if applicable

  • that you have written unit tests where possible
    In case you should have any uncertainty, please leave a comment and we will be happy to help you proceed with this pull request.
    If there is no further activity on this pull request, it will be closed in a week.

@github-actions github-actions bot added the stale Uh oh! Seems this work is abandoned, and the PR is about to close. label Jan 3, 2026
@lanckmann
Copy link
Contributor Author

lanckmann commented Jan 3, 2026

!!

@github-actions github-actions bot removed the stale Uh oh! Seems this work is abandoned, and the PR is about to close. label Jan 3, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Jan 18, 2026

The QGIS project highly values your contribution and would love to see this work merged! Unfortunately this PR has not had any activity in the last 14 days and is being automatically marked as "stale". If you think this pull request should be merged, please check

  • that all unit tests are passing

  • that all comments by reviewers have been addressed

  • that there is enough information for reviewers, in particular

    • link to any issues which this pull request fixes

    • add a description of workflows which this pull request fixes

    • add screenshots if applicable

  • that you have written unit tests where possible
    In case you should have any uncertainty, please leave a comment and we will be happy to help you proceed with this pull request.
    If there is no further activity on this pull request, it will be closed in a week.

@github-actions github-actions bot added the stale Uh oh! Seems this work is abandoned, and the PR is about to close. label Jan 18, 2026
@lanckmann
Copy link
Contributor Author

lanckmann commented Jan 18, 2026

No stale!

@github-actions github-actions bot removed the stale Uh oh! Seems this work is abandoned, and the PR is about to close. label Jan 18, 2026
@nyalldawson nyalldawson added the Freeze Exempt Feature Freeze exemption granted label Jan 27, 2026
ValentinBuira
ValentinBuira suggested changes Feb 6, 2026
View reviewed changes
Copy link
Contributor

@ValentinBuira ValentinBuira left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @lanckmann

Only commenting design wise, I didn't had a look at the code

Image

I think it is an interesting feature to explore, maybe you should go through a QEP process. This way you can gather feedbacks, and avoid common pitfalls. You would have a higher likelyhood to get this merged

python/plugins/processing/gui/ProcessingToolbox.py
QCoreApplication.translate("ProcessingToolbox", "Add to Queue…"),
popupmenu,
)
addToQueueAction.triggered.connect(self.addAlgorithmToQueue)
Copy link
Contributor

@ValentinBuira ValentinBuira Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This buttons act the same as the regular "Execute..."

python/plugins/processing/gui/ProcessingToolbox.py

# connect to existing providers
self.queueButton = QToolButton()
self.queueButton.setIcon(QgsApplication.getThemeIcon("/mTaskRunning.svg"))
Copy link
Contributor

@ValentinBuira ValentinBuira Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The running icon is not the most suitable, maybe something inspired by the queue data structure would be better ?

src/gui/processing/qgsprocessingalgorithmdialogbase.cpp
mButtonBox->addButton( mAdvancedButton, QDialogButtonBox::ResetRole );

mButtonAddToQueue = new QPushButton( tr( "Add to Queue…" ) );
mButtonBox->addButton( mButtonAddToQueue, QDialogButtonBox::ResetRole );
Copy link
Contributor

@ValentinBuira ValentinBuira Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why but there is two "Add to Queue" Buttons

Image

@github-actions github-actions bot added Processing Relating to QGIS Processing framework or individual Processing algorithms GUI/UX Related to QGIS application GUI or User Experience labels Feb 8, 2026
@nyalldawson nyalldawson added the Frozen Feature freeze - Do not merge! label Feb 20, 2026
@nyalldawson nyalldawson removed the Freeze Exempt Feature Freeze exemption granted label Feb 20, 2026
@nyalldawson nyalldawson removed the Frozen Feature freeze - Do not merge! label Mar 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@ValentinBuira ValentinBuira ValentinBuira requested changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

GUI/UX Related to QGIS application GUI or User Experience Processing Relating to QGIS Processing framework or individual Processing algorithms

Projects

None yet

Milestone

4.0.0

Development

Successfully merging this pull request may close these issues.

3 participants

@lanckmann @nyalldawson @ValentinBuira