Skip to content

Asynchronous data block processing #1469

@ml-evs

Description

@ml-evs

Placeholder issue to track this process. Current idea: add a prefers_async annotation to data blocks that need to be processed "offline". Develop an API spec for how a block should respond as its being processed, and let the UI poll and reload this. Then use APScheduler as our initial backend for offline tasks, following its inclusion in #1371.

Future work: use more robust/decentralised backend like RMQ/Arq + celery, but for now APScheduler will give us 90% of the benefit with limited extra work.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions