Skip to content

Parallelize async insight report jobs #154

@pnadolny13

Description

@pnadolny13

The tap gets reports using the async job workflow but we only kick off one job right now and wait for it to complete. We can kick off many of them at once and then poll for completion, even keeping them in order so the stream is sorted if we wanted. Since several other jobs will be processing while we poll for the first one, I'd expect all of them to be ready to extract once the first one is done. This would allow us to increase throughput significantly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions