Skip to content

Split app, request metadata store and request runtime data store in Server #1952

@eabatalov

Description

@eabatalov

In case Server state machine runs into an inconsistent state and has to be wiped we need to not lose apps that were deployed since the data store backup copy was created. We also don't want to loose requests that were created since the data store backup was created. This is why we need to split the current single Server data store into:

  1. Simple data store with applications and request metadata.
  2. Complex data store used for running requests.

On Server startup it needs to check if all the not finished requests in the simple store (1) have their state the complex "request runtime" store (2). If not then re-create these requests in store (2) so they can start running.

This allows to re-create the complex store (2) without losing ability to replay the requests that were running since the last complex store (2) backup was taken. Cool think is that with this approach the requests that were deleted together with complex store (2) wiping still have their intermediate blobs in blob store so we replay the requests without doing much extra work.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions