Race condition issue in credit check example

[azasorin-rss]

I'm not sure if this is a bug or misuse on my part, so apologies if this is better suited as an issue instead of a Q/A; this felt like a safer place to start.

In the credit check example, I run into a problem where the calls to the observer's next() race each other when multiple snapshots are emitted close together (e.g. starting an actor and then sending them an event). This results in a stale state being saved to MongoDB instead of the latest state. This doesn't happen every time, but it feels like roughly a 50% chance every time there's a state transition for the saved state to be stale. I am using a simpler state machine that has no invokes or actions and state transitions are triggered purely by events being sent to the API. Eventually, I will want to add invokes, but they're not needed yet. The example code and the video going over the example don't seem to imply that this is an invalid use case.

I have a repository with a minimal recreation of the issue: https://github.com/gepenrose/xstate-exmple-issue
The README contains steps to recreate this manually via CURL commands as well as instructions for running a script that repeat the sequence of steps 100 times to show how inconsistent the results are. To summarize the steps to recreate the issue:

1. POST to the workflows API to initialize a workflow and get the returned workflowId
2. POST to the workflows API with a "SUBMIT" event
3. GET the workflow using the workflowId and check if the saved state has correctly been saved as "PENDING"
4. POST to the workflows API with an "APPROVE" event
5. GET the workflow using the workflowId and check if the saved state has correctly been saved as "APPROVED"

I have a hacky temporary solution to get around this in my project, but I was wondering if there was any recommended way to fix this, if this is something that can be fixed in the example, or if this is something that can be fixed in XState. The only solution that comes to mind on the XState side is a mechanism to subscribe to emitted snapshots synchronously where the next() calls are awaited (like a syncSubscribe or something? 🤔 ). This sounds like it might go against the Actor model or XState's design, so I'm curious what y'all have to say 🙂.

[davidkpiano]

Thanks for the report! From initial glance, it seems like the best way to fix this is to use a task queue.

Whenever you get a new snapshot, add it to the task queue and process saving the snapshots one at a time.

[carlosbensant]

Yes, please. An example would be great.

I have the same use case and issues: Mismatching state values between transitions on actor.subscribe() when dealing with persisted states. We have the same workaround @azasorin-rss mentioned but were considering implementing a task queue.

[carlosbensant]

I want to revisit this bug to offer to do what I need to do to help replicate the issue and bring it to a resolution. This might not be the best example but is the one that I can come up with right now, it's somehow the combination of asynchronous and synchronous operations (observables): https://codesandbox.io/p/sandbox/hopeful-germain-qscpl3?file=%2Fmain.ts%3A34%2C1

[carlosbensant]

I forked the waitFor function to accomplish the following...

If you check the transition function I created, you'll find a few tricks to

1. Check if the event sent is part of the current/next event transitions (and terminate the transition early before calling actor.send() or actor.start() this last was triggering the actor.subscribe() and unnecessarily updating the entity in the database (when persisting the data).
2. Wait for the event transition to complete (including the part where we persist the snapshot/state in the database) - it was not possible with actor.subscribe().

For reference: https://codesandbox.io/p/sandbox/misty-fast-mz5958?file=%2Fmain.ts

What's the intention behind this?

For Frontend stuff, we usually have the same machine/actor for all the events and we expect xstate to internally handle the events coming through and update the state and context in no time.

But, for Backend stuff, we still have to update the state in the database for incoming events from different endpoints consumed by different users and different services that all together can send/trigger the same event type for the same entity and the machine ends up calling the same actions and invocations because when the actor was created it had the same initial state.

[azasorin-rss]

@carlosbensant I used the queue from the async package to accomplish this pretty easily. Here's a very stripped down example:

const actor = createActor(machine, { snapshot: persistedState });

const taskQueue = async.queue(async () => {
  const persistedState = actor.getPersistedSnapshot();
  
  // Db saving logic here
  // ...
}, 1);

actor.subscribe({
  next: () => {
    taskQueue.push({});
  },
});
actor.start();

[carlosbensant]

We're still integrating xstate into the project. We have plenty of functions that use/change the same data that is loaded in the state context and that persists. Eventually, we'll end up moving these functions to be part of xstate as actions and invocations.

Considering that, it wouldn't solve the issues I have for my use case. We use xstate as a framework to (1) map/check if the event applies, (2) do the transition), (3) call the different actions attached to the event or state and (4) persist the state transitions.

Functions outside xstate need to play with the latest value (snapshot), that's why we need to wait for the event transition to complete and to persist.

[Nirajkashyap]

@davidkpiano fan of your great work on xstate and introducing actor model to programming to many developers by xstate.

@azasorin-rss @davidkpiano ,
question 1: using queue to save state in db at end of event transition will give consistency of overall state ?
question 2: should queue be used in begging ? all events ( send(event) ) should go to queue and queue worker should take event one by one ?

want to understand question 1 ( using queue to save state in db at end of event transition will give consistency of overall state ? )
below is example

if using queue at end ( on subscribe next ) to save db state is working, @davidkpiano is there any plan to push, #5071 [core] Synchronizers #5071 into xstate V6.