Search then Get tests are intermittently failing — possible server race condition

[jwulf]

The Tasklist v1 API tests have started failing intermittently in CI.

I am disabling them to reduce friction, and have raised a few issues with the Tasklist team for further investigation.

These tests should be re-enabled after these have been addressed.

[jwulf]

The test code queries for tasks, then completes a task from the search result.

Here is the code:

const tasks = await PollingOperation({
		operation: () =>
			tasklist.searchTasks({
				state: 'CREATED',
			}),
		predicate: (tasks) => tasks.length > 0,
		interval: 100,
		timeout: 10000,
	})
	const zbc = new ZeebeRestClient()
	const res = await zbc.completeUserTask({
		userTaskKey: tasks[0].id,
	})

Sometimes this fails with a 404 for the task:

HTTPError: Response code 404 (Not Found) 
(POST http://localhost:8080/v1/user-tasks/2251799813685497/completion).
{
  "type":"about:blank","title":"NOT_FOUND",
  "status":404,
  "detail":"Command 'COMPLETE' rejected with code 'NOT_FOUND': Expected to complete user task with key '2251799813685497', but no such user task was found",
  "instance":"/v1/user-tasks/2251799813685497/completion"
}

This should either always fail, or never fail.

It indicates that the search query can return a task that the completeUserTask endpoint doesn't know about.

I may be able to make this test reliably pass by making the update operation a polling operation, but this seems to indicate that there is a server race condition between the query and update endpoints — possibly the index that they use.

[jwulf]

Related: #560

[jwulf]

Here is an example of the same failure on the Decision Endpoints:

FAIL src/__tests__/8.8/getDecisionInstance.spec.ts
  ● CamundaRestClient.getDecisionInstance › should fetch a decision instance and return all documented fields

    HTTPError: Response code 404 (Not Found) (GET ***/v2/decision-instances/6755399445909013-1). {"type":"about:blank","title":"NOT_FOUND","status":404,"detail":"Decision instance with key 6755399445909013-1 not found","instance":"/e4ce677f-e458-421b-ab33-3f0b02a3ebba/v2/decision-instances/6755399445909013-1"}. Enhanced stack trace available as error.source.

This is the code:

const res = await PollingOperation({
	operation: () =>
		client.searchDecisionInstances({ filter: { decisionInstanceKey } }),
	interval: 100,
	timeout: 6000,
})

// Now use the created decision instance key
const result = await client.getDecisionInstance(
	res.items[0].decisionInstanceId
)