(integ-runner): succeeds with errors

[moltar]

Describe the bug

Integration test run passes, yet the output shows errors.

Regression Issue

• Select this option if this issue appears to be a regression.

Last Known Working CDK Version

N/A

Expected Behavior

The errors to be reported thru normal error reporting channels and to not pass.

Current Behavior

Tests "pass".

Reproduction Steps

Running

integ-runner --config integ-runner.config.json --disable-update-workflow --update-on-failed --no-clean -vv --force test/dynamodb-table-session-storage.itest.ts

integ-runner.config.json

{
  "app": "tsx {filePath}",
  "language": "typescript",
  "parallel-regions": [
    "us-east-1"
  ],
  "directory": "test",
  "test-regex": [
    "\\.itest\\.ts$"
  ]
}

Possible Solution

N/A

Additional Information/Context

The tail of the output:

AwsLambdaFunctionUrlAdapterIntegTestDefaultTestDeployAssert216E26C9 | 29/32 | 1:53:43 p.m. | DELETE_COMPLETE      | AWS::CloudFormation::CustomResource | HttpApiCall4f0624126f0a6f58a6c1339a93bee774 
AwsLambdaFunctionUrlAdapterIntegTestDefaultTestDeployAssert216E26C9 | 30/32 | 1:53:43 p.m. | DELETE_COMPLETE      | AWS::CloudFormation::CustomResource | HttpApiCall8ea98cd5e7f2cd26d9787e8ce33efeae 
AwsLambdaFunctionUrlAdapterIntegTestDefaultTestDeployAssert216E26C9 | 31/32 | 1:53:43 p.m. | DELETE_COMPLETE      | AWS::CloudFormation::CustomResource | HttpApiCall2b4c670646c2c5f8f2c8f1bd5d21cfb5 
AwsLambdaFunctionUrlAdapterIntegTestDefaultTestDeployAssert216E26C9 | 32/32 | 1:53:43 p.m. | UPDATE_COMPLETE      | AWS::CloudFormation::Stack    | AwsLambdaFunctionUrlAdapterIntegTestDefaultTestDeployAssert216E26C9 

 ✅  AwsLambdaFunctionUrlAdapterIntegTest/DefaultTest/DeployAssert
✨  Deployment time: 39.92s
Outputs:
AwsLambdaFunctionUrlAdapterIntegTestDefaultTestDeployAssert216E26C9.AssertionResultsHttpApiCalla11420362f06765e8fa50bcbc92b57ae = {"status":"fail","message":"{\n  \"body\": {\n    \"cookie\": \"__session=x%3D.y\",\n!!   Expected 2 but received 1\n    \"counter\": 1\n  },\n  \"headers\": { ... },\n  \"ok\": true,\n  \"status\": 200,\n  \"statusText\": \"OK\"\n}"}
AwsLambdaFunctionUrlAdapterIntegTestDefaultTestDeployAssert216E26C9.AssertionResultsHttpApiCallbef3699880c06a026362e687054d8645 = {"status":"fail","message":"!! Expected type object but received undefined\nundefined"}
Stack ARN:
arn:aws:cloudformation:us-east-1:245748775536:stack/AwsLambdaFunctionUrlAdapterIntegTestDefaultTestDeployAssert216E26C9/634ded60-e25e-11f0-9284-12b1f8588f4d
✨  Total time: 39.92s

  SUCCESS    dynamodb-table-session-storage.itest-AwsLambdaFunctionUrlAdapterIntegTest/DefaultTest 87.719s

Test Results: 

Tests:    1 passed, 1 total
Not cleaning up stacks since "--no-clean" was used
   --- Integration test metrics ---
Profile undefined + Region us-east-1 total time: 87.719
  .../test/dynamodb-table-session-storage.itest.ts: 87.719

See the Outputs part:

AwsLambdaFunctionUrlAdapterIntegTestDefaultTestDeployAssert216E26C9.AssertionResultsHttpApiCalla11420362f06765e8fa50bcbc92b57ae = {"status":"fail","message":"{\n "body": {\n "cookie": "__session=x%3D.y",\n!! Expected 2 but received 1\n "counter": 1\n },\n "headers": { ... },\n "ok": true,\n "status": 200,\n "statusText": "OK"\n}"}
AwsLambdaFunctionUrlAdapterIntegTestDefaultTestDeployAssert216E26C9.AssertionResultsHttpApiCallbef3699880c06a026362e687054d8645 = {"status":"fail","message":"!! Expected type object but received undefined\nundefined"}

CDK CLI Version

2.1100.1

Framework Version

2.233.0

Node.js Version

22.16.0

OS

macOS

Language

TypeScript

Language Version

5.9.3

Other information

No response

[moltar]

It appears that this regression has been introduced in 2.191.0

This PR #906 is the only one in that release.

[moltar]

Using the --unstable=deprecated-cli-engine flag does indeed make it work again.