Failures in linux dds pkg tests

[a-siva]

There are new test failures on Add release date to CHANGELOG.md.

The tests

pkg/dds/test/dap/integration/debug_breakpoints_test RuntimeError (expected Pass)
pkg/dds/test/dap/integration/debug_eval_test RuntimeError (expected Pass)

are failing on configurations

unittest-asserts-debug-linux-x64

[a-siva]

Looks like these tests are racy, on one of the local runs I got this error

00:28 +16 -1: debug mode variables renders a Uint8List [E]

  Request "stackTrace" failed:
  {
      "seq": 19,
      "type": "response",
      "body": {
          "error": {
              "format": "{message}",
              "id": 1,
              "showUser": false,
              "variables": {
                  "message": "[Sentinel kind: Collected, valueAsString: <collected>] from getStack()",
                  "stack": "#0      new _OutstandingRequest (package:vm_service/src/vm_service.dart:265:34)\n#1      VmService._call.<anonymous closure> (package:vm_service/src/vm_service.dart:1921:25)\n#2      VmService._call (package:vm_service/src/vm_service.dart:1933:8)\n#3      VmService.getStack (package:vm_service/src/vm_service.dart:1277:7)\n#4      DartDebugAdapter.stackTraceRequest (package:dds/src/dap/adapters/dart.dart:1812:38)\n#5      BaseDebugAdapter.handle (package:dds/src/dap/base_debug_adapter.dart:141:20)\n#6      BaseDebugAdapter.handleIncomingRequest (package:dds/src/dap/base_debug_adapter.dart:442:7)\n#7      BaseDebugAdapter._handleIncomingMessage (package:dds/src/dap/base_debug_adapter.dart:311:7)\n#8      ByteStreamServerChannel._readMessage (package:dds/src/dap/protocol_stream.dart:82:18)\n#9      ByteStreamServerChannel.listen.<anonymous closure> (package:dds/src/dap/protocol_stream.dart:53:24)\n#10     _RootZone.runUnaryGuarded (dart:async/zone.dart:1778:10)\n#11     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:381:11)\n#12     _DelayedData.perform (dart:async/stream_impl.dart:573:14)\n#13     _PendingEvents.handleNext (dart:async/stream_impl.dart:678:11)\n#14     _PendingEvents.schedule.<anonymous closure> (dart:async/stream_impl.dart:649:7)\n#15     _microtaskLoop (dart:async/schedule_microtask.dart:40:21)\n#16     _startMicrotaskLoop (dart:async/schedule_microtask.dart:49:5)\n#17     _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:128:13)\n#18     _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:195:5)\n"
              }
          }
      },
      "command": "stackTrace",
      "message": "[Sentinel kind: Collected, valueAsString: <collected>] from getStack()",
      "request_seq": 6,
      "success": false
  }

[a-siva]

//cc @DanTup @derekxu16

[DanTup]

@a-siva how are you running the tests? I'm failing to repro the one you gave above, though I'm on Windows and running from the "Run" CodeLens in VS Code.. I'm not sure which is most likely to be different to you (although it could also be either, and something like the specs of our machine).

Otherwise, if you add forceVerboseLogging: true to the setup call:

This might result in more verbose output for the tests that fail.

Looking at the test, it appears that we hit a breakpoint before sending getStack(), but a verbose log might help verify if we are actually still paused when we send this (if we are, I think this may be a VM issue.. but if we are not, then it may be a DAP issue).

I don't know if the other failures are the same though. I'll have a dig through the logs and see if I can repro any locally to debug.

[DanTup]

When running these tests in WSL on Windows, I actually get a lot of failures and I think these are all related. There's a race condition that can result in us sending two readyToResume calls during startup, and this can result in us unpausing from the breakpoint (depending on timing) that we should not.

Working on a fix.

[DanTup]

I've opened https://dart-review.googlesource.com/c/sdk/+/410760 that fixes all of the flakes I see locally when running in WSL.

[a-siva]

Thanks for the fix @DanTup

[DanTup]

np! The failure I saw wasn't exactly the same as yours, however they seem like the same thing (the VM being unpaused when it shouldn't). If you do still see this failure/flake, please let me know.

There are still some other open issues about DAP flakes that I'm going to take another look through once this change has been in for a little while since it could explain some of the others too.