Skip to content

[libcxx] Test await_suspend control flow & coro_await_suspend_destroy attr #152820

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

[libcxx] Test await_suspend control flow & coro_await_suspend_destroy attr #152820

wants to merge 1 commit into from

Conversation

snarkmaster
Copy link

When reviewing #152623, @@ChuanqiXu9 suggested that it's better to separate clang and libcxx changes, so I did that here.

This event-replay test is compatible with compilers lacking the new attribute [[clang::coro_await_suspend_destroy]]. The two PRs can be landed in any order.

The test does not just verify the attribute's correct behavior. By comparing "gold" event traces, it also provides some "change detector" protection against future breakage in the complex control flow of the main C++ coroutine implementation. Since suspension remains an area of active optimization work, such bugs are not ruled out. For example, as a new contributor to LLVM, I made (and fixed) 2 control-flow bugs while working on #152623. This test would've caught both.

@snarkmaster
[libcxx] Change-detector for await_suspend control flow & end-to-en…
1b55fde 
…d test for [[clang::coro_await_suspend_destroy]]

When reviewing llvm#152623, @@ChuanqiXu9
suggested that it's better to separate `clang` and `libcxx` changes, so I did
that here.

This event-replay test is compatible with compilers lacking the new attribute.
The two PRs can be landed in any order.

Furthermore, the test does not just verify the attribute's correct behavior.
By storing "gold" event traces, it also provides some protection against future
breakage in the complex control flow of the C++ coroutine implementation.
Since suspension remains an area of active optimization work, such bugs are not
ruled out.  As a new contributor to LLVM, I myself made (and fixed) 2
control-flow bugs while working on llvm#152623.  This test would've caught both.
@snarkmaster snarkmaster requested a review from a team as a code owner August 9, 2025 02:29
@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 9, 2025

Thank you for submitting a Pull Request (PR) to the LLVM Project!

This PR will be automatically labeled and the relevant teams will be notified.

If you wish to, you can add reviewers by using the "Reviewers" section on this page.

If this is not working for you, it is probably because you do not have write permissions for the repository. In which case you can instead tag reviewers by name in a comment by using @ followed by their GitHub username.

If you have received no comments on your PR for a week, you can request a review by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate is once a week. Please remember that you are asking for valuable time from other developers.

If you have further questions, they may be answered by the LLVM GitHub User Guide.

You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums.

@llvmbot llvmbot added the libc++ libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi. label Aug 9, 2025
@llvmbot
Copy link
Member

llvmbot commented Aug 9, 2025

@llvm/pr-subscribers-libcxx

Author: None (snarkmaster)

Changes

When reviewing #152623, @@ChuanqiXu9 suggested that it's better to separate clang and libcxx changes, so I did that here.

This event-replay test is compatible with compilers lacking the new attribute [[clang::coro_await_suspend_destroy]]. The two PRs can be landed in any order.

The test does not just verify the attribute's correct behavior. By comparing "gold" event traces, it also provides some "change detector" protection against future breakage in the complex control flow of the main C++ coroutine implementation. Since suspension remains an area of active optimization work, such bugs are not ruled out. For example, as a new contributor to LLVM, I made (and fixed) 2 control-flow bugs while working on #152623. This test would've caught both.

Patch is 128.81 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/152820.diff

3 Files Affected:

  • (added) libcxx/test/std/language.support/support.coroutines/end.to.end/coro_await_suspend_destroy.golden-exceptions.h (+1163)
  • (added) libcxx/test/std/language.support/support.coroutines/end.to.end/coro_await_suspend_destroy.golden-no-exceptions.h (+1156)
  • (added) libcxx/test/std/language.support/support.coroutines/end.to.end/coro_await_suspend_destroy.pass.cpp (+560)
diff --git a/libcxx/test/std/language.support/support.coroutines/end.to.end/coro_await_suspend_destroy.golden-exceptions.h b/libcxx/test/std/language.support/support.coroutines/end.to.end/coro_await_suspend_destroy.golden-exceptions.h
new file mode 100644
index 0000000000000..2911e97ad9f59
--- /dev/null
+++ b/libcxx/test/std/language.support/support.coroutines/end.to.end/coro_await_suspend_destroy.golden-exceptions.h
@@ -0,0 +1,1163 @@
+// coro_await_suspend_destroy.golden-exceptions.h
++ test_coro_shortcircuits_to_empty(std::vector<test_driver>{
+    {.toggles_    = 0,
+     .threw_      = 0,
+     .result_     = {},
+     .next_event_ = 18,
+     .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 75, 2, 66, 64, 65}},
+    {.toggles_    = 1,
+     .threw_      = 1,
+     .result_     = -1,
+     .next_event_ = 19,
+     .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 75, 2, 66, 64, 81, 65}},
+    {.toggles_    = 2,
+     .threw_      = 0,
+     .result_     = {},
+     .next_event_ = 18,
+     .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 75, 2, 66, 64, 65}},
+    {.toggles_    = 3,
+     .threw_      = 1,
+     .result_     = -1,
+     .next_event_ = 19,
+     .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 75, 2, 66, 64, 81, 65}},
+    {.toggles_    = 4,
+     .threw_      = 0,
+     .result_     = {},
+     .next_event_ = 15,
+     .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 65}},
+    {.toggles_    = 5,
+     .threw_      = 1,
+     .result_     = -1,
+     .next_event_ = 16,
+     .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 81, 65}},
+    {.toggles_    = 6,
+     .threw_      = 0,
+     .result_     = {},
+     .next_event_ = 15,
+     .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 65}},
+    {.toggles_    = 7,
+     .threw_      = 1,
+     .result_     = -1,
+     .next_event_ = 16,
+     .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 81, 65}},
+    {.toggles_    = 8,
+     .threw_      = 0,
+     .result_     = {},
+     .next_event_ = 22,
+     .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 78, 2, 75, 2, 72, 69, 66, 64, 65}},
+    {.toggles_    = 9,
+     .threw_      = 1,
+     .result_     = -1,
+     .next_event_ = 23,
+     .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 78, 2, 75, 2, 72, 69, 66, 64, 81, 65}},
+    {.toggles_    = 10,
+     .threw_      = 0,
+     .result_     = {},
+     .next_event_ = 22,
+     .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 78, 2, 75, 2, 72, 69, 66, 64, 65}},
+    {.toggles_    = 11,
+     .threw_      = 1,
+     .result_     = -1,
+     .next_event_ = 23,
+     .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 78, 2, 75, 2, 72, 69, 66, 64, 81, 65}},
+    {.toggles_    = 12,
+     .threw_      = 0,
+     .result_     = {},
+     .next_event_ = 15,
+     .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 65}},
+    {.toggles_    = 13,
+     .threw_      = 1,
+     .result_     = -1,
+     .next_event_ = 16,
+     .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 81, 65}},
+    {.toggles_    = 14,
+     .threw_      = 0,
+     .result_     = {},
+     .next_event_ = 15,
+     .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 65}},
+    {.toggles_    = 15,
+     .threw_      = 1,
+     .result_     = -1,
+     .next_event_ = 16,
+     .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 81, 65}},
+    {.toggles_    = 16,
+     .threw_      = 0,
+     .result_     = {},
+     .next_event_ = 18,
+     .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 75, 2, 66, 64, 65}},
+    {.toggles_    = 17,
+     .threw_      = 1,
+     .result_     = -1,
+     .next_event_ = 19,
+     .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 75, 2, 66, 64, 81, 65}},
+    {.toggles_    = 18,
+     .threw_      = 0,
+     .result_     = {},
+     .next_event_ = 18,
+     .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 75, 2, 66, 64, 65}},
+    {.toggles_    = 19,
+     .threw_      = 1,
+     .result_     = -1,
+     .next_event_ = 19,
+     .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 75, 2, 66, 64, 81, 65}},
+    {.toggles_    = 20,
+     .threw_      = 0,
+     .result_     = {},
+     .next_event_ = 15,
+     .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 65}},
+    {.toggles_    = 21,
+     .threw_      = 1,
+     .result_     = -1,
+     .next_event_ = 16,
+     .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 81, 65}},
+    {.toggles_    = 22,
+     .threw_      = 0,
+     .result_     = {},
+     .next_event_ = 15,
+     .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 65}},
+    {.toggles_    = 23,
+     .threw_      = 1,
+     .result_     = -1,
+     .next_event_ = 16,
+     .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 81, 65}},
+    {.toggles_    = 24,
+     .threw_      = 0,
+     .result_     = {},
+     .next_event_ = 22,
+     .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 78, 2, 75, 2, 72, 69, 66, 64, 65}},
+    {.toggles_    = 25,
+     .threw_      = 1,
+     .result_     = -1,
+     .next_event_ = 23,
+     .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 78, 2, 75, 2, 72, 69, 66, 64, 81, 65}},
+    {.toggles_    = 26,
+     .threw_      = 0,
+     .result_     = {},
+     .next_event_ = 22,
+     .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 78, 2, 75, 2, 72, 69, 66, 64, 65}},
+    {.toggles_    = 27,
+     .threw_      = 1,
+     .result_     = -1,
+     .next_event_ = 23,
+     .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 78, 2, 75, 2, 72, 69, 66, 64, 81, 65}},
+    {.toggles_    = 28,
+     .threw_      = 0,
+     .result_     = {},
+     .next_event_ = 15,
+     .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 65}},
+    {.toggles_    = 29,
+     .threw_      = 1,
+     .result_     = -1,
+     .next_event_ = 16,
+     .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 81, 65}},
+    {.toggles_    = 30,
+     .threw_      = 0,
+     .result_     = {},
+     .next_event_ = 15,
+     .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 65}},
+    {.toggles_    = 31,
+     .threw_      = 1,
+     .result_     = -1,
+     .next_event_ = 16,
+     .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 81, 65}},
+}) +
+    test_coro_simple_await(std::vector<test_driver>{
+        {.toggles_    = 0,
+         .threw_      = 0,
+         .result_     = 12,
+         .next_event_ = 20,
+         .events_     = {67, 68, 73, 1, 74, 1, 73, 2, 74, 2, 70, 12, 75, 2, 75, 1, 69, 66, 64, 65}},
+        {.toggles_    = 1,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 21,
+         .events_     = {67, 68, 73, 1, 74, 1, 73, 2, 74, 2, 70, 12, 75, 2, 75, 1, 69, 66, 64, 81, 65}},
+        {.toggles_    = 2,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 22,
+         .events_     = {67, 68, 73, 1, 74, 1, 73, 2, 74, 2, 70, 12, 71, 75, 2, 75, 1, 72, 69, 66, 64, 65}},
+        {.toggles_    = 3,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 23,
+         .events_     = {67, 68, 73, 1, 74, 1, 73, 2, 74, 2, 70, 12, 71, 75, 2, 75, 1, 72, 69, 66, 64, 81, 65}},
+        {.toggles_    = 4,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 15,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 65}},
+        {.toggles_    = 5,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 16,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 81, 65}},
+        {.toggles_    = 6,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 15,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 65}},
+        {.toggles_    = 7,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 16,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 81, 65}},
+        {.toggles_    = 8,
+         .threw_      = 0,
+         .result_     = 12,
+         .next_event_ = 20,
+         .events_     = {67, 68, 73, 1, 74, 1, 73, 2, 74, 2, 70, 12, 75, 2, 75, 1, 69, 66, 64, 65}},
+        {.toggles_    = 9,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 21,
+         .events_     = {67, 68, 73, 1, 74, 1, 73, 2, 74, 2, 70, 12, 75, 2, 75, 1, 69, 66, 64, 81, 65}},
+        {.toggles_    = 10,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 22,
+         .events_     = {67, 68, 73, 1, 74, 1, 73, 2, 74, 2, 70, 12, 71, 75, 2, 75, 1, 72, 69, 66, 64, 65}},
+        {.toggles_    = 11,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 23,
+         .events_     = {67, 68, 73, 1, 74, 1, 73, 2, 74, 2, 70, 12, 71, 75, 2, 75, 1, 72, 69, 66, 64, 81, 65}},
+        {.toggles_    = 12,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 15,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 65}},
+        {.toggles_    = 13,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 16,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 81, 65}},
+        {.toggles_    = 14,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 15,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 65}},
+        {.toggles_    = 15,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 16,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 81, 65}},
+        {.toggles_    = 16,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 18,
+         .events_     = {67, 68, 73, 1, 74, 1, 73, 2, 79, 77, 2, 75, 2, 75, 1, 66, 64, 65}},
+        {.toggles_    = 17,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 19,
+         .events_     = {67, 68, 73, 1, 74, 1, 73, 2, 79, 77, 2, 75, 2, 75, 1, 66, 64, 81, 65}},
+        {.toggles_    = 18,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 18,
+         .events_     = {67, 68, 73, 1, 74, 1, 73, 2, 79, 77, 2, 75, 2, 75, 1, 66, 64, 65}},
+        {.toggles_    = 19,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 19,
+         .events_     = {67, 68, 73, 1, 74, 1, 73, 2, 79, 77, 2, 75, 2, 75, 1, 66, 64, 81, 65}},
+        {.toggles_    = 20,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 15,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 65}},
+        {.toggles_    = 21,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 16,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 81, 65}},
+        {.toggles_    = 22,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 15,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 65}},
+        {.toggles_    = 23,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 16,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 81, 65}},
+        {.toggles_    = 24,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 22,
+         .events_     = {67, 68, 73, 1, 74, 1, 73, 2, 79, 77, 2, 78, 2, 75, 2, 75, 1, 72, 69, 66, 64, 65}},
+        {.toggles_    = 25,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 23,
+         .events_     = {67, 68, 73, 1, 74, 1, 73, 2, 79, 77, 2, 78, 2, 75, 2, 75, 1, 72, 69, 66, 64, 81, 65}},
+        {.toggles_    = 26,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 22,
+         .events_     = {67, 68, 73, 1, 74, 1, 73, 2, 79, 77, 2, 78, 2, 75, 2, 75, 1, 72, 69, 66, 64, 65}},
+        {.toggles_    = 27,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 23,
+         .events_     = {67, 68, 73, 1, 74, 1, 73, 2, 79, 77, 2, 78, 2, 75, 2, 75, 1, 72, 69, 66, 64, 81, 65}},
+        {.toggles_    = 28,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 15,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 65}},
+        {.toggles_    = 29,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 16,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 81, 65}},
+        {.toggles_    = 30,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 15,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 65}},
+        {.toggles_    = 31,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 16,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 72, 69, 66, 64, 81, 65}},
+    }) +
+    test_coro_catching_shortcircuits_to_empty(std::vector<test_driver>{
+        {.toggles_    = 0,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 18,
+         .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 75, 2, 66, 64, 65}},
+        {.toggles_    = 1,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 19,
+         .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 75, 2, 66, 64, 81, 65}},
+        {.toggles_    = 2,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 18,
+         .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 75, 2, 66, 64, 65}},
+        {.toggles_    = 3,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 19,
+         .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 75, 2, 66, 64, 81, 65}},
+        {.toggles_    = 4,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 16,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 80, 72, 69, 66, 64, 65}},
+        {.toggles_    = 5,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 17,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 80, 72, 69, 66, 64, 81, 65}},
+        {.toggles_    = 6,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 16,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 80, 72, 69, 66, 64, 65}},
+        {.toggles_    = 7,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 17,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 80, 72, 69, 66, 64, 81, 65}},
+        {.toggles_    = 8,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 23,
+         .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 78, 2, 75, 2, 80, 72, 69, 66, 64, 65}},
+        {.toggles_    = 9,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 24,
+         .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 78, 2, 75, 2, 80, 72, 69, 66, 64, 81, 65}},
+        {.toggles_    = 10,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 23,
+         .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 78, 2, 75, 2, 80, 72, 69, 66, 64, 65}},
+        {.toggles_    = 11,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 24,
+         .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 78, 2, 75, 2, 80, 72, 69, 66, 64, 81, 65}},
+        {.toggles_    = 12,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 16,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 80, 72, 69, 66, 64, 65}},
+        {.toggles_    = 13,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 17,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 80, 72, 69, 66, 64, 81, 65}},
+        {.toggles_    = 14,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 16,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 80, 72, 69, 66, 64, 65}},
+        {.toggles_    = 15,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 17,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 80, 72, 69, 66, 64, 81, 65}},
+        {.toggles_    = 16,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 18,
+         .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 75, 2, 66, 64, 65}},
+        {.toggles_    = 17,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 19,
+         .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 75, 2, 66, 64, 81, 65}},
+        {.toggles_    = 18,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 18,
+         .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 75, 2, 66, 64, 65}},
+        {.toggles_    = 19,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 19,
+         .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 75, 2, 66, 64, 81, 65}},
+        {.toggles_    = 20,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 16,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 80, 72, 69, 66, 64, 65}},
+        {.toggles_    = 21,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 17,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 80, 72, 69, 66, 64, 81, 65}},
+        {.toggles_    = 22,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 16,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 80, 72, 69, 66, 64, 65}},
+        {.toggles_    = 23,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 17,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 80, 72, 69, 66, 64, 81, 65}},
+        {.toggles_    = 24,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 23,
+         .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 78, 2, 75, 2, 80, 72, 69, 66, 64, 65}},
+        {.toggles_    = 25,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 24,
+         .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 78, 2, 75, 2, 80, 72, 69, 66, 64, 81, 65}},
+        {.toggles_    = 26,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 23,
+         .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 78, 2, 75, 2, 80, 72, 69, 66, 64, 65}},
+        {.toggles_    = 27,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 24,
+         .events_     = {67, 68, 73, 1, 74, 1, 75, 1, 73, 2, 79, 77, 2, 78, 2, 75, 2, 80, 72, 69, 66, 64, 81, 65}},
+        {.toggles_    = 28,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 16,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 80, 72, 69, 66, 64, 65}},
+        {.toggles_    = 29,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 17,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 80, 72, 69, 66, 64, 81, 65}},
+        {.toggles_    = 30,
+         .threw_      = 0,
+         .result_     = {},
+         .next_event_ = 16,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 80, 72, 69, 66, 64, 65}},
+        {.toggles_    = 31,
+         .threw_      = 1,
+         .result_     = -1,
+         .next_event_ = 17,
+         .events_     = {67, 68, 73, 1, 74, 1, 76, 1, 75, 1, 80, 72, 69, 66, 64, 81, 65}},
+    }) +
+    test_coro_catching_simple_await(std::vector<test_driver>{
+        {.toggles_    = 0,
+         .threw_      = 0,
+         .result_     = 12,
+         .next_event_ = 20,...
[truncated]

@snarkmaster snarkmaster mentioned this pull request Aug 9, 2025
Open
snarkmaster added a commit to snarkmaster/llvm-project that referenced this pull request Aug 9, 2025
@snarkmaster
Split out the libcxx/test change into PR llvm#152820
62789ef
philnik777
philnik777 requested changes Aug 11, 2025
View reviewed changes
Copy link
Contributor

@philnik777 philnik777 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, why does this need to live in libc++? The attribute doesn't have anything to do with what we need to implement. This test also looks completely different from what any of our normal tests look like, and having "golden" in the name of a test seems very smelly to me.

@snarkmaster
Copy link
Author

snarkmaster commented Aug 11, 2025 •
edited
Loading

@philnik777, thanks for looking -- a few responses, please be understanding since this is my first time contributing to LLVM.

  1. This is an end-to-end test for the control flow of C++ coroutines. It does have a natural affinity for the <coroutine> header, but I will be the first to admit that the dependency on libcxx can be stubbed out with some effort.

  2. I would have preferred to put this in clang/, closer to the actual coroutine mechanics, but as far as I can tell, there is no place in clang/ that can run "end-to-end" tests of this sort. The specific things that the libcxx testing framework gives is -- it compiles AND executes the code with a large matrix of flags & architectures, including -fno-exceptions. I don't see any examples of this being doable in any clang/ tests.

  3. To your question about "golden", these traces are recording the expected sequencing of the various coroutine hooks. All this behavior is theoretically derivable from the standard, but it's not reasonable to manually test all the permutations. Recording & comparing traces is much more scalable.

  4. As it turns out the underlying implementation is complex enough that the code-local, "unit-style" tests of LLVM IR don't come close to catching all control flow bugs. Comparing traces caught some subtle and very problematic bugs during my work.

  5. If "golden" tests trigger an allergy, a watered-down version of this test is possible, where I just compare two trunk behaviors (await_suspend vs await_suspend_destroy), and drop the "change detector" part. But first, I would suggest that the coroutine stack maintainers weigh in on this, because I think they might see value in the "change detector" testing strategy for the core control flow.

I hope this gives some additional useful context! Please guide me in regards to these concrete decisions:

  • If this can't go in libcxx/, where in llvm-project/ can I get a test to run on a reasonably broad test matrix of flags and runtimes, to make sure that the underlying coroutine compiler logic is robust?

  • While I agree that "change detector" tests in general are smelly, in this case, we're detecting changes from standard-mandated behavior, so I think it's actually rather good. How strongly do you insist on removing the "gold" part of the test flow? Are you open to a conversation with the coroutine folks first?

@snarkmaster snarkmaster requested a review from philnik777 August 11, 2025 18:23
@philnik777
Copy link
Contributor

@philnik777, thanks for looking -- a few responses, please be understanding since this is my first time contributing to LLVM.

1. This is an end-to-end test for the control flow of C++ coroutines. It does have a natural affinity for the `<coroutine>` header, but I will be the first to admit that the dependency on `libcxx` can be stubbed out with some effort.

2. I would have preferred to put this in `clang/`, closer to the actual coroutine mechanics, but as far as I can tell, there is no place in `clang/` that can run "end-to-end" tests of this sort. The specific things that the `libcxx` testing framework gives is -- it compiles AND executes the code with a large matrix of flags & architectures, including `-fno-exceptions`. I don't see any examples of this being doable in any `clang/` tests.

It is quite funny to see that we have such good coverage that we're asked to run tests for other sub-projects. That may be an indication that other sub-projects need some more robust CI.

The main problem here is that the tests under libcxx/ are supposed to test libc++. We have to maintain these tests, but they're not actually testing anything we care directly about. It's also very different from other tests, so most contributors (including me currently) have no idea what they're even looking at.

3. To your question about "golden", these traces are recording the expected sequencing of the various coroutine hooks. All this behavior is theoretically derivable from the standard, but it's not reasonable to manually test all the permutations. Recording & comparing traces is much more scalable.

4. As it turns out the underlying implementation is complex enough that the code-local, "unit-style" tests of LLVM IR don't come close to catching all control flow bugs. Comparing traces caught some subtle and very problematic bugs during my work.

5. If "golden" tests trigger an allergy, a watered-down version of this test is possible, where I just compare two trunk behaviors (`await_suspend` vs `await_suspend_destroy`), and drop the "change detector" part. But first, I would suggest that the coroutine stack maintainers weigh in on this, because I think they might see value in the "change detector" testing strategy for the core control flow.

I'm sure there is value. However, the libc++ test suite is usually not run against the rest of the compiler. So any breaks in LLVM would break our CI. This is just not an acceptable way to handle this, especially since these are (as far as I understand currently) golden tests - i.e. they may be broken without the changes that broke them being incorrect.

I hope this gives some additional useful context! Please guide me in regards to these concrete decisions:

* If this can't go in `libcxx/`, where in `llvm-project/` can I get a test to **run** on a reasonably broad test matrix of flags and runtimes, to make sure that the underlying coroutine compiler logic is robust?

* While I agree that "change detector" tests in general are smelly, in this case, we're detecting changes from standard-mandated behavior, so I think it's actually rather good. How strongly do you insist on removing the "gold" part of the test flow? Are you open to a conversation with the coroutine folks first?

I'm happy to talk to people who work on coroutines. How happy I am about golden tests depend very much on how golden they actually are. If they're actually testing standards-mandated behaviour I wouldn't consider them golden - simply conformance tests. Though that still doesn't mean they should live inside libcxx/.

@snarkmaster
Copy link
Author

snarkmaster commented Aug 11, 2025 •
edited
Loading

Thanks @philnik777 , I agree that these effects you're flagging are quite problematic:

(1) For any breakage in these tests, the libcxx team has to triage them back to compiler coroutine folks. It feels like unnecessary toil.

(2) Since there's a lag between compiler changes and libcxx tests picking up the new compiler, the signal from these tests will arrive late, which isn't good for anyone either.

So, philosophically, I'm also in favor of moving this into clang/.

I think I did a reasonable job of looking at the existing clang/ test patterns, and neither clang/test/ nor clang/unittests/ seem to be a really great fit. Definitely not the IR tests in clang/test/. Perhaps clang/unittests/ could be forced into being "ok".

Being very new to the LLVM build-and-test flows (it seems to be CMake x {ninja, make, xcodebuild, ...} x {lit, ...}), I'm not sure I could volunteer to do the plumbing for a new type of compiler test, and actually do a good job of it.

Do you know who might be able to give me guidance on this?

@ChuanqiXu9, do you have any ideas here?

PS I agree about calling these "conformance tests" instead of the more nebulous "golden". I will do this in the next rev. Of course, being a "change detector" design, they could be measuring current non-conformance, but at least we know when the degree of conformance changes lol.

@yuxuanchen1997
Copy link
Member

Hi @snarkmaster, the usual location for tests related to your change are located in clang/test/CodeGenCoroutines/. Under such directory there's also an Inputs directory that contains a simple implementation of <coroutine> should you need it.

@snarkmaster
Copy link
Author

I was able to talk with @yuxuanchen1997 offline, and we got on the same page. These should go in test-suite instead. Reasons:

  • It's not possible to get this kind of combinatorially broad coverage from IR tests in clang/test/.
  • Neither libcxx/ nor clang/unittests/ are guaranteed to run with any frequency against clang trunk
  • With test-suite, we have a guarantee that it'll run against clang "often" (per the docs). So, even though there'll be a delay between "breakage" and "signal", at least it's predictable.

Unless I hear otherwise, I'll resubmit this PR to test-suite and put the files into C++/Coroutines/SuspendConformance.cpp here: https://github.com/llvm/llvm-test-suite/tree/main/MultiSource/UnitTests

What I'm doing is quite similar in spirit to this test: https://github.com/llvm/llvm-test-suite/tree/main/MultiSource/UnitTests/C%2B%2B11/frame_layout

I'll leave this open for feedback for another couple of days.

Thanks all for taking the time to guide me here!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@philnik777 philnik777 Awaiting requested review from philnik777

Assignees
No one assigned
Labels
libc++ libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@snarkmaster @llvmbot @philnik777 @yuxuanchen1997