ci: 👷 check that examples in header files compile #774

ion098 · 2025-09-08T18:19:09Z

Summary:

This PR adds a Python script to be ran during CI that isolates all the example code snippets in the include/pros directory and checks that they compile.
~~(Note: this script was partially written by Copilot)~~ The current code doesn't contain any part of the original code which contained code from Copilot

Motivation:

Prevents issues like #760.

Test Plan:

CI passes

ion098 · 2025-09-09T15:38:32Z

@Rocky14683 Can you please run the pipeline?

Rocky14683 · 2025-09-10T01:18:19Z

/azp run

azure-pipelines · 2025-09-10T01:18:31Z

Azure Pipelines successfully started running 1 pipeline(s).

Updated logging code and regex pattern for code blocks.

ion098 · 2025-09-10T22:17:11Z

@Rocky14683 can you please rerun pipeline?

BennyBot · 2025-09-10T22:36:35Z

/azp run

azure-pipelines · 2025-09-10T22:36:46Z

Azure Pipelines successfully started running 1 pipeline(s).

BennyBot · 2025-09-12T03:36:39Z

/azp run

azure-pipelines · 2025-09-12T03:36:50Z

Azure Pipelines successfully started running 1 pipeline(s).

BennyBot · 2025-09-12T04:07:06Z

/azp run

azure-pipelines · 2025-09-12T04:07:16Z

Azure Pipelines successfully started running 1 pipeline(s).

ion098 · 2025-09-12T20:39:44Z

@BennyBot can you please rerun the pipeline?

BennyBot · 2025-09-12T21:08:17Z

/azp run

azure-pipelines · 2025-09-12T21:08:27Z

Azure Pipelines successfully started running 1 pipeline(s).

check_example_code.py

ion098 · 2025-09-15T19:23:00Z

@BennyBot can you please rerun the pipeline?

Rocky14683 · 2025-09-15T19:37:48Z

/azp run

azure-pipelines · 2025-09-15T19:38:00Z

Azure Pipelines successfully started running 1 pipeline(s).

djava

this asyncio approach is much cleaner, nice job.

Last request besides what comments i left was just - type hints?

djava · 2025-09-16T14:41:28Z

check_example_code.py

+        "include/main.h",
+        "-std=c++23",
+        "-I", "include",
+        "-D", "_PROS_KERNEL_SUPPRESS_LLEMU_WARNING",


huge nitpick (like, dont bother changing this) but for the record its UB to have any identifier (including macros) that start with underscore then a capital letter lol.

actually idek if you wrote that macro or not but 🤷 just having fun being an asshole by mentioning it

djava · 2025-09-16T14:41:36Z

check_example_code.py


    Args:
        code_text (str): The C/C++ source code to compile.
-        filename (str): The name of the file.


was this remove a mistake?

Yeah, that shouldnt have been removed

djava · 2025-09-16T14:41:45Z

check_example_code.py

+        # Generate assembly (avoids linker errors)
+        "-S",


does the more standard -c not work? if not then probably give a longer comment to explain why youre doing -S because it took me a second for that to click

-c would probably work, but assembling the files seems like an unnecessary step.

Ok, maybe just a better comment - something like Just generate assembly, the further compilation steps are not necessary for this CI check maybe

djava · 2025-09-16T14:41:59Z

check_example_code.py

+        stdout, stderr = await process.communicate(input=code_text.encode("utf-8"))
+        success = (process.returncode == 0)


interesting - where do you await the compilation being done? im not super familiar with this but im confused. does process.communicate() imply blocking to wait for a response? is there a timeout?

Yes, communicate sends input via stdin and then waits for the process to finish running (no timeout though)

ok maybe a comment that says that because its not obvious imo

djava · 2025-09-16T16:29:53Z

check_example_code.py

+        "-x", "c++" if is_cpp else "c",
+        "-std=c++23" if is_cpp else "-std=c2x",
+        # Only need precompiled headers for C++
+        *(("-I", precompiled_include_directory) if is_cpp else ()),


weird syntax trick lol maybe just do it after the list initialization

BennyBot · 2025-09-17T17:47:52Z

/azp run

azure-pipelines · 2025-09-17T17:48:03Z

Azure Pipelines successfully started running 1 pipeline(s).

meisZWFLZ · 2025-09-23T23:34:35Z

Some silly notes I had while skimming the code:

Perhaps you could use the compiler flags from the makefile? Likely by adding a make rule to extract the flags
a. This could be applied to other configuration too, like the include directory
Idk how azure pipelines work, but the examples check can be done in parallel to the other checks
Maybe you're already doing this, but you could compile each snippet in parallel
A make rule could be added to run the script
Being able to specify a specific header to test would be useful for development

1 seems like it would be a lot of work, but everything else seems reasonable and justifiable

ion098 · 2025-09-24T17:07:50Z

re @meisZWFLZ

Perhaps you could use the compiler flags from the makefile? Likely by adding a make rule to extract the flags
a. This could be applied to other configuration too, like the include directory

That's probably a good idea in the long run, I didn't implement it because Makefile scares me a little.

Idk how azure pipelines work, but the examples check can be done in parallel to the other checks

Maybe you're already doing this, but you could compile each snippet in parallel

The code compiles everything in parallel using as many cores as possible with asyncio, so I don't think doing both checks in parallel would offer much speedup.

A make rule could be added to run the script

Being able to specify a specific header to test would be useful for development

That seems reasonable, but the script needs a few changes for development use---caching the example code for example, since the script always recompiles every example code snippet right now, which works for ci but makes it painful to repeatedly run during development.

meisZWFLZ · 2025-10-06T20:16:56Z

Do the other checks still run if the example check fails? I would think that would be ideal.

I kinda wonder whether this should be written in the makefile rather than a python script. Might make it a bit easier to implement caching with make rules.

ion098 added 3 commits September 8, 2025 11:08

ci: ✅ Add script to check that all example code compiles

810a55b

ci: 🐛 fix syntax issues and improve code clarity

f28a981

ci: 👷 add step to Azure pipelines to check example code

8e68045

ion098 changed the title ~~ci: 👷 check that example code compiles~~ ci: 👷 check that examples in header files compile Sep 9, 2025

fix: 🚧 Add logging to check_example_code.py

f0c2297

Updated logging code and regex pattern for code blocks.

ci: 🐛 check_example_code actually works now

b0b8163

ci: 🐛 change C standard from c23 to c2x to work with gcc 13

6caedfb

ion098 added 2 commits September 12, 2025 16:41

fix: 🚧 fix compiler errors in example code

e1617a7

ci: ⚡ compile example code snippets in parallel

4f234b1

djava reviewed Sep 12, 2025

View reviewed changes

check_example_code.py Outdated Show resolved Hide resolved

check_example_code.py Outdated Show resolved Hide resolved

ci: ⚡ use asyncio, precompile main.h for c++

2d9a547

djava reviewed Sep 16, 2025

View reviewed changes

docs: 🚧 fix compiler errors in example code

d32af1c

docs: 🚧 fix more compiler errors in examples

7b548e6

		stdout, stderr = await process.communicate(input=code_text.encode("utf-8"))
		success = (process.returncode == 0)

ci: 👷 check that examples in header files compile #774

Are you sure you want to change the base?

ci: 👷 check that examples in header files compile #774

Uh oh!

Conversation

ion098 commented Sep 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary:

Motivation:

Test Plan:

Uh oh!

ion098 commented Sep 9, 2025

Uh oh!

Rocky14683 commented Sep 10, 2025

Uh oh!

azure-pipelines bot commented Sep 10, 2025

Uh oh!

ion098 commented Sep 10, 2025

Uh oh!

BennyBot commented Sep 10, 2025

Uh oh!

azure-pipelines bot commented Sep 10, 2025

Uh oh!

BennyBot commented Sep 12, 2025

Uh oh!

azure-pipelines bot commented Sep 12, 2025

Uh oh!

BennyBot commented Sep 12, 2025

Uh oh!

azure-pipelines bot commented Sep 12, 2025

Uh oh!

ion098 commented Sep 12, 2025

Uh oh!

BennyBot commented Sep 12, 2025

Uh oh!

azure-pipelines bot commented Sep 12, 2025

Uh oh!

Uh oh!

Uh oh!

ion098 commented Sep 15, 2025

Uh oh!

Rocky14683 commented Sep 15, 2025

Uh oh!

azure-pipelines bot commented Sep 15, 2025

Uh oh!

djava left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ion098 Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

BennyBot commented Sep 17, 2025

Uh oh!

azure-pipelines bot commented Sep 17, 2025

Uh oh!

meisZWFLZ commented Sep 23, 2025

Uh oh!

ion098 commented Sep 24, 2025

Uh oh!

ion098 commented Sep 8, 2025 •

edited

Loading

ion098 Sep 16, 2025 •

edited

Loading