Skip to content

Conversation

@ratmice
Copy link
Collaborator

@ratmice ratmice commented Oct 17, 2025

To say this isn't very pretty is a bit of an understatement, but this is my first successful attempt at implementing
extra_files in the .yaml test output.

We want to use the CTLexerBuilder::lrpar_config callback which implements %grmtools{test_files} behavior.
In order to do that though, we must fix/hack up the lifetimes of the cgen_helper::run_test_files

We hack up those lifetimes in two ways:

  1. leak the Path sent to run_test_files
  2. reread and reparse the .test file yaml from within the closure.

The main issue being the borrow checker can't tell that the Path variable isn't used after the run_test_files function exits.
I spent quite a lot of time fighting this, and this is the only thing I could manage to get working.

I manually tested that when you change one of the .calc_input files to include any kind of malformed input that it is caught correctly.

@ratmice
Copy link
Collaborator Author

ratmice commented Oct 17, 2025

So just in the spirit of trying to brainstorm alternate ideas, if we had some sort of API for running the test_files manually, instead of using lrpar_config and relying on that function to call it, we could call it ourselves then continue to use the separate builders with the token map?

usize: num_traits::AsPrimitive<LexerTypesT::StorageT>,
{
lrpar_config: Option<Box<dyn Fn(CTParserBuilder<LexerTypesT>) -> CTParserBuilder<LexerTypesT>>>,
lrpar_config: Option<Box<dyn Fn(CTParserBuilder<LexerTypesT>) -> CTParserBuilder<LexerTypesT> + 'a>>,
Copy link
Collaborator Author

@ratmice ratmice Oct 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're willing to change the lifetime of this closure from being 'static to another lifetime like 'a here,
we can entirely get rid of the leak variables hacks.

This is here because Box is defaulting to 'static unless otherwise specified.
With it specified to something like 'a, the closure no longer needs to outlive the builder and main.

With that we can get rid of the leak hacks in this branch (at least, haven't looked into further cleanups yet)

Edit: It also allowed removal of the rereading and parsing of the .test yaml file.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going from 'static -> 'a seems fully backwards compatible (i.e. it's more general) to me, so I don't see a problem with this!

@ltratt
Copy link
Member

ltratt commented Oct 19, 2025

Please squash (if you wish).

@ratmice ratmice force-pushed the cttest_extra_files branch from 5bf06cd to 19e3ce6 Compare October 19, 2025 08:27
@ratmice
Copy link
Collaborator Author

ratmice commented Oct 19, 2025

Squashed, I moved the lifetime changes out to their own commit before the rest, so the changes to the library and cttests are somewhat separate from one another.

@ltratt
Copy link
Member

ltratt commented Oct 19, 2025

Thanks!

@ltratt ltratt added this pull request to the merge queue Oct 19, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Oct 19, 2025
Because `%grmtools{test_files}` needs *both* the lexer and parser
to work. We migrate this code from using the separate `CTLexerBuilder`
and `CTParserBuilder` with `rule_ids_map` to the combined method using
`lrpar_config`.

That causes `%grmtools{test_files}` to be invoked, also add an `extra_files`
keys to the `.test` file yaml.  These emit the files to `$OUT_DIR` where the
`test_files` globs are relative to.
@ratmice ratmice force-pushed the cttest_extra_files branch from 19e3ce6 to 0028126 Compare October 19, 2025 08:57
@ratmice
Copy link
Collaborator Author

ratmice commented Oct 19, 2025

Had to rustfmt the first commit, should be fixed.

@ltratt ltratt added this pull request to the merge queue Oct 19, 2025
Merged via the queue into softdevteam:master with commit 4e68283 Oct 19, 2025
2 checks passed
@ratmice ratmice deleted the cttest_extra_files branch October 22, 2025 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants