Skip to content

Renovation of the TestCase generation#3388

Merged
wadoon merged 5 commits intomainfrom
weigl/testgen
Sep 25, 2025
Merged

Renovation of the TestCase generation#3388
wadoon merged 5 commits intomainfrom
weigl/testgen

Conversation

@wadoon
Copy link
Copy Markdown
Member

@wadoon wadoon commented Jan 19, 2024

Intended Change

Make test case generation usable again, especially from the CLI in the new verification template.

The test case generation was not used for a long time. It was buggy and unstable, and could only be controlled by the UI. This PR tries to revive it a little bit:

Changes

  • Add a CLI + Facade for easy access from the CLI or to integrate it into build tools, e.g., Gradle or Maven.
  • Get rid of the old project layout, using shell scripts and manually set paths to JAR files.
    The new layout uses the Maven folder layout, and provides a build.gradle.kts and a pom.xml for building and testing
  • Add the test case from the book.
  • Robust generation of Java program code using javapoet. Nobody understands pages of StringBuilder foo.
  • Using picocli for the Command Line interface, also for the existing interfaces in key.ui, keyext.proofmanagement, ...
  • Clean up of old Java (introduction of records, sealed hierarchies, NPE checking, ...)

This is rather an MVP, more options and features can be thought of but first get it running.

Type of pull request

  • X Refactoring (behaviour should not change or only minimally change)
  • X New feature (non-breaking change which adds functionality)
  • X Breaking change (fix or feature that would cause existing functionality to change)
  • X There are changes to the (Java) code
  • X Renovations

@wadoon wadoon added 🐞 Bug Test Case Generator 🛠 Maintenance Code quality and related things w/o functional changes labels Jan 19, 2024
@wadoon wadoon added this to the v2.14.0 milestone Jan 19, 2024
@wadoon wadoon self-assigned this Jan 19, 2024
@wadoon wadoon force-pushed the weigl/testgen branch 2 times, most recently from 87f9f32 to 25a0200 Compare February 2, 2024 15:52
@wadoon wadoon force-pushed the weigl/testgen branch 2 times, most recently from 7abaaf2 to ddab633 Compare February 18, 2024 19:12
@KeYProject KeYProject deleted a comment from codecov bot May 23, 2024
@sonarqubecloud
Copy link
Copy Markdown

@KeYProject KeYProject deleted a comment from codecov bot Jul 20, 2024
@wadoon wadoon force-pushed the weigl/testgen branch 3 times, most recently from 2cedebc to 21573b2 Compare June 8, 2025 14:18
@wadoon wadoon force-pushed the weigl/testgen branch 3 times, most recently from 64bb3c5 to 82a27ce Compare July 14, 2025 00:37
@wadoon wadoon marked this pull request as ready for review July 14, 2025 00:37
@wadoon
Copy link
Copy Markdown
Member Author

wadoon commented Jul 14, 2025

@unp1 @mattulbrich Whoever has more experience with the hell of testgen.

@wadoon wadoon requested a review from unp1 July 14, 2025 00:38
@wadoon
Copy link
Copy Markdown
Member Author

wadoon commented Sep 20, 2025

I approve the PR once the problem with starting the version is resolved and I tested it on some small examples. Thanks a lot for the work.

Starting is resolved: Calling System.exit at the end of main does not work b/c the Swing instantiation is non-blocking.

For fostering the command line, the exit code is a good indicator of failure. We should reconsider the Main logic to respect this at some point.

@unp1
Copy link
Copy Markdown
Member

unp1 commented Sep 22, 2025

Thanks. I will complete the review later today

@unp1
Copy link
Copy Markdown
Member

unp1 commented Sep 22, 2025

Thanks I can open the UI now!
The following observations:

  • claims to have written test but cannot be found
Screenshot 2025-09-22 at 13 43 18 * selecting a folder where to store not possible via the magnifying class as folder cannot be selected and saved (only opened) * The result was different when not starting any proof and instead clicking the checkbox "Apply symbolic execution rules"

not implying that this PR should solve all these issues (in particular 3 might have been not working since longer and we can start the prover separately)

@unp1
Copy link
Copy Markdown
Member

unp1 commented Sep 22, 2025

Worked on CMdLine and I could now also find the directory. It was in my home directory (so it seems the GUI info is ignored?)

Maybe we should place it in the directory where it is called by default?

Anyways, I think the issues were present before and now at least the generation can be demoed an dI'll approve the change.

But please check if the mentioned issues should be dhave been resolved or if they fall under known issues.

Thanks!

Copy link
Copy Markdown
Member

@unp1 unp1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Makes it demoable but see my previous comment about known issues

@wadoon
Copy link
Copy Markdown
Member Author

wadoon commented Sep 22, 2025

  • claims to have written test but cannot be found

The test case from the book is here:
https://github.com/KeYProject/key/pull/3388/files#diff-3e5bc1e4d3a7fbee0d470dcd7d0b03e6f272fdd0c5ab1c8c8641023c7fd578e6R27

I tried to get binary search working, but was not so successful.

@wadoon
Copy link
Copy Markdown
Member Author

wadoon commented Sep 22, 2025

Thanks I can open the UI now! The following observations:

I avoided to go into the GUI at all. My user story would rather be the generation from inside of the project management / command line.

Clear is Objenesis and JML are resolved using Maven Central via pom.xml/build.xml/build.gradle.

I am also not sure, how the user story should be for the UI. Because, after the generation you need to leave KeY and go to the terminal.

wadoon and others added 5 commits September 25, 2025 21:35
# Conflicts:
#	key.core.testgen/src/main/java/de/uka/ilkd/key/testgen/TestCaseGenerator.java
#	key.core.testgen/src/test/java/de/uka/ilkd/key/testcase/smt/ce/TestCE.java
@wadoon
Copy link
Copy Markdown
Member Author

wadoon commented Sep 25, 2025

Rebased for better history. GUI issues are tracked in #3665

@wadoon wadoon added this pull request to the merge queue Sep 25, 2025
Merged via the queue into main with commit 6505edd Sep 25, 2025
35 checks passed
@wadoon wadoon deleted the weigl/testgen branch September 25, 2025 21:52
@wadoon wadoon modified the milestones: NextMajor, NextMinor, v3.0.0 Mar 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants