Skip to content

Add C++#532

Merged
jorg-vr merged 75 commits intomasterfrom
feat/add-cpp
Mar 24, 2025
Merged

Add C++#532
jorg-vr merged 75 commits intomasterfrom
feat/add-cpp

Conversation

@jorg-vr
Copy link
Contributor

@jorg-vr jorg-vr commented Jul 17, 2024

This pr adds c++ to TESTed.

It is compiled using gcc 10.2.1. The current version supports all c++ 17 features and some c++ 20 features.

Convention choices:

I have chosen to prefer std types over standard c types. So even though you could write valid c++ without using any std types, this won't work well with our judge as we try to follow best practices.

I used the string key cpp instead of c++. Both are commonly used, but the first is less likely to cause issues of being parsed incorrectly.

Heterogeneous collections are supported using std::variant.

Value arguments to a function are always cast to their type. This is done because not all types are well defined by how a value is written, and if the student uses templates for arguments this can cause compiler issues, without any type info present.

Limitations

TESTed cannot differentiate between applying functions on objects, pointers and classes or namespaces.
As all three have their own operator in c++ (., -> and :: respectively), I could not generate code that works in all cases. For this reason I made the following choices:

  • By default we always assume objects (and thus use the . operator)
  • If we got a PascalCase identifier, I've assumed it is a class and thus used ::. This is the only case where :: is generated.
  • pointers are not supported (-> is never generated)

Custom oracles written in c++ require either an object with methods if the filename is camelCase or a a class with static methods if the filename is PascalCase. Simple methods are not supported as TESTed always provides the filename of the oracle as namespace to the generator.

Typing statements isn't always easy and heavily reliant on the values. For some statements I simply do not have enough information to provide a type. Eg. foo: list = bar() and foo = [bar()] will both generate a variable foo of type std::vector<std::any>.
As all type information is lost on elements of type std::any these are hard to use for students and almost impossible to properly convert to json values for me. As type hinting is currently limited in TESTed, it would probably be best to use language specific statements in these cases.

Unrelated changes

I forgot to run half the tests for cpp as the language had to manually be specified for every test.
I have refactored this so most tests use predefined language sets from tests/language_markers.py.
These tests are now defined with a new helper function all_languages_except. This way, a new language is automatically tested on all tests, but it remains easy to exclude it from certain test if you decide this test shouldn't work for the given language.

Observations

Adding a language to TESTed requires three main parts:

  1. a config.py with a class inheriting from the default language class
  2. a generators.py which translates the internal TESTed format into a testfile in the desired language
  3. a templates folder containing language specific helper functions to convert return values and errors to the json format expected by tested

Extending the config.py is properly done using inheretance. Filling out the required methods, which are also mostly documented.

The generators.py is badly organized. There seems to be some kind of method naming convention, as all generators.py files seem to be largely copied from each other.
This results in lots of duplicate code. Because there is no clear API to implement, it is also not always clear which cases should be implemented. This part was the hardest part to write, but is mostly skimmed over in the documentation.
I think this part of the code could really benefit with some types of abstraction, such as an abstract class with methods to implement. Maybe with some different abstract subclasses for typed and untyped languages, which could also reduce code duplication. This issue could be solved with visitors as specified in #564

The templates folder was not really documented either. It took me a while to figure out what the files actually did. These files are technically not required by TESTed, but helpers for the code generated in generators.py. But we seem to use this for every language. So some form of specification/standardization would be helpful.
This was also hard to implement for me in c++, but that is just because my knowledge of c++ wasn't good enough to properly write type generics.

To end with a more positive observation: The test catch a lot of edge cases. While it was a frustrating experience to keep fixing bugs to get each test passing, it is very clear that the test have heavily improved my initial implementation.

Closes #530

Imran-imtiaz48

This comment was marked as spam.

@niknetniko niknetniko linked an issue Aug 23, 2024 that may be closed by this pull request
@jorg-vr
Copy link
Contributor Author

jorg-vr commented Mar 10, 2025

Thanks again @niknetniko for the detailed and good feedback. Although it took me another week it greatly improved my code.

Biggest changes:

  • I hadn't added c++ to almost half the test cases. Running them made me have to resolve a multitude of different edge cases. (again my compliments on the wide range of tests)
  • I have made the code completely independent of c and focused on following modern c++ standards. This made me rewrite so many parts that there is almost no code duplication left.
  • I have improved support for types of heterogeneous collections by traversing all values and creating std::variant types instead of defaulting to std::any. std::any is now only used for unknown values, such as function results (see limitations).

@jorg-vr jorg-vr marked this pull request as ready for review March 10, 2025 14:16
@jorg-vr jorg-vr requested a review from niknetniko March 10, 2025 14:16
Copy link
Member

@niknetniko niknetniko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@jorg-vr jorg-vr merged commit b705a9f into master Mar 24, 2025
9 checks passed
@jorg-vr jorg-vr deleted the feat/add-cpp branch March 24, 2025 14:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add C++ support

3 participants