Skip to content

Commit 8200416

Browse files
mk-mxpmk-mxp
committed
Re-write test generator with templates, unit tests
1 parent d70c33a commit 8200416

File tree

210 files changed

+2624
-263
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

210 files changed

+2624
-263
lines changed

‎.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,3 +11,4 @@ composer.lock
1111

1212
# IDE Files
1313
.idea
14+
.vscode

‎contribution/documentation/generator/README.md

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
# PHP code generator for Exercism PHP track exercises
2+
3+
- [Introduction](#introduction)
4+
- [Architecture](#architecture)
5+
- [Contribution](#contribution)
6+
7+
## Introduction
8+
9+
This is a simple code generator for practice exercises in the PHP track based on the [Exercism common problem specifications][exercism-problem-specifications].
10+
11+
> Please read and think about the exercise instructions!
12+
> Many problems require additional test failure messages and useful information to help students solve the exercise.
13+
> Generating code is not "being done"!
14+
15+
The majority of problems in problem specifications are *function oriented*.
16+
That means, all input goes into a single function call and no state of an object changes expected results.
17+
So the generator generates *function oriented* code.
18+
A fresh instance is created with no constructor arguments in `setUp()` for each test.
19+
The tests invoke methods with the input and compare actual results with expectations.
20+
21+
If the problem you generate code for requires object orientation, adjust the tests manually (e.g. replace `$this->subject->`).
22+
23+
The next decision to make is: How much freedom of implementation shall students have?
24+
For practice exercises we usually give maximum freedom of implementation.
25+
This freedom must be designed into the student facing interface.
26+
But there are good reasons to limit the freadom of choice.
27+
Has PHP an idiomatic way to solve such a class of problems?
28+
Like using an `enum` for certain result types.
29+
Then you should design that into the interface.
30+
31+
Mentoring is done to guide students towards "recommended" implementations.
32+
Some exercises do require stricter boundaries, like "Do not use language provided functions to solve this".
33+
Such restrictions need to be implemented manually.
34+
35+
Another decision required is the amount of prepared student interface code you want.
36+
The generator only produces the bare minimum, an empty class with a throwing constructor.
37+
This is a pragmatic choice, it is easy to implement. 🙂
38+
Adding predefined methods lowers the difficulty of the exercise.
39+
Testing for type declarations being set and / or testing for type safety raises the difficulty.
40+
So it is your choice, if you want to do so or not.
41+
42+
Now you have made the basic decisions. Time to use the generator!
43+
44+
- Follow the track README to install track tooling (`configlet`)
45+
- Run from track root:
46+
47+
```shell
48+
bin/configlet create --practice-exercise '<slug>'
49+
composer -d contribution/generator install
50+
contribution/generator/bin/console app:create-tests '<slug>'
51+
composer lint:fix
52+
vendor/bin/phpunit exercises/practice/'<slug>'/*Test.php
53+
```
54+
55+
- Run `git status` to see all the generated files.
56+
These are yours now.
57+
- Adjust the code as required.
58+
- Add more information to `.meta/*.append.md`.
59+
The generated files `.meta/*.md` are kept in sync with problem specifications.
60+
- Mark tests not implemented with `include = false` in `tests.toml`.
61+
- Open a PR to get feedback on your exercise **early**.
62+
63+
## Architecture
64+
65+
The `Exercise` interface supports both types of exercises, `PracticeExercise` and the planned `ConceptExercise`.
66+
For both exercise types, the test file and the students file are generated from `configlet` generated directory structures and files.
67+
It is planned to use a `canonical-data.json` similar to the problem specification for the concept exercises, too.
68+
69+
Between the symfony command(s) and the actual test / students file generation is the test boundary `ItemFactory`.
70+
By this, the integration testing (not yet implemented) needs to test only, that the expected files are generated with some sample text.
71+
The actual details of test / student file generation can be tested without mocking the filesystem or booting the symfony kernel.
72+
73+
![Top level sequence](sequence-diagram-top-level.svg)
74+
75+
`configlet` is used by `PracticeExercise`, wrapped in an own class.
76+
This is because the [cached problem specifications][exercism-problem-specifications] is required and `configlet` knows best where to find that.
77+
The actual path differs depending on the underlying operating system, and we shouldn't copy that from `configlet` sources.
78+
79+
We started with an implementation walking through the raw JSON data and used `nikic/php-parser` to produce PHP code from that.
80+
This hided the underlying data structures used to construct the varying canonical data sets in the problem specification.
81+
So we made them explicit:
82+
83+
- `Item`s are all data structures to construct the varying canonical data sets (interface).
84+
- `ItemFactory` turning raw data into the matching `Item` (object).
85+
- `Unknown` represents data structures that do not follow a known schema (object).
86+
- `TestCase`s represent tests with input and expected outcome of the students code (object).
87+
- `Group`s are sets of `Item`s, that share a common theme and may have some title, explanation and folding section markers (object).
88+
- `InnerGroup`s are the pure lists of `Item`s to convert to tests (object).
89+
- `CanonicalData` is the outermost group with the expected extra data for an exercise and an `InnerGroup` with `Item`s (object).
90+
91+
![Second level sequence](sequence-diagram-second-level.svg)
92+
93+
This allows to represent the JSON data as a tree of groups and test cases.
94+
Every data structure object examines the raw data and produces an instance only, if it can handle that structure.
95+
The tree is built using `ItemFactory`, which starts with the most specific data structure `CanonicalData` and goes out towards `Unknown` until an instance is made from the raw data given.
96+
Having all unknown data catched into an `Unknown` allows rendering any structure found into the output files.
97+
98+
Such a tree structure looks like doing the code production with the "visitor" pattern.
99+
But that pattern comes with a huge cost of complexity and should only be used, if the tree will have multiple visitors.
100+
So the simple way of iterating directly over the tree to produce the output from the data objects is prefered.
101+
102+
Also it is tempting to implement a transformation logic "data -> abstract syntax tree", which we had with `nikic/php-parser`.
103+
But this also adds a huge amount of complexity for no real win - the AST would only be used for pre-defined code output.
104+
Predefined code snippets - code templates with placeholders for the data given - are much simpler and allow direct control of formatting.
105+
Templates also don't require special knowledge about the AST library used.
106+
Simply edit the template using IDEs and other tools like any other file.
107+
For our purpose of producing code, that **should be** verified and corrected by humans, templates are the best solution.
108+
109+
[exercism-problem-specifications]: https://github.com/exercism/problem-specifications/
110+
111+
## Contribution
112+
113+
To get ready to contribute to the generator, run these commands:
114+
115+
```shell
116+
cd contribution/generator
117+
composer install
118+
vendor/bin/phpunit
119+
```
120+
121+
- Add unit tests for changes to `TrackData` classes
122+
- Document changes in [Introduction](#introduction) and [Architecture](#architecture)
123+
124+
### Architecture Diagrams
125+
126+
Use [PlantUML](http:://plantuml.com/) to add / modify diagrams.

‎contribution/documentation/generator/sequence-diagram-second-level.plantuml

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
@startuml
2+
participant Exercise as exercise
3+
participant CanonicalData as data
4+
participant InnerGroup as group
5+
participant Item as item
6+
7+
exercise -> data : ItemFactory::from($data)
8+
data -> group : ItemFactory::from($data)
9+
group -> group : ItemFactory::from($data)
10+
group -> item : ItemFactory::from($data)
11+
12+
exercise -> data : renderTestCode()
13+
data -> group : renderTestCode()
14+
group -> group : renderTestCode()
15+
group -> item : renderTestCode()
16+
17+
exercise -> data : renderSolutionCode()
18+
@enduml

‎contribution/documentation/generator/sequence-diagram-second-level.svg

Lines changed: 1 addition & 0 deletions
Loading

‎contribution/documentation/generator/sequence-diagram-top-level.plantuml

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
@startuml
2+
actor Developer as dev
3+
participant Command as command
4+
participant Exercise as exercise
5+
participant CanonicalData as data
6+
7+
dev -> command : create-exercise $slug
8+
command -> exercise : create($slug)
9+
exercise -> exercise : loadData($slug)
10+
exercise -> data : ItemFactory::from($data)
11+
command -> exercise : testFileName()
12+
command -> exercise : testFileContent()
13+
exercise -> data : renderTestCode()
14+
command -> exercise : solutionFileName()
15+
command -> exercise : solutionFileContent()
16+
exercise -> data : renderSolutionCode()
17+
@enduml

0 commit comments

Comments
 (0)