You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+9-9Lines changed: 9 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,7 +16,7 @@ You can modify the list of models in `models.jl`.
16
16
17
17
## I want to edit the HTML that gets generated!
18
18
19
-
The HTML is generated by the `collate.py` script.
19
+
The HTML is generated by the `ad.py` script.
20
20
You can edit it there.
21
21
22
22
Note that the links-to-existing-GitHub-issues in the table are also defined in this script.
@@ -32,20 +32,20 @@ Until then, it has to live here.
32
32
33
33
Under the hood, the main thing that actually runs the AD tests / benchmarks is `main.jl`.
34
34
You can run `julia --project=. main.jl` and it will print some usage information.
35
-
However, it is the Bash script `ad.sh` that controls how this Julia script is called.
35
+
However, it is the Python script `ad.py` that controls how this Julia script is called.
36
36
37
37
Fundamentally, the idea is that we want to loop over every combination of model and adtype and test it.
38
38
However, because GitHub Actions limits jobs to 6 hours, it is impractical to run every combination in the same job.
39
39
What we do is to run one job per model.
40
-
This is accomplished by first storing the names of the models and adtypes in the `$GITHUB_OUTPUT` variable, which can then be read by the next job.
40
+
This is accomplished by first storing the names of the models and adtypes in the `$GITHUB_OUTPUT` variable using `python ad.py setup`, which can then be read by the next job.
41
41
42
-
The next job is a CI matrix split by model name; each of the sub-jobs invokes `ad.sh`, which loops over each adtype and calls the Julia script with the model name and adtype as arguments.
43
-
The purpose of having this Bash -> Julia setup (as opposed to doing the looping inside Julia itself) is to guard against the Julia process crashing, which can happen sporadically with Enzyme.
44
-
If the Julia process successfully finishes, it will print the result which is picked up by the Bash script; if it crashes, we just record the result as 'error'.
42
+
The next job is a CI matrix split by model name; each of the sub-jobs invokes `python ad.py run --model {model_name}`, which loops over each adtype and calls the Julia script with the model name and adtype as arguments.
43
+
The purpose of having this Python -> Julia setup (as opposed to doing the looping inside Julia itself) is to guard against the Julia process crashing, which can happen sporadically with Enzyme.
44
+
If the Julia process successfully finishes, it will print the result which is picked up by the Python script; if it crashes, we just record the result as 'error'.
45
45
46
-
Finally, the results are collated and sent to the final job in the workflow, which is a Python script that uses the results to create the HTML page.
46
+
Finally, the results are collated and sent to the final job in the workflow, which is `python ad.py html`.
47
47
It _could_ be written in Julia (PRs are welcome).
48
-
However, it was much faster for me to prototype in Python.
48
+
However, it was much faster for me to do it in Python.
49
49
(Fun fact: collating these results is also somewhat involved because we can't just write to `$GITHUB_OUTPUT`; it turns out that [output from different jobs in a matrix will override each other](https://github.com/orgs/community/discussions/26639).
50
50
Thankfully, [there is an existing action](https://github.com/beacon-biosignals/matrix-output) which is designed to get around this problem by uploading artefacts.)
51
51
@@ -70,7 +70,7 @@ EOF
70
70
export RESULTS_JSON
71
71
```
72
72
73
-
Then you can run the Python script locally with `python collate.py`.
73
+
Then you can run the Python script locally with `python ad.py html`.
74
74
It will generate the HTML file in the `html` directory.
0 commit comments