You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Nov 26, 2025. It is now read-only.
Copy file name to clipboardExpand all lines: README.md
+86-1Lines changed: 86 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -142,4 +142,89 @@ Note: Arguments like `NET`, `BLK`, and `GRAPHIC` enable devices in QEMU, which t
142
142
143
143
## Test for oscomp testcases
144
144
145
-
We can run [testcases of the OS competition](https://github.com/oscomp/testsuits-for-oskernel/tree/pre-2025) with StarryOS. Guidence can be found in [Starry-Tutorial](https://azure-stars.github.io/Starry-Tutorial-Book/ch01-04.html#在-os-比赛上测试-starry).
145
+
We can run [testcases of the OS competition](https://github.com/oscomp/testsuits-for-oskernel/tree/pre-2025) with StarryOS. Guidence can be found in [Starry-Tutorial](https://azure-stars.github.io/Starry-Tutorial-Book/ch03-02.html).
146
+
147
+
148
+
And you can run the testcases with the following commands:
149
+
150
+
```bash
151
+
# Clone the base repository
152
+
./scripts/get_deps.sh
153
+
154
+
# run the testcases of oscomp on x86_64
155
+
$ make oscomp_run ARCH=x86_64 # If it reports an error: -accel kvm: failed to initialize kvm: Permission denied, please add `ACCEL=n` argument.
156
+
157
+
# run the testcases of oscomp on riscv64
158
+
$ make oscomp_run ARCH=riscv64
159
+
160
+
# run the testcases of oscomp on aarch64
161
+
$ make oscomp_run ARCH=aarch64
162
+
163
+
# run the testcases of oscomp on loongarch64
164
+
$ make oscomp_run ARCH=loongarch64
165
+
```
166
+
167
+
To run more testcases from oscomp, you can refer to the [oscomp README](./apps/oscomp/README.md).
168
+
169
+
## How to add new testcases
170
+
171
+
To allow the kernel to run user-written test cases, temporary test cases can be created. The specific steps are as follows:
172
+
173
+
1. Create a temporary test case folder
174
+
175
+
```sh
176
+
cd apps && mkdir custom &&cd custom
177
+
```
178
+
179
+
2. Create a `Makefile`in the current directory and fill in the following content:
180
+
181
+
```makefile
182
+
all: build
183
+
184
+
build:
185
+
186
+
clean:
187
+
rm -rf *.out
188
+
```
189
+
190
+
The reason for creating an empty build `Makefile` is that when Starry packages testcase images, it will first execute the test case's build program by default. However, since our temporary test case does not currently have a defined build program, `make all` does not need to perform any operations.
191
+
192
+
3. Copy your compiled executable file into the current directory.
193
+
194
+
Let's take a `hello_world` example. Create a `hello.c` file in the current directory and write the following content:
195
+
196
+
```c
197
+
#include "syscall.h"
198
+
199
+
int main()
200
+
{
201
+
char msg[] = "Hello, World!\n";
202
+
write(1, msg, sizeof(msg));
203
+
return 0;
204
+
}
205
+
```
206
+
207
+
Then, run the following command to compile it:
208
+
209
+
```sh
210
+
x86_64-linux-musl-gcc -static hello.c -o hello
211
+
```
212
+
213
+
This generates an x86_64 executable file in the current folder. Of course, you can also complete the compilation process elsewhere and directly copy the executable file into the current directory.
214
+
215
+
4. Create a `testcase_list` file in the current directory and add the relative path of the executable file that needs to be executed. Note that this path should be relative to `apps/custom` (i.e., the current directory). In our example, the content should be:
216
+
217
+
```sh
218
+
hello
219
+
```
220
+
221
+
5. Return to the project root directory and run the following command:
222
+
223
+
```sh
224
+
sudo ./build_img.sh -fs ext4 -file apps/custom
225
+
cp disk.img .arceos/disk.img
226
+
make defconfig
227
+
make AX_TESTCASE=custom EXTRA_CONFIG=../configs/x86_64.toml ARCH=x86_64 BLK=y NET=y FEATURES=fp_simd,lwext4_rs LOG=off ACCEL=n run
228
+
```
229
+
230
+
This completes the execution of the custom test case.
Then Starry will find `musl/basic/brk` from the sdcard images.And the working directory in the sdcard is default `/`.
15
15
16
16
## Judge scripts
17
+
17
18
Files like `judge_**.py` are the scripts that judge whether the output of the kernel is correct or not. The format ot the scripts can be seen at [judge](https://github.com/Azure-stars/oskernel-testsuits-cooperation/tree/master/judge).
18
19
19
20
When run `make oscomp_test` in the root directory, the testcases in the `testcase_list` will be run and the result will be shown in the terminal. And these scripts will be run to judge whether the output is correct or not.
20
21
21
22
## How to add new testcases
22
23
23
-
1. Enable testcases in the `judge_**.py` script. You can find `TODO` fields in the script and add the new testcases in the corresponding place. The format of the command can be seen in [judge](https://github.com/Azure-stars/oskernel-testsuits-cooperation/tree/master/judge).
24
+
### Add basic testcases
25
+
26
+
If we want to add `write` testcase, then we need:
27
+
28
+
1. Enable testcases in the [judge_basic.py](./judge_basic.py)。An example is that if the `target_testcase` in [judge_basic.py](./judge_basic.py) is below:
29
+
30
+
```sh
31
+
target_testcases = [
32
+
"test_brk",
33
+
"test_chdir",
34
+
"test_execve",
35
+
"test_pipe",
36
+
]
37
+
```
38
+
39
+
Then we need to enable `write` testcase by rewriting it as
40
+
41
+
```sh
42
+
target_testcases = [
43
+
"test_brk",
44
+
"test_chdir",
45
+
"test_execve",
46
+
"test_pipe",
47
+
"test_write",
48
+
]
49
+
```
50
+
51
+
2. Add the testcase in the [oscomp_test.sh](../../scripts/oscomp_test.sh)。An example is that if the `basic_testlist` in [oscomp_test.sh](../../scripts/oscomp_test.sh) is below:
52
+
53
+
```sh
54
+
# TODO: add more basic testcases
55
+
basic_testlist=(
56
+
"/$LIBC/basic/brk"
57
+
"/$LIBC/basic/chdir"
58
+
"/$LIBC/basic/clone"# add the new testcase here
59
+
)
60
+
```
61
+
62
+
Then we need to enable `write` testcase by rewriting it as
63
+
64
+
```sh
65
+
# TODO: add more basic testcases
66
+
basic_testlist=(
67
+
"/$LIBC/basic/brk"
68
+
"/$LIBC/basic/chdir"
69
+
"/$LIBC/basic/clone"
70
+
"/$LIBC/basic/write"
71
+
)
72
+
```
73
+
74
+
### Add libc testcases
75
+
76
+
The standard of the [judge_libctest.py](./judge_libctest.py) can be seen at [judge-std](https://github.com/Azure-stars/oskernel-testsuits-cooperation/tree/master/judge/judge_libctest.py).
77
+
78
+
If we want to add `entry-static.exe basename` testcase, and the `libctest_baseline` in the [judge_libctest.py](./judge_libctest.py) is as below:
79
+
80
+
```python
81
+
libctest_baseline ="""
82
+
========== START entry-static.exe argv ==========
83
+
Pass!
84
+
========== END entry-static.exe argv ==========
85
+
"""
86
+
```
87
+
88
+
Then we need to enable `entry-static.exe basename` testcase by rewriting it as
========== END entry-static.exe basename ==========
98
+
"""
99
+
```
100
+
101
+
That is, we **append the instructions or results used in the evaluation** to the script.
102
+
103
+
### Add busybox testcases
104
+
105
+
The standard of the [judge_busybox.py](./judge_busybox.py) can be seen at [judge-std](https://github.com/Azure-stars/oskernel-testsuits-cooperation/tree/master/judge/judge_busybox.py).
106
+
107
+
If we want to add `echo "#### independent command test"` testcase, and the `busybox_baseline` in the [judge_busybox.py](./judge_busybox.py) is as below:
108
+
109
+
```python
110
+
cmd =""""""
111
+
```
112
+
113
+
Then we need to enable `echo "#### independent command test"` testcase by rewriting it as
114
+
115
+
```python
116
+
cmd ="""
117
+
echo "#### independent command test"
118
+
"""
119
+
```
120
+
121
+
### Add lua testcases
122
+
123
+
The standard of the [judge_lua.py](./judge_lua.py) can be seen at [judge-std](https://github.com/Azure-stars/oskernel-testsuits-cooperation/tree/master/judge/judge_lua.py).
124
+
125
+
If we want to add `max_min.lua` testcase, and the `cmd` in the [judge_lua.py](./judge_lua.py) is as below:
126
+
127
+
```python
128
+
cmds =""""""
129
+
```
130
+
131
+
Then we need to enable `max_min.lua` testcase by rewriting it as
132
+
133
+
```python
134
+
cmds ="""
135
+
max_min.lua
136
+
"""
137
+
```
138
+
139
+
140
+
### About iozone testcases
24
141
25
-
2. If the testcase belongs to `basic`, you need to add it in the `./scripts/oscomp_test.sh`. You can find `TODO` fields in the script and add the new testcases in the corresponding place. An example to add `clone` testcases are shown below:
142
+
IOZONE is a special testcase, and the standard of the [judge_iozone.py](./judge_iozone.py)can be seen at [judge-std](https://github.com/Azure-stars/oskernel-testsuits-cooperation/tree/master/judge/judge_iozone.py).It measures both correctness and performance.
26
143
27
-
```shell
28
-
# TODO: add more basic testcases
29
-
basic_testlist=(
30
-
"/$LIBC/basic/brk"
31
-
"/$LIBC/basic/chdir"
32
-
"/$LIBC/basic/clone"# add the new testcase here
33
-
)
34
-
```
144
+
So we will directly enable all its subtest cases by rewriting `iozone_baseline` in the [judge_iozone.py](./judge_iozone.py) to that in the [judge-std](https://github.com/Azure-stars/oskernel-testsuits-cooperation/tree/master/judge/judge_iozone.py).
0 commit comments