You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/content/guides/administrator.md
+27-47Lines changed: 27 additions & 47 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,6 +7,8 @@ layout: subpage
7
7
8
8
# General Information
9
9
10
+
This page gives a brief overview of the underlying system and capabilities. It is meant to be read by people who would like to install the system. If you are provided such an instance please refer to the [Instructor's Guide](/guides/instructor)
11
+
10
12
The Infomark System can serve multiple courses within a single server instance. The linked background workers will then be shared amongst these courses.
11
13
12
14
## Configuration
@@ -18,7 +20,7 @@ The configuration is done within a YAML config file and has the following struct
Only the keys `rabbitmq_*` need to be shared between the server. In general, the defaults might do the job. For all secrets like passwords, tokens or keys we suggest to use `openssl rand -base64 32` to generate random high-quality secrets.
39
+
For an example configuration please refer to the `.infomark.example.yml` your downloaded release archive or inspect the file [on GitHub](https://github.com/cgtuebingen/infomark-backend/blob/master/.infomark.example.yml).
38
40
41
+
Only the keys `rabbitmq_*` need to be shared between the server. In general, the defaults might do the job. For all secrets like passwords, tokens or keys you should use `openssl rand -base64 32` to generate random high-quality secrets.
39
42
40
43
We discuss some important settings:
41
44
@@ -52,38 +55,40 @@ The sessions are handled on the server-side using cookies. Any session will stay
52
55
53
56
### Email
54
57
55
-
For technical reasons, our infrastructure only supports sendmail to deliver emails. If you remove the `sendmail_binary` key from the config, outgoing emails will be instead displayed in the terminal. Additionally, each outgoing email will have a footnote
58
+
For technical reasons, our infrastructure only supports the binary `sendmail` to deliver emails. If you remove the `sendmail_binary` key from the config, all emails from the system will be displayed in the terminal. Additionally, each outgoing email will have a footnote
56
59
57
60
```
58
61
Sent by: FirstName LastName
59
62
sent via Infomark
60
63
```
61
64
62
-
when these are composed by any identity.
65
+
when these are composed by any user.
63
66
64
-
We hard-coded the domain `uni-tuebingen.de` in the front-end such that any registration attempt outside this domain will get a warning. You might want to see the developer guide to change this value.
67
+
We hard-coded the domain `uni-tuebingen.de` in the front-end such that any registration attempt with an email not mathcing this domain will get a warning. You might want to see the [Developer's Guide](/guides/developer) on how to change this value.
65
68
66
69
### Directories
67
70
68
71
We use several directories to store uploads, generated files or common files like a privacy statement. The default values will work if the paths exist. We made these paths configurable as these files require different backup strategies. We suggest estimating beforehand the required space. From our experience 1000 submissions are equal to 20MB. You can limit the size of each submission file in the server settings using the key `max_request_submission_bytes`.
69
72
70
73
### Server-Settings
71
74
72
-
To balance the tradeoff between too many request and responsiveness, we added several strategies to avoid blocking actions. While each request is handled in a light-weight Go-routine we cannot stop these routines Go only has a fork-join thread-model. Instead we limit the number of bytes which are read from the client. The default of 1MB is enough for common JSON requests. The following keys can limit the request sizes:
75
+
To balance the tradeoff between too many request and responsiveness, we added several strategies to avoid blocking actions. We limit the number of bytes which are read from the client during a request. The default of 1MB is enough for common JSON requests. You can configure these limits for each kind of request using the following keys:
76
+
77
+
-`max_request_json_bytes` to limit any JSON request
78
+
-`max_request_avatar_bytes` to limit the image size for the avatar in the profil
79
+
-`max_request_submission_bytes` to limit the size for homework solutions
73
80
74
-
-`max_request_json_bytes`
75
-
-`max_request_avatar_bytes`
76
-
-`max_request_submission_bytes`
81
+
There is no limit when uploading slides or extra course material.
77
82
78
83
### Background-Worker
79
84
80
-
Background-Workers will ignore any queued submissions when `worker_void` is set to `true`. These workers can be turned off by`use_backend_worker: false`.
85
+
Background-Workers can be turned off when using the config`use_backend_worker: false`.
81
86
82
87
## Roles and Permissions
83
88
84
-
InfoMark has a relatively simple permission system. The permissions are linked to a request identity (the user). Each user can be either a normal user or `global admin`, which gives the user all permissions across all hosted courses.
89
+
InfoMark has a relatively simple permission system. The permissions are linked to a request identity (the user). Each user can be either a normal user or `global admin`. Any global admin will bypass all permission checks and gives the user all permissions across all hosted courses.
85
90
86
-
To upgrade/downgrade a user to `global admin`just use the console like
91
+
To upgrade/downgrade a user to `global admin` use the console like
87
92
88
93
```bash
89
94
# upgrade account
@@ -94,13 +99,13 @@ To upgrade/downgrade a user to `global admin` just use the console like
94
99
95
100
Only, a global admin can create new courses.
96
101
102
+
To get granularity in the permission system each user enrolled into a course will be assigned to one the following roles:
97
103
98
-
Further, every time a user enrolls into a course, there are three roles:
99
-
- admin: Admin for a course -- not a global admin
100
-
- tutor: Teaching assistants who will grade homework and lead exercise groups
101
-
- students: This is the default role.
104
+
-*admin*: Admin for a course -- not a global admin
105
+
-*tutor*: Teaching assistants who will grade homework and lead exercise groups
106
+
-*students*: This is the default role.
102
107
103
-
These permissions control which resources a user has access to, e.g. students cannot see other students personal information. Slides and material can be targeted to a specific group, e.g., distribute sample solution to TAs. These roles are subsets, i.e., an admin has all permissions a tutor and a student has. Tutors have additional permissions compared to students.
108
+
These permissions control which resources a user has access to, e.g. students cannot see other students personal information. Slides and material can be targeted to a specific role, e.g., distribute sample solution to TAs. These roles are subsets, i.e., an admin has all permissions, which a tutor and a student has. Tutors have additional permissions compared to students.
104
109
105
110
To upgrade a user to a course-tutor or course-admin use the console
106
111
@@ -117,10 +122,12 @@ Note, this enrolls the user to a course if the user is not enrolled. Otherwise,
117
122
118
123
## Auto-Tests
119
124
125
+
Please refer to the [Tutor's Guide](/guides/tutor) for more details about writing and using auto-tests.
126
+
120
127
Each task of a programming assignment can be linked to a docker-image and a zip file containing the test code.
121
-
Technically, the server process handling acting as a Restful JSON web server will communicate with separate processes, which are called `workers`. These workers can be started at different machines.
128
+
Technically, the server process acting as a Restful JSON web server will communicate using AMQP with separate processes, which are called `workers`. These workers can be started at different machines.
122
129
123
-
>Please note, in the current version these workers have global admin privileges. This will change in the future. Hence, only start these workers on machines you trust!
130
+
>Please note, in the current version these workers have global admin privileges. This will change in the future. Hence, only start these workers on machines you trust (which is anyway a good idea).
124
131
125
132
### Conventions
126
133
@@ -160,34 +167,7 @@ If the docker child returns an exit code != 0 a default message will be sent ins
160
167
161
168
The workers, will download the student submission over HTTP and run these tests locally. Make sure, the used docker file exists or is already pulled from e.g. docker-hub.
162
169
163
-
### Example
164
-
165
-
A basic dockerfile for testing a submission might be
166
-
167
-
```docker
168
-
# Dockerfile
169
-
FROM ubuntu:18.04
170
-
ADD scripts/run.sh /app/run.sh
171
-
ENTRYPOINT ["/app/run.sh"]
172
-
```
173
-
174
-
and script
175
-
176
-
```bash
177
-
# run.sh
178
-
DATA_DIR="/data"
179
-
SUBMISSION_FILE="$DATA_DIR/submission.zip"
180
-
TEST_FILE="$DATA_DIR/unittest.zip"
181
-
182
-
echo"this line will be ignored"
183
-
echo"--- BEGIN --- INFOMARK -- WORKER"
184
-
echo${SUBMISSION_FILE}
185
-
echo${TEST_FILE}
186
-
echo"--- END --- INFOMARK -- WORKER"
187
-
echo"this line will be ignored"
188
-
```
189
-
190
-
The testing-framework, e.g., JUnit has to ensure to suppress any potential output from the uploaded user code.
170
+
We provide [examples](/guides/tutor) for testing Java, Python and C++ programming assignment solutions.
Copy file name to clipboardExpand all lines: docs/content/guides/overview.md
+9-13Lines changed: 9 additions & 13 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,31 +11,27 @@ course management system supporting auto-testing of programming assignments scal
11
11
12
12
Uploaded solutions to programming assignments are tested automatically. TAs can grade these solutions online. The platform supports multiple courses each with multiple exercise groups, slides and course material.
13
13
14
+
For more information about writing such tests see our [Tutor's Guide](/guides/tutor/). On how to use the system please refer to the [Administrator's Guide](/guides/administrator/)
15
+
14
16
# Quick-Start
15
17
16
18
To locally test the system we suggest to run the following commands:
Copy file name to clipboardExpand all lines: docs/content/guides/tutor.md
+63-12Lines changed: 63 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,13 +1,13 @@
1
1
---
2
2
title: "Tutor's Guide"
3
3
date: 2019-04-21
4
-
lastmod: 2019-04-24
4
+
lastmod: 2019-05-13
5
5
layout: subpage
6
6
---
7
7
8
-
# Unit - Tests
8
+
# Auto - Tests
9
9
10
-
Unit-tests consists of
10
+
Auto-tests (unit-tests) consists of
11
11
12
12
* a submission `<STUDENT_UPLOAD.ZIP>` made by a student
13
13
* a testing framework `<TASK_SEPCIFIC_TEST.ZIP>` written by one of the tutors/instructors
@@ -23,13 +23,46 @@ docker run --rm -it --net="none" \
23
23
```
24
24
25
25
and capture the output which is store in the database and displayed the student resp. the tutor who grades the solution.
26
+
The most minimal and simple Dockerfile which handles uploaded solutions is
27
+
28
+
```docker
29
+
# Dockerfile
30
+
FROM ubuntu:18.04
31
+
ADD scripts/run.sh /app/run.sh
32
+
ENTRYPOINT ["/app/run.sh"]
33
+
```
34
+
35
+
which uses the script
36
+
37
+
```bash
38
+
# run.sh
39
+
DATA_DIR="/data"
40
+
SUBMISSION_FILE="$DATA_DIR/submission.zip"
41
+
TEST_FILE="$DATA_DIR/unittest.zip"
42
+
43
+
echo"this line will be ignored"
44
+
echo"--- BEGIN --- INFOMARK -- WORKER"
45
+
echo${SUBMISSION_FILE}
46
+
echo${TEST_FILE}
47
+
echo"--- END --- INFOMARK -- WORKER"
48
+
echo"this line will be ignored"
49
+
```
50
+
51
+
as an entry point. Please either use [one of our pre-defined test-examples](https://github.com/cgtuebingen/infomark/tree/master/unittests) or create your own.
52
+
By design we assume, that
53
+
54
+
* The testing-framework, e.g., JUnit, ensures that all stdout from the uploaded user code is suppressed.
55
+
* A timeout is handled in the tests (InfoMark does not kill any running test)
56
+
57
+
> We currently investigate if we want to add such a timeout-feature.
26
58
27
59
## Overview
28
60
29
61
There are some ways to ease the task of writing unit-tests.
30
62
A clear directory structure and a Makefile to automatically pack all necessary archives or/and run the unit-test locally can dramatically speed up the entire process and avoid debugging steps on the server.
63
+
31
64
The *makefile* should be able to clean temporary files, zip files and simulate the test result locally using the correct docker-image.
32
-
Further, specifying the docker-image in the *makefile* helps to set up the task in InfoMark as you will need to specify it there while creating a new exercise task.
65
+
Further, specifying the docker-image in the *makefile* helps to set up the task in InfoMark as you will need to specify it there while creating a new exercise task. An [example-Makefile](https://github.com/cgtuebingen/infomark/blob/master/unittests/python/makefile) is given in our repository.
33
66
34
67
InfoMark is language-agnostic. The system only records the docker-output. All post-processing of runs (processing JUNIT outputs) must be done *within* the docker container.
35
68
@@ -38,37 +71,37 @@ We provide several testing-templates and examples
38
71
| language | dockerimage (hub.docker.com) | test example | dockerfile |
| C++ |[patwie/test_cpp_submission:latest](https://hub.docker.com/r/patwie/test_cpp_submission)|[yes](https://github.com/cgtuebingen/infomark/tree/master/unittests/cpp)|[yes](https://github.com/cgtuebingen/infomark/tree/master/dockerimages/unittests/cpp)|
43
76
44
77
45
78
## Java 11
46
79
47
80
We suggest using our [docker-image](https://github.com/cgtuebingen/infomark/tree/master/dockerimages/unittests) to run follow the guide below.
48
81
49
-
We suggest keeping the following directory structure
82
+
We use the following directory structure to reduce the workload of writing re-usable unit-tests.
50
83
51
-
```
84
+
```bash
52
85
exercises
53
86
exercise<a>
54
87
makefile
55
-
tasks
88
+
tasks# any LaTeX related content describing the exercise
56
89
sheet.tex
57
-
solution
90
+
solution# sample solution
58
91
main
59
92
FileA.java
60
93
FileB.java
61
-
student_template_[<a>.<b>]
94
+
student_template_[<a>.<b>]# student-template for each task
62
95
main
63
96
FileA.java
64
97
FileB.java
65
-
unittest_public_[<a>.<b>]
98
+
unittest_public_[<a>.<b>]# all tests without output visible to students
66
99
src
67
100
__unittest
68
101
FileAtest.java
69
102
FileBtest.java
70
103
build.xml
71
-
unittest_private[<a>.<b>]
104
+
unittest_private[<a>.<b>]# all tests without output visible only to tutors/TAs
72
105
src
73
106
__unittest
74
107
FileAtest.java
@@ -366,6 +399,23 @@ int divide(int a, int b);
366
399
#endif// LIB_DIVIDE_H_
367
400
```
368
401
402
+
403
+
A simple way of testing C++ implementations uses the header-only library catch
404
+
405
+
```cpp
406
+
#include"lib/divide.h"
407
+
408
+
#defineCATCH_CONFIG_MAIN // This tells Catch to provide a main() - only do
409
+
// this in one cpp file
410
+
#include "catch.hpp"
411
+
412
+
TEST_CASE("Divide should be correct", "[divide]") {
413
+
REQUIRE(divide(6, 3) == 2);
414
+
}
415
+
```
416
+
417
+
> Reminder: InfoMark does not has any timeout when running the tests. We highly recommend to use any mechanism to avoid infinite loops. A potential solution is **timeout** from the **coreutils**.
418
+
369
419
As the implementation of `divide` is not correct the output will be:
370
420
371
421
Alpine clang version 5.0.1 (tags/RELEASE_501/final) (based on LLVM 5.0.1)
@@ -424,3 +474,4 @@ The output would also contain all linking issues (wrongly named function) like
424
474
> Currently, there is no way to skip non-existing methods. If a method does not exists or has the wrong signature
425
475
> the output will containg the linking error without any test-results from the run itself. This is caused by the nature
0 commit comments