Skip to content

Teaching Assistant Handbook

Jump to bottom
Mihai Baruta edited this page Nov 20, 2022 · 19 revisions

1. Introduction

To set up an assignment, you must create a private repository on Gitlab where the submissions will be checked and a public one from where the students can fork their own private repositories. Also, a special block must be added at the course page level on Moodle that will handle the pulling of data from Gitlab.

2. Gitlab Setup

The private and public repositories must have the following structure:

├── checker/
│   ├── checker.sh
│   ├── input/
│   │   ├── test0
│   └── output/
│       ├── test0.ref
├── Dockerfile
├── .gitlab-ci.yml
├── local.sh
└── src/
    └── skel.c

It is recommended to fork https://gitlab.cs.pub.ro/vmchecker/vmchecker-next-assignment for a faster start.

A typical assignment will have the above structure with the following files and folders of interest to teaching assistants:

  • src/ - Here the student will develop their solution according to the assignment's specification. Here, you can add a skeleton for the assignment.
  • local.sh - A utility script used to check the assignment locally. It simulates a Gitlab pipeline.
  • Dockerfile - Used to build the docker image for the Gitlab pipeline. The following is an example that should satisfy most cases:
FROM jokeswar/base-ctl
COPY ./checker ${CHECKER_DATA_DIRECTORY}

NOTE: The base-ctl image is based on the ubuntu:20.04 image and has the following additional packages installed: build-essential and python3.

NOTE: When the image is built the entire checker is copied into the docker image, to prevent the user from changing tests.

  • .gitlab-ci.yml - The definition of the pipeline. You must change the image used to the one you built.
  • checker/ - The checker directory.
  • checker/checker.sh - The entry point of the checker. When execution is passed to this script, the current working directory is set to the checker/ folder.

NOTE: In order for VMChecker Next to be able to grade the assignment it will look for the last string that matches the following regex: Total:\ *([0-9]+) (the grade must be an integer).

Valid format:

              Total:100/100
              Total:    0/100
Total:50/100

Invalid format:

              Total: 55.37/100       # the grade will be set as 55, not 55.37 on Moodle

NOTE: You will need to generate an access token that will allow VMChecker to access Gitlab's API. Also, the project ID for the private repository is needed.

2.1. Disable Merge Requests for the public repository

To avoid the possibility of students accidentally sharing their solution, Merge Requests for the public repository must be disabled. To do that:

  1. Go to your public project page
  2. On the left panel under Settings select General
  3. Under the Visibility, project features, permissions disable Merge Requests.

mr_disable

2.2. Set up a CD pipeline for the checker image

In .gitlab-ci.yml set the image name for the checker step to registry.gitlab.cs.pub.ro/<group-name>/<project-name>:<tag> (e.g: registry.gitlab.cs.pub.ro/ASC/TEMA-3-Public:latest).

2.2.1. Build the checker image trough Gitlab Pipelines

Set up the BUILD_DOCKER_IMAGE pipeline environment variable to true for your public repository. Each time a new commit is made to the repository a new Docker image build will be triggered. The image will be uploaded to Gitlab's own Docker registry.

2.2.2. Build the checker image manually

./local.sh docker build
./local.sh docker push --user "$GITLAB_USERNAME" --token "$GITLAB_ACCESS_TOKEN"

To create a GITLAB_ACCESS_TOKEN access token follow the documentation on Gitlab.

For more information on local.sh run ./local.sh -h.

2.3. Find the project ID

  1. Go to your project page
  2. The ID is found under the project name

project_id

3. Moodle Setup

First, set up a Moodle assignment with the following additional settings:

  • Submission types → Submission types → File submissions
  • Submission types → Maximum number of uploaded files → 1
  • Submission types → Maximum submission size → 50kb (The maximum submission size of VMChecker Next is 10 MB)
  • Submission types → Accepted file types → zip
  • Feedback types → Offline grading worksheet → true (enables feedback download)

Next, add a new VMChecker block on the current page by clicking on Add a block and selecting VMChecker block from the list.

add_block

Click on the gear and select Configure VMChecker block. Here you must fill in a couple of options for the VMChecker settings header:

  • Enable autograding of submissions - If you want the assignment to be automatically graded with the mark from the checker
  • Gitlab repository ID - The project ID of your private repository
  • Gitlab private token - The previously generated private access token
  • Assignment - From the list select the assignment you want VMChecker to manage

WARNING: There is a one-to-one relationship between an assignment and a VMChecker block. Make sure there is only one VMChecker block per assignment.

Save!

4. Test your assignment

After the setup is done, switch to the student test account and upload your submission. After a couple of minutes, your submission will run on the Gitlab CI. You can check your private repository. When the pipeline is finished the trace will be set as feedback for your assignment. If autograding is enabled, your submission will also be marked.

NOTE: On Moodle, the feeback string will be computed from the trace found on Gitlab pipeline, and will be the content between the start of the checker string mark (i.e. VMCHECKER_TRACE_CLEANUP) and the string that gardes the assignment.

<Pipeline output>
VMCHECKER_TRACE_CLEANUP
Test 1....0              ---|
.                           |
.                           |
.                           |   This will be the feedback for the student assignemnt.
Test n...0                  |
                            |
    Total 0/100          ---|
<some more Pipeline output>

5. Grading an assignment

5.1. Download all submissions

  1. Go to the course page on Moodle
  2. Click on the assignment
  3. Click on the gear and from the drop-down select Download all submissions

download_all

NOTE: If you do not have permissions, the assignment page will show the upload form only.

5.2. Download grading worksheet

  1. Go to the course page on Moodle
  2. Click on the assignment
  3. Click on View all submissions
  4. From grading action select Download grading worksheet

download_worksheet

NOTE: You can grade submissions and give feedback in the downloaded CSV.

5.3. Check a submission on Gitlab

To check the pipeline of a certain student, go to the assignment's private repository on Gitlab. From the menu on the left select Repository → Branches. Every student submission has a separate branch of the following format ${username}-${year}-${month}-${day}-${hour}-${minute}-${second}-${uuid}. Using the search bar, look for the desired submission.

branches

To check the user code click on the branch name and check the src folder.

If you want to see the pipeline output, click on the pipeline status icon.

pipeline_select

Next, click on Jobs and select the first job ID.

jobs

5.4. Import grades

  1. Go to the course page on Moodle
  2. Click on the assignment
  3. Click on View all submissions
  4. From grading action select Upload grading worksheet

upload_worksheet

  1. Upload the worksheet downloaded previously
  2. After you preview the changes click Confirm

6. VMChecker block

6.1. Recheck submission for ...

In the VMChecker block for the desired assignment select Recheck submission for ..., as action. Search in the list for the student, you can use the search bar for a faster lookup. Finally press Run action.

NOTE: Only students that have submitted their assignments will be found on the list.

recheck_for

6.2. Recheck all

In the VMChecker block for the desired assignment select Recheck all submissions, as action, and press Run action. All students that have submitted their homework will have their last submission rechecked.

recheck_all

Clone this wiki locally