This repository contains Docker and other configuration files needed to run and deploy the autograder system.
- Aug 26, 2024: Release 2024.08.v0 is out. See https://github.com/orgs/eecs-autograder/projects/2/views/1 for a list of addressed issues.
- Changes to the "Upgrading (Production Deployments)" section of this document regarding upgrade requirements.
- autograder-server and ag-website-vue submodules now use calendar versioning.
Documentation on how to configure projects, test cases, and more through the web interface can be found at https://eecs-autograder.github.io/autograder.io/
We continue to use calendar versioning, but we remove the v from the version number in previous versions:
{yyyy}.{mm}.{minor version}.{pre-release modifier}
We generally follow Python conventions
E.g., our next major release at time of writing will be 2025.08.0.
ag-client-typescript now also uses calendar versioning.
As of Jan. 2021, we use the following version scheme for release tags in this repo (autograder-full-stack):
{yyyy}.{mm}.v{X}
{yyyy}is the year of the release (e.g. "2021").{mm}is the month of the release (e.g. 01, 06, 08, 09 for Jan, June, Aug, or Sept).{X}is the minor version number, incremented for smaller changes (patches, bug fixes) between major releases.
(Written on Aug. 23, 2024): Starting with our next release, we will start using this calendar versioning scheme for the autograder-server and ag-website-vue sub-repositories. Note that since npm doesn't allow the "v" in the minor version portion, we will omit it in the package.json file in that repository. We will also omit the "v" in autograder-server for symmetry. These version labels will be synchronized (i.e., the submodules will have the same version as this repo's release tags) to make it easier to verify that the deployed repos are in sync. ag-client-typescript will continue to use semantic versioning.
See this tutorial.
See this tutorial.
See this tutorial.
We typically only release updates to the latest calendar version (e.g., 2025.08.x).
In the case of a critical issue, we may decide to backport certain updates to prior versions.
To upgrade to the latest release, replace {version} below with "latest".
To upgrade to a specific release, run the following:
git fetch origin
git fetch --tags
git checkout --recurse-submodules {version}
git submodule update --remote --recursive
Check that each submodule is pointing at the commit labelled with that version:
git submodule foreach --recursive git log --max-count 1
If the above command only prints log messages for two submodules, run the following:
git submodule update --init --recursive
git submodule update --remote --recursive
git submodule foreach --recursive git log --max-count 1
To upgrade from one version to the next, follow these steps:
-
Pull the
masterbranch in theautograder-full-stackrepo and pull the latest tags.cd autograder-full-stack git checkout master git pull git pull --tags -
Checkout the tag for the version you want to upgrade to.
# Replace {tag} with the appropriate version (e.g. 2021.01.v3) git checkout {tag}Legacy Note: If your deployment is behind by several major versions (the {yyyy}.{mm} portion of the version number), we recommend upgrading to each intermediate major version sequentially. This is important to ensure that database migrations run correctly.
Updated Note: Starting with release 2024.08.v0, we will generally try to make it possible to upgrade from 2024.08.v0 straight to the newest version without upgrading to each intermediate version. If a release requires upgrading from a specific previous version, we will note this requirement in the announcements section of this document.
-
Update the
autograder-serverandag-website-vuesubmodules. If you've made changes to the submodules (such as changing config settings), you may need to stash them first and then re-apply. Git will warn you if you need to do so.git submodule update --remote -
Re-deploy the docker containers and apply database migrations. This step varies slightly depending on which deployment strategy you're using (e.g., single server, swarm). Please refer to the appropriate tutorial for the specific commands: swarm, single server.
IMPORTANT: As of 2025.08.0, postgres 14 or later is required.
The postgres version is no longer hard-coded in the docker compose files.
Instead, we read it from the file autograder-full-stack/postgres_version
and pass the value as a docker build arg.
See the steps below for details.
Refer to the Postgres documentation for different possible upgrade approaches.
Here we outline the steps needed to do a dump and restore upgrade.
These steps assume we're upgrading to Postgres 14.
Replace "14" with the version of Postgres you want.
Replace ${postgres_container} with the name of the postgres container on your deployment.
For single-server deployments, the postgres container name will be ag-postgres.
Swarm deployments will need to find the container name with docker ps.
We recommend setting variables with these values to avoid mistakes.
- Schedule maintenance downtime.
- At the beginning of your maintenance window, stop or pause the django and nginx containers:
# Use "pause" for swarm deployments. "stop" will work for single-server docker pause ${django_container} ${nginx_container} - Make sure there are no submissions being graded. You may also choose to stop/pause your grading worker containers.
- Perform a full system backup.
- Dump the current database contents:
Store the
docker exec -it ${postgres_container} pg_dump --username=postgres --format=c postgres -f /db_backup docker cp ${postgres_container}:/db_backup .db_backupfile somewhere safe. - In your docker compose file, add a new postgres volume for the new version.
The example below uses Postgres 14 as an example version.
... volumes: ... # old version, keep it as-is pgdata: {} # ADD THIS LINE. Replace "14" with the version of postgres you want to upgrade to pgdata14: {} ...
- (
2025.08.0and later). Update the version of postgres inautograder-full-stack/postgres_version.echo 14 > postgres_version- (
2024.08.0and earlier). In your docker compose file, edit the postgres service to point at the new volume and to use the desired version.# Replace "14" with the version you want to upgrade to. postgres: # Don't change other settings ... # CHANGE THIS to the version you want image: postgres:14 volumes: # VERY IMPORTANT: Change "pgdata" on the left side of the colon to the # name of the volume you created in the previous step. - pgdata14:/var/lib/postgresql/data/ ... # Settings copied from the above service block
- (
- Rebulid and re-up the updated postgres service. DO NOT apply django migrations.
On a single-server deployment, the following commands will do so:
On swarm deployment, rebuild postgres with the following command, then redeploy the stack:
AG_POSTGRES_VERSION=$(cat postgres_version) docker compose -f docker-compose-single.yml build postgres AG_POSTGRES_VERSION=$(cat postgres_version) docker compose -f docker-compose-single.yml up -d postgresAG_POSTGRES_VERSION=$(cat postgres_version) docker compose build docker compose push docker stack deploy -c docker-compose.yml ag-stack - Restore the dumped contents into the updated postgres service. Note for swarm deployments: the name of the postgres container will have changed.
docker cp db_backup ${postgres_container}:/
docker exec -i ${postgres_container} pg_restore --username=postgres --format=c -d postgres /db_backup
Below are snapshots of what the changes to docker-compose-single.yml might look like during this process.
Start (your file may have some differences):
postgres:
container_name: ag-postgres
restart: unless-stopped
image: postgres:9.5
volumes:
- pgdata:/var/lib/postgresql/data/
environment:
POSTGRES_PASSWORD: 'redacted'
...
volumes:
redisdata: {}
pgdata: {}
rabbitmqdata: {}
sandbox_image_registry_data: {}New volume:
postgres:
container_name: ag-postgres
restart: unless-stopped
image: postgres:9.5
volumes:
- pgdata:/var/lib/postgresql/data/
environment:
POSTGRES_PASSWORD: 'redacted'
...
volumes:
redisdata: {}
pgdata: {}
pgdata14: {} # NEW
rabbitmqdata: {}
sandbox_image_registry_data: {}Update service:
postgres: # CHANGE
container_name: ag-postgres # DO NOT CHANGE
restart: unless-stopped
image: postgres:14 # CHANGE
volumes:
- pgdata14:/var/lib/postgresql/data/
environment:
POSTGRES_PASSWORD: 'redacted'
...
volumes:
redisdata: {}
pgdata: {}
pgdata14: {} # NEW
rabbitmqdata: {}
sandbox_image_registry_data: {}If you want to automate a task using a scripting language, you can use the Python HTTP client found at https://github.com/eecs-autograder/autograder-contrib and some ready-to-use python scripts at https://gitlab.eecs.umich.edu/akamil/autograder-tools.
- IMPORTANT: Make sure to use the correct URL for your deployment. In the former set of scripts, this is configurable with command-line arguments. In the latter, you may need to modify the source code.
Run the following in a django shell (docker exec -it ag-django python3 manage.py shell):
from django.contrib.auth.models import User, Permission
# UPDATE the email address
user = User.objects.get(username='@umich.edu')
user.user_permissions.add(Permission.objects.get(codename='create_course'))
This issue should be fixed as of 2024.08.0.
Notes on how to resolve are below just in case.
Occasionally a new project won't be correctly registered with the grading workers, and so submissions won't get past "queued" status. To manually register the project, run the following in a django shell:
from autograder.grading_tasks.tasks import register_project_queues
# UPDATE the project_pks list. The project primary key can be found in the url when viewing the project on the website.
register_project_queues(project_pks=[339])
This is now possible to resolve through the UI. See https://eecs-autograder.github.io/autograder.io/how_tos.html#rerunning-a-stuck-or-errored-submission
To create a custom api token, run the following in a django shell:
from django.contrib.auth.models import User
from rest_framework.authtoken.models import Token
# Replace <username> with whatever you want.
user = User.objects.create(username='<username>@autograder.io')
token, created = Token.objects.get_or_create(user=user)
# If that username is taken, try another one.
assert created
# Securely share this token with whomever needs it.
print(token)
Add the username (<username>@autograder.io) to the appropriate roster for your course to give the token user the permissions it needs.
This section is intended for developers.
As of 2025.08.0: Each release now has its own branch, and the master branch will no longer point to the latest version.
Instead, the latest release will have the tag latest.
Either checkout the latest tag or the release branch for the release you want to use.
Giving each release its own branch will make it easier for us to apply patches to multiple release versions when needed.
Use feature branches for all changes, and make a pull request against the develop branch.
This repo has two submodules: autograder-server and ag-website-vue.
The develop branch is for changes based on the develop branch of the submodules.
Update the submodules' develop branches when preparing a release or when starting work on a feature that depends on new autograder-server or ag-website-vue commits.
Use the following steps on your feature branch:
# Fetch latest submodule commits
git submodule update --remote
# git status should show new commits in the submodule
git status
git add autograder-server ag-client-typescript
git commit -m "Update submodules"
Name release branches as release-YYYY.MM.x, replacing YYYY with the full year and MM with the zero padded month (e.g., release-2024.08.x).
IMPORTANT: When you create a release branch, update the branch field in the autograder-server and ag-client-typescript entries of .gitmodules to point to the corresponding release branch in the submodules.
Then run git submodule update --remote and commit the changes.
Do NOT merge or rebase directly between the develop and release branches.
Once a release branch is created, it should only be updated with bugfix- or (rarely) feature-style branches.
Squash-and-merge for this type of PR.
After the squashed branch is merged into a release branch, cherry-pick the squashed commit on top of develop and open a pull request to merge the changes into develop.
The version of README.md (this file) on the develop branch is the source of truth.
Update this file on release branches just before publishing a release.
If instructions differ across releases, include both, and label which version the instructions apply to.
Publishing a new release is a multi-step process.
Run ./dev_scripts/start_release.sh {version} and follow the instructions printed.