feat: Build endpoints for admin and setup WSGI (#9)

- Endpoint for banning and unbanning
- Proper WSGI setup in docker compose to actually serve files
- Map deletion endpoint
- Update readme for the changes to docker-compose
- Improve docs for how static and media file serving works with django.
- Better container names
- Make the upload tests work on OSs that recognize map files as `ini` instead of just `text`.
- Beginnings of a make file
- Update `tmp_media_root` to stop overwriting the tmp folder path when `MEDIA_ROOT` is an absolute path
- Emulate file serving in pytest so that nginx doesn't need to run to run the test suite
- Update black and the line-length to match github
- Fix ci for changed env vars

Author: alexlambson

.gitignore

@@ -163,3 +163,5 @@ cython_debug/
 data/
 
 test.bmp
+# Photoshop files
+image-source/

.pre-commit-config.yaml

@@ -1,11 +1,11 @@
 repos:
   -   repo: https://github.com/pre-commit/pre-commit-hooks
-      rev: v2.3.0
+      rev: v5.0.0
       hooks:
         -   id: check-yaml
         -   id: end-of-file-fixer
         -   id: trailing-whitespace
   -   repo: https://github.com/psf/black
-      rev: 22.6.0
+      rev: 25.1.0
       hooks:
         -   id: black

Dockerfile

@@ -10,13 +10,13 @@ COPY .env /cncnet-map-api/
 # Copy files needed for build
 COPY requirements.txt /cncnet-map-api
 COPY requirements-dev.txt /cncnet-map-api
-COPY start.sh /cncnet-map-api
+COPY web_entry_point.sh /cncnet-map-api
 
 RUN apt-get update && apt-get install -y liblzo2-dev  # Compression library used by westwood.
 RUN apt-get install libmagic1  # File type checking.
 RUN pip install --upgrade pip
 # The cflags are needed to build the lzo library on Apple silicon.
 RUN CFLAGS=-I$(brew --prefix)/include LDFLAGS=-L$(brew --prefix)/lib pip install -r ./requirements-dev.txt
 
-RUN chmod +x /cncnet-map-api/start.sh
-ENTRYPOINT "/cncnet-map-api/start.sh"
+RUN chmod +x /cncnet-map-api/web_entry_point.sh
+ENTRYPOINT "/cncnet-map-api/web_entry_point.sh"

Makefile

@@ -0,0 +1,13 @@
+#desolate:
+#    POSTGRES_DATA_DIR = $(shell cat .env | grep POSTGRES_DATA_DIR)
+
+serve:
+	docker compose build django
+	docker compose up nginx-server
+
+stop:
+	docker compose stop
+
+test:
+	docker compose build django
+	docker compose run test

README.md

@@ -17,27 +17,32 @@ The mascot for the backend API is Kirovy, by [Direct & Dominate](https://www.you
 
 ## Frontend devs
 
-Just set up your environment file and run the full docker compose.
+Just set up your environment file and run `docker compose up web -d`.
+
+This will launch the database, run the migrations, and start the django web server.
 
 [Example env file](example.env)
 
 ## Backend devs
 
-You can use the docker files if you'd like, but Django + docker is known to have issue attaching
-to debuggers and hitting breakpoints, so here are the native OS instructions.
+You can use docker compose if you'd like, but here are the native OS instructions.
+
+### Linux and Mac
 
 1. Download and install [pyenv](https://github.com/pyenv/pyenv)
+   > You don't have to use `pyenv` but it makes life much easier when dealing with virtual environments.
 2. Install [PostgreSQL](https://www.postgresql.org/) for your system. This is required for Django
    - On Mac you can do `brew install postgresql` if you have brew installed.
 3. Install LibMagic for [Python Magic](https://github.com/ahupp/python-magic)
-   - On Mac you can do `brew install libmagic` if you have breq installed.
+   - On Mac you can do `brew install libmagic` if you have brew installed.
+   > LibMagic is used for checking file types.
 4. Checkout the repository
 5. Switch to the repository directory
 6. Setup Python
    - Install Python 3.12 `pyenv install 3.12` or whatever the latest python is.
    - Setup the virtual environments `pyenv virtualenv 3.12 cncnet-map-api`
-   - Set the virtual enviornment for the directory `pyenv local cncnet-map-api`
-7. Setup requirements `pip install -r requirements-dev.txt`
+   - Set the virtual environment for the directory `pyenv local cncnet-map-api`
+7. Install the dev requirements `pip install -r requirements-dev.txt`
    -  On Apple Silicon you'll need to install lzo with `brew install lzo` then run
       `CFLAGS=-I$(brew --prefix)/include LDFLAGS=-L$(brew --prefix)/lib pip install -r requirements-dev.txt`
       to get `python-lzo` to install. You shouldn't need to include those flags again unless `python-lzo` updates.
@@ -49,23 +54,37 @@ to debuggers and hitting breakpoints, so here are the native OS instructions.
    - If the app doesn't run due to a missing required variable, add said variable to `example.env` because the person
    who made the variable forgot to do so.
 10. Run the `db` service in `docker-compose`
-11. Load your `.env` file into your shell, (you can use `source load_env.sh && read_env`)
+11. Load your `.env` file into your shell, (you can use `source load_env.sh && read_env`)<a name="load-shell-env"></a>
 then migrate the database `./manage.py migrate`
-12. `./manage.py runserver`
+12. Run the django server with `./manage.py runserver`
+
+Tests can be run by following [these instructions](#running-tests-backend-devs)
+
+
+### Windows
+
+Chairman Bing of the Massivesoft corporation strikes again; getting the `LZO` libraries running
+natively on Windows is a... less-than-pleasant effort. So use docker instead.
+
+1. Install docker for windows. I have had success with [Rancher Desktop](https://rancherdesktop.io/)
+   or [Docker Desktop](https://docs.docker.com/desktop/setup/install/windows-install/)
+2. After docker is running, switch to your local git repo and run `docker compose up windows-dev -d`.
+   Make sure the build succeeds.
+3. Set `windows-dev` as your python interpreter for whichever editor you use.
 
-You can technically use PyCharm to launch everything via `docker-compose`, but there is some
-weird issue with breakpoints not triggering.
+> [!TIP]
+> In Pycharm you go to `Settings > Project > Python Interpreter > Add Interpreter > Docker Compose`
 
 
-## Running tests
+## Running tests (backend devs)
 
-I **strongly** recommend using PyCharm and the `.env` plugin for running the PyTests.
+I strongly recommend using PyCharm (or any other Python IDE with breakpoints) and the `.env` plugin for running the PyTests.
 All you need to do is run the database from `docker-compose`, then launch the tests via PyCharm.
 
 **If you want to run the tests via CLI:**
 
 - Make sure your database is running from the docker compose file. `docker-compose start db`
-- Make sure your environment variables are setup and loaded to your shell. See [backend dev setup](#backend-devs)
+- Make sure your environment variables are setup and loaded to your shell. See [backend dev setup](#load-shell-env)
 - Run `DJANGO_SETTINGS_MODULE="kirovy.settings.testing" pytest tests`
 
 Django should automatically run migrations as part of the test startup.

ci.env

@@ -3,8 +3,8 @@ POSTGRES_USER=kane
 POSTGRES_PASSWORD=cidevelopmentpasswordtechnologyofpeace
 POSTGRES_PORT=5432
 POSTGRES_DATA_DIR=/data/db/
-MEDIA_ROOT=/data/cnc_net_files/
-STATIC_ROOT=/data/cnc_net_static/
+HOST_MEDIA_ROOT=/data/cnc_net_files/
+HOST_STATIC_ROOT=/data/cnc_net_static/
 POSTGRES_TEST_HOST=db
 SECRET_KEY=";thetechnologyofpeaceforcidevwork6352722!@#$$#@"
 RUN_ENVIRONMENT=ci

docker-compose.yml

@@ -1,5 +1,3 @@
-version: "3.9"
-
 services:
   db:
     image: postgres
@@ -14,18 +12,35 @@ services:
     ports:
       - "127.0.0.1:${POSTGRES_PORT}:${POSTGRES_PORT}"
     command: -p ${POSTGRES_PORT}
-  web:
+
+  django:
+    # Won't serve files on its own. Launch nginx-server to run the whoe app.
     build: .
     volumes:
       - .:/cncnet-map-api
-      - ${MEDIA_ROOT}:/data/cncnet_files
-      - ${STATIC_ROOT}:/data/cncnet_static
+      - ${HOST_MEDIA_ROOT}:/data/cncnet_silo  # django will save user-uploaded files here. MEDIA_ROOT
+      - ${HOST_STATIC_ROOT}:/data/cncnet_static  # django will gather static files here. STATIC_ROOT
     ports:
       - "8000:8000"
     env_file:
       - .env
     depends_on:
       - db
+
+  nginx-server:
+    # This is the prod server service.
+    # `docker compose up nginx -d` will run the whole app.
+    # nginx proxies requests to django via gunicorn.
+    image: nginx:latest
+    volumes:
+      - ./nginx.conf:/etc/nginx/nginx.conf:ro
+      - ${HOST_STATIC_ROOT}:/usr/share/nginx/html/static  # website static assets.
+      - ${HOST_MEDIA_ROOT}:/usr/share/nginx/html/silo  # website user-uploaded files.
+    ports:
+      - "80:80"
+    depends_on:
+      - django
+
   test:
     build:
       context: ./
@@ -40,3 +55,18 @@ services:
       - db
     command:
       - pytest
+
+  windows-dev:
+    # Use this for windows. The LZO libraries are annoying to deal with without using docker.
+    # Chairman Bing of Massivesoft strikes again.
+    build:
+      context: ./
+      dockerfile: test.DockerFile
+    volumes:
+      - .:/cncnet-map-api
+    env_file:
+      - .env
+    environment:
+      POSTGRES_TEST_HOST: db  # Necessary to connect to docker db. Overrides the .env setting.
+    depends_on:
+      - db

example.env

@@ -19,12 +19,18 @@ POSTGRES_TEST_HOST=localhost
 # Port for postgres
 POSTGRES_PORT=5432
 
-# Required. The location where the app data (maps and such) will be saved. The docker volume mounts here
-MEDIA_ROOT=./data/cncnet_files/
-
-# Required. The location where static files will be served from.
-# Make sure to configure the webserver to serve files from here and run ``./manage.py collectstatic``
-STATIC_ROOT=./data/cncnet_static/
+# Required. The location where the app data (maps and such) will be saved on the docker host.
+# The docker volume mounts here.
+# You should have this backed up because it's where user-uploaded files will live.
+# nginx and django will serve these files under the url ``/silo/``
+HOST_MEDIA_ROOT=./data/cncnet_silo/
+
+# Required. The directory where collected static files will be saved on the docker host.
+# The docker volume mounts here.
+# ``collectstatic`` will gather all static assets here to be served by nginx.
+# Remember to run ``./manage.py collectstatic`` during build so that this directory will be populated.
+# nginx and django will serve these files under the url ``/static/``
+HOST_STATIC_ROOT=./data/cncnet_static/
 
 # Django debug, never enable on prod
 DEBUG=0

kirovy/exceptions/__init__.py

@@ -1,6 +1,7 @@
 """
 All exceptions for our app belong in this package.
 """
+
 from django.core.exceptions import *  # Import django exceptions for use elsewhere.
 from typing import Optional
 from django.utils.translation import gettext_lazy as _
@@ -74,3 +75,12 @@ def __init__(
         code: Optional[int] = None,
     ):
         super().__init__(game_name_or_slug, detail, code)
+
+
+class BanException(Exception):
+    """Raised when there is an issue during the ban process.
+
+    ``str(e)`` will be returned to the UI.
+    """
+
+    pass

kirovy/models/__init__.py

@@ -1,5 +1,14 @@
+import typing
+
+from django.db.models import UUIDField, Model
+
 from .cnc_game import CncGame, CncFileExtension
 from .cnc_map import CncMap, CncMapFile, MapCategory
 from .cnc_user import CncUser
 from .file_base import CncNetFileBaseModel
 from .map_preview import MapPreview
+
+
+class SupportsBan(typing.Protocol):
+    def set_ban(self, is_banned: bool, banned_by: CncUser) -> None:
+        ...