diff --git a/.env.template.docker b/.env.template.docker new file mode 100644 index 00000000..2ae84032 --- /dev/null +++ b/.env.template.docker @@ -0,0 +1,11 @@ +SECRET_KEY=<####SECRET####> +DEBUG=True +ALLOWED_HOSTS=0.0.0.0,127.0.0.1,localhost +DJANGO_SETTINGS_MODULE=djangosnippets.settings.development +SEARCHBOX_SSL_URL=http://elasticsearch:9200/ +SESSION_COOKIE_SECURE=False +DATABASE_URL=postgres://djangosnippets:djangosnippets@db/djangosnippets +POSTGRES_USER=djangosnippets +POSTGRES_PASSWORD=djangosnippets +POSTGRES_DB=djangosnippets +REDISTOGO_URL=redis://redis:6379/0 diff --git a/.env.template.local b/.env.template.local new file mode 100644 index 00000000..8ce5046c --- /dev/null +++ b/.env.template.local @@ -0,0 +1,8 @@ +SECRET_KEY=<####SECRET####> +DEBUG=True +ALLOWED_HOSTS=0.0.0.0,127.0.0.1,localhost +DJANGO_SETTINGS_MODULE=djangosnippets.settings.development +SEARCHBOX_SSL_URL=http://elasticsearch:9200/ +SESSION_COOKIE_SECURE=False +DATABASE_URL=postgres://djangosnippets:djangosnippets@db/djangosnippets +REDISTOGO_URL=redis://redis:6379/0 diff --git a/README.rst b/README.rst index f0ef0cc1..74c543fa 100644 --- a/README.rst +++ b/README.rst @@ -3,94 +3,166 @@ djangosnippets.org This code is used to power the snippet sharing site, `djangosnippets.org`_ -Database Setup Using Windows ------------------------------------ +Development Setup +================= -Download the latest version of PostgreSQL_. Click on the executable to start the installation setup wizard. +Prerequisites +------------- -Click ``Next``, keeping all the defaults as you work through the wizard. Make a note -of the password you choose for the database superuser (postgres). Select the default port 5432 and the default -locale. After it’s finished installing, you do not need to launch Stack Builder. Un-tick that box if you are asked, -and click ``Finish``. +- Python version 3.11 +- PostgreSQL -Open SQL Shell (psql). In the shell, select the default values for Server, Database, Port and Username -(basically, press Enter four times). +Installation +------------ -Type in the password you noted earlier and press enter. Run the command below, taking care to include the -semi-colon. :: +Basic Installation +~~~~~~~~~~~~~~~~~~ - $ CREATE DATABASE djangosnippets; +1. Clone the repo: -Close SQL Shell (psql). + .. code-block:: console -You need to copy .env.example to env.bat and configure to your needs. Use the template below, taking care to -include ``set`` at the start of each line, and to substitute the password you noted earlier into DATABASE_URL. -For development, DEBUG is set to True. :: + https://github.com/django/djangosnippets.org.git - set REDISTOGO_URL=redis://redis:6379/0 - set SECRET_KEY=p_o3vp1rg5)t^lxm9-43%0)s-=1qpeq%o7gfq+e4#*!t+_ev82 - set DEBUG=True - set ALLOWED_HOSTS=0.0.0.0,127.0.0.1 - set DATABASE_URL=postgres://postgres:your_password@:5432/djangosnippets - set DJANGO_SETTINGS_MODULE=djangosnippets.settings.development - set SEARCHBOX_SSL_URL=http://elasticsearch:9200/ - set SESSION_COOKIE_SECURE=False +2. Create your virtual environment: -Go back to your terminal. You will need to run the command below whenever you open a new terminal. :: + .. code-block:: console - $ env.bat + python -m venv venv -Your environment variables are now set and you can proceed with the instructions below. + Activate in Linux: -Development Setup ------------------ + .. code-block:: console -In a Python 3.11 virtual environment:: + source venv/bin/activate - $ cd requirements - $ pip install -r development.txt - $ cd .. - $ python manage.py tailwind install - $ python manage.py migrate + Activate in Windows: -Now you can start the development server:: + .. code-block:: console - $ python manage.py runserver + venv\Scripts\activate -Before you can actually use the site, you have to define at least one -language. If you just want to use the ones from djangosnippets.org, they -are included in the fixtures folder. Also included are five snippets to get you started:: +3. Connect to PostgreSQL - $ python manage.py createsuperuser - $ python manage.py loaddata fixtures/cab.json + Connect in Linux: -To use Tailwind, you need to start the Tailwind server:: + .. code-block:: console - $ python manage.py tailwind start + psql -U $(whoami) -d postgres -Now you should be able to use the development version of djangosnippets -on port 8000. + Connect in Windows: -To run tests:: + .. code-block:: console - $ python manage.py test --settings=djangosnippets.settings.testing + psql -U postgres + +4. Create a PostgreSQL database and role: + + .. code-block:: console + + postgres=# CREATE DATABASE djangosnippets; + postgres=# CREATE USER djangosnippets WITH SUPERUSER PASSWORD 'djangosnippets'; + postgres=# GRANT ALL PRIVILEGES ON DATABASE djangosnippets TO djangosnippets; + + Exit psql shell: + + .. code-block:: console + + postgres=# exit + +5. Install requirements: + + .. code-block:: console + + pip install -r requirements/development.txt + +6. Copy `.env.template.local` file, rename to `.env` and configure variables for your local postgres database. + + Copy in Linux: + + .. code-block:: console + + cp .env.template.local .env + + Copy in Windows: + + .. code-block:: console + + copy .env.template.local .env + +7. Run migrations and create superuser: + + Migrate: + + .. code-block:: console + + python manage.py migrate + + Optionally load data first: + + .. code-block:: console + + python manage.py loaddata fixtures/cab.json + + Create superuser: + + .. code-block:: console + + python manage.py createsuperuser + +8. Install tailwind (npm is required): + + .. code-block:: console + + python manage.py tailwind install + +9. Run server locally: + + .. code-block:: console + + python manage.py runserver_plus + +10. Run tailwind in another terminal locally: + + .. code-block:: console + + python manage.py tailwind start + +With Docker +~~~~~~~~~~~~~~~~~~~ + +Using `Docker `_ allows you to set up the development environment more quickly if Docker is installed 🐳 + +1. Build the Docker images: + + .. code-block:: console + + docker compose -f docker-compose.local.yml build + +2. Start the containers: + + .. code-block:: console + + docker compose -f docker-compose.local.yml up -d + +3. Go to: http://127.0.0.1:8000/ and enjoy πŸ™Œ Docker ------- +====== You need to copy .env.example to .env and configure to your needs. The example is fine to start with development. You may wish to use docker locally for production dependency testing and development; here are the setup instructions:: - $ docker-compose -f docker-compose.yml build - $ docker-compose -f docker-compose.yml up -d + $ docker-compose -f docker-compose.production.yml build + $ docker-compose -f docker-compose.production.yml up -d -d denotes running docker in a detached state:: - $ docker-compose -f docker-compose.yml run web python manage.py migrate - $ docker-compose -f docker-compose.yml run web python manage.py createsuperuser - $ docker-compose -f docker-compose.yml run web python manage.py loaddata fixtures/cab.json + $ docker-compose -f docker-compose.production.yml run web python manage.py migrate + $ docker-compose -f docker-compose.production.yml run web python manage.py createsuperuser + $ docker-compose -f docker-compose.production.yml run web python manage.py loaddata fixtures/cab.json $ npm run build - $ docker-compose -f docker-compose.yml run web python manage.py collectstatic + $ docker-compose -f docker-compose.production.yml run web python manage.py collectstatic The docker setup is running as close as possible to the production setup in Heroku: @@ -103,8 +175,14 @@ To run our tests with docker:: $ docker-compose -f docker-compose.yml run web python manage.py test --settings=djangosnippets.settings.testing +Test +====== +To run tests:: + + $ python manage.py test --settings=djangosnippets.settings.testing + Styling Contributor? --------------------- +==================== DjangoSnippets uses the Foundation_ framework as the core of its visual style. To get this working on your local machine you need compass_ and bower_ to compile @@ -126,7 +204,7 @@ configuration inside `djangosnippets/static/config.rb` is Production Setup ----------------- +================ The production setup is currently tailored to Heroku and, therefore, mostly automatic. The difference between these two setups is configured in diff --git a/compose/local/django/Dockerfile b/compose/local/django/Dockerfile new file mode 100644 index 00000000..4e5efc75 --- /dev/null +++ b/compose/local/django/Dockerfile @@ -0,0 +1,74 @@ +# define an alias for the specific python version used in this file. +FROM docker.io/python:3.11-slim-bookworm AS python + +# Python build stage +FROM python AS python-build-stage + +# Install apt packages +RUN apt-get update && apt-get install --no-install-recommends -y \ + # dependencies for building Python packages + build-essential \ + # psycopg dependencies + libpq-dev + +# Requirements are installed here to ensure they will be cached. +COPY ./requirements . + +# Create Python Dependency and Sub-Dependency Wheels. +RUN pip wheel --wheel-dir /usr/src/app/wheels \ + -r development.txt + +# Python 'run' stage +FROM python AS python-run-stage + +ENV PYTHONUNBUFFERED=1 +ENV PYTHONDONTWRITEBYTECODE=1 + +WORKDIR /app + +ARG NODE_MAJOR=20 + +RUN apt-get update \ + && apt-get install -y ca-certificates curl gnupg \ + && mkdir -p /etc/apt/keyrings \ + && curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg \ + && echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list \ + && apt-get update \ + && apt-get install nodejs -y \ + && rm -rf /var/lib/apt/lists/* /usr/share/doc /usr/share/man \ + && apt-get clean + +# devcontainer dependencies and utils +RUN apt-get update && apt-get install --no-install-recommends -y \ + sudo git bash-completion nano ssh + +# Create devcontainer user and add it to sudoers +RUN groupadd --gid 1000 dev-user \ + && useradd --uid 1000 --gid dev-user --shell /bin/bash --create-home dev-user \ + && echo dev-user ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/dev-user \ + && chmod 0440 /etc/sudoers.d/dev-user + +# Install required system dependencies +RUN apt-get update && apt-get install --no-install-recommends -y \ + # psycopg dependencies + libpq-dev \ + wait-for-it \ + # Translations dependencies + gettext \ + # cleaning up unused files + && apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false \ + && rm -rf /var/lib/apt/lists/* + +# All absolute dir copies ignore workdir instruction. All relative dir copies are wrt to the workdir instruction +# copy python dependency wheels from python-build-stage +COPY --from=python-build-stage /usr/src/app/wheels /wheels/ + +# use wheels to install python dependencies +RUN pip install --no-cache-dir --no-index --find-links=/wheels/ /wheels/* \ + && rm -rf /wheels/ + +COPY ./compose/local/django/start /start +RUN sed -i 's/\r$//g' /start +RUN chmod +x /start + +CMD ["/start"] diff --git a/compose/local/django/start b/compose/local/django/start new file mode 100644 index 00000000..1596d677 --- /dev/null +++ b/compose/local/django/start @@ -0,0 +1,9 @@ +#!/bin/bash + +set -o errexit +set -o pipefail +set -o nounset + +python manage.py migrate +python manage.py tailwind install +python manage.py runserver_plus 0.0.0.0:8000 diff --git a/djangosnippets/settings/base.py b/djangosnippets/settings/base.py index fdc2820e..3e1fe1c5 100644 --- a/djangosnippets/settings/base.py +++ b/djangosnippets/settings/base.py @@ -3,6 +3,9 @@ import dj_database_url from django.contrib import messages from django.urls import reverse +from dotenv import load_dotenv + +load_dotenv() def user_url(user): @@ -178,10 +181,9 @@ def user_url(user): } -DATABASES = {"default": dj_database_url.config(default="postgres:///djangosnippets")} +DATABASES = {"default": dj_database_url.config(conn_max_age=600, conn_health_checks=True)} DATABASES["default"]["ATOMIC_REQUESTS"] = True - REST_FRAMEWORK = { # Use Django's standard `django.contrib.auth` permissions, # or allow read-only access for unauthenticated users. diff --git a/docker-compose.local.yml b/docker-compose.local.yml new file mode 100644 index 00000000..205a6a03 --- /dev/null +++ b/docker-compose.local.yml @@ -0,0 +1,55 @@ +version: "3.7" + +x-app: &default-app + build: + context: . + dockerfile: ./compose/local/django/Dockerfile + restart: unless-stopped + volumes: + - .:/app:z + +services: + web: + <<: *default-app + container_name: djangosnippets-web-local + ports: + - 8000:8000 + env_file: + - ./.env.template.docker + depends_on: + - db + + tailwind: + <<: *default-app + container_name: djangosnippets-tailwind-local + command: "python manage.py tailwind start" + # Without tty, no stdin, and tailwind watcher aborts + # https://github.com/tailwindlabs/tailwindcss/issues/5324 + tty: true + + db: + image: postgres:15.13 + container_name: djangosnippets-db-local + restart: always + volumes: + - postgres_data:/var/lib/postgresql/data/ + env_file: + - ./.env.template.docker + ports: + - 5432:5432 + + redis: + image: redis:3.2.12 + container_name: djangosnippets-redis-local + restart: always + volumes: + - redisdata:/data + env_file: + - ./.env.template.docker + ports: + - 6379:6379 + +volumes: + web: + postgres_data: + redisdata: diff --git a/docker-compose.yml b/docker-compose.production.yml similarity index 100% rename from docker-compose.yml rename to docker-compose.production.yml diff --git a/requirements/base.txt b/requirements/base.txt index ade9b78f..70eb9f3c 100644 --- a/requirements/base.txt +++ b/requirements/base.txt @@ -1,6 +1,6 @@ bleach==6.2.0 Django==5.2.3 -dj-database-url==0.5.0 +dj-database-url==3.0.0 django-allauth==0.63.6 django-contrib-comments==2.2.0 django-extensions==3.2.3 @@ -11,14 +11,15 @@ django-taggit==5.0.1 django-tailwind==4.0.1 ipython==7.33.0 Markdown==3.4.4 -Pillow==9.1.0 +Pillow==11.2.1 Pygments==2.12.0 python-akismet==0.4.2 +python-dotenv==1.1.1 requests==2.32.4 six==1.15.0 urllib3==1.26.6 whitenoise==6.1.0 -psycopg2-binary==2.9.3 +psycopg2-binary==2.9.10 djangorestframework==3.14.0 django-htmx==1.15.0 gevent==22.10.2 # Updated version of gevent diff --git a/requirements/development.txt b/requirements/development.txt index c660921d..7b12af0c 100644 --- a/requirements/development.txt +++ b/requirements/development.txt @@ -1,4 +1,5 @@ -r base.txt +Werkzeug[watchdog]==3.0.6 flake8==4.0.1 isort==5.8.0 pre-commit==2.19.0