MALTrendsWeb

Django/React/Postgres build of a web app that provides scores and popularity time series visualizations of animes on MyAnimeList.

The data in this repo located at backend/anime_data is gathered from another project MALTrends, which is a web scraper for MyAnimeList statistics on the Wayback Machine internet archive.

About

A Django project with lots of state of the art libraries and tools like:

• React, for building interactive UIs
• django-js-reverse, for generating URLs on JS
• Bootstrap 4, for responsive styling
• Webpack, for bundling static assets
• Celery, for background worker tasks
• WhiteNoise with brotlipy, for efficient static files serving
• prospector and ESLint with pre-commit for automated quality assurance (does not replace proper testing!)

For continuous integration, a CircleCI configuration .circleci/config.yml is included.

Also, includes a Heroku app.json and a working Django production.py settings, enabling easy deployments with 'Deploy to Heroku' button. Those Heroku plugins are included in app.json:

• PostgreSQL, for DB
• Redis, for Celery
• Sendgrid, for e-mail sending
• Papertrail, for logs and platform errors alerts (must set them manually)

Running

Setup (plain python)

• Inside the backend folder, do the following:
• Create a copy of MALTrendsWeb/settings/local.py.example:  cp MALTrendsWeb/settings/local.py.example MALTrendsWeb/settings/local.py.
• Create a copy of .env.example: cp .env.example .env
• If you are using plain python:
  • Create the migrations for users app: python manage.py makemigrations
  • Run the migrations: python manage.py migrate
• If you are using docker:
  • Create the migrations for users app: docker-compose backend run python manage.py makemigrations
  • Run the migrations: docker-compose backend run python manage.py migrate

Tools

• Setup editorconfig, prospector and ESLint in the text editor you will use to develop.

Running the project (without docker)

• Open a command line window and go to the project's directory.
• pip install -r requirements.txt && pip install -r dev-requirements.txt
• npm install
• npm run start
• Open another command line window and go to the backend directory.
• workon MALTrendsWeb or source MALTrendsWeb/bin/activate depending on if you are using virtualenvwrapper or just virtualenv.
• python manage.py runserver

Running the project (with docker)

• Open a command line window and go to the project's directory.
• docker-compose up -d To access the logs for each service run docker-compose logs -f service_name (either backend, frontend, etc)

Celery

• Open a command line window and go to the project's directory
• workon MALTrendsWeb or source MALTrendsWeb/bin/activate depending on if you are using virtualenvwrapper or just virtualenv.
• python manage.py celery

Testing

make test

Will run django tests using --keepdb and --parallel. You may pass a path to the desired test module in the make command. E.g.:

make test someapp.tests.test_views

Adding new pypi libs

Add the libname to either requirements.in or dev-requirents.in, then either upgrade the libs with make upgrade or manually compile it and then, install. pip-compile requirements.in > requirements.txt (or make upgrade)
pip install -r requirements.txt

Deployment

Setup

This project comes with a app.json file for heroku, that can be used to create an app on heroku from a GitHub repository.

After setting up the project, you can init a repository and push it on GitHub. If your repository is public, you can use the following button:

If you are in a private repository, access the following link replacing $YOUR_REPOSITORY_LINK$ with your repository link.

• https://heroku.com/deploy?template=$YOUR_REPOSITORY_LINK$

Remember to fill the ALLOWED_HOSTS with the URL of your app, the default on heroku is appname.herokuapp.com. Replace appname with your heroku app name.

Sentry

Sentry is already set up on the project. For production, add SENTRY_DSN environment variable on Heroku, with your Sentry DSN as the value.

You can test your Sentry configuration by deploying the boilerplate with the sample page and clicking on the corresponding button.

Sentry source maps for JS files

The bin/post_compile script has a step to push Javascript source maps to Sentry, however some environment variables need to be set on Heroku.

You need to enable Heroku dyno metadata on your Heroku App. Use the following command on Heroku CLI:

• heroku labs:enable runtime-dyno-metadata -a <app name>

The environment variables that need to be set are:

• SENTRY_ORG - Name of the Sentry Organization that owns your Sentry Project.
• SENTRY_PROJECT_NAME - Name of the Sentry Project.
• SENTRY_API_KEY - Sentry API key that needs to be generated on Sentry. You can find or create authentication tokens within Sentry.

After enabling dyno metadata and setting the environment variables, your next Heroku Deploys will create a release on Sentry where the release name is the commit SHA, and it will push the source maps to it.

Linting

• Manually with prospector and npm run lint on project root.
• During development with an editor compatible with prospector and ESLint.

Pre-commit hooks

• Run pre-commit install to enable the hook into your git repo. The hook will run automatically for each commit.
• Run git commit -m "Your message" -n to skip the hook if you need.

Commercial Support

Copyright (c) 2019 Daniel Liu

Copyright (c) 2018 Vinta Serviços e Soluções Tecnológicas Ltda. MIT License