Prepare Databases for Manager

Guide variables

⚠️ Prepare the values of the following variables before working with this page and replace their occurrences with the values when you follow the guide.

{NS}	The etcd namespace
{ETCDADDR}	The etcd cluster address ({ETCDHOST}:{ETCDPORT}, localhost:2379 for development setup)
{DBADDR}	The PostgreSQL server address ({DBHOST}:{DBPORT}, localhost:5442 for development setup)
{DBUSER}	The database username (e.g., postgres for development setup)
{DBPASS}	The database password (e.g., develove for development setup)
{STRGMOUNT}	The path to the mounted network-shared storage. (Development setup: Use an arbitrary empty directory where Docker containers can also mount as volumes — e.g., Docker for Mac requires explicit configuration for mountable parent folders.)

Load initial etcd data

$ cd backend.ai-manager

Copy sample-configs/image-metadata.yml and sample-configs/image-aliases.yml and edit according to your setup.

$ cp sample-configs/image-metadata.yml image-metadata.yml
$ cp sample-configs/image-aliases.yml image-aliases.yml

By default you can pull the images listed in the sample via docker pull lablup/kernel-xxxx:tag(e.g. docker pull lablup/kernel-python-tensorflow:latest for the latest tensorflow) as they are hosted on the public Docker registry.

Load image registry metadata

$ python -m ai.backend.manager.cli etcd update-images \
>        --namespace={NS} --etcd-addr={ETCDADDR} \
>        -f image-metadata.yml

Load image aliases

$ python -m ai.backend.manager.cli etcd update-aliases \
>        --namespace={NS} --etcd-addr={ETCDADDR} \
>        -f image-aliases.yml

Set the default storage mount for virtual folders

You need to install a local etcdctl client.
Unfortunately, the client is always distributed with the server (which we don't need because we already installed our etcd cluster separately). To get the client executable only, grab it from a decompressed release binary tar downloaded from here. Ensure that the binary version matches with your cluster version. Since the client executable is self-contained, you don't need to install or configure anything else.

$ ETCDCTL_API=3 etcdctl --endpoints http://{ETCDADDR} \
>                       put /sorna/{NS}/volumes/_vfroot {STRGMOUNT}

(If you don't add etcdctl-v3.x to $PATH, replace etcdctl with (path)/etcdctl.)

Database Setup

Create a new database

$ psql -h {DBHOST} -p {DBPORT} -U {DBUSER}

postgres=# CREATE DATABASE backend;
postgres=# \q

Install database schema

Backend.AI uses alembic to manage database schema and its migration during version upgrades. First, localize the sample config:

$ cp alembic.ini.sample alembic.ini

Modify the line where sqlalchemy.url is set. You may use the following shell command: (ensure that special characters in your password are properly escaped)

$ sed -i'' -e 's!^sqlalchemy.url = .*$!sqlalchemy.url = postgresql://{DBUSER}:{DBPASS}@{DBHOST}:{DBPORT}/backend!' alembic.ini

$ python -m ai.backend.manager.cli schema oneshot head

NOTE: All sub-commands under "schema" uses alembic.ini to establish database connections.

Load initial fixtures

Edit ai/backend/manager/models/fixtures.py so that you have a randomized admin keypair.

(TODO: automate here!)

Then pour it to the database:

$ python -m ai.backend.manager.cli \
>        --db-addr={DBHOST}:{DBPORT} --db-user={DBUSER} --db-password={DBPASS} --db-name=backend \
>        fixture populate example_keypair