Skip to content

Commit cb85c9e

Browse files
datajohnsondatajohnson
authored
Merge pull request #17 from icefoganalytics/issue-16/dev-tooling--add-basic-test-suite-setup-to-project
Dev Tooling: Add Basic Test Suite Setup To Project
2 parents d95f1bf + 2fa28e2 commit cb85c9e

23 files changed

+13176
-5524
lines changed

‎README.md‎

Lines changed: 67 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -21,43 +21,97 @@ Writing code and developing in this application requires running three services:
2121
2. To run the database locally, you must have Docker installed as well as Docker Compose; afterwards, run the following command from the root directory:
2222

2323
```bash
24-
docker compose -f docker-compose.dev.yml up -d
24+
docker compose -f docker-compose.development.yaml up -d
2525
```
2626

2727
or if your docker compose is old
2828

2929
```bash
30-
docker-compose -f docker-compose.dev.yml up -d
30+
docker-compose -f docker-compose.development.yaml up -d
3131
```
3232

3333
This command will start API, SQL, Email, and S3 services, and bind them to appropriate ports on your local machine as specified in [docker-compose.development.yaml](./docker-compose.development.yaml).
3434

3535
3. When the database starts the first time, the database will be empty. To load some seed data, you must obtain a database backup, and put it into `/db/backups/sfa.bak`, then run the follow commands:
3636

37+
```bash
38+
dev up db
39+
#
40+
docker compose -f docker-compose.development.yaml up db
41+
```
42+
43+
If you need to debug the restore, you can connect to the running SQL Server via
44+
45+
```bash
46+
docker compose -f docker-compose.development.yaml exec -it db bash
47+
```
48+
49+
4. The first time you start the application, you must create a bucket named `documents` and an Access Key. Copy the access key id and secret and drop those values into the appropriate spots in the environment file. The Minio Web interface located at http://localhost:9090. Subsequent starts, it is not required to access the Minio interface.
50+
51+
5. To preview sent emails, the MailSlurper web interface is located at http://localhost:8081.
52+
53+
6. To boot the api, test, db, s3 and email services you can use the docker compose setup.
54+
55+
```bash
56+
dev up
57+
# or
58+
docker compose -f docker-compose.development.yaml up
59+
```
60+
61+
If you don't use docker see the "Without Docker" section
62+
63+
7. Last to start is the the Vue.js web front-end. To run this, open a second terminal window at this directory and run the following commands:
64+
65+
```bash
66+
cd src/web
67+
npm install
68+
npm run start
69+
```
70+
71+
You will now have the Vue CLI server hosting the application at http://localhost:8080 and you can begin editing the API or front-end code. **All changes to the files in the `src/api` and `src/web` will automatically reload there respective applications.**
72+
73+
8. Manually add your user account info to the database via
74+
3775
```bash
3876
docker compose \
39-
-f docker-compose.dev.yml \
77+
-f docker-compose.development.yaml \
4078
exec -it db \
4179
/opt/mssql-tools/bin/sqlcmd \
4280
-U sa \
4381
-s localhost \
4482
-P Testing1122 \
45-
-Q "RESTORE DATABASE SFADB_DEV FROM DISK = N'backups/sfa.bak' WITH FILE = 1"
83+
-d SFADB_DEV \
84+
-Q "INSERT INTO sfa.[USER](
85+
email
86+
, email_public
87+
, is_active
88+
, first_name
89+
, last_name
90+
, roles)
91+
VALUES (
92+
N'your.email@something.com'
93+
, N'your.email@something.com'
94+
, 1
95+
, N'YourFirstName'
96+
, N'YourLastName'
97+
, N'Admin');"
4698
```
4799

48-
If you need to debug the restore, you can connect to the running SQL Server via
100+
9. You should now be able to log in at http://localhost:8080, assuming you have an appropriate Auth0 or YNet account.
49101

50-
```bash
51-
docker compose -f docker-compose.dev.yml exec -it db bash
52-
```
102+
---
53103

54-
4. The first time you start the application, you must create a bucket named `documents` and an Access Key. Copy the access key id and secret and drop those values into the appropriate spots in the environment file. The Minio Web interface located at http://localhost:9090. Subsequent starts, it is not required to access the Minio interface.
104+
To access the Database console directly use:
55105

56-
5. To preview sent emails, the MailSlurper web interface is located at http://localhost:8081.
106+
```bash
107+
docker compose -f docker-compose.development.yaml exec db /opt/mssql-tools/bin/sqlcmd -U sa -s localhost -P Testing1122
108+
```
57109

58-
6. Install `asdf` using instructions at https://asdf-vm.com/guide/getting-started.html.
110+
#### Without Docker
59111

60-
7. Install the `nodejs` plugin via and the appropriate nodejs version.
112+
1. Install `asdf` using instructions at https://asdf-vm.com/guide/getting-started.html.
113+
114+
2. Install the `nodejs` plugin via and the appropriate nodejs version.
61115

62116
```bash
63117
asdf plugin add nodejs
@@ -71,7 +125,7 @@ Writing code and developing in this application requires running three services:
71125
node -v
72126
```
73127

74-
8. (optional) You will now have a local database with data ready for the API. To run the API, run the following commands:
128+
3. (optional) You will now have a local database with data ready for the API. To run the API, run the following commands:
75129

76130
```bash
77131
cd src/api
@@ -96,53 +150,6 @@ Writing code and developing in this application requires running three services:
96150

97151
> You no longer need this step if you are using docker compose.
98152

99-
12. Last to start is the the Vue.js web front-end. To run this, open a second terminal window at this directory and run the following commands:
100-
101-
```bash
102-
cd src/web
103-
npm install
104-
npm run start
105-
```
106-
107-
You will now have the Vue CLI server hosting the application at http://localhost:8080 and you can begin editing the API or front-end code. **All changes to the files in the `src/api` and `src/web` will automatically reload there respective applications.**
108-
109-
13. Manually add your user account info to the database via
110-
111-
```bash
112-
docker compose \
113-
-f docker-compose.dev.yml \
114-
exec -it db \
115-
/opt/mssql-tools/bin/sqlcmd \
116-
-U sa \
117-
-s localhost \
118-
-P Testing1122 \
119-
-d SFADB_DEV \
120-
-Q "INSERT INTO sfa.[USER](
121-
email
122-
, email_public
123-
, is_active
124-
, first_name
125-
, last_name
126-
, roles)
127-
VALUES (
128-
N'your.email@something.com'
129-
, N'your.email@something.com'
130-
, 1
131-
, N'YourFirstName'
132-
, N'YourLastName'
133-
, N'Admin');"
134-
```
135-
136-
14. You should now be able to log in at http://localhost:8080, assuming you have an appropriate Auth0 or YNet account.
137-
138-
---
139-
140-
To access the Database console directly use:
141-
142-
```bash
143-
docker compose -f docker-compose.dev.yml exec db /opt/mssql-tools/bin/sqlcmd -U sa -s localhost -P Testing1122
144-
```
145-
146153
## Dev Command Usage
147154

148155
If you want a simpler interface to interact with docker compose you can use the `bin/dev` helper.

‎bin/dev‎

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -59,19 +59,17 @@ class DevHelper
5959

6060
def debug
6161
api_container_id = container_id('api')
62-
puts 'Awaiting binding.pry to trigger...'
63-
puts "'ctrl-c' to exit"
62+
puts 'Waiting for breakpoint to trigger...'
63+
puts "'ctrl-c' to exit."
6464
command = "docker attach --detach-keys ctrl-c #{api_container_id}"
6565
puts "Running: #{command}"
6666
exec(command)
6767
exit 0
6868
end
6969

70-
# def mocha(*args, **kwargs)
71-
# run(*%w[test npm run test], *args, **kwargs)
72-
# end
73-
74-
# alias test mocha
70+
def test(*args, **kwargs)
71+
run(*%w[test npm run test], *args, **kwargs)
72+
end
7573

7674
def npm(*args, **kwargs)
7775
run(*%w[api npm], *args, **kwargs)

‎db/bin/configure-db.sh‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
#!/bin/bash
2+
3+
# Wait 60 seconds for SQL Server to start up by ensuring that
4+
# calling SQLCMD does not return an error code, which will ensure that sqlcmd is accessible
5+
# and that system and user databases return "0" which means all databases are in an "online" state
6+
# https://docs.microsoft.com/en-us/sql/relational-databases/system-catalog-views/sys-databases-transact-sql?view=sql-server-2017
7+
8+
DBSTATUS=1
9+
ERRCODE=1
10+
i=0
11+
12+
while [[ $DBSTATUS -ne 0 ]] && [[ $i -lt 3600 ]] && [[ $ERRCODE -ne 0 ]]; do
13+
i=$i+5
14+
DBSTATUS=$(/opt/mssql-tools/bin/sqlcmd -h -1 -t 1 -U sa -P "$MSSQL_SA_PASSWORD" -Q "SET NOCOUNT ON; Select SUM(state) from sys.databases WHERE name = 'master'")
15+
ERRCODE=$?
16+
sleep 5
17+
done
18+
19+
if [[ $DBSTATUS -ne 0 ]] || [[ $ERRCODE -ne 0 ]]; then
20+
echo "SQL Server took more than 3600 seconds to start up, master database is not in an ONLINE state"
21+
exit 1
22+
fi
23+
24+
# Run the setup script to create the DB and the schema in the DB
25+
/opt/mssql-tools/bin/sqlcmd \
26+
-S localhost \
27+
-U sa \
28+
-P "$MSSQL_SA_PASSWORD" \
29+
-d master \
30+
-i ./initializers/setup.sql

‎db/db.Dockerfile‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
FROM mcr.microsoft.com/mssql/server:2019-latest
2+
3+
# current user:group => $whoami:$(id -gn) => mssql:root
4+
USER root
5+
RUN mkdir -p /usr/src/db && chown mssql:root /usr/src/db
6+
USER mssql
7+
8+
WORKDIR /usr/src/db
9+
10+
# Data
11+
# COPY ./db/log /var/opt/mssql/log
12+
COPY ./db/backups /var/opt/mssql/data/backups
13+
14+
# Intialization
15+
COPY --chown=mssql:root --chmod=+x ./db/entrypoint.sh ./
16+
COPY --chown=mssql:root --chmod=+x ./db/bin ./bin
17+
COPY --chown=mssql:root ./db/initializers ./initializers
18+
19+
ENTRYPOINT ["/usr/src/db/entrypoint.sh"]

‎db/entrypoint.sh‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
#!/bin/bash
2+
3+
# Start the script to create the DB and user
4+
cd /usr/src/db
5+
./bin/configure-db.sh &
6+
7+
# Start SQL Server
8+
/opt/mssql/bin/sqlservr

‎db/initializers/setup.sql‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
/*
2+
3+
Enter custom T-SQL here that would run after SQL Server has started up.
4+
5+
*/
6+
7+
IF NOT EXISTS (
8+
SELECT name
9+
FROM sys.databases
10+
WHERE name = N'SFADB_DEV'
11+
)
12+
RESTORE DATABASE SFADB_DEV FROM DISK = N'backups/sfa.bak';
13+
14+
IF NOT EXISTS (
15+
SELECT name
16+
FROM sys.databases
17+
WHERE name = N'sfa_client_test'
18+
)
19+
RESTORE DATABASE sfa_client_test FROM DISK = N'backups/sfa.bak'
20+
WITH MOVE 'SFADB' TO '/var/opt/mssql/data/sfa_client_test.mdf',
21+
MOVE 'SFADB_log' TO '/var/opt/mssql/data/sfa_client_test_log.ldf';
22+
GO

‎docker-compose.development.yaml‎

Lines changed: 43 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -7,30 +7,65 @@ services:
77
dockerfile: development.Dockerfile
88
env_file:
99
- ./src/api/.env.development
10-
environment:
10+
environment: &environment-definition
1111
NODE_ENV: development
12+
DB_NAME: SFADB_DEV
1213
DB_HOST: db
14+
DB_USER: sa
15+
DB_PASS: &password-definition Testing1122
16+
DB_PORT: 1433
1317
DB_DEFAULT_SCHEMA: sfa
1418
API_PORT: ${API_PORT:-3000}
1519
BASE_URL: "http://localhost:${API_PORT:-3000}"
20+
tty: true
1621
ports:
1722
- "${API_PORT:-3000}:${API_PORT:-3000}"
1823
volumes:
1924
- ./src/api:/usr/src/api
2025
depends_on:
21-
- db
26+
db:
27+
condition: service_healthy
28+
29+
test:
30+
build:
31+
context: ./src/api
32+
dockerfile: development.Dockerfile
33+
command: /bin/true
34+
env_file:
35+
- ./src/api/.env.development
36+
environment:
37+
<<: *environment-definition
38+
NODE_ENV: test
39+
DB_NAME: sfa_client_test
40+
DB_DEFAULT_SCHEMA: dbo
41+
tty: true
42+
volumes:
43+
- ./src/api:/usr/src/api
44+
depends_on:
45+
db:
46+
condition: service_healthy
2247
db:
23-
image: mcr.microsoft.com/mssql/server:2019-latest
48+
build:
49+
context: .
50+
dockerfile: ./db/db.Dockerfile
2451
ports:
2552
- "1433:1433"
2653
user: root
27-
env_file:
28-
- db/sqlserver.env
29-
- db/sapassword.env
54+
environment:
55+
MSSQL_SA_PASSWORD: *password-definition
56+
ACCEPT_EULA: "Y"
3057
volumes:
31-
- ./db/data:/var/opt/mssql/data
58+
- ./db/bin:/usr/src/db/bin
59+
- ./db/initializers:/usr/src/db/initializers
60+
- db_data:/var/opt/mssql/data
3261
- ./db/log:/var/opt/mssql/log
3362
- ./db/backups:/var/opt/mssql/data/backups
63+
healthcheck: # https://github.com/Microsoft/mssql-docker/issues/133
64+
test: "/opt/mssql-tools/bin/sqlcmd -U sa -P \"$$MSSQL_SA_PASSWORD\" -Q \"SELECT 1\" || exit 1"
65+
interval: 5s
66+
timeout: 10s
67+
retries: 3
68+
start_period: 5s
3469
s3:
3570
image: "minio/minio:latest"
3671
ports:
@@ -51,4 +86,5 @@ services:
5186
- 8085:8085
5287

5388
volumes:
89+
db_data:
5490
s3storage:

‎src/api/bin/boot-app.sh‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
#!/bin/sh
2+
3+
# Start the script to run the migrations
4+
cd /usr/src/api
5+
npm run migrate:latest &
6+
7+
# Start Express Server
8+
npm run start

‎src/api/data/migrations/001_template.ts‎

Lines changed: 4 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,6 @@
1-
import { Knex } from "knex";
1+
// LEGACY: required until deleted from dbo.knex_migrations table production, after wich point they can be removed
2+
import { Knex } from "knex"
23

3-
exports.up = async function (knex: Knex, Promise: any) {
4-
5-
};
4+
exports.up = async function (knex: Knex) {}
65

7-
exports.down = async function (knex: Knex, Promise: any) {
8-
9-
};
6+
exports.down = async function (knex: Knex) {}

0 commit comments

Comments
 (0)