Skip to content
This repository is currently being migrated. It's locked while the migration is in progress.

Commit 29f0f35

Browse files
singhalssinghals
authored
Try to push to ecr (#2)
* Try to push to ecr * Remove other actions for now * Fix repo name * pull image from ecr repo * Setup pixi * Fix image name * Get working aws roles for github * Make sure to deploy the service * Try using family * Add describe service perm * Allow update service * Add registry to image name * only deploy on main * Remove scripts * Remove unused pixi
1 parent 145c57d commit 29f0f35

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

50 files changed

+3255
-6282
lines changed

‎.github/workflows/deploy.yml‎

Lines changed: 53 additions & 94 deletions
Original file line numberDiff line numberDiff line change
@@ -3,107 +3,66 @@ on:
33
push:
44
branches:
55
- main
6-
env:
7-
TF_CLOUD_ORGANIZATION: "shughesuk"
8-
CONFIG_DIRECTORY: "./"
6+
permissions:
7+
id-token: write # This is required for requesting tghe JWT
8+
contents: read # This is required for actions/checkout
99
jobs:
10-
deploy-backend:
11-
outputs:
12-
sha: ${{ steps.short_sha.outputs.sha }}
13-
environment: backend-production
14-
concurrency: backend-production
15-
permissions:
16-
id-token: write
17-
contents: read
10+
deploy-cube:
1811
runs-on: ubuntu-latest
1912
steps:
20-
- name: Checkout code
13+
- name: Checkout repo
2114
uses: actions/checkout@v4
22-
- name: Set up QEMU
23-
uses: docker/setup-qemu-action@v3
24-
- name: Set up Docker Buildx
25-
uses: docker/setup-buildx-action@v3
26-
- name: Login to Docker Hub
27-
uses: docker/login-action@v3
15+
16+
- name: Configure AWS credentials
17+
uses: aws-actions/configure-aws-credentials@v4
2818
with:
29-
username: ${{ secrets.DOCKERHUB_USERNAME }}
30-
password: ${{ secrets.DOCKERHUB_TOKEN }}
31-
- name: Get short SHA
32-
id: short_sha
33-
run: echo "sha=$(git rev-parse --short HEAD)" >> $GITHUB_OUTPUT
34-
- name: Build and push Docker image
35-
uses: docker/build-push-action@v5
19+
role-to-assume: arn:aws:iam::471881062455:role/system/github_actions_role
20+
role-session-name: GitHub_to_AWS_sync_svc_cube
21+
aws-region: us-east-1
22+
23+
- name: Login to Amazon ECR
24+
id: login-ecr
25+
uses: aws-actions/[email protected]
3626
with:
37-
context: .
38-
push: true
39-
tags: shughesuk/backend:${{ steps.short_sha.outputs.sha }}
40-
run-migrations:
41-
name: "Run Migrations"
42-
runs-on: ubuntu-latest
43-
needs: deploy-backend
44-
permissions:
45-
contents: read
46-
id-token: write
47-
steps:
48-
- name: Checkout
49-
uses: actions/checkout@v4
50-
- name: configure aws credentials
51-
uses: aws-actions/[email protected]
27+
mask-password: "false"
28+
29+
- name: Build, tag, and push docker image to Amazon ECR
30+
env:
31+
REGISTRY: ${{ steps.login-ecr.outputs.registry }}
32+
REPOSITORY: sync-svc-cube
33+
IMAGE_TAG: "${{ github.sha }}"
34+
run: |
35+
docker build -t $REGISTRY/$REPOSITORY:$IMAGE_TAG .
36+
docker push $REGISTRY/$REPOSITORY:$IMAGE_TAG
37+
38+
- name: Update cube-api Task Definition with latest image
39+
id: cube-api-task-def
40+
uses: aws-actions/[email protected]
5241
with:
53-
role-to-assume: arn:aws:iam::905418398753:role/github-actions-role
54-
role-session-name: GitHub_to_AWS_via_FederatedOIDC
55-
aws-region: "us-east-1"
56-
- uses: prefix-dev/[email protected]
42+
task-definition-family: cube_api
43+
container-name: cube-api
44+
image: ${{ steps.login-ecr.outputs.registry }}/sync-svc-cube:${{ github.sha }}
45+
46+
- name: Update cube-refresh-worker Task Definition with latest image
47+
id: cube-refresh-worker-task-def
48+
uses: aws-actions/[email protected]
5749
with:
58-
cache: true
59-
locked: true
60-
- name: Update task
61-
run: pixi run python scripts/update_task.py --task-definition production --container-name backend-api --image shughesuk/backend:${{ needs.deploy-backend.outputs.sha }}
62-
- name: Run migrations
63-
run: pixi run python scripts/run_task.py --task-definition production --cluster production --command "pixi run python manage.py migrate"
64-
terraform:
65-
needs:
66-
- deploy-backend
67-
- run-migrations
68-
name: "Terraform Apply"
69-
runs-on: ubuntu-latest
70-
permissions:
71-
contents: read
72-
steps:
73-
- name: Checkout
74-
uses: actions/checkout@v4
75-
- name: terraform-apply
76-
uses: dflook/[email protected]
50+
task-definition-family: cube_refresh_worker
51+
container-name: cube-refresh-worker
52+
image: ${{ steps.login-ecr.outputs.registry }}/sync-svc-cube:${{ github.sha }}
53+
54+
- name: Deploy cube-api task definition
55+
uses: aws-actions/[email protected]
7756
with:
78-
path: ./terraform
79-
auto_approve: true
80-
workspace: resume-workspace
81-
variables: |
82-
app_image = "shughesuk/backend:${{ needs.deploy-backend.outputs.sha }}"
83-
env:
84-
TERRAFORM_CLOUD_TOKENS: app.terraform.io=${{ secrets.TF_API_TOKEN }}
85-
deploy-frontend:
86-
needs: terraform
87-
environment: frontend-production
88-
concurrency: frontend-production
89-
permissions:
90-
id-token: write
91-
contents: read
92-
runs-on: ubuntu-latest
93-
steps:
94-
- name: Checkout
95-
uses: actions/checkout@v4
96-
- name: configure aws credentials
97-
uses: aws-actions/[email protected]
98-
with:
99-
role-to-assume: arn:aws:iam::905418398753:role/github-actions-role
100-
role-session-name: GitHub_to_AWS_via_FederatedOIDC
101-
aws-region: "us-east-1"
102-
- uses: prefix-dev/[email protected]
103-
env:
104-
ACTIONS_STEP_DEBUG: true
57+
task-definition: ${{ steps.cube-api-task-def.outputs.task-definition }}
58+
service: cube_api
59+
cluster: production
60+
wait-for-service-stability: true
61+
62+
- name: Deploy cube-refresh-worker task definition
63+
uses: aws-actions/[email protected]
10564
with:
106-
cache: true
107-
locked: true
108-
- name: Deploy
109-
run: pixi run frontend-deploy
65+
task-definition: ${{ steps.cube-refresh-worker-task-def.outputs.task-definition }}
66+
service: cube_refresh_worker
67+
cluster: production
68+
wait-for-service-stability: true

‎.github/workflows/pre-commit.yml‎

Lines changed: 0 additions & 22 deletions
This file was deleted.

‎.github/workflows/terraform-plan.yml‎

Lines changed: 0 additions & 79 deletions
This file was deleted.

‎Dockerfile‎

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,6 @@
1-
FROM ghcr.io/prefix-dev/pixi:0.18.0-bookworm-slim
1+
FROM cubejs/cube:v1.1.9
22

3-
COPY ./backend /opt/backend
4-
COPY ./pixi.lock /opt/backend/pixi.lock
5-
COPY ./pixi.toml /opt/backend/pixi.toml
6-
WORKDIR /opt/backend/
7-
RUN pixi install
8-
CMD pixi run uvicorn --port 80 --host 0.0.0.0 resume.asgi:application --log-config logging.yaml
3+
COPY cube.js cube.js
4+
COPY fetch.js fetch.js
5+
RUN mkdir model
6+
COPY model/ model/

‎README.md‎

Lines changed: 32 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -1,36 +1,42 @@
1-
# Samantha Hughes's Resume
1+
# sync-svc-cube
2+
A cube dev deployment creating a semantic layer for serving up a connected data model.
23

3-
[![Deployment](https://github.com/shughes-uk/resume/actions/workflows/deploy.yml/badge.svg)](https://d3k3m107rwvrzt.cloudfront.net/)
4+
## Standing up Cube locally
5+
### Prerequisites
6+
- Your machine is running docker
7+
- You have an Auth0 user
48

5-
## Getting Started
9+
### Standing up Cube locally
10+
1. Create a .env file at the root repo directory and add the following properties and get their values from dev:
611

7-
The repo uses [pixi](https://prefix.dev/docs/pixi/overview). Install it with
12+
1.1. Add properties and values to setup authentication for auth0:
13+
```
14+
CUBEJS_JWK_URL
15+
CUBEJS_JWT_AUDIENCE
16+
CUBEJS_JWT_ISSUER
17+
CUBEJS_JWT_ALGS
18+
CUBEJS_JWT_CLAIMS_NAMESPACE
19+
```
820
9-
```shell
10-
curl -fsSL https://pixi.sh/install.sh | bash
11-
```
21+
1.2. Add properties to connect to different Sync datastores like Postgres and Databricks Delta Tables
22+
```
23+
CUBEJS_DB_HOST
24+
CUBEJS_DB_PORT
25+
CUBEJS_DB_NAME
26+
CUBEJS_DB_USER
27+
CUBEJS_DB_PASS
28+
CUBEJS_DB_TYPE
29+
```
1230
13-
Then install everything you'll need
31+
2. Build docker image `docker build -t sync-cube-image .`
1432
15-
```shell
16-
pixi install
17-
pixi run frontend-install
18-
```
33+
3. Run `docker compose up`
1934
20-
Add the precommit with
35+
4. Run `curl -X GET 'http://localhost:4000/livez'` to ensure cube is properly running
2136
22-
```shell
23-
pre-commit install
24-
```
37+
5. You should now have a running cube server! Happy cubing! You can navigate to `http://localhost:4000/` to play with the cube playground.
2538
26-
Start the frontend dev server with
2739
28-
```shell
29-
pixi run frontend-dev
30-
```
31-
32-
Start hacking!
33-
34-
## Deployment
35-
36-
All pushes to main are immediately deployed!
40+
## Helpful Cube Dev documentation:
41+
* https://cube.dev/
42+
* https://github.com/cube-js/cube

0 commit comments

Comments
 (0)