-
Notifications
You must be signed in to change notification settings - Fork 19
[draft] AoU jupyter example #131
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
5534dad
e17423b
bf3ba3a
802b933
758d60e
1bc424b
20d1e56
7b3dc71
b32a41a
dd31167
056b9a6
b8ecfd9
2ba1adc
7ae7cd0
31891c9
35a8844
cab172a
be7f19d
a2cd37d
c2802cb
c6c398a
2170dca
70958cc
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| { | ||
| "name": "Jupyter app for AoU", | ||
| "dockerComposeFile": "docker-compose.yaml", | ||
| "service": "app", | ||
| "shutdownAction": "none", | ||
| "workspaceFolder": "/workspace", | ||
| "postCreateCommand": ["./startupscript/post-startup.sh", "jovyan", "/home/jovyan", "${templateOption:cloud}", "${templateOption:login}"], | ||
| // re-mount bucket files on container start up | ||
| "postStartCommand":["./startupscript/remount-on-restart.sh", "jovyan", "/home/jovyan", "${templateOption:cloud}", "${templateOption:login}"], | ||
| "features": { | ||
| "ghcr.io/devcontainers/features/aws-cli:1": {}, | ||
| "ghcr.io/dhoeric/features/google-cloud-cli:1": {}, | ||
| "ghcr.io/devcontainers/features/java:1": {} | ||
| }, | ||
| "remoteUser": "root" | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
|
|
||
| # Jupyter (jupyter) | ||
|
|
||
| A Template to run jupyter on workbench | ||
|
|
||
| ## Options | ||
|
|
||
| | Options Id | Description | Type | Default Value | | ||
| |-----|-----|-----|-----| | ||
| | cloud | VM cloud environment | string | gcp | | ||
| | login | Whether to log in to workbench CLI | string | false | | ||
|
|
||
|
|
||
|
|
||
| --- | ||
|
|
||
| _Note: This file was auto-generated from the [devcontainer-template.json](https://github.com/verily-src/workbench-app-devcontainers/blob/main/src/jupyter/devcontainer-template.json). Add additional notes to a `NOTES.md`._ |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| { | ||
| "id": "jupyter", | ||
| "version": "0.0.2", | ||
| "name": "Jupyter", | ||
| "description": "A Template to run jupyter on workbench", | ||
| "documentationURL": "https://github.com/verily-src/workbench-app-devcontainers/tree/master/src/jupyter", | ||
| "licenseURL": "https://github.com/verily-src/workbench-app-devcontainers/blob/master/LICENSE", | ||
| "options": { | ||
| "cloud": { | ||
| "type": "string", | ||
| "description": "VM cloud environment", | ||
| "proposals": [ | ||
| "gcp", | ||
| "aws" | ||
| ], | ||
| "default": "gcp" | ||
| }, | ||
| "login": { | ||
| "type": "string", | ||
| "description": "Whether to log in to workbench CLI", | ||
| "proposals": [ | ||
| "true", | ||
| "false" | ||
| ], | ||
| "default": "false" | ||
| } | ||
| }, | ||
| "platforms": [ | ||
| "Any" | ||
| ] | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| version: '2.4' | ||
| services: | ||
| app: | ||
| container_name: "application-server" | ||
| image: "gcr.io/deeplearning-platform-release/tf-gpu.2-11.py310" | ||
| restart: always | ||
| volumes: | ||
| - .:/workspace:cached | ||
| - work:/home/jovyan/work | ||
| - gcsfuse-mount:/mnt/gcs | ||
| ports: | ||
| - '8888:8888' | ||
| environment: | ||
| JUPYTER_ENABLE_LAB: "yes" | ||
| command: "start-notebook.py --ServerApp.token='' --ServerApp.password='' --ServerApp.root_dir=/home/jovyan/work --ServerApp.allow_origin='*'" | ||
| networks: | ||
| - app-network | ||
| devices: | ||
| - /dev/fuse | ||
| security_opt: | ||
| - apparmor:unconfined | ||
| wondershaper: | ||
| container_name: "wondershaper" | ||
| image: "us.gcr.io/broad-dsp-gcr-public/wondershaper:0.0.1" | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Wondershaper for rate limiting. Eventually, it's better to host this image in a VWB's public repo |
||
| restart: always | ||
| network_mode: "host" | ||
| cap_add: | ||
| - NET_ADMIN | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Net_admin is required by wondershaper sidecar |
||
| gcsfuse: | ||
| container_name: "gcsfuse" | ||
| image: "google/cloud-sdk:latest" | ||
| restart: always | ||
| volumes: | ||
| - gcsfuse-mount:/mnt/gcs | ||
| entrypoint: > | ||
| /bin/sh -c " | ||
| gcsfuse YOUR_BUCKET_NAME /mnt/gcs | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. todo - pass YOUR_BUCKET_NAME as env var from Mikey vm init script
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is a cool exploration. I think the workbench way is to use the workbench CLI and run |
||
| " | ||
| cap_add: | ||
| - SYS_ADMIN | ||
| devices: | ||
| - /dev/fuse | ||
| security_opt: | ||
| - apparmor:unconfined | ||
| networks: | ||
| app-network: | ||
| external: true | ||
| volumes: | ||
| work: | ||
| gcsfuse-mount: | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| { | ||
| "name": "Libreoffice app", | ||
| "dockerComposeFile": "docker-compose.yaml", | ||
| "service": "app", | ||
| "shutdownAction": "none", | ||
| "workspaceFolder": "/workspace", | ||
| "postCreateCommand": ["./startupscript/post-startup.sh", "abc", "/config", "${templateOption:cloud}", "${templateOption:login}"], | ||
| "postStartCommand": "/bin/bash -c 'sudo journalctl -u devcontainer.service > /tmp/system_logs.log & sleep 10; gsutil cp /tmp/system_logs.log gs://mountbucket-terra-vdevel-clean-pear-6753/debugging'", | ||
| "remoteUser": "root" | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
|
|
||
| # Rstudio (rstudio) | ||
|
|
||
| A Template to run rstudio on workbench | ||
|
|
||
| ## Options | ||
|
|
||
| | Options Id | Description | Type | Default Value | | ||
| |-----|-----|-----|-----| | ||
| | cloud | VM cloud environment | string | gcp | | ||
| | login | Whether to log in to workbench CLI | string | false | | ||
|
|
||
|
|
||
|
|
||
| --- | ||
|
|
||
| _Note: This file was auto-generated from the [devcontainer-template.json](https://github.com/verily-src/workbench-app-devcontainers/blob/main/src/rstudio/devcontainer-template.json). Add additional notes to a `NOTES.md`._ |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| { | ||
| "id": "libreoffice", | ||
| "version": "0.1.0", | ||
| "name": "Libreoffice", | ||
| "description": "A Template to libreoffice on workbench", | ||
| "documentationURL": "https://github.com/yonghaoy/workbench-app-devcontainers/tree/master/src/libreoffice", | ||
| "licenseURL": "https://github.com/yonghaoy/workbench-app-devcontainers/blob/master/LICENSE", | ||
| "options": { | ||
| "cloud": { | ||
| "type": "string", | ||
| "description": "VM cloud environment", | ||
| "proposals": [ | ||
| "gcp", | ||
| "aws" | ||
| ], | ||
| "default": "gcp" | ||
| }, | ||
| "login": { | ||
| "type": "string", | ||
| "description": "Whether to log in to workbench CLI", | ||
| "proposals": [ | ||
| "true", | ||
| "false" | ||
| ], | ||
| "default": "false" | ||
| } | ||
| }, | ||
| "platforms": [ | ||
| "Any" | ||
| ] | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| version: '1.0' | ||
| services: | ||
| app: | ||
| container_name: "application-server" | ||
| image: "lscr.io/linuxserver/libreoffice:latest" | ||
| restart: unless-stopped | ||
| ports: | ||
| - '3000:3000' | ||
| networks: | ||
| - app-network | ||
| cap_add: | ||
| - SYS_ADMIN | ||
| devices: | ||
| - /dev/fuse | ||
| security_opt: | ||
| - apparmor:unconfined | ||
| networks: | ||
| app-network: | ||
| external: true |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| { | ||
| "name": "nf-shard", | ||
| "dockerComposeFile": "docker-compose.yaml", | ||
| "service": "app", | ||
| "shutdownAction": "none", | ||
| "workspaceFolder": "/workspace", | ||
| "postCreateCommand": ["./startupscript/post-startup.sh", "abc", "/config", "${templateOption:cloud}", "${templateOption:login}"], | ||
| "postStartCommand": "/bin/bash -c 'sudo journalctl -u devcontainer.service > /tmp/system_logs.log & sleep 10; gsutil cp /tmp/system_logs.log gs://mountbucket-terra-vdevel-clean-pear-6753/debugging'", | ||
| "remoteUser": "root" | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
|
|
||
| # Rstudio (rstudio) | ||
|
|
||
| A Template to run rstudio on workbench | ||
|
|
||
| ## Options | ||
|
|
||
| | Options Id | Description | Type | Default Value | | ||
| |-----|-----|-----|-----| | ||
| | cloud | VM cloud environment | string | gcp | | ||
| | login | Whether to log in to workbench CLI | string | false | | ||
|
|
||
|
|
||
|
|
||
| --- | ||
|
|
||
| _Note: This file was auto-generated from the [devcontainer-template.json](https://github.com/verily-src/workbench-app-devcontainers/blob/main/src/rstudio/devcontainer-template.json). Add additional notes to a `NOTES.md`._ |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| { | ||
| "id": "nfshared", | ||
| "version": "0.1.0", | ||
| "name": "nf shared", | ||
| "description": "A Template to nf shared on workbench", | ||
| "documentationURL": "https://github.com/yonghaoy/workbench-app-devcontainers/tree/master/src/libreoffice", | ||
| "licenseURL": "https://github.com/yonghaoy/workbench-app-devcontainers/blob/master/LICENSE", | ||
| "options": { | ||
| "cloud": { | ||
| "type": "string", | ||
| "description": "VM cloud environment", | ||
| "proposals": [ | ||
| "gcp", | ||
| "aws" | ||
| ], | ||
| "default": "gcp" | ||
| }, | ||
| "login": { | ||
| "type": "string", | ||
| "description": "Whether to log in to workbench CLI", | ||
| "proposals": [ | ||
| "true", | ||
| "false" | ||
| ], | ||
| "default": "false" | ||
| } | ||
| }, | ||
| "platforms": [ | ||
| "Any" | ||
| ] | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| version: '1.0' | ||
| services: | ||
| app: | ||
| container_name: "application-server" | ||
| image: "augustinast/nf-shard:1.0.0" | ||
| restart: unless-stopped | ||
| ports: | ||
| - '3000:3000' | ||
| networks: | ||
| - app-network | ||
| cap_add: | ||
| - SYS_ADMIN | ||
| devices: | ||
| - /dev/fuse | ||
| security_opt: | ||
| - apparmor:unconfined | ||
| networks: | ||
| app-network: | ||
| external: true |
Uh oh!
There was an error while loading. Please reload this page.