Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions src/jupyter-aou/.devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"name": "Jupyter app for AoU",
"dockerComposeFile": "docker-compose.yaml",
"service": "app",
"shutdownAction": "none",
"workspaceFolder": "/workspace",
"postCreateCommand": ["./startupscript/post-startup.sh", "jovyan", "/home/jovyan", "${templateOption:cloud}", "${templateOption:login}"],
// re-mount bucket files on container start up
"postStartCommand":["./startupscript/remount-on-restart.sh", "jovyan", "/home/jovyan", "${templateOption:cloud}", "${templateOption:login}"],
"features": {
"ghcr.io/devcontainers/features/aws-cli:1": {},
"ghcr.io/dhoeric/features/google-cloud-cli:1": {},
"ghcr.io/devcontainers/features/java:1": {}
},
"remoteUser": "root"
}
17 changes: 17 additions & 0 deletions src/jupyter-aou/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@

# Jupyter (jupyter)

A Template to run jupyter on workbench

## Options

| Options Id | Description | Type | Default Value |
|-----|-----|-----|-----|
| cloud | VM cloud environment | string | gcp |
| login | Whether to log in to workbench CLI | string | false |



---

_Note: This file was auto-generated from the [devcontainer-template.json](https://github.com/verily-src/workbench-app-devcontainers/blob/main/src/jupyter/devcontainer-template.json). Add additional notes to a `NOTES.md`._
31 changes: 31 additions & 0 deletions src/jupyter-aou/devcontainer-template.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
{
"id": "jupyter",
"version": "0.0.2",
"name": "Jupyter",
"description": "A Template to run jupyter on workbench",
"documentationURL": "https://github.com/verily-src/workbench-app-devcontainers/tree/master/src/jupyter",
"licenseURL": "https://github.com/verily-src/workbench-app-devcontainers/blob/master/LICENSE",
"options": {
"cloud": {
"type": "string",
"description": "VM cloud environment",
"proposals": [
"gcp",
"aws"
],
"default": "gcp"
},
"login": {
"type": "string",
"description": "Whether to log in to workbench CLI",
"proposals": [
"true",
"false"
],
"default": "false"
}
},
"platforms": [
"Any"
]
}
50 changes: 50 additions & 0 deletions src/jupyter-aou/docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
version: '2.4'
services:
app:
container_name: "application-server"
image: "gcr.io/deeplearning-platform-release/tf-gpu.2-11.py310"
restart: always
volumes:
- .:/workspace:cached
- work:/home/jovyan/work
- gcsfuse-mount:/mnt/gcs
ports:
- '8888:8888'
environment:
JUPYTER_ENABLE_LAB: "yes"
command: "start-notebook.py --ServerApp.token='' --ServerApp.password='' --ServerApp.root_dir=/home/jovyan/work --ServerApp.allow_origin='*'"
networks:
- app-network
devices:
- /dev/fuse
security_opt:
- apparmor:unconfined
wondershaper:
container_name: "wondershaper"
image: "us.gcr.io/broad-dsp-gcr-public/wondershaper:0.0.1"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondershaper for rate limiting. Eventually, it's better to host this image in a VWB's public repo

restart: always
network_mode: "host"
cap_add:
- NET_ADMIN
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Net_admin is required by wondershaper sidecar

gcsfuse:
container_name: "gcsfuse"
image: "google/cloud-sdk:latest"
restart: always
volumes:
- gcsfuse-mount:/mnt/gcs
entrypoint: >
/bin/sh -c "
gcsfuse YOUR_BUCKET_NAME /mnt/gcs
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

todo - pass YOUR_BUCKET_NAME as env var from Mikey vm init script

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a cool exploration. I think the workbench way is to use the workbench CLI and run wb resource mount, which mounts all resource buckets in the workspace. This means we may need to build another image that specifically does this. The other problem is that now that users cannot remount buckets themselves, we'll need a cron in the mount sidecar that tries to remount every hour or so.

"
cap_add:
- SYS_ADMIN
devices:
- /dev/fuse
security_opt:
- apparmor:unconfined
networks:
app-network:
external: true
volumes:
work:
gcsfuse-mount:
10 changes: 10 additions & 0 deletions src/libreoffice/.devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"name": "Libreoffice app",
"dockerComposeFile": "docker-compose.yaml",
"service": "app",
"shutdownAction": "none",
"workspaceFolder": "/workspace",
"postCreateCommand": ["./startupscript/post-startup.sh", "abc", "/config", "${templateOption:cloud}", "${templateOption:login}"],
"postStartCommand": "/bin/bash -c 'sudo journalctl -u devcontainer.service > /tmp/system_logs.log & sleep 10; gsutil cp /tmp/system_logs.log gs://mountbucket-terra-vdevel-clean-pear-6753/debugging'",
"remoteUser": "root"
}
17 changes: 17 additions & 0 deletions src/libreoffice/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@

# Rstudio (rstudio)

A Template to run rstudio on workbench

## Options

| Options Id | Description | Type | Default Value |
|-----|-----|-----|-----|
| cloud | VM cloud environment | string | gcp |
| login | Whether to log in to workbench CLI | string | false |



---

_Note: This file was auto-generated from the [devcontainer-template.json](https://github.com/verily-src/workbench-app-devcontainers/blob/main/src/rstudio/devcontainer-template.json). Add additional notes to a `NOTES.md`._
31 changes: 31 additions & 0 deletions src/libreoffice/devcontainer-template.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
{
"id": "libreoffice",
"version": "0.1.0",
"name": "Libreoffice",
"description": "A Template to libreoffice on workbench",
"documentationURL": "https://github.com/yonghaoy/workbench-app-devcontainers/tree/master/src/libreoffice",
"licenseURL": "https://github.com/yonghaoy/workbench-app-devcontainers/blob/master/LICENSE",
"options": {
"cloud": {
"type": "string",
"description": "VM cloud environment",
"proposals": [
"gcp",
"aws"
],
"default": "gcp"
},
"login": {
"type": "string",
"description": "Whether to log in to workbench CLI",
"proposals": [
"true",
"false"
],
"default": "false"
}
},
"platforms": [
"Any"
]
}
19 changes: 19 additions & 0 deletions src/libreoffice/docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
version: '1.0'
services:
app:
container_name: "application-server"
image: "lscr.io/linuxserver/libreoffice:latest"
restart: unless-stopped
ports:
- '3000:3000'
networks:
- app-network
cap_add:
- SYS_ADMIN
devices:
- /dev/fuse
security_opt:
- apparmor:unconfined
networks:
app-network:
external: true
10 changes: 10 additions & 0 deletions src/nfshare/.devcontainer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"name": "nf-shard",
"dockerComposeFile": "docker-compose.yaml",
"service": "app",
"shutdownAction": "none",
"workspaceFolder": "/workspace",
"postCreateCommand": ["./startupscript/post-startup.sh", "abc", "/config", "${templateOption:cloud}", "${templateOption:login}"],
"postStartCommand": "/bin/bash -c 'sudo journalctl -u devcontainer.service > /tmp/system_logs.log & sleep 10; gsutil cp /tmp/system_logs.log gs://mountbucket-terra-vdevel-clean-pear-6753/debugging'",
"remoteUser": "root"
}
17 changes: 17 additions & 0 deletions src/nfshare/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@

# Rstudio (rstudio)

A Template to run rstudio on workbench

## Options

| Options Id | Description | Type | Default Value |
|-----|-----|-----|-----|
| cloud | VM cloud environment | string | gcp |
| login | Whether to log in to workbench CLI | string | false |



---

_Note: This file was auto-generated from the [devcontainer-template.json](https://github.com/verily-src/workbench-app-devcontainers/blob/main/src/rstudio/devcontainer-template.json). Add additional notes to a `NOTES.md`._
31 changes: 31 additions & 0 deletions src/nfshare/devcontainer-template.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
{
"id": "nfshared",
"version": "0.1.0",
"name": "nf shared",
"description": "A Template to nf shared on workbench",
"documentationURL": "https://github.com/yonghaoy/workbench-app-devcontainers/tree/master/src/libreoffice",
"licenseURL": "https://github.com/yonghaoy/workbench-app-devcontainers/blob/master/LICENSE",
"options": {
"cloud": {
"type": "string",
"description": "VM cloud environment",
"proposals": [
"gcp",
"aws"
],
"default": "gcp"
},
"login": {
"type": "string",
"description": "Whether to log in to workbench CLI",
"proposals": [
"true",
"false"
],
"default": "false"
}
},
"platforms": [
"Any"
]
}
19 changes: 19 additions & 0 deletions src/nfshare/docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
version: '1.0'
services:
app:
container_name: "application-server"
image: "augustinast/nf-shard:1.0.0"
restart: unless-stopped
ports:
- '3000:3000'
networks:
- app-network
cap_add:
- SYS_ADMIN
devices:
- /dev/fuse
security_opt:
- apparmor:unconfined
networks:
app-network:
external: true
2 changes: 1 addition & 1 deletion startupscript/post-startup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ fi
#############################
# Mount buckets
#############################
source "${CLOUD_SCRIPT_DIR}/resource-mount.sh"
# source "${CLOUD_SCRIPT_DIR}/resource-mount.sh"

###############################
# cloud platform specific setup
Expand Down