Skip to content

Commit 9d7262c

Browse files
committed
feat: DNA initialize math-gymnasium project
- Introduced configuration for Dockerized-NorLab project (DNA) setup. - Added PyTorch and CUDA verification scripts for dependency checks. - Included hyperparameter optimization pipelines via Hydra and Optuna. - Implemented CI test scripts for streamlined pytest execution. - Configured README documentation for project structure guidance.
1 parent 86416e1 commit 9d7262c

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

46 files changed

+1466
-6
lines changed

.dockerignore

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# =================================================================================================
2+
#
3+
# Specify files/directories to be ignored by the Dockerfile COPY/ADD clause
4+
# Note: content from "volumes" will still be accessible inside the container
5+
#
6+
# Syntax:
7+
# "*" match any number of characters
8+
# "?" match one character
9+
# "!" prevent ignore
10+
# "**" match any number of directories
11+
#
12+
# References: https://docs.docker.com/build/building/context/#dockerignore-files
13+
#
14+
# =================================================================================================
15+
16+
# ====Dockerized-NorLab(required)==================================================================
17+
!**/.dockerized_norlab/
18+
!**/version.txt
19+
!**/.git
20+
21+
# ====Dockerized-NorLab(recommended)===============================================================
22+
**/external_data/
23+
**/artifact/
24+
**/slurm_jobs_logs/*.log
25+
26+
# ====General======================================================================================
27+
# ....Repository related...........................................................................
28+
.github
29+
.releaserc.json
30+
CHANGELOG.md
31+
commit_msg_reference.md
32+
visual/
33+
34+
# ....Graphic/image related........................................................................
35+
**/*.svg
36+
**/*.svgz
37+
**/*.drawio
38+
**/drawio/
39+
40+
# ....Text related.................................................................................
41+
**/*.pdf
42+
43+
# ....Python related...............................................................................
44+
**/.pytest_cache
45+
**/.benchmarks
46+
**/__pycache__
47+
**/*.pyc
48+
49+
# ....Jetbrains....................................................................................
50+
/.run/
51+
/.idea/
52+
53+
# ....Log..........................................................................................
54+
**/*.out
55+
56+
# ....General......................................................................................
57+
**/.DS_Store
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# =================================================================================================
2+
# Dockerized-NorLab project application (DNA) super project configuration meta information(s).
3+
#
4+
# Reference: https://github.com/norlab-ulaval/dockerized-norlab-project
5+
# =================================================================================================
6+
DNA_CONFIG_SCHEME_VERSION=1

.dockerized_norlab/README.md

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# Dockerized-NorLab project application (DNA)
2+
3+
https://github.com/norlab-ulaval/dockerized-norlab-project.git
4+
5+
6+
## Getting started ... fast
7+
Check [DNA documentation](https://github.com/norlab-ulaval/dockerized-norlab-project?tab=readme-ov-file#documentation) for details
8+
9+
1. Setup/validate `.dockerized_norlab/configuration/` files:
10+
- Setup dotenv files: `.env`, `.env.dna` and `.env.local`;
11+
- Customize files in `project_requirements/`;
12+
- Customize files in `project_entrypoints/`. Add
13+
project-specific container runtime logic;
14+
- Customize `Dockerfile` to fit your need. It should work out of the box for most use cases;
15+
- Check `.dockerized_norlab/configuration/README.md` for more details.
16+
2. From your project `root`, execute the following
17+
```shell
18+
dna help
19+
20+
# Build your DN-project containers
21+
dna build
22+
23+
# Start your DN-project containers
24+
dna up
25+
26+
# Have fun
27+
# When your done, execute
28+
dna down
29+
```
Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# =================================================================================================
2+
# Set run time environment variable.
3+
#
4+
# Notes:
5+
# - DNA dotenv file loading precedence:
6+
# 1. .env.dna
7+
# 2. .env
8+
# 3. .env.local
9+
# 4. .env.dna-internal (DNA repository)
10+
#
11+
# =================================================================================================
12+
13+
# ....GPU..........................................................................................
14+
# Reference: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/docker-specialized.html
15+
16+
# Nvidia visible devices options: "all", "none", "void", gpu UUID e.g., "GPU-fef8089b or gpu indexes list e.g., "0,1,2"
17+
#NVIDIA_VISIBLE_DEVICES=all
18+
19+
# Nvidia driver capabilities options: "all" or a list of comma-separated driver's name e.g., "compute,utility"
20+
# Available driver: compute, compat32, graphics, utility, video or display
21+
#NVIDIA_DRIVER_CAPABILITIES=all
22+
23+
# ....ROS..........................................................................................
24+
#ROS_DOMAIN_ID=1
25+
26+
# Enable ROS2 log collouring
27+
#RCUTILS_COLORIZED_OUTPUT=1
28+
29+
# rmw implementation options: rmw_fastrtps_cpp, rmw_cyclonedds_cpp
30+
#RMW_IMPLEMENTATION=rmw_cyclonedds_cpp
31+
32+
#DDS_NETWORK_INTERFACE=eth0
33+
#CYCLONEDDS_URI="<CycloneDDS><Domain><General><NetworkInterface>${DDS_NETWORK_INTERFACE:?err}</></></></>"
34+
# Fix for warning "ros2: using network interface eth0 (udp/169.254.205.89) selected
35+
# arbitrarily from: eth0, wlan0, docker0".
36+
# Solution ref: https://answers.ros.org/question/375360/multiple-network-interfaces-with-rmw_cyclonedds_cpp/
37+
38+
# ....Python.......................................................................................
39+
#PYTHONUNBUFFERED=1
40+
#PYCHARM_DEBUG=1
41+
#PYTEST_DEBUG=1
42+
43+
# ....Hydra........................................................................................
44+
#HYDRA_FULL_ERROR=1
45+
46+
# Set omegaconf full error backtrace
47+
#OC_CAUSE=1
Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
# =================================================================================================
2+
# Set Dockerized-NorLab project application (DNA) environment variable.
3+
#
4+
# Notes:
5+
# - This dotenv file is use both at buildtime and runtime
6+
# - DNA dotenv file loading precedence:
7+
# 1. .env.dna
8+
# 2. .env
9+
# 3. .env.local
10+
# 4. .env.dna-internal (DNA repository)
11+
#
12+
# =================================================================================================
13+
14+
# ....Console option...............................................................................
15+
# Un-comment to override
16+
17+
#DN_ENTRYPOINT_TRACE_EXECUTION=true
18+
#DN_SHOW_DEBUG_INFO=true
19+
20+
# DNA container prompt style
21+
# Requirement:
22+
# - Ubuntu: "$ sudo apt-get install fonts-powerline" and select patched font in terminal, example "Ubuntu mono"
23+
# - MacOs: Any font from list "$ brew search nerd-font" -> "$ brew install font-FONT-NAME-nerd-font"
24+
# Example "$ brew install font-jetbrains-mono-nerd-font" and select patched font in terminal, "JetBrainsMon Nerd Font Mono"
25+
# Check preview at https://www.nerdfonts.com/font-downloads
26+
#DN_ACTIVATE_POWERLINE_PROMT=true
27+
28+
# ....Repository config............................................................................
29+
DN_PROJECT_GIT_REMOTE_URL=https://github.com/norlab-ulaval/math-gymnasium.git
30+
31+
# ....Docker config................................................................................
32+
DN_CONTAINER_NAME="IamDNA_mg"
33+
DN_PROJECT_ALIAS_PREFIX="mg"
34+
35+
# Set the project core base image
36+
# Un-comment to override (default to 'dockerized-norlab-dependencies-full')
37+
#DN_PROJECT_BASE_IMG=dockerized-norlab-dependencies-full
38+
39+
# Set the Dockerized-NorLab base image version
40+
# Use one of the following:
41+
# - 'latest' is the latest push to the repository default branch
42+
# - 'bleeding' is the latest push to the repository dev branch (i.e. bleeding-edge)
43+
# - you can use a specific release version tag number, e.g., 'v0.7.0'
44+
# - un-comment to override (default to 'latest')
45+
#DN_VERSION=bleeding
46+
47+
# This the Docker hub from where your project images will be pushed to and pulled from.
48+
# Un-comment to override (default to norlabulaval)
49+
#DN_PROJECT_HUB=norlabulaval
50+
51+
# ....Development config...........................................................................
52+
# Special user with dedicated shell configuration tailormade for ROS2 remote development
53+
#DN_SSH_SERVER_USER="${DN_PROJECT_ALIAS_PREFIX}-non-interactive-ros2"
54+
55+
# Port assignement
56+
#DN_SSH_SERVER_PORT=2222
57+
#DN_GDB_SERVER_PORT=7777
58+
59+
# Set the ssh daemon in no detach mode for server operation monitoring
60+
#DN_SSH_DAEMON_NO_DETACH=true
61+
62+
# ....deploy config................................................................................
63+
# Path to the src code that will be cloned into the DN project-deploy image
64+
# - Un-comment to override (default to repository root)
65+
#DN_PROJECT_DEPLOY_SRC_PATH="${SUPER_PROJECT_ROOT:?err}/arbitrary/path"
66+
67+
# Branch checkout for the project-deploy image
68+
# - Options: 'main', 'dev', '<feature-branch>' or tag version (ie 'v*.*.*')
69+
# - Un-comment to override (default to 'main')
70+
# - Automaticaly use the current checkout branch otherwise
71+
#DN_PROJECT_DEPLOY_REPO_BRANCH=dev
72+
73+
# ====Dockerized-NorLab============================================================================
74+
75+
# ....Tag..........................................................................................
76+
# The Dockerized-NorLab base image tag is generated by DNA internaly using various environment variables either set by the user or set by DNA.
77+
# The gerated base image tag and base image name follow this patern:
78+
#
79+
# <dn-project-hub>/<dn-project-base-img>:DN-<dn-version>-<tag-package>-<tag-os-version>
80+
#
81+
# Example:
82+
# norlabulaval/dockerized-norlab-dependencies-full:DN-bleeding-galactic-desktop-l4t-pytorch-r35.4.1
83+
#
84+
# See https://hub.docker.com/repositories/norlabulaval for available base images.
85+
86+
# The Dockerized-NorLab base image tag package name
87+
# Un-comment to override
88+
#TAG_PACKAGE=humble-base-l4t-pytorch
89+
90+
# The Linux-for-Tegra (Jetson OS) container tag version
91+
# Un-comment to override
92+
#TAG_OS_VERSION=r36.4.0
93+
94+
# ....Main config..................................................................................
95+
# Set the Dockerized-NorLab docker image hub where to pull DN_PROJECT_BASE_IMG from
96+
# Un-comment to override (default to 'norlabulaval')
97+
#DN_HUB=norlabulaval
98+
99+
# Set the Dockerized-NorLab repository branch for container internal tools
100+
# Accept 'main', 'dev', '<feature-branch-name>' or tag version 'v*.*.*'
101+
# Un-comment to override (default to 'main')
102+
#DN_GIT_BRANCH=dev
103+
104+
# ====Docker CLI/config/internal===================================================================
105+
# Docker context global overide
106+
# Un-comment to set in DNA
107+
#DOCKER_CONTEXT=desktop-linux
108+
109+
# Docker builder global overide.
110+
# Un-comment to set in DNA
111+
#BUILDX_BUILDER=desktop-linux
112+
# Note: DNA 'execute_compose.bash' automaticaly select the native builder if BUILDX_BUILDER is not set.
113+
114+
# For debuging docker build localy, use `docker build --progress=plain --no-cache` flags instead
115+
# Un-comment to set in DNA for debugging dockerfile
116+
#BUILDKIT_PROGRESS=plain
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# =================================================================================================
2+
# Set host bound environment variable.
3+
#
4+
# Notes:
5+
# - This file is added to '.gitignore' by default so any modification made here will only affect
6+
# the current host.
7+
# - This dotenv file is loaded last so it can be use as an environment variables override file.
8+
# DNA dotenv file loading precedence:
9+
# 1. .env.dna
10+
# 2. .env
11+
# 3. .env.local
12+
# 4. .env.dna-internal (DNA repository)
13+
#
14+
# =================================================================================================
Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
# =================================================================================================
2+
#
3+
# 👍 You can change the code in this file to take advantage of the Docker cache layer mechanism
4+
# and the Docker multi-stage build feature.
5+
#
6+
# =================================================================================================
7+
ARG BASE_IMAGE
8+
ARG BASE_IMAGE_TAG
9+
FROM ${BASE_IMAGE:?err}:${BASE_IMAGE_TAG:?err} AS user-project-custom-steps
10+
11+
ARG TARGETPLATFORM
12+
ARG BUILDPLATFORM
13+
WORKDIR ${DN_PROJECT_PATH:?'environment variable is not set'}
14+
15+
# ADD YOUR CODE HERE
16+
# ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
17+
# ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
18+
19+
# Example
20+
RUN <<EOF
21+
# ....Examples.................................................................................
22+
{
23+
echo "...Sanity check............................" && \
24+
echo "python version: $(n2st::which_python3_version)" && \
25+
python -c "import pytest" && \
26+
echo "..........................................." ;
27+
} || n2st::print_msg_error_and_exit "Failed sanity check!"
28+
EOF
29+
30+
RUN <<EOF
31+
# ....Examples with cuda.......................................................................
32+
# Note: Assuming the base image is nvidia cuda enable and has pycuda and pytorch installed
33+
if pip -qq show torch; then
34+
{
35+
echo "..........................................." && \
36+
echo "Sanity check" && \
37+
python -c "import torch" && \
38+
echo "..........................................." ;
39+
} || n2st::print_msg_error_and_exit "Failed torch sanity check!"
40+
(
41+
echo
42+
echo "alias dn-pytorch-cuda-check='python3 /ros2_ws/src/dockerized-norlab-project-mock/src/dna_example/try_pytorch.py'"
43+
echo
44+
) >> /dockerized-norlab/dockerized-norlab-images/container-tools/dn_bash_alias.bash
45+
fi
46+
47+
if pip -qq show pycuda; then
48+
{
49+
echo "..........................................." && \
50+
echo "Sanity check" && \
51+
python -c "import pycuda" && \
52+
echo "..........................................." ;
53+
} || n2st::print_msg_error_and_exit "Failed pycuda sanity check!"
54+
(
55+
echo
56+
echo "alias dn-pycuda-check='python3 /ros2_ws/src/dockerized-norlab-project-mock/src/dna_example/try_pycuda.py'"
57+
echo
58+
) >> /dockerized-norlab/dockerized-norlab-images/container-tools/dn_bash_alias.bash
59+
fi
60+
EOF
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
2+
#### DNA dotenv file loading precedence:
3+
4+
1. .env.dna
5+
2. .env
6+
3. .env.local
7+
4. .env.dna-internal (DNA repo)
8+
9+
#### References
10+
- [Docker-Compose environment variables precedence](https://docs.docker.com/compose/how-tos/environment-variables/envvars-precedence/)
11+
- Docker predefined environment variables
12+
- [Docker](https://docs.docker.com/reference/cli/docker/)
13+
- [Docker-Compose](https://docs.docker.com/compose/how-tos/environment-variables/envvars/)
Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
#!/bin/bash
2+
# =================================================================================================
3+
# Dockerized-NorLab container runtime entrypoint callback.
4+
# Is executed each time a shell is attach to a project-develop or project-deploy container.
5+
#
6+
# Usage:
7+
# Add project wide logic that need to be executed by each shell.
8+
#
9+
# Globals:
10+
# Read/write all environment variable exposed in DN at runtime
11+
#
12+
# =================================================================================================
13+
14+
# ....DNA-project internal logic...................................................................
15+
source /dna-lib-container-tools/project_entrypoints/entrypoint_helper.global.common.bash || exit 1
16+
17+
# ====DNA-project user defined logic===============================================================
18+
# Add your code here
19+
20+
# ....Examples: source ROS2 environment variables..................................................
21+
#dn::source_ros2_underlay_only
22+
#dn::source_ros2_overlay_only
23+
dn::source_ros2

0 commit comments

Comments
 (0)