Skip to content

Commit f3d5270

Browse files
committed
Merge branch 'dev' into beta
2 parents 508cd62 + 4aa6c19 commit f3d5270

File tree

14 files changed

+381
-281
lines changed

14 files changed

+381
-281
lines changed

.gitignore

Lines changed: 13 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,44 +1,42 @@
11

22
# ====Dockerized-NorLab============================================================================
3+
34
# ....Dev required.................................................................................
4-
/utilities/tmp/dockerized-norlab-project-mock
5+
/utilities/tmp/dockerized-norlab-project-mock/
56
!**/external_data/README.md
67
!**/artifact/README.md
78
!**/artifact/optuna_storage/README.md
89

910
# ....DNA common...................................................................................
10-
**/.dockerized_norlab/dn_container_env_variable
11-
**/external_data
12-
**/artifact
11+
**/.dockerized_norlab/dn_container_env_variable/
12+
**/external_data/
13+
**/artifact/
1314
**/slurm_jobs_logs/*.log
1415

15-
# ====General======================================================================================
16+
# ====Common=======================================================================================
1617

1718
# ....Jetbrains....................................................................................
18-
.idea
19+
.idea/
1920
*.iml
20-
out
21-
gen
2221

2322
# ....Jetbrains AI.................................................................................
24-
/.junie/iceboxed_plans
23+
/.junie/iceboxed_plans/
2524
/.junie/scratch.md
26-
!/.junie
25+
!/.junie/
2726

2827
# ....VM...........................................................................................
29-
**/.vagrant
28+
**/.vagrant/
3029

3130
# ....Python related...............................................................................
32-
**/.pytest_cache
31+
**/.pytest_cache/
3332
**/.benchmarks
3433
**/__pycache__
3534
**/*.pyc
3635

3736
# ....Log..........................................................................................
37+
out/
3838
**/*.out
3939
**/*.log
4040

41-
# ....General......................................................................................
41+
# ....OS...........................................................................................
4242
**/.DS_Store
43-
44-

.junie/guidelines.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -80,8 +80,8 @@ All instructions in sections _General Testing Instructions_ plus the following:
8080

8181
### Shell Script specific Mocking Instructions
8282
All instructions in sections _General Mocking Instruction_ plus the following:
83-
- You can mock shell core command an docker command.
8483
- You can mock `docker [OPTIONS|COMMAND]` commands and `git [OPTIONS|COMMAND]` commands.
84+
- Ask permission before mocking shell builtin commands.
8585
- Avoid mocking N2ST functions, at the exception of those in `${N2ST_PATH}/src/function_library/prompt_utilities.bash`. For example, instead of re-implementing `n2st::seek_and_modify_string_in_file`, just load the real one and test that the content of the file at `file_path` has been updated? You can find the real one in `${N2ST_PATH}/src/function_library/general_utilities.bash`.
8686
- Avoid mocking the `read` command. Instead use `echo 'y'` or `echo 'N'` for piping a keyboard input to the function who use the `read` command which in turn expect a single character, example: `run bash -c "echo 'y' | <the-tested-function>"`. Alternatively, use the `yes [n]` shell command which optionaly send [y|Y|yes] n time, example: `run bash -c "yes 2 | <the-tested-function>"`.
8787
- Use `timeouts 10 ` in integration tests that execute real scripts with mocked dependencies to prevent test hangs.

documentation/legacy_doc/README.md

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# Dockerized-NorLab project application (DNA)
2+
Reference: https://github.com/norlab-ulaval/dockerized-norlab-project.git
3+
4+
## Usage
5+
6+
1. Setup/validate `.dockerized_norlab/configuration/` files:
7+
- Setup dotenv files: `.env`, `.env.dna` and `.env.local`;
8+
- Customize files in `project_requirements/`;
9+
- Customize files in `project_entrypoints/`. Add
10+
project-specific container runtime logic;
11+
- Customize `Dockerfile` to fit your need. It should work out of the box for most use cases;
12+
- Check `.dockerized_norlab/configuration/README.md` for more details.
13+
2. From your project `root`, execute the following
14+
```shell
15+
dna help
16+
17+
# Build your DN-project containers
18+
dna build
19+
20+
# Start your DN-project containers
21+
dna up
22+
23+
# Have fun
24+
# When your done, execute
25+
dna down
26+
```
27+
28+
## Requirements:
29+
30+
- dependencies:
31+
- [Docker](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository) with
32+
`docker-buildx-plugin` and `docker-compose-plugin`
33+
- [nvidia-docker](https://github.com/NVIDIA/nvidia-docker) for GPU usage
34+
- The directory `.dockerized_norlab` need to be at the super-project repository root
35+
36+
# Setup PyCharm IDE
37+
38+
### Remote development
39+
- To work in a docker container on a remote machine, setup a remote developement configuration pointing to `<remote-machine-user>@<target-host-IP>:<remote-machine-ssh-port>` e.g.: `redleader@192.168.0.103:22`.
40+
- Set `rsync` ON and sudo OFF
41+
- Make sure `rsync` DO copy the `.git` directory to the remote host i.e. check if it is in the rsync excluded item list
42+
- **Remote display**:
43+
- Start your DN container project using command `dna up` or `dna run`
44+
- In the remote docker container terminal, execute command alias `dn_expose_container_env_variables` and download `.dockerized_norlab/dn_container_env_variable/.env.dn_expose_PLACEHOLDER_DN_CONTAINER_NAME` to your source machine
45+
and set `Paths to ".env" files` in run configuration window
46+
![set_interpreter_env.png](visual/set_run_config_env.png)
47+
48+
#### Rsync to remote machine
49+
- If you encounter rsync error such as the infamous `exit code 23`, check file/dir permission and group by executing `tree -aug <path/to/remote/project/dir>` and update them if needed by executing `sudo chown -R $(id -un):$(id -gn) <path/to/remote/project/dir>`.
50+
51+
### Python interpreter
52+
- Set up a ssh remote python interpreter pointing to `<dockerized-norlab-ssh-user>@<target-host-IP>:<dockerized-norlab-container-ssh-port>` e.g.: `pycharm-debugger@192.168.0.103:2222`. Note: use `localhost` explicitly if the container is running on your local machine i.e.: `pycharm-debugger@localhost:2222`
53+
- Remote python interpreter, i.e. `<user-name>@<remote-ip-adress>:2222`:
54+
- ⚠️ Uncheck `Execute code with root privileges` otherwise experimental data writen to disk will be owned by `root` instead of `PLACEHOLDER_SUPER_PROJECT_USER`
55+
- ⚠️ Uncheck `Automatically upload project files`. The project is mounted as a docker volume for develop images and is copied at build stage for deploy images.
56+
- Set path mapping to `/ros2_ws/src/PLACEHOLDER_SUPER_PROJECT_NAME` or delete the `Sync folders` entry.
57+
![python_interpreter_final_setup.png](visual/python_interpreter_final_setup.png)
58+
- Don't forget to add `/opt/ros/<distro>/lib/python<version>/site-packages` to the _Interpreter
59+
path_ once the docker based ssh interpreter is configured,
60+
e.g.: `/opt/ros/foxy/lib/python3.8/site-packages`
61+
62+
![python_interpreter.png](visual/python_interpreter.png)
63+
![interpreter_path.png](visual/interpreter_path.png)
64+
65+
### Run/debug configuration
66+
67+
- Don't forget to set run/debug configuration `path mapping` to: `~/PycharmProjects/PLACEHOLDER_SUPER_PROJECT_NAME` -> `/ros2_ws/src/PLACEHOLDER_SUPER_PROJECT_NAME`. It's critical for debug breakpoint remote/local resolution to work in regular/pytest run, regular/pytest debug and for IDE debug stacktrace remote/local code resolution for jumping to source code from stacktrace link.
68+
![img.png](visual/run_config_panel.png)

src/lib/template/.dockerized_norlab/visual/interpreter_path.png renamed to documentation/legacy_doc/visual/interpreter_path.png

File renamed without changes.

src/lib/template/.dockerized_norlab/visual/python_interpreter.png renamed to documentation/legacy_doc/visual/python_interpreter.png

File renamed without changes.

src/lib/template/.dockerized_norlab/visual/python_interpreter_final_setup.png renamed to documentation/legacy_doc/visual/python_interpreter_final_setup.png

File renamed without changes.

src/lib/template/.dockerized_norlab/visual/run_config_panel.png renamed to documentation/legacy_doc/visual/run_config_panel.png

File renamed without changes.

src/lib/template/.dockerized_norlab/visual/set_run_config_env.png renamed to documentation/legacy_doc/visual/set_run_config_env.png

File renamed without changes.

documentation/project_initialization_and_configuration.md

Lines changed: 19 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -54,24 +54,25 @@ After running `dna init`, your project will have the following structure:
5454

5555
```
5656
your-project-repository/
57-
├── .dockerized_norlab/ ← DNA configuration directory
58-
│ ├── configuration/ ← Main configuration files
59-
│ │ ├── .env ← Project environment variables
60-
│ │ ├── .env.dna ← DNA-specific variables
61-
│ │ ├── .env.local ← Local development overrides
62-
│ │ ├── Dockerfile ← Container build instructions
63-
│ │ ├── README.md ← Configuration documentation
64-
│ │ ├── project_entrypoints/ ← Container startup scripts
65-
│ │ └── project_requirements/ ← Dependency specifications
66-
│ ├── dn_container_env_variable/ ← Container environment exports
67-
│ └── visual/ ← Project-specific visuals
68-
├── artifact/ ← Runtime produced data (mounted)
69-
├── external_data/ ← Pre-existing data (mounted)
70-
├── src/ ← Your source code (mounted/copied)
71-
├── tests/ ← Your test code (mounted/copied)
72-
├── .dockerignore ← Docker build exclusions
73-
├── .gitignore ← Git exclusions
74-
└── README.md ← Project documentation
57+
├── .dockerized_norlab/ ← DNA configuration directory
58+
│ ├── configuration/ ← Main configuration files
59+
│ │ ├── project_entrypoints/ ← Container startup scripts
60+
│ │ ├── project_requirements/ ← Dependency specifications
61+
│ │ ├── Dockerfile ← Container build instructions
62+
│ │ ├── .env ← Project environment variables
63+
│ │ ├── .env.dna ← DNA-specific variables
64+
│ │ ├── .env.local ← Local development overrides
65+
│ │ └── README.md ← Configuration documentation
66+
│ ├── dn_container_env_variable/ ← Container environment exports
67+
│ ├── .env.your-project-repository ← Project DNA configuration meta
68+
│ └── README.md ← DNA configuration quick documentation
69+
├── artifact/ ← Runtime produced data (mounted)
70+
├── external_data/ ← Pre-existing data (mounted)
71+
├── src/ ← Your source code (mounted/copied)
72+
├── tests/ ← Your test code (mounted/copied)
73+
├── .dockerignore ← Docker build exclusions
74+
├── .gitignore ← Git exclusions
75+
└── README.md ← Project documentation
7576
```
7677

7778
### Directory Purposes

0 commit comments

Comments
 (0)