Skip to content
This repository was archived by the owner on Nov 30, 2023. It is now read-only.

Commit fcfe37b

Browse files
dnastraindnastrain
authored
Adding PostgreSQL support to Miniconda/Python3 Dev Container (#1282)
1 parent be0c74e commit fcfe37b

File tree

14 files changed

+559
-0
lines changed

14 files changed

+559
-0
lines changed

‎containers/python-3-miniconda-postgres/.devcontainer/.env

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
POSTGRES_USER=postgres
2+
POSTGRES_PASSWORD=postgres
3+
POSTGRES_DB=postgres
4+
POSTGRES_HOST=localhost

‎containers/python-3-miniconda-postgres/.devcontainer/Dockerfile

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
FROM mcr.microsoft.com/vscode/devcontainers/miniconda:0-3
2+
3+
# [Choice] Node.js version: none, lts/*, 16, 14, 12, 10
4+
ARG NODE_VERSION="none"
5+
RUN if [ "${NODE_VERSION}" != "none" ]; then su vscode -c "umask 0002 && . /usr/local/share/nvm/nvm.sh && nvm install ${NODE_VERSION} 2>&1"; fi
6+
7+
# Copy environment.yml (if found) to a temp location so we update the environment. Also
8+
# copy "noop.txt" so the COPY instruction does not fail if no environment.yml exists.
9+
COPY environment.yml* .devcontainer/noop.txt /tmp/conda-tmp/
10+
RUN if [ -f "/tmp/conda-tmp/environment.yml" ]; then umask 0002 && /opt/conda/bin/conda env update -n base -f /tmp/conda-tmp/environment.yml; fi \
11+
&& rm -rf /tmp/conda-tmp
12+
13+
# [Optional] Uncomment to install a different version of Python than the default
14+
# RUN conda install -y python=3.6 \
15+
# && pip install --no-cache-dir pipx \
16+
# && pipx reinstall-all
17+
18+
# [Optional] Uncomment this section to install additional OS packages.
19+
# RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
20+
# && apt-get -y install --no-install-recommends <your-package-list-here>

‎containers/python-3-miniconda-postgres/.devcontainer/devcontainer.json

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
{
2+
"name": "Miniconda (Python 3) & PostgreSQL",
3+
"dockerComposeFile": "docker-compose.yml",
4+
"service": "app",
5+
"workspaceFolder": "/workspace",
6+
7+
// Set *default* container specific settings.json values on container create.
8+
"settings": {
9+
"python.defaultInterpreterPath": "/opt/conda/bin/python",
10+
"python.linting.enabled": true,
11+
"python.linting.pylintEnabled": true,
12+
"python.formatting.autopep8Path": "/usr/local/py-utils/bin/autopep8",
13+
"python.formatting.blackPath": "/usr/local/py-utils/bin/black",
14+
"python.formatting.yapfPath": "/usr/local/py-utils/bin/yapf",
15+
"python.linting.banditPath": "/usr/local/py-utils/bin/bandit",
16+
"python.linting.flake8Path": "/usr/local/py-utils/bin/flake8",
17+
"python.linting.mypyPath": "/usr/local/py-utils/bin/mypy",
18+
"python.linting.pycodestylePath": "/usr/local/py-utils/bin/pycodestyle",
19+
"python.linting.pydocstylePath": "/usr/local/py-utils/bin/pydocstyle",
20+
"python.linting.pylintPath": "/usr/local/py-utils/bin/pylint"
21+
},
22+
23+
// Add the IDs of extensions you want installed when the container is created.
24+
"extensions": [
25+
"ms-python.python",
26+
"ms-python.vscode-pylance"
27+
],
28+
29+
// Use 'forwardPorts' to make a list of ports inside the container available locally.
30+
// "forwardPorts": [5432],
31+
32+
// Use 'postCreateCommand' to run commands after the container is created.
33+
// "postCreateCommand": "python --version",
34+
35+
// Comment out to connect as root instead. More info: https://aka.ms/vscode-remote/containers/non-root.
36+
"remoteUser": "vscode"
37+
}

‎containers/python-3-miniconda-postgres/.devcontainer/docker-compose.yml

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
version: '3.8'
2+
3+
services:
4+
app:
5+
build:
6+
context: ..
7+
dockerfile: .devcontainer/Dockerfile
8+
args:
9+
# Optional Node.js version to install
10+
NODE_VERSION: "lts/*"
11+
env_file:
12+
- .env
13+
14+
volumes:
15+
- ..:/workspace:cached
16+
17+
# Overrides default command so things don't shut down after the process ends.
18+
command: sleep infinity
19+
20+
# Runs app on the same network as the database container, allows "forwardPorts" in devcontainer.json function.
21+
network_mode: service:db
22+
23+
# Uncomment the next line to use a non-root user for all processes.
24+
# user: vscode
25+
26+
# Use "forwardPorts" in **devcontainer.json** to forward an app port locally.
27+
# (Adding the "ports" property to this file will not forward from a Codespace.)
28+
29+
db:
30+
image: postgres:latest
31+
restart: unless-stopped
32+
volumes:
33+
- postgres-data:/var/lib/postgresql/data
34+
env_file:
35+
- .env
36+
37+
# Add "forwardPorts": ["5432"] to **devcontainer.json** to forward PostgreSQL locally.
38+
# (Adding the "ports" property to this file will not forward from a Codespace.)
39+
40+
volumes:
41+
postgres-data:

‎containers/python-3-miniconda-postgres/.devcontainer/noop.txt

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
This file is copied into the container along with environment.yml* from the
2+
parent folder. This is done to prevent the Dockerfile COPY instruction from
3+
failing if no environment.yml is found.

‎containers/python-3-miniconda-postgres/.npmignore

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
README.md
2+
test-project
3+
history
4+
definition-manifest.json
5+
.devcontainer/library-scripts
6+
environment.yml
7+
.vscode
8+
.npmignore

‎containers/python-3-miniconda-postgres/.vscode/launch.json

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
{
2+
// Use IntelliSense to learn about possible attributes.
3+
// Hover to view descriptions of existing attributes.
4+
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
5+
"version": "0.2.0",
6+
"configurations": [
7+
{
8+
"name": "Python plot.py (Integrated Terminal)",
9+
"type": "python",
10+
"request": "launch",
11+
"program": "${workspaceFolder}/test-project/plot.py",
12+
"cwd":"${workspaceFolder}/test-project",
13+
"console": "integratedTerminal"
14+
},
15+
{
16+
"name": "Python database.py (Integrated Terminal)",
17+
"type": "python",
18+
"request": "launch",
19+
"program": "${workspaceFolder}/test-project/database.py",
20+
"cwd":"${workspaceFolder}/test-project",
21+
"console": "integratedTerminal"
22+
}
23+
]
24+
}

‎containers/python-3-miniconda-postgres/README.md

Lines changed: 158 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,158 @@
1+
# Miniconda (Python 3)
2+
3+
## Summary
4+
5+
*Develop Miniconda applications in Python 3. Installs dependencies from your environment.yml file and the Python extension.*
6+
7+
| Metadata | Value |
8+
|----------|-------|
9+
| *Contributors* | The [VS Code Python extension](https://marketplace.visualstudio.com/itemdetails?itemName=ms-python.python) team |
10+
| *Categories* | Core, Languages |
11+
| *Definition type* | Docker Compose |
12+
| *Supported architecture(s)* | x86-64 |
13+
| *Works in Codespaces* | Yes |
14+
| *Container host OS support* | Linux, macOS, Windows |
15+
| *Container OS* | Debian |
16+
| *Languages, platforms* | Python, Anaconda, Miniconda |
17+
18+
## Using this definition
19+
20+
This definition creates two containers, one for Minconda/Python and one for PostgreSQL. VS Code will attach to the Miniconda container, and from within that container the PostgreSQL container will be available on **`localhost`** port 5432. The default database is named `postgres` with a user of `postgres` whose password is `postgres`, and if desired this may be changed in `docker-compose.yml`. Data is stored in a volume named `postgres-data`.
21+
22+
While the definition itself works unmodified, you can also directly reference pre-built versions by updating the `FROM` statement in your own `Dockerfile` to the following. An example `Dockerfile` is included in this repository.
23+
24+
- `mcr.microsoft.com/vscode/devcontainers/minconda` (or `minconda:3`)
25+
26+
You can decide how often you want updates by referencing a [semantic version](https://semver.org/) of each image. For example:
27+
28+
- `mcr.microsoft.com/vscode/devcontainers/miniconda:0-3`
29+
- `mcr.microsoft.com/vscode/devcontainers/miniconda:0.201-3`
30+
- `mcr.microsoft.com/vscode/devcontainers/miniconda:0.201.4-3`
31+
32+
See the [python-3-miniconda history](https://github.com/microsoft/vscode-dev-containers/tree/main/containers/python-3-miniconda/history) for information on the contents of each version and [here for a complete list of available tags](https://mcr.microsoft.com/v2/vscode/devcontainers/miniconda/tags/list).
33+
34+
You also can connect to PostgreSQL from an external tool when using VS Code by updating `.devcontainer/devcontainer.json` as follows:
35+
36+
```json
37+
"forwardPorts": [ "5432" ]
38+
```
39+
40+
### Adding another service
41+
42+
You can add other services to your `docker-compose.yml` file [as described in Docker's documentation](https://docs.docker.com/compose/compose-file/#service-configuration-reference). However, if you want anything running in this service to be available in the container on localhost, or want to forward the service locally, be sure to add this line to the service config:
43+
44+
```yaml
45+
# Runs the service on the same network as the database container, allows "forwardPorts" in devcontainer.json function.
46+
network_mode: service:[$SERVICE_NAME]
47+
```
48+
49+
### Using Conda
50+
This dev container and its associated miniconda image includes [the `conda` package manager](https://aka.ms/vscode-remote/conda/about). Additional packages installed using Conda will be downloaded from Anaconda or another repository if you configure one. To reconfigure Conda in this container to access an alternative repository, please see information on [configuring Conda channels here](https://aka.ms/vscode-remote/conda/channel-setup).
51+
52+
Access to the Anaconda repository is covered by the [Anaconda Terms of Service](https://aka.ms/vscode-remote/conda/terms), which may require some organizations to obtain a commercial license from Anaconda. **However**, when this dev container or its associated image is used with GitHub Codespaces or GitHub Actions, **all users are permitted** to use the Anaconda Repository through the service, including organizations normally required by Anaconda to obtain a paid license for commercial activities. Note that third-party packages may be licensed by their publishers in ways that impact your intellectual property, and are used at your own risk.
53+
54+
#### Debug Configuration
55+
56+
Note that only the integrated terminal is supported by the Remote - Containers extension. You may need to modify `launch.json` configurations to include the following value if an external console is used.
57+
58+
```json
59+
"console": "integratedTerminal"
60+
```
61+
62+
#### Using the forwardPorts property
63+
64+
By default, frameworks like Flask only listens to localhost inside the container. As a result, we recommend using the `forwardPorts` property (available in v0.98.0+) to make these ports available locally.
65+
66+
```json
67+
"forwardPorts": [5000]
68+
```
69+
70+
The `appPort` property [publishes](https://docs.docker.com/config/containers/container-networking/#published-ports) rather than forwards the port, so applications need to listen to `*` or `0.0.0.0` for the application to be accessible externally. This conflicts with the defaults of some Python frameworks, but fortunately the `forwardPorts` property does not have this limitation.
71+
72+
If you've already opened your folder in a container, rebuild the container using the **Remote-Containers: Rebuild Container** command from the Command Palette (<kbd>F1</kbd>) so the settings take effect.
73+
74+
#### Installing Node.js
75+
76+
Given JavaScript front-end web client code written for use in conjunction with a Python back-end often requires the use of Node.js-based utilities to build, this container also includes `nvm` so that you can easily install Node.js. You can change the version of Node.js installed or disable its installation by updating the `args` property in `.devcontainer/docker-compose.yml`.
77+
78+
```yaml
79+
args:
80+
NODE_VERSION: "14" # Set to "none" to skip Node.js installation
81+
```
82+
83+
#### Installing or updating Python utilities
84+
85+
This container installs all Python development utilities using [pipx](https://pipxproject.github.io/pipx/) to avoid impacting the global Python environment. You can use this same utility add additional utilities in an isolated environment. For example:
86+
87+
```bash
88+
pipx install prospector
89+
```
90+
91+
Note that if you change the version of Python from the default, you'll need to run a few commands to update the utilities and `pipx`. More on that next.
92+
93+
#### Installing a different version of Python
94+
95+
As covered in the [user FAQ](https://docs.anaconda.com/anaconda/user-guide/faq) for Anaconda, you can install different versions of Python than the one in this image by running the following from a terminal:
96+
97+
```bash
98+
conda install python=3.6
99+
pip install --no-cache-dir pipx
100+
pipx uninstall pipx
101+
pipx reinstall-all
102+
```
103+
104+
Or in a Dockerfile:
105+
106+
```Dockerfile
107+
RUN conda install -y python=3.6 \
108+
&& pip install --no-cache-dir pipx \
109+
&& pipx uninstall pipx \
110+
&& pipx reinstall-all
111+
```
112+
113+
See the [pipx documentation](https://pipxproject.github.io/pipx/docs/) for additional information.
114+
115+
### [Optional] Adding the contents of environment.yml to the image
116+
117+
For convenience, this definition will automatically install dependencies from the `environment.yml` file in the parent folder when the container is built. You can change this behavior by altering this line in the `.devcontainer/Dockerfile`:
118+
119+
```Dockerfile
120+
RUN if [ -f "/tmp/conda-tmp/environment.yml" ]; then /opt/conda/bin/conda env update -n base -f /tmp/conda-tmp/environment.yml; fi \
121+
&& rm -rf /tmp/conda-tmp
122+
```
123+
124+
### Adding the definition to your folder
125+
126+
1. If this is your first time using a development container, please see getting started information on [setting up](https://aka.ms/vscode-remote/containers/getting-started) Remote-Containers or [creating a codespace](https://aka.ms/ghcs-open-codespace) using GitHub Codespaces.
127+
128+
2. Start VS Code and open your project folder or connect to a codespace.
129+
130+
3. Press <kbd>F1</kbd> select and **Add Development Container Configuration Files...** command for **Remote-Containers** or **Codespaces**.
131+
132+
> **Note:** If needed, you can drag-and-drop the `.devcontainer` folder from this sub-folder in a locally cloned copy of this repository into the VS Code file explorer instead of using the command.
133+
134+
4. Select this definition. You may also need to select **Show All Definitions...** for it to appear.
135+
136+
5. Finally, press <kbd>F1</kbd> and run **Remote-Containers: Reopen Folder in Container** or **Codespaces: Rebuild Container** to start using the definition.
137+
138+
## Testing the definition
139+
140+
This definition includes some test code that will help you verify it is working as expected on your system. Follow these steps:
141+
142+
1. If this is your first time using a development container, please follow the [getting started steps](https://aka.ms/vscode-remote/containers/getting-started) to set up your machine.
143+
2. Clone this repository.
144+
3. Start VS Code, press <kbd>F1</kbd>, and select **Remote-Containers: Open Folder in Container...**
145+
4. Select the `containers/python-3-miniconda-postgres` folder.
146+
5. After the folder has opened in the container, press <kbd>F5</kbd> to run the `plot.py` script in the project.
147+
6. You should see `Open test-project/plot.png to see the result!` in the terminal and a `test-project/plot.png` file should be added to the folder after it runs with the plot result.
148+
7. Next, open `test-project/plot.py` and press <kbd>ctrl/cmd</kbd>+<kbd>a</kbd> then <kbd>shift</kbd>+<kbd>enter</kbd>.
149+
8. You should see the `matplotlib` output in the interactive window.
150+
9. To run the `database.py` PostgreSQL connection script, navigate to the "Run and Debug" pane ( <kbd>ctrl/cmd</kbd>+<kbd>shift</kbd>+<kbd>D</kbd>) and select `Python database.py (Integrated Terminal)` from the dropdown and press <kbd>F5</kbd>.
151+
10. You should see `DATABASE CONNECTED` and `One database in this database server is: postgres` in the terminal.
152+
11. From here, you can add breakpoints or edit the contents of the `test-project` folder to do further testing.
153+
154+
## License
155+
156+
Copyright (c) Microsoft Corporation. All rights reserved.
157+
158+
Licensed under the MIT License. See [LICENSE](https://github.com/microsoft/vscode-dev-containers/blob/main/LICENSE)

‎containers/python-3-miniconda-postgres/environment.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
dependencies:
2+
- jupyter
3+
- numpy
4+
- matplotlib
5+
- psycopg2
6+
- pylint

‎containers/python-3-miniconda-postgres/test-project/.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
plot.png

0 commit comments

Comments
 (0)