Skip to content

Commit f73dcbd

Browse files
fracpetefracpete
committed
added documentation on how to install additional packages
1 parent 69b0934 commit f73dcbd

File tree

6 files changed

+184
-0
lines changed

6 files changed

+184
-0
lines changed

‎doc/source/docker.rst‎

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,3 +50,91 @@ you can map your local Weka packages into the container as follows:
5050
docker run --gpus=all -u $(id -u):$(id -g) \
5151
-v $HOME/wekafiles/:/workspace/wekafiles \
5252
-it fracpete/pww3:0.2.9_cuda10.2
53+
54+
55+
Additional Weka packages
56+
------------------------
57+
58+
When building Docker images for your environments, your code will most likely rely
59+
on additional Weka packages. You can install the packages by creating a little
60+
Python script that uses python-weka-wrapper3 to install them (just like you would
61+
normally do in a script). Here is the content of the ``install_packages.py``
62+
script:
63+
64+
.. code-block:: python
65+
66+
import weka.core.jvm as jvm
67+
import weka.core.packages as packages
68+
69+
jvm.start(packages=True)
70+
71+
# for reproducibility, we also specify the version
72+
packages.install_package("SelfOrganizingMap", version="1.0.3")
73+
74+
jvm.stop()
75+
76+
77+
A minimal ``Dockerfile`` (in the same directory as ``install_packages.py``) then looks
78+
like this (using pww3 0.2.9 for CPU):
79+
80+
::
81+
82+
FROM fracpete/pww3:0.2.9_cpu
83+
COPY install_packages.py /workspace/install_packages.py
84+
RUN python3 /workspace/install_packages.py
85+
86+
87+
You can then build this image just like any other Docker image:
88+
89+
.. code-block:: bash
90+
91+
docker build -t pww3-pkg .
92+
93+
94+
For testing, you can create a local script called ``test_packages.py`` with
95+
the content similar to this:
96+
97+
.. code-block:: python
98+
99+
import weka.core.jvm as jvm
100+
import weka.core.packages as packages
101+
from weka.clusterers import Clusterer
102+
103+
jvm.start(packages=True)
104+
105+
# list packages
106+
items = packages.installed_packages()
107+
for item in items:
108+
print(item.name + "/" + item.version + "\n " + item.url)
109+
110+
# instantiate from package
111+
cls = Clusterer(classname="weka.clusterers.SelfOrganizingMap")
112+
print(cls.to_commandline())
113+
114+
jvm.stop()
115+
116+
117+
The following command simply runs our ``test_packages.py`` script. To achieve this,
118+
the command maps the current directory (``pwd``) into the container's ``/workspace/scripts``
119+
directory:
120+
121+
.. code-block:: bash
122+
123+
docker run \
124+
-v `pwd`:/workspace/scripts \
125+
-t pww3-pkg:latest \
126+
python3 /workspace/scripts/test_packages.py
127+
128+
129+
The output will be something like this:
130+
131+
::
132+
133+
DEBUG:weka.core.jvm:Adding bundled jars
134+
DEBUG:weka.core.jvm:Classpath=['/usr/local/lib/python3.8/dist-packages/javabridge/jars/rhino-1.7R4.jar', '/usr/local/lib/python3.8/dist-packages/javabridge/jars/runnablequeue.jar', '/usr/local/lib/python3.8/dist-packages/javabridge/jars/cpython.jar', '/usr/local/lib/python3.8/dist-packages/weka/lib/weka.jar', '/usr/local/lib/python3.8/dist-packages/weka/lib/python-weka-wrapper.jar']
135+
DEBUG:weka.core.jvm:MaxHeapSize=default
136+
DEBUG:weka.core.jvm:Package support enabled
137+
SelfOrganizingMap/1.0.3
138+
http://prdownloads.sourceforge.net/wekann/SelfOrganizingMap1.0.3.zip?download
139+
weka.clusterers.SelfOrganizingMap -L 1.0 -O 2000 -C 1000 -H 2 -W 2
140+

‎docker/README.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,3 +16,9 @@ In order to make installation reproducible, the following Docker images
1616
The images themselves are available through Docker hub:
1717

1818
https://hub.docker.com/u/fracpete
19+
20+
21+
## Installing Weka packages
22+
23+
Instructions and example code/Dockerfile for installing additional Weka packages in a
24+
Docker image are located [here](packages).

‎docker/packages/Dockerfile‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
FROM fracpete/pww3:0.2.9_cpu
2+
3+
COPY install_packages.py /workspace/install_packages.py
4+
5+
RUN python3 /workspace/install_packages.py

‎docker/packages/README.md‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# Installing packages in Docker image
2+
3+
When building Docker images for your environments, your code will most likely rely
4+
on additional Weka packages. You can install the packages by creating a little
5+
Python script that uses python-weka-wrapper3 to install them (just like you would
6+
normally do in a script). Here is the content of the [install_packages.py](install_packages.py)
7+
script:
8+
9+
```python
10+
import weka.core.jvm as jvm
11+
import weka.core.packages as packages
12+
13+
jvm.start(packages=True)
14+
15+
# for reproducibility, we also specify the version of the package that we need
16+
packages.install_package("SelfOrganizingMap", version="1.0.3")
17+
18+
jvm.stop()
19+
```
20+
21+
A minimal `Dockerfile` then looks like this (using the CPU Docker base image of pww3 0.2.9):
22+
23+
```
24+
FROM fracpete/pww3:0.2.9_cpu
25+
COPY install_packages.py /workspace/install_packages.py
26+
RUN python3 /workspace/install_packages.py
27+
```
28+
29+
You can then build this image just like any other Docker image:
30+
31+
```bash
32+
docker build -t pww3-pkg .
33+
```
34+
35+
The following command simply runs the local [test_packages.py](test_packages.py) script
36+
to check whether the installation was successful. To achieve this, it maps the
37+
current directory (``pwd``) into the container's `/workspace/scripts` directory:
38+
39+
```bash
40+
docker run \
41+
-v `pwd`:/workspace/scripts \
42+
-t pww3-pkg:latest \
43+
python3 /workspace/scripts/test_packages.py
44+
```
45+
46+
The output will be something like this:
47+
48+
```
49+
DEBUG:weka.core.jvm:Adding bundled jars
50+
DEBUG:weka.core.jvm:Classpath=['/usr/local/lib/python3.8/dist-packages/javabridge/jars/rhino-1.7R4.jar', '/usr/local/lib/python3.8/dist-packages/javabridge/jars/runnablequeue.jar', '/usr/local/lib/python3.8/dist-packages/javabridge/jars/cpython.jar', '/usr/local/lib/python3.8/dist-packages/weka/lib/weka.jar', '/usr/local/lib/python3.8/dist-packages/weka/lib/python-weka-wrapper.jar']
51+
DEBUG:weka.core.jvm:MaxHeapSize=default
52+
DEBUG:weka.core.jvm:Package support enabled
53+
SelfOrganizingMap/1.0.3
54+
http://prdownloads.sourceforge.net/wekann/SelfOrganizingMap1.0.3.zip?download
55+
weka.clusterers.SelfOrganizingMap -L 1.0 -O 2000 -C 1000 -H 2 -W 2
56+
```

‎docker/packages/install_packages.py‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# helper script that is used by the Docker image to install additional packages
2+
3+
import weka.core.jvm as jvm
4+
import weka.core.packages as packages
5+
6+
jvm.start(packages=True)
7+
8+
# for reproducibility, we also specify the version of the package that we need
9+
packages.install_package("SelfOrganizingMap", version="1.0.3")
10+
11+
jvm.stop()

‎docker/packages/test_packages.py‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# script for testing the packages installed in the docker image by the "install_packages.py" script
2+
3+
import weka.core.jvm as jvm
4+
import weka.core.packages as packages
5+
from weka.clusterers import Clusterer
6+
7+
jvm.start(packages=True)
8+
9+
# list packages
10+
items = packages.installed_packages()
11+
for item in items:
12+
print(item.name + "/" + item.version + "\n " + item.url)
13+
14+
# instantiate from package
15+
cls = Clusterer(classname="weka.clusterers.SelfOrganizingMap")
16+
print(cls.to_commandline())
17+
18+
jvm.stop()

0 commit comments

Comments
 (0)