Skip to content

Commit 0004966

Browse files
tomasstupkatomasstupka
committed
[GR-57407] In-depth docs - maven and gradle plugins.
PullRequest: graalpython/3457
2 parents 93644d0 + e5d9d73 commit 0004966

File tree

2 files changed

+213
-33
lines changed

2 files changed

+213
-33
lines changed

‎docs/user/Embedding-Build-Tools.md

Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
---
2+
layout: docs
3+
toc_group: python
4+
link_title: Embedding Build Tools
5+
permalink: /reference-manual/python/Embedding-Build-Tools/
6+
---
7+
8+
# Embedding Build Tools
9+
10+
The GraalPy **Maven** and **Gradle** plugins provide functionality to manage Python related resources
11+
required for embedding Python code in Java-based applications:
12+
- *Python application files* provided by the user, for example, Python sources which are part of the project.
13+
- *Third-party Python packages* installed by the plugin during the build according to the plugin configuration.
14+
- *The Python standard library*, which is necessary to make Native Image generated executables self-contained.
15+
16+
Apart from physically managing and deploying those files, it is also necessary to make them available in Python at runtime by configuring the **GraalPy Context** in your Java code accordingly.
17+
The [GraalPyResources](https://github.com/oracle/graalpython/blob/master/graalpython/org.graalvm.python.embedding/src/org/graalvm/python/embedding/utils/GraalPyResources.java) API provides factory methods to create a Context preconfigured for accessing Python, embedding relevant resources with a **Virtual Filesystem** or from a dedicated **external directory**.
18+
19+
## Deployment
20+
21+
There are two modes how to deploy the resources: as Java resources using the Virtual Filesystem to access them in Python, or as an external directory.
22+
23+
### Virtual Filesystem
24+
25+
The Python related resources are embedded in the application file, either in JAR or Native Image generated
26+
executable, as standard Java resources.
27+
The GraalPy Virtual Filesystem accesses resource files as standard Java resources and makes them available to Python code running in GraalPy.
28+
This is transparent to the Python code, which can use standard Python IO to access those files.
29+
30+
Java resource files in a Maven or Gradle project are typically located in dedicated resources directories.
31+
All resources subdirectories named _org.graalvm.python.vfs_ are merged and mapped to a configurable Virtual Filesystem mount point at the Python side, by default `/graalpy_vfs`.
32+
For example, a Python file with the real filesystem path `${project_resources_directory}/org.graalvm.python.vfs/src/foo/bar.py` will be accessible as `/graalpy_vfs/src/foo/bar.py` in Python.
33+
34+
Use the following [GraalPyResources](https://github.com/oracle/graalpython/blob/master/graalpython/org.graalvm.python.embedding/src/org/graalvm/python/embedding/utils/GraalPyResources.java)
35+
factory methods to create GraalPy Context preconfigured for the use of the Virtual Filesystem:
36+
* `GraalPyResources.createContext()`
37+
* `GraalPyResources.contextBuilder()`
38+
* `GraalPyResources.contextBuilder(VirtualFileSystem)`
39+
40+
### External Directory
41+
42+
As an alternative to Java resources with the Virtual Filesystem, it is also possible to configure the Maven or Gradle plugin to manage the contents of an external directory, which will **not be embedded** as a Java resource into the resulting application.
43+
A user is then responsible for the deployment of such directory.
44+
Python code will access the files directly from the real filesystem.
45+
46+
Use the following [GraalPyResources](https://github.com/oracle/graalpython/blob/master/graalpython/org.graalvm.python.embedding/src/org/graalvm/python/embedding/utils/GraalPyResources.java) factory methods to create GraalPy Context preconfigured for the use of an external directory:
47+
* `GraalPyResources.createContextBuilder(Path)`
48+
49+
## Conventions
50+
51+
The factory methods in [GraalPyResources](https://github.com/oracle/graalpython/blob/master/graalpython/org.graalvm.python.embedding/src/org/graalvm/python/embedding/utils/GraalPyResources.java) rely on the following conventions, where the `${root}` is either an external directory, or a Virtual System mount point on the Python side and `${project_resources_directory}/org.graalvm.python.vfs` on the real filesystem:
52+
- `${root}/src`: used for Python application files. This directory will be configured as the default search path for Python module files (equivalent to `PYTHONPATH` environment variable).
53+
- `${root}/venv`: used for the Python virtual environment holding installed third-party Python packages.
54+
The Context will be configured as if it is executed from this virtual environment. Notably packages installed in this
55+
virtual environment will be automatically available for importing.
56+
- `${root}/home`: used for the Python standard library (equivalent to `PYTHONHOME` environment variable).
57+
58+
The Maven or Gradle plugin will fully manage the contents of the `venv` and `home` subdirectories.
59+
Any manual changes in these directories will be overridden by the plugin during the build.
60+
- `${root}/venv`: the plugin creates a virtual environment and installs required packages according to the plugin configuration in _pom.xml_ or _build.gradle_.
61+
- `${root}/home`: the plugin copies the required (also configurable) parts of the Python standard library into this directory.
62+
By default, the full standard library is used.
63+
64+
The _src_ subdirectory is left to be manually populated by the user with custom Python scripts or modules.
65+
66+
## GraalPy Maven Plugin Configuration
67+
68+
Add the plugin configuration in the `configuration` block of `graalpy-maven-plugin` in the _pom.xml_ file:
69+
```xml
70+
<plugin>
71+
<groupId>org.graalvm.python</groupId>
72+
<artifactId>graalpy-maven-plugin</artifactId>
73+
...
74+
<configuration>
75+
...
76+
</configuration>
77+
...
78+
</plugin>
79+
```
80+
The **packages** element declares a list of third-party Python packages to be downloaded and installed by the plugin.
81+
- The Python packages and their versions are specified as if used with `pip`:
82+
```xml
83+
<configuration>
84+
<packages>
85+
<package>termcolor==2.2</package>
86+
...
87+
</packages>
88+
...
89+
</configuration>
90+
```
91+
- The **pythonHome** subsection declares what parts of the standard library should be deployed.
92+
93+
Each `include` and `exclude` element is interpreted as a Java-like regular expression specifying which file paths should be included or excluded.
94+
```xml
95+
<configuration>
96+
<pythonHome>
97+
<includes>
98+
<include>.*</include>
99+
...
100+
</includes>
101+
<excludes>
102+
<exclude></exclude>
103+
...
104+
</excludes>
105+
</pythonHome>
106+
...
107+
</configuration>
108+
```
109+
- If the **pythonResourcesDirectory** element is specified, then the given directory is used as an [external directory](#external-directory) and no Java resources are embedded.
110+
Remember to use the appropriate `GraalPyResources` API to create the Context.
111+
```xml
112+
<configuration>
113+
<pythonResourcesDirectory>${basedir}/python-resources</pythonResourcesDirectory>
114+
...
115+
</configuration>
116+
```
117+
118+
## GraalPy Gradle Plugin Configuration
119+
120+
Add the plugin configuration in the `GraalPy` block in the _build.gradle_ file.
121+
The **packages** element declares a list of third-party Python packages to be downloaded and installed by the plugin.
122+
- The Python packages and their versions are specified as if used with `pip`.
123+
```
124+
graalPy {
125+
packages = ["termcolor==2.2"]
126+
...
127+
}
128+
```
129+
- The **pythonHome** subsection declares what parts of the standard library should be deployed.
130+
131+
Each element in the `includes` and `excludes` list is interpreted as a Java-like regular expression specifying which file paths should be included or excluded.
132+
```
133+
graalPy {
134+
pythonHome {
135+
includes = [".*"]
136+
excludes = []
137+
}
138+
...
139+
}
140+
```
141+
- If the **pythonResourcesDirectory** element is specified, then the given directory is used as an [external directory](#external-directory) and no Java resources are embedded.
142+
Remember to use the appropriate `GraalPyResources` API to create the Context.
143+
```
144+
graalPy {
145+
pythonResourcesDirectory = file("$rootDir/python-resources")
146+
...
147+
}
148+
```
149+
150+
### Related Documentation
151+
152+
* [Embedding Graal languages in Java](https://www.graalvm.org/reference-manual/embed-languages/)
153+
* [Permissions for Python Embeddings](Embedding-Permissions.md)

‎docs/user/README.md

Lines changed: 60 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -18,16 +18,15 @@ Other build systems (Ant, Make, CMake, ...) can also be used with a bit more man
1818

1919
GraalPy can generate a Maven project that embeds Python packages into a Java application using [Maven artefacts](https://mvnrepository.com/artifact/org.graalvm.python).
2020

21-
2221
1. Since version 24.0, the GraalPy project publishes a Maven archetype to generate a starter project:
2322
```bash
2423
mvn archetype:generate \
2524
-DarchetypeGroupId=org.graalvm.python \
2625
-DarchetypeArtifactId=graalpy-archetype-polyglot-app \
27-
-DarchetypeVersion=24.0.0
26+
-DarchetypeVersion=24.2.0
2827
```
2928

30-
2. Build a native executable using the [GraalVM Native Image](https://www.graalvm.org/latest/reference-manual/native-image/) plugin that was added for you automatically:
29+
2. Build a native executable using the [ GraalVM Native Image "tool"](https://www.graalvm.org/latest/reference-manual/native-image/) plugin that was added for you automatically:
3130
```bash
3231
mvn -Pnative package
3332
```
@@ -38,9 +37,10 @@ GraalPy can generate a Maven project that embeds Python packages into a Java app
3837
```
3938
The application prints "hello java" to the console.
4039

41-
The project uses the [GraalVM SDK Polyglot API](https://www.graalvm.org/sdk/javadoc/org/graalvm/polyglot/package-summary.html) with additional features to manage Python virtual environments and integrate Python package dependencies with a Maven workflow.
40+
The project uses the [GraalVM Polyglot API](https://www.graalvm.org/sdk/javadoc/org/graalvm/polyglot/package-summary.html) with additional features to manage Python virtual environments and integrate Python package dependencies with a Maven workflow.
4241
The Java code and the _pom.xml_ file are heavily documented and the generated code describes available features.
43-
(If you do not wish to use Maven, the archetype Java code also provides guidance to create a custom embedding.)
42+
43+
See also [Embedding Build Tools](Embedding-Build-Tools.md#graalpy-maven-plugin) for more information about the GraalPy Maven Plugin.
4444

4545
### Creating Cross-platform JARs with Native Python Packages
4646

@@ -67,7 +67,7 @@ In order to distribute the resulting application for other systems, follow these
6767

6868
## Gradle
6969

70-
1. Create a Java application with Gradle using the command below and follow the prompts (select a build script language, select a test framework, and so on):
70+
1. Create a Java application with Gradle using the command below and follow the prompts (select the Groovy build script language, select a test framework, and so on):
7171
```bash
7272
gradle init --type java-application \
7373
--project-name interop \
@@ -88,34 +88,14 @@ In order to distribute the resulting application for other systems, follow these
8888
```
8989

9090
2. Open your project configuration file, _app/build.gradle_, and modify it as follows.
91-
- Include the GraalPy support and the [GraalVM SDK Polyglot API](https://www.graalvm.org/sdk/javadoc/org/graalvm/polyglot/package-summary.html) in the `dependencies` section:
92-
93-
```
94-
implementation("org.graalvm.polyglot:polyglot:24.0.0")
95-
implementation("org.graalvm.polyglot:python:24.0.0")
96-
```
97-
98-
- We recommend you use the Java modules build. Add the appropriate plugin to the `plugins` section:
99-
```
100-
id("org.javamodularity.moduleplugin") version "1.8.12"
101-
```
91+
- Include the GraalPy support and the [GraalVM Polyglot API](https://www.graalvm.org/sdk/javadoc/org/graalvm/polyglot/package-summary.html) in the `dependencies` section:
10292

103-
- To run the application as a module rather than from the classpath, edit the `application` section to look like this:
10493
```
105-
application {
106-
mainClass.set("interop.App")
107-
mainModule.set("interop")
108-
}
94+
implementation("org.graalvm.polyglot:polyglot:24.2.0")
95+
implementation("org.graalvm.polyglot:python:24.2.0")
10996
```
11097

111-
3. Create a new file named _app/src/main/java/module-info.java_ with the following contents:
112-
```java
113-
module interop {
114-
requires org.graalvm.polyglot;
115-
}
116-
```
117-
118-
4. Finally, replace the code in the file named _App.java_ as follows for a small Python embedding:
98+
3. Finally, replace the code in the file named _App.java_ as follows for a small Python embedding:
11999
```java
120100
package interop;
121101
@@ -130,14 +110,61 @@ In order to distribute the resulting application for other systems, follow these
130110
}
131111
```
132112
133-
5. Run the application with Gradle:
113+
4. Run the application with Gradle:
134114
```bash
135115
./gradlew run
136116
```
137117
The application prints "Hello Python!" to the console.
138118
139119
> Note: The performance of the GraalPy runtime depends on the JDK in which you embed it. For more information, see [Runtime Optimization Support](https://www.graalvm.org/latest/reference-manual/embed-languages/#runtime-optimization-support).
140120
121+
5. Optionally, you can also use a third-party Python package:
122+
123+
5.1. In _app/build.gradle_:
124+
- add the graalpy-gradle-plugin to the `plugins` section:
125+
```
126+
id "org.graalvm.python" version "24.2.0"
127+
```
128+
129+
- configure the GraalPy Gradle plugin:
130+
```
131+
graalPy {
132+
packages = ["termcolor==2.2"]
133+
}
134+
```
135+
136+
5.2, In _settings.gradle_, add the following `pluginManagement` configuration.
137+
```
138+
pluginManagement {
139+
repositories {
140+
gradlePluginPortal()
141+
}
142+
}
143+
```
144+
145+
5.3. Update the file named _App.java_ as follows:
146+
```java
147+
package interop;
148+
149+
import org.graalvm.polyglot.*;
150+
import org.graalvm.python.embedding.utils.GraalPyResources;
151+
152+
class App {
153+
...
154+
public static void main(String[] args) {
155+
try (Context context = GraalPyResources.createContext()) {
156+
String src = """
157+
from termcolor import colored
158+
colored_text = colored("hello java", "red", attrs=["reverse", "blink"])
159+
print(colored_text)
160+
""";
161+
context.eval("python", src);
162+
}
163+
}
164+
```
165+
166+
See also [Embedding Build Tools](Embedding-Build-Tools.md) for more information about the GraalPy Gradle Plugin.
167+
141168
## Ant, CMake, Makefile or Other Build Systems Without Direct Support for Maven Dependencies
142169
143170
Some (often older) projects may be using Ant, Makefiles, CMake, or other build systems that do not directly support Maven dependencies.
@@ -160,13 +187,13 @@ GraalPy comes with a tool to obtain the required JAR files from Maven.
160187
In a POSIX shell:
161188
```bash
162189
export GRAALPY_HOME=$(graalpy -c 'print(__graalpython__.home)')
163-
"${GRAALPY_HOME}/libexec/graalpy-polyglot-get" -a python -o lib -v "24.0.0"
190+
"${GRAALPY_HOME}/libexec/graalpy-polyglot-get" -a python -o lib -v "24.2.0"
164191
```
165192
166193
In PowerShell:
167194
```
168195
$GRAALPY_HOME = graalpy -c "print(__graalpython__.home)"
169-
& "$GRAALPY_HOME/libexec/graalpy-polyglot-get" -a python -o lib -v "24.0.0"
196+
& "$GRAALPY_HOME/libexec/graalpy-polyglot-get" -a python -o lib -v "24.2.0"
170197
```
171198
172199
These commands download all GraalPy dependencies into the _lib_ directory.

0 commit comments

Comments
 (0)