|
1 | 1 | # GraalPy, the GraalVM Implementation of Python
|
2 | 2 |
|
3 | 3 | GraalPy is a high-performance implementation of the [Python](https://www.python.org/) language for the JVM built on [GraalVM](https://www.graalvm.org/) technology.
|
| 4 | +GraalPy has first-class support for embedding in Java and can turn Python applications into fast, standalone binaries. |
| 5 | + |
| 6 | +<sup> |
| 7 | +[Java embedding](https://github.com/timfel/graalpy-jbang) using [JBang](https://www.jbang.dev/) — [Standalone binary](https://github.com/timfel/racing-all-afternoon) of Python a game by [Joey Navarro](https://github.com/josephnavarro/racing-all-afternoon) with all dependencies included. |
| 8 | +</sup> |
| 9 | + |
| 10 | +## Benefits of GraalPy |
| 11 | + |
| 12 | +* **Low-overhead integration with Java and other languages.** |
| 13 | + |
| 14 | + GraalPy [integrates](docs/user/Interoperability.md) with Java, JavaScript, Ruby, and other languages on GraalVM JDK, Oracle JDK, or OpenJDK. |
| 15 | + You can easily add GraalPy to your Java application using [Maven build tools](docs/user/PythonStandaloneBinaries.md#embedding-graalpy-in-a-java-application). |
| 16 | + JVM tools such as VisualVM or JFR work as you would expect. |
| 17 | + Native Image compilation using GraalVM Native Image is [well supported](docs/user/PythonNativeimages.md). |
| 18 | + Python's low-level [OS APIs](docs/user/OsInterface.md#java-backend) are emulated, so Java code can intercept standard streams, file system and socket access, or subprocess execution. |
| 19 | + |
| 20 | +* **Compatible with the Python ecosystem** |
| 21 | + |
| 22 | + GraalPy supports many libraries including PyTorch, Tensorflow, SciPy. |
| 23 | + For example, models from [Hugging Face](https://huggingface.co/) like Stable Diffusion or GPT that use [PyTorch](https://pytorch.org/) usually just work. |
| 24 | + We run the CPython test suite on every commit and are passing ~85% of it. |
| 25 | + We also run the tests of the [top PyPI packages](https://hugovk.github.io/top-pypi-packages/) on GraalPy every day. |
| 26 | + To see if packages you use work, we have created a [Python Compatibility Checker](https://www.graalvm.org/python/compatibility/). |
| 27 | + <sup> |
| 28 | + For more than 96% of the top PyPI packages, there is at least one recent version that installs successfully and we are currently passing over 50% of all tests those top packages. |
| 29 | + </sup> |
| 30 | + |
| 31 | +* **Run Python code faster** |
| 32 | + |
| 33 | + GraalPy usually executes pure Python code faster than CPython after JIT compilation. |
| 34 | + Performance when C extensions are involved is close to CPython, but can vary a lot depending on the specific interactions of C extension code and Python code. |
| 35 | + We see a geomean speedup of ~4 on the [Python Performance Benchmark Suite](https://pyperformance.readthedocs.io/) over CPython. |
| 36 | + <sup> |
| 37 | + Benchmarks run via `pip install pyperformance && pyperformance run` on each of CPython, Jython, and GraalPy. |
| 38 | + Installation of each done via <tt>[pyenv](https://github.com/pyenv/pyenv)</tt>. |
| 39 | + Geomean speedup was calculated pair-wise against CPython on the intersection of benchmarks that run on both interpreters. |
| 40 | + </sup> |
4 | 41 |
|
5 | 42 | ## Getting Started
|
6 | 43 |
|
7 | 44 | <details>
|
8 | 45 | <summary><strong>Embedding GraalPy in Java</strong></summary>
|
9 | 46 |
|
10 |
| -GraalPy is [available on Maven Central](https://central.sonatype.com/artifact/org.graalvm.polyglot/python) for inclusion in Java projects: |
| 47 | +GraalPy is [available on Maven Central](https://central.sonatype.com/artifact/org.graalvm.polyglot/python) for inclusion in Java projects. |
| 48 | +Refer to our [embedding documentation](https://www.graalvm.org/latest/reference-manual/embed-languages/) for more details. |
| 49 | + |
| 50 | +* Maven |
| 51 | + ```xml |
| 52 | + <dependency> |
| 53 | + <groupId>org.graalvm.polyglot</groupId> |
| 54 | + <artifactId>polyglot</artifactId> |
| 55 | + <version>23.1.2</version> |
| 56 | + </dependency> |
| 57 | + <dependency> |
| 58 | + <groupId>org.graalvm.polyglot</groupId> |
| 59 | + <artifactId>python</artifactId> |
| 60 | + <version>23.1.2</version> |
| 61 | + <type>pom</type> |
| 62 | + </dependency> |
| 63 | + ``` |
11 | 64 |
|
12 |
| -``` |
13 |
| -<dependency> |
14 |
| - <groupId>org.graalvm.polyglot</groupId> |
15 |
| - <artifactId>python</artifactId> |
16 |
| - <version>23.1.2</version> |
17 |
| - <type>pom</type> |
18 |
| -</dependency> |
19 |
| -``` |
| 65 | +* Gradle |
| 66 | + ```kotlin |
| 67 | + implementation("org.graalvm.polyglot:polyglot:23.1.2") |
| 68 | + implementation("org.graalvm.polyglot:python:23.1.2") |
| 69 | + ``` |
20 | 70 |
|
21 | 71 | </details>
|
22 | 72 |
|
23 | 73 | <details>
|
24 | 74 | <summary><strong>Replacing CPython with GraalPy</strong></summary>
|
25 | 75 |
|
| 76 | +GraalPy should in many cases work as a drop-in replacement for CPython. |
| 77 | +You can use `pip` to install packages as usual. |
| 78 | +Packages with C code usually do not provide binaries for GraalPy, so they will be automatically compiled during installation. |
| 79 | +This means that build tools have to be available and installation will take longer. |
| 80 | +We provide [Github actions](scripts/wheelbuilder) to help you build binary packages with the correct dependencies. |
| 81 | +Thanks to our integration with GraalVM Native Image, we can deploy Python applications as [standalone binary](docs/user/PythonStandaloneBinaries.md), all dependencies included. |
| 82 | + |
26 | 83 | * Linux
|
27 | 84 |
|
28 | 85 | The easiest way to install GraalPy on Linux is to use [Pyenv](https://github.com/pyenv/pyenv) (the Python version manager).
|
@@ -87,6 +144,10 @@ The _setup-python_ action supports GraalPy:
|
87 | 144 | <summary><strong>Migrating Jython Scripts to GraalPy</strong></summary>
|
88 | 145 |
|
89 | 146 | To run Jython scripts, you will need a GraalPy distribution running on the JVM so you can access Java classes from Python scripts.
|
| 147 | +Most existing Jython code that uses Java integration will be based on a stable Jython release—however, these are only available in Python 2.x versions. |
| 148 | +First, to migrate your code from Python 2 to Python 3, follow [the official guide from the Python community](https://docs.python.org/3/howto/pyporting.html). |
| 149 | +To facilitate migration from Jython, GraalPy provides a mode that is "mostly compatible" with some of Jython's features, minus of course that Jython implements Python 2.7 and we implement Python 3+. |
| 150 | +We describe the current status of the compatibility mode [here](docs/user/Jython.md). |
90 | 151 |
|
91 | 152 | * Linux
|
92 | 153 |
|
@@ -116,100 +177,6 @@ To run Jython scripts, you will need a GraalPy distribution running on the JVM s
|
116 | 177 |
|
117 | 178 | </details>
|
118 | 179 |
|
119 |
| -## Aims and Benefits of GraalPy |
120 |
| -
|
121 |
| -* **Convenient, low-overhead integration with Java and other languages.** |
122 |
| -
|
123 |
| - Java, JavaScript, Ruby, and other languages that run on the JVM can be used [conveniently with GraalPy](docs/user/Interoperability.md). |
124 |
| - GraalPy also integrates with JVM tools such as VisualVM or JFR, and comes with [profilers, memory samplers, and debuggers](docs/user/Tooling.md) provided as part of GraalVM's Truffle framework. |
125 |
| - |
126 |
| -* **Compatible with the Python ecosystem** |
127 |
| -
|
128 |
| - We run the CPython test suite on every commit and are passing ~75% of it. |
129 |
| - We also run the tests of the [top 600 PyPI packages](https://hugovk.github.io/top-pypi-packages/) on GraalPy every day. |
130 |
| - For more than 96% of the top 600 packages, there is at least one recent version that installs successfully and we are currently passing over 50% of all test cases in those top 600 packages. |
131 |
| -
|
132 |
| -* **Run Python code faster** |
133 |
| -
|
134 |
| - GraalPy can usually execute pure Python code faster than CPython. |
135 |
| - We see a geomean speedup of 4 on the [Python Performance Benchmark Suite](https://pyperformance.readthedocs.io/) over CPython. |
136 |
| - |
137 |
| -
|
138 |
| -* **Support C extensions** |
139 |
| -
|
140 |
| - GraalPy supports most of the public [CPython C API](https://docs.python.org/3/c-api/index.html) as well as the [HPy API](https://hpyproject.org/). |
141 |
| - It supports many native libraries including PyTorch, Tensorflow, SciPy, as well many other data science and machine learning libraries from the rich Python ecosystem. |
142 |
| - Performance when C extensions are involved is close to CPython, but can vary a lot depending on the specific interactions of C extension code and Python code. |
143 |
| - |
144 |
| -## Current Status |
145 |
| -
|
146 |
| -Compatibility and performance is very good for many use cases, especially in the data science and machine learning space. |
147 |
| -For example, models from [Hugging Face](https://huggingface.co/) like Stable Diffusion or GPT that use [PyTorch](https://pytorch.org/) usually just work. |
148 |
| -
|
149 |
| -<figure> |
150 |
| -
|
151 |
| - |
152 |
| -<figcaption> |
153 |
| -
|
154 |
| -To check out if packages you use work, we have created a [Python Compatibility Checker](https://www.graalvm.org/python/compatibility/). |
155 |
| -</figcaption> |
156 |
| -</figure> |
157 |
| -
|
158 |
| -<figure> |
159 |
| -
|
160 |
| - |
161 |
| -<figcaption> |
162 |
| -
|
163 |
| -Benchmarks run via `pip install pyperformance && pyperformance run` on each of CPython, Jython, and GraalPy. |
164 |
| -Installation was done via [Pyenv](https://github.com/pyenv/pyenv). |
165 |
| -Geomean was calculated on pair-wise intersection of benchmarks that run on CPython & Jython or CPython & GraalPy. |
166 |
| -</figcaption> </figure> |
167 |
| -
|
168 |
| -## Python on the JVM |
169 |
| -
|
170 |
| -GraalPy is a first-class citizen of the JVM ecosystem that you can easily embed in your Java applications. |
171 |
| -You can use GraalPy with GraalVM JDK, Oracle JDK, or OpenJDK. |
172 |
| -You can easily add GraalPy to your Java application using [Maven build tools](docs/user/PythonStandaloneBinaries.md#embedding-graalpy-in-a-java-application). |
173 |
| -Python objects support the same [interoperability protocol](docs/user/Interoperability.md) as other GraalVM languages and behave naturally in Java. |
174 |
| -There are also [many options](docs/user/PythonNativeimages.md) to tweak GraalPy for GraalVM Native Images to reduce footprint. |
175 |
| -When embedded into Java, GraalPy also emulates many of the Python [OS interfaces](docs/user/OsInterface.md#java-backend) with standard Java APIs, so embedders have full control over standard streams, file system, or socket access. |
176 |
| -
|
177 |
| -<figure> |
178 |
| -
|
179 |
| - |
180 |
| -<figcaption><small> |
181 |
| -
|
182 |
| -[Java embedding](https://github.com/timfel/graalpy-jbang) using [JBang](https://www.jbang.dev/) |
183 |
| -</small> |
184 |
| -</figcaption> |
185 |
| -</figure> |
186 |
| -
|
187 |
| -## Migration from CPython |
188 |
| -
|
189 |
| -GraalPy should in many cases work as a drop-in replacement for CPython. |
190 |
| -You can use `pip` to install packages as usual. |
191 |
| -However, packages with C code usually do not provide binaries for GraalPy, so they will be automatically compiled during installation. |
192 |
| -This means that build tools have to be available and installation may take longer. |
193 |
| -We provide [Github actions](scripts/wheelbuilder) to help you build binary packages with the correct dependencies. |
194 |
| -Thanks to our integration with GraalVM Native Image, we can deploy Python applications as [standalone binary](docs/user/PythonStandaloneBinaries.md), all dependencies included. |
195 |
| -
|
196 |
| -<figure> |
197 |
| -
|
198 |
| - |
199 |
| -<figcaption><small> |
200 |
| -
|
201 |
| -[Standalone Python app binary](https://github.com/timfel/racing-all-afternoon) based on work by [Joey Navarro](https://github.com/josephnavarro/racing-all-afternoon) |
202 |
| -</small> |
203 |
| -</figcaption> |
204 |
| -</figure> |
205 |
| -
|
206 |
| -## Migration from Jython |
207 |
| -
|
208 |
| -Most existing Jython code that uses Java integration will be based on a stable Jython release—however, these are only available in Python 2.x versions. |
209 |
| -First, to migrate your code from Python 2 to Python 3, follow [the official guide from the Python community](https://docs.python.org/3/howto/pyporting.html). |
210 |
| -To facilitate migration from Jython, GraalPy provides a mode that is "mostly compatible" with some of Jython's features, minus of course that Jython implements Python 2.7 and we implement Python 3+. |
211 |
| -We describe the current status of the compatibility mode [here](docs/user/Jython.md). |
212 |
| -
|
213 | 180 | ## Documentation
|
214 | 181 |
|
215 | 182 | 
|
|
0 commit comments