Update documentation and test for the standalone module

Author: timfel

docs/user/PythonStandaloneBinaries.md

@@ -9,38 +9,48 @@ permalink: /reference-manual/python/standalone-binaries/
 It can be desirable to distribute Python applications or libraries as standalone binaries or JAR files without any external dependencies.
 The Truffle framework that GraalPy is built on and the Sulong LLVM runtime that GraalPy can leverage for managed execution of Python's native extensions allow us to completely virtualize all filesystem accesses of Python programs, including those to the standard library and installed packages.
 
-GraalPy comes with a module, `py2bin`, that creates a Maven project skeleton to bundle and run self-contained JAR files with Python code.
-The generated skeleton can also be used to generate a standalone binary that includes all the Python code using GraalVM Native Image.
+GraalPy comes with a module that can create standalone binaries or Java project skeletons.
+The binaries simply bundle everything into one native image.
+The Java skeletons are set up with Maven to build and run self-contained JAR files.
+They can also be used to generate a standalone binaries from those JARs later, so the Java skeletons offer more flexibility and control over the steps.
 
-## Usage
+## Creating GraalPy Binaries
 
 Suppose we have a simple Python script `my_script.py` that does some useful work when run directly.
 If we wanted to distribute it as a standalone native image, we run the following command:
 
 ```
-$ graalpy -m py2bin my_script_application/ my_script.py
+$ graalpy -m standalone binary --module my_script.py --output my_binary
 ```
 
-The target folder `my_script_application` will now include a `pom.xml` that makes it easy to generate a native image with maven.
-Make sure to set your `JAVA_HOME` to use a GraalVM distribution that includes the native-image binary:
+This will generate a standalone `my_binary` file which includes the Python code, the GraalPy runtime, and the Python standard library in a single, self-contained executable.
+Use `graalpy -m standalone binary --help` for further options.
+
+## Embedding GraalPy in Java Applications
+
+Suppose now we wanted to distribute our `my_script.py` script as a Jar that can run on any GraalVM that includes GraalPy.
+To prepare such a Java project, the `standalone` GraalPy tool has another command:
+
 ```
-$ mvn -Pnative package
+$ graalpy -m standalone java --output-directory MyJavaApplication --module my_script
 ```
 
-Afterwards, the standalone binary can be found in the `target` directory.
-It can be moved freely and launched directly, and does not require any additional resources.
+The target folder `MyJavaApplication` includes a `pom.xml` that makes it easy to generate a JAR or a GraalVM native image with Maven.
+You can open this Maven project with any Java IDE and edit the main class that was created to modify the Python embedding.
+To build, you can use either `mvn -Pjar package` to create a JAR, or `mvn -Pnative package` to create a GraalVM native image.
+For both, make sure to set your `JAVA_HOME` to use a GraalVM distribution.
 
 Take a look at the generated `pom.xml`.
 There are some options to tweak the performance and footprint trade-off.
 Review also our general documentation on Python native images to find out how to remove other unwanted components and further reduce the binary size.
 
 The generated project should be viewed as a starting point.
-It includes the entire Python standard library.
-This can be manually pruned to reduce it to the necessary amount, reducing both the size of the binary and the time to start up.
+It includes the entire Python standard library, so the Python code can invoke all of the standard library code.
+The resources can be manually pruned to reduce the included Python libraries to the necessary amount, reducing both the size of the package and the time to start up.
 The Java code demonstrates some useful default options for the Python context, but other settings may be desirable to further control what the Python code is allowed to do.
 
 ## Security considerations
 
-Creating a native image that includes the Python code could be seen as a mild form of obfuscation, but it does not protect your source code.
+Creating a native image or a JAR that includes the Python code could be seen as a mild form of obfuscation, but it does not protect your source code.
 While the Python sources are not stored verbatim into the image (only the GraalPy bytecode is), that bytecode is easy to convert back into Python sources.
 If stronger protection for the included Python source code is required, this needs to be handled by a means of e.g. encryption of the resources before building the native image, and adding approproate decryption into the generated virtual file system.

graalpython/lib-graalpython/modules/standalone/__main__.py

@@ -79,7 +79,9 @@ def ensure_directories(zf, path):
             zf.writestr(zipfile.ZipInfo(dirname), b"")
 
 
-def bundle_python_resources(zipname, vfs_prefix, home_prefix, venv_prefix, proj_prefix, project, venv=None):
+def bundle_python_resources(
+    zipname, vfs_prefix, home_prefix, venv_prefix, proj_prefix, project, venv=None
+):
     """
     Bundle the Python core, stdlib, venv, and module into a zip file.
     """
@@ -197,18 +199,16 @@ def main(args):
 
     subparsers = parser.add_subparsers(required=True)
 
-    module_argument_args = {
-        "dest": "module",
-        "help": "Python file or module folder to run",
-    }
-    venv_argument_args = {"dest": "venv", "help": "Python venv to use", "nargs": "?"}
-
     parser_bin = subparsers.add_parser(
         "binary", help="Create a standalone binary from the Python code directly."
     )
-    parser_bin.add_argument(**module_argument_args)
-    parser_bin.add_argument(**venv_argument_args)
-    parser_bin.add_argument('-o', '--output', required=True, help='Output filename for the binary')
+    parser_bin.add_argument(
+        "-m", "--module", help="Python file or module folder to run", required=True
+    )
+    parser_bin.add_argument("--venv", help="Python venv to bundle")
+    parser_bin.add_argument(
+        "-o", "--output", help="Output filename for the binary", required=True
+    )
     parser_bin.add_argument(
         "-Os", action="store_true", help="Optimize the binary for size, not speed"
     )
@@ -226,10 +226,15 @@ def main(args):
         help="Create a Java project from the Python code. This gives the most flexibility, as the project can be used to build both standalone Jar files or native binaries using Maven.",
     )
     parser_jar.add_argument(
-        "target", help="The directory to write the Java project to."
+        "-m", "--module", help="Python file or module folder to run", required=True
+    )
+    parser_jar.add_argument("--venv", help="Python venv to bundle")
+    parser_jar.add_argument(
+        "-o",
+        "--output-directory",
+        help="The directory to write the Java project to.",
+        required=True,
     )
-    parser_jar.add_argument(**module_argument_args)
-    parser_jar.add_argument(**venv_argument_args)
 
     parsed_args = parser.parse_args(args)
 
@@ -242,13 +247,13 @@ def main(args):
         proj_prefix,
     ) = parse_path_constants(java_launcher_template)
 
-    preparing_java_project = hasattr(parsed_args, 'target')
+    preparing_java_project = hasattr(parsed_args, "output_directory")
 
     if preparing_java_project:
         ni, jc = "", ""
         resource_prefix = os.path.join("src", "main", "resources")
         code_prefix = os.path.join("src", "main", "java")
-        targetdir = parsed_args.target
+        targetdir = parsed_args.output_directory
     else:
         ni, jc = get_tools()
         resource_prefix = ""

mx.graalpython/mx_graalpython.py

@@ -1211,7 +1211,7 @@ def graalpython_gate_runner(args, tasks):
                 tmpmain = os.path.join(tmpdir, "main.py")
                 with open(tmpmain, "w") as f:
                     f.write("print('hello standalone')")
-                mx.run([svm_image, "-m", "py2bin", "java", tmpstandalone, tmpmain])
+                mx.run([svm_image, "-m", "standalone", "java", "-o", tmpstandalone, "-m", tmpmain])
                 mx.run_maven(["-Pjar", "package"], cwd=tmpstandalone) # should compile without GraalVM
                 mx.run_maven(
                     ["-Pnative", "package"],
@@ -1229,7 +1229,7 @@ def graalpython_gate_runner(args, tasks):
                 if "hello standalone" not in out.data:
                     mx.abort('Output from generated SVM image "' + svm_image + '" did not match success pattern:\n' + success)
 
-                mx.run([svm_image, "-m", "py2bin", "binary", "-Os", "-o", os.path.join(tmpdir, "directlauncher"), tmpstandalone, tmpmain])
+                mx.run([svm_image, "-m", "standalone", "binary", "-Os", "-o", os.path.join(tmpdir, "directlauncher"), "-m", tmpmain])
                 out = mx.OutputCapture()
                 mx.run(
                     [os.path.join(tmpdir, "directlauncher")],