Merge pull request #44 from cachedjdk/semeru-versions

Clean up ibm-semeru-openj9 and graalvm version string quirk handling

Author: marktsuchida

.pre-commit-config.yaml

@@ -19,8 +19,8 @@ repos:
       - id: ruff
         args: ["--fix"]
       - id: ruff-format
-  - repo: https://github.com/executablebooks/mdformat
-    rev: 0.7.22
+  - repo: https://github.com/hukkin/mdformat
+    rev: 1.0.0
     hooks:
       - id: mdformat
         additional_dependencies:

docs/changelog.md

@@ -15,6 +15,11 @@ See also the section on [versioning](versioning-scheme).
 - `list_vendors()` and `ls-vendors` now filter vendors by OS and architecture,
   defaulting to the current platform.
 
+### Removed
+
+- The synthesized `ibm-semeru-openj9` vendor is no longer generated. Use
+  `ibm-semeru` instead (now available directly in the upstream index).
+
 ## [0.5.0] - 2026-01-07
 
 ### Added

docs/example-graal-hello.md

@@ -47,31 +47,11 @@ with open("Hello.java", "w") as fp:
     fp.write(java_source)
 ```
 
-Let's store the keyword arguments to `cjdk.java_env()` so that we can call it
-several times with the same configuration.
-
-```{code-cell} ipython3
-cjdk_config = dict(vendor="graalvm-java17", version="22.1.0")
-```
-
-The GraalVM `native-image` command is not included in the default install, so
-we need to use `gu` (the GraalVM updater) to install it.
-
-(On macOS, you may see warnings related to `setrlimit` in this and following
-steps. They can be ignored.)
-
-```{code-cell} ipython3
-with cjdk.java_env(**cjdk_config):
-    subprocess.run(
-        ["gu", "install", "--no-progress", "native-image"], check=True
-    )
-```
-
 Now let's compile the source, first with `javac` to byte code, then to a native
 image.
 
 ```{code-cell} ipython3
-with cjdk.java_env(**cjdk_config):
+with cjdk.java_env(vendor="graalvm-community", version="25.0.1"):
     subprocess.run(["javac", "Hello.java"], check=True)
     subprocess.run(["native-image", "Hello"], check=True)
 ```

docs/vendors.md

@@ -7,9 +7,9 @@ SPDX-License-Identifier: MIT
 # JDK vendors
 
 **cjdk** allows you to choose among JDKs and JREs released from different
-sources. Names such as `adoptium`, `zulu-jre`, or `graalvm-java17` are used to
-select a particular series of JDKs. These names are referred to as "vendors",
-even though they do not map 1:1 to companies.
+sources. Names such as `adoptium`, `zulu-jre`, or `graalvm-community` are used
+to select a particular series of JDKs. These names are referred to as
+"vendors", even though they do not map 1:1 to companies.
 
 If no vendor is specified, `adoptium` is used unless the environment variable
 [`CJDK_VENDOR`](environ-cjdk-vendor) is set to an alternative default.
@@ -19,18 +19,31 @@ If no vendor is specified, `adoptium` is used unless the environment variable
 The available set of vendors is determined by the [JDK index](./jdk-index.md)
 and is not built into **cjdk** itself.
 
-Common vendors include `adopt`, `adoptium`, `temurin`, `liberica`, `zulu`, and
-their JRE counterparts `adopt-jre`, `adoptium-jre`, `temurin-jre`,
-`liberica-jre`, `zulu-jre`.
-
-AdoptOpenJDK was
-[succeeded](https://blog.adoptium.net/2021/08/adoptium-celebrates-first-release/)
-by Eclipse Temurin by Adoptium in 2021. To specifically get AdoptOpenJDK
-releases, use `adopt`; to specifically get Temurin releases, use `temurin`;
-`adoptium` will get a Temurin release if available, falling back to
-AdoptOpenJDK for older versions. (Again, this behavior is defined by the index,
-not **cjdk** itself.)
-
-For GraalVM, `graalvm-java11`, `graalvm-java16`, and `graalvm-java17` are
-available at the time of writing (these each have [versions](./versions.md)
-that are numbered independently of the regular JDK version).
+Common vendor names for full JDKs include `temurin`, `zulu`, `liberica`,
+`corretto`, `ibm-semeru`, and `graalvm-community`. Common vendor names for JREs
+include `temurin-jre`, `zulu-jre`, and `liberica-jre`.
+
+```{note}
+**Eclipse Temurin** was
+[previously known](https://blog.adoptium.net/2021/08/adoptium-celebrates-first-release/)
+as **AdoptOpenJDK**. To specifically get AdoptOpenJDK releases, use `adopt`; to
+specifically get Temurin releases, use `temurin`; `adoptium` will get a Temurin
+release if available, falling back to AdoptOpenJDK for older versions. (This
+behavior is defined by the index, not **cjdk** itself.)
+```
+
+```{note}
+For **GraalVM**, the recommended vendor name is `graalvm-community`, which uses
+Java-version-aligned numbering (e.g., version `21.0.2` is for Java 21). Legacy
+vendors `graalvm` (Java 8), `graalvm-java11`, `graalvm-java17`, etc., are also
+available; these use GraalVM release version numbers (e.g., `22.3.3`) which are
+independent of the Java version. The `-javaN` suffix indicates the Java
+version.
+```
+
+```{note}
+For **IBM Semeru**, use `ibm-semeru`. There are also vendor names
+like `ibm-semeru-openj9-java11` (per Java major version), containing JDK
+versions that include an OpenJ9 VM version suffix (e.g.,
+`11.0.29+7_openj9-0.56.0`).
+```

docs/versions.md

@@ -14,6 +14,12 @@ The available versions for a given vendor (and OS, architecture) are defined by
 the [JDK index](./jdk-index.md). Different vendors use different numbering
 schemes.
 
+```{attention}
+For legacy GraalVM vendors (`graalvm`, `graalvm-java11`, etc.), version numbers
+are GraalVM release versions, not Java/JDK versions. See
+[vendors](./vendors.md) for details.
+```
+
 If you want to reproducibly install an exact JDK build, you should consult the
 index and specify an exact version in full.
 

src/cjdk/_index.py

@@ -5,8 +5,7 @@
 """
 JDK index handling.
 
-Fetches and caches the Coursier JDK index, parses JSON, normalizes vendor names
-(e.g., merges ibm-semeru-*-java## variants), and performs version
+Fetches and caches the Coursier JDK index, parses JSON, and performs version
 matching/resolution with support for version expressions like "17+".
 
 No actual operations except for caching the index itself. _index should be
@@ -143,44 +142,6 @@ def _read_index(path: Path) -> Index:
     except json.JSONDecodeError as e:
         raise InstallError(f"Invalid JSON in index file {path}: {e}") from e
 
-    return _postprocess_index(index)
-
-
-def _postprocess_index(index: Index) -> Index:
-    """
-    Post-process the index to normalize the data.
-
-    Some "vendors" include the major Java version,
-    so let's merge such entries. In particular:
-
-    * ibm-semuru-openj9-java<##>
-    * graalvm-java<##>
-
-    However: while the graalvm vendors follow this pattern, the version
-    numbers for graalvm are *not* JDK versions, but rather GraalVM versions,
-    which merely strongly resemble JDK version strings. For example,
-    graalvm-java17 version 22.3.3 bundles OpenJDK 17.0.8, but
-    unfortunately there is no way to know this from the index alone.
-    """
-
-    pattern = re.compile("^(jdk@ibm-semeru.*)-java\\d+$")
-    if not hasattr(index, "items"):
-        return index
-    for os, arches in index.items():
-        if not hasattr(arches, "items"):
-            continue
-        for arch, vendors in arches.items():
-            if not hasattr(vendors, "items"):
-                continue
-            for vendor, versions in vendors.copy().items():
-                if not vendor.startswith("jdk@graalvm") and (
-                    m := pattern.match(vendor)
-                ):
-                    true_vendor = m.group(1)
-                    if true_vendor not in index[os][arch]:
-                        index[os][arch][true_vendor] = {}
-                    index[os][arch][true_vendor].update(versions)
-
     return index
 
 
@@ -235,9 +196,10 @@ def _normalize_version(
     ver: str, *, remove_prefix_1: bool = False
 ) -> tuple[int | str, ...]:
     # Normalize requested version and candidates:
-    # - Split at dots and dashes (so we don't distinguish between '.' and '-')
-    # - Try to convert elements to integers (so that we can compare elements
-    #   numerically where feasible)
+    # - Handle _openj9- as a plain separator (for ibm-semeru-openj9-java*
+    #   versions); also handle -m2 suffix
+    # - Split at dots, dashes, plus signs, and underscores
+    # - Convert elements to integers (raise ValueError if not possible)
     # - If remove_prefix_1 and first element is 1, remove it (so JDK 1.8 == 8)
     # - Return as a tuple (so that we compare element by element)
     # - Trailing zero elements are NOT removed, so, e.g., 11 < 11.0 (for the
@@ -247,24 +209,38 @@ def _normalize_version(
     is_plus = ver.endswith("+")
     if is_plus:
         ver = ver[:-1]
+    plus = ("+",) if is_plus else ()
+
+    if "_openj9-" in ver:
+        # ibm-semeru-openj9-java* version numbers have a variable number of
+        # '.'-separated numbers before the '+'. Pad so that comparisons work.
+        first, second = ver.split("+", 1)
+        nfirst = _normalize_version(first, remove_prefix_1=remove_prefix_1)
+        while len(nfirst) < 4:
+            nfirst = nfirst + (0,)
+        nsecond = _normalize_version(
+            second.replace("-m", "-").replace("_openj9-", "-")
+        )
+        return nfirst + nsecond + plus
+
     if ver:
-        norm = tuple(re.split(_VER_SEPS, ver))
-        norm = tuple(_intify(e) for e in norm)
+        parts = re.split(_VER_SEPS, ver)
+        norm = []
+        for part in parts:
+            try:
+                norm.append(int(part))
+            except ValueError:
+                raise ValueError(
+                    f"Non-integer element '{part}' in version"
+                ) from None
+        norm = tuple(norm)
     else:
         norm = ()
-    plus = ("+",) if is_plus else ()
     if remove_prefix_1 and len(norm) and norm[0] == 1:
         return norm[1:] + plus
     return norm + plus
 
 
-def _intify(s: str) -> int | str:
-    try:
-        return int(s)
-    except ValueError:
-        return s
-
-
 def _is_version_compatible_with_spec(
     version: tuple[int | str, ...], spec: tuple[int | str, ...]
 ) -> bool:
@@ -282,25 +258,6 @@ def _is_version_compatible_with_spec(
     return len(version) >= len(spec) and version[: len(spec)] == spec
 
 
-class _VersionElement:
-    """Wrapper for version tuple elements enabling mixed int/str comparison."""
-
-    def __init__(self, value: int | str) -> None:
-        self.value = value
-
-    def __eq__(self, other: object) -> bool:
-        if not isinstance(other, _VersionElement):
-            return NotImplemented
-        if isinstance(self.value, int) and isinstance(other.value, int):
-            return self.value == other.value
-        return str(self.value) == str(other.value)
-
-    def __lt__(self, other: _VersionElement) -> bool:
-        if isinstance(self.value, int) and isinstance(other.value, int):
-            return self.value < other.value
-        return str(self.value) < str(other.value)
-
-
 def matching_jdk_versions(index: Index, conf: Configuration) -> list[str]:
     """
     Return all version strings matching the configuration, sorted by version.
@@ -313,10 +270,4 @@ def matching_jdk_versions(index: Index, conf: Configuration) -> list[str]:
     if not versions:
         return []
     matched = _match_versions(conf.vendor, versions, conf.version)
-
-    def version_sort_key(
-        item: tuple[tuple[int | str, ...], str],
-    ) -> tuple[_VersionElement, ...]:
-        return tuple(_VersionElement(e) for e in item[0])
-
-    return [v for _, v in sorted(matched.items(), key=version_sort_key)]
+    return [v for _, v in sorted(matched.items())]

tests/test_index.py

@@ -162,34 +162,6 @@ def test_read_index(tmp_path):
     assert _index._read_index(path) == data
 
 
-def test_postprocess_index():
-    index = {
-        "linux": {
-            "amd64": {
-                "jdk@ibm-semeru-openj9-java11": {
-                    "11.0.21+9_openj9-0.41.0": "a",
-                    "11.0.22+7_openj9-0.43.0": "b",
-                    "11.0.23+9_openj9-0.44.0": "c",
-                },
-                "jdk@ibm-semeru-openj9-java17": {
-                    "17.0.1+12_openj9-0.29.1": "d",
-                    "17.0.2+8_openj9-0.30.0": "e",
-                    "17.0.3+7_openj9-0.32.0": "f",
-                },
-                "jdk@ibm-semeru-openj9-java21": {
-                    "21.0.1+12_openj9-0.42.0": "g",
-                    "21.0.2+13_openj9-0.43.0": "h",
-                },
-                "jdk@not-semeru": {"8.0.252": "i"},
-            }
-        }
-    }
-    pp_index = _index._postprocess_index(index)
-    assert pp_index is index
-    assert "jdk@ibm-semeru-openj9" in index["linux"]["amd64"]
-    assert len(index["linux"]["amd64"]["jdk@ibm-semeru-openj9"]) == 8
-
-
 def test_match_versions():
     f = _index._match_versions
     assert f("adoptium", ["10", "11.0", "11.1", "1.12.0"], "11") == {
@@ -238,8 +210,14 @@ def test_normalize_version():
     assert f("1", remove_prefix_1=True) == ()
     assert f("1.8", remove_prefix_1=True) == (8,)
     assert f("1.8.0", remove_prefix_1=True) == (8, 0)
-    assert f("1.8u300", remove_prefix_1=True) == ("8u300",)
-    assert f("21.0.1+12_openj9-0.42.0") == (21, 0, 1, 12, "openj9", 0, 42, 0)
+    assert f("17.0.4.1+1_openj9-0.33.1") == (17, 0, 4, 1, 1, 0, 33, 1)
+    assert f("21.0.1+12_openj9-0.42.0") == (21, 0, 1, 0, 12, 0, 42, 0)
+    assert f("23+37_openj9-0.47.0.0.0") == (23, 0, 0, 0, 37, 0, 47, 0, 0, 0)
+    assert f("23.0.1+11_openj9-0.49.0-m2") == (23, 0, 1, 0, 11, 0, 49, 0, 2)
+    with pytest.raises(ValueError):
+        f("23.4.5_openj9-42")  # No '+' despite having _openj9-
+    with pytest.raises(ValueError):
+        f("1.8u300")  # No longer seen in index
 
 
 def test_is_version_compatible_with_spec():

tests/test_integration.py

@@ -11,7 +11,7 @@ def test_list_vendors():
     assert "adoptium" in vendors
     assert "corretto" in vendors
     assert "graalvm" in vendors
-    assert "ibm-semeru-openj9" in vendors
+    assert "ibm-semeru" in vendors
     assert "java-oracle" in vendors
     assert "liberica" in vendors
     assert "temurin" in vendors