Split precompiled data into a sub-project (and wheel)

The goal of this is the ability to generate wheels for precompiled
instances of uap-core, at whatever version we want.

1. It resolves #146 by splitting the versioning of the API and that of
   the (pre-compiled) data, this is an issue for 1.0 as that detaches
   uap-python's versioning from uap-core's.
2. It allows users to update the API and the precompiled dataset
   separately, something they would otherwise need to do via yaml.
3. It fixes #221 by allowing the regular release of "preview"
   precompiled regexes from uap-core snapshots e.g. we could release
   0.19.dev202412 at the start of december with whatever uap-core
   merged between the previous prerelease and then. This should not be
   picked up by pip by default, but would allow users to access those
   prerelases via `pip install --pre`.
4. If done well enough, it might allow users to build bespoke
   precompiled datasets so they don't have to pick between custom
   rules and precompiled (not sure there's any demand for this but it
   seems like it might be useful).
5. If it works well enough it might actually be possible to have 0.x
   use the legacy codegen package meaning it should not need to be
   updated anymore.

This is implemented via hatch build hooks (which seem seem simpler
than doing it via setuptools in the end).

Adding `regexes.yaml` to the sdist via artifacts is a bit strange but
necessary in order to generate a complete sdist which a wheel can be
built from (even though the release script will likely only push the
wheel).

Author: masklinn

.github/workflows/ci.yml

@@ -3,14 +3,16 @@ name: CI
 on:
   push:
   pull_request:
-  workflow_dispatch:
 
 jobs:
   checks:
     runs-on: ubuntu-latest
     steps:
     - name: Checkout working copy
       uses: actions/checkout@v4
+      with:
+        submodules: true
+        fetch-depth: 0
     - name: ruff check
       uses: chartboost/ruff-action@v1
     - name: ruff format
@@ -29,7 +31,7 @@ jobs:
       if: ${{ always() && steps.setup_python.conclusion == 'success' }}
       run: |
         python -mpip install --upgrade pip
-        python -mpip install mypy types-PyYaml
+        python -mpip install mypy types-PyYaml ./ua-parser-builtins
     - name: mypy
       if: ${{ always() && steps.install_mypy.conclusion == 'success' }}
       run: mypy
@@ -101,6 +103,7 @@ jobs:
       uses: actions/checkout@v4
       with:
         submodules: true
+        fetch-depth: 0
     - name: Set up Python ${{ matrix.python-version }}
       uses: actions/setup-python@v5
       with:
@@ -115,6 +118,7 @@ jobs:
             sudo apt install libyaml-dev
         fi
     - run: python -mpip install pytest pyyaml
+    - run: python -mpip install ./ua-parser-builtins
     # install rs accelerator if available, ignore if not
     - run: python -mpip install ua-parser-rs || true
     # re2 is basically impossible to install from source so don't

pyproject.toml

@@ -8,7 +8,7 @@ description = "Python port of Browserscope's user agent parser"
 version = "1.0.0a1"
 readme = "README.rst"
 requires-python = ">=3.9"
-dependencies = []
+dependencies = ["ua-parser-builtins"]
 
 license = {text = "Apache 2.0"}
 urls = {repository = "https://github.com/ua-parser/uap-python"}
@@ -57,8 +57,7 @@ where = ["src"]
 
 [tool.ruff]
 exclude = [
-    "src/ua_parser/_lazy.py",
-    "src/ua_parser/_matchers.py",
+    "src/ua_parser/generate_builtins.py",
 ]
 
 [tool.ruff.lint]

setup.cfg

@@ -52,7 +52,7 @@ def load_builtins() -> Matchers:
     further imports simply reference the existing datas.
 
     """
-    from ._matchers import MATCHERS
+    from ua_parser_builtins.matchers import MATCHERS
 
     # typing and mypy don't have safe upcast (#5756) and mypy is
     # unhappy about returning concrete matchers for a mixed type
@@ -66,7 +66,7 @@ def load_lazy_builtins() -> Matchers:
     further imports simply reference the existing datas.
 
     """
-    from ._lazy import MATCHERS
+    from ua_parser_builtins.lazy import MATCHERS
 
     return cast(Matchers, MATCHERS)
 

setup.py

@@ -521,4 +521,8 @@ def GetFilters(
     del SafeLoader
 else:
     # Just load our pre-compiled versions
-    from ._regexes import DEVICE_PARSERS, OS_PARSERS, USER_AGENT_PARSERS
+    from ua_parser_builtins.regexes import (
+        DEVICE_PARSERS,
+        OS_PARSERS,
+        USER_AGENT_PARSERS,
+    )