Skip to content

Commit 0cb4ed8

Browse files
author
Daniele Briggi
committed
feat(python): workflow to build and upload the pypi package
1 parent adae11e commit 0cb4ed8

File tree

7 files changed

+362
-0
lines changed

7 files changed

+362
-0
lines changed

‎.github/workflows/python-package.yml‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
name: Build and Publish Python Wheels
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
version:
7+
description: "Version to use for the Python package (e.g. 0.5.9)"
8+
required: true
9+
type: string
10+
release:
11+
types: [published]
12+
13+
jobs:
14+
build-and-publish:
15+
runs-on: ${{ matrix.os }}
16+
strategy:
17+
matrix:
18+
include:
19+
- os: ubuntu-latest
20+
platform: linux
21+
python-version: "3.10"
22+
arch: x86_64
23+
plat_name: linux_x86_64
24+
- os: ubuntu-latest
25+
platform: linux
26+
python-version: "3.10"
27+
arch: arm64
28+
plat_name: linux_aarch64
29+
- os: windows-latest
30+
platform: windows
31+
python-version: "3.10"
32+
arch: x86_64
33+
plat_name: win_amd64
34+
- os: macos-latest
35+
platform: macos
36+
python-version: "3.10"
37+
arch: x86_64
38+
plat_name: macosx_10_9_x86_64
39+
- os: macos-latest
40+
platform: macos
41+
python-version: "3.10"
42+
arch: arm64
43+
plat_name: macosx_11_0_arm64
44+
defaults:
45+
run:
46+
shell: bash
47+
steps:
48+
- uses: actions/checkout@v4
49+
with:
50+
submodules: false
51+
52+
- name: Set up Python
53+
uses: actions/setup-python@v5
54+
with:
55+
python-version: ${{ matrix.python-version }}
56+
57+
- name: Install build dependencies
58+
run: |
59+
python3 -m pip install --upgrade pip
60+
pip install .[dev]
61+
62+
- name: Get version
63+
id: get_version
64+
run: |
65+
if [[ "${{ github.event_name }}" == "release" ]]; then
66+
VERSION="${{ github.event.release.tag_name }}"
67+
else
68+
VERSION="${{ github.event.inputs.version }}"
69+
fi
70+
VERSION=${VERSION#v}
71+
echo "version=$VERSION" >> $GITHUB_OUTPUT
72+
73+
- name: Download artifacts for current platform
74+
run: |
75+
cd python-package
76+
python3 download_artifacts.py "${{ matrix.platform }}" "${{ steps.get_version.outputs.version }}"
77+
78+
- name: Build wheel
79+
env:
80+
PACKAGE_VERSION: ${{ steps.get_version.outputs.version }}
81+
run: |
82+
cd python-package
83+
python setup.py clean --all bdist_wheel --plat-name "${{ matrix.plat_name }}"
84+
85+
- name: Publish to PyPI
86+
if: github.event_name == 'release'
87+
uses: pypa/gh-action-pypi-publish@release/v1
88+
with:
89+
packages-dir: python-package/dist

‎python-package/MANIFEST.in‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
include README.md
2+
include LICENSE
3+
recursive-include src/sqliteai/binaries *

‎python-package/README.md‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
## SQLite AI Python package
2+
3+
This package provides the sqlite-ai extension prebuilt binaries for multiple platforms and architectures.
4+
5+
## Usage
6+
7+
```python
8+
import importlib.resources
9+
import sqlite3
10+
11+
# Connect to your SQLite database
12+
conn = sqlite3.connect("example.db")
13+
14+
# Load the sqlite-ai extension (Linux/CPU example)
15+
ext_path = importlib.resources.files("sqliteai.binaries.linux_x86_64") / "ai"
16+
17+
conn.enable_load_extension(True)
18+
conn.load_extension(str(ext_path))
19+
conn.enable_load_extension(False)
20+
21+
22+
# Now you can use sqlite-ai features in your SQL queries
23+
```
24+
25+
---
26+
27+
### Platform-specific extension paths
28+
29+
The sqlite-ai extension binary is included in a subpackage based on your platform and architecture.
30+
Use the following subpackage names to locate the extension:
31+
32+
| Platform | Arch | Subpackage name | Binary name |
33+
| ------------- | ------ | ------------------------------------ | ----------- |
34+
| Linux (CPU) | x86_64 | sqliteai.binaries.linux_x86_64 | ai.so |
35+
| Linux (GPU) | x86_64 | sqliteai.binaries.linux_x86_64_gpu | ai.so |
36+
| Linux (CPU) | arm64 | sqliteai.binaries.linux_arm64 | ai.so |
37+
| Linux (GPU) | arm64 | sqliteai.binaries.linux_arm64_gpu | ai.so |
38+
| Windows (CPU) | x86_64 | sqliteai.binaries.windows_x86_64 | ai.dll |
39+
| Windows (GPU) | x86_64 | sqliteai.binaries.windows_x86_64_gpu | ai.dll |
40+
| macOS (CPU) | x86_64 | sqliteai.binaries.macos_x86_64 | ai.dylib |
41+
| macOS (CPU) | arm64 | sqliteai.binaries.macos_arm64 | ai.dylib |
42+
43+
44+
#### Programmatic selection
45+
46+
You can use the `platform` module to select the correct subpackage and binary name, here's the example:
47+
48+
```python
49+
import platform
50+
import importlib.resources
51+
52+
system = platform.system().lower() # 'linux', 'windows', 'darwin'
53+
machine = platform.machine().lower() # 'x86_64', 'amd64', 'arm64', etc.
54+
55+
if system == "linux" and machine == "x86_64":
56+
subpkg = "sqliteai.binaries.linux_x86_64"
57+
binname = "ai.so"
58+
elif system == "linux" and machine == "arm64":
59+
subpkg = "sqliteai.binaries.linux_arm64"
60+
binname = "ai.so"
61+
elif system == "windows" and machine in ("x86_64", "amd64"):
62+
subpkg = "sqliteai.binaries.windows_x86_64"
63+
binname = "ai.dll"
64+
elif system == "darwin" and machine == "x86_64":
65+
subpkg = "sqliteai.binaries.macos_x86_64"
66+
binname = "ai.dylib"
67+
elif system == "darwin" and machine == "arm64":
68+
subpkg = "sqliteai.binaries.macos_arm64"
69+
binname = "ai.dylib"
70+
# Add more cases as needed
71+
72+
ext_path = importlib.resources.files(subpkg) / binname
73+
```

‎python-package/download_artifacts.py‎

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
import sys
2+
import zipfile
3+
import requests
4+
from pathlib import Path
5+
import shutil
6+
7+
8+
# == USAGE ==
9+
# python3 download_artifacts.py PLATFORM VERSION
10+
# eg: python3 download_artifacts.py linux_x86_64 "0.5.9"
11+
12+
REPO = "sqliteai/sqlite-ai"
13+
RELEASE_URL = f"https://github.com/{REPO}/releases/download"
14+
15+
# Map Python plat_name to artifact names
16+
ARTIFACTS = {
17+
"linux_x86_64": ["ai-linux-cpu-x86_64", "ai-linux-gpu-x86_64"],
18+
"linux_aarch64": [
19+
"ai-linux-cpu-arm64",
20+
"ai-linux-gpu-arm64",
21+
],
22+
"win_amd64": ["ai-windows-cpu-x86_64", "ai-windows-gpu-x86_64"],
23+
"macosx_10_9_x86_64": ["ai-macos"],
24+
"macosx_11_0_arm64": ["ai-macos"],
25+
}
26+
27+
BINARY_NAME = {
28+
"linux_x86_64": "ai.so",
29+
"linux_aarch64": "ai.so",
30+
"win_amd64": "ai.dll",
31+
"macosx_10_9_x86_64": "ai.dylib",
32+
"macosx_11_0_arm64": "ai.dylib",
33+
}
34+
35+
BINARIES_DIR = Path(__file__).parent / "src/sqliteai/binaries"
36+
37+
38+
def download_and_extract(artifact_name, bin_name, version):
39+
artifact = f"{artifact_name}-{version}.zip"
40+
url = f"{RELEASE_URL}/{version}/{artifact}"
41+
print(f"Downloading {url}")
42+
43+
r = requests.get(url)
44+
if r.status_code != 200:
45+
print(f"Failed to download {artifact}: {r.status_code}")
46+
sys.exit(1)
47+
48+
zip_path = BINARIES_DIR / artifact
49+
with open(zip_path, "wb") as f:
50+
f.write(r.content)
51+
52+
subdir = "gpu" if "gpu" in artifact_name else "cpu"
53+
out_dir = BINARIES_DIR / subdir
54+
out_dir.mkdir(parents=True, exist_ok=True)
55+
56+
with zipfile.ZipFile(zip_path, "r") as zip_ref:
57+
for member in zip_ref.namelist():
58+
if member.endswith(bin_name):
59+
zip_ref.extract(member, out_dir)
60+
61+
# Move to expected name/location
62+
src = out_dir / member
63+
dst = out_dir / bin_name
64+
src.rename(dst)
65+
66+
print(f"Extracted {dst}")
67+
68+
zip_path.unlink()
69+
70+
71+
def main():
72+
version = None
73+
platform = None
74+
if len(sys.argv) > 2:
75+
platform = sys.argv[1].lower()
76+
version = sys.argv[2]
77+
78+
if not version or not platform:
79+
print(
80+
'Error: Version is not specified.\nUsage: \n python3 download_artifacts.py linux_x86_64 "0.5.9"'
81+
)
82+
sys.exit(1)
83+
84+
print(BINARIES_DIR)
85+
if BINARIES_DIR.exists():
86+
shutil.rmtree(BINARIES_DIR)
87+
BINARIES_DIR.mkdir(parents=True, exist_ok=True)
88+
89+
platform_artifacts = ARTIFACTS.get(platform, [])
90+
if not platform_artifacts:
91+
print(f"Error: Unknown platform '{platform}'")
92+
sys.exit(1)
93+
94+
for artifact_name in platform_artifacts:
95+
download_and_extract(artifact_name, BINARY_NAME[platform], version)
96+
97+
98+
if __name__ == "__main__":
99+
main()

‎python-package/pyproject.toml‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
[build-system]
2+
requires = ["setuptools>=61.0", "wheel"]
3+
build-backend = "setuptools.build_meta"
4+
5+
[project]
6+
name = "sqlite-ai"
7+
dynamic = ["version"]
8+
description = "Python bindings and prebuilt binaries for SQLite AI extension."
9+
authors = [
10+
{ name = "SQLite AI Team" }
11+
]
12+
readme = "README.md"
13+
requires-python = ">=3"
14+
classifiers = []
15+
16+
[project.optional-dependencies]
17+
dev = [
18+
"requests",
19+
"toml",
20+
"setuptools",
21+
"wheel"
22+
]
23+
24+
[project.urls]
25+
Homepage = "https://sqlite.ai"
26+
Documentation = "https://github.com/sqliteai/sqlite-ai/blob/main/API.md"
27+
Repository = "https://github.com/sqliteai/sqlite-ai"
28+
Issues = "https://github.com/sqliteai/sqlite-ai/issues"

‎python-package/setup.py‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
import setuptools
2+
import toml
3+
import os
4+
import sys
5+
6+
usage = """
7+
Usage: python setup.py bdist_wheel --plat-name <platform>
8+
The PACKAGE_VERSION environment variable must be set to the desired version.
9+
10+
Example:
11+
PACKAGE_VERSION=0.5.9 python setup.py bdist_wheel --plat-name linux_x86_64
12+
"""
13+
14+
# Read pyproject.toml for metadata
15+
with open("pyproject.toml", "r") as f:
16+
pyproject = toml.load(f)
17+
18+
project = pyproject["project"]
19+
20+
# Get version from environment or default
21+
version = os.environ.get("PACKAGE_VERSION", "")
22+
if not version:
23+
print("PACKAGE_VERSION environment variable is not set.")
24+
print(usage)
25+
exit(1)
26+
27+
# Get Python platform name from --plat-name argument
28+
plat_name = None
29+
for i, arg in enumerate(sys.argv):
30+
if arg == "--plat-name" and i + 1 < len(sys.argv):
31+
plat_name = sys.argv[i + 1]
32+
break
33+
34+
if not plat_name:
35+
print("Error: --plat-name argument is required")
36+
print(usage)
37+
sys.exit(1)
38+
39+
# Map plat_name to classifier
40+
classifier_map = {
41+
"linux_x86_64": "Operating System :: POSIX :: Linux",
42+
"linux_aarch64": "Operating System :: POSIX :: Linux",
43+
"win_amd64": "Operating System :: Microsoft :: Windows",
44+
"macosx_10_9_x86_64": "Operating System :: MacOS",
45+
"macosx_11_0_arm64": "Operating System :: MacOS",
46+
}
47+
48+
classifier = classifier_map.get(plat_name)
49+
if not classifier:
50+
print(f"Unknown plat_name: {plat_name}")
51+
sys.exit(1)
52+
53+
setuptools.setup(
54+
name=project["name"],
55+
version=version,
56+
description=project.get("description", ""),
57+
author=project["authors"][0]["name"] if project.get("authors") else "",
58+
long_description=open("README.md").read(),
59+
long_description_content_type="text/markdown",
60+
url=project["urls"]["Homepage"],
61+
packages=setuptools.find_packages(where="src"),
62+
package_dir={"": "src"},
63+
include_package_data=True,
64+
python_requires=project.get("requires-python", ">=3"),
65+
classifiers=[
66+
"Programming Language :: Python :: 3",
67+
"License :: OSI Approved :: MIT License",
68+
classifier
69+
],
70+
)

‎python-package/src/sqliteai/__init__.py‎

Whitespace-only changes.

0 commit comments

Comments
 (0)