Skip to content

Commit 54339b7

Browse files
habedihabedi
committed
Improve project documentation
1 parent 0d61001 commit 54339b7

24 files changed

+892
-251
lines changed

‎.github/workflows/dist_pipeline.yml‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,11 +11,20 @@ concurrency:
1111
cancel-in-progress: true
1212

1313
jobs:
14-
build_extensions:
14+
duckdb-next-build:
1515
uses: duckdb/extension-ci-tools/.github/workflows/_extension_distribution.yml@main
1616
with:
1717
duckdb_version: main
1818
ci_tools_version: main
1919
extension_name: infera
2020
enable_rust: true
2121
exclude_archs: "linux_amd64_musl;windows_amd64_mingw;osx_amd64;wasm_mvp;wasm_eh;wasm_threads"
22+
23+
duckdb-stable-build:
24+
uses: duckdb/extension-ci-tools/.github/workflows/[email protected]
25+
with:
26+
duckdb_version: v1.4.0
27+
ci_tools_version: v1.4.0
28+
extension_name: infera
29+
enable_rust: true
30+
exclude_archs: "linux_amd64_musl;windows_amd64_mingw;osx_amd64;wasm_mvp;wasm_eh;wasm_threads"

‎README.md‎

Lines changed: 14 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -13,18 +13,18 @@
1313
[![Docs](https://img.shields.io/badge/docs-view-blue?style=flat&labelColor=282c34&logo=read-the-docs)](https://github.com/CogitatorTech/infera/tree/main/docs)
1414
[![License](https://img.shields.io/badge/license-MIT%2FApache--2.0-007ec6?style=flat&labelColor=282c34&logo=open-source-initiative)](https://github.com/CogitatorTech/infera)
1515

16-
Use machine learning models directly in your queries in DuckDB
16+
In-Database Machine Learning for DuckDB
1717

1818
</div>
1919

2020
---
2121

22-
Infera is DuckDB extension that lets you use machine learning (ML) models directly in SQL queries to perform inference
22+
Infera is DuckDB extension that allows you use machine learning (ML) models directly in SQL queries to perform inference
2323
on data stored in DuckDB tables.
2424
It is developed in Rust and uses [Tract](https://github.com/snipsco/tract) as the backend inference engine.
2525
Infera supports loading and running models in [ONNX](https://onnx.ai/) format.
26-
Check out the [ONNX Model Zoo](https://huggingface.co/onnxmodelzoo) repositors on Hugging Face for a very large collection of
27-
ready-to-use models that can be used with Infera.
26+
Check out the [ONNX Model Zoo](https://huggingface.co/onnxmodelzoo) repositors on Hugging Face for a very large
27+
collection of ready-to-use models that can be used with Infera.
2828

2929
### Motivation
3030

@@ -61,23 +61,28 @@ See the [ROADMAP.md](ROADMAP.md) for the list of implemented and planned feature
6161
LOAD
6262
infera;
6363

64-
-- 2. Load local ONNX model
65-
SELECT infera_load_model('linear_model', 'tests/models/linear.onnx');
64+
-- 2. Load a simple linear model from a remote URL
65+
SELECT infera_load_model('linear_model',
66+
'https://github.com/CogitatorTech/infera/raw/refs/heads/main/test/models/linear.onnx');
6667

6768
-- 3. Run a prediction using a very simple linear model
68-
-- Model is y = 2*x1 - 1*x2 + 0.5*x3 + 0.25
69+
-- Model: y = 2*x1 - 1*x2 + 0.5*x3 + 0.25
6970
SELECT infera_predict('linear_model', 1.0, 2.0, 3.0);
7071
-- Expected output: 1.75
7172

72-
-- 4. Unload the model to free memory
73+
-- 4. Unload the model when we're done with it
7374
SELECT infera_unload_model('linear_model');
7475
````
7576

7677
---
7778

7879
### Documentation
7980

80-
Check out the [docs](docs/README.md) directory for the API documentation and usage examples.
81+
Check out the [docs](docs/README.md) directory for the API documentation, how to build Infera from source, and more.
82+
83+
#### Examples
84+
85+
Check out the [examples](docs/examples) directory for SQL scripts that show how to use Infera.
8186

8287
---
8388

‎docs/README.md‎

Lines changed: 106 additions & 95 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,25 @@
11
### API Reference
22

3-
| Function | Return Type | Description |
4-
|:--------------------------------------------------------|:-----------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
5-
| `infera_load_model(name VARCHAR, path_or_url VARCHAR)` | `BOOLEAN` | Loads an ONNX model from a local path or remote URL. |
6-
| `infera_unload_model(name VARCHAR)` | `BOOLEAN` | Unloads a model from memory to free resources. |
7-
| `infera_set_autoload_dir(path VARCHAR)` | `VARCHAR (JSON)` | Scans a directory, loads all the `.onnx` files in it (models), and returns a JSON report. |
8-
| `infera_get_loaded_models()` | `VARCHAR (JSON)` | Returns a JSON array containing the names of models that are currently loaded and are ready to be used. |
9-
| `infera_get_model_info(name VARCHAR)` | `VARCHAR (JSON)` | Returns a JSON object with information about a specific (loaded) model. |
10-
| `infera_predict(name VARCHAR, features... FLOAT)` | `FLOAT` | Performs inference and returns a single float value. |
11-
| `infert_predict_multi(name VARCHAR, features... FLOAT)` | `VARCHAR (JSON)` | Performs inference and returns all outputs as a JSON array. This is useful for models that prodcue more than one predictions per sample, like in multi-target regression. |
12-
| `infera_predict_from_blob(name VARCHAR, data BLOB)` | `LIST[FLOAT]` | Performs inference on a raw `BLOB` of tensor data, like image data stored in the database. |
13-
| `infera_get_version()` | `VARCHAR (JSON)` | Returns a JSON object with the information about the current version of the extension. |
14-
15-
-----
3+
Table below includes the information about all SQL functions exposed by the Infera.
4+
5+
| Function | Return Type | Description |
6+
|:--------------------------------------------------------|:-----------------|:--------------------------------------------------------------------------------------------------------------------------------------------|
7+
| `infera_load_model(name VARCHAR, path_or_url VARCHAR)` | `BOOLEAN` | Loads an ONNX model from a local file path or a remote URL and assigns it a unique name. Returns `true` on success. |
8+
| `infera_unload_model(name VARCHAR)` | `BOOLEAN` | Unloads a model, freeing its associated resources. Returns `true` on success. |
9+
| `infera_set_autoload_dir(path VARCHAR)` | `VARCHAR (JSON)` | Scans a directory for `.onnx` files, loads them automatically, and returns a JSON report of loaded models and any errors. |
10+
| `infera_get_loaded_models()` | `VARCHAR (JSON)` | Returns a JSON array containing the names of all currently loaded models. |
11+
| `infera_get_model_info(name VARCHAR)` | `VARCHAR (JSON)` | Returns a JSON object with metadata about a specific loaded model, including its name, input/output shapes, and status. |
12+
| `infera_predict(name VARCHAR, features... FLOAT)` | `FLOAT` | Performs inference on a batch of data, returning a single float value for each input row. |
13+
| `infera_predict_multi(name VARCHAR, features... FLOAT)` | `VARCHAR (JSON)` | Performs inference and returns all outputs as a JSON-encoded array. This is useful for models that produce multiple predictions per sample. |
14+
| `infera_predict_from_blob(name VARCHAR, data BLOB)` | `LIST[FLOAT]` | Performs inference on raw `BLOB` data (for example, used for an image tensor), returning the result as a list of floats. |
15+
| `infera_get_version()` | `VARCHAR (JSON)` | Returns a JSON object with version and build information for the Infera extension. |
16+
17+
---
1618

1719
### Usage Examples
1820

21+
This section includes some examples of how to use the Infera functions.
22+
1923
#### Model Management
2024

2125
```sql
@@ -33,7 +37,7 @@ SELECT infera_get_loaded_models();
3337
SELECT infera_get_model_info('local_model');
3438
-- Output: {"name":"local_model","input_shape":[-1,3],"output_shape":[-1,1],"loaded":true}
3539

36-
-- Unload a model
40+
-- Unload a loaded model
3741
SELECT infera_unload_model('remote_model');
3842
```
3943

@@ -48,109 +52,116 @@ SELECT id,
4852
infera_predict('my_model', feature1, feature2, feature3) as prediction
4953
FROM features_table;
5054

51-
-- Get multiple outputs as a JSON array
55+
-- Get multiple outputs as a JSON array.
56+
-- This is useful models that return multiple outputs per prediction (like a non-binary classifier)
5257
SELECT infera_predict_multi('multi_output_model', 1.0, 2.0);
5358
-- Output: [0.85, 0.12, 0.03]
54-
```
55-
56-
-----
57-
58-
### Building from Source
59-
60-
#### Prerequisites
61-
62-
- Rust toolchain
63-
- CMake 3.21+
64-
- C++ compiler (GCC/Clang)
65-
- `cbindgen` for generating C headers (`cargo install cbindgen`)
66-
67-
#### Build Steps
68-
69-
1. Build the Rust library:
70-
```bash
71-
cargo build --release --manifest-path infera/Cargo.toml
72-
```
73-
2. Generate C header bindings (if needed):
74-
```bash
75-
make rust-binding-headers
76-
```
77-
3. Build the complete extension:
78-
```bash
79-
make release
80-
```
81-
82-
-----
8359

84-
### Testing the Extension
85-
86-
After building, you can run the included SQL test suite.
60+
-- Predict using raw BLOB data (like tensor data)
61+
SELECT infera_predict_from_blob('my_model', my_blob_column)
62+
FROM my_table;
63+
-- Expected output: [0.1, 0.2, 0.3, ...] (as a LIST<FLOAT>)
64+
```
8765

88-
```bash
89-
# Run all core tests
90-
./build/release/duckdb < tests/sql/test_core_functionality.sql
66+
> [!IMPORTANT]
67+
> When you use a model model for inference, in essence it will be executed on your machine.
68+
> So make sure you download and use models from trusted sources only.
9169
92-
# Run advanced feature tests
93-
./build/release/duckdb < tests/sql/test_advanced_features.sql
70+
#### Utility Functions
9471

95-
# Run integration and error tests
96-
./build/release/duckdb < tests/sql/test_integration_and_errors.sql
72+
```sql
73+
-- Get a JSON list of all loaded models
74+
SELECT infera_get_loaded_models();
75+
-- Output: ["linear_model", "squeezenet"]
76+
77+
-- Get detailed metadata for a specific model
78+
SELECT infera_get_model_info('squeezenet');
79+
/* Output:
80+
{
81+
"name": "squeezenet",
82+
"input_shape": [1, 3, 224, 224],
83+
"output_shape": [1, 1000],
84+
"loaded": true
85+
}
86+
*/
87+
88+
-- Load all models from the 'models/' directory
89+
SELECT infera_set_autoload_dir('path/to/your/models');
90+
/* Output:
91+
{
92+
"loaded": ["model1", "model2"],
93+
"errors": []
94+
}
95+
*/
9796
```
9897

99-
-----
98+
---
10099

101-
### Development Workflow
100+
### Building Infera from Source
102101

103-
#### Adding New Rust Functions
102+
To build Infera from source, you need to have GNU Make, CMake, and a C++ compiler (like GCC or Clang) installed.
103+
You also need to have Rust (nightly) and Cargo installed via `rustup`.
104104

105-
1. Add function to Rust code (`infera/src/lib.rs`):
105+
1. **Clone the repository:**
106106

107-
```rust
108-
#[no_mangle]
109-
pub unsafe extern "C" fn my_new_function() -> *mut c_char {
110-
// ... implementation ...
111-
}
107+
```bash
108+
git clone --recursive https://github.com/CogitatorTech/infera.git
109+
cd infera
112110
```
113111

114-
2. Add the function name to the `[export]` list in `infera/cbindgen.toml`.
112+
> [!NOTE]
113+
> The `--recursive` flag is important to clone the required submodules (like DuckDB).
115114
116-
3. Update C header (regenerate bindings):
115+
2. **Install dependencies:**
117116

117+
The project includes a `Makefile` target to help set up the development environment. For Debian-based systems, you
118+
can run:
118119
```bash
119-
make rust-binding-headers
120-
```
121-
122-
4. Register in C++ extension (`bindings/infera_extension.cpp`):
123-
124-
```cpp
125-
// Add a C++ wrapper and register it in LoadInternal()
126-
ScalarFunction my_func("my_new_function", {}, LogicalType::VARCHAR, MyNewFunctionWrapper);
127-
loader.RegisterFunction(my_func);
120+
make install-deps
128121
```
122+
This will install necessary system packages, Rust tools, and Python dependencies. For other operating systems, please
123+
refer to the `Makefile` to see the list of dependencies and install them manually.
129124

130-
5. Rebuild and test:
125+
3. **Build the extension:**
131126

127+
Run the following command to build the DuckDB shell with the Infera extension included:
132128
```bash
133129
make release
134-
./build/release/duckdb -c "SELECT my_new_function();"
135130
```
131+
This will create a `duckdb` executable inside the `build/release/` directory.
136132

137-
### Build from Source
133+
4. **Run the custom DuckDB shell:**
138134

139-
You can build Infera from source by following these steps:
140-
141-
```sh
142-
# Build in release mode
143-
make release
144-
```
145-
146-
```sh
147-
# Build in debug mode
148-
make debug
149-
```
150-
151-
After building, you can run the following binaries:
152-
153-
- `./build/{release or debug}/duckdb`: this is the newest stable version of duckdb with Infera statically linked to it.
154-
- `./build/{release or debug}/test/unittest`: this is the test runner of duckdb (for `.test` files).
155-
- `./build/{release or debug}/extension/infera/infera.duckdb_extension`: this is the loadable binary that is a `.so`,
156-
`.dylib`, or `.dll` file based on your platform.
135+
You can now run the custom-built DuckDB shell:
136+
```bash
137+
./build/release/duckdb
138+
```
139+
The Infera extension will be automatically available, and you can start using the `infera_*` functions right away
140+
without needing to run the `LOAD` command.
141+
142+
> [!NOTE]
143+
> After a successful build, you can run the following binaries:
144+
> - `./build/release/duckdb`: this is the newest stable version of duckdb with Infera statically linked to it.
145+
> - `./build/release/test/unittest`: this is the test runner of duckdb (for `.test` files).
146+
> - `./build/release/extension/infera/infera.duckdb_extension`: this is the loadable binary that is a `.so`,
147+
`.dylib`, or `.dll` file based on your platform.
148+
149+
---
150+
151+
### Architecture
152+
153+
Infera is made up of two main components:
154+
155+
1. **Rust Core (`infera/src/`)**: The core logic is implemented in Rust. This component is responsible for:
156+
* Loading ONNX models from local files or remote URLs.
157+
* Caching remote models for efficient reuse.
158+
* Managing the lifecycle of loaded models.
159+
* Performing the actual model inference using the [Tract](https://github.com/sonos/tract) engine.
160+
* Exposing a C-compatible Foreign Function Interface (FFI) so that it can be called from other languages.
161+
162+
2. **C++ DuckDB Bindings (`infera/bindings/`)**: A C++ layer that connects the Rust core and DuckDB. Its
163+
responsibilities include:
164+
* Defining the custom SQL functions (like `infera_load_model` and `infera_predict`).
165+
* Translating data from DuckDB's internal vector-based format into the raw data pointers expected by the Rust FFI.
166+
* Calling the Rust functions and handling the returned results.
167+
* Integrating with DuckDB's extension loading mechanism.

docs/examples/e1_test_core_functionality.sql renamed to docs/examples/e1_core_functionality.sql

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ SELECT infera_get_loaded_models() AS initial_models;
2020
SELECT '--- Testing Local Model Roundtrip ---';
2121

2222
-- Load the simple linear model.
23-
SELECT infera_load_model('linear', 'tests/models/linear.onnx') AS loaded;
23+
SELECT infera_load_model('linear', 'test/models/linear.onnx') AS loaded;
2424

2525
-- Verify the model appears in the list.
2626
SELECT instr(infera_get_loaded_models(), 'linear') > 0 AS after_load;
@@ -49,11 +49,11 @@ SELECT infera_get_loaded_models() AS after_unload;
4949
SELECT '--- Testing Autoload Directory ---';
5050

5151
-- Create a temporary directory and copy the model into it.
52-
.shell mkdir -p tests/temp_models
53-
.shell cp tests/models/linear.onnx tests/temp_models/
52+
.shell mkdir -p test/temp_models
53+
.shell cp test/models/linear.onnx test/temp_models/
5454

5555
-- Run the autoload function.
56-
SELECT infera_set_autoload_dir('tests/temp_models');
56+
SELECT infera_set_autoload_dir('test/temp_models');
5757

5858
-- Verify the model was loaded automatically.
5959
SELECT instr(infera_get_loaded_models(), 'linear') > 0 AS autoloaded_model_is_listed;
@@ -63,7 +63,7 @@ SELECT abs(infera_predict('linear', 1.0, 2.0, 3.0) - 1.75) < 1e-5 AS autoload_pr
6363

6464
-- Clean up.
6565
SELECT infera_unload_model('linear');
66-
.shell rm -rf tests/temp_models
66+
.shell rm -rf test/temp_models
6767

6868

6969
.echo off

docs/examples/e2_test_advanced_features.sql renamed to docs/examples/e2_advanced_features.sql

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ SELECT '--- Testing Remote Model Loading ---';
99

1010
-- Define macros for the model name and URL for clarity.
1111
CREATE OR REPLACE MACRO model_name() AS 'remote_linear_model';
12-
CREATE OR REPLACE MACRO model_url() AS 'https://github.com/CogitatorTech/infera/raw/refs/heads/main/tests/models/linear.onnx';
12+
CREATE OR REPLACE MACRO model_url() AS 'https://github.com/CogitatorTech/infera/raw/refs/heads/main/test/models/linear.onnx';
1313

1414
-- Load the model from the URL.
1515
SELECT infera_load_model(model_name(), model_url()) AS loaded_from_url;

docs/examples/e3_test_integration_and_errors.sql renamed to docs/examples/e3_integration_and_errors.sql

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ SELECT infera_unload_model('nonexistent_model');
2020
SELECT '--- Testing Batch Processing and Aggregation ---';
2121

2222
-- Load a model to use for batch tests.
23-
SELECT infera_load_model('linear', 'tests/models/linear.onnx');
23+
SELECT infera_load_model('linear', 'test/models/linear.onnx');
2424

2525
-- Create a table with sample feature data, casting to FLOAT.
2626
CREATE OR REPLACE TABLE features AS

‎infera/Cargo.toml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ hex = "0.4"
2626

2727
[dev-dependencies]
2828
tempfile = "3.10"
29+
mockito = "1.7.0"
2930

3031
[profile.release]
3132
opt-level = 3

0 commit comments

Comments
 (0)