Skip to content

Add laduck: LLM inference inside DuckDB via llama.cpp#1619

Open
hamidr wants to merge 1 commit intoduckdb:mainfrom
hamidr:add-laduck-extension
Open

Add laduck: LLM inference inside DuckDB via llama.cpp#1619
hamidr wants to merge 1 commit intoduckdb:mainfrom
hamidr:add-laduck-extension

Conversation

@hamidr
Copy link
Copy Markdown

@hamidr hamidr commented Mar 27, 2026

Summary

Adds laduck, a DuckDB extension that embeds llama.cpp for local LLM inference directly from SQL.

  • Text generation: llm_complete(text, model)
  • Embeddings: llm_embed(text, model) returns float arrays
  • Zero-shot classification: llm_classify(text, model, labels) in a single forward pass
  • Model sources: Local files, HuggingFace (hf://), or DuckDB internal storage (db://)
  • Portable databases: Store GGUF models inside .duckdb files
  • GPU acceleration: Metal (macOS), CUDA, Vulkan, ROCm

Build notes

llama.cpp is included as a git submodule (third_party/llama.cpp) and statically linked. The submodule adds build time but has no runtime dependencies. WASM platforms are excluded since llama.cpp requires native execution.

Repository

https://github.com/hamidr/laduck

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants