2.2.15 Backend: Nexa SDK

Nexa SDK

Handle: nexa
URL: http://localhost:34181

Nexa SDK is an on-device inference framework from Nexa AI. It runs GGUF models locally and exposes an OpenAI-compatible API server with streaming support.

Note

Nexa AI was acquired by Qualcomm and the upstream project has been rebranded to "GenieX" (v0.3.x), which ships Snapdragon/ARM64-only builds. Harbor pins the last NexaAI-branded release with Linux x86_64 binaries — v0.2.73 — via HARBOR_NEXA_VERSION. Do not bump this to a v0.3.x tag on x86_64 hosts.

Starting

# [Optional] pre-build the image
harbor build nexa

# [Optional] Check Nexa CLI is working
harbor nexa --help

# Start the service
harbor up nexa

On startup, the container pre-pulls the configured model (HARBOR_NEXA_MODEL) and then starts nexa serve bound to the Harbor network. The OpenAI-compatible API (including /v1/models and /v1/chat/completions) is served directly — no proxy sidecar is needed.

# Check the API
curl $(harbor url nexa)/v1/models

Models

# Get/set the model the server pre-pulls at startup
harbor nexa model
harbor nexa model Qwen/Qwen3-0.6B-GGUF

Models are Hugging Face repos in GGUF format (Qwen/Qwen3-0.6B-GGUF, NexaAI/...-GGUF, etc.). A specific quant can be selected with a :tag suffix, e.g. Qwen/Qwen3-0.6B-GGUF:Q8_0; otherwise the CLI's default quant is used. The model store is cached on the host at HARBOR_NEXA_CACHE (default ~/.cache/nexa). The container drops to your host user at startup, so downloaded models stay owned by you (any root-owned leftovers from older versions are repaired automatically).

/v1/models reports pulled models with their resolved quant suffix (e.g. Qwen/Qwen3-0.6B-GGUF:Q8_0), but completions accept either the suffixed id or the bare repo name — so integrations configured with HARBOR_NEXA_MODEL (e.g. cognee) work as-is. On CPU, the first completion after startup loads the model and can take several minutes; subsequent requests are fast.

Requests must reference a pulled model by its full name:

curl $(harbor url nexa)/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "Qwen/Qwen3-0.6B-GGUF", "messages": [{"role": "user", "content": "Hello!"}]}'

Configuration

• HARBOR_NEXA_HOST_PORT — host port (default 34181)
• HARBOR_NEXA_MODEL — model pre-pulled at startup (default Qwen/Qwen3-0.6B-GGUF)
• HARBOR_NEXA_VERSION — nexa-sdk release tag the image installs (default v0.2.73, the last x86_64 Linux release)
• HARBOR_NEXA_CACHE — host location of the model store

Environment variables can also be set with harbor env nexa <key> <value>.

CLI

Running harbor nexa <args> executes the nexa CLI in an ephemeral container:

harbor nexa --help
harbor nexa list

Note that the CLI's interactive commands (e.g. nexa infer, model pulls with quant selection) require a TTY.

Integrations

• Open WebUI: when webui and nexa run together, nexa is registered as an OpenAI-compatible connection at http://nexa:8000/v1.
• cognee and optillm can use nexa as their LLM backend via the corresponding cross-service configs.