[BUG] Ollama batch embedding fails after PR #621 (array input rejected)

[jlin53882]

Bug Description (Detailed Analysis)

After PR #621 fixed Ollama 0.20.5+ single embedding, batch embedding now fails because /api/embeddings only accepts a single string prompt field, not an array.

Error

Error: Failed to generate batch embeddings from Ollama: Ollama embedding failed: 400 Bad Request ??{"error":"json: cannot unmarshal array into Go struct field EmbeddingRequest.prompt of type string"}

Root Cause Analysis

PR #621 Background

PR #621 (fixing Issue #620) discovered that Ollama 0.20.5+ returns empty arrays when using /v1/embeddings endpoint. The fix switched to using Ollama's native /api/embeddings endpoint with the prompt field instead:

// PR #621 fix in embedWithNativeFetch()
const ollamaPayload = {
  model: payload.model,
  prompt: payload.input,  // <-- This works for single string
};

The Regression

The problem is that payload.input is not always a string. When embedMany() builds a batch payload, it passes an array:

// In embedMany()
const response = await this.embedWithRetry(
  this.buildPayload(validTexts, task),  // <-- validTexts is string[]
  signal,
);

// In buildPayload()
const payload = {
  model: this.model,
  input,  // <-- When input is string[], this becomes input: ["a", "b", "c"]
};

So when embedWithNativeFetch() receives the payload and sets prompt: payload.input, it sends an array where Ollama expects a string:

// What Ollama receives:
{
  "model": "jina-v5-retrieval-test",
  "prompt": ["a", "b", "c"]  // <-- Array, but Ollama expects string!
}

// Ollama error:
"json: cannot unmarshal array into Go struct field EmbeddingRequest.prompt of type string"

Why Single Works But Batch Fails

Scenario	payload.input type	Works?
embedPassage("hello")	string	✅
embedQuery("hello")	string	✅
embedBatchPassage(["a","b"])	string[]	❌

Suggested Fix

This is a routing fix based on verified behavior, not a universal solution.

• Single input: keep /api/embeddings + prompt (PR fix(embedder): use /api/embeddings + prompt for Ollama 0.20.5+ #621 unchanged)
• Batch input: route to /v1/embeddings + input[]
• Response validation: verify count and non-empty embeddings

Limitation: This assumes /v1/embeddings batch mode is valid for the target Ollama/model combination, which was verified in local testing with jina-v5-retrieval-test.

Testing

Local testing confirms both single and batch work correctly:

Test	Endpoint	Input	Result
Single	/api/embeddings	prompt: string	✅ SUCCESS (1024 dims)
Batch 3 items	/v1/embeddings	input: string[]	✅ SUCCESS (3 embeddings)
Batch 1 item	/v1/embeddings	input: string[]	✅ SUCCESS

Model: jina-v5-retrieval-test

Related

• [BUG] Ollama 0.20.5+ /v1/embeddings 回傳空陣列導致 recall 失敗 #620 (fixed by fix(embedder): use /api/embeddings + prompt for Ollama 0.20.5+ #621, but introduced this regression)
• PR fix(embedder): use /api/embeddings + prompt for Ollama 0.20.5+ #621

[jlin53882]

Test Coverage

Added test/embedder-ollama-batch-routing.test.mjs with 5 test cases:

Test Cases

#	Test	Validates
1	single requests use /api/embeddings with prompt field	Single embedding calls /api/embeddings with prompt: string (not input: string[])
2	batch requests use /v1/embeddings with input array	Batch calls /v1/embeddings with input: string[]
3	batch rejects response with wrong number of embeddings	Response validation: rejects when data.length !== input.length
4	batch rejects response with empty embedding array	Response validation: rejects when any embedding.length === 0
5	single-element batch still routes to /v1/embeddings	Routing based on Array.isArray(payload.input), not length

Test Execution

✔ single requests use /api/embeddings with prompt field (40.7992ms)
✔ batch requests use /v1/embeddings with input array (8.7916ms)
✔ batch rejects response with wrong number of embeddings (6.7713ms)
✔ batch rejects response with empty embedding array (5.7668ms)
✔ single-element batch still routes to /v1/embeddings (6.2722ms)
ℹ tests 5, pass 5, fail 0

Mock Server Design

• Uses fixed port 11434 so isOllamaProvider() returns true → native fetch path
• Each test creates its own server and closes it in finally block
• node:test runs tests sequentially by default, avoiding port conflicts
• Unexpected endpoints return 404 "unexpected endpoint" (not timeout)

Test Results

All 5 tests pass, confirming:

1. Single/batch routing is correct
2. Response validation catches count mismatch
3. Response validation catches empty embeddings
4. Single-element arrays route to batch path

[jlin53882]

Test Code

test/embedder-ollama-batch-routing.test.mjs:

import assert from "node:assert/strict";
import http from "node:http";
import { test } from "node:test";

import jitiFactory from "jiti";

const jiti = jitiFactory(import.meta.url, { interopDefault: true });
const { Embedder } = jiti("../src/embedder.ts");

const DIMS = 1024;
const OLLAMA_PORT = 11434;
const OLLAMA_BASE = `http://127.0.0.1:${OLLAMA_PORT}/v1`;

function readJson(req) {
  return new Promise((resolve, reject) => {
    let body = "";
    req.on("data", (chunk) => { body += chunk; });
    req.on("end", () => resolve(JSON.parse(body)));
    req.on("error", reject);
  });
}

function makeOllamaMock(handler) {
  return http.createServer((req, res) => {
    if (req.method === "POST" && req.url === "/api/embeddings") {
      handler(req, res, "api");
      return;
    }
    if (req.method === "POST" && req.url === "/v1/embeddings") {
      handler(req, res, "v1");
      return;
    }
    res.writeHead(404, { "Content-Type": "text/plain" });
    res.end("unexpected endpoint");
  });
}

function dims() {
  return Array.from({ length: DIMS }, () => Math.random() * 0.1);
}

test("single requests use /api/embeddings with prompt field", async () => {
  let capturedBody = null;

  const server = makeOllamaMock(async (req, res, route) => {
    capturedBody = await readJson(req);
    res.writeHead(200, { "Content-Type": "application/json" });
    res.end(JSON.stringify({ embedding: dims() }));
  });

  await new Promise((resolve) => server.listen(OLLAMA_PORT, "127.0.0.1", resolve));

  try {
    const embedder = new Embedder({
      provider: "openai-compatible",
      apiKey: "test-key",
      model: "mxbai-embed-large",
      baseURL: OLLAMA_BASE,
      dimensions: DIMS,
    });

    const result = await embedder.embedPassage("hello world");

    assert.equal(capturedBody?.model, "mxbai-embed-large");
    assert.equal(capturedBody?.prompt, "hello world");
    assert.equal(Array.isArray(capturedBody?.prompt), false, "prompt should be a string, not array");
    assert.equal(result.length, DIMS);
  } finally {
    await new Promise((resolve) => server.close(resolve));
  }
});

test("batch requests use /v1/embeddings with input array", async () => {
  let capturedBody = null;

  const server = makeOllamaMock(async (req, res, route) => {
    capturedBody = await readJson(req);
    const embeddings = capturedBody.input.map((_, i) => ({
      embedding: dims(),
      index: i,
    }));
    res.writeHead(200, { "Content-Type": "application/json" });
    res.end(JSON.stringify({ data: embeddings }));
  });

  await new Promise((resolve) => server.listen(OLLAMA_PORT, "127.0.0.1", resolve));

  try {
    const embedder = new Embedder({
      provider: "openai-compatible",
      apiKey: "test-key",
      model: "mxbai-embed-large",
      baseURL: OLLAMA_BASE,
      dimensions: DIMS,
    });

    const inputs = ["a", "b", "c"];
    const result = await embedder.embedBatchPassage(inputs);

    assert.equal(capturedBody?.model, "mxbai-embed-large");
    assert.deepEqual(capturedBody?.input, inputs);
    assert.equal(Array.isArray(capturedBody?.input), true, "input should be an array");
    assert.equal(result.length, 3);
    result.forEach((emb) => assert.equal(emb.length, DIMS));
  } finally {
    await new Promise((resolve) => server.close(resolve));
  }
});

test("batch rejects response with wrong number of embeddings", async () => {
  const server = makeOllamaMock(async (req, res, route) => {
    if (route !== "v1") {
      res.writeHead(404);
      res.end("unexpected route");
      return;
    }
    const body = await readJson(req);
    const embeddings = Array.from({ length: Math.max(1, body.input.length - 1) }, (_, i) => ({
      embedding: dims(),
      index: i,
    }));
    res.writeHead(200, { "Content-Type": "application/json" });
    res.end(JSON.stringify({ data: embeddings }));
  });

  await new Promise((resolve) => server.listen(OLLAMA_PORT, "127.0.0.1", resolve));

  try {
    const embedder = new Embedder({
      provider: "openai-compatible",
      apiKey: "test-key",
      model: "mxbai-embed-large",
      baseURL: OLLAMA_BASE,
      dimensions: DIMS,
    });

    const inputs = ["a", "b", "c"];
    await assert.rejects(
      async () => embedder.embedBatchPassage(inputs),
      (err) => {
        assert.ok(
          /unexpected result count|invalid response/i.test(err.message),
          `Expected count validation error, got: ${err.message}`,
        );
        return true;
      },
    );
  } finally {
    await new Promise((resolve) => server.close(resolve));
  }
});

test("batch rejects response with empty embedding array", async () => {
  const server = makeOllamaMock(async (req, res, route) => {
    if (route !== "v1") {
      res.writeHead(404);
      res.end("unexpected route");
      return;
    }
    const body = await readJson(req);
    const embeddings = body.input.map((_, i) => ({
      embedding: i === 1 ? [] : dims(),
      index: i,
    }));
    res.writeHead(200, { "Content-Type": "application/json" });
    res.end(JSON.stringify({ data: embeddings }));
  });

  await new Promise((resolve) => server.listen(OLLAMA_PORT, "127.0.0.1", resolve));

  try {
    const embedder = new Embedder({
      provider: "openai-compatible",
      apiKey: "test-key",
      model: "mxbai-embed-large",
      baseURL: OLLAMA_BASE,
      dimensions: DIMS,
    });

    const inputs = ["a", "b", "c"];
    await assert.rejects(
      async () => embedder.embedBatchPassage(inputs),
      (err) => {
        assert.ok(
          /invalid response/i.test(err.message),
          `Expected invalid response error, got: ${err.message}`,
        );
        return true;
      },
    );
  } finally {
    await new Promise((resolve) => server.close(resolve));
  }
});

test("single-element batch still routes to /v1/embeddings", async () => {
  let capturedRoute = null;

  const server = makeOllamaMock(async (req, res, route) => {
    capturedRoute = route;
    res.writeHead(200, { "Content-Type": "application/json" });
    res.end(JSON.stringify({ data: [{ embedding: dims(), index: 0 }] }));
  });

  await new Promise((resolve) => server.listen(OLLAMA_PORT, "127.0.0.1", resolve));

  try {
    const embedder = new Embedder({
      provider: "openai-compatible",
      apiKey: "test-key",
      model: "mxbai-embed-large",
      baseURL: OLLAMA_BASE,
      dimensions: DIMS,
    });

    const result = await embedder.embedBatchPassage(["only-one"]);

    assert.equal(capturedRoute, "v1", "single-element batch should use /v1/embeddings");
    assert.equal(result.length, 1);
    assert.equal(result[0].length, DIMS);
  } finally {
    await new Promise((resolve) => server.close(resolve));
  }
});

[jlin53882]

Proposed Fix (Updated)

Here's the final proposed fix for src/embedder.ts:

/**
 * Call embeddings.create using native fetch (bypasses OpenAI SDK).
 * Used exclusively for Ollama endpoints where AbortController must work
 * correctly to avoid long-lived stalled sockets.
 *
 * For Ollama 0.20.5+: /v1/embeddings may return empty arrays for some models,
 * so we use /api/embeddings with "prompt" field for single requests (PR #621).
 * For batch requests, we use /v1/embeddings with "input" array as it's more
 * efficient and confirmed working in local testing.
 *
 * See: https://github.com/CortexReach/memory-lancedb-pro/issues/620
 * Fix: https://github.com/CortexReach/memory-lancedb-pro/issues/629
 */
private async embedWithNativeFetch(payload: any, signal?: AbortSignal): Promise<any> {
  if (!this._baseURL) {
    throw new Error("embedWithNativeFetch requires a baseURL");
  }

  const base = this._baseURL.replace(/\/$/, "").replace(/\/v1$/, "");
  const apiKey = this.clients[0]?.apiKey ?? "ollama";

  // Handle batch requests with /v1/embeddings + input array
  if (Array.isArray(payload.input)) {
    const response = await fetch(base + "/v1/embeddings", {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        "Authorization": `Bearer ${apiKey}`,
      },
      body: JSON.stringify({
        model: payload.model,
        input: payload.input,
      }),
      signal,
    });

    if (!response.ok) {
      const body = await response.text().catch(() => "");
      throw new Error(
        `Ollama batch embedding failed: ${response.status} ${response.statusText} ??${body.slice(0, 200)}`
      );
    }

    const data = await response.json();

    // Validate response count and non-empty embeddings
    if (
      !Array.isArray(data?.data) ||
      data.data.length !== payload.input.length ||
      data.data.some((item: any) => {
        const embedding = item?.embedding;
        return !Array.isArray(embedding) || embedding.length === 0;
      })
    ) {
      throw new Error(
        `Ollama batch embedding returned invalid response for ${payload.input.length} inputs`
      );
    }

    return data;
  }

  // Single request: use /api/embeddings + prompt (PR #621 fix)
  const response = await fetch(base + "/api/embeddings", {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "Authorization": `Bearer ${apiKey}`,
    },
    body: JSON.stringify({
      model: payload.model,
      prompt: payload.input,
    }),
    signal,
  });

  if (!response.ok) {
    const body = await response.text().catch(() => "");
    throw new Error(
      `Ollama embedding failed: ${response.status} ${response.statusText} ??${body.slice(0, 200)}`
    );
  }

  const data = await response.json();

  // Ollama /api/embeddings returns { embedding: number[] },
  // convert to OpenAI-compatible shape { data: [{ embedding: number[] }] }
  return { data: [{ embedding: data.embedding }] };
}

Summary

• PR fix(embedder): use /api/embeddings + prompt for Ollama 0.20.5+ #621 single fix preserved: /api/embeddings + prompt
• Batch route: /v1/embeddings + input[]
• Response validation: count check + non-empty embedding check (improved to avoid null/undefined edge cases)