I'm hitting the same issue on a Mac Mini M1 with 8GB RAM.

Root cause: Both vsearch and query call expandQueryStructured() which requires the 1.28GB qmd-query-expansion-1.7B GGUF model. On 8GB machines with other processes running, the download gets OOM-killed before completing — and even if it downloaded, loading it for inference alongside everything else would be tight.

The 300MB embedding model (embeddinggemma-300M) downloads and runs fine. It's specifically the query expansion step that's the blocker.

This is related to #147 (GPU OOM with parallel vsearch/query) and #155 (CUDA OOM during embed) — same family of resource exhaustion issues where the LLM models exceed available memory. The common thread is that QMD's search pipeline assumes enough RAM/VRAM for multiple large models, which isn't always the case.

Proposed fix: A --no-expand flag on vsearch (and optionally query) that skips expandQueryStructured() and searches with the raw query only. ~5 line change in vectorSearch():

const vectorQueries: string[] = [query];
if (!opts.noExpand) {
  const queryables = await expandQueryStructured(query, false, opts.context, session);
  for (const q of queryables) {
    if (q.type === "vec" || q.type === "hyde") {
      if (q.text && q.text !== query) vectorQueries.push(q.text);
    }
  }
}

I patched this locally and raw vector search returns relevant results with good similarity scores (57-63%). Query expansion improves recall but isn't essential — having a working vsearch without it is much better than a broken one with it.

Env: Mac Mini M1, 8GB RAM, macOS Sonoma, Bun 1.3.8

qmd search and qmd vsearch both can work but qmd query always be auto killed. #158

Description

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions