add: with_lookup pitfalls

[Dylancouzon]

Propagates the operational caveats from landing_page #2362 into the search-types skill so agents recommending with_lookup flag the silent-failure modes the worked example introduced.

What changed

skills/qdrant-search-quality/search-strategies/hybrid-search/search-types/SKILL.md:

• Added with_lookup / group_by / empty lookup / sidecar / chunk-to-document triggers to the frontmatter description.
• New "When to Split into a Sidecar Collection with Lookup in Groups" section with a four-way classification — display payload (sidecar safe), retrieval vectors (denormalize), server-side filter/scoring fields like tenant_id/ACLs/recency dates (denormalize), and mixed.
• Storage formula documents × chunks_per_document × shared_bytes_per_point with the worked PR #2362 example (~1.4 GB duplicated vs ~60 MB split) as a parenthetical anchor, not the decision rule.
• Preconditions checklist: lookup is a join by point id (not vector search); lookup collection must be populated first; group_by values must be valid point IDs (unsigned int or UUID — arbitrary strings like DOIs need a stable join key); missing/mismatched ids return empty lookup silently; with_lookup=\"documents\" shorthand returns payload only; group_by array attaches one chunk to multiple groups.
• Payload index schema now matches the field value type (keyword for strings, integer for numeric) instead of unconditional keyword.
• Two #lookup-in-groups anchors → canonical ?s=lookup-in-groups.

skills/qdrant-search-quality/search-strategies/SKILL.md:

• One bubble-up trigger phrase in the description so chunk-to-document grouping queries reach this sub-tree.

evals/evals.json + scripts/run_eval.py:

• New eval id 7 (chunk/document with_lookup scenario, 7 expectations covering the silent-failure modes above) wired into SKILL_MAP against the search-quality chain.

Why

PR #2362 added the worked two-collection example because users were silently hitting empty lookup results, recall regressions from moving retrieval-relevant vectors behind the lookup, and filter behavior that broke under sidecar splits. The docs now teach the pattern; the skill should encode the failure-mode judgment so agents recommend the split safely.

Test plan

• Verified locally: `python3 scripts/validate_skills.py` — 0 errors.
• Not run locally: link check (lychee) and eval id 7 (`scripts/run_eval.py` needs the API key — runs in CI on merge to main per `eval-skills.yml`).
• Behavioral claims in the skill (point-ID type constraint, id-type silent failure, shorthand-returns-payload-only, etc.) are taken from landing_page PR #2362's worked example and snippets, not from a live Qdrant run.

Stacking

Stacks on #55 (link-multi-representation-search-tutorial). PR base is set accordingly; merge after #55 lands.

[szabosteve]

@Dylancouzon This is my first review in this repo and I'm still figuring this out, so please point out if I'm incorrect anywhere or made a wrong assumption. Also sorry for any inconvenience.

I left a few – mostly trivial – suggestions, please take or leave them. My only real concern is about the size of this skill. The new section adds a significant amount of content. Would it be worth splitting it into a sibling leaf, something like with-lookup-pitfalls/SKILL.md and linking that from the parent skill?

[Dylancouzon]

@szabosteve Thanks for the review and feedback. No inconvenience at all, this is exactly the kind of question worth raising.

Quick context on how I work in this repo: when I open a PR against landing_page, I make the architectural calls myself and mirror them here. The skills repo is different. Claude Code is the target audience, so I let it take the lead: it runs evals, tests changes, and makes the first call. The reasoning below is mostly its judgment, not just mine.

On the split: we landed on keeping it as one leaf. The with_lookup/sidecar content is the second half of the same decision as the multi-representation grouping section above it. “Group chunks back to documents” and “should document-level data live in a sidecar” are really one conversation, and an agent needs both to answer correctly. Splitting risks loading one without the other and giving incomplete advice.

We’re also already four levels deep, so adding a fifth sibling creates another routing decision and a near-duplicate description competing for the same queries. For this repo, that’s a net negative because we optimize more for discoverability than readability.

One caveat: the current evals don’t test discoverability. run_eval.py pre-loads the skill files, so it measures answer quality after a skill is loaded, not whether an agent finds it. A routing eval where skills are exposed but not pre-loaded, and we assert which one gets invoked, would let us settle this empirically.

[szabosteve]

Thanks for the changes and for the clear explanation! LGTM!

[generall]

should not use absolute link to search. subdomain. This triggers agent's built-in protectins about using unknown domains

[Dylancouzon]

Done, though flagging that this conflicts with our guidelines in CONTRIBUTING.md and AGENTS.md, both currently mandate search.qdrant.tech/md/documentation/.

[generall]

maybe https://skills.qdrant.tech/md/documentation/search/search?s=grouping-api better?

[Dylancouzon]

Done

[generall]

it feels unrelated to hybrid seach. Maybe a separate skill?

[generall]

Also keeping skills smaller seems more beneficial

[Dylancouzon]

Pulled grouping + sidecar/with_lookup into its own leaf, search-strategies/document-grouping/, as a sibling to hybrid-search/.

Eval id 7 now targets the new leaf.

[generall]

see comments

[github-actions]

this PR has been inactive for 14 days. it will be closed in 7 days unless there is new activity. if you're still working on it, push an update or leave a comment.

The base branch was changed.

[github-actions]

this PR has been inactive for 14 days. it will be closed in 7 days unless there is new activity. if you're still working on it, push an update or leave a comment.