Suggestion: add WFGY 16 Problem Map as a RAG failure-mode debugging resource #60
Description
Hi, thanks a lot for curating this list. It is super helpful for people building on Azure OpenAI with RAG and agents.
I wanted to ask if a RAG debugging / failure-mode taxonomy like WFGY Problem Map would be in scope for this repo.
What WFGY is
WFGY is an open source project on GitHub that maintains a 16-mode failure map for LLM + RAG systems (No.1–No.16). Each item is a concrete failure pattern such as:
- retrieval hallucination even when the right chunks are present
- vector store ingestion / bootstrap ordering issues
- config / secrets drift around first deploy
- prompt / tool contract mismatch, etc.
The Problem Map lives here:
The idea is to give RAG builders a common vocabulary for “what went wrong” and a checklist they can run through before blaming the model or the vendor.
Why it fits this repo
This repo already has great material on:
- building RAG and agents on Azure OpenAI
- patterns, tutorials and hands-on code
One thing many users struggle with is systematic debugging once the RAG pipeline starts to behave strangely. A link to WFGY Problem Map in a “Debugging / Failure modes / Best practices” section could help users map their bugs to a specific failure mode instead of guessing.
Concrete suggestion
If you think this is in scope, a simple entry like the following could work:
- WFGY 16 Problem Map – RAG failure-mode taxonomy
Open source 16-mode map of common LLM + RAG failures (No.1–No.16) with practical debugging checklists.
https://github.com/onestardao/WFGY/tree/main/ProblemMap#readme
If this does not match the focus of the list, no worries at all. I mainly wanted to check before sending a PR. Thanks again for maintaining this resource.