feat: add WFGY Problem Map for debugging scientific LLM tools #8
Description
Hi, thanks for maintaining this great list of AI tools for science.
I would like to propose adding WFGY Problem Map, an open source toolkit for debugging and hardening LLM-based assistants and RAG systems, which many scientific tools now rely on.
Repo: https://github.com/onestardao/WFGY
What WFGY provides
WFGY is centered on a 16-question checklist that captures common failure patterns in LLM systems:
- retrieval and chunking failures on domain-specific corpora (papers, lab notes, code)
- vector database and index skew after updates
- prompt / tool contract mismatches in complex workflows
- deployment issues that make a model silently “forget” data or tools
The checklist is paired with concrete fix recipes, so teams can move from “the model is acting weird” to “we know which failure mode this is, and how to fix it”.
Why it helps scientific AI tools
Many tools in this repo wrap LLMs to:
- answer questions over scientific literature,
- assist with data analysis and coding,
- orchestrate multi-step research workflows.
In those settings, silent failures or hallucinations are especially dangerous. WFGY is used as a provider-agnostic diagnostic layer: you run the checklist on failing examples, then repair the pipeline without touching model weights.
It could fit under a section for frameworks / infra / evaluation, for example:
WFGY Problem Map – 16-question checklist and toolkit for debugging RAG / LLM pipelines used in scientific tools (retrieval, vector stores, prompts, deployment).
If this is aligned with the scope of the list, I am happy to adjust the description and category.