🤖 Add AI/LLM Tools Section — New Chapter Proposal
What this PR does
Adds a new top-level chapter 🤖 AI/LLM Tools to the README, including TOC entry and 9 subcategories with 70+ curated tools and resources.
The Book of Secret Knowledge has zero coverage of the AI/LLM tooling ecosystem, which has become a core part of every sysadmin, DevOps engineer, security researcher, and developer's daily workflow. This fills that gap in the same format and spirit as the rest of the book.
All entries follow the existing format: [**Tool Name**](url) — short description.
All links verified as of March 2026. No existing content was modified.
Proposed Chapter: 🤖 AI/LLM Tools
To be inserted into the Table of Contents and as a new section in the README, following the existing formatting conventions.
▪️ CLI-Based LLM Interfaces
Ollama — run large language models locally on your machine via a simple CLI and REST API. Supports Llama 3, Mistral, Gemma, Phi, and many more.
llm — a CLI utility and Python library by Simon Willison for interacting with LLMs (OpenAI, Anthropic, local models via plugins) directly from your terminal.
aichat — all-in-one AI CLI tool featuring chat, shell assist, RAG, AI tools/agents, and multi-provider support (OpenAI, Claude, Gemini, Ollama, and more).
shell_gpt — a command-line productivity tool powered by AI models, designed to generate shell commands, code snippets, and natural language responses.
gpt-cli — a simple, powerful command-line interface for ChatGPT, Claude, and other LLMs with multi-turn conversation history support.
gorilla-cli — LLM-powered CLI assistant that suggests and executes CLI commands from natural language queries.
jan — open source ChatGPT alternative that runs fully offline on your computer; supports multiple local and cloud-based models.
tgpt — interact with AI chatbots in the terminal without needing API keys, using freely available backends.
fabric — an open source framework for augmenting humans with AI via a set of modular, reusable "patterns" (prompts) that can be piped through the CLI.
▪️ AI-Powered Terminal & IDE Assistants
GitHub Copilot CLI — AI assistant for your terminal that explains commands, translates natural language to shell/git/gh commands, and answers CLI questions.
Warp AI — a modern terminal (Rust-based) with built-in AI that can explain errors, generate commands, and assist with debugging in context.
Aider — AI pair programming tool in your terminal; connects to your local git repo and lets you code with Claude, GPT-4, or other models from the command line.
Claude Code — Anthropic's official agentic CLI coding assistant that understands entire codebases, writes and edits files, runs tests, and interacts with git.
Cursor — AI-first code editor (VS Code fork) with deeply integrated LLM capabilities for code generation, editing, and codebase Q&A.
Continue — open source AI code assistant for VS Code and JetBrains; supports any LLM, local or cloud-based.
Codeium — free AI code completion and chat tool that integrates with 40+ editors.
Zed — high-performance collaborative code editor with native AI assistant integration.
▪️ Local Model Runners & Management
LM Studio — discover, download, and run local LLMs (GGUF format) on your laptop with a GUI and a built-in local OpenAI-compatible server.
llama.cpp — efficient inference of Meta's LLaMA model (and many others) in pure C/C++; runs on CPU, Apple Silicon, CUDA, and more.
GPT4All — open source ecosystem for running powerful, private LLMs locally on consumer hardware, no GPU required.
koboldcpp — easy-to-use local AI text generation software based on llama.cpp with an accessible API and WebUI.
text-generation-webui — Gradio-based web UI for running large language models locally; supports many model formats and backends.
LocalAI — free, open source OpenAI API-compatible alternative that runs locally; supports text generation, image generation, TTS, STT, and more.
Msty — local and online AI model management app; designed for simple multi-model chat and comparison.
vllm — high-throughput and memory-efficient LLM inference engine, optimized for serving models at scale.
▪️ AI Agents & Automation Frameworks
AutoGPT — one of the first autonomous AI agent frameworks; gives GPT-4 the ability to browse the web, write files, and execute code to achieve long-horizon goals.
CrewAI — framework for orchestrating role-playing, autonomous AI agents that collaborate to complete complex tasks.
LangChain — framework for building context-aware, reasoning LLM applications; supports chains, agents, RAG, tools, and memory.
LlamaIndex — data framework for building LLM applications over custom data via indexing, retrieval, and query engines.
Microsoft AutoGen — framework for building multi-agent conversational AI systems where agents collaborate to solve tasks.
Flowise — drag-and-drop UI to build customized LLM orchestration flows and AI agents, powered by LangChain.
Dify — open source LLM application development platform; supports RAG, agent workflows, model management, and observability.
TaskWeaver — code-first agent framework by Microsoft for planning and executing data analytics tasks using LLMs.
▪️ Retrieval-Augmented Generation (RAG) & Vector Databases
Chroma — open source AI-native embedding database designed to make building LLM apps with context easy.
Qdrant — high-performance, production-ready vector database and similarity search engine.
Weaviate — open source vector database that stores both objects and vectors for semantic search and RAG.
pgvector — open source vector similarity search extension for PostgreSQL.
Milvus — cloud-native, open source vector database designed for scalable similarity search.
AnythingLLM — full-stack application to turn any document into a chatbot using RAG; supports local and cloud LLMs, multi-user, no-code setup.
privateGPT — interact privately with your documents using the power of LLMs without an internet connection; 100% private.
Verba — open source RAG chatbot powered by Weaviate; ingest documents and query them with LLMs.
▪️ Prompt Engineering Resources & Tools
Awesome Prompt Engineering — a curated list of resources for prompt engineering: papers, guides, tools, datasets, and examples.
Anthropic Prompt Engineering Docs — official, comprehensive guide to prompting Claude models effectively.
OpenAI Prompt Engineering Guide — official strategies and tactics for getting better results from GPT models.
Prompt Engineering Guide — a comprehensive guide covering techniques like zero-shot, few-shot, chain-of-thought, and more.
PromptPerfect — online tool to automatically optimize your prompts for various LLMs.
DSPy — Stanford's framework for algorithmically optimizing LLM prompts and weights rather than hand-crafting them.
Promptfoo — open source tool for testing, evaluating, and red-teaming LLM prompts and applications.
LMSYS Chatbot Arena — open platform for benchmarking LLMs side-by-side using human preference votes.
▪️ AI Security: Red Teaming, Prompt Injection & Jailbreaking Research
Garak — open source LLM vulnerability scanner by NVIDIA; probes models for hallucination, data leakage, prompt injection, toxicity, and more.
PyRIT — Python Risk Identification Toolkit for Generative AI by Microsoft; automates red teaming of LLM-based systems.
promptmap — automatically tests prompt injection attacks on ChatGPT instances.
PromptBench — unified library for evaluating and understanding the robustness of LLMs against adversarial prompts.
Pliny the Prompter — well-known community resource cataloging jailbreak prompts and bypasses for research and red-teaming.
AI Village CTF Resources — community hub for AI security research, CTF challenges, and adversarial ML papers.
LLM Security — curated list of papers, blog posts, and research on LLM security vulnerabilities including prompt injection, data poisoning, and model inversion.
Rebuff — open source prompt injection detection API designed to protect LLM-powered applications.
LLM Guard — security toolkit for LLM interactions; detects and sanitizes harmful inputs and outputs in production.
Vigil — open source LLM prompt injection and security scanner; detects jailbreaks, prompt leaking, and other adversarial inputs.
▪️ AI Observability, Evaluation & MLOps
Langfuse — open source LLM engineering platform for tracing, evaluation, prompt management, and metrics.
Phoenix — open source AI observability platform by Arize for LLM traces, evaluations, and dataset management.
Weights & Biases (W&B) — MLOps platform for tracking experiments, visualizing metrics, and managing LLM fine-tuning runs.
MLflow — open source platform for managing the ML lifecycle, including experiment tracking, model registry, and serving.
Helicone — open source LLM observability platform; logs, monitors, and analyzes every LLM API call.
Ragas — framework for evaluating RAG pipelines using metrics like faithfulness, answer relevancy, and context precision.
DeepEval — open source LLM evaluation framework; unit-test your LLM outputs with metrics like hallucination, toxicity, and bias.
▪️ AI-Powered Security & Sysadmin Tools
Nuclei AI — the widely-used vulnerability scanner now supports AI-generated templates for faster coverage of new CVEs.
Burp AI extensions — Burp Suite extensions that use LLMs to analyze HTTP traffic, suggest attack vectors, and explain vulnerabilities.
PentestGPT — GPT-powered penetration testing assistant that guides security professionals through testing methodology.
Vulnhuntr — uses Claude AI to autonomously find zero-day vulnerabilities in Python codebases via static analysis.
AISIRT / Incident Response AI — AI-assisted SIEM playbooks and triage tools; LLM-powered alert summarization and log analysis.
SWE-agent — AI agent that autonomously fixes bugs and resolves GitHub issues by interacting with a terminal like a human developer.
▪️ Inspiring Lists & Further Reading
Awesome LLM — curated list of large language model papers, frameworks, tools, and resources.
Awesome AI Agents — comprehensive list of AI autonomous agent projects and frameworks.
Awesome LLM Security — curated resources on LLM security, adversarial attacks, and defenses.
Awesome Ollama — community-curated tools, integrations, and resources for the Ollama ecosystem.
LLM Leaderboard (Open LLM) — Hugging Face's open benchmarking leaderboard for comparing local/open-source LLMs.
Simon Willison's Blog — one of the best blogs tracking the LLM tooling landscape, with practical guides and weekly AI news updates.
The Batch by deeplearning.ai — Andrew Ng's weekly newsletter covering the most important AI developments for practitioners.
AI Canon by a16z — curated reading list of the most important AI/ML papers, blog posts, and courses.
Why This Matters
The tools above are no longer "optional extras" — they are rapidly becoming core infrastructure for:
- Sysadmins: AI log analysis, config generation, runbook automation
- DevOps/SRE: AI-assisted incident triage, automated root cause analysis
- Pentesters/Security Researchers: LLM-powered fuzzing, vulnerability research, report generation
- Developers: Agentic coding assistants that write, test, and deploy code
The book covers vim, tmux, nmap, and Wireshark — all timeless, essential tools. Ollama, Claude Code, Aider, and Garak are their 2024–2026 equivalents.
Checklist
Thank you for maintaining this repository — it's an incredible resource. 🙏
🤖 Add AI/LLM Tools Section — New Chapter Proposal
What this PR does
Adds a new top-level chapter
🤖 AI/LLM Toolsto the README, including TOC entry and 9 subcategories with 70+ curated tools and resources.The Book of Secret Knowledge has zero coverage of the AI/LLM tooling ecosystem, which has become a core part of every sysadmin, DevOps engineer, security researcher, and developer's daily workflow. This fills that gap in the same format and spirit as the rest of the book.
All entries follow the existing format:
[**Tool Name**](url) — short description.All links verified as of March 2026. No existing content was modified.
Proposed Chapter:
🤖 AI/LLM Tools▪️ CLI-Based LLM Interfaces
Ollama — run large language models locally on your machine via a simple CLI and REST API. Supports Llama 3, Mistral, Gemma, Phi, and many more.
llm — a CLI utility and Python library by Simon Willison for interacting with LLMs (OpenAI, Anthropic, local models via plugins) directly from your terminal.
aichat — all-in-one AI CLI tool featuring chat, shell assist, RAG, AI tools/agents, and multi-provider support (OpenAI, Claude, Gemini, Ollama, and more).
shell_gpt — a command-line productivity tool powered by AI models, designed to generate shell commands, code snippets, and natural language responses.
gpt-cli — a simple, powerful command-line interface for ChatGPT, Claude, and other LLMs with multi-turn conversation history support.
gorilla-cli — LLM-powered CLI assistant that suggests and executes CLI commands from natural language queries.
jan — open source ChatGPT alternative that runs fully offline on your computer; supports multiple local and cloud-based models.
tgpt — interact with AI chatbots in the terminal without needing API keys, using freely available backends.
fabric — an open source framework for augmenting humans with AI via a set of modular, reusable "patterns" (prompts) that can be piped through the CLI.
▪️ AI-Powered Terminal & IDE Assistants
GitHub Copilot CLI — AI assistant for your terminal that explains commands, translates natural language to shell/git/gh commands, and answers CLI questions.
Warp AI — a modern terminal (Rust-based) with built-in AI that can explain errors, generate commands, and assist with debugging in context.
Aider — AI pair programming tool in your terminal; connects to your local git repo and lets you code with Claude, GPT-4, or other models from the command line.
Claude Code — Anthropic's official agentic CLI coding assistant that understands entire codebases, writes and edits files, runs tests, and interacts with git.
Cursor — AI-first code editor (VS Code fork) with deeply integrated LLM capabilities for code generation, editing, and codebase Q&A.
Continue — open source AI code assistant for VS Code and JetBrains; supports any LLM, local or cloud-based.
Codeium — free AI code completion and chat tool that integrates with 40+ editors.
Zed — high-performance collaborative code editor with native AI assistant integration.
▪️ Local Model Runners & Management
LM Studio — discover, download, and run local LLMs (GGUF format) on your laptop with a GUI and a built-in local OpenAI-compatible server.
llama.cpp — efficient inference of Meta's LLaMA model (and many others) in pure C/C++; runs on CPU, Apple Silicon, CUDA, and more.
GPT4All — open source ecosystem for running powerful, private LLMs locally on consumer hardware, no GPU required.
koboldcpp — easy-to-use local AI text generation software based on llama.cpp with an accessible API and WebUI.
text-generation-webui — Gradio-based web UI for running large language models locally; supports many model formats and backends.
LocalAI — free, open source OpenAI API-compatible alternative that runs locally; supports text generation, image generation, TTS, STT, and more.
Msty — local and online AI model management app; designed for simple multi-model chat and comparison.
vllm — high-throughput and memory-efficient LLM inference engine, optimized for serving models at scale.
▪️ AI Agents & Automation Frameworks
AutoGPT — one of the first autonomous AI agent frameworks; gives GPT-4 the ability to browse the web, write files, and execute code to achieve long-horizon goals.
CrewAI — framework for orchestrating role-playing, autonomous AI agents that collaborate to complete complex tasks.
LangChain — framework for building context-aware, reasoning LLM applications; supports chains, agents, RAG, tools, and memory.
LlamaIndex — data framework for building LLM applications over custom data via indexing, retrieval, and query engines.
Microsoft AutoGen — framework for building multi-agent conversational AI systems where agents collaborate to solve tasks.
Flowise — drag-and-drop UI to build customized LLM orchestration flows and AI agents, powered by LangChain.
Dify — open source LLM application development platform; supports RAG, agent workflows, model management, and observability.
TaskWeaver — code-first agent framework by Microsoft for planning and executing data analytics tasks using LLMs.
▪️ Retrieval-Augmented Generation (RAG) & Vector Databases
Chroma — open source AI-native embedding database designed to make building LLM apps with context easy.
Qdrant — high-performance, production-ready vector database and similarity search engine.
Weaviate — open source vector database that stores both objects and vectors for semantic search and RAG.
pgvector — open source vector similarity search extension for PostgreSQL.
Milvus — cloud-native, open source vector database designed for scalable similarity search.
AnythingLLM — full-stack application to turn any document into a chatbot using RAG; supports local and cloud LLMs, multi-user, no-code setup.
privateGPT — interact privately with your documents using the power of LLMs without an internet connection; 100% private.
Verba — open source RAG chatbot powered by Weaviate; ingest documents and query them with LLMs.
▪️ Prompt Engineering Resources & Tools
Awesome Prompt Engineering — a curated list of resources for prompt engineering: papers, guides, tools, datasets, and examples.
Anthropic Prompt Engineering Docs — official, comprehensive guide to prompting Claude models effectively.
OpenAI Prompt Engineering Guide — official strategies and tactics for getting better results from GPT models.
Prompt Engineering Guide — a comprehensive guide covering techniques like zero-shot, few-shot, chain-of-thought, and more.
PromptPerfect — online tool to automatically optimize your prompts for various LLMs.
DSPy — Stanford's framework for algorithmically optimizing LLM prompts and weights rather than hand-crafting them.
Promptfoo — open source tool for testing, evaluating, and red-teaming LLM prompts and applications.
LMSYS Chatbot Arena — open platform for benchmarking LLMs side-by-side using human preference votes.
▪️ AI Security: Red Teaming, Prompt Injection & Jailbreaking Research
Garak — open source LLM vulnerability scanner by NVIDIA; probes models for hallucination, data leakage, prompt injection, toxicity, and more.
PyRIT — Python Risk Identification Toolkit for Generative AI by Microsoft; automates red teaming of LLM-based systems.
promptmap — automatically tests prompt injection attacks on ChatGPT instances.
PromptBench — unified library for evaluating and understanding the robustness of LLMs against adversarial prompts.
Pliny the Prompter — well-known community resource cataloging jailbreak prompts and bypasses for research and red-teaming.
AI Village CTF Resources — community hub for AI security research, CTF challenges, and adversarial ML papers.
LLM Security — curated list of papers, blog posts, and research on LLM security vulnerabilities including prompt injection, data poisoning, and model inversion.
Rebuff — open source prompt injection detection API designed to protect LLM-powered applications.
LLM Guard — security toolkit for LLM interactions; detects and sanitizes harmful inputs and outputs in production.
Vigil — open source LLM prompt injection and security scanner; detects jailbreaks, prompt leaking, and other adversarial inputs.
▪️ AI Observability, Evaluation & MLOps
Langfuse — open source LLM engineering platform for tracing, evaluation, prompt management, and metrics.
Phoenix — open source AI observability platform by Arize for LLM traces, evaluations, and dataset management.
Weights & Biases (W&B) — MLOps platform for tracking experiments, visualizing metrics, and managing LLM fine-tuning runs.
MLflow — open source platform for managing the ML lifecycle, including experiment tracking, model registry, and serving.
Helicone — open source LLM observability platform; logs, monitors, and analyzes every LLM API call.
Ragas — framework for evaluating RAG pipelines using metrics like faithfulness, answer relevancy, and context precision.
DeepEval — open source LLM evaluation framework; unit-test your LLM outputs with metrics like hallucination, toxicity, and bias.
▪️ AI-Powered Security & Sysadmin Tools
Nuclei AI — the widely-used vulnerability scanner now supports AI-generated templates for faster coverage of new CVEs.
Burp AI extensions — Burp Suite extensions that use LLMs to analyze HTTP traffic, suggest attack vectors, and explain vulnerabilities.
PentestGPT — GPT-powered penetration testing assistant that guides security professionals through testing methodology.
Vulnhuntr — uses Claude AI to autonomously find zero-day vulnerabilities in Python codebases via static analysis.
AISIRT / Incident Response AI — AI-assisted SIEM playbooks and triage tools; LLM-powered alert summarization and log analysis.
SWE-agent — AI agent that autonomously fixes bugs and resolves GitHub issues by interacting with a terminal like a human developer.
▪️ Inspiring Lists & Further Reading
Awesome LLM — curated list of large language model papers, frameworks, tools, and resources.
Awesome AI Agents — comprehensive list of AI autonomous agent projects and frameworks.
Awesome LLM Security — curated resources on LLM security, adversarial attacks, and defenses.
Awesome Ollama — community-curated tools, integrations, and resources for the Ollama ecosystem.
LLM Leaderboard (Open LLM) — Hugging Face's open benchmarking leaderboard for comparing local/open-source LLMs.
Simon Willison's Blog — one of the best blogs tracking the LLM tooling landscape, with practical guides and weekly AI news updates.
The Batch by deeplearning.ai — Andrew Ng's weekly newsletter covering the most important AI developments for practitioners.
AI Canon by a16z — curated reading list of the most important AI/ML papers, blog posts, and courses.
Why This Matters
The tools above are no longer "optional extras" — they are rapidly becoming core infrastructure for:
The book covers
vim,tmux,nmap, andWireshark— all timeless, essential tools.Ollama,Claude Code,Aider, andGarakare their 2024–2026 equivalents.Checklist
##### ▪️ Categoryheaders,[**Name**](url) — descriptionentries)masterbranchThank you for maintaining this repository — it's an incredible resource. 🙏