feat: Add 75 AI Research Engineering Skills from AI-research-SKILLs #259
Conversation
Added comprehensive AI research engineering skills from the zechenzhangAGI/AI-research-SKILLs repository covering: Categories added (19): - Model Architecture (4 skills): litgpt, nanogpt, mamba, rwkv - Tokenization (2 skills): huggingface-tokenizers, sentencepiece - Fine-Tuning (4 skills): axolotl, llama-factory, peft, unsloth - Mechanistic Interpretability (4 skills): transformer-lens, saelens, pyvene, nnsight - Data Processing (2 skills): ray-data, nemo-curator - Post-Training (4 skills): trl-fine-tuning, grpo-rl-training, openrlhf, simpo - Safety & Alignment (3 skills): constitutional-ai, llamaguard, nemo-guardrails - Distributed Training (6 skills): accelerate, deepspeed, megatron-core, pytorch-fsdp, pytorch-lightning, ray-train - Infrastructure (3 skills): modal, skypilot, lambda-labs - Optimization (6 skills): flash-attention, bitsandbytes, gptq, awq, hqq, gguf - Evaluation (2 skills): lm-evaluation-harness, bigcode-evaluation-harness - Inference Serving (4 skills): vllm, tensorrt-llm, llama-cpp, sglang - MLOps (3 skills): weights-and-biases, mlflow, tensorboard - Agents (4 skills): langchain, llamaindex, crewai, autogpt - RAG (5 skills): chroma, faiss, pinecone, qdrant, sentence-transformers - Prompt Engineering (4 skills): dspy, instructor, guidance, outlines - Observability (2 skills): langsmith, phoenix - Multimodal (7 skills): clip, whisper, llava, stable-diffusion, segment-anything, blip-2, audiocraft - Emerging Techniques (6 skills): moe-training, model-merging, long-context, speculative-decoding, knowledge-distillation, model-pruning Total: 75 new skills added to cli-tool/components/skills/ai-research/ Components catalog updated: 259 → 334 skills Source: https://github.com/zechenzhangAGI/AI-research-SKILLs License: MIT Author: Orchestra Research
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
40 issues found across 268 files
Note: This PR contains a large number of files. cubic only reviews up to 75 files per PR, so some files may not have been reviewed.
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them.
<file name="cli-tool/components/skills/ai-research/agents-langchain/SKILL.md">
<violation number="1" location="cli-tool/components/skills/ai-research/agents-langchain/SKILL.md:145">
P0: Critical security vulnerability: Using `eval()` on user-controlled input enables arbitrary code execution. An LLM agent could pass malicious Python expressions to this calculator tool. Use a safe math expression parser like `numexpr`, `ast.literal_eval`, or `simpleeval` instead.</violation>
<violation number="2" location="cli-tool/components/skills/ai-research/agents-langchain/SKILL.md:303">
P2: Incorrect API usage: `WebBaseLoader.load()` doesn't accept URL arguments. The URL must be provided in the constructor. This example will fail at runtime.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/agents-langchain/references/agents.md">
<violation number="1" location="cli-tool/components/skills/ai-research/agents-langchain/references/agents.md:25">
P0: Using `eval()` on user input is a critical security vulnerability that allows arbitrary code execution. Documentation examples should demonstrate safe patterns since developers often copy them directly. Consider using a safer alternative like `ast.literal_eval()` for numeric literals, or a library like `simpleeval` for safe math expression evaluation.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/agents-crewai/SKILL.md">
<violation number="1" location="cli-tool/components/skills/ai-research/agents-crewai/SKILL.md:241">
P1: Security: Using `eval()` on untrusted input enables arbitrary code execution. In AI agent tools, inputs can come from LLMs (which may be prompt-injected) or external sources. Consider using a safe math expression parser like `simpleeval` or `numexpr`, or at minimum add a security warning about this being an unsafe example.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/infrastructure-lambda-labs/references/advanced-usage.md">
<violation number="1" location="cli-tool/components/skills/ai-research/infrastructure-lambda-labs/references/advanced-usage.md:85">
P2: Missing `import functools` - this example will raise a `NameError` when copied by users.</violation>
<violation number="2" location="cli-tool/components/skills/ai-research/infrastructure-lambda-labs/references/advanced-usage.md:234">
P2: Security: `AutoAddPolicy()` accepts any host key without verification, enabling MITM attacks. Consider using `RejectPolicy()` or `WarningPolicy()` with known_hosts verification, especially since this file has a 'Security Best Practices' section.</violation>
<violation number="3" location="cli-tool/components/skills/ai-research/infrastructure-lambda-labs/references/advanced-usage.md:240">
P1: Bug: `stderr.read()` is called twice. The first call consumes the stream, so the second call inside the print will always return empty bytes. Store the result in a variable first.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/distributed-training-ray-train/SKILL.md">
<violation number="1" location="cli-tool/components/skills/ai-research/distributed-training-ray-train/SKILL.md:99">
P1: The `dataloader` variable is used but never defined within `train_func`. In Ray Train, the training function runs on workers without access to the outer scope, so `dataloader` needs to be created inside the function or passed via `config`.</violation>
<violation number="2" location="cli-tool/components/skills/ai-research/distributed-training-ray-train/SKILL.md:238">
P1: Incorrect checkpoint pattern. Files should be saved to a directory first, then `Checkpoint.from_directory()` should be called on that directory. Also, `checkpoint.path / "model.pt"` may fail as `path` might be a string, not a Path object.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/distributed-training-ray-train/references/multi-node.md">
<violation number="1" location="cli-tool/components/skills/ai-research/distributed-training-ray-train/references/multi-node.md:366">
P1: SLURM script bug: `head_node_ip` is set to each node's own IP, so workers will try to connect to themselves instead of the head node. Use SLURM's node list to get the actual head node IP on all nodes.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/emerging-techniques-knowledge-distillation/SKILL.md">
<violation number="1" location="cli-tool/components/skills/ai-research/emerging-techniques-knowledge-distillation/SKILL.md:178">
P1: Undefined variable `train_dataset` in production training script. This will raise a `NameError` at runtime. Consider adding a `train_dataset` parameter to the function or loading it within the function body.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/agents-autogpt/references/advanced-usage.md">
<violation number="1" location="cli-tool/components/skills/ai-research/agents-autogpt/references/advanced-usage.md:75">
P2: The synchronous `OpenAI` client doesn't support `await`. Use `AsyncOpenAI` for async operations.</violation>
<violation number="2" location="cli-tool/components/skills/ai-research/agents-autogpt/references/advanced-usage.md:485">
P1: Flask-style tuple return doesn't work in FastAPI. This will return status 200 with the tuple as content, not a 401 error. Use `raise HTTPException(status_code=401, detail="Invalid signature")` or `return JSONResponse(status_code=401, content={"error": "Invalid signature"})`.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/emerging-techniques-long-context/references/fine_tuning.md">
<violation number="1" location="cli-tool/components/skills/ai-research/emerging-techniques-long-context/references/fine_tuning.md:47">
P2: Parameter `tokenizer` defaults to `None` but is used directly without a null check. This will raise `AttributeError` if called without providing a tokenizer. Consider either removing the default value or adding a validation check.</violation>
<violation number="2" location="cli-tool/components/skills/ai-research/emerging-techniques-long-context/references/fine_tuning.md:64">
P2: `next(dataset)` will raise `TypeError` because HuggingFace Dataset objects are not iterators. This needs to be refactored to use an iterator or index-based access.</violation>
<violation number="3" location="cli-tool/components/skills/ai-research/emerging-techniques-long-context/references/fine_tuning.md:406">
P2: `random` module is used but not imported. Add `import random` to the imports for this code example.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/distributed-training-accelerate/references/megatron-integration.md">
<violation number="1" location="cli-tool/components/skills/ai-research/distributed-training-accelerate/references/megatron-integration.md:265">
P2: Undefined variable `train_loader` in example code. The "Full Training Script" uses `train_loader` before it's defined, which will cause a `NameError`. Add dataloader creation before the `prepare()` call.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/inference-serving-vllm/references/troubleshooting.md">
<violation number="1" location="cli-tool/components/skills/ai-research/inference-serving-vllm/references/troubleshooting.md:377">
P2: Incorrect port for vLLM metrics endpoint. vLLM serves Prometheus metrics at `/metrics` on the same port as the API server (default 8000), not on port 9090. Port 9090 is the default Prometheus server port, not the vLLM metrics endpoint. Users following this guide will get connection refused errors.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/agents-llamaindex/SKILL.md">
<violation number="1" location="cli-tool/components/skills/ai-research/agents-llamaindex/SKILL.md:351">
P2: Deprecated Pinecone API: `pinecone.init()` was deprecated in SDK v3.0.0. This pattern will fail with current Pinecone versions. The repository's own Pinecone skill shows the correct modern API pattern using `Pinecone()` class instantiation.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/inference-serving-vllm/references/quantization.md">
<violation number="1" location="cli-tool/components/skills/ai-research/inference-serving-vllm/references/quantization.md:133">
P2: `quantize_config` is used in `from_pretrained()` before it's defined. Move the `quantize_config = BaseQuantizeConfig(...)` block above the `from_pretrained()` call.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/distributed-training-accelerate/references/custom-plugins.md">
<violation number="1" location="cli-tool/components/skills/ai-research/distributed-training-accelerate/references/custom-plugins.md:264">
P2: Undefined variable `threshold` in example code. This will cause a `NameError` at runtime. The threshold should be defined as a class parameter in `__init__` and stored as `self.threshold`.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/data-processing-nemo-curator/SKILL.md">
<violation number="1" location="cli-tool/components/skills/ai-research/data-processing-nemo-curator/SKILL.md:6">
P2: License inconsistency: frontmatter says MIT but the Resources section (and the actual NeMo Curator library) uses Apache 2.0. Update the frontmatter to match the correct license.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/inference-serving-sglang/SKILL.md">
<violation number="1" location="cli-tool/components/skills/ai-research/inference-serving-sglang/SKILL.md:43">
P2: Using `-i` replaces PyPI entirely, causing `sglang[all]` installation to fail. Use `--extra-index-url` instead to add the FlashInfer index while keeping PyPI as the primary source.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/inference-serving-tensorrt-llm/references/optimization.md">
<violation number="1" location="cli-tool/components/skills/ai-research/inference-serving-tensorrt-llm/references/optimization.md:68">
P2: Shell command line continuation is broken by inline comments. In bash, `\` must be immediately followed by a newline to continue the line. The spaces and comments after `\` break this. Move comments above each line or remove them.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/inference-serving-sglang/references/deployment.md">
<violation number="1" location="cli-tool/components/skills/ai-research/inference-serving-sglang/references/deployment.md:288">
P2: Bash line continuation broken by inline comment. In bash, `\` must be the last character on a line for continuation. Adding spaces and comments after `\` breaks the command. Move comments to separate lines above or remove them.</violation>
<violation number="2" location="cli-tool/components/skills/ai-research/inference-serving-sglang/references/deployment.md:298">
P2: Bash line continuation broken by inline comment. The `\` followed by spaces and `# More memory for batching` will cause a syntax error. Move comments to separate lines or remove them.</violation>
<violation number="3" location="cli-tool/components/skills/ai-research/inference-serving-sglang/references/deployment.md:299">
P2: Bash line continuation broken by inline comment. The `\` followed by space and `# Larger cache` breaks the command.</violation>
<violation number="4" location="cli-tool/components/skills/ai-research/inference-serving-sglang/references/deployment.md:309">
P2: Bash line continuation broken by inline comment. The `\` followed by spaces and the comment will cause a syntax error when this command is executed.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/distributed-training-megatron-core/SKILL.md">
<violation number="1" location="cli-tool/components/skills/ai-research/distributed-training-megatron-core/SKILL.md:58">
P2: Math error in parallelism table: 405B row claims 128 GPUs but TP×PP×DP×CP = 8×8×2×2 = 256. Based on the correct example shown later (lines 207-213), DP should be 1, not 2, to achieve 128 GPUs.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/agents-langchain/references/integration.md">
<violation number="1" location="cli-tool/components/skills/ai-research/agents-langchain/references/integration.md:40">
P2: Deprecated Pinecone API pattern. The `pinecone.init()` and `pinecone.create_index()` methods are from the old v2 client. The current Pinecone Python client (v3+) uses `Pinecone(api_key=...)` constructor and different method signatures.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/distributed-training-pytorch-lightning/references/callbacks.md">
<violation number="1" location="cli-tool/components/skills/ai-research/distributed-training-pytorch-lightning/references/callbacks.md:252">
P2: Forward hooks registered but never removed - this causes memory leaks. The handles returned by `register_forward_hook` should be stored and removed after use (e.g., in `on_train_batch_end`). Without cleanup, hooks accumulate each epoch.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/inference-serving-vllm/references/optimization.md">
<violation number="1" location="cli-tool/components/skills/ai-research/inference-serving-vllm/references/optimization.md:198">
P2: Inline comments after backslash line continuations break bash syntax. When users copy-paste this command, it will fail because `# comment` terminates the line, and subsequent arguments become separate (failing) commands. Move comments above each line or remove them.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/inference-serving-tensorrt-llm/SKILL.md">
<violation number="1" location="cli-tool/components/skills/ai-research/inference-serving-tensorrt-llm/SKILL.md:76">
P2: Invalid bash syntax: backslash line continuation must be the last character on the line. Having `\ # comment` will not work as intended - the backslash escapes the space, not the newline. Either remove inline comments or place them on separate lines above each flag.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/emerging-techniques-knowledge-distillation/references/minillm.md">
<violation number="1" location="cli-tool/components/skills/ai-research/emerging-techniques-knowledge-distillation/references/minillm.md:139">
P2: Incorrect loss computation in policy gradient example. Taking the mean of raw logits (`-student_outputs.logits.mean()`) is not a valid loss. Since labels are provided to the model call, use `student_outputs.loss` which contains the properly computed cross-entropy loss.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/distributed-training-megatron-core/references/training-recipes.md">
<violation number="1" location="cli-tool/components/skills/ai-research/distributed-training-megatron-core/references/training-recipes.md:482">
P2: Invalid parallelism configuration: 2 × 16 × 64 = 2,048 GPUs required, but only 1,024 specified. Data parallel degree of 0.5 is impossible. The configuration should use 2,048 GPUs or reduce parallelism dimensions.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/agents-langchain/references/rag.md">
<violation number="1" location="cli-tool/components/skills/ai-research/agents-langchain/references/rag.md:359">
P2: Undefined variables `calculator` and `search` in code example. These tools are referenced but never defined or imported, which would cause a `NameError` if users copy this code. Either define these tools or use only the defined `retriever_tool`.</violation>
<violation number="2" location="cli-tool/components/skills/ai-research/agents-langchain/references/rag.md:387">
P2: Function `create_agent` is not imported and doesn't exist in LangChain. This should use `create_tool_calling_agent` (which was imported earlier) with the correct parameter `llm=` instead of `model=`.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/emerging-techniques-long-context/references/extension_methods.md">
<violation number="1" location="cli-tool/components/skills/ai-research/emerging-techniques-long-context/references/extension_methods.md:277">
P2: Incorrect example output comment. The `get_slopes_power_of_2(8)` function actually produces `[0.5, 0.25, 0.125, 0.0625, 0.03125, 0.015625, 0.0078125, 0.00390625]` (decreasing values < 1), not the claimed `[0.0625, 0.125, 0.25, 0.5, 1.0, 2.0, 4.0, 8.0]`. This misleading documentation could confuse readers learning about ALiBi slopes.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/agents-crewai/references/tools.md">
<violation number="1" location="cli-tool/components/skills/ai-research/agents-crewai/references/tools.md:297">
P2: Using `eval()` with user input is dangerous, and the "safety check" (blocking letters) is insufficient. This documentation example teaches an insecure pattern that developers may copy. Consider using a safer alternative like the `simpleeval` library or `ast.literal_eval()` for simple cases, and update the comment to not imply the current approach is secure.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/inference-serving-vllm/references/server-deployment.md">
<violation number="1" location="cli-tool/components/skills/ai-research/inference-serving-vllm/references/server-deployment.md:201">
P2: The `--trust-remote-code` flag allows execution of arbitrary Python code from model repositories, which is a security risk in production. This documentation example should include a warning about its security implications or only recommend it for trusted model sources.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
| # Define custom tool | ||
| calculator = Tool( | ||
| name="Calculator", | ||
| func=lambda x: eval(x), |
There was a problem hiding this comment.
P0: Critical security vulnerability: Using eval() on user-controlled input enables arbitrary code execution. An LLM agent could pass malicious Python expressions to this calculator tool. Use a safe math expression parser like numexpr, ast.literal_eval, or simpleeval instead.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At cli-tool/components/skills/ai-research/agents-langchain/SKILL.md, line 145:
<comment>Critical security vulnerability: Using `eval()` on user-controlled input enables arbitrary code execution. An LLM agent could pass malicious Python expressions to this calculator tool. Use a safe math expression parser like `numexpr`, `ast.literal_eval`, or `simpleeval` instead.</comment>
<file context>
@@ -0,0 +1,480 @@
+# Define custom tool
+calculator = Tool(
+ name="Calculator",
+ func=lambda x: eval(x),
+ description="Useful for math calculations. Input: valid Python expression."
+)
</file context>
| # Define tools | ||
| def calculator(expression: str) -> str: | ||
| """Evaluate a math expression.""" | ||
| return str(eval(expression)) |
There was a problem hiding this comment.
P0: Using eval() on user input is a critical security vulnerability that allows arbitrary code execution. Documentation examples should demonstrate safe patterns since developers often copy them directly. Consider using a safer alternative like ast.literal_eval() for numeric literals, or a library like simpleeval for safe math expression evaluation.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At cli-tool/components/skills/ai-research/agents-langchain/references/agents.md, line 25:
<comment>Using `eval()` on user input is a critical security vulnerability that allows arbitrary code execution. Documentation examples should demonstrate safe patterns since developers often copy them directly. Consider using a safer alternative like `ast.literal_eval()` for numeric literals, or a library like `simpleeval` for safe math expression evaluation.</comment>
<file context>
@@ -0,0 +1,499 @@
+# Define tools
+def calculator(expression: str) -> str:
+ """Evaluate a math expression."""
+ return str(eval(expression))
+
+def search(query: str) -> str:
</file context>
|
|
||
| def _run(self, expression: str) -> str: | ||
| try: | ||
| result = eval(expression) |
There was a problem hiding this comment.
P1: Security: Using eval() on untrusted input enables arbitrary code execution. In AI agent tools, inputs can come from LLMs (which may be prompt-injected) or external sources. Consider using a safe math expression parser like simpleeval or numexpr, or at minimum add a security warning about this being an unsafe example.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At cli-tool/components/skills/ai-research/agents-crewai/SKILL.md, line 241:
<comment>Security: Using `eval()` on untrusted input enables arbitrary code execution. In AI agent tools, inputs can come from LLMs (which may be prompt-injected) or external sources. Consider using a safe math expression parser like `simpleeval` or `numexpr`, or at minimum add a security warning about this being an unsafe example.</comment>
<file context>
@@ -0,0 +1,498 @@
+
+ def _run(self, expression: str) -> str:
+ try:
+ result = eval(expression)
+ return f"Result: {result}"
+ except Exception as e:
</file context>
| if stderr.read(): | ||
| print(f"Error: {stderr.read().decode()}") |
There was a problem hiding this comment.
P1: Bug: stderr.read() is called twice. The first call consumes the stream, so the second call inside the print will always return empty bytes. Store the result in a variable first.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At cli-tool/components/skills/ai-research/infrastructure-lambda-labs/references/advanced-usage.md, line 240:
<comment>Bug: `stderr.read()` is called twice. The first call consumes the stream, so the second call inside the print will always return empty bytes. Store the result in a variable first.</comment>
<file context>
@@ -0,0 +1,611 @@
+ for cmd in commands:
+ stdin, stdout, stderr = client.exec_command(cmd)
+ print(stdout.read().decode())
+ if stderr.read():
+ print(f"Error: {stderr.read().decode()}")
+
</file context>
| if stderr.read(): | |
| print(f"Error: {stderr.read().decode()}") | |
| err = stderr.read() | |
| if err: | |
| print(f"Error: {err.decode()}") |
|
|
||
| # Prepare for distributed (automatic device placement) | ||
| model = train.torch.prepare_model(model) | ||
| dataloader = train.torch.prepare_data_loader(dataloader) |
There was a problem hiding this comment.
P1: The dataloader variable is used but never defined within train_func. In Ray Train, the training function runs on workers without access to the outer scope, so dataloader needs to be created inside the function or passed via config.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At cli-tool/components/skills/ai-research/distributed-training-ray-train/SKILL.md, line 99:
<comment>The `dataloader` variable is used but never defined within `train_func`. In Ray Train, the training function runs on workers without access to the outer scope, so `dataloader` needs to be created inside the function or passed via `config`.</comment>
<file context>
@@ -0,0 +1,406 @@
+
+ # Prepare for distributed (automatic device placement)
+ model = train.torch.prepare_model(model)
+ dataloader = train.torch.prepare_data_loader(dataloader)
+
+ for epoch in range(epochs):
</file context>
| # Create agent with retriever tool | ||
| agent = create_tool_calling_agent( | ||
| llm=llm, | ||
| tools=[retriever_tool, calculator, search], |
There was a problem hiding this comment.
P2: Undefined variables calculator and search in code example. These tools are referenced but never defined or imported, which would cause a NameError if users copy this code. Either define these tools or use only the defined retriever_tool.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At cli-tool/components/skills/ai-research/agents-langchain/references/rag.md, line 359:
<comment>Undefined variables `calculator` and `search` in code example. These tools are referenced but never defined or imported, which would cause a `NameError` if users copy this code. Either define these tools or use only the defined `retriever_tool`.</comment>
<file context>
@@ -0,0 +1,600 @@
+# Create agent with retriever tool
+agent = create_tool_calling_agent(
+ llm=llm,
+ tools=[retriever_tool, calculator, search],
+ system_prompt="Use python_docs tool for Python questions"
+)
</file context>
| **Example slopes for 8 heads**: | ||
| ```python | ||
| slopes = get_alibi_slopes(8) | ||
| # Output: [0.0625, 0.125, 0.25, 0.5, 1.0, 2.0, 4.0, 8.0] |
There was a problem hiding this comment.
P2: Incorrect example output comment. The get_slopes_power_of_2(8) function actually produces [0.5, 0.25, 0.125, 0.0625, 0.03125, 0.015625, 0.0078125, 0.00390625] (decreasing values < 1), not the claimed [0.0625, 0.125, 0.25, 0.5, 1.0, 2.0, 4.0, 8.0]. This misleading documentation could confuse readers learning about ALiBi slopes.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At cli-tool/components/skills/ai-research/emerging-techniques-long-context/references/extension_methods.md, line 277:
<comment>Incorrect example output comment. The `get_slopes_power_of_2(8)` function actually produces `[0.5, 0.25, 0.125, 0.0625, 0.03125, 0.015625, 0.0078125, 0.00390625]` (decreasing values < 1), not the claimed `[0.0625, 0.125, 0.25, 0.5, 1.0, 2.0, 4.0, 8.0]`. This misleading documentation could confuse readers learning about ALiBi slopes.</comment>
<file context>
@@ -0,0 +1,468 @@
+**Example slopes for 8 heads**:
+```python
+slopes = get_alibi_slopes(8)
+# Output: [0.0625, 0.125, 0.25, 0.5, 1.0, 2.0, 4.0, 8.0]
+
+# Each head has different slope
</file context>
| # Output: [0.0625, 0.125, 0.25, 0.5, 1.0, 2.0, 4.0, 8.0] | |
| # Output: [0.5, 0.25, 0.125, 0.0625, 0.03125, 0.015625, 0.0078125, 0.00390625] |
| for char in expression: | ||
| if char.isalpha(): | ||
| return "Error: Letters not allowed" | ||
| result = eval(expression) |
There was a problem hiding this comment.
P2: Using eval() with user input is dangerous, and the "safety check" (blocking letters) is insufficient. This documentation example teaches an insecure pattern that developers may copy. Consider using a safer alternative like the simpleeval library or ast.literal_eval() for simple cases, and update the comment to not imply the current approach is secure.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At cli-tool/components/skills/ai-research/agents-crewai/references/tools.md, line 297:
<comment>Using `eval()` with user input is dangerous, and the "safety check" (blocking letters) is insufficient. This documentation example teaches an insecure pattern that developers may copy. Consider using a safer alternative like the `simpleeval` library or `ast.literal_eval()` for simple cases, and update the comment to not imply the current approach is secure.</comment>
<file context>
@@ -0,0 +1,429 @@
+ for char in expression:
+ if char.isalpha():
+ return "Error: Letters not allowed"
+ result = eval(expression)
+ return f"Result: {result}"
+ except Exception as e:
</file context>
| # Concatenate multiple documents | ||
| buffer = tokens | ||
| while len(buffer) < target_length: | ||
| next_example = next(dataset) |
There was a problem hiding this comment.
P2: next(dataset) will raise TypeError because HuggingFace Dataset objects are not iterators. This needs to be refactored to use an iterator or index-based access.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At cli-tool/components/skills/ai-research/emerging-techniques-long-context/references/fine_tuning.md, line 64:
<comment>`next(dataset)` will raise `TypeError` because HuggingFace Dataset objects are not iterators. This needs to be refactored to use an iterator or index-based access.</comment>
<file context>
@@ -0,0 +1,611 @@
+ # Concatenate multiple documents
+ buffer = tokens
+ while len(buffer) < target_length:
+ next_example = next(dataset)
+ buffer.extend(tokenizer.encode(next_example['text']))
+
</file context>
| --max-num-seqs 512 \ | ||
| --gpu-memory-utilization 0.95 \ | ||
| --enable-prefix-caching \ | ||
| --trust-remote-code |
There was a problem hiding this comment.
P2: The --trust-remote-code flag allows execution of arbitrary Python code from model repositories, which is a security risk in production. This documentation example should include a warning about its security implications or only recommend it for trusted model sources.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At cli-tool/components/skills/ai-research/inference-serving-vllm/references/server-deployment.md, line 201:
<comment>The `--trust-remote-code` flag allows execution of arbitrary Python code from model repositories, which is a security risk in production. This documentation example should include a warning about its security implications or only recommend it for trusted model sources.</comment>
<file context>
@@ -0,0 +1,255 @@
+ --max-num-seqs 512 \
+ --gpu-memory-utilization 0.95 \
+ --enable-prefix-caching \
+ --trust-remote-code
+```
+
</file context>
| --trust-remote-code | |
| --trust-remote-code # WARNING: Only use with trusted model sources - allows arbitrary code execution |
There was a problem hiding this comment.
40 issues found across 268 files
Note: This PR contains a large number of files. cubic only reviews up to 75 files per PR, so some files may not have been reviewed.
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them.
<file name="cli-tool/components/skills/ai-research/agents-langchain/SKILL.md">
<violation number="1" location="cli-tool/components/skills/ai-research/agents-langchain/SKILL.md:145">
P0: Critical security vulnerability: Using `eval()` on user-controlled input enables arbitrary code execution. An LLM agent could pass malicious Python expressions to this calculator tool. Use a safe math expression parser like `numexpr`, `ast.literal_eval`, or `simpleeval` instead.</violation>
<violation number="2" location="cli-tool/components/skills/ai-research/agents-langchain/SKILL.md:303">
P2: Incorrect API usage: `WebBaseLoader.load()` doesn't accept URL arguments. The URL must be provided in the constructor. This example will fail at runtime.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/agents-langchain/references/agents.md">
<violation number="1" location="cli-tool/components/skills/ai-research/agents-langchain/references/agents.md:25">
P0: Using `eval()` on user input is a critical security vulnerability that allows arbitrary code execution. Documentation examples should demonstrate safe patterns since developers often copy them directly. Consider using a safer alternative like `ast.literal_eval()` for numeric literals, or a library like `simpleeval` for safe math expression evaluation.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/agents-crewai/SKILL.md">
<violation number="1" location="cli-tool/components/skills/ai-research/agents-crewai/SKILL.md:241">
P1: Security: Using `eval()` on untrusted input enables arbitrary code execution. In AI agent tools, inputs can come from LLMs (which may be prompt-injected) or external sources. Consider using a safe math expression parser like `simpleeval` or `numexpr`, or at minimum add a security warning about this being an unsafe example.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/infrastructure-lambda-labs/references/advanced-usage.md">
<violation number="1" location="cli-tool/components/skills/ai-research/infrastructure-lambda-labs/references/advanced-usage.md:85">
P2: Missing `import functools` - this example will raise a `NameError` when copied by users.</violation>
<violation number="2" location="cli-tool/components/skills/ai-research/infrastructure-lambda-labs/references/advanced-usage.md:234">
P2: Security: `AutoAddPolicy()` accepts any host key without verification, enabling MITM attacks. Consider using `RejectPolicy()` or `WarningPolicy()` with known_hosts verification, especially since this file has a 'Security Best Practices' section.</violation>
<violation number="3" location="cli-tool/components/skills/ai-research/infrastructure-lambda-labs/references/advanced-usage.md:240">
P1: Bug: `stderr.read()` is called twice. The first call consumes the stream, so the second call inside the print will always return empty bytes. Store the result in a variable first.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/distributed-training-ray-train/SKILL.md">
<violation number="1" location="cli-tool/components/skills/ai-research/distributed-training-ray-train/SKILL.md:99">
P1: The `dataloader` variable is used but never defined within `train_func`. In Ray Train, the training function runs on workers without access to the outer scope, so `dataloader` needs to be created inside the function or passed via `config`.</violation>
<violation number="2" location="cli-tool/components/skills/ai-research/distributed-training-ray-train/SKILL.md:238">
P1: Incorrect checkpoint pattern. Files should be saved to a directory first, then `Checkpoint.from_directory()` should be called on that directory. Also, `checkpoint.path / "model.pt"` may fail as `path` might be a string, not a Path object.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/distributed-training-ray-train/references/multi-node.md">
<violation number="1" location="cli-tool/components/skills/ai-research/distributed-training-ray-train/references/multi-node.md:366">
P1: SLURM script bug: `head_node_ip` is set to each node's own IP, so workers will try to connect to themselves instead of the head node. Use SLURM's node list to get the actual head node IP on all nodes.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/emerging-techniques-knowledge-distillation/SKILL.md">
<violation number="1" location="cli-tool/components/skills/ai-research/emerging-techniques-knowledge-distillation/SKILL.md:178">
P1: Undefined variable `train_dataset` in production training script. This will raise a `NameError` at runtime. Consider adding a `train_dataset` parameter to the function or loading it within the function body.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/agents-autogpt/references/advanced-usage.md">
<violation number="1" location="cli-tool/components/skills/ai-research/agents-autogpt/references/advanced-usage.md:75">
P2: The synchronous `OpenAI` client doesn't support `await`. Use `AsyncOpenAI` for async operations.</violation>
<violation number="2" location="cli-tool/components/skills/ai-research/agents-autogpt/references/advanced-usage.md:485">
P1: Flask-style tuple return doesn't work in FastAPI. This will return status 200 with the tuple as content, not a 401 error. Use `raise HTTPException(status_code=401, detail="Invalid signature")` or `return JSONResponse(status_code=401, content={"error": "Invalid signature"})`.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/emerging-techniques-long-context/references/fine_tuning.md">
<violation number="1" location="cli-tool/components/skills/ai-research/emerging-techniques-long-context/references/fine_tuning.md:47">
P2: Parameter `tokenizer` defaults to `None` but is used directly without a null check. This will raise `AttributeError` if called without providing a tokenizer. Consider either removing the default value or adding a validation check.</violation>
<violation number="2" location="cli-tool/components/skills/ai-research/emerging-techniques-long-context/references/fine_tuning.md:64">
P2: `next(dataset)` will raise `TypeError` because HuggingFace Dataset objects are not iterators. This needs to be refactored to use an iterator or index-based access.</violation>
<violation number="3" location="cli-tool/components/skills/ai-research/emerging-techniques-long-context/references/fine_tuning.md:406">
P2: `random` module is used but not imported. Add `import random` to the imports for this code example.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/distributed-training-accelerate/references/megatron-integration.md">
<violation number="1" location="cli-tool/components/skills/ai-research/distributed-training-accelerate/references/megatron-integration.md:265">
P2: Undefined variable `train_loader` in example code. The "Full Training Script" uses `train_loader` before it's defined, which will cause a `NameError`. Add dataloader creation before the `prepare()` call.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/inference-serving-vllm/references/troubleshooting.md">
<violation number="1" location="cli-tool/components/skills/ai-research/inference-serving-vllm/references/troubleshooting.md:377">
P2: Incorrect port for vLLM metrics endpoint. vLLM serves Prometheus metrics at `/metrics` on the same port as the API server (default 8000), not on port 9090. Port 9090 is the default Prometheus server port, not the vLLM metrics endpoint. Users following this guide will get connection refused errors.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/agents-llamaindex/SKILL.md">
<violation number="1" location="cli-tool/components/skills/ai-research/agents-llamaindex/SKILL.md:351">
P2: Deprecated Pinecone API: `pinecone.init()` was deprecated in SDK v3.0.0. This pattern will fail with current Pinecone versions. The repository's own Pinecone skill shows the correct modern API pattern using `Pinecone()` class instantiation.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/inference-serving-vllm/references/quantization.md">
<violation number="1" location="cli-tool/components/skills/ai-research/inference-serving-vllm/references/quantization.md:133">
P2: `quantize_config` is used in `from_pretrained()` before it's defined. Move the `quantize_config = BaseQuantizeConfig(...)` block above the `from_pretrained()` call.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/distributed-training-accelerate/references/custom-plugins.md">
<violation number="1" location="cli-tool/components/skills/ai-research/distributed-training-accelerate/references/custom-plugins.md:264">
P2: Undefined variable `threshold` in example code. This will cause a `NameError` at runtime. The threshold should be defined as a class parameter in `__init__` and stored as `self.threshold`.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/data-processing-nemo-curator/SKILL.md">
<violation number="1" location="cli-tool/components/skills/ai-research/data-processing-nemo-curator/SKILL.md:6">
P2: License inconsistency: frontmatter says MIT but the Resources section (and the actual NeMo Curator library) uses Apache 2.0. Update the frontmatter to match the correct license.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/inference-serving-sglang/SKILL.md">
<violation number="1" location="cli-tool/components/skills/ai-research/inference-serving-sglang/SKILL.md:43">
P2: Using `-i` replaces PyPI entirely, causing `sglang[all]` installation to fail. Use `--extra-index-url` instead to add the FlashInfer index while keeping PyPI as the primary source.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/inference-serving-tensorrt-llm/references/optimization.md">
<violation number="1" location="cli-tool/components/skills/ai-research/inference-serving-tensorrt-llm/references/optimization.md:68">
P2: Shell command line continuation is broken by inline comments. In bash, `\` must be immediately followed by a newline to continue the line. The spaces and comments after `\` break this. Move comments above each line or remove them.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/inference-serving-sglang/references/deployment.md">
<violation number="1" location="cli-tool/components/skills/ai-research/inference-serving-sglang/references/deployment.md:288">
P2: Bash line continuation broken by inline comment. In bash, `\` must be the last character on a line for continuation. Adding spaces and comments after `\` breaks the command. Move comments to separate lines above or remove them.</violation>
<violation number="2" location="cli-tool/components/skills/ai-research/inference-serving-sglang/references/deployment.md:298">
P2: Bash line continuation broken by inline comment. The `\` followed by spaces and `# More memory for batching` will cause a syntax error. Move comments to separate lines or remove them.</violation>
<violation number="3" location="cli-tool/components/skills/ai-research/inference-serving-sglang/references/deployment.md:299">
P2: Bash line continuation broken by inline comment. The `\` followed by space and `# Larger cache` breaks the command.</violation>
<violation number="4" location="cli-tool/components/skills/ai-research/inference-serving-sglang/references/deployment.md:309">
P2: Bash line continuation broken by inline comment. The `\` followed by spaces and the comment will cause a syntax error when this command is executed.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/distributed-training-megatron-core/SKILL.md">
<violation number="1" location="cli-tool/components/skills/ai-research/distributed-training-megatron-core/SKILL.md:58">
P2: Math error in parallelism table: 405B row claims 128 GPUs but TP×PP×DP×CP = 8×8×2×2 = 256. Based on the correct example shown later (lines 207-213), DP should be 1, not 2, to achieve 128 GPUs.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/agents-langchain/references/integration.md">
<violation number="1" location="cli-tool/components/skills/ai-research/agents-langchain/references/integration.md:40">
P2: Deprecated Pinecone API pattern. The `pinecone.init()` and `pinecone.create_index()` methods are from the old v2 client. The current Pinecone Python client (v3+) uses `Pinecone(api_key=...)` constructor and different method signatures.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/distributed-training-pytorch-lightning/references/callbacks.md">
<violation number="1" location="cli-tool/components/skills/ai-research/distributed-training-pytorch-lightning/references/callbacks.md:252">
P2: Forward hooks registered but never removed - this causes memory leaks. The handles returned by `register_forward_hook` should be stored and removed after use (e.g., in `on_train_batch_end`). Without cleanup, hooks accumulate each epoch.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/inference-serving-vllm/references/optimization.md">
<violation number="1" location="cli-tool/components/skills/ai-research/inference-serving-vllm/references/optimization.md:198">
P2: Inline comments after backslash line continuations break bash syntax. When users copy-paste this command, it will fail because `# comment` terminates the line, and subsequent arguments become separate (failing) commands. Move comments above each line or remove them.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/inference-serving-tensorrt-llm/SKILL.md">
<violation number="1" location="cli-tool/components/skills/ai-research/inference-serving-tensorrt-llm/SKILL.md:76">
P2: Invalid bash syntax: backslash line continuation must be the last character on the line. Having `\ # comment` will not work as intended - the backslash escapes the space, not the newline. Either remove inline comments or place them on separate lines above each flag.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/emerging-techniques-knowledge-distillation/references/minillm.md">
<violation number="1" location="cli-tool/components/skills/ai-research/emerging-techniques-knowledge-distillation/references/minillm.md:139">
P2: Incorrect loss computation in policy gradient example. Taking the mean of raw logits (`-student_outputs.logits.mean()`) is not a valid loss. Since labels are provided to the model call, use `student_outputs.loss` which contains the properly computed cross-entropy loss.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/distributed-training-megatron-core/references/training-recipes.md">
<violation number="1" location="cli-tool/components/skills/ai-research/distributed-training-megatron-core/references/training-recipes.md:482">
P2: Invalid parallelism configuration: 2 × 16 × 64 = 2,048 GPUs required, but only 1,024 specified. Data parallel degree of 0.5 is impossible. The configuration should use 2,048 GPUs or reduce parallelism dimensions.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/agents-langchain/references/rag.md">
<violation number="1" location="cli-tool/components/skills/ai-research/agents-langchain/references/rag.md:359">
P2: Undefined variables `calculator` and `search` in code example. These tools are referenced but never defined or imported, which would cause a `NameError` if users copy this code. Either define these tools or use only the defined `retriever_tool`.</violation>
<violation number="2" location="cli-tool/components/skills/ai-research/agents-langchain/references/rag.md:387">
P2: Function `create_agent` is not imported and doesn't exist in LangChain. This should use `create_tool_calling_agent` (which was imported earlier) with the correct parameter `llm=` instead of `model=`.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/emerging-techniques-long-context/references/extension_methods.md">
<violation number="1" location="cli-tool/components/skills/ai-research/emerging-techniques-long-context/references/extension_methods.md:277">
P2: Incorrect example output comment. The `get_slopes_power_of_2(8)` function actually produces `[0.5, 0.25, 0.125, 0.0625, 0.03125, 0.015625, 0.0078125, 0.00390625]` (decreasing values < 1), not the claimed `[0.0625, 0.125, 0.25, 0.5, 1.0, 2.0, 4.0, 8.0]`. This misleading documentation could confuse readers learning about ALiBi slopes.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/agents-crewai/references/tools.md">
<violation number="1" location="cli-tool/components/skills/ai-research/agents-crewai/references/tools.md:297">
P2: Using `eval()` with user input is dangerous, and the "safety check" (blocking letters) is insufficient. This documentation example teaches an insecure pattern that developers may copy. Consider using a safer alternative like the `simpleeval` library or `ast.literal_eval()` for simple cases, and update the comment to not imply the current approach is secure.</violation>
</file>
<file name="cli-tool/components/skills/ai-research/inference-serving-vllm/references/server-deployment.md">
<violation number="1" location="cli-tool/components/skills/ai-research/inference-serving-vllm/references/server-deployment.md:201">
P2: The `--trust-remote-code` flag allows execution of arbitrary Python code from model repositories, which is a security risk in production. This documentation example should include a warning about its security implications or only recommend it for trusted model sources.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
| # Define custom tool | ||
| calculator = Tool( | ||
| name="Calculator", | ||
| func=lambda x: eval(x), |
There was a problem hiding this comment.
P0: Critical security vulnerability: Using eval() on user-controlled input enables arbitrary code execution. An LLM agent could pass malicious Python expressions to this calculator tool. Use a safe math expression parser like numexpr, ast.literal_eval, or simpleeval instead.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At cli-tool/components/skills/ai-research/agents-langchain/SKILL.md, line 145:
<comment>Critical security vulnerability: Using `eval()` on user-controlled input enables arbitrary code execution. An LLM agent could pass malicious Python expressions to this calculator tool. Use a safe math expression parser like `numexpr`, `ast.literal_eval`, or `simpleeval` instead.</comment>
<file context>
@@ -0,0 +1,480 @@
+# Define custom tool
+calculator = Tool(
+ name="Calculator",
+ func=lambda x: eval(x),
+ description="Useful for math calculations. Input: valid Python expression."
+)
</file context>
| # Define tools | ||
| def calculator(expression: str) -> str: | ||
| """Evaluate a math expression.""" | ||
| return str(eval(expression)) |
There was a problem hiding this comment.
P0: Using eval() on user input is a critical security vulnerability that allows arbitrary code execution. Documentation examples should demonstrate safe patterns since developers often copy them directly. Consider using a safer alternative like ast.literal_eval() for numeric literals, or a library like simpleeval for safe math expression evaluation.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At cli-tool/components/skills/ai-research/agents-langchain/references/agents.md, line 25:
<comment>Using `eval()` on user input is a critical security vulnerability that allows arbitrary code execution. Documentation examples should demonstrate safe patterns since developers often copy them directly. Consider using a safer alternative like `ast.literal_eval()` for numeric literals, or a library like `simpleeval` for safe math expression evaluation.</comment>
<file context>
@@ -0,0 +1,499 @@
+# Define tools
+def calculator(expression: str) -> str:
+ """Evaluate a math expression."""
+ return str(eval(expression))
+
+def search(query: str) -> str:
</file context>
|
|
||
| def _run(self, expression: str) -> str: | ||
| try: | ||
| result = eval(expression) |
There was a problem hiding this comment.
P1: Security: Using eval() on untrusted input enables arbitrary code execution. In AI agent tools, inputs can come from LLMs (which may be prompt-injected) or external sources. Consider using a safe math expression parser like simpleeval or numexpr, or at minimum add a security warning about this being an unsafe example.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At cli-tool/components/skills/ai-research/agents-crewai/SKILL.md, line 241:
<comment>Security: Using `eval()` on untrusted input enables arbitrary code execution. In AI agent tools, inputs can come from LLMs (which may be prompt-injected) or external sources. Consider using a safe math expression parser like `simpleeval` or `numexpr`, or at minimum add a security warning about this being an unsafe example.</comment>
<file context>
@@ -0,0 +1,498 @@
+
+ def _run(self, expression: str) -> str:
+ try:
+ result = eval(expression)
+ return f"Result: {result}"
+ except Exception as e:
</file context>
| if stderr.read(): | ||
| print(f"Error: {stderr.read().decode()}") |
There was a problem hiding this comment.
P1: Bug: stderr.read() is called twice. The first call consumes the stream, so the second call inside the print will always return empty bytes. Store the result in a variable first.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At cli-tool/components/skills/ai-research/infrastructure-lambda-labs/references/advanced-usage.md, line 240:
<comment>Bug: `stderr.read()` is called twice. The first call consumes the stream, so the second call inside the print will always return empty bytes. Store the result in a variable first.</comment>
<file context>
@@ -0,0 +1,611 @@
+ for cmd in commands:
+ stdin, stdout, stderr = client.exec_command(cmd)
+ print(stdout.read().decode())
+ if stderr.read():
+ print(f"Error: {stderr.read().decode()}")
+
</file context>
| if stderr.read(): | |
| print(f"Error: {stderr.read().decode()}") | |
| err = stderr.read() | |
| if err: | |
| print(f"Error: {err.decode()}") |
|
|
||
| # Prepare for distributed (automatic device placement) | ||
| model = train.torch.prepare_model(model) | ||
| dataloader = train.torch.prepare_data_loader(dataloader) |
There was a problem hiding this comment.
P1: The dataloader variable is used but never defined within train_func. In Ray Train, the training function runs on workers without access to the outer scope, so dataloader needs to be created inside the function or passed via config.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At cli-tool/components/skills/ai-research/distributed-training-ray-train/SKILL.md, line 99:
<comment>The `dataloader` variable is used but never defined within `train_func`. In Ray Train, the training function runs on workers without access to the outer scope, so `dataloader` needs to be created inside the function or passed via `config`.</comment>
<file context>
@@ -0,0 +1,406 @@
+
+ # Prepare for distributed (automatic device placement)
+ model = train.torch.prepare_model(model)
+ dataloader = train.torch.prepare_data_loader(dataloader)
+
+ for epoch in range(epochs):
</file context>
| # Create agent with retriever tool | ||
| agent = create_tool_calling_agent( | ||
| llm=llm, | ||
| tools=[retriever_tool, calculator, search], |
There was a problem hiding this comment.
P2: Undefined variables calculator and search in code example. These tools are referenced but never defined or imported, which would cause a NameError if users copy this code. Either define these tools or use only the defined retriever_tool.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At cli-tool/components/skills/ai-research/agents-langchain/references/rag.md, line 359:
<comment>Undefined variables `calculator` and `search` in code example. These tools are referenced but never defined or imported, which would cause a `NameError` if users copy this code. Either define these tools or use only the defined `retriever_tool`.</comment>
<file context>
@@ -0,0 +1,600 @@
+# Create agent with retriever tool
+agent = create_tool_calling_agent(
+ llm=llm,
+ tools=[retriever_tool, calculator, search],
+ system_prompt="Use python_docs tool for Python questions"
+)
</file context>
| **Example slopes for 8 heads**: | ||
| ```python | ||
| slopes = get_alibi_slopes(8) | ||
| # Output: [0.0625, 0.125, 0.25, 0.5, 1.0, 2.0, 4.0, 8.0] |
There was a problem hiding this comment.
P2: Incorrect example output comment. The get_slopes_power_of_2(8) function actually produces [0.5, 0.25, 0.125, 0.0625, 0.03125, 0.015625, 0.0078125, 0.00390625] (decreasing values < 1), not the claimed [0.0625, 0.125, 0.25, 0.5, 1.0, 2.0, 4.0, 8.0]. This misleading documentation could confuse readers learning about ALiBi slopes.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At cli-tool/components/skills/ai-research/emerging-techniques-long-context/references/extension_methods.md, line 277:
<comment>Incorrect example output comment. The `get_slopes_power_of_2(8)` function actually produces `[0.5, 0.25, 0.125, 0.0625, 0.03125, 0.015625, 0.0078125, 0.00390625]` (decreasing values < 1), not the claimed `[0.0625, 0.125, 0.25, 0.5, 1.0, 2.0, 4.0, 8.0]`. This misleading documentation could confuse readers learning about ALiBi slopes.</comment>
<file context>
@@ -0,0 +1,468 @@
+**Example slopes for 8 heads**:
+```python
+slopes = get_alibi_slopes(8)
+# Output: [0.0625, 0.125, 0.25, 0.5, 1.0, 2.0, 4.0, 8.0]
+
+# Each head has different slope
</file context>
| # Output: [0.0625, 0.125, 0.25, 0.5, 1.0, 2.0, 4.0, 8.0] | |
| # Output: [0.5, 0.25, 0.125, 0.0625, 0.03125, 0.015625, 0.0078125, 0.00390625] |
| for char in expression: | ||
| if char.isalpha(): | ||
| return "Error: Letters not allowed" | ||
| result = eval(expression) |
There was a problem hiding this comment.
P2: Using eval() with user input is dangerous, and the "safety check" (blocking letters) is insufficient. This documentation example teaches an insecure pattern that developers may copy. Consider using a safer alternative like the simpleeval library or ast.literal_eval() for simple cases, and update the comment to not imply the current approach is secure.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At cli-tool/components/skills/ai-research/agents-crewai/references/tools.md, line 297:
<comment>Using `eval()` with user input is dangerous, and the "safety check" (blocking letters) is insufficient. This documentation example teaches an insecure pattern that developers may copy. Consider using a safer alternative like the `simpleeval` library or `ast.literal_eval()` for simple cases, and update the comment to not imply the current approach is secure.</comment>
<file context>
@@ -0,0 +1,429 @@
+ for char in expression:
+ if char.isalpha():
+ return "Error: Letters not allowed"
+ result = eval(expression)
+ return f"Result: {result}"
+ except Exception as e:
</file context>
| # Concatenate multiple documents | ||
| buffer = tokens | ||
| while len(buffer) < target_length: | ||
| next_example = next(dataset) |
There was a problem hiding this comment.
P2: next(dataset) will raise TypeError because HuggingFace Dataset objects are not iterators. This needs to be refactored to use an iterator or index-based access.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At cli-tool/components/skills/ai-research/emerging-techniques-long-context/references/fine_tuning.md, line 64:
<comment>`next(dataset)` will raise `TypeError` because HuggingFace Dataset objects are not iterators. This needs to be refactored to use an iterator or index-based access.</comment>
<file context>
@@ -0,0 +1,611 @@
+ # Concatenate multiple documents
+ buffer = tokens
+ while len(buffer) < target_length:
+ next_example = next(dataset)
+ buffer.extend(tokenizer.encode(next_example['text']))
+
</file context>
| --max-num-seqs 512 \ | ||
| --gpu-memory-utilization 0.95 \ | ||
| --enable-prefix-caching \ | ||
| --trust-remote-code |
There was a problem hiding this comment.
P2: The --trust-remote-code flag allows execution of arbitrary Python code from model repositories, which is a security risk in production. This documentation example should include a warning about its security implications or only recommend it for trusted model sources.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At cli-tool/components/skills/ai-research/inference-serving-vllm/references/server-deployment.md, line 201:
<comment>The `--trust-remote-code` flag allows execution of arbitrary Python code from model repositories, which is a security risk in production. This documentation example should include a warning about its security implications or only recommend it for trusted model sources.</comment>
<file context>
@@ -0,0 +1,255 @@
+ --max-num-seqs 512 \
+ --gpu-memory-utilization 0.95 \
+ --enable-prefix-caching \
+ --trust-remote-code
+```
+
</file context>
| --trust-remote-code | |
| --trust-remote-code # WARNING: Only use with trusted model sources - allows arbitrary code execution |
- Resolved conflicts in docs/components.json - Regenerated components.json to include both: - 75 new AI research skills from PR - 4 new Z.AI MCPs from main - GLM model version update - Website URL migration to aitmpl.com Total skills now: 339 (was 264) Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
|
Added comprehensive AI research engineering skills from the
zechenzhangAGI/AI-research-SKILLs repository covering:
Categories added (19):
Total: 75 new skills added to cli-tool/components/skills/ai-research/
Components catalog updated: 259 → 334 skills
Source: https://github.com/zechenzhangAGI/AI-research-SKILLs
License: MIT
Author: Orchestra Research
Summary by cubic
Added 75 AI research engineering skill components to expand the catalog to 339 total skills. This adds comprehensive coverage across agents, distributed training, data processing, RAG, inference, optimization, ops, and emerging techniques.
Written for commit be8e505. Summary will update on new commits.