experimental rules in run all mode; more explicit gates/ask-user

Author: GrosQuildu

plugins/static-analysis/skills/codeql/SKILL.md

@@ -4,8 +4,8 @@ description: >-
   Scans a codebase for security vulnerabilities using CodeQL's interprocedural data flow and
   taint tracking analysis. Triggers on "run codeql", "codeql scan", "codeql analysis", "build
   codeql database", or "find vulnerabilities with codeql". Supports "run all" (security-and-quality
-  suite) and "important only" (high-precision security findings) scan modes. Also handles
-  creating data extension models and processing CodeQL SARIF output.
+  + security-experimental suites) and "important only" (high-precision security findings) scan
+  modes. Also handles creating data extension models and processing CodeQL SARIF output.
 allowed-tools:
   - Bash
   - Read
@@ -155,6 +155,7 @@ Then execute the full pipeline: **build database → create data extensions →
 These shortcuts lead to missed findings. Do not accept them:
 
 - **"security-extended is enough"** - It is the baseline. Always check if Trail of Bits packs and Community Packs are available for the language. They catch categories `security-extended` misses entirely.
+- **"security-and-quality is the broadest suite"** - `security-and-quality` excludes all `experimental/` query paths. For run-all mode, import both `security-and-quality` and `security-experimental`. The delta is 1–52 queries depending on the language.
 - **"The database built, so it's good"** - A database that builds does not mean it extracted well. Always run quality assessment and check file counts against expected source files.
 - **"Data extensions aren't needed for standard frameworks"** - Even Django/Spring apps have custom wrappers that CodeQL does not model. Skipping extensions means missing vulnerabilities.
 - **"build-mode=none is fine for compiled languages"** - It produces severely incomplete analysis. Only use as an absolute last resort. On macOS, try the arm64 toolchain workaround or Rosetta first.

plugins/static-analysis/skills/codeql/references/ruleset-catalog.md

@@ -5,8 +5,10 @@
 | Suite | False Positives | Use Case |
 |-------|-----------------|----------|
 | `security-extended` | Low | **Default** - Security audits |
-| `security-and-quality` | Medium | Comprehensive review |
-| `security-experimental` | Higher | Research, vulnerability hunting |
+| `security-and-quality` | Medium | Comprehensive review (stable security + code quality) |
+| `security-experimental` | Higher | Research, vulnerability hunting (stable security + experimental security) |
+
+> **Suite hierarchy:** `security-and-quality` and `security-experimental` are complementary. `security-and-quality` excludes `experimental/` query paths. `security-experimental` includes them but excludes code quality queries. For maximum coverage (run-all mode), import both.
 
 **Usage:** `codeql/<lang>-queries:codeql-suites/<lang>-security-extended.qls`
 

plugins/static-analysis/skills/codeql/references/run-all-suite.md

@@ -6,17 +6,23 @@ In run-all mode, generate a custom `.qls` query suite file at runtime. This ensu
 
 When you pass a pack name directly to `codeql database analyze` (e.g., `-- codeql/cpp-queries`), CodeQL uses the pack's `defaultSuiteFile` field from `qlpack.yml`. For official packs, this is typically `codeql-suites/<lang>-code-scanning.qls`, which applies strict precision and severity filters. This silently drops many queries and can produce zero results for small codebases.
 
-The run-all suite explicitly references the broadest built-in suite (`security-and-quality`) for official packs and loads third-party packs with minimal filtering.
+The run-all suite explicitly imports both `security-and-quality` and `security-experimental` from official packs, plus third-party packs with minimal filtering.
+
+> **Why both suites?** `security-and-quality` = stable security + code quality (excludes `experimental/` paths). `security-experimental` = stable security + experimental security (re-includes `experimental/` paths tagged `security`). They are complementary — importing both is safe since CodeQL deduplicates shared queries automatically.
 
 ## Suite Template
 
 Generate this file as `run-all.qls` in the results directory before running analysis:
 
 ```yaml
-- description: Run-all — all security and quality queries from all installed packs
-# Official queries: use security-and-quality suite (broadest built-in suite)
+- description: Run-all — all security, experimental, and quality queries from all installed packs
+# Official queries: import BOTH suites (they are complementary, not hierarchical)
+# security-and-quality = stable security + code quality (excludes experimental/ paths)
+# security-experimental = stable security + experimental security (re-includes experimental/ with security tag)
 - import: codeql-suites/<CODEQL_LANG>-security-and-quality.qls
   from: codeql/<CODEQL_LANG>-queries
+- import: codeql-suites/<CODEQL_LANG>-security-experimental.qls
+  from: codeql/<CODEQL_LANG>-queries
 # Third-party packs (include only if installed, one entry per pack)
 # - queries: .
 #   from: trailofbits/<CODEQL_LANG>-queries
@@ -45,9 +51,11 @@ SUITE_FILE="$RAW_DIR/run-all.qls"
 # NOTE: INSTALLED_THIRD_PARTY_PACKS must be a space-separated list of pack names
 
 cat > "$SUITE_FILE" << HEADER
-- description: Run-all — all security and quality queries from all installed packs
+- description: Run-all — all security, experimental, and quality queries from all installed packs
 - import: codeql-suites/${CODEQL_LANG}-security-and-quality.qls
   from: codeql/${CODEQL_LANG}-queries
+- import: codeql-suites/${CODEQL_LANG}-security-experimental.qls
+  from: codeql/${CODEQL_LANG}-queries
 HEADER
 
 # Add each installed third-party pack
@@ -86,7 +94,7 @@ echo "Suite generated: $SUITE_FILE"
 
 | Aspect | Run all | Important only |
 |--------|---------|----------------|
-| Official pack suite | `security-and-quality` (all security + code quality) | All queries loaded, filtered by precision |
+| Official pack suites | `security-and-quality` + `security-experimental` (stable security + code quality + experimental security) | All queries loaded, filtered by precision |
 | Third-party packs | All `problem`/`path-problem` queries | Only `security`-tagged queries with precision metadata |
 | Precision filter | None | high/very-high always; medium only if security-severity >= 6.0 |
 | Post-analysis filter | None | Drops medium-precision results with security-severity < 6.0 |

plugins/static-analysis/skills/codeql/workflows/run-analysis.md

@@ -8,7 +8,7 @@ Two modes control analysis scope. Both use all installed packs — the differenc
 
 | Mode | Description | Suite Reference |
 |------|-------------|-----------------|
-| **Run all** | All queries from all installed packs via `security-and-quality` suite | [run-all-suite.md](../references/run-all-suite.md) |
+| **Run all** | All queries from all installed packs via `security-and-quality` + `security-experimental` suites | [run-all-suite.md](../references/run-all-suite.md) |
 | **Important only** | Security queries filtered by precision and security-severity threshold | [important-only-suite.md](../references/important-only-suite.md) |
 
 > **WARNING:** Do NOT pass pack names directly to `codeql database analyze` (e.g., `-- codeql/cpp-queries`). Each pack's `defaultSuiteFile` silently applies strict filters and can produce zero results. Always use an explicit suite reference.
@@ -31,10 +31,11 @@ TaskCreate: "Process and report results" (Step 5) - blockedBy: Step 4
 
 | Task | Gate Type | Cannot Proceed Until |
 |------|-----------|---------------------|
-| Step 2 | **SOFT GATE** | User selects mode; confirms installed/ignored for each missing pack |
-| Step 3 | **SOFT GATE** | User approves query packs, model packs, and threat model selection |
+| Step 2a | **SOFT GATE** | User selects scan mode. Skip only if user said "run all" or "important only" verbatim. |
+| Step 3a | **HARD GATE** | User confirms query pack selection. Always ask — no auto-skip. |
+| Step 3c | **HARD GATE** | User selects threat model. Always ask — no auto-skip. |
 
-**Auto-skip rule:** If the user already specified a choice in the invocation, skip the corresponding `AskUserQuestion` and use the provided value directly.
+**Auto-skip rules are per-gate.** Each gate documents its own skip condition. Choosing "full scan" or "run all" satisfies the scan mode gate (2a) but does not satisfy pack confirmation (3a) or threat model selection (3c).
 
 ---
 
@@ -94,7 +95,7 @@ If multi-language database, ask which language to analyze.
 
 #### 2a: Select Scan Mode
 
-**Skip if user already specified.** Otherwise use `AskUserQuestion`:
+**Skip only if user said "run all" or "important only" in their prompt.** "Full scan", "scan", or "analyze" do NOT count — ask.
 
 ```
 header: "Scan Mode"
@@ -139,13 +140,13 @@ Record all detected packs for Step 3.
 **Exit:** User confirmed query packs, model packs, and threat model selection; all flags built (`THREAT_MODEL_FLAG`, `MODEL_PACK_FLAGS`, `ADDITIONAL_PACK_FLAGS`)
 
 > **CHECKPOINT** — Present available packs to user for confirmation.
-> **Skip if user already specified pack preferences.**
+> **Always ask. Do not auto-skip.**
 
 #### 3a: Confirm Query Packs
 
 **Important-only mode:** Inform user all installed packs included with filtering. Proceed to 3b.
 
-**Run-all mode:** Use `AskUserQuestion` to confirm "Use all" or "Select individually".
+**Run-all mode:** Use `AskUserQuestion` to confirm "Use all" or "Select individually". Always ask — the user needs to see which packs will run.
 
 #### 3b: Select Model Packs (if any detected)
 
@@ -162,7 +163,7 @@ Use `AskUserQuestion`: "Use all (Recommended)" / "Select individually" / "Skip".
 
 Threat models control which input sources CodeQL treats as tainted. See [threat-models.md](../references/threat-models.md).
 
-Use `AskUserQuestion`:
+**Always ask.** Do not default to "remote only" without user confirmation. Use `AskUserQuestion`:
 
 ```
 header: "Threat Models"