Merge pull request #6 from algolia/fix/algolia-cli-validated-against-real-cli

fix: validate algolia-cli skill against real CLI

Author: leonardocuco

skills/algolia-cli/SKILL.md

@@ -85,11 +85,11 @@ Run `/algolia-cli:setup` to install the CLI and configure a profile, or follow [
 |----------------------------|---------------------------------------------------------------------|
 | Browse all records         | `algolia objects browse <index>`                                    |
 | Browse specific attributes | `algolia objects browse <index> --attributesToRetrieve title,price` |
-| Import records from file   | `algolia objects import <index> -F data.ndjson -y`                  |
-| Import from stdin          | `cat data.ndjson \| algolia objects import <index> -F - -y`         |
+| Import records from file   | `algolia objects import <index> -F data.ndjson`                     |
+| Import from stdin          | `cat data.ndjson \| algolia objects import <index> -F -`            |
 | Delete by IDs              | `algolia objects delete <index> --object-ids id1,id2 -y`            |
 | Delete by filter           | `algolia objects delete <index> --filters "type:obsolete" -y`       |
-| Partial update             | `algolia objects update <index> -F updates.ndjson -y`               |
+| Partial update             | `algolia objects update <index> -F updates.ndjson`                  |
 
 ### Settings
 
@@ -132,7 +132,7 @@ Choosing the right synonym type matters for search quality:
 
 ## Key Conventions
 
-1. **Always use non-interactive mode.** Every command that writes, deletes, or modifies data needs `-y` (or `--confirm`) to skip confirmation prompts. This includes `objects import`, `objects delete`, `objects update`, `indices delete/clear/copy/move`, `rules import/delete`, `synonyms import/delete`, and `apikeys delete`. Without `-y`, the CLI will hang waiting for user input.
+1. **Always use non-interactive mode.** Destructive commands need `-y` (or `--confirm`) to skip confirmation prompts. This includes `objects delete`, `indices delete/clear/copy/move`, `rules import/delete`, `synonyms delete`, and `apikeys delete`. Without `-y`, the CLI will hang waiting for user input. Note: `objects import`, `objects update`, and `synonyms import` do **not** have a `-y` flag — they run non-interactively by default.
 2. **ndjson format.** `objects browse`, `objects import`, `rules browse/import`, and `synonyms browse/import` use newline-delimited JSON (one JSON object per line), **not** JSON arrays.
 3. **Profile flag.** Use `-p <profile>` to target a non-default profile. Omit it to use the default.
 4. **Credential precedence.** Environment variables override all other configuration. The resolution order is: **env vars** > **CLI flags** (`--application-id`, `--api-key`) > **profile config file** > **default profile**. Supported env vars: `ALGOLIA_APPLICATION_ID`, `ALGOLIA_API_KEY`, `ALGOLIA_ADMIN_API_KEY`, `ALGOLIA_SEARCH_HOSTS`, `ALGOLIA_CRAWLER_USER_ID`, `ALGOLIA_CRAWLER_API_KEY`. If env vars are set, `--profile`/`-p` is ignored for those credentials.
@@ -145,7 +145,7 @@ Choosing the right synonym type matters for search quality:
 ### Migrate records between indices (with field filtering)
 ```bash
 algolia objects browse SOURCE --attributesToRetrieve objectID,title,price \
-  | algolia objects import DEST -F - -y -w
+  | algolia objects import DEST -F - -w
 ```
 
 ### Full index backup
@@ -159,10 +159,10 @@ Note: settings use `.json` (standard JSON), everything else uses `.ndjson` (newl
 
 ### Restore from backup
 ```bash
-algolia objects import MY_INDEX -F my_index_records.ndjson -y -w
-algolia settings import MY_INDEX -F my_index_settings.json -y -w
+algolia objects import MY_INDEX -F my_index_records.ndjson -w
+algolia settings import MY_INDEX -F my_index_settings.json -w
 algolia rules import MY_INDEX -F my_index_rules.ndjson -c -y -w
-algolia synonyms import MY_INDEX -F my_index_synonyms.ndjson -r -y -w
+algolia synonyms import MY_INDEX -F my_index_synonyms.ndjson -r -w
 ```
 
 ## Direct Invocation

skills/algolia-cli/evals/EVAL_RESULTS.md

@@ -26,10 +26,10 @@ The skill was evaluated across 3 realistic user scenarios, comparing **with-skil
 | Uses `--attributesToRetrieve` for field selection | FAIL | PASS | PASS |
 | Pipes browse to import with `-F -` | FAIL | PASS | PASS |
 | Uses `--attributesForFaceting` for category facet | PASS | PASS | PASS |
-| Includes `-y` flag on write commands | FAIL | FAIL | PASS |
+| Does NOT use `-y` on `objects import`/`settings set` (unsupported) | PASS | PASS | PASS |
 | Uses `-w` flag to sequence import before settings | FAIL | PASS | PASS |
 
-**Key finding:** Without the skill, Claude exported all fields and filtered with `jq` instead of using `--attributesToRetrieve`. It also used intermediate files instead of piping. The v1→v2 improvement fixed the missing `-y` flag by adding it to the quick reference tables.
+**Key finding:** Without the skill, Claude exported all fields and filtered with `jq` instead of using `--attributesToRetrieve`. It also used intermediate files instead of piping. Note: `objects import` and `settings set` do not support `-y` — they run non-interactively by default.
 
 ### Eval 2: Synonyms and Rules
 
@@ -66,7 +66,7 @@ The skill was evaluated across 3 realistic user scenarios, comparing **with-skil
 The biggest areas where the skill outperforms general knowledge:
 
 1. **ndjson format awareness** — Knowing which commands use newline-delimited JSON vs standard JSON
-2. **Non-interactive flags** — Consistently using `-y` to prevent CLI hangs
+2. **Non-interactive flags** — Knowing which commands need `-y` (destructive ops like `delete`, `clear`, `copy`, `move`, `rules import`) and which don't (`objects import`, `objects update`, `synonyms import`)
 3. **Piping patterns** — Using `-F -` for stdin instead of intermediate files
 4. **Synonym type selection** — Choosing `oneWaySynonym` vs `synonym` based on user intent
 5. **Correct CLI commands** — Preventing hallucinated commands like `algolia rules save --rule`
@@ -86,7 +86,7 @@ Low recall is a systemic limitation — Claude is confident enough in its Algoli
 
 ## Improvements Made (v1 → v2)
 
-1. **Added `-y` flag** to all write commands in quick reference tables
+1. **Corrected `-y` flag usage** — only on commands that actually support it (destructive operations), removed from `objects import`/`objects update`
 2. **New Synonym Type Guide** with decision rules for `synonym` vs `oneWaySynonym`
 3. **Expanded non-interactive convention** listing every command that needs `-y`
 4. **New Common Workflows section** (migrate, backup, restore patterns)

skills/algolia-cli/evals/evals.json

@@ -10,7 +10,7 @@
         "Uses 'algolia objects browse' with --attributesToRetrieve to select only title, price, and category",
         "Pipes the browse output to 'algolia objects import' using -F - for stdin",
         "Uses 'algolia settings set' with --attributesForFaceting to add 'category' as a facet",
-        "Includes -y flag on commands that require confirmation",
+        "Does NOT include -y flag on 'objects import' or 'settings set' (these commands do not support it)",
         "Uses -w (wait) flag on import before running the settings command, or chains them appropriately"
       ]
     },