Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
357 changes: 357 additions & 0 deletions docs/skill-interaction.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,357 @@
# Skill Interaction Methods

`OVOSSkill` provides two high-level methods for collecting structured user input: `ask_yesno` for binary yes/no questions and `ask_selection` for multiple-choice prompts. Both are backed by pluggable agent engines that can be swapped per-skill or system-wide.

---

## ask_yesno

```python
OVOSSkill.ask_yesno(prompt: str, data: Optional[dict] = None) -> Optional[str]
```

Speaks *prompt*, waits for the user's response, and classifies it as `"yes"`, `"no"`, or the raw response string if neither matched.

**Source**: `OVOSSkill.ask_yesno` — `ovos_workshop/skills/ovos.py`

### Parameters

| Parameter | Type | Description |
|-----------|------|-------------|
| `prompt` | `str` | Dialog ID (looked up in `locale/`) or a literal string to speak. |
| `data` | `dict \| None` | Template variables for Mustache rendering of the dialog string. |

### Return values

| Value | Meaning |
|-------|---------|
| `"yes"` | User confirmed (e.g. "yeah", "sure", "of course") |
| `"no"` | User declined (e.g. "nope", "nah", "definitely not") |
| `str` | User spoke something that could not be classified — raw transcript returned |
| `None` | No response received (timeout or user said nothing) |

### Basic usage

```python
class MySkill(OVOSSkill):
def handle_delete_intent(self, message):
if self.ask_yesno("confirm_delete") == "yes":
self._do_delete()
self.speak_dialog("deleted")
else:
self.speak_dialog("cancelled")
```

`locale/en-us/confirm_delete.dialog`:
```
Are you sure you want to delete this?
Do you really want to delete it?
```

### With template data

```python
answer = self.ask_yesno("confirm_action", data={"action": "restart the server"})
```

`locale/en-us/confirm_action.dialog`:
```
Are you sure you want to {{action}}?
```

### Handling all return values

```python
answer = self.ask_yesno("do_you_want_music")
if answer == "yes":
self.play_music()
elif answer == "no":
self.speak_dialog("okay_nevermind")
elif answer is None:
self.speak_dialog("no_response")
else:
# answer is the raw transcript — user said something unexpected
self.speak_dialog("did_not_understand")
```

### How it works internally

1. `get_response(dialog=prompt, data=data)` — speaks the prompt, records user reply.
2. `_get_yesno_engine()` — loads the configured `YesNoEngine` plugin (if any).
3. If a plugin is loaded: `engine.yes_or_no(question=prompt, response=resp, lang=self.lang)` → `True`, `False`, or `None`.
4. If no plugin: `YesNoSolver().match_yes_or_no(resp, lang=self.lang)` (built-in fallback, always available).
5. `True` → `"yes"`, `False` → `"no"`, `None`/unmatched → raw response.

---

## ask_selection

```python
OVOSSkill.ask_selection(
options: List[str],
dialog: str = '',
data: Optional[dict] = None,
min_conf: float = 0.65,
numeric: bool = False,
num_retries: int = -1,
) -> Optional[str]
```

Speaks the options list to the user, optionally follows with a dialog prompt, then resolves the user's spoken response to one of the options.

**Source**: `OVOSSkill.ask_selection` — `ovos_workshop/skills/ovos.py`

### Parameters

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `options` | `List[str]` | — | The predefined options to offer. |
| `dialog` | `str` | `''` | Dialog ID or literal string spoken **after** the options list. |
| `data` | `dict \| None` | `None` | Template variables for the dialog string. |
| `min_conf` | `float` | `0.65` | Minimum fuzzy-match confidence for the default plugin. Passed to the `OptionMatcherEngine` config if no plugin-level config overrides it. |
| `numeric` | `bool` | `False` | If `True`, speaks each option prefixed with its number ("one, pizza; two, pasta; …"). If `False`, speaks them as a joined list ("pizza, pasta, or salad?"). |
| `num_retries` | `int` | `-1` | How many times to re-prompt on no response. `-1` means use the system default. |

Comment on lines +104 to +114
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Document that min_conf parameter is currently unused.

The min_conf parameter is documented (line 111) but is not used in the current implementation since the legacy match_one was removed. Update the documentation to clarify this is passed to the engine configuration or mark it as deprecated.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/skill-interaction.md` around lines 104 - 114, The docs claim the
min_conf parameter is applied but the implementation no longer uses it directly;
update the documentation for the options parameter table to state that min_conf
is currently unused in the high-level API (or is forwarded only to the
OptionMatcherEngine configuration if applicable) or mark min_conf as deprecated.
Specifically, change the `min_conf` row to indicate its current behavior (e.g.,
"currently unused in the high-level selection flow; only used when configuring
OptionMatcherEngine" or "deprecated") and mention the OptionMatcherEngine symbol
so readers can find where to set/override this value.

### Return values

| Value | Meaning |
|-------|---------|
| `str` | One of the strings from `options`, exactly as provided. |
| `None` | No match, no response, or plugin failure. |

Special cases handled before user interaction:
- Empty `options` → `None` immediately.
- Single-element `options` → returns that element immediately (no prompt).

### Basic usage

```python
class MySkill(OVOSSkill):
def handle_transport_intent(self, message):
modes = ["bus", "train", "bicycle"]
choice = self.ask_selection(modes, dialog="which_transport")
if choice:
self.speak_dialog("you_chose", {"mode": choice})
```

`locale/en-us/which_transport.dialog`:
```
Which would you prefer?
How would you like to travel?
```

The skill speaks: *"bus, train, or bicycle? Which would you prefer?"*

The user can say:
- `"train"` — fuzzy-matched directly
- `"the second one"` / `"number two"` / `"two"` — position matched
- `"the last one"` — last-word matched
- `"option 3"` — numeric matched (requires `ovos-number-parser`)

### Numeric menu

```python
choice = self.ask_selection(options, numeric=True)
```

Speaks each option as: *"one, bus; two, train; three, bicycle"*. Useful when options are long or ambiguous. The user can then say *"two"* or *"the second one"*.

### Handling None

```python
choice = self.ask_selection(options, dialog="which_one", num_retries=1)
if choice is None:
self.speak_dialog("could_not_understand")
return
```

### How it works internally

1. Validates `options` (raises `ValueError` if not a list; returns immediately for 0 or 1 items).
2. Speaks options (as list or numbered menu based on `numeric`).
3. `get_response(dialog=dialog, data=data, num_retries=num_retries)` — speaks optional follow-up dialog, records reply.
4. `_get_selection_engine()` — loads the configured `OptionMatcherEngine` plugin.
5. `engine.match_option(utterance=resp, options=options, lang=self.lang)` — resolves response to a slot.
6. If the engine raises or returns `None`, `ask_selection` returns `None`.

---

## Plugin system

Both methods are backed by pluggable agent engines discovered and loaded via [ovos-plugin-manager](https://github.com/OpenVoiceOS/ovos-plugin-manager).

### ask_yesno — YesNoEngine

**Plugin type**: `YesNoEngine` (`opm.agents.yesno`)
**Config key**: `ask_yesno_plugin`
**Built-in fallback**: `ovos-solver-yes-no-plugin` (always available, no config needed)

`YesNoEngine` plugins implement:

```python
def yes_or_no(self, question: str, response: str, lang: Optional[str] = None) -> Optional[bool]:
... # True = yes, False = no, None = unclear
```

The `question` argument gives the plugin context about what was asked, enabling smarter inference (e.g. an LLM-backed plugin could use it to resolve ambiguous answers).

**Available plugins**:

| Plugin | Description |
|--------|-------------|
| `ovos-solver-yes-no-plugin` | Rule-based multilingual yes/no classifier (default fallback) |

### ask_selection — OptionMatcherEngine

**Plugin type**: `OptionMatcherEngine` (`opm.agents.option_matcher`)
**Config key**: `ask_selection_plugin`
**Default plugin**: `ovos-option-matcher-fuzzy-plugin` (installed as a dependency of `ovos-workshop`)

`OptionMatcherEngine` plugins implement:

```python
def match_option(self, utterance: str, options: List[str], lang: Optional[str] = None) -> Optional[str]:
... # returns one of options, or None
```

**Available plugins**:

| Plugin | Description |
|--------|-------------|
| `ovos-option-matcher-fuzzy-plugin` | Fuzzy + ordinal/cardinal vocab + numeric fallback. Default. |

---

## Configuration

### Global defaults — `mycroft.conf`

```json
{
"skills": {
"ask_yesno_plugin": "ovos-solver-yes-no-plugin",
"ask_selection_plugin": "ovos-option-matcher-fuzzy-plugin"
}
}
```

`ask_yesno_plugin` defaults to `None` (built-in `YesNoSolver` used). `ask_selection_plugin` defaults to `ovos-option-matcher-fuzzy-plugin`.

### Per-skill override — `settings.json`

Place in the skill's `settings.json` to override for that skill only:

```json
{
"ask_yesno_plugin": "my-llm-yesno-plugin",
"ask_selection_plugin": "my-embedding-option-matcher"
}
```

### Plugin config

Pass configuration to the plugin via the same settings block:

```json
{
"ask_selection_plugin": "ovos-option-matcher-fuzzy-plugin",
"ask_selection_plugin_config": {
"min_conf": 0.80
}
}
```

### Priority order

```
settings.json > mycroft.conf skills block > built-in default
```

Plugins are loaded lazily on first use and cached per plugin name for the lifetime of the skill instance.

---

## Writing a custom YesNoEngine plugin

```python
# my_yesno/__init__.py
from typing import Optional
from ovos_plugin_manager.templates.agents import YesNoEngine

class MyYesNoPlugin(YesNoEngine):
def yes_or_no(self, question: str, response: str,
lang: Optional[str] = None) -> Optional[bool]:
r = response.lower()
if "yes" in r or "sure" in r:
return True
if "no" in r or "never" in r:
return False
return None # unclear
```

```toml
# pyproject.toml
[project.entry-points."opm.agents.yesno"]
my-yesno-plugin = "my_yesno:MyYesNoPlugin"
```

Activate for a skill:

```json
{ "ask_yesno_plugin": "my-yesno-plugin" }
```

---

## Writing a custom OptionMatcherEngine plugin

```python
# my_matcher/__init__.py
from typing import List, Optional
from ovos_plugin_manager.templates.agents import OptionMatcherEngine

class MyOptionMatcher(OptionMatcherEngine):
def match_option(self, utterance: str, options: List[str],
lang: Optional[str] = None) -> Optional[str]:
# example: embedding similarity via a local model
scores = self._embed_and_score(utterance, options)
best_idx = max(range(len(scores)), key=lambda i: scores[i])
if scores[best_idx] >= self.config.get("min_conf", 0.6):
return options[best_idx]
return None
```

```toml
# pyproject.toml
[project.entry-points."opm.agents.option_matcher"]
my-option-matcher-plugin = "my_matcher:MyOptionMatcher"
```

Activate system-wide:

```json
{ "skills": { "ask_selection_plugin": "my-option-matcher-plugin" } }
```

---

## Failure behaviour

| Scenario | ask_yesno result | ask_selection result |
|----------|-----------------|---------------------|
| User says nothing (timeout) | `None` | `None` |
| User response unclassifiable | raw transcript string | `None` |
| Plugin fails to load | falls back to `YesNoSolver` | `None` |
| Plugin raises at runtime | falls back to `YesNoSolver` | `None` |
| No plugin configured | `YesNoSolver` used | default fuzzy plugin used |

`ask_selection` is intentionally strict: any failure returns `None` rather than guessing. Always handle the `None` case in your skill.

Comment on lines +340 to +349
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify ask_yesno exception handling
rg -n -A 10 "def ask_yesno" ovos_workshop/skills/ovos.py | head -30

Repository: OpenVoiceOS/ovos-workshop

Length of output: 693


🏁 Script executed:

#!/bin/bash
# Get complete ask_yesno method implementation
rg -n -A 25 "def ask_yesno" ovos_workshop/skills/ovos.py

Repository: OpenVoiceOS/ovos-workshop

Length of output: 1405


🏁 Script executed:

#!/bin/bash
# Check the documentation file at the specified lines
sed -n '335,355p' docs/skill-interaction.md

Repository: OpenVoiceOS/ovos-workshop

Length of output: 1030


🏁 Script executed:

#!/bin/bash
# Search for any try/except blocks in ask_yesno or related plugin handling
rg -n -B 5 -A 10 "yes_or_no\|YesNoSolver" ovos_workshop/skills/ovos.py

Repository: OpenVoiceOS/ovos-workshop

Length of output: 51


Documentation states fallback behavior that doesn't match code.

The documentation at lines 340-349 claims "Plugin raises at runtime → falls back to YesNoSolver" for ask_yesno, but the actual implementation at ovos_workshop/skills/ovos.py lines 1996-1997 calls engine.yes_or_no() without any try/except wrapper. If the engine raises an exception, it will propagate rather than falling back to YesNoSolver.

Update the code to add exception handling around engine.yes_or_no() to match the documented behavior, or update the documentation to reflect that exceptions propagate uncaught.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/skill-interaction.md` around lines 340 - 349, The doc claims ask_yesno
falls back to YesNoSolver if the plugin raises, but the code calls
engine.yes_or_no() directly; wrap the engine.yes_or_no() call inside a
try/except in the ask_yesno implementation so any exception from the engine is
caught, log the exception, and then invoke the fallback YesNoSolver path (same
return behavior as when plugin fails to load) to match the documentation; ensure
you reference engine.yes_or_no(), ask_yesno, and YesNoSolver when making the
change so the fallback is triggered on runtime errors.

---

## See also

- [`ovos-solver-yes-no-plugin`](https://github.com/OpenVoiceOS/ovos-solver-YesNo-plugin) — built-in yes/no classifier
- [`ovos-option-matcher-fuzzy-plugin`](https://github.com/OpenVoiceOS/ovos-option-matcher-fuzzy-plugin) — default selection plugin, with [full docs](https://github.com/OpenVoiceOS/ovos-option-matcher-fuzzy-plugin/tree/master/docs)
- `OVOSSkill.get_response` — lower-level method used internally by both
- [OPM agent templates](https://github.com/OpenVoiceOS/ovos-plugin-manager/blob/dev/ovos_plugin_manager/templates/agents.py) — `YesNoEngine`, `OptionMatcherEngine` base classes
Loading
Loading