Add GPT semantic command stack with guarded execution#198
Open
vovka wants to merge 2 commits intoC-Loftus:mainfrom
Open
Add GPT semantic command stack with guarded execution#198vovka wants to merge 2 commits intoC-Loftus:mainfrom
vovka wants to merge 2 commits intoC-Loftus:mainfrom
Conversation
Owner
|
Thanks for your PR. While I appreciate the work, I am a bit worried on the size of the changes and the amount that appears to be generated from AI. I think in order for me to review this thoroughly and feel comfortable merging, I would need to get buy in from other users in the talon community (i.e. that it seems useful and intuitive enough to warrant the additional code), ensure it works across platforms, and get some more real-world examples. Something like a screen recording demo would be ideal if possible just since the example plans are a bit abstract. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds a new standalone semantic command stack inside
GPT/semanticfortalon-ai-tools.The goal is to let users speak high-level requests (for example, “model semantic open google chrome and search for …”), generate a constrained action plan, preview it, and only execute after explicit confirmation.
Purpose
model semantic ...voice commands with preview/run/cancel/copy/repeat flows.Mechanics: How Semantic Commands Work
model semantic <request>.llmCLI or API endpoint).steps+ optionalsummary).run plan,cancel plan,copy plan, orrepeat last.run plan, executor runs steps in order with focus/settle synchronization.Examples:
Speech: model semantic open chrome and search for recent trends
Typical plan:
{ "steps": [ {"action": "switch_app", "args": {"app_name": "Google Chrome"}}, {"action": "new_tab", "args": {}}, {"action": "focus_address", "args": {}}, {"action": "insert_text", "args": {"text": "recent trends"}}, {"action": "key", "args": {"combo": "enter"}} ], "summary": "Open Chrome and run a web search" }Speech: model semantic open gedit
Typical plan:
{ "steps": [ {"action": "launch_app", "args": {"app_name": "gedit"}} ], "summary": "Launch text editor" }Speech: model semantic find budget in this page
Typical plan:
{ "steps": [ {"action": "find_text", "args": {"text": "budget"}} ], "summary": "Open in-page search for budget" }Speech: model semantic go to github.com in current tab
Typical plan:
{ "steps": [ {"action": "go_url", "args": {"url": "https://github.com"}} ], "summary": "Navigate current tab to GitHub" }Speech: model semantic select all and copy
Typical plan:
{ "steps": [ {"action": "select_all", "args": {}}, {"action": "copy", "args": {}} ], "summary": "Copy all content" }Speech: model semantic type hello world and press enter
Typical plan:
{ "steps": [ {"action": "insert_text", "args": {"text": "hello world"}}, {"action": "key", "args": {"combo": "enter"}} ], "summary": "Insert text and submit" }Speech: model semantic wait half a second then paste
Typical plan:
{ "steps": [ {"action": "sleep", "args": {"ms": 500}}, {"action": "paste", "args": {}} ], "summary": "Delay then paste" }User-facing behavior added
model semantic <user.text>model semantic run planmodel semantic cancel planmodel semantic copy planmodel semantic repeat last