You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
'Query and interact with the macOS accessibility interface to inspect UI elements of applications. This tool provides a powerful way to explore and manipulate the user interface elements of any application using the native macOS accessibility framework.\\n\\nThis tool exposes the complete macOS accessibility API capabilities, allowing detailed inspection of UI elements and their properties. It\'s particularly useful for automating interactions with applications that don\'t have robust AppleScript support or when you need to inspect the UI structure in detail.\\n\\n**Input Parameters:**\\n\\n* `cmd` (enum: \'query\' | \'perform\', required): The operation to perform.\\n * `query`: Retrieves information about UI elements.\\n * `perform`: Executes an action on a UI element (like clicking a button).\\n\\n* `locator` (object, required): Specifications to find the target element(s).\\n * `app` (string, required): The application to target, specified by either bundle ID or display name (e.g., "Safari", "com.apple.Safari").\\n * `role` (string, required): The accessibility role of the target element (e.g., "AXButton", "AXStaticText").\\n * `match` (object, required): Key-value pairs of attributes to match. Can be empty ({}) if not needed.\\n * `pathHint` (array of strings, optional): Path to navigate within the application hierarchy (e.g., ["window[1]", "toolbar[1]"]).\\n\\n* `multi` (boolean, optional): When `true`, returns all matching elements rather than just the first match. Default is `false`.\\n\\n* `attributes` (array of strings, optional): Specific attributes to query for matched elements. If not provided, common attributes will be included. Examples: ["AXRole", "AXTitle", "AXValue"]\\n\\n* `requireAction` (string, optional): Filter elements to only those supporting a specific action (e.g., "AXPress" for clickable elements).\\n\\n* `action` (string, optional, required when cmd="perform"): The accessibility action to perform on the matched element (e.g., "AXPress" to click a button).\\n\\n**Example Queries:**\\n\\n1. Find all text elements in the front Safari window:\\n```json\\n{\\n "cmd": "query",\\n "multi": true,\\n "locator": {\\n "app": "Safari",\\n "role": "AXStaticText",\\n "match": {},\\n "pathHint": ["window[1]"]\\n }\\n}\\n```\\n\\n2. Find and click a button with a specific title:\\n```json\\n{\\n "cmd": "perform",\\n "locator": {\\n "app": "System Settings",\\n "role": "AXButton",\\n "match": {"AXTitle": "General"}\\n },\\n "action": "AXPress"\\n}\\n```\\n\\n3. Get detailed information about the focused UI element:\\n```json\\n{\\n "cmd": "query",\\n "locator": {\\n "app": "Mail",\\n "role": "AXTextField",\\n "match": {"AXFocused": "true"}\\n },\\n "attributes": ["AXRole", "AXTitle", "AXValue", "AXDescription", "AXHelp", "AXPosition", "AXSize"]\\n}\\n```\\n\\n**Note:** Using this tool requires that the application running this server has the necessary Accessibility permissions in macOS System Settings > Privacy & Security > Accessibility.',
420
+
`Query and interact with the macOS accessibility interface to inspect UI elements of applications. This tool provides a powerful way to explore and manipulate the user interface elements of any application using the native macOS accessibility framework.
421
+
422
+
This tool exposes the complete macOS accessibility API capabilities, allowing detailed inspection of UI elements and their properties. It's particularly useful for automating interactions with applications that don't have robust AppleScript support or when you need to inspect the UI structure in detail.
423
+
424
+
**Input Parameters:**
425
+
426
+
* \`command\` (enum: 'query' | 'perform', required): The operation to perform.
427
+
* \`query\`: Retrieves information about UI elements.
428
+
* \`perform\`: Executes an action on a UI element (like clicking a button).
429
+
430
+
* \`locator\` (object, required): Specifications to find the target element(s).
431
+
* \`app\` (string, required): The application to target, specified by either bundle ID or display name (e.g., "Safari", "com.apple.Safari").
432
+
* \`role\` (string, required): The accessibility role of the target element (e.g., "AXButton", "AXStaticText").
433
+
* \`match\` (object, required): Key-value pairs of attributes to match. Can be empty (\`{}\`) if not needed.
434
+
* \`navigation_path_hint\` (array of strings, optional): Path to navigate within the application hierarchy (e.g., \`["window[1]", "toolbar[1]"]\`).
435
+
436
+
* \`return_all_matches\` (boolean, optional): When \`true\`, returns all matching elements rather than just the first match. Default is \`false\`.
437
+
438
+
* \`attributes_to_query\` (array of strings, optional): Specific attributes to query for matched elements. If not provided, common attributes will be included. Examples: \`["AXRole", "AXTitle", "AXValue"]\`
439
+
440
+
* \`required_action_name\` (string, optional): Filter elements to only those supporting a specific action (e.g., "AXPress" for clickable elements).
441
+
442
+
* \`action_to_perform\` (string, optional, required when \`command="perform"\`): The accessibility action to perform on the matched element (e.g., "AXPress" to click a button).
443
+
444
+
* \`report_execution_time\` (boolean, optional): If true, the tool will return an additional message containing the formatted script execution time. Defaults to false.
445
+
446
+
* \`limit\` (integer, optional): Maximum number of lines to return in the output. Defaults to 500. Output will be truncated if it exceeds this limit.
447
+
448
+
**Example Queries (Note: key names have changed to snake_case):**
449
+
450
+
1. **Find all text elements in the front Safari window:**
451
+
\`\`\`json
452
+
{
453
+
"command": "query",
454
+
"return_all_matches": true,
455
+
"locator": {
456
+
"app": "Safari",
457
+
"role": "AXStaticText",
458
+
"match": {},
459
+
"navigation_path_hint": ["window[1]"]
460
+
}
461
+
}
462
+
\`\`\`
463
+
464
+
2. **Find and click a button with a specific title:**
465
+
\`\`\`json
466
+
{
467
+
"command": "perform",
468
+
"locator": {
469
+
"app": "System Settings",
470
+
"role": "AXButton",
471
+
"match": {"AXTitle": "General"}
472
+
},
473
+
"action_to_perform": "AXPress"
474
+
}
475
+
\`\`\`
476
+
477
+
3. **Get detailed information about the focused UI element:**
**Note:** Using this tool requires that the application running this server has the necessary Accessibility permissions in macOS System Settings > Privacy & Security > Accessibility.`,
419
491
AXQueryInputShape,
420
492
async(args: unknown)=>{
493
+
let input: AXQueryInput;// Declare input here to make it accessible in catch
421
494
try{
422
-
constinput=AXQueryInputSchema.parse(args);
495
+
input=AXQueryInputSchema.parse(args);
423
496
logger.info('accessibility_query called with input:',input);
424
497
425
498
constresult=awaitaxQueryExecutor.execute(input);
426
499
427
500
// For cleaner output, especially for multi-element queries, format the response
0 commit comments