-
Notifications
You must be signed in to change notification settings - Fork 6
Open
Description
Description
Add tools for finding images on screen using computer vision.
SwiftAutoGUI API Reference
locateOnScreen(imagePath: String, confidence?: Float)- Find image on screenlocateCenterOnScreen(imagePath: String)- Find image centerlocateAllOnScreen(imagePath: String)- Find all image matches
Proposed Implementation
locateOnScreen
- Input:
imagePath(string) - Path to reference imageconfidence(number, optional) - Match confidence 0.0-1.0
- Output: {x, y, width, height} or null if not found
locateCenterOnScreen
- Input:
imagePath(string) - Path to reference imageconfidence(number, optional) - Match confidence
- Output: {x, y} center coordinates or null
locateAllOnScreen
- Input:
imagePath(string) - Path to reference imageconfidence(number, optional) - Match confidence
- Output: Array of {x, y, width, height} for all matches
Use Cases
- Automated UI testing
- Waiting for UI elements
- Game automation
- Visual verification
Technical Considerations
- OpenCV dependency (already in SwiftAutoGUI)
- Performance impact of image matching
- Handling retina displays and scaling
- Image format support
- Error handling for missing images
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels