UI testing MCP

[dts350z]

It seems like a logical next step would by an MCP that do UI testing/debugging. In my case, for windows applications (not just browser stuff).

Maybe integrate one of the existing Open-Source UI tools via MCP? AutoIt/Robot Framework, SikullX, etc.

My particular use case, at the moment, would be UI tests for audio plugins that run inside other closed source DAW software.

[dts350z]

There is this: https://testguild.com/top-model-context-protocols-mcp/

but at first look, I'm not sure I see a good fit.

[wonderwhy-er]

Yeah I have not seen good enough integrations with native UI
I think there was one that controlled android emulator or ios emulator

But not native apps.

For Mac you can try using apple script to pull that off.

For windows you would need something like Autoit

I am not sure that is best way to do debugging though. For good debugging you would need access to internal app state somehow and then its easier to just manipulate up internally instead of trough UI.

What is it exactly that you want to do in that sense?

[dts350z]

The audio plugin, which is instrumented for debug logging, runs inside a Digital Audio Workstation (DAW) host. The current manual test workflow is:

1. copy plugin from build area to deployment area
2. Launch DAW
3. Load DAW "layout", used for testing, which includes the plugin
4. Run the UI Test Plan, which includes clicking on UI controls both in the DAW and In the Plugin, which will write a log file
5. Close the DAW

Claude understands UI elements in screenshots, so screen captures could also be part of the test plan.

It just seems like the logical next step. If LLM + MCP can update code and build the project, why not also run the test plan to complete the feedback loop?

[dts350z]

Maybe someone already has an MCP for Autoit? https://www.reddit.com/r/mcp/comments/1k1pskq/local_desktop_softwares_mcps/

https://github.com/mario-andreschak/mcp-windows-desktop-automation

[wonderwhy-er]

Looks interesting.
Will need to give them a try