Skip to content

Latest commit

 

History

History
14 lines (10 loc) · 722 Bytes

File metadata and controls

14 lines (10 loc) · 722 Bytes

User Interface Inverse Dynamics Model

IDM used in MineRL here https://github.com/openai/Video-Pre-Training General Inverse Dynamics Model for Linux desktop environments. Collecting generic desktop navigation data is a security and privacy hazard, yet this data is priceless to teach AI agents how to use computer user interfaces: coordinating between multiple applications over long periods of time, just like humans do.

Here I open source tooling for data collection to self-train user interface navigation for agents.

Use an up-to-date version of OBS with websocket/remote control builtin. Tools->WebSocket Server Settings-> Enable WebSocket server Disable Authentication (or update idm.py file)