Add KeyEvent and MouseEvent creation and injection support for Python

[Chevi-Koren]

Type

• New feature (non-breaking change which adds functionality). Resolves Request to create and simulate KeyEvents to a window (Python) #7346

Motivation and Context

Currently, KeyEvent and MouseEvent objects in Open3D's Python GUI can only be generated from real user input. This limitation makes it difficult to write automated tests, reproduce interaction scenarios, or simulate GUI behavior programmatically.

This PR enables programmatic creation and injection of GUI input events from Python, allowing users to simulate keyboard and mouse interactions in a controlled and repeatable way.

Description

This PR adds Python support for creating and injecting KeyEvent and MouseEvent objects into an Open3D GUI window, addressing issue #7346.

The implementation follows the existing Open3D GUI architecture and ensures all injected events are processed safely on the GUI thread.

Changes Made

KeyEvent Support:

• Added Python constructor: gui.KeyEvent(type, key, is_repeat)
• Added: Window.post_key_event(event) to enqueue a KeyEvent for processing by the window.

MouseEvent Support:

• Added static factory methods for common mouse interactions:
  • gui.MouseEvent.move(...)
  • gui.MouseEvent.button_down(...)
  • gui.MouseEvent.button_up(...)
  • gui.MouseEvent.wheel(...)
• Added: Window.post_mouse_event(event) to enqueue a MouseEvent.

Thread Safety:

• Both post_key_event and post_mouse_event use Application::PostToMainThread internally.
• This guarantees that events are safely queued and handled on the GUI thread, even when injected from background threads.

Examples:

• Added: examples/python/visualization/gui_key_events.py
• Added: examples/python/visualization/gui_mouse_events.py

The examples demonstrate:

• Event creation
• Event injection
• Callback handling and verification

Due to the presence of a GUI window, an event loop, and timing dependencies, this functionality is demonstrated via executable examples rather than traditional unit tests.

Usage Example

import open3d.visualization.gui as gui

# Create and inject a KeyEvent
key_event = gui.KeyEvent(gui.KeyEvent.Type.DOWN, gui.KeyName.A, False)
window.post_key_event(key_event)

# Create and inject a MouseEvent
mouse_event = gui.MouseEvent.button_down(
    100, 100, gui.MouseButton.LEFT, 0
)
window.post_mouse_event(mouse_event)

## Checklist:

- [ ] I have run `python util/check_style.py --apply` to apply Open3D **code style** to my code.
- [x] This PR changes Open3D behavior or adds new functionality.
    - [x] Both C++ (Doxygen) and Python (Sphinx / Google style) **documentation** is updated accordingly.
    - [x] I have added or updated C++ and / or Python **unit tests** OR included **test results** (e.g. screenshots or numbers) here.
- [x] I will follow up and update the code if CI fails.
- [x] For fork PRs, I have selected **Allow edits from maintainers**.

[update-docs]

Thanks for submitting this pull request! The maintainers of this repository would appreciate if you could update the CHANGELOG.md based on your changes.

[Chevi-Koren]

@ssheorey
Could you please review this PR when you have a chance? Thanks.

[ssheorey]

Hi @Chevi-Koren your virtual environment was added to the PR. Can you please remove that?

[ssheorey]

Hi @Chevi-Koren have you looked at GUI automation tools? Here are some examples:

https://en.wikipedia.org/wiki/Comparison_of_GUI_testing_tools

https://github.com/jordansissel/xdotool

Not sure if there is a good case for including this in Open3D.

[Chevi-Koren]

@ssheorey Thanks for the suggestion and for pointing to existing GUI automation tools — that’s a fair point.

I wanted to clarify expectations before going further:
would you consider this issue effectively resolved by recommending external tools, or do you still see value in having a native, Python-level way to inject GUI events inside Open3D itself?

My main motivation here is enabling:

reproducible interaction scenarios

lightweight automation without OS-level dependencies

easier debugging and example-driven testing within Open3D

That said, if the maintainers’ preference is to rely on external automation tools instead, I’m happy to stop or redirect the work accordingly.
Just wanted to make sure I’m aligned with the intended direction