feat: Add multi-display support for Android automation

[ccxj]

feat: Add multi-display support for Android automation

Description

This PR implements comprehensive multi-display automation capabilities for Android devices, enabling robust testing of foldable phones, external displays, and multi-screen environments.

Technical Changes

New API Endpoints

• POST /session/:sessionId/appium/element/on_display - Find element on specific display
• POST /session/:sessionId/appium/elements/on_display - Find multiple elements on display
• GET /session/:sessionId/appium/device/display_info - Retrieve display information
• GET /session/:sessionId/appium/source/on_display - Get page source from display

Core Enhancements

• AndroidElement Interface: Added setDisplayId() method for display context tracking
• DisplayIdResolver: New utility for display identification and validation
• UiSelectorBuilder: Enhanced to support display-specific element queries
• Four New Handlers: Implement multi-display automation logic

Key Benefits

• Precision element location on specific displays
• Complete display topology awareness
• Support for foldable devices and external displays
• Fully backward compatible

Testing

• Validated on multi-display Android devices
• Verified backward compatibility
• Confirmed parameter validation and error handling

[linux-foundation-easycla]

• ❌ The email address for the commit (1c6c0f4) is not linked to the GitHub account, preventing the EasyCLA check. Consult this Help Article and GitHub Help to resolve. (To view the commit's email address, add .patch at the end of this PR page's URL.) For further assistance with EasyCLA, please submit a support request ticket.

[KazuCocoa]

I haven't dug into this implementation well yet, but what is the difference between currentDisplayId and this implementation?
You could find the description in https://github.com/appium/appium-uiautomator2-driver and search the implementation in this repository

[pinnnkman]

Hi @KazuCocoa
I would like to ask: if the interaction test of different screens, for example, when screen A has performed an operation, if the operation is performed using the currentDisplayId method, then switching to screen B needs to be re-instantiated, is it possible for us to operate a different screen after instantiating it once?

[mykola-mokhnach]

screen B needs to be re-instantiated

could you be more specific about that?

Eventually, I agree to @KazuCocoa. There is no need to re-introduce the same functionality twice if it already exists. Consider improving/fixing the existing implementation instead.

[pinnnkman]

@mykola-mokhnach

Here's my understanding for multi-screen operations, am I doing it right?

# main screen actions
screen_a = {
....,
currentDisplayId=0,
....}
driver_a = webdriver.Remote(str(remote_url), options=screen_a, strict_ssl=strict_ssl)
driver_a.actions...

# screen B actions
screen_b = {
....,
currentDisplayId=1,
....}
driver_b = webdriver.Remote(str(remote_url), options=screen_a, strict_ssl=strict_ssl)
driver_b.actions...

[KazuCocoa]

No, as the readme, currentDisplayId is under settings api. https://github.com/appium/appium-uiautomator2-driver?tab=readme-ov-file#settings-api, so a session handles multiple displaies in the session dynamically.
e.g. https://github.com/appium/python-client/blob/e59b853497ae04706137cf252bfa3b92e6a1e229/test/unit/webdriver/settings_test.py#L57

[pinnnkman]

@mykola-mokhnach got it, thank you for your solutions very much, current i already use this to do, but i mean if we can:

POST     /session/element/on_display    {display_id: 0}
POST     /session/click/on_display    {display_id: 0}
POST     /session/element/on_display    {display_id: 1}
POST     /session/click/on_display    

not:

POST     /session/element/
POST     /session/click/
POST     /session/settings/update    {display_id: 1}
POST     /session/element/
POST     /session/click/

[KazuCocoa]

Could you share the appium server log with timestamp for both? I assume you might concern about the delay with the extra endpoint call - /session/settings/update. As past comment in #729 (comment), we won't add the same functionality (and keep maintaining both.) Instead, we'd like to enhance the existing method with currentDisplayId

[pinnnkman]

@KazuCocoa I'm just raise a question and you have explained it well. I have no more questions, thanks.

[ccxj]

I haven't dug into this implementation well yet, but what is the difference between currentDisplayId and this implementation? You could find the description in https://github.com/appium/appium-uiautomator2-driver and search the implementation in this repository

We tested currentDisplayId on our conference hardware device in an HDMI multi-display scenario, but it only returned controls on the primary display. Therefore, we implemented a separate solution to support multi-display control and interaction.

[KazuCocoa]

Thank you for checking. Would it be sufficient to use a secondary display on an emulator to test out this multi-display behavior?

[update]
It looks like no. We'll need to add several error handling for the emulator case (#741)

[mykola-mokhnach]

@ccxj I've added some more helpers to address multi-display setups testing. Please check appium/appium-uiautomator2-driver#949 and share your feedback.