[🚀 Feature]: BiDi: Add low-level helper method to subscribe/unsubscribe from events

[Mr0grog]

Feature and motivation

I’ve started playing around with the BiDi support in the Ruby webdriver, and have noticed that subscribing to and unsubscribing from events is fairly complicated: you need to keep track of the event callbacks that have been added to the session (including by other actors) so you know whether to send a session.subscribe/session.unsubscribe command alongside adding/removing your callback.

For example, the Ruby LogHandler does this:

selenium/rb/lib/selenium/webdriver/bidi/log_handler.rb

Lines 44 to 59 in 164bf79

	def remove_message_handler(id)
	@bidi.remove_callback('log.entryAdded', id)
	unsubscribe_log_entry if @log_entry_subscribed && @bidi.callbacks['log.entryAdded'].empty?
	end
	private
	def subscribe_log_entry
	@bidi.session.subscribe('log.entryAdded')
	@log_entry_subscribed = true
	end
	def unsubscribe_log_entry
	@bidi.session.unsubscribe('log.entryAdded')
	@log_entry_subscribed = false
	end

The Python source follows a similar pattern in Script:

selenium/py/selenium/webdriver/common/bidi/script.py

Lines 38 to 52 in 164bf79

	def remove_console_message_handler(self, id):
	self.conn.remove_callback(LogEntryAdded, id)
	self._unsubscribe_from_log_entries()
	remove_javascript_error_handler = remove_console_message_handler
	def _subscribe_to_log_entries(self):
	if not self.log_entry_subscribed:
	self.conn.execute(session_subscribe(LogEntryAdded.event_class))
	self.log_entry_subscribed = True
	def _unsubscribe_from_log_entries(self):
	if self.log_entry_subscribed and LogEntryAdded.event_class not in self.conn.callbacks:
	self.conn.execute(session_unsubscribe(LogEntryAdded.event_class))
	self.log_entry_subscribed = False

I don’t know if this is a pattern that has crept in or a standard design/set of interfaces that are expected to exist across all the languages. Either way, having the subscriber need to keep track of who else might be subscribed seems like a problem — it adds a bit of complexity everywhere events are used and it’s easy to mess up if a user needs to listen to events that aren’t currently handled by a higher-level wrapper.

It would be really helpful if there were a subscribe or listen_to_event or some similar method (plus the corresponding unsubscribe method) on the low-level BiDi API that adds your callback and calls session.subscribe/session.unsubscribe as appropriate, e.g:

module Selenium
  module WebDriver
    class BiDi
      # This matches what `Script` does, but I imagine a more robust version of
      # this might also want to take an optional list of contexts and
      # subscribe/unsubscribe only the appropriate contexts. Then the handling
      # of callbacks might also filter based on what context the event is
      # coming from, rather than just hooking your callback directly up to the
      # @ws instance.
      def add_callback(event, &block)
        session.subscribe(event) if callbacks[event].empty?
        @ws.add_callback(event, &block)
      end

      def remove_callback(event, id)
        @ws.remove_callback(event, id)
        session.subscribe(event) if callbacks[event].empty?
      end

(BiDi seems like the obvious place to put this level of abstraction in Ruby, but the Python library currently has nothing sitting in-between the higher-level APIs like Script and the actual websocket connection. I have not looked at the other language packages.)

Usage example

Currently, subscribing/unsubscribing to BiDi events requires code like (this is Ruby, but it’s similar in Python and maybe other languages):

# Subscribe to navigation events:
driver_instance.bidi.session.subscribe('browsingContext.fragmentNavigated')
callback_id = driver_instance.bidi.add_callback('browsingContext.fragmentNavigated') do |params|
  # Do something with the event.
  puts "Fragment Navigated: #{params}"
end

# Do some work...

# Unsubscribe:
driver_instance.bidi.remove_callback('browsingContext.fragmentNavigated', callback_id)
driver_instance.bidi.session.unsubscribe('browsingContext.fragmentNavigated') if driver_instance.bidi.callbacks['browsingContext.fragmentNavigated'].empty?

Ideally, subscribing/unsubscribing would be one-liners without any conditionals:

callback_id = driver_instance.bidi.subscribe('browsingContext.fragmentNavigated') do |params|
  # Do something with the event.
  puts "Fragment Navigated: #{params}"
end

# Do some work...

# Unsubscribe:
driver_instance.bidi.unsubscribe('browsingContext.fragmentNavigated', callback_id)

Even nicer if allows filtering by context, but maybe not as critical?

callback_id = driver_instance.bidi.subscribe('browsingContext.fragmentNavigated', contexts: [a_context]) do |params|
  # Do something with the event.
  puts "Fragment Navigated: #{params}"
end

# Do some work...

# Unsubscribe (from which context is handled for you based on callback_id):
driver_instance.bidi.unsubscribe('browsingContext.fragmentNavigated', callback_id)

[github-actions]

@Mr0grog, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[titusfortner]

I'm not sure I'm following your concern.

The BiDi classes are all private API (I thought I'd marked them as such, need to add that). The plan is to provide nice wrapper methods to access everything. That means we are assuming that the driver is the source of truth and storing state. Users should never need to call driver_instance.bidi, users should never need to know the names of the domains they are subscribing to. If they do something that needs that subscription, it happens for them.

Subscribing/unsubscribing happens automatically when you use the API:

id = driver.script.add_console_message_handler { |log| log_entries << log }
driver.script.remove_console_message_handler(id)

So we'll probably have something like (domains and method names tbd):

id = driver.navigation.add_handler { |event| log_navigation << event}
driver.navigation.remove_handler(id)

[Mr0grog]

Ah, I guess this was my fundamental misunderstanding:

The BiDi classes are all private API (I thought I'd marked them as such, need to add that). The plan is to provide nice wrapper methods to access everything.

I assumed the low-level API was meant to be available for cases the high-level stuff does not (yet) cover, or when new BiDi stuff eventually shows up in various drivers before high-level wrappers show up, or as a general escape hatch from the way something is handled in the high-level wrappers (calling the wrappers "high-level API" all over the place also implied to me that there is meant low-level API to drop down to in this way).

Seeing all the discussion around deprecating devtools, etc. (both here and from vendors, e.g. https://fxdx.dev/deprecating-cdp-support-in-firefox-embracing-the-future-with-webdriver-bidi/) suggested to me that I should be getting started with BiDi. Since there is hardly any high-level coverage, I thought I was supposed to be using the low-level stuff for now. Is the actual intent that I should just not use BiDi yet? (Excepting very narrow use cases that have been covered.)

[titusfortner]

low-level API was meant to be available for cases the high-level stuff does not (yet) cover

This is also true! We want the classes to be available if someone wants to dig in and do interesting things (and then tell us that their use case should have a high-level API to make it easier). So, with that in mind, if there is something we can make easier in the low-level API as we build it out, we should do so. We've only got one implementation so far, so I'm assuming there are going to be some additional abstractions we'll be able to make to share patterns, and I just don't know them, yet.

[titusfortner]

@Mr0grog would love to discuss more on Slack/Matrix/IRC - https://www.selenium.dev/support/#ChatRoom

[github-actions]

This issue is stale because it has been open 280 days with no activity. Remove stale label or comment or this will be closed in 14 days.

[github-actions]

This issue was closed because it has been stalled for 14 days with no activity.

[Mr0grog]

I should have commented earlier here, but one of the sticking points is that this area of the spec has been under pretty active revision (for example, this PR just landed in the past day: w3c/webdriver-bidi#867). There have been several PRs and changes around event subscription in the spec over the past month or two, so it seems like there is a lot of instability in this area right now.

I think there’s plenty that could be done given the original way it worked (which I think is still supported in the spec and I assume will continue to be now that there are multiple shipping implementations of it), although now there are things like subscription IDs that you can subscribe and unsubscribe to. Those subscription IDs may take care of some aspects of this issue, but I’m not 100% clear on how it works if you have multiple subscription IDs involving the same (context/navigable + event). I am also not clear on what implementors are doing/planning, given the active spec changes in this area. 🤷

[github-actions]

This issue has been automatically locked since there has not been any recent activity since it was closed. Please open a new issue for related bugs.