Initial discovery of Gateways works fine, but when starting the integration with all gateways configured it can't find any gateway

[uschindler]

This is a followup when I applied the fix for #23:

Hi,
I tried to install the fork of @Iminet72. It install successfully and no python errors are happening. It detects the MobileAlerts Gateway successfully and adds the config for it. But when it then starts the integration it hangs repeating the same over and over (the first part of the log is printed with debugging enabled for the integration, so it successfully gets all information from the gateway and in CAN READ the configuration). But at end it can't enable the integration.

2024-10-19 15:55:36.154 DEBUG (MainThread) [custom_components.mobile_alerts.config_flow] async_step_single_gateway gateway MOBILEALERTS-Gateway V1.50, SerialNo: xxxx (id: xxxx)
Use DHCP: Yes
DHCP IP: 192.168.1.7
Fixed IP: 192.168.1.222
Fixed Netmask: 255.255.255.0
Fixed Gateway: 192.168.1.254
Fixed DNS: 192.168.1.253
Cloud Server: www.data199.com
Use Proxy: No
Proxy Server: 192.168.1.1
Proxy Port: 8080
Send data to cloud: Yes
Last Contact: Sat Oct 19 15:55:34 2024
2024-10-19 15:55:38.499 DEBUG (MainThread) [custom_components.mobile_alerts.config_flow] async_step_single_gateway gateway MOBILEALERTS-Gateway V1.50, SerialNo: xxxx(id: xxxxxx)
Use DHCP: Yes
DHCP IP: 192.168.1.7
Fixed IP: 192.168.1.222
Fixed Netmask: 255.255.255.0
Fixed Gateway: 192.168.1.254
Fixed DNS: 192.168.1.253
Cloud Server: www.data199.com
Use Proxy: No
Proxy Server: 192.168.1.1
Proxy Port: 8080
Send data to cloud: Yes
Last Contact: Sat Oct 19 15:55:34 2024
2024-10-19 15:55:38.500 DEBUG (MainThread) [custom_components.mobile_alerts] async_setup_entry {'created_at': '2024-10-19T13:55:38.499783+00:00', 'data': {'send_data_to_cloud': True}, 'disabled_by': None, 'domain': 'mobile_alerts', 'entry_id': '01JAJGZWC3CH0DMQ1ZKY7SW2T2', 'minor_version': 1, 'modified_at': '2024-10-19T13:55:38.499791+00:00', 'options': {}, 'pref_disable_new_entities': False, 'pref_disable_polling': False, 'source': 'user', 'title': 'xxxx', 'unique_id': 'xxxx', 'version': 1}
2024-10-19 15:56:13.863 DEBUG (MainThread) [custom_components.mobile_alerts] async_setup_entry {'created_at': '2024-10-19T13:55:38.499783+00:00', 'data': {'send_data_to_cloud': True}, 'disabled_by': None, 'domain': 'mobile_alerts', 'entry_id': '01JAJGZWC3CH0DMQ1ZKY7SW2T2', 'minor_version': 1, 'modified_at': '2024-10-19T13:55:38.499791+00:00', 'options': {}, 'pref_disable_new_entities': False, 'pref_disable_polling': False, 'source': 'user', 'title': 'xxxx', 'unique_id': 'xxxx', 'version': 1}
2024-10-19 15:56:54.379 DEBUG (MainThread) [custom_components.mobile_alerts] async_setup_entry {'created_at': '2024-10-19T13:55:38.499783+00:00', 'data': {'send_data_to_cloud': True}, 'disabled_by': None, 'domain': 'mobile_alerts', 'entry_id': '01JAJGZWC3CH0DMQ1ZKY7SW2T2', 'minor_version': 1, 'modified_at': '2024-10-19T13:55:38.499791+00:00', 'options': {}, 'pref_disable_new_entities': False, 'pref_disable_polling': False, 'source': 'user', 'title': 'xxxx', 'unique_id': 'xxxx', 'version': 1}
2024-10-19 15:57:44.668 DEBUG (MainThread) [custom_components.mobile_alerts] async_setup_entry {'created_at': '2024-10-19T13:55:38.499783+00:00', 'data': {'send_data_to_cloud': True}, 'disabled_by': None, 'domain': 'mobile_alerts', 'entry_id': '01JAJGZWC3CH0DMQ1ZKY7SW2T2', 'minor_version': 1, 'modified_at': '2024-10-19T13:55:38.499791+00:00', 'options': {}, 'pref_disable_new_entities': False, 'pref_disable_polling': False, 'source': 'user', 'title': 'xxxx', 'unique_id': 'xxxx', 'version': 1}
2024-10-19 15:58:55.096 DEBUG (MainThread) [custom_components.mobile_alerts] async_setup_entry {'created_at': '2024-10-19T13:55:38.499783+00:00', 'data': {'send_data_to_cloud': True}, 'disabled_by': None, 'domain': 'mobile_alerts', 'entry_id': '01JAJGZWC3CH0DMQ1ZKY7SW2T2', 'minor_version': 1, 'modified_at': '2024-10-19T13:55:38.499791+00:00', 'options': {}, 'pref_disable_new_entities': False, 'pref_disable_polling': False, 'source': 'user', 'title': 'xxxx', 'unique_id': 'xxxx', 'version': 1}
2024-10-19 16:00:45.376 DEBUG (MainThread) [custom_components.mobile_alerts] async_setup_entry {'created_at': '2024-10-19T13:55:38.499783+00:00', 'data': {'send_data_to_cloud': True}, 'disabled_by': None, 'domain': 'mobile_alerts', 'entry_id': '01JAJGZWC3CH0DMQ1ZKY7SW2T2', 'minor_version': 1, 'modified_at': '2024-10-19T13:55:38.499791+00:00', 'options': {}, 'pref_disable_new_entities': False, 'pref_disable_polling': False, 'source': 'user', 'title': 'xxxx', 'unique_id': 'xxxx', 'version': 1}
2024-10-19 16:02:35.572 DEBUG (MainThread) [custom_components.mobile_alerts] async_setup_entry {'created_at': '2024-10-19T13:55:38.499783+00:00', 'data': {'send_data_to_cloud': True}, 'disabled_by': None, 'domain': 'mobile_alerts', 'entry_id': '01JAJGZWC3CH0DMQ1ZKY7SW2T2', 'minor_version': 1, 'modified_at': '2024-10-19T13:55:38.499791+00:00', 'options': {}, 'pref_disable_new_entities': False, 'pref_disable_polling': False, 'source': 'user', 'title': 'xxxx', 'unique_id': 'xxxx', 'version': 1}

This repeats forever.

Unfortunately the Exception as printed in the UI (interestingly you can't find it in the log file its only printed on the integration configuration page) is not very helpful, it only says that it can't initialize the gateway. It is thrown here: https://github.com/Iminet72/ha_mobilealerts/blob/1d5a70b7f7da7e831ab3139ee82cd9adf430f1ef/custom_components/mobile_alerts/__init__.py#L30-L33

It returns false here in the low level gateway/proxy implementation causing the above exception: https://github.com/PlusPlus-ua/python-mobilealerts/blob/80b4591a3998dcbe62266e6f1294104b4ddeffd5/mobilealerts/gateway.py#L91-L100

I don't understand why the discovery of the gateway worked, but when it wants to enable the integration for the given gateway it fails while parsing the configuration.

The issue is simple:
My Hass server has multiple network interfaces and IP addresses. When the config workflow does the lookup it passes HASS IP as configured in the network interfaces as "base" address for the discovery (this is the one configured for broadcasts in the network settings of HASS). This IP address is retrieved from hass, see here:

ha_mobilealerts/custom_components/mobile_alerts/config_flow.py

Lines 115 to 120 in 8143b11

	gateways = []
	ip_address = await async_get_source_ip(self.hass)
	try:
	gateways = await Gateway.discover(ip_address)
	except socket.error as err:
	_LOGGER.error("Gateways discovery error %r", err)

Unfortunately when the implementation starts it can't find the gateway and more (to set the proxy port and IP address) of proxy, because the broadcast is sent with None as source address. In case of multiple network interfaces, the system does not know what interface to use to send the broadcast for reconfiguring the gateway.

Basically, my quick hack was simple, just modify the main.py file and hardcode my internal IP address as 2nd parameter to new Gateway(). The correct fix is to move the proxy_ip a few lines below up:

    proxy_ip = await async_get_source_ip(hass, gateway_ip)

and use it for discovery. I have to figure out and check the networking stack of hass to use the correct address for broadcasting in both cases. Once the address of gateway is discovery.

[Iminet72]

I'll check how the HA config is set for me right now. there is also LAN ip and IPv6. I haven't looked at the Wi-Fi connection for a long time to see if it's turned on or off.
If you send me a fix, I'll add it to mine in case it helps someone else.

[Iminet72]

Is your machine in multiple networks?
The other IP address also has another gateway, if I understand correctly?

[uschindler]

Hi,
yes the machine is on multiple networks (it also works as router) and has several docker containers. In the HASS confuration the networks interface for discovery/multicast is selected under "Settings -> System -> Network".

The other IP address also has another gateway, if I understand correctly?

The MobileAlerts Gateway is on the internal network that is also configured in the above HASS config setting. Don't confuse Gateway here, they have nothing to do with the issue here which is about broadcasts.

The problem here is simple: When sending broadcasts to discover the MobileAlerts gateway devices, it uses the official source IP address from the global HASS configuration (see the config flow):

ha_mobilealerts/custom_components/mobile_alerts/config_flow.py

Lines 115 to 120 in 8143b11

	gateways = []
	ip_address = await async_get_source_ip(self.hass)
	try:
	gateways = await Gateway.discover(ip_address)
	except socket.error as err:
	_LOGGER.error("Gateways discovery error %r", err)

This works well.

Unfortunately, when it sets up the gateway for normal use and configuring the proxy, it misses to pass that local IP address in the same way:

ha_mobilealerts/custom_components/mobile_alerts/__init__.py

Line 30 in 1d5a70b

gateway = Gateway(entry.unique_id)

What most people don't know: This code also needs to first find the MobileAlertsGateway using a broadcast, so it fully matters where to send it, it can't send it without knwoing the source IP address. The original developer just missed to set the second parameter in that line - was an oversight:

ha_mobilealerts/custom_components/mobile_alerts/__init__.py

Line 30 in 1d5a70b

gateway = Gateway(entry.unique_id)

If you add its in the same way like in the first code fragment, the gateway can be found successfully. The code works correct if there is only one network interface which can do IPv4 broadcasts, which is not the case when you have docker containers or WAN interfaces on the same network. So care must be taken to use the "official interface" as configured in HASS settings.

[uschindler]

I will provide a PR tomorrow with a correct implementation using the official APIs that HASS provides for discovery of devices. I will check both implementations to do the correct thing according to official HASS network code guidelines.

[uschindler]

Basically the code has to look like this and then the code is fully functional:

    local_ip = await async_get_source_ip(hass)
    gateway = Gateway(entry.unique_id, local_ip) # <-- second parameter here is important, like explained above!
    gateway.send_data_to_cloud = entry.options.get(CONF_SEND_DATA_TO_CLOUD, True)
    if not await gateway.init():
        raise ConfigEntryNotReady("Error initialization of MobileAlerts Gateway (%s)", gateway.gateway_id)

    gateway_ip = gateway.ip_address
    proxy_ip = await async_get_source_ip(hass, gateway_ip)
    _LOGGER.debug("async_setup_entry gateway_ip %s, proxy_ip %s", gateway_ip, proxy_ip)
    proxy = Proxy(None, proxy_ip)

[Iminet72]

They. If you send me a change, I'll try to include it as soon as possible. I have two other tasks right now, but I'll look at them as soon as I can

[uschindler]

@Iminet72 PR is #27

The fix is "incomplete" and does not comply with the standard HASS discovery, but I will fix it later. It works if you have only one HASS-enabled network interface, like this config: