[MQTT] (Auto)Discovery implementation

[tonhuisman]

Resolves #977
Resolves #4982
And Forum request(s) (here)

Features:

• MQTT (Auto)Discovery for all supported devices configured to use a MQTT Controller that has MQTT Discovery implemented
  • Controller settings for Trigger and Discovery topics, and an Enabled checkbox
    • Suggested defaults:
    • Discovery Trigger topic: homeassistant/status (listening for payload online).
    • Auto Discovery topic: homeassistant/%devclass%/%unique_id% (/config will be added by default) %unique_id% is a concatenation of the sysname, taskname and valuename, separated by an underscore _, and converted to lowercase.
    • Config suffix|online message (/config|online): /config (will use /config and online by default when left empty)
    • Retain Discovery: Unchecked, can be checked to send the AutoDiscovery with Retain flag.
    • Actions:
    • When discovery is triggered, a random delay between 1 and 10 seconds is inserted before starting the action, to spread the load on the receiving server.
    • Discovery is sent for all enabled tasks that have this controller enabled in their settings.
    • When the controller is enabled and Auto Discovery is enabled, on clicking the Submit button a full discovery is sent.
    • The Discovery is triggered when the Controller is started.
    • Discovery is sent using the Retain Discovery setting.
  • Device settings for Controller config to set an optional Group ID and Resend MQTT Discovery
    • When the task is enabled, the Resend MQTT Discovery checkbox is checked and the Submit button used, the Discovery for the current device is resent.
    • The Group field can be used to manually group multiple tasks as a single device on HA. When 0, all values of the device are grouped as a single, unique, device.
• [C005] Home Assistant (openHAB) MQTT: (partially) implemented (more device classes to add).
  • Device classes: switch, temperature, humidity, pressure, distance, pm1, pm25, pm10
• [P052] CO2 Senseair: implement device class config for supported values
• [P001] Switch: implemented all HA-supported MQTT Device Classes for binary_sensor, and Inversed Logic setting
• Plugins with user-configurable or undefined Value Types now have a selector per Value that is used by AutoDiscovery to determine how to send the discovery message
• [Derived Values] Add support for Value Type, so they can be used in MQTT AutoDiscovery
• [AutoDiscovery] Add support for Derived values, Unit of Measure and Value Type (Home assistant)
• Sending Derived values can be enabled per controller, if the controller supports that
• [C005] Add support for sending Derived values
• [C006] Add support for sending Derived values
• [C014] Add Unit of Measure to Homie protocol AutoDiscovery (/$unit)
• [C014] Add support for sending Derived values

TODO:

• Testing
• Enable actually sending the MQTT messages...
• Documentation updates
  • Add section to Development documentation (added to _Pxxx_PluginTemplate source)
  • Update C005 documentation
  • Update DevicePage documentation for Value Type selection
• Implement more device class handlers: lux, moisture, co2, weight, uv, uvindex, ir, distance, dust, aqi, nox, voltage, current, power, color, color temperature, (wind)direction, wind speed.
• Additional sensor handlers: dimmer, gps, analog. (may be implemented later)
• Implement more plugin specific value handlers (for plugins that allow user-selectable values)
• Add configurable trigger value (default: online)
• Add configurable Config topic suffix (/config)
• Add MQTT Device Class setting and Inverted to Sensor_VType::Switch devices

[KnatteAnka]

Some question to compare to my Node red Autodiscovery "Addon"

1. Does it handle setting values of a Dummy device?
2. Can you hide values as internal values per devide or even per variable? ( i use "_" to tell its internal for autodiscovery)
3. Does it check so mqtt string and values do not contain any character other than A-Z,_,0-9?
4. can you define a dummy devide as a Dimmer?
5. can you define a Select to input to espeasy different states?

[TD-er]

Not sure if Ton is already back home to reply...
So the only thing I know about this is about point 3, where I found out the hard way that a space should not be used.
Meaning Ton will change how a hostname may be used in these topics by replacing the space. Not sure what he will use, either '_' or '-'.
Would it be better to make this configurable? Or is a minus also not allowed?

Ton has added both unit-of-measure selections and value type. So I guess it should be possible to define a dummy task as dimmer. Not sure though what HA then expects how such a device should react on dimming updated from HA, or send dimming updates back to HA. The latter may not be an insignificant issue as I know from Domoticz that it was quite hard to make sure a device would not start to 'oscillate' by constantly toggling a relay for example.

Anyway, since this PR already has been merged, maybe best to open a new issue so we can have all information in 1 place.

[KnatteAnka]

hmm '-' should be allowed yes https://www.home-assistant.io/integrations/mqtt/

and not sure i will update and test this addon anytime soon :( so no worry if answers take time, when/if i get time to test around and see any problems i will create a new issue then.

right checked my documentation at:
https://github.com/KnatteAnka/NodeRED_ESPEASY_AutoDiscovery/blob/main/README.md

and for dimmers i used 2 devices on EE one MQTT import and one dummy device to tell HA what state it is right now, this as i could not get EE to understand messages from HA other than MQTT import or with run event command on device

for Buttons and switches it triggers a event on the EE with the name of the variable

[tonhuisman]

Some question to compare to my Node red Autodiscovery "Addon"

1. Does it handle setting values of a Dummy device?

Changes to the MQTT Controllers in the recent past include handling of the /cmd and /set endpoints for published task/values for performing general commands, and/or setting a value to the task/value, assuming it allows to be set (most sensors overwrite these values periodically per Interval setting).

2. Can you hide values as internal values per devide or even per variable? ( i use "_" to tell its internal for autodiscovery)

For predefined Value types you can't, currently, but for devices that allow setting the Value type (per value), you can set the Value Type to None so it won't be used in Auto Discovery. This does not stop the value from being published, though, currently there is no mechanism to do that. Not sure if that would be needed.
OTOH, HA allows to hide discovered entities, so that would also be a quite feasible/common way to handle that.

3. Does it check so mqtt string and values do not contain any character other than A-Z,_,0-9?

No, but I'll add that check in #5349 soon.

4. can you define a dummy devide as a Dimmer?

A dimmer is something that's rather hard to realize in ESPEasy right now, as we don't have a button that supports auto-repeat when being pressed. As mentioned in my reply to 1., you can send commands, and/or set value(s) to a task/value.

5. can you define a Select to input to espeasy different states?

As long as the value is numeric, that will work fine (the core of ESPEasy is using numeric/float values).
Using the new Presentation feature, added recently when introducing String-variables (see TaskValueSetPresentation command), you can map that value to a string to present to the user on the Devices page. Or in EasyFetch, but I assume you're not currently using that 😉.

Thanks for the pointers given (again) 👍

[tonhuisman]

hmm '-' should be allowed yes

Well, ESPEasy doesn't like the dash as part of a system, task or value name, it will in many situations be interpreted as a subtract operation 😮

[repa6]

@tonhuisman, HA MQTT discovery was changed.
I can get my fake SDM230 meter (TAC2100) working as follows:

config topic:
homeassistant/sensor/Energy_11/SDM230_ImpA/config
payload:
{ "dev":{ "name":"Energy_11 SDM230", "ids":["Energy_11_SDM230"], "mf":"ESPEasy", "sw":"20250725" }, "name":"SDM230 Import Active Energy", "unique_id":"energy_11_sdm230_ImpA", "device_class": "energy", "state_class": "total_increasing", "unit_of_measurement":"kWh", "state_topic":"EspEasy/Energy_11/SDM230s3/ImpA" }

[tonhuisman]

"state_class": "total_increasing",

Hmm, we currently don't have a setting for that, so I'll have to add some more code related to MQTT Discovery 😅

[repa6]

That is only needed if you want to be able to select as a meter on the energy panel in HA. May be useful for all Wh, kWh, MWh, VArh, kVArh, MVArh meters.