Fix private C API usage by implementing CustomAPIDevice

[alkuzad]

From the discussion on https://community.home-assistant.io/t/esphome-stability-too-much-breaking-changes-compared-to-tasmota/959500/20
some of the functionalities use private api and they should use public one, stable:

CustomAPIDevice instead of manually constructing Protobuf message

Bad - private usage:

api::HomeassistantServiceResponse resp;

resp.service = service;

resp.data.push_back(...);

send_homeassistant_service_call(resp);

Good - public API usage:

// This is stable public API

call_homeassistant_service("light.turn_on", {

{"entity_id", "light.my_light"},

{"brightness", "127"},

});

fire_homeassistant_event("esphome.something_happened", {

{"my_value", "500"},

});

CustomApiDevice:

https://github.com/esphome/esphome/blob/1f271e7c10e28596ff5002088b7ffae33feb4035/esphome/components/api/custom_api_device.h

[olicooper]

Thanks for the suggestion, I think it'll be good to document why this functionality is currently designed this way so it can be reviewed later...

I agree that public API usage is absolutely the right approach generally, but the main issue in this case is that esphome have not provided public API access to the data_template attribute as can be seen in custom_api_device.h. Because of this, I have no choice but to use the internal API as seen below:

esphome-nspanel-lovelace-native/components/nspanel_lovelace/nspanel_lovelace.cpp

Lines 2311 to 2320 in 98d0df5

	for (auto &it : data_template) {
	api::HomeassistantServiceMap kv;
	#if ESPHOME_VERSION_CODE >= VERSION_CODE(2025,8,0)
	kv.set_key(esphome::StringRef(it.first));
	#else
	kv.key = it.first;
	#endif
	kv.value = it.second;
	resp.data_template.push_back(kv);
	}

I think it is important to note that the function APIServer::send_homeassistant_service_call is marked as public so I personally think that any changes should really be handled more carefully by esphome so functions marked as public don't suddenly break. They are even calling the function directly in their own components which interact with homeassistant services e.g. this.

To solve this issue I would need esphome to create additional functions which provide access to the data members of the HomeassistantServiceResponse object (currently data, data_template, variables). I have opened unrelated tickets before but they are busy and I don't have the time to make PRs for esphome right now, but I will monitor and update if they make any changes 🤞