Evaporation sensor is not reliable

[puetsch]

Version of home_assistant

2025.1.2

How are you running home_assistant

HAOS

Version of the custom_component

2.1.26

Describe the bug

The sensor evaporation_in_the_next_24_hours behaves ususual.

1. It is unknown most of the time.
2. On many days (but not all) it has values from midnight to 11am and then goes back to unknown. (See graph below.)
3. The forecast, similarily, has non-null values only for some days.

I considered the case that null or unknown means zero but that doesn't check out because today is a sunny day and evaporation is almost certainly positive.

Note: This seems related to Issue #150

Current attributes

data: 
- datetime: '2025-01-13T06:00:00+00:00'
  value: null
- datetime: '2025-01-14T06:00:00+00:00'
  value: null
- datetime: '2025-01-15T06:00:00+00:00'
  value: 0.2
- datetime: '2025-01-16T06:00:00+00:00'
  value: null
- datetime: '2025-01-17T06:00:00+00:00'
  value: 0.1
- datetime: '2025-01-18T06:00:00+00:00'
  value: 0.2
- datetime: '2025-01-19T06:00:00+00:00'
  value: 0.1
- datetime: '2025-01-20T06:00:00+00:00'
  value: null
- datetime: '2025-01-21T06:00:00+00:00'
  value: null

forecast_time_utc: 2025-01-13T03:00:00+00:00
latest_update_utc: 2025-01-13T09:00:52.946058+00:00
station_id: 10803
station_name: Freiburg
attribution: Data provided by Deutscher Wetterdienst (DWD)
unit_of_measurement: kg/m2
device_class: 
icon: mdi:waves-arrow-up
friendly_name: Freiburg Evaporation in the next 24 hours```

[FL550]

Thanks for reporting @puetsch! Just to be on the same side: Are you using the normal update rate or forcing the hourly update?

I've checked the raw data of your station of the "normal" update which is every 6 hours and extracted the following values:

Timestamp of forecast	Timestamp of first evaporation data	Timestep difference	Reported value
1315	1506	38	0.0
1321	1506	32	0.0
1403	1506	26	0.0
1409	1606	44	0.3
1415	1606	38	0.2
1421	1606	32	0.3
1503	1606	26	0.2
1509	1706	44	0.3

The reported value is "Potential evapotranspiration within the last 24 hours" so we have to look a day in the future so you have to subtract one day. Analyzing this further give us this:

Timestamp of forecast	Forecast Day	First Evaporation Day	Day Difference	Evaporation available for today
1315	13	15	2	no
1321	13	15	2	no
1403	14	15	1	yes
1409	14	16	2	no
1415	14	16	2	no
1421	14	16	2	no
1503	15	16	1	yes
1509	15	17	2	no

So every day at 3 o'clock the data is available for the current day and every other report only for the next day. With a bit of imagination I can see this behavior in the graph you provided.

Regarding the null values after the first two days, these are days when there is indeed nothing reported by DWD for this specific day.

---

So to sum this up, there is an issue with the raw data and right now I'm not sure how to handle this properly. First impulse is to cache the value at 3 o'clock and insert it in the later reports. Maybe you have a better solution?

[puetsch]

Thank you for the root cause analysis, @FL550!

Yes, I am using the regular daily update sequence.

From my perspective, I wouldn't recommend trying to fix that missing data downstream, in the integration. Especially as in this case it's "expected evaporation in the next 24 hours" so if you show that value 12 hours later, it might be completely off.

(btw, in https://s3.eu-central-1.amazonaws.com/app-prod-static.warnwetter.de/v16/forecast_mosmix_10803.json I don't see any field for evaporation at all. :/ Is that the API endpoint you're using?

The way I look at it, it seems like your research has uncovered a data error on DWD's side. Maybe it makes most sense to alert them to it? Either it's a bug, or we don't know how to interpret the data points. (I'm with you that I'd expect a value reported for every hour, but at least consistently once a day at a fixed hour.) Might be best to try opendata@dwd.de first. I can help but I think you have most knowledge about which endpoints you used and which days are missing.

[FL550]

I've written a mail two weeks ago and wait now for an answer. I'll keep you posted!

[stasihasi]

I have the isue at location "voeklingen" and "berus", that the values are to high:

according to DWD internet homepage, that are the correct values:

thanks in advance for feedback!

[FL550]

@stasihasi
I haven't heard anything from DWD. Maybe you could also write them about this issue?