feat: Added configuration for standard and rolling target timeframes to determine minimum minutes for a slot to be evaluated (3.5 hours dev time)

Author: BottlecapDave

_docs/faq.md

@@ -4,6 +4,22 @@
 
 For some reason when you update the configuration via the integration page, the associated entities don't update. You'll need to reload the parent entry to get the configuration to take effect. This is something I'm currently investigating.
 
+## I've setup a target timeframe with the default time period (00:00-00:00) or a rolling target timeframe looking ahead for 24 hours but it's not updating. Is something broken?
+
+By default, the target timeframe sensors require the supporting data for the specified time periods to be available in order to be calculate. For example if it was `00:00` on `1/12/2025`, then the standard target timeframe would require data for _at least_ between `2025-12-01T00:00` and `2025-12-01T00:00`. If this is not the case, then the sensor will not be evaluated. This is made clearer by the `values_incomplete` attributes of the [target timeframe](./setup/target_timeframe.md#attributes) and [rolling target timeframe](./setup/rolling_target_timeframe.md#attributes).
+
+For some data sources, this might cause issues due to the data available (e.g. When you're on the Agile tariff of [Octopus Energy UK](./blueprints.md#octopus-energy) where data is available in advanced up to `23:00`).
+
+In this scenario, you have two options.
+
+1. The recommended approach would be to adjust the time period that the target timeframe looks at. See below for example suggestions 
+
+| Data Source | Standard Target Timeframe Recommendation | Rolling Target Timeframe Recommendation |
+|-|-|-|
+| Agile tariff for [Octopus Energy UK](./blueprints.md#octopus-energy) | Have an end time before or equal to `23:00` (e.g. `23:00-23:00` if you want to look at a full 24 hours) | Because data refreshes around `16:00` and will go up to `23:00`, then your look ahead hours should be no more than `7` to ensure it's working `99%` of the time |
+
+2. Set the configuration option to [calculate with incomplete data](./setup/target_timeframe.md#calculate-with-incomplete-data). This _could_ have undesired consequences in the calculations (e.g. picking times that look odd retrospectively because the full data wasn't available at the time of picking), so use with caution.
+
 ## How do I increase the logs for the integration?
 
 If you are having issues, it would be helpful to include Home Assistant logs as part of any raised issue. This can be done by following the [instructions](https://www.home-assistant.io/docs/configuration/troubleshooting/#enabling-debug-logging) outlined by Home Assistant.

_docs/setup/rolling_target_timeframe.md

@@ -121,6 +121,10 @@ These settings can have undesired effects and are not recommended to be changed,
 
 By default, the target timeframe isn't calculated if there isn't enough data for the period of time being evaluated. For example, if you have a look ahead hours set to 4 hours, it's 9pm and you only have data up to midnight, then the next target timeframe will not be calculated. If you turn this setting on, then the sensor will attempt to look for 4 hours worth of data if available, otherwise it will evaluate with whatever data is available (in this scenario 2 hours between 10pm and 12am).
 
+#### Minimum required minutes in slots
+
+By default, 30 minute slots that are part way through are not considered when evaluating rolling target time frames. For example, if you are looking for the best slots for the next 4 hours and it's 10:01, then only slots between 10:30 to 14:30 will be evaluated. This threshold can be changed here to a lower value if you want to take account of slots that are partially in the past. For example if this was set to 29, then the previous example would evaluate slots between 10:00 to 14:00.
+
 ## Attributes
 
 The following attributes are available on each sensor

_docs/setup/target_timeframe.md

@@ -131,6 +131,14 @@ These settings can have undesired effects and are not recommended to be changed,
 
 By default, the target timeframe isn't calculated if there isn't enough data for the period of time being evaluated. For example, if you have a timeframe looking between 10pm and 2am, it's 9pm and you only have data up to midnight, then the next target timeframe will not be calculated. If you turn this setting on, then the sensor will attempt to look for data between 10pm and 2am if available, otherwise it will evaluate with whatever data is available (in this scenario 10pm to 12am).
 
+#### Minimum required minutes in slots
+
+By default, 30 minute slots that are part way through are not considered when evaluating target time frames. For example, if you are looking for the best slots between 10:00 to 12:00 and it's 10:01, then only slots between 10:30 to 12:00 will be evaluated. This threshold can be changed here to a lower value if you want to take account of slots that are partially in the past. For example if this was set to 29, then the previous example would evaluate slots between 10:00 to 12:00.
+
+!!! warn
+
+    Changing this can cause sensors to not come on for the correct amount of time by up to 30 minutes.
+
 ## Attributes
 
 The following attributes are available on each sensor
@@ -163,6 +171,7 @@ The following attributes are available on each sensor
 | `next_max_value` | `float` | The average value for the next continuous discovered period. This will only be populated if `target_times` has been calculated and at least one period/block is in the future. |
 | `target_times_last_evaluated` | datetime | The datetime the target times collection was last evaluated. This will occur if all previous target times are in the past and all values are available for the requested future time period. For example, if you are targeting 16:00 (day 1) to 16:00 (day 2), and you only have values up to 23:00 (day 1), then the target values won't be calculated. |
 | `calculate_with_incomplete_data` | boolean | Determines if calculations should occur when there isn't enough data to satisfy the look ahead hours |
+| `minimum_required_minutes_in_slot` | integer | Determines the configured minimum number of minutes to be present in a slot for it to be considered |
 
 ## Services
 

custom_components/target_timeframes/config/rolling_target_timeframe.py

@@ -2,19 +2,22 @@
 
 from ..const import (
   CONFIG_ROLLING_TARGET_HOURS_LOOK_AHEAD,
+  CONFIG_TARGET_DANGEROUS_SETTINGS,
   CONFIG_TARGET_HOURS,
   CONFIG_TARGET_HOURS_MODE,
   CONFIG_TARGET_HOURS_MODE_EXACT,
   CONFIG_TARGET_HOURS_MODE_MINIMUM,
   CONFIG_TARGET_MAX_VALUE,
   CONFIG_TARGET_MIN_VALUE,
+  CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT,
   CONFIG_TARGET_NAME,
   CONFIG_TARGET_OFFSET,
   CONFIG_TARGET_TYPE,
   CONFIG_TARGET_TYPE_CONTINUOUS,
   CONFIG_TARGET_WEIGHTING,
   REGEX_ENTITY_NAME,
   REGEX_HOURS,
+  REGEX_INTEGER,
   REGEX_OFFSET_PARTS,
   REGEX_VALUE,
   REGEX_WEIGHTING
@@ -133,4 +136,25 @@ def validate_rolling_target_timeframe_config(data):
   if CONFIG_TARGET_HOURS not in errors and CONFIG_ROLLING_TARGET_HOURS_LOOK_AHEAD not in errors and data[CONFIG_ROLLING_TARGET_HOURS_LOOK_AHEAD] < data[CONFIG_TARGET_HOURS]:
     errors[CONFIG_ROLLING_TARGET_HOURS_LOOK_AHEAD] = "look_ahead_hours_not_long_enough"
 
+  minimum_required_minutes_in_slot: int | None = None
+  if (CONFIG_TARGET_DANGEROUS_SETTINGS in data and
+      CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT in data[CONFIG_TARGET_DANGEROUS_SETTINGS] and 
+      data[CONFIG_TARGET_DANGEROUS_SETTINGS][CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT] is not None):
+    
+    if isinstance(data[CONFIG_TARGET_DANGEROUS_SETTINGS][CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT], int) == False:
+      if isinstance(data[CONFIG_TARGET_DANGEROUS_SETTINGS][CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT], str):
+        matches = re.search(REGEX_INTEGER, data[CONFIG_TARGET_DANGEROUS_SETTINGS][CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT])
+        if matches is None:
+          errors[CONFIG_TARGET_DANGEROUS_SETTINGS] = "invalid_integer"
+        else:
+          minimum_required_minutes_in_slot = int(data[CONFIG_TARGET_DANGEROUS_SETTINGS][CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT])
+          data[CONFIG_TARGET_DANGEROUS_SETTINGS][CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT] = minimum_required_minutes_in_slot
+      else:
+        errors[CONFIG_TARGET_DANGEROUS_SETTINGS] = "invalid_integer"
+    else:
+      minimum_required_minutes_in_slot = data[CONFIG_TARGET_DANGEROUS_SETTINGS][CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT]
+
+  if minimum_required_minutes_in_slot is not None and (minimum_required_minutes_in_slot < 1 or minimum_required_minutes_in_slot > 30):
+    errors[CONFIG_TARGET_DANGEROUS_SETTINGS] = "invalid_minimum_required_minutes_in_slot"
+
   return errors

custom_components/target_timeframes/config/target_timeframe.py

@@ -4,13 +4,15 @@
 from homeassistant.util.dt import (parse_datetime)
 
 from ..const import (
+  CONFIG_TARGET_DANGEROUS_SETTINGS,
   CONFIG_TARGET_END_TIME,
   CONFIG_TARGET_HOURS,
   CONFIG_TARGET_HOURS_MODE,
   CONFIG_TARGET_HOURS_MODE_EXACT,
   CONFIG_TARGET_HOURS_MODE_MINIMUM,
   CONFIG_TARGET_MAX_VALUE,
   CONFIG_TARGET_MIN_VALUE,
+  CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT,
   CONFIG_TARGET_NAME,
   CONFIG_TARGET_OFFSET,
   CONFIG_TARGET_START_TIME,
@@ -19,6 +21,7 @@
   CONFIG_TARGET_WEIGHTING,
   REGEX_ENTITY_NAME,
   REGEX_HOURS,
+  REGEX_INTEGER,
   REGEX_OFFSET_PARTS,
   REGEX_VALUE,
   REGEX_TIME,
@@ -162,4 +165,25 @@ def validate_target_timeframe_config(data):
     if is_time_frame_long_enough(data[CONFIG_TARGET_HOURS], start_time, end_time) == False:
       errors[CONFIG_TARGET_HOURS] = "invalid_hours_time_frame"
 
+  minimum_required_minutes_in_slot: int | None = None
+  if (CONFIG_TARGET_DANGEROUS_SETTINGS in data and
+      CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT in data[CONFIG_TARGET_DANGEROUS_SETTINGS] and 
+      data[CONFIG_TARGET_DANGEROUS_SETTINGS][CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT] is not None):
+    
+    if isinstance(data[CONFIG_TARGET_DANGEROUS_SETTINGS][CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT], int) == False:
+      if isinstance(data[CONFIG_TARGET_DANGEROUS_SETTINGS][CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT], str):
+        matches = re.search(REGEX_INTEGER, data[CONFIG_TARGET_DANGEROUS_SETTINGS][CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT])
+        if matches is None:
+          errors[CONFIG_TARGET_DANGEROUS_SETTINGS] = "invalid_integer"
+        else:
+          minimum_required_minutes_in_slot = int(data[CONFIG_TARGET_DANGEROUS_SETTINGS][CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT])
+          data[CONFIG_TARGET_DANGEROUS_SETTINGS][CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT] = minimum_required_minutes_in_slot
+      else:
+        errors[CONFIG_TARGET_DANGEROUS_SETTINGS] = "invalid_integer"
+    else:
+      minimum_required_minutes_in_slot = data[CONFIG_TARGET_DANGEROUS_SETTINGS][CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT]
+
+  if minimum_required_minutes_in_slot is not None and (minimum_required_minutes_in_slot < 1 or minimum_required_minutes_in_slot > 30):
+    errors[CONFIG_TARGET_DANGEROUS_SETTINGS] = "invalid_minimum_required_minutes_in_slot"
+
   return errors

custom_components/target_timeframes/const.py

@@ -40,6 +40,8 @@
 CONFIG_TARGET_TARGET_TIMES_EVALUATION_MODE_ALWAYS = "always"
 CONFIG_TARGET_DANGEROUS_SETTINGS = "dangerous_settings"
 CONFIG_TARGET_CALCULATE_WITH_INCOMPLETE_DATA = "calculate_with_incomplete_data"
+CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT = "minimum_required_minutes_in_slot"
+CONFIG_TARGET_DEFAULT_MINIMUM_REQUIRED_MINUTES_IN_SLOT = 29
 
 CONFIG_ROLLING_TARGET_HOURS_LOOK_AHEAD = "look_ahead_hours"
 
@@ -67,6 +69,7 @@
 REGEX_OFFSET_PARTS = "^(-)?([0-1]?[0-9]|2[0-3]):([0-5][0-9]):([0-5][0-9])$"
 REGEX_DATE = "^[0-9]{4}-[0-9]{2}-[0-9]{2}$"
 REGEX_VALUE = "^(-)?[0-9]+(\\.[0-9]+)*$"
+REGEX_INTEGER = "^(-)?[0-9]+$"
 
 REGEX_WEIGHTING_NUMBERS = "([0-9]+\\.?[0-9]*(,[0-9]+\\.?[0-9]*+)*)"
 REGEX_WEIGHTING_START = "(\\*(,[0-9]+\\.?[0-9]*+)+)"
@@ -123,6 +126,7 @@
     vol.Schema(
         {
             vol.Required(CONFIG_TARGET_CALCULATE_WITH_INCOMPLETE_DATA, default=False): bool,
+            vol.Required(CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT, default=CONFIG_TARGET_DEFAULT_MINIMUM_REQUIRED_MINUTES_IN_SLOT): int,
         }
     ),
     {"collapsed": True},
@@ -172,6 +176,7 @@
     vol.Schema(
         {
             vol.Required(CONFIG_TARGET_CALCULATE_WITH_INCOMPLETE_DATA, default=False): bool,
+            vol.Required(CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT, default=CONFIG_TARGET_DEFAULT_MINIMUM_REQUIRED_MINUTES_IN_SLOT): int,
         }
     ),
     {"collapsed": True},

custom_components/target_timeframes/entities/__init__.py

@@ -6,7 +6,7 @@
 
 from homeassistant.util.dt import (as_utc, parse_datetime)
 
-from ..const import CONFIG_TARGET_TARGET_TIMES_EVALUATION_MODE_ALL_IN_FUTURE_OR_PAST, CONFIG_TARGET_TARGET_TIMES_EVALUATION_MODE_ALL_IN_PAST, CONFIG_TARGET_TARGET_TIMES_EVALUATION_MODE_ALWAYS, CONFIG_TARGET_HOURS_MODE_EXACT, CONFIG_TARGET_HOURS_MODE_MAXIMUM, CONFIG_TARGET_HOURS_MODE_MINIMUM, CONFIG_TARGET_KEYS, REGEX_OFFSET_PARTS, REGEX_WEIGHTING
+from ..const import CONFIG_TARGET_DEFAULT_MINIMUM_REQUIRED_MINUTES_IN_SLOT, CONFIG_TARGET_TARGET_TIMES_EVALUATION_MODE_ALL_IN_FUTURE_OR_PAST, CONFIG_TARGET_TARGET_TIMES_EVALUATION_MODE_ALL_IN_PAST, CONFIG_TARGET_TARGET_TIMES_EVALUATION_MODE_ALWAYS, CONFIG_TARGET_HOURS_MODE_EXACT, CONFIG_TARGET_HOURS_MODE_MAXIMUM, CONFIG_TARGET_HOURS_MODE_MINIMUM, CONFIG_TARGET_KEYS, REGEX_OFFSET_PARTS, REGEX_WEIGHTING
 
 _LOGGER = logging.getLogger(__name__)
 
@@ -43,7 +43,7 @@ def is_target_timeframe_complete_in_period(current_date: datetime, start_time: d
     target_timeframes[-1]["end"] <= current_date
   )
 
-def get_start_and_end_times(current_date: datetime, target_start_time: str, target_end_time: str, start_time_not_in_past = True, context: str = None):
+def get_start_and_end_times(current_date: datetime, target_start_time: str, target_end_time: str, minimum_slot_minutes = None, context: str = None):
   if (target_start_time is not None):
     target_start = parse_datetime(current_date.strftime(f"%Y-%m-%dT{target_start_time}:00%z"))
   else:
@@ -64,10 +64,15 @@ def get_start_and_end_times(current_date: datetime, target_start_time: str, targ
     else:
       target_end = target_end + timedelta(days=1)
 
-  # If our start date has passed, reset it to current_date to avoid picking a slot in the past
-  if (start_time_not_in_past == True and target_start < current_date and current_date < target_end):
-    _LOGGER.debug(f'{context} - Rolling target and {target_start} is in the past. Setting start to {current_date}')
-    target_start = current_date
+  if (minimum_slot_minutes is not None and target_start < current_date and current_date < target_end):
+    current_date_start = current_date.replace(minute=30 if current_date.minute >= 30 else 0, second=0, microsecond=0)
+    minutes_remaining_in_current_slot = 30 - ((current_date.replace(second=0, microsecond=0) - current_date_start).total_seconds() / 60)
+    if (minutes_remaining_in_current_slot >= minimum_slot_minutes):
+      _LOGGER.debug(f'{context} - Current slot is sufficient for minimum slot minutes, so using current date start: {current_date_start}')
+      target_start = current_date_start
+    else:
+      target_start = current_date_start + timedelta(minutes=30)
+      _LOGGER.debug(f'{context} - Current slot is not sufficient for minimum slot minutes, so using next slot start: {target_start}')
 
   # If our start and end are both in the past, then look to the next day
   if (target_start < current_date and target_end < current_date):
@@ -98,15 +103,22 @@ def get_fixed_applicable_time_periods(target_start: datetime, target_end: dateti
 
   return applicable_rates
 
-def get_rolling_applicable_time_periods(current_date: datetime, time_period_values: list, target_hours: float, calculate_with_incomplete_data = False, context: str = None):
+def get_rolling_applicable_time_periods(current_date: datetime, 
+                                        time_period_values: list,
+                                        target_hours: float,
+                                        minimum_slot_minutes: int = CONFIG_TARGET_DEFAULT_MINIMUM_REQUIRED_MINUTES_IN_SLOT,
+                                        calculate_with_incomplete_data = False,
+                                        context: str = None):
   # Retrieve the rates that are applicable for our target rate
   applicable_time_periods = []
   periods = target_hours * 2
 
   if time_period_values is not None:
-    for rate in time_period_values:
-      if rate["end"] >= current_date:
-        new_rate = dict(rate)
+    for time_period in time_period_values:
+
+      minutes_remaining_in_current_slot = 30 - ((current_date.replace(second=0, microsecond=0) - time_period["start"]).total_seconds() / 60)
+      if minutes_remaining_in_current_slot >= minimum_slot_minutes and time_period["end"] >= current_date:
+        new_rate = dict(time_period)
         applicable_time_periods.append(new_rate)
 
         if len(applicable_time_periods) >= periods:

custom_components/target_timeframes/entities/rolling_target_timeframe.py

@@ -22,6 +22,8 @@
   CONFIG_ROLLING_TARGET_HOURS_LOOK_AHEAD,
   CONFIG_TARGET_CALCULATE_WITH_INCOMPLETE_DATA,
   CONFIG_TARGET_DANGEROUS_SETTINGS,
+  CONFIG_TARGET_DEFAULT_MINIMUM_REQUIRED_MINUTES_IN_SLOT,
+  CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT,
   CONFIG_TARGET_TARGET_TIMES_EVALUATION_MODE,
   CONFIG_TARGET_HOURS_MODE,
   CONFIG_TARGET_MAX_VALUE,
@@ -147,10 +149,15 @@ async def async_update(self):
         if CONFIG_TARGET_DANGEROUS_SETTINGS in self._config and CONFIG_TARGET_CALCULATE_WITH_INCOMPLETE_DATA in self._config[CONFIG_TARGET_DANGEROUS_SETTINGS]:
           calculate_with_incomplete_data = self._config[CONFIG_TARGET_DANGEROUS_SETTINGS][CONFIG_TARGET_CALCULATE_WITH_INCOMPLETE_DATA]
 
+        minimum_slot_minutes = CONFIG_TARGET_DEFAULT_MINIMUM_REQUIRED_MINUTES_IN_SLOT
+        if CONFIG_TARGET_DANGEROUS_SETTINGS in self._config and CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT in self._config[CONFIG_TARGET_DANGEROUS_SETTINGS]:
+          minimum_slot_minutes = self._config[CONFIG_TARGET_DANGEROUS_SETTINGS][CONFIG_TARGET_MINIMUM_REQUIRED_MINUTES_IN_SLOT]
+
         applicable_time_periods = get_rolling_applicable_time_periods(
           current_local_date,
           self._data_source_data,
           self._config[CONFIG_ROLLING_TARGET_HOURS_LOOK_AHEAD],
+          minimum_slot_minutes,
           calculate_with_incomplete_data,
           self._config[CONFIG_TARGET_NAME]
         )
@@ -332,5 +339,7 @@ def update_default_attributes(self):
     calculate_with_incomplete_data = False
     if CONFIG_TARGET_DANGEROUS_SETTINGS in self._config and CONFIG_TARGET_CALCULATE_WITH_INCOMPLETE_DATA in self._config[CONFIG_TARGET_DANGEROUS_SETTINGS]:
       calculate_with_incomplete_data = self._config[CONFIG_TARGET_DANGEROUS_SETTINGS][CONFIG_TARGET_CALCULATE_WITH_INCOMPLETE_DATA]
-      del self._attributes[CONFIG_TARGET_DANGEROUS_SETTINGS]
-    self._attributes[CONFIG_TARGET_CALCULATE_WITH_INCOMPLETE_DATA] = calculate_with_incomplete_data
+    self._attributes[CONFIG_TARGET_CALCULATE_WITH_INCOMPLETE_DATA] = calculate_with_incomplete_data
+
+    if CONFIG_TARGET_DANGEROUS_SETTINGS in self._attributes:
+      del self._attributes[CONFIG_TARGET_DANGEROUS_SETTINGS]