Next Release (#1322)

Author: BottlecapDave

.husky/prepare-commit-msg

@@ -1 +1 @@
-exec < /dev/tty && npx cz --hook || true
+# exec < /dev/tty && npx cz --hook || true

_docs/entities/octoplus.md

@@ -41,6 +41,9 @@ Binary sensor to indicate if a saving session that the account has joined is act
 
 The state of this sensor states when the saving session events were last updated. The attributes of this sensor exposes the joined and available saving sessions.
 
+!!! note
+    This will only be available if you have enrolled into Octoplus. Once enrolled, reload the integration to gain access to this sensor.
+
 | Attribute | Type | Description |
 |-----------|------|-------------|
 | `available_events` | `array` | The collection of saving session events that you haven't joined |
@@ -76,6 +79,9 @@ This will indicate the baseline consumption that you need to be below for the cu
 
 You can use the [current period consumption](./electricity.md#current-interval-accumulative-consumption) sensor (if available) to see how on track you are.
 
+!!! note
+    This will only be available if you have enrolled into Octoplus. Once enrolled, reload the integration to gain access to this sensor.
+
 !!! note
     This is [disabled by default](../faq.md#there-are-entities-that-are-disabled-why-are-they-disabled-and-how-do-i-enable-them). 
 
@@ -119,9 +125,8 @@ Each item within `baselines` consists of the following attributes
 
 Binary sensor to indicate if a free electricity session is active.
 
-!!! warning
-
-    This sensor uses public information supplied by https://github.com/BottlecapDave/OctopusEnergyApi. However it is only applicable to your account if you have joined Octoplus and have signed up to [free electricity sessions](https://octopus.energy/free-electricity/). Once enrolled into Octoplus, reload the integration to gain access to this sensor.
+!!! note
+    This will only be available if you have enrolled into Octoplus. Once enrolled, reload the integration to gain access to this sensor. This is only applicable if you have signed up to [free electricity sessions](https://octopus.energy/free-electricity/). This sensor uses public information supplied by https://github.com/BottlecapDave/OctopusEnergyApi.
 
 !!! note
     This is [disabled by default](../faq.md#there-are-entities-that-are-disabled-why-are-they-disabled-and-how-do-i-enable-them). 
@@ -145,6 +150,9 @@ Binary sensor to indicate if a free electricity session is active.
 
 The state of this sensor states when the free electricity session events were last updated. The attributes of this sensor exposes the past, present and future free electricity sessions.
 
+!!! note
+    This will only be available if you have enrolled into Octoplus. Once enrolled, reload the integration to gain access to this sensor. This is only applicable if you have signed up to [free electricity sessions](https://octopus.energy/free-electricity/). This sensor uses public information supplied by https://github.com/BottlecapDave/OctopusEnergyApi.
+
 !!! note
     This is [disabled by default](../faq.md#there-are-entities-that-are-disabled-why-are-they-disabled-and-how-do-i-enable-them). 
 
@@ -169,6 +177,9 @@ This will indicate the baseline consumption that you need to be above for the cu
 
 You can use the [current period consumption](./electricity.md#current-interval-accumulative-consumption) sensor (if available) to see how on track you are.
 
+!!! note
+    This will only be available if you have enrolled into Octoplus. Once enrolled, reload the integration to gain access to this sensor. This is only applicable if you have signed up to [free electricity sessions](https://octopus.energy/free-electricity/). This sensor uses public information supplied by https://github.com/BottlecapDave/OctopusEnergyApi.
+
 !!! note
     This is [disabled by default](../faq.md#there-are-entities-that-are-disabled-why-are-they-disabled-and-how-do-i-enable-them). 
 

_docs/faq.md

@@ -182,6 +182,19 @@ Cost sensors are calculated by multiplying the consumption by the rate for each
 
 Each half hour block has it's consumption rounded to the nearest 0.01kwh before multiplying by the rate, which is rounded to the nearest penny. The rounding method used is rounding half to even, where numbers ending in 5 are rounded up or down, towards the nearest even hundredth decimal place. As a result, 0.015 would be rounded up to 0.02, while 0.025 is rounded down to 0.02. This is based on [Octopus Energy API documentation](https://developer.octopus.energy/rest/guides/endpoints)
 
+## I'm having issues with a sensor. Is there any way I can see the attributes for a certain point in time?
+
+The majority of attributes for entities are stored in the database for a short amount of time (default is around 10 days). Unfortunately, the only way of obtaining historic attributes is via the database. This can be done via the [SQLite Web Add-On](https://my.home-assistant.io/redirect/supervisor_addon/?addon=a0d7b954_sqlite-web), where the following SQL query can be executed. You'll need to change the target entity id from `binary_sensor.octopus_energy_xxx_intelligent_dispatching` to the one you want to target.
+
+```
+SELECT states_meta.entity_id, states.state, state_attributes.shared_attrs, DATETIME(last_changed_ts, 'unixepoch'), DATETIME(last_updated_ts, 'unixepoch')
+FROM "states"
+join state_attributes on states.attributes_id = state_attributes.attributes_id
+join states_meta on states_meta.metadata_id = states.metadata_id
+where states_meta.entity_id = 'binary_sensor.octopus_energy_xxx_intelligent_dispatching'
+order by last_updated_ts desc
+```
+
 ## Do you support older versions of the integration?
 
 Due to time constraints, I will only ever support the latest version of the integration. If you have an issue with an older version of the integration, my initial answer will always be to update to the latest version. This might be different to what HACS is reporting if you are not on the minimum supported Home Assistant version (which is highlighted in each release's changelog). 

_docs/repairs/no_active_tariff.md

@@ -3,7 +3,8 @@
 If you receive this repair notice it means that no active tariff was found for one or more of your meters. There are a few reasons why this might be the case.
 
 1. You are in the middle of switching tariffs. There can sometimes be a gap in your tariff information coming through when your existing tariff ends and your new tariff begins.
-2. You are new to OE and your tariff hasn't begun yet. Once your tariff starts, the integration will pick it up and should start working.
-3. There is a configuration issue with your account. In this scenario, you'll need to contact Octopus Energy support to get this rectified.
+2. You have had a meter swap. When your meter is replaced, it can take time for systems to fully update, leaving a gap in tariff information until this is complete.
+3. You are new to OE and your tariff hasn't begun yet. Once your tariff starts, the integration will pick it up and should start working.
+4. There is a configuration issue with your account. In this scenario, you'll need to contact Octopus Energy support to get this rectified.
 
-All of the above scenarios can be confirmed within your [diagnostic data](../faq.md#ive-been-asked-for-my-meter-information-in-a-bug-request-how-do-i-obtain-this). This will contain your account data including your meters. Under each of your meters, you'll have a list of tariff agreements. For each meter, you should have an agreement with a start date in the past and an end date either in the future or with no end date.
+All of the above scenarios can be confirmed within your [diagnostic data](../faq.md#ive-been-asked-for-my-meter-information-in-a-bug-request-how-do-i-obtain-this). This will contain your account data including your meters. Under each of your meters, you'll have a list of tariff agreements. For each meter, you should have an agreement with a start date in the past and an end date either in the future or with no end date.

_docs/setup/cost_tracker.md

@@ -88,6 +88,7 @@ Each item within the `tracked_changes` and `untracked_changes` have the followin
 | `rate` | `float` | The rate the consumption is charged at. This is in pounds and pence (e.g. 1.01 = £1.01) |
 | `consumption` | `float` | The consumption value of the specified period. This will be in `kwh`. |
 | `cost` | `float` | The cost of the consumption at the specified rate. This is in pounds and pence (e.g. 1.01 = £1.01) |
+| `cost_raw` | `float` | The raw cost of the consumption at the specified rate. This is in pounds and pence, but not rounded. This is to account for low cost devices |
 
 #### Variants
 

custom_components/octopus_energy/api_client/__init__.py

@@ -223,27 +223,21 @@
 }}'''
 
 intelligent_turn_on_bump_charge_mutation = '''mutation {{
-	triggerBoostCharge(
-    input: {{
-      accountNumber: "{account_id}"
-    }}
-  ) {{
-		krakenflexDevice {{
-			 krakenflexDeviceId
-		}}
-	}}
+	updateBoostCharge(input: {{
+    deviceId: "{device_id}"
+    action: BOOST
+  }}) {{
+    id
+  }}
 }}'''
 
 intelligent_turn_off_bump_charge_mutation = '''mutation {{
-	deleteBoostCharge(
-    input: {{
-      accountNumber: "{account_id}"
-    }}
-  ) {{
-		krakenflexDevice {{
-			 krakenflexDeviceId
-		}}
-	}}
+	updateBoostCharge(input: {{
+    deviceId: "{device_id}"
+    action: CANCEL
+  }}) {{
+    id
+  }}
 }}'''
 
 intelligent_turn_on_smart_charge_mutation = '''mutation {{
@@ -1593,7 +1587,7 @@ async def async_update_intelligent_car_target_time(
       raise TimeoutException()
 
   async def async_turn_on_intelligent_bump_charge(
-      self, account_id: str,
+      self, device_id: str,
     ):
     """Turn on an intelligent bump charge"""
     await self.async_refresh_token()
@@ -1603,7 +1597,7 @@ async def async_turn_on_intelligent_bump_charge(
       client = self._create_client_session()
       url = f'{self._base_url}/v1/graphql/'
       payload = { "query": intelligent_turn_on_bump_charge_mutation.format(
-        account_id=account_id,
+        device_id=device_id,
       ) }
 
       headers = { "Authorization": f"JWT {self._graphql_token}", integration_context_header: request_context }
@@ -1615,7 +1609,7 @@ async def async_turn_on_intelligent_bump_charge(
       raise TimeoutException()
 
   async def async_turn_off_intelligent_bump_charge(
-      self, account_id: str,
+      self, device_id: str,
     ):
     """Turn off an intelligent bump charge"""
     await self.async_refresh_token()
@@ -1625,7 +1619,7 @@ async def async_turn_off_intelligent_bump_charge(
       client = self._create_client_session()
       url = f'{self._base_url}/v1/graphql/'
       payload = { "query": intelligent_turn_off_bump_charge_mutation.format(
-        account_id=account_id,
+        device_id=device_id,
       ) }
 
       headers = { "Authorization": f"JWT {self._graphql_token}", integration_context_header: request_context }

custom_components/octopus_energy/api_client/heat_pump.py

@@ -11,9 +11,9 @@ class Connectivity(BaseModel):
 
 
 class Telemetry(BaseModel):
-    temperatureInCelsius: float
+    temperatureInCelsius: Optional[float]
     humidityPercentage: Optional[float]
-    retrievedAt: str
+    retrievedAt: Optional[str]
 
 
 class Sensor(BaseModel):

custom_components/octopus_energy/cost_tracker/__init__.py

@@ -1,11 +1,17 @@
 from datetime import datetime, timedelta
+import logging
 
 from homeassistant.components.sensor import (
   SensorStateClass,
 )
 
 from homeassistant.helpers.entity import DeviceInfo
 
+from ..utils.conversions import pence_to_pounds_pence, round_pounds, value_inc_vat_to_pounds
+from ..utils.cost import consumption_cost_in_pence
+
+_LOGGER = logging.getLogger(__name__)
+
 def get_device_info_from_device_entry(device_entry):
   if device_entry is None:
     return None
@@ -152,4 +158,82 @@ def accumulate_cost(current: datetime, accumulative_data: list, new_cost: float,
 
   return AccumulativeCostTrackerResult(new_accumulative_data, total_consumption, total_cost)
 
-  
+def __get_to(item):
+    return (item["end"].timestamp(), item["end"].fold)
+
+def __sort_consumption(consumption_data):
+  sorted = consumption_data.copy()
+  sorted.sort(key=__get_to)
+  return sorted
+
+def calculate_consumption_and_cost(
+    consumption_data,
+    rate_data,
+    standing_charge,
+    last_reset,
+    minimum_consumption_records = 0,
+    target_rate = None
+  ):
+  if (consumption_data is not None and len(consumption_data) >= minimum_consumption_records and rate_data is not None and len(rate_data) > 0 and standing_charge is not None):
+
+    sorted_consumption_data = __sort_consumption(consumption_data)
+
+    # Only calculate our consumption if our data has changed
+    if (last_reset is None or last_reset < sorted_consumption_data[0]["start"]):
+
+      charges = []
+      total_cost = 0
+      total_consumption = 0
+
+      for consumption in sorted_consumption_data:
+        consumption_value = consumption["consumption"]
+        consumption_from = consumption["start"]
+        consumption_to = consumption["end"]
+
+        try:
+          rate = next(r for r in rate_data if r["start"] == consumption_from and r["end"] == consumption_to)
+        except StopIteration:
+          raise Exception(f"Failed to find rate for consumption between {consumption_from} and {consumption_to}")
+
+        value = rate["value_inc_vat"]
+
+        if target_rate is not None and value != target_rate:
+          continue
+
+        total_consumption = total_consumption + consumption_value
+        cost = pence_to_pounds_pence(consumption_cost_in_pence(consumption_value, value))
+        cost_raw = (consumption_value * value) / 100
+        total_cost = total_cost + cost_raw
+
+        current_charge = {
+          "start": rate["start"],
+          "end": rate["end"],
+          "rate": value_inc_vat_to_pounds(value),
+          "consumption": consumption_value,
+          "cost": cost,
+          "cost_raw": cost_raw,
+        }
+
+        charges.append(current_charge)
+      
+      total_cost = round_pounds(total_cost)
+      total_cost_plus_standing_charge = total_cost + pence_to_pounds_pence(standing_charge)
+
+      last_reset = sorted_consumption_data[0]["start"] if len(sorted_consumption_data) > 0 else None
+      last_calculated_timestamp = sorted_consumption_data[-1]["end"] if len(sorted_consumption_data) > 0 else None
+
+      result = {
+        "standing_charge": pence_to_pounds_pence(standing_charge),
+        "total_cost_without_standing_charge": total_cost,
+        "total_cost": total_cost_plus_standing_charge,
+        "total_consumption": total_consumption,
+        "last_reset": last_reset,
+        "last_evaluated": last_calculated_timestamp,
+        "charges": charges,
+      }
+
+      return result
+    else:
+      _LOGGER.debug(f'Skipping consumption and cost calculation as last reset has not changed - last_reset: {last_reset}; consumption start: {sorted_consumption_data[0]["start"]}')
+  else:
+    _LOGGER.debug(f'Skipping consumption and cost calculation due to lack of data; consumption: {len(consumption_data) if consumption_data is not None else 0}; rates: {len(rate_data) if rate_data is not None else 0}; standing_charge: {standing_charge}')

custom_components/octopus_energy/cost_tracker/cost_tracker.py

@@ -41,7 +41,7 @@
 
 from ..coordinators.electricity_rates import ElectricityRatesCoordinatorResult
 from . import add_consumption, get_device_info_from_device_entry
-from ..electricity import calculate_electricity_consumption_and_cost
+from ..cost_tracker import calculate_consumption_and_cost
 from ..utils.rate_information import get_rate_index, get_unique_rates
 from ..utils.attributes import dict_to_typed_dict
 
@@ -283,7 +283,7 @@ def _recalculate_cost(self, current: datetime, tracked_consumption_data: list, u
       unique_rate_index = get_rate_index(len(unique_rates), self._peak_type)
       target_rate = unique_rates[unique_rate_index] if unique_rate_index is not None else None
 
-    tracked_result = calculate_electricity_consumption_and_cost(
+    tracked_result = calculate_consumption_and_cost(
       tracked_consumption_data,
       rates,
       0,
@@ -292,7 +292,7 @@ def _recalculate_cost(self, current: datetime, tracked_consumption_data: list, u
       target_rate=target_rate
     )
 
-    untracked_result = calculate_electricity_consumption_and_cost(
+    untracked_result = calculate_consumption_and_cost(
       untracked_consumption_data,
       rates,
       0,
@@ -309,15 +309,17 @@ def _recalculate_cost(self, current: datetime, tracked_consumption_data: list, u
         "end": charge["end"],
         "rate": charge["rate"],
         "consumption": charge["consumption"],
-        "cost": charge["cost"]
+        "cost": charge["cost"],
+        "cost_raw": charge["cost_raw"]
       }, tracked_result["charges"]))
 
       self._attributes["untracked_charges"] = list(map(lambda charge: {
         "start": charge["start"],
         "end": charge["end"],
         "rate": charge["rate"],
         "consumption": charge["consumption"],
-        "cost": charge["cost"]
+        "cost": charge["cost"],
+        "cost_raw": charge["cost_raw"]
       }, untracked_result["charges"]))
 
       self._attributes["total_consumption"] = tracked_result["total_consumption"] + untracked_result["total_consumption"]

custom_components/octopus_energy/electricity/previous_accumulative_cost_override.py

@@ -50,6 +50,8 @@ def __init__(self, hass: HomeAssistant, account_id: str, coordinator, client: Oc
     self._next_refresh = None
     self._last_retrieved  = None
     self._request_attempts = 1
+    self._rates = None
+    self._standing_charge = None
 
     CoordinatorEntity.__init__(self, coordinator)
     OctopusEnergyElectricitySensor.__init__(self, hass, meter, point)
@@ -124,18 +126,28 @@ async def async_update(self):
       target_tariff_code = self._config[CONFIG_TARIFF_COMPARISON_TARIFF_CODE]
 
       try:
-        _LOGGER.debug(f"Retrieving rates and standing charge overrides for '{self._mpan}/{self._serial_number}' ({period_from} - {period_to})...")
-        [rate_data, standing_charge] = await asyncio.gather(
-          self._client.async_get_electricity_rates(target_product_code, target_tariff_code, self._is_smart_meter, period_from, period_to),
-          self._client.async_get_electricity_standing_charge(target_product_code, target_tariff_code, period_from, period_to)
-        )
+        if (self._rates is None or
+            self._standing_charge is None or
+            self._rates[0]["start"] != period_from or
+            self._rates[-1]["end"] != period_to or
+            self._standing_charge["start"] != period_from or
+            self._standing_charge["end"] != period_to):
+
+          _LOGGER.debug(f"Retrieving rates and standing charge overrides for '{self._mpan}/{self._serial_number}' ({period_from} - {period_to})...")
+          [rate_data, standing_charge] = await asyncio.gather(
+            self._client.async_get_electricity_rates(target_product_code, target_tariff_code, self._is_smart_meter, period_from, period_to),
+            self._client.async_get_electricity_standing_charge(target_product_code, target_tariff_code, period_from, period_to)
+          )
+
+          self._rates = rate_data
+          self._standing_charge = standing_charge
 
-        _LOGGER.debug(f"Rates and standing charge overrides for '{self._mpan}/{self._serial_number}' ({period_from} - {period_to}) retrieved")
+          _LOGGER.debug(f"Rates and standing charge overrides for '{self._mpan}/{self._serial_number}' ({period_from} - {period_to}) retrieved")
 
         consumption_and_cost = calculate_electricity_consumption_and_cost(
           consumption_data,
-          rate_data,
-          standing_charge["value_inc_vat"] if standing_charge is not None else None,
+          self._rates,
+          self._standing_charge["value_inc_vat"] if self._standing_charge is not None else None,
           None
         )
 
@@ -170,7 +182,7 @@ async def async_update(self):
                                       "serial_number": self._serial_number,
                                       "product_code": target_product_code,
                                       "tariff_code": target_tariff_code,
-                                      "rates": private_rates_to_public_rates(rate_data) 
+                                      "rates": private_rates_to_public_rates(self._rates) 
                                     }))
 
           self._request_attempts = 1