[schedy] Reworked rule selection algorithm

Author: Robert Schindler

docs/apps/schedy/CHANGELOG.md

@@ -9,19 +9,25 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 ## Unreleased
 
 ### Fixed
-* Fixed a bug in schedule.next_results() expression helper that caused
-  some result changes to be skipped.
-* Simplified the algorithm that decides whether a rule is active or not
-  at a given point in time. It now handles rules spanning multiple
-  days correctly.
+* Fixed a bug in schedule.next_results() expression helper that caused some result
+  changes to be skipped.
+* Simplified the algorithm that decides whether a rule is active or not at a given
+  point in time. It should now handle all rules spanning multiple days correctly.
 
 ### Security
 
 ### Added
 
 ### Changed
+* The ``start`` and ``end`` rule parameters now accept day shifts, deprecating the
+  former ``end_plus_days``.
+* Constraints of rules with a sub-schedule attached are now only validated for the
+  day at which a particular rule starts. Hence rules of such sub-schedules spanning
+  midnight will now run until they're intended to end.
 
 ### Deprecated
+* 0.6: The ``end_plus_days`` rule parameter will be removed in favor of the new day
+  shifts specified with ``start`` and ``end``.
 
 ### Removed
 

docs/apps/schedy/events.rst

@@ -65,7 +65,7 @@ control Schedy's behaviour.
 
 All events have an optional ``app_name`` parameter that can be submitted
 when you have multiple instances of Schedy running for different purposes
-and you want to address exactly one of these instances. It's value has
+and you want to address exactly one of these instances. Its value has
 to be the name of the app instance as configured in AppDaemon. If you
 omit this parameter, all Schedy instances will react to the event. The
 app name is the name you start the app's configuration with:

docs/apps/schedy/schedules/basics.rst

@@ -49,7 +49,7 @@ the 16 degrees-rule and Schedy evaluates rules from top to bottom. From
    the value to set. Consequently, you should design your schedules with
    the most specific rules at the top and gradually generalize to wider
    time frames towards the bottom. Finally, there should be a fallback
-   rule without time constraints at all to ensure you have no time slot
+   rule without time restrictions at all to ensure you have no time slot
    left without a value defined for.
 
 The ``name`` parameter we specified here is completely optional and
@@ -66,6 +66,8 @@ still can't create different schedules for, for instance, the days of
 the week. Let's do this next.
 
 
+.. _schedy/schedules/basics/constraints:
+
 Constraints
 -----------
 
@@ -83,13 +85,14 @@ Constraints
 
     - v: 15
 
-With your knowledge so far, this should be self-explanatory. The only
-new parameter is ``weekdays``, which is a so called constraint.
+With your knowledge so far, this should be self-explanatory. The only new parameter is
+``weekdays``, which is a so called constraint.
 
-Constraints can be used to limit the starting days on which the rule is
-considered. There are a number of these constraints, namely:
+Constraints can be used to limit the days on which the rule should start to be
+active. There are a number of these constraints, namely:
 
-* ``years``: limit the years (e.g. ``years: 2016-2018``
+* ``years``: limit the years (e.g. ``years: 2016-2018``); only years from 1970 to
+  2099 are supported
 * ``months``: limit based on months of the year (e.g.
   ``months: 1-3, 10-12`` for Jan, Feb, Mar, Oct, Nov and Dec)
 * ``days``: limit based on days of the month (e.g.
@@ -111,11 +114,12 @@ considered. There are a number of these constraints, namely:
   provided, the nearest prior valid date (namely 2018-02-28 in this
   case) is assumed.
 
-All constraints you define need to be fulfilled for the rule to match.
+A date needs to fulfill all constraints you defined for a rule to be considered
+active at that specific date.
 
-The format used to specify values for the first five types of constraints
-is similar to that of crontab files. We call it range specification,
-and only integers are supported, no decimal values.
+The format used to specify values for the first five types of constraints is similar
+to that of crontab files. We call it range specification, and only integers are
+supported, no decimal values.
 
 * ``x``: the single number ``x``
 * ``x-y`` where ``x < y``: range of numbers from ``x`` to ``y``,
@@ -129,15 +133,14 @@ and only integers are supported, no decimal values.
 * ... and so on
 * Any spaces are ignored.
 
-If an exclamation mark (``!``) is prepended to the range specification,
-it's values are inverted. For instance, the constraint ``weekdays:
-"!4-5,7"`` expands to ``weekdays: 1,2,3,6`` and ``months: "!3"`` is
-equivalent to ``months: 1-2,4-12``.
+If an exclamation mark (``!``) is prepended to the range specification, its values are
+inverted. For instance, the constraint ``weekdays: "!4-5,7"`` expands to ``weekdays:
+1,2,3,6`` and ``months: "!3"`` is equivalent to ``months: 1-2,4-12``.
 
 .. note::
 
-   The ``!`` sign has a special meaning in YAML, hence inverted
-   specifications have to be enclosed in quotes.
+   The ``!`` sign has a special meaning in YAML, hence inverted specifications have
+   to be enclosed in quotes.
 
 
 Rules Spanning Multiple Days
@@ -157,36 +160,56 @@ If you omit the ``start`` parameter, Schedy assumes that you mean midnight
 a rule that ends the same moment it starts at wouldn't make sense. We
 expect it to count for the whole day instead.
 
-In order to express what we actually want, there's another parameter named
-``end_plus_days`` to tell Schedy how many midnights there are between
-the start and end time. As we didn't specify this parameter explicitly,
-it's value is determined by Schedy. If the end time of the rule is prior
-or equal to its start time, ``end_plus_days`` is assumed to be
-``1``, otherwise ``0``.
-
-.. note::
-
-   The value of ``end_plus_days`` can't be negative, meaning you can't
-   span a rule backwards in time. Only positive integers and ``0``
-   are allowed.
+In order to express what we actually want, we'd have to set ``end`` to ``"00:00+1d"``,
+which tells Schedy that there is one midnight between the start and end times. For
+convenience, Schedy automatically assumes one midnight between start and end when
+you don't specify a number of days explicitly and the start time is prior or equal
+to the end time, as in our case.
 
 .. note::
 
-   You don't need to care about setting ``end_plus_days`` yourself,
-   unless one of your rules should span more than 24 hours, requiring
-   ``end_plus_days: 2`` or greater.
+   You don't need to care about setting ``+?d`` yourself unless one of your rules
+   should span more than 24 hours, requiring ``+1d`` or greater.
 
 Having written out what Schedy assumes automatically would result in
 the following rule, which behaves exactly identical to what we begun with.
 
 ::
 
-    - { v: 16, start: "0:00", end: "0:00", end_plus_days: 1 }
+    - { v: 16, start: "0:00", end: "0:00+1d" }
+
+.. note::
+
+   The rule has been rewritten to take just a single line. This is no
+   special feature of Schedy, it's rather normal YAML. But writing rules
+   this way is often more readable, especially if you need to create
+   multiple similar ones which, for instance, only differ in weekdays,
+   time or value.
+
+Let's get back to :ref:`schedy/schedules/basics/constraints` briefly. We know that
+constraints limit the days on which a rule starts to be active. This explanation is
+not correct in all cases, as you'll see now.
+
+There are some days, such as the last day of a month, which can't be expressed
+using constraints explicitly. To allow targeting such days anyway, the ``start``
+parameter of a rule accepts a day shifting suffix as well. Your constraints are
+checked for some date, but the rule starts being active some days earlier or later,
+relative to the matching date.
+
+Even though you can't specify the last day of a month, you can well specify the
+1st. This rule is active on the last day of February from 6.00 pm to 10.00 pm,
+no matter if in a leap year or not::
+
+    - { v: 22, start: "18:00-1d", end: "22:00", days: 1, months: 3 }
+
+This one even runs until March 1st, 10.00 pm::
+
+    - { v: 22, start: "18:00-1d", end: "22:00+1d", days: 1, months: 3 }
 
-Note how the rule has been rewritten to take just a single line. This is
-no special feature of Schedy, it's rather normal YAML. But writing rules
-this way is often more readable, especially if you need to create multiple
-similar ones which, for instance, only differ in weekdays, time or value.
+As you noted, the day shift of ``start`` can be negative as well, but not that of
+``end``, meaning your rules can't span backwards in time. This design decision was
+made in order to keep rules readable and the evaluation algorithm simple. It neither
+has a technical reason nor does it reduce the expressiveness of rules.
 
 
 .. _schedy/schedules/basics/rules-with-sub-schedules:

hass_apps/schedy/config.py

@@ -33,9 +33,10 @@ def build_schedule_rule(rule: dict) -> schedule.Rule:
             )
 
     kwargs = {
-        "start_time": rule["start"],
-        "end_time": rule["end"],
-        "end_plus_days": rule["end_plus_days"],
+        "start_time": rule["start"][0],
+        "start_plus_days": rule["start"][1],
+        "end_time": rule["end"][0],
+        "end_plus_days": rule["end"][1],
         "constraints": constraints,
         "expr": expr,
         "expr_raw": expr_raw,
@@ -151,20 +152,33 @@ def config_post_hook(cfg: dict) -> dict:
 
     return cfg
 
+
 def schedule_rule_pre_hook(rule: dict) -> dict:
-    """Copy value for the value key over from alternative names."""
+    """Copy value for the expression and value keys over from alternative names."""
 
     rule = rule.copy()
     util.normalize_dict_key(rule, "expression", "x")
     util.normalize_dict_key(rule, "value", "v")
+    # Merge the legacy end_plus_days field into end
+    end = rule.get("end")
+    end_plus_days = rule.pop("end_plus_days", None)
+    if isinstance(end_plus_days, int):
+        if end is None:
+            end = ""
+        if isinstance(end, str) and "+" not in end and "-" not in end:
+            if end_plus_days < 0:
+                end += "{}".format(end_plus_days)
+            else:
+                end += "-{}".format(end_plus_days)
+            rule["end"] = end
     return rule
 
 def validate_rule_paths(sched: schedule.Schedule) -> schedule.Schedule:
     """A validator to be run after schedule creation to ensure
     each path contains at least one rule with an expression or value.
     A ValueError is raised when this check fails."""
 
-    for path in sched.unfold():
+    for path in sched.unfolded:
         if path.is_final and not list(path.rules_with_expr_or_value):
             raise ValueError(
                 "No expression or value specified along the path {}."
@@ -196,6 +210,10 @@ def build_range_spec_validator(min_value: int, max_value: int) -> vol.Schema:
     vol.Match(util.TIME_REGEXP),
     util.parse_time_string,
 )
+TIME_PLUS_DAYS_VALIDATOR = vol.All(
+    vol.Match(util.TIME_PLUS_DAYS_REGEXP),
+    util.parse_time_plus_days_string,
+)
 
 # This schema does no real validation and default value insertion,
 # it just ensures a dictionary containing string keys and dictionary
@@ -265,10 +283,15 @@ def parse_watched_entity_str(value: str) -> T.Dict[str, T.Any]:
         "expression": str,
         "value": object,
         vol.Optional("name", default=None): vol.Any(str, None),
-        vol.Optional("start", default=None): vol.Any(TIME_VALIDATOR, None),
-        vol.Optional("end", default=None): vol.Any(TIME_VALIDATOR, None),
-        vol.Optional("end_plus_days", default=None):
-            vol.Any(vol.All(int, vol.Range(min=0)), None),
+        vol.Optional("start", default=(None, None)):
+            vol.Any((None, None), TIME_PLUS_DAYS_VALIDATOR),
+        vol.Optional("end", default=(None, None)): vol.Any(
+            (None, None),
+            vol.All(
+                TIME_PLUS_DAYS_VALIDATOR,
+                (object, vol.Any(None, vol.Range(min=0))),
+            ),
+        ),
         vol.Optional("years"): build_range_spec_validator(1970, 2099),
         vol.Optional("months"): build_range_spec_validator(1, 12),
         vol.Optional("days"): build_range_spec_validator(1, 31),