subtasks proposal

[ferdinandyb]

This proposal tries to argue for a subtasks feature.

Current methods to approximate

A task-subtasks pairings generally would want to do two things:
- handle a group of tasks together for certain purposes
- describe a group of tasks together
- introduce a hierarchy among tasks for various reasons (even just ease of
reasoning about

Some of the above can be done via existing features, but not all of them together.

• projects can introduce hierarchy, but requires to manually create
  sub-projects at all steps
• projects and tags can be used to operate on a group of tasks together, but
  only incidentally, i.e. you can group-modify tasks, but the modifications are
  essentially separate
• blocking can be used as a sort of parent-child relationship, but the
  transitive properties of this is not, what one would expect of subtasks (if
  P is blocked by C1 and C2, then a T task blocking P essentially becomes C3,
  instead of blocking all of P, C1 and C2).

Example and usecase

You're planning a wedding, with the following high level tasks:

• T1: send invitations
• T2: fix venue (and date)
• T3: fix catering
• T4: buy décor

T1 has a bunch of subtasks (an order of magnitude more than high level tasks), namely each person you need to invite (In). T4 also has two subtasks, S1: things you can buy locally and S2: things you need a car for.

There is also an unrelated task U1: your car is broken and needs to be fixed.

T2 blocks T1 and thus all of In. U1 blocks S2. Décor needs to be bought while it's cheap, so before December.

Things I would want to do is,

• set blockings easily
• view the high level tasks only (otherwise In would clog the view)
• set due dates easily

Let's see how we can approximate this and the problems.

I make projects like wedding.send_invitations

• T2 cannot block In because it's not a task, if I make it a task, it falls out of the high level overview
• you are severely limited in what you can put in the "description" of a high level task since it needs to be a valid project name
• the only purpose this would solve that you can add the due date to S1 and S2 in one command

Tagging

It has similar limitations, but somewhat worse, because there is no hierarchy.

Using blockings

You imitate subtasks, by setting subtasks as blocking for their parent, that is all of In blocks T1, S1 and S2 block T4. The problems:

• it's possible, but extremely awkward to query programatically (for mass modification) all the blockers of a task, so I'd need to somewhat manually set up T2 blocking In
• in this view U1 is actually a subtask of S2
• in this view T2 is actually a subtask for each In
• it's not possible to query high level tasks, since being blocked does not mean this (In would show up being blocked but T2 would not)

Proposed solution

Introduce a new property to track parent-child relationships (an added benefit of this would be that this is what many caldav servers do for VTODO, so people syncing tasks to caldav could trivially map this property as well), somewhat similiarly how blocking works.

Unfortunately, parent is already taken, so maybe upstream or childof?

Properties

Maybe not all of these make sense, but these are thing I have in mind:

• a task may have only one upstream, DAG must be enforced
• I'd allow arbitrary depth, if there is a technical limitation, then a large enough so it's basically arbitrary
• if an upstream is blocked, all downstream tasks are also blocked
• computed attribute: depth (how far down a task is)
• ability to query all subtasks (for modifying together)
• new virtual tags UPSTREAM and DOWNSTREAM
• date properties of an upstream must sensibly restrict the downstream, i.e. a subtask's due date must not be later than the upstream's
• downstreams could (should?) inherit upstream's project, maybe allowing for being in a subproject
• downstreams could (should?) inherit upstream's set of tags, allowing for adding new ones, but not removing ones one the upstream (edit: this probably makes the least sense)

Obviously, if an upstream is changed (e.g. tags) or an upstream gets a new upstream things would need to be propagated. But implementing all of the above could be broken up into smaller steps easily, with enforcement of certain things, especially propagating updates could be implemented later.

Impact on downstream applications

Shouldn't be anything since the new subtasks are still just tasks, although I assume things like vit would want to visually put subtasks somehow under their parents.

[Necior]

Thank you for writing the proposal!

A task-subtasks pairings generally would want to do two things:

• handle a group of tasks together for certain purposes
• describe a group of tasks together
• introduce a hierarchy among tasks for various reasons (...)

Providing examples for each point and comparing using subtasks vs using existing mechanisms (e.g. dependencies or projects) would be helpful here. Personally, I'm mostly interested in specific use cases.

---

The main theme in the proposal is some kind of inheritance, propagation and restriction of attributes based on the parent-child relation. The tradeoff I see is between compactness (change one task's attributes and let the software propagate the change to its subtasks) and complexity (users need to remember how each attribute interacts subtasks).

If I tag a parent-task as +large, I don't want subtasks (which are smaller) to inherit this tag.

---

• Are subtasks of a task ordered? Are they expected to be completed in order?
• Can a parent-task be blocked by its subtask?
• Does completing all subtasks complete the parent task?
• Which existing commands should modify their behavior because of subtasks?

[ferdinandyb]

Thanks for the feedback! I added an example. You are probably quite right about tags. I've also been thinking that although date restrictions make sense in a way, introducing such restriction and propagation might actually be overcomplicating things, and maybe it could be something that is only taken into consideration when creating a subtask (e.g. copy due date). The only propagation I see necessary is blocking. I'd probably copy project and dates on create, but then leave it at that, but maybe only dates, who knows what people do with projects. Maybe even the dates are not necessary, as people could just write hooks to copy or not copy what they think they need.

Are subtasks of a task ordered? Are they expected to be completed in order?

Definitely no.

Can a parent-task be blocked by its subtask?

I don't think this makes sense, I mean obviously it's "blocking" from finishing it, but in terms of taswarrior blockers, this would be a different semantic

Does completing all subtasks complete the parent task?

This seems like a similar convenience which I had in mind with tagging, but you had a nice example of why you'd not want that, so I'd say no, or at least not by default. Maybe a virtual tag of that is essentially "ALLSUBTASKsCOMPLETE" could help though.

Which existing commands should modify their behavior because of subtasks?

This is a tougher one to list, but first try:

• we'd need a new filter to query all subtasks of a task
• add needs get a way of adding this
• list might want to be able to group subtasks visually under their parent
• urgency calculations need to be thought about
• modify would need to propagate at least blocking (at this point I'm more or less convinced that this is the only one needed)

Just a note, that dependencies via blocking is pretty hard to query: https://stackoverflow.com/a/66419579/2241241

[djmitche]

From a data-structures perspective, note that Taskwarrior's database is distributed and only eventually consistent. So global properties like "max depth" or "DAG" can't be enforced. As an example, consider task A -> B and C -> D. On replica 1, I add a dependency B -> C, and on replica 2 I add a dependency D -> A. On each of those replicas, no cycle is created, but after the changes are synchronized, the four tasks form a cycle.

Dependencies have caused a bit of a performance issue, in the sense that calculating a tasks' dependees (e.g., to determine +BLOCKING) requires scanning the database. Dependencies are, luckily, only allowed among pending tasks, so we can limit the scan to the working set. We use a helper data structure (DependencyMap) to make this a bit more efficient. Thinking about how to handle this in advance would be useful.

[djmitche]

That's right - you won't be able to rely on the graph being acyclic.

There's no check for a circular dependency chain, and I'm not sure a warning when one is discovered would be especially useful. However, if it's desirable then it could be added to task diag.

And, yes, if this hierarchy applies to all tasks, then there may be a performance hit. We have users with 100,000's of tasks, and scanning that many tasks on every run of the binary is very slow. In terms of performance, ideally the new functionality would only incur a performance penalty when it's used. For example, that might mean avoiding virtual tags (which are calculated on every report that shows tags, which is most of them).

A PoC is a good next step, and perhaps a document of some sort that we can share more broadly for review. We should think about what kinds of use-cases this supports, and what kinds of use-cases it might make more difficult.

[ferdinandyb]

I could copy (and touch up) the discussion into some other format, what would be the preference?

Re.: performance, that would probably mean that we'd need to keep a list of downstreams on the upstreams (basically book-keep the relationship in both places). That would mean that if a user does not use the feature, both fields will be empty, and we will know that there's nothing to be done. I.e. I assume a report showing tags could first pull up the tasks for the report and then just check the tasks themselves, no need to look through the entire DB to see if one of those tasks actually has a subtask.

[djmitche]

A Gist can be a good approach, or a google doc or whatever. The important thing is that it's formatted as a proposal and not a discussion, so it's clear what is being proposed rather than what has been discussed.

It is probably worth taking a closer look at the task model in TaskChampion. Modifications to properties occur in "operations", and Update always overwrites an existing value. So, we can't store "lists" of things. Consider a task with a "things" property containing "1, 2, 3". On replica A we read the list, add a new item to the end, and write it back in an Update operation with value "1, 2, 3, 10". On Replica B we read the list, add a new item to the end, and write it back with an Update operation with value "1, 2, 3, 20". When synchronizing, whichever of those operations occurred with a later timestamp - say, replica B's -- will win. The other will be lost. So the result would be "1, 2, 3, 20" and not the expected "1, 2, 3, 10, 20". The approach to address this is to store the list key in the property, and we've done this for dependencies (properties named dep_<uuid>), tags (tag_<name>), and annotations (annotation_<timestamp>). So, I think you'd want to do a similar thing here.

There are two other, related issues. First, in a distributed system like this it's possible that the double-bookkeeping would fail, leaving an upstream pointing downward without a corresponding upward pointer, or the reverse. So, the implementation would need to be resilient to this circumstance, and choose how to interpret it -- either re-adding the missing pointer, or deleting the un-paired pointer. The related issue is that the TaskChampion task model requires that any values for a task be treated as valid, so the implementation must, in fact, handle any combination of tasks and properties reasonably (that is, without an error).

These, and the performance issues, aren't insurmountable, but need to be addressed in the proposal.

[ferdinandyb]

At this point, I think I'll come back with a doc and some sort of PoC, just so that I better understand the current architecture.

[ferdinandyb]

Life is currently rather demanding of my time, so I'll probably only have time a couple of weeks from now for this. Just FYI.

[ferdinandyb]

@ryneeverett opening a new thread for the "how many upstream" topic (the DOM already has a parent for dependency so I'm gonna stick to upstream for now). Good question, because I mostly went intuitive there. Thinking aloud here:

So a simplified data model might have some benefits:

• this is just a hunch, but I think any simplification you can make to the data model probably has a positive performance impact, although, although it may be negligible
• inheritance definitely becomes more challenging if there isn't a single upstream
  • if the upstreams' projects are different, which project do you put them under?
  • calculating a depth becomes tricky (is it the shortest path or the longest or do you calculate an average?)

I tried to think of tricky blocking scenarios, but I don't think there's an issue there.

A somewhat unrelated question came to my mind: if all subtasks of a task are blocked should we block the task as well? But probably the answer is the same to whether we should complete it when all subtasks are completed (I'd say no).

[ryneeverett]

From my perspective, the simplest solution is to make this feature as similar to depends as possible, which is why I'm trying to understand the benefit of the divergence.

if the upstreams' projects are different, which project do you put them under?

Both. Is that a problem?

As an aside, I can think of three different ways inheritance could work which has an impact on the implementation challenges:

a) dynamic data inheritance: if an attribute changes on the parent it is propagated to existing children
b) static data inheritance: as a convenience certain attributes are propagated to children at the moment when the subtask relationship is initiated
c) virtual inheritance: frontends (like taskwarrior) can choose to treat certain attributes as inherited but that is not reflected in the database

My initial thought is that c would be optimal if it can be sufficiently performant.

calculating a depth becomes tricky (is it the shortest path or the longest or do you calculate an average?)

Can you explain why depth calculation is helpful?

[ryneeverett]

A similar issue I've been running into is that our notion of dependencies and blocking is insufficiently nuanced because tasks can have dependency relations without being blocking. A dependency relation indicates that the dependent task cannot be completed until the needed task is completed, while a blocking relation indicates that the dependent task cannot be started until the needed task is completed. It seems like the current UI conflates these two ideas.

[ryneeverett]

Ok, now I'm realizing that "non-blocking dependency" might be identical with this notion of parent/child relationships as long as multiple parents are allowed.

I also think the issue I'm describing might be the same as the "transitive blocking" bullet point in the original proposal.

[ferdinandyb]

Thanks, that's an interesting way to look at this. Life has been happening, but I hope to get back to this in about 2-3 week, and try to come up with a PoC.