import tasks from TW2 to TW3, preserving change history and timestamps #3835
Replies: 1 comment 5 replies
-
|
Wow, you've handled a number of ugly edge-cases in that script -- nice work! It's worth noting that https://github.com/GothenburgBitFactory/taskchampion-py exists, and it would have been possible for the script to interact directly with the replica via that package. In fact, that's probably a better option than adding complex functionality to |
Beta Was this translation helpful? Give feedback.
-
|
I did consider using the python interface, but this was something quick I could hack up which worked perfectly to let me move to tw3 (no such tool exists atm), preserving the change info, without learning about sync, taskchampion, or that whole interface.
I don't think it's complex to allow a time override rather than requiring modifications to always use the current time. In general, I don't think users should be required to use a programming language to update the task database with an arbitrary modification time... there is already the JSON import/export mechanism, but currently the update timestamps are stored magically out of band somewhere inaccessible to the ordinary user.
Indeed, eventually I'll migrate to use taskchampion directly and bypass taskwarrior where appropriate. There are a lot of possibilities, and in general I like and support the direction of the project now and the recent changes, including both sqlite3 and Rust. Hopefully at some point we can support more relational representations with certain backends like sqlite3, using columns for the tasks rather than serialized records, and taking advantage of some things that map to SQL well for queries, maybe using FTS5 for some searches of notes/description. But storing as serialized data is not so bad because the human task set is small and it's not so much computation or I/O to just re-read/recompute everything all the time. Actually I'm a heavy user of Timewarrior also and the hooks system, that's still all JSON based without any sync integration either so that's another area for me to explore later. |
Beta Was this translation helpful? Give feedback.
-
I think this is probably an implementation detail that most other users will not need or really understand how to use. But you're right that it's easy and unobtrusive enough to add, so I'd be happy to have a PR to do so. There's another issue, or discussion, somewhere about supporting a more sophisticated query language in TaskChampion. The trick is making it general enough that it can be used with multiple backends, but specific enough that it is useful. Maybe that was a conversation with you? I can't find it in a big of searching.. |
Beta Was this translation helpful? Give feedback.
-
Yeah, actually I find it puzzling that no one seems to care about losing their change logs, that represent potentially years of work, in the evolution of a large task. I often make use of the change log; the time that particular tags get added or removed is especially interesting. But you're right, and nobody else commented, so possibly I'm the only one that cares about change history :-)
Actually, when I saw your recent ImportV2, looking at that code area, I realized we could already do an auto-import from the undo file and preserve everything, rather than an "ordinary" import from completed/pending .data files. Most of the functions to read the undo data are already present in the codebase. However my C++ is weak, and so far nobody cares about the change log, so... However, I do think it's worth considering that task changes and timestamps should really be first class citizens, tied to the task. It seems like a lot of information to lose upon import/export!
Good memory, I see #2771 where this was discussed, and I also recall a more recent discussion where someone mentioned supporting more advanced operations if the backend allowed it and then falling back to serialized records if it didn't. I also can't find that one after searching... |
Beta Was this translation helpful? Give feedback.
-
|
As far as TaskChampion is concerned, changes are no first-class citizens (or at least almost - there are some approximations required in unusual sync situations). So that's good news :) |
Beta Was this translation helpful? Give feedback.
-
|
I just realized I made an important typo here -- should be "changes are now first-class citizens" |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
As someone who refers to task change history often, I was blocked from upgrading to taskwarrior3 until
journal.infosupport was [re]implemented recently (see #3671) (thanks @djmitche!)However, there's still the problem that dump+load to migrate would not preserve change histories, as individual changes are not stored in a JSON export, only the end-state. But then I noticed that the transaction log,
undo.data, does contain all information on all changes, including timestamps for each modification. It means full transaction history could be reconstructed using theundo.datarecords.I therefore made a script to import from the undo.log using individual imports wrapped by
faketime. Using this method, I get a fully correct import with preserved task change histories. Now, withrc.journal.info=1, resulting v3 task info is very close to the original, with no information loss. My undo data had 5335 records, which took 236 seconds to import with an older laptop i7.In case anyone else values their task change history and hasn't upgraded yet for that reason, I wanted to mention the script here so it could potentially help others. Hope someone else finds it useful!
P.S. I do think there should be some kind of command line flag to
task importthat lets one force the time of the operation without needing to wrap withfaketime. Even better, a special key in each JSON record to describe the time of the update (defaulting to the current time) would allow batched imports rather than repeated imports of individual records one at a time.Beta Was this translation helpful? Give feedback.
All reactions