Thoughts on hledger 2

[simonmichael]

Some thoughts on a hledger 2.x:

It's hard to deliver big features in a short enough time, as to justify a jump from 1.x to 2.x version numbering.

But I think lot tracking (#1015), which seems like it may require impactful changes to our data model - plus other impactful cleanups that could arise - might be a good moment for this.

1.x vs 2.x could also be a good demarcation between a codebase with almost zero AI-assisted development, and one where AI assistance is used more routinely.

What other big items or themes could make a 2.x series distinct from 1.x ?

How could migration go ? As I see it,

• hledger 1 would continue to exist as a stable known quantity, reference and fallback. Major and minor 1.x releases would still be possible if needed.

• hledger 2 would allow more aggressive cleanups than we've seen in 1, perhaps non-backward compatible. But there should always be a reasonably smooth path to migrate data from 1 to 2. (And ideally from 2 back to 1 as well, initially.) 2 would aim to continue or improve production readiness, and to eventually be a no-brainer upgrade.

• Radical, very disruptive experiments should be explored too, but would not be in scope for hledger 2. Call those hledger 3, 3000, or some other name.

[rickles42]

How could migration go ?

I'm still very new to hledger (only a couple weeks into adoption and still setting up my process). But if you're interested in a beginner's perspective, I'm happy to share some thoughts on migration.

tl;dr: I don't know whether or not it's appropriate to pursue "hledger 2" at this point in the project, but if you do, I'd ask you to consider pursuing a very crisp/explicit versioning story---including a total fork of your documentation and maybe even fully breaking supporting tool compat as well. For me, the biggest issue getting started with PTA was finding and following documentation for PTA projects. Being as explicit as possible about versioning, and about compat between the core tool and supporting tools in the ecosystem, makes a huge difference to those newly picking up the tools.

I was Beancount's user to lose, so to speak. I discovered them first; I'm familiar with Python but not Haskell; Beancount's "strict by default" ethos appealed to me; and I had the impression that Beancount was designed after ledger/hledger and thus might be able to take the best of ledger/hledger and improve upon it.

But I bounced off Beancount for several reasons, all related to its v2 -> v3 migration:

1. In the move to v3, it felt to me that Beancount disintegrated (decomposed?) from a tool for users into a set of libraries for coders. With v3, it's on me to code myself up a tool. I'm okay with wiring components together with configuration files, and maybe even writing an optional shell script once I get tired of repetitive commands. But Beancount v3 demands more: just to get a basic workflow running, you need to reason about class inheritance, query syntax, imports, dependency management, etc. Maybe I'm just missing something. But I'd never know it, because...
2. In the move to v3, Beancount also made a total mess of their docs. Rather than having a v2 docset and a v3 docset, the two versions are largely intermingled. Signposts occasionally point out, "Ignore this section if you're using v3" because nothing mentioned in that section even exists in v3 anymore. The main tutorial is still mostly about bean-report, which no longer exists in v3. A signpost at the start of the "Generating Reports" guide makes clear it's all deprecated, but doesn't point to any replacement.
3. Companion tools/packages likewise didn't always make it clear whether they supported Beancount v2, v3, or both. When errors arose, it wasn't clear to me (a beginner) whether I'd made a mistake (e.g. with dependency coordination) or if the tool just didn't support v3 yet.

So yeah, I suppose my main feedback on the topic of migration is: learn from Beancount's mistakes in that department.

[simonmichael]

Thanks for your insights. Everything you're saying about Beancount is right, and we must definitely avoid creating that kind of situation. Python 2 to 3 is another famous example.

I think we can do it more smoothly. Unlike the Beancount transition, we won't be rewriting the core in a different language, and it's quite unlikely we'd be dramatically restructuring the codebase - current language and architecture are working well. (Restructuring docs might be more likely.)

Why risk a big transition at all then ? I could keep just evolving 1.x "for ever". But I think there's value and opportunity in once in a while drawing a line, preserving a certain known product, signalling a bigger transition, and perhaps unlocking more experimentation, novelty, and velocity. There's some marketing value too. "hledger v2, now with excellent lot tracking".

I did push a draft branch recently, with some more notes about v1 vs v2: link

[simonmichael]

But... I'll ponder it more.

[Daniii44]

I’m equally new to hledger and fully agree with what @rickles42 has said. It personally took me quite a while to even choose between Ledger, hledger, and Beancount. Actually, I also looked into Beancount first but was deterred by how rough the v2/v3 transition was. At least, as far as I know, the Ledger files have remained backwards compatible.

Ideally, though, such a choice shouldn’t matter as much in the first place. The differences in syntax across these tools seem very minimal anyway. I would certainly hope that a potential hledger v2 would close this gap rather than make it larger. Rustledger has started a repo for PTA standards, which seems like a necessary undertaking to me (https://github.com/rustledger/pta-standards). Converging on a single syntax is probably not feasible but standardisation would be nice in my mind.

[adept]

How could migration go ? As I see it,

• hledger 1 would continue to exist as a stable known quantity, reference and fallback. Major and minor 1.x releases would still be possible if needed.
• hledger 2 would allow more aggressive cleanups than we've seen in 1, perhaps non-backward compatible. But there should always be a reasonably smooth path to migrate data from 1 to 2. (And ideally from 2 back to 1 as well, initially.) 2 would aim to continue or improve production readiness, and to eventually be a no-brainer upgrade.

In my opinion, a clear goal should come first. If there is some sort of clear goal that was not reachable before, because compatibilty and legacy gets in the way, but breaking it could unlock a clear path to said goal -- then absolutely, "breaking mould" could be a way to go, and a major version could be a nice way to do it.

On the other hand, history is littered by software that did a rewrite seemingly "just because" (Netscape Navigator, anyone?) and that led to its complete and utter downfall :)

So, unless there is a pressing need to break compatibility, maybe just ... dont? :)

[simonmichael]

Good feedback, thanks all.

My main motivations/desires are around making the best use of a "2.0" transition, one of the most distinctive version bumps a project can have:

1. using the opportunity to drop some costly cruft, in a well-signalled orderly manner, from the feature set and codebase
2. creating enough of a jump in features or surrounding resources to justify the 2.0 version number and avoid a let-down
3. using the marketing power of 2.0 effectively, eg giving people a chance to revisit and shift their perception of hledger
4. decoupling 2.0 development from the too-short 3-month hledger dev cycle, allowing more time to build and refine it while keeping regular release cadence
5. marking a boundary between no-AI and some-AI hledger development, eg for comparison/regrouping/forking

I could do a less disruptive 2.0 transition by skipping the separate branch plan: don't preemptively create two 1.x and 2.x trunks, just keep using master, and the next release after 1.52 could be 2.0.

Or if more time is wanted: 1.52, 1.99.1, 1.99.2, ..., 2.0 (meaning 1.52, 2.0 preview 1, preview 2, ..., 2.0. cabal requires numeric versions.)

And I could minimise breaking changes. There's a choice to be made:

• Take the opportunity to drop some cruft/make some more aggressive changes, for the sake of simplifying the product, increasing flexibility, and reducing future maintenance/support/learning costs.
• Or, be like Windows: never break legacy setups, keep supporting old things forever. Accept and preserve the cruft.

In practice we occasionally have breaking changes even in regular major releases; it's sometimes unavoidable. Of course we should always try to provide an easy workaround or migration path in such cases.

[simonmichael]

Current most likely plan:

• Keep using one master branch.
• On next release day (march 1st), release both 1.52 (minor updates) and 1.99.1 (2.0 preview 1, with lot tracking).
• Don't intentionally break anything, except in the usual way (as rarely as possible, with easy workarounds and deprecation periods).