diff --git a/.version b/.version index 0c353043ed6..3e9926483a2 100644 --- a/.version +++ b/.version @@ -1 +1 @@ -1.52.99 +1.99 diff --git a/README.md b/README.md index 1bdff339b66..a6d5bc33bc6 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,34 @@ # hledger + +## About this branch + +This is the `hledger2` branch, a development branch for the hledger 2.x releases. +This branch is mutable (it can be force-pushed). + +For the hledger 1.x series, see `hledger1`. +`master` is currently the same as `hledger1`. +It will absorb `hledger2`, and possibly be renamed to `main`, when hledger 2.0 is released. + +Some goals for the 2.x series: + +- continue and improve 1.x's reliability +- provide excellent lot tracking and capital gains calculation +- explore ethical use of AI as a dev tool +- more cleanup and simplification of code, docs, process, finance +- more speed +- more interoperability +- more use of jj for version management +- easier contribution + +and for 1.x: + +- continued stability, installability +- bugfix releases to fix newly discovered regressions, if any +- preserve the non-AI-assisted codebase, and try to keep it that way + +Related discussion: [Thoughts on hledger 2 #2547](https://github.com/simonmichael/hledger/issues/2547) + + ## Robust, intuitive plain text accounting [![license](https://img.shields.io/badge/license-GPLv3+-brightgreen.svg)](https://www.gnu.org/licenses/gpl.html) [![on hackage](https://img.shields.io/hackage/v/hledger.svg?label=hackage&colorB=green)](https://hackage.haskell.org/package/hledger) diff --git a/Shake.hs b/Shake.hs index 584564b4760..07988b804d4 100755 --- a/Shake.hs +++ b/Shake.hs @@ -1,6 +1,7 @@ #!/usr/bin/env stack {- stack script --resolver nightly-2025-09-30 --compile --extra-include-dirs /Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/include/ffi + --extra-dep shake-0.19.9 --package base-prelude --package directory --package extra diff --git a/doc/AI.md b/doc/AI.md new file mode 100644 index 00000000000..8c5f15d75c2 --- /dev/null +++ b/doc/AI.md @@ -0,0 +1,55 @@ +# AI policy + +The productivity benefits from AI-assisted software development are becoming obvious. +The many potential costs and risks will keep becoming clearer. + +Here are some current policies for AI use in the hledger project: + +- hledger does not use AI at runtime. + +- hledger 1.x (2007..2025) has been developed without AI assistance. + +- hledger 2.x (2026..) is developed with careful AI assistance. + +- This bears repeating: careful AI assistance is not "vibe coding". + We aim to increase quality and maintainability, not decrease them. + +- The codebase remains human maintainable. We can always stop using AI and keep moving forward. + +- We aim to use only the more principled/trustworthy/sustainable tools and providers. + For now that means we prefer Anthropic, Ecosia, local LLMs, and such. + +- We monitor and try to limit and optimise our AI resource usage (represented by tokens, cost, etc.) + +- We monitor the impact of AI tools on the project, ourselves, and the planet and make adjustments as needed. + +Justification for AI use in this project: + +- I needed it to fully design and implement robust tax lot tracking in hledger. + This is a feature that I have been wanting for years, but it was just too big/intricate to tackle. + Use of AI tools made it possible. I think it's unlikely hledger would have ever got this feature without them. + +- Although similar features exist in other free software (Beancount/Ledger/rustledger/BittyTax/rotki/RP2/..), + I believe this new implementation provides flexibility currently not available elsewhere - + private, plain text, and capable of modelling real world lot operations and US pre- and post-2025 booking methods. + This (I hope) will provide value to many. + +- I imagine it is overall more efficient in resources and human energy, + for developers to use AI to develop efficient deterministic software, + than to have everyone using AI individually to try and do the same tasks less efficiently and less reliably. + Ie, let's move the AI use upstream as far as possible - use it briefly at design/implementation time, + not repeatedly at usage time. + +- The "bitter lesson" says that general computation always eventually wins over special-purpose systems - + suggesting that the lifetime and value of specialised tools like hledger will decrease. + However, there is at least a time lag, and for some time yet there will be a gap in efficiency, cost, reliability, + and so on, making this work worthwhile. + +- We are making mindful limited use of unsustainable technologies for a short time, + in preparation for more sustainable versions (local LLMs, ASIC LLMs) coming soon. + +- It is a learning experiment that can be discontinued or even rolled back at any time. + +If you are a hledger user who objects to any use of AI, for one reason or another: I can understand. +AI is a tool, probably too dangerous for us, but it's here and we're going to have to try to survive it. +The AI-free hledger 1.x still exists, will continue to receive at least regression fixes, and can be revived or forked at any time if needed. diff --git a/doc/IMPACT.md b/doc/IMPACT.md new file mode 100644 index 00000000000..2f38372f601 --- /dev/null +++ b/doc/IMPACT.md @@ -0,0 +1,18 @@ +# Impact + +In the 21st century we are living in a time of collapse and disruption of many natural and human systems. + +As hledger users, contributors and especially developers, +we all are responsible and accountable for this project's impact on the world. + +On this page we try to understand and describe impacts +this project has, or could have, on things outside it. +Such as, eg: + +- [CLIMATE](CLIMATE.md) +- [AI](AI.md) use +- people using or needing accounting software +- FOSS developers and users +- people using or interested in Haskell +- the Plain Text Accounting community +- other PTA projects such as Ledger and Beancount diff --git a/doc/NOTE-2.x.md b/doc/NOTE-2.x.md new file mode 100644 index 00000000000..32ae496fe95 --- /dev/null +++ b/doc/NOTE-2.x.md @@ -0,0 +1,62 @@ +# hledger 2.x + +## Strategy + +Summary of discussion [Thoughts on hledger 2 #2547](https://github.com/simonmichael/hledger/issues/2547): + +### Positions + +**Simon's motivations**: drop costly cruft, marketing power of "2.0", lot tracking as flagship feature, AI-era demarcation, decouple from the 3-month release cycle. + +**New users (rickles42, Daniii44)**: Beancount's v2->v3 transition was a cautionary tale -- intermingled docs, broken companion tools, unclear what works with which version. Don't repeat that. + +**adept (collaborator)**: don't break compatibility without a clear goal that requires it. History is full of needless rewrites that killed projects. + +### Recommended: "Boring 2.0" + +Stay on master, release 2.0 when ready. Don't maintain parallel long-lived branches. + +1. **Linear release path**: 1.52 -> 1.99.1 (preview) -> 1.99.2 -> ... -> 2.0. Avoids the cost of maintaining two diverging trunks. + +2. **Minimise breaking changes; make them opt-in first**: new behaviours behind flags (like `--lots`), old behaviours deprecated with warnings for a release or two, then removed. Proven pattern (Rust editions, Python `__future__`, GHC extensions). + +3. **Lot tracking alone justifies 2.0**: it's a large, data-model-impacting feature. Combined with accumulated improvements, it's enough. Bundling too many breaking cleanups risks the Beancount trap. + +4. **Keep docs unified**: a single docset with "New in 2.0" / "Changed in 2.0" callouts, not two divergent doc trees. + +5. **AI demarcation is worth noting but shouldn't drive versioning**: it's a process change, not user-facing. Mention in release notes, not a reason to break compatibility. + +### Key Principle + +Take the marketing win (call it 2.0) but keep the technical disruption minimal. + +### Current most likely plan: + +- Keep using one master branch. +- On next release day (march 1st), release both 1.52 (minor updates) and 1.99.1 (2.0 preview 1, with lot tracking). +- Don't intentionally break anything, except in the usual way (as rarely as possible, with easy workarounds and deprecation periods). + +## Goals + +SM 2026-02:\ +Some goals for 2.x: + +- continue and improve 1.x's reliability +- excellent lots/capital gains tracking +- more interoperability/convertibility +- more speed +- more customisation paths + +and: + +- more use of AI as a dev tool; clarify policies +- more use of jj to simplify version management +- more aggressive cleanup and simplification of code/doc/process/finance.. +- easier contribution + +and for 1.x: + +- continued installability/usability +- preserve the stable/known hledger 1.x feature set +- preserve the non-AI-assisted codebase; draw a line between pre and post-AI eras + diff --git a/doc/NOTE-amount-classes.md b/doc/NOTE-amount-classes.md new file mode 100644 index 00000000000..a7f97d5d18e --- /dev/null +++ b/doc/NOTE-amount-classes.md @@ -0,0 +1,138 @@ +# Typeclass Semantics for Amounts + +**Date:** 2026-02-01 +**Context:** Design question on whether to define amount semantics via Monoid first, then Num, or vice versa. + +## Question + +Is it better to define amounts' semantics in terms of standard typeclasses like Monoid first, and then define Num operations in terms of those? + +## Current State + +### MixedAmount (hledger-lib/Hledger/Data/Amount.hs:848-863) + +```haskell +instance Semigroup MixedAmount where + (<>) = maPlus + +instance Monoid MixedAmount where + mempty = nullmixedamt + +instance Num MixedAmount where + (+) = maPlus -- delegates to Monoid operation + (*) = error "..." -- intentionally partial + signum = error "..." -- intentionally partial +``` + +### Amount (hledger-lib/Hledger/Data/Amount.hs:300-307) + +```haskell +instance Num Amount where + (+) = similarAmountsOp (+) -- errors if commodities differ + (*) = similarAmountsOp (*) -- questionable semantics + -- No Semigroup/Monoid instance at all +``` + +## Analysis + +### MixedAmount Follows Monoid-First Design + +**Strengths:** +- ✅ Core semantics defined via `maPlus :: MixedAmount -> MixedAmount -> MixedAmount` (line 905-906) +- ✅ Num instance delegates to Monoid (`(+) = maPlus`) +- ✅ Explicitly rejects nonsensical operations (multiplication, signum) +- ✅ Matches DESIGN-amounts-keys.md observation: "MixedAmount forms a commutative monoid" (line 114-121) + +### Amount Does NOT + +**Problems:** +- ❌ Only has Num, no Monoid/Semigroup +- ❌ Can't have a proper Monoid (no identity: `nullamt + $1` ≠ `$1` due to commodity difference) +- ❌ Num instance is **partial** (errors on mismatched commodities) +- ❌ Implements `(*)` with unclear semantics + +## Recommendation: Monoid-First is Better + +### Reasons + +1. **Semantic Clarity** + - The key operation is aggregation (combining amounts) + - Monoid captures exactly this - no more, no less + - When deciding "what goes in the key?", you're really asking "when does `<>` combine vs keep separate?" + +2. **Partial Num is a Code Smell** + - If you need `error` in Num methods, the type doesn't truly satisfy the Num contract + - Better to not have the instance than to have a partial one + +3. **Aligns with Design Documents** + - DESIGN-amounts-keys.md explicitly identifies the commutative monoid structure as fundamental (line 114-121) + - The core operation `add1` (line 123-129) is monoid addition + +4. **Better Composition** + - Monoid instances compose beautifully (e.g., `Map k v` is a Monoid if `v` is) + - This helps with the `MixedAmount = Map Key Amount` design + +5. **Forces Clear Thinking** + - Monoid has simpler laws (associativity + identity) + - Easier to verify and test + - Num brings unnecessary baggage (multiplication, division, etc.) + +### Theoretical Foundation + +From abstract algebra perspective: + +- **Monoid**: Just needs associativity, identity, and closure - minimal structure +- **Num**: Implies ring-like structure with multiplication - too much structure for amounts +- **For amounts**: Addition forms a commutative monoid, but multiplication is not well-defined + - What is `$1 * $1`? `$1`? Type error? `$²1`? + - What is `$1 * €1`? Nonsensical. + +## Concrete Suggestions + +### Short Term (Low Risk) +- Keep current MixedAmount design (already follows Monoid-first) +- Document that Num instance is a convenience wrapper around Monoid +- Consider deprecation warnings on partial operations + +### Medium Term (Moderate Risk) +- Drop Amount's Num instance entirely +- Add clear utility functions for amount arithmetic: + ```haskell + amountPlus :: Amount -> Amount -> Either String Amount + -- Returns Left if commodities don't match + ``` + +### Long Term (Requires Migration) +- Consider Amount Semigroup instance IF there's a sensible (<>) that doesn't require same commodity +- Or: Keep Amount without Monoid (since no proper identity) but use Semigroup for combining +- Make MixedAmount the primary abstraction for all arithmetic + +## Tradeoffs + +**Lost Convenience:** +- Can't write `amount1 + amount2` +- Must write `amount1 <> amount2` or explicit function calls + +**But:** +- This is already partial anyway (errors on commodity mismatch) +- Making it explicit forces handling the error case +- More honest API + +## Related + +- **DESIGN-amounts-keys.md** - Discusses MixedAmount monoid structure +- **hledger-lib/Hledger/Data/Amount.hs:905** - `maPlus` implementation +- **hledger-lib/Hledger/Data/Amount.hs:848** - Current Semigroup/Monoid instances + +## Open Questions + +1. Should Amount have a Semigroup instance? What would `<>` mean for amounts with different commodities? +2. If we remove Num from Amount, what impact on existing code? +3. Should we use a different type for "amounts that can be added" vs "amounts that must be kept separate"? +4. Could we use a phantom type to track whether an Amount is "simple" (can add) vs "complex" (need MixedAmount)? + +## Conclusion + +**Monoid-first design is theoretically cleaner and aligns better with amount semantics.** The current MixedAmount implementation already follows this pattern. Amount's Num instance is a historical convenience that causes more confusion than it solves. + +Recommendation: Embrace Monoid as the primary abstraction for amount aggregation, with Num as a thin, well-documented convenience layer that explicitly delegates to Monoid operations. diff --git a/doc/NOTE-amount-keys.md b/doc/NOTE-amount-keys.md new file mode 100644 index 00000000000..7be124df892 --- /dev/null +++ b/doc/NOTE-amount-keys.md @@ -0,0 +1,247 @@ +# Amount Keys and Aggregation Design Discussion + +**Date:** 2026-02-01 +**Context:** Discussion about MixedAmountKey design, when amounts should combine, and the semantics of @ vs @@ notation. + +## Core Questions + +### 1. When Should Amount Attributes Be Combined or Discarded? + +Different operations have different needs: + +| Operation | Preservation Need | Current Behavior | +|-----------|------------------|------------------| +| **Parsing/Printing** | Exact fidelity - preserve everything | Preserves all: @, @@, precision, style | +| **Transaction Balancing** | Check if amounts cancel | Depends on key function | +| **Lot Tracking (FIFO/LIFO)** | Keep acquisition costs separate | Use cost basis {} | +| **Balance Reports** | Aggregate "economically equivalent" amounts | Combines by MixedAmountKey | +| **Market Valuation** | Convert to current value | Discard transacted cost, apply market price | +| **Sorting** | Deterministic ordering | Compare by commodity, then cost | + +**Key Tension:** Balance reporting wants aggregation, lot tracking wants separation. + +### 2. Should Transacted Cost Be in Amount Keys? + +**Current Design (as of 2026-02-01):** +```haskell +-- Unit costs: quantity IS in key (amounts with different unit costs stay separate) +key (Amount "USD" 1 (UnitCost "EUR" 0.9)) = KeyUnit "USD" "EUR" 0.9 + +-- Total costs: quantity NOT in key (amounts with different totals combine) +key (Amount "USD" 1 (TotalCost "EUR" 90)) = KeyTotal "USD" "EUR" +``` + +**Implications:** +- `$1 @ €0.9` and `$1 @ €0.8` → separate (different unit costs) +- `$1 @@ €90` and `$1 @@ €80` → combine (total cost quantity ignored) +- When combining total costs, the cost quantities sum: `$1 @@ €1` + `$-2 @@ €1` = `$-1 @@ €2` + +**Design Question:** What is @ and @@ notation FOR? + +#### Interpretation A: Recording Only +- @ and @@ just record historical transaction cost +- NOT used for lot tracking (that's what {} is for) +- Balance reports ignore them or convert to cost +- **Key excludes all transacted costs** + +#### Interpretation B: @ is Lot Marker (Current) +- @ creates implicit lots (same unit cost = same lot) +- @@ is shorthand, not for lot tracking +- Balance reports keep @ amounts separate, combine @@ amounts +- **Current design: unit cost in key, total cost not in key** + +#### Interpretation C: Both Create Lots +- Both @ and @@ create lots (fine-grained separation) +- Explicit "aggregate" operation for reporting +- **Key includes all cost info** + +### 3. Redundancy Between MixedAmountKey and Amount + +**Current Redundancy:** +```haskell +-- Amount stores: +Amount {aCommodity = "USD", aQuantity = 100, aCost = Just (UnitCost "EUR" 0.9)} + +-- Key duplicates some fields: +KeyUnit "USD" "EUR" 0.9 + +-- Invariant: key(amount) must match map key +-- Problem: Easy to violate, memory duplication +``` + +**Alternative Designs:** + +#### Option A: Key is Projection (Current) +- Key derived from Amount +- Flexible but has invariant to maintain + +#### Option B: Key Owns Identity +```haskell +data AmountKey = AmountKey Commodity CostKey BasisKey +data AmountValue = AmountValue Quantity Style +type MixedAmount = Map AmountKey AmountValue +``` +- No redundancy, key is source of truth +- Harder to work with single amounts + +#### Option C: Different Types for Different Stages +```haskell +data ParsedAmount = ... -- everything preserved +data Amount = ... -- normalized, only cost basis matters +data MixedAmount = ... -- aggregated by key +``` +- Each type optimized for use case +- More types, conversion overhead + +#### Option D: Parameterized Amount +```haskell +data Amount cost = Amount Commodity Quantity cost +type ParsedAmount = Amount (Maybe CostNotation) +type LotAmount = Amount (Maybe CostBasis) +type SimpleAmount = Amount () +``` +- Very flexible, explicit +- Complex types + +## Key Insights from Greenfield Design Exercise + +From designing a clean-slate implementation: + +1. **The Key Function IS the Specification** + - Answers "when do amounts combine?" + - Everything else follows mechanically + +2. **MixedAmount Forms a Commutative Monoid** + ```haskell + a + b = b + a -- commutative + (a + b) + c = a + (b + c) -- associative + a + 0 = a -- identity + ``` + - Essential for order-independent processing + - Enables parallel aggregation + +3. **Core Operation: add1** + ```haskell + add1 :: Amount -> MixedAmount -> MixedAmount + add1 a (MixedAmount m) = MixedAmount $ M.insertWith combine (key a) a m + ``` + - All other operations build on this + - Simple and compositional + +4. **Total Cost Semantics (Current)** + - When amounts with total costs combine, costs sum + - `$1 @@ €1` + `$-2 @@ €1` = `$-1 @@ €2` + - Rationale: Accumulates cost of repeated purchases + +## Proposed Design Direction + +**Cleanest approach may be:** + +1. **@ and @@ are RECORDING ONLY, not lot tracking** + - Record what you paid in the transaction + - Don't use for lot tracking (use {} explicitly) + +2. **Normalize amounts after parsing** + ```haskell + normalize :: ParsedAmount -> Amount + -- Strips @ and @@ (for recording only) + -- Preserves {} (for lot tracking) + ``` + +3. **MixedAmount keys only on commodity and cost basis** + ```haskell + key :: Amount -> Key + key (Amount c _ (Just basis)) = KeyWithBasis c basis + key (Amount c _ Nothing) = KeySimple c + -- Transacted cost not in key at all + ``` + +4. **Choose aggregation level explicitly** + ```haskell + aggregateSimple :: MixedAmount -> Map Commodity Quantity + aggregateLots :: MixedAmount -> Map (Commodity, Basis) Quantity + convertToCost :: MixedAmount -> MixedAmount + ``` + +5. **For lot tracking, REQUIRE {} notation** + - Don't infer lots from @ or @@ + - Explicit is better than implicit + +## Open Questions + +1. **Should total costs sum when combining?** + - Current: Yes (`$1 @@ €1` + `$1 @@ €1` = `$2 @@ €2`) + - Alternative: Keep first cost (`$2 @@ €1`) + +2. **What about balance assertions with costs?** + - Should `= $100 @ €90` check the cost too? + - Or just the commodity quantity? + +3. **Conversion to cost:** + - When displaying `$100 @ €0.9` as cost, show `€90` + - What about `$100 @@ €88`? (unit cost would be `€0.88`, but we don't have it) + +4. **Mixed @ and @@ in same transaction:** + ```journal + assets:cash $100 @ €0.9 + assets:cash $50 @@ €44 + ``` + Should these combine? Currently they don't (different keys). + +## Related Files + +- **Implementation:** hledger-lib/Hledger/Data/Types.hs (MixedAmountKey) +- **Operations:** hledger-lib/Hledger/Data/Amount.hs (maAddAmount, sumSimilarAmounts) +- **Tests:** hledger-lib/Hledger/Data/Amount.hs:1425 ("adding mixed amounts with total costs") +- **Other docs:** + - doc/REFACTOR-prices-duplication.md + - doc/CODE.md + +## Language/Specification Explorations Considered + +Alternative ways to model/specify the semantics: + +1. **APL** - Array operations, compact but limited readability +2. **SQL** - GROUP BY semantics map perfectly to MixedAmount aggregation +3. **Equational Specification** - Mathematical laws as equations +4. **Alloy** - Constraint-based model finder for edge cases +5. **Property-Based Tests** - Laws as executable properties (QuickCheck) +6. **Prolog/Datalog** - Logic programming for rules +7. **TLA+** - Formal specification with model checking + +**Most promising for hledger:** +- Equational spec in doc (concise, mathematical) +- SQL prototype (test grouping interactively) +- Property tests in Haskell (verify laws) +- Alloy for edge case discovery (optional) + +## Summary + +The core design choice is: **What does the key function encode?** + +Current answer: +- Commodity (always) +- Unit cost with quantity (@ creates lots) +- Total cost without quantity (@@ doesn't create lots) +- Cost basis (always, {} creates lots) + +Alternative answer: +- Commodity (always) +- Cost basis only (only {} creates lots) +- @ and @@ are recording notation, not semantic + +The choice affects: +- How balance reports aggregate +- Whether lot tracking works implicitly or requires {} +- Memory usage and complexity + +**Next steps:** Consider whether @ should be recording-only, with explicit {} required for lot tracking. + +## 2026-03: Simplified to 3 constructors (commodity + transacted cost) + +Cost basis was removed from MixedAmountKey because lot identity is now carried +by subaccounts in `--lots` mode, and no reports display cost basis in aggregated output. + +Transacted cost remains in the key because balance assignments and balance +inference rely on keeping same-commodity different-cost amounts separate during +MixedAmount arithmetic. diff --git a/doc/NOTE-recompiling.md b/doc/NOTE-recompiling.md new file mode 100644 index 00000000000..0f21a7921fc --- /dev/null +++ b/doc/NOTE-recompiling.md @@ -0,0 +1,42 @@ +# Fast recompilation feedback + +## Problem + +The full rebuild cycle (`stack build hledger`) is slow when changes touch core modules like `Types.hs` — about 11 seconds due to cascading recompilation of all dependents. Leaf module changes are fast (~1.6s) but core changes are frequent during development. + +## Solution: `ghc -fno-code` type-checking + +Type-check without code generation, ~2s regardless of what changed: + +``` +stack exec -- ghc -fno-code -fforce-recomp \ + -Wall -Wno-incomplete-uni-patterns -Wno-missing-signatures -Wno-orphans -Wno-type-defaults -Wno-unused-do-bind \ + -DDEVELOPMENT -DVERSION="\"1.51.99\"" \ + -ihledger-lib -ihledger \ + TARGET_MODULE.hs +``` + +Replace `TARGET_MODULE.hs` with the module you changed (e.g. `hledger-lib/Hledger/Data/Lots.hs`). + +## Benchmarks (M-series Mac, 2026-02) + +| Scenario | `stack build hledger` | `ghc -fno-code` | +|---|---|---| +| After touching `Types.hs` | ~11s | ~2s | +| After touching `Lots.hs` (leaf) | ~1.6s | ~2s | + +`-fno-code` is ~5x faster for core module changes. For leaf-only changes `stack build` is comparable since it's already incremental. + +## Alternatives considered + +**ghcid** (`just ghcid`): Keeps a GHCi session loaded and does incremental `:reload` on file changes. Sub-second for leaf changes after initial load, but slow startup and requires managing a persistent background process. Not worth the complexity when `-fno-code` is already fast. + +**GHCi manual reload** (`just ghci` then `:reload`): Same incremental benefit as ghcid but requires an interactive session. Good for human developers; awkward for automated tooling. + +**HLS**: Provides continuous type-checking via LSP but requires an LSP client. + +## Recommended workflow + +1. After code changes: run `ghc -fno-code` targeting the changed module (~2s) +2. Before functional tests: run `stack build hledger` (need the actual binary) +3. Before committing: run `just functest --hide` for full regression check diff --git a/doc/PLAN-beancount-output.md b/doc/PLAN-beancount-output.md new file mode 100644 index 00000000000..abae289fe61 --- /dev/null +++ b/doc/PLAN-beancount-output.md @@ -0,0 +1,14 @@ +# Easier export to Beancount and rustledger + +Figure out if lot subaccounts are a problem for beancount export. +Avoid --lots ? What about explicit lot subaccounts ? + +Some current incompatibilities, seen with c.beancount: + +- beancount spurious no position matched errors for acquires +- rustledger won't balance a certain txn +- rustledger doesn't support single-posting transactions + +When generating `open` account directives, +we should add the account's lots tag value, if any, as the disposal method. +It should be a method supported by beancount (probably one of FIFO, LIFO, STRICT). diff --git a/doc/PLAN-doc.md b/doc/PLAN-doc.md new file mode 100644 index 00000000000..f4524c5a094 --- /dev/null +++ b/doc/PLAN-doc.md @@ -0,0 +1,22 @@ + +## hledger manual + +add a quick reference section near the top ? +is it feasible to have one page per major section ? + +## cost/price naming + +- cost -> price, cost basis -> basis and cost ? + Amount{acost,acostbasis} -> Amount{aprice,abasis} + CostBasis{cbcost} -> Basis{bcost} + --cost/--basis/-B, --price/-P, --value/-V/--market/--exchange/-X ? + --infer-costs + --infer-market-prices + P directive + cost and price are both unavoidably generic (price more so because less directional) + basis price/cost, acquisition price/cost, disposal price/sale amount, market price + Cost Reporting, Transacted Value Reporting, Market Value Reporting +- enable --infer-costs/equity/market-prices always ? +- clean up old --exchange, --market flags ? +- add --value types: tx (may exist as "cost"), basis ? + diff --git a/doc/PLAN-lots.md b/doc/PLAN-lots.md new file mode 100644 index 00000000000..c8ca87d2a0d --- /dev/null +++ b/doc/PLAN-lots.md @@ -0,0 +1,442 @@ +# Plan for calculating lots, 2026-02 + +## Initial plan + +For background, see SPEC-lots.md. + +In journalFinalise, after transaction balancing, we'll do all lot-related processing. +We will think of lot functionality as an optional addition that happens after all normal journal validation. + +There is a new step, journalCalculateLots, which can fail, something like Journal -> Either String Journal. +This will step through the journal's transactions, in date order, transforming them by adding lot information. +Specifically, + +- Where a dispose or transfer-from posting has a lot selector, that will be expanded to identify a specific lot. + If it selects multiple lots, the posting will be replicated so that there's one for each specific lot. + +- Each lotful posting will have a full lot name appended to its account name, as a subaccount. + +This will require a stateful loop, accumulating a map from accounts to their lot balances. + +Posting dates and secondary dates will not be used for calculating lots; only primary transaction dates. + +Lots/lot postings will be modified/selected as described in "Lot posting effects". Eg, + +- For acquire postings, a new lot name will be generated from the cost basis and/or posting date/transacted cost. + +- For transfer-from and dispose postings, one or more existing lots will be inferred from the lot selector, + and/or by following a lot selection method, FIFO. When multiple lots are selected, extra postings will be created. + +- For transfer-to postings, the lot name will be copied from the corresponding transfer-from posting in the transaction. + We assume that lot transfers are recorded in a single transaction. + +We'll add a general --lots flag. +It will affect reports, but perhaps also journal validation, so perhaps it should be in InputOpts, like --auto or --infer-costs. + +There will be two kinds of transaction balancing: + +1. Normal transaction balancing, using transacted costs; all journal entries must pass this as usual. + +2. Lot disposal balancing is a separate check that applies to disposal transactions only: + it uses cost basis instead of transacted cost for balancing, + and so it requires or infers an appropriate gain posting. + +Gain postings (identifiable by Gain account type) will be ignored by normal transaction balancing, +so that both kinds of balancing can succeed with the same journal entries. +This needs some exploration and testing to assess the impact on existing journal entries +not using the new lots features. + +Lot disposal balancing could be done always, for best error checking; or only when lots mode is enabled, if it helps performance. + +Splitting lot postings shouldn't be a problem for transaction balancing - their aggregate effect will be unchanged. + +We can enforce simplifying limitations where needed, eg for matching transfer postings. +In general it's important to get something simple working early; refinements can be added later. + +Lot labels are generated when needed, to satisfy lot id uniqueness rules. +Eg if we find multiple same-date-same-commodity acquisitions (across all accounts), with no labels, +we should add NNNN labels based on parse/processing order. +If we find such acquisitions that have user-added labels, we should check that the resulting lot ids are indeed unique +(again across all accounts, to be safe) and report an error otherwise. + +There are multiple lot selection (AKA reduction) methods: +FIFO, LIFO, HIFO, AVERAGE, SPECID, and their global validation variants +(FIFOALL, LIFOALL, HIFOALL, AVERAGEALL). Base methods are per-account. +*ALL variants select per-account but validate against the global ordering. +We should assume the method can be vary/be configured in a relatively fine-grained way - +per account, per commodity, per time period, perhaps even per posting. +In future we might want to add a {!METHOD, ...} part inside the lot syntax, +as another way to select lots or as an extra validation that the lot selection satisfies that method. + +We will want a data structure for the calculated lot state that is relatively efficient +for implementing all the planned lot selection methods. +Suggestions include Map Commodity (Map LotId (AccountName, Amount)) +or Map (AccountName, Commodity) (Map LotId Amount) for per-account selection. +We'll want to be able to search in order of date + label. +We might find more suitable alternatives to standard Map, in the base library or on Hackage. +But it's fine to start with a simple implementation even if it's a bit slow. + +In general, we should do the simplest thing that could possibly work for now, to narrow scope. +This applies generally to all unresolved questions, unless they really do need to be resolved now. + +## Initial claude assessment of spec + +What the spec describes + + SPEC-lots.md defines a comprehensive lot-tracking system for investment accounting. The core idea: a --lots flag enables automatic lot inference, tracking, and validation. Lots are identified by a cost basis (date + optional label + acquisition cost) which also serves as the lot name and becomes a + subaccount (e.g. assets:stocks:aaaa:{2026-02-10, "lot1", $50}). + + Postings are classified as acquire, dispose, transfer-from, or transfer-to. Lot selection for disposals/transfers can be explicit (SPECID) or automatic (FIFO). Gain/loss is calculated by comparing acquisition cost to disposal price. + +What's already implemented + +Substantial infrastructure is in place: + - CostBasis type with all-Maybe fields (no need for a separate PartialCostBasis) + - Gain account type (subtype of Revenue) + - journalInferPostingsCostBasis — infers cost basis from @ cost on lotful postings + - journalClassifyLotPostings — tags postings with _ptype (acquire/dispose/transfer-from/transfer-to) + - journalInferPostingsTransactedCost — infers @ cost from cost basis on acquire postings + - All three are wired into the journalFinalise pipeline + - Functional tests exist for the above + +What's planned (PLAN-lot-calculation.md) + + A new journalCalculateLots function, inserted after the existing pipeline steps, that: + 1. Walks transactions in date order with stateful lot-balance tracking + 2. Expands lot selectors to specific lots (FIFO or explicit) + 3. Splits postings when multiple lots are selected + 4. Appends full lot names as subaccount names + + Gain postings and transaction balancing are explicitly deferred. + +Problem areas and open questions + + 1. Transaction balancing interaction — The spec acknowledges this is unclear (lines 209-213). + Currently balancing happens after cost inference in the pipeline but before the planned journalCalculateLots. + If lot calculation splits postings or adds gain postings, it may need to re-balance, or balancing may need to be deferred/split into two passes. + This is the biggest architectural risk. + + 2. No --lots flag yet — The spec says lots mode is gated by --lots, but it doesn't exist. + Currently lot processing runs unconditionally based on lots: tags on commodities/accounts. + The plan doesn't address when/how to add the flag. + + 3. Lot id uniqueness scope — The spec notes lot ids must be unique per-commodity, + and scope (per-account vs. global) needs to be configurable for different tax regimes (pre-2025 vs. post-2025 US rules). + No design for this configuration exists yet. + + 4. Label generation — When no label is specified, hledger must generate one (using time-of-day or sequence number). + The plan doesn't detail this algorithm. + Getting this right matters for determinism and for lot ids to sort correctly. + + 5. All methods are per-account — lot selection is always scoped to the posting's account. + The stateful map design accommodates this with Map Commodity (Map LotId (Map AccountName Amount)). + + 6. Posting splitting — When a disposal/transfer selects multiple lots, postings are replicated. + This changes the structure of transactions after they've been parsed and partially processed, + which could interact poorly with other pipeline steps (auto postings, equity inference, etc.). + + 7. Transfer-to matching — The plan says "we assume lot transfers are in a single transaction" and the lot name is "copied from the corresponding transfer-from". + But matching transfer-from to transfer-to relies on the classification step's counterparty detection, + whose robustness in edge cases (multiple transfer pairs in one transaction, partial amounts) isn't discussed. + + 8. Gain deferral — Gain calculation and gain-posting inference are deferred, but they're tightly coupled to disposal processing. + The disposal path will need to be revisited when gains are added, so the initial implementation should leave clear extension points. + + 9. Loop style question — The plan asks whether to use a fold, scan, or State monad. + For this kind of stateful sequential transformation (accumulating lot balances, potentially splitting transactions), a State monad or mapAccumL would be natural in Haskell. + A fold would work but is less ergonomic when the output structure differs from the input. + +Overall assessment + + The foundation is solid — the classification and inference pipeline is already working. + The plan for journalCalculateLots is reasonable and well-scoped (deferring gains is wise). + The main risk is the transaction balancing interaction: the current pipeline balances transactions before lot calculation, + but lot calculation may need to modify transactions in ways that affect balance. + This needs a concrete design decision before implementation begins. + + +## Amendments + +1. LotState type: PLAN suggested Map Commodity (Map LotId (AccountName, Amount)). We use Map Commodity (Map LotId (Map AccountName Amount)) — triple-nested to support + partial lot transfers (same LotId at two accounts). + +2. selectLotsFIFO is parameterised: Takes Maybe AccountName — Nothing to search across all accounts (used for disposals), Just acct to search within one account + (used for transfers, since the lots must come from the source account). Both use the same FIFO method; the parameter controls scope, not method. + +3. reduceLotState is parameterised: Takes Maybe AccountName — Nothing for disposals (reduce across all accounts holding the lot), Just acct for transfers + (reduce only the source account's holding). This is about where to debit quantity, not about the lot selection method. + +4. Transfer pairing: PLAN said "pair by commodity". We pair by commodity, then sort by cost basis key within each commodity group to align explicit per-lot pairs. + PLAN didn't anticipate multiple transfer pairs per commodity per transaction. + +5. No transacted cost check on transfers: SPEC says "transacted costs are not expected". We removed the runtime check because {$X} sets acost via earlier pipeline + steps, making it indistinguishable from explicit @ $X. Transaction balancing serves as the practical guard instead. + +6. PLAN's "Third goal" phase 12.4 said handle transfers outside foldMPostings. We did this — transfers are partitioned and processed before foldMPostings runs on + remaining postings. + +7. Reorganised/expanded/updated lot tests. + +8. Transfer transacted cost check re-added: Amendment 5 removed this check because {$X} sets acost indistinguishably from explicit @ $X. We now re-add it + (presumably with smarter detection), erroring with "lot transfers should have no transacted cost" when an explicit @/@@ is present. + +9. {} on acquire infers cost basis from implicit balancing cost: A new inference path — `{}` is treated as a wildcard requesting inference, not just from explicit + @/@@ but also from the implicit cost determined by transaction balancing. SPEC updated: "no explicit cost basis annotation". + +10. Lot posting definition broadened for transfer-to: Bare postings matched by a transfer-from in the same transaction are now classified as transfer-to, even without + cost basis annotation and on non-lotful commodities. SPEC updated with a third criterion for lot posting identification. + +11. Non-asset accounts tolerated for lot operations: Cost basis annotation forces lot detection regardless of account type (e.g. equity:opening with {$50}). + This is a convenience for undeclared/unknown account types, not a semantically intended use — lot operations are still conceptually for asset accounts. + +12. Bare/@ disposals on lotful commodities get full lot inference: FIFO lot selection, cost basis, and transacted cost are all inferred from existing lots and + transaction balancing amounts. E.g. bare `-5 AAPL` on a lotful commodity becomes `-5 AAPL {$50} [2026-01-01] @ $55` with ptype:dispose. + +13. {} on acquire postings infers cost basis from transacted cost. Previously this worked only if there was no {} annotation at all. + +14. Bare/@ acquisitions on lotful commodities: Positive postings in a lotful commodity/account + are now classified as acquire even without explicit {} notation, symmetrically with how bare + negative lotful postings are classified as dispose (amendment 12). Cost basis is inferred from + transacted cost (explicit @ or balancer-inferred). The removed journalInferPostingsCostBasis + is no longer needed — classification detects bare acquires directly, and processAcquirePosting + infers cost basis from acost at lot calculation time (after balancing). + +(+ many followup improvements) + +## Assessment + +Lots branch: scope & complexity + +Size + +- 64 commits, 68 files changed +- ~8,400 insertions / ~3,800 deletions (net +4,600 lines) +- Core new module: Lots.hs at 987 lines — the single largest new artifact + +Breakdown by category + +| Category | Commits | What | +|--------------------|---------|------| +| imp (improvements) | 29 | Bulk of the work — lot tracking, disposal balancing, gain inference, beancount export, print formats | +| doc | 13 | Specs, plans, manual updates, embedded manuals | +| dev (refactoring) | 11 | Types reorganization, pipeline cleanup, test reorganization | +| feat (features) | 4 | Lotful commodities, cost basis classification, account lots tag, Gain type | +| fix | 4 | Disposal matching, gain balancing, cost basis inference | +| infra | 2 | Cabal file updates | +| test | 1 | (plus many tests embedded in other commits) | + +What it touches + +- Deep plumbing: Types.hs (+208 lines — LotId, CostBasis, new posting fields), Amount.hs, Balancing.hs (disposal-aware balancing), Journal.hs +(pipeline changes), Posting.hs, Transaction.hs +- Parser: Read/Common.hs (+222 lines — new lot/cost basis syntax), JournalReader.hs +- Output: Print.hs (+68 lines), new Ledger.hs writer, Beancount.hs improvements +- New module: Lots.hs (987 lines) — acquisition, disposal FIFO/LIFO, transfers, gain inference, error reporting +- Tests: ~1,800 lines across 9 test files (lots-acquire, lots-dispose, lots-transfer, lots-methods, etc.) +- Specs/plans: ~640 lines of design documentation + +Complexity assessment + +High. This is a fundamental extension to hledger's data model and processing pipeline: +- Adds new fields to core types (Posting, Amount) +- Modifies transaction balancing semantics (gain postings excluded from normal balancing) +- Adds a new pipeline stage (lot calculation runs after balancing in journalFinalise) +- Multiple reduction methods (FIFO, LIFO, HIFO, AVERAGE, SPECID) +- Gain posting inference (auto-creating revenue postings) +- New input syntax (consolidated lot syntax {...}) +- New output format (-O ledger) +- Cross-cutting: changes span parser → types → balancing → lot calculation → output + +Roughly equivalent to a medium-sized feature branch that a senior developer might produce over 2-4 weeks of focused work, or longer with the +design/spec iteration visible in the commit history. + +## Impact on existing journals + + Summary + + The lots branch is safe for existing journals. Transaction balancing behavior is unchanged for journals that balanced on master. The gain posting exclusion is strictly more permissive (allows + transactions that would have failed on master), never more restrictive. + + What's new on the lots branch (vs master) + + Already on master (no change) + + - Gain account type in Types.hs + - gainAccountRegex in AccountName.hs — infers Gain type from names like revenue:gains, income:capital-gains, etc. + - isAccountSubtypeOf Gain Revenue = True — Gain accounts are included in Revenue queries (income statement, type:R, etc.) + - {...} cost basis parsing populates acostbasis field + + New on lots branch, unconditional (always runs) + + 1. journalClassifyLotPostings (Common.hs:371) — adds hidden _ptype tags (acquire/dispose/transfer-from/transfer-to) to postings with cost basis {...}. Invisible in normal output; only visible with + --verbose-tags. + 2. journalInferPostingsTransactedCost (Common.hs:372) — infers @ $X from {$X} on positive (acquire) postings only. Strictly beneficial: makes transactions work that would have failed on master. + 3. Gain posting exclusion in Balancing.hs (lines 84-88, 117-120, 286-296, 373-376) — excludes Gain-type postings from transaction balancing/inference in disposal transactions (those tagged _ptype: + dispose). + + New on lots branch, gated by --lots + + - journalCalculateLots — lot subaccounts, splitting, FIFO/LIFO selection + - journalInferAndCheckDisposalBalancing — cost-basis balance check, gain posting inference + + Why existing journals are safe + + The gain posting exclusion can't break balanced transactions + + The gain exclusion only triggers when ALL of: + 1. Transaction has a _ptype: dispose posting (negative amount with cost basis {...}) + 2. A posting goes to a Gain-type account (revenue:gains, etc.) + 3. account_types_ is non-empty (always true in practice) + + Key insight: On master, a disposal + gain posting can NEVER balance: + assets:broker -5 AAPL {$50} @ $55 ; converts to -$275 + assets:cash $275 ; +$275 + revenue:gains -$25 ; -$25 + ; total: -$25 ≠ 0, FAILS on master + On the lots branch, the gain posting is excluded → remaining sum is $0 → passes. + + This means the exclusion only RELAXES the rules. Transactions that balanced on master still balance (the excluded gain posting was necessarily $0, or the transaction couldn't have balanced). + Transactions that failed on master may now succeed. + + Verified by test + + The journal above succeeds on the lots branch (tested manually) and would fail on master (no gain exclusion, sum = -$25). + + Edge case: amountless gain posting in disposal + + assets:broker -5 AAPL {$50} @ $55 + assets:cash $275 + revenue:gains ; amountless + - Master: inferred amount = $0 (sum of others = $0) + - Lots branch: gain posting excluded from inference, stays amountless; balance check passes (remaining postings sum to $0) + - Effect: benign difference in internal representation; identical print output + + Specific concern: revenue:gains account + + An existing account named revenue:gains would: + 1. Already be typed as Gain on master (via gainAccountRegex) + 2. Still appear in income statement (Gain is subtype of Revenue, Type [Revenue] includes it) + 3. Still appear in type:R queries + 4. Only be excluded from balancing inside disposal transactions (those with {...} cost basis) + 5. Never get an auto-inferred gain posting without --lots + + The gainAccountRegex pattern + + Matches (case-insensitive): + ^(income|revenue)s?:(capital[- ]?)?(gains?|loss(es)?)(:|$) + Examples: revenue:gains, income:capital-gains, revenues:losses, income:gain:realized + + Does NOT match: revenue:other-gains, expenses:gains, revenue:gaming + +## Future: transaction-level classification ? + +The current posting classification logic (`shouldClassify` and friends) is a chain of +per-posting rules with growing guard conditions. Each new edge case (bare disposals, +revenue postings, partial transfers with fees) adds another condition. This is getting +harder to reason about. + +A potentially simpler model: **classify at the transaction level first, then assign +roles to postings**. Instead of asking "what is this posting?" with counterpart lookups, +ask "what is this transaction doing with lotful commodities?": + +1. Look at all lotful commodity flows in the transaction. +2. If a commodity flows between two asset accounts (any quantity) → it's a **transfer** + (possibly with fees going to expense). +3. If a commodity flows from asset to non-asset (expense, equity) → it's a **disposal**. +4. If a commodity flows into an asset from non-asset or with a price → it's an **acquisition**. +5. Otherwise → **untracked** (not enough info to track lots). + +This is essentially what the counterpart maps approximate, but described as a +transaction-level pattern rather than per-posting guards. It could make the logic +more tractable and reduce the number of special cases. + +This could be attempted as a parallel code path alongside the current implementation, +validated against the same test suite, then swapped in once confirmed equivalent. + +## Make lot reduction account-scoped + +Global reduction methods (FIFO, LIFO, etc.) can conflict with the specific +accounts in journal entries, leading to confusing lot errors. For example, +a dispose from account B can consume account A's lots via global FIFO. + +The `lots:` tag configures only the reduction *order*; the *scope* is always +the posting's account. All methods are per-account. + +## Better detection of transfers with fees + +> It's common for a lot transfers to have a small disposal (or several) for fees, and these are not always recorded as matching posting pairs. Can you see a robust way we could detect these ? + +The problem has two parts: + +1. Classification — Transfer detection requires (commodity, |quantity|) exact match. In test.j line 688: +- assets:cc:x1 ETH -0.05075600 (source) +- expenses:fees ETH 0.00075600 (fee) +- assets:cc:x2 ETH 0.05000000 (destination) + +The source quantity (0.05075600) ≠ destination quantity (0.05000000), so counterpart detection fails. Neither posting gets classified. + +2. Processing — pairIndexedTransferPostings requires equal numbers of from/to postings per commodity, and processTransferPair uses the from-posting's full quantity. + +There are a few approaches: + +Approach A: Transaction-level net-flow detection + +After the existing exact-match classification pass, do a second pass looking at still-unclassified postings. For each lotful commodity in the transaction, compute: +- Net outflow from each asset account (negative lotful amounts) +- Net inflow to each asset account (positive amounts) + +If there's exactly one source and one destination for the same commodity, classify them as transfer-from/to. The fee postings (to expense accounts) are left unclassified or classified as dispose. + +Problem: the fee posting (expenses:fees ETH 0.00075600) is also lotful (ETH has lots: tag). It might need to be classified as a dispose to properly reduce lots. + +Approach B: Split the source posting + +Classify the full source as transfer-from, but during processing, transfer only the destination quantity's worth of lots, and dispose the remainder (fee). This keeps classification simple but makes processTransferPair handle the quantity mismatch. + +Problem: the fee posting is a separate posting on a different account — it's not part of the source posting. The lots consumed from x1 need to cover both the transfer and the fee. + +Approach C: Relaxed counterpart matching (simplest) + +Change the counterpart maps to be keyed by commodity only (not quantity), or add a secondary lookup that matches on commodity alone when exact-quantity matching fails. When a negative lotful asset posting has no exact counterpart but there IS a positive posting of the same +commodity on a different asset account, classify: +- The negative posting as transfer-from +- The positive posting as transfer-to +- Any other same-commodity postings on non-asset accounts (the fee) as dispose + +Then in processing, processTransferPair would transfer min(|from_qty|, to_qty) lots, and the excess on the source side would need separate dispose handling. + +This is the most robust but requires changes to both classification and processing. + +Approach D: Treat fees as part of the transfer + +The simplest pragmatic fix: when the source quantity exceeds the destination, the transfer processes the destination's quantity from the source's lots. The fee amount (source - dest) is simply "lost" from the lot perspective — those lots get reduced but no gain is recorded (the fee +is already recorded as an expense). This matches real-world semantics: you transferred X, the exchange took a cut. + +Should transfer+fee patterns also work when the fee posting has a price (e.g. `expenses:fees ETH 0.00075600 @ $1169`)? +That would make the fee posting look like a dispose (has a price), which could conflict with the transfer-from classification of the source. +→ Classify fee as dispose + +## Incompatibilities between hledger 1 and 2 + +Most of the new lots functionality is activated only when --lots is used. +But there are some exceptions. + +- Lot postings are always classified (with ptype tag). + +- Postings whose account is detected as Gain type are excluded from normal transaction balancing. + This means accounts declared with type:Gain, but also accounts like revenues:gain, income:gain etc. + whose name is matched by `^(income|revenue)s?:(capital[- ]?)?(gains?|loss(es)?)(:|$)`. + These are now excluded from transaction balancing, where before they were not. + Transactions involving them may be reported as unbalanced. + To work around: rename the account, or declare it as type:Asset not Gain. + +## Next ? + +- more testing with real world journals +- infer acquire price, dispose price from market price ? +- recognise some common commodity symbols as lotful ? +- consolidate lot tests ? +- insert ptype at start of comment, not at end ? + +Remember: don't over-engineer. Build the vision, build high quality, but most of all build what users actually need, and validate that with real users quickly. + diff --git a/doc/SPEC-finalising.md b/doc/SPEC-finalising.md new file mode 100644 index 00000000000..3bbcf0b4a04 --- /dev/null +++ b/doc/SPEC-finalising.md @@ -0,0 +1,174 @@ +# Journal finalising + +After parsing, a journal goes through a **finalisation** pipeline (`journalFinalise` in +`Hledger.Read.Common`) that infers missing information, checks validity, and enriches +postings with computed metadata. + +("Finalising" is not easy to say, better suggestions welcome.) + +## What gets inferred, and from what + +Here are the main kinds of information that finalisation infers or computes, roughly in the order they happen. + +Note: "cost" means @/@@ (AKA transacted cost); "cost basis" means {} (acquisition cost & info). + +| What is inferred | From what | Step | +|-------------------------------------------------------------|-----------------------------------------------------------|------------------------------------------| +| Account types | Account declarations, account names, parent accounts | journalAddAccountTypes | +| Consistent posting amount styles | Posting amounts written in journal | journalStyleAmounts | +| Forecast transactions | Periodic transaction rules + forecast period | journalAddForecast | +| Posting tags inherited from accounts | Account declarations | journalPostingsAddAccountTags | +| Location of conversion equity postings and costful postings | @/@@ annotations + adjacent conversion account postings | journalTagCostsAndEquityAndMaybeInferCosts(1st) | +| Auto postings | Auto posting rules + postings in journal | journalAddAutoPostings | +| Cost basis from lot subaccount names | Account names containing {…} subaccounts | journalInferBasisFromAccountNames | +| Lot posting types | Cost basis + amount sign + account type + lotful status + counterpostings | journalClassifyLotPostings | +| Cost from cost basis | Costless acquire postings with a cost basis | journalInferPostingsTransactedCost | +| Transaction-balancing amounts | Counterpostings (or their costs) | journalBalanceTransactions | +| Transaction-balancing costs | Costless two-commodity transactions | transactionInferBalancingCosts | +| Balance assignment amounts | Running account balances vs asserted balances | journalBalanceTransactions | +| Canonical commodity styles | All posting amounts now present after balancing | journalInferCommodityStyles | +| Posting tags inherited from commodities | Commodity declarations | journalPostingsAddCommodityTags | +| Costs from equity postings (--infer-costs) | Equity conversion posting pairs | journalTagCostsAndEquityAndMaybeInferCosts(2nd) | +| Equity postings from costs (--infer-equity) | Costful postings | journalInferEquityFromCosts | +| Market prices from costs | Costful postings | journalInferMarketPricesFromTransactions | +| Lot subaccounts, cost basis and cost for bare disposals | Lot state tracking (applying FIFO etc.) | journalCalculateLots | + +## Current pipeline sequence + +``` +journalFinalise + -- Setup + 1. journalSetLastReadTime + 2. journalAddFile + 3. journalReverse + + -- Account types and amount styles (pure, no errors) + 4. journalAddAccountTypes -- builds jaccounttypes map + 5. journalStyleAmounts -- infer preliminary commodity display styles, and apply to postings + + -- Generate forecast transactions + 6. journalAddForecast -- if --forecast, generate forecast transactions from periodic rules + + -- Account tags + 7. journalPostingsAddAccountTags -- propagate account tags to postings + + -- Pre-balancing cost/equity tagging + 8. journalTagCostsAndEquityAndMaybeInferCosts(1st) -- tag conversion equity postings + redundant costs (helps balancer ignore them) + + -- Generate auto postings + 9. journalAddAutoPostings -- if --auto, do transaction balancing (preliminary) to infer some missing amounts/costs, + -- then apply auto posting rules. Calls journalBalanceTransactions. + + -- Lot cost basis inference from account names, classification, and transacted cost inference (before balancing) + 10. journalInferBasisFromAccountNames -- if account name has a {…} lot subaccount, parse cost basis from it + 11. journalClassifyLotPostings -- tag lot postings as acquire/dispose/transfer-from/transfer-to + 12. journalInferPostingsTransactedCost -- infer cost from cost basis of acquire postings + + -- Transaction balancing (main) + 13. journalBalanceTransactions -- infer remaining balancing amounts, balancing costs, and balance assignment amounts; + -- and check transactions balanced and (unless --ignore-assertions) balance assertions satisfied. + + -- Post-balancing enrichment + 14. journalInferCommodityStyles -- infer canonical commodity styles, now with all amounts present + 15. journalPostingsAddCommodityTags -- propagate commodity tags to postings + 16. journalTagCostsAndEquityAndMaybeInferCosts(2nd) -- if --infer-costs, infer costs from equity conversion postings + 17. journalInferEquityFromCosts -- if --infer-equity, infer equity conversion postings from costs + 18. journalInferMarketPricesFromTransactions -- infer market prices from costs + 19. journalRenumberAccountDeclarations -- renumber account declarations for consistent ordering + + -- Lot calculation + 20. journalCalculateLots -- with --lots: evaluate lot selectors, apply reduction methods, + -- calculate lot balances, add explicit lot subaccounts, + -- infer cost basis for bare disposals, normalize transacted cost +``` + +## Sequencing constraints + +These are the known ordering requirements between steps. +An arrow A → B means "A must run before B". + +### Hard constraints + +- **journalAddAccountTypes → journalClassifyLotPostings** + Classification looks up account types to identify Asset accounts. + +- **journalPostingsAddAccountTags → journalClassifyLotPostings** + Classification may need `lots:` tags inherited from account declarations (in `ptags`). + +- **journalInferBasisFromAccountNames → journalClassifyLotPostings** + Classification checks `acostbasis` to identify lot postings; cost basis inferred from account names must be present first. + +- **journalTagCostsAndEquityAndMaybeInferCosts(1st) → journalBalanceTransactions** + The balancer needs to know which costs are redundant (equity-paired) to ignore them. + +- **journalClassifyLotPostings → journalInferPostingsTransactedCost** + Transacted cost inference skips `transfer-to` postings (which have no selling price), + so it needs the `_ptype` tag to be present. + +- **journalInferPostingsTransactedCost → journalBalanceTransactions** + The balancer needs transacted costs to correctly infer missing amounts + (e.g., infer `-$500` for cash, not `-10 AAPL {$50}`). + +- **journalBalanceTransactions → journalPostingsAddCommodityTags** + Balancing may infer missing posting amounts, changing their commodity from `AUTO` to a + real commodity. Commodity tag propagation should see the real commodity so it can + add the right tags. (In practice this is a soft constraint: `journalInferPostingsCostBasis` + reads `jdeclaredcommoditytags` directly rather than relying on commodity tags in `ptags`.) + +- **journalClassifyLotPostings → journalCalculateLots** + Lot calculation reads `_ptype` tags to identify acquire/dispose/transfer postings. + +### Design decisions + +- **Acquisitions on lotful commodities are detected without explicit `{}`.** + Classification identifies positive lotful postings as acquire even without cost basis + annotation, symmetrically with bare dispose detection. Cost basis is inferred from + transacted cost (explicit or balancer-inferred) at lot calculation time. + +- **Classification before balancing resolves the poriginal conflict.** + Since `_ptype` tags are added before the balancer sets `poriginal`, the tags are + naturally preserved in `poriginal` and visible in `print --verbose-tags` output. + +## Key fields on Amount + +These fields are central to the inference pipeline: + +- **acost** — transacted cost (`@ $50` or `@@ $500`). Used by balancer, equity tagging. + Sources: parsed from journal, inferred by balancer, + inferred from equity postings (--infer-costs), + inferred from cost basis. + +- **acostbasis** — lot cost basis (`{$50}` or `{2024-01-15, "lot1", $50}`). + Sources: parsed from journal (explicit `{}` required for lot tracking); + also set by `journalCalculateLots` for bare disposals on lotful commodities. + Used by lot posting classification and lot calculation. + +These two fields are sometimes both present on the same amount (both explicit, or one inferred). +`journalInferPostingsTransactedCost` and `transactionInferBalancingCosts` may infer +`acost` from `acostbasis` or from counterpostings, so in later steps you cannot assume +that the presence of `acost` means the user wrote `@ $X`. + +## Key fields on Posting + +- **ptags** — all tags (user-written + inherited from account/commodity + hidden computed tags). + Tags are added by: account tag propagation, commodity tag propagation, equity tagging, + lot classification (`_ptype`), auto posting generation (`_generated-posting`). + +- **poriginal** — snapshot of the posting before amount/cost inference, used by `print` + to show journal entries close to how they were written. Set by: `transactionInferBalancingCosts`, + `transactionInferBalancingAmount`, balance assignment processing, + `postingInferTransactedCost`, `processDisposePosting` (bare disposals). + Since classification runs before these steps, + `_ptype` tags are naturally included in `poriginal`. + +## Conditional steps + +Several steps only run with specific flags: + +| Step | Flag | +|-----------------------------------|-------------------| +| journalAddForecast | --forecast | +| journalAddAutoPostings | --auto | +| journalTagCostsAndEquity (2nd) | --infer-costs | +| journalInferEquityFromCosts | --infer-equity | +| journalCalculateLots | --lots | diff --git a/doc/SPEC-journal.md b/doc/SPEC-journal.md new file mode 100644 index 00000000000..5791d952ede --- /dev/null +++ b/doc/SPEC-journal.md @@ -0,0 +1,215 @@ +# journal format + +Key features of hledger's journal syntax: + +1. Indentation matters: Postings must be indented (at least one space or tab) +2. Double-space rule: In many contexts, a double space separates fields (like between account name and amount) +3. Status marks: * for cleared, ! for pending +4. Virtual postings: Accounts in parentheses (account) or brackets [account] denote virtual postings +5. Comments: Lines starting with ;, #, *, %, or any indented line starting with ; +6. Balance assertions: =, =*, ==, ==* variants +7. Costs: @ for unit cost, @@ for total cost, {} for lot costs +8. Flexible amounts: Commodity symbols can appear before or after quantities; different number styles supported + +## Grammar + +A rough approximation of hledger's journal format. For documentation only. +May not be completely in sync with the implementation (JournalReader.hs, Common.hs). + +This EBNF is simplified and doesn't capture all edge cases +(like virtual postings, lot dates, multipliers in transaction modifiers, etc.) +but covers the core syntax elements. + +See also: hledger.m4.md > Journal > Journal cheatsheet + +```ebnf +(* Journal Structure *) +journal = { journal-item } ; + +journal-item = transaction + | periodic-transaction + | transaction-modifier + | directive + | market-price-directive + | comment-line + | blank-line ; + +(* Comments *) +comment-line = ( ";" | "#" | "*" | "%" ), { any-char - newline }, newline ; +line-comment = ";", { any-char - newline } ; +blank-line = [ whitespace ], newline ; + +(* Transactions *) +transaction = simple-date, [ "=", secondary-date ], [ whitespace ], + [ status ], [ code ], [ description ], [ line-comment ], newline, + [ transaction-comment ], + { posting } ; + +periodic-transaction = "~", [ whitespace ], period-expr, [ whitespace ], + [ status ], [ code ], [ description ], [ line-comment ], newline, + [ transaction-comment ], + { posting } ; + +transaction-modifier = "=", [ whitespace ], query-expr, [ line-comment ], newline, + [ transaction-comment ], + { posting } ; + +simple-date = date-year, date-sep, date-month, date-sep, date-day ; +secondary-date = simple-date ; +date-year = digit, digit, digit, digit ; +date-month = digit, [ digit ] ; +date-day = digit, [ digit ] ; +date-sep = "/" | "-" | "." ; + +status = "*" (* cleared *) + | "!" (* pending *) + ; + +code = "(", { any-char - ")" }, ")" ; + +description = { any-char - ";" - newline } ; + +transaction-comment = { comment-line-indented } ; +comment-line-indented = whitespace-1, ";", { any-char - newline }, newline ; + +(* Postings *) +posting = whitespace-1, [ status ], [ whitespace ], + account-name, [ whitespace ], + [ amount-expr ], [ whitespace ], + [ balance-assertion ], [ whitespace ], + [ line-comment ], newline, + [ posting-comment ] ; + +posting-comment = { comment-line-indented } ; + +account-name = account-name-component, { ":", account-name-component } ; +account-name-component = account-char, { account-char } ; +account-char = any-char - ";" - newline - ":" - " " (* two spaces *) ; + +(* Amounts *) +amount-expr = amount, [ cost-expr ] ; + +amount = [ "-" | "+" ], commodity-symbol, quantity-no-sep + | [ "-" | "+" ], quantity, [ whitespace ], commodity-symbol ; + +quantity = { digit | digit-group-mark }, decimal-mark, { digit } + | { digit | digit-group-mark } + | decimal-mark, digit, { digit } ; + +quantity-no-sep = { digit }, [ decimal-mark, { digit } ] ; + +commodity-symbol = ( letter, { letter | digit | symbol-char } ) + | ( symbol-char, { symbol-char } ) + | ( '"', { any-char - '"' }, '"' ) ; + +decimal-mark = "." | "," ; +digit-group-mark = "," | "." | " " ; + +(* Cost notation *) +cost-expr = unit-cost | total-cost | lot-cost ; + +unit-cost = "@", [ whitespace ], amount ; +total-cost = "@@", [ whitespace ], amount ; + +lot-cost = "{", [ whitespace ], [ "=" ], [ whitespace ], amount, [ whitespace ], "}" + | "{{", [ whitespace ], [ "=" ], [ whitespace ], amount, [ whitespace ], "}}" + | "[", simple-date, "]" ; + +(* Balance Assertions *) +balance-assertion = "=", [ "=" ], [ whitespace ], amount + | "=", [ "=" ], [ "=" ], [ whitespace ], amount (* == for subaccount inclusive *) ; + +(* Directives *) +directive = [ "!" | "@" ], directive-keyword ; + +directive-keyword = account-directive + | commodity-directive + | default-commodity-directive + | default-year-directive + | alias-directive + | end-aliases-directive + | payee-directive + | tag-directive + | apply-account-directive + | end-apply-account-directive + | include-directive + | decimal-mark-directive ; + +account-directive = "account", whitespace-1, account-name, + [ line-comment ], newline, + { subdirective } ; + +commodity-directive = "commodity", whitespace-1, + ( amount | commodity-symbol ), + [ line-comment ], newline, + { subdirective } ; + +subdirective = whitespace-1, ( "format", whitespace-1, amount + | any-text ), newline ; + +default-commodity-directive = "D", whitespace-1, amount, newline ; + +default-year-directive = ( "Y" | "year" | "apply year" ), + [ whitespace ], date-year, newline ; + +alias-directive = "alias", whitespace-1, ( account-name, "=", account-name + | "/", regex, "/", "=", replacement ), newline ; + +end-aliases-directive = "end", whitespace-1, "aliases", newline ; + +payee-directive = "payee", whitespace-1, ( quoted-text | text ), [ line-comment ], newline ; + +tag-directive = "tag", whitespace-1, tag-name, [ line-comment ], newline ; + +apply-account-directive = "apply", whitespace-1, "account", whitespace-1, account-name, newline ; + +end-apply-account-directive = "end", whitespace-1, "apply", whitespace-1, "account", newline ; + +include-directive = "include", whitespace-1, file-path, [ line-comment ], newline ; + +decimal-mark-directive = "decimal-mark", whitespace-1, ( "." | "," ), newline ; + +(* Market prices *) +market-price-directive = "P", [ whitespace ], datetime, + whitespace-1, commodity-symbol, + whitespace-1, amount, newline ; + +datetime = simple-date, [ whitespace-1, time ] ; +time = digit, digit, ":", digit, digit, [ ":", digit, digit ], [ timezone ] ; +timezone = ( "+" | "-" ), digit, digit, digit, digit ; + +(* Period expressions for periodic transactions *) +period-expr = interval, [ whitespace-1, "from", whitespace-1, simple-date ], + [ whitespace-1, "to", whitespace-1, simple-date ] + | simple-date, [ whitespace-1, "to", whitespace-1, simple-date ] + | "every", whitespace-1, interval ; + +interval = "daily" | "weekly" | "monthly" | "quarterly" | "yearly" + | "every", whitespace-1, number, whitespace-1, ( "days" | "weeks" | "months" | "quarters" | "years" ) + | "every", whitespace-1, nth, whitespace-1, day-of-week, + [ whitespace-1, "of", whitespace-1, "month" ] ; + +day-of-week = "monday" | "tuesday" | "wednesday" | "thursday" | "friday" | "saturday" | "sunday" ; + +nth = "1st" | "2nd" | "3rd" | digit, "th" ; + +(* Common elements *) +query-expr = any-text ; +file-path = any-text ; +tag-name = letter, { letter | digit | "-" | "_" } ; +quoted-text = '"', { any-char - '"' }, '"' ; +text = { any-char - ";" - newline } ; +any-text = { any-char - newline } ; +number = digit, { digit } ; + +whitespace = { " " | tab } ; +whitespace-1 = ( " " | tab ), { " " | tab } ; + +digit = "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9" ; +letter = "a" | ... | "z" | "A" | ... | "Z" ; +symbol-char = "$" | "£" | "€" | "¥" | etc. ; +any-char = ? any character ? ; +newline = "\n" ; +tab = "\t" ; +``` + diff --git a/doc/SPEC-lots.md b/doc/SPEC-lots.md new file mode 100644 index 00000000000..0e5b6e92911 --- /dev/null +++ b/doc/SPEC-lots.md @@ -0,0 +1,447 @@ +# Lot tracking + +Here is the current specification for lots functionality, most of which has been implemented in the lots branch. + +See also +- hledger manual: Basis / lots +- hledger manual: Lot reporting +- +- + + +## Lots + +A lot is an amount of some commodity, acquired and held for investment purposes, +to be disposed of later, hopefully at a better price. + +A lot's acquisition price and date are preserved, to + +1. help comply with tax rules, and +2. calculate the capital gain or loss, both unrealised (before disposal), and realised (at disposal). + +Historically, hledger has not provided much lot-tracking assistance: +- you could track lots manually by using one subaccount per lot +- the `balance --gain` report calculates gain/loss in simple cases +- the `close` command and `hledger-move` script help record lot movements + +## Lots mode + +There is a new `--lots` general flag, which enables + +- automatic lot inference, tracking and error checking +- display of per-lot subaccounts and balances in all reports. + +In the journal, lot operations can be recorded + +1. implicitly, with minimal notation and maximum inference; +2. partly explicitly, with any missing parts inferred; +3. or fully explicitly, requiring no inference. + +A typical workflow is to use 1 primarily, or when processing/converting old journals; +and use `print` to convert to 3 for troubleshooting or reporting. + +## Transacted cost and cost basis + +- Transacted cost is the conversion rate used within a particular multi-commodity transaction. + It is recorded with a @ or @@ annotation after the amount(s). + +- Cost basis is the nominal acquisition cost of a lot, + along with its acquisition date and perhaps a label. + It is preserved (along with the lot's balance), throughout the lifetime of the lot, from acquisition through transfers to final disposal. + It is recorded with hledger lot syntax (consolidated {} notation); + we can also read ledger lot syntax (separate {}, [], () annotations). + +"Transacted cost" is an awkward term, overlapping with "cost basis", +but it avoids too much change from the historical "cost", and we don't have a better alternative, currently. + +The hledger manual has more detail on these. + +A posting amount can have transacted cost, cost basis, both, or neither. +When displaying a posting with both, we show cost basis before transacted cost (like beancount). + +## Lot names + +A lot's cost basis also serves as the lot name. + +In hledger, a full lot name is rendered as either {YYYY-MM-DD, "LABEL", COST} or {YYYY-MM-DD, COST}. +That is, two or three parts inside curly braces: +- a date in strict ISO format +- an optional label in double quotes +- a single-commodity hledger amount +with a comma and space between them. +When parsing, spaces inside the braces and around the commas are optional and ignored. +This is similar to Beancount's lot syntax, except it requires DLC order (date, label, cost) and it supports hledger's flexible amount syntax. + +Full lot names may be used as subaccount names, identifying lots within a parent account. +Eg assets:stocks:aaaa:{2026-02-10, "lot1", $50}. +So the label and the cost's commodity symbol may not contain double-quotes, colons, or semicolons. +When a lot subaccount is written explicitly by the user, the cost basis is parsed from the subaccount name +and applied to the posting's amounts (see "Inferring cost basis from lot subaccount names"). +When checking account names, lot subaccounts are ignored; only the base account needs to be declared. + +Partial lot names are also used; these have some or all of the parts missing. +The minimal partial lot name is rendered as {}. + +## Lot ids + +A lot's id is just the date and label parts. +Lot ids must be unique and ordered, so if there are multiple lots with the same date, +labels must be used to 1. disambiguate and 2. order them. +This is normally done by beginning the label with a time of day (HH:MM, or a more precise time as needed) +or a intra-day sequence number (NNNN, with enough leading zeros so that a day's lot ids sort nicely in numeric order; we'll assume four digits in total.) +Labels are generated only when needed to satisfy lot id uniqueness rules. +If there are multiple same-date, same-commodity acquisitions (across all accounts) with no labels, +hledger adds NNNN labels based on parse/processing order. +If such acquisitions do have user-provided labels, hledger checks that the resulting lot ids are unique +(across all accounts, to be safe) and reports an error otherwise. + +What is the scope of lot ids' uniqueness and ordering ? +It is 1. per commodity (lots of different commodities do not clash), +and 2. either per account, or across all accounts. +The latter needs to be configurable somehow, for different time periods. + +Eg in the US, tax rules require that before 2025, lots are tracked across all accounts, +whereas after 2025, lots are tracked separately within each account. + +## Lot selectors + +A full or partial lot name/cost basis, when used in a posting with a negative amount, +selects an existing lot, rather than creating a new one. +So in this case we call it a "lot selector". + +The terms "hledger lot syntax", "cost basis", "lot name", "lot selector" can sometimes be a bit interchangeable; +they all involve the same notation, which has different meanings depending on context. + +## Data types + +All fields are Maybe, so the same types serve for both definite and partial values: + +``` +data CostBasis = CostBasis { cbDate :: Maybe Day, cbLabel :: Maybe Text, cbCost :: Maybe Amount } +data LotId = LotId { lotDate :: Day, lotLabel :: Maybe Text } +``` + +A definite cost basis (used for a fully resolved lot) has all fields present except cbLabel which is only present when needed for uniqueness. +A partial cost basis (used as a lot selector or during inference) may have any fields missing. + +## Lotful commodities and accounts + +Commodities and/or accounts can be declared as lotful, by adding a "lots" tag to their declaration. +This signifies that their postings always involve a cost basis and lots, +so these should be inferred if not written explicitly. + +(In future, we may also recognise some common commodity symbols as lotful, even without the lots tag.) + +## Inferring cost basis from transacted cost + +In postings with a positive amount, involving a lotful commodity or account, +which have a transacted cost but no explicit cost basis annotation, +or an empty cost basis annotation (`{}`), +we infer a cost basis from the transacted cost. + +## Inferring cost basis from lot subaccount names + +When a posting's account name contains a lot subaccount (a final component starting with `{`), +the cost basis is parsed from the subaccount name and applied to the posting's amounts. +If the amount already has a cost basis annotation, the two are merged: +any `Nothing` fields are filled in from the other source, +and any fields present in both must agree (otherwise an error is reported). + +This allows `print --lots` output (which has explicit lot subaccounts) to be re-read +without losing cost basis information, and allows users to write lot subaccounts +directly without a redundant `{}` annotation on the amount. + +(journalInferBasisFromAccountNames, runs unconditionally before journalClassifyLotPostings) + +## Lot postings + +After inferring cost basis, we identify and classify lot postings. +A `ptype` tag is added to each classified posting to record its type: +acquire, dispose, transfer-from, transfer-to, or gain. + +(`journalClassifyLotPostings` → `transactionClassifyLotPostings`) + +### Classification summary + +In short: a lotful commodity entering an asset account is an **acquire**. +A lotful commodity leaving an asset account is a **dispose**. +A lotful commodity moving between asset accounts is a **transfer**. +The details below handle edge cases: bare postings without `{...}`, +equity transfers, partial transfers with fees, and cost source inference. + +### Classification rules + +Classification proceeds in several steps. Virtual (parenthesised) postings +are never classified. + +**1. Same-account transfer pairs.** +Within each account, negative and positive postings with the same commodity +and exact absolute quantity are paired as transfer-from / transfer-to. +When there are more of one sign than the other, the excess are left +unmatched and classified by the rules below. + +**2. Postings with cost basis (`{...}`).** +These are classified regardless of account type: + +- **Negative** → `dispose`, or `transfer-from` if a counterpart posting + (same commodity and quantity, different account) exists. +- **Positive** → `acquire`, or `transfer-to` if a counterpart exists. +- **Equity transfer override**: if the posting has no transacted price + (`@ ...`) and an equity counterpart posting (no cost basis) exists in + the transaction, it is classified as transfer-from/to instead of + dispose/acquire. This handles `close --clopen --lots` style equity + transfers where lots move to/from equity in separate transactions. + +**3. Bare postings on lotful asset accounts (no cost basis).** +These require an asset account type and a lotful commodity or account +(`lots:` tag). They are tried in this order: + +- **Negative lotful** → + `transfer-from` if a counterpart (same commodity, exact quantity, + different account) exists, or if another asset account in the same + transaction receives a positive lotful amount of the same commodity + (transfer+fee pattern, where source qty > dest qty due to fees). + Otherwise `dispose` if the posting has a transacted price. + +- **Positive lotful, no price, with transfer-from counterpart** → + `transfer-to`. The counterpart can match by exact quantity or by + commodity only (for transfer+fee patterns). This handles bare + transfer-to postings in lotful commodities/accounts that don't repeat + the `{...}` notation. + +- **Positive (any), no cost basis, with cost-basis transfer-from counterpart** → + `transfer-to`. The counterpart can match by exact quantity or by + commodity only (for transfer+fee patterns). This handles the receiving + side of transfers where the sending side has `{...}` but the receiving + side doesn't. + +- **Positive lotful with a plausible cost source** → + `acquire`. A cost source is plausible when the posting has a transacted + price (`@ ...`), or the transaction contains a different-commodity posting + (allowing the balancer to infer a cost), or a transfer-from counterpart + exists. Without any of these, no lot can be created and classification is + skipped. + +**4. Gain accounts.** +Postings in accounts with type `Gain` (and not otherwise classified) get +ptype `gain`. + +### Unclassified lotful postings + +With `--lots`, a real posting with a nonzero lotful commodity in an asset account +that was not classified (no `_ptype` tag) is an error. +This catches lotful postings that need lot tracking but weren't recognised. + +Zero-amount lotful postings (e.g. `0 AAPL = 100 AAPL` balance assertions) +are exempt: no lot movement occurs, so no classification is needed. +This applies regardless of whether the amount was written explicitly or left implicit. + +(`isUnclassifiedLotfulPosting` in Lots.hs) + +### Counterpart detection + +Transfer detection uses precomputed maps keyed by (commodity, |quantity|): + +- `negCBAccts`: accounts with negative postings that have cost basis + (any account type), or are bare lotful negatives on asset accounts. + Non-asset bare lotful negatives (e.g. revenue) are excluded. +- `posCBAccts`: accounts with positive postings that have cost basis. +- `posNoCBAccts`: accounts with positive asset postings without cost basis. + +A posting has a "counterpart" when the opposite-sign map contains a +different account for the same commodity and quantity. This requires exact +quantity matching for the primary check (`hasCounterpart`, +`hasTransferFromCounterpart`). + +A commodity-only fallback (`hasTransferFromCommodityMatch`) checks +`negCBAccts` for any entry with the same commodity in a different account, +ignoring quantity. This is used by `shouldClassifyLotful` and +`shouldClassifyBareTransferTo` to detect transfer-to postings in +transfer+fee patterns where the destination receives less than the source +sends. + +### Main functions + +- `journalClassifyLotPostings`: entry point, maps over transactions. +- `transactionClassifyLotPostings`: per-transaction classifier. + - `sameAcctTransferSet`: precomputed set of same-account transfer pair indices. + - `negCBAccts`, `posCBAccts`, `posNoCBAccts`: counterpart maps. + - `hasCounterpart`, `hasTransferFromCounterpart`, `hasTransferFromCommodityMatch`: counterpart lookups. + - `classifyAt`: per-posting dispatch. + - `shouldClassify` → `shouldClassifyWithCostBasis`, `shouldClassifyNegativeLotful`, + `shouldClassifyLotful`, `shouldClassifyBareTransferTo`, `shouldClassifyPositiveLotful`. + - `postingIsLotful`: checks for `lots:` tag on commodity or account. + +## Inferring transacted cost from cost basis + +After classifying lot postings, +in acquire postings which have no transacted cost annotation, +we infer a transacted cost from the cost basis. + +(journalInferPostingsTransactedCost) + +## Lot posting effects + +- An acquire posting creates a new lot, with a cost basis either specified + or inferred from the transacted cost (or perhaps market price, in future). + +- A transfer-from posting selects one or more lots to be transferred elsewhere, + following some selection/reduction method. Either + - it has a lot selector (a full or partial cost basis annotation), + which must unambiguously select a single existing lot ("SPECID" method) + - or it has no lot selector, in which case a default method is used ("FIFO" method), selecting one or more existing lots. + +- A transfer-to posting mirrors a corresponding transfer-from posting in the same transaction, + recreating its lot(s) under a new parent account. + It doesn't need a lot selector; if it has one, it must select the same lot as the transfer-from posting. + Transfer postings (both from and to) must not have explicit transacted cost (@ or @@); this is an error. + When the transfer-to quantity is less than the transfer-from quantity (a transfer+fee pattern), + lots are selected for the full source quantity, then split: the transfer portion's lots are + recreated at the destination, and the fee portion's lots are consumed from source only + (generating from-postings on lot subaccounts with no corresponding to-postings, like a + silent disposal with no gain). + +- An equity transfer is a variant of a lot transfer that happens in two parts across + separate transactions (e.g. a closing transaction transfers lots into equity, and an + opening transaction transfers them back out). In the closing transaction, transfer-from + postings reduce lots from the lot state. In the opening transaction, transfer-to + postings re-add the lots to the lot state, preserving their original cost basis. + The equity postings do not track lots. + +- A dispose posting selects one more lots to be disposed (sold), like a transfer-from posting. + It must also have a transacted cost, either explicit or inferred from transaction balancing + (or from market price, in future). + When the dispose posting has no cost basis annotation but involves a lotful commodity or account, + the cost basis is inferred from the selected lot, and the transacted cost + (if inferred by the balancer as @@) is normalized to unit cost (@). + +## Reduction methods + +The reduction methods are: + +FIFO (oldest first), LIFO (newest first), HIFO (highest cost first), +AVERAGE (weighted average cost basis), and SPECID (explicit selection via lot selector). +All methods are per-account: they only consider lots within the posting's account. + +Global validation variants: FIFOALL, LIFOALL, HIFOALL, AVERAGEALL. +These select per-account like the base methods, but validate that the selected lots +would also be chosen first if all accounts' lots were considered together. +If not, an error is raised showing which lots on other accounts have higher priority. +AVERAGEALL additionally computes the weighted average cost across the global pool +(all accounts), not just the posting's account. + +AVERAGE uses FIFO consumption order for bookkeeping, but applies +the pool's weighted average per-unit cost as the disposal cost basis. +The method is configurable per account and per commodity via the `lots:` tag, +and per posting via the `lots:` tag on a posting comment. + +In future, the method might be specified with an annotation like {!METHOD, ...} inside the lot syntax. + +## Lot transactions + +Lot transactions are transactions with lot postings. +We require that a transaction's lot postings are all of similar type: all acquire, or all transfer, or all dispose. +So lot transactions can be classified as "acquire", "transfer", or "dispose" (we don't record this explicitly). + +## Gain postings + +A gain posting is a posting to an account of Gain type (a subtype of Revenue). +We use this gain account to record capital gain and/or capital loss (depending on the amount sign). +The special account type helps hledger identify these postings. + +## Transaction balancing + +All journal entries, including lot-related ones, must pass normal transaction balancing. +When summing postings it uses their transacted costs (not cost basis), if any. +And it excludes (ignores) capital gain/loss postings, identified by their Gain account type. +When the postings' sum is nonzero, and amountless postings exist, it can infer one balancing amount in each unbalanced commodity. + +## Disposal balancing + +Journal entries involving lot disposals get this additional balancing pass. +When summing postings it uses their cost basis (not transacted cost), if any. +And it includes gain postings, or will infer one if needed. + +A disposal transaction's total realised capital gain/loss is calculated by +comparing the lot acquisition cost(s) for each dispose posting, and the total transacted disposal price. + +If the transaction contains a gain posting (or more than one), the recorded gain is expected to match the calculated gain. +Otherwise, a gain posting is inferred, posting the calculated gain to the alphabetically first Gain account. +Or if there is an amountless gain posting (at most one per commodity), we fill in its amount. +This helps the transaction to pass disposal balancing. + +The inclusion/exclusion gain postings allows both kinds of transaction balancing to succeed with the same journal entries. + +## Balance assertions + +A balance assertion on a dispose or transfer posting (eg `= 0 AAPL`) runs before `--lots` processing +(in `journalBalanceTransactions`), when the posting is still on the parent account — so it checks the +parent account's balance, as expected. + +When `--lots` later splits that posting onto lot subaccounts, the assertion is removed from the lot +postings and re-attached to a new zero-amount `_generated-posting` on the original parent account, +with `bainclusive = True` (ie the `=*` syntax). This makes the assertion check the inclusive balance +of the parent plus all its lot subaccounts, which is the semantically correct interpretation when the +output is re-read later (eg after `print --lots -x`). + +If the original posting's account is already an explicit lot subaccount (eg +`assets:stocks:{2026-01-15, $50}`), the assertion is left on the split posting unchanged, since it +already targets the right account. + +`close --lots` does not generate balance assertions on lot subaccount postings in the +closing transaction (e.g. `assets:stocks:{2026-01-15, $50}`), because these assertions +would be invalid when the output is re-read: balance assertions run before lot calculation, +so the lot subaccounts would not yet have their expected balances. Non-lot-subaccount +postings (e.g. `assets:cash`) and opening transaction postings retain their assertions. + +## Processing pipeline + +Most lot-related processing is optional, enabled by the --lots flag. +However, cost basis inference from lot subaccount names and lot classification run unconditionally +(since they affect transaction balancing and other pipeline stages). +See SPEC-finalising for more details of the implementation. + +## When might cost basis differ from the transacted cost ? + +In many real-world scenarios, a lot's cost basis (the value recorded for tax purposes) +can differ from the price actually paid to acquire it. These may include: + +- **Gifts** — the recipient inherits the donor's original cost basis (carryover basis), not the fair market value at the time of the gift. +- **Inheritance** — inherited assets get a "stepped-up" basis to fair market value at the date of death. +- **Employee stock options (NSOs)** — the bargain element (FMV minus exercise price) is taxed as ordinary income, and cost basis becomes the FMV at exercise, not the price paid. +- **Incentive stock options (ISOs)** — cost basis is the exercise price for regular tax, but FMV at exercise for AMT, so the same lot can have two different bases depending on tax context. +- **RSUs** — cost basis is FMV at vesting; the recipient paid nothing. +- **ESPPs** — shares bought at a discount; basis treatment depends on qualifying vs disqualifying disposition. +- **Wash sales** — disallowed loss from a prior sale is added to the cost basis of the replacement shares. +- **Corporate actions** — spin-offs, mergers, and stock splits cause cost basis to be allocated or adjusted in ways unrelated to any payment. + +## Examples + +### Disposal + +A very implicit disposal: + +``` +2026-03-01 sell + assets:stocks -15 AAPL + assets:cash $900 + revenue:gains +``` + +or: + +``` +2026-03-01 sell + assets:stocks -15 AAPL @ $60 + assets:cash + revenue:gains +``` + +Explanation: + +1. revenue:gains is recognised as a Gain account so ignored by normal transaction balancing +2. $60 sale price or $900 sale amount is inferred to balance the transaction + +if in --lots mode: +3. 15 AAPL are reduced from one or more existing lots selected with assets:stock's or AAPL's or default (FIFO) reduction method +4. disposal balancing infers the gain amount based on the reduction order, selected lot(s)' cost bases, and sale amount diff --git a/doc/SPEC-print.md b/doc/SPEC-print.md new file mode 100644 index 00000000000..29a5ca7ff33 --- /dev/null +++ b/doc/SPEC-print.md @@ -0,0 +1,49 @@ +# SPEC: print command + +Notes on some of print's behaviour. + +## Effects of certain output flags + +### No output flags (default output) + +By default, print tries to show each entry as it is written in the journal file, +except for alignment. And it shows entries in date-then-parse order. + +### `--round` + +Controls rounding/padding of displayed amounts: + +- `none` — show original decimal digits, as in the journal (default) +- `soft` — add or remove trailing decimal zeros to match commodity precision +- `hard` — round posting amounts to commodity precision (can unbalance transactions) +- `all` — also round cost amounts to commodity precision (can unbalance transactions) + +### `--verbose-tags` + +Makes certain normally-hidden tags visible (in comments): + +- `ptype: acquire/dispose/transfer-from/transfer-to` — lot posting classification +- `cost-tagged:` — marks postings that have or were given a transaction cost +- `conversion-tagged:` — marks equity conversion postings +- `generated-posting:` — marks auto-generated postings (from transaction modifiers or --infer-equity) +- `modified-transaction:` — marks transactions modified by auto posting rules +- `generated-transaction: ` — marks forecast transactions from periodic rules + +Without this flag, these tags still exist internally (queryable) but don't appear in print output. + +### `-x` / `--explicit` + +Shows all inferred balancing amounts and balancing costs: + +- Inferred amounts are shown +- Inferred costs are shown +- Balance assignment amounts are shown explicitly + +### `--lots` + +Triggers lot calculation, which restructures postings: + +- Cost basis fields are made explicit, with missing parts filled in +- Lots acquired on the same day get uniquifying labels added if needed +- All lot postings get specific lot subaccounts added (e.g. `assets:stocks` → `assets:stocks:{2026-01-15, $50}`) +- Transfer postings and dispose postings affecting multiple lots are split into one per lot diff --git a/doc/SPEC-special-postings.md b/doc/SPEC-special-postings.md new file mode 100644 index 00000000000..02671b366c4 --- /dev/null +++ b/doc/SPEC-special-postings.md @@ -0,0 +1,95 @@ +# Rules for detecting special postings + +Here's an overview of ways hledger infers things based on certain configurations of postings. + + 1. journalClassifyLotPostings (Lots.hs) + + Detects lot-related postings and tags them with _ptype. Classifications: + + ┌──────────────────┬───────────────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────┐ + │ Pattern │ Tag │ Conditions │ + ├──────────────────┼───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────┤ + │ acquire │ _ptype:acquire │ Positive amount + cost basis {}; OR positive lotful amount (commodity/account has lots: tag) │ + ├──────────────────┼───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────┤ + │ dispose │ _ptype:dispose │ Negative amount + cost basis; OR negative lotful amount, with no opposite-sign counterpart in │ + │ │ │ a different account │ + ├──────────────────┼───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────┤ + │ transfer-from │ _ptype:transfer-from │ Negative amount + cost basis (or lotful), AND a matching positive counterpart in a different │ + │ │ │ account with same commodity and exact quantity; OR negative lotful amount with no transacted │ + │ │ │ price and an equity counterpart posting (equity transfer) │ + ├──────────────────┼───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────┤ + │ │ │ Positive amount + cost basis with matching negative counterpart; OR positive lotful amount │ + │ transfer-to │ _ptype:transfer-to │ with no @ cost and a matching transfer-from counterpart; OR bare positive asset posting with a │ + │ │ │ matching transfer-from counterpart; OR positive lot posting with an equity counterpart (equity │ + │ │ │ transfer, e.g. opening balances from close --clopen --lots) │ + ├──────────────────┼───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────┤ + │ gain │ _ptype:gain │ Posting to a Gain-type account │ + ├──────────────────┼───────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────┤ + │ same-account │ _ptype:transfer-from / │ Within the same account, positive and negative postings with the same commodity and exact │ + │ transfer │ transfer-to │ absolute quantity are paired as transfers │ + └──────────────────┴───────────────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────┘ + + Detection is per-transaction. Transfer detection uses precomputed maps keyed by (commodity, |quantity|) for O(n) matching. + + --- + 2. journalTagCostsAndEquityAndMaybeInferCosts (Journal.hs → Transaction.hs) + + Detects equity conversion postings and corresponding costful postings. Called twice in the pipeline (before balancing with addcosts=False to detect, + after with addcosts=True to add costs when --infer-costs). + + ┌─────────────────────────┬─────────────────────┬───────────────────────────────────────────────────────────────────────────────────────────────┐ + │ Pattern │ Tag │ Conditions │ + ├─────────────────────────┼─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────┤ + │ conversion posting pair │ _conversion-posting │ Two adjacent postings to accounts declared as Conversion type (type V), each with a │ + │ │ │ single-commodity amount and no cost │ + ├─────────────────────────┼─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────┤ + │ costful posting │ _cost-posting │ A non-conversion posting whose amount matches -ca1 and whose cost matches ca2 (or vice │ + │ matching conversions │ │ versa), where ca1/ca2 are the conversion pair amounts │ + ├─────────────────────────┼─────────────────────┼───────────────────────────────────────────────────────────────────────────────────────────────┤ + │ costless posting │ _cost-posting │ (When no costful match exists) A costless single-commodity posting whose amount matches -ca1 │ + │ matching conversions │ │ or -ca2; must be unambiguous (no other costless posting with the same amount) │ + └─────────────────────────┴─────────────────────┴───────────────────────────────────────────────────────────────────────────────────────────────┘ + + In addcosts=True mode, the matched costless posting also gets a TotalCost added. + + The _cost-posting tag causes the balancer to strip costs from that posting (preventing double-counting with conversion postings). + + --- + 3. journalInferEquityFromCosts (Journal.hs → Transaction.hs → Posting.hs) + + Detects costful postings that lack corresponding equity conversion postings: + + ┌────────────────────────┬───────────────────────────────────────────────┬──────────────────────────────────────────────────────────────────────┐ + │ Pattern │ Action │ Conditions │ + ├────────────────────────┼───────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┤ + │ costful posting │ Generates a pair of _conversion-posting + │ Posting has cost amounts (@ or @@) AND is NOT already tagged │ + │ without conversions │ _generated-posting tagged postings │ _cost-posting (i.e., not already matched to existing conversion │ + │ │ │ postings) │ + └────────────────────────┴───────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────┘ + + For each cost amount, two conversion postings are generated under :: with amounts that offset the cost. + + --- + Summary of all hidden tags used + + ┌──────────────────────┬─────────────────────────────────────────────────────────────────────┬──────────────────────────────────────────────────┐ + │ Tag │ Set by │ Meaning │ + ├──────────────────────┼─────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────┤ + │ _ptype:acquire │ journalClassifyLotPostings │ Lot acquisition │ + ├──────────────────────┼─────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────┤ + │ _ptype:dispose │ journalClassifyLotPostings │ Lot disposal │ + ├──────────────────────┼─────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────┤ + │ _ptype:transfer-from │ journalClassifyLotPostings │ Lot transfer source │ + ├──────────────────────┼─────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────┤ + │ _ptype:transfer-to │ journalClassifyLotPostings │ Lot transfer destination │ + ├──────────────────────┼─────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────┤ + │ _ptype:gain │ journalClassifyLotPostings / journalInferAndCheckDisposalBalancing │ Capital gain/loss posting │ + ├──────────────────────┼─────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────┤ + │ _cost-posting │ journalTagCostsAndEquityAndMaybeInferCosts / │ Has (or could have) cost matching conversion │ + │ │ journalInferEquityFromCosts │ postings │ + ├──────────────────────┼─────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────┤ + │ _conversion-posting │ journalTagCostsAndEquityAndMaybeInferCosts / │ Equity conversion posting │ + │ │ journalInferEquityFromCosts │ │ + ├──────────────────────┼─────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────┤ + │ _generated-posting │ journalInferEquityFromCosts │ Machine-generated posting │ + └──────────────────────┴─────────────────────────────────────────────────────────────────────┴──────────────────────────────────────────────────┘ diff --git a/doc/dev.md b/doc/dev.md index 51ce8582d46..feb2064ce3d 100644 --- a/doc/dev.md +++ b/doc/dev.md @@ -12,6 +12,7 @@ and then symlinked into the hledger_site repo for rendering on hledger.org. - [Project README](dev-README.md) - [ROADMAP](ROADMAP.md) - [ACHIEVEMENTS](ACHIEVEMENTS.md) +- [AI](AI.md) - [BENCHMARKS](BENCHMARKS.md) - [CHANGELOGS](CHANGELOGS.md) - [CLIMATE](CLIMATE.md) @@ -25,6 +26,7 @@ and then symlinked into the hledger_site repo for rendering on hledger.org. - [EXAMPLES](EXAMPLES.md) - [FILES](FILES.md) - [FINANCE](FINANCE.md) +- [IMPACT](IMPACT.md) - [ISSUES](ISSUES.md) - [JUST-MAKE-SHAKE](JUST-MAKE-SHAKE.md) - [LINKS](LINKS.md) diff --git a/examples/investing/export-lots-workflow/Justfile b/examples/lots/export-lots-workflow/Justfile similarity index 100% rename from examples/investing/export-lots-workflow/Justfile rename to examples/lots/export-lots-workflow/Justfile diff --git a/examples/investing/export-lots-workflow/README.md b/examples/lots/export-lots-workflow/README.md similarity index 100% rename from examples/investing/export-lots-workflow/README.md rename to examples/lots/export-lots-workflow/README.md diff --git a/examples/investing/export-lots-workflow/beancount-gains-howto.md b/examples/lots/export-lots-workflow/beancount-gains-howto.md similarity index 100% rename from examples/investing/export-lots-workflow/beancount-gains-howto.md rename to examples/lots/export-lots-workflow/beancount-gains-howto.md diff --git a/examples/investing/export-lots-workflow/ledger-gains-howto.md b/examples/lots/export-lots-workflow/ledger-gains-howto.md similarity index 100% rename from examples/investing/export-lots-workflow/ledger-gains-howto.md rename to examples/lots/export-lots-workflow/ledger-gains-howto.md diff --git a/examples/investing/export-lots-workflow/rustledger-gains-howto.md b/examples/lots/export-lots-workflow/rustledger-gains-howto.md similarity index 100% rename from examples/investing/export-lots-workflow/rustledger-gains-howto.md rename to examples/lots/export-lots-workflow/rustledger-gains-howto.md diff --git a/examples/lots/lot-entries.journal b/examples/lots/lot-entries.journal new file mode 100644 index 00000000000..6016f6ae1fc --- /dev/null +++ b/examples/lots/lot-entries.journal @@ -0,0 +1,145 @@ +; * Example lot-tracking entries for hledger --lots. +; +; This file demonstrates common journal entry patterns for acquiring, +; transferring, and disposing of lot-tracked commodities (stocks, cryptocurrency, etc.). +; Use it as a starting point or reference for your own entries. +; +; To try it: hledger -f lots-entries.journal print --lots [--verbose-tags] +; hledger -f lots-entries.journal bal --lots [-M] +; +; Prerequisites: declare lot-tracked commodities with a lots: tag. + +commodity ETSY ; lots: +account revenue:gains ; type: G + +; ============================================================================ +; ** 1. ACQUIRE - buying or receiving a lot-tracked commodity +; ============================================================================ + +; 1a. Buy with no fee. +; {$50} records the per-unit cost basis. +2026-01-01 buy shares + assets:cash -$500 + assets:broker 10 ETSY {$50} + +; 1b. Buy with a fee in the base currency ($). +; The fee is a separate expense; the cost basis remains $60/share. +2026-02-01 buy shares, $ fee + assets:cash -$610 + expenses:fees $10 + assets:broker 10 ETSY {$60} + +; 1c. Buy with a fee in the acquired commodity. +; The broker takes 0.1 ETSY as a fee; you receive 9.9 ETSY. +; The fee posting needs @ so the lot system knows it's a disposal, not a transfer. +2026-03-01 buy shares, stock fee + assets:cash -$700 + expenses:fees 0.1 ETSY @ $70 + assets:broker 9.9 ETSY {$70} + +; 1d. Buy with fees in both commodities. +2026-04-01 buy shares, both fees + assets:cash -$810 + expenses:fees $10 + expenses:fees 0.1 ETSY @ $80 + assets:broker 9.9 ETSY {$80} + +; ============================================================================ +; ** 2. TRANSFER - moving lots between accounts, preserving cost basis +; ============================================================================ + +; 2a. Transfer with no fee. +; No transacted price or cost basis mentioned - the lots move with their original cost basis intact. +; By default, lots are reduced FIFO, oldest lot first ({2026-01-01, $50}). +2026-05-01 transfer to another broker + assets:broker -10 ETSY + assets:broker2 10 ETSY + +; 2b. Transfer with a fee in the base currency. +; FIFO selects the next oldest lot ({2026-02-01, $60}). +2026-06-01 transfer, $ fee + assets:broker -10 ETSY + assets:broker2 10 ETSY + assets:cash -$5 + expenses:fees $5 + +; 2c. Transfer with a fee in the transferred commodity. +; The fee (0.1 ETSY) is the difference between the from and to amounts. +; FIFO selects from the {2026-03-01, $70} lot (9.9 available). +2026-07-01 transfer, stock fee + assets:broker -5 ETSY + expenses:fees 0.1 ETSY + assets:broker2 4.9 ETSY + +; ============================================================================ +; ** 3. DISPOSE - selling or spending a lot-tracked commodity +; ============================================================================ + +; 3a. Sell with no fee, at a gain. +; {$50} selects the lot; @ $90 is the selling price. +; The gain posting balances the difference between cost and sale price. +2026-08-01 sell with at a gain, using specific identification + assets:broker2 -10 ETSY {$50} @ $90 + assets:cash $900 + revenue:gains -$400 + +; 3b. Sell with no fee, at a loss. +2026-09-01 sell at a loss + assets:broker2 -10 ETSY {$60} @ $55 + assets:cash $550 + revenue:gains $50 + +; 3c. Sell with a $ fee. +2026-10-01 sell with $ fee + assets:broker -4.8 ETSY {$70} @ $90 + assets:cash $422 + expenses:fees $10 + revenue:gains -$96 + +; 3d. Sell with a stock fee. +; You sell 5 from the {$80} lot; the broker keeps 0.1 ETSY as a fee. +2026-10-15 sell with stock fee + assets:broker -5 ETSY {$80} @ $90 + expenses:fees 0.1 ETSY {$80} @ $90 + assets:cash $441 + revenue:gains -$49 + +; 3e. Sell using commodity's/account's reduction method — +; {} chooses the most appropriate available lot(s) (FIFO by default). +; The gain posting's amount is inferred. +2026-11-01 sell using auto selection and inferred gain amount + assets:broker -4.9 ETSY {} @ $90 + assets:cash $441 + revenue:gains + +; 3f. Sell without writing a gain posting — it is added automatically, +; using the first account of Gain type, otherwise revenue:gains. +2026-12-01 sell, with inferred gain posting + assets:broker2 -4.9 ETSY {} @ $90 + assets:cash $441 + ; gains posting is inferred here + +; ============================================================================ +; ** 4. REVENUE & EXPENSES - lotful commodities in income/expense accounts +; ============================================================================ +; Revenue and expense accounts don't get lot subaccounts; +; only the asset account tracks the lot. + +; 4a. Receive stock as income (e.g. stock compensation or dividend). +; A new lot is acquired; the income is recorded in $. +2027-01-01 stock compensation + revenue:salary -$900 + assets:broker2 10 ETSY {$90} + +; 4b. Donate stock to charity. +; A disposal at market price; the difference from cost basis is recorded as a gain/loss. +2027-03-01 charitable donation + assets:broker2 -5 ETSY {$90} @ $100 + expenses:donations $500 + revenue:gains -$50 + +; 4c. Pay a contractor in stock. Similar to the above. +2027-04-01 pay contractor + assets:broker2 -5 ETSY {$90} @ $100 + expenses:contractors $500 + revenue:gains -$50 diff --git a/examples/investing/pta-lot-tracking-tests/Makefile b/examples/lots/pta-lot-tracking-tests/Makefile similarity index 100% rename from examples/investing/pta-lot-tracking-tests/Makefile rename to examples/lots/pta-lot-tracking-tests/Makefile diff --git a/examples/investing/pta-lot-tracking-tests/README.md b/examples/lots/pta-lot-tracking-tests/README.md similarity index 100% rename from examples/investing/pta-lot-tracking-tests/README.md rename to examples/lots/pta-lot-tracking-tests/README.md diff --git a/examples/investing/pta-lot-tracking-tests/beancount.beancount b/examples/lots/pta-lot-tracking-tests/beancount.beancount similarity index 100% rename from examples/investing/pta-lot-tracking-tests/beancount.beancount rename to examples/lots/pta-lot-tracking-tests/beancount.beancount diff --git a/examples/investing/pta-lot-tracking-tests/beancount1.beancount b/examples/lots/pta-lot-tracking-tests/beancount1.beancount similarity index 100% rename from examples/investing/pta-lot-tracking-tests/beancount1.beancount rename to examples/lots/pta-lot-tracking-tests/beancount1.beancount diff --git a/examples/investing/pta-lot-tracking-tests/hledger-lots.hledger b/examples/lots/pta-lot-tracking-tests/hledger-lots.hledger similarity index 100% rename from examples/investing/pta-lot-tracking-tests/hledger-lots.hledger rename to examples/lots/pta-lot-tracking-tests/hledger-lots.hledger diff --git a/examples/investing/pta-lot-tracking-tests/ledger.ledger b/examples/lots/pta-lot-tracking-tests/ledger.ledger similarity index 100% rename from examples/investing/pta-lot-tracking-tests/ledger.ledger rename to examples/lots/pta-lot-tracking-tests/ledger.ledger diff --git a/examples/investing/pta-lot-tracking-tests/lotter.hledger b/examples/lots/pta-lot-tracking-tests/lotter.hledger similarity index 100% rename from examples/investing/pta-lot-tracking-tests/lotter.hledger rename to examples/lots/pta-lot-tracking-tests/lotter.hledger diff --git a/examples/investing/pta-lot-tracking-tests/lotter.lotter b/examples/lots/pta-lot-tracking-tests/lotter.lotter similarity index 100% rename from examples/investing/pta-lot-tracking-tests/lotter.lotter rename to examples/lots/pta-lot-tracking-tests/lotter.lotter diff --git a/examples/investing/pta-lot-tracking-tests/subaccounts.beancount b/examples/lots/pta-lot-tracking-tests/subaccounts.beancount similarity index 100% rename from examples/investing/pta-lot-tracking-tests/subaccounts.beancount rename to examples/lots/pta-lot-tracking-tests/subaccounts.beancount diff --git a/examples/investing/pta-lot-tracking-tests/subaccounts.hledger b/examples/lots/pta-lot-tracking-tests/subaccounts.hledger similarity index 100% rename from examples/investing/pta-lot-tracking-tests/subaccounts.hledger rename to examples/lots/pta-lot-tracking-tests/subaccounts.hledger diff --git a/hledger-lib/.date.m4 b/hledger-lib/.date.m4 index 173ae01a69b..e510e9c9f95 100644 --- a/hledger-lib/.date.m4 +++ b/hledger-lib/.date.m4 @@ -1,2 +1,2 @@ m4_dnl Date to show in man pages. Updated by "Shake manuals" -m4_define({{_monthyear_}}, {{December 2025}})m4_dnl +m4_define({{_monthyear_}}, {{February 2026}})m4_dnl diff --git a/hledger-lib/.version b/hledger-lib/.version index 0c353043ed6..3e9926483a2 100644 --- a/hledger-lib/.version +++ b/hledger-lib/.version @@ -1 +1 @@ -1.52.99 +1.99 diff --git a/hledger-lib/.version.m4 b/hledger-lib/.version.m4 index de014a84661..4012f693a80 100644 --- a/hledger-lib/.version.m4 +++ b/hledger-lib/.version.m4 @@ -1,2 +1,2 @@ m4_dnl Version number to show in manuals. Updated by "Shake setversion" -m4_define({{_version_}}, {{1.52.99}})m4_dnl +m4_define({{_version_}}, {{1.99}})m4_dnl diff --git a/hledger-lib/Hledger/Data.hs b/hledger-lib/Hledger/Data.hs index 3cada9f9275..c849186cdcc 100644 --- a/hledger-lib/Hledger/Data.hs +++ b/hledger-lib/Hledger/Data.hs @@ -11,6 +11,7 @@ functionality. This package re-exports all the Hledger.Data.* modules module Hledger.Data ( module Hledger.Data.Account, module Hledger.Data.AccountName, + module Hledger.Data.AccountType, module Hledger.Data.Amount, module Hledger.Data.BalanceData, module Hledger.Data.Balancing, @@ -22,6 +23,7 @@ module Hledger.Data ( module Hledger.Data.JournalChecks, module Hledger.Data.Json, module Hledger.Data.Ledger, + module Hledger.Data.Lots, module Hledger.Data.Period, module Hledger.Data.PeriodData, module Hledger.Data.PeriodicTransaction, @@ -41,6 +43,7 @@ import Test.Tasty (testGroup) import Hledger.Data.Account import Hledger.Data.BalanceData import Hledger.Data.AccountName +import Hledger.Data.AccountType import Hledger.Data.Amount import Hledger.Data.Balancing import Hledger.Data.Currency @@ -51,6 +54,7 @@ import Hledger.Data.Journal import Hledger.Data.JournalChecks import Hledger.Data.Json import Hledger.Data.Ledger +import Hledger.Data.Lots import Hledger.Data.Period import Hledger.Data.PeriodData import Hledger.Data.PeriodicTransaction diff --git a/hledger-lib/Hledger/Data/AccountName.hs b/hledger-lib/Hledger/Data/AccountName.hs index 237c86e82a5..48acaac1808 100644 --- a/hledger-lib/Hledger/Data/AccountName.hs +++ b/hledger-lib/Hledger/Data/AccountName.hs @@ -172,23 +172,23 @@ accountNameLevel a = T.length (T.filter (==acctsepchar) a) + 1 unbudgetedAccountName :: T.Text unbudgetedAccountName = "" -accountNamePostingType :: AccountName -> PostingType +accountNamePostingType :: AccountName -> PostingRealness accountNamePostingType a - | T.null a = RegularPosting + | T.null a = RealPosting | T.head a == '[' && T.last a == ']' = BalancedVirtualPosting | T.head a == '(' && T.last a == ')' = VirtualPosting - | otherwise = RegularPosting + | otherwise = RealPosting accountNameWithoutPostingType :: AccountName -> AccountName accountNameWithoutPostingType a = case accountNamePostingType a of BalancedVirtualPosting -> textUnbracket a VirtualPosting -> textUnbracket a - RegularPosting -> a + RealPosting -> a -accountNameWithPostingType :: PostingType -> AccountName -> AccountName +accountNameWithPostingType :: PostingRealness -> AccountName -> AccountName accountNameWithPostingType BalancedVirtualPosting = wrap "[" "]" . accountNameWithoutPostingType accountNameWithPostingType VirtualPosting = wrap "(" ")" . accountNameWithoutPostingType -accountNameWithPostingType RegularPosting = accountNameWithoutPostingType +accountNameWithPostingType RealPosting = accountNameWithoutPostingType -- | Prefix one account name to another, preserving posting type -- indicators like concatAccountNames. @@ -200,7 +200,7 @@ joinAccountNames a b = concatAccountNames $ filter (not . T.null) [a,b] -- the resulting account name. concatAccountNames :: [AccountName] -> AccountName concatAccountNames as = accountNameWithPostingType t $ T.intercalate ":" $ map accountNameWithoutPostingType as - where t = headDef RegularPosting $ filter (/= RegularPosting) $ map accountNamePostingType as + where t = headDef RealPosting $ filter (/= RealPosting) $ map accountNamePostingType as -- | Rewrite an account name using all matching aliases from the given list, in sequence. -- Each alias sees the result of applying the previous aliases. diff --git a/hledger-lib/Hledger/Data/AccountType.hs b/hledger-lib/Hledger/Data/AccountType.hs new file mode 100644 index 00000000000..e047a8c569d --- /dev/null +++ b/hledger-lib/Hledger/Data/AccountType.hs @@ -0,0 +1,54 @@ +{-| + +Helpers for working with 'AccountType's. +Subtypes (Cash, Conversion, Gain) are recognised as their parent type +where appropriate. + +-} + +module Hledger.Data.AccountType ( + isAccountSubtypeOf, + isAssetType, + isLiabilityType, + isEquityType, + isRevenueType, + isExpenseType, +) where + +import Hledger.Data.Types (AccountType(..)) + +-- | Check whether the first argument is a subtype of the second: either equal +-- or one of the defined subtypes. +isAccountSubtypeOf :: AccountType -> AccountType -> Bool +isAccountSubtypeOf Asset Asset = True +isAccountSubtypeOf Liability Liability = True +isAccountSubtypeOf Equity Equity = True +isAccountSubtypeOf Revenue Revenue = True +isAccountSubtypeOf Expense Expense = True +isAccountSubtypeOf Cash Cash = True +isAccountSubtypeOf Cash Asset = True +isAccountSubtypeOf Conversion Conversion = True +isAccountSubtypeOf Conversion Equity = True +isAccountSubtypeOf Gain Gain = True +isAccountSubtypeOf Gain Revenue = True +isAccountSubtypeOf _ _ = False + +-- | Is this an Asset or Cash (subtype of Asset) account type ? +isAssetType :: AccountType -> Bool +isAssetType = (`isAccountSubtypeOf` Asset) + +-- | Is this a Liability account type ? +isLiabilityType :: AccountType -> Bool +isLiabilityType = (`isAccountSubtypeOf` Liability) + +-- | Is this an Equity or Conversion (subtype of Equity) account type ? +isEquityType :: AccountType -> Bool +isEquityType = (`isAccountSubtypeOf` Equity) + +-- | Is this a Revenue or Gain (subtype of Revenue) account type ? +isRevenueType :: AccountType -> Bool +isRevenueType = (`isAccountSubtypeOf` Revenue) + +-- | Is this an Expense account type ? +isExpenseType :: AccountType -> Bool +isExpenseType = (`isAccountSubtypeOf` Expense) diff --git a/hledger-lib/Hledger/Data/Amount.hs b/hledger-lib/Hledger/Data/Amount.hs index f767ffabb55..8b614e0d299 100644 --- a/hledger-lib/Hledger/Data/Amount.hs +++ b/hledger-lib/Hledger/Data/Amount.hs @@ -66,6 +66,7 @@ module Hledger.Data.Amount ( amountCost, amountIsZero, amountLooksZero, + amountSetQuantity, divideAmount, multiplyAmount, invertAmount, @@ -96,6 +97,8 @@ module Hledger.Data.Amount ( showAmountCostB, showAmountCostBasis, showAmountCostBasisB, + showAmountCostBasisLedger, + showAmountCostBasisLedgerB, cshowAmount, showAmountWithZeroCommodity, showAmountDebug, @@ -191,7 +194,7 @@ import Data.List (foldl') import Data.List.NonEmpty (NonEmpty(..), nonEmpty) import Data.Map.Strict qualified as M import Data.Set qualified as S -import Data.Maybe (fromMaybe, isNothing) +import Data.Maybe (catMaybes, fromMaybe, isNothing) import Data.Semigroup (Semigroup(..)) import Data.Text qualified as T import Data.Text.Lazy.Builder qualified as TB @@ -252,6 +255,7 @@ data AmountFormat = AmountFormat , displayCostBasis :: Bool -- ^ Whether to display Amounts' cost basis (Ledger-style lot syntax). , displayColour :: Bool -- ^ Whether to ansi-colourise negative Amounts. , displayQuotes :: Bool -- ^ Whether to enclose complex symbols in quotes (normally true) + , displayLedgerLotSyntax :: Bool -- ^ Whether to display cost basis using Ledger-style lot syntax ({COST} [DATE] (LABEL)) instead of hledger consolidated syntax. } deriving (Show) -- | By default, display amounts using @defaultFmt@ amount display options. @@ -272,6 +276,7 @@ defaultFmt = AmountFormat { , displayCostBasis = True , displayColour = False , displayQuotes = True + , displayLedgerLotSyntax = False } -- | Like defaultFmt but show zero amounts with commodity symbol and styling, like non-zero amounts. @@ -384,6 +389,13 @@ divideAmount n = transformAmount (/n) multiplyAmount :: Quantity -> Amount -> Amount multiplyAmount n = transformAmount (*n) +-- | Replace an amount's quantity, resetting display precision to NaturalPrecision. +-- This is the safe way to set a new quantity that may have different decimal places +-- than the original — NaturalPrecision ensures the exact value is always displayed. +-- Commodity styles will override the precision at rendering time. +amountSetQuantity :: Quantity -> Amount -> Amount +amountSetQuantity q a = a{aquantity=q, astyle=(astyle a){asprecision=NaturalPrecision}} + -- | Invert an amount (replace its quantity q with 1/q). -- The amount's transacted price, if any, is not changed. -- An amount with zero quantity is left unchanged. @@ -725,8 +737,8 @@ showAmountB ,displayForceDecimalMark, displayCost, displayCostBasis, displayColour, displayQuotes} a@Amount{astyle=style} = color $ case ascommodityside style of - L -> (if displayCommodity then wbFromText comm <> space else mempty) <> quantity' <> cost <> costbasis - R -> quantity' <> (if displayCommodity then space <> wbFromText comm else mempty) <> cost <> costbasis + L -> (if displayCommodity then wbFromText comm <> space else mempty) <> quantity' <> costbasis <> cost + R -> quantity' <> (if displayCommodity then space <> wbFromText comm else mempty) <> costbasis <> cost where color = if displayColour && isNegativeAmount a then colorB Dull Red else id quantity = showAmountQuantity displayForceDecimalMark $ @@ -736,7 +748,9 @@ showAmountB | otherwise = (quantity, (if displayQuotes then quoteCommoditySymbolIfNeeded else id) $ acommodity a) space = if not (T.null comm) && ascommodityspaced style then WideBuilder (TB.singleton ' ') 1 else mempty cost = if displayCost then showAmountCostB afmt a else mempty - costbasis = if displayCostBasis then showAmountCostBasisB afmt a else mempty + costbasis = if displayCostBasis then + (if displayLedgerLotSyntax afmt then showAmountCostBasisLedgerB else showAmountCostBasisB) afmt a + else mempty -- Show an amount's cost as @ UNITCOST or @@ TOTALCOST, plus a leading space, or "" if there's no cost. showAmountCost :: Amount -> String @@ -755,7 +769,7 @@ showAmountCostDebug Nothing = "" showAmountCostDebug (Just (UnitCost pa)) = "@ " ++ showAmountDebug pa showAmountCostDebug (Just (TotalCost pa)) = "@@ " ++ showAmountDebug pa --- | Show an amount's cost basis as Ledger-style lot syntax: {LOTCOST} [LOTDATE] (LOTNOTE). +-- | Show an amount's cost basis as consolidated lot syntax: {DATE, "LABEL", COST}. showAmountCostBasis :: Amount -> String showAmountCostBasis = wbUnpack . showAmountCostBasisB defaultFmt @@ -763,8 +777,34 @@ showAmountCostBasis = wbUnpack . showAmountCostBasisB defaultFmt showAmountCostBasisB :: AmountFormat -> Amount -> WideBuilder showAmountCostBasisB afmt amt = case acostbasis amt of Nothing -> mempty + Just CostBasis{cbCost=Nothing, cbDate=Nothing, cbLabel=Nothing} -> + WideBuilder (TB.fromString " {}") 3 Just CostBasis{cbCost, cbDate, cbLabel} -> - lotcost <> lotdate <> lotnote + case parts of + [] -> mempty + _ -> WideBuilder (TB.fromString " {") 2 <> contents <> WideBuilder (TB.singleton '}') 1 + where + parts = catMaybes + [ fmap (wbFromText . T.pack . show) cbDate + , fmap (\l -> wbFromText ("\"" <> l <> "\"")) cbLabel + , fmap (showAmountB afmt) cbCost + ] + separator = WideBuilder (TB.fromString ", ") 2 + contents = mconcat $ intersperse separator parts + +-- | Show an amount's cost basis as Ledger-style lot syntax: {LOTCOST} [LOTDATE] (LOTNOTE). +-- Kept for future --ledger-lot-syntax flag (step 3). +showAmountCostBasisLedger :: Amount -> String +showAmountCostBasisLedger = wbUnpack . showAmountCostBasisLedgerB defaultFmt + +-- showAmountCostBasisLedger, efficient builder version. +showAmountCostBasisLedgerB :: AmountFormat -> Amount -> WideBuilder +showAmountCostBasisLedgerB afmt amt = case acostbasis amt of + Nothing -> mempty + Just CostBasis{cbCost=Nothing, cbDate=Nothing, cbLabel=Nothing} -> + WideBuilder (TB.fromString " {}") 3 + Just CostBasis{cbCost, cbDate, cbLabel} -> + lotdate <> lotnote <> lotcost where lotcost = case cbCost of Nothing -> mempty @@ -864,13 +904,6 @@ instance Num MixedAmount where abs = mapMixedAmount (\amt -> amt { aquantity = abs (aquantity amt)}) signum = error' "error, mixed amounts do not support signum" --- | Calculate the key used to store an Amount within a MixedAmount. -amountKey :: Amount -> MixedAmountKey -amountKey amt@Amount{acommodity=c} = case acost amt of - Nothing -> MixedAmountKeyNoCost c - Just (TotalCost p) -> MixedAmountKeyTotalCost c (acommodity p) - Just (UnitCost p) -> MixedAmountKeyUnitCost c (acommodity p) (aquantity p) - -- | The empty mixed amount. nullmixedamt :: MixedAmount nullmixedamt = Mixed mempty @@ -885,7 +918,7 @@ missingmixedamt = mixedAmount missingamt -- instead it looks for missingamt among the Amounts. -- missingamt should always be alone, but detect it even if not. isMissingMixedAmount :: MixedAmount -> Bool -isMissingMixedAmount (Mixed ma) = amountKey missingamt `M.member` ma +isMissingMixedAmount (Mixed ma) = mixedAmountKey missingamt `M.member` ma -- | Convert amounts in various commodities into a mixed amount. mixed :: Foldable t => t Amount -> MixedAmount @@ -893,37 +926,34 @@ mixed = maAddAmounts nullmixedamt -- | Create a MixedAmount from a single Amount. mixedAmount :: Amount -> MixedAmount -mixedAmount a = Mixed $ M.singleton (amountKey a) a - --- | Add an Amount to a MixedAmount, normalising the result. --- Amounts with different costs are kept separate. -maAddAmount :: MixedAmount -> Amount -> MixedAmount -maAddAmount (Mixed ma) a = Mixed $ M.insertWith sumSimilarAmountsUsingFirstCost (amountKey a) a ma - --- | Add a collection of Amounts to a MixedAmount, normalising the result. --- Amounts with different costs are kept separate. -maAddAmounts :: Foldable t => MixedAmount -> t Amount -> MixedAmount -maAddAmounts = foldl' maAddAmount +mixedAmount a = Mixed $ M.singleton (mixedAmountKey a) a -- | Negate mixed amount's quantities (and total costs, if any). maNegate :: MixedAmount -> MixedAmount maNegate = transformMixedAmount negate --- | Sum two MixedAmount, keeping the cost of the first if any. --- Amounts with different costs are kept separate (since 2021). +-- | Sum two MixedAmounts. (Any cost basis on the amounts will be lost.) maPlus :: MixedAmount -> MixedAmount -> MixedAmount -maPlus (Mixed as) (Mixed bs) = Mixed $ M.unionWith sumSimilarAmountsUsingFirstCost as bs +maPlus (Mixed as) (Mixed bs) = Mixed $ M.unionWith sumSimilarAmounts as bs --- | Subtract a MixedAmount from another. --- Amounts with different costs are kept separate. +-- | Subtract a MixedAmount from another. (Any cost basis on the amounts will be lost.) maMinus :: MixedAmount -> MixedAmount -> MixedAmount maMinus a = maPlus a . maNegate --- | Sum a collection of MixedAmounts. --- Amounts with different costs are kept separate. -maSum :: Foldable t => t MixedAmount -> MixedAmount +-- | Sum a collection of MixedAmounts. (Any cost basis on the amounts will be lost.) +maSum :: (Foldable t) => t MixedAmount -> MixedAmount maSum = foldl' maPlus nullmixedamt +-- | Add an Amount to a MixedAmount, and then normalise that. +-- (Any cost basis on the amounts will be lost.) +maAddAmount :: MixedAmount -> Amount -> MixedAmount +maAddAmount (Mixed ma) a = Mixed $ M.insertWith sumSimilarAmounts (mixedAmountKey a) a ma + +-- | Add a collection of Amounts to a MixedAmount, and then normalise that. +-- (Any cost basis on the amounts will be lost.) +maAddAmounts :: (Foldable t) => MixedAmount -> t Amount -> MixedAmount +maAddAmounts = foldl' maAddAmount + -- | Divide a mixed amount's quantities (and total costs, if any) by a constant. divideMixedAmount :: Quantity -> MixedAmount -> MixedAmount divideMixedAmount n = transformMixedAmount (/n) @@ -1028,8 +1058,7 @@ amountsPreservingZeros (Mixed ma) -- | Get a mixed amount's component amounts without normalising zero and missing -- amounts. This is used for JSON serialisation, so the order is important. In -- particular, we want the Amounts given in the order of the MixedAmountKeys, --- i.e. lexicographically first by commodity, then by cost commodity, then by --- unit cost from most negative to most positive. +-- i.e. sorted by commodity and transacted cost. amountsRaw :: MixedAmount -> [Amount] amountsRaw (Mixed ma) = toList ma @@ -1052,11 +1081,12 @@ unifyMixedAmount = foldM combine 0 . amounts | acommodity amt == acommodity result = Just $ amt + result | otherwise = Nothing --- | Sum same-commodity amounts in a lossy way, applying the first --- cost to the result and discarding any other costs. Only used as a --- rendering helper. -sumSimilarAmountsUsingFirstCost :: Amount -> Amount -> Amount -sumSimilarAmountsUsingFirstCost a b = (a + b){acost=p} +-- | Sum amounts which have the same MixedAmountKey; ie they have the same commodity and the same transacted cost if any. +-- If they have total transacted costs, those are also summed. +-- If they have a unit cost, that is preserved. +-- If they have a lot cost basis, that is removed. +sumSimilarAmounts :: Amount -> Amount -> Amount +sumSimilarAmounts a b = (a + b){acost=p, acostbasis=Nothing} where p = case (acost a, acost b) of (Just (TotalCost ap), Just (TotalCost bp)) @@ -1067,17 +1097,8 @@ sumSimilarAmountsUsingFirstCost a b = (a + b){acost=p} filterMixedAmount :: (Amount -> Bool) -> MixedAmount -> MixedAmount filterMixedAmount p (Mixed ma) = Mixed $ M.filter p ma --- | Return an unnormalised MixedAmount containing just the amounts in the --- requested commodity from the original mixed amount. --- --- The result will contain at least one Amount of the requested commodity, --- even if the original mixed amount did not (with quantity zero in that case, --- and this would be discarded when the mixed amount is next normalised). --- --- The result can contain more than one Amount of the requested commodity, --- eg because there were several with different costs, --- or simply because the original mixed amount was was unnormalised. --- +-- | Return a MixedAmount containing just the amount of the requested commodity +-- that was in the original mixed amount (or zero if there was none). filterMixedAmountByCommodity :: CommoditySymbol -> MixedAmount -> MixedAmount filterMixedAmountByCommodity c (Mixed ma) | M.null ma' = mixedAmount nullamt{acommodity=c} @@ -1089,8 +1110,8 @@ mapMixedAmount :: (Amount -> Amount) -> MixedAmount -> MixedAmount mapMixedAmount f (Mixed ma) = mixed . map f $ toList ma -- | Apply a transform to a mixed amount's component 'Amount's, which does not --- affect the key of the amount (i.e. doesn't change the commodity, cost --- commodity, or unit cost amount). This condition is not checked. +-- affect the key of the amount (i.e. doesn't change the commodity or transacted cost). +-- This condition is not checked. mapMixedAmountUnsafe :: (Amount -> Amount) -> MixedAmount -> MixedAmount mapMixedAmountUnsafe f (Mixed ma) = Mixed $ M.map f ma -- Use M.map instead of fmap to maintain strictness @@ -1371,11 +1392,11 @@ mixedAmountSetPrecisionMin p = mapMixedAmountUnsafe (amountSetPrecisionMin p) mixedAmountSetPrecisionMax :: Word8 -> MixedAmount -> MixedAmount mixedAmountSetPrecisionMax p = mapMixedAmountUnsafe (amountSetPrecisionMax p) --- | Remove all costs from a MixedAmount. +-- | Remove all transacted costs and cost bases from a MixedAmount. mixedAmountStripCosts :: MixedAmount -> MixedAmount mixedAmountStripCosts (Mixed ma) = - foldl' (\m a -> maAddAmount m a{acost=Nothing}) (Mixed noCosts) withCosts - where (noCosts, withCosts) = M.partition (isNothing . acost) ma + foldl' (\m a -> maAddAmount m a{acost=Nothing, acostbasis=Nothing}) (Mixed noCosts) withCosts + where (noCosts, withCosts) = M.partition (\a -> isNothing (acost a) && isNothing (acostbasis a)) ma ------------------------------------------------------------------------------- diff --git a/hledger-lib/Hledger/Data/Balancing.hs b/hledger-lib/Hledger/Data/Balancing.hs index 85d6c33e35a..73357421ae6 100644 --- a/hledger-lib/Hledger/Data/Balancing.hs +++ b/hledger-lib/Hledger/Data/Balancing.hs @@ -51,7 +51,7 @@ import Safe (headErr) import Text.Printf (printf) import Hledger.Data.Types -import Hledger.Data.AccountName (isAccountNamePrefixOf) +import Hledger.Data.AccountName (accountNameType, isAccountNamePrefixOf) import Hledger.Data.Amount import Hledger.Data.Journal import Hledger.Data.Posting @@ -66,6 +66,7 @@ data BalancingOpts = BalancingOpts -- Distinct from InputOpts{infer_costs_}. , commodity_styles_ :: Maybe (M.Map CommoditySymbol AmountStyle) -- ^ commodity display styles , txn_balancing_ :: TransactionBalancingPrecision + , account_types_ :: M.Map AccountName AccountType -- ^ account type map, used to exclude Gain postings from balancing (with --lots) } deriving (Eq, Ord, Show) defbalancingopts :: BalancingOpts @@ -74,8 +75,24 @@ defbalancingopts = BalancingOpts , infer_balancing_costs_ = True , commodity_styles_ = Nothing , txn_balancing_ = TBPExact + , account_types_ = M.empty } +-- | Is this posting to a Gain-type account in a disposal transaction ? +-- Used to exclude gain/loss postings from normal transaction balancing, +-- since in disposals the gain posting balances at cost basis, not at selling price. +isGainPosting :: M.Map AccountName AccountType -> Transaction -> Posting -> Bool +isGainPosting atypes t p + | M.null atypes = False + | not (isDisposalTransaction atypes t) = False + | otherwise = accountNameType atypes (paccount p) == Just Gain + +-- | Does this transaction contain a disposal posting (one tagged _ptype: dispose) ? +isDisposalTransaction :: M.Map AccountName AccountType -> Transaction -> Bool +isDisposalTransaction atypes t + | M.null atypes = False + | otherwise = any (("_ptype", "dispose") `elem`) $ map ptags $ tpostings t + -- | Check that this transaction would appear balanced to a human when displayed. -- On success, returns the empty list, otherwise one or more error messages. -- @@ -94,13 +111,16 @@ defbalancingopts = BalancingOpts -- (using the given display styles if provided) -- transactionCheckBalanced :: BalancingOpts -> Transaction -> [String] -transactionCheckBalanced BalancingOpts{commodity_styles_=_mglobalstyles, txn_balancing_} t = errs +transactionCheckBalanced BalancingOpts{commodity_styles_=_mglobalstyles, txn_balancing_, account_types_} t = errs where + -- In disposal transactions, gain postings are excluded from normal balancing + isGain = isGainPosting account_types_ t + -- get real and balanced virtual postings, to be checked separately - (rps, bvps) = foldr partitionPosting ([], []) $ tpostings t + (rps, bvps) = foldr partitionPosting ([], []) $ filter (not . isGain) $ tpostings t where - partitionPosting p ~(l, r) = case ptype p of - RegularPosting -> (p:l, r) + partitionPosting p ~(l, r) = case preal p of + RealPosting -> (p:l, r) BalancedVirtualPosting -> (l, p:r) VirtualPosting -> (l, r) @@ -191,9 +211,9 @@ balanceTransactionHelper :: BalancingOpts -> Transaction -> Either String (Trans balanceTransactionHelper bopts t = do let lbl = lbl_ "balanceTransactionHelper" (t', inferredamtsandaccts) <- t - & (if infer_balancing_costs_ bopts then transactionInferBalancingCosts else id) + & (if infer_balancing_costs_ bopts then transactionInferBalancingCosts (account_types_ bopts) else id) & dbg9With (lbl "amounts after balancing-cost-inferring".show.map showMixedAmountOneLine.transactionAmounts) - & transactionInferBalancingAmount (fromMaybe M.empty $ commodity_styles_ bopts) + & transactionInferBalancingAmount (fromMaybe M.empty $ commodity_styles_ bopts) (account_types_ bopts) <&> dbg9With (lbl "balancing amounts inferred".show.map (second showMixedAmountOneLine).snd) case transactionCheckBalanced bopts t' of [] -> Right (txnTieKnot t', inferredamtsandaccts) @@ -242,9 +262,10 @@ transactionBalanceError t errs = printf "%s:\n%s\n\nThis %stransaction is unbala -- have the same price(s), and will be converted to the price commodity. transactionInferBalancingAmount :: M.Map CommoditySymbol AmountStyle -- ^ commodity display styles + -> M.Map AccountName AccountType -- ^ account type map (for excluding Gain postings) -> Transaction -> Either String (Transaction, [(AccountName, MixedAmount)]) -transactionInferBalancingAmount styles t@Transaction{tpostings=ps} +transactionInferBalancingAmount styles atypes t@Transaction{tpostings=ps} | length amountlessrealps > 1 = Left $ transactionBalanceError t ["There can't be more than one real posting with no amount." @@ -263,17 +284,20 @@ transactionInferBalancingAmount styles t@Transaction{tpostings=ps} ) where lbl = lbl_ "transactionInferBalancingAmount" - (amountfulrealps, amountlessrealps) = partition hasAmount (realPostings t) - realsum = sumPostings amountfulrealps + isGain = isGainPosting atypes t + (amountfulrealps, amountlessrealps) = partition hasAmount (filter (not . isGain) $ realPostings t) + realsum = maSum $ map (mixedAmountCost . pamount) amountfulrealps -- & dbg9With (lbl "real balancing amount".showMixedAmountOneLine) - (amountfulbvps, amountlessbvps) = partition hasAmount (balancedVirtualPostings t) - bvsum = sumPostings amountfulbvps + (amountfulbvps, amountlessbvps) = partition hasAmount (filter (not . isGain) $ balancedVirtualPostings t) + bvsum = maSum $ map (mixedAmountCost . pamount) amountfulbvps inferamount :: Posting -> (Posting, Maybe MixedAmount) - inferamount p = + inferamount p + | isGain p = (p, Nothing) -- gain postings are excluded from balancing + | otherwise = let - minferredamt = case ptype p of - RegularPosting | not (hasAmount p) -> Just realsum + minferredamt = case preal p of + RealPosting | not (hasAmount p) -> Just realsum BalancedVirtualPosting | not (hasAmount p) -> Just bvsum VirtualPosting | not (hasAmount p) -> Just 0 _ -> Nothing @@ -335,20 +359,21 @@ transactionInferBalancingAmount styles t@Transaction{tpostings=ps} -- use any decimal places. The minimum of 2 helps make the costs shown by the -- print command a bit less surprising in this case. Could do better.) -- -transactionInferBalancingCosts :: Transaction -> Transaction -transactionInferBalancingCosts t@Transaction{tpostings=ps} = t{tpostings=ps'} +transactionInferBalancingCosts :: M.Map AccountName AccountType -> Transaction -> Transaction +transactionInferBalancingCosts atypes t@Transaction{tpostings=ps} = t{tpostings=ps'} where - ps' = map (costInferrerFor t BalancedVirtualPosting . costInferrerFor t RegularPosting) ps + ps' = map (costInferrerFor atypes t BalancedVirtualPosting . costInferrerFor atypes t RealPosting) ps -- | Generate a posting update function which assigns a suitable cost to -- balance the posting, if and as appropriate for the given transaction and --- posting type (real or balanced virtual) (or if we cannot or should not infer +-- posting realness (real or balanced virtual) (or if we cannot or should not infer -- costs, leaves the posting unchanged). -costInferrerFor :: Transaction -> PostingType -> (Posting -> Posting) -costInferrerFor t pt = maybe id infercost inferFromAndTo +costInferrerFor :: M.Map AccountName AccountType -> Transaction -> PostingRealness -> (Posting -> Posting) +costInferrerFor atypes t pt = maybe id infercost inferFromAndTo where lbl = lbl_ "costInferrerFor" - postings = filter ((==pt).ptype) $ tpostings t + isGain = isGainPosting atypes t + postings = filter (\p -> preal p == pt && not (isGain p)) $ tpostings t pcommodities = map acommodity $ concatMap (amounts . pamount) postings sumamounts = amounts $ sumPostings postings -- amounts normalises to one amount per commodity & price @@ -370,7 +395,7 @@ costInferrerFor t pt = maybe id infercost inferFromAndTo -- and the commodity of the amount matches the amount we're converting from, -- then set its cost based on the ratio between fromamount and toamount. infercost (fromamount, toamount) p - | [a] <- amounts (pamount p), ptype p == pt, acommodity a == acommodity fromamount + | [a] <- amounts (pamount p), preal p == pt, acommodity a == acommodity fromamount = p{ pamount = mixedAmount a{acost=Just conversionprice} & dbg9With (lbl "inferred cost".showMixedAmountOneLine) , poriginal = Just $ originalPosting p } @@ -425,6 +450,7 @@ data BalancingState s = BalancingState { bsStyles :: Maybe (M.Map CommoditySymbol AmountStyle) -- ^ commodity display styles ,bsUnassignable :: S.Set AccountName -- ^ accounts where balance assignments may not be used (because of auto posting rules) ,bsAssrt :: Bool -- ^ whether to check balance assertions + ,bsAccountTypes :: M.Map AccountName AccountType -- ^ account type map (for excluding Gain postings from balancing) -- mutable ,bsBalances :: H.HashTable s AccountName MixedAmount -- ^ running account balances, initially empty ,bsTransactions :: STArray s Integer Transaction -- ^ a mutable array of the transactions being balanced @@ -531,7 +557,7 @@ journalBalanceTransactions bopts' j' = -- 2. Step through these items in date order (and preserved same-day order), -- keeping running balances for all accounts. runningbals <- lift $ H.newSized (length $ journalAccountNamesUsed j) - flip runReaderT (BalancingState styles autopostingaccts (not $ ignore_assertions_ bopts) runningbals balancedtxns) $ do + flip runReaderT (BalancingState styles autopostingaccts (not $ ignore_assertions_ bopts) (account_types_ bopts) runningbals balancedtxns) $ do -- On encountering any not-yet-balanced transaction with a balance assignment, -- enact the balance assignment then finish balancing the transaction. -- And, check any balance assertions encountered along the way. @@ -572,7 +598,8 @@ balanceTransactionAndCheckAssertionsB (Right t@Transaction{tpostings=ps}) = do -- infer any remaining missing amounts, and make sure the transaction is now fully balanced styles <- R.reader bsStyles - case balanceTransactionHelper defbalancingopts{commodity_styles_=styles} t{tpostings=ps'} of + atypes <- R.reader bsAccountTypes + case balanceTransactionHelper defbalancingopts{commodity_styles_=styles, account_types_=atypes} t{tpostings=ps'} of Left err -> throwError err Right (t', inferredacctsandamts) -> do -- for each amount just inferred, update the running balance @@ -793,10 +820,10 @@ tests_Balancing = testGroup "Balancing" [ testCase "transactionInferBalancingAmount" $ do - (fst <$> transactionInferBalancingAmount M.empty nulltransaction) @?= Right nulltransaction - (fst <$> transactionInferBalancingAmount M.empty nulltransaction{tpostings = ["a" `post` usd (-5), "b" `post` missingamt]}) @?= + (fst <$> transactionInferBalancingAmount M.empty M.empty nulltransaction) @?= Right nulltransaction + (fst <$> transactionInferBalancingAmount M.empty M.empty nulltransaction{tpostings = ["a" `post` usd (-5), "b" `post` missingamt]}) @?= Right nulltransaction{tpostings = ["a" `post` usd (-5), "b" `post` usd 5]} - (fst <$> transactionInferBalancingAmount M.empty nulltransaction{tpostings = ["a" `post` usd (-5), "b" `post` (eur 3 @@ usd 4), "c" `post` missingamt]}) @?= + (fst <$> transactionInferBalancingAmount M.empty M.empty nulltransaction{tpostings = ["a" `post` usd (-5), "b" `post` (eur 3 @@ usd 4), "c" `post` missingamt]}) @?= Right nulltransaction{tpostings = ["a" `post` usd (-5), "b" `post` (eur 3 @@ usd 4), "c" `post` usd 1]} , testGroup "balanceSingleTransaction" [ @@ -984,7 +1011,7 @@ tests_Balancing = [] [ posting {paccount = "b", pamount = mixedAmount (usd 1.00)} , posting {paccount = "c", pamount = mixedAmount (usd (-1.00))} - , posting {paccount = "d", pamount = mixedAmount (usd 100), ptype = VirtualPosting} + , posting {paccount = "d", pamount = mixedAmount (usd 100), preal = VirtualPosting} ] ,testCase "balanced virtual postings need to balance among themselves" $ assertBool "" $ @@ -1003,7 +1030,7 @@ tests_Balancing = [] [ posting {paccount = "b", pamount = mixedAmount (usd 1.00)} , posting {paccount = "c", pamount = mixedAmount (usd (-1.00))} - , posting {paccount = "d", pamount = mixedAmount (usd 100), ptype = BalancedVirtualPosting} + , posting {paccount = "d", pamount = mixedAmount (usd 100), preal = BalancedVirtualPosting} ] ,testCase "balanced virtual postings need to balance among themselves (2)" $ assertBool "" $ @@ -1021,8 +1048,8 @@ tests_Balancing = [] [ posting {paccount = "b", pamount = mixedAmount (usd 1.00)} , posting {paccount = "c", pamount = mixedAmount (usd (-1.00))} - , posting {paccount = "d", pamount = mixedAmount (usd 100), ptype = BalancedVirtualPosting} - , posting {paccount = "3", pamount = mixedAmount (usd (-100)), ptype = BalancedVirtualPosting} + , posting {paccount = "d", pamount = mixedAmount (usd 100), preal = BalancedVirtualPosting} + , posting {paccount = "3", pamount = mixedAmount (usd (-100)), preal = BalancedVirtualPosting} ] ] diff --git a/hledger-lib/Hledger/Data/Errors.hs b/hledger-lib/Hledger/Data/Errors.hs index 47f1fff1a11..a764295e671 100644 --- a/hledger-lib/Hledger/Data/Errors.hs +++ b/hledger-lib/Hledger/Data/Errors.hs @@ -7,9 +7,11 @@ Helpers for making error messages. module Hledger.Data.Errors ( makeAccountTagErrorExcerpt, + makeCommodityTagErrorExcerpt, makePriceDirectiveErrorExcerpt, makeTransactionErrorExcerpt, makePostingErrorExcerpt, + makePostingErrorExcerptByIndex, makePostingAccountErrorExcerpt, makeBalanceAssertionErrorExcerpt, transactionFindPostingIndex, @@ -59,6 +61,23 @@ showAccountDirective (a, AccountDeclarationInfo{..}) = "account " <> a <> (if not $ T.null adicomment then " ; " <> adicomment else "") +-- | Given a commodity and a problem tag within it: +-- render it as a megaparsec-style excerpt, showing the original line number. +-- Returns the file path, line number, column(s) if known, and the rendered excerpt. +makeCommodityTagErrorExcerpt :: Commodity -> TagName -> (FilePath, Int, Maybe (Int, Maybe Int), Text) +makeCommodityTagErrorExcerpt comm _t = (f, l, merrcols, ex) + where + SourcePos f pos _ = csourcepos comm + l = unPos pos + txt = showCommodityDirective comm & textChomp & (<>"\n") + ex = decorateExcerpt l merrcols txt + merrcols = Nothing + +showCommodityDirective :: Commodity -> Text +showCommodityDirective Commodity{..} = + "commodity " <> csymbol + <> (if not $ T.null ccomment then " ; " <> ccomment else "") + -- | Decorate a data excerpt with megaparsec-style left margin, line number, -- and marker/underline for the column(s) if known, for inclusion in an error message. decorateExcerpt :: Int -> Maybe (Int, Maybe Int) -> Text -> Text @@ -181,6 +200,24 @@ decoratePostingErrorExcerpt absline relline mcols txt = lineprefix = T.replicate marginw " " <> "| " where marginw = length (show absline) + 1 +-- | Like 'makePostingErrorExcerpt', but identifies the posting by its +-- 0-based index in the transaction rather than by equality search. +-- This avoids false mismatches when postings have been modified after parsing +-- (e.g. by the balancer), and is unambiguous when duplicate postings exist. +makePostingErrorExcerptByIndex :: Transaction -> Int -> (FilePath, Int, Maybe (Int, Maybe Int), Text) +makePostingErrorExcerptByIndex t idx = (f, errabsline, Nothing, ex) + where + (SourcePos f tl _) = fst $ tsourcepos t + errrelline = + commentExtraLines (tcomment t) + + sum (map postingLines $ take (idx + 1) $ tpostings t) + where + postingLines p' = 1 + commentExtraLines (pcomment p') + commentExtraLines c = max 0 (length (T.lines c) - 1) + errabsline = unPos tl + errrelline + txntxt = showTransaction t & textChomp & (<>"\n") + ex = decoratePostingErrorExcerpt errabsline errrelline Nothing txntxt + -- | Find the 1-based index of the first posting in this transaction -- satisfying the given predicate. transactionFindPostingIndex :: (Posting -> Bool) -> Transaction -> Maybe Int diff --git a/hledger-lib/Hledger/Data/Journal.hs b/hledger-lib/Hledger/Data/Journal.hs index 1c0f5b45fb3..5af87eb33a5 100644 --- a/hledger-lib/Hledger/Data/Journal.hs +++ b/hledger-lib/Hledger/Data/Journal.hs @@ -38,6 +38,13 @@ module Hledger.Data.Journal ( journalPostingsAddAccountTags, journalPostingsKeepAccountTagsOnly, journalPostingsAddCommodityTags, + journalInferPostingsCostBasis, + journalInferPostingsTransactedCost, + journalCommodityUsesLots, + journalCommodityLotsMethod, + postingLotsMethod, + parseReductionMethod, + journalCheckLotsTagValues, -- * Filtering filterJournalTransactions, filterJournalPostings, @@ -110,6 +117,7 @@ module Hledger.Data.Journal ( journalConcat, journalNumberTransactions, journalNumberAndTieTransactions, + journalTieTransactions, journalUntieTransactions, journalModifyTransactions, journalApplyAliases, @@ -134,7 +142,7 @@ import Data.List (foldl') #endif import Data.List.Extra (nubSort) import Data.Map.Strict qualified as M -import Data.Maybe (catMaybes, fromMaybe, mapMaybe, maybeToList) +import Data.Maybe (catMaybes, fromMaybe, isJust, isNothing, mapMaybe, maybeToList) import Data.Set qualified as S import Data.Text (Text) import Data.Text qualified as T @@ -149,6 +157,7 @@ import Hledger.Utils import Hledger.Data.Types import Hledger.Data.AccountName import Hledger.Data.Amount +import Hledger.Data.Errors (makeAccountTagErrorExcerpt, makeCommodityTagErrorExcerpt) import Hledger.Data.Posting import Hledger.Data.Transaction import Hledger.Data.TransactionModifier @@ -644,29 +653,153 @@ journalDeclaredAccountTypes Journal{jdeclaredaccounttypes} = -- | To all postings in the journal, add any tags from their account -- (including those inherited from parent accounts). +-- Tags are added to ptags (making them queryable) but not to pcomment (so they don't appear in print output). -- If a tag already exists on the posting, it is not changed (the account tag will be ignored). journalPostingsAddAccountTags :: Journal -> Journal journalPostingsAddAccountTags j = journalMapPostings addtags j where addtags p = p `postingAddTags` (journalInheritedAccountTags j $ paccount p) +-- | Remove all tags from the journal's postings except those provided by their account. +-- This is useful for the accounts report. +-- It does not remove tag declarations from the posting comments. +journalPostingsKeepAccountTagsOnly :: Journal -> Journal +journalPostingsKeepAccountTagsOnly j = journalMapPostings keepaccounttags j + where keepaccounttags p = p{ptags=[]} `postingAddTags` (journalInheritedAccountTags j $ paccount p) + -- | Get any tags declared for this commodity. journalCommodityTags :: Journal -> CommoditySymbol -> [Tag] journalCommodityTags Journal{jdeclaredcommoditytags} c = M.findWithDefault [] c jdeclaredcommoditytags +-- | Does this commodity have a 'lots:' tag declared ? +journalCommodityUsesLots :: Journal -> CommoditySymbol -> Bool +journalCommodityUsesLots j c = any ((== "lots") . T.toLower . fst) (journalCommodityTags j c) + +-- | Does this posting have a 'lots:' tag (eg inherited from its account) ? +postingUsesLots :: Posting -> Bool +postingUsesLots p = any ((== "lots") . T.toLower . fst) (ptags p) + +-- | Get the reduction method from a commodity's lots: tag value, if any. +journalCommodityLotsMethod :: Journal -> CommoditySymbol -> Maybe ReductionMethod +journalCommodityLotsMethod j c = + case [v | (k, v) <- journalCommodityTags j c, T.toLower k == "lots"] of + (v:_) -> parseReductionMethod v + [] -> Nothing + +-- | Get the reduction method from a posting's lots: tag value (typically inherited from its account), if any. +postingLotsMethod :: Posting -> Maybe ReductionMethod +postingLotsMethod p = + case [v | (k, v) <- ptags p, T.toLower k == "lots"] of + (v:_) -> parseReductionMethod v + [] -> Nothing + +-- | Parse a reduction method name from a lots: tag value. +parseReductionMethod :: Text -> Maybe ReductionMethod +parseReductionMethod t = case T.toUpper (T.strip t) of + "FIFO" -> Just FIFO + "LIFO" -> Just LIFO + "HIFO" -> Just HIFO + "AVERAGE" -> Just AVERAGE + "SPECID" -> Just SPECID + "FIFOALL" -> Just FIFOALL + "LIFOALL" -> Just LIFOALL + "HIFOALL" -> Just HIFOALL + "AVERAGEALL" -> Just AVERAGEALL + _ -> Nothing + +-- | Check that all lots: tag values on commodity and account declarations are recognised. +-- Empty values (bare @lots:@ tag) are valid and default to FIFO. +-- Non-empty values must be one of the known reduction methods. +journalCheckLotsTagValues :: Journal -> Either String Journal +journalCheckLotsTagValues j = do + mapM_ checkCommodity (M.toList $ jdeclaredcommoditytags j) + mapM_ checkAccount (jdeclaredaccounts j) + Right j + where + msg :: String + msg = unlines [ + "%s:%d:" + ,"%s" + ,"unrecognised lots: tag value %s." + ,"Use FIFO, LIFO, HIFO, AVERAGE, SPECID, FIFOALL, LIFOALL, HIFOALL, AVERAGEALL, or nothing (meaning FIFO)" + ] + + checkCommodity (sym, tags) = + case M.lookup sym (jdeclaredcommodities j) of + Just comm -> mapM_ (checkCommodityTag comm) tags + Nothing -> Right () + + checkCommodityTag comm (k, v) + | T.toLower k /= "lots" = Right () + | T.null (T.strip v) = Right () + | Just _ <- parseReductionMethod v = Right () + | otherwise = Left $ printf msg f l ex (show v) + where (f, l, _mcols, ex) = makeCommodityTagErrorExcerpt comm k + + checkAccount (acctName, adi) = + mapM_ (checkAccountTag acctName adi) (aditags adi) + + checkAccountTag acctName adi (k, v) + | T.toLower k /= "lots" = Right () + | T.null (T.strip v) = Right () + | Just _ <- parseReductionMethod v = Right () + | otherwise = Left $ printf msg f l ex (show v) + where (f, l, _mcols, ex) = makeAccountTagErrorExcerpt (acctName, adi) k + -- | To all postings in the journal, add any tags from their amount's commodities. +-- Tags are added to ptags (making them queryable) but not to pcomment (so they don't appear in print output). -- If a tag already exists on the posting, it is not changed (the commodity tag will be ignored). journalPostingsAddCommodityTags :: Journal -> Journal journalPostingsAddCommodityTags j = journalMapPostings addtags j where addtags p = p `postingAddTags` concatMap (journalCommodityTags j) (postingCommodities p) --- | Remove all tags from the journal's postings except those provided by their account. --- This is useful for the accounts report. --- It does not remove tag declarations from the posting comments. -journalPostingsKeepAccountTagsOnly :: Journal -> Journal -journalPostingsKeepAccountTagsOnly j = journalMapPostings keepaccounttags j - where keepaccounttags p = p{ptags=[]} `postingAddTags` (journalInheritedAccountTags j $ paccount p) +-- | For acquire postings (positive amounts) whose commodity or account has a 'lots:' tag, +-- infer cost basis (cost only, no date or label) from transacted cost. +-- The lot date will later default to the transaction date. +-- Must be called before journalClassifyLotPostings. +journalInferPostingsCostBasis :: Journal -> Journal +journalInferPostingsCostBasis j = journalMapPostings (postingInferCostBasis j) j + +postingInferCostBasis :: Journal -> Posting -> Posting +postingInferCostBasis j p = p{pamount = mapMixedAmount amountInferCostBasis $ pamount p} + where + amountInferCostBasis :: Amount -> Amount + amountInferCostBasis a + | aquantity a <= 0 = a -- only positive (acquire) amounts + | isJust (acostbasis a) = a -- already has cost basis + | Nothing <- acost a = a -- no transacted cost + | not (journalCommodityUsesLots j (acommodity a) || postingUsesLots p) = a -- commodity/account not lot-tracked + | Just cost <- acost a = a{acostbasis = Just (costToCostBasis (aquantity a) cost)} + + costToCostBasis :: Quantity -> AmountCost -> CostBasis + costToCostBasis qty cost = CostBasis{cbCost=Just ucost, cbDate=Nothing, cbLabel=Nothing} + where + ucost = case cost of + UnitCost amt -> amt + TotalCost amt | qty /= 0 -> amt{aquantity = aquantity amt / abs qty} + | otherwise -> amt + +-- | For positive postings with a cost basis, which are not lot transfers, +-- infer transacted cost from cost basis. +-- Must be called after journalClassifyLotPostings so ptype tags are available. +journalInferPostingsTransactedCost :: Journal -> Journal +journalInferPostingsTransactedCost = journalMapPostings postingInferTransactedCost + +postingInferTransactedCost :: Posting -> Posting +postingInferTransactedCost p + | ("_ptype", "transfer-to") `elem` ptags p = p -- not for transfer postings + | not (any needsInference $ amounts $ pamount p) = p -- nothing to infer + | otherwise = p'{poriginal = Just $ originalPosting p} + where + p' = p{pamount = mapMixedAmount amountInferTransactedCost $ pamount p} + needsInference a = aquantity a > 0 && isNothing (acost a) && hasCostBasisCost a + amountInferTransactedCost a + | needsInference a, Just CostBasis{cbCost=Just c} <- acostbasis a = a{acost = Just (UnitCost c)} + | otherwise = a + hasCostBasisCost a = case acostbasis a of + Just CostBasis{cbCost=Just _} -> True + _ -> False -- | The account name to use for conversion postings generated by --infer-equity. -- This is the first account declared with type V/Conversion, diff --git a/hledger-lib/Hledger/Data/JournalChecks.hs b/hledger-lib/Hledger/Data/JournalChecks.hs index 16fd9a65ad9..84bb347c62e 100644 --- a/hledger-lib/Hledger/Data/JournalChecks.hs +++ b/hledger-lib/Hledger/Data/JournalChecks.hs @@ -12,6 +12,7 @@ module Hledger.Data.JournalChecks ( journalCheckAccounts, journalCheckBalanceAssertions, journalCheckCommodities, + journalCheckLots, journalCheckPayees, journalCheckPairedConversionPostings, journalCheckRecentAssertions, @@ -35,13 +36,14 @@ import Hledger.Data.JournalChecks.Ordereddates import Hledger.Data.JournalChecks.Uniqueleafnames import Hledger.Data.Posting (isVirtual, postingDate, transactionAllTags, conversionPostingTagName, costPostingTagName, postingAsLines, generatedPostingTagName, generatedTransactionTagName, modifiedTransactionTagName) import Hledger.Data.Types -import Hledger.Data.Amount (amountIsZero, amountsRaw, missingamt, oneLineFmt, showMixedAmountWith) +import Hledger.Data.Amount (amountIsZero, amountsRaw, defaultFmt, missingamt, oneLineFmt, showMixedAmountWith) import Hledger.Data.Transaction (transactionPayee, showTransactionLineFirstPart, partitionAndCheckConversionPostings) import Data.Time (diffDays) import Hledger.Utils import Data.Ord import Hledger.Data.Dates (showDate) import Hledger.Data.Balancing (journalBalanceTransactions, defbalancingopts) +import Hledger.Data.Lots (lotBaseAccount, journalCalculateLots, journalInferAndCheckDisposalBalancing) -- | Run the extra -s/--strict checks on a journal, in order of priority, -- returning the first error message if any of them fail. @@ -58,7 +60,7 @@ journalCheckAccounts :: Journal -> Either String () journalCheckAccounts j = mapM_ checkacct (journalPostings j) where checkacct p@Posting{paccount=a} - | a `elem` journalAccountNamesDeclared j = Right () + | acct `elem` journalAccountNamesDeclared j = Right () | otherwise = Left $ printf (unlines [ "%s:%d:" ,"%s" @@ -67,8 +69,9 @@ journalCheckAccounts j = mapM_ checkacct (journalPostings j) ,"Consider adding an account directive. Examples:" ,"" ,"account %s" - ]) f l ex a a + ]) f l ex acct acct where + acct = lotBaseAccount a (f,l,_mcols,ex) = makePostingAccountErrorExcerpt p -- | Check all balance assertions in the journal and return an error message if any of them fail. @@ -316,7 +319,7 @@ findRecentAssertionError ps = do (showposting firsterrorp) where showposting p = - headDef "" $ first3 $ postingAsLines False True acctw amtw p{pcomment=""} + headDef "" $ first3 $ postingAsLines defaultFmt False True acctw amtw p{pcomment=""} where acctw = T.length $ paccount p amtw = length $ showMixedAmountWith oneLineFmt $ pamount p @@ -330,3 +333,13 @@ findRecentAssertionError ps = do -- (if baiLatestClearedAssertionStatus==Unmarked then " " else show baiLatestClearedAssertionStatus) -- (show baiLatestClearedAssertionDate) -- (diffDays today baiLatestClearedAssertionDate) + +-- | Check all lot tracking calculations: validate lot tag values on declarations, +-- calculate per-lot subaccounts, and verify disposal transactions balance correctly. +-- Equivalent to loading the journal with --lots --verbose-tags. +journalCheckLots :: Journal -> Either String () +journalCheckLots j = + journalCheckLotsTagValues j + >>= journalCalculateLots True + >>= journalInferAndCheckDisposalBalancing True + >> Right () diff --git a/hledger-lib/Hledger/Data/Json.hs b/hledger-lib/Hledger/Data/Json.hs index 12b6fbb0109..c121b91f7e4 100644 --- a/hledger-lib/Hledger/Data/Json.hs +++ b/hledger-lib/Hledger/Data/Json.hs @@ -107,7 +107,7 @@ instance ToJSON MixedAmount where instance ToJSON BalanceAssertion instance ToJSON AmountCost instance ToJSON MarketPrice -instance ToJSON PostingType +instance ToJSON PostingRealness instance ToJSON Posting where toJSON = object . postingKV @@ -127,7 +127,7 @@ postingKV Posting{..} = , "paccount" .= paccount , "pamount" .= pamount , "pcomment" .= pcomment - , "ptype" .= ptype + , "preal" .= preal , "ptags" .= ptags , "pbalanceassertion" .= pbalanceassertion -- To avoid a cycle, show just the parent transaction's index number @@ -219,7 +219,7 @@ instance FromJSON MixedAmount where instance FromJSON BalanceAssertion instance FromJSON AmountCost instance FromJSON MarketPrice -instance FromJSON PostingType +instance FromJSON PostingRealness instance FromJSON Posting instance FromJSON Transaction instance FromJSON AccountDeclarationInfo diff --git a/hledger-lib/Hledger/Data/Lots.hs b/hledger-lib/Hledger/Data/Lots.hs new file mode 100644 index 00000000000..cb7a41c1344 --- /dev/null +++ b/hledger-lib/Hledger/Data/Lots.hs @@ -0,0 +1,1721 @@ +{-| +Lot tracking for investment accounting. + +This module implements two pipeline stages (see doc\/SPEC-finalising.md): + +1. Classification ('journalClassifyLotPostings'): identifies lot postings + and tags them as acquire, dispose, transfer-from, or transfer-to. + +2. Calculation ('journalCalculateLots'): walks transactions in date order, + accumulating a map from commodities to their lots, and: + + - For acquire postings: generates lot names from cost basis and appends + them to the account name as subaccounts. + + - For dispose postings: selects an existing lot subaccount matched by + the posting's lot selector, using the configured reduction method + (FIFO by default, configurable per account\/commodity via @lots:@ tag). + If needed and if the lot selector permits it, selects multiple lots, + splitting the posting into one per lot. + Dispose postings with a transacted price (selling price) generate gain postings. + Bare disposes without a price (e.g. fee deductions) get lot subaccounts but no gain. + + - For transfer postings: selects lots from the source account (like + dispose) and recreates them under the destination account. The lot's + cost basis is preserved through the transfer. + Multi-lot transfers are supported (eg via @{}@ to transfer all lots). + +For background, see doc\/SPEC-lots.md and doc\/PLAN-lots.md. + +== Errors + +User-visible errors from this module. See also Hledger.Data.Errors and doc/ERRORS.md. + +journalCalculateLots: + +* validateUserLabels: + "lot id is not unique: commodity X, date D, label L" + +* processAcquirePosting: + "acquire posting has no cost basis", + "...has multiple cost basis amounts", + "X is lotful but this acquire posting has no cost basis or price", + "...has no lot cost", + "duplicate lot id: {...} for commodity X" + +* processDisposePosting: + "dispose posting has no cost basis", + "...has no transacted price (selling price)", + "...has non-negative quantity", + "SPECID requires a lot selector", + "lot ... has no cost basis (internal error)", + "lot subaccount ... does not match resolved lot" + +* pairIndexedTransferPostings: + "transfer-to/from posting ... has no matching ... posting", + -- "mismatched transfer postings for commodity X", + "... posting has no lotful commodity" + +* processTransferPair: + "transfer-from posting has no cost basis", + "...has multiple cost basis amounts", + "lot transfers should have no transacted price", + "transfer-from posting has non-negative quantity", + "lot ... has no cost basis (internal error)", + "lot cost basis ... does not match transfer-to cost basis" + +* selectLots: + "SPECID requires an explicit lot selector", + "no lots available for commodity X in account Y", + "lot selector is ambiguous, matches N lots in account Y", + "insufficient lots for commodity X in account Y" + +* validateGlobalCompliance: + "METHOD: lot(s) on other account(s) have higher priority than the lots in ACCT" + +* poolWeightedAvgCost: + "no lots with cost basis available for averaging", + "cannot average lots with different cost commodities", + "cannot average lots with zero total quantity" + +* foldMPostings (via isUnclassifiedLotfulPosting): + "X is declared lotful ... but this posting was not classified" + (exempt: zero-amount lotful postings) + +* journalInferAndCheckDisposalBalancing: + "This disposal transaction has multiple amountless gain postings", + "This disposal transaction is unbalanced at cost basis" + +-} + +{-# LANGUAGE CPP #-} +{-# LANGUAGE OverloadedStrings #-} +{-# LANGUAGE NamedFieldPuns #-} + +module Hledger.Data.Lots ( + journalClassifyLotPostings, + journalCalculateLots, + journalInferAndCheckDisposalBalancing, + isGainPosting, + lotBaseAccount, + lotSubaccountName, + mergeCostBasis, + parseLotName, + showLotName, +) where + +import Control.Applicative ((<|>)) +import Data.Bifunctor (first) +import Control.Monad (foldM, guard, unless, when) +import Data.List (intercalate, sort, sortOn) +import Data.Ord (Down(..)) +#if !MIN_VERSION_base(4,20,0) +import Data.List (foldl') +#endif +import Data.Map.Strict qualified as M +import Data.Maybe (catMaybes, fromMaybe, isJust, isNothing) +import Data.Set qualified as S +import Data.Text (Text) +import Data.Text qualified as T +import Data.Char (isDigit) +import Data.Time.Calendar (Day, addDays, fromGregorianValid) +import Text.Printf (printf) + +import Hledger.Data.AccountName (accountNameType) +import Hledger.Data.Dates (showDate) +import Hledger.Data.AccountType (isAssetType, isEquityType) +import Hledger.Data.Amount (amountSetQuantity, amountsRaw, isNegativeAmount, maNegate, mapMixedAmount, mixedAmount, mixedAmountLooksZero, nullmixedamt, noCostFmt, showAmountWith, showMixedAmountOneLine) +import Hledger.Data.Errors (makePostingErrorExcerpt, makePostingErrorExcerptByIndex, makeTransactionErrorExcerpt) +import Hledger.Data.Journal (journalAccountType, journalCommodityLotsMethod, journalCommodityUsesLots, journalFilePaths, journalInheritedAccountTags, journalMapTransactions, journalTieTransactions, parseReductionMethod) +import Hledger.Data.Posting (generatedPostingTagName, hasAmount, isReal, nullposting, originalPosting, postingAddHiddenAndMaybeVisibleTag, postingStripCosts) +import Hledger.Data.Transaction (txnTieKnot) +import Hledger.Data.Types +import Hledger.Utils (dbg5, dbg5With) + +-- Types + +-- | Map from commodity to (map from lot id to (map from account name to lot balance)). +-- Keyed by commodity at the top level so lots of different commodities don't clash. +-- The inner Map LotId is ordered by date then label, supporting FIFO/LIFO naturally. +-- The innermost Map AccountName allows the same lot to exist at multiple accounts +-- (eg after a partial lot transfer). +type LotState = M.Map CommoditySymbol (M.Map LotId (M.Map AccountName Amount)) + +-- | Resolve which reduction method to use for a posting, and where it came from. +-- Checks account-inherited tags first, then commodity tags, defaulting to FIFO. +-- Since commodity tags are propagated to ptags, we distinguish by checking the +-- commodity's own declared tags separately. +resolveReductionMethodWithSource :: Journal -> Posting -> CommoditySymbol -> (ReductionMethod, String) +resolveReductionMethodWithSource j p commodity = + case accountLotsMethod of + Just m -> (m, "from account tag on " ++ T.unpack (paccount p)) + Nothing -> case journalCommodityLotsMethod j commodity of + Just m -> (m, "from commodity tag on " ++ T.unpack commodity) + Nothing -> (FIFO, "default") + where + -- Check account-inherited tags only (excluding commodity-propagated tags). + accountLotsMethod = + let acctTags = journalInheritedAccountTags j (paccount p) + in case [v | (k, v) <- acctTags, T.toLower k == "lots"] of + (v:_) -> parseReductionMethod v + [] -> Nothing + +-- | Strip any trailing lot subaccount (a component starting with '{') from an account name. +-- E.g., @\"assets:broker:{2026-01-15, $50}\"@ becomes @\"assets:broker\"@. +-- Handles the case where a user writes the lot subaccount explicitly in a journal entry. +lotBaseAccount :: AccountName -> AccountName +lotBaseAccount a = + let (prefix, lastComp) = T.breakOnEnd ":" a + in if not (T.null prefix) && "{" `T.isPrefixOf` lastComp + then T.init prefix -- strip the trailing ':' + else a + +-- | Render a lot name in the consolidated hledger format for use as a subaccount name. +-- Format: @{YYYY-MM-DD, COST}@ or @{YYYY-MM-DD, \"LABEL\", COST}@. +showLotName :: CostBasis -> T.Text +showLotName CostBasis{cbDate, cbLabel, cbCost} = + "{" <> T.intercalate ", " parts <> "}" + where + parts = catMaybes + [ fmap (T.pack . show) cbDate + , fmap (\l -> "\"" <> l <> "\"") cbLabel + , fmap (T.pack . showAmountWith noCostFmt) cbCost + ] + +-- | Extract the lot subaccount name (the @{...}@ component) from an account name, +-- or @Nothing@ if there is none. +-- E.g., @\"assets:broker:{2026-01-15, $50}\"@ returns @Just \"{2026-01-15, $50}\"@. +lotSubaccountName :: AccountName -> Maybe Text +lotSubaccountName a = + let (prefix, lastComp) = T.breakOnEnd ":" a + in if not (T.null prefix) && "{" `T.isPrefixOf` lastComp + then Just lastComp + else Nothing + +-- | Parse a lot name (as produced by 'showLotName') back into a 'CostBasis'. +-- The input should be the full @{...}@ string including braces. +-- Parts are comma-separated and all optional: date (@YYYY-MM-DD@), +-- label (@\"LABEL\"@), and cost (parsed by the supplied callback). +-- The callback avoids an import cycle (Lots.hs cannot import Read-layer parsers). +parseLotName :: (String -> Maybe Amount) -> Text -> Either String CostBasis +parseLotName parseAmt t = do + inner <- case T.stripPrefix "{" t >>= T.stripSuffix "}" of + Just s -> Right (T.strip s) + Nothing -> Left $ "lot name must be enclosed in braces: " ++ T.unpack t + if T.null inner + then Right $ CostBasis Nothing Nothing Nothing + else do + let parts = map T.strip $ splitParts inner + parseParts parts + where + parseParts parts = go parts Nothing Nothing Nothing + where + go [] d l c = Right $ CostBasis d l c + go (p:ps) d l c + | isDatePart p = case parseDate p of + Just day -> go ps (Just day) l c + Nothing -> Left $ "invalid date in lot name: " ++ T.unpack p + | isLabelPart p = go ps d (Just (T.drop 1 (T.dropEnd 1 p))) c + | otherwise = case parseAmt (T.unpack p) of + Just amt -> go ps d l (Just amt) + Nothing -> Left $ "cannot parse lot name part: " ++ T.unpack p + + isDatePart p = T.length p == 10 && T.all (\c -> isDigit c || c == '-') p + isLabelPart p = "\"" `T.isPrefixOf` p && "\"" `T.isSuffixOf` p && T.length p >= 2 + + parseDate p = do + let s = T.unpack p + case s of + [y1,y2,y3,y4,'-',m1,m2,'-',d1,d2] -> + fromGregorianValid + (read [y1,y2,y3,y4]) + (read [m1,m2]) + (read [d1,d2]) + _ -> Nothing + + -- Split the inner text of a lot name into up to 3 parts (date, label, cost) + -- by peeling known-format prefixes in DLC order, exploiting the fact that + -- dates and labels have unambiguous syntax. This avoids splitting on commas, + -- which would break when the cost amount contains a decimal comma (e.g. @1,5@) + -- or when the commodity symbol contains commas (e.g. @"an, odd, commodity"@). + -- After each part, @,@ followed by optional whitespace is consumed. + -- + -- Examples: + -- @"2026-01-15, \"my, label\", €1,50"@ -> @["2026-01-15", "\"my, label\"", "€1,50"]@ + -- @"2026-01-15, \"an, odd, commodity\" 1,5"@ -> @["2026-01-15", "\"an, odd, commodity\" 1,5"]@ + -- @"2026-01-15, \"a, b\", \"an, odd, commodity\" 1,5" -> @["2026-01-15", "\"a, b\"", "\"an, odd, commodity\" 1,5"]@ + -- @"$100"@ -> @["$100"]@ + splitParts :: Text -> [Text] + splitParts s = + let (mdate, s1) = peelDate s + (mlabel, s2) = peelLabel s1 + mcost = let c = T.strip s2 in if T.null c then Nothing else Just c + in catMaybes [mdate, mlabel, mcost] + + -- Try to peel a date (YYYY-MM-DD) from the front. Returns the date text + -- and the remainder after stripping a comma separator, or Nothing and the + -- unchanged input. Requires that the date candidate is followed by end of + -- string, a comma, or whitespace (to avoid greedily consuming digits that + -- are part of a cost amount). + peelDate :: Text -> (Maybe Text, Text) + peelDate s + | T.length s >= 10 + , let (candidate, rest) = T.splitAt 10 s + , isDatePart candidate + , T.null rest || T.head rest == ',' || T.head rest == ' ' + = (Just candidate, stripSep rest) + | otherwise = (Nothing, s) + + -- Try to peel a double-quoted label from the front. Scans from the opening + -- quote to the next closing quote. A quoted string is only treated as a + -- label if it is followed by a comma separator or end of input. If it is + -- followed by digits or other amount-starting characters (after optional + -- whitespace), it is a quoted commodity symbol that belongs to the cost + -- amount, so we leave it alone. + peelLabel :: Text -> (Maybe Text, Text) + peelLabel s + | Just s' <- T.stripPrefix "\"" s + = let (inner, rest) = T.break (== '"') s' + in case T.uncons rest of + Just ('"', rest') + | looksLikeCostRemainder rest' -> (Nothing, s) -- quoted commodity symbol, not a label + | otherwise -> (Just ("\"" <> inner <> "\""), stripSep rest') + _ -> (Nothing, s) -- malformed, leave for parseAmt + | otherwise = (Nothing, s) + + -- After a closing quote, does the remainder look like it continues as a + -- cost amount (i.e. the quoted string was a commodity symbol, not a label)? + -- True when the next non-space character is a digit, sign, or decimal mark. + looksLikeCostRemainder :: Text -> Bool + looksLikeCostRemainder t' = + case T.uncons (T.stripStart t') of + Just (c, _) -> isDigit c || c == '+' || c == '-' || c == '.' + Nothing -> False + + -- Strip an optional comma and any surrounding whitespace. + stripSep :: Text -> Text + stripSep s0 = + let s1 = T.stripStart s0 + in case T.uncons s1 of + Just (',', s2) -> T.stripStart s2 + _ -> s1 + +-- | Merge two 'CostBasis' values. For each field, if both are @Just@, they +-- must agree (returns error if not); otherwise takes whichever is @Just@. +-- The first argument is typically the account-name basis (more complete), +-- the second is the amount's existing basis. +mergeCostBasis :: CostBasis -> CostBasis -> Either String CostBasis +mergeCostBasis a b = do + d <- mergeField "date" show (cbDate a) (cbDate b) + l <- mergeField "label" T.unpack (cbLabel a) (cbLabel b) + c <- mergeCostField (cbCost a) (cbCost b) + Right $ CostBasis d l c + where + mergeField :: Eq a => String -> (a -> String) -> Maybe a -> Maybe a -> Either String (Maybe a) + mergeField _ _ Nothing y = Right y + mergeField _ _ x Nothing = Right x + mergeField name showVal (Just x) (Just y) + | x == y = Right (Just x) + | otherwise = Left $ "conflicting cost basis " ++ name + ++ ": account name has " ++ showVal x + ++ " but amount has " ++ showVal y + + mergeCostField :: Maybe Amount -> Maybe Amount -> Either String (Maybe Amount) + mergeCostField Nothing y = Right y + mergeCostField x Nothing = Right x + mergeCostField (Just x) (Just y) + | acommodity x == acommodity y && aquantity x == aquantity y = Right (Just x) + | otherwise = Left $ "conflicting cost basis cost" + ++ ": account name has " ++ showAmountWith noCostFmt x + ++ " but amount has " ++ showAmountWith noCostFmt y + +-- Classification (pipeline stage 1) + +-- | Classify lot-related postings by adding ptype tags. +-- Must be called after journalAddAccountTypes so account types are available. +-- The verbosetags parameter controls whether the ptype tags will be made visible in comments. +journalClassifyLotPostings :: Bool -> Journal -> Journal +journalClassifyLotPostings verbosetags j = journalMapTransactions (transactionClassifyLotPostings verbosetags lookupType commodityIsLotful) j + where + lookupType = journalAccountType j + commodityIsLotful = journalCommodityUsesLots j + +-- | Classify lot-related postings by adding a ptype tag. +-- For each posting with a cost basis (any account type): +-- - determine if it's of type "acquire" or "dispose" (based on amount sign) +-- - or "transfer-from" or "transfer-to" (if counterposting with same commodity exists in a different account). +-- For asset postings without cost basis: +-- - a lotful posting can be classified as "transfer-to" (see shouldClassifyLotful) +-- - a bare positive posting can be classified as "transfer-to" if there's a +-- matching transfer-from counterpart (see shouldClassifyBareTransferTo). +-- The lookupAccountType function should typically be `journalAccountType journal`. +-- The commodityIsLotful function should typically be `journalCommodityUsesLots journal`. +-- The verbosetags parameter controls whether the tags are made visible in comments. +-- +-- For more detail on classification rules, please see doc/SPEC-lots.md > Lot postings. +-- +transactionClassifyLotPostings :: Bool -> (AccountName -> Maybe AccountType) -> (CommoditySymbol -> Bool) -> Transaction -> Transaction +transactionClassifyLotPostings verbosetags lookupAccountType commodityIsLotful t@Transaction{tpostings=ps} + | not (any hasLotRelevantAmount ps) && not (any isGainAcct ps) + = lotDbg t "no lot-relevant amounts, skipping" t + | otherwise = lotDbg t "classifying" $ t{tpostings=zipWith classifyAt [0..] ps} + where + hasCostBasis :: Posting -> Bool + hasCostBasis p = let amts = amountsRaw (pamount p) + result = any (isJust . acostbasis) amts + in dbg5 ("classifyLotPostings: hasCostBasis " ++ show (paccount p) ++ " amts=" ++ show (length amts)) result + + hasLotRelevantAmount :: Posting -> Bool + hasLotRelevantAmount p = isReal p && (hasCostBasis p || hasNegativeLotfulAmount p || hasPositiveLotfulAmount p) + + hasNegativeLotfulAmount :: Posting -> Bool + hasNegativeLotfulAmount p = + let amts = amountsRaw (pamount p) + in not (any (isJust . acostbasis) amts) + && any isNegativeAmount amts + && postingIsLotful p amts + + hasPositiveLotfulAmount :: Posting -> Bool + hasPositiveLotfulAmount p = + let amts = amountsRaw (pamount p) + in not (any (isJust . acostbasis) amts) + && not (any isNegativeAmount amts) + && any (\a -> aquantity a > 0) amts -- has a strictly positive amount (not zero or amountless) + && postingIsLotful p amts + + -- Same-account transfer pairs: within each account, match positive and negative + -- postings with the same commodity and absolute quantity as transfer pairs. + -- When there are more of one sign than the other, the excess are left unmatched + -- (and will be classified normally as acquire/dispose). + sameAcctTransferSet :: S.Set Int + sameAcctTransferSet = S.fromList $ concatMap matchPairs $ M.elems grouped + where + grouped :: M.Map (AccountName, CommoditySymbol, Quantity) ([Int], [Int]) + grouped = foldl' addPosting M.empty (zip [0..] ps) + addPosting m (i, p) + | not (isReal p) = m + | not (hasLotRelevantAmount p) = m + | otherwise = foldl' (addAmt i (lotBaseAccount (paccount p))) m (amountsRaw (pamount p)) + addAmt i acct m a + | q < 0 = M.insertWith mergePair (acct, acommodity a, negate q) ([i], []) m + | q > 0 = M.insertWith mergePair (acct, acommodity a, q) ([], [i]) m + | otherwise = m + where q = aquantity a + mergePair (n1, p1) (n2, p2) = (n1++n2, p1++p2) + matchPairs (negs, poss) = + let n = min (length negs) (length poss) + in take n negs ++ take n poss + + -- Precompute per-commodity, per-quantity transfer counterpart info (O(n)). + -- Keyed by (commodity, |quantity|) so that transfer detection requires exact quantity matching. + -- For each key, which accounts have: + -- negative postings with cost basis (transfer-from candidates; any account type) + -- positive postings with cost basis (transfer-to candidates, standard path; any account type) + -- positive asset postings without cost basis (transfer-to candidates, bare path; asset only) + -- Account lists are deduplicated (typically 1-2 accounts per commodity). + negCBAccts, posCBAccts, posNoCBAccts :: M.Map (CommoditySymbol, Quantity) [AccountName] + (negCBAccts, posCBAccts, posNoCBAccts) = foldl' collect (M.empty, M.empty, M.empty) (zip [0..] ps) + where + collect (!neg, !pos, !noCB) (i, p) + | not (isReal p) = (neg, pos, noCB) -- skip virtual postings + | i `S.member` sameAcctTransferSet = (neg, pos, noCB) -- skip same-account transfer pairs + | otherwise = + let baseAcct = lotBaseAccount (paccount p) + isAsset = maybe False isAssetType (lookupAccountType baseAcct) + amts = amountsRaw (pamount p) + acct = baseAcct + isNeg = any isNegativeAmount amts + hasCB = any (isJust . acostbasis) amts + isLotful = postingIsLotful p amts + cbKeys = [(acommodity a, abs (aquantity a)) | a <- amts, isJust (acostbasis a)] + allKeys = [(acommodity a, abs (aquantity a)) | a <- amts] + -- Include cost-basis negatives (any account type) and bare lotful + -- negatives on asset accounts. Non-asset bare lotful negatives + -- (e.g. revenue) are excluded — they shouldn't be transfer counterparts. + -- Equity transfers are handled separately via hasEquityCounterpart. + neg' = if isNeg && (hasCB || (isLotful && isAsset)) + then foldl' (addAcct acct) neg (if hasCB then cbKeys else allKeys) else neg + pos' = if not isNeg && hasCB + then foldl' (addAcct acct) pos cbKeys else pos + noCB' = if not isNeg && isAsset && not hasCB + then foldl' (addAcct acct) noCB allKeys else noCB + in (neg', pos', noCB') + -- Add an account to a (commodity, quantity) key's account list, deduplicating. + addAcct acct m k = M.insertWith (\_ old -> if acct `elem` old then old else acct : old) k [acct] m + + -- Is there a transfer counterpart for a posting in this account, with this sign, + -- in this commodity and quantity? + hasCounterpart :: AccountName -> Bool -> CommoditySymbol -> Quantity -> Bool + hasCounterpart acct isNeg c q + | isNeg = anyOtherAcct posCBAccts || anyOtherAcct posNoCBAccts + | otherwise = anyOtherAcct negCBAccts + where anyOtherAcct m = any (/= acct) (M.findWithDefault [] (c, abs q) m) + + -- Is there a transfer-from counterpart (negative with cost basis or lotful) + -- for this commodity and quantity? + hasTransferFromCounterpart :: AccountName -> CommoditySymbol -> Quantity -> Bool + hasTransferFromCounterpart acct c q = any (/= acct) (M.findWithDefault [] (c, abs q) negCBAccts) + + -- Commodity-only transfer-from check (ignores quantity). + -- Fallback for transfer+fee patterns where quantities don't match exactly. + hasTransferFromCommodityMatch :: AccountName -> CommoditySymbol -> Bool + hasTransferFromCommodityMatch acct c = + any (\((c', _), accts) -> c' == c && any (/= acct) accts) + (M.toList negCBAccts) + + isGainAcct :: Posting -> Bool + isGainAcct p = lookupAccountType (paccount p) == Just Gain + + classifyAt :: Int -> Posting -> Posting + classifyAt i p + | not (isReal p) = p -- skip virtual (parenthesised) postings + | i `S.member` sameAcctTransferSet = + let amts = amountsRaw (pamount p) + cls = if any isNegativeAmount amts then "transfer-from" else "transfer-to" + in addTag cls p + | otherwise = + case dbg5 ("classifyLotPostings: classifyPosting " ++ show (paccount p) ++ " result") $ shouldClassify p of + Just classification -> addTag classification p + Nothing + | isGainAcct p -> addTag "gain" p + | otherwise -> p + where addTag cls = postingAddHiddenAndMaybeVisibleTag True verbosetags (toHiddenTag ("ptype", cls)) + + -- Check if posting should be classified and return the classification: + -- one of acquire, dispose, transfer-from, transfer-to. + shouldClassify :: Posting -> Maybe Text + shouldClassify p = do + let amts = amountsRaw $ pamount p + baseAcct = lotBaseAccount (paccount p) + if any (isJust . acostbasis) amts + -- Cost basis present: classify regardless of account type (fix A) + then dbg5 ("classifyLotPostings: shouldClassify " ++ show (paccount p) ++ " withCostBasis") $ + shouldClassifyWithCostBasis p amts + else do + -- No cost basis: require asset account type + acctType <- dbg5 ("classifyLotPostings: shouldClassify " ++ show (paccount p) ++ " acctType") $ + lookupAccountType baseAcct + guard $ isAssetType acctType + dbg5 ("classifyLotPostings: shouldClassify " ++ show (paccount p) ++ " lotful/bare") $ + shouldClassifyNegativeLotful p amts <|> shouldClassifyLotful p amts <|> shouldClassifyBareTransferTo p amts <|> shouldClassifyPositiveLotful p amts + + -- True when the transaction has an equity posting with no explicit cost-basis amounts. + -- This indicates an equity transfer: lots move to/from equity in two parts + -- (e.g. close --clopen --lots generates a closing txn transferring lots into equity, + -- and an opening txn transferring them back out), allowing the negative lot postings + -- to be classified as transfer-from rather than dispose. + hasEquityCounterpart :: Bool + hasEquityCounterpart = any isEquityNonLotPosting ps + where + isEquityNonLotPosting q = + maybe False isEquityType (lookupAccountType (lotBaseAccount (paccount q))) + && not (any (isJust . acostbasis) (amountsRaw (pamount q))) + + -- Classify a posting that has cost basis: acquire, dispose, transfer-from, or transfer-to. + shouldClassifyWithCostBasis :: Posting -> [Amount] -> Maybe Text + shouldClassifyWithCostBasis p amts = do + let + baseAcct = lotBaseAccount (paccount p) + isNeg = any isNegativeAmount amts + primaryType = if isNeg then "dispose" else "acquire" + cbAmts = [(acommodity a, aquantity a) | a <- amts, isJust (acostbasis a)] + isTransfer = any (\(c, q) -> hasCounterpart baseAcct isNeg c q) cbAmts + -- Also treat as equity transfer when: no transacted price written, + -- and an equity counterpart posting is present. This handles lots moving + -- to/from equity (e.g. close --clopen --lots generates a closing txn with + -- negative lot postings and an opening txn with positive lot postings). + isEquityTransfer = not (any (isJust . acost) amts) && hasEquityCounterpart + if isTransfer || isEquityTransfer + then return $ if isNeg then "transfer-from" else "transfer-to" + else do + -- Don't classify income statement accounts (Revenue, Expense, Gain) as acquire/dispose. + -- These are flow accounts that should not track lots or get lot subaccounts. + -- E.g. expenses:fees 0.1 ETSY {$80} @ $90 in a stock-fee disposal. + guard $ not $ maybe False isIncomeStatementAccountType (lookupAccountType baseAcct) + return primaryType + + -- Classify a negative lotful posting without cost basis as dispose or transfer-from. + -- If the posting has no transacted price and another asset account in the same + -- transaction receives a positive lotful amount of the same commodity (even at + -- different quantity), skip classification — it's a transfer+fee pattern and + -- global FIFO will handle the lot reduction when the destination account trades. + shouldClassifyNegativeLotful :: Posting -> [Amount] -> Maybe Text + shouldClassifyNegativeLotful p amts = do + guard $ postingIsLotful p amts + guard $ any isNegativeAmount amts + let baseAcct = lotBaseAccount (paccount p) + hasPrice = any (isJust . acost) amts + negAmts = [a | a <- amts, isNegativeAmount a] + negCommodities = S.fromList [acommodity a | a <- negAmts] + amtPairs = [(acommodity a, aquantity a) | a <- amts] + isTransfer = any (\(c, q) -> hasCounterpart baseAcct True c q) amtPairs + -- Check for positive lotful amounts in other asset accounts (same commodity). + otherAssetReceives = any isOtherAssetWithLotful (filter (/= p) ps) + isOtherAssetWithLotful q = + let qAmts = amountsRaw (pamount q) + qBase = lotBaseAccount (paccount q) + in qBase /= baseAcct + && maybe False isAssetType (lookupAccountType qBase) + && postingIsLotful q qAmts + && any (\a -> aquantity a > 0 && acommodity a `S.member` negCommodities) qAmts + -- Does a non-asset posting receive exactly this commodity+quantity? + -- If so, this posting is likely a fee/dispose (e.g. paired with expenses:fees), + -- not part of a transfer to another asset account. + hasFeeCounterpart = any isFeeCounterpart (filter (/= p) ps) + isFeeCounterpart q = + let qAmts = amountsRaw (pamount q) + qBase = lotBaseAccount (paccount q) + in not (maybe False isAssetType (lookupAccountType qBase)) + && any (\a -> aquantity a > 0 + && any (\na -> acommodity na == acommodity a + && abs (aquantity na) == aquantity a) negAmts) qAmts + if isTransfer || (otherAssetReceives && not hasFeeCounterpart) + then return "transfer-from" + else do + guard $ hasPrice || hasFeeCounterpart || not otherAssetReceives + return "dispose" + + -- Classify a lotful posting without cost basis. + -- A positive posting in a lotful commodity/account, with no transacted price, + -- and with a matching transfer-from counterpart, is classified as transfer-to. + shouldClassifyLotful :: Posting -> [Amount] -> Maybe Text + shouldClassifyLotful p amts = do + guard $ postingIsLotful p amts + guard $ not $ any isNegativeAmount amts + guard $ not $ any (isJust . acost) amts + let baseAcct = lotBaseAccount (paccount p) + amtPairs = [(acommodity a, aquantity a) | a <- amts] + guard $ any (\(c, q) -> hasTransferFromCounterpart baseAcct c q) amtPairs + || any (\(c, _) -> hasTransferFromCommodityMatch baseAcct c) amtPairs + return "transfer-to" + + -- Classify a bare positive asset posting (no cost basis, not necessarily lotful) + -- as transfer-to if there's a matching transfer-from counterpart (fix C). + shouldClassifyBareTransferTo :: Posting -> [Amount] -> Maybe Text + shouldClassifyBareTransferTo p amts = do + guard $ not $ any isNegativeAmount amts + guard $ not $ any (isJust . acost) amts + let baseAcct = lotBaseAccount (paccount p) + amtPairs = [(acommodity a, aquantity a) | a <- amts] + guard $ any (\(c, q) -> hasTransferFromCounterpart baseAcct c q) amtPairs + || any (\(c, _) -> hasTransferFromCommodityMatch baseAcct c) amtPairs + return "transfer-to" + + -- Classify a positive lotful posting without cost basis as acquire. + -- This is the fallback for positive lotful postings that aren't transfer-to. + -- Requires a plausible cost source: transacted price, different-commodity posting + -- (for balancer inference), or a transfer-from counterpart (whose lot cost is inherited). + -- Without any of these, no lot can be created so we skip classification. + shouldClassifyPositiveLotful :: Posting -> [Amount] -> Maybe Text + shouldClassifyPositiveLotful p amts = do + guard $ postingIsLotful p amts + guard $ any (\a -> aquantity a > 0) amts + let commodities = S.fromList [acommodity a | a <- amts] + hasPrice = any (isJust . acost) amts + hasDiffCommodity = any (\q -> any ((`S.notMember` commodities) . acommodity) (amountsRaw (pamount q))) + (filter (\q -> q /= p && hasAmount q) ps) + baseAcct = lotBaseAccount (paccount p) + hasTransferFrom = any (\(c, q) -> hasTransferFromCounterpart baseAcct c q) + [(acommodity a, aquantity a) | a <- amts] + guard $ hasPrice || hasDiffCommodity || hasTransferFrom + return "acquire" + + -- Check if a posting is lotful: its commodity or account has a lots: tag. + postingIsLotful :: Posting -> [Amount] -> Bool + postingIsLotful p amts = + any ((== "lots") . T.toLower . fst) (ptags p) -- account lots: tag (inherited via ptags) + || any (commodityIsLotful . acommodity) amts -- commodity lots: tag + +-- Lot calculation (pipeline stage 2) + +-- | Calculate detailed lot movements by walking transactions in date order. +-- Handles acquire postings (generating lot names as subaccounts), +-- dispose postings (matching to existing lots using FIFO, splitting if needed), +-- and transfer postings (moving lots between accounts, preserving cost basis). +-- The verbosetags parameter controls whether generated-posting tags are made visible in comments. +-- All lot selection/classification failures are hard errors. +journalCalculateLots :: Bool -> Journal -> Either String Journal +journalCalculateLots verbosetags j + | not $ any (any isLotPosting . tpostings) txns = do + mapM_ checkUnclassified [(t, i, p) | t <- txns, (i, p) <- zip [0..] (tpostings t)] + Right j + | otherwise = do + validateUserLabels txns + let needsLabels = findDatesNeedingLabels txns + (_, txns') <- foldM (processTransaction verbosetags j needsLabels) (M.empty, []) (sortOn tdate txns) + Right (journalTieTransactions $ j{jtxns = reverse txns'}) + where + txns = jtxns j + checkUnclassified (t, i, p) + | isUnclassifiedLotfulPosting j p = Left (unclassifiedLotWarning j t i p) + | otherwise = Right () + +-- Disposal balancing + +-- | For each disposal transaction, infer any amountless gain posting's amount from cost basis, +-- then check that the transaction is balanced at cost basis. +-- This runs after journalCalculateLots, which has filled in cost basis info on dispose postings. +-- The verbosetags parameter controls whether the ptype tag will be made visible in comments. +journalInferAndCheckDisposalBalancing :: Bool -> Journal -> Either String Journal +journalInferAndCheckDisposalBalancing verbosetags j = do + txns' <- mapM inferGainInTransaction (jtxns j) + Right j{jtxns = txns'} + where + atypes = jaccounttypes j + + isGain :: Posting -> Bool + isGain p = accountNameType atypes (paccount p) == Just Gain + + gainAccount = case sort [a | (a, Gain) <- M.toList atypes] of + [] -> "revenue:gains" + (a:_) -> a + + tagGain :: Posting -> Posting + tagGain = postingAddHiddenAndMaybeVisibleTag True verbosetags (toHiddenTag ("ptype", "gain")) + + disposeHasPrice p = isDisposePosting p && any (isJust . acost) (amountsRaw (pamount p)) + + inferGainInTransaction t + | not (any isDisposePosting (tpostings t)) = Right t + | not (any disposeHasPrice (tpostings t)) = Right t + | otherwise = do + let (gainPs, otherPs) = partition' isGain (tpostings t) + (amountfulGainPs, amountlessGainPs) = partition' hasAmount gainPs + case amountlessGainPs of + -- No amountless gain postings + [] + -- No gain postings at all: create one if residual is nonzero + | null gainPs -> do + let residual = foldMap postingCostBasisAmount (tpostings t) + if mixedAmountLooksZero residual + then Right t + else do + let inferredAmt = maNegate residual + gp = postingAddHiddenAndMaybeVisibleTag False verbosetags (generatedPostingTagName, "") + $ tagGain nullposting{paccount = gainAccount, pamount = inferredAmt} + t' = txnTieKnot $ t{tpostings = tpostings t ++ [gp]} + checkBalance t' + Right t' + -- Has amountful gain postings, just check balance + | otherwise -> do + checkBalance t + Right t + -- One amountless gain posting: infer its amount, then check + [gp] -> do + let otherSum = foldMap postingCostBasisAmount (otherPs ++ amountfulGainPs) + inferredAmt = maNegate otherSum + gp' = gp{ pamount = inferredAmt + , poriginal = Just $ originalPosting gp + } + t' = t{tpostings = map (\p -> if p == gp then gp' else p) (tpostings t)} + checkBalance t' + Right t' + -- Multiple amountless gain postings: error + _ -> Left $ txnErrPrefix t + ++ "This disposal transaction has multiple amountless gain postings.\n" + ++ "At most one gain posting may have its amount inferred." + + checkBalance t = + let costBasisSum = foldMap postingCostBasisAmount (tpostings t) + in unless (mixedAmountLooksZero costBasisSum) $ + Left $ disposalBalanceError t costBasisSum + + -- Value a posting at cost basis: if it has a cost basis, use quantity * basis cost; + -- otherwise use the raw amount. Amountless postings contribute nothing. + postingCostBasisAmount :: Posting -> MixedAmount + postingCostBasisAmount p + | not (hasAmount p) = nullmixedamt + | otherwise = foldMap amountCostBasisValue (amountsRaw (pamount p)) + + amountCostBasisValue :: Amount -> MixedAmount + amountCostBasisValue a = case acostbasis a >>= cbCost of + Just basisCost -> mixedAmount basisCost{aquantity = aquantity a * aquantity basisCost} + Nothing -> mixedAmount a + + disposalBalanceError :: Transaction -> MixedAmount -> String + disposalBalanceError t residual = + txnErrPrefix t + ++ "This disposal transaction is unbalanced at cost basis.\n" + ++ "Residual: " ++ showMixedAmountOneLine residual + + -- Like Data.List.partition but preserves the type for the predicate + partition' :: (a -> Bool) -> [a] -> ([a], [a]) + partition' _ [] = ([], []) + partition' f (x:xs) + | f x = (x:yes, no) + | otherwise = (yes, x:no) + where (yes, no) = partition' f xs + +-- Posting type predicates + +-- | When an implicit-lot-subaccount posting (one whose account was a plain account, +-- not already a lot subaccount) is converted to explicit lot subaccount posting(s), +-- and it had a balance assertion, move the assertion to a new zero-amount generated +-- posting on the original parent account, making it subaccount-inclusive (=* style). +-- This preserves the assertion's meaning when the output is re-read without --lots: +-- the assertion checks the total of all lot subaccounts rather than the (empty) +-- direct balance of the parent. +-- If the original account was already a lot subaccount, the split postings are +-- returned unchanged (the assertion already targets the right account). +preserveParentAssertion :: Bool -> AccountName -> Maybe BalanceAssertion -> [Posting] -> [Posting] +preserveParentAssertion _ _ Nothing ps = ps +preserveParentAssertion _ origAcct (Just _) ps + | lotBaseAccount origAcct /= origAcct = ps -- already an explicit lot subaccount; leave as-is +preserveParentAssertion verbosetags origAcct (Just ba) ps = + map (\p -> p{pbalanceassertion = Nothing}) ps + ++ [ postingAddHiddenAndMaybeVisibleTag False verbosetags (generatedPostingTagName, "") + nullposting + { paccount = origAcct + , pamount = mixedAmount (baamount ba){aquantity = 0} + , pbalanceassertion = Just ba{bainclusive = True} + } ] + +-- | Check if a posting has any lot-related ptype tag. +isLotPosting :: Posting -> Bool +isLotPosting p = isAcquirePosting p || isDisposePosting p + || isTransferFromPosting p || isTransferToPosting p + || isGainPosting p + +-- | Check if a posting is an acquire posting (has _ptype:acquire tag). +isAcquirePosting :: Posting -> Bool +isAcquirePosting p = ("_ptype", "acquire") `elem` ptags p + +-- | Check if a posting is a dispose posting (has _ptype:dispose tag). +isDisposePosting :: Posting -> Bool +isDisposePosting p = ("_ptype", "dispose") `elem` ptags p + +-- | Check if a posting is a transfer-from posting (has _ptype:transfer-from tag). +isTransferFromPosting :: Posting -> Bool +isTransferFromPosting p = ("_ptype", "transfer-from") `elem` ptags p + +-- | Check if a posting is a transfer-to posting (has _ptype:transfer-to tag). +isTransferToPosting :: Posting -> Bool +isTransferToPosting p = ("_ptype", "transfer-to") `elem` ptags p + +-- | Check if a posting is a gain posting (has _ptype:gain tag). +isGainPosting :: Posting -> Bool +isGainPosting p = ("_ptype", "gain") `elem` ptags p + +-- | True if this posting involves a lotful commodity/account in an asset account +-- but has no _ptype tag (wasn't classified as acquire/dispose/transfer/gain). +-- Postings with zero amount in the lotful commodity are exempt (no lot tracking needed). +isUnclassifiedLotfulPosting :: Journal -> Posting -> Bool +isUnclassifiedLotfulPosting j p = + isReal p + && hasAmount p + && not (isLotPosting p) + && maybe False isAssetType (journalAccountType j (lotBaseAccount (paccount p))) + && hasNonzeroLotfulAmount + where + amts = amountsRaw (pamount p) + lotfulAmts = filter (journalCommodityUsesLots j . acommodity) amts + hasAccountLotsTag = any ((== "lots") . T.toLower . fst) (ptags p) + -- Flag only when the lotful commodity itself has nonzero quantity. + -- For account-level lots: tags with no commodity-lotful amounts, fall back to all amounts. + hasNonzeroLotfulAmount + | not (null lotfulAmts) = any ((/= 0) . aquantity) lotfulAmts + | hasAccountLotsTag = any ((/= 0) . aquantity) amts + | otherwise = False + +-- | Build an error message for an unclassified lotful posting. +-- Takes the transaction and the 0-based posting index for precise source location. +unclassifiedLotWarning :: Journal -> Transaction -> Int -> Posting -> String +unclassifiedLotWarning j t idx p = + let amts = amountsRaw (pamount p) + lotfulCommodities = [acommodity a | a <- amts, journalCommodityUsesLots j (acommodity a)] + hasAccountTag = any ((== "lots") . T.toLower . fst) (ptags p) + source = case (lotfulCommodities, hasAccountTag) of + (c:_, _) -> T.unpack c ++ " is declared lotful (commodity lots: tag)" + ([], True) -> T.unpack (paccount p) ++ " is declared lotful (account lots: tag)" + _ -> "posting involves a lotful commodity or account" + (f, line, _, ex) = makePostingErrorExcerptByIndex t idx + in printf "%s:%d:\n%s\n" f line ex + ++ source ++ " but this posting was not classified as\n" + ++ "acquire, dispose, or transfer. Lot state will not be updated.\n" + ++ "Possible fixes: add a cost basis ({$X}), a price (@ $X),\n" + ++ "or check the account type declaration." + +-- Validation and label generation + +-- | Validate that user-provided labels don't create duplicate lot ids. +validateUserLabels :: [Transaction] -> Either String () +validateUserLabels txns = + case M.toList duplicates of + [] -> Right () + (((c, d, l), t2:_):_) -> + let (f, line, _, ex) = makeTransactionErrorExcerpt t2 (const Nothing) + in Left $ printf (unlines [ + "%s:%d:" + ,"%s" + ,"lot id is not unique: commodity %s, date %s, label \"%s\"" + ]) f line ex (T.unpack c) (show d) (T.unpack l) + _ -> Right () -- shouldn't happen + where + labeled = [ ((acommodity a, getLotDate t cb, l), t) + | t <- txns + , p <- tpostings t + , isAcquirePosting p + , a <- amountsRaw (pamount p) + , Just cb <- [acostbasis a] + , Just l <- [cbLabel cb] + ] + txnsByKey = foldl' (\m (k, t) -> M.insertWith (++) k [t] m) M.empty labeled + duplicates = M.filter (\ts -> length ts > 1) txnsByKey + +-- | Find (commodity, date) pairs that have multiple acquisitions and thus need labels. +-- Only counts acquisitions that don't already have a user-provided label. +-- Includes bare acquire postings (no acostbasis), which use the transaction date. +findDatesNeedingLabels :: [Transaction] -> S.Set (CommoditySymbol, Day) +findDatesNeedingLabels txns = + M.keysSet $ M.filter (> 1) counts + where + counts = foldl' countAcquire M.empty + [ (acommodity a, acquireDate t a) + | t <- txns + , p <- tpostings t + , isAcquirePosting p + , a <- amountsRaw (pamount p) + ] + countAcquire m (c, d) = M.insertWith (+) (c, d) (1 :: Int) m + acquireDate t a = case acostbasis a of + Just cb -> getLotDate t cb + Nothing -> tdate t + +-- | Format a verbose error prefix for a transaction: "file:line:\nexcerpt\n\n". +-- Prepend to an error message to show source position and a transaction excerpt. +txnErrPrefix :: Transaction -> String +txnErrPrefix t = printf "%s:%d:\n%s\n" f line ex + where (f, line, _, ex) = makeTransactionErrorExcerpt t (const Nothing) + +-- | Format a verbose error prefix for a posting: "file:line:\nexcerpt\n\n". +-- Like txnErrPrefix but highlights the specific posting line. +-- Strips costs from the posting before lookup, since makePostingErrorExcerpt +-- matches by comparing cost-stripped transaction postings against the argument. +postingErrPrefix :: Posting -> String +postingErrPrefix p = printf "%s:%d:\n%s\n" f line ex + where (f, line, _, ex) = makePostingErrorExcerpt (postingStripCosts p) (\_ _ _ -> Nothing) + +-- | Emit a dbg5 trace for a lot operation: "lots: FILE:LINE DATE DESC: message". +lotDbg :: Transaction -> String -> a -> a +lotDbg t msg = dbg5With (\_ -> "lots: " ++ txnDbgPrefix t ++ ": " ++ msg) + +-- | Format a one-line transaction summary for debug traces: "FILE:LINE DATE DESC". +txnDbgPrefix :: Transaction -> String +txnDbgPrefix t = printf "%s:%d %s %s" f line (show (tdate t)) (T.unpack (tdescription t)) + where (f, line, _, _) = makeTransactionErrorExcerpt t (const Nothing) + +-- | Get the lot date from cost basis, falling back to the transaction date. +getLotDate :: Transaction -> CostBasis -> Day +getLotDate t cb = fromMaybe (tdate t) (cbDate cb) + +-- | Generate a label for a lot that needs one (due to same-date collision). +-- Uses a sequence number formatted as four (or more) digits: "0001", "0002", etc. +-- Uses O(log n) Map operations to count same-date lots efficiently. +generateLabel :: CommoditySymbol -> Day -> LotState -> T.Text +generateLabel commodity date lotState = + T.pack $ printf "%04d" nextNum + where + existingLots = M.findWithDefault M.empty commodity lotState + -- Use takeWhileAntitone/dropWhileAntitone for O(log n) range extraction. + -- LotId is ordered by date first, so same-date lots form a contiguous range. + sameDate = M.takeWhileAntitone (\(LotId d _) -> d == date) + $ M.dropWhileAntitone (\(LotId d _) -> d < date) existingLots + nextNum = M.size sameDate + 1 :: Int + +-- Transaction dispatch + +-- | Process a single transaction: transform its acquire, dispose, and transfer postings. +-- Transfer pairs are processed first (so that transferred lots are available for +-- subsequent disposals in the same transaction), then acquire and dispose postings. +-- Accumulates (LotState, [Transaction]) — transactions in reverse order. +processTransaction :: Bool -> Journal -> S.Set (CommoditySymbol, Day) -> (LotState, [Transaction]) -> Transaction + -> Either String (LotState, [Transaction]) +processTransaction verbosetags j needsLabels (ls, acc) t = do + -- Partition postings into transfer pairs and others + let (transferFroms, transferTos, otherPs) = partitionTransferPostings (tpostings t) + hasEquityOther = any (isEquityPosting j) otherPs + -- Closing equity transfer: transfer-from postings with no transfer-to counterpart, + -- where an equity posting receives the lots (e.g. close --clopen --lots). + -- Reduce lots from state; pass all postings through unchanged (equity does not track lots). + if not (null transferFroms) && null transferTos && hasEquityOther + then do + ls' <- foldM (reduceLotTransferToEquity j t) ls transferFroms + return (ls', t : acc) + -- Opening equity transfer: transfer-to postings with no transfer-from counterpart, + -- where an equity posting is the source (e.g. opening balances from close --clopen --lots). + -- Process transfer-to postings as acquires to add lots to the state. + else if null transferFroms && not (null transferTos) && hasEquityOther + then do + -- Build map of processed transfer-to postings, then reconstruct in original order. + let indexedTos = [(i, p) | (i, p) <- zip [0..] (tpostings t), isTransferToPosting p] + (ls', toMap) <- foldM (\(st, m) (i, p) -> do + (st', p') <- processAcquirePosting needsLabels txnDate t st p + return (st', M.insert i p' m) + ) (ls, M.empty) indexedTos + let allPs = [maybe p id (M.lookup i toMap) | (i, p) <- zip [0..] (tpostings t)] + return (ls', t{tpostings = allPs} : acc) + else do + let indexedFroms = [(i, p) | (i, p) <- zip [0..] (tpostings t), isTransferFromPosting p] + indexedTos = [(i, p) | (i, p) <- zip [0..] (tpostings t), isTransferToPosting p] + pairs <- pairIndexedTransferPostings t indexedFroms indexedTos + -- Process transfer pairs first, building an IntMap from original index to expanded postings. + (ls', transferMap) <- foldM processOnePair (ls, M.empty) pairs + -- Walk all postings in original order, substituting expanded results. + (ls'', allPs) <- foldMPostings ls' [] (zip [0..] (tpostings t)) transferMap + return (ls'', t{tpostings = reverse allPs} : acc) + where + txnDate = tdate t + + -- Process a transfer pair; record expanded postings keyed by original index. + processOnePair (st, m) (fromIdx, fromP, toIdx, toP) = do + (st', fromPs, toPs) <- processTransferPair verbosetags j t st fromP toP + let m' = M.insert fromIdx fromPs $ M.insert toIdx toPs m + return (st', m') + + -- Walk postings in original order, looking up transfer results or processing normally. + foldMPostings :: LotState -> [Posting] -> [(Int, Posting)] -> M.Map Int [Posting] + -> Either String (LotState, [Posting]) + foldMPostings st acc' [] _ = Right (st, acc') + foldMPostings st acc' ((i,p):ps) tmap + | Just expanded <- M.lookup i tmap = + foldMPostings st (reverse expanded ++ acc') ps tmap + | isAcquirePosting p = do + (st', p') <- processAcquirePosting needsLabels txnDate t st p + foldMPostings st' (p':acc') ps tmap + | isDisposePosting p = do + (st', newPs) <- processDisposePosting verbosetags j t st p + foldMPostings st' (reverse newPs ++ acc') ps tmap + | isUnclassifiedLotfulPosting j p = + Left (unclassifiedLotWarning j t i p) + | otherwise = + foldMPostings st (p:acc') ps tmap + +-- | True if the posting is in an equity account. +isEquityPosting :: Journal -> Posting -> Bool +isEquityPosting j p = maybe False isEquityType (journalAccountType j (lotBaseAccount (paccount p))) + +-- | Reduce lots from the lot state for a transfer-from posting going to an equity account. +-- Used when lots are transferred to equity (e.g. close --clopen --lots): reduces the lots +-- without requiring a matching transfer-to posting, since equity does not track lots. +reduceLotTransferToEquity :: Journal -> Transaction -> LotState -> Posting -> Either String LotState +reduceLotTransferToEquity j t ls p = + case [(a, cb) | a <- amountsRaw (pamount p), Just cb <- [acostbasis a], isNegativeAmount a] of + [(a, cb)] -> do + let commodity = acommodity a + qty = negate (aquantity a) + acct = lotBaseAccount (paccount p) + (method, methodSource) = resolveReductionMethodWithSource j p commodity + selected <- first (enrichLotError method methodSource acct commodity (tdate t) (journalFilePaths j)) + $ selectLots method (postingErrPrefix p) acct commodity qty cb ls + let consumed = [(lotId, qty') | (lotId, _, qty') <- selected] + return $ lotDbg t ("equity-transfer " ++ show qty ++ " " ++ T.unpack commodity + ++ " from " ++ T.unpack acct + ++ " (lots: " ++ showSelectedLots selected ++ ")") + $ reduceLotState acct commodity consumed ls + _ -> Right ls -- no single lot amount (e.g. cash posting): pass through + +-- | Partition a transaction's postings into transfer-from, transfer-to, and others. +partitionTransferPostings :: [Posting] -> ([Posting], [Posting], [Posting]) +partitionTransferPostings = go [] [] [] + where + go froms tos others [] = (reverse froms, reverse tos, reverse others) + go froms tos others (p:ps) + | isTransferFromPosting p = go (p:froms) tos others ps + | isTransferToPosting p = go froms (p:tos) others ps + | otherwise = go froms tos (p:others) ps + +-- | Pair indexed transfer-from and transfer-to postings by commodity. +-- Within each commodity group, froms and tos are sorted by cost basis fields +-- (date, label, cost) so that explicit per-lot pairs align correctly even when +-- interleaved. Cost basis mismatches are caught later by validation, not here. +-- Returns (fromIndex, fromPosting, toIndex, toPosting) tuples. +pairIndexedTransferPostings :: Transaction -> [(Int, Posting)] -> [(Int, Posting)] + -> Either String [(Int, Posting, Int, Posting)] +pairIndexedTransferPostings _ [] [] = Right [] +pairIndexedTransferPostings t froms tos = do + fromGroups <- groupByCommodity "transfer-from" froms + toGroups <- groupByCommodity "transfer-to" tos + let allComms = S.union (M.keysSet fromGroups) (M.keysSet toGroups) + concat <$> mapM (matchCommodityGroup fromGroups toGroups) (S.toList allComms) + where + showPos = txnErrPrefix t + + -- Group indexed postings by their lotful commodity. + groupByCommodity :: String -> [(Int, Posting)] -> Either String (M.Map CommoditySymbol [(Int, Posting)]) + groupByCommodity label ips = do + tagged <- mapM (\ip -> (,ip) <$> postingCommodity label (snd ip)) ips + Right $ M.map reverse $ M.fromListWith (++) [(c, [ip]) | (c, ip) <- tagged] + + postingCommodity :: String -> Posting -> Either String CommoditySymbol + postingCommodity label p = + case [acommodity a | a <- amountsRaw (pamount p), isJust (acostbasis a)] of + [c] -> Right c + -- Transfer-to postings without {} have no cost basis; use the raw commodity. + _ -> case [acommodity a | a <- amountsRaw (pamount p)] of + [c] -> Right c + _ -> Left $ showPos ++ label ++ " posting has no lotful commodity" + + -- Sort key for aligning pairs within a commodity group. + postingSortKey :: (Int, Posting) -> (Maybe Day, Maybe T.Text, Maybe (CommoditySymbol, Quantity)) + postingSortKey (_, p) = + case [cb | a <- amountsRaw (pamount p), Just cb <- [acostbasis a]] of + [cb] -> (cbDate cb, cbLabel cb, + fmap (\a -> (acommodity a, aquantity a)) (cbCost cb)) + _ -> (Nothing, Nothing, Nothing) + + matchCommodityGroup fromGroups toGroups comm = do + let fs = M.findWithDefault [] comm fromGroups + ts = M.findWithDefault [] comm toGroups + case (fs, ts) of + ([], _) -> Left $ showPos ++ "transfer-to posting for " ++ T.unpack comm + ++ " has no matching transfer-from posting" + (_, []) -> Left $ showPos ++ "transfer-from posting for " ++ T.unpack comm + ++ " has no matching transfer-to posting" + _ -> do + -- when (length fs /= length ts) $ + -- Left $ showPos ++ "mismatched transfer postings for commodity " ++ T.unpack comm + -- ++ ": " ++ show (length fs) ++ " transfer-from but " + -- ++ show (length ts) ++ " transfer-to" + let sortedFs = sortOn postingSortKey fs + sortedTs = sortOn postingSortKey ts + Right [(fi, fp, ti, tp) | ((fi, fp), (ti, tp)) <- zip sortedFs sortedTs] + +-- | Extract a per-unit cost Amount from an AmountCost, normalising TotalCost by quantity. +-- If quantity is zero, returns the TotalCost amount as-is (avoiding division by zero). +amountCostToUnitCost :: Quantity -> AmountCost -> Amount +amountCostToUnitCost _ (UnitCost c) = c +amountCostToUnitCost qty (TotalCost c) + | qty == 0 = c + | otherwise = c{aquantity = aquantity c / qty} + +-- | Normalize an amount's transacted cost to UnitCost form (converting TotalCost by dividing by quantity). +-- Returns Nothing if the amount has no transacted cost. +amountNormalizeCostToUnit :: Amount -> Maybe AmountCost +amountNormalizeCostToUnit a = fmap (UnitCost . amountCostToUnitCost (aquantity a)) (acost a) + +-- | Set the quantity of an amount matching the given commodity; leave others unchanged. +amountSetQuantityOf :: CommoditySymbol -> Quantity -> Amount -> Amount +amountSetQuantityOf c q a + | acommodity a == c = amountSetQuantity q a + | otherwise = a + +-- Per-type posting processing + +-- | Process a single acquire posting: generate a lot name and append it as a subaccount. +processAcquirePosting :: S.Set (CommoditySymbol, Day) -> Day -> Transaction -> LotState -> Posting + -> Either String (LotState, Posting) +processAcquirePosting needsLabels txnDate t lotState p = do + let lotAmts = [(a, cb) | a <- amountsRaw (pamount p), Just cb <- [acostbasis a]] + (lotAmt, cb, isBare) <- case lotAmts of + [x] -> Right (fst x, snd x, False) + _ -> do + let bareAmts = [a | a <- amountsRaw (pamount p), not (isNegativeAmount a)] + case bareAmts of + [a] -> Right (a, CostBasis Nothing Nothing Nothing, True) + _ -> Left $ showPos ++ "acquire posting has no cost basis" + + let commodity = acommodity lotAmt + date = fromMaybe txnDate (cbDate cb) + + -- Get the original (pre-balancing) amount to check for explicit transacted price. + -- A unit cost on the original can be used directly as the cost basis; + -- a total cost on the balanced amount is normalised to a per-unit cost basis. + let origAmt = case poriginal p of + Just orig -> case [a | a <- amountsRaw (pamount orig), acommodity a == commodity] of + (a:_) -> a + [] -> lotAmt + Nothing -> lotAmt + + let maybeLotBasis = case cbCost cb of + Just c -> Just c + Nothing + | Just (UnitCost c) <- acost origAmt -> Just c + | Just cost <- acost lotAmt -> Just $ amountCostToUnitCost (aquantity lotAmt) cost + | otherwise -> Nothing + + case maybeLotBasis of + Nothing | isBare -> Left $ showPos ++ T.unpack commodity + ++ " is lotful but this acquire posting has no cost basis or price.\n" + ++ "No lot will be created." + | otherwise -> Left $ showPos ++ "acquire posting has no lot cost" + Just lotBasis -> do + let cbInferred = isNothing (cbCost cb) + needsLabel = S.member (commodity, date) needsLabels + lotLabel' = cbLabel cb <|> if needsLabel then Just (generateLabel commodity date lotState) else Nothing + -- If the lot id already exists (e.g. an equity transfer-to on the same date + -- as a regular acquire, not predicted by findDatesNeedingLabels), auto-generate + -- a label to disambiguate. + existingLots = M.findWithDefault M.empty commodity lotState + lotId0 = LotId date lotLabel' + (lotId, lotLabel'') + | isNothing (cbLabel cb) && M.member lotId0 existingLots + = let l = generateLabel commodity date lotState + in (LotId date (Just l), Just l) + | otherwise = (lotId0, lotLabel') + fullCb = CostBasis{cbDate = Just date, cbLabel = lotLabel'', cbCost = Just lotBasis} + lotName = showLotName fullCb + -- When cost basis was inferred, fill it in on the user's original cb + -- so that print shows {$50} not {}. + filledCb = cb{cbCost = Just lotBasis} + -- The lot state always stores the full cost basis (with date/label/cost) + -- so that disposal selectors like {2026-01-15, $50} can match. + lotStateAmt = lotAmt{acostbasis = Just fullCb} + -- The posting amount preserves the user's original cost basis fields + -- (only filling in cost when inferred) for print output fidelity. + postingAmt = if cbInferred then lotAmt{acostbasis = Just filledCb} else lotAmt + + let baseAcct = lotBaseAccount (paccount p) + hasExplicitLotAcct = baseAcct /= paccount p + expectedAcct = baseAcct <> ":" <> lotName + + -- If the user wrote an explicit lot subaccount, check it matches the resolved lot. + when (hasExplicitLotAcct && paccount p /= expectedAcct) $ + Left $ showPos ++ "lot subaccount " ++ T.unpack (paccount p) + ++ " does not match the resolved lot " ++ T.unpack expectedAcct + + when (M.member lotId existingLots) $ + Left $ showPos ++ "duplicate lot id: " ++ T.unpack lotName + ++ " for commodity " ++ T.unpack commodity + + let -- For bare acquires with inferred CB, normalize transacted cost to UnitCost + -- in the posting's pamount (not in poriginal — that preserves what the user wrote). + p' = p{paccount = expectedAcct + ,pamount = mixedAmount $ if isBare && cbInferred then postingAmt{acost = amountNormalizeCostToUnit lotAmt} else postingAmt + ,poriginal = Just (originalPosting p)} + let lotState' = addLotState commodity lotId baseAcct lotStateAmt lotState + return $ lotDbg t ("acquired " ++ show (aquantity lotAmt) ++ " " + ++ T.unpack commodity ++ " " ++ T.unpack lotName + ++ " on " ++ T.unpack baseAcct) + (lotState', p') + where + showPos = txnErrPrefix t + +-- | Process a dispose posting: match to existing lots using the resolved reduction method, +-- split into multiple postings if the disposal spans multiple lots. +-- Returns the list of resulting postings (one per matched lot). +processDisposePosting :: Bool -> Journal -> Transaction -> LotState -> Posting + -> Either String (LotState, [Posting]) +processDisposePosting verbosetags j t lotState p = do + -- Extract lotful amount and lot selector. When cost basis is present, use it directly. + -- When absent (bare dispose on a lotful commodity), use a wildcard selector. + let lotAmts = [(a, cb) | a <- amountsRaw (pamount p), Just cb <- [acostbasis a]] + (lotAmt, cb, isBare) <- case lotAmts of + [x] -> Right (fst x, snd x, False) + _ -> do + let bareAmts = [a | a <- amountsRaw (pamount p), isNegativeAmount a] + case bareAmts of + [a] -> Right (a, CostBasis Nothing Nothing Nothing, True) + _ -> Left $ showPos ++ "dispose posting has no cost basis" + + let commodity = acommodity lotAmt + disposeQty = aquantity lotAmt + + -- Non-bare dispose (explicit {}) without price is an error. + -- Bare dispose without price proceeds to lot matching (but skips gain generation). + case acost lotAmt of + Nothing | not isBare -> Left $ showPos ++ "dispose posting has no transacted price (selling price) for " ++ T.unpack commodity + _ -> do + + when (disposeQty >= 0) $ + Left $ showPos ++ "dispose posting has non-negative quantity for " ++ T.unpack commodity + + let posQty = negate disposeQty + (method, methodSource) = resolveReductionMethodWithSource j p commodity + -- All methods are per-account, scoped to the posting's base account + -- (stripping any explicit lot subaccount the user may have written). + scopeAcct = lotBaseAccount (paccount p) + + when (isBare && method == SPECID) $ + Left $ showPos ++ "SPECID requires a lot selector on dispose postings" + + selected <- first (enrichLotError method methodSource scopeAcct commodity (tdate t) (journalFilePaths j)) + $ selectLots method (postingErrPrefix p) scopeAcct commodity posQty cb lotState + + -- For AVERAGE methods, compute the weighted average cost across the pool. + -- AVERAGEALL uses the global pool (all accounts); AVERAGE uses per-account scope. + mavgCost <- if methodIsAverage method + then do + let allLots = M.findWithDefault M.empty commodity lotState + flatLots = if methodIsGlobal method + then flattenAllAccountLots allLots + else M.mapMaybe (M.lookup scopeAcct) allLots + fmap Just $ poolWeightedAvgCost showPos flatLots + else Right Nothing + + let baseAcct = lotBaseAccount (paccount p) + hasExplicitLotAcct = baseAcct /= paccount p + mkPosting (lotId, storedAmt, consumedQty) = do + -- The original lot's cost basis is always needed for the lot subaccount name. + origBasis <- case acostbasis storedAmt >>= cbCost of + Just c -> Right c + Nothing -> Left $ showPos ++ "lot " ++ T.unpack (T.pack (show lotId)) + ++ " for commodity " ++ T.unpack commodity + ++ " has no cost basis (internal error)" + -- For AVERAGE methods, use the weighted average cost for the disposal amount. + -- For all other methods, use the original lot's cost. + let disposalBasis = fromMaybe origBasis mavgCost + -- Lot name uses the original cost (so the subaccount matches the acquisition). + lotCb = CostBasis + { cbDate = Just (lotDate lotId) + , cbLabel = lotLabel lotId + , cbCost = Just origBasis + } + -- Disposal cost basis uses average cost when applicable. + dispCb = lotCb{cbCost = Just disposalBasis} + lotName = showLotName lotCb + expectedAcct = baseAcct <> ":" <> lotName + -- If the user wrote an explicit lot subaccount, check it matches the resolved lot. + when (hasExplicitLotAcct && paccount p /= expectedAcct) $ + Left $ showPos ++ "lot subaccount " ++ T.unpack (paccount p) + ++ " does not match the resolved lot " ++ T.unpack expectedAcct + let acctWithLot = expectedAcct + -- Build the dispose amount: negative consumed quantity, + -- keeping the original amount's commodity, style, cost, and cost basis. + disposeAmt = (amountSetQuantity (negate consumedQty) lotAmt){acostbasis = Just dispCb} + let -- For bare disposes with a price, normalize transacted cost to UnitCost. + -- For bare disposes without a price (e.g. fee deductions), keep no cost. + disposeAmt' | isNothing (acost lotAmt) = disposeAmt{acost = Nothing} + | isBare = disposeAmt{acost = amountNormalizeCostToUnit lotAmt} + | otherwise = disposeAmt + -- poriginal preserves the user's original annotations, only updating quantity. + origP = originalPosting p + origP' = origP{pamount = mapMixedAmount (amountSetQuantityOf commodity (negate consumedQty)) $ pamount origP} + Right p{ paccount = acctWithLot + , pamount = mixedAmount disposeAmt' + , poriginal = Just origP' + } + + newPostings <- mapM mkPosting selected + let consumed = [(lotId, qty) | (lotId, _, qty) <- selected] + lotState' = reduceLotState scopeAcct commodity consumed lotState + + return $ lotDbg t ("disposed " ++ show posQty ++ " " ++ T.unpack commodity + ++ " from " ++ T.unpack scopeAcct + ++ " (" ++ show method ++ ", lots: " ++ showSelectedLots selected ++ ")") + (lotState', preserveParentAssertion verbosetags (paccount p) (pbalanceassertion p) newPostings) + where + showPos = txnErrPrefix t + +-- | Process a transfer pair: select lots from the source account (transfer-from) +-- and recreate them under the destination account (transfer-to). +-- Returns updated LotState and two lists of expanded postings (from, to). +processTransferPair :: Bool -> Journal -> Transaction -> LotState -> Posting -> Posting + -> Either String (LotState, [Posting], [Posting]) +processTransferPair verbosetags j t lotState fromP toP = do + -- Extract lotful amount and lot selector from transfer-from. + -- When cost basis is present, use it directly as the lot selector. + -- When absent (bare transfer on a lotful commodity), use a wildcard selector. + let fromAmts = [(a, cb) | a <- amountsRaw (pamount fromP), Just cb <- [acostbasis a]] + (fromAmt, fromCb) <- case fromAmts of + [x] -> Right x + _ -> do + let bareAmts = [a | a <- amountsRaw (pamount fromP), isNegativeAmount a] + case bareAmts of + [a] -> Right (a, CostBasis Nothing Nothing Nothing) + _ -> Left $ showPos ++ "transfer-from posting has no cost basis" + + let commodity = acommodity fromAmt + transferQty = aquantity fromAmt + + let toAmts = amountsRaw (pamount toP) + + -- Check that neither transfer posting has explicit transacted price (@ or @@). + -- Use originalPosting to distinguish user-written @ from pipeline-inferred acost. + let origFromAmts = amountsRaw $ pamount $ originalPosting fromP + origToAmts = amountsRaw $ pamount $ originalPosting toP + when (any (isJust . acost) origFromAmts || any (isJust . acost) origToAmts) $ + Left $ showPos ++ "lot transfers should have no transacted price" + + -- Validate transfer-from has negative quantity + when (transferQty >= 0) $ + Left $ showPos ++ "transfer-from posting has non-negative quantity for " ++ T.unpack commodity + + let fromQty = negate transferQty + -- Detect fee: if transfer-to has less qty than transfer-from, the difference is a fee. + toQty = case [aquantity a | a <- toAmts, acommodity a == commodity, aquantity a > 0] of + [q] | q < fromQty -> q + _ -> fromQty + feeQty = fromQty - toQty + -- Transfers are always per-account (scoped to source), but ordering follows the method. + (method, methodSource) = resolveReductionMethodWithSource j fromP commodity + fromBaseAcct = lotBaseAccount (paccount fromP) + + -- Select lots from source account for the full fromQty + selected <- first (enrichLotError method methodSource fromBaseAcct commodity (tdate t) (journalFilePaths j)) + $ selectLots method (postingErrPrefix fromP) fromBaseAcct commodity fromQty fromCb lotState + + -- Split selected lots into transfer portion and fee portion + let (transferLots, feeLots) = if feeQty > 0 + then splitLotsAt toQty selected + else (selected, []) + + -- Extract the transfer-to cost basis for optional validation + let toCb = case [(a, cb) | a <- toAmts, Just cb <- [acostbasis a]] of + [(_, cb)] -> Just cb + _ -> Nothing + + -- For each transfer lot, generate from and to postings + (transferFromPs, toPs) <- fmap unzip $ mapM (mkTransferPostings fromAmt toCb commodity) transferLots + + -- For fee lots, generate from-only postings (lots consumed from source, no destination) + feeFromPs <- mapM (mkFeeFromPosting fromAmt commodity) feeLots + + -- Update lot state: remove full fromQty from source, add only transfer portion to destination + let toBaseAcct = lotBaseAccount (paccount toP) + consumed = [(lotId, qty) | (lotId, _, qty) <- selected] + lotState' = reduceLotState fromBaseAcct commodity consumed lotState + lotState'' = foldl' (addTransferredLot commodity toBaseAcct) lotState' transferLots + + return $ lotDbg t ("transferred " ++ show fromQty ++ " " ++ T.unpack commodity + ++ " from " ++ T.unpack fromBaseAcct ++ " to " ++ T.unpack toBaseAcct + ++ " (lots: " ++ showSelectedLots selected + ++ if feeQty > 0 then "; fee: " ++ show feeQty ++ ")" else ")") + ( lotState'' + , preserveParentAssertion verbosetags (paccount fromP) (pbalanceassertion fromP) (transferFromPs ++ feeFromPs) + , preserveParentAssertion verbosetags (paccount toP) (pbalanceassertion toP) toPs + ) + where + showPos = txnErrPrefix t + + mkTransferPostings fromAmt toCb commodity (lotId, storedAmt, consumedQty) = do + lotBasis <- case acostbasis storedAmt >>= cbCost of + Just c -> Right c + Nothing -> Left $ showPos ++ "lot " ++ show lotId + ++ " for commodity " ++ T.unpack commodity + ++ " has no cost basis (internal error)" + let lotCb = CostBasis + { cbDate = Just (lotDate lotId) + , cbLabel = lotLabel lotId + , cbCost = Just lotBasis + } + lotName = showLotName lotCb + -- Use base accounts to avoid double-appending lot subaccounts. + fromAcct = lotBaseAccount (paccount fromP) <> ":" <> lotName + toAcct = lotBaseAccount (paccount toP) <> ":" <> lotName + + -- Validate transfer-to cost basis if it has specific fields + validateToCb toCb lotCb commodity + + let fromAmt' = (amountSetQuantity (negate consumedQty) fromAmt){acostbasis = Just lotCb} + toAmt' = amountSetQuantity consumedQty fromAmt' + -- poriginal preserves the user's original annotations, only updating quantity. + origFromP = originalPosting fromP + origFromP' = origFromP{pamount = mapMixedAmount (amountSetQuantityOf commodity (negate consumedQty)) (pamount origFromP)} + origToP = originalPosting toP + origToP' = origToP{pamount = mapMixedAmount (amountSetQuantityOf commodity consumedQty) (pamount origToP)} + fromP' = fromP{ paccount = fromAcct + , pamount = mixedAmount fromAmt' + , poriginal = Just origFromP' + } + toP' = toP{ paccount = toAcct + , pamount = mixedAmount toAmt' + , poriginal = Just origToP' + } + Right (fromP', toP') + + -- Generate a from-only posting for a fee-consumed lot (no destination). + mkFeeFromPosting fromAmt commodity (lotId, storedAmt, consumedQty) = do + lotBasis <- case acostbasis storedAmt >>= cbCost of + Just c -> Right c + Nothing -> Left $ showPos ++ "lot " ++ show lotId + ++ " for commodity " ++ T.unpack commodity + ++ " has no cost basis (internal error)" + let lotCb = CostBasis + { cbDate = Just (lotDate lotId) + , cbLabel = lotLabel lotId + , cbCost = Just lotBasis + } + lotName = showLotName lotCb + fromAcct = lotBaseAccount (paccount fromP) <> ":" <> lotName + fromAmt' = (amountSetQuantity (negate consumedQty) fromAmt){acostbasis = Just lotCb, acost = Nothing} + origFromP = originalPosting fromP + origFromP' = origFromP{pamount = mapMixedAmount (amountSetQuantityOf commodity (negate consumedQty)) (pamount origFromP)} + Right fromP{ paccount = fromAcct + , pamount = mixedAmount fromAmt' + , poriginal = Just origFromP' + } + + -- Validate that transfer-to cost basis (if specified with concrete fields) + -- matches the lot's cost basis. + validateToCb Nothing _ _ = Right () + validateToCb (Just toCb') lotCb commodity = do + case cbCost toCb' of + Just toCost | Just lotBasis <- cbCost lotCb -> + when (acommodity toCost /= acommodity lotBasis || aquantity toCost /= aquantity lotBasis) $ + Left $ showPos ++ "lot cost basis " ++ T.unpack (showLotName lotCb) + ++ " does not match transfer-to cost basis " ++ T.unpack (showLotName toCb') + ++ " for commodity " ++ T.unpack commodity + _ -> Right () + + -- Re-add a transferred lot to LotState under the destination account. + addTransferredLot commodity destAcct ls (lotId, storedAmt, consumedQty) = + let amt = storedAmt{aquantity = consumedQty} + in addLotState commodity lotId destAcct amt ls + +-- Lot state operations + +-- | Add an amount to LotState for a specific account/commodity/lot. +-- If the lot already exists on that account, quantities are summed (not overwritten). +-- This can happen when the same lot is transferred to the same destination account +-- by multiple transactions (e.g. two transfers on the same date both move portions +-- of the same lot). +addLotState :: CommoditySymbol -> LotId -> AccountName -> Amount -> LotState -> LotState +addLotState commodity lotId account amt = + M.insertWith (M.unionWith (M.unionWith addQty)) commodity + (M.singleton lotId (M.singleton account amt)) + where addQty a1 a2 = a1{aquantity = aquantity a1 + aquantity a2} + +-- | Enrich a selectLots error with reduction method info and a review hint. +enrichLotError :: ReductionMethod -> String -> AccountName -> CommoditySymbol + -> Day -> [FilePath] -> String -> String +enrichLotError method methodSource account commodity txnDate files err = + err ++ "\nUsing " ++ show method ++ " (" ++ methodSource ++ ")." + ++ "\nTo review lot movements: hledger" + ++ concatMap (" -f " ++) files + ++ " reg " ++ T.unpack account ++ " cur:" ++ T.unpack commodity + ++ " --lots -e " ++ T.unpack (showDate (addDays 1 txnDate)) + ++ " --verbose-tags" + +-- | Select lots to consume using the given reduction method. +-- All methods select from the specified account only. +-- Ordering: FIFO\/FIFOALL oldest-first; LIFO\/LIFOALL newest-first; +-- HIFO\/HIFOALL highest per-unit cost first; AVERAGE\/AVERAGEALL FIFO order; +-- SPECID requires an explicit selector matching one lot. +-- The *ALL variants additionally validate that the selected lots would also be +-- chosen first if all accounts' lots were considered together (see 'validateGlobalCompliance'). +-- The lot selector filters which lots are eligible: each non-Nothing field +-- in the selector must match the corresponding field in the lot's cost basis. +-- An all-Nothing selector (from @{}@) matches all lots. +-- Returns a list of (lot id, lot amount, quantity consumed from this lot). +-- Errors if total available quantity in matching lots is insufficient. +selectLots :: ReductionMethod -> String -> AccountName -> CommoditySymbol + -> Quantity -> CostBasis -> LotState + -> Either String [(LotId, Amount, Quantity)] +selectLots method posStr account commodity qty selector lotState = do + when (method == SPECID && isWildcardSelector selector) $ + Left $ posStr ++ "SPECID requires an explicit lot selector" + let allLots = M.findWithDefault M.empty commodity lotState + -- Flatten to (LotId, Amount) pairs, taking only the specified account's balance. + flatLots = M.mapMaybe (M.lookup account) allLots + matchingLots = M.filter (lotMatchesSelector selector) flatLots + when (M.null matchingLots) $ + Left $ posStr ++ "no lots available for commodity " ++ T.unpack commodity + ++ " in account " ++ T.unpack account + ++ showOtherAccountLots allLots + when (method == SPECID && M.size matchingLots > 1) $ + Left $ posStr ++ "lot selector is ambiguous, matches " ++ show (M.size matchingLots) + ++ " lots in account " ++ T.unpack account ++ ":" + ++ showLotList matchingLots + let available = sum [aquantity a | a <- M.elems matchingLots] + when (available < qty) $ + Left $ posStr ++ "insufficient lots for commodity " ++ T.unpack commodity + ++ " in account " ++ T.unpack account + ++ ": need " ++ show qty ++ " but only " ++ show available ++ " available" + ++ "\nAvailable lots in this account:" ++ showLotList matchingLots + ++ showOtherAccountLots allLots + let base = methodBaseOrdering method + orderedLots = case base of + FIFO -> M.toAscList matchingLots + LIFO -> M.toDescList matchingLots + HIFO -> sortOn (Down . lotPerUnitCost) (M.toList matchingLots) + AVERAGE -> M.toAscList matchingLots + SPECID -> M.toAscList matchingLots + _ -> M.toAscList matchingLots -- unreachable after methodBaseOrdering + selected = go qty orderedLots + when (methodIsGlobal method) $ + validateGlobalCompliance method posStr account commodity qty selector lotState selected + Right selected + where + go 0 _ = [] + go _ [] = [] -- shouldn't happen after the check above + go remaining ((lotId, lotAmt):rest) + | remaining >= lotBal = (lotId, lotAmt, lotBal) : go (remaining - lotBal) rest + | otherwise = [(lotId, lotAmt, remaining)] + where lotBal = aquantity lotAmt + + showLotList :: M.Map LotId Amount -> String + showLotList lots = concatMap fmt (M.toAscList lots) + where fmt (lid, a) = "\n " ++ T.unpack (showLotName (lotIdToCb lid a)) + ++ " " ++ show (aquantity a) + + showOtherAccountLots :: M.Map LotId (M.Map AccountName Amount) -> String + showOtherAccountLots allLots' = + let others = [(acct, lid, a) | (lid, acctMap) <- M.toAscList allLots' + , (acct, a) <- M.toList acctMap, acct /= account] + byAcct = M.fromListWith (++) [(acct, [(lid, a)]) | (acct, lid, a) <- others] + in if M.null byAcct then "" + else "\nLots of " ++ T.unpack commodity ++ " in other accounts:" + ++ concatMap fmtAcct (M.toAscList byAcct) + where fmtAcct (acct, lots) = "\n " ++ T.unpack acct ++ ": " + ++ intercalate ", " [T.unpack (showLotName (lotIdToCb lid a)) ++ " " ++ show (aquantity a) + | (lid, a) <- lots] + +-- | Split a list of selected lots at a quantity boundary. +-- Returns (lots for the first portion, lots for the remainder). +-- Used to separate transfer and fee portions when transfer qty < source qty. +splitLotsAt :: Quantity -> [(LotId, Amount, Quantity)] + -> ([(LotId, Amount, Quantity)], [(LotId, Amount, Quantity)]) +splitLotsAt 0 lots = ([], lots) +splitLotsAt _ [] = ([], []) +splitLotsAt remaining ((lid, amt, qty):rest) + | remaining >= qty = let (a, b) = splitLotsAt (remaining - qty) rest + in ((lid, amt, qty):a, b) + | otherwise = ([(lid, amt, remaining)], (lid, amt, qty - remaining):rest) + +-- | Extract the per-unit cost quantity from a lot entry, for HIFO sorting. +lotPerUnitCost :: (LotId, Amount) -> Quantity +lotPerUnitCost (_, a) = maybe 0 aquantity (acostbasis a >>= cbCost) + +-- | Whether a reduction method uses weighted average cost basis for disposals. +methodIsAverage :: ReductionMethod -> Bool +methodIsAverage AVERAGE = True +methodIsAverage AVERAGEALL = True +methodIsAverage _ = False + +-- | Whether a reduction method requires global validation across all accounts. +methodIsGlobal :: ReductionMethod -> Bool +methodIsGlobal FIFOALL = True +methodIsGlobal LIFOALL = True +methodIsGlobal HIFOALL = True +methodIsGlobal AVERAGEALL = True +methodIsGlobal _ = False + +-- | Map a *ALL method to its base ordering, or return the method unchanged. +methodBaseOrdering :: ReductionMethod -> ReductionMethod +methodBaseOrdering FIFOALL = FIFO +methodBaseOrdering LIFOALL = LIFO +methodBaseOrdering HIFOALL = HIFO +methodBaseOrdering AVERAGEALL = AVERAGE +methodBaseOrdering m = m + +-- | Flatten lots across all accounts, summing quantities for shared lot IDs. +-- From @Map LotId (Map AccountName Amount)@ to @Map LotId Amount@, +-- combining quantities across accounts (taking the first Amount's metadata). +flattenAllAccountLots :: M.Map LotId (M.Map AccountName Amount) -> M.Map LotId Amount +flattenAllAccountLots = M.mapMaybe flattenAccts + where + flattenAccts acctMap = + case M.elems acctMap of + [] -> Nothing + (a:rest) -> Just a{aquantity = aquantity a + sum (map aquantity rest)} + +-- | Validate that the per-account selected lots would also be chosen first +-- under a global ordering across all accounts. Errors if lots on other accounts +-- have higher priority than the selected lots. +validateGlobalCompliance :: ReductionMethod -> String -> AccountName -> CommoditySymbol + -> Quantity -> CostBasis -> LotState + -> [(LotId, Amount, Quantity)] -> Either String () +validateGlobalCompliance method posStr account commodity qty selector lotState selected = do + let allLots = M.findWithDefault M.empty commodity lotState + globalFlat = flattenAllAccountLots allLots + globalMatching = M.filter (lotMatchesSelector selector) globalFlat + base = methodBaseOrdering method + globalOrdered = case base of + FIFO -> M.toAscList globalMatching + LIFO -> M.toDescList globalMatching + HIFO -> sortOn (Down . lotPerUnitCost) (M.toList globalMatching) + AVERAGE -> M.toAscList globalMatching + _ -> M.toAscList globalMatching + -- Greedily consume qty from the globally-ordered list + globalSelectedIds = S.fromList $ map fst3 $ goConsume qty globalOrdered + selectedIds = S.fromList [lid | (lid, _, _) <- selected] + -- Lot IDs that would be globally selected but are NOT in the per-account selection + -- (i.e. they exist on other accounts and have higher priority) + higherPriorityElsewhere = S.difference globalSelectedIds selectedIds + unless (S.null higherPriorityElsewhere) $ do + -- Build detailed error showing which accounts hold the higher-priority lots + let otherLots = [(acct, lid, a) + | lid <- S.toList higherPriorityElsewhere + , Just acctMap <- [M.lookup lid allLots] + , (acct, a) <- M.toList acctMap] + byAcct = M.fromListWith (++) [(acct, [(lid, a)]) | (acct, lid, a) <- otherLots] + fmtAcct (acct, lots) = "\n " ++ T.unpack acct ++ ": " + ++ intercalate ", " [T.unpack (showLotName (lotIdToCb lid a)) ++ " " ++ show (aquantity a) + | (lid, a) <- lots] + fmtSelected = concatMap (\(lid, a, q) -> "\n " ++ T.unpack (showLotName (lotIdToCb lid a)) + ++ " " ++ show q) selected + Left $ posStr ++ show method ++ ": lot(s) on other account(s) have higher priority than the lots in " + ++ T.unpack account ++ ":" + ++ concatMap fmtAcct (M.toAscList byAcct) + ++ "\nSelected from " ++ T.unpack account ++ ":" + ++ fmtSelected + ++ "\nConsider disposing from the account(s) listed above first, or use " + ++ show (methodBaseOrdering method) ++ " for per-account scope." + where + goConsume 0 _ = [] + goConsume _ [] = [] + goConsume remaining ((lotId, lotAmt):rest) + | remaining >= aquantity lotAmt = (lotId, lotAmt, aquantity lotAmt) : goConsume (remaining - aquantity lotAmt) rest + | otherwise = [(lotId, lotAmt, remaining)] + fst3 (x, _, _) = x + +-- | Compute the weighted average per-unit cost across all lots in a pool. +-- All lots must have cost basis in the same commodity. +-- Returns a representative cost Amount with the weighted average quantity. +poolWeightedAvgCost :: String -> M.Map LotId Amount -> Either String Amount +poolWeightedAvgCost posStr lots = do + let costs = [(aquantity a, c) | a <- M.elems lots, Just cb <- [acostbasis a], Just c <- [cbCost cb]] + case costs of + [] -> Left $ posStr ++ "no lots with cost basis available for averaging" + ((_, firstCost):rest) + | any (\(_, c) -> acommodity c /= acommodity firstCost) rest -> + Left $ posStr ++ "cannot average lots with different cost commodities" + | otherwise -> + let totalQty = sum [q | (q, _) <- costs] + totalCost = sum [q * aquantity c | (q, c) <- costs] + in if totalQty == 0 + then Left $ posStr ++ "cannot average lots with zero total quantity" + else Right firstCost{aquantity = totalCost / totalQty} + +-- | Is this an all-Nothing (wildcard) lot selector, i.e. from @{}@? +isWildcardSelector :: CostBasis -> Bool +isWildcardSelector (CostBasis Nothing Nothing Nothing) = True +isWildcardSelector _ = False + +-- | Does a lot match a lot selector? +-- Each non-Nothing field in the selector must match the lot's stored cost basis. +lotMatchesSelector :: CostBasis -> Amount -> Bool +lotMatchesSelector selector a = + case acostbasis a of + Nothing -> False + Just lotCb -> matchCost (cbCost selector) (cbCost lotCb) + && matchField cbDate selector lotCb + && matchField cbLabel selector lotCb + where + matchField :: Eq b => (CostBasis -> Maybe b) -> CostBasis -> CostBasis -> Bool + matchField f sel lot = case f sel of + Nothing -> True -- selector doesn't constrain this field + Just v -> f lot == Just v + -- Compare costs by commodity and quantity, ignoring style differences. + matchCost :: Maybe Amount -> Maybe Amount -> Bool + matchCost Nothing _ = True + matchCost (Just _) Nothing = False + matchCost (Just sel) (Just lot) = acommodity sel == acommodity lot + && aquantity sel == aquantity lot + +-- | Subtract consumed quantities from LotState for a specific account. +-- Removes lot-account entries whose balance reaches zero. +-- Removes the lot entirely if no accounts remain. +reduceLotState :: AccountName -> CommoditySymbol -> [(LotId, Quantity)] -> LotState -> LotState +reduceLotState account commodity consumed = M.adjust adjustCommodity commodity + where + adjustCommodity lots = foldl' reduceLot lots consumed + reduceLot lots (lotId, qty) = M.update shrinkLot lotId lots + where + shrinkLot acctMap = + let acctMap' = M.update (shrinkAmt qty) account acctMap + in if M.null acctMap' then Nothing else Just acctMap' + shrinkAmt q a + | aquantity a <= q = Nothing + | otherwise = Just a{aquantity = aquantity a - q} + +-- Debug trace helpers + +-- | Reconstruct a CostBasis from a LotId and a stored Amount (for display in trace messages). +lotIdToCb :: LotId -> Amount -> CostBasis +lotIdToCb lid a = CostBasis (Just (lotDate lid)) (lotLabel lid) (acostbasis a >>= cbCost) + +-- | Show selected lots for trace messages: "{2026-01-15, $50} 5, {2026-02-01, $60} 3" +showSelectedLots :: [(LotId, Amount, Quantity)] -> String +showSelectedLots = intercalate ", " . map fmt + where fmt (lid, a, qty) = T.unpack (showLotName (lotIdToCb lid a)) ++ " " ++ show qty diff --git a/hledger-lib/Hledger/Data/Posting.hs b/hledger-lib/Hledger/Data/Posting.hs index f8dd1f74810..583a20c1d65 100644 --- a/hledger-lib/Hledger/Data/Posting.hs +++ b/hledger-lib/Hledger/Data/Posting.hs @@ -53,6 +53,7 @@ module Hledger.Data.Posting ( commentAddTag, commentAddTagUnspaced, commentAddTagNextLine, + commentPrependTag, generatedTransactionTagName, modifiedTransactionTagName, generatedPostingTagName, @@ -151,7 +152,7 @@ nullposting = Posting ,paccount="" ,pamount=nullmixedamt ,pcomment="" - ,ptype=RegularPosting + ,preal=RealPosting ,ptags=[] ,pbalanceassertion=Nothing ,ptransaction=Nothing @@ -167,7 +168,7 @@ post acc amt = posting {paccount=acc, pamount=mixedAmount amt} -- | Make a virtual (unbalanced) posting to an account. vpost :: AccountName -> Amount -> Posting -vpost acc amt = (post acc amt){ptype=VirtualPosting} +vpost acc amt = (post acc amt){preal=VirtualPosting} -- | Make a posting to an account, maybe with a balance assertion. post' :: AccountName -> Amount -> Maybe BalanceAssertion -> Posting @@ -175,7 +176,7 @@ post' acc amt ass = posting {paccount=acc, pamount=mixedAmount amt, pbalanceasse -- | Make a virtual (unbalanced) posting to an account, maybe with a balance assertion. vpost' :: AccountName -> Amount -> Maybe BalanceAssertion -> Posting -vpost' acc amt ass = (post' acc amt ass){ptype=VirtualPosting, pbalanceassertion=ass} +vpost' acc amt ass = (post' acc amt ass){preal=VirtualPosting, pbalanceassertion=ass} nullassertion :: BalanceAssertion nullassertion = BalanceAssertion @@ -215,14 +216,14 @@ originalPosting :: Posting -> Posting originalPosting p = fromMaybe p $ poriginal p showPosting :: Posting -> String -showPosting p = T.unpack . T.unlines $ postingsAsLines False [p] +showPosting p = T.unpack . T.unlines $ postingsAsLines defaultFmt False [p] -- | Render a posting, at the appropriate width for aligning with -- its siblings if any. Used by the rewrite command. showPostingLines :: Posting -> [Text] -showPostingLines p = first3 $ postingAsLines False False maxacctwidth maxamtwidth p +showPostingLines p = first3 $ postingAsLines defaultFmt False False maxacctwidth maxamtwidth p where - linesWithWidths = map (postingAsLines False False maxacctwidth maxamtwidth) . maybe [p] tpostings $ ptransaction p + linesWithWidths = map (postingAsLines defaultFmt False False maxacctwidth maxamtwidth) . maybe [p] tpostings $ ptransaction p maxacctwidth = maximumBound 0 $ map second3 linesWithWidths maxamtwidth = maximumBound 0 $ map third3 linesWithWidths @@ -244,10 +245,10 @@ showPostingLines p = first3 $ postingAsLines False False maxacctwidth maxamtwidt -- Amounts' display precisions, which may have been limited by commodity directives, -- will be increased if necessary to ensure this. -- -postingsAsLines :: Bool -> [Posting] -> [Text] -postingsAsLines onelineamounts ps = concatMap first3 linesWithWidths +postingsAsLines :: AmountFormat -> Bool -> [Posting] -> [Text] +postingsAsLines basefmt onelineamounts ps = concatMap first3 linesWithWidths where - linesWithWidths = map (postingAsLines False onelineamounts maxacctwidth maxamtwidth) ps + linesWithWidths = map (postingAsLines basefmt False onelineamounts maxacctwidth maxamtwidth) ps maxacctwidth = maximumBound 0 $ map second3 linesWithWidths maxamtwidth = maximumBound 0 $ map third3 linesWithWidths @@ -274,8 +275,8 @@ postingsAsLines onelineamounts ps = concatMap first3 linesWithWidths -- increased if needed to match the posting with the longest account name. -- This is used to align the amounts of a transaction's postings. -- -postingAsLines :: Bool -> Bool -> Int -> Int -> Posting -> ([Text], Int, Int) -postingAsLines elideamount onelineamounts acctwidth amtwidth p = +postingAsLines :: AmountFormat -> Bool -> Bool -> Int -> Int -> Posting -> ([Text], Int, Int) +postingAsLines basefmt elideamount onelineamounts acctwidth amtwidth p = (concatMap (++ newlinecomments) postingblocks, thisacctwidth, thisamtwidth) where -- This needs to be converted to strict Text in order to strip trailing @@ -295,7 +296,7 @@ postingAsLines elideamount onelineamounts acctwidth amtwidth p = pad amt = WideBuilder (TB.fromText $ T.replicate w " ") w <> amt where w = max 12 amtwidth - wbWidth amt -- min. 12 for backwards compatibility - pacctstr p' = showAccountName Nothing (ptype p') (paccount p') + pacctstr p' = showAccountName Nothing (preal p') (paccount p') pstatusandacct p' = pstatusprefix p' <> pacctstr p' pstatusprefix p' = case pstatus p' of Unmarked -> "" @@ -308,7 +309,7 @@ postingAsLines elideamount onelineamounts acctwidth amtwidth p = shownAmounts | elideamount = [mempty] | otherwise = showMixedAmountLinesB displayopts $ pamount p - where displayopts = defaultFmt{ + where displayopts = basefmt{ displayZeroCommodity=True, displayForceDecimalMark=True, displayOneLine=onelineamounts } thisamtwidth = maximumBound 0 $ map wbWidth shownAmounts @@ -329,11 +330,11 @@ postingAsLines elideamount onelineamounts acctwidth amtwidth p = c:cs -> (c,cs) -- | Show an account name, clipped to the given width if any, and --- appropriately bracketed/parenthesised for the given posting type. -showAccountName :: Maybe Int -> PostingType -> AccountName -> Text +-- appropriately bracketed/parenthesised for the given posting realness. +showAccountName :: Maybe Int -> PostingRealness -> AccountName -> Text showAccountName w = fmt where - fmt RegularPosting = maybe id T.take w + fmt RealPosting = maybe id T.take w fmt VirtualPosting = wrap "(" ")" . maybe id (T.takeEnd . subtract 2) w fmt BalancedVirtualPosting = wrap "[" "]" . maybe id (T.takeEnd . subtract 2) w @@ -359,13 +360,13 @@ commentSpace = (" "<>) isReal :: Posting -> Bool -isReal p = ptype p == RegularPosting +isReal p = preal p == RealPosting isVirtual :: Posting -> Bool -isVirtual p = ptype p == VirtualPosting +isVirtual p = preal p == VirtualPosting isBalancedVirtual :: Posting -> Bool -isBalancedVirtual p = ptype p == BalancedVirtualPosting +isBalancedVirtual p = preal p == BalancedVirtualPosting hasAmount :: Posting -> Bool hasAmount = not . isMissingMixedAmount . pamount @@ -470,11 +471,12 @@ postingAddTags p@Posting{ptags} tags = p{ptags=ptags `union` tags} -- | Add the given hidden tag to a posting; and with a true argument, -- also add the equivalent visible tag to the posting's tags and comment fields. -- If the posting already has these tags (with any value), do nothing. -postingAddHiddenAndMaybeVisibleTag :: Bool -> HiddenTag -> Posting -> Posting -postingAddHiddenAndMaybeVisibleTag verbosetags ht p@Posting{pcomment=c, ptags} = +postingAddHiddenAndMaybeVisibleTag :: Bool -> Bool -> HiddenTag -> Posting -> Posting +postingAddHiddenAndMaybeVisibleTag prepend verbosetags ht p@Posting{pcomment=c, ptags} = (p `postingAddTags` ([ht] <> [vt |verbosetags])) - {pcomment=if verbosetags && not hadtag then c `commentAddTag` vt else c} + {pcomment=if verbosetags && not hadtag then addFn c vt else c} where + addFn = if prepend then commentPrependTag else commentAddTag vt@(vname,_) = toVisibleTag ht hadtag = any ((== (T.toLower vname)) . T.toLower . fst) ptags -- XXX should regex-quote vname @@ -505,7 +507,7 @@ postingAddInferredEquityPostings verbosetags equityAcct p | costPostingTagName `elem` map fst (ptags p) = [p] -- tag the posting, and for each of its costs, add an equivalent pair of conversion postings after it | otherwise = - postingAddHiddenAndMaybeVisibleTag verbosetags (costPostingTagName,"") p : + postingAddHiddenAndMaybeVisibleTag False verbosetags (costPostingTagName,"") p : concatMap makeConversionPostings costs where costs = filter (isJust . acost) . amountsRaw $ pamount p @@ -523,8 +525,8 @@ postingAddInferredEquityPostings verbosetags equityAcct p amtCommodity = commodity amt costCommodity = commodity cost convp = nullposting{pdate=pdate p, pdate2=pdate2 p, pstatus=pstatus p, ptransaction=ptransaction p} - & postingAddHiddenAndMaybeVisibleTag verbosetags (conversionPostingTagName,"") - & postingAddHiddenAndMaybeVisibleTag verbosetags (generatedPostingTagName, "") + & postingAddHiddenAndMaybeVisibleTag False verbosetags (conversionPostingTagName,"") + & postingAddHiddenAndMaybeVisibleTag False verbosetags (generatedPostingTagName, "") accountPrefix = mconcat [ equityAcct, ":", T.intercalate "-" $ sort [amtCommodity, costCommodity], ":"] -- Take the commodity of an amount and collapse consecutive spaces to a single space commodity = T.unwords . filter (not . T.null) . T.words . acommodity @@ -558,6 +560,16 @@ commentAddTag c (t,v) c' = T.stripEnd c tag = t <> ": " <> v +-- | Add a tag at the start of a comment, comma-separated from any prior content. +-- A space is inserted following the colon, before the value. +commentPrependTag :: Text -> Tag -> Text +commentPrependTag c (t,v) + | T.null c' = tag + | otherwise = tag `commentJoin` c' + where + c' = T.stripEnd c + tag = t <> ": " <> v + -- | Like commentAddTag, but omits the space after the colon. commentAddTagUnspaced :: Text -> Tag -> Text commentAddTagUnspaced c (t,v) @@ -579,7 +591,7 @@ commentAddTagNextLine cmt (t,v) = tests_Posting = testGroup "Posting" [ testCase "accountNamePostingType" $ do - accountNamePostingType "a" @?= RegularPosting + accountNamePostingType "a" @?= RealPosting accountNamePostingType "(a)" @?= VirtualPosting accountNamePostingType "[a]" @?= BalancedVirtualPosting diff --git a/hledger-lib/Hledger/Data/Timeclock.hs b/hledger-lib/Hledger/Data/Timeclock.hs index e90d94a8c6f..fbc07d28c60 100644 --- a/hledger-lib/Hledger/Data/Timeclock.hs +++ b/hledger-lib/Hledger/Data/Timeclock.hs @@ -262,7 +262,7 @@ entryFromTimeclockInOut requiretimeordered i o "%s%s:\nThis clockout is earlier than the clockin." (makeTimeClockErrorExcerpt i "") (makeTimeClockErrorExcerpt o "") - ps = [posting{paccount=acctname, pamount=amt, ptype=VirtualPosting, ptransaction=Just t}] + ps = [posting{paccount=acctname, pamount=amt, preal=VirtualPosting, ptransaction=Just t}] -- tests diff --git a/hledger-lib/Hledger/Data/Transaction.hs b/hledger-lib/Hledger/Data/Transaction.hs index bfe7257ac28..9bd4bb2cabf 100644 --- a/hledger-lib/Hledger/Data/Transaction.hs +++ b/hledger-lib/Hledger/Data/Transaction.hs @@ -95,7 +95,7 @@ data TransactionBalancingPrecision = -- Some valid journals are rejected until commodity directives are added. -- Small unbalanced remainders can be hidden, and in accounts that are never reconciled, can accumulate over time. | TBPExact - -- ^ Simpler, more robust behaviour, as in Ledger: use precision inferred from the transaction. + -- ^ Simpler, more robust behaviour, like (I thought) Ledger: use precision inferred from the transaction. -- Display precision and transaction balancing precision are independent; display precision never affects journal reading. -- Valid journals from ledger or beancount are accepted without needing commodity directives. -- Every imbalance in a transaction is visibly accounted for in that transaction's journal entry. @@ -183,7 +183,7 @@ showTransactionHelper :: Bool -> Transaction -> TB.Builder showTransactionHelper onelineamounts t = TB.fromText descriptionline <> newline <> foldMap ((<> newline) . TB.fromText) newlinecomments - <> foldMap ((<> newline) . TB.fromText) (postingsAsLines onelineamounts $ tpostings t) + <> foldMap ((<> newline) . TB.fromText) (postingsAsLines defaultFmt onelineamounts $ tpostings t) <> newline where descriptionline = T.stripEnd $ showTransactionLineFirstPart t <> T.concat [desc, samelinecomment] @@ -368,8 +368,8 @@ transactionTagCostsAndEquityAndMaybeInferCosts verbosetags1 addcosts conversiona -- A function that adds a cost and/or tag to a numbered posting if appropriate. postingAddCostAndOrTag np costp (n,p) = - (n, if | n == np -> costp & postingAddHiddenAndMaybeVisibleTag verbosetags (costPostingTagName,"") -- if it's the specified posting number, replace it with the costful posting, and tag it - | n == n1 || n == n2 -> p & postingAddHiddenAndMaybeVisibleTag verbosetags (conversionPostingTagName,"") -- if it's one of the equity conversion postings, tag it + (n, if | n == np -> costp & postingAddHiddenAndMaybeVisibleTag False verbosetags (costPostingTagName,"") -- if it's the specified posting number, replace it with the costful posting, and tag it + | n == n1 || n == n2 -> p & postingAddHiddenAndMaybeVisibleTag False verbosetags (conversionPostingTagName,"") -- if it's one of the equity conversion postings, tag it | otherwise -> p) -- Annotate any errors with the conversion posting pair @@ -445,7 +445,7 @@ transactionTagCostsAndEquityAndMaybeInferCosts verbosetags1 addcosts conversiona deleteUniqueMatch p (x:xs) | p x = if any p xs then Nothing else Just xs | otherwise = (x:) <$> deleteUniqueMatch p xs deleteUniqueMatch _ [] = Nothing - annotateWithPostings xs str = T.unlines $ str : postingsAsLines False xs + annotateWithPostings xs str = T.unlines $ str : postingsAsLines defaultFmt False xs dbgShowAmountPrecision a = case asprecision $ astyle a of @@ -543,7 +543,7 @@ tests_Transaction = , paccount = "a" , pamount = mixed [usd 1, hrs 2] , pcomment = "pcomment1\npcomment2\n tag3: val3 \n" - , ptype = RegularPosting + , preal = RealPosting , ptags = [("ptag1", "val1"), ("ptag2", "val2")] } in showPostingLines p @?= @@ -572,27 +572,27 @@ tests_Transaction = -- unbalanced amounts when precision is limited (#931) -- t4 = nulltransaction {tpostings = ["a" `post` usd (-0.01), "b" `post` usd (0.005), "c" `post` usd (0.005)]} in testGroup "postingsAsLines" [ - testCase "null-transaction" $ postingsAsLines False (tpostings nulltransaction) @?= [] - , testCase "implicit-amount" $ postingsAsLines False (tpostings timp) @?= + testCase "null-transaction" $ postingsAsLines defaultFmt False (tpostings nulltransaction) @?= [] + , testCase "implicit-amount" $ postingsAsLines defaultFmt False (tpostings timp) @?= [ " a $1.00" , " b" -- implicit amount remains implicit ] - , testCase "explicit-amounts" $ postingsAsLines False (tpostings texp) @?= + , testCase "explicit-amounts" $ postingsAsLines defaultFmt False (tpostings texp) @?= [ " a $1.00" , " b $-1.00" ] - , testCase "one-explicit-amount" $ postingsAsLines False (tpostings texp1) @?= + , testCase "one-explicit-amount" $ postingsAsLines defaultFmt False (tpostings texp1) @?= [ " (a) $1.00" ] - , testCase "explicit-amounts-two-commodities" $ postingsAsLines False (tpostings texp2) @?= + , testCase "explicit-amounts-two-commodities" $ postingsAsLines defaultFmt False (tpostings texp2) @?= [ " a $1.00" , " b -1.00h @ $1.00" ] - , testCase "explicit-amounts-not-explicitly-balanced" $ postingsAsLines False (tpostings texp2b) @?= + , testCase "explicit-amounts-not-explicitly-balanced" $ postingsAsLines defaultFmt False (tpostings texp2b) @?= [ " a $1.00" , " b -1.00h" ] - , testCase "implicit-amount-not-last" $ postingsAsLines False (tpostings t3) @?= + , testCase "implicit-amount-not-last" $ postingsAsLines defaultFmt False (tpostings t3) @?= [" a $1.00", " b", " c $-1.00"] -- , testCase "ensure-visibly-balanced" $ -- in postingsAsLines False (tpostings t4) @?= @@ -617,7 +617,7 @@ tests_Transaction = , paccount = "a" , pamount = mixed [usd 1, hrs 2] , pcomment = "\npcomment2\n" - , ptype = RegularPosting + , preal = RealPosting , ptags = [("ptag1", "val1"), ("ptag2", "val2")] } ] diff --git a/hledger-lib/Hledger/Data/Types.hs b/hledger-lib/Hledger/Data/Types.hs index 6557a983f1b..ace9d4d68bc 100644 --- a/hledger-lib/Hledger/Data/Types.hs +++ b/hledger-lib/Hledger/Data/Types.hs @@ -114,8 +114,6 @@ data EFDay = Exact Day | Flex Day deriving (Eq,Generic,Show) -- EFDay's Ord instance treats them like ordinary dates, ignoring exact/flexible. instance Ord EFDay where compare d1 d2 = compare (fromEFDay d1) (fromEFDay d2) --- instance Ord EFDay where compare = maCompare - fromEFDay :: EFDay -> Day fromEFDay (Exact d) = d fromEFDay (Flex d) = d @@ -218,22 +216,6 @@ isIncomeStatementAccountType t = t `elem` [ Gain ] --- | Check whether the first argument is a subtype of the second: either equal --- or one of the defined subtypes. -isAccountSubtypeOf :: AccountType -> AccountType -> Bool -isAccountSubtypeOf Asset Asset = True -isAccountSubtypeOf Liability Liability = True -isAccountSubtypeOf Equity Equity = True -isAccountSubtypeOf Revenue Revenue = True -isAccountSubtypeOf Expense Expense = True -isAccountSubtypeOf Cash Cash = True -isAccountSubtypeOf Cash Asset = True -isAccountSubtypeOf Conversion Conversion = True -isAccountSubtypeOf Conversion Equity = True -isAccountSubtypeOf Gain Gain = True -isAccountSubtypeOf Gain Revenue = True -isAccountSubtypeOf _ _ = False - -- not worth the trouble, letters defined in accountdirectivep for now --instance Read AccountType -- where @@ -334,10 +316,11 @@ data DigitGroupStyle = DigitGroups !Char ![Word8] type CommoditySymbol = Text data Commodity = Commodity { - csymbol :: CommoditySymbol, - cformat :: Maybe AmountStyle, - ccomment :: Text, -- ^ any comment lines following the commodity directive - ctags :: [Tag] -- ^ tags extracted from the comment, if any + csymbol :: CommoditySymbol, + cformat :: Maybe AmountStyle, + ccomment :: Text, -- ^ any comment lines following the commodity directive + ctags :: [Tag], -- ^ tags extracted from the comment, if any + csourcepos :: SourcePos -- ^ source position of the commodity directive } deriving (Show,Eq,Generic) --,Ord) -- | The cost basis of an individual lot - some quantity of an asset acquired at a given date and time. @@ -345,11 +328,31 @@ data Commodity = Commodity { -- Or it can represent a cost basis matcher for selecting lots. -- Note: cost is always stored as a per-unit cost, even if the user specified total cost with {{}}. data CostBasis = CostBasis { - cbCost :: !(Maybe Amount), -- ^ nominal acquisition cost (per-unit) cbDate :: !(Maybe Day), -- ^ nominal acquisition date - cbLabel :: !(Maybe Text) -- ^ a short label to ensure uniqueness, correct intra-day order, or memorability, if needed + cbLabel :: !(Maybe Text), -- ^ a short label to ensure uniqueness, correct intra-day order, or memorability, if needed + cbCost :: !(Maybe Amount) -- ^ nominal acquisition cost (per-unit) } deriving (Show,Eq,Generic,Ord) +-- | Identifies a specific lot of a commodity, by its acquisition date and optional label. +-- Ordered by date first, then label (Nothing sorts before Just). +data LotId = LotId { + lotDate :: !Day, + lotLabel :: !(Maybe Text) +} deriving (Show,Eq,Ord,Generic) + +-- | The method used to select lots for disposal or transfer. +-- Per-account methods (scoped to the posting's account): +-- FIFO/LIFO select oldest/newest first. HIFO selects highest cost first. +-- AVERAGE uses weighted average cost basis for disposals (FIFO consumption order). +-- SPECID requires every disposal/transfer to have an explicit lot selector matching exactly one lot. +-- Global validation methods (*ALL variants): +-- FIFOALL/LIFOALL/HIFOALL select per-account but validate that the selected lots would also +-- be chosen first if all accounts' lots were considered together. Errors if not. +-- AVERAGEALL additionally computes weighted average cost across the global pool. +data ReductionMethod = FIFO | LIFO | HIFO | AVERAGE | SPECID + | FIFOALL | LIFOALL | HIFOALL | AVERAGEALL + deriving (Show,Read,Eq,Ord,Generic) + data Amount = Amount { acommodity :: !CommoditySymbol, -- commodity symbol, or special value "AUTO" aquantity :: !Quantity, -- numeric quantity, or zero in case of "AUTO" @@ -375,8 +378,51 @@ instance HasAmounts a => HasAmounts (Maybe a) where styleAmounts styles = fmap (styleAmounts styles) +-- | hledger's most general amount type. +-- It can contain multiple single-commodity Amounts, each possibly with a transacted cost and/or a lot cost basis attached. +-- Internally it is a map from MixedAmountKey to Amount, for efficiency and so that every mixed amount has a single canonical form. +newtype MixedAmount = Mixed (M.Map MixedAmountKey Amount) + deriving (Generic,Show) + +-- | The key used to group amounts within a MixedAmount: commodity and an optional unit or total transacted cost. +-- Amounts with the same commodity and transacted cost are combined; different transacted costs are kept separate. +-- (Lot cost basis is not part of the key, so not kept separate; subaccounts are used for that.) +data MixedAmountKey + = MixedAmountKeyNoCost + !CommoditySymbol -- ^ amount commodity + | MixedAmountKeyUnitCost + !CommoditySymbol -- ^ amount commodity + !CommoditySymbol -- ^ transacted cost commodity + !Quantity -- ^ transacted cost per unit + | MixedAmountKeyTotalCost + !CommoditySymbol -- ^ amount commodity + !CommoditySymbol -- ^ transacted cost commodity + deriving (Eq, Generic, Show) + +-- | Sort by commodity, then cost commodity (no cost first), then cost type and quantity. +instance Ord MixedAmountKey where + compare = comparing commodity <> comparing costCommodity <> comparing costDetail + where + commodity (MixedAmountKeyNoCost c) = c + commodity (MixedAmountKeyUnitCost c _ _) = c + commodity (MixedAmountKeyTotalCost c _) = c + + costCommodity (MixedAmountKeyNoCost _) = Nothing + costCommodity (MixedAmountKeyUnitCost _ pc _) = Just pc + costCommodity (MixedAmountKeyTotalCost _ pc) = Just pc -newtype MixedAmount = Mixed (M.Map MixedAmountKey Amount) deriving (Generic,Show) + costDetail (MixedAmountKeyNoCost _) = Nothing + costDetail (MixedAmountKeyUnitCost _ _ q) = Just (1 :: Int, Just q) + costDetail (MixedAmountKeyTotalCost _ _) = Just (0, Nothing) + +-- | Calculate the key for storing this Amount within a MixedAmount, +-- from its commodity and transacted cost (ignoring cost basis). +mixedAmountKey :: Amount -> MixedAmountKey +mixedAmountKey Amount{acommodity=c, acost} = + case acost of + Nothing -> MixedAmountKeyNoCost c + Just (UnitCost p) -> MixedAmountKeyUnitCost c (acommodity p) (aquantity p) + Just (TotalCost p) -> MixedAmountKeyTotalCost c (acommodity p) instance Eq MixedAmount where a == b = maCompare a b == EQ instance Ord MixedAmount where compare = maCompare @@ -398,39 +444,8 @@ maCompare (Mixed a) (Mixed b) = go (M.toList a) (M.toList b) Just (TotalCost p) -> aquantity p _ -> 0 --- | Stores the CommoditySymbol of the Amount, along with the CommoditySymbol of --- the cost, and its unit cost if being used. -data MixedAmountKey - = MixedAmountKeyNoCost !CommoditySymbol - | MixedAmountKeyTotalCost !CommoditySymbol !CommoditySymbol - | MixedAmountKeyUnitCost !CommoditySymbol !CommoditySymbol !Quantity - deriving (Eq,Generic,Show) - --- | We don't auto-derive the Ord instance because it would give an undesired ordering. --- We want the keys to be sorted lexicographically: --- (1) By the primary commodity of the amount. --- (2) By the commodity of the cost, with no cost being first. --- (3) By the unit cost, from most negative to most positive, with total costs --- before unit costs. --- For example, we would like the ordering to give --- MixedAmountKeyNoCost "X" < MixedAmountKeyTotalCost "X" "Z" < MixedAmountKeyNoCost "Y" -instance Ord MixedAmountKey where - compare = comparing commodity <> comparing pCommodity <> comparing pCost - where - commodity (MixedAmountKeyNoCost c) = c - commodity (MixedAmountKeyTotalCost c _) = c - commodity (MixedAmountKeyUnitCost c _ _) = c - - pCommodity (MixedAmountKeyNoCost _) = Nothing - pCommodity (MixedAmountKeyTotalCost _ pc) = Just pc - pCommodity (MixedAmountKeyUnitCost _ pc _) = Just pc - - pCost (MixedAmountKeyNoCost _) = Nothing - pCost (MixedAmountKeyTotalCost _ _) = Nothing - pCost (MixedAmountKeyUnitCost _ _ q) = Just q - -data PostingType = RegularPosting | VirtualPosting | BalancedVirtualPosting - deriving (Eq,Show,Generic) +data PostingRealness = RealPosting | VirtualPosting | BalancedVirtualPosting + deriving (Eq,Show,Generic) type TagName = Text type TagValue = Text @@ -498,13 +513,13 @@ data BalanceAssertion = BalanceAssertion { } deriving (Eq,Generic,Show) data Posting = Posting { - pdate :: Maybe Day, -- ^ this posting's date, if different from the transaction's - pdate2 :: Maybe Day, -- ^ this posting's secondary date, if different from the transaction's + pdate :: Maybe Day, -- ^ this posting's date, if different from the transaction's + pdate2 :: Maybe Day, -- ^ this posting's secondary date, if different from the transaction's pstatus :: Status, paccount :: AccountName, pamount :: MixedAmount, - pcomment :: Text, -- ^ this posting's comment lines, as a single non-indented multi-line string - ptype :: PostingType, + pcomment :: Text, -- ^ this posting's comment lines, as a single non-indented multi-line string + preal :: PostingRealness, -- ^ is this a normal balanced posting, or a virtual/unbalanced one ? ptags :: [Tag], -- ^ tag names and values, extracted from the posting comment -- and (after finalisation) the posting account's directive if any pbalanceassertion :: Maybe BalanceAssertion, -- ^ an expected balance in the account after this posting, @@ -532,7 +547,7 @@ instance Show Posting where ,"paccount=" ++ show paccount ,"pamount=" ++ show pamount ,"pcomment=" ++ show pcomment - ,"ptype=" ++ show ptype + ,"preal=" ++ show preal ,"ptags=" ++ show ptags ,"pbalanceassertion=" ++ show pbalanceassertion ,"ptransaction=" ++ show (ptransaction $> "txn") @@ -827,13 +842,15 @@ instance NFData DigitGroupStyle instance NFData EFDay instance NFData Interval instance NFData Journal +instance NFData LotId instance NFData MarketPrice +instance NFData ReductionMethod instance NFData MixedAmount instance NFData MixedAmountKey instance NFData Rounding instance NFData PayeeDeclarationInfo instance NFData PeriodicTransaction -instance NFData PostingType +instance NFData PostingRealness instance NFData PriceDirective instance NFData Side instance NFData Status diff --git a/hledger-lib/Hledger/Query.hs b/hledger-lib/Hledger/Query.hs index dbef1f3bea6..87bc95510e2 100644 --- a/hledger-lib/Hledger/Query.hs +++ b/hledger-lib/Hledger/Query.hs @@ -67,6 +67,7 @@ module Hledger.Query ( matchesMixedAmount, matchesAmount, matchesCommodity, + matchesCommodityExtra, matchesTag, -- patternsMatchTags, matchesPriceDirective, @@ -93,6 +94,7 @@ import Text.Megaparsec.Char (char, string, string') import Hledger.Utils hiding (words') import Hledger.Data.Types import Hledger.Data.AccountName +import Hledger.Data.AccountType import Hledger.Data.Amount (amountsRaw, mixedAmount, nullamt, usd) import Hledger.Data.Dates import Hledger.Data.Posting @@ -829,6 +831,17 @@ matchesCommodity (AnyPosting qs) s = all (`matchesCommodity` s) qs matchesCommodity (AllPostings qs) s = all1 (`matchesCommodity` s) qs matchesCommodity _ _ = False +-- | Like matchesCommodity, but also supporting Tag queries, +-- using the provided function to look up a commodity's tags. +matchesCommodityExtra :: (CommoditySymbol -> [Tag]) -> Query -> CommoditySymbol -> Bool +matchesCommodityExtra ctags (Not q) c = not $ matchesCommodityExtra ctags q c +matchesCommodityExtra ctags (Or qs) c = any (\q -> matchesCommodityExtra ctags q c) qs +matchesCommodityExtra ctags (And qs) c = all (\q -> matchesCommodityExtra ctags q c) qs +matchesCommodityExtra ctags (AnyPosting qs) c = all (\q -> matchesCommodityExtra ctags q c) qs +matchesCommodityExtra ctags (AllPostings qs) c = all1 (\q -> matchesCommodityExtra ctags q c) qs +matchesCommodityExtra ctags (Tag npat vpat) c = patternsMatchTags npat vpat $ ctags c +matchesCommodityExtra _ q c = matchesCommodity q c + -- | Does the match expression match this (simple) amount ? matchesAmount :: Query -> Amount -> Bool matchesAmount (Not q) a = not $ q `matchesAmount` a @@ -1167,9 +1180,9 @@ tests_Query = testGroup "Query" [ assertBool "" $ not $ (Not $ StatusQ Unmarked) `matchesPosting` nullposting{pstatus=Unmarked} ,testCase "positive match on true posting status acquired from transaction" $ assertBool "" $ (StatusQ Cleared) `matchesPosting` nullposting{pstatus=Unmarked,ptransaction=Just nulltransaction{tstatus=Cleared}} - ,testCase "real:1 on real posting" $ assertBool "" $ (Real True) `matchesPosting` nullposting{ptype=RegularPosting} - ,testCase "real:1 on virtual posting fails" $ assertBool "" $ not $ (Real True) `matchesPosting` nullposting{ptype=VirtualPosting} - ,testCase "real:1 on balanced virtual posting fails" $ assertBool "" $ not $ (Real True) `matchesPosting` nullposting{ptype=BalancedVirtualPosting} + ,testCase "real:1 on real posting" $ assertBool "" $ (Real True) `matchesPosting` nullposting{preal=RealPosting} + ,testCase "real:1 on virtual posting fails" $ assertBool "" $ not $ (Real True) `matchesPosting` nullposting{preal=VirtualPosting} + ,testCase "real:1 on balanced virtual posting fails" $ assertBool "" $ not $ (Real True) `matchesPosting` nullposting{preal=BalancedVirtualPosting} ,testCase "acct:" $ assertBool "" $ (Acct $ toRegex' "'b") `matchesPosting` nullposting{paccount="'b"} ,testCase "tag:" $ do assertBool "" $ not $ (Tag (toRegex' "a") (Just $ toRegex' "r$")) `matchesPosting` nullposting diff --git a/hledger-lib/Hledger/Read/Common.hs b/hledger-lib/Hledger/Read/Common.hs index d068979f389..ec965326e57 100644 --- a/hledger-lib/Hledger/Read/Common.hs +++ b/hledger-lib/Hledger/Read/Common.hs @@ -128,7 +128,7 @@ where --- ** imports import Control.Applicative.Permutations (runPermutation, toPermutationWithDefault) -import Control.Monad (foldM, join, liftM2, when, unless, (>=>), (<=<)) +import Control.Monad (foldM, liftM2, when, unless, (>=>), (<=<)) import Control.Monad.Fail qualified as Fail (fail) import Control.Monad.Except (ExceptT(..), liftEither, withExceptT) import Control.Monad.IO.Class (MonadIO, liftIO) @@ -152,7 +152,6 @@ import Data.Time.Clock.POSIX (getPOSIXTime) import Data.Time.LocalTime (LocalTime(..), TimeOfDay(..)) import Data.Word (Word8) import System.Directory (canonicalizePath) -import System.FilePath (takeFileName) import System.IO (Handle) import Text.Megaparsec import Text.Megaparsec.Char (char, char', digitChar, newline, string) @@ -208,8 +207,8 @@ isStdin f = case splitAtElement ':' f of -- | Parse an InputOpts from a RawOpts and a provided date. -- This will fail with a usage error if the forecast period expression cannot be parsed. -rawOptsToInputOpts :: Day -> Bool -> Bool -> RawOpts -> InputOpts -rawOptsToInputOpts day usecoloronstdout autopostingtags rawopts = +rawOptsToInputOpts :: Day -> Bool -> RawOpts -> InputOpts +rawOptsToInputOpts day usecoloronstdout rawopts = let -- Allow/disallow implicit-cost conversion transactions, according to policy in Check.md. @@ -244,12 +243,12 @@ rawOptsToInputOpts day usecoloronstdout autopostingtags rawopts = ,new_save_ = True ,pivot_ = stringopt "pivot" rawopts ,forecast_ = forecastPeriodFromRawOpts day rawopts - ,auto_posting_tags_ = autopostingtags ,verbose_tags_ = boolopt "verbose-tags" rawopts ,reportspan_ = DateSpan (Exact <$> queryStartDate False datequery) (Exact <$> queryEndDate False datequery) ,auto_ = boolopt "auto" rawopts ,infer_equity_ = boolopt "infer-equity" rawopts && conversionop_ ropts /= Just ToCost ,infer_costs_ = boolopt "infer-costs" rawopts + ,lots_ = boolopt "lots" rawopts ,balancingopts_ = defbalancingopts{ ignore_assertions_ = boolopt "ignore-assertions" rawopts , infer_balancing_costs_ = not noinferbalancingcosts @@ -330,94 +329,70 @@ initialiseAndParseJournal parser iopts f txt = do {- HLINT ignore journalFinalise "Redundant <&>" -} -- silence this warning, the code is clearer as is -- note this activates TH, may slow compilation ? https://github.com/ndmitchell/hlint/blob/master/README.md#customizing-the-hints -- --- | Post-process a Journal that has just been parsed or generated, in this order: --- --- - add misc info (file path, read time) --- --- - reverse transactions into their original parse order --- --- - apply canonical commodity styles --- --- - propagate account tags to postings --- --- - maybe add forecast transactions --- --- - propagate account tags to postings (again to affect forecast transactions) --- --- - maybe add auto postings --- --- - propagate account tags to postings (again to affect auto postings) --- --- - evaluate balance assignments and balance each transaction --- --- - maybe check balance assertions --- --- - maybe infer costs from equity postings --- --- - maybe infer equity postings from costs --- --- - manye infer market prices from costs --- --- One correctness check (parseable) has already passed when this function is called. --- Up to four more are performed here: --- --- - ordereddates (when enabled) --- --- - assertions (when enabled) --- --- - autobalanced (and with --strict, balanced ?), in the journalBalanceTransactions step. --- --- Others (commodities, accounts..) are done later by journalStrictChecks. --- +-- | Post-process a parsed Journal: infer missing information, check validity, +-- and enrich postings with computed metadata. +-- See doc\/SPEC-finalising.md for the full pipeline specification. journalFinalise :: InputOpts -> FilePath -> Text -> ParsedJournal -> ExceptT String IO Journal -journalFinalise iopts@InputOpts{auto_,balancingopts_,infer_costs_,infer_equity_,strict_,auto_posting_tags_,verbose_tags_,_ioDay} f txt pj = do +journalFinalise iopts@InputOpts{auto_,balancingopts_,infer_costs_,infer_equity_,lots_,strict_,verbose_tags_,_ioDay} f txt pj = do let BalancingOpts{commodity_styles_, ignore_assertions_} = balancingopts_ - fname = "journalFinalise " <> takeFileName f - lbl = lbl_ fname - -- Some not so pleasant hacks - -- We want to know when certain checks have been explicitly requested with the check command, - -- but it does not run until later. For now, inspect the command line with unsafePerformIO. + -- Hack: peek at the command line to know if certain checks were requested. checking checkname = "check" `elem` args && checkname `elem` args where args = progArgs - -- We will check ordered dates when "check ordereddates" is used. checkordereddates = checking "ordereddates" - -- We will check balance assertions by default, unless -I is used, but always if -s or "check assertions" are used. checkassertions = not ignore_assertions_ || strict_ || checking "assertions" t <- liftIO getPOSIXTime liftEither $ pj{jglobalcommoditystyles=fromMaybe mempty commodity_styles_} - & journalSetLastReadTime t -- save the last read time - & journalAddFile (f, txt) -- save the main file's info - & journalReverse -- convert all lists to the order they were parsed - & journalAddAccountTypes -- build a map of all known account types + + -- Setup + & journalSetLastReadTime t -- save the last read time + & journalAddFile (f, txt) -- save the main file's info + & journalReverse -- convert all lists to the order they were parsed + + -- Account types and amount styles + & journalAddAccountTypes -- build a map of all known account types -- XXX does not see conversion accounts generated by journalInferEquityFromCosts below, requiring a workaround in journalCheckAccounts. Do it later ? - & journalStyleAmounts -- Infer and apply commodity styles (but don't round) - should be done early - <&> journalAddForecast verbose_tags_ (forecastPeriod iopts pj) -- Add forecast transactions if enabled - <&> (if auto_posting_tags_ then journalPostingsAddAccountTags else id) -- Maybe propagate account tags to postings - >>= journalTagCostsAndEquityAndMaybeInferCosts verbose_tags_ False -- Tag equity conversion postings and redundant costs, to help journalBalanceTransactions ignore them. + & journalStyleAmounts -- infer and apply commodity styles (but don't round) - should be done early + + -- Forecast and account tags + <&> journalAddForecast verbose_tags_ (forecastPeriod iopts pj) -- add forecast transactions if enabled + <&> journalPostingsAddAccountTags -- propagate account tags to postings (queryable but hidden) + + -- Pre-balancing cost/equity tagging + >>= journalTagCostsAndEquityAndMaybeInferCosts verbose_tags_ False -- tag equity conversion postings and redundant costs, to help the transaction balancer ignore them + + -- Auto postings >>= (if auto_ && not (null $ jtxnmodifiers pj) - then journalAddAutoPostings verbose_tags_ _ioDay balancingopts_ -- Add auto postings if enabled, and account tags if needed. Does preliminary transaction balancing. + then journalAddAutoPostings verbose_tags_ _ioDay balancingopts_ -- add auto postings if enabled; does preliminary transaction balancing else pure) - -- XXX how to force debug output here ? - -- >>= Right . dbg0With (concatMap (T.unpack.showTransaction).jtxns) - -- >>= \j -> deepseq (concatMap (T.unpack.showTransaction).jtxns $ j) (return j) - <&> dbg9With (lbl "amounts after styling, forecasting, auto-posting".showJournalPostingAmountsDebug) - >>= (\j -> if checkordereddates then journalCheckOrdereddates j $> j else Right j) -- check ordereddates before assertions. The outer parentheses are needed. - >>= journalBalanceTransactions balancingopts_{ignore_assertions_=not checkassertions} -- infer balance assignments and missing amounts, and maybe check balance assertions. - <&> dbg9With (lbl "amounts after transaction-balancing".showJournalPostingAmountsDebug) - -- <&> dbg9With (("journalFinalise amounts after styling, forecasting, auto postings, transaction balancing"<>).showJournalPostingAmountsDebug) - >>= journalInferCommodityStyles -- infer commodity styles once more now that all posting amounts are present - -- >>= Right . dbg0With (pshow.journalCommodityStyles) - <&> (if auto_posting_tags_ then journalPostingsAddCommodityTags else id) -- Maybe propagate commodity tags to postings (after amounts are inferred) - >>= (if infer_costs_ then journalTagCostsAndEquityAndMaybeInferCosts verbose_tags_ True else pure) -- With --infer-costs, infer costs from equity postings where possible - <&> (if infer_equity_ then journalInferEquityFromCosts verbose_tags_ else id) -- With --infer-equity, infer equity postings from costs where possible - <&> dbg9With (lbl "amounts after equity-inferring".showJournalPostingAmountsDebug) - <&> journalInferMarketPricesFromTransactions -- infer market prices from commodity-exchanging transactions - -- <&> dbg6Msg fname -- debug logging - <&> dbgJournalAcctDeclOrder (fname <> ": acct decls : ") - <&> journalRenumberAccountDeclarations - <&> dbgJournalAcctDeclOrder (fname <> ": acct decls renumbered: ") + + -- Lot classification and transacted cost inference + >>= journalInferBasisFromAccountNames -- infer cost basis from lot subaccount names + <&> journalClassifyLotPostings verbose_tags_ -- detect and classify lot postings (acquire/dispose/transfer..), maybe with visible tags + <&> journalInferPostingsTransactedCost -- in acquire postings, infer a transacted cost from cost basis + + -- Transaction balancing + >>= (\j -> if checkordereddates then journalCheckOrdereddates j $> j else Right j) -- maybe check that journal entries are in date order + >>= (\j -> journalBalanceTransactions -- infer balance assignments/amounts, maybe check balance assertions + (balancingopts_{ignore_assertions_=not checkassertions, account_types_ = jaccounttypes j}) j) + + -- Post-balancing enrichment + >>= journalInferCommodityStyles -- infer commodity styles once more now that all posting amounts are present + <&> journalPostingsAddCommodityTags -- propagate amounts' commodity tags to postings (queryable but hidden) + + -- Cost/equity inference + >>= (if infer_costs_ then journalTagCostsAndEquityAndMaybeInferCosts verbose_tags_ True else pure) -- maybe infer costs from equity postings + <&> (if infer_equity_ then journalInferEquityFromCosts verbose_tags_ else id) -- maybe infer equity postings from costs + + -- Market prices and renumbering + <&> journalInferMarketPricesFromTransactions -- infer market prices from commodity-exchanging transactions + <&> journalRenumberAccountDeclarations -- renumber account declarations for consistent ordering + + -- Lot calculation + >>= (if lots_ then journalCheckLotsTagValues else pure) -- with --lots: validate lots: tag values + >>= (if lots_ then journalCalculateLots verbose_tags_ else pure) -- with --lots: evaluate lot selectors, calculate lot balances, add lot subaccounts + >>= (if lots_ then journalInferAndCheckDisposalBalancing verbose_tags_ else pure) -- with --lots: infer gain amounts and check disposal transactions balance at cost basis -- | Apply any auto posting rules to generate extra postings on this journal's transactions. -- With a true first argument, adds visible tags to generated postings and modified transactions. @@ -446,6 +421,36 @@ journalAddForecast verbosetags (Just forecastspan) j = j{jtxns = jtxns j ++ fore . concatMap (\pt -> runPeriodicTransaction verbosetags pt forecastspan) $ jperiodictxns j +-- | For each posting whose account name contains a lot subaccount (e.g. +-- @assets:broker:{2026-01-15, $50}@), parse the cost basis from the subaccount +-- name and set or merge it into the posting's amounts' @acostbasis@. +-- This allows users to write lot subaccounts explicitly without redundant @{...}@ +-- amount annotations. +journalInferBasisFromAccountNames :: Journal -> Either String Journal +journalInferBasisFromAccountNames j = do + txns' <- mapM processTransaction (jtxns j) + Right j{jtxns = txns'} + where + parseAmt s = case parseamount s of + Right a -> Just a + Left _ -> Nothing + + processTransaction t = do + ps' <- mapM processPosting (tpostings t) + Right t{tpostings = ps'} + + processPosting p = case lotSubaccountName (paccount p) of + Nothing -> Right p + Just name -> do + cb <- parseLotName parseAmt name + let updateAmt a = case acostbasis a of + Nothing -> Right a{acostbasis = Just cb} + Just existing -> do + merged <- mergeCostBasis cb existing + Right a{acostbasis = Just merged} + amts' <- mapM updateAmt (amountsRaw (pamount p)) + Right p{pamount = foldMap mixedAmount amts'} + setYear :: Year -> JournalParser m () setYear y = modify' (\j -> j{jparsedefaultyear=Just y}) @@ -838,17 +843,27 @@ amountp' mult = label "amount" $ do let spaces = lift $ skipNonNewlineSpaces amt <- simpleamountp mult <* spaces - (mcost, _valuationexpr, mlotcost, mlotdate, mlotnote) <- runPermutation $ + (mcost, _valuationexpr, mlotcb, mlotdate, mlotnote) <- runPermutation $ -- costp, valuationexprp, lotnotep all parse things beginning with parenthesis, try needed (,,,,) <$> toPermutationWithDefault Nothing (Just <$> try (costp amt) <* spaces) <*> toPermutationWithDefault Nothing (Just <$> valuationexprp <* spaces) -- XXX no try needed here ? <*> toPermutationWithDefault Nothing (Just <$> lotcostp (aquantity amt) <* spaces) <*> toPermutationWithDefault Nothing (Just <$> lotdatep <* spaces) <*> toPermutationWithDefault Nothing (Just <$> lotnotep <* spaces) + -- Reject mixing consolidated {DATE,...} or {"LABEL",...} with ledger-style [DATE] or (NOTE) + let isConsolidated = case mlotcb of + Just cb | isJust (cbDate cb) || isJust (cbLabel cb) -> True + _ -> False + when (isConsolidated && (isJust mlotdate || isJust mlotnote)) $ + Fail.fail "hledger lot syntax {...} cannot be combined with ledger-style [DATE] or (NOTE)" let mcostbasis = - case (mlotcost, mlotdate, mlotnote) of + case (mlotcb, mlotdate, mlotnote) of (Nothing, Nothing, Nothing) -> Nothing - _ -> Just $ CostBasis { cbCost = join mlotcost, cbDate = mlotdate, cbLabel = mlotnote } + _ -> Just $ CostBasis + { cbCost = mlotcb >>= cbCost + , cbDate = (mlotcb >>= cbDate) <|> mlotdate + , cbLabel = (mlotcb >>= cbLabel) <|> mlotnote + } pure $ amt { acost = mcost, acostbasis = mcostbasis } -- An amount with optional cost, but no cost basis. @@ -1031,25 +1046,75 @@ balanceassertionp = do , baposition = sourcepos } --- Parse a Ledger-style lot cost: --- {UNITCOST} or {{TOTALCOST}} or {=FIXEDUNITCOST} or {{=FIXEDTOTALCOST}} or {}. +-- Parse lot cost in curly braces. +-- Accepts either: +-- Ledger-style: {UNITCOST} or {{TOTALCOST}} or {=FIXEDUNITCOST} or {{=FIXEDTOTALCOST}} or {} +-- Consolidated: {DATE, "LABEL", COST} with all fields optional and in DLC order -- If total cost syntax {{}} is used, converts it to unit cost by dividing by the posting quantity. -lotcostp :: Quantity -> JournalParser m (Maybe Amount) +lotcostp :: Quantity -> JournalParser m CostBasis lotcostp postingqty = -- dbg "lotcostp" $ - label "ledger-style lot cost" $ do + label "lot cost" $ do char '{' doublebrace <- option False $ char '{' >> pure True lift skipNonNewlineSpaces - _fixed <- fmap isJust $ optional $ char '=' - lift skipNonNewlineSpaces - ma <- optional $ simpleamountp False - lift skipNonNewlineSpaces - char '}' - when (doublebrace) $ void $ char '}' - pure $ fmap (convertToUnitCost doublebrace) ma + if doublebrace + then ledgerCost True + else do + -- Peek to decide: consolidated vs ledger + -- consolidated starts with date (digit), label ("), or empty } + -- ledger starts with =, amount, or empty } + c <- lookAhead anySingle + case c of + '}' -> char '}' >> pure (CostBasis Nothing Nothing Nothing) + '=' -> ledgerCost False + '"' -> consolidatedNoDate -- no date, start with label + _ -> tryDateOrLedger where - -- Convert {{TOTALCOST}} to {UNITCOST} by dividing by posting quantity + tryDateOrLedger = do + -- Does input look like a date (YYYY-D...) ? If so, commit to consolidated + -- date parsing (no try), so invalid dates give clear errors instead of + -- falling through to the amount parser with a confusing message. + looksLikeDate <- option False $ lookAhead $ try $ + count 4 digitChar >> char '-' >> digitChar >> pure True + if looksLikeDate + then do + d <- datep + lift skipNonNewlineSpaces + void $ lookAhead (oneOf [',','}']) + consolidatedAfterDate d + else ledgerCost False -- not a date, parse as ledger amount + + consolidatedAfterDate d = do + -- after date: optional ", LABEL", optional ", COST", then } + mlabel <- optional $ try $ char ',' >> lift skipNonNewlineSpaces >> quotedLabelp <* lift skipNonNewlineSpaces + mcost <- optional $ char ',' >> lift skipNonNewlineSpaces >> simpleamountp False <* lift skipNonNewlineSpaces + char '}' + pure $ CostBasis (Just d) mlabel mcost + + consolidatedNoDate = do + -- parse "LABEL", then optional ", COST", then } + mlabel <- Just <$> quotedLabelp + lift skipNonNewlineSpaces + mcost <- optional $ char ',' >> lift skipNonNewlineSpaces >> simpleamountp False <* lift skipNonNewlineSpaces + char '}' + pure $ CostBasis Nothing mlabel mcost + + quotedLabelp = do + char '"' + lbl <- T.pack <$> many (noneOf ['"', '\n']) + char '"' + pure lbl + + ledgerCost doublebrace = do + _fixed <- fmap isJust $ optional $ char '=' + lift skipNonNewlineSpaces + ma <- optional $ simpleamountp False + lift skipNonNewlineSpaces + char '}' + when doublebrace $ void $ char '}' + pure $ CostBasis Nothing Nothing (fmap (convertToUnitCost doublebrace) ma) + convertToUnitCost isTotal lotamt = if isTotal && postingqty /= 0 then lotamt { aquantity = aquantity lotamt / postingqty } diff --git a/hledger-lib/Hledger/Read/InputOptions.hs b/hledger-lib/Hledger/Read/InputOptions.hs index b303dee60b0..69e3d8e2062 100644 --- a/hledger-lib/Hledger/Read/InputOptions.hs +++ b/hledger-lib/Hledger/Read/InputOptions.hs @@ -34,12 +34,12 @@ data InputOpts = InputOpts { ,new_save_ :: Bool -- ^ save latest new transactions state for next time ? ,pivot_ :: String -- ^ use the given field's value as the account name ,forecast_ :: Maybe DateSpan -- ^ span in which to generate forecast transactions - ,auto_posting_tags_ :: Bool -- ^ propagate commodity and account tags to postings ? Can be disabled (for beancount export). ,verbose_tags_ :: Bool -- ^ add user-visible tags when generating/modifying transactions & postings ? ,reportspan_ :: DateSpan -- ^ a dirty hack keeping the query dates in InputOpts. This rightfully lives in ReportSpec, but is duplicated here. ,auto_ :: Bool -- ^ generate extra postings according to auto posting rules ? ,infer_equity_ :: Bool -- ^ infer equity conversion postings from costs ? ,infer_costs_ :: Bool -- ^ infer costs from equity conversion postings ? distinct from BalancingOpts{infer_balancing_costs_} + ,lots_ :: Bool -- ^ calculate and display per-lot subaccounts for lotful commodities/accounts ? ,balancingopts_ :: BalancingOpts -- ^ options for transaction balancing ,strict_ :: Bool -- ^ do extra correctness checks ? ,_defer :: Bool -- ^ internal flag: postpone checks, because we are processing multiple files ? @@ -57,12 +57,12 @@ definputopts = InputOpts , new_save_ = True , pivot_ = "" , forecast_ = Nothing - , auto_posting_tags_ = False , verbose_tags_ = False , reportspan_ = nulldatespan , auto_ = False , infer_equity_ = False , infer_costs_ = False + , lots_ = False , balancingopts_ = defbalancingopts , strict_ = False , _defer = False diff --git a/hledger-lib/Hledger/Read/JournalReader.hs b/hledger-lib/Hledger/Read/JournalReader.hs index a95dc9e4831..38c11974278 100644 --- a/hledger-lib/Hledger/Read/JournalReader.hs +++ b/hledger-lib/Hledger/Read/JournalReader.hs @@ -621,15 +621,16 @@ commoditydirectivep = commoditydirectiveonelinep <|> commoditydirectivemultiline -- >>> Right _ <- rjp commoditydirectiveonelinep "commodity $1.00 ; blah\n" commoditydirectiveonelinep :: JournalParser m () commoditydirectiveonelinep = do - (off, Amount{acommodity,astyle}) <- try $ do + (off, pos, Amount{acommodity,astyle}) <- try $ do string "commodity" + pos <- getSourcePos lift skipNonNewlineSpaces1 off <- getOffset amt <- amountp - pure $ (off, amt) + pure $ (off, pos, amt) lift skipNonNewlineSpaces (comment, tags) <- lift transactioncommentp - let comm = Commodity{csymbol=acommodity, cformat=Just $ dbg7 "style from commodity directive" astyle, ccomment=comment, ctags=tags} + let comm = Commodity{csymbol=acommodity, cformat=Just $ dbg7 "style from commodity directive" astyle, ccomment=comment, ctags=tags, csourcepos=pos} if isNothing $ asdecimalmark astyle then customFailure $ parseErrorAt off pleaseincludedecimalpoint else modify' (\j -> j{jdeclaredcommodities=M.insert acommodity comm $ jdeclaredcommodities j @@ -653,13 +654,14 @@ pleaseincludedecimalpoint = chomp $ unlines [ commoditydirectivemultilinep :: JournalParser m () commoditydirectivemultilinep = do string "commodity" + pos <- getSourcePos lift skipNonNewlineSpaces1 sym <- lift commoditysymbolp (comment, tags) <- lift transactioncommentp -- read all subdirectives, saving format subdirectives as Lefts subdirectives <- many $ indented (eitherP (formatdirectivep sym) (lift restofline)) let mfmt = lastMay $ lefts subdirectives - let comm = Commodity{csymbol=sym, cformat=mfmt, ccomment=comment, ctags=tags} + let comm = Commodity{csymbol=sym, cformat=mfmt, ccomment=comment, ctags=tags, csourcepos=pos} modify' (\j -> j{jdeclaredcommodities=M.insert sym comm $ jdeclaredcommodities j ,jdeclaredcommoditytags=if null tags then jdeclaredcommoditytags j else M.insert sym tags $ jdeclaredcommoditytags j}) @@ -972,7 +974,7 @@ postingphelper isPostingRule mTransactionYear = do lift skipNonNewlineSpaces account <- modifiedaccountnamep True return (status, account) - let (ptype, account') = (accountNamePostingType account, textUnbracket account) + let (preal, account') = (accountNamePostingType account, textUnbracket account) lift skipNonNewlineSpaces mult <- if isPostingRule then multiplierp else pure False amt <- optional $ amountp' mult @@ -987,7 +989,7 @@ postingphelper isPostingRule mTransactionYear = do , paccount=account' , pamount=maybe missingmixedamt mixedAmount amt , pcomment=comment - , ptype=ptype + , preal=preal , ptags=tags , pbalanceassertion=massertion } @@ -1178,7 +1180,7 @@ tests_JournalReader = testGroup "JournalReader" [ paccount="a", pamount=mixedAmount (usd 1), pcomment="pcomment1\npcomment2\nptag1: val1\nptag2: val2\n", - ptype=RegularPosting, + preal=RealPosting, ptags=[("ptag1","val1"),("ptag2","val2")], ptransaction=Nothing } diff --git a/hledger-lib/Hledger/Read/RulesReader.hs b/hledger-lib/Hledger/Read/RulesReader.hs index 1a68c09cd91..dc8d076495a 100644 --- a/hledger-lib/Hledger/Read/RulesReader.hs +++ b/hledger-lib/Hledger/Read/RulesReader.hs @@ -1459,7 +1459,7 @@ transactionFromCsvRecord timesarezoned mtzin tzout sourcepos rules record = ,pbalanceassertion = mkBalanceAssertion rules record <$> mbalance ,pcomment = cmt ,ptags = tags - ,ptype = accountNamePostingType acct + ,preal = accountNamePostingType acct } ] diff --git a/hledger-lib/Hledger/Read/TimedotReader.hs b/hledger-lib/Hledger/Read/TimedotReader.hs index 41c0377ddd3..1a293d2d190 100644 --- a/hledger-lib/Hledger/Read/TimedotReader.hs +++ b/hledger-lib/Hledger/Read/TimedotReader.hs @@ -190,7 +190,7 @@ timedotentryp = do ps = [ nullposting{paccount=a ,pamount=mixedAmount $ nullamt{acommodity=c, aquantity=hours, astyle=s} - ,ptype=VirtualPosting + ,preal=VirtualPosting ,pcomment=comment ,ptags=tags } diff --git a/hledger-lib/Hledger/Reports/MultiBalanceReport.hs b/hledger-lib/Hledger/Reports/MultiBalanceReport.hs index 595ccf3f26f..c189fd29a5c 100644 --- a/hledger-lib/Hledger/Reports/MultiBalanceReport.hs +++ b/hledger-lib/Hledger/Reports/MultiBalanceReport.hs @@ -195,7 +195,7 @@ getPostings rspec@ReportSpec{_rsQuery=query, _rsReportOpts=ropts} j priceoracle -- If we're re-valuing every period, we need to have the unvalued start -- balance, so we can do it ourselves later. ropts' = if isJust (valuationAfterSum ropts) - then ropts{period_=dateSpanAsPeriod fullreportspan, value_=Nothing, conversionop_=Just NoConversionOp} -- If we're valuing after the sum, don't do it now + then ropts{period_=dateSpanAsPeriod fullreportspan, value_=Nothing} -- If we're valuing after the sum, don't do it now else ropts{period_=dateSpanAsPeriod fullreportspan} -- q projected back before the report start date. diff --git a/hledger-lib/Hledger/Reports/ReportOptions.hs b/hledger-lib/Hledger/Reports/ReportOptions.hs index 3a6d03a81bd..7bd471784a6 100644 --- a/hledger-lib/Hledger/Reports/ReportOptions.hs +++ b/hledger-lib/Hledger/Reports/ReportOptions.hs @@ -695,18 +695,12 @@ journalApplyValuationFromOptsWith rspec@ReportSpec{_rsReportOpts=ropts} j priceo mixedAmountApplyValuationAfterSumFromOptsWith :: ReportOpts -> Journal -> PriceOracle -> (Day -> MixedAmount -> MixedAmount) mixedAmountApplyValuationAfterSumFromOptsWith ropts j priceoracle = - case valuationAfterSum ropts of - Just mc -> case balancecalc_ ropts of - CalcGain -> gain mc - _ -> \d -> valuation mc d . costing - Nothing -> const id - where - valuation mc d = mixedAmountValueAtDate priceoracle styles mc d - gain mc d = mixedAmountGainAtDate priceoracle styles mc d - costing = case fromMaybe NoConversionOp $ conversionop_ ropts of - NoConversionOp -> id - ToCost -> styleAmounts styles . mixedAmountCost - styles = journalCommodityStyles j + case valuationAfterSum ropts of + Nothing -> const id + Just mc -> case balancecalc_ ropts of + CalcGain -> mixedAmountGainAtDate priceoracle styles mc + _ -> mixedAmountValueAtDate priceoracle styles mc + where styles = journalCommodityStyles j -- | If the ReportOpts specify that we are performing valuation after summing amounts, -- return Just of the commodity symbol we're converting to, Just Nothing for the default, diff --git a/hledger-lib/Hledger/Write/Beancount.hs b/hledger-lib/Hledger/Write/Beancount.hs index c5859e65c87..5deb9828529 100644 --- a/hledger-lib/Hledger/Write/Beancount.hs +++ b/hledger-lib/Hledger/Write/Beancount.hs @@ -6,6 +6,7 @@ Helpers for beancount output. module Hledger.Write.Beancount ( showTransactionBeancount, + showPriceDirectiveBeancount, -- postingsAsLinesBeancount, -- postingAsLinesBeancount, -- showAccountNameBeancount, @@ -78,6 +79,17 @@ showTransactionBeancount t = case renderCommentLines (tcomment t) of [] -> ("",[]) c:cs -> (c,cs) +-- | Render a PriceDirective in Beancount format: DATE price COMMODITY AMOUNT +showPriceDirectiveBeancount :: PriceDirective -> Text +showPriceDirectiveBeancount pd = + showDate (pddate pd) + <> " price " + <> commodityToBeancount (pdcommodity pd) + <> " " + <> wbToText (showAmountB beancountPriceFmt $ amountToBeancount $ pdamount pd) + where + beancountPriceFmt = defaultFmt{ displayZeroCommodity=True, displayForceDecimalMark=True, displayQuotes=False } + nl = "\n" type BMetadata = Tag @@ -200,11 +212,16 @@ postingAsLinesBeancount elideamount acctwidth amtwidth p = -- amtwidth at all. shownAmounts | elideamount = [mempty] - | otherwise = map addCostBasis $ showMixedAmountLinesPartsB displayopts a' + | otherwise = map addCostBasisAndCost amtParts where - displayopts = defaultFmt{ displayZeroCommodity=True, displayForceDecimalMark=True, displayQuotes=False, displayCostBasis=False } + -- render amounts without cost or cost basis; we append them in beancount order (costbasis before cost) below + basefmt = defaultFmt{ displayZeroCommodity=True, displayForceDecimalMark=True, displayQuotes=False, displayCost=False, displayCostBasis=False } + costfmt = defaultFmt{ displayZeroCommodity=True, displayForceDecimalMark=True, displayQuotes=False } a' = mapMixedAmount amountToBeancount $ pamount p - addCostBasis (builder, amt) = builder <> showAmountCostBasisBeancountB displayopts amt + -- get the display builders (with costs stripped) paired with the original amounts (with costs intact) + amtParts = zip (map fst $ showMixedAmountLinesPartsB basefmt a') (amounts a') + addCostBasisAndCost (builder, amt) = + builder <> showAmountCostBasisBeancountB costfmt amt <> showAmountCostB costfmt amt thisamtwidth = maximumBound 0 $ map wbWidth shownAmounts -- when there is a balance assertion, show it only on the last posting line @@ -217,7 +234,10 @@ postingAsLinesBeancount elideamount acctwidth amtwidth p = -- pad to the maximum account name width, plus 2 to leave room for status flags, to keep amounts aligned statusandaccount = postingIndent . fitText (Just $ 2 + acctwidth) Nothing False True $ pstatusandacct p thisacctwidth = realLength pacct - mds = tagsToBeancountMetadata $ ptags p + mds = tagsToBeancountMetadata $ filter (tagInComment (pcomment p)) (ptags p) + tagInComment c (n,_) = case toRegex ("\\b" <> n <> ":") of + Right r -> regexMatchText r c + Left _ -> False metadatalines = map (postingIndent . showBeancountMetadata (Just maxtagnamewidth)) mds where maxtagnamewidth = maximum' $ map (T.length . fst) mds (samelinecomment, newlinecomments) = @@ -247,8 +267,11 @@ accountNameToBeancount a = b cs1 = map accountNameComponentToBeancount $ accountNameComponents $ dbg9 "hledger account name " a + cs1' = case cs1 of + (c:cs) | T.toLower c `elem` ["revenue", "revenues"] -> "Income":cs + cs -> cs cs2 = - case cs1 of + case cs1' of c:_ | c `notElem` beancountTopLevelAccounts -> error' e where e = T.unpack $ T.unlines [ @@ -322,6 +345,8 @@ amountToBeancount a@Amount{acommodity=c,astyle=s,acost=mp} = a{acommodity=c', as showAmountCostBasisBeancountB :: AmountFormat -> Amount -> WideBuilder showAmountCostBasisBeancountB afmt amt = case acostbasis amt of Nothing -> mempty + Just CostBasis{cbCost=Nothing, cbDate=Nothing, cbLabel=Nothing} -> + WideBuilder (TB.fromString " {}") 3 Just CostBasis{cbCost, cbDate, cbLabel} -> case parts of [] -> mempty @@ -347,10 +372,11 @@ type BeancountCommoditySymbol = CommoditySymbol -- replaces spaces with dashes and other invalid characters with C, -- prepends a C if the first character is not a letter, -- appends a C if the last character is not a letter or digit, +-- doubles it if less than 2 characters, -- and disables hledger's enclosing double quotes. -- -- >>> commodityToBeancount "" --- "C" +-- "CC" -- >>> commodityToBeancount "$" -- "USD" -- >>> commodityToBeancount "Usd" @@ -359,8 +385,11 @@ type BeancountCommoditySymbol = CommoditySymbol -- "A1" -- >>> commodityToBeancount "\"A 1!\"" -- "A-1C21" +-- >>> commodityToBeancount "K" +-- "KK" -- commodityToBeancount :: CommoditySymbol -> BeancountCommoditySymbol +commodityToBeancount "" = "CC" commodityToBeancount com = dbg9 "beancount commodity name" $ let com' = stripquotes com @@ -372,6 +401,7 @@ commodityToBeancount com = & T.concatMap (\d -> if isBeancountCommodityChar d then T.singleton d else T.pack $ charToBeancount d) & fixstart & fixend + & fixshort where fixstart bcom = case T.uncons bcom of Just (c,_) | isBeancountCommodityStartChar c -> bcom @@ -379,6 +409,9 @@ commodityToBeancount com = fixend bcom = case T.unsnoc bcom of Just (_,c) | isBeancountCommodityEndChar c -> bcom _ -> bcom <> "C" + fixshort bcom + | T.length bcom < 2 = bcom <> bcom -- e.g. "K" -> "KK" + | otherwise = bcom -- | Is this a valid character in the middle of a Beancount commodity name (a capital letter, digit, or '._-) ? isBeancountCommodityChar :: Char -> Bool diff --git a/hledger-lib/Hledger/Write/Ledger.hs b/hledger-lib/Hledger/Write/Ledger.hs new file mode 100644 index 00000000000..e4b3238e955 --- /dev/null +++ b/hledger-lib/Hledger/Write/Ledger.hs @@ -0,0 +1,40 @@ +{-| +Helpers for Ledger-compatible output. +-} + +{-# LANGUAGE OverloadedStrings #-} + +module Hledger.Write.Ledger ( + showTransactionLedger, +) +where + +import Data.Text (Text) +import Data.Text qualified as T +import Data.Text.Lazy qualified as TL +import Data.Text.Lazy.Builder qualified as TB + +import Hledger.Data.Amount (defaultFmt, AmountFormat(..)) +import Hledger.Data.Posting (postingsAsLines, renderCommentLines) +import Hledger.Data.Transaction (showTransactionLineFirstPart) +import Hledger.Data.Types (Transaction(..), tdescription) + +ledgerFmt :: AmountFormat +ledgerFmt = defaultFmt{displayLedgerLotSyntax = True} + +-- | Like showTransaction, but renders cost basis using Ledger-style lot syntax +-- ({COST} [DATE] (LABEL)) instead of hledger consolidated syntax. +showTransactionLedger :: Transaction -> Text +showTransactionLedger t = + TL.toStrict . TB.toLazyText $ + TB.fromText descriptionline <> newline + <> foldMap ((<> newline) . TB.fromText) newlinecomments + <> foldMap ((<> newline) . TB.fromText) (postingsAsLines ledgerFmt False $ tpostings t) + <> newline + where + descriptionline = T.stripEnd $ showTransactionLineFirstPart t <> T.concat [desc, samelinecomment] + desc = if T.null d then "" else " " <> d where d = tdescription t + (samelinecomment, newlinecomments) = + case renderCommentLines (tcomment t) of [] -> ("",[]) + c:cs -> (c,cs) + newline = TB.singleton '\n' diff --git a/hledger-lib/hledger-lib.cabal b/hledger-lib/hledger-lib.cabal index 866755f2577..2ec93a62cfa 100644 --- a/hledger-lib/hledger-lib.cabal +++ b/hledger-lib/hledger-lib.cabal @@ -5,7 +5,7 @@ cabal-version: 2.2 -- see: https://github.com/sol/hpack name: hledger-lib -version: 1.52.99 +version: 1.99 synopsis: A library providing the core functionality of hledger description: This library contains hledger's core functionality. It is used by most hledger* packages so that they support the same @@ -56,6 +56,7 @@ library Hledger.Data Hledger.Data.Account Hledger.Data.AccountName + Hledger.Data.AccountType Hledger.Data.Amount Hledger.Data.BalanceData Hledger.Data.Balancing @@ -69,6 +70,7 @@ library Hledger.Data.JournalChecks.Uniqueleafnames Hledger.Data.Json Hledger.Data.Ledger + Hledger.Data.Lots Hledger.Data.Period Hledger.Data.PeriodData Hledger.Data.PeriodicTransaction @@ -113,6 +115,7 @@ library Hledger.Write.Html.Blaze Hledger.Write.Html.HtmlCommon Hledger.Write.Html.Lucid + Hledger.Write.Ledger Hledger.Write.Ods Hledger.Write.Spreadsheet Text.Tabular.AsciiWide @@ -124,7 +127,7 @@ library hs-source-dirs: ./ ghc-options: -Wall -Wno-incomplete-uni-patterns -Wno-missing-signatures -Wno-orphans -Wno-type-defaults -Wno-unused-do-bind - cpp-options: -DVERSION="1.52.99" + cpp-options: -DVERSION="1.99" build-depends: Decimal >=0.5.1 , Glob >=0.9 @@ -184,7 +187,7 @@ test-suite doctest hs-source-dirs: test ghc-options: -Wall -Wno-incomplete-uni-patterns -Wno-missing-signatures -Wno-orphans -Wno-type-defaults -Wno-unused-do-bind - cpp-options: -DVERSION="1.52.99" + cpp-options: -DVERSION="1.99" build-depends: Decimal >=0.5.1 , Glob >=0.7 @@ -245,7 +248,7 @@ test-suite unittest hs-source-dirs: test ghc-options: -Wall -Wno-incomplete-uni-patterns -Wno-missing-signatures -Wno-orphans -Wno-type-defaults -Wno-unused-do-bind - cpp-options: -DVERSION="1.52.99" + cpp-options: -DVERSION="1.99" build-depends: Decimal >=0.5.1 , Glob >=0.9 diff --git a/hledger-lib/package.yaml b/hledger-lib/package.yaml index 7f25d5d1cfa..d266ba16c15 100644 --- a/hledger-lib/package.yaml +++ b/hledger-lib/package.yaml @@ -1,5 +1,5 @@ name: hledger-lib -version: 1.52.99 +version: 1.99 license: GPL-3.0-or-later maintainer: Simon Michael author: Simon Michael @@ -95,7 +95,7 @@ dependencies: - extra >=1.7.11 - Glob >= 0.9 -cpp-options: -DVERSION="1.52.99" +cpp-options: -DVERSION="1.99" language: GHC2021 @@ -125,6 +125,7 @@ library: # - Hledger.Data # - Hledger.Data.Account # - Hledger.Data.AccountName + # - Hledger.Data.AccountType # - Hledger.Data.Amount # - Hledger.Data.BalanceData # - Hledger.Data.Balancing @@ -137,6 +138,7 @@ library: # - Hledger.Data.JournalChecks.Uniqueleafnames # - Hledger.Data.Json # - Hledger.Data.Ledger + # - Hledger.Data.Lots # - Hledger.Data.Period # - Hledger.Data.PeriodData # - Hledger.Data.PeriodicTransaction @@ -159,6 +161,7 @@ library: # - Hledger.Read.TimeclockReader # - Hledger.Write.Beancount # - Hledger.Write.Csv + # - Hledger.Write.Ledger # - Hledger.Write.Ods # - Hledger.Write.Html # - Hledger.Write.Html.Attribute diff --git a/hledger-ui/.date.m4 b/hledger-ui/.date.m4 index 173ae01a69b..e510e9c9f95 100644 --- a/hledger-ui/.date.m4 +++ b/hledger-ui/.date.m4 @@ -1,2 +1,2 @@ m4_dnl Date to show in man pages. Updated by "Shake manuals" -m4_define({{_monthyear_}}, {{December 2025}})m4_dnl +m4_define({{_monthyear_}}, {{February 2026}})m4_dnl diff --git a/hledger-ui/.version b/hledger-ui/.version index 0c353043ed6..3e9926483a2 100644 --- a/hledger-ui/.version +++ b/hledger-ui/.version @@ -1 +1 @@ -1.52.99 +1.99 diff --git a/hledger-ui/.version.m4 b/hledger-ui/.version.m4 index de014a84661..4012f693a80 100644 --- a/hledger-ui/.version.m4 +++ b/hledger-ui/.version.m4 @@ -1,2 +1,2 @@ m4_dnl Version number to show in manuals. Updated by "Shake setversion" -m4_define({{_version_}}, {{1.52.99}})m4_dnl +m4_define({{_version_}}, {{1.99}})m4_dnl diff --git a/hledger-ui/hledger-ui.1 b/hledger-ui/hledger-ui.1 index 4b5a9f29ea8..d05ae797add 100644 --- a/hledger-ui/hledger-ui.1 +++ b/hledger-ui/hledger-ui.1 @@ -1,5 +1,5 @@ -.TH "HLEDGER\-UI" "1" "December 2025" "hledger-ui-1.51.99 " "hledger User Manuals" +.TH "HLEDGER\-UI" "1" "February 2026" "hledger-ui-1.51.99 " "hledger User Manuals" diff --git a/hledger-ui/hledger-ui.cabal b/hledger-ui/hledger-ui.cabal index d2d37a10bbd..2cb2a216619 100644 --- a/hledger-ui/hledger-ui.cabal +++ b/hledger-ui/hledger-ui.cabal @@ -5,7 +5,7 @@ cabal-version: 2.2 -- see: https://github.com/sol/hpack name: hledger-ui -version: 1.52.99 +version: 1.99 synopsis: Terminal interface for the hledger accounting system description: A simple terminal user interface for the hledger accounting system. It can be a more convenient way to browse your accounts than the CLI. @@ -69,7 +69,7 @@ library hs-source-dirs: ./ ghc-options: -Wall -Wno-incomplete-uni-patterns -Wno-missing-signatures -Wno-orphans -Wno-type-defaults -Wno-unused-do-bind - cpp-options: -DVERSION="1.52.99" + cpp-options: -DVERSION="1.99" build-depends: ansi-terminal >=0.9 , async @@ -84,8 +84,8 @@ library , filepath , fsnotify >=0.4.2.0 && <0.5 , githash >=0.1.6.2 - , hledger >=1.52.99 && <1.53 - , hledger-lib >=1.52.99 && <1.53 + , hledger ==1.99.* + , hledger-lib ==1.99.* , megaparsec >=7.0.0 && <9.8 , microlens >=0.4 , microlens-platform >=0.2.3.1 @@ -100,7 +100,7 @@ library , transformers , vector , vty >=6.1 && <6.6 - , vty-crossplatform >=0.4.0.0 && <0.5.0.0 + , vty-crossplatform >=0.4.0.0 && <0.6.0.0 default-language: GHC2021 if (flag(debug)) cpp-options: -DDEBUG @@ -120,7 +120,7 @@ executable hledger-ui hs-source-dirs: app ghc-options: -Wall -Wno-incomplete-uni-patterns -Wno-missing-signatures -Wno-orphans -Wno-type-defaults -Wno-unused-do-bind -threaded - cpp-options: -DVERSION="1.52.99" + cpp-options: -DVERSION="1.99" build-depends: base >=4.18 && <4.23 , hledger-ui diff --git a/hledger-ui/hledger-ui.txt b/hledger-ui/hledger-ui.txt index c0ee7392983..d60bece4d2c 100644 --- a/hledger-ui/hledger-ui.txt +++ b/hledger-ui/hledger-ui.txt @@ -460,4 +460,4 @@ LICENSE SEE ALSO hledger(1), hledger-ui(1), hledger-web(1), ledger(1) -hledger-ui-1.51.99 December 2025 HLEDGER-UI(1) +hledger-ui-1.51.99 February 2026 HLEDGER-UI(1) diff --git a/hledger-ui/package.yaml b/hledger-ui/package.yaml index 257bde42f5f..c2f951d8a5c 100644 --- a/hledger-ui/package.yaml +++ b/hledger-ui/package.yaml @@ -1,5 +1,5 @@ name: hledger-ui -version: 1.52.99 +version: 1.99 license: GPL-3.0-or-later maintainer: Simon Michael author: Simon Michael @@ -41,7 +41,7 @@ flags: dependencies: - base >=4.18 && <4.23 -cpp-options: -DVERSION="1.52.99" +cpp-options: -DVERSION="1.99" language: GHC2021 @@ -68,8 +68,8 @@ library: # default: All modules in source-dirs less other-modules less modules mentioned in when # exposed-modules: dependencies: - - hledger-lib >=1.52.99 && <1.53 - - hledger >=1.52.99 && <1.53 + - hledger-lib >=1.99 && <1.100 + - hledger >=1.99 && <1.100 - ansi-terminal >=0.9 - async - cmdargs >=0.8 @@ -96,7 +96,7 @@ library: - vector - brick >=2.1.1 && <2.3.2 || >2.3.2 && < 2.11 - vty >=6.1 && <6.6 - - vty-crossplatform >= 0.4.0.0 && < 0.5.0.0 + - vty-crossplatform >= 0.4.0.0 && < 0.6.0.0 when: - condition: os(windows) then: diff --git a/hledger-web/.date.m4 b/hledger-web/.date.m4 index 173ae01a69b..e510e9c9f95 100644 --- a/hledger-web/.date.m4 +++ b/hledger-web/.date.m4 @@ -1,2 +1,2 @@ m4_dnl Date to show in man pages. Updated by "Shake manuals" -m4_define({{_monthyear_}}, {{December 2025}})m4_dnl +m4_define({{_monthyear_}}, {{February 2026}})m4_dnl diff --git a/hledger-web/.version b/hledger-web/.version index 0c353043ed6..3e9926483a2 100644 --- a/hledger-web/.version +++ b/hledger-web/.version @@ -1 +1 @@ -1.52.99 +1.99 diff --git a/hledger-web/.version.m4 b/hledger-web/.version.m4 index de014a84661..4012f693a80 100644 --- a/hledger-web/.version.m4 +++ b/hledger-web/.version.m4 @@ -1,2 +1,2 @@ m4_dnl Version number to show in manuals. Updated by "Shake setversion" -m4_define({{_version_}}, {{1.52.99}})m4_dnl +m4_define({{_version_}}, {{1.99}})m4_dnl diff --git a/hledger-web/Hledger/Web/Test.hs b/hledger-web/Hledger/Web/Test.hs index 8992de9988f..2048264679e 100644 --- a/hledger-web/Hledger/Web/Test.hs +++ b/hledger-web/Hledger/Web/Test.hs @@ -129,7 +129,7 @@ hledgerWebTest = do usecolor <- useColorOnStdout let rawopts = [("forecast","")] - iopts = rawOptsToInputOpts d usecolor True $ mkRawOpts rawopts + iopts = rawOptsToInputOpts d usecolor $ mkRawOpts rawopts f = "fake" -- need a non-null filename so forecast transactions get index 0 pj <- readJournal'' (T.pack $ unlines -- PARTIAL: readJournal'' should not fail ["~ monthly" diff --git a/hledger-web/Hledger/Web/Widget/AddForm.hs b/hledger-web/Hledger/Web/Widget/AddForm.hs index 2972ef770cd..a4835017bbb 100644 --- a/hledger-web/Hledger/Web/Widget/AddForm.hs +++ b/hledger-web/Hledger/Web/Widget/AddForm.hs @@ -137,7 +137,7 @@ validatePostings acctsRes amtsRes = let zipRow (Left e) (Left e') = Left (Just e, Just e') zipRow (Left e) (Right _) = Left (Just e, Nothing) zipRow (Right _) (Left e) = Left (Nothing, Just e) - zipRow (Right acct') (Right amt) = Right (nullposting {paccount = acct, ptype = atype, pamount = mixedAmount amt}) + zipRow (Right acct') (Right amt) = Right (nullposting {paccount = acct, preal = atype, pamount = mixedAmount amt}) where acct = accountNameWithoutPostingType acct' atype = accountNamePostingType acct' diff --git a/hledger-web/hledger-web.1 b/hledger-web/hledger-web.1 index 4f0e334938b..d47673a6df2 100644 --- a/hledger-web/hledger-web.1 +++ b/hledger-web/hledger-web.1 @@ -1,5 +1,5 @@ -.TH "HLEDGER\-WEB" "1" "December 2025" "hledger-web-1.51.99 " "hledger User Manuals" +.TH "HLEDGER\-WEB" "1" "February 2026" "hledger-web-1.51.99 " "hledger User Manuals" diff --git a/hledger-web/hledger-web.cabal b/hledger-web/hledger-web.cabal index d0d7a1179e2..d5bddae1f07 100644 --- a/hledger-web/hledger-web.cabal +++ b/hledger-web/hledger-web.cabal @@ -5,7 +5,7 @@ cabal-version: 2.2 -- see: https://github.com/sol/hpack name: hledger-web -version: 1.52.99 +version: 1.99 synopsis: Web user interface for the hledger accounting system description: A simple web user interface for the hledger accounting system, providing a more modern UI than the command-line or terminal interfaces. @@ -154,7 +154,7 @@ library hs-source-dirs: ./ ghc-options: -Wall -Wno-incomplete-uni-patterns -Wno-missing-signatures -Wno-orphans -Wno-type-defaults -Wno-unused-do-bind -threaded - cpp-options: -DVERSION="1.52.99" + cpp-options: -DVERSION="1.99" build-depends: Decimal >=0.5.1 , aeson >=1 && <2.3 @@ -176,8 +176,8 @@ library , filepath , githash >=0.1.6.2 , hjsmin - , hledger >=1.52.99 && <1.53 - , hledger-lib >=1.52.99 && <1.53 + , hledger ==1.99.* + , hledger-lib ==1.99.* , hspec , http-client , http-conduit @@ -222,7 +222,7 @@ executable hledger-web hs-source-dirs: app ghc-options: -Wall -Wno-incomplete-uni-patterns -Wno-missing-signatures -Wno-orphans -Wno-type-defaults -Wno-unused-do-bind -threaded - cpp-options: -DVERSION="1.52.99" + cpp-options: -DVERSION="1.99" build-depends: base >=4.18 && <4.23 , hledger-web @@ -242,7 +242,7 @@ test-suite apptest hs-source-dirs: test ghc-options: -Wall -Wno-incomplete-uni-patterns -Wno-missing-signatures -Wno-orphans -Wno-type-defaults -Wno-unused-do-bind -threaded - cpp-options: -DVERSION="1.52.99" + cpp-options: -DVERSION="1.99" build-depends: base >=4.18 && <4.23 , hledger-web diff --git a/hledger-web/hledger-web.txt b/hledger-web/hledger-web.txt index add5d331fe9..d485218ecec 100644 --- a/hledger-web/hledger-web.txt +++ b/hledger-web/hledger-web.txt @@ -480,4 +480,4 @@ LICENSE SEE ALSO hledger(1), hledger-ui(1), hledger-web(1), ledger(1) -hledger-web-1.51.99 December 2025 HLEDGER-WEB(1) +hledger-web-1.51.99 February 2026 HLEDGER-WEB(1) diff --git a/hledger-web/package.yaml b/hledger-web/package.yaml index c57bdd9b4a5..6dabf7af7dc 100644 --- a/hledger-web/package.yaml +++ b/hledger-web/package.yaml @@ -1,5 +1,5 @@ name: hledger-web -version: 1.52.99 +version: 1.99 license: GPL-3.0-or-later maintainer: Simon Michael author: Simon Michael @@ -67,7 +67,7 @@ flags: dependencies: - base >=4.18 && <4.23 -cpp-options: -DVERSION="1.52.99" +cpp-options: -DVERSION="1.99" language: GHC2021 @@ -103,8 +103,8 @@ library: # default: All modules in source-dirs less other-modules less modules mentioned in when # exposed-modules: dependencies: - - hledger-lib >=1.52.99 && <1.53 - - hledger >=1.52.99 && <1.53 + - hledger-lib >=1.99 && <1.100 + - hledger >=1.99 && <1.100 - aeson >=1 && <2.3 - base64 - blaze-html diff --git a/hledger.conf b/hledger.conf index dbd433dab0f..36b4f312389 100644 --- a/hledger.conf +++ b/hledger.conf @@ -15,3 +15,6 @@ #--debug # this doesn't work in config files yet + +# always hide explicit lots by default; --lots will show them (or re-add them..) +#--alias '/:{.*/=' diff --git a/hledger/.date.m4 b/hledger/.date.m4 index 173ae01a69b..e510e9c9f95 100644 --- a/hledger/.date.m4 +++ b/hledger/.date.m4 @@ -1,2 +1,2 @@ m4_dnl Date to show in man pages. Updated by "Shake manuals" -m4_define({{_monthyear_}}, {{December 2025}})m4_dnl +m4_define({{_monthyear_}}, {{February 2026}})m4_dnl diff --git a/hledger/.version b/hledger/.version index 0c353043ed6..3e9926483a2 100644 --- a/hledger/.version +++ b/hledger/.version @@ -1 +1 @@ -1.52.99 +1.99 diff --git a/hledger/.version.m4 b/hledger/.version.m4 index de014a84661..4012f693a80 100644 --- a/hledger/.version.m4 +++ b/hledger/.version.m4 @@ -1,2 +1,2 @@ m4_dnl Version number to show in manuals. Updated by "Shake setversion" -m4_define({{_version_}}, {{1.52.99}})m4_dnl +m4_define({{_version_}}, {{1.99}})m4_dnl diff --git a/hledger/Hledger/Cli/CliOptions.hs b/hledger/Hledger/Cli/CliOptions.hs index 5f8734edbd5..576772aba6a 100644 --- a/hledger/Hledger/Cli/CliOptions.hs +++ b/hledger/Hledger/Cli/CliOptions.hs @@ -182,6 +182,7 @@ inputflags = [ ]) ,flagNone ["infer-costs"] (setboolopt "infer-costs") "infer conversion equity postings from costs" ,flagNone ["infer-equity"] (setboolopt "infer-equity") "infer costs from conversion equity postings" + ,flagNone ["lots"] (setboolopt "lots") "calculate and show lots" -- history of this flag so far, lest we be confused: -- originally --infer-value -- 2021-02 --infer-market-price added, --infer-value deprecated @@ -637,11 +638,8 @@ rawOptsToCliOpts rawopts = do Just d -> either (const err) fromEFDay $ fixSmartDateStrEither' currentDay (T.pack d) where err = error' $ "Unable to parse date \"" ++ d ++ "\"" command = stringopt "command" rawopts - moutputformat = maybestringopt "output-format" rawopts - -- if printing beancount format, don't propagate account and commodity tags to postings - autopostingtags = not $ command == "print" && moutputformat == Just "beancount" usecolor <- useColorOnStdout - let iopts = rawOptsToInputOpts day usecolor autopostingtags rawopts + let iopts = rawOptsToInputOpts day usecolor rawopts rspec <- either error' pure $ rawOptsToReportSpec day usecolor rawopts -- PARTIAL: mtermwidth <- getTerminalWidth let availablewidth = fromMaybe defaultWidth mtermwidth @@ -652,7 +650,7 @@ rawOptsToCliOpts rawopts = do ,inputopts_ = iopts ,reportspec_ = rspec ,output_file_ = maybestringopt "output-file" rawopts - ,output_format_ = moutputformat + ,output_format_ = maybestringopt "output-format" rawopts ,pageropt_ = maybeynopt "pager" rawopts ,coloropt_ = maybeynaopt "color" rawopts ,debug_ = posintopt "debug" rawopts @@ -790,6 +788,16 @@ outputFormatFromOpts opts = Just ext | ext `elem` outputFormats -> ext _ -> defaultOutputFormat +-- | Like outputFormatFromOpts, but works on RawOpts (before CliOpts are constructed). +outputFormatFromRawOpts :: RawOpts -> String +outputFormatFromRawOpts rawopts = + case maybestringopt "output-format" rawopts of + Just f -> f + Nothing -> + case filePathExtension <$> maybestringopt "output-file" rawopts of + Just ext | ext `elem` outputFormats -> ext + _ -> defaultOutputFormat + -- -- | Get the file name without its last extension, from a file path. -- filePathBaseFileName :: FilePath -> String -- filePathBaseFileName = fst . splitExtension . snd . splitFileName diff --git a/hledger/Hledger/Cli/Commands.hs b/hledger/Hledger/Cli/Commands.hs index 27b7d1822dc..37dc916244f 100644 --- a/hledger/Hledger/Cli/Commands.hs +++ b/hledger/Hledger/Cli/Commands.hs @@ -534,7 +534,7 @@ tests_Commands = testGroup "Commands" [ j <- readJournal'' "apply account test\n2008/12/07 One\n (from) $-1\n (to) $1\n" -- PARTIAL: let p = headErr $ tpostings $ headErr $ jtxns j -- PARTIAL headErrs succeed because txns & postings provided paccount p @?= "test:from" - ptype p @?= VirtualPosting + preal p @?= VirtualPosting ] ,testCase "alias directive" $ do diff --git a/hledger/Hledger/Cli/Commands/Add.hs b/hledger/Hledger/Cli/Commands/Add.hs index b0a74c597d5..82468691e6f 100644 --- a/hledger/Hledger/Cli/Commands/Add.hs +++ b/hledger/Hledger/Cli/Commands/Add.hs @@ -246,7 +246,7 @@ transactionWizard previnput state@AddState{..} stack@(currentStage : _) = case c p = nullposting{paccount=T.pack $ stripbrackets account ,pamount=mixedamt ,pcomment=T.dropAround isNewline comment - ,ptype=accountNamePostingType $ T.pack account + ,preal=accountNamePostingType $ T.pack account ,pbalanceassertion = assertion ,pdate=pdate1 ,pdate2=pdate2 @@ -338,7 +338,7 @@ accountWizard :: PrevInput -> AddState -> Wizard Haskeline (Maybe String) accountWizard PrevInput{..} AddState{..} = do let pnum = length asPostings + 1 historicalp = fmap ((!! (pnum - 1)) . (++ (repeat nullposting)) . tpostings) asSimilarTransaction - historicalacct = case historicalp of Just p -> showAccountName Nothing (ptype p) (paccount p) + historicalacct = case historicalp of Just p -> showAccountName Nothing (preal p) (paccount p) Nothing -> "" def = headDef (T.unpack historicalacct) asArgs endmsg | canfinish && null def = " (or . or enter to finish this transaction)" diff --git a/hledger/Hledger/Cli/Commands/Check.hs b/hledger/Hledger/Cli/Commands/Check.hs index 205c8b7a6af..56710af9f9f 100644 --- a/hledger/Hledger/Cli/Commands/Check.hs +++ b/hledger/Hledger/Cli/Commands/Check.hs @@ -62,6 +62,7 @@ data Check = | Commodities | Accounts -- done when specified with the check command + | Lots | Ordereddates | Payees | Tags @@ -89,7 +90,7 @@ parseCheckArgument s = (checkname:checkargs) = words' s -- XXX do all of these print on stderr ? --- | Run the named error check, possibly with some arguments, +-- | Run the named error check, possibly with some arguments, -- on this journal with these options. runCheck :: CliOpts -> Journal -> (Check,[String]) -> IO () runCheck _opts j (chck,_) = do @@ -102,6 +103,7 @@ runCheck _opts j (chck,_) = do Assertions -> Right () Accounts -> journalCheckAccounts j Commodities -> journalCheckCommodities j + Lots -> journalCheckLots j Ordereddates -> journalCheckOrdereddates j Payees -> journalCheckPayees j Recentassertions -> journalCheckRecentAssertions j diff --git a/hledger/Hledger/Cli/Commands/Check.md b/hledger/Hledger/Cli/Commands/Check.md index d12d14ce80c..ce725720fc7 100644 --- a/hledger/Hledger/Cli/Commands/Check.md +++ b/hledger/Hledger/Cli/Commands/Check.md @@ -58,11 +58,19 @@ These provide extra error-catching power to help you keep your data clean and co - **accounts** - all account names used must be [declared](#account-error-checking). This prevents the use of mis-spelled or outdated account names. + (Except lot subaccounts, like `:{2026-01-15, $50}`, which are automatically exempt; + only their base account needs to be declared.) ### Other checks These are not wanted by everyone, but can be run using the `check` command: +- **lots** - all lot tracking calculations succeed. + Checks lots tag values on declarations, lot posting classification, + calculation of lot movements, capital gains, + and correct balancing of disposal transactions. + Stops at the first error. + - **tags** - all tags used must be [declared](#tag-directive). This prevents mis-spelled tag names. Note hledger fairly often finds unintended tags in comments. diff --git a/hledger/Hledger/Cli/Commands/Check.txt b/hledger/Hledger/Cli/Commands/Check.txt index 0de53e33311..6a3fa8ce3b6 100644 --- a/hledger/Hledger/Cli/Commands/Check.txt +++ b/hledger/Hledger/Cli/Commands/Check.txt @@ -59,7 +59,9 @@ correct: against mistyping or omitting commodity symbols. - accounts - all account names used must be declared. This prevents the - use of mis-spelled or outdated account names. + use of mis-spelled or outdated account names. (Except lot subaccounts, + like :{2026-01-15, $50}, which are automatically exempt; only their + base account needs to be declared.) Other checks diff --git a/hledger/Hledger/Cli/Commands/Close.hs b/hledger/Hledger/Cli/Commands/Close.hs index 052cdbd2553..525388fdc1d 100644 --- a/hledger/Hledger/Cli/Commands/Close.hs +++ b/hledger/Hledger/Cli/Commands/Close.hs @@ -134,6 +134,9 @@ close CliOpts{rawopts_=rawopts, reportspec_=rspec0} j = do -- amounts in opening/closing transactions should be too (#941, #1137) precise = amountSetFullPrecision + -- does this account have a lot subaccount component (e.g. assets:stocks:{2026-01-15, $50}) ? + isLotSubaccount a = lotBaseAccount a /= a + -- interleave equity postings next to the corresponding closing posting, or put them all at the end ? interleaved = boolopt "interleaved" rawopts @@ -164,8 +167,10 @@ close CliOpts{rawopts_=rawopts, reportspec_=rspec0} j = do ,pamount = mixedAmount $ precise b{aquantity=0, acost=Nothing} -- after each commodity's last posting, assert 0 balance (#1035) -- balance assertion amounts are unpriced (#824) + -- lot subaccounts are skipped: assertions run before lot calculation, + -- so the subaccount balance would be wrong when re-read ,pbalanceassertion = - if islast + if islast && not (isLotSubaccount a) then Just assertion{baamount=precise b} else Nothing } @@ -184,8 +189,9 @@ close CliOpts{rawopts_=rawopts, reportspec_=rspec0} j = do ,pamount = mixedAmount . precise $ negate b -- after each commodity's last posting, assert 0 balance (#1035) -- balance assertion amounts are unpriced (#824) + -- lot subaccounts are skipped (see Assert branch above) ,pbalanceassertion = - if islast + if islast && not (isLotSubaccount a) then Just assertion{baamount=precise b{aquantity=0, acost=Nothing}} else Nothing } diff --git a/hledger/Hledger/Cli/Commands/Close.md b/hledger/Hledger/Cli/Commands/Close.md index a00b19a645c..8f05bf55436 100644 --- a/hledger/Hledger/Cli/Commands/Close.md +++ b/hledger/Hledger/Cli/Commands/Close.md @@ -160,6 +160,9 @@ or restored to their previous balances in an opening transaction. These provide useful error checking, but you can ignore them temporarily with `-I`, or remove them if you prefer. +With `--lots`, balance assertions are not generated for the lot subaccount postings in +closing transactions (assertions on lot postings get confusing, because they apply to the parent account). + Single-commodity, subaccount-exclusive balance assertions (`=`) are generated by default. This can be changed with `--assertion-type='==*'` (eg). diff --git a/hledger/Hledger/Cli/Commands/Close.txt b/hledger/Hledger/Cli/Commands/Close.txt index d42b3777622..ec298c7c6f7 100644 --- a/hledger/Hledger/Cli/Commands/Close.txt +++ b/hledger/Hledger/Cli/Commands/Close.txt @@ -175,6 +175,10 @@ balances in an opening transaction. These provide useful error checking, but you can ignore them temporarily with -I, or remove them if you prefer. +With --lots, balance assertions are not generated for the lot subaccount +postings in closing transactions (assertions on lot postings get +confusing, because they apply to the parent account). + Single-commodity, subaccount-exclusive balance assertions (=) are generated by default. This can be changed with --assertion-type='==*' (eg). diff --git a/hledger/Hledger/Cli/Commands/Commodities.hs b/hledger/Hledger/Cli/Commands/Commodities.hs index 507dd602c25..77a2bc58ce6 100644 --- a/hledger/Hledger/Cli/Commands/Commodities.hs +++ b/hledger/Hledger/Cli/Commands/Commodities.hs @@ -39,7 +39,7 @@ commoditiesmode = hledgerCommandMode commodities :: CliOpts -> Journal -> IO () commodities opts@CliOpts{rawopts_=rawopts, reportspec_=ReportSpec{_rsQuery=query}} j = do let - filt = filter (matchesCommodity query) + filt = filter (matchesCommodityExtra (journalCommodityTags j) query) used = dbg5 "used" $ S.toList $ journalCommoditiesFromPriceDirectives j <> journalCommoditiesFromTransactions j declared' = dbg5 "declared" $ M.keys $ jdeclaredcommodities j unused = dbg5 "unused" $ declared' \\ used diff --git a/hledger/Hledger/Cli/Commands/Commodities.md b/hledger/Hledger/Cli/Commands/Commodities.md index 0c40df91796..93d08660450 100644 --- a/hledger/Hledger/Cli/Commands/Commodities.md +++ b/hledger/Hledger/Cli/Commands/Commodities.md @@ -21,4 +21,5 @@ or used but not declared, or declared but not used, or just the first one matched by a pattern (with `--find`, returning a non-zero exit code if it fails). -You can add `cur:` [query arguments](#queries) to further limit the commodities. +You can add [query arguments](#queries) to further limit the commodities; +at least `cur:` and `tag:` are supported. diff --git a/hledger/Hledger/Cli/Commands/Print.hs b/hledger/Hledger/Cli/Commands/Print.hs index 448ddffec37..5ef2d7cf63a 100644 --- a/hledger/Hledger/Cli/Commands/Print.hs +++ b/hledger/Hledger/Cli/Commands/Print.hs @@ -20,7 +20,7 @@ where import Data.Function ((&)) -import Data.List (intersperse, intercalate) +import Data.List (find, intersperse, intercalate, sortOn) import Data.List.Extra (nubSort) import Data.Text (Text) import Data.Map (Map) @@ -33,7 +33,8 @@ import Safe (lastMay, minimumDef) import System.Console.CmdArgs.Explicit import Hledger -import Hledger.Write.Beancount (accountNameToBeancount, showTransactionBeancount, showBeancountMetadata) +import Hledger.Write.Beancount (accountNameToBeancount, showTransactionBeancount, showBeancountMetadata, showPriceDirectiveBeancount) +import Hledger.Write.Ledger (showTransactionLedger) import Hledger.Write.Csv (CSV, printCSV, printTSV) import Hledger.Write.Ods (printFods) import Hledger.Write.Html.Lucid (styledTableHtml) @@ -59,7 +60,7 @@ printmode = hledgerCommandMode ,roundFlag ,flagReq ["base-url"] (\s opts -> Right $ setopt "base-url" s opts) "URLPREFIX" "in html output, generate links to hledger-web, with this prefix. (Usually the base url shown by hledger-web; can also be relative.)" - ,outputFormatFlag ["txt","beancount","csv","tsv","html","fods","json","sql"] + ,outputFormatFlag ["txt","ledger","beancount","csv","tsv","html","fods","json","sql"] ,outputFileFlag ]) cligeneralflagsgroups1 @@ -134,12 +135,14 @@ printEntries opts@CliOpts{rawopts_=rawopts, reportspec_=rspec} j = where -- print does user-specified rounding or (by default) no rounding, in all output formats styles = amountStylesSetRoundingFromRawOpts rawopts $ journalCommodityStyles j + styledPrices = map (\pd -> pd{pdamount = styleAmounts styles $ pdamount pd}) $ jpricedirectives j fmt = outputFormatFromOpts opts baseUrl = balance_base_url_ $ _rsReportOpts rspec query = querystring_ $ _rsReportOpts rspec render | fmt=="txt" = entriesReportAsText . styleAmounts styles . map maybeoriginalamounts - | fmt=="beancount" = entriesReportAsBeancount (jdeclaredaccounttags j) . styleAmounts styles . map maybeoriginalamounts + | fmt=="ledger" = entriesReportAsTextHelper showTransactionLedger . styleAmounts styles . map maybeoriginalamounts + | fmt=="beancount" = entriesReportAsBeancount (jdeclaredaccounttags j) styledPrices . styleAmounts styles . map fillBalanceAssignments | fmt=="csv" = printCSV . entriesReportAsCsv . styleAmounts styles | fmt=="tsv" = printTSV . entriesReportAsCsv . styleAmounts styles | fmt=="json" = toJsonText . styleAmounts styles @@ -167,6 +170,17 @@ printEntries opts@CliOpts{rawopts_=rawopts, reportspec_=rspec} j = -- Otherwise, keep the transaction's amounts close to how they were written in the journal. | otherwise = transactionWithMostlyOriginalPostings + -- Like maybeoriginalamounts, but also keeps the inferred amount for + -- balance assignment postings (which had no explicit amount). + -- Beancount requires all amounts to be explicit. + fillBalanceAssignments t = (maybeoriginalamounts t) + { tpostings = zipWith fillIfBalAssign (tpostings t) (tpostings $ maybeoriginalamounts t) } + where + fillIfBalAssign inferred reverted + | isJust (pbalanceassertion orig) && isMissingMixedAmount (pamount orig) = reverted { pamount = pamount inferred } + | otherwise = reverted + where orig = originalPosting inferred + -- | Replace this transaction's postings with the original postings if any, but keep the -- current possibly rewritten account names, and the inferred values of any auto postings. -- This is mainly for showing transactions with the amounts in their original journal format. @@ -190,15 +204,19 @@ entriesReportAsTextHelper showtxn = TB.toLazyText . foldMap (TB.fromText . showt -- in various ways when necessary (see Beancount.hs). It renders: -- account open directives for each account used (on their earliest posting dates), -- operating_currency directives (based on currencies used in costs), +-- sample tolerance options (commented), +-- price directives, -- and transaction entries. --- Transaction and posting tags are converted to metadata lines. --- Account tags are not propagated to the open directive, currently. -entriesReportAsBeancount :: Map AccountName [Tag] -> EntriesReport -> TL.Text -entriesReportAsBeancount atags ts = +-- Transaction, posting, and account tags are converted to metadata lines. +-- Account tags appear as metadata on the open directive. +entriesReportAsBeancount :: Map AccountName [Tag] -> [PriceDirective] -> EntriesReport -> TL.Text +entriesReportAsBeancount atags pricedirs ts = -- PERF: gathers and converts all account names, then repeats that work when showing each transaction TL.concat [ - TL.fromStrict operatingcurrencydirectives - ,TL.fromStrict openaccountdirectives + TL.fromStrict toleranceoptions + ,TL.fromStrict operatingcurrencyoptions + ,TL.fromStrict openaccounts + ,TL.fromStrict pricedirectives ,"\n" ,entriesReportAsTextHelper showTransactionBeancount ts3 ] @@ -216,24 +234,32 @@ entriesReportAsBeancount atags ts = -- Assume the simple case of no more than one cost + conversion posting group in each transaction. -- Actually that seems to be required by hledger right now. , let isredundantconvp p = - matchesPosting (Tag (toRegex' "conversion-posting") Nothing) p + matchesPosting (Tag (toRegex' conversionPostingTagName) Nothing) p && any (any (isJust.acost) . amounts . pamount) (tpostings t) ] + -- https://beancount.github.io/docs/beancount_language_syntax.html + -- https://beancount.github.io/docs/beancount_language_syntax.html#options + -- https://beancount.github.io/docs/beancount_options_reference.html + -- https://fava.pythonanywhere.com/example-beancount-file/help/beancount_syntax -- https://fava.pythonanywhere.com/example-beancount-file/help/options - -- "conversion-currencies + -- conversion-currencies -- When set, the currency conversion select dropdown in all charts will show the list of currencies specified in this option. -- By default, Fava lists all operating currencies and those currencies that match ISO 4217 currency codes." - -- http://furius.ca/beancount/doc/syntax - -- http://furius.ca/beancount/doc/options + -- https://beancount.github.io/docs/precision_tolerances.html + -- https://beancount.github.io/docs/precision_tolerances.html#configuration-for-default-tolerances + toleranceoptions = T.unlines [ + ";option \"inferred_tolerance_default\" \"*:0.005\"" + ] + -- "This option may be supplied multiple times ... -- A list of currencies that we single out during reporting and create dedicated columns for ... -- we use this to display these values in table cells without their associated unit strings ... -- This is used to indicate the main currencies that you work with in real life" -- We use: all currencies used in costs. - operatingcurrencydirectives + operatingcurrencyoptions | null basecurrencies = "" | otherwise = T.unlines (map (todirective . commodityToBeancount) basecurrencies) <> "\n" where @@ -253,20 +279,22 @@ entriesReportAsBeancount atags ts = concatMap tpostings ts3 - -- http://furius.ca/beancount/doc/syntax -- "there exists an “Open” directive that is used to provide the start date of each account. -- That can be located anywhere in the file, it does not have to appear in the file somewhere before you use an account name. -- You can just start using account names in transactions right away, -- though all account names that receive postings to them will eventually have to have -- a corresponding Open directive with a date that precedes all transactions posted to the account in the input file." - openaccountdirectives + openaccounts | null ts = "" | otherwise = T.unlines [ T.intercalate "\n" $ - firstdate <> " open " <> accountNameToBeancount a : + firstdate <> " open " <> accountNameToBeancount a <> disposalmethod : mdlines | a <- nubSort $ concatMap (map paccount.tpostings) ts3 - , let mds = tagsToBeancountMetadata $ fromMaybe [] $ Map.lookup a atags + , let tags' = fromMaybe [] $ Map.lookup a atags + , let lotsval = maybe "" snd $ find ((== "lots") . T.toLower . fst) tags' + , let disposalmethod = if T.null lotsval then "" else " \"" <> lotsval <> "\"" + , let mds = tagsToBeancountMetadata $ filter ((/= "lots") . T.toLower . fst) tags' , let maxwidth = maximum' $ map (T.length . fst) mds , let mdlines = map (postingIndent . showBeancountMetadata (Just maxwidth)) mds ] @@ -274,6 +302,12 @@ entriesReportAsBeancount atags ts = firstdate = showDate $ minimumDef err $ map tdate ts3 where err = error' "entriesReportAsBeancount: should not happen" + pricedirectives + | null pricedirs = "" + | otherwise = "\n" <> T.unlines (map showPriceDirectiveBeancount sortedpricedirs) + where + sortedpricedirs = sortOn pddate pricedirs + entriesReportAsSql :: EntriesReport -> TL.Text entriesReportAsSql txns = TB.toLazyText $ mconcat [ TB.fromText "create table if not exists postings(id serial,txnidx int,date1 date,date2 date,status text,code text,description text,comment text,account text,amount numeric,commodity text,credit numeric,debit numeric,posting_status text,posting_comment text);\n" @@ -363,7 +397,7 @@ postingToSpreadsheet fmt baseUrl query p = Spr.cellFromAmount fmt (Spr.Class "amount", (wbToText $ showAmountB machineFmt amt, amt)) status = T.pack . show $ pstatus p - account = showAccountName Nothing (ptype p) (paccount p) + account = showAccountName Nothing (preal p) (paccount p) comment = T.strip $ pcomment p addLocationTag :: Transaction -> Transaction diff --git a/hledger/Hledger/Cli/Commands/Print.md b/hledger/Hledger/Cli/Commands/Print.md index 7bee79a5d79..164ed1f225a 100644 --- a/hledger/Hledger/Cli/Commands/Print.md +++ b/hledger/Hledger/Cli/Commands/Print.md @@ -25,7 +25,8 @@ Flags: with this prefix. (Usually the base url shown by hledger-web; can also be relative.) -O --output-format=FMT select the output format. Supported formats: - txt, beancount, csv, tsv, html, fods, json, sql. + txt, ledger, beancount, csv, tsv, html, fods, json, + sql. -o --output-file=FILE write output to FILE. A file extension matching one of the above formats selects that format. ``` @@ -151,7 +152,11 @@ This command also supports the [output destination](hledger.html#output-destination) and [output format](hledger.html#output-format) options The output formats supported are -`txt`, `beancount` (*Added in 1.32*), `csv`, `tsv` (*Added in 1.32*), `json` and `sql`. +`txt`, `ledger`, `beancount` (*Added in 1.32*), `csv`, `tsv` (*Added in 1.32*), `json` and `sql`. + +The `ledger` format is currently the same as `txt` except it renders amounts' cost basis +using Ledger's lot syntax (`[DATE] (LABEL) {COST}`) +instead of hledger's (`{DATE, "LABEL", COST}`). The `beancount` format tries to produce Beancount-compatible output, as follows: diff --git a/hledger/Hledger/Cli/Commands/Print.txt b/hledger/Hledger/Cli/Commands/Print.txt index 6b6d284beb2..615bbac4e6d 100644 --- a/hledger/Hledger/Cli/Commands/Print.txt +++ b/hledger/Hledger/Cli/Commands/Print.txt @@ -24,7 +24,8 @@ Flags: with this prefix. (Usually the base url shown by hledger-web; can also be relative.) -O --output-format=FMT select the output format. Supported formats: - txt, beancount, csv, tsv, html, fods, json, sql. + txt, ledger, beancount, csv, tsv, html, fods, json, + sql. -o --output-file=FILE write output to FILE. A file extension matching one of the above formats selects that format. @@ -148,8 +149,12 @@ transaction, as a tag. print output format This command also supports the output destination and output format -options The output formats supported are txt, beancount (Added in 1.32), -csv, tsv (Added in 1.32), json and sql. +options The output formats supported are txt, ledger, beancount (Added +in 1.32), csv, tsv (Added in 1.32), json and sql. + +The ledger format is currently the same as txt except it renders +amounts' cost basis using Ledger's lot syntax ([DATE] (LABEL) {COST}) +instead of hledger's ({DATE, "LABEL", COST}). The beancount format tries to produce Beancount-compatible output, as follows: diff --git a/hledger/Hledger/Cli/Commands/Register.hs b/hledger/Hledger/Cli/Commands/Register.hs index 3a57f6ee5d3..bb8a775a7ca 100644 --- a/hledger/Hledger/Cli/Commands/Register.hs +++ b/hledger/Hledger/Cli/Commands/Register.hs @@ -159,7 +159,7 @@ postingsReportItemAsRecord opts@CliOpts{reportspec_=rspec} fmt baseUrl query (_, where clipAcct = clipOrEllipsifyAccountName (depth_ $ _rsReportOpts rspec) dropAcct = accountNameDrop (fromMaybe 0 $ readMay =<< maybestringopt "drop" (rawopts_ opts)) - bracket = case ptype p of + bracket = case preal p of BalancedVirtualPosting -> wrap "[" "]" VirtualPosting -> wrap "(" ")" _ -> id @@ -251,7 +251,7 @@ postingsReportItemAsText opts@CliOpts{reportspec_=rspec} preferredamtwidth prefe where clipAcct = clipOrEllipsifyAccountName (depth_ $ _rsReportOpts rspec) dropAcct = accountNameDrop (fromMaybe 0 $ readMay =<< maybestringopt "drop" (rawopts_ opts)) - (parenthesise, awidth) = case ptype p of + (parenthesise, awidth) = case preal p of BalancedVirtualPosting -> (wrap "[" "]", acctwidth-2) VirtualPosting -> (wrap "(" ")", acctwidth-2) _ -> (id,acctwidth) diff --git a/hledger/hledger.1 b/hledger/hledger.1 index be1f12d7eb9..99dc3681ccc 100644 --- a/hledger/hledger.1 +++ b/hledger/hledger.1 @@ -1,6 +1,6 @@ .\"t -.TH "HLEDGER" "1" "December 2025" "hledger-1.51.99 " "hledger User Manuals" +.TH "HLEDGER" "1" "February 2026" "hledger-1.51.99 " "hledger User Manuals" @@ -755,9 +755,9 @@ replacement string to reference capturing groups in the search regexp. Otherwise, if you write \f[CR]\[rs]1\f[R], it will match the digit \f[CR]1\f[R]. .IP "6." 3 -they do not support mode modifiers (\f[CR](?s)\f[R]), character classes -(\f[CR]\[rs]w\f[R], \f[CR]\[rs]d\f[R]), or anything else not mentioned -above. +they do not support lazy quantifiers (\f[CR]*?\f[R]), mode modifiers +(\f[CR](?s)\f[R]), character classes (\f[CR]\[rs]w\f[R], +\f[CR]\[rs]d\f[R]), or anything else not mentioned above. .IP "7." 3 they may not (I\[aq]m guessing not) properly support right\-to\-left or bidirectional text. @@ -936,7 +936,7 @@ Here are those commands and the formats currently supported: .PP .TS tab(@); -lw(20.6n) lw(5.1n) lw(6.2n) lw(9.3n) lw(6.2n) lw(11.3n) lw(5.1n) lw(6.2n). +lw(18.4n) lw(4.6n) lw(5.5n) lw(8.3n) lw(5.5n) lw(7.4n) lw(10.1n) lw(4.6n) lw(5.5n). T{ command T}@T{ @@ -948,6 +948,8 @@ csv/tsv T}@T{ fods T}@T{ +ledger +T}@T{ beancount T}@T{ sql @@ -968,6 +970,7 @@ Y T}@T{ T}@T{ T}@T{ +T}@T{ Y T} T{ @@ -983,6 +986,7 @@ Y T}@T{ T}@T{ T}@T{ +T}@T{ Y T} T{ @@ -998,6 +1002,7 @@ Y T}@T{ T}@T{ T}@T{ +T}@T{ Y T} T{ @@ -1013,6 +1018,7 @@ Y T}@T{ T}@T{ T}@T{ +T}@T{ Y T} T{ @@ -1028,6 +1034,7 @@ Y T}@T{ T}@T{ T}@T{ +T}@T{ Y T} T{ @@ -1043,6 +1050,7 @@ Y T}@T{ T}@T{ T}@T{ +T}@T{ Y T} T{ @@ -1061,6 +1069,8 @@ T}@T{ Y T}@T{ Y +T}@T{ +Y T} T{ register @@ -1075,6 +1085,7 @@ Y T}@T{ T}@T{ T}@T{ +T}@T{ Y T} .TE @@ -1169,15 +1180,21 @@ If you see junk characters, you might need to configure your pager to handle ANSI codes. Or you could disable colour as described above. .PP -If you are using the \f[CR]less\f[R] pager, hledger automatically -appends a number of options to the \f[CR]LESS\f[R] variable to enable -ANSI colour and a number of other conveniences. -(At the time of writing: \-\-chop\-long\-lines \-\-hilite\-unread -\-\-ignore\-case \-\-no\-init \-\-quit\-at\-eof -\-\-quit\-if\-one\-screen \-\-RAW\-CONTROL\-CHARS \-\-shift=8 -\-\-squeeze\-blank\-lines \-\-use\-backslash ). -If these don\[aq]t work well, you can set your preferred options in the -\f[CR]HLEDGER_LESS\f[R] variable, which will be used instead. +If you are using the \f[CR]less\f[R] pager, hledger tries to provide a +consistently pleasant experience by running it with some extra options +added to your \f[CR]LESS\f[R] environment variable: +.PP +\-\-chop\-long\-lines \-\-hilite\-unread \-\-ignore\-case \-\-no\-init +\-\-quit\-if\-one\-screen \-\-shift=8 \-\-squeeze\-blank\-lines +\-\-use\-backslash +.PP +and when colour output is enabled: +.PP +\-\-RAW\-CONTROL\-CHARS +.PP +You can prevent this by setting your preferred options in the +\f[CR]HLEDGER_LESS\f[R] variable, which will be used instead of +\f[CR]LESS\f[R]. .SS HTML output HTML output can be styled by an optional \f[CR]hledger.css\f[R] file in the same directory. @@ -1203,8 +1220,16 @@ borders. Btw. you can still extract CSV from FODS/ODS using various utilities like \f[CR]libreoffice \-\-headless\f[R] or ods2csv. +.SS Ledger output +This is a Ledger\-specific journal format supported by the +\f[CR]print\f[R] command. +It is currently identical to hledger\[aq]s default \f[CR]print\f[R] +output except that amounts\[aq] cost basis will use Ledger\[aq]s lot +syntax, (\f[CR]{COST} [DATE] (NOTE)\f[R]), not hledger\[aq]s +(\f[CR]{DATE, \[dq]LABEL\[dq], COST}\f[R]). .SS Beancount output -This is Beancount\[aq]s journal format. +This is Beancount\[aq]s journal format, supported by the +\f[CR]print\f[R] command. You can use this to export your hledger data to Beancount, eg to use the Fava web app. .PP @@ -1225,15 +1250,13 @@ There is one big adjustment you must handle yourself: for Beancount, the top level account names must be \f[CR]Assets\f[R], \f[CR]Liabilities\f[R], \f[CR]Equity\f[R], \f[CR]Income\f[R], and/or \f[CR]Expenses\f[R]. -You can use account aliases to rewrite your account names temporarily, -if needed, as in this hledger2beancount.conf config file. .PP -2024\-12\-20: Some more things not yet handled for you: -.IP \[bu] 2 -P directives are not converted automatically \- convert those yourself. -.IP \[bu] 2 -Balance assignments are not converted (Beancount doesn\[aq]t support -them) \- replace those with explicit amounts. +A top level hledger account named \f[CR]revenue\f[R] or +\f[CR]revenues\f[R] (case insensitive) will be converted to +\f[CR]Income\f[R] for Beancount. +To adjust other top level account names, you should use +\f[CR]\-\-alias\f[R] (see Account aliases, or this +hledger2beancount.conf file). .SS Beancount account names Aside from the top\-level names, hledger will adjust your account names to make valid Beancount account names, by capitalising each part, @@ -1250,6 +1273,15 @@ hledger will convert known currency symbols to ISO 4217 currency codes, capitalise letters, replace spaces with \f[CR]\-\f[R], replace other unsupported characters with \f[CR]C\f[R], and prepend or append \f[CR]C\f[R] if needed. +One\-letter symbols will be doubled. +The no\-symbol commodity will become \f[CR]CC\f[R]. +(Note, hledger tries to keep your commodities distinct, but collisions +are possiblel with short alphanumeric symbols like \f[CR]CC\f[R], +\f[CR]C\f[R], and no\-symbol, which are distinct in hledger but all +become \f[CR]CC\f[R] in beancount.) +.SS Beancount balance assignments +Beancount doesn\[aq]t support those; any balance assignments will be +converted to explicit amounts. .SS Beancount virtual postings Beancount doesn\[aq]t allow virtual postings; if you have any, they will be omitted from beancount output. @@ -1273,6 +1305,11 @@ hledger does. If you have any of these, the conversion postings will be omitted. Currently we support at most one cost + conversion postings group per transaction. +.SS Beancount tolerance +A sample \f[CR]inferred_tolerance_default\f[R] option is provided +(commented out). +If Beancount complains that transactions aren\[aq]t balanced, this is an +easy way to work around it. .SS Beancount operating currency Declaring an operating currency (or several) improves Beancount and Fava reports. @@ -2066,71 +2103,186 @@ displays them in output. This is explained in Commodity display style below. .PP .SS Costs -After a posting amount, you can note its cost (when buying) or selling -price (when selling) in another commodity, by writing either -\f[CR]\[at] UNITPRICE\f[R] or \f[CR]\[at]\[at] TOTALPRICE\f[R] after it. -This indicates a conversion transaction, where one commodity is -exchanged for another. -.PP -(You might also see this called \[dq]transaction price\[dq] in hledger -docs, discussions, or code; that term was directionally neutral and -reminded that it is a price specific to a transaction, but we now just -call it \[dq]cost\[dq], with the understanding that the transaction -could be a purchase or a sale.) -.PP -Costs are usually written explicitly with \f[CR]\[at]\f[R] or -\f[CR]\[at]\[at]\f[R], but can also be inferred automatically for simple -multi\-commodity transactions. -Note, if costs are inferred, the order of postings is significant; the -first posting will have a cost attached, in the commodity of the second. -.PP -As an example, here are several ways to record purchases of a foreign -currency in hledger, using the cost notation either explicitly or -implicitly: -.IP "1." 3 -Write the price per unit, as \f[CR]\[at] UNITPRICE\f[R] after the -amount: -.RS 4 +In traditional double entry bookkeeping, to record a transaction where +one commodity is exchanged for another, you add extra equity postings to +balance the two commodities. +Eg: .IP .EX -2009/1/1 - assets:euros €100 \[at] $1.35 ; one hundred euros purchased at $1.35 each - assets:dollars ; balancing amount is \-$135.00 +2026\-01\-01 buy euros + assets:dollars $\-123 + equity:conversion $123 + equity:conversion €\-100 + assets:euros €100 .EE -.RE +.PP +hledger offers a more convenient \[at]/\[at]\[at] \[dq]cost +notation\[dq] as an alternative: instead of equity postings, you can +write the \[dq]conversion rate\[dq] or \[dq]transacted price\[dq] after +a posting amount. +hledger docs generically call this \[dq]cost\[dq], whether buying or +selling. +(\[dq]cost\[dq] is an overloaded and generic term here, but we still use +it for historical reasons. +\[dq]Transacted price\[dq] is more precise.) +.PP +It can be written as either \f[CR]\[at] UNITPRICE\f[R] or +\f[CR]\[at]\[at] TOTALPRICE\f[R]. +Eg you could write the above as: +.IP +.EX +2026\-01\-01 buy euros + assets:dollars $\-123 + assets:euros €100 \[at] $1.23 ; unit cost (exchange rate) +.EE +.PP +or: +.IP +.EX +2026\-01\-01 buy euros + assets:dollars $\-123 + assets:euros €100 \[at]\[at] $123 ; total cost +.EE +.PP +The cost should normally be a positive amount. +Negative costs are supported, but can be confusing, as discussed at +\-\-infer\-market\-prices: market prices from transactions. +.PP +Costs participate in transaction balancing. +Amounts are converted to their cost before checking if the transaction +is balanced. +You could also write the above less redundantly, like so: +.IP +.EX +2026\-01\-01 buy euros + assets:dollars ; $\-123 is inferred + assets:euros €100 \[at] $1.23 +.EE +.PP +or: +.IP +.EX +2026\-01\-01 buy euros + assets:dollars ; $\-123 is inferred + assets:euros €100 \[at]\[at] $123 +.EE +.PP +or even: +.IP +.EX +2026\-01\-01 buy euros + assets:euros €100 ; \[at]\[at] $123 is inferred + assets:dollars $\-123 +.EE +.PP +This last form works for transactions involving exactly two commodities, +with neither cost notation nor equity postings. +Note, the order of postings is significant: the cost will be attached to +the first (top) posting. +So we had to switch the order of postings, to get the same meaning as +above. +Also, this form is the easiest to make undetected errors with; so it is +rejected by \f[CR]hledger check balanced\f[R], and by strict mode. +.PP +Advantages of cost notation: +.IP "1." 3 +it\[aq]s more compact and easier to read and write .IP "2." 3 -Write the total price, as \f[CR]\[at]\[at] TOTALPRICE\f[R] after the -amount: -.RS 4 +hledger reports can show such amounts converted to their cost, when you +add the \f[CR]\-B/\-\-cost\f[R] flag (see Cost reporting). +.PP +Advantages of equity postings +.IP "1." 3 +they help to keep the accounting equation balanced (if you care about +that) +.IP "2." 3 +they translate easily to any other double entry accounting system. +.PP +Most hledger users use cost notation and don\[aq]t use equity postings. +.PP +But you can always convert cost notation to equity postings by adding +\f[CR]\-\-infer\-equity\f[R]. +Eg try \f[CR]hledger print \-x \-\-infer\-equity\f[R]. +.PP +And you can usually convert equity postings to cost notation by adding +\f[CR]\-\-infer\-costs\f[R] (see Requirements for detecting equity +conversion postings). +Eg try \f[CR]hledger print \-x \-\-infer\-costs\f[R]. +.PP +Finally: using both equity postings and cost notation at the same time +is allowed, as long as the journal entry is well formed such that the +equity postings / cost equivalences can be detected. +(Otherwise you\[aq]ll get an error message saying that the transaction +is unbalanced.): .IP .EX -2009/1/1 - assets:euros €100 \[at]\[at] $135 ; one hundred euros purchased at $135 for the lot - assets:dollars +2026\-01\-01 buy euros + assets:dollars $\-123 + equity:conversion $123 + equity:conversion €\-100 + assets:euros €100 \[at] $1.23 .EE -.RE +.PP +So in principle you could enable both \f[CR]\-\-infer\-equity\f[R] and +\f[CR]\-\-infer\-costs\f[R] in your config file, and your reports would +have the advantages of both. +.SS Basis / lots +This is an advanced topic; skip over if you don\[aq]t need it. +For full details, see Lot reporting. +.PP +If you are buying an amount of some commodity to hold as an investment, +it may be important to keep track of its \f[I]cost basis\f[R] (AKA +\[dq]basis\[dq]) and \f[I]lots\f[R]. +(\[dq]Cost basis\[dq] and \[dq]cost\[dq] (see above) sound similar, and +often have the same amount in simple purchases; but they are distinct +concepts.) +.PP +The basis records: +.IP "1." 3 +the amount\[aq]s nominal acquisition cost (usually the same as the +transacted cost, ie what you paid for it, but not always) +.IP "2." 3 +the nominal acquisition date (usually the date you acquired it, but not +always) .IP "3." 3 -Specify amounts for all postings, using exactly two commodities, and let -hledger infer the price that balances the transaction. -Note the effect of posting order: the price is added to first posting, -making it \f[CR]€100 \[at]\[at] $135\f[R], as in example 2: -.RS 4 +and optionally a label, to disambiguate or identify lots. +.PP +An amount with a basis is called a lot. +The basis is a property of the lot, remaining with it throughout its +lifetime. +It is used to calculate capital gains/losses, eg for tax reporting. +.SS Lot syntax +A basis or lot can be described using \[dq]lot syntax\[dq]. +hledger supports two lot syntax styles: +.PP +\f[B]hledger lot syntax\f[R] puts all basis fields inside braces, +comma\-separated (like Beancount): .IP .EX -2009/1/1 - assets:euros €100 ; one hundred euros purchased - assets:dollars $\-135 ; for $135 +10 AAPL {2026\-01\-15, $50} +10 AAPL {2026\-01\-15, \[dq]lot1\[dq], $50} +10 AAPL {$50} +10 AAPL {} .EE -.RE .PP -Amounts can be converted to cost at report time using the -\f[CR]\-B/\-\-cost\f[R] flag; this is discussed more in the Cost -reporting section. +All fields are optional, but when present they must be in +date\-label\-cost order. +Dates must be in YYYY\-MM\-DD format, and labels must be double\-quoted. +The cost is a single\-commodity costless hledger amount. .PP -Note that the cost normally should be a positive amount, though it\[aq]s -not required to be. -This can be a little confusing, see discussion at -\-\-infer\-market\-prices: market prices from transactions. +When an amount has both a basis and a transacted price, like +\f[CR]\-10 AAPL {$50} \[at] $60\f[R], the preferred order is to write {} +before \[at]. +.PP +\f[B]Ledger lot syntax\f[R] uses separate annotations, in any order: +.IP +.EX +10 AAPL {$50} [2026\-01\-15] (lot1) +.EE +.PP +hledger accepts both styles on input. +Print output uses consolidated style by default, or Ledger style with +\f[CR]\-O ledger\f[R]. .SS Balance assertions hledger supports Ledger\-style balance assertions in journal files. These look like, for example, \f[CR]= EXPECTEDBALANCE\f[R] following a @@ -2578,8 +2730,8 @@ The second group above (generated\-transaction, etc.) are normally hidden, with a \f[CR]_\f[R] prefix added. This means \f[CR]print\f[R] doesn\[aq]t show them by default; but you can still use them in queries. -You can add the \f[CR]\-\-verbose\-tags\f[R] flag to make them visible, -which can be useful for troubleshooting. +You can add the \f[CR]\-\-verbose\-tags\f[R] flag to make them visible +in \f[CR]print\f[R] output, which can be useful for troubleshooting. .SS Directives Besides transactions, there is something else you can put in a \f[CR]journal\f[R] file: directives. @@ -2876,16 +3028,22 @@ account assets:bank:checking Text following \f[B]two or more spaces\f[R] and \f[CR];\f[R] at the end of an account directive line, and/or following \f[CR];\f[R] on indented lines immediately below it, form comments for that account. -They are ignored except they may contain tags, which are not ignored. .PP -The two\-space requirement for same\-line account comments is because -\f[CR];\f[R] is allowed in account names. +Same\-line account comments require two+ spaces before \f[CR];\f[R] +because that character can appear in account names. .IP .EX account assets:bank:checking ; same\-line comment, at least 2 spaces before the semicolon ; next\-line comment ; some tags \- type:A, acctnum:12345 .EE +.SS Account tags +An account directive\[aq]s comment may contain tags. +These will be inherited by all postings using that account, except where +the posting already has a value for that tag. +(A posting tag overrides an account tag.) +Note, these tags will be queryable but won\[aq]t be shown in +\f[CR]print\f[R] output, even with \-\-verbose\-tags. .SS Account error checking By default, accounts need not be declared; they come into existence when a posting references them. @@ -2917,6 +3075,10 @@ It\[aq]s currently not possible to declare \[dq]all possible subaccounts\[dq] with a wildcard; every account posted to must be declared. .IP \[bu] 2 +As an exception: lot subaccounts (a final account name component like +\f[CR]:{2026\-01\-15, $50}\f[R]) are always ignored by +\f[CR]check accounts\f[R], and need not be declared. +.IP \[bu] 2 If you use the \-\-infer\-equity flag, you will also need declarations for the account names it generates. .SS Account display order @@ -3010,29 +3172,36 @@ outflows T} .TE .PP -hledger also uses a couple of subtypes: +hledger also uses a few subtypes: .PP .TS tab(@); -l l l. +lw(23.3n) lw(23.3n) lw(23.3n). T{ \f[CR]Cash\f[R] T}@T{ \f[CR]C\f[R] T}@T{ -liquid assets +liquid assets (subtype of Asset) T} T{ \f[CR]Conversion\f[R] T}@T{ \f[CR]V\f[R] T}@T{ -commodity conversions equity +commodity conversions equity (subtype of Equity) +T} +T{ +\f[CR]Gain\f[R] +T}@T{ +\f[CR]G\f[R] +T}@T{ +capital gains/losses (subtype of Revenue) T} .TE .PP -As a convenience, hledger will detect these types automatically from -english account names. +As a convenience, hledger will detect most of these types automatically +from english account names. But it\[aq]s better to declare them explicitly by adding a \f[CR]type:\f[R] tag in the account directives. The tag\[aq]s value can be any of the types or one\-letter abbreviations @@ -3052,6 +3221,8 @@ account assets:bank ; type: C account assets:cash ; type: C account equity:conversion ; type: V + +account revenues:capital ; type: G .EE .PP This enables the easy balancesheet, balancesheetequity, cashflow and @@ -3366,7 +3537,7 @@ Commodity styles can be overridden by the (Related: #793) .SS Commodity directive syntax A commodity directive is normally the word \f[CR]commodity\f[R] followed -by a sample amount (and optionally a comment). +by a sample amount, and optionally a comment. Only the amount\[aq]s symbol and the number\[aq]s format is significant. Eg: .IP @@ -3376,8 +3547,6 @@ commodity 1.000,00 EUR commodity 1 000 000.0000 ; the no\-symbol commodity .EE .PP -Commodities do not have tags (tags in the comment will be ignored). -.PP A commodity directive\[aq]s sample amount must always include a period or comma decimal mark (this rule helps disambiguate decimal marks and digit group marks). @@ -3418,6 +3587,13 @@ commodity INR format INR 1,00,00,000.00 an unsupported subdirective ; ignored by hledger .EE +.SS Commodity tags +A commodity directive\[aq]s comment may contain tags. +These will be inherited by all postings using that commodity in their +main amount, except where the posting already has a value for that tag. +(A posting tag or an account tag overrides a commodity tag.) +Note, these tags will be queryable but won\[aq]t be shown in +\f[CR]print\f[R] output, even with \-\-verbose\-tags. .SS Commodity error checking In strict mode (\f[CR]\-s\f[R]/\f[CR]\-\-strict\f[R]) (or when you run \f[CR]hledger check commodities\f[R]), hledger will report an error if @@ -4125,72 +4301,22 @@ value EXPR .PP See also https://hledger.org/ledger.html for a detailed hledger/Ledger syntax comparison. -.SS Other cost/lot notations -A slight digression for Ledger and Beancount users. -.PP -\f[B]Ledger\f[R] has a number of cost/lot\-related notations: -.IP \[bu] 2 -\f[CR]\[at] UNITCOST\f[R] and \f[CR]\[at]\[at] TOTALCOST\f[R] -.RS 2 -.IP \[bu] 2 -expresses a conversion rate, as in hledger -.IP \[bu] 2 -when buying, also creates a lot that can be selected at selling time -.RE -.IP \[bu] 2 -\f[CR](\[at]) UNITCOST\f[R] and \f[CR](\[at]\[at]) TOTALCOST\f[R] -(virtual cost) -.RS 2 -.IP \[bu] 2 -like the above, but also means \[dq]this cost was exceptional, don\[aq]t -use it when inferring market prices\[dq]. -.RE -.IP \[bu] 2 -\f[CR]{=UNITCOST}\f[R] and \f[CR]{{=TOTALCOST}}\f[R] (fixed price) -.RS 2 -.IP \[bu] 2 -when buying, means \[dq]this cost is also the fixed value, don\[aq]t let -it fluctuate in value reports\[dq] -.RE -.IP \[bu] 2 -\f[CR]{UNITCOST}\f[R] and \f[CR]{{TOTALCOST}}\f[R] (lot price) -.RS 2 -.IP \[bu] 2 -can be used identically to \f[CR]\[at] UNITCOST\f[R] and -\f[CR]\[at]\[at] TOTALCOST\f[R], also creates a lot -.IP \[bu] 2 -when selling, combined with \f[CR]\[at] ...\f[R], selects an existing -lot by its cost basis. -Does not check if that lot is present. -.RE -.IP \[bu] 2 -\f[CR][YYYY/MM/DD]\f[R] (lot date) -.RS 2 -.IP \[bu] 2 -when buying, attaches this acquisition date to the lot -.IP \[bu] 2 -when selling, selects a lot by its acquisition date -.RE -.IP \[bu] 2 -\f[CR](SOME TEXT)\f[R] (lot note) -.RS 2 -.IP \[bu] 2 -when buying, attaches this note to the lot -.IP \[bu] 2 -when selling, selects a lot by its note -.RE -.PP -Currently, hledger -.IP \[bu] 2 -accepts any or all of the above in any order after the posting amount -.IP \[bu] 2 -supports \f[CR]\[at]\f[R] and \f[CR]\[at]\[at]\f[R] -.IP \[bu] 2 -treats \f[CR](\[at])\f[R] and \f[CR](\[at]\[at])\f[R] as synonyms for -\f[CR]\[at]\f[R] and \f[CR]\[at]\[at]\f[R] -.IP \[bu] 2 -and ignores the rest. -(This can break transaction balancing.) +.SS Ledger virtual costs +In Ledger, \f[CR](\[at]) UNITCOST\f[R] and +\f[CR](\[at]\[at]) TOTALCOST\f[R] are virtual costs, which do not +generate market prices. +In hledger, these are equivalent to \f[CR]\[at]\f[R] and +\f[CR]\[at]\[at]\f[R]. +.SS Ledger lot syntax +In Ledger, these annotations after an amount help specify or select a +lot\[aq]s cost basis: \f[CR]{LOTUNITCOST}\f[R] or +\f[CR]{{LOTTOTALCOST}}\f[R], \f[CR][LOTDATE]\f[R], and/or +\f[CR](LOTNOTE)\f[R]. +hledger will read these, as an alternative to its own lot syntax). +.PP +We also read Ledger\[aq]s fixed price) syntax, \f[CR]{=LOTUNITCOST}\f[R] +or \f[CR]{{=LOTTOTALCOST}}\f[R], treating it as equivalent to +\f[CR]{LOTUNITCOST}\f[R] or \f[CR]{{LOTTOTALCOST}}\f[R], .PP \f[B]Beancount\f[R] has simpler notation and different behaviour: .IP \[bu] 2 @@ -6555,6 +6681,14 @@ October 1st in current year \f[CR]last/this/next day/week/month/quarter/year\f[R] \-1, 0, 1 periods from the current period .TP +\f[CR]last/this/next tuesday\f[R] +the previous occurrence of the named day, or the next occurrence after +today +.TP +\f[CR]last/this/next february\f[R] +the previous occurrence of 1st of the named month, or the next +occurrence after the current month +.TP \f[CR]in n days/weeks/months/quarters/years\f[R] n periods from the current period .TP @@ -7190,13 +7324,19 @@ Match transaction descriptions. .PD Match dates (or with the \f[CR]\-\-date2\f[R] flag, secondary dates) within the specified period. -PERIODEXPR is a period expression with no report interval. +PERIODEXPR is a period expression. Examples: .PD 0 .P .PD \f[CR]date:2016\f[R], \f[CR]date:thismonth\f[R], \f[CR]date:2/1\-2/15\f[R], \f[CR]date:2021\-07\-27..nextquarter\f[R]. +.PP +PERIODEXPR may include a report interval (since 1.52). +On the command line, this is equivalent to specifying a report interval +with a command line option. +In other contexts (hledger\-ui, hledger\-web), the report interval may +be ignored. .SS date2: query \f[B]\f[CB]date2:PERIODEXPR\f[B]\f[R] .PD 0 @@ -7205,6 +7345,7 @@ Examples: If you use secondary dates: this matches secondary dates within the specified period. It is not affected by the \f[CR]\-\-date2\f[R] flag. +A report interval in PERIODEXPR will be ignored. .SS depth: query \f[B]\f[CB]depth:[REGEXP=]N\f[B]\f[R] .PD 0 @@ -7247,9 +7388,10 @@ Match unmarked, pending, or cleared transactions respectively. .PD Match by account type (see Declaring accounts > Account types). \f[CR]TYPECODES\f[R] is one or more of the single\-letter account type -codes \f[CR]ALERXCV\f[R], case insensitive. -Note \f[CR]type:A\f[R] and \f[CR]type:E\f[R] will also match their -respective subtypes \f[CR]C\f[R] (Cash) and \f[CR]V\f[R] (Conversion). +codes \f[CR]ALERXCVG\f[R], case insensitive. +Note \f[CR]type:A\f[R], \f[CR]type:E\f[R], and \f[CR]type:R\f[R] will +also match their respective subtypes \f[CR]C\f[R] (Cash), \f[CR]V\f[R] +(Conversion), and \f[CR]G\f[R] (Gain). Certain kinds of account alias can disrupt account types, see Rewriting accounts > Aliases and account types. .SS tag: query @@ -7526,6 +7668,9 @@ by auto posting rules. The \f[CR]\-\-forecast\f[R] option generates transactions from periodic transaction rules. .IP \[bu] 2 +The \f[CR]\-\-lots\f[R] flag adds extra lot subaccounts to postings for +detailed lot reporting. +.IP \[bu] 2 The \f[CR]balance \-\-budget\f[R] report infers budget goals from periodic transaction rules. .IP \[bu] 2 @@ -7550,6 +7695,97 @@ rules). Similar hidden tags (with an underscore prefix) are always present, also, so you can always match such data with queries like \f[CR]tag:generated\f[R] or \f[CR]tag:modified\f[R]. +.SH Detecting special postings +hledger detects certain kinds of postings, both generated and +non\-generated, and tags them for additional processing. +These are documented elsewhere, but this section gives an overview of +the special posting detection rules. +.PP +By default, the tags are hidden (with a \f[CR]_\f[R] prefix), so they +can be queried but they won\[aq]t appear in \f[CR]print\f[R] output. +To also add visible tags, use \f[CR]\-\-verbose\-tags\f[R] (useful for +troubleshooting). +.PP +.TS +tab(@); +lw(5.4n) lw(40.2n) lw(24.4n). +T{ +Tag +T}@T{ +Detected pattern +T}@T{ +Effect +T} +_ +T{ +\f[CR]conversion\-posting\f[R] +T}@T{ +A pair of adjacent, single\-commodity, costless postings to +\f[CR]Conversion\f[R]\-type accounts, with a nearby corresponding +costful or potentially corresponding costless posting +T}@T{ +Helps transaction balancer infer costs or avoid redundancy in commodity +conversions +T} +T{ +\f[CR]cost\-posting\f[R] +T}@T{ +A costful posting whose amount and transacted cost correspond to a +conversion postings pair; or a costless posting matching one of the pair +T}@T{ +Helps transaction balancer infer costs or avoid redundancy in commodity +conversions +T} +T{ +\f[CR]generated\-posting\f[R] +T}@T{ +Postings generated at runtime +T}@T{ +Helps users understand or find postings added at runtime by hledger +T} +T{ +\f[CR]ptype:acquire\f[R] +T}@T{ +Positive postings with lot annotations, or in a lotful +commodity/account, with no matching counterposting +T}@T{ +Creates a new lot +T} +T{ +\f[CR]ptype:dispose\f[R] +T}@T{ +Negative postings with lot annotations, or in a lotful +commodity/account, with no matching counterposting +T}@T{ +Selects and reduces existing lots +T} +T{ +\f[CR]ptype:transfer\-from\f[R] +T}@T{ +The negative posting of a pair of counterpostings, at least one with lot +annotation or a lotful commodity/account; or a negative lot posting with +an equity counterpart (equity transfer) +T}@T{ +Moves lots between accounts, preserving cost basis +T} +T{ +\f[CR]ptype:transfer\-to\f[R] +T}@T{ +The positive posting of a transfer pair; or a positive lot posting with +an equity counterpart (equity transfer, e.g. +opening balances) +T}@T{ +As above +T} +T{ +\f[CR]ptype:gain\f[R] +T}@T{ +A posting to a \f[CR]Gain\f[R]\-type account +T}@T{ +Helps transaction balancer avoid redundancy, helps disposal balancer +check realised capital gain/loss +T} +.TE .SH Forecasting Forecasting, or speculative future reporting, can be useful for estimating future balances, or for exploring different future scenarios. @@ -8053,6 +8289,8 @@ account with the \f[CR]V\f[R]/\f[CR]Conversion\f[R] account type. Note you will need to add account declarations for these to your journal, if you use \f[CR]check accounts\f[R] or \f[CR]check \-\-strict\f[R]. +(And unlike normal postings, generated equity postings do not inherit +tags from account declarations.) .SS Combining costs and equity conversion postings Finally, you can use both the \[at]/\[at]\[at] cost notation and equity postings at the same time. @@ -8885,6 +9123,343 @@ T} .PP \f[CR]\-\-cumulative\f[R] is omitted to save space, it works like \f[CR]\-H\f[R] but with a zero starting balance. +.SH Lot reporting +With the \f[CR]\-\-lots\f[R] flag, hledger can track investment lots +automatically: assigning lot subaccounts on acquisition, selecting lots +on disposal using configurable methods, calculating capital gains, and +showing per\-lot balances in all reports. +(Since 1.99.1, experimental. +For more technical details, see SPEC\-lots.md). +.SS Lotful commodities and accounts +Commodities and accounts can be declared as \[dq]lotful\[dq] by adding a +\f[CR]lots\f[R] tag in their declaration: +.IP +.EX +commodity AAPL ; lots: +account assets:stocks ; lots: +.EE +.PP +This tells hledger that postings involving these always involve lots, +enabling cost basis inference even when lot syntax is not written +explicitly. +.PP +The tag value can also specify a reduction method: +.IP +.EX +commodity AAPL ; lots: FIFO +account assets:stocks ; lots: LIFO +.EE +.PP +If no value is specified, the default is FIFO. +.SS \-\-lots +Add \f[CR]\-\-lots\f[R] to any command to enable lot tracking. +This activates: +.IP \[bu] 2 +\f[B]Lot posting classification\f[R] \[em] lot\-related postings are +tagged as \f[CR]acquire\f[R], \f[CR]dispose\f[R], +\f[CR]transfer\-from\f[R], \f[CR]transfer\-to\f[R], or \f[CR]gain\f[R] +(via a hidden \f[CR]ptype\f[R] tag, visible with +\f[CR]\-\-verbose\-tags\f[R], queryable with \f[CR]tag:ptype=...\f[R]). +.IP \[bu] 2 +\f[B]Cost basis inference\f[R] \[em] for lotful commodities/accounts, +cost basis is inferred from transacted cost and vice versa. +Or when the account name ends with a lot subaccount, cost basis can also +be inferred from that. +.IP \[bu] 2 +\f[B]Lot calculation\f[R] \[em] acquired lots become subaccounts; +disposals and transfers select from existing lots. +.IP \[bu] 2 +\f[B]Disposal balancing\f[R] \[em] disposal transactions are checked for +balance at cost basis; gain amounts/postings are inferred if missing. +.SS Lot subaccounts +With \f[CR]\-\-lots\f[R], each acquired lot becomes a subaccount named +by its cost basis: +.IP +.EX +commodity AAPL ; lots: + +2026\-01\-15 buy + assets:stocks 10 AAPL {$50} + assets:cash \-$500 +.EE +.IP +.EX +$ hledger bal assets:stocks \-\-lots \-N + 10 AAPL assets:stocks:{2026\-01\-15, $50} +.EE +.PP +You can also write lot subaccounts explicitly. +When a posting\[aq]s account name ends with a lot subaccount (like +\f[CR]:{2026\-01\-15, $50}\f[R]), the cost basis is parsed from it +automatically, so a \f[CR]{}\f[R] annotation on the amount is optional: +.IP +.EX +commodity AAPL ; lots: + +2026\-01\-15 buy + assets:stocks:{2026\-01\-15, $50} 10 AAPL + assets:cash +.EE +.PP +This is equivalent to writing \f[CR]10 AAPL {2026\-01\-15, $50}\f[R]. +(If both the account name and the amount specify a cost basis, they must +agree.) +.PP +When strictly checking account names, lot subaccounts are automatically +exempt \[em] you only need to declare the base account (eg +\f[CR]account assets:stocks\f[R]), not each individual lot subaccount. +.SS Lot operations +.IP \[bu] 2 +\f[B]Acquire\f[R]: a positive lot posting creates a new lot. +The cost basis can be specified explicitly with \f[CR]{}\f[R] on the +amount, inferred from the lot subaccount name, or inferred from the +transacted cost. +On lotful commodities/accounts, even a bare positive posting (no +\f[CR]{}\f[R] or \f[CR]\[at]\f[R]) can be detected as an acquire, with +cost inferred from the transaction\[aq]s other postings. +.IP \[bu] 2 +\f[B]Transfer\f[R]: a matching pair of negative/positive lot postings +moves a lot between accounts, preserving its cost basis. +Transfer postings should not have a transacted price. +.IP \[bu] 2 +\f[B]Dispose\f[R]: a negative lot posting sells from one or more +existing lots. +It must have a transacted price (the selling price), either explicit or +inferred. +.PP +An example disposal entry: +.IP +.EX +2026\-02\-01 sell + assets:stocks \-5 AAPL {$50} \[at] $60 + assets:cash $300 + revenue:gains \-$50 +.EE +.PP +With \f[CR]\-\-lots\f[R], this selects the specified quantity of the +matching lot (which must exist) and will show something like: +.IP +.EX +$ hledger print \-\-lots desc:sell +2026\-02\-01 sell + assets:stocks:{2026\-01\-15, $50} \-5 AAPL {2026\-01\-15, $50} \[at] $60 + assets:cash $300 + revenue:gains $\-50 +.EE +.SS Reduction methods +When a disposal or transfer doesn\[aq]t specify a particular lot (eg the +amount is \f[CR]\-5 AAPL\f[R] or \f[CR]\-5 AAPL {}\f[R]), hledger +selects lot(s) automatically using a reduction method. +The available methods are: +.PP +.TS +tab(@); +lw(16.9n) lw(16.9n) lw(17.7n) lw(18.6n). +T{ +Method +T}@T{ +Lots selected +T}@T{ +Scope +T}@T{ +Disposal cost basis +T} +_ +T{ +\f[B]FIFO\f[R] (default) +T}@T{ +oldest first +T}@T{ +across all accounts +T}@T{ +each lot\[aq]s cost +T} +T{ +\f[B]FIFO1\f[R] +T}@T{ +oldest first +T}@T{ +within one account +T}@T{ +each lot\[aq]s cost +T} +T{ +\f[B]LIFO\f[R] +T}@T{ +newest first +T}@T{ +across all accounts +T}@T{ +each lot\[aq]s cost +T} +T{ +\f[B]LIFO1\f[R] +T}@T{ +newest first +T}@T{ +within one account +T}@T{ +each lot\[aq]s cost +T} +T{ +\f[B]HIFO\f[R] +T}@T{ +highest cost first +T}@T{ +across all accounts +T}@T{ +each lot\[aq]s cost +T} +T{ +\f[B]HIFO1\f[R] +T}@T{ +highest cost first +T}@T{ +within one account +T}@T{ +each lot\[aq]s cost +T} +T{ +\f[B]AVERAGE\f[R] +T}@T{ +oldest first +T}@T{ +across all accounts +T}@T{ +weighted average cost +T} +T{ +\f[B]AVERAGE1\f[R] +T}@T{ +oldest first +T}@T{ +within one account +T}@T{ +weighted average cost +T} +T{ +\f[B]SPECID\f[R] +T}@T{ +one specified lot +T}@T{ +specified account +T}@T{ +specified lot\[aq]s cost +T} +.TE +.PP +An explicit lot selector (eg \f[CR]{2026\-01\-15, $50}\f[R] or +\f[CR]{$50}\f[R]) uses specific\-identification (SPECID). +.PP +\f[B]HIFO\f[R] (highest\-in\-first\-out) selects the lot with the +highest per\-unit cost first, which can be useful for tax optimization. +.PP +\f[B]AVERAGE\f[R] uses the weighted average per\-unit cost of the entire +pool as the disposal cost basis, rather than each lot\[aq]s individual +cost. +This is required in some jurisdictions (eg Canada\[aq]s Adjusted Cost +Base, France\[aq]s PMPA, UK\[aq]s S104 pools). +Lots are still consumed in FIFO order for bookkeeping purposes. +Configure the method via the \f[CR]lots:\f[R] tag on a commodity or +account declaration: +.IP +.EX +commodity AAPL ; lots: FIFO +account assets:stocks ; lots: AVERAGE +.EE +.PP +Account tags override commodity tags. +.SS Lot postings with balance assertions +On a dispose or transfer posting without an explicit lot subaccount, a +balance assertion always refers to the parent account\[aq]s balance. +So if lot subaccounts are added witih \f[CR]\-\-lots\f[R], the assertion +is not affected. +.PP +By contrast, in a journal entry where the lot subaccounts are recorded +explicitly, a balance assertion refers to the lot subaccount\[aq]s +balance. +.PP +This means that \f[CR]hledger print \-\-lots\f[R], if it adds explicit +lot subaccounts to a journal entry, could potentially change the meaning +of balance assertions, breaking them. +To avoid this, in such cases it will move the balance assertion to a new +zero\-amount posting to the parent account (and make sure it\[aq]s +subaccount\-inclusive). +(So eg +\f[CR]hledger \-f\- print \-\-lots \-x | hledger \-f\- check assertions\f[R] +will still pass.) +.SS Gain postings and disposal balancing +A \f[B]gain posting\f[R] is a posting to a Gain\-type account (type +\f[CR]G\f[R], a subtype of Revenue). +In disposal transactions, it records the capital gain or loss, which is +the difference between cost basis and selling price of the lots being +sold. +.PP +Accounts named like \f[CR]revenue:gains\f[R] or +\f[CR]income:capital\-gains\f[R] are detected as Gain accounts +automatically, or you can declare one explicitly: +.IP +.EX +account gain/loss ; type: G +.EE +.PP +Gain postings have special treatment: +.IP \[bu] 2 +\f[B]Normal transaction balancing\f[R] ignores gain postings (they +don\[aq]t count toward the balance check), and balances the transaction +using transacted price +.IP \[bu] 2 +\f[B]Disposal balancing\f[R] (with \f[CR]\-\-lots\f[R]) includes gain +postings, and balances the transaction using cost basis +.PP +An amountless gain posting in a disposal transaction will have its +amount filled in. +Or if a disposal transaction is unbalanced at cost basis and has no gain +posting, one is inferred automatically (posting to the first Gain +account, or \f[CR]revenue:gains\f[R] if none is declared). +.SS Lot reporting example +.IP +.EX +commodity AAPL ; lots: + +2026\-01\-15 buy low + assets:stocks 10 AAPL {$50} + assets:cash \-$500 + +2026\-02\-01 buy high + assets:stocks 10 AAPL {$60} + assets:cash \-$600 + +2026\-03\-01 sell some (FIFO, selects oldest lot first) + assets:stocks \-5 AAPL \[at] $70 + assets:cash $350 + revenue:gains +.EE +.IP +.EX +$ hledger bal assets:stocks \-\-lots \-N + 5 AAPL assets:stocks:{2026\-01\-15, $50} + 10 AAPL assets:stocks:{2026\-02\-01, $60} +.EE +.IP +.EX +$ hledger print \-\-lots \-x desc:sell +2026\-03\-01 sell some (FIFO, selects oldest lot first) + assets:stocks:{2026\-01\-15, $50} \-5 AAPL {2026\-01\-15, $50} \[at] $70 + assets:cash $350 + revenue:gains \-$100 +.EE +.IP +.EX +$ hledger print \-\-lots \-x desc:sell \-\-verbose\-tags +2026\-03\-01 sell some (FIFO, selects oldest lot first) + assets:stocks:{2026\-01\-15, $50} \-5 AAPL {2026\-01\-15, $50} \[at] $70 ; ptype: dispose + assets:cash $350 + revenue:gains $\-100 ; ptype: gain +.EE +.PP +The gain of $100 was inferred: 5 shares acquired at $50, sold at $70 = 5 +× ($70 \- $50) = $100. .SH PART 4: COMMANDS .PP Here are hledger\[aq]s standard subcommands. @@ -9342,6 +9917,12 @@ in the journal. .IP \[bu] 2 \f[CR]add\f[R] creates entries in journal format; it won\[aq]t work with timeclock or timedot files. +.IP \[bu] 2 +There is a known issue on Windows if this hledger version is built from +stackage: the prompts will show ANSI junk instead of colours (#2410). +You can avoid this by using official hledger release binaries or by +building it with haskeline >=0.8.4; or by running \f[CR]add\f[R] with +\f[CR]\-\-color=no\f[R], perhaps configured in your config file. .PP Examples: .IP \[bu] 2 @@ -9790,8 +10371,8 @@ but not declared, or declared but not used, or just the first one matched by a pattern (with \f[CR]\-\-find\f[R], returning a non\-zero exit code if it fails). .PP -You can add \f[CR]cur:\f[R] query arguments to further limit the -commodities. +You can add query arguments to further limit the commodities; at least +\f[CR]cur:\f[R] and \f[CR]tag:\f[R] are supported. .SS descriptions List the unique descriptions used in transactions. .IP @@ -10027,7 +10608,8 @@ Flags: with this prefix. (Usually the base url shown by hledger\-web; can also be relative.) \-O \-\-output\-format=FMT select the output format. Supported formats: - txt, beancount, csv, tsv, html, fods, json, sql. + txt, ledger, beancount, csv, tsv, html, fods, json, + sql. \-o \-\-output\-file=FILE write output to FILE. A file extension matching one of the above formats selects that format. .EE @@ -10166,9 +10748,14 @@ number to every transaction, as a tag. .SS print output format This command also supports the output destination and output format options The output formats supported are \f[CR]txt\f[R], -\f[CR]beancount\f[R] (\f[I]Added in 1.32\f[R]), \f[CR]csv\f[R], -\f[CR]tsv\f[R] (\f[I]Added in 1.32\f[R]), \f[CR]json\f[R] and -\f[CR]sql\f[R]. +\f[CR]ledger\f[R], \f[CR]beancount\f[R] (\f[I]Added in 1.32\f[R]), +\f[CR]csv\f[R], \f[CR]tsv\f[R] (\f[I]Added in 1.32\f[R]), +\f[CR]json\f[R] and \f[CR]sql\f[R]. +.PP +The \f[CR]ledger\f[R] format is currently the same as \f[CR]txt\f[R] +except it renders amounts\[aq] cost basis using Ledger\[aq]s lot syntax +(\f[CR][DATE] (LABEL) {COST}\f[R]) instead of hledger\[aq]s +(\f[CR]{DATE, \[dq]LABEL\[dq], COST}\f[R]). .PP The \f[CR]beancount\f[R] format tries to produce Beancount\-compatible output, as follows: @@ -10376,6 +10963,7 @@ Flags: description closest to DESC \-r \-\-related show postings\[aq] siblings instead \-\-invert display all amounts with reversed sign + \-\-drop=N omit N leading account name parts \-\-sort=FIELDS sort by: date, desc, account, amount, absamount, or a comma\-separated combination of these. For a descending sort, prefix with \-. (Default: date) @@ -10433,6 +11021,9 @@ $ hledger register checking \-b 2008/6 \-\-historical The \f[CR]\-\-depth\f[R] option limits the amount of sub\-account detail displayed. .PP +The \f[CR]\-\-drop\f[R] option will trim leading segments from account +names. +.PP The \f[CR]\-\-average\f[R]/\f[CR]\-A\f[R] flag shows the running average posting amount instead of the running total (so, the final number displayed is the average for the whole report period). @@ -10510,6 +11101,16 @@ intervals. This ensures that the first and last intervals are full length and comparable to the others in the report. .PP +If you have a deeply nested account tree some reports might benefit from +trimming leading segments from the account names using +\f[CR]\-\-drop\f[R]. +.IP +.EX +$ hledger register \-\-monthly income \-\-drop 1 +2008/01 salary $\-1 $\-1 +2008/06 gifts $\-1 $\-2 +.EE +.PP With \f[CR]\-m DESC\f[R]/\f[CR]\-\-match=DESC\f[R], register does a fuzzy search for one recent posting whose description is most similar to DESC. @@ -12573,6 +13174,10 @@ previous balances in an opening transaction. These provide useful error checking, but you can ignore them temporarily with \f[CR]\-I\f[R], or remove them if you prefer. .PP +With \f[CR]\-\-lots\f[R], balance assertions are not generated for the +lot subaccount postings in closing transactions (assertions on lot +postings get confusing, because they apply to the parent account). +.PP Single\-commodity, subaccount\-exclusive balance assertions (\f[CR]=\f[R]) are generated by default. This can be changed with \f[CR]\-\-assertion\-type=\[aq]==*\[aq]\f[R] @@ -12879,6 +13484,8 @@ This guards against mistyping or omitting commodity symbols. .IP \[bu] 2 \f[B]accounts\f[R] \- all account names used must be declared. This prevents the use of mis\-spelled or outdated account names. +(Except lot subaccounts, like \f[CR]:{2026\-01\-15, $50}\f[R], which are +automatically exempt; only their base account needs to be declared.) .SS Other checks These are not wanted by everyone, but can be run using the \f[CR]check\f[R] command: @@ -13145,7 +13752,7 @@ It depends on your shell, but running these commands in the terminal will work for many people; adapt if needed: .IP .EX -$ echo \[aq]export LEDGER_FILE=\[ti]/finance/my.journal\[aq] >> \[ti]/.profile +$ echo \[aq]export LEDGER_FILE=\[ti]/finance/main.journal\[aq] >> \[ti]/.profile $ source \[ti]/.profile .EE .PP @@ -13164,7 +13771,7 @@ Add an entry to \f[CR]\[ti]/.MacOSX/environment.plist\f[R] like .IP .EX { - \[dq]LEDGER_FILE\[dq] : \[dq]\[ti]/finance/my.journal\[dq] + \[dq]LEDGER_FILE\[dq] : \[dq]\[ti]/finance/main.journal\[dq] } .EE .RE @@ -13177,7 +13784,12 @@ When correctly configured for GUI applications: apps started from the dock or a spotlight search, such as a GUI Emacs, will be aware of the new LEDGER_FILE setting. .SS Set LEDGER_FILE on Windows -Using the gui is easiest: +It can be easier to create a default file at +\f[CR]C:\[rs]Users\[rs]USER\[rs].hledger.journal\f[R], and have it +include your other files. +See I\[aq]m on Windows, how do I keep my files in AppData? +.PP +Otherwise: using the gui is easiest: .IP "1." 3 In task bar, search for \f[CR]environment variables\f[R], and choose \[dq]Edit environment variables for your account\[dq]. @@ -13185,7 +13797,7 @@ In task bar, search for \f[CR]environment variables\f[R], and choose Create or change a \f[CR]LEDGER_FILE\f[R] setting in the User variables pane. A typical value would be -\f[CR]C:\[rs]Users\[rs]USERNAME\[rs]finance\[rs]my.journal\f[R]. +\f[CR]C:\[rs]Users\[rs]USER\[rs]finance\[rs]main.journal\f[R]. .IP "3." 3 Click OK to complete the change. .IP "4." 3 @@ -13195,7 +13807,7 @@ And open a new powershell window. Or at the command line, you can do it this way: .IP "1." 3 In a powershell window, run -\f[CR][Environment]::SetEnvironmentVariable(\[dq]LEDGER_FILE\[dq], \[dq]C:\[rs]User\[rs]USERNAME\[rs]finance\[rs]my.journal\[dq], [System.EnvironmentVariableTarget]::User)\f[R] +\f[CR][Environment]::SetEnvironmentVariable(\[dq]LEDGER_FILE\[dq], \[dq]C:\[rs]User\[rs]USER\[rs]finance\[rs]main.journal\[dq], [System.EnvironmentVariableTarget]::User)\f[R] .IP "2." 3 And open a new powershell window. (Existing windows won\[aq]t see the change.) diff --git a/hledger/hledger.cabal b/hledger/hledger.cabal index c559de73982..1711fb49a2a 100644 --- a/hledger/hledger.cabal +++ b/hledger/hledger.cabal @@ -5,7 +5,7 @@ cabal-version: 2.2 -- see: https://github.com/sol/hpack name: hledger -version: 1.52.99 +version: 1.99 synopsis: Command-line interface for the hledger accounting system description: The command-line interface for the hledger accounting system. Its basic function is to read a plain text file describing @@ -151,7 +151,7 @@ library hs-source-dirs: ./ ghc-options: -Wall -Wno-incomplete-uni-patterns -Wno-missing-signatures -Wno-orphans -Wno-type-defaults -Wno-unused-do-bind -optP-Wno-nonportable-include-path - cpp-options: -DVERSION="1.52.99" + cpp-options: -DVERSION="1.99" build-depends: Decimal >=0.5.1 , Diff >=0.2 @@ -168,7 +168,7 @@ library , githash >=0.1.6.2 , hashable >=1.2.4 , haskeline >=0.6 - , hledger-lib >=1.52.99 && <1.53 + , hledger-lib ==1.99.* , http-client , http-types , lucid @@ -208,7 +208,7 @@ executable hledger hs-source-dirs: app ghc-options: -Wall -Wno-incomplete-uni-patterns -Wno-missing-signatures -Wno-orphans -Wno-type-defaults -Wno-unused-do-bind -optP-Wno-nonportable-include-path -threaded -with-rtsopts=-T - cpp-options: -DVERSION="1.52.99" + cpp-options: -DVERSION="1.99" build-depends: Decimal >=0.5.1 , aeson >=1 && <2.3 @@ -224,7 +224,7 @@ executable hledger , githash >=0.1.6.2 , haskeline >=0.6 , hledger - , hledger-lib >=1.52.99 && <1.53 + , hledger-lib ==1.99.* , http-client , http-types , math-functions >=0.3.3.0 @@ -259,7 +259,7 @@ test-suite unittest hs-source-dirs: test ghc-options: -Wall -Wno-incomplete-uni-patterns -Wno-missing-signatures -Wno-orphans -Wno-type-defaults -Wno-unused-do-bind -optP-Wno-nonportable-include-path - cpp-options: -DVERSION="1.52.99" + cpp-options: -DVERSION="1.99" build-depends: Decimal >=0.5.1 , aeson >=1 && <2.3 @@ -275,7 +275,7 @@ test-suite unittest , githash >=0.1.6.2 , haskeline >=0.6 , hledger - , hledger-lib >=1.52.99 && <1.53 + , hledger-lib ==1.99.* , http-client , http-types , math-functions >=0.3.3.0 @@ -310,7 +310,7 @@ benchmark bench hs-source-dirs: bench ghc-options: -Wall -Wno-incomplete-uni-patterns -Wno-missing-signatures -Wno-orphans -Wno-type-defaults -Wno-unused-do-bind -optP-Wno-nonportable-include-path - cpp-options: -DVERSION="1.52.99" + cpp-options: -DVERSION="1.99" build-depends: Decimal >=0.5.1 , aeson >=1 && <2.3 @@ -327,7 +327,7 @@ benchmark bench , githash >=0.1.6.2 , haskeline >=0.6 , hledger - , hledger-lib >=1.52.99 && <1.53 + , hledger-lib ==1.99.* , html , http-client , http-types diff --git a/hledger/hledger.info b/hledger/hledger.info index aed80b16c7a..c3364605d9a 100644 --- a/hledger/hledger.info +++ b/hledger/hledger.info @@ -102,11 +102,13 @@ file" and "Setting opening balances" sections in PART 5: COMMON TASKS. * Queries:: * Pivoting:: * Generating data:: +* Detecting special postings:: * Forecasting:: * Budgeting:: * Amount formatting:: * Cost reporting:: * Value reporting:: +* Lot reporting:: * PART 4 COMMANDS:: * Help commands:: * User interface commands:: @@ -738,8 +740,9 @@ they support: aliases or CSV rules, where backreferences can be used in the replacement string to reference capturing groups in the search regexp. Otherwise, if you write '\1', it will match the digit '1'. - 6. they do not support mode modifiers ('(?s)'), character classes - ('\w', '\d'), or anything else not mentioned above. + 6. they do not support lazy quantifiers ('*?'), mode modifiers + ('(?s)'), character classes ('\w', '\d'), or anything else not + mentioned above. 7. they may not (I'm guessing not) properly support right-to-left or bidirectional text. @@ -922,16 +925,16 @@ File: hledger.info, Node: Output format, Next: Commodity styles, Prev: Output Some commands offer other kinds of output, not just text on the terminal. Here are those commands and the formats currently supported: -command txt html csv/tsv fods beancount sql json ------------------------------------------------------------------------------- -aregister Y Y Y Y Y -balance Y Y Y Y Y -balancesheet Y Y Y Y Y -balancesheetequity Y Y Y Y Y -cashflow Y Y Y Y Y -incomestatement Y Y Y Y Y -print Y Y Y Y Y Y Y -register Y Y Y Y Y +command txt html csv/tsv fods ledger beancount sql json +----------------------------------------------------------------------------------- +aregister Y Y Y Y Y +balance Y Y Y Y Y +balancesheet Y Y Y Y Y +balancesheetequity Y Y Y Y Y +cashflow Y Y Y Y Y +incomestatement Y Y Y Y Y +print Y Y Y Y Y Y Y Y +register Y Y Y Y Y You can also see which output formats a command supports by running 'hledger CMD -h' and looking for the '-O'/'--output-format=FMT' option, @@ -958,6 +961,7 @@ $ hledger balancesheet -o foo.txt -O csv # write CSV to foo.txt * HTML output:: * CSV / TSV output:: * FODS output:: +* Ledger output:: * Beancount output:: * SQL output:: * JSON output:: @@ -1046,13 +1050,19 @@ styling. If you see junk characters, you might need to configure your pager to handle ANSI codes. Or you could disable colour as described above. - If you are using the 'less' pager, hledger automatically appends a -number of options to the 'LESS' variable to enable ANSI colour and a -number of other conveniences. (At the time of writing: -chop-long-lines --hilite-unread -ignore-case -no-init -quit-at-eof -quit-if-one-screen --RAW-CONTROL-CHARS -shift=8 -squeeze-blank-lines -use-backslash ). If -these don't work well, you can set your preferred options in the -'HLEDGER_LESS' variable, which will be used instead. + If you are using the 'less' pager, hledger tries to provide a +consistently pleasant experience by running it with some extra options +added to your 'LESS' environment variable: + + -chop-long-lines -hilite-unread -ignore-case -no-init +-quit-if-one-screen -shift=8 -squeeze-blank-lines -use-backslash + + and when colour output is enabled: + + -RAW-CONTROL-CHARS + + You can prevent this by setting your preferred options in the +'HLEDGER_LESS' variable, which will be used instead of 'LESS'.  File: hledger.info, Node: HTML output, Next: CSV / TSV output, Prev: Text output, Up: Output format @@ -1077,7 +1087,7 @@ In CSV or TSV output, digit group marks (such as thousands separators) are disabled automatically.  -File: hledger.info, Node: FODS output, Next: Beancount output, Prev: CSV / TSV output, Up: Output format +File: hledger.info, Node: FODS output, Next: Ledger output, Prev: CSV / TSV output, Up: Output format 5.2.4 FODS output ----------------- @@ -1094,13 +1104,25 @@ CSV from FODS/ODS using various utilities like 'libreoffice --headless' or ods2csv.  -File: hledger.info, Node: Beancount output, Next: SQL output, Prev: FODS output, Up: Output format +File: hledger.info, Node: Ledger output, Next: Beancount output, Prev: FODS output, Up: Output format + +5.2.5 Ledger output +------------------- + +This is a Ledger-specific journal format supported by the 'print' +command. It is currently identical to hledger's default 'print' output +except that amounts' cost basis will use Ledger's lot syntax, ('{COST} +[DATE] (NOTE)'), not hledger's ('{DATE, "LABEL", COST}'). + + +File: hledger.info, Node: Beancount output, Next: SQL output, Prev: Ledger output, Up: Output format -5.2.5 Beancount output +5.2.6 Beancount output ---------------------- -This is Beancount's journal format. You can use this to export your -hledger data to Beancount, eg to use the Fava web app. +This is Beancount's journal format, supported by the 'print' command. +You can use this to export your hledger data to Beancount, eg to use the +Fava web app. hledger will try to adjust your data to suit Beancount, automatically. Be cautious and check the conversion until you are @@ -1114,30 +1136,28 @@ want to follow its conventions, for a cleaner conversion: There is one big adjustment you must handle yourself: for Beancount, the top level account names must be 'Assets', 'Liabilities', 'Equity', -'Income', and/or 'Expenses'. You can use account aliases to rewrite -your account names temporarily, if needed, as in this -hledger2beancount.conf config file. - - 2024-12-20: Some more things not yet handled for you: +'Income', and/or 'Expenses'. - * P directives are not converted automatically - convert those - yourself. - * Balance assignments are not converted (Beancount doesn't support - them) - replace those with explicit amounts. + A top level hledger account named 'revenue' or 'revenues' (case +insensitive) will be converted to 'Income' for Beancount. To adjust +other top level account names, you should use '--alias' (see Account +aliases, or this hledger2beancount.conf file). * Menu: * Beancount account names:: * Beancount commodity names:: +* Beancount balance assignments:: * Beancount virtual postings:: * Beancount metadata:: * Beancount costs:: +* Beancount tolerance:: * Beancount operating currency::  File: hledger.info, Node: Beancount account names, Next: Beancount commodity names, Up: Beancount output -5.2.5.1 Beancount account names +5.2.6.1 Beancount account names ............................... Aside from the top-level names, hledger will adjust your account names @@ -1148,9 +1168,9 @@ with a letter or digit, and appending ':A' to account names which have only one part.  -File: hledger.info, Node: Beancount commodity names, Next: Beancount virtual postings, Prev: Beancount account names, Up: Beancount output +File: hledger.info, Node: Beancount commodity names, Next: Beancount balance assignments, Prev: Beancount account names, Up: Beancount output -5.2.5.2 Beancount commodity names +5.2.6.2 Beancount commodity names ................................. hledger will adjust your commodity names to make valid Beancount @@ -1159,12 +1179,25 @@ or ''', '.', '_', '-', beginning with a letter and ending with a letter or digit. hledger will convert known currency symbols to ISO 4217 currency codes, capitalise letters, replace spaces with '-', replace other unsupported characters with 'C', and prepend or append -'C' if needed. +'C' if needed. One-letter symbols will be doubled. The no-symbol +commodity will become 'CC'. (Note, hledger tries to keep your +commodities distinct, but collisions are possiblel with short +alphanumeric symbols like 'CC', 'C', and no-symbol, which are distinct +in hledger but all become 'CC' in beancount.)  -File: hledger.info, Node: Beancount virtual postings, Next: Beancount metadata, Prev: Beancount commodity names, Up: Beancount output +File: hledger.info, Node: Beancount balance assignments, Next: Beancount virtual postings, Prev: Beancount commodity names, Up: Beancount output -5.2.5.3 Beancount virtual postings +5.2.6.3 Beancount balance assignments +..................................... + +Beancount doesn't support those; any balance assignments will be +converted to explicit amounts. + + +File: hledger.info, Node: Beancount virtual postings, Next: Beancount metadata, Prev: Beancount balance assignments, Up: Beancount output + +5.2.6.4 Beancount virtual postings .................................. Beancount doesn't allow virtual postings; if you have any, they will be @@ -1173,7 +1206,7 @@ omitted from beancount output.  File: hledger.info, Node: Beancount metadata, Next: Beancount costs, Prev: Beancount virtual postings, Up: Beancount output -5.2.5.4 Beancount metadata +5.2.6.5 Beancount metadata .......................... hledger tags will be converted to Beancount metadata (except for tags @@ -1188,9 +1221,9 @@ values. Eg an 'assets:cash' account might have both 'type:Asset' and the values combined, comma separated. Eg: 'type: "Asset, Cash"'.  -File: hledger.info, Node: Beancount costs, Next: Beancount operating currency, Prev: Beancount metadata, Up: Beancount output +File: hledger.info, Node: Beancount costs, Next: Beancount tolerance, Prev: Beancount metadata, Up: Beancount output -5.2.5.5 Beancount costs +5.2.6.6 Beancount costs ....................... Beancount doesn't allow redundant costs and conversion postings as @@ -1199,9 +1232,19 @@ omitted. Currently we support at most one cost + conversion postings group per transaction.  -File: hledger.info, Node: Beancount operating currency, Prev: Beancount costs, Up: Beancount output +File: hledger.info, Node: Beancount tolerance, Next: Beancount operating currency, Prev: Beancount costs, Up: Beancount output + +5.2.6.7 Beancount tolerance +........................... -5.2.5.6 Beancount operating currency +A sample 'inferred_tolerance_default' option is provided (commented +out). If Beancount complains that transactions aren't balanced, this is +an easy way to work around it. + + +File: hledger.info, Node: Beancount operating currency, Prev: Beancount tolerance, Up: Beancount output + +5.2.6.8 Beancount operating currency .................................... Declaring an operating currency (or several) improves Beancount and Fava @@ -1214,7 +1257,7 @@ option "operating_currency" "USD"  File: hledger.info, Node: SQL output, Next: JSON output, Prev: Beancount output, Up: Output format -5.2.6 SQL output +5.2.7 SQL output ---------------- SQL output is expected to work at least with SQLite, MySQL and Postgres. @@ -1235,7 +1278,7 @@ $ hledger print -O sql | sed 's/id serial/id INTEGER PRIMARY KEY AUTOINCREMENT N  File: hledger.info, Node: JSON output, Prev: SQL output, Up: Output format -5.2.7 JSON output +5.2.8 JSON output ----------------- Our JSON is rather large and verbose, since it is a faithful @@ -1387,6 +1430,8 @@ part. * Postings:: * Account names:: * Amounts:: +* Costs:: +* Basis / lots:: * Balance assertions:: * Posting comments:: * Transaction balancing:: @@ -1938,7 +1983,7 @@ postings. permanently, by account aliases.  -File: hledger.info, Node: Amounts, Next: Balance assertions, Prev: Account names, Up: Journal +File: hledger.info, Node: Amounts, Next: Costs, Prev: Account names, Up: Journal 8.11 Amounts ============ @@ -1983,7 +2028,6 @@ EUR 1E3 * Decimal marks:: * Digit group marks:: * Commodity:: -* Costs::  File: hledger.info, Node: Decimal marks, Next: Digit group marks, Up: Amounts @@ -2036,7 +2080,7 @@ INR 9,99,99,999.00 1 000 000.00 ; <- no-break space  -File: hledger.info, Node: Commodity, Next: Costs, Prev: Digit group marks, Up: Amounts +File: hledger.info, Node: Commodity, Prev: Digit group marks, Up: Amounts 8.11.3 Commodity ---------------- @@ -2063,63 +2107,176 @@ hledger displays them in output. This is explained in Commodity display style below.  -File: hledger.info, Node: Costs, Prev: Commodity, Up: Amounts +File: hledger.info, Node: Costs, Next: Basis / lots, Prev: Amounts, Up: Journal + +8.12 Costs +========== + +In traditional double entry bookkeeping, to record a transaction where +one commodity is exchanged for another, you add extra equity postings to +balance the two commodities. Eg: + +2026-01-01 buy euros + assets:dollars $-123 + equity:conversion $123 + equity:conversion €-100 + assets:euros €100 + + hledger offers a more convenient @/@@ "cost notation" as an +alternative: instead of equity postings, you can write the "conversion +rate" or "transacted price" after a posting amount. hledger docs +generically call this "cost", whether buying or selling. ("cost" is an +overloaded and generic term here, but we still use it for historical +reasons. "Transacted price" is more precise.) + + It can be written as either '@ UNITPRICE' or '@@ TOTALPRICE'. Eg you +could write the above as: + +2026-01-01 buy euros + assets:dollars $-123 + assets:euros €100 @ $1.23 ; unit cost (exchange rate) + + or: + +2026-01-01 buy euros + assets:dollars $-123 + assets:euros €100 @@ $123 ; total cost + + The cost should normally be a positive amount. Negative costs are +supported, but can be confusing, as discussed at -infer-market-prices: +market prices from transactions. + + Costs participate in transaction balancing. Amounts are converted to +their cost before checking if the transaction is balanced. You could +also write the above less redundantly, like so: + +2026-01-01 buy euros + assets:dollars ; $-123 is inferred + assets:euros €100 @ $1.23 + + or: + +2026-01-01 buy euros + assets:dollars ; $-123 is inferred + assets:euros €100 @@ $123 + + or even: + +2026-01-01 buy euros + assets:euros €100 ; @@ $123 is inferred + assets:dollars $-123 + + This last form works for transactions involving exactly two +commodities, with neither cost notation nor equity postings. Note, the +order of postings is significant: the cost will be attached to the first +(top) posting. So we had to switch the order of postings, to get the +same meaning as above. Also, this form is the easiest to make +undetected errors with; so it is rejected by 'hledger check balanced', +and by strict mode. + + Advantages of cost notation: + + 1. it's more compact and easier to read and write + 2. hledger reports can show such amounts converted to their cost, when + you add the '-B/--cost' flag (see Cost reporting). + + Advantages of equity postings + + 1. they help to keep the accounting equation balanced (if you care + about that) + 2. they translate easily to any other double entry accounting system. + + Most hledger users use cost notation and don't use equity postings. + + But you can always convert cost notation to equity postings by adding +'--infer-equity'. Eg try 'hledger print -x --infer-equity'. + + And you can usually convert equity postings to cost notation by +adding '--infer-costs' (see Requirements for detecting equity conversion +postings). Eg try 'hledger print -x --infer-costs'. + + Finally: using both equity postings and cost notation at the same +time is allowed, as long as the journal entry is well formed such that +the equity postings / cost equivalences can be detected. (Otherwise +you'll get an error message saying that the transaction is unbalanced.): + +2026-01-01 buy euros + assets:dollars $-123 + equity:conversion $123 + equity:conversion €-100 + assets:euros €100 @ $1.23 + + So in principle you could enable both '--infer-equity' and +'--infer-costs' in your config file, and your reports would have the +advantages of both. + + +File: hledger.info, Node: Basis / lots, Next: Balance assertions, Prev: Costs, Up: Journal -8.11.4 Costs ------------- +8.13 Basis / lots +================= -After a posting amount, you can note its cost (when buying) or selling -price (when selling) in another commodity, by writing either '@ -UNITPRICE' or '@@ TOTALPRICE' after it. This indicates a conversion -transaction, where one commodity is exchanged for another. +This is an advanced topic; skip over if you don't need it. For full +details, see Lot reporting. - (You might also see this called "transaction price" in hledger docs, -discussions, or code; that term was directionally neutral and reminded -that it is a price specific to a transaction, but we now just call it -"cost", with the understanding that the transaction could be a purchase -or a sale.) + If you are buying an amount of some commodity to hold as an +investment, it may be important to keep track of its _cost basis_ (AKA +"basis") and _lots_. ("Cost basis" and "cost" (see above) sound +similar, and often have the same amount in simple purchases; but they +are distinct concepts.) - Costs are usually written explicitly with '@' or '@@', but can also -be inferred automatically for simple multi-commodity transactions. -Note, if costs are inferred, the order of postings is significant; the -first posting will have a cost attached, in the commodity of the second. + The basis records: - As an example, here are several ways to record purchases of a foreign -currency in hledger, using the cost notation either explicitly or -implicitly: + 1. the amount's nominal acquisition cost (usually the same as the + transacted cost, ie what you paid for it, but not always) + 2. the nominal acquisition date (usually the date you acquired it, but + not always) + 3. and optionally a label, to disambiguate or identify lots. - 1. Write the price per unit, as '@ UNITPRICE' after the amount: + An amount with a basis is called a lot. The basis is a property of +the lot, remaining with it throughout its lifetime. It is used to +calculate capital gains/losses, eg for tax reporting. - 2009/1/1 - assets:euros €100 @ $1.35 ; one hundred euros purchased at $1.35 each - assets:dollars ; balancing amount is -$135.00 +* Menu: - 2. Write the total price, as '@@ TOTALPRICE' after the amount: +* Lot syntax:: - 2009/1/1 - assets:euros €100 @@ $135 ; one hundred euros purchased at $135 for the lot - assets:dollars + +File: hledger.info, Node: Lot syntax, Up: Basis / lots - 3. Specify amounts for all postings, using exactly two commodities, - and let hledger infer the price that balances the transaction. - Note the effect of posting order: the price is added to first - posting, making it '€100 @@ $135', as in example 2: +8.13.1 Lot syntax +----------------- - 2009/1/1 - assets:euros €100 ; one hundred euros purchased - assets:dollars $-135 ; for $135 +A basis or lot can be described using "lot syntax". hledger supports +two lot syntax styles: - Amounts can be converted to cost at report time using the '-B/--cost' -flag; this is discussed more in the Cost reporting section. + *hledger lot syntax* puts all basis fields inside braces, +comma-separated (like Beancount): - Note that the cost normally should be a positive amount, though it's -not required to be. This can be a little confusing, see discussion at --infer-market-prices: market prices from transactions. +10 AAPL {2026-01-15, $50} +10 AAPL {2026-01-15, "lot1", $50} +10 AAPL {$50} +10 AAPL {} + + All fields are optional, but when present they must be in +date-label-cost order. Dates must be in YYYY-MM-DD format, and labels +must be double-quoted. The cost is a single-commodity costless hledger +amount. + + When an amount has both a basis and a transacted price, like '-10 +AAPL {$50} @ $60', the preferred order is to write {} before @. + + *Ledger lot syntax* uses separate annotations, in any order: + +10 AAPL {$50} [2026-01-15] (lot1) + + hledger accepts both styles on input. Print output uses consolidated +style by default, or Ledger style with '-O ledger'.  -File: hledger.info, Node: Balance assertions, Next: Posting comments, Prev: Amounts, Up: Journal +File: hledger.info, Node: Balance assertions, Next: Posting comments, Prev: Basis / lots, Up: Journal -8.12 Balance assertions +8.14 Balance assertions ======================= hledger supports Ledger-style balance assertions in journal files. @@ -2159,7 +2316,7 @@ does not disable balance assignments, described below).  File: hledger.info, Node: Assertions and ordering, Next: Assertions and multiple files, Up: Balance assertions -8.12.1 Assertions and ordering +8.14.1 Assertions and ordering ------------------------------ hledger calculates and checks an account's balance assertions in date @@ -2176,7 +2333,7 @@ updating.  File: hledger.info, Node: Assertions and multiple files, Next: Assertions and costs, Prev: Assertions and ordering, Up: Balance assertions -8.12.2 Assertions and multiple files +8.14.2 Assertions and multiple files ------------------------------------ If an account has transactions appearing in multiple files, balance @@ -2202,7 +2359,7 @@ order. (Discussion welcome.)  File: hledger.info, Node: Assertions and costs, Next: Assertions and commodities, Prev: Assertions and multiple files, Up: Balance assertions -8.12.3 Assertions and costs +8.14.3 Assertions and costs --------------------------- Balance assertions ignore costs, and should normally be written without @@ -2220,7 +2377,7 @@ costs), and because balance _assignments_ do use costs (see below).  File: hledger.info, Node: Assertions and commodities, Next: Assertions and subaccounts, Prev: Assertions and costs, Up: Balance assertions -8.12.4 Assertions and commodities +8.14.4 Assertions and commodities --------------------------------- The balance assertions described so far are "*single commodity balance @@ -2269,7 +2426,7 @@ specified commodities and no others. It can be done by  File: hledger.info, Node: Assertions and subaccounts, Next: Assertions and status, Prev: Assertions and commodities, Up: Balance assertions -8.12.5 Assertions and subaccounts +8.14.5 Assertions and subaccounts --------------------------------- All of the balance assertions above (both '=' and '==') are @@ -2288,7 +2445,7 @@ by adding a star after the equals ('=*' or '==*'):  File: hledger.info, Node: Assertions and status, Next: Assertions and virtual postings, Prev: Assertions and subaccounts, Up: Balance assertions -8.12.6 Assertions and status +8.14.6 Assertions and status ---------------------------- Balance assertions always consider postings of all statuses (unmarked, @@ -2298,7 +2455,7 @@ pending, or cleared); they are not affected by the '-U'/'--unmarked' /  File: hledger.info, Node: Assertions and virtual postings, Next: Assertions and auto postings, Prev: Assertions and status, Up: Balance assertions -8.12.7 Assertions and virtual postings +8.14.7 Assertions and virtual postings -------------------------------------- Balance assertions always consider both real and virtual postings; they @@ -2307,7 +2464,7 @@ are not affected by the '--real/-R' flag or 'real:' query.  File: hledger.info, Node: Assertions and auto postings, Next: Assertions and precision, Prev: Assertions and virtual postings, Up: Balance assertions -8.12.8 Assertions and auto postings +8.14.8 Assertions and auto postings ----------------------------------- Balance assertions _are_ affected by the '--auto' flag, which generates @@ -2326,7 +2483,7 @@ these. So to avoid making fragile assertions, either:  File: hledger.info, Node: Assertions and precision, Next: Assertions and hledger add, Prev: Assertions and auto postings, Up: Balance assertions -8.12.9 Assertions and precision +8.14.9 Assertions and precision ------------------------------- Balance assertions compare the exactly calculated amounts, which are not @@ -2337,7 +2494,7 @@ assertion failure messages show exact amounts.  File: hledger.info, Node: Assertions and hledger add, Prev: Assertions and precision, Up: Balance assertions -8.12.10 Assertions and hledger add +8.14.10 Assertions and hledger add ---------------------------------- Balance assertions can be included in the amounts given in 'add'. All @@ -2357,7 +2514,7 @@ with '-I'.  File: hledger.info, Node: Posting comments, Next: Transaction balancing, Prev: Balance assertions, Up: Journal -8.13 Posting comments +8.15 Posting comments ===================== Text following ';', at the end of a posting line, and/or on indented @@ -2374,7 +2531,7 @@ tags, which are not ignored.  File: hledger.info, Node: Transaction balancing, Next: Tags, Prev: Posting comments, Up: Journal -8.14 Transaction balancing +8.16 Transaction balancing ========================== How exactly does hledger decide when a transaction is balanced ? @@ -2420,7 +2577,7 @@ hit this problem, it's easy to fix:  File: hledger.info, Node: Tags, Next: Directives, Prev: Transaction balancing, Up: Journal -8.15 Tags +8.17 Tags ========= Tags are a way to add extra labels or data fields to transactions, @@ -2477,7 +2634,7 @@ account assets:checking ; acct-number: 123-45-6789  File: hledger.info, Node: Tag propagation, Next: Displaying tags, Up: Tags -8.15.1 Tag propagation +8.17.1 Tag propagation ---------------------- In addition to what they are attached to, tags also affect related data @@ -2516,7 +2673,7 @@ transaction p2tag, atag posting, p2tag and atag from second  File: hledger.info, Node: Displaying tags, Next: When to use tags ?, Prev: Tag propagation, Up: Tags -8.15.2 Displaying tags +8.17.2 Displaying tags ---------------------- You can use the 'tags' command to list tag names or values. @@ -2529,7 +2686,7 @@ ways (eg appended to account names, like pseudo subaccounts).  File: hledger.info, Node: When to use tags ?, Next: Tag names, Prev: Displaying tags, Up: Tags -8.15.3 When to use tags ? +8.17.3 When to use tags ? ------------------------- Tags provide more dimensions of categorisation, complementing accounts @@ -2544,7 +2701,7 @@ your usual account categories.  File: hledger.info, Node: Tag names, Prev: When to use tags ?, Up: Tags -8.15.4 Tag names +8.17.4 Tag names ---------------- What is allowed in a tag name ? Most non-whitespace characters. Eg '😀 @@ -2578,13 +2735,13 @@ hledger. They are explained elsewhere, but here's a quick reference: The second group above (generated-transaction, etc.) are normally hidden, with a '_' prefix added. This means 'print' doesn't show them by default; but you can still use them in queries. You can add the -'--verbose-tags' flag to make them visible, which can be useful for -troubleshooting. +'--verbose-tags' flag to make them visible in 'print' output, which can +be useful for troubleshooting.  File: hledger.info, Node: Directives, Next: account directive, Prev: Tags, Up: Journal -8.16 Directives +8.18 Directives =============== Besides transactions, there is something else you can put in a 'journal' @@ -2624,7 +2781,7 @@ Declare market prices 'P'  File: hledger.info, Node: Directives and multiple files, Next: Directive effects, Up: Directives -8.16.1 Directives and multiple files +8.18.1 Directives and multiple files ------------------------------------ Directives vary in their scope, ie which journal entries and which input @@ -2644,7 +2801,7 @@ directives in your files.  File: hledger.info, Node: Directive effects, Prev: Directives and multiple files, Up: Directives -8.16.2 Directive effects +8.18.2 Directive effects ------------------------ Here are all hledger's directives, with their effects and scope @@ -2705,7 +2862,7 @@ directives*  File: hledger.info, Node: account directive, Next: alias directive, Prev: Directives, Up: Journal -8.17 'account' directive +8.19 'account' directive ======================== 'account' directives can be used to declare accounts (ie, the places @@ -2739,32 +2896,44 @@ account assets:bank:checking * Menu: * Account comments:: +* Account tags:: * Account error checking:: * Account display order:: * Account types::  -File: hledger.info, Node: Account comments, Next: Account error checking, Up: account directive +File: hledger.info, Node: Account comments, Next: Account tags, Up: account directive -8.17.1 Account comments +8.19.1 Account comments ----------------------- Text following *two or more spaces* and ';' at the end of an account directive line, and/or following ';' on indented lines immediately below -it, form comments for that account. They are ignored except they may -contain tags, which are not ignored. +it, form comments for that account. - The two-space requirement for same-line account comments is because -';' is allowed in account names. + Same-line account comments require two+ spaces before ';' because +that character can appear in account names. account assets:bank:checking ; same-line comment, at least 2 spaces before the semicolon ; next-line comment ; some tags - type:A, acctnum:12345  -File: hledger.info, Node: Account error checking, Next: Account display order, Prev: Account comments, Up: account directive +File: hledger.info, Node: Account tags, Next: Account error checking, Prev: Account comments, Up: account directive + +8.19.2 Account tags +------------------- + +An account directive's comment may contain tags. These will be +inherited by all postings using that account, except where the posting +already has a value for that tag. (A posting tag overrides an account +tag.) Note, these tags will be queryable but won't be shown in 'print' +output, even with -verbose-tags. + + +File: hledger.info, Node: Account error checking, Next: Account display order, Prev: Account tags, Up: account directive -8.17.2 Account error checking +8.19.3 Account error checking ----------------------------- By default, accounts need not be declared; they come into existence when @@ -2789,13 +2958,16 @@ account directive. Some notes: included files of all types. * It's currently not possible to declare "all possible subaccounts" with a wildcard; every account posted to must be declared. + * As an exception: lot subaccounts (a final account name component + like ':{2026-01-15, $50}') are always ignored by 'check accounts', + and need not be declared. * If you use the -infer-equity flag, you will also need declarations for the account names it generates.  File: hledger.info, Node: Account display order, Next: Account types, Prev: Account error checking, Up: account directive -8.17.3 Account display order +8.19.4 Account display order ---------------------------- Account directives also cause hledger to display accounts in a @@ -2837,7 +3009,7 @@ the other.  File: hledger.info, Node: Account types, Prev: Account display order, Up: account directive -8.17.4 Account types +8.19.5 Account types -------------------- hledger knows that in accounting there are three main account types: @@ -2851,15 +3023,20 @@ hledger knows that in accounting there are three main account types: 'Revenue' 'R' inflows (also known as 'Income') 'Expense' 'X' outflows - hledger also uses a couple of subtypes: + hledger also uses a few subtypes: -'Cash' 'C' liquid assets -'Conversion' 'V' commodity conversions equity +'Cash' 'C' liquid assets (subtype + of Asset) +'Conversion' 'V' commodity conversions + equity (subtype of + Equity) +'Gain' 'G' capital gains/losses + (subtype of Revenue) - As a convenience, hledger will detect these types automatically from -english account names. But it's better to declare them explicitly by -adding a 'type:' tag in the account directives. The tag's value can be -any of the types or one-letter abbreviations above. + As a convenience, hledger will detect most of these types +automatically from english account names. But it's better to declare +them explicitly by adding a 'type:' tag in the account directives. The +tag's value can be any of the types or one-letter abbreviations above. Here is a typical set of account type declarations. Subaccounts will inherit their parent's type, or can override it: @@ -2875,6 +3052,8 @@ account assets:cash ; type: C account equity:conversion ; type: V +account revenues:capital ; type: G + This enables the easy balancesheet, balancesheetequity, cashflow and incomestatement reports, and querying by type:. @@ -2920,7 +3099,7 @@ incomestatement reports, and querying by type:.  File: hledger.info, Node: alias directive, Next: commodity directive, Prev: account directive, Up: Journal -8.18 'alias' directive +8.20 'alias' directive ====================== You can define account alias rules which rewrite your account names, or @@ -2957,7 +3136,7 @@ more on this below.  File: hledger.info, Node: Basic aliases, Next: Regex aliases, Up: alias directive -8.18.1 Basic aliases +8.20.1 Basic aliases -------------------- To set an account alias, use the 'alias' directive in your journal file. @@ -2981,7 +3160,7 @@ alias checking = assets:bank:wells fargo:checking  File: hledger.info, Node: Regex aliases, Next: Combining aliases, Prev: Basic aliases, Up: alias directive -8.18.2 Regex aliases +8.20.2 Regex aliases -------------------- There is also a more powerful variant that uses a regular expression, @@ -3015,7 +3194,7 @@ of option argument), so it can contain trailing whitespace.  File: hledger.info, Node: Combining aliases, Next: Aliases and multiple files, Prev: Regex aliases, Up: alias directive -8.18.3 Combining aliases +8.20.3 Combining aliases ------------------------ You can define as many aliases as you like, using journal directives @@ -3052,7 +3231,7 @@ which aliases are being applied when.  File: hledger.info, Node: Aliases and multiple files, Next: end aliases directive, Prev: Combining aliases, Up: alias directive -8.18.4 Aliases and multiple files +8.20.4 Aliases and multiple files --------------------------------- As explained at Directives and multiple files, 'alias' directives do not @@ -3084,7 +3263,7 @@ include c.journal ; also affected  File: hledger.info, Node: end aliases directive, Next: Aliases can generate bad account names, Prev: Aliases and multiple files, Up: alias directive -8.18.5 'end aliases' directive +8.20.5 'end aliases' directive ------------------------------ You can clear (forget) all currently defined aliases (seen in the @@ -3095,7 +3274,7 @@ end aliases  File: hledger.info, Node: Aliases can generate bad account names, Next: Aliases and account types, Prev: end aliases directive, Up: alias directive -8.18.6 Aliases can generate bad account names +8.20.6 Aliases can generate bad account names --------------------------------------------- Be aware that account aliases can produce malformed account names, which @@ -3126,7 +3305,7 @@ $ hledger print --alias old="new USD" | hledger -f- print  File: hledger.info, Node: Aliases and account types, Prev: Aliases can generate bad account names, Up: alias directive -8.18.7 Aliases and account types +8.20.7 Aliases and account types -------------------------------- If an account with a type declaration (see Declaring accounts > Account @@ -3150,7 +3329,7 @@ $ hledger accounts --types -1 --alias assets=bassetts  File: hledger.info, Node: commodity directive, Next: decimal-mark directive, Prev: alias directive, Up: Journal -8.19 'commodity' directive +8.21 'commodity' directive ========================== The 'commodity' directive performs several functions: @@ -3193,24 +3372,23 @@ command line option. * Menu: * Commodity directive syntax:: +* Commodity tags:: * Commodity error checking::  -File: hledger.info, Node: Commodity directive syntax, Next: Commodity error checking, Up: commodity directive +File: hledger.info, Node: Commodity directive syntax, Next: Commodity tags, Up: commodity directive -8.19.1 Commodity directive syntax +8.21.1 Commodity directive syntax --------------------------------- A commodity directive is normally the word 'commodity' followed by a -sample amount (and optionally a comment). Only the amount's symbol and +sample amount, and optionally a comment. Only the amount's symbol and the number's format is significant. Eg: commodity $1000.00 commodity 1.000,00 EUR commodity 1 000 000.0000 ; the no-symbol commodity - Commodities do not have tags (tags in the comment will be ignored). - A commodity directive's sample amount must always include a period or comma decimal mark (this rule helps disambiguate decimal marks and digit group marks). If you don't want to show any decimal digits, write the @@ -3243,9 +3421,22 @@ commodity INR an unsupported subdirective ; ignored by hledger  -File: hledger.info, Node: Commodity error checking, Prev: Commodity directive syntax, Up: commodity directive +File: hledger.info, Node: Commodity tags, Next: Commodity error checking, Prev: Commodity directive syntax, Up: commodity directive + +8.21.2 Commodity tags +--------------------- + +A commodity directive's comment may contain tags. These will be +inherited by all postings using that commodity in their main amount, +except where the posting already has a value for that tag. (A posting +tag or an account tag overrides a commodity tag.) Note, these tags will +be queryable but won't be shown in 'print' output, even with +-verbose-tags. + + +File: hledger.info, Node: Commodity error checking, Prev: Commodity tags, Up: commodity directive -8.19.2 Commodity error checking +8.21.3 Commodity error checking ------------------------------- In strict mode ('-s'/'--strict') (or when you run 'hledger check @@ -3257,7 +3448,7 @@ have no commodity symbol.) It works like account error checking  File: hledger.info, Node: decimal-mark directive, Next: include directive, Prev: commodity directive, Up: Journal -8.20 'decimal-mark' directive +8.22 'decimal-mark' directive ============================= You can use a 'decimal-mark' directive - usually one per file, at the @@ -3277,7 +3468,7 @@ thousands separators).  File: hledger.info, Node: include directive, Next: P directive, Prev: decimal-mark directive, Up: Journal -8.21 'include' directive +8.23 'include' directive ======================== You can pull in the content of additional files by writing an include @@ -3331,7 +3522,7 @@ error that's hard to pinpoint: a good troubleshooting command is  File: hledger.info, Node: P directive, Next: payee directive, Prev: include directive, Up: Journal -8.22 'P' directive +8.24 'P' directive ================== The 'P' directive declares a market price, which is a conversion rate @@ -3361,7 +3552,7 @@ amount values in another commodity. See Value reporting.  File: hledger.info, Node: payee directive, Next: tag directive, Prev: P directive, Up: Journal -8.23 'payee' directive +8.25 'payee' directive ====================== 'payee PAYEE NAME' @@ -3384,7 +3575,7 @@ payee ""  File: hledger.info, Node: tag directive, Next: Periodic transactions, Prev: payee directive, Up: Journal -8.24 'tag' directive +8.26 'tag' directive ==================== 'tag TAGNAME' @@ -3404,7 +3595,7 @@ check your tags .  File: hledger.info, Node: Periodic transactions, Next: Auto postings, Prev: tag directive, Up: Journal -8.25 Periodic transactions +8.27 Periodic transactions ========================== The '~' directive declares a "periodic rule" which generates temporary @@ -3452,7 +3643,7 @@ this whole section, or at least the following tips:  File: hledger.info, Node: Periodic rule syntax, Next: Periodic rules and relative dates, Up: Periodic transactions -8.25.1 Periodic rule syntax +8.27.1 Periodic rule syntax --------------------------- A periodic transaction rule looks like a normal journal entry, with the @@ -3477,7 +3668,7 @@ dates).  File: hledger.info, Node: Periodic rules and relative dates, Next: Two spaces between period expression and description!, Prev: Periodic rule syntax, Up: Periodic transactions -8.25.2 Periodic rules and relative dates +8.27.2 Periodic rules and relative dates ---------------------------------------- Partial or relative dates (like '12/31', '25', 'tomorrow', 'last week', @@ -3496,7 +3687,7 @@ dates.  File: hledger.info, Node: Two spaces between period expression and description!, Prev: Periodic rules and relative dates, Up: Periodic transactions -8.25.3 Two spaces between period expression and description! +8.27.3 Two spaces between period expression and description! ------------------------------------------------------------ If the period expression is followed by a transaction description, these @@ -3521,7 +3712,7 @@ accidentally alter their meaning, as in this example:  File: hledger.info, Node: Auto postings, Next: Other syntax, Prev: Periodic transactions, Up: Journal -8.26 Auto postings +8.28 Auto postings ================== The '=' directive declares an "auto posting rule", which adds extra @@ -3618,7 +3809,7 @@ output into the journal file to make it permanent.  File: hledger.info, Node: Auto postings and multiple files, Next: Auto postings and dates, Up: Auto postings -8.26.1 Auto postings and multiple files +8.28.1 Auto postings and multiple files --------------------------------------- An auto posting rule can affect any transaction in the current file, or @@ -3628,7 +3819,7 @@ sibling files (when multiple '-f'/'--file' are used - see #1212).  File: hledger.info, Node: Auto postings and dates, Next: Auto postings and transaction balancing / inferred amounts / balance assertions, Prev: Auto postings and multiple files, Up: Auto postings -8.26.2 Auto postings and dates +8.28.2 Auto postings and dates ------------------------------ A posting date (or secondary date) in the matched posting, or (taking @@ -3638,7 +3829,7 @@ used in the generated posting.  File: hledger.info, Node: Auto postings and transaction balancing / inferred amounts / balance assertions, Next: Auto posting tags, Prev: Auto postings and dates, Up: Auto postings -8.26.3 Auto postings and transaction balancing / inferred amounts / +8.28.3 Auto postings and transaction balancing / inferred amounts / ------------------------------------------------------------------- balance assertions Currently, auto postings are added: @@ -3658,7 +3849,7 @@ infer amounts.  File: hledger.info, Node: Auto posting tags, Next: Auto postings on forecast transactions only, Prev: Auto postings and transaction balancing / inferred amounts / balance assertions, Up: Auto postings -8.26.4 Auto posting tags +8.28.4 Auto posting tags ------------------------ Automated postings will have some extra tags: @@ -3680,7 +3871,7 @@ will have these tags added:  File: hledger.info, Node: Auto postings on forecast transactions only, Prev: Auto posting tags, Up: Auto postings -8.26.5 Auto postings on forecast transactions only +8.28.5 Auto postings on forecast transactions only -------------------------------------------------- Tip: you can can make auto postings that will apply to forecast @@ -3691,7 +3882,7 @@ generating new journal entries to be saved in the journal.  File: hledger.info, Node: Other syntax, Prev: Auto postings, Up: Journal -8.27 Other syntax +8.29 Other syntax ================= hledger journal format supports quite a few other features, mainly to @@ -3713,12 +3904,13 @@ you decide if you want to use them. * Valuation expressions:: * Virtual postings:: * Other Ledger directives:: -* Other cost/lot notations:: +* Ledger virtual costs:: +* Ledger lot syntax::  File: hledger.info, Node: Balance assignments, Next: Bracketed posting dates, Up: Other syntax -8.27.1 Balance assignments +8.29.1 Balance assignments -------------------------- Ledger-style balance assignments are also supported. These are like @@ -3761,7 +3953,7 @@ trustworthy in an audit.  File: hledger.info, Node: Balance assignments and costs, Next: Balance assignments and multiple files, Up: Balance assignments -8.27.1.1 Balance assignments and costs +8.29.1.1 Balance assignments and costs ...................................... A cost in a balance assignment will cause the calculated amount to have @@ -3777,7 +3969,7 @@ $ hledger print --explicit  File: hledger.info, Node: Balance assignments and multiple files, Prev: Balance assignments and costs, Up: Balance assignments -8.27.1.2 Balance assignments and multiple files +8.29.1.2 Balance assignments and multiple files ............................................... Balance assignments handle multiple files like balance assertions. They @@ -3787,7 +3979,7 @@ but not from previous sibling or parent files.  File: hledger.info, Node: Bracketed posting dates, Next: D directive, Prev: Balance assignments, Up: Other syntax -8.27.2 Bracketed posting dates +8.29.2 Bracketed posting dates ------------------------------ For setting posting dates and secondary posting dates, Ledger's @@ -3804,7 +3996,7 @@ syntax.  File: hledger.info, Node: D directive, Next: apply account directive, Prev: Bracketed posting dates, Up: Other syntax -8.27.3 'D' directive +8.29.3 'D' directive -------------------- 'D AMOUNT' @@ -3850,7 +4042,7 @@ Ledger's 'D'.  File: hledger.info, Node: apply account directive, Next: Y directive, Prev: D directive, Up: Other syntax -8.27.4 'apply account' directive +8.29.4 'apply account' directive -------------------------------- This directive sets a default parent account, which will be prepended to @@ -3886,7 +4078,7 @@ portable, and less trustworthy in an audit.  File: hledger.info, Node: Y directive, Next: Secondary dates, Prev: apply account directive, Up: Other syntax -8.27.5 'Y' directive +8.29.5 'Y' directive -------------------- 'Y YEAR' @@ -3924,7 +4116,7 @@ date.  File: hledger.info, Node: Secondary dates, Next: Star comments, Prev: Y directive, Up: Other syntax -8.27.6 Secondary dates +8.29.6 Secondary dates ---------------------- A secondary date is written after the primary date, following an equals @@ -3960,7 +4152,7 @@ instead.  File: hledger.info, Node: Star comments, Next: Valuation expressions, Prev: Secondary dates, Up: Other syntax -8.27.7 Star comments +8.29.7 Star comments -------------------- Lines beginning with '*' (star/asterisk) are also comment lines. This @@ -3977,7 +4169,7 @@ losing ledger mode's features.  File: hledger.info, Node: Valuation expressions, Next: Virtual postings, Prev: Star comments, Up: Other syntax -8.27.8 Valuation expressions +8.29.8 Valuation expressions ---------------------------- Ledger allows a valuation function or value to be written in double @@ -3986,7 +4178,7 @@ parentheses after an amount. hledger ignores these.  File: hledger.info, Node: Virtual postings, Next: Other Ledger directives, Prev: Valuation expressions, Up: Other syntax -8.27.9 Virtual postings +8.29.9 Virtual postings ----------------------- A posting with parentheses around the account name, like '(some:account) @@ -4016,9 +4208,9 @@ bracketed, are called _real postings_. You can exclude virtual postings from reports with the '-R/--real' flag or a 'real:1' query.  -File: hledger.info, Node: Other Ledger directives, Next: Other cost/lot notations, Prev: Virtual postings, Up: Other syntax +File: hledger.info, Node: Other Ledger directives, Next: Ledger virtual costs, Prev: Virtual postings, Up: Other syntax -8.27.10 Other Ledger directives +8.29.10 Other Ledger directives ------------------------------- These other Ledger directives are currently accepted but ignored. This @@ -4047,49 +4239,29 @@ value EXPR hledger/Ledger syntax comparison.  -File: hledger.info, Node: Other cost/lot notations, Prev: Other Ledger directives, Up: Other syntax - -8.27.11 Other cost/lot notations --------------------------------- - -A slight digression for Ledger and Beancount users. - - *Ledger* has a number of cost/lot-related notations: +File: hledger.info, Node: Ledger virtual costs, Next: Ledger lot syntax, Prev: Other Ledger directives, Up: Other syntax - * '@ UNITCOST' and '@@ TOTALCOST' - * expresses a conversion rate, as in hledger - * when buying, also creates a lot that can be selected at - selling time - - * '(@) UNITCOST' and '(@@) TOTALCOST' (virtual cost) - * like the above, but also means "this cost was exceptional, - don't use it when inferring market prices". +8.29.11 Ledger virtual costs +---------------------------- - * '{=UNITCOST}' and '{{=TOTALCOST}}' (fixed price) - * when buying, means "this cost is also the fixed value, don't - let it fluctuate in value reports" +In Ledger, '(@) UNITCOST' and '(@@) TOTALCOST' are virtual costs, which +do not generate market prices. In hledger, these are equivalent to '@' +and '@@'. - * '{UNITCOST}' and '{{TOTALCOST}}' (lot price) - * can be used identically to '@ UNITCOST' and '@@ TOTALCOST', - also creates a lot - * when selling, combined with '@ ...', selects an existing lot - by its cost basis. Does not check if that lot is present. + +File: hledger.info, Node: Ledger lot syntax, Prev: Ledger virtual costs, Up: Other syntax - * '[YYYY/MM/DD]' (lot date) - * when buying, attaches this acquisition date to the lot - * when selling, selects a lot by its acquisition date +8.29.12 Ledger lot syntax +------------------------- - * '(SOME TEXT)' (lot note) - * when buying, attaches this note to the lot - * when selling, selects a lot by its note +In Ledger, these annotations after an amount help specify or select a +lot's cost basis: '{LOTUNITCOST}' or '{{LOTTOTALCOST}}', '[LOTDATE]', +and/or '(LOTNOTE)'. hledger will read these, as an alternative to its +own lot syntax). - Currently, hledger - - * accepts any or all of the above in any order after the posting - amount - * supports '@' and '@@' - * treats '(@)' and '(@@)' as synonyms for '@' and '@@' - * and ignores the rest. (This can break transaction balancing.) + We also read Ledger's fixed price) syntax, '{=LOTUNITCOST}' or +'{{=LOTTOTALCOST}}', treating it as equivalent to '{LOTUNITCOST}' or +'{{LOTTOTALCOST}}', *Beancount* has simpler notation and different behaviour: @@ -6370,6 +6542,14 @@ be 1-4. 'last/this/next day/week/month/quarter/year' -1, 0, 1 periods from the current period +'last/this/next tuesday' + + the previous occurrence of the named day, or the next occurrence + after today +'last/this/next february' + + the previous occurrence of 1st of the named month, or the next + occurrence after the current month 'in n days/weeks/months/quarters/years' n periods from the current period @@ -6896,11 +7076,15 @@ File: hledger.info, Node: date query, Next: date2 query, Prev: desc query, U *'date:PERIODEXPR'* Match dates (or with the '--date2' flag, secondary dates) within the -specified period. PERIODEXPR is a period expression with no report -interval. Examples: +specified period. PERIODEXPR is a period expression. Examples: 'date:2016', 'date:thismonth', 'date:2/1-2/15', 'date:2021-07-27..nextquarter'. + PERIODEXPR may include a report interval (since 1.52). On the +command line, this is equivalent to specifying a report interval with a +command line option. In other contexts (hledger-ui, hledger-web), the +report interval may be ignored. +  File: hledger.info, Node: date2 query, Next: depth query, Prev: date query, Up: Query types @@ -6909,7 +7093,8 @@ File: hledger.info, Node: date2 query, Next: depth query, Prev: date query, *'date2:PERIODEXPR'* If you use secondary dates: this matches secondary dates within the -specified period. It is not affected by the '--date2' flag. +specified period. It is not affected by the '--date2' flag. A report +interval in PERIODEXPR will be ignored.  File: hledger.info, Node: depth query, Next: note query, Prev: date2 query, Up: Query types @@ -6969,10 +7154,10 @@ File: hledger.info, Node: type query, Next: tag query, Prev: status query, U *'type:TYPECODES'* Match by account type (see Declaring accounts > Account types). 'TYPECODES' is one or more of the single-letter account type codes -'ALERXCV', case insensitive. Note 'type:A' and 'type:E' will also match -their respective subtypes 'C' (Cash) and 'V' (Conversion). Certain -kinds of account alias can disrupt account types, see Rewriting accounts -> Aliases and account types. +'ALERXCVG', case insensitive. Note 'type:A', 'type:E', and 'type:R' +will also match their respective subtypes 'C' (Cash), 'V' (Conversion), +and 'G' (Gain). Certain kinds of account alias can disrupt account +types, see Rewriting accounts > Aliases and account types.  File: hledger.info, Node: tag query, Prev: type query, Up: Query types @@ -7223,7 +7408,7 @@ $ hledger balance Income:Dues --pivot kind:member -2 EUR  -File: hledger.info, Node: Generating data, Next: Forecasting, Prev: Pivoting, Up: Top +File: hledger.info, Node: Generating data, Next: Detecting special postings, Prev: Pivoting, Up: Top 17 Generating data ****************** @@ -7243,6 +7428,8 @@ number of ways. Mostly, this is done only if you request it: auto posting rules. * The '--forecast' option generates transactions from periodic transaction rules. + * The '--lots' flag adds extra lot subaccounts to postings for + detailed lot reporting. * The 'balance --budget' report infers budget goals from periodic transaction rules. * Commands like 'close', 'rewrite', and 'hledger-interest' generate @@ -7264,9 +7451,61 @@ also, so you can always match such data with queries like 'tag:generated' or 'tag:modified'.  -File: hledger.info, Node: Forecasting, Next: Budgeting, Prev: Generating data, Up: Top +File: hledger.info, Node: Detecting special postings, Next: Forecasting, Prev: Generating data, Up: Top -18 Forecasting +18 Detecting special postings +***************************** + +hledger detects certain kinds of postings, both generated and +non-generated, and tags them for additional processing. These are +documented elsewhere, but this section gives an overview of the special +posting detection rules. + + By default, the tags are hidden (with a '_' prefix), so they can be +queried but they won't appear in 'print' output. To also add visible +tags, use '--verbose-tags' (useful for troubleshooting). + +Tag Detected pattern Effect +--------------------------------------------------------------------------- +'conversion-posting'A pair of adjacent, single-commodity,Helps transaction + costless postings to 'Conversion'-type balancer infer costs or + accounts, with a nearby corresponding avoid redundancy in + costful or potentially corresponding commodity conversions + costless posting +'cost-posting'A costful posting whose amount and Helps transaction + transacted cost correspond to a balancer infer costs or + conversion postings pair; or a costless avoid redundancy in + posting matching one of the pair commodity conversions +'generated-posting'Postings generated at runtime Helps users understand + or find postings added + at runtime by hledger +'ptype:acquire'Positive postings with lot annotations,Creates a new lot + or in a lotful commodity/account, with + no matching counterposting +'ptype:dispose'Negative postings with lot annotations,Selects and reduces + or in a lotful commodity/account, with existing lots + no matching counterposting +'ptype:transfer-from'The negative posting of a pair ofMoves lots between + counterpostings, at least one with lot accounts, preserving + annotation or a lotful cost basis + commodity/account; or a negative lot + posting with an equity counterpart + (equity transfer) +'ptype:transfer-to'The positive posting of a transferAs above + pair; or a positive lot posting with an + equity counterpart (equity transfer, + e.g. opening balances) +'ptype:gain'A posting to a 'Gain'-type account Helps transaction + balancer avoid + redundancy, helps + disposal balancer check + realised capital + gain/loss + + +File: hledger.info, Node: Forecasting, Next: Budgeting, Prev: Detecting special postings, Up: Top + +19 Forecasting ************** Forecasting, or speculative future reporting, can be useful for @@ -7289,7 +7528,7 @@ when you want to see them.  File: hledger.info, Node: --forecast, Next: Inspecting forecast transactions, Up: Forecasting -18.1 -forecast +19.1 -forecast ============== There is another way: with the '--forecast' option, hledger can generate @@ -7313,7 +7552,7 @@ that the '=' is required.  File: hledger.info, Node: Inspecting forecast transactions, Next: Forecast reports, Prev: --forecast, Up: Forecasting -18.2 Inspecting forecast transactions +19.2 Inspecting forecast transactions ===================================== 'print' is the best command for inspecting and troubleshooting forecast @@ -7357,7 +7596,7 @@ reproducible.)  File: hledger.info, Node: Forecast reports, Next: Forecast tags, Prev: Inspecting forecast transactions, Up: Forecasting -18.3 Forecast reports +19.3 Forecast reports ===================== Forecast transactions affect all reports, as you would expect. Eg: @@ -7382,7 +7621,7 @@ Balance changes in 2023-05-01..2023-09-30:  File: hledger.info, Node: Forecast tags, Next: Forecast period in detail, Prev: Forecast reports, Up: Forecasting -18.4 Forecast tags +19.4 Forecast tags ================== Forecast transactions generated by -forecast have a hidden tag, @@ -7398,7 +7637,7 @@ rule was responsible.  File: hledger.info, Node: Forecast period in detail, Next: Forecast troubleshooting, Prev: Forecast tags, Up: Forecasting -18.5 Forecast period, in detail +19.5 Forecast period, in detail =============================== Forecast start/end dates are chosen so as to do something useful by @@ -7429,7 +7668,7 @@ default in almost all situations, while also being flexible. Here are  File: hledger.info, Node: Forecast troubleshooting, Prev: Forecast period in detail, Up: Forecasting -18.6 Forecast troubleshooting +19.6 Forecast troubleshooting ============================= When -forecast is not doing what you expect, one of these tips should @@ -7457,7 +7696,7 @@ help:  File: hledger.info, Node: Budgeting, Next: Amount formatting, Prev: Forecasting, Up: Top -19 Budgeting +20 Budgeting ************ With the balance command's '--budget' report, each periodic transaction @@ -7474,7 +7713,7 @@ bal -M --budget --forecast ...'  File: hledger.info, Node: Amount formatting, Next: Cost reporting, Prev: Budgeting, Up: Top -20 Amount formatting +21 Amount formatting ******************** * Menu: @@ -7487,7 +7726,7 @@ File: hledger.info, Node: Amount formatting, Next: Cost reporting, Prev: Budg  File: hledger.info, Node: Commodity display style, Next: Rounding, Up: Amount formatting -20.1 Commodity display style +21.1 Commodity display style ============================ For the amounts in each commodity, hledger chooses a consistent display @@ -7530,7 +7769,7 @@ as decimal mark, and two decimal digits).  File: hledger.info, Node: Rounding, Next: Trailing decimal marks, Prev: Commodity display style, Up: Amount formatting -20.2 Rounding +21.2 Rounding ============= Amounts are stored internally as decimal numbers with up to 255 decimal @@ -7544,7 +7783,7 @@ decimal digits appears as "0".  File: hledger.info, Node: Trailing decimal marks, Next: Amount parseability, Prev: Rounding, Up: Amount formatting -20.3 Trailing decimal marks +21.3 Trailing decimal marks =========================== If you're wondering why your 'print' report sometimes shows trailing @@ -7578,7 +7817,7 @@ $ hledger print -c '$1,000.00' --round=soft  File: hledger.info, Node: Amount parseability, Prev: Trailing decimal marks, Up: Amount formatting -20.4 Amount parseability +21.4 Amount parseability ======================== More generally, hledger output falls into three rough categories, which @@ -7617,7 +7856,7 @@ by humans)*  File: hledger.info, Node: Cost reporting, Next: Value reporting, Prev: Amount formatting, Up: Top -21 Cost reporting +22 Cost reporting ***************** In some transactions - for example a currency conversion, or a purchase @@ -7640,7 +7879,7 @@ can show amounts "at cost", converted to the cost's commodity.  File: hledger.info, Node: Recording costs, Next: Reporting at cost, Up: Cost reporting -21.1 Recording costs +22.1 Recording costs ==================== We'll explore several ways of recording transactions involving costs. @@ -7694,7 +7933,7 @@ sure you have none of these by using '-s' (strict mode), or by running  File: hledger.info, Node: Reporting at cost, Next: Equity conversion postings, Prev: Recording costs, Up: Cost reporting -21.2 Reporting at cost +22.2 Reporting at cost ====================== Now when you add the '-B'/'--cost' flag to reports ("B" is from Ledger's @@ -7714,7 +7953,7 @@ they will be displayed "at cost" or "at sale price".  File: hledger.info, Node: Equity conversion postings, Next: Inferring equity conversion postings, Prev: Reporting at cost, Up: Cost reporting -21.3 Equity conversion postings +22.3 Equity conversion postings =============================== There is a problem with the entries above - they are not conventional @@ -7770,7 +8009,7 @@ $ hledger bal --infer-costs -B  File: hledger.info, Node: Inferring equity conversion postings, Next: Combining costs and equity conversion postings, Prev: Equity conversion postings, Up: Cost reporting -21.4 Inferring equity conversion postings +22.4 Inferring equity conversion postings ========================================= Can we go in the other direction ? Yes, if you have transactions @@ -7794,12 +8033,14 @@ symbol. You can customise the "equity:conversion" part by declaring an account with the 'V'/'Conversion' account type. Note you will need to add account declarations for these to your -journal, if you use 'check accounts' or 'check --strict'. +journal, if you use 'check accounts' or 'check --strict'. (And unlike +normal postings, generated equity postings do not inherit tags from +account declarations.)  File: hledger.info, Node: Combining costs and equity conversion postings, Next: Requirements for detecting equity conversion postings, Prev: Inferring equity conversion postings, Up: Cost reporting -21.5 Combining costs and equity conversion postings +22.5 Combining costs and equity conversion postings =================================================== Finally, you can use both the @/@@ cost notation and equity postings at @@ -7833,7 +8074,7 @@ $ hledger print -x --infer-costs --infer-equity  File: hledger.info, Node: Requirements for detecting equity conversion postings, Next: Infer cost and equity by default ?, Prev: Combining costs and equity conversion postings, Up: Cost reporting -21.6 Requirements for detecting equity conversion postings +22.6 Requirements for detecting equity conversion postings ========================================================== '--infer-costs' has certain requirements (unlike '--infer-equity', which @@ -7864,7 +8105,7 @@ fails, hledger raises an "unbalanced transaction" error.  File: hledger.info, Node: Infer cost and equity by default ?, Prev: Requirements for detecting equity conversion postings, Up: Cost reporting -21.7 Infer cost and equity by default ? +22.7 Infer cost and equity by default ? ======================================= Should '--infer-costs' and '--infer-equity' be enabled by default ? Try @@ -7875,9 +8116,9 @@ alias h="hledger --infer-equity --infer-costs" and let us know what problems you find.  -File: hledger.info, Node: Value reporting, Next: PART 4 COMMANDS, Prev: Cost reporting, Up: Top +File: hledger.info, Node: Value reporting, Next: Lot reporting, Prev: Cost reporting, Up: Top -22 Value reporting +23 Value reporting ****************** hledger can also show amounts "at market value", converted to some other @@ -7905,7 +8146,7 @@ transactions, by using the '--infer-market-prices' flag.  File: hledger.info, Node: -X Value in specified commodity, Next: -V Value in default commoditys, Up: Value reporting -22.1 -X: Value in specified commodity +23.1 -X: Value in specified commodity ===================================== The '-X COMM' (or '--exchange=COMM') option converts amounts to their @@ -7928,7 +8169,7 @@ special shell characters, if needed. Some examples:  File: hledger.info, Node: -V Value in default commoditys, Next: Valuation date, Prev: -X Value in specified commodity, Up: Value reporting -22.2 -V: Value in default commodity(s) +23.2 -V: Value in default commodity(s) ====================================== The '-V/--market' flag is a variant of '-X' where you don't have to @@ -7944,7 +8185,7 @@ better to use '-X', unless you're sure '-V' is doing what you want.  File: hledger.info, Node: Valuation date, Next: Finding market price, Prev: -V Value in default commoditys, Up: Value reporting -22.3 Valuation date +23.3 Valuation date =================== Market prices can change from day to day. hledger will use the prices @@ -7965,7 +8206,7 @@ can select either "then", "end", "now", or "custom" dates.  File: hledger.info, Node: Finding market price, Next: --infer-market-prices market prices from transactions, Prev: Valuation date, Up: Value reporting -22.4 Finding market price +23.4 Finding market price ========================= To convert a commodity A to its market value in another commodity B, @@ -7999,7 +8240,7 @@ converted.  File: hledger.info, Node: --infer-market-prices market prices from transactions, Next: Valuation commodity, Prev: Finding market price, Up: Value reporting -22.5 -infer-market-prices: market prices from transactions +23.5 -infer-market-prices: market prices from transactions ========================================================== Normally, market value in hledger is fully controlled by, and requires, @@ -8085,7 +8326,7 @@ P 2022-01-03 B A -1.0  File: hledger.info, Node: Valuation commodity, Next: --value Flexible valuation, Prev: --infer-market-prices market prices from transactions, Up: Value reporting -22.6 Valuation commodity +23.6 Valuation commodity ======================== *When you specify a valuation commodity ('-X COMM' or '--value @@ -8124,7 +8365,7 @@ converted.  File: hledger.info, Node: --value Flexible valuation, Next: Valuation examples, Prev: Valuation commodity, Up: Value reporting -22.7 -value: Flexible valuation +23.7 -value: Flexible valuation =============================== '-V' and '-X' are special cases of the more general '--value' option: @@ -8166,7 +8407,7 @@ this commodity, deducing market prices as described above.  File: hledger.info, Node: Valuation examples, Next: Interaction of valuation and queries, Prev: --value Flexible valuation, Up: Value reporting -22.8 Valuation examples +23.8 Valuation examples ======================= Here are some quick examples of '-V': @@ -8277,7 +8518,7 @@ $ hledger -f- print --value=2000-01-15  File: hledger.info, Node: Interaction of valuation and queries, Next: Effect of valuation on reports, Prev: Valuation examples, Up: Value reporting -22.9 Interaction of valuation and queries +23.9 Interaction of valuation and queries ========================================= When matching postings based on queries in the presence of valuation, @@ -8298,7 +8539,7 @@ the following happens:  File: hledger.info, Node: Effect of valuation on reports, Prev: Interaction of valuation and queries, Up: Value reporting -22.10 Effect of valuation on reports +23.10 Effect of valuation on reports ==================================== Here is a reference for how valuation is supposed to affect each part of @@ -8444,9 +8685,289 @@ average totals totals totals column a zero starting balance.  -File: hledger.info, Node: PART 4 COMMANDS, Next: Help commands, Prev: Value reporting, Up: Top +File: hledger.info, Node: Lot reporting, Next: PART 4 COMMANDS, Prev: Value reporting, Up: Top + +24 Lot reporting +**************** + +With the '--lots' flag, hledger can track investment lots automatically: +assigning lot subaccounts on acquisition, selecting lots on disposal +using configurable methods, calculating capital gains, and showing +per-lot balances in all reports. (Since 1.99.1, experimental. For more +technical details, see SPEC-lots.md). + +* Menu: + +* Lotful commodities and accounts:: +* --lots:: +* Lot subaccounts:: +* Lot operations:: +* Reduction methods:: +* Lot postings with balance assertions:: +* Gain postings and disposal balancing:: +* Lot reporting example:: + + +File: hledger.info, Node: Lotful commodities and accounts, Next: --lots, Up: Lot reporting + +24.1 Lotful commodities and accounts +==================================== + +Commodities and accounts can be declared as "lotful" by adding a 'lots' +tag in their declaration: + +commodity AAPL ; lots: +account assets:stocks ; lots: + + This tells hledger that postings involving these always involve lots, +enabling cost basis inference even when lot syntax is not written +explicitly. + + The tag value can also specify a reduction method: + +commodity AAPL ; lots: FIFO +account assets:stocks ; lots: LIFO + + If no value is specified, the default is FIFO. + + +File: hledger.info, Node: --lots, Next: Lot subaccounts, Prev: Lotful commodities and accounts, Up: Lot reporting + +24.2 -lots +========== + +Add '--lots' to any command to enable lot tracking. This activates: + + * *Lot posting classification* -- lot-related postings are tagged as + 'acquire', 'dispose', 'transfer-from', 'transfer-to', or 'gain' + (via a hidden 'ptype' tag, visible with '--verbose-tags', queryable + with 'tag:ptype=...'). + * *Cost basis inference* -- for lotful commodities/accounts, cost + basis is inferred from transacted cost and vice versa. Or when the + account name ends with a lot subaccount, cost basis can also be + inferred from that. + * *Lot calculation* -- acquired lots become subaccounts; disposals + and transfers select from existing lots. + * *Disposal balancing* -- disposal transactions are checked for + balance at cost basis; gain amounts/postings are inferred if + missing. + + +File: hledger.info, Node: Lot subaccounts, Next: Lot operations, Prev: --lots, Up: Lot reporting + +24.3 Lot subaccounts +==================== + +With '--lots', each acquired lot becomes a subaccount named by its cost +basis: + +commodity AAPL ; lots: + +2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:cash -$500 + +$ hledger bal assets:stocks --lots -N + 10 AAPL assets:stocks:{2026-01-15, $50} + + You can also write lot subaccounts explicitly. When a posting's +account name ends with a lot subaccount (like ':{2026-01-15, $50}'), the +cost basis is parsed from it automatically, so a '{}' annotation on the +amount is optional: + +commodity AAPL ; lots: + +2026-01-15 buy + assets:stocks:{2026-01-15, $50} 10 AAPL + assets:cash + + This is equivalent to writing '10 AAPL {2026-01-15, $50}'. (If both +the account name and the amount specify a cost basis, they must agree.) + + When strictly checking account names, lot subaccounts are +automatically exempt -- you only need to declare the base account (eg +'account assets:stocks'), not each individual lot subaccount. + + +File: hledger.info, Node: Lot operations, Next: Reduction methods, Prev: Lot subaccounts, Up: Lot reporting + +24.4 Lot operations +=================== + + * *Acquire*: a positive lot posting creates a new lot. The cost + basis can be specified explicitly with '{}' on the amount, inferred + from the lot subaccount name, or inferred from the transacted cost. + On lotful commodities/accounts, even a bare positive posting (no + '{}' or '@') can be detected as an acquire, with cost inferred from + the transaction's other postings. + * *Transfer*: a matching pair of negative/positive lot postings moves + a lot between accounts, preserving its cost basis. Transfer + postings should not have a transacted price. + * *Dispose*: a negative lot posting sells from one or more existing + lots. It must have a transacted price (the selling price), either + explicit or inferred. -23 PART 4: COMMANDS + An example disposal entry: + +2026-02-01 sell + assets:stocks -5 AAPL {$50} @ $60 + assets:cash $300 + revenue:gains -$50 + + With '--lots', this selects the specified quantity of the matching +lot (which must exist) and will show something like: + +$ hledger print --lots desc:sell +2026-02-01 sell + assets:stocks:{2026-01-15, $50} -5 AAPL {2026-01-15, $50} @ $60 + assets:cash $300 + revenue:gains $-50 + + +File: hledger.info, Node: Reduction methods, Next: Lot postings with balance assertions, Prev: Lot operations, Up: Lot reporting + +24.5 Reduction methods +====================== + +When a disposal or transfer doesn't specify a particular lot (eg the +amount is '-5 AAPL' or '-5 AAPL {}'), hledger selects lot(s) +automatically using a reduction method. The available methods are: + +Method Lots selected Scope Disposal cost + basis +--------------------------------------------------------------------------- +*FIFO* oldest first across all each lot's cost +(default) accounts +*FIFO1* oldest first within one each lot's cost + account +*LIFO* newest first across all each lot's cost + accounts +*LIFO1* newest first within one each lot's cost + account +*HIFO* highest cost across all each lot's cost + first accounts +*HIFO1* highest cost within one each lot's cost + first account +*AVERAGE* oldest first across all weighted average + accounts cost +*AVERAGE1* oldest first within one weighted average + account cost +*SPECID* one specified specified specified lot's + lot account cost + + An explicit lot selector (eg '{2026-01-15, $50}' or '{$50}') uses +specific-identification (SPECID). + + *HIFO* (highest-in-first-out) selects the lot with the highest +per-unit cost first, which can be useful for tax optimization. + + *AVERAGE* uses the weighted average per-unit cost of the entire pool +as the disposal cost basis, rather than each lot's individual cost. +This is required in some jurisdictions (eg Canada's Adjusted Cost Base, +France's PMPA, UK's S104 pools). Lots are still consumed in FIFO order +for bookkeeping purposes. Configure the method via the 'lots:' tag on a +commodity or account declaration: + +commodity AAPL ; lots: FIFO +account assets:stocks ; lots: AVERAGE + + Account tags override commodity tags. + + +File: hledger.info, Node: Lot postings with balance assertions, Next: Gain postings and disposal balancing, Prev: Reduction methods, Up: Lot reporting + +24.6 Lot postings with balance assertions +========================================= + +On a dispose or transfer posting without an explicit lot subaccount, a +balance assertion always refers to the parent account's balance. So if +lot subaccounts are added witih '--lots', the assertion is not affected. + + By contrast, in a journal entry where the lot subaccounts are +recorded explicitly, a balance assertion refers to the lot subaccount's +balance. + + This means that 'hledger print --lots', if it adds explicit lot +subaccounts to a journal entry, could potentially change the meaning of +balance assertions, breaking them. To avoid this, in such cases it will +move the balance assertion to a new zero-amount posting to the parent +account (and make sure it's subaccount-inclusive). (So eg 'hledger -f- +print --lots -x | hledger -f- check assertions' will still pass.) + + +File: hledger.info, Node: Gain postings and disposal balancing, Next: Lot reporting example, Prev: Lot postings with balance assertions, Up: Lot reporting + +24.7 Gain postings and disposal balancing +========================================= + +A *gain posting* is a posting to a Gain-type account (type 'G', a +subtype of Revenue). In disposal transactions, it records the capital +gain or loss, which is the difference between cost basis and selling +price of the lots being sold. + + Accounts named like 'revenue:gains' or 'income:capital-gains' are +detected as Gain accounts automatically, or you can declare one +explicitly: + +account gain/loss ; type: G + + Gain postings have special treatment: + + * *Normal transaction balancing* ignores gain postings (they don't + count toward the balance check), and balances the transaction using + transacted price + * *Disposal balancing* (with '--lots') includes gain postings, and + balances the transaction using cost basis + + An amountless gain posting in a disposal transaction will have its +amount filled in. Or if a disposal transaction is unbalanced at cost +basis and has no gain posting, one is inferred automatically (posting to +the first Gain account, or 'revenue:gains' if none is declared). + + +File: hledger.info, Node: Lot reporting example, Prev: Gain postings and disposal balancing, Up: Lot reporting + +24.8 Lot reporting example +========================== + +commodity AAPL ; lots: + +2026-01-15 buy low + assets:stocks 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 buy high + assets:stocks 10 AAPL {$60} + assets:cash -$600 + +2026-03-01 sell some (FIFO, selects oldest lot first) + assets:stocks -5 AAPL @ $70 + assets:cash $350 + revenue:gains + +$ hledger bal assets:stocks --lots -N + 5 AAPL assets:stocks:{2026-01-15, $50} + 10 AAPL assets:stocks:{2026-02-01, $60} + +$ hledger print --lots -x desc:sell +2026-03-01 sell some (FIFO, selects oldest lot first) + assets:stocks:{2026-01-15, $50} -5 AAPL {2026-01-15, $50} @ $70 + assets:cash $350 + revenue:gains -$100 + +$ hledger print --lots -x desc:sell --verbose-tags +2026-03-01 sell some (FIFO, selects oldest lot first) + assets:stocks:{2026-01-15, $50} -5 AAPL {2026-01-15, $50} @ $70 ; ptype: dispose + assets:cash $350 + revenue:gains $-100 ; ptype: gain + + The gain of $100 was inferred: 5 shares acquired at $50, sold at $70 += 5 × ($70 - $50) = $100. + + +File: hledger.info, Node: PART 4 COMMANDS, Next: Help commands, Prev: Lot reporting, Up: Top + +25 PART 4: COMMANDS ******************* Here are hledger's standard subcommands. You can list these by running @@ -8528,7 +9049,7 @@ can list all of a command's options by running 'hledger CMD -h'.  File: hledger.info, Node: Help commands, Next: User interface commands, Prev: PART 4 COMMANDS, Up: Top -24 Help commands +26 Help commands **************** * Menu: @@ -8540,7 +9061,7 @@ File: hledger.info, Node: Help commands, Next: User interface commands, Prev:  File: hledger.info, Node: commands, Next: demo, Up: Help commands -24.1 commands +26.1 commands ============= Show the hledger commands list. @@ -8551,7 +9072,7 @@ Flags:  File: hledger.info, Node: demo, Next: help, Prev: commands, Up: Help commands -24.2 demo +26.2 demo ========= Play demos of hledger usage in the terminal, if asciinema is installed. @@ -8583,7 +9104,7 @@ $ hledger demo install -s4 # play the "install" demo at 4x speed  File: hledger.info, Node: help, Prev: demo, Up: Help commands -24.3 help +26.3 help ========= Show the hledger user manual with 'info', 'man', or a pager. With a @@ -8625,7 +9146,7 @@ $ hledger help 'time periods' -m # use man, even if info is installed  File: hledger.info, Node: User interface commands, Next: Data entry commands, Prev: Help commands, Up: Top -25 User interface commands +27 User interface commands ************************** * Menu: @@ -8638,7 +9159,7 @@ File: hledger.info, Node: User interface commands, Next: Data entry commands,  File: hledger.info, Node: repl, Next: run, Up: User interface commands -25.1 repl +27.1 repl ========= Start an interactive prompt, where you can run any of hledger's @@ -8698,7 +9219,7 @@ shell (and then 'fg' to return to the REPL).  File: hledger.info, Node: Examples, Up: repl -25.1.1 Examples +27.1.1 Examples --------------- Start the REPL and enter some commands: @@ -8729,7 +9250,7 @@ Enter hledger commands. To exit, enter 'quit' or 'exit', or send EOF.  File: hledger.info, Node: run, Next: ui, Prev: repl, Up: User interface commands -25.2 run +27.2 run ======== Run a sequence of hledger commands, provided as files or command line @@ -8793,7 +9314,7 @@ name.  File: hledger.info, Node: Examples 2, Up: run -25.2.1 Examples +27.2.1 Examples --------------- Run commands from the command line: @@ -8833,7 +9354,7 @@ List of accounts in some.journal  File: hledger.info, Node: ui, Next: web, Prev: run, Up: User interface commands -25.3 ui +27.3 ui ======= Runs hledger-ui (if installed). @@ -8841,7 +9362,7 @@ Runs hledger-ui (if installed).  File: hledger.info, Node: web, Prev: ui, Up: User interface commands -25.4 web +27.4 web ======== Runs hledger-web (if installed). @@ -8849,7 +9370,7 @@ Runs hledger-web (if installed).  File: hledger.info, Node: Data entry commands, Next: Basic report commands, Prev: User interface commands, Up: Top -26 Data entry commands +28 Data entry commands ********************** * Menu: @@ -8862,7 +9383,7 @@ File: hledger.info, Node: Data entry commands, Next: Basic report commands, P  File: hledger.info, Node: add, Next: add and balance assertions, Up: Data entry commands -26.1 add +28.1 add ======== Add new transactions to a journal file, with interactive prompting. @@ -8908,6 +9429,11 @@ or press control-d or control-c to exit. commodity symbol repeated on amounts in the journal. * 'add' creates entries in journal format; it won't work with timeclock or timedot files. + * There is a known issue on Windows if this hledger version is built + from stackage: the prompts will show ANSI junk instead of colours + (#2410). You can avoid this by using official hledger release + binaries or by building it with haskeline >=0.8.4; or by running + 'add' with '--color=no', perhaps configured in your config file. Examples: @@ -8929,7 +9455,7 @@ or press control-d or control-c to exit.  File: hledger.info, Node: add and balance assertions, Next: add and balance assignments, Prev: add, Up: Data entry commands -26.2 add and balance assertions +28.2 add and balance assertions =============================== Since hledger 1.43, you can add a balance assertion by writing 'AMOUNT = @@ -8943,7 +9469,7 @@ disable balance assertion checking.  File: hledger.info, Node: add and balance assignments, Next: import, Prev: add and balance assertions, Up: Data entry commands -26.3 add and balance assignments +28.3 add and balance assignments ================================ Since hledger 1.51, you can add a balance assignment by writing '= @@ -8959,7 +9485,7 @@ assertion checking with '-I'.  File: hledger.info, Node: import, Prev: add and balance assignments, Up: Data entry commands -26.4 import +28.4 import =========== Import new transactions from one or more data files to the main journal. @@ -8998,7 +9524,7 @@ $ hledger import *.csv  File: hledger.info, Node: Import dry run, Next: Overlap detection, Up: import -26.4.1 Import dry run +28.4.1 Import dry run --------------------- It's useful to preview the import by running first with '--dry-run', to @@ -9027,7 +9553,7 @@ $ hledger import bank.csv  File: hledger.info, Node: Overlap detection, Next: First import, Prev: Import dry run, Up: import -26.4.2 Overlap detection +28.4.2 Overlap detection ------------------------ Reading CSV files is built in to hledger, and not specific to 'import'; @@ -9093,7 +9619,7 @@ works:  File: hledger.info, Node: First import, Next: Importing balance assignments, Prev: Overlap detection, Up: import -26.4.3 First import +28.4.3 First import ------------------- The first time you import from a file, when no corresponding .latest @@ -9126,7 +9652,7 @@ newer records.  File: hledger.info, Node: Importing balance assignments, Next: Import and commodity styles, Prev: First import, Up: import -26.4.4 Importing balance assignments +28.4.4 Importing balance assignments ------------------------------------ Journal entries added by import will have all posting amounts made @@ -9152,7 +9678,7 @@ that and send a pull request.)  File: hledger.info, Node: Import and commodity styles, Next: Import archiving, Prev: Importing balance assignments, Up: import -26.4.5 Import and commodity styles +28.4.5 Import and commodity styles ---------------------------------- Amounts in entries added by import will be formatted according to the @@ -9164,7 +9690,7 @@ directives or inferred from the journal's amounts.  File: hledger.info, Node: Import archiving, Next: Import special cases, Prev: Import and commodity styles, Up: import -26.4.6 Import archiving +28.4.6 Import archiving ----------------------- When importing from a CSV rules file ('hledger import bank.rules'), you @@ -9182,7 +9708,7 @@ pick the oldest, not the newest.  File: hledger.info, Node: Import special cases, Prev: Import archiving, Up: import -26.4.7 Import special cases +28.4.7 Import special cases --------------------------- * Menu: @@ -9194,7 +9720,7 @@ File: hledger.info, Node: Import special cases, Prev: Import archiving, Up: i  File: hledger.info, Node: Deduplication, Next: Varying file name, Up: Import special cases -26.4.7.1 Deduplication +28.4.7.1 Deduplication ...................... Here are two kinds of "deduplication" which 'import' does not handle @@ -9209,7 +9735,7 @@ data):  File: hledger.info, Node: Varying file name, Next: Multiple versions, Prev: Deduplication, Up: Import special cases -26.4.7.2 Varying file name +28.4.7.2 Varying file name .......................... If you have a download whose file name varies, you could rename it to a @@ -9219,7 +9745,7 @@ with a suitable glob pattern, and import from the .rules file.  File: hledger.info, Node: Multiple versions, Prev: Varying file name, Up: Import special cases -26.4.7.3 Multiple versions +28.4.7.3 Multiple versions .......................... Say you download 'bank.csv', import it, but forget to delete it from @@ -9242,7 +9768,7 @@ import bank.rules' will import and archive first 'bank.csv', then 'bank  File: hledger.info, Node: Basic report commands, Next: Standard report commands, Prev: Data entry commands, Up: Top -27 Basic report commands +29 Basic report commands ************************ * Menu: @@ -9261,7 +9787,7 @@ File: hledger.info, Node: Basic report commands, Next: Standard report command  File: hledger.info, Node: accounts, Next: codes, Up: Basic report commands -27.1 accounts +29.1 accounts ============= List the account names used or declared in the journal. @@ -9326,7 +9852,7 @@ $ hledger check accounts  File: hledger.info, Node: codes, Next: commodities, Prev: accounts, Up: Basic report commands -27.2 codes +29.2 codes ========== List the codes seen in transactions, in the order parsed. @@ -9377,7 +9903,7 @@ $ hledger codes -E  File: hledger.info, Node: commodities, Next: descriptions, Prev: codes, Up: Basic report commands -27.3 commodities +29.3 commodities ================ List the commodity symbols used or declared in the journal. @@ -9396,12 +9922,13 @@ or declared with 'commodity' directives, or used but not declared, or declared but not used, or just the first one matched by a pattern (with '--find', returning a non-zero exit code if it fails). - You can add 'cur:' query arguments to further limit the commodities. + You can add query arguments to further limit the commodities; at +least 'cur:' and 'tag:' are supported.  File: hledger.info, Node: descriptions, Next: files, Prev: commodities, Up: Basic report commands -27.4 descriptions +29.4 descriptions ================= List the unique descriptions used in transactions. @@ -9423,7 +9950,7 @@ Person A  File: hledger.info, Node: files, Next: notes, Prev: descriptions, Up: Basic report commands -27.5 files +29.5 files ========== List all files included in the journal. With a REGEX argument, only @@ -9435,7 +9962,7 @@ no command-specific flags  File: hledger.info, Node: notes, Next: payees, Prev: files, Up: Basic report commands -27.6 notes +29.6 notes ========== List the unique notes that appear in transactions. @@ -9457,7 +9984,7 @@ Snacks  File: hledger.info, Node: payees, Next: prices, Prev: notes, Up: Basic report commands -27.7 payees +29.7 payees =========== List the payee/payer names used or declared in the journal. @@ -9492,7 +10019,7 @@ Person A  File: hledger.info, Node: prices, Next: stats, Prev: payees, Up: Basic report commands -27.8 prices +29.8 prices =========== Print the market prices declared with P directives. With @@ -9517,7 +10044,7 @@ running the value report with -debug=2.  File: hledger.info, Node: stats, Next: tags, Prev: prices, Up: Basic report commands -27.9 stats +29.9 stats ========== Show journal and performance statistics. @@ -9575,7 +10102,7 @@ $ hledger stats -1 -f examples/10ktxns-1kaccts.journal  File: hledger.info, Node: tags, Prev: stats, Up: Basic report commands -27.10 tags +29.10 tags ========== List the tag names used or declared in the journal, or their values. @@ -9619,7 +10146,7 @@ also acquire tags from their postings.  File: hledger.info, Node: Standard report commands, Next: Advanced report commands, Prev: Basic report commands, Up: Top -28 Standard report commands +30 Standard report commands *************************** * Menu: @@ -9635,7 +10162,7 @@ File: hledger.info, Node: Standard report commands, Next: Advanced report comm  File: hledger.info, Node: print, Next: aregister, Up: Standard report commands -28.1 print +30.1 print ========== Show full journal entries, representing transactions. @@ -9662,7 +10189,8 @@ Flags: with this prefix. (Usually the base url shown by hledger-web; can also be relative.) -O --output-format=FMT select the output format. Supported formats: - txt, beancount, csv, tsv, html, fods, json, sql. + txt, ledger, beancount, csv, tsv, html, fods, json, + sql. -o --output-file=FILE write output to FILE. A file extension matching one of the above formats selects that format. @@ -9702,7 +10230,7 @@ $ hledger print -f examples/sample.journal date:200806  File: hledger.info, Node: print amount explicitness, Next: print alignment, Up: print -28.1.1 print amount explicitness +30.1.1 print amount explicitness -------------------------------- Normally, whether posting amounts are implicit or explicit is preserved. @@ -9723,7 +10251,7 @@ single-commodity postings, keeping the output parseable.  File: hledger.info, Node: print alignment, Next: print amount style, Prev: print amount explicitness, Up: print -28.1.2 print alignment +30.1.2 print alignment ---------------------- Amounts are shown right-aligned within each transaction (but not aligned @@ -9733,7 +10261,7 @@ Emacs).  File: hledger.info, Node: print amount style, Next: print parseability, Prev: print alignment, Up: print -28.1.3 print amount style +30.1.3 print amount style ------------------------- Amounts will be displayed mostly in their commodity's display style, @@ -9761,7 +10289,7 @@ entries, perhaps requiring manual fixup.  File: hledger.info, Node: print parseability, Next: print other features, Prev: print amount style, Up: print -28.1.4 print parseability +30.1.4 print parseability ------------------------- Normally, print's output is a valid hledger journal, which you can @@ -9791,7 +10319,7 @@ unparseable:  File: hledger.info, Node: print other features, Next: print output format, Prev: print parseability, Up: print -28.1.5 print, other features +30.1.5 print, other features ---------------------------- With '-B'/'--cost', amounts with costs are shown converted to cost. @@ -9815,12 +10343,16 @@ every transaction, as a tag.  File: hledger.info, Node: print output format, Prev: print other features, Up: print -28.1.6 print output format +30.1.6 print output format -------------------------- This command also supports the output destination and output format -options The output formats supported are 'txt', 'beancount' (_Added in -1.32_), 'csv', 'tsv' (_Added in 1.32_), 'json' and 'sql'. +options The output formats supported are 'txt', 'ledger', 'beancount' +(_Added in 1.32_), 'csv', 'tsv' (_Added in 1.32_), 'json' and 'sql'. + + The 'ledger' format is currently the same as 'txt' except it renders +amounts' cost basis using Ledger's lot syntax ('[DATE] (LABEL) {COST}') +instead of hledger's ('{DATE, "LABEL", COST}'). The 'beancount' format tries to produce Beancount-compatible output, as follows: @@ -9880,7 +10412,7 @@ $ hledger print -Ocsv  File: hledger.info, Node: aregister, Next: register, Prev: print, Up: Standard report commands -28.2 aregister +30.2 aregister ============== (areg) @@ -9981,7 +10513,7 @@ in 1.32_), 'html', 'fods' (_Added in 1.41_) and 'json'.  File: hledger.info, Node: aregister and posting dates, Up: aregister -28.2.1 aregister and posting dates +30.2.1 aregister and posting dates ---------------------------------- aregister always shows one line (and date and amount) per transaction. @@ -10001,7 +10533,7 @@ inaccurate running balance.  File: hledger.info, Node: register, Next: balancesheet, Prev: aregister, Up: Standard report commands -28.3 register +30.3 register ============= (reg) @@ -10020,6 +10552,7 @@ Flags: description closest to DESC -r --related show postings' siblings instead --invert display all amounts with reversed sign + --drop=N omit N leading account name parts --sort=FIELDS sort by: date, desc, account, amount, absamount, or a comma-separated combination of these. For a descending sort, prefix with -. (Default: date) @@ -10072,6 +10605,8 @@ $ hledger register checking -b 2008/6 --historical The '--depth' option limits the amount of sub-account detail displayed. + The '--drop' option will trim leading segments from account names. + The '--average'/'-A' flag shows the running average posting amount instead of the running total (so, the final number displayed is the average for the whole report period). This flag implies '--empty' (see @@ -10133,6 +10668,13 @@ these will be adjusted outward if necessary to contain a whole number of intervals. This ensures that the first and last intervals are full length and comparable to the others in the report. + If you have a deeply nested account tree some reports might benefit +from trimming leading segments from the account names using '--drop'. + +$ hledger register --monthly income --drop 1 +2008/01 salary $-1 $-1 +2008/06 gifts $-1 $-2 + With '-m DESC'/'--match=DESC', register does a fuzzy search for one recent posting whose description is most similar to DESC. DESC should contain at least two characters. If there is no similar-enough match, @@ -10145,7 +10687,7 @@ no posting will be shown and the program exit code will be non-zero.  File: hledger.info, Node: Custom register output, Up: register -28.3.1 Custom register output +30.3.1 Custom register output ----------------------------- register normally uses the full terminal width (or 80 columns if it @@ -10174,7 +10716,7 @@ options The output formats supported are 'txt', 'csv', 'tsv' (_Added in  File: hledger.info, Node: balancesheet, Next: balancesheetequity, Prev: register, Up: Standard report commands -28.4 balancesheet +30.4 balancesheet ================= (bs) @@ -10275,7 +10817,7 @@ options The output formats supported are 'txt', 'csv', 'tsv' (_Added in  File: hledger.info, Node: balancesheetequity, Next: cashflow, Prev: balancesheet, Up: Standard report commands -28.5 balancesheetequity +30.5 balancesheetequity ======================= (bse) @@ -10383,7 +10925,7 @@ and 'json'.  File: hledger.info, Node: cashflow, Next: incomestatement, Prev: balancesheetequity, Up: Standard report commands -28.6 cashflow +30.6 cashflow ============= (cf) @@ -10483,7 +11025,7 @@ options The output formats supported are 'txt', 'csv', 'tsv' (_Added in  File: hledger.info, Node: incomestatement, Prev: cashflow, Up: Standard report commands -28.7 incomestatement +30.7 incomestatement ==================== (is) @@ -10585,7 +11127,7 @@ options The output formats supported are 'txt', 'csv', 'tsv' (_Added in  File: hledger.info, Node: Advanced report commands, Next: Chart commands, Prev: Standard report commands, Up: Top -29 Advanced report commands +31 Advanced report commands *************************** * Menu: @@ -10596,7 +11138,7 @@ File: hledger.info, Node: Advanced report commands, Next: Chart commands, Pre  File: hledger.info, Node: balance, Next: roi, Up: Advanced report commands -29.1 balance +31.1 balance ============ (bal) @@ -10701,7 +11243,7 @@ more control, then use 'balance'.  File: hledger.info, Node: balance features, Next: Simple balance report, Up: balance -29.1.1 balance features +31.1.1 balance features ----------------------- Here's a quick overview of the 'balance' command's features, followed by @@ -10763,7 +11305,7 @@ amounts are shown in red.  File: hledger.info, Node: Simple balance report, Next: Balance report line format, Prev: balance features, Up: balance -29.1.2 Simple balance report +31.1.2 Simple balance report ---------------------------- With no arguments, 'balance' shows a list of all accounts and their @@ -10812,7 +11354,7 @@ $ hledger -f examples/sample.journal bal -E  File: hledger.info, Node: Balance report line format, Next: Filtered balance report, Prev: Simple balance report, Up: balance -29.1.3 Balance report line format +31.1.3 Balance report line format --------------------------------- For single-period balance reports displayed in the terminal (only), you @@ -10875,7 +11417,7 @@ may be needed to get pleasing results.  File: hledger.info, Node: Filtered balance report, Next: List or tree mode, Prev: Balance report line format, Up: balance -29.1.4 Filtered balance report +31.1.4 Filtered balance report ------------------------------ You can show fewer accounts, a different time period, totals from @@ -10890,7 +11432,7 @@ $ hledger -f examples/sample.journal bal --cleared assets date:200806  File: hledger.info, Node: List or tree mode, Next: Depth limiting, Prev: Filtered balance report, Up: balance -29.1.5 List or tree mode +31.1.5 List or tree mode ------------------------ By default, or with '-l/--flat', accounts are shown as a flat list with @@ -10933,7 +11475,7 @@ $ hledger -f examples/sample.journal balance  File: hledger.info, Node: Depth limiting, Next: Dropping top-level accounts, Prev: List or tree mode, Up: balance -29.1.6 Depth limiting +31.1.6 Depth limiting --------------------- With a 'depth:NUM' query, or '--depth NUM' option, or just '-NUM' (eg: @@ -10955,7 +11497,7 @@ $ hledger -f examples/sample.journal balance -1  File: hledger.info, Node: Dropping top-level accounts, Next: Showing declared accounts, Prev: Depth limiting, Up: balance -29.1.7 Dropping top-level accounts +31.1.7 Dropping top-level accounts ---------------------------------- You can also hide one or more top-level account name parts, using @@ -10971,7 +11513,7 @@ $ hledger -f examples/sample.journal bal expenses --drop 1  File: hledger.info, Node: Showing declared accounts, Next: Sorting by amount, Prev: Dropping top-level accounts, Up: balance -29.1.8 Showing declared accounts +31.1.8 Showing declared accounts -------------------------------- With '--declared', accounts which have been declared with an account @@ -10989,7 +11531,7 @@ accounts yet.  File: hledger.info, Node: Sorting by amount, Next: Percentages, Prev: Showing declared accounts, Up: balance -29.1.9 Sorting by amount +31.1.9 Sorting by amount ------------------------ With '-S/--sort-amount', accounts with the largest (most positive) @@ -11008,7 +11550,7 @@ balance reports ('bs', 'is'..), which flip the sign automatically (eg:  File: hledger.info, Node: Percentages, Next: Multi-period balance report, Prev: Sorting by amount, Up: balance -29.1.10 Percentages +31.1.10 Percentages ------------------- With '-%/--percent', balance reports show each account's value expressed @@ -11031,7 +11573,7 @@ $ hledger bal -% cur:€  File: hledger.info, Node: Multi-period balance report, Next: Balance change end balance, Prev: Percentages, Up: balance -29.1.11 Multi-period balance report +31.1.11 Multi-period balance report ----------------------------------- With a report interval (set by the '-D/--daily', '-W/--weekly', @@ -11091,7 +11633,7 @@ viewing in the terminal. Here are some ways to handle that:  File: hledger.info, Node: Balance change end balance, Next: Balance report modes, Prev: Multi-period balance report, Up: balance -29.1.12 Balance change, end balance +31.1.12 Balance change, end balance ----------------------------------- It's important to be clear on the meaning of the numbers shown in @@ -11128,7 +11670,7 @@ historical end balances:  File: hledger.info, Node: Balance report modes, Next: Budget report, Prev: Balance change end balance, Up: balance -29.1.13 Balance report modes +31.1.13 Balance report modes ---------------------------- The balance command is quite flexible; here is the full detail on how to @@ -11151,7 +11693,7 @@ experimentation to get familiar with all the report modes.  File: hledger.info, Node: Calculation mode, Next: Accumulation mode, Up: Balance report modes -29.1.13.1 Calculation mode +31.1.13.1 Calculation mode .......................... The basic calculation to perform for each table cell. It is one of: @@ -11169,7 +11711,7 @@ The basic calculation to perform for each table cell. It is one of:  File: hledger.info, Node: Accumulation mode, Next: Valuation mode, Prev: Calculation mode, Up: Balance report modes -29.1.13.2 Accumulation mode +31.1.13.2 Accumulation mode ........................... How amounts should accumulate across a report's subperiods/columns. @@ -11195,7 +11737,7 @@ each cell's calculation. It is one of:  File: hledger.info, Node: Valuation mode, Next: Combining balance report modes, Prev: Accumulation mode, Up: Balance report modes -29.1.13.3 Valuation mode +31.1.13.3 Valuation mode ........................ Which kind of value or cost conversion should be applied, if any, before @@ -11231,7 +11773,7 @@ together.  File: hledger.info, Node: Combining balance report modes, Prev: Valuation mode, Up: Balance report modes -29.1.13.4 Combining balance report modes +31.1.13.4 Combining balance report modes ........................................ Most combinations of these modes should produce reasonable reports, but @@ -11270,7 +11812,7 @@ Accumulation:v YYYY-MM-DD  File: hledger.info, Node: Budget report, Next: Balance report layout, Prev: Balance report modes, Up: balance -29.1.14 Budget report +31.1.14 Budget report --------------------- The '--budget' report is like a regular balance report, but with two @@ -11341,7 +11883,7 @@ https://plaintextaccounting.org/Budgeting has more on this topic.  File: hledger.info, Node: Using the budget report, Next: Budget date surprises, Up: Budget report -29.1.14.1 Using the budget report +31.1.14.1 Using the budget report ................................. Historically this report has been confusing and fragile. hledger's @@ -11397,7 +11939,7 @@ troubleshooting.  File: hledger.info, Node: Budget date surprises, Next: Selecting budget goals, Prev: Using the budget report, Up: Budget report -29.1.14.2 Budget date surprises +31.1.14.2 Budget date surprises ............................... With small data, or when starting out, some of the generated budget goal @@ -11436,7 +11978,7 @@ the trick.  File: hledger.info, Node: Selecting budget goals, Next: Budgeting vs forecasting, Prev: Budget date surprises, Up: Budget report -29.1.14.3 Selecting budget goals +31.1.14.3 Selecting budget goals ................................ By default, the budget report uses all available periodic transaction @@ -11456,7 +11998,7 @@ defined in your journal.  File: hledger.info, Node: Budgeting vs forecasting, Prev: Selecting budget goals, Up: Budget report -29.1.14.4 Budgeting vs forecasting +31.1.14.4 Budgeting vs forecasting .................................. '--forecast' and '--budget' both use the periodic transaction rules in @@ -11488,7 +12030,7 @@ uses all periodic rules uses all periodic rules; or  File: hledger.info, Node: Balance report layout, Next: Balance report output, Prev: Budget report, Up: balance -29.1.15 Balance report layout +31.1.15 Balance report layout ----------------------------- The '--layout' option affects how 'balance' and the other balance-like @@ -11527,7 +12069,7 @@ tidy Y  File: hledger.info, Node: Wide layout, Next: Tall layout, Up: Balance report layout -29.1.15.1 Wide layout +31.1.15.1 Wide layout ..................... With many commodities, reports can be very wide: @@ -11555,7 +12097,7 @@ Balance changes in 2012-01-01..2014-12-31:  File: hledger.info, Node: Tall layout, Next: Bare layout, Prev: Wide layout, Up: Balance report layout -29.1.15.2 Tall layout +31.1.15.2 Tall layout ..................... Each commodity gets a new line (may be different in each column), and @@ -11581,7 +12123,7 @@ Balance changes in 2012-01-01..2014-12-31:  File: hledger.info, Node: Bare layout, Next: Tidy layout, Prev: Tall layout, Up: Balance report layout -29.1.15.3 Bare layout +31.1.15.3 Bare layout ..................... Commodity symbols are kept in one column, each commodity has its own @@ -11628,7 +12170,7 @@ commodity-less, usually). This can break 'hledger-bar' confusingly  File: hledger.info, Node: Tidy layout, Prev: Bare layout, Up: Balance report layout -29.1.15.4 Tidy layout +31.1.15.4 Tidy layout ..................... This produces normalised "tidy data" (see @@ -11658,7 +12200,7 @@ $ hledger -f examples/bcexample.hledger bal assets:us:etrade -3 -Y -O csv --layo  File: hledger.info, Node: Balance report output, Next: Some useful balance reports, Prev: Balance report layout, Up: balance -29.1.16 Balance report output +31.1.16 Balance report output ----------------------------- As noted in Output format, if you choose HTML output (by using '-O html' @@ -11675,7 +12217,7 @@ links, like '--base-url="some/path"' or '--base-url=""'.)  File: hledger.info, Node: Some useful balance reports, Prev: Balance report output, Up: balance -29.1.17 Some useful balance reports +31.1.17 Some useful balance reports ----------------------------------- Some frequently used 'balance' options/reports are: @@ -11715,7 +12257,7 @@ Some frequently used 'balance' options/reports are:  File: hledger.info, Node: roi, Prev: balance, Up: Advanced report commands -29.2 roi +31.2 roi ======== Shows the time-weighted (TWR) and money-weighted (IRR) rate of return on @@ -11772,7 +12314,7 @@ annual rate.  File: hledger.info, Node: Spaces and special characters in --inv and --pnl, Next: Semantics of --inv and --pnl, Up: roi -29.2.1 Spaces and special characters in '--inv' and +31.2.1 Spaces and special characters in '--inv' and --------------------------------------------------- '--pnl' Note that '--inv' and '--pnl''s argument is a query, and queries @@ -11791,7 +12333,7 @@ $ hledger roi --inv="'Assets:Test 1'" --pnl="'Equity:Unrealized Profit and Loss'  File: hledger.info, Node: Semantics of --inv and --pnl, Next: IRR and TWR explained, Prev: Spaces and special characters in --inv and --pnl, Up: roi -29.2.2 Semantics of '--inv' and '--pnl' +31.2.2 Semantics of '--inv' and '--pnl' --------------------------------------- Query supplied to '--inv' has to match all transactions that are related @@ -11845,7 +12387,7 @@ postings in the example below would be classifed as:  File: hledger.info, Node: IRR and TWR explained, Prev: Semantics of --inv and --pnl, Up: roi -29.2.3 IRR and TWR explained +31.2.3 IRR and TWR explained ---------------------------- "ROI" stands for "return on investment". Traditionally this was @@ -11914,7 +12456,7 @@ cash in-flows and out-flows.  File: hledger.info, Node: Chart commands, Next: Data generation commands, Prev: Advanced report commands, Up: Top -30 Chart commands +32 Chart commands ***************** * Menu: @@ -11924,7 +12466,7 @@ File: hledger.info, Node: Chart commands, Next: Data generation commands, Pre  File: hledger.info, Node: activity, Up: Chart commands -30.1 activity +32.1 activity ============= Show an ascii barchart of posting counts per interval. @@ -11947,7 +12489,7 @@ $ hledger activity --quarterly  File: hledger.info, Node: Data generation commands, Next: Maintenance commands, Prev: Chart commands, Up: Top -31 Data generation commands +33 Data generation commands *************************** * Menu: @@ -11958,7 +12500,7 @@ File: hledger.info, Node: Data generation commands, Next: Maintenance commands  File: hledger.info, Node: close, Next: rewrite, Up: Data generation commands -31.1 close +33.1 close ========== (equity) @@ -12026,7 +12568,7 @@ value by providing an argument to the mode flag. Eg '--close=foo' or  File: hledger.info, Node: close --clopen, Next: close --close, Up: close -31.1.1 close -clopen +33.1.1 close -clopen -------------------- This is useful if migrating balances to a new journal file at the start @@ -12067,7 +12609,7 @@ the closed/opened accounts.  File: hledger.info, Node: close --close, Next: close --open, Prev: close --clopen, Up: close -31.1.2 close -close +33.1.2 close -close ------------------- This prints just the closing balances transaction of '--clopen'. It is @@ -12081,7 +12623,7 @@ to move just a portion of the balance, see hledger-move.)  File: hledger.info, Node: close --open, Next: close --assert, Prev: close --close, Up: close -31.1.3 close -open +33.1.3 close -open ------------------ This prints just the opening balances transaction of '--clopen'. (It is @@ -12090,7 +12632,7 @@ similar to Ledger's equity command.)  File: hledger.info, Node: close --assert, Next: close --assign, Prev: close --open, Up: close -31.1.4 close -assert +33.1.4 close -assert -------------------- This prints a transaction that asserts the account balances as they are @@ -12100,7 +12642,7 @@ documention and to guard against changes.  File: hledger.info, Node: close --assign, Next: close --retain, Prev: close --assert, Up: close -31.1.5 close -assign +33.1.5 close -assign -------------------- This prints a transaction that assigns the account balances as they are @@ -12117,7 +12659,7 @@ usually recommended.  File: hledger.info, Node: close --retain, Next: close customisation, Prev: close --assign, Up: close -31.1.6 close -retain +33.1.6 close -retain -------------------- This is like '--close', but it closes Revenue and Expense account @@ -12137,7 +12679,7 @@ report show a zero total, demonstrating that the accounting equation  File: hledger.info, Node: close customisation, Next: close and balance assertions, Prev: close --retain, Up: close -31.1.7 close customisation +33.1.7 close customisation -------------------------- In all modes, the following things can be overridden: @@ -12175,7 +12717,7 @@ hledger. (To move or dispose of lots, see the more capable  File: hledger.info, Node: close and balance assertions, Next: close examples, Prev: close customisation, Up: close -31.1.8 close and balance assertions +33.1.8 close and balance assertions ----------------------------------- 'close' adds balance assertions verifying that the accounts have been @@ -12184,6 +12726,10 @@ balances in an opening transaction. These provide useful error checking, but you can ignore them temporarily with '-I', or remove them if you prefer. + With '--lots', balance assertions are not generated for the lot +subaccount postings in closing transactions (assertions on lot postings +get confusing, because they apply to the parent account). + Single-commodity, subaccount-exclusive balance assertions ('=') are generated by default. This can be changed with '--assertion-type='==*'' (eg). @@ -12217,7 +12763,7 @@ transactions:  File: hledger.info, Node: close examples, Prev: close and balance assertions, Up: close -31.1.9 close examples +33.1.9 close examples --------------------- * Menu: @@ -12229,7 +12775,7 @@ File: hledger.info, Node: close examples, Prev: close and balance assertions,  File: hledger.info, Node: Retain earnings, Next: Migrate balances to a new file, Up: close examples -31.1.9.1 Retain earnings +33.1.9.1 Retain earnings ........................ Record 2022's revenues/expenses as retained earnings on 2022-12-31, @@ -12245,7 +12791,7 @@ $ hledger -f 2022.journal is not:desc:'retain earnings'  File: hledger.info, Node: Migrate balances to a new file, Next: More detailed close examples, Prev: Retain earnings, Up: close examples -31.1.9.2 Migrate balances to a new file +33.1.9.2 Migrate balances to a new file ....................................... Close assets/liabilities on 2022-12-31 and re-open them on 2023-01-01: @@ -12274,7 +12820,7 @@ $ hledger bs -Y -f 2023.j # unclosed file, no query needed  File: hledger.info, Node: More detailed close examples, Prev: Migrate balances to a new file, Up: close examples -31.1.9.3 More detailed close examples +33.1.9.3 More detailed close examples ..................................... See examples/multi-year. @@ -12282,7 +12828,7 @@ See examples/multi-year.  File: hledger.info, Node: rewrite, Prev: close, Up: Data generation commands -31.2 rewrite +33.2 rewrite ============ Print all transactions, rewriting the postings of matched transactions. @@ -12345,7 +12891,7 @@ commodity.  File: hledger.info, Node: Re-write rules in a file, Next: Diff output format, Up: rewrite -31.2.1 Re-write rules in a file +33.2.1 Re-write rules in a file ------------------------------- During the run this tool will execute so called "Automated Transactions" @@ -12383,7 +12929,7 @@ postings.  File: hledger.info, Node: Diff output format, Next: rewrite vs print --auto, Prev: Re-write rules in a file, Up: rewrite -31.2.2 Diff output format +33.2.2 Diff output format ------------------------- To use this tool for batch modification of your journal files you may @@ -12424,7 +12970,7 @@ output from 'hledger print'.  File: hledger.info, Node: rewrite vs print --auto, Prev: Diff output format, Up: rewrite -31.2.3 rewrite vs. print -auto +33.2.3 rewrite vs. print -auto ------------------------------ This command predates print -auto, and currently does much the same @@ -12444,7 +12990,7 @@ thing, but with these differences:  File: hledger.info, Node: Maintenance commands, Next: PART 5 COMMON TASKS, Prev: Data generation commands, Up: Top -32 Maintenance commands +34 Maintenance commands *********************** * Menu: @@ -12457,7 +13003,7 @@ File: hledger.info, Node: Maintenance commands, Next: PART 5 COMMON TASKS, Pr  File: hledger.info, Node: check, Next: diff, Up: Maintenance commands -32.1 check +34.1 check ========== Check for various kinds of errors in your data. @@ -12492,7 +13038,7 @@ reported.  File: hledger.info, Node: Basic checks, Next: Strict checks, Up: check -32.1.1 Basic checks +34.1.1 Basic checks ------------------- These important checks are performed by default, by almost all hledger @@ -12518,7 +13064,7 @@ commands:  File: hledger.info, Node: Strict checks, Next: Other checks, Prev: Basic checks, Up: check -32.1.2 Strict checks +34.1.2 Strict checks -------------------- When the '-s'/'--strict' flag is used (AKA strict mode), all commands @@ -12535,12 +13081,14 @@ clean and correct: guards against mistyping or omitting commodity symbols. * *accounts* - all account names used must be declared. This - prevents the use of mis-spelled or outdated account names. + prevents the use of mis-spelled or outdated account names. (Except + lot subaccounts, like ':{2026-01-15, $50}', which are automatically + exempt; only their base account needs to be declared.)  File: hledger.info, Node: Other checks, Next: Custom checks, Prev: Strict checks, Up: check -32.1.3 Other checks +34.1.3 Other checks ------------------- These are not wanted by everyone, but can be run using the 'check' @@ -12578,7 +13126,7 @@ command:  File: hledger.info, Node: Custom checks, Prev: Other checks, Up: check -32.1.4 Custom checks +34.1.4 Custom checks -------------------- You can build your own custom checks with add-on command scripts. See @@ -12593,7 +13141,7 @@ also Cookbook > Scripting. Here are some examples from hledger/bin/:  File: hledger.info, Node: diff, Next: setup, Prev: check, Up: Maintenance commands -32.2 diff +34.2 diff ========= Compares a particular account's transactions in two input files. It @@ -12632,7 +13180,7 @@ These transactions are in the second file only:  File: hledger.info, Node: setup, Next: test, Prev: diff, Up: Maintenance commands -32.3 setup +34.3 setup ========== Check the status of the hledger installation. @@ -12698,7 +13246,7 @@ journal  File: hledger.info, Node: test, Prev: setup, Up: Maintenance commands -32.4 test +34.4 test ========= Run built-in unit tests. @@ -12728,7 +13276,7 @@ $ hledger test -- -h # show tasty's options  File: hledger.info, Node: PART 5 COMMON TASKS, Next: BUGS, Prev: Maintenance commands, Up: Top -33 PART 5: COMMON TASKS +35 PART 5: COMMON TASKS *********************** Here are some quick examples of how to do some basic tasks with hledger. @@ -12748,7 +13296,7 @@ Here are some quick examples of how to do some basic tasks with hledger.  File: hledger.info, Node: Getting help, Next: Constructing command lines, Up: PART 5 COMMON TASKS -33.1 Getting help +35.1 Getting help ================= Here's how to list commands and view options and command docs: @@ -12771,7 +13319,7 @@ can be found at https://hledger.org/support.  File: hledger.info, Node: Constructing command lines, Next: Starting a journal file, Prev: Getting help, Up: PART 5 COMMON TASKS -33.2 Constructing command lines +35.2 Constructing command lines =============================== hledger has a flexible command line interface. We strive to keep it @@ -12791,7 +13339,7 @@ described in OPTIONS, here are some tips that might help:  File: hledger.info, Node: Starting a journal file, Next: Setting LEDGER_FILE, Prev: Constructing command lines, Up: PART 5 COMMON TASKS -33.3 Starting a journal file +35.3 Starting a journal file ============================ hledger looks for your accounting data in a journal file, @@ -12830,7 +13378,7 @@ Market prices : 0 ()  File: hledger.info, Node: Setting LEDGER_FILE, Next: Setting opening balances, Prev: Starting a journal file, Up: PART 5 COMMON TASKS -33.4 Setting LEDGER_FILE +35.4 Setting LEDGER_FILE ======================== * Menu: @@ -12842,13 +13390,13 @@ File: hledger.info, Node: Setting LEDGER_FILE, Next: Setting opening balances,  File: hledger.info, Node: Set LEDGER_FILE on unix, Next: Set LEDGER_FILE on mac, Up: Setting LEDGER_FILE -33.4.1 Set LEDGER_FILE on unix +35.4.1 Set LEDGER_FILE on unix ------------------------------ It depends on your shell, but running these commands in the terminal will work for many people; adapt if needed: -$ echo 'export LEDGER_FILE=~/finance/my.journal' >> ~/.profile +$ echo 'export LEDGER_FILE=~/finance/main.journal' >> ~/.profile $ source ~/.profile When correctly configured: @@ -12859,7 +13407,7 @@ $ source ~/.profile  File: hledger.info, Node: Set LEDGER_FILE on mac, Next: Set LEDGER_FILE on Windows, Prev: Set LEDGER_FILE on unix, Up: Setting LEDGER_FILE -33.4.2 Set LEDGER_FILE on mac +35.4.2 Set LEDGER_FILE on mac ----------------------------- In a terminal window, follow the unix procedure above. @@ -12869,7 +13417,7 @@ In a terminal window, follow the unix procedure above. 1. Add an entry to '~/.MacOSX/environment.plist' like { - "LEDGER_FILE" : "~/finance/my.journal" + "LEDGER_FILE" : "~/finance/main.journal" } 2. Run 'killall Dock' in a terminal window (or restart the machine), @@ -12883,16 +13431,20 @@ In a terminal window, follow the unix procedure above.  File: hledger.info, Node: Set LEDGER_FILE on Windows, Prev: Set LEDGER_FILE on mac, Up: Setting LEDGER_FILE -33.4.3 Set LEDGER_FILE on Windows +35.4.3 Set LEDGER_FILE on Windows --------------------------------- -Using the gui is easiest: +It can be easier to create a default file at +'C:\Users\USER\.hledger.journal', and have it include your other files. +See I'm on Windows, how do I keep my files in AppData + + Otherwise: using the gui is easiest: 1. In task bar, search for 'environment variables', and choose "Edit environment variables for your account". 2. Create or change a 'LEDGER_FILE' setting in the User variables pane. A typical value would be - 'C:\Users\USERNAME\finance\my.journal'. + 'C:\Users\USER\finance\main.journal'. 3. Click OK to complete the change. 4. And open a new powershell window. (Existing windows won't see the change.) @@ -12901,7 +13453,7 @@ Using the gui is easiest: 1. In a powershell window, run '[Environment]::SetEnvironmentVariable("LEDGER_FILE", - "C:\User\USERNAME\finance\my.journal", + "C:\User\USER\finance\main.journal", [System.EnvironmentVariableTarget]::User)' 2. And open a new powershell window. (Existing windows won't see the change.) @@ -12930,7 +13482,7 @@ other methods you may find online:  File: hledger.info, Node: Setting opening balances, Next: Recording transactions, Prev: Setting LEDGER_FILE, Up: PART 5 COMMON TASKS -33.5 Setting opening balances +35.5 Setting opening balances ============================= Pick a starting date for which you can look up the balances of some @@ -13013,7 +13565,7 @@ $ git commit -m 'initial balances' 2023.journal  File: hledger.info, Node: Recording transactions, Next: Reconciling, Prev: Setting opening balances, Up: PART 5 COMMON TASKS -33.6 Recording transactions +35.6 Recording transactions =========================== As you spend or receive money, you can record these transactions using @@ -13039,7 +13591,7 @@ and hledger.org for more ideas:  File: hledger.info, Node: Reconciling, Next: Reporting, Prev: Recording transactions, Up: PART 5 COMMON TASKS -33.7 Reconciling +35.7 Reconciling ================ Periodically you should reconcile - compare your hledger-reported @@ -13094,7 +13646,7 @@ $ git commit -m 'txns' 2023.journal  File: hledger.info, Node: Reporting, Next: Migrating to a new file, Prev: Reconciling, Up: PART 5 COMMON TASKS -33.8 Reporting +35.8 Reporting ============== Here are some basic reports. @@ -13242,7 +13794,7 @@ $ hledger activity -W  File: hledger.info, Node: Migrating to a new file, Prev: Reporting, Up: PART 5 COMMON TASKS -33.9 Migrating to a new file +35.9 Migrating to a new file ============================ At the end of the year, you may want to continue your journal in a new @@ -13255,7 +13807,7 @@ close command.  File: hledger.info, Node: BUGS, Prev: PART 5 COMMON TASKS, Up: Top -34 BUGS +36 BUGS ******* We welcome bug reports in the hledger issue tracker @@ -13284,7 +13836,7 @@ Ledger.  File: hledger.info, Node: Troubleshooting, Up: BUGS -34.1 Troubleshooting +36.1 Troubleshooting ==================== Here are some common issues you might encounter when you run hledger, @@ -13322,405 +13874,423 @@ See hledger and Ledger for full details.  Tag Table: Node: Top208 -Node: PART 1 USER INTERFACE4093 -Node: Input4232 -Node: Text encoding5324 -Node: Data formats6073 -Node: Standard input7807 -Node: Multiple files8196 -Node: Strict mode8933 -Node: Commands9767 -Node: Add-on commands11049 -Node: Options12100 -Node: Special characters19250 -Node: Escaping shell special characters20240 -Node: Escaping regular expression special characters21599 -Node: Escaping in other situations23114 -Node: Unicode characters24262 -Node: Regular expressions25683 -Node: hledger's regular expressions28942 -Node: Argument files30583 -Node: Config files31480 -Node: Shell completions34749 -Node: Output35238 -Node: Output destination35429 -Node: Output format35987 -Node: Text output37773 -Node: Box-drawing characters38757 -Node: Colour39257 -Node: Paging39843 -Node: HTML output41362 -Node: CSV / TSV output41780 -Node: FODS output42034 -Node: Beancount output42838 -Node: Beancount account names44339 -Node: Beancount commodity names44880 -Node: Beancount virtual postings45527 -Node: Beancount metadata45843 -Node: Beancount costs46623 -Node: Beancount operating currency47039 -Node: SQL output47489 -Node: JSON output48280 -Node: Commodity styles49097 -Node: Debug output50107 -Node: Environment50939 -Node: PART 2 DATA FORMATS52108 -Node: Journal52251 -Node: Journal cheatsheet54716 -Node: Comments60967 -Node: Transactions61911 -Node: Dates63048 -Node: Simple dates63200 -Node: Posting dates63816 -Node: Status64989 -Node: Code66755 -Node: Description67090 -Node: Payee and note67777 -Node: Transaction comments68868 -Node: Postings69384 -Node: Debits and credits70700 -Node: Account names71259 -Node: Two space delimiter72216 -Node: Account hierarchy73621 -Node: Other account name features74504 -Node: Amounts74922 -Node: Decimal marks75951 -Node: Digit group marks77055 -Node: Commodity77690 -Node: Costs78807 -Node: Balance assertions81059 -Node: Assertions and ordering82307 -Node: Assertions and multiple files83026 -Node: Assertions and costs84194 -Node: Assertions and commodities84841 -Node: Assertions and subaccounts86500 -Node: Assertions and status87160 -Node: Assertions and virtual postings87580 -Node: Assertions and auto postings87945 -Node: Assertions and precision88820 -Node: Assertions and hledger add89304 -Node: Posting comments90052 -Node: Transaction balancing90592 -Node: Tags92800 -Node: Tag propagation94319 -Node: Displaying tags95818 -Node: When to use tags ?96211 -Node: Tag names96875 -Node: Directives98856 -Node: Directives and multiple files100313 -Node: Directive effects101258 -Node: account directive104414 -Node: Account comments105864 -Node: Account error checking106523 -Node: Account display order108060 -Node: Account types109258 -Node: alias directive112533 -Node: Basic aliases113744 -Node: Regex aliases114619 -Node: Combining aliases115666 -Node: Aliases and multiple files117120 -Node: end aliases directive117903 -Node: Aliases can generate bad account names118271 -Node: Aliases and account types119104 -Node: commodity directive119996 -Node: Commodity directive syntax121789 -Node: Commodity error checking123438 -Node: decimal-mark directive123913 -Node: include directive124492 -Node: P directive126715 -Node: payee directive127749 -Node: tag directive128371 -Node: Periodic transactions128983 -Node: Periodic rule syntax131137 -Node: Periodic rules and relative dates131960 -Node: Two spaces between period expression and description!132737 -Node: Auto postings133698 -Node: Auto postings and multiple files136984 -Node: Auto postings and dates137389 -Node: Auto postings and transaction balancing / inferred amounts / balance assertions137830 -Node: Auto posting tags138676 -Node: Auto postings on forecast transactions only139571 -Node: Other syntax140041 -Node: Balance assignments140813 -Node: Balance assignments and costs142341 -Node: Balance assignments and multiple files142763 -Node: Bracketed posting dates143186 -Node: D directive143884 -Node: apply account directive145657 -Node: Y directive146524 -Node: Secondary dates147512 -Node: Star comments148997 -Node: Valuation expressions149689 -Node: Virtual postings149988 -Node: Other Ledger directives151612 -Node: Other cost/lot notations152374 -Node: CSV155215 -Node: CSV rules cheatsheet157415 -Node: source159713 -Node: Data cleaning / data generating commands161117 -Node: archive163016 -Node: encoding163944 -Node: separator164987 -Node: skip165640 -Node: date-format166290 -Node: timezone167235 -Node: newest-first168361 -Node: intra-day-reversed169074 -Node: decimal-mark169676 -Node: CSV fields and hledger fields170174 -Node: fields list172050 -Node: Field assignment173875 -Node: Field names175094 -Node: date field176426 -Node: date2 field176590 -Node: status field176785 -Node: code field176975 -Node: description field177163 -Node: comment field177380 -Node: account field177937 -Node: amount field178655 -Node: currency field181494 -Node: balance field181902 -Node: if block182425 -Node: Matchers183952 -Node: Multiple matchers185880 -Node: Match groups186688 -Node: if table187581 -Node: balance-type189644 -Node: include190471 -Node: Working with CSV191040 -Node: Rapid feedback191629 -Node: Valid CSV192212 -Node: File Extension193088 -Node: Reading CSV from standard input193823 -Node: Reading multiple CSV files194209 -Node: Reading files specified by rule194685 -Node: Valid transactions196082 -Node: Deduplicating importing196907 -Node: Regular expressions in CSV rules198153 -Node: Setting amounts199645 -Node: Amount signs202183 -Node: Setting currency/commodity203248 -Node: Amount decimal places204624 -Node: Referencing other fields205881 -Node: How CSV rules are evaluated206989 -Node: Well factored rules209706 -Node: CSV rules examples210196 -Node: Bank of Ireland210394 -Node: Coinbase211991 -Node: Amazon213174 -Node: Paypal215016 -Node: Timeclock222766 -Node: Timedot226819 -Node: Timedot examples230296 -Node: PART 3 REPORTING CONCEPTS232573 -Node: Time periods232737 -Node: Report start & end date233010 -Node: Smart dates234486 -Node: Report intervals236609 -Node: Date adjustments237183 -Node: Start date adjustment237403 -Node: End date adjustment238306 -Node: Period headings239087 -Node: Period expressions240020 -Node: Period expressions with a report interval241925 -Node: More complex report intervals242373 -Node: Multiple weekday intervals244489 -Node: Depth245500 -Node: Combining depth options246486 -Node: Queries247436 -Node: Query types250108 -Node: acct query250483 -Node: amt query250794 -Node: code query251491 -Node: cur query251686 -Node: desc query252292 -Node: date query252475 -Node: date2 query252871 -Node: depth query253162 -Node: note query253498 -Node: payee query253764 -Node: real query254045 -Node: status query254250 -Node: type query254490 -Node: tag query255023 -Node: Negative queries255652 -Node: not query255834 -Node: Space-separated queries256121 -Node: Boolean queries256809 -Node: expr query258127 -Node: any query258807 -Node: all query259260 -Node: Queries and command options259842 -Node: Queries and account aliases260290 -Node: Queries and valuation260615 -Node: Pivoting260977 -Node: Generating data263260 -Node: Forecasting265060 -Node: --forecast265716 -Node: Inspecting forecast transactions266817 -Node: Forecast reports268150 -Node: Forecast tags269259 -Node: Forecast period in detail269879 -Node: Forecast troubleshooting270967 -Node: Budgeting272038 -Node: Amount formatting272598 -Node: Commodity display style272842 -Node: Rounding274683 -Node: Trailing decimal marks275288 -Node: Amount parseability276221 -Node: Cost reporting277830 -Node: Recording costs278661 -Node: Reporting at cost280388 -Node: Equity conversion postings281153 -Node: Inferring equity conversion postings283798 -Node: Combining costs and equity conversion postings284940 -Node: Requirements for detecting equity conversion postings286165 -Node: Infer cost and equity by default ?287687 -Node: Value reporting288124 -Node: -X Value in specified commodity289082 -Node: -V Value in default commoditys289942 -Node: Valuation date290679 -Node: Finding market price291511 -Node: --infer-market-prices market prices from transactions292891 -Node: Valuation commodity295935 -Node: --value Flexible valuation297368 -Node: Valuation examples299211 -Node: Interaction of valuation and queries301355 -Node: Effect of valuation on reports302072 -Node: PART 4 COMMANDS309922 -Node: Help commands312711 -Node: commands312897 -Node: demo313105 -Node: help314198 -Node: User interface commands315903 -Node: repl316114 -Node: Examples318378 -Node: run318936 -Node: Examples 2321351 -Node: ui322375 -Node: web322512 -Node: Data entry commands322640 -Node: add322901 -Node: add and balance assertions325475 -Node: add and balance assignments326049 -Node: import326742 -Node: Import dry run327821 -Node: Overlap detection328769 -Node: First import331655 -Node: Importing balance assignments332850 -Node: Import and commodity styles333905 -Node: Import archiving334339 -Node: Import special cases335164 -Node: Deduplication335382 -Node: Varying file name335873 -Node: Multiple versions336257 -Node: Basic report commands337364 -Node: accounts337665 -Node: codes340177 -Node: commodities341199 -Node: descriptions342207 -Node: files342667 -Node: notes342964 -Node: payees343476 -Node: prices344633 -Node: stats345525 -Node: tags347583 -Node: Standard report commands349407 -Node: print349712 -Node: print amount explicitness352443 -Node: print alignment353381 -Node: print amount style353695 -Node: print parseability354925 -Node: print other features356202 -Node: print output format357164 -Node: aregister360449 -Node: aregister and posting dates364914 -Node: register365815 -Node: Custom register output373055 -Node: balancesheet374240 -Node: balancesheetequity379205 -Node: cashflow384540 -Node: incomestatement389353 -Node: Advanced report commands394202 -Node: balance394410 -Node: balance features399831 -Node: Simple balance report401934 -Node: Balance report line format403744 -Node: Filtered balance report406104 -Node: List or tree mode406623 -Node: Depth limiting408136 -Node: Dropping top-level accounts408903 -Node: Showing declared accounts409413 -Node: Sorting by amount410143 -Node: Percentages410997 -Node: Multi-period balance report411704 -Node: Balance change end balance414456 -Node: Balance report modes416093 -Node: Calculation mode416772 -Node: Accumulation mode417476 -Node: Valuation mode418577 -Node: Combining balance report modes419921 -Node: Budget report421951 -Node: Using the budget report424251 -Node: Budget date surprises426527 -Node: Selecting budget goals427891 -Node: Budgeting vs forecasting428839 -Node: Balance report layout430516 -Node: Wide layout431721 -Node: Tall layout434126 -Node: Bare layout435432 -Node: Tidy layout437496 -Node: Balance report output439040 -Node: Some useful balance reports439814 -Node: roi441074 -Node: Spaces and special characters in --inv and --pnl443321 -Node: Semantics of --inv and --pnl444047 -Node: IRR and TWR explained446134 -Node: Chart commands449545 -Node: activity449726 -Node: Data generation commands450223 -Node: close450429 -Node: close --clopen453494 -Node: close --close455390 -Node: close --open455914 -Node: close --assert456164 -Node: close --assign456491 -Node: close --retain457170 -Node: close customisation458027 -Node: close and balance assertions459705 -Node: close examples461227 -Node: Retain earnings461464 -Node: Migrate balances to a new file461967 -Node: More detailed close examples463329 -Node: rewrite463551 -Node: Re-write rules in a file466111 -Node: Diff output format467412 -Node: rewrite vs print --auto468682 -Node: Maintenance commands469396 -Node: check469615 -Node: Basic checks470698 -Node: Strict checks471764 -Node: Other checks472637 -Node: Custom checks474339 -Node: diff474778 -Node: setup475986 -Node: test478853 -Node: PART 5 COMMON TASKS479756 -Node: Getting help480205 -Node: Constructing command lines481106 -Node: Starting a journal file481971 -Node: Setting LEDGER_FILE483375 -Node: Set LEDGER_FILE on unix483663 -Node: Set LEDGER_FILE on mac484180 -Node: Set LEDGER_FILE on Windows484908 -Node: Setting opening balances486631 -Node: Recording transactions489973 -Node: Reconciling490718 -Node: Reporting493127 -Node: Migrating to a new file497261 -Node: BUGS497717 -Node: Troubleshooting498543 +Node: PART 1 USER INTERFACE4142 +Node: Input4281 +Node: Text encoding5373 +Node: Data formats6122 +Node: Standard input7856 +Node: Multiple files8245 +Node: Strict mode8982 +Node: Commands9816 +Node: Add-on commands11098 +Node: Options12149 +Node: Special characters19299 +Node: Escaping shell special characters20289 +Node: Escaping regular expression special characters21648 +Node: Escaping in other situations23163 +Node: Unicode characters24311 +Node: Regular expressions25732 +Node: hledger's regular expressions28991 +Node: Argument files30662 +Node: Config files31559 +Node: Shell completions34828 +Node: Output35317 +Node: Output destination35508 +Node: Output format36066 +Node: Text output37920 +Node: Box-drawing characters38904 +Node: Colour39404 +Node: Paging39990 +Node: HTML output41531 +Node: CSV / TSV output41949 +Node: FODS output42203 +Node: Ledger output43004 +Node: Beancount output43422 +Node: Beancount account names44879 +Node: Beancount commodity names45420 +Node: Beancount balance assignments46364 +Node: Beancount virtual postings46690 +Node: Beancount metadata47010 +Node: Beancount costs47790 +Node: Beancount tolerance48197 +Node: Beancount operating currency48561 +Node: SQL output49015 +Node: JSON output49806 +Node: Commodity styles50623 +Node: Debug output51633 +Node: Environment52465 +Node: PART 2 DATA FORMATS53634 +Node: Journal53777 +Node: Journal cheatsheet56269 +Node: Comments62520 +Node: Transactions63464 +Node: Dates64601 +Node: Simple dates64753 +Node: Posting dates65369 +Node: Status66542 +Node: Code68308 +Node: Description68643 +Node: Payee and note69330 +Node: Transaction comments70421 +Node: Postings70937 +Node: Debits and credits72253 +Node: Account names72812 +Node: Two space delimiter73769 +Node: Account hierarchy75174 +Node: Other account name features76057 +Node: Amounts76475 +Node: Decimal marks77481 +Node: Digit group marks78585 +Node: Commodity79220 +Node: Costs80323 +Node: Basis / lots84015 +Node: Lot syntax85068 +Node: Balance assertions86014 +Node: Assertions and ordering87267 +Node: Assertions and multiple files87986 +Node: Assertions and costs89154 +Node: Assertions and commodities89801 +Node: Assertions and subaccounts91460 +Node: Assertions and status92120 +Node: Assertions and virtual postings92540 +Node: Assertions and auto postings92905 +Node: Assertions and precision93780 +Node: Assertions and hledger add94264 +Node: Posting comments95012 +Node: Transaction balancing95552 +Node: Tags97760 +Node: Tag propagation99279 +Node: Displaying tags100778 +Node: When to use tags ?101171 +Node: Tag names101835 +Node: Directives103834 +Node: Directives and multiple files105291 +Node: Directive effects106236 +Node: account directive109392 +Node: Account comments110859 +Node: Account tags111446 +Node: Account error checking111922 +Node: Account display order113628 +Node: Account types114826 +Node: alias directive118530 +Node: Basic aliases119741 +Node: Regex aliases120616 +Node: Combining aliases121663 +Node: Aliases and multiple files123117 +Node: end aliases directive123900 +Node: Aliases can generate bad account names124268 +Node: Aliases and account types125101 +Node: commodity directive125993 +Node: Commodity directive syntax127805 +Node: Commodity tags129371 +Node: Commodity error checking129910 +Node: decimal-mark directive130373 +Node: include directive130952 +Node: P directive133175 +Node: payee directive134209 +Node: tag directive134831 +Node: Periodic transactions135443 +Node: Periodic rule syntax137597 +Node: Periodic rules and relative dates138420 +Node: Two spaces between period expression and description!139197 +Node: Auto postings140158 +Node: Auto postings and multiple files143444 +Node: Auto postings and dates143849 +Node: Auto postings and transaction balancing / inferred amounts / balance assertions144290 +Node: Auto posting tags145136 +Node: Auto postings on forecast transactions only146031 +Node: Other syntax146501 +Node: Balance assignments147291 +Node: Balance assignments and costs148819 +Node: Balance assignments and multiple files149241 +Node: Bracketed posting dates149664 +Node: D directive150362 +Node: apply account directive152135 +Node: Y directive153002 +Node: Secondary dates153990 +Node: Star comments155475 +Node: Valuation expressions156167 +Node: Virtual postings156466 +Node: Other Ledger directives158090 +Node: Ledger virtual costs158848 +Node: Ledger lot syntax159190 +Node: CSV160911 +Node: CSV rules cheatsheet163111 +Node: source165409 +Node: Data cleaning / data generating commands166813 +Node: archive168712 +Node: encoding169640 +Node: separator170683 +Node: skip171336 +Node: date-format171986 +Node: timezone172931 +Node: newest-first174057 +Node: intra-day-reversed174770 +Node: decimal-mark175372 +Node: CSV fields and hledger fields175870 +Node: fields list177746 +Node: Field assignment179571 +Node: Field names180790 +Node: date field182122 +Node: date2 field182286 +Node: status field182481 +Node: code field182671 +Node: description field182859 +Node: comment field183076 +Node: account field183633 +Node: amount field184351 +Node: currency field187190 +Node: balance field187598 +Node: if block188121 +Node: Matchers189648 +Node: Multiple matchers191576 +Node: Match groups192384 +Node: if table193277 +Node: balance-type195340 +Node: include196167 +Node: Working with CSV196736 +Node: Rapid feedback197325 +Node: Valid CSV197908 +Node: File Extension198784 +Node: Reading CSV from standard input199519 +Node: Reading multiple CSV files199905 +Node: Reading files specified by rule200381 +Node: Valid transactions201778 +Node: Deduplicating importing202603 +Node: Regular expressions in CSV rules203849 +Node: Setting amounts205341 +Node: Amount signs207879 +Node: Setting currency/commodity208944 +Node: Amount decimal places210320 +Node: Referencing other fields211577 +Node: How CSV rules are evaluated212685 +Node: Well factored rules215402 +Node: CSV rules examples215892 +Node: Bank of Ireland216090 +Node: Coinbase217687 +Node: Amazon218870 +Node: Paypal220712 +Node: Timeclock228462 +Node: Timedot232515 +Node: Timedot examples235992 +Node: PART 3 REPORTING CONCEPTS238269 +Node: Time periods238433 +Node: Report start & end date238706 +Node: Smart dates240182 +Node: Report intervals242553 +Node: Date adjustments243127 +Node: Start date adjustment243347 +Node: End date adjustment244250 +Node: Period headings245031 +Node: Period expressions245964 +Node: Period expressions with a report interval247869 +Node: More complex report intervals248317 +Node: Multiple weekday intervals250433 +Node: Depth251444 +Node: Combining depth options252430 +Node: Queries253380 +Node: Query types256052 +Node: acct query256427 +Node: amt query256738 +Node: code query257435 +Node: cur query257630 +Node: desc query258236 +Node: date query258419 +Node: date2 query259033 +Node: depth query259374 +Node: note query259710 +Node: payee query259976 +Node: real query260257 +Node: status query260462 +Node: type query260702 +Node: tag query261260 +Node: Negative queries261889 +Node: not query262071 +Node: Space-separated queries262358 +Node: Boolean queries263046 +Node: expr query264364 +Node: any query265044 +Node: all query265497 +Node: Queries and command options266079 +Node: Queries and account aliases266527 +Node: Queries and valuation266852 +Node: Pivoting267214 +Node: Generating data269497 +Node: Detecting special postings271407 +Node: Forecasting274162 +Node: --forecast274829 +Node: Inspecting forecast transactions275930 +Node: Forecast reports277263 +Node: Forecast tags278372 +Node: Forecast period in detail278992 +Node: Forecast troubleshooting280080 +Node: Budgeting281151 +Node: Amount formatting281711 +Node: Commodity display style281955 +Node: Rounding283796 +Node: Trailing decimal marks284401 +Node: Amount parseability285334 +Node: Cost reporting286943 +Node: Recording costs287774 +Node: Reporting at cost289501 +Node: Equity conversion postings290266 +Node: Inferring equity conversion postings292911 +Node: Combining costs and equity conversion postings294157 +Node: Requirements for detecting equity conversion postings295382 +Node: Infer cost and equity by default ?296904 +Node: Value reporting297341 +Node: -X Value in specified commodity298297 +Node: -V Value in default commoditys299157 +Node: Valuation date299894 +Node: Finding market price300726 +Node: --infer-market-prices market prices from transactions302106 +Node: Valuation commodity305150 +Node: --value Flexible valuation306583 +Node: Valuation examples308426 +Node: Interaction of valuation and queries310570 +Node: Effect of valuation on reports311287 +Node: Lot reporting319137 +Node: Lotful commodities and accounts319821 +Node: --lots320472 +Node: Lot subaccounts321428 +Node: Lot operations322527 +Node: Reduction methods323966 +Node: Lot postings with balance assertions326380 +Node: Gain postings and disposal balancing327408 +Node: Lot reporting example328672 +Node: PART 4 COMMANDS330075 +Node: Help commands332862 +Node: commands333048 +Node: demo333256 +Node: help334349 +Node: User interface commands336054 +Node: repl336265 +Node: Examples338529 +Node: run339087 +Node: Examples 2341502 +Node: ui342526 +Node: web342663 +Node: Data entry commands342791 +Node: add343052 +Node: add and balance assertions345977 +Node: add and balance assignments346551 +Node: import347244 +Node: Import dry run348323 +Node: Overlap detection349271 +Node: First import352157 +Node: Importing balance assignments353352 +Node: Import and commodity styles354407 +Node: Import archiving354841 +Node: Import special cases355666 +Node: Deduplication355884 +Node: Varying file name356375 +Node: Multiple versions356759 +Node: Basic report commands357866 +Node: accounts358167 +Node: codes360679 +Node: commodities361701 +Node: descriptions362744 +Node: files363204 +Node: notes363501 +Node: payees364013 +Node: prices365170 +Node: stats366062 +Node: tags368120 +Node: Standard report commands369944 +Node: print370249 +Node: print amount explicitness373016 +Node: print alignment373954 +Node: print amount style374268 +Node: print parseability375498 +Node: print other features376775 +Node: print output format377737 +Node: aregister381225 +Node: aregister and posting dates385690 +Node: register386591 +Node: Custom register output394312 +Node: balancesheet395497 +Node: balancesheetequity400462 +Node: cashflow405797 +Node: incomestatement410610 +Node: Advanced report commands415459 +Node: balance415667 +Node: balance features421088 +Node: Simple balance report423191 +Node: Balance report line format425001 +Node: Filtered balance report427361 +Node: List or tree mode427880 +Node: Depth limiting429393 +Node: Dropping top-level accounts430160 +Node: Showing declared accounts430670 +Node: Sorting by amount431400 +Node: Percentages432254 +Node: Multi-period balance report432961 +Node: Balance change end balance435713 +Node: Balance report modes437350 +Node: Calculation mode438029 +Node: Accumulation mode438733 +Node: Valuation mode439834 +Node: Combining balance report modes441178 +Node: Budget report443208 +Node: Using the budget report445508 +Node: Budget date surprises447784 +Node: Selecting budget goals449148 +Node: Budgeting vs forecasting450096 +Node: Balance report layout451773 +Node: Wide layout452978 +Node: Tall layout455383 +Node: Bare layout456689 +Node: Tidy layout458753 +Node: Balance report output460297 +Node: Some useful balance reports461071 +Node: roi462331 +Node: Spaces and special characters in --inv and --pnl464578 +Node: Semantics of --inv and --pnl465304 +Node: IRR and TWR explained467391 +Node: Chart commands470802 +Node: activity470983 +Node: Data generation commands471480 +Node: close471686 +Node: close --clopen474751 +Node: close --close476647 +Node: close --open477171 +Node: close --assert477421 +Node: close --assign477748 +Node: close --retain478427 +Node: close customisation479284 +Node: close and balance assertions480962 +Node: close examples482682 +Node: Retain earnings482919 +Node: Migrate balances to a new file483422 +Node: More detailed close examples484784 +Node: rewrite485006 +Node: Re-write rules in a file487566 +Node: Diff output format488867 +Node: rewrite vs print --auto490137 +Node: Maintenance commands490851 +Node: check491070 +Node: Basic checks492153 +Node: Strict checks493219 +Node: Other checks494234 +Node: Custom checks495936 +Node: diff496375 +Node: setup497583 +Node: test500450 +Node: PART 5 COMMON TASKS501353 +Node: Getting help501802 +Node: Constructing command lines502703 +Node: Starting a journal file503568 +Node: Setting LEDGER_FILE504972 +Node: Set LEDGER_FILE on unix505260 +Node: Set LEDGER_FILE on mac505779 +Node: Set LEDGER_FILE on Windows506509 +Node: Setting opening balances508414 +Node: Recording transactions511756 +Node: Reconciling512501 +Node: Reporting514910 +Node: Migrating to a new file519044 +Node: BUGS519500 +Node: Troubleshooting520326  End Tag Table diff --git a/hledger/hledger.m4.md b/hledger/hledger.m4.md index ad9146f08a3..fd41b53f0b9 100644 --- a/hledger/hledger.m4.md +++ b/hledger/hledger.m4.md @@ -647,16 +647,16 @@ $ hledger print -o - # write to stdout (the default) Some commands offer other kinds of output, not just text on the terminal. Here are those commands and the formats currently supported: -| command | txt | html | csv/tsv | fods | beancount | sql | json | -|--------------------|-----|------|---------|------|-----------|-----|------| -| aregister | Y | Y | Y | Y | | | Y | -| balance | Y | Y | Y | Y | | | Y | -| balancesheet | Y | Y | Y | Y | | | Y | -| balancesheetequity | Y | Y | Y | Y | | | Y | -| cashflow | Y | Y | Y | Y | | | Y | -| incomestatement | Y | Y | Y | Y | | | Y | -| print | Y | Y | Y | Y | Y | Y | Y | -| register | Y | Y | Y | Y | | | Y | +| command | txt | html | csv/tsv | fods | ledger | beancount | sql | json | +|--------------------|-----|------|---------|------|--------|-----------|-----|------| +| aregister | Y | Y | Y | Y | | | | Y | +| balance | Y | Y | Y | Y | | | | Y | +| balancesheet | Y | Y | Y | Y | | | | Y | +| balancesheetequity | Y | Y | Y | Y | | | | Y | +| cashflow | Y | Y | Y | Y | | | | Y | +| incomestatement | Y | Y | Y | Y | | | | Y | +| print | Y | Y | Y | Y | Y | Y | Y | Y | +| register | Y | Y | Y | Y | | | | Y | - -2024-12-20: Some more things not yet handled for you: -- P directives are not converted automatically - convert those yourself. -- Balance assignments are not converted (Beancount doesn't support them) - replace those with explicit amounts. +A top level hledger account named `revenue` or `revenues` (case insensitive) will be converted to `Income` for Beancount. +To adjust other top level account names, you should use `--alias` (see [Account aliases](#alias-directive), +or this [hledger2beancount.conf](https://github.com/simonmichael/hledger/blob/master/examples/hledger2beancount.conf) file). + #### Beancount account names @@ -849,7 +853,13 @@ hledger will adjust your commodity names to make valid which must be 2-24 uppercase letters, digits, or `'`, `.`, `_`, `-`, beginning with a letter and ending with a letter or digit. hledger will convert known currency symbols to [ISO 4217 currency codes](https://en.wikipedia.org/wiki/ISO_4217#Active_codes), capitalise letters, replace spaces with `-`, replace other unsupported characters with `C`, -and prepend or append `C` if needed. +and prepend or append `C` if needed. One-letter symbols will be doubled. The no-symbol commodity will become `CC`. +(Note, hledger tries to keep your commodities distinct, but collisions are possiblel with short alphanumeric symbols like +`CC`, `C`, and no-symbol, which are distinct in hledger but all become `CC` in beancount.) + +#### Beancount balance assignments + +Beancount doesn't support those; any balance assignments will be converted to explicit amounts. #### Beancount virtual postings @@ -873,6 +883,12 @@ Beancount doesn't allow [redundant costs and conversion postings](https://hledge If you have any of these, the conversion postings will be omitted. Currently we support at most one cost + conversion postings group per transaction. +#### Beancount tolerance + +A sample `inferred_tolerance_default` option is provided (commented out). +If Beancount complains that transactions aren't balanced, +this is an easy way to work around it. + #### Beancount operating currency Declaring an operating currency (or several) improves Beancount and Fava reports. @@ -1638,6 +1654,8 @@ you add extra equity postings to balance the two commodities. Eg: hledger offers a more convenient @/@@ "cost notation" as an alternative: instead of equity postings, you can write the "conversion rate" or "transacted price" after a posting amount. hledger docs generically call this "cost", whether buying or selling. +("cost" is an overloaded and generic term here, but we still use it for historical reasons. "Transacted price" is more precise.) + It can be written as either `@ UNITPRICE` or `@@ TOTALPRICE`. Eg you could write the above as: @@ -1724,31 +1742,49 @@ as long as the journal entry is well formed such that the equity postings / cost So in principle you could enable both `--infer-equity` and `--infer-costs` in your config file, and your reports would have the advantages of both. -## Cost basis / lot syntax +## Basis / lots + +This is an advanced topic; skip over if you don't need it. +For full details, see [Lot reporting](#lot-reporting). + +If you are buying an amount of some commodity to hold as an investment, +it may be important to keep track of its *cost basis* (AKA "basis") and *lots*. +("Cost basis" and "cost" (see above) sound similar, and often have the same amount in simple purchases; but they are distinct concepts.) + +The basis records: + +1. the amount's nominal acquisition cost (usually the same as the transacted cost, ie what you paid for it, but not always) +2. the nominal acquisition date (usually the date you acquired it, but not always) +3. and optionally a label, to disambiguate or identify lots. -If you are buying some commodity to hold as an investment, it may be important to keep track of +An amount with a basis is called a lot. +The basis is a property of the lot, remaining with it throughout its lifetime. +It is used to calculate capital gains/losses, eg for tax reporting. -1. its original acquisition cost -2. its original acquisition date -3. and a sequence number or label, if needed, to disambiguate multiple acquisitions on the same day, or to serve as a mnemonic for easy reference. +### Lot syntax -In hledger we call these three the "cost basis"; and if an amount being acquired has a cost basis, we call it a "lot". -Tax authorities often require that lots are tracked carefully and disposed of (sold) in a certain order. +A basis or lot can be described using "lot syntax". hledger supports two lot syntax styles: -Note, though "cost basis" sounds similar to the "cost" (transacted price) discussed above, they are distinct concepts. -In some transactions the transacted price and basis cost are the same, but in others they are not. +**hledger lot syntax** puts all basis fields inside braces, comma-separated (like Beancount): -So cost basis has its own syntax, also called "lot syntax". -hledger's lot syntax is like Ledger's: one or more of the following annotations, following the main amount: + 10 AAPL {2026-01-15, $50} + 10 AAPL {2026-01-15, "lot1", $50} + 10 AAPL {$50} + 10 AAPL {} -- `{LOTUNITCOST}` or `{{{{LOTTOTALCOST}}}}` (see [lot price][ledger: buying and selling stock]) -- `[LOTDATE]` (see [lot date][ledger: lot dates]) -- `(LOTLABEL)` (see [lot note][ledger: lot notes]) +All fields are optional, but when present they must be in date-label-cost order. +Dates must be in YYYY-MM-DD format, and labels must be double-quoted. +The cost is a single-commodity costless hledger amount. + +When an amount has both a basis and a transacted price, +like `-10 AAPL {$50} @ $60`, the preferred order is to write {} before @. + +**Ledger lot syntax** uses separate annotations, in any order: + + 10 AAPL {$50} [2026-01-15] (lot1) + +hledger accepts both styles on input. Print output uses consolidated style by default, or Ledger style with `-O ledger`. -hledger does not yet do anything with this lot syntax, except to preserve it and show it in `print`'s `txt`, `beancount`, and `json` output. -This means you can use this syntax in your hledger journals (plus an amountless extra posting to help transactions balance, if needed), -then use the `print` command to export to Ledger or Beancount or rustledger, to use their lots/gains reports -(see [Export Lots workflow](workflows.md#more-advanced-workflows)). ## Balance assertions @@ -2228,9 +2264,10 @@ account assets:bank:checking ; same-line comment, at least 2 spaces before th ### Account tags An account directive's comment may contain [tags](#tags). -These will be propagated to all postings using that account, as hidden but queryable posting tags, -except where the posting already a tag of the same name. -(Posting tags override account tags.) +These will be inherited by all postings using that account, +except where the posting already has a value for that tag. +(A posting tag overrides an account tag.) +Note, these tags will be queryable but won't be shown in `print` output, even with --verbose-tags. ### Account error checking @@ -2247,6 +2284,7 @@ Some notes: - The account directive's scope is "whole file and below" (see [directives](#directives)). This means it affects all of the current file, and any files it includes, but not parent or sibling files. The position of account directives within the file does not matter, though it's usual to put them at the top. - Accounts can only be declared in `journal` files, but will affect [included](#include-directive) files of all types. - It's currently not possible to declare "all possible subaccounts" with a wildcard; every account posted to must be declared. +- As an exception: lot subaccounts (a final account name component like `:{2026-01-15, $50}`) are always ignored by `check accounts`, and need not be declared. - If you use the [--infer-equity](#inferring-equity-conversion-postings) flag, you will also need declarations for the account names it generates. ### Account display order @@ -2658,9 +2696,10 @@ commodity INR ### Commodity tags A commodity directive's comment may contain [tags](#tags). -These will be propagated to all postings using that commodity in their main amount, as hidden but queryable posting tags, -except where the posting already a tag of the same name. -(Posting tags override account tags override commodity tags.) +These will be inherited by all postings using that commodity in their main amount, +except where the posting already has a value for that tag. +(A posting tag or an account tag overrides a commodity tag.) +Note, these tags will be queryable but won't be shown in `print` output, even with --verbose-tags. ### Commodity error checking @@ -3305,27 +3344,17 @@ In hledger, these are equivalent to `@` and `@@`. ### Ledger lot syntax -In Ledger, these optional annotations after an amount help specify the cost basis of a newly acquired lot, -or select existing lot(s) to dispose of: - -- `{LOTUNITCOST}` and `{{{{LOTTOTALCOST}}}}` ([lot price][ledger: buying and selling stock]) -- `[LOTDATE]` ([lot date][ledger: lot dates]) -- `(LOTNOTE)` ([lot note][ledger: lot notes]) - -hledger does not yet calculate lots itself, but it accepts these annotations and will show them in `print`'s `txt`, `beancount`, and `json` output formats. -This means you can use this syntax in your hledger journals (with an amountless extra posting to help transactions balance, when needed), -and use the `print` command to export to Ledger or Beancount when you want to calculate lots and capital gains. - -### Ledger fixed lot costs +In Ledger, these annotations after an amount help specify or select a lot's [cost basis](#basis-lots): +`{LOTUNITCOST}` or `{{{{LOTTOTALCOST}}}}`, `[LOTDATE]`, and/or `(LOTNOTE)`. +hledger will read these, as an alternative to its own [lot syntax](#lot-syntax)). -- `{=UNITCOST}` and `{{{{=TOTALCOST}}}}` ([fixed price][ledger: fixing lot prices]) - - when buying, means "this cost is also the fixed value, don't let it fluctuate in value reports" - -Probably equivalent to `@`/`@@`, I'm not sure. +We also read Ledger's [fixed price][ledger: fixing lot prices]) syntax, +`{=LOTUNITCOST}` or `{{{{=LOTTOTALCOST}}}}`, +treating it as equivalent to `{LOTUNITCOST}` or `{{{{LOTTOTALCOST}}}}`, +[ledger: fixing lot prices]: https://www.ledger-cli.org/3.0/doc/ledger3.html#Fixing-Lot-Prices [ledger: virtual posting costs]: https://www.ledger-cli.org/3.0/doc/ledger3.html#Virtual-posting-costs [ledger: buying and selling stock]: https://www.ledger-cli.org/3.0/doc/ledger3.html#Buying-and-Selling-Stock -[ledger: fixing lot prices]: https://www.ledger-cli.org/3.0/doc/ledger3.html#Fixing-Lot-Prices [ledger: lot dates]: https://www.ledger-cli.org/3.0/doc/ledger3.html#Lot-dates [ledger: lot notes]: https://www.ledger-cli.org/3.0/doc/ledger3.html#Lot-notes @@ -5972,6 +6001,7 @@ Mostly, this is done only if you request it: - The `--infer-market-prices` flag infers `P` price directives from costs. - The `--auto` flag adds extra postings to transactions matched by [auto posting rules](#auto-postings). - The `--forecast` option generates transactions from [periodic transaction rules](#periodic-transactions). +- The `--lots` flag adds extra lot subaccounts to postings for detailed [lot reporting](#lot-reporting). - The `balance --budget` report infers budget goals from periodic transaction rules. - Commands like `close`, `rewrite`, and `hledger-interest` generate transactions or postings. - CSV data is converted to transactions by applying CSV conversion rules.. etc. @@ -5986,6 +6016,25 @@ If you are curious what data is being generated and why, run `hledger print -x - Similar hidden tags (with an underscore prefix) are always present, also, so you can always match such data with queries like `tag:generated` or `tag:modified`. +# Detecting special postings + +hledger detects certain kinds of postings, both generated and non-generated, and tags them for additional processing. +These are documented elsewhere, but this section gives an overview of the special posting detection rules. + +By default, the [tags](#tags) are hidden (with a `_` prefix), so they can be queried but they won't appear in `print` output. +To also add visible tags, use `--verbose-tags` (useful for troubleshooting). + +| Tag | Detected pattern | Effect | +|-----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------| +| `conversion-posting` | A pair of adjacent, single-commodity, costless postings to `Conversion`-type accounts, with a nearby corresponding costful or potentially corresponding costless posting | Helps transaction balancer infer costs or avoid redundancy in commodity conversions | +| `cost-posting` | A costful posting whose amount and transacted cost correspond to a conversion postings pair; or a costless posting matching one of the pair | Helps transaction balancer infer costs or avoid redundancy in commodity conversions | +| `generated-posting` | Postings generated at runtime | Helps users understand or find postings added at runtime by hledger | +| `ptype:acquire` | Positive postings with [lot annotations](#lot-syntax), or in a lotful commodity/account, with no matching counterposting | Creates a new lot | +| `ptype:dispose` | Negative postings with lot annotations, or in a lotful commodity/account, with no matching counterposting | Selects and reduces existing lots | +| `ptype:transfer-from` | The negative posting of a pair of counterpostings, at least one with lot annotation or a lotful commodity/account; or a negative lot posting with an equity counterpart (equity transfer) | Moves lots between accounts, preserving cost basis | +| `ptype:transfer-to` | The positive posting of a transfer pair; or a positive lot posting with an equity counterpart (equity transfer, e.g. opening balances) | As above | +| `ptype:gain` | A posting to a `Gain`-type account | Helps transaction balancer avoid redundancy, helps disposal balancer check realised capital gain/loss | + # Forecasting Forecasting, or speculative future reporting, can be useful for estimating future balances, or for exploring different future scenarios. @@ -6901,6 +6950,244 @@ First, a quick glossary: `--cumulative` is omitted to save space, it works like `-H` but with a zero starting balance. +# Lot reporting + +With the `--lots` flag, hledger can track investment lots automatically: +assigning lot subaccounts on acquisition, selecting lots on disposal using +configurable methods, calculating capital gains, and showing per-lot balances +in all reports. +(Since 1.99.1, experimental. For more technical details, see [SPEC-lots.md](SPEC-lots.md)). + +## Lotful commodities and accounts + +Commodities and accounts can be declared as "lotful" by adding a `lots` tag +in their declaration: + +```journal +commodity AAPL ; lots: +account assets:stocks ; lots: +``` + +This tells hledger that postings involving these always involve lots, +enabling cost basis inference even when lot syntax is not written explicitly. + +The tag value can also specify a [reduction method](#reduction-methods): + +```journal +commodity AAPL ; lots: FIFO +account assets:stocks ; lots: LIFO +``` + +If no value is specified, the default is FIFO. + +## --lots + +Add `--lots` to any command to enable lot tracking. This activates: + +- **Lot posting classification** — lot-related postings are tagged as `acquire`, `dispose`, + `transfer-from`, `transfer-to`, or `gain` (via a hidden `ptype` tag, + visible with `--verbose-tags`, queryable with `tag:ptype=...`). +- **Cost basis inference** — for lotful commodities/accounts, cost basis + is inferred from transacted cost and vice versa. Or when the account name + ends with a lot subaccount, cost basis can also be inferred from that. +- **Lot calculation** — acquired lots become subaccounts; disposals and + transfers select from existing lots. +- **Disposal balancing** — disposal transactions are checked for balance + at cost basis; gain amounts/postings are inferred if missing. + +## Lot subaccounts + +With `--lots`, each acquired lot becomes a subaccount named by its cost basis: + +```journal +commodity AAPL ; lots: + +2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:cash -$500 +``` + +```cli +$ hledger bal assets:stocks --lots -N + 10 AAPL assets:stocks:{2026-01-15, $50} +``` + +You can also write lot subaccounts explicitly. When a posting's account name +ends with a lot subaccount (like `:{2026-01-15, $50}`), the cost basis is +parsed from it automatically, so a `{}` annotation on the amount is optional: + +```journal +commodity AAPL ; lots: + +2026-01-15 buy + assets:stocks:{2026-01-15, $50} 10 AAPL + assets:cash +``` + +This is equivalent to writing `10 AAPL {2026-01-15, $50}`. +(If both the account name and the amount specify a cost basis, they must agree.) + +When [strictly checking account names](#account-error-checking), lot subaccounts are automatically exempt — +you only need to declare the base account (eg `account assets:stocks`), not each individual lot subaccount. + +## Lot operations + +- **Acquire**: a positive lot posting creates a new lot. + The cost basis can be specified explicitly with `{}` on the amount, + inferred from the lot subaccount name, + or inferred from the transacted cost. + On lotful commodities/accounts, even a bare positive posting (no `{}` or `@`) can be detected as an acquire, + with cost inferred from the transaction's other postings. +- **Transfer**: a matching pair of negative/positive lot postings moves a lot between accounts, preserving its cost basis. + Transfer postings should not have a transacted price. + If the destination receives less than the source sends (eg due to a fee deducted by an exchange), + the fee portion of lots is consumed from the source without being recreated at the destination. +- **Dispose**: a negative lot posting sells from one or more existing lots. + It must have a transacted price (the selling price), either explicit or inferred. + +An example disposal entry: + +```journal +2026-02-01 sell + assets:stocks -5 AAPL {$50} @ $60 + assets:cash $300 + revenue:gains -$50 +``` + +With `--lots`, this selects the specified quantity of the matching lot (which must exist) and will show something like: + +```cli +$ hledger print --lots desc:sell +2026-02-01 sell + assets:stocks:{2026-01-15, $50} -5 AAPL {2026-01-15, $50} @ $60 + assets:cash $300 + revenue:gains $-50 +``` + +## Reduction methods + +When a disposal or transfer doesn't specify a particular lot (eg the amount is like `-5 AAPL {}`, or just `-5 AAPL`), +hledger selects lot(s) automatically using a reduction method. +You can configure the method via the `lots:` tag on a commodity or account declaration (account tags override commodity tags): + +```journal +commodity AAPL ; lots: FIFOALL +account assets:mutualfund ; lots: AVERAGE +``` + +The default method is `FIFO` (first in first out within one account). +The available methods are: + +| Method | Lots selected | Disposal cost basis | Error checking +|--------------------|--------------------|---------------------------|--------------------------------------------------------------------------------------- +| **SPECID** | one specific lot | specified lot's cost | A matching lot, with sufficient balance, exists in the account. +| **FIFO** (default) | oldest first | each lot's cost | Sufficient lot(s) exist in the account. +| **LIFO** | newest first | each lot's cost | " +| **HIFO** | highest cost first | each lot's cost | " +| **AVERAGE** | oldest first | weighted average cost | " +| **FIFOALL** | oldest first | each lot's cost | Sufficient lot(s) exist in the account, and are highest priority across all accounts. +| **LIFOALL** | newest first | each lot's cost | " +| **HIFOALL** | highest cost first | each lot's cost | " +| **AVERAGEALL** | oldest first | global weighted avg cost | " + +**SPECID** (specific identification) is what you're using if the journal entry contains +explicit lot selectors like `{2026-01-15, $50}` or `{$50}`, +or an explicit lot subaccount like `assets:broker:{2026-01-15, $50}`. + +**HIFO** (highest-in-first-out) selects the lot with the highest per-unit cost first, +which can be useful for tax optimization. + +**AVERAGE** consumes lots in FIFO order, +but uses the weighted average per-unit cost, within the specified account, +as the disposal cost basis, rather than each lot's individual cost. +This is required in some jurisdictions (eg Canada's Adjusted Cost Base, France's PMPA, UK's S104 pools). + +All of these methods select lots from the account mentioned in the posting. +But the **\*ALL** variants (FIFOALL, LIFOALL, HIFOALL, AVERAGEALL) additionally validate +that these lots are the ones that would be chosen if considering the global pool (all accounts holding that commodity). +So if there is a more appropriate lot in another account (eg an older lot when using FIFOALL), +they will raise an error showing which account holds it. +This is useful if you need to enforce a global disposal order across all accounts (brokers, exchanges, wallets etc). + +**AVERAGEALL** computes the weighted average cost across the global pool. + +## Lot postings with balance assertions + +On a dispose or transfer posting without an explicit lot subaccount, a [balance assertion](#balance-assertions) +always refers to the parent account's balance. So if lot subaccounts are added witih `--lots`, the assertion is not affected. + +By contrast, in a journal entry where the lot subaccounts are recorded explicitly, a balance assertion +refers to the lot subaccount's balance. + +This means that `hledger print --lots`, if it adds explicit lot subaccounts to a journal entry, +could potentially change the meaning of balance assertions, breaking them. To avoid this, in such cases it will move +the balance assertion to a new zero-amount posting to the parent account (and make sure it's subaccount-inclusive). +(So eg `hledger -f- print --lots -x | hledger -f- check assertions` will still pass.) + +## Gain postings and disposal balancing + +A **gain posting** is a posting to a [Gain-type account](#account-types) (type `G`, a subtype of Revenue). +In disposal transactions, it records the capital gain or loss, +which is the difference between cost basis and selling price of the lots being sold. + +Accounts named like `revenue:gains` or `income:capital-gains` are detected as Gain accounts automatically, +or you can declare one explicitly: + +```journal +account gain/loss ; type: G +``` + +Gain postings have special treatment: + +- **Normal transaction balancing** ignores gain postings (they don't count toward the balance check), and balances the transaction using transacted price +- **Disposal balancing** (with `--lots`) includes gain postings, and balances the transaction using cost basis + +An amountless gain posting in a disposal transaction will have its amount filled in. +Or if a disposal transaction is unbalanced at cost basis and has no gain posting, +one is inferred automatically (posting to the first Gain account, or `revenue:gains` if none is declared). + +## Lot reporting example + +```journal +commodity AAPL ; lots: + +2026-01-15 buy low + assets:stocks 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 buy high + assets:stocks 10 AAPL {$60} + assets:cash -$600 + +2026-03-01 sell some (FIFO, selects oldest lot first) + assets:stocks -5 AAPL @ $70 + assets:cash $350 + revenue:gains +``` + +```cli +$ hledger bal assets:stocks --lots -N + 5 AAPL assets:stocks:{2026-01-15, $50} + 10 AAPL assets:stocks:{2026-02-01, $60} +``` + +```cli +$ hledger print --lots -x desc:sell +2026-03-01 sell some (FIFO, selects oldest lot first) + assets:stocks:{2026-01-15, $50} -5 AAPL {2026-01-15, $50} @ $70 + assets:cash $350 + revenue:gains -$100 +``` +``` +$ hledger print --lots -x desc:sell --verbose-tags +2026-03-01 sell some (FIFO, selects oldest lot first) + assets:stocks:{2026-01-15, $50} -5 AAPL {2026-01-15, $50} @ $70 ; ptype: dispose + assets:cash $350 + revenue:gains $-100 ; ptype: gain +``` +The gain of $100 was inferred: 5 shares acquired at $50, sold at $70 = 5 × ($70 - $50) = $100. + + # PART 4: COMMANDS diff --git a/hledger/hledger.txt b/hledger/hledger.txt index a7164c90f03..9f68d7f844f 100644 --- a/hledger/hledger.txt +++ b/hledger/hledger.txt @@ -583,8 +583,8 @@ Options placement string to reference capturing groups in the search regexp. Otherwise, if you write \1, it will match the digit 1. - 6. they do not support mode modifiers ((?s)), character classes (\w, - \d), or anything else not mentioned above. + 6. they do not support lazy quantifiers (*?), mode modifiers ((?s)), + character classes (\w, \d), or anything else not mentioned above. 7. they may not (I'm guessing not) properly support right-to-left or bidirectional text. @@ -739,16 +739,16 @@ Output Some commands offer other kinds of output, not just text on the termi- nal. Here are those commands and the formats currently supported: - command txt html csv/tsv fods beancount sql json - -------------------------------------------------------------------------------------------- - aregister Y Y Y Y Y - balance Y Y Y Y Y - balancesheet Y Y Y Y Y - balancesheetequity Y Y Y Y Y - cashflow Y Y Y Y Y - incomestatement Y Y Y Y Y - print Y Y Y Y Y Y Y - register Y Y Y Y Y + command txt html csv/tsv fods ledger beancount sql json + ----------------------------------------------------------------------------------------------- + aregister Y Y Y Y Y + balance Y Y Y Y Y + balancesheet Y Y Y Y Y + balancesheetequity Y Y Y Y Y + cashflow Y Y Y Y Y + incomestatement Y Y Y Y Y + print Y Y Y Y Y Y Y Y + register Y Y Y Y Y You can also see which output formats a command supports by running hledger CMD -h and looking for the -O/--output-format=FMT option, @@ -825,14 +825,19 @@ Output pager to handle ANSI codes. Or you could disable colour as described above. - If you are using the less pager, hledger automatically appends a number - of options to the LESS variable to enable ANSI colour and a number of - other conveniences. (At the time of writing: --chop-long-lines - --hilite-unread --ignore-case --no-init --quit-at-eof - --quit-if-one-screen --RAW-CONTROL-CHARS --shift=8 - --squeeze-blank-lines --use-backslash ). If these don't work well, you - can set your preferred options in the HLEDGER_LESS variable, which will - be used instead. + If you are using the less pager, hledger tries to provide a consis- + tently pleasant experience by running it with some extra options added + to your LESS environment variable: + + --chop-long-lines --hilite-unread --ignore-case --no-init + --quit-if-one-screen --shift=8 --squeeze-blank-lines --use-backslash + + and when colour output is enabled: + + --RAW-CONTROL-CHARS + + You can prevent this by setting your preferred options in the + HLEDGER_LESS variable, which will be used instead of LESS. HTML output HTML output can be styled by an optional hledger.css file in the same @@ -858,9 +863,16 @@ Output extract CSV from FODS/ODS using various utilities like libreoffice --headless or ods2csv. + Ledger output + This is a Ledger-specific journal format supported by the print com- + mand. It is currently identical to hledger's default print output ex- + cept that amounts' cost basis will use Ledger's lot syntax, ({COST} + [DATE] (NOTE)), not hledger's ({DATE, "LABEL", COST}). + Beancount output - This is Beancount's journal format. You can use this to export your - hledger data to Beancount, eg to use the Fava web app. + This is Beancount's journal format, supported by the print command. + You can use this to export your hledger data to Beancount, eg to use + the Fava web app. hledger will try to adjust your data to suit Beancount, automatically. Be cautious and check the conversion until you are confident it is @@ -877,17 +889,12 @@ Output There is one big adjustment you must handle yourself: for Beancount, the top level account names must be Assets, Liabilities, Equity, In- - come, and/or Expenses. You can use account aliases to rewrite your ac- - count names temporarily, if needed, as in this hledger2beancount.conf - config file. - - 2024-12-20: Some more things not yet handled for you: + come, and/or Expenses. - o P directives are not converted automatically - convert those your- - self. - - o Balance assignments are not converted (Beancount doesn't support - them) - replace those with explicit amounts. + A top level hledger account named revenue or revenues (case insensi- + tive) will be converted to Income for Beancount. To adjust other top + level account names, you should use --alias (see Account aliases, or + this hledger2beancount.conf file). Beancount account names Aside from the top-level names, hledger will adjust your account names @@ -903,30 +910,44 @@ Output ', ., _, -, beginning with a letter and ending with a letter or digit. hledger will convert known currency symbols to ISO 4217 currency codes, capitalise letters, replace spaces with -, replace other unsupported - characters with C, and prepend or append C if needed. + characters with C, and prepend or append C if needed. + One-letter symbols will be doubled. The no-symbol commodity will be- + come CC. (Note, hledger tries to keep your commodities distinct, but + collisions are possiblel with short alphanumeric symbols like CC, C, + and no-symbol, which are distinct in hledger but all become CC in bean- + count.) + + Beancount balance assignments + Beancount doesn't support those; any balance assignments will be con- + verted to explicit amounts. Beancount virtual postings Beancount doesn't allow virtual postings; if you have any, they will be omitted from beancount output. Beancount metadata - hledger tags will be converted to Beancount metadata (except for tags + hledger tags will be converted to Beancount metadata (except for tags whose name begins with _). Metadata names will be adjusted to be Bean- count-compatible: beginning with a lowercase letter, at least two char- - acters long, and with unsupported characters encoded. Metadata values + acters long, and with unsupported characters encoded. Metadata values will use Beancount's string type. - In hledger, objects can have the same tag repeated with multiple val- + In hledger, objects can have the same tag repeated with multiple val- ues. Eg an assets:cash account might have both type:Asset and - type:Cash tags. For Beancount these will be combined into one, with + type:Cash tags. For Beancount these will be combined into one, with the values combined, comma separated. Eg: type: "Asset, Cash". Beancount costs - Beancount doesn't allow redundant costs and conversion postings as - hledger does. If you have any of these, the conversion postings will - be omitted. Currently we support at most one cost + conversion post- + Beancount doesn't allow redundant costs and conversion postings as + hledger does. If you have any of these, the conversion postings will + be omitted. Currently we support at most one cost + conversion post- ings group per transaction. + Beancount tolerance + A sample inferred_tolerance_default option is provided (commented out). + If Beancount complains that transactions aren't balanced, this is an + easy way to work around it. + Beancount operating currency Declaring an operating currency (or several) improves Beancount and Fava reports. Currently hledger will declare each currency used in @@ -1596,58 +1617,160 @@ Journal below. Costs - After a posting amount, you can note its cost (when buying) or selling - price (when selling) in another commodity, by writing either @ UNIT- - PRICE or @@ TOTALPRICE after it. This indicates a conversion transac- - tion, where one commodity is exchanged for another. + In traditional double entry bookkeeping, to record a transaction where + one commodity is exchanged for another, you add extra equity postings + to balance the two commodities. Eg: + + 2026-01-01 buy euros + assets:dollars $-123 + equity:conversion $123 + equity:conversion -100 + assets:euros 100 + + hledger offers a more convenient @/@@ "cost notation" as an alterna- + tive: instead of equity postings, you can write the "conversion rate" + or "transacted price" after a posting amount. hledger docs generically + call this "cost", whether buying or selling. ("cost" is an overloaded + and generic term here, but we still use it for historical reasons. + "Transacted price" is more precise.) + + It can be written as either @ UNITPRICE or @@ TOTALPRICE. Eg you could + write the above as: + + 2026-01-01 buy euros + assets:dollars $-123 + assets:euros 100 @ $1.23 ; unit cost (exchange rate) + + or: + + 2026-01-01 buy euros + assets:dollars $-123 + assets:euros 100 @@ $123 ; total cost + + The cost should normally be a positive amount. Negative costs are sup- + ported, but can be confusing, as discussed at --infer-market-prices: + market prices from transactions. + + Costs participate in transaction balancing. Amounts are converted to + their cost before checking if the transaction is balanced. You could + also write the above less redundantly, like so: + + 2026-01-01 buy euros + assets:dollars ; $-123 is inferred + assets:euros 100 @ $1.23 + + or: + + 2026-01-01 buy euros + assets:dollars ; $-123 is inferred + assets:euros 100 @@ $123 + + or even: + + 2026-01-01 buy euros + assets:euros 100 ; @@ $123 is inferred + assets:dollars $-123 + + This last form works for transactions involving exactly two commodi- + ties, with neither cost notation nor equity postings. Note, the order + of postings is significant: the cost will be attached to the first + (top) posting. So we had to switch the order of postings, to get the + same meaning as above. Also, this form is the easiest to make unde- + tected errors with; so it is rejected by hledger check balanced, and by + strict mode. + + Advantages of cost notation: + + 1. it's more compact and easier to read and write + + 2. hledger reports can show such amounts converted to their cost, when + you add the -B/--cost flag (see Cost reporting). + + Advantages of equity postings + + 1. they help to keep the accounting equation balanced (if you care + about that) + + 2. they translate easily to any other double entry accounting system. + + Most hledger users use cost notation and don't use equity postings. + + But you can always convert cost notation to equity postings by adding + --infer-equity. Eg try hledger print -x --infer-equity. + + And you can usually convert equity postings to cost notation by adding + --infer-costs (see Requirements for detecting equity conversion post- + ings). Eg try hledger print -x --infer-costs. + + Finally: using both equity postings and cost notation at the same time + is allowed, as long as the journal entry is well formed such that the + equity postings / cost equivalences can be detected. (Otherwise you'll + get an error message saying that the transaction is unbalanced.): + + 2026-01-01 buy euros + assets:dollars $-123 + equity:conversion $123 + equity:conversion -100 + assets:euros 100 @ $1.23 + + So in principle you could enable both --infer-equity and --infer-costs + in your config file, and your reports would have the advantages of + both. + + Basis / lots + This is an advanced topic; skip over if you don't need it. For full + details, see Lot reporting. - (You might also see this called "transaction price" in hledger docs, - discussions, or code; that term was directionally neutral and reminded - that it is a price specific to a transaction, but we now just call it - "cost", with the understanding that the transaction could be a purchase - or a sale.) + If you are buying an amount of some commodity to hold as an investment, + it may be important to keep track of its cost basis (AKA "basis") and + lots. ("Cost basis" and "cost" (see above) sound similar, and often + have the same amount in simple purchases; but they are distinct con- + cepts.) - Costs are usually written explicitly with @ or @@, but can also be in- - ferred automatically for simple multi-commodity transactions. Note, if - costs are inferred, the order of postings is significant; the first - posting will have a cost attached, in the commodity of the second. + The basis records: - As an example, here are several ways to record purchases of a foreign - currency in hledger, using the cost notation either explicitly or im- - plicitly: + 1. the amount's nominal acquisition cost (usually the same as the + transacted cost, ie what you paid for it, but not always) - 1. Write the price per unit, as @ UNITPRICE after the amount: + 2. the nominal acquisition date (usually the date you acquired it, but + not always) - 2009/1/1 - assets:euros 100 @ $1.35 ; one hundred euros purchased at $1.35 each - assets:dollars ; balancing amount is -$135.00 + 3. and optionally a label, to disambiguate or identify lots. - 2. Write the total price, as @@ TOTALPRICE after the amount: + An amount with a basis is called a lot. The basis is a property of the + lot, remaining with it throughout its lifetime. It is used to calcu- + late capital gains/losses, eg for tax reporting. - 2009/1/1 - assets:euros 100 @@ $135 ; one hundred euros purchased at $135 for the lot - assets:dollars + Lot syntax + A basis or lot can be described using "lot syntax". hledger supports + two lot syntax styles: - 3. Specify amounts for all postings, using exactly two commodities, and - let hledger infer the price that balances the transaction. Note the - effect of posting order: the price is added to first posting, making - it 100 @@ $135, as in example 2: + hledger lot syntax puts all basis fields inside braces, comma-separated + (like Beancount): - 2009/1/1 - assets:euros 100 ; one hundred euros purchased - assets:dollars $-135 ; for $135 + 10 AAPL {2026-01-15, $50} + 10 AAPL {2026-01-15, "lot1", $50} + 10 AAPL {$50} + 10 AAPL {} - Amounts can be converted to cost at report time using the -B/--cost - flag; this is discussed more in the Cost reporting section. + All fields are optional, but when present they must be in date-la- + bel-cost order. Dates must be in YYYY-MM-DD format, and labels must be + double-quoted. The cost is a single-commodity costless hledger amount. - Note that the cost normally should be a positive amount, though it's - not required to be. This can be a little confusing, see discussion at - --infer-market-prices: market prices from transactions. + When an amount has both a basis and a transacted price, like -10 AAPL + {$50} @ $60, the preferred order is to write {} before @. + + Ledger lot syntax uses separate annotations, in any order: + + 10 AAPL {$50} [2026-01-15] (lot1) + + hledger accepts both styles on input. Print output uses consolidated + style by default, or Ledger style with -O ledger. Balance assertions - hledger supports Ledger-style balance assertions in journal files. - These look like, for example, = EXPECTEDBALANCE following a posting's - amount. Eg here we assert the expected dollar balance in accounts a + hledger supports Ledger-style balance assertions in journal files. + These look like, for example, = EXPECTEDBALANCE following a posting's + amount. Eg here we assert the expected dollar balance in accounts a and b after each posting: 2013/1/1 @@ -1659,22 +1782,22 @@ Journal b $-1 = $-2 After reading a journal file, hledger will check all balance assertions - and report an error if any of them fail. Balance assertions can pro- - tect you from, eg, inadvertently disrupting reconciled balances while - cleaning up old entries. You can disable them temporarily with the + and report an error if any of them fail. Balance assertions can pro- + tect you from, eg, inadvertently disrupting reconciled balances while + cleaning up old entries. You can disable them temporarily with the -I/--ignore-assertions flag, which can be useful for troubleshooting or - for reading Ledger files. (Note: this flag currently does not disable + for reading Ledger files. (Note: this flag currently does not disable balance assignments, described below). Assertions and ordering - hledger calculates and checks an account's balance assertions in date + hledger calculates and checks an account's balance assertions in date order (and when there are multiple assertions on the same day, in parse - order). Note this is different from Ledger, which checks assertions + order). Note this is different from Ledger, which checks assertions always in parse order, ignoring dates. This means in hledger you can freely reorder transactions, postings, or files, and balance assertions will usually keep working. The exception - is when you reorder multiple postings on the same day, to the same ac- + is when you reorder multiple postings on the same day, to the same ac- count, which have balance assertions; those will likely need updating. Assertions and multiple files @@ -1682,20 +1805,20 @@ Journal sertions can still work - but only if those files are part of a hierar- chy made by include directives. - If the same files are specified with two -f options on the command - line, the assertions in the second will not see the balances from the + If the same files are specified with two -f options on the command + line, the assertions in the second will not see the balances from the first. - To work around this, arrange your files in a hierarchy with include. - Or, you could concatenate the files temporarily, and process them like + To work around this, arrange your files in a hierarchy with include. + Or, you could concatenate the files temporarily, and process them like one big file. - Why does it work this way ? It might be related to hledger's goal of - stable predictable reports. File hierarchy is considered "permanent", + Why does it work this way ? It might be related to hledger's goal of + stable predictable reports. File hierarchy is considered "permanent", part of your data, while the order of command line options/arguments is - not. We don't want transient changes to be able to change the meaning - of the data. Eg it would be frustrating if tomorrow all your balance - assertions broke because you wrote command line arguments in a differ- + not. We don't want transient changes to be able to change the meaning + of the data. Eg it would be frustrating if tomorrow all your balance + assertions broke because you wrote command line arguments in a differ- ent order. (Discussion welcome.) Assertions and costs @@ -1705,20 +1828,20 @@ Journal 2019/1/1 (a) $1 @ 1 = $1 - We do allow costs to be written in balance assertion amounts, however, - and print shows them, but they don't affect whether the assertion - passes or fails. This is for backward compatibility (hledger's close - command used to generate balance assertions with costs), and because + We do allow costs to be written in balance assertion amounts, however, + and print shows them, but they don't affect whether the assertion + passes or fails. This is for backward compatibility (hledger's close + command used to generate balance assertions with costs), and because balance assignments do use costs (see below). Assertions and commodities - The balance assertions described so far are "single commodity balance + The balance assertions described so far are "single commodity balance assertions": they assert and check the balance in one commodity, ignor- - ing any others that may be present. This is how balance assertions + ing any others that may be present. This is how balance assertions work in Ledger also. - If an account contains multiple commodities, you can assert their bal- - ances by writing multiple postings with balance assertions, one for + If an account contains multiple commodities, you can assert their bal- + ances by writing multiple postings with balance assertions, one for each commodity: 2013/1/1 @@ -1730,8 +1853,8 @@ Journal both 0 = $1 both 0 = 1 - In hledger you can make a stronger "sole commodity balance assertion" - by writing two equals signs (== EXPECTEDBALANCE). This also asserts + In hledger you can make a stronger "sole commodity balance assertion" + by writing two equals signs (== EXPECTEDBALANCE). This also asserts that there are no other commodities in the account besides the asserted one (or at least, that their current balance is zero): @@ -1741,12 +1864,12 @@ Journal both ;== $1 ; this one would fail because 'both' contains $ and It's less easy to make a "sole commodities balance assertion" (note the - plural) - ie, asserting that an account contains two or more specified + plural) - ie, asserting that an account contains two or more specified commodities and no others. It can be done by 1. isolating each commodity in a subaccount, and asserting those - 2. and also asserting there are no commodities in the parent account + 2. and also asserting there are no commodities in the parent account itself: 2013/1/1 @@ -1758,10 +1881,10 @@ Journal Assertions and subaccounts All of the balance assertions above (both = and ==) are "subaccount-ex- - clusive balance assertions"; they ignore any balances that exist in + clusive balance assertions"; they ignore any balances that exist in deeper subaccounts. - In hledger you can make "subaccount-inclusive balance assertions" by + In hledger you can make "subaccount-inclusive balance assertions" by adding a star after the equals (=* or ==*): 2019/1/1 @@ -1771,8 +1894,8 @@ Journal assets $0 ==* $20 ; assets + subaccounts contains $20 and nothing else Assertions and status - Balance assertions always consider postings of all statuses (unmarked, - pending, or cleared); they are not affected by the -U/--unmarked / + Balance assertions always consider postings of all statuses (unmarked, + pending, or cleared); they are not affected by the -U/--unmarked / -P/--pending / -C/--cleared flags or the status: query. Assertions and virtual postings @@ -1780,10 +1903,10 @@ Journal are not affected by the --real/-R flag or real: query. Assertions and auto postings - Balance assertions are affected by the --auto flag, which generates + Balance assertions are affected by the --auto flag, which generates auto postings, which can alter account balances. Because auto postings are optional in hledger, accounts affected by them effectively have two - balances. But balance assertions can only test one or the other of + balances. But balance assertions can only test one or the other of these. So to avoid making fragile assertions, either: o assert the balance calculated with --auto, and always use --auto with @@ -1796,19 +1919,19 @@ Journal avoid auto postings entirely). Assertions and precision - Balance assertions compare the exactly calculated amounts, which are - not always what is shown by reports. Eg a commodity directive may - limit the display precision, but this will not affect balance asser- + Balance assertions compare the exactly calculated amounts, which are + not always what is shown by reports. Eg a commodity directive may + limit the display precision, but this will not affect balance asser- tions. Balance assertion failure messages show exact amounts. Assertions and hledger add - Balance assertions can be included in the amounts given in add. All - types of assertions are supported, and assertions can be used as in a + Balance assertions can be included in the amounts given in add. All + types of assertions are supported, and assertions can be used as in a normal journal file. - All transactions, not just those that have an explicit assertion, are - validated against the existing assertions in the journal. This means - it is possible for an added transaction to fail even if its assertions + All transactions, not just those that have an explicit assertion, are + validated against the existing assertions in the journal. This means + it is possible for an added transaction to fail even if its assertions are correct as of the transaction date. If this assertion checking is not desired, then it can be disabled with @@ -1817,9 +1940,9 @@ Journal However, balance assignments are currently not supported. Posting comments - Text following ;, at the end of a posting line, and/or on indented - lines immediately below it, form comments for that posting. They are - reproduced by print but otherwise ignored, except they may contain + Text following ;, at the end of a posting line, and/or on indented + lines immediately below it, form comments for that posting. They are + reproduced by print but otherwise ignored, except they may contain tags, which are not ignored. 2012-01-01 @@ -1830,59 +1953,59 @@ Journal Transaction balancing How exactly does hledger decide when a transaction is balanced ? Espe- - cially when it involves costs, which often are not exact, because of + cially when it involves costs, which often are not exact, because of repeating decimals, or imperfect data from financial institutions ? In - each commodity, hledger sums the transaction's posting amounts, after - converting any with costs; then it checks if that sum is zero, when + each commodity, hledger sums the transaction's posting amounts, after + converting any with costs; then it checks if that sum is zero, when rounded to a suitable number of decimal digits - which we call the bal- ancing precision. Since version 1.50, hledger infers balancing precision in each transac- - tion from the amounts in that transaction's journal entry (like - Ledger). Ie, when checking the balance of commodity A, it uses the - highest decimal precision seen for A in the journal entry (excluding + tion from the amounts in that transaction's journal entry (like + Ledger). Ie, when checking the balance of commodity A, it uses the + highest decimal precision seen for A in the journal entry (excluding cost amounts). This makes transaction balancing robust; any imbalances - must be visibly accounted for in the journal entry, display precision - can be freely increased with -c, and compatibility with Ledger and + must be visibly accounted for in the journal entry, display precision + can be freely increased with -c, and compatibility with Ledger and Beancount journals is good. Note that hledger versions before 1.50 worked differently: they allowed - display precision to override the balancing precision. This masked - small imbalances and caused fragility (see issue #2402). As a result, + display precision to override the balancing precision. This masked + small imbalances and caused fragility (see issue #2402). As a result, some journal entries (or CSV rules) that worked with hledger <1.50, are - now rejected with an "unbalanced transaction" error. If you hit this + now rejected with an "unbalanced transaction" error. If you hit this problem, it's easy to fix: - o You can restore the old behaviour, by adding --txn-balancing=old to - the command or to your ~/.hledger.conf file. This lets you keep us- + o You can restore the old behaviour, by adding --txn-balancing=old to + the command or to your ~/.hledger.conf file. This lets you keep us- ing old journals unchanged, though without the above benefits. - o Or you can fix the problem entries (recommended). There are three + o Or you can fix the problem entries (recommended). There are three ways, use whichever seems best: 1. make cost amounts more precise (add more/better decimal digits) - 2. or make non-cost amounts less precise (remove unnecessary decimal + 2. or make non-cost amounts less precise (remove unnecessary decimal digits that are raising the precision) - 3. or add a posting to absorb the imbalance (eg "expenses:rounding". - Remember that one posting may omit the amount; that's convenient + 3. or add a posting to absorb the imbalance (eg "expenses:rounding". + Remember that one posting may omit the amount; that's convenient here.) Tags - Tags are a way to add extra labels or data fields to transactions, - postings, or accounts, which you can match with a tag: query in re- + Tags are a way to add extra labels or data fields to transactions, + postings, or accounts, which you can match with a tag: query in re- ports. (See queries below.) - Tags are a single word or hyphenated word, immediately followed by a - full colon, written within a comment. (Yes, storing data in comments + Tags are a single word or hyphenated word, immediately followed by a + full colon, written within a comment. (Yes, storing data in comments is slightly weird.) Here's a transaction with a tag: 2025-01-01 groceries ; some-tag: assets:checking expenses:food $1 - A tag can have a value, a single line of text written after the colon. + A tag can have a value, a single line of text written after the colon. Tag values can't contain newlines.: 2025-01-01 groceries ; tag1: this is tag1's value @@ -1896,14 +2019,14 @@ Journal 2025-01-01 groceries ; tag1:value 1, tag1:value 2 - You can write each tag on its own line of you prefer (but they still + You can write each tag on its own line of you prefer (but they still can't contain commas): 2025-01-01 groceries ; tag1: value 1 ; tag2: value 2 - Tags can be attached to individual postings, rather than the overall + Tags can be attached to individual postings, rather than the overall transaction: 2025-01-01 rent @@ -1923,10 +2046,10 @@ Journal 2. Transactions -> postings. Postings inherit tags from their transac- tion. - 3. Postings -> transactions. Transactions also acquire the tags of + 3. Postings -> transactions. Transactions also acquire the tags of their postings. - So when you use a tag: query to match whole transactions, individual + So when you use a tag: query to match whole transactions, individual postings, or accounts, it's good to understand how tags behave. Here's an example showing all three kinds of propagation: @@ -1943,9 +2066,9 @@ Journal ing account expenses:food atag atag: in comment account - assets:check- p1tag, ttag p1tag: in comment, ttag acquired from + assets:check- p1tag, ttag p1tag: in comment, ttag acquired from ing posting transaction - expenses:food p2tag, atag, p2tag: in comment, atag from account, ttag + expenses:food p2tag, atag, p2tag: in comment, atag from account, ttag posting ttag from transaction groceries ttag, p1tag, ttag: in comment, p1tag from first posting, transaction p2tag, atag p2tag and atag from second posting @@ -1955,27 +2078,27 @@ Journal The print command also shows tags. - You can use --pivot to display tag values in other reports, in various + You can use --pivot to display tag values in other reports, in various ways (eg appended to account names, like pseudo subaccounts). When to use tags ? - Tags provide more dimensions of categorisation, complementing accounts - and transaction descriptions. When to use each of these is somewhat a - matter of taste. Accounts have the most built-in support, and regex - queries on descriptions are also quite powerful. So you may not need - tags at all. But if you want to track multiple cross-cutting cate- - gories, they can be a good fit. For example, you could tag trip-re- + Tags provide more dimensions of categorisation, complementing accounts + and transaction descriptions. When to use each of these is somewhat a + matter of taste. Accounts have the most built-in support, and regex + queries on descriptions are also quite powerful. So you may not need + tags at all. But if you want to track multiple cross-cutting cate- + gories, they can be a good fit. For example, you could tag trip-re- lated transactions with trip: YEAR:PLACE, without disturbing your usual account categories. Tag names - What is allowed in a tag name ? Most non-whitespace characters. Eg : + What is allowed in a tag name ? Most non-whitespace characters. Eg : is a valid tag. - For extra error checking, you can declare valid tag names with the tag - directive, and then enforce these with the check command. But note - that tags are detected quite loosely at present, sometimes where you - didn't intend them. Eg a comment like ; see https://foo.com adds a + For extra error checking, you can declare valid tag names with the tag + directive, and then enforce these with the check command. But note + that tags are detected quite loosely at present, sometimes where you + didn't intend them. Eg a comment like ; see https://foo.com adds a https tag. There are several tag names which have special significance to hledger. @@ -1998,18 +2121,18 @@ Journal and which have an equivalent cost posting in the transaction The second group above (generated-transaction, etc.) are normally hid- - den, with a _ prefix added. This means print doesn't show them by de- - fault; but you can still use them in queries. You can add the --ver- - bose-tags flag to make them visible, which can be useful for trou- - bleshooting. + den, with a _ prefix added. This means print doesn't show them by de- + fault; but you can still use them in queries. You can add the --ver- + bose-tags flag to make them visible in print output, which can be use- + ful for troubleshooting. Directives - Besides transactions, there is something else you can put in a journal - file: directives. These are declarations, beginning with a keyword, - that modify hledger's behaviour. Some directives can have more spe- - cific subdirectives, indented below them. hledger's directives are + Besides transactions, there is something else you can put in a journal + file: directives. These are declarations, beginning with a keyword, + that modify hledger's behaviour. Some directives can have more spe- + cific subdirectives, indented below them. hledger's directives are similar to Ledger's in many cases, but there are also many differences. - Directives are not required, but can be useful. Here are the main di- + Directives are not required, but can be useful. Here are the main di- rectives: purpose directive @@ -2017,16 +2140,16 @@ Journal READING DATA: Rewrite account names alias Comment out sections of the file comment - Declare file's decimal mark, to help decimal-mark + Declare file's decimal mark, to help decimal-mark parse amounts accurately Include other data files include GENERATING DATA: - Generate recurring transactions or bud- ~ + Generate recurring transactions or bud- ~ get goals - Generate extra postings on existing = + Generate extra postings on existing = transactions CHECKING FOR ERRORS: - Define valid entities to provide more account, commodity, payee, tag + Define valid entities to provide more account, commodity, payee, tag error checking REPORTING: Declare accounts' type and display order account @@ -2034,23 +2157,23 @@ Journal Declare market prices P Directives and multiple files - Directives vary in their scope, ie which journal entries and which in- + Directives vary in their scope, ie which journal entries and which in- put files they affect. Most often, a directive will affect the follow- - ing entries and included files if any, until the end of the current + ing entries and included files if any, until the end of the current file - and no further. You might find this inconvenient! For example, - alias directives do not affect parent or sibling files. But there are + alias directives do not affect parent or sibling files. But there are usually workarounds; for example, put alias directives in your top-most file, before including other files. - The restriction, though it may be annoying at first, is in a good + The restriction, though it may be annoying at first, is in a good cause; it allows reports to be stable and deterministic, independent of - the order of input. Without it, reports could show different numbers - depending on the order of -f options, or the positions of include di- + the order of input. Without it, reports could show different numbers + depending on the order of -f options, or the positions of include di- rectives in your files. Directive effects - Here are all hledger's directives, with their effects and scope sum- - marised - nine main directives, plus four others which we consider + Here are all hledger's directives, with their effects and scope sum- + marised - nine main directives, plus four others which we consider non-essential: di- what it does ends @@ -2058,53 +2181,53 @@ Journal tive file end? -------------------------------------------------------------------------------------- - ac- Declares an account, for checking all entries in all files; and N + ac- Declares an account, for checking all entries in all files; and N count its display order and type. Subdirectives: any text, ignored. - alias Rewrites account names, in following entries until end of cur- Y + alias Rewrites account names, in following entries until end of cur- Y rent file or end aliases. Command line equivalent: --alias - com- Ignores part of the journal file, until end of current file or Y + com- Ignores part of the journal file, until end of current file or Y ment end comment. com- Declares up to four things: 1. a commodity symbol, for checking N,N,Y,Y - mod- all amounts in all files 2. the display style for all amounts - ity of this commodity 3. the decimal mark for parsing amounts of - this commodity, in the rest of this file and its children, if - there is no decimal-mark directive 4. the precision to use for - balanced-transaction checking in this commodity, in this file - and its children. Takes precedence over D. Subdirectives: + mod- all amounts in all files 2. the display style for all amounts + ity of this commodity 3. the decimal mark for parsing amounts of + this commodity, in the rest of this file and its children, if + there is no decimal-mark directive 4. the precision to use for + balanced-transaction checking in this commodity, in this file + and its children. Takes precedence over D. Subdirectives: format (ignored). Command line equivalent: -c/--commodity-style - deci- Declares the decimal mark, for parsing amounts of all commodi- Y + deci- Declares the decimal mark, for parsing amounts of all commodi- Y mal-mark ties in following entries until next decimal-mark or end of cur- - rent file. Included files can override. Takes precedence over + rent file. Included files can override. Takes precedence over commodity and D. - include Includes entries and directives from another file, as if they N - were written inline. Command line alternative: multiple + include Includes entries and directives from another file, as if they N + were written inline. Command line alternative: multiple -f/--file payee Declares a payee name, for checking all entries in all files. N P Declares the market price of a commodity on some date, for value N reports. - ~ Declares a periodic transaction rule that generates future N - (tilde) transactions with --forecast and budget goals with balance + ~ Declares a periodic transaction rule that generates future N + (tilde) transactions with --forecast and budget goals with balance --budget. Other syntax: - apply Prepends a common parent account to all account names, in fol- Y + apply Prepends a common parent account to all account names, in fol- Y account lowing entries until end of current file or end apply account. - D Sets a default commodity to use for no-symbol amounts;and, if Y,Y,N,N - there is no commodity directive for this commodity: its decimal + D Sets a default commodity to use for no-symbol amounts;and, if Y,Y,N,N + there is no commodity directive for this commodity: its decimal mark, balancing precision, and display style, as above. - Y Sets a default year to use for any yearless dates, in following Y + Y Sets a default year to use for any yearless dates, in following Y entries until end of current file. - = Declares an auto posting rule that generates extra postings on partly - (equals) matched transactions with --auto, in current, parent, and child + = Declares an auto posting rule that generates extra postings on partly + (equals) matched transactions with --auto, in current, parent, and child files (but not sibling files, see #1212). - Other Other directives from Ledger's file format are accepted but ig- + Other Other directives from Ledger's file format are accepted but ig- Ledger nored. direc- tives account directive account directives can be used to declare accounts (ie, the places that - amounts are transferred from and to). Though not required, these dec- + amounts are transferred from and to). Though not required, these dec- larations can provide several benefits: o They can document your intended chart of accounts, providing a refer- @@ -2116,17 +2239,17 @@ Journal o They can restrict which accounts may be posted to by transactions, eg in strict mode, which helps prevent errors. - o They influence account display order in reports, allowing non-alpha- + o They influence account display order in reports, allowing non-alpha- betic sorting (eg Revenues to appear above Expenses). - o They can help hledger know your accounts' types (asset, liability, + o They can help hledger know your accounts' types (asset, liability, equity, revenue, expense), enabling reports like balancesheet and in- comestatement. - o They help with account name completion (in hledger add, hledger-web, + o They help with account name completion (in hledger add, hledger-web, hledger-iadd, ledger-mode, etc.) - They are written as the word account followed by a hledger-style ac- + They are written as the word account followed by a hledger-style ac- count name. Eg: account assets:bank:checking @@ -2138,17 +2261,23 @@ Journal Account comments Text following two or more spaces and ; at the end of an account direc- - tive line, and/or following ; on indented lines immediately below it, - form comments for that account. They are ignored except they may con- - tain tags, which are not ignored. + tive line, and/or following ; on indented lines immediately below it, + form comments for that account. - The two-space requirement for same-line account comments is because ; - is allowed in account names. + Same-line account comments require two+ spaces before ; because that + character can appear in account names. account assets:bank:checking ; same-line comment, at least 2 spaces before the semicolon ; next-line comment ; some tags - type:A, acctnum:12345 + Account tags + An account directive's comment may contain tags. These will be inher- + ited by all postings using that account, except where the posting al- + ready has a value for that tag. (A posting tag overrides an account + tag.) Note, these tags will be queryable but won't be shown in print + output, even with --verbose-tags. + Account error checking By default, accounts need not be declared; they come into existence when a posting references them. This is convenient, but it means @@ -2176,12 +2305,16 @@ Journal o It's currently not possible to declare "all possible subaccounts" with a wildcard; every account posted to must be declared. - o If you use the --infer-equity flag, you will also need declarations + o As an exception: lot subaccounts (a final account name component like + :{2026-01-15, $50}) are always ignored by check accounts, and need + not be declared. + + o If you use the --infer-equity flag, you will also need declarations for the account names it generates. Account display order Account directives also cause hledger to display accounts in a particu- - lar order, not just alphabetically. Eg, here is a conventional order- + lar order, not just alphabetically. Eg, here is a conventional order- ing for the top-level accounts: account assets @@ -2199,21 +2332,21 @@ Journal revenues expenses - If there are undeclared accounts, those will be displayed last, in al- + If there are undeclared accounts, those will be displayed last, in al- phabetical order. Sorting is done within each group of sibling accounts, at each level of - the account tree. Eg, a declaration like account parent:child influ- + the account tree. Eg, a declaration like account parent:child influ- ences child's position among its siblings. - Note, it does not affect parent's position; for that, you need an ac- + Note, it does not affect parent's position; for that, you need an ac- count parent declaration. - Sibling accounts are always displayed together; hledger won't display + Sibling accounts are always displayed together; hledger won't display x:y in between a:b and a:c. - An account directive both declares an account as a valid posting tar- - get, and declares its display order; you can't easily do one without + An account directive both declares an account as a valid posting tar- + get, and declares its display order; you can't easily do one without the other. Account types @@ -2222,7 +2355,7 @@ Journal Asset A things you own Liability L things you owe Equity E owner's investment, - balances the two + balances the two above and two more representing changes in these: @@ -2231,16 +2364,20 @@ Journal as Income) Expense X outflows - hledger also uses a couple of subtypes: + hledger also uses a few subtypes: - Cash C liquid assets - Conversion V commodity conver- - sions equity + Cash C liquid assets (subtype + of Asset) + Conversion V commodity conversions + equity (subtype of Eq- + uity) + Gain G capital gains/losses + (subtype of Revenue) - As a convenience, hledger will detect these types automatically from - english account names. But it's better to declare them explicitly by - adding a type: tag in the account directives. The tag's value can be - any of the types or one-letter abbreviations above. + As a convenience, hledger will detect most of these types automatically + from english account names. But it's better to declare them explicitly + by adding a type: tag in the account directives. The tag's value can + be any of the types or one-letter abbreviations above. Here is a typical set of account type declarations. Subaccounts will inherit their parent's type, or can override it: @@ -2256,6 +2393,8 @@ Journal account equity:conversion ; type: V + account revenues:capital ; type: G + This enables the easy balancesheet, balancesheetequity, cashflow and incomestatement reports, and querying by type:. @@ -2524,28 +2663,26 @@ Journal Commodity directive syntax A commodity directive is normally the word commodity followed by a sam- - ple amount (and optionally a comment). Only the amount's symbol and - the number's format is significant. Eg: + ple amount, and optionally a comment. Only the amount's symbol and the + number's format is significant. Eg: commodity $1000.00 commodity 1.000,00 EUR commodity 1 000 000.0000 ; the no-symbol commodity - Commodities do not have tags (tags in the comment will be ignored). - - A commodity directive's sample amount must always include a period or - comma decimal mark (this rule helps disambiguate decimal marks and - digit group marks). If you don't want to show any decimal digits, + A commodity directive's sample amount must always include a period or + comma decimal mark (this rule helps disambiguate decimal marks and + digit group marks). If you don't want to show any decimal digits, write the decimal mark at the end: commodity 1000. AAAA ; show AAAA with no decimals - Commodity symbols containing spaces, numbers, or punctuation must be + Commodity symbols containing spaces, numbers, or punctuation must be enclosed in double quotes, as usual: commodity 1.0000 "AAAA 2023" - Commodity directives normally include a sample amount, but can declare + Commodity directives normally include a sample amount, but can declare only a symbol (ie, just function 1 above): commodity $ @@ -2554,7 +2691,7 @@ Journal commodity "" ; the no-symbol commodity Commodity directives may also be written with an indented format subdi- - rective, as in Ledger. The symbol is repeated and must be the same in + rective, as in Ledger. The symbol is repeated and must be the same in both places. Other subdirectives are currently ignored: ; display indian rupees with currency name on the left, @@ -2564,6 +2701,13 @@ Journal format INR 1,00,00,000.00 an unsupported subdirective ; ignored by hledger + Commodity tags + A commodity directive's comment may contain tags. These will be inher- + ited by all postings using that commodity in their main amount, except + where the posting already has a value for that tag. (A posting tag or + an account tag overrides a commodity tag.) Note, these tags will be + queryable but won't be shown in print output, even with --verbose-tags. + Commodity error checking In strict mode (-s/--strict) (or when you run hledger check commodi- ties), hledger will report an error if an undeclared commodity symbol @@ -3200,57 +3344,20 @@ Journal See also https://hledger.org/ledger.html for a detailed hledger/Ledger syntax comparison. - Other cost/lot notations - A slight digression for Ledger and Beancount users. - - Ledger has a number of cost/lot-related notations: - - o @ UNITCOST and @@ TOTALCOST - - o expresses a conversion rate, as in hledger - - o when buying, also creates a lot that can be selected at selling - time - - o (@) UNITCOST and (@@) TOTALCOST (virtual cost) - - o like the above, but also means "this cost was exceptional, don't - use it when inferring market prices". - - o {=UNITCOST} and {{=TOTALCOST}} (fixed price) - - o when buying, means "this cost is also the fixed value, don't let it - fluctuate in value reports" + Ledger virtual costs + In Ledger, (@) UNITCOST and (@@) TOTALCOST are virtual costs, which do + not generate market prices. In hledger, these are equivalent to @ and + @@. - o {UNITCOST} and {{TOTALCOST}} (lot price) + Ledger lot syntax + In Ledger, these annotations after an amount help specify or select a + lot's cost basis: {LOTUNITCOST} or {{LOTTOTALCOST}}, [LOTDATE], and/or + (LOTNOTE). hledger will read these, as an alternative to its own lot + syntax). - o can be used identically to @ UNITCOST and @@ TOTALCOST, also cre- - ates a lot - - o when selling, combined with @ ..., selects an existing lot by its - cost basis. Does not check if that lot is present. - - o [YYYY/MM/DD] (lot date) - - o when buying, attaches this acquisition date to the lot - - o when selling, selects a lot by its acquisition date - - o (SOME TEXT) (lot note) - - o when buying, attaches this note to the lot - - o when selling, selects a lot by its note - - Currently, hledger - - o accepts any or all of the above in any order after the posting amount - - o supports @ and @@ - - o treats (@) and (@@) as synonyms for @ and @@ - - o and ignores the rest. (This can break transaction balancing.) + We also read Ledger's fixed price) syntax, {=LOTUNITCOST} or {{=LOTTO- + TALCOST}}, treating it as equivalent to {LOTUNITCOST} or {{LOTTOTAL- + COST}}, Beancount has simpler notation and different behaviour: @@ -3258,8 +3365,8 @@ Journal o expresses a cost without creating a lot, as in hledger - o when buying (acquiring) or selling (disposing of) a lot, and com- - bined with {...}: is not used except to document the cost/selling + o when buying (acquiring) or selling (disposing of) a lot, and com- + bined with {...}: is not used except to document the cost/selling price o {UNITCOST} and {{TOTALCOST}} @@ -3276,10 +3383,10 @@ Journal o expresses the selling price for transaction balancing - o {}, {YYYY-MM-DD}, {"LABEL"}, {UNITCOST, "LABEL"}, {UNITCOST, + o {}, {YYYY-MM-DD}, {"LABEL"}, {UNITCOST, "LABEL"}, {UNITCOST, YYYY-MM-DD, "LABEL"} - o when selling, other combinations of date/cost/label, like the + o when selling, other combinations of date/cost/label, like the above, are accepted for selecting the lot. Currently, hledger @@ -3291,28 +3398,28 @@ Journal o and rejects the rest. CSV - hledger can read transactions from CSV (comma-separated values) files. - More precisely, it can read DSV (delimiter-separated values), from a + hledger can read transactions from CSV (comma-separated values) files. + More precisely, it can read DSV (delimiter-separated values), from a file or standard input. Comma-separated, semicolon-separated and - tab-separated are the most common variants, and hledger will recognise - these three automatically based on a .csv, .ssv or .tsv file name ex- + tab-separated are the most common variants, and hledger will recognise + these three automatically based on a .csv, .ssv or .tsv file name ex- tension or a csv:, ssv: or tsv: file path prefix. (To learn about producing CSV or TSV output, see Output format.) - Each CSV file must be described by a corresponding rules file. This - contains rules describing the CSV data (header line, fields layout, - date format etc.), how to construct hledger transactions from it, and - how to categorise transactions based on description or other attrib- + Each CSV file must be described by a corresponding rules file. This + contains rules describing the CSV data (header line, fields layout, + date format etc.), how to construct hledger transactions from it, and + how to categorise transactions based on description or other attrib- utes. - By default, hledger expects this rules file to be named like the CSV - file, with an extra .rules extension added, in the same directory. Eg - when asked to read foo/FILE.csv, hledger looks for foo/FILE.csv.rules. + By default, hledger expects this rules file to be named like the CSV + file, with an extra .rules extension added, in the same directory. Eg + when asked to read foo/FILE.csv, hledger looks for foo/FILE.csv.rules. You can specify a different rules file with the --rules option. - At minimum, the rules file must identify the date and amount fields, - and often it also specifies the date format and how many header lines + At minimum, the rules file must identify the date and amount fields, + and often it also specifies the date format and how many header lines there are. Here's a simple CSV file and a rules file for it: Date, Description, Id, Amount @@ -3328,35 +3435,35 @@ CSV expenses:unknown 10.23 income:unknown -10.23 - There's an introductory Tutorial: Import CSV data on hledger.org, and - more CSV rules examples below, and a larger collection at + There's an introductory Tutorial: Import CSV data on hledger.org, and + more CSV rules examples below, and a larger collection at https://github.com/simonmichael/hledger/tree/master/examples/csv. CSV rules cheatsheet The following kinds of rule can appear in the rules file, in any order. (Blank lines and lines beginning with # or ; or * are ignored.) - source optionally declare which file to read data + source optionally declare which file to read data from archive optionally enable an archive of imported files - encoding optionally declare which text encoding the + encoding optionally declare which text encoding the data has - separator declare the field separator, instead of rely- + separator declare the field separator, instead of rely- ing on file extension - decimal-mark declare the decimal mark used in CSV amounts, + decimal-mark declare the decimal mark used in CSV amounts, when ambiguous date-format declare how to parse CSV dates/date-times - timezone declare the time zone of ambiguous CSV + timezone declare the time zone of ambiguous CSV date-times - newest-first improve txn order when: there are multiple + newest-first improve txn order when: there are multiple records, newest first, all with the same date - intra-day-reversed improve txn order when: same-day txns are in + intra-day-reversed improve txn order when: same-day txns are in opposite order to the overall file skip (at top level) skip header line(s) at start of file - fields list name CSV fields for easy reference, and op- + fields list name CSV fields for easy reference, and op- tionally assign their values to hledger fields - Field assignment assign a CSV value or interpolated text value + Field assignment assign a CSV value or interpolated text value to a hledger field if block conditionally assign values to hledger fields, or skip a record or end (skip rest of file) @@ -3364,30 +3471,30 @@ CSV using compact syntax skip (inside an if rule) skip current record(s) end (inside an if rule) skip all remaining records - balance-type select which type of balance assertions/as- + balance-type select which type of balance assertions/as- signments to generate include inline another CSV rules file - Working with CSV tips can be found below, including How CSV rules are + Working with CSV tips can be found below, including How CSV rules are evaluated. source - If you tell hledger to read a csv file with -f foo.csv, it will look - for rules in foo.csv.rules. Or, you can tell it to read the rules - file, with -f foo.csv.rules, and it will look for data in foo.csv - (since 1.30). These are mostly equivalent, but the second method pro- - vides some extra features. For one, the data file can be missing, + If you tell hledger to read a csv file with -f foo.csv, it will look + for rules in foo.csv.rules. Or, you can tell it to read the rules + file, with -f foo.csv.rules, and it will look for data in foo.csv + (since 1.30). These are mostly equivalent, but the second method pro- + vides some extra features. For one, the data file can be missing, without causing an error; it is just considered empty. - For more flexibility, add a source rule, which lets you specify a dif- + For more flexibility, add a source rule, which lets you specify a dif- ferent data file: source ./Checking1.csv - If the file does not exist, it is just considered empty, without rais- + If the file does not exist, it is just considered empty, without rais- ing an error. - If you specify just a file name with no path, hledger will look for it + If you specify just a file name with no path, hledger will look for it in the ~/Downloads folder: source Checking1.csv @@ -3396,22 +3503,22 @@ CSV source Checking1*.csv - This has another benefit: if the pattern matches multiple files, + This has another benefit: if the pattern matches multiple files, hledger will read the newest (most recently modified) one. This avoids - problems if you have downloaded a file multiple times without cleaning + problems if you have downloaded a file multiple times without cleaning up. - All this enables a convenient workflow where can you just download CSV + All this enables a convenient workflow where can you just download CSV files, then run hledger import rules/*. See also "Working with CSV > Reading files specified by rule". Data cleaning / data generating commands After source's file pattern, you can write | (pipe) and a data cleaning - command (or command pipeline). If hledger's CSV rules aren't enough, - you can pre-process the downloaded data here with a shell command or - script, to make it more suitable for conversion. The command will be - executed by your default shell, in the directory of the rules file, + command (or command pipeline). If hledger's CSV rules aren't enough, + you can pre-process the downloaded data here with a shell command or + script, to make it more suitable for conversion. The command will be + executed by your default shell, in the directory of the rules file, will receive the data file's content as standard input, and should out- put zero or more lines of character-separated-values, suitable for con- version by the CSV rules. @@ -3425,7 +3532,7 @@ CSV Or, after source you can write | and a data generating command (with no file pattern before the |). This command receives no input, and should - output zero or more lines of character-separated values, suitable for + output zero or more lines of character-separated values, suitable for conversion by the CSV rules. Examples: @@ -3437,27 +3544,27 @@ CSV (paypal* and simplefin* scripts are in bin/) - Whenever hledger runs one of these commands, it will echo the command - on stderr. If the command produces error output, but exits success- + Whenever hledger runs one of these commands, it will echo the command + on stderr. If the command produces error output, but exits success- fully, hledger will show the error output as a warning. If the command - fails, hledger will fail and show the error output in the error mes- + fails, hledger will fail and show the error output in the error mes- sage. Added in 1.50; experimental. archive - With archive added to a rules file, the import command will archive - each successfully processed data file or data command output in a - nearby data/ directory. The archive file name will be based on the - rules file and the data file's modification date and extension (or for - a data-generating command, the current date and the ".csv" extension). + With archive added to a rules file, the import command will archive + each successfully processed data file or data command output in a + nearby data/ directory. The archive file name will be based on the + rules file and the data file's modification date and extension (or for + a data-generating command, the current date and the ".csv" extension). The original data file, if any, will be removed. - Also, in this mode import will prefer the oldest file matched by the - source rule's glob pattern, not the newest. (So if there are multiple + Also, in this mode import will prefer the oldest file matched by the + source rule's glob pattern, not the newest. (So if there are multiple downloads, they will be imported and archived oldest first.) - Archiving is optional, but it can be useful for troubleshooting your + Archiving is optional, but it can be useful for troubleshooting your CSV rules, regenerating entries with improved rules, checking for vari- ations in your bank's CSV, etc. @@ -3468,26 +3575,26 @@ CSV hledger normally expects non-ascii text to be using the system locale's text encoding. If you need to read CSV files which have some other en- - coding, you can do it by adding encoding ENCODING to your CSV rules. + coding, you can do it by adding encoding ENCODING to your CSV rules. Eg: encoding iso-8859-1. The following encodings are supported: ascii, utf-8, utf-16, utf-32, iso-8859-1, iso-8859-2, iso-8859-3, iso-8859-4, iso-8859-5, iso-8859-6, iso-8859-7, iso-8859-8, iso-8859-9, - iso-8859-10, iso-8859-11, iso-8859-13, iso-8859-14, iso-8859-15, - iso-8859-16, cp1250, cp1251, cp1252, cp1253, cp1254, cp1255, cp1256, - cp1257, cp1258, koi8-r, koi8-u, gb18030, macintosh, jis-x-0201, - jis-x-0208, iso-2022-jp, shift-jis, cp437, cp737, cp775, cp850, cp852, - cp855, cp857, cp860, cp861, cp862, cp863, cp864, cp865, cp866, cp869, + iso-8859-10, iso-8859-11, iso-8859-13, iso-8859-14, iso-8859-15, + iso-8859-16, cp1250, cp1251, cp1252, cp1253, cp1254, cp1255, cp1256, + cp1257, cp1258, koi8-r, koi8-u, gb18030, macintosh, jis-x-0201, + jis-x-0208, iso-2022-jp, shift-jis, cp437, cp737, cp775, cp850, cp852, + cp855, cp857, cp860, cp861, cp862, cp863, cp864, cp865, cp866, cp869, cp874, cp932. Added in 1.42. separator - You can use the separator rule to read other kinds of character-sepa- - rated data. The argument is any single separator character, or the - words tab or space (case insensitive). Eg, for comma-separated values + You can use the separator rule to read other kinds of character-sepa- + rated data. The argument is any single separator character, or the + words tab or space (case insensitive). Eg, for comma-separated values (CSV): separator , @@ -3500,32 +3607,32 @@ CSV separator TAB - If the input file has a .csv, .ssv or .tsv file extension (or a csv:, + If the input file has a .csv, .ssv or .tsv file extension (or a csv:, ssv:, tsv: prefix), the appropriate separator will be inferred automat- ically, and you won't need this rule. skip skip N - The word skip followed by a number (or no number, meaning 1) tells - hledger to ignore this many non-empty lines at the start of the input - data. You'll need this whenever your CSV data contains header lines. - Note, empty and blank lines are skipped automatically, so you don't + The word skip followed by a number (or no number, meaning 1) tells + hledger to ignore this many non-empty lines at the start of the input + data. You'll need this whenever your CSV data contains header lines. + Note, empty and blank lines are skipped automatically, so you don't need to count those. - skip has a second meaning: it can be used inside if blocks (described - below), to skip one or more records whenever the condition is true. + skip has a second meaning: it can be used inside if blocks (described + below), to skip one or more records whenever the condition is true. Records skipped in this way are ignored, except they are still required to be valid CSV. date-format date-format DATEFMT - This is a helper for the date (and date2) fields. If your CSV dates - are not formatted like YYYY-MM-DD, YYYY/MM/DD or YYYY.MM.DD, you'll - need to add a date-format rule describing them with a strptime-style - date parsing pattern - see https://hackage.haskell.org/pack- - age/time/docs/Data-Time-Format.html#v:formatTime. The pattern must + This is a helper for the date (and date2) fields. If your CSV dates + are not formatted like YYYY-MM-DD, YYYY/MM/DD or YYYY.MM.DD, you'll + need to add a date-format rule describing them with a strptime-style + date parsing pattern - see https://hackage.haskell.org/pack- + age/time/docs/Data-Time-Format.html#v:formatTime. The pattern must parse the CSV date value completely. Some examples: # MM/DD/YY @@ -3542,39 +3649,39 @@ CSV # Note the time and junk must be fully parsed, though only the date is used. date-format %-m/%-d/%Y %l:%M %p some other junk - Note currently there is no locale awareness for things like %b, and + Note currently there is no locale awareness for things like %b, and setting LC_TIME won't help. timezone timezone TIMEZONE - When CSV contains date-times that are implicitly in some time zone + When CSV contains date-times that are implicitly in some time zone other than yours, but containing no explicit time zone information, you - can use this rule to declare the CSV's native time zone, which helps + can use this rule to declare the CSV's native time zone, which helps prevent off-by-one dates. - When the CSV date-times do contain time zone information, you don't - need this rule; instead, use %Z in date-format (or %z, %EZ, %Ez; see + When the CSV date-times do contain time zone information, you don't + need this rule; instead, use %Z in date-format (or %z, %EZ, %Ez; see the formatTime link above). In either of these cases, hledger will do a time-zone-aware conversion, localising the CSV date-times to your current system time zone. If you prefer to localise to some other time zone, eg for reproducibility, you - can (on unix at least) set the output timezone with the TZ environment + can (on unix at least) set the output timezone with the TZ environment variable, eg: $ TZ=-1000 hledger print -f foo.csv # or TZ=-1000 hledger import foo.csv - timezone currently does not understand timezone names, except "UTC", - "GMT", "EST", "EDT", "CST", "CDT", "MST", "MDT", "PST", or "PDT". For + timezone currently does not understand timezone names, except "UTC", + "GMT", "EST", "EDT", "CST", "CDT", "MST", "MDT", "PST", or "PDT". For others, use numeric format: +HHMM or -HHMM. newest-first hledger tries to ensure that the generated transactions will be ordered chronologically, including same-day transactions. Usually it can - auto-detect how the CSV records are ordered. But if it encounters CSV + auto-detect how the CSV records are ordered. But if it encounters CSV where all records are on the same date, it assumes that the records are - oldest first. If in fact the CSV's records are normally newest first, + oldest first. If in fact the CSV's records are normally newest first, like: 2022-10-01, txn 3... @@ -3588,9 +3695,9 @@ CSV newest-first intra-day-reversed - If CSV records within a single day are ordered opposite to the overall - record order, you can add the intra-day-reversed rule to improve the - order of journal entries. Eg, here the overall record order is newest + If CSV records within a single day are ordered opposite to the overall + record order, you can add the intra-day-reversed rule to improve the + order of journal entries. Eg, here the overall record order is newest first, but same-day records are oldest first: 2022-10-02, txn 3... @@ -3608,35 +3715,35 @@ CSV decimal-mark , - hledger automatically accepts either period or comma as a decimal mark - when parsing numbers (cf Amounts). However if any numbers in the CSV - contain digit group marks, such as thousand-separating commas, you - should declare the decimal mark explicitly with this rule, to avoid + hledger automatically accepts either period or comma as a decimal mark + when parsing numbers (cf Amounts). However if any numbers in the CSV + contain digit group marks, such as thousand-separating commas, you + should declare the decimal mark explicitly with this rule, to avoid misparsed numbers. CSV fields and hledger fields This can be confusing, so let's start with an overview: - o CSV fields are provided by your data file. They are named by their - position in the CSV record, starting with 1. You can also give them + o CSV fields are provided by your data file. They are named by their + position in the CSV record, starting with 1. You can also give them a readable name. - o hledger fields are predefined; date, description, account1, amount1, - account2 are some of them. They correspond to parts of a transac- + o hledger fields are predefined; date, description, account1, amount1, + account2 are some of them. They correspond to parts of a transac- tion's journal entry, mostly. o The CSV fields and hledger fields are the only fields you'll be work- - ing with; you can't define new fields, or variables as in a program- - ming language. (But you could add extra CSV fields to the data in + ing with; you can't define new fields, or variables as in a program- + ming language. (But you could add extra CSV fields to the data in preprocessing, before running the rules.) - o For each CSV record, you'll assign values to one or more of the + o For each CSV record, you'll assign values to one or more of the hledger fields to build up a transaction (journal entry). Values can be static text, CSV field values from the current record, or a combi- nation of these. - o For simple cases, you can give a CSV field the same name as one of - the hledger fields, then its value will be automatically assigned to + o For simple cases, you can give a CSV field the same name as one of + the hledger fields, then its value will be automatically assigned to that hledger field. o CSV fields can only be read, not written to. They'll be on the right @@ -3646,12 +3753,12 @@ CSV o interpolating its value: HLEDGERFIELD %CSVFIELD - o hledger fields can only be written to, not read. They'll be on the + o hledger fields can only be written to, not read. They'll be on the left hand side (or in a fields list), with no prefix. Eg o setting the transaction's description to a value: description VALUE - o setting the transaction's description to the second CSV field's + o setting the transaction's description to the second CSV field's value: fields date, description, amount @@ -3661,17 +3768,17 @@ CSV A fields list (the word fields followed by comma-separated field names) is optional, but convenient. It does two things: - 1. It names the CSV field in each column. This can be convenient if - you are referencing them in other rules, so you can say %SomeField + 1. It names the CSV field in each column. This can be convenient if + you are referencing them in other rules, so you can say %SomeField instead of remembering %13. - 2. Whenever you use one of the special hledger field names (described - below), it assigns the CSV value in this position to that hledger - field. This is the quickest way to populate hledger's fields and + 2. Whenever you use one of the special hledger field names (described + below), it assigns the CSV value in this position to that hledger + field. This is the quickest way to populate hledger's fields and build a transaction. - Here's an example that says "use the 1st, 2nd and 4th fields as the - transaction's date, description and amount; name the last two fields + Here's an example that says "use the 1st, 2nd and 4th fields as the + transaction's date, description and amount; name the last two fields for later reference; and ignore the others": fields date, description, , amount, , , somefield, anotherfield @@ -3681,35 +3788,35 @@ CSV o There must be least two items in the list (at least one comma). - o Field names may not contain spaces. Spaces before/after field names + o Field names may not contain spaces. Spaces before/after field names are optional. o Field names may contain _ (underscore) or - (hyphen). - o Fields you don't care about can be given a dummy name or an empty + o Fields you don't care about can be given a dummy name or an empty name. - If the CSV contains column headings, it's convenient to use these for - your field names, suitably modified (eg lower-cased with spaces re- + If the CSV contains column headings, it's convenient to use these for + your field names, suitably modified (eg lower-cased with spaces re- placed by underscores). - Sometimes you may want to alter a CSV field name to avoid assigning to - a hledger field with the same name. Eg you could call the CSV's "bal- - ance" field balance_ to avoid directly setting hledger's balance field + Sometimes you may want to alter a CSV field name to avoid assigning to + a hledger field with the same name. Eg you could call the CSV's "bal- + ance" field balance_ to avoid directly setting hledger's balance field (and generating a balance assertion). Field assignment HLEDGERFIELD FIELDVALUE - Field assignments are the more flexible way to assign CSV values to + Field assignments are the more flexible way to assign CSV values to hledger fields. They can be used instead of or in addition to a fields list (see above). - To assign a value to a hledger field, write the field name (any of the - standard hledger field/pseudo-field names, defined below), a space, - followed by a text value on the same line. This text value may inter- - polate CSV fields, referenced either by their 1-based position in the - CSV record (%N) or by the name they were given in the fields list + To assign a value to a hledger field, write the field name (any of the + standard hledger field/pseudo-field names, defined below), a space, + followed by a text value on the same line. This text value may inter- + polate CSV fields, referenced either by their 1-based position in the + CSV record (%N) or by the name they were given in the fields list (%CSVFIELD), and regular expression match groups (\N). Some examples: @@ -3722,26 +3829,26 @@ CSV Tips: - o Interpolation strips outer whitespace (so a CSV value like " 1 " be- + o Interpolation strips outer whitespace (so a CSV value like " 1 " be- comes 1 when interpolated) (#1051). - o Interpolations always refer to a CSV field - you can't interpolate a + o Interpolations always refer to a CSV field - you can't interpolate a hledger field. (See Referencing other fields below). Field names - Note the two kinds of field names mentioned here, and used only in + Note the two kinds of field names mentioned here, and used only in hledger CSV rules files: - 1. CSV field names (CSVFIELD in these docs): you can optionally name - the CSV columns for easy reference (since hledger doesn't yet auto- + 1. CSV field names (CSVFIELD in these docs): you can optionally name + the CSV columns for easy reference (since hledger doesn't yet auto- matically recognise column headings in a CSV file), by writing arbi- trary names in a fields list, eg: fields When, What, Some_Id, Net, Total, Foo, Bar - 2. Special hledger field names (HLEDGERFIELD in these docs): you must - set at least some of these to generate the hledger transaction from - a CSV record, by writing them as the left hand side of a field as- + 2. Special hledger field names (HLEDGERFIELD in these docs): you must + set at least some of these to generate the hledger transaction from + a CSV record, by writing them as the left hand side of a field as- signment, eg: date %When @@ -3756,7 +3863,7 @@ CSV currency $ comment %Foo %Bar - Here are all the special hledger field names available, and what hap- + Here are all the special hledger field names available, and what hap- pens when you assign values to them: date field @@ -3779,7 +3886,7 @@ CSV commentN, where N is a number, sets the Nth posting's comment. - You can assign multi-line comments by writing literal \n in the code. + You can assign multi-line comments by writing literal \n in the code. A comment starting with \n will begin on a new line. Comments can contain tags, as usual. @@ -3791,99 +3898,99 @@ CSV Assigning to accountN, where N is 1 to 99, sets the account name of the Nth posting, and causes that posting to be generated. - Most often there are two postings, so you'll want to set account1 and - account2. Typically account1 is associated with the CSV file, and is - set once with a top-level assignment, while account2 is set based on + Most often there are two postings, so you'll want to set account1 and + account2. Typically account1 is associated with the CSV file, and is + set once with a top-level assignment, while account2 is set based on each transaction's description, in conditional rules. - If a posting's account name is left unset but its amount is set (see - below), a default account name will be chosen (like "expenses:unknown" + If a posting's account name is left unset but its amount is set (see + below), a default account name will be chosen (like "expenses:unknown" or "income:unknown"). amount field - There are several ways to set posting amounts from CSV, useful in dif- + There are several ways to set posting amounts from CSV, useful in dif- ferent situations. - 1. amount is the oldest and simplest. Assigning to this sets the + 1. amount is the oldest and simplest. Assigning to this sets the amount of the first and second postings. In the second posting, the - amount will be negated; also, if it has a cost attached, it will be + amount will be negated; also, if it has a cost attached, it will be converted to cost. - 2. amount-in and amount-out work exactly like the above, but should be - used when the CSV has two amount fields (such as "Debit" and + 2. amount-in and amount-out work exactly like the above, but should be + used when the CSV has two amount fields (such as "Debit" and "Credit", or "Inflow" and "Outflow"). Whichever field has a - non-zero value will be used as the amount of the first and second + non-zero value will be used as the amount of the first and second postings. Here are some tips to avoid confusion: - o It's not "amount-in for posting 1 and amount-out for posting 2", - it is "extract a single amount from the amount-in or amount-out + o It's not "amount-in for posting 1 and amount-out for posting 2", + it is "extract a single amount from the amount-in or amount-out field, and use that for posting 1 and (negated) for posting 2". - o Don't use both amount and amount-in/amount-out in the same rules + o Don't use both amount and amount-in/amount-out in the same rules file; choose based on whether the amount is in a single CSV field or spread across two fields. - o In each record, at most one of the two CSV fields should contain - a non-zero amount; the other field must contain a zero or noth- + o In each record, at most one of the two CSV fields should contain + a non-zero amount; the other field must contain a zero or noth- ing. - o hledger assumes both CSV fields contain unsigned numbers, and it + o hledger assumes both CSV fields contain unsigned numbers, and it automatically negates the amount-out values. - o If the data doesn't fit these requirements, you'll probably need + o If the data doesn't fit these requirements, you'll probably need an if rule (see below). 3. amountN (where N is a number from 1 to 99) sets the amount of only a - single posting: the Nth posting in the transaction. You'll usually - need at least two such assignments to make a balanced transaction. + single posting: the Nth posting in the transaction. You'll usually + need at least two such assignments to make a balanced transaction. You can also generate more than two postings, to represent more com- - plex transactions. The posting numbers don't have to be consecu- - tive; with if rules, higher posting numbers can be useful to ensure + plex transactions. The posting numbers don't have to be consecu- + tive; with if rules, higher posting numbers can be useful to ensure a certain order of postings. - 4. amountN-in and amountN-out work exactly like the above, but should - be used when the CSV has two amount fields. This is analogous to + 4. amountN-in and amountN-out work exactly like the above, but should + be used when the CSV has two amount fields. This is analogous to amount-in and amount-out, and those tips also apply here. 5. Remember that a fields list can also do assignments. So in a fields - list if you name a CSV field "amount", that counts as assigning to - amount. (If you don't want that, call it something else in the + list if you name a CSV field "amount", that counts as assigning to + amount. (If you don't want that, call it something else in the fields list, like "amount_".) - 6. The above don't handle every situation; if you need more flexibil- + 6. The above don't handle every situation; if you need more flexibil- ity, use an if rule to set amounts conditionally. See "Working with - CSV > Setting amounts" below for more on this and on amount-setting + CSV > Setting amounts" below for more on this and on amount-setting generally. currency field - currency sets a currency symbol, to be prepended to all postings' - amounts. You can use this if the CSV amounts do not have a currency + currency sets a currency symbol, to be prepended to all postings' + amounts. You can use this if the CSV amounts do not have a currency symbol, eg if it is in a separate column. currencyN prepends a currency symbol to just the Nth posting's amount. balance field - balanceN sets a balance assertion amount (or if the posting amount is + balanceN sets a balance assertion amount (or if the posting amount is left empty, a balance assignment) on posting N. balance is a compatibility spelling for hledger <1.17; it is equivalent to balance1. - You can adjust the type of assertion/assignment with the balance-type + You can adjust the type of assertion/assignment with the balance-type rule (see below). - See the Working with CSV tips below for more about setting amounts and + See the Working with CSV tips below for more about setting amounts and currency. if block - Rules can be applied conditionally, depending on patterns in the CSV - data. This allows flexibility; in particular, it is how you can cate- - gorise transactions, selecting an appropriate account name based on - their description (for example). There are two ways to write condi- - tional rules: "if blocks", described here, and "if tables", described + Rules can be applied conditionally, depending on patterns in the CSV + data. This allows flexibility; in particular, it is how you can cate- + gorise transactions, selecting an appropriate account name based on + their description (for example). There are two ways to write condi- + tional rules: "if blocks", described here, and "if tables", described below. - An if block is the word if and one or more "matcher" expressions (can + An if block is the word if and one or more "matcher" expressions (can be a word or phrase), one per line, starting either on the same or next line; followed by one or more indented rules. Eg, @@ -3899,11 +4006,11 @@ CSV RULE RULE - If any of the matchers succeeds, all of the indented rules will be ap- - plied. They are usually field assignments, but the following special + If any of the matchers succeeds, all of the indented rules will be ap- + plied. They are usually field assignments, but the following special rules may also be used within an if block: - o skip - skips the matched CSV record (generating no transaction from + o skip - skips the matched CSV record (generating no transaction from it) o end - skips the rest of the current CSV file. @@ -3929,39 +4036,39 @@ CSV Matchers There are two kinds of matcher: - 1. A whole record matcher is simplest: it is just a word, single-line - text fragment, or other regular expression, which hledger will try + 1. A whole record matcher is simplest: it is just a word, single-line + text fragment, or other regular expression, which hledger will try to match case-insensitively anywhere within the CSV record. Eg: whole foods. - 2. A field matcher has a percent-prefixed CSV field number or name be- + 2. A field matcher has a percent-prefixed CSV field number or name be- fore the pattern. Eg: %3 whole foods or %description whole foods. hledger will try to match the pattern just within the named CSV field. When using these, there's two things to be aware of: - 1. Whole record matchers don't see the exact original record; they see - a reconstruction of it, in which values are comma-separated, and - quotes enclosing values and whitespace outside those quotes are re- + 1. Whole record matchers don't see the exact original record; they see + a reconstruction of it, in which values are comma-separated, and + quotes enclosing values and whitespace outside those quotes are re- moved. Eg when reading an SSV record like: 2023-01-01 ; "Acme, Inc. " ; 1,000 the whole record matcher sees instead: 2023-01-01,Acme, Inc. ,1,000 2. Field matchers expect either a CSV field number, or a CSV field name - declared with fields. (Don't use a hledger field name here, unless - it is also a CSV field name.) A non-CSV field name will cause the - matcher to match against "" (the empty string), and does not raise - an error, allowing easier reuse of common rules with different CSV + declared with fields. (Don't use a hledger field name here, unless + it is also a CSV field name.) A non-CSV field name will cause the + matcher to match against "" (the empty string), and does not raise + an error, allowing easier reuse of common rules with different CSV files. You can also prefix a matcher with ! (and optional space) to negate it. - Eg ! whole foods, ! %3 whole foods, !%description whole foods will + Eg ! whole foods, ! %3 whole foods, !%description whole foods will match if "whole foods" is NOT present. Added in 1.32. - The pattern is, as usual in hledger, a POSIX extended regular expres- - sion that also supports GNU word boundaries (\b, \B, \<, \>) and noth- - ing else. For more details and tips, see Regular expressions in CSV + The pattern is, as usual in hledger, a POSIX extended regular expres- + sion that also supports GNU word boundaries (\b, \B, \<, \>) and noth- + ing else. For more details and tips, see Regular expressions in CSV rules below. Multiple matchers @@ -3969,28 +4076,28 @@ CSV o By default they are OR'd (any of them can match). - o Matcher lines beginning with & (or &&, since 1.42) are AND'ed with + o Matcher lines beginning with & (or &&, since 1.42) are AND'ed with the matcher above (all in the AND'ed group must match). - o Matcher lines beginning with & ! (since 1.41, or && !, since 1.42) + o Matcher lines beginning with & ! (since 1.41, or && !, since 1.42) are first negated and then AND'ed with the matcher above. - You can also combine multiple matchers one the same line separated by + You can also combine multiple matchers one the same line separated by && (AND) or && ! (AND NOT). Eg %description amazon && %date 2025-01-01 - will match only when the description field contains "amazon" and the + will match only when the description field contains "amazon" and the date field contains "2025-01-01". Added in 1.42. Match groups Added in 1.32 Matchers can define match groups: parenthesised portions of the regular - expression which are available for reference in field assignments. + expression which are available for reference in field assignments. Groups are enclosed in regular parentheses (( and )) and can be nested. - Each group is available in field assignments using the token \N, where - N is an index into the match groups for this conditional block (e.g. + Each group is available in field assignments using the token \N, where + N is an index into the match groups for this conditional block (e.g. \1, \2, etc.). - Example: Warp credit card payment postings to the beginning of the + Example: Warp credit card payment postings to the beginning of the billing period (Month start), to match how they are presented in state- ments, using posting dates: @@ -4004,8 +4111,8 @@ CSV account1 \1 if table - "if tables" are an alternative to if blocks; they can express many - matchers and field assignments in a more compact tabular format, like + "if tables" are an alternative to if blocks; they can express many + matchers and field assignments in a more compact tabular format, like this: if,HLEDGERFIELD1,HLEDGERFIELD2,... @@ -4016,21 +4123,21 @@ CSV The first character after if is taken to be this if table's field sepa- - rator. It is unrelated to the separator used in the CSV file. It + rator. It is unrelated to the separator used in the CSV file. It should be a non-alphanumeric character like , or | that does not appear - anywhere else in the table (it should not be used in field names or + anywhere else in the table (it should not be used in field names or matchers or values, and it cannot be escaped with a backslash). - Each line must contain the same number of separators; empty values are - allowed. Whitespace can be used in the matcher lines for readability - (but not in the if line, currently). You can use the comment lines in - the table body. The table must be terminated by an empty line (or end + Each line must contain the same number of separators; empty values are + allowed. Whitespace can be used in the matcher lines for readability + (but not in the if line, currently). You can use the comment lines in + the table body. The table must be terminated by an empty line (or end of file). - An if table like the above is interpreted as follows: try all of the + An if table like the above is interpreted as follows: try all of the lines with matchers; whenever a line with matchers succeeds, assign all of the values on that line to the corresponding hledger fields; If mul- - tiple lines match, later lines will override fields assigned by the + tiple lines match, later lines will override fields assigned by the earlier ones - just like the sequence of if blocks would behave. If table presented above is equivalent to this sequence of if blocks: @@ -4061,10 +4168,10 @@ CSV balance-type Balance assertions generated by assigning to balanceN are of the simple - = type by default, which is a single-commodity, subaccount-excluding + = type by default, which is a single-commodity, subaccount-excluding assertion. You may find the subaccount-including variants more useful, - eg if you have created some virtual subaccounts of checking to help - with budgeting. You can select a different type of assertion with the + eg if you have created some virtual subaccounts of checking to help + with budgeting. You can select a different type of assertion with the balance-type rule: # balance assertions will consider all commodities and all subaccounts @@ -4080,9 +4187,9 @@ CSV include include RULESFILE - This includes the contents of another CSV rules file at this point. - RULESFILE is an absolute file path or a path relative to the current - file's directory. This can be useful for sharing common rules between + This includes the contents of another CSV rules file at this point. + RULESFILE is an absolute file path or a path relative to the current + file's directory. This can be useful for sharing common rules between several rules files, eg: # someaccount.csv.rules @@ -4099,42 +4206,42 @@ CSV Some tips: Rapid feedback - It's a good idea to get rapid feedback while creating/troubleshooting + It's a good idea to get rapid feedback while creating/troubleshooting CSV rules. Here's a good way, using entr from eradman.com/entrproject: $ ls foo.csv* | entr bash -c 'echo ----; hledger -f foo.csv print desc:SOMEDESC' - A desc: query (eg) is used to select just one, or a few, transactions - of interest. "bash -c" is used to run multiple commands, so we can - echo a separator each time the command re-runs, making it easier to + A desc: query (eg) is used to select just one, or a few, transactions + of interest. "bash -c" is used to run multiple commands, so we can + echo a separator each time the command re-runs, making it easier to read the output. Valid CSV - Note that hledger will only accept valid CSV conforming to RFC 4180, + Note that hledger will only accept valid CSV conforming to RFC 4180, and equivalent SSV and TSV formats (like RFC 4180 but with semicolon or tab as separators). This means, eg: o Values may be enclosed in double quotes, or not. Enclosing in single quotes is not allowed. (Eg 'A','B' is rejected.) - o When values are enclosed in double quotes, spaces outside the quotes + o When values are enclosed in double quotes, spaces outside the quotes are not allowed. (Eg "A", "B" is rejected.) - o When values are not enclosed in quotes, they may not contain double + o When values are not enclosed in quotes, they may not contain double quotes. (Eg A"A, B is rejected.) - If your CSV/SSV/TSV is not valid in this sense, you'll need to trans- - form it before reading with hledger. Try using sed, or a more permis- + If your CSV/SSV/TSV is not valid in this sense, you'll need to trans- + form it before reading with hledger. Try using sed, or a more permis- sive CSV parser like python's csv lib. File Extension - To help hledger choose the CSV file reader and show the right error - messages (and choose the right field separator character by default), - it's best if CSV/SSV/TSV files are named with a .csv, .ssv or .tsv + To help hledger choose the CSV file reader and show the right error + messages (and choose the right field separator character by default), + it's best if CSV/SSV/TSV files are named with a .csv, .ssv or .tsv filename extension. (More about this at Data formats.) - When reading files with the "wrong" extension, you can ensure the CSV - reader (and the default field separator) by prefixing the file path + When reading files with the "wrong" extension, you can ensure the CSV + reader (and the default field separator) by prefixing the file path with csv:, ssv: or tsv:: Eg: $ hledger -f ssv:foo.dat print @@ -4143,29 +4250,29 @@ CSV if needed. Reading CSV from standard input - You'll need the file format prefix when reading CSV from stdin also, + You'll need the file format prefix when reading CSV from stdin also, since hledger assumes journal format by default. Eg: $ cat foo.dat | hledger -f ssv:- print Reading multiple CSV files - If you use multiple -f options to read multiple CSV files at once, - hledger will look for a correspondingly-named rules file for each CSV - file. But if you specify a rules file with --rules, that rules file + If you use multiple -f options to read multiple CSV files at once, + hledger will look for a correspondingly-named rules file for each CSV + file. But if you specify a rules file with --rules, that rules file will be used for all the CSV files. Reading files specified by rule Instead of specifying a CSV file in the command line, you can specify a - rules file, as in hledger -f foo.csv.rules CMD. By default this will - read data from foo.csv in the same directory, but you can add a source - rule to specify a different data file, perhaps located in your web + rules file, as in hledger -f foo.csv.rules CMD. By default this will + read data from foo.csv in the same directory, but you can add a source + rule to specify a different data file, perhaps located in your web browser's download directory. This feature was added in hledger 1.30, so you won't see it in most CSV - rules examples. But it helps remove some of the busywork of managing + rules examples. But it helps remove some of the busywork of managing CSV downloads. Most of your financial institutions's default CSV file- - names are different and can be recognised by a glob pattern. So you - can put a rule like source Checking1*.csv in foo-checking.csv.rules, + names are different and can be recognised by a glob pattern. So you + can put a rule like source Checking1*.csv in foo-checking.csv.rules, and then periodically follow a workflow like: 1. Download CSV from Foo's website, using your browser's defaults @@ -4173,45 +4280,45 @@ CSV 2. Run hledger import foo-checking.csv.rules to import any new transac- tions - After import, you can: discard the CSV, or leave it where it is for a - while, or move it into your archives, as you prefer. If you do noth- - ing, next time your browser will save something like Checking1-2.csv, - and hledger will use that because of the * wild card and because it is + After import, you can: discard the CSV, or leave it where it is for a + while, or move it into your archives, as you prefer. If you do noth- + ing, next time your browser will save something like Checking1-2.csv, + and hledger will use that because of the * wild card and because it is the most recent. Valid transactions After reading a CSV file, hledger post-processes and validates the gen- erated journal entries as it would for a journal file - balancing them, - applying balance assignments, and canonicalising amount styles. Any - errors at this stage will be reported in the usual way, displaying the + applying balance assignments, and canonicalising amount styles. Any + errors at this stage will be reported in the usual way, displaying the problem entry. There is one exception: balance assertions, if you have generated them, - will not be checked, since normally these will work only when the CSV - data is part of the main journal. If you do need to check balance as- + will not be checked, since normally these will work only when the CSV + data is part of the main journal. If you do need to check balance as- sertions generated from CSV right away, pipe into another hledger: $ hledger -f file.csv print | hledger -f- print Deduplicating, importing - When you download a CSV file periodically, eg to get your latest bank - transactions, the new file may overlap with the old one, containing + When you download a CSV file periodically, eg to get your latest bank + transactions, the new file may overlap with the old one, containing some of the same records. The import command will (a) detect the new transactions, and (b) append just those transactions to your main journal. It is idempotent, so you - don't have to remember how many times you ran it or with which version - of the CSV. (It keeps state in a hidden .latest.FILE.csv file.) This + don't have to remember how many times you ran it or with which version + of the CSV. (It keeps state in a hidden .latest.FILE.csv file.) This is the easiest way to import CSV data. Eg: # download the latest CSV files, then run this command. # Note, no -f flags needed here. $ hledger import *.csv [--dry] - This method works for most CSV files. (Where records have a stable + This method works for most CSV files. (Where records have a stable chronological order, and new records appear only at the new end.) - A number of other tools and workflows, hledger-specific and otherwise, + A number of other tools and workflows, hledger-specific and otherwise, exist for converting, deduplicating, classifying and managing CSV data. See: @@ -4220,7 +4327,7 @@ CSV o https://plaintextaccounting.org -> data import/conversion Regular expressions in CSV rules - Regular expressions in if conditions (AKA matchers) are POSIX extended + Regular expressions in if conditions (AKA matchers) are POSIX extended regular expressions, that also support GNU word boundaries (\b, \B, \<, \>), and nothing else. (For more detail, see Regular expressions.) @@ -4234,13 +4341,13 @@ CSV o Does it contain non-whitespace ? if %foo [^ ] - Testing the value of numeric fields is a little harder. You can't use - hledger queries like amt:0 or amt:>10 in CSV rules. But you can often + Testing the value of numeric fields is a little harder. You can't use + hledger queries like amt:0 or amt:>10 in CSV rules. But you can often achieve the same thing with a regular expression. - Note the content and layout of number fields in CSV varies, and can + Note the content and layout of number fields in CSV varies, and can change over time (eg if you switch data providers). So numeric regexps - are always somewhat specific to your particular CSV data; and it's a + are always somewhat specific to your particular CSV data; and it's a good idea to make them defensive and robust if you can. Here are some examples: @@ -4251,22 +4358,22 @@ CSV o Is it non-negative ? if ! %foo - - o Is it >= 10 ? if %foo [1-9][0-9]+\. (assuming a decimal period and + o Is it >= 10 ? if %foo [1-9][0-9]+\. (assuming a decimal period and no leading zeros) o Is it >= 10 and < 20 ? if %foo \b1[0-9]\. Setting amounts - Continuing from amount field above, here are more tips for amount-set- + Continuing from amount field above, here are more tips for amount-set- ting: 1. If the amount is in a single CSV field: a. If its sign indicates direction of flow: - Assign it to amountN, to set the Nth posting's amount. N is usu- + Assign it to amountN, to set the Nth posting's amount. N is usu- ally 1 or 2 but can go up to 99. b. If another field indicates direction of flow: - Use one or more conditional rules to set the appropriate amount + Use one or more conditional rules to set the appropriate amount sign. Eg: # assume a withdrawal unless Type contains "deposit": @@ -4274,15 +4381,15 @@ CSV if %Type deposit amount1 %Amount - 2. If the amount is in two CSV fields (such as Debit and Credit, or In + 2. If the amount is in two CSV fields (such as Debit and Credit, or In and Out): a. If both fields are unsigned: - Assign one field to amountN-in and the other to amountN-out. - hledger will automatically negate the "out" field, and will use + Assign one field to amountN-in and the other to amountN-out. + hledger will automatically negate the "out" field, and will use whichever field value is non-zero as posting N's amount. b. If either field is signed: - You will probably need to override hledger's sign for one or the + You will probably need to override hledger's sign for one or the other field, as in the following example: # Negate the -out value, but only if it is not empty: @@ -4290,12 +4397,12 @@ CSV if %amount1-out [1-9] amount1-out -%amount1-out - c. If both fields can contain a non-zero value (or both can be + c. If both fields can contain a non-zero value (or both can be empty): - The -in/-out rules normally choose the value which is - non-zero/non-empty. Some value pairs can be ambiguous, such as 1 + The -in/-out rules normally choose the value which is + non-zero/non-empty. Some value pairs can be ambiguous, such as 1 and none. For such cases, use conditional rules to help select the - amount. Eg, to handle the above you could select the value con- + amount. Eg, to handle the above you could select the value con- taining non-zero digits: fields date, description, in, out @@ -4308,8 +4415,8 @@ CSV Use the unnumbered amount (or amount-in and amount-out) syntax. 4. If the CSV has only balance amounts, not transaction amounts: - Assign to balanceN, to set a balance assignment on the Nth posting, - causing the posting's amount to be calculated automatically. balance + Assign to balanceN, to set a balance assignment on the Nth posting, + causing the posting's amount to be calculated automatically. balance with no number is equivalent to balance1. In this situation hledger is more likely to guess the wrong default account name, so you may need to set that explicitly. @@ -4325,20 +4432,20 @@ CSV o If an amount value is parenthesised: it will be de-parenthesised and sign-flipped: (AMT) becomes -AMT - o If an amount value has two minus signs (or two sets of parentheses, + o If an amount value has two minus signs (or two sets of parentheses, or a minus sign and parentheses): they cancel out and will be removed: --AMT or -(AMT) becomes AMT - o If an amount value contains just a sign (or just a set of parenthe- + o If an amount value contains just a sign (or just a set of parenthe- ses): - that is removed, making it an empty value. "+" or "-" or "()" becomes + that is removed, making it an empty value. "+" or "-" or "()" becomes "". - It's not possible (without preprocessing the CSV) to set an amount to + It's not possible (without preprocessing the CSV) to set an amount to its absolute value, ie discard its sign. Setting currency/commodity - If the currency/commodity symbol is included in the CSV's amount + If the currency/commodity symbol is included in the CSV's amount field(s): 2023-01-01,foo,$123.00 @@ -4357,7 +4464,7 @@ CSV 2023-01-01,foo,USD,123.00 You can assign that to the currency pseudo-field, which has the special - effect of prepending itself to every amount in the transaction (on the + effect of prepending itself to every amount in the transaction (on the left, with no separating space): fields date,description,currency,amount @@ -4366,7 +4473,7 @@ CSV expenses:unknown USD123.00 income:unknown USD-123.00 - Or, you can use a field assignment to construct the amount yourself, + Or, you can use a field assignment to construct the amount yourself, with more control. Eg to put the symbol on the right, and separated by a space: @@ -4377,38 +4484,38 @@ CSV expenses:unknown 123.00 USD income:unknown -123.00 USD - Note we used a temporary field name (cur) that is not currency - that + Note we used a temporary field name (cur) that is not currency - that would trigger the prepending effect, which we don't want here. Amount decimal places - When you are reading CSV data, eg with a command like hledger -f - foo.csv print, hledger will infer each commodity's decimal precision - (and other commodity display styles) from the amounts - much as when + When you are reading CSV data, eg with a command like hledger -f + foo.csv print, hledger will infer each commodity's decimal precision + (and other commodity display styles) from the amounts - much as when reading a journal file without commodity directives (see the link). - Note, the commodity styles are not inferred from the numbers in the + Note, the commodity styles are not inferred from the numbers in the original CSV data; rather, they are inferred from the amounts generated by the CSV rules. When you are importing CSV data with the import command, eg hledger im- - port foo.csv, there's another step: import tries to make the new en- - tries conform to the journal's existing styles. So for each commodity + port foo.csv, there's another step: import tries to make the new en- + tries conform to the journal's existing styles. So for each commodity - let's say it's EUR - import will choose: 1. the style declared for EUR by a commodity directive in the journal 2. otherwise, the style inferred from EUR amounts in the journal - 3. otherwise, the style inferred from EUR amounts generated by the CSV + 3. otherwise, the style inferred from EUR amounts generated by the CSV rules. - TLDR: if import is not generating the precisions or styles you want, + TLDR: if import is not generating the precisions or styles you want, add a commodity directive to specify them. Referencing other fields - In field assignments, you can interpolate only CSV fields, not hledger - fields. In the example below, there's both a CSV field and a hledger - field named amount1, but %amount1 always means the CSV field, not the + In field assignments, you can interpolate only CSV fields, not hledger + fields. In the example below, there's both a CSV field and a hledger + field named amount1, but %amount1 always means the CSV field, not the hledger field: # Name the third CSV field "amount1" @@ -4420,7 +4527,7 @@ CSV # Set comment to the CSV amount1 (not the amount1 assigned above) comment %amount1 - Here, since there's no CSV amount1 field, %amount1 will produce a lit- + Here, since there's no CSV amount1 field, %amount1 will produce a lit- eral "amount1": fields date,description,csvamount @@ -4428,7 +4535,7 @@ CSV # Can't interpolate amount1 here comment %amount1 - When there are multiple field assignments to the same hledger field, + When there are multiple field assignments to the same hledger field, only the last one takes effect. Here, comment's value will be be B, or C if "something" is matched, but never A: @@ -4439,27 +4546,27 @@ CSV How CSV rules are evaluated Here's how to think of CSV rules being evaluated. If you get a confus- - ing error while reading a CSV file, it may help to try to understand + ing error while reading a CSV file, it may help to try to understand which of these steps is failing: - 1. Any included rules files are inlined, from top to bottom, depth - first (scanning each included file for further includes, recur- + 1. Any included rules files are inlined, from top to bottom, depth + first (scanning each included file for further includes, recur- sively, before proceeding). - 2. Top level rules (date-format, fields, newest-first, skip etc) are + 2. Top level rules (date-format, fields, newest-first, skip etc) are read, top to bottom. "Top level rules" means non-conditional rules. - If a rule occurs more than once, the last one wins; except for + If a rule occurs more than once, the last one wins; except for skip/end rules, where the first one wins. - 3. The CSV file is read as text. Any non-ascii characters will be de- + 3. The CSV file is read as text. Any non-ascii characters will be de- coded using the text encoding specified by the encoding rule, other- wise the system locale's text encoding. - 4. Any top-level skip or end rule is applied. skip [N] immediately - skips the current or next N CSV records; end immediately skips all + 4. Any top-level skip or end rule is applied. skip [N] immediately + skips the current or next N CSV records; end immediately skips all remaining CSV records (not normally used at top level). - 5. Now any remaining CSV records are processed. For each CSV record, + 5. Now any remaining CSV records are processed. For each CSV record, in file order: o Is there a conditional skip/end rule that applies for this record @@ -4468,33 +4575,33 @@ CSV ber of CSV records, then continue at 5. Otherwise... - o Do some basic validation on this CSV record (eg, check that it + o Do some basic validation on this CSV record (eg, check that it has at least two fields). o For each hledger field (date, description, account1, etc.): - 1. Get the field's assigned value, first searching top level as- - signments, made directly or by the fields rule, then assign- - ments made inside succeeding if blocks. If there are more + 1. Get the field's assigned value, first searching top level as- + signments, made directly or by the fields rule, then assign- + ments made inside succeeding if blocks. If there are more than one, the last one wins. - 2. Compute the field's actual value (as text), by interpolating - any %CSVFIELD references within the assigned value; or by + 2. Compute the field's actual value (as text), by interpolating + any %CSVFIELD references within the assigned value; or by choosing a default value if there was no assignment. - o Generate a hledger transaction from the hledger field values, + o Generate a hledger transaction from the hledger field values, parsing them if needed (eg from text to an amount). - This is all done by the CSV reader, one of several readers hledger can + This is all done by the CSV reader, one of several readers hledger can use to read transactions from an input file. When all input files have - been read successfully, their transactions are passed to whichever + been read successfully, their transactions are passed to whichever hledger command the user specified. Well factored rules - Some things than can help reduce duplication and complexity in rules + Some things than can help reduce duplication and complexity in rules files: - o Extracting common rules usable with multiple CSV files into a com- + o Extracting common rules usable with multiple CSV files into a com- mon.rules, and adding include common.rules to each CSV's rules file. o Splitting if blocks into smaller if blocks, extracting the frequently @@ -4502,8 +4609,8 @@ CSV CSV rules examples Bank of Ireland - Here's a CSV with two amount fields (Debit and Credit), and a balance - field, which we can use to add balance assertions, which is not neces- + Here's a CSV with two amount fields (Debit and Credit), and a balance + field, which we can use to add balance assertions, which is not neces- sary but provides extra error checking: Date,Details,Debit,Credit,Balance @@ -4545,13 +4652,13 @@ CSV assets:bank:boi:checking EUR-5.0 = EUR126.0 expenses:unknown EUR5.0 - The balance assertions don't raise an error above, because we're read- - ing directly from CSV, but they will be checked if these entries are + The balance assertions don't raise an error above, because we're read- + ing directly from CSV, but they will be checked if these entries are imported into a journal file. Coinbase - A simple example with some CSV from Coinbase. The spot price is - recorded using cost notation. The legacy amount field name conve- + A simple example with some CSV from Coinbase. The spot price is + recorded using cost notation. The legacy amount field name conve- niently sets amount 2 (posting 2's amount) to the total cost. # Timestamp,Transaction Type,Asset,Quantity Transacted,Spot Price Currency,Spot Price at Transaction,Subtotal,Total (inclusive of fees and/or spread),Fees and/or Spread,Notes @@ -4573,7 +4680,7 @@ CSV Amazon Here we convert amazon.com order history, and use an if block to gener- - ate a third posting if there's a fee. (In practice you'd probably get + ate a third posting if there's a fee. (In practice you'd probably get this data from your bank instead, but it's an example.) "Date","Type","To/From","Name","Status","Amount","Fees","Transaction ID" @@ -4625,7 +4732,7 @@ CSV expenses:fees $1.00 Paypal - Here's a real-world rules file for (customised) Paypal CSV, with some + Here's a real-world rules file for (customised) Paypal CSV, with some Paypal-specific rules, and a second rules file included: "Date","Time","TimeZone","Name","Type","Status","Currency","Gross","Fee","Net","From Email Address","To Email Address","Transaction ID","Item Title","Item ID","Reference Txn ID","Receipt ID","Balance","Note" @@ -4774,13 +4881,13 @@ CSV expenses:banking:paypal $0.59 ; business: Timeclock - hledger can read time logs in the timeclock time logging format of - timeclock.el. As with Ledger, hledger's timeclock format is a sub- + hledger can read time logs in the timeclock time logging format of + timeclock.el. As with Ledger, hledger's timeclock format is a sub- set/variant of timeclock.el's. - hledger's timeclock format was updated in hledger 1.43 and 1.50. If - your old time logs are rejected, you should adapt them to modern - hledger; for now, you can restore the pre-1.43 behaviour with the + hledger's timeclock format was updated in hledger 1.43 and 1.50. If + your old time logs are rejected, you should adapt them to modern + hledger; for now, you can restore the pre-1.43 behaviour with the --old-timeclock flag. Here the timeclock format in hledger 1.50+: @@ -4799,18 +4906,18 @@ Timeclock i SIMPLEDATE HH:MM[:SS][+-ZZZZ] ACCOUNT[ DESCRIPTION][;COMMENT]] o SIMPLEDATE HH:MM[:SS][+-ZZZZ][ ACCOUNT][;COMMENT] - The date is a hledger simple date (YYYY-MM-DD or similar). The time - parts must use two digits. The seconds are optional. A + or - - four-digit time zone is accepted for compatibility, but currently ig- + The date is a hledger simple date (YYYY-MM-DD or similar). The time + parts must use two digits. The seconds are optional. A + or - + four-digit time zone is accepted for compatibility, but currently ig- nored; times are always interpreted as a local time. - In clock-in entries (i), the account name is required. A transaction + In clock-in entries (i), the account name is required. A transaction description, separated from the account name by 2+ spaces, is optional. - A transaction comment, beginning with ;, is also optional. (Indented + A transaction comment, beginning with ;, is also optional. (Indented following comment lines are also allowed, as in journal format.) In clock-out entries (o) have no description, but can have a comment if - you wish. A clock-in and clock-out pair form a "transaction" posting + you wish. A clock-in and clock-out pair form a "transaction" posting some number of hours to an account - also known as a session. Eg: i 2015/03/30 09:00:00 session1 @@ -4821,9 +4928,9 @@ Timeclock (session1) 1.00h Clock-ins and clock-outs are matched by their account/session name. If - a clock-out does not specify a name, the most recent unclosed clock-in - is closed. You can have multiple sessions active simultaneously. En- - tries are processed in the order they are parsed. Sessions spanning + a clock-out does not specify a name, the most recent unclosed clock-in + is closed. You can have multiple sessions active simultaneously. En- + tries are processed in the order they are parsed. Sessions spanning more than one day are automatically split at day boundaries. Eg, the following time log: @@ -4872,13 +4979,13 @@ Timeclock perhaps the extras in ledgerutils.el o or use the old ti and to scripts in the ledger 2.x repository. These - rely on a "timeclock" executable which I think is just the ledger 2 + rely on a "timeclock" executable which I think is just the ledger 2 executable renamed. Timedot - timedot format is hledger's human-friendly time logging format. Com- - pared to timeclock format, it is more convenient for quick, approxi- - mate, and retroactive time logging, and more human-readable (you can + timedot format is hledger's human-friendly time logging format. Com- + pared to timeclock format, it is more convenient for quick, approxi- + mate, and retroactive time logging, and more human-readable (you can see at a glance where time was spent). A quick example: 2023-05-01 @@ -4897,59 +5004,59 @@ Timedot (per:admin:finance) 0 A timedot file contains a series of transactions (usually one per day). - Each begins with a simple date (Y-M-D, Y/M/D, or Y.M.D), optionally be + Each begins with a simple date (Y-M-D, Y/M/D, or Y.M.D), optionally be followed on the same line by a transaction description, and/or a trans- action comment following a semicolon. After the date line are zero or more time postings, consisting of: - o An account name - any hledger-style account name, optionally in- + o An account name - any hledger-style account name, optionally in- dented. - o Two or more spaces - required if there is an amount (as in journal + o Two or more spaces - required if there is an amount (as in journal format). o A timedot amount, which can be o empty (representing zero) - o a number, optionally followed by a unit s, m, h, d, w, mo, or y, - representing a precise number of seconds, minutes, hours, days + o a number, optionally followed by a unit s, m, h, d, w, mo, or y, + representing a precise number of seconds, minutes, hours, days weeks, months or years (hours is assumed by default), which will be - converted to hours according to 60s = 1m, 60m = 1h, 24h = 1d, 7d = + converted to hours according to 60s = 1m, 60m = 1h, 24h = 1d, 7d = 1w, 30d = 1mo, 365d = 1y. - o one or more dots (period characters), each representing 0.25. - These are the dots in "timedot". Spaces are ignored and can be + o one or more dots (period characters), each representing 0.25. + These are the dots in "timedot". Spaces are ignored and can be used for grouping/alignment. - o Added in 1.32 one or more letters. These are like dots but they - also generate a tag t: (short for "type") with the letter as its - value, and a separate posting for each of the values. This pro- - vides a second dimension of categorisation, viewable in reports + o Added in 1.32 one or more letters. These are like dots but they + also generate a tag t: (short for "type") with the letter as its + value, and a separate posting for each of the values. This pro- + vides a second dimension of categorisation, viewable in reports with --pivot t. - o An optional comment following a semicolon (a hledger-style posting + o An optional comment following a semicolon (a hledger-style posting comment). - There is some flexibility to help with keeping time log data and notes + There is some flexibility to help with keeping time log data and notes in the same file: o Blank lines and lines beginning with # or ; are ignored. - o After the first date line, lines which do not contain a double space + o After the first date line, lines which do not contain a double space are parsed as postings with zero amount. (hledger's register reports will show these if you add -E). - o Before the first date line, lines beginning with * (eg org headings) - are ignored. And from the first date line onward, Emacs org mode + o Before the first date line, lines beginning with * (eg org headings) + are ignored. And from the first date line onward, Emacs org mode heading prefixes at the start of lines (one or more *'s followed by a - space) will be ignored. This means the time log can also be a org + space) will be ignored. This means the time log can also be a org outline. Timedot files don't support directives like journal files. So a common - pattern is to have a main journal file (eg time.journal) that contains - any needed directives, and then includes the timedot file (include + pattern is to have a main journal file (eg time.journal) that contains + any needed directives, and then includes the timedot file (include time.timedot). Timedot examples @@ -5057,21 +5164,21 @@ Timedot PART 3: REPORTING CONCEPTS Time periods Report start & end date - Most hledger reports will by default show the full time period repre- - sented by the journal. The report start date will be the earliest + Most hledger reports will by default show the full time period repre- + sented by the journal. The report start date will be the earliest transaction or posting date, and the report end date will be the latest transaction, posting, or market price date. Often you will want to see a shorter period, such as the current month. - You can specify a start and/or end date with the -b/--begin, -e/--end, - or -p/--period options, or a date: query argument, described below. + You can specify a start and/or end date with the -b/--begin, -e/--end, + or -p/--period options, or a date: query argument, described below. All of these accept the smart date syntax, also described below. End dates are exclusive; specify the day after the last day you want to see in the report. When dates are specified by multiple options, the last (right-most) op- - tion wins. And when date: queries and date options are combined, the + tion wins. And when date: queries and date options are combined, the report period will be their intersection. Examples: @@ -5099,18 +5206,18 @@ Time periods -b and -e) Smart dates - In hledger's user interfaces (though not in the journal file), you can - optionally use "smart date" syntax. Smart dates can be written with - english words, can be relative, and can have parts omitted. Missing - parts are inferred as 1, when needed. Smart dates can be interpreted + In hledger's user interfaces (though not in the journal file), you can + optionally use "smart date" syntax. Smart dates can be written with + english words, can be relative, and can have parts omitted. Missing + parts are inferred as 1, when needed. Smart dates can be interpreted as dates or periods depending on the context. Examples: 2004-01-01, 2004/10/1, 2004.9.1, 20240504, 2024Q1 : - Exact dates. The year must have at least four digits, the month must - be 1-12, the day must be 1-31, the separator can be - or / or . or - nothing. The q can be upper or lower case and the quarter number must + Exact dates. The year must have at least four digits, the month must + be 1-12, the day must be 1-31, the separator can be - or / or . or + nothing. The q can be upper or lower case and the quarter number must be 1-4. 2004-10 @@ -5133,6 +5240,14 @@ Time periods last/this/next day/week/month/quarter/year -1, 0, 1 periods from the current period + last/this/next tuesday + the previous occurrence of the named day, or the next occurrence + after today + + last/this/next february + the previous occurrence of 1st of the named month, or the next + occurrence after the current month + in n days/weeks/months/quarters/years n periods from the current period @@ -5526,18 +5641,23 @@ Queries date: query date:PERIODEXPR Match dates (or with the --date2 flag, secondary dates) within the - specified period. PERIODEXPR is a period expression with no report in- - terval. Examples: + specified period. PERIODEXPR is a period expression. Examples: date:2016, date:thismonth, date:2/1-2/15, date:2021-07-27..nextquarter. + PERIODEXPR may include a report interval (since 1.52). On the command + line, this is equivalent to specifying a report interval with a command + line option. In other contexts (hledger-ui, hledger-web), the report + interval may be ignored. + date2: query date2:PERIODEXPR If you use secondary dates: this matches secondary dates within the - specified period. It is not affected by the --date2 flag. + specified period. It is not affected by the --date2 flag. A report + interval in PERIODEXPR will be ignored. depth: query depth:[REGEXP=]N - Match (or display, depending on command) accounts at or above this + Match (or display, depending on command) accounts at or above this depth, optionally only for accounts matching a provided regular expres- sion. See Depth for detailed rules. @@ -5548,7 +5668,7 @@ Queries payee: query payee:REGEX - Match transaction payee/payer names (the part of the description left + Match transaction payee/payer names (the part of the description left of |, or the whole description if there's no |). real: query @@ -5561,18 +5681,18 @@ Queries type: query type:TYPECODES - Match by account type (see Declaring accounts > Account types). TYPE- - CODES is one or more of the single-letter account type codes ALERXCV, - case insensitive. Note type:A and type:E will also match their respec- - tive subtypes C (Cash) and V (Conversion). Certain kinds of account - alias can disrupt account types, see Rewriting accounts > Aliases and - account types. + Match by account type (see Declaring accounts > Account types). TYPE- + CODES is one or more of the single-letter account type codes ALERXCVG, + case insensitive. Note type:A, type:E, and type:R will also match + their respective subtypes C (Cash), V (Conversion), and G (Gain). Cer- + tain kinds of account alias can disrupt account types, see Rewriting + accounts > Aliases and account types. tag: query tag:NAMEREGEX[=VALREGEX] Match by tag name, and optionally also by tag value. Note: - o Both regular expressions do infix matching. If you need a complete + o Both regular expressions do infix matching. If you need a complete match, use ^ and $. Eg: tag:'^fullname$', tag:'^fullname$=^fullvalue$ @@ -5590,11 +5710,11 @@ Queries not:QUERY You can prepend not: to a query to negate the match. Eg: not:equity, not:desc:apple - (Also, a trick: not:not:... can sometimes solve query problems conve- + (Also, a trick: not:not:... can sometimes solve query problems conve- niently.) Space-separated queries - When given multiple space-separated query terms, most commands select + When given multiple space-separated query terms, most commands select things which match: o any of the description terms AND @@ -5618,84 +5738,84 @@ Queries Boolean queries You can write more complicated "boolean" query expressions, enclosed in quotes and prefixed with expr:. These can combine subqueries with NOT, - AND, OR operators (case insensitive), and parentheses for grouping. + AND, OR operators (case insensitive), and parentheses for grouping. Eg, to show transactions involving both cash and expense accounts: hledger print expr:'cash AND expenses' - The prefix and enclosing quotes are required, so don't write hledger + The prefix and enclosing quotes are required, so don't write hledger print cash AND expenses. That would be a space-separated query showing - transactions involving accounts with any of "cash", "and", "expenses" + transactions involving accounts with any of "cash", "and", "expenses" in their names. - You can write space-separated queries inside a boolean query, and they - will combine as described above, but it might be confusing and best - avoided. Eg these are equivalent, showing transactions involving cash + You can write space-separated queries inside a boolean query, and they + will combine as described above, but it might be confusing and best + avoided. Eg these are equivalent, showing transactions involving cash or expenses accounts: hledger print expr:'cash expenses' hledger print cash expenses - There is a restriction with date: queries: they may not be used inside + There is a restriction with date: queries: they may not be used inside OR expressions. - Actually, there are three types of boolean query: expr: for general + Actually, there are three types of boolean query: expr: for general use, and any: and all: variants which can be useful with print. expr: query expr:'QUERYEXPR' - For example, expr:'date:lastmonth AND NOT (food OR rent)' means "match - things which are dated in the last month and do not have food or rent + For example, expr:'date:lastmonth AND NOT (food OR rent)' means "match + things which are dated in the last month and do not have food or rent in the account name". - When using expr: with transaction-oriented commands like print, post- - ing-oriented query terms like acct: and amt: are considered to match + When using expr: with transaction-oriented commands like print, post- + ing-oriented query terms like acct: and amt: are considered to match the transaction if they match any of its postings. - So, hledger print expr:'cash and amt:>0' means "show transactions with + So, hledger print expr:'cash and amt:>0' means "show transactions with (at least one posting involving a cash account) and (at least one post- ing with a positive amount)". any: query any:'QUERYEXPR' - Like expr:, but when used with transaction-oriented commands like - print, it matches the transaction only if a posting can be matched by + Like expr:, but when used with transaction-oriented commands like + print, it matches the transaction only if a posting can be matched by all of QUERYEXPR. - So, hledger print any:'cash and amt:>0' means "show transactions where + So, hledger print any:'cash and amt:>0' means "show transactions where at least one posting posts a positive amount to a cash account". all: query all:'QUERYEXPR' - Like expr:, but when used with transaction-oriented commands like - print, it matches the transaction only if all postings are matched by + Like expr:, but when used with transaction-oriented commands like + print, it matches the transaction only if all postings are matched by all of QUERYEXPR (and there is at least one posting). - So, hledger print all:'cash and amt:0' means "show transactions where + So, hledger print all:'cash and amt:0' means "show transactions where all postings involve a cash account and have a zero amount". Or, hledger print all:'cash or checking' means "show transactions which touch only cash and/or checking accounts". Queries and command options - Some queries can also be expressed as command-line options: depth:2 is + Some queries can also be expressed as command-line options: depth:2 is equivalent to --depth 2, date:2023 is equivalent to -p 2023, etc. When - you mix command options and query arguments, generally the resulting + you mix command options and query arguments, generally the resulting query is their intersection. Queries and account aliases - When account names are rewritten with --alias or alias, acct: will + When account names are rewritten with --alias or alias, acct: will match either the old or the new account name. Queries and valuation - When amounts are converted to other commodities in cost or value re- - ports, cur: and amt: match the old commodity symbol and the old amount + When amounts are converted to other commodities in cost or value re- + ports, cur: and amt: match the old commodity symbol and the old amount quantity, not the new ones. (Except in hledger 1.22, #1625.) Pivoting - Normally, hledger groups amounts and displays their totals by account - (name). With --pivot PIVOTEXPR, some other field's (or multiple - fields') value is used as a synthetic account name, causing different + Normally, hledger groups amounts and displays their totals by account + (name). With --pivot PIVOTEXPR, some other field's (or multiple + fields') value is used as a synthetic account name, causing different grouping and display. PIVOTEXPR can be - o any of these standard transaction or posting fields (their value is - substituted): status, code, desc, payee, note, acct, comm/cur, amt, + o any of these standard transaction or posting fields (their value is + substituted): status, code, desc, payee, note, acct, comm/cur, amt, cost o or a tag name @@ -5710,7 +5830,7 @@ Pivoting o When pivoting a posting that has multiple values for a tag, the tag's first value will be used as the pivoted value. - o When a posting has multiple commodities, the pivoted value of + o When a posting has multiple commodities, the pivoted value of "comm"/"cur" will be "". Also when an unrecognised tag name or field is provided, its pivoted value will be "". (If this causes confusing output, consider excluding those postings from the report.) @@ -5744,7 +5864,7 @@ Pivoting -------------------- -2 EUR - Another way (the acct: query matches against the pivoted "account + Another way (the acct: query matches against the pivoted "account name"): $ hledger balance --pivot member acct:. @@ -5760,26 +5880,29 @@ Pivoting -2 EUR Generating data - hledger can enrich the data provided to it, or generate new data, in a + hledger can enrich the data provided to it, or generate new data, in a number of ways. Mostly, this is done only if you request it: - o Missing amounts or missing costs in transactions are inferred auto- + o Missing amounts or missing costs in transactions are inferred auto- matically when possible. - o The --infer-equity flag infers missing conversion equity postings + o The --infer-equity flag infers missing conversion equity postings from @/@@ costs. - o The --infer-costs flag infers missing costs from conversion equity + o The --infer-costs flag infers missing costs from conversion equity postings. o The --infer-market-prices flag infers P price directives from costs. - o The --auto flag adds extra postings to transactions matched by auto + o The --auto flag adds extra postings to transactions matched by auto posting rules. - o The --forecast option generates transactions from periodic transac- + o The --forecast option generates transactions from periodic transac- tion rules. + o The --lots flag adds extra lot subaccounts to postings for detailed + lot reporting. + o The balance --budget report infers budget goals from periodic trans- action rules. @@ -5802,36 +5925,81 @@ Generating data so you can always match such data with queries like tag:generated or tag:modified. +Detecting special postings + hledger detects certain kinds of postings, both generated and non-gen- + erated, and tags them for additional processing. These are documented + elsewhere, but this section gives an overview of the special posting + detection rules. + + By default, the tags are hidden (with a _ prefix), so they can be + queried but they won't appear in print output. To also add visible + tags, use --verbose-tags (useful for troubleshooting). + + Tag Detected pattern Effect + ------------------------------------------------------------------------------------ + con- A pair of adjacent, single-commodity, Helps transaction bal- + ver- costless postings to Conversion-type ac- ancer infer costs or + sion-post- counts, with a nearby corresponding avoid redundancy in com- + ing costful or potentially corresponding modity conversions + costless posting + cost-post- A costful posting whose amount and Helps transaction bal- + ing transacted cost correspond to a conver- ancer infer costs or + sion postings pair; or a costless post- avoid redundancy in com- + ing matching one of the pair modity conversions + gener- Postings generated at runtime Helps users understand + ated-post- or find postings added + ing at runtime by hledger + ptype:ac- Positive postings with lot annotations, Creates a new lot + quire or in a lotful commodity/account, with + no matching counterposting + ptype:dis- Negative postings with lot annotations, Selects and reduces ex- + pose or in a lotful commodity/account, with isting lots + no matching counterposting + ptype:trans- The negative posting of a pair of coun- Moves lots between ac- + fer-from terpostings, at least one with lot anno- counts, preserving cost + tation or a lotful commodity/account; or basis + a negative lot posting with an equity + counterpart (equity transfer) + ptype:trans- The positive posting of a transfer pair; As above + fer-to or a positive lot posting with an equity + counterpart (equity transfer, e.g. + opening balances) + ptype:gain A posting to a Gain-type account Helps transaction bal- + ancer avoid redundancy, + helps disposal balancer + check realised capital + gain/loss + Forecasting - Forecasting, or speculative future reporting, can be useful for esti- + Forecasting, or speculative future reporting, can be useful for esti- mating future balances, or for exploring different future scenarios. The simplest and most flexible way to do it with hledger is to manually record a bunch of future-dated transactions. You could keep these in a - separate future.journal and include that with -f only when you want to + separate future.journal and include that with -f only when you want to see them. --forecast - There is another way: with the --forecast option, hledger can generate - temporary "forecast transactions" for reporting purposes, according to - periodic transaction rules defined in the journal. Each rule can gen- - erate multiple recurring transactions, so by changing one rule you can + There is another way: with the --forecast option, hledger can generate + temporary "forecast transactions" for reporting purposes, according to + periodic transaction rules defined in the journal. Each rule can gen- + erate multiple recurring transactions, so by changing one rule you can change many forecasted transactions. - Forecast transactions usually start after ordinary transactions end. + Forecast transactions usually start after ordinary transactions end. By default, they begin after your latest-dated ordinary transaction, or - today, whichever is later, and they end six months from today. (The + today, whichever is later, and they end six months from today. (The exact rules are a little more complicated, and are given below.) This is the "forecast period", which need not be the same as the report - period. You can override it - eg to forecast farther into the future, + period. You can override it - eg to forecast farther into the future, or to force forecast transactions to overlap your ordinary transactions - - by giving the --forecast option a period expression argument, like - --forecast=..2099 or --forecast=2023-02-15... Note that the = is re- + - by giving the --forecast option a period expression argument, like + --forecast=..2099 or --forecast=2023-02-15... Note that the = is re- quired. Inspecting forecast transactions - print is the best command for inspecting and troubleshooting forecast + print is the best command for inspecting and troubleshooting forecast transactions. Eg: ~ monthly from 2022-12-20 rent @@ -5865,7 +6033,7 @@ Forecasting expenses:rent $1000 Here there are no ordinary transactions, so the forecasted transactions - begin on the first occurrence after today's date. (You won't normally + begin on the first occurrence after today's date. (You won't normally use --today; it's just to make these examples reproducible.) Forecast reports @@ -5889,19 +6057,19 @@ Forecasting || $1000 $1000 $1000 $1000 $1000 Forecast tags - Forecast transactions generated by --forecast have a hidden tag, _gen- - erated-transaction. So if you ever need to match forecast transac- + Forecast transactions generated by --forecast have a hidden tag, _gen- + erated-transaction. So if you ever need to match forecast transac- tions, you could use tag:_generated-transaction (or just tag:generated) in a query. - For troubleshooting, you can add the --verbose-tags flag. Then, visi- + For troubleshooting, you can add the --verbose-tags flag. Then, visi- ble generated-transaction tags will be added also, so you can view them - with the print command. Their value indicates which periodic rule was + with the print command. Their value indicates which periodic rule was responsible. Forecast period, in detail Forecast start/end dates are chosen so as to do something useful by de- - fault in almost all situations, while also being flexible. Here are + fault in almost all situations, while also being flexible. Here are (with luck) the exact rules, to help with troubleshooting: The forecast period starts on: @@ -5933,7 +6101,7 @@ Forecasting o otherwise: 180 days (~6 months) from today. Forecast troubleshooting - When --forecast is not doing what you expect, one of these tips should + When --forecast is not doing what you expect, one of these tips should help: o Remember to use the --forecast option. @@ -5943,22 +6111,22 @@ Forecasting o Test with print --forecast. - o Check for typos or too-restrictive start/end dates in your periodic + o Check for typos or too-restrictive start/end dates in your periodic transaction rule. - o Leave at least 2 spaces between the rule's period expression and de- + o Leave at least 2 spaces between the rule's period expression and de- scription fields. - o Check for future-dated ordinary transactions suppressing forecasted + o Check for future-dated ordinary transactions suppressing forecasted transactions. o Try setting explicit report start and/or end dates with -b, -e, -p or date: - o Try adding the -E flag to encourage display of empty periods/zero + o Try adding the -E flag to encourage display of empty periods/zero transactions. - o Try setting explicit forecast start and/or end dates with --fore- + o Try setting explicit forecast start and/or end dates with --fore- cast=START..END o Consult Forecast period, in detail, above. @@ -5966,13 +6134,13 @@ Forecasting o Check inside the engine: add --debug=2 (eg). Budgeting - With the balance command's --budget report, each periodic transaction - rule generates recurring budget goals in specified accounts, and goals - and actual performance can be compared. See the balance command's doc + With the balance command's --budget report, each periodic transaction + rule generates recurring budget goals in specified accounts, and goals + and actual performance can be compared. See the balance command's doc below. - You can generate budget goals and forecast transactions at the same - time, from the same or different periodic transaction rules: hledger + You can generate budget goals and forecast transactions at the same + time, from the same or different periodic transaction rules: hledger bal -M --budget --forecast ... See also: Budgeting and Forecasting. @@ -5980,17 +6148,17 @@ Budgeting Amount formatting Commodity display style For the amounts in each commodity, hledger chooses a consistent display - style (symbol placement, decimal mark and digit group marks, number of + style (symbol placement, decimal mark and digit group marks, number of decimal digits) to use in most reports. This is inferred as follows: - First, if there's a D directive declaring a default commodity, that - commodity symbol and amount format is applied to all no-symbol amounts + First, if there's a D directive declaring a default commodity, that + commodity symbol and amount format is applied to all no-symbol amounts in the journal. - Then each commodity's display style is determined from its commodity - directive. We recommend always declaring commodities with commodity + Then each commodity's display style is determined from its commodity + directive. We recommend always declaring commodities with commodity directives, since they help ensure consistent display styles and preci- - sions, and bring other benefits such as error checking for commodity + sions, and bring other benefits such as error checking for commodity symbols. Here's an example: # Set display styles (and decimal marks, for parsing, if there is no decimal-mark directive) @@ -6000,9 +6168,9 @@ Amount formatting commodity INR 9,99,99,999.00 commodity 1 000 000.9455 - But for convenience, if a commodity directive is not present, hledger - infers a commodity's display styles from its amounts as they are writ- - ten in the journal (excluding cost amounts and amounts in periodic + But for convenience, if a commodity directive is not present, hledger + infers a commodity's display styles from its amounts as they are writ- + ten in the journal (excluding cost amounts and amounts in periodic transaction rules or auto posting rules). It uses o the symbol placement and decimal mark of the first amount seen @@ -6011,7 +6179,7 @@ Amount formatting o and the maximum number of decimal digits seen across all amounts. - And as fallback if no applicable amounts are found, it would use a de- + And as fallback if no applicable amounts are found, it would use a de- fault style, like $1000.00 (symbol on the left with no space, period as decimal mark, and two decimal digits). @@ -6020,16 +6188,16 @@ Amount formatting Rounding Amounts are stored internally as decimal numbers with up to 255 decimal - places. They are displayed with their original journal precisions by - print and print-like reports, and rounded to their display precision + places. They are displayed with their original journal precisions by + print and print-like reports, and rounded to their display precision (the number of decimal digits specified by the commodity display style) - by other reports. When rounding, hledger uses banker's rounding (it + by other reports. When rounding, hledger uses banker's rounding (it rounds to the nearest even digit). So eg 0.5 displayed with zero deci- mal digits appears as "0". Trailing decimal marks If you're wondering why your print report sometimes shows trailing dec- - imal marks, with no decimal digits; it does this when showing amounts + imal marks, with no decimal digits; it does this when showing amounts that have digit group marks but no decimal digits, to disambiguate them and allow them to be re-parsed reliably (see Decimal marks). Eg: @@ -6043,7 +6211,7 @@ Amount formatting (a) $1,000. If this is a problem (eg when exporting to Ledger), you can avoid it by - disabling digit group marks, eg with -c/--commodity (for each affected + disabling digit group marks, eg with -c/--commodity (for each affected commodity): $ hledger print -c '$1000.00' @@ -6060,19 +6228,19 @@ Amount formatting More generally, hledger output falls into three rough categories, which format amounts a little bit differently to suit different consumers: - 1. "hledger-readable output" - should be readable by hledger (and by + 1. "hledger-readable output" - should be readable by hledger (and by humans) - o This is produced by reports that show full journal entries: print, + o This is produced by reports that show full journal entries: print, import, close, rewrite etc. - o It shows amounts with their original journal precisions, which may + o It shows amounts with their original journal precisions, which may not be consistent from one amount to the next. - o It adds a trailing decimal mark when needed to avoid showing ambigu- + o It adds a trailing decimal mark when needed to avoid showing ambigu- ous amounts. - o It can be parsed reliably (by hledger and ledger2beancount at least, + o It can be parsed reliably (by hledger and ledger2beancount at least, but perhaps not by Ledger..) 2. "human-readable output" - usually for humans @@ -6084,13 +6252,13 @@ Amount formatting o It shows ambiguous amounts unmodified. - o It can be parsed reliably in the context of a known report (when you + o It can be parsed reliably in the context of a known report (when you know decimals are consistently not being shown, you can assume a sin- gle mark is a digit group mark). 3. "machine-readable output" - usually for other software - o This is produced by all reports when an output format like csv, tsv, + o This is produced by all reports when an output format like csv, tsv, json, or sql is selected. o It shows amounts as 1 or 2 do, but without digit group marks. @@ -6100,17 +6268,17 @@ Amount formatting Cost reporting In some transactions - for example a currency conversion, or a purchase - or sale of stock - one commodity is exchanged for another. In these - transactions there is a conversion rate, also called the cost (when - buying) or selling price (when selling). (In hledger docs we just say - "cost" generically for convenience.) With the -B/--cost flag, hledger + or sale of stock - one commodity is exchanged for another. In these + transactions there is a conversion rate, also called the cost (when + buying) or selling price (when selling). (In hledger docs we just say + "cost" generically for convenience.) With the -B/--cost flag, hledger can show amounts "at cost", converted to the cost's commodity. Recording costs - We'll explore several ways of recording transactions involving costs. + We'll explore several ways of recording transactions involving costs. These are also summarised at hledger Cookbook > Cost notation. - Costs can be recorded explicitly in the journal, using the @ UNITCOST + Costs can be recorded explicitly in the journal, using the @ UNITCOST or @@ TOTALCOST notation described in Journal > Costs: Variant 1 @@ -6125,11 +6293,11 @@ Cost reporting assets:dollars $-135 assets:euros 100 @@ $135 ; $135 total cost - Typically, writing the unit cost (variant 1) is preferable; it can be + Typically, writing the unit cost (variant 1) is preferable; it can be more effort, requiring more attention to decimal digits; but it reveals the per-unit cost basis, and makes stock sales easier. - Costs can also be left implicit, and hledger will infer the cost that + Costs can also be left implicit, and hledger will infer the cost that is consistent with a balanced transaction: Variant 3 @@ -6138,49 +6306,49 @@ Cost reporting assets:dollars $-135 assets:euros 100 - Here, hledger will attach a @@ 100 cost to the first amount (you can - see it with hledger print -x). This form looks convenient, but there + Here, hledger will attach a @@ 100 cost to the first amount (you can + see it with hledger print -x). This form looks convenient, but there are downsides: - o It sacrifices some error checking. For example, if you accidentally + o It sacrifices some error checking. For example, if you accidentally wrote 10 instead of 100, hledger would not be able to detect the mis- take. - o It is sensitive to the order of postings - if they were reversed, a + o It is sensitive to the order of postings - if they were reversed, a different entry would be inferred and reports would be different. o The per-unit cost basis is not easy to read. - So generally this kind of entry is not recommended. You can make sure + So generally this kind of entry is not recommended. You can make sure you have none of these by using -s (strict mode), or by running hledger check balanced. Reporting at cost - Now when you add the -B/--cost flag to reports ("B" is from Ledger's - -B/--basis/--cost flag), any amounts which have been annotated with - costs will be converted to their cost's commodity (in the report out- + Now when you add the -B/--cost flag to reports ("B" is from Ledger's + -B/--basis/--cost flag), any amounts which have been annotated with + costs will be converted to their cost's commodity (in the report out- put). Ie they will be displayed "at cost" or "at sale price". Some things to note: - o Costs are attached to specific posting amounts in specific transac- - tions, and once recorded they do not change. This contrasts with + o Costs are attached to specific posting amounts in specific transac- + tions, and once recorded they do not change. This contrasts with market prices, which are ambient and fluctuating. - o Conversion to cost is performed before conversion to market value + o Conversion to cost is performed before conversion to market value (described below). Equity conversion postings - There is a problem with the entries above - they are not conventional - Double Entry Bookkeeping (DEB) notation, and because of the "magical" - transformation of one commodity into another, they cause an imbalance + There is a problem with the entries above - they are not conventional + Double Entry Bookkeeping (DEB) notation, and because of the "magical" + transformation of one commodity into another, they cause an imbalance in the Accounting Equation. This shows up as a non-zero grand total in balance reports like hledger bse. - For most hledger users, this doesn't matter in practice and can safely + For most hledger users, this doesn't matter in practice and can safely be ignored ! But if you'd like to learn more, keep reading. - Conventional DEB uses an extra pair of equity postings to balance the + Conventional DEB uses an extra pair of equity postings to balance the transaction. Of course you can do this in hledger as well: Variant 4 @@ -6191,10 +6359,10 @@ Cost reporting equity:conversion $135 equity:conversion -100 - Now the transaction is perfectly balanced according to standard DEB, + Now the transaction is perfectly balanced according to standard DEB, and hledger bse's total will not be disrupted. - And, hledger can still infer the cost for cost reporting, but it's not + And, hledger can still infer the cost for cost reporting, but it's not done by default - you must add the --infer-costs flag like so: $ hledger print --infer-costs @@ -6216,14 +6384,14 @@ Cost reporting o Instead of -B you must remember to type -B --infer-costs. - o --infer-costs works only where hledger can identify the two eq- - uity:conversion postings and match them up with the two non-equity - postings. So writing the journal entry in a particular format be- + o --infer-costs works only where hledger can identify the two eq- + uity:conversion postings and match them up with the two non-equity + postings. So writing the journal entry in a particular format be- comes more important. More on this below. Inferring equity conversion postings Can we go in the other direction ? Yes, if you have transactions writ- - ten with the @/@@ cost notation, hledger can infer the missing equity + ten with the @/@@ cost notation, hledger can infer the missing equity postings, if you add the --infer-equity flag. Eg: 2022-01-01 @@ -6237,18 +6405,20 @@ Cost reporting equity:conversion:$-: -100 equity:conversion:$-:$ $135.00 - The equity account names will be "equity:conversion:A-B:A" and "eq- - uity:conversion:A-B:B" where A is the alphabetically first commodity + The equity account names will be "equity:conversion:A-B:A" and "eq- + uity:conversion:A-B:B" where A is the alphabetically first commodity symbol. You can customise the "equity:conversion" part by declaring an account with the V/Conversion account type. - Note you will need to add account declarations for these to your jour- - nal, if you use check accounts or check --strict. + Note you will need to add account declarations for these to your jour- + nal, if you use check accounts or check --strict. (And unlike normal + postings, generated equity postings do not inherit tags from account + declarations.) Combining costs and equity conversion postings Finally, you can use both the @/@@ cost notation and equity postings at - the same time. This in theory gives the best of all worlds - preserv- - ing the accounting equation, revealing the per-unit cost basis, and + the same time. This in theory gives the best of all worlds - preserv- + ing the accounting equation, revealing the per-unit cost basis, and providing more flexibility in how you write the entry: Variant 5 @@ -6259,15 +6429,15 @@ Cost reporting equity:conversion -100 assets:euros 100 @ $1.35 - All the other variants above can (usually) be rewritten to this final + All the other variants above can (usually) be rewritten to this final form with: $ hledger print -x --infer-costs --infer-equity Downsides: - o The precise format of the journal entry becomes more important. If - hledger can't detect and match up the cost and equity postings, it + o The precise format of the journal entry becomes more important. If + hledger can't detect and match up the cost and equity postings, it will give a transaction balancing error. o The add command does not yet accept this kind of entry (#2056). @@ -6275,34 +6445,34 @@ Cost reporting o This is the most verbose form. Requirements for detecting equity conversion postings - --infer-costs has certain requirements (unlike --infer-equity, which + --infer-costs has certain requirements (unlike --infer-equity, which always works). It will infer costs only in transactions with: - o Two non-equity postings, in different commodities. Their order is + o Two non-equity postings, in different commodities. Their order is significant: the cost will be added to the first of them. - o Two postings to equity conversion accounts, next to one another, + o Two postings to equity conversion accounts, next to one another, which balance the two non-equity postings. This balancing is checked - to the same precision (number of decimal places) used in the conver- + to the same precision (number of decimal places) used in the conver- sion posting's amount. Equity conversion accounts are: o any accounts declared with account type V/Conversion, or their sub- accounts - o otherwise, accounts named equity:conversion, equity:trade, or eq- + o otherwise, accounts named equity:conversion, equity:trade, or eq- uity:trading, or their subaccounts. - And multiple such four-posting groups can coexist within a single - transaction. When --infer-costs fails, it does not infer a cost in - that transaction, and does not raise an error (ie, it infers costs + And multiple such four-posting groups can coexist within a single + transaction. When --infer-costs fails, it does not infer a cost in + that transaction, and does not raise an error (ie, it infers costs where it can). - Reading variant 5 journal entries, combining cost notation and equity - postings, has all the same requirements. When reading such an entry + Reading variant 5 journal entries, combining cost notation and equity + postings, has all the same requirements. When reading such an entry fails, hledger raises an "unbalanced transaction" error. Infer cost and equity by default ? - Should --infer-costs and --infer-equity be enabled by default ? Try + Should --infer-costs and --infer-equity be enabled by default ? Try using them always, eg with a shell alias: alias h="hledger --infer-equity --infer-costs" @@ -6310,26 +6480,26 @@ Cost reporting and let us know what problems you find. Value reporting - hledger can also show amounts "at market value", converted to some - other commodity using the market price or conversion rate on a certain + hledger can also show amounts "at market value", converted to some + other commodity using the market price or conversion rate on a certain date. - This is controlled by the --value=TYPE[,COMMODITY] option. We also - provide simpler -V and -X COMMODITY aliases for this, which are often + This is controlled by the --value=TYPE[,COMMODITY] option. We also + provide simpler -V and -X COMMODITY aliases for this, which are often sufficient. The market prices are declared with a special P directive, and/or they can be inferred from the costs recorded in transactions, by using the --infer-market-prices flag. -X: Value in specified commodity - The -X COMM (or --exchange=COMM) option converts amounts to their mar- + The -X COMM (or --exchange=COMM) option converts amounts to their mar- ket value in the specified commodity, using the market prices in effect on the valuation date(s), if any. (More on these in a minute.) Use this when you want to (eg) show everything in your base currency as - far as possible. (Commodities for which no conversion rate can be + far as possible. (Commodities for which no conversion rate can be found, will not be converted.) - COMM should be the full commodity symbol or name. Remember to quote + COMM should be the full commodity symbol or name. Remember to quote special shell characters, if needed. Some examples: o -X @@ -6345,90 +6515,90 @@ Value reporting -V: Value in default commodity(s) The -V/--market flag is a variant of -X where you don't have to specify COMM. Instead it tries to guess a default valuation commodity for each - original commodity, based on the market prices in effect on the valua- + original commodity, based on the market prices in effect on the valua- tion date(s). - -V can often be a convenient shortcut for -X MYCURRENCY, but not al- + -V can often be a convenient shortcut for -X MYCURRENCY, but not al- ways; depending on your data it could guess multiple valuation commodi- - ties. Usually you want to convert to a single commodity, so it's bet- + ties. Usually you want to convert to a single commodity, so it's bet- ter to use -X, unless you're sure -V is doing what you want. Valuation date - Market prices can change from day to day. hledger will use the prices - on a particular valuation date (or on more than one date). By default + Market prices can change from day to day. hledger will use the prices + on a particular valuation date (or on more than one date). By default hledger uses "end" dates for valuation. More specifically: - o For single period reports (including normal print and register re- + o For single period reports (including normal print and register re- ports): o If an explicit report end date is specified, that is used. - o Otherwise the latest transaction date or non-future P directive + o Otherwise the latest transaction date or non-future P directive date is used. o For multiperiod reports, each period is valued on its last day. - This can be customised with the --value option described below, which + This can be customised with the --value option described below, which can select either "then", "end", "now", or "custom" dates. Finding market price - To convert a commodity A to its market value in another commodity B, - hledger looks for a suitable market price (exchange rate) as follows, + To convert a commodity A to its market value in another commodity B, + hledger looks for a suitable market price (exchange rate) as follows, in this order of preference: - 1. A declared market price or inferred market price: A's latest market + 1. A declared market price or inferred market price: A's latest market price in B on or before the valuation date as declared by a P direc- tive, or (with the --infer-market-prices flag) inferred from costs. 2. A reverse market price: the inverse of a declared or inferred market price from B to A. - 3. A forward chain of market prices: a synthetic price formed by com- + 3. A forward chain of market prices: a synthetic price formed by com- bining the shortest chain of "forward" (only 1 above) market prices, leading from A to B. - 4. Any chain of market prices: a chain of any market prices, including - both forward and reverse prices (1 and 2 above), leading from A to + 4. Any chain of market prices: a chain of any market prices, including + both forward and reverse prices (1 and 2 above), leading from A to B. - There is a limit to the length of these price chains; if hledger - reaches that length without finding a complete chain or exhausting all - possibilities, it will give up (with a "gave up" message visible in + There is a limit to the length of these price chains; if hledger + reaches that length without finding a complete chain or exhausting all + possibilities, it will give up (with a "gave up" message visible in --debug=2 output). That limit is currently 1000. - Amounts for which no suitable market price can be found, are not con- + Amounts for which no suitable market price can be found, are not con- verted. --infer-market-prices: market prices from transactions Normally, market value in hledger is fully controlled by, and requires, P directives in your journal. Since adding and updating those can be a - chore, and since transactions usually take place at close to market - value, why not use the recorded costs as additional market prices (as - Ledger does) ? Adding the --infer-market-prices flag to -V, -X or + chore, and since transactions usually take place at close to market + value, why not use the recorded costs as additional market prices (as + Ledger does) ? Adding the --infer-market-prices flag to -V, -X or --value enables this. - So for example, hledger bs -V --infer-market-prices will get market - prices both from P directives and from transactions. If both occur on + So for example, hledger bs -V --infer-market-prices will get market + prices both from P directives and from transactions. If both occur on the same day, the P directive takes precedence. There is a downside: value reports can sometimes be affected in confus- - ing/undesired ways by your journal entries. If this happens to you, - read all of this Value reporting section carefully, and try adding + ing/undesired ways by your journal entries. If this happens to you, + read all of this Value reporting section carefully, and try adding --debug or --debug=2 to troubleshoot. --infer-market-prices can infer market prices from: o multicommodity transactions with explicit prices (@/@@) - o multicommodity transactions with implicit prices (no @, two commodi- - ties, unbalanced). (With these, the order of postings matters. + o multicommodity transactions with implicit prices (no @, two commodi- + ties, unbalanced). (With these, the order of postings matters. hledger print -x can be useful for troubleshooting.) o multicommodity transactions with equity postings, if cost is inferred with --infer-costs. - There is a limitation (bug) currently: when a valuation commodity is - not specified, prices inferred with --infer-market-prices do not help + There is a limitation (bug) currently: when a valuation commodity is + not specified, prices inferred with --infer-market-prices do not help select a default valuation commodity, as P prices would. So conversion might not happen because no valuation commodity was detected (--debug=2 will show this). To be safe, specify the valuation commmodity, eg: @@ -6438,8 +6608,8 @@ Value reporting o --value=then,EUR --infer-market-prices, not --value=then --infer-mar- ket-prices - Signed costs and market prices can be confusing. For reference, here - is the current behaviour, since hledger 1.25. (If you think it should + Signed costs and market prices can be confusing. For reference, here + is the current behaviour, since hledger 1.25. (If you think it should work differently, see #1870.) 2022-01-01 Positive Unit prices @@ -6469,7 +6639,7 @@ Value reporting b B -1 @@ A -1 All of the transactions above are considered balanced (and on each day, - the two transactions are considered equivalent). Here are the market + the two transactions are considered equivalent). Here are the market prices inferred for B: $ hledger -f- --infer-market-prices prices @@ -6482,34 +6652,34 @@ Value reporting Valuation commodity When you specify a valuation commodity (-X COMM or --value TYPE,COMM): - hledger will convert all amounts to COMM, wherever it can find a suit- + hledger will convert all amounts to COMM, wherever it can find a suit- able market price (including by reversing or chaining prices). - When you leave the valuation commodity unspecified (-V or --value + When you leave the valuation commodity unspecified (-V or --value TYPE): - For each commodity A, hledger picks a default valuation commodity as + For each commodity A, hledger picks a default valuation commodity as follows, in this order of preference: 1. The price commodity from the latest P-declared market price for A on or before valuation date. 2. The price commodity from the latest P-declared market price for A on - any date. (Allows conversion to proceed when there are inferred + any date. (Allows conversion to proceed when there are inferred prices before the valuation date.) - 3. If there are no P directives at all (any commodity or date) and the - --infer-market-prices flag is used: the price commodity from the + 3. If there are no P directives at all (any commodity or date) and the + --infer-market-prices flag is used: the price commodity from the latest transaction-inferred price for A on or before valuation date. This means: - o If you have P directives, they determine which commodities -V will + o If you have P directives, they determine which commodities -V will convert, and to what. - o If you have no P directives, and use the --infer-market-prices flag, + o If you have no P directives, and use the --infer-market-prices flag, costs determine it. - Amounts for which no valuation commodity can be found are not con- + Amounts for which no valuation commodity can be found are not con- verted. --value: Flexible valuation @@ -6526,26 +6696,26 @@ Value reporting The TYPE part selects cost or value and valuation date: --value=then - Convert amounts to their value in the default valuation commod- + Convert amounts to their value in the default valuation commod- ity, using market prices on each posting's date. --value=end - Convert amounts to their value in the default valuation commod- - ity, using market prices on the last day of the report period - (or if unspecified, the journal's end date); or in multiperiod + Convert amounts to their value in the default valuation commod- + ity, using market prices on the last day of the report period + (or if unspecified, the journal's end date); or in multiperiod reports, market prices on the last day of each subperiod. --value=now - Convert amounts to their value in the default valuation commod- - ity using current market prices (as of when report is gener- + Convert amounts to their value in the default valuation commod- + ity using current market prices (as of when report is gener- ated). --value=YYYY-MM-DD - Convert amounts to their value in the default valuation commod- + Convert amounts to their value in the default valuation commod- ity using market prices on this date. To select a different valuation commodity, add the optional ,COMM part: - a comma, then the target commodity's symbol. Eg: --value=now,EUR. + a comma, then the target commodity's symbol. Eg: --value=now,EUR. hledger will do its best to convert amounts to this commodity, deducing market prices as described above. @@ -6573,13 +6743,13 @@ Value reporting $ hledger -f t.j bal -N euros -V -e 2016/11/4 $110.00 assets:euros - What are they worth after 2016/12/21 ? (no report end date specified, + What are they worth after 2016/12/21 ? (no report end date specified, defaults to today) $ hledger -f t.j bal -N euros -V $103.00 assets:euros - Here are some examples showing the effect of --value, as seen with + Here are some examples showing the effect of --value, as seen with print: P 2000-01-01 A 1 B @@ -6617,7 +6787,7 @@ Value reporting 2000-02-01 (a) 2 B - With no report period specified, the latest transaction date or price + With no report period specified, the latest transaction date or price date is used as valuation date (2000-04-01): $ hledger -f- print --value=end @@ -6655,7 +6825,7 @@ Value reporting (a) 1 B Interaction of valuation and queries - When matching postings based on queries in the presence of valuation, + When matching postings based on queries in the presence of valuation, the following happens: 1. The query is separated into two parts: @@ -6669,14 +6839,14 @@ Value reporting 3. Valuation is applied to the postings. - 4. The postings are matched to the other parts of the query based on + 4. The postings are matched to the other parts of the query based on post-valued amounts. Related: #1625 Effect of valuation on reports - Here is a reference for how valuation is supposed to affect each part - of hledger's reports. It may be useful when troubleshooting. If you + Here is a reference for how valuation is supposed to affect each part + of hledger's reports. It may be useful when troubleshooting. If you find problems, please report them, ideally with a reproducible example. Related: #329, #1083. @@ -6684,29 +6854,29 @@ Value reporting cost calculated using price(s) recorded in the transaction(s). - value market value using available market price declarations, or the + value market value using available market price declarations, or the unchanged amount if no conversion rate can be found. report start - the first day of the report period specified with -b or -p or + the first day of the report period specified with -b or -p or date:, otherwise today. report or journal start - the first day of the report period specified with -b or -p or - date:, otherwise the earliest transaction date in the journal, + the first day of the report period specified with -b or -p or + date:, otherwise the earliest transaction date in the journal, otherwise today. report end - the last day of the report period specified with -e or -p or + the last day of the report period specified with -e or -p or date:, otherwise today. report or journal end - the last day of the report period specified with -e or -p or - date:, otherwise the latest transaction date in the journal, + the last day of the report period specified with -e or -p or + date:, otherwise the latest transaction date in the journal, otherwise today. report interval - a flag (-D/-W/-M/-Q/-Y) or period expression that activates the + a flag (-D/-W/-M/-Q/-Y) or period expression that activates the report's multi-period mode (whether showing one or many subperi- ods). @@ -6714,8 +6884,8 @@ Value reporting type --value=now -------------------------------------------------------------------------------------------- print - posting cost value at re- value at posting value at re- value at - amounts port end or date port or DATE/today + posting cost value at re- value at posting value at re- value at + amounts port end or date port or DATE/today today journal end balance unchanged unchanged unchanged unchanged unchanged asser- @@ -6731,7 +6901,7 @@ Value reporting (-H) with port or posting was made port or report journal journal interval start start - posting cost value at re- value at posting value at re- value at + posting cost value at re- value at posting value at re- value at amounts port or date port or DATE/today journal end journal end summary summarised value at pe- sum of postings value at pe- value at @@ -6747,8 +6917,8 @@ Value reporting balance (bs, bse, cf, is) - balance sums of value at re- value at posting value at re- value at - changes costs port end or date port or DATE/today of + balance sums of value at re- value at posting value at re- value at + changes costs port end or date port or DATE/today of today of journal end sums of post- sums of of sums of ings postings postings @@ -6756,7 +6926,7 @@ Value reporting amounts changes changes changes ances changes (--bud- get) - grand to- sum of dis- sum of dis- sum of displayed sum of dis- sum of dis- + grand to- sum of dis- sum of dis- sum of displayed sum of dis- sum of dis- tal played val- played val- valued played val- played values ues ues ues @@ -6782,7 +6952,7 @@ Value reporting end bal- sums of same as sums of values of period end value at ances costs of --value=end postings from be- balances, DATE/today of (bal -H, postings fore period start valued at sums of post- - is --H, from before to period end at period ends ings + is --H, from before to period end at period ends ings bs, cf) report start respective post- to period ing dates end @@ -6791,10 +6961,10 @@ Value reporting (--bud- balances balances ances balances get) row to- sums, aver- sums, aver- sums, averages of sums, aver- sums, aver- - tals, row ages of dis- ages of dis- displayed values ages of dis- ages of dis- + tals, row ages of dis- ages of dis- displayed values ages of dis- ages of dis- averages played val- played val- played val- played values (-T, -A) ues ues ues - column sums of dis- sums of dis- sums of displayed sums of dis- sums of dis- + column sums of dis- sums of dis- sums of displayed sums of dis- sums of dis- totals played val- played val- values played val- played values ues ues ues grand to- sum, average sum, average sum, average of sum, average sum, average @@ -6806,15 +6976,239 @@ Value reporting --cumulative is omitted to save space, it works like -H but with a zero starting balance. +Lot reporting + With the --lots flag, hledger can track investment lots automatically: + assigning lot subaccounts on acquisition, selecting lots on disposal + using configurable methods, calculating capital gains, and showing + per-lot balances in all reports. (Since 1.99.1, experimental. For + more technical details, see SPEC-lots.md). + + Lotful commodities and accounts + Commodities and accounts can be declared as "lotful" by adding a lots + tag in their declaration: + + commodity AAPL ; lots: + account assets:stocks ; lots: + + This tells hledger that postings involving these always involve lots, + enabling cost basis inference even when lot syntax is not written ex- + plicitly. + + The tag value can also specify a reduction method: + + commodity AAPL ; lots: FIFO + account assets:stocks ; lots: LIFO + + If no value is specified, the default is FIFO. + + --lots + Add --lots to any command to enable lot tracking. This activates: + + o Lot posting classification -- lot-related postings are tagged as ac- + quire, dispose, transfer-from, transfer-to, or gain (via a hidden + ptype tag, visible with --verbose-tags, queryable with + tag:ptype=...). + + o Cost basis inference -- for lotful commodities/accounts, cost basis + is inferred from transacted cost and vice versa. Or when the account + name ends with a lot subaccount, cost basis can also be inferred from + that. + + o Lot calculation -- acquired lots become subaccounts; disposals and + transfers select from existing lots. + + o Disposal balancing -- disposal transactions are checked for balance + at cost basis; gain amounts/postings are inferred if missing. + + Lot subaccounts + With --lots, each acquired lot becomes a subaccount named by its cost + basis: + + commodity AAPL ; lots: + + 2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:cash -$500 + + $ hledger bal assets:stocks --lots -N + 10 AAPL assets:stocks:{2026-01-15, $50} + + You can also write lot subaccounts explicitly. When a posting's ac- + count name ends with a lot subaccount (like :{2026-01-15, $50}), the + cost basis is parsed from it automatically, so a {} annotation on the + amount is optional: + + commodity AAPL ; lots: + + 2026-01-15 buy + assets:stocks:{2026-01-15, $50} 10 AAPL + assets:cash + + This is equivalent to writing 10 AAPL {2026-01-15, $50}. (If both the + account name and the amount specify a cost basis, they must agree.) + + When strictly checking account names, lot subaccounts are automatically + exempt -- you only need to declare the base account (eg account as- + sets:stocks), not each individual lot subaccount. + + Lot operations + o Acquire: a positive lot posting creates a new lot. The cost basis + can be specified explicitly with {} on the amount, inferred from the + lot subaccount name, or inferred from the transacted cost. On lotful + commodities/accounts, even a bare positive posting (no {} or @) can + be detected as an acquire, with cost inferred from the transaction's + other postings. + + o Transfer: a matching pair of negative/positive lot postings moves a + lot between accounts, preserving its cost basis. Transfer postings + should not have a transacted price. + + o Dispose: a negative lot posting sells from one or more existing lots. + It must have a transacted price (the selling price), either explicit + or inferred. + + An example disposal entry: + + 2026-02-01 sell + assets:stocks -5 AAPL {$50} @ $60 + assets:cash $300 + revenue:gains -$50 + + With --lots, this selects the specified quantity of the matching lot + (which must exist) and will show something like: + + $ hledger print --lots desc:sell + 2026-02-01 sell + assets:stocks:{2026-01-15, $50} -5 AAPL {2026-01-15, $50} @ $60 + assets:cash $300 + revenue:gains $-50 + + Reduction methods + When a disposal or transfer doesn't specify a particular lot (eg the + amount is -5 AAPL or -5 AAPL {}), hledger selects lot(s) automatically + using a reduction method. The available methods are: + + Method Lots selected Scope Disposal cost basis + --------------------------------------------------------------------------------- + FIFO (default) oldest first across all ac- each lot's cost + counts + FIFO1 oldest first within one account each lot's cost + LIFO newest first across all ac- each lot's cost + counts + LIFO1 newest first within one account each lot's cost + HIFO highest cost across all ac- each lot's cost + first counts + HIFO1 highest cost within one account each lot's cost + first + AVERAGE oldest first across all ac- weighted average + counts cost + AVERAGE1 oldest first within one account weighted average + cost + SPECID one specified lot specified account specified lot's + cost + + An explicit lot selector (eg {2026-01-15, $50} or {$50}) uses spe- + cific-identification (SPECID). + + HIFO (highest-in-first-out) selects the lot with the highest per-unit + cost first, which can be useful for tax optimization. + + AVERAGE uses the weighted average per-unit cost of the entire pool as + the disposal cost basis, rather than each lot's individual cost. This + is required in some jurisdictions (eg Canada's Adjusted Cost Base, + France's PMPA, UK's S104 pools). Lots are still consumed in FIFO order + for bookkeeping purposes. Configure the method via the lots: tag on a + commodity or account declaration: + + commodity AAPL ; lots: FIFO + account assets:stocks ; lots: AVERAGE + + Account tags override commodity tags. + + Lot postings with balance assertions + On a dispose or transfer posting without an explicit lot subaccount, a + balance assertion always refers to the parent account's balance. So if + lot subaccounts are added witih --lots, the assertion is not affected. + + By contrast, in a journal entry where the lot subaccounts are recorded + explicitly, a balance assertion refers to the lot subaccount's balance. + + This means that hledger print --lots, if it adds explicit lot subac- + counts to a journal entry, could potentially change the meaning of bal- + ance assertions, breaking them. To avoid this, in such cases it will + move the balance assertion to a new zero-amount posting to the parent + account (and make sure it's subaccount-inclusive). (So eg hledger -f- + print --lots -x | hledger -f- check assertions will still pass.) + + Gain postings and disposal balancing + A gain posting is a posting to a Gain-type account (type G, a subtype + of Revenue). In disposal transactions, it records the capital gain or + loss, which is the difference between cost basis and selling price of + the lots being sold. + + Accounts named like revenue:gains or income:capital-gains are detected + as Gain accounts automatically, or you can declare one explicitly: + + account gain/loss ; type: G + + Gain postings have special treatment: + + o Normal transaction balancing ignores gain postings (they don't count + toward the balance check), and balances the transaction using trans- + acted price + + o Disposal balancing (with --lots) includes gain postings, and balances + the transaction using cost basis + + An amountless gain posting in a disposal transaction will have its + amount filled in. Or if a disposal transaction is unbalanced at cost + basis and has no gain posting, one is inferred automatically (posting + to the first Gain account, or revenue:gains if none is declared). + + Lot reporting example + commodity AAPL ; lots: + + 2026-01-15 buy low + assets:stocks 10 AAPL {$50} + assets:cash -$500 + + 2026-02-01 buy high + assets:stocks 10 AAPL {$60} + assets:cash -$600 + + 2026-03-01 sell some (FIFO, selects oldest lot first) + assets:stocks -5 AAPL @ $70 + assets:cash $350 + revenue:gains + + $ hledger bal assets:stocks --lots -N + 5 AAPL assets:stocks:{2026-01-15, $50} + 10 AAPL assets:stocks:{2026-02-01, $60} + + $ hledger print --lots -x desc:sell + 2026-03-01 sell some (FIFO, selects oldest lot first) + assets:stocks:{2026-01-15, $50} -5 AAPL {2026-01-15, $50} @ $70 + assets:cash $350 + revenue:gains -$100 + + $ hledger print --lots -x desc:sell --verbose-tags + 2026-03-01 sell some (FIFO, selects oldest lot first) + assets:stocks:{2026-01-15, $50} -5 AAPL {2026-01-15, $50} @ $70 ; ptype: dispose + assets:cash $350 + revenue:gains $-100 ; ptype: gain + + The gain of $100 was inferred: 5 shares acquired at $50, sold at $70 = + 5 ($70 - $50) = $100. + PART 4: COMMANDS Here are hledger's standard subcommands. You can list these by running hledger. If you have installed more add-on commands, they also will be listed. - In the following command docs, each command's specific options are + In the following command docs, each command's specific options are shown. Most commands also support the general options described above, though some of them might have no effect. (Usually if there's a sensi- - ble way for a general option to affect a command, it will.) You can + ble way for a general option to affect a command, it will.) You can list all of a command's options by running hledger CMD -h. Help commands @@ -6869,7 +7263,7 @@ PART 4: COMMANDS o aregister (areg) - show transactions in a particular account - o register (reg) - show postings in one or more accounts & running to- + o register (reg) - show postings in one or more accounts & running to- tal o balancesheet (bs) - show assets, liabilities and net worth @@ -6922,16 +7316,16 @@ Help commands -s --speed=SPEED playback speed (1 is original speed, .5 is half, 2 is double, etc (default: 2)) - Run this command with no argument to list the demos. To play a demo, + Run this command with no argument to list the demos. To play a demo, write its number or a prefix or substring of its title. Tips: Make your terminal window large enough to see the demo clearly. - Use the -s/--speed SPEED option to set your preferred playback speed, + Use the -s/--speed SPEED option to set your preferred playback speed, eg -s4 to play at 4x original speed or -s.5 to play at half speed. The default speed is 2x. - During playback, several keys are available: SPACE to pause/unpause, . + During playback, several keys are available: SPACE to pause/unpause, . to step forward (while paused), CTRL-c quit. Examples: @@ -6943,7 +7337,7 @@ Help commands This command is experimental: there aren't many useful demos yet. help - Show the hledger user manual with info, man, or a pager. With a (case + Show the hledger user manual with info, man, or a pager. With a (case insensitive) TOPIC argument, try to open it at that section heading. Flags: @@ -6952,23 +7346,23 @@ Help commands -p show the manual with $PAGER or less (less is always used if TOPIC is specified) - This command shows the hledger manual built in to your hledger exe- - cutable. It can be useful when offline, or when you prefer the termi- + This command shows the hledger manual built in to your hledger exe- + cutable. It can be useful when offline, or when you prefer the termi- nal to a web browser, or when the appropriate hledger manual or viewers are not installed properly on your system. - By default it chooses the best viewer found in $PATH, trying in this - order: info, man, $PAGER, less, more, stdout. (If a TOPIC is speci- - fied, $PAGER and more are not tried.) You can force the use of info, - man, or a pager with the -i, -m, or -p flags. If no viewer can be - found, or if running non-interactively, it just prints the manual to + By default it chooses the best viewer found in $PATH, trying in this + order: info, man, $PAGER, less, more, stdout. (If a TOPIC is speci- + fied, $PAGER and more are not tried.) You can force the use of info, + man, or a pager with the -i, -m, or -p flags. If no viewer can be + found, or if running non-interactively, it just prints the manual to stdout. - When using info, TOPIC can match either the full heading or a prefix. + When using info, TOPIC can match either the full heading or a prefix. If your info --version is < 6, you'll need to upgrade it, eg with 'brew install texinfo' on mac. - When using man or less, TOPIC must match the full heading. For a pre- + When using man or less, TOPIC must match the full heading. For a pre- fix match, you can write 'TOPIC.*'. Examples @@ -6980,7 +7374,7 @@ Help commands User interface commands repl - Start an interactive prompt, where you can run any of hledger's com- + Start an interactive prompt, where you can run any of hledger's com- mands. Data files are parsed just once, so the commands run faster. Flags: @@ -6988,21 +7382,21 @@ User interface commands This command is experimental and could change in the future. - hledger repl starts a read-eval-print loop (REPL) where you can enter - commands interactively. As with the run command, each input file (or + hledger repl starts a read-eval-print loop (REPL) where you can enter + commands interactively. As with the run command, each input file (or each input file/input options combination) is parsed just once, so com- - mands will run more quickly than if you ran them individually at the + mands will run more quickly than if you ran them individually at the command line. Also like run, the input file(s) specified for the repl command will be - the default input for all interactive commands. You can override this - temporarily by specifying an -f option in particular commands. But - note that commands will not see any changes made to input files (eg by + the default input for all interactive commands. You can override this + temporarily by specifying an -f option in particular commands. But + note that commands will not see any changes made to input files (eg by add) until you exit and restart the REPL. The command syntax is the same as with run: - o enter one hledger command at a time, without the usual hledger first + o enter one hledger command at a time, without the usual hledger first word o empty lines and comment text from # to end of line are ignored @@ -7011,7 +7405,7 @@ User interface commands o type exit or quit or control-D to exit the REPL. - While it is running, the REPL remembers your command history, and you + While it is running, the REPL remembers your command history, and you can navigate in the usual ways: o Keypad or Emacs navigation keys to edit the current command line @@ -7022,9 +7416,9 @@ User interface commands o TAB to complete file paths. - Generally repl command lines should feel much like the normal hledger - CLI, but you may find differences. repl is a little stricter; eg it - requires full command names or official abbreviations (as seen in the + Generally repl command lines should feel much like the normal hledger + CLI, but you may find differences. repl is a little stricter; eg it + requires full command names or official abbreviations (as seen in the commands list). The commands and help commands, and the command help flags (CMD --tldr, @@ -7033,7 +7427,7 @@ User interface commands You can type control-C to cancel a long-running command (but only once; typing it a second time will exit the REPL). - And in most shells you can type control-Z to temporarily exit to the + And in most shells you can type control-Z to temporarily exit to the shell (and then fg to return to the REPL). Examples @@ -7063,8 +7457,8 @@ User interface commands ... run - Run a sequence of hledger commands, provided as files or command line - arguments. Data files are parsed just once, so the commands run + Run a sequence of hledger commands, provided as files or command line + arguments. Data files are parsed just once, so the commands run faster. Flags: @@ -7074,52 +7468,52 @@ User interface commands You can use run in three ways: - o hledger run -- CMD1 -- CMD2 -- CMD3 - read commands from the command + o hledger run -- CMD1 -- CMD2 -- CMD3 - read commands from the command line, separated by -- - o hledger run SCRIPTFILE1 SCRIPTFILE2 - read commands from one or more + o hledger run SCRIPTFILE1 SCRIPTFILE2 - read commands from one or more files o cat SCRIPTFILE1 | hledger run - read commands from standard input. run first loads the input file(s) specified by LEDGER_FILE or by -f op- tions, in the usual way. Then it runs each command in turn, each using - the same input data. But if you want a particular command to use dif- - ferent input, you can specify an -f option within that command. This + the same input data. But if you want a particular command to use dif- + ferent input, you can specify an -f option within that command. This will override (not add to) the default input, just for that command. Each input file (more precisely, each combination of input file and in- - put options) is parsed only once. This means that commands will not - see any changes made to these files, until the next run. But the com- - mands will run more quickly than if run individually (typically about + put options) is parsed only once. This means that commands will not + see any changes made to these files, until the next run. But the com- + mands will run more quickly than if run individually (typically about twice as fast). Command scripts, whether in a file or written on the command line, have a simple syntax: - o each line may contain a single hledger command and its arguments, + o each line may contain a single hledger command and its arguments, without the usual hledger first word o empty lines are ignored o text from # to end of line is a comment, and ignored - o you can use single or double quotes to quote arguments when needed, + o you can use single or double quotes to quote arguments when needed, as on the command line - o these extra commands are available: echo TEXT prints some text, and + o these extra commands are available: echo TEXT prints some text, and exit or quit ends the run. - On unix systems you can use #!/usr/bin/env hledger run in the first - line of a command file to make it a runnable script. If that gives an + On unix systems you can use #!/usr/bin/env hledger run in the first + line of a command file to make it a runnable script. If that gives an error, use #!/usr/bin/env -S hledger run. It's ok to use the run command recursively within a command script. - You may find some differences in behaviour between run command lines - and normal hledger command lines. run is a little stricter; eg it re- - quires full command names or official abbreviations (as seen in the - commands list), and command options must be written after the command + You may find some differences in behaviour between run command lines + and normal hledger command lines. run is a little stricter; eg it re- + quires full command names or official abbreviations (as seen in the + commands list), and command options must be written after the command name. Examples @@ -7127,8 +7521,8 @@ User interface commands hledger -f some.journal run -- balance assets --depth 2 -- balance liabilities -f /some/other.journal --depth 3 --transpose -- stats - This would load some.journal, run balance assets --depth 2 on it, then - run balance liabilities --depth 3 --transpose on /some/other.journal, + This would load some.journal, run balance assets --depth 2 on it, then + run balance liabilities --depth 3 --transpose on /some/other.journal, and finally run stats on some.journal Run commands from standard input: @@ -7170,30 +7564,30 @@ Data entry commands Flags: --no-new-accounts don't allow creating new accounts - Many hledger users edit their journals directly with a text editor, or - generate them from CSV. For more interactive data entry, there is the - add command, which prompts interactively on the console for new trans- - actions, and appends them to the main journal file (which should be in - journal format). Existing transactions are not changed. This is one - of the few hledger commands that writes to the journal file (see also + Many hledger users edit their journals directly with a text editor, or + generate them from CSV. For more interactive data entry, there is the + add command, which prompts interactively on the console for new trans- + actions, and appends them to the main journal file (which should be in + journal format). Existing transactions are not changed. This is one + of the few hledger commands that writes to the journal file (see also import). To use it, just run hledger add and follow the prompts. You can add as - many transactions as you like; when you are finished, enter . or press + many transactions as you like; when you are finished, enter . or press control-d or control-c to exit. Features: - o add tries to provide useful defaults, using the most similar (by de- - scription) recent transaction (filtered by the query, if any) as a + o add tries to provide useful defaults, using the most similar (by de- + scription) recent transaction (filtered by the query, if any) as a template. o You can also set the initial defaults with command line arguments. o Readline-style edit keys can be used during data entry. - o The tab key will auto-complete whenever possible - accounts, pay- - ees/descriptions, dates (yesterday, today, tomorrow). If the input + o The tab key will auto-complete whenever possible - accounts, pay- + ees/descriptions, dates (yesterday, today, tomorrow). If the input area is empty, it will insert the default value. o A parenthesised transaction code may be entered following a date. @@ -7202,20 +7596,26 @@ Data entry commands o If you make a mistake, enter < at any prompt to go one step backward. - o Input prompts are displayed in a different colour when the terminal + o Input prompts are displayed in a different colour when the terminal supports it. Notes: o If you enter a number with no commodity symbol, and you have declared - a default commodity with a D directive, you might expect add to add - this symbol for you. It does not do this; we assume that if you are - using a D directive you prefer not to see the commodity symbol re- + a default commodity with a D directive, you might expect add to add + this symbol for you. It does not do this; we assume that if you are + using a D directive you prefer not to see the commodity symbol re- peated on amounts in the journal. - o add creates entries in journal format; it won't work with timeclock + o add creates entries in journal format; it won't work with timeclock or timedot files. + o There is a known issue on Windows if this hledger version is built + from stackage: the prompts will show ANSI junk instead of colours + (#2410). You can avoid this by using official hledger release bina- + ries or by building it with haskeline >=0.8.4; or by running add with + --color=no, perhaps configured in your config file. + Examples: o Record new transactions, saving to the default journal file: @@ -7595,7 +7995,8 @@ Basic report commands clared but not used, or just the first one matched by a pattern (with --find, returning a non-zero exit code if it fails). - You can add cur: query arguments to further limit the commodities. + You can add query arguments to further limit the commodities; at least + cur: and tag: are supported. descriptions List the unique descriptions used in transactions. @@ -7604,7 +8005,7 @@ Basic report commands no command-specific flags This command lists the unique descriptions that appear in transactions, - in alphabetic order. You can add a query to select a subset of trans- + in alphabetic order. You can add a query to select a subset of trans- actions. Example: @@ -7615,7 +8016,7 @@ Basic report commands Person A files - List all files included in the journal. With a REGEX argument, only + List all files included in the journal. With a REGEX argument, only file names matching the regular expression (case sensitive) are shown. Flags: @@ -7628,8 +8029,8 @@ Basic report commands no command-specific flags This command lists the unique notes that appear in transactions, in al- - phabetic order. You can add a query to select a subset of transac- - tions. The note is the part of the transaction description after a | + phabetic order. You can add a query to select a subset of transac- + tions. The note is the part of the transaction description after a | character (or if there is no |, the whole description). Example: @@ -7649,16 +8050,16 @@ Basic report commands --find list the first payee matched by the first argument (a case-insensitive infix regexp) - This command lists unique payee/payer names - all of them by default, - or just the ones which have been used in transaction descriptions, or - declared with payee directives, or used but not declared, or declared - but not used, or just the first one matched by a pattern (with --find, + This command lists unique payee/payer names - all of them by default, + or just the ones which have been used in transaction descriptions, or + declared with payee directives, or used but not declared, or declared + but not used, or just the first one matched by a pattern (with --find, returning a non-zero exit code if it fails). - The payee/payer name is the part of the transaction description before + The payee/payer name is the part of the transaction description before a | character (or if there is no |, the whole description). - You can add query arguments to select a subset of transactions or pay- + You can add query arguments to select a subset of transactions or pay- ees. Example: @@ -7669,8 +8070,8 @@ Basic report commands Person A prices - Print the market prices declared with P directives. With --infer-mar- - ket-prices, also show any additional prices inferred from costs. With + Print the market prices declared with P directives. With --infer-mar- + ket-prices, also show any additional prices inferred from costs. With --show-reverse, also show additional prices inferred by reversing known prices. @@ -7678,14 +8079,14 @@ Basic report commands --show-reverse also show the prices inferred by reversing known prices - Price amounts are always displayed with their full precision, except + Price amounts are always displayed with their full precision, except for reverse prices which are limited to 8 decimal digits. Prices can be filtered by a date:, cur: or amt: query. Generally if you run this command with --infer-market-prices --show-re- - verse, it will show the same prices used internally to calculate value - reports. But if in doubt, you can inspect those directly by running + verse, it will show the same prices used internally to calculate value + reports. But if in doubt, you can inspect those directly by running the value report with --debug=2. stats @@ -7697,7 +8098,7 @@ Basic report commands -o --output-file=FILE write output to FILE. The stats command shows summary information for the whole journal, or a - matched part of it. With a reporting interval, it shows a report for + matched part of it. With a reporting interval, it shows a report for each report period. It also shows some performance statistics: @@ -7716,8 +8117,8 @@ Basic report commands With -v/--verbose, more details are shown: the full paths of all files, and the names of the commodities you work with. - With -1, only one line of output is shown, in a machine-friendly - tab-separated format: the program version, the main journal file name, + With -1, only one line of output is shown, in a machine-friendly + tab-separated format: the program version, the main journal file name, and the performance stats, The run time of stats is similar to that of a balance report. @@ -7741,7 +8142,7 @@ Basic report commands $ hledger stats -1 -f examples/10ktxns-1kaccts.journal 1.50.99-g0835a2485-20251119, mac-aarch64 10ktxns-1kaccts.journal 0.66 s elapsed 15244 txns/s 28 MB live 86 MB alloc - This command supports the -o/--output-file option (but not -O/--out- + This command supports the -o/--output-file option (but not -O/--out- put-format). tags @@ -7759,26 +8160,26 @@ Basic report commands including duplicates This command lists tag names - all of them by default, or just the ones - which have been used on transactions/postings/accounts, or declared - with tag directives, or used but not declared, or declared but not - used, or just the first one matched by a pattern (with --find, return- + which have been used on transactions/postings/accounts, or declared + with tag directives, or used but not declared, or declared but not + used, or just the first one matched by a pattern (with --find, return- ing a non-zero exit code if it fails). - Note this command's non-standard first argument: it is a case-insensi- - tive infix regular expression for matching tag names, which limits the - tags shown. Any additional arguments are standard query arguments, + Note this command's non-standard first argument: it is a case-insensi- + tive infix regular expression for matching tag names, which limits the + tags shown. Any additional arguments are standard query arguments, which limit the transactions, postings, or accounts providing tags. With --values, the tags' unique non-empty values are listed instead. With -E/--empty, blank/empty values are also shown. - With --parsed, tags or values are shown in the order they were parsed, - with duplicates included. (Except, tags from account declarations are + With --parsed, tags or values are shown in the order they were parsed, + with duplicates included. (Except, tags from account declarations are always shown first.) - Remember that accounts also acquire tags from their parents; postings - also acquire tags from their account and transaction; and transactions + Remember that accounts also acquire tags from their parents; postings + also acquire tags from their account and transaction; and transactions also acquire tags from their postings. Standard report commands @@ -7807,16 +8208,17 @@ Standard report commands with this prefix. (Usually the base url shown by hledger-web; can also be relative.) -O --output-format=FMT select the output format. Supported formats: - txt, beancount, csv, tsv, html, fods, json, sql. + txt, ledger, beancount, csv, tsv, html, fods, json, + sql. -o --output-file=FILE write output to FILE. A file extension matching one of the above formats selects that format. The print command displays full journal entries (transactions) from the journal file, sorted by date (or with --date2, by secondary date). - Directives and inter-transaction comments are not shown, currently. + Directives and inter-transaction comments are not shown, currently. This means the print command is somewhat lossy, and if you are using it - to reformat/regenerate your journal you should take care to also copy + to reformat/regenerate your journal you should take care to also copy over the directives and inter-transaction comments. Eg: @@ -7836,33 +8238,33 @@ Standard report commands assets:cash $-2 print amount explicitness - Normally, whether posting amounts are implicit or explicit is pre- + Normally, whether posting amounts are implicit or explicit is pre- served. For example, when an amount is omitted in the journal, it will - not appear in the output. Similarly, if a conversion cost is implied + not appear in the output. Similarly, if a conversion cost is implied but not written, it will not appear in the output. - You can use the -x/--explicit flag to force explicit display of all - amounts and costs. This can be useful for troubleshooting or for mak- - ing your journal more readable and robust against data entry errors. + You can use the -x/--explicit flag to force explicit display of all + amounts and costs. This can be useful for troubleshooting or for mak- + ing your journal more readable and robust against data entry errors. -x is also implied by using any of -B,-V,-X,--value. - The -x/--explicit flag will cause any postings with a multi-commodity - amount (which can arise when a multi-commodity transaction has an im- - plicit amount) to be split into multiple single-commodity postings, + The -x/--explicit flag will cause any postings with a multi-commodity + amount (which can arise when a multi-commodity transaction has an im- + plicit amount) to be split into multiple single-commodity postings, keeping the output parseable. print alignment - Amounts are shown right-aligned within each transaction (but not - aligned across all transactions; you can achieve that with ledger-mode + Amounts are shown right-aligned within each transaction (but not + aligned across all transactions; you can achieve that with ledger-mode in Emacs). print amount style - Amounts will be displayed mostly in their commodity's display style, - with standardised symbol placement, decimal mark, and digit group - marks. This does not apply to their decimal digits; print normally + Amounts will be displayed mostly in their commodity's display style, + with standardised symbol placement, decimal mark, and digit group + marks. This does not apply to their decimal digits; print normally shows the same decimal digits that are recorded in each journal entry. - You can override the decimal precisions with print's special --round + You can override the decimal precisions with print's special --round option (since 1.32). --round tries to show amounts with their commodi- ties' standard decimal precisions, increasingly strongly: @@ -7870,7 +8272,7 @@ Standard report commands o --round=soft add/remove decimal zeros in amounts (except costs) - o --round=hard round amounts (except costs), possibly hiding signifi- + o --round=hard round amounts (except costs), possibly hiding signifi- cant digits o --round=all round all amounts and costs @@ -7878,21 +8280,21 @@ Standard report commands soft is good for non-lossy cleanup, displaying more consistent decimals where possible, without making entries unbalanced. - hard or all can be good for stronger cleanup, when decimal rounding is - wanted. Note rounding can produce unbalanced journal entries, perhaps + hard or all can be good for stronger cleanup, when decimal rounding is + wanted. Note rounding can produce unbalanced journal entries, perhaps requiring manual fixup. print parseability - Normally, print's output is a valid hledger journal, which you can - "pipe" to a second hledger command for further processing. This is - sometimes convenient for achieving certain kinds of query (though less + Normally, print's output is a valid hledger journal, which you can + "pipe" to a second hledger command for further processing. This is + sometimes convenient for achieving certain kinds of query (though less needed now that queries have become more powerful): # Show running total of food expenses paid from cash. # -f- reads from stdin. -I/--ignore-assertions is sometimes needed. $ hledger print assets:cash | hledger -f- -I reg expenses:food - But here are some things which can cause print's output to become un- + But here are some things which can cause print's output to become un- parseable: o --round (see above) can disrupt transaction balancing. @@ -7900,12 +8302,12 @@ Standard report commands o Account aliases or pivoting can disrupt account names, balance asser- tions, or balance assignments. - o Value reporting also can disrupt balance assertions or balance as- + o Value reporting also can disrupt balance assertions or balance as- signments. o Auto postings can generate too many amountless postings. - o --infer-costs or --infer-equity can generate too-complex redundant + o --infer-costs or --infer-equity can generate too-complex redundant costs. o Because print always shows transactions in date order, balance asser- @@ -7915,26 +8317,30 @@ Standard report commands print, other features With -B/--cost, amounts with costs are shown converted to cost. - With --invert, posting amounts are shown with their sign flipped. It - could be useful if you have accidentally recorded some transactions + With --invert, posting amounts are shown with their sign flipped. It + could be useful if you have accidentally recorded some transactions with the wrong signs. With --new, print shows only transactions it has not seen on a previous - run. This uses the same deduplication system as the import command. + run. This uses the same deduplication system as the import command. (See import's docs for details.) With -m DESC/--match=DESC, print shows one recent transaction whose de- - scription is most similar to DESC. DESC should contain at least two - characters. If there is no similar-enough match, no transaction will + scription is most similar to DESC. DESC should contain at least two + characters. If there is no similar-enough match, no transaction will be shown and the program exit code will be non-zero. - With --locations, print adds the source file and line number to every + With --locations, print adds the source file and line number to every transaction, as a tag. print output format This command also supports the output destination and output format op- - tions The output formats supported are txt, beancount (Added in 1.32), - csv, tsv (Added in 1.32), json and sql. + tions The output formats supported are txt, ledger, beancount (Added in + 1.32), csv, tsv (Added in 1.32), json and sql. + + The ledger format is currently the same as txt except it renders + amounts' cost basis using Ledger's lot syntax ([DATE] (LABEL) {COST}) + instead of hledger's ({DATE, "LABEL", COST}). The beancount format tries to produce Beancount-compatible output, as follows: @@ -8127,6 +8533,7 @@ Standard report commands description closest to DESC -r --related show postings' siblings instead --invert display all amounts with reversed sign + --drop=N omit N leading account name parts --sort=FIELDS sort by: date, desc, account, amount, absamount, or a comma-separated combination of these. For a descending sort, prefix with -. (Default: date) @@ -8177,6 +8584,8 @@ Standard report commands The --depth option limits the amount of sub-account detail displayed. + The --drop option will trim leading segments from account names. + The --average/-A flag shows the running average posting amount instead of the running total (so, the final number displayed is the average for the whole report period). This flag implies --empty (see below). It @@ -8237,16 +8646,23 @@ Standard report commands tervals. This ensures that the first and last intervals are full length and comparable to the others in the report. - With -m DESC/--match=DESC, register does a fuzzy search for one recent + If you have a deeply nested account tree some reports might benefit + from trimming leading segments from the account names using --drop. + + $ hledger register --monthly income --drop 1 + 2008/01 salary $-1 $-1 + 2008/06 gifts $-1 $-2 + + With -m DESC/--match=DESC, register does a fuzzy search for one recent posting whose description is most similar to DESC. DESC should contain at least two characters. If there is no similar-enough match, no post- ing will be shown and the program exit code will be non-zero. Custom register output - register normally uses the full terminal width (or 80 columns if it + register normally uses the full terminal width (or 80 columns if it can't detect that). You can override this with the --width/-w option. - The description and account columns normally share the space equally + The description and account columns normally share the space equally (about half of (width - 40) each). You can adjust this by adding a de- scription width as part of --width's argument, comma-separated: --width W,D . Here's a diagram (won't display correctly in --help): @@ -8262,14 +8678,14 @@ Standard report commands $ hledger reg -w 100,40 # set overall width 100, description width 40 This command also supports the output destination and output format op- - tions The output formats supported are txt, csv, tsv (Added in 1.32), + tions The output formats supported are txt, csv, tsv (Added in 1.32), and json. balancesheet (bs) - Show the end balances in asset and liability accounts. Amounts are - shown with normal positive sign, as in conventional financial state- + Show the end balances in asset and liability accounts. Amounts are + shown with normal positive sign, as in conventional financial state- ments. Flags: @@ -8320,13 +8736,13 @@ Standard report commands -o --output-file=FILE write output to FILE. A file extension matching one of the above formats selects that format. - This command displays a balance sheet, showing historical ending bal- + This command displays a balance sheet, showing historical ending bal- ances of asset and liability accounts. (To see equity as well, use the balancesheetequity command.) Accounts declared with the Asset, Cash or Liability type are shown (see - account types). Or if no such accounts are declared, it shows - top-level accounts named asset or liability (case insensitive, plurals + account types). Or if no such accounts are declared, it shows + top-level accounts named asset or liability (case insensitive, plurals allowed) and their subaccounts. Example: @@ -8352,20 +8768,20 @@ Standard report commands Net: || 0 This command is a higher-level variant of the balance command, and sup- - ports many of that command's features, such as multi-period reports. - It is similar to hledger balance -H assets liabilities, but with - smarter account detection, and liabilities displayed with their sign + ports many of that command's features, such as multi-period reports. + It is similar to hledger balance -H assets liabilities, but with + smarter account detection, and liabilities displayed with their sign flipped. This command also supports the output destination and output format op- - tions The output formats supported are txt, csv, tsv (Added in 1.32), + tions The output formats supported are txt, csv, tsv (Added in 1.32), html, and json. balancesheetequity (bse) - This command displays a balance sheet, showing historical ending bal- - ances of asset, liability and equity accounts. Amounts are shown with + This command displays a balance sheet, showing historical ending bal- + ances of asset, liability and equity accounts. Amounts are shown with normal positive sign, as in conventional financial statements. Flags: @@ -8416,9 +8832,9 @@ Standard report commands -o --output-file=FILE write output to FILE. A file extension matching one of the above formats selects that format. - This report shows accounts declared with the Asset, Cash, Liability or - Equity type (see account types). Or if no such accounts are declared, - it shows top-level accounts named asset, liability or equity (case in- + This report shows accounts declared with the Asset, Cash, Liability or + Equity type (see account types). Or if no such accounts are declared, + it shows top-level accounts named asset, liability or equity (case in- sensitive, plurals allowed) and their subaccounts. Example: @@ -8449,14 +8865,14 @@ Standard report commands Net: || 0 This command is a higher-level variant of the balance command, and sup- - ports many of that command's features, such as multi-period reports. + ports many of that command's features, such as multi-period reports. It is similar to hledger balance -H assets liabilities equity, but with - smarter account detection, and liabilities/equity displayed with their + smarter account detection, and liabilities/equity displayed with their sign flipped. This report is the easiest way to see if the accounting equation (A+L+E - = 0) is satisfied (after you have done a close --retain to merge rev- - enues and expenses with equity, and perhaps added --infer-equity to + = 0) is satisfied (after you have done a close --retain to merge rev- + enues and expenses with equity, and perhaps added --infer-equity to balance your commodity conversions). This command also supports the output destination and output format op- @@ -8465,9 +8881,9 @@ Standard report commands cashflow (cf) - This command displays a (simple) cashflow statement, showing the in- - flows and outflows affecting "cash" (ie, liquid, easily convertible) - assets. Amounts are shown with normal positive sign, as in conven- + This command displays a (simple) cashflow statement, showing the in- + flows and outflows affecting "cash" (ie, liquid, easily convertible) + assets. Amounts are shown with normal positive sign, as in conven- tional financial statements. Flags: @@ -8519,10 +8935,10 @@ Standard report commands -o --output-file=FILE write output to FILE. A file extension matching one of the above formats selects that format. - This report shows accounts declared with the Cash type (see account + This report shows accounts declared with the Cash type (see account types). Or if no such accounts are declared, it shows accounts - o under a top-level account named asset (case insensitive, plural al- + o under a top-level account named asset (case insensitive, plural al- lowed) o whose name contains some variation of cash, bank, checking or saving. @@ -8549,19 +8965,19 @@ Standard report commands || $-1 This command is a higher-level variant of the balance command, and sup- - ports many of that command's features, such as multi-period reports. - It is similar to hledger balance assets not:fixed not:investment + ports many of that command's features, such as multi-period reports. + It is similar to hledger balance assets not:fixed not:investment not:receivable, but with smarter account detection. This command also supports the output destination and output format op- - tions The output formats supported are txt, csv, tsv (Added in 1.32), + tions The output formats supported are txt, csv, tsv (Added in 1.32), html, and json. incomestatement (is) - Show revenue inflows and expense outflows during the report period. - Amounts are shown with normal positive sign, as in conventional finan- + Show revenue inflows and expense outflows during the report period. + Amounts are shown with normal positive sign, as in conventional finan- cial statements. Flags: @@ -8613,12 +9029,12 @@ Standard report commands -o --output-file=FILE write output to FILE. A file extension matching one of the above formats selects that format. - This command displays an income statement, showing revenues and ex- + This command displays an income statement, showing revenues and ex- penses during one or more periods. - It shows accounts declared with the Revenue or Expense type (see ac- - count types). Or if no such accounts are declared, it shows top-level - accounts named revenue or income or expense (case insensitive, plurals + It shows accounts declared with the Revenue or Expense type (see ac- + count types). Or if no such accounts are declared, it shows top-level + accounts named revenue or income or expense (case insensitive, plurals allowed) and their subaccounts. Example: @@ -8645,20 +9061,20 @@ Standard report commands Net: || 0 This command is a higher-level variant of the balance command, and sup- - ports many of that command's features, such as multi-period reports. + ports many of that command's features, such as multi-period reports. It is similar to hledger balance '(revenues|income)' expenses, but with - smarter account detection, and revenues/income displayed with their + smarter account detection, and revenues/income displayed with their sign flipped. This command also supports the output destination and output format op- - tions The output formats supported are txt, csv, tsv (Added in 1.32), + tions The output formats supported are txt, csv, tsv (Added in 1.32), html, and json. Advanced report commands balance (bal) - A flexible, general purpose "summing" report that shows accounts with + A flexible, general purpose "summing" report that shows accounts with some kind of numeric data. This can be balance changes per period, end balances, budget performance, unrealised capital gains, etc. @@ -8725,19 +9141,19 @@ Advanced report commands -o --output-file=FILE write output to FILE. A file extension matching one of the above formats selects that format. - balance is one of hledger's oldest and most versatile commands, for - listing account balances, balance changes, values, value changes and + balance is one of hledger's oldest and most versatile commands, for + listing account balances, balance changes, values, value changes and more, during one time period or many. Generally it shows a table, with rows representing accounts, and columns representing periods. Note there are some variants of the balance command with convenient de- - faults, which are simpler to use: balancesheet, balancesheetequity, - cashflow and incomestatement. When you need more control, then use + faults, which are simpler to use: balancesheet, balancesheetequity, + cashflow and incomestatement. When you need more control, then use balance. balance features - Here's a quick overview of the balance command's features, followed by - more detailed descriptions and examples. Many of these work with the + Here's a quick overview of the balance command's features, followed by + more detailed descriptions and examples. Many of these work with the other balance-like commands as well (bs, cf, is..). balance can show.. @@ -8792,7 +9208,7 @@ Advanced report commands ..with.. - o totals (-T), averages (-A), percentages (-%), inverted sign (--in- + o totals (-T), averages (-A), percentages (-%), inverted sign (--in- vert) o rows and columns swapped (--transpose) @@ -8805,20 +9221,20 @@ Advanced report commands This command supports the output destination and output format options, with output formats txt, csv, tsv (Added in 1.32), json, and (multi-pe- - riod reports only:) html, fods (Added in 1.40). In txt output in a + riod reports only:) html, fods (Added in 1.40). In txt output in a colour-supporting terminal, negative amounts are shown in red. Simple balance report - With no arguments, balance shows a list of all accounts and their - change of balance - ie, the sum of posting amounts, both inflows and - outflows - during the entire period of the journal. ("Simple" here - means just one column of numbers, covering a single period. You can + With no arguments, balance shows a list of all accounts and their + change of balance - ie, the sum of posting amounts, both inflows and + outflows - during the entire period of the journal. ("Simple" here + means just one column of numbers, covering a single period. You can also have multi-period reports, described later.) - For real-world accounts, these numbers will normally be their end bal- + For real-world accounts, these numbers will normally be their end bal- ance at the end of the journal period; more on this below. - Accounts are sorted by declaration order if any, and then alphabeti- + Accounts are sorted by declaration order if any, and then alphabeti- cally by account name. For instance (using examples/sample.journal): $ hledger -f examples/sample.journal bal @@ -8833,7 +9249,7 @@ Advanced report commands 0 Accounts with a zero balance (and no non-zero subaccounts, in tree mode - - see below) are hidden by default. Use -E/--empty to show them (re- + - see below) are hidden by default. Use -E/--empty to show them (re- vealing assets:bank:checking here): $ hledger -f examples/sample.journal bal -E @@ -8848,12 +9264,12 @@ Advanced report commands -------------------- 0 - The total of the amounts displayed is shown as the last line, unless + The total of the amounts displayed is shown as the last line, unless -N/--no-total is used. Balance report line format For single-period balance reports displayed in the terminal (only), you - can use --format FMT to customise the format and content of each line. + can use --format FMT to customise the format and content of each line. Eg: $ hledger -f examples/sample.journal balance --format "%20(account) %12(total)" @@ -8870,7 +9286,7 @@ Advanced report commands --------------------------------- 0 - The FMT format string specifies the formatting applied to each ac- + The FMT format string specifies the formatting applied to each ac- count/balance pair. It may contain any suitable text, with data fields interpolated like so: @@ -8882,14 +9298,14 @@ Advanced report commands o FIELDNAME must be enclosed in parentheses, and can be one of: - o depth_spacer - a number of spaces equal to the account's depth, or + o depth_spacer - a number of spaces equal to the account's depth, or if MIN is specified, MIN * depth spaces. o account - the account's name o total - the account's balance/posted total, right justified - Also, FMT can begin with an optional prefix to control how multi-com- + Also, FMT can begin with an optional prefix to control how multi-com- modity amounts are rendered: o %_ - render on multiple lines, bottom-aligned (the default) @@ -8899,25 +9315,25 @@ Advanced report commands o %, - render on one line, comma-separated There are some quirks. Eg in one-line mode, %(depth_spacer) has no ef- - fect, instead %(account) has indentation built in. Experimentation + fect, instead %(account) has indentation built in. Experimentation may be needed to get pleasing results. Some example formats: o %(total) - the account's total - o %-20.20(account) - the account's name, left justified, padded to 20 + o %-20.20(account) - the account's name, left justified, padded to 20 characters and clipped at 20 characters - o %,%-50(account) %25(total) - account name padded to 50 characters, - total padded to 20 characters, with multiple commodities rendered on + o %,%-50(account) %25(total) - account name padded to 50 characters, + total padded to 20 characters, with multiple commodities rendered on one line - o %20(total) %2(depth_spacer)%-(account) - the default format for the + o %20(total) %2(depth_spacer)%-(account) - the default format for the single-column balance report Filtered balance report - You can show fewer accounts, a different time period, totals from + You can show fewer accounts, a different time period, totals from cleared transactions only, etc. by using query arguments or options to limit the postings being matched. Eg: @@ -8927,10 +9343,10 @@ Advanced report commands $-2 List or tree mode - By default, or with -l/--flat, accounts are shown as a flat list with + By default, or with -l/--flat, accounts are shown as a flat list with their full names visible, as in the examples above. - With -t/--tree, the account hierarchy is shown, with subaccounts' + With -t/--tree, the account hierarchy is shown, with subaccounts' "leaf" names indented below their parent: $ hledger -f examples/sample.journal balance @@ -8950,26 +9366,26 @@ Advanced report commands Notes: o "Boring" accounts are combined with their subaccount for more compact - output, unless --no-elide is used. Boring accounts have no balance - of their own and just one subaccount (eg assets:bank and liabilities + output, unless --no-elide is used. Boring accounts have no balance + of their own and just one subaccount (eg assets:bank and liabilities above). - o All balances shown are "inclusive", ie including the balances from - all subaccounts. Note this means some repetition in the output, + o All balances shown are "inclusive", ie including the balances from + all subaccounts. Note this means some repetition in the output, which requires explanation when sharing reports with non-plaintextac- - counting-users. A tree mode report's final total is the sum of the + counting-users. A tree mode report's final total is the sum of the top-level balances shown, not of all the balances shown. - o Each group of sibling accounts (ie, under a common parent) is sorted + o Each group of sibling accounts (ie, under a common parent) is sorted separately. Depth limiting - With a depth:NUM query, or --depth NUM option, or just -NUM (eg: -3) - balance reports will show accounts only to the specified depth, hiding - the deeper subaccounts. This can be useful for getting an overview + With a depth:NUM query, or --depth NUM option, or just -NUM (eg: -3) + balance reports will show accounts only to the specified depth, hiding + the deeper subaccounts. This can be useful for getting an overview without too much detail. - Account balances at the depth limit always include the balances from + Account balances at the depth limit always include the balances from any deeper subaccounts (even in list mode). Eg, limiting to depth 1: $ hledger -f examples/sample.journal balance -1 @@ -8981,7 +9397,7 @@ Advanced report commands 0 Dropping top-level accounts - You can also hide one or more top-level account name parts, using + You can also hide one or more top-level account name parts, using --drop NUM. This can be useful for hiding repetitive top-level account names: @@ -8992,54 +9408,54 @@ Advanced report commands $2 Showing declared accounts - With --declared, accounts which have been declared with an account di- - rective will be included in the balance report, even if they have no + With --declared, accounts which have been declared with an account di- + rective will be included in the balance report, even if they have no transactions. (Since they will have a zero balance, you will also need -E/--empty to see them.) - More precisely, leaf declared accounts (with no subaccounts) will be + More precisely, leaf declared accounts (with no subaccounts) will be included, since those are usually the more useful in reports. - The idea of this is to be able to see a useful "complete" balance re- + The idea of this is to be able to see a useful "complete" balance re- port, even when you don't have transactions in all of your declared ac- counts yet. Sorting by amount - With -S/--sort-amount, accounts with the largest (most positive) bal- - ances are shown first. Eg: hledger bal expenses -MAS shows your - biggest averaged monthly expenses first. When more than one commodity - is present, they will be sorted by the alphabetically earliest commod- - ity first, and then by subsequent commodities (if an amount is missing + With -S/--sort-amount, accounts with the largest (most positive) bal- + ances are shown first. Eg: hledger bal expenses -MAS shows your + biggest averaged monthly expenses first. When more than one commodity + is present, they will be sorted by the alphabetically earliest commod- + ity first, and then by subsequent commodities (if an amount is missing a commodity, it is treated as 0). - Revenues and liability balances are typically negative, however, so -S - shows these in reverse order. To work around this, you can add --in- - vert to flip the signs. Or you could use one of the higher-level bal- + Revenues and liability balances are typically negative, however, so -S + shows these in reverse order. To work around this, you can add --in- + vert to flip the signs. Or you could use one of the higher-level bal- ance reports (bs, is..), which flip the sign automatically (eg: hledger is -MAS). Percentages - With -%/--percent, balance reports show each account's value expressed + With -%/--percent, balance reports show each account's value expressed as a percentage of the (column) total. Note it is not useful to calculate percentages if the amounts in a col- - umn have mixed signs. In this case, make a separate report for each + umn have mixed signs. In this case, make a separate report for each sign, eg: $ hledger bal -% amt:`>0` $ hledger bal -% amt:`<0` - Similarly, if the amounts in a column have mixed commodities, convert - them to one commodity with -B, -V, -X or --value, or make a separate + Similarly, if the amounts in a column have mixed commodities, convert + them to one commodity with -B, -V, -X or --value, or make a separate report for each commodity: $ hledger bal -% cur:\\$ $ hledger bal -% cur: Multi-period balance report - With a report interval (set by the -D/--daily, -W/--weekly, - -M/--monthly, -Q/--quarterly, -Y/--yearly, or -p/--period flag), bal- - ance shows a tabular report, with columns representing successive time + With a report interval (set by the -D/--daily, -W/--weekly, + -M/--monthly, -Q/--quarterly, -Y/--yearly, or -p/--period flag), bal- + ance shows a tabular report, with columns representing successive time periods (and a title): $ hledger -f examples/sample.journal bal --quarterly income expenses -E @@ -9060,24 +9476,24 @@ Advanced report commands encompass the displayed subperiods (so that the first and last subpe- riods have the same duration as the others). - o Leading and trailing periods (columns) containing all zeroes are not + o Leading and trailing periods (columns) containing all zeroes are not shown, unless -E/--empty is used. - o Accounts (rows) containing all zeroes are not shown, unless + o Accounts (rows) containing all zeroes are not shown, unless -E/--empty is used. - o Amounts with many commodities are shown in abbreviated form, unless + o Amounts with many commodities are shown in abbreviated form, unless --no-elide is used. - o Average and/or total columns can be added with the -A/--average and + o Average and/or total columns can be added with the -A/--average and -T/--row-total flags. o The --transpose flag can be used to exchange rows and columns. - o The --pivot FIELD option causes a different transaction field to be + o The --pivot FIELD option causes a different transaction field to be used as "account name". See PIVOTING. - o The --summary-only flag (--summary also works) hides all but the To- + o The --summary-only flag (--summary also works) hides all but the To- tal and Average columns (those should be enabled with --row-total and -A/--average). @@ -9096,57 +9512,57 @@ Advanced report commands o Reduce the terminal's font size - o View with a pager like less, eg: hledger bal -D --color=yes | less + o View with a pager like less, eg: hledger bal -D --color=yes | less -RS - o Output as CSV and use a CSV viewer like visidata (hledger bal -D -O - csv | vd -f csv), Emacs' csv-mode (M-x csv-mode, C-c C-a), or a + o Output as CSV and use a CSV viewer like visidata (hledger bal -D -O + csv | vd -f csv), Emacs' csv-mode (M-x csv-mode, C-c C-a), or a spreadsheet (hledger bal -D -o a.csv && open a.csv) - o Output as HTML and view with a browser: hledger bal -D -o a.html && + o Output as HTML and view with a browser: hledger bal -D -o a.html && open a.html Balance change, end balance - It's important to be clear on the meaning of the numbers shown in bal- + It's important to be clear on the meaning of the numbers shown in bal- ance reports. Here is some terminology we use: - A balance change is the net amount added to, or removed from, an ac- + A balance change is the net amount added to, or removed from, an ac- count during some period. - An end balance is the amount accumulated in an account as of some date - (and some time, but hledger doesn't store that; assume end of day in + An end balance is the amount accumulated in an account as of some date + (and some time, but hledger doesn't store that; assume end of day in your timezone). It is the sum of previous balance changes. - We call it a historical end balance if it includes all balance changes + We call it a historical end balance if it includes all balance changes since the account was created. For a real world account, this means it - will match the "historical record", eg the balances reported in your + will match the "historical record", eg the balances reported in your bank statements or bank web UI. (If they are correct!) - In general, balance changes are what you want to see when reviewing + In general, balance changes are what you want to see when reviewing revenues and expenses, and historical end balances are what you want to see when reviewing or reconciling asset, liability and equity accounts. - balance shows balance changes by default. To see accurate historical + balance shows balance changes by default. To see accurate historical end balances: - 1. Initialise account starting balances with an "opening balances" - transaction (a transfer from equity to the account), unless the + 1. Initialise account starting balances with an "opening balances" + transaction (a transfer from equity to the account), unless the journal covers the account's full lifetime. 2. Include all of of the account's prior postings in the report, by not - specifying a report start date, or by using the -H/--historical + specifying a report start date, or by using the -H/--historical flag. (-H causes report start date to be ignored when summing post- ings.) Balance report modes - The balance command is quite flexible; here is the full detail on how - to control what it reports. If the following seems complicated, don't - worry - this is for advanced reporting, and it does take time and ex- + The balance command is quite flexible; here is the full detail on how + to control what it reports. If the following seems complicated, don't + worry - this is for advanced reporting, and it does take time and ex- perimentation to get familiar with all the report modes. There are three important option groups: - hledger balance [CALCULATIONMODE] [ACCUMULATIONMODE] [VALUATIONMODE] + hledger balance [CALCULATIONMODE] [ACCUMULATIONMODE] [VALUATIONMODE] ... Calculation mode @@ -9158,35 +9574,35 @@ Advanced report commands each account/period) o --valuechange : show the change in period-end historical balance val- - ues (caused by deposits, withdrawals, and/or market price fluctua- + ues (caused by deposits, withdrawals, and/or market price fluctua- tions) - o --gain : show the unrealised capital gain/loss, (the current valued + o --gain : show the unrealised capital gain/loss, (the current valued balance minus each amount's original cost) o --count : show the count of postings Accumulation mode - How amounts should accumulate across a report's subperiods/columns. - Another way to say it: which time period's postings should contribute + How amounts should accumulate across a report's subperiods/columns. + Another way to say it: which time period's postings should contribute to each cell's calculation. It is one of: - o --change : calculate with postings from column start to column end, - ie "just this column". Typically used to see revenues/expenses. + o --change : calculate with postings from column start to column end, + ie "just this column". Typically used to see revenues/expenses. (default for balance, cashflow, incomestatement) - o --cumulative : calculate with postings from report start to column - end, ie "previous columns plus this column". Typically used to show + o --cumulative : calculate with postings from report start to column + end, ie "previous columns plus this column". Typically used to show changes accumulated since the report's start date. Not often used. - o --historical/-H : calculate with postings from journal start to col- - umn end, ie "all postings from before report start date until this - column's end". Typically used to see historical end balances of as- - sets/liabilities/equity. (default for balancesheet, balancesheete- + o --historical/-H : calculate with postings from journal start to col- + umn end, ie "all postings from before report start date until this + column's end". Typically used to see historical end balances of as- + sets/liabilities/equity. (default for balancesheet, balancesheete- quity) Valuation mode - Which kind of value or cost conversion should be applied, if any, be- + Which kind of value or cost conversion should be applied, if any, be- fore displaying the report. See Cost reporting and Value reporting for more about conversions. @@ -9194,19 +9610,19 @@ Advanced report commands o no conversion : don't convert to cost or value (default) - o --value=cost[,COMM] : convert amounts to cost (then optionally to + o --value=cost[,COMM] : convert amounts to cost (then optionally to some other commodity) - o --value=then[,COMM] : convert amounts to market value on transaction + o --value=then[,COMM] : convert amounts to market value on transaction dates - o --value=end[,COMM] : convert amounts to market value on period end + o --value=end[,COMM] : convert amounts to market value on period end date(s) (default with --valuechange, --gain) o --value=now[,COMM] : convert amounts to market value on today's date - o --value=YYYY-MM-DD[,COMM] : convert amounts to market value on an- + o --value=YYYY-MM-DD[,COMM] : convert amounts to market value on an- other date or with the legacy -B/-V/-X options, which are equivalent and easier to @@ -9219,17 +9635,17 @@ Advanced report commands o -X COMM/--exchange COMM : like --value=end,COMM Note that --value can also convert to cost, as a convenience; but actu- - ally --cost and --value are independent options, and could be used to- + ally --cost and --value are independent options, and could be used to- gether. Combining balance report modes Most combinations of these modes should produce reasonable reports, but - if you find any that seem wrong or misleading, let us know. The fol- + if you find any that seem wrong or misleading, let us know. The fol- lowing restrictions are applied: o --valuechange implies --value=end - o --valuechange makes --change the default when used with the bal- + o --valuechange makes --change the default when used with the bal- ancesheet/balancesheetequity commands o --cumulative or --historical disables --row-total/-T @@ -9242,18 +9658,18 @@ Advanced report commands Accumu- /now lation:v ----------------------------------------------------------------------------------- - --change change in period sum of post- period-end DATE-value of - ing-date market value of change change in pe- + --change change in period sum of post- period-end DATE-value of + ing-date market value of change change in pe- values in period in period riod - --cumu- change from re- sum of post- period-end DATE-value of - lative port start to ing-date market value of change change from + --cumu- change from re- sum of post- period-end DATE-value of + lative port start to ing-date market value of change change from period end values from re- from report report start port start to pe- start to period to period end riod end end - --his- change from sum of post- period-end DATE-value of + --his- change from sum of post- period-end DATE-value of torical journal start to ing-date market value of change change from /-H period end (his- values from jour- from journal journal start - torical end bal- nal start to pe- start to period to period end + torical end bal- nal start to pe- start to period to period end ance) riod end end Budget report @@ -9264,11 +9680,11 @@ Advanced report commands o Accounts which don't have budget goals are hidden by default. - This is useful for comparing planned and actual income, expenses, time + This is useful for comparing planned and actual income, expenses, time usage, etc. - Periodic transaction rules are used to define budget goals. For exam- - ple, here's a periodic rule defining monthly goals for bus travel and + Periodic transaction rules are used to define budget goals. For exam- + ple, here's a periodic rule defining monthly goals for bus travel and food expenses: ;; Budget @@ -9310,66 +9726,66 @@ Advanced report commands || 0 [ 0% of $430] 0 [ 0% of $430] This is "goal-based budgeting"; you define goals for accounts and peri- - ods, often recurring, and hledger shows performance relative to the - goals. This contrasts with "envelope budgeting", which is more de- - tailed and strict - useful when cash is tight, but also quite a bit - more work. https://plaintextaccounting.org/Budgeting has more on this + ods, often recurring, and hledger shows performance relative to the + goals. This contrasts with "envelope budgeting", which is more de- + tailed and strict - useful when cash is tight, but also quite a bit + more work. https://plaintextaccounting.org/Budgeting has more on this topic. Using the budget report - Historically this report has been confusing and fragile. hledger's - version should be relatively robust and intuitive, but you may still - find surprises. Here are more notes to help with learning and trou- + Historically this report has been confusing and fragile. hledger's + version should be relatively robust and intuitive, but you may still + find surprises. Here are more notes to help with learning and trou- bleshooting. - o In the above example, expenses:bus and expenses:food are shown be- + o In the above example, expenses:bus and expenses:food are shown be- cause they have budget goals during the report period. - o Their parent expenses is also shown, with budget goals aggregated + o Their parent expenses is also shown, with budget goals aggregated from the children. - o The subaccounts expenses:food:groceries and expenses:food:dining are - not shown since they have no budget goal of their own, but they con- + o The subaccounts expenses:food:groceries and expenses:food:dining are + not shown since they have no budget goal of their own, but they con- tribute to expenses:food's actual amount. - o Unbudgeted accounts expenses:movies and expenses:gifts are also not + o Unbudgeted accounts expenses:movies and expenses:gifts are also not shown, but they contribute to expenses's actual amount. - o The other unbudgeted accounts income and assets:bank:checking are + o The other unbudgeted accounts income and assets:bank:checking are grouped as . - o --depth or depth: can be used to limit report depth in the usual way + o --depth or depth: can be used to limit report depth in the usual way (but will not reveal unbudgeted subaccounts). o Amounts are always inclusive of subaccounts (even in -l/--list mode). o Numbers displayed in a --budget report will not always agree with the - totals, because of hidden unbudgeted accounts; this is normal. + totals, because of hidden unbudgeted accounts; this is normal. -E/--empty can be used to reveal the hidden accounts. o In the periodic rules used for setting budget goals, unbalanced post- ings are convenient. - o You can filter budget reports with the usual queries, eg to focus on - particular accounts. It's common to restrict them to just expenses. - (The account is occasionally hard to exclude; this is + o You can filter budget reports with the usual queries, eg to focus on + particular accounts. It's common to restrict them to just expenses. + (The account is occasionally hard to exclude; this is because of date surprises, discussed below.) - o When you have multiple currencies, you may want to convert them to - one (-X COMM --infer-market-prices) and/or show just one at a time - (cur:COMM). If you do need to show multiple currencies at once, + o When you have multiple currencies, you may want to convert them to + one (-X COMM --infer-market-prices) and/or show just one at a time + (cur:COMM). If you do need to show multiple currencies at once, --layout bare can be helpful. - o You can "roll over" amounts (actual and budgeted) to the next period + o You can "roll over" amounts (actual and budgeted) to the next period with --cumulative. See also: https://hledger.org/budgeting.html. Budget date surprises - With small data, or when starting out, some of the generated budget - goal transaction dates might fall outside the report periods. Eg with - the following journal and report, the first period appears to have no - expenses:food budget. (Also the account should be ex- + With small data, or when starting out, some of the generated budget + goal transaction dates might fall outside the report periods. Eg with + the following journal and report, the first period appears to have no + expenses:food budget. (Also the account should be ex- cluded by the expenses query, but isn't.): ~ monthly in 2020 @@ -9389,64 +9805,64 @@ Advanced report commands ---------------++-------------------- || $400 [80% of $500] - In this case, the budget goal transactions are generated on first days - of of month (this can be seen with hledger print --forecast tag:gener- - ated expenses). Whereas the report period defaults to just the 15th - day of january (this can be seen from the report table's column head- + In this case, the budget goal transactions are generated on first days + of of month (this can be seen with hledger print --forecast tag:gener- + ated expenses). Whereas the report period defaults to just the 15th + day of january (this can be seen from the report table's column head- ings). - To fix this kind of thing, be more explicit about the report period - (and/or the periodic rules' dates). In this case, adding -b 2020 does + To fix this kind of thing, be more explicit about the report period + (and/or the periodic rules' dates). In this case, adding -b 2020 does the trick. Selecting budget goals - By default, the budget report uses all available periodic transaction - rules to generate goals. This includes rules with a different report - interval from your report. Eg if you have daily, weekly and monthly - periodic rules, all of these will contribute to the goals in a monthly + By default, the budget report uses all available periodic transaction + rules to generate goals. This includes rules with a different report + interval from your report. Eg if you have daily, weekly and monthly + periodic rules, all of these will contribute to the goals in a monthly budget report. - You can select a subset of periodic rules by providing an argument to - the --budget flag. --budget=DESCPAT will match all periodic rules + You can select a subset of periodic rules by providing an argument to + the --budget flag. --budget=DESCPAT will match all periodic rules whose description contains DESCPAT, a case-insensitive substring (not a - regular expression or query). This means you can give your periodic - rules descriptions (remember that two spaces are needed between period - expression and description), and then select from multiple budgets de- + regular expression or query). This means you can give your periodic + rules descriptions (remember that two spaces are needed between period + expression and description), and then select from multiple budgets de- fined in your journal. Budgeting vs forecasting - --forecast and --budget both use the periodic transaction rules in the - journal to generate temporary transactions for reporting purposes. - However they are separate features - though you can use both at the + --forecast and --budget both use the periodic transaction rules in the + journal to generate temporary transactions for reporting purposes. + However they are separate features - though you can use both at the same time if you want. Here are some differences between them: --forecast --budget -------------------------------------------------------------------------- - is a general option; it enables fore- is a balance command option; it - casting with all reports selects the balance report's + is a general option; it enables fore- is a balance command option; it + casting with all reports selects the balance report's budget mode - generates visible transactions which generates invisible transactions + generates visible transactions which generates invisible transactions appear in reports which produce goal amounts - generates forecast transactions from generates budget goal transac- - after the last regular transaction, to tions throughout the report pe- - the end of the report period; or with riod, optionally restricted by - an argument --forecast=PERIODEXPR gen- periods specified in the peri- - erates them throughout the specified odic transaction rules - period, both optionally restricted by - periods specified in the periodic + generates forecast transactions from generates budget goal transac- + after the last regular transaction, to tions throughout the report pe- + the end of the report period; or with riod, optionally restricted by + an argument --forecast=PERIODEXPR gen- periods specified in the peri- + erates them throughout the specified odic transaction rules + period, both optionally restricted by + periods specified in the periodic transaction rules uses all periodic rules uses all periodic rules; or with an argument --budget=DESCPAT - uses just the rules matched by + uses just the rules matched by DESCPAT Balance report layout The --layout option affects how balance and the other balance-like com- - mands show multi-commodity amounts and commodity symbols. It can im- + mands show multi-commodity amounts and commodity symbols. It can im- prove readability, for humans and/or machines (other software). It has four possible values: - o --layout=wide[,WIDTH]: commodities are shown on a single line, op- + o --layout=wide[,WIDTH]: commodities are shown on a single line, op- tionally elided to WIDTH o --layout=tall: each commodity is shown on a separate line @@ -9454,11 +9870,11 @@ Advanced report commands o --layout=bare: commodity symbols are in their own column, amounts are bare numbers - o --layout=tidy: data is normalised to easily-consumed "tidy" form, - with one row per data value. (This one is currently supported only + o --layout=tidy: data is normalised to easily-consumed "tidy" form, + with one row per data value. (This one is currently supported only by the balance command.) - Here are the --layout modes supported by each output format Only CSV + Here are the --layout modes supported by each output format Only CSV output supports all of them: - txt csv html json sql @@ -9494,7 +9910,7 @@ Advanced report commands || 10.00 ITOT, 337.18 USD, 2 more.. 70.00 GLD, 18.00 ITOT, 3 more.. -11.00 ITOT, 3 more.. 70.00 GLD, 17.00 ITOT, 3 more.. Tall layout - Each commodity gets a new line (may be different in each column), and + Each commodity gets a new line (may be different in each column), and account names are repeated: $ hledger -f examples/bcexample.hledger bal assets:us:etrade -3 -T -Y --layout=tall @@ -9515,7 +9931,7 @@ Advanced report commands || 18.00 VHT 294.00 VHT Bare layout - Commodity symbols are kept in one column, each commodity has its own + Commodity symbols are kept in one column, each commodity has its own row, amounts are bare numbers, account names are repeated: $ hledger -f examples/bcexample.hledger bal assets:us:etrade -3 -T -Y --layout=bare @@ -9551,15 +9967,15 @@ Advanced report commands "Total:","VEA","36.00" "Total:","VHT","294.00" - Bare layout will sometimes display an extra row for the no-symbol com- - modity, because of zero amounts (hledger treats zeroes as commod- + Bare layout will sometimes display an extra row for the no-symbol com- + modity, because of zero amounts (hledger treats zeroes as commod- ity-less, usually). This can break hledger-bar confusingly (workaround: add a cur: query to exclude the no-symbol row). Tidy layout This produces normalised "tidy data" (see https://cran.r-project.org/web/packages/tidyr/vignettes/tidy-data.html) - where every variable has its own column and each row represents a sin- + where every variable has its own column and each row represents a sin- gle data point. This is the easiest kind of data for other software to consume: @@ -9582,40 +9998,40 @@ Advanced report commands "Assets:US:ETrade","2014","2014-01-01","2014-12-31","VHT","170.00" Balance report output - As noted in Output format, if you choose HTML output (by using -O html + As noted in Output format, if you choose HTML output (by using -O html or -o somefile.html), you can create a hledger.css file in the same di- rectory to customise the report's appearance. The HTML and FODS output formats can generate hyperlinks to a - hledger-web register view for each account and period. E.g. if your + hledger-web register view for each account and period. E.g. if your hledger-web server is reachable at http://localhost:5000 then you might - run the balance command with the extra option --base-url=http://local- - host:5000. You can also produce relative links, like + run the balance command with the extra option --base-url=http://local- + host:5000. You can also produce relative links, like --base-url="some/path" or --base-url="".) Some useful balance reports Some frequently used balance options/reports are: o bal -M revenues expenses - Show revenues/expenses in each month. Also available as the incomes- + Show revenues/expenses in each month. Also available as the incomes- tatement command. o bal -M -H assets liabilities - Show historical asset/liability balances at each month end. Also + Show historical asset/liability balances at each month end. Also available as the balancesheet command. o bal -M -H assets liabilities equity - Show historical asset/liability/equity balances at each month end. + Show historical asset/liability/equity balances at each month end. Also available as the balancesheetequity command. o bal -M assets not:receivable - Show changes to liquid assets in each month. Also available as the + Show changes to liquid assets in each month. Also available as the cashflow command. Also: o bal -M expenses -2 -SA - Show monthly expenses summarised to depth 2 and sorted by average + Show monthly expenses summarised to depth 2 and sorted by average amount. o bal -M --budget expenses @@ -9629,7 +10045,7 @@ Advanced report commands Show top gainers [or losers] last week roi - Shows the time-weighted (TWR) and money-weighted (IRR) rate of return + Shows the time-weighted (TWR) and money-weighted (IRR) rate of return on your investments. Flags: @@ -9639,38 +10055,38 @@ Advanced report commands --profit-loss=QUERY --pnl query to select profit-and-loss or appreciation/valuation transactions - At a minimum, you need to supply a query (which could be just an ac- - count name) to select your investment(s) with --inv, and another query + At a minimum, you need to supply a query (which could be just an ac- + count name) to select your investment(s) with --inv, and another query to identify your profit and loss transactions with --pnl. - If you do not record changes in the value of your investment manually, - or do not require computation of time-weighted return (TWR), --pnl + If you do not record changes in the value of your investment manually, + or do not require computation of time-weighted return (TWR), --pnl could be an empty query (--pnl "" or --pnl STR where STR does not match any of your accounts). - This command will compute and display the internalized rate of return - (IRR, also known as money-weighted rate of return) and time-weighted - rate of return (TWR) for your investments for the time period re- - quested. IRR is always annualized due to the way it is computed, but - TWR is reported both as a rate over the chosen reporting period and as + This command will compute and display the internalized rate of return + (IRR, also known as money-weighted rate of return) and time-weighted + rate of return (TWR) for your investments for the time period re- + quested. IRR is always annualized due to the way it is computed, but + TWR is reported both as a rate over the chosen reporting period and as an annual rate. - Price directives will be taken into account if you supply appropriate + Price directives will be taken into account if you supply appropriate --cost or --value flags (see VALUATION). Note, in some cases this report can fail, for these reasons: - o Error (NotBracketed): No solution for Internal Rate of Return (IRR). - Possible causes: IRR is huge (>1000000%), balance of investment be- + o Error (NotBracketed): No solution for Internal Rate of Return (IRR). + Possible causes: IRR is huge (>1000000%), balance of investment be- comes negative at some point in time. - o Error (SearchFailed): Failed to find solution for Internal Rate of + o Error (SearchFailed): Failed to find solution for Internal Rate of Return (IRR). Either search does not converge to a solution, or con- verges too slowly. Examples: - o Using roi to compute total return of investment in stocks: + o Using roi to compute total return of investment in stocks: https://github.com/simonmichael/hledger/blob/master/examples/invest- ing/roi-unrealised.ledger @@ -9680,28 +10096,28 @@ Advanced report commands Note that --inv and --pnl's argument is a query, and queries could have several space-separated terms (see QUERIES). - To indicate that all search terms form single command-line argument, + To indicate that all search terms form single command-line argument, you will need to put them in quotes (see Special characters): $ hledger roi --inv 'term1 term2 term3 ...' - If any query terms contain spaces themselves, you will need an extra + If any query terms contain spaces themselves, you will need an extra level of nested quoting, eg: $ hledger roi --inv="'Assets:Test 1'" --pnl="'Equity:Unrealized Profit and Loss'" Semantics of --inv and --pnl - Query supplied to --inv has to match all transactions that are related + Query supplied to --inv has to match all transactions that are related to your investment. Transactions not matching --inv will be ignored. In these transactions, ROI will conside postings that match --inv to be - "investment postings" and other postings (not matching --inv) will be - sorted into two categories: "cash flow" and "profit and loss", as ROI - needs to know which part of the investment value is your contributions + "investment postings" and other postings (not matching --inv) will be + sorted into two categories: "cash flow" and "profit and loss", as ROI + needs to know which part of the investment value is your contributions and which is due to the return on investment. o "Cash flow" is depositing or withdrawing money, buying or selling as- - sets, or otherwise converting between your investment commodity and + sets, or otherwise converting between your investment commodity and any other commodity. Example: 2019-01-01 Investing in Snake Oil @@ -9718,12 +10134,12 @@ Advanced report commands investment:snake oil = $57 equity:unrealized profit or loss - All non-investment postings are assumed to be "cash flow", unless they - match --pnl query. Changes in value of your investment due to "profit - and loss" postings will be considered as part of your investment re- + All non-investment postings are assumed to be "cash flow", unless they + match --pnl query. Changes in value of your investment due to "profit + and loss" postings will be considered as part of your investment re- turn. - Example: if you use --inv snake --pnl equity:unrealized, then postings + Example: if you use --inv snake --pnl equity:unrealized, then postings in the example below would be classifed as: 2019-01-01 Snake Oil #1 @@ -9740,58 +10156,58 @@ Advanced report commands snake oil $50 ; investment posting IRR and TWR explained - "ROI" stands for "return on investment". Traditionally this was com- - puted as a difference between current value of investment and its ini- + "ROI" stands for "return on investment". Traditionally this was com- + puted as a difference between current value of investment and its ini- tial value, expressed in percentage of the initial value. However, this approach is only practical in simple cases, where invest- - ments receives no in-flows or out-flows of money, and where rate of + ments receives no in-flows or out-flows of money, and where rate of growth is fixed over time. For more complex scenarios you need differ- - ent ways to compute rate of return, and this command implements two of + ent ways to compute rate of return, and this command implements two of them: IRR and TWR. - Internal rate of return, or "IRR" (also called "money-weighted rate of - return") takes into account effects of in-flows and out-flows, and the - time between them. Investment at a particular fixed interest rate is - going to give you more interest than the same amount invested at the - same interest rate, but made later in time. If you are withdrawing - from your investment, your future gains would be smaller (in absolute - numbers), and will be a smaller percentage of your initial investment, + Internal rate of return, or "IRR" (also called "money-weighted rate of + return") takes into account effects of in-flows and out-flows, and the + time between them. Investment at a particular fixed interest rate is + going to give you more interest than the same amount invested at the + same interest rate, but made later in time. If you are withdrawing + from your investment, your future gains would be smaller (in absolute + numbers), and will be a smaller percentage of your initial investment, so your IRR will be smaller. And if you are adding to your investment, you will receive bigger absolute gains, which will be a bigger percent- age of your initial investment, so your IRR will be larger. - As mentioned before, in-flows and out-flows would be any cash that you + As mentioned before, in-flows and out-flows would be any cash that you personally put in or withdraw, and for the "roi" command, these are the - postings that match the query in the--inv argument and NOT match the + postings that match the query in the--inv argument and NOT match the query in the--pnl argument. - If you manually record changes in the value of your investment as - transactions that balance them against "profit and loss" (or "unreal- - ized gains") account or use price directives, then in order for IRR to - compute the precise effect of your in-flows and out-flows on the rate - of return, you will need to record the value of your investement on or + If you manually record changes in the value of your investment as + transactions that balance them against "profit and loss" (or "unreal- + ized gains") account or use price directives, then in order for IRR to + compute the precise effect of your in-flows and out-flows on the rate + of return, you will need to record the value of your investement on or close to the days when in- or out-flows occur. - In technical terms, IRR uses the same approach as computation of net + In technical terms, IRR uses the same approach as computation of net present value, and tries to find a discount rate that makes net present value of all the cash flows of your investment to add up to zero. This - could be hard to wrap your head around, especially if you haven't done + could be hard to wrap your head around, especially if you haven't done discounted cash flow analysis before. Implementation of IRR in hledger should produce results that match the =XIRR formula in Excel. - Second way to compute rate of return that roi command implements is - called "time-weighted rate of return" or "TWR". Like IRR, it will ac- - count for the effect of your in-flows and out-flows, but unlike IRR it - will try to compute the true rate of return of the underlying asset, - compensating for the effect that deposits and withdrawas have on the + Second way to compute rate of return that roi command implements is + called "time-weighted rate of return" or "TWR". Like IRR, it will ac- + count for the effect of your in-flows and out-flows, but unlike IRR it + will try to compute the true rate of return of the underlying asset, + compensating for the effect that deposits and withdrawas have on the apparent rate of growth of your investment. - TWR represents your investment as an imaginary "unit fund" where - in-flows/ out-flows lead to buying or selling "units" of your invest- - ment and changes in its value change the value of "investment unit". - Change in "unit price" over the reporting period gives you rate of re- - turn of your investment, and make TWR less sensitive than IRR to the + TWR represents your investment as an imaginary "unit fund" where + in-flows/ out-flows lead to buying or selling "units" of your invest- + ment and changes in its value change the value of "investment unit". + Change in "unit price" over the reporting period gives you rate of re- + turn of your investment, and make TWR less sensitive than IRR to the effects of cash in-flows and out-flows. References: @@ -9804,7 +10220,7 @@ Advanced report commands o IRR vs TWR - o Examples of computing IRR and TWR and discussion of the limitations + o Examples of computing IRR and TWR and discussion of the limitations of both metrics Chart commands @@ -9814,8 +10230,8 @@ Chart commands Flags: no command-specific flags - The activity command displays an ascii histogram showing transaction - counts by day, week, month or other reporting interval (by day is the + The activity command displays an ascii histogram showing transaction + counts by day, week, month or other reporting interval (by day is the default). With query arguments, it counts only matched transactions. Examples: @@ -9830,10 +10246,10 @@ Data generation commands close (equity) - close prints several kinds of "closing" and/or "opening" transactions, + close prints several kinds of "closing" and/or "opening" transactions, useful in various situations: migrating balances to a new journal file, - retaining earnings into equity, consolidating balances, viewing lot - costs.. Like print, it prints valid journal entries. You can copy + retaining earnings into equity, consolidating balances, viewing lot + costs.. Like print, it prints valid journal entries. You can copy these into your journal file(s) when you are happy with how they look. Flags: @@ -9864,22 +10280,22 @@ Data generation commands all - also round cost amounts to precision (can unbalance transactions) - close has six modes, selected by choosing one of the mode flags: - --clopen, --close (default), --open, --assert, --assign, or --retain. - They are all doing the same kind of operation, but with different de- + close has six modes, selected by choosing one of the mode flags: + --clopen, --close (default), --open, --assert, --assign, or --retain. + They are all doing the same kind of operation, but with different de- faults for different situations. - The journal entries generated by close will have a clopen: tag, which - is helpful when you want to exclude them from reports. If the main - journal file name contains a number, the tag's value will be that base - file name with the number incremented. Eg if the journal file is - 2025.journal, the tag will be clopen:2026. Or you can set the tag - value by providing an argument to the mode flag. Eg --close=foo or + The journal entries generated by close will have a clopen: tag, which + is helpful when you want to exclude them from reports. If the main + journal file name contains a number, the tag's value will be that base + file name with the number incremented. Eg if the journal file is + 2025.journal, the tag will be clopen:2026. Or you can set the tag + value by providing an argument to the mode flag. Eg --close=foo or --clopen=2025-main. close --clopen This is useful if migrating balances to a new journal file at the start - of a new year. It prints a "closing balances" transaction that zeroes + of a new year. It prints a "closing balances" transaction that zeroes out account balances (Asset and Liability accounts, by default), and an opposite "opening balances" transaction that restores them again. Typ- ically, you would run @@ -9890,40 +10306,40 @@ Data generation commands (and probably also update your LEDGER_FILE environment variable). Why might you do this ? If your reports are fast, you may not need it. - But at some point you will probably want to partition your data by - time, for performance or data integrity or regulatory reasons. A new - file or set of files per year is common. Then, having each file/file- - set "bookended" with opening and closing balance transactions will al- - low you to freely pick and choose which files to read - just the cur- + But at some point you will probably want to partition your data by + time, for performance or data integrity or regulatory reasons. A new + file or set of files per year is common. Then, having each file/file- + set "bookended" with opening and closing balance transactions will al- + low you to freely pick and choose which files to read - just the cur- rent year, any past year, any sequence of years, or all of them - while - showing correct account balances in each case. The earliest opening - balances transaction sets correct starting balances, and any later + showing correct account balances in each case. The earliest opening + balances transaction sets correct starting balances, and any later closing/opening pairs will harmlessly cancel each other out. - The balances will be transferred to and from equity:opening/closing - balances by default. You can override this by using --close-acct + The balances will be transferred to and from equity:opening/closing + balances by default. You can override this by using --close-acct and/or --open-acct. - You can select a different set of accounts to close/open by providing - an account query. Eg to add Equity accounts, provide arguments like - assets liabilities equity or type:ALE. When migrating to a new file, - you'll usually want to bring along the AL or ALE accounts, but not the + You can select a different set of accounts to close/open by providing + an account query. Eg to add Equity accounts, provide arguments like + assets liabilities equity or type:ALE. When migrating to a new file, + you'll usually want to bring along the AL or ALE accounts, but not the RX accounts (Revenue, Expense). - Assertions will be added indicating and checking the new balances of + Assertions will be added indicating and checking the new balances of the closed/opened accounts. close --close - This prints just the closing balances transaction of --clopen. It is + This prints just the closing balances transaction of --clopen. It is the default if you don't specify a mode. - More customisation options are described below. Among other things, + More customisation options are described below. Among other things, you can use close --close to generate a transaction moving the balances from any set of accounts, to a different account. (If you need to move just a portion of the balance, see hledger-move.) close --open - This prints just the opening balances transaction of --clopen. (It is + This prints just the opening balances transaction of --clopen. (It is similar to Ledger's equity command.) close --assert @@ -9933,29 +10349,29 @@ Data generation commands close --assign This prints a transaction that assigns the account balances as they are - on the end date (and adds an "assign:" tag). Unlike balance asser- + on the end date (and adds an "assign:" tag). Unlike balance asser- tions, assignments will post changes to balances as needed to reach the specified amounts. - This is another way to set starting balances when migrating to a new - file, and it will set them correctly even in the presence of earlier - files which do not have a closing balances transaction. However, it - can hide errors, and disturb the accounting equation, so --clopen is + This is another way to set starting balances when migrating to a new + file, and it will set them correctly even in the presence of earlier + files which do not have a closing balances transaction. However, it + can hide errors, and disturb the accounting equation, so --clopen is usually recommended. close --retain - This is like --close, but it closes Revenue and Expense account bal- - ances by default. They will be transferred to equity:retained earn- + This is like --close, but it closes Revenue and Expense account bal- + ances by default. They will be transferred to equity:retained earn- ings, or another account specified with --close-acct. - Revenues and expenses correspond to changes in equity. They are cate- + Revenues and expenses correspond to changes in equity. They are cate- gorised separately for reporting purposes, but traditionally at the end - of each accounting period, businesses consolidate them into equity, + of each accounting period, businesses consolidate them into equity, This is called "retaining earnings", or "closing the books". - In personal accounting, there's not much reason to do this, and most - people don't. (One reason to do it is to help the balancesheetequity - report show a zero total, demonstrating that the accounting equation + In personal accounting, there's not much reason to do this, and most + people don't. (One reason to do it is to help the balancesheetequity + report show a zero total, demonstrating that the accounting equation (A-L=E) is satisfied.) close customisation @@ -9967,56 +10383,60 @@ Data generation commands o the balancing account, with --close-acct=ACCT and/or --open-acct=ACCT - o the transaction descriptions, with --close-desc=DESC and + o the transaction descriptions, with --close-desc=DESC and --open-desc=DESC - o the transactions' clopen tag value, with a TAGVAL argument for the + o the transactions' clopen tag value, with a TAGVAL argument for the mode flag (see above). - By default, the closing date is yesterday, or the journal's end date, - whichever is later; and the opening date is always one day after the - closing date. You can change these by specifying a report end date; + By default, the closing date is yesterday, or the journal's end date, + whichever is later; and the opening date is always one day after the + closing date. You can change these by specifying a report end date; the closing date will be the last day of the report period. Eg -e 2024 means "close on 2023-12-31, open on 2024-01-01". With --x/--explicit, the balancing amount will be shown explicitly, and - if it involves multiple commodities, a separate posting will be gener- + if it involves multiple commodities, a separate posting will be gener- ated for each of them (similar to print -x). - With --interleaved, each individual transfer is shown with source and - destination postings next to each other (perhaps useful for trou- + With --interleaved, each individual transfer is shown with source and + destination postings next to each other (perhaps useful for trou- bleshooting). With --show-costs, balances' costs are also shown, with different costs - kept separate. This may generate very large journal entries, if you - have many currency conversions or investment transactions. close - --show-costs is currently the best way to view investment lots with - hledger. (To move or dispose of lots, see the more capable + kept separate. This may generate very large journal entries, if you + have many currency conversions or investment transactions. close + --show-costs is currently the best way to view investment lots with + hledger. (To move or dispose of lots, see the more capable hledger-move script.) close and balance assertions close adds balance assertions verifying that the accounts have been re- set to zero in a closing transaction or restored to their previous bal- - ances in an opening transaction. These provide useful error checking, + ances in an opening transaction. These provide useful error checking, but you can ignore them temporarily with -I, or remove them if you pre- fer. - Single-commodity, subaccount-exclusive balance assertions (=) are gen- - erated by default. This can be changed with --assertion-type='==*' + With --lots, balance assertions are not generated for the lot subac- + count postings in closing transactions (assertions on lot postings get + confusing, because they apply to the parent account). + + Single-commodity, subaccount-exclusive balance assertions (=) are gen- + erated by default. This can be changed with --assertion-type='==*' (eg). - When running close you should probably avoid using -C, -R, status: - (filtering by status or realness) or --auto (generating postings), + When running close you should probably avoid using -C, -R, status: + (filtering by status or realness) or --auto (generating postings), since the generated balance assertions would then require these. - Transactions with multiple dates (eg posting dates) spanning the file + Transactions with multiple dates (eg posting dates) spanning the file boundary also can disrupt the balance assertions: 2023-12-30 a purchase made in december, cleared in january expenses:food 5 assets:bank:checking -5 ; date: 2023-01-02 - To solve this you can transfer the money to and from a temporary ac- + To solve this you can transfer the money to and from a temporary ac- count, splitting the multi-day transaction into two single-day transac- tions: @@ -10037,7 +10457,7 @@ Data generation commands $ hledger close --retain -f 2022.journal -p 2022 >> 2022.journal - After this, to see 2022's revenues and expenses you must exclude the + After this, to see 2022's revenues and expenses you must exclude the retain earnings transaction: $ hledger -f 2022.journal is not:desc:'retain earnings' @@ -10049,12 +10469,12 @@ Data generation commands # copy/paste the closing transaction to the end of 2022.journal # copy/paste the opening transaction to the start of 2023.journal - After this, to see 2022's end-of-year balances you must exclude the + After this, to see 2022's end-of-year balances you must exclude the closing balances transaction: $ hledger -f 2022.journal bs not:desc:'closing balances' - For more flexibility, it helps to tag closing and opening transactions + For more flexibility, it helps to tag closing and opening transactions with eg clopen:NEWYEAR, then you can ensure correct balances by exclud- ing all opening/closing transactions except the first, like so: @@ -10070,7 +10490,7 @@ Data generation commands rewrite Print all transactions, rewriting the postings of matched transactions. - For now the only rewrite available is adding new postings, like print + For now the only rewrite available is adding new postings, like print --auto. Flags: @@ -10084,9 +10504,9 @@ Data generation commands patch tool This is a start at a generic rewriter of transaction entries. It reads - the default journal and prints the transactions, like print, but adds + the default journal and prints the transactions, like print, but adds one or more specified postings to any transactions matching QUERY. The - posting amounts can be fixed, or a multiplier of the existing transac- + posting amounts can be fixed, or a multiplier of the existing transac- tion's first posting amount. Examples: @@ -10102,7 +10522,7 @@ Data generation commands (reserve:grocery) *0.25 ; reserve 25% for grocery (reserve:) *0.25 ; reserve 25% for grocery - Note the single quotes to protect the dollar sign from bash, and the + Note the single quotes to protect the dollar sign from bash, and the two spaces between account and amount. More: @@ -10112,16 +10532,16 @@ Data generation commands $ hledger rewrite expenses:gifts --add-posting '(budget:gifts) *-1"' $ hledger rewrite ^income --add-posting '(budget:foreign currency) *0.25 JPY; diversify' - Argument for --add-posting option is a usual posting of transaction - with an exception for amount specification. More precisely, you can + Argument for --add-posting option is a usual posting of transaction + with an exception for amount specification. More precisely, you can use '*' (star symbol) before the amount to indicate that that this is a - factor for an amount of original matched posting. If the amount in- + factor for an amount of original matched posting. If the amount in- cludes a commodity name, the new posting amount will be in the new com- - modity; otherwise, it will be in the matched posting amount's commod- + modity; otherwise, it will be in the matched posting amount's commod- ity. Re-write rules in a file - During the run this tool will execute so called "Automated Transac- + During the run this tool will execute so called "Automated Transac- tions" found in any journal it process. I.e instead of specifying this operations in command line you can put them in a journal file. @@ -10136,7 +10556,7 @@ Data generation commands budget:gifts *-1 assets:budget *1 - Note that '=' (equality symbol) that is used instead of date in trans- + Note that '=' (equality symbol) that is used instead of date in trans- actions you usually write. It indicates the query by which you want to match the posting to add new ones. @@ -10149,12 +10569,12 @@ Data generation commands --add-posting 'assets:budget *1' \ > rewritten-tidy-output.journal - It is important to understand that relative order of such entries in - journal is important. You can re-use result of previously added post- + It is important to understand that relative order of such entries in + journal is important. You can re-use result of previously added post- ings. Diff output format - To use this tool for batch modification of your journal files you may + To use this tool for batch modification of your journal files you may find useful output in form of unified diff. $ hledger rewrite --diff -f examples/sample.journal '^income' --add-posting '(liabilities:tax) *.33' @@ -10178,10 +10598,10 @@ Data generation commands If you'll pass this through patch tool you'll get transactions contain- ing the posting that matches your query be updated. Note that multiple - files might be update according to list of input files specified via + files might be update according to list of input files specified via --file options and include directives inside of these files. - Be careful. Whole transaction being re-formatted in a style of output + Be careful. Whole transaction being re-formatted in a style of output from hledger print. See also: @@ -10189,17 +10609,17 @@ Data generation commands https://github.com/simonmichael/hledger/issues/99 rewrite vs. print --auto - This command predates print --auto, and currently does much the same + This command predates print --auto, and currently does much the same thing, but with these differences: - o with multiple files, rewrite lets rules in any file affect all other - files. print --auto uses standard directive scoping; rules affect + o with multiple files, rewrite lets rules in any file affect all other + files. print --auto uses standard directive scoping; rules affect only child files. - o rewrite's query limits which transactions can be rewritten; all are + o rewrite's query limits which transactions can be rewritten; all are printed. print --auto's query limits which transactions are printed. - o rewrite applies rules specified on command line or in the journal. + o rewrite applies rules specified on command line or in the journal. print --auto applies rules specified in the journal. Maintenance commands @@ -10209,62 +10629,64 @@ Maintenance commands Flags: no command-specific flags - hledger provides a number of built-in correctness checks to help vali- - date your data and prevent errors. Some are run automatically, some - when you enable --strict mode; or you can run any of them on demand by - providing them as arguments to the check command. check produces no + hledger provides a number of built-in correctness checks to help vali- + date your data and prevent errors. Some are run automatically, some + when you enable --strict mode; or you can run any of them on demand by + providing them as arguments to the check command. check produces no output and a zero exit code if all is well. Eg: hledger check # run basic checks hledger check -s # run basic and strict checks hledger check ordereddates payees # run basic checks and two others - If you are an Emacs user, you can also configure flycheck-hledger to + If you are an Emacs user, you can also configure flycheck-hledger to run these checks, providing instant feedback as you edit the journal. Here are the checks currently available. They are generally checked in - the order they are shown here, and only the first failure will be re- + the order they are shown here, and only the first failure will be re- ported. Basic checks - These important checks are performed by default, by almost all hledger + These important checks are performed by default, by almost all hledger commands: - o parseable - data files are in a supported format, with no syntax er- - rors and no invalid include directives. This ensures that all files + o parseable - data files are in a supported format, with no syntax er- + rors and no invalid include directives. This ensures that all files exist and are readable. o autobalanced - all transactions are balanced, after automatically in- - ferring missing amounts and conversion rates and then converting - amounts to cost. This ensures that each transaction's journal entry + ferring missing amounts and conversion rates and then converting + amounts to cost. This ensures that each transaction's journal entry is well formed. o assertions - all balance assertions in the journal are passing. Bal- - ance assertions are a strong defense against errors, catching many - problems. This check is on by default, but if it gets in your way, - you can disable it temporarily with -I/--ignore-assertions, or as a + ance assertions are a strong defense against errors, catching many + problems. This check is on by default, but if it gets in your way, + you can disable it temporarily with -I/--ignore-assertions, or as a default by adding that flag to your config file. (Then use -s/--strict or hledger check assertions when you want to enable it). Strict checks - When the -s/--strict flag is used (AKA strict mode), all commands will + When the -s/--strict flag is used (AKA strict mode), all commands will perform the following additional checks (and assertions, above). These provide extra error-catching power to help you keep your data clean and correct: - o balanced - like autobalanced, but implicit conversions between com- - modities are not allowed; all conversion transactions must use cost - notation or equity postings. This prevents wrong conversions caused + o balanced - like autobalanced, but implicit conversions between com- + modities are not allowed; all conversion transactions must use cost + notation or equity postings. This prevents wrong conversions caused by typos. - o commodities - all commodity symbols used must be declared. This + o commodities - all commodity symbols used must be declared. This guards against mistyping or omitting commodity symbols. - o accounts - all account names used must be declared. This prevents - the use of mis-spelled or outdated account names. + o accounts - all account names used must be declared. This prevents + the use of mis-spelled or outdated account names. (Except lot subac- + counts, like :{2026-01-15, $50}, which are automatically exempt; only + their base account needs to be declared.) Other checks - These are not wanted by everyone, but can be run using the check com- + These are not wanted by everyone, but can be run using the check com- mand: o tags - all tags used must be declared. This prevents mis-spelled tag @@ -10274,38 +10696,38 @@ Maintenance commands force you to declare any new payee name before using it. Most people will probably find this a bit too strict. - o ordereddates - within each file, transactions must be ordered by - date. This is a simple and effective error catcher. It's not in- + o ordereddates - within each file, transactions must be ordered by + date. This is a simple and effective error catcher. It's not in- cluded in strict mode, but you can add it by running hledger check -s ordereddates. If enabled, this check is performed before balance as- sertions. o recentassertions - all accounts with balance assertions must have one - that's within the 7 days before their latest posting. This will en- + that's within the 7 days before their latest posting. This will en- courage adding balance assertions for your active asset/liability ac- - counts, which in turn should encourage you to reconcile regularly - with those real world balances - another strong defense against er- - rors. (hledger close --assert >>$LEDGER_FILE is a convenient way to - add new balance assertions. Later these become quite redundant, and + counts, which in turn should encourage you to reconcile regularly + with those real world balances - another strong defense against er- + rors. (hledger close --assert >>$LEDGER_FILE is a convenient way to + add new balance assertions. Later these become quite redundant, and you might choose to remove them to reduce clutter.) o uniqueleafnames - no two accounts may have the same last account name - part (eg the checking in assets:bank:checking). This ensures each + part (eg the checking in assets:bank:checking). This ensures each account can be matched by a unique short name, easier to remember and to type. Custom checks - You can build your own custom checks with add-on command scripts. See + You can build your own custom checks with add-on command scripts. See also Cookbook > Scripting. Here are some examples from hledger/bin/: - o hledger-check-tagfiles - all tag values containing / exist as file + o hledger-check-tagfiles - all tag values containing / exist as file paths - o hledger-check-fancyassertions - more complex balance assertions are + o hledger-check-fancyassertions - more complex balance assertions are passing diff - Compares a particular account's transactions in two input files. It + Compares a particular account's transactions in two input files. It shows any transactions to this account which are in one file but not in the other. @@ -10313,16 +10735,16 @@ Maintenance commands no command-specific flags More precisely: for each posting affecting this account in either file, - this command looks for a corresponding posting in the other file which - posts the same amount to the same account (ignoring date, description, + this command looks for a corresponding posting in the other file which + posts the same amount to the same account (ignoring date, description, etc). Since it compares postings, not transactions, this also works when mul- tiple bank transactions have been combined into a single journal entry. - This command is useful eg if you have downloaded an account's transac- - tions from your bank (eg as CSV data): when hledger and your bank dis- - agree about the account balance, you can compare the bank data with + This command is useful eg if you have downloaded an account's transac- + tions from your bank (eg as CSV data): when hledger and your bank dis- + agree about the account balance, you can compare the bank data with your journal to find out the cause. Examples: @@ -10343,18 +10765,18 @@ Maintenance commands Flags: no command-specific flags - setup tests your hledger installation and prints a list of results, - sometimes with helpful hints. This is a good first command to run af- - ter installing hledger. Also after upgrading, or when something's not + setup tests your hledger installation and prints a list of results, + sometimes with helpful hints. This is a good first command to run af- + ter installing hledger. Also after upgrading, or when something's not working, or just when you want a reminder of where things are. - It makes one network request to detect the latest hledger release ver- - sion. It's ok if this fails or times out. It will use ANSI color by - default, unless disabled by NO_COLOR or --color=n. It does not use a + It makes one network request to detect the latest hledger release ver- + sion. It's ok if this fails or times out. It will use ANSI color by + default, unless disabled by NO_COLOR or --color=n. It does not use a pager or a config file. - It expects that the hledger version you are running is installed in - your PATH. If not, it will stop until you have done that (to keep + It expects that the hledger version you are running is installed in + your PATH. If not, it will stop until you have done that (to keep things simple). Example: @@ -10404,17 +10826,17 @@ Maintenance commands Flags: no command-specific flags - This command runs the unit tests built in to hledger and hledger-lib, - printing the results on stdout. If any test fails, the exit code will + This command runs the unit tests built in to hledger and hledger-lib, + printing the results on stdout. If any test fails, the exit code will be non-zero. - This is mainly used by hledger developers, but you can also use it to - sanity-check the installed hledger executable on your platform. All - tests are expected to pass - if you ever see a failure, please report + This is mainly used by hledger developers, but you can also use it to + sanity-check the installed hledger executable on your platform. All + tests are expected to pass - if you ever see a failure, please report as a bug! - Any arguments before a -- argument will be passed to the tasty test - runner as test-selecting -p patterns, and any arguments after -- will + Any arguments before a -- argument will be passed to the tasty test + runner as test-selecting -p patterns, and any arguments after -- will be passed to tasty unchanged. Examples: @@ -10424,7 +10846,7 @@ Maintenance commands $ hledger test -- -h # show tasty's options PART 5: COMMON TASKS - Here are some quick examples of how to do some basic tasks with + Here are some quick examples of how to do some basic tasks with hledger. Getting help @@ -10434,37 +10856,37 @@ PART 5: COMMON TASKS $ hledger --help # show common options $ hledger CMD --help # show CMD's options, common options and CMD's documentation - You can also view your hledger version's manual in several formats by + You can also view your hledger version's manual in several formats by using the help command. Eg: $ hledger help # show the hledger manual with info, man or $PAGER (best available) $ hledger help journal # show the journal topic in the hledger manual $ hledger help --help # find out more about the help command - To view manuals and introductory docs on the web, visit - https://hledger.org. Chat and mail list support and discussion + To view manuals and introductory docs on the web, visit + https://hledger.org. Chat and mail list support and discussion archives can be found at https://hledger.org/support. Constructing command lines - hledger has a flexible command line interface. We strive to keep it - simple and ergonomic, but if you run into one of the sharp edges de- + hledger has a flexible command line interface. We strive to keep it + simple and ergonomic, but if you run into one of the sharp edges de- scribed in OPTIONS, here are some tips that might help: - o command-specific options must go after the command (it's fine to put + o command-specific options must go after the command (it's fine to put common options there too: hledger CMD OPTS ARGS) - o you can run addon commands via hledger (hledger ui [ARGS]) or di- + o you can run addon commands via hledger (hledger ui [ARGS]) or di- rectly (hledger-ui [ARGS]) o enclose "problematic" arguments in single quotes - o if needed, also add a backslash to hide regular expression metachar- + o if needed, also add a backslash to hide regular expression metachar- acters from the shell o to see how a misbehaving command line is being parsed, add --debug=2. Starting a journal file - hledger looks for your accounting data in a journal file, + hledger looks for your accounting data in a journal file, $HOME/.hledger.journal by default: $ hledger stats @@ -10472,9 +10894,9 @@ PART 5: COMMON TASKS Please create it first, eg with "hledger add" or a text editor. Or, specify an existing journal file with -f or LEDGER_FILE. - You can override this by setting the LEDGER_FILE environment variable - (see below). It's a good practice to keep this important file under - version control, and to start a new file each year. So you could do + You can override this by setting the LEDGER_FILE environment variable + (see below). It's a good practice to keep this important file under + version control, and to start a new file each year. So you could do something like this: $ mkdir ~/finance @@ -10499,10 +10921,10 @@ PART 5: COMMON TASKS Setting LEDGER_FILE Set LEDGER_FILE on unix - It depends on your shell, but running these commands in the terminal + It depends on your shell, but running these commands in the terminal will work for many people; adapt if needed: - $ echo 'export LEDGER_FILE=~/finance/my.journal' >> ~/.profile + $ echo 'export LEDGER_FILE=~/finance/main.journal' >> ~/.profile $ source ~/.profile When correctly configured: @@ -10519,36 +10941,40 @@ PART 5: COMMON TASKS 1. Add an entry to ~/.MacOSX/environment.plist like { - "LEDGER_FILE" : "~/finance/my.journal" + "LEDGER_FILE" : "~/finance/main.journal" } - 2. Run killall Dock in a terminal window (or restart the machine), to + 2. Run killall Dock in a terminal window (or restart the machine), to complete the change. When correctly configured for GUI applications: - o apps started from the dock or a spotlight search, such as a GUI + o apps started from the dock or a spotlight search, such as a GUI Emacs, will be aware of the new LEDGER_FILE setting. Set LEDGER_FILE on Windows - Using the gui is easiest: + It can be easier to create a default file at + C:\Users\USER\.hledger.journal, and have it include your other files. + See I'm on Windows, how do I keep my files in AppData? + + Otherwise: using the gui is easiest: - 1. In task bar, search for environment variables, and choose "Edit en- + 1. In task bar, search for environment variables, and choose "Edit en- vironment variables for your account". - 2. Create or change a LEDGER_FILE setting in the User variables pane. - A typical value would be C:\Users\USERNAME\finance\my.journal. + 2. Create or change a LEDGER_FILE setting in the User variables pane. + A typical value would be C:\Users\USER\finance\main.journal. 3. Click OK to complete the change. - 4. And open a new powershell window. (Existing windows won't see the + 4. And open a new powershell window. (Existing windows won't see the change.) Or at the command line, you can do it this way: - 1. In a powershell window, run [Environment]::SetEnvironmentVari- - able("LEDGER_FILE", "C:\User\USERNAME\finance\my.journal", [Sys- - tem.EnvironmentVariableTarget]::User) + 1. In a powershell window, run [Environment]::SetEnvironmentVari- + able("LEDGER_FILE", "C:\User\USER\finance\main.journal", [System.En- + vironmentVariableTarget]::User) 2. And open a new powershell window. (Existing windows won't see the change.) @@ -10950,4 +11376,4 @@ LICENSE SEE ALSO hledger(1), hledger-ui(1), hledger-web(1), ledger(1) -hledger-1.51.99 December 2025 HLEDGER(1) +hledger-1.51.99 February 2026 HLEDGER(1) diff --git a/hledger/package.yaml b/hledger/package.yaml index fd680aa02c4..0ea9cfeb12c 100644 --- a/hledger/package.yaml +++ b/hledger/package.yaml @@ -1,5 +1,5 @@ name: hledger -version: 1.52.99 +version: 1.99 license: GPL-3.0-or-later maintainer: Simon Michael author: Simon Michael @@ -102,7 +102,7 @@ flags: dependencies: - base >=4.18 && <4.23 -- hledger-lib >=1.52.99 && <1.53 +- hledger-lib >=1.99 && <1.100 - aeson >=1 && <2.3 - ansi-terminal >=0.9 - bytestring @@ -143,7 +143,7 @@ dependencies: - utility-ht >=0.0.13 - wizards >=1.0 -cpp-options: -DVERSION="1.52.99" +cpp-options: -DVERSION="1.99" language: GHC2021 diff --git a/hledger/test/check-lots.test b/hledger/test/check-lots.test new file mode 100644 index 00000000000..7d8bf00657b --- /dev/null +++ b/hledger/test/check-lots.test @@ -0,0 +1,65 @@ +# * check lots + +# ** 1. succeeds with valid buy-only lot data +< +2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:checking + +$ hledger -f - check lots + +# ** 2. succeeds with valid buy and sell +< +2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:checking + +2026-02-01 sell + assets:stocks -10 AAPL {$50} @ $55 + assets:cash $550 + revenues:gains + +$ hledger -f - check lots + +# ** 3. fails with invalid lots: tag value on commodity declaration +< +commodity AAPL ; lots: BADMETHOD + +2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:checking + +$ hledger -f - check lots +>2 /unrecognised lots: tag value/ +>=1 + +# ** 4. check lots fails when selling more lots than available (strict mode) +< +2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:checking + +2026-02-01 sell + assets:stocks -15 AAPL {$50} @ $55 + assets:cash $825 + revenues:gains + +$ hledger -f - check lots +>2 /insufficient lots/ +>=1 + +# ** 5. check lots fails on disposal balance error +< +2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:checking + +2026-02-01 sell + assets:stocks -10 AAPL {$50} @ $55 + assets:cash $550 + revenues:gains $-999 + +$ hledger -f - check lots +>2 /unbalanced at cost basis/ +>=1 + diff --git a/hledger/test/journal/commodity-tags.test b/hledger/test/journal/commodity-tags.test index b42142e7c6d..a1e573c1670 100644 --- a/hledger/test/journal/commodity-tags.test +++ b/hledger/test/journal/commodity-tags.test @@ -37,3 +37,11 @@ account assets:bank ; source:account $ hledger -f - register tag:source=posting 2024-01-01 test assets:bank $100 $100 +# ** 4. Commodity tags can be queried with the commodities command. +< +commodity AAPL ; lots: +commodity $ + +$ hledger -f - commodities tag:lots +AAPL + diff --git a/hledger/test/journal/infer-cost.test b/hledger/test/journal/infer-cost.test new file mode 100644 index 00000000000..35d7968f232 --- /dev/null +++ b/hledger/test/journal/infer-cost.test @@ -0,0 +1,54 @@ +# * Infer transacted cost (@ ...) from cost basis ({...}). + +# ** 1. Transacted cost is inferred from cost basis when amount is positive. +# print shows the original form by default. +< +2024-01-01 buy stock + assets:brokerage 10 AAPL {$100} + assets:cash + +$ hledger -f - print +2024-01-01 buy stock + assets:brokerage 10 AAPL {$100} + assets:cash + +>= + +# ** 2. With -x, the inferred transacted cost is shown. +< +2024-01-01 buy stock + assets:brokerage 10 AAPL {$100} + assets:cash + +$ hledger -f - print -x +2024-01-01 buy stock + assets:brokerage 10 AAPL {$100} @ $100 + assets:cash $-1000 + +>= + +# ** 3. Transacted cost is not inferred for negative amounts (dispose postings). +< +2024-01-01 sell stock + assets:brokerage -10 AAPL {$100} + assets:cash + +$ hledger -f - print +2024-01-01 sell stock + assets:brokerage -10 AAPL {$100} + assets:cash + +>= + +# ** 4. Transacted cost is not inferred when already present. +< +2024-01-01 buy stock + assets:brokerage 10 AAPL {$95} @ $100 + assets:cash + +$ hledger -f - print +2024-01-01 buy stock + assets:brokerage 10 AAPL {$95} @ $100 + assets:cash + +>= diff --git a/hledger/test/journal/infer-costbasis.test b/hledger/test/journal/infer-costbasis.test new file mode 100644 index 00000000000..e686ddeee4b --- /dev/null +++ b/hledger/test/journal/infer-costbasis.test @@ -0,0 +1,93 @@ +# * Cost basis ({}) on lot postings. +# Acquisitions in lotful commodities/accounts require explicit {} cost basis annotation. +# Transacted cost (@ @@) is inferred from cost basis for acquire postings, to help transaction balancing. + +# ** 1. Explicit cost basis on lotful commodity is shown; transacted cost is inferred. +< +commodity AAPL ; lots: + +2024-01-01 buy stock + assets:brokerage 10 AAPL {$100} + assets:cash + +$ hledger -f - print +2024-01-01 buy stock + assets:brokerage 10 AAPL {$100} + assets:cash + +>= + +# ** 2. Cost basis is not inferred without lots: tag. +< +commodity AAPL + +2024-01-01 buy stock + assets:brokerage 10 AAPL @ $100 + assets:cash + +$ hledger -f - print +2024-01-01 buy stock + assets:brokerage 10 AAPL @ $100 + assets:cash + +>= + +# ** 3. Cost basis is not inferred for negative amounts (dispose postings). +< +commodity AAPL ; lots: + +2024-01-01 sell stock + assets:brokerage -10 AAPL @ $150 + assets:cash + +$ hledger -f - print +2024-01-01 sell stock + assets:brokerage -10 AAPL @ $150 + assets:cash + +>= + +# ** 4. Cost basis is not inferred when it has been explicitly recorded. +< +commodity AAPL ; lots: + +2024-01-01 buy stock + assets:brokerage 10 AAPL {$95} @ $100 + assets:cash + +$ hledger -f - print +2024-01-01 buy stock + assets:brokerage 10 AAPL {$95} @ $100 + assets:cash + +>= + +# ** 5. Without explicit cost basis, a lotful commodity posting with @ is classified as acquire. +< +commodity AAPL ; lots: + +2024-01-01 buy stock + assets:brokerage 10 AAPL @@ $1000 + assets:cash + +$ hledger -f - print --verbose-tags +2024-01-01 buy stock + assets:brokerage 10 AAPL @@ $1000 ; ptype: acquire + assets:cash + +>= + +# # ** 0. Posting date is used for inferred cost basis date +# < +# commodity AAPL ; lots: + +# 2024-03-15 buy stock +# assets:brokerage 10 AAPL @ $100 ; date:3/16 +# assets:cash + +# $ hledger -f - print +# 2024-03-15 buy stock +# assets:brokerage 10 AAPL @ $100 {$100} [2024-03-16] ; date:3/16 +# assets:cash + +# >= diff --git a/hledger/test/journal/lots-acquire-entries.test b/hledger/test/journal/lots-acquire-entries.test new file mode 100644 index 00000000000..2dbd9a4547e --- /dev/null +++ b/hledger/test/journal/lots-acquire-entries.test @@ -0,0 +1,185 @@ +# Testing lot acquisition journal entry variations, systematically. + +# * Acquire + +# ** With commodity/account not declared lotful + +# *** 1. acquire: implicit cost, non-lotful commmodity/account, asset. No lot posting detected. +< +2026-01-01 buy + assets:cash -$500 + assets:broker 10 AAPL + +$ hledger -f- print --lots --verbose-tags +2026-01-01 buy + assets:cash $-500 + assets:broker 10 AAPL + +>= + +# *** 2. acquire: @ $50, non-lotful commodity/account, asset. No lot posting detected. +< +2026-01-01 buy + assets:broker 10 AAPL @ $50 + assets:cash -$500 + +$ hledger -f- print --lots --verbose-tags +2026-01-01 buy + assets:broker 10 AAPL @ $50 + assets:cash $-500 + +>= + +# *** 3. acquire: {$50}, non-lotful commodity/account, non-asset. Cost basis forces acquisition even from non-asset account. +< +2026-01-01 buy + equity:opening 10 AAPL {$50} + assets:cash -$500 + +$ hledger -f- print --lots --verbose-tags +2026-01-01 buy + equity:opening:{2026-01-01, $50} 10 AAPL {$50} ; ptype: acquire + assets:cash $-500 + +>= + +# *** 4. acquire: {$50}, non-lotful commodity/account, asset. Acquisition detected. +< +2026-01-01 buy + assets:broker 10 AAPL {$50} + assets:cash -$500 + +$ hledger -f- print --lots --verbose-tags +2026-01-01 buy + assets:broker:{2026-01-01, $50} 10 AAPL {$50} ; ptype: acquire + assets:cash $-500 + +>= + +# ** With lotful commodity/account + +# *** 5. acquire: implicit cost, lotful commodity. Bare acquire detected, cost inferred from balancer. +< +commodity AAPL ; lots: +2026-01-01 buy + assets:broker 10 AAPL + assets:cash -$500 + +$ hledger -f- print --lots --verbose-tags +2026-01-01 buy + assets:broker:{2026-01-01, $50} 10 AAPL ; ptype: acquire + assets:cash $-500 + +>= + +# *** 6. acquire: @ $50, lotful commodity. Bare acquire detected, cost from @. +< +commodity AAPL ; lots: +2026-01-01 buy + assets:broker 10 AAPL @ $50 + assets:cash -$500 + +$ hledger -f- print --lots --verbose-tags +2026-01-01 buy + assets:broker:{2026-01-01, $50} 10 AAPL @ $50 ; ptype: acquire + assets:cash $-500 + +>= + +# *** 7. acquire: @@ $500, lotful commodity. Bare acquire detected, cost from @@. +< +commodity AAPL ; lots: +2026-01-01 buy + assets:broker 10 AAPL @@ $500 + assets:cash -$500 + +$ hledger -f- print --lots --verbose-tags +2026-01-01 buy + assets:broker:{2026-01-01, $50} 10 AAPL @@ $500 ; ptype: acquire + assets:cash $-500 + +>= + +# *** 8. acquire: {}. Acquisition detected, cost basis inferred from implicit cost. +< +commodity AAPL ; lots: +2026-01-01 buy + assets:broker 10 AAPL {} + assets:cash -$500 + +$ hledger -f- print --lots --verbose-tags +2026-01-01 buy + assets:broker:{2026-01-01, $50} 10 AAPL {} ; ptype: acquire + assets:cash $-500 + +>= + +# *** 9. acquire: {} @ $50. Acquisition detected, cost basis inferred from unit transacted cost. +< +commodity AAPL ; lots: +2026-01-01 buy + assets:broker 10 AAPL {} @ $50 + assets:cash -$500 + +$ hledger -f- print --lots --verbose-tags +2026-01-01 buy + assets:broker:{2026-01-01, $50} 10 AAPL {} @ $50 ; ptype: acquire + assets:cash $-500 + +>= + +# *** 10. acquire: {} @@ $500. Acquisition detected, cost basis inferred from total transacted cost. +< +commodity AAPL ; lots: +2026-01-01 buy + assets:broker 10 AAPL {} @@ $500 + assets:cash -$500 + +$ hledger -f- print --lots --verbose-tags +2026-01-01 buy + assets:broker:{2026-01-01, $50} 10 AAPL {} @@ $500 ; ptype: acquire + assets:cash $-500 + +>= + +# *** 11. acquire: {$50}. Acquisition detected. +< +commodity AAPL ; lots: +2026-01-01 buy + assets:broker 10 AAPL {$50} + assets:cash -$500 + +$ hledger -f- print --lots --verbose-tags +2026-01-01 buy + assets:broker:{2026-01-01, $50} 10 AAPL {$50} ; ptype: acquire + assets:cash $-500 + +>= + +# *** 12. acquire: {$50} @ $50. Acquisition detected. +< +commodity AAPL ; lots: +2026-01-01 buy + assets:broker 10 AAPL {$50} @ $50 + assets:cash -$500 + +$ hledger -f- print --lots --verbose-tags +2026-01-01 buy + assets:broker:{2026-01-01, $50} 10 AAPL {$50} @ $50 ; ptype: acquire + assets:cash $-500 + +>= + +# *** 13. acquire: {$50} @@ $500. Acquisition detected. +< +commodity AAPL ; lots: +2026-01-01 buy + assets:broker 10 AAPL {$50} @@ $500 + assets:cash -$500 + +$ hledger -f- print --lots --verbose-tags +2026-01-01 buy + assets:broker:{2026-01-01, $50} 10 AAPL {$50} @@ $500 ; ptype: acquire + assets:cash $-500 + +>= diff --git a/hledger/test/journal/lots-acquire.test b/hledger/test/journal/lots-acquire.test new file mode 100644 index 00000000000..2e7055ce7a5 --- /dev/null +++ b/hledger/test/journal/lots-acquire.test @@ -0,0 +1,300 @@ +# Testing lot acquisition scenarios. + +# * Lot acquisition + +# ** 1. Simple acquire - positive amount to Asset account with cost basis - gets ptype:acquire tag +< +2026-01-01 Buy stock + assets:broker 10 AAPL {$100} + assets:cash -$1000 + +$ hledger -f- print --verbose-tags +2026-01-01 Buy stock + assets:broker 10 AAPL {$100} ; ptype: acquire + assets:cash $-1000 + +>= + +# ** 2. No cost basis (@ instead of {}) - not recognised +< +2026-01-04 Buy with @ + assets:broker 10 AAPL @ $100 + assets:cash -$1000 + +$ hledger -f- print +2026-01-04 Buy with @ + assets:broker 10 AAPL @ $100 + assets:cash $-1000 + +>= + +# ** 3. Non-Asset account - should not be recognised even with cost basis +< +2026-01-05 Equity + equity:conversion 10 AAPL {$100} + assets:cash -$1000 + +$ hledger -f- print +2026-01-05 Equity + equity:conversion 10 AAPL {$100} + assets:cash $-1000 + +>= + +# ** 4. Cash accounts are recognised as asset accounts for lot classification. +< +commodity USDC ; lots: +account assets:usdc ; type: Cash + +2026-01-06 Buy USDC + assets:usdc 1000 USDC {$1} + assets:cash -$1000 + +$ hledger -f- print --verbose-tags +2026-01-06 Buy USDC + assets:usdc 1000 USDC {$1} ; ptype: acquire + assets:cash $-1000 + +>= + +# ** 5. Account-level lots: tag also triggers lot classification. +< +account assets:broker ; lots: + +2026-01-07 Buy stock + assets:broker 10 AAPL {$100} + assets:cash -$1000 + +$ hledger -f- print --verbose-tags +2026-01-07 Buy stock + assets:broker 10 AAPL {$100} ; ptype: acquire + assets:cash $-1000 + +>= + +# ** 6. With --lots, acquired lots are displayed in their own subaccounts. +< +2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:checking + +2026-02-01 buy more + assets:stocks 5 AAPL {$55} + assets:checking + +$ hledger -f- bal assets:stocks --lots -N + 10 AAPL assets:stocks:{2026-01-15, $50} + 5 AAPL assets:stocks:{2026-02-01, $55} + +# ** 7. Same-date acquisitions get labels added to ensure unique lot ids per commodity. +< +2026-01-15 buy morning + assets:stocks 10 AAPL {$50} + assets:checking + +2026-01-15 buy afternoon + assets:stocks 5 AAPL {$55} + assets:checking + +$ hledger -f- bal assets:stocks --lots -N + 10 AAPL assets:stocks:{2026-01-15, "0001", $50} + 5 AAPL assets:stocks:{2026-01-15, "0002", $55} + +# ** 8. User-provided labels are preserved. +< +2026-01-15 buy + assets:stocks 10 AAPL {$50} [2026-01-15] (morning-buy) + assets:checking + +$ hledger -f- bal assets:stocks --lots -N + 10 AAPL assets:stocks:{2026-01-15, "morning-buy", $50} + +# ** 9. User-provided labels resulting in non-unique lot ids are an error, +# with a source position and excerpt pointing to the duplicate transaction. +< +2026-01-15 buy first + assets:stocks 10 AAPL {$50} (mybuy) + assets:checking + +2026-01-15 buy second + assets:stocks 5 AAPL {$55} (mybuy) + assets:checking + +$ hledger -f- bal assets:stocks --lots +>2 /-:5:/ +>=1 + +# ** 10. Bare acquire with explicit @ on lotful commodity - classified as acquire. +< +commodity AAPL ; lots: + +2026-01-15 buy + assets:stocks 10 AAPL @ $50 + assets:cash -$500 + +$ hledger -f- print --verbose-tags +2026-01-15 buy + assets:stocks 10 AAPL @ $50 ; ptype: acquire + assets:cash $-500 + +>= + +# ** 11. Bare acquire with @ on lotful commodity creates a lot subaccount with --lots. +< +commodity AAPL ; lots: + +2026-01-15 buy + assets:stocks 10 AAPL @ $50 + assets:cash -$500 + +$ hledger -f- bal assets:stocks --lots -N + 10 AAPL assets:stocks:{2026-01-15, $50} + +# ** 12. Bare acquire with balancer-inferred cost on lotful commodity - classified as acquire. +< +commodity AAPL ; lots: + +2026-01-15 buy + assets:stocks 10 AAPL + assets:cash -$500 + +$ hledger -f- print --verbose-tags +2026-01-15 buy + assets:stocks 10 AAPL ; ptype: acquire + assets:cash $-500 + +>= + +# ** 13. Bare acquire with balancer-inferred cost creates a lot subaccount with --lots. +< +commodity AAPL ; lots: + +2026-01-15 buy + assets:stocks 10 AAPL + assets:cash -$500 + +$ hledger -f- bal assets:stocks --lots -N + 10 AAPL assets:stocks:{2026-01-15, $50} + +# ** 14. Without lots: tag, bare positive posting with @ is NOT classified as acquire. +< +2026-01-15 buy + assets:stocks 10 AAPL @ $50 + assets:cash -$500 + +$ hledger -f- print +2026-01-15 buy + assets:stocks 10 AAPL @ $50 + assets:cash $-500 + +>= + +# ** 15. Transfer-to takes priority over bare acquire when transfer-from counterpart exists. +< +commodity AAPL ; lots: + +2026-01-15 buy + assets:old 10 AAPL {$50} + assets:cash + +2026-02-01 transfer + assets:old -10 AAPL {$50} + assets:new 10 AAPL + +$ hledger -f- print --verbose-tags date:2026-02-01 +2026-02-01 transfer + assets:old -10 AAPL {$50} ; ptype: transfer-from + assets:new 10 AAPL ; ptype: transfer-to + +>= + +# ** 16. Bare lotful transfer: both postings bare (no {}) - still classified as transfer. +< +commodity AAPL ; lots: + +2026-01-15 buy + assets:old 10 AAPL {$50} + assets:cash + +2026-02-01 transfer + assets:old -10 AAPL + assets:new 10 AAPL + +$ hledger -f- print --verbose-tags date:2026-02-01 +2026-02-01 transfer + assets:old -10 AAPL ; ptype: transfer-from + assets:new 10 AAPL ; ptype: transfer-to + +>= + +# ** 17. Zero-amount balance assertion posting on lotful commodity is NOT classified as acquire. +< +commodity AAPL ; lots: + +2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:cash + +2026-02-01 assert + assets:stocks 0 AAPL = 10 AAPL + +$ hledger -f- bal assets:stocks --lots -N + 10 AAPL assets:stocks:{2026-01-15, $50} + +# ** 18. Bare positive lotful posting without inferable cost is a hard error with --lots. +< +commodity AAPL ; lots: + +2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:cash + +2026-02-01 gift received + assets:stocks 5 AAPL + equity:opening + +$ hledger -f- bal assets:stocks --lots 2>/dev/null +>=1 + +# ** 19. Explicit lot subaccount infers cost basis from account name (no {} on amount). +< +commodity AAPL ; lots: + +2026-01-15 buy + assets:stocks:{2026-01-15, $50} 10 AAPL + assets:cash + +$ hledger -f- print --lots -x +2026-01-15 buy + assets:stocks:{2026-01-15, $50} 10 AAPL {2026-01-15, $50} @ $50 + assets:cash $-500 + +>= + +# ** 20. Explicit lot subaccount with matching {} on amount - no double-append. +< +commodity AAPL ; lots: + +2026-01-15 buy + assets:stocks:{2026-01-15, $50} 10 AAPL {$50} + assets:cash + +$ hledger -f- print --lots -x +2026-01-15 buy + assets:stocks:{2026-01-15, $50} 10 AAPL {2026-01-15, $50} @ $50 + assets:cash $-500 + +>= + +# ** 21. Conflicting cost basis between lot subaccount and amount annotation is an error. +< +commodity AAPL ; lots: + +2026-01-15 buy + assets:stocks:{2026-01-15, $50} 10 AAPL {$99} + assets:cash + +$ hledger -f- print --lots +>2 /conflicting cost basis cost/ +>=1 + diff --git a/hledger/test/journal/lots-clopen.test b/hledger/test/journal/lots-clopen.test new file mode 100644 index 00000000000..98c1d494865 --- /dev/null +++ b/hledger/test/journal/lots-clopen.test @@ -0,0 +1,90 @@ +# * Tests for close --clopen with --lots (clopening workflow). + +# ** 1. Equity transfers with --lots: in a closing balances transaction, +# lot postings transferring to equity are classified as transfers, not disposals. +# Without this, --lots would error because dispose postings require a selling price. +< +commodity AAPL ; lots: + +2026-01-01 starting balance + assets:cash $1000 + equity:start + +2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:cash + +2026-07-01 buy more + assets:stocks 5 AAPL {$70} + assets:cash + +2026-12-31 closing balances + assets:cash $-150 = $0 + assets:stocks:{2026-01-15, $50} -10 AAPL + assets:stocks:{2026-07-01, $70} -5 AAPL + equity:start + +$ hledger -f- check --lots + +# ** 2. Opening equity transfers with --lots: in an opening balances transaction, +# positive lot postings from equity are classified as transfer-to, not acquire. +# This is the second half of a close --clopen --lots workflow. +< +commodity AAPL ; lots: + +2027-01-01 opening balances + assets:cash $150 + assets:stocks:{2026-01-15, $50} 10 AAPL + assets:stocks:{2026-07-01, $70} 5 AAPL + equity:start + +$ hledger -f- check --lots + +# ** 3. Full clopen round-trip: closing followed by opening, check --lots succeeds. +< +commodity AAPL ; lots: + +2026-01-01 starting balance + assets:cash $1000 + equity:start + +2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:cash + +2026-07-01 buy more + assets:stocks 5 AAPL {$70} + assets:cash + +2026-12-31 closing balances + assets:cash $-150 = $0 + assets:stocks:{2026-01-15, $50} -10 AAPL + assets:stocks:{2026-07-01, $70} -5 AAPL + equity:start + +2027-01-01 opening balances + assets:cash $150 + assets:stocks:{2026-01-15, $50} 10 AAPL + assets:stocks:{2026-07-01, $70} 5 AAPL + equity:start + +$ hledger -f- check --lots + +# ** 4. Opening equity transfer postings get ptype:transfer-to (visible with --verbose-tags). +< +commodity AAPL ; lots: + +2027-01-01 opening balances + assets:cash $150 + assets:stocks:{2026-01-15, $50} 10 AAPL + assets:stocks:{2026-07-01, $70} 5 AAPL + equity:start + +$ hledger -f- print --lots --verbose-tags +2027-01-01 opening balances + assets:cash $150 + assets:stocks:{2026-01-15, $50} 10 AAPL {2026-01-15, $50} ; ptype: transfer-to + assets:stocks:{2026-07-01, $70} 5 AAPL {2026-07-01, $70} ; ptype: transfer-to + equity:start + +>= diff --git a/hledger/test/journal/lots-dispose-entries.test b/hledger/test/journal/lots-dispose-entries.test new file mode 100644 index 00000000000..f2018bd53a8 --- /dev/null +++ b/hledger/test/journal/lots-dispose-entries.test @@ -0,0 +1,252 @@ +# Testing lot disposal journal entry variations, systematically. + +# * Dispose + +# ** With commodity/account not declared lotful + +# *** 1. dispose: {$50} @ $55, non-lotful commodity/account, non-asset. We'll let cost basis force a disposal, even from an unknown account type. +< +2026-01-01 buy + equity:opening 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 sell + equity:opening -5 AAPL {$50} @ $55 + assets:cash $275 + revenue:gains -$25 + +$ hledger -f- print desc:sell --lots --verbose-tags +2026-02-01 sell + equity:opening:{2026-01-01, $50} -5 AAPL {$50} @ $55 ; ptype: dispose + assets:cash $275 + revenue:gains $-25 ; ptype: gain + +>= + +# *** 2. dispose: {$50} @ $55, non-lotful commodity/account, asset. Disposal detected. +< +2026-01-01 buy + assets:broker 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 sell + assets:broker -5 AAPL {$50} @ $55 + assets:cash $275 + revenue:gains -$25 + +$ hledger -f- print desc:sell --lots --verbose-tags +2026-02-01 sell + assets:broker:{2026-01-01, $50} -5 AAPL {$50} @ $55 ; ptype: dispose + assets:cash $275 + revenue:gains $-25 ; ptype: gain + +>= + +# ** With lotful commodity/account + +# *** 3. dispose: implicit selling price. FIFO lot selected, disposal detected. +< +commodity AAPL ; lots: + +2026-01-01 buy + assets:broker 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 sell + assets:broker -5 AAPL + assets:cash $275 + revenue:gains -$25 + +$ hledger -f- print desc:sell --lots --verbose-tags +2026-02-01 sell + assets:broker:{2026-01-01, $50} -5 AAPL ; ptype: dispose + assets:cash $275 + revenue:gains $-25 ; ptype: gain + +>= + +# *** 4. dispose: @ $55. FIFO lot selected, disposal detected. +< +commodity AAPL ; lots: + +2026-01-01 buy + assets:broker 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 sell + assets:broker -5 AAPL @ $55 + assets:cash $275 + revenue:gains -$25 + +$ hledger -f- print desc:sell --lots --verbose-tags +2026-02-01 sell + assets:broker:{2026-01-01, $50} -5 AAPL @ $55 ; ptype: dispose + assets:cash $275 + revenue:gains $-25 ; ptype: gain + +>= + +# *** 5. dispose: {$50}, implicit selling price, explicit lot selector. Disposal detected. +< +commodity AAPL ; lots: + +2026-01-01 buy + assets:broker 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 sell + assets:broker -5 AAPL {$50} + assets:cash $275 + revenue:gains -$25 + +$ hledger -f- print desc:sell --lots --verbose-tags +2026-02-01 sell + assets:broker:{2026-01-01, $50} -5 AAPL {$50} ; ptype: dispose + assets:cash $275 + revenue:gains $-25 ; ptype: gain + +>= + +# *** 6. dispose: {} @ $55. Disposal detected. +< +commodity AAPL ; lots: + +2026-01-01 buy + assets:broker 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 sell + assets:broker -5 AAPL {} @ $55 + assets:cash $275 + revenue:gains -$25 + +$ hledger -f- print desc:sell --lots --verbose-tags +2026-02-01 sell + assets:broker:{2026-01-01, $50} -5 AAPL {} @ $55 ; ptype: dispose + assets:cash $275 + revenue:gains $-25 ; ptype: gain + +>= + +# *** 7. dispose: {$50} @ $55. Disposal detected. +# Also shows that a Gain-type account can be declared explicitly with type: G. +< +commodity AAPL ; lots: +account capital gains ; type: G + +2026-01-01 buy + assets:broker 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 sell + assets:broker -5 AAPL {$50} @ $55 + assets:cash $275 + capital gains -$25 + +$ hledger -f- print desc:sell --lots --verbose-tags +2026-02-01 sell + assets:broker:{2026-01-01, $50} -5 AAPL {$50} @ $55 ; ptype: dispose + assets:cash $275 + capital gains $-25 ; ptype: gain + +>= + +# *** 8. dispose: {$50} @@ $275. Disposal detected. +< +commodity AAPL ; lots: + +2026-01-01 buy + assets:broker 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 sell + assets:broker -5 AAPL {$50} @@ $275 + assets:cash $275 + revenue:gains -$25 + +$ hledger -f- print desc:sell --lots --verbose-tags +2026-02-01 sell + assets:broker:{2026-01-01, $50} -5 AAPL {$50} @@ $275 ; ptype: dispose + assets:cash $275 + revenue:gains $-25 ; ptype: gain + +>= + +# ** Disposal balancing validation + +# *** 9. dispose at cost: no gain/loss, transaction balances at cost basis. +< +commodity AAPL ; lots: + +2026-01-01 buy + assets:broker 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 sell + assets:broker -5 AAPL {$50} @ $50 + assets:cash $250 + +$ hledger -f- print desc:sell --lots --verbose-tags +2026-02-01 sell + assets:broker:{2026-01-01, $50} -5 AAPL {$50} @ $50 ; ptype: dispose + assets:cash $250 + +>= + +# *** 10. dispose: wrong gain amount. Disposal balancing rejects this. +< +commodity AAPL ; lots: + +2026-01-01 buy + assets:broker 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 sell + assets:broker -5 AAPL {$50} @ $55 + assets:cash $275 + revenue:gains -$30 + +$ hledger -f- print desc:sell --lots --verbose-tags +>2 /disposal transaction is unbalanced at cost basis/ +>=1 + +# *** 11. dispose: missing gain posting is inferred from Gain account. +< +commodity AAPL ; lots: +account revenue:gains ; type: G + +2026-01-01 buy + assets:broker 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 sell + assets:broker -5 AAPL {$50} @ $55 + assets:cash $275 + +$ hledger -f- print desc:sell --lots --verbose-tags +2026-02-01 sell + assets:broker:{2026-01-01, $50} -5 AAPL {$50} @ $55 ; ptype: dispose + assets:cash $275 + revenue:gains $-25 ; ptype: gain, generated-posting: + +>= + +# *** 12. dispose: missing gain posting, no Gain account declared → defaults to revenue:gains. +< +commodity AAPL ; lots: + +2026-01-01 buy + assets:broker 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 sell + assets:broker -5 AAPL {$50} @ $55 + assets:cash $275 + +$ hledger -f- print desc:sell --lots --verbose-tags +2026-02-01 sell + assets:broker:{2026-01-01, $50} -5 AAPL {$50} @ $55 ; ptype: dispose + assets:cash $275 + revenue:gains $-25 ; ptype: gain, generated-posting: + +>= diff --git a/hledger/test/journal/lots-dispose.test b/hledger/test/journal/lots-dispose.test new file mode 100644 index 00000000000..768e4925f24 --- /dev/null +++ b/hledger/test/journal/lots-dispose.test @@ -0,0 +1,413 @@ +# Testing lot disposal scenarios. + +# * Lot disposal + +# ** 1. Simple dispose - negative amount to Asset account with cost basis - gets ptype:dispose tag +< +2026-01-02 Sell stock + assets:broker -5 AAPL {$100} + assets:cash $500 + +$ hledger -f- print --verbose-tags +2026-01-02 Sell stock + assets:broker -5 AAPL {$100} ; ptype: dispose + assets:cash $500 + +>= + +# ** 2. Simple single-lot disposal: the sell posting selects the appropriate lot subaccount. +< +2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:checking + +2026-02-01 sell + assets:stocks -5 AAPL {$50} @ $50 + assets:cash $250 + +$ hledger -f- bal assets:stocks --lots -N + 5 AAPL assets:stocks:{2026-01-15, $50} + +# ** 3. Full lot consumption: the lot is fully consumed. +< +2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:checking + +2026-02-01 sell all + assets:stocks -10 AAPL {$50} @ $50 + assets:cash $500 + +$ hledger -f- bal assets:stocks --lots -N + +# ** 4. FIFO multi-lot disposal: sell with {} can select multiple lots, oldest first. +< +2026-01-15 buy cheap + assets:stocks 10 AAPL {$50} + assets:checking + +2026-02-01 buy expensive + assets:stocks 10 AAPL {$60} + assets:checking + +2026-03-01 sell some + assets:stocks -15 AAPL {} @ $55 + assets:cash $825 + revenue:gains -$25 + +$ hledger -f- bal assets:stocks --lots -N + 5 AAPL assets:stocks:{2026-02-01, $60} + +# ** 5. Insufficient lots error: disposing more than is available at the specified cost fails. +< +2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:checking + +2026-02-01 sell too many + assets:stocks -15 AAPL {$50} @ $55 + assets:cash $825 + +$ hledger -f- bal assets:stocks --lots +>2 /insufficient lots/ +>=1 + +# ** 6. Disposing when there's no lots at all gives a different error, apparently. ? +< +2026-02-01 sell without buying + assets:stocks -5 AAPL {$50} @ $55 + assets:cash $275 + +$ hledger -f- bal assets:stocks --lots +>2 /no lots available/ +>=1 + +# ** 7. Multiple disposals reduce lots correctly. +< +2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:checking + +2026-02-01 sell some + assets:stocks -3 AAPL {$50} @ $50 + assets:cash $150 + +2026-02-15 sell more + assets:stocks -4 AAPL {$50} @ $50 + assets:cash $200 + +2026-03-01 sell rest + assets:stocks -3 AAPL {$50} @ $50 + assets:cash $150 + +$ hledger -f- bal assets:stocks --lots -N + +# ** 8. A dispose posting with unknown transacted cost (selling price) is an error. +< +2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:checking + +2026-02-01 sell + assets:stocks -5 AAPL {$50} + assets:cash + +$ hledger -f- bal assets:stocks --lots +>2 /no transacted price/ +>=1 + +# ** 9. Explicit lot subaccount on dispose posting is accepted when it matches the resolved lot. +< +commodity AAPL ; lots: + +2026-01-15 buy + assets:stocks 10 AAPL {} @ $50 + assets:checking + +2026-02-01 sell + assets:stocks:{2026-01-15, $50} -5 AAPL {2026-01-15, $50} @ $60 + assets:cash $300 + revenue:gains -$50 + +$ hledger -f- print --lots +2026-01-15 buy + assets:stocks:{2026-01-15, $50} 10 AAPL {} @ $50 + assets:checking + +2026-02-01 sell + assets:stocks:{2026-01-15, $50} -5 AAPL {2026-01-15, $50} @ $60 + assets:cash $300 + revenue:gains $-50 + +>= + +# ** 10. Explicit lot subaccount that doesn't match the resolved lot is an error. +< +commodity AAPL ; lots: + +2026-01-15 buy + assets:stocks 10 AAPL {} @ $50 + assets:checking + +2026-02-01 sell + assets:stocks:{2026-02-01, $99} -5 AAPL {2026-01-15, $50} @ $60 + assets:cash $300 + revenue:gains -$50 + +$ hledger -f- print --lots +>2 /conflicting cost basis date/ +>=1 + +# ** 11. Implicit multi-lot disposal: bare negative amount with no cost basis, sale price inferred from cash. +< +commodity AAPL ; lots: + +2026-01-15 buy cheap + assets:stocks 10 AAPL {} @ $50 + assets:checking + +2026-02-01 buy expensive + assets:stocks 10 AAPL {} @ $55 + assets:checking + +2026-03-01 sell some + assets:stocks -15 AAPL + assets:cash $900 + revenue:gains + +$ hledger -f- print --lots -x desc:sell +2026-03-01 sell some + assets:stocks:{2026-01-15, $50} -10 AAPL {2026-01-15, $50} @ $60 + assets:stocks:{2026-02-01, $55} -5 AAPL {2026-02-01, $55} @ $60 + assets:cash $900 + revenue:gains $-125 + +>= + +# ** 12. Implicit multi-lot disposal: negative amount with explicit @ cost, cash and gains inferred. +< +commodity AAPL ; lots: + +2026-01-15 buy cheap + assets:stocks 10 AAPL {} @ $50 + assets:checking + +2026-02-01 buy expensive + assets:stocks 10 AAPL {} @ $55 + assets:checking + +2026-03-01 sell + assets:stocks -15 AAPL @ $60 + assets:cash + revenue:gains + +$ hledger -f- print --lots -x desc:sell +2026-03-01 sell + assets:stocks:{2026-01-15, $50} -10 AAPL {2026-01-15, $50} @ $60 + assets:stocks:{2026-02-01, $55} -5 AAPL {2026-02-01, $55} @ $60 + assets:cash $900 + revenue:gains $-125 + +>= + +# ** 11. Bare negative lotful posting without selling price gets a lot subaccount +# (dispose matches lot via FIFO, no gain posting since there's no selling price). +< +commodity AAPL ; lots: + +2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:cash + +2026-02-01 gift given + assets:stocks -5 AAPL + equity:opening + +$ hledger -f- bal assets:stocks --lots -N + 5 AAPL assets:stocks:{2026-01-15, $50} + +# ** 13. Balance assertion on a multi-lot disposal is moved to a generated parent-account +# posting with subaccount-inclusive checking (=*), so the assertion still passes on re-read. + +< +2026-01-15 buy cheap + assets:stocks 10 AAPL {$50} + assets:checking + +2026-02-01 buy expensive + assets:stocks 10 AAPL {$60} + assets:checking + +2026-03-01 sell all + assets:stocks -20 AAPL {} @ $70 = 0 AAPL + assets:cash $1400 + +$ hledger -f- print --lots -x desc:sell +2026-03-01 sell all + assets:stocks:{2026-01-15, $50} -10 AAPL {2026-01-15, $50} @ $70 + assets:stocks:{2026-02-01, $60} -10 AAPL {2026-02-01, $60} @ $70 + assets:stocks 0 AAPL =* 0 AAPL + assets:cash $1400 + revenue:gains $-300 + +>= + +# ** 14. A balance assertion on a dispose posting is moved to a generated parent-account +# posting with subaccount-inclusive checking (=*). +# Here after selling lot1, assets:stocks:{2026-01-15, $50} is 0 but assets:stocks:{2026-02-01, $60} is 10 AAPL. +# The assertion = 10 AAPL, converted to =*, checks the inclusive balance of assets:stocks = 10 AAPL. +< +2026-01-15 buy lot1 + assets:stocks 10 AAPL {$50} + assets:checking + +2026-02-01 buy lot2 + assets:stocks 10 AAPL {$60} + assets:checking + +2026-03-01 sell lot1 + assets:stocks -10 AAPL {} @ $70 = 10 AAPL + assets:cash $700 + +$ hledger -f- --lots check assertions + +# ** 15. But if a lot subaccount is written explicitly, a balance assertion checks that subaccount's balance. +< +2026-01-15 buy lot1 + assets:stocks:{2026-01-15, $50} 10 AAPL + assets:checking + +2026-02-01 buy lot2 + assets:stocks:{2026-02-01, $60} 10 AAPL + assets:checking + +2026-03-01 sell lot1 + assets:stocks:{2026-01-15, $50} -10 AAPL = 0 AAPL + assets:cash $700 + +$ hledger -f- check assertions + +# ** 16. Round-trip: print --lots -x output can be re-read and balance assertions still pass. +# The generated parent-account posting with =* preserves the assertion's semantics. +< +2026-01-15 buy cheap + assets:stocks 10 AAPL {$50} + assets:checking + +2026-02-01 buy expensive + assets:stocks 10 AAPL {$60} + assets:checking + +2026-03-01 sell all + assets:stocks -20 AAPL {} @ $70 = 0 AAPL + assets:cash $1400 + +$ hledger -f- print --lots -x | hledger -f- check assertions + +# ** 17. Dispose with explicit lot subaccount infers cost basis from account name. +< +commodity AAPL ; lots: + +2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:cash + +2026-02-01 sell with explicit lot subaccount + assets:stocks:{2026-01-15, $50} -5 AAPL @ $60 + assets:cash + revenue:gains + +$ hledger -f- print --lots -x desc:sell +2026-02-01 sell with explicit lot subaccount + assets:stocks:{2026-01-15, $50} -5 AAPL {2026-01-15, $50} @ $60 + assets:cash $300 + revenue:gains $-50 + +>= + +# ** 17. Insufficient lots error shows reduction method. +< +2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:checking + +2026-02-01 sell too many + assets:stocks -15 AAPL {$50} @ $55 + assets:cash $825 + +$ hledger -f- bal assets:stocks --lots +>2 /Using FIFO \(default\)/ +>=1 + +# ** 18. Insufficient lots error shows review hint. +< +2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:checking + +2026-02-01 sell too many + assets:stocks -15 AAPL {$50} @ $55 + assets:cash $825 + +$ hledger -f- bal assets:stocks --lots +>2 /hledger -f - reg assets:stocks cur:AAPL --lots -e 2026-02-02/ +>=1 + +# ** 19. No lots available error shows commodity tag as method source. +< +commodity AAPL ; lots: LIFO + +2026-02-01 sell without buying + assets:stocks -5 AAPL {$50} @ $55 + assets:cash $275 + +$ hledger -f- bal assets:stocks --lots +>2 /Using LIFO \(from commodity tag on AAPL\)/ +>=1 + +# ** 20. Error shows account tag as method source. +< +account assets:stocks ; lots: LIFO + +2026-02-01 sell without buying + assets:stocks -5 AAPL {$50} @ $55 + assets:cash $275 + +$ hledger -f- bal assets:stocks --lots +>2 /Using LIFO \(from account tag on assets:stocks\)/ +>=1 + +# ** 21. Lot error highlights the posting line, not the transaction line. +< +2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:checking + +2026-02-01 sell too many + assets:stocks -15 AAPL {$50} @ $55 + assets:cash $825 + +$ hledger -f- bal assets:stocks --lots +>2 /6 \| assets:stocks/ +>=1 + +# ** 22. Expense posting with cost basis does not get a lot subaccount. +< +commodity AAPL ; lots: +account revenue:gains ; type: G + +2026-01-15 buy + assets:stocks 10 AAPL {$80} + assets:cash + +2026-02-01 sell with stock fee + assets:stocks -5 AAPL {$80} @ $90 + expenses:fees 0.1 AAPL {$80} @ $90 + assets:cash $441 + revenue:gains -$49 + +$ hledger -f- bal --lots -N + $-359 assets:cash + 5.0 AAPL assets:stocks:{2026-01-15, $80} + 0.1 AAPL expenses:fees + $-49 revenue:gains diff --git a/hledger/test/journal/lots-methods.test b/hledger/test/journal/lots-methods.test new file mode 100644 index 00000000000..04ad5ddd379 --- /dev/null +++ b/hledger/test/journal/lots-methods.test @@ -0,0 +1,427 @@ +# Testing lot reduction methods: FIFO (default), LIFO, HIFO, AVERAGE, SPECID. +# Where possible, disposals use a selling price different from cost basis +# to exercise gain/loss posting exclusion from transaction balancing. + +# * Reduction methods + +# ** 1. LIFO disposal via commodity tag: newest lot consumed first. +< +commodity AAPL ; lots: LIFO + +2026-01-15 buy lot 1 + assets:stocks 10 AAPL {$50} + assets:checking + +2026-02-01 buy lot 2 + assets:stocks 10 AAPL {$50} + assets:checking + +2026-03-01 sell some + assets:stocks -15 AAPL {} @ $55 + assets:cash $825 + revenue:gains $-75 + +$ hledger -f- bal assets:stocks --lots -N + 5 AAPL assets:stocks:{2026-01-15, $50} + +# ** 2. LIFO disposal via account tag: newest lot consumed first. +< +account assets:stocks ; lots: LIFO + +2026-01-15 buy lot 1 + assets:stocks 10 AAPL {$50} + assets:checking + +2026-02-01 buy lot 2 + assets:stocks 10 AAPL {$50} + assets:checking + +2026-03-01 sell some + assets:stocks -10 AAPL {} @ $50 + assets:cash $500 + +$ hledger -f- bal assets:stocks --lots -N + 10 AAPL assets:stocks:{2026-01-15, $50} + +# ** 3. Account tag overrides commodity tag. +< +commodity AAPL ; lots: FIFO +account assets:stocks ; lots: LIFO + +2026-01-15 buy lot 1 + assets:stocks 10 AAPL {$50} + assets:checking + +2026-02-01 buy lot 2 + assets:stocks 10 AAPL {$50} + assets:checking + +2026-03-01 sell some + assets:stocks -10 AAPL {} @ $50 + assets:cash $500 + +$ hledger -f- bal assets:stocks --lots -N + 10 AAPL assets:stocks:{2026-01-15, $50} + +# ** 4. Bare lots: tag (no value) defaults to FIFO: oldest lot consumed first. +< +commodity AAPL ; lots: + +2026-01-15 buy lot 1 + assets:stocks 10 AAPL {$50} + assets:checking + +2026-02-01 buy lot 2 + assets:stocks 10 AAPL {$50} + assets:checking + +2026-03-01 sell some + assets:stocks -15 AAPL {} @ $55 + assets:cash $825 + revenue:gains $-75 + +$ hledger -f- bal assets:stocks --lots -N + 5 AAPL assets:stocks:{2026-02-01, $50} + +# ** 5. FIFO disposal with multiple accounts: per-account scope, oldest first. +# Lots in broker2 are untouched even though they exist. +< +commodity AAPL ; lots: FIFO + +2026-01-15 buy in broker1 + assets:broker1 10 AAPL {$50} + assets:checking + +2026-02-01 buy in broker2 + assets:broker2 5 AAPL {$50} + assets:checking + +2026-03-01 sell from broker1 + assets:broker1 -10 AAPL {} @ $50 + assets:cash $500 + +$ hledger -f- bal assets:broker --lots -N + 5 AAPL assets:broker2:{2026-02-01, $50} + +# ** 6. FIFO with insufficient lots in one account: fails +# even if another account has enough lots (per-account scope). +< +commodity AAPL ; lots: FIFO + +2026-01-15 buy in broker1 + assets:broker1 5 AAPL {$50} + assets:checking + +2026-02-01 buy in broker2 + assets:broker2 10 AAPL {$50} + assets:checking + +2026-03-01 sell too many from broker1 + assets:broker1 -8 AAPL {} @ $50 + assets:cash $400 + +$ hledger -f- bal assets:broker --lots +>2 /insufficient lots/ +>=1 + +# ** 7. LIFO disposal with multiple accounts: per-account scope, newest first. +# broker2's 2026-03-01 lot is newer globally but LIFO is scoped to broker1. +< +commodity AAPL ; lots: LIFO + +2026-01-15 buy in broker1 lot 1 + assets:broker1 10 AAPL {$50} + assets:checking + +2026-02-01 buy in broker1 lot 2 + assets:broker1 10 AAPL {$50} + assets:checking + +2026-03-01 buy in broker2 + assets:broker2 10 AAPL {$50} + assets:checking + +2026-04-01 sell from broker1 + assets:broker1 -10 AAPL {} @ $50 + assets:cash $500 + +$ hledger -f- bal assets:broker --lots -N + 10 AAPL assets:broker1:{2026-01-15, $50} + 10 AAPL assets:broker2:{2026-03-01, $50} + +# ** 8. LIFO transfer: lots are transferred newest first. +< +commodity AAPL ; lots: LIFO + +2026-01-15 buy lot 1 + assets:broker1 10 AAPL {$50} + assets:checking + +2026-02-01 buy lot 2 + assets:broker1 5 AAPL {$50} + assets:checking + +2026-03-01 transfer some + assets:broker1 -8 AAPL {} + assets:broker2 8 AAPL {} + +$ hledger -f- bal assets:broker --lots -N + 7 AAPL assets:broker1:{2026-01-15, $50} + 3 AAPL assets:broker2:{2026-01-15, $50} + 5 AAPL assets:broker2:{2026-02-01, $50} + +# ** 9. FIFO multi-lot disposal within one account: oldest lots consumed first. +< +account assets:broker ; lots: FIFO + +2026-01-15 buy lot 1 + assets:broker 10 AAPL {$50} + assets:checking + +2026-02-01 buy lot 2 + assets:broker 5 AAPL {$50} + assets:checking + +2026-03-01 sell spanning lots + assets:broker -12 AAPL {} @ $50 + assets:cash $600 + +$ hledger -f- bal assets:broker --lots -N + 3 AAPL assets:broker:{2026-02-01, $50} + +# ** 10. HIFO disposal: highest cost lot consumed first. +< +commodity AAPL ; lots: HIFO + +2026-01-15 buy lot 1 (cheap) + assets:stocks 10 AAPL {$40} + assets:checking + +2026-02-01 buy lot 2 (expensive) + assets:stocks 10 AAPL {$60} + assets:checking + +2026-03-01 buy lot 3 (medium) + assets:stocks 10 AAPL {$50} + assets:checking + +2026-04-01 sell some + assets:stocks -15 AAPL {} @ $65 + assets:cash $975 + revenue:gains $-125 + +$ hledger -f- bal assets:stocks --lots -N + 10 AAPL assets:stocks:{2026-01-15, $40} + 5 AAPL assets:stocks:{2026-03-01, $50} + +# ** 11. HIFO disposal with multiple accounts: per-account scope, highest cost first. +# broker2's expensive lot is untouched because HIFO is scoped to broker1. +< +commodity AAPL ; lots: HIFO + +2026-01-15 buy in broker1 (cheap) + assets:broker1 10 AAPL {$40} + assets:checking + +2026-02-01 buy in broker1 (expensive) + assets:broker1 10 AAPL {$60} + assets:checking + +2026-03-01 buy in broker2 (very expensive) + assets:broker2 10 AAPL {$80} + assets:checking + +2026-04-01 sell from broker1 + assets:broker1 -10 AAPL {} @ $65 + assets:cash $650 + revenue:gains $-50 + +$ hledger -f- bal assets:broker --lots -N + 10 AAPL assets:broker1:{2026-01-15, $40} + 10 AAPL assets:broker2:{2026-03-01, $80} + +# ** 12. AVERAGE disposal: weighted average cost basis. +# Two lots at different costs; disposal uses the weighted average cost ($50). +# Lot subaccounts retain original acquisition costs in their names. +# The gain posting ($-25) confirms average cost is used: 5 * ($55 - $50) = $25. +# With FIFO (non-average), the gain would be 5 * ($55 - $40) = $75. +< +commodity AAPL ; lots: AVERAGE + +2026-01-15 buy lot 1 + assets:stocks 10 AAPL {$40} + assets:checking + +2026-02-01 buy lot 2 + assets:stocks 10 AAPL {$60} + assets:checking + +2026-03-01 sell some (average cost is $50) + assets:stocks -5 AAPL {} @ $55 + assets:cash $275 + revenue:gains $-25 + +$ hledger -f- bal assets:stocks --lots -N + 5 AAPL assets:stocks:{2026-01-15, $40} + 10 AAPL assets:stocks:{2026-02-01, $60} + +# ** 13. AVERAGE disposal with multiple accounts: per-account weighted average cost basis. +# Each account has its own average. broker1 avg = (10*40 + 10*60)/20 = $50. +# broker2's $80 lots are excluded from the average since AVERAGE scopes per-account. +< +commodity AAPL ; lots: AVERAGE + +2026-01-15 buy in broker1 (cheap) + assets:broker1 10 AAPL {$40} + assets:checking + +2026-02-01 buy in broker1 (expensive) + assets:broker1 10 AAPL {$60} + assets:checking + +2026-03-01 buy in broker2 + assets:broker2 10 AAPL {$80} + assets:checking + +2026-04-01 sell from broker1 (avg cost $50) + assets:broker1 -5 AAPL {} @ $55 + assets:cash $275 + revenue:gains $-25 + +$ hledger -f- bal assets:broker --lots -N + 5 AAPL assets:broker1:{2026-01-15, $40} + 10 AAPL assets:broker1:{2026-02-01, $60} + 10 AAPL assets:broker2:{2026-03-01, $80} + +# ** 14. AVERAGE with partial disposal: average is of the entire pool. +# Pool avg = (10*30 + 20*60)/30 = $50. Only 3 units consumed from first lot. +# Gain = 3 * ($55 - $50) = $15. +< +commodity AAPL ; lots: AVERAGE + +2026-01-15 buy lot 1 + assets:stocks 10 AAPL {$30} + assets:checking + +2026-02-01 buy lot 2 + assets:stocks 20 AAPL {$60} + assets:checking + +2026-03-01 sell a few (pool avg = (10*30 + 20*60)/30 = $50) + assets:stocks -3 AAPL {} @ $55 + assets:cash $165 + revenue:gains $-15 + +$ hledger -f- bal assets:stocks --lots -N + 7 AAPL assets:stocks:{2026-01-15, $30} + 20 AAPL assets:stocks:{2026-02-01, $60} + +# * Global reduction methods (FIFOALL, LIFOALL, HIFOALL, AVERAGEALL) + +# ** 15. FIFOALL: dispose from account with globally-oldest lot passes. +< +commodity AAPL ; lots: FIFOALL + +2026-01-15 buy in broker1 (oldest globally) + assets:broker1 10 AAPL {$50} + assets:checking + +2026-02-01 buy in broker2 + assets:broker2 5 AAPL {$50} + assets:checking + +2026-03-01 sell from broker1 (has globally oldest lot, OK) + assets:broker1 -5 AAPL {} @ $55 + assets:cash $275 + revenue:gains $-25 + +$ hledger -f- bal assets:broker --lots -N + 5 AAPL assets:broker1:{2026-01-15, $50} + 5 AAPL assets:broker2:{2026-02-01, $50} + +# ** 16. FIFOALL: dispose from account but another account has an older lot: error. +< +commodity AAPL ; lots: FIFOALL + +2026-01-15 buy in broker1 (oldest globally) + assets:broker1 10 AAPL {$50} + assets:checking + +2026-02-01 buy in broker2 + assets:broker2 5 AAPL {$50} + assets:checking + +2026-03-01 sell from broker2 (but broker1 has older lot) + assets:broker2 -5 AAPL {} @ $55 + assets:cash $275 + revenue:gains $-25 + +$ hledger -f- bal assets:broker --lots -N +>2 /FIFOALL.*higher priority/ +>=1 + +# ** 17. LIFOALL: dispose from account but another account has a newer lot: error. +< +commodity AAPL ; lots: LIFOALL + +2026-01-15 buy in broker1 + assets:broker1 10 AAPL {$50} + assets:checking + +2026-02-01 buy in broker2 + assets:broker2 5 AAPL {$50} + assets:checking + +2026-03-01 sell from broker1 (but broker2 has newer lot under LIFO) + assets:broker1 -5 AAPL {} @ $55 + assets:cash $275 + revenue:gains $-25 + +$ hledger -f- bal assets:broker --lots -N +>2 /LIFOALL.*higher priority/ +>=1 + +# ** 18. HIFOALL: dispose from account but another account has a higher-cost lot: error. +< +commodity AAPL ; lots: HIFOALL + +2026-01-15 buy in broker1 (cheap) + assets:broker1 10 AAPL {$40} + assets:checking + +2026-02-01 buy in broker2 (expensive) + assets:broker2 5 AAPL {$60} + assets:checking + +2026-03-01 sell from broker1 (but broker2 has higher-cost lot under HIFO) + assets:broker1 -5 AAPL {} @ $55 + assets:cash $275 + revenue:gains $-75 + +$ hledger -f- bal assets:broker --lots -N +>2 /HIFOALL.*higher priority/ +>=1 + +# ** 19. AVERAGEALL: dispose uses global weighted average cost, not per-account. +# broker1 has 10 @ $40, broker2 has 10 @ $80. Global avg = (10*40 + 10*80)/20 = $60. +# Per-account avg for broker1 would be $40. +# Gain = 5 * ($65 - $60) = $25. (With per-account AVERAGE, gain would be 5 * ($65 - $40) = $125.) +< +commodity AAPL ; lots: AVERAGEALL + +2026-01-15 buy in broker1 (cheap) + assets:broker1 10 AAPL {$40} + assets:checking + +2026-02-01 buy in broker2 (expensive) + assets:broker2 10 AAPL {$80} + assets:checking + +2026-03-01 sell from broker1 (global avg cost $60) + assets:broker1 -5 AAPL {} @ $65 + assets:cash $325 + revenue:gains $-25 + +$ hledger -f- bal assets:broker --lots -N + 5 AAPL assets:broker1:{2026-01-15, $40} + 10 AAPL assets:broker2:{2026-02-01, $80} diff --git a/hledger/test/journal/lots-transfer-entries.test b/hledger/test/journal/lots-transfer-entries.test new file mode 100644 index 00000000000..1ea28a322d0 --- /dev/null +++ b/hledger/test/journal/lots-transfer-entries.test @@ -0,0 +1,276 @@ +# Testing lot transfer journal entry variations, systematically. + +# * Transfer + +# ** With commodity/account not declared lotful + +# *** 1. transfer: from {$50}, to costless basisless amount. Lot transfer detected. +< +2026-01-01 buy + assets:broker1 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 transfer + assets:broker1 -10 AAPL {$50} + assets:broker2 10 AAPL + +$ hledger -f- print desc:transfer --lots --verbose-tags +2026-02-01 transfer + assets:broker1:{2026-01-01, $50} -10 AAPL {$50} ; ptype: transfer-from + assets:broker2:{2026-01-01, $50} 10 AAPL ; ptype: transfer-to + +>= + +# *** 2. transfer: from {$50}, to {$50}, non-lotful commodity/account, non-asset. Not an asset, but we'll allow it here. +< +2026-01-01 buy + assets:broker 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 transfer + assets:broker -10 AAPL {$50} + equity:opening 10 AAPL {$50} + +$ hledger -f- print desc:transfer --lots --verbose-tags +2026-02-01 transfer + assets:broker:{2026-01-01, $50} -10 AAPL {$50} ; ptype: transfer-from + equity:opening:{2026-01-01, $50} 10 AAPL {$50} ; ptype: transfer-to + +>= + +# *** 3. transfer: from {$50}, to {$50}, non-lotful commodity/account, asset. +< +2026-01-01 buy + assets:broker1 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 transfer + assets:broker1 -10 AAPL {$50} + assets:broker2 10 AAPL {$50} + +$ hledger -f- print desc:transfer --lots --verbose-tags +2026-02-01 transfer + assets:broker1:{2026-01-01, $50} -10 AAPL {$50} ; ptype: transfer-from + assets:broker2:{2026-01-01, $50} 10 AAPL {$50} ; ptype: transfer-to + +>= + +# ** With lotful commodity/account + +# *** 4. transfer: from {$50}, to costless basisless amount. Lot transfer detected. +< +commodity AAPL ; lots: + +2026-01-01 buy + assets:broker1 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 transfer + assets:broker1 -10 AAPL {$50} + assets:broker2 10 AAPL + +$ hledger -f- print desc:transfer --lots --verbose-tags +2026-02-01 transfer + assets:broker1:{2026-01-01, $50} -10 AAPL {$50} ; ptype: transfer-from + assets:broker2:{2026-01-01, $50} 10 AAPL ; ptype: transfer-to + +>= + +# *** 5. transfer: from {$50}, to {}. Lot transfer detected. +< +commodity AAPL ; lots: + +2026-01-01 buy + assets:broker1 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 transfer + assets:broker1 -10 AAPL {$50} + assets:broker2 10 AAPL {} + +$ hledger -f- print desc:transfer --lots --verbose-tags +2026-02-01 transfer + assets:broker1:{2026-01-01, $50} -10 AAPL {$50} ; ptype: transfer-from + assets:broker2:{2026-01-01, $50} 10 AAPL {} ; ptype: transfer-to + +>= + +# *** 6. transfer: from {}, to {}. Lot transfer detected. +< +commodity AAPL ; lots: + +2026-01-01 buy + assets:broker1 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 transfer + assets:broker1 -10 AAPL {} + assets:broker2 10 AAPL {} + +$ hledger -f- print desc:transfer --lots --verbose-tags +2026-02-01 transfer + assets:broker1:{2026-01-01, $50} -10 AAPL {} ; ptype: transfer-from + assets:broker2:{2026-01-01, $50} 10 AAPL {} ; ptype: transfer-to + +>= + +# *** 7. transfer: from {$50}, to {$50}. Lot transfer detected. +< +commodity AAPL ; lots: + +2026-01-01 buy + assets:broker1 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 transfer + assets:broker1 -10 AAPL {$50} + assets:broker2 10 AAPL {$50} + +$ hledger -f- print desc:transfer --lots --verbose-tags +2026-02-01 transfer + assets:broker1:{2026-01-01, $50} -10 AAPL {$50} ; ptype: transfer-from + assets:broker2:{2026-01-01, $50} 10 AAPL {$50} ; ptype: transfer-to + +>= + +# *** 8. transfer: from {$50} @ $50, to {$50}. Error: unbalanced transaction. +< +commodity AAPL ; lots: + +2026-01-01 buy + assets:broker1 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 transfer + assets:broker1 -10 AAPL {$50} @ $50 + assets:broker2 10 AAPL {$50} + +$ hledger -f- print desc:transfer --lots --verbose-tags +>2 /transaction is unbalanced/ +>=1 + +# *** 9. transfer: from {$50} @ $50, to {$50} @ $50. Error: lot transfers should have no transacted price. +< +commodity AAPL ; lots: + +2026-01-01 buy + assets:broker1 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 transfer + assets:broker1 -10 AAPL {$50} @ $50 + assets:broker2 10 AAPL {$50} @ $50 + +$ hledger -f- print desc:transfer --lots --verbose-tags +>2 /lot transfers should have no transacted price/ +>=1 + +# *** 10. transfer: from {$50}, to {$60} (mismatched). Error: does not match. +< +commodity AAPL ; lots: + +2026-01-01 buy + assets:broker1 10 AAPL {$50} + assets:cash -$500 + +2026-02-01 transfer + assets:broker1 -10 AAPL {$50} + assets:broker2 10 AAPL {$60} + +$ hledger -f- print desc:transfer --lots --verbose-tags +>2 /does not match transfer-to cost basis/ +>=1 + +# ** Bare acquire (@ price) followed by transfer with varying {} placement + +# *** 11. transfer: bare acquire via @ price, no {} on either transfer posting. +< +commodity AAA 0.00 ; lots: + +2000-01-01 buy + assets:cash -100 USD + assets:x1 1 AAA @ 100 USD + +2000-01-01 transfer + assets:x1 -0.02 AAA + assets:x2 0.02 AAA + +$ hledger -f- print --verbose-tags --lots +2000-01-01 buy + assets:cash -100 USD + assets:x1:{2000-01-01, 100 USD} AAA 1 @ 100 USD ; ptype: acquire + +2000-01-01 transfer + assets:x1:{2000-01-01, 100 USD} AAA -0.02 ; ptype: transfer-from + assets:x2:{2000-01-01, 100 USD} AAA 0.02 ; ptype: transfer-to + +>=0 + +# *** 12. transfer: bare acquire via @ price, {} on from posting only. +< +commodity AAA 0.00 ; lots: + +2000-01-01 buy + assets:cash -100 USD + assets:x1 1 AAA @ 100 USD + +2000-01-01 transfer + assets:x1 -0.02 AAA {} + assets:x2 0.02 AAA + +$ hledger -f- print --verbose-tags --lots +2000-01-01 buy + assets:cash -100 USD + assets:x1:{2000-01-01, 100 USD} AAA 1 @ 100 USD ; ptype: acquire + +2000-01-01 transfer + assets:x1:{2000-01-01, 100 USD} AAA -0.02 {} ; ptype: transfer-from + assets:x2:{2000-01-01, 100 USD} AAA 0.02 ; ptype: transfer-to + +>=0 + +# *** 13. transfer: bare acquire via @ price, {} on to posting only. +< +commodity AAA 0.00 ; lots: + +2000-01-01 buy + assets:cash -100 USD + assets:x1 1 AAA @ 100 USD + +2000-01-01 transfer + assets:x1 -0.02 AAA + assets:x2 0.02 AAA {} + +$ hledger -f- print --verbose-tags --lots +2000-01-01 buy + assets:cash -100 USD + assets:x1:{2000-01-01, 100 USD} AAA 1 @ 100 USD ; ptype: acquire + +2000-01-01 transfer + assets:x1:{2000-01-01, 100 USD} AAA -0.02 ; ptype: transfer-from + assets:x2:{2000-01-01, 100 USD} AAA 0.02 {} ; ptype: transfer-to + +>=0 + +# *** 14. transfer: bare acquire via @ price, {} on both transfer postings. +< +commodity AAA 0.00 ; lots: + +2000-01-01 buy + assets:cash -100 USD + assets:x1 1 AAA @ 100 USD + +2000-01-01 transfer + assets:x1 -0.02 AAA {} + assets:x2 0.02 AAA {} + +$ hledger -f- print --verbose-tags --lots +2000-01-01 buy + assets:cash -100 USD + assets:x1:{2000-01-01, 100 USD} AAA 1 @ 100 USD ; ptype: acquire + +2000-01-01 transfer + assets:x1:{2000-01-01, 100 USD} AAA -0.02 {} ; ptype: transfer-from + assets:x2:{2000-01-01, 100 USD} AAA 0.02 {} ; ptype: transfer-to + +>=0 diff --git a/hledger/test/journal/lots-transfer.test b/hledger/test/journal/lots-transfer.test new file mode 100644 index 00000000000..4fe2a1b0fac --- /dev/null +++ b/hledger/test/journal/lots-transfer.test @@ -0,0 +1,448 @@ +# Testing lot transfer scenarios. + +# * Lot transfers + +# ** 1. Simple transfer - gets ptype:transfer-from and ptype:transfer-to tags +< +2026-01-03 Transfer + assets:broker1 -10 AAPL {$100} + assets:broker2 10 AAPL {$100} + +$ hledger -f- print --verbose-tags +2026-01-03 Transfer + assets:broker1 -10 AAPL {$100} ; ptype: transfer-from + assets:broker2 10 AAPL {$100} ; ptype: transfer-to + +>= + +# ** 2. Simple single-lot transfer between two broker accounts. +< +2026-01-15 buy + assets:broker1 10 AAPL {$50} + assets:checking + +2026-02-01 transfer + assets:broker1 -10 AAPL {$50} + assets:broker2 10 AAPL {$50} + +$ hledger -f- bal assets:broker --lots -N + 10 AAPL assets:broker2:{2026-01-15, $50} + +# ** 3. Wildcard multi-lot transfer with {}. +< +commodity AAPL ; lots: +account assets:broker1 ; lots: +account assets:broker2 ; lots: + +2026-01-15 buy cheap + assets:broker1 10 AAPL {$50} + assets:checking + +2026-02-01 buy expensive + assets:broker1 5 AAPL {$60} + assets:checking + +2026-03-01 transfer all + assets:broker1 -15 AAPL {} + assets:broker2 15 AAPL {} + +$ hledger -f- bal assets:broker --lots -N + 10 AAPL assets:broker2:{2026-01-15, $50} + 5 AAPL assets:broker2:{2026-02-01, $60} + +# ** 4. Transfer-to posting without {} is classified when commodity has lots: tag. +< +commodity AAPL ; lots: + +2026-01-15 buy + assets:broker1 10 AAPL {$50} + assets:checking + +2026-02-01 transfer + assets:broker1 -10 AAPL {$50} + assets:broker2 10 AAPL + +$ hledger -f- print --verbose-tags +2026-01-15 buy + assets:broker1 10 AAPL {$50} ; ptype: acquire + assets:checking + +2026-02-01 transfer + assets:broker1 -10 AAPL {$50} ; ptype: transfer-from + assets:broker2 10 AAPL ; ptype: transfer-to + +>= + +# ** 5. Transfer-to posting without {} is classified when account has lots: tag. +< +account assets:broker2 ; lots: + +2026-01-15 buy + assets:broker1 10 AAPL {$50} + assets:checking + +2026-02-01 transfer + assets:broker1 -10 AAPL {$50} + assets:broker2 10 AAPL + +$ hledger -f- print --verbose-tags +2026-01-15 buy + assets:broker1 10 AAPL {$50} ; ptype: acquire + assets:checking + +2026-02-01 transfer + assets:broker1 -10 AAPL {$50} ; ptype: transfer-from + assets:broker2 10 AAPL ; ptype: transfer-to + +>= + +# ** 6. Transfer then dispose from new account. +< +2026-01-15 buy + assets:broker1 10 AAPL {$50} + assets:checking + +2026-02-01 transfer + assets:broker1 -10 AAPL {$50} + assets:broker2 10 AAPL {$50} + +2026-03-01 sell from broker2 + assets:broker2 -5 AAPL {$50} @ $70 + assets:cash $350 + revenue:gains -$100 + +$ hledger -f- bal assets:broker --lots -N + 5 AAPL assets:broker2:{2026-01-15, $50} + +# ** 7. Transfer with matching cost basis on transfer-to (validation passes). +< +2026-01-15 buy + assets:broker1 10 AAPL {$50} + assets:checking + +2026-02-01 transfer + assets:broker1 -10 AAPL {$50} + assets:broker2 10 AAPL {$50} + +$ hledger -f- bal assets:broker --lots -N + 10 AAPL assets:broker2:{2026-01-15, $50} + +# ** 8. Transfer with mismatched cost basis on transfer-to is an error. +< +2026-01-15 buy + assets:broker1 10 AAPL {$50} + assets:checking + +2026-02-01 transfer + assets:broker1 -10 AAPL {$50} + assets:broker2 10 AAPL {$60} + +$ hledger -f- bal assets:broker --lots +>2 /does not match transfer-to cost basis/ +>=1 + +# ** 9. Transfer without prior acquisition is an error. +< +2026-02-01 transfer + assets:broker1 -10 AAPL {$50} + assets:broker2 10 AAPL {$50} + +$ hledger -f- bal assets:broker --lots +>2 /no lots available/ +>=1 + +# ** 10. Split transfers (1 from at qty 10, 2 to at qty 5) are not detected as transfers +# because transfer detection requires exact quantity matching. The postings are classified +# as dispose + acquire + acquire, and the transaction is unbalanced. +< +2026-01-15 buy + assets:broker1 10 AAPL {$50} + assets:checking + +2026-02-01 split transfer + assets:broker1 -10 AAPL {$50} + assets:broker2 5 AAPL {$50} + assets:broker3 5 AAPL {$50} + +$ hledger -f- bal assets:broker --lots +>2 /unbalanced/ +>=1 + +# ** 11. Cross-account transfer with fee: only matching-quantity postings are transfers. +< +commodity A ; lots: + +2026-01-15 buy + assets:a 1 A {$100} + assets:cash + +2026-02-17 transfer with fee + assets:a -1 A {$100} + assets:b 1 A {$100} + assets:a -0.02 A {$100} + expenses:fees $2 + +$ hledger -f- print --verbose-tags date:2026-02-17 +2026-02-17 transfer with fee + assets:a -1 A {$100} ; ptype: transfer-from + assets:b 1 A {$100} ; ptype: transfer-to + assets:a -0.02 A {$100} ; ptype: dispose + expenses:fees $2 + +>= + +# ** 12. Same-account transfer with fee: quantity matching pairs the transfer postings. +< +commodity A ; lots: + +2026-01-15 buy + assets:cc 1 A {$100} + assets:cash + +2026-02-17 reclassify with fee + assets:cc -1 A + assets:cc 1 A + assets:cc -0.02 A + expenses:fees $2 + +$ hledger -f- print --verbose-tags date:2026-02-17 +2026-02-17 reclassify with fee + assets:cc -1 A ; ptype: transfer-from + assets:cc 1 A ; ptype: transfer-to + assets:cc -0.02 A ; ptype: dispose + expenses:fees $2 + +>= + +# ** 13. Balance assertion on a multi-lot transfer is moved to a generated parent-account +# posting with subaccount-inclusive checking (=*), so the assertion still passes on re-read. +# Also, -x shows there are no hidden conversion prices inferred +# (the different AAPL cost bases don't affect the transaction balancer). +< +2026-01-15 buy lot1 + assets:broker1 10 AAPL {$50} + assets:checking + +2026-02-01 buy lot2 + assets:broker1 10 AAPL {$60} + assets:checking + +2026-03-01 transfer both lots + assets:broker1 -20 AAPL {} = 0 AAPL + assets:broker2 20 AAPL = 20 AAPL + +$ hledger -f- print --lots -x desc:transfer +2026-03-01 transfer both lots + assets:broker1:{2026-01-15, $50} -10 AAPL {2026-01-15, $50} + assets:broker1:{2026-02-01, $60} -10 AAPL {2026-02-01, $60} + assets:broker1 0 AAPL =* 0 AAPL + assets:broker2:{2026-01-15, $50} 10 AAPL {2026-01-15, $50} + assets:broker2:{2026-02-01, $60} 10 AAPL {2026-02-01, $60} + assets:broker2 0 AAPL =* 20 AAPL + +>= + +# ** 14. Transfer with explicit lot subaccounts - no double-append. +< +commodity AAPL ; lots: + +2026-01-15 buy + assets:broker1 10 AAPL {$50} + assets:cash + +2026-02-01 transfer with explicit lot subaccounts + assets:broker1:{2026-01-15, $50} -10 AAPL {2026-01-15, $50} + assets:broker2:{2026-01-15, $50} 10 AAPL {2026-01-15, $50} + +$ hledger -f- print --lots -x desc:transfer +2026-02-01 transfer with explicit lot subaccounts + assets:broker1:{2026-01-15, $50} -10 AAPL {2026-01-15, $50} + assets:broker2:{2026-01-15, $50} 10 AAPL {2026-01-15, $50} + +>= + +# ** 15. Bare transfer (no cost basis notation) splitting a quantity into lots +# with more decimal places. Regression test: the lot-split amounts must not +# appear as zero in balance reports due to insufficient display precision. +< +commodity 0.00000000 BTC ; lots: + +2026-01-01 buy + assets:exchange1 0.01 BTC @ $1200 + assets:cash + +2026-01-02 buy + assets:exchange1 0.01 BTC @ $1200 + assets:cash + +2026-01-03 buy + assets:exchange1 0.01 BTC @ $1200 + assets:cash + +2026-01-04 buy + assets:exchange1 0.01 BTC @ $1200 + assets:cash + +2026-01-05 buy + assets:exchange1 0.01 BTC @ $1200 + assets:cash + +2026-01-06 buy + assets:exchange1 0.01 BTC @ $1200 + assets:cash + +2026-01-07 buy + assets:exchange1 0.01 BTC @ $1200 + assets:cash + +2026-01-08 buy + assets:exchange1 0.01 BTC @ $1200 + assets:cash + +2026-01-09 buy + assets:exchange1 0.01 BTC @ $1200 + assets:cash + +2026-01-10 buy + assets:exchange1 0.01 BTC @ $1200 + assets:cash + +2026-02-01 transfer + assets:exchange1 -0.1 BTC + assets:exchange2 0.1 BTC + +$ hledger -f- bal cur:BTC acct:exchange2 --lots -N + 0.01000000 BTC assets:exchange2:{2026-01-01, $1200} + 0.01000000 BTC assets:exchange2:{2026-01-02, $1200} + 0.01000000 BTC assets:exchange2:{2026-01-03, $1200} + 0.01000000 BTC assets:exchange2:{2026-01-04, $1200} + 0.01000000 BTC assets:exchange2:{2026-01-05, $1200} + 0.01000000 BTC assets:exchange2:{2026-01-06, $1200} + 0.01000000 BTC assets:exchange2:{2026-01-07, $1200} + 0.01000000 BTC assets:exchange2:{2026-01-08, $1200} + 0.01000000 BTC assets:exchange2:{2026-01-09, $1200} + 0.01000000 BTC assets:exchange2:{2026-01-10, $1200} + +# ** 16. Bare disposal to expense (no selling price) gets lot subaccount, no gain posting. +< +commodity A ; lots: + +2026-01-15 buy + assets:x1 1 A {$100} + assets:cash + +2026-02-17 pay fee + assets:x1 -0.001 A + expenses:fees 0.001 A + +$ hledger -f- print --lots -x desc:fee +2026-02-17 pay fee + assets:x1:{2026-01-15, $100} -0.001 A {2026-01-15, $100} + expenses:fees 0.001 A + +>= + +# ** 17. Bare disposal to expense: remaining lot balance is reduced correctly. +< +commodity A ; lots: + +2026-01-15 buy + assets:x1 1 A {$100} + assets:cash + +2026-02-17 pay fee + assets:x1 -0.001 A + expenses:fees 0.001 A + +$ hledger -f- bal assets:x1 --lots -N + 0.999 A assets:x1:{2026-01-15, $100} + +# ** 18. Transfer+fee pattern: source qty > dest qty, fee deducted. +# The transfer is detected despite inexact quantity matching. +# Fee portion lots are consumed from source but not added to destination. +< +commodity A ; lots: + +2026-01-15 buy + assets:x1 1.05 A {$100} + assets:cash + +2026-02-17 transfer with fee + assets:x1 -1.05 A + expenses:fees 0.05 A + assets:x2 1.00 A + +$ hledger -f- print --lots -x desc:transfer +2026-02-17 transfer with fee + assets:x1:{2026-01-15, $100} -1.00 A {2026-01-15, $100} + assets:x1:{2026-01-15, $100} -0.05 A {2026-01-15, $100} + expenses:fees 0.05 A + assets:x2:{2026-01-15, $100} 1.00 A {2026-01-15, $100} + +>= + +# ** 19. Transfer+fee: destination gets only the transferred lots, not fee lots. +< +commodity A ; lots: + +2026-01-15 buy + assets:x1 1.05 A {$100} + assets:cash + +2026-02-17 transfer with fee + assets:x1 -1.05 A + expenses:fees 0.05 A + assets:x2 1.00 A + +$ hledger -f- bal assets:x2 --lots -N + 1.00 A assets:x2:{2026-01-15, $100} +>= + +# ** 20. Transfer+fee: subsequent disposal from destination works. +< +commodity A ; lots: + +2026-01-15 buy + assets:x1 2 A {$100} + assets:cash + +2026-02-17 transfer with fee + assets:x1 -1.05 A + expenses:fees 0.05 A + assets:x2 1.00 A + +2026-03-01 sell + assets:x2 -1.00 A @ $150 + assets:cash + revenue:gains + +$ hledger -f- bal assets:x1 assets:x2 --lots -N + 0.95 A assets:x1:{2026-01-15, $100} + +# ** 21. Two transfers moving portions of the same lot to the same destination account. +# The lot quantities should be summed, not overwritten. +< +commodity A ; lots: + +2026-01-15 buy + assets:x1 3 A {$100} + assets:cash + +2026-02-01 transfer batch 1 + assets:x1 -1.01 A + expenses:fees 0.01 A + assets:x2 1.00 A + +2026-02-01 transfer batch 2 + assets:x1 -1.01 A + expenses:fees 0.01 A + assets:x2 1.00 A + +2026-03-01 sell all from x2 + assets:x2 -2 A @ $150 + assets:cash + revenue:gains + +$ hledger -f- bal assets --lots -N + 0.98 A assets:x1:{2026-01-15, $100} + diff --git a/hledger/test/journal/lots-warn.test b/hledger/test/journal/lots-warn.test new file mode 100644 index 00000000000..459f46968aa --- /dev/null +++ b/hledger/test/journal/lots-warn.test @@ -0,0 +1,89 @@ +# Tests for lot error behavior (formerly testing --lots-warn mode, which has been removed). + +# ** 1. no lots available with --lots errors (exits 1) +< +2026-02-01 sell + assets:stocks -5 AAPL {$50} @ $55 + assets:cash $275 + revenues:gains + +$ hledger -f- print --lots 2>/dev/null +>=1 + +# ** 2. Unclassified lotful commodity posting in asset account is a hard error with --lots. +# AAPL has lots: tag, assets:broker is an asset account, but the posting has no cost basis +# and no price, so classification fails. With --lots this is an error. +< +commodity AAPL ; lots: + +2026-01-15 mysterious + assets:broker 10 AAPL + income:mystery + +$ hledger -f- bal assets --lots 2>&1 +> /was not classified/ +>=1 + +# ** 3. Zero-amount lotful posting (eg a balance assertion) is not an error with --lots. +< +commodity AAPL ; lots: + +2026-01-15 check balance + assets:broker 0 AAPL = 0 AAPL + income:mystery + +$ hledger -f- bal assets --lots 2>&1 +> !/was not classified/ +>=0 + +# ** 4. Implicit zero-amount lotful posting (balance assignment inferring zero) is not an error with --lots. +< +commodity AAPL ; lots: + +2026-01-15 check balance + assets:broker = 0 AAPL + income:mystery + +$ hledger -f- bal assets --lots 2>&1 +> !/was not classified/ +>=0 + +# ** 5. Non-lotful commodity in asset account does not error with --lots. +< +2026-01-15 normal + assets:checking $100 + income:salary + +$ hledger -f- bal assets --lots 2>&1 +> !/was not classified/ +>=0 + +# ** 6. Gain posting does not trigger unclassified error with --lots. +< +account revenue:gains ; type:V + +2026-01-15 buy + assets:stocks 10 AAPL {$50} + assets:cash + +2026-02-01 sell + assets:stocks -5 AAPL {$50} @ $60 + assets:cash $300 + revenue:gains + +$ hledger -f- bal assets --lots 2>&1 +> !/was not classified/ +>=0 + +# ** 7. Bare acquire with no inferable cost errors with --lots. +< +commodity T ; lots: + +2026-01-17 buy + assets:wj -52 A + assets:broker 2220 T ; @ $0.01 + equity:conversion + +$ hledger -f- bal assets --lots 2>&1 +> /has no cost basis or price/ +>=1 diff --git a/hledger/test/journal/lots.test b/hledger/test/journal/lots.test index c3a8bea568c..20161714283 100644 --- a/hledger/test/journal/lots.test +++ b/hledger/test/journal/lots.test @@ -1,6 +1,11 @@ -# * Lot syntax +# An overview of lot functional tests: +# 1. This file covers basic input/output of lot syntax. +# 2. lots-*-entries.test cover basic lot posting classification and calculation, for many journal entry variants. +# 3. lots-*.test cover more complex lot scenarios. -# ** 1. Ledger-style lot syntax is parsed and reproduced in print output. +# * Reading and writing lot syntax + +# ** 1. Ledger-style lot syntax is parsed and printed in consolidated format. < 2026-01-01 assets:investment 10 AAPL @ $100 {$100} [2026-01-01] (lot A) @@ -8,12 +13,150 @@ $ hledger -f- print 2026-01-01 - assets:investment 10 AAPL @ $100 {$100} [2026-01-01] (lot A) + assets:investment 10 AAPL {2026-01-01, "lot A", $100} @ $100 assets:cash >= -# ** 2. Lot info appears in the acostbasis field in JSON output. +# ** 2. hledger consolidated lot syntax {DATE, COST} is accepted in input. +< +2026-01-01 + assets:investment 10 AAPL {2026-01-01, $100} + assets:cash + +$ hledger -f- print +2026-01-01 + assets:investment 10 AAPL {2026-01-01, $100} + assets:cash + +>= + +# ** 3. hledger consolidated lot syntax with label {DATE, "LABEL", COST}. +< +2026-01-01 + assets:investment 10 AAPL {2026-01-01, "lot A", $100} + assets:cash + +$ hledger -f- print +2026-01-01 + assets:investment 10 AAPL {2026-01-01, "lot A", $100} + assets:cash + +>= + +# ** 4. Partial: date only. +< +2026-01-01 + assets:investment 10 AAPL {2026-01-01} + assets:cash + +$ hledger -f- print +2026-01-01 + assets:investment 10 AAPL {2026-01-01} + assets:cash + +>= + +# ** 5. Partial: label only. +< +2026-01-01 + assets:investment 10 AAPL {"lot1"} + assets:cash + +$ hledger -f- print +2026-01-01 + assets:investment 10 AAPL {"lot1"} + assets:cash + +>= + +# ** 6. Parsing: a lot subaccount name with date and cost, +# with commas in the commodity symbol and a comma decimal mark. +# The commas should not disrupt parsing of the parts. +< +2000-01-01 buy + assets:{2026-01-01, "an, odd, commodity" 1,5} 2 AAA + assets + +$ hledger -f- print +2000-01-01 buy + assets:{2026-01-01, "an, odd, commodity" 1,5} 2 AAA {2026-01-01, "an, odd, commodity" 1,5} + assets + +>= + +# ** 7. Parsing: like the above, but also with a label containing commas. +< +2000-01-01 buy + assets:{2026-01-01, "a, b, b", "an, odd, commodity" 1,5} 2 AAA + assets + +$ hledger -f- print +2000-01-01 buy + assets:{2026-01-01, "a, b, b", "an, odd, commodity" 1,5} 2 AAA {2026-01-01, "a, b, b", "an, odd, commodity" 1,5} + assets + +>= + +# ** 8. Parsing: like 7, but with the cost's commodity symbol on the right. +< +2000-01-01 buy + assets:{2026-01-01, "a, b, b", 1,5 "an, odd, commodity"} 2 AAA + assets + +$ hledger -f- print +2000-01-01 buy + assets:{2026-01-01, "a, b, b", 1,5 "an, odd, commodity"} 2 AAA {2026-01-01, "a, b, b", 1,5 "an, odd, commodity"} + assets + +>= + +# ** 9. Consolidated with @ transacted cost. +< +2026-01-01 + assets:broker 10 AAPL {2026-01-01, $100} @ $150 + assets:cash + +$ hledger -f- print +2026-01-01 + assets:broker 10 AAPL {2026-01-01, $100} @ $150 + assets:cash + +>= + +# ** 10. Consolidated with --lots pipeline. +< +2026-01-15 buy + assets:stocks 10 AAPL {2026-01-15, $50} + assets:checking + +$ hledger -f- bal assets:stocks --lots -N + 10 AAPL assets:stocks:{2026-01-15, $50} + +# ** 11. Consolidated syntax rejects trailing [DATE]. +< +2026-01-01 + assets:broker 10 AAPL {2026-01-01, $100} [2026-01-01] + assets:cash + +$ hledger -f- print +>2 /cannot be combined/ +>=1 + +# ** 12. Empty cost basis annotations like {} are also preserved in print output. +< +2026-01-01 + assets:broker 10 AAPL {} + assets:cash + +$ hledger -f- print +2026-01-01 + assets:broker 10 AAPL {} + assets:cash + +>= + +# ** 13. print's json output shows lot info in the acostbasis field. $ hledger -f- print -O json [ { @@ -25,52 +168,15 @@ $ hledger -f- print -O json "tindex": 1, "tpostings": [ { - "paccount": "assets:investment", + "paccount": "assets:broker", "pamount": [ { "acommodity": "AAPL", - "acost": { - "contents": { - "acommodity": "$", - "acost": null, - "acostbasis": null, - "aquantity": { - "decimalMantissa": 100, - "decimalPlaces": 0, - "floatingPoint": 100 - }, - "astyle": { - "ascommodityside": "L", - "ascommodityspaced": false, - "asdecimalmark": null, - "asdigitgroups": null, - "asprecision": 0, - "asrounding": "NoRounding" - } - }, - "tag": "UnitCost" - }, + "acost": null, "acostbasis": { - "cbCost": { - "acommodity": "$", - "acost": null, - "acostbasis": null, - "aquantity": { - "decimalMantissa": 100, - "decimalPlaces": 0, - "floatingPoint": 100 - }, - "astyle": { - "ascommodityside": "L", - "ascommodityspaced": false, - "asdecimalmark": null, - "asdigitgroups": null, - "asprecision": 0, - "asrounding": "NoRounding" - } - }, - "cbDate": "2026-01-01", - "cbLabel": "lot A" + "cbCost": null, + "cbDate": null, + "cbLabel": null }, "aquantity": { "decimalMantissa": 10, @@ -80,7 +186,7 @@ $ hledger -f- print -O json "astyle": { "ascommodityside": "R", "ascommodityspaced": true, - "asdecimalmark": null, + "asdecimalmark": ".", "asdigitgroups": null, "asprecision": 0, "asrounding": "NoRounding" @@ -92,27 +198,36 @@ $ hledger -f- print -O json "pdate": null, "pdate2": null, "poriginal": null, + "preal": "RealPosting", "pstatus": "Unmarked", - "ptags": [], - "ptransaction_": "1", - "ptype": "RegularPosting" + "ptags": [ + [ + "_ptype", + "acquire" + ] + ], + "ptransaction_": "1" }, { "paccount": "assets:cash", "pamount": [ { - "acommodity": "$", + "acommodity": "AAPL", "acost": null, - "acostbasis": null, + "acostbasis": { + "cbCost": null, + "cbDate": null, + "cbLabel": null + }, "aquantity": { - "decimalMantissa": -1000, + "decimalMantissa": -10, "decimalPlaces": 0, - "floatingPoint": -1000 + "floatingPoint": -10 }, "astyle": { - "ascommodityside": "L", - "ascommodityspaced": false, - "asdecimalmark": null, + "ascommodityside": "R", + "ascommodityspaced": true, + "asdecimalmark": ".", "asdigitgroups": null, "asprecision": 0, "asrounding": "NoRounding" @@ -124,10 +239,10 @@ $ hledger -f- print -O json "pdate": null, "pdate2": null, "poriginal": null, + "preal": "RealPosting", "pstatus": "Unmarked", "ptags": [], - "ptransaction_": "1", - "ptype": "RegularPosting" + "ptransaction_": "1" } ], "tprecedingcomment": "", @@ -148,7 +263,7 @@ $ hledger -f- print -O json } ] -# ** 3. Lot info appears in Beancount lot syntax in beancount output. +# ** 14. print -O beancount produces Beancount lot syntax. < 2026-01-01 Buy stocks with full lot info assets:investment 10 AAPL {$150} [2026-01-01] (lot1) @@ -167,6 +282,7 @@ $ hledger -f- print -O json assets:cash $ hledger -f- print -O beancount +;option "inferred_tolerance_default" "*:0.005" 2026-01-01 open Assets:Cash 2026-01-01 open Assets:Investment @@ -187,3 +303,67 @@ $ hledger -f- print -O beancount Assets:Cash >= + +# ** 15. In beancount output, an account's lots: tag value becomes the account's booking method +< +account assets:broker ; lots: FIFO + +2026-01-08 Buy stock + assets:broker 10 AAPL {$100} + assets:cash -$1000 + +$ hledger -f- print -O beancount +;option "inferred_tolerance_default" "*:0.005" +2026-01-08 open Assets:Broker "FIFO" +2026-01-08 open Assets:Cash + +2026-01-08 * "Buy stock" + Assets:Broker 10 AAPL {100 USD} + Assets:Cash -1000 USD + +>= + +# ** 16. In beancount output, balance assignment amounts are always made explicit. +< +2026-01-01 * opening balances + equity:opening + assets:cash = 2000 USD + +$ hledger -f- print -O beancount +;option "inferred_tolerance_default" "*:0.005" +2026-01-01 open Assets:Cash +2026-01-01 open Equity:Opening + +2026-01-01 * "opening balances" + Equity:Opening + Assets:Cash 2000 USD + +>= + +# ** 17. Most other reports ignore cost basis by default +< +2026-01-01 Buy with transaction cost + assets:broker 10 AAPL @ $160 + assets:cash + +2026-01-02 Buy with cost basis + assets:broker 10 AAPL {$150} + assets:cash + +$ hledger -f- bal assets:broker + 20 AAPL assets:broker +-------------------- + 20 AAPL + +# ** 18. print -O ledger produces Ledger-style lot syntax. +< +2026-01-01 + assets:investment 10 AAPL {2026-01-01, "lot A", $100} @ $150 + assets:cash + +$ hledger -f- print -O ledger +2026-01-01 + assets:investment 10 AAPL [2026-01-01] (lot A) {$100} @ $150 + assets:cash + +>= diff --git a/hledger/test/json.test b/hledger/test/json.test index 5489cab9318..c82d6925d4f 100644 --- a/hledger/test/json.test +++ b/hledger/test/json.test @@ -38,10 +38,10 @@ $ hledger -f- reg --output-format=json "pdate": null, "pdate2": null, "poriginal": null, + "preal": "VirtualPosting", "pstatus": "Unmarked", "ptags": [], - "ptransaction_": "1", - "ptype": "VirtualPosting" + "ptransaction_": "1" }, [ { diff --git a/hledger/test/ledger-compat/hledger-other.test b/hledger/test/ledger-compat/hledger-other.test index 9b770b18d46..f17d8ba4509 100644 --- a/hledger/test/ledger-compat/hledger-other.test +++ b/hledger/test/ledger-compat/hledger-other.test @@ -112,5 +112,3 @@ $ hledger -f- check Income:Capital Gains $-125.00 $ hledger -f- check ->2 /transaction is unbalanced/ ->=1 diff --git a/hledger/test/print/beancount.test b/hledger/test/print/beancount.test index e0ba8c6b328..b54b2398c83 100644 --- a/hledger/test/print/beancount.test +++ b/hledger/test/print/beancount.test @@ -16,6 +16,7 @@ $ hledger -f- print -O beancount equity:$-€:$ 0 USD $ hledger -f- print -O beancount +;option "inferred_tolerance_default" "*:0.005" 2000-01-01 open Assets:A 2000-01-01 open Equity:C24-C20ac:C24 @@ -34,11 +35,12 @@ $ hledger -f- print -O beancount assets 0 "size 2 pencils" $ hledger -f- print -O beancount +;option "inferred_tolerance_default" "*:0.005" 2000-01-01 open Assets:A 2000-01-01 * Assets:A 0 USD - Assets:A 0 C + Assets:A 0 CC Assets:A 0 C21 Assets:A 0 SIZE-2-PENCILS @@ -53,6 +55,7 @@ $ hledger -f- print -O beancount [c] $ hledger -f- print -O beancount +;option "inferred_tolerance_default" "*:0.005" 2000-01-01 open Assets:A 2000-01-01 * @@ -72,6 +75,7 @@ $ hledger -f- print -O beancount Assets €-1 $ hledger -f- print -O beancount +;option "inferred_tolerance_default" "*:0.005" 2000-01-01 open Assets:A 2000-01-01 open Equity:Conversion:C20ac-C24:C24 2000-01-01 open Equity:Conversion:C20ac-C24:C20ac @@ -94,6 +98,7 @@ $ hledger -f- print -O beancount Assets €-1 $ hledger -f- print -O beancount +;option "inferred_tolerance_default" "*:0.005" option "operating_currency" "EUR" 2000-01-01 open Assets:A @@ -144,6 +149,7 @@ account assets:cash ; type:C assets:cash 0 ; a: ptag $ hledger -f- print -O beancount +;option "inferred_tolerance_default" "*:0.005" 2000-01-01 open Assets:Cash type: "C" @@ -162,7 +168,100 @@ $ hledger -f- print -O beancount 2000-01-02 * "posting tags" ; a: ttag ma: "ttag" - Assets:Cash 0 C ; a: ptag + Assets:Cash 0 CC ; a: ptag ma: "ptag" >= + +# ** 9. Beancount output shows tags as beancount metadata (as well as in the comment); +# and does not expose posting tags that weren't in the comment, such as those inherited from commodity and account. +< +commodity AAPL ; lots: +account assets:broker ; type: Asset + +2026-01-01 Buy stock + assets:broker 10 AAPL {$100} + assets:cash -$1000 + +$ hledger -f- print -O beancount --verbose-tags +;option "inferred_tolerance_default" "*:0.005" +2026-01-01 open Assets:Broker + type: "Asset" +2026-01-01 open Assets:Cash + +2026-01-01 * "Buy stock" + Assets:Broker 10 AAPL {100 USD} ; ptype: acquire + ptype: "acquire" + Assets:Cash -1000 USD + +>= + +# ** 10. Price directives are converted to beancount price directives, sorted by date. +< +P 2026-01-15 AAPL 170.00 USD +P 2026-01-10 EUR 1.08 USD + +2026-01-01 + assets:cash $100 + equity:opening + +$ hledger -f- print -O beancount +;option "inferred_tolerance_default" "*:0.005" +2026-01-01 open Assets:Cash +2026-01-01 open Equity:Opening + +2026-01-10 price EUR 1.08 USD +2026-01-15 price AAPL 170.00 USD + +2026-01-01 * + Assets:Cash 100 USD + Equity:Opening + +>= + +# ** 11. "revenue" and "revenues" top-level account names are converted to "Income". +< +2026-01-01 + assets:cash $100 + revenue:salary + +2026-01-02 + assets:cash $200 + revenues:consulting + +$ hledger -f- print -O beancount +;option "inferred_tolerance_default" "*:0.005" +2026-01-01 open Assets:Cash +2026-01-01 open Income:Salary +2026-01-01 open Income:Consulting + +2026-01-01 * + Assets:Cash 100 USD + Income:Salary + +2026-01-02 * + Assets:Cash 200 USD + Income:Consulting + +>= + +# ** 12. Single-character commodity names are doubled to meet beancount's 2-character minimum. +< +2026-01-01 + assets:cash 1 K + equity:opening + +P 2026-01-01 K 2.00 USD + +$ hledger -f- print -O beancount +;option "inferred_tolerance_default" "*:0.005" +2026-01-01 open Assets:Cash +2026-01-01 open Equity:Opening + +2026-01-01 price KK 2.00 USD + +2026-01-01 * + Assets:Cash 1 KK + Equity:Opening + +>= diff --git a/stack.yaml b/stack.yaml index 8967dec5c46..2a87bef4df4 100644 --- a/stack.yaml +++ b/stack.yaml @@ -1,8 +1,5 @@ -# stack build plan using GHC 9.12.2 -# https://gitlab.haskell.org/ghc/ghc/-/wikis/migration/9.12 -# https://downloads.haskell.org/ghc/9.12.1/docs/users_guide/9.12.1-notes.html -# https://downloads.haskell.org/ghc/9.12.2/docs/users_guide/9.12.2-notes.html -# https://downloads.haskell.org/ghc/9.12.2/docs/users_guide/9.12.3-notes.html +# stack build plan using GHC 9.14 installed in PATH +# https://gitlab.haskell.org/ghc/ghc/-/wikis/migration/9.14 packages: - hledger-lib @@ -10,10 +7,10 @@ packages: - hledger-ui - hledger-web -resolver: nightly-2025-09-30 - -notify-if-ghc-untested: false -notify-if-cabal-untested: false +resolver: nightly-2026-03-23 +compiler: ghc-9.14 +# notify-if-ghc-untested: false +# notify-if-cabal-untested: false notify-if-no-run-tests: false extra-deps: @@ -24,26 +21,3 @@ extra-deps: - yesod-test-1.6.19 - cryptonite-0.30 - cryptonite-conduit-0.2.2 - -nix: - pure: false - packages: [perl gmp ncurses zlib] - -# ghc-options: -# "$locals": -Wno-x-partial -# "$locals": -fplugin Debug.Breakpoint - -# # for precise profiling, per https://www.tweag.io/posts/2020-01-30-haskell-profiling.html: -# # apply-ghc-options: everything -# # rebuild-ghc-options: true -# # stack build --profile --ghc-options="-fno-prof-auto" - -# # tell GHC to write hie files, eg for weeder. Rumoured to be slow. -# # ghc-options: -# # "$locals": -fwrite-ide-info - -# # ghc-options: -# # "$locals": -ddump-timings -# # "$targets": -Werror -# # "$everything": -O2 -# # some-package: -DSOME_CPP_FLAG diff --git a/stack912.yaml b/stack912.yaml new file mode 100644 index 00000000000..0cd44ed9ce7 --- /dev/null +++ b/stack912.yaml @@ -0,0 +1,45 @@ +# stack build plan using GHC 9.12.2 +# https://gitlab.haskell.org/ghc/ghc/-/wikis/migration/9.12 +# https://downloads.haskell.org/ghc/9.12.1/docs/users_guide/9.12.1-notes.html +# https://downloads.haskell.org/ghc/9.12.2/docs/users_guide/9.12.2-notes.html +# https://downloads.haskell.org/ghc/9.12.2/docs/users_guide/9.12.3-notes.html + +packages: +- hledger-lib +- hledger +- hledger-ui +- hledger-web + +resolver: nightly-2025-12-30 + +extra-deps: +# for #2410, #2512: +- haskeline-0.8.4.1 +# currently out of stackage: +- yesod-static-1.6.1.0 # <1.6.1.1 to avoid https://github.com/psibi/crypton-conduit/issues/3 +- yesod-test-1.6.19 +- cryptonite-0.30 +- cryptonite-conduit-0.2.2 + +nix: + pure: false + packages: [perl gmp ncurses zlib] + +# ghc-options: +# "$locals": -Wno-x-partial +# "$locals": -fplugin Debug.Breakpoint + +# # for precise profiling, per https://www.tweag.io/posts/2020-01-30-haskell-profiling.html: +# # apply-ghc-options: everything +# # rebuild-ghc-options: true +# # stack build --profile --ghc-options="-fno-prof-auto" + +# # tell GHC to write hie files, eg for weeder. Rumoured to be slow. +# # ghc-options: +# # "$locals": -fwrite-ide-info + +# # ghc-options: +# # "$locals": -ddump-timings +# # "$targets": -Werror +# # "$everything": -O2 +# # some-package: -DSOME_CPP_FLAG diff --git a/stack914.yaml b/stack914.yaml deleted file mode 100644 index 545a5560aea..00000000000 --- a/stack914.yaml +++ /dev/null @@ -1,23 +0,0 @@ -# stack build plan using GHC 9.14 installed in PATH -# https://gitlab.haskell.org/ghc/ghc/-/wikis/migration/9.14 - -resolver: nightly-2026-01-08 -compiler: ghc-9.14 -notify-if-ghc-untested: false -notify-if-cabal-untested: false -allow-newer: true - -packages: -- hledger-lib -- hledger -- hledger-ui -- hledger-web - -extra-deps: -# for #2410, #2512: -- haskeline-0.8.4.1 -# currently out of stackage: -- yesod-static-1.6.1.0 -- yesod-test-1.6.19 -- cryptonite-0.30 -- cryptonite-conduit-0.2.2