jtc v1.75 - JSON transformational chains

Release Notes for jtc v.1.75

New features:

• introduced a new semi-compact printing view. The view is engaged when the suffix c is appended to -t option, e.g.:-t2c, -tc. The semi-compact view is a middle ground between compact (-r) and pretty-printed (-t, default) views: when a JSON iterable is made of atomic values only (and/or empty iterables {}, []), it will be printed in a compact (one-line) format, the rest is pretty-printed

• introduced operations chaining via delimiter /:

  • chaining delimiter(s) pretty much replaces jtc ... | jtc ... | jtc ... notation with jtc ... / ... / ... - the advantage is huge: jtc now is capable of processing multiple chained operations w/o printing-parsing interim JSONs (which is quite expensive operation) - that speeds up operations and simplifies notation
    Another benefit is that it becomes possible to pass namespace(s) from one chain set into another (which is impossible with piping notation)
  • chain-delimiter / only splits options notations, not working when cited among file arguments

• introduced an optional step notation in range subscripts and search lexemes qualifiers: [N:M:S], <..>N:M:S: S must be strictly positive value. In search quantifiers <..>::{S} if after interpolation the value happens to be negative (or zero) then the default step 1 is applied

• new search lexemes <..>g and <..>G allow going over JSON elements in a sorted order (ascending and descending respectively). When applied w/o quantifiers allow finding min and max values respectively

• a new directive <..>Z - preserves into a namespace a selected (walked) JSON entry size (a recursive and non-recursive behaviors applied respectively). <..>Z1 lexeme (i.e., with quantifier 1) - saves into a namespace a currently walked JSON string size (if the walked JSON is not a string, the value -1 is saved)

• a new lexeme <..>W - preserves a current walk-path (as a JSON array) into a namespace variable

• introduced a new parsing behavior (-mm) allowing accepting ill-formed JSONs with clashing labels by collecting them into arrays (e.g.: { "a": 1, "a": 2 } will be parsed into { "a": [ 1, 2] }

• rebranded jtc into JSON transformational chains to reflect better tool's purpose and capability

Improvements, changes, fixes:

• enhanced template interpolation (-T...):

  • removed prior limitations: now, application of templates is universal to all operations - executed as a last step for the respective walk(s)
  • extended template-interpolations of JSON iterables into strings: the former could be interpolated into the string values as enumerations: the enumeration separator value (default ", ") will be taken from newly introduced namespace $#

• new namespaces added:

  • $#: holds the separator used when a JSON iterable is interpolated into a JSON string (default value ", ")
  • $_: holds the separator used when $path is interpolated to join path tokens (default value "_")
  • $$?: holds the separator used upon template expansion when interpolation token {$?} is used (default value ",")

• introduced quantifiers for F directive (both recursive and non-recursive):

  • a new semantic for <>Fn quantifier: if n > 0 (i.e. non-default), it will let continue walking past <>Fn directive skipping to nth lexeme from F: e.g.: <>F1 - will continue walking right from the immediately following lexeme, <>F2 will continue walking from 2nd lexeme past <>Fn (i.e., skipping the first one), etc.
  • a new semantic for ><Fn quantifier: if n > 0 (i.e. non-default), it allows additional replications of the entire walk (before the lexeme ><F) n times

• enhanced <..>I directive behavior:

  • initialization of the namespace value could be done now within the lexeme itself, e.g.: <c:100>I1 - will initialize counter c with the value 100 before the directive executes (unlike typical behavior where namespace initialization/preservation is applied as the last step end of lexeme walking)
  • a new additional semantic for <..>In:m quantifier, where n is an increment step (as before, no changes here), m - is a new multiplier (integer only), e.g.: <a:10>I5:2, after the first walking the namespace a will hold (10 + 5) * 2 = 30 - in such notation, first the increment is applied and then the multiplier
  • the directive also understands now an empty token {} for the increment and/or the multiplier : <..>I{}:{} - the empty token will will refer to the currently walked (numeric) value - this is the only lexeme where such empty token notion makes sense and supported

• improved -jj option behavior: now the clashing labels will override each other (thus, only the last value will be retained), to collect even clashing labels (into an array) use -m modifier

• improved behavior of -ll toggle - now it gleans all the labels, not just the first one (as before) - typically used together with -j

• performance improvements:

  • in the JSON library, for ARY/OBJ declarations stepped away from std::initializer_list to variadic templated arguments (that permits use of move semantic now in the initialization notations, which simplified the usage and improved performance
  • improved performance of buffered read from <stdin> (now, it's almost as fast as the read from files)
  • same way improved performance of file read in options (-i, -u, -c)
  • drastically improved performance of <>q, <>Q searches by making them cacheable: they are still quite memory hungry, still are the slowest among all searches, but now they are not prone to exponential decay and can be used on big JSONs with a predictable processing time

• added a few compilation options:

  • -DBG_FLOW: a new debug of the execution flows (tracing an entry and exit point of every DEBUGGABLE function/method). Add -DBG_FLOW flag when compiling to effectuate such debugging - complements nicely -DBG_CC flag when debugging copy-constructors for optimization
  • -DBG_mTS: lets debugging output to have time-stamp with milliseconds accuracy
  • -DBG_uTS: lets debugging output to have time-stamp with microseconds accuracy

• debugability improvements:

  • added printing backtrace in the unlikely event of a crash (only when debug is enabled). On MacOs/BSD it will print demangled back-tracing
  • improved parsing output when debugged - now it'll be auto-adjusted to terminal's width

• program design improvements:

  • simplified program design for all cases of source/destination walks - that also fixed the prior caveat with labels updates through the shell evaluation (now even nested labels could be updated, the caveat is removed)
  • enhanced a logical way of handlings for all directives where applicable - now, the directive is activated only once per a walk pass (applied to directives z, Z, W, v, k, I)
  • improved/fixed behavior for shell evaluation (-e with -i/-u) argument parsing behavior for Linux/GNU only (Macos/BSD were fine - getopt() GNU implementation works differently than MacOS/BSD's)

• more fixes and enhancements:

  • fixed issue: directives <..>I and <..>u also must support interpolated name-spaced quantifiers: <..>I{ns}, <...>u{ns}
  • fixed issue: fail-safe <>f directive should not fire after there have been successful matches in iterables
  • fixed/improved parsing of <..>j search lexeme when the content is a template
  • fixed Linux options parsing (to behave the same way like on Macos/BSD)
  • fixed a corner crash when move semantic applied on multiple walks and the prior walk deletes the object pointed by the subsequent, interleaved walk
  • fixed a corner crash when a search lexeme (i,o,c, etc) was matching a root iterable (array/object) and at the same time attempted saving it into a namespace
  • fixed a crash when blank (or white space only) input was combined with the streamed read (-a)