Releases: pygraphkit/graphtik
(30 Jun 2020, @ankostis): JSONP; net, evictions & sfxed fixes; conveyor fn; rename modules
-
FEAT(modifier): Dependencies with :term:
json pointer paththat can read/write
:term:subdoc\s (e.g. nested dicts & pandas).- feat(config): added :func:
set_layered_solution()into :term:configurations
which when True (or jsnops in the network if None (default)) all results
are stored in the given inputs to the pipeline
(this may become the regular behavior in the future). - feat(modifier, solution): +modifier with accessor functions to read/write Solution.
- doc: new section :ref:
hierarchical-dataputting together all advanced features
of the project in a "Weekly task runner".
- feat(config): added :func:
-
BREAK/REFACT: modules and objects renamed:
+---------------------------------+-----------------------------+
| FROM | TO |
+=================================+=============================+
| :file:modifierS.py| :file:modifier.py|
+---------------------------------+-----------------------------+
| func: modifiers.fn_kwarg | :func:.modifier.keyword()|
+---------------------------------+-----------------------------+
| :file:network.py| :file:planning.py|
+---------------------------------+-----------------------------+
| :file:op.py| :file:fnop.py|
+---------------------------------+-----------------------------+
| class: op.FunctionalOperation | :class:.fnop.FnOp|
+---------------------------------+-----------------------------+ -
FEAT(op): default :func:
.identity_function()acting as :term:conveyor operation. -
FIX(NET, EXECUTION): discovered and fixed bugs in pruning, evictions and rescheduling
with overwrites, while testing newjsonpmodifier; rely on dag alone while pruning
(and not digging into op needs/provides).- Dupe Evictions of pruned output were deliberately & wrongly consolidated, while
it is possible to need to evict repeatedly the same out from multiple ops
providing it. - Less aggressive prune-isolated-data permits SFX not to be asked explicitly,
and behave more like regular data.
Now For certain cases, the more specific error "Unreachable out" gets raised,
instead of the too generic "Unsolvable graph". - Prune-by-outputs was ignoring given inputs, chocking on computation cycles
that were possible to avoid!
- Dupe Evictions of pruned output were deliberately & wrongly consolidated, while
-
DROP(net):
_EvictionInstructionclass was obscuring modifier combinations, and
it didn't make sense any more, being the only instruction. -
FEAT(ops, pipelines, net, sol): unified :meth:
.Plottable.opsutility properties. -
ENH: Error reporting:
- enh(op, pipe): fail earlier if no function/name given when defining operations
and pipelines. - enh(op): when :envvar:
GRAPHTIK_DEBUGvar defined, any errors during inputs/needs
matching are raised immediately. - enh: improve tips & hints in exception messages; log past executed operations
when a pipeline fails.
- enh(op, pipe): fail earlier if no function/name given when defining operations
-
DOC(op): table explaining the differences between various dependency attributes of
:class:.FnOp... include:: ../../graphtik/fnop.py
:start-after: .. dep-attributes-start
:end-before: .. dep-attributes-end -
enh(op, pipe): restrict operation names to be strings (were :class:
collection.abc.Hashable). -
feat(modifier): public-ize :func:
modifier_withset()to produce modified
clones -- handle it with care. -
feat(doc): Add new section with most significant :ref:
featuresof this project. -
fix(travis): update
pytestor elsepip-install chokes with <https://travis-ci.org/github/ankostis/graphkit/jobs/700326904>_pytest-coverageplugin. -
enh(pytest): add
--logger-disabledCLI option when running TCs, as explained
inpytest-dev/pytest#7431 <https://github.com/pytest-dev/pytest/issues/7431>_. -
refact(tests): split big :file:
test/test_graphtik.pyTC file into multiple
ones, per functionality area (features).
(15 May 2020, @ankostis): subclass-able Op, plot edges from south-->north of nodes
- ENH(pipe): nest all Ops (not just FnOps), dropping
FunctionalOperation
dependency in network code, to allow for further sub-classing :class:Operation. - FIX(pipeline): due to a side-effect on a
kwdictionary, it was mixing the attributes
of earlier operations into later ones while merging them into pipelines. - REFACT(solution): facilitate inheriting Solution by extracting :meth:
.Solution._update_op_outsinto a separate method. - refact(pipe): move
build_net()--> back topipelinemodule,
dropping further network.py-->pipeline.py mod-dep. - enh(plot): StyleStack-ize data-io shape selection into separate theme-able dicts.
- DOC(exe, plotting): task-context section in Debugger
(14 May 2020, @ankostis): plot edges from south-->north of nodes
- ENH(plot): have all the link-edges between data and operations route out and into
the same point on the nodes (src: south, dst: north).
Distinguish needs edges from provides with a "dot".
(12 May 2020, @ankostis): mapped-->keyword, drop sol-finalize
- BREAK: rename
mapped --> keyword, which conveys the mot important meaning. - DROP Solution.finalized() method -- has stopped being used to reverse values
since sfxed have been introduced (v7+). - doc(modifiers): explain :term:
diacriticsymbols of dependencies when in printouts.
(11 May 2020, @ankostis): custom Solutions, Task-context,
- FEAT(exe):
compute()supports custom Solution classes. - FEAT(exe): underlying functions gain access to wrapping Operation with :data:
.execution.task_context.
(11 May 2020, @ankostis): drop last plan, Rename/Nest, Netop-->Pipeline, purify modules
-
DROP(pipeline): After solution class was introduced,
last_planattribute was redundant. -
REFACT: revert module splits and arrive back to :mod:
base.py, :mod:op.py&
:mod:pipeline.py, to facilitate development with smaller files, but still
with very few import-time dependencies.Importing project :term:
compositionclasses takes less than 4ms in a fast 2019 PC
(down from 300ms). -
FIX(plot): updated Legend, which had become outdated since v6+.
-
ENH(op): Rename & Nest operations with dictionary or callable.
-
enh(pipe): accept a dictionary with renames when doing :term:
operation nesting
(instead of callables or truthies). -
refact(op): force abstract :class:
.Operationto be :class:.Plottable. -
fix(modifiers):
dep_renamed()was faking sideffect-renaming only on repr(). -
enh(modifiers): add :attr:
._Modifier.cmdwith code to reproduce modifier.
(7 May 2020, @ankostis): re-MODULE; sideffect --> sfx; all DIACRITIC Modifiers; invert "merge" meaning
--((superseded immediately v8.0.1 & v8.0.2 with more restructurings)))--
-
BREAK: restructured
netop&&networkmodules:- BREAK: stopped(!) importing :mod:
.configtop-level. - BREAK: :mod:
.networkmodule was splitted into :mod:.executionwhich now
contains plan+solution; - BREAK: unified modules
op+netop--> :mod.composition. - DOC: module dependencies diagram in :doc:
reference;
now x60 fasterimport compositionfrom 300ms --> 5ms.
- BREAK: stopped(!) importing :mod:
-
BREAK: sideffect modifier functions shortened to :func:
.sfx& :func:.sfxed.- FEAT: +Sideffected :term:
varargish-- now :term:sideffectedfully mirror
a regular dependency. - ENH: change visual repr\esentation of modifiers with DIACRITICS only.
- refact(modifiers): use cstor matrix to combine modifier arguments; new utility
method for renaming dependencies :func:.dep_renamed()
(usefull when :ref:operation-nesting, see below). - ENH: possible to rename also sideffects; the actual sideffect string is now
stored in the modifier.
- FEAT: +Sideffected :term:
-
BREAK/ENH: invert ":term:
merge <operation merging>" meaning with (newly introduced)
":term:"nest `"; default is now is merge:- FEAT: introduce the :class:
NULL_OPoperation that can "erase" an existing
operation when merging pipelines. - ENH:
compose(..., nest=nest_cb)where the callback accepts :class:.NestArgs
and can perform any kind of renaming on data + operations before :term:combining pipelines <combine pipelines>. - doc: "merge" identically-named ops override each other, "nest" means they are prefixed,
"combine" means both operations. - DOC: re-written a :ref:
merge-vs-nest tutorial <operation-merging>for humanity.
- FEAT: introduce the :class:
-
DROP(op):
parentattribute is no longer maintained -- operation identity now
based only on name, which may implicitly be nested by dots(.). -
ENH(plot): accept bare dictionary as theme overrides when plotting.
-
doc: fix site configuration for using the standard
<s5defs>include
for colored/font-size sphinx roles.
(7 May 2020, @ankostis): superseded immediately BY v8.0.2 due to more restructs
--((superseded immediately BY v8.0.2 due to more restructs)))--
(7 May 2020, @ankostis): re-MODULE; sideffect --> sfx; all DIACRITIC Modifiers; invert "merge" meaning
-
BREAK: restructured
netop&&networkmodules:- :mod:
.networkmodule was splitted into :mod:.executionwhich now
contains plan+solution; - unified modules
op+netop--> :mod.composition. - DOC: module dependencies diagram in :doc:
reference.
- :mod:
-
BREAK: sideffect modifier functions shortened to :func:
.sfx& :func:.sfxed.- FEAT: +Sideffected :term:
varargish-- now :term:sideffectedfully mirror
a regular dependency. - ENH: change visual repr\esentation of modifiers with DIACRITICS only.
- refact(modifiers): use cstor matrix to combine modifier arguments; new utility
method for renaming dependencies :func:.dep_renamed()
(usefull when :ref:operation-nesting, see below). - ENH: possible to rename also sideffects; the actual sideffect string is now
stored in the modifier.
- FEAT: +Sideffected :term:
-
BREAK/ENH: invert ":term:
merge <operation merging>" meaning with (newly introduced)
":term:"nest `"; default is now is merge:- FEAT: introduce the :class:
NULL_OPoperation that can "erase" an existing
operation when merging pipelines. - ENH:
compose(..., nest=nest_cb)where the callback accepts :class:.NestArgs
and can perform any kind of renaming on data + operations before :term:combining pipelines <combine pipelines>. - doc: "merge" identically-named ops override each other, "nest" means they are prefixed,
"combine" means both operations. - DOC: re-written a :ref:
merge-vs-nest tutorial <operation-merging>for humanity.
- FEAT: introduce the :class:
-
DROP(op):
parentattribute is no longer maintained -- operation identity now
based only on name, which may implicitly be nested by dots(.). -
ENH(plot): accept bare dictionary as theme overrides when plotting.
-
doc: fix site configuration for using the standard
<s5defs>include
for colored/font-size sphinx roles.
(7 May 2020, @ankostis): superseded immediately by v8.0.1 due to +1 module restruct
-((superseded immediately BY v8.0.1 due to +1 module restruct)))--
(7 May 2020, @ankostis): re-MODULE; sideffect --> sfx; all DIACRITIC Modifiers; invert "merge" meaning
-
BREAK: restructured
netop&&networkmodules:- :mod:
.networkmodule was splitted into :mod:.executionwhich now
contains plan+solution; - renamed module
netop--> :mod:.pipeline. - DOC: module dependencies diagram in :doc:
reference.
- :mod:
-
BREAK: sideffect modifier functions shortened to :func:
.sfx& :func:.sfxed.- FEAT: +Sideffected :term:
varargish-- now :term:sideffectedfully mirror
a regular dependency. - ENH: change visual repr\esentation of modifiers with DIACRITICS only.
- refact(modifiers): use cstor matrix to combine modifier arguments; new utility
method for renaming dependencies :func:.dep_renamed()
(usefull when :ref:operation-nesting, see below). - ENH: possible to rename also sideffects; the actual sideffect string is now
stored in the modifier.
- FEAT: +Sideffected :term:
-
BREAK/ENH: invert ":term:
merge <operation merging>" meaning with (newly introduced)
":term:"nest `"; default is now is merge:- FEAT: introduce the :class:
NULL_OPoperation that can "erase" an existing
operation when merging pipelines. - ENH:
compose(..., nest=nest_cb)where the callback accepts :class:.NestArgs
and can perform any kind of renaming on data + operations before :term:combining pipelines <combine pipelines>. - doc: "merge" identically-named ops override each other, "nest" means they are prefixed,
"combine" means both operations. - DOC: re-written a :ref:
merge-vs-nest tutorial <operation-merging>for humanity.
- FEAT: introduce the :class:
-
DROP(op):
parentattribute is no longer maintained -- operation identity now
based only on name, which may implicitly be nested by dots(.). -
ENH(plot): accept bare dictionary as theme overrides when plotting.
-
doc: fix site configuration for using the standard
<s5defs>include
for colored/font-size sphinx roles.
(6 May 2020, @ankostis): minor reschedule fixes and refactoring
Actually it contains just what was originally planned for v7.1.1.
- FIX(op): v7.0.0 promise that
op.__call__delegates tocompute()was a fake;
now it is fixed. - fix(config): endurance flags were miss-behaving.
- refact(net): factor out a :meth:
._reschedule()method for both endurance & rescheduled ops. - feat(build): +script to launch pytest on a local clone repo before pushing.

