Releases: pygraphkit/graphtik
v2.0.1b0 (18 Oct 2019): better plan with perfect evictions
-
ENH(NET): new PLAN BULDING algorithm produces PERFECT EVICTIONS,
that is, it gradually eliminates from the solution all non-asked outputs.- enh: pruning now cleans isolated data.
- enh: eviction-instructions are inserted due to two different conditions:
once for unneeded data in the past, and another for unused produced data
(those not belonging typo the pruned dag). - enh: discard immediately irrelevant inputs.
-
ENH(net): changed results, now unrelated inputs are not included in solution.
-
refact(sideffect): store them as node-attributes in DAG, fix their combination
with pinning & eviction. -
fix(parallel): eviction was not working due to a typo 65 commits back!
(15 Oct 2019): Rebranded as "Graphttik" for PY6+, new DAG, better plots, +sideffects
===============================================================
REBRANDED from GraphKit --> Graphtik, and kept external API (hopefully) the same,
but revamped pruning algorithm and refactored network compute/compile structure,
so results may change; significantly enhanced plotting.
The only new feature actually is the :class:`sideffect`` modifier.
Network:
-
FIX(yahoo#18, yahoo#26, yahoo#29, yahoo#17, yahoo#20): Revamped DAG SOLVER
to fix bad pruning described in yahoo#24 & yahoo#25Pruning now works by breaking incoming provide-links to any given
intermedediate inputs dropping operations with partial inputs or without outputs.The end result is that operations in the graph that do not have all inputs satisfied,
they are skipped (in v1.2.4 they crashed).Also started annotating edges with optional/sideffects, to make proper use of
the underlyingnetworkxgraph.|v130-flowchart|
-
REFACT(yahoo#21, yahoo#29): Refactored Network and introduced :class:
ExecutionPlanto keep
compilation results (the oldstepslist, plus input/output names).Moved also the check for when to evict a value, from running the execution-plan,
to whenbuilding it; thus, execute methods don't need outputs anymore. -
ENH(yahoo#26): "Pin* input values that may be overriten by calculated ones.
This required the introduction of the new :class:
_PinInstructionin
the execution plan. -
FIX(yahoo#23, yahoo#22-2.4.3): Keep consistent order of
networkx.DiGraph
and sets, to generate deterministic solutions.Unfortunately, it non-determinism has not been fixed in < PY3.5, just
reduced the frequency ofspurious failures <https://travis-ci.org/yahoo/graphkit/builds/594729787>_, caused by
unstable dicts, and the use of subgraphs. -
REFACT: rename classes
Delete --> Evict, removedPlaceholder
from_DataNodes, privatized them. -
ENH: collect "jetsam" on errors and annotate exceptions with them.
-
enh: accept a single string as
outputswhen running graphs. -
enh: Mark outputs produced by :class:
NetworkOperation's needs asoptional.
TODO: subgraph network-operations would not be fully functional until
"optional outpus" are dealt with (see yahoo#22-2.5). -
enh: Annotate operation exceptions with
ExecutionPlanto aid debug sessions, -
drop: methods
list_layers()/show layers()not needed,repr()is
a better replacement.
Plotting:
-
ENH(yahoo#13, yahoo#26, yahoo#29): Now network remembers last plan and uses that
to overlay graphs with the internals of the planing and execution: |sample-plot|- execution-steps & order
- evict & pin instructions
- given inputs & asked outputs
- solution values (just if they are present)
- "optional" needs & broken links during pruning
-
REFACT: Move all API doc on plotting in a single module, splitted in 2 phases,
build DOT & render DOT -
FIX(yahoo#13): bring plot writing into files up-to-date from PY2; do not create plot-file
if given file-extension is not supported. -
FEAT: path
pydot library <https://pypi.org/project/pydot/>_ to support rendering
in Jupyter notebooks.
Testing & other code:
- Increased coverage from 77% --> 90%.
-
ENH(yahoo#28): use
pytest, to facilitate TCs parametrization. -
ENH(yahoo#30): doctest all code; enabled many assertions that were just print-outs
in v1.2.4. -
ENH(yahoo#31): check sphinx-site builds without any warnings.
-
FIX:
operation.__repr__()was crashing when not all arguments
had been set - a condition frequtnly met during debugging session or failed
TCs (inspired by @syamajala's 309338340). -
enh: Sped up parallel/multihtread TCs by reducing delays & repetitions.
.. tip::
You needpytest -m slowto run those slow tests. -
Established chores with build services for rebranded project:
- Travis (and auto-deploy to PyPi),
- codecov
- ReadTheDocs