Skip to content

Releases: pygraphkit/graphtik

v2.0.1b0 (18 Oct 2019): better plan with perfect evictions

18 Oct 16:32
v2.0.1b0

Choose a tag to compare

  • ENH(NET): new PLAN BULDING algorithm produces PERFECT EVICTIONS,
    that is, it gradually eliminates from the solution all non-asked outputs.

    • enh: pruning now cleans isolated data.
    • enh: eviction-instructions are inserted due to two different conditions:
      once for unneeded data in the past, and another for unused produced data
      (those not belonging typo the pruned dag).
    • enh: discard immediately irrelevant inputs.
  • ENH(net): changed results, now unrelated inputs are not included in solution.

  • refact(sideffect): store them as node-attributes in DAG, fix their combination
    with pinning & eviction.

  • fix(parallel): eviction was not working due to a typo 65 commits back!

(15 Oct 2019): Rebranded as "Graphttik" for PY6+, new DAG, better plots, +sideffects

15 Oct 19:28

Choose a tag to compare

===============================================================
REBRANDED from GraphKit --> Graphtik, and kept external API (hopefully) the same,
but revamped pruning algorithm and refactored network compute/compile structure,
so results may change; significantly enhanced plotting.
The only new feature actually is the :class:`sideffect`` modifier.

Network:

  • FIX(yahoo#18, yahoo#26, yahoo#29, yahoo#17, yahoo#20): Revamped DAG SOLVER
    to fix bad pruning described in yahoo#24 & yahoo#25

    Pruning now works by breaking incoming provide-links to any given
    intermedediate inputs dropping operations with partial inputs or without outputs.

    The end result is that operations in the graph that do not have all inputs satisfied,
    they are skipped (in v1.2.4 they crashed).

    Also started annotating edges with optional/sideffects, to make proper use of
    the underlying networkx graph.

    |v130-flowchart|

  • REFACT(yahoo#21, yahoo#29): Refactored Network and introduced :class:ExecutionPlan to keep
    compilation results (the old steps list, plus input/output names).

    Moved also the check for when to evict a value, from running the execution-plan,
    to whenbuilding it; thus, execute methods don't need outputs anymore.

  • ENH(yahoo#26): "Pin* input values that may be overriten by calculated ones.

    This required the introduction of the new :class:_PinInstruction in
    the execution plan.

  • FIX(yahoo#23, yahoo#22-2.4.3): Keep consistent order of networkx.DiGraph
    and sets, to generate deterministic solutions.

    Unfortunately, it non-determinism has not been fixed in < PY3.5, just
    reduced the frequency of spurious failures <https://travis-ci.org/yahoo/graphkit/builds/594729787>_, caused by
    unstable dicts, and the use of subgraphs.

  • REFACT: rename classes Delete --> Evict, removed Placeholder
    from _DataNodes, privatized them.

  • ENH: collect "jetsam" on errors and annotate exceptions with them.

  • enh: accept a single string as outputs when running graphs.

  • enh: Mark outputs produced by :class:NetworkOperation's needs as optional.
    TODO: subgraph network-operations would not be fully functional until
    "optional outpus" are dealt with (see yahoo#22-2.5).

  • enh: Annotate operation exceptions with ExecutionPlan to aid debug sessions,

  • drop: methods list_layers()/show layers() not needed, repr() is
    a better replacement.

Plotting:

  • ENH(yahoo#13, yahoo#26, yahoo#29): Now network remembers last plan and uses that
    to overlay graphs with the internals of the planing and execution: |sample-plot|

    • execution-steps & order
    • evict & pin instructions
    • given inputs & asked outputs
    • solution values (just if they are present)
    • "optional" needs & broken links during pruning
  • REFACT: Move all API doc on plotting in a single module, splitted in 2 phases,
    build DOT & render DOT

  • FIX(yahoo#13): bring plot writing into files up-to-date from PY2; do not create plot-file
    if given file-extension is not supported.

  • FEAT: path pydot library <https://pypi.org/project/pydot/>_ to support rendering
    in Jupyter notebooks.

Testing & other code:

  • Increased coverage from 77% --> 90%.
  • ENH(yahoo#28): use pytest, to facilitate TCs parametrization.

  • ENH(yahoo#30): doctest all code; enabled many assertions that were just print-outs
    in v1.2.4.

  • ENH(yahoo#31): check sphinx-site builds without any warnings.

  • FIX: operation.__repr__() was crashing when not all arguments
    had been set - a condition frequtnly met during debugging session or failed
    TCs (inspired by @syamajala's 309338340).

  • enh: Sped up parallel/multihtread TCs by reducing delays & repetitions.

    .. tip::
    You need pytest -m slow to run those slow tests.

  • Established chores with build services for rebranded project:

    • Travis (and auto-deploy to PyPi),
    • codecov
    • ReadTheDocs