Merge pull request #104 from predict-idlab/improve_docs

📦 improve docs

Author: jonasvdd

.gitignore

@@ -1,6 +1,7 @@
 venv*
 .cache_datasets/
 *.DS_Store
+file_system_store/
 
 # Sphinx documentation
 *_build/

README.md

@@ -158,6 +158,7 @@ In [this Plotly-Resampler demo](https://github.com/predict-idlab/plotly-resample
 
 * When running the code on a server, you should forward the port of the `FigureResampler.show_dash()` method to your local machine.<br>
   **Note** that you can add dynamic aggregation to plotly figures with the `FigureWidgetResampler` wrapper without needing to forward a port!
+* The `FigureWidgetResampler` *uses the IPython main thread* for its data aggregation functionality, so when this main thread is occupied, no resampling logic can be executed. For example; if you perform long computations within your notebook, the kernel will be occupied during these computations, and will only execute the resampling operations that take place during these computations after finishing that computation.
 * In general, when using downsampling one should be aware of (possible) [aliasing](https://en.wikipedia.org/wiki/Aliasing) effects.
   The <b style="color:orange">[R]</b> in the legend indicates when the corresponding trace is being resampled (and thus possibly distorted) or not. Additionally, the `~<range>` suffix represent the mean aggregation bin size in terms of the sequence index.
 * The plotly **autoscale** event (triggered by the autoscale button or a double-click within the graph), **does not reset the axes but autoscales the current graph-view** of plotly-resampler figures. This design choice was made as it seemed more intuitive for the developers to support this behavior with double-click than the default axes-reset behavior. The graph axes can ofcourse be resetted by using the `reset_axis` button.  If you want to give feedback and discuss this further with the developers, see issue [#49](https://github.com/predict-idlab/plotly-resampler/issues/49).

build.py

@@ -18,12 +18,13 @@
 def get_script_path():
     return os.path.dirname(os.path.realpath(sys.argv[0]))
 
+
 extensions = []
 if with_extensions:
     extensions = [
         Extension(
-            name="plotly_resampler.aggregation.algorithms.lttbcv2",
-            sources=["plotly_resampler/aggregation/algorithms/lttbcv2.c"],
+            name="plotly_resampler.aggregation.algorithms.lttbc",
+            sources=["plotly_resampler/aggregation/algorithms/lttbc.c"],
             define_macros=[("NPY_NO_DEPRECATED_API", "NPY_1_7_API_VERSION")],
             include_dirs=[np.get_include(), get_script_path()],
         ),
@@ -44,15 +45,26 @@ def run(self):
         try:
             build_ext.run(self)
         except (DistutilsPlatformError, FileNotFoundError) as e:
-            print("   Unable to build the C extensions.")
-            raise e
+            print(
+                "   Unable to build the C extensions, will use the slower python "
+                "fallback for LTTB"
+            )
+            print(e)
 
     def build_extension(self, ext):
         try:
             build_ext.build_extension(self, ext)
-        except (CCompilerError, DistutilsExecError, DistutilsPlatformError, ValueError) as e:
-            print('   Unable to build the "{}" C extension, '.format(ext.name))
-            raise e
+        except (
+            DistutilsPlatformError,
+            CCompilerError,
+            DistutilsExecError,
+            ValueError,
+        ) as e:
+            print(
+                '   Unable to build the "{}" C extension; '.format(ext.name)
+                + "will use the slower python fallback."
+            )
+            print(e)
 
 
 def build(setup_kwargs):

docs/sphinx/FAQ.rst

@@ -0,0 +1,162 @@
+.. role:: raw-html(raw)
+   :format: html
+
+.. |br| raw:: html
+
+   <br>
+
+
+FAQ ❓
+======
+
+.. raw:: html
+
+   <details>
+   <summary>
+      <a><b>What does the orange <b style="color:orange">~ time|number </b> suffix in legend name indicate?</b></a>
+   </summary>
+   <div style="margin-left:1em">
+
+
+This tilde suffix is only shown when the data is aggregated and represents the *mean aggregation bin size* which is the mean index-range difference between two consecutive aggregated samples.
+
+ * for *time-indexed data*: the mean time-range between 2 consecutive (sampled) samples.
+ * for *numeric-indexed data*: the mean numeric range between 2 consecutive (sampled) samples.
+
+When the index is a range-index; the *mean aggregation bin size* represents the *mean* downsample ratio; i.e., the mean number of samples that are aggregated into one sample.
+
+.. raw:: html
+
+   </div>
+   </details>
+   <br>
+   <details>
+   <summary>
+      <a><b>What is the difference between plotly-resampler figures and plain plotly figures?</b></a>
+   </summary>
+   <div style="margin-left:1em">
+
+plotly-resampler can be thought of as wrapper around plain plotly figures which adds line-chart visualization scalability by dynamically aggregating the data of the figures w.r.t. the front-end view. plotly-resampler thus adds dynamic aggregation functionality to plain plotly figures.
+
+**important to know**:
+
+* ``show`` *always* returns a static html view of the figure, i.e., no dynamic aggregation can be performed on that view.
+* To have dynamic aggregation:
+
+  * with ``FigureResampler``, you need to call ``show_dash`` (or output the object in a cell via ``IPython.display``) -> which spawns a dash-web app, and the dynamic aggregation is realized with dash callback
+  * with ``FigureWidgetResampler``, you need to use ``IPython.display`` on the object, which uses widget-events to realize dynamic aggregation (via the running IPython kernel).
+
+.. raw:: html
+
+   </div>
+   </details>
+   <br>
+   <details>
+   <summary>
+      <a><b>What does <code><a href="https://github.com/predict-idlab/trace-updater" target="_blank">TraceUpdater</a></code> do?</b></a>
+   </summary>
+   <div style="margin-left:1em">
+
+The ``TraceUpdater`` class is a custom dash component that aids ``dcc.Graph`` components to efficiently send and update (in our case aggregated) data to the front-end.
+
+For more information on how to use the trace-updater component together with the ``FigureResampler``, see our dash app `examples <https://github.com/predict-idlab/plotly-resampler/tree/main/examples>`_` and look at the `trace-updater <https://github.com/predict-idlab/trace-updater/blob/master/trace_updater/TraceUpdater.py>`_ its documentation.
+
+.. raw:: html
+
+   </div>
+   </details>
+   <br>
+      <details>
+   <summary>
+      <a><b>What is the difference in approach between plotly-resampler and datashader?</b></a>
+   </summary>
+   <div style="margin-left:1em">
+
+
+`Datashader <https://datashader.org/getting_started/Introduction.html>`_ is a highly scalable `open-source <https://github.com/holoviz/datashader>`_ library for analyzing and visualizing large datasets. More specifically, datashader *“rasterizes”* or *“aggregates”* datasets into regular grids that can be analyzed further or viewed as **images**. 
+
+
+**The main differences are**:
+
+Datashader can deal with various kinds of data (e.g., location related data, point clouds), whereas plotly-resampler is more tailored towards time-series data visualizations. 
+Furthermore, datashader outputs a **rasterized image/array** encompassing all traces their data, whereas plotly-resampler outputs an **aggregated series** per trace. Thus, datashader is more suited for analyzing data where you do not want to pin-out a certain series/trace.
+
+In our opinion, datashader truly shines (for the time series use case) when:
+
+* you want a global, overlaying view of all your traces
+* you want to visualize a large number of time series in a single plot (many traces)
+* there is a lot of noise on your high-frequency data and you want to uncover the underlying pattern
+* you want to render all data points in your visualization
+
+In our opinion, plotly-resampler shines when:
+
+* you need the capabilities to interact with the traces (e.g., hovering, toggling traces, hovertext per trace)
+* you want to use a less complex (but more restricted) visualization interface (as opposed to holoviews), i.e., plotly
+* you want to make existing plotly time-series figures more scalable and efficient
+* to build scalable Dash apps for time-series data visualization
+
+Furthermore combined with holoviews, datashader can also be employed in an interactive manner, see the example below.
+
+.. code:: python
+
+   from holoviews.operation.datashader import datashade
+   import datashader as ds
+   import holoviews as hv
+   import numpy as np
+   import pandas as pd
+   import panel as pn
+
+   hv.extension("bokeh")
+   pn.extension(comms='ipywidgets')
+
+   # Create the dummy dataframe
+   n = 1_000_000
+   x = np.arange(n)
+   noisy_sine = (np.sin(x / 3_000) + (np.random.randn(n) / 10)) * x / 5_000
+   df = pd.DataFrame(
+      {"ns": noisy_sine, "ns_abs": np.abs(noisy_sine),}
+   )
+
+   # Visualize interactively with datashader
+   opts = hv.opts.RGB(width=800, height=400)
+   ndoverlay = hv.NdOverlay({c:hv.Curve((df.index, df[c])) for c in df.columns})
+   datashade(ndoverlay, cnorm='linear', aggregator=ds.count(), line_width=3).opts(opts)
+
+.. image:: _static/datashader.png
+
+
+.. raw:: html
+
+   </div>
+   </details>
+   <br>
+   <details>
+   <summary>
+      <a><b>I get errors such as:</b><br><ul><li>
+         <code>RuntimeError: module compiled against API version 0x10 but this version of numpy is 0xe</code></li>  
+         <li><code>ImportError: numpy.core.multiarray failed to import</code></li>
+         </ul>
+      </a>
+   </summary>
+   <div style="margin-left:1em">
+
+   Plotly-resampler uses compiled C code (which uses the NumPy C API) to speed up the LTTB data-aggregation algorithm. This C code gets compiled during the building stage of the package (which might be before you install the package).<br><br>
+   If this C extension was build against a more recent NumPy version than your local version, you obtain a 
+   <a href="https://numpy.org/devdocs/user/troubleshooting-importerror.html#c-api-incompatibility"><i>NumPy C-API incompatibility</i></a> 
+   and the above error will be raised.<br><br>
+
+   These above mentioned errors can thus be resolved by running<br>
+   &nbsp;&nbsp;&nbsp;&nbsp;<code>pip install --upgrade numpy</code><br>
+   and reinstalling plotly-resampler afterwards.<br><br>
+
+   For more information about compatibility and building upon NumPy, you can consult 
+   <a href="https://numpy.org/doc/stable/user/depending_on_numpy.html?highlight=compiled#for-downstream-package-authors">NumPy's docs for downstream package authors</a>.
+
+   We aim to limit this issue as much as possible (by for example using <a href="https://github.com/scipy/oldest-supported-numpy">oldest-supported-numpy</a> in our build.py), 
+   but if you still experience issues, please open an issue on <a href="https://github.com/predict-idlab/plotly-resampler/issues">GitHub</a>.
+
+.. raw:: html
+
+   </div>
+   </details>
+   <br>

docs/sphinx/_static/annotate_twitter.gif

@@ -7,8 +7,8 @@
 
 
 
-Dash integration 🤝
-===================
+Dash apps 🤝
+============
 
 This documentation page describes how you can integrate ``plotly-resampler`` in a `dash <https://dash.plotly.com/>`_ application.
 
@@ -48,16 +48,19 @@ When you add a :class:`FigureResampler <plotly_resampler.figure_resampler.Figure
     fig.register_update_graph_callback(app, "graph-id", "trace-updater")
 
 
+.. warning::
+
+    The above example serves as an illustration, but uses a *global variable* to store the ``FigureResampler`` instance; this is not a good practice.
+    Ideally you should cache the ``FigureResampler`` per session on the server side.
+    In our `examples folder <https://github.com/predict-idlab/plotly-resampler/tree/main/examples>`_, we provide several dash app examples where we perform server side caching of such figures.
+
+
 .. tip::
 
     You can make the resampling faster by ensuring the
     `TraceUpdater <https://github.com/predict-idlab/trace-updater>`_ its
     ``sequentialUpdate`` argument is set to ``False``.
 
-
-* `This TraceUpdater-example <https://github.com/predict-idlab/trace-updater/blob/master/usage.py>`_ serves as a minimal working example.
-
-
 Limitations
 -----------
 ``plotly_resampler`` relies on `TraceUpdater <https://github.com/predict-idlab/trace-updater>`_ to ensure that the *updateData* is sent

docs/sphinx/_static/datashader.png

@@ -1,8 +1,8 @@
 .. role:: raw-html(raw)
    :format: html
 
-Getting started 🚀
-==================
+Get started 🚀
+==============
 
 ``plotly-resampler`` serves two main **modules**:
 
@@ -56,7 +56,7 @@ Dynamic resampling callbacks are realized:
 
     **To add dynamic resampling using a FigureWidget, you should**:
       1. wrap your plotly Figure (can be a ``go.Figure``) with :class:`FigureWidgetResampler <plotly_resampler.figure_resampler.FigureWidgetResampler>`
-      2. output the ```FigureWidgetResampler`` instance in a cell
+      2. output the ``FigureWidgetResampler`` instance in a cell
 
   .. tip::
 
@@ -114,6 +114,17 @@ The gif below demonstrates the example usage of of :class:`FigureWidgetResampler
 
 .. image:: https://raw.githubusercontent.com/predict-idlab/plotly-resampler/main/docs/sphinx/_static/figurewidget.gif
 
+
+.. raw:: html
+
+    <br><br>
+
+
+Furthermore, plotly's ``FigureWidget`` allows to conveniently add callbacks to for example click events. This allows creating a high-frequency time series annotation app in a couple of lines; as shown in the gif below and in `this notebook <https://github.com/predict-idlab/plotly-resampler/blob/main/examples/figurewidget_example.ipynb>`_.
+
+
+.. image:: _static/annotate_twitter.gif
+
 Important considerations & tips 🚨
 ----------------------------------
 
@@ -162,7 +173,7 @@ Working example ⬇️:
 .. tip::
 
     The ``FigureWidgetResampler`` graph will not be automatically redrawn after 
-    adjusting the fig its `hf_data` property,. The redrawning can be triggered by 
+    adjusting the fig its `hf_data` property,. The redrawing can be triggered by
     manually calling either:
 
     * :func:`FigureWidgetResampler.reload_data <plotly_resampler.figure_resampler.FigureWidgetResampler.reload_data>`, which keeps the current-graph range.

docs/sphinx/dash_app_integration.rst

@@ -40,8 +40,8 @@ As shown in the demo above, ``plotly-resampler`` maintains its interactiveness o
 
    getting_started
    dash_app_integration
-
    api_reference
+   FAQ