Merge branch 'main' of github.com:big-o/skdag into main

Author: big-o

doc/_static/img/cover.svg

@@ -5,7 +5,70 @@ scikit-dag (``skdag``) is an open-sourced, MIT-licenced library that provides ad
 workflow management to any machine learning operations that follow
 :mod:`sklearn` conventions. It does this by introducing Directed Acyclic
 Graphs (:class:`skdag.dag.DAG`) as a drop-in replacement for traditional scikit-learn
-:mod:`sklearn.pipeline.Pipeline`.
+:mod:`sklearn.pipeline.Pipeline`. This gives you a simple interface for a range of use
+cases including complex pre-processing, model stacking and benchmarking.
+
+.. code-block:: python
+
+   from skdag import DAGBuilder
+
+   dag = (
+      DAGBuilder()
+      .add_step("impute", SimpleImputer())
+      .add_step("vitals", "passthrough", deps={"impute": slice(0, 4)})
+      .add_step(
+         "blood",
+         PCA(n_components=2, random_state=0),
+         deps={"impute": slice(4, 10)}
+      )
+      .add_step(
+         "rf",
+         RandomForestRegressor(max_depth=5, random_state=0),
+         deps=["blood", "vitals"]
+      )
+      .add_step("svm", SVR(C=0.7), deps=["blood", "vitals"])
+      .add_step(
+         "knn",
+         KNeighborsRegressor(n_neighbors=5),
+         deps=["blood", "vitals"]
+      )
+      .add_step("meta", LinearRegression(), deps=["rf", "svm", "knn"])
+      .make_dag(n_jobs=2, verbose=True)
+   )
+
+   dag.show(detailed=True)
+
+.. image:: _static/img/cover.svg
+
+The above DAG imputes missing values, runs PCA on the columns relating to blood test
+results and leaves the other columns as they are. Then they get passed to three
+different regressors before being passed onto a final meta-estimator. Because DAGs
+(unlike pipelines) allow predictors in the middle or a workflow, you can use them to
+implement model stacking. We also chose to run the DAG steps in parallel wherever
+possible.
+
+After building our DAG, we can treat it as any other estimator:
+
+.. code-block:: python
+
+   from sklearn import datasets
+
+   X, y = datasets.load_diabetes(return_X_y=True, as_frame=True)
+   X_train, X_test, y_train, y_test = train_test_split(
+      X, y, test_size=0.2, random_state=0
+   )
+
+   dag.fit(X_train, y_train)
+   dag.predict(X_test)
+
+Just like a pipeline, you can optimise it with a gridsearch, pickle it etc.
+
+Note that this package does not deal with things like delayed dependencies and
+distributed architectures - consider an `established <https://airflow.apache.org/>`_
+`solution <https://dagster.io/>`_ for such use cases. ``skdag`` is just for building and
+executing local ensembles from estimators.
+
+:ref:`Read on<quickstart>` to learn more about ``skdag``...
 
 .. toctree::
    :maxdepth: 2

doc/index.rst

@@ -1,6 +1,8 @@
-#####################################
+.. _quickstart:
+
+######################
 Quick Start with skdag
-#####################################
+######################
 
 The following tutorial shows you how to write some simple directed acyclic graphs (DAGs)
 with ``skdag``.
@@ -21,8 +23,8 @@ to do this in Ubuntu:
 
     sudo apt install graphviz graphviz-dev
 
-Creating your own scikit-learn contribution package
-===================================================
+Creating a DAG
+==============
 
 The simplest DAGs are just a chain of singular dependencies. These DAGs may be
 created from the :meth:`skdag.dag.DAG.from_pipeline` method in the same way as a