Merge pull request #20 from jrzaurin/wide_embedding

Wide embedding

Author: jrzaurin

README.md

@@ -5,6 +5,12 @@
 
 [![Build Status](https://travis-ci.org/jrzaurin/pytorch-widedeep.svg?branch=master)](https://travis-ci.org/jrzaurin/pytorch-widedeep)
 [![Documentation Status](https://readthedocs.org/projects/pytorch-widedeep/badge/?version=latest)](https://pytorch-widedeep.readthedocs.io/en/latest/?badge=latest)
+[![Maintenance](https://img.shields.io/badge/Maintained%3F-yes-green.svg)](https://github.com/jrzaurin/pytorch-widedeep/graphs/commit-activity)
+
+Platform | Version Support
+---------|:---------------
+OSX      | [![Python 3.6 3.7](https://img.shields.io/badge/python-3.6%20%203.7-blue.svg)](https://www.python.org/)
+Linux    | [![Python 3.6 3.7 3.8](https://img.shields.io/badge/python-3.6%20%203.7%203.8-blue.svg)](https://www.python.org/)
 
 # pytorch-widedeep
 
@@ -34,11 +40,11 @@ few lines of code.
   <img width="600" src="docs/figures/architecture_1.png">
 </p>
 
-Architecture 1 combines the `Wide`, one-hot encoded features with the outputs
-from the `DeepDense`, `DeepText` and `DeepImage` components connected to a
-final output neuron or neurons, depending on whether we are performing a
-binary classification or regression, or a multi-class classification. The
-components within the faded-pink rectangles are concatenated.
+Architecture 1 combines the `Wide`, Linear model with the outputs from the
+`DeepDense`, `DeepText` and `DeepImage` components connected to a final output
+neuron or neurons, depending on whether we are performing a binary
+classification or regression, or a multi-class classification. The components
+within the faded-pink rectangles are concatenated.
 
 In math terms, and following the notation in the
 [paper](https://arxiv.org/abs/1606.07792), Architecture 1 can be formulated
@@ -65,10 +71,10 @@ otherwise".*
   <img width="600" src="docs/figures/architecture_2.png">
 </p>
 
-Architecture 2 combines the `Wide` one-hot encoded features with the Deep
-components of the model connected to the output neuron(s), after the different
-Deep components have been themselves combined through a FC-Head (that I refer
-as `deephead`).
+Architecture 2 combines the `Wide`, Linear model with the Deep components of
+the model connected to the output neuron(s), after the different Deep
+components have been themselves combined through a FC-Head (that I refer as
+`deephead`).
 
 In math terms, and following the notation in the
 [paper](https://arxiv.org/abs/1606.07792), Architecture 2 can be formulated
@@ -84,7 +90,8 @@ and `DeepImage` are optional. `pytorch-widedeep` includes standard text (stack
 of LSTMs) and image (pre-trained ResNets or stack of CNNs) models. However,
 the user can use any custom model as long as it has an attribute called
 `output_dim` with the size of the last layer of activations, so that
-`WideDeep` can be constructed. See the examples folder for more information.
+`WideDeep` can be constructed. See the examples folder or the docs for more
+information.
 
 
 ### Installation
@@ -112,14 +119,6 @@ cd pytorch-widedeep
 pip install -e .
 ```
 
-### Examples
-
-There are a number of notebooks in the `examples` folder plus some additional
-files. These notebooks cover most of the utilities of this package and can
-also act as documentation. In the case that github does not render the
-notebooks, or it renders them missing some parts, they are saved as markdown
-files in the `docs` folder.
-
 ### Quick start
 
 Binary classification with the [adult
@@ -128,6 +127,7 @@ using `Wide` and `DeepDense` and defaults settings.
 
 ```python
 import pandas as pd
+import numpy as np
 from sklearn.model_selection import train_test_split
 
 from pytorch_widedeep.preprocessing import WidePreprocessor, DensePreprocessor
@@ -166,7 +166,7 @@ target = df_train[target_col].values
 # wide
 preprocess_wide = WidePreprocessor(wide_cols=wide_cols, crossed_cols=cross_cols)
 X_wide = preprocess_wide.fit_transform(df_train)
-wide = Wide(wide_dim=X_wide.shape[1], pred_dim=1)
+wide = Wide(wide_dim=np.unique(X_wide).shape[0], pred_dim=1)
 
 # deepdense
 preprocess_deep = DensePreprocessor(embed_cols=embed_cols, continuous_cols=cont_cols)

VERSION

@@ -1 +1 @@
-0.4.2
+0.4.5

docs/wide_deep/callbacks.rst renamed to docs/callbacks.rst

@@ -18,7 +18,10 @@ Documentation
     Utilities <utils/index>
     Preprocessing <preprocessing>
     Model Components <model_components>
-    Wide and Deep Models <wide_deep/index>
+    Metrics <metrics>
+    Callbacks <callbacks>
+    Focal Loss <losses>
+    Wide and Deep Models <wide_deep>
     Examples <examples>
 
 
@@ -45,12 +48,11 @@ Architectures
    :width: 600px
    :align: center
 
-Architecture 1 combines the ``Wide``, one-hot encoded features with the
-outputs from the ``DeepDense``, ``DeepText`` and ``DeepImage`` components
-connected to a final output neuron or neurons, depending on whether we are
-performing a binary classification or regression, or a multi-class
-classification. The components within the faded-pink rectangles are
-concatenated.
+Architecture 1 combines the `Wide`, Linear model with the outputs from the
+`DeepDense`, `DeepText` and `DeepImage` components connected to a final output
+neuron or neurons, depending on whether we are performing a binary
+classification or regression, or a multi-class classification. The components
+within the faded-pink rectangles are concatenated.
 
 In math terms, and following the notation in the `paper
 <https://arxiv.org/abs/1606.07792>`_, Architecture 1 can be formulated as:
@@ -76,10 +78,10 @@ is the activation function.
    :width: 600px
    :align: center
 
-Architecture 2 combines the ``Wide`` one-hot encoded features with the Deep
-components of the model connected to the output neuron(s), after the different
-Deep components have been themselves combined through a FC-Head (referred as
-as ``deephead``).
+Architecture 2 combines the `Wide`, Linear model with the Deep components of
+the model connected to the output neuron(s), after the different Deep
+components have been themselves combined through a FC-Head (that I refer as
+`deephead`).
 
 In math terms, and following the notation in the `paper
 <https://arxiv.org/abs/1606.07792>`_, Architecture 2 can be formulated as:

docs/figures/architecture_1.png

@@ -1,10 +1,9 @@
 The ``models`` module
-=====================
+======================
 
 This module contains the four main Wide and Deep model component. These are:
 ``Wide``, ``DeepDense``, ``DeepText`` and ``DeepImage``.
 
-
 .. autoclass:: pytorch_widedeep.models.wide.Wide
     :members:
     :undoc-members:

docs/figures/architecture_2.png

@@ -15,6 +15,7 @@ The following code snippet is not directly related to ``pytorch-widedeep``.
 .. code-block:: python
 
     import pandas as pd
+    import numpy as np
     from sklearn.model_selection import train_test_split
 
     df = pd.read_csv("data/adult/adult.csv.zip")
@@ -23,6 +24,7 @@ The following code snippet is not directly related to ``pytorch-widedeep``.
     df_train, df_test = train_test_split(df, test_size=0.2, stratify=df.income_label)
 
 
+
 Prepare the wide and deep columns
 ---------------------------------
 
@@ -63,7 +65,7 @@ Preprocessing and model components definition
     # wide
     preprocess_wide = WidePreprocessor(wide_cols=wide_cols, crossed_cols=cross_cols)
     X_wide = preprocess_wide.fit_transform(df_train)
-    wide = Wide(wide_dim=X_wide.shape[1], pred_dim=1)
+    wide = Wide(wide_dim=np.unique(X_wide).shape[0], pred_dim=1)
 
     # deepdense
     preprocess_deep = DensePreprocessor(embed_cols=embed_cols, continuous_cols=cont_cols)