Merge pull request #115621 from likebupt/add-2-modules-0519

add 2 modules, group data into bins, convert word to vector

Author: ttorble

articles/machine-learning/algorithm-module-reference/convert-word-to-vector.md

@@ -0,0 +1,142 @@
+---
+title: "Convert Word to Vector"
+titleSuffix: Azure Machine Learning
+description: Learn how to use three provided Word2Vec models to extract a vocabulary and its corresponding word embeddings from a corpus of text.
+services: machine-learning
+ms.service: machine-learning
+ms.subservice: core
+ms.topic: reference
+
+author: likebupt
+ms.author: keli19
+ms.date: 05/19/2020
+---
+# Convert Word to Vector
+
+This article describes how to use the **Convert word to Vector** module in Azure Machine Learning designer (Preview), to apply various different Word2Vec models (Word2Vec, FastText, Glove pre-trained model) on the corpus of text that you specified as input, and generate a vocabulary with words embeddings.
+
+This module uses the Gensim library. For more information about Gensim, see its [official website](https://radimrehurek.com/gensim/apiref.html) that includes tutorials and explanation of algorithms.
+
+### More about Convert Word to Vector
+
+Generally speaking, converting word to vector, or word vectorization, is a natural language processing process, which uses language models or techniques to map words into vector space, that is, to represent each word by a vector of real numbers, and meanwhile, it allows words with similar meanings have similar representations.
+
+Word embeddings can be used as initial input for NLP downstream tasks such as text classification, sentiment analysis etc.
+
+Among various word embedding technologies, in this module, we implemented three widely used methods, including two online-training models, Word2Vec and FastText, and one pre-trained model, glove-wiki-gigaword-100. Online-training models are trained on your input data, while pre-trained models are trained off-line on a larger text corpus, (for example, Wikipedia, Google News) usually contains about 100 billion words, then, word embedding stays constant during word vectorization. Pre-trained word models provide benefits such as reduced training time, better word vectors encoded, and improved overall performance.
+
++ Word2Vec is one of the most popular techniques to learn word embeddings using shallow neural network, theory is discussed in this paper, available as a PDF download: [Efficient Estimation of Word Representations in Vector Space, Mikolov, Tomas, et al](https://arxiv.org/pdf/1301.3781.pdf). Implementation in this module is based on [gensim library for Word2Vec](https://radimrehurek.com/gensim/models/word2vec.html).
+
++ FastText theory is explained in this paper, available as a PDF download: [Enriching Word Vectors with Subword Information, Bojanowski, Piotr, et al](https://arxiv.org/pdf/1607.04606.pdf). Implementation in this module is based on [gensim library for FastText](https://radimrehurek.com/gensim/models/fasttext.html).
+
++ Glove pre-trained model: glove-wiki-gigaword-100, is a collection of pre-trained vectors based on Wikipedia text corpus, which contains 5.6B tokens and 400K uncased vocabulary, pdf is available: [GloVe: Global Vectors for Word Representation](https://nlp.stanford.edu/pubs/glove.pdf).
+
+## How to configure Convert Word to Vector
+
+This module requires a dataset that contains a column of text, preprocessed text is better.
+
+1. Add the **Convert Word to Vector** module to your pipeline.
+
+2. As input for the module, provide a dataset containing one or more text columns.
+
+3. For **Target column**, choose only one columns containing text to process.
+
+    In general, because this module creates a vocabulary from text, the content of different columns differs, which leads to different vocabulary contents, therefore, module only accept one target column.
+
+4. For  **Word2Vec strategy**, choose from `GloVe pretrained English Model`, `Gensim Word2Vec`, and `Gensim FastText`.
+
+5. if **Word2Vec strategy** is `Gensim Word2Vec` or `Gensim FastText`:
+
+    + **Word2Vec Training Algorithm**. Choose from `Skip_gram` and `CBOW`. Difference is introduced in the original [paper](https://arxiv.org/pdf/1301.3781.pdf).
+
+        Default method is `Skip_gram`.
+
+    + **Length of word embedding**. Specify the dimensionality of the word vectors. Corresponds to the `size` parameter in gensim.
+
+        Default embedding_size is 100.
+
+    + **Context window size**. Specify the maximum distance between the word being predicted and the current word. Corresponds to the `window` parameter in gensim.
+
+        Default window size is 5.
+
+    + **Number of epochs**. Specify the number of epochs (iterations) over the corpus. Corresponds to the `iter` parameter in gensim.
+
+        Default epochs number is 5.
+
+6. For **Maximum vocabulary size**, Specify the maximum number of the words in generated vocabulary.
+
+    If there are more unique words than this, then prune the infrequent ones.
+
+    Default vocabulary size is 10000.
+
+7. For **Minimum word count**, provide a minimum word count, which makes module ignores all words, which have a frequency lower than this value.
+
+    Default value is 5.
+
+8. Submit the pipeline.
+
+## Examples
+
+The module has one output:
+
++ **Vocabulary with embeddings**: Contains the generated vocabulary, together with each word's embedding, one dimension occupies one column.
+
+### Result examples
+
+To illustrate how the **Convert Word to Vector** module works, the following example applies this module with the default settings to the preprocessed Wikipedia SP 500 Dataset provided in Azure Machine Learning (preview).
+
+#### Source dataset
+
+The dataset contains a category column, as well as the full text fetched from Wikipedia. This table shows only a few representative examples.
+
+|text|
+|----------|
+|nasdaq 100 component s p 500 component foundation founder location city apple campus 1 infinite loop street infinite loop cupertino california cupertino california location country united states...|
+|br nasdaq 100 nasdaq 100 component br s p 500 s p 500 component industry computer software foundation br founder charles geschke br john warnock location adobe systems...|
+|s p 500 s p 500 component industry automotive industry automotive predecessor general motors corporation 1908 2009 successor...|
+|s p 500 s p 500 component industry conglomerate company conglomerate foundation founder location city fairfield connecticut fairfield connecticut location country usa area...|
+|br s p 500 s p 500 component foundation 1903 founder william s harley br arthur davidson harley davidson founder arthur davidson br walter davidson br william a davidson location...|
+
+#### Output vocabulary with embeddings
+
+The following table contains the output of this module taking Wikipedia SP 500 dataset as input. The leftmost column indicates the vocabulary, its embedding vector is represented by values of remaining columns in the same row.
+
+|Vocabulary|Embedding dim 0|Embedding dim 1|Embedding dim 2|Embedding dim 3|Embedding dim 4|Embedding dim 5|...|Embedding dim 99|
+|-------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|
+|nasdaq|-0.375865|0.609234|0.812797|-0.002236|0.319071|-0.591986|...|0.364276
+|component|0.081302|0.40001|0.121803|0.108181|0.043651|-0.091452|...|0.636587
+|s|-0.34355|-0.037092|-0.012167|0.151542|0.601019|0.084501|...|0.149419
+|p|-0.133407|0.073244|0.170396|0.326706|0.213463|-0.700355|...|0.530901
+foundation|-0.166819|0.10883|-0.07933|-0.073753|0.262137|0.045725|...|0.27487
+founder|-0.297408|0.493067|0.316709|-0.031651|0.455416|-0.284208|...|0.22798
+location|-0.375213|0.461229|0.310698|0.213465|0.200092|0.314288|...|0.14228
+city|-0.460828|0.505516|-0.074294|-0.00639|0.116545|0.494368|...|-0.2403
+apple|0.05779|0.672657|0.597267|-0.898889|0.099901|0.11833|...|0.4636
+campus|-0.281835|0.29312|0.106966|-0.031385|0.100777|-0.061452|...|0.05978
+infinite|-0.263074|0.245753|0.07058|-0.164666|0.162857|-0.027345|...|-0.0525
+loop|-0.391421|0.52366|0.141503|-0.105423|0.084503|-0.018424|...|-0.0521
+
+In this example, we used the default `Gensim Word2Vec` as the **Word2Vec strategy**, **Training Algorithm** is `Skip-gram`, **Length of word Embedding** is 100, therefore, we have 100 embedding columns.
+
+## Technical notes
+
+This section contains tips and answers to frequently asked questions.
+
++ Difference between online-train and pretrained model
+
+    In this **Convert word to Vector module**, we provided three different strategies, two online-training models, and one pre-trained model. Online-training model uses your input dataset as training data, generates vocabulary and word vectors during training, while pre-trained model is already trained by much larger text corpus such as Wikipedia or Twitter text, thus pre-trained model is actually a collection of (word, embedding) pair.  
+
+    If Glove pre-trained model is chosen as word vectorization strategy, it summarizes a vocabulary from the input dataset and generates embedding vector for each word from the pre-trained model, without online training, the use of pre-trained model could save training time, and has a better performance especially when the input dataset size is relatively small.
+
++ Embedding size
+
+    In general, the length of word embedding is set to a few hundred (for example, 100, 200, 300) to achieve good performance, because small embedding size means small vector space, which may cause word embedding collisions.  
+
+    For pretrained models, length of word embeddings are fixed, in this implementation, embedding size of glove-wiki-gigaword-100 is 100.
+
+
+## Next steps
+
+See the [set of modules available](module-reference.md) to Azure Machine Learning. 
+
+For a list of errors specific to the designer(preview) modules, see [Machine Learning Error codes](designer-error-codes.md).

articles/machine-learning/algorithm-module-reference/group-data-into-bins.md

@@ -0,0 +1,110 @@
+---
+title: "Group Data into Bins"
+titleSuffix: Azure Machine Learning
+description: Learn how to use the Group Data into Bins module to group numbers or change the distribution of continuous data.
+services: machine-learning
+ms.service: machine-learning
+ms.subservice: core
+ms.topic: reference
+
+author: likebupt
+ms.author: keli19
+ms.date: 05/19/2020
+---
+# Group Data into Bins
+
+This article describes how to use the [Group Data into Bins](group-data-into-bins.md) module in Azure Machine Learning designer (preview), to group numbers or change the distribution of continuous data.
+
+The [Group Data into Bins](group-data-into-bins.md) module supports multiple options for binning data. You can customize how the bin edges are set and how values are apportioned into the bins. For example, you can:  
+
++ Manually type a series of values to serve as the bin boundaries.  
++ Assign values to bins by using *quantiles*, or percentile ranks.  
++ Force an even distribution of values into the bins.  
+
+### More about binning and grouping
+
+*Binning* or  grouping data (sometimes called *quantization*) is an important tool in preparing numerical data for machine learning, and is useful in scenarios like these:
+
++ A column of continuous numbers has too many unique values to model effectively, so you automatically or manually assign the values to groups, to create a smaller set of discrete ranges.
+
++ Replace a column of numbers with categorical values that represent specific ranges.
+
+    For example, you might want to group values in an age column by specifying custom ranges, such as 1-15, 16-22, 23-30, and so forth for user demographics.
+
++ A dataset has a few extreme values, all well outside the expected range, and these values have an outsized influence on the trained model. To mitigate the bias in the model, you might transform the data to a uniform distribution, using the quantiles method.
+
+    With this method, the [Group Data into Bins](group-data-into-bins.md) module determines the ideal bin locations and bin widths to ensure that approximately the same number of samples fall into each bin. Then, depending on the normalization method you choose, the values in the bins are either transformed either to percentiles or mapped to a bin number.
+
+### Examples of binning
+
+The following diagram shows the distribution of numeric values before and after binning with the **quantiles** method. Notice that compared to the raw data at left, the data has been binned and transformed to a unit-normal scale.  
+
+'An example can be found from the result of this pipeline run: https://ml.azure.com/visualinterface/authoring/Normal/87270db9-4651-448e-bd28-8ef7428084dc?wsid=%2Fsubscriptions%2Fe9b2ec51-5c94-4fa8-809a-dc1e695e4896%2Fresourcegroups%2Fmodule-ws-rg%2Fworkspaces%2Fmodule-prerelease-119&flight=cm&tid=72f988bf-86f1-41af-91ab-2d7cd011db47&smtendpoint=https%3A%2F%2Fsmt-test1.azureml-test.net'
+
+Because there are so many ways to group data, all customizable, we recommend that you experiment with different methods and values. 
+
+## How to configure Group Data into Bins
+
+1. Add the **Group Data Into Bins** module to your pipeline in Designer (preview). You can find this module in the category **Data Transformation**.
+
+2. Connect the dataset that has numerical data to bin.  Quantization can be applied only to columns containing numeric data. 
+
+    If the dataset contains non-numeric columns, use the [Select Columns in Dataset](select-columns-in-dataset.md) module to select a subset of columns to work with.
+
+3. Specify the binning mode. The binning mode determines other parameters so be sure to select the **Binning mode** option first. The following types of binning are supported:
+
+    **Quantiles**: The quantile method assigns values to bins based on percentile ranks. Quantiles is also known as equal height binning.
+
+    **Equal Width**: With this option, you must specify the total number of bins. The values from the data column are placed in the bins such that each bin has the same interval between starting and end values. As a result, some bins might have more values if data is clumped around a certain point.
+
+    **Custom Edges**: You can specify the values that begin each bin. The edge value is always the lower boundary of the bin.  For example, assume you want to group values into two bins, one with values greater than 0, and one with values less than or equal to 0. In this case, for bin edges, you would type  0 in **Comma-separated list of bin edges**. The output of the module would be 1 and 2, indicating the bin index for each row value. Please note that the comma-separated value list must be in an ascending order, such as 1, 3, 5, 7.
+
+4. **Number of bins**: If you are using the **Quantiles**, and **Equal Width** binning modes, use this option to specify how many bins, or *quantiles*, that you want to create.
+
+5. For **Columns to bin**, use the Column Selector to choose the columns that have the values you want to bin. Columns must be a numeric data type.
+
+    The same binning rule is applied to all applicable columns that you choose. Therefore, if you need to bin some columns by using a different method, use a separate instance of [Group Data into Bins](group-data-into-bins.md) for each set of columns.
+
+    > [!WARNING]
+    > If you choose a column that is not an allowed type, a run-time error is generated. The module returns an error as soon as it finds any column of a disallowed type. If you get an error, review all selected columns. The error does not list all invalid columns.
+
+6. For **Output mode**, indicate how you want to output the quantized values.
+
+      + **Append**: Creates a new column with the binned values and appends that to the input table.
+
+      + **Inplace**: Replaces the original values with the new values in the dataset.
+
+      + **ResultOnly**: Returns just the result columns.
+
+7. If you select the **Quantiles** binning mode, use the **Quantile normalization** option to determine how values are normalized prior to sorting into quantiles. Note that normalizing values transforms the values, but does not affect the final number of bins.
+
+    The following normalization types are supported:
+
+    + **Percent**: Values are normalized within the range [0,100]
+
+    + **PQuantile**: Values are normalized within the range [0,1]
+
+    + **QuantileIndex**:  Values are normalized within the range [1,number of bins]
+
+8. If you choose the **Custom Edges** option, type a comma-separated list of numbers to use as *bin edges* in the + **Comma-separated list of bin edges** text box.  The values mark the point that divides bins, Therefore, if you type one bin edge value, two bins will be generated; if you type two bin edge values, three bins will be generated, and so forth.
+
+    The values must be sorted in order that the bins are created, from lowest to highest.
+
+10. **Tag columns as categorical**: Select this option to indicate that the quantized columns should be handled as categorical variables.
+
+11. Submit the pipeline.
+
+### Results
+
+The [Group Data into Bins](group-data-into-bins.md) module returns a dataset in which each element has been binned according to the specified mode. 
+
+It also returns a **Binning transformation**, which is a function that can be passed to the [Apply Transformation](apply-transformation.md) module to bin new samples of data using the same binning mode and parameters.  
+
+> [!TIP]
+> Remember, if you use binning on your training data, you must use the same binning method on data that you use for testing and prediction. This includes the binning method, bin locations, and bin widths. 
+> 
+> To ensure that data is always transformed by using the same binning method, we recommend that you save useful data transformations, and then apply them to other datasets, by using the [Apply Transformation](apply-transformation.md) module.
+
+## Next steps
+
+See the [set of modules available](module-reference.md) to Azure Machine Learning.