Merge remote-tracking branch 'upstream/master'

Author: joshchang1112

RELEASE.md

@@ -1,3 +1,39 @@
+# Release 1.3.0
+
+## Major Features and Improvements
+
+*   Added locality-sensitive hashing (LSH) support to the graph builder tool.
+    This allows the graph builder to scale up to larger input datasets. As part
+    of this change, the new `nsl.configs.GraphBuilderConfig` class was
+    introduced, as well as a new `nsl.tools.build_graph_from_config` function.
+    The new parameters for controlling the LSH algorithm are named `lsh_rounds`
+    and `lsh_splits`.
+
+## Bug Fixes and Other Changes
+
+*   Fixed a bug in `nsl.tools.read_tsv_graph` that was incrementing the edge
+    count too often.
+*   Changed `nsl.tools.add_edge` to return a boolean result indicating if a new
+    edge was added or not; previously, this function was not returning any
+    value.
+*   Removed Python 2 unit tests.
+*   Fixed a bug in `nsl.estimator.add_adversarial_regularization` and
+    `nsl.estimator.add_graph_regularization` so that the `UPDATE_OPS` can be
+    triggered correctly.
+*   Updated graph-NSL tutorials not to parse neighbor features during
+    evaluation.
+*   Added scaled graph and adversarial loss values as scalars to the summary in
+    `nsl.estimator.add_graph_regularization` and
+    `nsl.estimator.add_adversarial_regularization` respectively.
+*   Updated graph and adversarial regularization loss metrics in
+    `nsl.keras.GraphRegularization` and `nsl.keras.AdversarialRegularization`
+    respectively, to include scaled values for consistency with their respective
+    loss term contributions.
+
+## Thanks to our Contributors
+
+This release contains contributions from many people at Google.
+
 # Release 1.2.0
 
 ## Major Features and Improvements

g3doc/tutorials/graph_keras_lstm_imdb.ipynb

@@ -1381,7 +1381,7 @@
         "acc = graph_reg_history_dict['accuracy']\n",
         "val_acc = graph_reg_history_dict['val_accuracy']\n",
         "loss = graph_reg_history_dict['loss']\n",
-        "graph_loss = graph_reg_history_dict['graph_loss']\n",
+        "graph_loss = graph_reg_history_dict['scaled_graph_loss']\n",
         "val_loss = graph_reg_history_dict['val_loss']\n",
         "\n",
         "epochs = range(1, len(acc) + 1)\n",

neural_structured_learning/keras/layers/layers_test.py

@@ -37,14 +37,17 @@ def _make_functional_regularized_model(distance_config):
   def _make_unregularized_model(inputs, num_classes):
     """Makes standard 1 layer MLP with logistic regression."""
     x = tf.keras.layers.Dense(16, activation='relu')(inputs)
-    return tf.keras.Model(inputs, outputs=tf.keras.layers.Dense(num_classes)(x))
+    model = tf.keras.Model(inputs, tf.keras.layers.Dense(num_classes)(x))
+    return model
 
   # Each example has 4 features and 2 neighbors, each with an edge weight.
   inputs = (tf.keras.Input(shape=(4,), dtype=tf.float32, name='features'),
             tf.keras.Input(shape=(2, 4), dtype=tf.float32, name='neighbors'),
             tf.keras.Input(
                 shape=(2, 1), dtype=tf.float32, name='neighbor_weights'))
   features, neighbors, neighbor_weights = inputs
+  neighbors = tf.reshape(neighbors, (-1,) + tuple(features.shape[1:]))
+  neighbor_weights = tf.reshape(neighbor_weights, [-1, 1])
   unregularized_model = _make_unregularized_model(features, 3)
   logits = unregularized_model(features)
   model = tf.keras.Model(inputs=inputs, outputs=logits)

neural_structured_learning/version.py

@@ -15,7 +15,7 @@
 
 # We follow Semantic Versioning (https://semver.org/).
 _MAJOR_VERSION = '1'
-_MINOR_VERSION = '2'
+_MINOR_VERSION = '3'
 _PATCH_VERSION = '0'
 
 _VERSION_SUFFIX = ''

workshops/kdd_2020/README.md

@@ -36,6 +36,8 @@ We will begin the tutorial with an overview of the Neural Structured Learning
 framework and motivate the advantages of training neural networks with
 structured signals.
 
+[[Slides](slides/Introduction.pdf)]
+
 ### Data preprocessing in NSL
 
 This part of the tutorial will include a presentation discussing:
@@ -44,6 +46,8 @@ This part of the tutorial will include a presentation discussing:
 -   Augmenting training data for graph-based regularization in NSL
 -   Related tools in the NSL framework
 
+[[Slides](slides/Data_Preprocessing.pdf)]
+
 ### Graph regularization using natural graphs (Lab 1)
 
 Graph regularization [2] forces neural networks to learn similar
@@ -53,6 +57,9 @@ inherent relationship between each other. We will demonstrate via a practical
 tutorial, the use of natural graphs for graph regularization to classify the
 veracity of public message posts.
 
+[[Slides](slides/Natural_Graphs.pdf)]
+[[Colab tutorial](https://colab.research.google.com/github/tensorflow/neural-structured-learning/blob/master/workshops/kdd_2020/graph_regularization_pheme_natural_graph.ipynb)]
+
 ### Graph regularization using synthesized graphs (Lab 2)
 
 Input data may not always be represented as a graph. However, one can infer
@@ -62,6 +69,9 @@ for text classification using a practical tutorial. While graphs can be built in
 many ways, we will make use of text embeddings in this tutorial to build a
 graph.
 
+[[Slides](slides/Synthesized_Graphs.pdf)]
+[[Colab tutorial](https://colab.research.google.com/github/tensorflow/neural-structured-learning/blob/master/g3doc/tutorials/graph_keras_lstm_imdb.ipynb)]
+
 ### Adversarial regularization (Lab 3)
 
 Adversarial learning has been shown to be effective in improving the accuracy of
@@ -70,11 +80,16 @@ adversarial learning techniques [3,4] like *Fast Gradient Sign Method* (FGSM)
 and *Projected Gradient Descent* (PGD) for image classification using a
 practical tutorial.
 
+[[Slides](slides/Adversarial_Learning.pdf)]
+[[Colab tutorial](https://colab.research.google.com/github/tensorflow/neural-structured-learning/blob/master/workshops/kdd_2020/adversarial_regularization_mnist.ipynb)]
+
 ### NSL in TensorFlow Extended (TFX)
 
 -   Presentation on how Neural Structured Learning can be integrated with
     [TFX](https://www.tensorflow.org/tfx) pipelines.
 
+[[Slides](slides/NSL_in_TFX.pdf)]
+
 ### Research and Future Directions
 
 -   Presentation discussing:
@@ -84,12 +99,16 @@ practical tutorial.
 -   Prototype showing how NSL can be used with the
     [Graph Nets](https://github.com/deepmind/graph_nets) [9] library.
 
+[[Slides](slides/Research_and_Future_Directions.pdf)]
+
 ### Conclusion
 
 We will conclude our tutorial with a summary of the entire session, provide
 links to various NSL resources, and share a link to a brief survey to get
 feedback on the NSL framework and the hands-on tutorial.
 
+[[Slides](slides/Summary.pdf)]
+
 ## References
 
 1.  https://www.tensorflow.org/neural_structured_learning