No public description

PiperOrigin-RevId: 590661941

Author: tensorflower-gardener

docs/nlp/customize_encoder.ipynb

@@ -70,7 +70,7 @@
       "source": [
         "## Learning objectives\n",
         "\n",
-        "The [TensorFlow Models NLP library](https://github.com/tensorflow/models/tree/master/official/nlp/modeling) is a collection of tools for building and training modern high-performance natural language models.\n",
+        "The [TensorFlow Models NLP library](https://github.com/tensorflow/models/tree/master/official/nlp/modeling) is a collection of tools for building and training modern high performance natural language models.\n",
         "\n",
         "The `tfm.nlp.networks.EncoderScaffold` is the core of this library, and lots of new network architectures are proposed to improve the encoder. In this Colab notebook, we will learn how to customize the encoder to employ new network architectures."
       ]
@@ -151,7 +151,7 @@
       "source": [
         "## Canonical BERT encoder\n",
         "\n",
-        "Before learning how to customize the encoder, let's first create a canonical BERT encoder and use it to instantiate a `bert_classifier.BertClassifier` for the classification task."
+        "Before learning how to customize the encoder, let's firstly create a canonical BERT enoder and use it to instantiate a `bert_classifier.BertClassifier` for classification task."
       ]
     },
     {
@@ -256,9 +256,9 @@
       "source": [
         "#### Without Customization\n",
         "\n",
-        "Without any customization, `networks.EncoderScaffold` behaves the same as the canonical `networks.BertEncoder`.\n",
+        "Without any customization, `networks.EncoderScaffold` behaves the same the canonical `networks.BertEncoder`.\n",
         "\n",
-        "As shown in the following example, `networks.EncoderScaffold` can load `networks.BertEncoder`'s weights and output are the same values:"
+        "As shown in the following example, `networks.EncoderScaffold` can load `networks.BertEncoder`'s weights and output the same values:"
       ]
     },
     {
@@ -564,7 +564,7 @@
         "id": "MeidDfhlHKSO"
       },
       "source": [
-        "Inspecting the `albert_encoder`, we see it stacks the same `Transformer` layer multiple times (note the loop-back on the \"Transformer\" block below."
+        "Inspecting the `albert_encoder`, we see it stacks the same `Transformer` layer multiple times (note the loop-back on the \"Transformer\" block below.."
       ]
     },
     {

official/projects/text_classification_example/README.md

@@ -44,7 +44,7 @@ class ClassificationDataLoader(data_loader.DataLoader):
   ...
 ```
 
-Overall, loader will translate the tf.Example to appropriate format for model to
+Overall, loader will translate the tf.Example to approiate format for model to
 consume. Then in Task.build_inputs, link the dataset like
 
 ```python
@@ -88,7 +88,7 @@ task_config = classification_example.ClassificationExampleConfig()
 task = classification_example.ClassificationExampleTask(task_config)
 ```
 
-TIPs: You can also check the [unittest](https://github.com/tensorflow/models/blob/master/official/projects/text_classification_example/classification_example_test.py)
+TIPs: You can also check the [unittest](https://github.com/tensorflow/models/blob/master/official/nlp/projects/example/classification_example_test.py)
 for better understanding.
 
 ### Finetune

official/recommendation/movielens.py

@@ -89,7 +89,7 @@
 
 
 def _download_and_clean(dataset, data_dir):
-  """Download the MovieLens dataset in a standard format.
+  """Download MovieLens dataset in a standard format.
 
   This function downloads the specified MovieLens format and coerces it into a
   standard format. The only difference between the ml-1m and ml-20m datasets
@@ -148,10 +148,10 @@ def _transform_csv(input_path, output_path, names, skip_first, separator=","):
 
   Args:
     input_path: The path of the raw csv.
-    output_path: The location of the cleaned csv file.
-    names: The names of the csv columns.
-    skip_first: Boolean indicating whether the first line of the raw csv should be skipped.
-    separator: A character used in raw csv to separate fields.
+    output_path: The path of the cleaned csv.
+    names: The csv column names.
+    skip_first: Boolean of whether to skip the first line of the raw csv.
+    separator: Character used to separate fields in the raw csv.
   """
   if six.PY2:
     names = [six.ensure_text(n, "utf-8") for n in names]
@@ -179,17 +179,17 @@ def _regularize_1m_dataset(temp_dir):
   ratings.dat
     The file has no header row, and each line is in the following format:
     UserID::MovieID::Rating::Timestamp
-      - UserIDs range between 1 and 6040
-      - MovieIDs can range between 1 and 3952
+      - UserIDs range from 1 and 6040
+      - MovieIDs range from 1 and 3952
       - Ratings are made on a 5-star scale (whole-star ratings only)
-      - Timestamp is represented in seconds since midnight. Coordinated Universal
+      - Timestamp is represented in seconds since midnight Coordinated Universal
         Time (UTC) of January 1, 1970.
       - Each user has at least 20 ratings
 
   movies.dat
     Each line has the following format:
     MovieID::Title::Genres
-      - MovieIDs can range between 1 and 3952
+      - MovieIDs range from 1 and 3952
   """
   working_dir = os.path.join(temp_dir, ML_1M)
 
@@ -223,7 +223,7 @@ def _regularize_20m_dataset(temp_dir):
   movies.csv
     Each line has the following format:
     MovieID,Title,Genres
-      - MovieIDs can range between 1 and 3952
+      - MovieIDs range from 1 and 3952
   """
   working_dir = os.path.join(temp_dir, ML_20M)
 
@@ -265,7 +265,7 @@ def csv_to_joint_dataframe(data_dir, dataset):
 
 
 def integerize_genres(dataframe):
-  """Replace the genre string with a binary vector.
+  """Replace genre string with a binary vector.
 
   Args:
     dataframe: a pandas dataframe of movie data.
@@ -308,7 +308,7 @@ def define_data_download_flags():
 
 
 def main(_):
-  """Download and extract the data from the GroupLens website."""
+  """Download and extract the data from GroupLens website."""
   download(flags.FLAGS.dataset, flags.FLAGS.data_dir)
 
 

official/recommendation/ncf_common.py

@@ -191,7 +191,7 @@ def define_ncf_flags():
       default=None,
       help=flags_core.help_wrap(
           "The batch size used for evaluation. This should generally be larger"
-          "than the training batch size, as the lack of back propagation during"
+          "than the training batch size as the lack of back propagation during"
           "evaluation can allow for larger batch sizes to fit in memory. If not"
           "specified, the training batch size (--batch_size) will be used."))
 
@@ -257,7 +257,7 @@ def define_ncf_flags():
           "If passed, training will stop when the evaluation metric HR is "
           "greater than or equal to hr_threshold. For dataset ml-1m, the "
           "desired hr_threshold is 0.68 which is the result from the paper; "
-          "For the dataset ml-20m, the threshold can be set as 0.95 which is "
+          "For dataset ml-20m, the threshold can be set as 0.95 which is "
           "achieved by MLPerf implementation."))
 
   flags.DEFINE_enum(
@@ -308,7 +308,7 @@ def define_ncf_flags():
           "If set, output the MLPerf compliance logging. This is only useful "
           "if one is running the model for MLPerf. See "
           "https://github.com/mlperf/policies/blob/master/training_rules.adoc"
-          "#submission-compliance-logs for details. This uses sudo, and so it may "
+          "#submission-compliance-logs for details. This uses sudo and so may "
           "ask for your password, as root access is needed to clear the system "
           "caches, which is required for MLPerf compliance."))
 

official/vision/modeling/heads/dense_prediction_heads.py

@@ -143,7 +143,18 @@ def __init__(
         'bias_initializer': tf.constant_initializer(-np.log((1 - 0.01) / 0.01)),
         'bias_regularizer': self._config_dict['bias_regularizer'],
     }
-    if not self._config_dict['use_separable_conv']:
+    if self._config_dict['use_separable_conv']:
+      self._classifier_kwargs.update({
+          'depthwise_initializer': tf_keras.initializers.RandomNormal(
+              stddev=0.03
+          ),
+          'depthwise_regularizer': self._config_dict['kernel_regularizer'],
+          'pointwise_initializer': tf_keras.initializers.RandomNormal(
+              stddev=0.03
+          ),
+          'pointwise_regularizer': self._config_dict['kernel_regularizer'],
+      })
+    else:
       self._classifier_kwargs.update({
           'kernel_initializer': tf_keras.initializers.RandomNormal(stddev=1e-5),
           'kernel_regularizer': self._config_dict['kernel_regularizer'],
@@ -159,7 +170,18 @@ def __init__(
         'bias_initializer': tf.zeros_initializer(),
         'bias_regularizer': self._config_dict['bias_regularizer'],
     }
-    if not self._config_dict['use_separable_conv']:
+    if self._config_dict['use_separable_conv']:
+      self._box_regressor_kwargs.update({
+          'depthwise_initializer': tf_keras.initializers.RandomNormal(
+              stddev=0.03
+          ),
+          'depthwise_regularizer': self._config_dict['kernel_regularizer'],
+          'pointwise_initializer': tf_keras.initializers.RandomNormal(
+              stddev=0.03
+          ),
+          'pointwise_regularizer': self._config_dict['kernel_regularizer'],
+      })
+    else:
       self._box_regressor_kwargs.update({
           'kernel_initializer': tf_keras.initializers.RandomNormal(stddev=1e-5),
           'kernel_regularizer': self._config_dict['kernel_regularizer'],

official/vision/train.py

@@ -74,7 +74,7 @@ def _run_experiment_with_preemption_recovery(params, model_dir):
         preemption_watcher.block_until_worker_exit()
         logging.info(
             'Some TPU workers had been preempted (message: %s), '
-            'restarting training from the last checkpoint...',
+            'retarting training from the last checkpoint...',
             preemption_watcher.preemption_message)
         keep_training = True
       else:

official/vision/utils/object_detection/argmax_matcher.py

@@ -16,14 +16,14 @@
 """Argmax matcher implementation.
 
 This class takes a similarity matrix and matches columns to rows based on the
-maximum value per column. One can specify matched_threshold and
+maximum value per column. One can specify matched_thresholds and
 to prevent columns from matching to rows (generally resulting in a negative
-training example) and unmatched_threshold to ignore the match (generally
-resulting in neither a positive nor a negative training example).
+training example) and unmatched_theshold to ignore the match (generally
+resulting in neither a positive or negative training example).
 
 This matcher is used in Fast(er)-RCNN.
 
-Note: Matchers are used in TargetAssigners. There is a create_target_assigner
+Note: matchers are used in TargetAssigners. There is a create_target_assigner
 factory function for popular implementations.
 """
 import tensorflow as tf, tf_keras
@@ -33,22 +33,22 @@
 
 
 class ArgMaxMatcher(matcher.Matcher):
-  """Matcher based on the highest value.
+  """Matcher based on highest value.
 
   This class computes matches from a similarity matrix. Each column is matched
   to a single row.
 
-  To support object detection target assignment, this class enables setting both
+  To support object detection target assignment this class enables setting both
   matched_threshold (upper threshold) and unmatched_threshold (lower threshold)
   defining three categories of similarity which define whether examples are
   positive, negative, or ignored:
   (1) similarity >= matched_threshold: Highest similarity. Matched/Positive!
   (2) matched_threshold > similarity >= unmatched_threshold: Medium similarity.
           Depending on negatives_lower_than_unmatched, this is either
           Unmatched/Negative OR Ignore.
-  (3) unmatched_threshold > similarity: Lowest similarity. Depending on the flag
+  (3) unmatched_threshold > similarity: Lowest similarity. Depending on flag
           negatives_lower_than_unmatched, either Unmatched/Negative or Ignore.
-  For ignored matches, this class sets the values in the Match object to -2.
+  For ignored matches this class sets the values in the Match object to -2.
   """
 
   def __init__(self,

official/vision/utils/object_detection/balanced_positive_negative_sampler.py

@@ -14,21 +14,21 @@
 
 """Class to subsample minibatches by balancing positives and negatives.
 
-Subsamples minibatches based on a pre-specified positive fraction in the range
+Subsamples minibatches based on a pre-specified positive fraction in range
 [0,1]. The class presumes there are many more negatives than positive examples:
 if the desired batch_size cannot be achieved with the pre-specified positive
 fraction, it fills the rest with negative examples. If this is not sufficient
 for obtaining the desired batch_size, it returns fewer examples.
 
-The main function to call is Subsample(self, indicator, labels). For convenience,
-one can also call SubsampleWeights(self, weights, labels), which is defined in
+The main function to call is Subsample(self, indicator, labels). For convenience
+one can also call SubsampleWeights(self, weights, labels) which is defined in
 the minibatch_sampler base class.
 
 When is_static is True, it implements a method that guarantees static shapes.
-It also ensures that the length of the output of the subsample is always batch_size, even
+It also ensures the length of output of the subsample is always batch_size, even
 when number of examples set to True in indicator is less than batch_size.
 
-This is originally implemented in the TensorFlow Object Detection API.
+This is originally implemented in TensorFlow Object Detection API.
 """
 
 import tensorflow as tf, tf_keras