Updated Docs for 21.06 (#165)

Author: Ashwin Ramesh

docs/cli.md

@@ -33,9 +33,28 @@ in which it is being run. Currently model analyzer supports 2 modes.
 
 ### Online Mode
 
-This is the default mode. When in this mode, Model Analyzer will operate to find the optimal model
-configuration for an online inference scenario. In this scenario, Triton server will receive requests
-on demand with an expectation that latency will be minimized.
+This is the default mode. When in this mode, Model Analyzer will operate to find
+the optimal model configuration for an online inference scenario. In this
+scenario, Triton server will receive requests on demand with an expectation that
+latency will be minimized.
+
+By default in online mode, the best model configuration will be the one that
+minimizes latency. If a latency budget is specified the best model configuration
+will be the one with the highest throughput in the given budget. The analyze and
+report subcommands also generate summaries specific to online inference. See the
+example [online summary](../examples/online_summary.pdf) and [detailed
+report](../examples/online_summary.pdf).
+
+### Offline Mode
+
+The offline mode `--mode=offline` tells Model Analyzer to set its defaults to
+find a model that maximizes throughput. In the offline scenario, Triton
+processes requests offline and therefore inference throughput is the priority. A
+minimum throughput can be specified using `--min-throughput` to ignore any
+configuration that does not exceed a minimum number of inferences per second.
+Both the summary and the detailed report will contain alternative graphs in the
+offline mode. See the [offline summary](../examples/offline_summary.pdf) and
+[detailed report](../examples/offline_detailed_report.pdf) examples.
 
 ## Model Analyzer Subcommands
 

docs/config.md

@@ -220,6 +220,9 @@ analysis_models: <comma-delimited-string-list>
 # Shorthand that allows a user to specify a max latency constraint in ms
 [ latency_budget: <int>]
 
+# Shorthand that allows a user to specify a min throughput constraint
+[ min_throughput: <int>]
+
 # Specify path to config yaml file
 [ config_file: <string> ]
 ```
@@ -563,6 +566,26 @@ and two instance group combinations). If both `model_config_parameters` and
 `parameters` keys are specified, the list of sweep configurations will be the
 cartesian product of both of the lists.
 
+### `<cpu_only>`
+
+This flag tells the model analyzer that, whether performing a search during profiling
+or generating reports, this model should use CPU instances only. In order to run a model on CPU only you must provide a value of `true` for this flag.
+
+#### Example
+
+```yaml
+model_repository: /path/to/model/repository/
+profile_models:
+  model_1:
+    cpu_only: true
+  model_2:
+    perf_analyzer_flags:
+    percentile: 95
+    latency_report_file: /path/to/latency/report/file
+```
+The above config tells model analyzer to profile `model_1` on CPU only,
+but profile `model_2` using GPU.
+
 ### `<perf-analyzer-flags>`
 
 This field allows fine-grained control over the behavior of the `perf_analyzer`

docs/report.md

@@ -32,7 +32,7 @@ $ model-analyzer analyze --analysis-models <list of model names> --checkpoint-di
 
 The export directory will, by default, contain 3 subdirectories. The summary
 report for a model will be located in `[export-path]/reports/summaries/<model
-name>`. The report will look like the one shown [*here*](../examples/summary.pdf).
+name>`. The report will look like the one shown [*here*](../examples/online_summary.pdf).
 
 To disable summary report generation use `--summarize=false` or set the
 `summarize` yaml option to `false`.
@@ -54,7 +54,7 @@ particular model config with which the measurements are obtained, as well as
 extra configurable plots. The user can define the plots they would like to see
 in the detailed report using the YAML config file (See [**Configuring Model
 Analyzer**](./config.md) section for more details) The detailed report will
-look like the one shown [*here*](../examples/detailed_report.pdf).
+look like the one shown [*here*](../examples/online_detailed_report.pdf).
 
 
 See the [**quick start**](./quick_start.md#plots) and [**configuring model