Small fixes

Author: samtarver

articles/operator-insights/ingestion-agent-configuration-reference.md

@@ -153,9 +153,10 @@ source:
       # The path to a folder on the SFTP server that files will be uploaded to Azure Operator Insights from.
       base_path: /path/to/sftp/folder
       # Optional. A regular expression to specify which files in the base_path folder should be ingested. If not specified, the agent will attempt to ingest all files in the base_path folder (subject to exclude_pattern, settling_time and exclude_before_time).
-      include_pattern: "*\.csv$"
+      include_pattern: ".*\.csv$"
       # Optional. A regular expression to specify any files in the base_path folder which should not be ingested. Takes priority over include_pattern, so files which match both regular expressions will not be ingested.
-      exclude_pattern: '\.backup$'
+      # The exclude_pattern can also be used to ignore whole directories, but the pattern must still match all files under that directory. e.g. `^excluded-dir/.*$` or `^excluded-dir/` but *not* `^excluded-dir$`
+      exclude_pattern: "\.backup$|^\.staging/"
       # A duration, such as "10s", "5m", "1h".. During an upload run, any files last modified within the settling time are not selected for upload, as they may still be being modified.
       settling_time: 1m
       # Optional. A datetime that adheres to the RFC 3339 format. Any files last modified before this datetime will be ignored.

articles/operator-insights/monitor-troubleshoot-ingestion-agent.md

@@ -32,6 +32,8 @@ Metrics are reported in a simple human-friendly form.
 
 To collect a diagnostics package, SSH to the Virtual Machine and run the command `/usr/bin/microsoft/az-aoi-ingestion-gather-diags`. This command generates a date-stamped zip file in the current directory that you can copy from the system.
 
+If you have configued collection of logs through the Azure Monitor agent, you can view ingestion agent logs in the portal view of your Log Analytics workspace, and may not need to collect a diagnostics package to debug your issues.
+
 > [!NOTE]
 > Microsoft Support might request diagnostics packages when investigating an issue. Diagnostics packages don't contain any customer data or the value of any credentials.
 
@@ -118,6 +120,7 @@ Symptoms: No data appears in Azure Data Explorer. Logs of category `Ingestion` d
 - Check that the agent is running on all VMs and isn't reporting errors in logs.
 - Check that files exist in the correct location on the SFTP server, and that they aren't being excluded due to file source config (see [Files are missing](#files-are-missing)).
 - Check the network connectivity and firewall configuration between the ingestion agent VM and the Data Product's input storage account.
+- Ensure that the configured SFTP user can read all directories under the `base_path` which aren't being excluded by file source config.
 
 ### Files are missing
 

articles/operator-insights/set-up-ingestion-agent.md

@@ -139,6 +139,9 @@ On the SFTP server:
 
 1. Ensure port 22/TCP to the VM is open.
 1. Create a new user, or determine an existing user on the SFTP server that the ingestion agent should use to connect to the SFTP server.
+    - The ingestion agent searches every directory under the base path which is not excluded. Ensure this user has read permission on all files in the directories which are not excluded. 
+    > [!Note]
+    > Implicitly excluding directories by not specifying them in the included pattern is not sufficient to stop the agent searching those directories. See [the configuration reference](ingestion-agent-configuration-reference.md) for more detail on excluding directories.
 1. Determine the authentication method that the ingestion agent should use to connect to the SFTP server. The agent supports:
     - Password authentication
     - SSH key authentication
@@ -277,7 +280,12 @@ The configuration you need is specific to the type of source and your Data Produ
         - `user`: the name of the user on the SFTP server that the agent should use to connect.
         - Depending on the method of authentication you chose in [Prepare the VMs](#prepare-the-vms), set either `password` or `private_key`.
             - For password authentication, set `secret_name` to the name of the file containing the password in the `secrets_directory` folder. 
-            - For SSH key authentication, set `key_secret` to the name of the file containing the SSH key in the `secrets_directory` folder. If the private key is protected with a passphrase, set `passphrase_secret_name` to the name of the file containing the passphrase in the `secrets_directory` folder.
+            - For SSH key authentication, set `key_secret_name` to the name of the file containing the SSH key in the `secrets_directory` folder. If the private key is protected with a passphrase, set `passphrase_secret_name` to the name of the file containing the passphrase in the `secrets_directory` folder.
+            - All secret files should have permissions of `600` (`rw-------`), and an owner of `az-aoi-ingestion` so only the ingestion agent and privileged users can read them.
+            ```
+            sudo chmod 600 <secrets_directory>/*
+            sudo chown az-aoi-ingestion <secrets_directory>/*
+            ```
         
         For required or recommended values for other fields, refer to the documentation for your Data Product.
 
@@ -327,11 +335,12 @@ If you're running the ingestion agent on an Azure VM or on an on-premises VM con
 To collect ingestion agent logs, follow [the Azure Monitor documentation to install the Azure Monitor Agent and configure log collection](../azure-monitor/agents/data-collection-text-log.md).
 
 - These docs use the Az PowerShell module to create a logs table. Follow the [Az PowerShell module install documentation](/powershell/azure/install-azure-powershell) first.
-  - The `YourOptionalColumn` section from the sample `$tableParams` JSON is unnecessary for the ingestion agent, and can be removed.
+    - The `YourOptionalColumn` section from the sample `$tableParams` JSON is unnecessary for the ingestion agent, and can be removed.
 - When adding a data source to your data collection rule, add a `Custom Text Logs` source type, with file pattern `/var/log/az-aoi-ingestion/stdout.log`.
-- After adding the data collection rule, you can query these logs through the Log Analytics workspace. Use the following query to make them easier to work with:
+- We also recommend following [the documentation to add a `Linux Syslog` Data source](../azure-monitor/agents/data-collection-syslog.md) to your data collection rule, to allow for auditing of all processes running on the VM.
+- After adding the data collection rule, you can query the ingestion agent logs through the Log Analytics workspace. Use the following query to make them easier to work with:
   ```
-  RawAgentLogs_CL
+  <CustomTableName>_CL
   | extend RawData = replace_regex(RawData, '\\x1b\\[\\d{1,4}m', '')  // Remove any color tags
   | parse RawData with TimeGenerated: datetime '  ' Level ' ' Message  // Parse the log lines into the TimeGenerated, Level and Message columns for easy filtering
   | order by TimeGenerated desc