Skip to content

Commit e721806

Browse files
committed
DOCS-1266 - Token auth for HTTP sources - GA
1 parent a236085 commit e721806

File tree

5 files changed

+20
-159
lines changed

5 files changed

+20
-159
lines changed
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
---
2+
title: Token-Based Authentication for Secure HTTPS Logs and Metric Source (Collection)
3+
image: https://assets-www.sumologic.com/company-logos/_800x418_crop_center-center_82_none/SumoLogic_Preview_600x600.jpg?mtime=1617040082
4+
keywords:
5+
- http source
6+
- authentication
7+
hide_table_of_contents: true
8+
---
9+
10+
import useBaseUrl from '@docusaurus/useBaseUrl';
11+
12+
We're excited to introduce secure token-based authentication for the HTTP Logs and Metrics Source. This new capability allows you to authenticate using a unique token in the request header, maintaining the existing HTTPS endpoint behavior while adding token validation per source.
13+
14+
Obtain the token to use in an auth header when you configure an HTTP source or regenerate the URL. To learn more, see:
15+
* [Configure an HTTP Logs and Metrics Source](/docs/send-data/hosted-collectors/http-source/logs-metrics/#configure-an-httplogs-and-metrics-source)
16+
* [Generate a New URL for an HTTP Source](/docs/send-data/hosted-collectors/http-source/generate-new-url/)

docs/send-data/hosted-collectors/http-source/generate-new-url.md

Lines changed: 0 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -6,22 +6,6 @@ sidebar_label: Regenerate URL
66

77
import useBaseUrl from '@docusaurus/useBaseUrl';
88

9-
You can generate a new URL for an HTTP Source at any time. Generating a new URL completely invalidates the old URL.
10-
11-
To generate a new URL:
12-
13-
1. [**New UI**](/docs/get-started/sumo-logic-ui). In the Sumo Logic main menu select **Data Management**, and then under **Data Collection** select **Collection**. You can also click the **Go To...** menu at the top of the screen and select **Collection**.  <br/>[**Classic UI**](/docs/get-started/sumo-logic-ui-classic). In the main Sumo Logic menu, select **Manage Data > Collection > Collection**.
14-
1. On the **Manage Collection** page, click **Regenerate URL** next to the HTTP source.<br/><img src={useBaseUrl('img/send-data/regenerate-url.png')} alt="Regenerate URL link" style={{border: '1px solid gray'}} width="800" />
15-
1. In the **HTTP Source Address** dialog box, click **Generate**.<br/> <img src={useBaseUrl('img/send-data/generate-new-url.png')} alt="http_source_dialog_generate.png" width="350"/>
16-
1. When asked to confirm the generation, click **OK**.
17-
1. In the **HTTP Source Address** dialog box, the new URL is displayed. Copy and paste the URL, then click **OK**.<br/> <img src={useBaseUrl('img/send-data/http_source_dialog_with_button.png')} alt="http_source_dialog_generate.png" width="350"/>
18-
19-
:::note
20-
If you see a 401 (failed to authenticate) error message right after generating a new URL, wait a few minutes, then try the new URL again.
21-
:::
22-
23-
<!-- Per DCOS-1266, for GA replace the contents of this article with the following:
24-
259
You can generate a new URL for an HTTP Source at any time. Generating a new URL completely invalidates the old URL.
2610

2711
Sumo Logic offers secure token-based authentication for the HTTPS Logs and Metrics Source. This capability allows you to authenticate using a unique token in the request header, maintaining the existing HTTPS endpoint behavior while adding token validation per source. Obtain the token to use in an auth header when you configure an HTTP source or regenerate the URL.
@@ -41,5 +25,3 @@ To generate a new URL:
4125
:::note
4226
If you see a 401 (failed to authenticate) error message right after generating a new URL, wait a few minutes, then try the new URL again.
4327
:::
44-
45-
-->

docs/send-data/hosted-collectors/http-source/logs-metrics/index.md

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -53,15 +53,11 @@ To configure an HTTP Logs and Metrics Source:
5353
* **One Message Per Request.** Select this option if you'll be sending a single message with each HTTP request. For more information, see [Multiline options in HTTP sources](#multiline-options-in-http-sources)
5454
1. **Processing Rules.** Configure any desired filters, such as allowlist, denylist, hash, or mask, as described in Create a Processing Rule. Processing rules are applied to log data, but not to metric data.
5555
1. When you are finished configuring the Source, click **Save**.
56-
1. When the URL associated with the source is displayed, copy the URL so you can use it to upload data.<br/><img src={useBaseUrl('img/send-data/http-source-address.png')} alt="HTTP Source Address dialog " style={{border: '1px solid gray'}} width="500" />
57-
58-
<!-- Per DOCS-1266, replace the preceding step with the following:
5956
1. In the **HTTP Source Address** dialog box, select one of the following to copy the URL where the source data will be stored:
6057
* **Presigned URL**. Select to copy a presigned URL with embedded authentication.<br/><img src={useBaseUrl('img/send-data/http-source-address-new.png')} alt="HTTP Source Address with presigned URL" style={{border: '1px solid gray'}} width="600"/>
6158
* **Auth Header**. Select to copy the URL, as well as a separate authorization header that contains an authentication token. This option provides greater security than a presigned URL because placing the authentication token in the authorization header of a request prevents the token from being exposed in the URL.<br/><img src={useBaseUrl('img/send-data/http-source-address-and-auth-header.png')} alt="HTTP Source Address with authorization header" style={{border: '1px solid gray'}} width="600"/>
6259
1. Copy the URL (and header if applicable) and keep in a safe place. You will use the URL in the next step: [Upload data to the HTTP Logs and Metrics Source](#upload-data-to-the-httplogs-and-metrics-source).
6360
1. Click **Done**.
64-
-->
6561

6662
:::note
6763
* Metrics reported with a timestamp older than 24 hours ago or newer than 24 hours in the future from the time they are reported are dropped. Make sure that the Metrics sent to HTTP Endpoint have appropriate timestamps.

docs/send-data/hosted-collectors/http-source/logs-metrics/upload-logs.md

Lines changed: 4 additions & 74 deletions
Original file line numberDiff line numberDiff line change
@@ -29,21 +29,15 @@ We recommend that the POST data payload have a size, before compression, of 100K
2929
* Data line 2
3030
* Data line 3
3131
* Method: POST
32-
* URL: `https://[SumoEndpoint]/receiver/v1/http/[UniqueHTTPCollectorCode]` where
33-
* [SumoEndpoint] is your Sumo collection endpoint
34-
* [UniqueHTTPCollectorCode] is the string that follows the last forward slash (`/`) in the upload URL for the HTTP source
35-
* Body
36-
* Data line 1
37-
* Data line 2
38-
* Data line 3
39-
40-
<!-- Per DOCS-1266, replace the preceding "URL" bullet with the following:
4132
* URL: <br/>Enter the URL obtained when you [configure the HTTP Logs and Metrics Source](/docs/send-data/hosted-collectors/http-source/logs-metrics/#configure-an-httplogs-and-metrics-source) or when you [regenerate the URL](/docs/send-data/hosted-collectors/http-source/generate-new-url/). Enter either a presigned URL or a URL to be used with an auth header:
4233
* Presigned URL: `https://[SumoEndpoint]/receiver/v1/http/[UniqueHTTPCollectorCode]` <br/>where
4334
* [SumoEndpoint] is your Sumo collection endpoint
4435
* [UniqueHTTPCollectorCode] is the string that follows the last forward slash (`/`) in the upload URL for the HTTP source
4536
* URL used with auth header: `https://[SumoEndpoint]/receiver/v1/http` <br/>where [SumoEndpoint] is your Sumo collection endpoint
46-
-->
37+
* Body
38+
* Data line 1
39+
* Data line 2
40+
* Data line 3
4741

4842
## Supported HTTP Headers
4943

@@ -62,10 +56,7 @@ Overridden metadata field values are not returned with [Search Autocomplete](/do
6256
| Custom Source Host | `X-Sumo-Host` | Desired host name.<br/>Useful if you want to override the source host configured for the source. |
6357
| Custom Source Category | `X-Sumo-Category` | Desired source category.<br/>Useful if you want to override the source category configured for the source. |
6458
| Fields as custom metadata | `X-Sumo-Fields` | [Fields](/docs/manage/fields) need to be in a comma separated list of key-value pairs. |
65-
66-
<!-- Per DOCS-1266, add this row to the preceding table:
6759
| Token authentication | `x-sumo-token` | Token to be used for authentication in an authorization header. Obtain the token when you select **Auth Header** when you [configure the HTTP Logs and Metrics Source](/docs/send-data/hosted-collectors/http-source/logs-metrics/#configure-an-httplogs-and-metrics-source), or when you [regenerate the URL](/docs/send-data/hosted-collectors/http-source/generate-new-url/). |
68-
-->
6960

7061
## Command line examples
7162

@@ -77,65 +68,6 @@ Data is ingested according to the configured [processing rules](/docs/send-dat
7768

7869
When using cURL to POST data from a file: 
7970

80-
* Make sure to use the -T parameter to specify the file path, not -d. The -d parameter causes new lines to be removed from the content, which will interfere with message boundary detection.
81-
* Make sure that each line in the file follows the format specified by the Content-Type header for the HTTP request.
82-
83-
**POST upload**
84-
85-
```bash
86-
curl -v -X POST -T [local_file_name] https://collectors.sumologic.com/receiver/v1/http/[UniqueHTTPCollectorCode]
87-
```
88-
89-
**POST upload (gzip compressed data)** 
90-
91-
```bash
92-
curl -v -X POST -H 'Content-Encoding:gzip' -T [local_file_name.gz] https://collectors.sumologic.com/receiver/v1/http/[UniqueHTTPCollectorCode]
93-
```
94-
95-
**POST upload with custom Source Category**
96-
97-
```bash
98-
curl -v -X POST -H 'X-Sumo-Category:myNewCategory' -T [local_file_name] https://collectors.sumologic.com/receiver/v1/http/[UniqueHTTPCollectorCode]
99-
```
100-
101-
**POST upload with custom Fields**
102-
103-
```bash
104-
curl -v -X POST -H 'X-Sumo-Fields:environment=dev,cluster=k8s' -T [local_file_name] https://collectors.sumologic.com/receiver/v1/http/[UniqueHTTPCollectorCode]
105-
```
106-
107-
### PowerShell
108-
109-
**POST upload**
110-
111-
```bash
112-
Invoke-WebRequest -Method POST -InFile [local_file_name] 'https://collectors.sumologic.com/receiver/v1/http/[UniqueHTTPCollectorCode]'
113-
```
114-
115-
**POST upload (gzip compressed data)** 
116-
117-
```bash
118-
Invoke-WebRequest -Method POST -Headers @{'Content-Encoding' = 'gzip'} -InFile [local_file_name.gz] 'https://collectors.sumologic.com/receiver/v1/http/[UniqueHTTPCollectorCode]'
119-
```
120-
121-
**POST upload with custom Source Category**
122-
123-
```bash
124-
Invoke-WebRequest -Method POST -Headers @{'X-Sumo-Category' = 'myCustomCategory'} -InFile [local_file_name] 'https://collectors.sumologic.com/receiver/v1/http/[UniqueHTTPCollectorCode]'
125-
```
126-
127-
**POST upload with custom Field**
128-
129-
```bash
130-
Invoke-WebRequest -Method POST -Headers @{'X-Sumo-Fields' = 'environment=dev'} -InFile [local_file_name] 'https://collectors.sumologic.com/receiver/v1/http/[UniqueHTTPCollectorCode]'
131-
```
132-
133-
<!-- Per DOCS-1266, repleace the cURL and Powershell sections with the following:
134-
135-
### cURL
136-
137-
When using cURL to POST data from a file: 
138-
13971
* Make sure to use the -T parameter to specify the file path, not -d. The -d parameter causes new lines to be removed from the content, which will interfere with message boundary detection.
14072
* Make sure that each line in the file follows the format specified by the Content-Type header for the HTTP request.
14173
* Enter the URL (and auth header if applicable) obtained when you [configured the HTTP Logs and Metrics Source](/docs/send-data/hosted-collectors/http-source/logs-metrics/#configure-an-httplogs-and-metrics-source) or when you [regenerate the URL](/docs/send-data/hosted-collectors/http-source/generate-new-url/). If you use an auth header, enter it in this format: <br/>`-H "x-sumo-token: [TokenString]"`
@@ -244,5 +176,3 @@ URL with auth header:
244176
```bash
245177
Invoke-WebRequest -Method POST -Headers @{'X-Sumo-Fields' = 'environment=dev'} -InFile [local_file_name] 'https://collectors.sumologic.com/receiver/v1/http' -Headers $headers
246178
```
247-
248-
-->

docs/send-data/hosted-collectors/http-source/logs-metrics/upload-metrics.md

Lines changed: 0 additions & 63 deletions
Original file line numberDiff line numberDiff line change
@@ -93,10 +93,7 @@ Overridden metadata field values are not returned with [Search Autocomplete](/do
9393
| Custom Source Category | `X-Sumo-Category` | Desired source category.<br/>Useful if you want to override the source category configured for the source. |
9494
| Custom Metric Dimensions | `X-Sumo-Dimensions` | Comma-separated key=value list of dimensions to apply to every metric.<br/>For metrics only. Custom dimensions will allow you to query your metrics at a more granular level. |
9595
| Custom Metric Metadata | `X-Sumo-Metadata` | Comma-separated, key=value list of metadata to apply to every metric.<br/>For metrics only. Custom metadata will allow you to query your metrics at a more granular level. |
96-
97-
<!-- Per DOCS-1266, add this row to the table:
9896
| Token authentication | `x-sumo-token` | Token to be used for authentication in an authorization header. Obtain the token when you select **Auth Header** when you [configure the HTTP Logs and Metrics Source](/docs/send-data/hosted-collectors/http-source/logs-metrics/#configure-an-httplogs-and-metrics-source), or when you [regenerate the URL](/docs/send-data/hosted-collectors/http-source/generate-new-url/). |
99-
-->
10097

10198
## Data volume and metadata limits for metrics
10299

@@ -158,66 +155,6 @@ URL with auth header:
158155
curl -v -X POST -H 'Content-Type:application/vnd.sumologic.prometheus' -H "x-sumo-token: [TokenString]" -T [local_file_name] http://collectors.sumologic.com/receiver/v1/http
159156
```
160157

161-
<!-- Per DOCS-1266, replace the preceding section with this:
162-
163-
## Command line examples using cURL
164-
165-
When using cURL to POST data from a file: 
166-
167-
* Make sure to use the `-T` parameter to specify the file path, not `-d`. The `-d` parameter causes new lines to be removed from the content, which will interfere with message boundary detection.
168-
* Make sure that each line in the file follows the format specified by the `Content-Type` header for the HTTP request.
169-
* Enter the URL (and auth header if applicable) obtained when you [configured the HTTP Logs and Metrics Source](/docs/send-data/hosted-collectors/http-source/logs-metrics/#configure-an-httplogs-and-metrics-source) or when you [regenerate the URL](/docs/send-data/hosted-collectors/http-source/generate-new-url/). If you use an auth header, enter it in this format: <br/>`-H "x-sumo-token: [TokenString]"`
170-
171-
**POST upload ([Graphite](http://metrics20.org/implementations/)-formatted metrics)**
172-
173-
Presigned URL:
174-
```bash
175-
curl -v -X POST -H 'Content-Type:application/vnd.sumologic.graphite' -T [local_file_name] https://collectors.sumologic.com/receiver/v1/http/[UniqueHTTPCollectorCode]
176-
```
177-
178-
URL with auth header:
179-
```bash
180-
curl -v -X POST -H 'Content-Type:application/vnd.sumologic.graphite' -H "x-sumo-token: [TokenString]" -T [local_file_name] https://collectors.sumologic.com/receiver/v1/http
181-
```
182-
183-
**POST upload ([Carbon 2.0](http://graphite.readthedocs.io/en/latest/feeding-carbon.html#the-plaintext-protocol)-formatted metrics)**
184-
185-
Presigned URL:
186-
```bash
187-
curl -v -X POST -H 'Content-Type:application/vnd.sumologic.carbon2' -T [local_file_name] https://collectors.sumologic.com/receiver/v1/http/[UniqueHTTPCollectorCode]
188-
```
189-
190-
URL with auth header:
191-
```bash
192-
curl -v -X POST -H 'Content-Type:application/vnd.sumologic.carbon2' -H "x-sumo-token: [TokenString]" -T [local_file_name] https://collectors.sumologic.com/receiver/v1/http
193-
```
194-
195-
**POST upload (gzip compressed [Graphite](http://metrics20.org/implementations/)-formatted metrics)** 
196-
197-
Presigned URL:
198-
```bash
199-
curl -v -X POST -H 'Content-Encoding:gzip' -H 'Content-Type:application/vnd.sumologic.graphite' -T [local_file_name.gz] https://collectors.sumologic.com/receiver/v1/http/[UniqueHTTPCollectorCode]
200-
```
201-
202-
URL with auth header:
203-
```bash
204-
curl -v -X POST -H 'Content-Encoding:gzip' -H 'Content-Type:application/vnd.sumologic.graphite' -H "x-sumo-token: [TokenString]" -T [local_file_name.gz] https://collectors.sumologic.com/receiver/v1/http
205-
```
206-
207-
**POST upload ([Prometheus](https://github.com/prometheus/docs/blob/master/content/docs/instrumenting/exposition_formats.md)-formatted metrics)**
208-
209-
Presigned URL:
210-
```bash
211-
curl -v -X POST -H 'Content-Type:application/vnd.sumologic.prometheus' -T [local_file_name] http://collectors.sumologic.com/receiver/v1/http/[UniqueHTTPCollectorCode]
212-
```
213-
214-
URL with auth header:
215-
```bash
216-
curl -v -X POST -H 'Content-Type:application/vnd.sumologic.prometheus' -H "x-sumo-token: [TokenString]" -T [local_file_name] http://collectors.sumologic.com/receiver/v1/http
217-
```
218-
219-
-->
220-
221158
## Prometheus Metrics Not Accepted by Sumo
222159

223160
By design, Sumo does not ingest Prometheus comments. Sumo also rejects Prometheus metrics that do not conform to the Prometheus metric format. This page lists the conditions that will cause Sumo to reject Prometheus metrics

0 commit comments

Comments
 (0)