Skip to content

Commit aaad37b

Browse files
authored
Merge pull request #6296 from influxdata:jts-dar-522-3dist-partial-write-errors
Jts-dar-522-3dist-partial-write-errors Improve write data troubleshooting for influxdb3 distributed
2 parents fbfa1e1 + 81e86e3 commit aaad37b

File tree

4 files changed

+366
-254
lines changed

4 files changed

+366
-254
lines changed

content/influxdb3/cloud-dedicated/write-data/troubleshoot.md

Lines changed: 6 additions & 92 deletions
Original file line numberDiff line numberDiff line change
@@ -10,101 +10,15 @@ menu:
1010
influxdb3_cloud_dedicated:
1111
name: Troubleshoot issues
1212
parent: Write data
13-
influxdb3/cloud-dedicated/tags: [write, line protocol, errors]
13+
influxdb3/cloud-dedicated/tags: [write, line protocol, errors, partial writes]
1414
related:
15+
- /influxdb3/cloud-dedicated/get-started/write/
1516
- /influxdb3/cloud-dedicated/reference/syntax/line-protocol/
1617
- /influxdb3/cloud-dedicated/write-data/best-practices/
1718
- /influxdb3/cloud-dedicated/reference/internals/durability/
19+
source: /shared/influxdb3-write-guides/troubleshoot-distributed.md
1820
---
1921

20-
Learn how to avoid unexpected results and recover from errors when writing to {{% product-name %}}.
21-
22-
- [Handle write responses](#handle-write-responses)
23-
- [Review HTTP status codes](#review-http-status-codes)
24-
- [Troubleshoot failures](#troubleshoot-failures)
25-
- [Troubleshoot rejected points](#troubleshoot-rejected-points)
26-
27-
## Handle write responses
28-
29-
{{% product-name %}} does the following when you send a write request:
30-
31-
1. Validates the request.
32-
2. If successful, attempts to [ingest data](/influxdb3/cloud-dedicated/reference/internals/durability/#data-ingest) from the request body; otherwise, responds with an [error status](#review-http-status-codes).
33-
3. Ingests or rejects data in the batch and returns one of the following HTTP status codes:
34-
35-
- `204 No Content`: All data in the batch is ingested.
36-
- `400 Bad Request`: Some (_when **partial writes** are configured for the cluster_) or all of the data has been rejected. Data that has not been rejected is ingested and queryable.
37-
38-
The response body contains error details about [rejected points](#troubleshoot-rejected-points), up to 100 points.
39-
40-
Writes are synchronous--the response status indicates the final status of the write and all ingested data is queryable.
41-
42-
To ensure that InfluxDB handles writes in the order you request them,
43-
wait for the response before you send the next request.
44-
45-
### Review HTTP status codes
46-
47-
InfluxDB uses conventional HTTP status codes to indicate the success or failure of a request.
48-
The `message` property of the response body may contain additional details about the error.
49-
{{< product-name >}} returns one the following HTTP status codes for a write request:
50-
51-
| HTTP response code | Response body | Description |
52-
|:------------------------------|:------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
53-
| `204 No Content"` | no response body | If InfluxDB ingested all of the data in the batch |
54-
| `400 "Bad request"` | error details about rejected points, up to 100 points: `line` contains the first rejected line, `message` describes rejections | If some (_when **partial writes** are configured for the cluster_) or all request data isn't allowed (for example, if it is malformed or falls outside of the bucket's retention period)--the response body indicates whether a partial write has occurred or if all data has been rejected |
55-
| `401 "Unauthorized"` | | If the `Authorization` header is missing or malformed or if the [token](/influxdb3/cloud-dedicated/admin/tokens/) doesn't have [permission](/influxdb3/cloud-dedicated/reference/cli/influxctl/token/create/#examples) to write to the database. See [examples using credentials](/influxdb3/cloud-dedicated/get-started/write/#write-line-protocol-to-influxdb) in write requests. |
56-
| `404 "Not found"` | requested **resource type** (for example, "organization" or "database"), and **resource name** | If a requested resource (for example, organization or database) wasn't found |
57-
| `422 "Unprocessable Entity"` | `message` contains details about the error | If the data isn't allowed (for example, falls outside of the database’s retention period).
58-
| `500 "Internal server error"` | | Default status for an error |
59-
| `503 "Service unavailable"` | | If the server is temporarily unavailable to accept writes. The `Retry-After` header contains the number of seconds to wait before trying the write again.
60-
61-
The `message` property of the response body may contain additional details about the error.
62-
If your data did not write to the database, see how to [troubleshoot rejected points](#troubleshoot-rejected-points).
63-
64-
## Troubleshoot failures
65-
66-
If you notice data is missing in your database, do the following:
67-
68-
- Check the [HTTP status code](#review-http-status-codes) in the response.
69-
- Check the `message` property in the response body for details about the error.
70-
- If the `message` describes a field error, [troubleshoot rejected points](#troubleshoot-rejected-points).
71-
- Verify all lines contain valid syntax ([line protocol](/influxdb3/cloud-dedicated/reference/syntax/line-protocol/)).
72-
- Verify the timestamps in your data match the [precision parameter](/influxdb3/cloud-dedicated/reference/glossary/#precision) in your request.
73-
- Minimize payload size and network errors by [optimizing writes](/influxdb3/cloud-dedicated/write-data/best-practices/optimize-writes/).
74-
75-
## Troubleshoot rejected points
76-
77-
When writing points from a batch, InfluxDB rejects points that have syntax errors or schema conflicts.
78-
If InfluxDB processes the data in your batch and then rejects points, the [HTTP response](#handle-write-responses) body contains the following properties that describe rejected points:
79-
80-
- `code`: `"invalid"`
81-
- `line`: the line number of the _first_ rejected point in the batch.
82-
- `message`: a string that contains line-separated error messages, one message for each rejected point in the batch, up to 100 rejected points.
83-
84-
InfluxDB rejects points for the following reasons:
85-
86-
- a line protocol parsing error
87-
- an invalid timestamp
88-
- a schema conflict
89-
90-
Schema conflicts occur when you try to write data that contains any of the following:
91-
92-
- a wrong data type: the point falls within the same partition (default partitioning is measurement and day) as existing bucket data and contains a different data type for an existing field
93-
- a tag and a field that use the same key
94-
95-
### Example
96-
97-
The following example shows a response body for a write request that contains two rejected points:
98-
99-
```json
100-
{
101-
"code": "invalid",
102-
"line": 2,
103-
"message": "failed to parse line protocol:
104-
errors encountered on line(s):
105-
error parsing line 2 (1-based): Invalid measurement was provided
106-
error parsing line 4 (1-based): Unable to parse timestamp value '123461000000000000000000000000'"
107-
}
108-
```
109-
110-
Check for [field data type](/influxdb3/cloud-dedicated/reference/syntax/line-protocol/#data-types-and-format) differences between the rejected data point and points within the same database and partition--for example, did you attempt to write `string` data to an `int` field?
22+
<!-- The content for this page is at
23+
//SOURCE - content/shared/influxdb3-write-guides/troubleshoot-distributed.md
24+
-->

content/influxdb3/cloud-serverless/write-data/troubleshoot.md

Lines changed: 6 additions & 94 deletions
Original file line numberDiff line numberDiff line change
@@ -10,103 +10,15 @@ menu:
1010
influxdb3_cloud_serverless:
1111
name: Troubleshoot issues
1212
parent: Write data
13-
influxdb3/cloud-serverless/tags: [write, line protocol, errors]
13+
influxdb3/cloud-serverless/tags: [write, line protocol, errors, partial writes]
1414
related:
15+
- /influxdb3/cloud-serverless/get-started/write/
1516
- /influxdb3/cloud-serverless/reference/syntax/line-protocol/
1617
- /influxdb3/cloud-serverless/write-data/best-practices/
1718
- /influxdb3/cloud-serverless/reference/internals/durability/
19+
source: /shared/influxdb3-write-guides/troubleshoot-distributed.md
1820
---
1921

20-
Learn how to avoid unexpected results and recover from errors when writing to {{% product-name %}}.
21-
22-
23-
- [Handle write responses](#handle-write-responses)
24-
- [Review HTTP status codes](#review-http-status-codes)
25-
- [Troubleshoot failures](#troubleshoot-failures)
26-
- [Troubleshoot rejected points](#troubleshoot-rejected-points)
27-
28-
## Handle write responses
29-
30-
{{% product-name %}} does the following when you send a write request:
31-
32-
1. Validates the request.
33-
2. If successful, attempts to [ingest data](/influxdb3/cloud-serverless/reference/internals/durability/#data-ingest) from the request body; otherwise, responds with an [error status](#review-http-status-codes).
34-
3. Ingests or rejects data from the batch and returns one of the following HTTP status codes:
35-
36-
- `204 No Content`: All of the data is ingested and queryable.
37-
- `400 Bad Request`: Some or all of the data has been rejected. Data that has not been rejected is ingested and queryable.
38-
39-
The response body contains error details about [rejected points](#troubleshoot-rejected-points), up to 100 points.
40-
41-
Writes are synchronous--the response status indicates the final status of the write and all ingested data is queryable.
42-
43-
To ensure that InfluxDB handles writes in the order you request them,
44-
wait for the response before you send the next request.
45-
46-
### Review HTTP status codes
47-
48-
InfluxDB uses conventional HTTP status codes to indicate the success or failure of a request.
49-
The `message` property of the response body may contain additional details about the error.
50-
{{< product-name >}} returns one the following HTTP status codes for a write request:
51-
52-
| HTTP response code | Response body | Description |
53-
| :-------------------------------| :--------------------------------------------------------------- | :------------- |
54-
| `204 "No Content"` | no response body | If InfluxDB ingested all of the data in the batch |
55-
| `400 "Bad request"` | error details about rejected points, up to 100 points: `line` contains the first rejected line, `message` describes rejections | If some or all request data isn't allowed (for example, is malformed or falls outside of the bucket's retention period)--the response body indicates whether a partial write has occurred or if all data has been rejected |
56-
| `401 "Unauthorized"` | | If the `Authorization` header is missing or malformed or if the [token](/influxdb3/cloud-serverless/admin/tokens/) doesn't have [permission](/influxdb3/cloud-serverless/admin/tokens/create-token/) to write to the bucket. See [examples using credentials](/influxdb3/cloud-serverless/get-started/write/#write-line-protocol-to-influxdb) in write requests. |
57-
| `404 "Not found"` | requested **resource type** (for example, "organization" or "bucket"), and **resource name** | If a requested resource (for example, organization or bucket) wasn't found |
58-
| `413 “Request too large”` | cannot read data: points in batch is too large | If a request exceeds the maximum [global limit](/influxdb3/cloud-serverless/admin/billing/limits/) |
59-
| `429 “Too many requests”` | | If the number of requests exceeds the [adjustable service quota](/influxdb3/cloud-serverless/admin/billing/limits/#adjustable-service-quotas). The `Retry-After` header contains the number of seconds to wait before trying the write again. | If a request exceeds your plan's [adjustable service quotas](/influxdb3/cloud-serverless/admin/billing/limits/#adjustable-service-quotas)
60-
| `500 "Internal server error"` | | Default status for an error |
61-
| `503 "Service unavailable"` | | If the server is temporarily unavailable to accept writes. The `Retry-After` header contains the number of seconds to wait before trying the write again.
62-
63-
The `message` property of the response body may contain additional details about the error.
64-
If your data did not write to the bucket, see how to [troubleshoot rejected points](#troubleshoot-rejected-points).
65-
66-
## Troubleshoot failures
67-
68-
If you notice data is missing in your database, do the following:
69-
70-
- Check the [HTTP status code](#review-http-status-codes) in the response.
71-
- Check the `message` property in the response body for details about the error.
72-
- If the `message` describes a field error, [troubleshoot rejected points](#troubleshoot-rejected-points).
73-
- Verify all lines contain valid syntax ([line protocol](/influxdb3/cloud-serverless/reference/syntax/line-protocol/)).
74-
- Verify the timestamps in your data match the [precision parameter](/influxdb3/cloud-serverless/reference/glossary/#precision) in your request.
75-
- Minimize payload size and network errors by [optimizing writes](/influxdb3/cloud-serverless/write-data/best-practices/optimize-writes/).
76-
77-
## Troubleshoot rejected points
78-
79-
When writing points from a batch, InfluxDB rejects points that have syntax errors or schema conflicts.
80-
If InfluxDB processes the data in your batch and then rejects points, the [HTTP response](#handle-write-responses) body contains the following properties that describe rejected points:
81-
82-
- `code`: `"invalid"`
83-
- `line`: the line number of the _first_ rejected point in the batch.
84-
- `message`: a string that contains line-separated error messages, one message for each rejected point in the batch, up to 100 rejected points.
85-
86-
InfluxDB rejects points for the following reasons:
87-
88-
- a line protocol parsing error
89-
- an invalid timestamp
90-
- a schema conflict
91-
92-
Schema conflicts occur when you try to write data that contains any of the following:
93-
94-
- a wrong data type: the point falls within the same partition (default partitioning is measurement and day) as existing bucket data and contains a different data type for an existing field
95-
- a tag and a field that use the same key
96-
97-
### Example
98-
99-
The following example shows a response body for a write request that contains two rejected points:
100-
101-
```json
102-
{
103-
"code": "invalid",
104-
"line": 2,
105-
"message": "failed to parse line protocol:
106-
errors encountered on line(s):
107-
error parsing line 2 (1-based): Invalid measurement was provided
108-
error parsing line 4 (1-based): Unable to parse timestamp value '123461000000000000000000000000'"
109-
}
110-
```
111-
112-
Check for [field data type](/influxdb3/cloud-serverless/reference/syntax/line-protocol/#data-types-and-format) differences between the rejected data point and points within the same database and partition--for example, did you attempt to write `string` data to an `int` field?
22+
<!-- The content for this page is at
23+
//SOURCE - content/shared/influxdb3-write-guides/troubleshoot-distributed.md
24+
-->

0 commit comments

Comments
 (0)