Skip to content

Commit 32660dd

Browse files
author
pcoma55
committed
this PR adds two new documents around ingestion troubleshooting and updates links to other articles around ingestion
1 parent aea0cea commit 32660dd

11 files changed

+202
-9
lines changed

articles/planetary-computer/add-stac-item-to-collection.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -115,9 +115,8 @@ while True:
115115
if status not in {"Pending", "Running"}:
116116
break
117117
time.sleep(5)
118-
```
119118

120-
Once the items are ingested, use the `/stac/collections/{collection_id}/items` or `/stac/search` endpoints to get a paginated list of items, including your newly ingested items.
119+
Once the items are ingested, use the `/stac/collections/{collection_id}/items` or `/stac/search` endpoints to get a paginated list of items, including your newly ingested items. If the ingestion fails or you encounter errors, consult the [troubleshooting guide](./troubleshooting-ingestion.md) and the [list of error codes](./error-codes-ingestion.md).
121120

122121
```python
123122
items_response = requests.get(
@@ -174,3 +173,5 @@ Now that you added a few items, you should configure the data for visualization.
174173
- [Quickstart: Ingest data using the Microsoft Planetary Computer Pro web interface](./ingest-via-web-interface.md)
175174
- [Ingest data into Microsoft Planetary Computer Pro](./ingestion-overview.md)
176175
- [Ingestion source for Microsoft Planetary Computer Pro](./ingestion-source.md)
176+
- [Troubleshooting data ingestion](./troubleshooting-ingestion.md)
177+
- [Error codes: Microsoft Planetary Computer Pro ingestion](./error-codes-ingestion.md)

articles/planetary-computer/bulk-ingestion-api.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -315,7 +315,7 @@ In this final step, we're using the ingestion API to initiate a bulk ingestion w
315315
else:
316316
print(f"Failed to create ingestion run: {wf_response.text}")
317317
318-
Once the workflow is complete, you can query, retrieve, or visualize your geospatial data using the GeoCatalog STAC or data APIs, or with the Data Explorer.
318+
Once the workflow is complete, you can query, retrieve, or visualize your geospatial data using the GeoCatalog STAC or data APIs, or with the Data Explorer. If you encounter issues, refer to the [troubleshooting guide](./troubleshooting-ingestion.md) or the list of [ingestion error codes](./error-codes-ingestion.md).
319319
320320
## Clean up resources
321321
@@ -348,3 +348,5 @@ Now that you added a few items, you should configure the data for visualization.
348348
- [Quickstart: Ingest data using the Microsoft Planetary Computer Pro web interface](./ingest-via-web-interface.md)
349349
- [Ingest data into Microsoft Planetary Computer Pro](./ingestion-overview.md)
350350
- [Ingestion source for Microsoft Planetary Computer Pro](./ingestion-source.md)
351+
- [Troubleshooting data ingestion](./troubleshooting-ingestion.md)
352+
- [Error codes: Microsoft Planetary Computer Pro ingestion](./error-codes-ingestion.md)

articles/planetary-computer/create-stac-item.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -324,6 +324,9 @@ When pointed to a COG file, `rio-stac` automatically extracts and organizes esse
324324

325325
This automation significantly reduces the manual work needed to create valid STAC Items and ensures consistency in metadata
326326

327+
> [!NOTE]
328+
> GeoCatalog has limitations on characters that can be used in STAC Item IDs and Asset keys. Ensure that your IDs don't contain the following characters: `-`, `_`, `+`, `(`, `)`, and `.`. You may need to modify the `item_id` generation logic to replace or remove these characters from your filenames.
329+
327330
```python
328331
from rio_stac import create_stac_item
329332
from rasterio.io import MemoryFile
Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
---
2+
title: Ingestion error codes for Microsoft Planetary Computer Pro
3+
description: Learn how to troubleshoot ingestion errors when adding geospatial data to your Microsoft Planetary Computer Pro GeoCatalog.
4+
author: prasadko
5+
ms.author: prasadkomma
6+
ms.service: planetary-computer-pro
7+
ms.topic: error-reference
8+
ms.date: 06/16/2025
9+
10+
#customer intent: As a GeoCatalog user, I want to understand ingestion error codes so that I can troubleshoot and resolve issues when ingesting geospatial data.
11+
---
12+
13+
# Error codes: Microsoft Planetary Computer Pro ingestion
14+
15+
This article lists the error codes that can occur when ingesting geospatial data into a Microsoft Planetary Computer Pro GeoCatalog. Use this reference to identify the cause of ingestion failures and find resolution steps for each error type.
16+
17+
Ingestion errors can occur during various stages of the data ingestion process, including authentication, validation, transformation, and storage operations. The following table provides detailed information about each error code, its meaning, and steps to resolve the issue.
18+
19+
| Code | Error message | Details |
20+
| --- | --- | --- |
21+
| `AssetTransformationError` | Asset transformation failed | The system failed to transform the asset during ingestion. Verify that your asset files are in a [supported format](./supported-data-types.md) and aren't corrupted. Check that the asset URLs are accessible from your ingestion source. |
22+
| `CogTransformationError` | Error occurred during COG transformation | The Cloud Optimized GeoTIFF (COG) transformation process failed. Ensure your source files are valid GeoTIFF files with proper georeferencing information. Try validating your files with GDAL tools before ingestion. |
23+
| `CogTransformationTimeout` | COG transformation timed out | The transformation process exceeded the time limit. This error typically occurs with very large files. Consider splitting large files into smaller tiles or ensure your files are already in COG format to skip transformation. |
24+
| `CanceledOperation` | Operation was canceled | The ingestion operation was manually canceled or terminated. No action required unless this error was unexpected, in which case check system logs for automatic cancellation reasons. |
25+
| `CollectionDoesNotExist` | The specified collection doesn't exist | The STAC collection ID referenced in your ingestion request doesn't exist in the GeoCatalog. [Create the collection](./create-stac-collection.md) before attempting to ingest items into it. |
26+
| `EmptyAsset` | Asset content is empty | The asset file referenced in the STAC item has no content or is 0 bytes. Verify that the asset files exist at the specified URLs and contain valid data. |
27+
| `ErrorsInBatchOperation` | Multiple errors occurred in batch operation | One or more items in a bulk ingestion operation failed. Check the operation details for specific item failures. Consider ingesting failed items individually to isolate issues. |
28+
| `GenericHttpIngestionError` | HTTP error during ingestion | A general HTTP error occurred while accessing ingestion resources. Check network connectivity and verify that all asset URLs are accessible. Review HTTP status codes in logs for specific issues. |
29+
| `GenericIngestionError` | General ingestion error occurred | An unspecified error occurred during ingestion. Check the detailed error message and system logs for more information. Contact support if the issue persists. |
30+
| `ImportFileValidationFailed` | Import file validation failed | The STAC catalog or collection file failed validation. Ensure your STAC JSON files conform to the [STAC specification](./stac-overview.md) and include all required fields. |
31+
| `IngestionAuthenticationFailed` | Authentication to ingestion source failed | The GeoCatalog couldn't authenticate to the ingestion source. Verify your [ingestion source credentials](./ingestion-source.md) are valid. For SAS tokens, ensure the token isn't past its expiration date. |
32+
| `IngestionResourceConnectionFailed` | Connection to ingestion source failed | Unable to establish connection to the storage container. Check that the storage account is accessible and that network policies allow connections from GeoCatalog. |
33+
| `IngestionResourceForbidden` | Access to ingestion source is forbidden | The GeoCatalog lacks permissions to access the resource. Ensure your [managed identity has the Storage Blob Data Reader role](./set-up-ingestion-credentials-managed-identity.md) or that your SAS token has `read` permissions. |
34+
| `IngestionResourceTimeout` | Ingestion source access timed out | The request to access the ingestion source exceeded the time limit. This error can indicate network issues or an unresponsive storage service. Retry the operation or check Azure service health. |
35+
| `IngestionResourceUriNotFound` | Resource URI wasn't found | The specified asset or catalog URI doesn't exist. Verify all URLs in your STAC items point to existing files. Check for typos in file paths and ensure files haven't been moved or deleted. |
36+
| `InternalError` | Internal service error occurred | An unexpected error occurred within the GeoCatalog service. Retry the operation. If the error persists, contact support with the operation ID and timestamp. |
37+
| `InternalIntermitentError` | Intermittent internal error occurred | A temporary internal error occurred. Wait a few minutes and retry the operation. These errors typically resolve automatically. |
38+
| `InvalidAssetHref` | Asset href URL is invalid | The asset URL in the STAC item is malformed or invalid. Ensure all asset hrefs are properly formatted URLs pointing to accessible resources. Use absolute URLs, not relative paths. |
39+
| `InvalidGeoparquet` | GeoParquet file is invalid | The GeoParquet file doesn't meet the required format specifications. Validate your GeoParquet files using appropriate tools and ensure they contain valid geometry columns. |
40+
| `InvalidInputData` | Input data for ingestion is invalid | The STAC item or collection data contains invalid fields or values. Review the [STAC item creation guide](./create-stac-item.md) and validate your JSON against the STAC schema. |
41+
| `InvalidStacCatalog` | STAC catalog format is invalid | The STAC catalog structure doesn't conform to specifications. Ensure your catalog follows the [STAC catalog specification](https://github.com/radiantearth/stac-spec/blob/master/catalog-spec/catalog-spec.md) with proper links and child references. |
42+
| `ItemAlreadyExistsInCollection` | Item with this ID already exists | A STAC item with the same ID already exists in the collection. Use unique IDs for each item or delete the existing item before attempting to ingest again. |
43+
| `ItemSizeExceeded` | Item size exceeds limit | The STAC item JSON is too large. Reduce the number of assets or properties in the item. Consider splitting large items into multiple smaller items. |
44+
| `ItemTransformationError` | Item transformation error occurred | Failed to process or transform the STAC item. Verify that all required STAC fields are present and that datetime values are properly formatted. |
45+
| `ManagedIdentityInfoNotFound` | Managed identity information not found | The specified managed identity doesn't exist or isn't associated with the GeoCatalog. [Configure a managed identity](./set-up-ingestion-credentials-managed-identity.md) for your GeoCatalog. |
46+
| `NoRecordsToDelete` | No records found to delete | The delete operation didn't find any matching records. Verify the STAC item IDs or search criteria used for deletion. |
47+
| `NoRecordsToPatch` | No records found to patch | The patch operation didn't find any matching records. Ensure the STAC items you're trying to update exist in the collection. |
48+
| `NoRecordsToUpdate` | No records found to update | The update operation didn't find any matching records. Verify that the items exist before attempting updates. |
49+
| `OperationIsCanceled` | Operation was canceled | The ingestion operation was canceled before completion. Check if this was intentional or if system limits triggered automatic cancellation. |
50+
| `OperationNotExistsInCollection` | Operation not found in collection | The referenced operation ID doesn't exist for this collection. Verify you're using the correct operation ID and collection. |
51+
| `PatchValidation` | Patch validation failed | The patch operation contains invalid data or operations. Ensure your patch follows [JSON Patch](https://jsonpatch.com/) specifications and targets valid fields. |
52+
| `PostItemCollectionSizeExceeded` | Item collection POST size exceeded | The batch of items being posted is too large. Reduce the number of items in a single request or use the [bulk ingestion API](./bulk-ingestion-api.md) for large datasets. |
53+
| `PublicAccessRestricted` | The data you're trying to ingest restricted public access | The storage resource doesn't allow public access. Configure proper [ingestion sources](./ingestion-source.md) with appropriate authentication instead of using public URLs. |
54+
| `StacValidationFailed` | STAC item validation failed | The STAC item doesn't meet validation requirements. Use STAC validation tools to check your items and ensure all required fields are present with valid values. |
55+
| `TotalAssetsExceeded` | Maximum number of assets exceeded | The STAC item contains too many assets. Reduce the number of assets per item or split into multiple items. Check service limits for maximum assets per item. |
56+
57+
## Related content
58+
59+
- [Ingest data into Microsoft Planetary Computer Pro](./ingestion-overview.md)
60+
- [Configure an ingestion source using managed identity](./set-up-ingestion-credentials-managed-identity.md)
61+
- [Configure an ingestion source using SAS tokens](./set-up-ingestion-credentials-sas-tokens.md)
62+
- [Create a STAC Item](./create-stac-item.md)
63+
- [Bulk Ingestion API](./bulk-ingestion-api.md)

articles/planetary-computer/ingest-via-web-interface.md

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -67,10 +67,14 @@ Now that you have configured an ingestion source, refer to the following steps t
6767
8. When the ingestion run is complete, the **STAC items** tab in your collection displays your list of STAC items that have been ingested, with columns for the item ID and acquisition date.
6868
[ ![Screenshot of STAC items tab.](./media/stac-items.jpeg) ](./media/stac-items.jpeg#lightbox)
6969

70+
## Troubleshooting
7071

71-
## Next steps
72+
If you encounter issues during the ingestion process, such as failures in creating an ingestion source, starting a run, or if items don't appear as expected, refer to the following resources:
73+
74+
- [Troubleshooting data ingestion in Microsoft Planetary Computer Pro](./troubleshooting-ingestion.md): Provides guidance on diagnosing common ingestion problems.
75+
- [Error codes: Microsoft Planetary Computer Pro ingestion](./error-codes-ingestion.md): Lists common error codes and their meanings.
7276

73-
<!-- TODO: Update this link to point to the next article in the sequence once finalized. -->
77+
## Next steps
7478

7579
Once you have ingested data, it's time to configure the STAC Collection so you can visualize your data in the Explorer.
7680

articles/planetary-computer/ingestion-overview.md

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,13 @@ Given a preexisting STAC collection within a GeoCatalog, users can ingest new or
7070

7171
[ ![Screenshot of single-item ingestion GUI.](media/data-processing-user-interface-collections-items.png) ](media/data-processing-user-interface-collections-items.png#lightbox)
7272

73+
## Troubleshooting Ingestion
74+
75+
If you encounter issues during data ingestion, such as authentication failures, STAC validation errors, or problems with asset transformation, refer to our dedicated troubleshooting documentation:
76+
77+
- [Troubleshooting data ingestion in Microsoft Planetary Computer Pro](./troubleshooting-ingestion.md): This guide helps you identify and resolve common ingestion failures, including access permission problems and STAC validation errors.
78+
- [Error codes: Microsoft Planetary Computer Pro ingestion](./error-codes-ingestion.md): This reference lists specific error codes you might encounter during ingestion, along with their meanings and resolution steps.
79+
7380
## Next steps
7481
Learn more about how to setup an Ingestion Source:
7582

@@ -81,4 +88,6 @@ Learn more about how to setup an Ingestion Source:
8188

8289
- [Configure an ingestion source for Microsoft Planetary Computer Pro using managed identity](./set-up-ingestion-credentials-managed-identity.md)
8390
- [Add STAC Items to a Collection with GeoCatalog and Python](./add-stac-item-to-collection.md)
84-
- [Ingest data into GeoCatalog with the Bulk Ingestion API](./bulk-ingestion-api.md)
91+
- [Ingest data into GeoCatalog with the Bulk Ingestion API](./bulk-ingestion-api.md)
92+
- [Troubleshooting data ingestion](./troubleshooting-ingestion.md)
93+
- [Error codes: Microsoft Planetary Computer Pro ingestion](./error-codes-ingestion.md)

articles/planetary-computer/ingestion-source.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,9 @@ Microsoft Planetary Computer Pro currently only supports secure ingestion from [
3232
>[!IMPORTANT]
3333
> Don't include a trailing "/" after the container name.
3434
35+
> [!NOTE]
36+
> If your STAC metadata and data assets are stored in different containers or storage accounts, you must configure a separate ingestion source for each location. The GeoCatalog needs permission to read from all locations where your data resides.
37+
3538
## Ingestion source authentication mechanisms
3639

3740
Securely ingesting data requires users to provide an authentication mechanism which permits a GeoCatalog to read the data from a specific location. Planetary Computer Pro supports two mechanisms to support secure ingestion:
@@ -53,3 +56,5 @@ The following quickstarts are available to assist users in setting up ingestion
5356
- [Set up Ingestion Credentials for Planetary Computer Pro using SAS Tokens](./set-up-ingestion-credentials-sas-tokens.md)
5457
- [Adding an Item to a STAC Collection](./add-stac-item-to-collection.md)
5558
- [Ingest data into GeoCatalog with the Bulk Ingestion API](./add-stac-item-to-collection.md)
59+
- [Troubleshooting data ingestion](./troubleshooting-ingestion.md)
60+
- [Error codes: Microsoft Planetary Computer Pro ingestion](./error-codes-ingestion.md)
74.4 KB
Loading
1.69 MB
Loading

articles/planetary-computer/microsoft-planetary-computer-pro-overview.md

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,4 @@ The end-to-end GeoCatalog product lifecycle traverses the following stages:
8282
- [Bulk Ingestion API](./bulk-ingestion-api.md)
8383
- [Supported Data Types](./supported-data-types.md)
8484
- [Use Explorer](./use-explorer.md)
85-
- [Data Cube Quickstart](./data-cube-quickstart.md)
86-
- [Queryables for Explorer Custom Search Filter](./queryables-for-explorer-custom-search-filter.md)
87-
- [Delete a GeoCatalog](./delete-geocatalog-resource.md)
85+
- [Data Cube Quickstart](./data-cube-quickstart.md)

0 commit comments

Comments
 (0)