Skip to content

Commit a981c69

Browse files
authored
Merge pull request #178367 from jovanpop-msft/patch-221
Re-arranging delta known issues
2 parents dfc4351 + 9d625a4 commit a981c69

File tree

1 file changed

+22
-25
lines changed

1 file changed

+22
-25
lines changed

articles/synapse-analytics/sql/resources-self-help-sql-on-demand.md

Lines changed: 22 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -587,6 +587,28 @@ The easiest way is to grant yourself `Storage Blob Data Contributor` role on the
587587
- [Visit full guide on Azure Active Directory access control for storage for more information](../../storage/blobs/assign-azure-role-data-access.md).
588588
- [Visit Control storage account access for serverless SQL pool in Azure Synapse Analytics](develop-storage-files-storage-access-control.md)
589589

590+
### JSON text is not properly formatted
591+
592+
This error indicates that serverless SQL pool cannot read Delta Lake transaction log. You will probably see the error like the following error:
593+
594+
```
595+
Msg 13609, Level 16, State 4, Line 1
596+
JSON text is not properly formatted. Unexpected character '' is found at position 263934.
597+
Msg 16513, Level 16, State 0, Line 1
598+
Error reading external metadata.
599+
```
600+
Make sure that your Delta Lake data set is not corrupted. Verify that you can read the content of the Delta Lake folder using Apache Spark pool in Azure Synapse. This way you will ensure that the `_delta_log` file is not corrupted.
601+
602+
**Workaround** - try to create a checkpoint on Delta Lake data set using Apache Spark pool and re-run the query. The checkpoint will aggregate transactional json log files and might solve the issue.
603+
604+
If the data set is valid, [create a support ticket](../../azure-portal/supportability/how-to-create-azure-support-request.md#create-a-support-request) and provide an additional info:
605+
- Do not make any changes like adding/removing the columns or optimizing the table because this might change the state of Delta Lake transaction log files.
606+
- Copy the content of `_delta_log` folder into a new empty folder. **DO NOT** copy `.parquet data` files.
607+
- Try to read the content that you copied in new folder and verify that you are getting the same error.
608+
- Send the content of the copied `_delta_log` file to Azure support.
609+
610+
Now you can continue using Delta Lake folder with Spark pool. You will provide copied data to Microsoft support if you are allowed to share this. Azure team will investigate the content of the `delta_log` file and provide more info about the possible errors and the workarounds.
611+
590612
### Partitioning column returns NULL values
591613

592614
**Status**: Resolved
@@ -617,31 +639,6 @@ The easiest way is to grant yourself `Storage Blob Data Contributor` role on the
617639

618640
**Release**: November 2021
619641

620-
### JSON text is not properly formatted
621-
622-
This error indicates that serverless SQL pool cannot read Delta Lake transaction log. You will probably see the error like the following error:
623-
624-
```
625-
Msg 13609, Level 16, State 4, Line 1
626-
JSON text is not properly formatted. Unexpected character '{' is found at position 263934.
627-
Msg 16513, Level 16, State 0, Line 1
628-
Error reading external metadata.
629-
```
630-
First, make sure that your Delta Lake data set is not corrupted.
631-
- Verify that you can read the content of the Delta Lake folder using Apache Spark pool in Azure Synapse. This way you will ensure that the `_delta_log` file is not corrupted.
632-
- Verify that you can read the content of data files by specifying `FORMAT='PARQUET'` and using recursive wildcard `/**` at the end of the URI path. If you can read all Parquet files, the issue is in `_delta_log` transaction log folder.
633-
634-
**Workaround** - try to create a checkpoint on Delta Lake data set using Apache Spark pool and re-run the query. The checkpoint will aggregate transactional json log files and might solve the issue.
635-
636-
If the data set is valid, and the workarounds cannot help, report a support ticket and provide an additional info to Azure support:
637-
- Do not make any changes like adding/removing the columns or optimizing the table because this might change the state of Delta Lake transaction log files.
638-
- Copy the content of `_delta_log` folder into a new empty folder. **DO NOT** copy `.parquet data` files.
639-
- Try to read the content that you copied in new folder and verify that you are getting the same error.
640-
- Now you can continue using Delta Lake folder with Spark pool. You will provide copied data to Microsoft support if you are allowed to share this.
641-
- Send the content of the copied `_delta_log` file to Azure support.
642-
643-
Azure team will investigate the content of the `delta_log` file and provide more info about the possible errors and the workarounds.
644-
645642
### Resolving delta log on path ... failed with error: Cannot parse JSON object from log file
646643

647644
**Status**: Resolved

0 commit comments

Comments
 (0)