You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/data-factory/data-flow-troubleshoot-guide.md
+45-5Lines changed: 45 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@ ms.service: data-factory
8
8
ms.subservice: data-flows
9
9
ms.custom: ignite-2022
10
10
ms.topic: troubleshooting
11
-
ms.date: 11/02/2022
11
+
ms.date: 03/28/2023
12
12
---
13
13
14
14
# Troubleshoot mapping data flows in Azure Data Factory
@@ -330,7 +330,7 @@ This section lists common error codes and messages reported by mapping data flow
330
330
-**Cause**: Broadcast has a default timeout of 60 seconds in debug runs and 300 seconds in job runs. On the broadcast join, the stream chosen for broadcast is too large to produce data within this limit. If a broadcast join isn't used, the default broadcast by dataflow can reach the same limit.
331
331
-**Recommendation**: Turn off the broadcast option or avoid broadcasting large data streams for which the processing can take more than 60 seconds. Choose a smaller stream to broadcast. Large Azure SQL Data Warehouse tables and source files aren't typically good choices. In the absence of a broadcast join, use a larger cluster if this error occurs.
332
332
333
-
### Error code: DF-Executor-ColumnUnavailable
333
+
### Error code: DF-Executor-ColumnNotFound
334
334
335
335
-**Message**: Column name used in expression is unavailable or invalid.
336
336
-**Cause**: An invalid or unavailable column name is used in an expression.
@@ -440,7 +440,7 @@ This section lists common error codes and messages reported by mapping data flow
440
440
-**Cause**: The size of the data far exceeds the limit of the node memory.
441
441
-**Recommendation**: Increase the core count and switch to the memory optimized compute type.
442
442
443
-
### Error code: DF-Executor-ParseError
443
+
### Error code: DF-Executor-ExpressionParseError
444
444
445
445
-**Message**: Expression cannot be parsed.
446
446
-**Cause**: An expression generated parsing errors because of incorrect formatting.
@@ -484,6 +484,12 @@ This section lists common error codes and messages reported by mapping data flow
484
484
-**Cause**: Invalid store configuration is provided.
485
485
-**Recommendation**: Check the parameter value assignment in the pipeline. A parameter expression may contain invalid characters.
-**Message**: Column operands are not allowed in literal expressions.
490
+
-**Cause**: The value for a string parameter or an expected string value is not enclosed in single quotes.
491
+
-**Recommendation**: Near the mentioned line number(s) in the data flow script, ensure the value for a string parameter or an expected string value is enclosed in single quotes.
-**Message**: Implicit cartesian product for INNER join is not supported, use CROSS JOIN instead. Columns used in join should create a unique key for rows.
@@ -517,6 +523,23 @@ This section lists common error codes and messages reported by mapping data flow
517
523
-**Cause**: Privileged access approval is needed to copy data. It's a user configuration issue.
518
524
-**Recommendation**: Ask the tenant admin to approve your **Data Access Request** in Office365 in privileged access management (PAM) module.
519
525
526
+
### Error code: DF-Executor-DSLParseError
527
+
528
+
-**Message**: Data flow script cannot be parsed.
529
+
-**Cause**: The data flow script has parsing errors.
530
+
-**Recommendation**: Check for errors (example: missing symbol(s), unwanted symbol(s)) near mentioned line number(s) in the data flow script.
531
+
532
+
### Error code: DF-Executor-IncorrectQuery
533
+
534
+
-**Message**: Incorrect syntax. SQL Server error encountered while reading from the given table or while executing the given query.
535
+
-**Cause**: The query submitted was syntactically incorrect.
536
+
-**Recommendation**: Check the syntactical correctness of the given query. Ensure to have a non-quoted query string when it is referenced as a pipeline parameter.
537
+
538
+
### Error code: DF-Executor-ParameterParseError
539
+
-**Message**: Parameter stream has parsing errors. Not honoring the datatype of parameter(s) could be one of the causes.
540
+
-**Cause**: Parsing errors in given parameter(s).
541
+
-**Recommendation**: Check the parameter(s) having errors, ensure the usage of appropriate function(s), and honor the datatype(s) given.
542
+
520
543
### Error code: DF-File-InvalidSparkFolder
521
544
522
545
-**Message**: Failed to read footer for file.
@@ -525,7 +548,7 @@ This section lists common error codes and messages reported by mapping data flow
-**Message**: Either one of account key or tenant/spnId/spnCredential/spnCredentialType or miServiceUri/miServiceToken should be specified.
551
+
-**Message**: Either one of account key or SAS token or tenant/spnId/spnCredential/spnCredentialType or userAuth or miServiceUri/miServiceToken should be specified.
529
552
-**Cause**: An invalid credential is provided in the ADLS Gen2 linked service.
530
553
-**Recommendation**: Update the ADLS Gen2 linked service to have the right credential configuration.
531
554
@@ -589,10 +612,15 @@ This section lists common error codes and messages reported by mapping data flow
589
612
590
613
### Error code: DF-JSON-WrongDocumentForm
591
614
592
-
-**Message**: Malformed records are detected in schema inference. Parse Mode: FAILFAST.
615
+
-**Message**: Malformed records are detected in schema inference. Parse Mode: FAILFAST. It could be because of a wrong selection in document form to parse json file(s). Please try a different 'Document form' (Single document/Document per line/Array of documents) on the json source.
593
616
-**Cause**: Wrong document form is selected to parse JSON file(s).
594
617
-**Recommendation**: Try different **Document form** (**Single document**/**Document per line**/**Array of documents**) in JSON settings. Most cases of parsing errors are caused by wrong configuration.
595
618
619
+
### Error code: DF-MICROSOFT365-CONSENTPENDING
620
+
-**Message**: Admin Consent is pending.
621
+
-**Cause**: Admin Consent is missing.
622
+
-**Recommendation**: Provide the consent and then rerun the pipeline. To provide consent, refer to [PAM requests](/graph/data-connect-faq#how-can-i-approve-pam-requests-via-the-microsoft-365-admin-center).
623
+
596
624
### Error code: DF-MSSQL-ErrorRowsFound
597
625
598
626
-**Cause**: Error/Invalid rows were found while writing to Azure SQL Database sink.
@@ -865,6 +893,13 @@ This section lists common error codes and messages reported by mapping data flow
865
893
-**Cause**: An exception happened while writing error rows to the storage.
866
894
-**Recommendation**: Please check your rejected data linked service configuration.
-**Message**: The linked service is incorrectly configured as type 'Azure Synapse Analytics' instead of 'Azure SQL Database'. Please create a new linked service of type 'Azure SQL Database'<br>
899
+
Note: Please check that the given database is of type 'Dedicated SQL pool (formerly SQL DW)' for linked service type 'Azure Synapse Analytics'.
900
+
-**Cause**: The linked service is incorrectly configured as type **Azure Synapse Analytics** instead of **Azure SQL Database**.
901
+
-**Recommendation**: Create a new linked service of type **Azure SQL Database**, and check that the given database is of type Dedicated SQL pool (formerly SQL DW) for linked service type **Azure Synapse Analytics**.
902
+
868
903
### Error code: DF-SQLDW-InternalErrorUsingMSI
869
904
870
905
-**Message**: An internal error occurred while authenticating against Managed Service Identity in Azure Synapse Analytics instance. Please restart the Azure Synapse Analytics instance or contact Azure Synapse Analytics Dedicated SQL Pool support if this problem persists.
@@ -995,6 +1030,11 @@ This section lists common error codes and messages reported by mapping data flow
995
1030
-**Cause**: The pipeline expression passed in the Data Flow activity isn't being processed correctly because of a syntax error.
996
1031
-**Recommendation**: Check data flow activity name. Check expressions in activity monitoring to verify the expressions. For example, data flow activity name can't have a space or a hyphen.
997
1032
1033
+
### Error code: 127
1034
+
-**Message**: The spark job of Dataflow completed, but the runtime state is either null or still InProgress..
1035
+
-**Cause**: Transient issue with microservices involved in the execution can cause the run to fail.
1036
+
-**Recommendation**: Refer to [scenario 3 transient issues](#scenario-3-transient-issues).
1037
+
998
1038
### Error code: 2011
999
1039
1000
1040
-**Message**: The activity was running on Azure Integration Runtime and failed to decrypt the credential of data store or compute connected via a Self-hosted Integration Runtime. Please check the configuration of linked services associated with this activity, and make sure to use the proper integration runtime type.
0 commit comments