|
| 1 | +--- |
| 2 | +title: "Error management and troubleshooting for data integration projects" |
| 3 | +description: "Troubleshoot issues and manage errors when executing data integration projects using the Data Integrator service." |
| 4 | +ms.topic: conceptual |
| 5 | +ms.component: pa-admin |
| 6 | +ms.date: 11/19/2021 |
| 7 | +author: NHelgren |
| 8 | +ms.subservice: admin |
| 9 | +ms.author: nhelgren |
| 10 | +ms.reviewer: sericks |
| 11 | +search.audienceType: |
| 12 | + - admin |
| 13 | +--- |
| 14 | +# Data Integrator error management and troubleshooting |
| 15 | + |
| 16 | +The Data Integrator is a point-to-point integration service used to integrate data from multiple sources--for example, finance and operations apps, Dynamics 365 Sales, Salesforce, and Microsoft SQL (Preview)--into Microsoft Dataverse. It also supports integrating data into finance and operations apps and Dynamics 365 Sales. The [Integrate data into Dataverse](/power-platform/admin/data-integrator) topic provides detailed step-by-step instructions to help you set up projects for process-based integration scenarios like Prospect to Cash, Field Service, and Project Service integrations. |
| 17 | + |
| 18 | +While we are constantly evolving and driving fixes into the platform based on customer feedback, we understand there is a need to provide guidance when you run into issues. This topic walks you through error management and troubleshooting some of these issues. |
| 19 | + |
| 20 | +## Supported regions |
| 21 | + |
| 22 | +The Data Integrator is supported for the following regions: |
| 23 | +- Asia |
| 24 | +- Australia |
| 25 | +- Canada |
| 26 | +- Europe |
| 27 | +- India |
| 28 | +- Japan |
| 29 | +- South America |
| 30 | +- United Kingdom |
| 31 | +- United States |
| 32 | + |
| 33 | +There are no plans to enable Data Integrator for any other region. |
| 34 | + |
| 35 | +## View health of project executions |
| 36 | + |
| 37 | +Every time a data integration project is executed (manually or scheduled), you can view the status of the execution on the admin dashboard and/or the project list page. |
| 38 | + |
| 39 | +> [!div class="mx-imgBorder"] |
| 40 | +>  |
| 41 | +
|
| 42 | +The admin dashboard provides a one-stop real-time view of all your project runs and their status with a drill-down to view execution details. The dashboard shows you the individual and summarized count of executions. These are color-coded to show the status of each project: green for completed projects, yellow for completed projects with warnings, and red for projects with an error status. Similarly, the green, yellow, and red icons on the project list page indicate the status of your projects. |
| 43 | + |
| 44 | +Additionally, to view more details, you can drill through project executions via the admin dashboard by selecting individual bar charts. |
| 45 | + |
| 46 | +> [!div class="mx-imgBorder"] |
| 47 | +>  |
| 48 | +
|
| 49 | +Now you can drill through individual errors. |
| 50 | + |
| 51 | +> [!div class="mx-imgBorder"] |
| 52 | +>  |
| 53 | +
|
| 54 | +You can also view project execution details by selecting the individual projects on the project list page and viewing the historical executions and status on the **Execution history** tab. |
| 55 | + |
| 56 | +> [!div class="mx-imgBorder"] |
| 57 | +>  |
| 58 | +
|
| 59 | +If you get a warning or error, you can drill down more by clicking through the executions on the **Execution history** tab. |
| 60 | + |
| 61 | +> [!div class="mx-imgBorder"] |
| 62 | +>  |
| 63 | +
|
| 64 | +## Project monitoring |
| 65 | + |
| 66 | +We highly encourage our customers and partners to subscribe to email-based notifications so you receive email alerts on project executions that completed with either warnings or errors. For each project, on the **Scheduling** tab, you can select email-based notifications and provide multiple email addresses (including group addresses), separated by commas. |
| 67 | + |
| 68 | +> [!div class="mx-imgBorder"] |
| 69 | +>  |
| 70 | +
|
| 71 | +Any time a project completes with a warning or is in the error state, you get an email notification indicating the project execution status with a drillthrough link to the specific failure. |
| 72 | + |
| 73 | +> [!div class="mx-imgBorder"] |
| 74 | +>  |
| 75 | +
|
| 76 | +Selecting the link takes you directly to your project execution status, which you can further drill through for specific errors. |
| 77 | + |
| 78 | +> [!div class="mx-imgBorder"] |
| 79 | +>  |
| 80 | +
|
| 81 | +## Project execution status |
| 82 | + |
| 83 | +When a data integration project is executed (manually or scheduled), it creates a detailed log with project name, a time stamp showing the last update, and the project status. |
| 84 | + |
| 85 | +Each project execution is marked with the status Completed, Warning, or Error: |
| 86 | + |
| 87 | +* **Completed** |
| 88 | + |
| 89 | + Status if all records were upserted successfully. ("Upsert" or "update insert" is a logic to either update the record, if it already exists, or to insert a new record.) |
| 90 | + |
| 91 | + > [!div class="mx-imgBorder"] |
| 92 | + >  |
| 93 | +
|
| 94 | +* **Warning** |
| 95 | + |
| 96 | + Status if some records were upserted successfully, while some failed or errored out. |
| 97 | + |
| 98 | + > [!div class="mx-imgBorder"] |
| 99 | + >  |
| 100 | +
|
| 101 | +* **Error** |
| 102 | + |
| 103 | + Status if none of the records were successful and/or errored out, and there were no upserts or inserts in the destination. |
| 104 | + |
| 105 | + > [!div class="mx-imgBorder"] |
| 106 | + >  |
| 107 | +
|
| 108 | + If the project execution is in the **Error** state, then it will automatically retry execution at the next scheduled run. If you do not update the source record, it will not automatically be included in the next run. |
| 109 | + |
| 110 | +You can also manually retry an execution by selecting **Re-run execution** via the ellipsis (...) on the **Execution history** page. |
| 111 | + |
| 112 | +## Quick tips on troubleshooting common scenarios |
| 113 | + |
| 114 | +Here are some quick tips that will help you troubleshoot some of the common scenarios. |
| 115 | + |
| 116 | +### Connection or environment issues |
| 117 | + |
| 118 | +If you are unable to see your connections or environments in the drop-down while trying to create a Connection set, here are some of the things you can do to troubleshoot the issue: |
| 119 | + |
| 120 | +> [!div class="mx-imgBorder"] |
| 121 | +>  |
| 122 | +
|
| 123 | +* **Connection**: Ensure you have created your connections under Data/Connections on [https://make.powerapps.com ](https://make.powerapps.com) and that they are in the **Connected** state. If you see a **Fix Connection** notification, you should double-check the credentials used for the account, and use the **Switch account** option from the ellipsis (...) to reauthenticate. |
| 124 | + |
| 125 | + > [!div class="mx-imgBorder"] |
| 126 | + >  |
| 127 | +
|
| 128 | +* **Environment**: If you don't see your environments in the drop-down, ensure that the account you used to create the connections has the appropriate access to the entity. A good way to test this is by creating a *flow* (using Microsoft Power Automate). |
| 129 | + |
| 130 | + Here is an example of creating a simple flow to test your connection to finance and operations apps: |
| 131 | + |
| 132 | + 1. Create a new flow (choose **Create from blank**) under **Business logic/Flow** from [https://make.powerapps.com](https://make.powerapps.com). |
| 133 | + |
| 134 | + > [!div class="mx-imgBorder"] |
| 135 | + >  |
| 136 | +
|
| 137 | + 2. Select a **Recurrence** trigger. Under **New Step**, search for and select **Dynamics 365 finance and operations apps connector**. |
| 138 | + |
| 139 | + > [!div class="mx-imgBorder"] |
| 140 | + >  |
| 141 | +
|
| 142 | + 3. Select **Create record** as an action. In the drop-down, ensure that you are logged in with the appropriate account. This is the same account you use to create a connection for your data integration projects. |
| 143 | + |
| 144 | + > [!div class="mx-imgBorder"] |
| 145 | + >  |
| 146 | +
|
| 147 | + 4. Select the drop-down under **environment** to show all the finance and operations apps environments. This is a good step to verify that your account (from the previous step) has access to the environments. |
| 148 | + |
| 149 | + > [!div class="mx-imgBorder"] |
| 150 | + >  |
| 151 | +
|
| 152 | + 5. Once you have picked your environment, confirm that you have access to all the entities under it. |
| 153 | + |
| 154 | + > [!div class="mx-imgBorder"] |
| 155 | + >  |
| 156 | +
|
| 157 | +* **Organizations**: This is where you would specify the legal entity (for example, USMF) for finance and operations apps, the business unit for Dynamics 365 Sales, or the Dataverse organization name. If you miss this step, you get a message that contains valid names corresponding to your application that you then need to plug in under Organizations. |
| 158 | + |
| 159 | +### Project validation errors |
| 160 | + |
| 161 | +First, you validate a data integration project, and then execute it. Some of the top reasons for validation errors include: |
| 162 | + |
| 163 | +* Incorrect company/business unit selected during project creation |
| 164 | +* Missing mandatory columns |
| 165 | +* Incomplete or duplicate mapping |
| 166 | +* Field type mismatch |
| 167 | + |
| 168 | +Here is an example of how the error manifests in the case of duplicate mapping. The orange banner indicates mapping issues. |
| 169 | + |
| 170 | +> [!div class="mx-imgBorder"] |
| 171 | +>  |
| 172 | +
|
| 173 | +When you drill further into the project execution history, you see there is a duplicate field issue. |
| 174 | + |
| 175 | +> [!div class="mx-imgBorder"] |
| 176 | +>  |
| 177 | +
|
| 178 | +When you inspect the mapping, you can identify duplicates. In this case, the source field **fax** is incorrectly mapped to ADDRESSCITY. |
| 179 | + |
| 180 | +> [!div class="mx-imgBorder"] |
| 181 | +>  |
| 182 | +
|
| 183 | +Once you fix the mapping, the error should go away, and you should be able to execute the project successfully. |
| 184 | + |
| 185 | +> [!div class="mx-imgBorder"] |
| 186 | +>  |
| 187 | +
|
| 188 | +### Project execution issues |
| 189 | + |
| 190 | +If you are notified of a project execution that completed with a warning or is in an error state, the first step is to drill into the execution history. From the project list page, select the individual project and review the latest execution on the **Execution history** tab. You can then click through to the specific error. |
| 191 | + |
| 192 | +> [!div class="mx-imgBorder"] |
| 193 | +>  |
| 194 | +
|
| 195 | +If this is an integration project where finance and operations apps is the source, go to the Data Management workspace in finance and operations apps. Then filter projects based on your data integration project name, or specifically choose the type of import or export job. |
| 196 | + |
| 197 | +> [!div class="mx-imgBorder"] |
| 198 | +>  |
| 199 | +
|
| 200 | +Additionally, you can open the job history of the project and drill through the job ID based on the time stamp of your execution. You can also inspect the execution log, view historical runs, and view the staging data. |
| 201 | + |
| 202 | +> [!div class="mx-imgBorder"] |
| 203 | +>  |
| 204 | +
|
| 205 | + |
| 206 | + |
| 207 | + |
| 208 | + |
| 209 | + |
| 210 | +[!INCLUDE[footer-include](../../includes/footer-banner.md)] |
0 commit comments