You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: Data Integration Framework - Introduction.md
+5-5Lines changed: 5 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -44,13 +44,13 @@ The Data Integration provides a structured approach to data integration design f
44
44
45
45
To adapt to business needs the intended data solution should decouple ‘warehouse logic’ from ‘business logic’. The basic assumption is that requirements will change over time, and that any solution that specifically is designed for a certain output or requirement will fail over time when adjustments are made. Rather, the framework views requirements from a data perspective and aims to properly integrate and consolidate data before applying a certain view.
The diagram below outlines the components that are required to support a data solution and enable automation. The intent is to enable a standard and structured way for documenting decisions made related to system design and intended operation.
@@ -117,7 +117,7 @@ In this example the Design Pattern would refer to the ‘AGA Data Integration -
117
117
118
118
The Data Integration should be viewed as one part of the larger (enterprise) architecture. The purpose is to specify how the ETL and the data model can be configured for an optimal Enterprise Data Warehouse implementation. This is a detailed (albeit significant) component in the Data Warehouse architecture which in itself includes other components such as system landscape, subject areas and the Business Intelligence and Data domain.
In this overview the basic flow of data as specified in the Reference Architecture is:
89
89
@@ -106,7 +106,7 @@ The following diagram is a bottom-up overview of the detailed steps taken and ch
106
106
107
107
Each layer contains two (2) separate ‘areas’. These areas cover specific ETL functionality that support the overall purpose of the layer of which they are part of. Each area inherits the data modelling approach of the parent layer.
In this diagram Error/Exception handling and Operational Metadata are positioned as applicable for every process in the architecture. The left column of areas in the diagram specifies the mandatory areas (Staging, Integration and Reporting Structure).
112
112
@@ -280,7 +280,7 @@ Every Data Warehouse table contains a predefined set of metadata attributes, whi
280
280
281
281
Error handling and exception is applicable to every layer and area in the architecture. Each individual layer definition document will describe how error handling is used for that particular section of the architecture since the exception handling is very different between layers. The error handling and recycling document itself lists and explains the concepts that can be used and will provide an overview of the complete error and exception handling solution.
282
282
283
-

283
+

284
284
285
285
The details for error and exception handling are defined in the ETL Framework - 7 - Exception Handling v1.0’ document.
286
286
@@ -291,7 +291,7 @@ Similar to exception and error handling concepts the Metadata Model links in wit
291
291
292
292
The Metadata Model document itself will list and explain the available concepts and will provide an overview of the complete framework.
293
293
294
-

294
+

295
295
296
296
In general the metadata process model supports the ability to trace back what data has been loaded, when and in what way for every interface. A single attribute in any place in the architecture should be auditable back to the originating source system.
0 commit comments