|
| 1 | +# Solution Pattern - Data Modelling - Presentation Layer |
| 2 | + |
| 3 | +## Purpose |
| 4 | +This Implementation Pattern describes the data modelling conventions and architecture for the Presentation Layer. |
| 5 | +Structure |
| 6 | +In principle, there are two mechanisms towards preparing information for consumption in the Presentation Layer (created in the Presentation Layer database): |
| 7 | +Direct view on top of the Integration Layer (virtual information mart). In by far the most scenarios the first option (direct view / virtual) option is preferred as the subsequent layers in the (BI) architecture are typically MOLAP or in-memory. |
| 8 | +Table / persistence / physical storage using a view to join and prepare the data in the format that matches the table (logic) and can be used to incrementally load the table. |
| 9 | +In both cases these Presentation Layer objects will require one or more views to decouple the Business Intelligence (BI) and Data Warehouse (DWH) environments. These decoupling views are also intended to apply the history perspective at attribute level; e.g. how every attribute is displayed in time (e.g. Type1, Type2, Type 6). |
| 10 | +The following are general guidelines: |
| 11 | +The logic views and tables contain all history (Type2) by default. Any interpretation of history, such as ‘current state view’, can be queried using the decoupling views. |
| 12 | +There may be multiple decoupling views, but two views is the guideline, to represent history using different perspectives (e.g. current state / Type1 or mixed / Type2). |
| 13 | +The decoupling views are faced towards the BI / business side and aims to present information in the way it is easiest to consume. |
| 14 | +The decoupling views should be generated from metadata, and use Extended Properties defined at the view logic view to generate an accurate representation of history. |
| 15 | +The logic views used to populate tables are geared towards ETL and follows the rigor of naming conventions to support automation. BIML scripts are available to generate SSIS packages from the logic view to the Presentation Layer tables. |
| 16 | +The pres schema is the enterprise information / mart schema and contains the decoupling views, so this contains what is effectively the complete dimensional model exposed to the BI environment. This also allows the decoupling views to have the same name as the accompanying Dimension or Fact table. |
| 17 | +The ben schema contains the logic views and tables since objects (views and tables in this case) cannot be named identically (as they would be in the pres schema). The views are named with the ‘_VW’ suffix. |
| 18 | +Normal casing is used, with underscores (no spaces) for all tables and attributes. |
| 19 | +Definitions are maintained in the Confluence Business Glossary. |
| 20 | +Logic views are primarily manually developed (with some history merge scripts to assist) as these views handle the change from data handling to business use. |
| 21 | +Tables and decoupling views are generated from metadata. |
| 22 | +Decoupling views are used to expose history using additional metadata (‘extended properties’). |
| 23 | +The extract schema is used for data provision to support external systems (e.g. non-BI) and is therefore considered not to be part of the standard Presentation Layer. |
| 24 | +There is also a va schema which is specifically there to expose information to SAS Visual Analytics. |
| 25 | +There also is a temp schema which is strictly only used to store ETL required information / to support the performance and workings of the ETL. |
| 26 | +This is displayed in the following diagram: |
| 27 | + |
| 28 | +The modelling conventions for the Presentation Layer tables are outlined in the table below. |
| 29 | +Table Type |
| 30 | +Table name convention |
| 31 | +Mandatory attribute |
| 32 | +Comments |
| 33 | +Dimension |
| 34 | +ben.DIM_<name> |
| 35 | +table name>_SK |
| 36 | +OMD_INSERT_MODULE_INSTANCE_ID |
| 37 | +OMD_DELETED_RECORD_INDICATOR |
| 38 | +OMD_UPDATE_MODULE_INSTANCE_ID |
| 39 | +OMD_CHECKSUM_TYPE_1 |
| 40 | +OMD_CHECKSUM_TYPE_2 |
| 41 | +OMD_EFFECTIVE_DATETIME |
| 42 | +OMD_EXPIRY_DATETIME |
| 43 | +OMD_CURRENT_RECORD_INDICATOR |
| 44 | +<attributes> |
| 45 | +The first attribute (SK) is the primary key, and is a hash value (32 byte character) |
| 46 | +Optionally, a unique key / index is placed on the combination of level natural keys and the OMD_EFFECTIVE_DATETIME. This represents a unique point in time record. See the consequences section for more details |
| 47 | +Every attribute is specified as Type 0, Type 1, Type 2 (can be combined to type 3 or 6 - check the relevant pattern). This is specified in the model / database as an extended property |
| 48 | +Fact Table |
| 49 | +ben.FACT_<name> |
| 50 | +<table name>_SK |
| 51 | +<Dimension Keys> |
| 52 | +OMD_INSERT_MODULE_INSTANCE_ID |
| 53 | +OMD_INSERT_DATETIME |
| 54 | +OMD_INSERT_MODULE_INSTANCE_ID |
| 55 | +OMD_DELETED_RECORD_INDICATOR |
| 56 | +OMD_UPDATE_MODULE_INSTANCE_ID |
| 57 | +OMD_CHECKSUM_TYPE_1 |
| 58 | +OMD_CHECKSUM_TYPE_2 |
| 59 | +OMD_EFFECTIVE_DATETIME |
| 60 | +OMD_EXPIRY_DATETIME |
| 61 | +OMD_CURRENT_RECORD_INDICATOR |
| 62 | +<attributes> |
| 63 | +The first attribute (SK) is the primary key |
| 64 | +A unique key / index is placed on the combination of Dimension keys. |
| 65 | +Other |
| 66 | +ben.<name> |
| 67 | +<table name>_SK |
| 68 | +OMD_INSERT_MODULE_INSTANCE_ID |
| 69 | +OMD_INSERT_DATETIME |
| 70 | +<any OMD attributes required> |
| 71 | +<attributes> |
| 72 | +Not every delivery of information is necessarily in the form of a Star Schema / Dimensional Model. If a dataset is better delivered in a different format (wide table, normalised) this is preferred. |
| 73 | + |
| 74 | +The modelling conventions for the Presentation Layer views are outlined in the table below. |
| 75 | +View Type |
| 76 | +Table name convention |
| 77 | +Mandatory attribute |
| 78 | +Comments |
| 79 | +Logic View |
| 80 | +ben.<name>_VW |
| 81 | +<view name>_SK |
| 82 | +OMD_INSERT_MODULE_INSTANCE_ID |
| 83 | +OMD_INSERT_DATETIME |
| 84 | +OMD_INSERT_MODULE_INSTANCE_ID |
| 85 | +OMD_DELETED_RECORD_INDICATOR |
| 86 | +OMD_UPDATE_MODULE_INSTANCE_ID |
| 87 | +OMD_CHECKSUM_TYPE_1 |
| 88 | +OMD_CHECKSUM_TYPE_2 |
| 89 | +OMD_EFFECTIVE_DATETIME |
| 90 | +OMD_EXPIRY_DATETIME |
| 91 | +OMD_CURRENT_RECORD_INDICATOR |
| 92 | +<attributes> |
| 93 | +Used to load a standard Dimension or Fact table supported by BIML. |
| 94 | +The name of the view needs to match the name of the target table (except for the _VW suffix) |
| 95 | +The _VW suffix is required as there may be a table with the original name in the ben schema |
| 96 | +The checksums for Type 1 and Type 2 calculations will be handled by the BIML, and do not need to be present in the views. This allows for a more automated update if required |
| 97 | +All other OMD attributes required in the target table are handled by the BIML scripts |
| 98 | +Decoupling View |
| 99 | +pres.<name> (for regular views) |
| 100 | +pres.<name>_history (for history or mixed-history views) |
| 101 | +Underlying ‘ben’ table or logic view, but without OMD attributes. |
| 102 | +Surrogate keys optional. |
| 103 | +Business-facing, e.g. DIM_CUSTOMER, or DIM_CUSTOMER_HISTORY. |
| 104 | + |
| 105 | +Related Design Patterns |
| 106 | +Design Pattern 002 - Generic - Types of History |
| 107 | +Consequences |
| 108 | +Related to having Data Vault Surrogate (Hub) Keys (SK) in the Dimensional Model: it is OK to add Hub keys (Surrogate Keys) in the Presentation Layer for tracing and auditability purposes. However they cannot be adequately used as level keys as a level in a Dimension may not 100% map a business concept. For instance a 'business unit type' may not be modelled as a Hub in the Data Vault, but could be a level in a Dimension. By using Hub Keys for Dimension lookups a dependancy between the Integration and Presentation Layers is created that should be avoided. An example is where you have Business Unit Type, State, Counter and Ownership in the same Satellite (e.g. SAT_BUSINESS_UNIT). If these attributes are modelled in separate Dimensions in the Presentation Layer the Hub Key (from HUB_BUSINESS_UNIT) cannot be used, rather a separate Dimension Key must be created and a dedicated natural key must be selected appropriate for the dimension. In other words, lookups and constraints should be using natural keys. |
| 109 | +Discussion items (not yet to be implemented or used until final) |
| 110 | +None. |
0 commit comments