Skip to content

Feat/ghost records #53

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
DVJocelynShannon wants to merge 3 commits into
base: develop
Choose a base branch
from
Open

Feat/ghost records #53

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
da0708f
Extra GR docs
DVJocelynShannon Dec 30, 2022
95af473
Add updates pit code for BQ and SF
DVJocelynShannon Feb 24, 2023
7fe857a
PIT code for SQLServer and altered SF/BQ code
DVJocelynShannon Feb 24, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions docs/best_practices.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -427,3 +427,29 @@ input gives same output). Until we provide more configuration options, feel free
long as you stick to a standard that makes sense to you or your organisation. If you need
advice, [feel free to join our slack and ask our developers](https://join.slack.com/t/dbtvault/shared_invite/enQtODY5MTY3OTIyMzg2LWJlZDMyNzM4YzAzYjgzYTY0MTMzNTNjN2EyZDRjOTljYjY0NDYyYzEwMTlhODMzNGY3MmU2ODNhYWUxYmM2NjA)!

## Ghost Records

### What are ghost records?

Ghost records are 'dummy' records that are implemented in satellite tables. In Data Vault 1.0 the best practice was to
have a separate ghost record for each key. This was changed in Data Vault 2.0 and so now there is a single ghost records
inserted at the top of every satellite table.

The ghost records enable the PIT tables to have a complete set of historic data. This means if a key was introduced
after the initial creation or the keys are not updated at the same frequency, there will still be data in the table
instead of missing the record.

### What does a ghost record look like?

| Column | Datatype | Value |
|----------------------|---------------|------------------------------------------------------------------|
| Primary Key - MD5 | BINARY(16) | 00000000000000000000000000000000 |
| Primary Key - SHA256 | BINARY(32) | 0000000000000000000000000000000000000000000000000000000000000000 |
| Hashdiff - MD5 | BINARY(16) | 00000000000000000000000000000000 |
| Hashdiff - SHA256 | BINARY(32) | 0000000000000000000000000000000000000000000000000000000000000000 |
| Effective From | DATE/DATETIME | 1900-01-01 00:00:00.000000 |
| Load Date | DATE/DATETIME | 1900-01-01 00:00:00.000000 |
| Payload Columns | null | NULL |
| Extra Columns | null | NULL |


Loading