You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -19,9 +19,9 @@ By using GitHub Actions with Bytebase API, you can implement policy-as-code to a
19
19
20
20
This is Part 1 of our tutorial series on implementing automated database masking using GitHub Actions:
21
21
22
-
- Part 1: Column Masking (this one)
23
-
- Part 2: [Masking Algorithm](/docs/tutorials/github-action-data-masking-part2)
24
-
- Part 3: [Data Classification and Global Masking](/docs/tutorials/github-action-data-masking-part3)
22
+
- Part 1: [Semantic Type and Global Masking Rule](/docs/tutorials/github-action-data-masking-part1)
23
+
- Part 2: [Column Masking and Masking Exemption](/docs/tutorials/github-action-data-masking-part2)
24
+
- Part 3: [Data Classification](/docs/tutorials/github-action-data-masking-part3)
25
25
- Part 4: Data Export with Masking (TBD)
26
26
27
27
## Overview
@@ -31,7 +31,7 @@ In this tutorial, you'll learn how to automate database masking policies using G
31
31
- Manage data masking rules as code
32
32
- Automatically apply masking policies when PRs are merged
33
33
34
-
Here is [a merged pull request](https://github.com/bytebase/database-security-github-actions-example/pull/5) as an example.
34
+
Here is [a merged pull request](https://github.com/bytebase/database-security-github-actions-example/pull/81) as an example, for this tutorial, only "Semantic Type and Global Masking Rule" is covered.
35
35
36
36
<HintBlocktype="info">
37
37
@@ -78,56 +78,62 @@ Before you begin, make sure you have:
78
78
79
79
### Step 5 - Understanding the GitHub Workflow
80
80
81
-
Let's dig into the GitHub Actions workflow [code](https://github.com/bytebase/database-security-github-actions-example/blob/main/.github/workflows/bb-masking-1.yml):
81
+
Let's dig into the GitHub Actions workflow [code](https://github.com/bytebase/database-security-github-actions-example/blob/main/.github/workflows/1-bb-masking-semantic-type-global.yml):
82
82
83
83
1.**Trigger**: Workflow runs when PRs are merged to `main`.
84
84
85
85
1.**Authentication**: The step `Login Bytebase` will log in Bytebase using the official [bytebase-login](https://github.com/marketplace/actions/bytebase-login) action. The variables you configured in the GitHub **Secrets and variables** are mapped to the variables in the action.
86
86
87
-
1.**File Detection**: The step `Get changed files` will monitor the changed files in the pull request. For this workflow, we only care about column masking and masking exception. So `masking/databases/**/**/column-masking.json` and `masking/projects/**/masking-exception.json` are filtered out.
87
+
1.**File Detection**: The step `Get changed files` will monitor the changed files in the pull request. For this workflow, we only care about semantic type and global masking rule. So `masking/semantic-type.json` and `masking/global-masking-rule.json` are filtered out.
88
88
89
89
1.**PR Feedback**: The step `Comment on PR` will comment on the merged pull to notify the result.
90
90
91
-
## Column Masking
91
+
## Semantic Type
92
92
93
-
[Column Masking](/docs/security/data-masking/column-masking/)lets you specify table columns different Masking Level to mask the data.
93
+
You may define [semantic types](/docs/security/data-masking/semantic-types/)and apply them to global masking rule or columns of different tables. For example, you may define a semantic type `birth_date` with a masking algorithm to mask month and day.
94
94
95
-
In Bytebase console, go to a database page, then pick a table, you can specify masking level by clicking pen icon on table detail page.
95
+
### In Bytebase Console
96
96
97
-
In the GitHub workflow, find the step `Apply column masking`, which will apply the column masking to the database via API. First it will parse all the column masking files and then do a loop to apply the column masking to the database one by one. The code it calls Bytebase API is as follows:
97
+
Go to **Data Access > Semantic Types** and click **Add**. You can create a new semantic type with a name and description, and customize the masking algorithm.
Find the step `Apply semantic type`, which will apply the semantic type to the database via API. All the masking algorithms should be defined in one file in the root directory as `masking/semantic-type.json`. The code it calls Bytebase API is as follows:
By changing file `masking/databases/**/**/column-masking.json`, create a PR and then merge, the change will be appliedto the database.
114
+
By changing file `masking/semantic-type.json`, creating a PR and merging, the semantic type will be applied. Go to Bytebase console, click **Data Access > Semantic Types**, you can see the applied semantic types.
107
115
108
-
Log in Bytebase console, at the workspace level, click **Data Access > Data Masking**. Click **Explicit Masked Columns**, you can see the column masking is applied to the database.
Go to **Data Access > Global Masking** and click **Add**. You can create a new global masking rule with conditions and semantic type.
113
121
114
-
[Access Unmasked Data](/docs/security/data-masking/access-unmasked-data/) lets you relax the masking levels for the users. Full masked column to partial or partial masked column to none.
In the GitHub workflow, find the step `Apply masking exception`, which will apply the masking exception to the database and the process is similar, the code it calls Bytebase API is as follows:
124
+
### In GitHub Workflow
125
+
126
+
Find the step `Apply global masking rule`, which will apply the global masking rule to the database via API. All the global masking rules should be defined in one file in the root directory as `masking/global-masking-rule.json`. The code it calls Bytebase API is as follows:
By changing file `masking/projects/**/masking-exception.json`, create a PR and then merge, the change will be applied to the database.
126
-
127
-
Log in Bytebase console, go to the project `Sample Project`, click **Database > Masking Access**, you can see the masking exception is applied to the database.
By changing file `masking/global-masking-rule.json`, creating a PR and merge, you can apply the global masking rule to the database. Go to Bytebase console, click **Data Access > Global Masking** page, you can see the global masking rule is applied to the database.
130
136
131
137
## Next Steps
132
138
133
-
Now you have successfully applied data masking policies using GitHub Actions and Bytebase API. In the next part of this tutorial, you'll learn how to customize the masking algorithm. Stay tuned!
139
+
Now you have successfully define semantic types and apply global masking rule using GitHub Actions and Bytebase API. In the next part of this tutorial, you'll learn how to apply column masking and masking exemption. Stay tuned!
In the [previous tutorial](/docs/tutorials/github-action-data-masking-part1), you learned how to set up a GitHub Action that utilizes the Bytebase API to define data masking policies. In this tutorial, we will explore how to customize both the masking algorithm and semantic types.
14
+
In the [previous tutorial](/docs/tutorials/github-action-data-masking-part1), you learned how to set up a GitHub Action that utilizes the Bytebase API to define semantic types and global masking rule. In this tutorial, we will explore how to apply column masking and masking exemption.
15
15
16
16
---
17
17
18
18
This is Part 2 of our tutorial series on implementing automated database masking using GitHub Actions:
19
19
20
-
- Part 1: [Column masking](/docs/tutorials/github-action-data-masking-part1)
21
-
- Part 2: Masking Algorithm (this one)
22
-
- Part 3: [Data Classification and Global Masking](/docs/tutorials/github-action-data-masking-part3)
20
+
- Part 1: [Semantic Type and Global Masking Rule](/docs/tutorials/github-action-data-masking-part1)
21
+
- Part 2: Column Masking and Masking Exemption (this one)
22
+
- Part 3: [Data Classification](/docs/tutorials/github-action-data-masking-part3)
23
23
- Part 4: Data export with masking (TBD)
24
24
25
25
## Overview
26
26
27
-
In this tutorial, you'll learn how to automate database masking algorithms and semantic types using GitHub Actions and the Bytebase API. This integration allows you to:
27
+
In this tutorial, you'll learn how to automate column masking and masking exemption using GitHub Actions and the Bytebase API. This integration allows you to:
28
28
29
29
- Manage data masking rules as code
30
30
- Automatically apply masking policies when PRs are merged
31
31
32
-
Here is [a merged pull request](https://github.com/bytebase/database-security-github-actions-example/pull/18) as an example.
32
+
Here is [a merged pull request](https://github.com/bytebase/database-security-github-actions-example/pull/81) as an example.
33
33
34
34
<HintBlocktype="info">
35
35
@@ -39,52 +39,56 @@ The complete code for this tutorial is available at: [database-security-github-a
39
39
40
40
This tutorial skips the setup part, if you haven't set up the Bytebase and GitHub Action, please follow **Setup Instructions** section in the [previous tutorial](/docs/tutorials/github-action-data-masking-part1).
41
41
42
-
## Masking Algorithm
42
+
## Column Masking
43
43
44
-
You may customize your own [data masking algorithm](/docs/security/data-masking/masking-algorithm/)with the help of a predefined masking type, such as Full mask, Range mask, MD5 mask and Inner/Outer mask.
44
+
[Column Masking](/docs/security/data-masking/column-masking/)lets you specify table columns different semantic type to mask the data.
45
45
46
-
### In Bytebase console
46
+
### In Bytebase Console
47
47
48
-
Go to **Data Access > Data Masking**, click **Masking Algorithm** and click **Add**. You can create a new masking algorithm with a name and description, and later it can be used in the definition of semantic types.
48
+
Go to a database page, then pick a table, you can specify semantic type by clicking pen icon on table detail page.
In the GitHub workflow `bb-masking-2.yml`, find the step `Apply masking algorithm`, which will apply the masking algorithm to the database via API. All the masking algorithms should be defined in one file in the root directory of `masking/masking-algorithm.json`. The code it calls Bytebase API is as follows:
54
+
Find the step `Apply column masking`, which will apply the column masking to the database via API. First it will parse all the column masking files and then do a loop to apply the column masking to the database one by one. The code it calls Bytebase API is as follows:
By changing file `masking/masking-algorithm.json`, you can apply the masking algorithm to the database. Go to Bytebase console, click **Data Access > Data Masking**, go to **Masking Algorithm** page, you can see the masking algorithm is applied to the database.
63
+
By changing file `masking/databases/**/**/database-catalog.json`, create a PR and then merge, the change will be applied to the database.
64
+
65
+
Log in Bytebase console, at the workspace level, go to the database page, you can see the column semantic type is applied to the database.
64
66
65
-
## Semantic Type
67
+
## Masking Exemption
66
68
67
-
You may define [semantic types](/docs/security/data-masking/semantic-types/)and apply them to columns of different tables. Columns with the same semantic type will be masked with the same masking algorithm. For example, you may define a semantic type `mobile` and apply it to all the columns of phone number. Then you can define a masking algorithm `range 4-10`for the partial level masking for semantic type `mobile`.
69
+
[Masking Exemption](/docs/security/data-masking/masking-exemption/)lets you unmask data for specific users.
68
70
69
71
### In Bytebase Console
70
72
71
-
Go to **Data Access > Data Masking**, click **Semantic Types** and click **Add**. You can create a new semantic type with a name and description, and select the masking algorithm.
73
+
Go to a project page, then click **Manage > Masking Exemptions**, you can grant masking exemption to the database.
Find the step `Apply semantic type`, which will apply the semantic type to the database via API. All the masking algorithms should be defined in one file in the root directory as `masking/semantic-type.json`. The code it calls Bytebase API is as follows:
79
+
Find the step `Apply masking exception`, which will apply the masking exception to the database and the process is similar, the code it calls Bytebase API is as follows:
By changing file `masking/semantic-type.json`, you can apply the semantic type to the database. Go to Bytebase console, click **Data Access > Data Masking**, go to **Semantic Types** page, you can see the semantic type is applied to the database.
88
+
By changing file `masking/projects/**/masking-exception.json`, create a PR and then merge, the change will be applied to the database.
89
+
90
+
Log in Bytebase console, go to the project `Sample Project`, click **Manage > Masking Exemptions**, you can see the masking exemption is applied to the database.
87
91
88
92
## Next Steps
89
93
90
-
Now you have successfully applied data masking algorithm and semantic type using GitHub Actions and Bytebase API. In the next part of this tutorial, you'll learn how to use data classification and global masking with GitHub Actions. Stay tuned!
94
+
Now you have successfully applied column masking and masking exemption using GitHub Actions and Bytebase API. In the next part of this tutorial, you'll learn how to use data classification with GitHub Actions. Stay tuned!
0 commit comments