Skip to content

Commit df36bcf

Browse files
Copilotkobenguyent
Copilot
and
kobenguyent
committed
feat: Add configurable sensitive data masking with custom patterns (#5109)
* Initial plan * Complete sensitive data masking feature with custom patterns and documentation * fix: runner tests --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: kobenguyent <[email protected]>
1 parent e371c09 commit df36bcf

File tree

10 files changed

+452
-35
lines changed

10 files changed

+452
-35
lines changed

‎docs/secrets.md‎

Lines changed: 125 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,36 +1,150 @@
11
# Secrets
22

3-
It is possible to **mask out sensitive data** when passing it to steps. This is important when filling password fields, or sending secure keys to API endpoint.
3+
It is possible to **mask out sensitive data** when passing it to steps. This is important when filling password fields, or sending secure keys to API endpoint. CodeceptJS provides two approaches for masking sensitive data:
4+
5+
## 1. Using the `secret()` Function
46

57
Wrap data in `secret` function to mask sensitive values in output and logs.
68

79
For basic string `secret` just wrap a value into a string:
810

911
```js
10-
I.fillField('password', secret('123456'));
12+
I.fillField('password', secret('123456'))
1113
```
1214

1315
When executed it will be printed like this:
1416

1517
```
1618
I fill field "password" "*****"
1719
```
20+
1821
**Other Examples**
22+
1923
```js
20-
I.fillField('password', secret('123456'));
21-
I.append('password', secret('123456'));
22-
I.type('password', secret('123456'));
24+
I.fillField('password', secret('123456'))
25+
I.append('password', secret('123456'))
26+
I.type('password', secret('123456'))
2327
```
2428

2529
For an object, which can be a payload to POST request, specify which fields should be masked:
2630

2731
```js
28-
I.sendPostRequest('/login', secret({
29-
name: 'davert',
30-
password: '123456'
31-
}, 'password'))
32+
I.sendPostRequest(
33+
'/login',
34+
secret(
35+
{
36+
name: 'davert',
37+
password: '123456',
38+
},
39+
'password',
40+
),
41+
)
42+
```
43+
44+
The object created from `secret` is as Proxy to the object passed in. When printed password will be replaced with \*\*\*\*.
45+
46+
> ⚠️ Only direct properties of the object can be masked via `secret`
47+
48+
## 2. Global Sensitive Data Masking
49+
50+
CodeceptJS can automatically mask sensitive data in all output (logs, steps, debug messages, errors) using configurable patterns. This feature uses the `maskSensitiveData` configuration option.
51+
52+
### Basic Usage (Boolean)
53+
54+
Enable basic masking with predefined patterns:
55+
56+
```js
57+
// codecept.conf.js
58+
exports.config = {
59+
// ... other config
60+
maskSensitiveData: true,
61+
}
62+
```
63+
64+
This will mask common sensitive data patterns like:
65+
66+
- Authorization headers
67+
- API keys
68+
- Passwords
69+
- Tokens
70+
- Client secrets
71+
72+
### Advanced Usage (Custom Patterns)
73+
74+
Define your own masking patterns:
75+
76+
```js
77+
// codecept.conf.js
78+
exports.config = {
79+
// ... other config
80+
maskSensitiveData: {
81+
enabled: true,
82+
patterns: [
83+
{
84+
name: 'Email',
85+
regex: /(\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b)/gi,
86+
mask: '[MASKED_EMAIL]',
87+
},
88+
{
89+
name: 'Credit Card',
90+
regex: /\b(?:\d{4}[- ]?){3}\d{4}\b/g,
91+
mask: '[MASKED_CARD]',
92+
},
93+
{
94+
name: 'Phone Number',
95+
regex: /(\+?1[-.\s]?)?\(?([0-9]{3})\)?[-.\s]?([0-9]{3})[-.\s]?([0-9]{4})/g,
96+
mask: '[MASKED_PHONE]',
97+
},
98+
{
99+
name: 'SSN',
100+
regex: /\b\d{3}-\d{2}-\d{4}\b/g,
101+
mask: '[MASKED_SSN]',
102+
},
103+
],
104+
},
105+
}
106+
```
107+
108+
### Pattern Configuration
109+
110+
Each custom pattern object should have:
111+
112+
- `name`: A descriptive name for the pattern
113+
- `regex`: A JavaScript regular expression to match the sensitive data
114+
- `mask`: The replacement string to show instead of the sensitive data
115+
116+
### Examples
117+
118+
With the above configuration:
119+
120+
**Input:**
121+
32122
```
123+
User email: [email protected]
124+
Credit card: 4111 1111 1111 1111
125+
Phone: +1-555-123-4567
126+
```
127+
128+
**Output:**
129+
130+
```
131+
User email: [MASKED_EMAIL]
132+
Credit card: [MASKED_CARD]
133+
Phone: [MASKED_PHONE]
134+
```
135+
136+
### Where Masking Applies
137+
138+
Global sensitive data masking is applied to:
139+
140+
- Step descriptions and output
141+
- Debug messages (`--debug` mode)
142+
- Log messages (`--verbose` mode)
143+
- Error messages
144+
- Success messages
145+
146+
> ⚠️ Direct `console.log()` calls in helper functions are not masked. Use CodeceptJS output functions instead.
33147
34-
The object created from `secret` is as Proxy to the object passed in. When printed password will be replaced with ****.
148+
### Combining Both Approaches
35149

36-
> ⚠️ Only direct properties of the object can be masked via `secret`
150+
You can use both `secret()` function and global masking together. The `secret()` function is applied first, then global patterns are applied to the remaining output.

‎lib/output.js‎

Lines changed: 8 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
const colors = require('chalk')
22
const figures = require('figures')
3-
const { maskSensitiveData } = require('invisi-data')
3+
const { maskData, shouldMaskData, getMaskConfig } = require('./utils/mask_data')
44

55
const styles = {
66
error: colors.bgRed.white.bold,
@@ -59,7 +59,7 @@ module.exports = {
5959
* @param {string} msg
6060
*/
6161
debug(msg) {
62-
const _msg = isMaskedData() ? maskSensitiveData(msg) : msg
62+
const _msg = shouldMaskData() ? maskData(msg, getMaskConfig()) : msg
6363
if (outputLevel >= 2) {
6464
print(' '.repeat(this.stepShift), styles.debug(`${figures.pointerSmall} ${_msg}`))
6565
}
@@ -70,7 +70,7 @@ module.exports = {
7070
* @param {string} msg
7171
*/
7272
log(msg) {
73-
const _msg = isMaskedData() ? maskSensitiveData(msg) : msg
73+
const _msg = shouldMaskData() ? maskData(msg, getMaskConfig()) : msg
7474
if (outputLevel >= 3) {
7575
print(' '.repeat(this.stepShift), styles.log(truncate(` ${_msg}`, this.spaceShift)))
7676
}
@@ -81,15 +81,17 @@ module.exports = {
8181
* @param {string} msg
8282
*/
8383
error(msg) {
84-
print(styles.error(msg))
84+
const _msg = shouldMaskData() ? maskData(msg, getMaskConfig()) : msg
85+
print(styles.error(_msg))
8586
},
8687

8788
/**
8889
* Print a successful message
8990
* @param {string} msg
9091
*/
9192
success(msg) {
92-
print(styles.success(msg))
93+
const _msg = shouldMaskData() ? maskData(msg, getMaskConfig()) : msg
94+
print(styles.success(_msg))
9395
},
9496

9597
/**
@@ -124,7 +126,7 @@ module.exports = {
124126
stepLine += colors.grey(step.comment.split('\n').join('\n' + ' '.repeat(4)))
125127
}
126128

127-
const _stepLine = isMaskedData() ? maskSensitiveData(stepLine) : stepLine
129+
const _stepLine = shouldMaskData() ? maskData(stepLine, getMaskConfig()) : stepLine
128130
print(' '.repeat(this.stepShift), truncate(_stepLine, this.spaceShift))
129131
},
130132

@@ -278,7 +280,3 @@ function truncate(msg, gap = 0) {
278280
}
279281
return msg
280282
}
281-
282-
function isMaskedData() {
283-
return global.maskSensitiveData === true || false
284-
}

‎lib/utils/mask_data.js‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
const { maskSensitiveData } = require('invisi-data')
2+
3+
/**
4+
* Mask sensitive data utility for CodeceptJS
5+
* Supports both boolean and object configuration formats
6+
*
7+
* @param {string} input - The string to mask
8+
* @param {boolean|object} config - Masking configuration
9+
* @returns {string} - Masked string
10+
*/
11+
function maskData(input, config) {
12+
if (!config) {
13+
return input
14+
}
15+
16+
// Handle boolean config (backward compatibility)
17+
if (typeof config === 'boolean' && config === true) {
18+
return maskSensitiveData(input)
19+
}
20+
21+
// Handle object config with custom patterns
22+
if (typeof config === 'object' && config.enabled === true) {
23+
const customPatterns = config.patterns || []
24+
return maskSensitiveData(input, customPatterns)
25+
}
26+
27+
return input
28+
}
29+
30+
/**
31+
* Check if masking is enabled based on global configuration
32+
*
33+
* @returns {boolean|object} - Current masking configuration
34+
*/
35+
function getMaskConfig() {
36+
return global.maskSensitiveData || false
37+
}
38+
39+
/**
40+
* Check if data should be masked
41+
*
42+
* @returns {boolean} - True if masking is enabled
43+
*/
44+
function shouldMaskData() {
45+
const config = getMaskConfig()
46+
return config === true || (typeof config === 'object' && config.enabled === true)
47+
}
48+
49+
module.exports = {
50+
maskData,
51+
getMaskConfig,
52+
shouldMaskData,
53+
}

‎test/data/sandbox/codecept.bdd.boolean-masking.js‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
exports.config = {
2+
tests: './*_no_test.js',
3+
timeout: 10000,
4+
output: './output',
5+
helpers: {
6+
BDD: {
7+
require: './support/bdd_helper.js',
8+
},
9+
},
10+
// Traditional boolean masking configuration
11+
maskSensitiveData: true,
12+
gherkin: {
13+
features: './features/secret.feature',
14+
steps: ['./features/step_definitions/my_steps.js', './features/step_definitions/my_other_steps.js'],
15+
},
16+
include: {},
17+
bootstrap: false,
18+
mocha: {},
19+
name: 'sandbox-boolean-masking',
20+
}

‎test/data/sandbox/codecept.bdd.masking.js‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
exports.config = {
2+
tests: './*_no_test.js',
3+
timeout: 10000,
4+
output: './output',
5+
helpers: {
6+
BDD: {
7+
require: './support/bdd_helper.js',
8+
},
9+
},
10+
// New masking configuration with custom patterns
11+
maskSensitiveData: {
12+
enabled: true,
13+
patterns: [
14+
{
15+
name: 'Email',
16+
regex: /(\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b)/gi,
17+
mask: '[MASKED_EMAIL]',
18+
},
19+
{
20+
name: 'Credit Card',
21+
regex: /\b(?:\d{4}[- ]?){3}\d{4}\b/g,
22+
mask: '[MASKED_CARD]',
23+
},
24+
{
25+
name: 'Phone',
26+
regex: /(\+?1[-.\s]?)?\(?([0-9]{3})\)?[-.\s]?([0-9]{3})[-.\s]?([0-9]{4})/g,
27+
mask: '[MASKED_PHONE]',
28+
},
29+
],
30+
},
31+
gherkin: {
32+
features: './features/masking.feature',
33+
steps: ['./features/step_definitions/my_steps.js', './features/step_definitions/my_other_steps.js'],
34+
},
35+
include: {},
36+
bootstrap: false,
37+
mocha: {},
38+
name: 'sandbox-masking',
39+
}

‎test/data/sandbox/features/masking.feature‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
Feature: Custom Data Masking
2+
3+
Scenario: mask custom sensitive data in output
4+
Given I have user email "[email protected]"
5+
And I have credit card "4111 1111 1111 1111"
6+
And I have phone number "+1-555-123-4567"
7+
When I process user data
8+
Then I should see masked output

‎test/data/sandbox/features/step_definitions/my_steps.js‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,29 @@ Given('I login', () => {
3535
I.login('user', secret('password'))
3636
})
3737

38+
Given('I have user email {string}', email => {
39+
I.debug(`User email is: ${email}`)
40+
I.say(`Processing email: ${email}`)
41+
})
42+
43+
Given('I have credit card {string}', card => {
44+
I.debug(`Credit card is: ${card}`)
45+
I.say(`Processing card: ${card}`)
46+
})
47+
48+
Given('I have phone number {string}', phone => {
49+
I.debug(`Phone number is: ${phone}`)
50+
I.say(`Processing phone: ${phone}`)
51+
})
52+
53+
When('I process user data', () => {
54+
I.debug('Processing user data with sensitive information')
55+
})
56+
57+
Then('I should see masked output', () => {
58+
I.debug('All sensitive data should be masked in output')
59+
})
60+
3861
Given(/^I have this product in my cart$/, table => {
3962
let str = ''
4063
for (const id in table.rows) {

0 commit comments

Comments
 (0)