Skip to content

Commit 80c8c43

Browse files
cademcadem
authored
feat/6641435300-save-audit-in-self (#34738)
1 parent f9bcda5 commit 80c8c43

File tree

35 files changed

+1238
-56
lines changed

35 files changed

+1238
-56
lines changed

‎docs/en/08-operation/16-security.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,8 @@ Additionally, for an audit database:
9999

100100
Audit databases created before version 3.4.0.0 are incompatible with audit features in version 3.4.0.0 and later. In older versions, the `is_audit` parameter was not enforced, so there were no mandatory requirements for `DURATION`, `WAL_LEVEL`, and `ENCRYPT_ALGORITHM`. To enable new audit features for an old audit database, it must be dropped and recreated. As a workaround to access data from a pre-3.4.0.0 audit database in a newer version (without new audit features), you can disable `auditUseToken` by setting it to 0.
101101

102+
In versions 3.4.1.0 and later, audit information can be saved locally rather than sent to taoskeeper. To use this functionality, you need to set the parameter auditSaveInSelf to 1, and the number of vgroups in the audit database created must be limited to one.
103+
102104
### taosKeeper Configuration
103105

104106
Configure the related parameters for audit logs in the taosKeeper configuration file `keeper.toml`, as shown in the table below

‎docs/en/14-reference/01-components/01-taosd.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -243,6 +243,7 @@ The effective value of charset is UTF-8.
243243
| auditHttps | After 3.4.0.0 | Supported, effective immediately | Whether to use https to report audit data; Enterprise parameter; range 0 - 1, default value 0 (1: use https, 0: do not use). |
244244
| auditUseToken | After 3.4.0.0 | Supported, effective immediately | Whether to use token to report audit data; Enterprise parameter; range 0 - 1, default value 1 (1: use token, 0: do not use). |
245245
| auditCreateTable | | Supported, effective immediately | Whether to enable audit feature for creating subtables; Enterprise parameter |
246+
| auditSaveInSelf | After 3.4.1.0 | Supported, effective immediately | Whether to save audit information locally instead of sending it to taoskeeper. Range: 0-1, default: 0 (1: enabled, 0: disabled). |
246247
| encryptAlgorithm | | Not supported | Data encryption algorithm; Enterprise parameter |
247248
| encryptScope | | Not supported | Encryption scope; Enterprise parameter |
248249
| encryptExtDir | v3.4.0.0 | Not supported | User-defined encryption algorithms extensions path; Enterprise parameter |

‎docs/en/14-reference/09-error-code.md‎

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -683,11 +683,12 @@ Below are the business error codes for each module.
683683

684684
| Error Code | Description | Possible Error Scenarios or Reasons | Recommended Actions for Users |
685685
|------------|---------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------|
686-
| 0x80006103 | Audit database must be encrypted | Invalid param,eter | Check and correct the SQL statement |
687-
| 0x80006104 | Audit database wal_level must be 2 | Invalid param,eter | Check and correct the SQL statement |
688-
| 0x80006105 | Audit database keep2 must be greater than 1825d | Invalid param,eter | Check and correct the SQL statement |
689-
| 0x80006106 | Audit database already exist | Invalid param,eter | Check and correct the SQL statement |
690-
| 0x80006107 | Audit database is not allowed to change | Invalid param,eter | Check and correct the SQL statement |
686+
| 0x80006103 | Audit database must be encrypted | Invalid parameter | Check and correct the SQL statement |
687+
| 0x80006104 | Audit database wal_level must be 2 | Invalid parameter | Check and correct the SQL statement |
688+
| 0x80006105 | Audit database keep2 must be greater than 1825d | Invalid parameter | Check and correct the SQL statement |
689+
| 0x80006106 | Audit database already exist | Invalid parameter | Check and correct the SQL statement |
690+
| 0x80006107 | Audit database is not allowed to change | Invalid parameter | Check and correct the SQL statement |
691+
| 0x80006108 | Audit database is not allowed to keep multiple vgroups | Invalid parameter | Check and correct the SQL statement |
691692

692693
#### virtual table
693694

‎docs/zh/08-operation/16-security.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,8 @@ database_option: {
9292

9393
在 3.4.0.0 之前版本创建的审计库，与 3.4.0.0 及之后版本的审计库不兼容。3.4.0.0 之前版本的审计库无法开启 is_audit 参数，因此不会对 DURATION、WAL_LEVEL、ENCRYPT_ALGORITHM 做强制要求。对于 3.4.0.0 之前创建的审计库，如需使用新版本的审计能力，建议先 drop 该审计库后再重新创建。如果要在 3.4.0.0 之后的版本中继续使用由 3.4.0.0 之前版本创建的审计库，则需要将 auditUseToken 关闭（设置为 0）。
9494

95+
在 3.4.1.0 之后的版本可以将审计信息保存在自身，而不发送给 taoskeeper，若要使用该功能，需要将参数 auditSaveInSelf 设置为 1，并且在使用该功能时，创建的审计库的 vgroups 的数量只能为 1。
96+
9597
### taosKeeper 配置
9698

9799
在 taosKeeper 的配置文件 keeper.toml 中配置与审计日志有关的配置参数，如下表所示

‎docs/zh/14-reference/01-components/01-taosd.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1264,6 +1264,17 @@ charset 的有效值是 UTF-8。
12641264
- 动态修改：支持通过 SQL 修改，立即生效。
12651265
- 支持版本：从 v3.1.0.0 版本开始引入
12661266

1267+
#### auditSaveInSelf
1268+
1269+
- 说明：审计数据保存在自身，而不发送给 taoskeeper
1270+
- 类型：整数；0:关闭，1：开启。
1271+
- 默认值：0
1272+
- 最小值：0
1273+
- 最大值：1
1274+
- 参数类型：全局配置参数
1275+
- 动态修改：支持通过 SQL 修改，立即生效。
1276+
- 支持版本：从 v3.4.1.0 版本开始引入
1277+
12671278
#### encryptAlgorithm
12681279

12691280
- 说明：数据加密算法 **`企业版参数`**

‎docs/zh/14-reference/09-error-code.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -670,6 +670,7 @@ TSDB 错误码包括 taosc 客户端和服务端，所有语言的连接器无
670670
| 0x80006105 | Audit database keep2 must be greater than 1825d | 参数不正确 | 检查并修正 SQL 语句 |
671671
| 0x80006106 | Audit database already exist | 参数不正确 | 检查并修正 SQL 语句 |
672672
| 0x80006107 | Audit database is not allowed to change | 参数不正确 | 检查并修正 SQL 语句 |
673+
| 0x80006108 | Audit database is not allowed to keep multiple vgroups | 参数不正确 | 检查并修正 SQL 语句 |
673674

674675
#### virtual table
675676

‎include/common/tglobal.h‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -245,6 +245,7 @@ extern int32_t tsAuditLevel;
245245
extern int32_t tsAuditInterval;
246246
extern bool tsAuditHttps;
247247
extern bool tsAuditUseToken;
248+
extern bool tsAuditSaveInSelf;
248249

249250
// telem
250251
extern bool tsEnableTelem;

‎include/common/tmsg.h‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2813,6 +2813,8 @@ typedef struct {
28132813
int64_t timestamp;
28142814
char auditDB[TSDB_DB_FNAME_LEN];
28152815
char auditToken[TSDB_TOKEN_LEN];
2816+
SEpSet auditEpSet;
2817+
int32_t auditVgId;
28162818
} SStatusReq;
28172819

28182820
int32_t tSerializeSStatusReq(void* buf, int32_t bufLen, SStatusReq* pReq);
@@ -2912,6 +2914,8 @@ typedef struct {
29122914
int64_t timeWhiteVer;
29132915
char auditDB[TSDB_DB_FNAME_LEN];
29142916
char auditToken[TSDB_TOKEN_LEN];
2917+
SEpSet auditEpSet;
2918+
int32_t auditVgId;
29152919
} SStatusRsp;
29162920

29172921
int32_t tSerializeSStatusRsp(void* buf, int32_t bufLen, SStatusRsp* pRsp);
@@ -4069,6 +4073,13 @@ int32_t tSerializeSVArbSetAssignedLeaderReq(void* buf, int32_t bufLen, SVArbSetA
40694073
int32_t tDeserializeSVArbSetAssignedLeaderReq(void* buf, int32_t bufLen, SVArbSetAssignedLeaderReq* pReq);
40704074
void tFreeSVArbSetAssignedLeaderReq(SVArbSetAssignedLeaderReq* pReq);
40714075

4076+
typedef struct {
4077+
char* data;
4078+
} SVAuditRecordReq;
4079+
int32_t tSerializeSVAuditRecordReq(void* buf, int32_t bufLen, SVAuditRecordReq* pReq);
4080+
int32_t tDeserializeSVAuditRecordReq(void* buf, int32_t bufLen, SVAuditRecordReq* pReq);
4081+
void tFreeSVAuditRecordReq(SVAuditRecordReq* pReq);
4082+
40724083
typedef struct {
40734084
char* arbToken;
40744085
char* memberToken;

‎include/common/tmsgcb.h‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ typedef enum {
4343
STREAM_RUNNER_QUEUE,
4444
STREAM_READER_QUEUE,
4545
STREAM_TRIGGER_QUEUE,
46+
AUDIT_QUEUE,
4647
QUEUE_MAX,
4748
} EQueueType;
4849

‎include/common/tmsgdef.h‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -366,6 +366,7 @@
366366
TD_DEF_MSG_TYPE(TDMT_VND_QUERY_TRIM_PROGRESS, "vnode-query-trim-progress", NULL, NULL)
367367
TD_DEF_MSG_TYPE(TDMT_VND_SET_KEEP_VERSION, "vnode-set-keep-version", NULL, NULL)
368368
TD_DEF_MSG_TYPE(TDMT_VND_TRIM_WAL, "vnode-trim-wal", NULL, NULL)
369+
TD_DEF_MSG_TYPE(TDMT_VND_AUDIT_RECORD, "vnode-audit-record", NULL, NULL)
369370
TD_CLOSE_MSG_SEG(TDMT_VND_MSG)
370371

371372
TD_NEW_MSG_SEG(TDMT_SCH_MSG) // 3<<8

0 commit comments

Comments
 (0)