Regression test cases fail after enabling Transparent Data Encryption (TDE) functionality

[wsg314]

I made minor adjustments in the cloudberry code's gpAux/gpdemo directory, adding the '-T sm4' parameter to the gpinitsystem command line called by the demo_cluster.sh script to enable Transparent Data Encryption. After starting the gpdemo cluster, I executed 'make installcheck' in the src/test/regress directory, which resulted in test case failures. During the regression test execution, the cluster entered an abnormal state causing the regression testing to be interrupted.

Is the Transparent Data Encryption functionality currently available?

[kongfanshen-0801]

This feature worked in previous versions, but there may be problem with the latest code updates, which are uncertain. The database is currently in an abnormal state, what are the error logs. I won't be working this week, I'll go check on this issue next Monday.

[wsg314]

The code version I am using is 1a40e1f, which should correspond to version 1.6.0.

The regression logs were not preserved, but partial regression results are shown below:
"
test instr_in_shmem ... Duplicate specification "verbose|Verbose" for option "verbose"
ok 1757 ms (diff 367 ms)
test createdb ... Duplicate specification "verbose|Verbose" for option "verbose"
Duplicate specification "verbose|Verbose" for option "verbose"
FAILED 81489 ms (diff 1569 ms)

==================================
Cluster validation failed:
2 | 0 | m | p | n | d | 7002 | localhost.localdomain | localhost.localdomain | /home/wsg/openCode/cloudberry/gpAux/gpdemo/datadirs/dbfast1/demoDataDir0 | 0

========================
4 of 227 tests failed.

"

BTW : What was the commit hash of the previous version where the TDE function worked without issues?

[kongfanshen-0801]

The test case 'createdb' caused a database exception. TDE has a bug. Process panic, shared memory cleared. Causing abnormal value of key and data decryption failure. The way to recover from this problem is to restart the data.
Additionally, this bug also existed in previous versions.
I will fix this bug in a few days.

[wsg314]

Okay, please let me know when the bug is fixed. Thank you.

[tuhaihe]

The PR #1416 should be used to fix this issue.

[wsg314]

Thanks.

1、Should the size in the CreateSharedMemoryAndSemaphores function be increased by the size of BufEncCtx and BufDecCtx?

2、The 'createdb' test case passes, but there are some other failed test cases with '-T sm4' parameter

test: appendonly
test: aocs
test: rle rle_delta dsp not_out_of_shmem_exit_slots create_am_gp
test: ao_checksum_corruption AOCO_Compression AORO_Compression table_statistics
test: fts_recovery_in_progress
test: mirror_replay

[kongfanshen-0801]

test: appendonly
test: aocs
The reason for the failure of these two use cases is that the data is encrypted and random, and the compression algorithm determines that the storage space has not decreased after compression. So no compression was performed. The test case calls select get_ao_compression_ratio to calculate the compression ratio, which is 1, indicating that no compression has been performed. This is inconsistent with the expected compression ratio.

test: rle rle_delta dsp not_out_of_shmem_exit_slots create_am_gp
These test cases did not fail in my testing cluster

test: ao_checksum_corruption
This test case directly modifies the file by changing check_sum. But when TDE is enabled, all files are encrypted, so it's not about modifying check_stum. So the expected output of the use case is inconsistent.

test: AOCO_Compression AORO_Compression
The data is encrypted, compression ratio is not expected.

test: table_statistics
test: fts_recovery_in_progress
These test cases did not fail in my testing cluster

test: mirror_replay
This test case is skipped in greenplum_schedule

[kongfanshen-0801]

These test cases failed because TDE was enabled and the data was encrypted, which was as expected.

[wsg314]

thanks

[kongfanshen-0801]

Don't use share memory for BufEncCtx and BufDecCtx. Fix it in this PR:
#1424