[#23431] YSQL: Disable default partition creation

Summary:
This diff disables the  defualt partition creation in pg_partman as after creating default partition the further maintenance work of moving rows out of default partition into newly created partitions will cause this error

```
 updated partition constraint for default partition would be violated by some row
```

When `partition_data_proc` or `run_maintenance` functions  tries to move rows from default partition to a newly created child partition, it does that process in following steps

1.) Create a temporary table to hold the data that needs to be moved. This is done because child tables cannot be created in native partitioning if data that belongs to it exists in the default.

```
CREATE TEMP TABLE IF NOT EXISTS partman_temp_data_storage (LIKE %I.%I INCLUDING INDEXES) ON COMMIT DROP
```

2.)  Move rows from default table to the temp table.

```
    EXECUTE format('WITH partition_data AS (
            DELETE FROM %1$I.%2$I WHERE %3$I >= %4$s AND %3$I < %5$s RETURNING *)
        INSERT INTO partman_temp_data_storage (%6$s) SELECT %6$s FROM partition_data'
```

3.) Create the child partition via create_partition_id  function

```
PERFORM @[email protected]_partition_id(p_parent_table, v_partition_id, p_analyze);
```

4.) After creation of the child partition, move the data from temp table to the new child partition.

```
EXECUTE format('WITH partition_data AS (
            DELETE FROM ONLY %1$I.%2$I WHERE %3$I >= %4$s AND %3$I < %5$s RETURNING *)
        INSERT INTO %1$I.%6$I (%7$s) SELECT %7$s FROM partition_data
```
The issue is occurring at step 3 while creating the new child partition. Since currently YugabyteDB does not have transactional DDL support and the plsql functions are run inside a transactional context,   `create_partition_id`  and `partition_data_proc`  are in different transactional context. When `create_partition_id`  tries to attach the newly created child partition, that transaction still finds the default partition have violating rows present and the `ALTER TABLE ATTACH` query fails stating

```
updated partition constraint for default partition would be violated by some row .
```

Hence to not get fall in this scenario, default partition creation should be disabled in pg_partman. This diff also adds the check that if default partition is created deliberately then `partition_data_proc` and `run_maintenance` will early exit without performing any operation.
Jira: DB-12352

Test Plan: Jenkins: test regex: PgPartmanTest*

Reviewers: skumar, hsunder

Reviewed By: hsunder

Subscribers: yql

Differential Revision: https://phorge.dev.yugabyte.com/D39010

Author: Devansh Saxena

src/postgres/third-party-extensions/pg_partman/sql/functions/create_parent.sql

@@ -75,6 +75,7 @@ v_top_parent_schema             text := split_part(p_parent_table, '.', 1);
 v_top_parent_table              text := split_part(p_parent_table, '.', 2);
 v_unlogged                      char;
 yb_v_parent_table_default       regclass;
+yb_v_enable_default_partition   boolean := false;
 
 BEGIN
 /*
@@ -730,7 +731,10 @@ IF v_control_type = 'id' AND p_epoch = 'none' THEN
 
 END IF; -- End IF id
 
-IF p_type = 'native' AND current_setting('server_version_num')::int >= 110000 THEN
+/* YB: Disable default partition creation.
+ * TODO(#3109): Re-enable it after transactional DDL support.
+ */
+IF yb_v_enable_default_partition AND p_type = 'native' AND current_setting('server_version_num')::int >= 110000 THEN
     -- Add default partition to native sets in PG11+
 
     v_default_partition := @[email protected]_name_length(v_parent_tablename, '_default', FALSE);

src/postgres/third-party-extensions/pg_partman/sql/functions/partition_data_id.sql

@@ -106,6 +106,10 @@ ELSIF v_partition_type = 'native' AND current_setting('server_version_num')::int
         v_source_tablename := v_default_tablename;
 
         v_default_exists := true;
+        -- YB: Moving data from default partition is not supported
+        -- TODO(#3109): Re-enable it after transactional DDL support.
+        RAISE NOTICE 'Moving data from default partition to newly created child tables is not supported';
+        RETURN v_total_rows;
         EXECUTE format ('CREATE TEMP TABLE IF NOT EXISTS partman_temp_data_storage (LIKE %I.%I INCLUDING INDEXES) ON COMMIT DROP', v_source_schemaname, v_source_tablename);
     ELSE
         RAISE DEBUG 'No default table found when partition_data_id() was called';

src/postgres/third-party-extensions/pg_partman/sql/functions/partition_data_time.sql

@@ -118,6 +118,10 @@ ELSIF v_partition_type = 'native' AND current_setting('server_version_num')::int
         v_source_tablename := v_default_tablename;
 
         v_default_exists := true;
+        -- YB: Moving data from default partition is not supported
+        -- TODO(#3109): Re-enable it after transactional DDL support.
+        RAISE NOTICE 'Moving data from default partition to newly created child tables is not supported';
+        RETURN v_total_rows;
         EXECUTE format ('CREATE TEMP TABLE IF NOT EXISTS partman_temp_data_storage (LIKE %I.%I INCLUDING INDEXES) ON COMMIT DROP', v_source_schemaname, v_source_tablename);
     ELSE
         RAISE DEBUG 'No default table found when partition_data_id() was called';

src/postgres/third-party-extensions/pg_partman/sql/functions/run_maintenance.sql

@@ -172,6 +172,12 @@ LOOP
         WHERE n.nspname = v_parent_schema
         AND c.relname = v_default_tablename;
 
+        -- YB: TODO(#3109): Disable skip after transactional DDL support.
+        IF v_is_default = 'DEFAULT' THEN
+            RAISE NOTICE 'Skip run maintenace job for table % as it has default table attached', v_row.parent_table;
+            CONTINUE;
+        END IF;
+
         IF v_is_default != 'DEFAULT' THEN
             v_default_tablename := v_parent_tablename;
         END IF;

src/postgres/third-party-extensions/pg_partman/test/test_native/test_nonsuperuser/yb_pg_test-time-hourly-nonsuperuser-part2.sql

@@ -10,7 +10,7 @@
 -- BEGIN; YB: Transactional DDL not supported
 SELECT set_config('search_path','partman, public',false);
 
-SELECT plan(30); -- YB: decreased number of tests
+SELECT plan(28); -- YB: decreased number of tests
 
 CREATE TABLE partman_test.time_taptest_table (col1 serial, col2 text, col3 timestamp NOT NULL DEFAULT now()) PARTITION BY RANGE (col3);
 CREATE TABLE partman_test.time_taptest_undo (LIKE partman_test.time_taptest_table);
@@ -57,7 +57,9 @@ SELECT results_eq('SELECT count(*)::int FROM partman_test.time_taptest_table_p'|
 INSERT INTO partman_test.time_taptest_table (col1, col3) VALUES (generate_series(11,20), CURRENT_TIMESTAMP + '1 hour'::interval);
 INSERT INTO partman_test.time_taptest_table (col1, col3) VALUES (generate_series(21,25), CURRENT_TIMESTAMP + '2 hours'::interval);
 
-SELECT is_empty('SELECT * FROM partman_test.time_taptest_table_default', 'Check that default table has had no data inserted to it');
+-- YB: default partition creation is disabled.
+-- TODO(#3109): Re-enable it after transactional DDL support.
+-- SELECT is_empty('SELECT * FROM partman_test.time_taptest_table_default', 'Check that default table has had no data inserted to it'); --YB: Disabled default partition
 SELECT results_eq('SELECT count(*)::int FROM partman_test.time_taptest_table', ARRAY[25], 'Check count from time_taptest_table');
 SELECT results_eq('SELECT count(*)::int FROM partman_test.time_taptest_table_p'||to_char(date_trunc('hour', CURRENT_TIMESTAMP)+'1 hour'::interval, 'YYYY_MM_DD_HH24MI'), 
     ARRAY[10], 'Check count from time_taptest_table_p'||to_char(date_trunc('hour', CURRENT_TIMESTAMP)+'1 hour'::interval, 'YYYY_MM_DD_HH24MI'));
@@ -91,8 +93,10 @@ SELECT hasnt_table('partman_test', 'time_taptest_table_p'||to_char(date_trunc('h
 SELECT col_is_pk('partman_test', 'time_taptest_table_p'||to_char(date_trunc('hour', CURRENT_TIMESTAMP)+'8 hours'::interval, 'YYYY_MM_DD_HH24MI'), ARRAY['col1'], 
     'Check for primary key in time_taptest_table_p'||to_char(date_trunc('hour', CURRENT_TIMESTAMP)+'8 hours'::interval, 'YYYY_MM_DD_HH24MI'));
 
-INSERT INTO partman_test.time_taptest_table (col1, col3) VALUES (generate_series(200,210), CURRENT_TIMESTAMP + '20 hours'::interval);
-SELECT results_eq('SELECT count(*)::int FROM ONLY partman_test.time_taptest_table_default', ARRAY[11], 'Check that data outside trigger scope goes to default');
+-- YB: default partition creation is disabled
+-- TODO(#3109): Re-enable it after transactional DDL support.
+-- INSERT INTO partman_test.time_taptest_table (col1, col3) VALUES (generate_series(200,210), CURRENT_TIMESTAMP + '20 hours'::interval);
+-- SELECT results_eq('SELECT count(*)::int FROM ONLY partman_test.time_taptest_table_default', ARRAY[11], 'Check that data outside trigger scope goes to default');
 
 /* YB: undo_partition not supported
 -- Keep tables after undoing

src/postgres/third-party-extensions/pg_partman/test/test_native/yb_pg_test-id-native.sql

@@ -10,7 +10,7 @@
 -- BEGIN; YB: Transactional DDL not supported
 SELECT set_config('search_path','partman, public',false);
 
-SELECT plan(98); -- YB: decreased number of tests
+SELECT plan(99); -- YB: decreased number of tests
 CREATE SCHEMA partman_test;
 CREATE SCHEMA partman_retention_test;
 CREATE ROLE partman_basic;
@@ -62,7 +62,7 @@ SELECT has_table('partman_test', 'id_taptest_table_p3000000010', 'Check id_tapte
 SELECT has_table('partman_test', 'id_taptest_table_p3000000020', 'Check id_taptest_table_p3000000020 exists');
 SELECT has_table('partman_test', 'id_taptest_table_p3000000030', 'Check id_taptest_table_p3000000030 exists');
 SELECT has_table('partman_test', 'id_taptest_table_p3000000040', 'Check id_taptest_table_p3000000040 exists');
---SELECT has_table('partman_test', 'id_taptest_table_default', 'Check id_taptest_table_default exists');
+SELECT hasnt_table('partman_test', 'id_taptest_table_default', 'Check id_taptest_table_default exists'); -- YB: disable default partition creation
 SELECT hasnt_table('partman_test', 'id_taptest_table_p3000000050', 'Check id_taptest_table_p3000000050 doesn''t exists yet');
 SELECT col_is_pk('partman_test', 'id_taptest_table_p3000000000', ARRAY['col1'], 'Check for primary key in id_taptest_table_p3000000000');
 SELECT col_is_pk('partman_test', 'id_taptest_table_p3000000010', ARRAY['col1'], 'Check for primary key in id_taptest_table_p3000000010');

src/postgres/third-party-extensions/pg_partman/test/test_native/yb_pg_test-time-daily-native.sql

@@ -168,7 +168,7 @@ SELECT is_empty('SELECT * FROM ONLY partman_test.time_taptest_table', 'Check tha
 SELECT results_eq('SELECT count(*)::int FROM partman_test.time_taptest_table', ARRAY[10], 'Check count from parent table');
 SELECT results_eq('SELECT count(*)::int FROM partman_test.time_taptest_table_p'||to_char(CURRENT_TIMESTAMP, 'YYYY_MM_DD'), 
     ARRAY[10], 'Check count from time_taptest_table_p'||to_char(CURRENT_TIMESTAMP, 'YYYY_MM_DD'));
-SELECT results_eq('SELECT count(*)::int FROM pg_catalog.pg_publication_tables WHERE pubname = ''partman_test_publication''', ARRAY[10], 'Check that all child tables were added to publication. This test will fail in PG10 since default tables are not supported.');
+SELECT results_eq('SELECT count(*)::int FROM pg_catalog.pg_publication_tables WHERE pubname = ''partman_test_publication''', ARRAY[9], 'Check that all child tables were added to publication. This test will fail in PG10 since default tables are not supported.');
 
 REVOKE INSERT, UPDATE, DELETE, TRUNCATE, REFERENCES, TRIGGER ON partman_test.time_taptest_table FROM partman_revoke;
 INSERT INTO partman_test.time_taptest_table (col1, col3) VALUES (generate_series(11,20), CURRENT_TIMESTAMP + '1 day'::interval);
@@ -256,7 +256,7 @@ SELECT col_is_fk('partman_test', 'time_taptest_table_p'||to_char(CURRENT_TIMESTA
 SELECT is_empty('SELECT * FROM ONLY partman_test.time_taptest_table', 'Check that parent table has had no data inserted to it');
 SELECT results_eq('SELECT count(*)::int FROM partman_test.time_taptest_table_p'||to_char(CURRENT_TIMESTAMP+'5 days'::interval, 'YYYY_MM_DD'), 
     ARRAY[22], 'Check count from time_taptest_table_p'||to_char(CURRENT_TIMESTAMP+'5 days'::interval, 'YYYY_MM_DD'));
-SELECT results_eq('SELECT count(*)::int FROM pg_catalog.pg_publication_tables WHERE pubname = ''partman_test_publication''', ARRAY[16], 'Check that all child tables were added to publication. This test will fail in PG10 since default tables are not supported.');
+SELECT results_eq('SELECT count(*)::int FROM pg_catalog.pg_publication_tables WHERE pubname = ''partman_test_publication''', ARRAY[15], 'Check that all child tables were added to publication. This test will fail in PG10 since default tables are not supported.');
 
 SELECT table_privs_are('partman_test', 'time_taptest_table_p'||to_char(CURRENT_TIMESTAMP, 'YYYY_MM_DD'), 'partman_basic', ARRAY['SELECT','INSERT','UPDATE'], 
     'Check partman_basic privileges of time_taptest_table_p'||to_char(CURRENT_TIMESTAMP, 'YYYY_MM_DD'));
@@ -303,7 +303,7 @@ SELECT is_empty('SELECT * FROM ONLY partman_test.time_taptest_table', 'Check tha
 SELECT results_eq('SELECT count(*)::int FROM partman_test.time_taptest_table', ARRAY[148], 'Check count from parent table');
 SELECT results_eq('SELECT count(*)::int FROM partman_test.time_taptest_table_p'||to_char(CURRENT_TIMESTAMP+'6 days'::interval, 'YYYY_MM_DD'), 
     ARRAY[28], 'Check count from time_taptest_table_p'||to_char(CURRENT_TIMESTAMP+'6 days'::interval, 'YYYY_MM_DD'));
-SELECT results_eq('SELECT count(*)::int FROM pg_catalog.pg_publication_tables WHERE pubname = ''partman_test_publication''', ARRAY[18], 'Check that all child tables were added to publication. This test will fail in PG10 since default tables are not supported.');
+SELECT results_eq('SELECT count(*)::int FROM pg_catalog.pg_publication_tables WHERE pubname = ''partman_test_publication''', ARRAY[17], 'Check that all child tables were added to publication. This test will fail in PG10 since default tables are not supported.');
 
 SELECT has_table('partman_test', 'time_taptest_table_p'||to_char(CURRENT_TIMESTAMP+'11 days'::interval, 'YYYY_MM_DD'), 
     'Check time_taptest_table_p'||to_char(CURRENT_TIMESTAMP+'11 days'::interval, 'YYYY_MM_DD')||' exists');

src/postgres/third-party-extensions/pg_partman/test/test_procedure/yb_pg_test-id-procedure-source-table-part1.sql

@@ -33,7 +33,9 @@ CREATE TABLE partman_test.template_id_taptest_table (LIKE partman_test.id_taptes
 
 SELECT create_parent('partman_test.id_taptest_table', 'col1', 'native', '10000', p_jobmon := false, p_template_table := 'partman_test.template_id_taptest_table');
 
-SELECT has_table('partman_test', 'id_taptest_table_default', 'Check id_taptest_table_default exists');
+-- YB: default partition creation is disabled
+-- TODO(#3109): Re-enable it after transactional DDL support.
+SELECT hasnt_table('partman_test', 'id_taptest_table_default', 'Check id_taptest_table_default doesn''t exists');
 SELECT has_table('partman_test', 'id_taptest_table_p0', 'Check id_taptest_table_p0 exists');
 SELECT has_table('partman_test', 'id_taptest_table_p10000', 'Check id_taptest_table_p10000 exists');
 SELECT has_table('partman_test', 'id_taptest_table_p20000', 'Check id_taptest_table_p20000 exists');

src/postgres/third-party-extensions/pg_partman/test/test_procedure/yb_pg_test-id-procedure-source-table-part3.sql

@@ -2,7 +2,7 @@
 
 SELECT set_config('search_path','partman, public',false);
 
-SELECT plan(7);
+SELECT plan(5);
 
 -- YB: reduced number of rows by factor of 10 to prevent test timeout
 SELECT has_table('partman_test', 'id_taptest_table_p110000', 'Check id_taptest_table_p110000 exists');
@@ -11,11 +11,14 @@ SELECT has_table('partman_test', 'id_taptest_table_p130000', 'Check id_taptest_t
 SELECT has_table('partman_test', 'id_taptest_table_p140000', 'Check id_taptest_table_p140000 exists');
 SELECT hasnt_table('partman_test', 'id_taptest_table_p150000', 'Check id_taptest_table_p150000 doesn''t exists yet');
 
+/* YB: default partition creation is disabled.
+ * TODO(#3109): Re-enable it after transactional DDL support.
 -- Test default partition
 INSERT INTO partman_test.id_taptest_table (col1) VALUES (generate_series(200000, 200009));
 
 SELECT results_eq('SELECT count(*)::int FROM partman_test.id_taptest_table WHERE col1 > 100000', ARRAY[10], 'Check count from id_taptest_table for out of band data');
 SELECT results_eq('SELECT count(*)::int FROM partman_test.id_taptest_table_default', ARRAY[10], 'Check count from id_taptest_table_default');
+*/ -- YB
 
 SELECT diag('!!! In separate psql terminal, please run the following (adjusting schema if needed): "CALL partman.undo_partition_proc(''partman_test.id_taptest_table'', p_target_table := ''partman_test.id_taptest_table_target'', p_keep_table := false);".');
 SELECT diag('!!! After that, run part4 of this script to check result and cleanup tests !!!');