Skip to content

Conversation

@avamingli
Copy link
Contributor

In CBDB, row estimation is determined by the relation's rows and cluster segments.
However, when there is a parallel subquery scan path, each worker will process fewer rows (divided by parallel_workers).

set enable_parallel = off;
explain SELECT e.name
FROM employees e
WHERE e.salary > (
    SELECT AVG(salary)
    FROM employees
    WHERE department_id = e.department_id);
                                                         QUERY PLAN                                                         
----------------------------------------------------------------------------------------------------------------------------
 Gather Motion 3:1  (slice1; segments: 3)  (cost=163.42..307.76 rows=3767 width=218)
   ->  Hash Join  (cost=163.42..257.54 rows=1256 width=218)
         Hash Cond: (e.department_id = "Expr_SUBQUERY".csq_c0)
         Join Filter: (e.salary > "Expr_SUBQUERY".csq_c1)
         ->  Seq Scan on employees e  (cost=0.00..71.67 rows=3767 width=254)
         ->  Hash  (cost=150.92..150.92 rows=1000 width=36)
               ->  Broadcast Motion 3:3  (slice2; segments: 3)  (cost=130.09..150.92 rows=1000 width=36)
                     ->  Subquery Scan on "Expr_SUBQUERY"  (cost=130.09..137.59 rows=333 width=36)
                           ->  Finalize HashAggregate  (cost=130.09..134.26 rows=333 width=36)
                                 Group Key: employees.department_id
                                 ->  Redistribute Motion 3:3  (slice3; segments: 3)  (cost=90.50..122.67 rows=990 width=36)
                                       Hash Key: employees.department_id
                                       ->  Partial HashAggregate  (cost=90.50..102.87 rows=990 width=36)
                                             Group Key: employees.department_id
                                             ->  Seq Scan on employees  (cost=0.00..71.67 rows=3767 width=36)
 Optimizer: Postgres query optimizer
(16 rows)

Subquery Scan on "Expr_SUBQUERY" (cost=130.09..137.59 rows=333 width=36)
While, a parallel Subquery Scan has the same rows though cost is less than that.

set enable_parallel = on;
set min_parallel_table_scan_size = 0;
explain SELECT e.name
FROM employees e
WHERE e.salary > (
    SELECT AVG(salary)
    FROM employees
    WHERE department_id = e.department_id);
                                                        QUERY PLAN                                                         
---------------------------------------------------------------------------------------------------------------------------
 Gather Motion 6:1  (slice1; segments: 6)  (cost=131.17..245.45 rows=3767 width=218)
   ->  Parallel Hash Join  (cost=131.17..201.50 rows=628 width=218)
         Hash Cond: (e.department_id = "Expr_SUBQUERY".csq_c0)
         Join Filter: (e.salary > "Expr_SUBQUERY".csq_c1)
         ->  Parallel Seq Scan on employees e  (cost=0.00..52.83 rows=1883 width=254)
         ->  Parallel Hash  (cost=118.67..118.67 rows=1000 width=36)
               ->  Broadcast Workers Motion 6:6  (slice2; segments: 6)  (cost=99.92..118.67 rows=1000 width=36)
                     ->  Subquery Scan on "Expr_SUBQUERY"  (cost=99.92..105.33 rows=333 width=36)
                           ->  HashAggregate  (cost=99.92..102.00 rows=167 width=36)
                                 Group Key: employees.department_id
                                 ->  Redistribute Motion 6:6  (slice3; segments: 6)  (cost=0.00..90.50 rows=1883 width=36)
                                       Hash Key: employees.department_id
                                       Hash Module: 3
                                       ->  Parallel Seq Scan on employees  (cost=0.00..52.83 rows=1883 width=36)
 Optimizer: Postgres query optimizer
(15 rows)

This commit fixes that issue.

The correction not only makes parallel subquery estimation more accurate, but also enables the entire plan to be as parallel as possible, particularly for subqueries in complex queries.

Authored-by: Zhang Mingli [email protected]

Fixes #ISSUE_Number

What does this PR do?

Type of Change

  • Bug fix (non-breaking change)
  • New feature (non-breaking change)
  • Breaking change (fix or feature with breaking changes)
  • Documentation update

Breaking Changes

Test Plan

  • Unit tests added/updated
  • Integration tests added/updated
  • Passed make installcheck
  • Passed make -C src/test installcheck-cbdb-parallel

Impact

Performance:

User-facing changes:

Dependencies:

Checklist

Additional Context

CI Skip Instructions


@avamingli avamingli force-pushed the fix_subquery_rows branch 2 times, most recently from 414c999 to 0791ff6 Compare August 5, 2025 05:04
@avamingli
Copy link
Contributor Author

Previously, we attempted to disable window functions inside CASE WHEN
expressions due to concerns about unstable parallel results. However,
this was a misunderstanding. All expressions from the subquery are Var
columns, not the original expressions.

This issue was uncovered when we fixed the subquery row count
estimation, causing the cost to change in the upper plan.

Fixed at commit: Correct parallel window function in CASE WHEN.

EXPLAIN(COSTS OFF)
SELECT empno, depname, salary, bonus, depadj, MIN(bonus) OVER (ORDER BY
empno), MAX(depadj) OVER () FROM(
	SELECT *,
		CASE WHEN enroll_date < '2008-01-01' THEN 2008 -
extract(YEAR FROM enroll_date) END * 500 AS bonus,
		CASE WHEN
			AVG(salary) OVER (PARTITION BY depname) < salary
		THEN 200 END AS depadj FROM empsalary
)s;
                                        QUERY PLAN
--------------------------------------------------------------------------
 WindowAgg
   ->  WindowAgg
         Order By: s.empno
         ->  Gather Motion 6:1  (slice1; segments: 6)
               Merge Key: s.empno
               ->  Sort
                     Sort Key: s.empno
                     ->  Subquery Scan on s
                           ->  WindowAgg
                                 Partition By: empsalary.depname
                                 ->  Sort
                                       Sort Key: empsalary.depname
                                       ->  Redistribute Motion 6:6
(slice2; segments: 6)
                                             Hash Key: empsalary.depname
                                             Hash Module: 3
                                             ->  Parallel Seq Scan on
empsalary
 Optimizer: Postgres query optimizer
(17 rows)

@avamingli avamingli force-pushed the fix_subquery_rows branch 2 times, most recently from 519c515 to 06ee760 Compare August 8, 2025 08:53
In CBDB, path's row estimation is determined by subpath's rows
and cluster segments.

However, when there is a parallel subquery scan path, each
worker will process fewer rows (divided by parallel_workers).

This commit fixes that issue.

The correction not only makes parallel subquery estimation more
accurate, but also enables the entire plan to be as parallel as
possible, particularly for subqueries in complex queries.

Authored-by: Zhang Mingli [email protected]
Previously, we attempted to disable window functions inside CASE WHEN
expressions due to concerns about unstable parallel results. However,
this was a misunderstanding. All expressions from the subquery are Var
columns, not the original expressions.

This issue was uncovered when we fixed the subquery row count
estimation, causing the cost to change in the upper plan.

EXPLAIN(COSTS OFF)
SELECT empno, depname, salary, bonus, depadj, MIN(bonus) OVER (ORDER BY
empno), MAX(depadj) OVER () FROM(
	SELECT *,
		CASE WHEN enroll_date < '2008-01-01' THEN 2008 -
extract(YEAR FROM enroll_date) END * 500 AS bonus,
		CASE WHEN
			AVG(salary) OVER (PARTITION BY depname) < salary
		THEN 200 END AS depadj FROM empsalary
)s;
                                        QUERY PLAN
--------------------------------------------------------------------------
 WindowAgg
   ->  WindowAgg
         Order By: s.empno
         ->  Gather Motion 6:1  (slice1; segments: 6)
               Merge Key: s.empno
               ->  Sort
                     Sort Key: s.empno
                     ->  Subquery Scan on s
                           ->  WindowAgg
                                 Partition By: empsalary.depname
                                 ->  Sort
                                       Sort Key: empsalary.depname
                                       ->  Redistribute Motion 6:6
(slice2; segments: 6)
                                             Hash Key: empsalary.depname
                                             Hash Module: 3
                                             ->  Parallel Seq Scan on
empsalary
 Optimizer: Postgres query optimizer
(17 rows)

Authored-by: Zhang Mingli [email protected]
Copy link
Contributor

@my-ship-it my-ship-it left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@avamingli avamingli merged commit 8b01eaf into apache:main Aug 14, 2025
27 checks passed
@avamingli avamingli deleted the fix_subquery_rows branch August 14, 2025 08:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants