Skip to content

[Enhancement] Parallel CustomScan is not supported in the Cloudberry executor #1824

@roseduan

Description

@roseduan

Apache Cloudberry version

main

What happened

Cloudberry does not currently support running a CustomScan provider
under parallel execution. When enable_parallel = on and the plan
tree contains a CustomScanState, the CB-specific parallel-execution
walkers either skip CustomScan or wire only a subset of the per-node
parallel hooks, so DSM segment allocation, worker initialisation, and
per-worker state recovery for CustomScan nodes do not behave the same
way as for built-in parallel-aware nodes (SeqScan, IndexScan, etc.).

The upstream-style walkers in src/backend/executor/execParallel.c
do handle T_CustomScanState, but the CB-specific Gp* walkers in
the same file are the ones that drive parallelism inside CB segments,
and the CustomScan branches there are incomplete.

Functions that need to grow CustomScan support

src/backend/executor/execParallel.c:

  • EstimateGpParallelDSMEntrySize() (line 1548)
  • InitializeGpParallelWorkers() (line 1622)
  • InitializeGpParallelDSMEntry() (line 1689)

src/backend/executor/execProcnode.c:

  • planstate_walk_kids()

For reference, the existing parallel-CustomScan hook API in
src/include/executor/nodeCustom.h (ExecCustomScanEstimate,
ExecCustomScanInitializeDSM, ExecCustomScanReInitializeDSM,
ExecCustomScanInitializeWorker) is what the CB walkers should call;
the upstream walkers in the same file already demonstrate the calling
convention.

Why this matters

Cloudberry extensions that ship a CustomScan provider cannot rely on
parallel-aware execution today.
Tracking this as a single feature-gap issue so a CustomScan
provider in Cloudberry has a single reference point.

What you think should happen instead

No response

How to reproduce

none

Operating System

any

Anything else

No response

Are you willing to submit PR?

  • Yes, I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions