Skip to content

fix(inference): Synchronize cpu_threads configuration for HPI predictors#4972

Open
SankaraVenkatRam wants to merge 5 commits intoPaddlePaddle:developfrom
SankaraVenkatRam:fix/hpi-cpu-threads-sync
Open

fix(inference): Synchronize cpu_threads configuration for HPI predictors#4972
SankaraVenkatRam wants to merge 5 commits intoPaddlePaddle:developfrom
SankaraVenkatRam:fix/hpi-cpu-threads-sync

Conversation

@SankaraVenkatRam
Copy link

Summary

This PR fixes an issue where user-defined CPU thread settings (e.g. cpu_threads=2) were ignored when using the High Performance Inference (HPI) backend.

Although the value was correctly set in the legacy PaddlePredictorOption, it was never propagated to HPI, which instead used its own default (usually 10 threads). This could lead to unexpected CPU overuse, especially in containers or limited environments.


What changed

  • Synced CPU thread settings between PaddlePredictorOption and HPIConfig via a new sync_threads step in BasePredictor
  • Added support for the CPU_NUM_THREADS environment variable in:
    • OpenVINO and ONNX Runtime HPI configs
    • Default CPU thread handling in pp_option.py

Result

  • Explicit cpu_threads values are now respected by the HPI backend
  • CPU_NUM_THREADS provides a simple global override when no value is set in code
  • More predictable and resource-friendly CPU usage

Why it matters

This aligns user intent with actual runtime behavior and avoids silent over-allocation of CPU resources.

@paddle-bot
Copy link

paddle-bot bot commented Feb 4, 2026

Thanks for your contribution!

@paddle-bot paddle-bot bot added the contributor External developers label Feb 4, 2026
@SankaraVenkatRam
Copy link
Author

CI_XPU is failing during repository clone with a TLS handshake error on the XPU runner:
gnutls_handshake() failed

@SankaraVenkatRam
Copy link
Author

The PR is still blocked due to the same CI failure on the XPU runner:
gnutls_handshake() failed during repository clone.

This appears to be a TLS/network issue on the XPU CI runner rather than something introduced by this PR.

@SankaraVenkatRam
Copy link
Author

The CI failure is occurring during git fetch of the PR branch.

Logs show connection timeouts and proxy instability when accessing GitHub:

    Failed connect to github.com:443; Connection timed out
    
    RPC failed; result=52, HTTP code = 0
    
    The remote end hung up unexpectedly

This appears to be a CI runner network/proxy issue .

Kindly requesting maintainers to check the runner connectivity.


def sync_threads(self):
if self._pp_option and self._pp_option.cpu_threads:
# If the user specified threads in the old system,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pp_option and hpi_config are primarily designed for two orthogonal features. The former is intended for configuring the Paddle Inference Engine (pp referring to PaddlePaddle), while the latter is used for High-Performance Inference (HPI) settings. Although pp_option can take effect when the HPI backend is set to Paddle Inference, I am concerned that updating the CPU thread configuration via pp_option in a way that impacts other backends (e.g., ONNX Runtime) may not be appropriate. Such behavior could potentially lead to confusion.

"device_type": device_type,
"device_id": device_id,
"cpu_threads": 10,
"cpu_threads": int(os.getenv("CPU_NUM_THREADS", 10)),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest adding the prefix PADDLEX_PDX to the environment variable.

@SankaraVenkatRam SankaraVenkatRam force-pushed the fix/hpi-cpu-threads-sync branch from 88ecc8f to 8c4afeb Compare March 3, 2026 14:36
@SankaraVenkatRam
Copy link
Author

@Bobholamovic Thanks for the review! I've pushed a follow-up commit addressing both points:

1) Removed sync_threads() — each system now reads PADDLE_PDX_CPU_NUM_THREADS independently, keeping pp_option and hpi_config orthogonal.
2) Renamed env var to PADDLE_PDX_CPU_NUM_THREADS to follow the project convention

Copy link
Member

@Bobholamovic Bobholamovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@SankaraVenkatRam
Copy link
Author

It looks like the CI_XPU job failed during the checkout step due to a TLS connection error when cloning the repository

@PaddlePaddle PaddlePaddle locked and limited conversation to collaborators Mar 9, 2026
@PaddlePaddle PaddlePaddle unlocked this conversation Mar 9, 2026
@PaddlePaddle PaddlePaddle locked and limited conversation to collaborators Mar 9, 2026
@PaddlePaddle PaddlePaddle unlocked this conversation Mar 9, 2026
@luotao1
Copy link
Collaborator

luotao1 commented Mar 9, 2026

Sorry, there is something wrong with CI, could you please create a new PR to trigger CI?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor External developers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants