Deployment Challenges and Help Wanted: PaddleOCR on NVIDIA Blackwell/Grace (ARM) Infrastructure

[samilbeden]

This discussion highlights the experience of deploying PaddleOCR on NVIDIA Blackwell (Grace ARM64). While we achieved a stable CPU-based deployment, we faced significant hurdles with modern library versions (PaddleX/PaddleOCR v2.8+) and GPU acceleration. We are seeking community insights on stable versioning for ARM-based next-gen hardware.

Technical Overview:

1. Environment:
   Hardware: NVIDIA Blackwell (Grace CPU ARM64 / Blackwell GPU).
   Stack: Python 3.12, Docker, Flask.

32 The Library & Versioning Struggle:

• Incompatibility with Modern API: We observed that newer versions of the library (integrating with PaddleX) introduced breaking changes. Standard arguments like use_gpu, use_angle_cls, and show_log were deprecated or unrecognized, leading to Unexpected Keyword Argument crashes.
• Dependency Conflicts: Installing the latest Paddle suites on ARM64 caused internal conflicts where fonts and utility objects (paddlex.utils.fonts.Font) were no longer automatically handled, breaking standard JSON serialization.
• The Solution: We had to revert to a "barebones" implementation, stripping all advanced parameters and manually sanitizing the output to bypass these internal object errors.

3. CPU Success vs. GPU Failure:

• CPU: Stable after stripping all modern parameters.
• GPU (Help Needed): Despite trying various CUDA-enabled ARM images, enabling use_gpu=True consistently failed. We suspect a mismatch between Blackwell's new architecture and the pre-built binaries available for ARM64.

4. Request for Support:

• Has anyone identified a specific "Golden Version" of PaddlePaddle and PaddleOCR for NVIDIA Blackwell?
• Are there specific build flags or Docker images that correctly recognize the Blackwell GPU cores on ARM-based servers?

Greetings from Türkiye! 🇹🇷
We hope sharing our struggle with library versioning helps others avoid the same pitfalls on ARM-based infrastructure.