Tentatively eliminate graph break overhead#3741

Draft

cehongwang wants to merge 8 commits intomainfrom

Commits on Aug 1, 2025

Tentatively eliminate graph break overhead
cehongwang
committed

Commits on Aug 5, 2025

Added stream manipulation and output tensor reusage
cehongwang
committed

Commits on Aug 6, 2025

Commits on Aug 7, 2025

Added some comments and an edge case
cehongwang
committed

Commits on Aug 20, 2025

Uncommented some code
cehongwang
committed

Commits on Aug 27, 2025

Added cpp optimization still slower than python runtime
cehongwang
committed

Commits on Sep 9, 2025

Fixed the comments
cehongwang
committed