Merge branch 'PaddlePaddle:develop' into dec1

Author: Dayuxiaoshui

GraphNet_technical_report.pdf

@@ -3,20 +3,19 @@
 
 <div align="center">
 
-![](https://img.shields.io/badge/version-v0.1-brightgreen)
 ![](https://img.shields.io/github/issues/PaddlePaddle/GraphNet?label=open%20issues)
-[![Documentation](https://img.shields.io/badge/documentation-blue)](./GraphNet_technical_report.pdf)
+[![arXiv](https://img.shields.io/badge/arXiv-2510.24035-b31b1b.svg)](https://arxiv.org/abs/2510.24035)
 <a href="https://github.com/user-attachments/assets/125e3494-25c9-4494-9acd-8ad65ca85d03"><img src="https://img.shields.io/badge/微信-green?logo=wechat&amp"></a>
 </div>
 
 **GraphNet** is a large-scale dataset of deep learning **computation graphs**, built as a standard benchmark for **tensor compiler** optimization. It provides over 2.7K computation graphs extracted from state-of-the-art deep learning models spanning diverse tasks and ML frameworks. With standardized formats and rich metadata, GraphNet enables fair comparison and reproducible evaluation of the general optimization capabilities of tensor compilers, thereby supporting advanced research such as AI for System on compilers.
 
-## News
-- [2025-10-14] ✨ Our technical report is out: a detailed study of dataset construction and compiler benchmarking, introducing the novel performance metrics Speedup Score S(t) and Error-aware Speedup Score ES(t). [📘 GraphNet: A Large-Scale Computational Graph Dataset for Tensor Compiler Research](./GraphNet_technical_report.pdf)
+## 📣 News
+- [2025-10-14] ✨ Our technical report is out: a detailed study of dataset construction and compiler benchmarking, introducing the novel performance metrics Speedup Score S(t) and Error-aware Speedup Score ES(t). [📘 GraphNet: A Large-Scale Computational Graph Dataset for Tensor Compiler Research](https://arxiv.org/abs/2510.24035)
 - [2025-8-20] 🚀 The second round of [open contribution tasks](https://github.com/PaddlePaddle/Paddle/issues/74773) was released. (completed ✅)
 - [2025-7-30] 🚀 The first round of [open contribution tasks](https://github.com/PaddlePaddle/GraphNet/issues/44) was released.  (completed ✅)
-## Benchmark Results
-We evaluate two representative tensor compiler backends, CINN (PaddlePaddle) and TorchInductor (PyTorch), on GraphNet's NLP and CV subsets. The evaluation adopts two quantitative metrics proposed in the [Technical Report](./GraphNet_technical_report.pdf):
+## 📊 Benchmark Results
+We evaluate two representative tensor compiler backends, CINN (PaddlePaddle) and TorchInductor (PyTorch), on GraphNet's NLP and CV subsets. The evaluation adopts two quantitative metrics proposed in the [Technical Report](https://arxiv.org/abs/2510.24035):
 - **Speedup Score** S(t) — evaluates compiler performance under varying numerical tolerance levels.
 <div align="center">
   <img src="/pics/St-result.jpg" alt="Speedup Score S_t Results" width="80%">
@@ -28,7 +27,7 @@ We evaluate two representative tensor compiler backends, CINN (PaddlePaddle) and
 
 </div>
 
-## Quick Start
+## ⚡ Quick Start
 This section shows how to evaluate tensor compilers and reproduce benchmark results (for compiler users and developers),
 as well as how to contribute new computation graphs (for GraphNet contributors).
 
@@ -97,12 +96,12 @@ python -m graph_net.plot_violin \
 
 The scripts are designed to process a file structure as `/benchmark_path/category_name/`, and items on x-axis are identified by name of the sub-directories. After executing, several summary plots of result in categories (model tasks, libraries...) will be exported to `$GRAPH_NET_BENCHMARK_PATH`.
 
-## 🧱 Construction & Contribution Guide
+### 🧱 Construction & Contribution Guide
 Want to understand how GraphNet is built or contribute new samples?
 Check out the [Construction Guide](./docs/README_contribute.md) for details on the extraction and validation workflow.
 
 
-## Future Roadmap
+## 🚀 Future Roadmap
 
 1. Scale GraphNet to 10K+ graphs.
 2. Further annotate GraphNet samples into more granular sub-categories
@@ -136,10 +135,13 @@ GraphNet is released under the [MIT License](./LICENSE).
 If you find this project helpful, please cite:
 
 ```bibtex
-@article{li2025graphnet,
-  title     = {GraphNet: A Large-Scale Computational Graph Dataset for Tensor Compiler Research},
-  author    = {Xinqi Li and Yiqun Liu and Shan Jiang and Enrong Zheng and Huaijin Zheng and Wenhao Dai and Haodong Deng and Dianhai Yu and Yanjun Ma},
-  year      = {2025},
-  url       = {https://github.com/PaddlePaddle/GraphNet/blob/develop/GraphNet_technical_report.pdf}
+@misc{li2025graphnetlargescalecomputationalgraph,
+      title={GraphNet: A Large-Scale Computational Graph Dataset for Tensor Compiler Research}, 
+      author={Xinqi Li and Yiqun Liu and Shan Jiang and Enrong Zheng and Huaijin Zheng and Wenhao Dai and Haodong Deng and Dianhai Yu and Yanjun Ma},
+      year={2025},
+      eprint={2510.24035},
+      archivePrefix={arXiv},
+      primaryClass={cs.LG},
+      url={https://arxiv.org/abs/2510.24035}, 
 }
 ```

README.md

@@ -65,4 +65,23 @@ python -m graph_net.torch.validate \
   --model-path $GRAPH_NET_EXTRACT_WORKSPACE/model_name
 ```
 
-All the **construction constraints** will be examined automatically. After passing validation, a unique `graph_hash.txt` will be generated and later checked in CI procedure to avoid redundant.
+All the **construction constraints** will be examined automatically. After passing validation, a unique `graph_hash.txt` will be generated and later checked in CI procedure to avoid redundant.
+
+## 📁 Repository Structure
+This repository is organized as follows:
+
+| Directory | Description |
+|------------|--------------|
+| **graph_net/** | Core module for graph extraction, validation, and benchmarking |
+| **paddle_samples/** | Computation graph samples extracted from PaddlePaddle |
+| **samples/** | Computation graph samples extracted from PyTorch |
+| **docs/** | Technical documents and contributor guides|
+
+Below is the structure of the **graph_net/**:
+```text
+graph_net/
+ ├─ config/    # Config files, params
+ ├─ paddle/    # PaddlePaddle graph extraction & validation
+ ├─ torch/     # PyTorch graph extraction & validation
+ ├─ test/      # Unit tests and example scripts
+ └─ *.py       # Benchmark & analysis scripts 

CONTRIBUTE_TUTORIAL.md docs/CONTRIBUTE_TUTORIAL.mdCONTRIBUTE_TUTORIAL.md renamed to docs/CONTRIBUTE_TUTORIAL.md

@@ -254,6 +254,10 @@ def print_stat_info(
     # pi is a list of constants for t > 0 for each group
     pi = [0, 0]
 
+    is_correct_at_t1 = [False] * total_samples
+    speedup_at_t1 = [None] * total_samples
+    fail_type_at_t1 = ["CORRECT"] * total_samples
+
     final_correct_count = 0
     final_correct_negative_speedup_count = 0
     final_correct_speedups = []
@@ -291,8 +295,8 @@ def print_stat_info(
                             get_correctness(eager_dtypes[i], t_key, correctness_data, i)
                             for i in range(output_count)
                         )
-                    if not is_correct:
-                        fail_type = "accuracy"
+                if not is_correct:
+                    fail_type = "accuracy"
 
             # Collect statistics
             if is_correct:
@@ -306,6 +310,11 @@ def print_stat_info(
             if fail_type == "accuracy":
                 acc_failure_count += 1
 
+            if t_key == 1:
+                is_correct_at_t1[idx] = is_correct
+                speedup_at_t1[idx] = speedup
+                fail_type_at_t1[idx] = fail_type if fail_type is not None else "CORRECT"
+
             # S(t) calculation
             if fail_type is not None or speedup is None:
                 regularized_speedup = fpdb
@@ -320,37 +329,25 @@ def print_stat_info(
             # ES(t) calculation: based on state change
             rec_speedup_fake_degrad = 0
             if t_key < 1:
-                # When t < 1, ES behaves the same as S
                 if fail_type is not None or speedup is None:
                     rec_speedup_fake_degrad = fpdb
-                    # print(f"sample: {sample.get('configuration').get('model')}, fail_type: {fail_type}, rec_speedup_fake_degrad: {rec_speedup_fake_degrad}")
                 else:
                     rec_speedup_fake_degrad = (
                         speedup ** (negative_speedup_penalty + 1)
                         if speedup < 1
                         else speedup
                     )
             else:
-                # When t >= 1, ES starts applying stepwise logic
-                # ES curve's stepwise state, initialized as 'CORRECT'
-                es_status = ["CORRECT"] * total_samples
-                if es_status[idx] == "CORRECT" and fail_type is not None:
-                    es_status[idx] = fail_type
-
-                if (
-                    es_status[idx] is not None
-                    and es_status[idx] != "CORRECT"
-                    or speedup is None
-                ):
+                if not is_correct_at_t1[idx] or speedup_at_t1[idx] is None:
+                    fail_type_frozen = fail_type_at_t1[idx]
                     rec_speedup_fake_degrad = fake_perf_degrad(
-                        t_key, es_status[idx], fpdb
+                        t_key, fail_type_frozen, fpdb
                     )
-                    # print(f"sample: {sample.get('configuration').get('model')}, error type: {es_status[idx]}, rec_speedup_fake_degrad: {rec_speedup_fake_degrad}")
-                else:  # Still in a "CORRECT" state
+                else:
                     rec_speedup_fake_degrad = (
-                        speedup ** (negative_speedup_penalty + 1)
-                        if speedup < 1
-                        else speedup
+                        speedup_at_t1[idx] ** (negative_speedup_penalty + 1)
+                        if speedup_at_t1[idx] < 1
+                        else speedup_at_t1[idx]
                     )
             rectified_speedups_fake_degrad.append(rec_speedup_fake_degrad)
 
@@ -399,4 +396,3 @@ def print_stat_info(
     print(f"    - pi: {pi}")
 
     return s_scores, s_scores_fake_degrad
-    return s_scores, es_scores

CONTRIBUTE_TUTORIAL_cn.md docs/CONTRIBUTE_TUTORIAL_cn.mdCONTRIBUTE_TUTORIAL_cn.md renamed to docs/CONTRIBUTE_TUTORIAL_cn.md

@@ -0,0 +1,23 @@
+import paddle
+from graph_net.paddle.backend.graph_compiler_backend import GraphCompilerBackend
+
+
+class CinnBackend(GraphCompilerBackend):
+    def __call__(self, model, input_spec=None):
+        build_strategy = paddle.static.BuildStrategy()
+        compiled_model = paddle.jit.to_static(
+            model,
+            input_spec=input_spec,
+            build_strategy=build_strategy,
+            full_graph=True,
+        )
+        compiled_model.eval()
+        program = compiled_model.forward.concrete_program.main_program
+        return compiled_model
+
+    def synchronize(self):
+        if (
+            paddle.device.is_compiled_with_cuda()
+            or paddle.device.is_compiled_with_rocm()
+        ):
+            paddle.device.synchronize()

docs/README_contribute.md

@@ -0,0 +1,6 @@
+class GraphCompilerBackend:
+    def __call__(self, model, input_spec=None):
+        raise NotImplementedError()
+
+    def synchronize(self):
+        raise NotImplementedError()

graph_net/analysis_util.py

@@ -0,0 +1,14 @@
+import paddle
+from graph_net.paddle.backend.graph_compiler_backend import GraphCompilerBackend
+
+
+class NopeBackend(GraphCompilerBackend):
+    def __call__(self, model, input_spec=None):
+        return model
+
+    def synchronize(self):
+        if (
+            paddle.device.is_compiled_with_cuda()
+            or paddle.device.is_compiled_with_rocm()
+        ):
+            paddle.device.synchronize()