Skip to content

Commit 72376b6

Browse files
dedece35dedece35
authored
Merge pull request #386 from cleophass/AvoidNonPinnedMemoryForDataloaders
GCI102 AvoidNonPinnedMemoryForDataloaders #Python #DLG #RulesSpecifications
2 parents 9777b90 + 99498e2 commit 72376b6

File tree

6 files changed

+98
-0
lines changed

6 files changed

+98
-0
lines changed

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
99

1010
### Added
1111

12+
- [#386](https://github.com/green-code-initiative/creedengo-rules-specifications/pull/386) Add rule GCI102, recommending the use of pinned memory for the dataloader when transferring data from the CPU to the GPU.
1213
- [#385](https://github.com/green-code-initiative/creedengo-rules-specifications/pull/385) Added documentation for the rule : disables bias in convolutional layers preceding Batch Normalization.
1314
- [#384](https://github.com/green-code-initiative/creedengo-rules-specifications/pull/384) Add specifications for rule GCI100, this rule is specific to Python because it's based on the `PyTorch` library, a library used for Deep Learning.
1415
- [#379](https://github.com/green-code-initiative/creedengo-rules-specifications/pull/379) Add rule GCI99 Avoid CSV Format, this rule is designed for Python but it can be implemented in other languages. The rule suggests using more efficient formats like Feather or Parquet instead of CSV.

‎RULES.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,7 @@ Some are applicable for different technologies.
7878
| GCI99 | Data: Avoid CSV Format | The Parquet format is faster to write to, lighter in weight and faster to read data from. It is suitable for use cases where there would be a lot of data I/O, especially with Cloud storage. | | 🚀 | 🚀 | 🚀 | 🚀 | 🚀 | 🚀 | 🚀 |
7979
| GCI100 | Wrap PyTorch Inference in `torch.no_grad()` | Using a PyTorch model in evaluation mode without wrapping inference in `torch.no_grad()` leads to unnecessary gradient tracking | | 🚫 | 🚫 | 🚫 | 🚀 | 🚫 | 🚫 | 🚫 |
8080
| GCI101 | AI: Avoid Bias in Conv Layers Before Batch Norm | Disable bias in convolutional layers when it's followed by a batch norm layer | | 🚫 | 🚫 | 🚫 | 🚀 | 🚫 | 🚫 | 🚫 |
81+
| GCI102 | Use pinned memory on DataLoader when using GPU | This rule applies to PyTorch data loading, where the use of pinned memory can significantly optimize data transfer between CPU and GPU. | | 🚫 | 🚫 | 🚫 | ✅ | 🚫 | 🚫 | 🚫 |
8182
| GCI203 | Detect unoptimized file formats | When it is possible, to use svg format image over other image format | | 🚧 | 🚀 | 🚀 | ✅ | 🚀 | 🚀 | 🚫 |
8283
| GCI404 | Avoid list comprehension in iterations | Use generator comprehension instead of list comprehension in for loop declaration | | 🚫 | 🚫 | 🚫 | ✅ | 🚫 | 🚫 | 🚫 |
8384
| GCI522 | Sobriety: Brightness Override | To avoid draining the battery, iOS and Android devices adapt the brightness of the screen depending on the environment light. | | 🚫 | 🚫 | ✅ | 🚫 | 🚫 | 🚫 | 🚫 |

‎src/main/rules/GCI102/GCI102.json‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
{
2+
"title": "AI Use pinned memory on DataLoader when using GPU",
3+
"type": "CODE_SMELL",
4+
"status": "ready",
5+
"remediation": {
6+
"func": "Constant/Issue",
7+
"constantCost": "10min"
8+
},
9+
"tags": [
10+
"creedengo",
11+
"eco-design",
12+
"performance",
13+
"memory",
14+
"ai",
15+
"pytorch"
16+
],
17+
"defaultSeverity": "Minor"
18+
}

‎src/main/rules/GCI102/python/GCI102.asciidoc‎

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
This rule applies to PyTorch data loading, where the use of pinned memory can significantly optimize data transfer between CPU and GPU.
2+
3+
== Non Compliant Code Example
4+
5+
[source,python]
6+
----
7+
train_loader = torch.utils.data.DataLoader(
8+
dataset,
9+
batch_size=64,
10+
shuffle=True,
11+
pin_memory=False # Not using pinned memory
12+
)
13+
----
14+
15+
In this example, the DataLoader does not use pinned memory, which leads to slower host-to-device data transfers.
16+
17+
== Compliant Solution
18+
19+
[source,python]
20+
----
21+
train_loader = torch.utils.data.DataLoader(
22+
dataset,
23+
batch_size=64,
24+
shuffle=True,
25+
pin_memory=True # Enables faster transfer to GPU
26+
)
27+
----
28+
29+
When `pin_memory=True`, PyTorch allocates page-locked memory on the host side, allowing for faster data transfer to the GPU via DMA (Direct Memory Access).
30+
31+
== Relevance Analysis
32+
33+
Experiments were conducted to evaluate the performance and environmental impact of using pinned memory in DataLoaders.
34+
35+
=== Configuration
36+
37+
* Processor: Intel(R) Xeon(R) CPU 3.80GHz
38+
* RAM: 64GB
39+
* GPU: NVIDIA Quadro RTX 6000
40+
* CO₂ Emissions Measurement: https://mlco2.github.io/codecarbon/[CodeCarbon]
41+
* Framework: PyTorch
42+
43+
=== Context
44+
45+
Two training configurations were compared:
46+
- One using standard memory allocation (`pin_memory=False`)
47+
- One using pinned memory (`pin_memory=True`)
48+
49+
Metrics assessed:
50+
- Average batch processing time
51+
- Total training time
52+
- CO₂ emissions
53+
54+
=== Impact Analysis
55+
56+
image:image.png[]
57+
58+
image::results.png[]
59+
60+
- **Batch Processing Time:** Reduced from 0.0472s to 0.0378s (~20% improvement).
61+
- **Training Time:** Decreased by 9.82% when using pinned memory.
62+
- **Carbon Emissions:** Lowered by 7.56%, indicating a measurable environmental benefit.
63+
64+
The improvements observed are particularly significant in large-scale or long-running training tasks, where data transfer becomes a bottleneck.
65+
66+
== Conclusion
67+
68+
Enabling pinned memory in PyTorch DataLoaders:
69+
- Reduces batch processing time significantly
70+
- Slightly shortens total training duration
71+
- Contributes to lowering CO₂ emissions
72+
- Is a recommended best practice for GPU-accelerated training
73+
74+
== References
75+
Credit : https://github.com/AghilesAzzoug/GreenPyData
76+
77+
- https://pytorch.org/docs/stable/data.html
78+
- NVIDIA CUDA Documentation on Pinned Memory: https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#page-locked-host-memory

‎src/main/rules/GCI102/python/image.png‎

61.8 KB
Loading

‎src/main/rules/GCI102/python/results.png‎

41.8 KB
Loading

0 commit comments

Comments
 (0)