You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: data-exports/README.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -75,13 +75,13 @@ Cross-account access is possible but can be difficult to maintain, considering t
75
75
Yes. Throughout an organization's lifecycle, mergers and acquisitions may occur, so this approach prepares you for potential future scenarios.
76
76
77
77
### Can I use S3 Intelligent Tiering or S3 Infrequent Access (IA) for my CUR data connected to Athena?
78
-
We strongly recommend **against** using S3 IA or other storage tiers for CUR data that is connected to Athena, especially if you have active FinOps users querying this data. Here's why:
78
+
We strongly recommend **against** using S3 IA for CUR data that is connected to Athena, especially if you have active FinOps users querying this data. Here's why:
79
79
- CUDOS typically only retrieves data for the last 7 months, so theoretically older data could be moved to S3 IA or managed with Intelligent Tiering.
80
80
- Moving older CUR parquet files to IA could potentially reduce storage costs by up to 45%.
81
81
-**However**, this only saves money if the data isn't frequently accessed. With S3 IA, you're charged $0.01 per GB retrieved.
82
82
- Athena uses multiple computational nodes in parallel, and complex queries can multiply data reads dramatically. For every 1GB of data you want to scan, Athena might perform up to 75GB of S3 reads.
83
83
- If someone runs a query without properly limiting it to specific billing periods, the retrieval costs can be astronomical. For example:
84
84
* Scanning a full CUR of 600GB: `600GB × 75 × $0.01/GB` = `$450.00` for just one query!
85
85
- Due to this risk of human error, we do not use storage tiering as a default and strongly advise against it for CUR data connected to Athena.
86
-
87
-
If you still want to optimize storage costs, consider replicating your CUR to another bucket and configuring a lifecycle policy to delete older data from the main bucket connected to Athena.
86
+
We also advise agains Intelligent Tiering by default.
87
+
- KPI Dashboard - one of our foundational dashboards - scans the entire CUR (Cost and Usage Report) data to detect the first snapshot and determine its age. This prevents AWS Intelligent Tiering from functioning effectively, as it forces all data to remain in frequent access tiers and result is unnecessary additional monitoring costs with no cost-saving benefits.
0 commit comments