Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ The script supports the following environment variables:
* OPENCOST_PARQUET_RESOLUTION: Duration to use as resolution in Prometheus queries. Smaller values (i.e., higher resolutions) will provide better accuracy, but worse performance (i.e., slower query time, higher memory use). Larger values (i.e., lower resolutions) will perform better but at the expense of lower accuracy for short-running workloads.
* OPENCOST_PARQUET_ACCUMULATE: If `"true"`, sum the entire range of time intervals into a single set. Default value is `"false"`.
* OPENCOST_PARQUET_INCLUDE_IDLE: Whether to return the calculated __idle__ field for the query. Default is `"false"`.
* OPENCOST_PARQUET_SHARE_IDLE: Whether to distribute idle costs across all allocations proportionally. Default is `"false"`.
* OPENCOST_PARQUET_IDLE_BY_NODE: If `"true"`, idle allocations are created on a per-node basis, which will result in different values when shared and more idle allocations when split. Default is `"false"`.
* OPENCOST_PARQUET_STORAGE_BACKEND: The storage backend to use. Supports `aws`, `azure`, `gcp`. See below for Azure and GCP-specific variables.
* OPENCOST_PARQUET_JSON_SEPARATOR: The OpenCost API returns nested objects. The used [JSON normalization method](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.json_normalize.html) allows for a custom separator. Use this to specify the separator of your choice.
Expand Down
7 changes: 7 additions & 0 deletions src/opencost_parquet_exporter.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ def get_config(
accumulate=None,
storage_backend=None,
include_idle=None,
share_idle=None,
idle_by_node=None,
):
"""
Expand Down Expand Up @@ -85,6 +86,9 @@ def get_config(
- include_idle (str): Whether to return the calculated __idle__ field for the query,
defaults to the 'OPENCOST_PARQUET_INCLUDE_IDLE' environment
variable, or 'false' if not set.
- share_idle (str): Whether to distribute idle costs across all allocations proportionally,
defaults to the 'OPENCOST_PARQUET_SHARE_IDLE' environment
variable, or 'false' if not set.
- idle_by_node (str): If true, idle allocations are created on a per node basis,
defaults to the 'OPENCOST_PARQUET_IDLE_BY_NODE' environment
variable, or 'false' if not set.
Expand Down Expand Up @@ -125,6 +129,8 @@ def get_config(
idle_by_node = os.environ.get('OPENCOST_PARQUET_IDLE_BY_NODE', 'false')
if include_idle is None:
include_idle = os.environ.get('OPENCOST_PARQUET_INCLUDE_IDLE', 'false')
if share_idle is None:
share_idle = os.environ.get('OPENCOST_PARQUET_SHARE_IDLE', 'false')
if storage_backend is None:
storage_backend = os.environ.get(
'OPENCOST_PARQUET_STORAGE_BACKEND', 'aws') # For backward compatibility
Expand Down Expand Up @@ -163,6 +169,7 @@ def get_config(
config['params'] = [
("window", window),
("includeIdle", include_idle),
("shareIdle", share_idle),
("idleByNode", idle_by_node),
("includeProportionalAssetResourceCosts", "false"),
("format", "json")
Expand Down