Proxmox FOCUS Export

Export Proxmox virtual machine and disk usage data in the FOCUS (FinOps Open Cost and Usage Specification) CSV format. This enables standardized cloud cost reporting for on-premise Proxmox infrastructure, allowing integration with FinOps tools and processes.

Features

• FOCUS-compliant CSV output - Industry-standard format for cost data
• Historical runtime tracking - Uses Proxmox RRD to charge only for actual VM uptime (not just allocated resources)
• Harness CCM integration - Auto-upload cost data to Harness CCM for centralized cost visibility
• Multi-layer configuration - Environment variables, TOML files, and CLI arguments with precedence
• Flexible granularity - Monthly or daily cost line items
• Custom cost rates - Configure per-host and per-datastore pricing
• Comprehensive cost breakdown - Separate line items for CPU, memory, and disk storage

Installation

1. Clone this repository:

git clone <repository-url>
cd proxmox-focus-export

2. Install dependencies:

pip install -r requirements.txt

For Harness CCM integration (optional):

pip install harness-ccm-external-data

3. Set up configuration (see Configuration section below)

Quick Start

1. Copy the example files:

cp .env.example .env
cp config.example.toml config.toml

2. Edit .env with your Proxmox credentials:

PROXMOX_HOST=proxmox.example.com
PROXMOX_USER=monitoring@pam
PROXMOX_PASSWORD=your_password

3. Edit config.toml with your cost rates

4. Run the export:

python -m src.main \
  --config config.toml \
  --start-date 2026-02-01 \
  --end-date 2026-02-28 \
  --output february_costs.csv

Configuration

Configuration is loaded in this order (lowest to highest precedence):

1. Environment variables (lowest)
2. TOML configuration file
3. Command-line arguments (highest)

This allows flexible deployment scenarios:

• Use environment variables for containerized deployments
• Use TOML files for repeatable configurations
• Use CLI arguments for one-off overrides

Environment Variables

See .env.example for all available options. Key variables:

# Proxmox API
PROXMOX_HOST=proxmox.example.com
PROXMOX_PORT=8006
PROXMOX_USER=monitoring@pam
PROXMOX_PASSWORD=secure_password
PROXMOX_VERIFY_SSL=false

# Export settings
EXPORT_GRANULARITY=monthly
EXPORT_OUTPUT_FILE=output.csv

# Cost rates
COST_RATES_HOSTS_DEFAULT_CPU_PER_CORE=0.05
COST_RATES_HOSTS_DEFAULT_MEMORY_PER_GB=0.01
COST_RATES_DATASTORES_DEFAULT_STORAGE_PER_GB=0.0001

TOML Configuration File

See config.example.toml for a fully commented example. The TOML file supports:

• Proxmox API connection settings
• Export settings (granularity, output file, region)
• Default cost rates for hosts and datastores
• Host-specific rate overrides (by node hostname)
• Datastore-specific rate overrides (by storage name)

Example:

[proxmox]
host = "proxmox.example.com"
user = "monitoring@pam"
verify_ssl = false

[export]
granularity = "monthly"
output_file = "output.csv"
region = "us-datacenter-1"

[cost_rates.hosts.default]
cpu_per_core = 0.05
memory_per_gb = 0.01

[cost_rates.hosts."pve-node-premium"]
cpu_per_core = 0.08
memory_per_gb = 0.015

[cost_rates.datastores.default]
storage_per_gb = 0.0001

[cost_rates.datastores."ssd-storage"]
storage_per_gb = 0.0003

Command-Line Arguments

Command-line arguments have the highest precedence and override both environment variables and TOML config:

python -m src.main \
  --config config.toml \              # TOML configuration file
  --start-date 2026-02-01 \           # Billing period start (required)
  --end-date 2026-02-28 \             # Billing period end (required)
  --output monthly_costs.csv \        # Output CSV file
  --granularity daily \               # monthly or daily
  --region us-datacenter-1 \          # Region name for FOCUS
  --proxmox-host proxmox.example.com  # Proxmox connection overrides
  --proxmox-user monitoring@pam \
  --proxmox-password secret

Granularity Options

Monthly Granularity

One line item per resource for the entire billing period. More compact output, ideal for billing summaries.

python -m src.main --config config.toml --granularity monthly \
  --start-date 2026-02-01 --end-date 2026-02-28

Output: For a VM with 2 CPUs, 8GB RAM, and 100GB disk, you get 3 line items for the entire month:

• CPU: 672 hours (28 days × 24 hours)
• Memory: 5,376 GB-hours (8 GB × 672 hours)
• Disk: 67,200 GB-hours (100 GB × 672 hours)

Daily Granularity

One line item per resource per day. Better for trend analysis and detailed cost tracking.

python -m src.main --config config.toml --granularity daily \
  --start-date 2026-02-01 --end-date 2026-02-03

Output: For the same VM over 3 days, you get 9 line items (3 days × 3 resources):

• CPU: 24 hours per day
• Memory: 192 GB-hours per day (8 GB × 24 hours)
• Disk: 2,400 GB-hours per day (100 GB × 24 hours)

Authentication

Two authentication methods are supported:

Password Authentication

PROXMOX_USER=monitoring@pam
PROXMOX_PASSWORD=your_password

Token-Based Authentication (Recommended)

PROXMOX_USER=monitoring@pam
PROXMOX_TOKEN_NAME=monitoring
PROXMOX_TOKEN_VALUE=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

To create an API token in Proxmox:

1. Go to Datacenter → Permissions → API Tokens
2. Create a new token for your user
3. Save the token value securely

Historical Runtime Tracking (RRD)

By default, the tool uses Proxmox's RRD (Round Robin Database) to track actual VM runtime for more accurate cost calculations.

How It Works

• ✅ Accurate uptime tracking: Only charges for hours VMs were actually running
• ✅ Historical data: Uses existing Proxmox RRD metrics (no setup required)
• ✅ Smart fallback: Uses allocated hours if RRD unavailable (newly created VMs, API errors)
• ✅ Time-weighted averages: Accounts for resource allocation changes over the billing period

The tool queries the Proxmox RRD API to retrieve historical CPU/memory metrics for each VM. It determines runtime by analyzing CPU activity data points:

• If CPU metrics exist for a data point, the VM was running during that interval
• Runtime hours = (number of running data points × interval duration)
• Uses time-weighted average of CPU/memory allocation over the period

Configuration

Enable/disable RRD tracking in config.toml:

[export]
use_rrd_metrics = true  # Default: true

Or via environment variable:

EXPORT_USE_RRD_METRICS=true

Or via CLI:

# Use RRD (default)
python -m src.main --use-rrd --start-date 2026-02-01 --end-date 2026-02-28

# Disable RRD (legacy behavior - assumes 100% allocation)
python -m src.main --no-use-rrd --start-date 2026-02-01 --end-date 2026-02-28

Example Impact

Without RRD (legacy behavior):

• VM stopped for 20 days in a 30-day month
• Billed for entire 30-day month = $100

With RRD (default):

• VM stopped for 20 days in a 30-day month
• Billed for 10 running days only = $33.33

RRD Status Output

When RRD is enabled, the tool displays runtime information:

Fetching RRD historical data...
  VM 100: 336.0 hours runtime (50.0%, 100.0% coverage) - running
  VM 101: 672.0 hours runtime (100.0%, 100.0% coverage) - running
  VM 102: 0.0 hours runtime (0.0%, 100.0% coverage) - stopped

• Runtime hours: Actual hours the VM was running
• Runtime %: Percentage of billing period VM was running
• Coverage %: Percentage of billing period with RRD data available
• Status: Whether VM is running or stopped

Fallback Behavior

If RRD data is unavailable (newly created VM, API error, etc.), the tool:

1. Logs a warning message
2. Falls back to current VM configuration
3. Assumes 100% allocation for the billing period (legacy behavior)
4. Continues processing other VMs normally

This ensures the export always succeeds, even when RRD data is missing.

Harness CCM Integration

The tool can automatically upload generated FOCUS CSVs to Harness CCM (Cloud Cost Management) for centralized cost visibility across your entire infrastructure.

Why Harness CCM?

• Unified dashboard: View Proxmox costs alongside AWS, GCP, Azure in one place
• Cost anomaly detection: Automatic alerts for unexpected cost increases
• Budgets & forecasts: Set budgets and predict future costs
• Chargeback & showback: Allocate costs to teams/departments using tags
• FinOps insights: Optimization recommendations and cost trends

Setup

1. Get Harness credentials:

   • Sign up for Harness CCM at harness.io
   • Generate an API key: Account Settings → Access Control → API Keys
   • Note your Account ID from the URL: app.harness.io/ng/account/<ACCOUNT_ID>/...

2. Install the Harness SDK (if not already installed):

   pip install harness-ccm-external-data

3. Configure in config.toml:

   [harness]
   enabled = true
   account_id = "your-harness-account-id"
   api_key = "your-harness-api-key"
   provider = "Proxmox"
   data_source = "Production Cluster"  # Name for this Proxmox cluster

   Or via environment variables (recommended for security):

   HARNESS_ENABLED=true
   HARNESS_ACCOUNT_ID=your-account-id
   HARNESS_PLATFORM_API_KEY=your-api-key
   HARNESS_PROVIDER=Proxmox
   HARNESS_DATA_SOURCE="Production Cluster"

Usage

When enabled, the tool will prompt for confirmation before uploading to Harness (default behavior):

python -m src.main --config config.toml --start-date 2026-02-01 --end-date 2026-02-28

Output:

...
Export complete!

============================================================
Harness CCM Upload Confirmation
============================================================
Provider:     Proxmox
Data Source:  Production Cluster
CSV File:     output.csv
Total Cost:   $1,234.56
Line Items:   456
============================================================

Upload this data to Harness CCM? [y/N]: y

Uploading to Harness CCM...
  Provider: Proxmox
  Data Source: Production Cluster
  File: output.csv
✓ Successfully uploaded to Harness CCM

Skipping the Confirmation Prompt

To automatically upload without confirmation (useful for automation/CI/CD):

Option 1: CLI flag

python -m src.main --config config.toml --start-date 2026-02-01 --end-date 2026-02-28 --harness-yes

Option 2: Configuration file

[harness]
enabled = true
prompt_before_upload = false  # Disable confirmation prompt

Option 3: Environment variable

HARNESS_PROMPT_BEFORE_UPLOAD=false

Data Source Management

• First run: The tool automatically creates a new data source in Harness
• Subsequent runs: Data is added to the existing data source
• Multiple clusters: Use different data_source names for each Proxmox cluster

Upload Behavior

• Idempotent: Duplicate uploads are detected via MD5 hash and skipped
• Invoice period: Auto-detected from billing period dates
• Automatic ingestion: Harness begins processing data immediately after upload
• Fail-safe: If upload fails, the CSV is still saved locally

Configuration Options

Enable/disable upload:

[harness]
enabled = false  # Upload disabled

Or via environment:

HARNESS_ENABLED=false

Confirmation prompt (default: enabled):

[harness]
prompt_before_upload = true  # Show confirmation before upload (default)

Or via environment:

HARNESS_PROMPT_BEFORE_UPLOAD=true

Or via CLI:

--harness-yes  # Skip confirmation prompt (auto-confirm upload)

Troubleshooting

Upload fails with authentication error:

• Verify your API key has CCM permissions
• Check account ID is correct
• Ensure API key hasn't expired

Data source not appearing in Harness:

• Wait 1-2 minutes for ingestion to complete
• Check Harness CCM → Data Sources → External Data
• Verify provider name matches exactly

Duplicate data:

• The tool prevents duplicates via MD5 hash checking
• If you see duplicates, ensure you're not manually uploading the same CSV

Cost Calculation

Costs are calculated based on actual runtime (when RRD enabled) and allocated resources:

• CPU: cores × cpu_rate × runtime_hours
• Memory: GB × memory_rate × runtime_hours
• Storage: GB × storage_rate × hours (always uses full period hours)

When RRD is disabled, runtime_hours equals the full billing period hours (legacy behavior).

Rate Configuration

Configure different rates for different resources:

• Default rates: Applied to all hosts/datastores unless overridden
• Host-specific rates: Different rates for different Proxmox nodes (e.g., premium hardware)
• Datastore-specific rates: Different rates for different storage tiers (SSD, HDD, NVMe)

Example: If pve-node-1 has premium hardware, configure higher rates:

[cost_rates.hosts."pve-node-1"]
cpu_per_core = 0.08
memory_per_gb = 0.015

FOCUS Billing Metadata

Configure billing account and provider information that appears in FOCUS output:

[focus]
provider_name = "Proxmox"           # Cloud provider name
billing_account_id = "acct-12345"   # Your billing account ID
billing_account_name = "IT Infrastructure"  # Billing account display name
sub_account_id = "dept-eng"         # Optional: department/project ID
sub_account_name = "Engineering"    # Optional: department/project name

These fields help organize costs in FinOps tools and allow you to:

• Track costs across multiple billing accounts
• Allocate costs to departments or projects via sub-accounts
• Identify the cloud provider in multi-cloud environments

VM Tags

Tags are automatically pulled from Proxmox VMs and included in the FOCUS Tags field. To add tags to a VM in Proxmox:

1. Select the VM in the Proxmox web interface
2. Go to the VM's configuration
3. Add tags in the Tags field (semicolon-separated)
4. Tags will appear in the FOCUS CSV as a semicolon-separated string

Tags are useful for:

• Cost allocation by environment (production, staging, dev)
• Identifying application or service ownership
• Filtering and grouping in FinOps analysis tools

FOCUS Output Format

The generated CSV follows the FOCUS specification with these columns:

Core Billing Fields

• BillingAccountId - Your billing account ID (configurable)
• BillingAccountName - Billing account display name (configurable)
• BillingPeriodStart / BillingPeriodEnd - Billing period dates
• ChargePeriodStart / ChargePeriodEnd - Charge period dates (same as billing)
• SubAccountId - Optional sub-account/department ID (configurable)
• SubAccountName - Optional sub-account/department name (configurable)

Cost Fields

• ChargeCategory - Type of charge (Usage, Purchase, Tax, etc.) - currently "Usage"
• BilledCost - Calculated cost in USD
• EffectiveCost - Effective cost after discounts (same as BilledCost)
• ChargeDescription - Description of the charge

Resource Fields

• ProviderName - Cloud provider name (configurable, default: "Proxmox")
• ServiceName - Service type (Compute or Storage)
• ServiceCategory - Service category (Compute or Storage)
• ResourceId - VM ID or disk ID
• ResourceName - VM name or disk name
• ResourceType - Virtual Machine or Disk
• SkuId - SKU identifier (e.g., "Compute-CPU", "Storage-Storage")

Usage Fields

• UsageQuantity - Hours or GB-hours
• ConsumedQuantity - Same as UsageQuantity
• UsageUnit - Hour or GB-Hour

Location Fields

• RegionName - Proxmox cluster name
• AvailabilityZone - Node name (for compute) or datastore name (for storage)

Tags

• Tags - Semicolon-separated list of VM tags from Proxmox

Example Output

Monthly Granularity (3 line items per VM)

BillingPeriodStart,BillingPeriodEnd,ChargePeriodStart,ChargePeriodEnd,ServiceName,ServiceCategory,ResourceId,ResourceName,ResourceType,UsageQuantity,UsageUnit,BilledCost,ChargeDescription,Region,AvailabilityZone
2026-02-01T00:00:00Z,2026-02-28T23:59:59Z,2026-02-01T00:00:00Z,2026-02-28T23:59:59Z,Compute,Compute,100,web-server,Virtual Machine,672.0000,Hour,33.6000,CPU allocation (1 cores),proxmox-cluster,pve-node-1
2026-02-01T00:00:00Z,2026-02-28T23:59:59Z,2026-02-01T00:00:00Z,2026-02-28T23:59:59Z,Compute,Compute,100,web-server,Virtual Machine,6720.0000,GB-Hour,67.2000,Memory allocation (10.0 GB),proxmox-cluster,pve-node-1
2026-02-01T00:00:00Z,2026-02-28T23:59:59Z,2026-02-01T00:00:00Z,2026-02-28T23:59:59Z,Storage,Storage,100-scsi0,web-server-scsi0,Disk,67200.0000,GB-Hour,6.7200,Storage allocation (100.0 GB),proxmox-cluster,local-lvm

Daily Granularity (3 line items per VM per day)

BillingPeriodStart,BillingPeriodEnd,ChargePeriodStart,ChargePeriodEnd,ServiceName,ServiceCategory,ResourceId,ResourceName,ResourceType,UsageQuantity,UsageUnit,BilledCost,ChargeDescription,Region,AvailabilityZone
2026-02-01T00:00:00Z,2026-02-01T23:59:59Z,2026-02-01T00:00:00Z,2026-02-01T23:59:59Z,Compute,Compute,100,web-server,Virtual Machine,24.0000,Hour,1.2000,CPU allocation (1 cores),proxmox-cluster,pve-node-1
2026-02-01T00:00:00Z,2026-02-01T23:59:59Z,2026-02-01T00:00:00Z,2026-02-01T23:59:59Z,Compute,Compute,100,web-server,Virtual Machine,240.0000,GB-Hour,2.4000,Memory allocation (10.0 GB),proxmox-cluster,pve-node-1
2026-02-01T00:00:00Z,2026-02-01T23:59:59Z,2026-02-01T00:00:00Z,2026-02-01T23:59:59Z,Storage,Storage,100-scsi0,web-server-scsi0,Disk,2400.0000,GB-Hour,0.2400,Storage allocation (100.0 GB),proxmox-cluster,local-lvm
2026-02-02T00:00:00Z,2026-02-02T23:59:59Z,2026-02-02T00:00:00Z,2026-02-02T23:59:59Z,Compute,Compute,100,web-server,Virtual Machine,24.0000,Hour,1.2000,CPU allocation (1 cores),proxmox-cluster,pve-node-1
...

Testing

Run the test suite:

python -m pytest tests/

Template Handling

VM templates are handled differently from active VMs:

• Templates only incur disk storage costs - No CPU or memory charges
• Automatic detection - Templates are identified by the template flag in Proxmox
• Cost reporting - Template costs appear as Storage line items in the FOCUS output

This reflects the reality that templates are inactive and only consume storage space, not compute resources.

Assumptions & Limitations

• Single cluster: Currently supports one Proxmox cluster per export
• Allocated resources: Costs based on allocated CPU/memory/disk at time of measurement
  • With RRD enabled (default): Uses time-weighted average allocation over billing period
  • With RRD disabled: Uses current VM configuration
• Runtime tracking: With RRD enabled (default), VMs are only billed for actual running hours
  • Automatic VM uptime detection based on RRD CPU metrics
  • VMs created or deleted mid-period are handled automatically by RRD data availability
• Configuration changes: RRD uses time-weighted averages to account for mid-period VM resizes
  • Note: Exact resize timing may not be captured due to RRD aggregation intervals
• CPU utilization: Costs based on allocated CPU cores, not actual CPU % usage
• Template costs: Templates only have disk costs, no CPU/memory costs
• UTC timestamps: All times in FOCUS output are UTC
• QEMU VMs only: Currently only supports QEMU virtual machines (LXC containers not included)

Troubleshooting

SSL Certificate Errors

If you see SSL verification errors, set:

PROXMOX_VERIFY_SSL=false

Or in config.toml:

[proxmox]
verify_ssl = false

Authentication Errors

Ensure your user has the necessary permissions:

• VM.Audit (to read VM configurations)
• Sys.Audit (to read cluster information)

Missing Cost Rates

If you see "Cost rates not configured" errors, ensure you have default rates configured:

[cost_rates.hosts.default]
cpu_per_core = 0.05
memory_per_gb = 0.01

[cost_rates.datastores.default]
storage_per_gb = 0.0001

License

This project is provided as-is for use in FinOps cost reporting workflows.

Contributing

Contributions welcome! Please open an issue or pull request.