tf-migrate - Claude Context Guide

This document provides comprehensive context about the tf-migrate project for AI agents starting with empty context.

Project Overview
Architecture
How Migrations Work
Resource Transformers
Testing System
Drift Exemptions System
Development Guide
Common Patterns

Project Overview

tf-migrate is a CLI tool for automatically migrating Terraform configurations between different versions of the Cloudflare Terraform Provider. State migration is handled by the provider's built-in UpgradeState/MoveState mechanisms — tf-migrate only transforms config (.tf files).

What It Does

Transforms .tf configuration files - Updates resource types, attribute names, and block structures
Handles complex transformations - One-to-many resource splits, nested block restructuring, API-based migrations
Generates import blocks - For new v5 resources that don't exist in state yet

Current Support

v4 → v5: Cloudflare Provider v4 to v5 (60+ resource types)
Future: v5 → v6, etc.

Key Use Cases

Automated provider upgrades - Bulk migrate large Terraform codebases
API-aware migrations - Fetch data from Cloudflare API when needed (e.g., tunnel route UUIDs)
Testing infrastructure - Validate migrations with real resources via E2E tests

Technology Stack

Language: Go 1.25+
HCL Parsing: github.com/hashicorp/hcl/v2
JSON querying: github.com/tidwall/gjson (used in TransformConfig for config-level decisions)
Cloudflare API: github.com/cloudflare/cloudflare-go/v6
CLI Framework: github.com/spf13/cobra

Architecture

Design Patterns

Chain of Responsibility Pattern - Transformations flow through a pipeline of handlers:

Input → Preprocess → Parse → Transform → Format → Output

Core Components

tf-migrate/
├── cmd/
│   ├── tf-migrate/          # Main migration CLI binary
│   └── e2e-runner/          # E2E test runner CLI binary
│
├── internal/
│   ├── pipeline/            # Pipeline orchestration
│   │   └── pipeline.go      # BuildConfigPipeline
│   │
│   ├── handlers/            # Pipeline handlers (chain of responsibility)
│   │   ├── pre_process.go   # Resource filtering, validation
│   │   ├── parse.go         # HCL parsing
│   │   ├── resource_transform.go  # Resource transformation orchestration
│   │   └── formatter.go     # HCL formatting
│   │
│   ├── resources/           # Per-resource migration implementations
│   │   ├── dns_record/      # cloudflare_dns_record v4→v5
│   │   ├── zone_setting/    # cloudflare_zone_settings_override v4→v5
│   │   ├── bot_management/  # cloudflare_bot_management v4→v5
│   │   └── ...              # 60+ resources
│   │
│   ├── datasources/         # Data source migrations
│   │   ├── zone/
│   │   └── ...
│   │
│   ├── registry/            # Resource transformer registry
│   │   └── registry.go      # Register, GetTransformer
│   │
│   ├── transform/           # Transformation interfaces and utilities
│   │   ├── transformer.go   # ResourceTransformer interface, Context, TransformResult
│   │   ├── hcl/             # HCL manipulation helpers (RenameAttribute, etc.)
│   │   └── state/           # JSON utilities for reading state in TransformConfig
│   │
│   ├── e2e-runner/          # E2E test runner implementation
│   │   ├── runner.go        # Test orchestration
│   │   ├── drift.go         # Drift detection & exemptions
│   │   ├── init.go          # Test initialization
│   │   └── migrate.go       # Migration step
│   │
│   └── logger/              # Logging utilities
│
├── integration/             # Integration tests
│   ├── v4_to_v5/           # v4→v5 migration tests
│   │   ├── integration_test.go
│   │   └── testdata/        # Test fixtures per resource
│   │       ├── dns_record/
│   │       ├── zone_setting/
│   │       └── ...
│   └── test_runner.go       # Shared test infrastructure
│
├── e2e/                     # End-to-end tests
│   ├── tf/v4/              # v4 test resources (generated)
│   ├── migrated-v4_to_v5/  # Migration output (generated)
│   ├── global-drift-exemptions.yaml    # Global drift exemptions
│   ├── drift-exemptions/    # Resource-specific drift exemptions
│   │   ├── zone_setting.yaml
│   │   ├── zone_dnssec.yaml
│   │   └── bot_management.yaml
│   └── DRIFT-EXEMPTIONS.md  # Drift exemptions docs (deprecated, see CLAUDE.md)
│
└── scripts/                 # Helper scripts
    ├── run-e2e-tests.sh    # E2E test runner
    └── ...

Pipeline Flow

Config File Pipeline

1. Preprocess Handler
   ↓ (filters resources, validates)
2. Parse Handler
   ↓ (HCL → AST)
3. Resource Transform Handler
   ↓ (registry.GetTransformer → resource.TransformConfig)
4. Formatter Handler
   ↓ (AST → formatted HCL)
Output: Migrated .tf file

Resource Transformer Registry

All resource transformers register themselves in init():

// internal/resources/dns_record/v4_to_v5.go
func init() {
    registry.Register(registry.ResourceEntry{
        Version:      "v4_to_v5",
        ResourceType: "cloudflare_dns_record",
        Transformer:  &DNSRecordTransformer{},
    })
}

The registry provides:

GetTransformer(version, resourceType) - Lookup transformer
ListResources(version) - List all available resources
IsRegistered(version, resourceType) - Check if resource supported

How Migrations Work

Transformation Types

1. Simple Attribute Rename

# v4
resource "cloudflare_dns_record" "example" {
  zone_id = "abc123"
  name    = "example.com"
  type    = "A"
  value   = "192.0.2.1"
  proxied = true
}

# v5 (no changes needed for dns_record)
resource "cloudflare_dns_record" "example" {
  zone_id = "abc123"
  name    = "example.com"
  type    = "A"
  content = "192.0.2.1"  # value → content
  proxied = true
}

2. Resource Type Rename

# v4
resource "cloudflare_access_application" "example" {
  # ...
}

# v5
resource "cloudflare_zero_trust_access_application" "example" {
  # ...
}

3. One-to-Many Resource Split

Most complex transformation pattern

# v4 - ONE resource
resource "cloudflare_zone_settings_override" "example" {
  zone_id = "abc123"
  settings {
    tls_1_3 = "on"
    minify {
      css = "on"
      js  = "on"
    }
  }
}

# v5 - MULTIPLE resources
resource "cloudflare_zone_setting" "example_tls_1_3" {
  zone_id    = "abc123"
  setting_id = "tls_1_3"
  value      = "on"
}

resource "cloudflare_zone_setting" "example_minify" {
  zone_id    = "abc123"
  setting_id = "minify"
  value = {
    css = "on"
    js  = "on"
  }
}

4. Nested Block Restructuring

# v4
resource "cloudflare_access_policy" "example" {
  application_id = "abc123"
  include {
    email = ["user@example.com"]
  }
  exclude {
    email_domain = ["contractor.com"]
  }
}

# v5
resource "cloudflare_zero_trust_access_policy" "example" {
  application_id = "abc123"
  include = [{
    email = ["user@example.com"]
  }]
  exclude = [{
    email_domain = ["contractor.com"]
  }]
}

5. API-Based Migration

Some resources require API calls to complete migration:

# v4
resource "cloudflare_tunnel_route" "example" {
  account_id = "abc123"
  tunnel_id  = "def456"
  network    = "10.0.0.0/16"  # v4 used this as the ID
}

# v5
resource "cloudflare_zero_trust_tunnel_cloudflared_route" "example" {
  account_id = "abc123"
  tunnel_id  = "def456"
  network    = "10.0.0.0/16"
  # ID must be UUID from API (not the network CIDR)
}

The transformer calls the Cloudflare API to fetch the UUID inside TransformConfig, resolving the resource ID before writing the import block.

Resource Transformers

Each resource has a v4_to_v5.go file implementing:

type ResourceTransformer interface {
    // Transform HCL configuration
    TransformConfig(ctx *transform.Context, block *hclwrite.Block) (*transform.TransformResult, error)
}

Example: DNS Record Transformer

// internal/resources/dns_record/v4_to_v5.go
type DNSRecordTransformer struct{}

func (t *DNSRecordTransformer) TransformConfig(ctx *transform.Context, block *hclwrite.Block) (*transform.TransformResult, error) {
    // No changes needed for dns_record in v4→v5
    return &transform.TransformResult{
        Blocks:         []*hclwrite.Block{block},
        RemoveOriginal: false,
    }, nil
}

Common Transformation Utilities

HCL Utilities (internal/transform/hcl/):

RenameAttribute(block, old, new) - Rename attribute
RenameBlock(block, old, new) - Rename block type
RemoveAttribute(block, name) - Remove attribute
GetAttribute(block, name) - Get attribute value
SetAttribute(block, name, value) - Set attribute value

Multi-Name Resources and Cross-File References

Some resources accept multiple v4 resource type names. For example, zero_trust_tunnel_cloudflared_route accepts both:

cloudflare_tunnel_route (deprecated v4 name)
cloudflare_zero_trust_tunnel_route (preferred v4 name)

Both map to the same v5 name: cloudflare_zero_trust_tunnel_cloudflared_route

The Solution

The GetResourceRename() interface returns ALL v4 names that map to the v5 name:

type ResourceRenamer interface {
    GetResourceRename() (oldTypes []string, newType string)
}

// Example implementation with multiple v4 names
func (m *V4ToV5Migrator) GetResourceRename() ([]string, string) {
    return []string{
        "cloudflare_tunnel_route",
        "cloudflare_zero_trust_tunnel_route",
    }, "cloudflare_zero_trust_tunnel_cloudflared_route"
}

// Example implementation with single v4 name
func (m *V4ToV5Migrator) GetResourceRename() ([]string, string) {
    return []string{"cloudflare_dns_record"}, "cloudflare_dns_record"
}

The global postprocessing step in cmd/tf-migrate/main.go loops through all old types and updates all cross-file references, regardless of which v4 name was used.

Example that now works correctly:

# File: virtual_network.tf
resource "cloudflare_zero_trust_tunnel_virtual_network" "my_vnet" {
  # ... (using "second" v4 name)
}

# File: route.tf
resource "cloudflare_tunnel_route" "my_route" {
  virtual_network_id = cloudflare_zero_trust_tunnel_virtual_network.my_vnet.id
  # After migration, this reference WILL be updated! ✅
}

When to Use Multiple Old Names

Return multiple old names in GetResourceRename() when:

Your migrator calls internal.RegisterMigrator() multiple times with different v4 names
Users might have cross-file references using any of the v4 names
All v4 names map to the same v5 name

Important: Include ALL v4 names in the array, even if one of them matches the v5 name. This ensures cross-file references are updated correctly.

Resources Using Multiple Old Names

The following resources return multiple old names (22 total):

zero_trust_tunnel_cloudflared_route
- Old names: cloudflare_tunnel_route, cloudflare_zero_trust_tunnel_route
zero_trust_tunnel_cloudflared_virtual_network
- Old names: cloudflare_tunnel_virtual_network, cloudflare_zero_trust_tunnel_virtual_network
zero_trust_tunnel_cloudflared
- Old names: cloudflare_tunnel, cloudflare_zero_trust_tunnel_cloudflared
zero_trust_tunnel_cloudflared_config
- Old names: cloudflare_tunnel_config, cloudflare_zero_trust_tunnel_cloudflared_config
zero_trust_device_profiles
- Old names: cloudflare_zero_trust_device_profiles, cloudflare_device_settings_policy
zero_trust_local_fallback_domain
- Old names: cloudflare_zero_trust_local_fallback_domain, cloudflare_fallback_domain
zero_trust_dlp_custom_profile
- Old names: cloudflare_dlp_profile, cloudflare_zero_trust_dlp_profile
zero_trust_gateway_settings
- Old names: cloudflare_teams_account, cloudflare_zero_trust_gateway_settings
zero_trust_organization
- Old names: cloudflare_access_organization, cloudflare_zero_trust_access_organization
zero_trust_access_application
- Old names: cloudflare_access_application, cloudflare_zero_trust_access_application
zero_trust_access_group
- Old names: cloudflare_access_group, cloudflare_zero_trust_access_group
zero_trust_access_identity_provider
- Old names: cloudflare_access_identity_provider, cloudflare_zero_trust_access_identity_provider
zero_trust_access_mtls_certificate
- Old names: cloudflare_access_mutual_tls_certificate, cloudflare_zero_trust_access_mtls_certificate
zero_trust_access_service_token
- Old names: cloudflare_access_service_token, cloudflare_zero_trust_access_service_token
zero_trust_device_managed_networks
- Old names: cloudflare_device_managed_networks, cloudflare_zero_trust_device_managed_networks
zero_trust_device_posture_integration
- Old names: cloudflare_device_posture_integration, cloudflare_zero_trust_device_posture_integration
zero_trust_device_posture_rule
- Old names: cloudflare_device_posture_rule, cloudflare_zero_trust_device_posture_rule
zero_trust_dex_test
- Old names: cloudflare_device_dex_test, cloudflare_zero_trust_dex_test
worker_route
- Old names: cloudflare_workers_route, cloudflare_worker_route
workers_script
- Old names: cloudflare_workers_script, cloudflare_worker_script
workers_for_platforms_dispatch_namespace
- Old names: cloudflare_workers_for_platforms_namespace, cloudflare_workers_for_platforms_dispatch_namespace
zero_trust_split_tunnel
- Old names: cloudflare_split_tunnel, cloudflare_zero_trust_split_tunnel

Testing

The integration test in integration/v4_to_v5/testdata/zero_trust_tunnel_cloudflared_virtual_network/ validates that cross-resource references using the "second" v4 name are properly updated (see Pattern 9 in that test).

Resource-Specific Documentation

Each resource has a README.md explaining:

Changes from v4 to v5
Configuration examples
Special cases and limitations

Example: internal/resources/zone_setting/README.md

Testing System

Test Layers

┌─────────────────────────────────────────┐
│          E2E Tests (Real API)           │  ← Full workflow with real Cloudflare resources
│  ./scripts/run-e2e-tests.sh            │
└─────────────────────────────────────────┘
                  ↑
┌─────────────────────────────────────────┐
│      Integration Tests (Fixtures)       │  ← Complete migration workflow with test data
│  make test-integration                  │
└─────────────────────────────────────────┘
                  ↑
┌─────────────────────────────────────────┐
│          Unit Tests (Go)                │  ← Individual component testing
│  go test ./...                          │
└─────────────────────────────────────────┘

1. Unit Tests

Test individual transformers:

# Run all unit tests
go test ./...

# Test specific resource
go test ./internal/resources/dns_record -v

# With coverage
go test ./... -cover

2. Integration Tests

Located in integration/v4_to_v5/testdata/:

testdata/
├── dns_record/
│   ├── input/            # v4 configuration files
│   │   └── dns_record.tf
│   └── expected/         # Expected v5 output files
│       └── dns_record.tf
├── zone_setting/
│   └── ...
└── bot_management/
    └── ...

Run integration tests:

# All v4→v5 integration tests
make test-integration

# Specific resource
go test -v -run TestV4ToV5Migration/DNSRecord

# Single resource with environment variable
TEST_RESOURCE=dns_record go test -v -run TestSingleResource

# Keep temp directory for debugging
KEEP_TEMP=true TEST_RESOURCE=dns_record go test -v -run TestSingleResource

3. E2E Tests

Most comprehensive testing - Uses real Cloudflare infrastructure.

E2E Workflow

1. Init      → Copy integration testdata to e2e/tf/v4/
2. V4 Apply  → Create real resources with v4 provider
3. Migrate   → Run tf-migrate (config only; provider upgrades state on first apply)
4. V5 Apply  → Apply v5 configs to existing infrastructure
5. Drift     → Verify v5 plan shows "No changes"

Prerequisites

export CLOUDFLARE_ACCOUNT_ID="your-account-id"
export CLOUDFLARE_ZONE_ID="your-zone-id"
export CLOUDFLARE_DOMAIN="your-test-domain.com"

# Authentication (choose one)
export CLOUDFLARE_API_TOKEN="your-api-token"  # Recommended
# OR
export CLOUDFLARE_EMAIL="your-email@example.com"
export CLOUDFLARE_API_KEY="your-api-key"

Run E2E Tests

# Build and run full suite
./scripts/run-e2e-tests.sh --apply-exemptions

# Or manually
make build-all
./bin/e2e-runner run --apply-exemptions

# Test specific resources
./bin/e2e-runner run --resources dns_record,zone_setting

# Individual commands
./bin/e2e-runner init
./bin/e2e-runner migrate
./bin/e2e-runner clean --modules module.dns_record

E2E Runner Features

✅ Credential sanitization - Prevents secrets in logs
✅ Drift detection - Validates successful migration
✅ Resource filtering - Test specific resources
✅ Colored output - Clear success/failure indicators
✅ 88 unit tests - E2E runner itself is well-tested

Import Annotations

Some resources cannot be created, only imported (e.g., zero_trust_organization).

Use annotations in module files:

# integration/v4_to_v5/testdata/zero_trust_organization/_e2e.tf

# tf-migrate:import-address=${var.cloudflare_account_id}
resource "cloudflare_access_organization" "test" {
  account_id  = var.cloudflare_account_id
  name        = "Test Organization"
  auth_domain = "test.cloudflareaccess.com"
}

E2E runner generates import blocks:

# e2e/tf/v4/main.tf (auto-generated)

import {
  to = module.zero_trust_organization.cloudflare_access_organization.test
  id = var.cloudflare_account_id
}

Skipping Resources from E2E Tests

Some resources cannot be tested in E2E environments due to lifecycle constraints (cannot be created/destroyed) or external dependencies. To exclude a resource from E2E tests while keeping integration tests:

Add E2E-SKIP marker to the *_e2e.tf file:

# E2E-SKIP: Brief reason why E2E testing is not possible
#
# REASON FOR SKIP:
# - Detailed explanation of constraints
# - Why automated testing is not feasible
#
# TESTING COVERAGE:
# ✓ Integration tests: What IS tested
# ✓ Provider tests: What IS tested
# ✗ E2E tests: Why NOT tested

# Rest of file content...

Pattern Rules:

Marker location: Must be in the first 20 lines of the *_e2e.tf file
Format: # E2E-SKIP: (case-sensitive, must be in a comment)
Documentation: Include detailed reasoning and alternative test coverage
Integration tests: Continue to work normally (use separate input files)

Example: BYO IP Prefix

The byo_ip_prefix resource is skipped from E2E tests because:

Cannot be created via Terraform (requires manual account manager provisioning)
Cannot be destroyed (active bindings prevent deletion)
Requires manual intervention during migration

See: integration/v4_to_v5/testdata/byo_ip_prefix/input/byo_ip_prefix_e2e.tf

E2E Runner Behavior:

# When initializing E2E tests:
./bin/e2e-runner init

# Output shows skipped resources:
Syncing resource files from testdata...
  ✓ dns_record/dns_record.tf (from dns_record_e2e.tf)
  ⊗ Skipped byo_ip_prefix (E2E-SKIP)
  ✓ zone_setting/zone_setting.tf (from zone_setting_e2e.tf)

  Total: 120 files synced
  Skipped: 1 modules (E2E-SKIP)

When to Use E2E-SKIP:

Use the E2E-SKIP marker when:

Resources cannot be created via Terraform/API
Resources cannot be destroyed (lifecycle constraints)
Resources require manual provisioning or external setup
E2E testing would require pre-existing infrastructure

Do NOT use E2E-SKIP for:

Resources that can be imported (use import annotations instead)
Resources with slow operations (improve test efficiency instead)
Temporary test failures (fix the underlying issue)

Drift Exemptions System

The drift exemptions system allows you to define acceptable differences between v4 and v5 provider behavior during E2E testing, preventing false positives from failing CI while still catching real migration bugs.

Hierarchical Configuration

Exemptions are organized in two levels:

Global exemptions (e2e/global-drift-exemptions.yaml) - Apply to all resources
Resource-specific exemptions (e2e/drift-exemptions/{resource}.yaml) - Override global settings

File Locations

e2e/
├── global-drift-exemptions.yaml    # Global exemptions (all resources)
└── drift-exemptions/
    ├── zone_setting.yaml           # Zone setting specific
    ├── zone_dnssec.yaml            # Zone DNSSEC specific
    ├── bot_management.yaml         # Bot management specific
    └── {resource}.yaml             # Any resource can have one

Global Exemptions

File: e2e/global-drift-exemptions.yaml

version: 1

exemptions:
  # Standard computed field exemptions
  - name: "computed_value_refreshes"
    description: "Ignore attributes that refresh to 'known after apply'"
    patterns:
      - '\(known after apply\)'
      - '= \{\} -> null'
    enabled: true

settings:
  apply_exemptions: true
  verbose_exemptions: false
  warn_unused_exemptions: false
  load_resource_exemptions: true

Resource-Specific Exemptions

File: e2e/drift-exemptions/zone_setting.yaml

version: 1

exemptions:
  # One-to-many transformation: v4 zone_settings_override -> multiple v5 zone_setting
  - name: "allow_zone_setting_creation"
    description: "Zone settings are created during v4->v5 migration"
    allow_resource_creation: true
    enabled: true

  # Plan-specific features
  - name: "unsupported_plan_features"
    description: "Features not available on free/pro plans"
    patterns:
      - "0rtt"
      - "tls_1_3"
    enabled: true

  # Disable global exemption for this resource
  - name: "computed_value_refreshes"
    enabled: false

settings:
  apply_exemptions: true
  verbose_exemptions: false

Exemption Schema

Required Fields

name - Unique identifier for the exemption
description - Human-readable explanation (what and why)
enabled - Boolean to enable/disable the exemption

Scope Filters (optional)

resource_types - List of resource types (e.g., ["cloudflare_zone"])
resource_name_patterns - Regex patterns for resource names (e.g., ["module\\.zone_setting\\..*"])
attributes - List of attribute names to match

Pattern Matching

Regex Patterns:

patterns:
  - 'status.*->.*"active"'  # Status changes to active
  - '\(known after apply\)' # Computed fields
  - '- email = .* -> null'  # Deletions

Simplified Patterns (recommended):

# Allow entire resource to be created
allow_resource_creation: true

# Allow entire resource to be destroyed
allow_resource_destruction: true

# Allow entire resource to be replaced
allow_resource_replacement: true

Usage Examples

Example 1: Allow Resource Creation

When migrating from a single v4 resource to multiple v5 resources:

exemptions:
  - name: "allow_zone_setting_creation"
    description: "v4 zone_settings_override splits into multiple v5 zone_setting"
    allow_resource_creation: true
    resource_types:
      - "cloudflare_zone_setting"
    enabled: true

Example 2: Resource-Specific Computed Fields

Only exempt computed fields for test resources:

exemptions:
  - name: "test_policy_computed"
    description: "Computed fields for test policies only"
    resource_name_patterns:
      - 'module\.zero_trust_access_policy\..*\.test'
    patterns:
      - 'precedence.*\(known after apply\)'
    enabled: true

Example 3: Override Global Exemption

Disable a global exemption for a specific resource:

exemptions:
  # Disable global timestamp exemption - we want to catch drift here
  - name: "computed_value_refreshes"
    enabled: false

Example 4: Instance-Specific Pattern

exemptions:
  - name: "test_resources_only"
    description: "Only for test resources"
    resource_name_patterns:
      - 'module\.test_.*'
      - 'module\.staging\..*'
    patterns:
      - "some-pattern"
    enabled: true

Running E2E Tests with Exemptions

# With exemptions (default in CI)
./scripts/run-e2e-tests.sh --apply-exemptions

# Without exemptions (strict mode)
./scripts/run-e2e-tests.sh

# For specific resources
./scripts/run-e2e-tests.sh --apply-exemptions --resources zone_setting,bot_management

Creating Resource-Specific Exemptions

# Create new resource-specific exemption
cat > e2e/drift-exemptions/my_resource.yaml <<EOF
version: 1
exemptions:
  - name: "my_exemption"
    description: "Why this is needed"
    allow_resource_creation: true
    enabled: true
settings:
  apply_exemptions: true
EOF

# Test
./scripts/run-e2e-tests.sh --apply-exemptions --resources my_resource

Settings Reference

Global Settings

Setting	Default	Description
`apply_exemptions`	`false`	Master toggle for exemptions
`verbose_exemptions`	`false`	Show which exemptions matched
`warn_unused_exemptions`	`false`	Warn about unused exemptions
`load_resource_exemptions`	`true`	Load resource-specific configs

For Debugging

settings:
  verbose_exemptions: true        # See what's being exempted
  warn_unused_exemptions: true    # Find stale exemptions

Best Practices

1. Be Specific

# ❌ Too broad
patterns:
  - ".*"

# ✅ Specific
patterns:
  - 'status.*->.*"active"'
resource_types:
  - "cloudflare_zone_dnssec"

2. Use Simplified Patterns

# ❌ Complex
patterns:
  - "will be created"
  - "\\+ resource"
  - "\\+ id"
  # ... many more

# ✅ Simple
allow_resource_creation: true

3. Document Exemptions

# ✅ Well documented
- name: "zone_setting_migration_drift"
  description: "v4 zone_settings_override splits into multiple v5 zone_setting resources during migration"
  allow_resource_creation: true

4. Use Resource-Specific Configs

Keep global config clean - put resource-specific exemptions in their own files.

5. Enable Warnings

settings:
  warn_unused_exemptions: true

Troubleshooting

Exemption Not Matching

Enable verbose mode:

settings:
  verbose_exemptions: true

Check regex:

echo "your-text" | grep -E "your-pattern"

Remove filters:

# Comment out to match all resources
# resource_types: ["cloudflare_zone"]

Too Many False Positives

Create resource-specific exemption:

cat > e2e/drift-exemptions/my_resource.yaml <<EOF
version: 1
exemptions:
  - name: "my_specific_exemption"
    description: "Special case for my_resource"
    patterns:
      - "specific-pattern"
    enabled: true
settings:
  apply_exemptions: true
EOF

Output Format

E2E test output shows which exemptions were applied:

Step 4: Verifying stable state (v5 plan after apply)
✓ No drift detected

Drift Exemptions Applied:
  Global exemptions:
    - computed_value_refreshes: 12 matches (from global-drift-exemptions.yaml)

  Resource-specific exemptions:
    - allow_zone_setting_creation: 8 matches (from e2e/drift-exemptions/zone_setting.yaml)
    - unsupported_plan_features: 2 matches (from e2e/drift-exemptions/zone_setting.yaml)

Development Guide

Setting Up Development Environment

# Clone repository
git clone <repository-url>
cd tf-migrate

# Install dependencies
go mod download

# Build binaries
make build-all

# Run tests
make test

Adding a New Resource Transformer

Step 1: Create Resource Directory

mkdir -p internal/resources/my_resource

Step 2: Implement Transformer

// internal/resources/my_resource/v4_to_v5.go
package my_resource

import (
    "github.com/cloudflare/tf-migrate/internal/registry"
    "github.com/cloudflare/tf-migrate/internal/transform"
)

type MyResourceTransformer struct{}

func init() {
    registry.Register(registry.ResourceEntry{
        Version:      "v4_to_v5",
        ResourceType: "cloudflare_my_resource",
        Transformer:  &MyResourceTransformer{},
    })
}

func (t *MyResourceTransformer) TransformConfig(ctx *transform.Context, block *hclwrite.Block) (*transform.TransformResult, error) {
    // Transform HCL configuration
    // Use tfhcl.RenameAttribute, tfhcl.RenameBlock, etc.

    return &transform.TransformResult{
        Blocks:         []*hclwrite.Block{block},
        RemoveOriginal: false,
    }, nil
}

Step 3: Add Tests

// internal/resources/my_resource/v4_to_v5_test.go
package my_resource

import (
    "testing"
    "github.com/stretchr/testify/assert"
    "github.com/cloudflare/tf-migrate/internal/transform"
)

func TestMyResourceTransformer_TransformConfig(t *testing.T) {
    input := `
resource "cloudflare_my_resource" "test" {
  name = "example"
}
`

    transformer := &MyResourceTransformer{}
    // Parse into HCL and call TransformConfig with the resource block
    // See existing resource test files for the standard test pattern
    _ = transformer
    _ = input
}

Step 4: Add Integration Test Data

mkdir -p integration/v4_to_v5/testdata/my_resource

Create test fixtures:

# integration/v4_to_v5/testdata/my_resource/main.tf
resource "cloudflare_my_resource" "test" {
  name = "example"
}

# integration/v4_to_v5/testdata/my_resource/expected.tf
resource "cloudflare_my_resource" "test" {
  name = "example"
}

Step 5: Add Documentation

# internal/resources/my_resource/README.md

# My Resource Migration Guide (v4 → v5)

## Changes

| Aspect | v4 | v5 | Change |
|--------|----|----|--------|
| Resource name | `cloudflare_my_resource` | `cloudflare_my_resource` | No change |
| `name` attribute | Required | Required | No change |

## Examples

...

Step 6: Run Tests

# Unit tests
go test ./internal/resources/my_resource -v

# Integration tests
TEST_RESOURCE=my_resource go test -v -run TestSingleResource

# E2E tests (if applicable)
./scripts/run-e2e-tests.sh --resources my_resource --apply-exemptions

Makefile Targets

make build           # Build tf-migrate binary
make build-all       # Build both tf-migrate and e2e-runner
make test            # Run all tests
make test-unit       # Run unit tests only
make test-integration # Run integration tests
make test-e2e        # Run e2e-runner unit tests
make lint            # Run linter
make lint-testdata   # Lint testdata naming conventions
make clean           # Clean build artifacts

Debugging Tips

Enable Debug Logging

./bin/tf-migrate migrate --log-level debug --dry-run

Inspect Intermediate Results

# Keep temp files in integration tests
KEEP_TEMP=true TEST_RESOURCE=my_resource go test -v -run TestSingleResource

# Check generated files
ls -la /tmp/tf-migrate-test-*

Debug E2E Tests

# Enable verbose drift exemptions
# Edit e2e/global-drift-exemptions.yaml:
settings:
  verbose_exemptions: true

# Run with debug output
./bin/e2e-runner run --resources my_resource --apply-exemptions

Common Patterns

Pattern 1: Simple Attribute Rename

func (t *MyTransformer) TransformConfig(ctx *transform.Context, block *hclwrite.Block) (*transform.TransformResult, error) {
    tfhcl.RenameAttribute(block.Body(), "old_name", "new_name")
    return &transform.TransformResult{Blocks: []*hclwrite.Block{block}, RemoveOriginal: false}, nil
}

Pattern 2: Resource Type Rename

func (t *MyTransformer) TransformConfig(ctx *transform.Context, block *hclwrite.Block) (*transform.TransformResult, error) {
    tfhcl.RenameResourceType(block, "cloudflare_old_name", "cloudflare_new_name")
    return &transform.TransformResult{Blocks: []*hclwrite.Block{block}, RemoveOriginal: false}, nil
}

Pattern 3: Nested Block to Attribute

// v4: nested block
// block {
//   nested {
//     value = "foo"
//   }
// }

// v5: attribute
// nested = { value = "foo" }

func (t *MyTransformer) TransformConfig(ctx *transform.Context, block *hclwrite.Block) (*transform.TransformResult, error) {
    // Use HCL traversal to restructure
    // See internal/resources/zero_trust_access_policy for examples
    return &transform.TransformResult{Blocks: []*hclwrite.Block{block}, RemoveOriginal: false}, nil
}

Pattern 4: One-to-Many Split

// See internal/resources/zone_setting/v4_to_v5.go for full example

func (t *ZoneSettingTransformer) TransformConfig(ctx *transform.Context, block *hclwrite.Block) (*transform.TransformResult, error) {
    // 1. Parse original resource block
    // 2. Extract settings from settings block
    // 3. Generate N new resource blocks (one per setting)
    // 4. Copy meta-arguments to all blocks
    // 5. Return {Blocks: newBlocks, RemoveOriginal: true}
}

Pattern 5: API-Based Migration

API calls are made inside TransformConfig to resolve import IDs or resource references:

func (t *TunnelRouteTransformer) TransformConfig(ctx *transform.Context, block *hclwrite.Block) (*transform.TransformResult, error) {
    // 1. Read resource attributes from HCL block
    network := tfhcl.ExtractStringAttribute(block.Body(), "network")

    // 2. Call Cloudflare API to fetch the UUID
    routes, err := ctx.CloudflareClient.ListTunnelRoutes(...)

    // 3. Generate import block with the resolved UUID
    for _, route := range routes {
        if route.Network == network {
            importBlock := tfhcl.CreateImportBlock(block, route.ID)
            return &transform.TransformResult{Blocks: []*hclwrite.Block{block, importBlock}}, nil
        }
    }
    return &transform.TransformResult{Blocks: []*hclwrite.Block{block}}, nil
}

Pattern 6: Conditional Transformation

func (t *MyTransformer) TransformConfig(ctx *transform.Context, block *hclwrite.Block) (*transform.TransformResult, error) {
    // Check if attribute exists before transforming
    if block.Body().GetAttribute("old_attr") != nil {
        tfhcl.RenameAttribute(block.Body(), "old_attr", "new_attr")
    }
    return &transform.TransformResult{Blocks: []*hclwrite.Block{block}, RemoveOriginal: false}, nil
}

Pattern 7: Manual Intervention with Warning Comments

When required fields cannot be automatically populated (e.g., values must come from external sources), add warning comments to guide users:

// See: internal/resources/byo_ip_prefix/v4_to_v5.go
// See: internal/resources/list_item/v4_to_v5.go

func (m *V4ToV5Migrator) TransformConfig(ctx *transform.Context, block *hclwrite.Block) (*transform.TransformResult, error) {
    body := block.Body()

    // Remove deprecated fields
    tfhcl.RemoveAttributes(body, "old_field_1", "old_field_2")

    // Add warning comment for fields requiring manual intervention
    warningMsg := "This resource requires manual intervention to add v5 required fields 'field_a' and 'field_b'. Find values in [source]. See migration documentation for details."
    tfhcl.AppendWarningComment(body, warningMsg)

    return &transform.TransformResult{
        Blocks:         []*hclwrite.Block{block},
        RemoveOriginal: false,
    }, nil
}

Result in HCL:

resource "cloudflare_example" "test" {
  account_id = "abc123"
  # MIGRATION WARNING: This resource requires manual intervention to add v5 required fields 'field_a' and 'field_b'. Find values in [source]. See migration documentation for details.
}

When to use:

New v5 required fields don't exist in v4
Values must come from external sources (API, dashboard, user)
No reasonable default value exists
Field values are account/environment-specific

Integration test strategy:

Expected output files include the warning comment
E2E tests simulate user adding fields from environment variables
Provider tests verify warning exists before manual intervention step

Additional Resources

Key Files to Understand

internal/pipeline/pipeline.go - Pipeline orchestration
internal/registry/registry.go - Resource transformer registration
internal/transform/interfaces.go - Core interfaces
internal/e2e-runner/runner.go - E2E test orchestration
internal/e2e-runner/drift.go - Drift detection & exemptions
internal/resources/zone_setting/v4_to_v5.go - Complex one-to-many example
internal/resources/zero_trust_tunnel_cloudflared_route/v4_to_v5.go - API-based migration example

External Documentation

Getting Help

Check resource-specific README.md files
Review integration test fixtures for examples
Use --dry-run to preview changes
Enable debug logging: --log-level debug
Review E2E test output for drift exemptions

Quick Command Reference

# Build
make build-all

# Test
go test ./...                              # All unit tests
make test-integration                      # Integration tests
./scripts/run-e2e-tests.sh --apply-exemptions  # E2E tests

# Run migration
./bin/tf-migrate migrate --source-version v4 --target-version v5
./bin/tf-migrate migrate --dry-run  # Preview only

# E2E runner
./bin/e2e-runner run --apply-exemptions
./bin/e2e-runner run --resources dns_record,zone_setting
./bin/e2e-runner init
./bin/e2e-runner migrate

# Debug
./bin/tf-migrate migrate --log-level debug --dry-run
KEEP_TEMP=true TEST_RESOURCE=dns_record go test -v -run TestSingleResource

Last Updated: 2026-03-20 Version: Based on v4→v5 migration implementation

FilesExpand file tree

agents.md

Latest commit

History