[ISSUE] intersight_fabric_vlan resource fails to create multiple vlans due to policy validation

[liftconfig]

Description

When adding multiple VLANs using intersight_fabric_vlan resource to a VLAN policy attached to a UCS domain profile, the following error is returned after the first few VLANs are created:

Error: error occurred while creating FabricVlan: 400 Bad Request Response from endpoint: {"code":"InvalidRequest","message":"The VLAN policy is attached to profiles in Validating state. Wait for validation to complete and retry. Use bulk API for editing multiple VLANs.","messageId":"gobi_fabric_vlan_create_error","traceId":"I6lpTAE11lv54PRU56wLIN_Sm1qlcHSsIm7J_tTeTW6GnrMs5nka9Q=="}

https://community.cisco.com/t5/unified-computing-system-ucs/intersight-bulk-policy-changes/td-p/5255213

The VLAN policy is validated after every VLAN is added causing the API to reject additional updates while the policy is in a validating state.

Terraform-provider-intersight version

TF provider version 1.0.55. Using SaaS version of Intersight

Workaround

Is there a resource or way to utilise the bulk intersight API (such as the intersight_bulk_request resource) to create multiple VLANs?

[the-akhil-nair]

Hi @liftconfig

Thank you for reaching out to us. Could you please share the Terraform configuration used to create the VLANs?

[liftconfig]

Hi @the-akhil-nair

FI profile, fabric eth network policy, and fabric vlan. Issue only occurs when the switch profile has been applied to live UCS FI

resource "intersight_fabric_switch_profile" "fi_a_profile" {
  name        = "fi-a-profile"
  switch_cluster_profile {
    object_type = "fabric.SwitchClusterProfile"
    moid        = intersight_fabric_switch_cluster_profile.cluster_switch_profile.moid
  }

  policy_bucket { # VLAN Configuration
    moid        = intersight_fabric_eth_network_policy.fabric_eth_network_policy.moid
    object_type = intersight_fabric_eth_network_policy.fabric_eth_network_policy.object_type
  }

  policy_bucket { # VSAN Configuration
    moid        = intersight_fabric_fc_network_policy.fabric_fc_network_policy_vsan_a.moid
    object_type = intersight_fabric_fc_network_policy.fabric_fc_network_policy_vsan_a.object_type
  }

  policy_bucket { # Port Policy
    moid        = intersight_fabric_port_policy.fi_a_port_policy_6454.moid
    object_type = intersight_fabric_port_policy.fi_a_port_policy_6454.object_type
  }

  policy_bucket { # System QoS Policy
    moid        = data.intersight_fabric_system_qos_policy.fabric_system_qos_policy.results[0].moid
    object_type = data.intersight_fabric_system_qos_policy.fabric_system_qos_policy.results[0].object_type
  }

  policy_bucket { # Switch Control Policy
    moid        = data.intersight_fabric_switch_control_policy.fabric_switch_control_policy.results[0].moid
    object_type = data.intersight_fabric_switch_control_policy.fabric_switch_control_policy.results[0].object_type
  }

  policy_bucket { # NTP Policy
    moid        = data.intersight_ntp_policy.ntp_aest.results[0].moid
    object_type = data.intersight_ntp_policy.ntp_aest.results[0].object_type
  }

  policy_bucket { # Network Connectivity
    moid        = intersight_networkconfig_policy.fi_attached_network_connectivity.moid
    object_type = intersight_networkconfig_policy.fi_attached_network_connectivity.object_type
  }

  policy_bucket { # Syslog
    object_type = intersight_syslog_policy.fi_attached_syslog_policy.object_type
    moid        = intersight_syslog_policy.fi_attached_syslog_policy.moid
  }
}

resource "intersight_fabric_eth_network_policy" "fabric_eth_network_policy" {
  name = "Fabric-Eth-Network-Policy
  organization {
    moid = data.intersight_organization_organization.results[0].moid
  }
}

resource "intersight_fabric_vlan" "vlans_group" {
  for_each = var.vlans

  name                  = each.key
  auto_allow_on_uplinks = false
  is_native             = false
  vlan_id               = each.value.vlan

  eth_network_policy {
    moid = intersight_fabric_eth_network_policy.fabric_eth_network_policy.moid
  }

  multicast_policy {
    moid = data.intersight_fabric_multicast_policy.fabric_multicast_policy.results[0].moid
  }
}

variable "vlans" {
  type = map(object({
    vlan = number
    qos  = string
  }))
  default = {
    v101-test1 = { vlan = 101, qos = "gold" }
    v102-test2 = { vlan = 102, qos = "gold" }
    v103-test3 = { vlan = 103, qos = "gold" }
    v104-test4 = { vlan = 104, qos = "gold" }
    v105-test5 = { vlan = 105, qos = "gold" }
  }
}

[jingiris05]

Hi @liftconfig,

This behavior occurs because VLANs were being created individually while the associated domain or switch profile was still in a validating/configuring state.
During this time, the Intersight backend restricts VLAN policy changes to prevent inconsistent configurations, which led to the observed failure.

The issue can be resolved by using the correct resource creation order using the depends_on argument.
By doing this, Terraform ensures that all VLANs are created first before the switch (or domain) profile is applied.

Below is the configuration that you could give a try, Please let us know if you encounter any difficulties:

Config:

# Create VLANs first, before attaching to profiles
resource "intersight_fabric_vlan" "vlans_group" {
  for_each = var.vlans

  name                  = each.key
  auto_allow_on_uplinks = false
  is_native             = false
  vlan_id               = each.value.vlan

  eth_network_policy {
    moid = data.intersight_fabric_eth_network_policy.fabric_eth_network_policy_vlan.moid
  }

  multicast_policy {
    moid = data.intersight_fabric_multicast_policy.fabric_multicast_policy.moid
  }
}

resource "intersight_fabric_switch_cluster_profile" "cluster_switch_profile_1" {
  name = "cluster_switch_profile_1"

  organization {
    moid = data.intersight_organization_organization.organization.results[0].moid
  }

}

# Create switch profile after cluster profile and VLANs
resource "intersight_fabric_switch_profile" "fi_a_profile" {
  name        = "fi-a-profile"
  
  switch_cluster_profile {
    object_type = "fabric.SwitchClusterProfile"
    moid        = intersight_fabric_switch_cluster_profile.cluster_switch_profile_1.moid
  }

  policy_bucket { # VLAN Configuration
    moid        = data.intersight_fabric_eth_network_policy.fabric_eth_network_policy_vlan.moid
    object_type = "fabric.EthNetworkPolicy"
  }

  policy_bucket { # VSAN Configuration
    moid        = data.intersight_fabric_fc_network_policy.fabric_fc_network_policy_vsan_a.moid
    object_type = "fabric.FcNetworkPolicy"
  }

  policy_bucket { # Port Policy
    moid        = data.intersight_fabric_port_policy.fi_a_port_policy_6454.moid
    object_type = "fabric.PortPolicy"
  }

  policy_bucket { # System QoS Policy
    moid        = data.intersight_fabric_system_qos_policy.fabric_system_qos_policy.moid
    object_type = "fabric.SystemQosPolicy"
  }

  policy_bucket { # Switch Control Policy
    moid        = data.intersight_fabric_switch_control_policy.fabric_switch_control_policy.moid
    object_type = "fabric.SwitchControlPolicy"
  }

  policy_bucket { # NTP Policy
    moid        = data.intersight_ntp_policy.ntp_1.moid
    object_type = "ntp.Policy"
  }

  policy_bucket { # Network Connectivity
    moid        = data.intersight_networkconfig_policy.fi_attached_network_connectivity.moid
    object_type = "networkconfig.Policy"
  }

  policy_bucket { # Syslog
    moid        = data.intersight_syslog_policy.fi_attached_syslog_policy.moid
    object_type = "syslog.Policy"
  }

  depends_on = [
    intersight_fabric_vlan.vlans_group,
    intersight_fabric_switch_cluster_profile.cluster_switch_profile_1
  ]
}

variable "vlans" {
  type = map(object({
    vlan = number
    qos  = string
  }))
  default = {
    v101-test1 = { vlan = 101, qos = "gold" }
    v102-test2 = { vlan = 102, qos = "gold" }
    v103-test3 = { vlan = 103, qos = "gold" }
    v104-test4 = { vlan = 104, qos = "gold" }
    v105-test5 = { vlan = 105, qos = "gold" }
  }
}