Updated for multiple nics (#428)

Co-authored-by: Doug Clayton <[email protected]>

Author: dougclayton

articles/cyclecloud/cluster-references/network-interface-reference.md

@@ -11,40 +11,62 @@ ms.custom: compute-evergreen
 
 Network Interface objects are rank 3, and subordinate to `node` or `nodearray`. `network-interface` represents an Azure Network Interface.
 
-Multiple network interfaces may be attached to a single node for multi-homed VMs.   Refer to the [VM Sizes](/azure/virtual-machines/linux/sizes) documentation to find the maxinum number of NICs for the selected VM SKU.
+Multiple network interfaces may be attached to a single node for multi-homed VMs.   Refer to the [VM Sizes](/azure/virtual-machines/linux/sizes) documentation to find the maximum number of NICs for the selected VM SKU.
 
 ### Example
 
-Adding a `[[[network-interface]]]` section to a node adds controls to the default NIC. You can also add additional NICs and attach them to a VM.
+Nodes will get a single network interface by default. Adding a `[[[network-interface]]]` section to a node lets the defaults be overridden. You can also add additional NICs and attach them to a VM.
 
-This example attaches an existing NIC to a VM:
+This example creates a node with two network interfaces, and places the second nic in a different subnet with two application security groups:
 
 ``` ini
 [cluster my-cluster]
   [[node my-node]]
     Credentials = $Credentials
-    SubnetId = $SubnetId
+    SubnetId = my-rg/my-vnet/subnet2
     MachineType = $MachineType
     ImageName = $ImageName
 
-    [[[network-interface my-nic]]]
+    [[[network-interface nic1]]]
       NetworkInterfaceId = /subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/resourceGroups/my-rg/providers/Microsoft.Network/networkInterfaces/my-nic
+
+    [[[network-interface nic2]]]
+      SubnetId = my-rg/my-vnet2/subnet
+      ApplicationSecurityGroups = /subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/resourceGroups/my-rg/providers/Microsoft.Network/applicationSecurityGroups/asg1, /subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/resourceGroups/my-rg/providers/Microsoft.Network/applicationSecurityGroups/asg2
 ```
 
 Attribute values that begin with `$` are referencing parameters.
 
 ## Attribute Reference
 
+The `[[[network-interface]]]` block is most commonly used for single nodes and the attributes that reference singular properties (such as a private IP address) do not apply to node arrays. However, the block may be used to apply an existing Network Security Group or one or more Application Security Groups to nodes in an array.
+
 Attribute | Type | Definition
------- | ----- | ----------
-AssociatePublicIpAddress | Boolean | Associate a public IP address with the NIC (node only)
-StaticPublicIpAddress | Boolean | If true, the IP address will be persisted between node restarts (node only)
-EnableIpForwarding | Boolean | If true, allow IP forwarding (node only)
+--------- | ---- | ----------
+AssociatePublicIpAddress | Boolean | Associate a public IP address with the NIC
+EnableIpForwarding | Boolean | If true, allow IP forwarding
+SecurityGroup | String | Specify an existing Network Security Group Resource ID (overrides the default NSG created when a public IP is specified). This overrides the `NetworkSecurityGroupId` [node attribute](node-nodearray-reference.md), if any.
+ApplicationSecurityGroups | String (list) | List of Application Security Groups by Resource ID
+SubnetId | String | Subnet definition in the form `${rg}/${vnet}/${subnet}`. This overrides the `SubnetId` [node attribute](node-nodearray-reference.md).
+Primary | Boolean | If set, marks this NIC as "primary" for the operating system.
 PrivateIp | String | Assign a specific private IP address (node only)
 NetworkInterfaceId | String | Specify an existing NIC by resource ID (node only)
-SecurityGroup | String | Specify an existing Network Security Group Resouce ID (overrides the default NSG created when a public IP is specified)
-ApplicationSecurityGroups | String (list) | List of Application Security Groups by Resource ID and separated by comma
+StaticPublicIpAddress | Boolean | If true, the IP address will be persisted between node restarts (node only)
+
+### Existing network interfaces
+
+For head nodes, you can create a NIC separately and attach it to a node:
 
-### Nodearray Network Interface Attributes
+``` ini
+[cluster my-cluster]
+  [[node my-node]]
+    Credentials = $Credentials
+    SubnetId = $SubnetId
+    MachineType = $MachineType
+    ImageName = $ImageName
+
+    [[[network-interface my-nic]]]
+      NetworkInterfaceId = /subscriptions/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/resourceGroups/my-rg/providers/Microsoft.Network/networkInterfaces/my-nic
+```
 
-The `[[[network-interface]]]` block is most commonly used for single nodes/VMs and most of the attributes do not apply to nodearrays.   However, the block may be used to apply an existing Network Security Group or one or more Application Security Groups to all nodes/VMs in the nodearray.
+This network interface will not be modified or deleted by CycleCloud.

articles/cyclecloud/cluster-references/node-nodearray-reference.md

@@ -61,7 +61,7 @@ There are a minimum of four required attributes to successfully start a node:
 Attribute | Type | Definition
 ------ | ----- | ----------
 MachineType | String | The Azure VM Size
-SubnetId | String | Subnet definition in the form `${rg}/${vnet}/${subnet}`
+SubnetId | String | Subnet definition in the form `${rg}/${vnet}/${subnet}`. Note this is *not* the full resource id.
 Credentials | String | Name of the Cloud Provider account.
 
 The fourth required attribute is related to an image. An image attribute is required
@@ -94,6 +94,7 @@ KeepAlive | Boolean | If true, CycleCloud will prevent the termination of this n
 Locker | String | Specify the name of the locker from which to download project specs. See [Use Projects](~/how-to/projects.md)
 BootDiagnosticsUri | String | Storage URI for boot diagnostics (example: https://mystorageaccount.blob.core.windows.net/), if specified. Storage charges will apply.
 HybridBenefit | Boolean | If true, enables "Azure Hybrid Benefit" licensing for Windows VMs
+NetworkSecurityGroupId | String | If given, the full resource id for a Network Security Group to use for this node. Note: this can also be specified as `SecurityGroup` on a [network interface](network-interface-reference.md).
 EnableTerminateNotification (8.2.0+) | Boolean | If true, enables [Terminate Notification](/azure/virtual-machine-scale-sets/virtual-machine-scale-sets-terminate-notification) to send events on VM deletion to the VM for local handling. This only applies to scaleset VMs.
 TerminateNotificationTimeout (8.2.2+) | Relative Time | If terminate-notification is enabled, this controls how long VMs are given to handle the event before being deleted.
 ThrottleCapacity (8.2.2+) | Boolean | If true, this nodearray will report 0 capacity to autoscalers for a default of 5 minutes after encountering a capacity issue