Skip to content

Support attaching multiple network interfaces to the same network card #2855

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Dec 19, 2024
Merged

Support attaching multiple network interfaces to the same network card #2855

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions ...arallelcluster-environment/files/amazon-2023/network_interfaces/configure_nw_interface.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,16 @@ set -ex

if
[ -z "${DEVICE_NAME}" ] || # name of the device
[ -z "${DEVICE_NUMBER}" ] || # number of the device
[ -z "${DEVICE_NUMBER}" ] || # index of the device
[ -z "${NETWORK_CARD_INDEX}" ] || # index of the network card
[ -z "${DEVICE_IP_ADDRESS}" ] || # ip of the device
[ -z "${MAC}" ] || # mac address of the device
[ -z "${CIDR_BLOCK}" ] # CIDR block of the subnet
then
echo 'One or more environment variables missing'
exit 1
fi
echo "Configuring NIC, Device name: ${DEVICE_NAME}, Device number: ${DEVICE_NUMBER}"
echo "Configuring NIC, Device name: ${DEVICE_NAME}, Device number: ${DEVICE_NUMBER}, Network card index:${NETWORK_CARD_INDEX}"

configuration_directory="/etc/systemd/network"
file_name="70-${DEVICE_NAME}.network"
Expand All @@ -23,12 +24,13 @@ fi

cd "$configuration_directory"

ROUTE_TABLE=100${DEVICE_NUMBER}
SUFFIX=$(printf "%03d" $NETWORK_CARD_INDEX)$(printf "%03d" $DEVICE_NUMBER)
Copy link
Contributor

@enrico-usai enrico-usai Jan 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would be good to add an example of the expected output in a comment

ROUTE_TABLE=1${SUFFIX}

ln -s /usr/lib/systemd/network/80-ec2.network ${file_name} # Use default EC2 configuration. This include MTU, etc.

/bin/cat <<EOF > ${sub_directory}/eni.conf
# Configuration for ${DEVICE_NUMBER} generated by ParallelCluster
# Configuration for network card: ${NETWORK_CARD_INDEX}, device number: ${DEVICE_NUMBER} generated by ParallelCluster
# This is inspired by https://github.com/amazonlinux/amazon-ec2-net-utils/blob/v2.4.1/lib/lib.sh
[Match]
MACAddress=${MAC}
Expand Down
9 changes: 6 additions & 3 deletions ...ws-parallelcluster-environment/files/default/network_interfaces/configure_nw_interface.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ set -e

if
[ -z "${DEVICE_NAME}" ] || # name of the device
[ -z "${DEVICE_NUMBER}" ] || # number of the device
[ -z "${DEVICE_NUMBER}" ] || # index of the device
[ -z "${NETWORK_CARD_INDEX}" ] || # index of the network card
[ -z "${GW_IP_ADDRESS}" ] || # gateway ip address
[ -z "${DEVICE_IP_ADDRESS}" ] || # ip address to assign to the interface
[ -z "${CIDR_PREFIX_LENGTH}" ] || # the prefix length of the device IP cidr block
Expand All @@ -20,9 +21,11 @@ then
exit 1
fi

ROUTE_TABLE="100${DEVICE_NUMBER}"
SUFFIX=$(printf "%03d" $NETWORK_CARD_INDEX)$(printf "%03d" $DEVICE_NUMBER)

echo "Configuring ${DEVICE_NAME} with IP:${DEVICE_IP_ADDRESS} CIDR_PREFIX:${CIDR_PREFIX_LENGTH} NETMASK:${NETMASK} GW:${GW_IP_ADDRESS} ROUTING_TABLE:${ROUTE_TABLE}"
ROUTE_TABLE="1${SUFFIX}"

echo "Configuring device name: ${DEVICE_NAME} with IP:${DEVICE_IP_ADDRESS} CIDR_PREFIX:${CIDR_PREFIX_LENGTH} NETMASK:${NETMASK} GW:${GW_IP_ADDRESS} ROUTING_TABLE:${ROUTE_TABLE}"
Copy link
Contributor

@enrico-usai enrico-usai Jan 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about adding network card index too? ${NETWORK_CARD_INDEX}


# config file
FILE="/etc/sysconfig/network-scripts/ifcfg-${DEVICE_NAME}"
Expand Down
10 changes: 6 additions & 4 deletions ...s-parallelcluster-environment/files/redhat-8.network_interfaces/configure_nw_interface.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ set -e

if
[ -z "${DEVICE_NAME}" ] || # name of the device
[ -z "${DEVICE_NUMBER}" ] || # number of the device
[ -z "${DEVICE_NUMBER}" ] || # index of the device
[ -z "${NETWORK_CARD_INDEX}" ] || # index of the network card
[ -z "${GW_IP_ADDRESS}" ] || # gateway ip address
[ -z "${DEVICE_IP_ADDRESS}" ] || # ip address to assign to the interface
[ -z "${CIDR_PREFIX_LENGTH}" ] # the prefix length of the device IP cidr block
Expand All @@ -23,9 +24,10 @@ then
fi

con_name="System ${DEVICE_NAME}"
route_table="100${DEVICE_NUMBER}"
priority="100${DEVICE_NUMBER}"
metric="100${DEVICE_NUMBER}"
SUFFIX=$(printf "%03d" $NETWORK_CARD_INDEX)$(printf "%03d" $DEVICE_NUMBER)
Copy link
Contributor

@enrico-usai enrico-usai Jan 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be consistent with existing code we should use lowercase variable names for internal variables, so we should have suffix rather than SUFFIX.

route_table="1${SUFFIX}"
priority="1${SUFFIX}"
metric="1${SUFFIX}"

# Rename connection
original_con_name=`nmcli -t -f GENERAL.CONNECTION device show ${DEVICE_NAME} | cut -f2 -d':'`
Expand Down
10 changes: 6 additions & 4 deletions .../aws-parallelcluster-environment/files/rocky/network_interfaces/configure_nw_interface.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ set -e

if
[ -z "${DEVICE_NAME}" ] || # name of the device
[ -z "${DEVICE_NUMBER}" ] || # number of the device
[ -z "${DEVICE_NUMBER}" ] || # index of the device
[ -z "${NETWORK_CARD_INDEX}" ] || # index of the network card
[ -z "${GW_IP_ADDRESS}" ] || # gateway ip address
[ -z "${DEVICE_IP_ADDRESS}" ] || # ip address to assign to the interface
[ -z "${CIDR_PREFIX_LENGTH}" ] # the prefix length of the device IP cidr block
Expand All @@ -23,9 +24,10 @@ then
fi

con_name="System ${DEVICE_NAME}"
route_table="100${DEVICE_NUMBER}"
priority="100${DEVICE_NUMBER}"
metric="100${DEVICE_NUMBER}"
SUFFIX=$(printf "%03d" $NETWORK_CARD_INDEX)$(printf "%03d" $DEVICE_NUMBER)
route_table="1${SUFFIX}"
priority="1${SUFFIX}"
metric="1${SUFFIX}"

# Rename connection
original_con_name=`nmcli -t -f GENERAL.CONNECTION device show ${DEVICE_NAME} | cut -f2 -d':'`
Expand Down
6 changes: 4 additions & 2 deletions ...aws-parallelcluster-environment/files/ubuntu/network_interfaces/configure_nw_interface.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ set -e

if
[ -z "${DEVICE_NAME}" ] || # name of the device
[ -z "${DEVICE_NUMBER}" ] || # number of the device
[ -z "${DEVICE_NUMBER}" ] || # index of the device
[ -z "${NETWORK_CARD_INDEX}" ] || # index of the network card
[ -z "${GW_IP_ADDRESS}" ] || # gateway ip address
[ -z "${DEVICE_IP_ADDRESS}" ] || # ip address to assign to the interface
[ -z "${CIDR_PREFIX_LENGTH}" ] || # the prefix length of the device IP cidr block
Expand Down Expand Up @@ -40,7 +41,8 @@ if [ "${STATIC_IP_CONFIG}" = "" ]
fi

FILE="/etc/netplan/${DEVICE_NAME}.yaml"
ROUTE_TABLE="100${DEVICE_NUMBER}"
SUFFIX=$(printf "%03d" $NETWORK_CARD_INDEX)$(printf "%03d" $DEVICE_NUMBER)
ROUTE_TABLE="1${SUFFIX}"
Copy link
Contributor

@enrico-usai enrico-usai Jan 23, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would have expected kitchen tests to fail in the https://github.com/aws/aws-parallelcluster-cookbook/blob/develop/cookbooks/aws-parallelcluster-environment/test/controls/network_interfaces_spec.rb after this changes.

Could you please run them and double check if we can improve existing coverage?


echo "Configuring ${DEVICE_NAME} with IP:${DEVICE_IP_ADDRESS} CIDR_PREFIX:${CIDR_PREFIX_LENGTH} NETMASK:${NETMASK} GW:${GW_IP_ADDRESS} ROUTING_TABLE:${ROUTE_TABLE}"

Expand Down
10 changes: 9 additions & 1 deletion cookbooks/aws-parallelcluster-environment/recipes/config/network_interfaces.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,13 @@
return if on_docker?

def network_card_index(mac, token)
# This IMDS call is not available on single NIC instance, therefore fallback to 0
Copy link
Contributor

@enrico-usai enrico-usai Jan 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to link in a comment the public documentation where this behavior is described.

uri = URI("http://169.254.169.254/latest/meta-data/network/interfaces/macs/#{mac}/network-card")
get_metadata_with_token(token, uri) || 0
end

def device_number(mac, token)
uri = URI("http://169.254.169.254/latest/meta-data/network/interfaces/macs/#{mac}/device-number")
get_metadata_with_token(token, uri)
end

Expand Down Expand Up @@ -69,6 +75,7 @@ def cidr_to_netmask(cidr)
# Configure nw interfaces
macs.each do |mac|
device_name = device_name(mac)
device_number = device_number(mac, token)
network_card_index = network_card_index(mac, token)
gw_ip_address = gateway_address
device_ip_address = device_ip(mac, token)
Expand All @@ -84,7 +91,8 @@ def cidr_to_netmask(cidr)
environment(
# TODO: The variables are a superset of what's required by individual scripts. Consider simplification.
'DEVICE_NAME' => device_name,
'DEVICE_NUMBER' => "#{network_card_index}", # in configure_nw_interface DEVICE_NUMBER actually means network card index
'DEVICE_NUMBER' => "#{device_number}",
'NETWORK_CARD_INDEX' => "#{network_card_index}",
'GW_IP_ADDRESS' => gw_ip_address,
'DEVICE_IP_ADDRESS' => device_ip_address,
'CIDR_PREFIX_LENGTH' => cidr_prefix_length,
Expand Down
Loading