Skip to content
This repository was archived by the owner on Dec 9, 2025. It is now read-only.

feat: add pcieroot as a device attribute#114

Merged
aojea merged 1 commit intogoogle:mainfrom
michaelasp:pcieRoot
Jun 18, 2025
Merged

feat: add pcieroot as a device attribute#114
aojea merged 1 commit intogoogle:mainfrom
michaelasp:pcieRoot

Conversation

@michaelasp
Copy link
Collaborator

kubernetes/enhancements#5316 provides a standardized device attribute for pcieroot. Use that definition. Fixes #111

@michaelasp
Copy link
Collaborator Author

/cc @aojea

@michaelasp michaelasp requested a review from aojea June 12, 2025 23:46
@michaelasp michaelasp changed the title feat: add pcieroot as a standard value feat: add pcieroot as a device attribute Jun 13, 2025
device.Attributes["dra.net/pciAddressFunction"] = resourceapi.DeviceAttribute{StringValue: &address.function}
}
if address.domain != "" && address.bus != "" {
pcieRoot := fmt.Sprintf("pci%s:%s", address.bus, address.device)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have you validate this is the correct format expected by all drivers?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm using the definition from the kep, although I realized I used the wrong values in the sprintf. Good catch!

 1. `resource.kubernetes.io/pcieRoot`: A string value in the format
//     `pci<domain>:<bus>`, referring to a PCIe (Peripheral Component
//     Interconnect Express) Root Complex. This attribute can be used to
//     identify devices that share the same PCIe Root Complex. DRA drivers MAY
//     determine this value by inspecting the hierarchical path of the device's
//     entry in sysfs (e.g., `/sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0`),
//     where the `pci<domain>:<bus>` segment at the beginning of the real path
//     identifies the Root Complex (e.g., `pci0000:00`).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on discussion in kubernetes/kubernetes#132296 this needs some update because the root pci device may have a different hub than the device itself.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated, PTAL again whenever you have a chance!

device.Attributes["dra.net/pciAddressFunction"] = resourceapi.DeviceAttribute{StringValue: &address.function}
}
if address.domain != "" && address.bus != "" {
pcieRoot := fmt.Sprintf("pci%s:%s", address.bus, address.device)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm using the definition from the kep, although I realized I used the wrong values in the sprintf. Good catch!

 1. `resource.kubernetes.io/pcieRoot`: A string value in the format
//     `pci<domain>:<bus>`, referring to a PCIe (Peripheral Component
//     Interconnect Express) Root Complex. This attribute can be used to
//     identify devices that share the same PCIe Root Complex. DRA drivers MAY
//     determine this value by inspecting the hierarchical path of the device's
//     entry in sysfs (e.g., `/sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0`),
//     where the `pci<domain>:<bus>` segment at the beginning of the real path
//     identifies the Root Complex (e.g., `pci0000:00`).

device.Attributes["dra.net/pciAddressFunction"] = resourceapi.DeviceAttribute{StringValue: &address.function}
}
if address.domain != "" && address.bus != "" {
pcieRoot := fmt.Sprintf("pci%s:%s", address.bus, address.device)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on discussion in kubernetes/kubernetes#132296 this needs some update because the root pci device may have a different hub than the device itself.

@michaelasp michaelasp force-pushed the pcieRoot branch 2 times, most recently from a1d386a to 60ee798 Compare June 16, 2025 16:20
@michaelasp michaelasp requested a review from aojea June 16, 2025 16:20
@michaelasp
Copy link
Collaborator Author

Tested locally, looks like it's working as expected:

      name: eth3
    - basic:
        attributes:
          ...
          dra.net/pciAddressBus:
            string: "08"
          dra.net/pciAddressDevice:
            string: "00"
          dra.net/pciAddressDomain:
            string: "0000"
          dra.net/pciAddressFunction:
            string: "0"
          dra.net/pciVendor:
            string: Google, Inc.
          dra.net/rdma:
            bool: false
          dra.net/sriov:
            bool: false
          dra.net/state:
            string: up
          dra.net/tcxProgramNames:
            string: ""
          dra.net/type:
            string: device
          dra.net/virtual:
            bool: false
          gce.dra.net/networkName:
            string: maspinwall-test-dra-net-net-4
          gce.dra.net/networkProjectNumber:
            int: 455207029971
          resource.kubernetes.io/pcieRoot:
            string: pci0000:08

kubernetes/enhancements#5316 provides a standardized device attribute for pcieroot. Use that definition.
@aojea
Copy link
Collaborator

aojea commented Jun 17, 2025

@gauravkghildiyal please review and once you LGTM I merge

@aojea
Copy link
Collaborator

aojea commented Jun 18, 2025

We know need to document how to use this with an example

@aojea aojea merged commit f00a8c1 into google:main Jun 18, 2025
6 checks passed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add standard attribute resource.kubernetes.io/pcieRoot

3 participants