Skip to content

Commit c3772a6

Browse files
committed
specs-go/config: add Intel RDT/CAT Linux support
Add support for Intel Resource Director Technology (RDT) / Cache Allocation Technology (CAT). Add L3 cache resource constraints in Linux-specific configuration. This is the prerequisite of this runc proposal: opencontainers/runc#433 For more information about Intel RDT/CAT, please refer to: opencontainers/runc#433 Signed-off-by: Xiaochen Shen <[email protected]>
1 parent 5398f4e commit c3772a6

File tree

2 files changed

+98
-0
lines changed

2 files changed

+98
-0
lines changed

config-linux.md

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -476,6 +476,94 @@ The following parameters can be specified to setup the controller:
476476
}
477477
```
478478

479+
## Intel RDT
480+
481+
Intel platforms with new Xeon CPU support Intel Resource Director Technology
482+
(RDT). Cache Allocation Technology (CAT) is a sub-feature of RDT, which
483+
currently supports L3 cache resource allocation.
484+
485+
This feature provides a way for the software to restrict cache allocation to a
486+
defined 'subset' of L3 cache which may be overlapping with other 'subsets'.
487+
The different subsets are identified by class of service (CLOS) and each CLOS
488+
has a capacity bitmask (CBM).
489+
490+
In Linux kernel, it is exposed via "resource control" filesystem, which is a
491+
"cgroup-like" interface.
492+
493+
Comparing with cgroups, it has similar process management lifecycle and
494+
interfaces in a container. But unlike cgroups' hierarchy, it has single level
495+
filesystem layout.
496+
497+
Intel RDT "resource control" filesystem hierarchy:
498+
```
499+
mount -t resctrl resctrl /sys/fs/resctrl
500+
tree /sys/fs/resctrl
501+
/sys/fs/resctrl/
502+
|-- info
503+
| |-- L3
504+
| |-- cbm_mask
505+
| |-- min_cbm_bits
506+
| |-- num_closids
507+
|-- cpus
508+
|-- schemata
509+
|-- tasks
510+
|-- <container_id>
511+
|-- cpus
512+
|-- schemata
513+
|-- tasks
514+
515+
```
516+
517+
For containers, we can make use of `tasks` and `schemata` configuration for
518+
L3 cache resource constraints if hardware and kernel support Intel RDT/CAT.
519+
520+
The file `tasks` has a list of tasks that belongs to this group (e.g.,
521+
<container_id>" group). Tasks can be added to a group by writing the task ID
522+
to the "tasks" file (which will automatically remove them from the previous
523+
group to which they belonged). New tasks created by fork(2) and clone(2) are
524+
added to the same group as their parent. If a pid is not in any sub group, it
525+
is in root group.
526+
527+
The file `schemata` has allocation masks/values for L3 cache on each socket,
528+
which contains L3 cache id and capacity bitmask (CBM).
529+
```
530+
Format: "L3:<cache_id0>=<cbm0>;<cache_id1>=<cbm1>;..."
531+
```
532+
For example, on a two-socket machine, L3's schema line could be `L3:0=ff;1=c0`
533+
Which means L3 cache id 0's CBM is 0xff, and L3 cache id 1's CBM is 0xc0.
534+
535+
The valid L3 cache CBM is a *contiguous bits set* and number of bits that can
536+
be set is less than the max bit. The max bits in the CBM is varied among
537+
supported Intel Xeon platforms. In Intel RDT "resource control" filesystem
538+
layout, the CBM in a group should be a subset of the CBM in root. Kernel will
539+
check if it is valid when writing. e.g., 0xfffff in root indicates the max bits
540+
of CBM is 20 bits, which mapping to entire L3 cache capacity. Some valid CBM
541+
values to set in a group: 0xf, 0xf0, 0x3ff, 0x1f00 and etc.
542+
543+
**`intelRdt`** (object, OPTIONAL) represents the L3 cache resource constraints in Intel Xeon platforms.
544+
It is part of `resources` field of the Linux configuration.
545+
546+
For more information, see [Intel RDT/CAT kernel interface][intel-rdt-cat-kernel-interface].
547+
548+
The following parameters can be specified for the container:
549+
550+
* **`l3CacheSchema`** *(string, OPTIONAL)* - specifies the schema for L3 cache id and capacity bitmask (CBM)
551+
552+
###### Example
553+
```json
554+
There are two L3 caches in the two-socket machine, the default CBM is 0xfffff
555+
and the max CBM length is 20 bits. This configuration assigns 4/5 of L3 cache
556+
id 0 and the whole L3 cache id 1 for the container:
557+
558+
"linux": {
559+
"resources": {
560+
"intelRdt": {
561+
"l3CacheSchema": "L3:0=ffff0;1=fffff"
562+
}
563+
}
564+
}
565+
```
566+
479567
## Sysctl
480568

481569
**`sysctl`** (object, OPTIONAL) allows kernel parameters to be modified at runtime for the container.
@@ -621,3 +709,4 @@ The values MUST be absolute paths in the [container namespace][container-namespa
621709
[random.4]: http://man7.org/linux/man-pages/man4/random.4.html
622710
[tty.4]: http://man7.org/linux/man-pages/man4/tty.4.html
623711
[zero.4]: http://man7.org/linux/man-pages/man4/zero.4.html
712+
[intel-rdt-cat-kernel-interface]: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/x86/intel_rdt_ui.txt

specs-go/config.go

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -311,6 +311,13 @@ type LinuxNetwork struct {
311311
Priorities []LinuxInterfacePriority `json:"priorities,omitempty"`
312312
}
313313

314+
// LinuxIntelRdt for Linux Intel RDT/CAT resource management (Linux 4.10)
315+
type LinuxIntelRdt struct {
316+
// The schema for L3 cache id and capacity bitmask (CBM)
317+
// Format: "L3:<cache_id0>=<cbm0>;<cache_id1>=<cbm1>;..."
318+
L3CacheSchema *string `json:"l3CacheSchema,omitempty"`
319+
}
320+
314321
// LinuxResources has container runtime resource constraints
315322
type LinuxResources struct {
316323
// Devices configures the device whitelist.
@@ -331,6 +338,8 @@ type LinuxResources struct {
331338
HugepageLimits []LinuxHugepageLimit `json:"hugepageLimits,omitempty"`
332339
// Network restriction configuration
333340
Network *LinuxNetwork `json:"network,omitempty"`
341+
// IntelRdt restriction configuration
342+
IntelRdt *LinuxIntelRdt `json:"intelRdt,omitempty"`
334343
}
335344

336345
// LinuxDevice represents the mknod information for a Linux special device file

0 commit comments

Comments
 (0)