Skip to content

Commit 50bf471

Browse files
committed
specs-go/config: add Intel RDT/CAT Linux support
Add support for Intel Resource Director Technology (RDT) / Cache Allocation Technology (CAT). Add L3 cache resource constraints in Linux-specific configuration. This is the prerequisite of this runc proposal: opencontainers/runc#433 For more information about Intel RDT/CAT, please refer to: opencontainers/runc#433 Signed-off-by: Xiaochen Shen <[email protected]>
1 parent 7839cbb commit 50bf471

File tree

2 files changed

+98
-0
lines changed

2 files changed

+98
-0
lines changed

config-linux.md

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -474,6 +474,93 @@ The following parameters can be specified to setup the controller:
474474
}
475475
```
476476

477+
## Intel RDT
478+
479+
Intel platforms with new Xeon CPU support Intel Resource Director Technology
480+
(RDT). Cache Allocation Technology (CAT) is a sub-feature of RDT, which
481+
currently supports L3 cache resource allocation.
482+
483+
This feature provides a way for the software to restrict cache allocation to a
484+
defined 'subset' of L3 cache which may be overlapping with other 'subsets'.
485+
The different subsets are identified by class of service (CLOS) and each CLOS
486+
has a capacity bitmask (CBM).
487+
488+
In Linux kernel, it is exposed via "resource control" filesystem, which is a
489+
"cgroup-like" interface.
490+
491+
Comparing with cgroups, it has similar process management lifecycle and
492+
interfaces in a container. But unlike cgroups' hierarchy, it has single level
493+
filesystem layout.
494+
495+
Intel RDT "resource control" filesystem hierarchy:
496+
```
497+
mount -t resctrl resctrl /sys/fs/resctrl
498+
tree /sys/fs/resctrl
499+
/sys/fs/resctrl/
500+
|-- info
501+
| |-- L3
502+
| |-- cbm_mask
503+
| |-- num_closids
504+
|-- cpus
505+
|-- schemata
506+
|-- tasks
507+
|-- <container_id>
508+
|-- cpus
509+
|-- schemata
510+
|-- tasks
511+
512+
```
513+
514+
For containers, we can make use of `tasks` and `schemata` configuration for
515+
L3 cache resource constraints if hardware and kernel support Intel RDT/CAT.
516+
517+
The file `tasks` has a list of tasks that belongs to this group (e.g.,
518+
<container_id>" group). Tasks can be added to a group by writing the task ID
519+
to the "tasks" file (which will automatically remove them from the previous
520+
group to which they belonged). New tasks created by fork(2) and clone(2) are
521+
added to the same group as their parent. If a pid is not in any sub group, it
522+
is in root group.
523+
524+
The file `schemata` has allocation masks/values for L3 cache on each socket,
525+
which contains L3 cache id and capacity bitmask (CBM).
526+
```
527+
Format: "L3:<cache_id0>=<cbm0>;<cache_id1>=<cbm1>;..."
528+
```
529+
For example, on a two-socket machine, L3's schema line could be `L3:0=ff;1=c0`
530+
Which means L3 cache id 0's CBM is 0xff, and L3 cache id 1's CBM is 0xc0.
531+
532+
The valid L3 cache CBM is a *contiguous bits set* and number of bits that can
533+
be set is less than the max bit. The max bits in the CBM is varied among
534+
supported Intel Xeon platforms. In Intel RDT "resource control" filesystem
535+
layout, the CBM in a group should be a subset of the CBM in root. Kernel will
536+
check if it is valid when writing. e.g., 0xfffff in root indicates the max bits
537+
of CBM is 20 bits, which mapping to entire L3 cache capacity. Some valid CBM
538+
values to set in a group: 0xf, 0xf0, 0x3ff, 0x1f00 and etc.
539+
540+
**`intelRdt`** (object, OPTIONAL) represents the L3 cache resource constraints in Intel Xeon platforms.
541+
It is part of `resources` field of the Linux configuration.
542+
543+
For more information, see [Intel RDT/CAT kernel interface][intel-rdt-cat-kernel-interface].
544+
545+
The following parameters can be specified for the container:
546+
547+
* **`l3CacheSchema`** *(string, OPTIONAL)* - specifies the schema for L3 cache id and capacity bitmask (CBM)
548+
549+
###### Example
550+
```json
551+
There are two L3 caches in the two-socket machine, the default CBM is 0xfffff
552+
and the max CBM length is 20 bits. This configuration assigns 4/5 of L3 cache
553+
id 0 and the whole L3 cache id 1 for the container:
554+
555+
"linux": {
556+
"resources": {
557+
"intelRdt": {
558+
"l3CacheSchema": "L3:0=ffff0;1=fffff"
559+
}
560+
}
561+
}
562+
```
563+
477564
## Sysctl
478565

479566
**`sysctl`** (object, OPTIONAL) allows kernel parameters to be modified at runtime for the container.
@@ -618,3 +705,5 @@ The values MUST be absolute paths in the [container namespace][container-namespa
618705
[random.4]: http://man7.org/linux/man-pages/man4/random.4.html
619706
[tty.4]: http://man7.org/linux/man-pages/man4/tty.4.html
620707
[zero.4]: http://man7.org/linux/man-pages/man4/zero.4.html
708+
[intel-rdt-cat-kernel-interface]: https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git/commit/?h=x86/cache&id=f20e57892806ad244eaec7a7ae365e78fee53377
709+

specs-go/config.go

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -311,6 +311,13 @@ type LinuxNetwork struct {
311311
Priorities []LinuxInterfacePriority `json:"priorities,omitempty"`
312312
}
313313

314+
// LinuxIntelRdt for Linux Intel RDT/CAT resource management (Linux 4.10)
315+
type LinuxIntelRdt struct {
316+
// The schema for L3 cache id and capacity bitmask (CBM)
317+
// Format: "L3:<cache_id0>=<cbm0>;<cache_id1>=<cbm1>;..."
318+
L3CacheSchema *string `json:"l3CacheSchema,omitempty"`
319+
}
320+
314321
// LinuxResources has container runtime resource constraints
315322
type LinuxResources struct {
316323
// Devices configures the device whitelist.
@@ -331,6 +338,8 @@ type LinuxResources struct {
331338
HugepageLimits []LinuxHugepageLimit `json:"hugepageLimits,omitempty"`
332339
// Network restriction configuration
333340
Network *LinuxNetwork `json:"network,omitempty"`
341+
// IntelRdt restriction configuration
342+
IntelRdt *LinuxIntelRdt `json:"intelRdt,omitempty"`
334343
}
335344

336345
// LinuxDevice represents the mknod information for a Linux special device file

0 commit comments

Comments
 (0)