Skip to content

Commit b11ade4

Browse files
committed
config-linux: RFC 2119 wording for intelRdt
So we can compliance-test runtimes for these settings. Also remove the tutorial, since the kernel docs should provide sufficient documentation on that front. The kernel can be patched if they do not, and we do not include tutorials for other config-linux settings in this spec. The updated example was recommended by Xiaochen to compensate for the removed inline tutorial [1]. [1]: #787 (comment) Signed-off-by: W. Trevor King <[email protected]>
1 parent 101e6f9 commit b11ade4

File tree

1 file changed

+14
-71
lines changed

1 file changed

+14
-71
lines changed

config-linux.md

Lines changed: 14 additions & 71 deletions
Original file line numberDiff line numberDiff line change
@@ -478,86 +478,29 @@ The following parameters can be specified to setup the controller:
478478

479479
## <a name="configLinuxIntelRdt" />IntelRdt
480480

481-
Intel platforms with new Xeon CPU support Intel Resource Director Technology
482-
(RDT). Cache Allocation Technology (CAT) is a sub-feature of RDT, which
483-
currently supports L3 cache resource allocation.
481+
**`intelRdt`** (object, OPTIONAL) represents the [Intel Resource Director Technology][intel-rdt-cat-kernel-interface].
482+
If `intelRdt` is set, the runtime MUST write the container process ID to the `<container-id>/tasks` file in a mounted `resctrl` pseudo-filesystem, using the container ID from [`start`](runtime.md#start) and creating the `<container-id>` directory if necessary.
483+
If no mounted `resctrl` pseudo-filesystem is available in the [runtime mount namespace](glossary.md#runtime-namespace), the runtime MUST [generate an error](runtime.md#errors).
484484

485-
This feature provides a way for the software to restrict cache allocation to a
486-
defined 'subset' of L3 cache which may be overlapping with other 'subsets'.
487-
The different subsets are identified by class of service (CLOS) and each CLOS
488-
has a capacity bitmask (CBM).
485+
If `intelRdt` is not set, the runtime MUST NOT manipulate any `resctrl` psuedo-filesystems.
489486

490-
In Linux kernel, it is exposed via "resource control" filesystem, which is a
491-
"cgroup-like" interface.
492-
493-
Comparing with cgroups, it has similar process management lifecycle and
494-
interfaces in a container. But unlike cgroups' hierarchy, it has single level
495-
filesystem layout.
496-
497-
Intel RDT "resource control" filesystem hierarchy:
498-
```
499-
mount -t resctrl resctrl /sys/fs/resctrl
500-
tree /sys/fs/resctrl
501-
/sys/fs/resctrl/
502-
|-- info
503-
| |-- L3
504-
| |-- cbm_mask
505-
| |-- min_cbm_bits
506-
| |-- num_closids
507-
|-- cpus
508-
|-- schemata
509-
|-- tasks
510-
|-- <container_id>
511-
|-- cpus
512-
|-- schemata
513-
|-- tasks
514-
515-
```
516-
517-
For containers, we can make use of `tasks` and `schemata` configuration for
518-
L3 cache resource constraints if hardware and kernel support Intel RDT/CAT.
519-
520-
The file `tasks` has a list of tasks that belongs to this group (e.g.,
521-
<container_id>" group). Tasks can be added to a group by writing the task ID
522-
to the "tasks" file (which will automatically remove them from the previous
523-
group to which they belonged). New tasks created by fork(2) and clone(2) are
524-
added to the same group as their parent. If a pid is not in any sub group, it
525-
is in root group.
526-
527-
The file `schemata` has allocation masks/values for L3 cache on each socket,
528-
which contains L3 cache id and capacity bitmask (CBM).
529-
```
530-
Format: "L3:<cache_id0>=<cbm0>;<cache_id1>=<cbm1>;..."
531-
```
532-
For example, on a two-socket machine, L3's schema line could be `L3:0=ff;1=c0`
533-
Which means L3 cache id 0's CBM is 0xff, and L3 cache id 1's CBM is 0xc0.
534-
535-
The valid L3 cache CBM is a *contiguous bits set* and number of bits that can
536-
be set is less than the max bit. The max bits in the CBM is varied among
537-
supported Intel Xeon platforms. In Intel RDT "resource control" filesystem
538-
layout, the CBM in a group should be a subset of the CBM in root. Kernel will
539-
check if it is valid when writing. e.g., 0xfffff in root indicates the max bits
540-
of CBM is 20 bits, which mapping to entire L3 cache capacity. Some valid CBM
541-
values to set in a group: 0xf, 0xf0, 0x3ff, 0x1f00 and etc.
487+
The following parameters can be specified for the container:
542488

543-
**`intelRdt`** (object, OPTIONAL) represents the L3 cache resource constraints in Intel Xeon platforms.
489+
* **`l3CacheSchema`** *(string, OPTIONAL)* - specifies the schema for L3 cache id and capacity bitmask (CBM).
490+
If `l3CacheSchema` is set, runtimes MUST write the value to the `schemata` file in the `<container-id>` directory discussed in `intelRdt`.
544491

545-
For more information, see [Intel RDT/CAT kernel interface][intel-rdt-cat-kernel-interface].
492+
If `l3CacheSchema` is not set, runtimes MUST NOT write to `schemata` files in any `resctrl` psuedo-filesystems.
546493

547-
The following parameters can be specified for the container:
494+
### Example
548495

549-
* **`l3CacheSchema`** *(string, OPTIONAL)* - specifies the schema for L3 cache id and capacity bitmask (CBM)
496+
Consider a two-socket machine with two L3 caches where the default CBM is 0xfffff and the max CBM length is 20 bits.
497+
Tasks inside the container only have access to the "upper" 80% of L3 cache id 0 and the "lower" 50% L3 cache id 1:
550498

551-
###### Example
552499
```json
553-
There are two L3 caches in the two-socket machine, the default CBM is 0xfffff
554-
and the max CBM length is 20 bits. This configuration assigns 4/5 of L3 cache
555-
id 0 and the whole L3 cache id 1 for the container:
556-
557500
"linux": {
558-
"intelRdt": {
559-
"l3CacheSchema": "L3:0=ffff0;1=fffff"
560-
}
501+
"intelRdt": {
502+
"l3CacheSchema": "L3:0=ffff0;1=3ff"
503+
}
561504
}
562505
```
563506

0 commit comments

Comments
 (0)