libct/cg/OpenFile: check cgroupFd on error

opencontainers/runc issue 3026 describes a scenario in which OpenFile
failed to open a legitimate existing cgroupfs file. Added debug
(similar to what this commit does) shown that cgroupFd is no longer
opened to "/sys/fs/cgroup", but to "/" (it's not clear what caused it,
and the source code is not available, but they might be using the same
process on the both sides of the container/chroot/pivot_root/mntns
boundary, or remounting /sys/fs/cgroup).

Consider such use incorrect, but give a helpful hint as two what is
going on by wrapping the error in a more useful message.

NB: this can potentially be fixed by reopening the cgroupFd once we
detected that it's screwed, and retrying openat2. Alas I do not have
a test case for this, so left this as a TODO suggestion.

Signed-off-by: Kir Kolyshkin <[email protected]>

Author: kolyshkin

libcontainer/cgroups/file.go

@@ -6,6 +6,7 @@ import (
 	"fmt"
 	"os"
 	"path"
+	"strconv"
 	"strings"
 	"sync"
 
@@ -137,7 +138,23 @@ func openFile(dir, file string, flags int) (*os.File, error) {
 			Mode:    uint64(mode),
 		})
 	if err != nil {
-		return nil, &os.PathError{Op: "openat2", Path: path, Err: err}
+		err = &os.PathError{Op: "openat2", Path: path, Err: err}
+		// Check if cgroupFd is still opened to cgroupfsDir
+		// (happens when this package is incorrectly used
+		// across the chroot/pivot_root/mntns boundary, or
+		// when /sys/fs/cgroup is remounted).
+		//
+		// TODO: if such usage will ever be common, amend this
+		// to reopen cgroupFd and retry openat2.
+		fdStr := strconv.Itoa(cgroupFd)
+		fdDest, _ := os.Readlink("/proc/self/fd/" + fdStr)
+		if fdDest != cgroupfsDir {
+			// Wrap the error so it is clear that cgroupFd
+			// is opened to an unexpected/wrong directory.
+			err = fmt.Errorf("cgroupFd %s unexpectedly opened to %s != %s: %w",
+				fdStr, fdDest, cgroupfsDir, err)
+		}
+		return nil, err
 	}
 
 	return os.NewFile(uintptr(fd), path), nil