bpf: Add support for KF_RET_RCU flag

Add a kfunc annotation 'KF_RET_RCU' to signal that the return type must
be marked MEM_RCU, to return objects that are RCU protected. Naturally,
this must imply that the kfunc is invoked in an RCU critical section,
and thus the presence of this flag implies the presence of the
KF_RCU_PROTECTED flag. Upcoming kfunc scx_bpf_cpu_curr() [0] will be
made to make use of this flag.

  [0]: https://lore.kernel.org/all/[email protected]

Signed-off-by: Kumar Kartikeya Dwivedi <[email protected]>

Author: kkdwvd

Documentation/bpf/kfuncs.rst

@@ -346,10 +346,19 @@ arguments are at least RCU protected pointers. This may transitively imply that
 RCU protection is ensured, but it does not work in cases of kfuncs which require
 RCU protection but do not take RCU protected arguments.
 
+2.4.9 KF_RET_RCU flag
+---------------------
+
+The KF_RET_RCU flag is used for kfuncs which return pointers to RCU protected
+objects. Since this only works when the invocation of the kfunc is made in an
+active RCU critical section, the usage of this flag implies ``KF_RCU_PROTECTED``
+flag automatically. This flag may be combined with other return value modifiers,
+such as ``KF_RET_NULL``.
+
 .. _KF_deprecated_flag:
 
-2.4.9 KF_DEPRECATED flag
-------------------------
+2.4.10 KF_DEPRECATED flag
+-------------------------
 
 The KF_DEPRECATED flag is used for kfuncs which are scheduled to be
 changed or removed in a subsequent kernel release. A kfunc that is

include/linux/btf.h

@@ -79,6 +79,7 @@
 #define KF_ARENA_RET    (1 << 13) /* kfunc returns an arena pointer */
 #define KF_ARENA_ARG1   (1 << 14) /* kfunc takes an arena pointer as its first argument */
 #define KF_ARENA_ARG2   (1 << 15) /* kfunc takes an arena pointer as its second argument */
+#define KF_RET_RCU      ((1 << 16) | KF_RCU_PROTECTED) /* kfunc returns an RCU protected pointer, implies KF_RCU_PROTECTED */
 
 /*
  * Tag marking a kernel function as a kfunc. This is meant to minimize the

kernel/bpf/verifier.c

@@ -12342,6 +12342,11 @@ static bool is_kfunc_ret_null(struct bpf_kfunc_call_arg_meta *meta)
 	return meta->kfunc_flags & KF_RET_NULL;
 }
 
+static bool is_kfunc_ret_rcu(struct bpf_kfunc_call_arg_meta *meta)
+{
+	return meta->kfunc_flags & KF_RET_RCU;
+}
+
 static bool is_kfunc_bpf_rcu_read_lock(struct bpf_kfunc_call_arg_meta *meta)
 {
 	return meta->func_id == special_kfunc_list[KF_bpf_rcu_read_lock];
@@ -14042,6 +14047,8 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
 
 			if (meta.func_id == special_kfunc_list[KF_bpf_get_kmem_cache])
 				regs[BPF_REG_0].type |= PTR_UNTRUSTED;
+			else if (is_kfunc_ret_rcu(&meta))
+				regs[BPF_REG_0].type |= MEM_RCU;
 
 			if (is_iter_next_kfunc(&meta)) {
 				struct bpf_reg_state *cur_iter;