Does not work on kernel 4.15

[leitao]

Unfortunately this code does not work on kernel 4.15, with the following error:

      member 'sp1' of class or structure 'x86_hw_tss' may result in an unaligned pointer value
      [-Werror,-Waddress-of-packed-member]

This is the full log:

gcc -O2 -Wall -I ./   -c -o bpf/bpf.o bpf/bpf.c
gcc -O2 -Wall -I ./   -c -o bpf_load.o bpf_load.c
bpf_load.c: In function ‘parse_relo_and_apply’:
bpf_load.c:272:7: warning: unused variable ‘j’ [-Wunused-variable]
   int j, map_idx;
       ^
bpf_load.c: In function ‘load_elf_maps_section’:
bpf_load.c:328:6: warning: unused variable ‘copy_sz’ [-Wunused-variable]
  int copy_sz;
      ^~~~~~~
gcc -O2 -Wall -I ./ bpf/bpf.o bpf_load.o -lelf -o memcpy_kprobe memcpy_kprobe_user.c
gcc -O2 -Wall -I ./ bpf/bpf.o bpf_load.o -lelf -o memcpy_stat memcpy_stat_user.c
#it will generate .ll file which is actually a LLVM assembly code
clang -S -nostdinc -isystem /usr/lib/gcc/x86_64-linux-gnu/7/include -I/lib/modules/4.15.0-36-generic/build//arch/x86/include/generated/uapi -I/lib/modules/4.15.0-36-generic/build//arch/x86/include/generated -I/lib/modules/4.15.0-36-generic/build//arch/x86/include -I/lib/modules/4.15.0-36-generic/build//arch/x86/include/uapi -I/lib/modules/4.15.0-36-generic/build//include -I/lib/modules/4.15.0-36-generic/build//include/uapi -include /lib/modules/4.15.0-36-generic/build//include/linux/kconfig.h -I/lib/modules/4.15.0-36-generic/build//include/generated/uapi -Werror \
    -D__KERNEL__ -D__ASM_SYSREG_H -Wno-unused-value -Wno-pointer-sign \
    -Wno-compare-distinct-pointer-types \
    -Wno-gnu-variable-sized-type-not-at-end \
    -Wno-tautological-compare \
    -O2 -emit-llvm -c memcpy_kprobe_kern.c
In file included from memcpy_kprobe_kern.c:1:
In file included from /lib/modules/4.15.0-36-generic/build//include/linux/filter.h:11:
In file included from /lib/modules/4.15.0-36-generic/build//include/linux/refcount.h:6:
In file included from /lib/modules/4.15.0-36-generic/build//include/linux/mutex.h:20:
/lib/modules/4.15.0-36-generic/build//arch/x86/include/asm/processor.h:537:17: error: taking address of packed
      member 'sp0' of class or structure 'x86_hw_tss' may result in an unaligned pointer value
      [-Werror,-Waddress-of-packed-member]
        this_cpu_write(cpu_tss_rw.x86_tss.sp0, sp0);
                       ^~~~~~~~~~~~~~~~~~~~~~
/lib/modules/4.15.0-36-generic/build//include/linux/percpu-defs.h:510:68: note: expanded from macro 'this_cpu_write'
#define this_cpu_write(pcp, val)        __pcpu_size_call(this_cpu_write_, pcp, val)
                                                                          ^~~foobar ➜  learn-bpf git:(master)
/lib/modules/4.15.0-36-generic/build//include/linux/percpu-defs.h:377:22: note: expanded from macro
      '__pcpu_size_call'
        __verify_pcpu_ptr(&(variable));                                 \
                            ^~~~~~~~
/lib/modules/4.15.0-36-generic/build//include/linux/percpu-defs.h:221:47: note: expanded from macro
      '__verify_pcpu_ptr'
        const void __percpu *__vpp_verify = (typeof((ptr) + 0))NULL;    \
                                                     ^~~
In file included from memcpy_kprobe_kern.c:1:
In file included from /lib/modules/4.15.0-36-generic/build//include/linux/filter.h:11:
In file included from /lib/modules/4.15.0-36-generic/build//include/linux/refcount.h:6:
In file included from /lib/modules/4.15.0-36-generic/build//include/linux/mutex.h:20:
/lib/modules/4.15.0-36-generic/build//arch/x86/include/asm/processor.h:554:30: error: taking address of packed
      member 'sp1' of class or structure 'x86_hw_tss' may result in an unaligned pointer value
      [-Werror,-Waddress-of-packed-member]
        return this_cpu_read_stable(cpu_current_top_of_stack);
                                    ^~~~~~~~~~~~~~~~~~~~~~~~
/lib/modules/4.15.0-36-generic/build//arch/x86/include/asm/processor.h:379:34: note: expanded from macro
      'cpu_current_top_of_stack'
#define cpu_current_top_of_stack cpu_tss_rw.x86_tss.sp1
                                 ^~~~~~~~~~~~~~~~~~~~~~
/lib/modules/4.15.0-36-generic/build//arch/x86/include/asm/percpu.h:392:59: note: expanded from macro
      'this_cpu_read_stable'
#define this_cpu_read_stable(var)       percpu_stable_op("mov", var)
                                                                ^~~
/lib/modules/4.15.0-36-generic/build//arch/x86/include/asm/percpu.h:219:16: note: expanded from macro
      'percpu_stable_op'
                    : "p" (&(var)));                    \
                             ^~~
In file included from memcpy_kprobe_kern.c:1:
In file included from /lib/modules/4.15.0-36-generic/build//include/linux/filter.h:11:
In file included from /lib/modules/4.15.0-36-generic/build//include/linux/refcount.h:6:
In file included from /lib/modules/4.15.0-36-generic/build//include/linux/mutex.h:20:
/lib/modules/4.15.0-36-generic/build//arch/x86/include/asm/processor.h:554:30: error: taking address of packed
      member 'sp1' of class or structure 'x86_hw_tss' may result in an unaligned pointer value
      [-Werror,-Waddress-of-packed-member]
        return this_cpu_read_stable(cpu_current_top_of_stack);
                                    ^~~~~~~~~~~~~~~~~~~~~~~~
/lib/modules/4.15.0-36-generic/build//arch/x86/include/asm/processor.h:379:34: note: expanded from macro
      'cpu_current_top_of_stack'
#define cpu_current_top_of_stack cpu_tss_rw.x86_tss.sp1
                                 ^~~~~~~~~~~~~~~~~~~~~~
/lib/modules/4.15.0-36-generic/build//arch/x86/include/asm/percpu.h:392:59: note: expanded from macro
      'this_cpu_read_stable'
#define this_cpu_read_stable(var)       percpu_stable_op("mov", var)
                                                                ^~~
/lib/modules/4.15.0-36-generic/build//arch/x86/include/asm/percpu.h:224:16: note: expanded from macro
      'percpu_stable_op'
                    : "p" (&(var)));                    \
                             ^~~
In file included from memcpy_kprobe_kern.c:1:
In file included from /lib/modules/4.15.0-36-generic/build//include/linux/filter.h:11:
In file included from /lib/modules/4.15.0-36-generic/build//include/linux/refcount.h:6:
In file included from /lib/modules/4.15.0-36-generic/build//include/linux/mutex.h:20:
/lib/modules/4.15.0-36-generic/build//arch/x86/include/asm/processor.h:554:30: error: taking address of packed
      member 'sp1' of class or structure 'x86_hw_tss' may result in an unaligned pointer value
      [-Werror,-Waddress-of-packed-member]
        return this_cpu_read_stable(cpu_current_top_of_stack);
                                    ^~~~~~~~~~~~~~~~~~~~~~~~
/lib/modules/4.15.0-36-generic/build//arch/x86/include/asm/processor.h:379:34: note: expanded from macro
      'cpu_current_top_of_stack'
#define cpu_current_top_of_stack cpu_tss_rw.x86_tss.sp1
                                 ^~~~~~~~~~~~~~~~~~~~~~
/lib/modules/4.15.0-36-generic/build//arch/x86/include/asm/percpu.h:392:59: note: expanded from macro
      'this_cpu_read_stable'
#define this_cpu_read_stable(var)       percpu_stable_op("mov", var)
                                                                ^~~
/lib/modules/4.15.0-36-generic/build//arch/x86/include/asm/percpu.h:229:16: note: expanded from macro
      'percpu_stable_op'
                    : "p" (&(var)));                    \
                             ^~~
In file included from memcpy_kprobe_kern.c:1:
In file included from /lib/modules/4.15.0-36-generic/build//include/linux/filter.h:11:
In file included from /lib/modules/4.15.0-36-generic/build//include/linux/refcount.h:6:
In file included from /lib/modules/4.15.0-36-generic/build//include/linux/mutex.h:20:
/lib/modules/4.15.0-36-generic/build//arch/x86/include/asm/processor.h:554:30: error: taking address of packed
      member 'sp1' of class or structure 'x86_hw_tss' may result in an unaligned pointer value
      [-Werror,-Waddress-of-packed-member]
        return this_cpu_read_stable(cpu_current_top_of_stack);
                                    ^~~~~~~~~~~~~~~~~~~~~~~~
/lib/modules/4.15.0-36-generic/build//arch/x86/include/asm/processor.h:379:34: note: expanded from macro
      'cpu_current_top_of_stack'
#define cpu_current_top_of_stack cpu_tss_rw.x86_tss.sp1
                                 ^~~~~~~~~~~~~~~~~~~~~~
/lib/modules/4.15.0-36-generic/build//arch/x86/include/asm/percpu.h:392:59: note: expanded from macro
      'this_cpu_read_stable'
#define this_cpu_read_stable(var)       percpu_stable_op("mov", var)
                                                                ^~~
/lib/modules/4.15.0-36-generic/build//arch/x86/include/asm/percpu.h:234:16: note: expanded from macro
      'percpu_stable_op'
                    : "p" (&(var)));                    \
                             ^~~

5 errors generated.
Makefile:55: recipe for target 'memcpy_kprobe_kern.o' failed
make: *** [memcpy_kprobe_kern.o] Error 1


# uname -a
Linux foobar 4.15.0-36-generic #39-Ubuntu SMP Mon Sep 24 16:19:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

[meishaoqing]

I have the same problem at 4.15.

[drug007]

As a workaround you can disable the warnings as error option. Comment this line in Makefile (put # in the beggining of the line):

EXTRA_CFLAGS=-Werror

[hsane2001]

I resolved this by 2 things on kernel 5.3

1. Used LLVM9 compiler
2. Add a compiler flag "-Wno-address-of-packed-member"

This compiles well but gives an error when I run it such as:
sh: 1: echo: echo: I/O error
failed to open event memcpy

[pobetiger]

In bpf_load.c:

 57 static int load_and_attach(const char *event, struct bpf_insn *prog, int size)
 58 {
 59     bool is_socket = strncmp(event, "socket", 6) == 0;
...
135         snprintf(buf, sizeof(buf),
136              "echo '%c:%s %s' >> /sys/kernel/debug/tracing/kprobe_events",
137              is_kprobe ? 'p' : 'r', event, event);

this apparently results in the following:

echo "p:memcpy memcpy" >> /sys/kernel/debug/tracing/kprobe_events

the error is complaining that the tracing is not enabled. By doing the following:

echo 1 > /sys/kernel/debug/tracing/tracing_on

Prior to starting the program, I was able to get past the IO error (failed to open event...error). The code is missing the command to turn on tracing.

Correction:
The command still fails to add the necessary kprobe point, if you have other probe events in the system, it will dump those out instead.