linuxtv-rkvdec-work-in-progress branch fails to compile hantro and rkvdec drivers

[rubenvb]

I'm trying to compile the current work-in-progress branch for my rk3399 and am running into compile errors. System is Archlinux ARM aarch64 using GCC 10.2.0.

The Hantro driver fails here:

drivers/staging/media/hantro/hantro_h264.c: In function 'prepare_table':
drivers/staging/media/hantro/hantro_h264.c:244:9: error: variable 'parity' has initializer but incomplete type
  244 |    enum v4l2_h264_field_reference parity = (i & 0x1) ?
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~
drivers/staging/media/hantro/hantro_h264.c:244:35: error: storage size of 'parity' isn't known
  244 |    enum v4l2_h264_field_reference parity = (i & 0x1) ?
      |                                   ^~~~~~
drivers/staging/media/hantro/hantro_h264.c:247:19: error: 'const struct v4l2_h264_dpb_entry' has no member named 'reference'; did you mean 'reference_ts'?
  247 |        dpb[i / 2].reference & parity)
      |                   ^~~~~~~~~
      |                   reference_ts
drivers/staging/media/hantro/hantro_h264.c:244:35: warning: unused variable 'parity' [-Wunused-variable]
  244 |    enum v4l2_h264_field_reference parity = (i & 0x1) ?
      |                                   ^~~~~~

The rkvdec driver (when I disable the hantro module in kernel config) fails here:

drivers/staging/media/rkvdec/rkvdec-h264.c: In function 'assemble_hw_rps':
drivers/staging/media/rkvdec/rkvdec-h264.c:786:8: error: variable 'a_parity' has initializer but incomplete type
  786 |   enum v4l2_h264_field_reference a_parity =
      |        ^~~~~~~~~~~~~~~~~~~~~~~~~
drivers/staging/media/rkvdec/rkvdec-h264.c:786:34: error: storage size of 'a_parity' isn't known
  786 |   enum v4l2_h264_field_reference a_parity =
      |                                  ^~~~~~~~
drivers/staging/media/rkvdec/rkvdec-h264.c:789:8: error: variable 'b_parity' has initializer but incomplete type
  789 |   enum v4l2_h264_field_reference b_parity =
      |        ^~~~~~~~~~~~~~~~~~~~~~~~~
drivers/staging/media/rkvdec/rkvdec-h264.c:789:34: error: storage size of 'b_parity' isn't known
  789 |   enum v4l2_h264_field_reference b_parity =
      |                                  ^~~~~~~~
drivers/staging/media/rkvdec/rkvdec-h264.c:805:19: error: 'const struct v4l2_h264_dpb_entry' has no member named 'reference'; did you mean 'reference_ts'?
  805 |     if ((dpb[idx].reference & a_parity) == a_parity &&
      |                   ^~~~~~~~~
      |                   reference_ts
drivers/staging/media/rkvdec/rkvdec-h264.c:820:19: error: 'const struct v4l2_h264_dpb_entry' has no member named 'reference'; did you mean 'reference_ts'?
  820 |     if ((dpb[idx].reference & b_parity) == b_parity &&
      |                   ^~~~~~~~~
      |                   reference_ts
drivers/staging/media/rkvdec/rkvdec-h264.c:789:34: warning: unused variable 'b_parity' [-Wunused-variable]
  789 |   enum v4l2_h264_field_reference b_parity =
      |                                  ^~~~~~~~
drivers/staging/media/rkvdec/rkvdec-h264.c:786:34: warning: unused variable 'a_parity' [-Wunused-variable]
  786 |   enum v4l2_h264_field_reference a_parity =
      |                                  ^~~~~~~~

[zillevdr]

The same error with default branch here. rkvdec does not compile:

drivers/staging/media/rkvdec/rkvdec-h264.c: In function ‘assemble_hw_rps’:
drivers/staging/media/rkvdec/rkvdec-h264.c:786:8: error: variable ‘a_parity’ has initializer but incomplete type
  786 |   enum v4l2_h264_field_reference a_parity =
      |        ^~~~~~~~~~~~~~~~~~~~~~~~~
drivers/staging/media/rkvdec/rkvdec-h264.c:786:34: error: storage size of ‘a_parity’ isn’t known
  786 |   enum v4l2_h264_field_reference a_parity =
      |                                  ^~~~~~~~
drivers/staging/media/rkvdec/rkvdec-h264.c:789:8: error: variable ‘b_parity’ has initializer but incomplete type
  789 |   enum v4l2_h264_field_reference b_parity =
      |        ^~~~~~~~~~~~~~~~~~~~~~~~~
drivers/staging/media/rkvdec/rkvdec-h264.c:789:34: error: storage size of ‘b_parity’ isn’t known
  789 |   enum v4l2_h264_field_reference b_parity =
      |                                  ^~~~~~~~
drivers/staging/media/rkvdec/rkvdec-h264.c:805:19: error: ‘const struct v4l2_h264_dpb_entry’ has no member named ‘reference’; did you mean ‘reference_ts’?
  805 |     if ((dpb[idx].reference & a_parity) == a_parity &&
      |                   ^~~~~~~~~
      |                   reference_ts
drivers/staging/media/rkvdec/rkvdec-h264.c:820:19: error: ‘const struct v4l2_h264_dpb_entry’ has no member named ‘reference’; did you mean ‘reference_ts’?
  820 |     if ((dpb[idx].reference & b_parity) == b_parity &&
      |                   ^~~~~~~~~
      |                   reference_ts
drivers/staging/media/rkvdec/rkvdec-h264.c:789:34: warning: unused variable ‘b_parity’ [-Wunused-variable]
  789 |   enum v4l2_h264_field_reference b_parity =
      |                                  ^~~~~~~~
drivers/staging/media/rkvdec/rkvdec-h264.c:786:34: warning: unused variable ‘a_parity’ [-Wunused-variable]
  786 |   enum v4l2_h264_field_reference a_parity =
      |                                  ^~~~~~~~

[rubenvb]

I fixed the build error by replacing enum v4l2_h264_field_reference with plain int (which has pretty much the same meaning, especially in C). It seems the enum doesn't exist anymore (yet?) in kernel 5.10.