Working on IMX214 on RockPro64- no stream #67
Description
While I was working on the the IMX214 already last year with limited success, I picked it up again and I need some buddies to discuss the problem I'm facing.
As it seems it is a mixture of device tree problems and problems with the library itself (probably). There is no forum to collect all the findings I made so far, so this is an attempt to do so and make the IMX214 finally work with libcamera - and to provide a working solution that hopefully makes use of libcamera as well.
To start off, here's what I did:
- the Rockchip-ISP1 driver is compiling nicely on the latest kernels from 5.15.89-rockchip64, so that should provide a working basis.
- the libcamera compiles fine, but lacks support for the IMX214. I opened a pull request with the two minor changes (thx to Marvin Schmidt for providing me with this).
- the mainline IMX214 driver lacks certain capabilities that libcamera requires. Therefore I patched it to fix the missing properties. Here is a version with Makefile to compile it outside the kernel branch. Loads fine and finds the camera. I2Ctransfer also does a nice job and gets data out.
- the device tree itself for the RockPro64 - it's a mess, but I would say it is 95% working...
/dts-v1/;
/plugin/;
#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/pinctrl/rockchip.h>
#include <dt-bindings/clock/rk3399-cru.h>
/ {
compatible = "rockchip,rk3399";
fragment@0 {
target = <&isp0_mmu>;
__overlay__ {
status = "okay";
};
};
fragment@1 {
target = <&mipi_dphy_rx0>;
__overlay__ {
status = "okay";
};
};
fragment@2 {
target = <&isp0>;
__overlay__ {
status = "okay";
ports {
port@0 {
#address-cells = <1>;
#size-cells = <0>;
mipi_in_wcam: endpoint@0 {
reg = <0>;
remote-endpoint = <&wcam_out>;
data-lanes = <1 2 3 4>;
};
};
};
};
};
fragment@3 {
target = <&sdio0>;
__overlay__ {
status = "disabled";
};
};
fragment@4 {
target = <&i2c1>;
__overlay__ {
status = "okay";
// https://github.com/friendlyarm/kernel-rockchip/blob/nanopi-r2-v5.15.y/arch/arm64/boot/dts/rockchip/rk3399-gru-scarlet.dtsi
clock-frequency = <400000>;
i2c-scl-falling-time-ns = <50>;
i2c-scl-rising-time-ns = <300>;
// 24M mclk is shared between world and user cameras
pinctrl-0 = <&i2c1_xfer &cif_clkouta>;
// pinctrl-1 = <&wcam_rst>;
#address-cells = <1>;
#size-cells = <0>;
wcam: imx214@10 {
compatible = "sony,imx214";
reg = <0x10>;
vdddo-supply = <&vcc1v8_mipi>;
vddd-supply = <&vcc1v2_mipi_dvdd>;
vdda-supply = <&vcc2v8_mipi>;
reset-gpios = <&gpio0 RK_PB4 GPIO_ACTIVE_LOW>; // GPIO0_B4
clocks = <&cru SCLK_CIF_OUT>;
clock-names = "xvclk";
rotation = <180>;
orientation = <2>;
// ORIGINAL BEGIN
port {
wcam_out: endpoint {
data-lanes = <1 2 3 4>;
link-frequencies = /bits/ 64 <480000000>;
remote-endpoint = <&mipi_in_wcam>;
};
};
// ORIGINAL END
};
};
};
fragment@5 {
target = <&pinctrl>;
__overlay__ {
dvp {
dvp_pwr_en: dvp-pwr-en {
rockchip,pins = <1 RK_PC7 RK_FUNC_GPIO &pcfg_pull_none>;
};
};
camera {
wcam_rst: wcam_rst {
rockchip,pins = <0 RK_PB4 RK_FUNC_GPIO &pcfg_pull_none>;
};
};
cif {
cif_clkouta: cif-clkouta {
rockchip,pins = <2 RK_PB3 3 &pcfg_pull_up>;
};
};
};
};
fragment@6 {
target-path = "/";
__overlay__ {
vcc2v8_mipi: vcc2v8-mipi-regulator {
compatible = "regulator-fixed";
enable-active-high;
gpio = <&gpio1 RK_PC7 GPIO_ACTIVE_HIGH>;
pinctrl-names = "default";
pinctrl-0 = <&dvp_pwr_en>;
regulator-name = "vcc2v8_mipi";
regulator-always-on;
regulator-boot-on;
regulator-min-microvolt = <2800000>;
regulator-max-microvolt = <2800000>;
startup-delay-us = <1000>;
vin-supply = <&vcc5v0_sys>;
};
// turned on automatically as vcc2v8_mipi is turned on...
vcc1v8_mipi: vcc1v8-mipi-regulator {
compatible = "regulator-fixed";
regulator-name = "vcc1v8_mipi";
regulator-always-on;
regulator-boot-on;
regulator-min-microvolt = <1800000>;
regulator-max-microvolt = <1800000>;
startup-delay-us = <1000>;
vin-supply = <&vcc3v3_sys>;
};
vcc2v8_mipi_af: vcc2v8-mipi-af-regulator {
compatible = "regulator-fixed";
regulator-name = "vcc2v8_mipi_af";
regulator-always-on;
regulator-boot-on;
regulator-min-microvolt = <2800000>;
regulator-max-microvolt = <2800000>;
startup-delay-us = <1000>;
vin-supply = <&vcc5v0_sys>;
};
vcc1v2_mipi_dvdd: vcc1v2-mipi-dvdd-regulator {
compatible = "regulator-fixed";
regulator-name = "vcc1v2_mipi_dvdd";
regulator-always-on;
regulator-boot-on;
regulator-min-microvolt = <1200000>;
regulator-max-microvolt = <1200000>;
startup-delay-us = <1000>;
vin-supply = <&vcc3v3_sys>;
};
};
};
};
I originally had some trouble setting up the clocks, but those seem to be fine thx to a pro from the Discord forum. According to the specs there is GPIO2_B4 and GPIO0_B4 for the PowerDown-Pins, depending on the MIPI port used, but I can't really make sense out of them.
Anyway, here's the issue: the libcamera works and finds the camera, but stream mapping is broken (it gets stuck). Similarly lc_compliance gets stuck already at the first test. There is also this media-ctl sequence, which should get some data out at some point:
"media-ctl" "-d" "platform:rkisp1" "-l" "'imx214 1-0010':0 -> 'rkisp1_isp':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_mainpath':0 [0]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"imx214 1-0010":0 [fmt:SRGGB10_1X10/1920x1080]'
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":0 [fmt:SRGGB10_1X10/1920x1080 crop: (0,0)/1920x1080]'
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":2 [fmt:YUYV8_2X8/1920x1080 crop: (0,0)/1920x1080]'
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":0 [fmt:YUYV8_2X8/1920x1080 crop: (0,0)/1920x1080]'
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":1 [fmt:YUYV8_2X8/1920x1080]'
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_mainpath":0 [fmt:YUYV8_2X8/1920x1080 crop: (0,0)/1920x1080]'
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_mainpath":1 [fmt:YUYV8_2X8/1920x1080]'
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1920,height=1080,"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "pixelformat=422P"
and ultimately
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" --stream-to=single.raw
fails with "broken pipe" or "invalid device" or something similar. So it seems that libcamera has the same problem with the raw stream capture by media-ctl/video-ctl..., and I can't get rid of the feeling that I'm missing something somewhere. There was a conversation on the DietPi forum about this, but no real working solution at the end either...
I have a similar setup for the OV13850 as well, for which I also already patched the devicetree and libcamera - that's a slightly different story, but the outcome is the same... will try to summarize this in an other submission...