focal length

[LIUMeng0823]

Great job on iris tracking in python!

however, could you give some detials about why you choose focal length = img_width?
and the following computation: x = np.sqrt(focal_length_pixel ** 2 + y ** 2) && depth = human_iris_size_in_mm * x / iris_size in "iris_lm_depth.py"

Thanks!

[Rassibassi]

hi,
the variable in question is the focal length in pixel for solving the intercept theorem, which this depth calculation is based on.

In my code I provide the possibility of setting that value yourself, or it chooses the image width in pixel by default, which roughly worked for my camera (which I calibrated via opencv).

From the cpp implementation of mediapipe, following the lines here [1]:

if (cc->InputSidePackets().HasTag(kFocalLengthPixelTag)) {
#if defined(__APPLE__)
    focal_length_pixels_ = *cc->InputSidePackets()
                                .Tag(kFocalLengthPixelTag)
                                .Get<std::unique_ptr<float>>();
#else
    focal_length_pixels_ =
        cc->InputSidePackets().Tag(kFocalLengthPixelTag).Get<float>();
#endif
    compute_depth_from_iris_ = true;
  } else if (cc->InputSidePackets().HasTag(kImageFilePropertiesTag)) {
    const auto& properties = cc->InputSidePackets()
                                 .Tag(kImageFilePropertiesTag)
                                 .Get<ImageFileProperties>();
    focal_length_pixels_ = properties.focal_length_pixels();
    compute_depth_from_iris_ = true;
  }

Above you can see, the focal length is set directly by the developer via kFocalLengthPixelTag or retrieved from a kImageFilePropertiesTag side packet. So following that kImageFilePropertiesTag tag, you can find a way of calculating the focal length in pixel.

The kImageFilePropertiesTag side packet is populated here [2]:

# Computes the focal length in pixels based on EXIF information stored in the
# image file. The output is an ImageFileProperties object containing relevant
# image EXIF information along with focal length in pixels.
node {
  calculator: "ImageFilePropertiesCalculator"
  input_stream: "input_image_bytes"
  output_side_packet: "image_file_properties"
}

and the focal length in pixel is then calculated here [3]:

absl::StatusOr<double> ComputeFocalLengthInPixels(int image_width,
                                                  int image_height,
                                                  double focal_length_35mm,
                                                  double focal_length_mm) {
  // TODO: Allow returning image file properties even when focal length
  // computation is not possible.
  if (image_width == 0 || image_height == 0) {
    return absl::InternalError(
        "Image dimensions should be non-zero to compute focal length in "
        "pixels.");
  }
  if (focal_length_mm == 0) {
    return absl::InternalError(
        "Focal length in mm should be non-zero to compute focal length in "
        "pixels.");
  }
  if (focal_length_35mm == 0) {
    return absl::InternalError(
        "Focal length in 35 mm should be non-zero to compute focal length in "
        "pixels.");
  }
  // Derived from
  // https://en.wikipedia.org/wiki/35_mm_equivalent_focal_length#Calculation.
  /// Using focal_length_35mm = focal_length_mm * SENSOR_DIAGONAL_35MM /
  /// sensor_diagonal_mm, we can calculate the diagonal length of the sensor in
  /// millimeters i.e. sensor_diagonal_mm.
  double sensor_diagonal_mm =
      SENSOR_DIAGONAL_35MM / focal_length_35mm * focal_length_mm;
  // Note that for the following computations, the longer dimension is treated
  // as image width and the shorter dimension is treated as image height.
  int width = image_width;
  int height = image_height;
  if (image_height > image_width) {
    width = image_height;
    height = image_width;
  }
  double inv_aspect_ratio = (double)height / width;
  // Compute sensor width.
  /// Using Pythagoras theorem, sensor_width^2 + sensor_height^2 =
  /// sensor_diagonal_mm^2. We can substitute sensor_width / sensor_height with
  /// the aspect ratio calculated in pixels to compute the sensor width.
  double sensor_width = std::sqrt((sensor_diagonal_mm * sensor_diagonal_mm) /
                                  (1.0 + inv_aspect_ratio * inv_aspect_ratio));

  // Compute focal length in pixels.
  double focal_length_pixels = width * focal_length_mm / sensor_width;
  return focal_length_pixels;
}

The cpp function ComputeFocalLengthInPixels is not implemented in my python version.

[1] https://github.com/google/mediapipe/blob/ecb5b5f44ab23ea620ef97a479407c699e424aa7/mediapipe/graphs/iris_tracking/calculators/iris_to_depth_calculator.cc#L140
[2] https://github.com/google/mediapipe/blob/ecb5b5f44ab23ea620ef97a479407c699e424aa7/mediapipe/graphs/iris_tracking/iris_depth_cpu.pbtxt#L18
[3] https://github.com/google/mediapipe/blob/ecb5b5f44ab23ea620ef97a479407c699e424aa7/mediapipe/calculators/image/image_file_properties_calculator.cc#L31

[LIUMeng0823]

thanks a lot for your explaination! it seems that the focal length in pixel equals to "fx" in the camera's calibration, since it is calculated as "width * focal_length_mm / sensor_width"