Skip to content

Add dual fisheye .jpg mode (Fusion)  #31

@himynamesdave

Description

@himynamesdave

Another video mode, this time will accept dual jpg images shot in fisheye.

GoPro Fusion cameras also produce a dual .jpg format (two fisheye images that can be stitched together to create a single equirectangular 360).

Gopro2gsv validates as follows:

  • GoPro2GSV checks the directory only contains .jpg files (case insensitive). It ignores any hidden dot files that might be present, however, will fail if any other type of file or directory is detected.
  • GoPro2GSV checks the filename prefixes all match to a single timelapse recorded on the camera and that there is a pair of GPFR and GPBK with same 4 digit number, e.g. GPFR0001.JPG, GPBK0001.JPG, GPFR0002.JPG, GPBK0002.JPG
  • GoPro2GSV checks all valid images from previous step have the same dimensions by ensuring all images have the same values for File:ImageWidth and File:ImageWidth properties

We already have some old code to join

Tool: https://github.com/trek-view/fusion2sphere
Example implementation in another tool: https://github.com/trek-view/gopro2frames/blob/main/docs/LOGIC.md#7b-fusion2sphere-flow-2-gopro-fisheye-frames

Note: resusing code is fine, but do not copy the whole codebase of this implementation.

After this, the GPS points are then embedded to the equirectangular images (same as #26) and the final video is processed in the same way as the image->video pipeline including the new function; #25

In this mode, a user should be able to set --keep_extracted_frames which will also store a copy of the extracted images used to create final video (with geotags). If not set, frames extracted will be removed on completion.

===== below example docs

dual fisheye timelapse frames -> fusion2sphere -> timelapse frames -> final video (for Fusion cameras)
* this mode take two fisheye timelapse frames, stitches them together into equirectangular frames using fusion2sphere, removes erroneous GPS points/frames, and the extracted frames are then rebuilt into an equirectangular video, or set of equirectangular videos. The user options are the same as equirectangular timelapse frames -> final video

Mode 5: dual fisheye timelapse frames -> fusion2sphere -> timelapse frames -> final video (for Fusion cameras)

Fusion cameras produce a pair of two .jpg fisheye images. These need to be converted into equirectangular format for use with most major viewing software (e.g. GSV).

Overview:

  • 4a: user input
  • 4b: validation
  • 4c:

5a: User options on input

Same as 2a: User options on input

5b: Validation

Gopro2gsv validates as follows:

  1. GoPro2GSV checks the directory for .jpg files (case insensitive). It ignores any other files that might be present.
  2. Checks there are >=40 frames in the input (20 final images)
  3. GoPro2GSV checks the filename prefixes all match to a single timelapse recorded on the camera and that there is a pair of GPFR and GPBK with same 4 digit number, e.g. GPFR0001.JPG+GPBK0001.JPG, GPFR0002.JPG+GPBK0002.JPG ...
  4. GoPro2GSV checks the metadata of all valid GPFR images from previous step using exiftool, and images that are not valid are discarded;
    • file extension must be .jpg (case insensitive)
    • GPS:GPSLatitudeRef must be present
    • GPS:GPSLatitude must be present
    • GPS:GPSLongitudeRef must be present
    • GPS:GPSLongitude must be present
    • GPS:GPSAltitudeRef must be present
    • GPS:GPSAltitude must be present
    • GPS:GPSTimeStamp must be present
    • GPS:GPSDateStamp must be present
  5. GoPro2GSV checks all valid images from previous step have the same dimensions by ensuring all images have the same values for File:ImageWidth and File:ImageHeight properties
exiftool -ee -G3 -api LargeFileSupport=1 -ProjectionType -GPSLatitudeRef -GPSLatitude -GPSLongitudeRef -GPSLongitude -GPSAltitudeRef -GPSAltitude -GPSTimeStamp -GPSDateStamp -ImageWidth -ImageHeight GPFR<IMAGE>.jpg
  1. GoPro2GSV sorts all valid images from previous step by time (GPS:GPSDateStamp+GPS:GPSDateStamp), earliest to latest.
  2. GoPro2GSV checks that time between consecutive GPFR images (GPS:GPSDateStamp+GPS:GPSDateStamp) is not >20 seconds. If true then the whole directory is not considered and an ERROR log is reported
  3. GoPro2GSV checks that a directory with all valid images from previous step has >=40 valid images after the previous step. If <40 valid images then the whole directory is not considered and an ERROR log is reported
  4. GoPro2GSV renames the images files in format NNNNN.jpg, starting from 00001.jpg and ascends, e.g. 00001.jpg, 00002.jpg...
  5. GoPro2GSV tracks the data of each timelapse image, including those that failed validation, in the local database. The original and new filename is included so that it is clear what files were not considered in the final video (and why)

5c: Processing

Useful supporting information for this section;

Processing is broken up into four key parts;

  1. extract GPS from video
  2. extracting frames from .360 input
  3. turning extracted frames into equirectangular projections using max2sphere
  4. geotag extracted frames

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions