Can't display gazebo video

[hamishwillee]

Gazebo Simulation · PX4 Developer Guide

On ubuntu 18.04 using Gazebo, I'm starting PX4 using: HEADLESS=1 make px4_sitl gazebo_typhoon_h480__sonoma_raceway.
This works, in QGC I can see the raceway on the map. I can see the video widget , and sometimes I can see the legs of the plane. But I cannot see the world - it is just shown as a grey sceen.

I tried to follow instructions of using gst-launch-1.0 as shown below. The UI launches and shows the same grey screen. I have included debug below from app. Am I missing something obvious?

@Jaeyoung-Lim @mrivi Would appreciate your advice.

gst-launch-1.0  -v udpsrc port=5600 caps='application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264' ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink fps-update-interval=1000 sync=false
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstUDPSrc:udpsrc0.GstPad:src: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstRtpH264Depay:rtph264depay0.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264
/GstPipeline:pipeline0/GstRtpH264Depay:rtph264depay0.GstPad:src: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, codec_data=(buffer)0164001fffe1001e6764001facd940a02ff97016a0c020b4a0000003002000000f11e30632c001000468efbcb0, level=(string)3.1, profile=(string)high
/GstPipeline:pipeline0/avdec_h264:avdec_h264-0.GstPad:sink: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, codec_data=(buffer)0164001fffe1001e6764001facd940a02ff97016a0c020b4a0000003002000000f11e30632c001000468efbcb0, level=(string)3.1, profile=(string)high
/GstPipeline:pipeline0/avdec_h264:avdec_h264-0.GstPad:src: caps = video/x-raw, format=(string)I420, width=(int)640, height=(int)360, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)bt601, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps = video/x-raw, width=(int)640, height=(int)360, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1, format=(string)YV12
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw, width=(int)640, height=(int)360, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1, format=(string)YV12
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage.GstPad:sink: caps = video/x-raw, width=(int)640, height=(int)360, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1, format=(string)YV12
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0.GstGhostPad:sink: caps = video/x-raw, width=(int)640, height=(int)360, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1, format=(string)YV12
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps = video/x-raw, format=(string)I420, width=(int)640, height=(int)360, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)bt601, framerate=(fraction)60/1
Redistribute latency...

[hamishwillee]

FYI, the test pattern ball works as a source, so it looks like QGC is good.
gst-launch-1.0 videotestsrc pattern=ball ! video/x-raw,width=640,height=480 ! x264enc ! rtph264pay ! udpsink host=127.0.0.1 port=5600

[hamishwillee]

To get video working with gazebo and on ubuntu 18.04 (in well resourced VM) I needed sudo apt install ubuntu-restricted-extras with associated clickthrough (to get avdec_h264). Is this something everyone will need in reality? OR is it just because I'm debugging?

For mcmillan airfield things are perfect

HEADLESS=1 make px4_sitl gazebo_typhoon_h480__mcmillan_airfield

For warehouse I get this crappy world.

For empty world and sonoma raceway I get a grey screen.

For yosemite I get errpr

ubuntu@ubuntu:~/github/Firmware$ HEADLESS=1 make px4_sitl gazebo_typhoon_h480__yosemite
ninja: error: unknown target 'gazebo_typhoon_h480__yosemite'
Makefile:205: recipe for target 'px4_sitl' failed
make: *** [px4_sitl] Error 1

For ksql_airport it looks like something, but not an airport.

Baylands starts looks same as above, but then turns grey.

gst-launch-1.0  -v udpsrc port=5600 caps='application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264' ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink fps-update-interval=1000 sync=false
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstUDPSrc:udpsrc0.GstPad:src: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264
/GstPipeline:pipeline0/GstRtpH264Depay:rtph264depay0.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264
/GstPipeline:pipeline0/GstRtpH264Depay:rtph264depay0.GstPad:src: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, codec_data=(buffer)0164001fffe1001e6764001facd940a02ff97016a0c020b4a0000003002000000f11e30632c001000468efbcb0, level=(string)3.1, profile=(string)high
/GstPipeline:pipeline0/avdec_h264:avdec_h264-0.GstPad:sink: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, codec_data=(buffer)0164001fffe1001e6764001facd940a02ff97016a0c020b4a0000003002000000f11e30632c001000468efbcb0, level=(string)3.1, profile=(string)high
/GstPipeline:pipeline0/avdec_h264:avdec_h264-0.GstPad:src: caps = video/x-raw, format=(string)I420, width=(int)640, height=(int)360, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)bt601, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps = video/x-raw, width=(int)640, height=(int)360, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1, format=(string)YV12
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw, width=(int)640, height=(int)360, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1, format=(string)YV12
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage.GstPad:sink: caps = video/x-raw, width=(int)640, height=(int)360, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1, format=(string)YV12
/GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0.GstGhostPad:sink: caps = video/x-raw, width=(int)640, height=(int)360, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)60/1, format=(string)YV12
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps = video/x-raw, format=(string)I420, width=(int)640, height=(int)360, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)bt601, framerate=(fraction)60/1
Redistribute latency...

[Jaeyoung-Lim]

@hamishwillee This is very different from what I see on my local system.

How does it look on the gazebo client? To reproduce this, should I also run a VM?

[hamishwillee]

@Jaeyoung-Lim I can get this working on a virtualbox VM, but not a VMWare VM. Therefore we know that this can work, and it is more likely a driver or similar issue. I'll do some debugging to see if I can get any further with "why". For now, thanks so much for your help, but hold off until (perhaps) I can find something useful to share.

[Jaeyoung-Lim]

@hamishwillee one thing I noticed with AGS is that id doesnt work with certain graphics drivers. For example of you use the Xorg graphics driver it wont wont work but if you use the nvidia graphics driver it will

Not sure if this js relevant to virtualbox...but might be worth checking!

[hamishwillee]

@Jaeyoung-Lim OK, I THINK this is a setup/dependency issue., but I am not sure how to fix it.

When you install (same) ubuntu on Virtualbox or VMWARE you get a slightly different installation process. For Virtualbox you can select "install third party components etc" as a checkbox. If you do that then gazebo video just works (note, I have removed the avdec line out of the setup for this during testing - I think it may remove errors but still not "be right")

However for VMWARE you don't get this options. Instead you get an error when you run the gst-launch-1.0 ... test, and you have to install the extras using sudo apt install ubuntu-restricted-extras (and do a clickthrough on that).
The error message goes, but you don't get the rendered video in this case.

Below shows the video vindo in both cases. Note the different header in the renderer on virtualbox. Could this be a hint?

Anyway, is any of that helpful to be sure of the likely cause?