Create a version that works with Picamera2

[woutersf]

I attempted to run this on my Raspberry pi 4 with Picamera2 and coultn't get it to work.
the picamera2 docs only have a mjpeg server example and a seperate H264 quantiser example

Is there another branch/version of this repo where this works with picamera2 ?

I suspect the changes are something like the following however I don't speak python fluently.

To be added:

from picamera2 import Picamera2
from picamera2.encoders import H264Encoder
from picamera2.outputs import CircularOutput  # (or FileOutput?)

and then the camera references should be removed or replaced by picam2
where picam2 iis intialised and starts streaming.

Also in the try:

    picam2 = Picamera2()
    fps = 30
    dur = 5
    micro = int((1 / fps) * 1000000)
    vconfig = picam2.create_video_configuration()
    vconfig['controls']['FrameDurationLimits'] = (micro, micro)
    picam2.configure(vconfig)
    encoder = H264Encoder()
    output = CircularOutput(buffersize=int(fps * (dur + 0.2)), outputtofile=False)
    output.fileoutput = "file.h264"
    picam2.start_recording(encoder, output)
    application = tornado.web.Application(requestHandlers)
    application.listen(serverPort)
    streamBuffer.setLoop(loop)
    loop.start()

If anyone has any hints on getting this to work, I'm willing to give it some attempts but it's not familiar to me, I would need some guidance.

[dans98]

Honestly, You would be better to start from scratch. the picamera2 module is completely different, and I'm not sure it can be made to work in the same fashion.

[nikola-j]

Here is a picamera2 version:

import io
import os
import socket
from string import Template
from threading import Condition

import tornado.ioloop
import tornado.web
import tornado.websocket
from picamera2 import Picamera2
from picamera2.encoders import H264Encoder
from picamera2.outputs import Output

# start configuration
serverPort = 8000

picam2 = Picamera2()
picam2.configure(picam2.create_video_configuration(main={"size": (640, 480)}))

framerate = 10

s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.connect(('8.8.8.8', 0))
serverIp = s.getsockname()[0]

abspath = os.path.abspath(__file__)
dname = os.path.dirname(abspath)
os.chdir(dname)


def getFile(filePath):
    file = open(filePath, 'r')
    content = file.read()
    file.close()
    return content


def templatize(content, replacements):
    tmpl = Template(content)
    return tmpl.substitute(replacements)


indexHtml = templatize(getFile('index.html'), {'ip': serverIp, 'port': serverPort, 'fps': framerate})
jmuxerJs = getFile('jmuxer.min.js')


class StreamingOutput(Output):
    def __init__(self):
        super().__init__()
        self.loop = None
        self.buffer = io.BytesIO()

    def setLoop(self, loop):
        self.loop = loop

    def outputframe(self, frame, keyframe=True, timestamp=None):
        self.buffer.write(frame)
        if self.loop is not None and wsHandler.hasConnections():
            self.loop.add_callback(callback=wsHandler.broadcast, message=self.buffer.getvalue())
        self.buffer.seek(0)
        self.buffer.truncate()


class wsHandler(tornado.websocket.WebSocketHandler):
    connections = []

    def open(self):
        self.connections.append(self)

    def on_close(self):
        self.connections.remove(self)

    def on_message(self, message):
        pass

    @classmethod
    def hasConnections(cl):
        if len(cl.connections) == 0:
            return False
        return True

    @classmethod
    async def broadcast(cl, message):
        for connection in cl.connections:
            try:
                await connection.write_message(message, True)
            except tornado.websocket.WebSocketClosedError:
                pass
            except tornado.iostream.StreamClosedError:
                pass

    def check_origin(self, origin):
        return True


class indexHandler(tornado.web.RequestHandler):
    def get(self):
        self.write(indexHtml)


class jmuxerHandler(tornado.web.RequestHandler):
    def get(self):
        self.set_header('Content-Type', 'text/javascript')
        self.write(jmuxerJs)


requestHandlers = [
    (r"/ws/", wsHandler),
    (r"/", indexHandler),
    (r"/jmuxer.min.js", jmuxerHandler)
]

try:
    output = StreamingOutput()
    encoder = H264Encoder(repeat=True, framerate=framerate, qp=20)
    encoder.output = output
    picam2.start_recording(encoder, output)

    application = tornado.web.Application(requestHandlers)
    application.listen(serverPort)
    loop = tornado.ioloop.IOLoop.current()
    output.setLoop(loop)
    loop.start()
except KeyboardInterrupt:
    picam2.stop_recording()
    loop.stop()

[chradev]

Hi guys,

I have just forked @nikola-j and @kroketio repository pi-h264-to-browser and make some changes to stream video from both RPi 5 cameras to a browser and setup the range-of-interest by picamera2 property ScalerCrop. My code is on pi-h264-to-browser-stramer but my experience in Python is very weak so I will be extremely grateful for a little help to solve following problems in server.py:

• avoiding duplication of the StreamingOutput and wsHandler classes
• using of {OffsetX, OffsetY, Width, Height} values in the StreamingOutput class but calculated by another application (independent process) - maybe by using of Unix sockets or something alike

I am working on RPi 5 & bookworm RPi official OS with 2x RPi cameras v.2.

Best regards
Christo

[leeprevost]

I'll checkout your fork @chradev. Thanks for example @nikola-j.

I'm also working on a picamera2 derivative -- but am using FASTApi rather than tornado. I haven't used jumuxer but rather closely following the picamera2 examples mjpyeg_server_2.py. This essentially takes video off picamera2, encodes with MJPEG, captures in a custom buffer, and then uses a frame generator to generate the MJPEG video rendered directly in the browser.

my thinking, since I'm trying to make all this work in a PiZero2 is to use hardware encoding and hardware buffered as much as I can leaving CPU headroom for other work.

i've got the stream working in FASTApi but I'm having some instability issues with thread locks particularly when two browsers sessions hit it or when a user accesses the endpoint/route for the stream directly (vs. through the HTML index page). I think that is probably because the picamera2 example was not designed for async so am down a rabbit hole now on making the buffer/frame generator asynchronous.

[UltimateCodeWarrior]

Hi All,

Just food for thought:
While the whole Remote PI Camera thing was a great journey to learn python, I fell out of love with the RPI 4/5 as a stand alone camera / remote view app.

Why not just use a H264 POE camera, at least the night vision works.  I wasn't able to find a worthy PI Camera that responded well to IR Light and that was a big deal breaker.   Plus, you couldn't do 12mp video as I had hoped.  It's close to a 4k.      You can offload a lot of AI functions over to a POE Camera, might even be able to poll for new clips and just process the clips.  Wont be real time, but will save a lot space and work for initial project. 

I Like the Amcrest $99 turret cameras, they have an on-board wifi config interface. No stupid download app to configure.
They have a limited HTTP API, but at least you can lock onto the Video/Audio with PyAv and mux them into a composite MP4 file. Then you can just use the PI (or any other powerhouse) to connect as your NVR to the camera. Usually they can support multiple simultaneous streams, and the H264 only takes about 1 megabit of bandwidth at 4k /20fps resolution. This also saves your SD/NVME drive from needing to be a speed demon and from filling up too fast. 4k MPEG was taking 20mbits for just one stream. Not good.

A better option than the PI would be the NVIDIA Jetson one SBC if you want to do your own motion detection/object detection image processing on the stream. Power consumption is a beast, but so is the processing. If you just process the motion clips stored on the camera, you might be able to take that $250 board and be running AI on multiple cameras on the cheap.

And I had a lot of issues with WiFi H264 feeds on the pi. It would work for a bit and then get into some 'resend the packet' feedback loop that would take down the PI. Did much better with a WavLink POE Hat and hardwired ethernet connection. You can get away with a Wireless Bridge like a TPLink 800Megabit,but there is still video ghosting at only 300feet. Be careful of powering any larger 12v fans off that hat, the EMF will 'warp' the video coming across those ribbons. Those ribbons were also another headache. I don't know why they didn't just use USB3!

[leeprevost]

Ok thanks for that. I'll log this away for when I do a custom security setup.

Right now, my application is for a small underwater battery powered device. So low power, small, and low latency are critical. Also would like to do as much on the "edge" as possible around detection but with a very simple model. Streaming back 800x600 topside just for monitoring, not detecting or recording. Would love to use that Jetson but power budget and $$ budget doesn't allow.

[UltimateCodeWarrior]

You could always use a drone camera with a transmitter the submerged part, and have a pi/receiver for your computer https://www.youtube.com/watch?v=rN-V3slnIcE

[leeprevost]

I seriously thought about that approach but I went down the piz2 route because

• Low power
• I want to use Linux rsynch and networking for multiple camera.
• I haven't seen much on edge detection with drones. I expect my WiFi to be up and down on turns and with conditions so I want some local processing to continue even without a connection. Kind of like the jamming problem with drones.

I haven't ruled that approach out yet.

I also got mediamtx/webrtc working today so I may go that path rather than try to solve the mjpeg thread lock issue. If it weren't for non standard cameras (arducam) I'd be done by now but arducam's use of non standard libcam is maddening!