Interesting coincidence

[Interpause]

While working on a fairly complicated project, and somewhat inspired by Mediapipe, I somehow ended up writing something similar to this. So far, I have been using asyncio for my Source, Analyze (instead of Transform lol) and Sink worker loops. I only use separate processes for Analysis workers so far. I also stumbled across dearPyGui while desperately looking for an asyncio-compatible GUI. But I am quite far behind your project in that I lack an editor. I spent too much time on fancy logging and configuration systems and trying to get the entire thing to package using PyInstaller (which works btw).

https://github.com/Interpause/nicepipe

Probably useful to you would be the fact I have written code for sending both the video feed and prediction to clients via socket.io and webrtc.

If I give up by next month, I might end up using and contributing to your project. Thanks for the awesome work.

[Interpause]

Actually let me reexplain. Nicepipe aims to be multi-process computer vision pipeline. It takes in video & outputs analysis using components such as feature detectors, tracking & object detection. This analysis is then received by a frontend client for further visualization.

Essentially, heavy machine learning approaches in the backend, visualization and actuation in the frontend.

In the long run, I also aim for it to be distributed. Multiple nicepipe instances can run across multiple machines. Downstream instances receive data from upstream and performs higher level analysis on it. The frontend can connect to multiple backends in order to receive data as well as reconfigure them live. Basically an overly ambitious idea.

The difference in our projects is that I am focused on the analysis (separating out visualization of data) while your project is more on video processing, I assume. That said, I lack a solid video processing pipeline, which is useful for preprocessing inputs for analysis components.

Also both our repository names are not great. Nicepipe is obviously a bad name, and a quick search on Github reveals too many repositories called Heron.

[Interpause]

My current progress

[georgedimitriadis]

Hey, Thanks for letting me know that there are other people out there with similar requirements in software tools. It is really cool that you made your software run from an .exe. This is something I would like to do also but for the time being I needed as soon as possible something that could run experimental pipelines so I was thinking (selfisly) more of my own needs rather than those of other users. "In the long run, I also aim for it to be distributed. Multiple nicepipe instances can run across multiple machines. Downstream instances receive data from upstream and performs higher level analysis on it. The frontend can connect to multiple backends in order to receive data as well as reconfigure them live. Basically an overly ambitious idea." Not that ambitious. If I understood well what you are aiming for, Heron currently kind of has that, with the frontend being the Heron gui and every Node (i.e. the worker and com processes of it) being the backend(s). Which machine the worker process of its Node will run on is defined in the gui and Heron takes care of the data exchange and lifetime of each process between machines. What you are aiming for is full instances of your software running on separate machines. Heron cannot run headless yet but this is definitely on the to do list (it will also solve a rather weird issue Heron has at the moment). So the plan is to have Heron run headless and have Heron Nodes that can connect a headless instance of Heron to other, individual Nodes. "while your project is more on video processing". I do not think of Heron like that. Although I have written a few more Nodes that capture and save video (simply because the requirements of the experiments I am running at the moment demand lots of cameras) Heron is more generic than a specific pipeline. It is designed to facilitate most pipelines one can possibly need. I haven't written any Nodes yet that would grab pre saved data and pass them through an analysis pipeline but it is a very easy thing to do and I am planning to do so soon (as my experiment turns from data capture to data analysis mode). And that is the bottom line of Heron. When you need a functionality you write the individual Nodes that will generate it and then Heron takes care of the rest. Using dearpygui I also plan to create Nodes that can become parts of an analysis specific GUI. For the moment I have a visualisation Node and I want to add 'data selector' Nodes i.e. things like sliders and buttons and take it from there. I chose the name Heron because I think of it as a constructor of automata and the first person I know of who did that was Heron or Alexandria. So I do not mind that much that Heron is an often-used name. The way I see it, when this software shows up in the first 3 links of a google search then I will know people really like it 🙂. Again thank you very much for coming in contact. If you end up using Heron I am more than happy to help with any questions. I am releasing a paper that describes things a little bit better in two to three weeks and soon after I will release the first manual. If you want to contribute either at the level of the basic Heron capabilities or at the level of writing Nodes let me know. Again I am very happy to help George

…

________________________________ From: John-Henry Lim ***@***.***> Sent: 15 May 2022 05:14 To: Heron-Repositories/Heron ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [Heron-Repositories/Heron] Interesting coincidence (Discussion #3) Actually let me reexplain. Nicepipe aims to be multi-process computer vision pipeline. It takes in video & outputs analysis using components such as feature detectors, tracking & object detection. This analysis is then received by a frontend client for further visualization. Essentially, heavy machine learning approaches in the backend, visualization and actuation in the frontend. In the long run, I also aim for it to be distributed. Multiple nicepipe instances can run across multiple machines. Downstream instances receive data from upstream and performs higher level analysis on it. The frontend can connect to multiple backends in order to receive data as well as reconfigure them live. Basically an overly ambitious idea. The difference in our projects is that I am focused on the analysis (separating out visualization of data) while your project is more on video processing, I assume. That said, I lack a solid video processing pipeline, which is useful for preprocessing inputs for analysis components. Also both our repository names are not great. Nicepipe is obviously a bad name, and a quick search on Github reveals too many repositories called Heron. — Reply to this email directly, view it on GitHub<#3 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADCLS4ZBBSSWXPWRO3HRDUTVKCB45ANCNFSM5V6PGPKQ>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[Interpause]

Its been a long while, I ended up abandoning the project in favour of using the Robot Operating System (ROS), which works surprisingly well for computer vision too (see Nvidia's NITROS project).

After realising I was using the wrong search terms, I actually found quite a few projects that would have been suitable. For example, Dask's has an underlying library, Dask Distributed, that is very similar to what both of us were trying to do...