Proposal for tcMenu remote control using a 2nd embedded target fitted with switches, encoders, and an LCD for an ergonomic UI

[AtmosphEng]

Hi Dave and co.,
I have a feature request which would allow a kind of embedControl functionality but running on a 2nd dedicated wired or wireless target device. My project devices are usually ESP32-WROOM and/or ESP32-S3. I've already mentioned to you this functionality a while back as an unofficial hack. However this has actually been tested and is available now if two (identical or similar) target devices are programmed with the same tcMenu tree with tcMenuDesigner Generated Code, and IOT remote control is set to e.g. Serial2 on both target devices.

If both target devices (say device-A and device-B) are now connected with a cross-over (x-over) cable (i.e. TxA to RxB, and TxB to RxA) then one or both target devices can control their own tcMenu-A and the other target tcMenu-B interchangeably. I've tested this at 115200-8-n-1. The advantage of this simple interconnection is that it allows the embedded tcMenu rendering of device-B to be almost obtained almost for 'free' after the device-A code is working as expected on its own. The remote control is now possible without requiring embedControl to be installed on a separate host.

In addition to the wired x-over above, I've recently done some extra wireless testing with this Serial x-over method with good success too. To achieve this I've added two more devices each with Serial2 ports and programmed them to act as a transparent wireless serial bridge using the following wireless connection types:
(1) Bluetooth pairing,
(2) TCPIP over Wifi, and
(3) ESP-NOW peer-to-peer.
However, adding the extra transparent wireless serial bridge to achieve this is not optimal, and it would be great if a Code Generation option could be added so that at least for ESP32 devices, their onboard wireless hardware of Bluetooth and Wifi could be used instead - Wifi would be my strong preference for the greater range distance. Perhaps the second target device could be designated as an embedded remote control 'Client' for such connections.

I apologise in advance for the hack, as this is rarely popular with the original developer(s). But the advantages seem too good to me to be ignored. Two good use cases come to mind:
(a) an embedded remote control 'Client' can be fitted with more ergonomic input devices for operation e.g. switches and rotary encoders, than is easily possible with host-based embedControl, and
(b) it can help with the eternal embedded problem of running out of IO pins by adding a second device for the User Interface (UI). device-A just needs to allocate 2 pins at most for a UI hardware Serial port and device-B can have the switches and/or encoders and/or local LCD fitted.

Please let me know if you would be interested in making the above embedded remote control a standard part of the tcMenu feature set. I can sponsor a modest amount to help this along. If you need more details, test setups or diagrams, please let me know.
Thanks for reading.

[AtmosphEng]

Could you please give me some feedback on the feasibility of adding an embedded 'client' via wifi in the tcMenuDesigner Code Generator, as outlined above? I made a sponsorship contribution to support this request.

[davetcc]

Sorry for the exceptionally long delay.

Honestly, I had to take a break from developing TcMenu for a while as it became overwhelming.

I think that at present, with effectively one developer working on it part-time, we can't do more than keep on the lights.

After the break I thought about which direction the project should take, and I think personally, the focus has to be on a working core offering (menu structure, displays, input). I'm happy to discuss it with you offline, and maybe discuss making you a custom plugin or something like that, but I don't want to build that kind of complexity into the Designer UI, it is already a very complex UI without adding multiple devices into code generator along with more protocols into the mix.

[AtmosphEng]

notes - arduino ide com0com - for tcmenu embedcontrol with mcu as host – virtserial

Hi davetcc, great to hear from you. I totally understand you could need a break from tcMenu. The community you’ve created here are beholding greatly to you, and that any output and support you provide will always be greatly welcomed and appreciated.

In the meantime, I’ve already implemented a workaround (with some limitations) for a tcMenu tagValue protocol Remote Controller (RC) using a esp32s3 MCU and Serial crossover communications based on Wifi.
From the outset I tried to minimise the changes required for this tagValue RC MCU feature in the tcMenu codebase. The small changes needed in the tcmenu codebase are made only in tcMenuDesigner generated file project*_menu.cpp. Please see below for a simple example of the changes required.

I have a couple of private repository libraries which implement this called VirtSerial and CommsHelper. If you are interested I can clean them up the best I can and make them public.

Please let me have your thoughts, and your assessment of this remote control extension to MCUs. I’m happy to discuss it further here or by some other means. Please read on for a basic outline of the changes.

INTRODUCTION TO tcMenu SERIAL UART CROSSOVER FOR tagValue REMOTE CONTROL VIA esp32s3 MCU.

I wanted to use tcMenu embedControl tagValue protocol, but with an MCU as host e.g. an esp32s3. EmbedControl used Java-based hosts such as Rpi, Linux, Windows, Android Nomone Desktop, and Apple. Having had some experience with tcMenu in an MCU such as the Lilygo esp32s3 t-display-s3 as a controller with e.g. an integrated LCD and WiFi, I tried to build an MCU based tagValue protocol host to use in applications similar to a Remote-Control (RC) model car scenario where a tagValue protocol host such as a t-display-s3 fitted with external Rotary Encoders (RE) could be used to control the speed and position of the RC car. In my application this was actually for a custom designed and built model train.

In earlier experiments I configured 2x tcMenu t-display-s3’s fitted with up to 3x RE’s and with identical code on each. Serial communications was enabled between the 2x MCUs using the tcMenuDesigner Generate Code Option of ESP Serial remote control with a Serial Port choice of Serial2. In the wiring the Tx and Rx of each MCU was wired with a simple crossover arrangement. When the 2x Serial2 ports were run at 115200-8-n-1, then the primary RE of each MCU could not only control the tcMenu LCD display of the ‘local’ t-display-s3 MCU , but also that of the ‘remote’ MCU and vice-versa. This was a ‘light-bulb’ moment for me as I loved using tcMenu as a ‘local’ MCU controller, and now it seemed that for a little effort that an RC version of tcMenu could be run on an identical or similar ‘remote’ controller MCU. Indeed, if tcMenuDesigner Code Generator was configured to have an additional IoT/Remote then 3-way control was achieved using the Linux or Windows app versions of embedControl.

So far, so good. All worked well, but could the Serial crossover communications between the 2x tcMenu MCUs work with Wifi? After some experimentation I found the Wifi results were less good but still usable. I found that if one of the REs was turned too fast then the tcMenu control variable on each of the MCUs became unsynchronised. This lack of synchronisation could be fixed by making the tcMenu control variable on one of the MCU’s to be ‘Read-Only’, and this is acceptable where control is actually essential on the ‘remote’ MCU only for example. I also tried the wireless ESP-NOW protocol and had better synchronisation success, but I’ve reverted back to Wifi in the meantime for ease of use. It seems that the increased latency of the Serial crossover Wifi compared to the wired UART is responsible for the tcMenu control variable problems with lack of synchronisation.

WIRELESS WIFI IMPLEMENTATION WORKAROUND FOR tcMenu SERIAL CROSSOVER FOR tagValue REMOTE CONTROL VIA esp32s3 MCU.

From the outset I tried to minimise the changes required for this tagValue RC MCU feature in the tcMenu codebase. Maybe you are familiar or heard-of the com0com virtual serial port software app for Windows? For example, if 2x pre-written (existing) Windows apps want to have mutual Serial comms using a COMx port, then instead of using 2x external COMx ports and crossover wiring, com0com could be used to create a virtual Serial port which was then used for a software solution without the need for external COMx ports and wiring. So I created my version of com0com (which I called VirtSerial) for Arduino IDE so that a relatively simple manual edit in the tcMenuDesigner generated file project*_menu.cpp would be the only changes required to the tcMenu codebase.

AN EXAMPLE OF THE SMALL CHANGES NEEDED IN THE TCMENU CODEBASE, MADE IN tcMenuDesigner generated file project*_menu.cpp
When the tcMenuDesigner Code Generator for ESP Serial remote control is specified as Serial port: Serial2, then the extract of the generated file project*_menu.cpp is as expected previously

e.g.
SerialTagValueTransport serialTransport(&Serial2);

and with the VirtSerial Arduino IDE myCom0com equivalent it becomes:
//**************************************
#include "../../../libraries/VirtSerial/src/VirtSerial.h"
extern VirtSerial myCom0com;
#if(1)
SerialTagValueTransport serialTransport((HardwareSerial*)&myCom0com);
#else
SerialTagValueTransport serialTransport(&Serial2);
#endif
//**************************************