HID services

[kdschlosser]

I am wanting to either connect a USB mouse to the jtag controller on an ESP32-S3 or use a bluetooth mouse. With having the ability to use lvgl and micropython it seems like connecting things like mice and keyboards either bluetooth or USB seems like a logical thing to do. I have not been able to locate any examples of hot to do this written for use without micropython or for micropython.

I have not found anything in the IDF that parses either bluetooth or USB data past the base class identifier. Meaning I am able to tell if the device attached is HID but that is as far as it goes.... The USB and also the bluetooth seems mostly geared at the ESP32 being an emulator or a client and transmitting instead of being a host and receiving. If a display is able to be connected and a GUI is able to be displayed then there needs to be ways of providing user input. Now I know that touchscreens are available but what if someone doesn't want to have to use up 4 GPIO pins to use an SPI touchscreen or 2 pins for an I2C touchscreen? Bluetooth is there and doesn't use any pins. As screens get larger in size at a cheaper price point it would make sense to provide other forms of user input beyond touchscreens.

If anyone has some kind of boilerplate code for doing this that I can use as an example of how to go about it that would be a starting point.

any help is appreciated.

--kevin

[jimmo]

connect a USB mouse to the jtag controller on an ESP32-S3

I'm not quite sure what this would mean...

Perhaps you mean using the S3's USB OTG as a USB host (e.g. https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/api-reference/peripherals/usb_host.html)

Unfortunately this is not supported by MicroPython (although USB Host mode in general is on the TODO list).

or use a bluetooth mouse

This is possible. I'm not aware of any examples of this specifically, however there is a pending example of the reverse (i.e. making the ESP32 appear as a BLE mouse). It should be possible to adapt this code to be the client instead the server -- #6559

[kdschlosser]

Yes using the native USB in the S3 to connect a keyboard or a mouse to it and be able to type using the keyboard or use the mouse to click on things that are being displayed on the display.

All of the examples I can find are for the ESP emulating a keyboard or a mouse so when the ESP is plugged into a computer the computer sees the ESP as an HID keyboard or mouse. That is not what I am talking about. I want to plug a keyboard or a mouse into the ESP and be able to read mouse movements or key presses on the keyboard.

I believe that Bluetooth might be a better thing to use because of the lack of power on the USB that is provided by display development kits that have the ESP onboard. None of them that I have seen so far provide power to the devices connected to that USB.

As a side note.. Having that USB connected to a PC and the ESP showing up as a mass storage controller to be able to access the filesystem on the ESP would be a pretty cool thing to have. Having to use ampy or the like to upload files is horrible. A drag and drop interface would be a whole lot nicer to deal with. 480Mbps is a whole lot faster than 921600bps which is the limit of the cp2x UART to USB IC's that are commonly used. IDK what the limit is to the onboard USB for speed but from what I have read it appears to support USB2.0 high speed. IDK if the trasnfer speeds hold true to that. I would imagine that both the repl and the file transfer can be done over the one USB connection which would mean that UART0 would only need to be used for flashing the module and micropython would not have to bind the repl to it.

[robert-hh]

Having that USB connected to a PC and the ESP showing up as a mass storage controller to be able to access the filesystem on the ESP would be a pretty cool thing to have. Having to use ampy or the like to upload files is horrible. A drag and drop interface would be a whole lot nicer to deal with.

You can use ftp to transfer files, using e.g. filezilla or even some file managers, A suitable ftp server for the board is e.g. https://github.com/robert-hh/FTP-Server-for-ESP8266-ESP32-and-PYBD

Making MSC available is an ongoing discussion. It is available for STM32 board, but using it is leads sooner or later to a corrupted file system on the board. It hit me several times until I decided not to use it any more. There are PRs for RP2 and MIMXRT supporting MSC by avoiding file system corruption, but these are in the "to be considered" stall. The other drawback of MSC support is the need to use a FAT file system on the board, which is not the best choice. There are experimental drivers for the pC to deal with a LFS file system, but nor very convenient to use.

[kdschlosser]

I have been messing about with Dokan and that is another option that could be used for Windows and for nix based systems you have Fuse that can be used. I am writing a python binding to Dokan and one can also be written for Fuse pretty easily I am sure. This would make cross platform pretty easy to do. Can use the existing mechanisms for transferring files whether it be over WiFi connecting to the repl or through serial again through the repl. It would be restricted to the speed in which either of those works but at least the items would show up in the OS as a removable drive and file can be browsed, deleted, copied. Ideally there would need to be some kind of hard coded mechanism that can be accessed using a ctrl code in the repl to be able to transfer files and get filesystem information without having to constantly send code to the repl to open a file for read or write or run an os command to return data about the file/folder structure.

File system corruption would not be ideal to have happen. not sure why it would happen, maybe there is some incompatibility been the FAT file system used on the MCU's and the OS on a computer.

[kdschlosser]

I exposed all of the functions, enums, structs and constants in the usb component to micropython using the script that LVGL uses to expose LVGL to micropython. It actually worked. I am going to mess about with getting the HID devices added that way. I am more comfortable writing code in Python then I am with C/CPP

[robert-hh]

File system corruption would not be ideal to have happen. not sure why it would happen, maybe there is some incompatibility been the FAT file system used on the MCU's and the OS on a computer.

The corruption happens because the PC assumes to be the only entity writing to the storage. It caches for instance the directory and data to be written. If then the board writes to the file system itself or reboots before the PC has written all data, then the result is garbage. I tried a long time, to be careful, not to write by the PC and board in a session, doing a sync before ejecting the board, ... But after a while things went bad.

[kdschlosser]

on Windows at least you can tell it to write the files right away instead of doing the whole caching and waiting thing.