USB endpoints (including trace) broken on Black Pill (STM32F4xx)

[DavidCPlatt]

While waiting for my official BMP 2.3 order to arrive, I've been experimenting with "building my own" using a WeAct STM32F411 board.

Basic functionality (debug and flash) is working.

SWO tracing does not work at all, in either the current 1.10 pre-release, or in the 1.9.2 build I did. The symptom of the failure is that the host attempt to read from the trace endpoint fails immediately, with a "broken pipe" error (transfer stalled, apparently).

The identical version of 1.9.2, built for "stlink" and flashed into a "Baite" ST-Link v2 clone, does not exhibit this error. SWO doesn't capture anything on the host (due to the fact that the trace line isn't brought out to the connector) but the host-to-USB connection doesn't flail and die.

I've been digging into the problem for a day or so, and I've come to the sad conclusion that trace-capture may never be able to work on the Black Pill. In fact, I'm sorta surprised that BMP runs as well on this board/chip as it does. The reason: the USB OTG controller used in the STM32F411 (and apparently the F107 as well) has hardware support for one control endpoint (#0), three IN endpoints (0x81-0x83), and three OUT endpoints(1-3)... period. No others need apply. You can have zero, one, two or three. Counteth thee not to four. Five is right out.

BMP uses four IN endpoints (for the two CDC-ACM), so it's already out of range.

I looked through the logic in libopencm3/.../usb_dwc_common.c to see how it accesses the hardware registers, and was somewhat troubled to see that it doesn't bother to range-check the endpoint index at all. If you pass it endpoint numbers in the valid ranges (0, 1-3, 0x81-0x83) it hits the right hardware registers. If you pass larger numbers (like 0x85 for trace) it appears to just go ahead and read/write quite happily to unimplemented register addresses. This doesn't seem to cause hardware to throw exceptions, but I rather suspect that the results are quite undefined and may result in purple monkey bus dishwasher.

I've looked in detail at the hardware-register definitions, in the hope that one might be allowed three active IN endpoints at a time, and be able to multiplex those three slots among a larger number of logical endpoints. I haven't found any way to do so - the endpoint index/number is implied by the register offset, and isn't set explicitly in any register I've found.

I looked at a Wireshark/usbmon trace of the traffic between host and Black Pill when I try to start reading the trace data. The host fires off a bunch of bulk-read URBs in rapid sequence. A fraction of a second later the Black Pill starts sending responses - a combination of "broken pipe" and "no such address" is how Wireshark interprets them. I'd guess that the F411 USB-OTG core is taking exception to being asked to send data from endpoint numbers it has no registers/FIFO for.

In theory I suspect that this same problem might affect the VCP CDC-ACM, since it uses endpoint 0x84 for control. Maybe it's never sending anything of significance on that endpoint?

So, that's the unfortunate news. Unless I'm gravely wrong (and I certainly may be) it won't be possible to put a full-function BMP firmware into any of the STM32 devices that have this particular USB-OTG core. Some functionality will have to be sacrificed - either trace won't work, or the VCP "UART" will have to be configured out, and the trace endpoint moved down to 0x83.

[ALTracer]

Welcome, Dave. Here's what I can clarify as a Blackmagic community contributor and blackpill-f411ce platform user for 4 months.
#1152 (comment)
As I stated in PR1152, it seems indeed impossible to use raw trace data from platforms using dwc2 core (stm32f4) because of limited endpoints. So that leaves orbuculum suite off the table. It's dictated by BMP-common endpoint allocation to dual CDC-ACM interfaces, and changing it may break Windows 7 clients which require an .inf and don't honor BOS descriptors.

Second/VCP port allows getting/setting line coding (baudrate, parity, stop) from the defaults of 38400-8n1, and also sends RTS/DTR (platform gpio) states over that interrupt endpoint, where present. AFAIK pins for RTS and DTR are only declared for swlink platform, but could be easily added for others. These are useful for external RS-485 transceivers, secondary non-reset lines for target boot mode manipulation and such.

However, since #664 BMF core supports decoding traceswo channels in-probe, and dumps selected channels to VCP interface, so that ITM_printf() messages can be observed without libusb desktop software and raw bulk endpoint access. If that covers your usecase -- try it and report back how well it performs.

Finally, there are a couple low-priority PRs regarding TraceSWO, namely PR1092 which evolved from PR423 and introduces FIFOs for Manchester capture driver IIUC; and my #1529 which attempts to switch F4 platforms from Manchester-only to faster UART-type capture driver, which is already used by swlink/stlink platforms (but harms their heap robustness) and employs a hardware UART+DMA instead of hardware timer capture. That one may perform at 2.25 Mbaud rather than 100kHz, if only in short bursts.

No endpoint multiplexing (time-division) can happen as BMD composite device uses USB Class drivers (DFU, CDC-ACM serial) and a raw bulk export endpoint understood by SWO traffic consuming software. Additionally, I think even multi-config or switching configs will break operation flow of existing tooling (BMDA, bmpflash, etc.)

[DavidCPlatt]

Hi, ALTracer! Thanks for the comprehensive reply!

It seems that I've independently rediscovered an issue you folks were aware of - I missed seeing it when I searched the open issues. I think we've come to quite similar conclusions.

I will try using the VCP-decoding again, as it would certainly cover my initial use case (debug output from the Zephyr logging back-end). I haven't been able to get it to work yet - my 'scope can see by board "wiggling" the trace-output line, but nothing ever decodes in BMP. I suspect I don't yet have the Manchester communication working correctly. Zephyr defaults to asynchronous, I've had to patch the source code to change this (they don't yet have a config option to select Manchester) and I need to experiment with bit rates some more.

I agree that there's probably no way to work around the limited endpoint support in this USB core, and that any alternate approaches would probably break a lot of the back-end tools. Pity.

One slightly-out-of-the-box approach I did think of, would be to move away from USB-composite entirely, and switch to a different communication method - say, Ethernet-over-USB.

A semi-straightforward (but quite nontrivial) way to do this would be to port the BMP core logic to run as an application on an RTOS which has Ethernet-over-USB support and also a usable TCP/IP stack. Zephyr looks as if it would quality, and I'm sure there are others. One could run three or more TCP services - e.g. the GDB-server would be one (easy for GDB to connect to), the VCP could be another (TELNET?), and the non-decoded TraceSWO could be a simple "connect to this TCP port, and the fire-hose starts running" stream.

This would obviously result in a substantial increase in code size and data size - I can't see that STLINK/SWLINK and similar '103-sized chips would ever be able to support it. Something like the 411 or 401, would probably be enough. As to performance... I don't know. Given that the 411 Manchester is limited to relatively low bit rates, TCP-over-IP-over-Ethernet-over-USB might be fast enough to keep up.

If I understand correctly there may be at least one BMP derivative which has gone in this general direction.

FYI, my applications so far are two-fold, both running under Zephyr on Black Pill or similar-sized STM32 platforms. I've coded up a new amateur-ratio TNC (basically a Bell 202-compatible firmware-DSP "modem" coupled with AX.25 HDLC-based packet management and host data-packet communication), and a control-panel interface for a homebrew RF test-signal generator (drives an LCD, decodes buttons and a rotary encoder, and handles configuration management and setup which is then SPI'ed over to a small FPGA board). BMP has been quite useful in the development process. Having decent-rate trace output (decoded or otherwise) is thus quite desirable.

[ALTracer]

One slightly-out-of-the-box approach I did think of, would be to move away from USB-composite entirely, and switch to a different communication method - say, Ethernet-over-USB.

Funny story, I also raised the idea of CDC-ECM in Discord. While it will require substantially more flash space, it can actually solve the problem for F4 platforms which have bigger flash but fewer endpoints.

This would obviously result in a substantial increase in code size and data size - I can't see that STLINK/SWLINK and similar '103-sized chips would ever be able to support it. Something like the 411 or 401, would probably be enough. As to performance... I don't know. Given that the 411 Manchester is limited to relatively low bit rates, TCP-over-IP-over-Ethernet-over-USB might be fast enough to keep up.

Sometime in February I tried majbthrd/stm32ecm on a 32F072B-Disco, and even managed to shove an additional iperf2 server there. The project occupied most of SRAM (15.5/16 KiB), used 50 KiB of code and 40 KiB of static resources (for the html webpage, ssi, and svg of a toaster). I'm not sure how many lwIP components would be required here (dhcpd, dns), but it definitely has to provide multiple tcp servers -- mandatory gdb_if, aux_serial and raw traceswo, optional decoded traceswo, redirected semihosting writes, rtt_if and debug_serial for BMP-internal logging. Also it needs some way of reconfiguring DHCPd away from default served subnet if it collides with a given corporate network. In fact, this approach only uses 3 endpoints, if losing DFU over EP0 is deemed acceptable, and does not need to be composite.

If I understand correctly there may be at least one BMP derivative which has gone in this general direction.

Correct. But also they used beefier chips, and lost the simplicity and elegance of upstream BMP (RTOS-less and bit-banging!) in the process of porting. The only important benefit I see they might have is the ability of serving multiple GDB sockets and hence reading memory of a running target (how brittle that may be -- is a good question).

TBH this topic is better suited for GH Discussions. Feel free to close after confirming Zephyr ITM-prints as Manchester (or UART) and your blackpill-f411ce can see it (as slow Manchester!). When in doubt, read the BMD docs on TraceSWO usage, or fall back to f103cb-based adapter platforms (these are confirmed to work with traceswo/async).

[DavidCPlatt]

I've built a Blue Pill (F103) version, and unfortunately it isn't capturing SWO traces with my Zephyr applications either.

At this point I rather suspect that something is wrong on the Zephyr side, unrelated to the specific BMP implementations I'm using. Maybe pin setup is wrong/absent/missing for the Zephyr board config?

In any case, I'm content to close this issue as "known problem, infeasible to fix due to limitations of the specific chip platform involved".

[dragonmux]

That works for us - we've been quietly following along. What we'll suggest then is we'll mark the issue "Won't fix", not to dismiss the issue but because there's not much we can do for the STM32F4 platforms on this front. If you then find it's actually a bug in the BMD SWO implementation we'll happily change the issue markings and work with you to figure out what and how to fix it.

Just let us know what you'd like to do here and we'll work with you on it. Marking for v2.0 so we track it to the right milestone.