Track and clarify differences with upstream OpenOCD

[en-sc]

After the most recent merge #976 the difference with upstream version used looks like so (some differences are omitted):

> git checkout 1f512eac32e614e893565741f93ea3739a522797
...
> git diff d4575b647a3603200a9bb4a784d170f792ab88d0 --stat
 .github/*
 ...
 HACKING                                                                     |   26 +
 Makefile.am                                                                 |    6 +-
 configure.ac                                                                |    3 +-
 contrib/loaders/flash/gd32v/gd32vf103.inc                                   |    8 +
 doc/openocd.texi                                                            |  278 +++++++++-
 git-hooks/*
 src/flash/nor/tcl.c                                                         |    3 +-
 src/helper/Makefile.am                                                      |    2 +
 src/helper/base64.c                                                         |  157 ++++++
 src/helper/base64.h                                                         |   17 +
 src/jtag/drivers/ftdi.c                                                     |  448 +++++++++++++++-
 src/rtos/FreeRTOS.c                                                         |  763 ++++++++++++++++++++-------
 src/rtos/hwthread.c                                                         |   49 +-
 src/rtos/rtos.c                                                             |  266 ++++++++--
 src/rtos/rtos.h                                                             |   52 +-
 src/rtos/rtos_standard_stackings.c                                          |  213 +++++++-
 src/rtos/rtos_standard_stackings.h                                          |    5 +
 src/server/gdb_server.c                                                     |  176 +++++--
 src/target/breakpoints.c                                                    |  213 ++++----
 src/target/breakpoints.h                                                    |    2 +-
 src/target/dsp563xx.c                                                       |    3 +-
 src/target/riscv/*
 ...
 src/target/target.c                                                         |   96 ++--
 src/target/target.h                                                         |   12 +-
 tcl/board/esp32c2-*
 ...
 tcl/board/sifive-*
 ...
 tcl/interface/ftdi/arty-onboard-ftdi.cfg                                    |    7 +
 tcl/interface/ftdi/digilent-hs2-cjtag.cfg                                   |   17 +
 tcl/interface/ftdi/olimex-arm-jtag-cjtag.cfg                                |   27 +
 tcl/interface/ftdi/olimex-arm-usb-ocd-h-cjtag.cfg                           |   22 +
 tcl/interface/ftdi/olimex-arm-usb-tiny-h-cjtag.cfg                          |   27 +
 "tcl/target/1986\320\262\320\2651\321\202.cfg" => tcl/target/1986Be1T.cfg   |    0
 "tcl/target/\320\2721879x\320\2611\321\217.cfg" => tcl/target/K1879x61R.cfg |    0
 tcl/target/esp*
 tcl/target/gd32vf103.cfg                                                    |  105 +++-
 tools/filter_openocd_log.py                                                 |  120 +++++

Some of these files seem suspicious, namely:

 contrib/loaders/flash/gd32v/gd32vf103.inc
 src/target/dsp563xx.c
 tcl/interface/ftdi/*
 tcl/target/gd32vf103.cfg 

[en-sc]

Regarding contrib/loaders/flash/gd32v/gd32vf103.inc -- it was moved and re-generated in the upstream.

> git grep gd32vf103.inc
contrib/loaders/flash/gd32vf103/Makefile:all: gd32vf103.inc
src/flash/nor/stm32f1x.c:#include "../../../contrib/loaders/flash/gd32vf103/gd32vf103.inc"
> diff contrib/loaders/flash/gd32vf103/gd32vf103.inc contrib/loaders/flash/gd32v/gd32vf103.inc
2,4c2,8
< 0x83,0x57,0x06,0x00,0x13,0x06,0x26,0x00,0x23,0x90,0xf6,0x00,0x83,0x27,0x05,0x00,
< 0x13,0xf7,0x17,0x00,0xe3,0x1c,0x07,0xfe,0x93,0xf7,0x47,0x01,0x63,0x98,0x07,0x00,
< 0x93,0x85,0xf5,0xff,0x93,0x86,0x26,0x00,0xe3,0x9c,0x05,0xfc,0x73,0x00,0x10,0x00,
---
> 0x6f,0x00,0x80,0x00,0x73,0x00,0x10,0x00,0x03,0x2b,0x06,0x00,0x63,0x0c,0x0b,0x04,
> 0x83,0x2a,0x46,0x00,0xb3,0x87,0x6a,0x41,0xe3,0x88,0x07,0xfe,0x03,0xdb,0x0a,0x00,
> 0x23,0x10,0x67,0x01,0x93,0x8a,0x2a,0x00,0x13,0x07,0x27,0x00,0x83,0x2b,0xc5,0x00,
> 0x93,0xf7,0x1b,0x00,0xe3,0x9c,0x07,0xfe,0x93,0xf7,0x4b,0x01,0x63,0x90,0x07,0x02,
> 0x63,0xe6,0xda,0x00,0x93,0x0a,0x06,0x00,0x93,0x8a,0x8a,0x00,0x23,0x22,0x56,0x01,
> 0x93,0x85,0xf5,0xff,0x63,0x88,0x05,0x00,0x6f,0xf0,0x1f,0xfb,0x13,0x05,0x00,0x00,
> 0x23,0x22,0xa6,0x00,0x13,0x85,0x0b,0x00,0x6f,0xf0,0xdf,0xf9,

[en-sc]

The change to src/target/dsp563xx.c is made by 53ec10b to create riscv repeat_read

[en-sc]

There is a significant difference in src/jtag/drivers/ftdi.c and tcl/interface/ftdi/*. Some commits are quite recent (2022). @timsifive, can you please clarify whether these changes are RISC-V specific in some way? Is it possible to start upstreaming them without upstreaming the main RISC-V-specific part?

[timsifive]

The ftdi changes let a user debug a target using cJTAG OSCAN1 protocol. It was added in #320. I'm not sure if it's ever seen significant use.

There are significant FreeRTOS changes as well. They support 64-bit targets, and also reorganize a bunch of code so it was easier to support RISC-V with 2 different call stack conventions. This is the one that seems the hardest to upstream since you probably need to check that it still works on some non-RISC-V target before sending it upstream. I don't remember any users ever asking about RISC-V FreeRTOS so probably it's not used much.

The breakpoints changes are recent, and are causing quite a few conflicts when merging changes down. I'd prioritize upstreaming these just to reduce the work required in resolving these conflicts.

The rtos changes primarily let gdb correctly access registers from individual threads that are not the current one. This is probably important in conjunction with hwthread and multiple harts.

[TommyMurphyTM1234]

Might this be an opportune time to consider a full "two way/up and down" sync of the RISC-V fork with the upstream OpenOCD?

• https://sourceforge.net/p/openocd/mailman/message/59186930/

Preparing for OpenOCD v1.0.0-rc1

From: Antonio B. <bor...@gm...> - 2025-05-23 10:16:54

Hello,

we are going to tag OpenOCD v1.0.0-rc1 in around one month.
It will follow a period of code freeze, during which only patches for
bug fixes will be merged.
The release v1.0.0 is expected before the end of the year.

...

I understand that such work is always dependent/contingent on there being sufficient voluntary support to assist with execution and this is often in short supply. If I personally can be of any assistance I'm happy to help.

I'm not sure if there are specific issues that might represent difficulties with such a sync?

Copying @ilg-ul and @AntonioBorneo in case they are interested.

[AntonioBorneo]

The code freeze period upstream could be a good opportunity to upstream some code from this fork, without the need to track big changes in the other direction.
I could merge them upstream once v1.0.0 is released.

Another way could be to "review" the code in this fork (actually the delta with upstream) and push here changes to simplify the following review upstream.
I have pushed riscv/riscv-debug-spec#1122 that goes in this direction, but I don't see it progressing...

[ilg-ul]

I'll be happy to know that the risc-v patches are upstreamed and I'll be glad to make a new xPack release once v1.0.0 is out, but I'm afraid I can't be of great help to review the patches, my experience with risc-v debuggers is minimal.

[en-sc]

@tom-van, @aap-sc, @borneoa, @MarekVCodasip let's transfer the discussion from Gerrit to here.

Current status of the diff with RISC-V OpenOCD

I've tried to resolve all the merge conflicts and merge the current RISC-V Sync into the current sync with the mainline to assess what is left to be upstreamed.
You can check the results in en-sc#1.
Running

git diff en-sc/riscv-sync-diff^2 --stat

(at 52aa6f3)

• .github/workflows/*, git-hooks/* -- irrelevant (CI).
• HACKING -- the difference is entirely c3c76bf. IMHO, it can be dropped, since we have d4a64e3.
• doc/openocd.texi -- FreeRTOS related.
• src/rtos/freertos.c -- FreeRTOS support for RISC-V, huge diff.
• src/rtos/rtos_standard_stackings.* -- AFAIU, for RISC-V FreeRTOS the RTOS stackings were redesigned.
• tcl/* -- there is a number of new scripts. I don't really know how to upstream these.
• src/target/riscv/riscv-013.c -- due to Fix bug in scratch_reserve #1295
• src/jtag/jtag.h -- a FIXME comment introduced in f3ed0ab and addressed in 8646
• src/rtos/hwthread.c -- non-functional, addressed in TODO.
• src/rtos/rtos.* -- functional changes related to stackings and some cosmetics.
• src/server/gdb_server.c -- cosmetic, except for one mysterious line:

This is the problematic diff:

@@ -3186,19 +3190,15 @@ static bool gdb_handle_vcont_packet(struct connection *connection, const char *p
                gdb_connection->output_flag = GDB_OUTPUT_ALL;
                target_call_event_callbacks(ct, TARGET_EVENT_GDB_START);

-               /*
-                * work around an annoying gdb behaviour: when the current thread
-                * is changed in gdb, it assumes that the target can follow and also
-                * make the thread current. This is an assumption that cannot hold
-                * for a real target running a multi-threading OS. We just fake
-                * the step to not trigger an internal error in gdb. See
-                * https://sourceware.org/bugzilla/show_bug.cgi?id=22925 for details
-                */
                if (fake_step) {
+                       /* We just fake the step to not trigger an internal error in
+                        * gdb. See https://sourceware.org/bugzilla/show_bug.cgi?id=22925
+                        * for details. */
                        int sig_reply_len;
                        char sig_reply[128];

                        LOG_DEBUG("fake step thread %"PRIx64, thread_id);
+                       target->rtos->current_threadid = thread_id;

                        sig_reply_len = snprintf(sig_reply, sizeof(sig_reply),
                                                                        "T05thread:%016"PRIx64";", thread_id);

I don't understand where this

+                       target->rtos->current_threadid = thread_id;

comes from.
The blame shows 868ebdd, but running git blame 868ebdd89c^2 -- src/server/gdb_server.c results in:

5f46de2e79 (MadSquirrel        2022-06-04 22:01:11 +0200 3100)          /*
5f46de2e79 (MadSquirrel        2022-06-04 22:01:11 +0200 3101)           * work around an annoying gdb behaviour: when the current thread
5f46de2e79 (MadSquirrel        2022-06-04 22:01:11 +0200 3102)           * is changed in gdb, it assumes that the target can follow and also
5f46de2e79 (MadSquirrel        2022-06-04 22:01:11 +0200 3103)           * make the thread current. This is an assumption that cannot hold
5f46de2e79 (MadSquirrel        2022-06-04 22:01:11 +0200 3104)           * for a real target running a multi-threading OS. We just fake
5f46de2e79 (MadSquirrel        2022-06-04 22:01:11 +0200 3105)           * the step to not trigger an internal error in gdb. See
5f46de2e79 (MadSquirrel        2022-06-04 22:01:11 +0200 3106)           * https://sourceware.org/bugzilla/show_bug.cgi?id=22925 for details
5f46de2e79 (MadSquirrel        2022-06-04 22:01:11 +0200 3107)           */
5f46de2e79 (MadSquirrel        2022-06-04 22:01:11 +0200 3108)          if (fake_step) {
5f46de2e79 (MadSquirrel        2022-06-04 22:01:11 +0200 3109)                  int sig_reply_len;
5f46de2e79 (MadSquirrel        2022-06-04 22:01:11 +0200 3110)                  char sig_reply[128];
935f0c5cc2 (Matthias Welwarsky 2018-03-05 14:50:51 +0100 3111)
5f46de2e79 (MadSquirrel        2022-06-04 22:01:11 +0200 3112)                  LOG_DEBUG("fake step thread %"PRIx64, thread_id);
935f0c5cc2 (Matthias Welwarsky 2018-03-05 14:50:51 +0100 3113)
5f46de2e79 (MadSquirrel        2022-06-04 22:01:11 +0200 3114)                  sig_reply_len = snprintf(sig_reply, sizeof(sig_reply),
5f46de2e79 (MadSquirrel        2022-06-04 22:01:11 +0200 3115)                                                                  "T05thread:%016"PRIx64";", thread_id);
935f0c5cc2 (Matthias Welwarsky 2018-03-05 14:50:51 +0100 3116)
5f46de2e79 (MadSquirrel        2022-06-04 22:01:11 +0200 3117)                  gdb_put_packet(connection, sig_reply, sig_reply_len);
5f46de2e79 (MadSquirrel        2022-06-04 22:01:11 +0200 3118)                  gdb_connection->output_flag = GDB_OUTPUT_NO;
bae76053dc (Tomas Vanek        2018-02-23 19:27:28 +0100 3119)
5f46de2e79 (MadSquirrel        2022-06-04 22:01:11 +0200 3120)                  return true;
5f46de2e79 (MadSquirrel        2022-06-04 22:01:11 +0200 3121)          }

no target->rtos->current_threadid = thread_id; line.
So it seems like the line was introduced by Tim's conflict resolution. And I don't get why.

• src/server/server.h, src/target/image.c -- See 8118

• src/target/target.* -- 8118 and 6964

• src/target/breakpoints.c -- there is a significant difference. Perhaps I have resolved a conflict erroneously. Will investigate.

• src/target/breakpoints.h -- See 9175

• tools/filter_openocd_log.py -- this tool is unique to RISC-V OpenOCD.

Summary:

This is what needs to be dealt with to sync the codebases:

1. Tcl scripts.
2. The diff in src/target/breakpoints.c.
3. The mysterious line from src/server/gdb_server.c.
4. The diff related to FreeRTOS and RTOS stackings.

I believe (4) should not be part of this sync and should be dealt with separately.
I don't know how to approach (1). Any thoughts?

[en-sc]

Also I have a suggestion:
Perhaps we could split the sync in two, first merging everything related to RISC-V and then dealing with other changes?

I've tried to come up with the list of patches that I suggest to merge first

• https://review.openocd.org/c/openocd/+/8893
• https://review.openocd.org/c/openocd/+/8896
• https://review.openocd.org/c/openocd/+/9069
• https://review.openocd.org/c/openocd/+/9071
• https://review.openocd.org/c/openocd/+/9072
• https://review.openocd.org/c/openocd/+/9123
• https://review.openocd.org/c/openocd/+/9171
• https://review.openocd.org/c/openocd/+/9127
• https://review.openocd.org/c/openocd/+/9130
• https://review.openocd.org/c/openocd/+/9132
• https://review.openocd.org/c/openocd/+/9133
• https://review.openocd.org/c/openocd/+/9135
• https://review.openocd.org/c/openocd/+/9138
• https://review.openocd.org/c/openocd/+/9141
• https://review.openocd.org/c/openocd/+/9142
• https://review.openocd.org/c/openocd/+/9143
• https://review.openocd.org/c/openocd/+/9144
• https://review.openocd.org/c/openocd/+/9163
• https://review.openocd.org/c/openocd/+/9164
• https://review.openocd.org/c/openocd/+/9165

This will make RISC-V support in mainline OpenOCD on par with the fork and will allow us to freeze the development of the fork.

The thing is: other patches, related to SMP/RTOS hwthread/unavailable targets seem quite complex.

As for testing with riscv-tests on Spike I would suggest to target spike32/64 at this stage, disregarding failures of multiprocessor targets for now.

[tom-van]

Ad 1. Tcl scripts:
Recently lot of target/board configs were moved to vendor folders on the upstream repo. Better compare to commit b1c1dd1 (before moving started)

[borneoa]

From a quick check, the scripts in this fork that need to go upstream are only:

tcl/target/esp32c2.cfg
tcl/target/esp32c3.cfg
tcl/target/esp32c6.cfg
tcl/target/esp32h2.cfg
tcl/target/esp_common.cfg
tcl/target/gd32vf103.cfg

tcl/board/esp32c2-ftdi.cfg
tcl/board/esp32c3-builtin.cfg
tcl/board/esp32c3-ftdi.cfg
tcl/board/esp32c6-builtin.cfg
tcl/board/esp32h2-builtin.cfg

[tom-van]

From a quick check, the scripts in this fork that need to go upstream are only:

tcl/target/gd32vf103.cfg

I believe the upstream gd32vf103.cfg is the most recent, the riscv-collab version is screwed after bad merge from upstream.

[en-sc]

@tom-van, I've tried to fix the diff in #1302