bluetooth: shell: refactor shell print for Bluetooth-specific context #74652

ndrs-pst · 2024-06-21T08:51:11Z

This PR aims to eliminate the dependency on ctx_shell in the Bluetooth host/shell/*, making the code more maintainable.

Introduced bt_shell_private.c and bt_shell_private.h to provide common functions for the Bluetooth shell.
Limit the usage of ctx_shell to cases where printing requires it and sh is not available.
(Interim commit before switch to bt_shell_*)
Refactor shell print to eliminate ctx_shell usage in the Bluetooth host/shell/*.

Fixes #41796

Thalley

I don't think this is the right direction (but please do convince me if you believe so).

Some of the changes are welcome, but the idea of a bt_shell is IMO not.

In your commit message you mention that you want to remove the code footprint, but you also mention an increase cost of this new bt_shell. Aren't those two sentences conflicting?

The BT shell definitely needs improvement. I created a issue #70945 related to this, that requests the removed of the ctx_shell. The removal of the ctx_shell would directly conflict with the majority of this PR, as all the bt_shell functions you've defined will simply call the shell functions directly (as they would need the shell pointer again).

Could you provide some data on the codesize that you've increase/decreased?

ndrs-pst · 2024-06-22T09:34:44Z

@Thalley Thank you for your feedback. It's good to hear that you're addressing ctx_shell. 😃

Aren't those two sentences conflicting?

At first glance, this might seem controversial.
Currently, many shell print require passing parameters like sh and color repeatedly.
By centralizing these interactions within specific bt_shell functions, we streamline the code.

Could you provide some data on the codesize that you've increase/decreased?

By picking up periodic_adv_conn sample with nrf52840dk boards and adding CONFIG_BT_SHELL=y
the footprint of shell\* which get from rom_report shown that

MODULE	FLASH SIZE (BEFORE)	FLASH SIZE (AFTER)	INCREASE/DECREASE
`bluetooth\shell\*`	15,795 B	14,963 B	-832 B
`bt.c`	13,251 B	12,275 B	-976 B
`bt_shell_private.c`	0 B	144 B	+144 B

Here is the footprint from each wrapper

ndrs-pst · 2024-06-22T09:46:20Z

To stretch further, I think the functions that should be improved are
shell_fprintf_impl(sh, color, fmt, ##__VA_ARGS__) itself to have a wrapper for the color parameter.

void __printf_like(2, 3) shell_fprintf_error_impl(const struct shell *sh, const char *fmt, ...);
void __printf_like(2, 3) shell_fprintf_info_impl(const struct shell *sh, const char *fmt, ...);
void __printf_like(2, 3) shell_fprintf_print_impl(const struct shell *sh, const char *fmt, ...);
void __printf_like(2, 3) shell_fprintf_warn_impl(const struct shell *sh, const char *fmt, ...);

Since in reality the same message printed out inside the shell is always fixed to a specific color, and if we prioritize the code size, this should make sense.

Otherwise, every time, for example in Cortex-M, r1 needs to be allocated to pass the color value (while r0 is for sh), which causes the caller code to save the previously used r1.

Thalley · 2024-06-28T09:18:14Z

Just tried your PR with the BT shell test application for the nrf5340dk_nrf5340_cpuapp board and got the following results:

With PR

Memory region         Used Size  Region Size  %age Used
           FLASH:      317356 B         1 MB     30.27%
             RAM:       52992 B       448 KB     11.55%
        IDT_LIST:          0 GB        32 KB      0.00%

Without PR

Memory region         Used Size  Region Size  %age Used
           FLASH:      318668 B         1 MB     30.39%
             RAM:       52992 B       448 KB     11.55%
        IDT_LIST:          0 GB        32 KB      0.00%

So while I do not agree with the direction of this PR, as we really need to get rid of ctx_shell, I can't disagree with the 1312 Bytes saved.

I'm not at a point where I'll approve, but I'm also not going to reject this change. I'd like to get input from the other BT collaborators on this.

Not sure yet

ndrs-pst · 2024-06-28T09:32:00Z

@Thalley, thank you for the additional information. It seems we should start by getting rid of ctx_shell and then find a way to reduce the footprint.

ndrs-pst · 2024-07-28T07:20:56Z

To stretch further, I think the functions that should be improved are shell_fprintf_impl(sh, color, fmt, ##__VA_ARGS__) itself to have a wrapper for the color parameter.
void __printf_like(2, 3) shell_fprintf_error_impl(const struct shell *sh, const char *fmt, ...);
void __printf_like(2, 3) shell_fprintf_info_impl(const struct shell *sh, const char *fmt, ...);
void __printf_like(2, 3) shell_fprintf_print_impl(const struct shell *sh, const char *fmt, ...);
void __printf_like(2, 3) shell_fprintf_warn_impl(const struct shell *sh, const char *fmt, ...);
Since in reality the same message printed out inside the shell is always fixed to a specific color, and if we prioritize the code size, this should make sense.

Otherwise, every time, for example in Cortex-M, r1 needs to be allocated to pass the color value (while r0 is for sh), which causes the caller code to save the previously used r1.

@Thalley Just a reminder that this stretch was merged in this PR: #75340. 👀

github-actions · 2024-09-27T00:32:42Z

This pull request has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this pull request will automatically be closed in 14 days. Note, that you can always re-open a closed pull request at any time.

ndrs-pst · 2024-10-14T19:47:29Z

@Thalley I just added another commit that introduces CONFIG_BT_SHELL_BACKEND for Bluetooth shell output when the shell context is unavailable (e.g., in callbacks).

While exploring ways to eliminate ctx_shell, could we initialize it during startup and hide it from other shell commands?
Since ctx_shell is currently used in multiple places like audio/shell, mesh/shell, and services/ias/shell.

subsys/bluetooth/host/shell/bt.c

ndrs-pst · 2024-11-15T13:58:57Z

Changes made as requested—hope this looks good!

Thalley · 2024-11-15T14:22:39Z

Excellent job and great that you updated it so fast :)

jhedberg · 2024-11-15T20:34:11Z

subsys/bluetooth/host/shell/bt_shell_private.c

Is there anything really Bluetooth specific in this c-file? Wouldn't other modules benefit from having a "wall" API? Seems to me like this should just be a generic API instead of a Bluetooth specific one.

It's true that this C-file isn't Bluetooth-specific, but since it's designed to address a Bluetooth-shell problem, keeping it focused on Bluetooth is fine for now. If similar needs arise elsewhere, it can be refactored into a generic API later.

🤞 Once this is merged, I'll open another PR for audio/shell, mesh/shell, and classic/shell too.

@jakub-uC @carlescufi any comments on this? How easily could this generic API be accepted as part of the shell subsystem?

@jakub-uC @carlescufi I'm still waiting for some response to the above question.

Since the ctx_shell is only used in BT context, I think we can move forward with this PR. We can later port these to a non-BT place if there's any use for it

Thalley · 2024-12-04T14:47:55Z

@ndrs-pst I think this PR actually solves #41796 - Should we add fixes https://github.com/zephyrproject-rtos/zephyr/issues/41796 in the PR description here?

jhedberg

Some style comments. I'm still a bit concerned with the total silence from the shell maintainers.

jhedberg · 2024-12-04T15:21:34Z

subsys/bluetooth/host/shell/bt.c

Can't this be just one line?

jhedberg · 2024-12-04T15:21:58Z

subsys/bluetooth/host/shell/bt.c

Merge to one line?

jhedberg · 2024-12-04T15:22:33Z

subsys/bluetooth/host/shell/bt.c

Seems like at least the string literal should be mergeable to the first line

jhedberg · 2024-12-04T15:22:42Z

subsys/bluetooth/host/shell/bt.c

jhedberg · 2024-12-04T15:25:23Z

subsys/bluetooth/host/shell/bt.c

One line perhaps? I'll stop pointing more of this out, but in general, please pay attention to when you're shortening a line that was previously split if there's the possibility to merge the next line into it. I realize this is some extra work since you probably did an automated search-and-replace, however if we don't do it then the code will gradually start looking quite odd.

Hmm, the way that it's formatted looks like clang-format, but where the line length is set to 80 instead of 100. Chances are that the author is using a misconfigured auto-formatter

As far as I can recall, I used the in-tree .clang-format with some manual adjustments to align it with the previous one.
For the next push, I'll change it according to the clang-format suggestion.

ndrs-pst · 2024-12-04T16:25:24Z

@ndrs-pst I think this PR actually solves #41796 - Should we add fixes https://github.com/zephyrproject-rtos/zephyr/issues/41796 in the PR description here?

Got it.

Introduced `bt_shell_private.c` and `bt_shell_private.h` to provide common functions for the Bluetooth `shell_wall_print`. These functions are equivalent to `shell_fprintf`, `shell_info`, `shell_print`, `shell_warn`, `shell_error` and `shell_hexdump` but without requiring the `sh` parameter. The cost of the newly added `bt_shell_fprintf_info` ... `_error` functions will be negligible when there are many individual calls that need to pass both the `sh` and `color` parameters each time. Signed-off-by: Pisit Sawangvonganan <[email protected]>

Limit the usage of `ctx_shell` to cases where printing requires it and `sh` is not available. Signed-off-by: Pisit Sawangvonganan <[email protected]>

This change aims to eliminate the dependency on `ctx_shell` in the Bluetooth `host/shell/*`, making the code more maintainable. Replaced `shell_*` functions that depended on `ctx_shell` with the appropriate `bt_shell_*` functions. The shell-less functions `bt_do_scan_filter_clear_name`, `bt_do_scan_off`, and `bt_do_connect_le` were added so they can be called without `sh`. Signed-off-by: Pisit Sawangvonganan <[email protected]>

jhedberg

Approving, but there's potential cleanup still to be done in follow-up PRs (see my inline comments). Please also engage actively with the shell maintainers to make this a generic API eventually. Thanks.

jhedberg · 2024-12-04T18:32:33Z

subsys/bluetooth/host/shell/bt.c

+	bt_shell_print("LE conn  param req: int (0x%04x, 0x%04x) lat %d"
+		       " to %d", param->interval_min, param->interval_max,
+		       param->latency, param->timeout);

 	return true;
 }

 static void le_param_updated(struct bt_conn *conn, uint16_t interval,
 			     uint16_t latency, uint16_t timeout)
 {
-	shell_print(ctx_shell, "LE conn param updated: int 0x%04x lat %d "
-		     "to %d", interval, latency, timeout);
+	bt_shell_print("LE conn param updated: int 0x%04x lat %d "
+		       "to %d", interval, latency, timeout);


I don't mean to keep bikeshedding here, and in most cases line splitting details are really not that critical, but splitting format strings is something that should be avoided whenever possible. The reason is that splitting them makes it hard/impossible to grep the code for the location where something was outputted to the console, since what's in the console will not match any single line in the source code. I see the original code had the split already, but I'm pretty sure the string can now be merged into a single line in both of these cases.

Agreed. If the author has setup clang-format in their IDE with only formatting changed lines using the .clang-format from Zephyr, these would have been formatted like you want.

jhedberg · 2024-12-04T18:32:41Z

subsys/bluetooth/host/shell/bt.c

-		shell_print(ctx_shell, "Security failed: %s level %u "
-			    "reason: %s (%d)",
-			    addr, level, security_err_str(err), err);
+		bt_shell_print("Security failed: %s level %u "
+			       "reason: %s (%d)",
+			       addr, level, security_err_str(err), err);


jhedberg · 2024-12-04T18:32:54Z

subsys/bluetooth/host/shell/bt.c

+		bt_shell_print("Remote LMP version %s (0x%02x) subversion 0x%04x "
+			       "manufacturer 0x%04x", bt_hci_get_ver_str(remote_info->version),


jhedberg · 2024-12-04T18:33:02Z

subsys/bluetooth/host/shell/bt.c

+	bt_shell_print("LE data len updated: TX (len: %d time: %d)"
+		       " RX (len: %d time: %d)", info->tx_max_len,


ndrs-pst · 2024-12-05T15:49:46Z

Thank you for the approval and feedback!
I'll address the potential cleanup you mentioned in a follow-up PR.
Cheers 🎉

zephyrbot added area: Bluetooth Host Bluetooth Host (excluding BR/EDR) area: Bluetooth labels Jun 21, 2024

zephyrbot requested review from Thalley, Vudentz, alwa-nordic, asbjornsabo, hermabe, jhedberg, jori-nordic, sjanc and theob-pro June 21, 2024 08:51

zephyrbot assigned jhedberg Jun 21, 2024

ndrs-pst force-pushed the pr_bt_shell_refactor branch 2 times, most recently from 1f6268c to 204e867 Compare June 21, 2024 09:22

Thalley previously requested changes Jun 22, 2024

View reviewed changes

Thalley self-requested a review June 28, 2024 09:18

ndrs-pst marked this pull request as draft June 28, 2024 09:32

ndrs-pst mentioned this pull request Aug 16, 2024

drivers: can: shell: use shell_fprintf_normal instead of shell_fprintf #77165

Merged

github-actions bot added the Stale label Sep 27, 2024

ndrs-pst force-pushed the pr_bt_shell_refactor branch from 204e867 to 45075db Compare October 4, 2024 11:29

Thalley removed the Stale label Oct 4, 2024

ndrs-pst force-pushed the pr_bt_shell_refactor branch from 45075db to d3dffb7 Compare October 14, 2024 19:38

ndrs-pst dismissed alwa-nordic’s stale review via 5243ac9 November 15, 2024 11:52

ndrs-pst force-pushed the pr_bt_shell_refactor branch from 0bf9681 to 5243ac9 Compare November 15, 2024 11:52

Thalley requested changes Nov 15, 2024

View reviewed changes

subsys/bluetooth/host/shell/bt.c Outdated Show resolved Hide resolved

subsys/bluetooth/host/shell/bt.c Outdated Show resolved Hide resolved

ndrs-pst force-pushed the pr_bt_shell_refactor branch from 5243ac9 to 90f439f Compare November 15, 2024 13:52

Thalley previously approved these changes Nov 15, 2024

View reviewed changes

jhedberg reviewed Nov 15, 2024

View reviewed changes

alwa-nordic previously approved these changes Nov 16, 2024

View reviewed changes

Thalley mentioned this pull request Nov 26, 2024

usb: shell: Remove use of ctx_shell #81395

Merged

Thalley requested a review from jhedberg December 4, 2024 14:46

Thalley mentioned this pull request Dec 4, 2024

shell_print crashes the board on bt bonds and bt connections #41796

Closed

jhedberg reviewed Dec 4, 2024

View reviewed changes

ndrs-pst added 3 commits December 4, 2024 23:35

bluetooth: shell: replace ctx_shell with sh where applicable

274e7e5

Limit the usage of `ctx_shell` to cases where printing requires it and `sh` is not available. Signed-off-by: Pisit Sawangvonganan <[email protected]>

ndrs-pst dismissed stale reviews from alwa-nordic and Thalley via 15ed4ec December 4, 2024 17:47

ndrs-pst force-pushed the pr_bt_shell_refactor branch from 90f439f to 15ed4ec Compare December 4, 2024 17:47

Thalley approved these changes Dec 4, 2024

View reviewed changes

alwa-nordic approved these changes Dec 5, 2024

View reviewed changes

Thalley requested a review from jhedberg December 5, 2024 13:45

jhedberg approved these changes Dec 5, 2024

View reviewed changes

kartben merged commit aa1a38e into zephyrproject-rtos:main Dec 5, 2024
26 checks passed

ndrs-pst deleted the pr_bt_shell_refactor branch December 6, 2024 06:48

This was referenced Dec 7, 2024

bluetooth: shell: code formatting cleanup in host/shell #82684

Merged

bluetooth: shell: eliminate ctx_shell usage #82695

Merged

		bt_shell_print("Remote LMP version %s (0x%02x) subversion 0x%04x "
		"manufacturer 0x%04x", bt_hci_get_ver_str(remote_info->version),

		bt_shell_print("LE data len updated: TX (len: %d time: %d)"
		" RX (len: %d time: %d)", info->tx_max_len,

bluetooth: shell: refactor shell print for Bluetooth-specific context #74652

bluetooth: shell: refactor shell print for Bluetooth-specific context #74652

Uh oh!

Conversation

ndrs-pst commented Jun 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Thalley left a comment

Choose a reason for hiding this comment

Uh oh!

ndrs-pst commented Jun 22, 2024

Uh oh!

ndrs-pst commented Jun 22, 2024

Uh oh!

Thalley commented Jun 28, 2024

Uh oh!

ndrs-pst commented Jun 28, 2024

Uh oh!

ndrs-pst commented Jul 28, 2024

Uh oh!

github-actions bot commented Sep 27, 2024

Uh oh!

ndrs-pst commented Oct 14, 2024

Uh oh!

Uh oh!

Uh oh!

ndrs-pst commented Nov 15, 2024

Uh oh!

Thalley commented Nov 15, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ndrs-pst Nov 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Thalley commented Dec 4, 2024

Uh oh!

jhedberg left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ndrs-pst commented Dec 4, 2024

Uh oh!

jhedberg left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ndrs-pst commented Dec 5, 2024

Uh oh!

Uh oh!

Reviewers

ndrs-pst commented Jun 21, 2024 •

edited

Loading

ndrs-pst Nov 16, 2024 •

edited

Loading