[Bug] mq_open() / mq_unlink() build /dev/mqueue/<name> with rt_sprintf() into a fixed stack buffer, allowing stack overflow

[XueDugu]

RT-Thread Version

master, verified on origin/master at commit 25295501c0cc7181d6a541a867fdf7214879ddf8, first appear in v5.0.2 at commit d6adf6708f6e0666d040e553a81b24edcc92c374

Hardware Type/Architectures

Any architecture using components/libc/posix/ipc/mqueue.c

Develop Toolchain

GCC

Describe the bug

Summary

A stack-based buffer overflow exists in RT-Thread's POSIX message queue implementation.

mq_open() and mq_unlink() build the path string "/dev/mqueue/<name>" into a fixed-size stack buffer using unbounded rt_sprintf().

The buffer is declared as:

char mq_name[RT_NAME_MAX + 12];

However, the required size for the resulting string is:

• strlen("/dev/mqueue/") + strlen(name) + 1
• 12 + strlen(name) + 1

So the true safe condition is strlen(name) <= RT_NAME_MAX - 1. Current code does not enforce that condition.

---

Affected Locations

• components/libc/posix/ipc/mqueue.c:121
• components/libc/posix/ipc/mqueue.c:164
• components/libc/posix/ipc/mqueue.c:465

---

Vulnerability Details

1. mq_open() contains an off-by-one stack overflow

mq_open() checks:

if (len > RT_NAME_MAX) {
    rt_set_errno(ENAMETOOLONG);
    return (mqd_t)(-1);
}

But it later uses:

rt_sprintf(mq_name, "%s%s", mq_path, name);

with:

char mq_name[RT_NAME_MAX + 12];

Since the prefix "/dev/mqueue/" is 12 bytes and sprintf also writes the terminating NUL byte, len == RT_NAME_MAX already overflows the stack buffer by 1 byte. The current guard is insufficient.

2. mq_unlink() contains an unbounded stack overflow

mq_unlink() uses the same fixed-size stack buffer and the same rt_sprintf() path construction logic, but does not perform an equivalent length check first.

A long name can therefore overflow the stack buffer by more than 1 byte — potentially by an arbitrarily large amount depending on the input length.

---

Reachability

This is not a remote issue. However, it is practically reachable by local callers.

In systems using LWP/MMU, the syscall layer copies the user-provided message queue name into kernel memory and then invokes mq_open() / mq_unlink() without imposing a stricter length limit that would prevent this bug.

Relevant reachability points:

• lwp_syscall.c:9057
• lwp_syscall.c:9122

An untrusted local userspace process can supply an oversized queue name and trigger these kernel-side stack overflows.

---

Impact

• Local kernel stack overflow
• Local denial of service
• Possible local privilege escalation, depending on platform, compiler, calling convention, stack protector configuration, and surrounding memory layout

If the product does not run untrusted local processes, this may be treated as a severe robustness bug. If the product uses LWP userspace isolation and executes untrusted local code, this is a real local security vulnerability.

---

Steps to Reproduce

mq_open() off-by-one case

1. Build RT-Thread with POSIX mqueue support enabled.

2. Call mq_open() with a queue name whose length is exactly RT_NAME_MAX.

3. Ensure the path construction reaches:

   rt_sprintf(mq_name, "%s%s", mq_path, name);

4. Observe that the fixed stack buffer is overrun by 1 byte.

mq_unlink() unbounded case

1. Build RT-Thread with POSIX mqueue support enabled.
2. Call mq_unlink() with a very long queue name.
3. Observe that the same fixed stack buffer is used with rt_sprintf() and no sufficient length guard.
4. The stack buffer can be overflowed by a larger attacker-controlled amount.

---

Expected Behavior

RT-Thread should safely construct the queue path without overflowing the stack buffer.

At minimum:

• Use bounded formatting, e.g. rt_snprintf(mq_name, sizeof(mq_name), "%s%s", mq_path, name)
• Reject names that cannot fit including the prefix and terminating NUL
• Treat len >= RT_NAME_MAX as too long in the current buffer sizing scheme
• Apply the same validation to both mq_open() and mq_unlink()

---

Actual Behavior

• mq_open() allows len == RT_NAME_MAX, which causes a deterministic 1-byte stack overflow
• mq_unlink() uses the same fixed stack buffer and unbounded rt_sprintf() without adequate length validation, allowing larger stack overflows

---

Suggested Fix

1. Replace unbounded rt_sprintf() with bounded variant

// Before
rt_sprintf(mq_name, "%s%s", mq_path, name);

// After
rt_snprintf(mq_name, sizeof(mq_name), "%s%s", mq_path, name);

2. Fix the off-by-one in mq_open() length check

// Before
if (len > RT_NAME_MAX)

// After
if (len >= RT_NAME_MAX)

3. Add equivalent length validation to mq_unlink()

Apply the same length check before formatting the path, consistent with the corrected mq_open() guard.

Kindly let me know if you intend to request a CVE ID upon confirmation of the vulnerability.

Other additional context

No response