libbtrfsutil: don't leave subvolume iterator pointing at unfilled search buffer

subvolume_iterator_next_tree_search() sets nr_items to 4096 before
BTRFS_IOC_TREE_SEARCH and returns BTRFS_UTIL_ERROR_SEARCH_FAILED without
restoring it when the ioctl fails. items_pos and buf_off stay at 0, so the
next call to btrfs_util_subvolume_iterator_next() takes the 'items_pos <
nr_items' branch and parses btrfs_ioctl_search_header structures out of a
buffer the failed ioctl never filled.

Each phantom item advances buf_off by the header size plus the length read
from the uninitialized memory, so the walk quickly leaves the 3992 byte
buffer; with 4096 items it ends about 128 KB past its start. Depending on
what follows the buffer this either segfaults or spins on an unbounded
number of bogus items.

An unprivileged caller reaches this easily: use_tree_search is true whenever
top != 0, so BTRFS_UTIL_SUBVOLUME_ITERATOR_* users that pass a subvolume id
take the TREE_SEARCH path, the iterator is created successfully, and the
first ioctl then fails with EPERM.

Reset the entry on the error path so the search buffer is never parsed
before an ioctl has filled it.

Pull-request: #1159
Issue: #1158
Assisted-by: LLM
Signed-off-by: David Sterba <dsterba@suse.com>

Author: wehrwolfmann

libbtrfsutil/subvolume.c

@@ -1501,8 +1501,18 @@ static enum btrfs_util_error subvolume_iterator_next_tree_search(struct btrfs_ut
 			} else {
 				top->search.key.nr_items = 4096;
 				ret = ioctl(iter->fd, BTRFS_IOC_TREE_SEARCH, &top->search);
-				if (ret == -1)
+				if (ret == -1) {
+					/*
+					 * nr_items was set above but the ioctl
+					 * left buf untouched. Clear it so that
+					 * a caller which keeps iterating does
+					 * not walk the stale buffer.
+					 */
+					top->search.key.nr_items = 0;
+					top->items_pos = 0;
+					top->buf_off = 0;
 					return BTRFS_UTIL_ERROR_SEARCH_FAILED;
+				}
 				top->items_pos = 0;
 				top->buf_off = 0;