gdb: support zero inode in generate-core-file command

It is possible, when creating a shared memory segment (i.e. with
shmget), that the id of the segment will be zero.

When looking at the segment in /proc/PID/smaps, the inode field of the
entry holds the shared memory segment id.

And so, it can be the case that an entry (in the smaps file) will have
an inode of zero.

When GDB generates a core file, with the generate-core-file (or its
gcore alias) command, the shared memory segment should be written into
the core file.

Fedora GDB has, since 2008, carried a patch that tests this case.
There is no fix for GDB associated with the test, and unfortunately,
the motivation for the test has been lost to the mists of time.  This
likely means that a fix was merged upstream without a suitable test,
but I've not been able to find and relevant commit.  The test seems to
be checking that the shared memory segment with id zero, is being
written to the core file.

While looking at this test and trying to work out if it should be
posted upstream, I saw that GDB does appear to write the shared memory
segment into the core file (as expected), which is good.  However, GDB
still isn't getting this case exactly right, there appears to be no
NT_FILE entry for the shared memory mapping if the mapping had an id
of zero.

In gcore_memory_sections (gcore.c) we call back into linux-tdep.c (via
the gdbarch_find_memory_regions call) to correctly write the shared
memory segment into the core file, however, in
linux_make_mappings_corefile_notes, when we use
linux_find_memory_regions_full to create the NT_FILE note, we call
back in to dump_note_entry_p for each mapping, and in here we reject
any mapping with a zero inode.

The result of this, is that, for a shared memory segment with a
non-zero id, after loading the core file, the shared memory segment
will appear in the 'proc info mappings' output.  But, for a shared
memory segment with a zero id, the segment will not appear in the
'proc info mappings' output.

I initially tried just dropping the inode check in this function (see
previous commit 1e21c84, which I then reverted in commit
998165b.

The problem with dropping the inode check is that the special kernel
mappings, e.g. '[vvar]' would now get a NT_FILE entry.  In fact, any
special entry except '[vdso]' and '[vsyscall]' which are specifically
checked for in dump_note_entry_p would get a NT_FILE entry, which is
not correct.

So, instead, I propose that if the inode is zero, and the filename
starts with '[' and finished with ']' then we should not create a
NT_FILE entry.  But otherwise a zero inode should not prevent a
NT_FILE entry being created.

The test for this change is a bit tricky.  The original Fedora
test (mentioned above) has a loop that tries to grab the shared memory
mapping with id zero.  This was, unfortunately, not very reliable.

I tried to make this more reliable by going multi-threaded, and
waiting for longer, see my proposal here:

  https://inbox.sourceware.org/gdb-patches/0d389b435cbb0924335adbc9eba6cf30b4a2c4ee.1741776651.git.aburgess@redhat.com

But this was still not great.  On further testing this was only
passing (i.e. managing to find the shared memory mapping with id zero)
about 60% of the time.

However, I realised that GDB finds the shared memory id by reading the
/proc/PID/smaps file.  But we don't really _need_ the shared memory id
for anything, we just use the value (as an inode) to decide if the
segment should be included in the core file or not.  The id isn't even
written to the core file.  So, if we could intercept the read of the
smaps file, then maybe, we could lie to GDB, and tell it that the id
was zero, and then see how GDB handles this.

And luckily, we can do that using a preload library!

We already have a test that uses a preload library to modify GDB, see
gdb.threads/attach-slow-waitpid.exp.

So, I have created a new preload library.  This one intercepts open,
open64, close, read, and pread.  When GDB attempts to open
/proc/PID/smaps, the library spots this and loads the file contents
into a memory buffer.  The buffer is then modified to change the id of
any shared memory mapping to zero.  Any reads from this file are
served from the modified memory buffer.

I tested on x86-64, AArch64, PPC, s390, and ARM, all running various
versions of GNU/Linux.  The requirement for open64() came from my ARM
testing.  The other targets used plain open().

And so, the test is now simple.  Start GDB with the preload library in
place, start the inferior and generate a core file.  Then restart GDB,
load the core file, and check the shared memory mapping was included.
This test will fail with an unpatched GDB, and succeed with the patch
applied.

Tested-By: Guinevere Larsen <[email protected]>

Author: T-J-Teru

gdb/linux-tdep.c

@@ -805,13 +805,12 @@ dump_note_entry_p (filter_flags filterflags, const smaps_data &map)
   if (map.filename.length () == 0)
     return false;
 
-  /* Don't add NT_FILE entries for mappings with a zero inode.  */
-  if (map.inode == 0)
-    return false;
-
-  /* vDSO and vsyscall mappings will end up in the core file.  Don't
-     put them in the NT_FILE note.  */
-  if (map.filename == "[vdso]" || map.filename == "[vsyscall]")
+  /* Special kernel mappings, those with names like '[vdso]' and
+     '[vsyscall]' will be placed in the core file, but shouldn't get an
+     NT_FILE entry.  These special mappings all have a zero inode.  */
+  if (map.inode == 0
+      && map.filename.front () == '['
+      && map.filename.back () == ']')
     return false;
 
   /* Otherwise, any other file-based mapping should be placed in the