L2ARC: unexpected change from label, to da1p1

[grahamperrin]

FreeBSD 14.0-CURRENT.

zfsd(8) https://man.freebsd.org/cgi/man.cgi?query=zfsd&sektion=8&apropos=0&manpath=FreeBSD%2014.0-current enabled.

When I added a cache device (s/n 60A44C426697AF20F000167F) to a pool, I used its label:

• gpt/cache-august.

---

To the best of my recollection, with reference to the attached file …

messages etc.txt

At 12:32 today, one of my cats closed the case of my notebook, just enough to take three of three cache devices offline. I opened the case a few seconds later.

Some time later, I realised that something wasn't right. As a possible workaround I decided to sleep then wake the computer. Logged:

Jun 17 13:10:31 mowa219-gjp4-8570p-freebsd acpi[96520]: suspend at 20230617 13:10:31
Jun 17 13:10:36 mowa219-gjp4-8570p-freebsd acpi[96570]: resumed at 20230617 13:10:36

There followed a problem with DRM (graphics).

Things looked ugly, around this time I chose to physically remove all three devices, a brief wait then I reconnected all three, in order:

1. gpt/cache-august
2. gpt/cache2-august
3. gpt/cache3-august.

I can't make sense of what's logged, I don't expect anyone else to make sense, but the end result was da1 instead of gpt/cache-august in output from zpool status -v.

I chose to shutdown (and reboot) at 13:35.

---

% zpool status -x
all pools are healthy
% lsblk
DEVICE         MAJ:MIN SIZE TYPE                                          LABEL MOUNT
ada0             0:123 932G GPT                                               - -
  ada0p1         0:125 260M efi                                    gpt/efiboot0 -
  <FREE>         -:-   1.0M -                                                 - -
  ada0p2         0:127  16G freebsd-swap                              gpt/swap0 SWAP
  ada0p2.eli     2:50   16G freebsd-swap                                      - SWAP
  ada0p3         0:129 915G freebsd-zfs                                gpt/zfs0 <ZFS>
  ada0p3.eli     0:136 915G -                                                 - -
  <FREE>         -:-   708K -                                                 - -
da0              0:154 7.2G GPT                                               - -
  <FREE>         -:-   1.0M -                                                 - -
  da0p1          0:155 7.2G freebsd-zfs                       gpt/cache3-august <ZFS>
  <FREE>         -:-   1.0M -                                                 - -
da1              0:172  14G GPT                                               - -
  <FREE>         -:-   1.0M -                                                 - -
  da1p1          0:173  14G freebsd-zfs                        gpt/cache-august <ZFS>
  <FREE>         -:-   1.0M -                                                 - -
da2              0:189  14G GPT                                               - -
  <FREE>         -:-   1.0M -                                                 - -
  da2p1          0:190  14G freebsd-zfs                       gpt/cache2-august <ZFS>
  <FREE>         -:-   1.0M -                                                 - -
% zpool status -v
  pool: august
 state: ONLINE
status: Some supported and requested features are not enabled on the pool.
        The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: scrub repaired 0B in 11:06:38 with 0 errors on Mon Jun 12 01:56:37 2023
config:

        NAME                 STATE     READ WRITE CKSUM
        august               ONLINE       0     0     0
          ada0p3.eli         ONLINE       0     0     0
        cache
          gpt/cache2-august  ONLINE       0     0     0
          da1p1              ONLINE       0     0     0
          gpt/cache3-august  ONLINE       0     0     0

errors: No known data errors
% 

Resolved, in single user mode, be removing then (again) adding the device:

Script started on Sat Jun 17 15:46:58 2023
# zpool status -v
  pool: august
 state: ONLINE
status: One or more devices has been taken offline by the administrator.
	Sufficient replicas exist for the pool to continue functioning in a
	degraded state.
action: Online the device using 'zpool online' or replace the device with
	'zpool replace'.
  scan: scrub repaired 0B in 11:06:38 with 0 errors on Mon Jun 12 01:56:37 2023
config:

	NAME                 STATE     READ WRITE CKSUM
	august               ONLINE       0     0     0
	  ada0p3.eli         ONLINE       0     0     0
	cache
	  gpt/cache2-august  ONLINE       0     0     0
	  da1p1              OFFLINE      0     0     0
	  gpt/cache3-august  ONLINE       0     0     0

errors: No known data errors
# zpool remove august da1p1
# zpool add august cache gpt/cache-august
# zpool status -v
  pool: august
 state: ONLINE
status: Some supported and requested features are not enabled on the pool.
	The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
	the pool may no longer be accessible by software that does not support
	the features. See zpool-features(7) for details.
  scan: scrub repaired 0B in 11:06:38 with 0 errors on Mon Jun 12 01:56:37 2023
config:

	NAME                 STATE     READ WRITE CKSUM
	august               ONLINE       0     0     0
	  ada0p3.eli         ONLINE       0     0     0
	cache
	  gpt/cache2-august  ONLINE       0     0     0
	  gpt/cache3-august  ONLINE       0     0     0
	  gpt/cache-august   ONLINE       0     0     0

errors: No known data errors
# exit

Script done on Sat Jun 17 15:48:19 2023