Skip to content

Commit b27b12a

Browse files
Merge pull request ceph#63659 from zdover23/wip-doc-2025-06-03-backport-63632-to-tentacle
tentacle: doc/dev/cephfs-mirroring: edit file 4 of x
2 parents 494a2a3 + 99bb1fc commit b27b12a

File tree

1 file changed

+68
-41
lines changed

1 file changed

+68
-41
lines changed

doc/dev/cephfs-mirroring.rst

Lines changed: 68 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -376,14 +376,19 @@ status:
376376
}
377377
}
378378

379-
`Peers` section in the command output above shows the peer information such as unique
380-
peer-id (UUID) and specification. The peer-id is required to remove an existing peer
381-
as mentioned in the `Mirror Module and Interface` section.
379+
The ``Peers`` section in the command output above shows the peer information
380+
such as unique peer-id (UUID) and specification. The peer-id is required to
381+
remove an existing peer as mentioned in the `Mirror Module and Interface`
382+
section.
382383

383-
Command with `fs mirror peer status` prefix provide peer synchronization status. This
384-
command is of format `filesystem-name@filesystem-id peer-uuid`::
384+
Commands with the ``fs mirror peer status`` prefix return peer synchronization
385+
status. Commands of this kind take the form ``filesystem-name@filesystem-id peer-uuid``, as in the following example:
386+
387+
.. prompt:: bash $
388+
389+
ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror peer status cephfs@360 a2dc7784-e7a1-4723-b103-03ee8d8768f8
390+
::
385391

386-
$ ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror peer status cephfs@360 a2dc7784-e7a1-4723-b103-03ee8d8768f8
387392
{
388393
"/d0": {
389394
"state": "idle",
@@ -399,27 +404,36 @@ command is of format `filesystem-name@filesystem-id peer-uuid`::
399404
}
400405
}
401406

402-
Synchronization stats such as `snaps_synced`, `snaps_deleted` and `snaps_renamed` are reset
403-
on daemon restart and/or when a directory is reassigned to another mirror daemon (when
404-
multiple mirror daemons are deployed).
407+
Synchronization stats such as ``snaps_synced``, ``snaps_deleted`` and
408+
``snaps_renamed`` are reset when the daemon is restarted or (when multiple
409+
mirror daemons are deployed), when a directory is reassigned to another mirror
410+
daemon.
411+
405412

406413
A directory can be in one of the following states::
407414

408415
- `idle`: The directory is currently not being synchronized
409416
- `syncing`: The directory is currently being synchronized
410417
- `failed`: The directory has hit upper limit of consecutive failures
411418

412-
When a directory hits a configured number of consecutive synchronization failures, the
413-
mirror daemon marks it as `failed`. Synchronization for these directories are retried.
414-
By default, the number of consecutive failures before a directory is marked as failed
415-
is controlled by `cephfs_mirror_max_consecutive_failures_per_directory` configuration
416-
option (default: 10) and the retry interval for failed directories is controlled via
417-
`cephfs_mirror_retry_failed_directories_interval` configuration option (default: 60s).
419+
When a directory hits a configured number of consecutive synchronization
420+
failures, the mirror daemon marks it as ``failed``. Synchronization for these
421+
directories is retried. By default, the number of consecutive failures before a
422+
directory is marked as failed is controlled by the
423+
``cephfs_mirror_max_consecutive_failures_per_directory`` configuration option
424+
(default: ``10``). The retry interval for failed directories is controlled by
425+
the ``cephfs_mirror_retry_failed_directories_interval`` configuration option
426+
(default: ``60s``).
427+
428+
For example, adding a regular file for synchronization results in a ``failed``
429+
status:
430+
431+
.. prompt:: bash $
418432

419-
E.g., adding a regular file for synchronization would result in failed status::
433+
ceph fs snapshot mirror add cephfs /f0
434+
ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror peer status cephfs@360 a2dc7784-e7a1-4723-b103-03ee8d8768f8
435+
::
420436

421-
$ ceph fs snapshot mirror add cephfs /f0
422-
$ ceph --admin-daemon /var/run/ceph/cephfs-mirror.asok fs mirror peer status cephfs@360 a2dc7784-e7a1-4723-b103-03ee8d8768f8
423437
{
424438
"/d0": {
425439
"state": "idle",
@@ -441,53 +455,66 @@ E.g., adding a regular file for synchronization would result in failed status::
441455
}
442456
}
443457

444-
This allows a user to add a non-existent directory for synchronization. The mirror daemon
445-
would mark the directory as failed and retry (less frequently). When the directory comes
446-
to existence, the mirror daemons would unmark the failed state upon successful snapshot
447-
synchronization.
458+
This allows a user to add a non-existent directory for synchronization. The
459+
mirror daemon marks the directory as failed and retries (less frequently).
460+
When the directory comes into existence, the mirror daemons notice the
461+
successful snapshot synchronization and unmark the failed state.
462+
463+
When mirroring is disabled, the ``fs mirror status`` command for the file
464+
system will not show up in command help.
448465

449-
When mirroring is disabled, the respective `fs mirror status` command for the file system
450-
will not show up in command help.
466+
The mirroring module provides a couple of commands to display directory mapping
467+
and distribution information. To check which mirror daemon a directory has been
468+
mapped to use, run a command of the following form:
451469

452-
Mirroring module provides a couple of commands to display directory mapping and distribution
453-
information. To check which mirror daemon a directory has been mapped to use::
470+
.. prompt:: bash $
471+
472+
ceph fs snapshot mirror dirmap cephfs /d0/d1/d2
473+
::
454474

455-
$ ceph fs snapshot mirror dirmap cephfs /d0/d1/d2
456475
{
457476
"instance_id": "404148",
458477
"last_shuffled": 1601284516.10986,
459478
"state": "mapped"
460479
}
461480

462-
.. note:: `instance_id` is the RADOS instance-id associated with a mirror daemon.
481+
.. note:: ``instance_id`` is the RADOS instance-id associated with a mirror
482+
daemon.
463483

464-
Other information such as `state` and `last_shuffled` are interesting when running
465-
multiple mirror daemons.
484+
Other information such as ``state`` and ``last_shuffled`` are interesting when
485+
running multiple mirror daemons.
466486

467-
When no mirror daemons are running the above command shows::
487+
If no mirror daemons are running, the same command shows the following:
488+
489+
.. prompt:: bash $
490+
491+
ceph fs snapshot mirror dirmap cephfs /d0/d1/d2
492+
::
468493

469-
$ ceph fs snapshot mirror dirmap cephfs /d0/d1/d2
470494
{
471495
"reason": "no mirror daemons running",
472496
"state": "stalled"
473497
}
474498

475-
Signifying that no mirror daemons are running and mirroring is stalled.
499+
This signifies that no mirror daemons are running and that mirroring is
500+
stalled.
476501

477502
Re-adding Peers
478503
---------------
479504

480-
When re-adding (reassigning) a peer to a file system in another cluster, ensure that
481-
all mirror daemons have stopped synchronization to the peer. This can be checked
482-
via `fs mirror status` admin socket command (the `Peer UUID` should not show up
483-
in the command output). Also, it is recommended to purge synchronized directories
484-
from the peer before re-adding it to another file system (especially those directories
485-
which might exist in the new primary file system). This is not required if re-adding
486-
a peer to the same primary file system it was earlier synchronized from.
505+
When re-adding (reassigning) a peer to a file system in another cluster, ensure
506+
that all mirror daemons have stopped synchronizing with the peer. This can be
507+
checked via the ``fs mirror status`` admin socket command (the ``Peer UUID``
508+
should not show up in the command output). We recommend purging
509+
synchronized directories from the peer before re-adding them to another file
510+
system (especially those directories which might exist in the new primary file
511+
system). This is not required if you re-add a peer to the same primary file
512+
system it was synchronized from before.
487513

488514
Feature Status
489515
--------------
490516

491-
`cephfs-mirror` daemon is built by default (follows `WITH_CEPHFS` CMake rule).
517+
The ``cephfs-mirror`` daemon is built by default. It follows the
518+
``WITH_CEPHFS`` CMake rule).
492519

493520
.. _CephFS Snapdiff Feature: https://croit.io/blog/cephfs-snapdiff-feature

0 commit comments

Comments
 (0)