Skip to content

Add columns for common arguments, descriptions #563

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Add columns for common arguments, descriptions #563

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9c62d73
Add columns for common arguments, descriptions
voetberg May 19, 2025
7a392c5
Apply suggestions from code review
voetberg May 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 88 additions & 40 deletions docs/started/main_components/daemons.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,47 +10,47 @@ and then write some data back into the DB.

Usually one daemon will create some work for another daemon and vice-versa.
In Rucio realm, daemons communicate to others by the DB.
The following table represents a high level view of the responsibility of each of the daemons.
The following table represents a high level view of the responsibility of each of the daemons and basic functionality.

## Daemons

Name | Domain| Purpose | Details
-----------| ----------------- | -------| ----- |
rucio-abacus-account | Accounting | Account usage | [Details](bin/rucio-abacus-account.md)
rucio-abacus-collection-replica | Accounting | Updates collection replicas | [Details](bin/rucio-abacus-collection-replica.md)
rucio-abacus-rse | Accounting | Updates RSE counters | [Details](bin/rucio-abacus-rse.md)
rucio-atropos | Replica | End the life of the rules according to the Lifetime Model | [Details](bin/rucio-atropos.md)
rucio-auditor | Replica | Find inconsistencies on storage, for example, dark data discovery | [Details](bin/rucio-auditor.md)
rucio-automatix | Replica | Used for testing: injects random data in RSEs to check liveness | [Details](bin/rucio-automatix.md)
rucio-bb8 | Replica | Rebalance data across RSEs | [Details](bin/rucio-bb8.md)
rucio-cache-client | Replica | Populates information of replicas on volatile storage | [Details](bin/rucio-cache-client.md)
rucio-cache-consumer | Replica | Adds and deletes cache replicas to the Rucio catalog | [Details](bin/rucio-cache-consumer.md)
rucio-conveyor-finisher | Transfer | Updates Rucio internal state after the file transfer has finished | [Details](bin/rucio-conveyor-finisher.md)
rucio-conveyor-poller | Transfer | Polls updates from the transfer tool to check the transfer state | [Details](bin/rucio-conveyor-poller.md)
rucio-conveyor-preparer | Transfer | Prepares data transfers | [Details](bin/rucio-conveyor-preparer.md)
rucio-conveyor-receiver | Transfer | Sister of poller, instead of polling for updates, it reads transfer tools notifications to check transfer state | [Details](bin/rucio-conveyor-receiver.md)
rucio-conveyor-stager | Transfer | Issues staging (bring online) requests to tape RSEs | [Details](bin/rucio-conveyor-stager.md)
rucio-conveyor-submitter | Transfer | Submit transfer requests to the transfer tool (prepares the transfer as well if the conveyor-preparer is not enabled) | [Details](bin/rucio-conveyor-submitter.md)
rucio-conveyor-throttler | Transfer | Queues transfer requests inside Rucio, applying limits, ex: only one transfer at a time, etc ... | [Details](bin/rucio-conveyor-throttler.md)
rucio-dark-reaper | Deletion | Deletes quarantined replicas | [Details](bin/rucio-dark-reaper.md)
rucio-dumper | Consistency | Dumps file lists. The rucio-auditor consumes these dumps to discover dark data | [Details](bin/rucio-dumper.md)
rucio-follower | Telemetry | Aggregates events affecting DIDs | [Details](bin/rucio-follower.md)
rucio-hermes | Telemetry | Sends Rucio messages to external services (InfluxDB, OpenSearch, ActiveMQ, ...) | [Details](bin/rucio-hermes.md)
rucio-judge-cleaner | Rule | Cleans expired replication rules | [Details](bin/rucio-judge-cleaner.md)
rucio-judge-evaluator | Rule | Creates and evaluates replication rules based on their state (OK/REPL/STUCK) | [Details](bin/rucio-judge-evaluator.md)
rucio-judge-injector | Rule | Asynchronously injects replication rules | [Details](bin/rucio-judge-injector.md)
rucio-judge-repairer | Rule | Repairs stuck replication rules (STATE=STUCK) | [Details](bin/rucio-judge-repairer.md)
rucio-kronos | Telemetry | Consumes Rucio tracing messages, updates access time of replicas and access count of DIDs | [Details](bin/rucio-kronos.md)
rucio-minos | Replica | Reads list of physical file names (PFNs) declared bad and classifies them in: temporary unavailable and permanently unavailable (to be recovered by the necromancer daemon) | [Details](bin/rucio-minos.md)
rucio-minos-temporary-expiration | Replica | Moves back TEMPORARY_UNAVAILABLE replicas into AVAILABLE state | [Details](bin/rucio-minos-temporary-expiration.md)
rucio-necromancer | Deletion | Works on permanently unavailable replicas, it tries to recover the data from other valid replicas if any, else declares the replica as lost | [Details](bin/rucio-necromancer.md)
rucio-oauth-manager | Auth/Authz | Deletes expired access tokens (in case there is a valid refresh token, expired access tokens will be kept until refresh_token expires as well.) and deletion of expired OAuth session parameters | [Details](bin/rucio-oauth-manager.md)
rucio-reaper | Deletion | Deletes replicas that don't have locks anymore, i.e. they have a tombstone set | [Details](bin/rucio-reaper.md)
rucio-suspicious-replica-recoverer | Replica | Declares suspicious replicas as bad if they are found available on other RSEs, so necromancer will work on them | [Details](bin/rucio-replica-recoverer.md)
rucio-rse-decommissioner | Deletion | Decommissions an RSE. The actions to perform are specified in decommissioning profiles (delete all data, move replicas, etc ...) | [Details](bin/rucio-rse-decommissioner.md)
rucio-storage-consistency-actions | Consistency | Applies corrective actions as a result of a consistency check on an RSE | [Details](bin/rucio-storage-consistency-actions.md)
rucio-transmogrifier | Rule | Creates replication rules for DIDs matching a subscription | [Details](bin/rucio-transmogrifier.md)
rucio-undertaker | Deletion | Manages expired DIDs, deleting them (does not delete replicas) | [Details](bin/rucio-undertaker.md)
Name | Domain| Purpose | Batched? | Threaded? | Multi-VO? | Details
-----------| ----------------- | -------| ----- | ----- | ----- | ----- |
rucio-abacus-account | Accounting | Account usage | ❌ | ✅ | ❌ | [Details](bin/rucio-abacus-account.md)
rucio-abacus-collection-replica | Accounting | Updates collection replicas | ❌ | ✅ | ❌ | [Details](bin/rucio-abacus-collection-replica.md)
rucio-abacus-rse | Accounting | Updates RSE counters | ❌ | ✅ | ❌ | [Details](bin/rucio-abacus-rse.md)
rucio-atropos | Replica | End the life of the rules according to the Lifetime Model | ❌ | ✅ | ❌ | [Details](bin/rucio-atropos.md)
rucio-auditor | Replica | Find inconsistencies on storage, for example, dark data discovery | ❌ | ✅ | ❌ | [Details](bin/rucio-auditor.md)
rucio-automatix | Replica | Used for testing: injects random data in RSEs to check liveness | ❌ | ✅ | ❌ | [Details](bin/rucio-automatix.md)
rucio-bb8 | Replica | Rebalance data across RSEs | ❌ | ✅ | ❌ | [Details](bin/rucio-bb8.md)
rucio-cache-client | Replica | Populates information of replicas on volatile storage | ❌ | ❌ | ❌ | [Details](bin/rucio-cache-client.md)
rucio-cache-consumer | Replica | Adds and deletes cache replicas to the Rucio catalog | ❌ | ✅ | ❌ | [Details](bin/rucio-cache-consumer.md)
rucio-conveyor-finisher | Transfer | Updates Rucio internal state after the file transfer has finished | ✅ | ✅ | ✅ | [Details](bin/rucio-conveyor-finisher.md)
rucio-conveyor-poller | Transfer | Polls updates from the transfer tool to check the transfer state | ✅ | ✅ | ❌ | [Details](bin/rucio-conveyor-poller.md)
rucio-conveyor-preparer | Transfer | Prepares data transfers | ✅ | ✅ | ❌ | [Details](bin/rucio-conveyor-preparer.md)
rucio-conveyor-receiver | Transfer | Sister of poller, instead of polling for updates, it reads transfer tools notifications to check transfer state | ❌ | ✅ | ❌ | [Details](bin/rucio-conveyor-receiver.md)
rucio-conveyor-stager | Transfer | Issues staging (bring online) requests to tape RSEs | ✅ | ✅ | ✅ | [Details](bin/rucio-conveyor-stager.md)
rucio-conveyor-submitter | Transfer | Submit transfer requests to the transfer tool (prepares the transfer as well if the conveyor-preparer is not enabled) | ✅ | ✅ | ✅ | [Details](bin/rucio-conveyor-submitter.md)
rucio-conveyor-throttler | Transfer | Queues transfer requests inside Rucio, applying limits, ex: only one transfer at a time, etc ... | ❌ | ❌ | ❌ | [Details](bin/rucio-conveyor-throttler.md)
rucio-dark-reaper | Deletion | Deletes quarantined replicas | ✅ | ✅ | ✅ | [Details](bin/rucio-dark-reaper.md)
rucio-dumper | Consistency | Dumps file lists. The rucio-auditor consumes these dumps to discover dark data | ❌ | ❌ | ❌ | [Details](bin/rucio-dumper.md)
rucio-follower | Telemetry | Aggregates events affecting DIDs | ❌ | ✅ | ❌ | [Details](bin/rucio-follower.md)
rucio-hermes | Telemetry | Sends Rucio messages to external services (InfluxDB, OpenSearch, ActiveMQ, ...) | ✅ | ✅ | ❌ | [Details](bin/rucio-hermes.md)
rucio-judge-cleaner | Rule | Cleans expired replication rules | ❌ | ✅ | ❌ | [Details](bin/rucio-judge-cleaner.md)
rucio-judge-evaluator | Rule | Creates and evaluates replication rules based on their state (OK/REPL/STUCK) | ❌ | ✅ | ❌ | [Details](bin/rucio-judge-evaluator.md)
rucio-judge-injector | Rule | Asynchronously injects replication rules | ❌ | ✅ | ❌ | [Details](bin/rucio-judge-injector.md)
rucio-judge-repairer | Rule | Repairs stuck replication rules (STATE=STUCK) | ❌ | ✅ | ❌ | [Details](bin/rucio-judge-repairer.md)
rucio-kronos | Telemetry | Consumes Rucio tracing messages, updates access time of replicas and access count of DIDs | ❌ | ✅ | ❌ | [Details](bin/rucio-kronos.md)
rucio-minos | Replica | Reads list of physical file names (PFNs) declared bad and classifies them in: temporary unavailable and permanently unavailable (to be recovered by the necromancer daemon) | ✅ | ✅ | ❌ | [Details](bin/rucio-minos.md)
rucio-minos-temporary-expiration | Replica | Moves back TEMPORARY_UNAVAILABLE replicas into AVAILABLE state | ✅ | ✅ | ❌ | [Details](bin/rucio-minos-temporary-expiration.md)
rucio-necromancer | Deletion | Works on permanently unavailable replicas, it tries to recover the data from other valid replicas if any, else declares the replica as lost | ✅ | ✅ | ❌ | [Details](bin/rucio-necromancer.md)
rucio-oauth-manager | Auth/Authz | Deletes expired access tokens (in case there is a valid refresh token, expired access tokens will be kept until refresh_token expires as well.) and deletion of expired OAuth session parameters | ✅ | ✅ | ❌ | [Details](bin/rucio-oauth-manager.md)
rucio-reaper | Deletion | Deletes replicas that don't have locks anymore, i.e. they have a tombstone set | ✅ | ✅ | ✅ | [Details](bin/rucio-reaper.md)
rucio-suspicious-replica-recoverer | Replica | Declares suspicious replicas as bad if they are found available on other RSEs, so necromancer will work on them | ❌ | ❌ | ✅ | [Details](bin/rucio-replica-recoverer.md)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I notice the daemon name rucio-suspicious-replica-recoverer in the table doesn't match the actual executable rucio-replica-recoverer that exists in the codebase. Should we update the table to use the correct executable name, or is there a reason for this discrepancy?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be updated

rucio-rse-decommissioner | Deletion | Decommissions an RSE. The actions to perform are specified in decommissioning profiles (delete all data, move replicas, etc ...) | ❌ | ❌ | ❌ | [Details](bin/rucio-rse-decommissioner.md)
rucio-storage-consistency-actions | Consistency | Applies corrective actions as a result of a consistency check on an RSE | ❌ | ✅ | ❌ | [Details](bin/rucio-storage-consistency-actions.md)
rucio-transmogrifier | Rule | Creates replication rules for DIDs matching a subscription | ✅ | ✅ | ❌ | [Details](bin/rucio-transmogrifier.md)
rucio-undertaker | Deletion | Manages expired DIDs, deleting them (does not delete replicas) | ✅ | ✅ | ❌ | [Details](bin/rucio-undertaker.md)

## FAQ

Expand Down Expand Up @@ -80,7 +80,7 @@ Dark reaper is the one deleting this dark data to free up space from the quarant
When replicas are healthy, the judge-cleaner will set a tombstone on replicas where the lifetime has expired. These replicas are taken by the reaper and they are deleted.
Sometimes, replicas can become unhealthy. A dump is created by the dumper daemon. The auditor checks these dumps and declares replicas as suspicious.

## How is a replica declared bad?
### How is a replica declared bad?
- An operator can declare a replica as bad issuing rucio cli commands.
```
rucio-admin replicas declare-bad [-h] --reason REASON [--inputfile [INPUTFILE]] [--allow-collection] [--lfns [LFNS]] [--scope [SCOPE]] [--rse [RSE]] [listbadfiles ...]
Expand All @@ -90,6 +90,54 @@ These bad replicas are taken by the necromancer daemon and then deleted if they
- the suspicious-replica-recoverer is a daemon that will analyze different counters (transfer errors, download errors, etc ...) to mark replicas as being suspicious to have an issue. After hitting
a certain limit (configurable by `--nattempts` flag), the replica is marked as bad and eventually consumed by the necromancer.

## What is the purpose of undertaker?
### What is the purpose of undertaker?
A dataset is "never" deleted, however, when the dataset is known to be bad, there is no point having it in the catalog. The undertaker daemon takes care to remove these datasets.
An operator will set an expiration date in the past of the DIDs and this daemon will delete the dataset from the DB. If there were any replicas attached, the replicas will be deleted as well.

## Daemon arguments
A full description for each daemon's arguments can be found by running `rucio-{daemon} --help` or viewing the `Details` in the daemon description table above.
Listed below are common definitions between different daemons.

- **run-once** - Only run one iteration of the daemon. If executed with this argument the daemon will run once and close.
- **sleep-time** - How long a daemon will sleep between iterations, mutually exclusive with `run-once`. Units of seconds.
- **threads**, **total-workers**, **threads-per-process**, **nprocs** - _[Present in threaded daemons]_ Run in threaded mode.
- **bulk**, **chunk-size**, **max-rows** - _[Present in batched daemons]_ Provide a limit of the number of operations a single instance of a daemon can run in an iteration.
- **dry-run** - Run once, showing logs of the daemon's operations without performing any action. Useful for verifying settings of the instance and daemon.
- **vos** - _[Present in Multi-VO daemons]_ Provide a list of VOs with which the daemon can interact. Used when VOs use different settings for their daemons.

### Batched Daemons

Some daemons can run over large backlogs depending on the traffic of an instance.
To prevent the daemon from running too long, or submitting too many requests at once to an external system, a daemon's workload can be batched.
Between batches of work, the daemon sleeps for `sleep-time`.

This setting is present in daemons that either submit requests (e.g. transfer requests) or process multiple replicas or DIDs (e.g. setting statuses, running deletion).

### Threaded Daemons

When daemons are run with threaded arguments, the database query used by the daemons has a `threads` argument applied to the query string.
This is applicable to `oracle`, `postgres` and `mysql` databases.

> **_Note:_**
This definition doesn't apply for producer/consumer daemons. When producer/consumer daemons are run in threaded mode, it creates multiple instances of producers and consumers. This includes the threaded `conveyor` daemons.


### Multi-VO Daemons

Daemons that have a `vos` option can be set to have separate settings per vo running in a multi-vo instance.
When this is set, the daemon will only interact with objects that are explicitly included in the specific VO.

By default, a daemon with multi-VO options interact with all VOs on the instance.
For example, this is the log displayed by the `replica-recoverer` daemon.

```bash
$ rucio-replica-recoverer --run-once
2020-07-28 15:15:14,151 5461 INFO replica_recoverer: This instance will work on VOs: def, abc, xyz, 123


$ rucio-replica-recoverer --run-once --vos abc xyz
2020-07-28 15:16:36,066 5474 INFO replica_recoverer: This instance will work on VOs: abc, xyz
```

> **_Note:_**
Multi-VO daemons can still be used in single-VO instances, and the `vos` option does not need to be set.