|
| 1 | +--- |
| 2 | +id: transfers_poller_receiver |
| 3 | +title: Transfers Poller/Receiver |
| 4 | +--- |
| 5 | + |
| 6 | +`conveyor-poller` and `conveyor-receiver` are daemons responsible for tracking transfer status and updating it in the system. |
| 7 | +**You need to run one or both of them** depending on the transfer tool you are using: |
| 8 | + |
| 9 | +- If you are using only FTS3, it is **recommended to use the receiver** for scalability. Although poller also works. |
| 10 | +- If you are using only Globus and/or BitTorrent, you only need to run the poller. |
| 11 | +- If both FTS3 and Globus and/or BitTorrent, you need both receiver and poller. |
| 12 | + |
| 13 | +## Poller |
| 14 | + |
| 15 | +The `conveyor-poller` daemon periodically queries the transfer tools (such as FTS, Globus, or BitTorrent) directly to retrieve the status of ongoing transfers. |
| 16 | +Based on the results, it updates the state of each transfer in the system and/or refreshes the last access time. |
| 17 | +The transfer tool to query is taken from the transfer information internally and **no additional configuration is needed**. |
| 18 | + |
| 19 | +**If using Globus or BitTorrent as the transfer tool, you need to use the poller.** |
| 20 | + |
| 21 | +## Receiver |
| 22 | + |
| 23 | +The `conveyor-receiver` daemon subscribes to an ActiveMQ message queue and continuously listens for transfer status updates. |
| 24 | +When it receives a message, it processes the update, changes the transfer's state accordingly, and/or updates the last access time. |
| 25 | + |
| 26 | +To set up FTS to send transfer status updates to ActiveMQ broker, see the instructions [FTS3 Messaging Guide](https://fts3-docs.web.cern.ch/fts3-docs/docs/messaging.html). |
| 27 | + |
| 28 | +> **Note:** The receiver connects to ActiveMQ using the **STOMP protocol**, so ensure your ActiveMQ instance has STOMP support enabled. As of this writing (May 2025), CERN FTS uses [ActiveMQ-Classic](https://activemq.apache.org/components/classic/) and |
| 29 | +not tested with [ActiveMQ-Artemis](https://activemq.apache.org/components/artemis/). |
| 30 | + |
| 31 | +To configure `conveyor-receiver`, your rucio server config i.e. `rucio.cfg`, include the following section with described options: |
| 32 | + |
| 33 | +```cfg |
| 34 | +[messaging-fts3] |
| 35 | +# Set to True to use SSL with certificate/key authentication, or False to use username/password |
| 36 | +use_ssl = True |
| 37 | +
|
| 38 | +# A comma-separated list of ActiveMQ broker DNS hostnames or aliases |
| 39 | +brokers = activemq1.example.org,activemq2.example.org |
| 40 | +
|
| 41 | +# Only required if use_ssl = False |
| 42 | +username = your-username |
| 43 | +password = your-password |
| 44 | +
|
| 45 | +# Port to use when NOT using SSL (i.e., use_ssl = False) |
| 46 | +nonssl_port = 61613 |
| 47 | +
|
| 48 | +# Port to use when using SSL (i.e., use_ssl = True) |
| 49 | +port = 61617 |
| 50 | +
|
| 51 | +# Required if use_ssl = True |
| 52 | +ssl_key_file = /path/to/hostkey.pem |
| 53 | +ssl_cert_file = /path/to/hostcert.pem |
| 54 | +
|
| 55 | +# The message topic or queue where FTS publishes transfer status (complete) updates. |
| 56 | +destination = /topic/transfer.fts_monitoring_complete |
| 57 | +
|
| 58 | +# Optional: virtual host name used to connect to the broker |
| 59 | +# This is only needed if your ActiveMQ setup requires a virtual host (e.g., in multi-tenant environments) |
| 60 | +broker_virtual_host = /atlas |
| 61 | +``` |
| 62 | + |
0 commit comments