Merge pull request #7975 from chaen/cherry-pick-2-5aac36b18-integration

[sweep:integration] allow to disable DM transfer as an FTS failover

Author: fstagni

.pre-commit-config.yaml

@@ -11,7 +11,7 @@ exclude: |
 
 repos:
   - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v4.4.0
+    rev: v5.0.0
     hooks:
       - id: trailing-whitespace
       - id: end-of-file-fixer

dirac.cfg

@@ -467,6 +467,7 @@ Systems
           {
             Location = DIRAC/DataManagementSystem/Agent/RequestOperations/ReplicateAndRegister
             FTSMode = True # If True, will use FTS to transfer files
+            DMMode = True # If false, DataManager will not be used as a failover of FTS transfers
             FTSBannedGroups = lhcb_user # list of groups for which not to use FTS
           }
           SetFileStatus

docs/source/AdministratorGuide/Systems/RequestManagement/rmsObjects.rst

@@ -188,6 +188,7 @@ Details: :py:mod:`~DIRAC.DataManagementSystem.Agent.RequestOperations.ReplicateA
 Extra configuration options:
 
 * `FTSMode`: If True, will use FTS to transfer files
+* `DMMode`: if False, will not use DataManager transfer as FTS failover
 * `FTSBannedGroups` : list of groups for which not to use FTS
 
 ------

docs/source/DeveloperGuide/Systems/RequestManagement/index.rst

@@ -238,6 +238,7 @@ The timeout for the operation is then calculated from this value and the number
 
 The `ReplicateAndRegister` section accepts extra attributes, specific to FTSTransfers:
   * FTSMode (default False): if True, delegate transfers to FTS
+  * DMMode (default True): if False, will not use DataManager as a failover transfer for FTS
   * FTSBannedGroups: list of DIRAC group whose transfers should not go through FTS.
 
 This of course does not cover all possible needs for a specific VO, hence all developers are encouraged to create and keep

src/DIRAC/DataManagementSystem/Agent/RequestOperations/ReplicateAndRegister.py

@@ -518,7 +518,12 @@ def dmTransfer(self, fromFTS=False):
             return S_OK()
         # # loop over files
         if fromFTS:
-            self.log.info("Trying transfer using replica manager as FTS failed")
+            if getattr(self, "DMMode", True):
+                self.log.info("Trying transfer using DataManager as FTS failed")
+            else:
+                self.log.info("DataManager replication disabled, skipping")
+                return S_OK()
+
         else:
             self.log.info("Transferring files using Data manager...")
         errors = defaultdict(int)

src/DIRAC/RequestManagementSystem/ConfigTemplate.cfg

@@ -70,6 +70,7 @@ Agents
       {
         Location = DIRAC/DataManagementSystem/Agent/RequestOperations/ReplicateAndRegister
         FTSMode = False
+        DMMode = True
         FTSBannedGroups = dirac_user, lhcb_user
         LogLevel = INFO
         MaxAttempts = 256