Concurrency fix for Global cache PoC

[jorgee]

This pull request introduces distributed locking for task work directory creation in Nextflow, enabling safe concurrent execution in environments using a global cache (such as cloud storage). It adds a new GlobalCacheLockManager that uses atomic file creation for distributed locks, updates the S3 file system provider to support atomic file creation with CREATE_NEW semantics, and refactors locking interfaces for extensibility. Comprehensive tests for the new lock manager are also included.

Distributed Locking for Global Cache:

• Added GlobalCacheLockManager (GlobalCacheLockManager.groovy), which implements distributed locks using atomic file creation in cloud storage, and the LockHandle interface to standardize lock release. ([[1]](https://github.com/nextflow-io/nextflow/pull/6832/files#diff-568d1408d477494155d7329f8929018eb157a09264142a10c39ab24f73a782d6R1-R122), [[2]](https://github.com/nextflow-io/nextflow/pull/6832/files#diff-5085a999ac12b57fa5d9a066631a888fb8855a1c4bc44c595ca3814979414528R1-R23))
• Updated TaskProcessor to use safeCheckAndCreateDir, ensuring that only one pipeline creates a task work directory when using a global cache, preventing race conditions. ([[1]](https://github.com/nextflow-io/nextflow/pull/6832/files#diff-3250bcd11fe7bb7aeeaf9839c44ac5d737dc27ddc9a734954789d09eb9dab399L825-R841), [[2]](https://github.com/nextflow-io/nextflow/pull/6832/files#diff-3250bcd11fe7bb7aeeaf9839c44ac5d737dc27ddc9a734954789d09eb9dab399R853-R888), [[3]](https://github.com/nextflow-io/nextflow/pull/6832/files#diff-3250bcd11fe7bb7aeeaf9839c44ac5d737dc27ddc9a734954789d09eb9dab399R100))

Atomic File Creation Support for S3 and Az:

• Modified S3FileSystemProvider and S3OutputStream to support the CREATE_NEW option, using If-None-Match: * precondition for atomic object creation. Throws FileAlreadyExistsException if the object already exists, ensuring correct lock semantics. ([[1]](https://github.com/nextflow-io/nextflow/pull/6832/files#diff-0fe8c2b4ba2ecc70a4c3409eafb255f6f3a3abb2a0ead25f6290a5b55e5faa38R223-R224), [[2]](https://github.com/nextflow-io/nextflow/pull/6832/files#diff-0fe8c2b4ba2ecc70a4c3409eafb255f6f3a3abb2a0ead25f6290a5b55e5faa38L237-R239), [[3]](https://github.com/nextflow-io/nextflow/pull/6832/files#diff-0fe8c2b4ba2ecc70a4c3409eafb255f6f3a3abb2a0ead25f6290a5b55e5faa38L261-R263), [[4]](https://github.com/nextflow-io/nextflow/pull/6832/files#diff-0fe8c2b4ba2ecc70a4c3409eafb255f6f3a3abb2a0ead25f6290a5b55e5faa38L336-R338), [[5]](https://github.com/nextflow-io/nextflow/pull/6832/files#diff-0fe8c2b4ba2ecc70a4c3409eafb255f6f3a3abb2a0ead25f6290a5b55e5faa38L349-R352), [[6]](https://github.com/nextflow-io/nextflow/pull/6832/files#diff-9be7f9a49f9f4da9db70ad595bc09e6396b5b5bb1326640548d0c40405c61519R27-R37), [[7]](https://github.com/nextflow-io/nextflow/pull/6832/files#diff-9be7f9a49f9f4da9db70ad595bc09e6396b5b5bb1326640548d0c40405c61519R147-R151), [[8]](https://github.com/nextflow-io/nextflow/pull/6832/files#diff-9be7f9a49f9f4da9db70ad595bc09e6396b5b5bb1326640548d0c40405c61519R214-R218), [[9]](https://github.com/nextflow-io/nextflow/pull/6832/files#diff-9be7f9a49f9f4da9db70ad595bc09e6396b5b5bb1326640548d0c40405c61519R634-R653))

Testing:

• Added GlobalCacheLockManagerTest, S3GlobalCacheLockManagerTest, AzGlobalCacheLockManagerTest and GsGlobalCacheLockManagerTest with comprehensive tests for lock acquisition, release, concurrency, and error handling to ensure robust distributed locking behavior.