|
| 1 | +# Using Secrets for Database Credentials |
| 2 | + |
| 3 | +For production deployments, it's recommended to use Nextflow secrets instead of hardcoding credentials in configuration files. This is especially important when working with cloud databases like AWS Athena. |
| 4 | + |
| 5 | +## Configuration with Secrets |
| 6 | + |
| 7 | +When using [Nextflow secrets](https://www.nextflow.io/docs/latest/secrets.html) (available in Nextflow 25.04.0+), you can reference workspace or user-level secrets in your database configuration: |
| 8 | + |
| 9 | +```groovy |
| 10 | +sql { |
| 11 | + db { |
| 12 | + athena { |
| 13 | + url = 'jdbc:awsathena://AwsRegion=us-east-1;S3OutputLocation=s3://bucket;Workgroup=CompBio' |
| 14 | + user = secrets.ATHENA_USER |
| 15 | + password = secrets.ATHENA_PASSWORD |
| 16 | + driver = 'com.simba.athena.jdbc.Driver' |
| 17 | + } |
| 18 | + } |
| 19 | +} |
| 20 | +``` |
| 21 | + |
| 22 | +## Setting Up Secrets in Seqera Platform |
| 23 | + |
| 24 | +1. **Workspace Secrets**: Navigate to your workspace → Secrets → Add secret |
| 25 | +2. **User Secrets**: Navigate to Your profile → Secrets → Add secret |
| 26 | +3. Create secrets with names matching your configuration (e.g., `ATHENA_USER`, `ATHENA_PASSWORD`) |
| 27 | + |
| 28 | +## Troubleshooting Secrets Issues |
| 29 | + |
| 30 | +If you encounter authentication errors like "Missing credentials error", verify: |
| 31 | + |
| 32 | +- **Secret Names**: Ensure secret names in your configuration match exactly (case-sensitive) |
| 33 | +- **Permissions**: Verify you have access to workspace secrets or have defined user-level secrets |
| 34 | +- **Nextflow Version**: Secrets require Nextflow >=25.04.0 |
| 35 | +- **Secret Values**: Ensure secrets contain valid credentials (no empty values) |
| 36 | + |
| 37 | +Common error patterns: |
| 38 | +- `user=sa; password=null` indicates secrets were not resolved |
| 39 | +- `Unresolved secret detected` means secret names don't match or aren't accessible |
| 40 | + |
| 41 | +## Local Development |
| 42 | + |
| 43 | +For local testing, you can use the Nextflow secrets command: |
| 44 | + |
| 45 | +```bash |
| 46 | +# Set secrets locally |
| 47 | +nextflow secrets set ATHENA_USER "your-username" |
| 48 | +nextflow secrets set ATHENA_PASSWORD "your-password" |
| 49 | + |
| 50 | +# List secrets |
| 51 | +nextflow secrets list |
| 52 | + |
| 53 | +# Run pipeline with secrets |
| 54 | +nextflow run your-pipeline.nf |
| 55 | +``` |
0 commit comments