Skip to content

Commit 6d6f724

Browse files
committed
docs: add comprehensive secrets usage documentation
Add docs/secrets.md with detailed guidance on: - Configuring workspace secrets with database credentials - Setting up secrets in Seqera Platform - Local development with Nextflow secrets command - Troubleshooting common secrets configuration issues - Error pattern identification and resolution Update README.md to reference the new secrets documentation. Provides users with complete guidance for secure credential management in production deployments, especially for cloud databases like AWS Athena. Signed-off-by: Edmund Miller <[email protected]>
1 parent 77caa0c commit 6d6f724

File tree

2 files changed

+57
-0
lines changed

2 files changed

+57
-0
lines changed

README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,8 @@ The following options are available:
5757
`sql.db.'<DB-NAME>'.password`
5858
: The database connection password.
5959

60+
For information on using secrets with database credentials, see [docs/secrets.md](docs/secrets.md).
61+
6062
## Dataflow Operators
6163

6264
This plugin provides the following dataflow operators for querying from and inserting into database tables.

docs/secrets.md

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# Using Secrets for Database Credentials
2+
3+
For production deployments, it's recommended to use Nextflow secrets instead of hardcoding credentials in configuration files. This is especially important when working with cloud databases like AWS Athena.
4+
5+
## Configuration with Secrets
6+
7+
When using [Nextflow secrets](https://www.nextflow.io/docs/latest/secrets.html) (available in Nextflow 25.04.0+), you can reference workspace or user-level secrets in your database configuration:
8+
9+
```groovy
10+
sql {
11+
db {
12+
athena {
13+
url = 'jdbc:awsathena://AwsRegion=us-east-1;S3OutputLocation=s3://bucket;Workgroup=workgroup'
14+
user = secrets.ATHENA_USER
15+
password = secrets.ATHENA_PASSWORD
16+
driver = 'com.simba.athena.jdbc.Driver'
17+
}
18+
}
19+
}
20+
```
21+
22+
## Setting Up Secrets in Seqera Platform
23+
24+
1. **Workspace Secrets**: Navigate to your workspace → Secrets → Add secret
25+
2. **User Secrets**: Navigate to Your profile → Secrets → Add secret
26+
3. Create secrets with names matching your configuration (e.g., `ATHENA_USER`, `ATHENA_PASSWORD`)
27+
28+
## Troubleshooting Secrets Issues
29+
30+
If you encounter authentication errors like "Missing credentials error", verify:
31+
32+
- **Secret Names**: Ensure secret names in your configuration match exactly (case-sensitive)
33+
- **Permissions**: Verify you have access to workspace secrets or have defined user-level secrets
34+
- **Nextflow Version**: Secrets require Nextflow >=25.04.0
35+
- **Secret Values**: Ensure secrets contain valid credentials (no empty values)
36+
37+
Common error patterns:
38+
- `user=sa; password=null` indicates secrets were not resolved
39+
- `Unresolved secret detected` means secret names don't match or aren't accessible
40+
41+
## Local Development
42+
43+
For local testing, you can use the Nextflow secrets command:
44+
45+
```bash
46+
# Set secrets locally
47+
nextflow secrets set ATHENA_USER "your-username"
48+
nextflow secrets set ATHENA_PASSWORD "your-password"
49+
50+
# List secrets
51+
nextflow secrets list
52+
53+
# Run pipeline with secrets
54+
nextflow run your-pipeline.nf
55+
```

0 commit comments

Comments
 (0)