Skip to content

Add -v/--verbose flag for detailed HTTP request logging (issue #27) #48

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Tahoora-Tabassum wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Add -v/--verbose flag for detailed HTTP request logging (issue #27) #48

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
2fb3f06
Fix --version-id -> --versionid in CLI
Tahoora-Tabassum Dec 23, 2025
d4bb454
Fix --version-id -> --versionid in test script
Tahoora-Tabassum Dec 23, 2025
189458e
cli: add mkdist validations, completion helper, tests and docs
Tahoora-Tabassum Dec 26, 2025
4d605f8
Resolve merge conflicts with upstream/main (README, cli.py)
Tahoora-Tabassum Dec 26, 2025
d72a444
Resolve remaining merge markers in cli.py
Tahoora-Tabassum Dec 26, 2025
af27e18
Use api_deploy.create_distribution to avoid circular import
Tahoora-Tabassum Dec 26, 2025
a7b361f
Add verbose CLI flag with redacted HTTP logging
Tahoora-Tabassum Jan 4, 2026
0cd28c7
Add verbose (-v) flag using logging; redact Authorization and X-API-K…
Tahoora-Tabassum Jan 6, 2026
faadd58
Add verbose logging support and stabilize auth tests
Tahoora-Tabassum Jan 7, 2026
a6bb139
Resolve merge conflicts with upstream/main
Tahoora-Tabassum Jan 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,11 @@ All notable changes to this project will be documented in this file.
### Notes
- Version 0.15 skips 0.13 and 0.14 as requested in issue #35
- This release updates the PyPI package to align with current repository features


## [Unreleased]

### Added
- Add `-v/--verbose` global CLI option to enable redacted HTTP request/response logging for debugging (`databusclient -v ...`)
- Ensure `Authorization` and `X-API-KEY` headers are redacted in verbose output
- Add unit tests and README documentation for verbose mode
19 changes: 19 additions & 0 deletions PR_BODY.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
Title: Add verbose CLI flag (-v) with redacted HTTP logging

Short description:
- Add a global `-v/--verbose` CLI flag to enable redacted HTTP request/response logging to help debug interactions with the Databus and Vault.

What changed:
- Add global `-v/--verbose` option to `databusclient` CLI and propagate it to API calls.
- Implement redacted HTTP logging helper (redacts `Authorization` and `X-API-KEY` headers).
- Instrument `download` and Vault token exchange flows to print HTTP request/response details when `-v` is enabled.
- Add unit tests ensuring verbose logs are printed and sensitive tokens are redacted.
- Update `README.md` and add a `CHANGELOG.md` entry.

Why:
- Provides safe, actionable debugging output for issues involving HTTP communication and auth problems without exposing secrets.

Security note:
- Authorization and API-key headers are redacted in verbose output. Avoid enabling verbose output in public CI logs.

Closes #27
90 changes: 86 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,10 +50,26 @@ python3 -m pip install --upgrade databusclient==0.15
You can then use the client in the command line:

```bash
# Python
databusclient --help
databusclient deploy --help
databusclient delete --help
databusclient download --help

# Example output:
# Usage: databusclient [OPTIONS] COMMAND [ARGS]...
#
# Options:
# --install-completion [bash|zsh|fish|powershell|pwsh] Install completion for the specified shell.
# --show-completion [bash|zsh|fish|powershell|pwsh] Show completion for the specified shell.
# --help Show this message and exit.
#
# Commands:
# deploy
# download
# delete
# mkdist
# completion
```

### Download command
```

### Docker
Expand Down Expand Up @@ -172,6 +188,8 @@ docker run --rm -v $(pwd):/data dbpedia/databus-python-client download $DOWNLOAD
- If the dataset/files to be downloaded require vault authentication, you need to provide a vault token with `--vault-token /path/to/vault-token.dat`. See [Registration (Access Token)](#registration-access-token) for details on how to get a vault token.

Note: Vault tokens are only required for certain protected Databus hosts (for example: `data.dbpedia.io`, `data.dev.dbpedia.link`). The client now detects those hosts and will fail early with a clear message if a token is required but not provided. Do not pass `--vault-token` for public downloads.
- `-v, --verbose`
- Enable verbose HTTP request/response output for debugging. Headers that may contain secrets (for example `Authorization` and `X-API-KEY`) are redacted in the output. Use with caution and avoid enabling in public CI logs.
- `--databus-key`
- If the databus is protected and needs API key authentication, you can provide the API key with `--databus-key YOUR_API_KEY`.

Expand Down Expand Up @@ -289,7 +307,7 @@ Usage: databusclient deploy [OPTIONS] [DISTRIBUTIONS]...
- Upload & deploy via Nextcloud (--webdav-url, --remote, --path)

Options:
--version-id TEXT Target databus version/dataset identifier of the form <h
--versionid TEXT Target databus version/dataset identifier of the form <h
Copy link

@coderabbitai coderabbitai bot Jan 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Fix inconsistent option name.

Line 310 shows --versionid but the CLI code and other examples use --version-id (with hyphen). Please correct this to --version-id for consistency.

🤖 Prompt for AI Agents 
In @README.md at line 310, The README has an inconsistent CLI option name
`--versionid`; change it to the hyphenated `--version-id` to match the CLI code
and other examples—update the example line that currently shows `--versionid
TEXT` to `--version-id TEXT` and verify any surrounding examples or references
use the same `--version-id` token for consistency.

ttps://databus.dbpedia.org/$ACCOUNT/$GROUP/$ARTIFACT/$VE
RSION> [required]
--title TEXT Dataset title [required]
Expand All @@ -303,6 +321,18 @@ Options:
--remote TEXT rclone remote name (e.g., 'nextcloud')
--path TEXT Remote path on Nextcloud (e.g., 'datasets/mydataset')
--help Show this message and exit.
<<<<<<< HEAD

```
#### Examples of using deploy command
##### Mode 1: Classic Deploy (Distributions)
```
databusclient deploy --versionid https://databus.dbpedia.org/user1/group1/artifact1/2022-05-18 --title title1 --abstract abstract1 --description description1 --license http://dalicc.net/licenselibrary/AdaptivePublicLicense10 --apikey MYSTERIOUS 'https://raw.githubusercontent.com/dbpedia/databus/master/server/app/api/swagger.yml|type=swagger'
```

```
databusclient deploy --versionid https://dev.databus.dbpedia.org/denis/group1/artifact1/2022-05-18 --title "Client Testing" --abstract "Testing the client...." --description "Testing the client...." --license http://dalicc.net/licenselibrary/AdaptivePublicLicense10 --apikey MYSTERIOUS 'https://raw.githubusercontent.com/dbpedia/databus/master/server/app/api/swagger.yml|type=swagger'
=======
```

### Mode 1: Classic Deploy (Distributions)
Expand All @@ -326,6 +356,7 @@ docker run --rm -v $(pwd):/data dbpedia/databus-python-client deploy \
--license http://dalicc.net/licenselibrary/AdaptivePublicLicense10 \
--apikey MYSTERIOUS \
'https://raw.githubusercontent.com/dbpedia/databus/master/server/app/api/swagger.yml|type=swagger
>>>>>>> upstream/main
```
A few more notes for CLI usage:

Expand All @@ -342,6 +373,10 @@ All files referenced there will be registered on the Databus.
```bash
# Python
databusclient deploy \
<<<<<<< HEAD
--metadata /home/metadata.json \
--versionid https://databus.org/user/dataset/version/1.0 \
=======
--metadata ./metadata.json \
--version-id https://databus.dbpedia.org/user1/group1/artifact1/1.0 \
--title "Metadata Deploy Example" \
Expand All @@ -353,6 +388,7 @@ databusclient deploy \
docker run --rm -v $(pwd):/data dbpedia/databus-python-client deploy \
--metadata ./metadata.json \
--version-id https://databus.dbpedia.org/user1/group1/artifact1/1.0 \
>>>>>>> upstream/main
--title "Metadata Deploy Example" \
--abstract "This is a short abstract of the dataset." \
--description "This dataset was uploaded using metadata.json." \
Expand Down Expand Up @@ -388,6 +424,9 @@ databusclient deploy \
--webdav-url https://cloud.example.com/remote.php/webdav \
--remote nextcloud \
--path datasets/mydataset \
<<<<<<< HEAD
--versionid https://databus.org/user/dataset/version/1.0 \
=======
--version-id https://databus.dbpedia.org/user1/group1/artifact1/1.0 \
--title "Test Dataset" \
--abstract "Short abstract of dataset" \
Expand All @@ -402,6 +441,7 @@ docker run --rm -v $(pwd):/data dbpedia/databus-python-client deploy \
--remote nextcloud \
--path datasets/mydataset \
--version-id https://databus.dbpedia.org/user1/group1/artifact1/1.0 \
>>>>>>> upstream/main
--title "Test Dataset" \
--abstract "Short abstract of dataset" \
--description "This dataset was uploaded for testing the Nextcloud → Databus pipeline." \
Expand Down Expand Up @@ -487,6 +527,48 @@ databusclient delete https://databus.dbpedia.org/dbpedia/collections/dbpedia-sna
docker run --rm -v $(pwd):/data dbpedia/databus-python-client delete https://databus.dbpedia.org/dbpedia/collections/dbpedia-snapshot-2022-12 --databus-key YOUR_API_KEY
```

### mkdist command

Create a distribution string from components.

Usage:
```
databusclient mkdist URL --cv key=value --cv key2=value2 --format ttl --compression gz --sha-length <sha256hex>:<length>
```

Example:
```
python -m databusclient mkdist "https://example.org/file.ttl" --cv lang=en --cv part=sorted --format ttl --compression gz --sha-length aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:12345
```

## Completion

Enable shell completion (bash example):
```
eval "$(_DATABUSCLIENT_COMPLETE=source_bash python -m databusclient)"
```

### mkdist command

Create a distribution string from components.

Usage:
```
databusclient mkdist URL --cv key=value --cv key2=value2 --format ttl --compression gz --sha-length <sha256hex>:<length>
```

Example:
```
python -m databusclient mkdist "https://example.org/file.ttl" --cv lang=en --cv part=sorted --format ttl --compression gz --sha-length aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:12345
```

## Completion

Enable shell completion (bash example):
```
eval "$(_DATABUSCLIENT_COMPLETE=source_bash python -m databusclient)"
```
Comment on lines +530 to +570
Copy link

@coderabbitai coderabbitai bot Jan 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Critical: Duplicate sections must be removed.

The mkdist and Completion sections are duplicated (lines 530-549 and 551-570). This duplication likely resulted from incomplete merge conflict resolution.

Please:

  1. Remove the duplicate sections
  2. Keep only one instance of each section
  3. Ensure proper formatting and language specification for code blocks (as flagged by markdownlint)
🧰 Tools
🪛 markdownlint-cli2 (0.18.1)

535-535: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

540-540: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

547-547: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

556-556: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

561-561: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

565-565: Multiple headings with the same content

(MD024, no-duplicate-heading)

568-568: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

🤖 Prompt for AI Agents 
In @README.md around lines 530 - 570, Remove the duplicated "### mkdist command"
and "## Completion" sections so only one copy of each remains; locate the
repeated headings ("mkdist command" and "Completion") and delete the second
occurrence (the block starting at the duplicate heading), preserving the first
instance with its Usage, Example, and code blocks; also ensure fenced code
blocks are marked with the language (e.g., ```bash or ```text/```python as
appropriate) to satisfy markdownlint and keep formatting consistent.


## Module Usage

<a id="module-deploy"></a>
Expand Down
Loading