-
Notifications
You must be signed in to change notification settings - Fork 0
Document avendehut cli tool #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
a9ac4b3
5d23e78
951c77d
881b800
85167f8
53d61ea
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| root = true | ||
|
|
||
| [*] | ||
| charset = utf-8 | ||
| end_of_line = lf | ||
| insert_final_newline = true | ||
| indent_style = space | ||
| indent_size = 2 | ||
| max_line_length = 100 | ||
|
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| # OneDrive / Microsoft Graph configuration | ||
| ONEDRIVE_CLIENT_ID= # TODO: set your Azure app client ID | ||
| ONEDRIVE_CLIENT_SECRET= # TODO: set your Azure app client secret (if used) | ||
| ONEDRIVE_TENANT_ID=consumers # TODO: 'consumers' for personal Microsoft accounts | ||
|
|
||
| # When using OneDrive source | ||
| SRC_FOLDER=onedrive:/Documents/Books # TODO: OneDrive path or local folder | ||
|
|
||
| # Output folder for build artifacts | ||
| OUT_FOLDER=dist # TODO: output directory | ||
|
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| name: CI | ||
|
|
||
| on: | ||
| push: | ||
| branches: ["main"] | ||
| pull_request: | ||
| branches: ["main"] | ||
|
|
||
| jobs: | ||
| build: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: '3.11' | ||
| - name: Install Poetry | ||
| run: pipx install poetry | ||
| - name: Install deps | ||
| run: poetry install | ||
| - name: Lint | ||
| run: | | ||
| poetry run black --check . | ||
| poetry run flake8 | ||
| - name: Type check | ||
| run: poetry run mypy avendehut | ||
| - name: Test | ||
| run: poetry run pytest --maxfail=1 --disable-warnings -q | ||
| - name: Build sample | ||
| run: | | ||
| mkdir -p examples/books | ||
| # Generate a tiny PDF | ||
| python - <<'PY' | ||
| from pypdf import PdfWriter | ||
| from pathlib import Path | ||
| src = Path('examples/books'); src.mkdir(parents=True, exist_ok=True) | ||
| writer = PdfWriter(); writer.add_blank_page(width=72, height=72); writer.add_metadata({"/Title": "CI PDF", "/Author": "CI"}) | ||
| with open(src / 'ci.pdf', 'wb') as f: | ||
| writer.write(f) | ||
| PY | ||
| poetry run avendehut build --src examples/books --out dist | ||
|
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| ## Contributors | ||
|
|
||
| Thanks to everyone who has contributed to `avendehut`. | ||
|
|
||
| - Your Name Here | ||
|
|
||
| _Please add your name here when you contribute to the project._ | ||
|
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,120 @@ | ||
| ## avendehut | ||
|
|
||
| A CLI tool to scan a folder of books (local or OneDrive), extract metadata, generate tags, and produce a searchable HTML catalog. | ||
|
|
||
| Inspired by the historical figure [Juan Hispalense (siglo XII)](https://es.wikipedia.org/wiki/Juan_Hispalense_%28siglo_XII%29), also known as Avendehut Hispanus. | ||
|
|
||
| ### Features | ||
|
|
||
| - **📚 Metadata extraction**: EPUB, PDF, MOBI (extensible to other formats). | ||
| - **🏷 Automatic tag generation**: From metadata and file content. | ||
| - **🌐 HTML catalog generation**: | ||
| - Generates `index.html`, `data.json`, and static assets. | ||
| - Searchable, filterable, and responsive UI. | ||
| - Supports tag filtering, instant search, and lazy loading for large datasets. | ||
| - **⚡ Incremental builds**: Uses a `.manifest.json` to skip unchanged files. | ||
| - **☁ OneDrive integration**: Via Microsoft Graph API. | ||
| - **🔍 CLI search**: Search without opening the HTML site. | ||
| - **📤 Export catalog**: To CSV or JSON. | ||
| - **🖥 Cross‑platform**: macOS and Linux. | ||
|
|
||
| ### Requirements | ||
|
|
||
| - **OS**: macOS or Linux | ||
| - **Python**: 3.10+ | ||
| - **Dependency management**: Poetry | ||
| - **For OneDrive integration**: | ||
| - Microsoft account | ||
| - Microsoft Graph API app registration | ||
|
|
||
| ### Installation | ||
|
|
||
| 1. Clone the repository. | ||
| ```bash | ||
| git clone https://github.com/khnumdev/avendehut.git | ||
| cd avendehut | ||
| ``` | ||
| 2. Install dependencies with Poetry. | ||
| ```bash | ||
| poetry install | ||
| ``` | ||
| 3. Set environment variables for OneDrive (if needed). | ||
| - `ONEDRIVE_CLIENT_ID` | ||
| - `ONEDRIVE_CLIENT_SECRET` | ||
| - `SRC_FOLDER` (OneDrive folder path) | ||
| - `OUT_FOLDER` (local output path) | ||
|
|
||
| ### OneDrive Setup | ||
|
|
||
| - If `--src` points to a OneDrive path, the tool uses Microsoft Graph API. | ||
| - Required environment variables: | ||
| - `ONEDRIVE_CLIENT_ID` | ||
| - `ONEDRIVE_CLIENT_SECRET` | ||
| - `ONEDRIVE_TENANT_ID` | ||
| - `SRC_FOLDER` (OneDrive folder path) | ||
| - `OUT_FOLDER` (local output path) | ||
| - Register an app and configure permissions by following Microsoft's official docs: [Register an application](https://learn.microsoft.com/en-us/graph/auth-register-app-v2). | ||
|
|
||
| ### CLI Commands | ||
|
Owner
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @cursoragent add a basic "usage" section There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Owner
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @cursoragent add basic usage section, to process from local and for one drive There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
|
|
||
| | Command | Description | Options | | ||
| | --- | --- | --- | | ||
| | `avendehut build` | Scans source folder, processes new/updated books, generates HTML site. | `--src <path>` (source folder), `--out <path>` (output folder), `--format html` (default), `--force` (reprocess all) | | ||
| | `avendehut watch` | Watches source folder for changes, auto‑rebuilds HTML. | Same as build plus `--interval <seconds>` | | ||
| | `avendehut clean` | Deletes generated output and manifest. | `--out <path>` | | ||
| | `avendehut open` | Opens generated HTML in default browser. | `--out <path>` | | ||
| | `avendehut search` | CLI search in local index. | `--query "<text>"`, `--tags "<tag1,tag2>"` | | ||
| | `avendehut export` | Exports catalog to CSV or JSON. | `--out <file>`, `--format csv|json` | | ||
|
|
||
| ### Usage | ||
|
|
||
| ```bash | ||
| # Build a catalog from a local folder | ||
| poetry run avendehut build --src ./books --out ./dist | ||
|
|
||
| # Open the generated HTML | ||
| poetry run avendehut open --out ./dist | ||
|
|
||
| # Search from the CLI | ||
| poetry run avendehut search --out ./dist --query "Dune" --tags "sci-fi,epub" | ||
|
|
||
| # Export catalog | ||
| poetry run avendehut export --src-out ./dist --format csv --out ./dist/catalog.csv | ||
| ``` | ||
|
|
||
| ### HTML Catalog Features | ||
|
|
||
| - Responsive design (mobile, tablet, desktop). | ||
| - Instant search with highlighting. | ||
| - Tag filtering (multi‑select). | ||
| - Lazy loading for large datasets. | ||
| - Split `data.json` into chunks if > 5 MB. | ||
| - Keyboard shortcuts for navigation. | ||
| - Dark mode toggle. | ||
|
|
||
| ### CI/CD Integration | ||
|
|
||
| - Use GitHub Actions for: | ||
| - Linting (`flake8`, `black`). | ||
| - Running tests (`pytest`). | ||
| - Building HTML output. | ||
| - Deploying to GitHub Pages (optional). | ||
|
|
||
| - Example workflow: | ||
| - Trigger on push to `main`. | ||
| - Install Python & Poetry. | ||
| - Run `poetry install`, `poetry run pytest`. | ||
| - If tests pass, run `avendehut build` and deploy. | ||
|
|
||
| ### Development Guide | ||
|
|
||
| - Run in development mode: `poetry run avendehut --help`. | ||
| - Add new commands: extend the CLI module and register the command. | ||
| - Extend metadata extraction: add new extractor classes/modules. | ||
| - Contribute: see [`CONTRIBUTORS.md`](CONTRIBUTORS.md). | ||
|
|
||
| ### License | ||
|
|
||
| Apache 2.0 License. | ||
|
|
||
| # avendehut | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| """avendehut package. | ||
|
|
||
| CLI to scan books, extract metadata, and generate a searchable HTML catalog. | ||
| """ | ||
|
|
||
| __all__ = [ | ||
| "__version__", | ||
| ] | ||
|
|
||
| __version__ = "0.1.0" | ||
|
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| from __future__ import annotations | ||
|
|
||
| import sys | ||
|
|
||
| import click | ||
| from rich.console import Console | ||
|
|
||
| from .commands.build import build_command | ||
| from .commands.watch import watch_command | ||
| from .commands.clean import clean_command | ||
| from .commands.open import open_command | ||
| from .commands.search import search_command | ||
| from .commands.export import export_command | ||
|
|
||
|
|
||
| console = Console() | ||
|
|
||
|
|
||
| @click.group(context_settings={"help_option_names": ["-h", "-help", "--help"]}) | ||
| @click.version_option(package_name="avendehut") | ||
| def main() -> None: | ||
| """avendehut - build and search a local HTML catalog of books. | ||
|
|
||
| Inspired by Juan Hispalense (Avendehut Hispanus). | ||
| """ | ||
|
|
||
|
|
||
| # Register subcommands | ||
| main.add_command(build_command, name="build") | ||
| main.add_command(watch_command, name="watch") | ||
| main.add_command(clean_command, name="clean") | ||
| main.add_command(open_command, name="open") | ||
| main.add_command(search_command, name="search") | ||
| main.add_command(export_command, name="export") | ||
|
|
||
|
|
||
| if __name__ == "__main__": # pragma: no cover | ||
| try: | ||
| main() | ||
| except click.ClickException as e: # pragma: no cover | ||
| console.print(f"[red]Error:[/red] {e}") | ||
| sys.exit(e.exit_code if hasattr(e, "exit_code") else 1) | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@cursoragent This should be for development.
Ideally, I'd like to have this as a package published.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Working on it! I'll start making changes in this branch.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@cursoragent still pending
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You need to increase your spend limit or enable usage-based billing to run background agents. Go to Cursor