Skip to content

Commit c6d539c

Browse files
reroweprerowep
committed
initial: add rero-invenio-thumbnails extension
Lightweight Invenio extension for discovering book thumbnail URLs from multiple providers. Features: - Multi-provider architecture with fallback chain - 6 built-in providers: Files, OpenLibrary, BNF, DNB, GoogleBooks, GoogleAPI - Plugin-based system via entry points for custom providers - Redis-based caching with configurable TTL - RESTful JSON API: GET /thumbnails-url/<isbn> - HTTP retry logic with exponential backoff - Comprehensive test suite: 112 tests, 97% coverage Providers: - FilesProvider: Serve from local filesystem directory - OpenLibraryProvider: Community-driven book cover database - BnfProvider: Bibliothèque nationale de France covers - GoogleBooksProvider: Google Books preview service - GoogleApiProvider: Official Google Books API Architecture: - URL discovery only (no image serving/manipulation) - Chainable provider pattern with first-match-wins strategy - Configurable provider order and retry behavior - Full Sphinx documentation with API reference Co-Authored-by: Peter Weber <peter.weber@rero.ch>
0 parents  commit c6d539c

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

60 files changed

+9158
-0
lines changed

‎.dockerignore‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
.git
2+
*.gitignore
3+
4+
*.mo
5+
*.pyc
6+
*.swp
7+
*.swo
8+
*.~
9+
10+
.dockerignore
11+
Dockerfile
12+
docker-compose.yml
13+
docker-compose-dev.yml
14+
15+
Procfile*

‎.editorconfig‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
# -*- coding: utf-8 -*-
2+
#
3+
# This file is part of RERO ILS.
4+
# Copyright (C) 2017 RERO.
5+
#
6+
# RERO ILS is free software; you can redistribute it
7+
# and/or modify it under the terms of the GNU General Public License as
8+
# published by the Free Software Foundation; either version 2 of the
9+
# License, or (at your option) any later version.
10+
#
11+
# RERO ILS is distributed in the hope that it will be
12+
# useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
13+
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
14+
# General Public License for more details.
15+
#
16+
# You should have received a copy of the GNU General Public License
17+
# along with RERO ILS; if not, write to the
18+
# Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston,
19+
# MA 02111-1307, USA.
20+
#
21+
# In applying this license, RERO does not
22+
# waive the privileges and immunities granted to it by virtue of its status
23+
# as an Intergovernmental Organization or submit itself to any jurisdiction.
24+
25+
root = true
26+
27+
[*]
28+
indent_style = space
29+
end_of_line = lf
30+
insert_final_newline = true
31+
trim_trailing_whitespace = true
32+
charset = utf-8
33+
34+
# Python files
35+
[*.py]
36+
indent_size = 4
37+
# isort plugin configuration
38+
known_first_party = rero_ils
39+
multi_line_output = 2
40+
default_section = THIRDPARTY
41+
skip = .eggs
42+
# Needed for isort since version 4.3.10
43+
reverse_relative = true
44+
45+
# RST files (used by sphinx)
46+
[*.rst]
47+
indent_size = 4
48+
49+
# CSS, HTML, JS, JSON, YML, Typescript
50+
[*.{scss,css,html,js,json,yml,ts}]
51+
indent_size = 2
52+
53+
# Matches the exact files either package.json or .travis.yml
54+
[{package.json,.travis.yml}]
55+
indent_size = 2
56+
57+
# Dockerfile
58+
[Dockerfile]
59+
indent_size = 4

‎.github/auto_assign.yml‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# Set to true to add reviewers to pull requests
2+
addReviewers: false
3+
4+
# Set to true to add assignees to pull requests
5+
addAssignees: author
6+
7+
runOnDraft: true
8+
9+
# A number of reviewers added to the pull request
10+
# Set 0 to add all the reviewers (default: 0)
11+
# numberOfReviewers: 1
12+
13+
# A number of assignees to add to the pull request
14+
# Set to 0 to add all of the assignees.
15+
# Uses numberOfReviewers if unset.
16+
# numberOfAssignees: 2
17+
18+
19+
# A list of keywords to be skipped the process that add reviewers if pull requests include it
20+
# skipKeywords:
21+
# - wip

‎.github/workflows/auto_assign.yml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
name: 'Auto Assign'
2+
on:
3+
pull_request_target:
4+
types: [opened, ready_for_review]
5+
6+
jobs:
7+
add-reviews:
8+
runs-on: ubuntu-latest
9+
steps:
10+
- uses: kentaro-m/auto-assign-action@v2.0.0

‎.github/workflows/continuous-integration-test.yml‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# RERO Invenio Base
2+
# Copyright (C) 2025 RERO.
3+
#
4+
# This program is free software: you can redistribute it and/or modify
5+
# it under the terms of the GNU Affero General Public License as published by
6+
# the Free Software Foundation, version 3 of the License.
7+
#
8+
# This program is distributed in the hope that it will be useful,
9+
# but WITHOUT ANY WARRANTY; without even the implied warranty of
10+
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
11+
# GNU Affero General Public License for more details.
12+
#
13+
# You should have received a copy of the GNU Affero General Public License
14+
# along with this program. If not, see <http://www.gnu.org/licenses/>.
15+
16+
name: CI
17+
on: [push, pull_request, workflow_dispatch]
18+
19+
jobs:
20+
test:
21+
runs-on: ubuntu-latest
22+
strategy:
23+
matrix:
24+
python-version: ["3.12"]
25+
dependencies: ["dev", "deploy"]
26+
steps:
27+
28+
- uses: actions/checkout@v4
29+
30+
- name: Install uv
31+
uses: astral-sh/setup-uv@v5
32+
with:
33+
enable-cache: true
34+
cache-dependency-glob: uv.lock
35+
36+
- name: Set up Python
37+
run: uv python install ${{ matrix.python-version }}
38+
39+
- name: Update and install dependencies
40+
if: ${{ matrix.dependencies == 'dev' }}
41+
run: uv sync --upgrade
42+
43+
- name: Install locked dependencies
44+
if: ${{ matrix.dependencies == 'deploy' }}
45+
run: uv sync --frozen
46+
47+
- name: Run tests
48+
run: uv run poe run_tests
49+
50+
- name: Coverage XML
51+
if: ${{ matrix.dependencies == 'dev' }}
52+
run: uv run coverage xml
53+
54+
- name: Coveralls
55+
if: ${{ matrix.dependencies == 'dev' }}
56+
uses: coverallsapp/github-action@v2
57+
with:
58+
format: cobertura

‎.github/workflows/pypi-publish.yml‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
# RERO Invenio Thumbnails
2+
# Copyright (C) 2026 RERO.
3+
#
4+
# This program is free software: you can redistribute it and/or modify
5+
# it under the terms of the GNU Affero General Public License as published by
6+
# the Free Software Foundation, version 3 of the License.
7+
#
8+
# This program is distributed in the hope that it will be useful,
9+
# but WITHOUT ANY WARRANTY; without even the implied warranty of
10+
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
11+
# GNU Affero General Public License for more details.
12+
#
13+
# You should have received a copy of the GNU Affero General Public License
14+
# along with this program. If not, see <http://www.gnu.org/licenses/>.
15+
16+
name: Publish
17+
18+
on:
19+
release:
20+
types: ["published"]
21+
22+
jobs:
23+
run:
24+
name: "Build and publish release"
25+
runs-on: ubuntu-latest
26+
environment: pypi
27+
permissions:
28+
id-token: write
29+
30+
steps:
31+
- uses: actions/checkout@v4
32+
33+
- name: Install uv
34+
uses: astral-sh/setup-uv@v5
35+
with:
36+
enable-cache: true
37+
cache-dependency-glob: uv.lock
38+
39+
- name: Set up Python
40+
run: uv python install 3.12
41+
42+
- name: Build wheel
43+
run: uv build
44+
45+
- name: Publish package
46+
run: uv publish

‎.gitignore‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
# Byte-compiled / optimized / DLL files
2+
__pycache__/
3+
*.py[cod]
4+
5+
# Idea software family
6+
.idea/
7+
8+
# C extensions
9+
*.so
10+
11+
# Distribution / packaging
12+
.Python
13+
env/
14+
build/
15+
develop-eggs/
16+
dist/
17+
downloads/
18+
eggs/
19+
.eggs/
20+
lib/
21+
lib64/
22+
parts/
23+
sdist/
24+
var/
25+
*.egg-info/
26+
.installed.cfg
27+
*.egg
28+
Pipfile
29+
Pipfile.lock
30+
31+
# PyInstaller
32+
# Usually these files are written by a python script from a template
33+
# before PyInstaller builds the exe, so as to inject date/other infos into it.
34+
*.manifest
35+
*.spec
36+
37+
# Installer logs
38+
pip-log.txt
39+
pip-delete-this-directory.txt
40+
41+
# Virtual environments
42+
.venv/
43+
44+
# Unit test / coverage reports
45+
htmlcov/
46+
.tox/
47+
.coverage
48+
.coverage.*
49+
.cache
50+
nosetests.xml
51+
coverage.xml
52+
*,cover
53+
54+
# Translations
55+
*.mo
56+
57+
# Django stuff:
58+
*.log
59+
60+
# Sphinx documentation
61+
docs/_build/
62+
63+
# PyBuilder
64+
target/
65+
66+
# Vim swapfiles
67+
.*.sw?

‎AUTHORS.rst‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
..
2+
Copyright (C) 2026 RERO.
3+
4+
rero-invenio-thumbnails is free software; you can redistribute it
5+
and/or modify it under the terms of the MIT License; see LICENSE file for
6+
more details.
7+
8+
Authors
9+
=======
10+
11+
RERO Invenio module that adds thumbnails
12+
13+
- RERO <software@rero.ch>

‎CHANGES.rst‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
..
2+
Copyright (C) 2026 RERO.
3+
4+
rero-invenio-thumbnails is free software; you can redistribute it
5+
and/or modify it under the terms of the MIT License; see LICENSE file for
6+
more details.
7+
8+
Changes
9+
=======
10+
11+
Version 0.2.0 (unreleased)
12+
--------------------------
13+
14+
**Breaking Changes:**
15+
16+
- **API Return Format Change**: All provider methods and `get_thumbnail_url()` now return
17+
a tuple `(url, provider_name)` instead of just the URL. This provides transparency about
18+
which provider supplied the thumbnail.
19+
20+
Migration example::
21+
22+
# Old code (v0.1.0)
23+
url = get_thumbnail_url("9780134685991")
24+
25+
# New code (v0.2.0+)
26+
url, provider = get_thumbnail_url("9780134685991")
27+
28+
- **HTTP Response Format**: The `/thumbnails-url/<isbn>` endpoint now includes a `provider`
29+
field in the JSON response::
30+
31+
{
32+
"url": "https://covers.openlibrary.org/b/isbn/9780134685991-L.jpg",
33+
"isbn": "9780134685991",
34+
"provider": "open library"
35+
}
36+
37+
**Improvements:**
38+
39+
- **New endpoint**: Added `/thumbnails/<isbn>` endpoint to serve actual thumbnail image
40+
files directly from local storage (in addition to the existing `/thumbnails-url/<isbn>`
41+
endpoint that returns JSON with URLs)
42+
- **Client-side caching**: The `/thumbnails/<isbn>` endpoint now supports efficient
43+
client-side caching with ETag and Last-Modified headers, plus conditional requests
44+
(If-None-Match, If-Modified-Since) returning 304 Not Modified responses
45+
- Cache entries now store provider information using JSON format (prevents issues with pipe characters in URLs)
46+
- Use `RERO_INVENIO_THUMBNAILS_HTTP_CACHE_MAX_AGE` constant from config.py consistently
47+
- BNF and DNB providers now validate image content before returning URLs
48+
- Improved error handling with provider attribution in error tuples
49+
- Modernized exception handling using `contextlib.suppress` where appropriate
50+
51+
**Bug Fixes:**
52+
53+
- Fixed cache serialization to handle URLs containing pipe characters (`|`) by using JSON instead of pipe-delimited strings
54+
55+
0.1.0 (2026-01-15)
56+
-------------------
57+
58+
Initial release
59+
60+
- Multi-provider thumbnail URL discovery system with fallback chain
61+
- Six thumbnail providers:
62+
- Files: Serve from local filesystem directory
63+
- Open Library: Free, community-driven book cover database
64+
- BNF (Bibliothèque nationale de France): French national library covers
65+
- DNB (Deutsche Nationalbibliothek): German national library covers via SRU
66+
- Google Books API: Official Google Books API integration
67+
- Google Books: Google Books preview service
68+
- Plugin-based architecture via entry points for custom providers
69+
- Redis-based caching via `invenio_cache` with configurable TTL
70+
- Flask blueprint with `/thumbnails-url/<isbn>` endpoint returning JSON
71+
- HTTP retry logic with exponential backoff for external providers
72+
- Configurable retry behavior via `RERO_INVENIO_THUMBNAILS_RETRY_*` settings
73+
- Environment variable `RERO_THUMBNAILS_DISABLE_RETRIES` to disable retries globally
74+
- Comprehensive test suite with 120 tests and 97% code coverage
75+
- Full Sphinx documentation with API reference and usage examples

0 commit comments

Comments
 (0)