Skip to content

Conversation

lovasoa
Copy link
Collaborator

@lovasoa lovasoa commented Aug 29, 2025

Convert database BLOBs to data URIs, add documentation and tests.

lovasoa added 15 commits August 29, 2025 21:47
- Add comprehensive BLOB support for all supported databases:
  * PostgreSQL: BYTEA columns with data URL conversion
  * MySQL/MariaDB: BLOB columns with data URL conversion
  * MSSQL: VARBINARY, BIGVARBINARY, BINARY, IMAGE columns
  * SQLite: BLOB columns with data URL conversion

- Create shared data URL conversion functions to eliminate code duplication
- Add comprehensive tests for all database types
- Update CHANGELOG.md with detailed feature description

- All blob data is now consistently converted to data URLs with base64 encoding
- Cross-database compatibility ensures identical blob behavior across all backends
- Comprehensive testing validates functionality across PostgreSQL, MySQL, MariaDB, MSSQL, and SQLite
- Change blob function parameters from Vec<u8> to &[u8] to avoid unnecessary copying
- Use inline format args in data URL construction
- Update function calls to borrow parameters correctly
- Maintain backward compatibility and functionality
- Implement automatic MIME type detection based on file signatures (magic bytes)
- Support common file formats: PNG, JPEG, GIF, BMP, WebP, SVG, PDF, DOCX, XLSX, PPTX, JSON, XML, ZIP
- Automatic fallback to 'application/octet-stream' for unknown formats
- Update CHANGELOG.md with comprehensive feature description
- Add comprehensive tests for MIME type detection functionality

- BLOB data now automatically returns appropriate data URLs:
  * PNG files: 'data:image/png;base64,...'
  * PDF files: 'data:application/pdf;base64,...'
  * SVG files: 'data:image/svg+xml;base64,...'
  * Unknown files: 'data:application/octet-stream;base64,...'

- Improves user experience by providing correct MIME types for downloads and displays
- Eliminates need for manual MIME type specification in most cases
- Use bytes.starts_with() for cleaner magic byte detection
- Remove verbose comments for each MIME type
- Maintain same functionality with cleaner, more readable code
- Reduce code duplication and improve maintainability
- Use byte string literals (b"string") for better readability
- Remove UTF-8 parsing for text-based formats, use direct byte comparisons
- Maintain same functionality with cleaner, more performant code
- PNG: b"\x89PNG\r\n\x1a\n" instead of hex arrays
- JPEG: b"\xFF\xD8" instead of [0xFF, 0xD8]
- Text formats: Direct byte matching without String::from_utf8_lossy
- Update all tests to use new byte string format
- Create new mime_detection.rs module for better code organization
- Move detect_mime_type() function and tests to dedicated module
- Update sql_to_json.rs to import from mime_detection module
- Remove unused import from functions.rs
- Maintain same functionality with improved code structure

Benefits:
- Better separation of concerns
- Improved code organization and maintainability
- Easier to extend MIME detection in the future
- Cleaner module boundaries

File changes:
- NEW: src/webserver/database/mime_detection.rs (MIME detection + tests)
- MOD: src/webserver/database/mod.rs (add mime_detection module)
- MOD: src/webserver/database/sql_to_json.rs (use mime_detection module)
- MOD: src/webserver/database/sqlpage_functions/functions.rs (remove unused import)
- Rename module from mime_detection to blob_to_data_url for better clarity
- Follow project naming conventions (similar to sql_to_json.rs)
- Update all imports and references to use new module name
- Maintain same functionality with improved code organization

File changes:
- RENAMED: src/webserver/database/mime_detection.rs → src/webserver/database/blob_to_data_url.rs
- MOD: src/webserver/database/mod.rs (update module declaration)
- MOD: src/webserver/database/sql_to_json.rs (update import path)
- MOD: src/webserver/database/sqlpage_functions/functions.rs (update import path)
- Add #[must_use] attribute to detect_mime_type for better usage indication
- Replace empty byte check with is_empty() for clarity
- Update tests for improved readability with formatted assertions
- Remove unnecessary blank lines in sql_to_json.rs

Benefits:
- Improved code clarity and maintainability
- Enhanced test readability
- Replace direct call to vec_to_data_uri_value with updated import from blob_to_data_url
- Remove deprecated vec_to_data_uri and vec_to_data_uri_value functions for cleaner code
- Maintain existing functionality while improving code organization
- Remove unnecessary empty byte check and streamline conditions for text-based formats
- Enhance readability by consolidating checks for XML and JSON formats
- Maintain existing functionality while improving code clarity
- CHANGELOG.md : details on BLOB support and automatic MIME type detection
- Add examples in extensions-to-sql.md illustrating data type conversions and JSON object structure
- Update SQL examples in migrations to reflect new BLOB handling capabilities
@lovasoa lovasoa changed the title enhanced blob support blob support: decode binary database values as data URLs Aug 30, 2025
@lovasoa lovasoa merged commit 60ab81c into main Aug 30, 2025
10 checks passed
@lovasoa lovasoa deleted the feature/enhanced-blob-support branch August 30, 2025 07:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant