Add piece cache feature for fileless torrent operation

[ismdevteam]

Piece Cache Feature for Fileless Torrent Operation

Overview

This PR adds fileless torrent operation capability to libtorrent, allowing seeding without creating original content files using a piece-based cache system.

Key Features

• Fileless Operation (-Z mode): Seed torrents without creating original content files
• Piece-based Caching: Store pieces in a cache directory instead of original files
• Resume Support: Maintain resume data in cache directory
• Backward Compatible: Works alongside traditional file storage
• Seed-from-Cache Mode (-S mode): Seed exclusively from piece cache without creating any original files

Testing with libtorrent v2.0.11

This feature has been extensively tested with libtorrent v2.0.11:

Test Environment:

• libtorrent Version: v2.0.11
• Compiler: GCC 11.3.0 / Clang 14.0.0
• Platform: Linux x86_64
• Build System: CMake 3.22 + Ninja 1.10

Test Coverage:

• ✅ Fileless mode (-Z flag) functionality
• ✅ Seed-from-cache mode (-S flag) functionality
• ✅ Piece caching during download (-C flag)
• ✅ Resume data management in cache directory
• ✅ Both torrent file and magnet link support
• ✅ Seeding from cache without original files
• ✅ Memory and disk cache consistency
• ✅ Error handling and edge cases

Files Added

• examples/piece_cache_manager.hpp - Main cache management interface
• examples/piece_cache_manager.cpp - Cache implementation with disk storage
• examples/simple_piece_cache.hpp - Integration layer with libtorrent session
• examples/client_test_piece_cache.cpp - Enhanced client with piece cache support

Building

git clone --recurse-submodules -b piece-cache-feature https://github.com/ismdevteam/libtorrent.git
cd ./libtorrent
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_STANDARD=17 -G Ninja .. -Dbuild_examples=ON
ninja client_test_piece_cache

Usage Examples

# Fileless mode with torrent file (download + seed)
./client_test_piece_cache -Z -s /tmp/save_path torrent.torrent

# Seed from cache only - no original files created at all
./client_test_piece_cache -S -s /tmp/save_path torrent.torrent

# Fileless mode with cache during download
./client_test_piece_cache -Z -C -s /tmp/save_path torrent.torrent

# Custom cache directory with seed-from-cache mode
./client_test_piece_cache -S --cache_root=/custom/cache torrent.torrent

# Seed from cache with magnet link
./client_test_piece_cache -S "magnet:?xt=urn:btih:..."

Benefits

• Disk Space Efficiency: Store only cached pieces, not entire files
• Privacy: No original content files created on disk
• Performance: Reduced disk I/O for seeding-only scenarios
• Flexibility: Switch between fileless and traditional modes
• Pure Seeding: -S mode enables seeding without any file creation

Backward Compatibility

• Fully backward compatible with existing libtorrent functionality
• Optional feature disabled by default
• No breaking changes to existing APIs

[USBhost]

Is the point of this to help seed if you do not care about the content?

[ismdevteam]

Is the point of this to help seed if you do not care about the content?

This pull request implements functionality similar to what qBittorrent offers with its "SQLite database (experimental)" resume data storage option, but directly within libtorrent itself and without the SQLite dependency.

What it does:

qBittorrent has an advanced setting where you can choose:

• Fastresume files (traditional .fastresume files per torrent)
• SQLite database (stores all torrent metadata in a single SQLite database)

This PR implements a third approach within libtorrent:

• Piece cache storage - stores both resume data AND torrent pieces as blob files in a cache directory
• All done natively in libtorrent without external database dependencies

The main purpose: Enable efficient seeding when you don't want to keep the actual content files - perfect for pure seeding scenarios or temporary downloads.

How it works (vs. SQLite approach):
Instead of using SQLite's structured database tables, this implements a simpler, native libtorrent solution:

• Resume metadata stored alongside cached piece blobs in regular files
• No SQLite dependency or overhead
• Direct piece access without database queries
• Enables fileless seeding from cache-only storage

[arvidn]

this appears to be a feature of the client_test example only. It's not adding a feature to libtorrent, only to the example. It's not necessarily a problem, but it seems like quite a lot of code to be an example of how to use the disk io interface.

I think the main problem is that you clone the client_test.cpp rather than introducing this feature as an option to the existing program. Is there a reason for that? There's quite a lot of duplicated code in your PR.

Additionally, by introducing a new test, without updating the CI workflows, there's no CI coverage of this new code at all. Not building it, let alone running it.

Can you think of a way to test this feature on CI, to demonstrate it working?