FileStorage

To avoid too memoy consume, LiteDB has a limit document size in 1Mb. To text documents, this is a huge size. But for store files, 1Mb is too small. So, LiteDB implements FileStorage, a custom colleciton to store files and streams.

LiteDB use 2 special collections to split files in 1Mb each chunck.

_files collection store file reference and metadata only

{
    _id: "my-photo",
    filename: "my-photo.jpg",
    mimeType: "image/jpg",
    length: { $numberLong: "2340000" },
    uploadDate: { $date: "2015-01-01T00:00:00Z" },
    metadata: { "key1": "value1", "key2": "value2" }
}

_chunks collection stores binary data in 1MB each chunck.

{
    _id: "my-photo\00001",
    data: { $binary: "VHlwZSAob3Igc ... GUpIGhlcmUuLi4" }
}
{
    _id: "my-photo\00002",
    data: { $binary: "pGaGhlcmUuLi4 ... VHlwZSAob3Igc" }
}
{
   ...
}

Each file in FileStorage are identified by an _id. This _id must be a string value, with folling rules:

• Starts with a letter, number, _, -, $, @, !, +, %, ; or .
• If contains a /, must be sequence with chars above

To better organize many files, you can use _id as a directory/file_id. This will be a great solution to quick find all files in a directory using Find method.

Example: $/photos/2014/picture-01.jpg

To access files, FileStorage collection contains simple methods like:

• Upload: Send file or stream to database. Can be used with file or Stream
• SetMetadata: Update a stored file metadata. This method doesn't change data bytes, affect only _files.metadata
• Download: Get your file from database.
• Delete: Delete a file reference and all data chunks
• Find: Find one or many files in _files collection. Returns LiteFileInfo class, that can be download data after.

// Upload a file from file system
db.FileStorage.Upload("$/photos/2014/picture-01.jpg", @"C:\Temp\picture-01.jpg");

// Upload a file from a Stream
db.FileStorage.Upload("$/photos/2014/picture-01.jpg", strem);

// Find file reference only (no binary data yet) - returns null if not found
var file = db.FileStoage.FindById("$/photos/2014/picture-01.jpg");

// Now, load data and save to file system
file.SaveAs(@"C:\Temp\new-picture.jpg");

// Or get data as Stream and copy to another Stream
using(var stream = file.OpenRead())
{
    stream.CopyTo(Response.OutputStream);
}

// Find all files references in a "directory"
var files = db.FileStoage.Find("$/photos/2014/");

FileStorage do not support transaction. The reason for that is to avoid put all file in memory before store in disk. Transactions are used per chunck. Each uploaded chunck are in a single transaction.

FileStorage keeps _files.length in 0 during _chuncks uploads. When all chuncks finish FileStorage updates _files.length to real data size. If you try to download a zero bytes length file, the uploaded file are corrupted. Corrupted files doesn't mean corrupt database.