Issue with split cache files

[AppleSheeple]

Hello.

Media larger than ~10MiB seem to be cached to multiple files. But the decoded output doesn't seem to be correct.

With a video cached to two files, I found skipping 12 bytes from the start of the first decoded split file got me to an MP4 header. But beyond that, simply appending the second cached file (pre or post-decoding) didn't work.

[AppleSheeple]

So, I kind of figured out what's going on.

From telegram-desktop, file media/streaming/media_streaming_reader.cpp:

QByteArray Reader::Slices::serializeComplexSlice(const Slice &slice) const {
	auto result = QByteArray();
	const auto count = slice.parts.size();
	const auto intSize = sizeof(int32);
	result.reserve(count * kPartSize + 2 * intSize * (count + 1));
	const auto appendInt = [&](int value) {
		auto serialized = int32(value);
		result.append(
			reinterpret_cast<const char*>(&serialized),
			intSize);
	};
	appendInt(count);
	for (const auto &[offset, part] : slice.parts) {
		appendInt(offset);
		appendInt(part.size());
		result.append(part);
	}
	return result;
}

The relevant part is:

	appendInt(count);
	for (const auto &[offset, part] : slice.parts) {
		appendInt(offset);
		appendInt(part.size());
		result.append(part);
	}

appendInt(n) writes uint32 values (little-endian). A slice can have 64 parts max. That's the count value. And a part max size is 128*1024.

The offset value is the offset in the original (non-serialized) media file.

Note that offsets are not necessarily in order. This reader emulates a media player, so if an MP4 file has a moov atom necessary for playback at the end of the media file, the reader will seek to the end and read from there, then come back.

Now, the next split file(s) are not actually serialized. You can simply append them. But do note that you need to discard the forward seeks mentioned above. The first serialized file shouldn't be writing anything in order larger than 8MiB. So taking the first 8MiBs from the deserialized output of the first file, then appending the next split files should give you a full working file.

Final note, there are a few bytes left after the parsed slices in the serialized file. I don't know what they are. But simply discarding them worked for me.

If anyone is interested, I wrote a quick rust program that writes a deserialized file from a serialized one.

[superalino]

If anyone is interested, I wrote a quick rust program that writes a deserialized file from a serialized one.

Hi, yes I'm interested, can you share please the script?

[AppleSheeple]

@superalino

https://github.com/AppleSheeple/telegram-media-deserialize