Transmitting the parity shards when an object doesn't need to be fixed

[zzhpro]

In the implementation of GetObject, it seems that parity shards of an object are transmitted to the coordinator regardless of whether the object need to be fixed. This wastes network bandwidth and can harm cluster performance. Is there a reason for doing this or simply am I wrong about the behavior?

[weisd]

to ensure the integrity of the data, it is necessary.

[zzhpro]

Thanks for your reply!

The code fragment of decode method of LowRateDecoder in reed-solomon-simd shows that, if there are enough original data shards, it will return them directly, without recalculating and matching the parity shards. So in the normal case, passing the parity shards doesn't help us verify the integrity of the object.

fn decode(&mut self) -> Result<DecoderResult, Error> {
        let Some((mut work, original_count, recovery_count, received)) =
            self.work.decode_begin()?
        else {
            // Nothing to do, original data is complete.
            return Ok(DecoderResult::new(&mut self.work));
        };
// ignored
}

If we can distinguish the majority case of normal objects, and the rare case of damaged objects, and only transmit the parity shards in the second case, we can save some network IO. Actually, MinIO doesn't transmit the parity shards when an object isn't corrupted.

[weisd]

You are right. There is indeed room for optimization here. As long as enough shards are obtained, the complete data can be decoded.

[loverustfs]

Thank you @zzhpro @weisd .