avoid possible overflow in delta size filtering computation

On a 32-bit system, the maximum possible size for an object is less than
4GB, while 64-bit systems may cope with larger objects.  Due to this
limitation, variables holding object sizes are using an unsigned long
type (32 bits on 32-bit systems, or 64 bits on 64-bit systems).

When large objects are encountered, and/or people play with large delta
depth values, it is possible for the maximum allowed delta size
computation to overflow, especially on a 32-bit system.  When this
occurs, surviving result bits may represent a value much smaller than
what it is supposed to be, or even zero.  This prevents some objects
from being deltified although they do get deltified when a smaller depth
limit is used.  Fix this by always performing a 64-bit multiplication.

Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Junio C Hamano <[email protected]>

Author: Nicolas Pitre

builtin-pack-objects.c

@@ -1265,7 +1265,7 @@ static int try_delta(struct unpacked *trg, struct unpacked *src,
 		max_size = trg_entry->delta_size;
 		ref_depth = trg->depth;
 	}
-	max_size = max_size * (max_depth - src->depth) /
+	max_size = (uint64_t)max_size * (max_depth - src->depth) /
 						(max_depth - ref_depth + 1);
 	if (max_size == 0)
 		return 0;