make --max-pack-size argument to 'git pack-object' count in bytes

The value passed to --max-pack-size used to count in MiB which was
inconsistent with the corresponding configuration variable as well as
other command arguments which are defined to count in bytes with an
optional unit suffix.  This brings --max-pack-size in line with the
rest of Git.

Also, in order not to cause havoc with people used to the previous
megabyte scale, and because this is a sane thing to do anyway, a
minimum size of 1 MiB is enforced to avoid an explosion of pack files.

Adjust and extend test suite accordingly.

Signed-off-by: Nicolas Pitre <[email protected]>
Signed-off-by: Junio C Hamano <[email protected]>

Author: npitre

Documentation/RelNotes-1.7.0.txt

@@ -38,14 +38,20 @@ Notes on behaviour change
    whitespaces is reported with zero exit status when run with
    --exit-code, and there is no "diff --git" header for such a change.
 
- * external diff and textconv helpers are now executed using the shell.
+ * External diff and textconv helpers are now executed using the shell.
    This makes them consistent with other programs executed by git, and
    allows you to pass command-line parameters to the helpers. Any helper
    paths containing spaces or other metacharacters now need to be
    shell-quoted.  The affected helpers are GIT_EXTERNAL_DIFF in the
    environment, and diff.*.command and diff.*.textconv in the config
    file.
 
+ * The --max-pack-size argument to 'git repack' and 'git pack-objects' was
+   assuming the provided size to be expressed in MiB, unlike the
+   corresponding config variable and other similar options accepting a size
+   value.  It is now expecting a size expressed in bytes, with a possible
+   unit suffix of 'k', 'm', or 'g'.
+
 Updates since v1.6.6
 --------------------
 

Documentation/config.txt

@@ -1368,10 +1368,13 @@ you can use linkgit:git-index-pack[1] on the *.pack file to regenerate
 the `{asterisk}.idx` file.
 
 pack.packSizeLimit::
-	The default maximum size of a pack.  This setting only affects
-	packing to a file, i.e. the git:// protocol is unaffected.  It
-	can be overridden by the `\--max-pack-size` option of
-	linkgit:git-repack[1].
+	The maximum size of a pack.  This setting only affects
+	packing to a file when repacking, i.e. the git:// protocol
+	is unaffected.  It can be overridden by the `\--max-pack-size`
+	option of linkgit:git-repack[1]. The minimum size allowed is
+	limited to 1 MiB. The default is unlimited.
+	Common unit suffixes of 'k', 'm', or 'g' are
+	supported.
 
 pager.<cmd>::
 	Allows turning on or off pagination of the output of a

Documentation/git-pack-objects.txt

@@ -105,8 +105,9 @@ base-name::
 	`--window-memory=0` makes memory usage unlimited, which is the
 	default.
 
---max-pack-size=<n>::
-	Maximum size of each output packfile, expressed in MiB.
+--max-pack-size=[N]::
+	Maximum size of each output pack file. The size can be suffixed with
+	"k", "m", or "g". The minimum size allowed is limited to 1 MiB.
 	If specified,  multiple packfiles may be created.
 	The default is unlimited, unless the config variable
 	`pack.packSizeLimit` is set.

Documentation/git-repack.txt

@@ -98,10 +98,12 @@ other objects in that pack they already have locally.
 	`--window-memory=0` makes memory usage unlimited, which is the
 	default.
 
---max-pack-size=<n>::
-	Maximum size of each output packfile, expressed in MiB.
+--max-pack-size=[N]::
+	Maximum size of each output pack file. The size can be suffixed with
+	"k", "m", or "g". The minimum size allowed is limited to 1 MiB.
 	If specified,  multiple packfiles may be created.
-	The default is unlimited.
+	The default is unlimited, unless the config variable
+	`pack.packSizeLimit` is set.
 
 
 Configuration

builtin-pack-objects.c

@@ -77,7 +77,7 @@ static int allow_ofs_delta;
 static const char *base_name;
 static int progress = 1;
 static int window = 10;
-static uint32_t pack_size_limit, pack_size_limit_cfg;
+static unsigned long pack_size_limit, pack_size_limit_cfg;
 static int depth = 50;
 static int delta_search_threads;
 static int pack_to_stdout;
@@ -2192,10 +2192,8 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 			continue;
 		}
 		if (!prefixcmp(arg, "--max-pack-size=")) {
-			char *end;
 			pack_size_limit_cfg = 0;
-			pack_size_limit = strtoul(arg+16, &end, 0) * 1024 * 1024;
-			if (!arg[16] || *end)
+			if (!git_parse_ulong(arg+16, &pack_size_limit))
 				usage(pack_usage);
 			continue;
 		}
@@ -2335,9 +2333,12 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 
 	if (!pack_to_stdout && !pack_size_limit)
 		pack_size_limit = pack_size_limit_cfg;
-
 	if (pack_to_stdout && pack_size_limit)
 		die("--max-pack-size cannot be used to build a pack for transfer.");
+	if (pack_size_limit && pack_size_limit < 1024*1024) {
+		warning("minimum pack size limit is 1 MiB");
+		pack_size_limit = 1024*1024;
+	}
 
 	if (!pack_to_stdout && thin)
 		die("--thin cannot be used to build an indexable pack.");

t/t5300-pack-object.sh

@@ -16,7 +16,9 @@ test_expect_success \
      perl -e "print \"a\" x 4096;" > a &&
      perl -e "print \"b\" x 4096;" > b &&
      perl -e "print \"c\" x 4096;" > c &&
-     git update-index --add a b c &&
+     test-genrandom "seed a" 2097152 > a_big &&
+     test-genrandom "seed b" 2097152 > b_big &&
+     git update-index --add a a_big b b_big c &&
      cat c >d && echo foo >>d && git update-index --add d &&
      tree=`git write-tree` &&
      commit=`git commit-tree $tree </dev/null` && {
@@ -375,19 +377,19 @@ test_expect_success 'index-pack with --strict' '
 '
 
 test_expect_success 'honor pack.packSizeLimit' '
-	git config pack.packSizeLimit 200 &&
+	git config pack.packSizeLimit 3m &&
 	packname_10=$(git pack-objects test-10 <obj-list) &&
-	test 3 = $(ls test-10-*.pack | wc -l)
+	test 2 = $(ls test-10-*.pack | wc -l)
 '
 
 test_expect_success 'verify resulting packs' '
 	git verify-pack test-10-*.pack
 '
 
-test_expect_success 'tolerate absurdly small packsizelimit' '
-	git config pack.packSizeLimit 2 &&
+test_expect_success 'tolerate packsizelimit smaller than biggest object' '
+	git config pack.packSizeLimit 1 &&
 	packname_11=$(git pack-objects test-11 <obj-list) &&
-	test $(wc -l <obj-list) = $(ls test-11-*.pack | wc -l)
+	test 3 = $(ls test-11-*.pack | wc -l)
 '
 
 test_expect_success 'verify resulting packs' '