Skip to content

Commit 30b939c

Browse files
peffgitster
authored andcommitted
fast-export: do not load blob objects twice
When fast-export wants to export a blob object, it first calls parse_object to get a "struct object" and check whether we have already shown the object. If we haven't shown it, we then use read_sha1_file to pull it from disk and write it out. That means we load each blob from disk twice: once for parse_object to find its type and check its sha1, and a second time when we actually output it. We can drop this to a single load by using lookup_object to check the SHOWN flag, and then checking the signature on and outputting a single buffer. This provides modest speedups on git.git (best-of-five, "git fast-export HEAD >/dev/null"): [before] [after] real 0m14.347s real 0m13.780s user 0m14.084s user 0m13.620s sys 0m0.208s sys 0m0.100s and somewhat more on more blob-heavy repos (this is a repository full of media files): [before] [after] real 0m52.236s real 0m44.451s user 0m50.568s user 0m43.000s sys 0m1.536s sys 0m1.284s Signed-off-by: Jeff King <[email protected]> Signed-off-by: Junio C Hamano <[email protected]>
1 parent f9b54e2 commit 30b939c

File tree

1 file changed

+10
-6
lines changed

1 file changed

+10
-6
lines changed

builtin/fast-export.c

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -119,23 +119,26 @@ static void export_blob(const unsigned char *sha1)
119119
enum object_type type;
120120
char *buf;
121121
struct object *object;
122+
int eaten;
122123

123124
if (no_data)
124125
return;
125126

126127
if (is_null_sha1(sha1))
127128
return;
128129

129-
object = parse_object(sha1);
130-
if (!object)
131-
die ("Could not read blob %s", sha1_to_hex(sha1));
132-
133-
if (object->flags & SHOWN)
130+
object = lookup_object(sha1);
131+
if (object && object->flags & SHOWN)
134132
return;
135133

136134
buf = read_sha1_file(sha1, &type, &size);
137135
if (!buf)
138136
die ("Could not read blob %s", sha1_to_hex(sha1));
137+
if (check_sha1_signature(sha1, buf, size, typename(type)) < 0)
138+
die("sha1 mismatch in blob %s", sha1_to_hex(sha1));
139+
object = parse_object_buffer(sha1, type, size, buf, &eaten);
140+
if (!object)
141+
die("Could not read blob %s", sha1_to_hex(sha1));
139142

140143
mark_next_object(object);
141144

@@ -147,7 +150,8 @@ static void export_blob(const unsigned char *sha1)
147150
show_progress();
148151

149152
object->flags |= SHOWN;
150-
free(buf);
153+
if (!eaten)
154+
free(buf);
151155
}
152156

153157
static int depth_first(const void *a_, const void *b_)

0 commit comments

Comments
 (0)