line-log: fix assertion error

When line-level log is invoked with more than one disjoint line range
in the same file, and one of the commits happens to change that file
such that:

  - the last line of a line range R(n) immediately preceeds the first line
    modified or added by a hunk H, and
  - subtracting the number of lines added by hunk H from the start and
    end of the subsequent line range R(n+1) would result in a range
    overlapping with line range R(n),

then git aborts with an assertion error, because those overlapping
line ranges violate the invariants:

  $ git log --oneline -p
  73e4e2f (HEAD -> master) Add lines 6 7 8 9 10
  diff --git a/file b/file
  index 572d5d9..00935f1 100644
  --- a/file
  +++ b/file
  @@ -3,3 +3,8 @@ Line 2
   Line 3
   Line 4
   Line 5
  +Line 6
  +Line 7
  +Line 8
  +Line 9
  +Line 10
  66e3561 Add lines 1 2 3 4 5
  diff --git a/file b/file
  new file mode 100644
  index 0000000..572d5d9
  --- /dev/null
  +++ b/file
  @@ -0,0 +1,5 @@
  +Line 1
  +Line 2
  +Line 3
  +Line 4
  +Line 5
  $ git log --oneline -L3,5:file -L7,8:file
  git: line-log.c:73: range_set_append: Assertion `rs->nr == 0 || rs->ranges[rs->nr-1].end <= a' failed.
  Aborted (core dumped)

The line-log machinery encodes line and diff ranges internally as
[start, end) pairs, i.e. include 'start' but exclude 'end', and line
numbering starts at 0 (as opposed to the -LX,Y option, where it starts
at 1, IOW the parameter -L3,5 is represented internally as { start =
2, end = 5 }).

The reason for this assertion error and some related issues is that
there are a couple of places where 'end' is mistakenly considered to
be part of the range:

  - When a commit modifies an interesting path, the line-log machinery
    first checks which diff range (i.e. hunk) modify any line ranges.
    This is done in diff_ranges_filter_touched(), where the outer loop
    iterates over the diff ranges, and in each iteration the inner
    loop advances the line ranges supposedly until the current line
    range ends at or after the current diff range starts, and then the
    current diff and line ranges are checked for overlap.

    For HEAD in the above example the first line range [2, 5) ends
    just before the diff range [5, 10) starts, so the inner loop
    should advance, and then the second line range [6, 8) and the diff
    range should be checked for overlap.

    Unfortunately, the condition of the inner loop mistakenly
    considers 'end' as part of the line range, and, seeing the diff
    range starting at 5 and the line range ending at 5, it doesn't
    skip the first range.  Consequently, the diff range and the first
    line range are checked for overlap, and after that the outer loop
    runs out of diff ranges, and then the processing goes on in the
    false belief that this commit didn't touch any of the interesting
    line ranges.

    The line-log machinery later shifts the line ranges to account for
    any added/removed lines in the diff ranges preceeding each line
    range.  This leaves the first line range intact, but attempts to
    shift the second line range [6, 8) by 5 lines towards the
    beginning of the file, resulting in [1, 3), triggering the
    assertion error, because the two overlapping line ranges violate
    the invariants.

    Fix that loop condition in diff_ranges_filter_touched() to not
    treat 'end' as part of the line range.

  - With the above fix the assertion error is gone... but, alas, we
    now get stuck in an endless loop!

    This happens in range_set_difference(), where a couple of nested
    loops iterate over the line and diff ranges, and a condition is
    supposed to break the middle loop when the current line range ends
    before the current diff range, so processing could continue with
    the next line range.

    For HEAD in the above example the first line range [2, 5) ends
    just before the diff range [5, 10) starts, so this condition
    should trigger and break the middle loop.

    Unfortunately, just like in the case of the assertion error, this
    conditions mistakenly considers 'end' as part of the line range,
    and, seeing the line range ending at 5 and the diff range starting
    at 5, it doesn't break the loop, which will then go on and on.

    Fix this condition in range_set_difference() to not treat 'end' as
    part of the line range.

  - With the above fix the endless loop is gone... but, alas, the
    output is now wrong, as it shows both line ranges for HEAD, even
    though the first line range is not modified by that commit:

      $ git log --oneline -L3,5:file -L7,8:file
      73e4e2f (HEAD -> master) Add lines 6 7 8 9 10

      diff --git a/file b/file
      --- a/file
      +++ b/file
      @@ -3,3 +3,3 @@
       Line 3
       Line 4
       Line 5
      @@ -6,0 +7,2 @@
      +Line 7
      +Line 8
      66e3561 Add lines 1 2 3 4 5

      diff --git a/file b/file
      --- /dev/null
      +++ b/file
      @@ -0,0 +3,3 @@
      +Line 3
      +Line 4
      +Line 5

    In dump_diff_hacky_one() a couple of nested loops are responsible
    for finding and printing the modified line ranges: the big outer
    loop iterates over all line ranges, and the first inner loop skips
    over the diff ranges that end before the start of the current line
    range.  This is followed by a condition checking whether the
    current diff range starts after the end of the current line range,
    which, when fulfilled, continues and advances the outer loop to
    the next line range.

    For HEAD in the above example the first line range [2, 5) ends
    just before the diff range [5, 10), so this condition should
    trigger, and the outer loop should advance to the second line
    range.

    Unfortunately, just like in the previous cases, this condition
    mistakenly considers 'end' as part of the line range, and, seeing
    the first line range ending at 5 and the diff range starting at 5,
    it doesn't continue to advance the outher loop, but goes on to
    show the (unmodified) first line range.

    Fix this condition to not treat 'end' as part of the line range,
    just like in the previous cases.

After all this the command in the above example finally finishes and
produces the right output:

  $ git log --oneline -L3,5:file -L7,8:file
  73e4e2f (HEAD -> master) Add lines 6 7 8 9 10

  diff --git a/file b/file
  --- a/file
  +++ b/file
  @@ -6,0 +7,2 @@
  +Line 7
  +Line 8
  66e3561 Add lines 1 2 3 4 5

  diff --git a/file b/file
  --- /dev/null
  +++ b/file
  @@ -0,0 +3,3 @@
  +Line 3
  +Line 4
  +Line 5

Add a canned test similar to the above example, with the line ranges
adjusted to the test repository's history.

Reported-by: Evgeni Chasnovski <[email protected]>
Signed-off-by: SZEDER Gábor <[email protected]>
Signed-off-by: Junio C Hamano <[email protected]>

Author: szeder

line-log.c

@@ -201,7 +201,7 @@ static void range_set_difference(struct range_set *out,
 				 * b: ------|
 				 */
 				j++;
-			if (j >= b->nr || end < b->ranges[j].start) {
+			if (j >= b->nr || end <= b->ranges[j].start) {
 				/*
 				 * b exhausted, or
 				 * a:  ----|
@@ -408,7 +408,7 @@ static void diff_ranges_filter_touched(struct diff_ranges *out,
 	assert(out->target.nr == 0);
 
 	for (i = 0; i < diff->target.nr; i++) {
-		while (diff->target.ranges[i].start > rs->ranges[j].end) {
+		while (diff->target.ranges[i].start >= rs->ranges[j].end) {
 			j++;
 			if (j == rs->nr)
 				return;
@@ -941,7 +941,7 @@ static void dump_diff_hacky_one(struct rev_info *rev, struct line_log_data *rang
 
 		while (j < diff->target.nr && diff->target.ranges[j].end < t_start)
 			j++;
-		if (j == diff->target.nr || diff->target.ranges[j].start > t_end)
+		if (j == diff->target.nr || diff->target.ranges[j].start >= t_end)
 			continue;
 
 		/* Scan ahead to determine the last diff that falls in this range */

t/t4211-line-log.sh

@@ -78,6 +78,8 @@ canned_test "-L :main:a.c -L 4,18:a.c simple" multiple-overlapping
 canned_test "-L 4:a.c -L 8,12:a.c simple" multiple-superset
 canned_test "-L 8,12:a.c -L 4:a.c simple" multiple-superset
 
+canned_test "-L 10,16:b.c -L 18,26:b.c main" no-assertion-error
+
 test_bad_opts "-L" "switch.*requires a value"
 test_bad_opts "-L b.c" "argument not .start,end:file"
 test_bad_opts "-L 1:" "argument not .start,end:file"

t/t4211/sha1/expect.no-assertion-error

@@ -0,0 +1,90 @@
+commit 0d8dcfc6b968e06a27d5215bad1fdde3de9d6235
+Author: Thomas Rast <[email protected]>
+Date:   Thu Feb 28 10:50:24 2013 +0100
+
+    move within the file
+
+diff --git a/b.c b/b.c
+--- a/b.c
++++ b/b.c
+@@ -25,0 +18,9 @@
++long f(long x)
++{
++	int s = 0;
++	while (x) {
++		x /= 2;
++		s++;
++	}
++	return s;
++}
+
+commit 4659538844daa2849b1a9e7d6fadb96fcd26fc83
+Author: Thomas Rast <[email protected]>
+Date:   Thu Feb 28 10:48:43 2013 +0100
+
+    change back to complete line
+
+diff --git a/a.c b/a.c
+--- a/a.c
++++ b/a.c
+@@ -18,5 +18,7 @@
+ int main ()
+ {
+ 	printf("%ld\n", f(15));
+ 	return 0;
+-}
+\ No newline at end of file
++}
++
++/* incomplete lines are bad! */
+
+commit 100b61a6f2f720f812620a9d10afb3a960ccb73c
+Author: Thomas Rast <[email protected]>
+Date:   Thu Feb 28 10:48:10 2013 +0100
+
+    change to an incomplete line at end
+
+diff --git a/a.c b/a.c
+--- a/a.c
++++ b/a.c
+@@ -18,5 +18,5 @@
+ int main ()
+ {
+ 	printf("%ld\n", f(15));
+ 	return 0;
+-}
++}
+\ No newline at end of file
+
+commit a6eb82647d5d67f893da442f8f9375fd89a3b1e2
+Author: Thomas Rast <[email protected]>
+Date:   Thu Feb 28 10:45:16 2013 +0100
+
+    touch both functions
+
+diff --git a/a.c b/a.c
+--- a/a.c
++++ b/a.c
+@@ -17,5 +17,5 @@
+ int main ()
+ {
+-	printf("%d\n", f(15));
++	printf("%ld\n", f(15));
+ 	return 0;
+ }
+
+commit de4c48ae814792c02a49c4c3c0c757ae69c55f6a
+Author: Thomas Rast <[email protected]>
+Date:   Thu Feb 28 10:44:48 2013 +0100
+
+    initial
+
+diff --git a/a.c b/a.c
+--- /dev/null
++++ b/a.c
+@@ -0,0 +16,5 @@
++int main ()
++{
++	printf("%d\n", f(15));
++	return 0;
++}

t/t4211/sha256/expect.no-assertion-error

@@ -0,0 +1,90 @@
+commit eb871b8aa9aff323e484723039c9a92ab0266e060bc0ef2afb08fadda25c5ace
+Author: Thomas Rast <[email protected]>
+Date:   Thu Feb 28 10:50:24 2013 +0100
+
+    move within the file
+
+diff --git a/b.c b/b.c
+--- a/b.c
++++ b/b.c
+@@ -25,0 +18,9 @@
++long f(long x)
++{
++	int s = 0;
++	while (x) {
++		x /= 2;
++		s++;
++	}
++	return s;
++}
+
+commit 5526ed05c2476b56af8b7be499e8f78bd50f490740733a9ca7e1f55878fa90a9
+Author: Thomas Rast <[email protected]>
+Date:   Thu Feb 28 10:48:43 2013 +0100
+
+    change back to complete line
+
+diff --git a/a.c b/a.c
+--- a/a.c
++++ b/a.c
+@@ -18,5 +18,7 @@
+ int main ()
+ {
+ 	printf("%ld\n", f(15));
+ 	return 0;
+-}
+\ No newline at end of file
++}
++
++/* incomplete lines are bad! */
+
+commit 29f32ac3141c48b22803e5c4127b719917b67d0f8ca8c5248bebfa2a19f7da10
+Author: Thomas Rast <[email protected]>
+Date:   Thu Feb 28 10:48:10 2013 +0100
+
+    change to an incomplete line at end
+
+diff --git a/a.c b/a.c
+--- a/a.c
++++ b/a.c
+@@ -18,5 +18,5 @@
+ int main ()
+ {
+ 	printf("%ld\n", f(15));
+ 	return 0;
+-}
++}
+\ No newline at end of file
+
+commit ccf97b9878189c40a981da50b15713bb80a35755326320ec80900caf22ced46f
+Author: Thomas Rast <[email protected]>
+Date:   Thu Feb 28 10:45:16 2013 +0100
+
+    touch both functions
+
+diff --git a/a.c b/a.c
+--- a/a.c
++++ b/a.c
+@@ -17,5 +17,5 @@
+ int main ()
+ {
+-	printf("%d\n", f(15));
++	printf("%ld\n", f(15));
+ 	return 0;
+ }
+
+commit 1dd7e9b2b1699324b53b341e728653b913bc192a14dfea168c5b51f2b3d03592
+Author: Thomas Rast <[email protected]>
+Date:   Thu Feb 28 10:44:48 2013 +0100
+
+    initial
+
+diff --git a/a.c b/a.c
+--- /dev/null
++++ b/a.c
+@@ -0,0 +16,5 @@
++int main ()
++{
++	printf("%d\n", f(15));
++	return 0;
++}