Ending position of nested list item appears to be incorrect when followed by a non-list container.

[c-forrest]

Hello! Thank you a lot for this excellent repo!

I recently encountered a seemingly incorrect behavior of this micromark repo but I do not know whether this is expected and/or how to avoid it.

If we are using remark to parse the following document:

- a
  - b
    - b1
  - c
    - c1
  > e

with the following script:

import { unified } from 'unified'
import remarkParse from 'remark-parse'
import remarkStringify from 'remark-stringify'

import { read } from 'to-vfile'
import { inspect } from 'unist-util-inspect'

const processor = unified()
  .use(remarkParse)
  .use(remarkStringify)

const tree = processor.parse(await read('example.md'))
console.log(inspect(tree))

we might get the following result: (some lines omitted)

root[1] (1:1-6:6, 0-39)
└─0 list[1] (1:1-6:6, 0-39)
    └─0 listItem[3] (1:1-6:6, 0-39)
        ├─0 paragraph[1] (1:3-1:4, 2-3)
        │   └─0 text "a" (1:3-1:4, 2-3)
        ├─1 list[2] (2:3-6:3, 6-36)
        │   ├─0 listItem[2] (2:3-3:9, 6-18)
        │   │   ├─0 paragraph[1] (2:5-2:6, 8-9)
        │   │   │   └─0 text "b" (2:5-2:6, 8-9)
        │   │   └─1 list[1] (3:5-3:9, 14-18)
        │   │       └─0 listItem[1] (3:5-3:9, 14-18)
        │   │           └─0 paragraph[1] (3:7-3:9, 16-18)
        │   │               └─0 text "b1" (3:7-3:9, 16-18)
        │   └─1 listItem[2] (4:3-6:3, 21-36)
        │       ├─0 paragraph[1] (4:5-4:6, 23-24)
        │       │   └─0 text "c" (4:5-4:6, 23-24)
        │       └─1 list[1] (5:5-6:3, 29-36)
        │           └─0 listItem[1] (5:5-5:9, 29-33)
        │               └─0 paragraph[1] (5:7-5:9, 31-33)
        │                   └─0 text "c1" (5:7-5:9, 31-33)
        └─2 blockquote[1] (6:3-6:6, 36-39)
            └─0 paragraph[1] (6:5-6:6, 38-39)
                └─0 text "e" (6:5-6:6, 38-39)

It appears to me that the list item c ends at the wrong position. It should be the same as the ending position of c1 instead of the current 6:3, which is the starting position of the block quote. This shall be compared with the list item b, which ends at 3:9 instead of 4:3.

This troubles me because I want to use this tool to format my markdown files. This will cause the program to wrongly identify a nonspread list to be a spread one, and add unintended empty lines between those list items.

Thus, I tried to figure out what happens, but I quickly got lost in the micromark source codes. As far as I understand, the micromark context pushes an exit event of the listUnordered container when it finds a line not belonging to the current list. But this occurs after it has touches the starting position of next container (i.e., the blockQuote container here). Then, it adjusts the position of the exit event through documentContinue function and some postprocesses. However, in this case, for whatever reason, when exiting the listUnordered container, the documentContinue function does not read a self.containerState._closeFlow marker and adjust the ending position of the previous container. And all the following position fixing works are not executed.

This is possibly caused by the fact that the notInCurrentItem function is not executed (or later reverted) when the list construct finds the next line is not another list item.

I did not know how to proceed now, so I need your help. Could you please let me know whether this is expected or not, and what is the minimal patch to adjust the ending position in the way I expect?

Actually I am trying to write a new micromark extension for more markdown extensions, but I won't bother you with the details because I find exactly the same thing occurs within the original micromark framework. Therefore, an easy fix without touching codes of the micromark is greatly appreciated, so that I can simply add a patch in my own repo.

Thanks in advance!