Regex corner case

[skeggse]

Likely inherent from limitations of javascript's regex implementation, this module does not handle greedy regular expressions (there are also probably other variants that are problematic).

// logs null [ 'one', '', 'two' ] instead of null [ 'one', 'two' ]
pull(
  pull.values(['one\n', '\ntwo']),
  split(/\n+/),
  pull.collect(console.log));

It seems difficult to appropriately handle this, so perhaps we should simply document this shortcoming?

[dominictarr]

Hmm, well you could handle this particular case by having a way to trim the start of inputs (reasonable, would merge a PR for this)

However, if you had an exact splitter such as \n\n (I often use this because then you can have pretty printed json, which is viably human readable) then an empty string between that could be significant.

[skeggse]

The given example was mostly to demonstrate this problem - best I can tell, you can't use this library will any arbitrary regex and expect it to work.

I'm not sure what you mean by "trim the start of inputs." Are you proposing an option that removes a given string or regex from the beginning of a value? How would that handle pull.values(['one', '\ntwo'])?

[dominictarr]

Ah... it would only use the trimmer if it had just handled a break. so 'one\n' '\ntwo' would trigger at the start of the second string. 'one\n\nt' 'wo' it would call it with t, and 'one\n\n' '\n\n' '\ntwo' would call it with \n\n (it would return undefined, no output) then \ntwo (would respond with two)

Yeah, I could imagine more complex cases where splitting was even more abigious, but you'd have to convice me that "arbitary" regexp was a good idea. Simpler is better, especially with regexp! I'm kinda curious exactly where the bounds are here what is the simplest regexp that would be ambigious?

Another technique would be to use the regexp to match the item not the separator, then you could match a sequence of non \n characters, this has it's limitations also, of course.

[skeggse]

// logs null [ 'one', '', 'two' ] instead of [ 'one', 'two' ]
pull(
  pull.values(['one\r\n\r', '\ntwo']),
  split(/(?:\r\n)+/),
  pull.collect(console.log));

// logs null [ 'one', '\ntwo' ] (as expected)
pull(
  pull.values(['one\r\n', '\ntwo']),
  split(/(?:\r\n)+/),
  pull.collect(console.log));

For the above examples, what would you suggest for trimming? This might seem somewhat contrived, and if so I can try to come up with a better example.

If you just do a general String#trim, the second example won't have its prefixed newline. This requires a more implementation-specific trim, which takes as input the most recently matched separator, which would further complicate the trimmer.