Make Line Ending Explicit

[JoshClose]

RFC 4180 states that a line ending is CRLF and if a CR or LF is in a field, it doesn't need to be escaped. Currently the parser looks for CR, LF, or CRLF. This presents a problem reading a file that is actually valid where the field contains a CR or LF and the line endings are CRLF.

Changing the line ending from being either CRLF, CR, or LF, to specific string that you set that would default to CRLF would fix this issue, but could cause another. People on Linux systems that used the system line ending of LF instead of the correct CRLF would need to specify their line ending. This means files won't parse automatically. I don't know what the majority of files written on Linux use for line endings, so I don't know how prevalent this is.

Thoughts?

#1632

[JoshClose]

@StefanBertels I'd like your opinion here if have a minute.

[StefanBertels]

Hi, just checked RFC4180 and could not find that CR or LF are allowed in non-escaped.

[JoshClose]

6. Fields containing line breaks (CRLF), double quotes, and commas
   should be enclosed in double-quotes. For example:

   "aaa","b CRLF
   bb","ccc" CRLF
   zzz,yyy,xxx

It doesn't say anything about just CR or LF. You could interpret line break as CR or LF, but they do specify CRLF. Gotta love not having an exact spec.

[StefanBertels]

CR and LF are not allowed in unquoted fields if I read RFC correctly.
Look at https://tools.ietf.org/html/rfc4180#section-2 and search for "TEXTDATA".
If you look at "escaped" then you can see that CR and LF are specified as valid within quotes.

[JoshClose]

Awesome! I missed that. That'll make my life a lot easier.