fix: handle case where doc does not lex as ocaml (#2684)

* fix: handle case where doc does not lex as ocaml

Fixes #2679

Formatting `mld` files can crash when the input file cannot be lexed as
ocaml. This can happen with LaTeX is found in `{m ...}` or C code in
`{@c[ ... ]}`, for example.

In that case we would crash when building the list of tokens in the
file; however it's not used in the `mld` case.

Author: emillon

CHANGES.md

@@ -43,6 +43,10 @@ profile. This started with version 0.26.0.
 
 - Fix crash due to edge case with asterisk-prefixed comments (#2674, @Julow)
 
+
+- Fix crash when formatting `mld` files that cannot be lexed as ocaml (e.g.
+  containing LaTeX or C code) (#2684, @emillon)
+
 - \* Fix double parens around module constraint in functor application :
   `module M = F ((A : T))` becomes `module M = F (A : T)`. (#2678, @EmileTrotignon)
 

lib/Parse_with_comments.ml

@@ -95,8 +95,9 @@ let parse ?(disable_w50 = false) ?(disable_deprecated = false) parse fragment
           List.map ~f:mk_cmt (Lexer.comments ())
         in
         let tokens =
+          (* mld files can not always be lexed using the ocaml lexer *)
           let lexbuf, _ = fresh_lexbuf source in
-          tokens lexbuf
+          try tokens lexbuf with Lexer.Error _ -> []
         in
         let source = Source.create ~text:source ~tokens in
         {ast; comments; prefix= hash_bang; source} )

test/passing/refs.ahrefs/doc.mld.ref

@@ -162,3 +162,7 @@ For more about [odoc] commands, simply invoke [odoc --help] in your shell.
 
 Preserve the space between a link/reference and its text: {{:foo}bar}
 {{:foo} bar} {{!foo}bar} {{!foo} bar}
+
+{1 Some math}
+
+{m \Gamma}

test/passing/refs.default/doc.mld.ref

@@ -162,3 +162,7 @@ For more about [odoc] commands, simply invoke [odoc --help] in your shell.
 
 Preserve the space between a link/reference and its text: {{:foo}bar}
 {{:foo} bar} {{!foo}bar} {{!foo} bar}
+
+{1 Some math}
+
+{m \Gamma}

test/passing/refs.janestreet/doc.mld.ref

@@ -154,3 +154,7 @@ For more about [odoc] commands, simply invoke [odoc --help] in your shell.
 
 Preserve the space between a link/reference and its text:
 {{:foo}bar} {{:foo} bar} {{!foo}bar} {{!foo} bar}
+
+{1 Some math}
+
+{m \Gamma}

test/passing/refs.ocamlformat/doc.mld.ref

@@ -162,3 +162,7 @@ For more about [odoc] commands, simply invoke [odoc --help] in your shell.
 
 Preserve the space between a link/reference and its text: {{:foo}bar}
 {{:foo} bar} {{!foo}bar} {{!foo} bar}
+
+{1 Some math}
+
+{m \Gamma}

test/passing/tests/doc.mld

@@ -160,3 +160,7 @@ For more about [odoc] commands, simply invoke [odoc --help] in your shell.
 
 Preserve the space between a link/reference and its text:
 {{:foo}bar} {{:foo} bar} {{!foo}bar} {{!foo} bar}
+
+{1 Some math}
+
+{m \Gamma}