Fix multiline expectations (#35)

* Allows using leading comments in expected output for editor style

* Updated expected generated tests

* Fix typo in docstring

* Add unit test for removing leading comments

* Update expected generated tests

* Allow multiple leading comment characters

* Add test cases

* Add integration test case for leading semicolon support for adoc

* Add test cases based on more involved mixing of eval and stdout

* docs: examples: add section to explain multiline

Create a dedicated section to explain multiline output for example.md
and example.adoc.

These both explain to the user and act as tests for test-doc-blocks.

* Multiline eval expectations complete when parsable

We now know  that an eval expectation is complete when its form
parses successfully.

Added the lovely nubank matcher combinators dep to help with test
assertions.

Now that things are a bit more complicated, algorithm and naming in
code. It might be a bit easier to follow now? Hope so.

* review feedback: fix typo

* doc: examples: update single semicolon advice

We now would only ever need the single semicolon distinguisher for
`=stdout=>` and `=stderr=>` blocks.

* lint: add nubank/matcher-combinators config

* doc: changelog

Closes #33
---------

Co-authored-by: lread <[email protected]>

Author: mattiuusitalo

.clj-kondo/imports/nubank/matcher-combinators/config.edn

@@ -0,0 +1,4 @@
+{:linters
+ {:unresolved-symbol
+  {:exclude [(cljs.test/is [match? thrown-match?])
+             (clojure.test/is [match? thrown-match?])]}}}

CHANGELOG.adoc

@@ -21,7 +21,11 @@ A release with known breaking changes is marked with:
 
 * Other changes
 ** Test-doc-blocks now correctly specifies `true` for ClojureScript `include-macros`
+https://github.com/lread/test-doc-blocks/issues/29[#29]
 (thanks for raising https://github.com/NoahTheDuke[@NoahTheDuke]! thanks for fixing https://github.com/mattiuusitalo[@mattiuusitalo]!)
+** Editor-style multiline expectations can be fully commented
+https://github.com/lread/test-doc-blocks/issues/33[#33]
+(thanks https://github.com/mattiuusitalo[@mattiuusitalo]!)
 ** Bumped deps (used during test generation only)
 
 == v1.1.20 - 2024-05-06 [[v1.1.20]]

deps.edn

@@ -62,7 +62,8 @@
            ;;
 
            ;; for kaocha see also tests.edn
-           :kaocha {:extra-deps {lambdaisland/kaocha {:mvn/version "1.91.1392"}}
+           :kaocha {:extra-deps {lambdaisland/kaocha {:mvn/version "1.91.1392"}
+                                 nubank/matcher-combinators {:mvn/version "3.9.1"}}
                     :extra-paths ["test"]
                     :main-opts ["-m" "kaocha.runner"]}
 

doc/example.adoc

@@ -43,6 +43,11 @@ user=> (+ 1 2 3)
 (+ 1 2 3 4)
 ;; => 10
 
+;; the expected value can span multiple lines
+(assoc {} :foo :bar :quu :baz)
+;; => {:foo :bar
+;;     :quu :baz}
+
 ;; it understands that Clojure and ClojureScript can evaluate differently
 \C
 ;; =clj=> \C
@@ -98,6 +103,54 @@ It is sometimes just nice to know that your code snippet will run without except
      (apply str))
 ----
 
+== Multiline Expectations
+
+It is normal for `=stdout=>` and `=stderr=>` output to span multiple lines.
+You can span other expected output over multiple lines for readability.
+
+//#:test-doc-blocks {:platform :clj :test-ns example-adoc-out-test}
+[source,clj]
+----
+;; A snippit of Clojure where we check result, stderr and stdout
+(do
+  (println "Hi ho\nHi ho\nTo out I will go")
+  (binding [*out* *err*] (println "To err is human\nTo forgive\nIs devine"))
+  (range 1 16))
+;; => ( 1  2  3  4  5
+;;      6  7  8  9 10
+;;     11 12 13 14 15)
+;; =stdout=> Hi ho
+;; Hi ho
+;; To out I will go
+;; =stderr=>
+;; To err is human
+;; To forgive
+;; Is devine
+----
+
+If your expected stdout or stderr contains subsequent lines starting with test-doc-block `+=*>+` markers, you can distinguish by using a single semicolon prefix for subsequent lines:
+
+//#:test-doc-blocks {:platform :clj :test-ns example-adoc-out-test}
+[source,clj]
+----
+;; A snippit of Clojure where we check result, stderr and stdout
+(do
+  (println "Hi ho\nHi ho\nTo out I will go\n=stderr=> still part of stdout")
+  (binding [*out* *err*] (println "To err is human\nTo forgive\nIs devine\n=> part of stderr"))
+  [:bip :bop '=stdout> :bap :borp])
+;; => [:bip :bop
+;;     =stdout> :bap
+;;     :borp]
+;; =stdout=> Hi ho
+; Hi ho
+; To out I will go
+; =stderr=> still part of stdout
+;; =stderr=>
+; To err is human
+; To forgive
+; Is devine
+; => part of stderr
+----
 
 == Inline Options
 You can, to some limited extent, communicate your intent to test-doc-blocks.

doc/example.md

@@ -37,6 +37,11 @@ Within your Clojure code blocks, test-doc-blocks automatically generates asserti
    (+ 1 2 3 4)
    ;; => 10
 
+   ;; the expected value can span multiple lines
+   (assoc {} :foo :bar :quu :baz)
+   ;; => {:foo :bar
+   ;;     :quu :baz}
+
    ;; it understands that Clojure and ClojureScript can evaluate differently
    \C
    ;; =clj=> \C
@@ -52,7 +57,6 @@ Within your Clojure code blocks, test-doc-blocks automatically generates asserti
    ; is this right?
    ; or not?
    ```
-
    Sometimes we might care about evaluated result, stderr and stdout.
    <!-- #:test-doc-blocks {:platform :clj :test-ns example-md-out-test} -->
    ```clj
@@ -65,7 +69,6 @@ Within your Clojure code blocks, test-doc-blocks automatically generates asserti
    ;; =stderr=> To err is human
    ;; =stdout=> To out I go
    ```
-
    <!-- #:test-doc-blocks {:platform :cljs :test-ns example-md-out-test} -->
    ```cljs
    ;; And the same idea for ClojureScript
@@ -87,6 +90,51 @@ It is sometimes just nice to know that your code snippet will run without except
      reverse
      (apply str))
 ```
+## Multiline Expectations
+
+It is normal for `=stdout=>` and `=stderr=>` output to span multiple lines.
+You can span other expected output over multiple lines for readability.
+
+<!-- #:test-doc-blocks {:platform :clj :test-ns example-md-out-test} -->
+```clj
+;; A snippit of Clojure where we check result, stderr and stdout
+(do
+  (println "Hi ho\nHi ho\nTo out I will go")
+  (binding [*out* *err*] (println "To err is human\nTo forgive\nIs devine"))
+  (range 1 16))
+;; => ( 1  2  3  4  5
+;;      6  7  8  9 10
+;;     11 12 13 14 15) 
+;; =stdout=> Hi ho
+;; Hi ho
+;; To out I will go
+;; =stderr=> 
+;; To err is human
+;; To forgive
+;; Is devine
+```
+If your expected stdout or stderr contains subsequent lines starting with test-doc-block `+=*>+` markers, you can distinguish by using a single semicolon prefix for subsequent lines:
+
+<!-- #:test-doc-blocks {:platform :clj :test-ns example-md-out-test} -->
+```clj
+;; A snippit of Clojure where we check result, stderr and stdout
+(do
+  (println "Hi ho\nHi ho\nTo out I will go\n=stderr=> still part of stdout")
+  (binding [*out* *err*] (println "To err is human\nTo forgive\nIs devine\n=> part of stderr"))
+  [:bip :bop '=stdout> :bap :borp])
+;; => [:bip :bop
+;;     =stdout> :bap
+;;     :borp]
+;; =stdout=> Hi ho
+; Hi ho
+; To out I will go
+; =stderr=> still part of stdout
+;; =stderr=> 
+; To err is human
+; To forgive
+; Is devine
+; => part of stderr
+```
 
 ## Inline Options
 You can, to some limited extent, communicate your intent to test-doc-blocks.

src/lread/test_doc_blocks/impl/body_prep.clj

@@ -1,6 +1,14 @@
 (ns ^:no-doc lread.test-doc-blocks.impl.body-prep
   "Prep a doc block to be converted into a test."
-  (:require [clojure.string :as string]))
+  (:require [clojure.string :as string]
+            [rewrite-clj.zip :as z]))
+
+(defn- parseable? [s]
+  (try
+    (let [zloc (z/of-string s)]
+      (not= :forms (z/tag zloc)))
+    (catch Exception _ex
+      false)))
 
 (defn ^:no-doc prep-block-for-conversion-to-test
   "Convert doc block to Clojure that can be more easily parsed.
@@ -29,62 +37,128 @@
   "
   [block-text]
   (let [re-editor-style-out-expected #"^\s*(?:;;\s*){0,1}(=stdout=>|=stderr=>)(?:\s*$| {0,1}(.*))"
-        re-out-continue #"^\s*;(?:\s*$| {0,1}(.*))"
-        re-editor-style-expected #"^\s*(?:;;\s*){0,1}(=clj=>|=cljs=>|=>)\s*(.*$)"]
-    (-> (loop [acc {:body ""}
+        re-line-continue             #"^\s*;+(?:\s*$| {0,1}(.*))"
+        re-editor-style-expected     #"^\s*(?:;;\s*){0,1}(=clj=>|=cljs=>|=>)\s*(.*$)"]
+    (-> (loop [{:keys [body parsing assert-token expectation eo-assert?] :as state} {:body ""}
                [line :as lines] (string/split-lines block-text)]
           (let [line (and line (string/trimr line))
-                [_ assert-token payload] (when line
-                                           (or (re-matches re-editor-style-expected line)
-                                               (re-matches re-editor-style-out-expected line)))]
+                [_ line-assert-token line-payload] (when line
+                                                     (or (re-matches re-editor-style-expected line)
+                                                         (re-matches re-editor-style-out-expected line)))]
             (cond
-              ;; out expectation ends
-              (and (:out acc) (or (not line)
-                                  assert-token
-                                  (not (re-matches re-out-continue line))
-                                  (re-matches re-editor-style-expected line)))
-              (recur (-> acc
-                         (update :body str (str (:out-token acc) " "
-                                                (conj (:out acc)) "\n"))
-                         (dissoc :out :out-token))
-                     lines)
+              (= :eval-expectation parsing)
+              (cond
+                ;; end of eval expectation?
+                (or (not line) eo-assert?)
+                (recur {:body (str body
+                                   assert-token " "
+                                   (string/join "\n" expectation)
+                                   "\n")}
+                       lines)
+
+                :else
+                (let [[_ line-continue] (re-matches re-line-continue line)
+                      eval-line (or line-continue line)]
+                  (recur (let [{:keys [expectation] :as state} (update state :expectation conj (or eval-line ""))]
+                           (if (parseable? (string/join "\n" expectation))
+                             (assoc state :eo-assert? true)
+                             state))
+                         (rest lines))))
+
+              (= :out-expectation parsing)
+              (cond
+                ;; end of expecation?
+                (or (not line)
+                    line-assert-token
+                    eo-assert?
+                    (not (re-matches re-line-continue line)))
+                (recur {:body (str body
+                                   assert-token " "
+                                   (conj expectation)
+                                   "\n")}
+                       lines)
+
+                (re-matches re-line-continue line)
+                (let [[_ line-continue] (re-matches re-line-continue line)]
+                  (recur (update state :expectation conj (or line-continue ""))
+                         (rest lines)))
+
+                :else
+                (recur (assoc state :eo-assert? true) lines))
 
               ;; all done?
               (not line)
-              acc
-
-              ;; collecting stdout/stderr expectation
-              (and (:out acc)
-                   (not assert-token)
-                   (re-matches re-out-continue line))
-              (let [[_ out-line] (re-matches re-out-continue line)]
-                (recur (update acc :out conj (or out-line ""))
-                       (rest lines)))
-
-              ;; out expectation starts
-              (or (= "=stdout=>" assert-token)
-                  (= "=stderr=>" assert-token))
-              (recur (assoc acc
-                            :out (if payload [payload] [])
-                            :out-token assert-token)
+              state
+
+              ;; start of out expection
+              (or (= "=stdout=>" line-assert-token)
+                  (= "=stderr=>" line-assert-token))
+              (recur (assoc state
+                            :parsing :out-expectation
+                            :expectation (if line-payload [line-payload] [])
+                            :assert-token line-assert-token)
                      (rest lines))
 
-              ;; editor style evaluation expectation:
-              ;; actual
-              ;; ;;=> expected
-              assert-token
-              (recur (update acc :body str (str assert-token " " payload "\n"))
+              ;; editor style expectation
+              line-assert-token
+              (recur (let [{:keys [expectation] :as state} (assoc state
+                                                                  :parsing :eval-expectation
+                                                                  :expectation (if line-payload [line-payload] [])
+                                                                  :assert-token line-assert-token)]
+                       (if (parseable? (string/join "\n" expectation))
+                         (assoc state :eo-assert? true)
+                         state))
                      (rest lines))
 
-              ;; other lines
               :else
-              (recur (update acc :body str (str line "\n"))
+              (recur (update state :body str (str line "\n"))
                      (rest lines)))))
         :body)))
 
 
 (comment
-  (prep-block-for-conversion-to-test ";; =stdout=> line1")
-
-
-  )
+  (parseable? " , ")
+
+  (prep-block-for-conversion-to-test ";; =stdout=> line1
+; line2")
+  ;; => "=stdout=> [\"line1\" \"line2\"]\n"
+
+  (prep-block-for-conversion-to-test
+   "(assoc {} :foo :bar :baz :kikka)
+;; => {:foo :bar
+;  :baz :kikka}"
+   )
+  ;; => "(assoc {} :foo :bar :baz :kikka)\n=> {:foo :bar\n :baz :kikka}\n"
+
+  (prep-block-for-conversion-to-test
+   "(assoc {} :foo :bar :baz :kikka)
+;; => ^{:foo :bar
+;;     :baz :kikka} []
+;; some other comment
+;; more comments
+;; =stdout=> foo
+"
+)
+  ;; => "(assoc {} :foo :bar :baz :kikka)\n=> ^{:foo :bar\n    :baz :kikka} []\n;; some other comment\n;; more comments\n=stdout=> [\"foo\"]\n"
+
+
+  (parseable? "   ")
+  ;; => false
+  (parseable? "   32 ")
+  ;; => true
+
+  ;; only :a will be significant
+  (parseable? ":a 2 3")
+  ;; => true
+
+  (parseable? "[:a 2\n;;some foo\n 3]")
+  ;; => true
+
+  (parseable? ";; foo")
+  ;; => false
+
+  ;; for now don't skip discards, user should get error for this eventualy
+  (parseable? "#_ 3")
+  ;; => true
+
+  :eoc)

test-resources/expected/test-doc-blocks/test/example_adoc_cljs_test.cljs

@@ -6,7 +6,7 @@
   (:import goog.events.EventType))
 
 (clojure.test/deftest block-0001
-  (clojure.test/testing  "doc/example.adoc - line 252 - Specifying The Platform - :platform"
+  (clojure.test/testing  "doc/example.adoc - line 305 - Specifying The Platform - :platform"
 ;; this code block will generate a test under example-adoc-cljs-test ns to a .cljs file
 
 nil

test-resources/expected/test-doc-blocks/test/example_adoc_inline_ns_test.cljc

@@ -10,7 +10,7 @@
   (:import #?@(:clj [java.util.List java.util.Queue java.util.Set] :cljs [goog.math.Long goog.math.Vec2 goog.math.Vec3])))
 
 (clojure.test/deftest block-0001
-  (clojure.test/testing  "doc/example.adoc - line 472 - Inline requires and imports"
+  (clojure.test/testing  "doc/example.adoc - line 525 - Inline requires and imports"
 ;; Stick the basics for requires, shorthand notation isn't supported
 
 ;; Some examples:

test-resources/expected/test-doc-blocks/test/example_adoc_inline_refer_clojure_test.cljc

@@ -7,7 +7,7 @@
             [clojure.edn :refer [read-string]]))
 
 (clojure.test/deftest block-0001
-  (clojure.test/testing  "doc/example.adoc - line 503 - Inline refer-clojure"
+  (clojure.test/testing  "doc/example.adoc - line 556 - Inline refer-clojure"
 ;; a contrived example that uses uses clojure.edn/read-string in place
 ;; of clojure.core/read-string and excludes clojure.core/for
 nil

test-resources/expected/test-doc-blocks/test/example_adoc_new_ns/ns1_test.cljc

@@ -6,7 +6,7 @@
             [clojure.string :as string]))
 
 (clojure.test/deftest block-0001
-  (clojure.test/testing  "doc/example.adoc - line 220 - Specifying Test Namespace - :test-ns"
+  (clojure.test/testing  "doc/example.adoc - line 273 - Specifying Test Namespace - :test-ns"
 ;; this code block will generate tests under example-adoc-new-ns.ns1-test
 
 nil