docs: on parsing of invalid code, sexpr-able?

There is, rightly so, confusion on how sexpr-able? behaves.

There was also a lack of detail around what technically invalid
Clojure code rewrite-clj parses.

I've updated user guide and docstrings to clarify.

Closes #150, closes #151

Author: lread

doc/01-user-guide.adoc

@@ -698,6 +698,85 @@ image::map-nodes.png[map nodes]
 
 This is convenient when navigating through the source, but when we want to logically treat any map as a map the difference is admittedly bit awkward.
 
+== Parsing Peculiarities
+
+Rewrite-clj can, in some specific cases, parse technically invalid Clojure.
+Some folks have come to rely on this over the years, so these are behaviours we will preserve.
+
+[[unbalanced-maps]]
+=== Unbalanced Maps
+An unbalanced map is one where there is a key with no value.
+
+Rewrite-clj can parse and emit unbalanced maps:
+[source,clojure]
+----
+(require '[rewrite-clj.zip :as z])
+
+(-> "{:a 1 :b 2 :c}"
+    z/of-string
+    z/root-string)
+;; => "{:a 1 :b 2 :c}"
+----
+
+An attempt to convert an unbalanced map to a Clojure form will throw:
+//#:test-doc-blocks {:reader-cond :clj}
+[source,clojure]
+----
+(try
+  (-> "{:a 1 :b 2 :c}"
+      z/of-string
+      z/sexpr)
+  (catch Throwable e
+    (.getMessage e)))
+;; => "No value supplied for key: :c"
+----
+
+NOTE: `sexpr-able?` considers the current node element type only and will return `true` for all maps, balanced or not.
+
+[[maps-with-duplicate-keys]]
+=== Maps with Duplicate keys
+Rewrite-clj can parse and emit maps with duplicate keys:
+
+[source,clojure]
+----
+(-> "{:a 1 :b 2 :a 3 :a 4 :a 5 :a 6}"
+    z/of-string
+    z/root-string)
+;; => "{:a 1 :b 2 :a 3 :a 4 :a 5 :a 6}"
+----
+
+But when converting to a Clojure form, duplicate keys are not valid in a map, so only the last key/value pair for duplicate keys will be included:
+[source,clojure]
+----
+(-> "{:a 1 :b 2 :a 3 :a 4 :a 5 :a 6}"
+    z/of-string
+    z/sexpr)
+;; => {:b 2, :a 6}
+----
+
+[[sets-with-duplicate-values]]
+=== Sets with Duplicate values
+
+Rewrite-clj can parse and emit sets with duplicate values:
+
+[source,clojure]
+----
+(-> "#{:a :b :a :a :a}"
+    z/of-string
+    z/root-string)
+;; => "#{:a :b :a :a :a}"
+----
+
+But when converting to a Clojure form, duplicate values in a set are not valid Clojure, so the duplicates are omitted:
+
+[source,clojure]
+----
+(-> "#{:a :b :a :a :a}"
+    z/of-string
+    z/sexpr)
+;; => #{:b :a}
+----
+
 [#sexpr-nuances]
 == Sexpr Nuances
 
@@ -754,15 +833,24 @@ The whitespace that a rewrite-clj so carefully preserves is lost when converting
 ; {:b 2, :a 1}
 ----
 
-=== Not all Source is Sexpr-able
+[[not-all-clojure-is-sexpr-able]]
+=== Not all Clojure Elements are Sexpr-able
 
-Some source code elements are not sexpr-able:
+Some source code element types are not sexpr-able:
 
 * Reader ignore/discard `#_` (also known as "uneval" in rewrite-clj)
 * Comments
 * Clojure whitespace (which includes commas)
 
-Both the zip and node APIs include `sexpr-able?` to check if sexpr is supported for a node.
+Both the zip and node APIs include `sexpr-able?` to check if sexpr is supported for the current node element type.
+
+[NOTE]
+====
+`sexpr-able?` only looks at the current node element type. This means that `sexpr` will still throw when:
+
+1. called on a node with an element type that is `sepxr-able?` but, for whatever reason, has a child node that fails to `sexpr`, see link:#unbalanced-maps[unbalanced maps].
+2. called directly on an link:#unbalanced-maps[unbalanced maps].
+====
 
 [source, clojure]
 ----
@@ -823,6 +911,19 @@ Both the zip and node APIs include `sexpr-able?` to check if sexpr is supported
 ----
 <1> Notice the use of `next*` to include normally skipped nodes.
 
+Remember that child nodes with element types that are not `sexpr-able?` are skipped for `sexpr`:
+
+[source,clojure]
+----
+(-> (str "[1 #_:child-discard-will-be-skipped\n"
+         " ;; comment will be skipped\n"
+         " ,,, ,,, ,,, \n"
+         " 2]")
+    z/of-string
+    z/sexpr)
+;; => [1 2]
+----
+
 === Differences in Clojure Platforms
 
 Clojure and ClojureScript have differences.

src/rewrite_clj/node.cljc

@@ -165,7 +165,9 @@
 
 ;; DO NOT EDIT FILE, automatically imported from: rewrite-clj.node.protocols
 (defn sexpr-able?
-  "Return true if [[sexpr]] is supported for `node`."
+  "Return true if [[sexpr]] is supported for `node`'s element type.
+
+   See [related docs in user guide](/doc/01-user-guide.adoc#not-all-clojure-is-sexpr-able)"
   [node] (rewrite-clj.node.protocols/sexpr-able? node))
 
 ;; DO NOT EDIT FILE, automatically imported from: rewrite-clj.node.protocols
@@ -551,7 +553,7 @@
 ;; DO NOT EDIT FILE, automatically imported from: rewrite-clj.node.seq
 (defn list-node
   "Create a node representing a list with `children`.
-   
+
    ```Clojure
    (require '[rewrite-clj.node :as n])
 
@@ -569,7 +571,7 @@
 (defn map-node
   "Create a node representing a map with `children`.
    ```Clojure
-   (require '[rewrite-clj.node :as n]) 
+   (require '[rewrite-clj.node :as n])
 
    (-> (n/map-node [(n/keyword-node :a)
                     (n/spaces 1)
@@ -579,41 +581,66 @@
                     (n/spaces 1)
                     (n/token-node 2)])
        (n/string))
-   ;; => \"{:a 1 :b 2}\" 
+   ;; => \"{:a 1 :b 2}\"
    ```
 
-   Note that rewrite-clj allows unbalanced maps:
+   Note that rewrite-clj allows the, technically illegal, unbalanced map:
    ```Clojure
    (-> (n/map-node [(n/keyword-node :a)])
        (n/string))
    ;; => \"{:a}\"
    ```
-   Note also that [[sexpr]] will fail on an unbalanced map."
+   See [docs on unbalanced maps](/doc/01-user-guide.adoc#unbalanced-maps).
+
+   Rewrite-clj also allows the, also technically illegal, map with duplicate keys:
+   ```Clojure
+   (-> (n/map-node [(n/keyword-node :a)
+                    (n/spaces 1)
+                    (n/token-node 1)
+                    (n/spaces 1)
+                    (n/keyword-node :a)
+                    (n/spaces 1)
+                    (n/token-node 2)])
+       (n/string))
+   ;; => \"{:a 1 :a 2}\"
+   ```
+   See [docs on maps with duplicate keys](/doc/01-user-guide.adoc#maps-with-duplicate-keys)."
   [children] (rewrite-clj.node.seq/map-node children))
 
 ;; DO NOT EDIT FILE, automatically imported from: rewrite-clj.node.seq
 (defn set-node
   "Create a node representing a set with `children`.
-   
+
    ```Clojure
-   (require '[rewrite-clj.node :as n]) 
+   (require '[rewrite-clj.node :as n])
 
    (-> (n/set-node [(n/token-node 1)
                     (n/spaces 1)
                     (n/token-node 2)
                     (n/spaces 1)
                     (n/token-node 3)])
-       n/string) 
+       n/string)
    ;; => \"#{1 2 3}\"
-   ```"
+   ```
+
+   Note that rewrite-clj allows the, technically illegal, set with duplicate values:
+   ```Clojure
+   (-> (n/set-node [(n/token-node 1)
+                    (n/spaces 1)
+                    (n/token-node 1)])
+       (n/string))
+   ;; => \"#{1 1}\"
+   ```
+
+   See [docs on sets with duplicate values](/doc/01-user-guide.adoc#sets-with-duplicate-values)."
   [children] (rewrite-clj.node.seq/set-node children))
 
 ;; DO NOT EDIT FILE, automatically imported from: rewrite-clj.node.seq
 (defn vector-node
   "Create a node representing a vector with `children`.
 
    ```Clojure
-   (require '[rewrite-clj.node :as n]) 
+   (require '[rewrite-clj.node :as n])
 
    (-> (n/vector-node [(n/token-node 1)
                        (n/spaces 1)

src/rewrite_clj/node/protocols.cljc

@@ -32,7 +32,9 @@
   (string [this] (pr-str this)))
 
 (defn sexpr-able?
-  "Return true if [[sexpr]] is supported for `node`."
+  "Return true if [[sexpr]] is supported for `node`'s element type.
+
+   See [related docs in user guide](/doc/01-user-guide.adoc#not-all-clojure-is-sexpr-able)"
   [node]
   (not (printable-only? node)))
 

src/rewrite_clj/node/seq.cljc

@@ -41,7 +41,7 @@
 
 (defn list-node
   "Create a node representing a list with `children`.
-   
+
    ```Clojure
    (require '[rewrite-clj.node :as n])
 
@@ -60,7 +60,7 @@
   "Create a node representing a vector with `children`.
 
    ```Clojure
-   (require '[rewrite-clj.node :as n]) 
+   (require '[rewrite-clj.node :as n])
 
    (-> (n/vector-node [(n/token-node 1)
                        (n/spaces 1)
@@ -75,25 +75,36 @@
 
 (defn set-node
   "Create a node representing a set with `children`.
-   
+
    ```Clojure
-   (require '[rewrite-clj.node :as n]) 
+   (require '[rewrite-clj.node :as n])
 
    (-> (n/set-node [(n/token-node 1)
                     (n/spaces 1)
                     (n/token-node 2)
                     (n/spaces 1)
                     (n/token-node 3)])
-       n/string) 
+       n/string)
    ;; => \"#{1 2 3}\"
-   ```"
+   ```
+
+   Note that rewrite-clj allows the, technically illegal, set with duplicate values:
+   ```Clojure
+   (-> (n/set-node [(n/token-node 1)
+                    (n/spaces 1)
+                    (n/token-node 1)])
+       (n/string))
+   ;; => \"#{1 1}\"
+   ```
+
+   See [docs on sets with duplicate values](/doc/01-user-guide.adoc#sets-with-duplicate-values)."
   [children]
   (->SeqNode :set "#{%s}" 3 set children))
 
 (defn map-node
   "Create a node representing a map with `children`.
    ```Clojure
-   (require '[rewrite-clj.node :as n]) 
+   (require '[rewrite-clj.node :as n])
 
    (-> (n/map-node [(n/keyword-node :a)
                     (n/spaces 1)
@@ -103,15 +114,29 @@
                     (n/spaces 1)
                     (n/token-node 2)])
        (n/string))
-   ;; => \"{:a 1 :b 2}\" 
+   ;; => \"{:a 1 :b 2}\"
    ```
 
-   Note that rewrite-clj allows unbalanced maps:
+   Note that rewrite-clj allows the, technically illegal, unbalanced map:
    ```Clojure
    (-> (n/map-node [(n/keyword-node :a)])
        (n/string))
    ;; => \"{:a}\"
    ```
-   Note also that [[sexpr]] will fail on an unbalanced map."
+   See [docs on unbalanced maps](/doc/01-user-guide.adoc#unbalanced-maps).
+
+   Rewrite-clj also allows the, also technically illegal, map with duplicate keys:
+   ```Clojure
+   (-> (n/map-node [(n/keyword-node :a)
+                    (n/spaces 1)
+                    (n/token-node 1)
+                    (n/spaces 1)
+                    (n/keyword-node :a)
+                    (n/spaces 1)
+                    (n/token-node 2)])
+       (n/string))
+   ;; => \"{:a 1 :a 2}\"
+   ```
+   See [docs on maps with duplicate keys](/doc/01-user-guide.adoc#maps-with-duplicate-keys)."
   [children]
   (->SeqNode :map "{%s}" 2 #(apply hash-map %) children))

src/rewrite_clj/zip.cljc

@@ -238,7 +238,9 @@
 
 ;; DO NOT EDIT FILE, automatically imported from: rewrite-clj.zip.base
 (defn sexpr-able?
-  "Return true if current node in `zloc` can be [[sexpr]]-ed."
+  "Return true if current node's element type in `zloc` can be [[sexpr]]-ed.
+
+   See [related docs in user guide](/doc/01-user-guide.adoc#not-all-clojure-is-sexpr-able)"
   [zloc] (rewrite-clj.zip.base/sexpr-able? zloc))
 
 ;; DO NOT EDIT FILE, automatically imported from: rewrite-clj.zip.base

src/rewrite_clj/zip/base.cljc

@@ -58,7 +58,9 @@
   (some-> zloc zraw/node node/tag))
 
 (defn sexpr-able?
-  "Return true if current node in `zloc` can be [[sexpr]]-ed."
+  "Return true if current node's element type in `zloc` can be [[sexpr]]-ed.
+
+   See [related docs in user guide](/doc/01-user-guide.adoc#not-all-clojure-is-sexpr-able)"
   [zloc]
   (some-> zloc zraw/node node/sexpr-able?))