docs: update serrializer/tests

Author: Gerit Wagner

docs/source/dev_docs/parser_development.rst

@@ -75,7 +75,9 @@ Implement ``parse_query_tree()`` to build the query object, creating nested quer
 
 .. note::
 
-   Check whether ``SearchFields`` can be created for nested queries (e.g., ``TI=(eHealth OR mHealth)``or only for individual terms, e.g., ``eHealth[ti] OR mHealth[ti]``.)
+    Parsers can be developed as top-down parsers (see PubMed) or bottom-up parsers (see Web of Science).
+
+Check whether ``SearchFields`` can be created for nested queries (e.g., ``TI=(eHealth OR mHealth)``or only for individual terms, e.g., ``eHealth[ti] OR mHealth[ti]``.)
 
 **Parser Skeleton**
 

docs/source/dev_docs/parser_skeleton_tests.py

@@ -0,0 +1,27 @@
+Serializers
+===========
+
+Serializers convert a query object into a string representation.
+This enables the query to be rendered for human inspection, logging, or submission to search engines.
+
+Each serializer implements a function that takes a `Query` object and returns a string.
+This supports various output formats including debugging views and platform-specific syntaxes.
+
+Interface
+---------
+Serializers are typically implemented as standalone functions. The core interface is:
+
+.. literalinclude:: serializer_skeleton.py
+   :language: python
+
+
+Serializers follow a shared conceptual pattern:
+
+- Accept a `Query` object.
+- Recursively traverse the query tree.
+- Render each node (logical operator, term, field) into a string.
+- Combine child nodes with appropriate formatting and syntax.
+
+.. note::
+
+  Avoid embedding platform-specific validation logic (use linters for that).

docs/source/dev_docs/serializer_development.rst

@@ -0,0 +1,29 @@
+#!/usr/bin/env python3
+"""Example serializer template for a custom platform."""
+from __future__ import annotations
+
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from search_query.query import Query
+
+
+def to_string_custom(query: Query) -> str:
+
+    # Leaf node (no children)
+    if not query.children:
+        field = query.search_field.value if query.search_field else ""
+        return f"{field}{query.value}"
+
+    # Composite node (operator with children)
+    serialized_children = [to_string_custom(child) for child in query.children]
+    joined_children = f" {query.value} ".join(serialized_children)
+
+    # Add parentheses to clarify grouping
+    if len(query.children) > 1:
+        joined_children = f"({joined_children})"
+
+    # Prefix with field if applicable
+    if query.search_field:
+        return f"{query.search_field.value}{joined_children}"
+    return joined_children

docs/source/dev_docs/serializer_skeleton.py

@@ -1,5 +1,128 @@
 Tests
-----------------
+============
 
-.. literalinclude:: parser_skeleton_tests.py
-   :language: python
+This section outlines best practices for writing unit tests in the `search_query` package.
+Tests are primarily written using `pytest` and are organized by module (`parser`, `linter`, `translator`, etc.).
+
+
+To run all tests:
+::
+
+    pytest test
+
+Test Types
+----------
+
+1. **Tokenization Tests**
+    - Purpose: Verify that a query string is tokenized correctly into expected tokens.
+    - Tools: `pytest.mark.parametrize` for multiple cases.
+    - Example:
+
+   .. code-block:: python
+
+            @pytest.mark.parametrize(
+               "query_str, expected_tokens",
+               [
+                  (
+                        "AB=(Health)",
+                        [
+                           Token(value="AB=", type=TokenTypes.FIELD, position=(0, 3)),
+                           Token(value="(", type=TokenTypes.PARENTHESIS_OPEN, position=(3, 4)),
+                           Token(value="Health", type=TokenTypes.SEARCH_TERM, position=(4, 10)),
+                           Token(value=")", type=TokenTypes.PARENTHESIS_CLOSED, position=(10, 11)),
+                        ],
+                  )
+               ],
+            )
+            def test_tokenization(query_str: str, expected_tokens: list) -> None:
+               print(
+                  f"Run query parser for: \n  {Colors.GREEN}{query_str}{Colors.END}\n--------------------\n"
+               )
+
+               parser = XYParser(query_str)
+               parser.tokenize()
+
+               assert parser.tokens == expected_tokens, (
+                  f"\nExpected: {expected_tokens}\nGot     : {parser.tokens}"
+               )
+
+2. **Linter Message Tests**
+    - Purpose: Verify that the linter raises expected warnings or errors for malformed input.
+    - Approach:
+      - Catch exceptions where necessary.
+      - Use structured comparison with linter messages.
+    - Example:
+
+   .. code-block:: python
+
+         @pytest.mark.parametrize(
+            "query_str, search_field_general, messages",
+            [
+               (
+                     '("health tracking" OR "remote monitoring") AND (("mobile application" OR "wearable device")',
+                     "Title",
+                     [
+                        {
+                           "code": "F1001",
+                           "label": "unbalanced-parentheses",
+                           "message": "Parentheses are unbalanced in the query",
+                           "is_fatal": True,
+                           "position": (47, 48),
+                           "details": "Unbalanced opening parenthesis",
+                        },
+                        {
+                           "code": "E0001",
+                           "label": "search-field-missing",
+                           "message": "Expected search field is missing",
+                           "is_fatal": False,
+                           "position": (-1, -1),
+                           "details": "Search fields should be specified in the query instead of the search_field_general",
+                        },
+                     ],
+               ),
+               # add more cases here as needed...
+            ],
+         )
+         def test_linter(query_str: str, search_field_general: str, messages: list[dict]) -> None:
+
+            parser = XYParser(query_str, search_field_general=search_field_general)
+            try:
+               parser.parse()
+            except SearchQueryException:
+               pass  # Errors are expected in some cases
+
+            actual_messages = parser.linter.messages
+            if actual_messages != messages:
+               print("Expected:")
+               for m in messages:
+                     print(f"  - {m}")
+               print("Got:")
+               for m in actual_messages:
+                     print(f"  - {m}")
+
+            assert actual_messages == messages
+
+3. **Translation Tests**
+    - Purpose: Confirm that parsing + serialization results in the expected generic or structured query string.
+    - Example:
+
+      **TODO:**
+
+      ::
+
+        @pytest.mark.parametrize(
+            "query_string, expected_translation",
+            [
+                ("TS=(eHealth) AND TS=(Review)", "AND[eHealth[TS=], Review[TS=]]"),
+            ],
+        )
+        def test_parser_translation(query_string, expected_translation):
+            parser = XYParser(query_string)
+            query_tree = parser.parse()
+            assert query_tree.to_generic_string() == expected_translation
+
+
+.. note::
+
+   - Use helper functions like `print_debug_tokens()` to ease debugging.
+   - Use `assert ... == ...` with fallbacks for `print(...)` for inspection.

docs/source/dev_docs/tests.rst

@@ -180,4 +180,5 @@ Below is a high-level overview of the core functionalities:
    dev_docs/parser_development
    dev_docs/linter_development
    dev_docs/translator_development
-   dev_docs/tests
+   dev_docs/serializer_development
+   dev_docs/tests