docs/qapi_domain: add namespace support to cross-references

This patch does three things:

1. Record the current namespace context in pending_xrefs so it can be
   used for link resolution later,
2. Pass that recorded namespace context to find_obj() when resolving a
   reference, and
3. Wildly and completely rewrite find_obj().

cross-reference support is expanded to tolerate the presence or absence
of either namespace or module, and to cope with the presence or absence
of contextual information for either.

References now work like this:

1. If the explicit reference target is recorded in the domain's object
   registry, we link to that target and stop looking. We do this lookup
   regardless of how fully qualified the target is, which allows direct
   references to modules (which don't have a module component to their
   names) or direct references to definitions that may or may not belong
   to a namespace or module.

2. If contextual information is available from qapi:namespace or
   qapi:module directives, try using those components to find a direct
   match to the implied target name.

3. If both prior lookups fail, generate a series of regular expressions
   looking for wildcard matches in order from most to least
   specific. Any explicitly provided components (namespace, module)
   *must* match exactly, but both contextual and entirely omitted
   components are allowed to differ from the search result. Note that if
   more than one result is found, Sphinx will emit a warning (a build
   error for QEMU) and list all of the candidate references.

The practical upshot is that in the large majority of cases, namespace
and module information is not required when creating simple `references`
to definitions from within the same context -- even when identical
definitions exist in other contexts.

Even when using simple `references` from elsewhere in the QEMU
documentation manual, explicit namespace info is not required if there
is only one definition by that name.

Disambiguation *will* be required from outside of the QAPI documentation
when referencing e.g. block-core definitions, which are shared between
QEMU QMP and the QEMU Storage Daemon. In that case, there are two
options:

A: References can be made partially or fully explicit,
   e.g. `QMP:block-dirty-bitmap-add` will link to the QEMU version of
   the definition, while `QSD:block-dirty-bitmap-add` would link to the
   QSD version.

B: If all of the references in a document are intended to go to the same
   place, you can insert a "qapi:namespace:: QMP" directive to influence
   the fuzzy-searching for later references.

Signed-off-by: John Snow <[email protected]>
Message-ID: <[email protected]>
Acked-by: Markus Armbruster <[email protected]>
[Commit message typo fixed]
Signed-off-by: Markus Armbruster <[email protected]>

Author: jnsnow

docs/devel/qapi-domain.rst

@@ -400,11 +400,10 @@ Namespaces
 Mimicking the `Python domain target specification syntax
 <https://www.sphinx-doc.org/en/master/usage/domains/python.html#target-specification>`_,
 QAPI allows you to specify the fully qualified path for a data
-type. QAPI enforces globally unique names, so it's unlikely you'll need
-this specific feature, but it may be extended in the near future to
-allow referencing identically named commands and data types from
-different utilities; i.e. QEMU Storage Daemon vs QMP.
+type.
 
+* A namespace can be explicitly provided;
+  e.g. ``:qapi:type:`QMP:BitmapSyncMode``
 * A module can be explicitly provided;
   ``:qapi:type:`block-core.BitmapSyncMode``` will render to:
   :qapi:type:`block-core.BitmapSyncMode`
@@ -413,6 +412,28 @@ different utilities; i.e. QEMU Storage Daemon vs QMP.
   will render to: :qapi:type:`~block-core.BitmapSyncMode`
 
 
+Target resolution
+-----------------
+
+Any cross-reference to a QAPI type, whether using the ```any``` style of
+reference or the more explicit ```:qapi:any:`target``` syntax, allows
+for the presence or absence of either the namespace or module
+information.
+
+When absent, their value will be inferred from context by the presence
+of any ``qapi:namespace`` or ``qapi:module`` directives preceding the
+cross-reference.
+
+If no results are found when using the inferred values, other
+namespaces/modules will be searched as a last resort; but any explicitly
+provided values must always match in order to succeed.
+
+This allows for efficient cross-referencing with a minimum of syntax in
+the large majority of cases, but additional context or namespace markup
+may be required outside of the QAPI reference documents when linking to
+items that share a name across multiple documented QAPI schema.
+
+
 Custom link text
 ----------------
 
@@ -492,6 +513,11 @@ directives are associated with the most recent namespace. This affects
 the definition's "fully qualified name", allowing two different
 namespaces to create an otherwise identically named definition.
 
+This directive also influences how reference resolution works for any
+references that do not explicity specify a namespace, so this directive
+can be used to nudge references into preferring targets from within that
+namespace.
+
 Example::
 
    .. qapi:namespace:: QMP

docs/sphinx/qapi_domain.py

@@ -7,6 +7,7 @@
 
 from __future__ import annotations
 
+import re
 from typing import (
     TYPE_CHECKING,
     List,
@@ -94,6 +95,7 @@ def process_link(
         title: str,
         target: str,
     ) -> tuple[str, str]:
+        refnode["qapi:namespace"] = env.ref_context.get("qapi:namespace")
         refnode["qapi:module"] = env.ref_context.get("qapi:module")
 
         # Cross-references that begin with a tilde adjust the title to
@@ -830,66 +832,102 @@ def merge_domaindata(
                 self.objects[fullname] = obj
 
     def find_obj(
-        self, modname: str, name: str, typ: Optional[str]
-    ) -> list[tuple[str, ObjectEntry]]:
+        self, namespace: str, modname: str, name: str, typ: Optional[str]
+    ) -> List[Tuple[str, ObjectEntry]]:
         """
-        Find a QAPI object for "name", perhaps using the given module.
+        Find a QAPI object for "name", maybe using contextual information.
 
         Returns a list of (name, object entry) tuples.
 
-        :param modname: The current module context (if any!)
-                        under which we are searching.
-        :param name: The name of the x-ref to resolve;
-                     may or may not include a leading module.
-        :param type: The role name of the x-ref we're resolving, if provided.
-                     (This is absent for "any" lookups.)
+        :param namespace: The current namespace context (if any!) under
+            which we are searching.
+        :param modname: The current module context (if any!) under
+            which we are searching.
+        :param name: The name of the x-ref to resolve; may or may not
+            include leading context.
+        :param type: The role name of the x-ref we're resolving, if
+            provided. This is absent for "any" role lookups.
         """
         if not name:
             return []
 
-        names: list[str] = []
-        matches: list[tuple[str, ObjectEntry]] = []
+        # ##
+        # what to search for
+        # ##
 
-        fullname = name
-        if "." in fullname:
-            # We're searching for a fully qualified reference;
-            # ignore the contextual module.
-            pass
-        elif modname:
-            # We're searching for something from somewhere;
-            # try searching the current module first.
-            # e.g. :qapi:cmd:`query-block` or `query-block` is being searched.
-            fullname = f"{modname}.{name}"
+        parts = list(QAPIDescription.split_fqn(name))
+        explicit = tuple(bool(x) for x in parts)
+
+        # Fill in the blanks where possible:
+        if namespace and not parts[0]:
+            parts[0] = namespace
+        if modname and not parts[1]:
+            parts[1] = modname
+
+        implicit_fqn = ""
+        if all(parts):
+            implicit_fqn = f"{parts[0]}:{parts[1]}.{parts[2]}"
 
         if typ is None:
-            # type isn't specified, this is a generic xref.
-            # search *all* qapi-specific object types.
+            # :any: lookup, search everything:
             objtypes: List[str] = list(self.object_types)
         else:
             # type is specified and will be a role (e.g. obj, mod, cmd)
             # convert this to eligible object types (e.g. command, module)
             # using the QAPIDomain.object_types table.
             objtypes = self.objtypes_for_role(typ, [])
 
-        if name in self.objects and self.objects[name].objtype in objtypes:
-            names = [name]
-        elif (
-            fullname in self.objects
-            and self.objects[fullname].objtype in objtypes
-        ):
-            names = [fullname]
-        else:
-            # exact match wasn't found; e.g. we are searching for
-            # `query-block` from a different (or no) module.
-            searchname = "." + name
-            names = [
-                oname
-                for oname in self.objects
-                if oname.endswith(searchname)
-                and self.objects[oname].objtype in objtypes
-            ]
+        # ##
+        # search!
+        # ##
+
+        def _search(needle: str) -> List[str]:
+            if (
+                needle
+                and needle in self.objects
+                and self.objects[needle].objtype in objtypes
+            ):
+                return [needle]
+            return []
 
-        matches = [(oname, self.objects[oname]) for oname in names]
+        if found := _search(name):
+            # Exact match!
+            pass
+        elif found := _search(implicit_fqn):
+            # Exact match using contextual information to fill in the gaps.
+            pass
+        else:
+            # No exact hits, perform applicable fuzzy searches.
+            searches = []
+
+            esc = tuple(re.escape(s) for s in parts)
+
+            # Try searching for ns:*.name or ns:name
+            if explicit[0] and not explicit[1]:
+                searches.append(f"^{esc[0]}:([^\\.]+\\.)?{esc[2]}$")
+            # Try searching for *:module.name or module.name
+            if explicit[1] and not explicit[0]:
+                searches.append(f"(^|:){esc[1]}\\.{esc[2]}$")
+            # Try searching for context-ns:*.name or context-ns:name
+            if parts[0] and not (explicit[0] or explicit[1]):
+                searches.append(f"^{esc[0]}:([^\\.]+\\.)?{esc[2]}$")
+            # Try searching for *:context-mod.name or context-mod.name
+            if parts[1] and not (explicit[0] or explicit[1]):
+                searches.append(f"(^|:){esc[1]}\\.{esc[2]}$")
+            # Try searching for *:name, *.name, or name
+            if not (explicit[0] or explicit[1]):
+                searches.append(f"(^|:|\\.){esc[2]}$")
+
+            for search in searches:
+                if found := [
+                    oname
+                    for oname in self.objects
+                    if re.search(search, oname)
+                    and self.objects[oname].objtype in objtypes
+                ]:
+                    break
+
+        matches = [(oname, self.objects[oname]) for oname in found]
         if len(matches) > 1:
             matches = [m for m in matches if not m[1].aliased]
         return matches
@@ -904,8 +942,9 @@ def resolve_xref(
         node: pending_xref,
         contnode: Element,
     ) -> nodes.reference | None:
+        namespace = node.get("qapi:namespace")
         modname = node.get("qapi:module")
-        matches = self.find_obj(modname, target, typ)
+        matches = self.find_obj(namespace, modname, target, typ)
 
         if not matches:
             # Normally, we could pass warn_dangling=True to QAPIXRefRole(),
@@ -958,7 +997,9 @@ def resolve_any_xref(
         contnode: Element,
     ) -> List[Tuple[str, nodes.reference]]:
         results: List[Tuple[str, nodes.reference]] = []
-        matches = self.find_obj(node.get("qapi:module"), target, None)
+        matches = self.find_obj(
+            node.get("qapi:namespace"), node.get("qapi:module"), target, None
+        )
         for name, obj in matches:
             rolename = self.role_for_objtype(obj.objtype)
             assert rolename is not None