Implement new mlir_specs Python function to get resource counts from MLIR passes

.codecov.yml

@@ -7,3 +7,7 @@ coverage:
   status:
     project: false
     patch: true
+
+# TODO: add coverage for the python compiler interface
+ignore:
+  - "frontend/catalyst/python_interface"

.github/workflows/check-catalyst.yaml

@@ -473,6 +473,9 @@ jobs:
         # macOS requirements.txt
         python3 -m pip install cuda-quantum==0.6.0
         python3 -m pip install oqc-qcaas-client
+        # Install graphviz for testing the mlir-op-graph integration
+        sudo apt-get install -y graphviz
+        python3 -m pip install graphviz
         make frontend
 
     - name: Get Cached LLVM Build
@@ -556,6 +559,9 @@ jobs:
         sudo apt-get install -y libasan6 make
         python3 --version | grep ${{ needs.constants.outputs.primary_python_version }}
         python3 -m pip install -r requirements.txt
+        # Install graphviz for testing the mlir-op-graph integration
+        sudo apt-get install -y graphviz
+        python3 -m pip install graphviz
         make frontend
 
     - name: Get Cached LLVM Build

doc/releases/changelog-dev.md

@@ -73,6 +73,48 @@
 
 <h3>Improvements 🛠</h3>
 
+* Add an experimental `outline_state_evolution_pass` xDSL pass to `catalyst.python_interface.transforms`,
+  which moves all quantum gate operations to a private callable.
+  [(#8367)](https://github.com/PennyLaneAI/pennylane/pull/8367)
+
+* A new experimental `split_non_commuting_pass` compiler pass has been added to
+  `catalyst.python_interface.transforms`. This pass splits quantum functions that
+  measure observables on the same wires into multiple function executions, where
+  each execution measures observables on different wires (using the "wires" grouping
+  strategy). The original function is replaced with calls to these generated functions,
+  and the results are combined appropriately.
+  [(#8531)](https://github.com/PennyLaneAI/pennylane/pull/8531)
+
+* Add the `PCPhaseOp` operation to the xDSL Quantum dialect.
+  [(#8621)](https://github.com/PennyLaneAI/pennylane/pull/8621)
+
+* Users can now apply xDSL passes without the need to pass the `pass_plugins` argument to
+  the `qjit` decorator.
+  [(#8572)](https://github.com/PennyLaneAI/pennylane/pull/8572)
+  [(#8573)](https://github.com/PennyLaneAI/pennylane/pull/8573)
+  [(#2169)](https://github.com/PennyLaneAI/catalyst/pull/2169)
+  [(#2183)](https://github.com/PennyLaneAI/catalyst/pull/2183)
+
+* The :meth:`catalyst.python_interface.transforms.convert_to_mbqc_formalism_pass` now
+  supports :class:`~xdsl.dialects.scf.IndexSwitchOp` in IR and ignores regions that have no body.
+  [(#8632)](https://github.com/PennyLaneAI/pennylane/pull/8632)
+
+* The `convert_to_mbqc_formalism` compilation pass now outlines the operations to represent a gate
+  in the MBQC formalism into subroutines in order to reduce the IR size for large programs.
+  [(#8619)](https://github.com/PennyLaneAI/pennylane/pull/8619)
+
+* The :meth:`catalyst.python_interface.Compiler.run` method now accepts a string as input,
+  which is parsed and transformed with xDSL.
+  [(#8587)](https://github.com/PennyLaneAI/pennylane/pull/8587)
+
+* An `is_xdsl_pass` function has been added to the `catalyst.python_interface.pass_api` module.
+  This function checks if a pass name corresponds to an xDSL implemented pass.
+  [(#8572)](https://github.com/PennyLaneAI/pennylane/pull/8572)
+
+* A new `catalyst.python_interface.utils` submodule has been added, containing general-purpose utilities for
+  working with xDSL. This includes a function that extracts the concrete value of scalar, constant SSA values.
+  [(#8514)](https://github.com/PennyLaneAI/pennylane/pull/8514)
+
 * Catalyst can now use the new `pass_name` property of pennylane transform objects. Passes can now
   be created using `qml.transform(pass_name=pass_name)` instead of `PassPipelineWrapper`.
   [(#2149](https://github.com/PennyLaneAI/catalyst/pull/2149)
@@ -197,6 +239,20 @@
 
 <h3>Bug fixes 🐛</h3>
 
+* The experimental xDSL :func:`~catalyst.python_interface.transforms.measurements_from_samples_pass`
+  pass has been updated to support `shots` defined by an `arith.constant` operation.
+  [(#8460)](https://github.com/PennyLaneAI/pennylane/pull/8460)
+
+* The experimental xDSL :func:`~catalyst.python_interface.transforms.diagonalize_measurements`
+  pass has been updated to fix a bug that included the wrong SSA value for final qubit insertion
+  and deallocation at the end of the circuit. A clear error is now also raised when there are
+  observables with overlapping wires.
+  [(#8383)](https://github.com/PennyLaneAI/pennylane/pull/8383)
+
+* Fixes a bug in the constructor of the xDSL Quantum dialect's `QubitUnitaryOp` that
+  prevented an instance from being constructed.
+  [(#8456)](https://github.com/PennyLaneAI/pennylane/pull/8456)
+
 * Added missing `detensorize-function-boundary` and `symbol-dce` passes to `Pipelines.cpp`.
   [(#2266)](https://github.com/PennyLaneAI/catalyst/pull/2266)
 
@@ -272,6 +328,20 @@
 
 <h3>Internal changes ⚙️</h3>
 
+* A new `catalyst.python_interface.inspection.mlir_specs` method has been added to facilitate
+  PennyLane's new pass-by-pass specs feature. This function returns information gathered by parsing
+  the xDSL generated by a given QJIT object, such as gate counts, measurements, or qubit allocations.
+  [(#2238)](https://github.com/PennyLaneAI/catalyst/pull/2238)
+
+* The `catalyst.python_interface.visualization` module has been renamed to
+  `catalyst.python_interface.inspection`, and various utility functions within this module
+  have been streamlined.
+  [(#2237)](https://github.com/PennyLaneAI/catalyst/pull/2237)
+
+* Migrated the `pennylane.compiler.python_compiler` submodule from PennyLane to Catalyst.
+  It is now accessible as `catalyst.python_interface`.
+  [(#2199)](https://github.com/PennyLaneAI/catalyst/pull/2199)
+
 * Resource tracking now writes out at device destruction time instead of qubit deallocation
   time. The written resources will be the total amount of resources collected throughout the
   lifetime of the execution. For executions that split work between multiple functions,
@@ -340,17 +410,24 @@
   [(#2245)](https://github.com/PennyLaneAI/catalyst/pull/2245)
   [(#2254)](https://github.com/PennyLaneAI/catalyst/pull/2254)
 
+  * Removed the `catalyst.python_interface.dialects.transform` module in favor of
+  using the `xdsl.dialects.transform` module directly.
+  [(#2261)](https://github.com/PennyLaneAI/catalyst/pull/2261)
 
   * Refactor QEC tablegen files to separate QEC operations into a new `QECOp.td` file
   [(#2253](https://github.com/PennyLaneAI/catalyst/pull/2253)
 
-
   * Removed the `getRotationKind` and `setRotationKind` methods from
   the QEC interface `QECOpInterface` to simplify the interface.
   [(#2250)](https://github.com/PennyLaneAI/catalyst/pull/2250)
 
 <h3>Documentation 📝</h3>
 
+* Added a "Unified Compiler Cookbook" RST file, along with tutorials, to `catalyst.python_interface.doc`,
+  which provides a quickstart guide for getting started with xDSL and its integration with PennyLane and
+  Catalyst.
+  [(#8571)](https://github.com/PennyLaneAI/pennylane/pull/8571)
+
 * A typo in the code example for :func:`~.passes.ppr_to_ppm` has been corrected.
   [(#2136)](https://github.com/PennyLaneAI/catalyst/pull/2136)
 

frontend/catalyst/compiler.py

@@ -384,13 +384,13 @@ def to_mlir_opt(
     *args, stdin=None, options: Optional[CompileOptions] = None, using_python_compiler=False
 ):
     """echo ${input} | catalyst --tool=opt *args *opts -"""
-    # Check if we need to use Python compiler for xDSL passes
+    # Check if we need to use the Python interface for xDSL passes
     if using_python_compiler:
-        # Use Python compiler path for xDSL passes
+        # Use the Python interface path for xDSL passes
         # pylint: disable-next=import-outside-toplevel
-        from pennylane.compiler.python_compiler import Compiler as PythonCompiler
+        from catalyst.python_interface import Compiler as UnifiedCompiler
 
-        compiler = PythonCompiler()
+        compiler = UnifiedCompiler()
         stdin = compiler.run(stdin, callback=None)
 
     # These are the options that may affect compilation
@@ -548,7 +548,7 @@ def check_nested_operations(op):
 
     @debug_logger
     def is_using_python_compiler(self, mlir_module=None):
-        """Returns true if we need the Python compiler path.
+        """Returns true if we need the Python interface path.
 
         This happens when:
         1. xDSL plugin is explicitly loaded (legacy), OR
@@ -594,7 +594,8 @@ def _create_xdsl_pass_save_callback(self, workspace):
             workspace: The workspace directory path
 
         Returns:
-            Callable or None: The callback function if intermediate saving is enabled, None otherwise
+            Callable or None: The callback function if intermediate saving is enabled, None
+            otherwise
         """
         if not (workspace and self.options.keep_intermediate >= KeepIntermediateLevel.CHANGED):
             return None
@@ -606,7 +607,7 @@ def _create_xdsl_pass_save_callback(self, workspace):
         os.makedirs(user_transform_dir, exist_ok=True)
 
         class SavePassIRCallback:
-            """Callback to save IR after each pass in python_compiler."""
+            """Callback to save IR after each pass in python_interface."""
 
             def __init__(self, transform_dir):
                 self.transform_dir = transform_dir
@@ -666,10 +667,10 @@ def run(self, mlir_module, *args, **kwargs):
             # We keep this module here to keep xDSL requirement optional
             # Only move this is it has been decided that xDSL is no longer optional.
             # pylint: disable-next=import-outside-toplevel
-            from pennylane.compiler.python_compiler import Compiler as PythonCompiler
+            from catalyst.python_interface import Compiler as UnifiedCompiler
 
             callback = self._create_xdsl_pass_save_callback(workspace)
-            compiler = PythonCompiler()
+            compiler = UnifiedCompiler()
             ir = compiler.run(ir, callback=callback)
 
         return self.run_from_ir(

frontend/catalyst/jax_primitives_utils.py

@@ -398,9 +398,7 @@ def transform_named_sequence_lowering(jax_ctx: mlir.LoweringRuleContext, pipelin
 
                 try:
                     # pylint: disable=import-outside-toplevel
-                    from pennylane.compiler.python_compiler.pass_api import (
-                        is_xdsl_pass,
-                    )
+                    from catalyst.python_interface.pass_api import is_xdsl_pass
 
                     if is_xdsl_pass(name):
                         uses_xdsl_passes = True

frontend/catalyst/python_interface/__init__.py

@@ -0,0 +1,27 @@
+# Copyright 2025 Xanadu Quantum Technologies Inc.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Unified Compiler API for integration of Catalyst with xDSL."""
+
+from .compiler import Compiler
+from .inspection import QMLCollector, mlir_specs
+from .parser import QuantumParser
+from .pass_api import compiler_transform
+
+__all__ = [
+    "Compiler",
+    "compiler_transform",
+    "QuantumParser",
+    "QMLCollector",
+    "mlir_specs",
+]

frontend/catalyst/python_interface/compiler.py

@@ -0,0 +1,83 @@
+# Copyright 2025 Xanadu Quantum Technologies Inc.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""This file contains the implementation of the PennyLane-xDSL integration API."""
+
+
+import io
+
+from jax._src.interpreters import mlir
+from jaxlib.mlir.dialects import stablehlo
+from jaxlib.mlir.ir import Context as jaxContext
+from jaxlib.mlir.ir import Module as jaxModule
+from pennylane.typing import Callable
+from xdsl.context import Context as xContext
+from xdsl.dialects.builtin import ModuleOp
+from xdsl.passes import ModulePass, PassPipeline
+from xdsl.printer import Printer
+
+from catalyst.python_interface.parser import QuantumParser
+from catalyst.python_interface.pass_api import ApplyTransformSequence
+
+
+# pylint: disable=too-few-public-methods
+class Compiler:
+    """Compiler namespace"""
+
+    @staticmethod
+    def run(
+        module: jaxModule | str,
+        callback: Callable[[ModulePass, ModuleOp, ModulePass], None] | None = None,
+    ) -> jaxModule | str:
+        """Runs the apply-transform-sequence pass.
+
+        The apply-transform-sequence pass is a "meta-pass". In other words,
+        it is a pass that runs other passes.
+
+        Args:
+            module: Either a Jax MLIR module or MLIR IR as a string
+            callback: Optional callback function called between passes
+
+        Returns:
+            jaxModule | str: jaxModule if the input was a jaxModule, else a string.
+        """
+        # Convert to generic text format
+        is_jax_module = isinstance(module, jaxModule)
+        if is_jax_module:
+            gentxtmod = module.operation.get_asm(
+                binary=False, print_generic_op_form=True, assume_verified=True
+            )
+        else:
+            gentxtmod = module
+
+        # Parse and transform with xDSL
+        ctx = xContext(allow_unregistered=True)
+        parser = QuantumParser(ctx, gentxtmod)
+        # xmod is modified in place
+        xmod = parser.parse_module()
+        pipeline = PassPipeline((ApplyTransformSequence(callback=callback),))
+        pipeline.apply(ctx, xmod)
+
+        # Convert back to string
+        buffer = io.StringIO()
+        Printer(stream=buffer, print_generic_format=True).print_op(xmod)
+
+        # Convert back to jaxModule if input was jaxModule
+        if is_jax_module:
+            with jaxContext() as jctx:
+                jctx.allow_unregistered_dialects = True
+                jctx.append_dialect_registry(mlir.upstream_dialects)
+                stablehlo.register_dialect(jctx)  # pylint: disable=no-member
+                newmod: jaxModule = jaxModule.parse(buffer.getvalue())
+            return newmod
+        return buffer.getvalue()