Merge branch 'master' into fix/rich_progressbar

Author: GdoongMathew

.github/CODEOWNERS

@@ -5,19 +5,19 @@
 # the repo. Unless a later match takes precedence,
 # @global-owner1 and @global-owner2 will be requested for
 # review when someone opens a pull request.
-* @lantiga @borda @tchaton @justusschock @ethanwharris
+* @lantiga @tchaton @justusschock @ethanwharris
 
 # Docs
-/.github/*.md                               @williamfalcon @lantiga @borda
+/.github/*.md                               @williamfalcon @lantiga
 /docs/source-fabric/index.rst               @williamfalcon @lantiga
 /docs/source-pytorch/index.rst              @williamfalcon @lantiga
 /docs/source-pytorch/levels                 @williamfalcon @lantiga
 
 /.github/CODEOWNERS                  @williamfalcon
 /SECURITY.md                         @williamfalcon @lantiga
 /README.md                           @williamfalcon @lantiga
-/src/pytorch_lightning/__about__.py  @williamfalcon @lantiga @borda
-/src/lightning_fabric/__about__.py   @williamfalcon @lantiga @borda
+/src/pytorch_lightning/__about__.py  @williamfalcon @lantiga
+/src/lightning_fabric/__about__.py   @williamfalcon @lantiga
 
 /src/lightning/fabric/loggers        @williamfalcon
 /src/lightning/pytorch/loggers       @williamfalcon

.github/CONTRIBUTING.md

@@ -212,6 +212,8 @@ We welcome any useful contribution! For your convenience here's a recommended wo
    - [Test README](https://github.com/Lightning-AI/pytorch-lightning/blob/master/tests/README.md)
    - [CI/CD README](https://github.com/Lightning-AI/pytorch-lightning/tree/master/.github/workflows#readme)
 
+1. Once you have a PR opened (and thereby a PR number), please update the respective changelog for [fabric](https://github.com/Lightning-AI/pytorch-lightning/blob/master/src/lightning/fabric/CHANGELOG.md) or [pytorch](https://github.com/Lightning-AI/pytorch-lightning/blob/master/src/lightning/pytorch/CHANGELOG.md) subpackage depending on where you made your changes.
+
 1. When you feel ready for integrating your work, mark your PR "Ready for review".
 
    - Your code should be readable and follow the project's design principles.

Makefile

@@ -10,7 +10,7 @@ export PACKAGE_NAME=pytorch
 
 # In Lightning Studio, the `lightning` package comes pre-installed.
 # Uninstall it first to ensure the editable install works correctly.
-setup:
+setup: update
 	uv pip uninstall lightning pytorch-lightning lightning-fabric || true
 	uv pip install -r requirements.txt \
 	    -r requirements/pytorch/base.txt \

docs/source-fabric/guide/callbacks.rst

@@ -83,6 +83,30 @@ The :meth:`~lightning.fabric.fabric.Fabric.call` calls the callback objects in t
 Not all objects registered via ``Fabric(callbacks=...)`` must implement a method with the given name.
 The ones that have a matching method name will get called.
 
+The different callbacks can have different method signatures. Fabric automatically filters keyword arguments based on
+each callback's function signature, allowing callbacks with different signatures to work together seamlessly.
+
+.. code-block:: python
+
+    class TrainingMetricsCallback:
+        def on_train_epoch_end(self, train_loss):
+            print(f"Training loss: {train_loss:.4f}")
+
+    class ValidationMetricsCallback:
+        def on_train_epoch_end(self, val_accuracy):
+            print(f"Validation accuracy: {val_accuracy:.4f}")
+
+    class ComprehensiveCallback:
+        def on_train_epoch_end(self, epoch, **kwargs):
+            print(f"Epoch {epoch} complete with metrics: {kwargs}")
+
+    fabric = Fabric(
+        callbacks=[TrainingMetricsCallback(), ValidationMetricsCallback(), ComprehensiveCallback()]
+    )
+
+    # Each callback receives only the arguments it can handle
+    fabric.call("on_train_epoch_end", epoch=5, train_loss=0.1, val_accuracy=0.95, learning_rate=0.001)
+
 
 ----
 

docs/source-pytorch/community/governance.rst

@@ -19,15 +19,15 @@ Role: All final decisions related to Lightning.
 Maintainers
 -----------
 - Luca Antiga (`lantiga <https://github.com/lantiga>`_)
-- Jirka Borovec (`Borda <https://github.com/Borda>`_)
+- Ethan Harris (`ethanwharris <https://github.com/ethanwharris>`_) (Torchbearer founder)
 - Justus Schock (`justusschock <https://github.com/justusschock>`_)
 
 
 Emeritus Maintainers
 --------------------
-- Ethan Harris (`ethanwharris <https://github.com/ethanwharris>`_) (Torchbearer founder)
 - Nicki Skafte (`SkafteNicki <https://github.com/SkafteNicki>`_)
 - Thomas Chaton (`tchaton <https://github.com/tchaton>`_)
+- Jirka Borovec (`Borda <https://github.com/Borda>`_)
 
 
 Alumni

requirements/pytorch/extra.txt

@@ -5,7 +5,7 @@
 matplotlib>3.1, <3.11.0
 omegaconf >=2.2.3, <2.4.0
 hydra-core >=1.2.0, <1.4.0
-jsonargparse[signatures,jsonnet] >=4.39.0, <4.43.0
+jsonargparse[signatures,jsonnet] >=4.39.0, <4.44.0
 rich >=12.3.0, <14.3.0
 tensorboardX >=2.2, <2.7.0  # min version is set by torch.onnx missing attribute
 bitsandbytes >=0.45.2,<0.47.0; platform_system != "Darwin"

src/lightning/fabric/CHANGELOG.md

@@ -5,18 +5,28 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
 
 
-## [unreleased] - YYYY-MM-DD
+## [Unreleased] - YYYY-MM-DD
 
 ### Added
 
 -
 
+### Changed
+
+-
 
 ### Removed
 
 -
 
 
+## [2.6.0] - 2025-11-28
+
+### Added
+
+- Added kwargs-filtering for `Fabric.call` to support different callback method signatures ([#21258](https://github.com/Lightning-AI/pytorch-lightning/pull/21258))
+
+
 ### Changed
 
 - Expose `weights_only` argument for `Trainer.{fit,validate,test,predict}` and let `torch` handle default value ([#21072](https://github.com/Lightning-AI/pytorch-lightning/pull/21072))
@@ -25,7 +35,12 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
 
 ### Fixed
 
--
+- Fixed issue in detecting MPIEnvironment with partial mpi4py installation ([#21353](https://github.com/Lightning-AI/pytorch-lightning/pull/21353))
+
+- Learning rate scheduler is stepped at the end of epoch when `on_train_batch_start` returns -1 ([#21296](https://github.com/Lightning-AI/pytorch-lightning/issues/21296)).
+
+
+- Fixed FSDP mixed precision semantics and added user warning ([#21361](https://github.com/Lightning-AI/pytorch-lightning/pull/21361))
 
 
 ---

src/lightning/fabric/fabric.py

@@ -985,6 +985,34 @@ def train_function(fabric):
             )
         return self._wrap_and_launch(function, self, *args, **kwargs)
 
+    def _filter_kwargs_for_callback(self, method: Callable, kwargs: dict[str, Any]) -> dict[str, Any]:
+        """Filter keyword arguments to only include those that match the callback method's signature.
+
+        Args:
+            method: The callback method to inspect
+            kwargs: The keyword arguments to filter
+
+        Returns:
+            A filtered dictionary of keyword arguments that match the method's signature
+
+        """
+        try:
+            sig = inspect.signature(method)
+        except (ValueError, TypeError):
+            # If we can't inspect the signature, pass all kwargs to maintain backward compatibility
+            return kwargs
+
+        filtered_kwargs = {}
+        for name, param in sig.parameters.items():
+            # If the method accepts **kwargs, pass all original kwargs directly
+            if param.kind == inspect.Parameter.VAR_KEYWORD:
+                return kwargs
+            # If the parameter exists in the incoming kwargs, add it to filtered_kwargs
+            if name in kwargs:
+                filtered_kwargs[name] = kwargs[name]
+
+        return filtered_kwargs
+
     def call(self, hook_name: str, *args: Any, **kwargs: Any) -> None:
         r"""Trigger the callback methods with the given name and arguments.
 
@@ -994,7 +1022,9 @@ def call(self, hook_name: str, *args: Any, **kwargs: Any) -> None:
         Args:
             hook_name: The name of the callback method.
             *args: Optional positional arguments that get passed down to the callback method.
-            **kwargs: Optional keyword arguments that get passed down to the callback method.
+            **kwargs: Optional keyword arguments that get passed down to the callback method. Keyword arguments
+                that are not present in the callback's signature will be filtered out automatically, allowing
+                callbacks to have different signatures for the same hook.
 
         Example::
 
@@ -1016,13 +1046,8 @@ def on_train_epoch_end(self, results):
                 )
                 continue
 
-            method(*args, **kwargs)
-
-            # TODO(fabric): handle the following signatures
-            # method(self, fabric|trainer, x, y=1)
-            # method(self, fabric|trainer, *args, x, y=1)
-            # method(self, *args, y=1)
-            # method(self, *args, **kwargs)
+            filtered_kwargs = self._filter_kwargs_for_callback(method, kwargs)
+            method(*args, **filtered_kwargs)
 
     def log(self, name: str, value: Any, step: Optional[int] = None) -> None:
         """Log a scalar to all loggers that were added to Fabric.

src/lightning/fabric/plugins/environments/mpi.py

@@ -73,7 +73,11 @@ def detect() -> bool:
         if not _MPI4PY_AVAILABLE:
             return False
 
-        from mpi4py import MPI
+        try:
+            # mpi4py may be installed without MPI being present
+            from mpi4py import MPI
+        except ImportError:
+            return False
 
         return MPI.COMM_WORLD.Get_size() > 1
 

src/lightning/fabric/plugins/precision/fsdp.py

@@ -24,6 +24,7 @@
 from lightning.fabric.plugins.precision.amp import _optimizer_handles_unscaling
 from lightning.fabric.plugins.precision.precision import Precision
 from lightning.fabric.plugins.precision.utils import _convert_fp_tensor, _DtypeContextManager
+from lightning.fabric.utilities import rank_zero_warn
 from lightning.fabric.utilities.types import Optimizable
 
 if TYPE_CHECKING:
@@ -84,19 +85,18 @@ def convert_module(self, module: Module) -> Module:
     def mixed_precision_config(self) -> "TorchMixedPrecision":
         from torch.distributed.fsdp.fully_sharded_data_parallel import MixedPrecision as TorchMixedPrecision
 
-        if self.precision == "16-mixed":
-            param_dtype = torch.float32
-            reduce_dtype = buffer_dtype = torch.float16
-        elif self.precision == "bf16-mixed":
-            param_dtype = torch.float32
-            reduce_dtype = buffer_dtype = torch.bfloat16
-        elif self.precision == "16-true":
+        if self.precision in ("16-true", "bf16-true"):
+            rank_zero_warn(
+                f"FSDP with `{self.precision}` enables computation in lower precision. "
+                "FSDP will always retain a full-precision copy of the model parameters for sharding."
+            )
+
+        if self.precision in ("16-true", "16-mixed"):
             param_dtype = reduce_dtype = buffer_dtype = torch.float16
-        elif self.precision == "bf16-true":
+        elif self.precision in ("bf16-true", "bf16-mixed"):
             param_dtype = reduce_dtype = buffer_dtype = torch.bfloat16
         elif self.precision == "32-true":
-            param_dtype = torch.float32
-            reduce_dtype = buffer_dtype = torch.float32
+            param_dtype = reduce_dtype = buffer_dtype = torch.float32
         else:
             raise ValueError(f"Was unable to infer precision type, received {self.precision!r}.")