Preserve custom variables and events in built-in model to_config (#5411)

* Preserve custom variables and events in built-in model to_config

When users modify a built-in model (add/remove variables, change events),
to_config() now detects these changes and includes them as optional
override keys (extra_variables, removed_variables, events) in the
compact config format. from_config() applies them after constructing
the fresh model.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: pre-commit fixes

* update changelog

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Author: 3 people

CHANGELOG.md

@@ -3,6 +3,7 @@
 ## Features
 
 - Improved the performance of processed variables by replacing `casadi.vertcat` input stacking with numpy vectors. ([#5413](https://github.com/pybamm-team/PyBaMM/pull/5413))
+- Preserve custom variables and events in built-in model to_config ([#5411](https://github.com/pybamm-team/PyBaMM/pull/5411))
 - Allow out of bounds initial state of charge to enable initialising a simulation at a voltage outside the voltage limits. ([#5386](https://github.com/pybamm-team/PyBaMM/pull/5386))
 - Added `cache_esoh` option to `Simulation` that caches the electrode SOH computation across repeated `solve` calls, avoiding redundant recalculation when eSOH-relevant parameters have not changed. The cached eSOH solver/simulation object is also reused on cache misses to skip expensive model rebuilding. ([#5408](https://github.com/pybamm-team/PyBaMM/pull/5408))
 - Eliminated the mass matrix inverse and temporary dense matrix objects when building the simulation. ([#5391](https://github.com/pybamm-team/PyBaMM/pull/5391))

docs/source/api/models/submodels/electrolyte_conductivity/full_conductivity.rst

@@ -3,4 +3,3 @@ Full Model
 
 .. autoclass:: pybamm.electrolyte_conductivity.Full
     :members:
-    :inherited-members:

src/pybamm/models/base_model.py

@@ -15,6 +15,7 @@
 
 import pybamm
 from pybamm.expression_tree.operations.serialise import Serialise
+from pybamm.models.event import EventType
 from pybamm.models.symbol_processor import SymbolProcessor
 
 
@@ -2041,6 +2042,82 @@ def _find_builtin_module(cls):
                     return mod_name
         return None
 
+    def serialise_builtin_overrides(self, model_config: dict) -> None:
+        """Detect and serialise user modifications to a built-in model.
+
+        Compares the current model's ``variables`` keys and ``events``
+        against a freshly-constructed reference model (same class and
+        options).  Any differences are written into *model_config* using
+        the following optional keys:
+
+        ``custom_variables``
+            ``dict[str, json]`` – variables the user *added* (keys not
+            present in the reference model).  Each value is the symbolic
+            expression serialised via
+            :func:`convert_symbol_to_json`.
+        ``removed_variables``
+            ``list[str]`` – variable names that exist in the reference
+            model but have been deleted by the user.
+        ``events``
+            ``list[dict]`` – full serialised events list, included
+            **only** when the events differ from the reference model
+            (by count or by name set).
+
+        If the model is unmodified, no extra keys are added and the
+        config stays identical to the previous compact format.
+
+        .. note::
+
+           Only added/removed variable *keys* are tracked.  Overwriting
+           an existing variable's expression with a new one is **not**
+           detected in this version.
+
+        Parameters
+        ----------
+        model_config : dict
+            The compact config dict (mutated in-place).
+        """
+        from pybamm.expression_tree.operations.serialise import (
+            convert_symbol_to_json,
+        )
+
+        # Build a pristine reference with the same options.
+        model_cls = type(self)
+        options = model_config.get("options", {})
+        ref = model_cls(options=options) if options else model_cls()
+
+        # --- Variables diff (added / removed keys only) ---
+        current_keys = set(self.variables.keys())
+        ref_keys = set(ref.variables.keys())
+
+        added = current_keys - ref_keys
+        removed = ref_keys - current_keys
+
+        if added:
+            model_config["custom_variables"] = {
+                name: convert_symbol_to_json(self.variables[name])
+                for name in sorted(added)
+            }
+        if removed:
+            model_config["removed_variables"] = sorted(removed)
+
+        # --- Events diff (full replacement when different) ---
+        current_event_names = {e.name for e in self.events}
+        ref_event_names = {e.name for e in ref.events}
+
+        if (
+            len(self.events) != len(ref.events)
+            or current_event_names != ref_event_names
+        ):
+            model_config["events"] = [
+                {
+                    "name": event.name,
+                    "expression": convert_symbol_to_json(event.expression),
+                    "event_type": event.event_type.value,
+                }
+                for event in self.events
+            ]
+
     def to_config(
         self,
         filename: str | Path | None = None,
@@ -2082,6 +2159,13 @@ def to_config(
             }
             if hasattr(self, "options"):
                 model_config["options"] = dict(self.options)
+
+            # Detect user modifications to variables and events.
+            # A fresh reference model is instantiated with the same options
+            # so we can diff against it. Only added/removed variable *keys*
+            # are tracked; overwriting an existing variable's expression is
+            # NOT detected (out of scope for v1).
+            self.serialise_builtin_overrides(model_config)
         else:
             # Custom / user-defined model — full serialised format
             model_config = {
@@ -2212,11 +2296,60 @@ def from_config(config: str | dict) -> BaseModel:
                     k: tuple(v) if isinstance(v, list) else v
                     for k, v in options.items()
                 }
-            return model_cls(options=options) if options else model_cls()
+            model = model_cls(options=options) if options else model_cls()
+            BaseModel.apply_builtin_overrides(model, data)
+            return model
 
         # Fallback: raw to_json dict (no "type" key)
         return Serialise.load_custom_model(data)
 
+    @staticmethod
+    def apply_builtin_overrides(model: BaseModel, data: dict) -> None:
+        """Apply variable and event overrides from a built-in config.
+
+        This is the inverse of
+        :meth:`serialise_builtin_overrides`.  It mutates *model*
+        in-place.
+
+        Parameters
+        ----------
+        model : BaseModel
+            A freshly-constructed built-in model.
+        data : dict
+            The config dictionary, which may contain
+            ``custom_variables``, ``removed_variables``, and/or
+            ``events``.
+        """
+        from pybamm.expression_tree.operations.serialise import (
+            convert_symbol_from_json,
+        )
+
+        # --- Custom variables ---
+        extra = data.get("custom_variables")
+        if extra:
+            for name, expr_json in extra.items():
+                model.variables[name] = convert_symbol_from_json(expr_json)
+
+        # --- Removed variables ---
+        removed = data.get("removed_variables")
+        if removed:
+            for name in removed:
+                model.variables.pop(name, None)
+
+        # --- Events override ---
+        events_data = data.get("events")
+        if events_data is not None:
+            model.events = [
+                pybamm.Event(
+                    e["name"],
+                    convert_symbol_from_json(e["expression"]),
+                    EventType(e["event_type"])
+                    if isinstance(e["event_type"], int)
+                    else e["event_type"],
+                )
+                for e in events_data
+            ]
+
 
 def load_model(filename, battery_model: BaseModel | None = None):
     """

tests/unit/test_models/test_model_to_json.py

@@ -369,3 +369,133 @@ def test_to_config_builtin_file_round_trip(self, tmp_path):
         assert file_path.exists()
         loaded = pybamm.BaseModel.from_config(str(file_path))
         assert type(loaded) is type(model)
+
+    # ---- Built-in model override tests ----
+
+    def test_to_config_builtin_unmodified_has_no_overrides(self):
+        """Unmodified built-in model config has no override keys."""
+        model = pybamm.lithium_ion.SPM()
+        config = model.to_config()
+        assert config["type"] == "SPM"
+        assert "custom_variables" not in config
+        assert "removed_variables" not in config
+        assert "events" not in config
+
+    def test_to_config_builtin_with_extra_variable_round_trip(self):
+        """Added variable survives to_config / from_config round-trip."""
+        model = pybamm.lithium_ion.SPM()
+        model.variables["My variable"] = 2 * model.variables["Voltage [V]"]
+        config = model.to_config()
+
+        # Still compact format with override
+        assert config["type"] == "SPM"
+        assert "custom_variables" in config
+        assert "My variable" in config["custom_variables"]
+
+        loaded = pybamm.BaseModel.from_config(config)
+        assert type(loaded) is type(model)
+        assert "My variable" in loaded.variables
+
+    def test_to_config_builtin_with_cleared_events_round_trip(self):
+        """Clearing events survives to_config / from_config round-trip."""
+        model = pybamm.lithium_ion.SPM()
+        model.events = []
+        config = model.to_config()
+
+        assert config["type"] == "SPM"
+        assert "events" in config
+        assert config["events"] == []
+
+        loaded = pybamm.BaseModel.from_config(config)
+        assert type(loaded) is type(model)
+        assert loaded.events == []
+
+    def test_to_config_builtin_with_removed_variable_round_trip(self):
+        """Removed variable is absent after from_config round-trip."""
+        model = pybamm.lithium_ion.SPM()
+        assert "Voltage [V]" in model.variables
+        del model.variables["Voltage [V]"]
+        config = model.to_config()
+
+        assert config["type"] == "SPM"
+        assert "removed_variables" in config
+        assert "Voltage [V]" in config["removed_variables"]
+
+        loaded = pybamm.BaseModel.from_config(config)
+        assert type(loaded) is type(model)
+        assert "Voltage [V]" not in loaded.variables
+
+    def test_to_config_builtin_with_added_event_round_trip(self):
+        """Custom event added to built-in model survives round-trip."""
+        model = pybamm.lithium_ion.SPM()
+        original_count = len(model.events)
+        model.events.append(
+            pybamm.Event(
+                "my_custom_event",
+                pybamm.Scalar(1),
+                pybamm.EventType.TERMINATION,
+            )
+        )
+        config = model.to_config()
+
+        assert config["type"] == "SPM"
+        assert "events" in config
+        assert len(config["events"]) == original_count + 1
+
+        loaded = pybamm.BaseModel.from_config(config)
+        assert type(loaded) is type(model)
+        event_names = {e.name for e in loaded.events}
+        assert "my_custom_event" in event_names
+
+    def test_to_config_builtin_combined_variable_and_event_changes(self):
+        """Matches example.py: add variable + clear events, round-trip."""
+        model = pybamm.lithium_ion.SPM()
+        model.events = []
+        model.variables["My variable"] = 2 * model.variables["Voltage [V]"]
+        config = model.to_config()
+
+        assert config["type"] == "SPM"
+        assert "custom_variables" in config
+        assert "events" in config
+        assert config["events"] == []
+
+        loaded = pybamm.BaseModel.from_config(config)
+        assert type(loaded) is type(model)
+        assert "My variable" in loaded.variables
+        assert loaded.events == []
+
+    def test_to_config_builtin_event_type_survives_json_round_trip(self):
+        """Event type enum survives JSON serialisation round-trip."""
+        model = pybamm.lithium_ion.SPM()
+        model.events.append(
+            pybamm.Event(
+                "custom_event",
+                pybamm.Scalar(1),
+                pybamm.EventType.TERMINATION,
+            )
+        )
+        config = model.to_config()
+
+        # JSON round-trip (would fail if event_type is a raw enum)
+        json_str = json.dumps(config)
+        reloaded = json.loads(json_str)
+
+        loaded = pybamm.BaseModel.from_config(reloaded)
+        custom = [e for e in loaded.events if e.name == "custom_event"]
+        assert len(custom) == 1
+        assert custom[0].event_type == pybamm.EventType.TERMINATION
+
+    def test_to_config_builtin_overrides_json_serializable(self):
+        """Config with overrides survives JSON round-trip."""
+        model = pybamm.lithium_ion.SPM()
+        model.events = []
+        model.variables["My variable"] = 2 * model.variables["Voltage [V]"]
+        config = model.to_config()
+
+        json_str = json.dumps(config)
+        assert isinstance(json_str, str)
+        reloaded = json.loads(json_str)
+
+        loaded = pybamm.BaseModel.from_config(reloaded)
+        assert "My variable" in loaded.variables
+        assert loaded.events == []