refactor(plugins): decouple plugin framework data models

[araujof]

Summary

Eliminates reverse dependencies from the plugin framework (mcpgateway/plugins/framework/) back into the outer gateway package (mcpgateway.common, mcpgateway.utils), and introduces hook payload policies for controlled modification of plugin payloads.

After this change, the plugin framework has zero imports from mcpgateway.common or mcpgateway.utils, making it viable as a standalone, extractable package.

This PR depends on #2829 (refactor/plugins_settings).

Closes: #2859

Changes

Protocol definitions for boundary types

• New protocols.py with @runtime_checkable MessageLike and PromptResultLike protocols
• Replaces concrete imports of Message and PromptResult from mcpgateway.common.models
• The framework never instantiates these types — protocols express the structural contract without creating a dependency

Immutable payloads

• Promoted PluginPayload from a bare TypeAlias = BaseModel to a proper frozen base class (ConfigDict(frozen=True, arbitrary_types_allowed=True))
• Enforces copy-on-write semantics — plugins use model_copy(update={...}) instead of in-place mutation
• Eliminates the need for defensive deep copies in the executor

Internal framework types

• Defined TransportType enum internally in models.py (was imported from mcpgateway.common.models)
• Created self-contained SecurityValidator in validators.py with only the two methods used by the framework (validate_url, path validation)
• Copied ORJSONResponse into utils.py (was imported from mcpgateway.utils.orjson_response)

Hook payload policies

• New hooks/policies.py with HookPayloadPolicy, DefaultHookPolicy, and apply_policy() — the framework provides the mechanism
• New mcpgateway/plugins/policy.py — the gateway defines the concrete policies per hook type (tool, prompt, resource, agent)
• Policies are injected into PluginManager via dependency injection, keeping the framework decoupled from gateway-specific decisions
• default_hook_policy setting (PLUGINS_DEFAULT_HOOK_POLICY env var) controls behavior for hooks without explicit policies: allow (default, backwards compatible) or deny (strict mode)

Policy enforcement in executor

• PluginExecutor applies per-plugin policy filtering after each plugin returns
• Only fields listed in writable_fields are accepted; all other modifications are silently reverted to the original values
• Hooks without a defined policy fall back to default_hook_policy

JSON deserialization for Any-typed fields

• Added StructuredData model and coerce_nested() utility for recursive dict-to-object conversion
• Applied as a Pydantic field validator on PromptPosthookPayload.result so that external server flows (gRPC, MCP stdio, Unix socket, streamable HTTP) correctly reconstruct nested objects with attribute access

Plugin updates for frozen payloads

• Updated vault, search-replace, PII filter, and Cedar plugins to use model_copy(update={...}) instead of direct attribute assignment
• Updated test fixture HeadersPlugin to use the same pattern

Files changed

Action	File	Description
New	framework/protocols.py	MessageLike, PromptResultLike protocol classes
New	framework/validators.py	Self-contained SecurityValidator subset
New	framework/hooks/policies.py	HookPayloadPolicy, DefaultHookPolicy, apply_policy
New	plugins/policy.py	Gateway-side HOOK_PAYLOAD_POLICIES dictionary
New	tests/.../test_policies.py	20 tests: policies, protocols, frozen payloads, import isolation
Modified	framework/models.py	Internal TransportType enum, frozen PluginPayload base class
Modified	framework/hooks/agents.py	Use Any + protocol docs instead of Message import
Modified	framework/hooks/prompts.py	Use Any + field validator instead of PromptResult import
Modified	framework/hooks/registry.py	Minor import cleanup
Modified	framework/manager.py	Accept hook_policies via DI, enforce in executor loop
Modified	framework/settings.py	Add default_hook_policy field
Modified	framework/utils.py	Add StructuredData, coerce_nested, ORJSONResponse
Modified	framework/__init__.py	Inject HOOK_PAYLOAD_POLICIES when creating manager
Modified	framework/external/mcp/client.py	Import TransportType from framework
Modified	framework/external/mcp/server/runtime.py	Import ORJSONResponse from framework
Modified	plugins/vault/vault_plugin.py	Use model_copy for frozen payload compatibility
Modified	plugins/regex_filter/search_replace.py	Use model_copy for frozen payload compatibility
Modified	plugins/pii_filter/pii_filter.py	Use model_copy for frozen payload compatibility
Modified	plugins/external/cedar/.../plugin.py	Use model_copy for frozen payload compatibility
Modified	tests/.../headers.py	Use model_copy for frozen payload compatibility

Note to maintainer

Please merge #2829 first.

[crivetimihai]

Thanks for this substantial and well-structured decoupling work, @araujof. The protocol-based type contracts and hook payload policies are clean architectural improvements. A few items:

Bug: LazySettingsWrapper.enabled ignores .env / pydantic-settings resolution
settings.py:~1962-1971: When PLUGINS_ENABLED is absent from os.environ, the property hard-returns False instead of delegating to get_settings().enabled. If PLUGINS_ENABLED=true is set in a .env file (which pydantic-settings would parse), it gets silently ignored. Same issue as noted in PR #2829 — consider falling back to get_settings().enabled.

Bug: regex_filter/search_replace.py dict branch still mutates frozen payload in-place
Lines 139-144 (unchanged) still do payload.result[key] = value inside a loop. While frozen=True only guards attribute reassignment, this bypasses the model_copy-based modification pattern and won't produce a properly tracked modified_payload. The str branch was correctly fixed; the dict branch needs the same treatment.

Duplicated TransportType enum
models.py now defines its own TransportType identical to common/models.py. This risks divergence. Consider re-exporting via a thin alias or adding a test asserting both enums have the same members.

lru_cache on get_settings() prevents test isolation
settings.py:1934: @lru_cache() means env var changes in tests won't take effect after the first call. Consider exposing get_settings.cache_clear().

[araujof]

Thanks for this substantial and well-structured decoupling work, @araujof. The protocol-based type contracts and hook payload policies are clean architectural improvements. A few items:

Bug: LazySettingsWrapper.enabled ignores .env / pydantic-settings resolution settings.py:~1962-1971: When PLUGINS_ENABLED is absent from os.environ, the property hard-returns False instead of delegating to get_settings().enabled. If PLUGINS_ENABLED=true is set in a .env file (which pydantic-settings would parse), it gets silently ignored. Same issue as noted in PR #2829 — consider falling back to get_settings().enabled.

Bug: regex_filter/search_replace.py dict branch still mutates frozen payload in-place Lines 139-144 (unchanged) still do payload.result[key] = value inside a loop. While frozen=True only guards attribute reassignment, this bypasses the model_copy-based modification pattern and won't produce a properly tracked modified_payload. The str branch was correctly fixed; the dict branch needs the same treatment.

Duplicated TransportType enum models.py now defines its own TransportType identical to common/models.py. This risks divergence. Consider re-exporting via a thin alias or adding a test asserting both enums have the same members.

lru_cache on get_settings() prevents test isolation settings.py:1934: @lru_cache() means env var changes in tests won't take effect after the first call. Consider exposing get_settings.cache_clear().

Summary of changes:

• Fix frozen payload mutation in search_replace plugin (plugins/regex_filter/search_replace.py)

  • tool_pre_invoke: dict branch now copies payload.args into a local dict, applies regex replacements on the copy, then uses model_copy(update={"args": modified_args}) instead of mutating in-place.
  • tool_post_invoke: same treatment for the payload.result dict branch — copy, modify, model_copy. This aligns both dict branches with the str branch pattern already in place.

• Add TransportType enum parity test (tests/unit/mcpgateway/plugins/framework/test_plugin_models.py)

  • New test_transport_type_enum_parity() asserts that framework.models.TransportType and common.models.TransportType have identical members, guarding against silent divergence.

• Other issues addressed in refactor(plugins): decouple plugin framework from gateway settings #2829 and rebased here. Merge refactor(plugins): decouple plugin framework from gateway settings #2829 first.