Advanced Reflection usage with LLM - Update 0.10.0

[IvanMurzak]

Unity-MCP: Reflection, LLMs, and Custom ReflectionConverters

This post introduces the powerful new features around JSON schema, reflection, and extensibility in Unity-MCP, as updated by PR #159.

---

🚀 Reflection + LLM: Supercharging Unity and C#

Unity-MCP now leverages .NET Reflection to create a flexible bridge between Large Language Models (LLMs) and Unity/C# projects. This unlocks a new level of automation, extensibility, and AI-driven manipulation for your Unity scenes and C# objects.

What Can You Do with Reflection & LLM?

• Dynamic Object Modification: LLMs can inspect, modify, and create C# objects or Unity GameObjects at runtime.
• API Discovery & Invocation: Any method in your API can be found and called, with smart handling for input parameters.
• Unity Scene Automation: Add, configure, or remove objects, components, or settings from Unity scenes and assets—directly via LLM instructions.
• File Manipulation: Unity files and metadata can be analyzed and changed programmatically.

---

🧩 Custom ReflectionConverters: Like JsonConverters, but for Reflection

Just as JsonConverter lets you control JSON serialization, ReflectionConverter gives you full control over how reflection-based reads and writes are handled for specific types.

Why Use ReflectionConverters?

• Type Safety & Customization: Handle complex Unity or C# types (e.g., GameObjects, custom components) in a robust, extensible way.
• Plug-and-Play: Easily add your own converters for any type—anyone can contribute or override how certain types are handled.
• LLM-Friendly: Ensures that even AI-generated modifications or API calls respect your type safety and business logic.

How to Create a Custom ReflectionConverter

Implement your own converter following the provided interface, much like you would with a custom JsonConverter. Register it with the MCP plugin, and it will be used automatically where appropriate.

// Example skeleton
public class MyCustomConverter : ReflectionConverter<MyType>
{
    public override SerializedMember Serialize(object value) { /* custom logic */ }
    public override object Deserialize(SerializedMember value) { /* custom logic */ }
}

🧠 Example: LLM-Driven API Calls in Unity

With this system, an LLM can:

1. Discover any available API or method in your Unity/C# project.
2. Invoke that method, providing any required input parameters—even for complex types.
3. Modify existing objects (e.g., adjust properties, run methods, destroy components).
4. Return results or errors, with improved logging and type name formatting.

Example:
“Change the color of all lights in my scene to blue.”
The LLM inspects the scene via reflection, finds all Light components, and invokes their color setters.

---

⚡️ What's New in PR #159?

• Json Schema Update: Enhanced metadata and scene setup, including a new NoLightmap.unity scene.
• API Improvements:
  • Refined GameObject.DestroyComponents and GameObject.Modify for better type clarity and error messages.
  • Improved usage of named parameters and logger injection for all reflection-based APIs.
• Metadata & Dev Experience:
  • More robust .meta file handling.
  • New project configuration for easier development.

Details and file changes:
👉 PR #159 File Changes

---

🌱 Contribute

Anyone can create and register their own ReflectionConverters. Whether you want to handle a new Unity type or add advanced logic, just implement a new converter and plug it in.

---

💬 Questions?

See the Unity-MCP Discussions or open a conversation or simply reply in comments!

---