-
Notifications
You must be signed in to change notification settings - Fork 38
feat: add play/stop controls for Studio simulation #50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
This adds a new MCP tool that allows creating or updating Script, LocalScript, or ModuleScript instances with provided Luau source code. The tool uses ScriptEditorService:UpdateSourceAsync() to safely write script source, which is the recommended approach since direct Script.Source access is blocked by Roblox. Features: - Create scripts in any service (ServerScriptService, ReplicatedStorage, etc.) - Automatically create intermediate folders for nested paths - Update existing scripts with new source code Parameters: - path: Path to script in game hierarchy (e.g., 'ServerScriptService.GameManager') - source: The Luau source code to write to the script Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Add header explaining this is an enhanced fork - Document the write_script tool and its capabilities - Add installation instructions specific to this fork - Include upstream sync instructions - Preserve original README content below Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add comprehensive AI-assisted game development tools: - capture_screenshot: Capture Studio window as JPEG (macOS/Windows) - read_output: Read Output window with filtering and buffering - get_studio_state: Query current edit/play/run mode - start_playtest: Start play mode with player character - start_simulation: Start physics-only simulation - stop_simulation/stop_playtest: Return to edit mode These tools enable AI assistants to: - Visually inspect game state and debug UI issues - Monitor script output and errors automatically - Safely control play/simulation modes - Build complete, functional games Co-Authored-By: Claude Opus 4.5 <[email protected]>
Fix for start_playtest not workingThe original implementation called Changes Made:
Important Note:During playtest, |
Add three new tools to enable automated game testing: 1. move_character - Move or teleport characters in the workspace - Supports instant teleport or walking to position via Humanoid:MoveTo() - Works in simulation mode where plugin has direct game access - Can target specific characters by name 2. simulate_input - Fire simulated input events via BindableEvent - Creates MCPInputEvent in ReplicatedStorage - Supports keyboard keys (WASD, Space, abilities, etc.) - Supports mouse buttons (Left, Right, Middle) - Actions: begin, end, tap - Games connect to this event to handle simulated input 3. click_gui - Programmatically interact with GUI elements - Finds elements by path (e.g., "StarterGui.MainUI.Button") - Fires MCPGuiClickEvent for game code to respond - Returns element info (position, size, visibility) These tools enable AI-assisted automated testing of Roblox games: - Navigate characters through the game world - Trigger abilities and interactions - Interact with UI elements - Verify game mechanics via screenshots Note: Due to Roblox security restrictions, true input injection is not possible. These tools work by firing BindableEvents that game scripts can listen to, or by directly manipulating game state in simulation mode. Co-Authored-By: Claude Opus 4.5 <[email protected]>
During playtest, RunService:IsRunning() returns false from the plugin context even though the game is running. The plugin can still manipulate workspace objects and fire BindableEvents, so remove the check and let operations fail naturally if they can't be performed. Co-Authored-By: Claude Opus 4.5 <[email protected]>
- StartPlaytest now uses StudioTestService:ExecutePlayModeAsync() to start actual playtest mode (F5) with player character - GetStudioState now distinguishes between "playtest" and "simulation" modes using RunService:IsRunMode() Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Use task.defer instead of task.spawn for better deferred execution
- Pass {} instead of nil to ExecutePlayModeAsync to avoid argument issues
- Fall back to RunService:Run() if StudioTestService fails
- Return detailed JSON response with mode, warnings, and error info
- Don't rely on IsRunning() check (returns false from plugin context)
The tool now works without hanging and provides clear feedback about
which mode was started (playtest vs simulation).
Co-Authored-By: Claude Opus 4.5 <[email protected]>
The plugin runs in a separate DataModel during playtest, causing RunService:IsRunning() to return false even when playtest is running. This caused the code to incorrectly fall back to simulation mode. The fix removes the state verification check and trusts that ExecutePlayModeAsync() was called successfully via task.defer. Verified working: screenshot confirms playtest starts correctly with player character spawned. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Same issue as start_playtest - RunService:IsRunning() returns false from plugin context during playtest. Removed the check and wrapped calls in pcall to handle errors gracefully. Co-Authored-By: Claude Opus 4.5 <[email protected]>
The edit/playtest DataModel isolation prevents BindableEvents from crossing the boundary. This implements HTTP polling as the solution: - Rust server queues commands at /mcp/input endpoint - simulate_input and click_gui now queue directly (no Luau plugin) - Added MCPInputPoller.lua template for games to poll and execute - Games must enable HttpService and include the poller script Removed: - plugin/src/Tools/SimulateInput.luau (no longer needed) - plugin/src/Tools/ClickGui.luau (no longer needed) Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Added simulate_input, click_gui, move_character tool documentation - Updated MCPInputPoller.lua with server/client architecture - Added input simulation setup instructions - Added test script for verification Co-Authored-By: Claude Opus 4.5 <[email protected]>
MCP tools (simulate_input, click_gui) were writing to a local queue that wasn't shared with the HTTP endpoint. Commands never reached the game's poller. Fix: MCP tools now POST to /mcp/input endpoint, which adds commands to the shared queue that get_input_commands_handler drains. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Update log messages to match actual MCPInputPoller output. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Prevents API error 400 when conversation accumulates multiple screenshots. API enforces 2000px max for multi-image requests. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Add MCPMovementController script for WASD movement and jumping - Add ability integration pattern for game ability systems - Add usage examples showing common MCP input operations - Update installation instructions with optional scripts - Add "How It Works" section explaining the data flow Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Document MCPMovementController for WASD movement - Document ability integration pattern - Clarify required vs optional scripts - Improve installation instructions formatting Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Add ASCII architecture diagram showing data flow - Clearly separate required vs optional scripts - Add "WHAT YOU NEED TO ADD TO YOUR GAME" section - Add step-by-step "QUICK START" guide - Add "TROUBLESHOOTING" section - Better explain that these are game-side scripts, not MCP server code Co-Authored-By: Claude Opus 4.5 <[email protected]>
**Problem:** - start_playtest/stop_playtest returned success immediately without verifying the state actually changed - Used task.defer() which executed after HTTP response was sent - No timeout protection on Rust side for hanging operations - Error: "Previous call to start play session has not been completed" **Solution:** - Remove task.defer(), call ExecutePlayModeAsync() directly (it yields) - Add waitForState() function that polls RunService:IsRunning() with timeout - Verify state actually changed before returning success - Add 30s server-side timeout to prevent indefinite hangs - Return structured JSON with success/verified/error fields - Check current state before operations (prevent double start/stop) **Changes:** - plugin/src/Tools/StartPlaytest.luau: Direct call + verification polling - plugin/src/Tools/Simulation.luau: Same pattern for start/stop simulation - src/rbx_studio_server.rs: Add TOOL_EXECUTION_TIMEOUT (30s) to generic_tool_run Co-Authored-By: Claude Opus 4.5 <[email protected]>
Adds new MCP tool to execute Luau code in the actual game server context
during playtest, unlike run_code which executes in plugin context.
**New Tool: run_server_code**
- Executes code in ServerScriptService context
- Access to server-side _G values, DataStores, game state
- Useful for testing server logic, verifying initialization, debugging
**Architecture:**
- MCP server queues code commands at /mcp/server_code endpoint
- Game ServerScript (MCPServerCodeRunner.lua) polls for pending code
- Script executes code and POSTs results back
- Tool waits for result with 30s timeout
**New Files:**
- MCPServerCodeRunner.lua - Server script for games to include
**Changes:**
- src/rbx_studio_server.rs: Add ServerCodeCommand, ServerCodeResult types,
run_server_code tool, and endpoint handlers
- src/main.rs: Register /mcp/server_code routes
**Usage:**
1. Copy MCPServerCodeRunner.lua to ServerScriptService
2. Start playtest (F5)
3. Use run_server_code({ code = "return _G.MyValue" })
Co-Authored-By: Claude Opus 4.5 <[email protected]>
Fixes the error: [UNEXPECTED ERROR] user_MCPStudioPlugin.rbxm.MCPStudioPlugin.Tools.RunCode:79: invalid value (boolean) at index 2 in table for 'concat' The toStrTable function now explicitly converts boolean values to strings before table.concat, which only accepts strings and numbers. Co-Authored-By: Claude Opus 4.5 <[email protected]>
New MCP tools to test RemoteFunctions and RemoteEvents during playtest
without needing to click through UI.
**invoke_remote**
- Invokes a RemoteFunction by path (e.g., 'ReplicatedStorage.Remotes.GetData')
- Accepts JSON array of arguments
- Returns the result from the server handler
- Validates that the target is actually a RemoteFunction
**fire_remote**
- Fires a RemoteEvent by path
- Supports three directions:
- ToServer: Simulates client firing to server
- ToClient: Server fires to specific player (by name)
- ToAllClients: Server fires to all connected clients
- Accepts JSON array of arguments
- Validates that the target is actually a RemoteEvent
Both tools:
- Use the same MCPServerCodeRunner infrastructure as run_server_code
- Provide clear error messages when remote not found
- Have 30s timeout for response
**Usage Examples:**
- invoke_remote({ path: "ReplicatedStorage.Remotes.GetPlayerData", args: "[\"player1\"]" })
- fire_remote({ path: "ReplicatedStorage.Remotes.PlayerAction", direction: "ToServer", args: "[\"jump\"]" })
Co-Authored-By: Claude Opus 4.5 <[email protected]>
- start_playtest now uses task.spawn to call ExecutePlayModeAsync without blocking the HTTP response - Removed broken verification polling (RunService:IsRunning() returns false from plugin context during playtest due to DataModel isolation) - Updated stop_playtest note to explain that RunService:Stop() only works for simulation mode, not playtest mode - For playtest, stopping requires run_server_code with EndTest() or manual stop (F6) Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Add warning section at top about DataModel isolation - Document which tools work in simulation vs playtest mode - Add run_server_code documentation - Add Server Code Execution Setup section - Update start_playtest/stop_playtest docs with limitations - Explain how to enable programmatic playtest stop Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Add kill_existing_server_on_port() that runs before binding to port 44755 - On macOS/Linux: uses lsof to find and kill existing processes - On Windows: uses netstat and taskkill - Prevents conflicts when old MCP server instances linger Also update MCPServerCodeRunner.lua: - Add built-in commands (STOP, PING, PLAYERS, STATE) that work without loadstring - Increase poll interval to 500ms to avoid rate limiting - Improve documentation about LoadStringEnabled property Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Remove invoke_remote tool: Roblox's OnServerInvoke is write-only, cannot be called directly from server context - Update fire_remote description to clarify ToServer isn't supported - ToClient and ToAllClients work correctly for server-to-client events Roblox limitations: - RemoteFunction.OnServerInvoke: can SET but not GET/call - RemoteEvent.OnServerEvent: RBXScriptSignal, cannot be manually fired Co-Authored-By: Claude Opus 4.5 <[email protected]>
e6ba8b3 to
c07a213
Compare
Summary
Add tools for Claude to control Studio's play/simulation state:
get_studio_state- Check current mode (edit/playtest/simulation)start_simulation- Start run mode (F8 - physics without player)start_playtest- Start play mode (F5 - with player character)stop_simulation/stop_playtest- Return to edit modestop_playtestcannot stop playtest mode (F5) from the plugin.This is a fundamental Roblox limitation, not a bug:
RunService:Stop()from plugin context has no effect on playtestRunService:IsRunning()returnsfalsefrom plugin context during playtestWhat Works
start_simulationstart_playteststop_simulationstop_playtestHow to Enable Programmatic Playtest Stop
To stop playtest programmatically, you must call
StudioTestService:EndTest()from inside the running game. This requires adding a server script that can receive commands.Option 1: Use
run_server_code(requires MCPServerCodeRunner)MCPServerCodeRunner.luato your game'sServerScriptServicerun_server_codetool:Option 2: Manual Stop
Press F6 or click the Stop button in Studio.
Implementation Details
start_playtest
Uses
StudioTestService:ExecutePlayModeAsync()wrapped intask.spawn()to avoid blocking the HTTP response. Returns immediately after dispatching the command.start_simulation
Uses
RunService:Run()with verification polling (works because simulation runs in same DataModel as plugin).get_studio_state
Returns current mode using
RunService:IsRunning()andRunService:IsRunMode():"edit"- Studio is in edit mode"playtest"- Play mode (F5) with player character"simulation"- Run mode (F8) without playerNote: During playtest,
get_studio_statemay incorrectly report "edit" due to DataModel isolation.Files Changed
plugin/src/Tools/StartPlaytest.luau- New toolplugin/src/Tools/Simulation.luau- start/stop simulationplugin/src/Tools/GetStudioState.luau- State detectionsrc/rbx_studio_server.rs- Tool definitionsTest Plan
start_playtestspawns player characterstart_simulationruns physics without playerstop_simulationstops simulation modestop_playteststops simulation mode (not playtest - documented limitation)get_studio_stateidentifies edit and simulation modes🤖 Generated with Claude Code