gopls/internal/mcp: document MCP for v0.20.0; some CLI improvements

Add documentation to our features and release notes for the MCP server
added in v0.20.0. Additionally, make the following CLI improvements:

- s/mcp-listen/mcp.listen, to be consistent with the rpc.trace flag.
- Add an -instructions flag to print the instructions. Users
  installing gopls from the module proxy will not have access to its
  source.

Fixes golang/go#73580

Change-Id: Id2f7d9b2076126e84db1510d7bfcb80702e4d6b4
Reviewed-on: https://go-review.googlesource.com/c/tools/+/687435
Reviewed-by: Alan Donovan <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>

Author: findleyr

gopls/doc/features/index.md

@@ -64,6 +64,7 @@ when making significant changes to existing features or when adding new ones.
   - [go.mod and go.work files](modfiles.md): Go module and workspace manifests
   - [Go *.s assembly files](assembly.md): Go assembly files
 - [Command-line interface](../command-line.md): CLI for debugging and scripting (unstable)
+- [Model Context Protocol](mcp.md): use some features in AI-assisted environments
 
 You can find this page from within your editor by executing the
 `gopls.doc.features` [code action](transformation.md#code-actions),

gopls/doc/features/mcp.md

@@ -0,0 +1,74 @@
+---
+title: "Gopls: Model Context Protocol support"
+---
+
+Gopls includes an experimental built-in server for the [Model Context
+Protocol](https://modelcontextprotocol.io/introduction) (MCP), allowing it to
+expose a subset of its functionality to AI assistants in the form of MCP tools.
+
+## Running the MCP server
+
+There are two modes for running this server: 'attached' and 'detached'. In
+attached mode, the MCP server operates in the context of an active gopls LSP
+session, and so is able to share memory with your LSP session and observe the
+current unsaved buffer state. In detached mode, gopls interacts with a headless
+LSP session, and therefore only sees saved files on disk.
+
+### Attached mode
+
+To use the 'attached' mode, run gopls with the `-mcp.listen` flag. For
+example:
+
+```
+gopls serve -mcp.listen=localhost:8092`
+```
+
+This exposes an HTTP based MCP server using the server-sent event transport
+(SSE), available at `http://localhost:8092/sessions/1` (assuming you have only
+one [session](../daemon.md) on your gopls instance).
+
+### Detached mode
+
+To use the 'detached' mode, run the `mcp` subcommand:
+
+```
+gopls mcp
+```
+
+This runs a standalone gopls instance that speaks MCP over stdin/stdout.
+
+## Instructions to the model
+
+This gopls MCP server includes model instructions for its usage, describing
+workflows for interacting with Go code using its available tools. These
+instructions are automatically published during the MCP server initialization,
+but you may want to also load them as additional context in your AI-assisted
+session, to emphasize their importance. The `-instructions` flag causes them to
+be printed, so that you can do, for example:
+
+```
+gopls mcp -instructions` > /path/to/contextFile.md
+```
+
+## Security considerations
+
+The gopls MCP server is a wrapper around the functionality ordinarily exposed
+by gopls through the Language Server Protocol (LSP). As such, gopls' tools
+may perform any of the operations gopls normally performs, including:
+
+- reading files from the file system, and returning their contents in tool
+  results (such as when providing context);
+- executing the `go` command to load package information, which may result in
+  calls to https://proxy.golang.org to download Go modules, and writes to go
+  caches;
+- writing to gopls' cache or persistant configuration files; and
+- uploading weekly telemetry data **if you have opted in** to [Go telemetry](https://go.dev/doc/telemetry).
+
+The gopls MCP server does not perform any operations not already performed by
+gopls in an ordinary IDE session. Like most LSP servers, gopls does not
+generally write directly to your source tree, though it may instruct the client
+to apply edits. Nor does it make arbitrary requests over the network, though it
+may make narrowly scoped requests to certain services such as the Go module
+mirror or the Go vulnerability database, which can't readily be exploited as a
+vehicle for exfiltration by a confused agent. Nevertheless, these capabilities
+may require additional consideration when used as part of an AI-enabled system.

gopls/doc/release/v0.20.0.md

@@ -2,8 +2,12 @@
 title: "Gopls release v0.20.0 (forthcoming)"
 ---
 
-Gopls' documentation is now available on the Go project's website at
-https://go.dev/gopls. (Note: this link will not be accurate until the
+This release contains a new experimental Model Context Protocol server for
+gopls, which may be used to integrate a subset of gopls' features in
+AI-assisted environments.
+
+Additionally, gopls' documentation is now available on the Go project's website
+at https://go.dev/gopls. (Note: this link will not be accurate until the
 completion of the release. Dueing the pre-release period, please use
 https://tip.golang.org/gopls, which reflects the latest commit.)
 
@@ -12,9 +16,7 @@ web search index.
 
 ## Configuration Changes
 
-## Navigation features
-
-### Web-based features
+## Web-based features
 
 ### "Split package" tool
 
@@ -46,6 +48,20 @@ https://go.dev/issue#xxxxx
 
 ## Editing features
 
+### Model Context Protocol server
+
+Gopls now includes an experimental built-in server for the Model Context
+Protocol (MCP), allowing it to expose a subset of its functionality to
+AI assistants in the form of MCP tools.
+
+See the [documentation](../features/mcp.md) for more information.
+
+**Caveats:** This is a brand new mode of operation for gopls, and so we're
+still experimenting with the best set of tools and instructions to provide.
+Please let us know how well it works for you. Also, please be aware that using
+gopls in an AI powered environment may carry additional security
+considerations, as discussed in the documentation above.
+
 ## Analysis features
 
 ### `ignoredError` inlay hint

gopls/internal/cmd/mcp.go

@@ -23,9 +23,10 @@ import (
 type headlessMCP struct {
 	app *Application
 
-	Address  string `flag:"listen" help:"the address on which to run the mcp server"`
-	Logfile  string `flag:"logfile" help:"filename to log to; if unset, logs to stderr"`
-	RPCTrace bool   `flag:"rpc.trace" help:"print MCP rpc traces; cannot be used with -listen"`
+	Address      string `flag:"listen" help:"the address on which to run the mcp server"`
+	Logfile      string `flag:"logfile" help:"filename to log to; if unset, logs to stderr"`
+	RPCTrace     bool   `flag:"rpc.trace" help:"print MCP rpc traces; cannot be used with -listen"`
+	Instructions bool   `flag:"instructions" help:"if set, print gopls' MCP instructions and exit"`
 }
 
 func (m *headlessMCP) Name() string      { return "mcp" }
@@ -46,6 +47,10 @@ Examples:
 }
 
 func (m *headlessMCP) Run(ctx context.Context, args ...string) error {
+	if m.Instructions {
+		fmt.Println(mcp.Instructions)
+		return nil
+	}
 	if m.Address != "" && m.RPCTrace {
 		// There's currently no way to plumb logging instrumentation into the SSE
 		// transport that is created on connections to the HTTP handler, so we must

gopls/internal/cmd/serve.go

@@ -41,7 +41,7 @@ type Serve struct {
 	RemoteLogfile       string        `flag:"remote.logfile" help:"when used with -remote=auto, the -logfile value used to start the daemon"`
 
 	// MCP Server related configurations.
-	MCPAddress string `flag:"mcp-listen" help:"experimental: address on which to listen for model context protocol connections. If port is localhost:0, pick a random port in localhost instead."`
+	MCPAddress string `flag:"mcp.listen" help:"experimental: address on which to listen for model context protocol connections. If port is localhost:0, pick a random port in localhost instead."`
 
 	app *Application
 }

gopls/internal/cmd/usage/mcp.hlp

@@ -9,6 +9,8 @@ Starts the server over stdio or sse with http, depending on whether the listen f
 Examples:
   $ gopls mcp -listen=localhost:3000
   $ gopls mcp  //start over stdio
+  -instructions
+    	if set, print gopls' MCP instructions and exit
   -listen=string
     	the address on which to run the mcp server
   -logfile=string

gopls/internal/cmd/usage/serve.hlp

@@ -16,7 +16,7 @@ server-flags:
     	when used with -listen, shut down the server when there are no connected clients for this duration
   -logfile=string
     	filename to log to. if value is "auto", then logging to a default output file is enabled
-  -mcp-listen=string
+  -mcp.listen=string
     	experimental: address on which to listen for model context protocol connections. If port is localhost:0, pick a random port in localhost instead.
   -mode=string
     	no effect

gopls/internal/cmd/usage/usage-v.hlp

@@ -60,7 +60,7 @@ flags:
     	when used with -listen, shut down the server when there are no connected clients for this duration
   -logfile=string
     	filename to log to. if value is "auto", then logging to a default output file is enabled
-  -mcp-listen=string
+  -mcp.listen=string
     	experimental: address on which to listen for model context protocol connections. If port is localhost:0, pick a random port in localhost instead.
   -mode=string
     	no effect

gopls/internal/cmd/usage/usage.hlp

@@ -57,7 +57,7 @@ flags:
     	when used with -listen, shut down the server when there are no connected clients for this duration
   -logfile=string
     	filename to log to. if value is "auto", then logging to a default output file is enabled
-  -mcp-listen=string
+  -mcp.listen=string
     	experimental: address on which to listen for model context protocol connections. If port is localhost:0, pick a random port in localhost instead.
   -mode=string
     	no effect

gopls/internal/mcp/mcp.go

@@ -25,7 +25,7 @@ import (
 )
 
 //go:embed instructions.md
-var instructions string
+var Instructions string
 
 // A handler implements various MCP tools for an LSP session.
 type handler struct {
@@ -156,7 +156,7 @@ func newServer(session *cache.Session, lspServer protocol.Server) *mcp.Server {
 		lspServer: lspServer,
 	}
 	mcpServer := mcp.NewServer("gopls", "v0.1.0", &mcp.ServerOptions{
-		Instructions: instructions,
+		Instructions: Instructions,
 	})
 
 	defaultTools := []*mcp.ServerTool{