Skip to content

Commit d076c70

Browse files
meghanmurphy1kibanamachine
meghanmurphy1
and
kibanamachine
authored
[WorplaceAI] GitHub MCP Connector part of Data Source (#247659)
## Summary The GitHub Data Source will now use the MCP connector rather than the GitHub connector-v2 spec and bulk imports primitive tools that are namespaced with the data source and create workflows for searching. This PR: - separates the bulk import logic from the MCP connector API into an exported function `bulkCreateMcpTools` - adds `getNamedMcpTools` to get back the tools that are listed as part of `importedTools` in the Github`DataTypeDefinition.` - adds `createStackConnector `util to create the stack connector and build the secrets based on the connector v2 spec or MCP config. - update to use `credentials` in api instead of `token` for clarity that it can be other forms of auth as well. - added workflows that become workflow tools for searching that can later have steps to sanitize/rerank. ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [ ] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
1 parent 3df8126 commit d076c70

File tree

17 files changed

+1399
-288
lines changed

17 files changed

+1399
-288
lines changed

‎x-pack/platform/plugins/shared/agent_builder/server/routes/internal/tools.ts‎

Lines changed: 7 additions & 92 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,6 @@
77

88
import { schema } from '@kbn/config-schema';
99
import { listSearchSources } from '@kbn/agent-builder-genai-utils';
10-
import { ToolType, validateToolId } from '@kbn/agent-builder-common';
1110
import { CONNECTOR_ID as MCP_CONNECTOR_ID } from '@kbn/connector-schemas/mcp/constants';
1211
import type { ListToolsResponse } from '@kbn/mcp-client';
1312
import type { ActionTypeExecutorResult } from '@kbn/actions-plugin/common';
@@ -24,7 +23,6 @@ import type {
2423
GetToolHealthResponse,
2524
ListToolHealthResponse,
2625
BulkCreateMcpToolsResponse,
27-
BulkCreateMcpToolResult,
2826
ListConnectorsResponse,
2927
ConnectorItem,
3028
ListMcpToolsResponse,
@@ -34,10 +32,9 @@ import type {
3432
McpToolHealthStatus,
3533
ValidateNamespaceResponse,
3634
} from '../../../common/http_api/tools';
37-
import { validateConnector } from '../../services/tools/tool_types/mcp/validate_configuration';
3835
import { apiPrivileges } from '../../../common/features';
3936
import { internalApiPath } from '../../../common/constants';
40-
import { getToolTypeInfo } from '../../services/tools/utils';
37+
import { getToolTypeInfo, bulkCreateMcpTools } from '../../services/tools/utils';
4138
import { toConnectorItem } from '../utils';
4239

4340
export function registerInternalToolsRoutes({
@@ -129,99 +126,17 @@ export function registerInternalToolsRoutes({
129126
const [, { actions }] = await coreSetup.getStartServices();
130127
const registry = await toolService.getRegistry({ request });
131128

132-
// Validate namespace if provided (must be valid tool ID segment)
133-
if (namespace) {
134-
const namespaceError = validateToolId({ toolId: namespace, builtIn: false });
135-
if (namespaceError) {
136-
return response.badRequest({
137-
body: { message: `Invalid namespace: ${namespaceError}` },
138-
});
139-
}
140-
}
141-
142-
// Validate connector is MCP type
143-
await validateConnector({
129+
const { results, summary } = await bulkCreateMcpTools({
130+
registry,
144131
actions,
145132
request,
146133
connectorId,
134+
tools,
135+
namespace,
136+
tags,
137+
skipExisting,
147138
});
148139

149-
// Precompute tool metadata (toolId, mcpToolName) once per tool
150-
// MCP tool names are server-generated and typically well-formed (e.g., snake_case)
151-
// We just lowercase them; validation in registry.create() handles edge cases
152-
const toolsWithIds = tools.map((tool) => {
153-
const toolName = tool.name.toLowerCase();
154-
const toolId = namespace ? `${namespace}.${toolName}` : toolName;
155-
return { toolId, mcpToolName: tool.name, description: tool.description };
156-
});
157-
158-
// Process tools in parallel using Promise.allSettled (matches bulk delete pattern)
159-
const createResults = await Promise.allSettled(
160-
toolsWithIds.map(async ({ toolId, mcpToolName, description }) => {
161-
// Check if tool already exists
162-
const exists = await registry.has(toolId);
163-
if (exists && skipExisting) {
164-
return { toolId, mcpToolName, skipped: true as const };
165-
}
166-
167-
// Create the MCP tool
168-
await registry.create({
169-
id: toolId,
170-
type: ToolType.mcp,
171-
description: description ?? '',
172-
tags,
173-
configuration: {
174-
connector_id: connectorId,
175-
tool_name: mcpToolName,
176-
},
177-
});
178-
179-
return { toolId, mcpToolName, skipped: false as const };
180-
})
181-
);
182-
183-
// Map results to response format (matches bulk delete pattern)
184-
const results: BulkCreateMcpToolResult[] = createResults.map((result, index) => {
185-
const { toolId, mcpToolName } = toolsWithIds[index];
186-
187-
if (result.status === 'rejected') {
188-
return {
189-
toolId,
190-
mcpToolName,
191-
success: false as const,
192-
reason: result.reason?.toJSON?.() ?? {
193-
error: { message: result.reason?.message ?? 'Unknown error' },
194-
},
195-
};
196-
}
197-
198-
if (result.value.skipped) {
199-
return {
200-
toolId: result.value.toolId,
201-
mcpToolName: result.value.mcpToolName,
202-
success: true as const,
203-
skipped: true as const,
204-
};
205-
}
206-
207-
return {
208-
toolId: result.value.toolId,
209-
mcpToolName: result.value.mcpToolName,
210-
success: true as const,
211-
};
212-
});
213-
214-
// Compute summary counts
215-
const summary = results.reduce(
216-
(acc, r) => {
217-
if (!r.success) acc.failed++;
218-
else if ('skipped' in r && r.skipped) acc.skipped++;
219-
else acc.created++;
220-
return acc;
221-
},
222-
{ total: results.length, created: 0, skipped: 0, failed: 0 }
223-
);
224-
225140
return response.ok<BulkCreateMcpToolsResponse>({
226141
body: {
227142
results,

‎x-pack/platform/plugins/shared/agent_builder/server/services/tools/tool_types/mcp/index.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,4 +5,4 @@
55
* 2.0.
66
*/
77

8-
export { getMcpToolType, listMcpTools } from './tool_type';
8+
export { getMcpToolType, listMcpTools, getNamedMcpTools } from './tool_type';

‎x-pack/platform/plugins/shared/agent_builder/server/services/tools/tool_types/mcp/tool_type.ts‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ import type { KibanaRequest } from '@kbn/core-http-server';
1212
import type { PluginStartContract as ActionsPluginStart } from '@kbn/actions-plugin/server';
1313
import type { ListToolsResponse } from '@kbn/mcp-client';
1414
import { jsonSchemaToZod } from '@n8n/json-schema-to-zod';
15+
import type { Logger } from '@kbn/core/server';
1516
import type { ToolTypeDefinition } from '../definitions';
1617
import { configurationSchema, configurationUpdateSchema } from './schemas';
1718
import { validateConfig } from './validate_configuration';
@@ -69,6 +70,35 @@ async function getMcpToolInputSchema({
6970
}
7071
}
7172

73+
/**
74+
* Retrieves a specific MCP tool by name by calling listTools on the connector.
75+
* Returns undefined if the connector or tool is not found.
76+
*/
77+
export async function getNamedMcpTools({
78+
actions,
79+
request,
80+
connectorId,
81+
toolNames,
82+
logger,
83+
}: {
84+
actions: ActionsPluginStart;
85+
request: KibanaRequest;
86+
connectorId: string;
87+
toolNames: string[];
88+
logger: Logger;
89+
}): Promise<Array<{ name: string; description?: string }> | undefined> {
90+
try {
91+
const { tools } = await listMcpTools({ actions, request, connectorId });
92+
return tools
93+
.filter((t) => toolNames.includes(t.name))
94+
.map((tool) => ({ name: tool.name, description: tool.description }));
95+
} catch (error) {
96+
// Connector not found or other error - return undefined
97+
logger.error('Error getting named MCP tools: ', error.message ? error.message : String(error));
98+
return undefined;
99+
}
100+
}
101+
72102
/**
73103
* Discriminated union for MCP tool execution results
74104
*/

‎x-pack/platform/plugins/shared/agent_builder/server/services/tools/utils/bulk_create_mcp_tools.ts‎

Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
/*
2+
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
3+
* or more contributor license agreements. Licensed under the Elastic License
4+
* 2.0; you may not use this file except in compliance with the Elastic License
5+
* 2.0.
6+
*/
7+
8+
import { ToolType, validateToolId } from '@kbn/agent-builder-common';
9+
import type { KibanaRequest } from '@kbn/core-http-server';
10+
import type { PluginStartContract as ActionsPluginStart } from '@kbn/actions-plugin/server';
11+
import type { ToolRegistry } from '../tool_registry';
12+
import { validateConnector } from '../tool_types/mcp/validate_configuration';
13+
import type {
14+
BulkCreateMcpToolResult,
15+
BulkCreateMcpToolsResponse,
16+
} from '../../../../common/http_api/tools';
17+
18+
export interface BulkCreateMcpToolsParams {
19+
registry: ToolRegistry;
20+
actions: ActionsPluginStart;
21+
request: KibanaRequest;
22+
connectorId: string;
23+
tools: Array<{
24+
name: string;
25+
description?: string;
26+
}>;
27+
namespace?: string;
28+
skipExisting?: boolean;
29+
tags?: string[];
30+
}
31+
32+
/**
33+
* Bulk create MCP tools from a connector.
34+
* This function can be used both in API routes and server-side code.
35+
*
36+
* @param params - Parameters for bulk creating MCP tools
37+
* @returns Response with results and summary
38+
*/
39+
export async function bulkCreateMcpTools({
40+
registry,
41+
actions,
42+
request,
43+
connectorId,
44+
tools,
45+
namespace,
46+
skipExisting,
47+
tags = [],
48+
}: BulkCreateMcpToolsParams): Promise<BulkCreateMcpToolsResponse> {
49+
// Validate namespace if provided (must be valid tool ID segment)
50+
if (namespace) {
51+
const namespaceError = validateToolId({ toolId: namespace, builtIn: false });
52+
if (namespaceError) {
53+
throw new Error(`Invalid namespace: ${namespaceError}`);
54+
}
55+
}
56+
57+
// Validate connector is MCP type
58+
await validateConnector({
59+
actions,
60+
request,
61+
connectorId,
62+
});
63+
64+
// Precompute tool metadata (toolId, mcpToolName) once per tool
65+
// MCP tool names are server-generated and typically well-formed (e.g., snake_case)
66+
// We just lowercase them; validation in registry.create() handles edge cases
67+
const toolsWithIds = tools.map((tool) => {
68+
const toolName = tool.name.toLowerCase();
69+
const toolId = namespace ? `${namespace}.${toolName}` : toolName;
70+
return { toolId, mcpToolName: tool.name, description: tool.description };
71+
});
72+
73+
// Process tools in parallel using Promise.allSettled (matches bulk delete pattern)
74+
const createResults = await Promise.allSettled(
75+
toolsWithIds.map(async ({ toolId, mcpToolName, description }) => {
76+
// Check if tool already exists
77+
const exists = await registry.has(toolId);
78+
if (exists && skipExisting) {
79+
return { toolId, mcpToolName, skipped: true as const };
80+
}
81+
82+
// Create the MCP tool
83+
await registry.create({
84+
id: toolId,
85+
type: ToolType.mcp,
86+
description: description ?? '',
87+
tags,
88+
configuration: {
89+
connector_id: connectorId,
90+
tool_name: mcpToolName,
91+
},
92+
});
93+
94+
return { toolId, mcpToolName, skipped: false as const };
95+
})
96+
);
97+
98+
// Map results to response format (matches bulk delete pattern)
99+
const results: BulkCreateMcpToolResult[] = createResults.map((result, index) => {
100+
const { toolId, mcpToolName } = toolsWithIds[index];
101+
102+
if (result.status === 'rejected') {
103+
return {
104+
toolId,
105+
mcpToolName,
106+
success: false as const,
107+
reason: result.reason?.toJSON?.() ?? {
108+
error: { message: result.reason?.message ?? 'Unknown error' },
109+
},
110+
};
111+
}
112+
113+
if (result.value.skipped) {
114+
return {
115+
toolId: result.value.toolId,
116+
mcpToolName: result.value.mcpToolName,
117+
success: true as const,
118+
skipped: true as const,
119+
};
120+
}
121+
122+
return {
123+
toolId: result.value.toolId,
124+
mcpToolName: result.value.mcpToolName,
125+
success: true as const,
126+
};
127+
});
128+
129+
// Compute summary counts
130+
const summary = results.reduce(
131+
(acc, r) => {
132+
if (!r.success) acc.failed++;
133+
else if ('skipped' in r && r.skipped) acc.skipped++;
134+
else acc.created++;
135+
return acc;
136+
},
137+
{ total: results.length, created: 0, skipped: 0, failed: 0 }
138+
);
139+
140+
return {
141+
results,
142+
summary,
143+
};
144+
}

‎x-pack/platform/plugins/shared/agent_builder/server/services/tools/utils/index.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,3 +7,5 @@
77

88
export { toDescriptorWithSchema, toExecutableTool } from './tool_conversion';
99
export { getToolTypeInfo } from './get_tool_type_info';
10+
export { bulkCreateMcpTools } from './bulk_create_mcp_tools';
11+
export type { BulkCreateMcpToolsParams } from './bulk_create_mcp_tools';

‎x-pack/platform/plugins/shared/data_catalog/common/data_source_spec.ts‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,19 @@
55
* 2.0.
66
*/
77

8+
import type { SavedObjectAttributes } from '@kbn/core/server';
9+
10+
/**
11+
* Type for Stack connector secrets.
12+
*/
13+
export interface ConnectorSecrets extends SavedObjectAttributes {
14+
token?: string;
15+
apiKey?: string;
16+
user?: string;
17+
password?: string;
18+
secretHeaders?: Record<string, string>;
19+
}
20+
821
/**
922
* OAuth providers supported by EARS
1023
*/
@@ -47,6 +60,7 @@ export interface CustomOAuthConfiguration {
4760
export interface StackConnectorConfig {
4861
type: string;
4962
config: Record<string, unknown>;
63+
importedTools?: string[];
5064
}
5165

5266
/**

‎x-pack/platform/plugins/shared/data_sources/moon.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,7 @@ dependsOn:
4747
- '@kbn/agent-builder-plugin'
4848
- '@kbn/deeplinks-workflows'
4949
- '@kbn/deeplinks-agent-builder'
50+
- '@kbn/connector-schemas'
5051
tags:
5152
- plugin
5253
- prod

0 commit comments

Comments
 (0)