[8.x] [Obs Ai Assistant] Add system message (#209773) (#211396)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Obs Ai Assistant] Add system message
(#209773)](#209773)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Arturo
Lidueña","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-02-17T10:12:56Z","message":"[Obs
Ai Assistant] Add system message (#209773)\n\nFix: System Message
Missing in Inference Plugin\r\nCloses #209548\r\n## Summary\r\n\r\nA
regression was introduced in
8.18\r\n([#199286](#199286)),
where the\r\nsystem message is no longer passed to the inference plugin
and,\r\nconsequently, the LLM.\r\n\r\nCurrently, only user messages are
being sent, which impacts conversation\r\nguidance and guardrails. The
system message is crucial for steering\r\nresponses and maintaining
contextual integrity.\r\n\r\nThe filtering of the system message happens
here:\r\n\r\nhttps://github.com/elastic/kibana/blob/771a080ffa99e501e72c9cb98c833795769483ae/x-pack/platform/plugins/shared/observability_ai_assistant/server/service/client/index.ts#L510-L512\r\n\r\nFix
Approach\r\n- Ensure the `system` message is included as a parameter
in\r\n`inferenceClient.chatComplete.`\r\n```typescript\r\nconst options
= {\r\n connectorId,\r\n system,\r\n messages:
convertMessagesForInference(messages),\r\n toolChoice,\r\n tools,\r\n
functionCalling: (simulateFunctionCalling ? 'simulated' : 'native') as
FunctionCallingMode,\r\n };\r\n if (stream) {\r\n return defer(() =>\r\n
this.dependencies.inferenceClient.chatComplete({\r\n ...options,\r\n
stream: true,\r\n })\r\n ).pipe(\r\n
convertInferenceEventsToStreamingEvents(),\r\n
instrumentAndCountTokens(name),\r\n failOnNonExistingFunctionCall({
functions }),\r\n tap((event) => {\r\n if (\r\n event.type ===
StreamingChatResponseEventType.ChatCompletionChunk &&\r\n
this.dependencies.logger.isLevelEnabled('trace')\r\n ) {\r\n
this.dependencies.logger.trace(`Received chunk:
${JSON.stringify(event.message)}`);\r\n }\r\n }),\r\n shareReplay()\r\n
) as TStream extends true\r\n ? Observable<ChatCompletionChunkEvent |
TokenCountEvent | ChatCompletionMessageEvent>\r\n : never;\r\n } else
{\r\n return this.dependencies.inferenceClient.chatComplete({\r\n
...options,\r\n stream: false,\r\n }) as TStream extends true ? never :
Promise<ChatCompleteResponse>;\r\n }\r\n }\r\n ```\r\n- Add an API test
to verify that the system message is correctly passed to the
LLM.","sha":"0ae28aa8bc73ce682166ffd5666828b1fdb7c017","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","v9.0.0","Team:Obs
AI
Assistant","ci:project-deploy-observability","backport:version","v9.1.0","v8.19.0"],"title":"[Obs
Ai Assistant] Add system
message","number":209773,"url":"https://github.com/elastic/kibana/pull/209773","mergeCommit":{"message":"[Obs
Ai Assistant] Add system message (#209773)\n\nFix: System Message
Missing in Inference Plugin\r\nCloses #209548\r\n## Summary\r\n\r\nA
regression was introduced in
8.18\r\n([#199286](#199286)),
where the\r\nsystem message is no longer passed to the inference plugin
and,\r\nconsequently, the LLM.\r\n\r\nCurrently, only user messages are
being sent, which impacts conversation\r\nguidance and guardrails. The
system message is crucial for steering\r\nresponses and maintaining
contextual integrity.\r\n\r\nThe filtering of the system message happens
here:\r\n\r\nhttps://github.com/elastic/kibana/blob/771a080ffa99e501e72c9cb98c833795769483ae/x-pack/platform/plugins/shared/observability_ai_assistant/server/service/client/index.ts#L510-L512\r\n\r\nFix
Approach\r\n- Ensure the `system` message is included as a parameter
in\r\n`inferenceClient.chatComplete.`\r\n```typescript\r\nconst options
= {\r\n connectorId,\r\n system,\r\n messages:
convertMessagesForInference(messages),\r\n toolChoice,\r\n tools,\r\n
functionCalling: (simulateFunctionCalling ? 'simulated' : 'native') as
FunctionCallingMode,\r\n };\r\n if (stream) {\r\n return defer(() =>\r\n
this.dependencies.inferenceClient.chatComplete({\r\n ...options,\r\n
stream: true,\r\n })\r\n ).pipe(\r\n
convertInferenceEventsToStreamingEvents(),\r\n
instrumentAndCountTokens(name),\r\n failOnNonExistingFunctionCall({
functions }),\r\n tap((event) => {\r\n if (\r\n event.type ===
StreamingChatResponseEventType.ChatCompletionChunk &&\r\n
this.dependencies.logger.isLevelEnabled('trace')\r\n ) {\r\n
this.dependencies.logger.trace(`Received chunk:
${JSON.stringify(event.message)}`);\r\n }\r\n }),\r\n shareReplay()\r\n
) as TStream extends true\r\n ? Observable<ChatCompletionChunkEvent |
TokenCountEvent | ChatCompletionMessageEvent>\r\n : never;\r\n } else
{\r\n return this.dependencies.inferenceClient.chatComplete({\r\n
...options,\r\n stream: false,\r\n }) as TStream extends true ? never :
Promise<ChatCompleteResponse>;\r\n }\r\n }\r\n ```\r\n- Add an API test
to verify that the system message is correctly passed to the
LLM.","sha":"0ae28aa8bc73ce682166ffd5666828b1fdb7c017"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/209773","number":209773,"mergeCommit":{"message":"[Obs
Ai Assistant] Add system message (#209773)\n\nFix: System Message
Missing in Inference Plugin\r\nCloses #209548\r\n## Summary\r\n\r\nA
regression was introduced in
8.18\r\n([#199286](#199286)),
where the\r\nsystem message is no longer passed to the inference plugin
and,\r\nconsequently, the LLM.\r\n\r\nCurrently, only user messages are
being sent, which impacts conversation\r\nguidance and guardrails. The
system message is crucial for steering\r\nresponses and maintaining
contextual integrity.\r\n\r\nThe filtering of the system message happens
here:\r\n\r\nhttps://github.com/elastic/kibana/blob/771a080ffa99e501e72c9cb98c833795769483ae/x-pack/platform/plugins/shared/observability_ai_assistant/server/service/client/index.ts#L510-L512\r\n\r\nFix
Approach\r\n- Ensure the `system` message is included as a parameter
in\r\n`inferenceClient.chatComplete.`\r\n```typescript\r\nconst options
= {\r\n connectorId,\r\n system,\r\n messages:
convertMessagesForInference(messages),\r\n toolChoice,\r\n tools,\r\n
functionCalling: (simulateFunctionCalling ? 'simulated' : 'native') as
FunctionCallingMode,\r\n };\r\n if (stream) {\r\n return defer(() =>\r\n
this.dependencies.inferenceClient.chatComplete({\r\n ...options,\r\n
stream: true,\r\n })\r\n ).pipe(\r\n
convertInferenceEventsToStreamingEvents(),\r\n
instrumentAndCountTokens(name),\r\n failOnNonExistingFunctionCall({
functions }),\r\n tap((event) => {\r\n if (\r\n event.type ===
StreamingChatResponseEventType.ChatCompletionChunk &&\r\n
this.dependencies.logger.isLevelEnabled('trace')\r\n ) {\r\n
this.dependencies.logger.trace(`Received chunk:
${JSON.stringify(event.message)}`);\r\n }\r\n }),\r\n shareReplay()\r\n
) as TStream extends true\r\n ? Observable<ChatCompletionChunkEvent |
TokenCountEvent | ChatCompletionMessageEvent>\r\n : never;\r\n } else
{\r\n return this.dependencies.inferenceClient.chatComplete({\r\n
...options,\r\n stream: false,\r\n }) as TStream extends true ? never :
Promise<ChatCompleteResponse>;\r\n }\r\n }\r\n ```\r\n- Add an API test
to verify that the system message is correctly passed to the
LLM.","sha":"0ae28aa8bc73ce682166ffd5666828b1fdb7c017"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Arturo Lidueña <[email protected]>

Author: kibanamachine

x-pack/platform/packages/shared/kbn-ai-assistant/src/chat/chat_body.tsx

@@ -368,7 +368,7 @@ export function ChatBody({
               paddingSize="m"
               className={animClassName}
             >
-              {connectors.connectors?.length === 0 || messages.length === 1 ? (
+              {connectors.connectors?.length === 0 || messages.length === 0 ? (
                 <WelcomeMessage
                   connectors={connectors}
                   knowledgeBase={knowledgeBase}

x-pack/platform/packages/shared/kbn-ai-assistant/src/chat/chat_item_avatar.tsx

@@ -7,7 +7,7 @@
 
 import React from 'react';
 import { UserAvatar } from '@kbn/user-profile-components';
-import { EuiAvatar, EuiLoadingSpinner } from '@elastic/eui';
+import { EuiLoadingSpinner } from '@elastic/eui';
 import type { AuthenticatedUser } from '@kbn/security-plugin/common';
 import { MessageRole } from '@kbn/observability-ai-assistant-plugin/public';
 import { AssistantAvatar } from '@kbn/ai-assistant-icon';
@@ -34,9 +34,6 @@ export function ChatItemAvatar({ currentUser, role, loading }: ChatAvatarProps)
     case MessageRole.Function:
       return <AssistantAvatar name="Elastic Assistant" color="subdued" size="m" />;
 
-    case MessageRole.System:
-      return <EuiAvatar name="system" iconType="dot" color="subdued" />;
-
     default:
       return null;
   }

x-pack/platform/packages/shared/kbn-ai-assistant/src/hooks/use_conversation.test.tsx

@@ -116,16 +116,8 @@ describe('useConversation', () => {
       });
     });
 
-    it('returns only the system message', () => {
-      expect(hookResult.result.current.messages).toEqual([
-        {
-          '@timestamp': expect.any(String),
-          message: {
-            content: '',
-            role: MessageRole.System,
-          },
-        },
-      ]);
+    it('returns empty messages', () => {
+      expect(hookResult.result.current.messages).toEqual([]);
     });
 
     it('returns a ready state', () => {
@@ -157,15 +149,8 @@ describe('useConversation', () => {
       });
     });
 
-    it('returns the system message and the initial messages', () => {
+    it('returns the initial messages', () => {
       expect(hookResult.result.current.messages).toEqual([
-        {
-          '@timestamp': expect.any(String),
-          message: {
-            content: '',
-            role: MessageRole.System,
-          },
-        },
         {
           '@timestamp': expect.any(String),
           message: {
@@ -183,14 +168,8 @@ describe('useConversation', () => {
         conversation: {
           id: 'my-conversation-id',
         },
+        systemMessage: 'System',
         messages: [
-          {
-            '@timestamp': new Date().toISOString(),
-            message: {
-              role: MessageRole.System,
-              content: 'System',
-            },
-          },
           {
             '@timestamp': new Date().toISOString(),
             message: {
@@ -218,14 +197,8 @@ describe('useConversation', () => {
         conversation: {
           id: 'my-conversation-id',
         },
+        systemMessage: 'System',
         messages: [
-          {
-            '@timestamp': expect.any(String),
-            message: {
-              content: 'System',
-              role: MessageRole.System,
-            },
-          },
           {
             '@timestamp': expect.any(String),
             message: {
@@ -239,13 +212,6 @@ describe('useConversation', () => {
 
     it('sets messages to the messages of the conversation', () => {
       expect(hookResult.result.current.messages).toEqual([
-        {
-          '@timestamp': expect.any(String),
-          message: {
-            content: expect.any(String),
-            role: MessageRole.System,
-          },
-        },
         {
           '@timestamp': expect.any(String),
           message: {
@@ -255,10 +221,6 @@ describe('useConversation', () => {
         },
       ]);
     });
-
-    it('overrides the system message', () => {
-      expect(hookResult.result.current.messages[0].message.content).toBe('');
-    });
   });
 
   describe('with a conversation id that fails to load', () => {
@@ -282,21 +244,14 @@ describe('useConversation', () => {
     });
 
     it('resets the messages', () => {
-      expect(hookResult.result.current.messages.length).toBe(1);
+      expect(hookResult.result.current.messages.length).toBe(0);
     });
   });
 
   describe('when chat completes', () => {
     const subject: Subject<StreamingChatResponseEventWithoutError> = new Subject();
     let onConversationUpdate: jest.Mock;
     const expectedMessages = [
-      {
-        '@timestamp': expect.any(String),
-        message: {
-          role: MessageRole.System,
-          content: '',
-        },
-      },
       {
         '@timestamp': expect.any(String),
         message: {
@@ -333,6 +288,7 @@ describe('useConversation', () => {
             conversation: {
               id: 'my-conversation-id',
             },
+            systemMessage: '',
             messages: expectedMessages,
           },
           (request as any).params.body

x-pack/platform/packages/shared/kbn-ai-assistant/src/utils/builders.ts

@@ -32,19 +32,6 @@ export function buildMessage(params: BuildMessageProps): Message {
   );
 }
 
-export function buildSystemMessage(
-  params?: Omit<BuildMessageProps, 'message'> & {
-    message: DeepPartial<Omit<Message['message'], 'role'>>;
-  }
-) {
-  return buildMessage(
-    // @ts-expect-error upgrade typescript v5.1.6
-    merge({}, params, {
-      message: { role: MessageRole.System },
-    })
-  );
-}
-
 export function buildUserMessage(
   params?: Omit<BuildMessageProps, 'message'> & {
     message?: DeepPartial<Omit<Message['message'], 'role'>>;
@@ -117,7 +104,8 @@ export function buildConversation(params?: Partial<Conversation>): Conversation
       title: '',
       last_updated: '',
     },
-    messages: [buildSystemMessage()],
+    systemMessage: '',
+    messages: [],
     labels: {},
     numeric_labels: {},
     namespace: '',

x-pack/platform/packages/shared/kbn-ai-assistant/src/utils/create_mock_chat_service.ts

@@ -8,7 +8,6 @@
 import type { DeeplyMockedKeys } from '@kbn/utility-types-jest';
 import {
   FunctionDefinition,
-  MessageRole,
   ObservabilityAIAssistantChatService,
 } from '@kbn/observability-ai-assistant-plugin/public';
 import { BehaviorSubject } from 'rxjs';
@@ -25,13 +24,7 @@ export const createMockChatService = (): MockedChatService => {
     hasFunction: jest.fn().mockReturnValue(false),
     hasRenderFunction: jest.fn().mockReturnValue(true),
     renderFunction: jest.fn(),
-    getSystemMessage: jest.fn().mockReturnValue({
-      '@timestamp': new Date().toISOString(),
-      message: {
-        role: MessageRole.System,
-        content: '',
-      },
-    }),
+    getSystemMessage: jest.fn().mockReturnValue('system message'),
     getScopes: jest.fn(),
   };
   return mockChatService;

x-pack/platform/packages/shared/kbn-ai-assistant/src/utils/get_role_translation.ts

@@ -15,12 +15,6 @@ export function getRoleTranslation(role: MessageRole) {
     });
   }
 
-  if (role === MessageRole.System) {
-    return i18n.translate('xpack.aiAssistant.chatTimeline.messages.system.label', {
-      defaultMessage: 'System',
-    });
-  }
-
   return i18n.translate('xpack.aiAssistant.chatTimeline.messages.elasticAssistant.label', {
     defaultMessage: 'Elastic Assistant',
   });

x-pack/platform/packages/shared/kbn-ai-assistant/src/utils/get_timeline_items_from_conversation.test.tsx

@@ -59,13 +59,6 @@ describe('getTimelineItemsFromConversation', () => {
         },
         chatState: ChatState.Ready,
         messages: [
-          {
-            '@timestamp': new Date().toISOString(),
-            message: {
-              role: MessageRole.System,
-              content: 'System',
-            },
-          },
           {
             '@timestamp': new Date().toISOString(),
             message: {
@@ -77,7 +70,7 @@ describe('getTimelineItemsFromConversation', () => {
         onActionClick: jest.fn(),
       });
     });
-    it('excludes the system message', () => {
+    it('includes the opening message and the user message', () => {
       expect(items.length).toBe(2);
       expect(items[0].title).toBe('started a conversation');
     });
@@ -113,13 +106,6 @@ describe('getTimelineItemsFromConversation', () => {
         hasConnector: true,
         chatState: ChatState.Ready,
         messages: [
-          {
-            '@timestamp': new Date().toISOString(),
-            message: {
-              role: MessageRole.System,
-              content: 'System',
-            },
-          },
           {
             '@timestamp': new Date().toISOString(),
             message: {
@@ -204,13 +190,6 @@ describe('getTimelineItemsFromConversation', () => {
         hasConnector: true,
         chatState: ChatState.Ready,
         messages: [
-          {
-            '@timestamp': new Date().toISOString(),
-            message: {
-              role: MessageRole.System,
-              content: 'System',
-            },
-          },
           {
             '@timestamp': new Date().toISOString(),
             message: {
@@ -282,13 +261,6 @@ describe('getTimelineItemsFromConversation', () => {
         hasConnector: true,
         chatState: ChatState.Ready,
         messages: [
-          {
-            '@timestamp': new Date().toISOString(),
-            message: {
-              role: MessageRole.System,
-              content: 'System',
-            },
-          },
           {
             '@timestamp': new Date().toISOString(),
             message: {
@@ -364,13 +336,6 @@ describe('getTimelineItemsFromConversation', () => {
         },
         chatState: ChatState.Ready,
         messages: [
-          {
-            '@timestamp': new Date().toISOString(),
-            message: {
-              role: MessageRole.System,
-              content: 'System',
-            },
-          },
           {
             '@timestamp': new Date().toISOString(),
             message: {
@@ -415,13 +380,6 @@ describe('getTimelineItemsFromConversation', () => {
         hasConnector: true,
         chatState: ChatState.Ready,
         messages: [
-          {
-            '@timestamp': new Date().toISOString(),
-            message: {
-              role: MessageRole.System,
-              content: 'System',
-            },
-          },
           {
             '@timestamp': new Date().toISOString(),
             message: {
@@ -491,13 +449,6 @@ describe('getTimelineItemsFromConversation', () => {
         hasConnector: true,
         chatState: ChatState.Loading,
         messages: [
-          {
-            '@timestamp': new Date().toISOString(),
-            message: {
-              role: MessageRole.System,
-              content: 'System',
-            },
-          },
           {
             '@timestamp': new Date().toISOString(),
             message: {

x-pack/platform/packages/shared/kbn-ai-assistant/src/utils/get_timeline_items_from_conversation.tsx

@@ -80,10 +80,6 @@ export function getTimelineItemsfromConversation({
     payload: ChatActionClickPayload;
   }) => void;
 }): ChatTimelineItem[] {
-  const messagesWithoutSystem = messages.filter(
-    (message) => message.message.role !== MessageRole.System
-  );
-
   const items: ChatTimelineItem[] = [
     {
       id: v4(),
@@ -100,18 +96,16 @@ export function getTimelineItemsfromConversation({
       }),
       role: MessageRole.User,
     },
-    ...messagesWithoutSystem.map((message, index) => {
+    ...messages.map((message, index) => {
       const id = v4();
 
       let title: React.ReactNode = '';
       let content: string | undefined;
       let element: React.ReactNode | undefined;
 
       const prevFunctionCall =
-        message.message.name &&
-        messagesWithoutSystem[index - 1] &&
-        messagesWithoutSystem[index - 1].message.function_call
-          ? messagesWithoutSystem[index - 1].message.function_call
+        message.message.name && messages[index - 1] && messages[index - 1].message.function_call
+          ? messages[index - 1].message.function_call
           : undefined;
 
       let role = message.message.function_call?.trigger || message.message.role;

x-pack/platform/plugins/private/translations/translations/fr-FR.json

@@ -9840,7 +9840,6 @@
     "xpack.aiAssistant.chatTimeline.actions.editPrompt": "Modifier l'invite",
     "xpack.aiAssistant.chatTimeline.actions.inspectPrompt": "Inspecter l'invite",
     "xpack.aiAssistant.chatTimeline.messages.elasticAssistant.label": "Assistant d'Elastic",
-    "xpack.aiAssistant.chatTimeline.messages.system.label": "Système",
     "xpack.aiAssistant.chatTimeline.messages.user.label": "Vous",
     "xpack.aiAssistant.checkingKbAvailability": "Vérification de la disponibilité de la base de connaissances",
     "xpack.aiAssistant.conversationList.dateGroupTitle.lastMonth": "Le mois dernier",

x-pack/platform/plugins/private/translations/translations/ja-JP.json

@@ -9715,7 +9715,6 @@
     "xpack.aiAssistant.chatTimeline.actions.editPrompt": "プロンプトを編集",
     "xpack.aiAssistant.chatTimeline.actions.inspectPrompt": "プロンプトを検査",
     "xpack.aiAssistant.chatTimeline.messages.elasticAssistant.label": "Elastic Assistant",
-    "xpack.aiAssistant.chatTimeline.messages.system.label": "システム",
     "xpack.aiAssistant.chatTimeline.messages.user.label": "あなた",
     "xpack.aiAssistant.checkingKbAvailability": "ナレッジベースの利用可能性を確認中",
     "xpack.aiAssistant.conversationList.dateGroupTitle.lastMonth": "先月",