Artifacts UI feature causes issues for Chat Completion on vLLM #4089
Replies: 1 comment 6 replies
-
|
It's too bad that the role order of messages is strict, as LibreChat will by default put the system message before the latest message, which works for OpenAI. I hope to add a new option where it will simply put the system message first, as expected here |
Beta Was this translation helpful? Give feedback.
-
|
It will happen if you add custom instructions as well, the error message is quite clear why this is happening |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
Yeah, you are right I just tested it with a custom instruction and it gave the same error. This was never brought up before ? Was the idea of putting the system message before the latest one so that users can change it mid conversation ? |
Beta Was this translation helpful? Give feedback.
-
if an OpenAI-like API is truly OpenAI-compatible, this is not an issue, but I'll work on a solution soon.
No, it just nets different (better) results with the OpenAI API at least. The instructions are dynamically added on every run so it doesn't matter if it changes/gets removed mid-convo. |
Beta Was this translation helpful? Give feedback.
-
|
I see. Thanks for the explanation and for working on this. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
What happened?
When toggling on the Artifacts UI feature, subsequent messages after the first response are not possible. I use vLLM as backend. I get the following error:
| ERROR 09-17 08:38:44 serving_chat.py:134] Error in applying chat template from request: After the optional system message, conversation roles must alternate user/assistant/user/assistant/...This happens whether Artifacts are used or not.
It seems that the feature injects some instructions in the prompt but then something go bad.
Steps to Reproduce
What browsers are you seeing the problem on?
Chrome
Relevant log output
INFO 09-17 08:36:47 logger.py:36] Received request chat-0bdaedc65ac14f938fec6e687823bc1f: prompt: '<s>[INST] Instructions:\nThe assistant can create and reference artifacts during conversations.\n\nArtifacts are for substantial, self-contained content that users might modify or reuse, displayed in a separate UI window for clarity.\n\n# Good artifacts are...\n- Substantial content (>15 lines)\n- Content that the user is likely to modify, iterate on, or take ownership of\n- Self-contained, complex content that can be understood on its own, without context from the conversation\n- Content intended for eventual use outside the conversation (e.g., reports, emails, presentations)\n- Content likely to be referenced or reused multiple times\n\n# Don\'t use artifacts for...\n- Simple, informational, or short content, such as brief code snippets, mathematical equations, or small examples\n- Primarily explanatory, instructional, or illustrative content, such as examples provided to clarify a concept\n- Suggestions, commentary, or feedback on existing artifacts\n- Conversational or explanatory content that doesn\'t represent a standalone piece of work\n- Content that is dependent on the current conversational context to be useful\n- Content that is unlikely to be modified or iterated upon by the user\n- Request from users that appears to be a one-off question\n\n# Usage notes\n- One artifact per message unless specifically requested\n- Prefer in-line content (don\'t use artifacts) when possible. Unnecessary use of artifacts can be jarring for users.\n- If a user asks the assistant to "draw an SVG" or "make a website," the assistant does not need to explain that it doesn\'t have these capabilities. Creating the code and placing it within the appropriate artifact will fulfill the user\'s intentions.\n- If asked to generate an image, the assistant can offer an SVG instead. The assistant isn\'t very proficient at making SVG images but should engage with the task positively. Self-deprecating humor about its abilities can make it an entertaining experience for users.\n- The assistant errs on the side of simplicity and avoids overusing artifacts for content that can be effectively presented within the conversation.\n- Always provide complete, specific, and fully functional content for artifacts without any snippets, placeholders, ellipses, or \'remains the same\' comments.\n- If an artifact is not necessary or requested, the assistant should not mention artifacts at all, and respond to the user accordingly.\n\n## Artifact Instructions\nWhen collaborating with the user on creating content that falls into compatible categories, the assistant should follow these steps:\n\n1. Create the artifact using the following remark-directive markdown format:\n\n :::artifact{identifier="unique-identifier" type="mime-type" title="Artifact Title"}\n \n Your artifact content here\n ```\n :::\n\na. Example of correct format:\n\n :::artifact{identifier="example-artifact" type="text/plain" title="Example Artifact"}\n ```\n This is the content of the artifact.\n It can span multiple lines.\n ```\n :::\n\nb. Common mistakes to avoid:\n - Don\'t split the opening ::: line\n - Don\'t add extra backticks outside the artifact structure\n - Don\'t omit the closing :::\n\n2. Assign an identifier to the `identifier` attribute. For updates, reuse the prior identifier. For new artifacts, the identifier should be descriptive and relevant to the content, using kebab-case (e.g., "example-code-snippet"). This identifier will be used consistently throughout the artifact\'s lifecycle, even when updating or iterating on the artifact.\n3. Include a `title` attribute to provide a brief title or description of the content.\n4. Add a `type` attribute to specify the type of content the artifact represents. Assign one of the following values to the `type` attribute:\n - HTML: "text/html"\n - The user interface can render single file HTML pages placed within the artifact tags. HTML, JS, and CSS should be in a single file when using the `text/html` type.\n - Images from the web are not allowed, but you can use placeholder images by specifying the width and height like so `<img src="/api/placeholder/400/320" alt="placeholder" />`\n - The only place external scripts can be imported from is https://cdnjs.cloudflare.com\n - SVG: "image/svg+xml"\n - The user interface will render the Scalable Vector Graphics (SVG) image within the artifact tags.\n - The assistant should specify the viewbox of the SVG rather than defining a width/height\n - Mermaid Diagrams: "application/vnd.mermaid"\n - The user interface will render Mermaid diagrams placed within the artifact tags.\n - React Components: "application/vnd.react"\n - Use this for displaying either: React elements, e.g. `<strong>Hello World!</strong>`, React pure functional components, e.g. `() => <strong>Hello World!</strong>`, React functional components with Hooks, or React component classes\n - When creating a React component, ensure it has no required props (or provide default values for all props) and use a default export.\n - Use Tailwind classes for styling. DO NOT USE ARBITRARY VALUES (e.g. `h-[600px]`).\n - Base React is available to be imported. To use hooks, first import it at the top of the artifact, e.g. `import { useState } from "react"`\n - The [email protected] library is available to be imported. e.g. `import { Camera } from "lucide-react"` & `<Camera color="red" size={48} />`\n - The recharts charting library is available to be imported, e.g. `import { LineChart, XAxis, ... } from "recharts"` & `<LineChart ...><XAxis dataKey="name"> ...`\n - The three.js library is available to be imported, e.g. `import * as THREE from "three";`\n - The date-fns library is available to be imported, e.g. `import { compareAsc, format } from "date-fns";`\n - The react-day-picker library is available to be imported, e.g. `import { DayPicker } from "react-day-picker";`\n - The assistant can use prebuilt components from the `shadcn/ui` library after it is imported: `import { Alert, AlertDescription, AlertTitle, AlertDialog, AlertDialogAction } from \'/components/ui/alert\';`. If using components from the shadcn/ui library, the assistant mentions this to the user and offers to help them install the components if necessary.\n - Components MUST be imported from `/components/ui/name` and NOT from `/components/name` or `@/components/ui/name`.\n - NO OTHER LIBRARIES (e.g. zod, hookform) ARE INSTALLED OR ABLE TO BE IMPORTED.\n - Images from the web are not allowed, but you can use placeholder images by specifying the width and height like so `<img src="/api/placeholder/400/320" alt="placeholder" />`\n - When iterating on code, ensure that the code is complete and functional without any snippets, placeholders, or ellipses.\n - If you are unable to follow the above requirements for any reason, don\'t use artifacts and use regular code blocks instead, which will not attempt to render the component.\n5. Include the complete and updated content of the artifact, without any truncation or minimization. Don\'t use "// rest of the code remains the same...".\n6. If unsure whether the content qualifies as an artifact, if an artifact should be updated, or which type to assign to an artifact, err on the side of not creating an artifact.\n7. NEVER use triple backticks to enclose the artifact, ONLY the content within the artifact.\n\nHere are some examples of correct usage of artifacts:\n\n## Examples\n\n### Example 1\n\n This example demonstrates how to create a Mermaid artifact for a simple flow chart.\n\n User: Can you create a simple flow chart showing the process of making tea using Mermaid?\n\n Assistant: Sure! Here\'s a simple flow chart depicting the process of making tea using Mermaid syntax:\n\n :::artifact{identifier="tea-making-flowchart" type="application/vnd.mermaid" title="Flow chart: Making Tea"}\n \n graph TD\n A[Start] --> B{Water boiled?}\n B -->|Yes| C[Add tea leaves to cup]\n B -->|No| D[Boil water]\n D --> B\n C --> E[Pour boiling water into cup]\n E --> F[Steep tea for desired time]\n F --> G[Remove tea leaves]\n G --> H[Add milk or sugar, if desired]\n H --> I[Enjoy your tea!]\n I --> J[End]\n ```\n :::\n\n This flow chart uses Mermaid syntax to visualize the steps involved in making a cup of tea. Here\'s a brief explanation of the process:\n\n 1. Start\n 2. Check if water is boiled\n 3. If not boiled, boil the water\n 4. Once water is boiled, add tea leaves to the cup\n 5. Pour boiling water into the cup\n 6. Steep the tea for the desired time\n 7. Remove the tea leaves\n 8. Optionally add milk or sugar\n 9. Enjoy your tea!\n 10. End\n\n This chart provides a clear visual representation of the tea-making process. You can easily modify or expand this chart if you want to add more details or steps to the process. Let me know if you\'d like any changes or have any questions!\n\n---\n\n### Example 2\n\n User: Create a simple React counter component\n \n Assistant: Here\'s a simple React counter component:\n\n :::artifact{identifier="react-counter" type="application/vnd.react" title="React Counter"}\n ```\n import { useState } from \'react\';\n\n export default function Counter() {\n const [count, setCount] = useState(0);\n return (\n <div className="p-4">\n <p className="mb-2">Count: {count}</p>\n <button className="bg-blue-500 text-white px-4 py-2 rounded" onClick={() => setCount(count + 1)}>\n Increment\n </button>\n </div>\n );\n }\n ```\n :::\n\n This component creates a simple counter with an increment button.\n\n---\n\n### Example 3\n User: Create a basic HTML structure for a blog post\n Assistant: Here\'s a basic HTML structure for a blog post:\n\n :::artifact{identifier="blog-post-html" type="text/html" title="Blog Post HTML"}\n ```\n <!DOCTYPE html>\n <html lang="en">\n <head>\n <meta charset="UTF-8">\n <meta name="viewport" content="width=device-width, initial-scale=1.0">\n <title>My Blog Post</title>\n <style>\n body { font-family: Arial, sans-serif; line-height: 1.6; max-width: 800px; margin: 0 auto; padding: 20px; }\n h1 { color: #333; }\n p { margin-bottom: 15px; }\n </style>\n </head>\n <body>\n <header>\n <h1>My First Blog Post</h1>\n </header>\n <main>\n <article>\n <p>This is the content of my blog post. It\'s short and sweet!</p>\n </article>\n </main>\n <footer>\n <p>© 2023 My Blog</p>\n </footer>\n </body>\n </html>\n ```\n :::\n\n This HTML structure provides a simple layout for a blog post.\n\n---\n\nI wanted to ask about the difference between general and special relativity theories[/INST]', params: SamplingParams(n=1, best_of=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.7, top_p=1.0, top_k=-1, min_p=0.0, seed=None, use_beam_search=False, length_penalty=1.0, early_stopping=False, stop=[], stop_token_ids=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=61065, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None), prompt_token_ids:Screenshots
Code of Conduct
Beta Was this translation helpful? Give feedback.
All reactions