🤖 Documentation Chat Bot

[squidfunk]

We've got something exciting to share with you today: we're launching an experimental chat bot within our documentation! You can access this cool new feature by simply clicking on the adorable little robot icon located in the search bar, and start asking it questions relating to Material for MkDocs, like:

• How to set a background color for social cards?
• How can I use tooltips?
• Explain Material for MkDocs like I'm five.
• ...

We've designed this chat bot to make our documentation search experience more interactive and user-friendly. Before giving it a spin, bear in mind that:

This is an experiment

This chat bot is currently in an exploratory phase. While it's a playful addition to our documentation, it's important to note that we're not yet considering it as a ready-for-primetime feature for integration into Material for MkDocs. Given that we are in the process of completely overhauling our search engine, we see this chat interface as an experimental addition. We are keen to understand if it could be a valuable supplement to the new documentation search capabilities.

Your feedback matters

As we experiment with this chat bot, we need your help and feedback. You can contribute by rating the answers it provides. Feel free to give a thumbs up 👍 or down 👎 based on the accuracy and helpfulness of the responses you receive. Your feedback is invaluable in helping us evaluate whether this chat bot can be a valuable addition to the Material for MkDocs experience. We want to make sure it genuinely enhances your documentation search process and doesn't just add a cute robot to the mix.

Additionally, we invite you to share any feedback, good or bad, in this discussion.

Where we go from here

With your feedback, we will continue to improve the chat bot to learn whether we can make it good enough to be a useful tool that individuals and organizations can add to their documentation with as little effort as possible, giving them a unique new way for users to interface with their projects and products.

So, give it a try, have some fun, and let us know what you think! Your insights will play a crucial role in shaping the future of this feature and determining whether it becomes a permanent part of our toolkit.

Thank you for being a part of our community, and happy exploring with our new experimental chat bot! 🤖📚

[frollow]

An amazing idea, I ask you to implement it even in an intermediate version. We were just looking at external solutions for exactly the same functionality, so that users would ask the bot instead of looking for documentation. This is difficult for many people.

[tlofreso]

Love this idea. The ability to tie LLMs to internal documentation is highly desired. One use case that comes to mind... We use the blog extension to author posts highlighting large projects. It would be great to be able to ask the documentation chat bot for a summary of major projects completed last year / last quarter. Perhaps informing the prompt with tags associated with those posts.

• Provide a summary of major projects completed last quarter.
• What are some new capabilities the Platform Engineering team has enabled recently?

Subscribed to this discussion. Great work.

[squidfunk]

Very valuable feedback, thank you!

[hhslepicka]

I am very interested in this feature! I started looking into LangChain + GPT-4 to ingest our markdown files from our documentation portal and train a chatbot for this exact purpose. Excited to give it a try

[mdsanima]

This is a fantastic experience! I am extremely pleased with the assistance I received. I specifically requested information and instructions in Polish, and I was provided with detailed and precise guidance. The support I received was exceptional, and it exceeded my expectations. I am truly grateful for the outstanding service I received.

[alexvoss]

Yeah, being able to interact in different languages is a pretty cool feature of the generative AI models.

For me it seems to have produced a hallucination, though - I think - not sure I have heard of a custom_css attribute before. It should have been extra_css at the top level of mkdocs.yml.

Well, the English version is interesting for different reasons:

Good job we have @squidfunk's awesome documentation to go back to to check!

[normandy7]

Would be great if it worked reliably, but unless priority is a hidden option, there's some hallucination going on as well...

Before that, I tried asking how to exclude a paragraph from search. I meant something like an inline paragraph, but the bot considered it synonymous with section and only suggested exclusion of entire sections and pages. So I again manually sought out the information I needed, but only because I had a memory that it should be possible to exclude any block.

[alexvoss]

Wonder if it is possible to "protect" technical content from the GAI messing with it? In two of the examples, it has swapped terms, put them into the wrong context and had no understanding (surprise!) of the significance of values. In the search configuration the right term would have been "boost", so it got this close to getting the term right! Not sure if it stands much of a chance to get numbers right, given how these things work?

I will certainly try and use this as a tool to rapidly author some examples and tutorials over the weekend. With some manual tweaks they should be fine and I can save myself a lot of typing. Fortunately, in computing, there is an easy proof of the pudding - it either works or does not. For things of manageable complexity, I'd rather spend my own time reviewing/writing tests/debugging than generating first drafts of code. I am much more worried about other uses of AI.

[squidfunk]

So this is a prime example why we don't consider this ready for prime time 😉 There are hallucinations right now – we are aware of this problem. The whole topic of retrieval augmented generation (RAG) is one of the hottest research topics right now, and it is far from being a solved problem. Our plan is to learn how people use our little bot and maybe next year have a prototype that we can give into the hands of our users to deploy on their documentation! In fact, deploying this bot on our site we already had several learnings that we haven't anticipated, so it's a great experiment to evaluate whether it makes sense to go down this path.

[alexvoss]

Here is a nice example of the real value of the chat bot, which IMHO does not lie in accurately producing versions of the documentation but in adding things on that it has 'learned' elsewhere.

I wanted to know how to manually publish my documents. The audience I am writing for may manage to push changes into a repo but I think that GitHub Actions is not the way to go for them, so the instructions contained in the Material for MkDocs documentation would not help much.

The answer provided by the bot was a very good starting point, though. It suggested creating a gh-pages branch that contains only the site directory. I did not know this was possible - thought that GitHub Pages have to have their own repo. May need to explore the options to see what is possible to use for my audience. Excellent starting point for thinking about this, though.

[ojacques]

I was looking to do the same for myself. Well done on the implementation, it's very simple for the user, and where I would expect it to be.

• For useability, given that some of the markdown we write are code blocks themselves, the example answer sometimes just does not show (but we see the bot typing it!). For example, here I cannot see where to add the title. This may be fixed with a prompt engineered in a way to return escaped markdown.
  
• When answering, can we add a reference back to the documentation? I can do it with a different prompt.
  
• In addition to the mkdocs site itself, I found many answers in the issues. Maybe worth feeding that in (may create more hallucinations though).

I'm happy to contribute - maybe in insiders.

I'm impressed!

[squidfunk]

We'll update this discussion once we consider this as an addition to the core of the theme ☺️

[tyson-swetnam]

This is very exciting - our university coursework and research pages being written in Material would see an immediate benefit from this ChatBot.

I just found this thread because I was looking to add this feature to our current pages!

[sgustafsson]

Awesome feature! Just wanted to share what I think is a hallunication: the documentation lists card grids as sponsors/insiders feature, but the chatbot insists that everyone can use it:

[squidfunk]

#6240 (reply in thread) ☺️

[AbdelrhmanHamouda]

Is it possible to share the technical approach you took to train this bot and how to integrate it with an mkdocs-material site?

[squidfunk]

Currently not, sorry. It's experimental and subject to change. I can only say that we did nothing too fancy (yet).

[lazka]

It told me "I am an AI language model powered by OpenAI's GPT-3" at least. That doesn't have to be true of course :)

[Wcowin]

Amazing job!
But it still has some problem such for the wrong code

[squidfunk]

#6240 (reply in thread) 😉

[rhartsfield]

Is it possible to get the overall chat interface as a plugin that simply sends a rest request to a configurable endpoint? That way we can handle the RAG ourselves and dispatch to a locally hosted LLM.

Would be nice to build docs, ship markdown to get vector embedded for our langchain/llamaindex service to work with, and then just hit that service from docs.

[oliverjfletcher]

Love the idea, once you get closer to a preview it would be good to understand how privately hosted bots or LLMs can be leveraged and which of the public cloud provider services for those you would be supporting, thanks for continuing to build an awesome product :)

[jromal]

I did some RAG tests with my dicumentation and was thinking how could this be integrated into the search.

From this initiative and the new search I can just say that I fully believe you are moving into the right direction. Modularity and caution in the implementation are very important. But having so many people ready to test, do not hesitate to share interim versions.

Thanks for your work

[RyanBaig]

Ping me if it gets added please!

[stozn]

It's really helpful !

[samzong]

It will help us.

[StarfallProjects]

I was excited to see this, it would definitely be helpful for us (at n8n) I'd love to see this as a feature :-)

[fabge]

@squidfunk you are providing the obufscated and minified javascript code for the chatbot under https://chat.squidfunk.com/client.js.
Is it possible to provide the unobfuscated javascript code to take a look and adapt it to ones needs?

[squidfunk]

Currently not – this is only an experiment and very much hacked together. Please watch out for #6307, which is a prerequisite to building a deep integration. You might also consider sponsoring the project, as it allows us to iterate faster if you want to see this feature happen! 🚀

[ianderrington]

Very interested in this. Presently building chatbot one as a separate app using embedchain/streamlit as it will be also parsing pdfs/github etc to help with www.managen.ai

[Vasperous]

@squidfunk do you have any idea or thoughts on when you would make this available to some users as a preview etc.? Looking to implement something similar but would rather have it integrated like you do here.

[squidfunk]

I'm very happy about the interest the prototype sparked, but please understand that I cannot put a specific ETA on this except for a rough "in 2024". It will definitely be part of the new search I'm working on (as another mode of exploring the site), which is another whole project of its own. I still need some time to finish it, which I'm fighting for. As you ca imagine, my time is also needed to fix reported and unreported bugs, answer questions, etc.

Honestly, we're really hoping and pushing hard for more sponsorships right now, because we're in the process of growing our team so we can iterate even faster, but this means we need more funds. Having more to distribute would mean we could pay more individuals for helping out. Please remember that this is an Open Source project currently maintained by 3 individuals, 2 of which are not working on the core of the project.

That being said, I really want to thank you for sponsoring the project for so long!

[Vasperous]

Thanks for the info and I appreciate all the things you (and the contributors) are doing! 😄

[boegel]

Any updates on the current status of this, be it experimental, in development, etc.?

It would be great if a chat bot could be made available as a part of Material for MkDocs, even if it has to somehow a warning of some kind with every answer it gives, since hallucinations will be difficult to 100% prevent...

[squidfunk]

I'm sorry this is taking so long. We're a very small team working on this project (basically only me working on the core) and we needed to re-prioritize other, bigger picture topics which we're working tirelessly on in the background. There will be an announcement in the coming weeks. Once we got the other topics sorted and search is out (which is currently paused as well for the same reasons), we'll start work on the chat bot.

[HydrogenSulfate]

This is really great! LLM is indeed very well-suited for projects that require a bit of complexity and precise configurations. However, I have a question: why can't I see the robot now? Has it gone offline?

[squidfunk]

... at the expense of running an LLM or integrating with an upstream service like ChatGPT 😉 Nonetheless, yes, before just slapping a chatbot into our current implementation of search, we started rethinking search entirely in #6307 and LLMs will play an important, yet always optional part in the very near future.

[pete-willisSP]

Hi There, I am interested if there is progress on this and if it would be implemented. Many thanks.

[squidfunk]

No progress, we're currently focusing on foundational work.

[benomahony]

@squidfunk are you open to contributions for this feature?

[squidfunk]

Thanks for offering! Currently not, as we're in feature freeze right now due to the aforementioned foundational work. Once that's out the door, we'll re-evaluate. Thanks for your patience!

[squidfunk]

Hey all! As I've mentioned several times in this discussion and in other places, we're currently focusing on foundational work, addressing all of the short comings and problems when creating technical documentation and knowledge bases. This concerns many of the currently open feature requests in Material for MkDocs, but also performance issues coming from MkDocs itself.

Thus, we're currently not considering progressing with integrating a documentation chatbot interface. We'll revisit LLMs as part of the new search we've also been working on, which we'll release once the foundational work is done and out the door.

To mitigate the occasional necroposting, I'm closing and locking this discussion for now. We kindly ask for your understanding 🤟