configure classifiers and transitions from extensions

[ryancurrah]

When connecting to an extension server it would awesome if we could send the bot our own classifiers and transitions. This would allow us to programmatically generate questions and answers.

An example where this would be used is the company I work at. We have an internal StackOverflow site. We could write an extension that gets all the questions and answers from that site and generate classifiers and transitions and register them with the bot.

It would be super powerful for automatically training the bot to respond to questions with a set of predefined answers from other tools.

[jaimeteb]

Wow, yeah, this would be a great way to constantly train the bot and make it keep learning even after the development stage. This is actually something that we've wanted to do with other chatbots at work, so I see the potential.

However, I don't know if extensions are the right place for this feature, maybe a new /bot endpoint could handle this, I'm not sure. Do you have a more concrete idea of how to do this?

[ryancurrah]

I've been thinking about this issue as well as #24 and #18 and a solution that can solve all three.

Change extension servers to use events for receiving a command by using WebSockets

Add a feature to the Bot that will produce extension events over WebSockets instead of calling an extension REST/RPC endpoint.

The extension server would then no longer need to implement a web service and the complexities. Like firewall access from the bot, authorization, and setting up endpoints.

The extension server would then become an event consumer. We could use a tool similar to https://www.asyncapi.com/ to generate consumer clients in multiple languages. This would address #24 as generated clients and consumers are easier to implement than generated servers.

Generate client SDKs

We would then generate and publish client SDKs for the Bot REST API and Extension WebSocket Event API in multiple languages. Extension servers would import these clients.

Classifier and FSM instance per extension

When an extension server comes online it can register its extensions with the Bot along with Classifiers and FSM transitions. The Bot would have the endpoints /bot/classifiers/<extension_name> and /bot/fsm/<extension_name>. Which provides the mechanism to register. In the Classifier and FSM there would be multiple instances, the default instance which is loaded by the Bot at the start, and an instance per extension. Having an instance per extension allows configuring without worrying about overwriting each other.

[jaimeteb]

I like the idea of using the Bot as the centerpiece of the system. If I understand correctly, what you're proposing is: if an extension is to be run, the Bot produces an event, then the extension server consumes it, and finally, the extension makes a request to a Bot endpoint to register its result.

Some remarks:

• I wouldn't want to get rid of the REST extension server, because of how easy it is to integrate. For example, I'm developing a serverless Chatto bot, which involves a serverless extension server as well, and the easiest way to trigger it is via REST/HTTP.

  I'm all for the WebSocket idea, as long as it is flexible and versatile. I'll do some research, perhaps I'm too tied to REST and HTTP.

• I'm not sure I'm convinced by that last part, I feel like multiple instances of FSM/Classifier are essentially multiple bots, so I don't know if hosting them in the same bot server is the best solution. Or maybe I'm misunderstanding. I thought you were referring to an "auto-training" feature, where an extension would be able to retrain the bot by sending new FSM/Classifier data, and maybe we could have versioning of the files so that original or previous data isn't lost.

  Another issue I see with the multiple instances is that with Improve classifier, and KNN with word vectors #41, we'll now be able to write/read Classifier models to/from files, and if you're using Word Vectors, these files become very large. If there are multiple instances, there would be many of these huge files.

Still thinking about this, not discarding anything because these are great ideas, but let me give it some thought.

[ryancurrah]

No problem I understand this kind of changes the model a bit.

Remark 1 response

I would prefer to stick with REST as well. The hard part about REST is requiring extension servers to implement concrete endpoints correctly. Writing service endpoints is harder to get right and easy to get wrong. So turning the model into an event driven one makes it a little easier. Where developers only need to worry about consuming events as a client and posting responses to a REST endpoint.

The AWS provider for serverless seems to support WebSocket https://www.serverless.com/framework/docs/providers/aws/events/websocket/.

Remark 2 response

Yeah this one is a little more difficult. The trick is to update the configuration/train the bot without wiping the other configurations/trainings and preventing race conditions.

Separating by extension solves that although like you mentioned will become a huge issue for Classifiers. I think it might be okay to have a unique FSM implementation per extension. I'm not 100% until we do some testing. Then have a global Classifier because the issue won't be as big of a deal there.

Saving the training seems unnecessary if we can get the extensions to re-register themselves. Maybe through a WebSocket event to tell extensions to re-register.

I can put an example PR together to flesh it out more and you can 👍 or 👎 it.

[jaimeteb]

1

You're right about that, REST can get very messy. Besides, the event-driven approach makes things a bit more modern I think. What I like about REST is that it is platform/language agnostic, but WebSockets are too, right?

2

For sure, maybe with something more tangible we would see the possibilities. Again, I'm very influenced by Rasa, and I was imagining something like Rasa X, a tool that can update the bot's model, but it doesn't overwrite the previous one, instead, it saves versions of the bot.

---

I think we could try the WebSockets first, and after that, the second feature. I'll keep thinking about it, I'm sure we're onto something cool here.

[jaimeteb]

BTW, take a look at the new classifier features, maybe they'll come into play for this.

[ryancurrah]

The new classier features look awesome. Will have to understand them a bit more. As someone who knows nothing about ML I'm not sure why I would choose to use some of these features. I will google them and see if I can also add some info as to why I would use them in the docs.

Good call about WebSockets first. And yes they are ubiquitous in all languages, even web browsers and generally pretty easy to implement.