[feature] app as openai-compatible server

[x-0D]

It would be very useful to have the RWKV_APP provide an "enable API server" button that is starting API server compatible with the OpenAI API specification. This would allow integration with existing tools and systems built around the OpenAI API, when you have good phone hardware and bad outdated PC.

Suggested functionality:

• chat/completions endpoint with parameters similar to OpenAI (model, messages, max_tokens, temperature, etc.)
• Authentication compatible with OpenAI API keys or a simple token system
• Response format matching OpenAI’s structure for chat completions
• Option to select different RWKV models transparently via the API parameter

Many thanks to all the developers who are working on the project. This app makes life better and has the potential to become a local mobile personal AI, fine-tuned to user needs, if it would have openai-compatible endpoint and on-device fine-tune feature. <3

[x-0D]

Do you think it's possible to have embedding service based on RWKV on phone? Is there any work done to have RWKV embeddings?

At the moment I'm using ollama(cpu) on termux as embedding service for my IDE on low-end computer. It would be great to have opanai or ollama compatible embeddings with RWKV_APP.

[HaloWang]

We already have a server on the desktop version of RWKV Chat but it's not OpenAI-compatible. We are now using this server as our offline translation chrome extension.

Do you think it's possible to have embedding service based on RWKV on phone?

I think it's possible.

Is there any work done to have RWKV embeddings?

Yes. @MollySophia has done something, running Soduku game.

I'm now thinking to build a matrix with multiple edge-deivces. Each of them running domain-specific RWKV model. And connect them by websocket or any other real time communication techs. Pretty like MoE. 🤣

[HaloWang]

I'm now working on other part of this App. I will get back to this issue in the future.

[MollySophia]

What. How's sudoku game related to embedding models :\

[HaloWang]

What. How's sudoku game related to embedding models :\

Apologies, that was a misunderstanding on my part. I've been quite busy with other tasks lately.

We will look into the embedding work in the future. I'll update this thread when we start working on it.

[x-0D]

Bump. This is still actual Feature