Skip to content

Commit ffec491

Browse files
czerwiukkMiloszFilimowskiAHGIJMKLKKZNPJKQR
authored
Release 0.22.0 (#195)
## Description Describe your changes in detail --------- Co-authored-by: Miłosz Filimowski <[email protected]> Co-authored-by: Tomasz Mazur <[email protected]>
1 parent 1652b56 commit ffec491

File tree

287 files changed

+16059
-291
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

287 files changed

+16059
-291
lines changed

api/fishjam-server

api/room-manager

docs/_common/agents/definition.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
## What is an Agent?
22

33
An agent is a piece of software that allows your backend server to participate in a Fishjam room, similar to how the Fishjam client SDKs allow your client-side application to participate in a Fishjam room.
4-
They can be used to implement features such as real-time audio transcription, audio recording, real-time content moderation and more.
4+
They can be used to implement features such as real-time audio transcription, real-time content moderation, conversations with AI agents and more.
55

66
You can simply think of an agent as a peer running within your backend application.
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
:::danger Remember to disconnect your agents!
2+
It's important to disconnect agents, because **every connected agent generates usage** just as a normal peer.
3+
:::

docs/explanation/agent-internals.mdx

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ sidebar_position: 5
77
import Tabs from "@theme/Tabs";
88
import TabItem from "@theme/TabItem";
99
import AgentDefinition from "../_common/agents/definition.mdx";
10+
import RememberToDisconnect from "../_common/agents/remember-to-disconnect.mdx";
1011
import Subscriptions from "../_common/agents/subscriptions.mdx";
1112

1213
# Agent Internals
@@ -50,6 +51,8 @@ curl -XPOST -H "Authorization: Bearer $FISHJAM_MANAGEMENT_TOKEN" \
5051
### Step 2. Connecting to the Room
5152

5253
Agents connect to Fishjam by initiating a WebSocket connection on the `/socket/agent/websocket` REST API endpoint.
54+
All communication over the WebSocket between Agent the Fishjam is done using [Protobuf messages](https://protobuf.dev/).
55+
Check [Fishjam Protobuf reference](../api/reference#protobufs) for more info.
5356
Once connected, the agent must authenticate with Fishjam by sending an [`AgentRequest.AuthRequest` protobuf message](../api/reference#protobufs).
5457
An example of initiating a connection using [websocat](https://github.com/vi/websocat) and [Buf CLI](https://buf.build/docs/cli/) would look like:
5558

@@ -69,11 +72,21 @@ In a backend application, we suggest using the [Protobuf compiler](https://proto
6972

7073
Once connected, an agent will receive chunks of audio as [`AgentRequest.TrackData` protobuf message](../api/reference#protobufs).
7174

72-
### Step 4. Disconnecting from the Room
75+
### Step 4. Sending Audio
76+
77+
All further steps are done via the WebSocket connection created in [Step 2](#step-2-connecting-to-the-room).
78+
79+
To send audio to an agent, you need to create an audio track first.
80+
This is done by sending an [`AgentRequest.CreateTrack` protobuf message](../api/reference#protobufs).
81+
Once the track is created, you can send audio to it using [`AgentRequest.TrackData` protobuf message](../api/reference#protobufs).
82+
83+
### Step 5. Disconnecting from the Room
7384

7485
An agent disconnects from the room by closing the WebSocket connection created in [Step 2](#step-2-connecting-to-the-room).
7586
It may reconnect using the same token it provided in [Step 2](#step-2-connecting-to-the-room).
7687

88+
<RememberToDisconnect />
89+
7790
## Peer subscriptions
7891

7992
<Subscriptions />
@@ -90,4 +103,4 @@ Currently, only **16-bit PCM** (raw) audio output is supported.
90103
The output can have either a sample rate of **16kHz** or **24kHz**.
91104

92105
The above values aim to be compatible with popular real-time AI audio APIs.
93-
If you require a different output format, then make sure to contact us at [email protected].
106+
If you require a different output format, then make sure to contact us at `[email protected]`.

docs/explanation/architecture.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ sequenceDiagram
6060
FM->>BE: Return room ID
6161
BE->>FM: Create peer (Server SDK)
6262
FM->>BE: Return peer token
63-
BE->>Client: Send peer token + Fishjam URL
63+
BE->>Client: Send peer token + Fishjam ID
6464
```
6565

6666
### 2. Media Streaming Flow

docs/explanation/sandbox-api-concept.mdx

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -40,13 +40,13 @@ import {
4040
RoomConfigRoomTypeEnum,
4141
} from "@fishjam-cloud/js-server-sdk";
4242
import express from "express";
43-
const fishjamUrl = "";
43+
const fishjamId = "";
4444
const managementToken = "";
4545
const app = express();
4646

4747
// ---cut---
4848
// What Sandbox API does internally (simplified)
49-
const fishjamClient = new FishjamClient({ fishjamUrl, managementToken });
49+
const fishjamClient = new FishjamClient({ fishjamId, managementToken });
5050

5151
app.get("/", async (req: express.Request, res: express.Response) => {
5252
const { roomName, peerName, roomType } = req.query;
@@ -59,7 +59,7 @@ app.get("/", async (req: express.Request, res: express.Response) => {
5959
// Create or get peer
6060
const { peer, peerToken } = await fishjamClient.createPeer(room.id);
6161

62-
res.json({ peerToken, url: fishjamUrl });
62+
res.json({ peerToken });
6363
});
6464
```
6565

docs/explanation/security-tokens.mdx

Lines changed: 3 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -71,10 +71,10 @@ Peer tokens give permission to:
7171
import { FishjamClient, RoomId } from "@fishjam-cloud/js-server-sdk";
7272
import express from "express";
7373
const res = {} as any;
74-
const fishjamUrl = "";
7574
const managementToken = "";
75+
const fishjamId = "";
7676
const roomId = "" as unknown as RoomId;
77-
const fishjamClient = new FishjamClient({ fishjamUrl, managementToken });
77+
const fishjamClient = new FishjamClient({ fishjamId, managementToken });
7878

7979
// ---cut---
8080
// Backend generates peer token (using management token)
@@ -85,7 +85,6 @@ const { peer, peerToken } = await fishjamClient.createPeer(roomId, {
8585
// Backend sends peer token to client
8686
res.json({
8787
peerToken, // Client uses this to connect
88-
fishjamUrl, // Fishjam server URL
8988
roomId, // Room information
9089
});
9190
```
@@ -95,15 +94,11 @@ res.json({
9594
```typescript
9695
import { useConnection } from "@fishjam-cloud/react-client";
9796
const joinRoom = {} as any;
98-
const fishjamUrl = "";
9997
const peerToken = "";
10098

10199
// ---cut---
102100
// Client uses peer token to connect (safe to use in frontend)
103-
await joinRoom({
104-
url: fishjamUrl,
105-
peerToken: peerToken, // This is safe in client code
106-
});
101+
await joinRoom({ peerToken }); // This is safe in client code
107102
```
108103

109104
## Next Steps

docs/how-to/backend/fastapi-example.mdx

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ If you wish to see more general info, visit [**Set up your server**](../../how-t
1111
## Minimal setup
1212

1313
The same way as in the [**FastAPI docs**](https://fastapi.tiangolo.com/tutorial/first-steps/), the code below can be copied and run immediately.
14-
Just make sure you set `FISHJAM_URL` and `FISHJAM_MANAGEMENT_TOKEN` environment variables.
14+
Just make sure you set `FISHJAM_ID` and `FISHJAM_MANAGEMENT_TOKEN` environment variables.
1515
The `Depends` function allows the `FishjamClient` to be provided to the route function as a dependency, which avoids code duplication and allows easy mocking.
1616

1717
```python
@@ -24,9 +24,9 @@ from fishjam import FishjamClient
2424
app = FastAPI()
2525

2626
def fishjam_client():
27-
fishjam_url = os.environ["FISHJAM_URL"]
27+
fishjam_id = os.environ["FISHJAM_ID"]
2828
management_token = os.environ["FISHJAM_MANAGEMENT_TOKEN"]
29-
return FishjamClient(fishjam_url=fishjam_url, management_token=management_token)
29+
return FishjamClient(fishjam_id=fishjam_id, management_token=management_token)
3030

3131

3232
@app.get("/")
@@ -77,7 +77,7 @@ It doesn't interact with the FastAPI library per se, just start the event loop a
7777
import asyncio
7878
from fishjam import FishjamNotifier
7979

80-
notifier = FishjamNotifier(server_address=fishjam_url, server_api_token=management_token)
80+
notifier = FishjamNotifier(server_address=fishjam_id, server_api_token=management_token)
8181

8282
@notifier.on_server_notification
8383
def handle_notification(notification):

0 commit comments

Comments
 (0)