Skip to content

Commit 6a22bad

Browse files
skydovesskydoves
committed
Merge branch 'develop'
2 parents 8c67f3e + f8cae48 commit 6a22bad

File tree

107 files changed

+17893
-7165
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

107 files changed

+17893
-7165
lines changed

‎README.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,13 @@ If you're interested in customizing the UI components for the Video SDK, check o
3939
<img src="https://github.com/GetStream/stream-video-android/assets/24237865/3cc08121-c8c8-4b71-8a96-0cf33b9f2c68" width="32%"/>
4040
</p>
4141

42+
## 👔 Sample Projects
43+
44+
You can find sample projects below that demonstrates use cases of Stream Video SDK for Android:
45+
46+
- [Dogfooding](https://github.com/GetStream/stream-video-android/tree/develop/dogfooding): Dogfooding demonstrates Stream Video SDK for Android with modern Android tech stacks, such as Compose, Hilt, and Coroutines.
47+
- [Meeting Room Compose](https://github.com/GetStream/meeting-room-compose): A real-time meeting room app built with Jetpack Compose to demonstrate video communications.
48+
4249
## 👩‍💻 Free for Makers 👨‍💻
4350

4451
Stream is free for most side and hobby projects. To qualify, your project/company needs to have < 5 team members and < $10k in monthly revenue. Makers get $100 in monthly credit for video for free.
@@ -116,6 +123,9 @@ Video roadmap and changelog is available [here](https://github.com/GetStream/pro
116123
### 0.4.0 milestone
117124

118125
- [X] Screensharing from mobile
126+
- [ ] Implement Chat overlay for Dogfooding
127+
- [ ] Add Dogfooding instructions + directs Google Play
128+
- [ ] Reaction dialog API for Compose
119129
- [ ] Complete Livestreaming APIs and Tutorials for hosting & watching
120130
- [ ] Android SDK development.md cleanup (Daniel)
121131
- [ ] Upgrade to more recent versions of webrtc (Kanat)

‎app/src/main/kotlin/io/getstream/video/android/app/ui/CallScreen.kt‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ import androidx.compose.runtime.Composable
2121
import androidx.compose.ui.Modifier
2222
import androidx.compose.ui.platform.LocalContext
2323
import androidx.compose.ui.tooling.preview.Preview
24+
import io.getstream.video.android.app.BuildConfig
2425
import io.getstream.video.android.compose.theme.VideoTheme
2526
import io.getstream.video.android.compose.ui.components.call.activecall.CallContent
2627
import io.getstream.video.android.core.Call
@@ -50,6 +51,7 @@ fun CallScreen(
5051
else -> Unit
5152
}
5253
},
54+
enableDiagnostics = BuildConfig.DEBUG,
5355
)
5456
}
5557

‎docusaurus/docs/Android/05-ui-cookbook/01-overview.mdx‎

Lines changed: 105 additions & 78 deletions
Original file line numberDiff line numberDiff line change
@@ -3,98 +3,125 @@ title: Overview
33
description: Overview of the UI cookbook
44
---
55

6-
Stream UI components are highly customizable and allow you to fully customize styles to your taste. This UI Cookbook will walk you through how to customize each component in your video call.
7-
8-
## UI Cookbook
9-
10-
export const CookbookCardGrid = ({ children }) => (
11-
<ol
6+
export const CookbookCard = ({ title, link, img }) => (
7+
<div
128
style={{
13-
display: "flex",
14-
flexWrap: "wrap",
15-
gap: "0.5rem",
16-
padding: 0,
9+
display: 'flex',
10+
flexDirection: 'column',
11+
justifyContent: 'start',
12+
alignItems: 'center',
13+
gap: '0.5rem',
1714
}}
1815
>
19-
{children}
20-
</ol>
16+
<h4 style={{ margin: '0px' }}>{title}</h4>
17+
<a href={link}>
18+
<p style={{ margin: '0', padding: '0' }}>
19+
<img src={img} />
20+
</p>
21+
</a>
22+
</div>
2123
);
2224

23-
export const CookbookCard = ({ title, children }) => (
24-
<li
25-
style={{
26-
display: "flex",
27-
flexDirection: "column",
28-
margin: "0 0.5rem",
29-
listStyleType: "none",
30-
color: "#d3d3d3",
31-
width: "250px",
32-
height: "200px",
33-
}}
34-
>
35-
<h4>{title}</h4>
36-
{children}
37-
</li>
38-
);
25+
import CallControls from '../assets/cookbook/replacing-call-controls.png';
26+
import Label from '../assets/cookbook/removing-label-and-indicators.png';
27+
import VideoLayout from '../assets/cookbook/custom-video-layout.png';
28+
import IncomingCall from '../assets/cookbook/incoming-call.png';
29+
import LobbyPreview from '../assets/cookbook/lobby-preview.png';
30+
import VideoFallback from '../assets/cookbook/no-video-fallback-avatar.png';
31+
32+
import PermissionRequests from '../assets/cookbook/permission-requests.png';
33+
import VolumeIndicator from '../assets/cookbook/audio-volume-indicator.png';
34+
import Reactions from '../assets/cookbook/reactions.png';
35+
import WatchingLivestream from '../assets/cookbook/livestream-live-label.png';
3936

40-
This cookbook aims to show you how to build your own UI elements for video calling.
37+
import ConnectionQuality from '../assets/cookbook/network-quality.png';
38+
import SpeakingWhileMuted from '../assets/cookbook/speaking-while-muted.png';
39+
import ConnectionWarning from '../assets/cookbook/connection-unstable.png';
40+
41+
Stream UI components are highly customizable and allow you to fully customize styles to your taste. This UI Cookbook will walk you through how to customize each component in your video call.
4142

4243
### Video Calls & Ringing
4344

44-
<div>
45-
<CookbookCardGrid>
46-
<CookbookCard title="Replacing Call Controls">
47-
<img src={require("../assets/cookbook/replacing-call-controls.png").default} />
48-
</CookbookCard>
49-
<CookbookCard title="Custom label &amp; indicators">
50-
<img src={require("../assets/cookbook/removing-label-and-indicators.png").default} />
51-
</CookbookCard>
52-
<CookbookCard title="Custom Video Layout">
53-
<img src={require("../assets/cookbook/custom-video-layout.png").default} />
54-
</CookbookCard>
55-
<CookbookCard title="Incoming Call">
56-
<img src={require("../assets/cookbook/incoming-call.png").default} />
57-
</CookbookCard>
58-
<CookbookCard title="Lobby/Preview">
59-
<img src={require("../assets/cookbook/lobby-preview.png").default} />
60-
</CookbookCard>
61-
<CookbookCard title="No-video fallback">
62-
<img src={require("../assets/cookbook/no-video-fallback-avatar.png").default} />
63-
</CookbookCard>
64-
</CookbookCardGrid>
45+
<div
46+
style={{ display: 'grid', gridTemplateColumns: '1fr 1fr 1fr', gap: '1.2rem' }}
47+
>
48+
<CookbookCard
49+
title="Replacing Control Actions"
50+
link="../control-actions"
51+
img={CallControls}
52+
></CookbookCard>
53+
<CookbookCard
54+
title="Custom label &amp; indicators"
55+
link="../participant-label"
56+
img={Label}
57+
></CookbookCard>
58+
<CookbookCard
59+
title="Video Renderer"
60+
link="../video-renderer"
61+
img={VideoLayout}
62+
></CookbookCard>
63+
<CookbookCard
64+
title="Incoming Call &amp; Outging Call"
65+
link="../incoming-and-outgoing-call"
66+
img={VideoLayout}
67+
></CookbookCard>
68+
<CookbookCard
69+
title="Call Lobby"
70+
link="../lobby-preview"
71+
img={LobbyPreview}
72+
></CookbookCard>
73+
<CookbookCard
74+
title="Video Fallback"
75+
link="../video-fallback"
76+
img={VideoFallback}
77+
></CookbookCard>
6578
</div>
6679

6780
### Audio rooms & Livestreams
6881

69-
<div>
70-
<CookbookCardGrid>
71-
<CookbookCard title="Permission Requests">
72-
<img src={require("../assets/cookbook/permission-requests.png").default} />
73-
</CookbookCard>
74-
<CookbookCard title="Audio volume Indicator">
75-
<img src={require("../assets/cookbook/audio-volume-indicator.png").default} />
76-
</CookbookCard>
77-
<CookbookCard title="Reactions">
78-
<img src={require("../assets/cookbook/reactions.png").default} />
79-
</CookbookCard>
80-
<CookbookCard title="Watching a Livestream">
81-
<img src={require("../assets/cookbook/reactions.png").default} />
82-
</CookbookCard>
83-
</CookbookCardGrid>
82+
<div
83+
style={{ display: 'grid', gridTemplateColumns: '1fr 1fr 1fr', gap: '1.2rem' }}
84+
>
85+
<CookbookCard
86+
title="Permission Requests"
87+
link="../permission-requests"
88+
img={PermissionRequests}
89+
></CookbookCard>
90+
<CookbookCard
91+
title="Audio volume Indicator"
92+
link="../audio-volume-indicator"
93+
img={VolumeIndicator}
94+
></CookbookCard>
95+
<CookbookCard
96+
title="Reactions"
97+
link="../reactions"
98+
img={Reactions}
99+
></CookbookCard>
100+
<CookbookCard
101+
title="Watching a Livestream"
102+
link="../watching-livestream"
103+
img={WatchingLivestream}
104+
></CookbookCard>
84105
</div>
85106

86107
### Small Components
87108

88-
<div>
89-
<CookbookCardGrid>
90-
<CookbookCard title="Network Quality Indicator">
91-
<img src={require("../assets/cookbook/network-quality.png").default} />
92-
</CookbookCard>
93-
<CookbookCard title="Speaking While Muted">
94-
<img src={require("../assets/cookbook/speaking-while-muted.png").default} />
95-
</CookbookCard>
96-
<CookbookCard title="Connection Unstable">
97-
<img src={require("../assets/cookbook/connection-unstable.png").default} />
98-
</CookbookCard>
99-
</CookbookCardGrid>
100-
</div>
109+
<div
110+
style={{ display: 'grid', gridTemplateColumns: '1fr 1fr 1fr', gap: '1.2rem' }}
111+
>
112+
<CookbookCard
113+
title="Network Quality Indicator"
114+
link="../network-quality-indicator"
115+
img={PermissionRequests}
116+
></CookbookCard>
117+
<CookbookCard
118+
title="Speaking While Muted"
119+
link="../speaking-while-muted"
120+
img={VolumeIndicator}
121+
></CookbookCard>
122+
<CookbookCard
123+
title="Connection Unstable"
124+
link="../network-quality-indicator"
125+
img={Reactions}
126+
></CookbookCard>
127+
</div>

‎docusaurus/docs/Android/06-advanced/05-apply-video-filters.mdx‎

Lines changed: 127 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,132 @@ title: Video & Audio filters
33
description: How to build video or audio filters
44
---
55

6-
## Apply Custom Video Filters
6+
## Video Filters
77

8-
// TODO - cover how to apply custom filters, where they live/exist and some common examples
8+
Some calling apps allow filters to be applied to the current user's video, such as blurring the background, adding AR elements (glasses, moustaches, etc) or applying image filters (such as sepia, bloom etc). StreamVideo's Android SDK has support for injecting your custom filter into the calling experience.
99

10-
// sepia
11-
// grayscale
10+
How does this work? You can inject a filter through `Call.videoFilter`, you will receive each frame of the user's local video as `Bitmap`, allowing you to apply the filters (by mutating the `Bitmap`). This way you have complete freedom over the processing pipeline.
11+
12+
## Adding a Video Filter
13+
14+
Create a `BitmapVideoFilter` or `RawVideoFilter` instance in your project. Here is how the abstract classes look like:
15+
16+
```kotlin
17+
abstract class BitmapVideoFilter : VideoFilter() {
18+
fun filter(bitmap: Bitmap)
19+
}
20+
21+
abstract class RawVideoFilter : VideoFilter() {
22+
abstract fun filter(videoFrame: VideoFrame, surfaceTextureHelper: SurfaceTextureHelper): VideoFrame
23+
}
24+
```
25+
The `BitmapVideoFilter` is a simpler filter that gives you a `Bitmap` which you can then manipulate directly. But it's less performant than using the `RawVideoFilter` which gives you direct access to `VideoFrame` from WebRTC and there is no overhead compared to `BitmapVideoFilter` (like YUV <-> ARGB conversions).
26+
27+
And then set the video filter into `Call.videoFilter`.
28+
29+
We can create and set a simple black and white filter like this:
30+
```kotlin
31+
call.videoFilter = object: BitmapVideoFilter() {
32+
override fun filter(bitmap: Bitmap) {
33+
val c = Canvas(bitmap)
34+
val paint = Paint()
35+
val cm = ColorMatrix()
36+
cm.setSaturation(0f)
37+
val f = ColorMatrixColorFilter(cm)
38+
paint.colorFilter = f
39+
c.drawBitmap(bitmap, 0f, 0f, paint)
40+
}
41+
}
42+
```
43+
44+
:::note
45+
You need to manipulate the original bitmap instance to apply the filters. You can of course create a new bitmap in the process, but you need to then draw it on the `bitmap` instance you get in the `filter` callback
46+
:::
47+
48+
## Adding AI Filters
49+
50+
In some cases, you might also want to apply AI filters. That can be an addition to the user's face (glasses, moustaches, etc), or an ML filter. In this section this use-case will be covered. Specifically, you will use the [Selfie Segmentation](https://developers.google.com/ml-kit/vision/selfie-segmentation/android) from Google's ML kit to change the background behind you.
51+
52+
First include the necessary dependency (check for latest version [here](https://developers.google.com/ml-kit/vision/selfie-segmentation/android#before_you_begin)
53+
54+
```gradle
55+
dependencies {
56+
implementation 'com.google.mlkit:segmentation-selfie:16.0.0-beta4'
57+
}
58+
```
59+
60+
Create a class that will hold your custom filter. The initialisation of the `Segmentation` class is done according to the [official docs](https://developers.google.com/ml-kit/vision/selfie-segmentation/android#enable_raw_size_mask).
61+
62+
```kotlin
63+
class SelfieSegmentation {
64+
65+
private val options =
66+
SelfieSegmenterOptions.Builder()
67+
.setDetectorMode(SelfieSegmenterOptions.STREAM_MODE)
68+
.enableRawSizeMask()
69+
.build()
70+
private val segmenter = Segmentation.getClient(options)
71+
72+
fun applyFilter(bitmap: Bitmap) {
73+
// Send the bitmap into ML Kit for processing
74+
val mlImage = InputImage.fromBitmap(bitmap, 0)
75+
val task = segmenter.process(mlImage)
76+
// Wait for result synchronously on same thread
77+
val mask = Tasks.await(task)
78+
79+
val isRawSizeMaskEnabled = mask.width != bitmap.width || mask.height != bitmap.height
80+
val scaleX = bitmap.width * 1f / mask.width
81+
val scaleY = bitmap.height * 1f / mask.height
82+
83+
// Create a bitmap mask to cover the background
84+
val maskBitmap = Bitmap.createBitmap(
85+
maskColorsFromByteBuffer(mask), mask.width, mask.height, Bitmap.Config.ARGB_8888
86+
)
87+
// Create a canvas from the frame bitmap
88+
val canvas = Canvas(bitmap)
89+
val matrix = Matrix()
90+
if (isRawSizeMaskEnabled) {
91+
matrix.preScale(scaleX, scaleY)
92+
}
93+
// And now draw the bitmap mask onto the original bitmap
94+
canvas.drawBitmap(maskBitmap, matrix, null)
95+
96+
maskBitmap.recycle()
97+
}
98+
99+
private fun maskColorsFromByteBuffer(mask: SegmentationMask): IntArray {
100+
val colors = IntArray(mask.width * mask.height)
101+
for (i in 0 until mask.width * mask.height) {
102+
val backgroundLikelihood = 1 - mask.buffer.float
103+
if (backgroundLikelihood > 0.9) {
104+
colors[i] = Color.argb(128, 0, 0, 255)
105+
} else if (backgroundLikelihood > 0.2) {
106+
val alpha = (182.9 * backgroundLikelihood - 36.6 + 0.5).toInt()
107+
colors[i] = Color.argb(alpha, 0, 0, 255)
108+
}
109+
}
110+
return colors
111+
}
112+
}
113+
```
114+
115+
And now set the custom filter into our SDK:
116+
117+
```kotlin
118+
call.videoFilter = object: BitmapVideoFilter() {
119+
120+
val selfieFilter = SelfieSegmentation()
121+
122+
override fun filter(bitmap: Bitmap) {
123+
selfieFilter.applyFilter(bitmap)
124+
}
125+
}
126+
```
127+
128+
The result:
129+
130+
![Stream Filter](../assets/screenshot_video_filter.png)
131+
132+
## Audio Filters
133+
134+
TODO

‎docusaurus/docs/Android/assets/screenshot_video_filter.png‎

2.53 MB
Loading

0 commit comments

Comments
 (0)