[Bug]: Using PreviewView doesn't work as pointed out in the example

[SharkFourSix]

Version

3.0.2

Environment that reproduces the issue

• Samsung Galaxy Note 8
• Galaxy A35

RTMP/SRT/... Server

http://192.168.1.45:8088/stream.m3u

Audio configuration

No response

Video configuration

No response

Is it reproducible in the demos application?

Yes

Reproduction steps

Streaming to remote server works fine. Video appears fine on other clients, but the streaming device's preview is just blank. SurfaceView, TextureView, AutoFitTextureView all work, albeit the video gets stretched and skewed.

Following this snippet in the documentation (repo page):

val streamer = cameraSingleStreamer(context = requireContext()) // Already instantiated streamer
val preview = findViewById<PreviewView>(R.id.preview) // Already inflated preview
/**
 * If the preview is a `PreviewView`
 */
preview.streamer = streamer
// Then start the preview
streamer.startPreview()

The app crashes. Reason: Preview surface is not set:

Expected result

The goal is to prevent the video from stretching and skewing. I wanted to use AutoFitSurfaceView, which is deprecated in favor of PreviewView. However, PreviewView either doesn't work or crashes the app.

Preview should be visible.

Actual result

Preview is not visible if calling preview.startPreview().

Calling streamer.startPreview() causes a crash with following output

Additional context

No response

Relevant logs output

[ThibaultBee]

Hi,
Thanks for reporting.
I am currently trying to improve the preview view and its stability. But you might have to wait till mid-January. It is the thing I want to work on before releasing 3.1.0.
I guess a workaround for your case would be to delay the call to start preview.
As a second woraround, you can directly use camerax viewfinder package. The previewview relies on this package.

[SharkFourSix]

Thanks for the prompt reply. May I ask where the viewfinder package is located at?

[ThibaultBee]

It is part of camerax: https://developer.android.com/jetpack/androidx/releases/camera-viewfinder

[ThibaultBee]

By the way, I plan to change the behavior of the PreviewView in 3.1.0.
There won't be any startPreview nor stopPreview API, the new PreviewView will just start preview when it is displayed and stops the preview on detach.

[SharkFourSix]

It is part of camerax: https://developer.android.com/jetpack/androidx/releases/camera-viewfinder

Alright. Thanks. I was under the impression that StreamPack already included Camera2/X libraries.

Or maybe you know a way of preventing the video previews from stretching?

[ThibaultBee]

Indeed.
The core package uses camera2. Camera2 is an Android API.
And the UI package uses CameraX viewfinder.

The best way to size the view is to use the camerax viewfinder as I already said. It is a workaround till I fix the PreviewView.

[SharkFourSix]

So I tried Viewfinder. It still stretches the videos but it's responsive to the quality of the resolution (Low, MID, HD) (I implemented the quality profiles based on the following https://developer.android.com/media/platform/supported-formats#video-encoding)

So perhaps this could actually be a device issue. I'll need to look for a non-Samsung test device because all my devices are Samsung.

[ThibaultBee]

Hi, I don't think this is a device issue. It is more likely a configuration issue. You can try on an emulator.

Could you share a code snippet?

[SharkFourSix]

Excuse the comments.

But this is what I'm testing with. The stream shows up fine and properly scaled on viewers' devices. But the streamer's preview (the code below) is what stretches.

Publiisher.kt

import android.Manifest
import android.annotation.SuppressLint
import android.content.Context
import android.media.AudioFormat
import android.media.MediaFormat
import android.util.Size
import android.view.Surface
import androidx.annotation.RequiresPermission
import io.github.thibaultbee.streampack.core.elements.sources.video.camera.extensions.defaultCameraId
import io.github.thibaultbee.streampack.core.elements.sources.video.camera.extensions.frontCameras
import io.github.thibaultbee.streampack.core.interfaces.startPreview
import io.github.thibaultbee.streampack.core.interfaces.startStream
import io.github.thibaultbee.streampack.core.interfaces.stopPreview
import io.github.thibaultbee.streampack.core.streamers.single.AudioConfig
import io.github.thibaultbee.streampack.core.streamers.single.SingleStreamer
import io.github.thibaultbee.streampack.core.streamers.single.VideoConfig
import io.github.thibaultbee.streampack.core.streamers.single.cameraSingleStreamer
import io.github.thibaultbee.streampack.ui.views.PreviewView


interface StreamSource {
    suspend fun start(
        endpoint: String,
        preview: Surface,
        profile: StreamProfile = StreamProfile.Medium
    )

    suspend fun stop()
    suspend fun prepare(): Boolean
    suspend fun release()
}

/*
                    SD (Low quality) 	SD (High quality) 	HD 720p (N/A on all devices)
Video resolution 	176 x 144 px 	    480 x 360 px 	    1280 x 720 px
Video frame rate 	12 fps 	            30 fps 	            30 fps
Video bitrate 	    56 Kbps 	        500 Kbps 	        2 Mbps
Audio codec 	    AAC-LC 	            AAC-LC 	            AAC-LC
Audio channels 	    1 (mono) 	        2 (stereo) 	        2 (stereo)
Audio bitrate 	    24 Kbps 	        128 Kbps 	        192 Kbps

*/

sealed class StreamProfile(val audioConfig: AudioConfig, val videoConfig: VideoConfig) {
    object Low : StreamProfile(
        audioConfig = AudioConfig(
            channelConfig = AudioFormat.CHANNEL_IN_MONO,
            startBitrate = 24_000,
            mimeType = MediaFormat.MIMETYPE_AUDIO_AAC
        ),
        videoConfig = VideoConfig(
            resolution = Size(176, 144),
            fps = 12,
            startBitrate = 56_000
        )
    )

    object Medium : StreamProfile(
        audioConfig = AudioConfig(
            channelConfig = AudioFormat.CHANNEL_IN_STEREO,
            startBitrate = 128_000,
            mimeType = MediaFormat.MIMETYPE_AUDIO_AAC
        ),
        videoConfig = VideoConfig(
            resolution = Size(480, 360),
            fps = 30,
            startBitrate = 500_000
        )
    )

    object High : StreamProfile(
        audioConfig = AudioConfig(
            channelConfig = AudioFormat.CHANNEL_IN_STEREO,
            startBitrate = 192_000,
            mimeType = MediaFormat.MIMETYPE_AUDIO_AAC
        ),
        videoConfig = VideoConfig(
            resolution = Size(1280, 720),
            fps = 30,
            startBitrate = 2_000_000
        )
    )
}

class StreamPublisher(val appContext: Context) : StreamSource {
    private lateinit var streamer: SingleStreamer
    private lateinit var preview: PreviewView

    private val cameraID = if (appContext.frontCameras.isNotEmpty()) {
        appContext.frontCameras[0]
    } else {
        appContext.defaultCameraId
    }

    @SuppressLint("MissingPermission")
    override suspend fun start(endpoint: String, preview: Surface, profile: StreamProfile) {
        prepare()

        streamer.setConfig(audioConfig = profile.audioConfig, videoConfig = profile.videoConfig)

        // this.preview = preview
        //preview.streamer = streamer
        streamer.startStream(endpoint)
        streamer.startPreview(preview)

    }

    override suspend fun stop() {
        streamer.stopStream()
        streamer.stopPreview()
    }

    @SuppressLint("MissingPermission")
    @RequiresPermission(Manifest.permission.CAMERA)
    override suspend fun prepare(): Boolean {
        streamer = cameraSingleStreamer(context = appContext, cameraId = cameraID).apply {
            /*val audioConfig = AudioConfig(
                startBitrate = 128000,
                sampleRate = 44100
            )

            val videoConfig = VideoConfig(
                fps = 30
            )*/

            /* val videoConfig = VideoConfig(
                 startBitrate = 2000000, // 2 Mb/s
                 resolution = Size(1280, 720),
                 fps = 30
             )*/


        }

        return true
    }

    override suspend fun release() {
        stop()
        streamer.release()
    }
}

Producer.kt

@Composable
@androidx.annotation.RequiresPermission(android.Manifest.permission.CAMERA)
fun StreamerPreviewTest(onDismissRequest: () -> Unit) {
    val context = LocalContext.current
    var source = remember { StreamPublisher(context) }
    val scope = rememberCoroutineScope()
    var preview by remember { mutableStateOf<Surface?>(null) }
    var errorMessage by remember { mutableStateOf("") }

    val profile by lazy {
        StreamProfile.Medium
    }

    DisposableEffect(Unit) {
        onDispose {
            scope.launch {
                source.release()
            }
        }
    }

    val sr = ViewfinderSurfaceRequest(
        width = profile.videoConfig.resolution.width,
        height = profile.videoConfig.resolution.height
    )

    Box(modifier = Modifier.fillMaxSize()) {
        Viewfinder(
            surfaceRequest = sr,
            modifier = Modifier.fillMaxSize(),
            contentScale = ContentScale.FillBounds // I've played with this (no dice, still stretches noticeably) 
        ) {
            onSurfaceSession {
                preview = this.surface
            }
        }
        
        /*AndroidView(factory = {
            //Viewfinder() { }
            SurfaceView(context).apply {
                preview = this

            }
            *//*
                        AutoFitSurfaceView(context).apply {
                            preview=this

                        }*//*
        })*/
    }

    LaunchedEffect(preview) {
        if (preview != null) {
            
            try {
                source.start(
                    "srt://fedora:8890?streamid=publish:test",
                    preview!!,
                    profile = profile
                )
            } catch (e: Exception) {
                errorMessage = "Error creating stream"
                println("damn. exception: ${e.message}")
            }

         
        }
    }

    Prompt(show = errorMessage.isNotEmpty(), text = errorMessage) {
        onDismissRequest()
    }
}

Prompt.kt

val NopeCancel: () -> Unit = {}

@OptIn(ExperimentalMaterial3Api::class)
@Composable
fun Prompt(
    show: Boolean,
    title: String = "",
    text: String,
    onCancel: () -> Unit = NopeCancel,
    onOk: () -> Unit
) {
    if (show) {
        BasicAlertDialog(
            onDismissRequest = { },
            properties = DialogProperties(dismissOnBackPress = false, dismissOnClickOutside = false)
        ) {
            Surface(
                modifier = Modifier
                    .wrapContentWidth()
                    .wrapContentHeight(),
                shape = MaterialTheme.shapes.large,
                tonalElevation = AlertDialogDefaults.TonalElevation
            ) {
                Column(modifier = Modifier.fillMaxWidth()) {
                    Column(modifier = Modifier.padding(16.dp)) {
                        if (title.isNotEmpty()) {
                            Text(title, fontSize = 16.sp)
                            Spacer(modifier = Modifier.height(16.dp))
                        }
                        Text(text)
                    }
                    HorizontalDivider()
                    Row(
                        modifier = Modifier
                            .fillMaxWidth()
                            .padding(horizontal = 16.dp, vertical = 4.dp),
                        horizontalArrangement = Arrangement.End
                    ) {
                        if (onCancel != NopeCancel) {
                            TextButton(onClick = {
                                onCancel()
                            }) {
                                Text("Cancel")
                            }
                        }
                        TextButton(onClick = {
                            onOk()
                        }) {
                            Text("OK")
                        }
                    }
                }
            }
        }
    }
}

[ThibaultBee]

Have you tried using setScaleType?

[ThibaultBee]

Also, not sure all device cameras support 12 fps.

[SharkFourSix]

Have you tried using setScaleType?

This is exposed through the contentScale parameter in the compose API.

[SharkFourSix]

Also, not sure all device cameras support 12 fps.

This was my attempt at trying to get the picture not to stretch and skew. So I 'm trying everything

[ThibaultBee]

The answer is probably there or in the modifier.

In the ViewfinderSurfaceRequest, have you tried passing the view size instead of the video configuration size?

Sorry, I am not familiar enough with compose to help you and this is a camerax viewfinder issue, it is not directly related to StreamPack.