Implement MSC2530: render caption for image and video if available and add white backround to images in timeline. #2570
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Implement MSC2530 (Body field as media caption) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -16,39 +16,126 @@ | |||||
|
|
||||||
| package io.element.android.features.messages.impl.timeline.components.event | ||||||
|
|
||||||
| import android.text.SpannedString | ||||||
| import androidx.compose.foundation.background | ||||||
| import androidx.compose.foundation.layout.Box | ||||||
| import androidx.compose.foundation.layout.Column | ||||||
| import androidx.compose.foundation.layout.Spacer | ||||||
| import androidx.compose.foundation.layout.fillMaxWidth | ||||||
| import androidx.compose.foundation.layout.height | ||||||
| import androidx.compose.foundation.layout.padding | ||||||
| import androidx.compose.foundation.layout.widthIn | ||||||
| import androidx.compose.foundation.shape.RoundedCornerShape | ||||||
| import androidx.compose.runtime.Composable | ||||||
| import androidx.compose.runtime.getValue | ||||||
| import androidx.compose.runtime.mutableStateOf | ||||||
| import androidx.compose.runtime.remember | ||||||
| import androidx.compose.runtime.setValue | ||||||
| import androidx.compose.ui.Alignment | ||||||
| import androidx.compose.ui.Modifier | ||||||
| import androidx.compose.ui.draw.clip | ||||||
| import androidx.compose.ui.graphics.Color | ||||||
| import androidx.compose.ui.layout.ContentScale | ||||||
| import androidx.compose.ui.platform.LocalInspectionMode | ||||||
| import androidx.compose.ui.res.stringResource | ||||||
| import androidx.compose.ui.semantics.contentDescription | ||||||
| import androidx.compose.ui.semantics.semantics | ||||||
| import androidx.compose.ui.tooling.preview.PreviewParameter | ||||||
| import androidx.compose.ui.unit.dp | ||||||
| import coil.compose.AsyncImage | ||||||
| import coil.compose.AsyncImagePainter | ||||||
| import io.element.android.features.messages.impl.timeline.aTimelineItemEvent | ||||||
| import io.element.android.features.messages.impl.timeline.components.ATimelineItemEventRow | ||||||
| import io.element.android.features.messages.impl.timeline.components.layout.ContentAvoidingLayout | ||||||
| import io.element.android.features.messages.impl.timeline.components.layout.ContentAvoidingLayoutData | ||||||
| import io.element.android.features.messages.impl.timeline.model.TimelineItemGroupPosition | ||||||
| import io.element.android.features.messages.impl.timeline.model.event.TimelineItemImageContent | ||||||
| import io.element.android.features.messages.impl.timeline.model.event.TimelineItemImageContentProvider | ||||||
| import io.element.android.features.messages.impl.timeline.model.event.aTimelineItemImageContent | ||||||
| import io.element.android.libraries.designsystem.components.blurhash.blurHashBackground | ||||||
| import io.element.android.libraries.designsystem.preview.ElementPreview | ||||||
| import io.element.android.libraries.designsystem.preview.PreviewsDayNight | ||||||
| import io.element.android.libraries.matrix.api.timeline.item.event.MessageFormat | ||||||
| import io.element.android.libraries.matrix.ui.media.MediaRequestData | ||||||
| import io.element.android.libraries.textcomposer.ElementRichTextEditorStyle | ||||||
| import io.element.android.libraries.ui.strings.CommonStrings | ||||||
| import io.element.android.wysiwyg.compose.EditorStyledText | ||||||
|
|
||||||
| @Composable | ||||||
| fun TimelineItemImageView( | ||||||
| content: TimelineItemImageContent, | ||||||
| onContentLayoutChanged: (ContentAvoidingLayoutData) -> Unit, | ||||||
| modifier: Modifier = Modifier, | ||||||
| ) { | ||||||
| val description = stringResource(CommonStrings.common_image) | ||||||
| Column( | ||||||
| modifier = modifier.semantics { contentDescription = description }, | ||||||
| ) { | ||||||
| val containerModifier = if (content.showCaption) { | ||||||
| Modifier.padding(top = 6.dp).clip(RoundedCornerShape(6.dp)) | ||||||
| } else { | ||||||
| Modifier | ||||||
| } | ||||||
| TimelineItemAspectRatioBox( | ||||||
| modifier = containerModifier.blurHashBackground(content.blurhash, alpha = 0.9f), | ||||||
| aspectRatio = content.aspectRatio, | ||||||
| ) { | ||||||
| var isLoaded by remember { mutableStateOf(false) } | ||||||
| AsyncImage( | ||||||
| modifier = Modifier | ||||||
| .fillMaxWidth() | ||||||
| .then(if (isLoaded) Modifier.background(Color.White) else Modifier), | ||||||
| model = MediaRequestData(content.preferredMediaSource, MediaRequestData.Kind.File(content.body, content.mimeType)), | ||||||
| contentScale = ContentScale.Fit, | ||||||
| alignment = Alignment.Center, | ||||||
| contentDescription = description, | ||||||
| onState = { isLoaded = it is AsyncImagePainter.State.Success }, | ||||||
| ) | ||||||
| } | ||||||
|
|
||||||
| if (content.showCaption) { | ||||||
| Spacer(modifier = Modifier.height(8.dp)) | ||||||
| val caption = if (LocalInspectionMode.current) { | ||||||
| SpannedString(content.caption) | ||||||
| } else { | ||||||
| content.formatted?.body?.takeIf { content.formatted.format == MessageFormat.HTML } ?: SpannedString(content.caption) | ||||||
| } | ||||||
| Box { | ||||||
| EditorStyledText( | ||||||
|
There was a problem hiding this comment. I think you need to provide a LocalContentColor, as per this place, this will fix the fact that color of caption is not the same for my captions and for other people captions. (see the recorded screenshot) |
||||||
| modifier = Modifier | ||||||
| .widthIn(min = MIN_HEIGHT_IN_DP.dp * content.aspectRatio!!, max = MAX_HEIGHT_IN_DP.dp * content.aspectRatio), | ||||||
| text = caption, | ||||||
| style = ElementRichTextEditorStyle.textStyle(), | ||||||
| releaseOnDetach = false, | ||||||
| onTextLayout = ContentAvoidingLayout.measureLegacyLastTextLine(onContentLayoutChanged = onContentLayoutChanged), | ||||||
| ) | ||||||
| } | ||||||
| } | ||||||
| } | ||||||
| } | ||||||
|
|
||||||
| @PreviewsDayNight | ||||||
| @Composable | ||||||
| internal fun TimelineItemImageViewPreview(@PreviewParameter(TimelineItemImageContentProvider::class) content: TimelineItemImageContent) = ElementPreview { | ||||||
| TimelineItemImageView(content, {}) | ||||||
| } | ||||||
|
|
||||||
| @PreviewsDayNight | ||||||
| @Composable | ||||||
| internal fun TimelineImageWithCaptionRowPreview() = ElementPreview { | ||||||
| Column { | ||||||
| sequenceOf(false, true).forEach { | ||||||
|
||||||
| sequenceOf(false, true).forEach { | |
| sequenceOf(false, true).forEach { isMine -> |
(do not apply suggestion, it will not compile!)
Note that this is not done everywhere, so maybe an opportunity to replace all occurences of isMine = it in the codebase :)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I know this has been reported internally, but I am pretty sure adding a white bg can cause more troubles than fixes (rendering in Dark theme for instance), but let's see.
FTR last time I checked, Element Web does not add white BG to images.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the white background the cause of these strange artifacts on the border/corners?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, but Matthew said it was a mistake. i.e. png images with no background and black letters were impossible to read in dark mode.
Sadly, I think that's the case...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, so with a white background, how a png image with no background and white letters will render?
I'd say this is the problem for only a minority of files, probably built just to make applications fail (not only Element).
Adding a background to image with transparency is like not supporting image with transparency, this is a shame. I still believe we should not add a white background. This degrades the user experience (stickers, etc.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You might be right. If the users find it annoying we can always remove this white background.
Stickers use a different view, so they won't have this problem.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we only add the background if we detect that the image is both transparent, and has a specific contrast ratio relative to the app colour scheme?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To check if an image is transparent we'd have to load it twice in memory if I'm not mistaken, which isn't really efficient, or add some transformation when Coil loads them, but it seems like it could affect animated images (GIFs, WebP...), so I don't think it's something we want to invest too much time on.