Design: camera API

[sargunv]

This conversation is ported over from an ephemeral Slack channel in order to preserve it. Original URL (will not live forever): https://osmus.slack.com/archives/C06U5MM097B/p1731638690170239

@westnordost

on the API, hmm... the issue (I have) with MapLibre camera animations is that it is currently not possible to animate several properties of the camera at once.

E.g. a common use case in which you would want to do that is navigation (e.g. 3rd person view of a tracked train 😜)

camera follows current GPS position continuously (i.e. with a lot of smoothing, which means that the camera position is updated more or less each frame)

there's e.g. a zoom in/zoom out button, which should animate the zoom level in or out. Or, e.g. there's a button to animate the view into a tilted navigation view and back to north-up top-down view

The properties animated are distinct (position vs zoom/tilt/rotation), yet, the current animation API at least on Android doesn't allow this. Any new animation will overwrite the last.

I do something like this in my app. So, I have an interest in being able to animate these properties separately.

One idea was to not have an animation API at all there, but rely on the Compose (Animatable) api altogether. I.e. from the MapLibre point of view, the camera position is always (re)set completely, while any animation is handled by Compose.

On how exactly then this would look even on the API side, I didn't think about that.

Maybe that not CameraState is Animatable but its properties zoom , position, rotation etc. are?

One downside of this idea is that I think MapLibre has some interesting "fly to" animation with a custom animation sequence (i.e. when there's a large distance between two points, it also zooms out, then later in again, so that it looks like one is taking a giant leap), IIRC. When camera properties are always handled independently, this would have to be re-implemented as a helper(?)

@sargunv

So the current API does have support for setting position directly, which means you can use the compose animation stuff instead of the built in fly animation, though it's not super easy atm.

I've thought about providing the helpers to use with Animateable (mainly the vector converter). Pitch, padding, and zoom are easy but am unsure how to handle angular values in the animation vector. Since you probably wouldn't want longitude 179 animating to -179 to go all the way around the world 😅 also applies to bearing, say 1 to 359

@westnordost

maybe MapLibre solves this automatically?

ah, no, nevermind, I get it, if MapLibre is not involved in animation, it must be handled by train-tracker-maplibre-wrapper

@sargunv

MapLibre does I think, if you use .flyTo with such a longitude

But if you're setting position directly (controlled by some Compose animation) it won't understand +179 should increase to wrap around to -179, so it'll animate the other way around

yeah

Could work around it by putting sin/cos of the angle in the animation vector, but then the animation is faster at certain longitudes than others

@westnordost

hmm, you can rotate things in normal compose API too, so there must be a solution to this

A function I often use in context of rotation/degrees is

/** returns a number between [startAt] - [startAt]+360 */
fun normalizeDegrees(value: Double, startAt: Double = 0.0): Double {
    var result = value % 360 // is now -360..360
    result = (result + 360) % 360 // is now 0..360
    if (result > startAt + 360) result -= 360
    return result
}

So you could normalize the diff angle (target angle - currentangle) with startAt=-180 and you get a number from -180 to 180. And then you animate from current angle to current angle + diff angle

@sargunv

so that would have to be a function on top of the compose animation libs, something like

val animatedPosiiton = Animateable(cameraPosition, converter)
animatedPosition.animateTo(calculateTargetPosition(currentPosition, targetPosition))

(or a wrapper around Animateable)

Some other problems to consider:

compose's animation vector only supports up to 4 dimensions, but a camera position has 9 (lat, lon, bearing, zoom, pitch, padding x4) so we'd need to destructure into separate animateables

setting camera position directly every frame (as we'd do when using compose to animate) interferes with the built in camera movement gestures (fling velocity would zero out every frame, and I think even before fling it gets weird)

dimensions are definitely solvable, I'm more concerned about interfering with the standard camera movement

@westnordost

wouldn't every single one (position, zoom, pitch, padding, rotation) not each be an own Animatable? (Sorry, didn't look at Animatables for a bit, so I don't know what you mean with Animation vectors)

@sargunv

Right now CameraPosition is a single object for all those dimensions (mirroring the underlying SDK), you can update one dimension at a time with cameraState.position = cameraState.position.copy(zoom = foo)

so I don't know what you mean with Animation vectors
Compose supports up to four dimensions in one Animateable, think colors with RGBA all animating together

so for example lat/lon could be a 2d animation vector

@westnordost

Hmm, interesting... I think, the normal solution in Compose would be to try to "hoist" that state, if possible. I.e. don't let the gesture change the camera position at all directly, but instead get notified via callback that e.g. the map is being moved and then (in compose code) update the camera
@ map gestures

Not sure if this is possible at all, or if it is hardwired

@sargunv

It's hardwired unfortunately. I tried to hack around it to still hoist state but it didn't work well

Maybe possible if we disable all the built in gestures and implement it all on top

@westnordost

well, the gestures are all defined in platform code. Is it possible to replace them?
(by the way, @louwers created some ticket for "Let's offer a bare bones C API". I didn't find it when I looked for it today, but (if you find it), it might be a good place to dump some wishlist on shortcomings on what is currently exposed on Android / iOS)

@louwers

There is only this PR, no issue yet. But you can document what you are running into in the KMP issue.

@westnordost

On Android, there's MapLibreMap::setGesturesManager

It looks like the only way to replace gestures with different behavior is to extend AndroidGesturesManager which is passed as first parameter to above mentioned function. Because, all the other classes involved (MapGestureDetector, *GestureListener) are all final .

The extended MyOwnAndroidGesturesManager would then need to reimplement/copy much of the behavior seen in MapGestureDetector.

That's about 1000 LOC to essentially have to copy and maintain to be in sync with upstream. 😕 (edited)

@sargunv

Is it possible to replace them?

I haven't looked deeply but from the single tap convo in the other channel I think it's possible yeah. Just a lot more work to replicate everything and make it feel right for each platform

Let's offer a bare bones C API

I want to explore someday calling into the native core directly and skipping the platform SDKs. Kotlin Native can't call C++ directly so a C api would be helpful for iOS (to avoid an objc/swift interop layer). On the Android side we'd need to use the NDK anyway so I don't think a C api would change much (unless, a kotlin native common wrapper around the core, then the iOS side and Android side both depend on that, the latter via NDK?)

That's about 1000 LOC to essentially have to copy and maintain to be in sync with upstream.

If it's required to get the best possible Compose API, then that's pretty reasonable I think. But not something I'd prioritize now if there's "good enough" solutions that avoid reimplementing all the gestures

Although I'm curious, how much of the camera stuff is in the native core vs in the platform code? I've noticed the flight animations at least seem to be calling native code rather than managing it in platform. Is fling velocity in core or platform?

And wrt "hoisting" state, I'm curious what other gesture based composables do. Thinking like, any scrolling column, or web views, or google maps. I'm only familiar with hoisting state for more discrete stuff like form fields.

I think my current approach is similar to google maps at least though I haven't looked deeply

(I wonder if the transformable compose modifier works with native Android and UIKit views, could be an elegant way to re-implement the main gestures)

@westnordost

sounds reasonable to also look at how the Compose API for Google Maps looks like. On the whole camera stuff, note that @ian Wagner mentioned that it was complex to get this to work well in compose for the maplibre-compose-playground project, so maybe he also has some input on that. (Just mentioning him here for a ping)

@sargunv

Google Maps Compose docs: https://developers.google.com/maps/documentation/android-sdk/maps-compose

val singapore = LatLng(1.35, 103.87)
val cameraPositionState = rememberCameraPositionState {
  position = CameraPosition.fromLatLngZoom(singapore, 10f)
}
GoogleMap(
  modifier = Modifier.fillMaxSize(),
  cameraPositionState = cameraPositionState
) {}

and the state exposes a var position and a suspend fun animate, with new position and duration, basically identical to what we have now

ScrollState seems similar, val value: Int and a fun to scroll instantly and a suspend fun to scroll with animation

I think, a couple TODOs:

• Try to mock up a navigation style camera experience with the current API (following a moving target with user controllable zoom gestures and buttons)
• Explore compose gesture modifiers (mainly transformable) to see if it's a viable shortcut to reimplementing gestures (mainly, do they work with AndroidView and UIKitViewController) and hoisting state
• And 3: look at what other MapLibre wrappers have done (@amir Hammad maybe has encountered this already? And the Ramani compose wrapper, React Native, Flutter, SwiftUI)

@sargunv

Explore compose gesture modifiers (mainly transformable) to see if it’s a viable shortcut to reimplementing gestures

I looked into this part just now; I was able to get a proof of concept of pan/zoom/rotate working with modifier.pointerInput but when looking into how to get the math right for zooming and rotating around the centroid, I realized that’s actually in the native core, not the platform SDKs. If we want to control the camera entirely from compose, we’d need to reimplement that stuff too. Also the native view seems to be consuming some touch events despite removing all gesture detectors; I had to make my code ignore whether touches were consumed. Probably solvable but not pursuing this route further atm

@westnordost

Back when I used Tangram-ES for display of the map, I remember that that library also didn't support parallel camera animations. (Initially, they did, but it was kind of buggy so they removed it without replacement.)

So, what I did was to implement this replacing the normal animations API:
https://github.com/streetcomplete/StreetComplete/blob/v58.2/app/src/main/java/de/w[…]ordost/streetcomplete/screens/main/map/tangram/CameraManager.kt

In a nutshell, I re-implemented all animating of the camera with Android ObjectAnimators. They animate a set of camera properties to the target value(s), always starting from the current camera properties and cancelling any prior animations whose animated properties overlap on start. (E.g. new animation of zoom+tilt cancels prior animation of zoom+rotate but doesn't cancel prior animation of position). I think when a gesture was detected, I just called cancelAllAnimations on that class.

I didn't re-do this after migration from Tangram-ES to MapLibre-Native because I hoped that either it would work out of the box or it would some day be implemented upstream (for Tangram-ES, there was no chance that it would be implemented upstream, because it had been unmaintained for a number of years)

Anyway, I guess it would be most prudent to rather implement the ability to do parallel animations upstream instead of trying to work around it, I did create a ticket for it.

@sargunv

That's great context!

I think when a gesture was detected, I just called cancelAllAnimations on that class.

I think the current solution could be updated to support that pattern without much trouble. The google maps compose state exposes some state for the cause of current movement, and if we do the same then the user could cancel their own compose animations when a gesture is detected

Anyway, I guess it would be most prudent to rather implement the ability to do parallel animations upstream instead of trying to work around it

That's fair. I think there's two separate but related issues here:

The first is animating the SDK's internal camera state by individual properties, such that animating one property doesn't automatically cancel in-progress animations on other properties. I think this is something that is best done upstream in the SDK, or if a user needs it now, done as a workaround similar to what you described with Tangram. For that, the compose SDK should support detecting when a gesture starts (like google maps does)

The second is hoisting camera state out of the SDK entirely for the most compose-y APIs. This is possible today, though it requires re-implementing both the gesture processing (currently in platform SDKs) and camera transforms based on gesture updates, like "rotate 1 degree around focal point x=123;y=456" (currently in the native core as mutating functions). I think the most upstream can/should do here is expose the camera transform math as pure function that does the calculation and returns a new camera position, and then the compose implementation can handle integrating it into platform gestures and hoisted state.

So I think my take on a reasonable plan:

• Short term, maplibre-compose mimics google maps-compose, no hoisted state, add camera move reason state so user can do your tangram workaround for independent animations
  If/when the upstream SDKs support independent camera animations, add that support to maplibre-compose
• Long term, if/when maplibre-compose integrates with the native core directly, implement all the gestures and camera state at the composable level, ideally rely on the native core for the common math around camera transforms but the standard compose APIs for gesture implementation

I gate this behind "integrate with the native core directly" because that way I don't have to work around platform SDK gestures interfering with compose gestures