Phoenix LiveView Phoenix.Sync.LiveView.sync_stream evaluation

[Valian]

Hello 👋🏻

I'm evaluating Phoenix.Sync.LiveView.sync_stream in my Phoenix LiveView application. Particular use case I'm checking is notification stream - I want to display notifications and keep them up to date in the UI.

I'm using it in the "standard" way:

defmodule MyWeb.MyLive do
  use Phoenix.LiveView
  import Phoenix.Sync.LiveView

  def mount(_params, _session, socket) do
    {:ok, sync_stream(socket, :todos, Todos.Todo)}
  end

  def handle_info({:sync, event}, socket) do
    {:noreply, sync_stream_update(socket, event)}
  end
end

After playing with it for some time, I have these observations that surprised me.

1. dead render always has no data in stream - it's not ideal in tests and differs from the original LiveView approach.
2. initial live render stream has old (sometimes very old) data. It means there's a lot of catching up to be done on each page refresh (since I'm not using client-side caching)
3. Limit query seems to be not working, it silently ignores it.

So my questions:

1. Regular phoenix applications are being "mounted" frequently, on each page navigation. Currently it means each such mount is followed by a lot of catching up, which is visible in UI. I understand it's done that way to reduce database load, but it's not ideal. Is there an easy option to configure max snapshot age before it's refreshed?

2. Is there a way to get similar behaviour to phoenix streams in tests? I'm asking about dead render having fresh data (without cache)

3. Is there a way to respect limit query in the initial render? Phoenix.Sync.LiveView.sync_stream(.., ..,.., limit: 10) doesn't seem to work.

In general, I have a feeling that sync_stream is promising, but not yet battle-tested utility. Which is a shame.

What I was hoping for:

I would love a solution that would work in the same way as "stock" phoenix live view streams

1. initial data is always "current", present for both dead and initial live render, so there's not a lot of catching up, with limits working
2. I don't have to "babysit" these streams with lots of pubsub broadcasts and subscribes, it would just stay synchronized out of the box.

All in all, I'm rooting for this project! It's really impressive how it's all working, just wanted to drop my own observations.

[magnetised]

@Valian thanks for the warm words and the excellent feedback. You have it absolutely right -- this is exactly the current implementation and it does have the issues you have highlighted.

initial data is always "current", present for both dead and initial live render, so there's not a lot of catching up, with limits working

In 0.5.0 we added the concept of a Shape which maintains a constantly updated in-memory view of the postgres table plus allows processes to subscribe to change events.

It should be possible to use that to do exactly what you want, new instances get the current snapshot and then subscribe for updates.

This is something we should look at so that applications where the initial state is very important or frequently requested can just pull from the shape.

Is there a way to respect limit query in the initial render? Phoenix.Sync.LiveView.sync_stream(.., ..,.., limit: 10) doesn't seem to work.

We need to think about this -- there's currently no order by equivalent for electric shapes and without that limit is difficult to implement.

Having said that we could add it to the sync_live implementation once we have integrated with the Shape implementation with a user-defined order function.

We're juggling a lot of priorities here so I can't promise an immediate fix but will do our best.

Thanks again for the enthusiasm! Much appreciated.

[Valian]

Hi @magnetised, thanks for the response!

I fully understand the priorities, it's a lot of effort to maintain and develop such a big project. If I might suggest, adding an explanation of current limitations might be a huge help for users considering if this library solves their use case. If I'm correct:

• single-table stream
• no order by
• no limits
• no data on dead render
• on live render, data is not available during mount but later received via "handle_event"
• no control about snapshot invalidation (or I'm wrong on this? Gemini said it's being cached on disk and to invalidate these files would need to be removed)

Here's my discussion with Gemini about the codebase, some things might be incorrect.

[javiercr]

there's currently no order by equivalent for electric shapes and without that limit is difficult to implement.

@magnetised could you clarify what this means in practice? Is it documented anywhere?

For example, if we wanted to sync a stream of messages (like in an AI chatbox), does this limitation mean we can’t reliably guarantee the ordering of messages in the stream? And in a scenario with millions of messages, would the client always have to query the full dataset on first load?

I’m currently evaluating ElectricSQL + Phoenix Sync for a large project, and this point is a bit concerning. Any guidance or best practices would be greatly appreciated!

[thruflo]

As @magnetised says, Electric shapes don’t currently support order by or limit. Typically you use them to filter data and then sort on the client.

There’s a solution coming which will add order and limit support but it’s still in development.