Add support for optimistic writes (#14)

Specifically supporting https://github.com/TanStack/optimistic

---------

Co-authored-by: James Arthur <[email protected]>

Author: magnetised

CHANGELOG.md

@@ -5,7 +5,7 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## 0.3.4
+## [0.3.4] - 2025-03-25
 
 ### Changed
 

README.md

@@ -1,15 +1,15 @@
 # Phoenix.Sync
 
+Real-time sync for Postgres-backed [Phoenix](https://www.phoenixframework.org/) applications.
+
 <p>
-  <br />
   <a href="https://hexdocs.pm/phoenix_sync" target="_blank">
     <picture>
       <img alt="Phoenix sync illustration"
           src="https://github.com/electric-sql/phoenix_sync/raw/main/docs/phoenix-sync.png"
       />
     </picture>
   </a>
-  <br />
 </p>
 
 [![Hex.pm](https://img.shields.io/hexpm/v/phoenix_sync.svg)](https://hex.pm/packages/phoenix_sync)
@@ -18,29 +18,30 @@
 [![Status](https://img.shields.io/badge/status-beta-orange)](https://github.com/electric-sql/phoenix_sync)
 [![Discord](https://img.shields.io/discord/933657521581858818?color=5969EA&label=discord)](https://discord.electric-sql.com)
 
-Sync is the best way of building modern apps. Phoenix.Sync enables real-time sync for Postgres-backed [Phoenix](https://www.phoenixframework.org/) applications.
-
 Documentation is available at [hexdocs.pm/phoenix_sync](https://hexdocs.pm/phoenix_sync).
 
-## Build real-time apps on locally synced data
+## Build real-time apps on sync
+
+Phoenix.Sync is a library that adds real-time sync to Postgres-backed [Phoenix](https://www.phoenixframework.org/) applications. Use it to sync data into both LiveView and front-end web and mobile applications.
 
-- sync data into Elixir, `LiveView` and frontend web and mobile applications
 - integrates with `Plug` and `Phoenix.{Controller, LiveView, Router, Stream}`
-- uses [ElectricSQL](https://electric-sql.com) for scalable data delivery and fan out
-- maps `Ecto` queries to [Shapes](https://electric-sql.com/docs/guides/shapes) for partial replication
+- uses [ElectricSQL](https://electric-sql.com) for core sync, fan-out and data delivery
+- maps `Ecto.Query`s to [Shapes](https://electric-sql.com/docs/guides/shapes) for partial replication
 
-## Usage
+There are four key APIs for [read-path sync](#read-path-sync) out of Postgres:
 
-There are four key APIs:
+- `Phoenix.Sync.Client.stream/2` for low level usage in Elixir
+- `Phoenix.Sync.LiveView.sync_stream/4` to sync into a LiveView
+- `Phoenix.Sync.Router.sync/2` macro to expose a shape in your Router
+- `Phoenix.Sync.Controller.sync_render/3` to return shapes from a Controller
 
-- [`Phoenix.Sync.Client.stream/2`](https://hexdocs.pm/phoenix_sync/Phoenix.Sync.Client.html#stream/2) for low level usage in Elixir
-- [`Phoenix.Sync.LiveView.sync_stream/4`](https://hexdocs.pm/phoenix_sync/Phoenix.Sync.LiveView.html#sync_stream/4) to sync into a LiveView stream
-- [`Phoenix.Sync.Router.sync/2`](https://hexdocs.pm/phoenix_sync/Phoenix.Sync.Router.html#sync/2) macro to expose a statically defined shape in your Router
-- [`Phoenix.Sync.Controller.sync_render/3`](https://hexdocs.pm/phoenix_sync/Phoenix.Sync.Controller.html#sync_render/3) to expose dynamically constructed shapes from a Controller
+And a `Phoenix.Sync.Writer` module for handling [write-path sync](#write-path-sync) back into Postgres.
+
+## Read-path sync
 
 ### Low level usage in Elixir
 
-Use [`Phoenix.Sync.Client.stream/2`](https://hexdocs.pm/phoenix_sync/Phoenix.Sync.Client.html#stream/2) to convert an `Ecto.Query` into an Elixir `Stream`:
+Use `Phoenix.Sync.Client.stream/2` to convert an `Ecto.Query` into an Elixir `Stream`:
 
 ```elixir
 stream = Phoenix.Sync.Client.stream(Todos.Todo)
@@ -52,7 +53,7 @@ stream =
 
 ### Sync into a LiveView stream
 
-Swap out `Phoenix.LiveView.stream/3` for [`Phoenix.Sync.LiveView.sync_stream/4`](https://hexdocs.pm/phoenix_sync/Phoenix.Sync.LiveView.html#sync_stream/4) to automatically keep a LiveView up-to-date with the state of your Postgres database:
+Swap out `Phoenix.LiveView.stream/3` for `Phoenix.Sync.LiveView.sync_stream/4` to automatically keep a LiveView up-to-date with the state of your Postgres database:
 
 ```elixir
 defmodule MyWeb.MyLive do
@@ -75,7 +76,7 @@ This means you can build fully end-to-end real-time multi-user applications with
 
 ### Sync shapes through your Router
 
-Use the [`Phoenix.Sync.Router.sync/2`](https://hexdocs.pm/phoenix_sync/Phoenix.Sync.Router.html#sync/2) macro to expose statically (compile-time) defined shapes in your Router:
+Use the `Phoenix.Sync.Router.sync/2` macro to expose statically (compile-time) defined shapes in your Router:
 
 ```elixir
 defmodule MyWeb.Router do
@@ -98,7 +99,7 @@ Because the shapes are exposed through your Router, the client connects through
 
 ### Sync dynamic shapes from a Controller
 
-Sync shapes from any standard Controller using the [`Phoenix.Sync.Controller.sync_render/3`](https://hexdocs.pm/phoenix_sync/Phoenix.Sync.Controller.html#sync_render/3) view function:
+Sync shapes from any standard Controller using the `Phoenix.Sync.Controller.sync_render/3` view function:
 
 ```elixir
 defmodule Phoenix.Sync.LiveViewTest.TodoController do
@@ -122,7 +123,7 @@ This allows you to define and personalise the shape definition at runtime using
 
 You can sync _into_ any client in any language that [speaks HTTP and JSON](https://electric-sql.com/docs/api/http).
 
-For example, using the Electric [Typescript client](https://electric-sql.com/docs/api/clients/typescript):
+For example, using the Electric [TypeScript client](https://electric-sql.com/docs/api/clients/typescript):
 
 ```typescript
 import { Shape, ShapeStream } from "@electric-sql/client";
@@ -152,6 +153,51 @@ const MyComponent = () => {
 
 See the Electric [demos](https://electric-sql.com/demos) and [documentation](https://electric-sql.com/docs) for more client-side usage examples.
 
+## Write-path sync
+
+The `Phoenix.Sync.Writer` module allows you to ingest batches of writes from the client.
+
+The idea is that the front-end can batch up [local optimistic writes](https://electric-sql.com/docs/guides/writes). For example using a library like [@TanStack/optimistic](https://github.com/TanStack/optimistic) or by [monitoring changes to a local embedded database](https://electric-sql.com/docs/guides/writes#through-the-db).
+
+These changes can be POSTed to a `Phoenix.Controller`, which then constructs a `Phoenix.Sync.Writer` instance. The writer instance authorizes and validates the writes before applying them to the database. Under the hood this uses `Ecto.Multi`, to ensure that transactions (batches of writes) are applied atomically.
+
+For example, the controller below handles local writes made to a project management app. It constructs a writer instance and pipes it through a series of `Phoenix.Sync.Writer.allow/3` calls. These register functions against `Ecto.Schema`s (in this case `Projects.Project` and `Projects.Issue`):
+
+```elixir
+defmodule MutationController do
+  use Phoenix.Controller, formats: [:json]
+
+  alias Phoenix.Sync.Writer
+  alias Phoenix.Sync.Writer.Format
+
+  def mutate(conn, %{"transaction" => transaction} = _params) do
+    user_id = conn.assigns.user_id
+
+    {:ok, txid, _changes} =
+      Phoenix.Sync.Writer.new()
+      |> Phoenix.Sync.Writer.allow(
+        Projects.Project,
+        check: reject_invalid_params/2,
+        load: &Projects.load_for_user(&1, user_id),
+        validate: &Projects.Project.changeset/2
+      )
+      |> Phoenix.Sync.Writer.allow(
+        Projects.Issue,
+        # Use the sensible defaults:
+        # validate: Projects.Issue.changeset/2
+        # etc.
+      )
+      |> Phoenix.Sync.Writer.apply(transaction, Repo, format: Format.TanstackOptimistic)
+
+    render(conn, :mutations, txid: txid)
+  end
+end
+```
+
+This facilitates incrementally adding bi-directional sync support to a Phoenix application, re-using your existing auth and schema/validation logic.
+
+See the `Phoenix.Sync.Writer` module docs for more information.
+
 ## Installation and configuration
 
 `Phoenix.Sync` can be used in two modes:
@@ -356,7 +402,7 @@ You can also include `replica` (see below) in your static shape definitions:
 sync "/incomplete-todos", Todos.Todo, where: "completed = false", replica: :full
 ```
 
-For anything else more dyanamic, or to use Ecto queries, you should switch from using the `sync` macros in your router to using `sync_render/3` in a controller.
+For anything else more dynamic, or to use Ecto queries, you should switch from using the `sync` macros in your router to using `sync_render/3` in a controller.
 
 ### Using a keyword list
 
@@ -372,4 +418,3 @@ The available options are:
 - `replica` (optional). By default Electric will only send primary keys + changed columns on updates. Set `replica: :full` to receive the full row, not just the changed columns.
 
 See the [Electric Shapes guide](https://electric-sql.com/docs/guides/shapes) for more information.
-

config/test.exs

@@ -17,7 +17,8 @@ config :phoenix_sync, Support.Repo,
   port: 54321,
   stacktrace: true,
   show_sensitive_data_on_connection_error: true,
-  pool_size: 10
+  pool_size: 10,
+  pool: Ecto.Adapters.SQL.Sandbox
 
 config :phoenix_sync, Support.ConfigTestRepo,
   username: "postgres",

lib/phoenix/sync.ex

@@ -1,4 +1,10 @@
 defmodule Phoenix.Sync do
+  @moduledoc """
+  Real-time sync for Postgres-backed Phoenix applications.
+
+  See the [docs](../../README.md) for more information.
+  """
+
   alias Electric.Client.ShapeDefinition
 
   @shape_keys [:namespace, :where, :columns]
@@ -18,7 +24,54 @@ defmodule Phoenix.Sync do
           | {:columns, String.t()}
   @type param_overrides :: [param_override()]
 
+  @doc """
+  Returns the required adapter configuration for your Phoenix Endpoint or
+  `Plug.Router`.
+
+  ## Phoenix
+
+  Configure your endpoint with the configuration at runtime by passing the
+  `phoenix_sync` configuration to your endpoint in the `Application.start/2`
+  callback:
+
+      def start(_type, _args) do
+        children = [
+          # ...
+          {MyAppWeb.Endpoint, phoenix_sync: Phoenix.Sync.plug_opts()}
+        ]
+      end
+
+  ## Plug
+
+  Add the configuration to the Plug opts in your server configuration:
+
+      children = [
+        {Bandit, plug: {MyApp.Router, phoenix_sync: Phoenix.Sync.plug_opts()}}
+      ]
+
+  Your `Plug.Router` must be configured with
+  [`copy_opts_to_assign`](https://hexdocs.pm/plug/Plug.Builder.html#module-options) and you should `use` the rele
+
+      defmodule MyApp.Router do
+        use Plug.Router, copy_opts_to_assign: :options
+
+        use Phoenix.Sync.Controller
+        use Phoenix.Sync.Router
+
+        plug :match
+        plug :dispatch
+
+        sync "/shapes/todos", Todos.Todo
+
+        get "/shapes/user-todos" do
+          %{"user_id" => user_id} = conn.params
+          sync_render(conn, from(t in Todos.Todo, where: t.owner_id == ^user_id)
+        end
+      end
+  """
   defdelegate plug_opts(), to: Phoenix.Sync.Application
+
+  @doc false
   defdelegate plug_opts(config), to: Phoenix.Sync.Application
 
   defdelegate client!(), to: Phoenix.Sync.Client, as: :new!

lib/phoenix/sync/application.ex

@@ -1,4 +1,6 @@
 defmodule Phoenix.Sync.Application do
+  @moduledoc false
+
   use Application
 
   require Logger
@@ -52,51 +54,6 @@ defmodule Phoenix.Sync.Application do
     }
   end
 
-  @doc """
-  Returns the required adapter configuration for your Phoenix Endpoint or
-  `Plug.Router`.
-
-  ## Phoenix
-
-  Configure your endpoint with the configuration at runtime by passing the
-  `phoenix_sync` configuration to your endpoint in the `Application.start/2`
-  callback:
-
-      def start(_type, _args) do
-        children = [
-          # ...
-          {MyAppWeb.Endpoint, phoenix_sync: Phoenix.Sync.plug_opts()}
-        ]
-      end
-
-  ## Plug
-
-  Add the configuration to the Plug opts in your server configuration:
-
-      children = [
-        {Bandit, plug: {MyApp.Router, phoenix_sync: Phoenix.Sync.plug_opts()}}
-      ]
-
-  Your `Plug.Router` must be configured with
-  [`copy_opts_to_assign`](https://hexdocs.pm/plug/Plug.Builder.html#module-options) and you should `use` the rele
-
-      defmodule MyApp.Router do
-        use Plug.Router, copy_opts_to_assign: :options
-
-        use Phoenix.Sync.Controller
-        use Phoenix.Sync.Router
-
-        plug :match
-        plug :dispatch
-
-        sync "/shapes/todos", Todos.Todo
-
-        get "/shapes/user-todos" do
-          %{"user_id" => user_id} = conn.params
-          sync_render(conn, from(t in Todos.Todo, where: t.owner_id == ^user_id)
-        end
-      end
-  """
   @spec plug_opts() :: keyword()
   def plug_opts do
     config() |> plug_opts()

lib/phoenix/sync/client.ex

@@ -1,4 +1,16 @@
 defmodule Phoenix.Sync.Client do
+  @moduledoc """
+  Low level Elixir client. Converts an `Ecto.Query` into an Elixir `Stream`:
+
+  ```elixir
+  stream = Phoenix.Sync.Client.stream(Todos.Todo)
+
+  stream =
+    Ecto.Query.from(t in Todos.Todo, where: t.completed == false)
+    |> Phoenix.Sync.Client.stream()
+  ```
+  """
+
   alias Phoenix.Sync.PredefinedShape
 
   @doc """

lib/phoenix/sync/live_view.ex

@@ -1,4 +1,24 @@
 defmodule Phoenix.Sync.LiveView do
+  @moduledoc """
+  Swap out `Phoenix.LiveView.stream/3` for `Phoenix.Sync.LiveView.sync_stream/4` to
+  automatically keep a LiveView up-to-date with the state of your Postgres database:
+
+  ```elixir
+  defmodule MyWeb.MyLive do
+    use Phoenix.LiveView
+    import Phoenix.Sync.LiveView
+
+    def mount(_params, _session, socket) do
+      {:ok, sync_stream(socket, :todos, Todos.Todo)}
+    end
+
+    def handle_info({:sync, event}, socket) do
+      {:noreply, sync_stream_update(socket, event)}
+    end
+  end
+  ```
+  """
+
   use Phoenix.Component
 
   alias Electric.Client.Message
@@ -63,9 +83,7 @@ defmodule Phoenix.Sync.LiveView do
         {:noreply, Phoenix.Sync.LiveView.sync_stream_update(socket, event)}
       end
 
-  See the docs for
-  [`Phoenix.LiveView.stream/4`](https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.html#stream/4)
-  for details on using LiveView streams.
+  See the docs for `Phoenix.LiveView.stream/4` for details on using LiveView streams.
 
   ## Lifecycle Events
 

lib/phoenix/sync/plug.ex

@@ -42,10 +42,8 @@ defmodule Phoenix.Sync.Plug do
   ```
 
   You can add additional authentication/authorization for shapes using
-  [Phoenix's
-  pipelines](https://hexdocs.pm/phoenix/Phoenix.Router.html#pipeline/2) or
-  other [`plug`
-  calls](https://hexdocs.pm/phoenix/Phoenix.Router.html#plug/2).
+  [Phoenix's pipelines](https://hexdocs.pm/phoenix/Phoenix.Router.html#pipeline/2)
+  or other [`plug` calls](https://hexdocs.pm/phoenix/Phoenix.Router.html#plug/2).
 
   ## Plug.Router