Skip to content

Commit 3e59a72

Browse files
author
Hein
committed
Add client-id mapping tutorial
1 parent 994afb5 commit 3e59a72

File tree

4 files changed

+351
-3
lines changed

4 files changed

+351
-3
lines changed

‎mint.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -399,7 +399,8 @@
399399
"group": "Data Management",
400400
"pages": [
401401
"tutorials/client/data/overview",
402-
"tutorials/client/data/cascading-delete"
402+
"tutorials/client/data/cascading-delete",
403+
"tutorials/client/data/map-local-uuid"
403404
]
404405
}
405406
]

‎tutorials/client/data/map-local-uuid.mdx‎

Lines changed: 344 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,344 @@
1+
---
2+
title: Auto-Incrementing ID Mapping
3+
description: In this tutorial we will show you how to map the local uuid to a remote auto increment id.
4+
sidebarTitle: Auto-Incrementing ID Mapping
5+
keywords: ["data", "uuid", "map", "auto increment", "id", "sequential id"]
6+
---
7+
8+
# Introduction
9+
When auto-incrementing / sequential IDs are used on the backend database, the ID can only be generated on the backend database, and not on the client while offline.
10+
To handle this, you can use a secondary UUID on the client, then map them to a sequential ID when performing an update on the backend database.
11+
This allows using a sequential primary key for each record, with a UUID as a secondary ID.
12+
13+
<Note>
14+
This mapping must be performed wherever the UUIDs are referenced, including for every foreign key column.
15+
</Note>
16+
17+
To illustrate this, we will use the [To-Do List demo app](https://github.com/powersync-ja/powersync-js/tree/main/demos/react-supabase-todolist) and modify it to use UUIDs
18+
on the client and map them to sequential IDs on the backend database (Supabase in this case).
19+
20+
### Overview
21+
Before we get started, let's outline the changes we will have to make:
22+
<Steps>
23+
<Step title={"Data model"}>
24+
Update the `lists` and `todos` tables
25+
</Step>
26+
27+
<Step title={"Create SQL triggers"}>
28+
Add two triggers that will map the UUID to the integer ID and vice versa.
29+
</Step>
30+
31+
<Step title={"Update Sync Rules"}>
32+
Update the Sync Rules to use the new integer ID instead of the UUID column.
33+
</Step>
34+
35+
<Step title={"Update client to use UUIDs."}>
36+
The following components/files will have to be updated:
37+
- *Files*:
38+
- `AppSchema.ts`
39+
- `fts_setup.ts`
40+
- `SupabaseConnector.ts`
41+
- *Components*:
42+
- `lists.tsx`
43+
- `page.tsx`
44+
- `SearchBarWidget.tsx`
45+
- `TodoListsWidget.tsx`
46+
</Step>
47+
</Steps>
48+
49+
# Data Model
50+
51+
In order to map the UUID to the integer ID, we need to update the
52+
- `lists` table by adding a `uuid` column, which will be the secondary ID, and
53+
- `todos` table by adding a `uuid` column, and a `list_uuid` foreign key column which references the `uuid` column in the `lists` table.
54+
55+
<CodeGroup>
56+
```sql data model {3, 13, 21, 26}
57+
create table public.lists (
58+
id serial,
59+
uuid uuid not null unique,
60+
created_at timestamp with time zone not null default now(),
61+
name text not null,
62+
owner_id uuid not null,
63+
constraint lists_pkey primary key (id),
64+
constraint lists_owner_id_fkey foreign key (owner_id) references auth.users (id) on delete cascade
65+
) tablespace pg_default;
66+
67+
create table public.todos (
68+
id serial,
69+
uuid uuid not null unique,
70+
created_at timestamp with time zone not null default now(),
71+
completed_at timestamp with time zone null,
72+
description text not null,
73+
completed boolean not null default false,
74+
created_by uuid null,
75+
completed_by uuid null,
76+
list_id int not null,
77+
list_uuid uuid not null,
78+
constraint todos_pkey primary key (id),
79+
constraint todos_created_by_fkey foreign key (created_by) references auth.users (id) on delete set null,
80+
constraint todos_completed_by_fkey foreign key (completed_by) references auth.users (id) on delete set null,
81+
constraint todos_list_id_fkey foreign key (list_id) references lists (id) on delete cascade,
82+
constraint todos_list_uuid_fkey foreign key (list_uuid) references lists (uuid) on delete cascade
83+
) tablespace pg_default;
84+
```
85+
</CodeGroup>
86+
87+
With the data mode updated, we now need a method to synchronize and map the `list_id` and `list_uuid` in the `todos` table, with the `id` and `uuid` columns in the `lists` table.
88+
We can achieve this by creating SQL triggers.
89+
90+
# Create SQL Triggers
91+
92+
We need to create triggers that can look up the integer ID for the given UUID and vice versa.
93+
These triggers will maintain consistency between `list_id` and `list_uuid` in the `todos` table by ensuring that they remain synchronized with the `id` and `uuid` columns in the `lists` table;
94+
even if changes are made to either field.
95+
96+
We will create the following two triggers that cover either scenario of updating the `list_id` or `list_uuid` in the `todos` table:
97+
1. `update_integer_id`, and
98+
2. `update_uuid_column`
99+
100+
<AccordionGroup>
101+
<Accordion title="Trigger 1: update_integer_id" defaultOpen={true}>
102+
The `update_integer_id` trigger ensures that whenever a `list_uuid` value is inserted or updated in the `todos` table,
103+
the corresponding `list_id` is fetched from the `lists` table and updated automatically. It also validates that the `list_uuid` exists in the `lists` table; otherwise, it raises an exception.
104+
105+
```sql
106+
CREATE OR REPLACE FUNCTION func_update_integer_id()
107+
RETURNS TRIGGER AS $$
108+
BEGIN
109+
IF TG_OP = 'INSERT' THEN
110+
-- Always update list_id on INSERT
111+
SELECT id INTO NEW.list_id
112+
FROM lists
113+
WHERE uuid = NEW.list_uuid;
114+
115+
IF NOT FOUND THEN
116+
RAISE EXCEPTION 'UUID % does not exist in lists', NEW.list_uuid;
117+
END IF;
118+
119+
ELSIF TG_OP = 'UPDATE' THEN
120+
-- Only update list_id if list_uuid changes
121+
IF NEW.list_uuid IS DISTINCT FROM OLD.list_uuid THEN
122+
SELECT id INTO NEW.list_id
123+
FROM lists
124+
WHERE uuid = NEW.list_uuid;
125+
126+
IF NOT FOUND THEN
127+
RAISE EXCEPTION 'UUID % does not exist in lists', NEW.list_uuid;
128+
END IF;
129+
END IF;
130+
END IF;
131+
132+
RETURN NEW;
133+
END;
134+
$$ LANGUAGE plpgsql;
135+
136+
CREATE TRIGGER update_integer_id
137+
BEFORE INSERT OR UPDATE ON todos
138+
FOR EACH ROW
139+
EXECUTE FUNCTION func_update_integer_id();
140+
```
141+
</Accordion>
142+
<Accordion title="Trigger 2: update_uuid_column" defaultOpen={true}>
143+
The `update_uuid_column` trigger ensures that whenever a `list_id` value is inserted or updated in the todos table, the corresponding `list_uuid` is fetched from the
144+
`lists` table and updated automatically. It also validates that the `list_id` exists in the `lists` table.
145+
146+
```sql update_uuid_column
147+
CREATE OR REPLACE FUNCTION func_update_uuid_column()
148+
RETURNS TRIGGER AS $$
149+
BEGIN
150+
IF TG_OP = 'INSERT' THEN
151+
-- Always update list_uuid on INSERT
152+
SELECT uuid INTO NEW.list_uuid
153+
FROM lists
154+
WHERE id = NEW.list_id;
155+
156+
IF NOT FOUND THEN
157+
RAISE EXCEPTION 'ID % does not exist in lists', NEW.list_id;
158+
END IF;
159+
160+
ELSIF TG_OP = 'UPDATE' THEN
161+
-- Only update list_uuid if list_id changes
162+
IF NEW.list_id IS DISTINCT FROM OLD.list_id THEN
163+
SELECT uuid INTO NEW.list_uuid
164+
FROM lists
165+
WHERE id = NEW.list_id;
166+
167+
IF NOT FOUND THEN
168+
RAISE EXCEPTION 'ID % does not exist in lists', NEW.list_id;
169+
END IF;
170+
END IF;
171+
END IF;
172+
173+
RETURN NEW;
174+
END;
175+
$$ LANGUAGE plpgsql;
176+
177+
CREATE TRIGGER update_uuid_column
178+
BEFORE INSERT OR UPDATE ON todos
179+
FOR EACH ROW
180+
EXECUTE FUNCTION func_update_uuid_column();
181+
```
182+
</Accordion>
183+
</AccordionGroup>
184+
185+
We now have triggers in place that will handle the mapping for our updated data model and
186+
can move on to updating the Sync Rules to use the UUID column instead of the integer ID.
187+
188+
# Update Sync Rules
189+
190+
As sequential IDs can only be created on the backend database, we need to use UUIDs in the client. This can be done by updating both the `parameters` and `data` queries to use the new `uuid` columns.
191+
The `parameters` query is updated by removing the `list_id` alias (this is removed to avoid any confusion between the `list_id` column in the `todos` table), and
192+
the `data` query is updated to use the `uuid` column as the `id` column for the `lists` and `todos` tables. We also explicitly define which columns to select, as `list_id` is no longer required in the client.
193+
194+
```yaml sync_rules.yaml {4, 7-8}
195+
bucket_definitions:
196+
user_lists:
197+
# Separate bucket per todo list
198+
parameters: select id from lists where owner_id = request.user_id()
199+
data:
200+
# Explicitly define all the columns
201+
- select uuid as id, created_at, name, owner_id from lists where id = bucket.id
202+
- select uuid as id, created_at, completed_at, description, completed, created_by, list_uuid from todos where list_id = bucket.id
203+
```
204+
205+
With the Sync Rules updated, we can now move on to updating the client to use UUIDs.
206+
207+
# Update Client to Use UUIDs
208+
209+
With our Sync Rules updated, we no longer have the `list_id` column in the `todos` table.
210+
We start by updating `AppSchema.ts` and replacing `list_id` with `list_uuid` in the `todos` table.
211+
```typescript AppSchema.ts {3, 11}
212+
const todos = new Table(
213+
{
214+
list_uuid: column.text,
215+
created_at: column.text,
216+
completed_at: column.text,
217+
description: column.text,
218+
created_by: column.text,
219+
completed_by: column.text,
220+
completed: column.integer
221+
},
222+
{ indexes: { list: ['list_uuid'] } }
223+
);
224+
```
225+
226+
The `uploadData` function in `SupabaseConnector.ts` needs to be updated to use the new `uuid` column in both tables.
227+
228+
```typescript SupabaseConnector.ts {13, 17, 20}
229+
export class SupabaseConnector extends BaseObserver<SupabaseConnectorListener> implements PowerSyncBackendConnector {
230+
// other code
231+
232+
async uploadData(database: AbstractPowerSyncDatabase): Promise<void> {
233+
// other code
234+
try {
235+
for (const op of transaction.crud) {
236+
lastOp = op;
237+
const table = this.client.from(op.table);
238+
let result: any;
239+
switch (op.op) {
240+
case UpdateType.PUT:
241+
const record = { ...op.opData, uuid: op.id };
242+
result = await table.upsert(record);
243+
break;
244+
case UpdateType.PATCH:
245+
result = await table.update(op.opData).eq('uuid', op.id);
246+
break;
247+
case UpdateType.DELETE:
248+
result = await table.delete().eq('uuid', op.id);
249+
break;
250+
}
251+
}
252+
} catch (ex: any) {
253+
// other code
254+
}
255+
}
256+
}
257+
```
258+
259+
<Note>
260+
For the remaining files, we simply need to replace any reference to `list_id` with `list_uuid`.
261+
</Note>
262+
263+
```typescript fts_setup.ts {3}
264+
export async function configureFts(): Promise<void> {
265+
await createFtsTable('lists', ['name'], 'porter unicode61');
266+
await createFtsTable('todos', ['description', 'list_uuid']);
267+
}
268+
```
269+
270+
```tsx page.tsx {4, 14}
271+
const TodoEditSection = () => {
272+
// code
273+
const { data: todos } = useQuery<TodoRecord>(
274+
`SELECT * FROM ${TODOS_TABLE} WHERE list_uuid=? ORDER BY created_at DESC, id`,
275+
[listID]
276+
);
277+
278+
// code
279+
const createNewTodo = async (description: string) => {
280+
// other code
281+
await powerSync.execute(
282+
`INSERT INTO
283+
${TODOS_TABLE}
284+
(id, created_at, created_by, description, list_uuid)
285+
VALUES
286+
(uuid(), datetime(), ?, ?, ?)`,
287+
[userID, description, listID!]
288+
);
289+
}
290+
}
291+
```
292+
293+
```tsx TodoListWidget.tsx {10, 18}
294+
export function TodoListsWidget(props: TodoListsWidgetProps) {
295+
// hooks and navigation
296+
297+
const { data: listRecords, isLoading } = useQuery<ListRecord & { total_tasks: number; completed_tasks: number }>(`
298+
SELECT
299+
${LISTS_TABLE}.*, COUNT(${TODOS_TABLE}.id) AS total_tasks, SUM(CASE WHEN ${TODOS_TABLE}.completed = true THEN 1 ELSE 0 END) as completed_tasks
300+
FROM
301+
${LISTS_TABLE}
302+
LEFT JOIN ${TODOS_TABLE}
303+
ON ${LISTS_TABLE}.id = ${TODOS_TABLE}.list_uuid
304+
GROUP BY
305+
${LISTS_TABLE}.id;
306+
`);
307+
308+
const deleteList = async (id: string) => {
309+
await powerSync.writeTransaction(async (tx) => {
310+
// Delete associated todos
311+
await tx.execute(`DELETE FROM ${TODOS_TABLE} WHERE list_uuid = ?`, [id]);
312+
// Delete list record
313+
await tx.execute(`DELETE FROM ${LISTS_TABLE} WHERE id = ?`, [id]);
314+
});
315+
};
316+
}
317+
```
318+
319+
```tsx SearchBarWidget.tsx {8, 19}
320+
export const SearchBarWidget: React.FC<any> = () => {
321+
const handleInputChange = async (value: string) => {
322+
if (value.length !== 0) {
323+
let listsSearchResults: any[] = [];
324+
const todoItemsSearchResults = await searchTable(value, 'todos');
325+
for (let i = 0; i < todoItemsSearchResults.length; i++) {
326+
const res = await powersync.get<ListRecord>(`SELECT * FROM ${LISTS_TABLE} WHERE id = ?`, [
327+
todoItemsSearchResults[i]['list_uuid']
328+
]);
329+
todoItemsSearchResults[i]['list_name'] = res.name;
330+
}
331+
if (!todoItemsSearchResults.length) {
332+
listsSearchResults = await searchTable(value, 'lists');
333+
}
334+
const formattedListResults: SearchResult[] = listsSearchResults.map(
335+
(result) => new SearchResult(result['id'], result['name'])
336+
);
337+
const formattedTodoItemsResults: SearchResult[] = todoItemsSearchResults.map((result) => {
338+
return new SearchResult(result['list_uuid'], result['list_name'] ?? '', result['description']);
339+
});
340+
setSearchResults([...formattedTodoItemsResults, ...formattedListResults]);
341+
}
342+
};
343+
}
344+
```

‎tutorials/client/data/overview.mdx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,4 +5,5 @@ description: "A collection of tutorials showcasing various data management strat
55

66
<CardGroup>
77
<Card title="Cascading Delete" icon="database" href="/tutorials/client/data/cascading-delete" horizontal/>
8+
<Card title="Auto-Incrementing ID Mapping " icon="database" href="/tutorials/client/data/map-local-uuid" horizontal/>
89
</CardGroup>

‎usage/sync-rules/client-id.mdx‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,8 @@ Care must be taken if a user can populate the same records from different device
5050

5151
#### Option 3: Use an ID mapping
5252

53-
Use UUIDs on the client, then map them to sequential IDs when performing an update on the server. This allows using a sequential primary key for each record, with an UUID as a secondary ID.
53+
Use UUIDs on the client, then map them to sequential IDs when performing an update on the server. This allows using a sequential primary key for each record, with a UUID as a secondary ID.
5454

55-
This mapping must be performed wherever the UUIDs are referenced, including for every foreign key column
55+
This mapping must be performed wherever the UUIDs are referenced, including for every foreign key column.
56+
57+
For more information, have a look at the [ID mapping tutorial](/tutorials/client/data/map-local-uuid).

0 commit comments

Comments
 (0)