Request for example using sveltekit

[eranmiller]

Greetings Max,

I hope you can help me.
I cannot seem to integrate this very promising library into my Sveltekit project. Without a complete example to reference I keep hitting roadblocks.
If you don't have time to create a complete example, perhaps you can show me where my configuration/understanding is flawed.

My setup includes dependencies:
"@signaldb/core": "^1.2.3",
"@signaldb/indexeddb": "^1.0.0",
"@signaldb/sync": "^1.1.1",

1. I get the following error, though I never explicitly import @signaldb/devtools, nor do I call methods from devtools.

Uncaught (in promise) TypeError: The specifier “@signaldb/devtools” was a bare specifier, but was not remapped to anything. Relative module specifiers must start with “./”, “../” or “/”.

2. When I attempt to sync data from a MongoDb (MDb) database with an IndexedDb (IDb) database:

• the IDb gets the same document inserted over and over.
• its not using the MDb default _id as the primary key, instead it's creating a unique id - even when I use transform within new Collection options. Note, output is the same without transform.

My svelte component
<script lang="ts">
import { syncManager } from '$lib/services/Sync'
import type { CatalogItem } from '$lib/types/Catalog'
import { Collection } from '@signaldb/core'
import createIndexedDBAdapter from '@signaldb/indexeddb'
import { onMount } from 'svelte'

let initialLoadComplete = $state(false)
let items = $state.raw<CatalogItem[]>([])
let error = $state<string | null>(null)
let Catalog: Collection<CatalogItem>

onMount(() => {
  try {
    Catalog = new Collection<CatalogItem>({
      name: 'Catalog',
      persistence: createIndexedDBAdapter('Catalog'),
      transform: (data: CatalogItem): CatalogItem => {
        return {
          ...data,
          id: data._id ?? data.id,
          _id: undefined,
        }
      },
      reactivity: {
        create() {
          let dep = $state(0)
          return {
            depend() {
              if (!initialLoadComplete) return
              dep
            },
            notify() {
              if (!initialLoadComplete) return
              console.log('Notify called, current dep:', dep)
              dep += 1
            },
          }
        },
        isInScope: () => !!$effect.tracking(),
      },
    })
    syncManager.addCollection(Catalog, {
      name: 'catalog',
      apiPath: '/catalog',
    })

    syncManager.sync('catalog', {
      force: true,
    })

    initialLoadComplete = true
  } catch (e) {
    error = e instanceof Error ? e.message : 'Initialization failed'
  }
})

$effect(() => {
  const cursor = Catalog.find({})
  items = cursor.fetch()
  return () => {
    cursor.cleanup()
  }
})
</script>

{#if error}
  <div class="error">{error}</div>
{:else}
  <ul>
    Items: {items.length}
    {#each items as item}
      <li>
        {item.id}
        <strong>{item.meta.title}</strong> by {item.meta.artist} ({item.meta.year}) --{item.meta.duration}
      </li>
    {/each}
  </ul>
{/if}

3. I reduced the push method to log statements A complete example to reference would be incredibly helpful, I am finding the documentation difficult to follow.

• SyncManager pull method lastFinishedSyncStart is always undefined. https://signaldb.js.org/sync/#implementing-the-pull-method

my Sync.ts file
import { EventEmitter } from '@signaldb/core'
import { SyncManager } from '@signaldb/sync'

const errorEmitter = new EventEmitter()
errorEmitter.on('error', message => {
  console.error(message)
})

const apiBaseUrl = '/api/v1'
export const syncManager = new SyncManager({
  pull: async ({ apiPath, lastFinishedSyncStart }) => {
    console.log('Pull called with timestamp:', lastFinishedSyncStart)
    const url = lastFinishedSyncStart
      ? `${apiBaseUrl}${apiPath}?since=${lastFinishedSyncStart}`
      : `${apiBaseUrl}${apiPath}`

    const response = await fetch(url)
    const data = await response.json()
    const timestamp = Date.now()

    return { items: data, timestamp }
  },

  push: async ({ apiPath }, { changes }) => {
    if (
      changes.added.length ||
      changes.modified.length ||
      changes.removed.length
    ) {
      console.log('Push changes:', changes)
    }
    return Promise.resolve()
  },
})

The log output is:

Notify called, current dep: 0 [SignalDB.svelte:35:22](http://localhost:5173/src/routes/(app)/(catalog)/catalog/SignalDB.svelte)
Pulling from /api/v1/catalog [Sync.ts:13:12](http://localhost:5173/src/lib/services/Sync.ts)
Notify called, current dep: 0 2 [SignalDB.svelte:35:22](http://localhost:5173/src/routes/(app)/(catalog)/catalog/SignalDB.svelte)
Pulling from /api/v1/catalog [Sync.ts:13:12](http://localhost:5173/src/lib/services/Sync.ts)
Notify called, current dep: 1 [SignalDB.svelte:35:22](http://localhost:5173/src/routes/(app)/(catalog)/catalog/SignalDB.svelte)
Pulling from /api/v1/catalog [Sync.ts:13:12](http://localhost:5173/src/lib/services/Sync.ts)
updating data /catalog 
Object { added: (414) […], modified: [], removed: (1) […] }
[Sync.ts:39:12](http://localhost:5173/src/lib/services/Sync.ts)
Pulling from /api/v1/catalog [Sync.ts:13:12](http://localhost:5173/src/lib/services/Sync.ts)
updating data /catalog 
Object { added: (414) […], modified: [], removed: (1) […] }
[Sync.ts:39:12](http://localhost:5173/src/lib/services/Sync.ts)
Pulling from /api/v1/catalog [Sync.ts:13:12](http://localhost:5173/src/lib/services/Sync.ts)
Notify called, current dep: 2 [SignalDB.svelte:35:22](http://localhost:5173/src/routes/(app)/(catalog)/catalog/SignalDB.svelte)
Notify called, current dep: 0 [SignalDB.svelte:35:22](http://localhost:5173/src/routes/(app)/(catalog)/catalog/SignalDB.svelte)
Notify called, current dep: 1 [SignalDB.svelte:35:22](http://localhost:5173/src/routes/(app)/(catalog)/catalog/SignalDB.svelte)
Pulling from /api/v1/catalog [Sync.ts:13:12](http://localhost:5173/src/lib/services/Sync.ts)
Notify called, current dep: 3 [SignalDB.svelte:35:22](http://localhost:5173/src/routes/(app)/(catalog)/catalog/SignalDB.svelte)
Notify called, current dep: 4 [SignalDB.svelte:35:22](http://localhost:5173/src/routes/(app)/(catalog)/catalog/SignalDB.svelte)
Notify called, current dep: 0 [SignalDB.svelte:35:22](http://localhost:5173/src/routes/(app)/(catalog)/catalog/SignalDB.svelte)
Pulling from /api/v1/catalog [Sync.ts:13:12](http://localhost:5173/src/lib/services/Sync.ts)
Notify called, current dep: 5 [SignalDB.svelte:35:22](http://localhost:5173/src/routes/(app)/(catalog)/catalog/SignalDB.svelte)
updating data /catalog 
Object { added: (415) […], modified: [], removed: (1) […] }
[Sync.ts:39:12](http://localhost:5173/src/lib/services/Sync.ts)
Pulling from /api/v1/catalog [Sync.ts:13:12](http://localhost:5173/src/lib/services/Sync.ts)
updating data /catalog 
Object { added: (415) […], modified: [], removed: (1) […] }
[Sync.ts:39:12](http://localhost:5173/src/lib/services/Sync.ts)
Pulling from /api/v1/catalog

And the IDb data contains multiple entries of the same document from MDb - in other words, only one document ever gets imported to the IDb, not the 414 items that are received from MDb:

a7d2faee7403d8:"{"id":"a7d2faee7403d8","_id":"6791418657b7f7a30b72ddb4","meta":{"title":"A Test Song","artist":["A Test Artist"],"year":1974,"album":"Best Year Ever","duration":60,"key":"E","genres":["Eran"],"cover":"","tempo":null,"styles":null,"api_track_id":null},"modified_at":1713415028234,"is_deleted":false,"transpose_to":0,"is_draft":true,"sections":[...]"

Any help you can provide would be enormously appreciated.
Thank you!

[maxnowack]

Start by checking the Svelte guide: https://signaldb.js.org/guides/svelte/. However, this guide does not cover syncing with a remote API. For that, refer to the examples in the repository: https://github.com/maxnowack/signaldb/tree/main/examples. These examples use React, but the sync implementation concepts remain the same. Also, review the Sync Documentation: https://signaldb.js.org/sync/.

The TypeError related to @signaldb/devtools should be resolved once #1419 is merged and released.

If the API you’re fetching data from doesn’t use id as the primary key (e.g., it uses _id), you need to inform SignalDB. Otherwise, SignalDB will generate a new id for every document, assuming they don’t have one. In your push method, make sure to map id back to _id so your API correctly recognizes the documents.

Example:

const syncManager = new SyncManager({
  pull: async ({ apiPath, lastFinishedSyncStart }) => {
    // ...
    const response = await fetch(url)
    const data = await response.json()
    return {
      items: data.map(item => ({ id: item._id, ...item })),
    }
  },

  push: async ({ apiPath }, { changes }) => {
    const changesWithMongoId = {
      added: changes.added.map(item => ({ _id: item.id, ...item })),
      modified: changes.modified.map(item => ({ _id: item.id, ...item })),
      removed: changes.removed.map(item => ({ _id: item.id, ...item })),
    }
    // ...
  },
})

If lastFinishedSyncStart is undefined, it might be due to the SyncManager not persisting its state. Try adding a persistenceAdapter to ensure the internal state is saved and restored between sessions.

Example using IndexedDB for persistence:

import createIndexedDBAdapter from '@signaldb/indexeddb'

const syncManager = new SyncManager({
  persistenceAdapter: name => createIndexedDBAdapter(name),
})

I hope this solves the majority of your issues. Please let me know if something is still unclear 🙂