Specify refs when uploading data

[employee451]

Hi, it would be great if it was possible to specify the refs for documents when uploading data through FGU. This is useful if you need to make relationships between different documents that you upload.

[chj-web]

Hey!

This should already be possible, at least to some extent. You could create a data object like this:

{
  collection: "User",
  index: "user_by_id",
  key: "id",
  data: [
    { id: "1", name: "User1" }
  ]
}

And then use that index (user_by_id) in another data resource to get the ref of the user. Like this:

// This gets the ref of the (hopefully) previously created user.
const user = q.Select(
  "ref"
  q.Get(
    q.Match(
      q.Index("user_by_id")
      "1",
    )
  )
);

const data: DataResource = {
  collection: "Post",
  index: "post_by_slug",
  key: "slug",
  data: [
    // The user field will now contain a ref
    { slug: "hello-world", user: user }
  ]
}

Now, I do realize that this is a bit cumbersome, especially if you need to create multiple relationships. There is also the potential issue of resources being created in the wrong order, resulting in reference errors.

We could create one or more utility functions to handle this use case, to both make it easier to work with relationships and also prevent reference errors.

It would then be possible to write references in a way similar to the following example:

import { getDataRef } from "fauna-gql-upload";

// Import the user data resource
import UserData from "./user.ts";

const post: DataResource = {
  collection: "Post",
  index: "post_by_slug",
  key: "slug",
  data: [
    { slug: "hello-world", user: getDataRef(UserData, "1") }
  ]
}

Let me know if you think this kind of implementation would be helpful.

[employee451]

Hey! Thanks for the response.

This solution would not work for us, as we need specific values for the refs (we point to them from other places).
It should be possible to specify the ref when creating documents, like so:

Create(Ref(Collection("foo"), "298227357544612362"),
  {
    data: { bar: true, baz: 1 }
  }
)

It's unclear to me what the key is actually used for?

[chj-web]

Hey!

Okay, I see what you want to do. That is in fact not possible with FGU at the moment. This would be fairly easy to support, though. We could just add an optional field to each of the data entries so that you could write them like this:

{
  data: [
    { slug: "hello-world", ref: Ref(Collection("foo"), "298227357544612362")) }
  ]
}

where the data entries containing a ref field will receive special treatment and probably be translated to something similar to your example. We could even omit the Collection() part, since it is already specified in the resource.

As for the key field, it helps keep the data idempotent. Its purpose is to identify a field where the data will be unique and thus identify each specific data entry. It is the field that will be passed to the specified index.

Now, I do think this interface is a bit confusing (I'm having trouble with it myself), so it will probably be replaced in a future major release.

But anyway, let me know if the example above would be an acceptable solution. Also, let me know if you have any suggestions on how the interface should work.

[employee451]

That seems like a good solution!

In a major release it might make more sense to match the structure Fauna uses though (although it would be a breaking change and maybe annoying if you only need to provide some data):

{
  ref: Ref(Collection("spells"), "181388642046968320"),
  data: {
    name: 'Fire Beak',
    element: [ 'air', 'fire' ],
    spellbook: Ref(Collection("spellbooks"), "181388642139243008")
  }
}

Thanks for the clarification about the key field.

[chj-web]

Yes absolutely!

Matching the Fauna structure is definitely something we'll aim for in a major release. But, as you say, it might not be enough to warrant a breaking change/major release on its own.

Anyway, I will add the

{
  data: [
    { slug: "hello-world", ref: Ref(Collection("foo"), "298227357544612362")) }
  ]
}

behavior as soon as I can and notify you here when it's released.

[employee451]

Hey again,
What's the status on this? Let me know if you need any help implementing this.

[chj-web]

Hey,

I'm actually just about finished with this. Only need to do some more testing before publishing it. I'm planning on releasing 2.5.0 with a few other features soon, so I'll bundle this in with that. I'll probably get around to that next weekend.

[employee451]

Thanks for the update, this sounds great!

[chj-web]

Hey!

This was just released with version 2.5.0!