Greater/Less than comparators for strings

[ericchernuka]

We have a use case where we're building the ability to navigate backwards with cursor-based pagination. To facilitate sorting, we need a way to do a where clause that possibly uses a greater than/less than query against a property. Before I do a PR to add this, I wanted to see if there were prior reasons for not including gt|gte|lt|lte for strings in the past.

Reference: https://brunoscheufler.com/blog/2022-01-01-paginating-large-ordered-datasets-with-cursor-based-pagination#ordering-with-two-attributes

[kettanaito]

Hey, @ericchernuka. Thanks for the proposal.

How would you implement these new comparators? Simply as A > B, or do you have something else in mind?

I don't like the semantics breach that happens when we handle strings this way, to be honest. I'd much prefer to go with something like { id: { satisfies(left, right): boolean } } but then I remembered that the queries must be serializable. We definitely need to define this better before moving forward so thanks for raising this discussion.

[ericchernuka]

Yea, the serializability makes it interesting. I had thought I only needed basic A > B, but you bring up a good point on something more explicit. I'm all ears for any ideas you might have. We found a way around this by just doing it all in memory after the initial where clause.

[kettanaito]

I'm honestly at the stage of considering if comparison operators can be semantically applied to strings. Sure, in JavaScript land everything is possible through the magic of coercion but the real question to ask is whether that's the right thing. I'd say that it's incorrect to compare strings with > and < but perhaps I haven't got a use case for it in my entire engineering career, making me the wrong person to judge this.

satisfies() is great but it cannot be serialized. We can technically make it serializable by stringifying its function body and then doing eval() whenever we need to. But this puts too many restrictions on how you should write that function body (well, it must be pure). I'm not a fan of this.

I generally don't have anything against adding what you've suggested just want to make sure this is logical to people.

[ericchernuka]

I understand. To give context, we were just trying to mimic how SQL makes string comparisons, but it's limited in its ability here. A case in point would be an example of a1, a2, a3, a10, if we made any of the comparisons, it wouldn't work due to the numbers, but if we did ISO8601 dates, it would work correctly due to the format. Sadly it's not a bulletproof and straightforward solution :(

[kettanaito]

I think since you can technically compare strings in JavaScript, I don't see a reason why this library would block you from doing so. TIL that it's not coercion that powers this string comparison but actual comparison of unicode symbols, which gives me more hope in its semantics.

Overall, I'm for this change. If you have time, would you be interested in opening a pull request?

[ericchernuka]

You bet! I already have a few things I'm supposed to do for this library, and my work gave me the power to contribute back during office hours, so I'll find some time in the next week.

Thanks for the conversation about the idea. Was super helpful, and I appreciate the back and forth.

[kettanaito]

Update

Since v1.0, you can implement your own comparators (in fact, the library no longer ships any built-in comparators at all). This is how you get a gt/lt comparator for strings:

const users = new Collection({
  schema: z.object({
    firstName: z.string(),
    lastName: z.string(),
  }),
})

users.findFirst((q) => q.where({
  firstName: (firstName) => firstName > 'Alice',
  lastName: (lastName) => lastName < 'Doe',
}))