Skip to content

lsdb-server cannot filter based on string columns #11

@gitosaurus

Description

@gitosaurus

Bug report

Back-end filtering works for numeric quantities but not for string/enum quantities. Both of these attempts fail:

gaia3 = lsdb.read_hats(
    "http://epyc.astro.washington.edu:43210/hats/gaia_dr3/gaia/",
    filters=[
                ["phot_variable_flag", "=", "CONSTANT"],  # bare string
            ],
)
gaia3 = lsdb.read_hats(
    "http://epyc.astro.washington.edu:43210/hats/gaia_dr3/gaia/",
    filters=[
                ["phot_variable_flag", "=", '"CONSTANT"'],  # forced string literal
            ],
)

Both of these cause this back-end error from lsdb-server:

called `Result::unwrap()` on an `Err` value: NotYetImplemented("Data type Utf8 not yet implemented")

The first approach works with https://data.lsdb.io/hats/gaia_dr3/gaia, and the second does not (and perhaps should not).

NOTE: there are no columns in Gaia DR3 where phot_variable_flag has the value CONSTANT. There is only NOT_AVAILABLE and VARIABLE; be aware of that when testing.

Before submitting
Please check the following:

  • I have described the situation in which the bug arose, including what code was executed, information about my environment, and any applicable data others will need to reproduce the problem.
  • I have included available evidence of the unexpected behavior (including error messages, screenshots, and/or plots) as well as a description of what I expected instead.
  • If I have a solution in mind, I have provided an explanation and/or pseudocode and/or task list.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions