Skip to content

[Rubin] diaObjectId is not interpreted correctly in datatransfer #228

@JulienPeloton

Description

@JulienPeloton
# on 20260120
In [36]: import glob
    ...: import pandas as pd
    ...: 
    ...: fns = glob.glob("ftransfer_lsst_2026-01-27_422049/*.parquet")
    ...: pdf = pd.DataFrame()
    ...: for fn in fns:
    ...:     pdf = pd.concat((pdf, pd.read_parquet(fn)))
    ...: pdf = pdf.reset_index()

In [37]: objname = 169830580431093809 # as int64

In [38]: mask = pdf["diaObject"].apply(lambda x: x["diaObjectId"] == objname)

In [39]: pdf[mask]
Out[39]: 
      index         diaSourceId  ... tns_type_recomputed          conv
5475      1  169896545491091497  ...             Unknown  1.698306e+17
5848      5  169896548713889829  ...             Unknown  1.698306e+17

[2 rows x 30 columns]

In [35]: conversion = pdf["diaObject"].apply(lambda x: 0 if x["diaObjectId"] is None else x["diaObjectId"])

In [36]: mask = conversion.values == objname

In [37]: pdf[mask]
Out[37]: 
      index         diaSourceId  ... tns_type_recomputed          conv
5475      1  169896545491091497  ...             Unknown  1.698306e+17
5848      5  169896548713889829  ...             Unknown  1.698306e+17
8774      6  169896545604861979  ...             Unknown  1.698306e+17

[3 rows x 30 columns]

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions