Error when changing metadata to individuals in table of tree sequence

[Jesson-mark]

Hi tskit team,

thanks for the great software! I am currently learning tskit to process my data. I followed the codes from this tutorial and found that my codes raised a TypeError.

The codes I used are as below:

ts = tskit.load("tutorials/data/tables_example.trees")
new_tables = ts.dump_tables()  # make a copy of the tree sequence tables, for editing
new_tables.sort()
altered_ts = new_tables.tree_sequence()  

tables = altered_ts.dump_tables()

# Change the name "Bob" to "Robert"
tables.individuals[1] = tables.individuals[1].replace(metadata={"name": "Robert"})

The error messages are as below:

TypeError                                 Traceback (most recent call last)
Cell In[15], line 5
      2 tables = altered_ts.dump_tables()
      4 # Change the name "Bob" to "Robert"
----> 5 tables.individuals[1] = tables.individuals[1].replace(metadata={"name": "Robert"})
      6 edited_ts = tables.tree_sequence()
      7 edited_ts.tables.individuals

File /opt/miniforge3/envs/py3.10/lib/python3.10/site-packages/tskit/tables.py:802, in MutableBaseTable.__setitem__(self, index, new_row)
    798 # Encode the metadata - note that if this becomes a perf bottleneck it is
    799 # possible to use the cached, encoded metadata in the row object, rather than
    800 # decode and reencode
    801 if "metadata" in row_data:
--> 802     row_data["metadata"] = self.metadata_schema.validate_and_encode_row(
    803         row_data["metadata"]
    804     )
    806 self.ll_table.update_row(row_index=index, **row_data)

File /opt/miniforge3/envs/py3.10/lib/python3.10/site-packages/tskit/metadata.py:858, in MetadataSchema.validate_and_encode_row(self, row)
    856 if not self._bypass_validation:
    857     try:
--> 858         self._validate_row(row)
    859     except jsonschema.exceptions.ValidationError as ve:
    860         raise exceptions.MetadataValidationError(str(ve)) from ve

File /opt/miniforge3/envs/py3.10/lib/python3.10/site-packages/tskit/metadata.py:768, in validate_bytes(data)
    766 def validate_bytes(data: bytes | None) -> None:
    767     if data is not None and not isinstance(data, bytes):
--> 768         raise TypeError(
    769             f"If no encoding is set metadata should be bytes, found {type(data)}"
    770         )

TypeError: If no encoding is set metadata should be bytes, found <class 'dict'>

Do you have any idea about this error? Any input would be greatly appreciated!

Best regards,

Jie

[hyanwong]

I'm moving this to the tutorials repo

[hyanwong]

I think you are loading "tutorials/data/tables_example.trees" whereas the tutorial loads "data/tables_example_muts.trees". The second file has a permissive json schema set on the individuals table, and the first one doesn't.

You can get your version to work by changing the metadata schema and re-loading the rows. Something like the following:

import json

tables.individuals.clear()
tables.individuals.metadata_schema = tskit.MetadataSchema.permissive_json()
for ind in altered_ts.individuals():
     tables.individuals.append(ind.replace(metadata=json.loads(ind.metadata.decode())))

[Jesson-mark]

Thank you for your suggestions! Using the "data/tables_example_muts.trees" file solved this error.