Skip to content

High memory usage when calculating chunked Hamilton-Kerr relationship matrix #1333

@timothymillar

Description

@timothymillar

This is a bug in pedigree_kinship when used with method='Hamilton-Kerr', return_relationship=True and chunks=.... Converting the (chucked) kinships to relationships requires multiplying by a combination of the (potentially non-equal) ploidies. This is currently being achieved with a non-chunked matrix resulting in high memory usage when evaluating a single chunk of the returned relationship matrix. The offending code block is here:
https://github.com/sgkit-dev/sgkit/blob/main/sgkit/stats/pedigree.py#L1202

This should be simple to fix with a chunked ploidy matrix.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions