Changes to .janno and .ssf columns for Poseidon v3.0.0

[nevrome]

This PR is part of the preparation for Poseidon v3.0.0 as specified here: poseidon-framework/poseidon-schema#93

As documented in #351 I implemented the changes to .janno and .ssf.

While doing so I noticed two minor things I wanted to change in the new schema specification: poseidon-framework/poseidon-schema@31900ef & poseidon-framework/poseidon-schema@a256a62

Adding and modifying the columns in poseidon-hs also required (or: made desirable) some minor changes in the surrounding code -- but nothing spectacular.

[codecov]

Codecov Report

❌ Patch coverage is 51.93237% with 199 lines in your changes missing coverage. Please review.
✅ Project coverage is 55.68%. Comparing base (fec6b1f) to head (aa4e4db).
⚠️ Report is 1 commits behind head on Schema_300_dev.

Files with missing lines	Patch %	Lines
src/Poseidon/ColumnTypesJanno.hs	42.96%	32 Missing and 45 partials ⚠️
src/Poseidon/Janno.hs	60.92%	3 Missing and 56 partials ⚠️
src/Poseidon/SequencingSource.hs	45.16%	7 Missing and 27 partials ⚠️
src/Poseidon/ColumnTypesSSF.hs	37.14%	9 Missing and 13 partials ⚠️
src/Poseidon/ColumnTypesUtils.hs	78.57%	0 Missing and 3 partials ⚠️
src/Poseidon/PoseidonVersion.hs	33.33%	0 Missing and 2 partials ⚠️
src/Poseidon/CLI/Validate.hs	50.00%	0 Missing and 1 partial ⚠️
src/Poseidon/Package.hs	87.50%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@                Coverage Diff                 @@
##           Schema_300_dev     #357      +/-   ##
==================================================
- Coverage           56.89%   55.68%   -1.22%     
==================================================
  Files                  33       33              
  Lines                5043     5237     +194     
  Branches              548      632      +84     
==================================================
+ Hits                 2869     2916      +47     
- Misses               1626     1689      +63     
- Partials              548      632      +84     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[stschiff]

Thanks, I will take a look.

[nevrome]

Meeting 2025/12/01: Clemens goes through this PR and documents it properly to make a review possible.

[nevrome]

This PR is pretty straight forward, actually. It just adds new columns and some special handling for the ones whose semantics change with Poseidon v3.0.0. This affects so many files because changes in the .janno columns change a lot of reference data for the golden tests.

The only interesting bit is the way _Note columns are sorted when writing .janno files.

There are still two open PRs in the schema repository that should be worked into this PR, as soon as they are merged:

• add Individual_ID as mandatory column poseidon-schema#109
• Another capture type and some documentation on the individual protocols poseidon-schema#114

When this is done, the review can proceed.

[nevrome]

OK - I think this is ready for review. It seems overwhelming with all these files affected, but the actual changes are realtively minimal.

I suggest you start in src/Poseidon/ColumnTypesUtils.hs. That's where the new FromFieldVersioned lives that conceptually powers a lot of the new versioned .janno/.ssf parsing machinery. The consequences of this change are well visible then in src/Poseidon/ColumnTypesJanno.hs and beyond, mixed with the few individual column-wise changes of Poseidon v3.0.0.