Force-included sequences are missing dates

[victorlin]

Description

Some bug fixes in Augur surfaced an issue with the data used in this repo. Here's a preview of the error message when pathogen-repo-ci is run with changes from nextstrain/augur#1913 + nextstrain/augur#1961:

ERROR: The following sequence ids are missing valid dates:

  'AF202541' has date 'XXXX-XX-XX'
  'AF206518' has date 'XXXX-XX-XX'
  'AF260968' has date 'XXXX-XX-XX'
  'AF481864' has date 'XXXX-XX-XX'

Such ambiguous dates are normally excluded by augur filter, but here they have been added back for various reasons:

WNV/phylogenetic/defaults/all-lineages/include.txt

Line 1 in ea0fe14

AF260968 # Egypt 1951 all-lineages reference

WNV/phylogenetic/defaults/all-lineages/include.txt

Line 7 in ea0fe14

AF481864 # pre-NY

WNV/phylogenetic/defaults/all-lineages/include.txt

Lines 17 to 18 in ea0fe14

	AF202541 # NY99
	AF206518 # NY99

Possible solutions

• AF260968 already has an annotation for date=1951-XX-XX. Update example data to reflect this.
• Add date info for all other force-included sequences that need it.

[victorlin]

Hmm... there may be a larger issue and better solution here. Looking at metadata.tsv.zst, nearly 70% of data has date=XXXX-XX-XX.

EDIT: nevermind, this is just how the data is. There are 25k sequences in the ingest workflow output. Pathoplexus only has 8.5k sequences, which is just 1k more than the 7.5k sequences with date info in our metadata. This lines up with the improved date coverage that I observed in #113 (comment).