multichrom work for outputFull() and readFromPopulationFile()

Author: bhaller

QtSLiM/QtSLiMWindow.cpp

@@ -5882,8 +5882,7 @@ void QtSLiMWindow::dumpPopulationClicked(void)
             slim_tick_t species_cycle = displaySpecies->Cycle();
 
             // dump the population
-            SLIM_OUTSTREAM << "#OUT: " << community->tick_ << " " << species_cycle << " A" << std::endl;
-            displaySpecies->population_.PrintAll(SLIM_OUTSTREAM, true, true, false, false);	// output spatial positions and ages if available, but not ancestral sequence
+            displaySpecies->population_.PrintAll(SLIM_OUTSTREAM, true, true, false, false, false );	// output spatial positions and ages if available, but not ancestral sequence or tags
 
             // dump fixed substitutions also; so the dump in SLiMgui is like outputFull() + outputFixedMutations()
             SLIM_OUTSTREAM << std::endl;

QtSLiM/help/SLiMHelpClasses.html

@@ -958,8 +958,7 @@
 <p class="p6"><span class="s3">In SLiM 3.3 and later, the output format includes the nucleotides associated with any nucleotide-based mutations.</span></p>
 <p class="p4">Output is generally done in a <span class="s1">late()</span> event, so that the output reflects the state of the simulation at the end of a tick.</p>
 <p class="p3">– (integer$)readFromPopulationFile(string$ filePath<span class="s5">, [No&lt;Dictionary&gt;$ subpopMap = NULL]</span>)</p>
-<p class="p6">Read from a population initialization file, whether in text or binary format as previously specified to <span class="s1">outputFull()</span>, and return the tick counter value represented by the file’s contents (i.e., the tick at which the file was generated).<span class="Apple-converted-space">  </span>Although this is most commonly used to set up initial populations (often in an Eidos event set to run in tick 1, immediately after simulation initialization), it may be called in any <span class="s1">early()</span> or <span class="s1">late()</span> Eidos event; the current state of all populations in the target species will be wiped and replaced by the state in the file at <span class="s1">filePath</span>.<span class="Apple-converted-space">  </span>All Eidos variables that are of type <span class="s1">object</span> and have element type <span class="s1">Subpopulation</span>, <span class="s6">Haplosome</span>, <span class="s1">Mutation</span>, <span class="s1">Individual</span>, or <span class="s1">Substitution</span> will be removed as a side effect of this method if they contain any element that belongs to the target species, because those objects will no longer exist in the SLiM simulation; if you want to preserve any of that state, you should output it or save it to a file prior to this call.<span class="Apple-converted-space">  </span>New symbols will be defined to refer to the new <span class="s1">Subpopulation</span> objects loaded from the file.</p>
-<p class="p6">If the file being read was written by a version of SLiM prior to 2.3, then for backward compatibility fitness values will be calculated immediately for any new subpopulations created by this call, which will trigger the calling of any activated and applicable <span class="s1">mutationEffect()</span> and <span class="s1">fitnessEffect()</span> callbacks.<span class="Apple-converted-space">  </span>When reading files written by SLiM 2.3 or later, fitness values are not calculated as a side effect of this call (because the simulation will often need to evaluate interactions or modify other state prior to doing so).</p>
+<p class="p6">Read from a population initialization file, whether in text or binary format as previously specified to <span class="s1">outputFull()</span>, and return the tick counter value represented by the file’s contents (i.e., the tick at which the file was generated).<span class="Apple-converted-space">  </span>Although this is most commonly used to set up initial populations (often in an Eidos event set to run in tick 1, immediately after simulation initialization), it may be called in any <span class="s1">early()</span> or <span class="s1">late()</span> Eidos event; the current state of all populations in the target species will be wiped and replaced by the state in the file at <span class="s1">filePath</span>.<span class="Apple-converted-space">  </span>All Eidos variables that are of type <span class="s1">object</span> and have element type <span class="s1">Subpopulation</span>, <span class="s1">Haplosome</span>, <span class="s1">Mutation</span>, <span class="s1">Individual</span>, or <span class="s1">Substitution</span> will be removed as a side effect of this method if they contain any element that belongs to the target species, because those objects will no longer exist in the SLiM simulation; if you want to preserve any of that state, you should output it or save it to a file prior to this call.<span class="Apple-converted-space">  </span>New symbols will be defined to refer to the new <span class="s1">Subpopulation</span> objects loaded from the file.<span class="Apple-converted-space">  </span>Note that fitness values are not calculated as a side effect of this call (because the simulation will often need to evaluate interactions or modify other state prior to doing so).</p>
 <p class="p6"><span class="s3">In SLiM 2.3 and later when using the WF model, calling </span><span class="s4">readFromPopulationFile()</span><span class="s3"> from any context other than a </span><span class="s4">late()</span><span class="s3"> event causes a warning; calling from a </span><span class="s4">late()</span><span class="s3"> event is almost always correct in WF models, so that fitness values can be automatically recalculated by SLiM at the usual time in the tick cycle without the need to force their recalculation (see comments on </span><span class="s4">recalculateFitness()</span><span class="s3">).</span></p>
 <p class="p6"><span class="s3">In SLiM 3.0 when using the nonWF model, calling </span><span class="s4">readFromPopulationFile()</span><span class="s3"> from any context other than an </span><span class="s4">early()</span><span class="s3"> event causes a warning; calling from an </span><span class="s4">early()</span><span class="s3"> event is almost always correct in nonWF models, so that fitness values can be automatically recalculated by SLiM at the usual time in the tick cycle without the need to force their recalculation (see comments on </span><span class="s4">recalculateFitness()</span><span class="s3">).</span></p>
 <p class="p6">As of SLiM 2.1, this method changes the tick and cycle counters to the tick and cycle read from the file.<span class="Apple-converted-space">  </span>If you do not want these counters to be changed, you can change them back after reading, by setting <span class="s1">community.tick</span> and <span class="s1">sim.cycle</span> to whatever values you wish.<span class="Apple-converted-space">  </span>Note that restoring a saved past state and running forward again will not yield the same simulation results, because the random number generator’s state will not be the same; to ensure reproducibility from a given time point, <span class="s1">setSeed()</span> can be used to establish a new seed value.<span class="Apple-converted-space">  </span>Any changes made to structure of the species (mutation types, genomic element types, etc.) will not be wiped and re-established by <span class="s1">readFromPopulationFile()</span>; this method loads only the population’s state, not the species configuration, so care should be taken to ensure that the species structure meshes coherently with the loaded data.<span class="Apple-converted-space">  </span>Indeed, state such as the selfing and cloning rates of subpopulations, values set into <span class="s1">tag</span> properties, and values set onto objects with <span class="s1">setValue()</span> will also be lost, since it is not saved out by <span class="s1">outputFull()</span>.<span class="Apple-converted-space">  </span>Only information saved by <span class="s1">outputFull()</span> will be restored; all other state associated with the species – subpopulations, individuals, haplosomes, mutations, and substitutions – will be lost, and should be re-established by the model if it is still needed.</p>

SLiMgui/SLiMHelpClasses.rtf

@@ -8731,7 +8731,7 @@ In SLiM 3.3 and later, the output format includes the nucleotides associated wit
 \pard\pardeftab720\li720\fi-446\ri720\sb180\sa60\partightenfactor0
 
 \f3\fs18 \cf0 \'96\'a0(integer$)readFromPopulationFile(string$\'a0filePath\cf2 , [No<Dictionary>$\'a0subpopMap\'a0=\'a0NULL]\cf0 )\
-\pard\pardeftab397\li547\ri720\sb60\sa60\partightenfactor0
+\pard\pardeftab543\li547\ri720\sb60\sa60\partightenfactor0
 
 \f4\fs20 \cf2 Read from a population initialization file, whether in text or binary format as previously specified to 
 \f3\fs18 outputFull()
@@ -8746,24 +8746,17 @@ In SLiM 3.3 and later, the output format includes the nucleotides associated wit
 \f4\fs20  and have element type 
 \f3\fs18 Subpopulation
 \f4\fs20 , 
-\f3\fs18 \cf0 Haplosome
-\f4\fs20 \cf2 , 
+\f3\fs18 Haplosome
+\f4\fs20 , 
 \f3\fs18 Mutation
 \f4\fs20 , 
 \f3\fs18 Individual
 \f4\fs20 , or 
 \f3\fs18 Substitution
 \f4\fs20  will be removed as a side effect of this method if they contain any element that belongs to the target species, because those objects will no longer exist in the SLiM simulation; if you want to preserve any of that state, you should output it or save it to a file prior to this call.  New symbols will be defined to refer to the new 
 \f3\fs18 Subpopulation
-\f4\fs20  objects loaded from the file.\
-\pard\pardeftab720\li547\ri720\sb60\sa60\partightenfactor0
-\cf2 If the file being read was written by a version of SLiM prior to 2.3, then for backward compatibility fitness values will be calculated immediately for any new subpopulations created by this call, which will trigger the calling of any activated and applicable 
-\f3\fs18 mutationEffect()
-\f4\fs20  and 
-\f3\fs18 fitnessEffect()
-\f4\fs20  callbacks.  When reading files written by SLiM 2.3 or later, fitness values are not calculated as a side effect of this call (because the simulation will often need to evaluate interactions or modify other state prior to doing so).\
-\pard\pardeftab543\li547\ri720\sb60\sa60\partightenfactor0
-\cf2 \expnd0\expndtw0\kerning0
+\f4\fs20  objects loaded from the file.  Note that fitness values are not calculated as a side effect of this call (because the simulation will often need to evaluate interactions or modify other state prior to doing so).\
+\expnd0\expndtw0\kerning0
 In SLiM 2.3 and later when using the WF model, calling 
 \f3\fs18 readFromPopulationFile()
 \f4\fs20  from any context other than a 

SLiMgui/SLiMWindowController.mm

@@ -3098,8 +3098,7 @@ - (IBAction)dumpPopulationToOutput:(id)sender
 		slim_tick_t species_cycle = displaySpecies->Cycle();
 
 		// dump the population
-		SLIM_OUTSTREAM << "#OUT: " << community->tick_ << " " << species_cycle << " A" << std::endl;
-		displaySpecies->population_.PrintAll(SLIM_OUTSTREAM, true, true, false, false);	// output spatial positions and ages if available, but not ancestral sequence
+		displaySpecies->population_.PrintAll(SLIM_OUTSTREAM, true, true, false, false, false);	// output spatial positions and ages if available, but not ancestral sequence or tags
 
 		// dump fixed substitutions also; so the dump in SLiMgui is like outputFull() + outputFixedMutations()
 		SLIM_OUTSTREAM << std::endl;
@@ -3290,10 +3289,8 @@ - (IBAction)exportPopulation:(id)sender
 
 			// BCH 3/6/2022: Note that the species cycle has been added here for SLiM 4, in keeping with SLiM's native output formats.
 			Species *displaySpecies = [self focalDisplaySpecies];
-			slim_tick_t species_cycle = displaySpecies->Cycle();
 
-			outstring << "#OUT: " << community->tick_ << " " << species_cycle << " A " << std::endl;
-			displaySpecies->population_.PrintAll(outstring, true, true, true, false);	// include spatial positions, ages, and ancestral sequence, if available
+			displaySpecies->population_.PrintAll(outstring, true, true, true, false, true);	// include spatial positions, ages, ancestral sequence, and tags, if available
 
 			std::string &&population_dump = outstring.str();
 			NSString *populationDump = [NSString stringWithUTF8String:population_dump.c_str()];

VERSIONS

@@ -126,6 +126,10 @@ development head (in the master branch):
 			Subpopulation method outputVCFSample(): check that all haplosomes belong to a single chromosome; use chromosome symbols for the CHROM column
 			add [l$ groupAsIndividuals = T] parameter; if F, each haplosome is emitted as a haploid call rather than forming diploids, regardless of the chromosome's ploidy
 			Haplosome_Class::ExecuteMethod_readFromVCF(): require VCF data that is single-chromosome, as with output
+		fix built-in SLiM-format full input/output methods for multiple chromosomes:
+			Species method outputFull(): allow multi-chromosome output with a new sectioned format, for both text and binary
+			Species method readFromPopulationFile(): support reading multi-chromosome output, for both text and binary
+			policy change: readFromPopulationFile() breaks backward compatibility with old file formats, for both text and binary; too much has changed
 
 
 version 4.3 (Eidos version 3.3):