REVISIONS.txt

Revision log
2022.11.23
v1.4.9

* removed the dependency on the "corpus" package. 

2022.11.23
v1.4.8

* added ", useNames = TRUE" in two calls to matrixStats::rowQuantiles to 
accommodate impending changes in the matrixStats package

2022.11.02
v1.4.7

* changed is.finite(tmp) in obsCols_ltp to all(is.finite(tmp)) for stability
* amended the checkDate function for compatibility with new as.Date in R 4.3.0

2021.06.16
v1.4.6

* corrected a GUI display error for MLEs and CIs for r statistics in the 
table in the Analysis|Carcass Persistence|Estimates tab

* corrected an error that prevent the command line function desc from 
calculating summary CP statistics for a single Ir value (prior versions
required a vector of Ir values)

2021.01.29
v1.4.5.0

* fixed a downstream bug on labels for CI columns for Ir.

2020.11.16
v1.4.5

* fixed a bug that put the wrong labels on CI columns for Ir in the "desc" 
function for cp models when search intervals were entered in the arg list in
non-increasing order. This error would never occur in the GUI and would be 
rare on the command line side, limited to a single summay table, not affecting 
any other calculations.

* updated the email address of the maintainer

2020.06.06
v1.4.4

* In previous versions, the GUI would crash with the gray screen of death if a 
user tried to input data files that were not formatted properly as .csv files. 
Error-handling to prevent this instability has been added. 

* An SE model with one p covariate and fixed k was not fittable when there was
a cell with all 0s or 1s. This inconvience has been fixed. Also, the AICc score
and likelihood for determining the reference model for models in which there 
were cells with all 0s or 1s in the data were previously calculated using the 
adjusted data set (see revisions for v1.3.1) rather than the original data set.
This resulting in artificially poor AIC scores for the adjusted models and 
incorrect specification of the "reference" model in some cases. The error has 
been corrected.

* ambiguous date column formatting (i.e., where all dates in a column could be 
read either as m/d/y or d/m/y) now triggers an informative warning message in 
addition to simply returning NULL from the checkDate function.

2020.05.13
v1.4.3

* CP models for trial data in which all carcasses persisted beyond the study
period (i.e., FirstAbsent = Inf for all carcasses) or were removed before the 
first search were unstable, sometimes failing and sometimes returning 
implausible r statistics without warning. This is an inherent problem with 
asymptotically normal MLE estimators with data near the boundaries of the 
parameter space. 

* SE models for trial data in which the names of a factor level for one of the
predictors contained a period character ( . ) and either all carcasses or no 
carcasses were recovered on the first search after carcass placement for one of
the model cells, the model fitting was unstable and would mistakenly throw an 
error in certain cases. The error has been corrected.

2020.05.12
v1.4.2.1

* A more comprehensive summary table for download of estgGeneric. The summary
for "Download" from the GUI and for summary.estgGeneric now includes model
choices and alpha and beta parameters for fitted ghat's (following EoA 
convention).

* Entering a carcass class column when there was only one class would cause 
the SE and CP models to fail. Data columns with no variation are no longer
offered as choices in the "Carcass Class Column (optional)" drop down menu.
This change improves the stability of the GUI only but has no impact on any
results.

* ppersist with lognormal distribution would generate NaN when persistence 
times were extremely long. The error has been corrected.

* calcg function args have been made more flexible. Previously, param_SE
and param_CP needed to be two column arrays or data frames or to be vectors
with two elements. Now, they can also be multiple column arrays or data frames
with named columns. This will simplify the formatting for certain custom
analyses on the command line and will have no effect on GUI analyses.

* pkm and pk0 can now be called with a single column for search data rather 
than requiring a second column (for carcass ID or covariate or second search).
This will streamline certain custom command line analyses (esp. simulation)
but will have no effect on GUI analyses.

* pkm0 would fail on certain (rare) occasions where it was possible but not easy
to fit k because the default starting point for optimization (kInit = 0.7) is 
does not work for every conceivable situation. Now, if the initial kInit = 0.7
fails, GenEst will also try 0.05 and 0.95 before, which improves the stability
of the model fitting.

* estg now replaces values of ghat < 1e-7 with 1e-7. Very small values of ghat
could result in estM returning NA; the change solves that problem and makes the
estimator more stable in cases with low g and high variance in ghat.

2020.03.14
v1.4.2
* The `sticky` package may soon be "archived" at CRAN because the maintainers 
have not updated it to meet new CRAN requirements. To maintain ready 
availability of GenEst, the dependency on `sticky` had to be removed. The change
will have no impact on the GUI usage or results and is highly unlikely to affect
any command-line analyses.

* The "common scale" option for mortality figures with two splitting covariates
was not working in v1.4.1. Functionality has been restored.

* Running plot.estM from the command-line immediately after calling certain 
specialty plotting functions (e.g., plot.pkm) would look not good and in some
cases would result in "Error in plot.new() : figure margins too large". The 
problem was that the specialized plotting functions use par() settings for the 
margins and overall plot that are specially designed for the given function
and are not compatible with most other plot functions. By R convention, the 
par() settings are not reset until the plot device has been closed. plot.pkm,
plot.cpm, plot.splitSummary, plot.pkmSet, plot.cpmSet, and plot.gGeneric have 
been updated so that after their plotting is done, par() margins and layouts are 
reset to the R defaults: list(mfrow = c(1, 1), mar = c(5.1, 4.1, 4.1, 2.1), 
oma = c(0, 0, 0, 0), cex = 1, fig = c(0, 1, 0, 1)). This allows new figures to 
be drawn immediately to the same graphics device that had earlier housed one 
of the specialty graphs. The change affects the command-line functions only.

2020.03.03
v1.4.1

* For certain combinations of predictors, estgGeneric was returning two copies 
of estimates for some cells and none for others. The error has been corrected.

* A bug in `shiny` was preventing tables and figures from rendering in the GUI
in MS Edge and MS Internet Explorer whenever GenEst was started for the first
time in a new Windows session. GUI users would not be able to see uploaded data
tables until they closed and reopened GenEst. An update to `shiny` dependency
`httpuv` on 29 Aug 2019 fixes the bug. GenEst now requires `shiny` version 1.4.0
or greater.

* Some Windows users were having difficulty downloading the .csv versions of the 
example data files. The solution appears to be beyond the scope of the software,
requiring users to adjust their Windows settings by hand, and possibly requiring
installation of new software. The csv versions of the example data sets have now
been removed from the app and stored on the GenEst websites instead:
https://www.usgs.gov/software/genest-a-generalized-estimator-mortality
https://code.usgs.gov/ecosystems/GenEst/-/releases

* If covariates that were not included in CO were used in model_CP or model_SE, 
GenEst would fail to show a meaningful error message on "Estimate M" and would 
give a gray screen of death on "Estimate" under the "Detection Probability" tab.
The error messaging has been improved, and estimating generic detection 
probability under that scenario has been corrected. 

* An error-check in the routine to estimate generic detection probability was 
disallowing a case where a covariate used in the CP (or SE) model was not
included in the SE (or CP) data. That case (rightly) is no longer tagged as an
error.

* Code in calcg has been streamlined and simplified. A minor error in the 
previous code (version <= 1.4.0.2) that resulted in g values that were slightly 
too small under certain conditions has been corrected. The change will have no 
impact whatsoever on SE and CP modeling or on estimation of M (i.e., any 
funtions outside the "Detection Probability" tab in the GUI). The affected 
functions are calcg and functions that call calcg (estgGeneric, 
estgGenericSize). There  will no effect when nsearch = 1 (not including t = 0), 
negligible effect when nsearch > 3, slight effect when nsearch = 3 
(g_new/g_old - 1 < less than 10% and typically well below 10%), and a possibly
substantial effect when nsearch = 2 (not including t = 0, so 3 searches in all).
In all cases, g_new >= g_old. 

* Figures for generic g estimates have been upgraded.

* Font size in several figures in the GUI has been increased to improve
legibility.

* Certain legitimate mortality splits were generating an error: "Second split 
too fine for plotting. Consider transposing." The error has been corrected.

* estg (and by implication, estM) and defineUnitCol can now accept prepSS 
objects for data_SS. This allows for slightly streamlined estimation and faster 
run times in certain custom command-line analyses that involve simulation. The 
traditional format for data_SS files may still be used. The changes have no 
impact on GUI usage or performance.

* Function calcg no longer requires arrays of CP and SE parameters to have 
column names. This slightly simplifies the command-line implementation of 
certain custom analyses and has no impact on the GUI. 

* Function calcg now accepts single parameter sets rather than requiring arrays
of multiple parameter sets. 

* The estg function with arg model_DWP = NULL was failing to run. This could 
potentially cause inconvenience to a command-line user who would have to create
a data structure by hand in order to calculate estimates that assumed DWP = 1.
That inconvenience has been fixed, and model_DWP = NULL will run with the 
assumption that DWP = 1 for all units.

2020.02.05
v1.4.0.2

Added "..." to the arg list for pkm to meet new CRAN requirements. This change
will have no impact on any usage of GenEst.

2019.11.20
v1.4.0.1

* fixed a simple link in GenEst GUI authors that was causing the GUI to
crash upon opening under certain browser settings.

2019.11.06
v1.4.0

* updated the CP figures with improved font and readability; format now more 
closely matches that of the SE figures. Changes are limited to plot.cpmSet,
while the plot.cpm function for plotting individual models in isolation
remains unchanged (for now).

2019.11.02
v1.3.3

* corrected a data frame subsetting function that returned a data frame rather
than the expected vector. This was causing a the output matrix for CP model 
fits to be reconstructed as a list, which was causing gray-screen crashes when
CP output to the screen was for models other than the Weibull (GUI only). The
error has been corrected.

* an error in formatting data for displaying tables of summary statistics
for exponential CP models was preventing the tables from appearing under
the "Estimates" tab for CP models in the GUI. This bug has been fixed.

* labels were not properly matched to pk figures in the GUI when there were
two covariates and a fig for a "*" model for p was drawn after a fig for a "+"
model was drawn. Cell labels are now correctly matched to cell figs.

* an error in the function for downloading a comprehensive set of summary 
statistics for estimated mortality (including names of all data sets used, 
names and summary statistics for selected SE and CP models, AIC tables used  
for model selection, summary statistics for M_hat and splits) made it so only
the CP table was downloaded. The error has been corrected so that all the 
intended data is downloaded.

* An error in the calculation of extreme tail probabilities of loglogistic 
persistence distributions has been corrected. This bug would only be a problem
in estimating g (and M) when the CP distribution was loglogistic with very
short persistence times relative to the search intervals. The user would have
seen a cryptic error message when trying to estimate M or a gray screen (GUI)
when trying to estimate g.

* removed the "seed" arguments seed_SE, seed_CP, seed_g, and seed_M, in estg, 
estgGeneric, and estM. The utility of providing these seeds in the arg lists 
would be for minute management of creating strictly reprodicible results in 
command-line analyses. However, this purpose can normally be just as readily 
and more safely accomplished by management of seeds outside the function calls, 
as, e.g., set.seed(1234); estM(...) rather than estM(..., seed = 1234). In 
addition, ensuring that the internal seeds maintain their expected behavior 
after software updates unnecessarily complicates GenEst code maintenance.

* the old plot.gGeneric function determined covariate cells by parsing the 
short form cell names by splitting the names by ".". This precludes having "."
in the name of any covariate level. No other functions have this restriction.
The new plot.gGeneric extracts the cells from x$predictors rather than parsing
concatenated cell names.

* SE figures have been amended to correct several problems:
-- labels for cellwise plots showing summaries of data and fitted models have
   been corrected (under certain combinations of selected p and k formulas 
   with 2 predictors, some of the labels were incorrect)
-- illegibly small font sizes have been increased
-- figures produced from command-line using plot.pkm have been prettified
-- labels for cellwise p and k summaries and summaries of data with fitted 
   values were changed from cell names in the panel corners to margin labels
   that show both the predictor name and the levels
-- the superfluous "app" arg in plot.pkm and plot.pkmSet has been removed
-- functions plotSEFigure, pkmParamPlot, pkmSECellPlot, plotSEHeader, 
   plotSEBoxPlots, pkmSetSpecParamPlot, plotSEBoxTemplate, plotSECells, 
   pkmSetSpecSECellPlot, and SEcols have been replaced with a modified version
   of plotSEFigure, which calls two new functions (SEpanel and SEboxes) that
   are not exported but called by plot.pkm and plot.pkmSet

* added pda & pdb and location & scale to the download file for CP (GUI)

* in the $cell_ab table of descriptive stastistics for CP models with
loglogistic or Weibull distributions, the lower and upper bounds of the CIs
for the pda parameter were transposed (in cpm objects on the command-line). 
The error has been corrected.
* an internal database (sysdata.rda) of parameters used by the GUI has been 
created to store, e.g., default colors and other graphics settings.

* quantiles of the marginal distributions of p_hat and k_hat were calculated 
by simulation, which is slow and relatively inaccurate; v1.3.3 introduces a 
new function "qpk" to return the quantiles directly from the estimated means
and variances that were previously used to simulate data for calculating the 
quantiles empirically from the simulated data.

* functions that are only used internally are no longer exported, following 
suggestion in Wickham (2015) [http://r-pkgs.had.co.nz/namespace.html]

* An error in array indexing for the "2n fix" (added in v1.3.1) to allow 
estimation of searcher efficiency when all SE trial carcasses in a cell are 
either found or not found on the first search essentially nullified the fix
in most cases. Data sets that require the "2n fix" are not common, and the 
error would be limited to scenarios where formula_p has 1 predictor and an 
offending cell that is not the first cell in the data or formula_p has is
the interaction model with an offending cell that is not the first cell in
the data and the GenEst version is v1.3.1 or v1.3.2. Earlier versions would 
throw an error and abort the calculation rather than apply the "2n fix". The
error has been corrected in development version v1.3.3.1 and all subsequent
production versions (>=1.3.3). 

2019.09.13
v1.3.2

* correction of bug that incorrectly caused a pk model error when all carcasses
were either found or not found on the first search in SE field trials with no
covariates (GUI and command line). 

* correction of bug in error-checking for carcass ID column selection (GUI).
GUI users may notice better software stability in the "Estimate" button under 
the "Mortality Estimation" tab.

* removed dependency on cbinom and removed supporting functions for cbinom 
because we no longer use cbinom

* changed output table for CP model summary to show point estimates and CIs for
medianCP and r statistics rather than showing point estimates and CIs for 
location and scale parameters and just point estimates for medianCP and r 
statistics (GUI)

* added a function [desc(model_CP,...)] to calculate medianCP and r statistics 
(with CIs) for user-specified intervals (command-line). This function is used 
internally in the GUI for producing display tables for CP models.

* simplified the code flow for writing CP summary tables:
(1) eliminated the superfluous data structures rv$modTabPretty_CP and 
rv$modTabDL_CP, instead storing the table of summary statistics for a selected 
CP model in rv$modTab_CP. Previously, there were three extremely similar sets of 
data stored under different names. Now there is only one.
(2) create "pretty" data table for GUI rendering in one function call and only
when needed

* removed the function "renderDTns", which simply called an existing function 
[renderDataTable(...)] with arg "server = FALSE" and was superfluous.

* added DT:: prefixes to calls to renderDataTable to disambiguate the functions
in 'shiny' and 'DT'

* other superfluous functions were removed: trueLength, isNeverDecreasing, 
Ecbinom, cbmean, DWPbyCarcass

* the .rda was removed because it had data strictly for processing cbinom bias
corrections

* corrected an error that led to omission of the $n column in the $cell_ab table
in cpm objects that were fit with scale parameter modeled as an additive
function of two predictors or a function of more than two predictors. 

* changed the display and help name for "size" to "class" to reflect broader
interpretation of carcass types beyond just size. In the underlying code, the 
word "size" is preserved to maintain robustness and to be compatible with legacy
code that command-line users may have developed for analyses or explorations.

* changed pk model summary statistics output table to show only 3 digits instead
of 6 (GUI).

* moved the cell names from the first column of the descriptive statistics for 
CP model into the row names instead. This makes it easier to work with the data
sets because they are all numeric rather than being a mix of names and numeric.
modes

2019.08.22
v1.3.1

* adjustment to pk models to allow fitting in the uncommon scenarios where all
field trial carcasses are found are are missed in a cell. Data from offending
cells are duplicated but with one carcass having opposite fate from the others.
Result is that p_hat in the offending cell = 1/(2n), with variance structure
that closely approximates a Bayesian posterior of p based on p ~ beta with a 
uniform prior. The revised pk model is identical to the previous version except
that it as adapted to fitting a wider array of data scenarios.

* simple correction to reset kFixed to null when a pk model is cleared (GUI).

* replaced the bias-corrected cbinom(x/g, g) bootstrap for xtilde with the more 
efficient bias-corrected binom(round(x/g), g) bootstrap. Users are unlikely to
notice any differences in results but will notice faster computation times
for the estimating M.

2019.07.17
v1.3.0

* more informative data downloads with more useful default file names (GUI)

* option to plot multiple 'splits' graphics panels on common y-axis scale (GUI)

* unique carcass ID used as row names in the Mhat element of estM objects. 
This will help facilitate the performing of complicated, custom mortality splits
from the command-line.

* new placeholder functions for accommodating the incorporation of a DWP 
module sometime in the not-to-distant future. Currently, the only function
available is a null model that simply reads DWP data that was generated outside
GenEst. The models (including the null model) can accomodate uncertainties
in the estimation of DWP.

2019.04.12
v1.2.4

* an error in extracting data from splits arrays for Mhat resulted in boxplots
that were constructed from plot.splitSummary (or the "Split Estimate" button in
the GUI) having lower tails that were too long and having all other quantiles 
shifted slightly downward. The error has been corrected. Users may notice the
change in the lengths of the lower tails in the splits plots.

* corrected the lower bound of the CIs shown on the histogram for Mhat 
estimates so that it cannot exceed the number of carcasses observed after the
first search

* minor editing of some GUI labels and messages to improve readability

2019.03.07
v1.2.3

* added a error check to return a more meaningful message when there are 
mismatches between covariate levels in CO and SE or CP. 
Previously: "subscript out of bounds"
Now: The offending data formatting error is identified and referenced 
specifically and clearly in an error message.

* extended the KM lines on the CP figs in cases where there are many
right-censored data points.

* an error in loop indexing in the error-checking routine for identifying the
"unit" column inadvertently required that in cases where there were two or more
column names in common between the CO and DWP files, the unit column in the CO
file had to be one of the first few columns; otherwise, an error message was 
generated and the mortality calculation could not be completed. The error has
been corrected.

None of the changes affect analyses or results but all will improve user 
experience with the software.

2019.02.04
v1.2.2

* added a note for Windows users about downloading example data sets.

* with data sets with only 1 unit, prepSS was not properly assigning unit
names to rows of searches_unit array. This caused an error in prepSS and its
dependencies (estM and estg), which was expressed in the error message: 
"no 'dimnames' attribute for array". This has been corrected.

* corrected the download button for figures showing transposed mortality 
splits (previously, the untransposed figs were downloaded)

* corrected an error in calculating average SS under certain conditions,
the error affected default values in a convenience funtion and had
zero potential for producing erroneous or misleading results but would 
require users to enter specific values into a input box for which reasonable
default values should have been made available.

2018.12.09
v1.2.1

* changed the conditional run_M and run_g buttons to use output.filename_SS
rather than output.data_SS as a check. The check on output.data_SS required
users who uploaded example data into the GUI to click on the "Search Schedule"
panel before the run_M and run_g buttons could be activated. This change
relieves an inconvenience that GUI users would have when analyzing the example
data sets (GUI).

* corrected a check on data structure that was preventing estimation of 
mortality in some very simple data sets, namely, data with no carcass size
distinctions, only one search occasion represented in the SE data, and using 
a fixed k parameter (GUI). 

* fixed a bug that was preventing mortality estimation in cases where there 
was more than one size class and more than one estimate of mortality was to 
be made in succession (GUI). 

* streamlined the interface under the "Detection Probability" tab by removing 
the requirement that, after entering new search schedule parameters, users must
calculate the new search schedule derived from the parameters before clicking 
"Estimate" to get the new g's based on the new search schedule. Also, added the
search schedule to the results table's caption to make it clear which search 
parameters the g_hat's are associated with (GUI). 

* the default search schedule that is entered into the numericInput boxes under
the "Detection Probability" calculation tab is now NULL when no data_SS is 
loaded and averageSS when a data_SS is loaded. The new defaults are a better 
representation (averageSS) of the uploaded data than are the previous
one-size-fits-all parameters of I = 7, span = 364. However, the "Detection 
Probability" tab SS parameters are not used in mortality estimation and are 
strictly a user convenience for data exploration and can be readily changed 
with no downstream consequences on results outside the specialized "Detection
Probability" page (GUI).

* two-dimensional transposeSplits was assigning some splits estimates to the 
wrong splits levels because of an array ordering issue. This has been corrected
by initial ordering of CO splits in calcSplits. (command line and GUI)

* added additional error-checks for failure to select CP and SE models for all
size classes. Previously, if models were selected for the first size class but
no others, the "Estimate" buttons were activated for both M and genericg, but
clicking them would crash GenEst (gray screen). Now, the buttons are still 
activated, but a new error-check catches the problem and displays a useful
error message in lieu of crashing. (GUI)

* added code to convert dataframe "factors" to "characters" prior to performing
calculations in pkm0, cpm0, estM, estg, 

* figure downloaded for estimated mortality corrected to properly reflect user's
custom-selected CL

2018.12.06
v1.2.0

* updated the .zip download buttons to no longer require an unzip utility to
be in the path statement in Windows (using zip::zip rather than utils::zip). 

* more robust error-checking and more informative error messages. Changes
affect especially the functions under the "Mortality Estimation" in the GUI, 
which have the most demanding data formatting requirements.

* corrected the mortality splits results summary table to sort mortality
quantiles numerically rather than alphabetically.

* removed the requirement that GUI users both enter a value for fixedK and
check a box indicating that a value is entered. User simply enters a value
for fixedK or leaves the field blank, whichever is appropriate for the 
desired analysis.

2018.11.23
v1.1.3

* improved error-checking and error messages for auto-parsing of CO, SS, and
DWP files to find unit column when not provided by user in arg list to
estg or estM. The error-checking is extracted into a short function that is 
called by estg and estM when necessary.

2018.11.22
v1.1.2

* example data sets can now be loaded directly into the GUI from buttons on
the (newly named "Example Data" tab...formerly "Downloads") rather than
requiring that they be first downloaded as .csv files and then uploaded
one by one into the GUI. (The data sets can still be downloaded for use
as input file templates or other purposes if desired.)

* improved error-checking and error messages for estM function

* export the more informative estM error messages from command line function
to the GUI

* updated links to User Guide and technical manual to point to doi's rather
than pdf's

2018.11.19
v1.1.1

* estg: error-check added for carcasses discovered before searches begin

* estM: error-check added for CO units not included in DWP

* cpm0: sample size was missing from cp model tables when s parameter has an
additive model (e.g., s ~ A + B). Fixed

2018.11.14
v1.1.0

* added links to published GenEst User Guide and GenEst Statistical Models

* corrected a GUI bug that prevented mortality estimation when CO and DWP files
had more than one column name in common

* corrected a GUI bug that caused intermittent failure to update some results
figs when user selected new models. The bug was caused by figs sometimes not
being properly suspended when hidden.

* corrected a bug that caused ppersist to return NaN or 0 for Weibull 
distributed persistence times when the variance of the scale parameter was very
large. The error occured when all CP field trial carcasses were removed in the
span of two intervals. Improved error-checking and an updated calculation of 
Weibull ppersist for data that generates fits with very large uncertainty in
scale parameter.

* corrected a GUI bug that erroneously allowed the drop-down selection menus for
mortality splits to permit more than two covariates to be selected in certain
cases where an improper model was first entered, later corrected, and then a 
third covariate selected from the temporal splits menu. Results figures and 
tables would show the proper covariates, but they would not be properly 
reflected in the drop-downs. This was due to an error in the conditionals for
the drop-down lists not being properly updated. It had no effect on calculations
or results.

* GUI code was revised for readability and easier maintenance: some groups of
very small functions were consolidated into larger functions, overly long 
variable names were shortened, some groups of short files were consolidated into
longer files. 

* Documentation for GUI code was updated to facilitate code maintenance.