PESTPP-GLM does not expand list of tested lambdas and does not use lambda_scale_facs over 1

[mmorphew]

The following is a snippet from the PEST++ manual:

"In contrast to PEST, PESTPP-GLM does not use the control variables specified on the fifth line of the “control data” section of the PEST control file to govern how it chooses Marquardt lambdas. Instead, it receives Marquardt lambda control information from PEST++ control variables. Two of these are lambdas() and lambda_scale_fac(). Default settings for these variables are as follows:

lambdas(0.1, 1, 10,100,100)

lambda_scale_fac(0.9, 0.8, 0.7, 0.5)

As is apparent, more than one value can be associated with each of these control variables. Values supplied for lambdas() specify the Marquardt lambdas that PESTPP-GLM must use during each iteration of the inversion process. Then, if values are supplied for lambda_scale_fac(), PESTPP-GLM calculates multiple upgraded parameter sets for each Marquardt lambda. These sets are those that correspond to various fractional lengths along the upgrade vector that is calculated on the basis of the Marquardt lambda alone, the latter corresponding to a lambda_scale_fac() value of 1.0. Note that parameter change limits and parameter bounds are not enforced on scale factors greater than 1.0.

If only a single value is supplied for lambdas(), then this value of lambda is used in all iterations of the inversion process. In contrast, if more than one value is supplied for lambdas(), then PESTPP-GLM expands the user-supplied lambda list over the course of the inversion process. The expanded list includes other lambda values that are above and/or below those already comprising the list. A lambda is added to the list if the best parameters obtained during any particular iteration of the inversion process were calculated using an end member of the current list. The value chosen for a new lambda ensures that the previous best lambda is bracketed by members of the expanded list."

As far as I can tell from looking at the source code (and from my own experience using PEST++GLM), this section is only partially accurate. Namely, there does not appear to be any code for expanding the user-supplied lambda list over the course of the inversion process. lambda_scale_facs over 1.0 also appear to be ignored entirely (from SVDSolver.cpp line 1138-), rather than ignoring parameter change limits and parameter bounds.

for (double i_scale : lambda_scale_vec) { if (i_scale >= 1.0) continue;

I bring this issue up because A) not expanding the list of tested lambda values can really hamper GLM's performance and B) I've been looking into implementing a dynamic lambda_vec construction process, and these issues would be easy to correct/modify in the process of doing that. However, before doing that, I wanted to better understand the team's philosophy/preference behind the current implementation in the code vs. what's in the manual. What behavior is intended vs. unintentional, and would there be an objection to submitting a PR to change this behavior?

[jtwhite79]

I think the expanded lambda search concept was removed long ago for reasons I cant remember and I agree that expanding/tuning/optimizing the lambda values dynamically during a run can be very beneficial. So can "extending" the upgrade vector with a scale factor greater than 1.0.

pestpp-ies tries to tune the lambda search as it goes, but Im sure there are more clever and efficient things that can be done...happy for any and all attempts to improve pestpp-glm!