[WIP] COBYLA solver: Track best-performing parameters during optimization#1240
[WIP] COBYLA solver: Track best-performing parameters during optimization#1240carolinafernandezp wants to merge 3 commits intojonescompneurolab:masterfrom
Conversation
…ed objective value rather than the final iterate (COBYLA solver)
|
Thanks Carolina, is this ready to go or is it still a work-in-progress? If it's ready to go, is it okay if I push some commits in order to fix the tests breaking? We just pushed a commit to master yesterday that fixes an issue with our circleci step failing (1fb6074 ). I can add it to your branch myself if that's okay with you (along with other commits that will fix unit test breakage). |
Yes that sounds great, thank you @asoplata! |
|
@carolinafernandezp Is this ready for review / merge? Or is it still a work in progress? |
Currently, both the script and notebook versions of our example file `examples/workflows/plot_simulate_somato.py` fail because of an upstream breakage in how the MNE sample data is downloaded and unpacked (see here fatiando/pooch#518 This change fixes the issue by "capping" the Pooch version to a prior version which does not contain the bug.
|
This appears to be a WIP work in progress, since there are legitimate test failures occurring |
asoplata
changed the title
|
Hey @carolinafernandezp I've made a pull request to your branch on your fork here carolinafernandezp#1 that should fix the failing test, see that for details |
2 participants
The COBYLA algorithm implemented by SciPy returns the set of parameters that attempts to minimize the objective function at termination, instead of the argmin over all evaluated parameter sets. This means that, currently,
optim.net_whenoptim = Optimizer(net, solver='cobyla')does not return the optimized network, but the network that was last explored.The fix I'm implementing here is to track the best parameter set and corresponding objective value over all evaluated parameter sets. In other words, the proposed fix is to store and return the parameter set corresponding to the lowest observed objective value rather than the final iterate for the COBYLA solver.
We already do this with the Bayesian optimization function we implemented in externals, which is why we don't run into the same issue with the bayesian solver.