Random failure for CREST pre-run optimizations

[chertianser]

Describe the bug
crestopt.log is sometimes empty when starting a run with crest.

 -----------------------------
 Initial Geometry Optimization
 -----------------------------

  Initial geometry optimization failed!
  Please check your input and, if present, crestopt.log.
 CREST terminated with failures.

It's sometimes possible to repeatedly run the same calculation over and over again with the same flags on a clean directory and the calculation will work. When the initial optimization succeeds with crest (rc_A_crestopt.xyz), the structure matches up very closely with the one optimized separately from GFN2-xTB (rc_A_xtbopt.xyz, RMSD = 0.00366687). I succeeded at least once running crest with rc_A.xyz, but have not been able to get rc_B.xyz to converge the pre-optimization with crest. If I optimize rc_B.xyz separately with xtb (rc_B_xtbopt.xyz), that structure will proceed with crest.

I suspect this to be an issue with tblite, as rc_A.xyz works with the --legacy CLI flag.

To Reproduce

crest struc.xyz --chrg -1

rc_A.xyz

13
rc_A.xyz
  C          -0.00832751238325      0.01417029172866     -0.37717680515120
  C          -0.06386907941367      0.16909265481552      1.10613620102408
  F           1.28577057663188     -0.16033746560355     -0.88204609219719
  H          -2.45803807638385     -0.56953060445928      1.97113471782028
  H          -1.16149079270809      0.24165215775680      1.37560949293040
  H          -0.62131255127982     -0.80144807414511     -0.74815975684843
  H           0.45687673981754      1.08266358937492      1.39732016225023
  N           0.38600408938011     -0.99938487351295      1.84525550895292
  H           0.99183657438869     -1.60654015344020      1.31007803903066
  H          -0.45520898142410     -1.50779053038184      2.13190349348799
  N          -0.55405813724919      1.26806383991956     -1.02959842655361
  O           0.05403235428282      2.31805648080460     -0.84679973450201
  O          -1.60250020929174      1.15567489483862     -1.65143900593181

rc_A_crestopt.xyz

  13
 Etot=         -26.1188257789
 C          0.2568634786       -0.0673194118       -0.2993925561
 C          1.2168325170       -0.6393616685        0.7572749878
 F          0.4868288232        1.2760668603       -0.4843850441
 H          1.3239837293       -1.3790505073       -2.2846538541
 H          0.8068964029       -1.6034629952        1.0692788280
 H          0.5367756223       -0.6092585847       -1.2880391634
 H          1.2478646766        0.0391780549        1.6182547168
 N          2.5626397485       -0.8438211299        0.2697432879
 H          2.9481887303        0.0549594609       -0.0130311558
 H          2.4898021078       -1.3898293745       -0.5909988034
 N         -1.1225762936       -0.2591446564        0.0035718981
 O         -1.9201947006        0.6449982260       -0.2423816714
 O         -1.4581118929       -1.3603402089        0.4517890090

rc_A_xtbopt.xyz

13
 energy: -26.118825561467 gnorm: 0.000926138693 xtb: 6.7.1 (edcfbbe)
C           -0.36890203957863        0.06839474529215       -0.21799718606839
C            0.43319284485530        0.19393619810669        1.08827975028258
F            0.21852755328549       -0.84765709484228       -1.06051383475186
H           -2.44146095695492       -1.03635933756095        0.91217568539925
H            0.10150744776697        1.11443938591143        1.57556400639366
H           -1.40107810705499       -0.36117883023981        0.10079874536998
H            1.50030798593510        0.27281360284300        0.84829931096320
N            0.23555611297315       -0.91043519685418        2.00022939770942
H            0.54747458102886       -1.76762995889812        1.54864979025030
H           -0.77284947978932       -1.02007997392603        2.12396539313847
N           -0.53544894039030        1.29681308927106       -0.91938572142976
O           -0.46694889283732        1.29975735779472       -2.14759843286175
O           -0.80016311486998        2.30152822079822       -0.25024911008300

rc_B.xyz

15
rc_B.xyz
  C           0.18098243175343     -0.03259072815111      0.04119792485350
  H           1.71244186062529      1.42863545141993      0.35922752584614
  F          -1.17566977163396     -0.06596579946779      0.43536703647111
  H           2.82615602324739     -0.44168405796081     -0.35717408159295
  C           0.65943631766357      1.43842116697278      0.05106757680634
  H           0.76568587016038     -0.57950042387475      0.78193968550339
  N           0.22023423050839     -0.60122832572799     -1.27804226813380
  H          -0.38063132206932     -1.41933108267002     -1.26537463600169
  H           1.19801357036353     -0.89438636033470     -1.41787980432207
  N          -0.07588749114977      2.11013326099781      1.23621529630402
  O          -0.80393222653200      3.07279622234350      1.01537851354470
  O           0.16661905792617      1.66917617915922      2.34902141316331
  N           0.44088276311204      2.12359693826971     -1.17516785501051
  H           0.05520792355347      3.04309825809766     -1.01946287983142
  H          -0.15866654429107      1.57306792204982     -1.77616771093345

rc_B_xtbopt.xyz

15
 energy: -29.551152110860 gnorm: 0.000565544720 xtb: 6.7.1 (edcfbbe)
C           -0.11681285209301       -0.11379726041925       -0.06225075402819
H            1.59350384605907        1.11286732405908       -0.08628243195221
F           -1.50466939646238       -0.03650113192665       -0.32279599524991
H            2.88400511517798        0.38750185853898       -0.40194938610477
C            0.45698791677834        1.30015004107207       -0.04850031189155
H           -0.06695224028354       -0.52635112564010        0.96052118356449
N            0.47594914838597       -0.90864769134538       -1.07819576912660
H            0.49141864635006       -1.88834648090736       -0.82255806977351
H            1.43464346164401       -0.58886595292615       -1.21604375419343
N            0.18015566009900        1.97608800655589        1.20713943512412
O           -0.03235823574102        3.19074814471329        1.16056131233719
O            0.25558064955968        1.34540135774954        2.25935478345236
N            0.08458633484646        2.06661481596572       -1.19498195555755
H            0.42108250742319        3.01738059445740       -1.08196945992868
H           -0.92624786850660        2.08999612117527       -1.29190309000366

Expected behavior
I expected the initial pre-optimization to return some optimization trace in crestopt.log for rc_A.xyz and rc_B.xyz.

[pprcht]

have you checked with --opt or --sp even to see if the SCF (from tblite) may have troubles?

[pprcht]

Hm yeah, the SCC runs into the max cycles (for rc_A.xyz). If the charge is truly -1, that is. Both rc_A.xyz and rc_B.xyz look more like +1 charge to me, but maybe you are trying to simulate something else?

[chertianser]

I will get back to you on --opt and --sp, but the charge is meant to be -1 (simulating a hydride ion).

[pprcht]

Ok, I see. Yeah, it does seem that tblite is unhappy with the hydride. with --sp you can see that it runs into the limit of 500 SCC cycles.

Now, it seems to be fixable by initializing the tblite calculation from a CEH density guess. Unfortunately that implementation is not yet in the CREST main branch. I'm prepping it for 3.0.3.
If you'd like to have an experimental version tough, you can find one here: https://github.com/pprcht/crest/tree/3.0.3-maintenance (only the gfortran/CMake files are set up there at the moment). The cli flag for CEH initialization is --ceh_guess and it does that automatically before starting the GFN2-xTB calculation via tblite. On the plus side, the problem from #458 also seems fixable with that.

[chertianser]

I can confirm the 3.0.3-maintenance build works for the calculations mentioned. Is there an intention to build a rescue procedure for the failed optimizations, or raising some kind of exception to suggest that the user uses --ceh_guess for an error like this given the current printout is not verbose in the reason why it fails (given crestopt.log is empty)?

[pprcht]

Thanks for testing, yes, that is a good idea. As the build is very much work-in-progress, implementing such fallback should be very doable. I'll write it down as a TODO.