Define expected improvement for regression problems

[adowling2]

Action items:

• Derive formula
• Prototype formula in Python file
• Check if this formula has been derived elsewhere in literature, compare

[adowling2]

Main reference:
https://stats.stackexchange.com/questions/426782/intuitive-understanding-of-expected-improvement-for-gaussian-process

[adowling2]

My work from today:

Normal distribution PDF:
https://en.wikipedia.org/wiki/Normal_distribution

Integral for final term:
https://www.wolframalpha.com/input/?i=integrate+x%5E2%2F%28%5Csqrt%282+pi%29%29+e%5E%7B-x%5E2%2F2%7D

[adowling2]

@nil23121994 I derived the formula. As a next step, I'll prototype this in Python for you.

@bbefort This will be helpful for Bayesian optimization for forcefield tuning. Happy to explain more today.

[NilaySystemsBio]

Thank you for sharing this. I was looking into the portion of math where you estimate the bounds of integral eps_1 and eps_2. The quadratic equation you defined over epsilon cancels out to 0 as shown in above calculations indicating that the quation has infinite slutions. Am I doing something wrong when I am substituting e* with the term you defined? What do you think @adowling2 ?

[adowling2]

@nil23121994 I got something different.

Here e^star is the best least squared error observed so far. Unless the simulation matches the target exactly, e^star> 0 . This formula makes sense, because as e^star goes to zero 0 ), the integration bounds get tighter and the expected improvement gets smaller.

[adowling2]

@nil23121994 Is this enough for you to move forward? I recommend creating a simple .py file to calculate the EI for a given mu, sigma, f^bar, and f^{*}.

[adowling2]

This paper uses Kriging (Gaussian process) regression for model calibration. It discusses the expected improvement of SSE, although I think our approach is more straightforward:
https://aiche.onlinelibrary.wiley.com/doi/full/10.1002/aic.15352

[NilaySystemsBio]

@adowling2 Yes, it looks good to me. I will try to code the formulae up and share it with you for a review. Thank you!

[adowling2]

Perfect.