configurable Constraints for each sample

[SimonHerter]

Hi,
i'm currently using Botorch to optimize my simulations, therefore i need to constraint my parameter space. I'm wondering if it's possible to adapt the coefficient passed to the constraints based on each sample. So i would like to dynamicly adjust the coefficients based on the actual candidates created.
For example i would like to pass something like this as constraints:
inequality_constraints = [(torch.tensor([0,1,2,3,4,5]), torch.tensor([Variable]6), 0.0)]
and Variable=7X[0]+6*X[1]*X[2]X[3]+8X[4]+X[5]**2 (for my problem this is complex Function determining if all boundary conditions are met, but such a function is illustrating my problem well)
In this example X is the actual candidate created, by the aquisition function. Is something like this possible?
Or is there a work around?
If it's not possible to constraint the Optimizer in such a way, is it possible to use outcome constraint? I was thinking of asigning each candidates of parameters violating my constraints a certain value and then passing this values as an outcome constraint.
kind regards,
Simon

[Balandat]

We currently don't have support for nonlinear inequality constraints like these built out, but it should not be too hard to pass them into the optimizer and have scipy deal with them. Essentially, what one would do is to, in addition to equality and inequality constraints, also allow other generic callables as an input to https://github.com/pytorch/botorch/blob/main/botorch/optim/optimize.py#L40, and then properly pass it through the stack to gen_candidates_scipy and ultimately scipy.optimize.minimize in https://github.com/pytorch/botorch/blob/a3f8aa98abf55148a18b8f6971137d43d6fb8a9e/botorch/generation/gen.py#L166.

We'd be happy to accept a PR for this. A couple of words of warning though:

1. This can make the optimization significantly harder, so generation might take much longer.
2. If the constraint is such that the feasible domain is disconnected / has little local volume around feasible points, then modeling with a GP might become problematic.

[SimonHerter]

Hi,
thank you a lot for the quick response. I will try to implement you're suggested method. Do you have any thoughts on the about using outcome constraints as a solution for my problem?
I'm thinking about asigning the objective a certain value e.g. 0 if the 'complex parameter' constraints are violated. As all feasible samples show an hihger objective, i could use linear outcome constraints, but i'm not sure if this is an appropriate way to perform Bayesian Optimization as i'm new to this method.
kind regards
Simon

[Balandat]

Black box constraints are not assumed to be linear, they are just that: black boxes. You could use outcome constraints to express your parameter constraints and hook things up they way you suggested. In principle that should work. However, by doing so you'd be leaving a lot of information (namely the explicit form of the constraint) on the table; the constraint model would have to learn that from scratch. You'd likely get worse results with higher computational overhead than if you used the parameter constraints directly.