Introduction of objective constraints in botorch - unclear implementation and runtime issues

[MoBurmeister]

Hello together,

thanks for providing and maintaining the botorch library!

I am having some issues with the introduction of objective/outcome constraints in multi-objective optimization.
Right now I have two targets to optimize, where one target (a temperature) should be maximized but should not exceed a specific temperature limit.
I looked into the guidance of introducing outcome constraints, but I am not sure if I am getting it right.

I am using the qNoisyExpectedHypervolumeImprovement acquisition function, where I am trying to introduce the constraints.

acq_function = qNoisyExpectedHypervolumeImprovement(model=self.multiobjective_model.gp_model, ref_point=self.reference_point, X_baseline=self.multiobjective_model.dataset_manager.initial_dataset.input_data, constraints=self.output_constraints, prune_baseline=True)

my constraint function looks like this:

def constraint_func(outputs: torch.Tensor) -> torch.Tensor: second_dimension = outputs[..., 1] constraint_values = second_dimension - 2000.0 return constraint_values

After introducing this, I observe the following:

• the optimization slows down significantly and now takes a really long time
• there are still parameter combinations that result in exceeding the the constraint by a significant amount (not slightly exceeding it, e.g. reaching 4000 or more, so more then doubling it)

So I am not sure if I introduced the constraint correctly and what is causing these huge runtime increases.
Also I was wondering what generally the best way is to introduce such constraints. I also thought about introducing it as an additional objective. Is there something else I need to consider or pay attention to?

Furthermore, I was how I should adjust the optimize_acqf values:
candidate, acq_value = optimize_acqf( acq_function=acq_function, bounds=self.multiobjective_model.dataset_manager.initial_dataset.bounds_list, q=1, num_restarts=40, raw_samples=512, )

Should I tune them down when implementing constraints?

Many thanks in advance for any help or comments!

[Balandat]

one target (a temperature) should be maximized but should not exceed a specific temperature limit.

Interesting; this is a somewhat non-standard setting. Typically, we assume that if an objective is maximized, we don't "cap" it.

The proper way to incorporate this into the multi-objective optimization acquisition functions themselves, in this case qNoisyExpectedHypervolumeImprovement. At this point Hypervolume-based acquisition functions don't support this, their ref_point argument https://github.com/pytorch/botorch/blob/e44280e32f64de9c8deed8ca989c258410ac423d/botorch/acquisition/multi_objective/monte_carlo.py#L331 essentially assumes that there is only a one-sided lower threshold for each of the objective). It should be possible to incorporate this into the algorithm, essentially just by extending things to a "reference range" rather than a reference point (cc @sdaulton for his thoughts here). However, that would likely be a relatively large effort to change the algorithms and incorporate this capability into the signature of all acquisition functions.

A workaround could be for you to, rather than maximize temperature up to some upper threshold t_max, simply maximize a transformed objective, e.g. -(t_max - t)**2. The optimum then is t_max and the the higher t is the better (for t < t_max). That would allow you to avoid using an outcome constraint.

the optimization slows down significantly and now takes a really long time

It is expected for the optimization to take quite a bit longer with your setup. Each outcome constraint will be modeled by an additional GP model, and the optimization problem also becomes more challenging. You can avoid these issues with the workaround I mentioned above.

there are still parameter combinations that result in exceeding the the constraint by a significant amount (not slightly exceeding it, e.g. reaching 4000 or more, so more then doubling it)

That the constraint is exceeded is expected somewhat frequently is expected (since it's modeled as a black-box constraint). By how much it exceeds the constraint depends somewhat on how uncertain the model is about certain regions. The constrained optimization works via probability-weighting the expected HV improvement, so you have sth like EHVI(t_obj, other_param) * Prob(t_con < 2000), but the acquisition function doesn't know that t_obj is in fact the same as t_con (both are modeled independently). So, likely what is happening is that the acquisition function is ending up with high EHVI values but low Prob values for high t, so that the product is still comparable with cases where EHVI is low but Prob is high for more reasonable values of t. This is simply not the right formulation for the problem. The workaround above should also avoid this.

cc @SebastianAment

[SebastianAment]

@MoBurmeister Thank you for reaching out. Additional observations and suggestions here:

there are still parameter combinations that result in exceeding the the constraint by a significant amount (not slightly exceeding it, e.g. reaching 4000 or more, so more then doubling it)

This is most likely primarily caused by either

1. Bad extrapolations of the constraint model (i.e. the objective model), or
2. poor acquisition function optimization, which could leave the candidates stuck in a region that is predicted to be infeasible.

You could analyze if the constraint model predicts the candidates returned by optimize_acqf to be likely feasible or not, in order to differentiate the two causes, or determine that both are present.

To ameliorate 1): You could improve the model either with a better prior mean (e.g. you could put a monotonic / linear function with a positive slope here if you know a-priori that certain inputs should generally increase the temperature. Otherwise, you could add a LinearKernel to the covariance kernel you are currently using, if you believe that this type of overall monotonic structure is present, but you don't have a good initial guess as to what the linear coefficients should be.

In any case, and to ameliorate 2), we generally and strongly recommend replacing qNoisyExpectedHypervolumeImprovement with qLogNoisyExpectedHypervolumeImprovement, which fixes the vanishing gradient problems that qNEHVI encounters with or without constraints. This is a drop-in replacement with the same API, just adding "Log" in the name. See our NeurIPS article for details.

Let me know if this helps or you have further thoughts or questions.

[MoBurmeister]

Thank you very much for your detailed responses!

It is indeed a non-typical setup and after trying around a bit the best solution is probably to change the objective into the minimization of the temperature difference. This worked out quite fine for me

Thanks for explaining how the constraints internally work, I will consider this in the future.
Also thanks for pointing out to use qLogNoisyExpectedHypervolumeImprovement as an acquisition function.