Botorch with cardinality constraint via sampling #301
Conversation
Waschenbacher
requested review from
Scienfitz,
AdrianSosic and
AVHopp
as code owners
Waschenbacher
force-pushed
the
feature/cardinality_constraint_to_botorch_via_sampling
branch
from
0be3285
to
1f7783d
Compare
There was a problem hiding this comment.
Hi @Waschenbacher, thanks for the work. This is not yet a review but only a first batch of very high-level comments that I would like you to address before we can go into the actual review process. The reason being that the main functionality brought by this PR is currently rather convoluted and hard to parse, so I'd prefer to to work with a more readable version, tbh
Waschenbacher
force-pushed
the
feature/cardinality_constraint_to_botorch_via_sampling
branch
from
1f7783d
to
9d28b49
Compare
AdrianSosic
force-pushed
the
feature/cardinality_constraint_to_botorch_via_sampling
branch
from
f68ce4f
to
fc52875
Compare
There was a problem hiding this comment.
Hi @Waschenbacher, I've finally managed to spend some time on this important PR, thanks again for your preparation work. I've already refactored some parts that were clear to me. However, I'm not yet certain about the design at the remaining places. I have the feeling that it can potentially be simplified a lot, depending on whether it is possible to reuse the idea of reduced subspaces. Have marked the corresponding places with comments. I think we need to discuss this part first before we can continue.
baybe/searchspace/continuous.py
Outdated
| # TODO: Revise function name/docstring and arguments. In particular: why | ||
| # does the function expect the inactive parameters instead of the active ones? | ||
|
|
||
| # TODO: Shouldn't the x != 0 constraints be applied on the level of the | ||
| # individual constrains, also taking into account whether min_cardinality > 0? | ||
|
|
||
| # TODO: Instead of adding additional constraints, why not alter the parameter | ||
| # bounds? In case we keep the constraints: is the sign of the threshold | ||
| # correct? |
There was a problem hiding this comment.
Here some points for discussion
There was a problem hiding this comment.
I made the following changes:
- Ensure active parameters != 0 by altering the parameter bounds
- By introducing
_FixedNumericalContinuousParameterclass, we ensure inactive parameters = 0 and active parameters != 0 in the copied subspace object. This is one way of communicating the subspace with inactive parameters. As discussed, we could also go with the option of the reduced subspace, then we need to manipulated the_botorch_acqf.
| acqf_values = torch.max(torch.cat(acqf_values_all)) | ||
| return points, acqf_values | ||
|
|
||
| def _recommend_continuous_without_cardinality_constraints( |
There was a problem hiding this comment.
I find the design here a bit weird. The method is called ..._without_cardinality_constraint but somehow still takes care of inactive parameters internally. My general question here is: instead of fixing inactive values to 0.0, can't we apply the same logic of creating a reduced space instead? If that was possible, I think it would also drastically reduce the amount necessary code
There was a problem hiding this comment.
I believe design in the revised version is not that misleading. Regarding the inactive parameters, please see my reply above #301 (comment)
… parameter setting
Waschenbacher
force-pushed
the
feature/cardinality_constraint_to_botorch_via_sampling
branch
from
e9d0048
to
120d717
Compare
Waschenbacher
force-pushed
the
feature/cardinality_constraint_to_botorch_via_sampling
branch
from
2767dd5
to
e6248b5
Compare
This PR attempts to support cardinality constraints in botorch recommender. The core idea is
There are two mechanisms for determining the configuration of inactive parameters.
In addition, a validation is added to ensure that zero is included in the bounds of all cardinality constraint parameters.