Callback #73

javdrher · 2017-08-22T13:27:22Z

Implementation of the final use case defined in #7 . This implements a callback strategy to plug a user defined callable in to BayesianOptimizer which is called each iteration and gives full controls over the models. All GPflow manipulations are possible (assigning priors, modifying transforms, fixing parameters). Goal of those callbacks is to assure optimizations are successful, which can be very application specific.

Combined with the optimize_restarts feature, following use-cases are possible:

By settings optimize_restarts = 0 on the objects it is even possible to do the model optimization manually, e.g. using a multi-step approaches by fixing some parameters first in a first stage.
Optimize restarts = 1 means the callback sets the initial starting point, optimization is done by the framework
Optimize restarts > 1: same, but followed by some randomized restarts.

This PR depends on #68 and #72 and should be merged after.

…control over the models between iterations. Slightly reworked MCMCAcquisition to support this scenario

codecov-io · 2017-08-22T13:27:23Z

Codecov Report

Merging #73 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff            @@
##           master     #73      +/-   ##
=========================================
+ Coverage    99.8%   99.8%   +<.01%     
=========================================
  Files          17      17              
  Lines        1013    1040      +27     
=========================================
+ Hits         1011    1038      +27     
  Misses          2       2

Impacted Files	Coverage Δ
gpflowopt/acquisition/acquisition.py	`100% <100%> (ø)`	⬆️
gpflowopt/bo.py	`98.94% <100%> (+0.28%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 10da813...339b699. Read the comment docs.

icouckuy · 2017-08-22T15:12:11Z

testing/test_optimizers.py

+                self.counter = 0
+
+            def __call__(self, models):
+                self.counter += 1


Lets think about the callback signature some more. Is there any information we want to pass that might be useful for model building?

For instance, to let the model building strategy depend on the iteration number (we can stop optimizing the hyps after a while like in the MES paper). Although we can also look at the data set size.

What about model building strategies that changes model.X en model.Y (like replace clusters etc.). Not sure if that fits here or is even relevant (the GPflow model should be able to cope with it).

I think the model contains all the data you need to accomplish something. I believe X and Y can even be updated in this callback as long as the model supports it (all models in GPflow do).

If at some point some information is really missing, this can be added.

icouckuy · 2017-08-25T11:40:36Z

GPflowOpt/acquisition/acquisition.py

        # the call to the constructor of the parent classes, will optimize acquisition, so it obtains the MLE solution.
-        super(MCMCAcquistion, self).__init__([acquisition] + copies)
+        super(MCMCAcquistion, self).__init__([acquisition]*n_slices)


Does this make deep copies? I assumed you used the old way to assure that it were deep copies

Ah I see, need_new_copies = True makes sure deep copies are made later

This version does shallow copies, its mostly to assure the copy later on is aware of the amount of copies required without serious overhead.

icouckuy · 2017-08-25T11:43:56Z

GPflowOpt/acquisition/acquisition.py

        self._sample_opt = kwargs

    def _optimize_models(self):
        # Optimize model #1
        self.operands[0]._optimize_models()

+        # Copy it again if needed due to changed free state
+        if self._needs_new_copies:
+            new_copies = [copy.deepcopy(self.operands[0]) for _ in range(len(self.operands) - 1)]


copy.deepcopy([self.operands[0]]*len(self.operands))

not tested, works too?

no, the * syntax are shallow copies so the deepcopy will copy the object they are all pointing to.

icouckuy · 2017-08-25T11:45:21Z

GPflowOpt/acquisition/acquisition.py

+
+    def _kill_autoflow(self):
+        """
+        Following the recompilation of models, the free state might have changed. This means updating the samples can


"""
Flag for recreation on next optimize.

Following the ...
"""

icouckuy · 2017-08-25T11:46:10Z

GPflowOpt/acquisition/acquisition.py

+        cause inconsistencies and errors. Flag for recreation on next optimize
+        """
+        super(MCMCAcquistion, self)._kill_autoflow()
+        self._needs_new_copies = True


I assume we cant use needs_setup for this?

_needs_setup is triggered by a simple set_data. This doesn't require new copies, only in case a callback changes the models (this should happen)

icouckuy · 2017-08-25T11:47:55Z

GPflowOpt/bo.py

+def jitchol_callback(models):
+    """
+    Default callback for BayesianOptimizer. For all GPR models, increase the likelihood variance in case of cholesky
+    faillures. This is similar to the use of jitchol in GPy


failures

"""
Increase the likelihood ...

This is similar to ... Default callback for BayesianOptimizers. Only usable with GPR models.
"""

icouckuy · 2017-08-25T11:59:45Z

GPflowOpt/bo.py

 from .pareto import non_dominated_sort


+def jitchol_callback(models):


callbacks can be in a separate callbacks.py file?

I do not plan on shipping any additional callbacks (I might even get rid of this one, it got comitted by accident but it might improve stability?) so that file would be quite empty.

Ok, I'm not in favor of including jitchol. I think there are other ways users can improve stability. First and foremost putting priors and transforms on the hyps.

Given #74 I think we should really consider this. For standard scenario's with GPRs (which is what most people will start with) I think this might give an additional automated stability support (which can be disabled by setting the callback to None)

icouckuy · 2017-08-25T12:00:30Z

GPflowOpt/bo.py

@@ -51,6 +74,12 @@ def __init__(self, domain, acquisition, optimizer=None, initial=None, scaling=Tr
            are obtained using Hamiltonian MC.
            (see `GPflow documentation <https://gpflow.readthedocs.io/en/latest//>`_ for details) for each model.
            The acquisition score is computed for each draw, and averaged.
+        :param callable callback: (optional) this function or object will be called after each evaluate, after the
+            data of all models has been updated with all models as retrieved by acquisition.models as argument without
+            the wrapping model handling any scaling . This allows custom model optimization strategies to be implemented.


if we do a separate callbacks.py file some of the explanation can be moved there + module link

icouckuy · 2017-08-25T12:01:05Z

GPflowOpt/bo.py

@@ -69,6 +98,8 @@ def __init__(self, domain, acquisition, optimizer=None, initial=None, scaling=Tr
        initial = initial or EmptyDesign(domain)
        self.set_initial(initial.generate())

+        self._iter_callback = callback


why call it iter_callback and not model_callback?

icouckuy · 2017-08-25T12:07:54Z

GPflowOpt/bo.py

@@ -86,6 +117,8 @@ def _update_model_data(self, newX, newY):
        assert self.acquisition.data[0].shape[1] == newX.shape[-1]
        assert self.acquisition.data[1].shape[1] == newY.shape[-1]
        assert newX.shape[0] == newY.shape[0]
+        if newX.size == 0:


will this ever happen? As far as I know we cant empty GPflow models so data[0] will never be empty.

this line avoids _needs_setup = True in case i.e. the EmptyDesign is configured as initial design (as is by default)

As a sidenote, as GPflow doesn't support models with no data I actually see no use case for BOptimizer having an initial design parameter.

icouckuy · 2017-08-25T12:10:58Z

GPflowOpt/bo.py

+            # If callback specified, and acquisition has the setup flag enabled (indicating an upcoming compilation,
+            # run the callback.
+            if self._iter_callback and self.acquisition._needs_setup:
+                self._iter_callback([m.wrapped for m in self.acquisition.models])


if there is no callback:

setup is run and models are optimized on the first evaluate
with a callback:

models are optimized here but setup probably has not been run yet and needs_setup is still True -> models are optimized again on first evaluate? right?

You confuse something here: you can optimize your model in the callback but this is one of the scenarios (which would require optimize_restarts to be 0 in order to avoid two optimizes). The primary use case is to only set the initial starting point.

(The reason the jitchol callback runs the optimization for a small number of steps is to check if no cholesky error occurs, not to optimize the model. )

Ok, there was indeed some confusion here. I thought the callback would implement the complete model building strategy: setting hyps, running one or more optimizations, etc. This is still possible but you have to set optimize_restarts = 0

icouckuy · 2017-08-31T16:07:53Z

gpflowopt/acquisition/acquisition.py

+        Flag for recreation on next optimize.
+
+        Following the recompilation of models, the free state might have changed. This means updating the samples can
+        cause inconsistencies and errors. Flag for recreation on next optimize


duplicate "Flag for recreation on next optimize"

icouckuy · 2017-08-31T16:08:11Z

gpflowopt/bo.py

+
+def jitchol_callback(models):
+    """
+    Increase the likelihood in case of cholesky faillures.


icouckuy · 2017-08-31T16:15:03Z

gpflowopt/bo.py

+            jitchol_callback(m.wrapped)  # pragma: no cover
+
+        if not isinstance(m, GPR):
+            continue


maybe show a warning?

icouckuy · 2017-08-31T16:46:58Z

gpflowopt/bo.py

@@ -190,6 +228,10 @@ def inverse_acquisition(x):

        # Optimization loop
        for i in range(n_iter):
+            # If callback specified, and acquisition has the setup flag enabled (indicating an upcoming compilation,


If a callback is specified,...

and close brackets :)

javdrher added 3 commits August 21, 2017 17:08

Merge branch 'lazy_setup' into hyper_callback

2fd99a6

Added support to configure a callback in BayesianOptimizer, allowing …

98aad5e

…control over the models between iterations. Slightly reworked MCMCAcquisition to support this scenario

Adjusted documentation, parameter renaming and making tests more strict

107b5c5

javdrher added the do not merge yet label Aug 22, 2017

icouckuy reviewed Aug 22, 2017

View reviewed changes

javdrher added 6 commits August 22, 2017 23:25

Merge branch 'recompilation_fix' into hyper_callback

abcff50

Merge branch 'lazy_setup' into hyper_callback

4d9b669

Fixes in tests

58b9482

Added a jitchol-like callback for raising the likelihood variance

86fedb1

Merge branch 'master' into hyper_callback

18a4436

Improved the jitchol callback

1282569

javdrher added this to the 0.1.0 release milestone Aug 23, 2017

icouckuy reviewed Aug 25, 2017

View reviewed changes

javdrher mentioned this pull request Aug 27, 2017

Initial implementation of sequential batch optimization #74

Open

javdrher added 3 commits August 30, 2017 22:14

Merge branch 'master' into hyper_callback

cb8421c

Addressing code review comments

3331fa0

Coverage fix

0d5d4bb

javdrher removed the do not merge yet label Aug 30, 2017

icouckuy reviewed Aug 31, 2017

View reviewed changes

icouckuy approved these changes Aug 31, 2017

View reviewed changes

Fix comments

339b699

javdrher merged commit d19ef28 into master Aug 31, 2017

javdrher deleted the hyper_callback branch August 31, 2017 19:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Callback #73

Callback #73

javdrher commented Aug 22, 2017

codecov-io commented Aug 22, 2017 •

edited

Loading

icouckuy Aug 22, 2017

javdrher Aug 22, 2017

icouckuy Aug 25, 2017

icouckuy Aug 25, 2017

javdrher Aug 25, 2017

icouckuy Aug 25, 2017

javdrher Aug 25, 2017

icouckuy Aug 25, 2017

icouckuy Aug 25, 2017

javdrher Aug 27, 2017

icouckuy Aug 25, 2017

icouckuy Aug 25, 2017

javdrher Aug 25, 2017

icouckuy Aug 26, 2017

javdrher Aug 27, 2017

icouckuy Aug 25, 2017

javdrher Aug 25, 2017

icouckuy Aug 25, 2017

icouckuy Aug 25, 2017

javdrher Aug 25, 2017

icouckuy Aug 26, 2017

icouckuy Aug 25, 2017

javdrher Aug 25, 2017

icouckuy Aug 26, 2017

icouckuy Aug 31, 2017

icouckuy Aug 31, 2017

icouckuy Aug 31, 2017

icouckuy Aug 31, 2017

		from .pareto import non_dominated_sort


		def jitchol_callback(models):

Callback #73

Callback #73

Conversation

javdrher commented Aug 22, 2017

codecov-io commented Aug 22, 2017 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-io commented Aug 22, 2017 •

edited

Loading