You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The test test_boptim in test/test_boptim.py is flaky when the seed is removed. The test failed 500 out of 500 times that I ran.
Interestingly, the test always only fails for the first parameter ("ei") and passes for the other two: "poi" and "cb". I observed that this is true even if I change the order of the parameters i.e., whatever comes first fails! Error message below.
Do you know why this happens or how can this be resolved? I will be happy to raise a PR if you have any suggestions!
Thanks!
Error message:
_________________________________________________________ test_boptim[ei-/home/saikat/projects/borntobeflaky/projects/GPim/test/test_data/test_ei.npy] _________________________________________________________
acqf = 'ei', result = '/home/saikat/projects/borntobeflaky/projects/GPim/test/test_data/test_ei.npy'
@pytest.mark.parametrize(
"acqf, result",
[("ei", test_img_ei),
# ("poi", test_img_poi),
# ("cb", test_img_cb)
])
def test_boptim(acqf, result):
Z_sparse = initial_seed()
X_full = gprutils.get_full_grid(Z_sparse)
X_sparse = gprutils.get_sparse_grid(Z_sparse)
expected_result = np.load(result)
boptim = boptimizer(
X_sparse, Z_sparse, X_full,
trial_func, acquisition_function=acqf,
exploration_steps=20,
use_gpu=False, verbose=1)
boptim.run()
#print(expected_result)
> assert_allclose(boptim.target_func_vals[-1], expected_result)
test/test_boptim.py:60:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
x = array([[3.69114071e-08, nan, nan, nan,
nan, nan, ... nan, nan,
nan, nan, nan, nan,
7.57299443e-34]])
y = array([[3.69114071e-08, nan, nan, nan,
nan, nan, 3.6477... nan, nan,
nan, nan, nan, nan,
7.57299443e-34]])
func = <ufunc 'isnan'>, hasval = 'nan'
def func_assert_same_pos(x, y, func=isnan, hasval='nan'):
"""Handling nan/inf.
Combine results of running func on x and y, checking that they are True
at the same locations.
"""
x_id = func(x)
y_id = func(y)
# We include work-arounds here to handle three types of slightly
# pathological ndarray subclasses:
# (1) all() on `masked` array scalars can return masked arrays, so we
# use != True
# (2) __eq__ on some ndarray subclasses returns Python booleans
# instead of element-wise comparisons, so we cast to bool_() and
# use isinstance(..., bool) checks
# (3) subclasses with bare-bones __array_function__ implementations may
# not implement np.all(), so favor using the .all() method
# We are not committed to supporting such subclasses, but it's nice to
# support them if possible.
if bool_(x_id == y_id).all() != True: msg = build_err_msg([x, y],
err_msg + '\nx and y %s location mismatch:'
% (hasval), verbose=verbose, header=header,
names=('x', 'y'), precision=precision)
> raise AssertionError(msg)
E AssertionError:
E Not equal to tolerance rtol=1e-07, atol=0
E
E x and y nan location mismatch:
E x: array([[3.691141e-08, nan, nan, nan,
E nan, nan, nan, nan,
E 1.886172e-02, nan, nan, 2.844261e-02,...
E y: array([[3.691141e-08, nan, nan, nan,
E nan, nan, 3.647780e-03, nan,
E nan, 2.844261e-02, nan, nan,...
../../../../anaconda3/envs/GPim/lib/python3.6/site-packages/numpy/testing/_private/utils.py:740: AssertionError
The text was updated successfully, but these errors were encountered:
The test
test_boptim
intest/test_boptim.py
is flaky when the seed is removed. The test failed 500 out of 500 times that I ran.Interestingly, the test always only fails for the first parameter ("ei") and passes for the other two: "poi" and "cb". I observed that this is true even if I change the order of the parameters i.e., whatever comes first fails! Error message below.
Do you know why this happens or how can this be resolved? I will be happy to raise a PR if you have any suggestions!
Thanks!
Error message:
The text was updated successfully, but these errors were encountered: