Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RDF issue that occasionally happens in the CI #504

Open
PythonFZ opened this issue Feb 23, 2022 · 1 comment
Open

RDF issue that occasionally happens in the CI #504

PythonFZ opened this issue Feb 23, 2022 · 1 comment
Assignees
Labels
bug Something isn't working calculator Issue with calculator data management Issue with data management

Comments

@PythonFZ
Copy link
Member

PythonFZ commented Feb 23, 2022

traj_file = '/tmp/pytest-of-runner/pytest-0/NaCl_gk_i_q.lammpstraj'
true_values = {'Cl_Cl': {'left': 0.2372193090684762, 'min_pomf': 5.4953035102048034e-11, 'pomf': [inf, inf, inf, inf, inf, inf, ...]...20018507094386176, 0.0030027760641579263, 0.004003701418877235, 0.005004626773596544, 0.006005552128315853, ...], ...}}
tmp_path = PosixPath('/tmp/pytest-of-runner/pytest-0/test_experiment1')

    def test_experiment(traj_file, true_values, tmp_path):
        """Test the pomf called from the experiment class"""
        os.chdir(tmp_path)
        project = mds.Project()
        project.add_experiment(
            "NaCl", simulation_data=traj_file, timestep=0.002, temperature=1400
        )
    
>       computation = project.experiments["NaCl"].run.PotentialOfMeanForce(plot=False)

/home/runner/work/MDSuite/MDSuite/CI/integration_tests/calculators/test_potential_of_mean_force.py:75: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/home/runner/work/MDSuite/MDSuite/mdsuite/calculators/calculator.py:125: in inner
    cls.run_analysis()
/home/runner/work/MDSuite/MDSuite/mdsuite/calculators/calculator.py:321: in run_analysis
    self.run_calculator()
/home/runner/work/MDSuite/MDSuite/mdsuite/calculators/potential_of_mean_force.py:236: in run_calculator
    calculations = self.experiment.run.RadialDistributionFunction(plot=False)
/home/runner/work/MDSuite/MDSuite/mdsuite/calculators/calculator.py:125: in inner
    cls.run_analysis()
/home/runner/work/MDSuite/MDSuite/mdsuite/calculators/calculator.py:321: in run_analysis
    self.run_calculator()
/home/runner/work/MDSuite/MDSuite/mdsuite/calculators/radial_distribution_function.py:854: in run_calculator
    enumerate(batch_ds), ncols=70, disable=batch_tqm, total=self.n_batches
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/tqdm/std.py:1180: in __iter__
    for obj in iterable:
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/tensorflow/python/data/ops/iterator_ops.py:836: in __next__
    return self._next_internal()
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/tensorflow/python/data/ops/iterator_ops.py:822: in _next_internal
    output_shapes=self._flat_output_shapes)
/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/tensorflow/python/ops/gen_dataset_ops.py:2923: in iterator_get_next
    _ops.raise_from_not_ok_status(e, name)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

e = _NotOkStatusException(), name = None

    def raise_from_not_ok_status(e, name):
      e.message += (" name: " + name if name is not None else "")
>     raise core._status_to_exception(e) from None  # pylint: disable=protected-access
E     tensorflow.python.framework.errors_impl.UnknownError: IndexError: list index out of range
E     Traceback (most recent call last):
E     
E       File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/tensorflow/python/ops/script_ops.py", line 271, in __call__
E         ret = func(*args)
E     
E       File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/tensorflow/python/autograph/impl/api.py", line 642, in wrapper
E         return func(*args, **kwargs)
E     
E       File "/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 1004, in generator_py_func
E         values = next(generator_state.get_iterator(iterator_id))
E     
E       File "/home/runner/work/MDSuite/MDSuite/mdsuite/database/data_manager.py", line 204, in generator
E         select_slice = np.s_[self.atom_selection, loop_array[batch]]
E     
E     IndexError: list index out of range
E     
E     
E     	 [[{{node PyFunc}}]] [Op:IteratorGetNext]

/opt/hostedtoolcache/Python/3.7.12/x64/lib/python3.7/site-packages/tensorflow/python/framework/ops.py:7186: UnknownError

see also https://github.com/zincware/MDSuite/runs/5309604368?check_suite_focus=true

@PythonFZ PythonFZ added bug Something isn't working calculator Issue with calculator data management Issue with data management p1-high Should be completed as soon as possible. labels Feb 23, 2022
@SamTov
Copy link
Member

SamTov commented Mar 16, 2022

So I have tried again in #493 to resolve this by essentially further ensuring that there can be NO remainder in the batch but there could be a better way to avoid this. I would change the priority of this until we see it arise again but not close it until we can test this property. Perhaps in #476 this can actually be tested correctly by forcing a remainder in the batch.

@SamTov SamTov removed the p1-high Should be completed as soon as possible. label Mar 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working calculator Issue with calculator data management Issue with data management
Projects
None yet
Development

No branches or pull requests

2 participants