Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When RabbitMQ active queues is none, it throws an exception instead of failing gracefully #321

Open
lifenautjoe opened this issue Mar 12, 2022 · 0 comments

Comments

@lifenautjoe
Copy link

Hi Kristian,

First of all, thank you for the amazing package! If you have a place where we can invite you some coffee's, we're thrilled to do so.

We're seeing some exceptions and thought someone else might also benefit from a resolution.

Bug

We were experiencing issues with our RabbitMQ instance but instead of notifying us that there were issues with it, our API was throwing 500 errors.

Logs

[somus-api-production-sfo2] [2022-03-12 17:37:49] AttributeError: 'NoneType' object has no attribute 'values'
[somus-api-production-sfo2] [2022-03-12 17:37:49] 2022-03-12 18:37:49,145 django.request ERROR    Internal Server Error: /health/
[somus-api-production-sfo2] [2022-03-12 17:37:49] Traceback (most recent call last):
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/site-packages/asgiref/sync.py", line 451, in thread_handler
[somus-api-production-sfo2] [2022-03-12 17:37:49]     raise exc_info[1]
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/site-packages/django/core/handlers/exception.py", line 38, in inner
[somus-api-production-sfo2] [2022-03-12 17:37:49]     response = await get_response(request)
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/site-packages/django/core/handlers/base.py", line 233, in _get_response_async
[somus-api-production-sfo2] [2022-03-12 17:37:49]     response = await wrapped_callback(request, *callback_args, **callback_kwargs)
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/site-packages/asgiref/sync.py", line 414, in __call__
[somus-api-production-sfo2] [2022-03-12 17:37:49]     ret = await asyncio.wait_for(future, timeout=None)
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/asyncio/tasks.py", line 455, in wait_for
[somus-api-production-sfo2] [2022-03-12 17:37:49]     return await fut
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
[somus-api-production-sfo2] [2022-03-12 17:37:49]     result = self.fn(*self.args, **self.kwargs)
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/site-packages/asgiref/sync.py", line 455, in thread_handler
[somus-api-production-sfo2] [2022-03-12 17:37:49]     return func(*args, **kwargs)
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/site-packages/sentry_sdk/integrations/django/views.py", line 67, in sentry_wrapped_callback
[somus-api-production-sfo2] [2022-03-12 17:37:49]     return callback(request, *args, **kwargs)
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/site-packages/django/views/generic/base.py", line 70, in view
[somus-api-production-sfo2] [2022-03-12 17:37:49]     return self.dispatch(request, *args, **kwargs)
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/site-packages/django/views/generic/base.py", line 98, in dispatch
[somus-api-production-sfo2] [2022-03-12 17:37:49]     return handler(request, *args, **kwargs)
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/site-packages/django/utils/decorators.py", line 43, in _wrapper
[somus-api-production-sfo2] [2022-03-12 17:37:49]     return bound_method(*args, **kwargs)
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/site-packages/django/views/decorators/cache.py", line 44, in _wrapped_view_func
[somus-api-production-sfo2] [2022-03-12 17:37:49]     response = view_func(request, *args, **kwargs)
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/site-packages/health_check/views.py", line 86, in get
[somus-api-production-sfo2] [2022-03-12 17:37:49]     status_code = 500 if self.errors else 200
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/site-packages/health_check/mixins.py", line 16, in errors
[somus-api-production-sfo2] [2022-03-12 17:37:49]     self._errors = self.run_check()
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/site-packages/health_check/mixins.py", line 40, in run_check
[somus-api-production-sfo2] [2022-03-12 17:37:49]     for plugin in executor.map(_run, self.plugins):
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 619, in result_iterator
[somus-api-production-sfo2] [2022-03-12 17:37:49]     yield fs.pop().result()
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 444, in result
[somus-api-production-sfo2] [2022-03-12 17:37:49]     return self.__get_result()
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
[somus-api-production-sfo2] [2022-03-12 17:37:49]     raise self._exception
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
[somus-api-production-sfo2] [2022-03-12 17:37:49]     result = self.fn(*self.args, **self.kwargs)
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/site-packages/health_check/mixins.py", line 32, in _run
[somus-api-production-sfo2] [2022-03-12 17:37:49]     plugin.run_check()
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/site-packages/health_check/backends.py", line 30, in run_check
[somus-api-production-sfo2] [2022-03-12 17:37:49]     self.check_status()
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/site-packages/health_check/contrib/celery_ping/backends.py", line 33, in check_status
[somus-api-production-sfo2] [2022-03-12 17:37:49]     self._check_ping_result(ping_result)
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/site-packages/health_check/contrib/celery_ping/backends.py", line 50, in _check_ping_result
[somus-api-production-sfo2] [2022-03-12 17:37:49]     self._check_active_queues(active_workers)
[somus-api-production-sfo2] [2022-03-12 17:37:49]   File "/usr/local/lib/python3.8/site-packages/health_check/contrib/celery_ping/backends.py", line 61, in _check_active_queues
[somus-api-production-sfo2] [2022-03-12 17:37:49]     for queues in app.control.inspect(active_workers).active_queues().values():
[somus-api-production-sfo2] [2022-03-12 17:37:49] AttributeError: 'NoneType' object has no attribute 'values'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant