Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How does amqpstorm handle Rabbitmq memory alarms #118

Open
mehdigmira opened this issue Apr 21, 2022 · 4 comments
Open

How does amqpstorm handle Rabbitmq memory alarms #118

mehdigmira opened this issue Apr 21, 2022 · 4 comments

Comments

@mehdigmira
Copy link

mehdigmira commented Apr 21, 2022

Hello,

Thanks for your work on this library 👍
I have a question regarding Rabbitmq memory watermarks and the way amqpstorm handles them.

Imagine the following:

  1. I have a process that keeps publishing messages (basically an infinite loop for this example)
  2. At a certain point memory alarm is triggered on RMQ, and further publishes are blocked (Writes on a blocked connection will time out or fail with an I/O write exception.), also the publisher get notified that a connexion is blocked (Compatible AMQP 0-9-1 clients will be [notified] when they are blocked and unblocked.).
  3. Then Rabbitmq flushes out some of the data to disk, and unblocks publisher

What happens on AMQPstorm side ? Does it raise ? do we lose some messages ? does the block/unblock logic trigger some logic in AMQPStorm ?

Thanks for the help

Links: RMQ doc

@eandersson
Copy link
Owner

The state is exposed on the connection level using the is_blocked property.

e.g.

while connection.is_blocked:
    time.sleep(1)
channel.publish(...)

I suspect that if you enable confirm_deliveries on the channel level it would raise an exception as well, or at the very least return False, since the message wasn't delivered successfully.

@jhogg
Copy link
Collaborator

jhogg commented Apr 22, 2022 via email

@mehdigmira
Copy link
Author

Thanks for the swift replies. The is_blocked property is asynchronous, since it relies on a notification sent by Rabbitmq, so we might be trying to publish on a blocked connection, even if we use is_blocked. So my question is: what does amqpstorm do when i try to write on a blocked connection ?
Rabbitmq doc states Writes on a blocked connection will time out or fail with an I/O write exception..
So how does that transpire ? Will channel.publish(...) raise ?

@jhogg
Copy link
Collaborator

jhogg commented Apr 22, 2022 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants