galapagos compatibility with managed apache kafka #21
-
|
Hi there, So my question to the community: Is Galapagos compatible with such of managed service like confluent cloud (confluence.io), Amazon MSK (https://aws.amazon.com/msk/) or Azure HDInsight (https://azure.microsoft.com/de-de/services/hdinsight/) ? Thank's in advance for your hints, tips and thank's sharing your experiences. |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 3 replies
-
|
Hi @MrTB , Indeed, as with most standard software today, it is much better to not operate Kafka yourself, but use a managed Kafka like one of the products you mentioned. Especially for Kafka, this offers additional advantages like auto-scaling, pay-per-use etc. We at Hermes have not had this opportunity yet, as e.g. the Confluent product is not compatible with our current cloud provider, so we really operate our own clusters now. I have talked with Confluent about a year ago, and it looks like Galapagos is compatible to their product. Galapagos needs a Kafka product to offer the following:
Using these criteria, you can easily evaluate every Kafka-based product for compatibility with Galapagos. Once we have evaluated some, of course we will report our results here at GitHub, but we are looking forward to hear from others and their experience! |
Beta Was this translation helpful? Give feedback.
-
|
Just FYI: there is currently (Status from 18 February 2021) not possible to put CA cert in Kafka trust-store on confluent.io. |
Beta Was this translation helpful? Give feedback.
-
|
Yep, that's correct ATM. Currently, you could only use Confluent Platform as managed product from Confluent (that was the compatible product we determined last year - and also in a call two weeks ago). But we are currently designing a modular concept for application authentication, which would also allow for using Confluent's API Keys and thus completely remove the need for a custom PKI. This is in the very beginning; as soon as we make progress there, we will document it here on GitHub! |
Beta Was this translation helpful? Give feedback.
-
|
@albrechtflo-hg please don't get me wrong. I appreciated that u guys created such of great tool. In between we use Azure HDInsight for proof of concept since Azure HDInsight allows us to put CA in kafka trust-store. And I think, AWS MSK also allows it too. |
Beta Was this translation helpful? Give feedback.
-
|
@MrTB FYI: Galapagos 2.1.0 is now finally released and does support Confluent Cloud - in fact, we are using Galapagos together with Confluent Cloud already for some months now at our company. If you plan to give it a try, we are looking forward to any feedback :-) |
Beta Was this translation helpful? Give feedback.
-
|
@albrechtflo-hg thanks for remind me. I appreciate your support. Vielen herzlichen Dank! |
Beta Was this translation helpful? Give feedback.
Hi @MrTB ,
Indeed, as with most standard software today, it is much better to not operate Kafka yourself, but use a managed Kafka like one of the products you mentioned. Especially for Kafka, this offers additional advantages like auto-scaling, pay-per-use etc.
We at Hermes have not had this opportunity yet, as e.g. the Confluent product is not compatible with our current cloud provider, so we really operate our own clusters now.
But as we are shifting to a new provider, we plan to evaluate Managed Kafka products.
I have talked with Confluent about a year ago, and it looks like Galapagos is compatible to their product. Galapagos needs a Kafka product to offer the following: