Skip to content

Latest commit

 

History

History
45 lines (42 loc) · 1.25 KB

README.md

File metadata and controls

45 lines (42 loc) · 1.25 KB
Raw

Multi-broker Kafka setup over Kubernetes

In this repo, I'm gonna share my deployment of the Multi-broker Kafka installation and configuration. Let's assume we have Kubernetes installed already ;).

Lets build the pods with the below order.

  • Zookeeper & zookeeper service
  • Kafka broker and its service
  • Schema registry and its service
  • Kafka connect components and its service Here we go :
  1. Zookeeper installation
kubectl apply -f zookeeper.yaml
kubectl apply -f zookeeper-service.yaml
  1. Kafka-broker installation
kubectl apply -f kafka-broker.yaml
kubectl apply -f kafka-service.yaml
  1. Schema registry installation
kubectl apply -f schema-registry.yaml
kubectl apply -f schema-registry-svc.yaml
  1. Kafka connect installation
kubectl apply -f kafka-producer.yaml
kubectl apply -f kafka-producer-svc.yaml
kubectl apply -f kafka-consumer.yaml
kubectl apply -f kafka-consumer-svc.yaml

We need to ensure that all pods and services are up and running. Let's check.

kubectl get pod
kubectl get service

If you observe something wrong like pod status is in 'RESTARTING | FAILED' or something else, you will need to investigate the root cause.

kubectl logs <pod_name>
kubectl get events
kubectl describe <resource>