A sample project demonstrating how to provision a simple service on Kubernetes, monitor metrics via Prometheus and visualize with Grafana.
Install minikube - see instructions at: https://github.com/kubernetes/minikube
Start Minikube
minikube start
Apply all manifests (sample-app, prometheus-operator, grafana)
./manifests/apply.sh
Navigate to Grafana local address (obtained via minikube cli)
minikube service grafana --url
Login (foo:foo) to view the selection of infrastructural and business metric dashboards:
- Plot business metrics properly
- Define a smooth dashboard workflow. I.e create -> update -> version -> provision
- All RBAC
- Consistent labels / expression for service monitor selectors
- Proper namespacing
- Alert Manager / Prometheus Rules
- Extra Monitoring Pieces i.e kube-controller-manager, kube-scheduler, apiserver, etcd
This project contains a simple Golang app for metric demonstration purposes. This will be deployed as a service with 3 replicated pods. The container image itself is publicly available so no need to build from source locally.
If you do wish to build the sample app locally from the command line:
env GOOS=linux GOARCH=386 go build && docker build -t test . && docker run -p 8081:8080 test
minikube service sample-app --url
Chosen to deploy prometheus using the operator to ease the complex configuration of maintaining a server manually see:
https://coreos.com/operators/prometheus/docs/latest/user-guides/getting-started.html
minikube service prometheus --url
Modelled on example configuration found here: https://github.com/coreos/prometheus-operator/tree/master/contrib/kube-prometheus/manifests/grafana
Adding the json for the dashboards by hand is pretty painful, discussions and tools below: prometheus-operator/prometheus-operator#272 https://github.com/coreos/prometheus-operator/blob/master/contrib/kube-prometheus/docs/developing-alerts-and-dashboards.md https://github.com/coreos/prometheus-operator/tree/master/contrib/kube-prometheus/hack/grafana-dashboards-configmap-generator
minikube service grafana --url
Modelled on example configuration found here: https://github.com/coreos/prometheus-operator/tree/master/contrib/kube-prometheus