Updated doc

Readme.md

@@ -1,58 +1,97 @@
-# Kubernetes 1.1 Vagrant Machine
+# How to setup a lightweight Kubernetes Vagrant Machine
 
-Please execute this command on your host to make sure you can reach the services inside the vagrant machine:
+## TL;DR
 
-OSX:
 ```
-sudo route -n add 10.0.0.0/24 10.10.0.3
+make
+make add-box
+cd test; vagrant up
 ```
 
-Linux:
-```
-sudo ip route add 10.0.0.0/24 via 10.10.0.3
-```
+## Motivation
+There are already a number of tutorials out there on how to setup a Kubernetes cluster, but no setup met our expectations how a local cluster for application development should look like. For this machine we have the following goals:
 
-Windows:
-```
-route add 10.0.0.0 mask 255.255.255.0 10.10.0.3
-```
+- Easy setup for the developer
+- Robust system
+- Small in size and efficient resource usage
 
+## Requirements
 
-On Arch Linux:
-```
-systemctl start nfs-server rpcbind
-```
+- Vagrant
+- Make
 
-## Try it out (inside the VM (`vagrant ssh`)):
+We will use Debian Jessie as the base system, but all steps are easily transferable to other distributions.
+## Kubernetes Overview
+A Kubernetes cluster consists of master and node related services which we need to install on our Vagrant machine:
 
-Start a pdf rendering service
-```
-kubectl run mfb-pdf --image=dcr.mfb.io/mfb-service-pdf
-kubectl expose rc mfb-pdf --port=80
-```
+![Kubernetes Overview](doc/overview.png)
 
-Create a dev environment
-```
-kubectl create -f /data/dev.rc.yml
-kubectl create -f /data/dev.svc.yml
-```
+### Master related services (yellow)
+- `etcd` as a data store
+- `kube-apiserver` the public/private interface of the kubernetes cluster
+- `kube-scheduler` schedules the pods on the different nodes
+- `kube-controllermanager` manages different resources of the cluster (e.g. storage)
 
-List all endpoints
-```
-kubectl get endpoints
-```
+### Node related services (green, red)
+- `kubelet` manages resources and containers on the node
+- `kube-proxy` provides load balancing to services
+- `docker` as our container engine
 
-List all running services
+### Kubernetes plugins (purple)
+- `kube-dns` provides cluster internal DNS for services
+
+We will install all of the master and node related services on the same machine and manage them via `systemd`. This keeps the setup simple and makes it easy to check the logs via `journalctl`. The Kubernetes plugins will be managed via Kubernetes itself.
+
+### Networking
+An important part of setting up a kubernetes cluster is getting the networking right.
+We chose a L2 based approach because of the simplicity and small scale of the system.
+
+![Kubernetes Overview](doc/networking.png)
+
+Via a Vagrant private network (blue) the host and virtual machine can communicate on the L2 level with each other.
+We can also bridge the Docker network (red) to the same physical network, so that the host can communicate with the Docker containers directly and we don't need to setup any routing.
+This works for the pod level networking, but Kubernetes services have a separate virtual IP address range.
+This means we need to add a static route on the host, so that the host can also reach the services (green).
+We also configure NAT for the containers, so that they can reach the internet.
+
+For our example setup we choose `10.10.0.0/24` as the network for the containers and `10.0.0.0/24` as the virtual network for the services. You might want to adjust these to prevent conflicts in your network.
+
+## Provisioning
+Now that we know how we want our machine to look like we can write a provisioning shell script. The complete script is in the git repository as it would be too long to explain it here, but these are the parts you can modifiy:
+
+We hardcoded our versions to know good ones:
 ```
-kubectl get services
+ETCD_VERSION=2.2.3
+KUBERNETES_VERSION=1.1.4
+DOCKER_VERSION=1.8.3
 ```
 
-Update a container
+And networking:
 ```
-kubectl rolling-update mfb-dev mfb-dev-v2 --image=dcr.mfb.io/mfb-symfony-php7:latest
+NET_CIRD=10.10.0.0/24      # shared network between host/VM/Docker
+DOCKER_CIRD=10.10.0.128/25 # address range Docker will be using for containers, starts at .128 to prevent conflicts with the host/bridge IP
+BRIDGE_IP=10.10.0.2        # address of Docker bridge (cbr0)
+BRIDGE_MASK=255.255.255.0  # netmask of it
+PORTAL_CIRD=10.0.0.1/24    # virtual IP address range for services
+CLUSTERDNS_IP=10.0.0.10    # virtual IP of the kube-dns service
+DOMAIN=example.local       # top level domain for the kube-dns plugin
 ```
 
-Get some insight
-```
-csysdig
-```
+## Packaging the box
+We use make to package the box automatically. The following targets are available:
+
+- `make` build the box
+- `make stripped.box` just build the box
+- `make box.meta` just build a metadata file
+- `make clean` remove all files created during build
+- `make dist-clean` also remove cached, downloaded files
+
+## Testing
+
+- `make add-box` add the box to your local Vagrant
+- `cd test; vagrant up` start a test machine
+
+## Remarks
+- If one of the `vagrant up` steps fails you have to manually reset the build process via `make clean`.
+- For internal use we build our own Kubernetes and etcd Debian packages. You might want to do that, too.
+- It's not possible to run two of the Vagrant machines at the time on a single machine, because of networking conflicts.

doc/networking.xml

@@ -0,0 +1 @@
+<mxfile type="device" userAgent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36" version="5.2.7.6" editor="www.draw.io"><diagram>7VpZc+I4EP41PCZl40CYx5Bj8jCztVVs7fEosLC1MRYliyPz66clWrZ8gQnGIakARdktWZa+PtSHet79YvtdkGX4k/s06vUdf9vzHnr9/tDz4F8RXncEd/TN2VECwXykZYQJ+0WRaLqtmE+TXEfJeSTZMk+c8TimM5mjESH4Jt9tzqP8W5ckMG/MCJMZicrUf5gvwx111B9m9GfKgtC82R1+27VMyewlEHwV4/t6fW+uP7vmBTFj6YV6jwCi4ByGUVeL7T2NFJAGox0aTzWt6SQFjXEi+x+AJvXAmkQrXOczT+SOmMhXs/hkwxYRieFuvAmZpJMlmammDfAaaKFcRHDnwuUyIiy+CgR5xZGpkBSFoGJ2moRT+075gkoBDzr4AMoMiszVYICysMkY4A6RFlrgD26QSJDpQTp0hgVcIBzV0ODLLWjWU76NqYSxhxG8aeyzNVwG6tJ1ruEHfzBq/8Z0mArTbijQbD1VwrkRuFM1oSbgaubWgouSYdDtOxXgmj42uAbwU7AFhArYVmBKZZhCXcSt1Hl7rb/t4Nw+uimSXaA7aIJuxBtj6/ZvQbS1cF8mukbduwAX7b0FLth2CQpZZRP0N2cSDoK9ZgSGN/Yk5dx5TEkgKI1b4IeLImf4cdshQ25LDPmbwP4DE+87P8ksZLBtXcp+5jbY0QbG42l9Rxt9AquruX2RVhfdPQvdMTiy4FQWYaBAnOAtFzLkAY9J9JhRx0lIlqo1YvEL3GrnkarXOHnE6JbJfxX5eoB3/5mWGGZvNalb1aYGSCQR8k65xECIudYCTXtianG6D439Qg+gWO3/Uylf0UknK8mBlC3lB+fAWj2PRAr+Qu95xIVevOfoD7TMeSxxABc4itxX2OznPUDJV0KLkBUdwPQDanRQK1mtjIBlHaHbi2Li4SCCRkSydX4CVSKBY//JmbYypgufzxOYRFFm0ik0EiOzooNaWrsblzqDhX+CTeUdPNCjtLVTD9TF2R6llwcV0VxrVbsGR8zWxJwearW8TE0sWJ83KibudDnF1F5CWRwa611j5qJ+n5O5t1/MzTNXexddMLccmFfYvNlUNHdjLG/77NG7ABFr3XR6ZtfoxHSWo/c17EfnV7mPqXE+SUK9xszHPVrfMLyy9c1KF1YJyBWI9HCIYxl3+KL8nHKWQknRDsbUtzmPF1NkUZua6Y7e06vB7MSXFp5DCzF+zmlhTazR/q5XTnNUGl0QXKURFhcFTdgvMtUdFDJLpc16JoNxb/Cg4PDGJGJBDLQZAEABy7HSAQbVjztsWDDf19ISkSmNxmlNw8Ieqxp1oV2FVmGtBieXFivyGRUt0vvtXDGtgsO/1c5VVx9MxNCuFSznZDIrWOXXfFAreNOpFSynYu5hrbA6EG3IrZ6QDqzGrHF6cD9onskGImgj3CJtzEwe0MbMNaXUU0AzzLBAs1OBVbUulVA8hyDOYSOosiv11r5ZblaLRb3IFnKHxkHpQmLNdC3ws/rM+SoxFwl8l8Wcivr3h7QUHZuKcgR+0FSACHwgiT1U7npPU1GOvj+RqTgO+E5NxWeoqh+VVOoU3XJZ/WU1pVdLwbcwUAk8ttRRQ1Kfq3tbIb7bovpx7OiyqJ7Gv60mEnToX51ISIuWb04kzCKSJGyWyyWoUU/PJdSkC+zcQEVqAO1089MB7k0h4kyDppM4WY4w/7j7Cwi9PizMmUye9WlIsSHCZ3FwkQmFkxIIB7RMlYNHiJHJpreSPiiciUGFajV7YM4cfKX7mpW1bI2tKmFVpZ+OU1nPSOBJJ0/LoXjNUZJLUM4EtkNlOLyHk3N9O3nen+tz8rqKAdyJupqvk7SS6oPb7Bj3rnt2MN57/A0=</diagram></mxfile>

doc/overview.xml

@@ -0,0 +1 @@
+<mxfile type="device" userAgent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36" version="5.2.7.7" editor="www.draw.io"><diagram>5VlNc5s8EP41PqYDxh9wbNMk7+HtTGdyaHtUQAHVQmKEiO3++qxgZb7shsQ29kx98EiPJKR9dp/VDky823TzoEiWfJMR5ZOpE20m3tfJdBr4S/g3wLYCXN/xKiRWLEKsBh7ZH4qgg2jBIpq3JmopuWZZGwylEDTULYwoJdftac+St3fNSGx3rIHHkPA++oNFOqlQf47HM/h/lMWJ3dl1cOSJhKtYyULgfpOp91z+quGU2GeV8707IFFJCY8xrXRzS7kh0nJUPfT+wOjukIoKPMjfF/i4ItdbayiNwG7sSqUTGUtB+F2Nfkl0ymHQhSbdMP0T2s6nOfZ+mZ5pC622jSHTNWNm1W+q9RZdTAotAap3+l/KDOflWskVvZVcqvJsnlP+YKTkkxojTK9vtTVLFipEu5YYgZqomOI0H31kbG4sRK4eqEwpHBsmKMqJZi/tWCAYUvFuXs00NJDs/cTb07wQXuBTV8UTvSEZy6l6oarnl3XCNH3MSGnPGkTW9kTGCRM3UhEBphziA56rKUrygKU4Op3NqyWo15vAR2BdR7+L0eUkjchfIHYMO9YvDXaoDqNr48R1lmOSMvuAVttCea9yp2eQ7iCx+gB0xWolczqxlks/K0XMKjshk0zovPHk7waooyDoBIHvVsAuxXbmT22MHFgAjeoIdRDsbBkWF8hVN5XkYUKjgl88lXgd+5fzMUWDe/0TosGK6gpF0wuCy6sGyeqqBupH8AIH2aREQFF3afW4XSJGlU+AueXU8gGjxiwXqYggWuQa4JCTPGdhBd4zY3m5eJjCFn2FBThv/Boy8M/jnCqB7XdO5bimd3IgQ1t2hRRmkxJDcgfyP1aO3OPBK8mRszdSXm+B90ZSPT5HIlmdHMmBuA8lxVhRKk6QE2fdQnzckiIYvaToq24suWCKuUK5eEv05fXIBcnqlhSZkpsqwi6mGM++RrtMFYER07D+OD2ct0TAS6y+n8z4kGtukKRsvdCqIdAZF6ghbCjVMRvJcPXRsleBS4+P1nnvJrQEjRKteOudOL83o7kM4JFSuKXuvCl8MLl4mm6OjESZ6t8fcVmhMlhwhqAb9Y3n7nNBg5h8m2uavvut5xle2Sz2ULGwZh9JBXTr7x3VrVt/QfLuXgE=</diagram></mxfile>

example/Readme.md

@@ -0,0 +1,58 @@
+# Kubernetes 1.1 Vagrant Machine
+
+Please execute this command on your host to make sure you can reach the services inside the vagrant machine:
+
+OSX:
+```
+sudo route -n add 10.0.0.0/24 10.10.0.3
+```
+
+Linux:
+```
+sudo ip route add 10.0.0.0/24 via 10.10.0.3
+```
+
+Windows:
+```
+route add 10.0.0.0 mask 255.255.255.0 10.10.0.3
+```
+
+
+On Arch Linux:
+```
+systemctl start nfs-server rpcbind
+```
+
+## Try it out (inside the VM (`vagrant ssh`)):
+
+Start a pdf rendering service
+```
+kubectl run mfb-pdf --image=dcr.mfb.io/mfb-service-pdf
+kubectl expose rc mfb-pdf --port=80
+```
+
+Create a dev environment
+```
+kubectl create -f /data/dev.rc.yml
+kubectl create -f /data/dev.svc.yml
+```
+
+List all endpoints
+```
+kubectl get endpoints
+```
+
+List all running services
+```
+kubectl get services
+```
+
+Update a container
+```
+kubectl rolling-update mfb-dev mfb-dev-v2 --image=dcr.mfb.io/mfb-symfony-php7:latest
+```
+
+Get some insight
+```
+csysdig
+```