From 73b4b870310d03711e7268a99d613dfff5e25366 Mon Sep 17 00:00:00 2001 From: Ishaanj18 <93217075+Ishaanj18@users.noreply.github.com> Date: Sat, 4 May 2024 17:39:41 +0530 Subject: [PATCH] Update kubearmor_visibility.md Signed-off-by: Ishaan --- getting-started/kubearmor_visibility.md | 339 +++++++----------------- 1 file changed, 92 insertions(+), 247 deletions(-) diff --git a/getting-started/kubearmor_visibility.md b/getting-started/kubearmor_visibility.md index 8a47aa5fdf..8508b9a251 100644 --- a/getting-started/kubearmor_visibility.md +++ b/getting-started/kubearmor_visibility.md @@ -1,8 +1,6 @@ # KubeArmor Visibility -KubeArmor currently supports enabling visibility for containers and hosts. - -Visibility for hosts is not enabled by default, however it is enabled by default for containers . +There are only two sources of visibility configuration: the `kubearmor-visibility` annotation on the namespace and the visibility set in the `kubearmor-config` ConfigMap. The `karmor` tool provides access to both using `karmor logs`. @@ -17,7 +15,6 @@ The `karmor` tool provides access to both using `karmor logs`. - ### Prerequisites * If you don't have access to a K8s cluster, please follow [this](https://github.com/kubearmor/KubeArmor/blob/main/getting-started/deployment_guide.md#prerequisites) to set one up. @@ -32,273 +29,119 @@ kubectl apply -f https://raw.githubusercontent.com/kubearmor/KubeArmor/main/exam ``` This sample policy blocks execution of the `apt` and `apt-get` commands in wordpress pods with label selector `app: wordpress`. -### Getting Container Visibility +### ConfigMap Visibility + KubeArmor has the ability to let the user select what kind of events have to be traced by changing the `visibility` at the `KubeArmorConfig` configmap. + However it is not recommended to manually tamper with the configmap and the right way to do it is via editing the `KubeArmorConfig` resource named `kubearmor-default` in `kubearmor` namespace and setting `defaultVisibility`. -* Checking default visibility +* Checking ConfigMap visibility configuration - * Container visibility is enabled by default. We can check it using `kubectl describe` and grep `kubearmor-visibility` + * Visibility configuration in ConfigMap can be checked using `kubectl describe`. ```text - POD_NAME=$(kubectl get pods -n wordpress-mysql -l app=wordpress -o jsonpath='{.items[0].metadata.name}') && kubectl describe -n wordpress-mysql pod $POD_NAME | grep kubearmor-visibility + kubectl get configmap kubearmor-config -n kubearmor -o yaml | grep visibility - kubearmor-visibility: process, file, network, capabilities - ``` - * **For pre-existing workloads :** Enable visibility using `kubectl annotate`. Currently KubeArmor supports `process`, `file`, `network`, `capabilities` - ```text - kubectl annotate pods -n wordpress-mysql "kubearmor-visibility=process,file,network,capabilities" + visibility: process,network ``` -* Open up a terminal, and watch logs using the `karmor` cli - ```text - karmor logs - ``` -* In another terminal, simulate a policy violation . Try `sleep` inside a pod + * Visibility configuration in `kubearmor-default` resource can be checked using `kubectl describe`. ```text - POD_NAME=$(kubectl get pods -n wordpress-mysql -l app=wordpress -o jsonpath='{.items[0].metadata.name}') && kubectl -n wordpress-mysql exec -it $POD_NAME -- bash - # apt update - ``` -* In the terminal running `karmor logs`, the policy violation along with container visibility is shown, in this case for example -
- Click to expand + kubectl describe kubearmorconfig kubearmor-default -n kubearmor | grep "defaultVisibillity" - ```text - == Alert / 2023-01-04 04:58:37.689182 == - ClusterName: default - HostName: sibashi-asus - NamespaceName: wordpress-mysql - PodName: wordpress-787f45786f-mm2bm - Labels: app=wordpress - ContainerName: wordpress - ContainerID: 9af5246810fd0a732e74d391b32b95f65e4c08e655d1ab10b49b04b148cc1c24 - ContainerImage: docker.io/library/wordpress:4.8-apache@sha256:6216f64ab88fc51d311e38c7f69ca3f9aaba621492b4f1fa93ddf63093768845 - Type: MatchedPolicy - PolicyName: ksp-wordpress-block-process - Severity: 3 - Source: /bin/bash - Resource: /usr/bin/apt update - Operation: Process - Action: Block - Data: syscall=SYS_EXECVE - Enforcer: AppArmor - Result: Permission denied - HostPID: 17462 - HostPPID: 17293 - PID: 199 - PPID: 193 - ParentProcessName: /bin/bash - ProcessName: /usr/bin/apt + defaultVisibility: process,network ``` - -
- - -* The logs can also be generated in JSON format using `karmor logs --json ` - -
- Click to expand - - ```json - { - "Timestamp":1672808328, - "UpdatedTime":"2023-01-04T04:58:48.838991Z", - "ClusterName":"default","HostName":"sibashi-asus", - "NamespaceName":"wordpress-mysql","PodName":"wordpress-787f45786f-mm2bm", - "Labels":"app=wordpress", - "ContainerID":"9af5246810fd0a732e74d391b32b95f65e4c08e655d1ab10b49b04b148cc1c24", - "ContainerName":"wordpress", - "ContainerImage":"docker.io/library/wordpress:4.8-apache@sha256:6216f64ab88fc51d311e38c7f69ca3f9aaba621492b4f1fa93ddf63093768845", - "HostPPID":17293, - "HostPID":17526, - "PPID":193, - "PID":200, - "ParentProcessName":"/bin/bash", - "ProcessName":"/usr/bin/apt", - "PolicyName":"ksp-wordpress-block-process", - "Severity":"3", - "Type":"MatchedPolicy", - "Source":"/bin/bash", - "Operation":"Process", - "Resource":"/usr/bin/apt update", - "Data":"syscall=SYS_EXECVE", - "Enforcer":"AppArmor", - "Action":"Block", - "Result":"Permission denied" - } + * **To update the visibility configuration of configMap :** It's recommended to edit the KubeArmorConfig resource named `kubearmor-default` in the kubearmor namespace and set the `defaultVisibility` field. + + ```text + kubectl edit kubearmorconfig kubearmor-default -n kubearmor ``` -
- -### Getting Host Visibility - -* Host Visibility is not enabled by default . To enable Host Visibility we need to annotate the node using `kubectl annotate node` -``` - kubectl annotate node "kubearmor-visibility=process,file,network,capabilities" -``` - -* To confirm it use `kubectl describe` and grep `kubearmor-visibility` -```text -kubectl describe node | grep kubearmor-visibility -``` - -* Now we can get general telemetry events in the context of the host using `karmor logs` .The logs related to Host Visibility will have type `Type: HostLog`and `Operation: File | Process | Network ` - -```text -karmor logs --logFilter=all -``` +* Now we can get general telemetry events using karmor logs. Open up a terminal, and watch logs using the karmor cli + ```text + karmor logs --logFilter=system -
- Click to expand + ``` + +* In another terminal, let's exec into the pod and run some process commands . Try `ls` inside the pod ```text - == Alert / 2023-01-04 04:58:37.689182 == -== Log / 2023-01-27 14:41:49.017709 == -ClusterName: default -HostName: kubearmor-dev2 -Type: HostLog -Source: /usr/bin/dockerd -Resource: /usr/bin/runc --version -Operation: Process -Data: syscall=SYS_EXECVE -Result: Passed -HostPID: 193088 -HostPPID: 914 -PID: 193088 -PPID: 914 -ParentProcessName: /usr/bin/dockerd -ProcessName: /usr/bin/runc -== Log / 2023-01-27 14:41:49.018951 == -ClusterName: default -HostName: kubearmor-dev2 -Type: HostLog -Source: /usr/bin/runc --version -Resource: /lib/x86_64-linux-gnu/libc.so.6 -Operation: File -Data: syscall=SYS_OPENAT fd=-100 flags=O_RDONLY|O_CLOEXEC -Result: Passed -HostPID: 193088 -HostPPID: 914 -PID: 193088 -PPID: 914 -ParentProcessName: /usr/bin/dockerd -ProcessName: /usr/bin/runc -== Log / 2023-01-27 14:41:49.018883 == -ClusterName: default -HostName: kubearmor-dev2 -Type: HostLog -Source: /usr/bin/runc --version -Resource: /etc/ld.so.cache -Operation: File -Data: syscall=SYS_OPENAT fd=-100 flags=O_RDONLY|O_CLOEXEC -Result: Passed -HostPID: 193088 -HostPPID: 914 -PID: 193088 -PPID: 914 -ParentProcessName: /usr/bin/dockerd -ProcessName: /usr/bin/runc -== Log / 2023-01-27 14:41:49.020905 == -ClusterName: default -HostName: kubearmor-dev2 -Type: HostLog -Source: /var/lib/rancher/k3s/data/2949af7261ce923f6a5091396d266a0e9d9436dcee976fcd548edc324eb277bb/bin/k3s -Resource: /var/lib/rancher/k3s/data/2949af7261ce923f6a5091396d266a0e9d9436dcee976fcd548edc324eb277bb/bin/portmap -Operation: Process -Data: syscall=SYS_EXECVE -Result: Passed -HostPID: 193090 -HostPPID: 5627 -PID: 193090 -PPID: 5627 -ParentProcessName: /var/lib/rancher/k3s/data/2949af7261ce923f6a5091396d266a0e9d9436dcee976fcd548edc324eb277bb/bin/k3s -ProcessName: /var/lib/rancher/k3s/data/2949af7261ce923f6a5091396d266a0e9d9436dcee976fcd548edc324eb277bb/bin/portmap - + POD_NAME=$(kubectl get pods -n wordpress-mysql -l app=wordpress -o jsonpath='{.items[0].metadata.name}') && kubectl -n wordpress-mysql exec -it $POD_NAME -- bash + # ls ``` -* The logs can also be generated in JSON format using `karmor logs --logFilter=all --json ` + Now, we can notice logs have been generated for the above command and logs with only `Operation: Network` and `Operation: Process` are shown as configured in default visibility. -
+
Click to expand - ```json + ```text - { - "Timestamp": 1674889485, - "UpdatedTime": "2023-01-28T07:04:45.436854Z", - "ClusterName": "default", - "HostName": "kubearmor-dev2", - "ParentProcessName": "/usr/bin/containerd-shim", - "ProcessName": "/usr/bin/containerd", - "HostPPID": 80989, - "HostPID": 614074, - "PPID": 80989, - "PID": 614074, - "Type": "HostLog", - "Source": "/usr/bin/containerd --address /run/containerd/containerd.sock publish --topic /tasks/exit --namespace moby", - "Operation": "File", - "Resource": "/lib/x86_64-linux-gnu/libdl.so.2", - "Data": "syscall=SYS_OPENAT fd=-100 flags=O_RDONLY|O_CLOEXEC", - "Result": "Passed", - }, - { - "Timestamp": 1674889485, - "UpdatedTime": "2023-01-28T07:04:45.440804Z", - "ClusterName": "default", - "HostName": "kubearmor-dev2", - "ParentProcessName": "/usr/bin/containerd-shim", - "ProcessName": "/usr/bin/containerd", - "HostPPID": 80989, - "HostPID": 614074, - "PPID": 80989, - "PID": 614074, - "Type": "HostLog", - "Source": "/usr/bin/containerd --address /run/containerd/containerd.sock publish --topic /tasks/exit --namespace moby", - "Operation": "File", - "Resource": "/usr/bin/containerd", - "Data": "syscall=SYS_OPENAT fd=-100 flags=O_RDONLY|O_CLOEXEC", - "Result": "Passed", - }, - { - "Timestamp": 1674889485, - "UpdatedTime": "2023-01-28T07:04:45.427658Z", - "ClusterName": "default", - "HostName": "kubearmor-dev2", - "ParentProcessName": "/bin/bash", - "ProcessName": "/usr/bin/pgrep", - "HostPPID": 614066, - "HostPID": 614073, - "PPID": 614066, - "PID": 614073, - "Type": "HostLog", - "Source": "/bin/bash", - "Operation": "Process", - "Resource": "/usr/bin/pgrep kubearmor", - "Data": "syscall=SYS_EXECVE", - "Result": "Passed", - }, - { - "Timestamp": 1674889485, - "UpdatedTime": "2023-01-28T07:04:45.450454Z", - "ClusterName": "default", - "HostName": "kubearmor-dev2", - "ParentProcessName": "/usr/bin/containerd-shim", - "ProcessName": "/usr/bin/containerd", - "HostPPID": 80989, - "HostPID": 614074, - "PPID": 80989, - "PID": 614074, - "Type": "HostLog", - "Source": "/usr/bin/containerd --address /run/containerd/containerd.sock publish --topic /tasks/exit --namespace moby", - "Operation": "Network", - "Resource": "domain=AF_UNIX type=SOCK_STREAM|SOCK_NONBLOCK|SOCK_CLOEXEC protocol=0", - "Data": "syscall=SYS_SOCKET", - "Result": "Passed" - } + == Log / 2023-01-27 14:41:49.017709 == + ClusterName: default + HostName: kubearmor-dev2 + Type: HostLog + Source: /usr/bin/dockerd + Resource: /usr/bin/runc --version + Operation: Process + Data: syscall=SYS_EXECVE + Result: Passed + HostPID: 193088 + HostPPID: 914 + PID: 193088 + PPID: 914 + ParentProcessName: /usr/bin/dockerd + ProcessName: /usr/bin/runc + == Log / 2023-01-27 14:41:49.018951 == + ClusterName: default + HostName: kubearmor-dev2 + Type: HostLog + Source: /usr/bin/runc --version + Resource: /lib/x86_64-linux-gnu/libc.so.6 + Operation: File + Data: syscall=SYS_OPENAT fd=-100 flags=O_RDONLY|O_CLOEXEC + Result: Passed + HostPID: 193088 + HostPPID: 914 + PID: 193088 + PPID: 914 + ParentProcessName: /usr/bin/dockerd + ProcessName: /usr/bin/runc + == Log / 2023-01-27 14:41:49.018883 == + ClusterName: default + HostName: kubearmor-dev2 + Type: HostLog + Source: /usr/bin/runc --version + Resource: /etc/ld.so.cache + Operation: File + Data: syscall=SYS_OPENAT fd=-100 flags=O_RDONLY|O_CLOEXEC + Result: Passed + HostPID: 193088 + HostPPID: 914 + PID: 193088 + PPID: 914 + ParentProcessName: /usr/bin/dockerd + ProcessName: /usr/bin/runc + == Log / 2023-01-27 14:41:49.020905 == + ClusterName: default + HostName: kubearmor-dev2 + Type: HostLog + Source: /var/lib/rancher/k3s/data/2949af7261ce923f6a5091396d266a0e9d9436dcee976fcd548edc324eb277bb/bin/k3s + Resource: /var/lib/rancher/k3s/data/2949af7261ce923f6a5091396d266a0e9d9436dcee976fcd548edc324eb277bb/bin/portmap + Operation: Process + Data: syscall=SYS_EXECVE + Result: Passed + HostPID: 193090 + HostPPID: 5627 + PID: 193090 + PPID: 5627 + ParentProcessName: /var/lib/rancher/k3s/data/2949af7261ce923f6a5091396d266a0e9d9436dcee976fcd548edc324eb277bb/bin/k3s + ProcessName: /var/lib/rancher/k3s/data/2949af7261ce923f6a5091396d266a0e9d9436dcee976fcd548edc324eb277bb/bin/portmap + ``` +
- ``` -
+> Note: While default visibility settings set through the ConfigMap provide a broad stroke for monitoring events within a Kubernetes cluster, there are cases where specific applications or use cases necessitate finer control over visibility settings like as in a case like [this](https://github.com/kubearmor/KubeArmor/blob/main/getting-started/deployment_guide.md#audit-access-to-folderspaths). Namespace visibility settings can override the default visibility configurations set at the ConfigMap level, allowing for tailored monitoring and enforcement within specific namespaces. ### Updating Namespace Visibility @@ -313,6 +156,8 @@ KubeArmor has the ability to let the user select what kind of events have to be kubearmor-visibility: process, file, network, capabilities ``` + > Note: By default namespace annotations are set to none. + * **To update the visibility of namespace :** Now let's update Kubearmor visibility using `kubectl annotate`. Currently KubeArmor supports `process`, `file`, `network`, `capabilities`. Lets try to update visibility for the namespace `wordpress-mysql` @@ -378,4 +223,4 @@ KubeArmor has the ability to let the user select what kind of events have to be ProcessName: /usr/bin/apt ``` -
\ No newline at end of file +