Kubernets cluster in Azure cloud
Links
Kubernets config
Use Declarative, we declare the state and kubectl implements using
kubectl apply -R -f configs/
Setup Cluster, using the Azure az commands and azure aks install-cli kubectl
- Use az tool, with docker run -it microsoft/azure-cli
in the container add the kubectl
az aks install-cli az aks get-credentials --resource-group <RG> --name <name> --subscription <Hex-ID>
list subscriptions
az account list --output table
set subscription to the one that contains k8s
az account set --subscription xx-xx-xx
run az proxy to connect the browser to kubernets admin in cloud
Proxy running on http://127.0.0.1:8001/ Press CTRL+C to close the tunnel... Forwarding from 127.0.0.1:8001 -> 9090 ## Problem only binds to loopback, in a container, if not using container for microsoft/azure-cli skip next command. nc -v -lk -p 8001 -s $(hostname -i) -e /usr/bin/nc 127.0.0.1 8001 ## Web dashboard no right - nodes is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list nodes at the cluster scope ## RBAC ClusterRoleBinding must be created for Kubernetes dashboard kubectl create clusterrolebinding kubernetes-dashboard \ --clusterrole=cluster-admin \ --serviceaccount=kube-system:kubernetes-dashboard
List nodes
kubectl get nodes
If this fails with "Unable to connect to the server: dial tcp: lookup ...." reset with
rm .kube/config az aks get-credentials --resource-group <nameRG> --name >nameClusterInRG> kubectl get nodes
List namespaces
kubectl get namespaces
Reset ssh key and password to get access to k8s Node in Azure 2020-05
reset ssh key and password
AZ_RG=aks-<xxxx>-nodes AZ_SUBSCRIPTION="xxx" RANDOM_PWD=$( ( head /dev/urandom ; date +%s) | sha256sum | base64 | head -c32 ) SCALE_SET_NAME=$(az vmss list --resource-group $AZ_RG --subscription "$AZ_SUBSCRIPTION" --query [0].name -o tsv) echo "SCALE_SET_NAME=$SCALE_SET_NAME RANDOM_PWD=$RANDOM_PWD AZ_SUBSCRIPTION=$AZ_SUBSCRIPTION" az vmss extension set --resource-group "$AZ_RG" --vmss-name "$SCALE_SET_NAME" --name VMAccessForLinux --publisher Microsoft.OSTCExtensions --version 1.4 --protected-settings "{\"reset_ssh\": true, \"username\": \"azureuser\", \"password\": \"$RANDOM_PWD\", \"ssh_key\": \"$(cat ~/.ssh/id_rsa.pub)\"}" ( set -x ; az vmss update-instances --instance-ids '*' --resource-group "$AZ_RG" --name "$SCALE_SET_NAME" )
Then get the ip of the node with $ kubectl get nodes -o wide
Launch container in k8s cluster
$ kubectl run -i --tty --rm MyContainer -generator="run-pod/v1" --image=debian /bin/bash # apt-get update && apt-get install openssh-client -y
Copy pvt ssh key into container $ kubectl cp ~/.ssh/id_rsa MyContainer-xxx:/id_rsa
ssh to node. # ssh -i /id_rsa azureuser@10.240.0.x
Test
- az - setup proxy tunnel to web admin
az aks get-credentials --resource-group K8S-xxx --name K8S-xxx
Merged "K8S-INF" as current context in /root/.kube/config
- kubectl get nodes
Create azure-vote.yml from https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough
kubectl apply -f azure-vote.yaml
deployment.apps/azure-vote-back created service/azure-vote-back created deployment.apps/azure-vote-front created service/azure-vote-front created bash-4.4#
kubectl get service azure-vote-front --watch
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE azure-vote-front LoadBalancer 10.0.61.46 1.5.1.39 80:31745/TCP 101s
Own namespace
- Created with json file that gets deployed
Set default namespace to mynamespace e.g.
# kubectl config set-context $(kubectl config current-context) --namespace=MyNameSpace # kubectl config view | grep namespace
Perf monitoring
- kubectl top nodes
- kubectl describe nodes
- kubectl top pod
Prometheus - https://github.com/google-cloud-tools/kube-eagle
AZ check clusters and PSP/NetPolicy
export azure subscription and run
export azsub=--subscription \"Non-Prod\" " az aks list $azsub | jq "[ .[] | {name: .name, k8sV: .kubernetesVersion, sp: .servicePrincipalProfile.clientId, NetPolicy: .networkProfile.networkPolicy, Nodes: ((.agentPoolProfiles[0].count|tostring) + \"/\" + ( .agentPoolProfiles[0].maxCount|tostring ) + \" \" + .agentPoolProfiles[0].provisioningState), psp: .enablePodSecurityPolicy, rbac: .enableRbac, apiAuth: .apiServerAccessProfile.authorizedIpRanges, }]"
...