Kubernets cluster in Azure cloud

Links

• k8s/helm

• Quickstart: AKS cluster

• 2018-Helm in Azure Kubernetes AKS

• cheatsheet

• Azure KeyVault exposed to k8s pods as flexVol

Kubernets config

• Use Declarative, we declare the state and kubectl implements using

  kubectl apply -R -f configs/

Setup Cluster, using the Azure az commands and azure aks install-cli kubectl

• Use az tool, with docker run -it microsoft/azure-cli

• in the container add the kubectl

  az aks install-cli
  az aks get-credentials --resource-group <RG> --name <name> --subscription <Hex-ID>

• list subscriptions

  az account list --output table 

• set subscription to the one that contains k8s

  az account set --subscription xx-xx-xx

• run az proxy to connect the browser to kubernets admin in cloud

  Proxy running on http://127.0.0.1:8001/
  Press CTRL+C to close the tunnel...
  Forwarding from 127.0.0.1:8001 -> 9090
  
  ## Problem only binds to loopback, in a container, if not using container for microsoft/azure-cli skip next command.
  nc -v -lk -p 8001 -s $(hostname -i) -e /usr/bin/nc 127.0.0.1 8001
  
  ## Web dashboard no right - nodes is forbidden: User "system:serviceaccount:kube-system:kubernetes-dashboard" cannot list nodes at the cluster scope
  ## RBAC ClusterRoleBinding must be created for Kubernetes dashboard
  kubectl create clusterrolebinding kubernetes-dashboard \
      --clusterrole=cluster-admin \
      --serviceaccount=kube-system:kubernetes-dashboard

  • List nodes

    kubectl get nodes

    • If this fails with "Unable to connect to the server: dial tcp: lookup ...." reset with

      rm .kube/config
      az aks get-credentials --resource-group <nameRG> --name >nameClusterInRG>
      kubectl get nodes

  • List namespaces

    kubectl get namespaces

Reset ssh key and password to get access to k8s Node in Azure

CLUSTER_RESOURCE_GROUP=aks-<xxxx>-nodes
RANDOM_PWD=$( ( head /dev/urandom ; date +%s) | sha256sum | base64 | head -c32 )

SCALE_SET_NAME=$(az vmss list --resource-group $CLUSTER_RESOURCE_GROUP --query [0].name -o tsv)
echo "SCALE_SET_NAME=$SCALE_SET_NAME"

az vmss extension set --resource-group "$CLUSTER_RESOURCE_GROUP" --vmss-name "$SCALE_SET_NAME" --name VMAccessForLinux --publisher Microsoft.OSTCExtensions --version 1.4 --protected-settings "{\"reset_ssh\": true, \"username\": \"azureuser\", \"password\": \"$RANDOM_PWD\", \"ssh_key\": \"$(cat ~/.ssh/id_rsa.pub)\"}"

az vmss update-instances --instance-ids '*' --resource-group "$CLUSTER_RESOURCE_GROUP" --name "$SCALE_SET_NAME"

• Then get the ip of the node with  $ kubectl get nodes -o wide 

• Lauche container in k8s cluster

  $ kubectl run -i --tty --rm MyContainer --image=debian
  # apt-get update && apt-get install openssh-client -y

• Copy pvt ssh key into container   $ kubectl cp ~/.ssh/id_rsa MyContainer-xxx:/id_rsa 

• ssh to node.  # ssh -i /id_rsa azureuser@10.240.0.x 

Test

• az - setup proxy tunnel to web admin

  • az aks get-credentials --resource-group K8S-xxx --name K8S-xxx

    Merged "K8S-INF" as current context in /root/.kube/config 

  • kubectl get nodes

  • Create azure-vote.yml from https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough

  • kubectl apply -f azure-vote.yaml

    deployment.apps/azure-vote-back created
    service/azure-vote-back created
    deployment.apps/azure-vote-front created
    service/azure-vote-front created
    bash-4.4# 

  • kubectl get service azure-vote-front --watch

    NAME               TYPE           CLUSTER-IP   EXTERNAL-IP     PORT(S)        AGE
    azure-vote-front   LoadBalancer   10.0.61.46   1.5.1.39        80:31745/TCP   101s

Own namespace

• Created with json file that gets deployed

• Set default namespace to mynamespace e.g.

  # kubectl config set-context $(kubectl config current-context) --namespace=MyNameSpace
  # kubectl config view | grep namespace

Perf monitoring

• kubectl top nodes
• kubectl describe nodes
• kubectl top pod

• Prometheus - https://github.com/google-cloud-tools/kube-eagle

AZ check clusters and PSP/NetPolicy

• export azure subscription and run

  export azsub=--subscription \"Non-Prod\" "
  az aks list $azsub | jq "[ .[] | 
          {name: .name, k8sV: .kubernetesVersion,
           sp: .servicePrincipalProfile.clientId,
           NetPolicy: .networkProfile.networkPolicy,
           Nodes: ((.agentPoolProfiles[0].count|tostring) + \"/\" + ( .agentPoolProfiles[0].maxCount|tostring ) + \" \" + .agentPoolProfiles[0].provisioningState),
           psp: .enablePodSecurityPolicy,
           rbac: .enableRbac,
           apiAuth: .apiServerAccessProfile.authorizedIpRanges,
           }]"

...

CategoryK8sKubernetes CategoryK8sKubernetes