Note for k8s the hard way

Tools
 cfssl, cfssljson, and kubectl.

Provision servers
 create vpc, subnet, internet gateway, route table, security group, nlb
 create ssh key
 create 3 controllers
 create 3 workers

CA, TLS certificates
 create CA
 create client, server certificates
  - admin client
  - kubelet client
  - controller manager client
  - kube proxy client
  - scheduler client
  - kubernetes api server
  - Service Account key pair

Distribute Client and Server certificates
  scp ca.pem, worker[i]-key.pem worker[i].pem to each worker
  scp ca-key.pem, kubernetes-key.pem kubernetes.pem, service-account-key.pem, service-account.pem to each controller

Kubernetes Configuration for authentication
  generate kubeconfig files for `controller manager`, `kubelet`, `kube-proxy`, `scheduler` clients and the `admin` user

  get KUBERNETES_PUBLIC_ADDRESS (loadbalancer)
  generate worker kubeconfig using node0,1,2 private key and k8s public ip (worker-0.kubeconfig, worker-1.kubeconfig, worker-2.kubeconfig)
  generate kube-proxy kubeconfig using kube-proxy private key and k8s public ip
  generate kube-controller-manager using kube-controller-manager private key and k8s public ip
  genrate kube-scheduler using kube-scheduler private key and k8s public ip
  generate kubeconfig for `admin` user

Distribute kubconfig files
  scp kubeconfig for kubelet and kube-proxy to each worker
  scp kubeconfig for kube-controller-manager and kube-scheduler to each controller


Create the Data Encryption Config and key
  generate an encryption key and create EncryptionConfig
  distribute to each controller

Bootstrapping etcd cluster
  k8s stores cluster state in etcd

  each controller
    install etcd binary files
    cp ca.pem, kubernetes-key.pem
    get instance private ip
    create etcd.service with parameters, keys, private ip
    start etcd service

  once etcd in all controller installed, verify by listing etcd cluster member

Bootstrapping control plane
  install these binaries, kube-api, kube-controller-manager, kube-scheduler, kubectl
  cp ca.pem, kubernetes-key.pem, kubernetes.pem, service-account.pem, service-account.pem, encryption-config.yaml
  get private ip
  create services for those conponents
  start service and verify kubectl get componentstatuses

RBAC for Kubelet Authorization
  configure RBAC permissions to allow the Kubernetes API Server to access the Kubelet API on each worker node

Bootstrapping control plane
  install worker binaries
  configure CNI network
  configure Containerd
  configure Kubelet
  configure Kube-proxy
  start worker services
  verify node `kubectl get nodes --kubeconfig admin.kubeconfig`

Configure kubectl for remote access
  set kubeconfig and verify `kubectl get componentstatuses` and `kubectl get nodes`

Provisioning Pod Network Routes
  create Route Table and Route in aws
  validate Routes

Deploy DNS cluster add on
  install kube DNS
  run dns lookup from Pod

Smoke Test
  Data encryption
  Deployment
    Portforward
    Logs
    Exec
  Services
  Untrusted Workloads
  Check images/pods/containers on worker nodes using crictl

Cleanup