API tools faq
paste
Guest User

k3d v3.0.2 - leaderelection lost

a guest
Sep 20th, 2020
168
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Bash 117.54 KB | None | 0 0
raw download clone embed print report
  1. $ docker logs k3d-testing-server-0
  2. time="2020-09-20T15:12:13.117426111Z" level=info msg="Starting k3s v1.18.6+k3s1 (6f56fa1d)"
  3. time="2020-09-20T15:12:13.652207951Z" level=info msg="Testing connection to peers [172.27.0.2:6443]"
  4. time="2020-09-20T15:12:13.654267205Z" level=info msg="Active TLS secret  (ver=) (count 8): map[listener.cattle.io/cn-0.0.0.0:0.0.0.0 listener.cattle.io/cn-10.43.0.1:10.43.0.1 listener.cattle.io/cn-127.0.0.1:127.0.0.1 listener.cattle.io/cn-172.27.0.2:172.27.0.2 listener.cattle.io/cn-kubernetes:kubernetes listener.cattle.io/cn-kubernetes.default:kubernetes.default listener.cattle.io/cn-kubernetes.default.svc.cluster.local:kubernetes.default.svc.cluster.local listener.cattle.io/cn-localhost:localhost listener.cattle.io/hash:975a58fa049171a3cd99dc91a2a922ec83aeee121b28cf3ca23262809bdf0215]"
  5. time="2020-09-20T15:12:13.654888096Z" level=info msg="Connection OK to peers [172.27.0.2:6443]"
  6. time="2020-09-20T15:12:13.730259446Z" level=info msg="Kine listening on unix://kine.sock"
  7. time="2020-09-20T15:12:13.735182186Z" level=info msg="Running kube-apiserver --advertise-port=6443 --allow-privileged=true --anonymous-auth=false --api-audiences=unknown --authorization-mode=Node,RBAC --basic-auth-file=/var/lib/rancher/k3s/server/cred/passwd --bind-address=127.0.0.1 --cert-dir=/var/lib/rancher/k3s/server/tls/temporary-certs --client-ca-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --enable-admission-plugins=NodeRestriction --etcd-servers=unix://kine.sock --insecure-port=0 --kubelet-certificate-authority=/var/lib/rancher/k3s/server/tls/server-ca.crt --kubelet-client-certificate=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.crt --kubelet-client-key=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.key --proxy-client-cert-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt --proxy-client-key-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.key --requestheader-allowed-names=system:auth-proxy --requestheader-client-ca-file=/var/lib/rancher/k3s/server/tls/request-header-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6444 --service-account-issuer=k3s --service-account-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-account-signing-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-cluster-ip-range=10.43.0.0/16 --storage-backend=etcd3 --tls-cert-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt --tls-private-key-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key"
  8. Flag --basic-auth-file has been deprecated, Basic authentication mode is deprecated and will be removed in a future release. It is not recommended for production environments.
  9. I0920 15:12:13.738807       6 server.go:645] external host was not specified, using 172.27.0.2
  10. I0920 15:12:13.740305       6 server.go:162] Version: v1.18.6+k3s1
  11. I0920 15:12:14.749462       6 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook.
  12. I0920 15:12:14.749869       6 plugins.go:161] Loaded 10 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,CertificateSubjectRestriction,ValidatingAdmissionWebhook,ResourceQuota.
  13. I0920 15:12:14.752571       6 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook.
  14. I0920 15:12:14.752863       6 plugins.go:161] Loaded 10 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,CertificateSubjectRestriction,ValidatingAdmissionWebhook,ResourceQuota.
  15. I0920 15:12:14.816070       6 master.go:270] Using reconciler: lease
  16. I0920 15:12:14.949471       6 rest.go:113] the default service ipfamily for this cluster is: IPv4
  17. W0920 15:12:15.880067       6 genericapiserver.go:409] Skipping API batch/v2alpha1 because it has no resources.
  18. W0920 15:12:15.901340       6 genericapiserver.go:409] Skipping API discovery.k8s.io/v1alpha1 because it has no resources.
  19. W0920 15:12:15.926339       6 genericapiserver.go:409] Skipping API node.k8s.io/v1alpha1 because it has no resources.
  20. W0920 15:12:15.963945       6 genericapiserver.go:409] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources.
  21. W0920 15:12:15.973106       6 genericapiserver.go:409] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources.
  22. W0920 15:12:16.009749       6 genericapiserver.go:409] Skipping API storage.k8s.io/v1alpha1 because it has no resources.
  23. W0920 15:12:16.059416       6 genericapiserver.go:409] Skipping API apps/v1beta2 because it has no resources.
  24. W0920 15:12:16.059724       6 genericapiserver.go:409] Skipping API apps/v1beta1 because it has no resources.
  25. I0920 15:12:16.082412       6 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook.
  26. I0920 15:12:16.082741       6 plugins.go:161] Loaded 10 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,CertificateSubjectRestriction,ValidatingAdmissionWebhook,ResourceQuota.
  27. I0920 15:12:20.202643       6 secure_serving.go:178] Serving securely on 127.0.0.1:6444
  28. I0920 15:12:20.203354       6 apiservice_controller.go:94] Starting APIServiceRegistrationController
  29. I0920 15:12:20.203534       6 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller
  30. I0920 15:12:20.203824       6 dynamic_cafile_content.go:167] Starting request-header::/var/lib/rancher/k3s/server/tls/request-header-ca.crt
  31. I0920 15:12:20.203968       6 dynamic_serving_content.go:130] Starting serving-cert::/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt::/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key
  32. I0920 15:12:20.204095       6 tlsconfig.go:240] Starting DynamicServingCertificateController
  33. I0920 15:12:20.209518       6 cluster_authentication_trust_controller.go:440] Starting cluster_authentication_trust_controller controller
  34. I0920 15:12:20.209902       6 shared_informer.go:223] Waiting for caches to sync for cluster_authentication_trust_controller
  35. I0920 15:12:20.211083       6 controller.go:81] Starting OpenAPI AggregationController
  36. I0920 15:12:20.211392       6 available_controller.go:387] Starting AvailableConditionController
  37. I0920 15:12:20.211532       6 cache.go:32] Waiting for caches to sync for AvailableConditionController controller
  38. I0920 15:12:20.211726       6 autoregister_controller.go:141] Starting autoregister controller
  39. I0920 15:12:20.211860       6 cache.go:32] Waiting for caches to sync for autoregister controller
  40. I0920 15:12:20.212754       6 crd_finalizer.go:266] Starting CRDFinalizer
  41. I0920 15:12:20.213040       6 dynamic_cafile_content.go:167] Starting client-ca-bundle::/var/lib/rancher/k3s/server/tls/client-ca.crt
  42. I0920 15:12:20.213884       6 dynamic_cafile_content.go:167] Starting client-ca-bundle::/var/lib/rancher/k3s/server/tls/client-ca.crt
  43. I0920 15:12:20.214125       6 dynamic_cafile_content.go:167] Starting request-header::/var/lib/rancher/k3s/server/tls/request-header-ca.crt
  44. I0920 15:12:20.223704       6 crdregistration_controller.go:111] Starting crd-autoregister controller
  45. I0920 15:12:20.224038       6 shared_informer.go:223] Waiting for caches to sync for crd-autoregister
  46. I0920 15:12:20.224329       6 controller.go:86] Starting OpenAPI controller
  47. I0920 15:12:20.224494       6 customresource_discovery_controller.go:209] Starting DiscoveryController
  48. I0920 15:12:20.224664       6 naming_controller.go:291] Starting NamingConditionController
  49. I0920 15:12:20.224814       6 establishing_controller.go:76] Starting EstablishingController
  50. I0920 15:12:20.224974       6 nonstructuralschema_controller.go:186] Starting NonStructuralSchemaConditionController
  51. I0920 15:12:20.225125       6 apiapproval_controller.go:186] Starting KubernetesAPIApprovalPolicyConformantConditionController
  52. I0920 15:12:20.408210       6 cache.go:39] Caches are synced for APIServiceRegistrationController controller
  53. I0920 15:12:20.410247       6 shared_informer.go:230] Caches are synced for cluster_authentication_trust_controller
  54. I0920 15:12:20.422100       6 cache.go:39] Caches are synced for AvailableConditionController controller
  55. I0920 15:12:20.422498       6 cache.go:39] Caches are synced for autoregister controller
  56. I0920 15:12:20.424310       6 shared_informer.go:230] Caches are synced for crd-autoregister
  57. E0920 15:12:20.530802       6 controller.go:151] Unable to perform initial Kubernetes service initialization: Service "kubernetes" is invalid: spec.clusterIP: Invalid value: "10.43.0.1": cannot allocate resources of type serviceipallocations at this time
  58. E0920 15:12:20.538311       6 controller.go:156] Unable to remove old endpoints from kubernetes service: StorageError: key not found, Code: 1, Key: /registry/masterleases/172.27.0.2, ResourceVersion: 0, AdditionalErrorMsg:
  59. I0920 15:12:21.201667       6 controller.go:130] OpenAPI AggregationController: action for item : Nothing (removed from the queue).
  60. I0920 15:12:21.202375       6 controller.go:130] OpenAPI AggregationController: action for item k8s_internal_local_delegation_chain_0000000000: Nothing (removed from the queue).
  61. I0920 15:12:21.254192       6 storage_scheduling.go:134] created PriorityClass system-node-critical with value 2000001000
  62. I0920 15:12:21.268138       6 storage_scheduling.go:134] created PriorityClass system-cluster-critical with value 2000000000
  63. I0920 15:12:21.268645       6 storage_scheduling.go:143] all system priority classes are created successfully or already exist.
  64. I0920 15:12:23.072752       6 controller.go:606] quota admission added evaluator for: roles.rbac.authorization.k8s.io
  65. I0920 15:12:23.215974       6 controller.go:606] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
  66. I0920 15:12:23.401364       6 registry.go:150] Registering EvenPodsSpread predicate and priority function
  67. I0920 15:12:23.401759       6 registry.go:150] Registering EvenPodsSpread predicate and priority function
  68. time="2020-09-20T15:12:23.405652313Z" level=info msg="Running kube-scheduler --bind-address=127.0.0.1 --kubeconfig=/var/lib/rancher/k3s/server/cred/scheduler.kubeconfig --port=10251 --secure-port=0"
  69. time="2020-09-20T15:12:23.416307385Z" level=info msg="Running kube-controller-manager --allocate-node-cidrs=true --bind-address=127.0.0.1 --cluster-cidr=10.42.0.0/16 --cluster-signing-cert-file=/var/lib/rancher/k3s/server/tls/server-ca.crt --cluster-signing-key-file=/var/lib/rancher/k3s/server/tls/server-ca.key --kubeconfig=/var/lib/rancher/k3s/server/cred/controller.kubeconfig --port=10252 --root-ca-file=/var/lib/rancher/k3s/server/tls/server-ca.crt --secure-port=0 --service-account-private-key-file=/var/lib/rancher/k3s/server/tls/service.key --use-service-account-credentials=true"
  70. I0920 15:12:23.423918       6 controllermanager.go:161] Version: v1.18.6+k3s1
  71. I0920 15:12:23.425433       6 deprecated_insecure_serving.go:53] Serving insecurely on [::]:10252
  72. I0920 15:12:23.425719       6 leaderelection.go:242] attempting to acquire leader lease  kube-system/kube-controller-manager...
  73. time="2020-09-20T15:12:23.473643439Z" level=info msg="Waiting for cloudcontroller rbac role to be created"
  74. I0920 15:12:23.477963       6 registry.go:150] Registering EvenPodsSpread predicate and priority function
  75. I0920 15:12:23.478312       6 registry.go:150] Registering EvenPodsSpread predicate and priority function
  76. W0920 15:12:23.480662       6 authorization.go:47] Authorization is disabled
  77. W0920 15:12:23.481458       6 authentication.go:40] Authentication is disabled
  78. I0920 15:12:23.481691       6 deprecated_insecure_serving.go:51] Serving healthz insecurely on [::]:10251
  79. time="2020-09-20T15:12:23.518726913Z" level=info msg="Creating CRD addons.k3s.cattle.io"
  80. I0920 15:12:23.521773       6 controller.go:606] quota admission added evaluator for: endpoints
  81. I0920 15:12:23.549752       6 controller.go:606] quota admission added evaluator for: leases.coordination.k8s.io
  82. time="2020-09-20T15:12:23.559111916Z" level=info msg="Creating CRD helmcharts.helm.cattle.io"
  83. I0920 15:12:23.590393       6 leaderelection.go:242] attempting to acquire leader lease  kube-system/kube-scheduler...
  84. W0920 15:12:23.611531       6 lease.go:224] Resetting endpoints for master service "kubernetes" to [172.27.0.2]
  85. time="2020-09-20T15:12:23.659969469Z" level=info msg="Waiting for CRD addons.k3s.cattle.io to become available"
  86. I0920 15:12:23.664660       6 leaderelection.go:252] successfully acquired lease kube-system/kube-controller-manager
  87. I0920 15:12:23.666486       6 event.go:278] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"kube-controller-manager", UID:"5c883b80-f3e3-40d3-b7c8-693ad7dd085c", APIVersion:"v1", ResourceVersion:"152", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' k3d-testing-server-0_392478b6-b490-4800-b192-75aa4977e926 became leader
  88. I0920 15:12:23.666870       6 event.go:278] Event(v1.ObjectReference{Kind:"Lease", Namespace:"kube-system", Name:"kube-controller-manager", UID:"c321b345-bd4b-431a-955e-823608a6b3c1", APIVersion:"coordination.k8s.io/v1", ResourceVersion:"158", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' k3d-testing-server-0_392478b6-b490-4800-b192-75aa4977e926 became leader
  89. I0920 15:12:23.912999       6 leaderelection.go:252] successfully acquired lease kube-system/kube-scheduler
  90. I0920 15:12:23.961259       6 controller.go:606] quota admission added evaluator for: endpointslices.discovery.k8s.io
  91. time="2020-09-20T15:12:24.164999831Z" level=info msg="Done waiting for CRD addons.k3s.cattle.io to become available"
  92. time="2020-09-20T15:12:24.165416189Z" level=info msg="Waiting for CRD helmcharts.helm.cattle.io to become available"
  93. time="2020-09-20T15:12:24.683732794Z" level=info msg="Waiting for cloudcontroller rbac role to be created"
  94. time="2020-09-20T15:12:24.925199333Z" level=info msg="Done waiting for CRD helmcharts.helm.cattle.io to become available"
  95. time="2020-09-20T15:12:24.976406821Z" level=info msg="Writing static file: /var/lib/rancher/k3s/server/static/charts/traefik-1.81.0.tgz"
  96. time="2020-09-20T15:12:24.978539907Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/coredns.yaml"
  97. time="2020-09-20T15:12:24.979309857Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/aggregated-metrics-reader.yaml"
  98. time="2020-09-20T15:12:24.979810541Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/resource-reader.yaml"
  99. time="2020-09-20T15:12:24.980343919Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/rolebindings.yaml"
  100. time="2020-09-20T15:12:24.980743966Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/ccm.yaml"
  101. time="2020-09-20T15:12:24.981232243Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/local-storage.yaml"
  102. time="2020-09-20T15:12:24.981710530Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/auth-delegator.yaml"
  103. time="2020-09-20T15:12:24.982125912Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/auth-reader.yaml"
  104. time="2020-09-20T15:12:24.982534767Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/metrics-apiservice.yaml"
  105. time="2020-09-20T15:12:24.983030848Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/metrics-server-deployment.yaml"
  106. time="2020-09-20T15:12:24.983528945Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/metrics-server-service.yaml"
  107. time="2020-09-20T15:12:24.983948499Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/traefik.yaml"
  108. I0920 15:12:25.106413       6 request.go:621] Throttling request took 1.001938999s, request: GET:https://127.0.0.1:6444/apis/policy/v1beta1?timeout=32s
  109. time="2020-09-20T15:12:25.330769668Z" level=info msg="Node token is available at /var/lib/rancher/k3s/server/token"
  110. time="2020-09-20T15:12:25.331222526Z" level=info msg="To join node to cluster: k3s agent -s https://172.27.0.2:6443 -t ${NODE_TOKEN}"
  111. time="2020-09-20T15:12:25.332391862Z" level=info msg="Starting k3s.cattle.io/v1, Kind=Addon controller"
  112. time="2020-09-20T15:12:25.334538149Z" level=info msg="Waiting for master node  startup: resource name may not be empty"
  113. I0920 15:12:25.334905       6 leaderelection.go:242] attempting to acquire leader lease  kube-system/k3s...
  114. 2020-09-20 15:12:25.442799 I | http: TLS handshake error from 127.0.0.1:48468: remote error: tls: bad certificate
  115. I0920 15:12:25.577046       6 leaderelection.go:252] successfully acquired lease kube-system/k3s
  116. time="2020-09-20T15:12:25.582035488Z" level=info msg="Wrote kubeconfig /output/kubeconfig.yaml"
  117. time="2020-09-20T15:12:25.582391799Z" level=info msg="Run: k3s kubectl"
  118. time="2020-09-20T15:12:25.582604063Z" level=info msg="k3s is up and running"
  119. time="2020-09-20T15:12:25.583355113Z" level=info msg="module overlay was already loaded"
  120. time="2020-09-20T15:12:25.583715325Z" level=info msg="module nf_conntrack was already loaded"
  121. time="2020-09-20T15:12:25.583981565Z" level=info msg="module br_netfilter was already loaded"
  122. 2020-09-20 15:12:25.711329 I | http: TLS handshake error from 127.0.0.1:48476: remote error: tls: bad certificate
  123. time="2020-09-20T15:12:25.799684475Z" level=info msg="Waiting for cloudcontroller rbac role to be created"
  124. I0920 15:12:25.840757       6 controller.go:606] quota admission added evaluator for: addons.k3s.cattle.io
  125. I0920 15:12:25.859869       6 plugins.go:100] No cloud provider specified.
  126. I0920 15:12:25.864648       6 shared_informer.go:223] Waiting for caches to sync for tokens
  127. 2020-09-20 15:12:25.869011 I | http: TLS handshake error from 127.0.0.1:48482: remote error: tls: bad certificate
  128. time="2020-09-20T15:12:25.904324239Z" level=error msg="--disable-agent is not compatible with dqlite"
  129. time="2020-09-20T15:12:25.904805052Z" level=info msg="Starting helm.cattle.io/v1, Kind=HelmChart controller"
  130. time="2020-09-20T15:12:25.904979263Z" level=info msg="Starting batch/v1, Kind=Job controller"
  131. time="2020-09-20T15:12:25.905123843Z" level=info msg="Starting /v1, Kind=Node controller"
  132. time="2020-09-20T15:12:25.905248024Z" level=info msg="Starting /v1, Kind=Service controller"
  133. time="2020-09-20T15:12:25.905469821Z" level=info msg="Starting /v1, Kind=Pod controller"
  134. time="2020-09-20T15:12:25.905693834Z" level=info msg="Starting /v1, Kind=Endpoints controller"
  135. time="2020-09-20T15:12:25.905913638Z" level=info msg="Starting /v1, Kind=Secret controller"
  136. I0920 15:12:25.911431       6 controller.go:606] quota admission added evaluator for: serviceaccounts
  137. time="2020-09-20T15:12:25.915929081Z" level=info msg="Active TLS secret k3s-serving (ver=176) (count 8): map[listener.cattle.io/cn-0.0.0.0:0.0.0.0 listener.cattle.io/cn-10.43.0.1:10.43.0.1 listener.cattle.io/cn-127.0.0.1:127.0.0.1 listener.cattle.io/cn-172.27.0.2:172.27.0.2 listener.cattle.io/cn-kubernetes:kubernetes listener.cattle.io/cn-kubernetes.default:kubernetes.default listener.cattle.io/cn-kubernetes.default.svc.cluster.local:kubernetes.default.svc.cluster.local listener.cattle.io/cn-localhost:localhost listener.cattle.io/hash:975a58fa049171a3cd99dc91a2a922ec83aeee121b28cf3ca23262809bdf0215]"
  138. I0920 15:12:25.972763       6 shared_informer.go:230] Caches are synced for tokens
  139. I0920 15:12:25.977529       6 controllermanager.go:533] Started "disruption"
  140. I0920 15:12:25.978114       6 disruption.go:331] Starting disruption controller
  141. I0920 15:12:25.978327       6 shared_informer.go:223] Waiting for caches to sync for disruption
  142. time="2020-09-20T15:12:26.081113941Z" level=info msg="Logging containerd to /var/lib/rancher/k3s/agent/containerd/containerd.log"
  143. time="2020-09-20T15:12:26.082663777Z" level=info msg="Running containerd -c /var/lib/rancher/k3s/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /var/lib/rancher/k3s/agent/containerd"
  144. time="2020-09-20T15:12:26.095355035Z" level=info msg="Waiting for containerd startup: rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial unix /run/k3s/containerd/containerd.sock: connect: no such file or directory\""
  145. I0920 15:12:26.219740       6 controllermanager.go:533] Started "cronjob"
  146. W0920 15:12:26.221325       6 controllermanager.go:512] "tokencleaner" is disabled
  147. I0920 15:12:26.222421       6 cronjob_controller.go:97] Starting CronJob Manager
  148. time="2020-09-20T15:12:26.419595884Z" level=info msg="Waiting for master node k3d-testing-server-0 startup: nodes \"k3d-testing-server-0\" not found"
  149. I0920 15:12:26.488213       6 controllermanager.go:533] Started "serviceaccount"
  150. I0920 15:12:26.493809       6 serviceaccounts_controller.go:117] Starting service account controller
  151. I0920 15:12:26.494241       6 shared_informer.go:223] Waiting for caches to sync for service account
  152. I0920 15:12:26.589426       6 controller.go:606] quota admission added evaluator for: deployments.apps
  153. I0920 15:12:26.618631       6 controllermanager.go:533] Started "podgc"
  154. I0920 15:12:26.623739       6 gc_controller.go:89] Starting GC controller
  155. I0920 15:12:26.624035       6 shared_informer.go:223] Waiting for caches to sync for GC
  156. I0920 15:12:26.753835       6 controllermanager.go:533] Started "statefulset"
  157. I0920 15:12:26.755034       6 stateful_set.go:146] Starting stateful set controller
  158. I0920 15:12:26.755265       6 shared_informer.go:223] Waiting for caches to sync for stateful set
  159. I0920 15:12:26.839127       6 controllermanager.go:533] Started "csrsigning"
  160. I0920 15:12:26.840073       6 certificate_controller.go:119] Starting certificate controller "csrsigning"
  161. I0920 15:12:26.841333       6 shared_informer.go:223] Waiting for caches to sync for certificate-csrsigning
  162. I0920 15:12:26.842129       6 dynamic_serving_content.go:130] Starting csr-controller::/var/lib/rancher/k3s/server/tls/server-ca.crt::/var/lib/rancher/k3s/server/tls/server-ca.key
  163. time="2020-09-20T15:12:26.852312977Z" level=info msg="Running cloud-controller-manager --allocate-node-cidrs=true --allow-untagged-cloud=true --bind-address=127.0.0.1 --cloud-provider=k3s --cluster-cidr=10.42.0.0/16 --kubeconfig=/var/lib/rancher/k3s/server/cred/cloud-controller.kubeconfig --node-status-update-frequency=1m --secure-port=0"
  164. Flag --allow-untagged-cloud has been deprecated, This flag is deprecated and will be removed in a future release. A cluster-id will be required on cloud instances.
  165. I0920 15:12:26.886445       6 controllermanager.go:120] Version: v1.18.6+k3s1
  166. W0920 15:12:26.889053       6 controllermanager.go:132] detected a cluster without a ClusterID.  A ClusterID will be required in the future.  Please tag your cluster to avoid any future issues
  167. I0920 15:12:26.889455       6 leaderelection.go:242] attempting to acquire leader lease  kube-system/cloud-controller-manager...
  168. W0920 15:12:27.039884       6 probe.go:268] Flexvolume plugin directory at /usr/libexec/kubernetes/kubelet-plugins/volume/exec/ does not exist. Recreating.
  169. I0920 15:12:27.054072       6 controllermanager.go:533] Started "attachdetach"
  170. I0920 15:12:27.055172       6 attach_detach_controller.go:348] Starting attach detach controller
  171. I0920 15:12:27.055506       6 shared_informer.go:223] Waiting for caches to sync for attach detach
  172. time="2020-09-20T15:12:27.115717746Z" level=info msg="Waiting for containerd startup: rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial unix /run/k3s/containerd/containerd.sock: connect: no such file or directory\""
  173. I0920 15:12:27.262205       6 leaderelection.go:252] successfully acquired lease kube-system/cloud-controller-manager
  174. I0920 15:12:27.271974       6 node_controller.go:110] Sending events to api server.
  175. I0920 15:12:27.284494       6 controllermanager.go:247] Started "cloud-node"
  176. I0920 15:12:27.279499       6 controllermanager.go:533] Started "endpointslice"
  177. I0920 15:12:27.279538       6 endpointslice_controller.go:213] Starting endpoint slice controller
  178. I0920 15:12:27.282582       6 event.go:278] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"cloud-controller-manager", UID:"fa2e82d7-b032-4496-aa43-0b378a4cabc1", APIVersion:"v1", ResourceVersion:"211", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' k3d-testing-server-0_b0955284-487c-45ed-893e-ba43673a43d9 became leader
  179. I0920 15:12:27.300907       6 node_lifecycle_controller.go:78] Sending events to api server
  180. I0920 15:12:27.306617       6 shared_informer.go:223] Waiting for caches to sync for endpoint_slice
  181. I0920 15:12:27.306786       6 event.go:278] Event(v1.ObjectReference{Kind:"Lease", Namespace:"kube-system", Name:"cloud-controller-manager", UID:"91e8634f-076a-4406-91a9-fdcf332dcdf2", APIVersion:"coordination.k8s.io/v1", ResourceVersion:"216", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' k3d-testing-server-0_b0955284-487c-45ed-893e-ba43673a43d9 became leader
  182. I0920 15:12:27.307087       6 controllermanager.go:247] Started "cloud-node-lifecycle"
  183. E0920 15:12:27.310085       6 core.go:90] Failed to start service controller: the cloud provider does not support external load balancers
  184. W0920 15:12:27.310271       6 controllermanager.go:244] Skipping "service"
  185. W0920 15:12:27.310363       6 core.go:108] configure-cloud-routes is set, but cloud provider does not support routes. Will not configure cloud provider routes.
  186. W0920 15:12:27.310451       6 controllermanager.go:244] Skipping "route"
  187. I0920 15:12:27.343534       6 controllermanager.go:533] Started "csrcleaner"
  188. I0920 15:12:27.345590       6 cleaner.go:82] Starting CSR cleaner controller
  189. time="2020-09-20T15:12:27.605799160Z" level=info msg="Waiting for master node k3d-testing-server-0 startup: nodes \"k3d-testing-server-0\" not found"
  190. E0920 15:12:27.671093       6 core.go:89] Failed to start service controller: WARNING: no cloud provider provided, services of type LoadBalancer will fail
  191. W0920 15:12:27.671432       6 controllermanager.go:525] Skipping "service"
  192. I0920 15:12:27.812463       6 controllermanager.go:533] Started "pv-protection"
  193. W0920 15:12:27.812762       6 controllermanager.go:525] Skipping "root-ca-cert-publisher"
  194. I0920 15:12:27.813192       6 pv_protection_controller.go:83] Starting PV protection controller
  195. I0920 15:12:27.813377       6 shared_informer.go:223] Waiting for caches to sync for PV protection
  196. I0920 15:12:27.942234       6 controllermanager.go:533] Started "endpoint"
  197. I0920 15:12:27.943177       6 endpoints_controller.go:182] Starting endpoint controller
  198. I0920 15:12:27.943352       6 shared_informer.go:223] Waiting for caches to sync for endpoint
  199. time="2020-09-20T15:12:28.194636641Z" level=warning msg="Disabling CPU quotas due to missing cpu.cfs_period_us"
  200. time="2020-09-20T15:12:28.201740479Z" level=info msg="Running kubelet --address=0.0.0.0 --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=cgroupfs --client-ca-file=/var/lib/rancher/k3s/agent/client-ca.crt --cloud-provider=external --cluster-dns=10.43.0.10 --cluster-domain=cluster.local --cni-bin-dir=/bin --cni-conf-dir=/var/lib/rancher/k3s/agent/etc/cni/net.d --container-runtime-endpoint=/run/k3s/containerd/containerd.sock --container-runtime=remote --containerd=/run/k3s/containerd/containerd.sock --cpu-cfs-quota=false --eviction-hard=imagefs.available<5%,nodefs.available<5% --eviction-minimum-reclaim=imagefs.available=10%,nodefs.available=10% --fail-swap-on=false --healthz-bind-address=127.0.0.1 --hostname-override=k3d-testing-server-0 --kubeconfig=/var/lib/rancher/k3s/agent/kubelet.kubeconfig --kubelet-cgroups=/systemd --node-labels= --read-only-port=0 --resolv-conf=/tmp/k3s-resolv.conf --runtime-cgroups=/systemd --serialize-image-pulls=false --tls-cert-file=/var/lib/rancher/k3s/agent/serving-kubelet.crt --tls-private-key-file=/var/lib/rancher/k3s/agent/serving-kubelet.key"
  201. Flag --containerd has been deprecated, This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.
  202. I0920 15:12:28.204525       6 server.go:413] Version: v1.18.6+k3s1
  203. time="2020-09-20T15:12:28.212701167Z" level=info msg="Running kube-proxy --cluster-cidr=10.42.0.0/16 --healthz-bind-address=127.0.0.1 --hostname-override=k3d-testing-server-0 --kubeconfig=/var/lib/rancher/k3s/agent/kubeproxy.kubeconfig --proxy-mode=iptables"
  204. W0920 15:12:28.213248       6 server.go:225] WARNING: all flags other than --config, --write-config-to, and --cleanup are deprecated. Please begin using a config file ASAP.
  205. W0920 15:12:28.217244       6 proxier.go:625] Failed to read file /lib/modules/5.4.0-40-generic/modules.builtin with error open /lib/modules/5.4.0-40-generic/modules.builtin: no such file or directory. You can ignore this message when kube-proxy is running inside container without mounting /lib/modules
  206. W0920 15:12:28.218676       6 proxier.go:635] Failed to load kernel module ip_vs with modprobe. You can ignore this message when kube-proxy is running inside container without mounting /lib/modules
  207. W0920 15:12:28.219485       6 proxier.go:635] Failed to load kernel module ip_vs_rr with modprobe. You can ignore this message when kube-proxy is running inside container without mounting /lib/modules
  208. W0920 15:12:28.220185       6 proxier.go:635] Failed to load kernel module ip_vs_wrr with modprobe. You can ignore this message when kube-proxy is running inside container without mounting /lib/modules
  209. W0920 15:12:28.220929       6 proxier.go:635] Failed to load kernel module ip_vs_sh with modprobe. You can ignore this message when kube-proxy is running inside container without mounting /lib/modules
  210. W0920 15:12:28.221691       6 proxier.go:635] Failed to load kernel module nf_conntrack with modprobe. You can ignore this message when kube-proxy is running inside container without mounting /lib/modules
  211. time="2020-09-20T15:12:28.224337294Z" level=warning msg="Running modprobe ip_vs failed with message: `modprobe: can't change directory to '5.4.0-40-generic': No such file or directory`, error: exit status 1"
  212. I0920 15:12:28.274000       6 dynamic_cafile_content.go:167] Starting client-ca-bundle::/var/lib/rancher/k3s/agent/client-ca.crt
  213. W0920 15:12:28.275849       6 info.go:51] Couldn't collect info from any of the files in "/etc/machine-id,/var/lib/dbus/machine-id"
  214. I0920 15:12:28.277955       6 server.go:645] --cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /
  215. I0920 15:12:28.279014       6 container_manager_linux.go:277] container manager verified user specified cgroup-root exists: []
  216. I0920 15:12:28.279244       6 container_manager_linux.go:282] Creating Container Manager object based on Node Config: {RuntimeCgroupsName:/systemd SystemCgroupsName: KubeletCgroupsName:/systemd ContainerRuntime:remote CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:cgroupfs KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: ReservedSystemCPUs: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[{Signal:imagefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.05} GracePeriod:0s MinReclaim:<nil>} {Signal:nodefs.available Operator:LessThan Value:{Quantity:<nil> Percentage:0.05} GracePeriod:0s MinReclaim:<nil>}]} QOSReserved:map[] ExperimentalCPUManagerPolicy:none ExperimentalCPUManagerReconcilePeriod:10s ExperimentalPodPidsLimit:-1 EnforceCPULimits:false CPUCFSQuotaPeriod:100ms ExperimentalTopologyManagerPolicy:none}
  217. I0920 15:12:28.279569       6 topology_manager.go:126] [topologymanager] Creating topology manager with none policy
  218. I0920 15:12:28.279656       6 container_manager_linux.go:312] [topologymanager] Initializing Topology Manager with none policy
  219. I0920 15:12:28.279767       6 container_manager_linux.go:317] Creating device plugin manager: true
  220. W0920 15:12:28.280551       6 util_unix.go:103] Using "/run/k3s/containerd/containerd.sock" as endpoint is deprecated, please consider using full url format "unix:///run/k3s/containerd/containerd.sock".
  221. W0920 15:12:28.280896       6 util_unix.go:103] Using "/run/k3s/containerd/containerd.sock" as endpoint is deprecated, please consider using full url format "unix:///run/k3s/containerd/containerd.sock".
  222. I0920 15:12:28.281256       6 kubelet.go:317] Watching apiserver
  223. I0920 15:12:28.300066       6 kuberuntime_manager.go:211] Container runtime containerd initialized, version: v1.3.3-k3s2, apiVersion: v1alpha2
  224. I0920 15:12:28.301253       6 server.go:1124] Started kubelet
  225. I0920 15:12:28.357611       6 fs_resource_analyzer.go:64] Starting FS ResourceAnalyzer
  226. I0920 15:12:28.359324       6 server.go:145] Starting to listen on 0.0.0.0:10250
  227. I0920 15:12:28.361466       6 server.go:393] Adding debug handlers to kubelet server.
  228. I0920 15:12:28.371537       6 volume_manager.go:265] Starting Kubelet Volume Manager
  229. I0920 15:12:28.374591       6 desired_state_of_world_populator.go:139] Desired state populator starts to run
  230. I0920 15:12:28.412653       6 status_manager.go:158] Starting to sync pod status with apiserver
  231. I0920 15:12:28.413050       6 kubelet.go:1822] Starting kubelet main sync loop.
  232. E0920 15:12:28.413449       6 kubelet.go:1846] skipping pod synchronization - [container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]
  233. E0920 15:12:28.441879       6 cri_stats_provider.go:375] Failed to get the info of the filesystem with mountpoint "/var/lib/rancher/k3s/agent/containerd/io.containerd.snapshotter.v1.overlayfs": unable to find data in memory cache.
  234. E0920 15:12:28.442150       6 kubelet.go:1306] Image garbage collection failed once. Stats initialization may not have completed yet: invalid capacity 0 on image filesystem
  235. I0920 15:12:28.505082       6 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
  236. E0920 15:12:28.506710       6 kubelet.go:2268] node "k3d-testing-server-0" not found
  237. E0920 15:12:28.561071       6 node.go:125] Failed to retrieve node info: nodes "k3d-testing-server-0" not found
  238. time="2020-09-20T15:12:28.570832691Z" level=info msg="waiting for node k3d-testing-server-0: nodes \"k3d-testing-server-0\" not found"
  239. E0920 15:12:28.573906       6 kubelet.go:1846] skipping pod synchronization - container runtime status check may not have completed yet
  240. I0920 15:12:28.608467       6 kubelet_node_status.go:70] Attempting to register node k3d-testing-server-0
  241. I0920 15:12:28.619601       6 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
  242. E0920 15:12:28.622551       6 kubelet.go:2268] node "k3d-testing-server-0" not found
  243. I0920 15:12:28.665754       6 cpu_manager.go:184] [cpumanager] starting with none policy
  244. I0920 15:12:28.666091       6 cpu_manager.go:185] [cpumanager] reconciling every 10s
  245. I0920 15:12:28.666375       6 state_mem.go:36] [cpumanager] initializing new in-memory state store
  246. I0920 15:12:28.669750       6 policy_none.go:43] [cpumanager] none policy: Start
  247. time="2020-09-20T15:12:28.691265211Z" level=info msg="Waiting for master node k3d-testing-server-0 startup: nodes \"k3d-testing-server-0\" not found"
  248. E0920 15:12:28.693137       6 controller.go:228] failed to get node "k3d-testing-server-0" when trying to set owner ref to the node lease: nodes "k3d-testing-server-0" not found
  249. I0920 15:12:28.759381       6 node_controller.go:325] Initializing node k3d-testing-server-0 with cloud provider
  250. I0920 15:12:28.764216       6 kubelet_node_status.go:73] Successfully registered node k3d-testing-server-0
  251. W0920 15:12:28.828086       6 manager.go:597] Failed to retrieve checkpoint for "kubelet_internal_checkpoint": checkpoint is not found
  252. I0920 15:12:28.828888       6 plugin_manager.go:114] Starting Kubelet Plugin Manager
  253. I0920 15:12:28.898306       6 reconciler.go:157] Reconciler: start to sync state
  254. I0920 15:12:29.044234       6 controllermanager.go:533] Started "horizontalpodautoscaling"
  255. I0920 15:12:29.044427       6 horizontal.go:169] Starting HPA controller
  256. I0920 15:12:29.044435       6 shared_informer.go:223] Waiting for caches to sync for HPA
  257. time="2020-09-20T15:12:29.065789174Z" level=info msg="couldn't find node internal ip label on node k3d-testing-server-0"
  258. time="2020-09-20T15:12:29.065831609Z" level=info msg="couldn't find node hostname label on node k3d-testing-server-0"
  259. time="2020-09-20T15:12:29.089803827Z" level=info msg="Updated coredns node hosts entry [172.27.0.2 k3d-testing-server-0]"
  260. E0920 15:12:29.392716       6 summary_sys_containers.go:47] Failed to get system container stats for "/systemd": failed to get cgroup stats for "/systemd": failed to get container info for "/systemd": unknown container "/systemd"
  261. E0920 15:12:29.392980       6 summary_sys_containers.go:47] Failed to get system container stats for "/systemd": failed to get cgroup stats for "/systemd": failed to get container info for "/systemd": unknown container "/systemd"
  262. I0920 15:12:29.482085       6 trace.go:116] Trace[1156132399]: "GuaranteedUpdate etcd3" type:*core.Node (started: 2020-09-20 15:12:28.979341864 +0000 UTC m=+16.326331137) (total time: 502.689852ms):
  263. Trace[1156132399]: [135.647401ms] [135.467703ms] Transaction prepared
  264. Trace[1156132399]: [304.475669ms] [168.828268ms] Transaction committed
  265. Trace[1156132399]: [502.450833ms] [195.845817ms] Transaction committed
  266. I0920 15:12:29.482670       6 trace.go:116] Trace[247533618]: "Patch" url:/api/v1/nodes/k3d-testing-server-0/status,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:12:28.978950831 +0000 UTC m=+16.325940068) (total time: 503.674479ms):
  267. Trace[247533618]: [135.454751ms] [134.866311ms] About to check admission control
  268. Trace[247533618]: [304.993803ms] [169.539052ms] About to apply patch
  269. Trace[247533618]: [503.535011ms] [196.981721ms] Object stored in database
  270. I0920 15:12:29.534146       6 node_lifecycle_controller.go:78] Sending events to api server
  271. E0920 15:12:29.534506       6 core.go:229] failed to start cloud node lifecycle controller: no cloud provider provided
  272. W0920 15:12:29.534649       6 controllermanager.go:525] Skipping "cloud-node-lifecycle"
  273. I0920 15:12:29.654821       6 controllermanager.go:533] Started "persistentvolume-expander"
  274. I0920 15:12:29.655466       6 expand_controller.go:319] Starting expand controller
  275. I0920 15:12:29.655638       6 shared_informer.go:223] Waiting for caches to sync for expand
  276. I0920 15:12:29.743931       6 node.go:136] Successfully retrieved node IP: 172.27.0.2
  277. I0920 15:12:29.746815       6 server_others.go:187] Using iptables Proxier.
  278. I0920 15:12:29.752191       6 server.go:583] Version: v1.18.6+k3s1
  279. I0920 15:12:29.758860       6 conntrack.go:52] Setting nf_conntrack_max to 131072
  280. I0920 15:12:29.796396       6 conntrack.go:103] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_established' to 86400
  281. I0920 15:12:29.796539       6 conntrack.go:103] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_close_wait' to 3600
  282. I0920 15:12:29.802608       6 config.go:315] Starting service config controller
  283. I0920 15:12:29.802657       6 shared_informer.go:223] Waiting for caches to sync for service config
  284. I0920 15:12:29.802706       6 config.go:133] Starting endpoints config controller
  285. I0920 15:12:29.802732       6 shared_informer.go:223] Waiting for caches to sync for endpoints config
  286. time="2020-09-20T15:12:29.892817278Z" level=info msg="master role label has been set succesfully on node: k3d-testing-server-0"
  287. I0920 15:12:29.938094       6 shared_informer.go:230] Caches are synced for service config
  288. I0920 15:12:29.938574       6 shared_informer.go:230] Caches are synced for endpoints config
  289. I0920 15:12:30.292231       6 controllermanager.go:533] Started "replicationcontroller"
  290. I0920 15:12:30.292828       6 replica_set.go:181] Starting replicationcontroller controller
  291. I0920 15:12:30.293027       6 shared_informer.go:223] Waiting for caches to sync for ReplicationController
  292. I0920 15:12:30.370158       6 trace.go:116] Trace[1854841042]: "Create" url:/apis/k3s.cattle.io/v1/namespaces/kube-system/addons,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:12:29.866744666 +0000 UTC m=+17.213733852) (total time: 503.32162ms):
  293. Trace[1854841042]: [501.916372ms] [501.634069ms] Object stored in database
  294. I0920 15:12:30.631528       6 node_ipam_controller.go:94] Sending events to api server.
  295. I0920 15:12:30.674793       6 trace.go:116] Trace[1548480728]: "GuaranteedUpdate etcd3" type:*core.ServiceAccount (started: 2020-09-20 15:12:30.108116355 +0000 UTC m=+17.455105556) (total time: 566.619131ms):
  296. Trace[1548480728]: [566.564606ms] [566.211012ms] Transaction committed
  297. I0920 15:12:30.675332       6 trace.go:116] Trace[1738941790]: "Update" url:/api/v1/namespaces/kube-system/serviceaccounts/expand-controller,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/tokens-controller,client:127.0.0.1 (started: 2020-09-20 15:12:30.107878011 +0000 UTC m=+17.454867217) (total time: 567.414212ms):
  298. Trace[1738941790]: [567.297107ms] [567.180983ms] Object stored in database
  299. I0920 15:12:30.729180       6 trace.go:116] Trace[104974683]: "GuaranteedUpdate etcd3" type:*core.ConfigMap (started: 2020-09-20 15:12:30.207880489 +0000 UTC m=+17.554869806) (total time: 521.241381ms):
  300. Trace[104974683]: [521.139489ms] [520.558774ms] Transaction committed
  301. I0920 15:12:30.729735       6 trace.go:116] Trace[873063152]: "Update" url:/api/v1/namespaces/kube-system/configmaps/k3s,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:12:30.207542414 +0000 UTC m=+17.554531624) (total time: 522.148882ms):
  302. Trace[873063152]: [521.965366ms] [521.691566ms] Object stored in database
  303. I0920 15:12:30.927338       6 trace.go:116] Trace[21267881]: "GuaranteedUpdate etcd3" type:*core.Endpoints (started: 2020-09-20 15:12:30.227610523 +0000 UTC m=+17.574599733) (total time: 699.671525ms):
  304. Trace[21267881]: [699.600765ms] [698.874197ms] Transaction committed
  305. I0920 15:12:30.927527       6 trace.go:116] Trace[1883582428]: "Update" url:/api/v1/namespaces/kube-system/endpoints/cloud-controller-manager,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:12:30.227380644 +0000 UTC m=+17.574369847) (total time: 700.115175ms):
  306. Trace[1883582428]: [700.035604ms] [699.860701ms] Object stored in database
  307. time="2020-09-20T15:12:30.934514916Z" level=info msg="waiting for node k3d-testing-server-0 CIDR not assigned yet"
  308. I0920 15:12:30.951199       6 trace.go:116] Trace[817037361]: "GuaranteedUpdate etcd3" type:*core.Node (started: 2020-09-20 15:12:30.415962563 +0000 UTC m=+17.762951775) (total time: 535.195631ms):
  309. Trace[817037361]: [535.120605ms] [533.229285ms] Transaction committed
  310. I0920 15:12:30.953743       6 trace.go:116] Trace[821331171]: "Update" url:/api/v1/nodes/k3d-testing-server-0,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/node-controller,client:127.0.0.1 (started: 2020-09-20 15:12:30.415603273 +0000 UTC m=+17.762592563) (total time: 538.025168ms):
  311. Trace[821331171]: [537.869461ms] [537.592199ms] Object stored in database
  312. time="2020-09-20T15:12:30.994098000Z" level=info msg="couldn't find node internal ip label on node k3d-testing-server-0"
  313. time="2020-09-20T15:12:30.994145430Z" level=info msg="couldn't find node hostname label on node k3d-testing-server-0"
  314. I0920 15:12:30.994179       6 node_controller.go:397] Successfully initialized node k3d-testing-server-0 with cloud provider
  315. I0920 15:12:30.994222       6 node_controller.go:325] Initializing node k3d-testing-server-0 with cloud provider
  316. I0920 15:12:31.254710       6 node_controller.go:325] Initializing node k3d-testing-server-0 with cloud provider
  317. I0920 15:12:31.516428       6 node_controller.go:325] Initializing node k3d-testing-server-0 with cloud provider
  318. I0920 15:12:31.787193       6 node_controller.go:325] Initializing node k3d-testing-server-0 with cloud provider
  319. I0920 15:12:32.178289       6 trace.go:116] Trace[506186294]: "GuaranteedUpdate etcd3" type:*core.ServiceAccount (started: 2020-09-20 15:12:31.599792446 +0000 UTC m=+18.946781656) (total time: 578.440246ms):
  320. Trace[506186294]: [578.349117ms] [566.835082ms] Transaction committed
  321. I0920 15:12:32.178506       6 trace.go:116] Trace[1536852442]: "Update" url:/api/v1/namespaces/kube-system/serviceaccounts/metrics-server,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/tokens-controller,client:127.0.0.1 (started: 2020-09-20 15:12:31.599561662 +0000 UTC m=+18.946550847) (total time: 578.905416ms):
  322. Trace[1536852442]: [578.805057ms] [578.630736ms] Object stored in database
  323. I0920 15:12:32.179103       6 trace.go:116] Trace[28654448]: "Get" url:/api/v1/namespaces/kube-system/endpoints/kube-scheduler,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:12:31.649907905 +0000 UTC m=+18.996897113) (total time: 529.154143ms):
  324. Trace[28654448]: [529.075348ms] [529.022398ms] About to write a response
  325. I0920 15:12:35.253257       6 trace.go:116] Trace[2002468278]: "Create" url:/apis/k3s.cattle.io/v1/namespaces/kube-system/addons,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:12:31.903191573 +0000 UTC m=+19.250181114) (total time: 3.350008362s):
  326. Trace[2002468278]: [3.349677756s] [3.349319741s] Object stored in database
  327. I0920 15:12:36.309838       6 trace.go:116] Trace[1159059653]: "Get" url:/api/v1/nodes/k3d-testing-server-0,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/node-controller,client:127.0.0.1 (started: 2020-09-20 15:12:32.044612121 +0000 UTC m=+19.391601324) (total time: 4.265156027s):
  328. Trace[1159059653]: [4.264998441s] [4.26497144s] About to write a response
  329. I0920 15:12:36.311107       6 trace.go:116] Trace[1739418422]: "Get" url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:12:32.263636335 +0000 UTC m=+19.610625634) (total time: 4.047416096s):
  330. Trace[1739418422]: [4.047307949s] [4.047263048s] About to write a response
  331. time="2020-09-20T15:12:36.316354252Z" level=error msg="Unable to watch for tunnel endpoints: unknown (get endpoints)"
  332. I0920 15:12:36.794079       6 node_controller.go:325] Initializing node k3d-testing-server-0 with cloud provider
  333. I0920 15:12:36.892632       6 trace.go:116] Trace[1831783429]: "Get" url:/api/v1/nodes/k3d-testing-server-0,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:12:35.164716814 +0000 UTC m=+22.511706109) (total time: 1.727846475s):
  334. Trace[1831783429]: [1.727385614s] [1.727352802s] About to write a response
  335. I0920 15:12:36.958215       6 trace.go:116] Trace[314276057]: "Get" url:/api/v1/namespaces/kube-system/endpoints/cloud-controller-manager,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:12:35.270204036 +0000 UTC m=+22.617193239) (total time: 1.687943914s):
  336. Trace[314276057]: [1.687818528s] [1.687795065s] About to write a response
  337. I0920 15:12:36.982923       6 trace.go:116] Trace[566545035]: "Get" url:/api/v1/namespaces/kube-system/configmaps/k3s,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:12:35.252336716 +0000 UTC m=+22.599325901) (total time: 1.730511865s):
  338. Trace[566545035]: [1.730336849s] [1.730320493s] About to write a response
  339. I0920 15:12:36.988528       6 trace.go:116] Trace[1959441266]: "Get" url:/api/v1/namespaces/default,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:12:35.27054723 +0000 UTC m=+22.617536431) (total time: 1.717890971s):
  340. Trace[1959441266]: [1.717765638s] [1.717759216s] About to write a response
  341. time="2020-09-20T15:12:36.990728428Z" level=info msg="waiting for node k3d-testing-server-0 CIDR not assigned yet"
  342. I0920 15:12:37.293115       6 trace.go:116] Trace[1207278053]: "GuaranteedUpdate etcd3" type:*core.Endpoints (started: 2020-09-20 15:12:35.250390791 +0000 UTC m=+22.597379981) (total time: 2.042658989s):
  343. Trace[1207278053]: [2.042576055s] [2.041779588s] Transaction committed
  344. I0920 15:12:37.293556       6 trace.go:116] Trace[1953041730]: "Update" url:/api/v1/namespaces/kube-system/endpoints/kube-controller-manager,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:12:35.250145381 +0000 UTC m=+22.597134598) (total time: 2.043351937s):
  345. Trace[1953041730]: [2.043085794s] [2.042907417s] Object stored in database
  346. I0920 15:12:38.441611       6 trace.go:116] Trace[2133178562]: "GuaranteedUpdate etcd3" type:*v1.Endpoints (started: 2020-09-20 15:12:37.534968717 +0000 UTC m=+24.881957909) (total time: 906.585342ms):
  347. Trace[2133178562]: [177.073326ms] [177.073326ms] initial value restored
  348. Trace[2133178562]: [390.875258ms] [213.801932ms] Transaction prepared
  349. Trace[2133178562]: [906.539696ms] [515.664438ms] Transaction committed
  350. I0920 15:12:38.594868       6 trace.go:116] Trace[1625647298]: "Create" url:/api/v1/namespaces/kube-system/services,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:12:37.499420411 +0000 UTC m=+24.846409604) (total time: 1.09537336s):
  351. Trace[1625647298]: [1.093652602s] [1.093267479s] Object stored in database
  352. I0920 15:12:39.523245       6 trace.go:116] Trace[1711510853]: "GuaranteedUpdate etcd3" type:*apiregistration.APIService (started: 2020-09-20 15:12:38.729923317 +0000 UTC m=+26.076912522) (total time: 793.286766ms):
  353. Trace[1711510853]: [793.119084ms] [787.396243ms] Transaction committed
  354. I0920 15:12:39.534962       6 trace.go:116] Trace[968869755]: "Update" url:/apis/apiregistration.k8s.io/v1/apiservices/v1beta1.metrics.k8s.io/status,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:12:38.729566366 +0000 UTC m=+26.076555547) (total time: 805.308286ms):
  355. Trace[968869755]: [805.155463ms] [804.8773ms] Object stored in database
  356. I0920 15:12:39.695285       6 trace.go:116] Trace[1838483081]: "Create" url:/apis/k3s.cattle.io/v1/namespaces/kube-system/addons,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:12:38.810835564 +0000 UTC m=+26.157824777) (total time: 884.397432ms):
  357. Trace[1838483081]: [884.110066ms] [883.692836ms] Object stored in database
  358. time="2020-09-20T15:12:39.936831264Z" level=info msg="waiting for node k3d-testing-server-0 CIDR not assigned yet"
  359. I0920 15:12:40.959887       6 trace.go:116] Trace[1918663301]: "Create" url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:12:40.264429922 +0000 UTC m=+27.611419104) (total time: 695.388587ms):
  360. Trace[1918663301]: [695.303271ms] [695.183833ms] Object stored in database
  361. time="2020-09-20T15:12:41.066941673Z" level=info msg="Updating TLS secret for k3s-serving (count: 9): map[listener.cattle.io/cn-0.0.0.0:0.0.0.0 listener.cattle.io/cn-10.43.0.1:10.43.0.1 listener.cattle.io/cn-127.0.0.1:127.0.0.1 listener.cattle.io/cn-172.27.0.2:172.27.0.2 listener.cattle.io/cn-k3d-testing-server-0:k3d-testing-server-0 listener.cattle.io/cn-kubernetes:kubernetes listener.cattle.io/cn-kubernetes.default:kubernetes.default listener.cattle.io/cn-kubernetes.default.svc.cluster.local:kubernetes.default.svc.cluster.local listener.cattle.io/cn-localhost:localhost listener.cattle.io/hash:92b7b37fe3e2b94c828a2c8c07e2595d3f600d4ffc1bb4436388d0b6e6a94801]"
  362. I0920 15:12:41.617114       6 trace.go:116] Trace[1117407556]: "Create" url:/api/v1/namespaces/kube-system/serviceaccounts/node-controller/token,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/kube-controller-manager,client:127.0.0.1 (started: 2020-09-20 15:12:41.096646854 +0000 UTC m=+28.443636068) (total time: 520.415453ms):
  363. Trace[1117407556]: [520.287557ms] [520.152102ms] Object stored in database
  364. I0920 15:12:41.647727       6 trace.go:116] Trace[1838303263]: "Update" url:/api/v1/namespaces/kube-system/secrets/k3s-serving,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:12:41.140878937 +0000 UTC m=+28.487868147) (total time: 506.796659ms):
  365. Trace[1838303263]: [506.624827ms] [506.323085ms] Object stored in database
  366. time="2020-09-20T15:12:41.706415782Z" level=info msg="Active TLS secret k3s-serving (ver=311) (count 9): map[listener.cattle.io/cn-0.0.0.0:0.0.0.0 listener.cattle.io/cn-10.43.0.1:10.43.0.1 listener.cattle.io/cn-127.0.0.1:127.0.0.1 listener.cattle.io/cn-172.27.0.2:172.27.0.2 listener.cattle.io/cn-k3d-testing-server-0:k3d-testing-server-0 listener.cattle.io/cn-kubernetes:kubernetes listener.cattle.io/cn-kubernetes.default:kubernetes.default listener.cattle.io/cn-kubernetes.default.svc.cluster.local:kubernetes.default.svc.cluster.local listener.cattle.io/cn-localhost:localhost listener.cattle.io/hash:92b7b37fe3e2b94c828a2c8c07e2595d3f600d4ffc1bb4436388d0b6e6a94801]"
  367. I0920 15:12:41.769216       6 trace.go:116] Trace[1535517935]: "GuaranteedUpdate etcd3" type:*core.Endpoints (started: 2020-09-20 15:12:41.178406787 +0000 UTC m=+28.525396000) (total time: 590.775927ms):
  368. Trace[1535517935]: [590.709436ms] [590.15376ms] Transaction committed
  369. I0920 15:12:41.772437       6 trace.go:116] Trace[1035808158]: "Update" url:/api/v1/namespaces/kube-system/endpoints/kube-controller-manager,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:12:41.17801884 +0000 UTC m=+28.525008026) (total time: 594.359005ms):
  370. Trace[1035808158]: [594.228219ms] [593.888008ms] Object stored in database
  371. time="2020-09-20T15:12:41.803247835Z" level=error msg="Unable to watch for tunnel endpoints: unknown (get endpoints)"
  372. I0920 15:12:41.872609       6 trace.go:116] Trace[334388292]: "GuaranteedUpdate etcd3" type:*core.Endpoints (started: 2020-09-20 15:12:41.335280736 +0000 UTC m=+28.682269957) (total time: 537.287024ms):
  373. Trace[334388292]: [537.23146ms] [527.934845ms] Transaction committed
  374. I0920 15:12:41.872798       6 trace.go:116] Trace[2005595269]: "Update" url:/api/v1/namespaces/kube-system/endpoints/kube-scheduler,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:12:41.310032793 +0000 UTC m=+28.657022009) (total time: 562.735072ms):
  375. Trace[2005595269]: [562.650065ms] [537.489669ms] Object stored in database
  376. time="2020-09-20T15:12:42.039982373Z" level=info msg="Active TLS secret k3s-serving (ver=311) (count 9): map[listener.cattle.io/cn-0.0.0.0:0.0.0.0 listener.cattle.io/cn-10.43.0.1:10.43.0.1 listener.cattle.io/cn-127.0.0.1:127.0.0.1 listener.cattle.io/cn-172.27.0.2:172.27.0.2 listener.cattle.io/cn-k3d-testing-server-0:k3d-testing-server-0 listener.cattle.io/cn-kubernetes:kubernetes listener.cattle.io/cn-kubernetes.default:kubernetes.default listener.cattle.io/cn-kubernetes.default.svc.cluster.local:kubernetes.default.svc.cluster.local listener.cattle.io/cn-localhost:localhost listener.cattle.io/hash:92b7b37fe3e2b94c828a2c8c07e2595d3f600d4ffc1bb4436388d0b6e6a94801]"
  377. I0920 15:12:42.062342       6 range_allocator.go:82] Sending events to api server.
  378. I0920 15:12:42.062710       6 range_allocator.go:110] No Service CIDR provided. Skipping filtering out service addresses.
  379. I0920 15:12:42.062726       6 range_allocator.go:116] No Secondary Service CIDR provided. Skipping filtering out secondary service addresses.
  380. I0920 15:12:42.062783       6 controllermanager.go:533] Started "nodeipam"
  381. W0920 15:12:42.062813       6 core.go:243] configure-cloud-routes is set, but no cloud provider specified. Will not configure cloud provider routes.
  382. W0920 15:12:42.062820       6 controllermanager.go:525] Skipping "route"
  383. I0920 15:12:42.063122       6 node_ipam_controller.go:162] Starting ipam controller
  384. I0920 15:12:42.063141       6 shared_informer.go:223] Waiting for caches to sync for node
  385. I0920 15:12:42.367853       6 log.go:172] http: TLS handshake error from 172.27.0.3:36386: remote error: tls: bad certificate
  386. time="2020-09-20T15:12:42.557356670Z" level=info msg="waiting for node k3d-testing-server-0 CIDR not assigned yet"
  387. I0920 15:12:42.835508       6 log.go:172] http: TLS handshake error from 172.27.0.3:36394: remote error: tls: bad certificate
  388. I0920 15:12:43.236107       6 controllermanager.go:533] Started "pvc-protection"
  389. I0920 15:12:43.236460       6 pvc_protection_controller.go:101] Starting PVC protection controller
  390. I0920 15:12:43.236473       6 shared_informer.go:223] Waiting for caches to sync for PVC protection
  391. W0920 15:12:43.978089       6 handler_proxy.go:102] no RequestInfo found in the context
  392. E0920 15:12:43.978195       6 controller.go:114] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
  393. , Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
  394. I0920 15:12:43.978213       6 controller.go:127] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
  395. I0920 15:12:44.200789       6 controllermanager.go:533] Started "deployment"
  396. I0920 15:12:44.203417       6 deployment_controller.go:153] Starting deployment controller
  397. I0920 15:12:44.203464       6 shared_informer.go:223] Waiting for caches to sync for deployment
  398. I0920 15:12:45.058530       6 trace.go:116] Trace[333475220]: "Get" url:/api/v1/namespaces/kube-system/serviceaccounts/deployment-controller,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/tokens-controller,client:127.0.0.1 (started: 2020-09-20 15:12:44.388784202 +0000 UTC m=+31.735773412) (total time: 669.690631ms):
  399. Trace[333475220]: [669.604434ms] [669.587874ms] About to write a response
  400. I0920 15:12:45.142478       6 trace.go:116] Trace[971228073]: "Create" url:/apis/k3s.cattle.io/v1/namespaces/kube-system/addons,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:12:43.663079969 +0000 UTC m=+31.010069153) (total time: 1.479347778s):
  401. Trace[971228073]: [1.479122694s] [1.478852327s] Object stored in database
  402. I0920 15:12:45.174385       6 trace.go:116] Trace[1827295251]: "Get" url:/api/v1/namespaces/kube-system/serviceaccounts/daemon-set-controller,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/kube-controller-manager,client:127.0.0.1 (started: 2020-09-20 15:12:44.20455974 +0000 UTC m=+31.551548963) (total time: 969.774687ms):
  403. Trace[1827295251]: [969.774687ms] [969.760412ms] END
  404. I0920 15:12:46.095853       6 trace.go:116] Trace[1330801612]: "Get" url:/api/v1/namespaces/kube-system/configmaps/k3s,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:12:45.316090329 +0000 UTC m=+32.663079526) (total time: 779.712338ms):
  405. Trace[1330801612]: [779.589255ms] [779.578002ms] About to write a response
  406. time="2020-09-20T15:12:46.852343328Z" level=info msg="Tunnel endpoint watch event: [172.27.0.2:6443]"
  407. time="2020-09-20T15:12:46.852448054Z" level=info msg="Connecting to proxy" url="wss://172.27.0.2:6443/v1-k3s/connect"
  408. time="2020-09-20T15:12:46.938997452Z" level=info msg="Handling backend connection request [k3d-testing-server-0]"
  409. I0920 15:12:46.976439       6 trace.go:116] Trace[1500721328]: "GuaranteedUpdate etcd3" type:*core.ServiceAccount (started: 2020-09-20 15:12:43.931595666 +0000 UTC m=+31.278584879) (total time: 3.04479236s):
  410. Trace[1500721328]: [3.044734491s] [3.044311347s] Transaction committed
  411. I0920 15:12:46.976638       6 trace.go:116] Trace[1028854164]: "Update" url:/api/v1/namespaces/kube-system/serviceaccounts/pvc-protection-controller,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/tokens-controller,client:127.0.0.1 (started: 2020-09-20 15:12:43.929223834 +0000 UTC m=+31.276213028) (total time: 3.047383569s):
  412. Trace[1028854164]: [3.047283087s] [3.045035794s] Object stored in database
  413. I0920 15:12:48.271885       6 trace.go:116] Trace[1407424968]: "Get" url:/api/v1/namespaces/kube-system/endpoints/cloud-controller-manager,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:12:44.983565462 +0000 UTC m=+32.330554650) (total time: 3.288266s):
  414. Trace[1407424968]: [3.288161158s] [3.288143983s] About to write a response
  415. I0920 15:12:48.289679       6 trace.go:116] Trace[1029851414]: "Get" url:/api/v1/namespaces/kube-system/endpoints/kube-scheduler,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:12:44.729957465 +0000 UTC m=+32.076946662) (total time: 3.559652509s):
  416. Trace[1029851414]: [3.559570086s] [3.559534446s] About to write a response
  417. I0920 15:12:48.303840       6 trace.go:116] Trace[686059183]: "Get" url:/api/v1/namespaces/kube-system/endpoints/kube-controller-manager,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:12:44.679116024 +0000 UTC m=+32.026105228) (total time: 3.624565214s):
  418. Trace[686059183]: [3.624370832s] [3.624313045s] About to write a response
  419. I0920 15:12:48.330175       6 trace.go:116] Trace[1418972815]: "List etcd3" key:/helm.cattle.io/helmcharts,resourceVersion:,limit:0,continue: (started: 2020-09-20 15:12:45.166599035 +0000 UTC m=+32.513588233) (total time: 3.163533263s):
  420. Trace[1418972815]: [3.163533263s] [3.163533263s] END
  421. I0920 15:12:48.330411       6 trace.go:116] Trace[1538250742]: "List" url:/apis/helm.cattle.io/v1/helmcharts,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:12:45.166474768 +0000 UTC m=+32.513464011) (total time: 3.163902454s):
  422. Trace[1538250742]: [3.163736963s] [3.16362682s] Listing from storage done
  423. I0920 15:12:48.347206       6 controller.go:606] quota admission added evaluator for: helmcharts.helm.cattle.io
  424. I0920 15:12:48.359111       6 trace.go:116] Trace[1651048002]: "Get" url:/api/v1/nodes/k3d-testing-server-0,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:12:44.675542799 +0000 UTC m=+32.022532022) (total time: 3.68352945s):
  425. Trace[1651048002]: [3.683232952s] [3.683190235s] About to write a response
  426. I0920 15:12:48.362956       6 trace.go:116] Trace[263741449]: "Get" url:/api/v1/namespaces/default,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:12:45.17724775 +0000 UTC m=+32.524236956) (total time: 3.185636795s):
  427. Trace[263741449]: [3.185574139s] [3.185559816s] About to write a response
  428. I0920 15:12:48.363263       6 trace.go:116] Trace[486027255]: "Get" url:/api/v1/namespaces/kube-system,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/kube-controller-manager,client:127.0.0.1 (started: 2020-09-20 15:12:45.196110251 +0000 UTC m=+32.543099454) (total time: 3.167131316s):
  429. Trace[486027255]: [3.167103186s] [3.16708861s] About to write a response
  430. time="2020-09-20T15:12:48.363920502Z" level=info msg="waiting for node k3d-testing-server-0 CIDR not assigned yet"
  431. I0920 15:12:48.407263       6 trace.go:116] Trace[216243803]: "Create" url:/api/v1/namespaces/kube-system/secrets,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/tokens-controller,client:127.0.0.1 (started: 2020-09-20 15:12:45.092790338 +0000 UTC m=+32.439779520) (total time: 3.314430464s):
  432. Trace[216243803]: [3.314348143s] [3.314208907s] Object stored in database
  433. I0920 15:12:48.538940       6 trace.go:116] Trace[1383852456]: "List etcd3" key:/jobs,resourceVersion:,limit:500,continue: (started: 2020-09-20 15:12:47.659780277 +0000 UTC m=+35.006769490) (total time: 879.108379ms):
  434. Trace[1383852456]: [879.108379ms] [879.108379ms] END
  435. I0920 15:12:48.540302       6 trace.go:116] Trace[174285684]: "List" url:/apis/batch/v1/jobs,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/system:serviceaccount:kube-system:cronjob-controller,client:127.0.0.1 (started: 2020-09-20 15:12:47.659617943 +0000 UTC m=+35.006607150) (total time: 880.624601ms):
  436. Trace[174285684]: [880.461413ms] [880.31327ms] Listing from storage done
  437. I0920 15:12:48.646502       6 trace.go:116] Trace[1386769560]: "GuaranteedUpdate etcd3" type:*core.ConfigMap (started: 2020-09-20 15:12:46.109721139 +0000 UTC m=+33.456710349) (total time: 2.536714162s):
  438. Trace[1386769560]: [2.536655427s] [2.475546694s] Transaction committed
  439. I0920 15:12:48.646743       6 trace.go:116] Trace[1671624436]: "Update" url:/api/v1/namespaces/kube-system/configmaps/k3s,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:12:46.108721786 +0000 UTC m=+33.455711021) (total time: 2.537989169s):
  440. Trace[1671624436]: [2.537857419s] [2.537044661s] Object stored in database
  441. I0920 15:12:48.820788       6 controllermanager.go:533] Started "daemonset"
  442. I0920 15:12:48.834126       6 daemon_controller.go:285] Starting daemon sets controller
  443. I0920 15:12:48.834163       6 shared_informer.go:223] Waiting for caches to sync for daemon sets
  444. I0920 15:12:49.427330       6 trace.go:116] Trace[1324748938]: "List etcd3" key:/cronjobs,resourceVersion:,limit:500,continue: (started: 2020-09-20 15:12:48.61659262 +0000 UTC m=+35.963581849) (total time: 810.692219ms):
  445. Trace[1324748938]: [810.692219ms] [810.692219ms] END
  446. I0920 15:12:49.442396       6 trace.go:116] Trace[1080809577]: "List" url:/apis/batch/v1beta1/cronjobs,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/system:serviceaccount:kube-system:cronjob-controller,client:127.0.0.1 (started: 2020-09-20 15:12:48.616537708 +0000 UTC m=+35.963526932) (total time: 825.777125ms):
  447. Trace[1080809577]: [824.16715ms] [824.128431ms] Listing from storage done
  448. I0920 15:12:49.430678       6 trace.go:116] Trace[2109367052]: "Get" url:/api/v1/namespaces/kube-system/serviceaccounts/replicaset-controller,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/kube-controller-manager,client:127.0.0.1 (started: 2020-09-20 15:12:48.835385658 +0000 UTC m=+36.182374856) (total time: 595.236333ms):
  449. Trace[2109367052]: [595.236333ms] [595.216137ms] END
  450. I0920 15:12:49.430969       6 trace.go:116] Trace[379349998]: "GuaranteedUpdate etcd3" type:*core.ServiceAccount (started: 2020-09-20 15:12:48.46394638 +0000 UTC m=+35.810935600) (total time: 966.998191ms):
  451. Trace[379349998]: [966.953857ms] [966.675583ms] Transaction committed
  452. I0920 15:12:49.445772       6 trace.go:116] Trace[2124853093]: "Update" url:/api/v1/namespaces/kube-system/serviceaccounts/deployment-controller,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/tokens-controller,client:127.0.0.1 (started: 2020-09-20 15:12:48.463744854 +0000 UTC m=+35.810734123) (total time: 981.981236ms):
  453. Trace[2124853093]: [981.837887ms] [981.688855ms] Object stored in database
  454. I0920 15:12:49.460089       6 trace.go:116] Trace[191475551]: "GuaranteedUpdate etcd3" type:*core.Endpoints (started: 2020-09-20 15:12:48.424685807 +0000 UTC m=+35.771675019) (total time: 1.035374684s):
  455. Trace[191475551]: [1.035352224s] [1.027279768s] Transaction committed
  456. I0920 15:12:49.460238       6 trace.go:116] Trace[920030425]: "Update" url:/api/v1/namespaces/kube-system/endpoints/kube-scheduler,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:12:48.424441662 +0000 UTC m=+35.771430870) (total time: 1.035768756s):
  457. Trace[920030425]: [1.035708121s] [1.035523705s] Object stored in database
  458. I0920 15:12:49.473176       6 trace.go:116] Trace[1152290832]: "GuaranteedUpdate etcd3" type:*core.Endpoints (started: 2020-09-20 15:12:48.507609439 +0000 UTC m=+35.854598649) (total time: 965.528ms):
  459. Trace[1152290832]: [965.49537ms] [964.768264ms] Transaction committed
  460. I0920 15:12:49.475431       6 trace.go:116] Trace[59479683]: "Update" url:/api/v1/namespaces/kube-system/endpoints/kube-controller-manager,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:12:48.507319423 +0000 UTC m=+35.854308607) (total time: 968.064394ms):
  461. Trace[59479683]: [965.911988ms] [965.694886ms] Object stored in database
  462. I0920 15:12:49.519363       6 trace.go:116] Trace[1825766939]: "Create" url:/api/v1/namespaces/kube-system/serviceaccounts,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:12:48.61596794 +0000 UTC m=+35.962957147) (total time: 903.343423ms):
  463. Trace[1825766939]: [903.040061ms] [897.424762ms] Object stored in database
  464. I0920 15:12:49.560845       6 trace.go:116] Trace[238997453]: "GuaranteedUpdate etcd3" type:*v1.Endpoints (started: 2020-09-20 15:12:48.571166327 +0000 UTC m=+35.918155541) (total time: 989.635121ms):
  465. Trace[238997453]: [989.596213ms] [813.694487ms] Transaction committed
  466. I0920 15:12:49.567104       6 trace.go:116] Trace[260363846]: "Create" url:/apis/k3s.cattle.io/v1/namespaces/kube-system/addons,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:12:48.606744211 +0000 UTC m=+35.953733407) (total time: 960.307766ms):
  467. Trace[260363846]: [960.110761ms] [959.774013ms] Object stored in database
  468. I0920 15:12:49.628083       6 controllermanager.go:533] Started "replicaset"
  469. W0920 15:12:49.633453       6 controllermanager.go:525] Skipping "ttl-after-finished"
  470. I0920 15:12:49.633793       6 replica_set.go:181] Starting replicaset controller
  471. I0920 15:12:49.633809       6 shared_informer.go:223] Waiting for caches to sync for ReplicaSet
  472. I0920 15:12:49.818625       6 controller.go:606] quota admission added evaluator for: jobs.batch
  473. E0920 15:12:50.096791       6 namespaced_resources_deleter.go:161] unable to get all supported resources from server: unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
  474. I0920 15:12:50.098816       6 controllermanager.go:533] Started "namespace"
  475. I0920 15:12:50.099099       6 namespace_controller.go:200] Starting namespace controller
  476. I0920 15:12:50.099110       6 shared_informer.go:223] Waiting for caches to sync for namespace
  477. E0920 15:12:50.382344       6 memcache.go:206] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request
  478. time="2020-09-20T15:12:50.401909715Z" level=info msg="waiting for node k3d-testing-server-0 CIDR not assigned yet"
  479. E0920 15:12:50.751214       6 memcache.go:111] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request
  480. W0920 15:12:50.754241       6 garbagecollector.go:644] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
  481. E0920 15:12:50.866379       6 memcache.go:206] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request
  482. E0920 15:12:51.207365       6 memcache.go:111] couldn't get resource list for metrics.k8s.io/v1beta1: the server is currently unable to handle the request
  483. I0920 15:12:51.216366       6 controllermanager.go:533] Started "garbagecollector"
  484. I0920 15:12:51.217542       6 garbagecollector.go:133] Starting garbage collector controller
  485. I0920 15:12:51.221399       6 shared_informer.go:223] Waiting for caches to sync for garbage collector
  486. I0920 15:12:51.221797       6 graph_builder.go:282] GraphBuilder running
  487. I0920 15:12:51.227604       6 controllermanager.go:533] Started "csrapproving"
  488. W0920 15:12:51.233428       6 controllermanager.go:512] "bootstrapsigner" is disabled
  489. I0920 15:12:51.233914       6 certificate_controller.go:119] Starting certificate controller "csrapproving"
  490. I0920 15:12:51.234032       6 shared_informer.go:223] Waiting for caches to sync for certificate-csrapproving
  491. I0920 15:12:51.457837       6 log.go:172] http: TLS handshake error from 172.27.0.4:53074: remote error: tls: bad certificate
  492. I0920 15:12:51.492777       6 log.go:172] http: TLS handshake error from 172.27.0.4:53082: remote error: tls: bad certificate
  493. E0920 15:12:51.692546       6 resource_quota_controller.go:161] initial discovery check failure, continuing and counting on future sync update: unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
  494. W0920 15:12:51.692964       6 shared_informer.go:461] resyncPeriod 53803125190509 is smaller than resyncCheckPeriod 61994098782019 and the informer has already started. Changing it to 61994098782019
  495. I0920 15:12:51.693400       6 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for serviceaccounts
  496. I0920 15:12:51.693605       6 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for ingresses.extensions
  497. I0920 15:12:51.693770       6 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for deployments.apps
  498. I0920 15:12:51.693886       6 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for jobs.batch
  499. I0920 15:12:51.694046       6 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for poddisruptionbudgets.policy
  500. I0920 15:12:51.694204       6 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for addons.k3s.cattle.io
  501. I0920 15:12:51.694634       6 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for roles.rbac.authorization.k8s.io
  502. I0920 15:12:51.694918       6 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for events.events.k8s.io
  503. I0920 15:12:51.695112       6 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for horizontalpodautoscalers.autoscaling
  504. I0920 15:12:51.695354       6 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for cronjobs.batch
  505. I0920 15:12:51.695523       6 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for helmcharts.helm.cattle.io
  506. I0920 15:12:51.695724       6 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for daemonsets.apps
  507. I0920 15:12:51.696102       6 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for ingresses.networking.k8s.io
  508. I0920 15:12:51.696244       6 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for endpointslices.discovery.k8s.io
  509. I0920 15:12:51.696379       6 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for podtemplates
  510. I0920 15:12:51.696511       6 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for endpoints
  511. I0920 15:12:51.696637       6 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for controllerrevisions.apps
  512. I0920 15:12:51.696768       6 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for statefulsets.apps
  513. I0920 15:12:51.696882       6 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for replicasets.apps
  514. I0920 15:12:51.697089       6 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for rolebindings.rbac.authorization.k8s.io
  515. I0920 15:12:51.697224       6 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for limitranges
  516. I0920 15:12:51.697411       6 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for networkpolicies.networking.k8s.io
  517. I0920 15:12:51.697541       6 resource_quota_monitor.go:228] QuotaMonitor created object count evaluator for leases.coordination.k8s.io
  518. I0920 15:12:51.697626       6 controllermanager.go:533] Started "resourcequota"
  519. I0920 15:12:51.698242       6 resource_quota_controller.go:272] Starting resource quota controller
  520. I0920 15:12:51.698373       6 shared_informer.go:223] Waiting for caches to sync for resource quota
  521. I0920 15:12:51.698478       6 resource_quota_monitor.go:303] QuotaMonitor running
  522. I0920 15:12:51.777506       6 controllermanager.go:533] Started "ttl"
  523. I0920 15:12:51.778042       6 ttl_controller.go:118] Starting TTL controller
  524. I0920 15:12:51.778218       6 shared_informer.go:223] Waiting for caches to sync for TTL
  525. I0920 15:12:51.824671       6 node_lifecycle_controller.go:384] Sending events to api server.
  526. I0920 15:12:51.825596       6 taint_manager.go:163] Sending events to api server.
  527. I0920 15:12:51.825998       6 node_lifecycle_controller.go:512] Controller will reconcile labels.
  528. I0920 15:12:51.826258       6 controllermanager.go:533] Started "nodelifecycle"
  529. I0920 15:12:51.826843       6 node_lifecycle_controller.go:546] Starting node controller
  530. I0920 15:12:51.827069       6 shared_informer.go:223] Waiting for caches to sync for taint
  531. I0920 15:12:51.871496       6 controllermanager.go:533] Started "persistentvolume-binder"
  532. I0920 15:12:51.873092       6 pv_controller_base.go:295] Starting persistent volume controller
  533. I0920 15:12:51.873724       6 shared_informer.go:223] Waiting for caches to sync for persistent volume
  534. I0920 15:12:51.961499       6 controllermanager.go:533] Started "clusterrole-aggregation"
  535. I0920 15:12:51.962175       6 clusterroleaggregation_controller.go:149] Starting ClusterRoleAggregator
  536. I0920 15:12:51.962467       6 shared_informer.go:223] Waiting for caches to sync for ClusterRoleAggregator
  537. I0920 15:12:52.017727       6 controllermanager.go:533] Started "job"
  538. I0920 15:12:52.020014       6 job_controller.go:144] Starting job controller
  539. I0920 15:12:52.020290       6 shared_informer.go:223] Waiting for caches to sync for job
  540. I0920 15:12:52.086467       6 shared_informer.go:230] Caches are synced for ReplicaSet
  541. I0920 15:12:52.087091       6 shared_informer.go:230] Caches are synced for HPA
  542. I0920 15:12:52.129883       6 shared_informer.go:230] Caches are synced for service account
  543. I0920 15:12:52.130564       6 shared_informer.go:230] Caches are synced for namespace
  544. I0920 15:12:52.130960       6 shared_informer.go:230] Caches are synced for deployment
  545. W0920 15:12:52.136709       6 actual_state_of_world.go:506] Failed to update statusUpdateNeeded field in actual state of world: Failed to set statusUpdateNeeded to needed true, because nodeName="k3d-testing-server-0" does not exist
  546. I0920 15:12:52.143527       6 shared_informer.go:230] Caches are synced for daemon sets
  547. I0920 15:12:52.163708       6 shared_informer.go:230] Caches are synced for node
  548. I0920 15:12:52.164208       6 range_allocator.go:172] Starting range CIDR allocator
  549. I0920 15:12:52.164382       6 shared_informer.go:223] Waiting for caches to sync for cidrallocator
  550. I0920 15:12:52.164457       6 shared_informer.go:230] Caches are synced for cidrallocator
  551. I0920 15:12:52.166310       6 shared_informer.go:230] Caches are synced for ClusterRoleAggregator
  552. I0920 15:12:52.207172       6 shared_informer.go:230] Caches are synced for TTL
  553. I0920 15:12:52.224364       6 controller.go:606] quota admission added evaluator for: replicasets.apps
  554. I0920 15:12:52.231319       6 shared_informer.go:230] Caches are synced for PV protection
  555. I0920 15:12:52.232388       6 shared_informer.go:230] Caches are synced for job
  556. I0920 15:12:52.233686       6 shared_informer.go:230] Caches are synced for GC
  557. I0920 15:12:52.242332       6 range_allocator.go:373] Set node k3d-testing-server-0 PodCIDR to [10.42.0.0/24]
  558. I0920 15:12:52.248980       6 request.go:621] Throttling request took 1.013417594s, request: GET:https://127.0.0.1:6444/apis/apiextensions.k8s.io/v1beta1?timeout=32s
  559. I0920 15:12:52.283656       6 event.go:278] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"kube-system", Name:"metrics-server", UID:"7b419a71-d09c-4309-a09f-e900f63d597a", APIVersion:"apps/v1", ResourceVersion:"290", FieldPath:""}): type: 'Normal' reason: 'ScalingReplicaSet' Scaled up replica set metrics-server-7566d596c8 to 1
  560. I0920 15:12:52.290242       6 event.go:278] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"kube-system", Name:"coredns", UID:"f7d025bb-08b5-4214-be2b-0c2667a101bc", APIVersion:"apps/v1", ResourceVersion:"197", FieldPath:""}): type: 'Normal' reason: 'ScalingReplicaSet' Scaled up replica set coredns-8655855d6 to 1
  561. I0920 15:12:52.308401       6 event.go:278] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"kube-system", Name:"local-path-provisioner", UID:"6ffbb0e1-9f05-4a78-9389-1d580e8cf853", APIVersion:"apps/v1", ResourceVersion:"224", FieldPath:""}): type: 'Normal' reason: 'ScalingReplicaSet' Scaled up replica set local-path-provisioner-6d59f47c7 to 1
  562. I0920 15:12:52.344385       6 kuberuntime_manager.go:978] updating runtime config through cri with podcidr 10.42.0.0/24
  563. I0920 15:12:52.345790       6 shared_informer.go:230] Caches are synced for taint
  564. I0920 15:12:52.346315       6 node_lifecycle_controller.go:1433] Initializing eviction metric for zone:
  565. W0920 15:12:52.346784       6 node_lifecycle_controller.go:1048] Missing timestamp for Node k3d-testing-server-0. Assuming now as a timestamp.
  566. I0920 15:12:52.346973       6 node_lifecycle_controller.go:1249] Controller detected that zone  is now in state Normal.
  567. I0920 15:12:52.348965       6 taint_manager.go:187] Starting NoExecuteTaintManager
  568. I0920 15:12:52.349771       6 event.go:278] Event(v1.ObjectReference{Kind:"Node", Namespace:"", Name:"k3d-testing-server-0", UID:"55115179-a7f2-440f-a0c6-0d9cffc9273e", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RegisteredNode' Node k3d-testing-server-0 event: Registered Node k3d-testing-server-0 in Controller
  569. I0920 15:12:52.366634       6 kubelet_network.go:77] Setting Pod CIDR:  -> 10.42.0.0/24
  570. I0920 15:12:52.402579       6 shared_informer.go:230] Caches are synced for disruption
  571. I0920 15:12:52.402882       6 disruption.go:339] Sending events to api server.
  572. I0920 15:12:52.403496       6 shared_informer.go:230] Caches are synced for ReplicationController
  573. I0920 15:12:52.474333       6 shared_informer.go:230] Caches are synced for PVC protection
  574. I0920 15:12:52.480455       6 flannel.go:92] Determining IP address of default interface
  575. I0920 15:12:52.481462       6 flannel.go:105] Using interface with name eth0 and address 172.27.0.2
  576. I0920 15:12:52.486894       6 kube.go:117] Waiting 10m0s for node controller to sync
  577. I0920 15:12:52.487359       6 kube.go:300] Starting kube subnet manager
  578. I0920 15:12:52.502977       6 shared_informer.go:230] Caches are synced for stateful set
  579. I0920 15:12:52.503287       6 shared_informer.go:230] Caches are synced for attach detach
  580. I0920 15:12:52.503620       6 shared_informer.go:230] Caches are synced for expand
  581. I0920 15:12:52.504069       6 shared_informer.go:230] Caches are synced for persistent volume
  582. I0920 15:12:52.544414       6 shared_informer.go:230] Caches are synced for certificate-csrapproving
  583. I0920 15:12:52.575641       6 shared_informer.go:230] Caches are synced for certificate-csrsigning
  584. I0920 15:12:52.576486       6 shared_informer.go:230] Caches are synced for endpoint
  585. I0920 15:12:52.632391       6 shared_informer.go:230] Caches are synced for resource quota
  586. I0920 15:12:52.633812       6 shared_informer.go:230] Caches are synced for endpoint_slice
  587. time="2020-09-20T15:12:52.683651840Z" level=info msg="Failed to update node k3d-testing-server-0: Operation cannot be fulfilled on nodes \"k3d-testing-server-0\": the object has been modified; please apply your changes to the latest version and try again"
  588. I0920 15:12:52.732492       6 shared_informer.go:230] Caches are synced for garbage collector
  589. I0920 15:12:52.732844       6 garbagecollector.go:142] Garbage collector: all resource monitors have synced. Proceeding to collect garbage
  590. I0920 15:12:52.843650       6 event.go:278] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"kube-system", Name:"local-path-provisioner-6d59f47c7", UID:"1a5a9569-5002-4884-9163-66b10994da32", APIVersion:"apps/v1", ResourceVersion:"385", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: local-path-provisioner-6d59f47c7-cx7fr
  591. I0920 15:12:52.897542       6 event.go:278] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"kube-system", Name:"metrics-server-7566d596c8", UID:"78f76514-434b-4c9f-a7d1-9f7e28a78a82", APIVersion:"apps/v1", ResourceVersion:"383", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: metrics-server-7566d596c8-8bdgf
  592. I0920 15:12:52.920763       6 event.go:278] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"kube-system", Name:"coredns-8655855d6", UID:"4ff2e6c5-2287-40ca-90fa-1a994ea4b9c5", APIVersion:"apps/v1", ResourceVersion:"384", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: coredns-8655855d6-h6nfg
  593. I0920 15:12:52.933350       6 controller.go:606] quota admission added evaluator for: events.events.k8s.io
  594. I0920 15:12:52.957039       6 event.go:278] Event(v1.ObjectReference{Kind:"Job", Namespace:"kube-system", Name:"helm-install-traefik", UID:"7de285c9-cc6f-42ea-816c-002df8a656f0", APIVersion:"batch/v1", ResourceVersion:"349", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: helm-install-traefik-ksk4q
  595. I0920 15:12:53.087161       6 trace.go:116] Trace[1258160998]: "GuaranteedUpdate etcd3" type:*core.Node (started: 2020-09-20 15:12:52.408505623 +0000 UTC m=+39.755494875) (total time: 678.591759ms):
  596. Trace[1258160998]: [351.87298ms] [348.756616ms] Transaction committed
  597. Trace[1258160998]: [678.330316ms] [323.885933ms] Transaction committed
  598. I0920 15:12:53.087960       6 trace.go:116] Trace[66842401]: "Patch" url:/api/v1/nodes/k3d-testing-server-0,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/system:serviceaccount:kube-system:node-controller,client:127.0.0.1 (started: 2020-09-20 15:12:52.408344689 +0000 UTC m=+39.755333873) (total time: 679.535993ms):
  599. Trace[66842401]: [352.163049ms] [349.417524ms] About to apply patch
  600. Trace[66842401]: [679.371584ms] [325.432326ms] Object stored in database
  601. I0920 15:12:53.197911       6 event.go:278] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"", Name:"metrics-server", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'FailedToCreateEndpoint' Failed to create endpoint for service kube-system/metrics-server: endpoints "metrics-server" already exists
  602. I0920 15:12:53.489051       6 kube.go:124] Node controller sync successful
  603. I0920 15:12:53.489459       6 vxlan.go:121] VXLAN config: VNI=1 Port=0 GBP=false Learning=false DirectRouting=false
  604. I0920 15:12:53.572347       6 topology_manager.go:233] [topologymanager] Topology Admit Handler
  605. I0920 15:12:53.654754       6 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "tmp-dir" (UniqueName: "kubernetes.io/empty-dir/df39d1de-d0ae-4031-bded-6b1d56e07602-tmp-dir") pod "metrics-server-7566d596c8-8bdgf" (UID: "df39d1de-d0ae-4031-bded-6b1d56e07602")
  606. I0920 15:12:53.655199       6 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "metrics-server-token-mfblk" (UniqueName: "kubernetes.io/secret/df39d1de-d0ae-4031-bded-6b1d56e07602-metrics-server-token-mfblk") pod "metrics-server-7566d596c8-8bdgf" (UID: "df39d1de-d0ae-4031-bded-6b1d56e07602")
  607. E0920 15:12:53.657587       6 resource_quota_controller.go:408] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
  608. I0920 15:12:53.658020       6 shared_informer.go:223] Waiting for caches to sync for resource quota
  609. I0920 15:12:53.658199       6 shared_informer.go:230] Caches are synced for resource quota
  610. I0920 15:12:53.728436       6 topology_manager.go:233] [topologymanager] Topology Admit Handler
  611. I0920 15:12:53.751428       6 topology_manager.go:233] [topologymanager] Topology Admit Handler
  612. I0920 15:12:53.853942       6 topology_manager.go:233] [topologymanager] Topology Admit Handler
  613. I0920 15:12:53.972047       6 trace.go:116] Trace[631281456]: "GuaranteedUpdate etcd3" type:*apps.ReplicaSet (started: 2020-09-20 15:12:53.338846007 +0000 UTC m=+40.685835218) (total time: 633.081635ms):
  614. Trace[631281456]: [632.893965ms] [631.12278ms] Transaction committed
  615. I0920 15:12:54.008451       6 trace.go:116] Trace[314569415]: "Update" url:/apis/apps/v1/namespaces/kube-system/replicasets/coredns-8655855d6/status,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/system:serviceaccount:kube-system:replicaset-controller,client:127.0.0.1 (started: 2020-09-20 15:12:53.338510754 +0000 UTC m=+40.685499939) (total time: 669.900371ms):
  616. Trace[314569415]: [634.009034ms] [633.764805ms] Object stored in database
  617. I0920 15:12:54.050806       6 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "config-volume" (UniqueName: "kubernetes.io/configmap/061e2ac1-fe66-4fa2-a502-e9b9c5d691ad-config-volume") pod "coredns-8655855d6-h6nfg" (UID: "061e2ac1-fe66-4fa2-a502-e9b9c5d691ad")
  618. I0920 15:12:54.050850       6 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "coredns-token-9dft8" (UniqueName: "kubernetes.io/secret/061e2ac1-fe66-4fa2-a502-e9b9c5d691ad-coredns-token-9dft8") pod "coredns-8655855d6-h6nfg" (UID: "061e2ac1-fe66-4fa2-a502-e9b9c5d691ad")
  619. I0920 15:12:54.050876       6 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "config-volume" (UniqueName: "kubernetes.io/configmap/16c592b9-d07d-4c4b-b052-da23b2e14e87-config-volume") pod "local-path-provisioner-6d59f47c7-cx7fr" (UID: "16c592b9-d07d-4c4b-b052-da23b2e14e87")
  620. I0920 15:12:54.050903       6 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "local-path-provisioner-service-account-token-gmbj8" (UniqueName: "kubernetes.io/secret/16c592b9-d07d-4c4b-b052-da23b2e14e87-local-path-provisioner-service-account-token-gmbj8") pod "local-path-provisioner-6d59f47c7-cx7fr" (UID: "16c592b9-d07d-4c4b-b052-da23b2e14e87")
  621. I0920 15:12:54.050945       6 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "values" (UniqueName: "kubernetes.io/configmap/5329ace8-c782-48c9-b57c-7cc2706401ab-values") pod "helm-install-traefik-ksk4q" (UID: "5329ace8-c782-48c9-b57c-7cc2706401ab")
  622. I0920 15:12:54.050966       6 reconciler.go:224] operationExecutor.VerifyControllerAttachedVolume started for volume "helm-traefik-token-qd949" (UniqueName: "kubernetes.io/secret/5329ace8-c782-48c9-b57c-7cc2706401ab-helm-traefik-token-qd949") pod "helm-install-traefik-ksk4q" (UID: "5329ace8-c782-48c9-b57c-7cc2706401ab")
  623. I0920 15:12:54.052538       6 trace.go:116] Trace[449087098]: "GuaranteedUpdate etcd3" type:*apps.ReplicaSet (started: 2020-09-20 15:12:53.349203302 +0000 UTC m=+40.696192543) (total time: 703.308635ms):
  624. Trace[449087098]: [703.228392ms] [701.922663ms] Transaction committed
  625. I0920 15:12:54.052712       6 trace.go:116] Trace[1195230689]: "Update" url:/apis/apps/v1/namespaces/kube-system/replicasets/metrics-server-7566d596c8/status,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/system:serviceaccount:kube-system:replicaset-controller,client:127.0.0.1 (started: 2020-09-20 15:12:53.34897818 +0000 UTC m=+40.695967373) (total time: 703.712571ms):
  626. Trace[1195230689]: [703.607202ms] [703.458017ms] Object stored in database
  627. I0920 15:12:54.299603       6 trace.go:116] Trace[496232544]: "GuaranteedUpdate etcd3" type:*core.ServiceAccount (started: 2020-09-20 15:12:53.435968797 +0000 UTC m=+40.782958081) (total time: 863.592071ms):
  628. Trace[496232544]: [863.545389ms] [862.837687ms] Transaction committed
  629. I0920 15:12:54.305598       6 trace.go:116] Trace[858063449]: "Update" url:/api/v1/namespaces/default/serviceaccounts/default,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/tokens-controller,client:127.0.0.1 (started: 2020-09-20 15:12:53.435318343 +0000 UTC m=+40.782307577) (total time: 870.223335ms):
  630. Trace[858063449]: [870.089257ms] [869.522114ms] Object stored in database
  631. I0920 15:12:54.382585       6 trace.go:116] Trace[201125337]: "GuaranteedUpdate etcd3" type:*batch.Job (started: 2020-09-20 15:12:53.425028173 +0000 UTC m=+40.772017449) (total time: 957.510983ms):
  632. Trace[201125337]: [957.389364ms] [956.211715ms] Transaction committed
  633. I0920 15:12:54.383252       6 trace.go:116] Trace[514116635]: "Update" url:/apis/batch/v1/namespaces/kube-system/jobs/helm-install-traefik/status,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/system:serviceaccount:kube-system:job-controller,client:127.0.0.1 (started: 2020-09-20 15:12:53.424563264 +0000 UTC m=+40.771552600) (total time: 958.625297ms):
  634. Trace[514116635]: [958.410964ms] [958.026515ms] Object stored in database
  635. W0920 15:12:54.874557       6 handler_proxy.go:102] no RequestInfo found in the context
  636. E0920 15:12:54.875328       6 controller.go:114] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
  637. , Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
  638. I0920 15:12:54.903434       6 controller.go:127] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
  639. I0920 15:12:54.917535       6 trace.go:116] Trace[423602771]: "Create" url:/api/v1/namespaces/kube-system/secrets,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/tokens-controller,client:127.0.0.1 (started: 2020-09-20 15:12:53.467811754 +0000 UTC m=+40.814801050) (total time: 1.44965596s):
  640. Trace[423602771]: [1.449553655s] [1.449438308s] Object stored in database
  641. I0920 15:12:54.889748       6 trace.go:116] Trace[1079494548]: "Get" url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cloud-controller-manager,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:12:54.136541866 +0000 UTC m=+41.483531179) (total time: 753.126673ms):
  642. Trace[1079494548]: [753.04204ms] [753.031844ms] About to write a response
  643. W0920 15:12:54.946991       6 garbagecollector.go:644] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
  644. I0920 15:12:54.948104       6 shared_informer.go:223] Waiting for caches to sync for garbage collector
  645. I0920 15:12:54.948430       6 shared_informer.go:230] Caches are synced for garbage collector
  646. I0920 15:12:54.966729       6 trace.go:116] Trace[700077391]: "Create" url:/api/v1/namespaces/kube-system/events,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/system:serviceaccount:kube-system:replicaset-controller,client:127.0.0.1 (started: 2020-09-20 15:12:53.504350374 +0000 UTC m=+40.851339571) (total time: 1.46230357s):
  647. Trace[700077391]: [1.462182322s] [1.462023159s] Object stored in database
  648. I0920 15:12:54.977238       6 trace.go:116] Trace[10893548]: "Create" url:/apis/events.k8s.io/v1beta1/namespaces/kube-system/events,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/scheduler,client:127.0.0.1 (started: 2020-09-20 15:12:53.554730583 +0000 UTC m=+40.901719798) (total time: 1.422435712s):
  649. Trace[10893548]: [1.422357571s] [1.422244354s] Object stored in database
  650. I0920 15:12:55.001621       6 trace.go:116] Trace[840504134]: "GuaranteedUpdate etcd3" type:*apps.Deployment (started: 2020-09-20 15:12:53.603693811 +0000 UTC m=+40.950683014) (total time: 1.397882408s):
  651. Trace[840504134]: [1.397723817s] [1.396170841s] Transaction committed
  652. I0920 15:12:55.012984       6 trace.go:116] Trace[1756540435]: "Update" url:/apis/apps/v1/namespaces/kube-system/deployments/local-path-provisioner/status,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/system:serviceaccount:kube-system:deployment-controller,client:127.0.0.1 (started: 2020-09-20 15:12:53.537532067 +0000 UTC m=+40.884521255) (total time: 1.475387345s):
  653. Trace[1756540435]: [1.464608944s] [1.398550494s] Object stored in database
  654. I0920 15:12:55.038236       6 trace.go:116] Trace[1741232844]: "GuaranteedUpdate etcd3" type:*core.Node (started: 2020-09-20 15:12:53.600214198 +0000 UTC m=+40.947203418) (total time: 1.437974816s):
  655. Trace[1741232844]: [1.437829731s] [1.4349277s] Transaction committed
  656. I0920 15:12:55.039139       6 trace.go:116] Trace[1647325753]: "Patch" url:/api/v1/nodes/k3d-testing-server-0/status,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:12:53.59997726 +0000 UTC m=+40.946966461) (total time: 1.439103119s):
  657. Trace[1647325753]: [1.438688879s] [1.436092032s] Object stored in database
  658. I0920 15:12:55.054003       6 trace.go:116] Trace[498775431]: "GuaranteedUpdate etcd3" type:*discovery.EndpointSlice (started: 2020-09-20 15:12:53.681188091 +0000 UTC m=+41.028177304) (total time: 1.372768842s):
  659. Trace[498775431]: [1.372710432s] [1.369972343s] Transaction committed
  660. I0920 15:12:55.054983       6 trace.go:116] Trace[1842650393]: "Update" url:/apis/discovery.k8s.io/v1beta1/namespaces/kube-system/endpointslices/metrics-server-p9rqb,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/system:serviceaccount:kube-system:endpointslice-controller,client:127.0.0.1 (started: 2020-09-20 15:12:53.681023971 +0000 UTC m=+41.028013267) (total time: 1.373903906s):
  661. Trace[1842650393]: [1.373808436s] [1.373701652s] Object stored in database
  662. I0920 15:12:55.076085       6 trace.go:116] Trace[1015583404]: "Create" url:/apis/events.k8s.io/v1beta1/namespaces/kube-system/events,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/scheduler,client:127.0.0.1 (started: 2020-09-20 15:12:53.88842664 +0000 UTC m=+41.235415842) (total time: 1.186680196s):
  663. Trace[1015583404]: [1.186602644s] [1.186435523s] Object stored in database
  664. I0920 15:12:55.091200       6 trace.go:116] Trace[777396569]: "Create" url:/apis/events.k8s.io/v1beta1/namespaces/kube-system/events,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/scheduler,client:127.0.0.1 (started: 2020-09-20 15:12:53.771715799 +0000 UTC m=+41.118705083) (total time: 1.319433378s):
  665. Trace[777396569]: [1.31936162s] [1.319124756s] Object stored in database
  666. I0920 15:12:55.097185       6 trace.go:116] Trace[1951529860]: "Create" url:/apis/events.k8s.io/v1beta1/namespaces/kube-system/events,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/scheduler,client:127.0.0.1 (started: 2020-09-20 15:12:53.775809333 +0000 UTC m=+41.122798573) (total time: 1.321321964s):
  667. Trace[1951529860]: [1.321210112s] [1.274744712s] Object stored in database
  668. I0920 15:12:55.105763       6 trace.go:116] Trace[1296971871]: "GuaranteedUpdate etcd3" type:*core.Pod (started: 2020-09-20 15:12:53.831576876 +0000 UTC m=+41.178566093) (total time: 1.274105477s):
  669. Trace[1296971871]: [1.274004188s] [1.168637694s] Transaction committed
  670. I0920 15:12:55.106954       6 trace.go:116] Trace[513198728]: "Patch" url:/api/v1/namespaces/kube-system/pods/metrics-server-7566d596c8-8bdgf/status,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:12:53.831367945 +0000 UTC m=+41.178357154) (total time: 1.275531519s):
  671. Trace[513198728]: [105.136978ms] [104.722225ms] About to check admission control
  672. Trace[513198728]: [1.275046924s] [1.169909946s] Object stored in database
  673. I0920 15:12:55.124681       6 trace.go:116] Trace[1611736316]: "GuaranteedUpdate etcd3" type:*apps.Deployment (started: 2020-09-20 15:12:54.173745277 +0000 UTC m=+41.520734561) (total time: 941.828723ms):
  674. Trace[1611736316]: [941.6758ms] [938.579895ms] Transaction committed
  675. I0920 15:12:55.127452       6 trace.go:116] Trace[1611844821]: "Update" url:/apis/apps/v1/namespaces/kube-system/deployments/coredns/status,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/system:serviceaccount:kube-system:deployment-controller,client:127.0.0.1 (started: 2020-09-20 15:12:54.172964662 +0000 UTC m=+41.519953886) (total time: 954.431526ms):
  676. Trace[1611844821]: [952.958945ms] [952.362018ms] Object stored in database
  677. time="2020-09-20T15:12:55.149685170Z" level=info msg="Failed to update node k3d-testing-server-0: Operation cannot be fulfilled on nodes \"k3d-testing-server-0\": the object has been modified; please apply your changes to the latest version and try again"
  678. I0920 15:12:55.152270       6 trace.go:116] Trace[170497176]: "GuaranteedUpdate etcd3" type:*apps.Deployment (started: 2020-09-20 15:12:54.308217389 +0000 UTC m=+41.655206602) (total time: 844.021133ms):
  679. Trace[170497176]: [843.920173ms] [837.569638ms] Transaction committed
  680. I0920 15:12:55.153074       6 trace.go:116] Trace[175149560]: "Update" url:/apis/apps/v1/namespaces/kube-system/deployments/metrics-server/status,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/system:serviceaccount:kube-system:deployment-controller,client:127.0.0.1 (started: 2020-09-20 15:12:54.307902593 +0000 UTC m=+41.654891871) (total time: 844.880206ms):
  681. Trace[175149560]: [844.773411ms] [844.552255ms] Object stored in database
  682. I0920 15:12:55.441930       6 iptables.go:145] Some iptables rules are missing; deleting and recreating rules
  683. I0920 15:12:55.442258       6 iptables.go:167] Deleting iptables rule: -s 10.42.0.0/16 -d 10.42.0.0/16 -j RETURN
  684. I0920 15:12:55.443781       6 iptables.go:167] Deleting iptables rule: -s 10.42.0.0/16 ! -d 224.0.0.0/4 -j MASQUERADE --random-fully
  685. I0920 15:12:55.445517       6 iptables.go:167] Deleting iptables rule: ! -s 10.42.0.0/16 -d 10.42.0.0/24 -j RETURN
  686. I0920 15:12:55.446774       6 iptables.go:167] Deleting iptables rule: ! -s 10.42.0.0/16 -d 10.42.0.0/16 -j MASQUERADE --random-fully
  687. I0920 15:12:55.447962       6 iptables.go:155] Adding iptables rule: -s 10.42.0.0/16 -d 10.42.0.0/16 -j RETURN
  688. I0920 15:12:55.501906       6 iptables.go:145] Some iptables rules are missing; deleting and recreating rules
  689. I0920 15:12:55.502310       6 iptables.go:167] Deleting iptables rule: -s 10.42.0.0/16 -j ACCEPT
  690. I0920 15:12:55.503493       6 iptables.go:167] Deleting iptables rule: -d 10.42.0.0/16 -j ACCEPT
  691. I0920 15:12:55.504677       6 iptables.go:155] Adding iptables rule: -s 10.42.0.0/16 -j ACCEPT
  692. I0920 15:12:55.507373       6 iptables.go:155] Adding iptables rule: -d 10.42.0.0/16 -j ACCEPT
  693. I0920 15:12:55.509607       6 flannel.go:78] Wrote subnet file to /run/flannel/subnet.env
  694. I0920 15:12:55.509881       6 flannel.go:82] Running backend.
  695. I0920 15:12:55.510061       6 vxlan_network.go:60] watching for new subnet leases
  696. I0920 15:12:55.513830       6 iptables.go:155] Adding iptables rule: -s 10.42.0.0/16 ! -d 224.0.0.0/4 -j MASQUERADE --random-fully
  697. I0920 15:12:55.516252       6 iptables.go:155] Adding iptables rule: ! -s 10.42.0.0/16 -d 10.42.0.0/24 -j RETURN
  698. I0920 15:12:55.535737       6 iptables.go:155] Adding iptables rule: ! -s 10.42.0.0/16 -d 10.42.0.0/16 -j MASQUERADE --random-fully
  699. time="2020-09-20T15:12:56.411834294Z" level=info msg="labels have been set successfully on node: k3d-testing-server-0"
  700. I0920 15:12:56.538713       6 network_policy_controller.go:149] Starting network policy controller
  701. I0920 15:13:01.732023       6 trace.go:116] Trace[837727469]: "GuaranteedUpdate etcd3" type:*coordination.Lease (started: 2020-09-20 15:13:01.191666006 +0000 UTC m=+48.538655235) (total time: 540.298918ms):
  702. Trace[837727469]: [540.211876ms] [538.812682ms] Transaction committed
  703. I0920 15:13:01.732795       6 trace.go:116] Trace[474025118]: "Update" url:/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/k3d-testing-server-0,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:13:01.191246849 +0000 UTC m=+48.538236062) (total time: 541.483894ms):
  704. Trace[474025118]: [541.320716ms] [541.047635ms] Object stored in database
  705. I0920 15:13:05.128406       6 trace.go:116] Trace[1460941295]: "Get" url:/api/v1/namespaces/kube-system/endpoints/kube-controller-manager,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:13:04.48961815 +0000 UTC m=+51.836607340) (total time: 638.684761ms):
  706. Trace[1460941295]: [638.584279ms] [638.509993ms] About to write a response
  707. W0920 15:13:06.287743       6 manager.go:1131] Failed to process watch event {EventType:0 Name:/kubepods/besteffort/poddf39d1de-d0ae-4031-bded-6b1d56e07602/db0777faf38346780eb114ef13e8aadf6c858e0aedcb739e41158c06e4574b14 WatchSource:0}: task db0777faf38346780eb114ef13e8aadf6c858e0aedcb739e41158c06e4574b14 not found: not found
  708. I0920 15:13:08.556543       6 trace.go:116] Trace[1195869368]: "GuaranteedUpdate etcd3" type:*core.Endpoints (started: 2020-09-20 15:13:07.970837605 +0000 UTC m=+55.317826786) (total time: 585.608954ms):
  709. Trace[1195869368]: [585.580242ms] [584.412753ms] Transaction committed
  710. I0920 15:13:08.558017       6 trace.go:116] Trace[571872525]: "Update" url:/api/v1/namespaces/kube-system/endpoints/kube-controller-manager,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:13:07.970311278 +0000 UTC m=+55.317300474) (total time: 587.668721ms):
  711. Trace[571872525]: [587.583236ms] [587.201663ms] Object stored in database
  712. I0920 15:13:15.096406       6 trace.go:116] Trace[1395050087]: "Get" url:/api/v1/namespaces/kube-system/endpoints/cloud-controller-manager,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:13:14.548529775 +0000 UTC m=+61.895518969) (total time: 547.822007ms):
  713. Trace[1395050087]: [547.777394ms] [547.761843ms] About to write a response
  714. I0920 15:13:15.458837       6 trace.go:116] Trace[690553907]: "Get" url:/api/v1/namespaces/kube-system/configmaps/k3s,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:13:14.600781712 +0000 UTC m=+61.947770894) (total time: 857.864307ms):
  715. Trace[690553907]: [857.559396ms] [857.539723ms] About to write a response
  716. I0920 15:13:15.503241       6 trace.go:116] Trace[1379600235]: "Get" url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:13:14.607806246 +0000 UTC m=+61.954795477) (total time: 895.370183ms):
  717. Trace[1379600235]: [895.27825ms] [895.242192ms] About to write a response
  718. I0920 15:13:16.315622       6 trace.go:116] Trace[801424339]: "GuaranteedUpdate etcd3" type:*core.ConfigMap (started: 2020-09-20 15:13:15.75367597 +0000 UTC m=+63.100665217) (total time: 561.900771ms):
  719. Trace[801424339]: [561.740773ms] [453.846259ms] Transaction committed
  720. I0920 15:13:16.322026       6 trace.go:116] Trace[556678468]: "Update" url:/api/v1/namespaces/kube-system/configmaps/k3s,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:13:15.70466992 +0000 UTC m=+63.051659112) (total time: 612.97493ms):
  721. Trace[556678468]: [611.549616ms] [563.007499ms] Object stored in database
  722. I0920 15:13:16.610379       6 trace.go:116] Trace[1286714156]: "GuaranteedUpdate etcd3" type:*core.Endpoints (started: 2020-09-20 15:13:15.836381004 +0000 UTC m=+63.183370216) (total time: 773.883707ms):
  723. Trace[1286714156]: [773.788438ms] [772.91665ms] Transaction committed
  724. I0920 15:13:16.612216       6 trace.go:116] Trace[799142590]: "Update" url:/api/v1/namespaces/kube-system/endpoints/kube-scheduler,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:13:15.835960903 +0000 UTC m=+63.182950102) (total time: 776.1484ms):
  725. Trace[799142590]: [775.89225ms] [775.614531ms] Object stored in database
  726. I0920 15:13:16.879726       6 trace.go:116] Trace[2131113100]: "GuaranteedUpdate etcd3" type:*core.Endpoints (started: 2020-09-20 15:13:16.275044957 +0000 UTC m=+63.622034186) (total time: 604.536408ms):
  727. Trace[2131113100]: [604.379792ms] [603.026336ms] Transaction committed
  728. I0920 15:13:16.925999       6 trace.go:116] Trace[1403436437]: "Update" url:/api/v1/namespaces/kube-system/endpoints/kube-controller-manager,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:13:16.273961374 +0000 UTC m=+63.620950877) (total time: 651.982899ms):
  729. Trace[1403436437]: [651.870947ms] [650.841768ms] Object stored in database
  730. I0920 15:13:17.025716       6 trace.go:116] Trace[1695245381]: "GuaranteedUpdate etcd3" type:*v1.Endpoints (started: 2020-09-20 15:13:15.967731352 +0000 UTC m=+63.314720565) (total time: 1.057938561s):
  731. Trace[1695245381]: [318.184232ms] [318.184232ms] initial value restored
  732. Trace[1695245381]: [1.032967232s] [714.351132ms] Transaction committed
  733. I0920 15:13:20.198392       6 trace.go:116] Trace[1486049375]: "Get" url:/api/v1/namespaces/default/endpoints/kubernetes,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:13:17.14949733 +0000 UTC m=+64.496486538) (total time: 3.04862419s):
  734. Trace[1486049375]: [3.048184232s] [3.048168632s] About to write a response
  735. I0920 15:13:20.504227       6 trace.go:116] Trace[1026152727]: "Get" url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:13:17.144512184 +0000 UTC m=+64.491501497) (total time: 3.359671253s):
  736. Trace[1026152727]: [3.359503066s] [3.357181304s] About to write a response
  737. I0920 15:13:20.792666       6 trace.go:116] Trace[552814966]: "Get" url:/api/v1/namespaces/kube-system/configmaps/k3s,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:13:18.795118889 +0000 UTC m=+66.142108100) (total time: 1.99742882s):
  738. Trace[552814966]: [1.997143259s] [1.99710958s] About to write a response
  739. I0920 15:13:20.862406       6 trace.go:116] Trace[22208770]: "GuaranteedUpdate etcd3" type:*coordination.Lease (started: 2020-09-20 15:13:17.323590368 +0000 UTC m=+64.670579590) (total time: 3.538764892s):
  740. Trace[22208770]: [3.53871931s] [3.538006144s] Transaction committed
  741. I0920 15:13:20.862986       6 trace.go:116] Trace[1565140689]: "Update" url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/cloud-controller-manager,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:13:17.32331495 +0000 UTC m=+64.670304140) (total time: 3.53961801s):
  742. Trace[1565140689]: [3.539485975s] [3.539297846s] Object stored in database
  743. I0920 15:13:21.223218       6 trace.go:116] Trace[1869699985]: "List etcd3" key:/jobs,resourceVersion:,limit:500,continue: (started: 2020-09-20 15:13:19.919041961 +0000 UTC m=+67.266031167) (total time: 1.304101819s):
  744. Trace[1869699985]: [1.304101819s] [1.304101819s] END
  745. I0920 15:13:21.282032       6 trace.go:116] Trace[915782787]: "List" url:/apis/batch/v1/jobs,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/system:serviceaccount:kube-system:cronjob-controller,client:127.0.0.1 (started: 2020-09-20 15:13:19.918872383 +0000 UTC m=+67.265861576) (total time: 1.363091347s):
  746. Trace[915782787]: [1.304672963s] [1.304554906s] Listing from storage done
  747. I0920 15:13:21.419766       6 trace.go:116] Trace[1817817918]: "List etcd3" key:/masterleases/,resourceVersion:0,limit:0,continue: (started: 2020-09-20 15:13:20.202408812 +0000 UTC m=+67.549398026) (total time: 1.217243833s):
  748. Trace[1817817918]: [1.217243833s] [1.217243833s] END
  749. I0920 15:13:22.014438       6 trace.go:116] Trace[1839279473]: "Get" url:/apis/discovery.k8s.io/v1beta1/namespaces/default/endpointslices/kubernetes,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:13:21.438648202 +0000 UTC m=+68.785637403) (total time: 575.729723ms):
  750. Trace[1839279473]: [575.524458ms] [575.488871ms] About to write a response
  751. I0920 15:13:22.296228       6 trace.go:116] Trace[1514537780]: "Get" url:/api/v1/namespaces/kube-system,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:13:20.974197681 +0000 UTC m=+68.321186891) (total time: 1.321909978s):
  752. Trace[1514537780]: [1.321774708s] [1.321756126s] About to write a response
  753. I0920 15:13:22.326663       6 trace.go:116] Trace[452922841]: "GuaranteedUpdate etcd3" type:*coordination.Lease (started: 2020-09-20 15:13:16.947790087 +0000 UTC m=+64.294779299) (total time: 5.37877379s):
  754. Trace[452922841]: [5.378679021s] [5.377995642s] Transaction committed
  755. I0920 15:13:22.344153       6 trace.go:116] Trace[2130913937]: "Update" url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:13:16.947550293 +0000 UTC m=+64.294539488) (total time: 5.396510388s):
  756. Trace[2130913937]: [5.379875898s] [5.379707488s] Object stored in database
  757. I0920 15:13:23.354877       6 trace.go:116] Trace[1593025230]: "Get" url:/api/v1/namespaces/kube-node-lease,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1,client:127.0.0.1 (started: 2020-09-20 15:13:22.350176044 +0000 UTC m=+69.697165247) (total time: 1.004624216s):
  758. Trace[1593025230]: [1.003322298s] [1.003307875s] About to write a response
  759. I0920 15:13:24.250315       6 trace.go:116] Trace[1053842330]: "Get" url:/api/v1/namespaces/kube-system/endpoints/cloud-controller-manager,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:13:23.135205872 +0000 UTC m=+70.482195103) (total time: 1.114983757s):
  760. Trace[1053842330]: [1.109868021s] [1.109822973s] About to write a response
  761. I0920 15:13:24.329039       6 trace.go:116] Trace[2025658136]: "GuaranteedUpdate etcd3" type:*coordination.Lease (started: 2020-09-20 15:13:20.520870707 +0000 UTC m=+67.867859892) (total time: 3.808135046s):
  762. Trace[2025658136]: [3.808068199s] [3.709372624s] Transaction committed
  763. I0920 15:13:24.419628       6 trace.go:116] Trace[864624396]: "Update" url:/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/leader-election,client:127.0.0.1 (started: 2020-09-20 15:13:20.520606549 +0000 UTC m=+67.867595773) (total time: 3.898967535s):
  764. Trace[864624396]: [3.898869517s] [3.898726045s] Object stored in database
  765. I0920 15:13:24.463375       6 trace.go:116] Trace[390511180]: "List etcd3" key:/cronjobs,resourceVersion:,limit:500,continue: (started: 2020-09-20 15:13:21.322709422 +0000 UTC m=+68.669698684) (total time: 3.140618839s):
  766. Trace[390511180]: [3.140618839s] [3.140618839s] END
  767. I0920 15:13:24.726672       6 trace.go:116] Trace[1987801515]: "List" url:/apis/batch/v1beta1/cronjobs,user-agent:k3s/v1.18.6+k3s1 (linux/amd64) kubernetes/6f56fa1/system:serviceaccount:kube-system:cronjob-controller,client:127.0.0.1 (started: 2020-09-20 15:13:21.322614996 +0000 UTC m=+68.669604226) (total time: 3.403972197s):
  768. Trace[1987801515]: [3.403655795s] [3.403584779s] Listing from storage done
  769. E0920 15:13:24.811553       6 leaderelection.go:356] Failed to update lock: Put https://127.0.0.1:6444/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager?timeout=10s: context deadline exceeded
  770. I0920 15:13:24.826539       6 leaderelection.go:277] failed to renew lease kube-system/kube-controller-manager: timed out waiting for the condition
  771. E0920 15:13:24.850053       6 resource_quota_controller.go:408] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
  772. I0920 15:13:24.883134       6 event.go:278] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"kube-system", Name:"kube-controller-manager", UID:"5c883b80-f3e3-40d3-b7c8-693ad7dd085c", APIVersion:"v1", ResourceVersion:"527", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' k3d-testing-server-0_392478b6-b490-4800-b192-75aa4977e926 stopped leading
  773. I0920 15:13:25.069655       6 event.go:278] Event(v1.ObjectReference{Kind:"Lease", Namespace:"", Name:"", UID:"", APIVersion:"coordination.k8s.io/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' k3d-testing-server-0_392478b6-b490-4800-b192-75aa4977e926 stopped leading
  774. I0920 15:13:24.915213       6 garbagecollector.go:151] Shutting down garbage collector controller
  775. I0920 15:13:25.079164       6 cleaner.go:90] Shutting down CSR cleaner controller
  776. I0920 15:13:25.079547       6 dynamic_serving_content.go:145] Shutting down csr-controller::/var/lib/rancher/k3s/server/tls/server-ca.crt::/var/lib/rancher/k3s/server/tls/server-ca.key
  777. I0920 15:13:25.079740       6 cronjob_controller.go:101] Shutting down CronJob Manager
  778. I0920 15:13:24.945260       6 endpointslice_controller.go:229] Shutting down endpoint slice controller
  779. I0920 15:13:24.958440       6 resource_quota_controller.go:291] Shutting down resource quota controller
  780. I0920 15:13:24.958733       6 endpoints_controller.go:199] Shutting down endpoint controller
  781. I0920 15:13:25.173620       6 resource_quota_controller.go:260] resource quota controller worker shutting down
  782. I0920 15:13:25.174009       6 resource_quota_controller.go:260] resource quota controller worker shutting down
  783. I0920 15:13:25.176272       6 resource_quota_controller.go:260] resource quota controller worker shutting down
  784. I0920 15:13:25.209471       6 resource_quota_controller.go:260] resource quota controller worker shutting down
  785. I0920 15:13:25.210647       6 resource_quota_controller.go:260] resource quota controller worker shutting down
  786. I0920 15:13:24.958780       6 certificate_controller.go:131] Shutting down certificate controller "csrsigning"
  787. I0920 15:13:24.962342       6 certificate_controller.go:131] Shutting down certificate controller "csrapproving"
  788. I0920 15:13:24.963020       6 pv_controller_base.go:311] Shutting down persistent volume controller
  789. I0920 15:13:24.963083       6 expand_controller.go:331] Shutting down expand controller
  790. I0920 15:13:25.465540       6 pv_controller_base.go:448] volume worker queue shutting down
  791. I0920 15:13:24.963142       6 attach_detach_controller.go:387] Shutting down attach detach controller
  792. I0920 15:13:24.963178       6 stateful_set.go:158] Shutting down statefulset controller
  793. I0920 15:13:24.963282       6 pvc_protection_controller.go:113] Shutting down PVC protection controller
  794. I0920 15:13:24.963390       6 replica_set.go:193] Shutting down replicationcontroller controller
  795. I0920 15:13:24.963440       6 disruption.go:348] Shutting down disruption controller
  796. F0920 15:13:24.987668       6 controllermanager.go:279] leaderelection lost
  797.  
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!