Advertisement
Guest User

Untitled

a guest
Apr 22nd, 2020
30
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 29.33 KB | None
  1. > kubectl logs flannel-migration-rg6gp -n kube-system -f
  2. 2020-04-22 18:33:27.705 [INFO][1] main.go 88: Loaded configuration from environment config=&config.Config{LogLevel:"info", ReconcilerPeriod:"5m", CompactionPeriod:"10m", EnabledControllers:"flannelmigration", WorkloadEndpointWorkers:1, ProfileWorkers:1, PolicyWorkers:1, NodeWorkers:1, Kubeconfig:"", HealthEnabled:true, SyncNodeLabels:true, DatastoreType:"kubernetes"}
  3. W0422 18:33:27.707042 1 client_config.go:541] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
  4. 2020-04-22 18:33:27.707 [INFO][1] main.go 109: Ensuring Calico datastore is initialized
  5. 2020-04-22 18:33:27.715 [INFO][1] main.go 165: Loaded Flannel configuration from environment flannelConfig=&flannelmigration.Config{FlannelNetwork:"", FlannelDaemonsetName:"kube-flannel-ds-amd64", FlannelMTU:0, FlannelIPMasq:true, FlannelSubnetLen:24, FlannelAnnotationPrefix:"flannel.alpha.coreos.com", FlannelVNI:1, FlannelPort:8472, CalicoDaemonsetName:"calico-node", CniConfigDir:"/etc/cni/net.d", PodNodeName:"pt-rancheragent-1-worker", FlannelSubnetEnv:"", DebugWaitBeforeExit:0, DebugWaitBeforeStart:0}
  6. 2020-04-22 18:33:27.715 [INFO][1] main.go 183: Starting status report routine
  7. 2020-04-22 18:33:27.715 [INFO][1] main.go 368: Starting controller ControllerType="FlannelMigration"
  8. 2020-04-22 18:33:27.715 [INFO][1] migration_controller.go 154: Starting Migration controller
  9. 2020-04-22 18:33:27.726 [INFO][1] migration_controller.go 303: Canal daemonset exists, we are migrating from Canal to Calico.
  10. 2020-04-22 18:33:27.732 [INFO][1] migration_controller.go 258: Trying to read Flannel env config by executing into daemonet pod.
  11. 2020-04-22 18:33:28.135 [INFO][1] k8s_resources.go 500: Kubectl exec canal-m8zrl(kube-flannel) completed successfully.
  12. ---[exec canal-m8zrl --namespace=kube-system -c=kube-flannel cat run/flannel/subnet.env]---
  13. FLANNEL_NETWORK=10.42.0.0/16
  14. FLANNEL_SUBNET=10.42.3.1/24
  15. FLANNEL_MTU=8951
  16. FLANNEL_IPMASQ=true
  17.  
  18. ------
  19. 2020-04-22 18:33:28.135 [INFO][1] migration_controller.go 279: Flannel env config parsed successfully. flannelConfig=&flannelmigration.Config{FlannelNetwork:"10.42.0.0/16", FlannelDaemonsetName:"canal", FlannelMTU:8951, FlannelIPMasq:true, FlannelSubnetLen:24, FlannelAnnotationPrefix:"flannel.alpha.coreos.com", FlannelVNI:1, FlannelPort:8472, CalicoDaemonsetName:"calico-node", CniConfigDir:"/etc/cni/net.d", PodNodeName:"pt-rancheragent-1-worker", FlannelSubnetEnv:"", DebugWaitBeforeExit:0, DebugWaitBeforeStart:0}
  20. 2020-04-22 18:33:28.141 [INFO][1] k8s_resources.go 572: Config map flannel-migration-config updated flannel_subnet_env=FLANNEL_NETWORK=10.42.0.0/16;FLANNEL_SUBNET=10.42.3.1/24;FLANNEL_MTU=8951;FLANNEL_IPMASQ=true;.
  21. 2020-04-22 18:33:28.141 [INFO][1] migration_controller.go 288: Flannel subnet.env stored in migration config map: 'FLANNEL_NETWORK=10.42.0.0/16;FLANNEL_SUBNET=10.42.3.1/24;FLANNEL_MTU=8951;FLANNEL_IPMASQ=true;'.
  22. 2020-04-22 18:33:28.143 [INFO][1] k8s_resources.go 562: Config map calico-config has veth_mtu=8951 already. No update needed.
  23. 2020-04-22 18:33:28.148 [INFO][1] network_migrator.go 83: Network migrator initialised, container image calico/node:v3.13.2, CNI config /etc/cni/net.d/10-calico.conflist.
  24. 2020-04-22 18:33:28.148 [INFO][1] ipam_migrator.go 289: Ensure default IPv4 pool (cidr 10.42.0.0/16, blockSize 26, nat true, vxlanMode Always).
  25. 2020-04-22 18:33:28.150 [INFO][1] ipam_migrator.go 357: Use current default IPv4 pool.
  26. 2020-04-22 18:33:28.153 [INFO][1] ipam_migrator.go 405: Default Felix configration got correct vxlanEnabled, VNI(1), port(8472), mtu(8951).
  27. 2020-04-22 18:33:28.153 [INFO][1] migration_controller.go 177: Waiting to sync with Kubernetes API (Nodes)
  28. 2020-04-22 18:33:28.253 [INFO][1] migration_controller.go 181: Finished syncing with Kubernetes API (Nodes)
  29. 2020-04-22 18:33:28.258 [INFO][1] migration_controller.go 408: Migration controller is running on node pt-rancheragent-1-worker.
  30. 2020-04-22 18:33:28.261 [INFO][1] migration_controller.go 439: Controller node pt-rancheragent-1-worker is last node to be migrated.
  31. 2020-04-22 18:33:28.261 [INFO][1] ipam_migrator.go 183: Start IPAM migration process for 3 nodes.
  32. 2020-04-22 18:33:28.261 [INFO][1] ipam_migrator.go 185: Start setting up Calico IPAM for node pt-rancheragent-0-etcd[index 0].
  33. 2020-04-22 18:33:28.261 [INFO][1] ipam_migrator.go 161: node pt-rancheragent-0-etcd has vxlan setup from Flannel (vtepMac: '9e:80:ab:be:0c:b6', vtepIP: '10.42.1.0').
  34. 2020-04-22 18:33:28.267 [INFO][1] ipam_block_reader_writer.go 129: Trying to create affinity in pending state host="pt-rancheragent-0-etcd" subnet=10.42.1.0/26
  35. 2020-04-22 18:33:28.271 [INFO][1] ipam_block_reader_writer.go 140: Block affinity already exists, getting existing affinity host="pt-rancheragent-0-etcd" subnet=10.42.1.0/26
  36. 2020-04-22 18:33:28.274 [INFO][1] ipam_block_reader_writer.go 148: Got existing affinity host="pt-rancheragent-0-etcd" subnet=10.42.1.0/26
  37. 2020-04-22 18:33:28.276 [INFO][1] ipam_block_reader_writer.go 156: Existing affinity is already confirmed host="pt-rancheragent-0-etcd" subnet=10.42.1.0/26
  38. 2020-04-22 18:33:28.276 [INFO][1] ipam_block_reader_writer.go 182: Attempting to create a new block host="pt-rancheragent-0-etcd" subnet=10.42.1.0/26
  39. 2020-04-22 18:33:28.284 [INFO][1] ipam_block_reader_writer.go 187: The block already exists, getting it from data store host="pt-rancheragent-0-etcd" subnet=10.42.1.0/26
  40. 2020-04-22 18:33:28.292 [INFO][1] ipam_block_reader_writer.go 203: Block is already claimed by this host, confirm the affinity host="pt-rancheragent-0-etcd" subnet=10.42.1.0/26
  41. 2020-04-22 18:33:28.292 [INFO][1] ipam_block_reader_writer.go 234: Confirming affinity host="pt-rancheragent-0-etcd" subnet=10.42.1.0/26
  42. 2020-04-22 18:33:28.296 [INFO][1] ipam_block_reader_writer.go 249: Successfully confirmed affinity host="pt-rancheragent-0-etcd" subnet=10.42.1.0/26
  43. 2020-04-22 18:33:28.296 [INFO][1] ipam_block_reader_writer.go 129: Trying to create affinity in pending state host="pt-rancheragent-0-etcd" subnet=10.42.1.64/26
  44. 2020-04-22 18:33:28.358 [INFO][1] ipam_block_reader_writer.go 140: Block affinity already exists, getting existing affinity host="pt-rancheragent-0-etcd" subnet=10.42.1.64/26
  45. 2020-04-22 18:33:28.362 [INFO][1] ipam_block_reader_writer.go 148: Got existing affinity host="pt-rancheragent-0-etcd" subnet=10.42.1.64/26
  46. 2020-04-22 18:33:28.362 [INFO][1] ipam_block_reader_writer.go 156: Existing affinity is already confirmed host="pt-rancheragent-0-etcd" subnet=10.42.1.64/26
  47. 2020-04-22 18:33:28.362 [INFO][1] ipam_block_reader_writer.go 182: Attempting to create a new block host="pt-rancheragent-0-etcd" subnet=10.42.1.64/26
  48. 2020-04-22 18:33:28.369 [INFO][1] ipam_block_reader_writer.go 187: The block already exists, getting it from data store host="pt-rancheragent-0-etcd" subnet=10.42.1.64/26
  49. 2020-04-22 18:33:28.371 [INFO][1] ipam_block_reader_writer.go 203: Block is already claimed by this host, confirm the affinity host="pt-rancheragent-0-etcd" subnet=10.42.1.64/26
  50. 2020-04-22 18:33:28.371 [INFO][1] ipam_block_reader_writer.go 234: Confirming affinity host="pt-rancheragent-0-etcd" subnet=10.42.1.64/26
  51. 2020-04-22 18:33:28.374 [INFO][1] ipam_block_reader_writer.go 249: Successfully confirmed affinity host="pt-rancheragent-0-etcd" subnet=10.42.1.64/26
  52. 2020-04-22 18:33:28.374 [INFO][1] ipam_block_reader_writer.go 129: Trying to create affinity in pending state host="pt-rancheragent-0-etcd" subnet=10.42.1.128/26
  53. 2020-04-22 18:33:28.383 [INFO][1] ipam_block_reader_writer.go 140: Block affinity already exists, getting existing affinity host="pt-rancheragent-0-etcd" subnet=10.42.1.128/26
  54. 2020-04-22 18:33:28.385 [INFO][1] ipam_block_reader_writer.go 148: Got existing affinity host="pt-rancheragent-0-etcd" subnet=10.42.1.128/26
  55. 2020-04-22 18:33:28.385 [INFO][1] ipam_block_reader_writer.go 156: Existing affinity is already confirmed host="pt-rancheragent-0-etcd" subnet=10.42.1.128/26
  56. 2020-04-22 18:33:28.385 [INFO][1] ipam_block_reader_writer.go 182: Attempting to create a new block host="pt-rancheragent-0-etcd" subnet=10.42.1.128/26
  57. 2020-04-22 18:33:28.394 [INFO][1] ipam_block_reader_writer.go 187: The block already exists, getting it from data store host="pt-rancheragent-0-etcd" subnet=10.42.1.128/26
  58. 2020-04-22 18:33:28.398 [INFO][1] ipam_block_reader_writer.go 203: Block is already claimed by this host, confirm the affinity host="pt-rancheragent-0-etcd" subnet=10.42.1.128/26
  59. 2020-04-22 18:33:28.398 [INFO][1] ipam_block_reader_writer.go 234: Confirming affinity host="pt-rancheragent-0-etcd" subnet=10.42.1.128/26
  60. 2020-04-22 18:33:28.402 [INFO][1] ipam_block_reader_writer.go 249: Successfully confirmed affinity host="pt-rancheragent-0-etcd" subnet=10.42.1.128/26
  61. 2020-04-22 18:33:28.402 [INFO][1] ipam_block_reader_writer.go 129: Trying to create affinity in pending state host="pt-rancheragent-0-etcd" subnet=10.42.1.192/26
  62. 2020-04-22 18:33:28.407 [INFO][1] ipam_block_reader_writer.go 140: Block affinity already exists, getting existing affinity host="pt-rancheragent-0-etcd" subnet=10.42.1.192/26
  63. 2020-04-22 18:33:28.409 [INFO][1] ipam_block_reader_writer.go 148: Got existing affinity host="pt-rancheragent-0-etcd" subnet=10.42.1.192/26
  64. 2020-04-22 18:33:28.409 [INFO][1] ipam_block_reader_writer.go 156: Existing affinity is already confirmed host="pt-rancheragent-0-etcd" subnet=10.42.1.192/26
  65. 2020-04-22 18:33:28.409 [INFO][1] ipam_block_reader_writer.go 182: Attempting to create a new block host="pt-rancheragent-0-etcd" subnet=10.42.1.192/26
  66. 2020-04-22 18:33:28.414 [INFO][1] ipam_block_reader_writer.go 187: The block already exists, getting it from data store host="pt-rancheragent-0-etcd" subnet=10.42.1.192/26
  67. 2020-04-22 18:33:28.416 [INFO][1] ipam_block_reader_writer.go 203: Block is already claimed by this host, confirm the affinity host="pt-rancheragent-0-etcd" subnet=10.42.1.192/26
  68. 2020-04-22 18:33:28.416 [INFO][1] ipam_block_reader_writer.go 234: Confirming affinity host="pt-rancheragent-0-etcd" subnet=10.42.1.192/26
  69. 2020-04-22 18:33:28.419 [INFO][1] ipam_block_reader_writer.go 249: Successfully confirmed affinity host="pt-rancheragent-0-etcd" subnet=10.42.1.192/26
  70. 2020-04-22 18:33:28.419 [INFO][1] ipam_migrator.go 169: 4 IPAM blocks claimed for node pt-rancheragent-0-etcd.
  71. 2020-04-22 18:33:28.419 [INFO][1] ipam_migrator.go 198: Updating Calico Node pt-rancheragent-0-etcd with vtep IP 10.42.1.0, Mac 9e:80:ab:be:0c:b6.
  72. 2020-04-22 18:33:28.424 [INFO][1] ipam_migrator.go 207: Calico Node pt-rancheragent-0-etcd vtep IP been assigned already.
  73. 2020-04-22 18:33:28.427 [INFO][1] ipam_migrator.go 252: Calico Node current value: &{TypeMeta:{Kind:Node APIVersion:projectcalico.org/v3} ObjectMeta:{Name:pt-rancheragent-0-etcd GenerateName: Namespace: SelfLink: UID:b20fd5a5-aa01-458b-bbaf-600e88d785f3 ResourceVersion:7124 Generation:0 CreationTimestamp:2020-04-22 17:59:19 +0000 UTC DeletionTimestamp:<nil> DeletionGracePeriodSeconds:<nil> Labels:map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:pt-rancheragent-0-etcd kubernetes.io/os:linux node-role.kubernetes.io/etcd:true projectcalico.org/node-flannel-migration-in-progress:true projectcalico.org/node-network-during-migration:none] Annotations:map[projectcalico.org/kube-labels:{"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux","kubernetes.io/arch":"amd64","kubernetes.io/hostname":"pt-rancheragent-0-etcd","kubernetes.io/os":"linux","node-role.kubernetes.io/etcd":"true","projectcalico.org/node-flannel-migration-in-progress":"true","projectcalico.org/node-network-during-migration":"none"}] OwnerReferences:[] Finalizers:[] ClusterName: ManagedFields:[]} Spec:{BGP:0xc00030af00 IPv4VXLANTunnelAddr:10.42.1.0 VXLANTunnelMACAddr:9e:80:ab:be:0c:b6 OrchRefs:[]}}.
  74. 2020-04-22 18:33:28.432 [INFO][1] ipam_migrator.go 265: Calico Node pt-rancheragent-0-etcd vtep IP/Mac/PublicIP updated.
  75. 2020-04-22 18:33:28.432 [INFO][1] ipam_migrator.go 177: Setting up Calico IPAM for node pt-rancheragent-0-etcd completed successfully.
  76. 2020-04-22 18:33:28.432 [INFO][1] ipam_migrator.go 185: Start setting up Calico IPAM for node pt-rancheragent-0-worker[index 1].
  77. 2020-04-22 18:33:28.432 [INFO][1] ipam_migrator.go 161: node pt-rancheragent-0-worker has vxlan setup from Flannel (vtepMac: '2e:5b:d3:1d:4e:c8', vtepIP: '10.42.2.0').
  78. 2020-04-22 18:33:28.437 [INFO][1] ipam_block_reader_writer.go 129: Trying to create affinity in pending state host="pt-rancheragent-0-worker" subnet=10.42.2.0/26
  79. 2020-04-22 18:33:28.441 [INFO][1] ipam_block_reader_writer.go 140: Block affinity already exists, getting existing affinity host="pt-rancheragent-0-worker" subnet=10.42.2.0/26
  80. 2020-04-22 18:33:28.442 [INFO][1] ipam_block_reader_writer.go 148: Got existing affinity host="pt-rancheragent-0-worker" subnet=10.42.2.0/26
  81. 2020-04-22 18:33:28.442 [INFO][1] ipam_block_reader_writer.go 156: Existing affinity is already confirmed host="pt-rancheragent-0-worker" subnet=10.42.2.0/26
  82. 2020-04-22 18:33:28.443 [INFO][1] ipam_block_reader_writer.go 182: Attempting to create a new block host="pt-rancheragent-0-worker" subnet=10.42.2.0/26
  83. 2020-04-22 18:33:28.450 [INFO][1] ipam_block_reader_writer.go 187: The block already exists, getting it from data store host="pt-rancheragent-0-worker" subnet=10.42.2.0/26
  84. 2020-04-22 18:33:28.456 [INFO][1] ipam_block_reader_writer.go 203: Block is already claimed by this host, confirm the affinity host="pt-rancheragent-0-worker" subnet=10.42.2.0/26
  85. 2020-04-22 18:33:28.456 [INFO][1] ipam_block_reader_writer.go 234: Confirming affinity host="pt-rancheragent-0-worker" subnet=10.42.2.0/26
  86. 2020-04-22 18:33:28.459 [INFO][1] ipam_block_reader_writer.go 249: Successfully confirmed affinity host="pt-rancheragent-0-worker" subnet=10.42.2.0/26
  87. 2020-04-22 18:33:28.459 [INFO][1] ipam_block_reader_writer.go 129: Trying to create affinity in pending state host="pt-rancheragent-0-worker" subnet=10.42.2.64/26
  88. 2020-04-22 18:33:28.465 [INFO][1] ipam_block_reader_writer.go 140: Block affinity already exists, getting existing affinity host="pt-rancheragent-0-worker" subnet=10.42.2.64/26
  89. 2020-04-22 18:33:28.467 [INFO][1] ipam_block_reader_writer.go 148: Got existing affinity host="pt-rancheragent-0-worker" subnet=10.42.2.64/26
  90. 2020-04-22 18:33:28.467 [INFO][1] ipam_block_reader_writer.go 156: Existing affinity is already confirmed host="pt-rancheragent-0-worker" subnet=10.42.2.64/26
  91. 2020-04-22 18:33:28.467 [INFO][1] ipam_block_reader_writer.go 182: Attempting to create a new block host="pt-rancheragent-0-worker" subnet=10.42.2.64/26
  92. 2020-04-22 18:33:28.471 [INFO][1] ipam_block_reader_writer.go 187: The block already exists, getting it from data store host="pt-rancheragent-0-worker" subnet=10.42.2.64/26
  93. 2020-04-22 18:33:28.473 [INFO][1] ipam_block_reader_writer.go 203: Block is already claimed by this host, confirm the affinity host="pt-rancheragent-0-worker" subnet=10.42.2.64/26
  94. 2020-04-22 18:33:28.473 [INFO][1] ipam_block_reader_writer.go 234: Confirming affinity host="pt-rancheragent-0-worker" subnet=10.42.2.64/26
  95. 2020-04-22 18:33:28.476 [INFO][1] ipam_block_reader_writer.go 249: Successfully confirmed affinity host="pt-rancheragent-0-worker" subnet=10.42.2.64/26
  96. 2020-04-22 18:33:28.476 [INFO][1] ipam_block_reader_writer.go 129: Trying to create affinity in pending state host="pt-rancheragent-0-worker" subnet=10.42.2.128/26
  97. 2020-04-22 18:33:28.479 [INFO][1] ipam_block_reader_writer.go 140: Block affinity already exists, getting existing affinity host="pt-rancheragent-0-worker" subnet=10.42.2.128/26
  98. 2020-04-22 18:33:28.481 [INFO][1] ipam_block_reader_writer.go 148: Got existing affinity host="pt-rancheragent-0-worker" subnet=10.42.2.128/26
  99. 2020-04-22 18:33:28.481 [INFO][1] ipam_block_reader_writer.go 156: Existing affinity is already confirmed host="pt-rancheragent-0-worker" subnet=10.42.2.128/26
  100. 2020-04-22 18:33:28.481 [INFO][1] ipam_block_reader_writer.go 182: Attempting to create a new block host="pt-rancheragent-0-worker" subnet=10.42.2.128/26
  101. 2020-04-22 18:33:28.487 [INFO][1] ipam_block_reader_writer.go 187: The block already exists, getting it from data store host="pt-rancheragent-0-worker" subnet=10.42.2.128/26
  102. 2020-04-22 18:33:28.489 [INFO][1] ipam_block_reader_writer.go 203: Block is already claimed by this host, confirm the affinity host="pt-rancheragent-0-worker" subnet=10.42.2.128/26
  103. 2020-04-22 18:33:28.489 [INFO][1] ipam_block_reader_writer.go 234: Confirming affinity host="pt-rancheragent-0-worker" subnet=10.42.2.128/26
  104. 2020-04-22 18:33:28.492 [INFO][1] ipam_block_reader_writer.go 249: Successfully confirmed affinity host="pt-rancheragent-0-worker" subnet=10.42.2.128/26
  105. 2020-04-22 18:33:28.492 [INFO][1] ipam_block_reader_writer.go 129: Trying to create affinity in pending state host="pt-rancheragent-0-worker" subnet=10.42.2.192/26
  106. 2020-04-22 18:33:28.495 [INFO][1] ipam_block_reader_writer.go 140: Block affinity already exists, getting existing affinity host="pt-rancheragent-0-worker" subnet=10.42.2.192/26
  107. 2020-04-22 18:33:28.497 [INFO][1] ipam_block_reader_writer.go 148: Got existing affinity host="pt-rancheragent-0-worker" subnet=10.42.2.192/26
  108. 2020-04-22 18:33:28.497 [INFO][1] ipam_block_reader_writer.go 156: Existing affinity is already confirmed host="pt-rancheragent-0-worker" subnet=10.42.2.192/26
  109. 2020-04-22 18:33:28.497 [INFO][1] ipam_block_reader_writer.go 182: Attempting to create a new block host="pt-rancheragent-0-worker" subnet=10.42.2.192/26
  110. 2020-04-22 18:33:28.501 [INFO][1] ipam_block_reader_writer.go 187: The block already exists, getting it from data store host="pt-rancheragent-0-worker" subnet=10.42.2.192/26
  111. 2020-04-22 18:33:28.503 [INFO][1] ipam_block_reader_writer.go 203: Block is already claimed by this host, confirm the affinity host="pt-rancheragent-0-worker" subnet=10.42.2.192/26
  112. 2020-04-22 18:33:28.503 [INFO][1] ipam_block_reader_writer.go 234: Confirming affinity host="pt-rancheragent-0-worker" subnet=10.42.2.192/26
  113. 2020-04-22 18:33:28.506 [INFO][1] ipam_block_reader_writer.go 249: Successfully confirmed affinity host="pt-rancheragent-0-worker" subnet=10.42.2.192/26
  114. 2020-04-22 18:33:28.506 [INFO][1] ipam_migrator.go 169: 4 IPAM blocks claimed for node pt-rancheragent-0-worker.
  115. 2020-04-22 18:33:28.506 [INFO][1] ipam_migrator.go 198: Updating Calico Node pt-rancheragent-0-worker with vtep IP 10.42.2.0, Mac 2e:5b:d3:1d:4e:c8.
  116. 2020-04-22 18:33:28.511 [INFO][1] ipam_migrator.go 207: Calico Node pt-rancheragent-0-worker vtep IP been assigned already.
  117. 2020-04-22 18:33:28.513 [INFO][1] ipam_migrator.go 252: Calico Node current value: &{TypeMeta:{Kind:Node APIVersion:projectcalico.org/v3} ObjectMeta:{Name:pt-rancheragent-0-worker GenerateName: Namespace: SelfLink: UID:ce3467aa-ab15-47a1-97aa-523ef8f83199 ResourceVersion:6438 Generation:0 CreationTimestamp:2020-04-22 18:01:00 +0000 UTC DeletionTimestamp:<nil> DeletionGracePeriodSeconds:<nil> Labels:map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:pt-rancheragent-0-worker kubernetes.io/os:linux node-role.kubernetes.io/worker:true projectcalico.org/node-network-during-migration:flannel] Annotations:map[projectcalico.org/kube-labels:{"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux","kubernetes.io/arch":"amd64","kubernetes.io/hostname":"pt-rancheragent-0-worker","kubernetes.io/os":"linux","node-role.kubernetes.io/worker":"true","projectcalico.org/node-network-during-migration":"flannel"}] OwnerReferences:[] Finalizers:[] ClusterName: ManagedFields:[]} Spec:{BGP:0xc0002edcc0 IPv4VXLANTunnelAddr:10.42.2.0 VXLANTunnelMACAddr:2e:5b:d3:1d:4e:c8 OrchRefs:[]}}.
  118. 2020-04-22 18:33:28.519 [INFO][1] ipam_migrator.go 265: Calico Node pt-rancheragent-0-worker vtep IP/Mac/PublicIP updated.
  119. 2020-04-22 18:33:28.519 [INFO][1] ipam_migrator.go 177: Setting up Calico IPAM for node pt-rancheragent-0-worker completed successfully.
  120. 2020-04-22 18:33:28.519 [INFO][1] ipam_migrator.go 185: Start setting up Calico IPAM for node pt-rancheragent-1-worker[index 2].
  121. 2020-04-22 18:33:28.519 [INFO][1] ipam_migrator.go 161: node pt-rancheragent-1-worker has vxlan setup from Flannel (vtepMac: 'ea:79:d4:2b:f3:0f', vtepIP: '10.42.3.0').
  122. 2020-04-22 18:33:28.524 [INFO][1] ipam_block_reader_writer.go 129: Trying to create affinity in pending state host="pt-rancheragent-1-worker" subnet=10.42.3.0/26
  123. 2020-04-22 18:33:28.527 [INFO][1] ipam_block_reader_writer.go 140: Block affinity already exists, getting existing affinity host="pt-rancheragent-1-worker" subnet=10.42.3.0/26
  124. 2020-04-22 18:33:28.529 [INFO][1] ipam_block_reader_writer.go 148: Got existing affinity host="pt-rancheragent-1-worker" subnet=10.42.3.0/26
  125. 2020-04-22 18:33:28.529 [INFO][1] ipam_block_reader_writer.go 156: Existing affinity is already confirmed host="pt-rancheragent-1-worker" subnet=10.42.3.0/26
  126. 2020-04-22 18:33:28.529 [INFO][1] ipam_block_reader_writer.go 182: Attempting to create a new block host="pt-rancheragent-1-worker" subnet=10.42.3.0/26
  127. 2020-04-22 18:33:28.534 [INFO][1] ipam_block_reader_writer.go 187: The block already exists, getting it from data store host="pt-rancheragent-1-worker" subnet=10.42.3.0/26
  128. 2020-04-22 18:33:28.536 [INFO][1] ipam_block_reader_writer.go 203: Block is already claimed by this host, confirm the affinity host="pt-rancheragent-1-worker" subnet=10.42.3.0/26
  129. 2020-04-22 18:33:28.536 [INFO][1] ipam_block_reader_writer.go 234: Confirming affinity host="pt-rancheragent-1-worker" subnet=10.42.3.0/26
  130. 2020-04-22 18:33:28.539 [INFO][1] ipam_block_reader_writer.go 249: Successfully confirmed affinity host="pt-rancheragent-1-worker" subnet=10.42.3.0/26
  131. 2020-04-22 18:33:28.539 [INFO][1] ipam_block_reader_writer.go 129: Trying to create affinity in pending state host="pt-rancheragent-1-worker" subnet=10.42.3.64/26
  132. 2020-04-22 18:33:28.543 [INFO][1] ipam_block_reader_writer.go 140: Block affinity already exists, getting existing affinity host="pt-rancheragent-1-worker" subnet=10.42.3.64/26
  133. 2020-04-22 18:33:28.545 [INFO][1] ipam_block_reader_writer.go 148: Got existing affinity host="pt-rancheragent-1-worker" subnet=10.42.3.64/26
  134. 2020-04-22 18:33:28.545 [INFO][1] ipam_block_reader_writer.go 156: Existing affinity is already confirmed host="pt-rancheragent-1-worker" subnet=10.42.3.64/26
  135. 2020-04-22 18:33:28.545 [INFO][1] ipam_block_reader_writer.go 182: Attempting to create a new block host="pt-rancheragent-1-worker" subnet=10.42.3.64/26
  136. 2020-04-22 18:33:28.549 [INFO][1] ipam_block_reader_writer.go 187: The block already exists, getting it from data store host="pt-rancheragent-1-worker" subnet=10.42.3.64/26
  137. 2020-04-22 18:33:28.551 [INFO][1] ipam_block_reader_writer.go 203: Block is already claimed by this host, confirm the affinity host="pt-rancheragent-1-worker" subnet=10.42.3.64/26
  138. 2020-04-22 18:33:28.551 [INFO][1] ipam_block_reader_writer.go 234: Confirming affinity host="pt-rancheragent-1-worker" subnet=10.42.3.64/26
  139. 2020-04-22 18:33:28.553 [INFO][1] ipam_block_reader_writer.go 249: Successfully confirmed affinity host="pt-rancheragent-1-worker" subnet=10.42.3.64/26
  140. 2020-04-22 18:33:28.553 [INFO][1] ipam_block_reader_writer.go 129: Trying to create affinity in pending state host="pt-rancheragent-1-worker" subnet=10.42.3.128/26
  141. 2020-04-22 18:33:28.557 [INFO][1] ipam_block_reader_writer.go 140: Block affinity already exists, getting existing affinity host="pt-rancheragent-1-worker" subnet=10.42.3.128/26
  142. 2020-04-22 18:33:28.559 [INFO][1] ipam_block_reader_writer.go 148: Got existing affinity host="pt-rancheragent-1-worker" subnet=10.42.3.128/26
  143. 2020-04-22 18:33:28.559 [INFO][1] ipam_block_reader_writer.go 156: Existing affinity is already confirmed host="pt-rancheragent-1-worker" subnet=10.42.3.128/26
  144. 2020-04-22 18:33:28.559 [INFO][1] ipam_block_reader_writer.go 182: Attempting to create a new block host="pt-rancheragent-1-worker" subnet=10.42.3.128/26
  145. 2020-04-22 18:33:28.562 [INFO][1] ipam_block_reader_writer.go 187: The block already exists, getting it from data store host="pt-rancheragent-1-worker" subnet=10.42.3.128/26
  146. 2020-04-22 18:33:28.564 [INFO][1] ipam_block_reader_writer.go 203: Block is already claimed by this host, confirm the affinity host="pt-rancheragent-1-worker" subnet=10.42.3.128/26
  147. 2020-04-22 18:33:28.564 [INFO][1] ipam_block_reader_writer.go 234: Confirming affinity host="pt-rancheragent-1-worker" subnet=10.42.3.128/26
  148. 2020-04-22 18:33:28.567 [INFO][1] ipam_block_reader_writer.go 249: Successfully confirmed affinity host="pt-rancheragent-1-worker" subnet=10.42.3.128/26
  149. 2020-04-22 18:33:28.567 [INFO][1] ipam_block_reader_writer.go 129: Trying to create affinity in pending state host="pt-rancheragent-1-worker" subnet=10.42.3.192/26
  150. 2020-04-22 18:33:28.571 [INFO][1] ipam_block_reader_writer.go 140: Block affinity already exists, getting existing affinity host="pt-rancheragent-1-worker" subnet=10.42.3.192/26
  151. 2020-04-22 18:33:28.573 [INFO][1] ipam_block_reader_writer.go 148: Got existing affinity host="pt-rancheragent-1-worker" subnet=10.42.3.192/26
  152. 2020-04-22 18:33:28.573 [INFO][1] ipam_block_reader_writer.go 156: Existing affinity is already confirmed host="pt-rancheragent-1-worker" subnet=10.42.3.192/26
  153. 2020-04-22 18:33:28.573 [INFO][1] ipam_block_reader_writer.go 182: Attempting to create a new block host="pt-rancheragent-1-worker" subnet=10.42.3.192/26
  154. 2020-04-22 18:33:28.576 [INFO][1] ipam_block_reader_writer.go 187: The block already exists, getting it from data store host="pt-rancheragent-1-worker" subnet=10.42.3.192/26
  155. 2020-04-22 18:33:28.578 [INFO][1] ipam_block_reader_writer.go 203: Block is already claimed by this host, confirm the affinity host="pt-rancheragent-1-worker" subnet=10.42.3.192/26
  156. 2020-04-22 18:33:28.578 [INFO][1] ipam_block_reader_writer.go 234: Confirming affinity host="pt-rancheragent-1-worker" subnet=10.42.3.192/26
  157. 2020-04-22 18:33:28.581 [INFO][1] ipam_block_reader_writer.go 249: Successfully confirmed affinity host="pt-rancheragent-1-worker" subnet=10.42.3.192/26
  158. 2020-04-22 18:33:28.581 [INFO][1] ipam_migrator.go 169: 4 IPAM blocks claimed for node pt-rancheragent-1-worker.
  159. 2020-04-22 18:33:28.581 [INFO][1] ipam_migrator.go 198: Updating Calico Node pt-rancheragent-1-worker with vtep IP 10.42.3.0, Mac ea:79:d4:2b:f3:0f.
  160. 2020-04-22 18:33:28.592 [INFO][1] ipam_migrator.go 207: Calico Node pt-rancheragent-1-worker vtep IP been assigned already.
  161. 2020-04-22 18:33:28.594 [INFO][1] ipam_migrator.go 252: Calico Node current value: &{TypeMeta:{Kind:Node APIVersion:projectcalico.org/v3} ObjectMeta:{Name:pt-rancheragent-1-worker GenerateName: Namespace: SelfLink: UID:4e94fc51-1154-4003-baa0-16536999ab52 ResourceVersion:6521 Generation:0 CreationTimestamp:2020-04-22 18:01:01 +0000 UTC DeletionTimestamp:<nil> DeletionGracePeriodSeconds:<nil> Labels:map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux kubernetes.io/arch:amd64 kubernetes.io/hostname:pt-rancheragent-1-worker kubernetes.io/os:linux node-role.kubernetes.io/worker:true projectcalico.org/node-network-during-migration:flannel] Annotations:map[projectcalico.org/kube-labels:{"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux","kubernetes.io/arch":"amd64","kubernetes.io/hostname":"pt-rancheragent-1-worker","kubernetes.io/os":"linux","node-role.kubernetes.io/worker":"true","projectcalico.org/node-network-during-migration":"flannel"}] OwnerReferences:[] Finalizers:[] ClusterName: ManagedFields:[]} Spec:{BGP:0xc000764b40 IPv4VXLANTunnelAddr:10.42.3.0 VXLANTunnelMACAddr:ea:79:d4:2b:f3:0f OrchRefs:[]}}.
  162. 2020-04-22 18:33:28.601 [INFO][1] ipam_migrator.go 265: Calico Node pt-rancheragent-1-worker vtep IP/Mac/PublicIP updated.
  163. 2020-04-22 18:33:28.601 [INFO][1] ipam_migrator.go 177: Setting up Calico IPAM for node pt-rancheragent-1-worker completed successfully.
  164. 2020-04-22 18:33:28.601 [INFO][1] ipam_migrator.go 191: 3 nodes completed IPAM migration process.
  165. 2020-04-22 18:33:28.603 [INFO][1] network_migrator.go 226: Start network migration process for 3 nodes.
  166. 2020-04-22 18:33:28.603 [INFO][1] network_migrator.go 244: Start setting up Calico network for node pt-rancheragent-0-etcd[index 0].
  167. 2020-04-22 18:33:28.603 [INFO][1] network_migrator.go 139: Setting node label to disable Flannel daemonset pod on node pt-rancheragent-0-etcd.
  168. 2020-04-22 18:33:28.605 [INFO][1] k8s_resources.go 506: Start drain node pt-rancheragent-0-etcd
  169. 2020-04-22 18:33:28.705 [INFO][1] k8s_resources.go 514: Drain node pt-rancheragent-0-etcd completed successfully.
  170. ---Drain Node Logs---
  171. node/pt-rancheragent-0-etcd already cordoned
  172. WARNING: ignoring DaemonSet-managed Pods: cattle-system/cattle-node-agent-djkx2
  173. node/pt-rancheragent-0-etcd drained
  174.  
  175. ------
  176. 2020-04-22 18:33:28.705 [INFO][1] network_migrator.go 158: Removing flannel tunnel device/routes on pt-rancheragent-0-etcd.
  177. 2020-04-22 18:33:28.705 [INFO][1] k8s_resources.go 201: Create pod on node pt-rancheragent-0-etcd to run [ ip link show flannel.1 || exit 0 && { echo '{ "name": "dummy", "plugins": [{ "type": "flannel-migration-in-progress" }]}' > /host//etc/cni/net.d/10-calico.conflist ; ip link delete flannel.1; } && exit 0 || exit 1 ].
  178. 2020-04-22 18:33:33.722 [ERROR][1] k8s_resources.go 253: Error waiting for pod remove-flannel-j84dv success or timeout. error=pods "remove-flannel-j84dv" not found
  179. 2020-04-22 18:33:33.723 [INFO][1] k8s_resources.go 258: Failed to get log. pod remove-flannel-j84dv container remove-flannel. error=unknown (get pods remove-flannel-j84dv)
  180. 2020-04-22 18:33:43.723 [ERROR][1] network_migrator.go 165: failed to remove flannel network on node pt-rancheragent-0-etcd error=pods "remove-flannel-j84dv" not found
  181. 2020-04-22 18:33:43.726 [ERROR][1] migration_controller.go 202: Error running network migration. error=pods "remove-flannel-j84dv" not found
  182. 2020-04-22 18:33:43.726 [FATAL][1] migration_controller.go 138: Migration controller stopped.
Advertisement
RAW Paste Data Copied
Advertisement