API tools faq
paste
dfarrell07

subm_random_fail_20201102

dfarrell07
Nov 2nd, 2020 (edited)
100
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 1.69 MB | None | 0 0
raw download clone embed print report
  1. 2020-11-02T11:47:32.8025850Z ##[section]Starting: Request a runner to run this job
  2. 2020-11-02T11:47:33.7767208Z Can't find any online and idle self-hosted runner in current repository that matches the required labels: 'ubuntu-latest'
  3. 2020-11-02T11:47:33.7767322Z Can't find any online and idle self-hosted runner in current repository's account/organization that matches the required labels: 'ubuntu-latest'
  4. 2020-11-02T11:47:33.7767538Z Found online and idle hosted runner in current repository's account/organization that matches the required labels: 'ubuntu-latest'
  5. 2020-11-02T11:47:33.9053517Z ##[section]Finishing: Request a runner to run this job
  6. 2020-11-02T11:47:43.0189131Z Current runner version: '2.273.5'
  7. 2020-11-02T11:47:43.0217297Z ##[group]Operating System
  8. 2020-11-02T11:47:43.0218031Z Ubuntu
  9. 2020-11-02T11:47:43.0218321Z 18.04.5
  10. 2020-11-02T11:47:43.0218620Z LTS
  11. 2020-11-02T11:47:43.0218861Z ##[endgroup]
  12. 2020-11-02T11:47:43.0219223Z ##[group]Virtual Environment
  13. 2020-11-02T11:47:43.0219660Z Environment: ubuntu-18.04
  14. 2020-11-02T11:47:43.0220037Z Version: 20201026.1
  15. 2020-11-02T11:47:43.0220776Z Included Software: https://github.com/actions/virtual-environments/blob/ubuntu18/20201026.1/images/linux/Ubuntu1804-README.md
  16. 2020-11-02T11:47:43.0221652Z ##[endgroup]
  17. 2020-11-02T11:47:43.0222704Z Prepare workflow directory
  18. 2020-11-02T11:47:43.0418514Z Prepare all required actions
  19. 2020-11-02T11:47:43.0428436Z Getting action download info
  20. 2020-11-02T11:47:43.2707450Z Download action repository 'actions/checkout@v2'
  21. 2020-11-02T11:47:45.4049805Z ##[group]Run actions/checkout@v2
  22. 2020-11-02T11:47:45.4050671Z with:
  23. 2020-11-02T11:47:45.4051282Z repository: submariner-io/submariner
  24. 2020-11-02T11:47:45.4052092Z token: ***
  25. 2020-11-02T11:47:45.4052447Z ssh-strict: true
  26. 2020-11-02T11:47:45.4052880Z persist-credentials: true
  27. 2020-11-02T11:47:45.4053285Z clean: true
  28. 2020-11-02T11:47:45.4053623Z fetch-depth: 1
  29. 2020-11-02T11:47:45.4053905Z lfs: false
  30. 2020-11-02T11:47:45.4054297Z submodules: false
  31. 2020-11-02T11:47:45.4054667Z ##[endgroup]
  32. 2020-11-02T11:47:46.3287090Z Syncing repository: submariner-io/submariner
  33. 2020-11-02T11:47:46.3287999Z ##[group]Getting Git version info
  34. 2020-11-02T11:47:46.3288836Z Working directory is '/home/runner/work/submariner/submariner'
  35. 2020-11-02T11:47:46.3289404Z [command]/usr/bin/git version
  36. 2020-11-02T11:47:46.3289832Z git version 2.29.0
  37. 2020-11-02T11:47:46.3290590Z ##[endgroup]
  38. 2020-11-02T11:47:46.3291275Z Deleting the contents of '/home/runner/work/submariner/submariner'
  39. 2020-11-02T11:47:46.3292402Z ##[group]Initializing the repository
  40. 2020-11-02T11:47:46.3292905Z [command]/usr/bin/git init /home/runner/work/submariner/submariner
  41. 2020-11-02T11:47:46.3293557Z Initialized empty Git repository in /home/runner/work/submariner/submariner/.git/
  42. 2020-11-02T11:47:46.3294571Z [command]/usr/bin/git remote add origin https://github.com/submariner-io/submariner
  43. 2020-11-02T11:47:46.3295171Z ##[endgroup]
  44. 2020-11-02T11:47:46.3295653Z ##[group]Disabling automatic garbage collection
  45. 2020-11-02T11:47:46.3296322Z [command]/usr/bin/git config --local gc.auto 0
  46. 2020-11-02T11:47:46.3296722Z ##[endgroup]
  47. 2020-11-02T11:47:46.3298156Z ##[group]Setting up auth
  48. 2020-11-02T11:47:46.3298927Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
  49. 2020-11-02T11:47:46.3300059Z [command]/usr/bin/git submodule foreach --recursive git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :
  50. 2020-11-02T11:47:46.3301192Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
  51. 2020-11-02T11:47:46.3302719Z [command]/usr/bin/git submodule foreach --recursive git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :
  52. 2020-11-02T11:47:46.3304141Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic ***
  53. 2020-11-02T11:47:46.3304778Z ##[endgroup]
  54. 2020-11-02T11:47:46.3305146Z ##[group]Fetching the repository
  55. 2020-11-02T11:47:46.3306330Z [command]/usr/bin/git -c protocol.version=2 fetch --no-tags --prune --progress --no-recurse-submodules --depth=1 origin +7bbf15868a67ab9669e8b42266b47e8c04adceff:refs/remotes/pull/915/merge
  56. 2020-11-02T11:47:46.3307252Z remote: Enumerating objects: 230, done.
  57. 2020-11-02T11:47:46.3307679Z remote: Counting objects: 0% (1/230)
  58. 2020-11-02T11:47:46.3308069Z remote: Counting objects: 1% (3/230)
  59. 2020-11-02T11:47:46.3308470Z remote: Counting objects: 2% (5/230)
  60. 2020-11-02T11:47:46.3308889Z remote: Counting objects: 3% (7/230)
  61. 2020-11-02T11:47:46.3309237Z remote: Counting objects: 4% (10/230)
  62. 2020-11-02T11:47:46.3310020Z remote: Counting objects: 5% (12/230)
  63. 2020-11-02T11:47:46.3310444Z remote: Counting objects: 6% (14/230)
  64. 2020-11-02T11:47:46.3310843Z remote: Counting objects: 7% (17/230)
  65. 2020-11-02T11:47:46.3311256Z remote: Counting objects: 8% (19/230)
  66. 2020-11-02T11:47:46.3311816Z remote: Counting objects: 9% (21/230)
  67. 2020-11-02T11:47:46.3312204Z remote: Counting objects: 10% (23/230)
  68. 2020-11-02T11:47:46.3312595Z remote: Counting objects: 11% (26/230)
  69. 2020-11-02T11:47:46.3313009Z remote: Counting objects: 12% (28/230)
  70. 2020-11-02T11:47:46.3313410Z remote: Counting objects: 13% (30/230)
  71. 2020-11-02T11:47:46.3313757Z remote: Counting objects: 14% (33/230)
  72. 2020-11-02T11:47:46.3314305Z remote: Counting objects: 15% (35/230)
  73. 2020-11-02T11:47:46.3314720Z remote: Counting objects: 16% (37/230)
  74. 2020-11-02T11:47:46.3315108Z remote: Counting objects: 17% (40/230)
  75. 2020-11-02T11:47:46.3315502Z remote: Counting objects: 18% (42/230)
  76. 2020-11-02T11:47:46.3315892Z remote: Counting objects: 19% (44/230)
  77. 2020-11-02T11:47:46.3316272Z remote: Counting objects: 20% (46/230)
  78. 2020-11-02T11:47:46.3316694Z remote: Counting objects: 21% (49/230)
  79. 2020-11-02T11:47:46.3317042Z remote: Counting objects: 22% (51/230)
  80. 2020-11-02T11:47:46.3317433Z remote: Counting objects: 23% (53/230)
  81. 2020-11-02T11:47:46.3317834Z remote: Counting objects: 24% (56/230)
  82. 2020-11-02T11:47:46.3318222Z remote: Counting objects: 25% (58/230)
  83. 2020-11-02T11:47:46.3319044Z remote: Counting objects: 26% (60/230)
  84. 2020-11-02T11:47:46.3319457Z remote: Counting objects: 27% (63/230)
  85. 2020-11-02T11:47:46.3319880Z remote: Counting objects: 28% (65/230)
  86. 2020-11-02T11:47:46.3320421Z remote: Counting objects: 29% (67/230)
  87. 2020-11-02T11:47:46.3320789Z remote: Counting objects: 30% (69/230)
  88. 2020-11-02T11:47:46.3321259Z remote: Counting objects: 31% (72/230)
  89. 2020-11-02T11:47:46.3321677Z remote: Counting objects: 32% (74/230)
  90. 2020-11-02T11:47:46.3322403Z remote: Counting objects: 33% (76/230)
  91. 2020-11-02T11:47:46.3322795Z remote: Counting objects: 34% (79/230)
  92. 2020-11-02T11:47:46.3323537Z remote: Counting objects: 35% (81/230)
  93. 2020-11-02T11:47:46.3324111Z remote: Counting objects: 36% (83/230)
  94. 2020-11-02T11:47:46.3324540Z remote: Counting objects: 37% (86/230)
  95. 2020-11-02T11:47:46.3324908Z remote: Counting objects: 38% (88/230)
  96. 2020-11-02T11:47:46.3325350Z remote: Counting objects: 39% (90/230)
  97. 2020-11-02T11:47:46.3325795Z remote: Counting objects: 40% (92/230)
  98. 2020-11-02T11:47:46.3326801Z remote: Counting objects: 41% (95/230)
  99. 2020-11-02T11:47:46.3327419Z remote: Counting objects: 42% (97/230)
  100. 2020-11-02T11:47:46.3327813Z remote: Counting objects: 43% (99/230)
  101. 2020-11-02T11:47:46.3328201Z remote: Counting objects: 44% (102/230)
  102. 2020-11-02T11:47:46.3328844Z remote: Counting objects: 45% (104/230)
  103. 2020-11-02T11:47:46.3329538Z remote: Counting objects: 46% (106/230)
  104. 2020-11-02T11:47:46.3329941Z remote: Counting objects: 47% (109/230)
  105. 2020-11-02T11:47:46.3330381Z remote: Counting objects: 48% (111/230)
  106. 2020-11-02T11:47:46.3330811Z remote: Counting objects: 49% (113/230)
  107. 2020-11-02T11:47:46.3331301Z remote: Counting objects: 50% (115/230)
  108. 2020-11-02T11:47:46.3331867Z remote: Counting objects: 51% (118/230)
  109. 2020-11-02T11:47:46.3332262Z remote: Counting objects: 52% (120/230)
  110. 2020-11-02T11:47:46.3333042Z remote: Counting objects: 53% (122/230)
  111. 2020-11-02T11:47:46.3333454Z remote: Counting objects: 54% (125/230)
  112. 2020-11-02T11:47:46.3333812Z remote: Counting objects: 55% (127/230)
  113. 2020-11-02T11:47:46.3334224Z remote: Counting objects: 56% (129/230)
  114. 2020-11-02T11:47:46.3334616Z remote: Counting objects: 57% (132/230)
  115. 2020-11-02T11:47:46.3335183Z remote: Counting objects: 58% (134/230)
  116. 2020-11-02T11:47:46.3335569Z remote: Counting objects: 59% (136/230)
  117. 2020-11-02T11:47:46.3335951Z remote: Counting objects: 60% (138/230)
  118. 2020-11-02T11:47:46.3336333Z remote: Counting objects: 61% (141/230)
  119. 2020-11-02T11:47:46.3336714Z remote: Counting objects: 62% (143/230)
  120. 2020-11-02T11:47:46.3337058Z remote: Counting objects: 63% (145/230)
  121. 2020-11-02T11:47:46.3337456Z remote: Counting objects: 64% (148/230)
  122. 2020-11-02T11:47:46.3337841Z remote: Counting objects: 65% (150/230)
  123. 2020-11-02T11:47:46.3338224Z remote: Counting objects: 66% (152/230)
  124. 2020-11-02T11:47:46.3338813Z remote: Counting objects: 67% (155/230)
  125. 2020-11-02T11:47:46.3340295Z remote: Counting objects: 68% (157/230)
  126. 2020-11-02T11:47:46.3340737Z remote: Counting objects: 69% (159/230)
  127. 2020-11-02T11:47:46.3341145Z remote: Counting objects: 70% (161/230)
  128. 2020-11-02T11:47:46.3341856Z remote: Counting objects: 71% (164/230)
  129. 2020-11-02T11:47:46.3342461Z remote: Counting objects: 72% (166/230)
  130. 2020-11-02T11:47:46.3342793Z remote: Counting objects: 73% (168/230)
  131. 2020-11-02T11:47:46.3343164Z remote: Counting objects: 74% (171/230)
  132. 2020-11-02T11:47:46.3343554Z remote: Counting objects: 75% (173/230)
  133. 2020-11-02T11:47:46.3343967Z remote: Counting objects: 76% (175/230)
  134. 2020-11-02T11:47:46.3344336Z remote: Counting objects: 77% (178/230)
  135. 2020-11-02T11:47:46.3344705Z remote: Counting objects: 78% (180/230)
  136. 2020-11-02T11:47:46.3345072Z remote: Counting objects: 79% (182/230)
  137. 2020-11-02T11:47:46.3345439Z remote: Counting objects: 80% (184/230)
  138. 2020-11-02T11:47:46.3345815Z remote: Counting objects: 81% (187/230)
  139. 2020-11-02T11:47:46.3346150Z remote: Counting objects: 82% (189/230)
  140. 2020-11-02T11:47:46.3346593Z remote: Counting objects: 83% (191/230)
  141. 2020-11-02T11:47:46.3347024Z remote: Counting objects: 84% (194/230)
  142. 2020-11-02T11:47:46.3347416Z remote: Counting objects: 85% (196/230)
  143. 2020-11-02T11:47:46.3364602Z remote: Counting objects: 86% (198/230)
  144. 2020-11-02T11:47:46.3365316Z remote: Counting objects: 87% (201/230)
  145. 2020-11-02T11:47:46.3365801Z remote: Counting objects: 88% (203/230)
  146. 2020-11-02T11:47:46.3366424Z remote: Counting objects: 89% (205/230)
  147. 2020-11-02T11:47:46.3366969Z remote: Counting objects: 90% (207/230)
  148. 2020-11-02T11:47:46.3367403Z remote: Counting objects: 91% (210/230)
  149. 2020-11-02T11:47:46.3368007Z remote: Counting objects: 92% (212/230)
  150. 2020-11-02T11:47:46.3368647Z remote: Counting objects: 93% (214/230)
  151. 2020-11-02T11:47:46.3369450Z remote: Counting objects: 94% (217/230)
  152. 2020-11-02T11:47:46.3370226Z remote: Counting objects: 95% (219/230)
  153. 2020-11-02T11:47:46.3370658Z remote: Counting objects: 96% (221/230)
  154. 2020-11-02T11:47:46.3371104Z remote: Counting objects: 97% (224/230)
  155. 2020-11-02T11:47:46.3371486Z remote: Counting objects: 98% (226/230)
  156. 2020-11-02T11:47:46.3372163Z remote: Counting objects: 99% (228/230)
  157. 2020-11-02T11:47:46.3372582Z remote: Counting objects: 100% (230/230)
  158. 2020-11-02T11:47:46.3373358Z remote: Counting objects: 100% (230/230), done.
  159. 2020-11-02T11:47:46.3373805Z remote: Compressing objects: 0% (1/187)
  160. 2020-11-02T11:47:46.3374273Z remote: Compressing objects: 1% (2/187)
  161. 2020-11-02T11:47:46.3374751Z remote: Compressing objects: 2% (4/187)
  162. 2020-11-02T11:47:46.3375361Z remote: Compressing objects: 3% (6/187)
  163. 2020-11-02T11:47:46.3375980Z remote: Compressing objects: 4% (8/187)
  164. 2020-11-02T11:47:46.3376393Z remote: Compressing objects: 5% (10/187)
  165. 2020-11-02T11:47:46.3376846Z remote: Compressing objects: 6% (12/187)
  166. 2020-11-02T11:47:46.3377484Z remote: Compressing objects: 7% (14/187)
  167. 2020-11-02T11:47:46.3378600Z remote: Compressing objects: 8% (15/187)
  168. 2020-11-02T11:47:46.3379150Z remote: Compressing objects: 9% (17/187)
  169. 2020-11-02T11:47:46.3379647Z remote: Compressing objects: 10% (19/187)
  170. 2020-11-02T11:47:46.3380154Z remote: Compressing objects: 11% (21/187)
  171. 2020-11-02T11:47:46.3380653Z remote: Compressing objects: 12% (23/187)
  172. 2020-11-02T11:47:46.3381101Z remote: Compressing objects: 13% (25/187)
  173. 2020-11-02T11:47:46.3381903Z remote: Compressing objects: 14% (27/187)
  174. 2020-11-02T11:47:46.3382411Z remote: Compressing objects: 15% (29/187)
  175. 2020-11-02T11:47:46.3382876Z remote: Compressing objects: 16% (30/187)
  176. 2020-11-02T11:47:46.3383382Z remote: Compressing objects: 17% (32/187)
  177. 2020-11-02T11:47:46.3383941Z remote: Compressing objects: 18% (34/187)
  178. 2020-11-02T11:47:46.3384429Z remote: Compressing objects: 19% (36/187)
  179. 2020-11-02T11:47:46.3385214Z remote: Compressing objects: 20% (38/187)
  180. 2020-11-02T11:47:46.3385840Z remote: Compressing objects: 21% (40/187)
  181. 2020-11-02T11:47:46.3386428Z remote: Compressing objects: 22% (42/187)
  182. 2020-11-02T11:47:46.3386885Z remote: Compressing objects: 23% (44/187)
  183. 2020-11-02T11:47:46.3387346Z remote: Compressing objects: 24% (45/187)
  184. 2020-11-02T11:47:46.3387859Z remote: Compressing objects: 25% (47/187)
  185. 2020-11-02T11:47:46.3388688Z remote: Compressing objects: 26% (49/187)
  186. 2020-11-02T11:47:46.3389343Z remote: Compressing objects: 27% (51/187)
  187. 2020-11-02T11:47:46.3389823Z remote: Compressing objects: 28% (53/187)
  188. 2020-11-02T11:47:46.3390301Z remote: Compressing objects: 29% (55/187)
  189. 2020-11-02T11:47:46.3390728Z remote: Compressing objects: 30% (57/187)
  190. 2020-11-02T11:47:46.3391375Z remote: Compressing objects: 31% (58/187)
  191. 2020-11-02T11:47:46.3392530Z remote: Compressing objects: 32% (60/187)
  192. 2020-11-02T11:47:46.3392971Z remote: Compressing objects: 33% (62/187)
  193. 2020-11-02T11:47:46.3393440Z remote: Compressing objects: 34% (64/187)
  194. 2020-11-02T11:47:46.3393878Z remote: Compressing objects: 35% (66/187)
  195. 2020-11-02T11:47:46.3394319Z remote: Compressing objects: 36% (68/187)
  196. 2020-11-02T11:47:46.3394945Z remote: Compressing objects: 37% (70/187)
  197. 2020-11-02T11:47:46.3395402Z remote: Compressing objects: 38% (72/187)
  198. 2020-11-02T11:47:46.3395802Z remote: Compressing objects: 39% (73/187)
  199. 2020-11-02T11:47:46.3396357Z remote: Compressing objects: 40% (75/187)
  200. 2020-11-02T11:47:46.3397147Z remote: Compressing objects: 41% (77/187)
  201. 2020-11-02T11:47:46.3397622Z remote: Compressing objects: 42% (79/187)
  202. 2020-11-02T11:47:46.3398163Z remote: Compressing objects: 43% (81/187)
  203. 2020-11-02T11:47:46.3398817Z remote: Compressing objects: 44% (83/187)
  204. 2020-11-02T11:47:46.3399289Z remote: Compressing objects: 45% (85/187)
  205. 2020-11-02T11:47:46.3399719Z remote: Compressing objects: 46% (87/187)
  206. 2020-11-02T11:47:46.3400218Z remote: Compressing objects: 47% (88/187)
  207. 2020-11-02T11:47:46.3400682Z remote: Compressing objects: 48% (90/187)
  208. 2020-11-02T11:47:46.3401142Z remote: Compressing objects: 49% (92/187)
  209. 2020-11-02T11:47:46.3401789Z remote: Compressing objects: 50% (94/187)
  210. 2020-11-02T11:47:46.3402528Z remote: Compressing objects: 51% (96/187)
  211. 2020-11-02T11:47:46.3403002Z remote: Compressing objects: 52% (98/187)
  212. 2020-11-02T11:47:46.3403647Z remote: Compressing objects: 53% (100/187)
  213. 2020-11-02T11:47:46.3404294Z remote: Compressing objects: 54% (101/187)
  214. 2020-11-02T11:47:46.3404773Z remote: Compressing objects: 55% (103/187)
  215. 2020-11-02T11:47:46.3405605Z remote: Compressing objects: 56% (105/187)
  216. 2020-11-02T11:47:46.3406056Z remote: Compressing objects: 57% (107/187)
  217. 2020-11-02T11:47:46.3406496Z remote: Compressing objects: 58% (109/187)
  218. 2020-11-02T11:47:46.3407082Z remote: Compressing objects: 59% (111/187)
  219. 2020-11-02T11:47:46.3407557Z remote: Compressing objects: 60% (113/187)
  220. 2020-11-02T11:47:46.3408013Z remote: Compressing objects: 61% (115/187)
  221. 2020-11-02T11:47:46.3408850Z remote: Compressing objects: 62% (116/187)
  222. 2020-11-02T11:47:46.3409338Z remote: Compressing objects: 63% (118/187)
  223. 2020-11-02T11:47:46.3409982Z remote: Compressing objects: 64% (120/187)
  224. 2020-11-02T11:47:46.3410633Z remote: Compressing objects: 65% (122/187)
  225. 2020-11-02T11:47:46.3411051Z remote: Compressing objects: 66% (124/187)
  226. 2020-11-02T11:47:46.3411543Z remote: Compressing objects: 67% (126/187)
  227. 2020-11-02T11:47:46.3412379Z remote: Compressing objects: 68% (128/187)
  228. 2020-11-02T11:47:46.3412991Z remote: Compressing objects: 69% (130/187)
  229. 2020-11-02T11:47:46.3413477Z remote: Compressing objects: 70% (131/187)
  230. 2020-11-02T11:47:46.3413998Z remote: Compressing objects: 71% (133/187)
  231. 2020-11-02T11:47:46.3414481Z remote: Compressing objects: 72% (135/187)
  232. 2020-11-02T11:47:46.3414970Z remote: Compressing objects: 73% (137/187)
  233. 2020-11-02T11:47:46.3415590Z remote: Compressing objects: 74% (139/187)
  234. 2020-11-02T11:47:46.3416044Z remote: Compressing objects: 75% (141/187)
  235. 2020-11-02T11:47:46.3416504Z remote: Compressing objects: 76% (143/187)
  236. 2020-11-02T11:47:46.3416957Z remote: Compressing objects: 77% (144/187)
  237. 2020-11-02T11:47:46.3417448Z remote: Compressing objects: 78% (146/187)
  238. 2020-11-02T11:47:46.3417902Z remote: Compressing objects: 79% (148/187)
  239. 2020-11-02T11:47:46.3418360Z remote: Compressing objects: 80% (150/187)
  240. 2020-11-02T11:47:46.3419065Z remote: Compressing objects: 81% (152/187)
  241. 2020-11-02T11:47:46.3419496Z remote: Compressing objects: 82% (154/187)
  242. 2020-11-02T11:47:46.3419966Z remote: Compressing objects: 83% (156/187)
  243. 2020-11-02T11:47:46.3420464Z remote: Compressing objects: 84% (158/187)
  244. 2020-11-02T11:47:46.3420927Z remote: Compressing objects: 85% (159/187)
  245. 2020-11-02T11:47:46.3421554Z remote: Compressing objects: 86% (161/187)
  246. 2020-11-02T11:47:46.3422007Z remote: Compressing objects: 87% (163/187)
  247. 2020-11-02T11:47:46.3422460Z remote: Compressing objects: 88% (165/187)
  248. 2020-11-02T11:47:46.3422916Z remote: Compressing objects: 89% (167/187)
  249. 2020-11-02T11:47:46.3423385Z remote: Compressing objects: 90% (169/187)
  250. 2020-11-02T11:47:46.3423793Z remote: Compressing objects: 91% (171/187)
  251. 2020-11-02T11:47:46.3424245Z remote: Compressing objects: 92% (173/187)
  252. 2020-11-02T11:47:46.3424719Z remote: Compressing objects: 93% (174/187)
  253. 2020-11-02T11:47:46.3425180Z remote: Compressing objects: 94% (176/187)
  254. 2020-11-02T11:47:46.3425634Z remote: Compressing objects: 95% (178/187)
  255. 2020-11-02T11:47:46.3426082Z remote: Compressing objects: 96% (180/187)
  256. 2020-11-02T11:47:46.3426521Z remote: Compressing objects: 97% (182/187)
  257. 2020-11-02T11:47:46.3616789Z remote: Compressing objects: 98% (184/187)
  258. 2020-11-02T11:47:46.3617255Z remote: Compressing objects: 99% (186/187)
  259. 2020-11-02T11:47:46.3617672Z remote: Compressing objects: 100% (187/187)
  260. 2020-11-02T11:47:46.3618096Z remote: Compressing objects: 100% (187/187), done.
  261. 2020-11-02T11:47:46.3618488Z Receiving objects: 0% (1/230)
  262. 2020-11-02T11:47:46.3618836Z Receiving objects: 1% (3/230)
  263. 2020-11-02T11:47:46.3619177Z Receiving objects: 2% (5/230)
  264. 2020-11-02T11:47:46.3619514Z Receiving objects: 3% (7/230)
  265. 2020-11-02T11:47:46.3619856Z Receiving objects: 4% (10/230)
  266. 2020-11-02T11:47:46.3620196Z Receiving objects: 5% (12/230)
  267. 2020-11-02T11:47:46.3620556Z Receiving objects: 6% (14/230)
  268. 2020-11-02T11:47:46.3620901Z Receiving objects: 7% (17/230)
  269. 2020-11-02T11:47:46.3621245Z Receiving objects: 8% (19/230)
  270. 2020-11-02T11:47:46.3621585Z Receiving objects: 9% (21/230)
  271. 2020-11-02T11:47:46.3622165Z Receiving objects: 10% (23/230)
  272. 2020-11-02T11:47:46.3622507Z Receiving objects: 11% (26/230)
  273. 2020-11-02T11:47:46.3622846Z Receiving objects: 12% (28/230)
  274. 2020-11-02T11:47:46.3623185Z Receiving objects: 13% (30/230)
  275. 2020-11-02T11:47:46.3623518Z Receiving objects: 14% (33/230)
  276. 2020-11-02T11:47:46.3623859Z Receiving objects: 15% (35/230)
  277. 2020-11-02T11:47:46.3624197Z Receiving objects: 16% (37/230)
  278. 2020-11-02T11:47:46.3624537Z Receiving objects: 17% (40/230)
  279. 2020-11-02T11:47:46.3624876Z Receiving objects: 18% (42/230)
  280. 2020-11-02T11:47:46.3625214Z Receiving objects: 19% (44/230)
  281. 2020-11-02T11:47:46.3625555Z Receiving objects: 20% (46/230)
  282. 2020-11-02T11:47:46.3625888Z Receiving objects: 21% (49/230)
  283. 2020-11-02T11:47:46.3626230Z Receiving objects: 22% (51/230)
  284. 2020-11-02T11:47:46.3626659Z Receiving objects: 23% (53/230)
  285. 2020-11-02T11:47:46.3627018Z Receiving objects: 24% (56/230)
  286. 2020-11-02T11:47:46.3627357Z Receiving objects: 25% (58/230)
  287. 2020-11-02T11:47:46.3627707Z Receiving objects: 26% (60/230)
  288. 2020-11-02T11:47:46.3628045Z Receiving objects: 27% (63/230)
  289. 2020-11-02T11:47:46.3628384Z Receiving objects: 28% (65/230)
  290. 2020-11-02T11:47:46.3628715Z Receiving objects: 29% (67/230)
  291. 2020-11-02T11:47:46.3629058Z Receiving objects: 30% (69/230)
  292. 2020-11-02T11:47:46.3629397Z Receiving objects: 31% (72/230)
  293. 2020-11-02T11:47:46.3629738Z Receiving objects: 32% (74/230)
  294. 2020-11-02T11:47:46.3630076Z Receiving objects: 33% (76/230)
  295. 2020-11-02T11:47:46.3630416Z Receiving objects: 34% (79/230)
  296. 2020-11-02T11:47:46.3630756Z Receiving objects: 35% (81/230)
  297. 2020-11-02T11:47:46.3631090Z Receiving objects: 36% (83/230)
  298. 2020-11-02T11:47:46.3631432Z Receiving objects: 37% (86/230)
  299. 2020-11-02T11:47:46.3631932Z Receiving objects: 38% (88/230)
  300. 2020-11-02T11:47:46.3632269Z Receiving objects: 39% (90/230)
  301. 2020-11-02T11:47:46.3632597Z Receiving objects: 40% (92/230)
  302. 2020-11-02T11:47:46.3632925Z Receiving objects: 41% (95/230)
  303. 2020-11-02T11:47:46.3633255Z Receiving objects: 42% (97/230)
  304. 2020-11-02T11:47:46.3633590Z Receiving objects: 43% (99/230)
  305. 2020-11-02T11:47:46.3633913Z Receiving objects: 44% (102/230)
  306. 2020-11-02T11:47:46.3634246Z Receiving objects: 45% (104/230)
  307. 2020-11-02T11:47:46.3634579Z Receiving objects: 46% (106/230)
  308. 2020-11-02T11:47:46.3634907Z Receiving objects: 47% (109/230)
  309. 2020-11-02T11:47:46.3635242Z Receiving objects: 48% (111/230)
  310. 2020-11-02T11:47:46.3635572Z Receiving objects: 49% (113/230)
  311. 2020-11-02T11:47:46.3635901Z Receiving objects: 50% (115/230)
  312. 2020-11-02T11:47:46.3636222Z Receiving objects: 51% (118/230)
  313. 2020-11-02T11:47:46.3636552Z Receiving objects: 52% (120/230)
  314. 2020-11-02T11:47:46.3636881Z Receiving objects: 53% (122/230)
  315. 2020-11-02T11:47:46.3637212Z Receiving objects: 54% (125/230)
  316. 2020-11-02T11:47:46.3637540Z Receiving objects: 55% (127/230)
  317. 2020-11-02T11:47:46.3637877Z Receiving objects: 56% (129/230)
  318. 2020-11-02T11:47:46.3638214Z Receiving objects: 57% (132/230)
  319. 2020-11-02T11:47:46.3638546Z Receiving objects: 58% (134/230)
  320. 2020-11-02T11:47:46.3639098Z Receiving objects: 59% (136/230)
  321. 2020-11-02T11:47:46.3639979Z Receiving objects: 60% (138/230)
  322. 2020-11-02T11:47:46.3640518Z Receiving objects: 61% (141/230)
  323. 2020-11-02T11:47:46.3641066Z Receiving objects: 62% (143/230)
  324. 2020-11-02T11:47:46.3641453Z Receiving objects: 63% (145/230)
  325. 2020-11-02T11:47:46.3641994Z Receiving objects: 64% (148/230)
  326. 2020-11-02T11:47:46.3642366Z Receiving objects: 65% (150/230)
  327. 2020-11-02T11:47:46.3643080Z Receiving objects: 66% (152/230)
  328. 2020-11-02T11:47:46.3643405Z Receiving objects: 67% (155/230)
  329. 2020-11-02T11:47:46.3643734Z Receiving objects: 68% (157/230)
  330. 2020-11-02T11:47:46.3644224Z Receiving objects: 69% (159/230)
  331. 2020-11-02T11:47:46.3644554Z Receiving objects: 70% (161/230)
  332. 2020-11-02T11:47:46.3644881Z Receiving objects: 71% (164/230)
  333. 2020-11-02T11:47:46.3645221Z Receiving objects: 72% (166/230)
  334. 2020-11-02T11:47:46.3645551Z Receiving objects: 73% (168/230)
  335. 2020-11-02T11:47:46.3645875Z Receiving objects: 74% (171/230)
  336. 2020-11-02T11:47:46.3646315Z Receiving objects: 75% (173/230)
  337. 2020-11-02T11:47:46.3646646Z Receiving objects: 76% (175/230)
  338. 2020-11-02T11:47:46.3646979Z Receiving objects: 77% (178/230)
  339. 2020-11-02T11:47:46.3647315Z Receiving objects: 78% (180/230)
  340. 2020-11-02T11:47:46.3647641Z Receiving objects: 79% (182/230)
  341. 2020-11-02T11:47:46.3647971Z Receiving objects: 80% (184/230)
  342. 2020-11-02T11:47:46.3648304Z Receiving objects: 81% (187/230)
  343. 2020-11-02T11:47:46.3648627Z Receiving objects: 82% (189/230)
  344. 2020-11-02T11:47:46.3648962Z Receiving objects: 83% (191/230)
  345. 2020-11-02T11:47:46.3649298Z Receiving objects: 84% (194/230)
  346. 2020-11-02T11:47:46.3649631Z Receiving objects: 85% (196/230)
  347. 2020-11-02T11:47:46.3650582Z remote: Total 230 (delta 26), reused 102 (delta 16), pack-reused 0
  348. 2020-11-02T11:47:46.3651000Z Receiving objects: 86% (198/230)
  349. 2020-11-02T11:47:46.3651449Z Receiving objects: 87% (201/230)
  350. 2020-11-02T11:47:46.3651805Z Receiving objects: 88% (203/230)
  351. 2020-11-02T11:47:46.3652128Z Receiving objects: 89% (205/230)
  352. 2020-11-02T11:47:46.3652475Z Receiving objects: 90% (207/230)
  353. 2020-11-02T11:47:46.3652810Z Receiving objects: 91% (210/230)
  354. 2020-11-02T11:47:46.3653147Z Receiving objects: 92% (212/230)
  355. 2020-11-02T11:47:46.3653485Z Receiving objects: 93% (214/230)
  356. 2020-11-02T11:47:46.3653819Z Receiving objects: 94% (217/230)
  357. 2020-11-02T11:47:46.3654155Z Receiving objects: 95% (219/230)
  358. 2020-11-02T11:47:46.3654477Z Receiving objects: 96% (221/230)
  359. 2020-11-02T11:47:46.3654818Z Receiving objects: 97% (224/230)
  360. 2020-11-02T11:47:46.3655154Z Receiving objects: 98% (226/230)
  361. 2020-11-02T11:47:46.3655489Z Receiving objects: 99% (228/230)
  362. 2020-11-02T11:47:46.3655825Z Receiving objects: 100% (230/230)
  363. 2020-11-02T11:47:46.3656202Z Receiving objects: 100% (230/230), 166.22 KiB | 13.85 MiB/s, done.
  364. 2020-11-02T11:47:46.3656583Z Resolving deltas: 0% (0/26)
  365. 2020-11-02T11:47:46.3656913Z Resolving deltas: 3% (1/26)
  366. 2020-11-02T11:47:46.3657228Z Resolving deltas: 7% (2/26)
  367. 2020-11-02T11:47:46.3657557Z Resolving deltas: 11% (3/26)
  368. 2020-11-02T11:47:46.3657892Z Resolving deltas: 15% (4/26)
  369. 2020-11-02T11:47:46.3658219Z Resolving deltas: 19% (5/26)
  370. 2020-11-02T11:47:46.3658548Z Resolving deltas: 23% (6/26)
  371. 2020-11-02T11:47:46.3658875Z Resolving deltas: 26% (7/26)
  372. 2020-11-02T11:47:46.3659199Z Resolving deltas: 30% (8/26)
  373. 2020-11-02T11:47:46.3659706Z Resolving deltas: 34% (9/26)
  374. 2020-11-02T11:47:46.3660032Z Resolving deltas: 38% (10/26)
  375. 2020-11-02T11:47:46.3660370Z Resolving deltas: 42% (11/26)
  376. 2020-11-02T11:47:46.3660706Z Resolving deltas: 46% (12/26)
  377. 2020-11-02T11:47:46.3661043Z Resolving deltas: 50% (13/26)
  378. 2020-11-02T11:47:46.3661380Z Resolving deltas: 53% (14/26)
  379. 2020-11-02T11:47:46.3661873Z Resolving deltas: 57% (15/26)
  380. 2020-11-02T11:47:46.3662200Z Resolving deltas: 61% (16/26)
  381. 2020-11-02T11:47:46.3662514Z Resolving deltas: 65% (17/26)
  382. 2020-11-02T11:47:46.3663036Z Resolving deltas: 69% (18/26)
  383. 2020-11-02T11:47:46.3663379Z Resolving deltas: 73% (19/26)
  384. 2020-11-02T11:47:46.3663714Z Resolving deltas: 76% (20/26)
  385. 2020-11-02T11:47:46.3664066Z Resolving deltas: 80% (21/26)
  386. 2020-11-02T11:47:46.3664400Z Resolving deltas: 84% (22/26)
  387. 2020-11-02T11:47:46.3664894Z Resolving deltas: 88% (23/26)
  388. 2020-11-02T11:47:46.3665218Z Resolving deltas: 92% (24/26)
  389. 2020-11-02T11:47:46.3665532Z Resolving deltas: 96% (25/26)
  390. 2020-11-02T11:47:46.3665858Z Resolving deltas: 100% (26/26)
  391. 2020-11-02T11:47:46.3666200Z Resolving deltas: 100% (26/26), done.
  392. 2020-11-02T11:47:46.3666908Z From https://github.com/submariner-io/submariner
  393. 2020-11-02T11:47:46.3667736Z * [new ref] 7bbf15868a67ab9669e8b42266b47e8c04adceff -> pull/915/merge
  394. 2020-11-02T11:47:46.3668302Z ##[endgroup]
  395. 2020-11-02T11:47:46.3668678Z ##[group]Determining the checkout info
  396. 2020-11-02T11:47:46.3669153Z ##[endgroup]
  397. 2020-11-02T11:47:46.3669460Z ##[group]Checking out the ref
  398. 2020-11-02T11:47:46.3670125Z [command]/usr/bin/git checkout --progress --force refs/remotes/pull/915/merge
  399. 2020-11-02T11:47:46.3670785Z Note: switching to 'refs/remotes/pull/915/merge'.
  400. 2020-11-02T11:47:46.3671192Z
  401. 2020-11-02T11:47:46.3671794Z You are in 'detached HEAD' state. You can look around, make experimental
  402. 2020-11-02T11:47:46.3672503Z changes and commit them, and you can discard any commits you make in this
  403. 2020-11-02T11:47:46.3673046Z state without impacting any branches by switching back to a branch.
  404. 2020-11-02T11:47:46.3673373Z
  405. 2020-11-02T11:47:46.3674907Z If you want to create a new branch to retain commits you create, you may
  406. 2020-11-02T11:47:46.3675926Z do so (now or later) by using -c with the switch command. Example:
  407. 2020-11-02T11:47:46.3676273Z
  408. 2020-11-02T11:47:46.3676796Z git switch -c <new-branch-name>
  409. 2020-11-02T11:47:46.3677082Z
  410. 2020-11-02T11:47:46.3677422Z Or undo this operation with:
  411. 2020-11-02T11:47:46.3677691Z
  412. 2020-11-02T11:47:46.3678113Z git switch -
  413. 2020-11-02T11:47:46.3678321Z
  414. 2020-11-02T11:47:46.3679013Z Turn off this advice by setting config variable advice.detachedHead to false
  415. 2020-11-02T11:47:46.3680015Z
  416. 2020-11-02T11:47:46.3680765Z HEAD is now at 7bbf158 Merge 19a9740783ec846a54ca78f77090198c2c5b95ac into ddbb7a1346be07267df18e38ca052c64ebca6337
  417. 2020-11-02T11:47:46.3681596Z ##[endgroup]
  418. 2020-11-02T11:47:46.3715907Z [command]/usr/bin/git log -1 --format='%H'
  419. 2020-11-02T11:47:46.3716622Z '7bbf15868a67ab9669e8b42266b47e8c04adceff'
  420. 2020-11-02T11:47:46.3794895Z ##[group]Run sudo swapoff -a
  421. 2020-11-02T11:47:46.3795360Z sudo swapoff -a
  422. 2020-11-02T11:47:46.3795688Z sudo rm -f /swapfile
  423. 2020-11-02T11:47:46.3795965Z df -h
  424. 2020-11-02T11:47:46.3796414Z free -h
  425. 2020-11-02T11:47:46.3849382Z shell: /bin/bash -e {0}
  426. 2020-11-02T11:47:46.3849724Z ##[endgroup]
  427. 2020-11-02T11:47:46.4661529Z Filesystem Size Used Avail Use% Mounted on
  428. 2020-11-02T11:47:46.4662224Z udev 3.4G 0 3.4G 0% /dev
  429. 2020-11-02T11:47:46.4662655Z tmpfs 696M 680K 695M 1% /run
  430. 2020-11-02T11:47:46.4663092Z /dev/sdb1 84G 63G 21G 76% /
  431. 2020-11-02T11:47:46.4664521Z tmpfs 3.4G 8.0K 3.4G 1% /dev/shm
  432. 2020-11-02T11:47:46.4665283Z tmpfs 5.0M 0 5.0M 0% /run/lock
  433. 2020-11-02T11:47:46.4665665Z tmpfs 3.4G 0 3.4G 0% /sys/fs/cgroup
  434. 2020-11-02T11:47:46.4666004Z /dev/sdb15 105M 3.6M 101M 4% /boot/efi
  435. 2020-11-02T11:47:46.4666550Z /dev/sda1 14G 4.1G 9.0G 32% /mnt
  436. 2020-11-02T11:47:46.4799036Z total used free shared buff/cache available
  437. 2020-11-02T11:47:46.4799860Z Mem: 6.8G 451M 4.9G 25M 1.4G 6.0G
  438. 2020-11-02T11:47:46.4800521Z Swap: 0B 0B 0B
  439. 2020-11-02T11:47:46.4866527Z ##[group]Run make e2e using=" operator libreswan"
  440. 2020-11-02T11:47:46.4867019Z make e2e using=" operator libreswan"
  441. 2020-11-02T11:47:46.4908020Z shell: /bin/bash -e {0}
  442. 2020-11-02T11:47:46.4908325Z ##[endgroup]
  443. 2020-11-02T11:47:46.5011983Z Downloading dapper
  444. 2020-11-02T11:47:46.6859337Z .dapper.tmp version v0.5.3
  445. 2020-11-02T11:47:46.6888978Z ./.dapper -m bind make e2e -- using=operator\ libreswan
  446. 2020-11-02T11:47:47.6867808Z Sending build context to Docker daemon 6.407MB
  447. 2020-11-02T11:47:47.6868951Z
  448. 2020-11-02T11:47:47.6944824Z Step 1/5 : FROM quay.io/submariner/shipyard-dapper-base:devel
  449. 2020-11-02T11:47:47.8916218Z devel: Pulling from submariner/shipyard-dapper-base
  450. 2020-11-02T11:47:47.9007884Z ee7e89337106: Pulling fs layer
  451. 2020-11-02T11:47:47.9015979Z 15b785f9b2f0: Pulling fs layer
  452. 2020-11-02T11:47:47.9017028Z f22377f2f0d9: Pulling fs layer
  453. 2020-11-02T11:47:47.9018675Z 14821382b808: Pulling fs layer
  454. 2020-11-02T11:47:47.9019623Z 72c0738702f0: Pulling fs layer
  455. 2020-11-02T11:47:47.9020798Z 213eace38d21: Pulling fs layer
  456. 2020-11-02T11:47:47.9023133Z 555aaae525c9: Pulling fs layer
  457. 2020-11-02T11:47:47.9081306Z 14821382b808: Waiting
  458. 2020-11-02T11:47:47.9081780Z 72c0738702f0: Waiting
  459. 2020-11-02T11:47:47.9082194Z 213eace38d21: Waiting
  460. 2020-11-02T11:47:47.9082612Z 555aaae525c9: Waiting
  461. 2020-11-02T11:47:48.8175566Z f22377f2f0d9: Verifying Checksum
  462. 2020-11-02T11:47:48.8176108Z f22377f2f0d9: Download complete
  463. 2020-11-02T11:47:48.8176936Z 14821382b808: Verifying Checksum
  464. 2020-11-02T11:47:48.8177311Z 14821382b808: Download complete
  465. 2020-11-02T11:47:48.8177739Z ee7e89337106: Verifying Checksum
  466. 2020-11-02T11:47:48.8178625Z ee7e89337106: Download complete
  467. 2020-11-02T11:47:48.8179029Z 72c0738702f0: Verifying Checksum
  468. 2020-11-02T11:47:48.8179574Z 72c0738702f0: Download complete
  469. 2020-11-02T11:47:48.8179986Z 213eace38d21: Verifying Checksum
  470. 2020-11-02T11:47:48.8180564Z 213eace38d21: Download complete
  471. 2020-11-02T11:47:48.8180973Z 555aaae525c9: Verifying Checksum
  472. 2020-11-02T11:47:48.8181523Z 555aaae525c9: Download complete
  473. 2020-11-02T11:47:49.9851387Z 15b785f9b2f0: Verifying Checksum
  474. 2020-11-02T11:47:49.9852219Z 15b785f9b2f0: Download complete
  475. 2020-11-02T11:47:51.8119696Z ee7e89337106: Pull complete
  476. 2020-11-02T11:47:59.6165012Z 15b785f9b2f0: Pull complete
  477. 2020-11-02T11:48:00.3975059Z f22377f2f0d9: Pull complete
  478. 2020-11-02T11:48:00.4540052Z 14821382b808: Pull complete
  479. 2020-11-02T11:48:00.5218392Z 72c0738702f0: Pull complete
  480. 2020-11-02T11:48:00.5776409Z 213eace38d21: Pull complete
  481. 2020-11-02T11:48:00.6481411Z 555aaae525c9: Pull complete
  482. 2020-11-02T11:48:00.6528296Z Digest: sha256:2d5f6b3b673026b5f6db375a578397f919124c2d9cd35679fc323e68091fc40e
  483. 2020-11-02T11:48:00.6563030Z Status: Downloaded newer image for quay.io/submariner/shipyard-dapper-base:devel
  484. 2020-11-02T11:48:00.6574430Z ---> 45327d47bb4a
  485. 2020-11-02T11:48:00.6575961Z Step 2/5 : ENV DAPPER_ENV="REPO TAG QUAY_USERNAME QUAY_PASSWORD GITHUB_SHA CLUSTERS_ARGS DEPLOY_ARGS IMAGES_ARGS RELEASE_ARGS" DAPPER_SOURCE=/go/src/github.com/submariner-io/submariner DAPPER_DOCKER_SOCKET=true
  486. 2020-11-02T11:48:08.3796796Z ---> Running in 24593d963516
  487. 2020-11-02T11:48:09.1505006Z Removing intermediate container 24593d963516
  488. 2020-11-02T11:48:09.1506474Z ---> a9cebcf01e35
  489. 2020-11-02T11:48:09.1506834Z Step 3/5 : ENV DAPPER_OUTPUT=${DAPPER_SOURCE}/output
  490. 2020-11-02T11:48:09.1696270Z ---> Running in e28a5cfba45a
  491. 2020-11-02T11:48:10.0535072Z Removing intermediate container e28a5cfba45a
  492. 2020-11-02T11:48:10.0536990Z ---> ff686b1d708f
  493. 2020-11-02T11:48:10.0538638Z Step 4/5 : WORKDIR ${DAPPER_SOURCE}
  494. 2020-11-02T11:48:10.0730930Z ---> Running in 802db7148ab6
  495. 2020-11-02T11:48:11.0401219Z Removing intermediate container 802db7148ab6
  496. 2020-11-02T11:48:11.0402134Z ---> 3b7e8c3dc3c5
  497. 2020-11-02T11:48:11.0402460Z Step 5/5 : ENTRYPOINT ["/opt/shipyard/scripts/entry"]
  498. 2020-11-02T11:48:11.0640207Z ---> Running in b7eb79177a2e
  499. 2020-11-02T11:48:12.0418837Z Removing intermediate container b7eb79177a2e
  500. 2020-11-02T11:48:12.0424727Z ---> 7f2abde6349f
  501. 2020-11-02T11:48:12.0476697Z Successfully built 7f2abde6349f
  502. 2020-11-02T11:48:12.0667253Z Successfully tagged submariner:HEAD
  503. 2020-11-02T11:48:12.5008934Z [submariner]$ trap chown -R 1001:116 . exit
  504. 2020-11-02T11:48:12.5087834Z [submariner]$ mkdir -p bin dist output
  505. 2020-11-02T11:48:12.5088825Z [submariner]$ make e2e -- using=operator libreswan
  506. 2020-11-02T11:48:12.5477649Z fatal: No names found, cannot describe anything.
  507. 2020-11-02T11:48:12.5625782Z /opt/shipyard/scripts/e2e.sh --focus .\* cluster2 cluster3 cluster1
  508. 2020-11-02T11:48:12.5653356Z Downloading shflags 1.0.3
  509. 2020-11-02T11:48:12.5718842Z % Total % Received % Xferd Average Speed Time Time Time Current
  510. 2020-11-02T11:48:12.5719314Z Dload Upload Total Spent Left Speed
  511. 2020-11-02T11:48:12.5719537Z
  512. 2020-11-02T11:48:12.5922067Z 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
  513. 2020-11-02T11:48:12.5923408Z 100 31091 100 31091 0 0 1445k 0 --:--:-- --:--:-- --:--:-- 1445k
  514. 2020-11-02T11:48:12.6522455Z [submariner]$ source /opt/shipyard/scripts/lib/utils
  515. 2020-11-02T11:48:12.6532762Z [submariner]$ . /opt/shipyard/scripts/lib/source_only
  516. 2020-11-02T11:48:12.6542434Z [submariner]$ script_name=utils
  517. 2020-11-02T11:48:12.6554659Z [submariner]$ exec_name=e2e.sh
  518. 2020-11-02T11:48:12.6572633Z [submariner]$ source /opt/shipyard/scripts/lib/cluster_settings
  519. 2020-11-02T11:48:12.6585605Z [submariner]$ . /opt/shipyard/scripts/lib/source_only
  520. 2020-11-02T11:48:12.6594477Z [submariner]$ script_name=cluster_settings
  521. 2020-11-02T11:48:12.6631337Z [submariner]$ exec_name=e2e.sh
  522. 2020-11-02T11:48:12.6652443Z [submariner]$ broker=cluster1
  523. 2020-11-02T11:48:12.6664654Z [submariner]$ declare -gA cluster_nodes
  524. 2020-11-02T11:48:12.6677055Z [submariner]$ cluster_nodes[cluster1]=control-plane worker
  525. 2020-11-02T11:48:12.6691643Z [submariner]$ cluster_nodes[cluster2]=control-plane worker
  526. 2020-11-02T11:48:12.6703469Z [submariner]$ cluster_nodes[cluster3]=control-plane worker worker
  527. 2020-11-02T11:48:12.6715098Z [submariner]$ declare -gA cluster_subm
  528. 2020-11-02T11:48:12.6726613Z [submariner]$ cluster_subm[cluster1]=true
  529. 2020-11-02T11:48:12.6736783Z [submariner]$ cluster_subm[cluster2]=true
  530. 2020-11-02T11:48:12.6746549Z [submariner]$ cluster_subm[cluster3]=true
  531. 2020-11-02T11:48:12.6756970Z [submariner]$ declare -gA cluster_cni
  532. 2020-11-02T11:48:12.6767466Z [submariner]$ cluster_cni[cluster2]=weave
  533. 2020-11-02T11:48:12.6818367Z [submariner]$ cluster_cni[cluster3]=weave
  534. 2020-11-02T11:48:12.6819357Z [submariner]$ declare_kubeconfig
  535. 2020-11-02T11:48:12.6824862Z [submariner]$ declare_kubeconfig
  536. 2020-11-02T11:48:12.6825586Z [submariner]$ source /opt/shipyard/scripts/lib/kubecfg
  537. 2020-11-02T11:48:12.6833364Z [submariner]$ export KUBECONFIG
  538. 2020-11-02T11:48:12.6866295Z [submariner]$ KUBECONFIG=
  539. 2020-11-02T11:48:12.6881882Z [submariner]$ find /go/src/github.com/submariner-io/submariner/output/kubeconfigs -type f -printf %p:
  540. 2020-11-02T11:48:12.6892798Z find: '/go/src/github.com/submariner-io/submariner/output/kubeconfigs': No such file or directory
  541. 2020-11-02T11:48:12.6903516Z [submariner]$ :
  542. 2020-11-02T11:48:12.6918880Z [submariner]$ deploy_env_once
  543. 2020-11-02T11:48:12.6929282Z [submariner]$ deploy_env_once
  544. 2020-11-02T11:48:12.6942589Z 
  545. 2020-11-02T11:48:12.9400661Z [submariner]$ make deploy
  546. 2020-11-02T11:48:12.9428014Z make[1]: Entering directory '/go/src/github.com/submariner-io/submariner'
  547. 2020-11-02T11:48:12.9860766Z fatal: No names found, cannot describe anything.
  548. 2020-11-02T11:48:12.9996926Z /opt/shipyard/scripts/clusters.sh --cluster_settings /go/src/github.com/submariner-io/submariner/scripts/cluster_settings
  549. 2020-11-02T11:48:13.0721922Z Running with: k8s_version=1.17.0, olm_version=0.14.1, olm=false, globalnet=false, registry_inmemory=true, cluster_settings=/go/src/github.com/submariner-io/submariner/scripts/cluster_settings, timeout=5m
  550. 2020-11-02T11:48:13.0740634Z [submariner]$ source /opt/shipyard/scripts/lib/utils
  551. 2020-11-02T11:48:13.0751180Z [submariner]$ . /opt/shipyard/scripts/lib/source_only
  552. 2020-11-02T11:48:13.0761832Z [submariner]$ script_name=utils
  553. 2020-11-02T11:48:13.0773239Z [submariner]$ exec_name=clusters.sh
  554. 2020-11-02T11:48:13.0789368Z [submariner]$ source /opt/shipyard/scripts/lib/cluster_settings
  555. 2020-11-02T11:48:13.0799067Z [submariner]$ . /opt/shipyard/scripts/lib/source_only
  556. 2020-11-02T11:48:13.0807033Z [submariner]$ script_name=cluster_settings
  557. 2020-11-02T11:48:13.0815882Z [submariner]$ exec_name=clusters.sh
  558. 2020-11-02T11:48:13.0831579Z [submariner]$ broker=cluster1
  559. 2020-11-02T11:48:13.0839791Z [submariner]$ declare -gA cluster_nodes
  560. 2020-11-02T11:48:13.0851544Z [submariner]$ cluster_nodes[cluster1]=control-plane worker
  561. 2020-11-02T11:48:13.0861620Z [submariner]$ cluster_nodes[cluster2]=control-plane worker
  562. 2020-11-02T11:48:13.0878482Z [submariner]$ cluster_nodes[cluster3]=control-plane worker worker
  563. 2020-11-02T11:48:13.0888206Z [submariner]$ declare -gA cluster_subm
  564. 2020-11-02T11:48:13.0896397Z [submariner]$ cluster_subm[cluster1]=true
  565. 2020-11-02T11:48:13.0905496Z [submariner]$ cluster_subm[cluster2]=true
  566. 2020-11-02T11:48:13.0915220Z [submariner]$ cluster_subm[cluster3]=true
  567. 2020-11-02T11:48:13.0925006Z [submariner]$ declare -gA cluster_cni
  568. 2020-11-02T11:48:13.0936754Z [submariner]$ cluster_cni[cluster2]=weave
  569. 2020-11-02T11:48:13.0943938Z [submariner]$ cluster_cni[cluster3]=weave
  570. 2020-11-02T11:48:13.0954066Z [submariner]$ source /go/src/github.com/submariner-io/submariner/scripts/cluster_settings
  571. 2020-11-02T11:48:13.0965790Z [submariner]$ cluster_nodes[cluster1]=control-plane worker
  572. 2020-11-02T11:48:13.0975793Z [submariner]$ cluster_nodes[cluster2]=control-plane worker worker
  573. 2020-11-02T11:48:13.0985332Z [submariner]$ cluster_nodes[cluster3]=control-plane worker worker
  574. 2020-11-02T11:48:13.1033389Z [submariner]$ cat
  575. 2020-11-02T11:48:13.1051391Z [submariner]$ typeset -p cluster_nodes
  576. 2020-11-02T11:48:13.1063355Z [submariner]$ cut -f 2- -d=
  577. 2020-11-02T11:48:13.1090087Z [submariner]$ typeset -p cluster_subm
  578. 2020-11-02T11:48:13.1100595Z [submariner]$ cut -f 2- -d=
  579. 2020-11-02T11:48:13.1121106Z Cluster settings::
  580. 2020-11-02T11:48:13.1121620Z broker - 'cluster1'
  581. 2020-11-02T11:48:13.1122062Z clusters - 'cluster1' 'cluster2' 'cluster3'
  582. 2020-11-02T11:48:13.1123025Z nodes per cluster - ([cluster3]="control-plane worker worker" [cluster2]="control-plane worker worker" [cluster1]="control-plane worker" )
  583. 2020-11-02T11:48:13.1125247Z install submariner - ([cluster3]="true" [cluster2]="true" [cluster1]="true" )
  584. 2020-11-02T11:48:13.1138022Z [submariner]$ rm -rf /go/src/github.com/submariner-io/submariner/output/kubeconfigs
  585. 2020-11-02T11:48:13.1155008Z [submariner]$ mkdir -p /go/src/github.com/submariner-io/submariner/output/kubeconfigs
  586. 2020-11-02T11:48:13.1177197Z [submariner]$ run_local_registry
  587. 2020-11-02T11:48:13.1188236Z [submariner]$ run_local_registry
  588. 2020-11-02T11:48:13.1196622Z [submariner]$ registry_running
  589. 2020-11-02T11:48:13.1206283Z [submariner]$ registry_running
  590. 2020-11-02T11:48:13.1215933Z [submariner]$ docker ps --filter name=^/?kind-registry$
  591. 2020-11-02T11:48:13.1225946Z [submariner]$ grep kind-registry
  592. 2020-11-02T11:48:13.4224333Z [submariner]$ return 0
  593. 2020-11-02T11:48:13.4225132Z Deploying local registry kind-registry to serve images centrally.
  594. 2020-11-02T11:48:13.4236304Z [submariner]$ local volume_flag
  595. 2020-11-02T11:48:13.4256134Z [submariner]$ volume_flag=-v /dev/shm/kind-registry:/var/lib/registry
  596. 2020-11-02T11:48:13.4266497Z [submariner]$ docker run -d -v /dev/shm/kind-registry:/var/lib/registry -p 5000:5000 --restart=always --name kind-registry registry:2
  597. 2020-11-02T11:48:13.7248523Z Unable to find image 'registry:2' locally
  598. 2020-11-02T11:48:13.9252032Z 2: Pulling from library/registry
  599. 2020-11-02T11:48:14.0798894Z cbdbe7a5bc2a: Already exists
  600. 2020-11-02T11:48:14.0830316Z 47112e65547d: Pulling fs layer
  601. 2020-11-02T11:48:14.0830656Z 46bcb632e506: Pulling fs layer
  602. 2020-11-02T11:48:14.0831084Z c1cc712bcecd: Pulling fs layer
  603. 2020-11-02T11:48:14.0831455Z 3db6272dcbfa: Pulling fs layer
  604. 2020-11-02T11:48:14.0831769Z 3db6272dcbfa: Waiting
  605. 2020-11-02T11:48:14.1289755Z 47112e65547d: Download complete
  606. 2020-11-02T11:48:14.1766173Z c1cc712bcecd: Verifying Checksum
  607. 2020-11-02T11:48:14.1766825Z c1cc712bcecd: Download complete
  608. 2020-11-02T11:48:14.1968578Z 46bcb632e506: Download complete
  609. 2020-11-02T11:48:14.2144940Z 3db6272dcbfa: Verifying Checksum
  610. 2020-11-02T11:48:14.2145754Z 3db6272dcbfa: Download complete
  611. 2020-11-02T11:48:14.2914472Z 47112e65547d: Pull complete
  612. 2020-11-02T11:48:14.5561480Z 46bcb632e506: Pull complete
  613. 2020-11-02T11:48:14.6282232Z c1cc712bcecd: Pull complete
  614. 2020-11-02T11:48:14.6902225Z 3db6272dcbfa: Pull complete
  615. 2020-11-02T11:48:14.6930137Z Digest: sha256:8be26f81ffea54106bae012c6f349df70f4d5e7e2ec01b143c46e2c03b9e551d
  616. 2020-11-02T11:48:14.6950876Z Status: Downloaded newer image for registry:2
  617. 2020-11-02T11:48:15.0132685Z 011a33acee6fa3df1d879599134c8313e1499751b1372570a2e1dc143e3d9da4
  618. 2020-11-02T11:48:15.8318865Z [submariner]$ registry_ip=172.17.0.3
  619. 2020-11-02T11:48:15.8332926Z [submariner]$ docker inspect -f {{.NetworkSettings.IPAddress}} kind-registry
  620. 2020-11-02T11:48:16.1136778Z [submariner]$ declare_cidrs
  621. 2020-11-02T11:48:16.1140971Z [submariner]$ declare_cidrs
  622. 2020-11-02T11:48:16.1146934Z [submariner]$ declare -gA cluster_CIDRs service_CIDRs global_CIDRs
  623. 2020-11-02T11:48:16.1158061Z [submariner]$ i=1
  624. 2020-11-02T11:48:16.1170433Z [submariner]$ [cluster1] add_cluster_cidrs 1 cluster1
  625. 2020-11-02T11:48:16.1182806Z [submariner]$ [cluster1] add_cluster_cidrs 1 cluster1
  626. 2020-11-02T11:48:16.1194415Z [submariner]$ [cluster1] local val=1
  627. 2020-11-02T11:48:16.1204817Z [submariner]$ [cluster1] local idx=cluster1
  628. 2020-11-02T11:48:16.1219280Z [submariner]$ [cluster1] cluster_CIDRs[cluster1]=10.1.0.0/16
  629. 2020-11-02T11:48:16.1233120Z [submariner]$ [cluster1] service_CIDRs[cluster1]=100.1.0.0/16
  630. 2020-11-02T11:48:16.1243388Z [submariner]$ [cluster1] i=2
  631. 2020-11-02T11:48:16.1259741Z [submariner]$ [cluster2] add_cluster_cidrs 2 cluster2
  632. 2020-11-02T11:48:16.1276272Z [submariner]$ [cluster2] add_cluster_cidrs 2 cluster2
  633. 2020-11-02T11:48:16.1287047Z [submariner]$ [cluster2] local val=2
  634. 2020-11-02T11:48:16.1298624Z [submariner]$ [cluster2] local idx=cluster2
  635. 2020-11-02T11:48:16.1312483Z [submariner]$ [cluster2] cluster_CIDRs[cluster2]=10.2.0.0/16
  636. 2020-11-02T11:48:16.1321711Z [submariner]$ [cluster2] service_CIDRs[cluster2]=100.2.0.0/16
  637. 2020-11-02T11:48:16.1336339Z [submariner]$ [cluster2] i=3
  638. 2020-11-02T11:48:16.1349843Z [submariner]$ [cluster3] add_cluster_cidrs 3 cluster3
  639. 2020-11-02T11:48:16.1361433Z [submariner]$ [cluster3] add_cluster_cidrs 3 cluster3
  640. 2020-11-02T11:48:16.1372799Z [submariner]$ [cluster3] local val=3
  641. 2020-11-02T11:48:16.1380351Z [submariner]$ [cluster3] local idx=cluster3
  642. 2020-11-02T11:48:16.1391477Z [submariner]$ [cluster3] cluster_CIDRs[cluster3]=10.3.0.0/16
  643. 2020-11-02T11:48:16.1400124Z [submariner]$ [cluster3] service_CIDRs[cluster3]=100.3.0.0/16
  644. 2020-11-02T11:48:16.1412252Z [submariner]$ [cluster3] i=4
  645. 2020-11-02T11:48:16.1423975Z [submariner]$ [cluster3] run_all_clusters with_retries 3 create_kind_cluster
  646. 2020-11-02T11:48:16.1435772Z [submariner]$ [cluster3] run_all_clusters with_retries 3 create_kind_cluster
  647. 2020-11-02T11:48:16.1446734Z [submariner]$ [cluster3] run_parallel cluster1 cluster2 cluster3 with_retries 3 create_kind_cluster
  648. 2020-11-02T11:48:16.1461870Z [submariner]$ [cluster3] run_parallel cluster1 cluster2 cluster3 cluster1 cluster2 cluster3 with_retries 3 create_kind_cluster
  649. 2020-11-02T11:48:16.1471281Z [submariner]$ [cluster3] local cmnd=with_retries
  650. 2020-11-02T11:48:16.1485669Z [submariner]$ [cluster3] declare -A pids
  651. 2020-11-02T11:48:16.1507613Z [submariner]$ [cluster3] eval echo cluster1 cluster2 cluster3
  652. 2020-11-02T11:48:16.1534426Z [submariner]$ [cluster1] pids[cluster1]=344
  653. 2020-11-02T11:48:16.1551890Z [submariner]$ [cluster1] set -o pipefail
  654. 2020-11-02T11:48:16.1553000Z [submariner]$ [cluster2] pids[cluster2]=347
  655. 2020-11-02T11:48:16.1573258Z [submariner]$ [cluster3] pids[cluster3]=349
  656. 2020-11-02T11:48:16.1580129Z [submariner]$ [cluster2] set -o pipefail
  657. 2020-11-02T11:48:16.1589980Z [submariner]$ [cluster3] set -o pipefail
  658. 2020-11-02T11:48:16.1601736Z [submariner]$ [cluster2] with_retries 3 create_kind_cluster
  659. 2020-11-02T11:48:16.1602382Z [submariner]$ [cluster1] with_retries 3 create_kind_cluster
  660. 2020-11-02T11:48:16.1615506Z [submariner]$ [cluster3] with_retries 3 create_kind_cluster
  661. 2020-11-02T11:48:16.1625456Z [submariner]$ [cluster3] wait 349
  662. 2020-11-02T11:48:16.1628805Z [submariner]$ [cluster2] sed /\[cluster2]/!s/^/[cluster2] /
  663. 2020-11-02T11:48:16.1639421Z [submariner]$ [cluster1] sed /\[cluster1]/!s/^/[cluster1] /
  664. 2020-11-02T11:48:16.1647236Z [submariner]$ [cluster3] sed /\[cluster3]/!s/^/[cluster3] /
  665. 2020-11-02T11:52:32.0786406Z [submariner]$ [cluster1] with_retries
  666. 2020-11-02T11:52:32.0787022Z [submariner]$ [cluster1] local retries=3
  667. 2020-11-02T11:52:32.0787607Z [submariner]$ [cluster1] local cmnd=create_kind_cluster
  668. 2020-11-02T11:52:32.0788186Z [submariner]$ [cluster1] iteration=1
  669. 2020-11-02T11:52:32.0788694Z [submariner]$ [cluster1] wait 380
  670. 2020-11-02T11:52:32.0789241Z [submariner]$ [cluster1] create_kind_cluster
  671. 2020-11-02T11:52:32.0789808Z [submariner]$ [cluster1] create_kind_cluster
  672. 2020-11-02T11:52:32.0790761Z [submariner]$ [cluster1] export KUBECONFIG=/go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster1
  673. 2020-11-02T11:52:32.0791981Z [submariner]$ [cluster1] rm -f /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster1
  674. 2020-11-02T11:52:32.0792824Z [submariner]$ [cluster1] kind get clusters
  675. 2020-11-02T11:52:32.0793367Z [submariner]$ [cluster1] grep -q ^cluster1$
  676. 2020-11-02T11:52:32.0793749Z [cluster1] No kind clusters found.
  677. 2020-11-02T11:52:32.0794094Z [cluster1] Creating KIND cluster...
  678. 2020-11-02T11:52:32.0794639Z [submariner]$ [cluster1] generate_cluster_yaml
  679. 2020-11-02T11:52:32.0795214Z [submariner]$ [cluster1] generate_cluster_yaml
  680. 2020-11-02T11:52:32.0795776Z [submariner]$ [cluster1] local pod_cidr=10.1.0.0/16
  681. 2020-11-02T11:52:32.0796342Z [submariner]$ [cluster1] local service_cidr=100.1.0.0/16
  682. 2020-11-02T11:52:32.0796968Z [submariner]$ [cluster1] local dns_domain=cluster1.local
  683. 2020-11-02T11:52:32.0797589Z [submariner]$ [cluster1] local disable_cni=false
  684. 2020-11-02T11:52:32.0798279Z [submariner]$ [cluster1] local nodes
  685. 2020-11-02T11:52:32.0798731Z [submariner]$ [cluster1] nodes=
  686. 2020-11-02T11:52:32.0799205Z [cluster1] - role: control-plane
  687. 2020-11-02T11:52:32.0799684Z [submariner]$ [cluster1] nodes=
  688. 2020-11-02T11:52:32.0800159Z [cluster1] - role: control-plane
  689. 2020-11-02T11:52:32.0800592Z [cluster1] - role: worker
  690. 2020-11-02T11:52:32.0800959Z [cluster1] 
  691. 2020-11-02T11:52:32.0801303Z [cluster1] 
  692. 2020-11-02T11:52:32.0801723Z [submariner]$ [cluster1] eval echo "---
  693. 2020-11-02T11:52:32.0802025Z [cluster1] kind: Cluster
  694. 2020-11-02T11:52:32.0802523Z [cluster1] apiVersion: kind.x-k8s.io/v1alpha4
  695. 2020-11-02T11:52:32.0803061Z [cluster1] networking:
  696. 2020-11-02T11:52:32.0805310Z [cluster1] disableDefaultCNI: ${disable_cni}
  697. 2020-11-02T11:52:32.0805791Z [cluster1] podSubnet: ${pod_cidr}
  698. 2020-11-02T11:52:32.0806198Z [cluster1] serviceSubnet: ${service_cidr}
  699. 2020-11-02T11:52:32.0806706Z [cluster1] containerdConfigPatches:
  700. 2020-11-02T11:52:32.0807655Z [cluster1] - |-
  701. 2020-11-02T11:52:32.0808349Z [cluster1] [plugins.\"io.containerd.grpc.v1.cri\".registry.mirrors.\"localhost:5000\"]
  702. 2020-11-02T11:52:32.0809021Z [cluster1] endpoint = [\"http://${registry_ip}:5000\"]
  703. 2020-11-02T11:52:32.0809471Z [cluster1] kubeadmConfigPatches:
  704. 2020-11-02T11:52:32.0809960Z [cluster1] - |
  705. 2020-11-02T11:52:32.0810320Z [cluster1] apiVersion: kubeadm.k8s.io/v1beta2
  706. 2020-11-02T11:52:32.0810792Z [cluster1] kind: ClusterConfiguration
  707. 2020-11-02T11:52:32.0811173Z [cluster1] metadata:
  708. 2020-11-02T11:52:32.0811447Z [cluster1] name: config
  709. 2020-11-02T11:52:32.0811748Z [cluster1] networking:
  710. 2020-11-02T11:52:32.0812077Z [cluster1] podSubnet: ${pod_cidr}
  711. 2020-11-02T11:52:32.0812466Z [cluster1] serviceSubnet: ${service_cidr}
  712. 2020-11-02T11:52:32.0812853Z [cluster1] dnsDomain: ${dns_domain}
  713. 2020-11-02T11:52:32.0813317Z [cluster1] nodes: ${nodes}"
  714. 2020-11-02T11:52:32.0814288Z [submariner]$ [cluster1] cat /opt/shipyard/scripts/resources/kind-cluster-config.yaml
  715. 2020-11-02T11:52:32.0815062Z [submariner]$ [cluster1] local image_flag=
  716. 2020-11-02T11:52:32.0815709Z [submariner]$ [cluster1] image_flag=--image=kindest/node:v1.17.0
  717. 2020-11-02T11:52:32.1508109Z [submariner]$ [cluster1] kind create cluster --image=kindest/node:v1.17.0 --name=cluster1 --config=/opt/shipyard/scripts/resources/cluster1-config.yaml
  718. 2020-11-02T11:52:32.1508856Z [cluster1] Creating cluster "cluster1" ...
  719. 2020-11-02T11:52:32.1509488Z [cluster1] • Ensuring node image (kindest/node:v1.17.0) 🖼 ...
  720. 2020-11-02T11:52:32.1510107Z [cluster1] ✓ Ensuring node image (kindest/node:v1.17.0) 🖼
  721. 2020-11-02T11:52:32.1510808Z [cluster1] • Preparing nodes 📦 📦 ...
  722. 2020-11-02T11:52:32.1511324Z [cluster1] ✓ Preparing nodes 📦 📦
  723. 2020-11-02T11:52:32.1511823Z [cluster1] • Writing configuration 📜 ...
  724. 2020-11-02T11:52:32.1512333Z [cluster1] ✓ Writing configuration 📜
  725. 2020-11-02T11:52:32.1512837Z [cluster1] • Starting control-plane 🕹️ ...
  726. 2020-11-02T11:52:32.1513351Z [cluster1] ✓ Starting control-plane 🕹️
  727. 2020-11-02T11:52:32.1514195Z [cluster1] • Installing CNI 🔌 ...
  728. 2020-11-02T11:52:32.1514816Z [cluster1] ✓ Installing CNI 🔌
  729. 2020-11-02T11:52:32.1515364Z [cluster1] • Installing StorageClass 💾 ...
  730. 2020-11-02T11:52:32.1515905Z [cluster1] ✓ Installing StorageClass 💾
  731. 2020-11-02T11:52:32.1516418Z [cluster1] • Joining worker nodes 🚜 ...
  732. 2020-11-02T11:52:32.1518187Z [cluster1] ✓ Joining worker nodes 🚜
  733. 2020-11-02T11:52:32.1520335Z [cluster1] Set kubectl context to "kind-cluster1"
  734. 2020-11-02T11:52:32.1521154Z [cluster1] You can now use your cluster with:
  735. 2020-11-02T11:52:32.1521682Z [cluster1]
  736. 2020-11-02T11:52:32.1522374Z [cluster1] kubectl cluster-info --context kind-cluster1
  737. 2020-11-02T11:52:32.1523128Z [cluster1]
  738. 2020-11-02T11:52:32.1524726Z [cluster1] Have a nice day! 👋
  739. 2020-11-02T11:52:32.1525488Z [submariner]$ [cluster1] kind_fixup_config
  740. 2020-11-02T11:52:32.1526090Z [submariner]$ [cluster1] kind_fixup_config
  741. 2020-11-02T11:52:32.1526721Z [submariner]$ [cluster1] local master_ip=172.17.0.9
  742. 2020-11-02T11:52:32.1527823Z [submariner]$ [cluster1] docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}} cluster1-control-plane
  743. 2020-11-02T11:52:32.1529238Z [submariner]$ [cluster1] head -n 1
  744. 2020-11-02T11:52:32.9316607Z [submariner]$ [cluster1] sed -i -- s/server: .*/server: https:\/\/172.17.0.9:6443/g /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster1
  745. 2020-11-02T11:52:32.9318592Z [submariner]$ [cluster1] sed -i -- s/user: kind-.*/user: cluster1/g /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster1
  746. 2020-11-02T11:52:32.9320637Z [submariner]$ [cluster1] sed -i -- s/name: kind-.*/name: cluster1/g /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster1
  747. 2020-11-02T11:52:32.9322587Z [submariner]$ [cluster1] sed -i -- s/cluster: kind-.*/cluster: cluster1/g /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster1
  748. 2020-11-02T11:52:32.9324730Z [submariner]$ [cluster1] sed -i -- s/current-context: .*/current-context: cluster1/g /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster1
  749. 2020-11-02T11:52:32.9326201Z [submariner]$ [cluster1] chmod a+r /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster1
  750. 2020-11-02T11:52:32.9327313Z [submariner]$ [cluster1] wait 1814
  751. 2020-11-02T11:52:32.9327887Z [submariner]$ [cluster1] deploy_cluster_capabilities
  752. 2020-11-02T11:52:32.9328533Z [submariner]$ [cluster1] deploy_cluster_capabilities
  753. 2020-11-02T11:52:32.9329103Z [submariner]$ [cluster1] deploy_cni
  754. 2020-11-02T11:52:32.9329606Z [submariner]$ [cluster1] deploy_cni
  755. 2020-11-02T11:52:32.9330086Z [submariner]$ [cluster1] return 0
  756. 2020-11-02T11:52:32.9330726Z [submariner]$ [cluster1] return 0
  757. 2020-11-02T11:52:50.1135329Z [submariner]$ [cluster2] with_retries
  758. 2020-11-02T11:52:50.1136041Z [submariner]$ [cluster2] local retries=3
  759. 2020-11-02T11:52:50.1137063Z [submariner]$ [cluster2] local cmnd=create_kind_cluster
  760. 2020-11-02T11:52:50.1137957Z [submariner]$ [cluster2] iteration=1
  761. 2020-11-02T11:52:50.1138496Z [submariner]$ [cluster2] wait 386
  762. 2020-11-02T11:52:50.1139052Z [submariner]$ [cluster2] create_kind_cluster
  763. 2020-11-02T11:52:50.1139648Z [submariner]$ [cluster2] create_kind_cluster
  764. 2020-11-02T11:52:50.1140670Z [submariner]$ [cluster2] export KUBECONFIG=/go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster2
  765. 2020-11-02T11:52:50.1142240Z [submariner]$ [cluster2] rm -f /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster2
  766. 2020-11-02T11:52:50.1143169Z [submariner]$ [cluster2] kind get clusters
  767. 2020-11-02T11:52:50.1143882Z [submariner]$ [cluster2] grep -q ^cluster2$
  768. 2020-11-02T11:52:50.1144298Z [cluster2] No kind clusters found.
  769. 2020-11-02T11:52:50.1144689Z [cluster2] Creating KIND cluster...
  770. 2020-11-02T11:52:50.1145405Z [submariner]$ [cluster2] generate_cluster_yaml
  771. 2020-11-02T11:52:50.1146151Z [submariner]$ [cluster2] generate_cluster_yaml
  772. 2020-11-02T11:52:50.1146778Z [submariner]$ [cluster2] local pod_cidr=10.2.0.0/16
  773. 2020-11-02T11:52:50.1147376Z [submariner]$ [cluster2] local service_cidr=100.2.0.0/16
  774. 2020-11-02T11:52:50.1148132Z [submariner]$ [cluster2] local dns_domain=cluster2.local
  775. 2020-11-02T11:52:50.1148797Z [submariner]$ [cluster2] local disable_cni=false
  776. 2020-11-02T11:52:50.1149388Z [submariner]$ [cluster2] disable_cni=true
  777. 2020-11-02T11:52:50.1149947Z [submariner]$ [cluster2] local nodes
  778. 2020-11-02T11:52:50.1150465Z [submariner]$ [cluster2] nodes=
  779. 2020-11-02T11:52:50.1150972Z [cluster2] - role: control-plane
  780. 2020-11-02T11:52:50.1151595Z [submariner]$ [cluster2] nodes=
  781. 2020-11-02T11:52:50.1152114Z [cluster2] - role: control-plane
  782. 2020-11-02T11:52:50.1152593Z [cluster2] - role: worker
  783. 2020-11-02T11:52:50.1153073Z [submariner]$ [cluster2] nodes=
  784. 2020-11-02T11:52:50.1153588Z [cluster2] - role: control-plane
  785. 2020-11-02T11:52:50.1154051Z [cluster2] - role: worker
  786. 2020-11-02T11:52:50.1154470Z [cluster2] - role: worker
  787. 2020-11-02T11:52:50.1154877Z [cluster2] 
  788. 2020-11-02T11:52:50.1155256Z [cluster2] 
  789. 2020-11-02T11:52:50.1155789Z [submariner]$ [cluster2] eval echo "---
  790. 2020-11-02T11:52:50.1156142Z [cluster2] kind: Cluster
  791. 2020-11-02T11:52:50.1156682Z [cluster2] apiVersion: kind.x-k8s.io/v1alpha4
  792. 2020-11-02T11:52:50.1157097Z [cluster2] networking:
  793. 2020-11-02T11:52:50.1157657Z [cluster2] disableDefaultCNI: ${disable_cni}
  794. 2020-11-02T11:52:50.1158116Z [cluster2] podSubnet: ${pod_cidr}
  795. 2020-11-02T11:52:50.1158528Z [cluster2] serviceSubnet: ${service_cidr}
  796. 2020-11-02T11:52:50.1159042Z [cluster2] containerdConfigPatches:
  797. 2020-11-02T11:52:50.1159608Z [cluster2] - |-
  798. 2020-11-02T11:52:50.1160451Z [cluster2] [plugins.\"io.containerd.grpc.v1.cri\".registry.mirrors.\"localhost:5000\"]
  799. 2020-11-02T11:52:50.1161147Z [cluster2] endpoint = [\"http://${registry_ip}:5000\"]
  800. 2020-11-02T11:52:50.1161735Z [cluster2] kubeadmConfigPatches:
  801. 2020-11-02T11:52:50.1162239Z [cluster2] - |
  802. 2020-11-02T11:52:50.1162614Z [cluster2] apiVersion: kubeadm.k8s.io/v1beta2
  803. 2020-11-02T11:52:50.1163106Z [cluster2] kind: ClusterConfiguration
  804. 2020-11-02T11:52:50.1163502Z [cluster2] metadata:
  805. 2020-11-02T11:52:50.1163803Z [cluster2] name: config
  806. 2020-11-02T11:52:50.1164470Z [cluster2] networking:
  807. 2020-11-02T11:52:50.1164815Z [cluster2] podSubnet: ${pod_cidr}
  808. 2020-11-02T11:52:50.1165223Z [cluster2] serviceSubnet: ${service_cidr}
  809. 2020-11-02T11:52:50.1165611Z [cluster2] dnsDomain: ${dns_domain}
  810. 2020-11-02T11:52:50.1166136Z [cluster2] nodes: ${nodes}"
  811. 2020-11-02T11:52:50.1166896Z [submariner]$ [cluster2] cat /opt/shipyard/scripts/resources/kind-cluster-config.yaml
  812. 2020-11-02T11:52:50.1167912Z [submariner]$ [cluster2] local image_flag=
  813. 2020-11-02T11:52:50.1168716Z [submariner]$ [cluster2] image_flag=--image=kindest/node:v1.17.0
  814. 2020-11-02T11:52:50.1194717Z [submariner]$ [cluster2] kind create cluster --image=kindest/node:v1.17.0 --name=cluster2 --config=/opt/shipyard/scripts/resources/cluster2-config.yaml
  815. 2020-11-02T11:52:50.1195509Z [cluster2] Creating cluster "cluster2" ...
  816. 2020-11-02T11:52:50.1196168Z [cluster2] • Ensuring node image (kindest/node:v1.17.0) 🖼 ...
  817. 2020-11-02T11:52:50.1196806Z [cluster2] ✓ Ensuring node image (kindest/node:v1.17.0) 🖼
  818. 2020-11-02T11:52:50.1197352Z [cluster2] • Preparing nodes 📦 📦 📦 ...
  819. 2020-11-02T11:52:50.1197879Z [cluster2] ✓ Preparing nodes 📦 📦 📦
  820. 2020-11-02T11:52:50.1198414Z [cluster2] • Writing configuration 📜 ...
  821. 2020-11-02T11:52:50.1198959Z [cluster2] ✓ Writing configuration 📜
  822. 2020-11-02T11:52:50.1199510Z [cluster2] • Starting control-plane 🕹️ ...
  823. 2020-11-02T11:52:50.1200307Z [cluster2] ✓ Starting control-plane 🕹️
  824. 2020-11-02T11:52:50.1200850Z [cluster2] • Installing StorageClass 💾 ...
  825. 2020-11-02T11:52:50.1201384Z [cluster2] ✓ Installing StorageClass 💾
  826. 2020-11-02T11:52:50.1201937Z [cluster2] • Joining worker nodes 🚜 ...
  827. 2020-11-02T11:52:50.1202411Z [cluster2] ✓ Joining worker nodes 🚜
  828. 2020-11-02T11:52:50.1202947Z [cluster2] Set kubectl context to "kind-cluster2"
  829. 2020-11-02T11:52:50.1203378Z [cluster2] You can now use your cluster with:
  830. 2020-11-02T11:52:50.1203691Z [cluster2]
  831. 2020-11-02T11:52:50.1204393Z [cluster2] kubectl cluster-info --context kind-cluster2
  832. 2020-11-02T11:52:50.1204778Z [cluster2]
  833. 2020-11-02T11:52:50.1205551Z [cluster2] Have a question, bug, or feature request? Let us know! https://kind.sigs.k8s.io/#community 🙂
  834. 2020-11-02T11:52:50.1206378Z [submariner]$ [cluster2] kind_fixup_config
  835. 2020-11-02T11:52:50.1206923Z [submariner]$ [cluster2] kind_fixup_config
  836. 2020-11-02T11:52:50.1207474Z [submariner]$ [cluster2] local master_ip=172.17.0.8
  837. 2020-11-02T11:52:50.1208433Z [submariner]$ [cluster2] docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}} cluster2-control-plane
  838. 2020-11-02T11:52:55.0674617Z [submariner]$ [cluster2[submariner]$ [cluster3] with_retries
  839. 2020-11-02T11:52:55.0675403Z [submariner]$ [cluster3] local retries=3
  840. 2020-11-02T11:52:55.0676083Z [submariner]$ [cluster3] local cmnd=create_kind_cluster
  841. 2020-11-02T11:52:55.0676739Z [submariner]$ [cluster3] iteration=1
  842. 2020-11-02T11:52:55.0677303Z [submariner]$ [cluster3] wait 382
  843. 2020-11-02T11:52:55.0678412Z [submariner]$ [cluster3] create_kind_cluster
  844. 2020-11-02T11:52:55.0679443Z [submariner]$ [cluster3] create_kind_cluster
  845. 2020-11-02T11:52:55.0680515Z [submariner]$ [cluster3] export KUBECONFIG=/go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster3
  846. 2020-11-02T11:52:55.0681786Z [submariner]$ [cluster3] rm -f /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster3
  847. 2020-11-02T11:52:55.0682654Z [submariner]$ [cluster3] kind get clusters
  848. 2020-11-02T11:52:55.0683220Z [submariner]$ [cluster3] grep -q ^cluster3$
  849. 2020-11-02T11:52:55.0683634Z [cluster3] No kind clusters found.
  850. 2020-11-02T11:52:55.0684738Z [cluster3] Creating KIND cluster...
  851. 2020-11-02T11:52:55.0685362Z [submariner]$ [cluster3] generate_cluster_yaml
  852. 2020-11-02T11:52:55.0686195Z [submariner]$ [cluster3] generate_cluster_yaml
  853. 2020-11-02T11:52:55.0686786Z [submariner]$ [cluster3] local pod_cidr=10.3.0.0/16
  854. 2020-11-02T11:52:55.0687367Z [submariner]$ [cluster3] local service_cidr=100.3.0.0/16
  855. 2020-11-02T11:52:55.0688225Z [submariner]$ [cluster3] local dns_domain=cluster3.local
  856. 2020-11-02T11:52:55.0688874Z [submariner]$ [cluster3] local disable_cni=false
  857. 2020-11-02T11:52:55.0689452Z [submariner]$ [cluster3] disable_cni=true
  858. 2020-11-02T11:52:55.0689992Z [submariner]$ [cluster3] local nodes
  859. 2020-11-02T11:52:55.0690499Z [submariner]$ [cluster3] nodes=
  860. 2020-11-02T11:52:55.0690988Z [cluster3] - role: control-plane
  861. 2020-11-02T11:52:55.0691836Z [submariner]$ [cluster3] nodes=
  862. 2020-11-02T11:52:55.0692365Z [cluster3] - role: control-plane
  863. 2020-11-02T11:52:55.0692829Z [cluster3] - role: worker
  864. 2020-11-02T11:52:55.0693289Z [submariner]$ [cluster3] nodes=
  865. 2020-11-02T11:52:55.0694144Z [cluster3] - role: control-plane
  866. 2020-11-02T11:52:55.0694643Z [cluster3] - role: worker
  867. 2020-11-02T11:52:55.0695074Z [cluster3] - role: worker
  868. 2020-11-02T11:52:55.0695486Z [cluster3] 
  869. 2020-11-02T11:52:55.0695865Z [cluster3] 
  870. 2020-11-02T11:52:55.0696675Z [submariner]$ [cluster3] eval echo "---
  871. 2020-11-02T11:52:55.0697183Z [cluster3] kind: Cluster
  872. 2020-11-02T11:52:55.0697892Z [cluster3] apiVersion: kind.x-k8s.io/v1alpha4
  873. 2020-11-02T11:52:55.0698309Z [cluster3] networking:
  874. 2020-11-02T11:52:55.0698727Z [cluster3] disableDefaultCNI: ${disable_cni}
  875. 2020-11-02T11:52:55.0699305Z [cluster3] podSubnet: ${pod_cidr}
  876. 2020-11-02T11:52:55.0699724Z [cluster3] serviceSubnet: ${service_cidr}
  877. 2020-11-02T11:52:55.0700237Z [cluster3] containerdConfigPatches:
  878. 2020-11-02T11:52:55.0700954Z [cluster3] - |-
  879. 2020-11-02T11:52:55.0701748Z [cluster3] [plugins.\"io.containerd.grpc.v1.cri\".registry.mirrors.\"localhost:5000\"]
  880. 2020-11-02T11:52:55.0702440Z [cluster3] endpoint = [\"http://${registry_ip}:5000\"]
  881. 2020-11-02T11:52:55.0703063Z [cluster3] kubeadmConfigPatches:
  882. 2020-11-02T11:52:55.0703589Z [cluster3] - |
  883. 2020-11-02T11:52:55.0704150Z [cluster3] apiVersion: kubeadm.k8s.io/v1beta2
  884. 2020-11-02T11:52:55.0704677Z [cluster3] kind: ClusterConfiguration
  885. 2020-11-02T11:52:55.0705119Z [cluster3] metadata:
  886. 2020-11-02T11:52:55.0705444Z [cluster3] name: config
  887. 2020-11-02T11:52:55.0705779Z [cluster3] networking:
  888. 2020-11-02T11:52:55.0706298Z [cluster3] podSubnet: ${pod_cidr}
  889. 2020-11-02T11:52:55.0707023Z [cluster3] serviceSubnet: ${service_cidr}
  890. 2020-11-02T11:52:55.0707413Z [cluster3] dnsDomain: ${dns_domain}
  891. 2020-11-02T11:52:55.0708089Z [cluster3] nodes: ${nodes}"
  892. 2020-11-02T11:52:55.0709421Z [submariner]$ [cluster3] cat /opt/shipyard/scripts/resources/kind-cluster-config.yaml
  893. 2020-11-02T11:52:55.0710447Z [submariner]$ [cluster3] local image_flag=
  894. 2020-11-02T11:52:55.0711139Z [submariner]$ [cluster3] image_flag=--image=kindest/node:v1.17.0
  895. 2020-11-02T11:52:55.0712290Z [submariner]$ [cluster3] kind create cluster --image=kindest/node:v1.17.0 --name=cluster3 --config=/opt/shipyard/scripts/resources/cluster3-config.yaml
  896. 2020-11-02T11:52:55.0713514Z [cluster3] Creating cluster "cluster3" ...
  897. 2020-11-02T11:52:55.0715911Z [cluster3] • Ensuring node image (kindest/node:v1.17.0) 🖼 ...
  898. 2020-11-02T11:52:55.0717851Z [cluster3] ✓ Ensuring node image (kindest/node:v1.17.0) 🖼
  899. 2020-11-02T11:52:55.0718789Z [cluster3] • Preparing nodes 📦 📦 📦 ...
  900. 2020-11-02T11:52:55.0719280Z [cluster3] ✓ Preparing nodes 📦 📦 📦
  901. 2020-11-02T11:52:55.0719800Z [cluster3] • Writing configuration 📜 ...
  902. 2020-11-02T11:52:55.0720347Z [cluster3] ✓ Writing configuration 📜
  903. 2020-11-02T11:52:55.0721085Z [cluster3] • Starting control-plane 🕹️ ...
  904. 2020-11-02T11:52:55.0721703Z [cluster3] ✓ Starting control-plane 🕹️
  905. 2020-11-02T11:52:55.0722270Z [cluster3] • Installing StorageClass 💾 ...
  906. 2020-11-02T11:52:55.0722820Z [cluster3] ✓ Installing StorageClass 💾
  907. 2020-11-02T11:52:55.0723354Z [cluster3] • Joining worker nodes 🚜 ...
  908. 2020-11-02T11:52:55.0724158Z [cluster3] ✓ Joining worker nodes 🚜
  909. 2020-11-02T11:52:55.0724762Z [cluster3] Set kubectl context to "kind-cluster3"
  910. 2020-11-02T11:52:55.0725218Z [cluster3] You can now use your cluster with:
  911. 2020-11-02T11:52:55.0725576Z [cluster3]
  912. 2020-11-02T11:52:55.0726300Z [cluster3] kubectl cluster-info --context kind-cluster3
  913. 2020-11-02T11:52:55.0726699Z [cluster3]
  914. 2020-11-02T11:52:55.0727458Z [cluster3] Have a question, bug, or feature request? Let us know! https://kind.sigs.k8s.io/#community 🙂
  915. 2020-11-02T11:52:55.0728269Z [submariner]$ [cluster3] kind_fixup_config
  916. 2020-11-02T11:52:55.0728830Z [submariner]$ [cluster3] kind_fixup_config
  917. 2020-11-02T11:52:55.0729607Z [submariner]$ [cluster3] local master_ip=172.17.0.10
  918. 2020-11-02T11:52:55.0730603Z [submariner]$ [cluster3] docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}} cluster3-control-plane
  919. 2020-11-02T11:53:43.5552493Z [submariner]$ [cluster] head -n 1
  920. 2020-11-02T11:53:43.5553859Z [submariner]$ [cluster2] sed -i -- s/server: .*/server: https:\/\/172.17.0.8:6443/g /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster2
  921. 2020-11-02T11:53:43.5555507Z [submariner]$ [cluster2] sed -i -- s/user: kind-.*/user: cluster2/g /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster2
  922. 2020-11-02T11:53:43.5556925Z [submariner]$ [cluster2] sed -i -- s/name: kind-.*/name: cluster2/g /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster2
  923. 2020-11-02T11:53:43.5558760Z [submariner]$ [cluster2] sed -i -- s/cluster: kind-.*/cluster: cluster2/g /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster2
  924. 2020-11-02T11:53:43.5563063Z [submariner]$ [cluster2] sed -i -- s/current-context: .*/current-context: cluster2/g /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster2
  925. 2020-11-02T11:53:43.5566101Z [submariner]$ [cluster2] chmod a+r /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster2
  926. 2020-11-02T11:53:43.5567092Z [submariner]$ [cluster2] wait 1974
  927. 2020-11-02T11:53:43.5568705Z [submariner]$ [cluster2] deploy_cluster_capabilities
  928. 2020-11-02T11:53:43.5569921Z [submariner]$ [cluster2] deploy_cluster_capabilities
  929. 2020-11-02T11:53:43.5570588Z [submariner]$ [cluster2] deploy_cni
  930. 2020-11-02T11:53:43.5571111Z [submariner]$ [cluster2] deploy_cni
  931. 2020-11-02T11:53:43.5571658Z [submariner]$ [cluster2] eval deploy_weave_cni
  932. 2020-11-02T11:53:43.5572220Z [submariner]$ [cluster2] deploy_weave_cni
  933. 2020-11-02T11:53:43.5572784Z [submariner]$ [cluster2] deploy_weave_cni
  934. 2020-11-02T11:53:43.5573174Z [cluster2] Applying weave network...
  935. 2020-11-02T11:53:43.5574130Z [submariner]$ [cluster2] kubectl apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.17.0&env.IPALLOC_RANGE=10.2.0.0/16
  936. 2020-11-02T11:53:43.5575543Z [submariner]$ [cluster2] kubectl apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.17.0&env.IPALLOC_RANGE=10.2.0.0/16
  937. 2020-11-02T11:53:43.5577189Z [submariner]$ [cluster2] command kubectl --context=cluster2 apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.17.0&env.IPALLOC_RANGE=10.2.0.0/16
  938. 2020-11-02T11:53:43.5578243Z [cluster2] serviceaccount/weave-net created
  939. 2020-11-02T11:53:43.5579144Z [cluster2] clusterrole.rbac.authorization.k8s.io/weave-net created
  940. 2020-11-02T11:53:43.5580439Z [cluster2] clusterrolebinding.rbac.authorization.k8s.io/weave-net created
  941. 2020-11-02T11:53:43.5581612Z [cluster2] role.rbac.authorization.k8s.io/weave-net created
  942. 2020-11-02T11:53:43.5582712Z [cluster2] rolebinding.rbac.authorization.k8s.io/weave-net created
  943. 2020-11-02T11:53:43.5583603Z [cluster2] daemonset.apps/weave-net created
  944. 2020-11-02T11:53:43.5584761Z [cluster2] Waiting for weave-net pods to be ready...
  945. 2020-11-02T11:53:43.5586341Z [submariner]$ [cluster2] kubectl wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m
  946. 2020-11-02T11:53:43.5587314Z [submariner]$ [cluster2] kubectl wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m
  947. 2020-11-02T11:53:43.5588368Z [submariner]$ [cluster2] command kubectl --context=cluster2 wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m
  948. 2020-11-02T11:53:43.5589457Z [submariner]$ [cluster2] kubectl --context=cluster2 wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m
  949. 2020-11-02T11:53:43.5590989Z [cluster2] pod/weave-net-gqks5 condition met
  950. 2020-11-02T11:53:43.5591598Z [cluster2] pod/weave-net-pntqf condition met
  951. 2020-11-02T11:53:43.5592608Z [cluster2] pod/weave-net-prp6w condition met
  952. 2020-11-02T11:53:43.5593260Z [cluster2] Waiting for core-dns deployment to be ready...
  953. 2020-11-02T11:53:43.5594014Z [submariner]$ [cluster2] kubectl -n kube-system rollout status deploy/coredns --timeout=5m
  954. 2020-11-02T11:53:43.5595053Z [submariner]$ [cluster2] kubectl -n kube-system rollout status deploy/coredns --timeout=5m
  955. 2020-11-02T11:53:43.5596439Z [submariner]$ [cluster2] command kubectl --context=cluster2 -n kube-system rollout status deploy/coredns --timeout=5m
  956. 2020-11-02T11:53:43.5597731Z [submariner]$ [cluster2] kubectl --context=cluster2 -n kube-system rollout status deploy/coredns --timeout=5m
  957. 2020-11-02T11:53:43.5599021Z [cluster2] Waiting for deployment "coredns" rollout to finish: 1 of 2 updated replicas are available...
  958. 2020-11-02T11:53:43.5599680Z [cluster2] deployment "coredns" successfully rolled out
  959. 2020-11-02T11:53:43.5600470Z [submariner]$ [cluster2] return 0
  960. 2020-11-02T11:53:44.7264762Z 3] head -n 1
  961. 2020-11-02T11:53:44.7265850Z [submariner]$ [cluster3] sed -i -- s/server: .*/server: https:\/\/172.17.0.10:6443/g /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster3
  962. 2020-11-02T11:53:44.7267167Z [submariner]$ [cluster3] sed -i -- s/user: kind-.*/user: cluster3/g /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster3
  963. 2020-11-02T11:53:44.7268495Z [submariner]$ [cluster3] sed -i -- s/name: kind-.*/name: cluster3/g /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster3
  964. 2020-11-02T11:53:44.7269820Z [submariner]$ [cluster3] sed -i -- s/cluster: kind-.*/cluster: cluster3/g /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster3
  965. 2020-11-02T11:53:44.7271244Z [submariner]$ [cluster3] sed -i -- s/current-context: .*/current-context: cluster3/g /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster3
  966. 2020-11-02T11:53:44.7272563Z [submariner]$ [cluster3] chmod a+r /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster3
  967. 2020-11-02T11:53:44.7273348Z [submariner]$ [cluster3] wait 2071
  968. 2020-11-02T11:53:44.7274057Z [submariner]$ [cluster3] deploy_cluster_capabilities
  969. 2020-11-02T11:53:44.7274660Z [submariner]$ [cluster3] deploy_cluster_capabilities
  970. 2020-11-02T11:53:44.7275192Z [submariner]$ [cluster3] deploy_cni
  971. 2020-11-02T11:53:44.7275659Z [submariner]$ [cluster3] deploy_cni
  972. 2020-11-02T11:53:44.7276542Z [submariner]$ [cluster3] eval deploy_weave_cni
  973. 2020-11-02T11:53:44.7277368Z [submariner]$ [cluster3] deploy_weave_cni
  974. 2020-11-02T11:53:44.7278822Z [submariner]$ [cluster3] deploy_weave_cni
  975. 2020-11-02T11:53:44.7279289Z [cluster3] Applying weave network...
  976. 2020-11-02T11:53:44.7280360Z [submariner]$ [cluster3] kubectl apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.17.0&env.IPALLOC_RANGE=10.3.0.0/16
  977. 2020-11-02T11:53:44.7281758Z [submariner]$ [cluster3] kubectl apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.17.0&env.IPALLOC_RANGE=10.3.0.0/16
  978. 2020-11-02T11:53:44.7284086Z [submariner]$ [cluster3] command kubectl --context=cluster3 apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.17.0&env.IPALLOC_RANGE=10.3.0.0/16
  979. 2020-11-02T11:53:44.7285037Z [cluster3] serviceaccount/weave-net created
  980. 2020-11-02T11:53:44.7285878Z [cluster3] clusterrole.rbac.authorization.k8s.io/weave-net created
  981. 2020-11-02T11:53:44.7287328Z [cluster3] clusterrolebinding.rbac.authorization.k8s.io/weave-net created
  982. 2020-11-02T11:53:44.7288450Z [cluster3] role.rbac.authorization.k8s.io/weave-net created
  983. 2020-11-02T11:53:44.7289619Z [cluster3] rolebinding.rbac.authorization.k8s.io/weave-net created
  984. 2020-11-02T11:53:44.7290436Z [cluster3] daemonset.apps/weave-net created
  985. 2020-11-02T11:53:44.7291224Z [cluster3] Waiting for weave-net pods to be ready...
  986. 2020-11-02T11:53:44.7292006Z [submariner]$ [cluster3] kubectl wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m
  987. 2020-11-02T11:53:44.7293169Z [submariner]$ [cluster3] kubectl wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m
  988. 2020-11-02T11:53:44.7294365Z [submariner]$ [cluster3] command kubectl --context=cluster3 wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m
  989. 2020-11-02T11:53:44.7295420Z [submariner]$ [cluster3] kubectl --context=cluster3 wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m
  990. 2020-11-02T11:53:44.7296184Z [cluster3] pod/weave-net-65xmt condition met
  991. 2020-11-02T11:53:44.7296975Z [cluster3] pod/weave-net-br9qq condition met
  992. 2020-11-02T11:53:44.7297559Z [cluster3] pod/weave-net-xbnsr condition met
  993. 2020-11-02T11:53:44.7298151Z [cluster3] Waiting for core-dns deployment to be ready...
  994. 2020-11-02T11:53:44.7298898Z [submariner]$ [cluster3] kubectl -n kube-system rollout status deploy/coredns --timeout=5m
  995. 2020-11-02T11:53:44.7299909Z [submariner]$ [cluster3] kubectl -n kube-system rollout status deploy/coredns --timeout=5m
  996. 2020-11-02T11:53:44.7301166Z [submariner]$ [cluster3] command kubectl --context=cluster3 -n kube-system rollout status deploy/coredns --timeout=5m
  997. 2020-11-02T11:53:44.7302184Z [submariner]$ [cluster3] kubectl --context=cluster3 -n kube-system rollout status deploy/coredns --timeout=5m
  998. 2020-11-02T11:53:44.7302937Z [cluster3] Waiting for deployment "coredns" rollout to finish: 0 of 2 updated replicas are available...
  999. 2020-11-02T11:53:44.7303666Z [cluster3] Waiting for deployment "coredns" rollout to finish: 1 of 2 updated replicas are available...
  1000. 2020-11-02T11:53:44.7304280Z [cluster3] deployment "coredns" successfully rolled out
  1001. 2020-11-02T11:53:44.7304852Z [submariner]$ [cluster3] return 0
  1002. 2020-11-02T11:53:44.7328954Z [submariner]$ [cluster3] wait 347
  1003. 2020-11-02T11:53:44.7339941Z [submariner]$ [cluster3] wait 344
  1004. 2020-11-02T11:53:44.7381977Z [submariner]$ [cluster3] print_clusters_message
  1005. 2020-11-02T11:53:44.7405051Z [submariner]$ [cluster3] print_clusters_message
  1006. 2020-11-02T11:53:44.7444738Z [submariner]$ [cluster3] cat
  1007. 2020-11-02T11:53:44.7619821Z Your virtual cluster(s) are deployed and working properly and can be accessed with:
  1008. 2020-11-02T11:53:44.7620306Z
  1009. 2020-11-02T11:53:44.7621275Z export KUBECONFIG=$(find $(git rev-parse --show-toplevel)/output/kubeconfigs/ -type f -printf %p:)
  1010. 2020-11-02T11:53:44.7621702Z
  1011. 2020-11-02T11:53:44.7622481Z $ kubectl config use-context cluster1 # or cluster2, cluster3..
  1012. 2020-11-02T11:53:44.7622826Z
  1013. 2020-11-02T11:53:44.7623137Z To clean everthing up, just run: make cleanup
  1014. 2020-11-02T11:53:44.7661071Z go mod download
  1015. 2020-11-02T11:54:01.0500980Z go mod vendor
  1016. 2020-11-02T11:54:04.7832224Z fatal: No names found, cannot describe anything.
  1017. 2020-11-02T11:54:04.7840637Z /opt/shipyard/scripts/compile.sh bin/submariner-engine main.go --ldflags '-X main.VERSION='
  1018. 2020-11-02T11:54:04.8871102Z [submariner]$ mkdir -p bin
  1019. 2020-11-02T11:54:04.8954631Z Building 'bin/submariner-engine' (ldflags: '-X main.VERSION=')
  1020. 2020-11-02T11:54:04.8970585Z [submariner]$ ldflags=-s -w -X main.VERSION=
  1021. 2020-11-02T11:54:04.8986403Z [submariner]$ CGO_ENABLED=0 go build -ldflags -s -w -X main.VERSION= -o bin/submariner-engine main.go
  1022. 2020-11-02T11:55:12.0974274Z [submariner]$ upx bin/submariner-engine
  1023. 2020-11-02T11:55:18.6213191Z Ultimate Packer for eXecutables
  1024. 2020-11-02T11:55:18.6214930Z Copyright (C) 1996 - 2020
  1025. 2020-11-02T11:55:18.6215815Z UPX 3.96 Markus Oberhumer, Laszlo Molnar & John Reiser Jan 23rd 2020
  1026. 2020-11-02T11:55:18.6216309Z
  1027. 2020-11-02T11:55:18.6216776Z File size Ratio Format Name
  1028. 2020-11-02T11:55:18.6218330Z -------------------- ------ ----------- -----------
  1029. 2020-11-02T11:55:18.6219236Z 29446144 -> 8706736 29.57% linux/amd64 submariner-engine
  1030. 2020-11-02T11:55:18.6223705Z
  1031. 2020-11-02T11:55:18.6223944Z Packed 1 file.
  1032. 2020-11-02T11:55:18.6743881Z fatal: No names found, cannot describe anything.
  1033. 2020-11-02T11:55:18.6753811Z /opt/shipyard/scripts/compile.sh bin/submariner-route-agent ./pkg/routeagent/main.go --ldflags '-X main.VERSION='
  1034. 2020-11-02T11:55:18.7375764Z [submariner]$ mkdir -p bin
  1035. 2020-11-02T11:55:18.7392247Z Building 'bin/submariner-route-agent' (ldflags: '-X main.VERSION=')
  1036. 2020-11-02T11:55:18.7403414Z [submariner]$ ldflags=-s -w -X main.VERSION=
  1037. 2020-11-02T11:55:18.7417305Z [submariner]$ CGO_ENABLED=0 go build -ldflags -s -w -X main.VERSION= -o bin/submariner-route-agent ./pkg/routeagent/main.go
  1038. 2020-11-02T11:55:41.8421186Z [submariner]$ upx bin/submariner-route-agent
  1039. 2020-11-02T11:55:48.2517772Z Ultimate Packer for eXecutables
  1040. 2020-11-02T11:55:48.2519031Z Copyright (C) 1996 - 2020
  1041. 2020-11-02T11:55:48.2519601Z UPX 3.96 Markus Oberhumer, Laszlo Molnar & John Reiser Jan 23rd 2020
  1042. 2020-11-02T11:55:48.2520027Z
  1043. 2020-11-02T11:55:48.2520346Z File size Ratio Format Name
  1044. 2020-11-02T11:55:48.2520835Z -------------------- ------ ----------- -----------
  1045. 2020-11-02T11:55:48.2521381Z 29470720 -> 8506456 28.86% linux/amd64 submariner-route-agent
  1046. 2020-11-02T11:55:48.2521828Z
  1047. 2020-11-02T11:55:48.2522026Z Packed 1 file.
  1048. 2020-11-02T11:55:48.3057413Z fatal: No names found, cannot describe anything.
  1049. 2020-11-02T11:55:48.3070851Z /opt/shipyard/scripts/compile.sh bin/submariner-globalnet ./pkg/globalnet/main.go --ldflags '-X main.VERSION='
  1050. 2020-11-02T11:55:48.3598907Z [submariner]$ mkdir -p bin
  1051. 2020-11-02T11:55:48.3619097Z Building 'bin/submariner-globalnet' (ldflags: '-X main.VERSION=')
  1052. 2020-11-02T11:55:48.3629930Z [submariner]$ ldflags=-s -w -X main.VERSION=
  1053. 2020-11-02T11:55:48.3645633Z [submariner]$ CGO_ENABLED=0 go build -ldflags -s -w -X main.VERSION= -o bin/submariner-globalnet ./pkg/globalnet/main.go
  1054. 2020-11-02T11:55:51.9341411Z [submariner]$ upx bin/submariner-globalnet
  1055. 2020-11-02T11:55:58.0663372Z Ultimate Packer for eXecutables
  1056. 2020-11-02T11:55:58.0664408Z Copyright (C) 1996 - 2020
  1057. 2020-11-02T11:55:58.0664955Z UPX 3.96 Markus Oberhumer, Laszlo Molnar & John Reiser Jan 23rd 2020
  1058. 2020-11-02T11:55:58.0670641Z
  1059. 2020-11-02T11:55:58.0670983Z File size Ratio Format Name
  1060. 2020-11-02T11:55:58.0671572Z -------------------- ------ ----------- -----------
  1061. 2020-11-02T11:55:58.0672415Z 29315072 -> 8447232 28.82% linux/amd64 submariner-globalnet
  1062. 2020-11-02T11:55:58.0672886Z
  1063. 2020-11-02T11:55:58.0673105Z Packed 1 file.
  1064. 2020-11-02T11:55:58.1274141Z fatal: No names found, cannot describe anything.
  1065. 2020-11-02T11:55:58.1284754Z /opt/shipyard/scripts/compile.sh bin/submariner-networkplugin-syncer ./pkg/networkplugin-syncer/main.go --ldflags '-X main.VERSION='
  1066. 2020-11-02T11:55:58.2076455Z [submariner]$ mkdir -p bin
  1067. 2020-11-02T11:55:58.2094898Z Building 'bin/submariner-networkplugin-syncer' (ldflags: '-X main.VERSION=')
  1068. 2020-11-02T11:55:58.2107633Z [submariner]$ ldflags=-s -w -X main.VERSION=
  1069. 2020-11-02T11:55:58.2119358Z [submariner]$ CGO_ENABLED=0 go build -ldflags -s -w -X main.VERSION= -o bin/submariner-networkplugin-syncer ./pkg/networkplugin-syncer/main.go
  1070. 2020-11-02T11:55:58.7673969Z [submariner]$ upx bin/submariner-networkplugin-syncer
  1071. 2020-11-02T11:55:59.1654734Z Ultimate Packer for eXecutables
  1072. 2020-11-02T11:55:59.1657408Z Copyright (C) 1996 - 2020
  1073. 2020-11-02T11:55:59.1658150Z UPX 3.96 Markus Oberhumer, Laszlo Molnar & John Reiser Jan 23rd 2020
  1074. 2020-11-02T11:55:59.1658618Z
  1075. 2020-11-02T11:55:59.1659273Z File size Ratio Format Name
  1076. 2020-11-02T11:55:59.1659953Z -------------------- ------ ----------- -----------
  1077. 2020-11-02T11:55:59.1660868Z 835584 -> 354504 42.43% linux/amd64 submariner-networkplugin-syncer
  1078. 2020-11-02T11:55:59.1662570Z
  1079. 2020-11-02T11:55:59.1662916Z Packed 1 file.
  1080. 2020-11-02T11:55:59.5379875Z /opt/shipyard/scripts/build_image.sh -i submariner -f package/Dockerfile.submariner
  1081. 2020-11-02T11:55:59.5894775Z fatal: No names found, cannot describe anything.
  1082. 2020-11-02T11:55:59.6887290Z [submariner]$ set -e
  1083. 2020-11-02T11:55:59.6898460Z [submariner]$ local_image=quay.io/submariner/submariner:dev
  1084. 2020-11-02T11:55:59.6910499Z [submariner]$ cache_image=quay.io/submariner/submariner:devel
  1085. 2020-11-02T11:55:59.6925086Z [submariner]$ cache_flag=
  1086. 2020-11-02T11:55:59.6937532Z [submariner]$ cache_flag=--cache-from quay.io/submariner/submariner:devel
  1087. 2020-11-02T11:55:59.6952650Z [submariner]$ docker image ls -q quay.io/submariner/submariner:devel
  1088. 2020-11-02T11:56:00.0289311Z [submariner]$ docker pull quay.io/submariner/submariner:devel
  1089. 2020-11-02T11:56:00.4729185Z devel: Pulling from submariner/submariner
  1090. 2020-11-02T11:56:00.4758097Z ee7e89337106: Already exists
  1091. 2020-11-02T11:56:00.4805721Z 9da6274d90a1: Pulling fs layer
  1092. 2020-11-02T11:56:00.4806246Z fe0b30e0483d: Pulling fs layer
  1093. 2020-11-02T11:56:00.4806574Z 111d7c31d937: Pulling fs layer
  1094. 2020-11-02T11:56:00.4806887Z d9ad2fa71aba: Pulling fs layer
  1095. 2020-11-02T11:56:00.4809272Z d9ad2fa71aba: Waiting
  1096. 2020-11-02T11:56:00.5769597Z 9da6274d90a1: Download complete
  1097. 2020-11-02T11:56:00.6458458Z 111d7c31d937: Verifying Checksum
  1098. 2020-11-02T11:56:00.6461207Z 111d7c31d937: Download complete
  1099. 2020-11-02T11:56:00.6585381Z d9ad2fa71aba: Verifying Checksum
  1100. 2020-11-02T11:56:00.6586969Z d9ad2fa71aba: Download complete
  1101. 2020-11-02T11:56:00.7361945Z fe0b30e0483d: Verifying Checksum
  1102. 2020-11-02T11:56:00.7362435Z fe0b30e0483d: Download complete
  1103. 2020-11-02T11:56:01.3849404Z 9da6274d90a1: Pull complete
  1104. 2020-11-02T11:56:02.5042759Z fe0b30e0483d: Pull complete
  1105. 2020-11-02T11:56:02.6770857Z 111d7c31d937: Pull complete
  1106. 2020-11-02T11:56:02.7624660Z d9ad2fa71aba: Pull complete
  1107. 2020-11-02T11:56:02.7649248Z Digest: sha256:151322de6a829aa0dcbe998485ecf691027089bda754d304009e7a00c6104e85
  1108. 2020-11-02T11:56:02.7678300Z Status: Downloaded newer image for quay.io/submariner/submariner:devel
  1109. 2020-11-02T11:56:02.7703760Z quay.io/submariner/submariner:devel
  1110. 2020-11-02T11:56:02.7762552Z [submariner]$ grep FROM package/Dockerfile.submariner
  1111. 2020-11-02T11:56:02.7768692Z [submariner]$ cut -f2 -d 
  1112. 2020-11-02T11:56:02.7769194Z [submariner]$ grep -v scratch
  1113. 2020-11-02T11:56:02.7782963Z [submariner]$ cache_flag+= --cache-from fedora:33
  1114. 2020-11-02T11:56:02.7786746Z [submariner]$ docker pull fedora:33
  1115. 2020-11-02T11:56:03.4253656Z 33: Pulling from library/fedora
  1116. 2020-11-02T11:56:03.5013548Z ee7e89337106: Already exists
  1117. 2020-11-02T11:56:03.5514534Z Digest: sha256:b9ec86d36fca7b1d3de39cd7c258e8d90c377d312c21a7748071ce49069b8db4
  1118. 2020-11-02T11:56:03.5538668Z Status: Downloaded newer image for fedora:33
  1119. 2020-11-02T11:56:03.5539238Z docker.io/library/fedora:33
  1120. 2020-11-02T11:56:03.5572612Z [submariner]$ buildargs_flag=--build-arg BUILDKIT_INLINE_CACHE=1
  1121. 2020-11-02T11:56:03.5587442Z [submariner]$ DOCKER_BUILDKIT=1 docker build -t quay.io/submariner/submariner:dev --cache-from quay.io/submariner/submariner:devel --cache-from fedora:33 -f package/Dockerfile.submariner --build-arg BUILDKIT_INLINE_CACHE=1 .
  1122. 2020-11-02T11:56:03.8920857Z #1 [internal] load .dockerignore
  1123. 2020-11-02T11:56:03.8922524Z #1 DONE 0.0s
  1124. 2020-11-02T11:56:04.0216230Z
  1125. 2020-11-02T11:56:04.0219671Z #1 [internal] load .dockerignore
  1126. 2020-11-02T11:56:04.0220104Z #1 transferring context: 2B 0.0s done
  1127. 2020-11-02T11:56:04.0220420Z #1 DONE 0.0s
  1128. 2020-11-02T11:56:04.0220576Z
  1129. 2020-11-02T11:56:04.0221012Z #2 [internal] load build definition from Dockerfile.submariner
  1130. 2020-11-02T11:56:04.0221561Z #2 transferring dockerfile: 581B done
  1131. 2020-11-02T11:56:04.0221881Z #2 DONE 0.0s
  1132. 2020-11-02T11:56:04.0222035Z
  1133. 2020-11-02T11:56:04.0222394Z #3 [internal] load metadata for docker.io/library/fedora:33
  1134. 2020-11-02T11:56:04.0222768Z #3 DONE 0.0s
  1135. 2020-11-02T11:56:04.0223150Z
  1136. 2020-11-02T11:56:04.0224097Z #9 importing cache manifest from fedora:33
  1137. 2020-11-02T11:56:04.0224455Z #9 DONE 0.0s
  1138. 2020-11-02T11:56:04.0224612Z
  1139. 2020-11-02T11:56:04.0225190Z #11 importing cache manifest from quay.io/submariner/submariner:devel
  1140. 2020-11-02T11:56:04.0225619Z #11 DONE 0.0s
  1141. 2020-11-02T11:56:04.0225789Z
  1142. 2020-11-02T11:56:04.0226329Z #10 [1/5] FROM docker.io/library/fedora:33
  1143. 2020-11-02T11:56:04.0226765Z #10 DONE 0.0s
  1144. 2020-11-02T11:56:04.0226930Z
  1145. 2020-11-02T11:56:04.0227192Z #6 [internal] load build context
  1146. 2020-11-02T11:56:04.1709572Z #6 transferring context: 8.71MB 0.1s done
  1147. 2020-11-02T11:56:04.1709993Z #6 DONE 0.1s
  1148. 2020-11-02T11:56:04.1710169Z
  1149. 2020-11-02T11:56:04.1710446Z #4 [2/5] WORKDIR /var/submariner
  1150. 2020-11-02T11:56:04.1710739Z #4 CACHED
  1151. 2020-11-02T11:56:04.1710897Z
  1152. 2020-11-02T11:56:04.1714255Z #5 [3/5] RUN dnf -y install --nodocs --setopt=install_weak_deps=0 ...
  1153. 2020-11-02T11:56:04.1714649Z #5 CACHED
  1154. 2020-11-02T11:56:04.1714799Z
  1155. 2020-11-02T11:56:04.1715458Z #7 [4/5] COPY package/submariner.sh package/pluto bin/submariner-engine /us...
  1156. 2020-11-02T11:56:06.0759662Z #7 DONE 2.0s
  1157. 2020-11-02T11:56:06.0760072Z
  1158. 2020-11-02T11:56:06.0761084Z #8 [5/5] COPY package/iptables-wrapper.in /usr/sbin/
  1159. 2020-11-02T11:56:06.0761499Z #8 DONE 0.0s
  1160. 2020-11-02T11:56:07.0828531Z
  1161. 2020-11-02T11:56:07.0829046Z #12 exporting to image
  1162. 2020-11-02T11:56:07.0829369Z #12 exporting layers done
  1163. 2020-11-02T11:56:07.0830182Z #12 writing image sha256:d8c626a8c5bb551f473b19df31e167d22079bd31b35ad54aafa8041219de1359
  1164. 2020-11-02T11:56:07.0902377Z #12 writing image sha256:d8c626a8c5bb551f473b19df31e167d22079bd31b35ad54aafa8041219de1359 done
  1165. 2020-11-02T11:56:07.0903340Z #12 naming to quay.io/submariner/submariner:dev done
  1166. 2020-11-02T11:56:07.0903692Z #12 DONE 0.0s
  1167. 2020-11-02T11:56:07.0903869Z
  1168. 2020-11-02T11:56:07.0904113Z #13 exporting cache
  1169. 2020-11-02T11:56:07.0904463Z #13 preparing build cache for export done
  1170. 2020-11-02T11:56:07.0904762Z #13 DONE 0.0s
  1171. 2020-11-02T11:56:07.0968270Z [submariner]$ docker tag quay.io/submariner/submariner:dev quay.io/submariner/submariner:devel
  1172. 2020-11-02T11:56:07.4082461Z touch package/.image.submariner
  1173. 2020-11-02T11:56:07.7863299Z /opt/shipyard/scripts/build_image.sh -i submariner-route-agent -f package/Dockerfile.submariner-route-agent
  1174. 2020-11-02T11:56:07.8334407Z fatal: No names found, cannot describe anything.
  1175. 2020-11-02T11:56:07.9222964Z [submariner]$ set -e
  1176. 2020-11-02T11:56:07.9234934Z [submariner]$ local_image=quay.io/submariner/submariner-route-agent:dev
  1177. 2020-11-02T11:56:07.9245469Z [submariner]$ cache_image=quay.io/submariner/submariner-route-agent:devel
  1178. 2020-11-02T11:56:07.9258248Z [submariner]$ cache_flag=
  1179. 2020-11-02T11:56:07.9275002Z [submariner]$ cache_flag=--cache-from quay.io/submariner/submariner-route-agent:devel
  1180. 2020-11-02T11:56:07.9292937Z [submariner]$ docker image ls -q quay.io/submariner/submariner-route-agent:devel
  1181. 2020-11-02T11:56:08.2568101Z [submariner]$ docker pull quay.io/submariner/submariner-route-agent:devel
  1182. 2020-11-02T11:56:08.7541705Z devel: Pulling from submariner/submariner-route-agent
  1183. 2020-11-02T11:56:08.7543105Z 0fd3b5213a9b: Pulling fs layer
  1184. 2020-11-02T11:56:08.7543816Z aebb8c556853: Pulling fs layer
  1185. 2020-11-02T11:56:08.7544260Z f5092dbc8061: Pulling fs layer
  1186. 2020-11-02T11:56:08.7544732Z 45712d871266: Pulling fs layer
  1187. 2020-11-02T11:56:08.7545064Z 3cafd23f580a: Pulling fs layer
  1188. 2020-11-02T11:56:08.7545403Z 13b7038a41b9: Pulling fs layer
  1189. 2020-11-02T11:56:08.7545689Z 45712d871266: Waiting
  1190. 2020-11-02T11:56:08.7546006Z 3cafd23f580a: Waiting
  1191. 2020-11-02T11:56:08.7546283Z 13b7038a41b9: Waiting
  1192. 2020-11-02T11:56:08.8058547Z f5092dbc8061: Verifying Checksum
  1193. 2020-11-02T11:56:08.8059026Z f5092dbc8061: Download complete
  1194. 2020-11-02T11:56:08.8084593Z aebb8c556853: Verifying Checksum
  1195. 2020-11-02T11:56:08.8084998Z aebb8c556853: Download complete
  1196. 2020-11-02T11:56:08.9058053Z 45712d871266: Verifying Checksum
  1197. 2020-11-02T11:56:08.9058472Z 45712d871266: Download complete
  1198. 2020-11-02T11:56:09.0108115Z 3cafd23f580a: Verifying Checksum
  1199. 2020-11-02T11:56:09.0109066Z 3cafd23f580a: Download complete
  1200. 2020-11-02T11:56:09.0213770Z 13b7038a41b9: Verifying Checksum
  1201. 2020-11-02T11:56:09.0214782Z 13b7038a41b9: Download complete
  1202. 2020-11-02T11:56:09.2716472Z 0fd3b5213a9b: Verifying Checksum
  1203. 2020-11-02T11:56:09.2718111Z 0fd3b5213a9b: Download complete
  1204. 2020-11-02T11:56:11.8690813Z 0fd3b5213a9b: Pull complete
  1205. 2020-11-02T11:56:11.9499138Z aebb8c556853: Pull complete
  1206. 2020-11-02T11:56:12.0377162Z f5092dbc8061: Pull complete
  1207. 2020-11-02T11:56:12.3369936Z 45712d871266: Pull complete
  1208. 2020-11-02T11:56:12.5093396Z 3cafd23f580a: Pull complete
  1209. 2020-11-02T11:56:12.5902028Z 13b7038a41b9: Pull complete
  1210. 2020-11-02T11:56:12.5929680Z Digest: sha256:e363e4dd64aaebe588d320fe548d05b1607fa2e3cf87c7c4ec4b85cfc6e57555
  1211. 2020-11-02T11:56:12.5956890Z Status: Downloaded newer image for quay.io/submariner/submariner-route-agent:devel
  1212. 2020-11-02T11:56:12.5988918Z quay.io/submariner/submariner-route-agent:devel
  1213. 2020-11-02T11:56:12.6027181Z [submariner]$ grep FROM package/Dockerfile.submariner-route-agent
  1214. 2020-11-02T11:56:12.6042567Z [submariner]$ cut -f2 -d 
  1215. 2020-11-02T11:56:12.6043598Z [submariner]$ grep -v scratch
  1216. 2020-11-02T11:56:12.6056619Z [submariner]$ cache_flag+= --cache-from registry.access.redhat.com/ubi8/ubi-minimal
  1217. 2020-11-02T11:56:12.6071356Z [submariner]$ docker pull registry.access.redhat.com/ubi8/ubi-minimal
  1218. 2020-11-02T11:56:12.9317225Z Using default tag: latest
  1219. 2020-11-02T11:56:13.5220071Z latest: Pulling from ubi8/ubi-minimal
  1220. 2020-11-02T11:56:13.6553716Z 0fd3b5213a9b: Already exists
  1221. 2020-11-02T11:56:13.6572160Z aebb8c556853: Already exists
  1222. 2020-11-02T11:56:13.7952635Z Digest: sha256:5cfbaf45ca96806917830c183e9f37df2e913b187aadb32e89fd83fa455ebaa6
  1223. 2020-11-02T11:56:13.7978061Z Status: Downloaded newer image for registry.access.redhat.com/ubi8/ubi-minimal:latest
  1224. 2020-11-02T11:56:13.7983882Z registry.access.redhat.com/ubi8/ubi-minimal:latest
  1225. 2020-11-02T11:56:13.8017219Z [submariner]$ buildargs_flag=--build-arg BUILDKIT_INLINE_CACHE=1
  1226. 2020-11-02T11:56:13.8032609Z [submariner]$ DOCKER_BUILDKIT=1 docker build -t quay.io/submariner/submariner-route-agent:dev --cache-from quay.io/submariner/submariner-route-agent:devel --cache-from registry.access.redhat.com/ubi8/ubi-minimal -f package/Dockerfile.submariner-route-agent --build-arg BUILDKIT_INLINE_CACHE=1 .
  1227. 2020-11-02T11:56:14.2632677Z #1 [internal] load .dockerignore
  1228. 2020-11-02T11:56:14.2633153Z #1 transferring context: 2B done
  1229. 2020-11-02T11:56:14.2633445Z #1 DONE 0.0s
  1230. 2020-11-02T11:56:14.2633835Z
  1231. 2020-11-02T11:56:14.2635002Z #2 [internal] load build definition from Dockerfile.submariner-route-agent
  1232. 2020-11-02T11:56:14.2635817Z #2 transferring dockerfile: 513B done
  1233. 2020-11-02T11:56:14.2636092Z #2 DONE 0.0s
  1234. 2020-11-02T11:56:14.2636245Z
  1235. 2020-11-02T11:56:14.2636907Z #3 [internal] load metadata for registry.access.redhat.com/ubi8/ubi-minimal...
  1236. 2020-11-02T11:56:14.2637407Z #3 DONE 0.0s
  1237. 2020-11-02T11:56:14.2637546Z
  1238. 2020-11-02T11:56:14.2638170Z #4 importing cache manifest from quay.io/submariner/submariner-route-agent:...
  1239. 2020-11-02T11:56:14.2638641Z #4 DONE 0.0s
  1240. 2020-11-02T11:56:14.2638778Z
  1241. 2020-11-02T11:56:14.2639416Z #5 importing cache manifest from registry.access.redhat.com/ubi8/ubi-minima...
  1242. 2020-11-02T11:56:14.2639915Z #5 DONE 0.0s
  1243. 2020-11-02T11:56:14.2640051Z
  1244. 2020-11-02T11:56:14.2640600Z #6 [1/5] FROM registry.access.redhat.com/ubi8/ubi-minimal
  1245. 2020-11-02T11:56:14.2641021Z #6 DONE 0.0s
  1246. 2020-11-02T11:56:14.2641174Z
  1247. 2020-11-02T11:56:14.2641403Z #9 [internal] load build context
  1248. 2020-11-02T11:56:14.4138771Z #9 transferring context: 8.51MB 0.1s done
  1249. 2020-11-02T11:56:14.4139225Z #9 DONE 0.1s
  1250. 2020-11-02T11:56:14.4139390Z
  1251. 2020-11-02T11:56:14.4139645Z #7 [2/5] WORKDIR /var/submariner
  1252. 2020-11-02T11:56:14.4139922Z #7 CACHED
  1253. 2020-11-02T11:56:14.4140069Z
  1254. 2020-11-02T11:56:14.4141477Z #8 [3/5] RUN microdnf -y install --nodocs iproute iptables && microdnf ...
  1255. 2020-11-02T11:56:14.4141853Z #8 CACHED
  1256. 2020-11-02T11:56:14.4141996Z
  1257. 2020-11-02T11:56:14.4143020Z #10 [4/5] COPY package/submariner-route-agent.sh bin/submariner-route-agent ...
  1258. 2020-11-02T11:56:17.2871790Z #10 DONE 2.8s
  1259. 2020-11-02T11:56:17.2872180Z
  1260. 2020-11-02T11:56:17.2873325Z #11 [5/5] COPY package/iptables-wrapper.in /usr/sbin/
  1261. 2020-11-02T11:56:17.2873700Z #11 DONE 0.0s
  1262. 2020-11-02T11:56:18.0135657Z
  1263. 2020-11-02T11:56:18.0137291Z #12 exporting to image
  1264. 2020-11-02T11:56:18.0139409Z #12 exporting layers done
  1265. 2020-11-02T11:56:18.0200120Z #12 writing image sha256:898fd60cceb42cf12c67e336cf25b1c65b4a9c03922ff3a54b9e2c418d41742a done
  1266. 2020-11-02T11:56:18.0202146Z #12 naming to quay.io/submariner/submariner-route-agent:dev done
  1267. 2020-11-02T11:56:18.0202617Z #12 DONE 0.0s
  1268. 2020-11-02T11:56:18.0202787Z
  1269. 2020-11-02T11:56:18.0203177Z #13 exporting cache
  1270. 2020-11-02T11:56:18.0203531Z #13 preparing build cache for export done
  1271. 2020-11-02T11:56:18.0209473Z #13 DONE 0.0s
  1272. 2020-11-02T11:56:18.0263829Z [submariner]$ docker tag quay.io/submariner/submariner-route-agent:dev quay.io/submariner/submariner-route-agent:devel
  1273. 2020-11-02T11:56:18.3518198Z touch package/.image.submariner-route-agent
  1274. 2020-11-02T11:56:18.7091089Z /opt/shipyard/scripts/build_image.sh -i submariner-globalnet -f package/Dockerfile.submariner-globalnet
  1275. 2020-11-02T11:56:18.7582667Z fatal: No names found, cannot describe anything.
  1276. 2020-11-02T11:56:18.8484429Z [submariner]$ set -e
  1277. 2020-11-02T11:56:18.8494485Z [submariner]$ local_image=quay.io/submariner/submariner-globalnet:dev
  1278. 2020-11-02T11:56:18.8506802Z [submariner]$ cache_image=quay.io/submariner/submariner-globalnet:devel
  1279. 2020-11-02T11:56:18.8520791Z [submariner]$ cache_flag=
  1280. 2020-11-02T11:56:18.8537385Z [submariner]$ cache_flag=--cache-from quay.io/submariner/submariner-globalnet:devel
  1281. 2020-11-02T11:56:18.8555766Z [submariner]$ docker image ls -q quay.io/submariner/submariner-globalnet:devel
  1282. 2020-11-02T11:56:19.1917331Z [submariner]$ docker pull quay.io/submariner/submariner-globalnet:devel
  1283. 2020-11-02T11:56:19.6338626Z devel: Pulling from submariner/submariner-globalnet
  1284. 2020-11-02T11:56:19.6339585Z 0fd3b5213a9b: Already exists
  1285. 2020-11-02T11:56:19.6355097Z aebb8c556853: Already exists
  1286. 2020-11-02T11:56:19.6375047Z f5092dbc8061: Already exists
  1287. 2020-11-02T11:56:19.6395608Z 45712d871266: Already exists
  1288. 2020-11-02T11:56:19.6428636Z 27090f5c05fc: Pulling fs layer
  1289. 2020-11-02T11:56:19.6429175Z 905e755ba173: Pulling fs layer
  1290. 2020-11-02T11:56:19.7007423Z 905e755ba173: Verifying Checksum
  1291. 2020-11-02T11:56:19.7007795Z 905e755ba173: Download complete
  1292. 2020-11-02T11:56:19.7741507Z 27090f5c05fc: Verifying Checksum
  1293. 2020-11-02T11:56:19.7749240Z 27090f5c05fc: Download complete
  1294. 2020-11-02T11:56:19.9557098Z 27090f5c05fc: Pull complete
  1295. 2020-11-02T11:56:20.0285199Z 905e755ba173: Pull complete
  1296. 2020-11-02T11:56:20.0312687Z Digest: sha256:2ce4ef1b2ae430ec33fd9fccf26c3f1f9a383547f045e455ae37b214d067ceca
  1297. 2020-11-02T11:56:20.0341524Z Status: Downloaded newer image for quay.io/submariner/submariner-globalnet:devel
  1298. 2020-11-02T11:56:20.0371929Z quay.io/submariner/submariner-globalnet:devel
  1299. 2020-11-02T11:56:20.0426139Z [submariner]$ grep FROM package/Dockerfile.submariner-globalnet
  1300. 2020-11-02T11:56:20.0426906Z [submariner]$ cut -f2 -d 
  1301. 2020-11-02T11:56:20.0427380Z [submariner]$ grep -v scratch
  1302. 2020-11-02T11:56:20.0441270Z [submariner]$ cache_flag+= --cache-from registry.access.redhat.com/ubi8/ubi-minimal
  1303. 2020-11-02T11:56:20.0452774Z [submariner]$ docker pull registry.access.redhat.com/ubi8/ubi-minimal
  1304. 2020-11-02T11:56:20.3622256Z Using default tag: latest
  1305. 2020-11-02T11:56:20.6145176Z latest: Pulling from ubi8/ubi-minimal
  1306. 2020-11-02T11:56:20.7637853Z Digest: sha256:5cfbaf45ca96806917830c183e9f37df2e913b187aadb32e89fd83fa455ebaa6
  1307. 2020-11-02T11:56:20.7641847Z Status: Image is up to date for registry.access.redhat.com/ubi8/ubi-minimal:latest
  1308. 2020-11-02T11:56:20.7642795Z registry.access.redhat.com/ubi8/ubi-minimal:latest
  1309. 2020-11-02T11:56:20.7675365Z [submariner]$ buildargs_flag=--build-arg BUILDKIT_INLINE_CACHE=1
  1310. 2020-11-02T11:56:20.7692736Z [submariner]$ DOCKER_BUILDKIT=1 docker build -t quay.io/submariner/submariner-globalnet:dev --cache-from quay.io/submariner/submariner-globalnet:devel --cache-from registry.access.redhat.com/ubi8/ubi-minimal -f package/Dockerfile.submariner-globalnet --build-arg BUILDKIT_INLINE_CACHE=1 .
  1311. 2020-11-02T11:56:21.2105247Z #2 [internal] load .dockerignore
  1312. 2020-11-02T11:56:21.2114087Z #2 transferring context: 2B done
  1313. 2020-11-02T11:56:21.2114502Z #2 DONE 0.0s
  1314. 2020-11-02T11:56:21.2114686Z
  1315. 2020-11-02T11:56:21.2116177Z #1 [internal] load build definition from Dockerfile.submariner-globalnet
  1316. 2020-11-02T11:56:21.2116892Z #1 transferring dockerfile: 505B done
  1317. 2020-11-02T11:56:21.2117211Z #1 DONE 0.0s
  1318. 2020-11-02T11:56:21.2117388Z
  1319. 2020-11-02T11:56:21.2118158Z #3 [internal] load metadata for registry.access.redhat.com/ubi8/ubi-minimal...
  1320. 2020-11-02T11:56:21.2118737Z #3 DONE 0.0s
  1321. 2020-11-02T11:56:21.2118900Z
  1322. 2020-11-02T11:56:21.2119827Z #10 importing cache manifest from registry.access.redhat.com/ubi8/ubi-minima...
  1323. 2020-11-02T11:56:21.2120564Z #10 DONE 0.0s
  1324. 2020-11-02T11:56:21.2120727Z
  1325. 2020-11-02T11:56:21.2121450Z #11 importing cache manifest from quay.io/submariner/submariner-globalnet:de...
  1326. 2020-11-02T11:56:21.2121973Z #11 DONE 0.0s
  1327. 2020-11-02T11:56:21.2122291Z
  1328. 2020-11-02T11:56:21.2123145Z #4 [1/5] FROM registry.access.redhat.com/ubi8/ubi-minimal
  1329. 2020-11-02T11:56:21.2124566Z #4 DONE 0.0s
  1330. 2020-11-02T11:56:21.2124745Z
  1331. 2020-11-02T11:56:21.2125013Z #7 [internal] load build context
  1332. 2020-11-02T11:56:21.2125413Z #7 transferring context: 8.45MB 0.1s done
  1333. 2020-11-02T11:56:21.3608252Z #7 DONE 0.1s
  1334. 2020-11-02T11:56:21.3609066Z
  1335. 2020-11-02T11:56:21.3609497Z #5 [2/5] WORKDIR /var/submariner
  1336. 2020-11-02T11:56:21.3610050Z #5 CACHED
  1337. 2020-11-02T11:56:21.3610432Z
  1338. 2020-11-02T11:56:21.3611406Z #6 [3/5] RUN microdnf -y install --nodocs iproute iptables && microdnf ...
  1339. 2020-11-02T11:56:21.3611834Z #6 CACHED
  1340. 2020-11-02T11:56:21.3612004Z
  1341. 2020-11-02T11:56:21.3612701Z #8 [4/5] COPY package/submariner-globalnet.sh bin/submariner-globalnet /usr...
  1342. 2020-11-02T11:56:21.6770662Z #8 DONE 0.4s
  1343. 2020-11-02T11:56:21.6784695Z
  1344. 2020-11-02T11:56:21.6788856Z #9 [5/5] COPY package/iptables-wrapper.in /usr/sbin/
  1345. 2020-11-02T11:56:21.6789436Z #9 DONE 0.0s
  1346. 2020-11-02T11:56:22.0249873Z
  1347. 2020-11-02T11:56:22.0250475Z #12 exporting to image
  1348. 2020-11-02T11:56:22.0251502Z #12 exporting layers done
  1349. 2020-11-02T11:56:22.0252350Z #12 writing image sha256:82568159ffa0dc68be225b52e420c97f8d627e91a51ff2faf8f87f1510b38695 done
  1350. 2020-11-02T11:56:22.0256497Z #12 naming to quay.io/submariner/submariner-globalnet:dev done
  1351. 2020-11-02T11:56:22.0257082Z #12 DONE 0.0s
  1352. 2020-11-02T11:56:22.0257297Z
  1353. 2020-11-02T11:56:22.0258316Z #13 exporting cache
  1354. 2020-11-02T11:56:22.0258652Z #13 preparing build cache for export done
  1355. 2020-11-02T11:56:22.0259008Z #13 DONE 0.0s
  1356. 2020-11-02T11:56:22.0298755Z [submariner]$ docker tag quay.io/submariner/submariner-globalnet:dev quay.io/submariner/submariner-globalnet:devel
  1357. 2020-11-02T11:56:22.3486555Z touch package/.image.submariner-globalnet
  1358. 2020-11-02T11:56:22.6983183Z /opt/shipyard/scripts/build_image.sh -i submariner-networkplugin-syncer -f package/Dockerfile.submariner-networkplugin-syncer
  1359. 2020-11-02T11:56:22.7456534Z fatal: No names found, cannot describe anything.
  1360. 2020-11-02T11:56:22.8505331Z [submariner]$ set -e
  1361. 2020-11-02T11:56:22.8520914Z [submariner]$ local_image=quay.io/submariner/submariner-networkplugin-syncer:dev
  1362. 2020-11-02T11:56:22.8535260Z [submariner]$ cache_image=quay.io/submariner/submariner-networkplugin-syncer:devel
  1363. 2020-11-02T11:56:22.8549518Z [submariner]$ cache_flag=
  1364. 2020-11-02T11:56:22.8560978Z [submariner]$ cache_flag=--cache-from quay.io/submariner/submariner-networkplugin-syncer:devel
  1365. 2020-11-02T11:56:22.8575939Z [submariner]$ docker image ls -q quay.io/submariner/submariner-networkplugin-syncer:devel
  1366. 2020-11-02T11:56:23.2125634Z [submariner]$ docker pull quay.io/submariner/submariner-networkplugin-syncer:devel
  1367. 2020-11-02T11:56:23.6115561Z Error response from daemon: unauthorized: access to the requested resource is not authorized
  1368. 2020-11-02T11:56:23.6147775Z [submariner]$ :
  1369. 2020-11-02T11:56:23.6168586Z [submariner]$ grep FROM package/Dockerfile.submariner-networkplugin-syncer
  1370. 2020-11-02T11:56:23.6188019Z [submariner]$ cut -f2 -d 
  1371. 2020-11-02T11:56:23.6215197Z [submariner]$ grep -v scratch
  1372. 2020-11-02T11:56:23.6264026Z [submariner]$ cache_flag+= --cache-from registry.access.redhat.com/ubi8/ubi-minimal
  1373. 2020-11-02T11:56:23.6274244Z [submariner]$ docker pull registry.access.redhat.com/ubi8/ubi-minimal
  1374. 2020-11-02T11:56:23.9397414Z Using default tag: latest
  1375. 2020-11-02T11:56:24.1969114Z latest: Pulling from ubi8/ubi-minimal
  1376. 2020-11-02T11:56:24.3432645Z Digest: sha256:5cfbaf45ca96806917830c183e9f37df2e913b187aadb32e89fd83fa455ebaa6
  1377. 2020-11-02T11:56:24.3434429Z Status: Image is up to date for registry.access.redhat.com/ubi8/ubi-minimal:latest
  1378. 2020-11-02T11:56:24.3435595Z registry.access.redhat.com/ubi8/ubi-minimal:latest
  1379. 2020-11-02T11:56:24.3464344Z [submariner]$ buildargs_flag=--build-arg BUILDKIT_INLINE_CACHE=1
  1380. 2020-11-02T11:56:24.3476677Z [submariner]$ DOCKER_BUILDKIT=1 docker build -t quay.io/submariner/submariner-networkplugin-syncer:dev --cache-from quay.io/submariner/submariner-networkplugin-syncer:devel --cache-from registry.access.redhat.com/ubi8/ubi-minimal -f package/Dockerfile.submariner-networkplugin-syncer --build-arg BUILDKIT_INLINE_CACHE=1 .
  1381. 2020-11-02T11:56:24.6752215Z #2 [internal] load .dockerignore
  1382. 2020-11-02T11:56:24.8251236Z #2 transferring context: 2B done
  1383. 2020-11-02T11:56:24.8251849Z #2 DONE 0.0s
  1384. 2020-11-02T11:56:24.8252000Z
  1385. 2020-11-02T11:56:24.8253261Z #1 [internal] load build definition from Dockerfile.submariner-networkplugi...
  1386. 2020-11-02T11:56:24.8254170Z #1 transferring dockerfile: 369B done
  1387. 2020-11-02T11:56:24.8254512Z #1 DONE 0.0s
  1388. 2020-11-02T11:56:24.8254656Z
  1389. 2020-11-02T11:56:24.8255394Z #3 [internal] load metadata for registry.access.redhat.com/ubi8/ubi-minimal...
  1390. 2020-11-02T11:56:24.8256098Z #3 DONE 0.0s
  1391. 2020-11-02T11:56:24.8256297Z
  1392. 2020-11-02T11:56:24.8256982Z #5 [1/3] FROM registry.access.redhat.com/ubi8/ubi-minimal
  1393. 2020-11-02T11:56:24.8257572Z #5 DONE 0.0s
  1394. 2020-11-02T11:56:24.8257763Z
  1395. 2020-11-02T11:56:24.8258045Z #7 [internal] load build context
  1396. 2020-11-02T11:56:24.8258345Z #7 DONE 0.0s
  1397. 2020-11-02T11:56:24.8258482Z
  1398. 2020-11-02T11:56:24.8259184Z #9 importing cache manifest from registry.access.redhat.com/ubi8/ubi-minima...
  1399. 2020-11-02T11:56:24.8259721Z #9 DONE 0.0s
  1400. 2020-11-02T11:56:24.8259861Z
  1401. 2020-11-02T11:56:24.8260839Z #4 importing cache manifest from quay.io/submariner/submariner-networkplugi...
  1402. 2020-11-02T11:56:24.9352962Z #4 ERROR: unexpected status code https://quay.io/v2/submariner/submariner-networkplugin-syncer/manifests/devel: 401 Unauthorized
  1403. 2020-11-02T11:56:24.9353938Z
  1404. 2020-11-02T11:56:24.9354254Z #6 [2/3] WORKDIR /var/submariner
  1405. 2020-11-02T11:56:24.9354512Z #6 CACHED
  1406. 2020-11-02T11:56:24.9354713Z
  1407. 2020-11-02T11:56:24.9355036Z #7 [internal] load build context
  1408. 2020-11-02T11:56:24.9355422Z #7 transferring context: 355.05kB done
  1409. 2020-11-02T11:56:24.9355694Z #7 DONE 0.0s
  1410. 2020-11-02T11:56:24.9355892Z
  1411. 2020-11-02T11:56:24.9356904Z #8 [3/3] COPY package/submariner-networkplugin-syncer.sh bin/submariner-net...
  1412. 2020-11-02T11:56:24.9357590Z #8 DONE 0.0s
  1413. 2020-11-02T11:56:25.1948158Z
  1414. 2020-11-02T11:56:25.1949103Z #10 exporting to image
  1415. 2020-11-02T11:56:25.1949436Z #10 exporting layers done
  1416. 2020-11-02T11:56:25.2131063Z #10 writing image sha256:30b92879bb8a919796442d3628fc6a90f397e2e761f8d7ae71acbfd1b6da02af done
  1417. 2020-11-02T11:56:25.2133584Z #10 naming to quay.io/submariner/submariner-networkplugin-syncer:dev done
  1418. 2020-11-02T11:56:25.2134269Z #10 DONE 0.0s
  1419. 2020-11-02T11:56:25.2134493Z
  1420. 2020-11-02T11:56:25.2134732Z #11 exporting cache
  1421. 2020-11-02T11:56:25.2135137Z #11 preparing build cache for export done
  1422. 2020-11-02T11:56:25.2135505Z #11 DONE 0.0s
  1423. 2020-11-02T11:56:25.2135911Z ------
  1424. 2020-11-02T11:56:25.2136811Z > importing cache manifest from quay.io/submariner/submariner-networkplugin-syncer:devel:
  1425. 2020-11-02T11:56:25.2137983Z ------
  1426. 2020-11-02T11:56:25.2145215Z [submariner]$ docker tag quay.io/submariner/submariner-networkplugin-syncer:dev quay.io/submariner/submariner-networkplugin-syncer:devel
  1427. 2020-11-02T11:56:25.5407938Z touch package/.image.submariner-networkplugin-syncer
  1428. 2020-11-02T11:56:25.5434268Z /opt/shipyard/scripts/deploy.sh --deploytool_submariner_args '--cable-driver libreswan'
  1429. 2020-11-02T11:56:25.6251929Z Running with: globalnet='false', deploytool='operator', deploytool_broker_args='', deploytool_submariner_args='--cable-driver libreswan', cluster_settings='', timeout=5m
  1430. 2020-11-02T11:56:25.6276751Z [submariner]$ source /opt/shipyard/scripts/lib/version
  1431. 2020-11-02T11:56:25.6289001Z [submariner]$ . /opt/shipyard/scripts/lib/source_only
  1432. 2020-11-02T11:56:25.6301716Z [submariner]$ script_name=version
  1433. 2020-11-02T11:56:25.6326495Z [submariner]$ exec_name=deploy.sh
  1434. 2020-11-02T11:56:25.6384886Z [submariner]$ git describe --tags --dirty=-dev --exclude=devel --exclude=latest
  1435. 2020-11-02T11:56:25.7037913Z fatal: No names found, cannot describe anything.
  1436. 2020-11-02T11:56:25.7052715Z [submariner]$ source /opt/shipyard/scripts/lib/utils
  1437. 2020-11-02T11:56:25.7068805Z [submariner]$ . /opt/shipyard/scripts/lib/source_only
  1438. 2020-11-02T11:56:25.7081546Z [submariner]$ script_name=utils
  1439. 2020-11-02T11:56:25.7095327Z [submariner]$ exec_name=deploy.sh
  1440. 2020-11-02T11:56:25.7115502Z [submariner]$ source /opt/shipyard/scripts/lib/deploy_funcs
  1441. 2020-11-02T11:56:25.7130673Z [submariner]$ . /opt/shipyard/scripts/lib/source_only
  1442. 2020-11-02T11:56:25.7141995Z [submariner]$ script_name=deploy_funcs
  1443. 2020-11-02T11:56:25.7158245Z [submariner]$ exec_name=deploy.sh
  1444. 2020-11-02T11:56:25.7178914Z [submariner]$ source /opt/shipyard/scripts/lib/cluster_settings
  1445. 2020-11-02T11:56:25.7193574Z [submariner]$ . /opt/shipyard/scripts/lib/source_only
  1446. 2020-11-02T11:56:25.7207781Z [submariner]$ script_name=cluster_settings
  1447. 2020-11-02T11:56:25.7217974Z [submariner]$ exec_name=deploy.sh
  1448. 2020-11-02T11:56:25.7238605Z [submariner]$ broker=cluster1
  1449. 2020-11-02T11:56:25.7252991Z [submariner]$ declare -gA cluster_nodes
  1450. 2020-11-02T11:56:25.7266213Z [submariner]$ cluster_nodes[cluster1]=control-plane worker
  1451. 2020-11-02T11:56:25.7278637Z [submariner]$ cluster_nodes[cluster2]=control-plane worker
  1452. 2020-11-02T11:56:25.7292281Z [submariner]$ cluster_nodes[cluster3]=control-plane worker worker
  1453. 2020-11-02T11:56:25.7304646Z [submariner]$ declare -gA cluster_subm
  1454. 2020-11-02T11:56:25.7316631Z [submariner]$ cluster_subm[cluster1]=true
  1455. 2020-11-02T11:56:25.7329022Z [submariner]$ cluster_subm[cluster2]=true
  1456. 2020-11-02T11:56:25.7341505Z [submariner]$ cluster_subm[cluster3]=true
  1457. 2020-11-02T11:56:25.7351445Z [submariner]$ declare -gA cluster_cni
  1458. 2020-11-02T11:56:25.7362594Z [submariner]$ cluster_cni[cluster2]=weave
  1459. 2020-11-02T11:56:25.7373678Z [submariner]$ cluster_cni[cluster3]=weave
  1460. 2020-11-02T11:56:25.7384264Z [submariner]$ declare_cidrs
  1461. 2020-11-02T11:56:25.7396258Z [submariner]$ declare_cidrs
  1462. 2020-11-02T11:56:25.7407027Z [submariner]$ declare -gA cluster_CIDRs service_CIDRs global_CIDRs
  1463. 2020-11-02T11:56:25.7418971Z [submariner]$ i=1
  1464. 2020-11-02T11:56:25.7432057Z [submariner]$ [cluster1] add_cluster_cidrs 1 cluster1
  1465. 2020-11-02T11:56:25.7451002Z [submariner]$ [cluster1] add_cluster_cidrs 1 cluster1
  1466. 2020-11-02T11:56:25.7466159Z [submariner]$ [cluster1] local val=1
  1467. 2020-11-02T11:56:25.7478395Z [submariner]$ [cluster1] local idx=cluster1
  1468. 2020-11-02T11:56:25.7490385Z [submariner]$ [cluster1] cluster_CIDRs[cluster1]=10.1.0.0/16
  1469. 2020-11-02T11:56:25.7503206Z [submariner]$ [cluster1] service_CIDRs[cluster1]=100.1.0.0/16
  1470. 2020-11-02T11:56:25.7515937Z [submariner]$ [cluster1] i=2
  1471. 2020-11-02T11:56:25.7530087Z [submariner]$ [cluster2] add_cluster_cidrs 2 cluster2
  1472. 2020-11-02T11:56:25.7541687Z [submariner]$ [cluster2] add_cluster_cidrs 2 cluster2
  1473. 2020-11-02T11:56:25.7555375Z [submariner]$ [cluster2] local val=2
  1474. 2020-11-02T11:56:25.7581753Z [submariner]$ [cluster2] local idx=cluster2
  1475. 2020-11-02T11:56:25.7596471Z [submariner]$ [cluster2] cluster_CIDRs[cluster2]=10.2.0.0/16
  1476. 2020-11-02T11:56:25.7609441Z [submariner]$ [cluster2] service_CIDRs[cluster2]=100.2.0.0/16
  1477. 2020-11-02T11:56:25.7624341Z [submariner]$ [cluster2] i=3
  1478. 2020-11-02T11:56:25.7638415Z [submariner]$ [cluster3] add_cluster_cidrs 3 cluster3
  1479. 2020-11-02T11:56:25.7654250Z [submariner]$ [cluster3] add_cluster_cidrs 3 cluster3
  1480. 2020-11-02T11:56:25.7669124Z [submariner]$ [cluster3] local val=3
  1481. 2020-11-02T11:56:25.7684710Z [submariner]$ [cluster3] local idx=cluster3
  1482. 2020-11-02T11:56:25.7697882Z [submariner]$ [cluster3] cluster_CIDRs[cluster3]=10.3.0.0/16
  1483. 2020-11-02T11:56:25.7747370Z [submariner]$ [cluster3] service_CIDRs[cluster3]=100.3.0.0/16
  1484. 2020-11-02T11:56:25.7759916Z [submariner]$ [cluster3] i=4
  1485. 2020-11-02T11:56:25.7774419Z [submariner]$ [cluster3] declare_kubeconfig
  1486. 2020-11-02T11:56:25.7787264Z [submariner]$ [cluster3] declare_kubeconfig
  1487. 2020-11-02T11:56:25.7800265Z [submariner]$ [cluster3] source /opt/shipyard/scripts/lib/kubecfg
  1488. 2020-11-02T11:56:25.7829652Z [submariner]$ [cluster3] export KUBECONFIG
  1489. 2020-11-02T11:56:25.7899608Z [submariner]$ [cluster3] KUBECONFIG=/go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster2:/go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster3:/go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster1:
  1490. 2020-11-02T11:56:25.7922009Z [submariner]$ [cluster3] find /go/src/github.com/submariner-io/submariner/output/kubeconfigs -type f -printf %p:
  1491. 2020-11-02T11:56:25.7953912Z [submariner]$ [cluster3] import_image quay.io/submariner/nettest
  1492. 2020-11-02T11:56:25.7967705Z [submariner]$ [cluster3] import_image quay.io/submariner/nettest
  1493. 2020-11-02T11:56:25.7980081Z [submariner]$ [cluster3] local orig_image=quay.io/submariner/nettest
  1494. 2020-11-02T11:56:25.7991210Z [submariner]$ [cluster3] local versioned_image=quay.io/submariner/nettest:dev
  1495. 2020-11-02T11:56:25.8006599Z [submariner]$ [cluster3] local local_image=localhost:5000/nettest:local
  1496. 2020-11-02T11:56:25.8019498Z [submariner]$ [cluster3] docker tag quay.io/submariner/nettest:dev localhost:5000/nettest:local
  1497. 2020-11-02T11:56:26.1133092Z Error response from daemon: No such image: quay.io/submariner/nettest:dev
  1498. 2020-11-02T11:56:26.1166220Z [submariner]$ [cluster3] docker pull quay.io/submariner/nettest:devel
  1499. 2020-11-02T11:56:26.6048486Z devel: Pulling from submariner/nettest
  1500. 2020-11-02T11:56:26.6049288Z 188c0c94c7c5: Pulling fs layer
  1501. 2020-11-02T11:56:26.6049679Z 56cefa24c928: Pulling fs layer
  1502. 2020-11-02T11:56:26.6050028Z 629d5148bcbc: Pulling fs layer
  1503. 2020-11-02T11:56:26.6050361Z 630288c35ea1: Pulling fs layer
  1504. 2020-11-02T11:56:26.6050613Z 630288c35ea1: Waiting
  1505. 2020-11-02T11:56:26.6624475Z 56cefa24c928: Verifying Checksum
  1506. 2020-11-02T11:56:26.6625707Z 56cefa24c928: Download complete
  1507. 2020-11-02T11:56:26.7070906Z 188c0c94c7c5: Download complete
  1508. 2020-11-02T11:56:26.7231352Z 630288c35ea1: Verifying Checksum
  1509. 2020-11-02T11:56:26.7231953Z 630288c35ea1: Download complete
  1510. 2020-11-02T11:56:26.7392915Z 629d5148bcbc: Verifying Checksum
  1511. 2020-11-02T11:56:26.7393411Z 629d5148bcbc: Download complete
  1512. 2020-11-02T11:56:26.9006701Z 188c0c94c7c5: Pull complete
  1513. 2020-11-02T11:56:26.9762670Z 56cefa24c928: Pull complete
  1514. 2020-11-02T11:56:27.3307856Z 629d5148bcbc: Pull complete
  1515. 2020-11-02T11:56:27.4060688Z 630288c35ea1: Pull complete
  1516. 2020-11-02T11:56:27.4089787Z Digest: sha256:897b8f7645e3942b17fdd1e4560c4e64e19b97020677912e39f075b68e5e943a
  1517. 2020-11-02T11:56:27.4119596Z Status: Downloaded newer image for quay.io/submariner/nettest:devel
  1518. 2020-11-02T11:56:27.4205621Z quay.io/submariner/nettest:devel
  1519. 2020-11-02T11:56:27.4209528Z [submariner]$ [cluster3] docker tag quay.io/submariner/nettest:devel quay.io/submariner/nettest:dev
  1520. 2020-11-02T11:56:27.7481901Z [submariner]$ [cluster3] docker tag quay.io/submariner/nettest:dev localhost:5000/nettest:local
  1521. 2020-11-02T11:56:28.0561140Z [submariner]$ [cluster3] docker push localhost:5000/nettest:local
  1522. 2020-11-02T11:56:28.3531131Z The push refers to repository [localhost:5000/nettest]
  1523. 2020-11-02T11:56:28.4184215Z 38c1b9018ad3: Preparing
  1524. 2020-11-02T11:56:28.4184717Z f0cf3a781ab4: Preparing
  1525. 2020-11-02T11:56:28.4185077Z d26ed8773de6: Preparing
  1526. 2020-11-02T11:56:28.4185719Z ace0eda3e3be: Preparing
  1527. 2020-11-02T11:56:29.2729578Z d26ed8773de6: Pushed
  1528. 2020-11-02T11:56:29.2822361Z 38c1b9018ad3: Pushed
  1529. 2020-11-02T11:56:29.2851702Z ace0eda3e3be: Pushed
  1530. 2020-11-02T11:56:30.1894061Z f0cf3a781ab4: Pushed
  1531. 2020-11-02T11:56:30.2091740Z local: digest: sha256:897b8f7645e3942b17fdd1e4560c4e64e19b97020677912e39f075b68e5e943a size: 1155
  1532. 2020-11-02T11:56:30.2130072Z [submariner]$ [cluster3] import_image quay.io/submariner/submariner
  1533. 2020-11-02T11:56:30.2151591Z [submariner]$ [cluster3] import_image quay.io/submariner/submariner
  1534. 2020-11-02T11:56:30.2155417Z [submariner]$ [cluster3] local orig_image=quay.io/submariner/submariner
  1535. 2020-11-02T11:56:30.2168949Z [submariner]$ [cluster3] local versioned_image=quay.io/submariner/submariner:dev
  1536. 2020-11-02T11:56:30.2207007Z [submariner]$ [cluster3] local local_image=localhost:5000/submariner:local
  1537. 2020-11-02T11:56:30.2239456Z [submariner]$ [cluster3] docker tag quay.io/submariner/submariner:dev localhost:5000/submariner:local
  1538. 2020-11-02T11:56:30.5260845Z [submariner]$ [cluster3] docker push localhost:5000/submariner:local
  1539. 2020-11-02T11:56:30.8351211Z The push refers to repository [localhost:5000/submariner]
  1540. 2020-11-02T11:56:30.8461342Z 7fd30380ce4c: Preparing
  1541. 2020-11-02T11:56:30.8461784Z 9d27e5761ec3: Preparing
  1542. 2020-11-02T11:56:30.8462089Z 79f310a5b43c: Preparing
  1543. 2020-11-02T11:56:30.8462524Z af4259372ae6: Preparing
  1544. 2020-11-02T11:56:30.8462905Z f3c6dcc5e8bc: Preparing
  1545. 2020-11-02T11:56:31.2254609Z 7fd30380ce4c: Pushed
  1546. 2020-11-02T11:56:31.4989961Z af4259372ae6: Pushed
  1547. 2020-11-02T11:56:31.5363746Z 9d27e5761ec3: Pushed
  1548. 2020-11-02T11:56:35.7431328Z 79f310a5b43c: Pushed
  1549. 2020-11-02T11:56:45.2929372Z f3c6dcc5e8bc: Pushed
  1550. 2020-11-02T11:56:45.3135098Z local: digest: sha256:e25998f0e01c4d8db384180156d56b93b1d0aba78f4bdd0c2cdcb8f23dc8b274 size: 1366
  1551. 2020-11-02T11:56:45.3179119Z [submariner]$ [cluster3] import_image quay.io/submariner/submariner-route-agent
  1552. 2020-11-02T11:56:45.3192869Z [submariner]$ [cluster3] import_image quay.io/submariner/submariner-route-agent
  1553. 2020-11-02T11:56:45.3216798Z [submariner]$ [cluster3] local orig_image=quay.io/submariner/submariner-route-agent
  1554. 2020-11-02T11:56:45.3229994Z [submariner]$ [cluster3] local versioned_image=quay.io/submariner/submariner-route-agent:dev
  1555. 2020-11-02T11:56:45.3245256Z [submariner]$ [cluster3] local local_image=localhost:5000/submariner-route-agent:local
  1556. 2020-11-02T11:56:45.3261613Z [submariner]$ [cluster3] docker tag quay.io/submariner/submariner-route-agent:dev localhost:5000/submariner-route-agent:local
  1557. 2020-11-02T11:56:45.6333171Z [submariner]$ [cluster3] docker push localhost:5000/submariner-route-agent:local
  1558. 2020-11-02T11:56:45.9410219Z The push refers to repository [localhost:5000/submariner-route-agent]
  1559. 2020-11-02T11:56:45.9514250Z d8d4aad8c4e5: Preparing
  1560. 2020-11-02T11:56:45.9514752Z 4bb901307c7a: Preparing
  1561. 2020-11-02T11:56:45.9515133Z bc50e5baf75a: Preparing
  1562. 2020-11-02T11:56:45.9516612Z 98629952edb8: Preparing
  1563. 2020-11-02T11:56:45.9517684Z 3485805ce47c: Preparing
  1564. 2020-11-02T11:56:45.9518189Z b0e2911c99f3: Preparing
  1565. 2020-11-02T11:56:45.9518541Z b0e2911c99f3: Waiting
  1566. 2020-11-02T11:56:46.6182858Z d8d4aad8c4e5: Pushed
  1567. 2020-11-02T11:56:46.6183375Z 3485805ce47c: Pushed
  1568. 2020-11-02T11:56:46.6285454Z 98629952edb8: Pushed
  1569. 2020-11-02T11:56:46.6627702Z 4bb901307c7a: Pushed
  1570. 2020-11-02T11:56:46.9411883Z bc50e5baf75a: Pushed
  1571. 2020-11-02T11:56:56.4933883Z b0e2911c99f3: Pushed
  1572. 2020-11-02T11:56:56.5153411Z local: digest: sha256:f10fcdae4da8697ed54b212608189440846bbabbee8843e8b2ff6fd372a94cc3 size: 1573
  1573. 2020-11-02T11:56:56.5196355Z [submariner]$ [cluster3] load_deploytool operator
  1574. 2020-11-02T11:56:56.5212705Z [submariner]$ [cluster3] load_deploytool operator
  1575. 2020-11-02T11:56:56.5221704Z [submariner]$ [cluster3] local deploytool=operator
  1576. 2020-11-02T11:56:56.5231068Z [submariner]$ [cluster3] local deploy_lib=/opt/shipyard/scripts/lib/deploy_operator
  1577. 2020-11-02T11:56:56.5243348Z [submariner]$ [cluster3] . /opt/shipyard/scripts/lib/deploy_operator
  1578. 2020-11-02T11:56:56.5244278Z Will deploy submariner using operator
  1579. 2020-11-02T11:56:56.5258252Z [submariner]$ [cluster3] . /opt/shipyard/scripts/lib/source_only
  1580. 2020-11-02T11:56:56.5272067Z [submariner]$ [cluster3] script_name=deploy_operator
  1581. 2020-11-02T11:56:56.5285394Z [submariner]$ [cluster3] exec_name=deploy.sh
  1582. 2020-11-02T11:56:56.5305749Z [submariner]$ [cluster3] deploytool_prereqs
  1583. 2020-11-02T11:56:56.5314753Z [submariner]$ [cluster3] deploytool_prereqs
  1584. 2020-11-02T11:56:56.5327354Z [submariner]$ [cluster3] test -x /go/bin/subctl
  1585. 2020-11-02T11:56:56.5341037Z [submariner]$ [cluster3] /go/bin/subctl version
  1586. 2020-11-02T11:56:56.7506549Z subctl version: v0.7.0-8-g282818e
  1587. 2020-11-02T11:56:56.7539454Z [submariner]$ [cluster3] import_image quay.io/submariner/submariner-operator
  1588. 2020-11-02T11:56:56.7552855Z [submariner]$ [cluster3] import_image quay.io/submariner/submariner-operator
  1589. 2020-11-02T11:56:56.7567870Z [submariner]$ [cluster3] local orig_image=quay.io/submariner/submariner-operator
  1590. 2020-11-02T11:56:56.7580125Z [submariner]$ [cluster3] local versioned_image=quay.io/submariner/submariner-operator:dev
  1591. 2020-11-02T11:56:56.7593251Z [submariner]$ [cluster3] local local_image=localhost:5000/submariner-operator:local
  1592. 2020-11-02T11:56:56.7612338Z [submariner]$ [cluster3] docker tag quay.io/submariner/submariner-operator:dev localhost:5000/submariner-operator:local
  1593. 2020-11-02T11:56:57.0812558Z Error response from daemon: No such image: quay.io/submariner/submariner-operator:dev
  1594. 2020-11-02T11:56:57.0833707Z [submariner]$ [cluster3] docker pull quay.io/submariner/submariner-operator:devel
  1595. 2020-11-02T11:56:57.5159724Z devel: Pulling from submariner/submariner-operator
  1596. 2020-11-02T11:56:57.5160377Z 95ecb56ba51a: Pulling fs layer
  1597. 2020-11-02T11:56:57.6478180Z 95ecb56ba51a: Download complete
  1598. 2020-11-02T11:56:58.1470158Z 95ecb56ba51a: Pull complete
  1599. 2020-11-02T11:56:58.1498082Z Digest: sha256:0782ca4c712c2db646d03c9d9963bbfd1346f8e5ef28fb49a4e189e7df1b0058
  1600. 2020-11-02T11:56:58.1522785Z Status: Downloaded newer image for quay.io/submariner/submariner-operator:devel
  1601. 2020-11-02T11:56:58.1568030Z quay.io/submariner/submariner-operator:devel
  1602. 2020-11-02T11:56:58.1572797Z [submariner]$ [cluster3] docker tag quay.io/submariner/submariner-operator:devel quay.io/submariner/submariner-operator:dev
  1603. 2020-11-02T11:56:58.4856531Z [submariner]$ [cluster3] docker tag quay.io/submariner/submariner-operator:dev localhost:5000/submariner-operator:local
  1604. 2020-11-02T11:56:58.7929971Z [submariner]$ [cluster3] docker push localhost:5000/submariner-operator:local
  1605. 2020-11-02T11:56:59.1075867Z The push refers to repository [localhost:5000/submariner-operator]
  1606. 2020-11-02T11:56:59.1117966Z d7181d385c8c: Preparing
  1607. 2020-11-02T11:57:01.6704386Z d7181d385c8c: Pushed
  1608. 2020-11-02T11:57:01.6934044Z local: digest: sha256:0782ca4c712c2db646d03c9d9963bbfd1346f8e5ef28fb49a4e189e7df1b0058 size: 529
  1609. 2020-11-02T11:57:01.6974034Z [submariner]$ [cluster3] run_subm_clusters prepare_cluster submariner-operator
  1610. 2020-11-02T11:57:01.6994121Z [submariner]$ [cluster3] run_subm_clusters prepare_cluster submariner-operator
  1611. 2020-11-02T11:57:01.7013632Z [submariner]$ [cluster3] declare -a subm_clusters
  1612. 2020-11-02T11:57:01.7073220Z [submariner]$ [cluster3] run_parallel cluster1 cluster2 cluster3 prepare_cluster submariner-operator
  1613. 2020-11-02T11:57:01.7087860Z [submariner]$ [cluster3] run_parallel cluster1 cluster2 cluster3 cluster1 cluster2 cluster3 prepare_cluster submariner-operator
  1614. 2020-11-02T11:57:01.7100682Z [submariner]$ [cluster3] local cmnd=prepare_cluster
  1615. 2020-11-02T11:57:01.7112794Z [submariner]$ [cluster3] declare -A pids
  1616. 2020-11-02T11:57:01.7134651Z [submariner]$ [cluster3] eval echo cluster1 cluster2 cluster3
  1617. 2020-11-02T11:57:01.7162583Z [submariner]$ [cluster1] pids[cluster1]=5469
  1618. 2020-11-02T11:57:01.7177328Z [submariner]$ [cluster1] set -o pipefail
  1619. 2020-11-02T11:57:01.7189302Z [submariner]$ [cluster2] pids[cluster2]=5472
  1620. 2020-11-02T11:57:01.7196851Z [submariner]$ [cluster2] set -o pipefail
  1621. 2020-11-02T11:57:01.7209781Z [submariner]$ [cluster1] prepare_cluster submariner-operator
  1622. 2020-11-02T11:57:01.7216989Z [submariner]$ [cluster3] pids[cluster3]=5476
  1623. 2020-11-02T11:57:01.7226304Z [submariner]$ [cluster2] prepare_cluster submariner-operator
  1624. 2020-11-02T11:57:01.7240711Z [submariner]$ [cluster1] sed /\[cluster1]/!s/^/[cluster1] /
  1625. 2020-11-02T11:57:01.7245751Z [submariner]$ [cluster3] wait 5476
  1626. 2020-11-02T11:57:01.7253779Z [submariner]$ [cluster3] set -o pipefail
  1627. 2020-11-02T11:57:01.7261280Z [submariner]$ [cluster2] sed /\[cluster2]/!s/^/[cluster2] /
  1628. 2020-11-02T11:57:01.7270621Z [submariner]$ [cluster3] prepare_cluster submariner-operator
  1629. 2020-11-02T11:57:01.7312625Z [submariner]$ [cluster3] sed /\[cluster3]/!s/^/[cluster3] /
  1630. 2020-11-02T11:57:03.5895835Z [submariner]$ [cluster1] prepare_cluster
  1631. 2020-11-02T11:57:03.5896605Z [submariner]$ [cluster1] local namespace=submariner-operator
  1632. 2020-11-02T11:57:03.5897160Z [cluster1] 
  1633. 2020-11-02T11:57:03.5897549Z [cluster1] 
  1634. 2020-11-02T11:57:03.5897903Z [cluster1] 
  1635. 2020-11-02T11:57:03.5898363Z [submariner]$ [cluster1] add_subm_gateway_label
  1636. 2020-11-02T11:57:03.5898938Z [submariner]$ [cluster1] add_subm_gateway_label
  1637. 2020-11-02T11:57:03.5899886Z [submariner]$ [cluster1] kubectl label node cluster1-worker submariner.io/gateway=true --overwrite
  1638. 2020-11-02T11:57:03.5900879Z [submariner]$ [cluster1] kubectl label node cluster1-worker submariner.io/gateway=true --overwrite
  1639. 2020-11-02T11:57:03.5902175Z [submariner]$ [cluster1] command kubectl --context=cluster1 label node cluster1-worker submariner.io/gateway=true --overwrite
  1640. 2020-11-02T11:57:03.5903354Z [submariner]$ [cluster1] kubectl --context=cluster1 label node cluster1-worker submariner.io/gateway=true --overwrite
  1641. 2020-11-02T11:57:03.5904157Z [cluster1] node/cluster1-worker labeled
  1642. 2020-11-02T11:57:03.8798317Z [submariner]$ [cluster3] prepare_cluster
  1643. 2020-11-02T11:57:03.8799199Z [submariner]$ [cluster3] local namespace=submariner-operator
  1644. 2020-11-02T11:57:03.8799927Z [cluster3] 
  1645. 2020-11-02T11:57:03.8800281Z [cluster3] 
  1646. 2020-11-02T11:57:03.8800769Z [cluster3] 
  1647. 2020-11-02T11:57:03.8801276Z [submariner]$ [cluster3] add_subm_gateway_label
  1648. 2020-11-02T11:57:03.8801845Z [submariner]$ [cluster3] add_subm_gateway_label
  1649. 2020-11-02T11:57:03.8802786Z [submariner]$ [cluster3] kubectl label node cluster3-worker submariner.io/gateway=true --overwrite
  1650. 2020-11-02T11:57:03.8803723Z [submariner]$ [cluster3] kubectl label node cluster3-worker submariner.io/gateway=true --overwrite
  1651. 2020-11-02T11:57:03.8805210Z [submariner]$ [cluster3] command kubectl --context=cluster3 label node cluster3-worker submariner.io/gateway=true --overwrite
  1652. 2020-11-02T11:57:03.8806304Z [submariner]$ [cluster3] kubectl --context=cluster3 label node cluster3-worker submariner.io/gateway=true --overwrite
  1653. 2020-11-02T11:57:03.8807353Z [cluster3] node/cluster3-worker labeled
  1654. 2020-11-02T11:57:03.8817511Z [submariner]$ [cluster3] wait 5472
  1655. 2020-11-02T11:57:04.0420935Z [submariner]$ [cluster2] prepare_cluster
  1656. 2020-11-02T11:57:04.0421712Z [submariner]$ [cluster2] local namespace=submariner-operator
  1657. 2020-11-02T11:57:04.0422267Z [cluster2] 
  1658. 2020-11-02T11:57:04.0422623Z [cluster2] 
  1659. 2020-11-02T11:57:04.0422976Z [cluster2] 
  1660. 2020-11-02T11:57:04.0423433Z [submariner]$ [cluster2] add_subm_gateway_label
  1661. 2020-11-02T11:57:04.0424033Z [submariner]$ [cluster2] add_subm_gateway_label
  1662. 2020-11-02T11:57:04.0424831Z [submariner]$ [cluster2] kubectl label node cluster2-worker submariner.io/gateway=true --overwrite
  1663. 2020-11-02T11:57:04.0425897Z [submariner]$ [cluster2] kubectl label node cluster2-worker submariner.io/gateway=true --overwrite
  1664. 2020-11-02T11:57:04.0426982Z [submariner]$ [cluster2] command kubectl --context=cluster2 label node cluster2-worker submariner.io/gateway=true --overwrite
  1665. 2020-11-02T11:57:04.0428120Z [submariner]$ [cluster2] kubectl --context=cluster2 label node cluster2-worker submariner.io/gateway=true --overwrite
  1666. 2020-11-02T11:57:04.0428920Z [cluster2] node/cluster2-worker labeled
  1667. 2020-11-02T11:57:04.0445939Z [submariner]$ [cluster3] wait 5469
  1668. 2020-11-02T11:57:04.0462277Z [submariner]$ [cluster3] with_context cluster1 setup_broker
  1669. 2020-11-02T11:57:04.0475951Z [submariner]$ [cluster3] with_context cluster1 setup_broker
  1670. 2020-11-02T11:57:04.0489541Z [submariner]$ [cluster3] local cluster=cluster1
  1671. 2020-11-02T11:57:04.0501761Z [submariner]$ [cluster1] local cmnd=setup_broker
  1672. 2020-11-02T11:57:04.0514811Z [submariner]$ [cluster1] setup_broker
  1673. 2020-11-02T11:57:04.0530537Z [submariner]$ [cluster1] setup_broker
  1674. 2020-11-02T11:57:04.0540959Z [submariner]$ [cluster1] local gn
  1675. 2020-11-02T11:57:04.0546045Z Installing broker...
  1676. 2020-11-02T11:57:04.0563410Z [submariner]$ [cluster1] cd /go/src/github.com/submariner-io/submariner/output
  1677. 2020-11-02T11:57:05.0378997Z [output]$ [cluster1] subctl deploy-broker --kubeconfig /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster1
  1678. 2020-11-02T11:57:05.0388388Z • Deploying broker ...
  1679. 2020-11-02T11:57:05.0389186Z ✓ Deploying broker
  1680. 2020-11-02T11:57:05.0390325Z • Creating broker-info.subm file ...
  1681. 2020-11-02T11:57:05.0391171Z ✓ Creating broker-info.subm file
  1682. 2020-11-02T11:57:05.0392508Z ✓ A new IPsec PSK will be generated for broker-info.subm
  1683. 2020-11-02T11:57:05.0393719Z [submariner]$ [cluster3] install_subm_all_clusters
  1684. 2020-11-02T11:57:05.0394625Z [submariner]$ [cluster3] install_subm_all_clusters
  1685. 2020-11-02T11:57:05.0395556Z [submariner]$ [cluster3] run_subm_clusters subctl_install_subm
  1686. 2020-11-02T11:57:05.0396956Z [submariner]$ [cluster3] run_subm_clusters subctl_install_subm
  1687. 2020-11-02T11:57:05.0398247Z [submariner]$ [cluster3] declare -a subm_clusters
  1688. 2020-11-02T11:57:05.0400329Z [submariner]$ [cluster3] run_parallel cluster1 cluster2 cluster3 subctl_install_subm
  1689. 2020-11-02T11:57:05.0402198Z [submariner]$ [cluster3] run_parallel cluster1 cluster2 cluster3 cluster1 cluster2 cluster3 subctl_install_subm
  1690. 2020-11-02T11:57:05.0404513Z [submariner]$ [cluster3] local cmnd=subctl_install_subm
  1691. 2020-11-02T11:57:05.0406496Z [submariner]$ [cluster3] declare -A pids
  1692. 2020-11-02T11:57:05.0409885Z [submariner]$ [cluster3] eval echo cluster1 cluster2 cluster3
  1693. 2020-11-02T11:57:05.0412066Z [submariner]$ [cluster1] pids[cluster1]=5646
  1694. 2020-11-02T11:57:05.0414115Z [submariner]$ [cluster1] set -o pipefail
  1695. 2020-11-02T11:57:05.0415942Z [submariner]$ [cluster2] pids[cluster2]=5649
  1696. 2020-11-02T11:57:05.0422729Z [submariner]$ [cluster1] subctl_install_subm
  1697. 2020-11-02T11:57:05.0427546Z [submariner]$ [cluster2] set -o pipefail
  1698. 2020-11-02T11:57:05.0434057Z [submariner]$ [cluster1] sed /\[cluster1]/!s/^/[cluster1] /
  1699. 2020-11-02T11:57:05.0442260Z [submariner]$ [cluster3] pids[cluster3]=5653
  1700. 2020-11-02T11:57:05.0444475Z [submariner]$ [cluster2] subctl_install_subm
  1701. 2020-11-02T11:57:05.0448596Z [submariner]$ [cluster3] wait 5653
  1702. 2020-11-02T11:57:05.0454897Z [submariner]$ [cluster3] set -o pipefail
  1703. 2020-11-02T11:57:05.0457323Z [submariner]$ [cluster2] sed /\[cluster2]/!s/^/[cluster2] /
  1704. 2020-11-02T11:57:05.0461992Z [submariner]$ [cluster3] subctl_install_subm
  1705. 2020-11-02T11:57:05.0466886Z [submariner]$ [cluster3] sed /\[cluster3]/!s/^/[cluster3] /
  1706. 2020-11-02T11:57:13.2980984Z [submariner]$ [cluster2] subctl_install_subm
  1707. 2020-11-02T11:57:13.2984500Z [submariner]$ [cluster2] subctl join --kubeconfig /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster2 --clusterid cluster2 --repository localhost:5000 --version local --nattport 4500 --ikeport 500 --colorcodes blue --globalnet-cidr --disable-nat --cable-driver libreswan /go/src/github.com/submariner-io/submariner/output/broker-info.subm
  1708. 2020-11-02T11:57:13.2986520Z [cluster2] * /go/src/github.com/submariner-io/submariner/output/broker-info.subm says broker is at: https://172.17.0.9:6443
  1709. 2020-11-02T11:57:13.2987380Z [cluster2] * There are 1 labeled nodes in the cluster:
  1710. 2020-11-02T11:57:13.2987892Z [cluster2] - cluster2-worker
  1711. 2020-11-02T11:57:13.2988411Z [cluster2] • Discovering network details ...
  1712. 2020-11-02T11:57:13.2988808Z [cluster2] Discovered network details:
  1713. 2020-11-02T11:57:13.2989417Z [cluster2] Network plugin: weave-net
  1714. 2020-11-02T11:57:13.2989768Z [cluster2] Service CIDRs: [100.2.0.0/16]
  1715. 2020-11-02T11:57:13.2990087Z [cluster2] Cluster CIDRs: [10.2.0.0/16]
  1716. 2020-11-02T11:57:13.2990591Z [cluster2] ✓ Discovering network details
  1717. 2020-11-02T11:57:13.2991314Z [cluster2] • Discovering multi cluster details ...
  1718. 2020-11-02T11:57:13.2991914Z [cluster2] • Validating Globalnet configurations ...
  1719. 2020-11-02T11:57:13.2992587Z [cluster2] ✓ Validating Globalnet configurations
  1720. 2020-11-02T11:57:13.2993180Z [cluster2] ✓ Discovering multi cluster details
  1721. 2020-11-02T11:57:13.2993755Z [cluster2] • Deploying the Submariner operator ...
  1722. 2020-11-02T11:57:13.2994485Z [cluster2] ✓ Deploying the Submariner operator
  1723. 2020-11-02T11:57:13.2994990Z [cluster2] ✓ Created operator CRDs
  1724. 2020-11-02T11:57:13.2995750Z [cluster2] ✓ Created operator namespace: submariner-operator
  1725. 2020-11-02T11:57:13.2996374Z [cluster2] ✓ Created operator service account and role
  1726. 2020-11-02T11:57:13.2997390Z [cluster2] ✓ Created lighthouse service account and role
  1727. 2020-11-02T11:57:13.2998086Z [cluster2] ✓ Created Lighthouse service accounts and roles
  1728. 2020-11-02T11:57:13.2998674Z [cluster2] ✓ Deployed the operator successfully
  1729. 2020-11-02T11:57:13.2999195Z [cluster2] • Creating SA for cluster ...
  1730. 2020-11-02T11:57:13.2999663Z [cluster2] ✓ Creating SA for cluster
  1731. 2020-11-02T11:57:13.3000136Z [cluster2] • Deploying Submariner ...
  1732. 2020-11-02T11:57:13.3000660Z [cluster2] ✓ Deploying Submariner
  1733. 2020-11-02T11:57:13.3001146Z [cluster2] ✓ Submariner is up and running
  1734. 2020-11-02T11:57:16.8485147Z [submariner]$ [cluster3] subctl_install_subm
  1735. 2020-11-02T11:57:16.8487317Z [submariner]$ [cluster3] subctl join --kubeconfig /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster3 --clusterid cluster3 --repository localhost:5000 --version local --nattport 4500 --ikeport 500 --colorcodes blue --globalnet-cidr --disable-nat --cable-driver libreswan /go/src/github.com/submariner-io/submariner/output/broker-info.subm
  1736. 2020-11-02T11:57:16.8489373Z [cluster3] * /go/src/github.com/submariner-io/submariner/output/broker-info.subm says broker is at: https://172.17.0.9:6443
  1737. 2020-11-02T11:57:16.8490483Z [cluster3] * There are 1 labeled nodes in the cluster:
  1738. 2020-11-02T11:57:16.8491183Z [cluster3] - cluster3-worker
  1739. 2020-11-02T11:57:16.8491872Z [cluster3] • Discovering network details ...
  1740. 2020-11-02T11:57:16.8492736Z [cluster3] Discovered network details:
  1741. 2020-11-02T11:57:16.8493501Z [cluster3] Network plugin: weave-net
  1742. 2020-11-02T11:57:16.8494014Z [cluster3] Service CIDRs: [100.3.0.0/16]
  1743. 2020-11-02T11:57:16.8494365Z [cluster3] Cluster CIDRs: [10.3.0.0/16]
  1744. 2020-11-02T11:57:16.8495275Z [cluster3] ✓ Discovering network details
  1745. 2020-11-02T11:57:16.8495880Z [cluster3] • Discovering multi cluster details ...
  1746. 2020-11-02T11:57:16.8496537Z [cluster3] • Validating Globalnet configurations ...
  1747. 2020-11-02T11:57:16.8497558Z [cluster3] ✓ Validating Globalnet configurations
  1748. 2020-11-02T11:57:16.8498323Z [cluster3] ✓ Discovering multi cluster details
  1749. 2020-11-02T11:57:16.8499046Z [cluster3] • Deploying the Submariner operator ...
  1750. 2020-11-02T11:57:16.8499606Z [cluster3] ✓ Deploying the Submariner operator
  1751. 2020-11-02T11:57:16.8500108Z [cluster3] ✓ Created operator CRDs
  1752. 2020-11-02T11:57:16.8500708Z [cluster3] ✓ Created operator namespace: submariner-operator
  1753. 2020-11-02T11:57:16.8501363Z [cluster3] ✓ Created operator service account and role
  1754. 2020-11-02T11:57:16.8502123Z [cluster3] ✓ Created lighthouse service account and role
  1755. 2020-11-02T11:57:16.8502852Z [cluster3] ✓ Created Lighthouse service accounts and roles
  1756. 2020-11-02T11:57:16.8503773Z [cluster3] ✓ Deployed the operator successfully
  1757. 2020-11-02T11:57:16.8520682Z [cluster3] • Creating SA for cluster ...
  1758. 2020-11-02T11:57:16.8521288Z [cluster3] ✓ Creating SA for cluster
  1759. 2020-11-02T11:57:16.8521949Z [cluster3] • Deploying Submariner ...
  1760. 2020-11-02T11:57:16.8522522Z [cluster3] ✓ Deploying Submariner
  1761. 2020-11-02T11:57:16.8523186Z [cluster3] ✓ Submariner is up and running
  1762. 2020-11-02T11:57:16.8523748Z [submariner]$ [cluster3] wait 5649
  1763. 2020-11-02T11:57:16.8525039Z [submariner]$ [cluster3] wait 5646
  1764. 2020-11-02T11:57:25.1084365Z [submariner]$ [cluster1] subctl_install_subm
  1765. 2020-11-02T11:57:25.1086654Z [submariner]$ [cluster1] subctl join --kubeconfig /go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster1 --clusterid cluster1 --repository localhost:5000 --version local --nattport 4500 --ikeport 500 --colorcodes blue --globalnet-cidr --disable-nat --cable-driver libreswan /go/src/github.com/submariner-io/submariner/output/broker-info.subm
  1766. 2020-11-02T11:57:25.1089518Z [cluster1] * /go/src/github.com/submariner-io/submariner/output/broker-info.subm says broker is at: https://172.17.0.9:6443
  1767. 2020-11-02T11:57:25.1090280Z [cluster1] * There are 1 labeled nodes in the cluster:
  1768. 2020-11-02T11:57:25.1090973Z [cluster1] - cluster1-worker
  1769. 2020-11-02T11:57:25.1091721Z [cluster1] • Discovering network details ...
  1770. 2020-11-02T11:57:25.1092155Z [cluster1] Discovered network details:
  1771. 2020-11-02T11:57:25.1092542Z [cluster1] Network plugin: generic
  1772. 2020-11-02T11:57:25.1092890Z [cluster1] Service CIDRs: [100.1.0.0/16]
  1773. 2020-11-02T11:57:25.1093193Z [cluster1] Cluster CIDRs: [10.1.0.0/16]
  1774. 2020-11-02T11:57:25.1093755Z [cluster1] ✓ Discovering network details
  1775. 2020-11-02T11:57:25.1094336Z [cluster1] • Discovering multi cluster details ...
  1776. 2020-11-02T11:57:25.1094954Z [cluster1] • Validating Globalnet configurations ...
  1777. 2020-11-02T11:57:25.1095782Z [cluster1] ✓ Validating Globalnet configurations
  1778. 2020-11-02T11:57:25.1096566Z [cluster1] ✓ Discovering multi cluster details
  1779. 2020-11-02T11:57:25.1097659Z [cluster1] • Deploying the Submariner operator ...
  1780. 2020-11-02T11:57:25.1098316Z [cluster1] ✓ Deploying the Submariner operator
  1781. 2020-11-02T11:57:25.1098894Z [cluster1] ✓ Created operator CRDs
  1782. 2020-11-02T11:57:25.1099627Z [cluster1] ✓ Created operator namespace: submariner-operator
  1783. 2020-11-02T11:57:25.1100567Z [cluster1] ✓ Created operator service account and role
  1784. 2020-11-02T11:57:25.1101378Z [cluster1] ✓ Created lighthouse service account and role
  1785. 2020-11-02T11:57:25.1102041Z [cluster1] ✓ Created Lighthouse service accounts and roles
  1786. 2020-11-02T11:57:25.1102684Z [cluster1] ✓ Deployed the operator successfully
  1787. 2020-11-02T11:57:25.1103406Z [cluster1] • Creating SA for cluster ...
  1788. 2020-11-02T11:57:25.1104086Z [cluster1] ✓ Creating SA for cluster
  1789. 2020-11-02T11:57:25.1104591Z [cluster1] • Deploying Submariner ...
  1790. 2020-11-02T11:57:25.1105084Z [cluster1] ✓ Deploying Submariner
  1791. 2020-11-02T11:57:25.1105599Z [cluster1] ✓ Submariner is up and running
  1792. 2020-11-02T11:57:25.1143375Z [submariner]$ [cluster3] with_context cluster3 with_retries 30 verify_gw_status
  1793. 2020-11-02T11:57:25.1156554Z [submariner]$ [cluster3] with_context cluster3 with_retries 30 verify_gw_status
  1794. 2020-11-02T11:57:25.1172049Z [submariner]$ [cluster3] local cluster=cluster3
  1795. 2020-11-02T11:57:25.1186256Z [submariner]$ [cluster3] local cmnd=with_retries
  1796. 2020-11-02T11:57:25.1301211Z [submariner]$ [cluster3] with_retries 30 verify_gw_status
  1797. 2020-11-02T11:57:25.1411294Z [submariner]$ [cluster3] with_retries
  1798. 2020-11-02T11:57:25.1476577Z [submariner]$ [cluster3] local retries=30
  1799. 2020-11-02T11:57:25.1524912Z [submariner]$ [cluster3] local cmnd=verify_gw_status
  1800. 2020-11-02T11:57:25.1615851Z [submariner]$ [cluster3] iteration=1
  1801. 2020-11-02T11:57:25.1772814Z [submariner]$ [cluster3] wait 5703
  1802. 2020-11-02T11:57:25.1830868Z [submariner]$ [cluster3] verify_gw_status
  1803. 2020-11-02T11:57:25.1831567Z [submariner]$ [cluster3] verify_gw_status
  1804. 2020-11-02T11:57:25.1911258Z [submariner]$ [cluster3] sleep_duration=6
  1805. 2020-11-02T11:57:25.1926841Z [submariner]$ [cluster3] subctl show connections
  1806. 2020-11-02T11:57:25.1982727Z [submariner]$ [cluster3] grep the server could not find the requested resource
  1807. 2020-11-02T11:57:26.7751529Z [submariner]$ [cluster3] subctl show connections
  1808. 2020-11-02T11:57:26.7774578Z [submariner]$ [cluster3] grep connected
  1809. 2020-11-02T11:57:29.8736252Z iter: 1. Clusters not yet connected. sleeping for 6 secs
  1810. 2020-11-02T11:57:29.8764220Z [submariner]$ [cluster3] sleep 6
  1811. 2020-11-02T11:57:35.9106118Z [submariner]$ [cluster3] return 1
  1812. 2020-11-02T11:57:35.9266225Z [submariner]$ [cluster3] wait 5729
  1813. 2020-11-02T11:57:35.9273501Z [submariner]$ [cluster3] verify_gw_status
  1814. 2020-11-02T11:57:35.9392888Z [submariner]$ [cluster3] verify_gw_status
  1815. 2020-11-02T11:57:35.9407287Z [submariner]$ [cluster3] sleep_duration=6
  1816. 2020-11-02T11:57:35.9439840Z [submariner]$ [cluster3] subctl show connections
  1817. 2020-11-02T11:57:35.9711234Z [submariner]$ [cluster3] grep the server could not find the requested resource
  1818. 2020-11-02T11:57:36.9103496Z [submariner]$ [cluster3] subctl show connections
  1819. 2020-11-02T11:57:36.9220226Z [submariner]$ [cluster3] grep connected
  1820. 2020-11-02T11:57:38.2631229Z iter: 2. Clusters not yet connected. sleeping for 6 secs
  1821. 2020-11-02T11:57:38.2650335Z [submariner]$ [cluster3] sleep 6
  1822. 2020-11-02T11:57:44.2681177Z [submariner]$ [cluster3] return 1
  1823. 2020-11-02T11:57:44.2720791Z [submariner]$ [cluster3] wait 5756
  1824. 2020-11-02T11:57:44.2728155Z [submariner]$ [cluster3] verify_gw_status
  1825. 2020-11-02T11:57:44.2742983Z [submariner]$ [cluster3] verify_gw_status
  1826. 2020-11-02T11:57:44.2774729Z [submariner]$ [cluster3] sleep_duration=6
  1827. 2020-11-02T11:57:44.2855300Z [submariner]$ [cluster3] subctl show connections
  1828. 2020-11-02T11:57:44.2901788Z [submariner]$ [cluster3] grep the server could not find the requested resource
  1829. 2020-11-02T11:57:46.0779799Z [submariner]$ [cluster3] subctl show connections
  1830. 2020-11-02T11:57:46.0780549Z [submariner]$ [cluster3] grep connected
  1831. 2020-11-02T11:57:46.4883501Z iter: 3. Clusters not yet connected. sleeping for 6 secs
  1832. 2020-11-02T11:57:46.4898794Z [submariner]$ [cluster3] sleep 6
  1833. 2020-11-02T11:57:52.4924498Z [submariner]$ [cluster3] return 1
  1834. 2020-11-02T11:57:52.4963506Z [submariner]$ [cluster3] wait 5782
  1835. 2020-11-02T11:57:52.4975141Z [submariner]$ [cluster3] verify_gw_status
  1836. 2020-11-02T11:57:52.4989510Z [submariner]$ [cluster3] verify_gw_status
  1837. 2020-11-02T11:57:52.5000546Z [submariner]$ [cluster3] sleep_duration=6
  1838. 2020-11-02T11:57:52.5012725Z [submariner]$ [cluster3] subctl show connections
  1839. 2020-11-02T11:57:52.5119936Z [submariner]$ [cluster3] grep the server could not find the requested resource
  1840. 2020-11-02T11:57:53.0993923Z [submariner]$ [cluster3] subctl show connections
  1841. 2020-11-02T11:57:53.1056535Z [submariner]$ [cluster3] grep connected
  1842. 2020-11-02T11:57:55.3234528Z iter: 4. Clusters not yet connected. sleeping for 6 secs
  1843. 2020-11-02T11:57:55.3246722Z [submariner]$ [cluster3] sleep 6
  1844. 2020-11-02T11:58:01.3271715Z [submariner]$ [cluster3] return 1
  1845. 2020-11-02T11:58:01.3347083Z [submariner]$ [cluster3] wait 5808
  1846. 2020-11-02T11:58:01.3376611Z [submariner]$ [cluster3] verify_gw_status
  1847. 2020-11-02T11:58:01.3435419Z [submariner]$ [cluster3] verify_gw_status
  1848. 2020-11-02T11:58:01.3477106Z [submariner]$ [cluster3] sleep_duration=6
  1849. 2020-11-02T11:58:01.3509844Z [submariner]$ [cluster3] subctl show connections
  1850. 2020-11-02T11:58:01.3657654Z [submariner]$ [cluster3] grep the server could not find the requested resource
  1851. 2020-11-02T11:58:01.7342006Z [submariner]$ [cluster3] subctl show connections
  1852. 2020-11-02T11:58:01.7414306Z [submariner]$ [cluster3] grep connected
  1853. 2020-11-02T11:58:02.5003499Z iter: 5. Clusters not yet connected. sleeping for 6 secs
  1854. 2020-11-02T11:58:02.5022365Z [submariner]$ [cluster3] sleep 6
  1855. 2020-11-02T11:58:08.5058750Z [submariner]$ [cluster3] return 1
  1856. 2020-11-02T11:58:08.5096200Z [submariner]$ [cluster3] wait 5836
  1857. 2020-11-02T11:58:08.5107072Z [submariner]$ [cluster3] verify_gw_status
  1858. 2020-11-02T11:58:08.5132631Z [submariner]$ [cluster3] verify_gw_status
  1859. 2020-11-02T11:58:08.5147795Z [submariner]$ [cluster3] sleep_duration=6
  1860. 2020-11-02T11:58:08.5158735Z [submariner]$ [cluster3] subctl show connections
  1861. 2020-11-02T11:58:08.5225294Z [submariner]$ [cluster3] grep the server could not find the requested resource
  1862. 2020-11-02T11:58:09.6643021Z [submariner]$ [cluster3] subctl show connections
  1863. 2020-11-02T11:58:09.6657387Z [submariner]$ [cluster3] grep connected
  1864. 2020-11-02T11:58:10.4355468Z iter: 6. Clusters not yet connected. sleeping for 6 secs
  1865. 2020-11-02T11:58:10.4369466Z [submariner]$ [cluster3] sleep 6
  1866. 2020-11-02T11:58:16.4974186Z [submariner]$ [cluster3] return 1
  1867. 2020-11-02T11:58:16.5064613Z [submariner]$ [cluster3] wait 5862
  1868. 2020-11-02T11:58:16.5117102Z [submariner]$ [cluster3] verify_gw_status
  1869. 2020-11-02T11:58:16.5131520Z [submariner]$ [cluster3] verify_gw_status
  1870. 2020-11-02T11:58:16.5165659Z [submariner]$ [cluster3] sleep_duration=6
  1871. 2020-11-02T11:58:16.5180241Z [submariner]$ [cluster3] subctl show connections
  1872. 2020-11-02T11:58:16.5202926Z [submariner]$ [cluster3] grep the server could not find the requested resource
  1873. 2020-11-02T11:58:17.5520293Z [submariner]$ [cluster3] subctl show connections
  1874. 2020-11-02T11:58:17.5544545Z [submariner]$ [cluster3] grep connected
  1875. 2020-11-02T11:58:18.7429119Z iter: 7. Clusters not yet connected. sleeping for 6 secs
  1876. 2020-11-02T11:58:18.7449173Z [submariner]$ [cluster3] sleep 6
  1877. 2020-11-02T11:58:24.7665772Z [submariner]$ [cluster3] return 1
  1878. 2020-11-02T11:58:24.7704091Z [submariner]$ [cluster3] wait 5888
  1879. 2020-11-02T11:58:24.7712647Z [submariner]$ [cluster3] verify_gw_status
  1880. 2020-11-02T11:58:24.7723258Z [submariner]$ [cluster3] verify_gw_status
  1881. 2020-11-02T11:58:24.7787893Z [submariner]$ [cluster3] sleep_duration=6
  1882. 2020-11-02T11:58:24.7803312Z [submariner]$ [cluster3] subctl show connections
  1883. 2020-11-02T11:58:24.7822062Z [submariner]$ [cluster3] grep the server could not find the requested resource
  1884. 2020-11-02T11:58:25.0707681Z [submariner]$ [cluster3] subctl show connections
  1885. 2020-11-02T11:58:25.0719903Z [submariner]$ [cluster3] grep connected
  1886. 2020-11-02T11:58:25.3546430Z iter: 8. Clusters not yet connected. sleeping for 6 secs
  1887. 2020-11-02T11:58:25.3585579Z [submariner]$ [cluster3] sleep 6
  1888. 2020-11-02T11:58:31.3616514Z [submariner]$ [cluster3] return 1
  1889. 2020-11-02T11:58:31.3680969Z [submariner]$ [cluster3] verify_gw_status
  1890. 2020-11-02T11:58:31.3681628Z [submariner]$ [cluster3] wait 5914
  1891. 2020-11-02T11:58:31.3741827Z [submariner]$ [cluster3] verify_gw_status
  1892. 2020-11-02T11:58:31.3757797Z [submariner]$ [cluster3] sleep_duration=6
  1893. 2020-11-02T11:58:31.3777056Z [submariner]$ [cluster3] subctl show connections
  1894. 2020-11-02T11:58:31.3796099Z [submariner]$ [cluster3] grep the server could not find the requested resource
  1895. 2020-11-02T11:58:31.6426838Z [submariner]$ [cluster3] subctl show connections
  1896. 2020-11-02T11:58:31.6442295Z [submariner]$ [cluster3] grep connected
  1897. 2020-11-02T11:58:31.9338375Z cluster3-worker cluster3 172.17.0.7 libreswan 100.3.0.0/16, 10.3.0.0/16 connected
  1898. 2020-11-02T11:58:31.9339291Z cluster1-worker cluster1 172.17.0.6 libreswan 100.1.0.0/16, 10.1.0.0/16 connected
  1899. 2020-11-02T11:58:31.9340176Z cluster2-worker cluster2 172.17.0.4 libreswan 100.2.0.0/16, 10.2.0.0/16 connected
  1900. 2020-11-02T11:58:31.9340889Z cluster1-worker cluster1 172.17.0.6 libreswan 100.1.0.0/16, 10.1.0.0/16 connected
  1901. 2020-11-02T11:58:31.9341598Z cluster2-worker cluster2 172.17.0.4 libreswan 100.2.0.0/16, 10.2.0.0/16 connected
  1902. 2020-11-02T11:58:31.9342296Z cluster3-worker cluster3 172.17.0.7 libreswan 100.3.0.0/16, 10.3.0.0/16 connected
  1903. 2020-11-02T11:58:31.9353425Z [submariner]$ [cluster3] return 0
  1904. 2020-11-02T11:58:31.9367981Z [submariner]$ [cluster3] return 0
  1905. 2020-11-02T11:58:31.9380854Z [submariner]$ [cluster3] with_context cluster3 connectivity_tests cluster2
  1906. 2020-11-02T11:58:31.9393728Z [submariner]$ [cluster3] with_context cluster3 connectivity_tests cluster2
  1907. 2020-11-02T11:58:31.9412696Z [submariner]$ [cluster3] local cluster=cluster3
  1908. 2020-11-02T11:58:31.9415860Z [submariner]$ [cluster3] local cmnd=connectivity_tests
  1909. 2020-11-02T11:58:31.9426180Z [submariner]$ [cluster3] connectivity_tests cluster2
  1910. 2020-11-02T11:58:31.9436531Z [submariner]$ [cluster3] connectivity_tests
  1911. 2020-11-02T11:58:31.9447094Z [submariner]$ [cluster3] target_cluster=cluster2
  1912. 2020-11-02T11:58:31.9458128Z [submariner]$ [cluster3] deploy_resource /opt/shipyard/scripts/resources/netshoot.yaml
  1913. 2020-11-02T11:58:31.9474528Z [submariner]$ [cluster3] deploy_resource /opt/shipyard/scripts/resources/netshoot.yaml
  1914. 2020-11-02T11:58:31.9491203Z [submariner]$ [cluster3] local resource_file=/opt/shipyard/scripts/resources/netshoot.yaml
  1915. 2020-11-02T11:58:31.9505955Z [submariner]$ [cluster3] local resource_name
  1916. 2020-11-02T11:58:31.9543257Z [submariner]$ [cluster3] resource_name=netshoot
  1917. 2020-11-02T11:58:31.9562892Z [submariner]$ [cluster3] basename /opt/shipyard/scripts/resources/netshoot.yaml .yaml
  1918. 2020-11-02T11:58:31.9589193Z [submariner]$ [cluster3] kubectl apply -f /opt/shipyard/scripts/resources/netshoot.yaml
  1919. 2020-11-02T11:58:31.9602205Z [submariner]$ [cluster3] kubectl apply -f /opt/shipyard/scripts/resources/netshoot.yaml
  1920. 2020-11-02T11:58:31.9613021Z [submariner]$ [cluster3] command kubectl --context=cluster3 apply -f /opt/shipyard/scripts/resources/netshoot.yaml
  1921. 2020-11-02T11:58:31.9625818Z [submariner]$ [cluster3] kubectl --context=cluster3 apply -f /opt/shipyard/scripts/resources/netshoot.yaml
  1922. 2020-11-02T11:58:32.4301492Z deployment.apps/netshoot created
  1923. 2020-11-02T11:58:32.4386856Z Waiting for netshoot pods to be ready.
  1924. 2020-11-02T11:58:32.4417812Z [submariner]$ [cluster3] kubectl rollout status deploy/netshoot --timeout=5m
  1925. 2020-11-02T11:58:32.4433497Z [submariner]$ [cluster3] kubectl rollout status deploy/netshoot --timeout=5m
  1926. 2020-11-02T11:58:32.4453137Z [submariner]$ [cluster3] command kubectl --context=cluster3 rollout status deploy/netshoot --timeout=5m
  1927. 2020-11-02T11:58:32.4507509Z [submariner]$ [cluster3] kubectl --context=cluster3 rollout status deploy/netshoot --timeout=5m
  1928. 2020-11-02T11:58:32.8953599Z Waiting for deployment "netshoot" rollout to finish: 0 of 1 updated replicas are available...
  1929. 2020-11-02T11:58:34.5634303Z deployment "netshoot" successfully rolled out
  1930. 2020-11-02T11:58:34.5687669Z [submariner]$ [cluster3] with_context cluster2 deploy_resource /opt/shipyard/scripts/resources/nginx-demo.yaml
  1931. 2020-11-02T11:58:34.5699892Z [submariner]$ [cluster3] with_context cluster2 deploy_resource /opt/shipyard/scripts/resources/nginx-demo.yaml
  1932. 2020-11-02T11:58:34.5713586Z [submariner]$ [cluster3] local cluster=cluster2
  1933. 2020-11-02T11:58:34.5725820Z [submariner]$ [cluster2] local cmnd=deploy_resource
  1934. 2020-11-02T11:58:34.5738258Z [submariner]$ [cluster2] deploy_resource /opt/shipyard/scripts/resources/nginx-demo.yaml
  1935. 2020-11-02T11:58:34.5755391Z [submariner]$ [cluster2] deploy_resource
  1936. 2020-11-02T11:58:34.5771215Z [submariner]$ [cluster2] local resource_file=/opt/shipyard/scripts/resources/nginx-demo.yaml
  1937. 2020-11-02T11:58:34.5792906Z [submariner]$ [cluster2] local resource_name
  1938. 2020-11-02T11:58:34.5832406Z [submariner]$ [cluster2] resource_name=nginx-demo
  1939. 2020-11-02T11:58:34.5869524Z [submariner]$ [cluster2] basename /opt/shipyard/scripts/resources/nginx-demo.yaml .yaml
  1940. 2020-11-02T11:58:34.5893403Z [submariner]$ [cluster2] kubectl apply -f /opt/shipyard/scripts/resources/nginx-demo.yaml
  1941. 2020-11-02T11:58:34.5905882Z [submariner]$ [cluster2] kubectl apply -f /opt/shipyard/scripts/resources/nginx-demo.yaml
  1942. 2020-11-02T11:58:34.5919137Z [submariner]$ [cluster2] command kubectl --context=cluster2 apply -f /opt/shipyard/scripts/resources/nginx-demo.yaml
  1943. 2020-11-02T11:58:34.5933645Z [submariner]$ [cluster2] kubectl --context=cluster2 apply -f /opt/shipyard/scripts/resources/nginx-demo.yaml
  1944. 2020-11-02T11:58:35.3145898Z deployment.apps/nginx-demo created
  1945. 2020-11-02T11:58:35.3514985Z service/nginx-demo created
  1946. 2020-11-02T11:58:35.3662374Z Waiting for nginx-demo pods to be ready.
  1947. 2020-11-02T11:58:35.3679507Z [submariner]$ [cluster2] kubectl rollout status deploy/nginx-demo --timeout=5m
  1948. 2020-11-02T11:58:35.3697974Z [submariner]$ [cluster2] kubectl rollout status deploy/nginx-demo --timeout=5m
  1949. 2020-11-02T11:58:35.3722857Z [submariner]$ [cluster2] command kubectl --context=cluster2 rollout status deploy/nginx-demo --timeout=5m
  1950. 2020-11-02T11:58:35.3739341Z [submariner]$ [cluster2] kubectl --context=cluster2 rollout status deploy/nginx-demo --timeout=5m
  1951. 2020-11-02T11:58:36.0445254Z Waiting for deployment "nginx-demo" rollout to finish: 0 of 2 updated replicas are available...
  1952. 2020-11-02T11:58:39.0844266Z Waiting for deployment "nginx-demo" rollout to finish: 1 of 2 updated replicas are available...
  1953. 2020-11-02T11:58:40.1228774Z deployment "nginx-demo" successfully rolled out
  1954. 2020-11-02T11:58:40.1279775Z [submariner]$ [cluster3] local netshoot_pod nginx_svc_ip
  1955. 2020-11-02T11:58:40.4932564Z [submariner]$ [cluster3] netshoot_pod=netshoot-789f6cf54f-vsgqf
  1956. 2020-11-02T11:58:40.4973258Z [submariner]$ [cluster3] kubectl get pods -l app=netshoot
  1957. 2020-11-02T11:58:40.4991530Z [submariner]$ [cluster3] awk FNR == 2 {print $1}
  1958. 2020-11-02T11:58:40.4999214Z [submariner]$ [cluster3] kubectl get pods -l app=netshoot
  1959. 2020-11-02T11:58:40.5015238Z [submariner]$ [cluster3] command kubectl --context=cluster3 get pods -l app=netshoot
  1960. 2020-11-02T11:58:40.5040054Z [submariner]$ [cluster3] kubectl --context=cluster3 get pods -l app=netshoot
  1961. 2020-11-02T11:58:41.1460411Z [submariner]$ [cluster3] nginx_svc_ip=100.2.197.33
  1962. 2020-11-02T11:58:41.1489758Z [submariner]$ [cluster3] with_context cluster2 get_svc_ip nginx-demo
  1963. 2020-11-02T11:58:41.1502733Z [submariner]$ [cluster3] with_context cluster2 get_svc_ip nginx-demo
  1964. 2020-11-02T11:58:41.1517606Z [submariner]$ [cluster3] local cluster=cluster2
  1965. 2020-11-02T11:58:41.1530107Z [submariner]$ [cluster2] local cmnd=get_svc_ip
  1966. 2020-11-02T11:58:41.1541777Z [submariner]$ [cluster2] get_svc_ip nginx-demo
  1967. 2020-11-02T11:58:41.1556009Z [submariner]$ [cluster2] get_svc_ip
  1968. 2020-11-02T11:58:41.1570941Z [submariner]$ [cluster2] local svc_name=nginx-demo
  1969. 2020-11-02T11:58:41.1596145Z [submariner]$ [cluster2] local svc_ip
  1970. 2020-11-02T11:58:41.5082873Z [submariner]$ [cluster2] svc_ip=100.2.197.33
  1971. 2020-11-02T11:58:41.5114501Z [submariner]$ [cluster2] kubectl --context=cluster2 get svc -l app=nginx-demo
  1972. 2020-11-02T11:58:41.5140291Z [submariner]$ [cluster2] kubectl --context=cluster2 get svc -l app=nginx-demo
  1973. 2020-11-02T11:58:41.5154286Z [submariner]$ [cluster2] awk FNR == 2 {print $3}
  1974. 2020-11-02T11:58:41.5172033Z [submariner]$ [cluster2] command kubectl --context=cluster2 get svc -l app=nginx-demo
  1975. 2020-11-02T11:58:41.5185197Z [submariner]$ [cluster2] kubectl --context=cluster2 get svc -l app=nginx-demo
  1976. 2020-11-02T11:58:41.8081459Z [submariner]$ [cluster3] with_retries 5 test_connection netshoot-789f6cf54f-vsgqf 100.2.197.33
  1977. 2020-11-02T11:58:41.8095434Z [submariner]$ [cluster3] with_retries 5 test_connection netshoot-789f6cf54f-vsgqf 100.2.197.33
  1978. 2020-11-02T11:58:41.8108218Z [submariner]$ [cluster3] local retries=5
  1979. 2020-11-02T11:58:41.8118807Z [submariner]$ [cluster3] local cmnd=test_connection
  1980. 2020-11-02T11:58:41.8131827Z [submariner]$ [cluster3] iteration=1
  1981. 2020-11-02T11:58:41.8157065Z [submariner]$ [cluster3] wait 6080
  1982. 2020-11-02T11:58:41.8171824Z [submariner]$ [cluster3] test_connection netshoot-789f6cf54f-vsgqf 100.2.197.33
  1983. 2020-11-02T11:58:41.8184184Z [submariner]$ [cluster3] test_connection
  1984. 2020-11-02T11:58:41.8226074Z [submariner]$ [cluster3] local source_pod=netshoot-789f6cf54f-vsgqf
  1985. 2020-11-02T11:58:41.8243018Z [submariner]$ [cluster3] local target_address=100.2.197.33
  1986. 2020-11-02T11:58:41.8243811Z Attempting connectivity between clusters - netshoot-789f6cf54f-vsgqf --> 100.2.197.33
  1987. 2020-11-02T11:58:41.8256033Z [submariner]$ [cluster3] wait_time=30
  1988. 2020-11-02T11:58:41.8306928Z [submariner]$ [cluster3] start_time=1604318321
  1989. 2020-11-02T11:58:41.8320235Z [submariner]$ [cluster3] date +%s
  1990. 2020-11-02T11:58:41.8352132Z [submariner]$ [cluster3] kubectl exec netshoot-789f6cf54f-vsgqf -- curl --output /dev/null -m 30 --silent --head --fail 100.2.197.33
  1991. 2020-11-02T11:58:41.8361550Z [submariner]$ [cluster3] kubectl exec netshoot-789f6cf54f-vsgqf -- curl --output /dev/null -m 30 --silent --head --fail 100.2.197.33
  1992. 2020-11-02T11:58:41.8406999Z [submariner]$ [cluster3] command kubectl --context=cluster3 exec netshoot-789f6cf54f-vsgqf -- curl --output /dev/null -m 30 --silent --head --fail 100.2.197.33
  1993. 2020-11-02T11:58:41.8464338Z [submariner]$ [cluster3] kubectl --context=cluster3 exec netshoot-789f6cf54f-vsgqf -- curl --output /dev/null -m 30 --silent --head --fail 100.2.197.33
  1994. 2020-11-02T11:58:42.3031640Z Connection test was successful!
  1995. 2020-11-02T11:58:42.3044868Z [submariner]$ [cluster3] return 0
  1996. 2020-11-02T11:58:42.3057030Z [submariner]$ [cluster3] remove_resource /opt/shipyard/scripts/resources/netshoot.yaml
  1997. 2020-11-02T11:58:42.3069401Z [submariner]$ [cluster3] remove_resource /opt/shipyard/scripts/resources/netshoot.yaml
  1998. 2020-11-02T11:58:42.3087229Z [submariner]$ [cluster3] local resource_file=/opt/shipyard/scripts/resources/netshoot.yaml
  1999. 2020-11-02T11:58:42.3102377Z [submariner]$ [cluster3] kubectl delete -f /opt/shipyard/scripts/resources/netshoot.yaml
  2000. 2020-11-02T11:58:42.3119888Z [submariner]$ [cluster3] kubectl delete -f /opt/shipyard/scripts/resources/netshoot.yaml
  2001. 2020-11-02T11:58:42.3130271Z [submariner]$ [cluster3] command kubectl --context=cluster3 delete -f /opt/shipyard/scripts/resources/netshoot.yaml
  2002. 2020-11-02T11:58:42.3142042Z [submariner]$ [cluster3] kubectl --context=cluster3 delete -f /opt/shipyard/scripts/resources/netshoot.yaml
  2003. 2020-11-02T11:58:42.6005023Z deployment.apps "netshoot" deleted
  2004. 2020-11-02T11:58:42.6080354Z [submariner]$ [cluster3] with_context cluster2 remove_resource /opt/shipyard/scripts/resources/nginx-demo.yaml
  2005. 2020-11-02T11:58:42.6095275Z [submariner]$ [cluster3] with_context cluster2 remove_resource /opt/shipyard/scripts/resources/nginx-demo.yaml
  2006. 2020-11-02T11:58:42.6113430Z [submariner]$ [cluster3] local cluster=cluster2
  2007. 2020-11-02T11:58:42.6127820Z [submariner]$ [cluster2] local cmnd=remove_resource
  2008. 2020-11-02T11:58:42.6140776Z [submariner]$ [cluster2] remove_resource /opt/shipyard/scripts/resources/nginx-demo.yaml
  2009. 2020-11-02T11:58:42.6155708Z [submariner]$ [cluster2] remove_resource
  2010. 2020-11-02T11:58:42.6169739Z [submariner]$ [cluster2] local resource_file=/opt/shipyard/scripts/resources/nginx-demo.yaml
  2011. 2020-11-02T11:58:42.6183586Z [submariner]$ [cluster2] kubectl delete -f /opt/shipyard/scripts/resources/nginx-demo.yaml
  2012. 2020-11-02T11:58:42.6203533Z [submariner]$ [cluster2] kubectl delete -f /opt/shipyard/scripts/resources/nginx-demo.yaml
  2013. 2020-11-02T11:58:42.6224958Z [submariner]$ [cluster2] command kubectl --context=cluster2 delete -f /opt/shipyard/scripts/resources/nginx-demo.yaml
  2014. 2020-11-02T11:58:42.6238749Z [submariner]$ [cluster2] kubectl --context=cluster2 delete -f /opt/shipyard/scripts/resources/nginx-demo.yaml
  2015. 2020-11-02T11:58:42.9374719Z deployment.apps "nginx-demo" deleted
  2016. 2020-11-02T11:58:43.0148898Z service "nginx-demo" deleted
  2017. 2020-11-02T11:58:43.0249371Z make[1]: Leaving directory '/go/src/github.com/submariner-io/submariner'
  2018. 2020-11-02T11:58:43.0359795Z [submariner]$ declare_kubeconfig
  2019. 2020-11-02T11:58:43.0374697Z [submariner]$ declare_kubeconfig
  2020. 2020-11-02T11:58:43.0387220Z [submariner]$ source /opt/shipyard/scripts/lib/kubecfg
  2021. 2020-11-02T11:58:43.0407511Z [submariner]$ export KUBECONFIG
  2022. 2020-11-02T11:58:43.0595568Z [submariner]$ KUBECONFIG=/go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster2:/go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster3:/go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster1:
  2023. 2020-11-02T11:58:43.0633366Z [submariner]$ find /go/src/github.com/submariner-io/submariner/output/kubeconfigs -type f -printf %p:
  2024. 2020-11-02T11:58:43.0761924Z [submariner]$ test_with_e2e_tests
  2025. 2020-11-02T11:58:43.0797236Z [submariner]$ test_with_e2e_tests
  2026. 2020-11-02T11:58:43.0858396Z [submariner]$ cd /go/src/github.com/submariner-io/submariner/test/e2e
  2027. 2020-11-02T11:58:43.0914658Z [e2e]$ go test -v -timeout 30m -args -ginkgo.v -ginkgo.randomizeAllSpecs -ginkgo.trace -submariner-namespace submariner-operator -dp-context cluster2 -dp-context cluster3 -dp-context cluster1 -ginkgo.reportPassed -test.timeout 15m -ginkgo.focus \[.*\] -ginkgo.reportFile /go/src/github.com/submariner-io/submariner/output/e2e-junit.xml
  2028. 2020-11-02T11:58:43.0931898Z [e2e]$ tee /go/src/github.com/submariner-io/submariner/output/e2e-tests.log
  2029. 2020-11-02T11:58:43.0967947Z [e2e]$ generate_context_flags
  2030. 2020-11-02T11:58:43.0978246Z [e2e]$ generate_context_flags
  2031. 2020-11-02T11:58:43.0993124Z [e2e]$ [cluster2] printf -dp-context cluster2
  2032. 2020-11-02T11:58:43.1007648Z [e2e]$ [cluster3] printf -dp-context cluster3
  2033. 2020-11-02T11:58:43.1021420Z [e2e]$ [cluster1] printf -dp-context cluster1
  2034. 2020-11-02T11:59:56.1598074Z === RUN TestE2E
  2035. 2020-11-02T11:59:56.1709888Z Running Suite: Submariner E2E suite
  2036. 2020-11-02T11:59:56.1710528Z ===================================
  2037. 2020-11-02T11:59:56.1711952Z Random Seed: 1604318395 - Will randomize all specs
  2038. 2020-11-02T11:59:56.1712719Z Will run 19 of 20 specs
  2039. 2020-11-02T11:59:56.1713053Z
  2040. 2020-11-02T11:59:56.1728858Z STEP: Creating kubernetes clients
  2041. 2020-11-02T11:59:56.2250411Z STEP: Creating submariner clients
  2042. 2020-11-02T11:59:56.2492292Z [dataplane-globalnet] Basic TCP connectivity tests across overlapping clusters without discovery when a pod with HostNetworking connects via TCP to the globalIP of a remote service when the pod is on a gateway and the remote service is not on a gateway
  2043. 2020-11-02T11:59:56.2496116Z should have sent the expected data from the pod to the other pod
  2044. 2020-11-02T11:59:56.2497518Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:15
  2045. 2020-11-02T11:59:56.2498860Z STEP: Creating namespace objects with basename "dataplane-gn-conn-nd"
  2046. 2020-11-02T11:59:56.2632337Z STEP: Generated namespace "e2e-tests-dataplane-gn-conn-nd-pxv8w" in cluster "cluster2" to execute the tests in
  2047. 2020-11-02T11:59:56.2634694Z STEP: Creating namespace "e2e-tests-dataplane-gn-conn-nd-pxv8w" in cluster "cluster3"
  2048. 2020-11-02T11:59:56.3543833Z STEP: Creating namespace "e2e-tests-dataplane-gn-conn-nd-pxv8w" in cluster "cluster1"
  2049. 2020-11-02T11:59:56.3701884Z Nov 2 11:59:56.369: INFO: Globalnet is not enabled, skipping the test...
  2050. 2020-11-02T11:59:56.3713459Z STEP: Deleting namespace "e2e-tests-dataplane-gn-conn-nd-pxv8w" on cluster "cluster2"
  2051. 2020-11-02T11:59:56.4492082Z STEP: Deleting namespace "e2e-tests-dataplane-gn-conn-nd-pxv8w" on cluster "cluster3"
  2052. 2020-11-02T11:59:56.4572870Z STEP: Deleting namespace "e2e-tests-dataplane-gn-conn-nd-pxv8w" on cluster "cluster1"
  2053. 2020-11-02T11:59:56.4681232Z
  2054. 2020-11-02T11:59:56.4686424Z S [SKIPPING] [0.219 seconds]
  2055. 2020-11-02T11:59:56.4688247Z [dataplane-globalnet] Basic TCP connectivity tests across overlapping clusters without discovery
  2056. 2020-11-02T11:59:56.4689349Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:9
  2057. 2020-11-02T11:59:56.4690241Z when a pod with HostNetworking connects via TCP to the globalIP of a remote service
  2058. 2020-11-02T11:59:56.4691306Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:56
  2059. 2020-11-02T11:59:56.4692372Z when the pod is on a gateway and the remote service is not on a gateway
  2060. 2020-11-02T11:59:56.4693453Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:66
  2061. 2020-11-02T11:59:56.4694229Z should have sent the expected data from the pod to the other pod [It]
  2062. 2020-11-02T11:59:56.4695000Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:15
  2063. 2020-11-02T11:59:56.4695433Z
  2064. 2020-11-02T11:59:56.4695909Z Nov 2 11:59:56.370: Globalnet is not enabled, skipping the test...
  2065. 2020-11-02T11:59:56.4696330Z
  2066. 2020-11-02T11:59:56.4697028Z /go/src/github.com/submariner-io/submariner/vendor/github.com/submariner-io/shipyard/test/e2e/framework/logging.go:42
  2067. 2020-11-02T11:59:56.4698136Z ------------------------------
  2068. 2020-11-02T11:59:56.4703874Z [dataplane] Basic TCP connectivity tests across clusters without discovery when a pod with HostNetworking connects via TCP to a remote pod when the pod is not on a gateway and the remote pod is not on a gateway
  2069. 2020-11-02T11:59:56.4705095Z should have sent the expected data from the pod to the other pod
  2070. 2020-11-02T11:59:56.4705854Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_pod_connectivity.go:15
  2071. 2020-11-02T11:59:56.4706645Z STEP: Creating namespace objects with basename "dataplane-conn-nd"
  2072. 2020-11-02T11:59:56.4774377Z STEP: Generated namespace "e2e-tests-dataplane-conn-nd-lj9p6" in cluster "cluster2" to execute the tests in
  2073. 2020-11-02T11:59:56.4775590Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-lj9p6" in cluster "cluster3"
  2074. 2020-11-02T11:59:56.4844977Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-lj9p6" in cluster "cluster1"
  2075. 2020-11-02T11:59:56.4954208Z STEP: Creating a listener pod in cluster "cluster3", which will wait for a handshake over TCP
  2076. 2020-11-02T12:00:02.5985600Z Nov 2 12:00:02.598: INFO: Will send traffic to IP: 10.3.128.3
  2077. 2020-11-02T12:00:02.5986760Z STEP: Creating a connector pod in cluster "cluster2", which will attempt the specific UUID handshake over TCP
  2078. 2020-11-02T12:00:02.6199989Z STEP: Waiting for the connector pod "tcp-check-podjfbr4" to exit, returning what connector sent
  2079. 2020-11-02T12:00:07.6341567Z Nov 2 12:00:07.633: INFO: Pod "tcp-check-podjfbr4" output:
  2080. 2020-11-02T12:00:07.6342052Z 10.3.128.3 (10.3.128.3:1234) open
  2081. 2020-11-02T12:00:07.6342688Z [dataplane] listener says a7244c39-c52d-4477-8493-4067284848ea
  2082. 2020-11-02T12:00:07.6342993Z
  2083. 2020-11-02T12:00:07.6343753Z STEP: Waiting for the listener pod "tcp-check-listener9ddd8" to exit, returning what listener sent
  2084. 2020-11-02T12:00:07.6378711Z Nov 2 12:00:07.637: INFO: Pod "tcp-check-listener9ddd8" output:
  2085. 2020-11-02T12:00:07.6379406Z listening on 0.0.0.0:1234 ...
  2086. 2020-11-02T12:00:07.6379854Z connect to 10.3.128.3:1234 from 10.2.128.0:38603 (10.2.128.0:38603)
  2087. 2020-11-02T12:00:07.6380665Z [dataplane] connector says 5e255cdc-f7d7-42af-aa6d-57254d1c2e50
  2088. 2020-11-02T12:00:07.6381314Z
  2089. 2020-11-02T12:00:07.6381683Z Nov 2 12:00:07.637: INFO: Connector pod has IP: 172.17.0.11
  2090. 2020-11-02T12:00:07.6382623Z STEP: Verifying that the listener got the connector's data and the connector got the listener's data
  2091. 2020-11-02T12:00:07.6383835Z STEP: Verifying that globalIP annotation does not exist on the connector POD
  2092. 2020-11-02T12:00:07.6384979Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-lj9p6" on cluster "cluster2"
  2093. 2020-11-02T12:00:07.6445275Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-lj9p6" on cluster "cluster3"
  2094. 2020-11-02T12:00:07.6492435Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-lj9p6" on cluster "cluster1"
  2095. 2020-11-02T12:00:07.6556751Z •
  2096. 2020-11-02T12:00:07.6557348Z ------------------------------
  2097. 2020-11-02T12:00:07.6558333Z STEP: Creating namespace objects with basename "dataplane-conn-nd"
  2098. 2020-11-02T12:00:07.6559925Z STEP: Generated namespace "e2e-tests-dataplane-conn-nd-lj9p6" in cluster "cluster2" to execute the tests in
  2099. 2020-11-02T12:00:07.6561467Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-lj9p6" in cluster "cluster3"
  2100. 2020-11-02T12:00:07.6563007Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-lj9p6" in cluster "cluster1"
  2101. 2020-11-02T12:00:07.6565307Z STEP: Creating a listener pod in cluster "cluster3", which will wait for a handshake over TCP
  2102. 2020-11-02T12:00:07.6566173Z Nov 2 12:00:02.598: INFO: Will send traffic to IP: 10.3.128.3
  2103. 2020-11-02T12:00:07.6567240Z STEP: Creating a connector pod in cluster "cluster2", which will attempt the specific UUID handshake over TCP
  2104. 2020-11-02T12:00:07.6568439Z STEP: Waiting for the connector pod "tcp-check-podjfbr4" to exit, returning what connector sent
  2105. 2020-11-02T12:00:07.6569509Z Nov 2 12:00:07.633: INFO: Pod "tcp-check-podjfbr4" output:
  2106. 2020-11-02T12:00:07.6570019Z 10.3.128.3 (10.3.128.3:1234) open
  2107. 2020-11-02T12:00:07.6570645Z [dataplane] listener says a7244c39-c52d-4477-8493-4067284848ea
  2108. 2020-11-02T12:00:07.6571072Z
  2109. 2020-11-02T12:00:07.6571899Z STEP: Waiting for the listener pod "tcp-check-listener9ddd8" to exit, returning what listener sent
  2110. 2020-11-02T12:00:07.6572839Z Nov 2 12:00:07.637: INFO: Pod "tcp-check-listener9ddd8" output:
  2111. 2020-11-02T12:00:07.6573412Z listening on 0.0.0.0:1234 ...
  2112. 2020-11-02T12:00:07.6573742Z connect to 10.3.128.3:1234 from 10.2.128.0:38603 (10.2.128.0:38603)
  2113. 2020-11-02T12:00:07.6574878Z [dataplane] connector says 5e255cdc-f7d7-42af-aa6d-57254d1c2e50
  2114. 2020-11-02T12:00:07.6575474Z
  2115. 2020-11-02T12:00:07.6576117Z Nov 2 12:00:07.637: INFO: Connector pod has IP: 172.17.0.11
  2116. 2020-11-02T12:00:07.6577245Z STEP: Verifying that the listener got the connector's data and the connector got the listener's data
  2117. 2020-11-02T12:00:07.6578169Z STEP: Verifying that globalIP annotation does not exist on the connector POD
  2118. 2020-11-02T12:00:07.6579400Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-lj9p6" on cluster "cluster2"
  2119. 2020-11-02T12:00:07.6580820Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-lj9p6" on cluster "cluster3"
  2120. 2020-11-02T12:00:07.6582339Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-lj9p6" on cluster "cluster1"
  2121. 2020-11-02T12:00:07.6583021Z
  2122. 2020-11-02T12:00:07.6583211Z
  2123. 2020-11-02T12:00:07.6584281Z [dataplane] Basic TCP connectivity tests across clusters without discovery when a pod connects via TCP to a remote pod when the pod is on a gateway and the remote pod is on a gateway
  2124. 2020-11-02T12:00:07.6585544Z should have sent the expected data from the pod to the other pod
  2125. 2020-11-02T12:00:07.6586595Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_pod_connectivity.go:15
  2126. 2020-11-02T12:00:07.6587882Z STEP: Creating namespace objects with basename "dataplane-conn-nd"
  2127. 2020-11-02T12:00:07.6617881Z STEP: Generated namespace "e2e-tests-dataplane-conn-nd-5lll7" in cluster "cluster2" to execute the tests in
  2128. 2020-11-02T12:00:07.6619883Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-5lll7" in cluster "cluster3"
  2129. 2020-11-02T12:00:07.6873391Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-5lll7" in cluster "cluster1"
  2130. 2020-11-02T12:00:07.6926076Z STEP: Creating a listener pod in cluster "cluster3", which will wait for a handshake over TCP
  2131. 2020-11-02T12:00:13.7531305Z Nov 2 12:00:13.752: INFO: Will send traffic to IP: 10.3.192.3
  2132. 2020-11-02T12:00:13.7532666Z STEP: Creating a connector pod in cluster "cluster2", which will attempt the specific UUID handshake over TCP
  2133. 2020-11-02T12:00:13.7583912Z STEP: Waiting for the connector pod "tcp-check-podp4lsz" to exit, returning what connector sent
  2134. 2020-11-02T12:00:18.7799012Z Nov 2 12:00:18.779: INFO: Pod "tcp-check-podp4lsz" output:
  2135. 2020-11-02T12:00:18.7800180Z 10.3.192.3 (10.3.192.3:1234) open
  2136. 2020-11-02T12:00:18.7801461Z [dataplane] listener says 96279e35-fae8-4e8e-a6c9-ba8ce3d8376e
  2137. 2020-11-02T12:00:18.7802200Z
  2138. 2020-11-02T12:00:18.7803319Z STEP: Waiting for the listener pod "tcp-check-listenerhdfmf" to exit, returning what listener sent
  2139. 2020-11-02T12:00:18.7833356Z Nov 2 12:00:18.783: INFO: Pod "tcp-check-listenerhdfmf" output:
  2140. 2020-11-02T12:00:18.7834558Z listening on 0.0.0.0:1234 ...
  2141. 2020-11-02T12:00:18.7835163Z connect to 10.3.192.3:1234 from 10.2.0.5:33555 (10.2.0.5:33555)
  2142. 2020-11-02T12:00:18.7836515Z [dataplane] connector says eeafa0da-e569-4825-9f7b-5e533e268073
  2143. 2020-11-02T12:00:18.7836940Z
  2144. 2020-11-02T12:00:18.7837422Z Nov 2 12:00:18.783: INFO: Connector pod has IP: 10.2.0.5
  2145. 2020-11-02T12:00:18.7840167Z STEP: Verifying that the listener got the connector's data and the connector got the listener's data
  2146. 2020-11-02T12:00:18.7843208Z STEP: Verifying the output of listener pod which must contain the source IP
  2147. 2020-11-02T12:00:18.7846000Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-5lll7" on cluster "cluster2"
  2148. 2020-11-02T12:00:18.7877006Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-5lll7" on cluster "cluster3"
  2149. 2020-11-02T12:00:18.7959672Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-5lll7" on cluster "cluster1"
  2150. 2020-11-02T12:00:18.8030808Z •
  2151. 2020-11-02T12:00:18.8032288Z ------------------------------
  2152. 2020-11-02T12:00:18.8034129Z STEP: Creating namespace objects with basename "dataplane-conn-nd"
  2153. 2020-11-02T12:00:18.8036410Z STEP: Generated namespace "e2e-tests-dataplane-conn-nd-5lll7" in cluster "cluster2" to execute the tests in
  2154. 2020-11-02T12:00:18.8038224Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-5lll7" in cluster "cluster3"
  2155. 2020-11-02T12:00:18.8039835Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-5lll7" in cluster "cluster1"
  2156. 2020-11-02T12:00:18.8041496Z STEP: Creating a listener pod in cluster "cluster3", which will wait for a handshake over TCP
  2157. 2020-11-02T12:00:18.8046599Z Nov 2 12:00:13.752: INFO: Will send traffic to IP: 10.3.192.3
  2158. 2020-11-02T12:00:18.8048840Z STEP: Creating a connector pod in cluster "cluster2", which will attempt the specific UUID handshake over TCP
  2159. 2020-11-02T12:00:18.8051094Z STEP: Waiting for the connector pod "tcp-check-podp4lsz" to exit, returning what connector sent
  2160. 2020-11-02T12:00:18.8051949Z Nov 2 12:00:18.779: INFO: Pod "tcp-check-podp4lsz" output:
  2161. 2020-11-02T12:00:18.8052327Z 10.3.192.3 (10.3.192.3:1234) open
  2162. 2020-11-02T12:00:18.8052948Z [dataplane] listener says 96279e35-fae8-4e8e-a6c9-ba8ce3d8376e
  2163. 2020-11-02T12:00:18.8053648Z
  2164. 2020-11-02T12:00:18.8054548Z STEP: Waiting for the listener pod "tcp-check-listenerhdfmf" to exit, returning what listener sent
  2165. 2020-11-02T12:00:18.8055444Z Nov 2 12:00:18.783: INFO: Pod "tcp-check-listenerhdfmf" output:
  2166. 2020-11-02T12:00:18.8055879Z listening on 0.0.0.0:1234 ...
  2167. 2020-11-02T12:00:18.8056152Z connect to 10.3.192.3:1234 from 10.2.0.5:33555 (10.2.0.5:33555)
  2168. 2020-11-02T12:00:18.8056836Z [dataplane] connector says eeafa0da-e569-4825-9f7b-5e533e268073
  2169. 2020-11-02T12:00:18.8057252Z
  2170. 2020-11-02T12:00:18.8057530Z Nov 2 12:00:18.783: INFO: Connector pod has IP: 10.2.0.5
  2171. 2020-11-02T12:00:18.8058277Z STEP: Verifying that the listener got the connector's data and the connector got the listener's data
  2172. 2020-11-02T12:00:18.8059432Z STEP: Verifying the output of listener pod which must contain the source IP
  2173. 2020-11-02T12:00:18.8060621Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-5lll7" on cluster "cluster2"
  2174. 2020-11-02T12:00:18.8061824Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-5lll7" on cluster "cluster3"
  2175. 2020-11-02T12:00:18.8063010Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-5lll7" on cluster "cluster1"
  2176. 2020-11-02T12:00:18.8063572Z
  2177. 2020-11-02T12:00:18.8063710Z
  2178. 2020-11-02T12:00:18.8064951Z [dataplane] Basic TCP connectivity tests across clusters without discovery when a pod connects via TCP to a remote service when the pod is not on a gateway and the remote service is not on a gateway
  2179. 2020-11-02T12:00:18.8066177Z should have sent the expected data from the pod to the other pod
  2180. 2020-11-02T12:00:18.8067105Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_pod_connectivity.go:15
  2181. 2020-11-02T12:00:18.8068218Z STEP: Creating namespace objects with basename "dataplane-conn-nd"
  2182. 2020-11-02T12:00:18.8238887Z STEP: Generated namespace "e2e-tests-dataplane-conn-nd-495xd" in cluster "cluster2" to execute the tests in
  2183. 2020-11-02T12:00:18.8240860Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-495xd" in cluster "cluster3"
  2184. 2020-11-02T12:00:18.8365846Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-495xd" in cluster "cluster1"
  2185. 2020-11-02T12:00:18.8679617Z STEP: Creating a listener pod in cluster "cluster3", which will wait for a handshake over TCP
  2186. 2020-11-02T12:00:24.9277543Z STEP: Pointing a service ClusterIP to the listener pod in cluster "cluster3"
  2187. 2020-11-02T12:00:24.9379683Z Nov 2 12:00:24.937: INFO: Will send traffic to IP: 100.3.252.192
  2188. 2020-11-02T12:00:24.9425123Z STEP: Creating a connector pod in cluster "cluster2", which will attempt the specific UUID handshake over TCP
  2189. 2020-11-02T12:00:24.9688191Z STEP: Waiting for the connector pod "tcp-check-podlzkqq" to exit, returning what connector sent
  2190. 2020-11-02T12:00:30.0110799Z Nov 2 12:00:30.010: INFO: Pod "tcp-check-podlzkqq" output:
  2191. 2020-11-02T12:00:30.0112226Z 100.3.252.192 (100.3.252.192:1234) open
  2192. 2020-11-02T12:00:30.0113414Z [dataplane] listener says 6815c2cb-e892-401d-875f-b494a8bd3a23
  2193. 2020-11-02T12:00:30.0114028Z
  2194. 2020-11-02T12:00:30.0115012Z STEP: Waiting for the listener pod "tcp-check-listenertv52j" to exit, returning what listener sent
  2195. 2020-11-02T12:00:30.0149316Z Nov 2 12:00:30.014: INFO: Pod "tcp-check-listenertv52j" output:
  2196. 2020-11-02T12:00:30.0150073Z listening on 0.0.0.0:1234 ...
  2197. 2020-11-02T12:00:30.0150534Z connect to 10.3.128.3:1234 from 10.2.128.4:46101 (10.2.128.4:46101)
  2198. 2020-11-02T12:00:30.0151425Z [dataplane] connector says f2368978-4fc9-487f-85df-f9b2f62980a8
  2199. 2020-11-02T12:00:30.0151959Z
  2200. 2020-11-02T12:00:30.0152371Z Nov 2 12:00:30.014: INFO: Connector pod has IP: 10.2.128.4
  2201. 2020-11-02T12:00:30.0153709Z STEP: Verifying that the listener got the connector's data and the connector got the listener's data
  2202. 2020-11-02T12:00:30.0155225Z STEP: Verifying the output of listener pod which must contain the source IP
  2203. 2020-11-02T12:00:30.0162605Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-495xd" on cluster "cluster2"
  2204. 2020-11-02T12:00:30.0323763Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-495xd" on cluster "cluster3"
  2205. 2020-11-02T12:00:30.0376995Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-495xd" on cluster "cluster1"
  2206. 2020-11-02T12:00:30.0430403Z •
  2207. 2020-11-02T12:00:30.0431396Z ------------------------------
  2208. 2020-11-02T12:00:30.0437096Z STEP: Creating namespace objects with basename "dataplane-conn-nd"
  2209. 2020-11-02T12:00:30.0438467Z STEP: Generated namespace "e2e-tests-dataplane-conn-nd-495xd" in cluster "cluster2" to execute the tests in
  2210. 2020-11-02T12:00:30.0439879Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-495xd" in cluster "cluster3"
  2211. 2020-11-02T12:00:30.0441203Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-495xd" in cluster "cluster1"
  2212. 2020-11-02T12:00:30.0442397Z STEP: Creating a listener pod in cluster "cluster3", which will wait for a handshake over TCP
  2213. 2020-11-02T12:00:30.0443842Z STEP: Pointing a service ClusterIP to the listener pod in cluster "cluster3"
  2214. 2020-11-02T12:00:30.0445204Z Nov 2 12:00:24.937: INFO: Will send traffic to IP: 100.3.252.192
  2215. 2020-11-02T12:00:30.0446195Z STEP: Creating a connector pod in cluster "cluster2", which will attempt the specific UUID handshake over TCP
  2216. 2020-11-02T12:00:30.0447382Z STEP: Waiting for the connector pod "tcp-check-podlzkqq" to exit, returning what connector sent
  2217. 2020-11-02T12:00:30.0448704Z Nov 2 12:00:30.010: INFO: Pod "tcp-check-podlzkqq" output:
  2218. 2020-11-02T12:00:30.0449296Z 100.3.252.192 (100.3.252.192:1234) open
  2219. 2020-11-02T12:00:30.0450363Z [dataplane] listener says 6815c2cb-e892-401d-875f-b494a8bd3a23
  2220. 2020-11-02T12:00:30.0450917Z
  2221. 2020-11-02T12:00:30.0452289Z STEP: Waiting for the listener pod "tcp-check-listenertv52j" to exit, returning what listener sent
  2222. 2020-11-02T12:00:30.0453498Z Nov 2 12:00:30.014: INFO: Pod "tcp-check-listenertv52j" output:
  2223. 2020-11-02T12:00:30.0453950Z listening on 0.0.0.0:1234 ...
  2224. 2020-11-02T12:00:30.0454242Z connect to 10.3.128.3:1234 from 10.2.128.4:46101 (10.2.128.4:46101)
  2225. 2020-11-02T12:00:30.0455216Z [dataplane] connector says f2368978-4fc9-487f-85df-f9b2f62980a8
  2226. 2020-11-02T12:00:30.0455571Z
  2227. 2020-11-02T12:00:30.0455872Z Nov 2 12:00:30.014: INFO: Connector pod has IP: 10.2.128.4
  2228. 2020-11-02T12:00:30.0456685Z STEP: Verifying that the listener got the connector's data and the connector got the listener's data
  2229. 2020-11-02T12:00:30.0458218Z STEP: Verifying the output of listener pod which must contain the source IP
  2230. 2020-11-02T12:00:30.0459498Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-495xd" on cluster "cluster2"
  2231. 2020-11-02T12:00:30.0461034Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-495xd" on cluster "cluster3"
  2232. 2020-11-02T12:00:30.0463503Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-495xd" on cluster "cluster1"
  2233. 2020-11-02T12:00:30.0464762Z
  2234. 2020-11-02T12:00:30.0464918Z
  2235. 2020-11-02T12:00:30.0465835Z [redundancy] Gateway fail-over tests when any gateway node is configured
  2236. 2020-11-02T12:00:30.0466723Z should be reported to the Gateway API
  2237. 2020-11-02T12:00:30.0467531Z /go/src/github.com/submariner-io/submariner/test/e2e/redundancy/gateway_failover.go:25
  2238. 2020-11-02T12:00:30.0468470Z STEP: Creating namespace objects with basename "gateway-redundancy"
  2239. 2020-11-02T12:00:30.0497898Z STEP: Generated namespace "e2e-tests-gateway-redundancy-qbcmh" in cluster "cluster2" to execute the tests in
  2240. 2020-11-02T12:00:30.0499252Z STEP: Creating namespace "e2e-tests-gateway-redundancy-qbcmh" in cluster "cluster3"
  2241. 2020-11-02T12:00:30.0560982Z STEP: Creating namespace "e2e-tests-gateway-redundancy-qbcmh" in cluster "cluster1"
  2242. 2020-11-02T12:00:30.0596166Z STEP: Ensuring that only one gateway reports as active "cluster2"
  2243. 2020-11-02T12:00:30.1252175Z STEP: Ensuring that the gateway "cluster2-worker" is reporting connections
  2244. 2020-11-02T12:00:30.1345444Z STEP: Deleting namespace "e2e-tests-gateway-redundancy-qbcmh" on cluster "cluster2"
  2245. 2020-11-02T12:00:30.1417731Z STEP: Deleting namespace "e2e-tests-gateway-redundancy-qbcmh" on cluster "cluster3"
  2246. 2020-11-02T12:00:30.1466234Z STEP: Deleting namespace "e2e-tests-gateway-redundancy-qbcmh" on cluster "cluster1"
  2247. 2020-11-02T12:00:30.1596704Z •
  2248. 2020-11-02T12:00:30.1597319Z ------------------------------
  2249. 2020-11-02T12:00:30.1598279Z STEP: Creating namespace objects with basename "gateway-redundancy"
  2250. 2020-11-02T12:00:30.1599624Z STEP: Generated namespace "e2e-tests-gateway-redundancy-qbcmh" in cluster "cluster2" to execute the tests in
  2251. 2020-11-02T12:00:30.1600730Z STEP: Creating namespace "e2e-tests-gateway-redundancy-qbcmh" in cluster "cluster3"
  2252. 2020-11-02T12:00:30.1601777Z STEP: Creating namespace "e2e-tests-gateway-redundancy-qbcmh" in cluster "cluster1"
  2253. 2020-11-02T12:00:30.1602622Z STEP: Ensuring that only one gateway reports as active "cluster2"
  2254. 2020-11-02T12:00:30.1603301Z STEP: Ensuring that the gateway "cluster2-worker" is reporting connections
  2255. 2020-11-02T12:00:30.1604971Z STEP: Deleting namespace "e2e-tests-gateway-redundancy-qbcmh" on cluster "cluster2"
  2256. 2020-11-02T12:00:30.1606239Z STEP: Deleting namespace "e2e-tests-gateway-redundancy-qbcmh" on cluster "cluster3"
  2257. 2020-11-02T12:00:30.1607826Z STEP: Deleting namespace "e2e-tests-gateway-redundancy-qbcmh" on cluster "cluster1"
  2258. 2020-11-02T12:00:30.1608344Z
  2259. 2020-11-02T12:00:30.1608498Z
  2260. 2020-11-02T12:00:30.1609621Z [dataplane-globalnet] Basic TCP connectivity tests across overlapping clusters without discovery when a pod connects via TCP to the globalIP of a remote service when the pod is on a gateway and the remote service is not on a gateway
  2261. 2020-11-02T12:00:30.1610698Z should have sent the expected data from the pod to the other pod
  2262. 2020-11-02T12:00:30.1611475Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:15
  2263. 2020-11-02T12:00:30.1616444Z STEP: Creating namespace objects with basename "dataplane-gn-conn-nd"
  2264. 2020-11-02T12:00:30.1698508Z STEP: Generated namespace "e2e-tests-dataplane-gn-conn-nd-529t2" in cluster "cluster2" to execute the tests in
  2265. 2020-11-02T12:00:30.1699741Z STEP: Creating namespace "e2e-tests-dataplane-gn-conn-nd-529t2" in cluster "cluster3"
  2266. 2020-11-02T12:00:30.1767534Z STEP: Creating namespace "e2e-tests-dataplane-gn-conn-nd-529t2" in cluster "cluster1"
  2267. 2020-11-02T12:00:30.1808241Z Nov 2 12:00:30.180: INFO: Globalnet is not enabled, skipping the test...
  2268. 2020-11-02T12:00:30.1823544Z STEP: Deleting namespace "e2e-tests-dataplane-gn-conn-nd-529t2" on cluster "cluster2"
  2269. 2020-11-02T12:00:30.2462410Z STEP: Deleting namespace "e2e-tests-dataplane-gn-conn-nd-529t2" on cluster "cluster3"
  2270. 2020-11-02T12:00:30.2651958Z STEP: Deleting namespace "e2e-tests-dataplane-gn-conn-nd-529t2" on cluster "cluster1"
  2271. 2020-11-02T12:00:30.2701079Z
  2272. 2020-11-02T12:00:30.2705654Z S [SKIPPING] [0.111 seconds]
  2273. 2020-11-02T12:00:30.2706421Z [dataplane-globalnet] Basic TCP connectivity tests across overlapping clusters without discovery
  2274. 2020-11-02T12:00:30.2707349Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:9
  2275. 2020-11-02T12:00:30.2707934Z when a pod connects via TCP to the globalIP of a remote service
  2276. 2020-11-02T12:00:30.2708892Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:33
  2277. 2020-11-02T12:00:30.2709509Z when the pod is on a gateway and the remote service is not on a gateway
  2278. 2020-11-02T12:00:30.2710295Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:47
  2279. 2020-11-02T12:00:30.2711244Z should have sent the expected data from the pod to the other pod [It]
  2280. 2020-11-02T12:00:30.2712248Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:15
  2281. 2020-11-02T12:00:30.2712698Z
  2282. 2020-11-02T12:00:30.2713580Z Nov 2 12:00:30.180: Globalnet is not enabled, skipping the test...
  2283. 2020-11-02T12:00:30.2714036Z
  2284. 2020-11-02T12:00:30.2714952Z /go/src/github.com/submariner-io/submariner/vendor/github.com/submariner-io/shipyard/test/e2e/framework/logging.go:42
  2285. 2020-11-02T12:00:30.2715652Z ------------------------------
  2286. 2020-11-02T12:00:30.2716821Z [dataplane-globalnet] Basic TCP connectivity tests across overlapping clusters without discovery when a pod connects via TCP to the globalIP of a remote service when the pod is on a gateway and the remote service is on a gateway
  2287. 2020-11-02T12:00:30.2718061Z should have sent the expected data from the pod to the other pod
  2288. 2020-11-02T12:00:30.2718811Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:15
  2289. 2020-11-02T12:00:30.2719640Z STEP: Creating namespace objects with basename "dataplane-gn-conn-nd"
  2290. 2020-11-02T12:00:30.2782454Z STEP: Generated namespace "e2e-tests-dataplane-gn-conn-nd-m9mtf" in cluster "cluster2" to execute the tests in
  2291. 2020-11-02T12:00:30.2783764Z STEP: Creating namespace "e2e-tests-dataplane-gn-conn-nd-m9mtf" in cluster "cluster3"
  2292. 2020-11-02T12:00:30.2988173Z STEP: Creating namespace "e2e-tests-dataplane-gn-conn-nd-m9mtf" in cluster "cluster1"
  2293. 2020-11-02T12:00:30.3122363Z Nov 2 12:00:30.311: INFO: Globalnet is not enabled, skipping the test...
  2294. 2020-11-02T12:00:30.3127846Z STEP: Deleting namespace "e2e-tests-dataplane-gn-conn-nd-m9mtf" on cluster "cluster2"
  2295. 2020-11-02T12:00:30.3408272Z STEP: Deleting namespace "e2e-tests-dataplane-gn-conn-nd-m9mtf" on cluster "cluster3"
  2296. 2020-11-02T12:00:30.3496241Z STEP: Deleting namespace "e2e-tests-dataplane-gn-conn-nd-m9mtf" on cluster "cluster1"
  2297. 2020-11-02T12:00:30.3560946Z
  2298. 2020-11-02T12:00:30.3568026Z S [SKIPPING] [0.086 seconds]
  2299. 2020-11-02T12:00:30.3568797Z [dataplane-globalnet] Basic TCP connectivity tests across overlapping clusters without discovery
  2300. 2020-11-02T12:00:30.3569872Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:9
  2301. 2020-11-02T12:00:30.3575881Z when a pod connects via TCP to the globalIP of a remote service
  2302. 2020-11-02T12:00:30.3576805Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:33
  2303. 2020-11-02T12:00:30.3577394Z when the pod is on a gateway and the remote service is on a gateway
  2304. 2020-11-02T12:00:30.3578141Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:51
  2305. 2020-11-02T12:00:30.3578889Z should have sent the expected data from the pod to the other pod [It]
  2306. 2020-11-02T12:00:30.3579653Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:15
  2307. 2020-11-02T12:00:30.3580431Z
  2308. 2020-11-02T12:00:30.3582910Z Nov 2 12:00:30.311: Globalnet is not enabled, skipping the test...
  2309. 2020-11-02T12:00:30.3583389Z
  2310. 2020-11-02T12:00:30.3584333Z /go/src/github.com/submariner-io/submariner/vendor/github.com/submariner-io/shipyard/test/e2e/framework/logging.go:42
  2311. 2020-11-02T12:00:30.3585404Z ------------------------------
  2312. 2020-11-02T12:00:30.3586565Z [dataplane] Basic TCP connectivity tests across clusters without discovery when a pod connects via TCP to a remote pod when the pod is on a gateway and the remote pod is not on a gateway
  2313. 2020-11-02T12:00:30.3587766Z should have sent the expected data from the pod to the other pod
  2314. 2020-11-02T12:00:30.3588612Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_pod_connectivity.go:15
  2315. 2020-11-02T12:00:30.3589663Z STEP: Creating namespace objects with basename "dataplane-conn-nd"
  2316. 2020-11-02T12:00:30.3819657Z STEP: Generated namespace "e2e-tests-dataplane-conn-nd-4tvm2" in cluster "cluster2" to execute the tests in
  2317. 2020-11-02T12:00:30.3820967Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-4tvm2" in cluster "cluster3"
  2318. 2020-11-02T12:00:30.4035181Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-4tvm2" in cluster "cluster1"
  2319. 2020-11-02T12:00:30.4079947Z STEP: Creating a listener pod in cluster "cluster3", which will wait for a handshake over TCP
  2320. 2020-11-02T12:00:36.4708111Z Nov 2 12:00:36.470: INFO: Will send traffic to IP: 10.3.128.3
  2321. 2020-11-02T12:00:36.4710174Z STEP: Creating a connector pod in cluster "cluster2", which will attempt the specific UUID handshake over TCP
  2322. 2020-11-02T12:00:36.4978581Z STEP: Waiting for the connector pod "tcp-check-podmdcw5" to exit, returning what connector sent
  2323. 2020-11-02T12:00:41.5358666Z Nov 2 12:00:41.535: INFO: Pod "tcp-check-podmdcw5" output:
  2324. 2020-11-02T12:00:41.5359085Z 10.3.128.3 (10.3.128.3:1234) open
  2325. 2020-11-02T12:00:41.5360135Z [dataplane] listener says bc4a024c-8888-4866-90c5-d54a3b50b6ea
  2326. 2020-11-02T12:00:41.5360777Z
  2327. 2020-11-02T12:00:41.5367048Z STEP: Waiting for the listener pod "tcp-check-listenerqkt4z" to exit, returning what listener sent
  2328. 2020-11-02T12:00:41.5384379Z Nov 2 12:00:41.538: INFO: Pod "tcp-check-listenerqkt4z" output:
  2329. 2020-11-02T12:00:41.5385482Z listening on 0.0.0.0:1234 ...
  2330. 2020-11-02T12:00:41.5386434Z connect to 10.3.128.3:1234 from 10.2.0.5:44663 (10.2.0.5:44663)
  2331. 2020-11-02T12:00:41.5388199Z [dataplane] connector says a1d0b017-aa6c-422a-a564-e572ed0bc206
  2332. 2020-11-02T12:00:41.5388890Z
  2333. 2020-11-02T12:00:41.5389405Z Nov 2 12:00:41.538: INFO: Connector pod has IP: 10.2.0.5
  2334. 2020-11-02T12:00:41.5390243Z STEP: Verifying that the listener got the connector's data and the connector got the listener's data
  2335. 2020-11-02T12:00:41.5391179Z STEP: Verifying the output of listener pod which must contain the source IP
  2336. 2020-11-02T12:00:41.5392237Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-4tvm2" on cluster "cluster2"
  2337. 2020-11-02T12:00:41.5478890Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-4tvm2" on cluster "cluster3"
  2338. 2020-11-02T12:00:41.5553300Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-4tvm2" on cluster "cluster1"
  2339. 2020-11-02T12:00:41.5612070Z •
  2340. 2020-11-02T12:00:41.5615075Z ------------------------------
  2341. 2020-11-02T12:00:41.5615807Z STEP: Creating namespace objects with basename "dataplane-conn-nd"
  2342. 2020-11-02T12:00:41.5617001Z STEP: Generated namespace "e2e-tests-dataplane-conn-nd-4tvm2" in cluster "cluster2" to execute the tests in
  2343. 2020-11-02T12:00:41.5618314Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-4tvm2" in cluster "cluster3"
  2344. 2020-11-02T12:00:41.5619761Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-4tvm2" in cluster "cluster1"
  2345. 2020-11-02T12:00:41.5620787Z STEP: Creating a listener pod in cluster "cluster3", which will wait for a handshake over TCP
  2346. 2020-11-02T12:00:41.5622188Z Nov 2 12:00:36.470: INFO: Will send traffic to IP: 10.3.128.3
  2347. 2020-11-02T12:00:41.5622955Z STEP: Creating a connector pod in cluster "cluster2", which will attempt the specific UUID handshake over TCP
  2348. 2020-11-02T12:00:41.5623812Z STEP: Waiting for the connector pod "tcp-check-podmdcw5" to exit, returning what connector sent
  2349. 2020-11-02T12:00:41.5624492Z Nov 2 12:00:41.535: INFO: Pod "tcp-check-podmdcw5" output:
  2350. 2020-11-02T12:00:41.5624801Z 10.3.128.3 (10.3.128.3:1234) open
  2351. 2020-11-02T12:00:41.5625305Z [dataplane] listener says bc4a024c-8888-4866-90c5-d54a3b50b6ea
  2352. 2020-11-02T12:00:41.5625603Z
  2353. 2020-11-02T12:00:41.5626395Z STEP: Waiting for the listener pod "tcp-check-listenerqkt4z" to exit, returning what listener sent
  2354. 2020-11-02T12:00:41.5627391Z Nov 2 12:00:41.538: INFO: Pod "tcp-check-listenerqkt4z" output:
  2355. 2020-11-02T12:00:41.5627788Z listening on 0.0.0.0:1234 ...
  2356. 2020-11-02T12:00:41.5628225Z connect to 10.3.128.3:1234 from 10.2.0.5:44663 (10.2.0.5:44663)
  2357. 2020-11-02T12:00:41.5629033Z [dataplane] connector says a1d0b017-aa6c-422a-a564-e572ed0bc206
  2358. 2020-11-02T12:00:41.5629405Z
  2359. 2020-11-02T12:00:41.5629658Z Nov 2 12:00:41.538: INFO: Connector pod has IP: 10.2.0.5
  2360. 2020-11-02T12:00:41.5630688Z STEP: Verifying that the listener got the connector's data and the connector got the listener's data
  2361. 2020-11-02T12:00:41.5631447Z STEP: Verifying the output of listener pod which must contain the source IP
  2362. 2020-11-02T12:00:41.5632462Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-4tvm2" on cluster "cluster2"
  2363. 2020-11-02T12:00:41.5633474Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-4tvm2" on cluster "cluster3"
  2364. 2020-11-02T12:00:41.5634482Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-4tvm2" on cluster "cluster1"
  2365. 2020-11-02T12:00:41.5634942Z
  2366. 2020-11-02T12:00:41.5635073Z
  2367. 2020-11-02T12:00:41.5635971Z [dataplane] Basic TCP connectivity tests across clusters without discovery when a pod connects via TCP to a remote service when the pod is not on a gateway and the remote service is on a gateway
  2368. 2020-11-02T12:00:41.5637274Z should have sent the expected data from the pod to the other pod
  2369. 2020-11-02T12:00:41.5638396Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_pod_connectivity.go:15
  2370. 2020-11-02T12:00:41.5639472Z STEP: Creating namespace objects with basename "dataplane-conn-nd"
  2371. 2020-11-02T12:00:41.5720595Z STEP: Generated namespace "e2e-tests-dataplane-conn-nd-xj7k5" in cluster "cluster2" to execute the tests in
  2372. 2020-11-02T12:00:41.5721872Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-xj7k5" in cluster "cluster3"
  2373. 2020-11-02T12:00:41.5961508Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-xj7k5" in cluster "cluster1"
  2374. 2020-11-02T12:00:41.5996119Z STEP: Creating a listener pod in cluster "cluster3", which will wait for a handshake over TCP
  2375. 2020-11-02T12:00:47.6430256Z STEP: Pointing a service ClusterIP to the listener pod in cluster "cluster3"
  2376. 2020-11-02T12:00:47.6636884Z Nov 2 12:00:47.663: INFO: Will send traffic to IP: 100.3.77.234
  2377. 2020-11-02T12:00:47.6639371Z STEP: Creating a connector pod in cluster "cluster2", which will attempt the specific UUID handshake over TCP
  2378. 2020-11-02T12:00:47.6735382Z STEP: Waiting for the connector pod "tcp-check-podbn4rk" to exit, returning what connector sent
  2379. 2020-11-02T12:00:52.6804925Z Nov 2 12:00:52.679: INFO: Pod "tcp-check-podbn4rk" output:
  2380. 2020-11-02T12:00:52.6805735Z 100.3.77.234 (100.3.77.234:1234) open
  2381. 2020-11-02T12:00:52.6806509Z [dataplane] listener says fb8680da-36d8-4073-adb8-f462a9730af9
  2382. 2020-11-02T12:00:52.6806979Z
  2383. 2020-11-02T12:00:52.6810810Z STEP: Waiting for the listener pod "tcp-check-listener8p5vx" to exit, returning what listener sent
  2384. 2020-11-02T12:00:52.6839515Z Nov 2 12:00:52.683: INFO: Pod "tcp-check-listener8p5vx" output:
  2385. 2020-11-02T12:00:52.6840337Z listening on 0.0.0.0:1234 ...
  2386. 2020-11-02T12:00:52.6841442Z connect to 10.3.192.3:1234 from 10.2.128.4:43927 (10.2.128.4:43927)
  2387. 2020-11-02T12:00:52.6842599Z [dataplane] connector says 606dd02f-6f0d-48b8-98da-21c56a2ac07f
  2388. 2020-11-02T12:00:52.6843102Z
  2389. 2020-11-02T12:00:52.6843468Z Nov 2 12:00:52.683: INFO: Connector pod has IP: 10.2.128.4
  2390. 2020-11-02T12:00:52.6844984Z STEP: Verifying that the listener got the connector's data and the connector got the listener's data
  2391. 2020-11-02T12:00:52.6849102Z STEP: Verifying the output of listener pod which must contain the source IP
  2392. 2020-11-02T12:00:52.6850279Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-xj7k5" on cluster "cluster2"
  2393. 2020-11-02T12:00:52.6890659Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-xj7k5" on cluster "cluster3"
  2394. 2020-11-02T12:00:52.7020078Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-xj7k5" on cluster "cluster1"
  2395. 2020-11-02T12:00:52.7090713Z •
  2396. 2020-11-02T12:00:52.7091158Z ------------------------------
  2397. 2020-11-02T12:00:52.7092628Z STEP: Creating namespace objects with basename "dataplane-conn-nd"
  2398. 2020-11-02T12:00:52.7094501Z STEP: Generated namespace "e2e-tests-dataplane-conn-nd-xj7k5" in cluster "cluster2" to execute the tests in
  2399. 2020-11-02T12:00:52.7097781Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-xj7k5" in cluster "cluster3"
  2400. 2020-11-02T12:00:52.7098882Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-xj7k5" in cluster "cluster1"
  2401. 2020-11-02T12:00:52.7100200Z STEP: Creating a listener pod in cluster "cluster3", which will wait for a handshake over TCP
  2402. 2020-11-02T12:00:52.7101210Z STEP: Pointing a service ClusterIP to the listener pod in cluster "cluster3"
  2403. 2020-11-02T12:00:52.7101724Z Nov 2 12:00:47.663: INFO: Will send traffic to IP: 100.3.77.234
  2404. 2020-11-02T12:00:52.7102793Z STEP: Creating a connector pod in cluster "cluster2", which will attempt the specific UUID handshake over TCP
  2405. 2020-11-02T12:00:52.7104448Z STEP: Waiting for the connector pod "tcp-check-podbn4rk" to exit, returning what connector sent
  2406. 2020-11-02T12:00:52.7105821Z Nov 2 12:00:52.679: INFO: Pod "tcp-check-podbn4rk" output:
  2407. 2020-11-02T12:00:52.7106190Z 100.3.77.234 (100.3.77.234:1234) open
  2408. 2020-11-02T12:00:52.7106795Z [dataplane] listener says fb8680da-36d8-4073-adb8-f462a9730af9
  2409. 2020-11-02T12:00:52.7107302Z
  2410. 2020-11-02T12:00:52.7108369Z STEP: Waiting for the listener pod "tcp-check-listener8p5vx" to exit, returning what listener sent
  2411. 2020-11-02T12:00:52.7109388Z Nov 2 12:00:52.683: INFO: Pod "tcp-check-listener8p5vx" output:
  2412. 2020-11-02T12:00:52.7109812Z listening on 0.0.0.0:1234 ...
  2413. 2020-11-02T12:00:52.7110076Z connect to 10.3.192.3:1234 from 10.2.128.4:43927 (10.2.128.4:43927)
  2414. 2020-11-02T12:00:52.7110903Z [dataplane] connector says 606dd02f-6f0d-48b8-98da-21c56a2ac07f
  2415. 2020-11-02T12:00:52.7111237Z
  2416. 2020-11-02T12:00:52.7111497Z Nov 2 12:00:52.683: INFO: Connector pod has IP: 10.2.128.4
  2417. 2020-11-02T12:00:52.7112191Z STEP: Verifying that the listener got the connector's data and the connector got the listener's data
  2418. 2020-11-02T12:00:52.7112992Z STEP: Verifying the output of listener pod which must contain the source IP
  2419. 2020-11-02T12:00:52.7114048Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-xj7k5" on cluster "cluster2"
  2420. 2020-11-02T12:00:52.7115282Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-xj7k5" on cluster "cluster3"
  2421. 2020-11-02T12:00:52.7116369Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-xj7k5" on cluster "cluster1"
  2422. 2020-11-02T12:00:52.7116872Z
  2423. 2020-11-02T12:00:52.7117009Z
  2424. 2020-11-02T12:00:52.7117926Z [dataplane] Basic TCP connectivity tests across clusters without discovery when a pod connects via TCP to a remote pod when the pod is not on a gateway and the remote pod is on a gateway
  2425. 2020-11-02T12:00:52.7118895Z should have sent the expected data from the pod to the other pod
  2426. 2020-11-02T12:00:52.7119693Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_pod_connectivity.go:15
  2427. 2020-11-02T12:00:52.7120702Z STEP: Creating namespace objects with basename "dataplane-conn-nd"
  2428. 2020-11-02T12:00:52.7154049Z STEP: Generated namespace "e2e-tests-dataplane-conn-nd-cbrjj" in cluster "cluster2" to execute the tests in
  2429. 2020-11-02T12:00:52.7155395Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-cbrjj" in cluster "cluster3"
  2430. 2020-11-02T12:00:52.7207854Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-cbrjj" in cluster "cluster1"
  2431. 2020-11-02T12:00:52.7348508Z STEP: Creating a listener pod in cluster "cluster3", which will wait for a handshake over TCP
  2432. 2020-11-02T12:00:58.8097126Z Nov 2 12:00:58.809: INFO: Will send traffic to IP: 10.3.192.3
  2433. 2020-11-02T12:00:58.8098749Z STEP: Creating a connector pod in cluster "cluster2", which will attempt the specific UUID handshake over TCP
  2434. 2020-11-02T12:00:58.8176669Z STEP: Waiting for the connector pod "tcp-check-pod4p2fm" to exit, returning what connector sent
  2435. 2020-11-02T12:01:03.8363221Z Nov 2 12:01:03.835: INFO: Pod "tcp-check-pod4p2fm" output:
  2436. 2020-11-02T12:01:03.8363630Z 10.3.192.3 (10.3.192.3:1234) open
  2437. 2020-11-02T12:01:03.8364923Z [dataplane] listener says 94fd9faa-0b03-4f9f-9580-d3ca08be4abe
  2438. 2020-11-02T12:01:03.8365291Z
  2439. 2020-11-02T12:01:03.8365957Z STEP: Waiting for the listener pod "tcp-check-listenerzr6kp" to exit, returning what listener sent
  2440. 2020-11-02T12:01:03.8397730Z Nov 2 12:01:03.839: INFO: Pod "tcp-check-listenerzr6kp" output:
  2441. 2020-11-02T12:01:03.8398168Z listening on 0.0.0.0:1234 ...
  2442. 2020-11-02T12:01:03.8398414Z connect to 10.3.192.3:1234 from 10.2.128.4:38457 (10.2.128.4:38457)
  2443. 2020-11-02T12:01:03.8399199Z [dataplane] connector says 3c4e236b-1a53-41cb-ab1f-8f4adb827b6e
  2444. 2020-11-02T12:01:03.8399549Z
  2445. 2020-11-02T12:01:03.8399799Z Nov 2 12:01:03.839: INFO: Connector pod has IP: 10.2.128.4
  2446. 2020-11-02T12:01:03.8405033Z STEP: Verifying that the listener got the connector's data and the connector got the listener's data
  2447. 2020-11-02T12:01:03.8405844Z STEP: Verifying the output of listener pod which must contain the source IP
  2448. 2020-11-02T12:01:03.8406711Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-cbrjj" on cluster "cluster2"
  2449. 2020-11-02T12:01:03.8453785Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-cbrjj" on cluster "cluster3"
  2450. 2020-11-02T12:01:03.8499599Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-cbrjj" on cluster "cluster1"
  2451. 2020-11-02T12:01:03.8550535Z •
  2452. 2020-11-02T12:01:03.8551106Z ------------------------------
  2453. 2020-11-02T12:01:03.8551720Z STEP: Creating namespace objects with basename "dataplane-conn-nd"
  2454. 2020-11-02T12:01:03.8552844Z STEP: Generated namespace "e2e-tests-dataplane-conn-nd-cbrjj" in cluster "cluster2" to execute the tests in
  2455. 2020-11-02T12:01:03.8553916Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-cbrjj" in cluster "cluster3"
  2456. 2020-11-02T12:01:03.8555061Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-cbrjj" in cluster "cluster1"
  2457. 2020-11-02T12:01:03.8555952Z STEP: Creating a listener pod in cluster "cluster3", which will wait for a handshake over TCP
  2458. 2020-11-02T12:01:03.8556413Z Nov 2 12:00:58.809: INFO: Will send traffic to IP: 10.3.192.3
  2459. 2020-11-02T12:01:03.8557079Z STEP: Creating a connector pod in cluster "cluster2", which will attempt the specific UUID handshake over TCP
  2460. 2020-11-02T12:01:03.8557917Z STEP: Waiting for the connector pod "tcp-check-pod4p2fm" to exit, returning what connector sent
  2461. 2020-11-02T12:01:03.8558594Z Nov 2 12:01:03.835: INFO: Pod "tcp-check-pod4p2fm" output:
  2462. 2020-11-02T12:01:03.8558901Z 10.3.192.3 (10.3.192.3:1234) open
  2463. 2020-11-02T12:01:03.8559434Z [dataplane] listener says 94fd9faa-0b03-4f9f-9580-d3ca08be4abe
  2464. 2020-11-02T12:01:03.8559768Z
  2465. 2020-11-02T12:01:03.8560381Z STEP: Waiting for the listener pod "tcp-check-listenerzr6kp" to exit, returning what listener sent
  2466. 2020-11-02T12:01:03.8561130Z Nov 2 12:01:03.839: INFO: Pod "tcp-check-listenerzr6kp" output:
  2467. 2020-11-02T12:01:03.8561639Z listening on 0.0.0.0:1234 ...
  2468. 2020-11-02T12:01:03.8561888Z connect to 10.3.192.3:1234 from 10.2.128.4:38457 (10.2.128.4:38457)
  2469. 2020-11-02T12:01:03.8562472Z [dataplane] connector says 3c4e236b-1a53-41cb-ab1f-8f4adb827b6e
  2470. 2020-11-02T12:01:03.8562800Z
  2471. 2020-11-02T12:01:03.8563023Z Nov 2 12:01:03.839: INFO: Connector pod has IP: 10.2.128.4
  2472. 2020-11-02T12:01:03.8563658Z STEP: Verifying that the listener got the connector's data and the connector got the listener's data
  2473. 2020-11-02T12:01:03.8565196Z STEP: Verifying the output of listener pod which must contain the source IP
  2474. 2020-11-02T12:01:03.8566052Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-cbrjj" on cluster "cluster2"
  2475. 2020-11-02T12:01:03.8567060Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-cbrjj" on cluster "cluster3"
  2476. 2020-11-02T12:01:03.8568263Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-cbrjj" on cluster "cluster1"
  2477. 2020-11-02T12:01:03.8568748Z
  2478. 2020-11-02T12:01:03.8568915Z
  2479. 2020-11-02T12:01:03.8569224Z P [PENDING]
  2480. 2020-11-02T12:01:03.8569602Z [expansion] Test expanding/shrinking an existing cluster fleet
  2481. 2020-11-02T12:01:03.8570331Z /go/src/github.com/submariner-io/submariner/test/e2e/cluster/add_remove_cluster.go:12
  2482. 2020-11-02T12:01:03.8570996Z Should be able to add and remove third cluster
  2483. 2020-11-02T12:01:03.8571657Z /go/src/github.com/submariner-io/submariner/test/e2e/cluster/add_remove_cluster.go:15
  2484. 2020-11-02T12:01:03.8572208Z ------------------------------
  2485. 2020-11-02T12:01:03.8573316Z [dataplane-globalnet] Basic TCP connectivity tests across overlapping clusters without discovery when a pod connects via TCP to the globalIP of a remote service when the pod is not on a gateway and the remote service is not on a gateway
  2486. 2020-11-02T12:01:03.8574396Z should have sent the expected data from the pod to the other pod
  2487. 2020-11-02T12:01:03.8575290Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:15
  2488. 2020-11-02T12:01:03.8576081Z STEP: Creating namespace objects with basename "dataplane-gn-conn-nd"
  2489. 2020-11-02T12:01:03.8635563Z STEP: Generated namespace "e2e-tests-dataplane-gn-conn-nd-slvv6" in cluster "cluster2" to execute the tests in
  2490. 2020-11-02T12:01:03.8636946Z STEP: Creating namespace "e2e-tests-dataplane-gn-conn-nd-slvv6" in cluster "cluster3"
  2491. 2020-11-02T12:01:03.8877547Z STEP: Creating namespace "e2e-tests-dataplane-gn-conn-nd-slvv6" in cluster "cluster1"
  2492. 2020-11-02T12:01:03.8915991Z Nov 2 12:01:03.891: INFO: Globalnet is not enabled, skipping the test...
  2493. 2020-11-02T12:01:03.8925801Z STEP: Deleting namespace "e2e-tests-dataplane-gn-conn-nd-slvv6" on cluster "cluster2"
  2494. 2020-11-02T12:01:03.9188262Z STEP: Deleting namespace "e2e-tests-dataplane-gn-conn-nd-slvv6" on cluster "cluster3"
  2495. 2020-11-02T12:01:03.9550304Z STEP: Deleting namespace "e2e-tests-dataplane-gn-conn-nd-slvv6" on cluster "cluster1"
  2496. 2020-11-02T12:01:03.9665710Z
  2497. 2020-11-02T12:01:03.9670188Z S [SKIPPING] [0.112 seconds]
  2498. 2020-11-02T12:01:03.9674121Z [dataplane-globalnet] Basic TCP connectivity tests across overlapping clusters without discovery
  2499. 2020-11-02T12:01:03.9678397Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:9
  2500. 2020-11-02T12:01:03.9682110Z when a pod connects via TCP to the globalIP of a remote service
  2501. 2020-11-02T12:01:03.9686213Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:33
  2502. 2020-11-02T12:01:03.9689854Z when the pod is not on a gateway and the remote service is not on a gateway
  2503. 2020-11-02T12:01:03.9693771Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:39
  2504. 2020-11-02T12:01:03.9697744Z should have sent the expected data from the pod to the other pod [It]
  2505. 2020-11-02T12:01:03.9703451Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:15
  2506. 2020-11-02T12:01:03.9707137Z
  2507. 2020-11-02T12:01:03.9713603Z Nov 2 12:01:03.891: Globalnet is not enabled, skipping the test...
  2508. 2020-11-02T12:01:03.9716947Z
  2509. 2020-11-02T12:01:03.9726276Z /go/src/github.com/submariner-io/submariner/vendor/github.com/submariner-io/shipyard/test/e2e/framework/logging.go:42
  2510. 2020-11-02T12:01:03.9730116Z ------------------------------
  2511. 2020-11-02T12:01:03.9749498Z [dataplane] Basic TCP connectivity tests across clusters without discovery when a pod connects via TCP to a remote service when the pod is on a gateway and the remote service is not on a gateway
  2512. 2020-11-02T12:01:03.9750467Z should have sent the expected data from the pod to the other pod
  2513. 2020-11-02T12:01:03.9751204Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_pod_connectivity.go:15
  2514. 2020-11-02T12:01:03.9752190Z STEP: Creating namespace objects with basename "dataplane-conn-nd"
  2515. 2020-11-02T12:01:03.9820315Z STEP: Generated namespace "e2e-tests-dataplane-conn-nd-6wtkj" in cluster "cluster2" to execute the tests in
  2516. 2020-11-02T12:01:03.9824658Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-6wtkj" in cluster "cluster3"
  2517. 2020-11-02T12:01:04.0056874Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-6wtkj" in cluster "cluster1"
  2518. 2020-11-02T12:01:04.0135851Z STEP: Creating a listener pod in cluster "cluster3", which will wait for a handshake over TCP
  2519. 2020-11-02T12:01:09.2564777Z STEP: Pointing a service ClusterIP to the listener pod in cluster "cluster3"
  2520. 2020-11-02T12:01:09.2946195Z Nov 2 12:01:09.293: INFO: Will send traffic to IP: 100.3.156.69
  2521. 2020-11-02T12:01:09.2947846Z STEP: Creating a connector pod in cluster "cluster2", which will attempt the specific UUID handshake over TCP
  2522. 2020-11-02T12:01:09.3125816Z STEP: Waiting for the connector pod "tcp-check-pod9xzrh" to exit, returning what connector sent
  2523. 2020-11-02T12:01:14.3964732Z Nov 2 12:01:14.395: INFO: Pod "tcp-check-pod9xzrh" output:
  2524. 2020-11-02T12:01:14.3965880Z 100.3.156.69 (100.3.156.69:1234) open
  2525. 2020-11-02T12:01:14.3966731Z [dataplane] listener says ec922a7f-598b-4c7d-b67f-93591099b36d
  2526. 2020-11-02T12:01:14.3967290Z
  2527. 2020-11-02T12:01:14.3968175Z STEP: Waiting for the listener pod "tcp-check-listenerkl94x" to exit, returning what listener sent
  2528. 2020-11-02T12:01:14.3993954Z Nov 2 12:01:14.399: INFO: Pod "tcp-check-listenerkl94x" output:
  2529. 2020-11-02T12:01:14.3995564Z listening on 0.0.0.0:1234 ...
  2530. 2020-11-02T12:01:14.3996474Z connect to 10.3.128.3:1234 from 10.2.0.5:33205 (10.2.0.5:33205)
  2531. 2020-11-02T12:01:14.3997409Z [dataplane] connector says 2c1d91bf-b6a2-42e7-b12c-b5bc0fa25585
  2532. 2020-11-02T12:01:14.3998079Z
  2533. 2020-11-02T12:01:14.3998697Z Nov 2 12:01:14.399: INFO: Connector pod has IP: 10.2.0.5
  2534. 2020-11-02T12:01:14.4000273Z STEP: Verifying that the listener got the connector's data and the connector got the listener's data
  2535. 2020-11-02T12:01:14.4001504Z STEP: Verifying the output of listener pod which must contain the source IP
  2536. 2020-11-02T12:01:14.4002857Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-6wtkj" on cluster "cluster2"
  2537. 2020-11-02T12:01:14.4061123Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-6wtkj" on cluster "cluster3"
  2538. 2020-11-02T12:01:14.4117710Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-6wtkj" on cluster "cluster1"
  2539. 2020-11-02T12:01:14.4160729Z •
  2540. 2020-11-02T12:01:14.4161157Z ------------------------------
  2541. 2020-11-02T12:01:14.4204834Z STEP: Creating namespace objects with basename "dataplane-conn-nd"
  2542. 2020-11-02T12:01:14.4205813Z STEP: Generated namespace "e2e-tests-dataplane-conn-nd-6wtkj" in cluster "cluster2" to execute the tests in
  2543. 2020-11-02T12:01:14.4206864Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-6wtkj" in cluster "cluster3"
  2544. 2020-11-02T12:01:14.4207848Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-6wtkj" in cluster "cluster1"
  2545. 2020-11-02T12:01:14.4208968Z STEP: Creating a listener pod in cluster "cluster3", which will wait for a handshake over TCP
  2546. 2020-11-02T12:01:14.4209681Z STEP: Pointing a service ClusterIP to the listener pod in cluster "cluster3"
  2547. 2020-11-02T12:01:14.4210116Z Nov 2 12:01:09.293: INFO: Will send traffic to IP: 100.3.156.69
  2548. 2020-11-02T12:01:14.4210768Z STEP: Creating a connector pod in cluster "cluster2", which will attempt the specific UUID handshake over TCP
  2549. 2020-11-02T12:01:14.4211812Z STEP: Waiting for the connector pod "tcp-check-pod9xzrh" to exit, returning what connector sent
  2550. 2020-11-02T12:01:14.4212529Z Nov 2 12:01:14.395: INFO: Pod "tcp-check-pod9xzrh" output:
  2551. 2020-11-02T12:01:14.4212848Z 100.3.156.69 (100.3.156.69:1234) open
  2552. 2020-11-02T12:01:14.4213377Z [dataplane] listener says ec922a7f-598b-4c7d-b67f-93591099b36d
  2553. 2020-11-02T12:01:14.4213706Z
  2554. 2020-11-02T12:01:14.4214332Z STEP: Waiting for the listener pod "tcp-check-listenerkl94x" to exit, returning what listener sent
  2555. 2020-11-02T12:01:14.4215470Z Nov 2 12:01:14.399: INFO: Pod "tcp-check-listenerkl94x" output:
  2556. 2020-11-02T12:01:14.4215843Z listening on 0.0.0.0:1234 ...
  2557. 2020-11-02T12:01:14.4216090Z connect to 10.3.128.3:1234 from 10.2.0.5:33205 (10.2.0.5:33205)
  2558. 2020-11-02T12:01:14.4216652Z [dataplane] connector says 2c1d91bf-b6a2-42e7-b12c-b5bc0fa25585
  2559. 2020-11-02T12:01:14.4216979Z
  2560. 2020-11-02T12:01:14.4217218Z Nov 2 12:01:14.399: INFO: Connector pod has IP: 10.2.0.5
  2561. 2020-11-02T12:01:14.4217854Z STEP: Verifying that the listener got the connector's data and the connector got the listener's data
  2562. 2020-11-02T12:01:14.4218706Z STEP: Verifying the output of listener pod which must contain the source IP
  2563. 2020-11-02T12:01:14.4219501Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-6wtkj" on cluster "cluster2"
  2564. 2020-11-02T12:01:14.4220444Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-6wtkj" on cluster "cluster3"
  2565. 2020-11-02T12:01:14.4221565Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-6wtkj" on cluster "cluster1"
  2566. 2020-11-02T12:01:14.4222194Z
  2567. 2020-11-02T12:01:14.4222324Z
  2568. 2020-11-02T12:01:14.4223490Z [dataplane-globalnet] Basic TCP connectivity tests across overlapping clusters without discovery when a pod with HostNetworking connects via TCP to the globalIP of a remote service when the pod is not on a gateway and the remote service is not on a gateway
  2569. 2020-11-02T12:01:14.4224888Z should have sent the expected data from the pod to the other pod
  2570. 2020-11-02T12:01:14.4225643Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:15
  2571. 2020-11-02T12:01:14.4226416Z STEP: Creating namespace objects with basename "dataplane-gn-conn-nd"
  2572. 2020-11-02T12:01:14.4298507Z STEP: Generated namespace "e2e-tests-dataplane-gn-conn-nd-fv9mh" in cluster "cluster2" to execute the tests in
  2573. 2020-11-02T12:01:14.4303824Z STEP: Creating namespace "e2e-tests-dataplane-gn-conn-nd-fv9mh" in cluster "cluster3"
  2574. 2020-11-02T12:01:14.4384619Z STEP: Creating namespace "e2e-tests-dataplane-gn-conn-nd-fv9mh" in cluster "cluster1"
  2575. 2020-11-02T12:01:14.4482114Z Nov 2 12:01:14.447: INFO: Globalnet is not enabled, skipping the test...
  2576. 2020-11-02T12:01:14.4495321Z STEP: Deleting namespace "e2e-tests-dataplane-gn-conn-nd-fv9mh" on cluster "cluster2"
  2577. 2020-11-02T12:01:14.5066655Z STEP: Deleting namespace "e2e-tests-dataplane-gn-conn-nd-fv9mh" on cluster "cluster3"
  2578. 2020-11-02T12:01:14.5151964Z STEP: Deleting namespace "e2e-tests-dataplane-gn-conn-nd-fv9mh" on cluster "cluster1"
  2579. 2020-11-02T12:01:14.5198186Z
  2580. 2020-11-02T12:01:14.5198777Z S [SKIPPING] [0.099 seconds]
  2581. 2020-11-02T12:01:14.5199672Z [dataplane-globalnet] Basic TCP connectivity tests across overlapping clusters without discovery
  2582. 2020-11-02T12:01:14.5200752Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:9
  2583. 2020-11-02T12:01:14.5201663Z when a pod with HostNetworking connects via TCP to the globalIP of a remote service
  2584. 2020-11-02T12:01:14.5202556Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:56
  2585. 2020-11-02T12:01:14.5203171Z when the pod is not on a gateway and the remote service is not on a gateway
  2586. 2020-11-02T12:01:14.5204135Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:62
  2587. 2020-11-02T12:01:14.5212456Z should have sent the expected data from the pod to the other pod [It]
  2588. 2020-11-02T12:01:14.5213257Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:15
  2589. 2020-11-02T12:01:14.5213986Z
  2590. 2020-11-02T12:01:14.5214790Z Nov 2 12:01:14.447: Globalnet is not enabled, skipping the test...
  2591. 2020-11-02T12:01:14.5215070Z
  2592. 2020-11-02T12:01:14.5215787Z /go/src/github.com/submariner-io/submariner/vendor/github.com/submariner-io/shipyard/test/e2e/framework/logging.go:42
  2593. 2020-11-02T12:01:14.5216474Z ------------------------------
  2594. 2020-11-02T12:01:14.5217156Z [redundancy] Gateway fail-over tests when one gateway node is configured and the submariner engine pod fails
  2595. 2020-11-02T12:01:14.5218124Z should start a new submariner engine pod and be able to connect from another cluster
  2596. 2020-11-02T12:01:14.5218912Z /go/src/github.com/submariner-io/submariner/test/e2e/redundancy/gateway_failover.go:31
  2597. 2020-11-02T12:01:14.5219789Z STEP: Creating namespace objects with basename "gateway-redundancy"
  2598. 2020-11-02T12:01:14.5449311Z STEP: Generated namespace "e2e-tests-gateway-redundancy-v7drt" in cluster "cluster2" to execute the tests in
  2599. 2020-11-02T12:01:14.5450738Z STEP: Creating namespace "e2e-tests-gateway-redundancy-v7drt" in cluster "cluster3"
  2600. 2020-11-02T12:01:14.5507627Z STEP: Creating namespace "e2e-tests-gateway-redundancy-v7drt" in cluster "cluster1"
  2601. 2020-11-02T12:01:14.5561731Z STEP: Sanity check - ensuring there's only one gateway node on "cluster2"
  2602. 2020-11-02T12:01:14.5828138Z STEP: Found gateway on node "cluster2-worker" on "cluster2"
  2603. 2020-11-02T12:01:14.5957429Z STEP: Found submariner engine pod "submariner-gateway-tktjw" on "cluster2"
  2604. 2020-11-02T12:01:14.5958205Z STEP: Ensuring that the gateway reports as active on "cluster2"
  2605. 2020-11-02T12:01:14.5986030Z STEP: Deleting submariner engine pod "submariner-gateway-tktjw"
  2606. 2020-11-02T12:01:29.6380149Z STEP: Found new submariner engine pod "submariner-gateway-kcnxr"
  2607. 2020-11-02T12:01:29.6384283Z STEP: Waiting for the gateway to be up and connected "submariner-gateway-kcnxr"
  2608. 2020-11-02T12:01:39.6432663Z STEP: Verifying TCP connectivity from gateway node on "cluster3" to gateway node on "cluster2"
  2609. 2020-11-02T12:01:39.6434180Z STEP: Creating a listener pod in cluster "cluster2", which will wait for a handshake over TCP
  2610. 2020-11-02T12:01:44.6565616Z Nov 2 12:01:44.656: INFO: Will send traffic to IP: 10.2.0.5
  2611. 2020-11-02T12:01:44.6566867Z STEP: Creating a connector pod in cluster "cluster3", which will attempt the specific UUID handshake over TCP
  2612. 2020-11-02T12:01:44.6650209Z STEP: Waiting for the connector pod "tcp-check-podlc4bx" to exit, returning what connector sent
  2613. 2020-11-02T12:04:54.6754242Z STEP: Waiting for the listener pod "tcp-check-listenerhd2pj" to exit, returning what listener sent
  2614. 2020-11-02T12:04:54.6850222Z Nov 2 12:04:54.684: INFO: Pod "tcp-check-listenerhd2pj" output:
  2615. 2020-11-02T12:04:54.6850710Z listening on 0.0.0.0:1234 ...
  2616. 2020-11-02T12:04:54.6850929Z nc: timeout
  2617. 2020-11-02T12:04:54.6851062Z
  2618. 2020-11-02T12:04:54.6851295Z Nov 2 12:04:54.684: INFO: Connector pod has IP: 10.3.192.3
  2619. 2020-11-02T12:04:54.6856824Z STEP: Deleting namespace "e2e-tests-gateway-redundancy-v7drt" on cluster "cluster2"
  2620. 2020-11-02T12:04:54.6930090Z STEP: Deleting namespace "e2e-tests-gateway-redundancy-v7drt" on cluster "cluster3"
  2621. 2020-11-02T12:04:54.7080666Z STEP: Deleting namespace "e2e-tests-gateway-redundancy-v7drt" on cluster "cluster1"
  2622. 2020-11-02T12:04:54.8071468Z
  2623. 2020-11-02T12:04:54.8181920Z • Failure [220.286 seconds]
  2624. 2020-11-02T12:04:54.8187556Z [redundancy] Gateway fail-over tests
  2625. 2020-11-02T12:04:54.8188823Z /go/src/github.com/submariner-io/submariner/test/e2e/redundancy/gateway_failover.go:18
  2626. 2020-11-02T12:04:54.8189516Z when one gateway node is configured and the submariner engine pod fails
  2627. 2020-11-02T12:04:54.8191066Z /go/src/github.com/submariner-io/submariner/test/e2e/redundancy/gateway_failover.go:30
  2628. 2020-11-02T12:04:54.8191933Z should start a new submariner engine pod and be able to connect from another cluster [It]
  2629. 2020-11-02T12:04:54.8192755Z /go/src/github.com/submariner-io/submariner/test/e2e/redundancy/gateway_failover.go:31
  2630. 2020-11-02T12:04:54.8193147Z
  2631. 2020-11-02T12:04:54.8193454Z Expected
  2632. 2020-11-02T12:04:54.8193688Z <int32>: 1
  2633. 2020-11-02T12:04:54.8193889Z to equal
  2634. 2020-11-02T12:04:54.8194204Z <int32>: 0
  2635. 2020-11-02T12:04:54.8194342Z
  2636. 2020-11-02T12:04:54.8195065Z /go/src/github.com/submariner-io/submariner/vendor/github.com/submariner-io/shipyard/test/e2e/framework/network_pods.go:159
  2637. 2020-11-02T12:04:54.8195553Z
  2638. 2020-11-02T12:04:54.8195895Z Full Stack Trace
  2639. 2020-11-02T12:04:54.8196647Z github.com/submariner-io/shipyard/test/e2e/framework.(*NetworkPod).CheckSuccessfulFinish(0xc00001a000)
  2640. 2020-11-02T12:04:54.8198386Z /go/src/github.com/submariner-io/submariner/vendor/github.com/submariner-io/shipyard/test/e2e/framework/network_pods.go:159 +0x258
  2641. 2020-11-02T12:04:54.8199585Z github.com/submariner-io/shipyard/test/e2e/tcp.RunConnectivityTest(0xc0003a8eb0, 0x0, 0x12, 0x7, 0x1, 0x1, 0x0, 0x1, 0x0, 0x0, ...)
  2642. 2020-11-02T12:04:54.8201055Z /go/src/github.com/submariner-io/submariner/vendor/github.com/submariner-io/shipyard/test/e2e/tcp/connectivity.go:43 +0xa8
  2643. 2020-11-02T12:04:54.8202709Z github.com/submariner-io/submariner/test/e2e/redundancy.testEnginePodRestartScenario(0xc0000b6f60)
  2644. 2020-11-02T12:04:54.8205051Z /go/src/github.com/submariner-io/submariner/test/e2e/redundancy/gateway_failover.go:83 +0xaf9
  2645. 2020-11-02T12:04:54.8209682Z github.com/submariner-io/submariner/test/e2e/redundancy.glob..func1.2.1()
  2646. 2020-11-02T12:04:54.8210977Z /go/src/github.com/submariner-io/submariner/test/e2e/redundancy/gateway_failover.go:32 +0x2a
  2647. 2020-11-02T12:04:54.8212814Z github.com/submariner-io/shipyard/test/e2e.RunE2ETests(0xc000219e00, 0xc34d5b442c)
  2648. 2020-11-02T12:04:54.8213995Z /go/src/github.com/submariner-io/submariner/vendor/github.com/submariner-io/shipyard/test/e2e/e2e.go:77 +0x125
  2649. 2020-11-02T12:04:54.8215349Z github.com/submariner-io/submariner/test/e2e.TestE2E(0xc000219e00)
  2650. 2020-11-02T12:04:54.8216129Z /go/src/github.com/submariner-io/submariner/test/e2e/e2e_test.go:13 +0x2b
  2651. 2020-11-02T12:04:54.8216808Z testing.tRunner(0xc000219e00, 0x1605c98)
  2652. 2020-11-02T12:04:54.8217442Z /usr/lib/golang/src/testing/testing.go:1127 +0xef
  2653. 2020-11-02T12:04:54.8217971Z created by testing.(*T).Run
  2654. 2020-11-02T12:04:54.8218497Z /usr/lib/golang/src/testing/testing.go:1178 +0x386
  2655. 2020-11-02T12:04:54.8218959Z ------------------------------
  2656. 2020-11-02T12:04:54.8219760Z [redundancy] Gateway fail-over tests when a new node is labeled as a gateway node and the label on the existing gateway node is removed
  2657. 2020-11-02T12:04:54.8220904Z should start a submariner engine on the new gateway node and be able to connect from another cluster
  2658. 2020-11-02T12:04:54.8221766Z /go/src/github.com/submariner-io/submariner/test/e2e/redundancy/gateway_failover.go:37
  2659. 2020-11-02T12:04:54.8222607Z STEP: Creating namespace objects with basename "gateway-redundancy"
  2660. 2020-11-02T12:04:54.8235000Z STEP: Generated namespace "e2e-tests-gateway-redundancy-pxlnz" in cluster "cluster2" to execute the tests in
  2661. 2020-11-02T12:04:54.8236956Z STEP: Creating namespace "e2e-tests-gateway-redundancy-pxlnz" in cluster "cluster3"
  2662. 2020-11-02T12:04:54.8581165Z STEP: Creating namespace "e2e-tests-gateway-redundancy-pxlnz" in cluster "cluster1"
  2663. 2020-11-02T12:04:54.8794128Z STEP: Found gateway node "cluster2-worker" on "cluster2"
  2664. 2020-11-02T12:04:54.8884858Z STEP: Found non-gateway node "cluster2-worker2" on "cluster2"
  2665. 2020-11-02T12:04:54.8969528Z STEP: Found submariner engine pod "submariner-gateway-kcnxr" on "cluster2"
  2666. 2020-11-02T12:04:54.9020616Z STEP: Found submariner endpoint for "cluster2": &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"Endpoint", APIVersion:"submariner.io/v1"}, ObjectMeta:v1.ObjectMeta{Name:"cluster2-submariner-cable-cluster2-172-17-0-4", GenerateName:"", Namespace:"submariner-operator", SelfLink:"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", UID:"8e4ce5be-6521-4f69-bef3-cde34ed638a9", ResourceVersion:"2845", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63739915300, loc:(*time.Location)(0x2035440)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster2", CableName:"submariner-cable-cluster2-172-17-0-4", Hostname:"cluster2-worker", Subnets:[]string{"100.2.0.0/16", "10.2.0.0/16"}, PrivateIP:"172.17.0.4", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string(nil)}}
  2667. 2020-11-02T12:04:54.9025763Z STEP: Setting the gateway label for node "cluster2-worker2" to true
  2668. 2020-11-02T12:04:54.9321173Z STEP: Ensuring that two Gateways become available in cluster "cluster2"
  2669. 2020-11-02T12:05:10.0557274Z STEP: Setting the gateway label for node "cluster2-worker" to false
  2670. 2020-11-02T12:05:10.0685978Z STEP: Verifying that the gateway "cluster2-worker" was deleted
  2671. 2020-11-02T12:05:15.1575028Z STEP: Found new submariner engine pod "submariner-gateway-whn7f"
  2672. 2020-11-02T12:05:15.1576033Z STEP: Waiting for the new pod "submariner-gateway-whn7f" to report as active and fully connected
  2673. 2020-11-02T12:05:40.1652029Z STEP: Found new submariner endpoint for "cluster2": &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"Endpoint", APIVersion:"submariner.io/v1"}, ObjectMeta:v1.ObjectMeta{Name:"cluster2-submariner-cable-cluster2-172-17-0-11", GenerateName:"", Namespace:"submariner-operator", SelfLink:"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-11", UID:"4dbea653-9ed8-4f2e-9fd3-5ab655122bcd", ResourceVersion:"3729", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63739915530, loc:(*time.Location)(0x2035440)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster2", CableName:"submariner-cable-cluster2-172-17-0-11", Hostname:"cluster2-worker2", Subnets:[]string{"100.2.0.0/16", "10.2.0.0/16"}, PrivateIP:"172.17.0.11", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string(nil)}}
  2674. 2020-11-02T12:05:40.1658308Z STEP: Verifying TCP connectivity from gateway node on "cluster3" to gateway node on "cluster2"
  2675. 2020-11-02T12:05:40.1659637Z STEP: Creating a listener pod in cluster "cluster2", which will wait for a handshake over TCP
  2676. 2020-11-02T12:05:45.1811125Z Nov 2 12:05:45.180: INFO: Will send traffic to IP: 10.2.128.4
  2677. 2020-11-02T12:05:45.1812934Z STEP: Creating a connector pod in cluster "cluster3", which will attempt the specific UUID handshake over TCP
  2678. 2020-11-02T12:05:45.1916042Z STEP: Waiting for the connector pod "tcp-check-podfq77r" to exit, returning what connector sent
  2679. 2020-11-02T12:05:50.1968472Z Nov 2 12:05:50.196: INFO: Pod "tcp-check-podfq77r" output:
  2680. 2020-11-02T12:05:50.1968832Z 10.2.128.4 (10.2.128.4:1234) open
  2681. 2020-11-02T12:05:50.1969766Z [dataplane] listener says 29937eae-030a-4201-844a-f017c435561c
  2682. 2020-11-02T12:05:50.1970058Z
  2683. 2020-11-02T12:05:50.1970886Z STEP: Waiting for the listener pod "tcp-check-listener8qcsd" to exit, returning what listener sent
  2684. 2020-11-02T12:05:50.2022648Z Nov 2 12:05:50.200: INFO: Pod "tcp-check-listener8qcsd" output:
  2685. 2020-11-02T12:05:50.2023145Z listening on 0.0.0.0:1234 ...
  2686. 2020-11-02T12:05:50.2023447Z connect to 10.2.128.4:1234 from 10.3.192.3:34533 (10.3.192.3:34533)
  2687. 2020-11-02T12:05:50.2024303Z [dataplane] connector says 068c9c0b-b8c7-49e9-80c8-a719a546eeb8
  2688. 2020-11-02T12:05:50.2024799Z
  2689. 2020-11-02T12:05:50.2025241Z Nov 2 12:05:50.201: INFO: Connector pod has IP: 10.3.192.3
  2690. 2020-11-02T12:05:50.2027234Z STEP: Verifying that the listener got the connector's data and the connector got the listener's data
  2691. 2020-11-02T12:05:50.2035374Z STEP: Verifying the output of listener pod which must contain the source IP
  2692. 2020-11-02T12:05:50.2040385Z STEP: Verifying TCP connectivity from non-gateway node on "cluster3" to non-gateway node on "cluster2"
  2693. 2020-11-02T12:05:50.2058005Z STEP: Creating a listener pod in cluster "cluster2", which will wait for a handshake over TCP
  2694. 2020-11-02T12:05:55.2257140Z Nov 2 12:05:55.225: INFO: Will send traffic to IP: 10.2.0.5
  2695. 2020-11-02T12:05:55.2259377Z STEP: Creating a connector pod in cluster "cluster3", which will attempt the specific UUID handshake over TCP
  2696. 2020-11-02T12:05:55.2316808Z STEP: Waiting for the connector pod "tcp-check-pod452ks" to exit, returning what connector sent
  2697. 2020-11-02T12:06:00.2369085Z Nov 2 12:06:00.236: INFO: Pod "tcp-check-pod452ks" output:
  2698. 2020-11-02T12:06:00.2369994Z 10.2.0.5 (10.2.0.5:1234) open
  2699. 2020-11-02T12:06:00.2370861Z [dataplane] listener says 842b76f3-2080-401b-b17d-3e278b957f0c
  2700. 2020-11-02T12:06:00.2371393Z
  2701. 2020-11-02T12:06:00.2372678Z STEP: Waiting for the listener pod "tcp-check-listener27sqr" to exit, returning what listener sent
  2702. 2020-11-02T12:06:00.2398674Z Nov 2 12:06:00.239: INFO: Pod "tcp-check-listener27sqr" output:
  2703. 2020-11-02T12:06:00.2399444Z listening on 0.0.0.0:1234 ...
  2704. 2020-11-02T12:06:00.2400151Z connect to 10.2.0.5:1234 from 10.3.128.3:40045 (10.3.128.3:40045)
  2705. 2020-11-02T12:06:00.2400921Z [dataplane] connector says 8005781a-7dde-4d36-a3c0-38d4e3fcfead
  2706. 2020-11-02T12:06:00.2401389Z
  2707. 2020-11-02T12:06:00.2401980Z Nov 2 12:06:00.239: INFO: Connector pod has IP: 10.3.128.3
  2708. 2020-11-02T12:06:00.2403074Z STEP: Verifying that the listener got the connector's data and the connector got the listener's data
  2709. 2020-11-02T12:06:00.2404858Z STEP: Verifying the output of listener pod which must contain the source IP
  2710. 2020-11-02T12:06:00.2408868Z STEP: Deleting namespace "e2e-tests-gateway-redundancy-pxlnz" on cluster "cluster2"
  2711. 2020-11-02T12:06:00.2444076Z STEP: Deleting namespace "e2e-tests-gateway-redundancy-pxlnz" on cluster "cluster3"
  2712. 2020-11-02T12:06:00.2534858Z STEP: Deleting namespace "e2e-tests-gateway-redundancy-pxlnz" on cluster "cluster1"
  2713. 2020-11-02T12:06:00.2646950Z
  2714. 2020-11-02T12:06:00.2647572Z • [SLOW TEST:65.458 seconds]
  2715. 2020-11-02T12:06:00.2648031Z [redundancy] Gateway fail-over tests
  2716. 2020-11-02T12:06:00.2662999Z /go/src/github.com/submariner-io/submariner/test/e2e/redundancy/gateway_failover.go:18
  2717. 2020-11-02T12:06:00.2663726Z when a new node is labeled as a gateway node and the label on the existing gateway node is removed
  2718. 2020-11-02T12:06:00.2664754Z /go/src/github.com/submariner-io/submariner/test/e2e/redundancy/gateway_failover.go:36
  2719. 2020-11-02T12:06:00.2665604Z should start a submariner engine on the new gateway node and be able to connect from another cluster
  2720. 2020-11-02T12:06:00.2666648Z /go/src/github.com/submariner-io/submariner/test/e2e/redundancy/gateway_failover.go:37
  2721. 2020-11-02T12:06:00.2667464Z ------------------------------
  2722. 2020-11-02T12:06:00.2668342Z [dataplane] Basic TCP connectivity tests across clusters without discovery when a pod connects via TCP to a remote pod when the pod is not on a gateway and the remote pod is not on a gateway
  2723. 2020-11-02T12:06:00.2669419Z should have sent the expected data from the pod to the other pod
  2724. 2020-11-02T12:06:00.2670372Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_pod_connectivity.go:15
  2725. 2020-11-02T12:06:00.2671382Z STEP: Creating namespace objects with basename "dataplane-conn-nd"
  2726. 2020-11-02T12:06:00.2696201Z STEP: Generated namespace "e2e-tests-dataplane-conn-nd-rq7w4" in cluster "cluster2" to execute the tests in
  2727. 2020-11-02T12:06:00.2697594Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-rq7w4" in cluster "cluster3"
  2728. 2020-11-02T12:06:00.2886308Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-rq7w4" in cluster "cluster1"
  2729. 2020-11-02T12:06:00.2974808Z STEP: Creating a listener pod in cluster "cluster3", which will wait for a handshake over TCP
  2730. 2020-11-02T12:06:06.3408715Z Nov 2 12:06:06.340: INFO: Will send traffic to IP: 10.3.128.3
  2731. 2020-11-02T12:06:06.3410647Z STEP: Creating a connector pod in cluster "cluster2", which will attempt the specific UUID handshake over TCP
  2732. 2020-11-02T12:06:06.3462962Z STEP: Waiting for the connector pod "tcp-check-pod6cnd4" to exit, returning what connector sent
  2733. 2020-11-02T12:06:11.3601149Z Nov 2 12:06:11.359: INFO: Pod "tcp-check-pod6cnd4" output:
  2734. 2020-11-02T12:06:11.3601553Z 10.3.128.3 (10.3.128.3:1234) open
  2735. 2020-11-02T12:06:11.3602407Z [dataplane] listener says ed0829d3-36f5-4af5-aa0f-6d1b3d9f568a
  2736. 2020-11-02T12:06:11.3602759Z
  2737. 2020-11-02T12:06:11.3603472Z STEP: Waiting for the listener pod "tcp-check-listenergj692" to exit, returning what listener sent
  2738. 2020-11-02T12:06:11.3635714Z Nov 2 12:06:11.363: INFO: Pod "tcp-check-listenergj692" output:
  2739. 2020-11-02T12:06:11.3636179Z listening on 0.0.0.0:1234 ...
  2740. 2020-11-02T12:06:11.3636453Z connect to 10.3.128.3:1234 from 10.2.0.5:38215 (10.2.0.5:38215)
  2741. 2020-11-02T12:06:11.3637364Z [dataplane] connector says b43eab33-dddb-4684-944a-1a9c9fcbce30
  2742. 2020-11-02T12:06:11.3637750Z
  2743. 2020-11-02T12:06:11.3638025Z Nov 2 12:06:11.363: INFO: Connector pod has IP: 10.2.0.5
  2744. 2020-11-02T12:06:11.3638746Z STEP: Verifying that the listener got the connector's data and the connector got the listener's data
  2745. 2020-11-02T12:06:11.3643613Z STEP: Verifying the output of listener pod which must contain the source IP
  2746. 2020-11-02T12:06:11.3645418Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-rq7w4" on cluster "cluster2"
  2747. 2020-11-02T12:06:11.3678044Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-rq7w4" on cluster "cluster3"
  2748. 2020-11-02T12:06:11.3745640Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-rq7w4" on cluster "cluster1"
  2749. 2020-11-02T12:06:11.3808425Z •
  2750. 2020-11-02T12:06:11.3810728Z ------------------------------
  2751. 2020-11-02T12:06:11.3826371Z STEP: Creating namespace objects with basename "dataplane-conn-nd"
  2752. 2020-11-02T12:06:11.3829938Z STEP: Generated namespace "e2e-tests-dataplane-conn-nd-rq7w4" in cluster "cluster2" to execute the tests in
  2753. 2020-11-02T12:06:11.3831094Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-rq7w4" in cluster "cluster3"
  2754. 2020-11-02T12:06:11.3832776Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-rq7w4" in cluster "cluster1"
  2755. 2020-11-02T12:06:11.3833998Z STEP: Creating a listener pod in cluster "cluster3", which will wait for a handshake over TCP
  2756. 2020-11-02T12:06:11.3834548Z Nov 2 12:06:06.340: INFO: Will send traffic to IP: 10.3.128.3
  2757. 2020-11-02T12:06:11.3835543Z STEP: Creating a connector pod in cluster "cluster2", which will attempt the specific UUID handshake over TCP
  2758. 2020-11-02T12:06:11.3836473Z STEP: Waiting for the connector pod "tcp-check-pod6cnd4" to exit, returning what connector sent
  2759. 2020-11-02T12:06:11.3837532Z Nov 2 12:06:11.359: INFO: Pod "tcp-check-pod6cnd4" output:
  2760. 2020-11-02T12:06:11.3838131Z 10.3.128.3 (10.3.128.3:1234) open
  2761. 2020-11-02T12:06:11.3838723Z [dataplane] listener says ed0829d3-36f5-4af5-aa0f-6d1b3d9f568a
  2762. 2020-11-02T12:06:11.3839363Z
  2763. 2020-11-02T12:06:11.3840092Z STEP: Waiting for the listener pod "tcp-check-listenergj692" to exit, returning what listener sent
  2764. 2020-11-02T12:06:11.3841719Z Nov 2 12:06:11.363: INFO: Pod "tcp-check-listenergj692" output:
  2765. 2020-11-02T12:06:11.3842610Z listening on 0.0.0.0:1234 ...
  2766. 2020-11-02T12:06:11.3843390Z connect to 10.3.128.3:1234 from 10.2.0.5:38215 (10.2.0.5:38215)
  2767. 2020-11-02T12:06:11.3844870Z [dataplane] connector says b43eab33-dddb-4684-944a-1a9c9fcbce30
  2768. 2020-11-02T12:06:11.3845450Z
  2769. 2020-11-02T12:06:11.3845844Z Nov 2 12:06:11.363: INFO: Connector pod has IP: 10.2.0.5
  2770. 2020-11-02T12:06:11.3846898Z STEP: Verifying that the listener got the connector's data and the connector got the listener's data
  2771. 2020-11-02T12:06:11.3848198Z STEP: Verifying the output of listener pod which must contain the source IP
  2772. 2020-11-02T12:06:11.3849109Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-rq7w4" on cluster "cluster2"
  2773. 2020-11-02T12:06:11.3850172Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-rq7w4" on cluster "cluster3"
  2774. 2020-11-02T12:06:11.3851508Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-rq7w4" on cluster "cluster1"
  2775. 2020-11-02T12:06:11.3852038Z
  2776. 2020-11-02T12:06:11.3852164Z
  2777. 2020-11-02T12:06:11.3853154Z [dataplane] Basic TCP connectivity tests across clusters without discovery when a pod connects via TCP to a remote service when the pod is on a gateway and the remote service is on a gateway
  2778. 2020-11-02T12:06:11.3854226Z should have sent the expected data from the pod to the other pod
  2779. 2020-11-02T12:06:11.3855273Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_pod_connectivity.go:15
  2780. 2020-11-02T12:06:11.3856130Z STEP: Creating namespace objects with basename "dataplane-conn-nd"
  2781. 2020-11-02T12:06:11.3951535Z STEP: Generated namespace "e2e-tests-dataplane-conn-nd-cpwmp" in cluster "cluster2" to execute the tests in
  2782. 2020-11-02T12:06:11.3952823Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-cpwmp" in cluster "cluster3"
  2783. 2020-11-02T12:06:11.4338146Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-cpwmp" in cluster "cluster1"
  2784. 2020-11-02T12:06:11.4448457Z STEP: Creating a listener pod in cluster "cluster3", which will wait for a handshake over TCP
  2785. 2020-11-02T12:06:17.5145988Z STEP: Pointing a service ClusterIP to the listener pod in cluster "cluster3"
  2786. 2020-11-02T12:06:17.5289675Z Nov 2 12:06:17.528: INFO: Will send traffic to IP: 100.3.23.238
  2787. 2020-11-02T12:06:17.5293657Z STEP: Creating a connector pod in cluster "cluster2", which will attempt the specific UUID handshake over TCP
  2788. 2020-11-02T12:06:17.5587547Z STEP: Waiting for the connector pod "tcp-check-podqdq8b" to exit, returning what connector sent
  2789. 2020-11-02T12:06:22.5746651Z Nov 2 12:06:22.574: INFO: Pod "tcp-check-podqdq8b" output:
  2790. 2020-11-02T12:06:22.5747093Z 100.3.23.238 (100.3.23.238:1234) open
  2791. 2020-11-02T12:06:22.5747703Z [dataplane] listener says d0aadfd7-c98f-4847-9f13-162bdf968d6a
  2792. 2020-11-02T12:06:22.5748062Z
  2793. 2020-11-02T12:06:22.5748721Z STEP: Waiting for the listener pod "tcp-check-listenert489x" to exit, returning what listener sent
  2794. 2020-11-02T12:06:22.5766812Z Nov 2 12:06:22.576: INFO: Pod "tcp-check-listenert489x" output:
  2795. 2020-11-02T12:06:22.5767260Z listening on 0.0.0.0:1234 ...
  2796. 2020-11-02T12:06:22.5767525Z connect to 10.3.192.3:1234 from 10.2.128.4:34535 (10.2.128.4:34535)
  2797. 2020-11-02T12:06:22.5768108Z [dataplane] connector says 6b89aaa9-b193-4774-886b-550d94dc4eb1
  2798. 2020-11-02T12:06:22.5768411Z
  2799. 2020-11-02T12:06:22.5768660Z Nov 2 12:06:22.576: INFO: Connector pod has IP: 10.2.128.4
  2800. 2020-11-02T12:06:22.5774048Z STEP: Verifying that the listener got the connector's data and the connector got the listener's data
  2801. 2020-11-02T12:06:22.5775144Z STEP: Verifying the output of listener pod which must contain the source IP
  2802. 2020-11-02T12:06:22.5776324Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-cpwmp" on cluster "cluster2"
  2803. 2020-11-02T12:06:22.5806264Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-cpwmp" on cluster "cluster3"
  2804. 2020-11-02T12:06:22.5873498Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-cpwmp" on cluster "cluster1"
  2805. 2020-11-02T12:06:22.5924478Z •
  2806. 2020-11-02T12:06:22.5929894Z ------------------------------
  2807. 2020-11-02T12:06:22.5930495Z STEP: Creating namespace objects with basename "dataplane-conn-nd"
  2808. 2020-11-02T12:06:22.5931816Z STEP: Generated namespace "e2e-tests-dataplane-conn-nd-cpwmp" in cluster "cluster2" to execute the tests in
  2809. 2020-11-02T12:06:22.5933070Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-cpwmp" in cluster "cluster3"
  2810. 2020-11-02T12:06:22.5934135Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-cpwmp" in cluster "cluster1"
  2811. 2020-11-02T12:06:22.5935058Z STEP: Creating a listener pod in cluster "cluster3", which will wait for a handshake over TCP
  2812. 2020-11-02T12:06:22.5935813Z STEP: Pointing a service ClusterIP to the listener pod in cluster "cluster3"
  2813. 2020-11-02T12:06:22.5936282Z Nov 2 12:06:17.528: INFO: Will send traffic to IP: 100.3.23.238
  2814. 2020-11-02T12:06:22.5937001Z STEP: Creating a connector pod in cluster "cluster2", which will attempt the specific UUID handshake over TCP
  2815. 2020-11-02T12:06:22.5938041Z STEP: Waiting for the connector pod "tcp-check-podqdq8b" to exit, returning what connector sent
  2816. 2020-11-02T12:06:22.5938754Z Nov 2 12:06:22.574: INFO: Pod "tcp-check-podqdq8b" output:
  2817. 2020-11-02T12:06:22.5939077Z 100.3.23.238 (100.3.23.238:1234) open
  2818. 2020-11-02T12:06:22.5939627Z [dataplane] listener says d0aadfd7-c98f-4847-9f13-162bdf968d6a
  2819. 2020-11-02T12:06:22.5939997Z
  2820. 2020-11-02T12:06:22.5940777Z STEP: Waiting for the listener pod "tcp-check-listenert489x" to exit, returning what listener sent
  2821. 2020-11-02T12:06:22.5941593Z Nov 2 12:06:22.576: INFO: Pod "tcp-check-listenert489x" output:
  2822. 2020-11-02T12:06:22.5941977Z listening on 0.0.0.0:1234 ...
  2823. 2020-11-02T12:06:22.5942221Z connect to 10.3.192.3:1234 from 10.2.128.4:34535 (10.2.128.4:34535)
  2824. 2020-11-02T12:06:22.5942772Z [dataplane] connector says 6b89aaa9-b193-4774-886b-550d94dc4eb1
  2825. 2020-11-02T12:06:22.5943091Z
  2826. 2020-11-02T12:06:22.5943531Z Nov 2 12:06:22.576: INFO: Connector pod has IP: 10.2.128.4
  2827. 2020-11-02T12:06:22.5944242Z STEP: Verifying that the listener got the connector's data and the connector got the listener's data
  2828. 2020-11-02T12:06:22.5944963Z STEP: Verifying the output of listener pod which must contain the source IP
  2829. 2020-11-02T12:06:22.5945812Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-cpwmp" on cluster "cluster2"
  2830. 2020-11-02T12:06:22.5946822Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-cpwmp" on cluster "cluster3"
  2831. 2020-11-02T12:06:22.5947842Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-cpwmp" on cluster "cluster1"
  2832. 2020-11-02T12:06:22.5948324Z
  2833. 2020-11-02T12:06:22.5948438Z
  2834. 2020-11-02T12:06:22.5949497Z [dataplane-globalnet] Basic TCP connectivity tests across overlapping clusters without discovery when a pod connects via TCP to the globalIP of a remote service when the pod is not on a gateway and the remote service is on a gateway
  2835. 2020-11-02T12:06:22.5950973Z should have sent the expected data from the pod to the other pod
  2836. 2020-11-02T12:06:22.5951780Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:15
  2837. 2020-11-02T12:06:22.5952650Z STEP: Creating namespace objects with basename "dataplane-gn-conn-nd"
  2838. 2020-11-02T12:06:22.5980369Z STEP: Generated namespace "e2e-tests-dataplane-gn-conn-nd-9vmfx" in cluster "cluster2" to execute the tests in
  2839. 2020-11-02T12:06:22.5981868Z STEP: Creating namespace "e2e-tests-dataplane-gn-conn-nd-9vmfx" in cluster "cluster3"
  2840. 2020-11-02T12:06:22.6067028Z STEP: Creating namespace "e2e-tests-dataplane-gn-conn-nd-9vmfx" in cluster "cluster1"
  2841. 2020-11-02T12:06:22.6153653Z Nov 2 12:06:22.615: INFO: Globalnet is not enabled, skipping the test...
  2842. 2020-11-02T12:06:22.6161550Z STEP: Deleting namespace "e2e-tests-dataplane-gn-conn-nd-9vmfx" on cluster "cluster2"
  2843. 2020-11-02T12:06:22.6636463Z STEP: Deleting namespace "e2e-tests-dataplane-gn-conn-nd-9vmfx" on cluster "cluster3"
  2844. 2020-11-02T12:06:22.6803490Z STEP: Deleting namespace "e2e-tests-dataplane-gn-conn-nd-9vmfx" on cluster "cluster1"
  2845. 2020-11-02T12:06:22.6867040Z
  2846. 2020-11-02T12:06:22.6868000Z S [SKIPPING] [0.093 seconds]
  2847. 2020-11-02T12:06:22.6868925Z [dataplane-globalnet] Basic TCP connectivity tests across overlapping clusters without discovery
  2848. 2020-11-02T12:06:22.6869848Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:9
  2849. 2020-11-02T12:06:22.6870467Z when a pod connects via TCP to the globalIP of a remote service
  2850. 2020-11-02T12:06:22.6871483Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:33
  2851. 2020-11-02T12:06:22.6872102Z when the pod is not on a gateway and the remote service is on a gateway
  2852. 2020-11-02T12:06:22.6873066Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:43
  2853. 2020-11-02T12:06:22.6873878Z should have sent the expected data from the pod to the other pod [It]
  2854. 2020-11-02T12:06:22.6874691Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_gn_pod_connectivity.go:15
  2855. 2020-11-02T12:06:22.6875156Z
  2856. 2020-11-02T12:06:22.6875846Z Nov 2 12:06:22.615: Globalnet is not enabled, skipping the test...
  2857. 2020-11-02T12:06:22.6876129Z
  2858. 2020-11-02T12:06:22.6877036Z /go/src/github.com/submariner-io/submariner/vendor/github.com/submariner-io/shipyard/test/e2e/framework/logging.go:42
  2859. 2020-11-02T12:06:22.6877739Z ------------------------------
  2860. 2020-11-02T12:06:22.6878936Z [dataplane] Basic TCP connectivity tests across clusters without discovery when a pod with HostNetworking connects via TCP to a remote pod when the pod is on a gateway and the remote pod is not on a gateway
  2861. 2020-11-02T12:06:22.6879932Z should have sent the expected data from the pod to the other pod
  2862. 2020-11-02T12:06:22.6881236Z /go/src/github.com/submariner-io/submariner/test/e2e/dataplane/tcp_pod_connectivity.go:15
  2863. 2020-11-02T12:06:22.6882172Z STEP: Creating namespace objects with basename "dataplane-conn-nd"
  2864. 2020-11-02T12:06:22.7147417Z STEP: Generated namespace "e2e-tests-dataplane-conn-nd-72kjs" in cluster "cluster2" to execute the tests in
  2865. 2020-11-02T12:06:22.7148881Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-72kjs" in cluster "cluster3"
  2866. 2020-11-02T12:06:22.7369195Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-72kjs" in cluster "cluster1"
  2867. 2020-11-02T12:06:22.7448101Z STEP: Creating a listener pod in cluster "cluster3", which will wait for a handshake over TCP
  2868. 2020-11-02T12:06:28.8570143Z Nov 2 12:06:28.856: INFO: Will send traffic to IP: 10.3.128.3
  2869. 2020-11-02T12:06:28.8571872Z STEP: Creating a connector pod in cluster "cluster2", which will attempt the specific UUID handshake over TCP
  2870. 2020-11-02T12:06:28.8956714Z STEP: Waiting for the connector pod "tcp-check-podzx97t" to exit, returning what connector sent
  2871. 2020-11-02T12:06:33.9645674Z Nov 2 12:06:33.963: INFO: Pod "tcp-check-podzx97t" output:
  2872. 2020-11-02T12:06:33.9646491Z 10.3.128.3 (10.3.128.3:1234) open
  2873. 2020-11-02T12:06:33.9647538Z [dataplane] listener says c860d88a-b494-426f-924e-e9182b884c36
  2874. 2020-11-02T12:06:33.9649403Z
  2875. 2020-11-02T12:06:33.9650926Z STEP: Waiting for the listener pod "tcp-check-listenerh79tq" to exit, returning what listener sent
  2876. 2020-11-02T12:06:33.9677482Z Nov 2 12:06:33.967: INFO: Pod "tcp-check-listenerh79tq" output:
  2877. 2020-11-02T12:06:33.9678375Z listening on 0.0.0.0:1234 ...
  2878. 2020-11-02T12:06:33.9678918Z connect to 10.3.128.3:1234 from 10.2.128.0:43793 (10.2.128.0:43793)
  2879. 2020-11-02T12:06:33.9679856Z [dataplane] connector says a549da27-7b7c-427c-b95e-a49556f61dda
  2880. 2020-11-02T12:06:33.9680364Z
  2881. 2020-11-02T12:06:33.9680734Z Nov 2 12:06:33.967: INFO: Connector pod has IP: 172.17.0.11
  2882. 2020-11-02T12:06:33.9681720Z STEP: Verifying that the listener got the connector's data and the connector got the listener's data
  2883. 2020-11-02T12:06:33.9682756Z STEP: Verifying that globalIP annotation does not exist on the connector POD
  2884. 2020-11-02T12:06:33.9684058Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-72kjs" on cluster "cluster2"
  2885. 2020-11-02T12:06:33.9725067Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-72kjs" on cluster "cluster3"
  2886. 2020-11-02T12:06:33.9786041Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-72kjs" on cluster "cluster1"
  2887. 2020-11-02T12:06:33.9839882Z •
  2888. 2020-11-02T12:06:33.9840283Z ------------------------------
  2889. 2020-11-02T12:06:33.9840896Z STEP: Creating namespace objects with basename "dataplane-conn-nd"
  2890. 2020-11-02T12:06:33.9842939Z STEP: Generated namespace "e2e-tests-dataplane-conn-nd-72kjs" in cluster "cluster2" to execute the tests in
  2891. 2020-11-02T12:06:33.9844955Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-72kjs" in cluster "cluster3"
  2892. 2020-11-02T12:06:33.9847854Z STEP: Creating namespace "e2e-tests-dataplane-conn-nd-72kjs" in cluster "cluster1"
  2893. 2020-11-02T12:06:33.9848862Z STEP: Creating a listener pod in cluster "cluster3", which will wait for a handshake over TCP
  2894. 2020-11-02T12:06:33.9849383Z Nov 2 12:06:28.856: INFO: Will send traffic to IP: 10.3.128.3
  2895. 2020-11-02T12:06:33.9850140Z STEP: Creating a connector pod in cluster "cluster2", which will attempt the specific UUID handshake over TCP
  2896. 2020-11-02T12:06:33.9851256Z STEP: Waiting for the connector pod "tcp-check-podzx97t" to exit, returning what connector sent
  2897. 2020-11-02T12:06:33.9852052Z Nov 2 12:06:33.963: INFO: Pod "tcp-check-podzx97t" output:
  2898. 2020-11-02T12:06:33.9852401Z 10.3.128.3 (10.3.128.3:1234) open
  2899. 2020-11-02T12:06:33.9852951Z [dataplane] listener says c860d88a-b494-426f-924e-e9182b884c36
  2900. 2020-11-02T12:06:33.9853270Z
  2901. 2020-11-02T12:06:33.9853972Z STEP: Waiting for the listener pod "tcp-check-listenerh79tq" to exit, returning what listener sent
  2902. 2020-11-02T12:06:33.9855149Z Nov 2 12:06:33.967: INFO: Pod "tcp-check-listenerh79tq" output:
  2903. 2020-11-02T12:06:33.9855594Z listening on 0.0.0.0:1234 ...
  2904. 2020-11-02T12:06:33.9855875Z connect to 10.3.128.3:1234 from 10.2.128.0:43793 (10.2.128.0:43793)
  2905. 2020-11-02T12:06:33.9857107Z [dataplane] connector says a549da27-7b7c-427c-b95e-a49556f61dda
  2906. 2020-11-02T12:06:33.9857466Z
  2907. 2020-11-02T12:06:33.9857746Z Nov 2 12:06:33.967: INFO: Connector pod has IP: 172.17.0.11
  2908. 2020-11-02T12:06:33.9858503Z STEP: Verifying that the listener got the connector's data and the connector got the listener's data
  2909. 2020-11-02T12:06:33.9859369Z STEP: Verifying that globalIP annotation does not exist on the connector POD
  2910. 2020-11-02T12:06:33.9860349Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-72kjs" on cluster "cluster2"
  2911. 2020-11-02T12:06:33.9861680Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-72kjs" on cluster "cluster3"
  2912. 2020-11-02T12:06:33.9862827Z STEP: Deleting namespace "e2e-tests-dataplane-conn-nd-72kjs" on cluster "cluster1"
  2913. 2020-11-02T12:06:33.9863353Z
  2914. 2020-11-02T12:06:33.9929429Z
  2915. 2020-11-02T12:06:33.9930264Z JUnit path was configured: /go/src/github.com/submariner-io/submariner/output/e2e-junit.xml
  2916. 2020-11-02T12:06:33.9942208Z
  2917. 2020-11-02T12:06:33.9943040Z JUnit report was created: /go/src/github.com/submariner-io/submariner/output/e2e-junit.xml
  2918. 2020-11-02T12:06:33.9943445Z
  2919. 2020-11-02T12:06:33.9943584Z
  2920. 2020-11-02T12:06:33.9944162Z Summarizing 1 Failure:
  2921. 2020-11-02T12:06:33.9944558Z
  2922. 2020-11-02T12:06:33.9946145Z [Fail] [redundancy] Gateway fail-over tests when one gateway node is configured and the submariner engine pod fails [It] should start a new submariner engine pod and be able to connect from another cluster 
  2923. 2020-11-02T12:06:33.9947461Z /go/src/github.com/submariner-io/submariner/vendor/github.com/submariner-io/shipyard/test/e2e/framework/network_pods.go:159
  2924. 2020-11-02T12:06:33.9948177Z
  2925. 2020-11-02T12:06:33.9948761Z Ran 13 of 20 Specs in 397.822 seconds
  2926. 2020-11-02T12:06:33.9949683Z FAIL! -- 12 Passed | 1 Failed | 1 Pending | 6 Skipped
  2927. 2020-11-02T12:06:33.9950347Z --- FAIL: TestE2E (397.83s)
  2928. 2020-11-02T12:06:33.9950585Z FAIL
  2929. 2020-11-02T12:06:33.9985617Z exit status 1
  2930. 2020-11-02T12:06:33.9986373Z FAIL github.com/submariner-io/submariner/test/e2e 398.241s
  2931. 2020-11-02T12:06:34.1133791Z make: *** [/opt/shipyard/Makefile.inc:40: e2e] Error 1
  2932. 2020-11-02T12:06:34.1211793Z [submariner]$ make e2e -- using=operator libreswan
  2933. 2020-11-02T12:06:40.4196137Z time="2020-11-02T12:06:40Z" level=fatal msg="exit status 2"
  2934. 2020-11-02T12:06:40.4207058Z Makefile.dapper:14: recipe for target 'e2e' failed
  2935. 2020-11-02T12:06:40.4209328Z make: *** [e2e] Error 1
  2936. 2020-11-02T12:06:40.4255020Z ##[error]Process completed with exit code 2.
  2937. 2020-11-02T12:06:40.4352709Z ##[group]Run df -h
  2938. 2020-11-02T12:06:40.4353370Z df -h
  2939. 2020-11-02T12:06:40.4353587Z free -h
  2940. 2020-11-02T12:06:40.4354000Z make post-mortem
  2941. 2020-11-02T12:06:40.5932943Z shell: /bin/bash -e {0}
  2942. 2020-11-02T12:06:40.5933206Z ##[endgroup]
  2943. 2020-11-02T12:06:40.6074833Z Filesystem Size Used Avail Use% Mounted on
  2944. 2020-11-02T12:06:40.6075165Z udev 3.4G 0 3.4G 0% /dev
  2945. 2020-11-02T12:06:40.6075399Z tmpfs 696M 1.4M 695M 1% /run
  2946. 2020-11-02T12:06:40.6075636Z /dev/sdb1 84G 79G 4.7G 95% /
  2947. 2020-11-02T12:06:40.6075885Z tmpfs 3.4G 174M 3.3G 5% /dev/shm
  2948. 2020-11-02T12:06:40.6076137Z tmpfs 5.0M 0 5.0M 0% /run/lock
  2949. 2020-11-02T12:06:40.6076401Z tmpfs 3.4G 0 3.4G 0% /sys/fs/cgroup
  2950. 2020-11-02T12:06:40.6076675Z /dev/sdb15 105M 3.6M 101M 4% /boot/efi
  2951. 2020-11-02T12:06:40.6076928Z /dev/sda1 14G 4.1G 9.0G 32% /mnt
  2952. 2020-11-02T12:06:40.6110722Z total used free shared buff/cache available
  2953. 2020-11-02T12:06:40.6111143Z Mem: 6.8G 3.9G 270M 268M 2.7G 2.7G
  2954. 2020-11-02T12:06:40.6111423Z Swap: 0B 0B 0B
  2955. 2020-11-02T12:06:40.6143271Z ./.dapper -m bind make post-mortem
  2956. 2020-11-02T12:06:43.4629799Z Sending build context to Docker daemon 67.43MB
  2957. 2020-11-02T12:06:43.4631067Z
  2958. 2020-11-02T12:06:43.4725549Z Step 1/5 : FROM quay.io/submariner/shipyard-dapper-base:devel
  2959. 2020-11-02T12:06:43.4742886Z ---> 45327d47bb4a
  2960. 2020-11-02T12:06:43.4744651Z Step 2/5 : ENV DAPPER_ENV="REPO TAG QUAY_USERNAME QUAY_PASSWORD GITHUB_SHA CLUSTERS_ARGS DEPLOY_ARGS IMAGES_ARGS RELEASE_ARGS" DAPPER_SOURCE=/go/src/github.com/submariner-io/submariner DAPPER_DOCKER_SOCKET=true
  2961. 2020-11-02T12:06:43.4764706Z ---> Using cache
  2962. 2020-11-02T12:06:43.4765694Z ---> a9cebcf01e35
  2963. 2020-11-02T12:06:43.4766644Z Step 3/5 : ENV DAPPER_OUTPUT=${DAPPER_SOURCE}/output
  2964. 2020-11-02T12:06:43.4778782Z ---> Using cache
  2965. 2020-11-02T12:06:43.4780083Z ---> ff686b1d708f
  2966. 2020-11-02T12:06:43.4780608Z Step 4/5 : WORKDIR ${DAPPER_SOURCE}
  2967. 2020-11-02T12:06:43.4791381Z ---> Using cache
  2968. 2020-11-02T12:06:43.4791910Z ---> 3b7e8c3dc3c5
  2969. 2020-11-02T12:06:43.4792588Z Step 5/5 : ENTRYPOINT ["/opt/shipyard/scripts/entry"]
  2970. 2020-11-02T12:06:43.4810937Z ---> Using cache
  2971. 2020-11-02T12:06:43.4811360Z ---> 7f2abde6349f
  2972. 2020-11-02T12:06:43.5659284Z Successfully built 7f2abde6349f
  2973. 2020-11-02T12:06:43.5707424Z Successfully tagged submariner:HEAD
  2974. 2020-11-02T12:06:44.3258081Z [submariner]$ trap chown -R 1001:116 . exit
  2975. 2020-11-02T12:06:44.3264459Z [submariner]$ mkdir -p bin dist output
  2976. 2020-11-02T12:06:44.3319199Z [submariner]$ make post-mortem
  2977. 2020-11-02T12:06:44.3892070Z fatal: No names found, cannot describe anything.
  2978. 2020-11-02T12:06:44.4085996Z /opt/shipyard/scripts/post_mortem.sh
  2979. 2020-11-02T12:06:44.4145363Z [submariner]$ source /opt/shipyard/scripts/lib/utils
  2980. 2020-11-02T12:06:44.4159585Z [submariner]$ . /opt/shipyard/scripts/lib/source_only
  2981. 2020-11-02T12:06:44.4171068Z [submariner]$ script_name=utils
  2982. 2020-11-02T12:06:44.4183020Z [submariner]$ exec_name=post_mortem.sh
  2983. 2020-11-02T12:06:44.4203761Z [submariner]$ declare_kubeconfig
  2984. 2020-11-02T12:06:44.4214483Z [submariner]$ declare_kubeconfig
  2985. 2020-11-02T12:06:44.4222927Z [submariner]$ source /opt/shipyard/scripts/lib/kubecfg
  2986. 2020-11-02T12:06:44.4239875Z [submariner]$ export KUBECONFIG
  2987. 2020-11-02T12:06:44.4274457Z [submariner]$ KUBECONFIG=/go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster2:/go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster3:/go/src/github.com/submariner-io/submariner/output/kubeconfigs/kind-config-cluster1:
  2988. 2020-11-02T12:06:44.4291369Z [submariner]$ find /go/src/github.com/submariner-io/submariner/output/kubeconfigs -type f -printf %p:
  2989. 2020-11-02T12:06:44.4369965Z [submariner]$ kind get clusters
  2990. 2020-11-02T12:06:44.8823738Z [submariner]$ run_sequential cluster1 cluster2 cluster3 post_analyze
  2991. 2020-11-02T12:06:44.8834582Z [submariner]$ run_sequential cluster1 cluster2 cluster3 post_analyze
  2992. 2020-11-02T12:06:44.8846539Z [submariner]$ local cmnd=post_analyze
  2993. 2020-11-02T12:06:44.8886992Z [submariner]$ eval echo cluster1 cluster2 cluster3
  2994. 2020-11-02T12:06:44.8911539Z [submariner]$ [cluster1] post_analyze
  2995. 2020-11-02T12:06:44.8929146Z [submariner]$ [cluster1] sed s/^/[cluster1] /
  2996. 2020-11-02T12:06:44.8935024Z [submariner]$ [cluster1] post_analyze
  2997. 2020-11-02T12:06:44.8948368Z [submariner]$ [cluster1] kubectl get all --all-namespaces
  2998. 2020-11-02T12:06:44.8959494Z [submariner]$ [cluster1] kubectl get all --all-namespaces
  2999. 2020-11-02T12:06:44.8974830Z [submariner]$ [cluster1] command kubectl --context=cluster1 get all --all-namespaces
  3000. 2020-11-02T12:06:44.8983172Z [submariner]$ [cluster1] kubectl --context=cluster1 get all --all-namespaces
  3001. 2020-11-02T12:06:45.3448980Z [cluster1] ======================= Post mortem cluster1 =======================
  3002. 2020-11-02T12:06:45.3449743Z [cluster1] NAMESPACE NAME READY STATUS RESTARTS AGE
  3003. 2020-11-02T12:06:45.3451670Z [cluster1] kube-system pod/coredns-6955765f44-4trkn 1/1 Running 0 14m
  3004. 2020-11-02T12:06:45.3452948Z [cluster1] kube-system pod/coredns-6955765f44-lqn8h 1/1 Running 0 14m
  3005. 2020-11-02T12:06:45.3454397Z [cluster1] kube-system pod/etcd-cluster1-control-plane 1/1 Running 0 14m
  3006. 2020-11-02T12:06:45.3455469Z [cluster1] kube-system pod/kindnet-pmfpv 1/1 Running 0 14m
  3007. 2020-11-02T12:06:45.3456413Z [cluster1] kube-system pod/kindnet-wh6gn 1/1 Running 0 14m
  3008. 2020-11-02T12:06:45.3457592Z [cluster1] kube-system pod/kube-apiserver-cluster1-control-plane 1/1 Running 0 14m
  3009. 2020-11-02T12:06:45.3459176Z [cluster1] kube-system pod/kube-controller-manager-cluster1-control-plane 1/1 Running 0 14m
  3010. 2020-11-02T12:06:45.3460526Z [cluster1] kube-system pod/kube-proxy-266pl 1/1 Running 0 14m
  3011. 2020-11-02T12:06:45.3461472Z [cluster1] kube-system pod/kube-proxy-mq8dj 1/1 Running 0 14m
  3012. 2020-11-02T12:06:45.3463180Z [cluster1] kube-system pod/kube-scheduler-cluster1-control-plane 1/1 Running 0 14m
  3013. 2020-11-02T12:06:45.3464627Z [cluster1] local-path-storage pod/local-path-provisioner-7745554f7f-lxdzm 1/1 Running 0 14m
  3014. 2020-11-02T12:06:45.3465994Z [cluster1] submariner-operator pod/submariner-gateway-dtktc 1/1 Running 0 9m20s
  3015. 2020-11-02T12:06:45.3467529Z [cluster1] submariner-operator pod/submariner-lighthouse-agent-79ccdfd54-htpkb 0/1 ImagePullBackOff 0 9m18s
  3016. 2020-11-02T12:06:45.3469594Z [cluster1] submariner-operator pod/submariner-lighthouse-coredns-557485fbc-shr4s 0/1 ImagePullBackOff 0 9m17s
  3017. 2020-11-02T12:06:45.3471675Z [cluster1] submariner-operator pod/submariner-lighthouse-coredns-557485fbc-zt6ls 0/1 ImagePullBackOff 0 9m16s
  3018. 2020-11-02T12:06:45.3475123Z [cluster1] submariner-operator pod/submariner-operator-6675977db7-khlg6 1/1 Running 0 9m39s
  3019. 2020-11-02T12:06:45.3585733Z [cluster1] submariner-operator pod/submariner-routeagent-5vp7b 1/1 Running 0 9m19s
  3020. 2020-11-02T12:06:45.3587869Z [cluster1] submariner-operator pod/submariner-routeagent-c4hh4 1/1 Running 0 9m19s
  3021. 2020-11-02T12:06:45.3588695Z [cluster1]
  3022. 2020-11-02T12:06:45.3589551Z [cluster1] NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
  3023. 2020-11-02T12:06:45.3590318Z [cluster1] default service/kubernetes ClusterIP 100.1.0.1 <none> 443/TCP 15m
  3024. 2020-11-02T12:06:45.3591318Z [cluster1] kube-system service/kube-dns ClusterIP 100.1.0.10 <none> 53/UDP,53/TCP,9153/TCP 14m
  3025. 2020-11-02T12:06:45.3592835Z [cluster1] submariner-operator service/submariner-lighthouse-coredns ClusterIP 100.1.240.245 <none> 53/UDP 9m15s
  3026. 2020-11-02T12:06:45.3594872Z [cluster1] submariner-operator service/submariner-operator-metrics ClusterIP 100.1.152.254 <none> 8383/TCP,8686/TCP 9m28s
  3027. 2020-11-02T12:06:45.3595651Z [cluster1]
  3028. 2020-11-02T12:06:45.3596522Z [cluster1] NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
  3029. 2020-11-02T12:06:45.3597586Z [cluster1] kube-system daemonset.apps/kindnet 2 2 2 2 2 <none> 14m
  3030. 2020-11-02T12:06:45.3598674Z [cluster1] kube-system daemonset.apps/kube-proxy 2 2 2 2 2 beta.kubernetes.io/os=linux 14m
  3031. 2020-11-02T12:06:45.3599987Z [cluster1] submariner-operator daemonset.apps/submariner-gateway 1 1 1 1 1 submariner.io/gateway=true 9m20s
  3032. 2020-11-02T12:06:45.3601870Z [cluster1] submariner-operator daemonset.apps/submariner-routeagent 2 2 2 2 2 <none> 9m20s
  3033. 2020-11-02T12:06:45.3602987Z [cluster1]
  3034. 2020-11-02T12:06:45.3634891Z [submariner]$ [cluster1] kubectl get pods -A
  3035. 2020-11-02T12:06:45.3646676Z [submariner]$ [cluster1] tail -n +2
  3036. 2020-11-02T12:06:45.3652170Z [submariner]$ [cluster1] kubectl get pods -A
  3037. 2020-11-02T12:06:45.3668584Z [submariner]$ [cluster1] grep -v Running
  3038. 2020-11-02T12:06:45.3681267Z [submariner]$ [cluster1] command kubectl --context=cluster1 get pods -A
  3039. 2020-11-02T12:06:45.3696251Z [submariner]$ [cluster1] sed s/ */;/g
  3040. 2020-11-02T12:06:45.3706051Z [submariner]$ [cluster1] kubectl --context=cluster1 get pods -A
  3041. 2020-11-02T12:06:45.6871023Z [submariner]$ [cluster1] ns=submariner-operator
  3042. 2020-11-02T12:06:45.6884918Z [submariner]$ [cluster1] cut -f1 -d;
  3043. 2020-11-02T12:06:45.6921560Z [submariner]$ [cluster1] name=submariner-lighthouse-agent-79ccdfd54-htpkb
  3044. 2020-11-02T12:06:45.6936508Z [submariner]$ [cluster1] cut -f2 -d;
  3045. 2020-11-02T12:06:45.6958996Z [submariner]$ [cluster1] kubectl -n submariner-operator describe pod submariner-lighthouse-agent-79ccdfd54-htpkb
  3046. 2020-11-02T12:06:45.6968823Z [submariner]$ [cluster1] kubectl -n submariner-operator describe pod submariner-lighthouse-agent-79ccdfd54-htpkb
  3047. 2020-11-02T12:06:45.7015989Z [submariner]$ [cluster1] command kubectl --context=cluster1 -n submariner-operator describe pod submariner-lighthouse-agent-79ccdfd54-htpkb
  3048. 2020-11-02T12:06:45.7028987Z [submariner]$ [cluster1] kubectl --context=cluster1 -n submariner-operator describe pod submariner-lighthouse-agent-79ccdfd54-htpkb
  3049. 2020-11-02T12:06:46.0050438Z [cluster1] NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
  3050. 2020-11-02T12:06:46.0051283Z [cluster1] kube-system deployment.apps/coredns 2/2 2 2 14m
  3051. 2020-11-02T12:06:46.0052205Z [cluster1] local-path-storage deployment.apps/local-path-provisioner 1/1 1 1 14m
  3052. 2020-11-02T12:06:46.0054060Z [cluster1] submariner-operator deployment.apps/submariner-lighthouse-agent 0/1 1 0 9m19s
  3053. 2020-11-02T12:06:46.0055521Z [cluster1] submariner-operator deployment.apps/submariner-lighthouse-coredns 0/2 2 0 9m18s
  3054. 2020-11-02T12:06:46.0056984Z [cluster1] submariner-operator deployment.apps/submariner-operator 1/1 1 1 9m39s
  3055. 2020-11-02T12:06:46.0057558Z [cluster1]
  3056. 2020-11-02T12:06:46.0057952Z [cluster1] NAMESPACE NAME DESIRED CURRENT READY AGE
  3057. 2020-11-02T12:06:46.0058788Z [cluster1] kube-system replicaset.apps/coredns-6955765f44 2 2 2 14m
  3058. 2020-11-02T12:06:46.0059907Z [cluster1] local-path-storage replicaset.apps/local-path-provisioner-7745554f7f 1 1 1 14m
  3059. 2020-11-02T12:06:46.0061366Z [cluster1] submariner-operator replicaset.apps/submariner-lighthouse-agent-79ccdfd54 1 1 0 9m19s
  3060. 2020-11-02T12:06:46.0062970Z [cluster1] submariner-operator replicaset.apps/submariner-lighthouse-coredns-557485fbc 2 2 0 9m17s
  3061. 2020-11-02T12:06:46.0065580Z [cluster1] submariner-operator replicaset.apps/submariner-operator-6675977db7 1 1 1 9m39s
  3062. 2020-11-02T12:06:46.0067491Z [cluster1] ======================= submariner-lighthouse-agent-79ccdfd54-htpkb - submariner-operator ============================
  3063. 2020-11-02T12:06:46.0069274Z [cluster1] Name: submariner-lighthouse-agent-79ccdfd54-htpkb
  3064. 2020-11-02T12:06:46.0070321Z [cluster1] Namespace: submariner-operator
  3065. 2020-11-02T12:06:46.0070743Z [cluster1] Priority: 0
  3066. 2020-11-02T12:06:46.0071270Z [cluster1] Node: cluster1-worker/172.17.0.6
  3067. 2020-11-02T12:06:46.0071672Z [cluster1] Start Time: Mon, 02 Nov 2020 11:57:28 +0000
  3068. 2020-11-02T12:06:46.0072339Z [cluster1] Labels: app=submariner-lighthouse-agent
  3069. 2020-11-02T12:06:46.0073113Z [cluster1] component=submariner-lighthouse
  3070. 2020-11-02T12:06:46.0073780Z [cluster1] pod-template-hash=79ccdfd54
  3071. 2020-11-02T12:06:46.0074209Z [cluster1] Annotations: <none>
  3072. 2020-11-02T12:06:46.0074723Z [cluster1] Status: Pending
  3073. 2020-11-02T12:06:46.0075040Z [cluster1] IP: 10.1.1.3
  3074. 2020-11-02T12:06:46.0075314Z [cluster1] IPs:
  3075. 2020-11-02T12:06:46.0075596Z [cluster1] IP: 10.1.1.3
  3076. 2020-11-02T12:06:46.0076643Z [cluster1] Controlled By: ReplicaSet/submariner-lighthouse-agent-79ccdfd54
  3077. 2020-11-02T12:06:46.0077382Z [cluster1] Containers:
  3078. 2020-11-02T12:06:46.0078010Z [cluster1] submariner-lighthouse-agent:
  3079. 2020-11-02T12:06:46.0078677Z [cluster1] Container ID:
  3080. 2020-11-02T12:06:46.0079302Z [cluster1] Image: localhost:5000/lighthouse-agent:local
  3081. 2020-11-02T12:06:46.0079729Z [cluster1] Image ID:
  3082. 2020-11-02T12:06:46.0080027Z [cluster1] Port: <none>
  3083. 2020-11-02T12:06:46.0080341Z [cluster1] Host Port: <none>
  3084. 2020-11-02T12:06:46.0080670Z [cluster1] State: Waiting
  3085. 2020-11-02T12:06:46.0081067Z [cluster1] Reason: ImagePullBackOff
  3086. 2020-11-02T12:06:46.0081840Z [cluster1] Ready: False
  3087. 2020-11-02T12:06:46.0082148Z [cluster1] Restart Count: 0
  3088. 2020-11-02T12:06:46.0082487Z [cluster1] Environment:
  3089. 2020-11-02T12:06:46.0083418Z [cluster1] SUBMARINER_NAMESPACE: submariner-operator
  3090. 2020-11-02T12:06:46.0084838Z [cluster1] SUBMARINER_CLUSTERID: cluster1
  3091. 2020-11-02T12:06:46.0085776Z [cluster1] SUBMARINER_EXCLUDENS: submariner,kube-system,operators
  3092. 2020-11-02T12:06:46.0086509Z [cluster1] SUBMARINER_DEBUG: false
  3093. 2020-11-02T12:06:46.0086991Z [cluster1] SUBMARINER_GLOBALNET_ENABLED: false
  3094. 2020-11-02T12:06:46.0087465Z [cluster1] BROKER_K8S_APISERVER: 172.17.0.9:6443
  3095. 2020-11-02T12:06:46.0234294Z [cluster1] BROKER_K8S_APISERVERTOKEN: eyJhbGciOiJSUzI1NiIsImtpZCI6InhlWjlWdWVUMTJ5dnpDRDZhc09MU0M3YlJheHl4ZkNMdDhXaW40bDdrdmsifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJzdWJtYXJpbmVyLWs4cy1icm9rZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlY3JldC5uYW1lIjoiY2x1c3Rlci1jbHVzdGVyMS10b2tlbi1xdGdnbCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJjbHVzdGVyLWNsdXN0ZXIxIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiYjVlYzhlYzgtZDJjNi00YzUzLWI3OTktNDM5MTkxYjM4NTU3Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OnN1Ym1hcmluZXItazhzLWJyb2tlcjpjbHVzdGVyLWNsdXN0ZXIxIn0.jhfv1ZJIIo6YXvnN7ZWlYl3WTLD6UYxYy0gjSDI4TkWxFAdPvajZpH2Og05xX2RvhuZbZXfKUjb8U8Kfkfq7ASSjCPP_q3LHe4wTQhqz5GKXpoRnUwruRCtJst-pWpVMHaNPZQzloOimYTJJWXNvNvIZarlRpq3L5MJk9U2csB3TYfM9PdrgjSOZvX5-2TMl1Hjsuyh0ur0q4gH4F2h67yH_VQJgwQ7TpW1yFL7x4D0f0r44E15T5Rc9SLU2KdWxb8tOeCiASQX6psIyLuLYlWDa_RmA72OlYpkArtr7ReFzwu_Je6xN4ChvvinWn2xIw54ABZ6vc6YIK_36fKOElw
  3096. 2020-11-02T12:06:46.0401952Z [submariner]$ [cluster1] kubectl -n submariner-operator logs submariner-lighthouse-agent-79ccdfd54-htpkb
  3097. 2020-11-02T12:06:46.0403586Z [submariner]$ [cluster1] kubectl -n submariner-operator logs submariner-lighthouse-agent-79ccdfd54-htpkb
  3098. 2020-11-02T12:06:46.0405865Z [submariner]$ [cluster1] command kubectl --context=cluster1 -n submariner-operator logs submariner-lighthouse-agent-79ccdfd54-htpkb
  3099. 2020-11-02T12:06:46.0407966Z [submariner]$ [cluster1] kubectl --context=cluster1 -n submariner-operator logs submariner-lighthouse-agent-79ccdfd54-htpkb
  3100. 2020-11-02T12:06:46.3210519Z Error from server (BadRequest): container "submariner-lighthouse-agent" in pod "submariner-lighthouse-agent-79ccdfd54-htpkb" is waiting to start: trying and failing to pull image
  3101. 2020-11-02T12:06:46.3275279Z [submariner]$ [cluster1] ns=submariner-operator
  3102. 2020-11-02T12:06:46.3298387Z [submariner]$ [cluster1] cut -f1 -d;
  3103. 2020-11-02T12:06:46.3349739Z [submariner]$ [cluster1] name=submariner-lighthouse-coredns-557485fbc-shr4s
  3104. 2020-11-02T12:06:46.3370152Z [submariner]$ [cluster1] cut -f2 -d;
  3105. 2020-11-02T12:06:46.3395116Z [submariner]$ [cluster1] kubectl -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-shr4s
  3106. 2020-11-02T12:06:46.3405520Z [submariner]$ [cluster1] kubectl -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-shr4s
  3107. 2020-11-02T12:06:46.3420540Z [submariner]$ [cluster1] command kubectl --context=cluster1 -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-shr4s
  3108. 2020-11-02T12:06:46.3434895Z [submariner]$ [cluster1] kubectl --context=cluster1 -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-shr4s
  3109. 2020-11-02T12:06:46.6500443Z [cluster1] BROKER_K8S_REMOTENAMESPACE: submariner-k8s-broker
  3110. 2020-11-02T12:06:46.6968797Z [cluster1] BROKER_K8S_CA: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJd01URXdNakV4TlRBMU1sb1hEVE13TVRBek1URXhOVEExTWxvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTVUwClVkaVlaV0xudTBjLzRsL1pEVUk2L2dyd3ZMakQ3T2lJczJSSjdNZnNuMXdrb2U3UE1Od2Y4YjQ2Y1MrdkJxR0oKS2xjUHhnTWpkYWtGdndSb3ZjR2JVWHBsdUlLZkNMTWxVKzdYenhuUnc4TTRwaFBndkxCRjhWRzg4WFBJTXpEcgpsT2tZQXdmY2V2L2ZVeWdxdTNBNzJqd3lQNGFodm0rRHcyN1htQ3R5cUloRmVGc1dzOXRBWDdNYk1BYk53Z09ICm82L2VUcEREQ05zc0ZiMUJhaWJrNDJEbm1SUVVkNHdnYSt1UEZPS2oxTEF5LzZaSWExVkJYckJIb2xUVkVnOWYKV3BCZEF5Um5IWis4cUxZeWVtUXQ0RHJrUXBuRy83a3JNcVB5Q2tBRTVONGppTGVvbXpVZlpEWlVCNWxEY1BLMwpuS2JibXhDZkhCQXNqNVhrUWtFQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFLeGUzc3RHbE9PaVIvdFpFUWlBcHRkNE0wOFAKVGIrTDNEVFNLZGlnQjVTNWhFakR6L3RzTHI5NUhQbEMvQXNUeW9yWDV2eGJHeHdsNHVmb0hXRzVtM0xJL3JMUgpNVmFITExVSmwzaWZ0c0R1SXVMaFZMT2pIM3NOelhmcGdFZ0o5Q1VjalhsTTlkMm4wTGlzK24rdy83OTJvREpiClRsMCtxZTdDcDR0eXdNYzN4MWJlZzN5YTFUbEdxaUV1UkRDUFZaWU5wazd3NlpReGN4dUlhRDQ1SW5YVHVqV3gKTHpCdURFSlErQlFkR2VORDc5T1k4SEdyZEYyaG5TdEdBcFMwUUtRWktNZ2ppOEo5MW5VQzNRNFZXT1B4U2Q0LwpCRENHdUJ4RzhXTTEvak5LdCs1MmJWeDEvdGF6VEdrSWhXVnd1UHl2c1IxRk5UWUpDTkRFVkhGT3A3MD0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
  3111. 2020-11-02T12:06:46.7484807Z [cluster1] Mounts:
  3112. 2020-11-02T12:06:46.7486571Z [cluster1] /var/run/secrets/kubernetes.io/serviceaccount from submariner-lighthouse-token-vvdp7 (ro)
  3113. 2020-11-02T12:06:46.7487365Z [cluster1] Conditions:
  3114. 2020-11-02T12:06:46.7487694Z [cluster1] Type Status
  3115. 2020-11-02T12:06:46.7488039Z [cluster1] Initialized True
  3116. 2020-11-02T12:06:46.7488382Z [cluster1] Ready False
  3117. 2020-11-02T12:06:46.7488797Z [cluster1] ContainersReady False
  3118. 2020-11-02T12:06:46.7489189Z [cluster1] PodScheduled True
  3119. 2020-11-02T12:06:46.7489534Z [cluster1] Volumes:
  3120. 2020-11-02T12:06:46.7490267Z [cluster1] submariner-lighthouse-token-vvdp7:
  3121. 2020-11-02T12:06:46.7490921Z [cluster1] Type: Secret (a volume populated by a Secret)
  3122. 2020-11-02T12:06:46.7491786Z [cluster1] SecretName: submariner-lighthouse-token-vvdp7
  3123. 2020-11-02T12:06:46.7492401Z [cluster1] Optional: false
  3124. 2020-11-02T12:06:46.7492938Z [cluster1] QoS Class: BestEffort
  3125. 2020-11-02T12:06:46.7493683Z [cluster1] Node-Selectors: <none>
  3126. 2020-11-02T12:06:46.7494441Z [cluster1] Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
  3127. 2020-11-02T12:06:46.7495483Z [cluster1] node.kubernetes.io/unreachable:NoExecute for 300s
  3128. 2020-11-02T12:06:46.7495948Z [cluster1] Events:
  3129. 2020-11-02T12:06:46.7496306Z [cluster1] Type Reason Age From Message
  3130. 2020-11-02T12:06:46.7496866Z [cluster1] ---- ------ ---- ---- -------
  3131. 2020-11-02T12:06:46.7498290Z [cluster1] Normal Scheduled 9m18s default-scheduler Successfully assigned submariner-operator/submariner-lighthouse-agent-79ccdfd54-htpkb to cluster1-worker
  3132. 2020-11-02T12:06:46.7500010Z [cluster1] Normal Pulling 7m4s (x4 over 9m14s) kubelet, cluster1-worker Pulling image "localhost:5000/lighthouse-agent:local"
  3133. 2020-11-02T12:06:46.7502453Z [cluster1] Warning Failed 7m4s (x4 over 8m27s) kubelet, cluster1-worker Failed to pull image "localhost:5000/lighthouse-agent:local": rpc error: code = Unknown desc = failed to pull and unpack image "localhost:5000/lighthouse-agent:local": failed to resolve reference "localhost:5000/lighthouse-agent:local": failed to do request: Head https://localhost:5000/v2/lighthouse-agent/manifests/local: dial tcp 127.0.0.1:5000: connect: connection refused
  3134. 2020-11-02T12:06:46.7504708Z [cluster1] Warning Failed 7m4s (x4 over 8m27s) kubelet, cluster1-worker Error: ErrImagePull
  3135. 2020-11-02T12:06:46.7506403Z [cluster1] Normal BackOff 6m38s (x6 over 8m26s) kubelet, cluster1-worker Back-off pulling image "localhost:5000/lighthouse-agent:local"
  3136. 2020-11-02T12:06:46.7507520Z [cluster1] Warning Failed 4m13s (x16 over 8m26s) kubelet, cluster1-worker Error: ImagePullBackOff
  3137. 2020-11-02T12:06:46.7508864Z [cluster1] ===================== END submariner-lighthouse-agent-79ccdfd54-htpkb - submariner-operator ==========================
  3138. 2020-11-02T12:06:46.7510517Z [cluster1] ======================= submariner-lighthouse-coredns-557485fbc-shr4s - submariner-operator ============================
  3139. 2020-11-02T12:06:46.7520490Z [cluster1] Name: submariner-lighthouse-coredns-557485fbc-shr4s
  3140. 2020-11-02T12:06:46.7521565Z [cluster1] Namespace: submariner-operator
  3141. 2020-11-02T12:06:46.7521993Z [cluster1] Priority: 0
  3142. 2020-11-02T12:06:46.7522522Z [cluster1] Node: cluster1-worker/172.17.0.6
  3143. 2020-11-02T12:06:46.7524502Z [submariner]$ [cluster1] kubectl -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-shr4s
  3144. 2020-11-02T12:06:46.7526582Z [submariner]$ [cluster1] kubectl -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-shr4s
  3145. 2020-11-02T12:06:46.7528766Z [submariner]$ [cluster1] command kubectl --context=cluster1 -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-shr4s
  3146. 2020-11-02T12:06:46.7531718Z [submariner]$ [cluster1] kubectl --context=cluster1 -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-shr4s
  3147. 2020-11-02T12:06:47.0008766Z Error from server (BadRequest): container "submariner-lighthouse-coredns" in pod "submariner-lighthouse-coredns-557485fbc-shr4s" is waiting to start: trying and failing to pull image
  3148. 2020-11-02T12:06:47.0113054Z [submariner]$ [cluster1] ns=submariner-operator
  3149. 2020-11-02T12:06:47.0136130Z [submariner]$ [cluster1] cut -f1 -d;
  3150. 2020-11-02T12:06:47.0179593Z [submariner]$ [cluster1] name=submariner-lighthouse-coredns-557485fbc-zt6ls
  3151. 2020-11-02T12:06:47.0210817Z [submariner]$ [cluster1] cut -f2 -d;
  3152. 2020-11-02T12:06:47.0248851Z [submariner]$ [cluster1] kubectl -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-zt6ls
  3153. 2020-11-02T12:06:47.0261012Z [submariner]$ [cluster1] kubectl -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-zt6ls
  3154. 2020-11-02T12:06:47.0277623Z [submariner]$ [cluster1] command kubectl --context=cluster1 -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-zt6ls
  3155. 2020-11-02T12:06:47.0293046Z [submariner]$ [cluster1] kubectl --context=cluster1 -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-zt6ls
  3156. 2020-11-02T12:06:47.3347519Z [cluster1] Start Time: Mon, 02 Nov 2020 11:57:30 +0000
  3157. 2020-11-02T12:06:47.3350985Z [cluster1] Labels: app=submariner-lighthouse-coredns
  3158. 2020-11-02T12:06:47.3351881Z [cluster1] component=submariner-lighthouse
  3159. 2020-11-02T12:06:47.3352484Z [cluster1] pod-template-hash=557485fbc
  3160. 2020-11-02T12:06:47.3352858Z [cluster1] Annotations: <none>
  3161. 2020-11-02T12:06:47.3353179Z [cluster1] Status: Pending
  3162. 2020-11-02T12:06:47.3353438Z [cluster1] IP: 10.1.1.4
  3163. 2020-11-02T12:06:47.3353680Z [cluster1] IPs:
  3164. 2020-11-02T12:06:47.3353933Z [cluster1] IP: 10.1.1.4
  3165. 2020-11-02T12:06:47.3354758Z [cluster1] Controlled By: ReplicaSet/submariner-lighthouse-coredns-557485fbc
  3166. 2020-11-02T12:06:47.3355440Z [cluster1] Containers:
  3167. 2020-11-02T12:06:47.3356039Z [cluster1] submariner-lighthouse-coredns:
  3168. 2020-11-02T12:06:47.3356508Z [cluster1] Container ID:
  3169. 2020-11-02T12:06:47.3357075Z [cluster1] Image: localhost:5000/lighthouse-coredns:local
  3170. 2020-11-02T12:06:47.3357462Z [cluster1] Image ID:
  3171. 2020-11-02T12:06:47.3357734Z [cluster1] Port: <none>
  3172. 2020-11-02T12:06:47.3358018Z [cluster1] Host Port: <none>
  3173. 2020-11-02T12:06:47.3358522Z [cluster1] Args:
  3174. 2020-11-02T12:06:47.3358922Z [cluster1] -conf
  3175. 2020-11-02T12:06:47.3359213Z [cluster1] /etc/coredns/Corefile
  3176. 2020-11-02T12:06:47.3359530Z [cluster1] State: Waiting
  3177. 2020-11-02T12:06:47.3359870Z [cluster1] Reason: ImagePullBackOff
  3178. 2020-11-02T12:06:47.3360414Z [cluster1] Ready: False
  3179. 2020-11-02T12:06:47.3360719Z [cluster1] Restart Count: 0
  3180. 2020-11-02T12:06:47.3361037Z [cluster1] Environment: <none>
  3181. 2020-11-02T12:06:47.3361332Z [cluster1] Mounts:
  3182. 2020-11-02T12:06:47.3361850Z [cluster1] /etc/coredns from config-volume (ro)
  3183. 2020-11-02T12:06:47.3362784Z [cluster1] /var/run/secrets/kubernetes.io/serviceaccount from submariner-lighthouse-token-vvdp7 (ro)
  3184. 2020-11-02T12:06:47.3363489Z [cluster1] Conditions:
  3185. 2020-11-02T12:06:47.3363789Z [cluster1] Type Status
  3186. 2020-11-02T12:06:47.3364537Z [cluster1] Initialized True
  3187. 2020-11-02T12:06:47.3364855Z [cluster1] Ready False
  3188. 2020-11-02T12:06:47.3365204Z [cluster1] ContainersReady False
  3189. 2020-11-02T12:06:47.3365608Z [cluster1] PodScheduled True
  3190. 2020-11-02T12:06:47.3365916Z [cluster1] Volumes:
  3191. 2020-11-02T12:06:47.3366381Z [cluster1] config-volume:
  3192. 2020-11-02T12:06:47.3366803Z [cluster1] Type: ConfigMap (a volume populated by a ConfigMap)
  3193. 2020-11-02T12:06:47.3367647Z [cluster1] Name: submariner-lighthouse-coredns
  3194. 2020-11-02T12:06:47.3368119Z [cluster1] Optional: false
  3195. 2020-11-02T12:06:47.3368951Z [cluster1] submariner-lighthouse-token-vvdp7:
  3196. 2020-11-02T12:06:47.3369547Z [cluster1] Type: Secret (a volume populated by a Secret)
  3197. 2020-11-02T12:06:47.3370607Z [cluster1] SecretName: submariner-lighthouse-token-vvdp7
  3198. 2020-11-02T12:06:47.3371166Z [cluster1] Optional: false
  3199. 2020-11-02T12:06:47.3371492Z [cluster1] QoS Class: BestEffort
  3200. 2020-11-02T12:06:47.3371970Z [cluster1] Node-Selectors: <none>
  3201. 2020-11-02T12:06:47.3372942Z [cluster1] Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
  3202. 2020-11-02T12:06:47.3373742Z [cluster1] node.kubernetes.io/unreachable:NoExecute for 300s
  3203. 2020-11-02T12:06:47.3374527Z [cluster1] Events:
  3204. 2020-11-02T12:06:47.3374871Z [cluster1] Type Reason Age From Message
  3205. 2020-11-02T12:06:47.3375425Z [cluster1] ---- ------ ---- ---- -------
  3206. 2020-11-02T12:06:47.3377302Z [cluster1] Normal Scheduled 9m16s default-scheduler Successfully assigned submariner-operator/submariner-lighthouse-coredns-557485fbc-shr4s to cluster1-worker
  3207. 2020-11-02T12:06:47.3379041Z [cluster1] Normal Pulling 6m52s (x4 over 9m13s) kubelet, cluster1-worker Pulling image "localhost:5000/lighthouse-coredns:local"
  3208. 2020-11-02T12:06:47.3381692Z [cluster1] Warning Failed 6m52s (x4 over 8m27s) kubelet, cluster1-worker Failed to pull image "localhost:5000/lighthouse-coredns:local": rpc error: code = Unknown desc = failed to pull and unpack image "localhost:5000/lighthouse-coredns:local": failed to resolve reference "localhost:5000/lighthouse-coredns:local": failed to do request: Head https://localhost:5000/v2/lighthouse-coredns/manifests/local: dial tcp 127.0.0.1:5000: connect: connection refused
  3209. 2020-11-02T12:06:47.3384819Z [cluster1] Warning Failed 6m52s (x4 over 8m27s) kubelet, cluster1-worker Error: ErrImagePull
  3210. 2020-11-02T12:06:47.3386193Z [cluster1] Normal BackOff 6m26s (x7 over 8m26s) kubelet, cluster1-worker Back-off pulling image "localhost:5000/lighthouse-coredns:local"
  3211. 2020-11-02T12:06:47.3387232Z [cluster1] Warning Failed 4m2s (x17 over 8m26s) kubelet, cluster1-worker Error: ImagePullBackOff
  3212. 2020-11-02T12:06:47.3388635Z [cluster1] ===================== END submariner-lighthouse-coredns-557485fbc-shr4s - submariner-operator ==========================
  3213. 2020-11-02T12:06:47.3390255Z [submariner]$ [cluster1] kubectl -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-zt6ls
  3214. 2020-11-02T12:06:47.3396450Z [cluster1] ======================= submariner-lighthouse-coredns-557485fbc-zt6ls - submariner-operator ============================
  3215. 2020-11-02T12:06:47.3397986Z [cluster1] Name: submariner-lighthouse-coredns-557485fbc-zt6ls
  3216. 2020-11-02T12:06:47.3398904Z [cluster1] Namespace: submariner-operator
  3217. 2020-11-02T12:06:47.3399285Z [cluster1] Priority: 0
  3218. 2020-11-02T12:06:47.3400301Z [cluster1] Node: cluster1-worker/172.17.0.6
  3219. 2020-11-02T12:06:47.3400693Z [cluster1] Start Time: Mon, 02 Nov 2020 11:57:30 +0000
  3220. 2020-11-02T12:06:47.3401362Z [cluster1] Labels: app=submariner-lighthouse-coredns
  3221. 2020-11-02T12:06:47.3402139Z [cluster1] component=submariner-lighthouse
  3222. 2020-11-02T12:06:47.3402767Z [cluster1] pod-template-hash=557485fbc
  3223. 2020-11-02T12:06:47.3404243Z [submariner]$ [cluster1] kubectl -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-zt6ls
  3224. 2020-11-02T12:06:47.3409687Z [submariner]$ [cluster1] command kubectl --context=cluster1 -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-zt6ls
  3225. 2020-11-02T12:06:47.3424484Z [submariner]$ [cluster1] kubectl --context=cluster1 -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-zt6ls
  3226. 2020-11-02T12:06:47.6571441Z Error from server (BadRequest): container "submariner-lighthouse-coredns" in pod "submariner-lighthouse-coredns-557485fbc-zt6ls" is waiting to start: trying and failing to pull image
  3227. 2020-11-02T12:06:47.6608295Z [submariner]$ [cluster1] namespace=kube-system
  3228. 2020-11-02T12:06:47.6628059Z [submariner]$ [cluster1] kubectl get pods --selector=k8s-app=kube-proxy -n kube-system -o jsonpath={.items[*].metadata.name}
  3229. 2020-11-02T12:06:47.6640166Z [submariner]$ [cluster1] kubectl get pods --selector=k8s-app=kube-proxy -n kube-system -o jsonpath={.items[*].metadata.name}
  3230. 2020-11-02T12:06:47.6655909Z [submariner]$ [cluster1] command kubectl --context=cluster1 get pods --selector=k8s-app=kube-proxy -n kube-system -o jsonpath={.items[*].metadata.name}
  3231. 2020-11-02T12:06:47.6676159Z [submariner]$ [cluster1] kubectl --context=cluster1 get pods --selector=k8s-app=kube-proxy -n kube-system -o jsonpath={.items[*].metadata.name}
  3232. 2020-11-02T12:06:47.9593413Z [submariner]$ [cluster1] kubectl -n kube-system logs kube-proxy-266pl
  3233. 2020-11-02T12:06:47.9605323Z [submariner]$ [cluster1] kubectl -n kube-system logs kube-proxy-266pl
  3234. 2020-11-02T12:06:47.9616102Z [submariner]$ [cluster1] command kubectl --context=cluster1 -n kube-system logs kube-proxy-266pl
  3235. 2020-11-02T12:06:47.9627300Z [submariner]$ [cluster1] kubectl --context=cluster1 -n kube-system logs kube-proxy-266pl
  3236. 2020-11-02T12:06:48.2545994Z [cluster1] Annotations: <none>
  3237. 2020-11-02T12:06:48.2546381Z [cluster1] Status: Pending
  3238. 2020-11-02T12:06:48.2546655Z [cluster1] IP: 10.1.1.5
  3239. 2020-11-02T12:06:48.2546909Z [cluster1] IPs:
  3240. 2020-11-02T12:06:48.2547161Z [cluster1] IP: 10.1.1.5
  3241. 2020-11-02T12:06:48.2548469Z [cluster1] Controlled By: ReplicaSet/submariner-lighthouse-coredns-557485fbc
  3242. 2020-11-02T12:06:48.2549178Z [cluster1] Containers:
  3243. 2020-11-02T12:06:48.2549806Z [cluster1] submariner-lighthouse-coredns:
  3244. 2020-11-02T12:06:48.2550292Z [cluster1] Container ID:
  3245. 2020-11-02T12:06:48.2550881Z [cluster1] Image: localhost:5000/lighthouse-coredns:local
  3246. 2020-11-02T12:06:48.2551291Z [cluster1] Image ID:
  3247. 2020-11-02T12:06:48.2551573Z [cluster1] Port: <none>
  3248. 2020-11-02T12:06:48.2551865Z [cluster1] Host Port: <none>
  3249. 2020-11-02T12:06:48.2552141Z [cluster1] Args:
  3250. 2020-11-02T12:06:48.2552521Z [cluster1] -conf
  3251. 2020-11-02T12:06:48.2552823Z [cluster1] /etc/coredns/Corefile
  3252. 2020-11-02T12:06:48.2553149Z [cluster1] State: Waiting
  3253. 2020-11-02T12:06:48.2553499Z [cluster1] Reason: ImagePullBackOff
  3254. 2020-11-02T12:06:48.2553861Z [cluster1] Ready: False
  3255. 2020-11-02T12:06:48.2554166Z [cluster1] Restart Count: 0
  3256. 2020-11-02T12:06:48.2554731Z [cluster1] Environment: <none>
  3257. 2020-11-02T12:06:48.2555038Z [cluster1] Mounts:
  3258. 2020-11-02T12:06:48.2555580Z [cluster1] /etc/coredns from config-volume (ro)
  3259. 2020-11-02T12:06:48.2556539Z [cluster1] /var/run/secrets/kubernetes.io/serviceaccount from submariner-lighthouse-token-vvdp7 (ro)
  3260. 2020-11-02T12:06:48.2557265Z [cluster1] Conditions:
  3261. 2020-11-02T12:06:48.2557580Z [cluster1] Type Status
  3262. 2020-11-02T12:06:48.2557883Z [cluster1] Initialized True
  3263. 2020-11-02T12:06:48.2558200Z [cluster1] Ready False
  3264. 2020-11-02T12:06:48.2558551Z [cluster1] ContainersReady False
  3265. 2020-11-02T12:06:48.2558932Z [cluster1] PodScheduled True
  3266. 2020-11-02T12:06:48.2559249Z [cluster1] Volumes:
  3267. 2020-11-02T12:06:48.2559692Z [cluster1] config-volume:
  3268. 2020-11-02T12:06:48.2560128Z [cluster1] Type: ConfigMap (a volume populated by a ConfigMap)
  3269. 2020-11-02T12:06:48.2560846Z [cluster1] Name: submariner-lighthouse-coredns
  3270. 2020-11-02T12:06:48.2561338Z [cluster1] Optional: false
  3271. 2020-11-02T12:06:48.2562019Z [cluster1] submariner-lighthouse-token-vvdp7:
  3272. 2020-11-02T12:06:48.2562630Z [cluster1] Type: Secret (a volume populated by a Secret)
  3273. 2020-11-02T12:06:48.2563419Z [cluster1] SecretName: submariner-lighthouse-token-vvdp7
  3274. 2020-11-02T12:06:48.2564492Z [cluster1] Optional: false
  3275. 2020-11-02T12:06:48.2564964Z [cluster1] QoS Class: BestEffort
  3276. 2020-11-02T12:06:48.2565537Z [cluster1] Node-Selectors: <none>
  3277. 2020-11-02T12:06:48.2566195Z [cluster1] Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
  3278. 2020-11-02T12:06:48.2566825Z [cluster1] node.kubernetes.io/unreachable:NoExecute for 300s
  3279. 2020-11-02T12:06:48.2567258Z [cluster1] Events:
  3280. 2020-11-02T12:06:48.2567589Z [cluster1] Type Reason Age From Message
  3281. 2020-11-02T12:06:48.2568098Z [cluster1] ---- ------ ---- ---- -------
  3282. 2020-11-02T12:06:48.2569534Z [cluster1] Normal Scheduled 9m17s default-scheduler Successfully assigned submariner-operator/submariner-lighthouse-coredns-557485fbc-zt6ls to cluster1-worker
  3283. 2020-11-02T12:06:48.2571185Z [cluster1] Normal Pulling 7m3s (x4 over 9m13s) kubelet, cluster1-worker Pulling image "localhost:5000/lighthouse-coredns:local"
  3284. 2020-11-02T12:06:48.2573551Z [cluster1] Warning Failed 7m3s (x4 over 8m28s) kubelet, cluster1-worker Failed to pull image "localhost:5000/lighthouse-coredns:local": rpc error: code = Unknown desc = failed to pull and unpack image "localhost:5000/lighthouse-coredns:local": failed to resolve reference "localhost:5000/lighthouse-coredns:local": failed to do request: Head https://localhost:5000/v2/lighthouse-coredns/manifests/local: dial tcp 127.0.0.1:5000: connect: connection refused
  3285. 2020-11-02T12:06:48.2575490Z [cluster1] Warning Failed 7m3s (x4 over 8m28s) kubelet, cluster1-worker Error: ErrImagePull
  3286. 2020-11-02T12:06:48.2576409Z [cluster1] Warning Failed 6m24s (x7 over 8m27s) kubelet, cluster1-worker Error: ImagePullBackOff
  3287. 2020-11-02T12:06:48.2577503Z [cluster1] Normal BackOff 4m1s (x16 over 8m27s) kubelet, cluster1-worker Back-off pulling image "localhost:5000/lighthouse-coredns:local"
  3288. 2020-11-02T12:06:48.2579101Z [cluster1] ===================== END submariner-lighthouse-coredns-557485fbc-zt6ls - submariner-operator ==========================
  3289. 2020-11-02T12:06:48.2580315Z [cluster1] +++++++++++++++++++++: Logs for Pod kube-proxy-266pl in namespace kube-system :++++++++++++++++++++++
  3290. 2020-11-02T12:06:48.2580942Z [cluster1] W1102 11:52:34.405030 1 server_others.go:323] Unknown proxy mode "", assuming iptables proxy
  3291. 2020-11-02T12:06:48.2581516Z [cluster1] I1102 11:52:34.413436 1 node.go:135] Successfully retrieved node IP: 172.17.0.6
  3292. 2020-11-02T12:06:48.2582037Z [cluster1] I1102 11:52:34.413465 1 server_others.go:145] Using iptables Proxier.
  3293. 2020-11-02T12:06:48.2582508Z [cluster1] I1102 11:52:34.413844 1 server.go:571] Version: v1.17.0
  3294. 2020-11-02T12:06:48.2583089Z [cluster1] I1102 11:52:34.414514 1 conntrack.go:52] Setting nf_conntrack_max to 131072
  3295. 2020-11-02T12:06:48.2583984Z [cluster1] I1102 11:52:34.414899 1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_established' to 86400
  3296. 2020-11-02T12:06:48.2599407Z [submariner]$ [cluster1] kubectl -n kube-system logs kube-proxy-mq8dj
  3297. 2020-11-02T12:06:48.2613196Z [submariner]$ [cluster1] kubectl -n kube-system logs kube-proxy-mq8dj
  3298. 2020-11-02T12:06:48.2626957Z [submariner]$ [cluster1] command kubectl --context=cluster1 -n kube-system logs kube-proxy-mq8dj
  3299. 2020-11-02T12:06:48.2640345Z [submariner]$ [cluster1] kubectl --context=cluster1 -n kube-system logs kube-proxy-mq8dj
  3300. 2020-11-02T12:06:48.5760950Z [submariner]$ [cluster1] namespace=submariner-operator
  3301. 2020-11-02T12:06:48.5777433Z [submariner]$ [cluster1] kubectl get pods --selector=name=submariner-operator -n submariner-operator -o jsonpath={.items[*].metadata.name}
  3302. 2020-11-02T12:06:48.5789909Z [submariner]$ [cluster1] kubectl get pods --selector=name=submariner-operator -n submariner-operator -o jsonpath={.items[*].metadata.name}
  3303. 2020-11-02T12:06:48.5803253Z [submariner]$ [cluster1] command kubectl --context=cluster1 get pods --selector=name=submariner-operator -n submariner-operator -o jsonpath={.items[*].metadata.name}
  3304. 2020-11-02T12:06:48.5814054Z [submariner]$ [cluster1] kubectl --context=cluster1 get pods --selector=name=submariner-operator -n submariner-operator -o jsonpath={.items[*].metadata.name}
  3305. 2020-11-02T12:06:48.8770239Z [submariner]$ [cluster1] kubectl -n submariner-operator logs submariner-operator-6675977db7-khlg6
  3306. 2020-11-02T12:06:48.8785399Z [submariner]$ [cluster1] kubectl -n submariner-operator logs submariner-operator-6675977db7-khlg6
  3307. 2020-11-02T12:06:48.8809375Z [submariner]$ [cluster1] command kubectl --context=cluster1 -n submariner-operator logs submariner-operator-6675977db7-khlg6
  3308. 2020-11-02T12:06:48.8824689Z [submariner]$ [cluster1] kubectl --context=cluster1 -n submariner-operator logs submariner-operator-6675977db7-khlg6
  3309. 2020-11-02T12:06:49.1797860Z [cluster1] I1102 11:52:34.415045 1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_close_wait' to 3600
  3310. 2020-11-02T12:06:49.1798537Z [cluster1] I1102 11:52:34.418515 1 config.go:313] Starting service config controller
  3311. 2020-11-02T12:06:49.1799100Z [cluster1] I1102 11:52:34.418695 1 shared_informer.go:197] Waiting for caches to sync for service config
  3312. 2020-11-02T12:06:49.1799812Z [cluster1] I1102 11:52:34.419734 1 config.go:131] Starting endpoints config controller
  3313. 2020-11-02T12:06:49.1800367Z [cluster1] I1102 11:52:34.419758 1 shared_informer.go:197] Waiting for caches to sync for endpoints config
  3314. 2020-11-02T12:06:49.1800930Z [cluster1] I1102 11:52:34.518975 1 shared_informer.go:204] Caches are synced for service config
  3315. 2020-11-02T12:06:49.1801491Z [cluster1] I1102 11:52:34.520010 1 shared_informer.go:204] Caches are synced for endpoints config
  3316. 2020-11-02T12:06:49.1802676Z [cluster1] +++++++++++++++++++++: Logs for Pod kube-proxy-mq8dj in namespace kube-system :++++++++++++++++++++++
  3317. 2020-11-02T12:06:49.1803519Z [cluster1] W1102 11:52:14.089053 1 server_others.go:323] Unknown proxy mode "", assuming iptables proxy
  3318. 2020-11-02T12:06:49.1804846Z [cluster1] I1102 11:52:14.106056 1 node.go:135] Successfully retrieved node IP: 172.17.0.9
  3319. 2020-11-02T12:06:49.1805819Z [cluster1] I1102 11:52:14.106101 1 server_others.go:145] Using iptables Proxier.
  3320. 2020-11-02T12:06:49.1806251Z [cluster1] I1102 11:52:14.106408 1 server.go:571] Version: v1.17.0
  3321. 2020-11-02T12:06:49.1807025Z [cluster1] I1102 11:52:14.106976 1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_max' to 131072
  3322. 2020-11-02T12:06:49.1807537Z [cluster1] I1102 11:52:14.107013 1 conntrack.go:52] Setting nf_conntrack_max to 131072
  3323. 2020-11-02T12:06:49.1808602Z [cluster1] I1102 11:52:14.108708 1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_established' to 86400
  3324. 2020-11-02T12:06:49.1809471Z [cluster1] I1102 11:52:14.108873 1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_close_wait' to 3600
  3325. 2020-11-02T12:06:49.1810057Z [cluster1] I1102 11:52:14.144098 1 config.go:131] Starting endpoints config controller
  3326. 2020-11-02T12:06:49.1810610Z [cluster1] I1102 11:52:14.144129 1 shared_informer.go:197] Waiting for caches to sync for endpoints config
  3327. 2020-11-02T12:06:49.1811150Z [cluster1] I1102 11:52:14.144457 1 config.go:313] Starting service config controller
  3328. 2020-11-02T12:06:49.1811683Z [cluster1] I1102 11:52:14.144473 1 shared_informer.go:197] Waiting for caches to sync for service config
  3329. 2020-11-02T12:06:49.1812236Z [cluster1] I1102 11:52:14.244504 1 shared_informer.go:204] Caches are synced for endpoints config
  3330. 2020-11-02T12:06:49.1812775Z [cluster1] I1102 11:52:14.244789 1 shared_informer.go:204] Caches are synced for service config
  3331. 2020-11-02T12:06:49.1813181Z [cluster1] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  3332. 2020-11-02T12:06:49.1814411Z [cluster1] +++++++++++++++++++++: Logs for Pod submariner-operator-6675977db7-khlg6 in namespace submariner-operator :++++++++++++++++++++++
  3333. 2020-11-02T12:06:49.1815603Z [cluster1] {"level":"info","ts":1604318231.4451654,"logger":"cmd","msg":"Go Version: go1.15.2"}
  3334. 2020-11-02T12:06:49.1816048Z [cluster1] {"level":"info","ts":1604318231.445195,"logger":"cmd","msg":"Go OS/Arch: linux/amd64"}
  3335. 2020-11-02T12:06:49.1816750Z [cluster1] {"level":"info","ts":1604318231.4451993,"logger":"cmd","msg":"Version of operator-sdk: v0.17.0"}
  3336. 2020-11-02T12:06:49.1817653Z [cluster1] {"level":"info","ts":1604318231.4452028,"logger":"cmd","msg":"Submariner operator version: was not built correctly"}
  3337. 2020-11-02T12:06:49.1818228Z [cluster1] {"level":"info","ts":1604318231.4453318,"logger":"leader","msg":"Trying to become the leader."}
  3338. 2020-11-02T12:06:49.1818942Z [cluster1] {"level":"info","ts":1604318231.9721751,"logger":"leader","msg":"No pre-existing lock was found."}
  3339. 2020-11-02T12:06:49.1819448Z [cluster1] {"level":"info","ts":1604318231.9786093,"logger":"leader","msg":"Became the leader."}
  3340. 2020-11-02T12:06:49.1820496Z [cluster1] {"level":"info","ts":1604318232.4821186,"logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":"0.0.0.0:8383"}
  3341. 2020-11-02T12:06:49.1821173Z [cluster1] {"level":"info","ts":1604318232.4831007,"logger":"cmd","msg":"Registering Components."}
  3342. 2020-11-02T12:06:49.1821661Z [cluster1] {"level":"info","ts":1604318232.483279,"logger":"cmd","msg":"Creating the engine CRDs"}
  3343. 2020-11-02T12:06:49.1822319Z [cluster1] {"level":"info","ts":1604318232.5589535,"logger":"cmd","msg":"Creating the Lighthouse CRDs"}
  3344. 2020-11-02T12:06:49.1822894Z [cluster1] {"level":"info","ts":1604318232.6954536,"logger":"cmd","msg":"The Lighthouse CRDs were updated, restarting..."}
  3345. 2020-11-02T12:06:49.1823433Z [cluster1] {"level":"info","ts":1604318232.7772844,"logger":"cmd","msg":"Go Version: go1.15.2"}
  3346. 2020-11-02T12:06:49.1823875Z [cluster1] {"level":"info","ts":1604318232.7773192,"logger":"cmd","msg":"Go OS/Arch: linux/amd64"}
  3347. 2020-11-02T12:06:49.1825199Z [cluster1] {"level":"info","ts":1604318232.7773242,"logger":"cmd","msg":"Version of operator-sdk: v0.17.0"}
  3348. 2020-11-02T12:06:49.1825805Z [cluster1] {"level":"info","ts":1604318232.777328,"logger":"cmd","msg":"Submariner operator version: was not built correctly"}
  3349. 2020-11-02T12:06:49.1826382Z [cluster1] {"level":"info","ts":1604318232.7846718,"logger":"leader","msg":"Trying to become the leader."}
  3350. 2020-11-02T12:06:49.1827010Z [cluster1] {"level":"info","ts":1604318233.3305757,"logger":"leader","msg":"Found existing lock with my name. I was likely restarted."}
  3351. 2020-11-02T12:06:49.1827738Z [cluster1] {"level":"info","ts":1604318233.330802,"logger":"leader","msg":"Continuing as the leader."}
  3352. 2020-11-02T12:06:49.1829984Z [cluster1] {"level":"info","ts":1604318233.840264,"logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":"0.0.0.0:8383"}
  3353. 2020-11-02T12:06:49.1831027Z [cluster1] {"level":"info","ts":1604318233.8405793,"logger":"cmd","msg":"Registering Components."}
  3354. 2020-11-02T12:06:49.1831640Z [cluster1] {"level":"info","ts":1604318233.840743,"logger":"cmd","msg":"Creating the engine CRDs"}
  3355. 2020-11-02T12:06:49.1832662Z [cluster1] {"level":"info","ts":1604318234.3785741,"logger":"cmd","msg":"Creating the Lighthouse CRDs"}
  3356. 2020-11-02T12:06:49.1833887Z [cluster1] {"level":"info","ts":1604318237.2031915,"logger":"metrics","msg":"Metrics Service object created","Service.Name":"submariner-operator-metrics","Service.Namespace":"submariner-operator"}
  3357. 2020-11-02T12:06:49.1835199Z [cluster1] {"level":"info","ts":1604318237.7045047,"logger":"cmd","msg":"Could not create ServiceMonitor object","error":"no ServiceMonitor registered with the API"}
  3358. 2020-11-02T12:06:49.1836796Z [cluster1] {"level":"info","ts":1604318237.7045481,"logger":"cmd","msg":"Install prometheus-operator in your cluster to create ServiceMonitor objects","error":"no ServiceMonitor registered with the API"}
  3359. 2020-11-02T12:06:49.1837775Z [cluster1] {"level":"info","ts":1604318237.7045531,"logger":"cmd","msg":"Starting the Cmd."}
  3360. 2020-11-02T12:06:49.1838713Z [cluster1] {"level":"info","ts":1604318237.7050633,"logger":"controller-runtime.manager","msg":"starting metrics server","path":"/metrics"}
  3361. 2020-11-02T12:06:49.1840302Z [cluster1] {"level":"info","ts":1604318237.705219,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"servicediscovery-controller","source":"kind source: /, Kind="}
  3362. 2020-11-02T12:06:49.1842160Z [cluster1] {"level":"info","ts":1604318237.7053692,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"submariner-controller","source":"kind source: /, Kind="}
  3363. 2020-11-02T12:06:49.1844113Z [cluster1] {"level":"info","ts":1604318237.8055308,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"submariner-controller","source":"kind source: /, Kind="}
  3364. 2020-11-02T12:06:49.1846306Z [cluster1] {"level":"info","ts":1604318237.8054774,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"servicediscovery-controller","source":"kind source: /, Kind="}
  3365. 2020-11-02T12:06:49.1852539Z [cluster1] {"level":"info","ts":1604318237.905946,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"submariner-controller","source":"kind source: /, Kind="}
  3366. 2020-11-02T12:06:49.1854519Z [cluster1] {"level":"info","ts":1604318237.906212,"logger":"controller-runtime.controller","msg":"Starting Controller","controller":"servicediscovery-controller"}
  3367. 2020-11-02T12:06:49.1857024Z [cluster1] {"level":"info","ts":1604318238.0072563,"logger":"controller-runtime.controller","msg":"Starting workers","controller":"servicediscovery-controller","worker count":1}
  3368. 2020-11-02T12:06:49.1858900Z [cluster1] {"level":"info","ts":1604318238.0073764,"logger":"controller-runtime.controller","msg":"Starting Controller","controller":"submariner-controller"}
  3369. 2020-11-02T12:06:49.1860880Z [cluster1] {"level":"info","ts":1604318238.007392,"logger":"controller-runtime.controller","msg":"Starting workers","controller":"submariner-controller","worker count":1}
  3370. 2020-11-02T12:06:49.1862634Z [cluster1] {"level":"info","ts":1604318245.0848403,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3371. 2020-11-02T12:06:49.1863574Z [cluster1] {"level":"info","ts":1604318245.8536696,"logger":"controller_submariner","msg":"Cluster network discovered"}
  3372. 2020-11-02T12:06:49.1864707Z [cluster1] {"level":"info","ts":1604318245.8537014,"logger":"controller_submariner","msg":"Discovered K8s network details","plugin":"generic","clusterCIDRs":["10.1.0.0/16"],"serviceCIDRs":["100.1.0.0/16"]}
  3373. 2020-11-02T12:06:49.1865877Z [cluster1] {"level":"info","ts":1604318245.8537579,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3374. 2020-11-02T12:06:49.1866886Z [cluster1] {"level":"info","ts":1604318245.8537621,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3375. 2020-11-02T12:06:49.1868823Z [cluster1] {"level":"info","ts":1604318245.912207,"logger":"controller_submariner","msg":"Created a new DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3376. 2020-11-02T12:06:49.1871217Z [cluster1] {"level":"info","ts":1604318245.9765806,"logger":"controller_submariner","msg":"Created a new DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3377. 2020-11-02T12:06:49.1873494Z [cluster1] {"level":"info","ts":1604318246.3512332,"logger":"controller_servicediscovery","msg":"Reconciling ServiceDiscovery","Request.Namespace":"submariner-operator","Request.Name":"service-discovery"}
  3378. 2020-11-02T12:06:49.1875828Z [cluster1] {"level":"info","ts":1604318246.4615843,"logger":"controller_submariner","msg":"Created Service Discovery CR","Request.Namespace":"submariner-operator","Request.Name":"submariner","Namespace":"submariner-operator","Name":"service-discovery"}
  3379. 2020-11-02T12:06:49.1878285Z [cluster1] {"level":"info","ts":1604318246.4616544,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3380. 2020-11-02T12:06:49.1881011Z [cluster1] {"level":"info","ts":1604318246.8138392,"logger":"controller_servicediscovery","msg":"Created a new Deployment","Request.Namespace":"submariner-operator","Request.Name":"service-discovery","Deployment.Namespace":"submariner-operator","Deployment.Name":"submariner-lighthouse-agent"}
  3381. 2020-11-02T12:06:49.1883017Z [cluster1] {"level":"info","ts":1604318247.3521671,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3382. 2020-11-02T12:06:49.1883815Z [cluster1] {"level":"info","ts":1604318247.3521955,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3383. 2020-11-02T12:06:49.1888156Z [cluster1] {"level":"info","ts":1604318247.6223507,"logger":"controller_servicediscovery","msg":"Created a new ConfigMap","Request.Namespace":"submariner-operator","Request.Name":"service-discovery","ConfigMap.Namespace":"submariner-operator","ConfigMap.Name":"submariner-lighthouse-coredns"}
  3384. 2020-11-02T12:06:49.1891965Z [cluster1] {"level":"info","ts":1604318248.782181,"logger":"controller_servicediscovery","msg":"Created a new Deployment","Request.Namespace":"submariner-operator","Request.Name":"service-discovery","Deployment.Namespace":"submariner-operator","Deployment.Name":"submariner-lighthouse-coredns"}
  3385. 2020-11-02T12:06:49.1895209Z [cluster1] {"level":"info","ts":1604318249.891252,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3386. 2020-11-02T12:06:49.1898278Z [cluster1] {"level":"info","ts":1604318250.712128,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3387. 2020-11-02T12:06:49.1901104Z [cluster1] {"level":"info","ts":1604318251.2428436,"logger":"controller_servicediscovery","msg":"Created a new Service","Request.Namespace":"submariner-operator","Request.Name":"service-discovery","Service.Namespace":"submariner-operator","Service.Name":"submariner-lighthouse-coredns"}
  3388. 2020-11-02T12:06:49.1906030Z [cluster1] {"level":"info","ts":1604318251.2434936,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3389. 2020-11-02T12:06:49.1908323Z [cluster1] {"level":"info","ts":1604318251.386975,"logger":"controller_servicediscovery","msg":"coredns configmap does not have lighthouse configuration hence creating","Request.Namespace":"submariner-operator","Request.Name":"service-discovery"}
  3390. 2020-11-02T12:06:49.1912577Z [cluster1] {"level":"info","ts":1604318251.387009,"logger":"controller_servicediscovery","msg":"Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n forward . 100.1.240.245\n}\n#lighthouse-end\n.:53 {\n errors\n health {\n lameduck 5s\n }\n ready\n kubernetes cluster1.local in-addr.arpa ip6.arpa {\n pods insecure\n fallthrough in-addr.arpa ip6.arpa\n ttl 30\n }\n prometheus :9153\n forward . /etc/resolv.conf\n cache 30\n loop\n reload\n loadbalance\n}\n"}
  3391. 2020-11-02T12:06:49.1922184Z [cluster1] {"level":"error","ts":1604318251.5103388,"logger":"controller-runtime.controller","msg":"Reconciler error","controller":"submariner-controller","name":"submariner","namespace":"submariner-operator","error":"Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\tsubmariner-operator/vendor/github.com/go-logr/zapr/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsubmariner-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:248\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsubmariner-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:222\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\tsubmariner-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:201\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\tsubmariner-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\tsubmariner-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\tsubmariner-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"}
  3392. 2020-11-02T12:06:49.1952522Z [cluster1] {"level":"info","ts":1604318251.8178568,"logger":"controller_servicediscovery","msg":"Reconciling ServiceDiscovery","Request.Namespace":"submariner-operator","Request.Name":"service-discovery"}
  3393. 2020-11-02T12:06:49.1955275Z [cluster1] {"level":"info","ts":1604318251.9670835,"logger":"controller_servicediscovery","msg":"Updated existing Deployment","Request.Namespace":"submariner-operator","Request.Name":"service-discovery","Deployment.Namespace":"submariner-operator","Deployment.Name":"submariner-lighthouse-agent"}
  3394. 2020-11-02T12:06:49.1958560Z [cluster1] {"level":"info","ts":1604318252.0627096,"logger":"controller_servicediscovery","msg":"Updated existing Deployment","Request.Namespace":"submariner-operator","Request.Name":"service-discovery","Deployment.Namespace":"submariner-operator","Deployment.Name":"submariner-lighthouse-coredns"}
  3395. 2020-11-02T12:06:49.1961150Z [cluster1] {"level":"info","ts":1604318252.1097326,"logger":"controller_servicediscovery","msg":"coredns configmap has lighthouse configuration hence updating","Request.Namespace":"submariner-operator","Request.Name":"service-discovery"}
  3396. 2020-11-02T12:06:49.1964014Z [cluster1] {"level":"info","ts":1604318252.1097927,"logger":"controller_servicediscovery","msg":"Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n forward . 100.1.240.245\n}\n#lighthouse-end\n.:53 {\n errors\n health {\n lameduck 5s\n }\n ready\n kubernetes cluster1.local in-addr.arpa ip6.arpa {\n pods insecure\n fallthrough in-addr.arpa ip6.arpa\n ttl 30\n }\n prometheus :9153\n forward . /etc/resolv.conf\n cache 30\n loop\n reload\n loadbalance\n}\n\n"}
  3397. 2020-11-02T12:06:49.1966694Z [cluster1] {"level":"info","ts":1604318252.5165303,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3398. 2020-11-02T12:06:49.1968092Z [cluster1] {"level":"info","ts":1604318252.5673025,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3399. 2020-11-02T12:06:49.1968935Z [cluster1] {"level":"info","ts":1604318252.5673287,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3400. 2020-11-02T12:06:49.1970853Z [cluster1] {"level":"info","ts":1604318252.6391218,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3401. 2020-11-02T12:06:49.1973019Z [cluster1] {"level":"info","ts":1604318252.6910934,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3402. 2020-11-02T12:06:49.1975230Z [cluster1] {"level":"info","ts":1604318288.3170278,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3403. 2020-11-02T12:06:49.1976543Z [cluster1] {"level":"info","ts":1604318288.346502,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3404. 2020-11-02T12:06:49.1977568Z [cluster1] {"level":"info","ts":1604318288.3465238,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3405. 2020-11-02T12:06:49.1979171Z [cluster1] {"level":"info","ts":1604318288.3853123,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3406. 2020-11-02T12:06:49.1981256Z [cluster1] {"level":"info","ts":1604318288.4004676,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3407. 2020-11-02T12:06:49.1982962Z [cluster1] {"level":"info","ts":1604318288.454066,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3408. 2020-11-02T12:06:49.1983849Z [cluster1] {"level":"info","ts":1604318288.4741662,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3409. 2020-11-02T12:06:49.1986708Z [cluster1] {"level":"info","ts":1604318288.4741998,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3410. 2020-11-02T12:06:49.1989287Z [cluster1] {"level":"info","ts":1604318288.4832227,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3411. 2020-11-02T12:06:49.1992165Z [cluster1] {"level":"info","ts":1604318288.494848,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3412. 2020-11-02T12:06:49.1994069Z [cluster1] {"level":"info","ts":1604318289.2729318,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3413. 2020-11-02T12:06:49.1995376Z [cluster1] {"level":"info","ts":1604318289.3047597,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3414. 2020-11-02T12:06:49.1996095Z [cluster1] {"level":"info","ts":1604318289.3047812,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3415. 2020-11-02T12:06:49.1998535Z [cluster1] {"level":"info","ts":1604318289.3180523,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3416. 2020-11-02T12:06:49.2003844Z [cluster1] {"level":"info","ts":1604318289.3485947,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3417. 2020-11-02T12:06:49.2005776Z [cluster1] {"level":"info","ts":1604318289.4431558,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3418. 2020-11-02T12:06:49.2007711Z [cluster1] {"level":"info","ts":1604318289.5550907,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3419. 2020-11-02T12:06:49.2008574Z [cluster1] {"level":"info","ts":1604318289.555117,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3420. 2020-11-02T12:06:49.2010932Z [cluster1] {"level":"info","ts":1604318289.5929625,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3421. 2020-11-02T12:06:49.2013707Z [cluster1] {"level":"info","ts":1604318289.6108859,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3422. 2020-11-02T12:06:49.2015852Z [cluster1] {"level":"info","ts":1604318289.793422,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3423. 2020-11-02T12:06:49.2018885Z [cluster1] {"level":"info","ts":1604318289.8618183,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3424. 2020-11-02T12:06:49.2020019Z [cluster1] {"level":"info","ts":1604318289.8618455,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3425. 2020-11-02T12:06:49.2022165Z [cluster1] {"level":"info","ts":1604318289.8805523,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3426. 2020-11-02T12:06:49.2024934Z [cluster1] {"level":"info","ts":1604318289.8974562,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3427. 2020-11-02T12:06:49.2026918Z [cluster1] {"level":"info","ts":1604318289.9273858,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3428. 2020-11-02T12:06:49.2028045Z [cluster1] {"level":"info","ts":1604318289.9671872,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3429. 2020-11-02T12:06:49.2028773Z [cluster1] {"level":"info","ts":1604318289.9672117,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3430. 2020-11-02T12:06:49.2031274Z [cluster1] {"level":"info","ts":1604318290.0201788,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3431. 2020-11-02T12:06:49.2042874Z [cluster1] {"level":"info","ts":1604318290.0335083,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3432. 2020-11-02T12:06:49.2045965Z [cluster1] {"level":"info","ts":1604318294.7976341,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3433. 2020-11-02T12:06:49.2047300Z [cluster1] {"level":"info","ts":1604318294.8224459,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3434. 2020-11-02T12:06:49.2048012Z [cluster1] {"level":"info","ts":1604318294.8224714,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3435. 2020-11-02T12:06:49.2049824Z [cluster1] {"level":"info","ts":1604318294.8305976,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3436. 2020-11-02T12:06:49.2052057Z [cluster1] {"level":"info","ts":1604318294.8347232,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3437. 2020-11-02T12:06:49.2054086Z [cluster1] {"level":"info","ts":1604318301.752168,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3438. 2020-11-02T12:06:49.2063219Z [cluster1] {"level":"info","ts":1604318301.7728062,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3439. 2020-11-02T12:06:49.2064204Z [cluster1] {"level":"info","ts":1604318301.7728271,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3440. 2020-11-02T12:06:49.2066198Z [cluster1] {"level":"info","ts":1604318301.7767537,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3441. 2020-11-02T12:06:49.2068924Z [cluster1] {"level":"info","ts":1604318301.781179,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3442. 2020-11-02T12:06:49.2076135Z [cluster1] {"level":"info","ts":1604318301.7901776,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3443. 2020-11-02T12:06:49.2077540Z [cluster1] {"level":"info","ts":1604318301.8049154,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3444. 2020-11-02T12:06:49.2078269Z [cluster1] {"level":"info","ts":1604318301.8049312,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3445. 2020-11-02T12:06:49.2080105Z [cluster1] {"level":"info","ts":1604318301.809012,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3446. 2020-11-02T12:06:49.2082337Z [cluster1] {"level":"info","ts":1604318301.8131294,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3447. 2020-11-02T12:06:49.2084297Z [cluster1] {"level":"info","ts":1604318302.2888916,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3448. 2020-11-02T12:06:49.2085256Z [cluster1] {"level":"info","ts":1604318302.3074226,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3449. 2020-11-02T12:06:49.2085958Z [cluster1] {"level":"info","ts":1604318302.3074965,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3450. 2020-11-02T12:06:49.2090062Z [cluster1] {"level":"info","ts":1604318302.316143,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3451. 2020-11-02T12:06:49.2092472Z [cluster1] {"level":"info","ts":1604318302.336873,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3452. 2020-11-02T12:06:49.2094411Z [cluster1] {"level":"info","ts":1604318302.364972,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3453. 2020-11-02T12:06:49.2095328Z [cluster1] {"level":"info","ts":1604318302.3741064,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3454. 2020-11-02T12:06:49.2096021Z [cluster1] {"level":"info","ts":1604318302.3742626,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3455. 2020-11-02T12:06:49.2101481Z [cluster1] {"level":"info","ts":1604318302.3783393,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3456. 2020-11-02T12:06:49.2104279Z [cluster1] {"level":"info","ts":1604318302.386812,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3457. 2020-11-02T12:06:49.2106098Z [cluster1] {"level":"info","ts":1604318306.7705164,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3458. 2020-11-02T12:06:49.2107620Z [cluster1] {"level":"info","ts":1604318306.774567,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3459. 2020-11-02T12:06:49.2108399Z [cluster1] {"level":"info","ts":1604318306.774586,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3460. 2020-11-02T12:06:49.2110777Z [cluster1] {"level":"info","ts":1604318306.7791884,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3461. 2020-11-02T12:06:49.2114112Z [cluster1] {"level":"info","ts":1604318306.7825115,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3462. 2020-11-02T12:06:49.2116162Z [cluster1] {"level":"info","ts":1604318311.7855222,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3463. 2020-11-02T12:06:49.2118318Z [cluster1] {"level":"info","ts":1604318311.7980192,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3464. 2020-11-02T12:06:49.2119085Z [cluster1] {"level":"info","ts":1604318311.7980359,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3465. 2020-11-02T12:06:49.2121555Z [cluster1] {"level":"info","ts":1604318311.8113637,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3466. 2020-11-02T12:06:49.2124671Z [cluster1] {"level":"info","ts":1604318311.8197765,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3467. 2020-11-02T12:06:49.2127157Z [cluster1] {"level":"info","ts":1604318311.8305833,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3468. 2020-11-02T12:06:49.2128181Z [cluster1] {"level":"info","ts":1604318311.8415854,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3469. 2020-11-02T12:06:49.2129108Z [cluster1] {"level":"info","ts":1604318311.8416016,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3470. 2020-11-02T12:06:49.2130984Z [cluster1] {"level":"info","ts":1604318311.8474672,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3471. 2020-11-02T12:06:49.2136641Z [cluster1] {"level":"info","ts":1604318311.8516343,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3472. 2020-11-02T12:06:49.2139525Z [cluster1] {"level":"info","ts":1604318316.8328028,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3473. 2020-11-02T12:06:49.2141615Z [cluster1] {"level":"info","ts":1604318316.8419213,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3474. 2020-11-02T12:06:49.2142517Z [cluster1] {"level":"info","ts":1604318316.84194,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3475. 2020-11-02T12:06:49.2144674Z [cluster1] {"level":"info","ts":1604318316.8465714,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3476. 2020-11-02T12:06:49.2147066Z [cluster1] {"level":"info","ts":1604318316.8501976,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3477. 2020-11-02T12:06:49.2148999Z [cluster1] {"level":"info","ts":1604318321.8490956,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3478. 2020-11-02T12:06:49.2150014Z [cluster1] {"level":"info","ts":1604318321.8599052,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3479. 2020-11-02T12:06:49.2153276Z [cluster1] {"level":"info","ts":1604318321.859925,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3480. 2020-11-02T12:06:49.2155065Z [cluster1] {"level":"info","ts":1604318321.8664546,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3481. 2020-11-02T12:06:49.2157604Z [cluster1] {"level":"info","ts":1604318321.8738716,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3482. 2020-11-02T12:06:49.2159859Z [cluster1] {"level":"info","ts":1604318326.885754,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3483. 2020-11-02T12:06:49.2160992Z [cluster1] {"level":"info","ts":1604318326.893581,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3484. 2020-11-02T12:06:49.2161717Z [cluster1] {"level":"info","ts":1604318326.8936043,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3485. 2020-11-02T12:06:49.2164012Z [cluster1] {"level":"info","ts":1604318326.8978,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3486. 2020-11-02T12:06:49.2168521Z [cluster1] {"level":"info","ts":1604318326.9010017,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3487. 2020-11-02T12:06:49.2171400Z [cluster1] {"level":"info","ts":1604318331.9228876,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3488. 2020-11-02T12:06:49.2173452Z [cluster1] {"level":"info","ts":1604318331.9326766,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3489. 2020-11-02T12:06:49.2174371Z [cluster1] {"level":"info","ts":1604318331.9327323,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3490. 2020-11-02T12:06:49.2176266Z [cluster1] {"level":"info","ts":1604318331.9391642,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3491. 2020-11-02T12:06:49.2179190Z [cluster1] {"level":"info","ts":1604318331.9417837,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3492. 2020-11-02T12:06:49.2183894Z [cluster1] {"level":"info","ts":1604318336.943529,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3493. 2020-11-02T12:06:49.2184906Z [cluster1] {"level":"info","ts":1604318336.9559078,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3494. 2020-11-02T12:06:49.2185981Z [cluster1] {"level":"info","ts":1604318336.9559324,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3495. 2020-11-02T12:06:49.2188196Z [cluster1] {"level":"info","ts":1604318336.973851,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3496. 2020-11-02T12:06:49.2192911Z [cluster1] {"level":"info","ts":1604318336.9810393,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3497. 2020-11-02T12:06:49.2195927Z [cluster1] {"level":"info","ts":1604318341.9623225,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3498. 2020-11-02T12:06:49.2197527Z [cluster1] {"level":"info","ts":1604318341.9742541,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3499. 2020-11-02T12:06:49.2198480Z [cluster1] {"level":"info","ts":1604318341.9742744,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3500. 2020-11-02T12:06:49.2200763Z [cluster1] {"level":"info","ts":1604318341.98084,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3501. 2020-11-02T12:06:49.2203128Z [cluster1] {"level":"info","ts":1604318341.9842637,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3502. 2020-11-02T12:06:49.2205743Z [cluster1] {"level":"info","ts":1604318346.982152,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3503. 2020-11-02T12:06:49.2206919Z [cluster1] {"level":"info","ts":1604318347.0047262,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3504. 2020-11-02T12:06:49.2207663Z [cluster1] {"level":"info","ts":1604318347.0047617,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3505. 2020-11-02T12:06:49.2209378Z [cluster1] {"level":"info","ts":1604318347.0149908,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3506. 2020-11-02T12:06:49.2212105Z [cluster1] {"level":"info","ts":1604318347.0187325,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3507. 2020-11-02T12:06:49.2214237Z [cluster1] {"level":"info","ts":1604318352.0059497,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3508. 2020-11-02T12:06:49.2215227Z [cluster1] {"level":"info","ts":1604318352.0213158,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3509. 2020-11-02T12:06:49.2215945Z [cluster1] {"level":"info","ts":1604318352.0213394,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3510. 2020-11-02T12:06:49.2217633Z [cluster1] {"level":"info","ts":1604318352.0254638,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3511. 2020-11-02T12:06:49.2219938Z [cluster1] {"level":"info","ts":1604318352.0321784,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3512. 2020-11-02T12:06:49.2221798Z [cluster1] {"level":"info","ts":1604318357.024767,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3513. 2020-11-02T12:06:49.2222772Z [cluster1] {"level":"info","ts":1604318357.1618023,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3514. 2020-11-02T12:06:49.2223492Z [cluster1] {"level":"info","ts":1604318357.1618567,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3515. 2020-11-02T12:06:49.2225147Z [cluster1] {"level":"info","ts":1604318357.1695793,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3516. 2020-11-02T12:06:49.2228806Z [cluster1] {"level":"info","ts":1604318357.1846125,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3517. 2020-11-02T12:06:49.2231098Z [cluster1] {"level":"info","ts":1604318362.16847,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3518. 2020-11-02T12:06:49.2232085Z [cluster1] {"level":"info","ts":1604318362.1805968,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3519. 2020-11-02T12:06:49.2232954Z [cluster1] {"level":"info","ts":1604318362.1806154,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3520. 2020-11-02T12:06:49.2234820Z [cluster1] {"level":"info","ts":1604318362.2102716,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3521. 2020-11-02T12:06:49.2237457Z [cluster1] {"level":"info","ts":1604318362.2163653,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3522. 2020-11-02T12:06:49.2239601Z [cluster1] {"level":"info","ts":1604318367.195513,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3523. 2020-11-02T12:06:49.2240550Z [cluster1] {"level":"info","ts":1604318367.2022896,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3524. 2020-11-02T12:06:49.2241254Z [cluster1] {"level":"info","ts":1604318367.2023091,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3525. 2020-11-02T12:06:49.2242933Z [cluster1] {"level":"info","ts":1604318367.205901,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3526. 2020-11-02T12:06:49.2245376Z [cluster1] {"level":"info","ts":1604318367.208686,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3527. 2020-11-02T12:06:49.2247397Z [cluster1] {"level":"info","ts":1604318372.2139256,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3528. 2020-11-02T12:06:49.2248388Z [cluster1] {"level":"info","ts":1604318372.2716324,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3529. 2020-11-02T12:06:49.2249110Z [cluster1] {"level":"info","ts":1604318372.2716618,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3530. 2020-11-02T12:06:49.2252228Z [cluster1] {"level":"info","ts":1604318372.2793567,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3531. 2020-11-02T12:06:49.2255596Z [cluster1] {"level":"info","ts":1604318372.284946,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3532. 2020-11-02T12:06:49.2258079Z [cluster1] {"level":"info","ts":1604318377.2666159,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3533. 2020-11-02T12:06:49.2259928Z [cluster1] {"level":"info","ts":1604318377.2721095,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3534. 2020-11-02T12:06:49.2260959Z [cluster1] {"level":"info","ts":1604318377.2721455,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3535. 2020-11-02T12:06:49.2262828Z [cluster1] {"level":"info","ts":1604318377.2764885,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3536. 2020-11-02T12:06:49.2265573Z [cluster1] {"level":"info","ts":1604318377.2795417,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3537. 2020-11-02T12:06:49.2268431Z [cluster1] {"level":"info","ts":1604318382.2770135,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3538. 2020-11-02T12:06:49.2269791Z [cluster1] {"level":"info","ts":1604318382.2847385,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3539. 2020-11-02T12:06:49.2270738Z [cluster1] {"level":"info","ts":1604318382.284757,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3540. 2020-11-02T12:06:49.2272452Z [cluster1] {"level":"info","ts":1604318382.288473,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3541. 2020-11-02T12:06:49.2274753Z [cluster1] {"level":"info","ts":1604318382.30597,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3542. 2020-11-02T12:06:49.2277413Z [cluster1] {"level":"info","ts":1604318387.2958715,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3543. 2020-11-02T12:06:49.2278449Z [cluster1] {"level":"info","ts":1604318387.3021302,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3544. 2020-11-02T12:06:49.2279557Z [cluster1] {"level":"info","ts":1604318387.3021502,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3545. 2020-11-02T12:06:49.2281413Z [cluster1] {"level":"info","ts":1604318387.3059852,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3546. 2020-11-02T12:06:49.2285669Z [cluster1] {"level":"info","ts":1604318387.309677,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3547. 2020-11-02T12:06:49.2288658Z [cluster1] {"level":"info","ts":1604318392.3152106,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3548. 2020-11-02T12:06:49.2290328Z [cluster1] {"level":"info","ts":1604318392.330355,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3549. 2020-11-02T12:06:49.2291077Z [cluster1] {"level":"info","ts":1604318392.3303707,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3550. 2020-11-02T12:06:49.2292857Z [cluster1] {"level":"info","ts":1604318392.3371446,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3551. 2020-11-02T12:06:49.2295243Z [cluster1] {"level":"info","ts":1604318392.3478856,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3552. 2020-11-02T12:06:49.2297898Z [cluster1] {"level":"info","ts":1604318397.3622377,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3553. 2020-11-02T12:06:49.2299064Z [cluster1] {"level":"info","ts":1604318397.3695834,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3554. 2020-11-02T12:06:49.2300088Z [cluster1] {"level":"info","ts":1604318397.3696086,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3555. 2020-11-02T12:06:49.2302524Z [cluster1] {"level":"info","ts":1604318397.375013,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3556. 2020-11-02T12:06:49.2305390Z [cluster1] {"level":"info","ts":1604318397.378932,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3557. 2020-11-02T12:06:49.2307395Z [cluster1] {"level":"info","ts":1604318402.3759656,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3558. 2020-11-02T12:06:49.2308450Z [cluster1] {"level":"info","ts":1604318402.3809168,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3559. 2020-11-02T12:06:49.2309217Z [cluster1] {"level":"info","ts":1604318402.3809364,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3560. 2020-11-02T12:06:49.2311122Z [cluster1] {"level":"info","ts":1604318402.385459,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3561. 2020-11-02T12:06:49.2313635Z [cluster1] {"level":"info","ts":1604318402.3888154,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3562. 2020-11-02T12:06:49.2315768Z [cluster1] {"level":"info","ts":1604318407.394539,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3563. 2020-11-02T12:06:49.2316777Z [cluster1] {"level":"info","ts":1604318407.401688,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3564. 2020-11-02T12:06:49.2317776Z [cluster1] {"level":"info","ts":1604318407.401712,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3565. 2020-11-02T12:06:49.2320189Z [cluster1] {"level":"info","ts":1604318407.4049244,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3566. 2020-11-02T12:06:49.2323051Z [cluster1] {"level":"info","ts":1604318407.4082832,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3567. 2020-11-02T12:06:49.2325179Z [cluster1] {"level":"info","ts":1604318412.41032,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3568. 2020-11-02T12:06:49.2326396Z [cluster1] {"level":"info","ts":1604318412.4156358,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3569. 2020-11-02T12:06:49.2327216Z [cluster1] {"level":"info","ts":1604318412.4156523,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3570. 2020-11-02T12:06:49.2328992Z [cluster1] {"level":"info","ts":1604318412.4189763,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3571. 2020-11-02T12:06:49.2331606Z [cluster1] {"level":"info","ts":1604318412.4219325,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3572. 2020-11-02T12:06:49.2334004Z [cluster1] {"level":"info","ts":1604318417.424918,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3573. 2020-11-02T12:06:49.2335221Z [cluster1] {"level":"info","ts":1604318417.4310384,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3574. 2020-11-02T12:06:49.2339190Z [cluster1] {"level":"info","ts":1604318417.431062,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3575. 2020-11-02T12:06:49.2345175Z [cluster1] {"level":"info","ts":1604318417.436785,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3576. 2020-11-02T12:06:49.2347863Z [cluster1] {"level":"info","ts":1604318417.4401214,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3577. 2020-11-02T12:06:49.2350008Z [cluster1] {"level":"info","ts":1604318422.4428062,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3578. 2020-11-02T12:06:49.2351057Z [cluster1] {"level":"info","ts":1604318422.4509497,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3579. 2020-11-02T12:06:49.2351832Z [cluster1] {"level":"info","ts":1604318422.4509697,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3580. 2020-11-02T12:06:49.2353761Z [cluster1] {"level":"info","ts":1604318422.455565,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3581. 2020-11-02T12:06:49.2356136Z [cluster1] {"level":"info","ts":1604318422.4587784,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3582. 2020-11-02T12:06:49.2358068Z [cluster1] {"level":"info","ts":1604318427.469491,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3583. 2020-11-02T12:06:49.2359084Z [cluster1] {"level":"info","ts":1604318427.474478,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3584. 2020-11-02T12:06:49.2359834Z [cluster1] {"level":"info","ts":1604318427.4744997,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3585. 2020-11-02T12:06:49.2361557Z [cluster1] {"level":"info","ts":1604318427.4780047,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3586. 2020-11-02T12:06:49.2364106Z [cluster1] {"level":"info","ts":1604318427.480567,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3587. 2020-11-02T12:06:49.2368606Z [cluster1] {"level":"info","ts":1604318432.490149,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3588. 2020-11-02T12:06:49.2381484Z [cluster1] {"level":"info","ts":1604318432.498024,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3589. 2020-11-02T12:06:49.2382461Z [cluster1] {"level":"info","ts":1604318432.498041,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3590. 2020-11-02T12:06:49.2384363Z [cluster1] {"level":"info","ts":1604318432.5009792,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3591. 2020-11-02T12:06:49.2386767Z [cluster1] {"level":"info","ts":1604318432.503939,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3592. 2020-11-02T12:06:49.2388697Z [cluster1] {"level":"info","ts":1604318437.5077732,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3593. 2020-11-02T12:06:49.2389721Z [cluster1] {"level":"info","ts":1604318437.5198064,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3594. 2020-11-02T12:06:49.2390564Z [cluster1] {"level":"info","ts":1604318437.5198243,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3595. 2020-11-02T12:06:49.2392323Z [cluster1] {"level":"info","ts":1604318437.525298,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3596. 2020-11-02T12:06:49.2395371Z [cluster1] {"level":"info","ts":1604318437.5302088,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3597. 2020-11-02T12:06:49.2397437Z [cluster1] {"level":"info","ts":1604318442.5225024,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3598. 2020-11-02T12:06:49.2398421Z [cluster1] {"level":"info","ts":1604318442.527723,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3599. 2020-11-02T12:06:49.2399159Z [cluster1] {"level":"info","ts":1604318442.52774,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3600. 2020-11-02T12:06:49.2401297Z [cluster1] {"level":"info","ts":1604318442.5315032,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3601. 2020-11-02T12:06:49.2403648Z [cluster1] {"level":"info","ts":1604318442.534488,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3602. 2020-11-02T12:06:49.2405781Z [cluster1] {"level":"info","ts":1604318447.5376616,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3603. 2020-11-02T12:06:49.2406769Z [cluster1] {"level":"info","ts":1604318447.5445552,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3604. 2020-11-02T12:06:49.2407493Z [cluster1] {"level":"info","ts":1604318447.5445812,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3605. 2020-11-02T12:06:49.2409148Z [cluster1] {"level":"info","ts":1604318447.54819,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3606. 2020-11-02T12:06:49.2413142Z [cluster1] {"level":"info","ts":1604318447.550597,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3607. 2020-11-02T12:06:49.2415370Z [cluster1] {"level":"info","ts":1604318452.5520053,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3608. 2020-11-02T12:06:49.2416676Z [cluster1] {"level":"info","ts":1604318452.5569422,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3609. 2020-11-02T12:06:49.2417400Z [cluster1] {"level":"info","ts":1604318452.557131,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3610. 2020-11-02T12:06:49.2419381Z [cluster1] {"level":"info","ts":1604318452.5600429,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3611. 2020-11-02T12:06:49.2421912Z [cluster1] {"level":"info","ts":1604318452.562755,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3612. 2020-11-02T12:06:49.2423983Z [cluster1] {"level":"info","ts":1604318457.5632355,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3613. 2020-11-02T12:06:49.2424964Z [cluster1] {"level":"info","ts":1604318457.6543796,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3614. 2020-11-02T12:06:49.2425687Z [cluster1] {"level":"info","ts":1604318457.6544077,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3615. 2020-11-02T12:06:49.2427352Z [cluster1] {"level":"info","ts":1604318457.6609628,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3616. 2020-11-02T12:06:49.2429653Z [cluster1] {"level":"info","ts":1604318457.665648,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3617. 2020-11-02T12:06:49.2431513Z [cluster1] {"level":"info","ts":1604318462.6609652,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3618. 2020-11-02T12:06:49.2432638Z [cluster1] {"level":"info","ts":1604318462.6717775,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3619. 2020-11-02T12:06:49.2433342Z [cluster1] {"level":"info","ts":1604318462.6717997,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3620. 2020-11-02T12:06:49.2434948Z [cluster1] {"level":"info","ts":1604318462.6769943,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3621. 2020-11-02T12:06:49.2437722Z [cluster1] {"level":"info","ts":1604318462.6798196,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3622. 2020-11-02T12:06:49.2439582Z [cluster1] {"level":"info","ts":1604318467.6802716,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3623. 2020-11-02T12:06:49.2440656Z [cluster1] {"level":"info","ts":1604318467.6874218,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3624. 2020-11-02T12:06:49.2442060Z [cluster1] {"level":"info","ts":1604318467.687443,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3625. 2020-11-02T12:06:49.2444345Z [cluster1] {"level":"info","ts":1604318467.6919062,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3626. 2020-11-02T12:06:49.2447562Z [cluster1] {"level":"info","ts":1604318467.7012916,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3627. 2020-11-02T12:06:49.2449439Z [cluster1] {"level":"info","ts":1604318472.6947675,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3628. 2020-11-02T12:06:49.2450426Z [cluster1] {"level":"info","ts":1604318472.7063375,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3629. 2020-11-02T12:06:49.2451262Z [cluster1] {"level":"info","ts":1604318472.7063785,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3630. 2020-11-02T12:06:49.2460595Z [cluster1] {"level":"info","ts":1604318472.7100277,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3631. 2020-11-02T12:06:49.2464481Z [cluster1] {"level":"info","ts":1604318472.712832,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3632. 2020-11-02T12:06:49.2466606Z [cluster1] {"level":"info","ts":1604318477.7121108,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3633. 2020-11-02T12:06:49.2467630Z [cluster1] {"level":"info","ts":1604318477.7185793,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3634. 2020-11-02T12:06:49.2468384Z [cluster1] {"level":"info","ts":1604318477.7186012,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3635. 2020-11-02T12:06:49.2470085Z [cluster1] {"level":"info","ts":1604318477.72182,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3636. 2020-11-02T12:06:49.2473041Z [cluster1] {"level":"info","ts":1604318477.7249432,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3637. 2020-11-02T12:06:49.2475134Z [cluster1] {"level":"info","ts":1604318482.724666,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3638. 2020-11-02T12:06:49.2476469Z [cluster1] {"level":"info","ts":1604318482.7326224,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3639. 2020-11-02T12:06:49.2484904Z [submariner]$ [cluster1] namespace=submariner-operator
  3640. 2020-11-02T12:06:49.2486303Z [submariner]$ [cluster1] kubectl get pods --selector=app=submariner-globalnet -n submariner-operator -o jsonpath={.items[*].metadata.name}
  3641. 2020-11-02T12:06:49.2488057Z [submariner]$ [cluster1] kubectl get pods --selector=app=submariner-globalnet -n submariner-operator -o jsonpath={.items[*].metadata.name}
  3642. 2020-11-02T12:06:49.2489773Z [submariner]$ [cluster1] command kubectl --context=cluster1 get pods --selector=app=submariner-globalnet -n submariner-operator -o jsonpath={.items[*].metadata.name}
  3643. 2020-11-02T12:06:49.2491347Z [submariner]$ [cluster1] kubectl --context=cluster1 get pods --selector=app=submariner-globalnet -n submariner-operator -o jsonpath={.items[*].metadata.name}
  3644. 2020-11-02T12:06:49.2492372Z [cluster1] {"level":"info","ts":1604318482.7326431,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3645. 2020-11-02T12:06:49.2494096Z [cluster1] {"level":"info","ts":1604318482.7358034,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3646. 2020-11-02T12:06:49.2496922Z [cluster1] {"level":"info","ts":1604318482.7393627,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3647. 2020-11-02T12:06:49.2502713Z [cluster1] {"level":"info","ts":1604318487.736096,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3648. 2020-11-02T12:06:49.2503783Z [cluster1] {"level":"info","ts":1604318487.74143,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3649. 2020-11-02T12:06:49.2504535Z [cluster1] {"level":"info","ts":1604318487.7414465,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3650. 2020-11-02T12:06:49.2506421Z [cluster1] {"level":"info","ts":1604318487.7454093,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3651. 2020-11-02T12:06:49.2508820Z [cluster1] {"level":"info","ts":1604318487.748196,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3652. 2020-11-02T12:06:49.2510749Z [cluster1] {"level":"info","ts":1604318492.7498624,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3653. 2020-11-02T12:06:49.2511990Z [cluster1] {"level":"info","ts":1604318492.7550335,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3654. 2020-11-02T12:06:49.2512752Z [cluster1] {"level":"info","ts":1604318492.7550557,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3655. 2020-11-02T12:06:49.2514627Z [cluster1] {"level":"info","ts":1604318492.7593513,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3656. 2020-11-02T12:06:49.2516944Z [cluster1] {"level":"info","ts":1604318492.763093,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3657. 2020-11-02T12:06:49.2518817Z [cluster1] {"level":"info","ts":1604318497.7606838,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3658. 2020-11-02T12:06:49.2519801Z [cluster1] {"level":"info","ts":1604318497.7678368,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3659. 2020-11-02T12:06:49.2520519Z [cluster1] {"level":"info","ts":1604318497.76786,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3660. 2020-11-02T12:06:49.2522520Z [cluster1] {"level":"info","ts":1604318497.7729635,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3661. 2020-11-02T12:06:49.2525019Z [cluster1] {"level":"info","ts":1604318497.776824,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3662. 2020-11-02T12:06:49.2528102Z [cluster1] {"level":"info","ts":1604318502.7816567,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3663. 2020-11-02T12:06:49.2529104Z [cluster1] {"level":"info","ts":1604318502.78812,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3664. 2020-11-02T12:06:49.2529840Z [cluster1] {"level":"info","ts":1604318502.788152,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3665. 2020-11-02T12:06:49.2531627Z [cluster1] {"level":"info","ts":1604318502.792025,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3666. 2020-11-02T12:06:49.2535864Z [cluster1] {"level":"info","ts":1604318502.7949862,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3667. 2020-11-02T12:06:49.2538106Z [cluster1] {"level":"info","ts":1604318502.801899,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3668. 2020-11-02T12:06:49.2539272Z [cluster1] {"level":"info","ts":1604318502.8126297,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3669. 2020-11-02T12:06:49.2540130Z [cluster1] {"level":"info","ts":1604318502.8126466,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3670. 2020-11-02T12:06:49.2541740Z [cluster1] {"level":"info","ts":1604318502.8153672,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3671. 2020-11-02T12:06:49.2547121Z [cluster1] {"level":"info","ts":1604318502.8178363,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3672. 2020-11-02T12:06:49.2549180Z [cluster1] {"level":"info","ts":1604318507.7950025,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3673. 2020-11-02T12:06:49.2550490Z [cluster1] {"level":"info","ts":1604318507.801001,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3674. 2020-11-02T12:06:49.2551340Z [cluster1] {"level":"info","ts":1604318507.8010228,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3675. 2020-11-02T12:06:49.2556608Z [cluster1] {"level":"info","ts":1604318507.803918,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3676. 2020-11-02T12:06:49.2559048Z [cluster1] {"level":"info","ts":1604318507.8066127,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3677. 2020-11-02T12:06:49.2561082Z [cluster1] {"level":"info","ts":1604318512.8135548,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3678. 2020-11-02T12:06:49.2562043Z [cluster1] {"level":"info","ts":1604318512.8228908,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3679. 2020-11-02T12:06:49.2562748Z [cluster1] {"level":"info","ts":1604318512.8229108,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3680. 2020-11-02T12:06:49.2564852Z [cluster1] {"level":"info","ts":1604318512.8263314,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3681. 2020-11-02T12:06:49.2567124Z [cluster1] {"level":"info","ts":1604318512.8292916,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3682. 2020-11-02T12:06:49.2569035Z [cluster1] {"level":"info","ts":1604318517.822518,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3683. 2020-11-02T12:06:49.2570001Z [cluster1] {"level":"info","ts":1604318517.8293798,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3684. 2020-11-02T12:06:49.2570703Z [cluster1] {"level":"info","ts":1604318517.8294535,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3685. 2020-11-02T12:06:49.2577410Z [cluster1] {"level":"info","ts":1604318517.8330698,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3686. 2020-11-02T12:06:49.2580103Z [cluster1] {"level":"info","ts":1604318517.835996,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3687. 2020-11-02T12:06:49.2582784Z [cluster1] {"level":"info","ts":1604318522.8389978,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3688. 2020-11-02T12:06:49.2583734Z [cluster1] {"level":"info","ts":1604318522.8450089,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3689. 2020-11-02T12:06:49.2584708Z [cluster1] {"level":"info","ts":1604318522.8450332,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3690. 2020-11-02T12:06:49.2586366Z [cluster1] {"level":"info","ts":1604318522.8480363,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3691. 2020-11-02T12:06:49.2589173Z [cluster1] {"level":"info","ts":1604318522.8518674,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3692. 2020-11-02T12:06:49.2591508Z [cluster1] {"level":"info","ts":1604318527.854673,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3693. 2020-11-02T12:06:49.2592684Z [cluster1] {"level":"info","ts":1604318527.8601613,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3694. 2020-11-02T12:06:49.2593431Z [cluster1] {"level":"info","ts":1604318527.8601975,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3695. 2020-11-02T12:06:49.2595607Z [cluster1] {"level":"info","ts":1604318527.866062,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3696. 2020-11-02T12:06:49.2597840Z [cluster1] {"level":"info","ts":1604318527.8690891,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3697. 2020-11-02T12:06:49.2599847Z [cluster1] {"level":"info","ts":1604318532.868489,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3698. 2020-11-02T12:06:49.2601017Z [cluster1] {"level":"info","ts":1604318532.8757777,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3699. 2020-11-02T12:06:49.2601773Z [cluster1] {"level":"info","ts":1604318532.875816,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3700. 2020-11-02T12:06:49.2604153Z [cluster1] {"level":"info","ts":1604318532.8791049,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3701. 2020-11-02T12:06:49.2606449Z [cluster1] {"level":"info","ts":1604318532.8836145,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3702. 2020-11-02T12:06:49.2613091Z [cluster1] {"level":"info","ts":1604318537.8861287,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3703. 2020-11-02T12:06:49.2614431Z [cluster1] {"level":"info","ts":1604318537.8915863,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3704. 2020-11-02T12:06:49.2615141Z [cluster1] {"level":"info","ts":1604318537.8916101,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3705. 2020-11-02T12:06:49.2616985Z [cluster1] {"level":"info","ts":1604318537.8950844,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3706. 2020-11-02T12:06:49.2619652Z [cluster1] {"level":"info","ts":1604318537.8979666,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3707. 2020-11-02T12:06:49.2621638Z [cluster1] {"level":"info","ts":1604318542.8976343,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3708. 2020-11-02T12:06:49.2622694Z [cluster1] {"level":"info","ts":1604318542.9069986,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3709. 2020-11-02T12:06:49.2623397Z [cluster1] {"level":"info","ts":1604318542.9070141,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3710. 2020-11-02T12:06:49.2625792Z [cluster1] {"level":"info","ts":1604318542.9114397,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3711. 2020-11-02T12:06:49.2628255Z [cluster1] {"level":"info","ts":1604318542.915487,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3712. 2020-11-02T12:06:49.2630282Z [cluster1] {"level":"info","ts":1604318547.9183037,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3713. 2020-11-02T12:06:49.2631237Z [cluster1] {"level":"info","ts":1604318547.924063,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3714. 2020-11-02T12:06:49.2632127Z [cluster1] {"level":"info","ts":1604318547.9240842,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3715. 2020-11-02T12:06:49.2633779Z [cluster1] {"level":"info","ts":1604318547.928257,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3716. 2020-11-02T12:06:49.2636553Z [cluster1] {"level":"info","ts":1604318547.9317822,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3717. 2020-11-02T12:06:49.2638436Z [cluster1] {"level":"info","ts":1604318552.9363718,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3718. 2020-11-02T12:06:49.2639396Z [cluster1] {"level":"info","ts":1604318552.94117,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3719. 2020-11-02T12:06:49.2640094Z [cluster1] {"level":"info","ts":1604318552.9411862,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3720. 2020-11-02T12:06:49.2641720Z [cluster1] {"level":"info","ts":1604318552.9447956,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3721. 2020-11-02T12:06:49.2644297Z [cluster1] {"level":"info","ts":1604318552.9480007,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3722. 2020-11-02T12:06:49.2646400Z [cluster1] {"level":"info","ts":1604318557.960382,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3723. 2020-11-02T12:06:49.2647352Z [cluster1] {"level":"info","ts":1604318557.9675455,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3724. 2020-11-02T12:06:49.2648059Z [cluster1] {"level":"info","ts":1604318557.967567,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3725. 2020-11-02T12:06:49.2649679Z [cluster1] {"level":"info","ts":1604318557.9714048,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3726. 2020-11-02T12:06:49.2651911Z [cluster1] {"level":"info","ts":1604318557.974467,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3727. 2020-11-02T12:06:49.2653838Z [cluster1] {"level":"info","ts":1604318562.9773471,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3728. 2020-11-02T12:06:49.2654758Z [cluster1] {"level":"info","ts":1604318562.9983084,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3729. 2020-11-02T12:06:49.2655561Z [cluster1] {"level":"info","ts":1604318562.9983308,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3730. 2020-11-02T12:06:49.2657591Z [cluster1] {"level":"info","ts":1604318563.0032308,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3731. 2020-11-02T12:06:49.2659818Z [cluster1] {"level":"info","ts":1604318563.0061307,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3732. 2020-11-02T12:06:49.2662395Z [cluster1] {"level":"info","ts":1604318568.0030086,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3733. 2020-11-02T12:06:49.2663338Z [cluster1] {"level":"info","ts":1604318568.0139647,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3734. 2020-11-02T12:06:49.2664021Z [cluster1] {"level":"info","ts":1604318568.0139892,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3735. 2020-11-02T12:06:49.2665754Z [cluster1] {"level":"info","ts":1604318568.0225465,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3736. 2020-11-02T12:06:49.2667960Z [cluster1] {"level":"info","ts":1604318568.0273192,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3737. 2020-11-02T12:06:49.2669705Z [cluster1] {"level":"info","ts":1604318573.0207503,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3738. 2020-11-02T12:06:49.2670634Z [cluster1] {"level":"info","ts":1604318573.026718,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3739. 2020-11-02T12:06:49.2671316Z [cluster1] {"level":"info","ts":1604318573.0267413,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3740. 2020-11-02T12:06:49.2672870Z [cluster1] {"level":"info","ts":1604318573.0303993,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3741. 2020-11-02T12:06:49.2675016Z [cluster1] {"level":"info","ts":1604318573.0328655,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3742. 2020-11-02T12:06:49.2679599Z [cluster1] {"level":"info","ts":1604318578.0330706,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3743. 2020-11-02T12:06:49.2680744Z [cluster1] {"level":"info","ts":1604318578.038372,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3744. 2020-11-02T12:06:49.2681454Z [cluster1] {"level":"info","ts":1604318578.0383935,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3745. 2020-11-02T12:06:49.2683120Z [cluster1] {"level":"info","ts":1604318578.0413787,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3746. 2020-11-02T12:06:49.2687815Z [cluster1] {"level":"info","ts":1604318578.0442903,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3747. 2020-11-02T12:06:49.2691157Z [cluster1] {"level":"info","ts":1604318583.049238,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3748. 2020-11-02T12:06:49.2692244Z [cluster1] {"level":"info","ts":1604318583.056562,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3749. 2020-11-02T12:06:49.2693306Z [cluster1] {"level":"info","ts":1604318583.0565856,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3750. 2020-11-02T12:06:49.2695140Z [cluster1] {"level":"info","ts":1604318583.0611787,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3751. 2020-11-02T12:06:49.2698771Z [cluster1] {"level":"info","ts":1604318583.0636988,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3752. 2020-11-02T12:06:49.2701092Z [cluster1] {"level":"info","ts":1604318588.065111,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3753. 2020-11-02T12:06:49.2702250Z [cluster1] {"level":"info","ts":1604318588.0706236,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3754. 2020-11-02T12:06:49.2702930Z [cluster1] {"level":"info","ts":1604318588.070639,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3755. 2020-11-02T12:06:49.2704510Z [cluster1] {"level":"info","ts":1604318588.0763016,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3756. 2020-11-02T12:06:49.2707661Z [cluster1] {"level":"info","ts":1604318588.0796652,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3757. 2020-11-02T12:06:49.2709865Z [cluster1] {"level":"info","ts":1604318593.0787454,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3758. 2020-11-02T12:06:49.2711496Z [cluster1] {"level":"info","ts":1604318593.0846438,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3759. 2020-11-02T12:06:49.2712204Z [cluster1] {"level":"info","ts":1604318593.0846663,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3760. 2020-11-02T12:06:49.2714419Z [cluster1] {"level":"info","ts":1604318593.0879672,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3761. 2020-11-02T12:06:49.2717031Z [cluster1] {"level":"info","ts":1604318593.0916688,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3762. 2020-11-02T12:06:49.2719149Z [cluster1] {"level":"info","ts":1604318598.0901456,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3763. 2020-11-02T12:06:49.2720048Z [cluster1] {"level":"info","ts":1604318598.0959122,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3764. 2020-11-02T12:06:49.2720832Z [cluster1] {"level":"info","ts":1604318598.0959346,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3765. 2020-11-02T12:06:49.2722628Z [cluster1] {"level":"info","ts":1604318598.0994282,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3766. 2020-11-02T12:06:49.2725584Z [cluster1] {"level":"info","ts":1604318598.1049273,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3767. 2020-11-02T12:06:49.2727382Z [cluster1] {"level":"info","ts":1604318603.102506,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3768. 2020-11-02T12:06:49.2728313Z [cluster1] {"level":"info","ts":1604318603.1085749,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3769. 2020-11-02T12:06:49.2729483Z [cluster1] {"level":"info","ts":1604318603.1085901,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3770. 2020-11-02T12:06:49.2731490Z [cluster1] {"level":"info","ts":1604318603.1120827,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3771. 2020-11-02T12:06:49.2734586Z [cluster1] {"level":"info","ts":1604318603.115246,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3772. 2020-11-02T12:06:49.2736774Z [cluster1] {"level":"info","ts":1604318608.1201026,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3773. 2020-11-02T12:06:49.2738227Z [cluster1] {"level":"info","ts":1604318608.1360683,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3774. 2020-11-02T12:06:49.2738928Z [cluster1] {"level":"info","ts":1604318608.136101,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3775. 2020-11-02T12:06:49.2740677Z [cluster1] {"level":"info","ts":1604318608.1408448,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3776. 2020-11-02T12:06:49.2742830Z [cluster1] {"level":"info","ts":1604318608.144489,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3777. 2020-11-02T12:06:49.2744786Z [cluster1] {"level":"info","ts":1604318613.147837,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3778. 2020-11-02T12:06:49.2745738Z [cluster1] {"level":"info","ts":1604318613.155124,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3779. 2020-11-02T12:06:49.2746431Z [cluster1] {"level":"info","ts":1604318613.1551414,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3780. 2020-11-02T12:06:49.2748033Z [cluster1] {"level":"info","ts":1604318613.1605625,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3781. 2020-11-02T12:06:49.2750683Z [cluster1] {"level":"info","ts":1604318613.1638613,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3782. 2020-11-02T12:06:49.2752706Z [cluster1] {"level":"info","ts":1604318618.1653204,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3783. 2020-11-02T12:06:49.2753625Z [cluster1] {"level":"info","ts":1604318618.17438,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3784. 2020-11-02T12:06:49.2754304Z [cluster1] {"level":"info","ts":1604318618.1744018,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3785. 2020-11-02T12:06:49.2756671Z [cluster1] {"level":"info","ts":1604318618.179326,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3786. 2020-11-02T12:06:49.2759325Z [cluster1] {"level":"info","ts":1604318618.1824775,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3787. 2020-11-02T12:06:49.2761119Z [cluster1] {"level":"info","ts":1604318623.1832983,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3788. 2020-11-02T12:06:49.2762032Z [cluster1] {"level":"info","ts":1604318623.1891096,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3789. 2020-11-02T12:06:49.2762711Z [cluster1] {"level":"info","ts":1604318623.1891313,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3790. 2020-11-02T12:06:49.2764622Z [cluster1] {"level":"info","ts":1604318623.1921952,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3791. 2020-11-02T12:06:49.2768256Z [cluster1] {"level":"info","ts":1604318623.1967707,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3792. 2020-11-02T12:06:49.2771296Z [cluster1] {"level":"info","ts":1604318628.195256,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3793. 2020-11-02T12:06:49.2772455Z [cluster1] {"level":"info","ts":1604318628.2022767,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3794. 2020-11-02T12:06:49.2773496Z [cluster1] {"level":"info","ts":1604318628.2022989,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3795. 2020-11-02T12:06:49.2775742Z [cluster1] {"level":"info","ts":1604318628.20546,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3796. 2020-11-02T12:06:49.2778677Z [cluster1] {"level":"info","ts":1604318628.2079597,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3797. 2020-11-02T12:06:49.2780494Z [cluster1] {"level":"info","ts":1604318633.2092338,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3798. 2020-11-02T12:06:49.2781846Z [cluster1] {"level":"info","ts":1604318633.2141898,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3799. 2020-11-02T12:06:49.2782541Z [cluster1] {"level":"info","ts":1604318633.2142107,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3800. 2020-11-02T12:06:49.2784188Z [cluster1] {"level":"info","ts":1604318633.217588,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3801. 2020-11-02T12:06:49.2786354Z [cluster1] {"level":"info","ts":1604318633.2197063,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3802. 2020-11-02T12:06:49.2788711Z [cluster1] {"level":"info","ts":1604318638.2275064,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3803. 2020-11-02T12:06:49.2789734Z [cluster1] {"level":"info","ts":1604318638.2343075,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3804. 2020-11-02T12:06:49.2790781Z [cluster1] {"level":"info","ts":1604318638.2343302,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3805. 2020-11-02T12:06:49.2792708Z [cluster1] {"level":"info","ts":1604318638.2375476,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3806. 2020-11-02T12:06:49.2794859Z [cluster1] {"level":"info","ts":1604318638.2404506,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3807. 2020-11-02T12:06:49.2797112Z [cluster1] {"level":"info","ts":1604318643.2434587,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3808. 2020-11-02T12:06:49.2798375Z [cluster1] {"level":"info","ts":1604318643.2538242,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3809. 2020-11-02T12:06:49.2799073Z [cluster1] {"level":"info","ts":1604318643.2538452,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3810. 2020-11-02T12:06:49.2800862Z [cluster1] {"level":"info","ts":1604318643.2572262,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3811. 2020-11-02T12:06:49.2803221Z [cluster1] {"level":"info","ts":1604318643.2596025,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3812. 2020-11-02T12:06:49.2805179Z [cluster1] {"level":"info","ts":1604318648.260034,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3813. 2020-11-02T12:06:49.2806130Z [cluster1] {"level":"info","ts":1604318648.2655733,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3814. 2020-11-02T12:06:49.2806825Z [cluster1] {"level":"info","ts":1604318648.265588,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3815. 2020-11-02T12:06:49.2808425Z [cluster1] {"level":"info","ts":1604318648.269561,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3816. 2020-11-02T12:06:49.2811006Z [cluster1] {"level":"info","ts":1604318648.2721512,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3817. 2020-11-02T12:06:49.2813157Z [cluster1] {"level":"info","ts":1604318653.2774365,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3818. 2020-11-02T12:06:49.2814071Z [cluster1] {"level":"info","ts":1604318653.2883081,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3819. 2020-11-02T12:06:49.2814744Z [cluster1] {"level":"info","ts":1604318653.2883248,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3820. 2020-11-02T12:06:49.2816287Z [cluster1] {"level":"info","ts":1604318653.2929351,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3821. 2020-11-02T12:06:49.2818520Z [cluster1] {"level":"info","ts":1604318653.2963583,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3822. 2020-11-02T12:06:49.2820306Z [cluster1] {"level":"info","ts":1604318658.294538,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3823. 2020-11-02T12:06:49.2821225Z [cluster1] {"level":"info","ts":1604318658.2991092,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3824. 2020-11-02T12:06:49.2821900Z [cluster1] {"level":"info","ts":1604318658.2991254,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3825. 2020-11-02T12:06:49.2823460Z [cluster1] {"level":"info","ts":1604318658.303197,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3826. 2020-11-02T12:06:49.2825968Z [cluster1] {"level":"info","ts":1604318658.305685,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3827. 2020-11-02T12:06:49.2828054Z [cluster1] {"level":"info","ts":1604318663.3096497,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3828. 2020-11-02T12:06:49.2828971Z [cluster1] {"level":"info","ts":1604318663.3245738,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3829. 2020-11-02T12:06:49.2829652Z [cluster1] {"level":"info","ts":1604318663.324607,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3830. 2020-11-02T12:06:49.2831202Z [cluster1] {"level":"info","ts":1604318663.3303554,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3831. 2020-11-02T12:06:49.2834066Z [cluster1] {"level":"info","ts":1604318663.332746,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3832. 2020-11-02T12:06:49.2836351Z [cluster1] {"level":"info","ts":1604318668.3258898,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3833. 2020-11-02T12:06:49.2837562Z [cluster1] {"level":"info","ts":1604318668.3316207,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3834. 2020-11-02T12:06:49.2838217Z [cluster1] {"level":"info","ts":1604318668.3316433,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3835. 2020-11-02T12:06:49.2839764Z [cluster1] {"level":"info","ts":1604318668.3351297,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3836. 2020-11-02T12:06:49.2842064Z [cluster1] {"level":"info","ts":1604318668.3449013,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3837. 2020-11-02T12:06:49.2843811Z [cluster1] {"level":"info","ts":1604318673.3383858,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3838. 2020-11-02T12:06:49.2844968Z [cluster1] {"level":"info","ts":1604318673.344085,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3839. 2020-11-02T12:06:49.2845716Z [cluster1] {"level":"info","ts":1604318673.3441064,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3840. 2020-11-02T12:06:49.2847289Z [cluster1] {"level":"info","ts":1604318673.347276,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3841. 2020-11-02T12:06:49.2849379Z [cluster1] {"level":"info","ts":1604318673.3496683,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3842. 2020-11-02T12:06:49.2851081Z [cluster1] {"level":"info","ts":1604318678.3503027,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3843. 2020-11-02T12:06:49.2852164Z [cluster1] {"level":"info","ts":1604318678.358031,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3844. 2020-11-02T12:06:49.2852842Z [cluster1] {"level":"info","ts":1604318678.3580534,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3845. 2020-11-02T12:06:49.2854394Z [cluster1] {"level":"info","ts":1604318678.3612053,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3846. 2020-11-02T12:06:49.2856783Z [cluster1] {"level":"info","ts":1604318678.363528,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3847. 2020-11-02T12:06:49.2860616Z [cluster1] {"level":"info","ts":1604318683.364665,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3848. 2020-11-02T12:06:49.2861614Z [cluster1] {"level":"info","ts":1604318683.3691406,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3849. 2020-11-02T12:06:49.2863028Z [cluster1] {"level":"info","ts":1604318683.3691568,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3850. 2020-11-02T12:06:49.2864937Z [cluster1] {"level":"info","ts":1604318683.3732386,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3851. 2020-11-02T12:06:49.2867733Z [cluster1] {"level":"info","ts":1604318683.3758223,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3852. 2020-11-02T12:06:49.2869826Z [cluster1] {"level":"info","ts":1604318688.3819466,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3853. 2020-11-02T12:06:49.2871011Z [cluster1] {"level":"info","ts":1604318688.388229,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3854. 2020-11-02T12:06:49.2871688Z [cluster1] {"level":"info","ts":1604318688.3882453,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3855. 2020-11-02T12:06:49.2874357Z [cluster1] {"level":"info","ts":1604318688.3918045,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3856. 2020-11-02T12:06:49.2877190Z [cluster1] {"level":"info","ts":1604318688.394556,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3857. 2020-11-02T12:06:49.2878936Z [cluster1] {"level":"info","ts":1604318693.3967717,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3858. 2020-11-02T12:06:49.2880211Z [cluster1] {"level":"info","ts":1604318693.4021118,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3859. 2020-11-02T12:06:49.2880916Z [cluster1] {"level":"info","ts":1604318693.402133,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3860. 2020-11-02T12:06:49.2882682Z [cluster1] {"level":"info","ts":1604318693.4049633,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3861. 2020-11-02T12:06:49.2885997Z [cluster1] {"level":"info","ts":1604318693.407203,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3862. 2020-11-02T12:06:49.2889898Z [cluster1] {"level":"info","ts":1604318698.4147432,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3863. 2020-11-02T12:06:49.2891147Z [cluster1] {"level":"info","ts":1604318698.428379,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3864. 2020-11-02T12:06:49.2891839Z [cluster1] {"level":"info","ts":1604318698.4284275,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3865. 2020-11-02T12:06:49.2893769Z [cluster1] {"level":"info","ts":1604318698.4448302,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3866. 2020-11-02T12:06:49.2896621Z [cluster1] {"level":"info","ts":1604318698.4520311,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3867. 2020-11-02T12:06:49.2901426Z [cluster1] {"level":"info","ts":1604318703.4338977,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3868. 2020-11-02T12:06:49.2902905Z [cluster1] {"level":"info","ts":1604318703.4625602,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3869. 2020-11-02T12:06:49.2907344Z [cluster1] {"level":"info","ts":1604318703.46258,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3870. 2020-11-02T12:06:49.2909377Z [cluster1] {"level":"info","ts":1604318703.5051248,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3871. 2020-11-02T12:06:49.2913146Z [cluster1] {"level":"info","ts":1604318703.5085495,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3872. 2020-11-02T12:06:49.2916669Z [cluster1] {"level":"info","ts":1604318708.4593103,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3873. 2020-11-02T12:06:49.2917751Z [cluster1] {"level":"info","ts":1604318708.465629,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3874. 2020-11-02T12:06:49.2918649Z [cluster1] {"level":"info","ts":1604318708.4656484,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3875. 2020-11-02T12:06:49.2920491Z [cluster1] {"level":"info","ts":1604318708.470041,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3876. 2020-11-02T12:06:49.2924128Z [cluster1] {"level":"info","ts":1604318708.4731653,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3877. 2020-11-02T12:06:49.2926829Z [cluster1] {"level":"info","ts":1604318713.4803746,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3878. 2020-11-02T12:06:49.2927788Z [cluster1] {"level":"info","ts":1604318713.4871583,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3879. 2020-11-02T12:06:49.2928471Z [cluster1] {"level":"info","ts":1604318713.4871829,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3880. 2020-11-02T12:06:49.2930078Z [cluster1] {"level":"info","ts":1604318713.4906185,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3881. 2020-11-02T12:06:49.2932314Z [cluster1] {"level":"info","ts":1604318713.4929516,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3882. 2020-11-02T12:06:49.2934369Z [cluster1] {"level":"info","ts":1604318718.4930296,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3883. 2020-11-02T12:06:49.2935336Z [cluster1] {"level":"info","ts":1604318718.4982686,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3884. 2020-11-02T12:06:49.2937309Z [cluster1] {"level":"info","ts":1604318718.4982917,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3885. 2020-11-02T12:06:49.2939354Z [cluster1] {"level":"info","ts":1604318718.5017145,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3886. 2020-11-02T12:06:49.2942132Z [cluster1] {"level":"info","ts":1604318718.5048473,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3887. 2020-11-02T12:06:49.2945288Z [cluster1] {"level":"info","ts":1604318723.5107937,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3888. 2020-11-02T12:06:49.2947942Z [cluster1] {"level":"info","ts":1604318723.5189555,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3889. 2020-11-02T12:06:49.2949019Z [cluster1] {"level":"info","ts":1604318723.518972,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3890. 2020-11-02T12:06:49.2950890Z [cluster1] {"level":"info","ts":1604318723.5228949,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3891. 2020-11-02T12:06:49.2953795Z [cluster1] {"level":"info","ts":1604318723.5266767,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3892. 2020-11-02T12:06:49.2958882Z [cluster1] {"level":"info","ts":1604318728.5268536,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3893. 2020-11-02T12:06:49.2960614Z [cluster1] {"level":"info","ts":1604318728.5311205,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3894. 2020-11-02T12:06:49.2961497Z [cluster1] {"level":"info","ts":1604318728.531143,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3895. 2020-11-02T12:06:49.2964090Z [cluster1] {"level":"info","ts":1604318728.5340374,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3896. 2020-11-02T12:06:49.2966728Z [cluster1] {"level":"info","ts":1604318728.5360973,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3897. 2020-11-02T12:06:49.2968982Z [cluster1] {"level":"info","ts":1604318733.536966,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3898. 2020-11-02T12:06:49.2969973Z [cluster1] {"level":"info","ts":1604318733.5430424,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3899. 2020-11-02T12:06:49.2970695Z [cluster1] {"level":"info","ts":1604318733.5430655,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3900. 2020-11-02T12:06:49.2972359Z [cluster1] {"level":"info","ts":1604318733.54996,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3901. 2020-11-02T12:06:49.2975753Z [cluster1] {"level":"info","ts":1604318733.554215,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3902. 2020-11-02T12:06:49.2979416Z [cluster1] {"level":"info","ts":1604318733.5648096,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3903. 2020-11-02T12:06:49.2980619Z [cluster1] {"level":"info","ts":1604318733.5692356,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3904. 2020-11-02T12:06:49.2981350Z [cluster1] {"level":"info","ts":1604318733.5692549,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3905. 2020-11-02T12:06:49.2983942Z [cluster1] {"level":"info","ts":1604318733.5722325,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3906. 2020-11-02T12:06:49.2986579Z [cluster1] {"level":"info","ts":1604318733.5752597,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3907. 2020-11-02T12:06:49.2988582Z [cluster1] {"level":"info","ts":1604318738.5529742,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3908. 2020-11-02T12:06:49.2989660Z [cluster1] {"level":"info","ts":1604318738.5584576,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3909. 2020-11-02T12:06:49.2990383Z [cluster1] {"level":"info","ts":1604318738.5584805,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3910. 2020-11-02T12:06:49.2992276Z [cluster1] {"level":"info","ts":1604318738.5617044,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3911. 2020-11-02T12:06:49.2994570Z [cluster1] {"level":"info","ts":1604318738.564149,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3912. 2020-11-02T12:06:49.2996381Z [cluster1] {"level":"info","ts":1604318743.564064,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3913. 2020-11-02T12:06:49.2997329Z [cluster1] {"level":"info","ts":1604318743.571144,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3914. 2020-11-02T12:06:49.2998086Z [cluster1] {"level":"info","ts":1604318743.5711675,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3915. 2020-11-02T12:06:49.2999702Z [cluster1] {"level":"info","ts":1604318743.5748098,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3916. 2020-11-02T12:06:49.3002355Z [cluster1] {"level":"info","ts":1604318743.578106,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3917. 2020-11-02T12:06:49.3004732Z [cluster1] {"level":"info","ts":1604318748.5780685,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3918. 2020-11-02T12:06:49.3005817Z [cluster1] {"level":"info","ts":1604318748.584814,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3919. 2020-11-02T12:06:49.3006494Z [cluster1] {"level":"info","ts":1604318748.5848377,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3920. 2020-11-02T12:06:49.3008443Z [cluster1] {"level":"info","ts":1604318748.5879645,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3921. 2020-11-02T12:06:49.3010672Z [cluster1] {"level":"info","ts":1604318748.590696,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3922. 2020-11-02T12:06:49.3012477Z [cluster1] {"level":"info","ts":1604318753.5919795,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3923. 2020-11-02T12:06:49.3013420Z [cluster1] {"level":"info","ts":1604318753.598125,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3924. 2020-11-02T12:06:49.3014790Z [cluster1] {"level":"info","ts":1604318753.5981445,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3925. 2020-11-02T12:06:49.3016985Z [cluster1] {"level":"info","ts":1604318753.6016676,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3926. 2020-11-02T12:06:49.3020737Z [cluster1] {"level":"info","ts":1604318753.6040316,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3927. 2020-11-02T12:06:49.3023103Z [cluster1] {"level":"info","ts":1604318758.6074917,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3928. 2020-11-02T12:06:49.3024060Z [cluster1] {"level":"info","ts":1604318758.6161082,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3929. 2020-11-02T12:06:49.3024777Z [cluster1] {"level":"info","ts":1604318758.6161346,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3930. 2020-11-02T12:06:49.3026389Z [cluster1] {"level":"info","ts":1604318758.621269,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3931. 2020-11-02T12:06:49.3029291Z [cluster1] {"level":"info","ts":1604318758.6255255,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3932. 2020-11-02T12:06:49.3031216Z [cluster1] {"level":"info","ts":1604318763.6288302,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3933. 2020-11-02T12:06:49.3032658Z [cluster1] {"level":"info","ts":1604318763.6354594,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3934. 2020-11-02T12:06:49.3033344Z [cluster1] {"level":"info","ts":1604318763.6354806,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3935. 2020-11-02T12:06:49.3034903Z [cluster1] {"level":"info","ts":1604318763.6393445,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3936. 2020-11-02T12:06:49.3037289Z [cluster1] {"level":"info","ts":1604318763.64182,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3937. 2020-11-02T12:06:49.3039920Z [cluster1] {"level":"info","ts":1604318768.6453164,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3938. 2020-11-02T12:06:49.3041467Z [cluster1] {"level":"info","ts":1604318768.6523519,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3939. 2020-11-02T12:06:49.3042349Z [cluster1] {"level":"info","ts":1604318768.6523838,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3940. 2020-11-02T12:06:49.3044230Z [cluster1] {"level":"info","ts":1604318768.6706011,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3941. 2020-11-02T12:06:49.3046493Z [cluster1] {"level":"info","ts":1604318768.6739237,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3942. 2020-11-02T12:06:49.3048305Z [cluster1] {"level":"info","ts":1604318773.6785011,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3943. 2020-11-02T12:06:49.3049382Z [cluster1] {"level":"info","ts":1604318773.688979,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3944. 2020-11-02T12:06:49.3050098Z [cluster1] {"level":"info","ts":1604318773.6890275,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3945. 2020-11-02T12:06:49.3051730Z [cluster1] {"level":"info","ts":1604318773.6935632,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3946. 2020-11-02T12:06:49.3053953Z [cluster1] {"level":"info","ts":1604318773.698483,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3947. 2020-11-02T12:06:49.3055972Z [cluster1] {"level":"info","ts":1604318778.707179,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3948. 2020-11-02T12:06:49.3057087Z [cluster1] {"level":"info","ts":1604318778.7303247,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3949. 2020-11-02T12:06:49.3057794Z [cluster1] {"level":"info","ts":1604318778.7303507,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3950. 2020-11-02T12:06:49.3059452Z [cluster1] {"level":"info","ts":1604318778.7477858,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3951. 2020-11-02T12:06:49.3061686Z [cluster1] {"level":"info","ts":1604318778.7518084,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3952. 2020-11-02T12:06:49.3064535Z [cluster1] {"level":"info","ts":1604318783.759681,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3953. 2020-11-02T12:06:49.3065785Z [cluster1] {"level":"info","ts":1604318783.7677612,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3954. 2020-11-02T12:06:49.3066464Z [cluster1] {"level":"info","ts":1604318783.7677836,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3955. 2020-11-02T12:06:49.3068194Z [cluster1] {"level":"info","ts":1604318783.7762609,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3956. 2020-11-02T12:06:49.3072114Z [cluster1] {"level":"info","ts":1604318783.7798424,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3957. 2020-11-02T12:06:49.3075006Z [cluster1] {"level":"info","ts":1604318788.7740457,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3958. 2020-11-02T12:06:49.3076294Z [cluster1] {"level":"info","ts":1604318788.779934,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3959. 2020-11-02T12:06:49.3077006Z [cluster1] {"level":"info","ts":1604318788.7799575,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3960. 2020-11-02T12:06:49.3079774Z [cluster1] {"level":"info","ts":1604318788.7826715,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3961. 2020-11-02T12:06:49.3084237Z [cluster1] {"level":"info","ts":1604318788.7904606,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3962. 2020-11-02T12:06:49.3087793Z [cluster1] {"level":"info","ts":1604318793.7859218,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3963. 2020-11-02T12:06:49.3089115Z [cluster1] {"level":"info","ts":1604318793.7932038,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3964. 2020-11-02T12:06:49.3090216Z [cluster1] {"level":"info","ts":1604318793.793222,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3965. 2020-11-02T12:06:49.3096561Z [cluster1] {"level":"info","ts":1604318793.7982082,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3966. 2020-11-02T12:06:49.3098971Z [cluster1] {"level":"info","ts":1604318793.8008504,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3967. 2020-11-02T12:06:49.3101533Z [cluster1] {"level":"info","ts":1604318798.8010936,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3968. 2020-11-02T12:06:49.3103276Z [cluster1] {"level":"info","ts":1604318798.8100774,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3969. 2020-11-02T12:06:49.3104454Z [cluster1] {"level":"info","ts":1604318798.8101056,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3970. 2020-11-02T12:06:49.3106465Z [cluster1] {"level":"info","ts":1604318798.8146825,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3971. 2020-11-02T12:06:49.3109346Z [cluster1] {"level":"info","ts":1604318798.8188145,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3972. 2020-11-02T12:06:49.3117976Z [cluster1] {"level":"info","ts":1604318803.868159,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3973. 2020-11-02T12:06:49.3118960Z [cluster1] {"level":"info","ts":1604318803.874162,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3974. 2020-11-02T12:06:49.3119666Z [cluster1] {"level":"info","ts":1604318803.8741972,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3975. 2020-11-02T12:06:49.3122042Z [cluster1] {"level":"info","ts":1604318803.8788602,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3976. 2020-11-02T12:06:49.3125523Z [cluster1] {"level":"info","ts":1604318803.8819213,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3977. 2020-11-02T12:06:49.3128580Z [cluster1] {"level":"info","ts":1604318808.891386,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  3978. 2020-11-02T12:06:49.3142743Z [cluster1] {"level":"info","ts":1604318808.8974924,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.1.0.0/16"}
  3979. 2020-11-02T12:06:49.3143472Z [cluster1] {"level":"info","ts":1604318808.897509,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.1.0.0/16"}
  3980. 2020-11-02T12:06:49.5216297Z [submariner]$ [cluster1] kubectl get pods --selector=app=submariner-engine -n submariner-operator -o jsonpath={.items[*].metadata.name}
  3981. 2020-11-02T12:06:49.5228124Z [submariner]$ [cluster1] kubectl get pods --selector=app=submariner-engine -n submariner-operator -o jsonpath={.items[*].metadata.name}
  3982. 2020-11-02T12:06:49.5239047Z [submariner]$ [cluster1] command kubectl --context=cluster1 get pods --selector=app=submariner-engine -n submariner-operator -o jsonpath={.items[*].metadata.name}
  3983. 2020-11-02T12:06:49.5251975Z [submariner]$ [cluster1] kubectl --context=cluster1 get pods --selector=app=submariner-engine -n submariner-operator -o jsonpath={.items[*].metadata.name}
  3984. 2020-11-02T12:06:49.7967109Z [submariner]$ [cluster1] kubectl -n submariner-operator logs submariner-gateway-dtktc
  3985. 2020-11-02T12:06:49.7995076Z [submariner]$ [cluster1] kubectl -n submariner-operator logs submariner-gateway-dtktc
  3986. 2020-11-02T12:06:49.8005796Z [submariner]$ [cluster1] command kubectl --context=cluster1 -n submariner-operator logs submariner-gateway-dtktc
  3987. 2020-11-02T12:06:49.8016871Z [submariner]$ [cluster1] kubectl --context=cluster1 -n submariner-operator logs submariner-gateway-dtktc
  3988. 2020-11-02T12:06:50.0861716Z [cluster1] {"level":"info","ts":1604318808.901036,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  3989. 2020-11-02T12:06:50.0864347Z [cluster1] {"level":"info","ts":1604318808.9046443,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  3990. 2020-11-02T12:06:50.0865491Z [cluster1] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  3991. 2020-11-02T12:06:50.0865770Z [cluster1] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  3992. 2020-11-02T12:06:50.0866543Z [cluster1] +++++++++++++++++++++: Logs for Pod submariner-gateway-dtktc in namespace submariner-operator :++++++++++++++++++++++
  3993. 2020-11-02T12:06:50.0867553Z [cluster1] + trap 'exit 1' SIGTERM SIGINT
  3994. 2020-11-02T12:06:50.0867956Z [cluster1] + export CHARON_PID_FILE=/var/run/charon.pid
  3995. 2020-11-02T12:06:50.0868373Z [cluster1] + CHARON_PID_FILE=/var/run/charon.pid
  3996. 2020-11-02T12:06:50.0868869Z [cluster1] + rm -f /var/run/charon.pid
  3997. 2020-11-02T12:06:50.0869225Z [cluster1] + SUBMARINER_VERBOSITY=2
  3998. 2020-11-02T12:06:50.0869681Z [cluster1] + '[' false == true ']'
  3999. 2020-11-02T12:06:50.0870069Z [cluster1] + DEBUG=-v=2
  4000. 2020-11-02T12:06:50.0870449Z [cluster1] + mkdir -p /etc/ipsec
  4001. 2020-11-02T12:06:50.0870861Z [cluster1] net.ipv4.conf.all.send_redirects = 0
  4002. 2020-11-02T12:06:50.0871555Z [cluster1] iptables-save is present on the host at /usr/sbin/iptables-save
  4003. 2020-11-02T12:06:50.0872262Z [cluster1] + sysctl -w net.ipv4.conf.all.send_redirects=0
  4004. 2020-11-02T12:06:50.0872868Z [cluster1] + export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/libexec/strongswan
  4005. 2020-11-02T12:06:50.0873502Z [cluster1] + PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/libexec/strongswan
  4006. 2020-11-02T12:06:50.0874637Z [cluster1] + for f in iptables-save iptables
  4007. 2020-11-02T12:06:50.0875207Z [cluster1] ++ find_iptables_on_host iptables-save
  4008. 2020-11-02T12:06:50.0875783Z [cluster1] ++ chroot /host test -x /usr/sbin/iptables-save
  4009. 2020-11-02T12:06:50.0876131Z [cluster1] ++ echo /usr/sbin
  4010. 2020-11-02T12:06:50.0876572Z [cluster1] ++ return
  4011. 2020-11-02T12:06:50.0876878Z [cluster1] + location=/usr/sbin
  4012. 2020-11-02T12:06:50.0877369Z [cluster1] + '[' /usr/sbin '!=' unknown ']'
  4013. 2020-11-02T12:06:50.0877989Z [cluster1] + echo 'iptables-save is present on the host at /usr/sbin/iptables-save'
  4014. 2020-11-02T12:06:50.0878705Z [cluster1] + sed 's!@@PATH@@!/usr/sbin!' /usr/sbin/iptables-wrapper.in
  4015. 2020-11-02T12:06:50.0879300Z [cluster1] + for f in iptables-save iptables
  4016. 2020-11-02T12:06:50.0879692Z [cluster1] ++ find_iptables_on_host iptables
  4017. 2020-11-02T12:06:50.0880194Z [cluster1] ++ chroot /host test -x /usr/sbin/iptables
  4018. 2020-11-02T12:06:50.0880540Z [cluster1] ++ echo /usr/sbin
  4019. 2020-11-02T12:06:50.0880809Z [cluster1] ++ return
  4020. 2020-11-02T12:06:50.0881096Z [cluster1] + location=/usr/sbin
  4021. 2020-11-02T12:06:50.0881545Z [cluster1] + '[' /usr/sbin '!=' unknown ']'
  4022. 2020-11-02T12:06:50.0882096Z [cluster1] + echo 'iptables is present on the host at /usr/sbin/iptables'
  4023. 2020-11-02T12:06:50.0882766Z [cluster1] + sed 's!@@PATH@@!/usr/sbin!' /usr/sbin/iptables-wrapper.in
  4024. 2020-11-02T12:06:50.0883267Z [cluster1] iptables is present on the host at /usr/sbin/iptables
  4025. 2020-11-02T12:06:50.0884286Z [cluster1] + exec submariner-engine -v=2 -alsologtostderr
  4026. 2020-11-02T12:06:50.0884927Z [cluster1] I1102 11:58:09.693663 1 main.go:70] Starting the submariner gateway engine
  4027. 2020-11-02T12:06:50.0885939Z [cluster1] W1102 11:58:09.696271 1 client_config.go:543] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
  4028. 2020-11-02T12:06:50.0886603Z [cluster1] I1102 11:58:09.705594 1 main.go:95] Creating the cable engine
  4029. 2020-11-02T12:06:50.0890113Z [cluster1] I1102 11:58:09.707183 1 syncer.go:61] CableEngine syncer started
  4030. 2020-11-02T12:06:50.0892422Z [cluster1] I1102 11:58:15.711313 1 main.go:252] Gateway leader election config values: main.leaderConfig{LeaseDuration:10, RenewDeadline:5, RetryPeriod:2}
  4031. 2020-11-02T12:06:50.0894620Z [cluster1] I1102 11:58:15.711520 1 main.go:269] Using namespace "submariner-operator" for the leader election lock
  4032. 2020-11-02T12:06:50.0895985Z [cluster1] I1102 11:58:15.711549 1 leaderelection.go:242] attempting to acquire leader lease submariner-operator/submariner-engine-lock...
  4033. 2020-11-02T12:06:50.0897584Z [cluster1] I1102 11:58:16.011766 1 leaderelection.go:252] successfully acquired lease submariner-operator/submariner-engine-lock
  4034. 2020-11-02T12:06:50.0924688Z [cluster1] I1102 11:58:16.012138 1 event.go:281] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"submariner-operator", Name:"submariner-engine-lock", UID:"6892ed8f-2ddf-4283-966b-316997d9b9da", APIVersion:"v1", ResourceVersion:"1668", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' cluster1-worker-submariner-engine became leader
  4035. 2020-11-02T12:06:50.0926358Z [cluster1] I1102 11:58:16.012162 1 main.go:144] Creating the datastore syncer
  4036. 2020-11-02T12:06:50.0926807Z [cluster1] I1102 11:58:16.058857 1 libreswan.go:347] Starting Pluto
  4037. 2020-11-02T12:06:50.0927223Z [cluster1] Initializing NSS database
  4038. 2020-11-02T12:06:50.0927528Z [cluster1]
  4039. 2020-11-02T12:06:50.0927809Z [cluster1] nflog ipsec capture disabled
  4040. 2020-11-02T12:06:50.0928343Z [cluster1] I1102 11:58:21.065569 1 cableengine.go:71] CableEngine controller started, driver: "libreswan"
  4041. 2020-11-02T12:06:50.0928991Z [cluster1] I1102 11:58:21.065685 1 datastoresyncer.go:60] Starting the datastore syncer
  4042. 2020-11-02T12:06:50.0929535Z [cluster1] I1102 11:58:21.067471 1 tunnel.go:22] Starting the tunnel controller
  4043. 2020-11-02T12:06:50.0930303Z [cluster1] I1102 11:58:21.068298 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  4044. 2020-11-02T12:06:50.0931116Z [cluster1] I1102 11:58:21.068357 1 resource_syncer.go:180] Starting syncer "Tunnel controller"
  4045. 2020-11-02T12:06:50.0931742Z [cluster1] I1102 11:58:21.068371 1 resource_syncer.go:193] Syncer "Tunnel controller" waiting for informer cache to sync
  4046. 2020-11-02T12:06:50.0932429Z [cluster1] I1102 11:58:21.168721 1 resource_syncer.go:202] Syncer "Tunnel controller" started
  4047. 2020-11-02T12:06:50.0933236Z [cluster1] I1102 11:58:27.069854 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  4048. 2020-11-02T12:06:50.0934262Z [cluster1] I1102 11:58:27.121404 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  4049. 2020-11-02T12:06:50.0938210Z [cluster1] I1102 11:58:27.128152 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  4050. 2020-11-02T12:06:50.0939538Z [cluster1] I1102 11:58:27.128289 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  4051. 2020-11-02T12:06:50.0940575Z [cluster1] I1102 11:58:27.128463 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  4052. 2020-11-02T12:06:50.0942351Z [cluster1] I1102 11:58:27.128542 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  4053. 2020-11-02T12:06:50.0943165Z [cluster1] I1102 11:58:27.128706 1 datastoresyncer.go:80] Starting the broker syncer
  4054. 2020-11-02T12:06:50.0944111Z [cluster1] I1102 11:58:27.128712 1 resource_syncer.go:180] Starting syncer "local -> broker for *v1.Cluster"
  4055. 2020-11-02T12:06:50.0944991Z [cluster1] I1102 11:58:27.128719 1 resource_syncer.go:193] Syncer "local -> broker for *v1.Cluster" waiting for informer cache to sync
  4056. 2020-11-02T12:06:50.0945825Z [cluster1] I1102 11:58:27.229021 1 resource_syncer.go:202] Syncer "local -> broker for *v1.Cluster" started
  4057. 2020-11-02T12:06:50.0951075Z [cluster1] I1102 11:58:27.229049 1 resource_syncer.go:180] Starting syncer "broker -> local for *v1.Cluster"
  4058. 2020-11-02T12:06:50.0952028Z [cluster1] I1102 11:58:27.229073 1 resource_syncer.go:202] Syncer "broker -> local for *v1.Cluster" started
  4059. 2020-11-02T12:06:50.0952806Z [cluster1] I1102 11:58:27.229076 1 resource_syncer.go:180] Starting syncer "local -> broker for *v1.Endpoint"
  4060. 2020-11-02T12:06:50.0953664Z [cluster1] I1102 11:58:27.229081 1 resource_syncer.go:193] Syncer "local -> broker for *v1.Endpoint" waiting for informer cache to sync
  4061. 2020-11-02T12:06:50.0959990Z [cluster1] I1102 11:58:27.237177 1 resource_syncer.go:264] Syncer "broker -> local for *v1.Cluster" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:19Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2", "namespace":"submariner-k8s-broker", "resourceVersion":"1679", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/clusters/cluster2", "uid":"4ede1863-af75-4242-84c7-e67b484c3c22"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.2.0.0/16"}, "cluster_id":"cluster2", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.2.0.0/16"}}}}
  4062. 2020-11-02T12:06:50.0966501Z [cluster1] I1102 11:58:27.237263 1 resource_syncer.go:276] Syncing resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:19Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2", "namespace":"submariner-k8s-broker", "resourceVersion":"1679", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/clusters/cluster2", "uid":"4ede1863-af75-4242-84c7-e67b484c3c22"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.2.0.0/16"}, "cluster_id":"cluster2", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.2.0.0/16"}}}}
  4063. 2020-11-02T12:06:50.0976870Z [cluster1] I1102 11:58:27.237411 1 federator.go:42] In Distribute for &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:19Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2", "namespace":"submariner-k8s-broker", "resourceVersion":"1679", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/clusters/cluster2", "uid":"4ede1863-af75-4242-84c7-e67b484c3c22"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.2.0.0/16"}, "cluster_id":"cluster2", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.2.0.0/16"}}}}
  4064. 2020-11-02T12:06:50.0990640Z [cluster1] I1102 11:58:27.237600 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  4065. 2020-11-02T12:06:50.0993373Z [cluster1] I1102 11:58:27.239696 1 create_or_update.go:32] Creating resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2", "namespace":"submariner-operator"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.2.0.0/16"}, "cluster_id":"cluster2", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.2.0.0/16"}}}}
  4066. 2020-11-02T12:06:50.0995570Z [cluster1] I1102 11:58:27.249431 1 resource_syncer.go:285] Syncer "broker -> local for *v1.Cluster" successfully synced "submariner-k8s-broker/cluster2"
  4067. 2020-11-02T12:06:50.1035639Z [cluster1] I1102 11:58:27.249683 1 resource_syncer.go:264] Syncer "broker -> local for *v1.Cluster" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3", "namespace":"submariner-k8s-broker", "resourceVersion":"1721", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/clusters/cluster3", "uid":"b0da8c3d-b10e-4fc0-b95e-e145f3a95836"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.3.0.0/16"}, "cluster_id":"cluster3", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.3.0.0/16"}}}}
  4068. 2020-11-02T12:06:50.1041571Z [cluster1] I1102 11:58:27.249738 1 resource_syncer.go:276] Syncing resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3", "namespace":"submariner-k8s-broker", "resourceVersion":"1721", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/clusters/cluster3", "uid":"b0da8c3d-b10e-4fc0-b95e-e145f3a95836"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.3.0.0/16"}, "cluster_id":"cluster3", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.3.0.0/16"}}}}
  4069. 2020-11-02T12:06:50.1044907Z [submariner]$ [cluster1] kubectl get pods --selector=app=submariner-routeagent -n submariner-operator -o jsonpath={.items[*].metadata.name}
  4070. 2020-11-02T12:06:50.1046190Z [submariner]$ [cluster1] kubectl get pods --selector=app=submariner-routeagent -n submariner-operator -o jsonpath={.items[*].metadata.name}
  4071. 2020-11-02T12:06:50.1048765Z [submariner]$ [cluster1] command kubectl --context=cluster1 get pods --selector=app=submariner-routeagent -n submariner-operator -o jsonpath={.items[*].metadata.name}
  4072. 2020-11-02T12:06:50.1050810Z [submariner]$ [cluster1] kubectl --context=cluster1 get pods --selector=app=submariner-routeagent -n submariner-operator -o jsonpath={.items[*].metadata.name}
  4073. 2020-11-02T12:06:50.1056193Z [cluster1] I1102 11:58:27.249808 1 federator.go:42] In Distribute for &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3", "namespace":"submariner-k8s-broker", "resourceVersion":"1721", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/clusters/cluster3", "uid":"b0da8c3d-b10e-4fc0-b95e-e145f3a95836"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.3.0.0/16"}, "cluster_id":"cluster3", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.3.0.0/16"}}}}
  4074. 2020-11-02T12:06:50.1059928Z [cluster1] I1102 11:58:27.249890 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  4075. 2020-11-02T12:06:50.1063452Z [cluster1] I1102 11:58:27.252743 1 create_or_update.go:32] Creating resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3", "namespace":"submariner-operator"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.3.0.0/16"}, "cluster_id":"cluster3", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.3.0.0/16"}}}}
  4076. 2020-11-02T12:06:50.1069770Z [cluster1] I1102 11:58:27.254783 1 resource_syncer.go:264] Syncer "local -> broker for *v1.Cluster" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2", "namespace":"submariner-operator", "resourceVersion":"1737", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/clusters/cluster2", "uid":"47c8f744-0a61-4726-857c-b438854de541"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.2.0.0/16"}, "cluster_id":"cluster2", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.2.0.0/16"}}}}
  4077. 2020-11-02T12:06:50.1073397Z [cluster1] I1102 11:58:27.255035 1 resource_syncer.go:437] Found cluster ID label "cluster2" - not syncing resource "cluster2"
  4078. 2020-11-02T12:06:50.1123441Z [cluster1] I1102 11:58:27.257369 1 resource_syncer.go:285] Syncer "broker -> local for *v1.Cluster" successfully synced "submariner-k8s-broker/cluster3"
  4079. 2020-11-02T12:06:50.1128554Z [cluster1] I1102 11:58:27.261637 1 resource_syncer.go:264] Syncer "local -> broker for *v1.Cluster" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3", "namespace":"submariner-operator", "resourceVersion":"1738", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/clusters/cluster3", "uid":"9a1ea5ca-46ba-4784-84ab-511e6ed7ec59"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.3.0.0/16"}, "cluster_id":"cluster3", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.3.0.0/16"}}}}
  4080. 2020-11-02T12:06:50.1131406Z [cluster1] I1102 11:58:27.261704 1 resource_syncer.go:437] Found cluster ID label "cluster3" - not syncing resource "cluster3"
  4081. 2020-11-02T12:06:50.1132565Z [cluster1] I1102 11:58:27.329297 1 resource_syncer.go:202] Syncer "local -> broker for *v1.Endpoint" started
  4082. 2020-11-02T12:06:50.1133383Z [cluster1] I1102 11:58:27.329583 1 resource_syncer.go:180] Starting syncer "broker -> local for *v1.Endpoint"
  4083. 2020-11-02T12:06:50.1134311Z [cluster1] I1102 11:58:27.329646 1 resource_syncer.go:202] Syncer "broker -> local for *v1.Endpoint" started
  4084. 2020-11-02T12:06:50.1135126Z [cluster1] I1102 11:58:27.329670 1 datastoresyncer.go:108] Ensuring we are the only endpoint active for this cluster
  4085. 2020-11-02T12:06:50.1136678Z [cluster1] I1102 11:58:27.329693 1 datastoresyncer.go:143] Creating local submariner Cluster: types.SubmarinerCluster{ID:"cluster1", Spec:v1.ClusterSpec{ClusterID:"cluster1", ColorCodes:[]string{"blue"}, ServiceCIDR:[]string{"100.1.0.0/16"}, ClusterCIDR:[]string{"10.1.0.0/16"}, GlobalCIDR:[]string{}}}
  4086. 2020-11-02T12:06:50.1140278Z [cluster1] I1102 11:58:27.329864 1 federator.go:42] In Distribute for &v1.Cluster{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster1", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.ClusterSpec{ClusterID:"cluster1", ColorCodes:[]string{"blue"}, ServiceCIDR:[]string{"100.1.0.0/16"}, ClusterCIDR:[]string{"10.1.0.0/16"}, GlobalCIDR:[]string{}}}
  4087. 2020-11-02T12:06:50.1143752Z [cluster1] I1102 11:58:27.330087 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  4088. 2020-11-02T12:06:50.1146318Z [cluster1] I1102 11:58:27.333555 1 create_or_update.go:32] Creating resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"name":"cluster1", "namespace":"submariner-operator"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.1.0.0/16"}, "cluster_id":"cluster1", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.1.0.0/16"}}}}
  4089. 2020-11-02T12:06:50.1149640Z [cluster1] I1102 11:58:27.337025 1 datastoresyncer.go:156] Creating local submariner Endpoint: types.SubmarinerEndpoint{Spec:v1.EndpointSpec{ClusterID:"cluster1", CableName:"submariner-cable-cluster1-172-17-0-6", Hostname:"cluster1-worker", Subnets:[]string{"100.1.0.0/16", "10.1.0.0/16"}, PrivateIP:"172.17.0.6", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string{}}}
  4090. 2020-11-02T12:06:50.1155219Z [cluster1] I1102 11:58:27.337061 1 federator.go:42] In Distribute for &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster1-submariner-cable-cluster1-172-17-0-6", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster1", CableName:"submariner-cable-cluster1-172-17-0-6", Hostname:"cluster1-worker", Subnets:[]string{"100.1.0.0/16", "10.1.0.0/16"}, PrivateIP:"172.17.0.6", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string{}}}
  4091. 2020-11-02T12:06:50.1158848Z [cluster1] I1102 11:58:27.337156 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  4092. 2020-11-02T12:06:50.1163495Z [cluster1] I1102 11:58:27.337327 1 resource_syncer.go:264] Syncer "broker -> local for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:19Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-k8s-broker", "resourceVersion":"1681", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"2d4d0eb4-8d26-4d4c-92fe-e2b47a023c8a"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  4093. 2020-11-02T12:06:50.1223850Z [cluster1] I1102 11:58:27.337403 1 resource_syncer.go:276] Syncing resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:19Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-k8s-broker", "resourceVersion":"1681", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"2d4d0eb4-8d26-4d4c-92fe-e2b47a023c8a"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  4094. 2020-11-02T12:06:50.1232060Z [cluster1] I1102 11:58:27.337445 1 federator.go:42] In Distribute for &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:19Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-k8s-broker", "resourceVersion":"1681", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"2d4d0eb4-8d26-4d4c-92fe-e2b47a023c8a"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  4095. 2020-11-02T12:06:50.1236071Z [cluster1] I1102 11:58:27.337501 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  4096. 2020-11-02T12:06:50.1239281Z [cluster1] I1102 11:58:27.339586 1 create_or_update.go:32] Creating resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"name":"cluster1-submariner-cable-cluster1-172-17-0-6", "namespace":"submariner-operator"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster1-172-17-0-6", "cluster_id":"cluster1", "hostname":"cluster1-worker", "nat_enabled":false, "private_ip":"172.17.0.6", "public_ip":"", "subnets":[]interface {}{"100.1.0.0/16", "10.1.0.0/16"}}}}
  4097. 2020-11-02T12:06:50.1244916Z [cluster1] I1102 11:58:27.339776 1 create_or_update.go:32] Creating resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-operator"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  4098. 2020-11-02T12:06:50.1249621Z [cluster1] I1102 11:58:27.341304 1 resource_syncer.go:264] Syncer "local -> broker for *v1.Cluster" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "name":"cluster1", "namespace":"submariner-operator", "resourceVersion":"1739", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/clusters/cluster1", "uid":"73e178bc-a242-4043-bf1c-750d0af46c6d"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.1.0.0/16"}, "cluster_id":"cluster1", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.1.0.0/16"}}}}
  4099. 2020-11-02T12:06:50.1254440Z [cluster1] I1102 11:58:27.341489 1 resource_syncer.go:276] Syncing resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "name":"cluster1", "namespace":"submariner-operator", "resourceVersion":"1739", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/clusters/cluster1", "uid":"73e178bc-a242-4043-bf1c-750d0af46c6d"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.1.0.0/16"}, "cluster_id":"cluster1", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.1.0.0/16"}}}}
  4100. 2020-11-02T12:06:50.1259278Z [cluster1] I1102 11:58:27.341636 1 federator.go:42] In Distribute for &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "name":"cluster1", "namespace":"submariner-operator", "resourceVersion":"1739", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/clusters/cluster1", "uid":"73e178bc-a242-4043-bf1c-750d0af46c6d"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.1.0.0/16"}, "cluster_id":"cluster1", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.1.0.0/16"}}}}
  4101. 2020-11-02T12:06:50.1261716Z [cluster1] I1102 11:58:27.341710 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  4102. 2020-11-02T12:06:50.1264583Z [cluster1] I1102 11:58:27.344932 1 create_or_update.go:32] Creating resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1", "namespace":"submariner-k8s-broker"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.1.0.0/16"}, "cluster_id":"cluster1", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.1.0.0/16"}}}}
  4103. 2020-11-02T12:06:50.1267345Z [cluster1] I1102 11:58:27.346553 1 resource_syncer.go:285] Syncer "broker -> local for *v1.Endpoint" successfully synced "submariner-k8s-broker/cluster2-submariner-cable-cluster2-172-17-0-4"
  4104. 2020-11-02T12:06:50.1273156Z [cluster1] I1102 11:58:27.346758 1 resource_syncer.go:264] Syncer "broker -> local for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3-submariner-cable-cluster3-172-17-0-7", "namespace":"submariner-k8s-broker", "resourceVersion":"1722", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster3-submariner-cable-cluster3-172-17-0-7", "uid":"f4e3ecdd-5302-4901-bfde-9d6483becbcb"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster3-172-17-0-7", "cluster_id":"cluster3", "hostname":"cluster3-worker", "nat_enabled":false, "private_ip":"172.17.0.7", "public_ip":"", "subnets":[]interface {}{"100.3.0.0/16", "10.3.0.0/16"}}}}
  4105. 2020-11-02T12:06:50.1280964Z [cluster1] I1102 11:58:27.346817 1 resource_syncer.go:276] Syncing resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3-submariner-cable-cluster3-172-17-0-7", "namespace":"submariner-k8s-broker", "resourceVersion":"1722", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster3-submariner-cable-cluster3-172-17-0-7", "uid":"f4e3ecdd-5302-4901-bfde-9d6483becbcb"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster3-172-17-0-7", "cluster_id":"cluster3", "hostname":"cluster3-worker", "nat_enabled":false, "private_ip":"172.17.0.7", "public_ip":"", "subnets":[]interface {}{"100.3.0.0/16", "10.3.0.0/16"}}}}
  4106. 2020-11-02T12:06:50.1289588Z [cluster1] I1102 11:58:27.346857 1 federator.go:42] In Distribute for &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3-submariner-cable-cluster3-172-17-0-7", "namespace":"submariner-k8s-broker", "resourceVersion":"1722", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster3-submariner-cable-cluster3-172-17-0-7", "uid":"f4e3ecdd-5302-4901-bfde-9d6483becbcb"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster3-172-17-0-7", "cluster_id":"cluster3", "hostname":"cluster3-worker", "nat_enabled":false, "private_ip":"172.17.0.7", "public_ip":"", "subnets":[]interface {}{"100.3.0.0/16", "10.3.0.0/16"}}}}
  4107. 2020-11-02T12:06:50.1294748Z [cluster1] I1102 11:58:27.346943 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  4108. 2020-11-02T12:06:50.1299566Z [cluster1] I1102 11:58:27.347928 1 resource_syncer.go:264] Syncer "local -> broker for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-operator", "resourceVersion":"1740", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"78218d29-51fa-4e18-9649-07670fababa0"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  4109. 2020-11-02T12:06:50.1304524Z [cluster1] I1102 11:58:27.347994 1 resource_syncer.go:437] Found cluster ID label "cluster2" - not syncing resource "cluster2-submariner-cable-cluster2-172-17-0-4"
  4110. 2020-11-02T12:06:50.1309591Z [cluster1] I1102 11:58:27.348025 1 resource_syncer.go:264] Syncer "Tunnel controller" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-operator", "resourceVersion":"1740", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"78218d29-51fa-4e18-9649-07670fababa0"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  4111. 2020-11-02T12:06:50.1318786Z [cluster1] I1102 11:58:27.348535 1 tunnel.go:56] Tunnel controller processing added or updated submariner Endpoint object: &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster2-submariner-cable-cluster2-172-17-0-4", GenerateName:"", Namespace:"submariner-operator", SelfLink:"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", UID:"78218d29-51fa-4e18-9649-07670fababa0", ResourceVersion:"1740", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63739915107, loc:(*time.Location)(0x2020aa0)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"submariner-io/clusterID":"cluster2"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster2", CableName:"submariner-cable-cluster2-172-17-0-4", Hostname:"cluster2-worker", Subnets:[]string{"100.2.0.0/16", "10.2.0.0/16"}, PrivateIP:"172.17.0.4", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string(nil)}}
  4112. 2020-11-02T12:06:50.1324189Z [cluster1] I1102 11:58:27.348586 1 cableengine.go:101] Installing Endpoint cable "submariner-cable-cluster2-172-17-0-4"
  4113. 2020-11-02T12:06:50.1325160Z [cluster1] I1102 11:58:27.348592 1 libreswan.go:164] Active connections: []
  4114. 2020-11-02T12:06:50.1325726Z [cluster1] 002 listening for IKE messages
  4115. 2020-11-02T12:06:50.1326061Z [cluster1] 002 forgetting secrets
  4116. 2020-11-02T12:06:50.1326440Z [cluster1] 002 loading secrets from "/etc/ipsec.secrets"
  4117. 2020-11-02T12:06:50.1326928Z [cluster1] 002 loading secrets from "/etc/ipsec.d/submariner.secrets"
  4118. 2020-11-02T12:06:50.1328155Z [cluster1] I1102 11:58:27.350853 1 libreswan.go:245] Creating connection(s) for {{cluster2 submariner-cable-cluster2-172-17-0-4 cluster2-worker [100.2.0.0/16 10.2.0.0/16] 172.17.0.4 false libreswan map[]}}
  4119. 2020-11-02T12:06:50.1329927Z [cluster1] I1102 11:58:27.350958 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-0-0 --id 172.17.0.6 --host 172.17.0.6 --client 172.17.0.6/32 --to --id 172.17.0.4 --host 172.17.0.4 --client 172.17.0.4/32]
  4120. 2020-11-02T12:06:50.1331238Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-4-0-0"
  4121. 2020-11-02T12:06:50.1332805Z [cluster1] I1102 11:58:27.360919 1 resource_syncer.go:285] Syncer "local -> broker for *v1.Cluster" successfully synced "submariner-operator/cluster1"
  4122. 2020-11-02T12:06:50.1336416Z [cluster1] I1102 11:58:27.362177 1 resource_syncer.go:264] Syncer "broker -> local for *v1.Cluster" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1", "namespace":"submariner-k8s-broker", "resourceVersion":"1741", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/clusters/cluster1", "uid":"025b70bc-43a5-4d06-b4ff-a733ee10d897"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.1.0.0/16"}, "cluster_id":"cluster1", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.1.0.0/16"}}}}
  4123. 2020-11-02T12:06:50.1339226Z [cluster1] I1102 11:58:27.362384 1 resource_syncer.go:443] Cluster ID label "cluster1" not present or matches local cluster ID "cluster1" - not syncing resource "cluster1"
  4124. 2020-11-02T12:06:50.1340287Z [cluster1] 181 "submariner-cable-cluster2-172-17-0-4-0-0" #1: initiating IKEv2 IKE SA
  4125. 2020-11-02T12:06:50.1342532Z [cluster1] I1102 11:58:27.442452 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-0-1 --id 172.17.0.6 --host 172.17.0.6 --client 172.17.0.6/32 --to --id 172.17.0.4 --host 172.17.0.4 --client 100.2.0.0/16]
  4126. 2020-11-02T12:06:50.1344020Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-4-0-1"
  4127. 2020-11-02T12:06:50.1346215Z [cluster1] 000 "submariner-cable-cluster2-172-17-0-4-0-1": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #1 "submariner-cable-cluster2-172-17-0-4-0-0"
  4128. 2020-11-02T12:06:50.1348177Z [cluster1] I1102 11:58:27.495687 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-0-2 --id 172.17.0.6 --host 172.17.0.6 --client 172.17.0.6/32 --to --id 172.17.0.4 --host 172.17.0.4 --client 10.2.0.0/16]
  4129. 2020-11-02T12:06:50.1349473Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-4-0-2"
  4130. 2020-11-02T12:06:50.1350248Z [cluster1] I1102 11:58:27.534382 1 datastoresyncer.go:99] Datastore syncer started
  4131. 2020-11-02T12:06:50.1354837Z [cluster1] I1102 11:58:27.539317 1 resource_syncer.go:264] Syncer "local -> broker for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "name":"cluster1-submariner-cable-cluster1-172-17-0-6", "namespace":"submariner-operator", "resourceVersion":"1742", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster1-submariner-cable-cluster1-172-17-0-6", "uid":"3e804d9f-ca6d-45f7-9088-b1a9a8f4f4d3"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster1-172-17-0-6", "cluster_id":"cluster1", "hostname":"cluster1-worker", "nat_enabled":false, "private_ip":"172.17.0.6", "public_ip":"", "subnets":[]interface {}{"100.1.0.0/16", "10.1.0.0/16"}}}}
  4132. 2020-11-02T12:06:50.1363811Z [cluster1] I1102 11:58:27.539564 1 resource_syncer.go:276] Syncing resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "name":"cluster1-submariner-cable-cluster1-172-17-0-6", "namespace":"submariner-operator", "resourceVersion":"1742", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster1-submariner-cable-cluster1-172-17-0-6", "uid":"3e804d9f-ca6d-45f7-9088-b1a9a8f4f4d3"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster1-172-17-0-6", "cluster_id":"cluster1", "hostname":"cluster1-worker", "nat_enabled":false, "private_ip":"172.17.0.6", "public_ip":"", "subnets":[]interface {}{"100.1.0.0/16", "10.1.0.0/16"}}}}
  4133. 2020-11-02T12:06:50.1372571Z [cluster1] I1102 11:58:27.540272 1 federator.go:42] In Distribute for &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "name":"cluster1-submariner-cable-cluster1-172-17-0-6", "namespace":"submariner-operator", "resourceVersion":"1742", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster1-submariner-cable-cluster1-172-17-0-6", "uid":"3e804d9f-ca6d-45f7-9088-b1a9a8f4f4d3"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster1-172-17-0-6", "cluster_id":"cluster1", "hostname":"cluster1-worker", "nat_enabled":false, "private_ip":"172.17.0.6", "public_ip":"", "subnets":[]interface {}{"100.1.0.0/16", "10.1.0.0/16"}}}}
  4134. 2020-11-02T12:06:50.1376307Z [cluster1] I1102 11:58:27.541529 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  4135. 2020-11-02T12:06:50.1379868Z [cluster1] I1102 11:58:27.546917 1 create_or_update.go:32] Creating resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1-submariner-cable-cluster1-172-17-0-6", "namespace":"submariner-k8s-broker"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster1-172-17-0-6", "cluster_id":"cluster1", "hostname":"cluster1-worker", "nat_enabled":false, "private_ip":"172.17.0.6", "public_ip":"", "subnets":[]interface {}{"100.1.0.0/16", "10.1.0.0/16"}}}}
  4136. 2020-11-02T12:06:50.1383168Z [cluster1] I1102 11:58:27.554951 1 resource_syncer.go:285] Syncer "local -> broker for *v1.Endpoint" successfully synced "submariner-operator/cluster1-submariner-cable-cluster1-172-17-0-6"
  4137. 2020-11-02T12:06:50.1385327Z [cluster1] 000 "submariner-cable-cluster2-172-17-0-4-0-2": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #1 "submariner-cable-cluster2-172-17-0-4-0-0"
  4138. 2020-11-02T12:06:50.1387557Z [cluster1] I1102 11:58:27.600578 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-1-0 --id 172.17.0.6 --host 172.17.0.6 --client 100.1.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 172.17.0.4/32]
  4139. 2020-11-02T12:06:50.1388871Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-4-1-0"
  4140. 2020-11-02T12:06:50.1391335Z [cluster1] 000 "submariner-cable-cluster2-172-17-0-4-1-0": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #1 "submariner-cable-cluster2-172-17-0-4-0-0"
  4141. 2020-11-02T12:06:50.1393149Z [cluster1] I1102 11:58:27.612983 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-1-1 --id 172.17.0.6 --host 172.17.0.6 --client 100.1.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 100.2.0.0/16]
  4142. 2020-11-02T12:06:50.1394484Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-4-1-1"
  4143. 2020-11-02T12:06:50.1396040Z [cluster1] 000 "submariner-cable-cluster2-172-17-0-4-1-1": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #1 "submariner-cable-cluster2-172-17-0-4-0-0"
  4144. 2020-11-02T12:06:50.1398476Z [cluster1] I1102 11:58:27.649286 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-1-2 --id 172.17.0.6 --host 172.17.0.6 --client 100.1.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 10.2.0.0/16]
  4145. 2020-11-02T12:06:50.1400378Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-4-1-2"
  4146. 2020-11-02T12:06:50.1402471Z [cluster1] 000 "submariner-cable-cluster2-172-17-0-4-1-2": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #1 "submariner-cable-cluster2-172-17-0-4-0-0"
  4147. 2020-11-02T12:06:50.1404433Z [cluster1] I1102 11:58:27.659710 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-2-0 --id 172.17.0.6 --host 172.17.0.6 --client 10.1.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 172.17.0.4/32]
  4148. 2020-11-02T12:06:50.1405781Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-4-2-0"
  4149. 2020-11-02T12:06:50.1407333Z [cluster1] 000 "submariner-cable-cluster2-172-17-0-4-2-0": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #1 "submariner-cable-cluster2-172-17-0-4-0-0"
  4150. 2020-11-02T12:06:50.1409234Z [cluster1] I1102 11:58:27.672970 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-2-1 --id 172.17.0.6 --host 172.17.0.6 --client 10.1.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 100.2.0.0/16]
  4151. 2020-11-02T12:06:50.1410584Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-4-2-1"
  4152. 2020-11-02T12:06:50.1412127Z [cluster1] 000 "submariner-cable-cluster2-172-17-0-4-2-1": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #1 "submariner-cable-cluster2-172-17-0-4-0-0"
  4153. 2020-11-02T12:06:50.1413906Z [cluster1] I1102 11:58:27.683345 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-2-2 --id 172.17.0.6 --host 172.17.0.6 --client 10.1.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 10.2.0.0/16]
  4154. 2020-11-02T12:06:50.1415210Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-4-2-2"
  4155. 2020-11-02T12:06:50.1416747Z [cluster1] 000 "submariner-cable-cluster2-172-17-0-4-2-2": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #1 "submariner-cable-cluster2-172-17-0-4-0-0"
  4156. 2020-11-02T12:06:50.1418384Z [cluster1] I1102 11:58:27.697783 1 cableengine.go:150] Successfully installed Endpoint cable "submariner-cable-cluster2-172-17-0-4" with remote IP 172.17.0.4
  4157. 2020-11-02T12:06:50.1419784Z [cluster1] I1102 11:58:27.698371 1 tunnel.go:68] Tunnel controller successfully installed Endpoint cable submariner-cable-cluster2-172-17-0-4 in the engine
  4158. 2020-11-02T12:06:50.1420971Z [cluster1] I1102 11:58:27.699200 1 resource_syncer.go:352] Syncer "Tunnel controller": transform function returned nil - not syncing - requeue: false
  4159. 2020-11-02T12:06:50.1425070Z [cluster1] I1102 11:58:27.700689 1 resource_syncer.go:264] Syncer "Tunnel controller" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "name":"cluster1-submariner-cable-cluster1-172-17-0-6", "namespace":"submariner-operator", "resourceVersion":"1742", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster1-submariner-cable-cluster1-172-17-0-6", "uid":"3e804d9f-ca6d-45f7-9088-b1a9a8f4f4d3"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster1-172-17-0-6", "cluster_id":"cluster1", "hostname":"cluster1-worker", "nat_enabled":false, "private_ip":"172.17.0.6", "public_ip":"", "subnets":[]interface {}{"100.1.0.0/16", "10.1.0.0/16"}}}}
  4160. 2020-11-02T12:06:50.1432954Z [cluster1] I1102 11:58:27.700899 1 tunnel.go:56] Tunnel controller processing added or updated submariner Endpoint object: &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster1-submariner-cable-cluster1-172-17-0-6", GenerateName:"", Namespace:"submariner-operator", SelfLink:"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster1-submariner-cable-cluster1-172-17-0-6", UID:"3e804d9f-ca6d-45f7-9088-b1a9a8f4f4d3", ResourceVersion:"1742", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63739915107, loc:(*time.Location)(0x2020aa0)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster1", CableName:"submariner-cable-cluster1-172-17-0-6", Hostname:"cluster1-worker", Subnets:[]string{"100.1.0.0/16", "10.1.0.0/16"}, PrivateIP:"172.17.0.6", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string(nil)}}
  4161. 2020-11-02T12:06:50.1438001Z [cluster1] I1102 11:58:27.700975 1 cableengine.go:92] Not installing cable for local cluster
  4162. 2020-11-02T12:06:50.1439190Z [cluster1] I1102 11:58:27.700980 1 tunnel.go:68] Tunnel controller successfully installed Endpoint cable submariner-cable-cluster1-172-17-0-6 in the engine
  4163. 2020-11-02T12:06:50.1440495Z [cluster1] I1102 11:58:27.700985 1 resource_syncer.go:352] Syncer "Tunnel controller": transform function returned nil - not syncing - requeue: false
  4164. 2020-11-02T12:06:50.1443557Z [cluster1] I1102 11:58:27.731638 1 create_or_update.go:32] Creating resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3-submariner-cable-cluster3-172-17-0-7", "namespace":"submariner-operator"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster3-172-17-0-7", "cluster_id":"cluster3", "hostname":"cluster3-worker", "nat_enabled":false, "private_ip":"172.17.0.7", "public_ip":"", "subnets":[]interface {}{"100.3.0.0/16", "10.3.0.0/16"}}}}
  4165. 2020-11-02T12:06:50.1446904Z [cluster1] I1102 11:58:27.937419 1 resource_syncer.go:285] Syncer "broker -> local for *v1.Endpoint" successfully synced "submariner-k8s-broker/cluster3-submariner-cable-cluster3-172-17-0-7"
  4166. 2020-11-02T12:06:50.1451702Z [cluster1] I1102 11:58:27.937460 1 resource_syncer.go:264] Syncer "broker -> local for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1-submariner-cable-cluster1-172-17-0-6", "namespace":"submariner-k8s-broker", "resourceVersion":"1743", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster1-submariner-cable-cluster1-172-17-0-6", "uid":"b8932e09-5864-498e-b04f-0147ed950e52"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster1-172-17-0-6", "cluster_id":"cluster1", "hostname":"cluster1-worker", "nat_enabled":false, "private_ip":"172.17.0.6", "public_ip":"", "subnets":[]interface {}{"100.1.0.0/16", "10.1.0.0/16"}}}}
  4167. 2020-11-02T12:06:50.1456015Z [cluster1] I1102 11:58:27.937632 1 resource_syncer.go:443] Cluster ID label "cluster1" not present or matches local cluster ID "cluster1" - not syncing resource "cluster1-submariner-cable-cluster1-172-17-0-6"
  4168. 2020-11-02T12:06:50.1461865Z [cluster1] I1102 11:58:27.942434 1 resource_syncer.go:264] Syncer "local -> broker for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3-submariner-cable-cluster3-172-17-0-7", "namespace":"submariner-operator", "resourceVersion":"1744", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster3-submariner-cable-cluster3-172-17-0-7", "uid":"89e19f24-5be8-4c1a-acc0-06c182b6ed78"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster3-172-17-0-7", "cluster_id":"cluster3", "hostname":"cluster3-worker", "nat_enabled":false, "private_ip":"172.17.0.7", "public_ip":"", "subnets":[]interface {}{"100.3.0.0/16", "10.3.0.0/16"}}}}
  4169. 2020-11-02T12:06:50.1466399Z [cluster1] I1102 11:58:27.943445 1 resource_syncer.go:437] Found cluster ID label "cluster3" - not syncing resource "cluster3-submariner-cable-cluster3-172-17-0-7"
  4170. 2020-11-02T12:06:50.1471290Z [cluster1] I1102 11:58:27.944729 1 resource_syncer.go:264] Syncer "Tunnel controller" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3-submariner-cable-cluster3-172-17-0-7", "namespace":"submariner-operator", "resourceVersion":"1744", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster3-submariner-cable-cluster3-172-17-0-7", "uid":"89e19f24-5be8-4c1a-acc0-06c182b6ed78"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster3-172-17-0-7", "cluster_id":"cluster3", "hostname":"cluster3-worker", "nat_enabled":false, "private_ip":"172.17.0.7", "public_ip":"", "subnets":[]interface {}{"100.3.0.0/16", "10.3.0.0/16"}}}}
  4171. 2020-11-02T12:06:50.1480213Z [cluster1] I1102 11:58:27.945526 1 tunnel.go:56] Tunnel controller processing added or updated submariner Endpoint object: &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster3-submariner-cable-cluster3-172-17-0-7", GenerateName:"", Namespace:"submariner-operator", SelfLink:"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster3-submariner-cable-cluster3-172-17-0-7", UID:"89e19f24-5be8-4c1a-acc0-06c182b6ed78", ResourceVersion:"1744", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63739915107, loc:(*time.Location)(0x2020aa0)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"submariner-io/clusterID":"cluster3"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster3", CableName:"submariner-cable-cluster3-172-17-0-7", Hostname:"cluster3-worker", Subnets:[]string{"100.3.0.0/16", "10.3.0.0/16"}, PrivateIP:"172.17.0.7", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string(nil)}}
  4172. 2020-11-02T12:06:50.1485818Z [cluster1] I1102 11:58:27.946725 1 cableengine.go:101] Installing Endpoint cable "submariner-cable-cluster3-172-17-0-7"
  4173. 2020-11-02T12:06:50.1487016Z [cluster1] I1102 11:58:27.948612 1 libreswan.go:164] Active connections: [submariner-cable-cluster2-172-17-0-4]
  4174. 2020-11-02T12:06:50.1487880Z [cluster1] 002 listening for IKE messages
  4175. 2020-11-02T12:06:50.1488622Z [cluster1] 002 adding interface vx-submariner/vx-submariner (esp-hw-offload not supported by kernel) 240.17.0.6:500
  4176. 2020-11-02T12:06:50.1489412Z [cluster1] 002 adding interface vx-submariner/vx-submariner 240.17.0.6:4500
  4177. 2020-11-02T12:06:50.1489843Z [cluster1] 002 forgetting secrets
  4178. 2020-11-02T12:06:50.1490236Z [cluster1] 002 loading secrets from "/etc/ipsec.secrets"
  4179. 2020-11-02T12:06:50.1490725Z [cluster1] 002 loading secrets from "/etc/ipsec.d/submariner.secrets"
  4180. 2020-11-02T12:06:50.1491946Z [cluster1] I1102 11:58:27.958917 1 libreswan.go:245] Creating connection(s) for {{cluster3 submariner-cable-cluster3-172-17-0-7 cluster3-worker [100.3.0.0/16 10.3.0.0/16] 172.17.0.7 false libreswan map[]}}
  4181. 2020-11-02T12:06:50.1493944Z [cluster1] I1102 11:58:27.974602 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster3-172-17-0-7-0-0 --id 172.17.0.6 --host 172.17.0.6 --client 172.17.0.6/32 --to --id 172.17.0.7 --host 172.17.0.7 --client 172.17.0.7/32]
  4182. 2020-11-02T12:06:50.1495284Z [cluster1] 002 added connection description "submariner-cable-cluster3-172-17-0-7-0-0"
  4183. 2020-11-02T12:06:50.1496405Z [cluster1] 181 "submariner-cable-cluster3-172-17-0-7-0-0" #6: initiating IKEv2 IKE SA
  4184. 2020-11-02T12:06:50.1497833Z [cluster1] I1102 11:58:28.126141 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster3-172-17-0-7-0-1 --id 172.17.0.6 --host 172.17.0.6 --client 172.17.0.6/32 --to --id 172.17.0.7 --host 172.17.0.7 --client 100.3.0.0/16]
  4185. 2020-11-02T12:06:50.1499131Z [cluster1] 002 added connection description "submariner-cable-cluster3-172-17-0-7-0-1"
  4186. 2020-11-02T12:06:50.1500800Z [cluster1] 000 "submariner-cable-cluster3-172-17-0-7-0-1": queuing pending IPsec SA negotiating with 172.17.0.7 IKE SA #6 "submariner-cable-cluster3-172-17-0-7-0-0"
  4187. 2020-11-02T12:06:50.1502712Z [cluster1] I1102 11:58:28.230459 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster3-172-17-0-7-0-2 --id 172.17.0.6 --host 172.17.0.6 --client 172.17.0.6/32 --to --id 172.17.0.7 --host 172.17.0.7 --client 10.3.0.0/16]
  4188. 2020-11-02T12:06:50.1504247Z [cluster1] 002 added connection description "submariner-cable-cluster3-172-17-0-7-0-2"
  4189. 2020-11-02T12:06:50.1506319Z [cluster1] I1102 11:58:28.291536 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster3-172-17-0-7-1-0 --id 172.17.0.6 --host 172.17.0.6 --client 100.1.0.0/16 --to --id 172.17.0.7 --host 172.17.0.7 --client 172.17.0.7/32]
  4190. 2020-11-02T12:06:50.1507952Z [cluster1] 002 added connection description "submariner-cable-cluster3-172-17-0-7-1-0"
  4191. 2020-11-02T12:06:50.1509971Z [cluster1] I1102 11:58:28.316051 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster3-172-17-0-7-1-1 --id 172.17.0.6 --host 172.17.0.6 --client 100.1.0.0/16 --to --id 172.17.0.7 --host 172.17.0.7 --client 100.3.0.0/16]
  4192. 2020-11-02T12:06:50.1512115Z [cluster1] 002 added connection description "submariner-cable-cluster3-172-17-0-7-1-1"
  4193. 2020-11-02T12:06:50.1513603Z [cluster1] I1102 11:58:28.350506 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster3-172-17-0-7-1-2 --id 172.17.0.6 --host 172.17.0.6 --client 100.1.0.0/16 --to --id 172.17.0.7 --host 172.17.0.7 --client 10.3.0.0/16]
  4194. 2020-11-02T12:06:50.1514904Z [cluster1] 002 added connection description "submariner-cable-cluster3-172-17-0-7-1-2"
  4195. 2020-11-02T12:06:50.1518124Z [cluster1] I1102 11:58:28.408717 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster3-172-17-0-7-2-0 --id 172.17.0.6 --host 172.17.0.6 --client 10.1.0.0/16 --to --id 172.17.0.7 --host 172.17.0.7 --client 172.17.0.7/32]
  4196. 2020-11-02T12:06:50.1520211Z [cluster1] 002 added connection description "submariner-cable-cluster3-172-17-0-7-2-0"
  4197. 2020-11-02T12:06:50.1521919Z [cluster1] I1102 11:58:28.506402 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster3-172-17-0-7-2-1 --id 172.17.0.6 --host 172.17.0.6 --client 10.1.0.0/16 --to --id 172.17.0.7 --host 172.17.0.7 --client 100.3.0.0/16]
  4198. 2020-11-02T12:06:50.1523264Z [cluster1] 002 added connection description "submariner-cable-cluster3-172-17-0-7-2-1"
  4199. 2020-11-02T12:06:50.1525554Z [cluster1] I1102 11:58:28.549201 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster3-172-17-0-7-2-2 --id 172.17.0.6 --host 172.17.0.6 --client 10.1.0.0/16 --to --id 172.17.0.7 --host 172.17.0.7 --client 10.3.0.0/16]
  4200. 2020-11-02T12:06:50.1527275Z [cluster1] 002 added connection description "submariner-cable-cluster3-172-17-0-7-2-2"
  4201. 2020-11-02T12:06:50.1528581Z [cluster1] I1102 11:58:28.564585 1 cableengine.go:150] Successfully installed Endpoint cable "submariner-cable-cluster3-172-17-0-7" with remote IP 172.17.0.7
  4202. 2020-11-02T12:06:50.1529990Z [cluster1] I1102 11:58:28.565950 1 tunnel.go:68] Tunnel controller successfully installed Endpoint cable submariner-cable-cluster3-172-17-0-7 in the engine
  4203. 2020-11-02T12:06:50.1531189Z [cluster1] I1102 11:58:28.566075 1 resource_syncer.go:352] Syncer "Tunnel controller": transform function returned nil - not syncing - requeue: false
  4204. 2020-11-02T12:06:50.1532673Z [cluster1] I1102 12:01:40.295503 1 resource_syncer.go:294] Syncer "broker -> local for *v1.Endpoint" informed of deleted resource "submariner-k8s-broker/cluster2-submariner-cable-cluster2-172-17-0-4"
  4205. 2020-11-02T12:06:50.1538155Z [cluster1] I1102 12:01:40.295527 1 resource_syncer.go:315] Syncer "broker -> local for *v1.Endpoint" deleting resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:19Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-k8s-broker", "resourceVersion":"2627", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"2d4d0eb4-8d26-4d4c-92fe-e2b47a023c8a"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  4206. 2020-11-02T12:06:50.1542532Z [cluster1] I1102 12:01:40.295757 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  4207. 2020-11-02T12:06:50.1547136Z [cluster1] I1102 12:01:40.295768 1 federator.go:73] Deleting resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:19Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-operator", "resourceVersion":"2627", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"2d4d0eb4-8d26-4d4c-92fe-e2b47a023c8a"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  4208. 2020-11-02T12:06:50.1551264Z [cluster1] I1102 12:01:40.304942 1 resource_syncer.go:330] Syncer "broker -> local for *v1.Endpoint" successfully deleted "submariner-k8s-broker/cluster2-submariner-cable-cluster2-172-17-0-4"
  4209. 2020-11-02T12:06:50.1553130Z [cluster1] I1102 12:01:40.307987 1 resource_syncer.go:294] Syncer "Tunnel controller" informed of deleted resource "submariner-operator/cluster2-submariner-cable-cluster2-172-17-0-4"
  4210. 2020-11-02T12:06:50.1555398Z [cluster1] I1102 12:01:40.308050 1 resource_syncer.go:294] Syncer "local -> broker for *v1.Endpoint" informed of deleted resource "submariner-operator/cluster2-submariner-cable-cluster2-172-17-0-4"
  4211. 2020-11-02T12:06:50.1557239Z [cluster1] I1102 12:01:40.308066 1 resource_syncer.go:437] Found cluster ID label "cluster2" - not syncing resource "cluster2-submariner-cable-cluster2-172-17-0-4"
  4212. 2020-11-02T12:06:50.1563268Z [cluster1] I1102 12:01:40.308315 1 tunnel.go:74] Tunnel controller processing removed submariner Endpoint object: &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster2-submariner-cable-cluster2-172-17-0-4", GenerateName:"", Namespace:"submariner-operator", SelfLink:"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", UID:"78218d29-51fa-4e18-9649-07670fababa0", ResourceVersion:"2628", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63739915107, loc:(*time.Location)(0x2020aa0)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"submariner-io/clusterID":"cluster2"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster2", CableName:"submariner-cable-cluster2-172-17-0-4", Hostname:"cluster2-worker", Subnets:[]string{"100.2.0.0/16", "10.2.0.0/16"}, PrivateIP:"172.17.0.4", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string(nil)}}
  4213. 2020-11-02T12:06:50.1569797Z [cluster1] I1102 12:01:40.308404 1 cableengine.go:166] Removing Endpoint cable "submariner-cable-cluster2-172-17-0-4"
  4214. 2020-11-02T12:06:50.1572234Z [cluster1] I1102 12:01:40.308594 1 libreswan.go:300] Deleting connection to {{cluster2 submariner-cable-cluster2-172-17-0-4 cluster2-worker [100.2.0.0/16 10.2.0.0/16] 172.17.0.4 false libreswan map[]}}
  4215. 2020-11-02T12:06:50.1573909Z [cluster1] I1102 12:01:40.308609 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-0-0]
  4216. 2020-11-02T12:06:50.1575118Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-0-0": terminating SAs using this connection
  4217. 2020-11-02T12:06:50.1576602Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-0-0" #27: deleting state (STATE_V2_IPSEC_R) aged 6.109s and sending notification
  4218. 2020-11-02T12:06:50.1577861Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-0-0" #27: ESP traffic information: in=0B out=0B
  4219. 2020-11-02T12:06:50.1583083Z [cluster1] I1102 12:01:40.343905 1 resource_syncer.go:264] Syncer "broker -> local for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:01:40Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-k8s-broker", "resourceVersion":"2629", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"bb7e9be5-2d33-4f1d-947e-4a46c6904408"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  4220. 2020-11-02T12:06:50.1590922Z [cluster1] I1102 12:01:40.343965 1 resource_syncer.go:276] Syncing resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:01:40Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-k8s-broker", "resourceVersion":"2629", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"bb7e9be5-2d33-4f1d-947e-4a46c6904408"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  4221. 2020-11-02T12:06:50.1598508Z [cluster1] I1102 12:01:40.344001 1 federator.go:42] In Distribute for &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:01:40Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-k8s-broker", "resourceVersion":"2629", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"bb7e9be5-2d33-4f1d-947e-4a46c6904408"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  4222. 2020-11-02T12:06:50.1602294Z [cluster1] I1102 12:01:40.344059 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  4223. 2020-11-02T12:06:50.1609477Z [cluster1] I1102 12:01:40.346429 1 create_or_update.go:32] Creating resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-operator"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  4224. 2020-11-02T12:06:50.1612854Z [cluster1] I1102 12:01:40.359241 1 resource_syncer.go:285] Syncer "broker -> local for *v1.Endpoint" successfully synced "submariner-k8s-broker/cluster2-submariner-cable-cluster2-172-17-0-4"
  4225. 2020-11-02T12:06:50.1618078Z [cluster1] I1102 12:01:40.365726 1 resource_syncer.go:264] Syncer "local -> broker for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:01:40Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-operator", "resourceVersion":"2630", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"c09b80a0-c12c-4246-a9ea-692c9619737f"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  4226. 2020-11-02T12:06:50.1622434Z [cluster1] I1102 12:01:40.365853 1 resource_syncer.go:437] Found cluster ID label "cluster2" - not syncing resource "cluster2-submariner-cable-cluster2-172-17-0-4"
  4227. 2020-11-02T12:06:50.1623953Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-0-0" #7: deleting state (STATE_V2_IPSEC_I) aged 192.246s and sending notification
  4228. 2020-11-02T12:06:50.1626105Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-0-0" #7: ESP traffic information: in=0B out=0B
  4229. 2020-11-02T12:06:50.1627467Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-0-0" #3: deleting state (STATE_V2_IPSEC_R) aged 192.527s and sending notification
  4230. 2020-11-02T12:06:50.1628918Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-0-0" #3: ESP traffic information: in=0B out=0B
  4231. 2020-11-02T12:06:50.1630175Z [cluster1] I1102 12:01:40.372491 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-0-1]
  4232. 2020-11-02T12:06:50.1631412Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-0-1": terminating SAs using this connection
  4233. 2020-11-02T12:06:50.1632893Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-0-1" #30: deleting state (STATE_V2_IPSEC_R) aged 6.070s and sending notification
  4234. 2020-11-02T12:06:50.1634354Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-0-1" #30: ESP traffic information: in=0B out=0B
  4235. 2020-11-02T12:06:50.1635883Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-0-1" #8: deleting state (STATE_V2_IPSEC_R) aged 192.208s and sending notification
  4236. 2020-11-02T12:06:50.1638738Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-0-1" #8: ESP traffic information: in=0B out=0B
  4237. 2020-11-02T12:06:50.1640010Z [cluster1] I1102 12:01:40.411267 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-0-2]
  4238. 2020-11-02T12:06:50.1641243Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-0-2": terminating SAs using this connection
  4239. 2020-11-02T12:06:50.1642903Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-0-2" #33: deleting state (STATE_V2_IPSEC_R) aged 6.060s and sending notification
  4240. 2020-11-02T12:06:50.1644750Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-0-2" #33: ESP traffic information: in=0B out=0B
  4241. 2020-11-02T12:06:50.1648312Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-0-2" #17: deleting state (STATE_V2_IPSEC_R) aged 192.056s and sending notification
  4242. 2020-11-02T12:06:50.1649834Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-0-2" #17: ESP traffic information: in=0B out=0B
  4243. 2020-11-02T12:06:50.1651258Z [cluster1] I1102 12:01:40.434377 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-1-0]
  4244. 2020-11-02T12:06:50.1652748Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-1-0": terminating SAs using this connection
  4245. 2020-11-02T12:06:50.1654099Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-1-0" #28: deleting state (STATE_V2_IPSEC_R) aged 6.218s and sending notification
  4246. 2020-11-02T12:06:50.1655908Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-1-0" #28: ESP traffic information: in=0B out=0B
  4247. 2020-11-02T12:06:50.1657276Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-1-0" #4: deleting state (STATE_V2_IPSEC_R) aged 192.558s and sending notification
  4248. 2020-11-02T12:06:50.1659651Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-1-0" #4: ESP traffic information: in=0B out=0B
  4249. 2020-11-02T12:06:50.1661030Z [cluster1] I1102 12:01:40.482748 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-1-1]
  4250. 2020-11-02T12:06:50.1662278Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-1-1": terminating SAs using this connection
  4251. 2020-11-02T12:06:50.1663587Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-1-1" #31: deleting state (STATE_V2_IPSEC_R) aged 6.162s and sending notification
  4252. 2020-11-02T12:06:50.1664827Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-1-1" #31: ESP traffic information: in=0B out=0B
  4253. 2020-11-02T12:06:50.1666327Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-1-1" #11: deleting state (STATE_V2_IPSEC_R) aged 192.222s and sending notification
  4254. 2020-11-02T12:06:50.1667637Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-1-1" #11: ESP traffic information: in=0B out=0B
  4255. 2020-11-02T12:06:50.1669062Z [cluster1] I1102 12:01:40.513468 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-1-2]
  4256. 2020-11-02T12:06:50.1670298Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-1-2": terminating SAs using this connection
  4257. 2020-11-02T12:06:50.1671601Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-1-2" #34: deleting state (STATE_V2_IPSEC_R) aged 6.148s and sending notification
  4258. 2020-11-02T12:06:50.1674307Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-1-2" #34: ESP traffic information: in=0B out=0B
  4259. 2020-11-02T12:06:50.1676095Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-1-2" #20: deleting state (STATE_V2_IPSEC_R) aged 192.102s and sending notification
  4260. 2020-11-02T12:06:50.1677602Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-1-2" #20: ESP traffic information: in=0B out=0B
  4261. 2020-11-02T12:06:50.1679290Z [cluster1] I1102 12:01:40.532740 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-2-0]
  4262. 2020-11-02T12:06:50.1680742Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-2-0": terminating SAs using this connection
  4263. 2020-11-02T12:06:50.1696663Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-2-0" #29: deleting state (STATE_V2_IPSEC_R) aged 6.246s and sending notification
  4264. 2020-11-02T12:06:50.1698726Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-2-0" #29: ESP traffic information: in=0B out=0B
  4265. 2020-11-02T12:06:50.1700364Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-2-0" #5: deleting state (STATE_V2_IPSEC_R) aged 192.439s and sending notification
  4266. 2020-11-02T12:06:50.1701754Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-2-0" #5: ESP traffic information: in=0B out=0B
  4267. 2020-11-02T12:06:50.1703014Z [cluster1] I1102 12:01:40.633032 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-2-1]
  4268. 2020-11-02T12:06:50.1704256Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-2-1": terminating SAs using this connection
  4269. 2020-11-02T12:06:50.1705881Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-2-1" #32: deleting state (STATE_V2_IPSEC_R) aged 6.299s and sending notification
  4270. 2020-11-02T12:06:50.1707137Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-2-1" #32: ESP traffic information: in=0B out=0B
  4271. 2020-11-02T12:06:50.1708750Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-2-1" #14: deleting state (STATE_V2_IPSEC_R) aged 192.337s and sending notification
  4272. 2020-11-02T12:06:50.1710038Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-2-1" #14: ESP traffic information: in=0B out=0B
  4273. 2020-11-02T12:06:50.1711434Z [cluster1] I1102 12:01:40.728348 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-2-2]
  4274. 2020-11-02T12:06:50.1713116Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-2-2": terminating SAs using this connection
  4275. 2020-11-02T12:06:50.1714501Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-2-2" #35: deleting state (STATE_V2_IPSEC_R) aged 6.347s and sending notification
  4276. 2020-11-02T12:06:50.1716348Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-2-2" #35: ESP traffic information: in=0B out=0B
  4277. 2020-11-02T12:06:50.1717707Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-2-2" #25: deleting state (STATE_V2_REKEY_CHILD_I0) aged 25.613s and NOT sending notification
  4278. 2020-11-02T12:06:50.1719408Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-2-2" #26: deleting state (STATE_PARENT_R2) aged 6.595s and sending notification
  4279. 2020-11-02T12:06:50.1720728Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-2-2" #2: deleting state (STATE_PARENT_R2) aged 192.912s and sending notification
  4280. 2020-11-02T12:06:50.1722219Z [cluster1] I1102 12:01:40.803549 1 cableengine.go:176] Successfully removed Endpoint cable "submariner-cable-cluster2-172-17-0-4"
  4281. 2020-11-02T12:06:50.1723614Z [cluster1] I1102 12:01:40.803738 1 tunnel.go:85] Tunnel controller successfully removed Endpoint cable submariner-cable-cluster2-172-17-0-4 from the engine
  4282. 2020-11-02T12:06:50.1725192Z [cluster1] I1102 12:01:40.803834 1 resource_syncer.go:352] Syncer "Tunnel controller": transform function returned nil - not syncing - requeue: false
  4283. 2020-11-02T12:06:50.1729512Z [cluster1] I1102 12:01:40.803985 1 resource_syncer.go:264] Syncer "Tunnel controller" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:01:40Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-operator", "resourceVersion":"2630", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"c09b80a0-c12c-4246-a9ea-692c9619737f"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  4284. 2020-11-02T12:06:50.1739007Z [cluster1] I1102 12:01:40.804304 1 tunnel.go:56] Tunnel controller processing added or updated submariner Endpoint object: &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster2-submariner-cable-cluster2-172-17-0-4", GenerateName:"", Namespace:"submariner-operator", SelfLink:"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", UID:"c09b80a0-c12c-4246-a9ea-692c9619737f", ResourceVersion:"2630", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63739915300, loc:(*time.Location)(0x2020aa0)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"submariner-io/clusterID":"cluster2"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster2", CableName:"submariner-cable-cluster2-172-17-0-4", Hostname:"cluster2-worker", Subnets:[]string{"100.2.0.0/16", "10.2.0.0/16"}, PrivateIP:"172.17.0.4", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string(nil)}}
  4285. 2020-11-02T12:06:50.1745047Z [cluster1] I1102 12:01:40.804480 1 cableengine.go:101] Installing Endpoint cable "submariner-cable-cluster2-172-17-0-4"
  4286. 2020-11-02T12:06:50.1746274Z [cluster1] I1102 12:01:40.804572 1 libreswan.go:164] Active connections: [submariner-cable-cluster3-172-17-0-7]
  4287. 2020-11-02T12:06:50.1747865Z [cluster1] I1102 12:01:40.808967 1 libreswan.go:245] Creating connection(s) for {{cluster2 submariner-cable-cluster2-172-17-0-4 cluster2-worker [100.2.0.0/16 10.2.0.0/16] 172.17.0.4 false libreswan map[]}}
  4288. 2020-11-02T12:06:50.1749480Z [cluster1] I1102 12:01:40.808989 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-0-0 --id 172.17.0.6 --host 172.17.0.6 --client 172.17.0.6/32 --to --id 172.17.0.4 --host 172.17.0.4 --client 172.17.0.4/32]
  4289. 2020-11-02T12:06:50.1750310Z [cluster1] 002 listening for IKE messages
  4290. 2020-11-02T12:06:50.1750645Z [cluster1] 002 forgetting secrets
  4291. 2020-11-02T12:06:50.1751030Z [cluster1] 002 loading secrets from "/etc/ipsec.secrets"
  4292. 2020-11-02T12:06:50.1751704Z [cluster1] 002 loading secrets from "/etc/ipsec.d/submariner.secrets"
  4293. 2020-11-02T12:06:50.1753426Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-4-0-0"
  4294. 2020-11-02T12:06:50.1755586Z [cluster1] 181 "submariner-cable-cluster2-172-17-0-4-0-0" #36: initiating IKEv2 IKE SA
  4295. 2020-11-02T12:06:50.1757032Z [cluster1] I1102 12:01:40.888071 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-0-1 --id 172.17.0.6 --host 172.17.0.6 --client 172.17.0.6/32 --to --id 172.17.0.4 --host 172.17.0.4 --client 100.2.0.0/16]
  4296. 2020-11-02T12:06:50.1758347Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-4-0-1"
  4297. 2020-11-02T12:06:50.1760192Z [cluster1] 000 "submariner-cable-cluster2-172-17-0-4-0-1": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #36 "submariner-cable-cluster2-172-17-0-4-0-0"
  4298. 2020-11-02T12:06:50.1762182Z [cluster1] I1102 12:01:40.959440 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-0-2 --id 172.17.0.6 --host 172.17.0.6 --client 172.17.0.6/32 --to --id 172.17.0.4 --host 172.17.0.4 --client 10.2.0.0/16]
  4299. 2020-11-02T12:06:50.1764642Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-4-0-2"
  4300. 2020-11-02T12:06:50.1766236Z [cluster1] 000 "submariner-cable-cluster2-172-17-0-4-0-2": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #36 "submariner-cable-cluster2-172-17-0-4-0-0"
  4301. 2020-11-02T12:06:50.1768228Z [cluster1] I1102 12:01:41.020150 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-1-0 --id 172.17.0.6 --host 172.17.0.6 --client 100.1.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 172.17.0.4/32]
  4302. 2020-11-02T12:06:50.1769526Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-4-1-0"
  4303. 2020-11-02T12:06:50.1771076Z [cluster1] 000 "submariner-cable-cluster2-172-17-0-4-1-0": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #36 "submariner-cable-cluster2-172-17-0-4-0-0"
  4304. 2020-11-02T12:06:50.1773090Z [cluster1] I1102 12:01:41.027656 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-1-1 --id 172.17.0.6 --host 172.17.0.6 --client 100.1.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 100.2.0.0/16]
  4305. 2020-11-02T12:06:50.1775772Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-4-1-1"
  4306. 2020-11-02T12:06:50.1777913Z [cluster1] 000 "submariner-cable-cluster2-172-17-0-4-1-1": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #36 "submariner-cable-cluster2-172-17-0-4-0-0"
  4307. 2020-11-02T12:06:50.1780450Z [cluster1] I1102 12:01:41.036909 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-1-2 --id 172.17.0.6 --host 172.17.0.6 --client 100.1.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 10.2.0.0/16]
  4308. 2020-11-02T12:06:50.1781838Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-4-1-2"
  4309. 2020-11-02T12:06:50.1783330Z [cluster1] I1102 12:01:41.043963 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-2-0 --id 172.17.0.6 --host 172.17.0.6 --client 10.1.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 172.17.0.4/32]
  4310. 2020-11-02T12:06:50.1785074Z [cluster1] 000 "submariner-cable-cluster2-172-17-0-4-1-2": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #36 "submariner-cable-cluster2-172-17-0-4-0-0"
  4311. 2020-11-02T12:06:50.1787033Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-4-2-0"
  4312. 2020-11-02T12:06:50.1788729Z [cluster1] 000 "submariner-cable-cluster2-172-17-0-4-2-0": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #36 "submariner-cable-cluster2-172-17-0-4-0-0"
  4313. 2020-11-02T12:06:50.1790810Z [cluster1] I1102 12:01:41.054930 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-2-1 --id 172.17.0.6 --host 172.17.0.6 --client 10.1.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 100.2.0.0/16]
  4314. 2020-11-02T12:06:50.1792795Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-4-2-1"
  4315. 2020-11-02T12:06:50.1794535Z [cluster1] 000 "submariner-cable-cluster2-172-17-0-4-2-1": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #36 "submariner-cable-cluster2-172-17-0-4-0-0"
  4316. 2020-11-02T12:06:50.1797036Z [cluster1] I1102 12:01:41.060361 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-2-2 --id 172.17.0.6 --host 172.17.0.6 --client 10.1.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 10.2.0.0/16]
  4317. 2020-11-02T12:06:50.1798473Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-4-2-2"
  4318. 2020-11-02T12:06:50.1800282Z [cluster1] 000 "submariner-cable-cluster2-172-17-0-4-2-2": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #36 "submariner-cable-cluster2-172-17-0-4-0-0"
  4319. 2020-11-02T12:06:50.1802350Z [cluster1] I1102 12:01:41.064507 1 cableengine.go:150] Successfully installed Endpoint cable "submariner-cable-cluster2-172-17-0-4" with remote IP 172.17.0.4
  4320. 2020-11-02T12:06:50.1804025Z [cluster1] I1102 12:01:41.064594 1 tunnel.go:68] Tunnel controller successfully installed Endpoint cable submariner-cable-cluster2-172-17-0-4 in the engine
  4321. 2020-11-02T12:06:50.1806042Z [cluster1] I1102 12:01:41.064639 1 resource_syncer.go:352] Syncer "Tunnel controller": transform function returned nil - not syncing - requeue: false
  4322. 2020-11-02T12:06:50.1808635Z [cluster1] I1102 12:05:30.606224 1 resource_syncer.go:294] Syncer "broker -> local for *v1.Endpoint" informed of deleted resource "submariner-k8s-broker/cluster2-submariner-cable-cluster2-172-17-0-4"
  4323. 2020-11-02T12:06:50.1814657Z [cluster1] I1102 12:05:30.606252 1 resource_syncer.go:315] Syncer "broker -> local for *v1.Endpoint" deleting resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:01:40Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-k8s-broker", "resourceVersion":"3453", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"bb7e9be5-2d33-4f1d-947e-4a46c6904408"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  4324. 2020-11-02T12:06:50.1820527Z [cluster1] I1102 12:05:30.606418 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  4325. 2020-11-02T12:06:50.1825107Z [cluster1] I1102 12:05:30.606432 1 federator.go:73] Deleting resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:01:40Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-operator", "resourceVersion":"3453", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"bb7e9be5-2d33-4f1d-947e-4a46c6904408"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  4326. 2020-11-02T12:06:50.1831831Z [cluster1] I1102 12:05:30.613588 1 resource_syncer.go:330] Syncer "broker -> local for *v1.Endpoint" successfully deleted "submariner-k8s-broker/cluster2-submariner-cable-cluster2-172-17-0-4"
  4327. 2020-11-02T12:06:50.1834793Z [cluster1] I1102 12:05:30.618589 1 resource_syncer.go:294] Syncer "local -> broker for *v1.Endpoint" informed of deleted resource "submariner-operator/cluster2-submariner-cable-cluster2-172-17-0-4"
  4328. 2020-11-02T12:06:50.1837461Z [cluster1] I1102 12:05:30.618625 1 resource_syncer.go:437] Found cluster ID label "cluster2" - not syncing resource "cluster2-submariner-cable-cluster2-172-17-0-4"
  4329. 2020-11-02T12:06:50.1839375Z [cluster1] I1102 12:05:30.619568 1 resource_syncer.go:294] Syncer "Tunnel controller" informed of deleted resource "submariner-operator/cluster2-submariner-cable-cluster2-172-17-0-4"
  4330. 2020-11-02T12:06:50.1846753Z [cluster1] I1102 12:05:30.619771 1 tunnel.go:74] Tunnel controller processing removed submariner Endpoint object: &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster2-submariner-cable-cluster2-172-17-0-4", GenerateName:"", Namespace:"submariner-operator", SelfLink:"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", UID:"c09b80a0-c12c-4246-a9ea-692c9619737f", ResourceVersion:"3454", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63739915300, loc:(*time.Location)(0x2020aa0)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"submariner-io/clusterID":"cluster2"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster2", CableName:"submariner-cable-cluster2-172-17-0-4", Hostname:"cluster2-worker", Subnets:[]string{"100.2.0.0/16", "10.2.0.0/16"}, PrivateIP:"172.17.0.4", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string(nil)}}
  4331. 2020-11-02T12:06:50.1852854Z [cluster1] I1102 12:05:30.619839 1 cableengine.go:166] Removing Endpoint cable "submariner-cable-cluster2-172-17-0-4"
  4332. 2020-11-02T12:06:50.1854402Z [cluster1] I1102 12:05:30.619848 1 libreswan.go:300] Deleting connection to {{cluster2 submariner-cable-cluster2-172-17-0-4 cluster2-worker [100.2.0.0/16 10.2.0.0/16] 172.17.0.4 false libreswan map[]}}
  4333. 2020-11-02T12:06:50.1857297Z [cluster1] I1102 12:05:30.619859 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-0-0]
  4334. 2020-11-02T12:06:50.1858688Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-0-0": terminating SAs using this connection
  4335. 2020-11-02T12:06:50.1860489Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-0-0": IKE SA is shared - only terminating IPsec SA
  4336. 2020-11-02T12:06:50.1861902Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-0-0" #47: deleting state (STATE_V2_IPSEC_I) aged 229.401s and sending notification
  4337. 2020-11-02T12:06:50.1863217Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-0-0" #47: ESP traffic information: in=0B out=0B
  4338. 2020-11-02T12:06:50.1864586Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-0-0" #39: deleting state (STATE_V2_IPSEC_R) aged 229.505s and sending notification
  4339. 2020-11-02T12:06:50.1866023Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-0-0" #39: ESP traffic information: in=0B out=0B
  4340. 2020-11-02T12:06:50.1867568Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-0-0" #36: deleting state (STATE_PARENT_I3) aged 229.766s and sending notification
  4341. 2020-11-02T12:06:50.1868965Z [cluster1] I1102 12:05:30.649345 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-0-1]
  4342. 2020-11-02T12:06:50.1870383Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-0-1": terminating SAs using this connection
  4343. 2020-11-02T12:06:50.1871702Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-0-1" #46: deleting state (STATE_V2_IPSEC_I) aged 229.431s and NOT sending notification
  4344. 2020-11-02T12:06:50.1872969Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-0-1" #46: ESP traffic information: in=0B out=0B
  4345. 2020-11-02T12:06:50.1878089Z [cluster1] I1102 12:05:30.671680 1 resource_syncer.go:264] Syncer "broker -> local for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:05:30Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-11", "namespace":"submariner-k8s-broker", "resourceVersion":"3455", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-11", "uid":"59c1f9b2-d440-4647-9bdd-1450a56598e9"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-11", "cluster_id":"cluster2", "hostname":"cluster2-worker2", "nat_enabled":false, "private_ip":"172.17.0.11", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  4346. 2020-11-02T12:06:50.1886333Z [cluster1] I1102 12:05:30.677637 1 resource_syncer.go:276] Syncing resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:05:30Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-11", "namespace":"submariner-k8s-broker", "resourceVersion":"3455", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-11", "uid":"59c1f9b2-d440-4647-9bdd-1450a56598e9"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-11", "cluster_id":"cluster2", "hostname":"cluster2-worker2", "nat_enabled":false, "private_ip":"172.17.0.11", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  4347. 2020-11-02T12:06:50.1894266Z [cluster1] I1102 12:05:30.678741 1 federator.go:42] In Distribute for &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:05:30Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-11", "namespace":"submariner-k8s-broker", "resourceVersion":"3455", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-11", "uid":"59c1f9b2-d440-4647-9bdd-1450a56598e9"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-11", "cluster_id":"cluster2", "hostname":"cluster2-worker2", "nat_enabled":false, "private_ip":"172.17.0.11", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  4348. 2020-11-02T12:06:50.1898639Z [cluster1] I1102 12:05:30.679456 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  4349. 2020-11-02T12:06:50.1902603Z [cluster1] I1102 12:05:30.681632 1 create_or_update.go:32] Creating resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-11", "namespace":"submariner-operator"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-11", "cluster_id":"cluster2", "hostname":"cluster2-worker2", "nat_enabled":false, "private_ip":"172.17.0.11", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  4350. 2020-11-02T12:06:50.1907744Z [cluster1] I1102 12:05:30.713147 1 resource_syncer.go:285] Syncer "broker -> local for *v1.Endpoint" successfully synced "submariner-k8s-broker/cluster2-submariner-cable-cluster2-172-17-0-11"
  4351. 2020-11-02T12:06:50.1909630Z [cluster1] I1102 12:05:30.714163 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-0-2]
  4352. 2020-11-02T12:06:50.1910868Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-0-2": terminating SAs using this connection
  4353. 2020-11-02T12:06:50.1912815Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-0-2" #45: deleting state (STATE_V2_IPSEC_I) aged 229.495s and NOT sending notification
  4354. 2020-11-02T12:06:50.1914348Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-0-2" #45: ESP traffic information: in=0B out=0B
  4355. 2020-11-02T12:06:50.1920473Z [cluster1] I1102 12:05:30.718544 1 resource_syncer.go:264] Syncer "local -> broker for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:05:30Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-11", "namespace":"submariner-operator", "resourceVersion":"3456", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-11", "uid":"79f4d7cc-a617-4f29-b5a4-300f464cb55f"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-11", "cluster_id":"cluster2", "hostname":"cluster2-worker2", "nat_enabled":false, "private_ip":"172.17.0.11", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  4356. 2020-11-02T12:06:50.1925280Z [cluster1] I1102 12:05:30.718746 1 resource_syncer.go:437] Found cluster ID label "cluster2" - not syncing resource "cluster2-submariner-cable-cluster2-172-17-0-11"
  4357. 2020-11-02T12:06:50.1927471Z [cluster1] I1102 12:05:30.738746 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-1-0]
  4358. 2020-11-02T12:06:50.1928882Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-1-0": terminating SAs using this connection
  4359. 2020-11-02T12:06:50.1930929Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-1-0" #44: deleting state (STATE_V2_IPSEC_I) aged 229.520s and NOT sending notification
  4360. 2020-11-02T12:06:50.1932184Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-1-0" #44: ESP traffic information: in=0B out=0B
  4361. 2020-11-02T12:06:50.1933436Z [cluster1] I1102 12:05:30.756122 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-1-1]
  4362. 2020-11-02T12:06:50.1934678Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-1-1": terminating SAs using this connection
  4363. 2020-11-02T12:06:50.1936387Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-1-1" #43: deleting state (STATE_V2_IPSEC_I) aged 229.539s and NOT sending notification
  4364. 2020-11-02T12:06:50.1938354Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-1-1" #43: ESP traffic information: in=0B out=0B
  4365. 2020-11-02T12:06:50.1940041Z [cluster1] I1102 12:05:30.780381 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-1-2]
  4366. 2020-11-02T12:06:50.1941270Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-1-2": terminating SAs using this connection
  4367. 2020-11-02T12:06:50.1942580Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-1-2" #42: deleting state (STATE_V2_IPSEC_I) aged 229.561s and NOT sending notification
  4368. 2020-11-02T12:06:50.1943847Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-1-2" #42: ESP traffic information: in=0B out=0B
  4369. 2020-11-02T12:06:50.1945108Z [cluster1] I1102 12:05:30.795191 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-2-0]
  4370. 2020-11-02T12:06:50.1946721Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-2-0": terminating SAs using this connection
  4371. 2020-11-02T12:06:50.1948124Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-2-0" #41: deleting state (STATE_V2_IPSEC_I) aged 229.576s and NOT sending notification
  4372. 2020-11-02T12:06:50.1949794Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-2-0" #41: ESP traffic information: in=0B out=0B
  4373. 2020-11-02T12:06:50.1951373Z [cluster1] I1102 12:05:30.845581 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-2-1]
  4374. 2020-11-02T12:06:50.1952663Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-2-1": terminating SAs using this connection
  4375. 2020-11-02T12:06:50.1953974Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-2-1" #40: deleting state (STATE_V2_IPSEC_I) aged 229.626s and NOT sending notification
  4376. 2020-11-02T12:06:50.1955251Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-2-1" #40: ESP traffic information: in=0B out=0B
  4377. 2020-11-02T12:06:50.1956931Z [cluster1] I1102 12:05:30.906881 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-2-2]
  4378. 2020-11-02T12:06:50.1958246Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-2-2": terminating SAs using this connection
  4379. 2020-11-02T12:06:50.1960495Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-2-2" #38: deleting state (STATE_V2_IPSEC_I) aged 229.767s and NOT sending notification
  4380. 2020-11-02T12:06:50.1961874Z [cluster1] 005 "submariner-cable-cluster2-172-17-0-4-2-2" #38: ESP traffic information: in=0B out=0B
  4381. 2020-11-02T12:06:50.1963207Z [cluster1] 002 "submariner-cable-cluster2-172-17-0-4-2-2" #37: deleting state (STATE_PARENT_R2) aged 229.804s and sending notification
  4382. 2020-11-02T12:06:50.1965122Z [cluster1] I1102 12:05:30.970035 1 cableengine.go:176] Successfully removed Endpoint cable "submariner-cable-cluster2-172-17-0-4"
  4383. 2020-11-02T12:06:50.1967139Z [cluster1] I1102 12:05:30.970369 1 tunnel.go:85] Tunnel controller successfully removed Endpoint cable submariner-cable-cluster2-172-17-0-4 from the engine
  4384. 2020-11-02T12:06:50.1968444Z [cluster1] I1102 12:05:30.970562 1 resource_syncer.go:352] Syncer "Tunnel controller": transform function returned nil - not syncing - requeue: false
  4385. 2020-11-02T12:06:50.1973389Z [cluster1] I1102 12:05:30.970732 1 resource_syncer.go:264] Syncer "Tunnel controller" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:05:30Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-11", "namespace":"submariner-operator", "resourceVersion":"3456", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-11", "uid":"79f4d7cc-a617-4f29-b5a4-300f464cb55f"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-11", "cluster_id":"cluster2", "hostname":"cluster2-worker2", "nat_enabled":false, "private_ip":"172.17.0.11", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  4386. 2020-11-02T12:06:50.1983503Z [cluster1] I1102 12:05:30.971109 1 tunnel.go:56] Tunnel controller processing added or updated submariner Endpoint object: &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster2-submariner-cable-cluster2-172-17-0-11", GenerateName:"", Namespace:"submariner-operator", SelfLink:"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-11", UID:"79f4d7cc-a617-4f29-b5a4-300f464cb55f", ResourceVersion:"3456", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63739915530, loc:(*time.Location)(0x2020aa0)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"submariner-io/clusterID":"cluster2"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster2", CableName:"submariner-cable-cluster2-172-17-0-11", Hostname:"cluster2-worker2", Subnets:[]string{"100.2.0.0/16", "10.2.0.0/16"}, PrivateIP:"172.17.0.11", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string(nil)}}
  4387. 2020-11-02T12:06:50.1991393Z [cluster1] I1102 12:05:30.971368 1 cableengine.go:101] Installing Endpoint cable "submariner-cable-cluster2-172-17-0-11"
  4388. 2020-11-02T12:06:50.1992643Z [cluster1] I1102 12:05:30.971456 1 libreswan.go:164] Active connections: [submariner-cable-cluster3-172-17-0-7]
  4389. 2020-11-02T12:06:50.1993344Z [cluster1] 002 listening for IKE messages
  4390. 2020-11-02T12:06:50.1993672Z [cluster1] 002 forgetting secrets
  4391. 2020-11-02T12:06:50.1994060Z [cluster1] 002 loading secrets from "/etc/ipsec.secrets"
  4392. 2020-11-02T12:06:50.1994567Z [cluster1] 002 loading secrets from "/etc/ipsec.d/submariner.secrets"
  4393. 2020-11-02T12:06:50.4034510Z [cluster1] I1102 12:05:30.975707 1 libreswan.go:245] Creating connection(s) for {{cluster2 submariner-cable-cluster2-172-17-0-11 cluster2-worker2 [100.2.0.0/16 10.2.0.0/16] 172.17.0.11 false libreswan map[]}}
  4394. 2020-11-02T12:06:50.4036811Z [cluster1] I1102 12:05:30.975905 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-11-0-0 --id 172.17.0.6 --host 172.17.0.6 --client 172.17.0.6/32 --to --id 172.17.0.11 --host 172.17.0.11 --client 172.17.0.11/32]
  4395. 2020-11-02T12:06:50.4038505Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-11-0-0"
  4396. 2020-11-02T12:06:50.4040324Z [cluster1] I1102 12:05:31.058245 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-11-0-1 --id 172.17.0.6 --host 172.17.0.6 --client 172.17.0.6/32 --to --id 172.17.0.11 --host 172.17.0.11 --client 100.2.0.0/16]
  4397. 2020-11-02T12:06:50.4042305Z [cluster1] 181 "submariner-cable-cluster2-172-17-0-11-0-0" #49: initiating IKEv2 IKE SA
  4398. 2020-11-02T12:06:50.4043664Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-11-0-1"
  4399. 2020-11-02T12:06:50.4046267Z [cluster1] I1102 12:05:31.177478 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-11-0-2 --id 172.17.0.6 --host 172.17.0.6 --client 172.17.0.6/32 --to --id 172.17.0.11 --host 172.17.0.11 --client 10.2.0.0/16]
  4400. 2020-11-02T12:06:50.4047832Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-11-0-2"
  4401. 2020-11-02T12:06:50.4049576Z [cluster1] I1102 12:05:31.248774 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-11-1-0 --id 172.17.0.6 --host 172.17.0.6 --client 100.1.0.0/16 --to --id 172.17.0.11 --host 172.17.0.11 --client 172.17.0.11/32]
  4402. 2020-11-02T12:06:50.4051301Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-11-1-0"
  4403. 2020-11-02T12:06:50.4055331Z [cluster1] I1102 12:05:31.254661 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-11-1-1 --id 172.17.0.6 --host 172.17.0.6 --client 100.1.0.0/16 --to --id 172.17.0.11 --host 172.17.0.11 --client 100.2.0.0/16]
  4404. 2020-11-02T12:06:50.4057257Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-11-1-1"
  4405. 2020-11-02T12:06:50.4059038Z [cluster1] I1102 12:05:31.257725 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-11-1-2 --id 172.17.0.6 --host 172.17.0.6 --client 100.1.0.0/16 --to --id 172.17.0.11 --host 172.17.0.11 --client 10.2.0.0/16]
  4406. 2020-11-02T12:06:50.4060656Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-11-1-2"
  4407. 2020-11-02T12:06:50.4062520Z [cluster1] I1102 12:05:31.260665 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-11-2-0 --id 172.17.0.6 --host 172.17.0.6 --client 10.1.0.0/16 --to --id 172.17.0.11 --host 172.17.0.11 --client 172.17.0.11/32]
  4408. 2020-11-02T12:06:50.4064026Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-11-2-0"
  4409. 2020-11-02T12:06:50.4067477Z [cluster1] I1102 12:05:31.275130 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-11-2-1 --id 172.17.0.6 --host 172.17.0.6 --client 10.1.0.0/16 --to --id 172.17.0.11 --host 172.17.0.11 --client 100.2.0.0/16]
  4410. 2020-11-02T12:06:50.4069122Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-11-2-1"
  4411. 2020-11-02T12:06:50.4070972Z [cluster1] I1102 12:05:31.280549 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-11-2-2 --id 172.17.0.6 --host 172.17.0.6 --client 10.1.0.0/16 --to --id 172.17.0.11 --host 172.17.0.11 --client 10.2.0.0/16]
  4412. 2020-11-02T12:06:50.4072651Z [cluster1] 002 added connection description "submariner-cable-cluster2-172-17-0-11-2-2"
  4413. 2020-11-02T12:06:50.4074191Z [cluster1] I1102 12:05:31.285815 1 cableengine.go:150] Successfully installed Endpoint cable "submariner-cable-cluster2-172-17-0-11" with remote IP 172.17.0.11
  4414. 2020-11-02T12:06:50.4075802Z [cluster1] I1102 12:05:31.286061 1 tunnel.go:68] Tunnel controller successfully installed Endpoint cable submariner-cable-cluster2-172-17-0-11 in the engine
  4415. 2020-11-02T12:06:50.4079939Z [cluster1] I1102 12:05:31.286255 1 resource_syncer.go:352] Syncer "Tunnel controller": transform function returned nil - not syncing - requeue: false
  4416. 2020-11-02T12:06:50.4080551Z [cluster1] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  4417. 2020-11-02T12:06:50.4081669Z [submariner]$ [cluster1] kubectl -n submariner-operator logs submariner-routeagent-5vp7b
  4418. 2020-11-02T12:06:50.4083006Z [submariner]$ [cluster1] kubectl -n submariner-operator logs submariner-routeagent-5vp7b
  4419. 2020-11-02T12:06:50.4084540Z [submariner]$ [cluster1] command kubectl --context=cluster1 -n submariner-operator logs submariner-routeagent-5vp7b
  4420. 2020-11-02T12:06:50.4103786Z [submariner]$ [cluster1] kubectl --context=cluster1 -n submariner-operator logs submariner-routeagent-5vp7b
  4421. 2020-11-02T12:06:50.6842123Z [cluster1] +++++++++++++++++++++: Logs for Pod submariner-routeagent-5vp7b in namespace submariner-operator :++++++++++++++++++++++
  4422. 2020-11-02T12:06:50.6842984Z [cluster1] + trap 'exit 1' SIGTERM SIGINT
  4423. 2020-11-02T12:06:50.6843354Z [cluster1] + SUBMARINER_VERBOSITY=2
  4424. 2020-11-02T12:06:50.6843819Z [cluster1] + '[' false == true ']'
  4425. 2020-11-02T12:06:50.6844594Z [cluster1] + DEBUG=-v=2
  4426. 2020-11-02T12:06:50.6845063Z [cluster1] + for f in iptables-save iptables
  4427. 2020-11-02T12:06:50.6845654Z [cluster1] ++ find_iptables_on_host iptables-save
  4428. 2020-11-02T12:06:50.6846235Z [cluster1] ++ chroot /host test -x /usr/sbin/iptables-save
  4429. 2020-11-02T12:06:50.6846584Z [cluster1] ++ echo /usr/sbin
  4430. 2020-11-02T12:06:50.6846853Z [cluster1] ++ return
  4431. 2020-11-02T12:06:50.6847140Z [cluster1] + location=/usr/sbin
  4432. 2020-11-02T12:06:50.6847600Z [cluster1] + '[' /usr/sbin '!=' unknown ']'
  4433. 2020-11-02T12:06:50.6848408Z [cluster1] + echo 'iptables-save is present on the host at /usr/sbin/iptables-save'
  4434. 2020-11-02T12:06:50.6849191Z [cluster1] + sed 's!@@PATH@@!/usr/sbin!' /usr/sbin/iptables-wrapper.in
  4435. 2020-11-02T12:06:50.6850091Z [cluster1] iptables-save is present on the host at /usr/sbin/iptables-save
  4436. 2020-11-02T12:06:50.6850691Z [cluster1] + for f in iptables-save iptables
  4437. 2020-11-02T12:06:50.6851071Z [cluster1] ++ find_iptables_on_host iptables
  4438. 2020-11-02T12:06:50.6851566Z [cluster1] ++ chroot /host test -x /usr/sbin/iptables
  4439. 2020-11-02T12:06:50.6851893Z [cluster1] ++ echo /usr/sbin
  4440. 2020-11-02T12:06:50.6852159Z [cluster1] ++ return
  4441. 2020-11-02T12:06:50.6852433Z [cluster1] + location=/usr/sbin
  4442. 2020-11-02T12:06:50.6852871Z [cluster1] + '[' /usr/sbin '!=' unknown ']'
  4443. 2020-11-02T12:06:50.6853412Z [cluster1] + echo 'iptables is present on the host at /usr/sbin/iptables'
  4444. 2020-11-02T12:06:50.6854240Z [cluster1] + sed 's!@@PATH@@!/usr/sbin!' /usr/sbin/iptables-wrapper.in
  4445. 2020-11-02T12:06:50.6854909Z [cluster1] iptables is present on the host at /usr/sbin/iptables
  4446. 2020-11-02T12:06:50.6855921Z [cluster1] + exec submariner-route-agent -v=2 -alsologtostderr
  4447. 2020-11-02T12:06:50.6856753Z [cluster1] I1102 11:58:08.758698 1 main.go:47] Starting submariner-route-agent
  4448. 2020-11-02T12:06:50.6857799Z [cluster1] W1102 11:58:08.807884 1 client_config.go:543] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
  4449. 2020-11-02T12:06:50.6859065Z [cluster1] I1102 11:58:08.811158 1 route.go:192] Starting Route Controller. ClusterID: cluster1, localClusterCIDR: [10.1.0.0/16], localServiceCIDR: [100.1.0.0/16]
  4450. 2020-11-02T12:06:50.6859793Z [cluster1] I1102 11:58:08.811358 1 route.go:196] Waiting for Endpoint informer caches to sync.
  4451. 2020-11-02T12:06:50.6860544Z [cluster1] I1102 11:58:08.926288 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.9
  4452. 2020-11-02T12:06:50.6861308Z [cluster1] I1102 11:58:08.926952 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.6
  4453. 2020-11-02T12:06:50.6861818Z [cluster1] I1102 11:58:09.016031 1 driver.go:37] Interface "lo" has "127.0.0.1" address
  4454. 2020-11-02T12:06:50.6862299Z [cluster1] I1102 11:58:09.016186 1 driver.go:37] Interface "vethd1aa2860" has "10.1.0.1" address
  4455. 2020-11-02T12:06:50.6862890Z [cluster1] I1102 11:58:09.016228 1 driver.go:42] Found CNI Interface "vethd1aa2860" that has IP "10.1.0.1" from ClusterCIDR "10.1.0.0/16"
  4456. 2020-11-02T12:06:50.6863599Z [cluster1] I1102 11:58:09.016494 1 driver.go:60] Successfully configured rp_filter to loose mode(2) on cniInterface "vethd1aa2860"
  4457. 2020-11-02T12:06:50.6864681Z [cluster1] I1102 11:58:09.081305 1 iptables.go:22] Install/ensure SUBMARINER-POSTROUTING chain exists
  4458. 2020-11-02T12:06:50.6865653Z [cluster1] I1102 11:58:09.118901 1 iptables.go:28] Insert SUBMARINER-POSTROUTING rule that has rules for inter-cluster traffic
  4459. 2020-11-02T12:06:50.6866575Z [cluster1] I1102 11:58:09.143679 1 iptables.go:34] Install/ensure SUBMARINER-INPUT chain exists
  4460. 2020-11-02T12:06:50.6867397Z [cluster1] I1102 11:58:09.373030 1 iptables.go:45] Allow VxLAN incoming traffic in SUBMARINER-INPUT Chain
  4461. 2020-11-02T12:06:50.6868298Z [cluster1] I1102 11:58:09.576151 1 iptables.go:53] Insert rule to allow traffic over vx-submariner interface in FORWARDing Chain
  4462. 2020-11-02T12:06:50.6869328Z [cluster1] I1102 11:58:09.674435 1 iptables.go:65] Installing rule for host network to remote cluster communication: -s 240.0.0.0/8 -o vx-submariner -j SNAT --to 10.1.0.1
  4463. 2020-11-02T12:06:50.6870363Z [cluster1] I1102 11:58:09.858445 1 route.go:254] In cluster1, podIP of submariner-route-agent[0] is 172.17.0.9
  4464. 2020-11-02T12:06:50.6871224Z [cluster1] I1102 11:58:09.858495 1 route.go:254] In cluster1, podIP of submariner-route-agent[1] is 172.17.0.6
  4465. 2020-11-02T12:06:50.6872293Z [cluster1] I1102 11:58:09.858500 1 route.go:268] Hostname is "cluster1-control-plane" and routeAgentNodeName is "cluster1-control-plane"
  4466. 2020-11-02T12:06:50.6873511Z [cluster1] I1102 11:58:10.008274 1 driver.go:87] Successfully annotated node "cluster1-control-plane" with cniIfaceIP "10.1.0.1"
  4467. 2020-11-02T12:06:50.6874342Z [cluster1] I1102 11:58:10.008540 1 route.go:286] Route agent workers started
  4468. 2020-11-02T12:06:50.6875397Z [cluster1] I1102 11:58:10.008689 1 route.go:507] Processing submariner-route-agent pod "submariner-routeagent-5vp7b" with host IP "172.17.0.9", pod IP "172.17.0.9"
  4469. 2020-11-02T12:06:50.6887327Z [cluster1] I1102 11:58:10.008859 1 route.go:531] Successfully processed submariner-route-agent pod "submariner-routeagent-5vp7b"
  4470. 2020-11-02T12:06:50.6888690Z [cluster1] I1102 11:58:10.008966 1 route.go:507] Processing submariner-route-agent pod "submariner-routeagent-c4hh4" with host IP "172.17.0.6", pod IP "172.17.0.6"
  4471. 2020-11-02T12:06:50.6890130Z [cluster1] I1102 11:58:10.009072 1 route.go:531] Successfully processed submariner-route-agent pod "submariner-routeagent-c4hh4"
  4472. 2020-11-02T12:06:50.6891703Z [cluster1] I1102 11:58:27.351462 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  4473. 2020-11-02T12:06:50.6893172Z [cluster1] I1102 11:58:27.379319 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.1.0.0/16 -d 100.2.0.0/16 -j ACCEPT
  4474. 2020-11-02T12:06:50.6894032Z [cluster1] I1102 11:58:27.396803 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 100.2.0.0/16 -d 10.1.0.0/16 -j ACCEPT
  4475. 2020-11-02T12:06:50.6894942Z [cluster1] I1102 11:58:27.411063 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.1.0.0/16 -d 10.2.0.0/16 -j ACCEPT
  4476. 2020-11-02T12:06:50.6895804Z [cluster1] I1102 11:58:27.423351 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 10.2.0.0/16 -d 10.1.0.0/16 -j ACCEPT
  4477. 2020-11-02T12:06:50.6897332Z [cluster1] I1102 11:58:27.539422 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4478. 2020-11-02T12:06:50.6898603Z [cluster1] I1102 11:58:27.539721 1 route.go:612] This route agent is running on a non-gateway/non-active node
  4479. 2020-11-02T12:06:50.6899599Z [cluster1] I1102 11:58:27.539829 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.6
  4480. 2020-11-02T12:06:50.6900308Z [cluster1] I1102 11:58:27.550293 1 route.go:771] Reconciling routes to gw: 240.17.0.6
  4481. 2020-11-02T12:06:50.6900819Z [cluster1] I1102 11:58:27.552073 1 route.go:787] Processing route {Ifindex: 5 Dst: 240.0.0.0/8 Src: 240.17.0.9 Gw: <nil> Flags: [] Table: 254}
  4482. 2020-11-02T12:06:50.6901455Z [cluster1] I1102 11:58:27.552635 1 route.go:790] Found nil gw or dst
  4483. 2020-11-02T12:06:50.6902569Z [cluster1] I1102 11:58:27.553157 1 route.go:647] Successfully processed local Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6"
  4484. 2020-11-02T12:06:50.6904237Z [cluster1] I1102 11:58:27.942974 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  4485. 2020-11-02T12:06:50.6905504Z [cluster1] I1102 11:58:27.986706 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.1.0.0/16 -d 100.3.0.0/16 -j ACCEPT
  4486. 2020-11-02T12:06:50.6906355Z [cluster1] I1102 11:58:28.050893 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 100.3.0.0/16 -d 10.1.0.0/16 -j ACCEPT
  4487. 2020-11-02T12:06:50.6907201Z [cluster1] I1102 11:58:28.095378 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.1.0.0/16 -d 10.3.0.0/16 -j ACCEPT
  4488. 2020-11-02T12:06:50.6908058Z [cluster1] I1102 11:58:28.110072 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 10.3.0.0/16 -d 10.1.0.0/16 -j ACCEPT
  4489. 2020-11-02T12:06:50.6909432Z [cluster1] I1102 11:58:38.895621 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  4490. 2020-11-02T12:06:50.6911198Z [cluster1] I1102 11:58:38.895657 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4491. 2020-11-02T12:06:50.6912670Z [cluster1] I1102 11:58:38.895666 1 route.go:612] This route agent is running on a non-gateway/non-active node
  4492. 2020-11-02T12:06:50.6913494Z [cluster1] I1102 11:58:38.895675 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.6
  4493. 2020-11-02T12:06:50.6914122Z [cluster1] I1102 11:58:38.895901 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  4494. 2020-11-02T12:06:50.6914655Z [cluster1] I1102 11:58:39.003488 1 route.go:771] Reconciling routes to gw: 240.17.0.6
  4495. 2020-11-02T12:06:50.6915148Z [cluster1] I1102 11:58:39.003777 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4496. 2020-11-02T12:06:50.6915771Z [cluster1] I1102 11:58:39.003860 1 route.go:793] Found route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4497. 2020-11-02T12:06:50.6955180Z [cluster1] I1102 11:58:39.003887 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4498. 2020-11-02T12:06:50.6955981Z [cluster1] I1102 11:58:39.003913 1 route.go:793] Found route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4499. 2020-11-02T12:06:50.6956610Z [cluster1] I1102 11:58:39.003936 1 route.go:787] Processing route {Ifindex: 5 Dst: 240.0.0.0/8 Src: 240.17.0.9 Gw: <nil> Flags: [] Table: 254}
  4500. 2020-11-02T12:06:50.6957075Z [cluster1] I1102 11:58:39.003965 1 route.go:790] Found nil gw or dst
  4501. 2020-11-02T12:06:50.6957478Z [cluster1] I1102 11:58:39.004054 1 route.go:829] Found equivalent route, not adding
  4502. 2020-11-02T12:06:50.6957955Z [cluster1] I1102 11:58:39.004081 1 route.go:829] Found equivalent route, not adding
  4503. 2020-11-02T12:06:50.6959401Z [cluster1] I1102 11:58:39.004188 1 route.go:647] Successfully processed local Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6"
  4504. 2020-11-02T12:06:50.6961104Z [cluster1] I1102 11:58:39.004231 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  4505. 2020-11-02T12:06:50.6962815Z [cluster1] I1102 11:59:08.896322 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  4506. 2020-11-02T12:06:50.6965223Z [cluster1] I1102 11:59:08.896516 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4507. 2020-11-02T12:06:50.6966519Z [cluster1] I1102 11:59:08.896527 1 route.go:612] This route agent is running on a non-gateway/non-active node
  4508. 2020-11-02T12:06:50.6967374Z [cluster1] I1102 11:59:08.896639 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.6
  4509. 2020-11-02T12:06:50.6968022Z [cluster1] I1102 11:59:08.896952 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  4510. 2020-11-02T12:06:50.6968565Z [cluster1] I1102 11:59:09.505949 1 route.go:771] Reconciling routes to gw: 240.17.0.6
  4511. 2020-11-02T12:06:50.6969097Z [cluster1] I1102 11:59:09.509277 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4512. 2020-11-02T12:06:50.6969735Z [cluster1] I1102 11:59:09.509316 1 route.go:793] Found route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4513. 2020-11-02T12:06:50.6970366Z [cluster1] I1102 11:59:09.509373 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4514. 2020-11-02T12:06:50.6971218Z [cluster1] I1102 11:59:09.509383 1 route.go:793] Found route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4515. 2020-11-02T12:06:50.6971885Z [cluster1] I1102 11:59:09.509396 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4516. 2020-11-02T12:06:50.6972515Z [cluster1] I1102 11:59:09.509408 1 route.go:793] Found route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4517. 2020-11-02T12:06:50.6973150Z [cluster1] I1102 11:59:09.509416 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4518. 2020-11-02T12:06:50.6973777Z [cluster1] I1102 11:59:09.509428 1 route.go:793] Found route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4519. 2020-11-02T12:06:50.6974392Z [cluster1] I1102 11:59:09.509436 1 route.go:787] Processing route {Ifindex: 5 Dst: 240.0.0.0/8 Src: 240.17.0.9 Gw: <nil> Flags: [] Table: 254}
  4520. 2020-11-02T12:06:50.6974869Z [cluster1] I1102 11:59:09.509447 1 route.go:790] Found nil gw or dst
  4521. 2020-11-02T12:06:50.6975298Z [cluster1] I1102 11:59:09.510375 1 route.go:829] Found equivalent route, not adding
  4522. 2020-11-02T12:06:50.6975782Z [cluster1] I1102 11:59:09.510389 1 route.go:829] Found equivalent route, not adding
  4523. 2020-11-02T12:06:50.6976264Z [cluster1] I1102 11:59:09.510406 1 route.go:829] Found equivalent route, not adding
  4524. 2020-11-02T12:06:50.6976750Z [cluster1] I1102 11:59:09.510437 1 route.go:829] Found equivalent route, not adding
  4525. 2020-11-02T12:06:50.6977985Z [cluster1] I1102 11:59:09.510481 1 route.go:647] Successfully processed local Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6"
  4526. 2020-11-02T12:06:50.6979705Z [cluster1] I1102 11:59:09.510505 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  4527. 2020-11-02T12:06:50.6981469Z [cluster1] I1102 11:59:38.896496 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  4528. 2020-11-02T12:06:50.6983256Z [cluster1] I1102 11:59:38.896708 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4529. 2020-11-02T12:06:50.6985689Z [cluster1] I1102 11:59:38.896851 1 route.go:612] This route agent is running on a non-gateway/non-active node
  4530. 2020-11-02T12:06:50.6987033Z [cluster1] I1102 11:59:38.896942 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.6
  4531. 2020-11-02T12:06:50.6987807Z [cluster1] I1102 11:59:38.897382 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  4532. 2020-11-02T12:06:50.6988759Z [cluster1] I1102 11:59:38.910760 1 route.go:771] Reconciling routes to gw: 240.17.0.6
  4533. 2020-11-02T12:06:50.6989288Z [cluster1] I1102 11:59:38.911677 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4534. 2020-11-02T12:06:50.6989905Z [cluster1] I1102 11:59:38.913729 1 route.go:793] Found route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4535. 2020-11-02T12:06:50.6990531Z [cluster1] I1102 11:59:38.914122 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4536. 2020-11-02T12:06:50.6991168Z [cluster1] I1102 11:59:38.914403 1 route.go:793] Found route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4537. 2020-11-02T12:06:50.6991795Z [cluster1] I1102 11:59:38.914680 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4538. 2020-11-02T12:06:50.6992506Z [cluster1] I1102 11:59:38.914784 1 route.go:793] Found route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4539. 2020-11-02T12:06:50.6993146Z [cluster1] I1102 11:59:38.914905 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4540. 2020-11-02T12:06:50.6993770Z [cluster1] I1102 11:59:38.914999 1 route.go:793] Found route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4541. 2020-11-02T12:06:50.6994455Z [cluster1] I1102 11:59:38.915126 1 route.go:787] Processing route {Ifindex: 5 Dst: 240.0.0.0/8 Src: 240.17.0.9 Gw: <nil> Flags: [] Table: 254}
  4542. 2020-11-02T12:06:50.6994933Z [cluster1] I1102 11:59:38.915326 1 route.go:790] Found nil gw or dst
  4543. 2020-11-02T12:06:50.6995368Z [cluster1] I1102 11:59:38.915527 1 route.go:829] Found equivalent route, not adding
  4544. 2020-11-02T12:06:50.6995860Z [cluster1] I1102 11:59:38.915622 1 route.go:829] Found equivalent route, not adding
  4545. 2020-11-02T12:06:50.6996341Z [cluster1] I1102 11:59:38.915715 1 route.go:829] Found equivalent route, not adding
  4546. 2020-11-02T12:06:50.6996822Z [cluster1] I1102 11:59:38.915820 1 route.go:829] Found equivalent route, not adding
  4547. 2020-11-02T12:06:50.6998036Z [cluster1] I1102 11:59:38.916263 1 route.go:647] Successfully processed local Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6"
  4548. 2020-11-02T12:06:50.7015991Z [cluster1] I1102 11:59:38.916435 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  4549. 2020-11-02T12:06:50.7017810Z [cluster1] I1102 12:00:08.896870 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  4550. 2020-11-02T12:06:50.7019829Z [cluster1] I1102 12:00:08.897011 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4551. 2020-11-02T12:06:50.7021154Z [cluster1] I1102 12:00:08.897066 1 route.go:612] This route agent is running on a non-gateway/non-active node
  4552. 2020-11-02T12:06:50.7022167Z [cluster1] I1102 12:00:08.897095 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.6
  4553. 2020-11-02T12:06:50.7022819Z [cluster1] I1102 12:00:08.897403 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  4554. 2020-11-02T12:06:50.7064828Z [cluster1] I1102 12:00:08.901010 1 route.go:771] Reconciling routes to gw: 240.17.0.6
  4555. 2020-11-02T12:06:50.7065365Z [cluster1] I1102 12:00:08.901351 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4556. 2020-11-02T12:06:50.7066015Z [cluster1] I1102 12:00:08.901520 1 route.go:793] Found route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4557. 2020-11-02T12:06:50.7066646Z [cluster1] I1102 12:00:08.901550 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4558. 2020-11-02T12:06:50.7067278Z [cluster1] I1102 12:00:08.901564 1 route.go:793] Found route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4559. 2020-11-02T12:06:50.7067908Z [cluster1] I1102 12:00:08.901577 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4560. 2020-11-02T12:06:50.7068545Z [cluster1] I1102 12:00:08.901587 1 route.go:793] Found route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4561. 2020-11-02T12:06:50.7069169Z [cluster1] I1102 12:00:08.901595 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4562. 2020-11-02T12:06:50.7069900Z [cluster1] I1102 12:00:08.901604 1 route.go:793] Found route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4563. 2020-11-02T12:06:50.7070541Z [cluster1] I1102 12:00:08.901642 1 route.go:787] Processing route {Ifindex: 5 Dst: 240.0.0.0/8 Src: 240.17.0.9 Gw: <nil> Flags: [] Table: 254}
  4564. 2020-11-02T12:06:50.7071070Z [cluster1] I1102 12:00:08.901664 1 route.go:790] Found nil gw or dst
  4565. 2020-11-02T12:06:50.7071589Z [cluster1] I1102 12:00:08.901756 1 route.go:829] Found equivalent route, not adding
  4566. 2020-11-02T12:06:50.7072080Z [cluster1] I1102 12:00:08.901767 1 route.go:829] Found equivalent route, not adding
  4567. 2020-11-02T12:06:50.7072746Z [cluster1] I1102 12:00:08.901778 1 route.go:829] Found equivalent route, not adding
  4568. 2020-11-02T12:06:50.7073338Z [cluster1] I1102 12:00:08.901786 1 route.go:829] Found equivalent route, not adding
  4569. 2020-11-02T12:06:50.7074727Z [cluster1] I1102 12:00:08.901818 1 route.go:647] Successfully processed local Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6"
  4570. 2020-11-02T12:06:50.7078103Z [cluster1] I1102 12:00:08.901834 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  4571. 2020-11-02T12:06:50.7079990Z [cluster1] I1102 12:00:38.897007 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4572. 2020-11-02T12:06:50.7081272Z [cluster1] I1102 12:00:38.897051 1 route.go:612] This route agent is running on a non-gateway/non-active node
  4573. 2020-11-02T12:06:50.7082142Z [cluster1] I1102 12:00:38.897062 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.6
  4574. 2020-11-02T12:06:50.7082794Z [cluster1] I1102 12:00:38.897552 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  4575. 2020-11-02T12:06:50.7083343Z [cluster1] I1102 12:00:38.902022 1 route.go:771] Reconciling routes to gw: 240.17.0.6
  4576. 2020-11-02T12:06:50.7084022Z [cluster1] I1102 12:00:38.902302 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4577. 2020-11-02T12:06:50.7084664Z [cluster1] I1102 12:00:38.902339 1 route.go:793] Found route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4578. 2020-11-02T12:06:50.7085292Z [cluster1] I1102 12:00:38.902348 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4579. 2020-11-02T12:06:50.7086084Z [cluster1] I1102 12:00:38.902356 1 route.go:793] Found route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4580. 2020-11-02T12:06:50.7086722Z [cluster1] I1102 12:00:38.902456 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4581. 2020-11-02T12:06:50.7087348Z [cluster1] I1102 12:00:38.902471 1 route.go:793] Found route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4582. 2020-11-02T12:06:50.7087974Z [cluster1] I1102 12:00:38.902481 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4583. 2020-11-02T12:06:50.7088599Z [cluster1] I1102 12:00:38.902568 1 route.go:793] Found route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4584. 2020-11-02T12:06:50.7089232Z [cluster1] I1102 12:00:38.902577 1 route.go:787] Processing route {Ifindex: 5 Dst: 240.0.0.0/8 Src: 240.17.0.9 Gw: <nil> Flags: [] Table: 254}
  4585. 2020-11-02T12:06:50.7089717Z [cluster1] I1102 12:00:38.902584 1 route.go:790] Found nil gw or dst
  4586. 2020-11-02T12:06:50.7090222Z [cluster1] I1102 12:00:38.902783 1 route.go:829] Found equivalent route, not adding
  4587. 2020-11-02T12:06:50.7090699Z [cluster1] I1102 12:00:38.902800 1 route.go:829] Found equivalent route, not adding
  4588. 2020-11-02T12:06:50.7091180Z [cluster1] I1102 12:00:38.902810 1 route.go:829] Found equivalent route, not adding
  4589. 2020-11-02T12:06:50.7091658Z [cluster1] I1102 12:00:38.902926 1 route.go:829] Found equivalent route, not adding
  4590. 2020-11-02T12:06:50.7092891Z [cluster1] I1102 12:00:38.903029 1 route.go:647] Successfully processed local Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6"
  4591. 2020-11-02T12:06:50.7094775Z [cluster1] I1102 12:00:38.903056 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  4592. 2020-11-02T12:06:50.7096578Z [cluster1] I1102 12:00:38.903069 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  4593. 2020-11-02T12:06:50.7098458Z [cluster1] I1102 12:01:08.897631 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  4594. 2020-11-02T12:06:50.7100252Z [cluster1] I1102 12:01:08.897845 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  4595. 2020-11-02T12:06:50.7102033Z [cluster1] I1102 12:01:08.897912 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4596. 2020-11-02T12:06:50.7103323Z [cluster1] I1102 12:01:08.898000 1 route.go:612] This route agent is running on a non-gateway/non-active node
  4597. 2020-11-02T12:06:50.7104171Z [cluster1] I1102 12:01:08.898104 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.6
  4598. 2020-11-02T12:06:50.7104824Z [cluster1] I1102 12:01:08.898756 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  4599. 2020-11-02T12:06:50.7105370Z [cluster1] I1102 12:01:08.915491 1 route.go:771] Reconciling routes to gw: 240.17.0.6
  4600. 2020-11-02T12:06:50.7105895Z [cluster1] I1102 12:01:08.915907 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4601. 2020-11-02T12:06:50.7106528Z [cluster1] I1102 12:01:08.916095 1 route.go:793] Found route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4602. 2020-11-02T12:06:50.7107157Z [cluster1] I1102 12:01:08.916217 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4603. 2020-11-02T12:06:50.7107928Z [cluster1] I1102 12:01:08.917529 1 route.go:793] Found route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4604. 2020-11-02T12:06:50.7108565Z [cluster1] I1102 12:01:08.917770 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4605. 2020-11-02T12:06:50.7109180Z [cluster1] I1102 12:01:08.917895 1 route.go:793] Found route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4606. 2020-11-02T12:06:50.7109806Z [cluster1] I1102 12:01:08.921571 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4607. 2020-11-02T12:06:50.7110431Z [cluster1] I1102 12:01:08.921845 1 route.go:793] Found route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4608. 2020-11-02T12:06:50.7111067Z [cluster1] I1102 12:01:08.922847 1 route.go:787] Processing route {Ifindex: 5 Dst: 240.0.0.0/8 Src: 240.17.0.9 Gw: <nil> Flags: [] Table: 254}
  4609. 2020-11-02T12:06:50.7111545Z [cluster1] I1102 12:01:08.923032 1 route.go:790] Found nil gw or dst
  4610. 2020-11-02T12:06:50.7114651Z [cluster1] I1102 12:01:08.923289 1 route.go:829] Found equivalent route, not adding
  4611. 2020-11-02T12:06:50.7156513Z [cluster1] I1102 12:01:08.923449 1 route.go:829] Found equivalent route, not adding
  4612. 2020-11-02T12:06:50.7157118Z [cluster1] I1102 12:01:08.923581 1 route.go:829] Found equivalent route, not adding
  4613. 2020-11-02T12:06:50.7157693Z [cluster1] I1102 12:01:08.923727 1 route.go:829] Found equivalent route, not adding
  4614. 2020-11-02T12:06:50.7159283Z [cluster1] I1102 12:01:08.923908 1 route.go:647] Successfully processed local Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6"
  4615. 2020-11-02T12:06:50.7170116Z [cluster1] I1102 12:01:38.897868 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  4616. 2020-11-02T12:06:50.7182756Z [cluster1] I1102 12:01:38.897908 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4617. 2020-11-02T12:06:50.7184122Z [cluster1] I1102 12:01:38.897948 1 route.go:612] This route agent is running on a non-gateway/non-active node
  4618. 2020-11-02T12:06:50.7184965Z [cluster1] I1102 12:01:38.898006 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.6
  4619. 2020-11-02T12:06:50.7185617Z [cluster1] I1102 12:01:38.898353 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  4620. 2020-11-02T12:06:50.7186159Z [cluster1] I1102 12:01:38.902481 1 route.go:771] Reconciling routes to gw: 240.17.0.6
  4621. 2020-11-02T12:06:50.7186698Z [cluster1] I1102 12:01:38.903203 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4622. 2020-11-02T12:06:50.7187329Z [cluster1] I1102 12:01:38.903238 1 route.go:793] Found route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4623. 2020-11-02T12:06:50.7187965Z [cluster1] I1102 12:01:38.903248 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4624. 2020-11-02T12:06:50.7188834Z [cluster1] I1102 12:01:38.903488 1 route.go:793] Found route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4625. 2020-11-02T12:06:50.7189867Z [cluster1] I1102 12:01:38.903513 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4626. 2020-11-02T12:06:50.7190600Z [cluster1] I1102 12:01:38.903525 1 route.go:793] Found route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4627. 2020-11-02T12:06:50.7191579Z [cluster1] I1102 12:01:38.903532 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4628. 2020-11-02T12:06:50.7192319Z [cluster1] I1102 12:01:38.903560 1 route.go:793] Found route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4629. 2020-11-02T12:06:50.7193041Z [cluster1] I1102 12:01:38.903572 1 route.go:787] Processing route {Ifindex: 5 Dst: 240.0.0.0/8 Src: 240.17.0.9 Gw: <nil> Flags: [] Table: 254}
  4630. 2020-11-02T12:06:50.7193617Z [cluster1] I1102 12:01:38.903579 1 route.go:790] Found nil gw or dst
  4631. 2020-11-02T12:06:50.7194142Z [cluster1] I1102 12:01:38.903753 1 route.go:829] Found equivalent route, not adding
  4632. 2020-11-02T12:06:50.7194720Z [cluster1] I1102 12:01:38.903776 1 route.go:829] Found equivalent route, not adding
  4633. 2020-11-02T12:06:50.7195456Z [cluster1] I1102 12:01:38.903783 1 route.go:829] Found equivalent route, not adding
  4634. 2020-11-02T12:06:50.7196226Z [cluster1] I1102 12:01:38.903856 1 route.go:829] Found equivalent route, not adding
  4635. 2020-11-02T12:06:50.7197632Z [cluster1] I1102 12:01:38.903969 1 route.go:647] Successfully processed local Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6"
  4636. 2020-11-02T12:06:50.7199851Z [cluster1] I1102 12:01:38.903994 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  4637. 2020-11-02T12:06:50.7204304Z [cluster1] I1102 12:01:40.305568 1 route.go:682] Informed of removed endpoint: {"metadata":{"name":"cluster2-submariner-cable-cluster2-172-17-0-4","namespace":"submariner-operator","selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4","uid":"78218d29-51fa-4e18-9649-07670fababa0","resourceVersion":"2628","generation":1,"creationTimestamp":"2020-11-02T11:58:27Z","labels":{"submariner-io/clusterID":"cluster2"}},"spec":{"cluster_id":"cluster2","cable_name":"submariner-cable-cluster2-172-17-0-4","hostname":"cluster2-worker","subnets":["100.2.0.0/16","10.2.0.0/16"],"private_ip":"172.17.0.4","public_ip":"","nat_enabled":false,"backend":"libreswan"}}
  4638. 2020-11-02T12:06:50.7208181Z [cluster1] I1102 12:01:40.360727 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  4639. 2020-11-02T12:06:50.7210110Z [cluster1] I1102 12:02:08.897912 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  4640. 2020-11-02T12:06:50.7229439Z [cluster1] I1102 12:02:08.898180 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4641. 2020-11-02T12:06:50.7230995Z [cluster1] I1102 12:02:08.898207 1 route.go:612] This route agent is running on a non-gateway/non-active node
  4642. 2020-11-02T12:06:50.7232010Z [cluster1] I1102 12:02:08.898353 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.6
  4643. 2020-11-02T12:06:50.7232800Z [cluster1] I1102 12:02:08.898693 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  4644. 2020-11-02T12:06:50.7233459Z [cluster1] I1102 12:02:08.902178 1 route.go:771] Reconciling routes to gw: 240.17.0.6
  4645. 2020-11-02T12:06:50.7234082Z [cluster1] I1102 12:02:08.902571 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4646. 2020-11-02T12:06:50.7234990Z [cluster1] I1102 12:02:08.902634 1 route.go:793] Found route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4647. 2020-11-02T12:06:50.7235730Z [cluster1] I1102 12:02:08.902645 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4648. 2020-11-02T12:06:50.7236689Z [cluster1] I1102 12:02:08.902883 1 route.go:793] Found route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4649. 2020-11-02T12:06:50.7237428Z [cluster1] I1102 12:02:08.902894 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4650. 2020-11-02T12:06:50.7238154Z [cluster1] I1102 12:02:08.902902 1 route.go:793] Found route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4651. 2020-11-02T12:06:50.7238872Z [cluster1] I1102 12:02:08.902936 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4652. 2020-11-02T12:06:50.7239740Z [cluster1] I1102 12:02:08.903173 1 route.go:793] Found route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4653. 2020-11-02T12:06:50.7240463Z [cluster1] I1102 12:02:08.903234 1 route.go:787] Processing route {Ifindex: 5 Dst: 240.0.0.0/8 Src: 240.17.0.9 Gw: <nil> Flags: [] Table: 254}
  4654. 2020-11-02T12:06:50.7241040Z [cluster1] I1102 12:02:08.903243 1 route.go:790] Found nil gw or dst
  4655. 2020-11-02T12:06:50.7241667Z [cluster1] I1102 12:02:08.903434 1 route.go:829] Found equivalent route, not adding
  4656. 2020-11-02T12:06:50.7242286Z [cluster1] I1102 12:02:08.903460 1 route.go:829] Found equivalent route, not adding
  4657. 2020-11-02T12:06:50.7242863Z [cluster1] I1102 12:02:08.903557 1 route.go:829] Found equivalent route, not adding
  4658. 2020-11-02T12:06:50.7243437Z [cluster1] I1102 12:02:08.903625 1 route.go:829] Found equivalent route, not adding
  4659. 2020-11-02T12:06:50.7244923Z [cluster1] I1102 12:02:08.903794 1 route.go:647] Successfully processed local Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6"
  4660. 2020-11-02T12:06:50.7247506Z [cluster1] I1102 12:02:08.903897 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  4661. 2020-11-02T12:06:50.7273860Z [submariner]$ [cluster1] kubectl -n submariner-operator logs submariner-routeagent-c4hh4
  4662. 2020-11-02T12:06:50.7275394Z [cluster1] I1102 12:02:38.898190 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  4663. 2020-11-02T12:06:50.7277354Z [cluster1] I1102 12:02:38.898240 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4664. 2020-11-02T12:06:50.7278749Z [cluster1] I1102 12:02:38.898250 1 route.go:612] This route agent is running on a non-gateway/non-active node
  4665. 2020-11-02T12:06:50.7279589Z [cluster1] I1102 12:02:38.898257 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.6
  4666. 2020-11-02T12:06:50.7280348Z [cluster1] I1102 12:02:38.898799 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  4667. 2020-11-02T12:06:50.7280882Z [cluster1] I1102 12:02:38.903397 1 route.go:771] Reconciling routes to gw: 240.17.0.6
  4668. 2020-11-02T12:06:50.7281417Z [cluster1] I1102 12:02:38.903975 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4669. 2020-11-02T12:06:50.7282051Z [cluster1] I1102 12:02:38.904036 1 route.go:793] Found route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4670. 2020-11-02T12:06:50.7282685Z [cluster1] I1102 12:02:38.904084 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4671. 2020-11-02T12:06:50.7283312Z [cluster1] I1102 12:02:38.904113 1 route.go:793] Found route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4672. 2020-11-02T12:06:50.7284504Z [cluster1] I1102 12:02:38.904181 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4673. 2020-11-02T12:06:50.7285169Z [cluster1] I1102 12:02:38.904201 1 route.go:793] Found route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4674. 2020-11-02T12:06:50.7285820Z [cluster1] I1102 12:02:38.904210 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4675. 2020-11-02T12:06:50.7286471Z [cluster1] I1102 12:02:38.904219 1 route.go:793] Found route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4676. 2020-11-02T12:06:50.7287115Z [cluster1] I1102 12:02:38.904262 1 route.go:787] Processing route {Ifindex: 5 Dst: 240.0.0.0/8 Src: 240.17.0.9 Gw: <nil> Flags: [] Table: 254}
  4677. 2020-11-02T12:06:50.7287612Z [cluster1] I1102 12:02:38.904272 1 route.go:790] Found nil gw or dst
  4678. 2020-11-02T12:06:50.7288056Z [cluster1] I1102 12:02:38.904383 1 route.go:829] Found equivalent route, not adding
  4679. 2020-11-02T12:06:50.7288556Z [cluster1] I1102 12:02:38.904453 1 route.go:829] Found equivalent route, not adding
  4680. 2020-11-02T12:06:50.7289144Z [cluster1] I1102 12:02:38.904465 1 route.go:829] Found equivalent route, not adding
  4681. 2020-11-02T12:06:50.7289813Z [cluster1] I1102 12:02:38.904476 1 route.go:829] Found equivalent route, not adding
  4682. 2020-11-02T12:06:50.7291051Z [cluster1] I1102 12:02:38.904668 1 route.go:647] Successfully processed local Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6"
  4683. 2020-11-02T12:06:50.7293088Z [cluster1] I1102 12:02:38.904691 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  4684. 2020-11-02T12:06:50.7294922Z [cluster1] I1102 12:03:08.898316 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  4685. 2020-11-02T12:06:50.7298355Z [cluster1] I1102 12:03:08.898362 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  4686. 2020-11-02T12:06:50.7300996Z [cluster1] I1102 12:03:08.898372 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4687. 2020-11-02T12:06:50.7302912Z [cluster1] I1102 12:03:08.898379 1 route.go:612] This route agent is running on a non-gateway/non-active node
  4688. 2020-11-02T12:06:50.7303781Z [cluster1] I1102 12:03:08.898387 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.6
  4689. 2020-11-02T12:06:50.7304639Z [cluster1] I1102 12:03:08.899000 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  4690. 2020-11-02T12:06:50.7305399Z [cluster1] I1102 12:03:08.902979 1 route.go:771] Reconciling routes to gw: 240.17.0.6
  4691. 2020-11-02T12:06:50.7305965Z [cluster1] I1102 12:03:08.903250 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4692. 2020-11-02T12:06:50.7306958Z [cluster1] I1102 12:03:08.903287 1 route.go:793] Found route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4693. 2020-11-02T12:06:50.7333021Z [cluster1] I1102 12:03:08.903430 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4694. 2020-11-02T12:06:50.7333742Z [cluster1] I1102 12:03:08.903456 1 route.go:793] Found route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4695. 2020-11-02T12:06:50.7334445Z [cluster1] I1102 12:03:08.903466 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4696. 2020-11-02T12:06:50.7335323Z [cluster1] I1102 12:03:08.903565 1 route.go:793] Found route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4697. 2020-11-02T12:06:50.7336024Z [cluster1] I1102 12:03:08.903589 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4698. 2020-11-02T12:06:50.7336874Z [cluster1] I1102 12:03:08.903599 1 route.go:793] Found route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4699. 2020-11-02T12:06:50.7337535Z [cluster1] I1102 12:03:08.903606 1 route.go:787] Processing route {Ifindex: 5 Dst: 240.0.0.0/8 Src: 240.17.0.9 Gw: <nil> Flags: [] Table: 254}
  4700. 2020-11-02T12:06:50.7338033Z [cluster1] I1102 12:03:08.903614 1 route.go:790] Found nil gw or dst
  4701. 2020-11-02T12:06:50.7338481Z [cluster1] I1102 12:03:08.903783 1 route.go:829] Found equivalent route, not adding
  4702. 2020-11-02T12:06:50.7338988Z [cluster1] I1102 12:03:08.903807 1 route.go:829] Found equivalent route, not adding
  4703. 2020-11-02T12:06:50.7339470Z [cluster1] I1102 12:03:08.903817 1 route.go:829] Found equivalent route, not adding
  4704. 2020-11-02T12:06:50.7340122Z [cluster1] I1102 12:03:08.903824 1 route.go:829] Found equivalent route, not adding
  4705. 2020-11-02T12:06:50.7341550Z [cluster1] I1102 12:03:08.903861 1 route.go:647] Successfully processed local Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6"
  4706. 2020-11-02T12:06:50.7343309Z [cluster1] I1102 12:03:38.898517 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  4707. 2020-11-02T12:06:50.7345087Z [cluster1] I1102 12:03:38.898718 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4708. 2020-11-02T12:06:50.7346554Z [cluster1] I1102 12:03:38.898853 1 route.go:612] This route agent is running on a non-gateway/non-active node
  4709. 2020-11-02T12:06:50.7347395Z [cluster1] I1102 12:03:38.898875 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.6
  4710. 2020-11-02T12:06:50.7348052Z [cluster1] I1102 12:03:38.899267 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  4711. 2020-11-02T12:06:50.7348594Z [cluster1] I1102 12:03:38.903975 1 route.go:771] Reconciling routes to gw: 240.17.0.6
  4712. 2020-11-02T12:06:50.7349122Z [cluster1] I1102 12:03:38.904256 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4713. 2020-11-02T12:06:50.7349931Z [cluster1] I1102 12:03:38.904291 1 route.go:793] Found route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4714. 2020-11-02T12:06:50.7350581Z [cluster1] I1102 12:03:38.904301 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4715. 2020-11-02T12:06:50.7351233Z [cluster1] I1102 12:03:38.904497 1 route.go:793] Found route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4716. 2020-11-02T12:06:50.7351890Z [cluster1] I1102 12:03:38.904507 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4717. 2020-11-02T12:06:50.7352713Z [cluster1] I1102 12:03:38.904751 1 route.go:793] Found route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4718. 2020-11-02T12:06:50.7353541Z [cluster1] I1102 12:03:38.904764 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4719. 2020-11-02T12:06:50.7354189Z [cluster1] I1102 12:03:38.904773 1 route.go:793] Found route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4720. 2020-11-02T12:06:50.7354947Z [cluster1] I1102 12:03:38.904785 1 route.go:787] Processing route {Ifindex: 5 Dst: 240.0.0.0/8 Src: 240.17.0.9 Gw: <nil> Flags: [] Table: 254}
  4721. 2020-11-02T12:06:50.7355423Z [cluster1] I1102 12:03:38.904794 1 route.go:790] Found nil gw or dst
  4722. 2020-11-02T12:06:50.7356035Z [cluster1] I1102 12:03:38.904869 1 route.go:829] Found equivalent route, not adding
  4723. 2020-11-02T12:06:50.7356700Z [cluster1] I1102 12:03:38.904886 1 route.go:829] Found equivalent route, not adding
  4724. 2020-11-02T12:06:50.7357197Z [cluster1] I1102 12:03:38.904898 1 route.go:829] Found equivalent route, not adding
  4725. 2020-11-02T12:06:50.7357693Z [cluster1] I1102 12:03:38.904911 1 route.go:829] Found equivalent route, not adding
  4726. 2020-11-02T12:06:50.7358931Z [cluster1] I1102 12:03:38.904944 1 route.go:647] Successfully processed local Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6"
  4727. 2020-11-02T12:06:50.7360832Z [cluster1] I1102 12:03:38.904967 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  4728. 2020-11-02T12:06:50.7362844Z [cluster1] I1102 12:04:08.898857 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  4729. 2020-11-02T12:06:50.7392857Z [cluster1] I1102 12:04:08.898896 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  4730. 2020-11-02T12:06:50.7394925Z [cluster1] I1102 12:04:08.898905 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4731. 2020-11-02T12:06:50.7396475Z [cluster1] I1102 12:04:08.898927 1 route.go:612] This route agent is running on a non-gateway/non-active node
  4732. 2020-11-02T12:06:50.7397542Z [cluster1] I1102 12:04:08.898935 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.6
  4733. 2020-11-02T12:06:50.7398239Z [cluster1] I1102 12:04:08.899287 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  4734. 2020-11-02T12:06:50.7398973Z [cluster1] I1102 12:04:08.902975 1 route.go:771] Reconciling routes to gw: 240.17.0.6
  4735. 2020-11-02T12:06:50.7399527Z [cluster1] I1102 12:04:08.903288 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4736. 2020-11-02T12:06:50.7400168Z [cluster1] I1102 12:04:08.903317 1 route.go:793] Found route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4737. 2020-11-02T12:06:50.7400937Z [cluster1] I1102 12:04:08.903325 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4738. 2020-11-02T12:06:50.7401585Z [cluster1] I1102 12:04:08.903395 1 route.go:793] Found route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4739. 2020-11-02T12:06:50.7402241Z [cluster1] I1102 12:04:08.903414 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4740. 2020-11-02T12:06:50.7402901Z [cluster1] I1102 12:04:08.903422 1 route.go:793] Found route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4741. 2020-11-02T12:06:50.7403551Z [cluster1] I1102 12:04:08.903428 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4742. 2020-11-02T12:06:50.7404373Z [cluster1] I1102 12:04:08.903498 1 route.go:793] Found route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4743. 2020-11-02T12:06:50.7405024Z [cluster1] I1102 12:04:08.903506 1 route.go:787] Processing route {Ifindex: 5 Dst: 240.0.0.0/8 Src: 240.17.0.9 Gw: <nil> Flags: [] Table: 254}
  4744. 2020-11-02T12:06:50.7405699Z [cluster1] I1102 12:04:08.903512 1 route.go:790] Found nil gw or dst
  4745. 2020-11-02T12:06:50.7406142Z [cluster1] I1102 12:04:08.903691 1 route.go:829] Found equivalent route, not adding
  4746. 2020-11-02T12:06:50.7406694Z [cluster1] I1102 12:04:08.903710 1 route.go:829] Found equivalent route, not adding
  4747. 2020-11-02T12:06:50.7407191Z [cluster1] I1102 12:04:08.903720 1 route.go:829] Found equivalent route, not adding
  4748. 2020-11-02T12:06:50.7407686Z [cluster1] I1102 12:04:08.903727 1 route.go:829] Found equivalent route, not adding
  4749. 2020-11-02T12:06:50.7408960Z [cluster1] I1102 12:04:08.903820 1 route.go:647] Successfully processed local Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6"
  4750. 2020-11-02T12:06:50.7410910Z [cluster1] I1102 12:04:38.898923 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  4751. 2020-11-02T12:06:50.7412806Z [cluster1] I1102 12:04:38.898961 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4752. 2020-11-02T12:06:50.7414450Z [cluster1] I1102 12:04:38.898970 1 route.go:612] This route agent is running on a non-gateway/non-active node
  4753. 2020-11-02T12:06:50.7415438Z [cluster1] I1102 12:04:38.898978 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.6
  4754. 2020-11-02T12:06:50.7416108Z [cluster1] I1102 12:04:38.899230 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  4755. 2020-11-02T12:06:50.7416776Z [cluster1] I1102 12:04:38.902607 1 route.go:771] Reconciling routes to gw: 240.17.0.6
  4756. 2020-11-02T12:06:50.7417318Z [cluster1] I1102 12:04:38.902765 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4757. 2020-11-02T12:06:50.7417980Z [cluster1] I1102 12:04:38.902796 1 route.go:793] Found route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4758. 2020-11-02T12:06:50.7418628Z [cluster1] I1102 12:04:38.902812 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4759. 2020-11-02T12:06:50.7419281Z [cluster1] I1102 12:04:38.902820 1 route.go:793] Found route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4760. 2020-11-02T12:06:50.7419927Z [cluster1] I1102 12:04:38.902828 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4761. 2020-11-02T12:06:50.7420580Z [cluster1] I1102 12:04:38.902843 1 route.go:793] Found route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4762. 2020-11-02T12:06:50.7421226Z [cluster1] I1102 12:04:38.902850 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4763. 2020-11-02T12:06:50.7421877Z [cluster1] I1102 12:04:38.902858 1 route.go:793] Found route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4764. 2020-11-02T12:06:50.7422531Z [cluster1] I1102 12:04:38.902871 1 route.go:787] Processing route {Ifindex: 5 Dst: 240.0.0.0/8 Src: 240.17.0.9 Gw: <nil> Flags: [] Table: 254}
  4765. 2020-11-02T12:06:50.7423023Z [cluster1] I1102 12:04:38.902879 1 route.go:790] Found nil gw or dst
  4766. 2020-11-02T12:06:50.7423466Z [cluster1] I1102 12:04:38.902948 1 route.go:829] Found equivalent route, not adding
  4767. 2020-11-02T12:06:50.7423964Z [cluster1] I1102 12:04:38.902965 1 route.go:829] Found equivalent route, not adding
  4768. 2020-11-02T12:06:50.7424458Z [cluster1] I1102 12:04:38.902974 1 route.go:829] Found equivalent route, not adding
  4769. 2020-11-02T12:06:50.7424954Z [cluster1] I1102 12:04:38.902986 1 route.go:829] Found equivalent route, not adding
  4770. 2020-11-02T12:06:50.7426278Z [cluster1] I1102 12:04:38.903018 1 route.go:647] Successfully processed local Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6"
  4771. 2020-11-02T12:06:50.7428041Z [cluster1] I1102 12:04:38.903041 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  4772. 2020-11-02T12:06:50.7429848Z [cluster1] I1102 12:05:08.899158 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  4773. 2020-11-02T12:06:50.7431676Z [cluster1] I1102 12:05:08.899209 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4774. 2020-11-02T12:06:50.7432982Z [cluster1] I1102 12:05:08.899218 1 route.go:612] This route agent is running on a non-gateway/non-active node
  4775. 2020-11-02T12:06:50.7433850Z [cluster1] I1102 12:05:08.899251 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.6
  4776. 2020-11-02T12:06:50.7434515Z [cluster1] I1102 12:05:08.899595 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  4777. 2020-11-02T12:06:50.7435149Z [cluster1] I1102 12:05:08.903230 1 route.go:771] Reconciling routes to gw: 240.17.0.6
  4778. 2020-11-02T12:06:50.7435700Z [cluster1] I1102 12:05:08.903463 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4779. 2020-11-02T12:06:50.7436353Z [cluster1] I1102 12:05:08.903499 1 route.go:793] Found route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4780. 2020-11-02T12:06:50.7437001Z [cluster1] I1102 12:05:08.903508 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4781. 2020-11-02T12:06:50.7437803Z [cluster1] I1102 12:05:08.903516 1 route.go:793] Found route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4782. 2020-11-02T12:06:50.7438436Z [cluster1] I1102 12:05:08.903567 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4783. 2020-11-02T12:06:50.7439065Z [cluster1] I1102 12:05:08.903600 1 route.go:793] Found route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4784. 2020-11-02T12:06:50.7439694Z [cluster1] I1102 12:05:08.903643 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4785. 2020-11-02T12:06:50.7440320Z [cluster1] I1102 12:05:08.903657 1 route.go:793] Found route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4786. 2020-11-02T12:06:50.7440946Z [cluster1] I1102 12:05:08.903664 1 route.go:787] Processing route {Ifindex: 5 Dst: 240.0.0.0/8 Src: 240.17.0.9 Gw: <nil> Flags: [] Table: 254}
  4787. 2020-11-02T12:06:50.7441430Z [cluster1] I1102 12:05:08.903708 1 route.go:790] Found nil gw or dst
  4788. 2020-11-02T12:06:50.7441858Z [cluster1] I1102 12:05:08.903811 1 route.go:829] Found equivalent route, not adding
  4789. 2020-11-02T12:06:50.7442331Z [cluster1] I1102 12:05:08.903865 1 route.go:829] Found equivalent route, not adding
  4790. 2020-11-02T12:06:50.7442811Z [cluster1] I1102 12:05:08.903876 1 route.go:829] Found equivalent route, not adding
  4791. 2020-11-02T12:06:50.7443293Z [cluster1] I1102 12:05:08.903884 1 route.go:829] Found equivalent route, not adding
  4792. 2020-11-02T12:06:50.7461271Z [cluster1] I1102 12:05:08.903938 1 route.go:647] Successfully processed local Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6"
  4793. 2020-11-02T12:06:50.7463179Z [cluster1] I1102 12:05:08.903993 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  4794. 2020-11-02T12:06:50.7467484Z [cluster1] I1102 12:05:30.614880 1 route.go:682] Informed of removed endpoint: {"metadata":{"name":"cluster2-submariner-cable-cluster2-172-17-0-4","namespace":"submariner-operator","selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4","uid":"c09b80a0-c12c-4246-a9ea-692c9619737f","resourceVersion":"3454","generation":1,"creationTimestamp":"2020-11-02T12:01:40Z","labels":{"submariner-io/clusterID":"cluster2"}},"spec":{"cluster_id":"cluster2","cable_name":"submariner-cable-cluster2-172-17-0-4","hostname":"cluster2-worker","subnets":["100.2.0.0/16","10.2.0.0/16"],"private_ip":"172.17.0.4","public_ip":"","nat_enabled":false,"backend":"libreswan"}}
  4795. 2020-11-02T12:06:50.7471891Z [cluster1] I1102 12:05:30.716286 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11" with subnets [100.2.0.0/16 10.2.0.0/16]
  4796. 2020-11-02T12:06:50.7474285Z [cluster1] I1102 12:05:38.899441 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11" with subnets [100.2.0.0/16 10.2.0.0/16]
  4797. 2020-11-02T12:06:50.7476390Z [cluster1] I1102 12:05:38.899480 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4798. 2020-11-02T12:06:50.7477710Z [cluster1] I1102 12:05:38.899494 1 route.go:612] This route agent is running on a non-gateway/non-active node
  4799. 2020-11-02T12:06:50.7478547Z [cluster1] I1102 12:05:38.899503 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.6
  4800. 2020-11-02T12:06:50.7480444Z [cluster1] I1102 12:05:38.899858 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  4801. 2020-11-02T12:06:50.7481014Z [cluster1] I1102 12:05:38.903110 1 route.go:771] Reconciling routes to gw: 240.17.0.6
  4802. 2020-11-02T12:06:50.7481587Z [cluster1] I1102 12:05:38.903383 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4803. 2020-11-02T12:06:50.7482265Z [cluster1] I1102 12:05:38.903497 1 route.go:793] Found route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4804. 2020-11-02T12:06:50.7484346Z [cluster1] I1102 12:05:38.903568 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4805. 2020-11-02T12:06:50.7485002Z [cluster1] I1102 12:05:38.903721 1 route.go:793] Found route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4806. 2020-11-02T12:06:50.7485634Z [cluster1] I1102 12:05:38.903832 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4807. 2020-11-02T12:06:50.7486263Z [cluster1] I1102 12:05:38.903856 1 route.go:793] Found route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4808. 2020-11-02T12:06:50.7486898Z [cluster1] I1102 12:05:38.903865 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4809. 2020-11-02T12:06:50.7487532Z [cluster1] I1102 12:05:38.903872 1 route.go:793] Found route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4810. 2020-11-02T12:06:50.7488166Z [cluster1] I1102 12:05:38.903959 1 route.go:787] Processing route {Ifindex: 5 Dst: 240.0.0.0/8 Src: 240.17.0.9 Gw: <nil> Flags: [] Table: 254}
  4811. 2020-11-02T12:06:50.7488646Z [cluster1] I1102 12:05:38.903969 1 route.go:790] Found nil gw or dst
  4812. 2020-11-02T12:06:50.7489081Z [cluster1] I1102 12:05:38.904074 1 route.go:829] Found equivalent route, not adding
  4813. 2020-11-02T12:06:50.7489568Z [cluster1] I1102 12:05:38.904167 1 route.go:829] Found equivalent route, not adding
  4814. 2020-11-02T12:06:50.7490195Z [cluster1] I1102 12:05:38.904252 1 route.go:829] Found equivalent route, not adding
  4815. 2020-11-02T12:06:50.7491024Z [cluster1] I1102 12:05:38.904288 1 route.go:829] Found equivalent route, not adding
  4816. 2020-11-02T12:06:50.7492535Z [cluster1] I1102 12:05:38.904368 1 route.go:647] Successfully processed local Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6"
  4817. 2020-11-02T12:06:50.7494276Z [cluster1] I1102 12:05:38.904494 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  4818. 2020-11-02T12:06:50.7496471Z [cluster1] I1102 12:06:08.899594 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  4819. 2020-11-02T12:06:50.7498478Z [cluster1] I1102 12:06:08.899795 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11" with subnets [100.2.0.0/16 10.2.0.0/16]
  4820. 2020-11-02T12:06:50.7500281Z [cluster1] I1102 12:06:08.899888 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4821. 2020-11-02T12:06:50.7502251Z [cluster1] I1102 12:06:08.899915 1 route.go:612] This route agent is running on a non-gateway/non-active node
  4822. 2020-11-02T12:06:50.7503163Z [cluster1] I1102 12:06:08.899961 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.6
  4823. 2020-11-02T12:06:50.7503814Z [cluster1] I1102 12:06:08.900286 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  4824. 2020-11-02T12:06:50.7504537Z [cluster1] I1102 12:06:08.903671 1 route.go:771] Reconciling routes to gw: 240.17.0.6
  4825. 2020-11-02T12:06:50.7505063Z [cluster1] I1102 12:06:08.903957 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4826. 2020-11-02T12:06:50.7505757Z [cluster1] I1102 12:06:08.903992 1 route.go:793] Found route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4827. 2020-11-02T12:06:50.7506390Z [cluster1] I1102 12:06:08.904001 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4828. 2020-11-02T12:06:50.7507023Z [cluster1] I1102 12:06:08.904010 1 route.go:793] Found route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4829. 2020-11-02T12:06:50.7507652Z [cluster1] I1102 12:06:08.904110 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4830. 2020-11-02T12:06:50.7508280Z [cluster1] I1102 12:06:08.904121 1 route.go:793] Found route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4831. 2020-11-02T12:06:50.7508910Z [cluster1] I1102 12:06:08.904128 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4832. 2020-11-02T12:06:50.7509533Z [cluster1] I1102 12:06:08.904212 1 route.go:793] Found route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4833. 2020-11-02T12:06:50.7510161Z [cluster1] I1102 12:06:08.904270 1 route.go:787] Processing route {Ifindex: 5 Dst: 240.0.0.0/8 Src: 240.17.0.9 Gw: <nil> Flags: [] Table: 254}
  4834. 2020-11-02T12:06:50.7510640Z [cluster1] I1102 12:06:08.904336 1 route.go:790] Found nil gw or dst
  4835. 2020-11-02T12:06:50.7511070Z [cluster1] I1102 12:06:08.904522 1 route.go:829] Found equivalent route, not adding
  4836. 2020-11-02T12:06:50.7512208Z [cluster1] I1102 12:06:08.904575 1 route.go:829] Found equivalent route, not adding
  4837. 2020-11-02T12:06:50.7512837Z [cluster1] I1102 12:06:08.904601 1 route.go:829] Found equivalent route, not adding
  4838. 2020-11-02T12:06:50.7513453Z [cluster1] I1102 12:06:08.904622 1 route.go:829] Found equivalent route, not adding
  4839. 2020-11-02T12:06:50.7515308Z [cluster1] I1102 12:06:08.904777 1 route.go:647] Successfully processed local Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6"
  4840. 2020-11-02T12:06:50.7517315Z [cluster1] I1102 12:06:38.899677 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11" with subnets [100.2.0.0/16 10.2.0.0/16]
  4841. 2020-11-02T12:06:50.7520153Z [submariner]$ [cluster1] kubectl -n submariner-operator logs submariner-routeagent-c4hh4
  4842. 2020-11-02T12:06:50.7521639Z [submariner]$ [cluster1] command kubectl --context=cluster1 -n submariner-operator logs submariner-routeagent-c4hh4
  4843. 2020-11-02T12:06:50.7522884Z [submariner]$ [cluster1] kubectl --context=cluster1 -n submariner-operator logs submariner-routeagent-c4hh4
  4844. 2020-11-02T12:06:51.0102119Z [cluster1] I1102 12:06:38.899795 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4845. 2020-11-02T12:06:51.0104104Z [cluster1] I1102 12:06:38.899821 1 route.go:612] This route agent is running on a non-gateway/non-active node
  4846. 2020-11-02T12:06:51.0105649Z [cluster1] I1102 12:06:38.899865 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.6
  4847. 2020-11-02T12:06:51.0106705Z [cluster1] I1102 12:06:38.900086 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  4848. 2020-11-02T12:06:51.0107435Z [cluster1] I1102 12:06:38.903606 1 route.go:771] Reconciling routes to gw: 240.17.0.6
  4849. 2020-11-02T12:06:51.0108076Z [cluster1] I1102 12:06:38.904002 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4850. 2020-11-02T12:06:51.0108885Z [cluster1] I1102 12:06:38.904098 1 route.go:793] Found route {Ifindex: 5 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4851. 2020-11-02T12:06:51.0111138Z [cluster1] I1102 12:06:38.904108 1 route.go:787] Processing route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4852. 2020-11-02T12:06:51.0112134Z [cluster1] I1102 12:06:38.904133 1 route.go:793] Found route {Ifindex: 5 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4853. 2020-11-02T12:06:51.0112904Z [cluster1] I1102 12:06:38.904172 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4854. 2020-11-02T12:06:51.0113652Z [cluster1] I1102 12:06:38.904505 1 route.go:793] Found route {Ifindex: 5 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4855. 2020-11-02T12:06:51.0114803Z [cluster1] I1102 12:06:38.904531 1 route.go:787] Processing route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254}
  4856. 2020-11-02T12:06:51.0115710Z [cluster1] I1102 12:06:38.904541 1 route.go:793] Found route {Ifindex: 5 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.6 Flags: [] Table: 254} with gw 240.17.0.6 already installed
  4857. 2020-11-02T12:06:51.0116557Z [cluster1] I1102 12:06:38.904548 1 route.go:787] Processing route {Ifindex: 5 Dst: 240.0.0.0/8 Src: 240.17.0.9 Gw: <nil> Flags: [] Table: 254}
  4858. 2020-11-02T12:06:51.0117180Z [cluster1] I1102 12:06:38.904555 1 route.go:790] Found nil gw or dst
  4859. 2020-11-02T12:06:51.0117618Z [cluster1] I1102 12:06:38.904649 1 route.go:829] Found equivalent route, not adding
  4860. 2020-11-02T12:06:51.0118107Z [cluster1] I1102 12:06:38.904848 1 route.go:829] Found equivalent route, not adding
  4861. 2020-11-02T12:06:51.0118588Z [cluster1] I1102 12:06:38.904936 1 route.go:829] Found equivalent route, not adding
  4862. 2020-11-02T12:06:51.0119232Z [cluster1] I1102 12:06:38.904949 1 route.go:829] Found equivalent route, not adding
  4863. 2020-11-02T12:06:51.0120497Z [cluster1] I1102 12:06:38.905009 1 route.go:647] Successfully processed local Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6"
  4864. 2020-11-02T12:06:51.0122389Z [cluster1] I1102 12:06:38.905157 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  4865. 2020-11-02T12:06:51.0123769Z [cluster1] +++++++++++++++++++++: Logs for Pod submariner-routeagent-c4hh4 in namespace submariner-operator :++++++++++++++++++++++
  4866. 2020-11-02T12:06:51.0124755Z [cluster1] + trap 'exit 1' SIGTERM SIGINT
  4867. 2020-11-02T12:06:51.0125096Z [cluster1] + SUBMARINER_VERBOSITY=2
  4868. 2020-11-02T12:06:51.0125535Z [cluster1] + '[' false == true ']'
  4869. 2020-11-02T12:06:51.0125907Z [cluster1] + DEBUG=-v=2
  4870. 2020-11-02T12:06:51.0126354Z [cluster1] + for f in iptables-save iptables
  4871. 2020-11-02T12:06:51.0126885Z [cluster1] ++ find_iptables_on_host iptables-save
  4872. 2020-11-02T12:06:51.0127443Z [cluster1] ++ chroot /host test -x /usr/sbin/iptables-save
  4873. 2020-11-02T12:06:51.0128103Z [cluster1] iptables-save is present on the host at /usr/sbin/iptables-save
  4874. 2020-11-02T12:06:51.0128500Z [cluster1] ++ echo /usr/sbin
  4875. 2020-11-02T12:06:51.0128761Z [cluster1] ++ return
  4876. 2020-11-02T12:06:51.0129035Z [cluster1] + location=/usr/sbin
  4877. 2020-11-02T12:06:51.0129474Z [cluster1] + '[' /usr/sbin '!=' unknown ']'
  4878. 2020-11-02T12:06:51.0130178Z [cluster1] + echo 'iptables-save is present on the host at /usr/sbin/iptables-save'
  4879. 2020-11-02T12:06:51.0130917Z [cluster1] + sed 's!@@PATH@@!/usr/sbin!' /usr/sbin/iptables-wrapper.in
  4880. 2020-11-02T12:06:51.0131491Z [cluster1] + for f in iptables-save iptables
  4881. 2020-11-02T12:06:51.0131870Z [cluster1] ++ find_iptables_on_host iptables
  4882. 2020-11-02T12:06:51.0132354Z [cluster1] ++ chroot /host test -x /usr/sbin/iptables
  4883. 2020-11-02T12:06:51.0132683Z [cluster1] ++ echo /usr/sbin
  4884. 2020-11-02T12:06:51.0132944Z [cluster1] ++ return
  4885. 2020-11-02T12:06:51.0133219Z [cluster1] + location=/usr/sbin
  4886. 2020-11-02T12:06:51.0133657Z [cluster1] + '[' /usr/sbin '!=' unknown ']'
  4887. 2020-11-02T12:06:51.0134203Z [cluster1] + echo 'iptables is present on the host at /usr/sbin/iptables'
  4888. 2020-11-02T12:06:51.0134840Z [cluster1] + sed 's!@@PATH@@!/usr/sbin!' /usr/sbin/iptables-wrapper.in
  4889. 2020-11-02T12:06:51.0135542Z [cluster1] + exec submariner-route-agent -v=2 -alsologtostderr
  4890. 2020-11-02T12:06:51.0136738Z [cluster1] iptables is present on the host at /usr/sbin/iptables
  4891. 2020-11-02T12:06:51.0137808Z [cluster1] I1102 11:58:22.272143 1 main.go:47] Starting submariner-route-agent
  4892. 2020-11-02T12:06:51.0138982Z [cluster1] W1102 11:58:22.272673 1 client_config.go:543] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
  4893. 2020-11-02T12:06:51.0140066Z [cluster1] I1102 11:58:22.277167 1 route.go:192] Starting Route Controller. ClusterID: cluster1, localClusterCIDR: [10.1.0.0/16], localServiceCIDR: [100.1.0.0/16]
  4894. 2020-11-02T12:06:51.0141256Z [cluster1] I1102 11:58:22.277281 1 route.go:196] Waiting for Endpoint informer caches to sync.
  4895. 2020-11-02T12:06:51.0142138Z [cluster1] I1102 11:58:22.309540 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.6
  4896. 2020-11-02T12:06:51.0143052Z [cluster1] I1102 11:58:22.309688 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.9
  4897. 2020-11-02T12:06:51.0143683Z [cluster1] I1102 11:58:22.377829 1 driver.go:37] Interface "lo" has "127.0.0.1" address
  4898. 2020-11-02T12:06:51.0144259Z [cluster1] I1102 11:58:22.378065 1 driver.go:37] Interface "vethf136f981" has "10.1.1.1" address
  4899. 2020-11-02T12:06:51.0145320Z [cluster1] I1102 11:58:22.378168 1 driver.go:42] Found CNI Interface "vethf136f981" that has IP "10.1.1.1" from ClusterCIDR "10.1.0.0/16"
  4900. 2020-11-02T12:06:51.0146160Z [cluster1] I1102 11:58:22.378450 1 driver.go:60] Successfully configured rp_filter to loose mode(2) on cniInterface "vethf136f981"
  4901. 2020-11-02T12:06:51.0147272Z [cluster1] I1102 11:58:22.385297 1 iptables.go:22] Install/ensure SUBMARINER-POSTROUTING chain exists
  4902. 2020-11-02T12:06:51.0148472Z [cluster1] I1102 11:58:22.408892 1 iptables.go:28] Insert SUBMARINER-POSTROUTING rule that has rules for inter-cluster traffic
  4903. 2020-11-02T12:06:51.0149775Z [cluster1] I1102 11:58:22.420241 1 iptables.go:34] Install/ensure SUBMARINER-INPUT chain exists
  4904. 2020-11-02T12:06:51.0150799Z [cluster1] I1102 11:58:22.450178 1 iptables.go:45] Allow VxLAN incoming traffic in SUBMARINER-INPUT Chain
  4905. 2020-11-02T12:06:51.0151909Z [cluster1] I1102 11:58:22.461299 1 iptables.go:53] Insert rule to allow traffic over vx-submariner interface in FORWARDing Chain
  4906. 2020-11-02T12:06:51.0153161Z [cluster1] I1102 11:58:22.473545 1 iptables.go:65] Installing rule for host network to remote cluster communication: -s 240.0.0.0/8 -o vx-submariner -j SNAT --to 10.1.1.1
  4907. 2020-11-02T12:06:51.0154336Z [cluster1] I1102 11:58:22.494972 1 route.go:254] In cluster1, podIP of submariner-route-agent[0] is 172.17.0.9
  4908. 2020-11-02T12:06:51.0155394Z [cluster1] I1102 11:58:22.495112 1 route.go:254] In cluster1, podIP of submariner-route-agent[1] is 172.17.0.6
  4909. 2020-11-02T12:06:51.0156551Z [cluster1] I1102 11:58:22.495185 1 route.go:268] Hostname is "cluster1-worker" and routeAgentNodeName is "cluster1-worker"
  4910. 2020-11-02T12:06:51.0157696Z [cluster1] I1102 11:58:22.502663 1 driver.go:87] Successfully annotated node "cluster1-worker" with cniIfaceIP "10.1.1.1"
  4911. 2020-11-02T12:06:51.0158502Z [cluster1] I1102 11:58:22.502935 1 route.go:286] Route agent workers started
  4912. 2020-11-02T12:06:51.0159754Z [cluster1] I1102 11:58:22.503513 1 route.go:507] Processing submariner-route-agent pod "submariner-routeagent-c4hh4" with host IP "172.17.0.6", pod IP "172.17.0.6"
  4913. 2020-11-02T12:06:51.0161506Z [cluster1] I1102 11:58:22.503631 1 route.go:531] Successfully processed submariner-route-agent pod "submariner-routeagent-c4hh4"
  4914. 2020-11-02T12:06:51.0163271Z [cluster1] I1102 11:58:22.503714 1 route.go:507] Processing submariner-route-agent pod "submariner-routeagent-5vp7b" with host IP "172.17.0.9", pod IP "172.17.0.9"
  4915. 2020-11-02T12:06:51.0172296Z [cluster1] I1102 11:58:22.503820 1 route.go:531] Successfully processed submariner-route-agent pod "submariner-routeagent-5vp7b"
  4916. 2020-11-02T12:06:51.0174358Z [cluster1] I1102 11:58:27.348759 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  4917. 2020-11-02T12:06:51.0176334Z [cluster1] I1102 11:58:27.369222 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.1.0.0/16 -d 100.2.0.0/16 -j ACCEPT
  4918. 2020-11-02T12:06:51.0177438Z [cluster1] I1102 11:58:27.389035 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 100.2.0.0/16 -d 10.1.0.0/16 -j ACCEPT
  4919. 2020-11-02T12:06:51.0178658Z [cluster1] I1102 11:58:27.412693 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.1.0.0/16 -d 10.2.0.0/16 -j ACCEPT
  4920. 2020-11-02T12:06:51.0179709Z [cluster1] I1102 11:58:27.427217 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 10.2.0.0/16 -d 10.1.0.0/16 -j ACCEPT
  4921. 2020-11-02T12:06:51.0181278Z [cluster1] I1102 11:58:27.542930 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4922. 2020-11-02T12:06:51.0182523Z [cluster1] I1102 11:58:27.544197 1 route.go:756] Processing route {Ifindex: 12 Dst: <nil> Src: <nil> Gw: 172.17.0.1 Flags: [] Table: 254}
  4923. 2020-11-02T12:06:51.0193418Z [cluster1] I1102 11:58:27.545161 1 route.go:759] Found nil gw or dst
  4924. 2020-11-02T12:06:51.0194218Z [cluster1] I1102 11:58:27.545310 1 route.go:756] Processing route {Ifindex: 12 Dst: 10.1.0.0/24 Src: <nil> Gw: 172.17.0.9 Flags: [] Table: 254}
  4925. 2020-11-02T12:06:51.0195431Z [cluster1] I1102 11:58:27.545421 1 route.go:756] Processing route {Ifindex: 2 Dst: 10.1.1.2/32 Src: <nil> Gw: <nil> Flags: [] Table: 254}
  4926. 2020-11-02T12:06:51.0196036Z [cluster1] I1102 11:58:27.545720 1 route.go:759] Found nil gw or dst
  4927. 2020-11-02T12:06:51.0196868Z [cluster1] I1102 11:58:27.545847 1 route.go:756] Processing route {Ifindex: 3 Dst: 10.1.1.3/32 Src: <nil> Gw: <nil> Flags: [] Table: 254}
  4928. 2020-11-02T12:06:51.0197466Z [cluster1] I1102 11:58:27.545978 1 route.go:759] Found nil gw or dst
  4929. 2020-11-02T12:06:51.0198064Z [cluster1] I1102 11:58:27.546141 1 route.go:756] Processing route {Ifindex: 4 Dst: 10.1.1.4/32 Src: <nil> Gw: <nil> Flags: [] Table: 254}
  4930. 2020-11-02T12:06:51.0198972Z [cluster1] I1102 11:58:27.546244 1 route.go:759] Found nil gw or dst
  4931. 2020-11-02T12:06:51.0199540Z [cluster1] I1102 11:58:27.546351 1 route.go:756] Processing route {Ifindex: 5 Dst: 10.1.1.5/32 Src: <nil> Gw: <nil> Flags: [] Table: 254}
  4932. 2020-11-02T12:06:51.0200132Z [cluster1] I1102 11:58:27.546459 1 route.go:759] Found nil gw or dst
  4933. 2020-11-02T12:06:51.0200707Z [cluster1] I1102 11:58:27.546621 1 route.go:756] Processing route {Ifindex: 12 Dst: 172.17.0.0/16 Src: 172.17.0.6 Gw: <nil> Flags: [] Table: 254}
  4934. 2020-11-02T12:06:51.0201266Z [cluster1] I1102 11:58:27.546885 1 route.go:759] Found nil gw or dst
  4935. 2020-11-02T12:06:51.0201816Z [cluster1] I1102 11:58:27.547030 1 route.go:582] This route agent is running on the active gateway node
  4936. 2020-11-02T12:06:51.0203612Z [cluster1] I1102 11:58:27.547238 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  4937. 2020-11-02T12:06:51.0205183Z [cluster1] I1102 11:58:27.552894 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.9 00:00:00:00:00:00
  4938. 2020-11-02T12:06:51.0205875Z [cluster1] I1102 11:58:27.553450 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.6 00:00:00:00:00:00
  4939. 2020-11-02T12:06:51.0206873Z [cluster1] I1102 11:58:27.553672 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  4940. 2020-11-02T12:06:51.0208702Z [cluster1] I1102 11:58:27.942107 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  4941. 2020-11-02T12:06:51.0210155Z [cluster1] I1102 11:58:27.955472 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.1.0.0/16 -d 100.3.0.0/16 -j ACCEPT
  4942. 2020-11-02T12:06:51.0211177Z [cluster1] I1102 11:58:27.989703 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 100.3.0.0/16 -d 10.1.0.0/16 -j ACCEPT
  4943. 2020-11-02T12:06:51.0212409Z [cluster1] I1102 11:58:28.052446 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.1.0.0/16 -d 10.3.0.0/16 -j ACCEPT
  4944. 2020-11-02T12:06:51.0214853Z [cluster1] I1102 11:58:28.081708 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 10.3.0.0/16 -d 10.1.0.0/16 -j ACCEPT
  4945. 2020-11-02T12:06:51.0216244Z [submariner]$ [cluster1] subctl show all
  4946. 2020-11-02T12:06:51.0218099Z [cluster1] I1102 11:58:52.295265 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  4947. 2020-11-02T12:06:51.0220690Z [cluster1] I1102 11:58:52.295313 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  4948. 2020-11-02T12:06:51.0223083Z [cluster1] I1102 11:58:52.295321 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4949. 2020-11-02T12:06:51.0225566Z [cluster1] I1102 11:58:52.295544 1 route.go:756] Processing route {Ifindex: 6 Dst: 240.0.0.0/8 Src: 240.17.0.6 Gw: <nil> Flags: [] Table: 254}
  4950. 2020-11-02T12:06:51.0226257Z [cluster1] I1102 11:58:52.295565 1 route.go:759] Found nil gw or dst
  4951. 2020-11-02T12:06:51.0226845Z [cluster1] I1102 11:58:52.295571 1 route.go:582] This route agent is running on the active gateway node
  4952. 2020-11-02T12:06:51.0229984Z [cluster1] I1102 11:58:52.295577 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  4953. 2020-11-02T12:06:51.0230983Z [cluster1] I1102 11:58:52.295699 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  4954. 2020-11-02T12:06:51.0231875Z [cluster1] I1102 11:58:52.295733 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.6 00:00:00:00:00:00
  4955. 2020-11-02T12:06:51.0232568Z [cluster1] I1102 11:58:52.295780 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.9 00:00:00:00:00:00
  4956. 2020-11-02T12:06:51.0233593Z [cluster1] I1102 11:58:52.295840 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  4957. 2020-11-02T12:06:51.0239984Z [cluster1] I1102 11:59:22.295765 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  4958. 2020-11-02T12:06:51.0242228Z [cluster1] I1102 11:59:22.295817 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4959. 2020-11-02T12:06:51.0243829Z [cluster1] I1102 11:59:22.296142 1 route.go:756] Processing route {Ifindex: 6 Dst: 240.0.0.0/8 Src: 240.17.0.6 Gw: <nil> Flags: [] Table: 254}
  4960. 2020-11-02T12:06:51.0245864Z [cluster1] I1102 11:59:22.296164 1 route.go:759] Found nil gw or dst
  4961. 2020-11-02T12:06:51.0247391Z [cluster1] I1102 11:59:22.296172 1 route.go:582] This route agent is running on the active gateway node
  4962. 2020-11-02T12:06:51.0289137Z [cluster1] I1102 11:59:22.296179 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  4963. 2020-11-02T12:06:51.0290060Z [cluster1] I1102 11:59:22.296352 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  4964. 2020-11-02T12:06:51.0290661Z [cluster1] I1102 11:59:22.296451 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.9 00:00:00:00:00:00
  4965. 2020-11-02T12:06:51.0291206Z [cluster1] I1102 11:59:22.296487 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.6 00:00:00:00:00:00
  4966. 2020-11-02T12:06:51.0292735Z [cluster1] I1102 11:59:22.296578 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  4967. 2020-11-02T12:06:51.0294447Z [cluster1] I1102 11:59:22.296660 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  4968. 2020-11-02T12:06:51.0296929Z [cluster1] I1102 11:59:52.295959 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  4969. 2020-11-02T12:06:51.0299432Z [cluster1] I1102 11:59:52.295994 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4970. 2020-11-02T12:06:51.0300692Z [cluster1] I1102 11:59:52.296180 1 route.go:756] Processing route {Ifindex: 6 Dst: 240.0.0.0/8 Src: 240.17.0.6 Gw: <nil> Flags: [] Table: 254}
  4971. 2020-11-02T12:06:51.0301178Z [cluster1] I1102 11:59:52.296215 1 route.go:759] Found nil gw or dst
  4972. 2020-11-02T12:06:51.0301649Z [cluster1] I1102 11:59:52.296221 1 route.go:582] This route agent is running on the active gateway node
  4973. 2020-11-02T12:06:51.0302676Z [cluster1] I1102 11:59:52.296227 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  4974. 2020-11-02T12:06:51.0303347Z [cluster1] I1102 11:59:52.296338 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  4975. 2020-11-02T12:06:51.0303954Z [cluster1] I1102 11:59:52.296379 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.9 00:00:00:00:00:00
  4976. 2020-11-02T12:06:51.0304690Z [cluster1] I1102 11:59:52.297143 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.6 00:00:00:00:00:00
  4977. 2020-11-02T12:06:51.0305691Z [cluster1] I1102 11:59:52.297220 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  4978. 2020-11-02T12:06:51.0307081Z [cluster1] I1102 11:59:52.297276 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  4979. 2020-11-02T12:06:51.0309234Z [cluster1] I1102 12:00:22.296167 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  4980. 2020-11-02T12:06:51.0311019Z [cluster1] I1102 12:00:22.296210 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4981. 2020-11-02T12:06:51.0312292Z [cluster1] I1102 12:00:22.296518 1 route.go:756] Processing route {Ifindex: 6 Dst: 240.0.0.0/8 Src: 240.17.0.6 Gw: <nil> Flags: [] Table: 254}
  4982. 2020-11-02T12:06:51.0312792Z [cluster1] I1102 12:00:22.296547 1 route.go:759] Found nil gw or dst
  4983. 2020-11-02T12:06:51.0313267Z [cluster1] I1102 12:00:22.296555 1 route.go:582] This route agent is running on the active gateway node
  4984. 2020-11-02T12:06:51.0315783Z [cluster1] I1102 12:00:22.296562 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  4985. 2020-11-02T12:06:51.0316558Z [cluster1] I1102 12:00:22.296707 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  4986. 2020-11-02T12:06:51.0317228Z [cluster1] I1102 12:00:22.296744 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.9 00:00:00:00:00:00
  4987. 2020-11-02T12:06:51.0318436Z [cluster1] I1102 12:00:22.296775 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.6 00:00:00:00:00:00
  4988. 2020-11-02T12:06:51.0319317Z [cluster1] I1102 12:00:22.296829 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  4989. 2020-11-02T12:06:51.0320661Z [cluster1] I1102 12:00:22.296891 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  4990. 2020-11-02T12:06:51.0322403Z [cluster1] I1102 12:00:52.296270 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  4991. 2020-11-02T12:06:51.0326022Z [cluster1] I1102 12:00:52.296310 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  4992. 2020-11-02T12:06:51.0327175Z [cluster1] I1102 12:00:52.296584 1 route.go:756] Processing route {Ifindex: 6 Dst: 240.0.0.0/8 Src: 240.17.0.6 Gw: <nil> Flags: [] Table: 254}
  4993. 2020-11-02T12:06:51.0327835Z [cluster1] I1102 12:00:52.296614 1 route.go:759] Found nil gw or dst
  4994. 2020-11-02T12:06:51.0328297Z [cluster1] I1102 12:00:52.296621 1 route.go:582] This route agent is running on the active gateway node
  4995. 2020-11-02T12:06:51.0329180Z [cluster1] I1102 12:00:52.296638 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  4996. 2020-11-02T12:06:51.0329825Z [cluster1] I1102 12:00:52.296771 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  4997. 2020-11-02T12:06:51.0330419Z [cluster1] I1102 12:00:52.296845 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.9 00:00:00:00:00:00
  4998. 2020-11-02T12:06:51.0330962Z [cluster1] I1102 12:00:52.296883 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.6 00:00:00:00:00:00
  4999. 2020-11-02T12:06:51.0331774Z [cluster1] I1102 12:00:52.296926 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  5000. 2020-11-02T12:06:51.0333313Z [cluster1] I1102 12:00:52.296987 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  5001. 2020-11-02T12:06:51.0336899Z [cluster1] I1102 12:01:22.296363 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  5002. 2020-11-02T12:06:51.0339486Z [cluster1] I1102 12:01:22.296495 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  5003. 2020-11-02T12:06:51.0341067Z [cluster1] I1102 12:01:22.296698 1 route.go:756] Processing route {Ifindex: 6 Dst: 240.0.0.0/8 Src: 240.17.0.6 Gw: <nil> Flags: [] Table: 254}
  5004. 2020-11-02T12:06:51.0341548Z [cluster1] I1102 12:01:22.296727 1 route.go:759] Found nil gw or dst
  5005. 2020-11-02T12:06:51.0342021Z [cluster1] I1102 12:01:22.296734 1 route.go:582] This route agent is running on the active gateway node
  5006. 2020-11-02T12:06:51.0342862Z [cluster1] I1102 12:01:22.296740 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  5007. 2020-11-02T12:06:51.0343506Z [cluster1] I1102 12:01:22.296914 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  5008. 2020-11-02T12:06:51.0344096Z [cluster1] I1102 12:01:22.296959 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.9 00:00:00:00:00:00
  5009. 2020-11-02T12:06:51.0344819Z [cluster1] I1102 12:01:22.296994 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.6 00:00:00:00:00:00
  5010. 2020-11-02T12:06:51.0345650Z [cluster1] I1102 12:01:22.297063 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  5011. 2020-11-02T12:06:51.0347112Z [cluster1] I1102 12:01:22.297124 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  5012. 2020-11-02T12:06:51.0351373Z [cluster1] I1102 12:01:40.303169 1 route.go:682] Informed of removed endpoint: {"metadata":{"name":"cluster2-submariner-cable-cluster2-172-17-0-4","namespace":"submariner-operator","selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4","uid":"78218d29-51fa-4e18-9649-07670fababa0","resourceVersion":"2628","generation":1,"creationTimestamp":"2020-11-02T11:58:27Z","labels":{"submariner-io/clusterID":"cluster2"}},"spec":{"cluster_id":"cluster2","cable_name":"submariner-cable-cluster2-172-17-0-4","hostname":"cluster2-worker","subnets":["100.2.0.0/16","10.2.0.0/16"],"private_ip":"172.17.0.4","public_ip":"","nat_enabled":false,"backend":"libreswan"}}
  5013. 2020-11-02T12:06:51.0361720Z [cluster1] I1102 12:01:40.364898 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  5014. 2020-11-02T12:06:51.0363840Z [cluster1] I1102 12:01:52.296573 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  5015. 2020-11-02T12:06:51.0366063Z [cluster1] I1102 12:01:52.296615 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  5016. 2020-11-02T12:06:51.0367188Z [cluster1] I1102 12:01:52.296849 1 route.go:756] Processing route {Ifindex: 6 Dst: 240.0.0.0/8 Src: 240.17.0.6 Gw: <nil> Flags: [] Table: 254}
  5017. 2020-11-02T12:06:51.0367851Z [cluster1] I1102 12:01:52.296891 1 route.go:759] Found nil gw or dst
  5018. 2020-11-02T12:06:51.0368317Z [cluster1] I1102 12:01:52.296898 1 route.go:582] This route agent is running on the active gateway node
  5019. 2020-11-02T12:06:51.0369126Z [cluster1] I1102 12:01:52.296905 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  5020. 2020-11-02T12:06:51.0370426Z [cluster1] I1102 12:01:52.297037 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  5021. 2020-11-02T12:06:51.0371394Z [cluster1] I1102 12:01:52.297082 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.9 00:00:00:00:00:00
  5022. 2020-11-02T12:06:51.0371978Z [cluster1] I1102 12:01:52.297112 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.6 00:00:00:00:00:00
  5023. 2020-11-02T12:06:51.0372893Z [cluster1] I1102 12:01:52.297164 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  5024. 2020-11-02T12:06:51.0374716Z [cluster1] I1102 12:01:52.297227 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  5025. 2020-11-02T12:06:51.0378361Z [cluster1] I1102 12:02:22.296657 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  5026. 2020-11-02T12:06:51.0380650Z [cluster1] I1102 12:02:22.296696 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  5027. 2020-11-02T12:06:51.0383134Z [cluster1] I1102 12:02:22.296985 1 route.go:756] Processing route {Ifindex: 6 Dst: 240.0.0.0/8 Src: 240.17.0.6 Gw: <nil> Flags: [] Table: 254}
  5028. 2020-11-02T12:06:51.0384181Z [cluster1] I1102 12:02:22.297040 1 route.go:759] Found nil gw or dst
  5029. 2020-11-02T12:06:51.0384806Z [cluster1] I1102 12:02:22.297047 1 route.go:582] This route agent is running on the active gateway node
  5030. 2020-11-02T12:06:51.0385762Z [cluster1] I1102 12:02:22.297054 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  5031. 2020-11-02T12:06:51.0387600Z [cluster1] I1102 12:02:22.297220 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  5032. 2020-11-02T12:06:51.0388937Z [cluster1] I1102 12:02:22.297267 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.9 00:00:00:00:00:00
  5033. 2020-11-02T12:06:51.0389528Z [cluster1] I1102 12:02:22.297309 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.6 00:00:00:00:00:00
  5034. 2020-11-02T12:06:51.0390446Z [cluster1] I1102 12:02:22.297367 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  5035. 2020-11-02T12:06:51.0391791Z [cluster1] I1102 12:02:22.297429 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  5036. 2020-11-02T12:06:51.0404069Z [cluster1] I1102 12:02:52.296906 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  5037. 2020-11-02T12:06:51.0406400Z [cluster1] I1102 12:02:52.296962 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  5038. 2020-11-02T12:06:51.0409180Z [cluster1] I1102 12:02:52.296972 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  5039. 2020-11-02T12:06:51.0412514Z [cluster1] I1102 12:02:52.297197 1 route.go:756] Processing route {Ifindex: 6 Dst: 240.0.0.0/8 Src: 240.17.0.6 Gw: <nil> Flags: [] Table: 254}
  5040. 2020-11-02T12:06:51.0413031Z [cluster1] I1102 12:02:52.297218 1 route.go:759] Found nil gw or dst
  5041. 2020-11-02T12:06:51.0413666Z [cluster1] I1102 12:02:52.297225 1 route.go:582] This route agent is running on the active gateway node
  5042. 2020-11-02T12:06:51.0417082Z [cluster1] I1102 12:02:52.297231 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  5043. 2020-11-02T12:06:51.0417800Z [cluster1] I1102 12:02:52.297351 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  5044. 2020-11-02T12:06:51.0418442Z [cluster1] I1102 12:02:52.297386 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.9 00:00:00:00:00:00
  5045. 2020-11-02T12:06:51.0419024Z [cluster1] I1102 12:02:52.297414 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.6 00:00:00:00:00:00
  5046. 2020-11-02T12:06:51.0419950Z [cluster1] I1102 12:02:52.297472 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  5047. 2020-11-02T12:06:51.0421967Z [cluster1] I1102 12:03:22.297092 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  5048. 2020-11-02T12:06:51.0425570Z [cluster1] I1102 12:03:22.297130 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  5049. 2020-11-02T12:06:51.0427452Z [cluster1] I1102 12:03:22.297317 1 route.go:756] Processing route {Ifindex: 6 Dst: 240.0.0.0/8 Src: 240.17.0.6 Gw: <nil> Flags: [] Table: 254}
  5050. 2020-11-02T12:06:51.0427952Z [cluster1] I1102 12:03:22.297346 1 route.go:759] Found nil gw or dst
  5051. 2020-11-02T12:06:51.0428419Z [cluster1] I1102 12:03:22.297353 1 route.go:582] This route agent is running on the active gateway node
  5052. 2020-11-02T12:06:51.0429227Z [cluster1] I1102 12:03:22.297360 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  5053. 2020-11-02T12:06:51.0430021Z [cluster1] I1102 12:03:22.297481 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  5054. 2020-11-02T12:06:51.0431365Z [cluster1] I1102 12:03:22.297524 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.9 00:00:00:00:00:00
  5055. 2020-11-02T12:06:51.0432246Z [cluster1] I1102 12:03:22.297560 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.6 00:00:00:00:00:00
  5056. 2020-11-02T12:06:51.0433143Z [cluster1] I1102 12:03:22.297614 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  5057. 2020-11-02T12:06:51.0434620Z [cluster1] I1102 12:03:22.297687 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  5058. 2020-11-02T12:06:51.0436792Z [cluster1] I1102 12:03:52.297364 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  5059. 2020-11-02T12:06:51.0437871Z [cluster1] I1102 12:03:52.297603 1 route.go:756] Processing route {Ifindex: 6 Dst: 240.0.0.0/8 Src: 240.17.0.6 Gw: <nil> Flags: [] Table: 254}
  5060. 2020-11-02T12:06:51.0438363Z [cluster1] I1102 12:03:52.297624 1 route.go:759] Found nil gw or dst
  5061. 2020-11-02T12:06:51.0439084Z [cluster1] I1102 12:03:52.297631 1 route.go:582] This route agent is running on the active gateway node
  5062. 2020-11-02T12:06:51.0440051Z [cluster1] I1102 12:03:52.297637 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  5063. 2020-11-02T12:06:51.0440700Z [cluster1] I1102 12:03:52.297759 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  5064. 2020-11-02T12:06:51.0441288Z [cluster1] I1102 12:03:52.297793 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.9 00:00:00:00:00:00
  5065. 2020-11-02T12:06:51.0443035Z [cluster1] I1102 12:03:52.297821 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.6 00:00:00:00:00:00
  5066. 2020-11-02T12:06:51.0444523Z [cluster1] I1102 12:03:52.297878 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  5067. 2020-11-02T12:06:51.0446416Z [cluster1] I1102 12:03:52.297936 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  5068. 2020-11-02T12:06:51.0448245Z [cluster1] I1102 12:03:52.297951 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  5069. 2020-11-02T12:06:51.0452016Z [cluster1] I1102 12:04:22.297650 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  5070. 2020-11-02T12:06:51.0457480Z [cluster1] I1102 12:04:22.297695 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  5071. 2020-11-02T12:06:51.0460315Z [cluster1] I1102 12:04:22.297704 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  5072. 2020-11-02T12:06:51.0462254Z [cluster1] I1102 12:04:22.297931 1 route.go:756] Processing route {Ifindex: 6 Dst: 240.0.0.0/8 Src: 240.17.0.6 Gw: <nil> Flags: [] Table: 254}
  5073. 2020-11-02T12:06:51.0462744Z [cluster1] I1102 12:04:22.297995 1 route.go:759] Found nil gw or dst
  5074. 2020-11-02T12:06:51.0463217Z [cluster1] I1102 12:04:22.298002 1 route.go:582] This route agent is running on the active gateway node
  5075. 2020-11-02T12:06:51.0464091Z [cluster1] I1102 12:04:22.298008 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  5076. 2020-11-02T12:06:51.0464926Z [cluster1] I1102 12:04:22.298215 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  5077. 2020-11-02T12:06:51.0466642Z [cluster1] I1102 12:04:22.298267 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.9 00:00:00:00:00:00
  5078. 2020-11-02T12:06:51.0467597Z [cluster1] I1102 12:04:22.298320 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.6 00:00:00:00:00:00
  5079. 2020-11-02T12:06:51.0468996Z [cluster1] I1102 12:04:22.298384 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  5080. 2020-11-02T12:06:51.0471013Z [cluster1] I1102 12:04:52.297663 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  5081. 2020-11-02T12:06:51.0473300Z [cluster1] I1102 12:04:52.297721 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  5082. 2020-11-02T12:06:51.0475142Z [cluster1] I1102 12:04:52.297731 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  5083. 2020-11-02T12:06:51.0476419Z [cluster1] I1102 12:04:52.297984 1 route.go:756] Processing route {Ifindex: 6 Dst: 240.0.0.0/8 Src: 240.17.0.6 Gw: <nil> Flags: [] Table: 254}
  5084. 2020-11-02T12:06:51.0476924Z [cluster1] I1102 12:04:52.298035 1 route.go:759] Found nil gw or dst
  5085. 2020-11-02T12:06:51.0477402Z [cluster1] I1102 12:04:52.298043 1 route.go:582] This route agent is running on the active gateway node
  5086. 2020-11-02T12:06:51.0478228Z [cluster1] I1102 12:04:52.298049 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  5087. 2020-11-02T12:06:51.0480418Z [cluster1] I1102 12:04:52.298196 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  5088. 2020-11-02T12:06:51.0481050Z [cluster1] I1102 12:04:52.298263 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.9 00:00:00:00:00:00
  5089. 2020-11-02T12:06:51.0481591Z [cluster1] I1102 12:04:52.298320 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.6 00:00:00:00:00:00
  5090. 2020-11-02T12:06:51.0482507Z [cluster1] I1102 12:04:52.298406 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  5091. 2020-11-02T12:06:51.0483838Z [cluster1] I1102 12:05:22.297890 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  5092. 2020-11-02T12:06:51.0486023Z [cluster1] I1102 12:05:22.297929 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  5093. 2020-11-02T12:06:51.0487148Z [cluster1] I1102 12:05:22.298159 1 route.go:756] Processing route {Ifindex: 6 Dst: 240.0.0.0/8 Src: 240.17.0.6 Gw: <nil> Flags: [] Table: 254}
  5094. 2020-11-02T12:06:51.0487999Z [cluster1] I1102 12:05:22.298202 1 route.go:759] Found nil gw or dst
  5095. 2020-11-02T12:06:51.0488751Z [cluster1] I1102 12:05:22.298209 1 route.go:582] This route agent is running on the active gateway node
  5096. 2020-11-02T12:06:51.0489858Z [cluster1] I1102 12:05:22.298215 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  5097. 2020-11-02T12:06:51.0490596Z [cluster1] I1102 12:05:22.298338 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  5098. 2020-11-02T12:06:51.0491869Z [cluster1] I1102 12:05:22.298372 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.9 00:00:00:00:00:00
  5099. 2020-11-02T12:06:51.0492408Z [cluster1] I1102 12:05:22.298400 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.6 00:00:00:00:00:00
  5100. 2020-11-02T12:06:51.0493252Z [cluster1] I1102 12:05:22.298466 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  5101. 2020-11-02T12:06:51.0494598Z [cluster1] I1102 12:05:22.298521 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  5102. 2020-11-02T12:06:51.0499190Z [cluster1] I1102 12:05:30.614500 1 route.go:682] Informed of removed endpoint: {"metadata":{"name":"cluster2-submariner-cable-cluster2-172-17-0-4","namespace":"submariner-operator","selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4","uid":"c09b80a0-c12c-4246-a9ea-692c9619737f","resourceVersion":"3454","generation":1,"creationTimestamp":"2020-11-02T12:01:40Z","labels":{"submariner-io/clusterID":"cluster2"}},"spec":{"cluster_id":"cluster2","cable_name":"submariner-cable-cluster2-172-17-0-4","hostname":"cluster2-worker","subnets":["100.2.0.0/16","10.2.0.0/16"],"private_ip":"172.17.0.4","public_ip":"","nat_enabled":false,"backend":"libreswan"}}
  5103. 2020-11-02T12:06:51.0503536Z [cluster1] I1102 12:05:30.715593 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11" with subnets [100.2.0.0/16 10.2.0.0/16]
  5104. 2020-11-02T12:06:51.0506171Z [cluster1] I1102 12:05:52.297938 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11" with subnets [100.2.0.0/16 10.2.0.0/16]
  5105. 2020-11-02T12:06:51.0508586Z [cluster1] I1102 12:05:52.298020 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  5106. 2020-11-02T12:06:51.0509891Z [cluster1] I1102 12:05:52.298259 1 route.go:756] Processing route {Ifindex: 6 Dst: 240.0.0.0/8 Src: 240.17.0.6 Gw: <nil> Flags: [] Table: 254}
  5107. 2020-11-02T12:06:51.0511093Z [cluster1] I1102 12:05:52.298291 1 route.go:759] Found nil gw or dst
  5108. 2020-11-02T12:06:51.0511561Z [cluster1] I1102 12:05:52.298307 1 route.go:582] This route agent is running on the active gateway node
  5109. 2020-11-02T12:06:51.0512390Z [cluster1] I1102 12:05:52.298315 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  5110. 2020-11-02T12:06:51.0513030Z [cluster1] I1102 12:05:52.298462 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  5111. 2020-11-02T12:06:51.0513616Z [cluster1] I1102 12:05:52.298508 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.9 00:00:00:00:00:00
  5112. 2020-11-02T12:06:51.0514162Z [cluster1] I1102 12:05:52.298547 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.6 00:00:00:00:00:00
  5113. 2020-11-02T12:06:51.0515483Z [cluster1] I1102 12:05:52.298596 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  5114. 2020-11-02T12:06:51.0517369Z [cluster1] I1102 12:05:52.298658 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  5115. 2020-11-02T12:06:51.0519263Z [cluster1] I1102 12:06:22.298823 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11" with subnets [100.2.0.0/16 10.2.0.0/16]
  5116. 2020-11-02T12:06:51.0521378Z [cluster1] I1102 12:06:22.298859 1 route.go:570] Processing local Endpoint cluster1-submariner-cable-cluster1-172-17-0-6 with IP 172.17.0.6, Host cluster1-worker for this cluster
  5117. 2020-11-02T12:06:51.0522455Z [cluster1] I1102 12:06:22.299039 1 route.go:756] Processing route {Ifindex: 6 Dst: 240.0.0.0/8 Src: 240.17.0.6 Gw: <nil> Flags: [] Table: 254}
  5118. 2020-11-02T12:06:51.0523078Z [cluster1] I1102 12:06:22.299059 1 route.go:759] Found nil gw or dst
  5119. 2020-11-02T12:06:51.0523542Z [cluster1] I1102 12:06:22.299066 1 route.go:582] This route agent is running on the active gateway node
  5120. 2020-11-02T12:06:51.0524806Z [cluster1] I1102 12:06:22.299072 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  5121. 2020-11-02T12:06:51.0525458Z [cluster1] I1102 12:06:22.299201 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  5122. 2020-11-02T12:06:51.0526418Z [cluster1] I1102 12:06:22.299253 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.9 00:00:00:00:00:00
  5123. 2020-11-02T12:06:51.0527225Z [cluster1] I1102 12:06:22.299282 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.6 00:00:00:00:00:00
  5124. 2020-11-02T12:06:51.3476492Z [cluster1] I1102 12:06:22.299337 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  5125. 2020-11-02T12:06:51.3479532Z [cluster1] I1102 12:06:22.299395 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  5126. 2020-11-02T12:06:51.3480481Z [cluster1] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  5127. 2020-11-02T12:06:51.3480975Z [cluster1]
  5128. 2020-11-02T12:06:51.3481367Z [cluster1] Showing information for cluster "cluster2":
  5129. 2020-11-02T12:06:51.3481817Z [cluster1] Discovered network details:
  5130. 2020-11-02T12:06:51.3482433Z [cluster1] Network plugin: weave-net
  5131. 2020-11-02T12:06:51.3482971Z [cluster1] Service CIDRs: [100.2.0.0/16]
  5132. 2020-11-02T12:06:51.3483471Z [cluster1] Cluster CIDRs: [10.2.0.0/16]
  5133. 2020-11-02T12:06:51.3483731Z [cluster1]
  5134. 2020-11-02T12:06:51.3484655Z [cluster1] CLUSTER ID ENDPOINT IP PUBLIC IP CABLE DRIVER TYPE
  5135. 2020-11-02T12:06:51.3485148Z [cluster1] cluster2 172.17.0.11 libreswan local
  5136. 2020-11-02T12:06:51.3485579Z [cluster1] cluster3 172.17.0.7 libreswan remote
  5137. 2020-11-02T12:06:51.3486015Z [cluster1] cluster1 172.17.0.6 libreswan remote
  5138. 2020-11-02T12:06:51.3486353Z [cluster1]
  5139. 2020-11-02T12:06:51.3486760Z [cluster1] GATEWAY CLUSTER REMOTE IP CABLE DRIVER SUBNETS STATUS
  5140. 2020-11-02T12:06:51.3487651Z [cluster1] cluster3-worker cluster3 172.17.0.7 libreswan 100.3.0.0/16, 10.3.0.0/16 connected
  5141. 2020-11-02T12:06:51.3488623Z [cluster1] cluster1-worker cluster1 172.17.0.6 libreswan 100.1.0.0/16, 10.1.0.0/16 connected
  5142. 2020-11-02T12:06:51.3489178Z [cluster1]
  5143. 2020-11-02T12:06:51.3489647Z [cluster1] NODE HA STATUS SUMMARY
  5144. 2020-11-02T12:06:51.3490480Z [cluster1] cluster2-worker2 active All connections (2) are established
  5145. 2020-11-02T12:06:51.3490917Z [cluster1]
  5146. 2020-11-02T12:06:51.3491277Z [cluster1] COMPONENT REPOSITORY VERSION
  5147. 2020-11-02T12:06:51.3491740Z [cluster1] submariner localhost:5000 local
  5148. 2020-11-02T12:06:51.3492581Z [cluster1] submariner-operator localhost:5000 local
  5149. 2020-11-02T12:06:51.3493688Z [cluster1] service-discovery localhost:5000 local
  5150. 2020-11-02T12:06:51.3494074Z [cluster1]
  5151. 2020-11-02T12:06:51.3494538Z [cluster1] Showing information for cluster "cluster3":
  5152. 2020-11-02T12:06:51.3495140Z [cluster1] Discovered network details:
  5153. 2020-11-02T12:06:51.3495704Z [cluster1] Network plugin: weave-net
  5154. 2020-11-02T12:06:51.3496041Z [cluster1] Service CIDRs: [100.3.0.0/16]
  5155. 2020-11-02T12:06:51.3496518Z [cluster1] Cluster CIDRs: [10.3.0.0/16]
  5156. 2020-11-02T12:06:51.3496961Z [cluster1]
  5157. 2020-11-02T12:06:51.3497533Z [cluster1] CLUSTER ID ENDPOINT IP PUBLIC IP CABLE DRIVER TYPE
  5158. 2020-11-02T12:06:51.3498025Z [cluster1] cluster3 172.17.0.7 libreswan local
  5159. 2020-11-02T12:06:51.3498475Z [cluster1] cluster1 172.17.0.6 libreswan remote
  5160. 2020-11-02T12:06:51.3498921Z [cluster1] cluster2 172.17.0.11 libreswan remote
  5161. 2020-11-02T12:06:51.3499263Z [cluster1]
  5162. 2020-11-02T12:06:51.3499821Z [cluster1] GATEWAY CLUSTER REMOTE IP CABLE DRIVER SUBNETS STATUS
  5163. 2020-11-02T12:06:51.3501194Z [cluster1] cluster1-worker cluster1 172.17.0.6 libreswan 100.1.0.0/16, 10.1.0.0/16 connected
  5164. 2020-11-02T12:06:51.3502111Z [cluster1] cluster2-worker2 cluster2 172.17.0.11 libreswan 100.2.0.0/16, 10.2.0.0/16 connected
  5165. 2020-11-02T12:06:51.3502553Z [cluster1]
  5166. 2020-11-02T12:06:51.3502860Z [cluster1] NODE HA STATUS SUMMARY
  5167. 2020-11-02T12:06:51.3503748Z [cluster1] cluster3-worker active All connections (2) are established
  5168. 2020-11-02T12:06:51.3504335Z [cluster1]
  5169. 2020-11-02T12:06:51.3504679Z [cluster1] COMPONENT REPOSITORY VERSION
  5170. 2020-11-02T12:06:51.3505124Z [cluster1] submariner localhost:5000 local
  5171. 2020-11-02T12:06:51.3505950Z [cluster1] submariner-operator localhost:5000 local
  5172. 2020-11-02T12:06:51.4244495Z [cluster1] service-discovery localhost:5000 local
  5173. 2020-11-02T12:06:51.4244902Z [cluster1]
  5174. 2020-11-02T12:06:51.4245324Z [cluster1] Showing information for cluster "cluster1":
  5175. 2020-11-02T12:06:51.4245745Z [cluster1] Discovered network details:
  5176. 2020-11-02T12:06:51.4246108Z [cluster1] Network plugin: generic
  5177. 2020-11-02T12:06:51.4246435Z [cluster1] Service CIDRs: [100.1.0.0/16]
  5178. 2020-11-02T12:06:51.4246902Z [cluster1] Cluster CIDRs: [10.1.0.0/16]
  5179. 2020-11-02T12:06:51.4247313Z [cluster1]
  5180. 2020-11-02T12:06:51.4247800Z [cluster1] CLUSTER ID ENDPOINT IP PUBLIC IP CABLE DRIVER TYPE
  5181. 2020-11-02T12:06:51.4248223Z [cluster1] cluster1 172.17.0.6 libreswan local
  5182. 2020-11-02T12:06:51.4248616Z [cluster1] cluster3 172.17.0.7 libreswan remote
  5183. 2020-11-02T12:06:51.4248997Z [cluster1] cluster2 172.17.0.11 libreswan remote
  5184. 2020-11-02T12:06:51.4249292Z [cluster1]
  5185. 2020-11-02T12:06:51.4249654Z [cluster1] GATEWAY CLUSTER REMOTE IP CABLE DRIVER SUBNETS STATUS
  5186. 2020-11-02T12:06:51.4250457Z [cluster1] cluster3-worker cluster3 172.17.0.7 libreswan 100.3.0.0/16, 10.3.0.0/16 connected
  5187. 2020-11-02T12:06:51.4251212Z [cluster1] cluster2-worker2 cluster2 172.17.0.11 libreswan 100.2.0.0/16, 10.2.0.0/16 connected
  5188. 2020-11-02T12:06:51.4251589Z [cluster1]
  5189. 2020-11-02T12:06:51.4252075Z [cluster1] NODE HA STATUS SUMMARY
  5190. 2020-11-02T12:06:51.4252722Z [cluster1] cluster1-worker active All connections (2) are established
  5191. 2020-11-02T12:06:51.4253102Z [cluster1]
  5192. 2020-11-02T12:06:51.4253591Z [cluster1] COMPONENT REPOSITORY VERSION
  5193. 2020-11-02T12:06:51.4254259Z [cluster1] submariner localhost:5000 local
  5194. 2020-11-02T12:06:51.4254925Z [cluster1] submariner-operator localhost:5000 local
  5195. 2020-11-02T12:06:51.4255616Z [cluster1] service-discovery localhost:5000 local
  5196. 2020-11-02T12:06:51.4256055Z [cluster1] ===================== END Post mortem cluster1 =====================
  5197. 2020-11-02T12:06:51.4271519Z [submariner]$ [cluster2] post_analyze
  5198. 2020-11-02T12:06:51.4288271Z [submariner]$ [cluster2] sed s/^/[cluster2] /
  5199. 2020-11-02T12:06:51.4298729Z [submariner]$ [cluster2] post_analyze
  5200. 2020-11-02T12:06:51.4313122Z [submariner]$ [cluster2] kubectl get all --all-namespaces
  5201. 2020-11-02T12:06:51.4326134Z [submariner]$ [cluster2] kubectl get all --all-namespaces
  5202. 2020-11-02T12:06:51.4341399Z [submariner]$ [cluster2] command kubectl --context=cluster2 get all --all-namespaces
  5203. 2020-11-02T12:06:51.4359883Z [submariner]$ [cluster2] kubectl --context=cluster2 get all --all-namespaces
  5204. 2020-11-02T12:06:51.8874123Z [cluster2] ======================= Post mortem cluster2 =======================
  5205. 2020-11-02T12:06:51.8874644Z [cluster2] NAMESPACE NAME READY STATUS RESTARTS AGE
  5206. 2020-11-02T12:06:51.8876329Z [cluster2] kube-system pod/coredns-6955765f44-9nwgj 1/1 Running 0 14m
  5207. 2020-11-02T12:06:51.8877253Z [cluster2] kube-system pod/coredns-6955765f44-vcgpn 1/1 Running 0 14m
  5208. 2020-11-02T12:06:51.8878204Z [cluster2] kube-system pod/etcd-cluster2-control-plane 1/1 Running 0 14m
  5209. 2020-11-02T12:06:51.8879532Z [cluster2] kube-system pod/kube-apiserver-cluster2-control-plane 1/1 Running 0 14m
  5210. 2020-11-02T12:06:51.8881076Z [cluster2] kube-system pod/kube-controller-manager-cluster2-control-plane 1/1 Running 0 14m
  5211. 2020-11-02T12:06:51.8882401Z [cluster2] kube-system pod/kube-proxy-88gzn 1/1 Running 0 14m
  5212. 2020-11-02T12:06:51.8885144Z [cluster2] kube-system pod/kube-proxy-jm52h 1/1 Running 0 14m
  5213. 2020-11-02T12:06:51.8887153Z [cluster2] kube-system pod/kube-proxy-mq6xw 1/1 Running 0 14m
  5214. 2020-11-02T12:06:51.8889555Z [cluster2] kube-system pod/kube-scheduler-cluster2-control-plane 1/1 Running 0 14m
  5215. 2020-11-02T12:06:51.8890939Z [cluster2] kube-system pod/weave-net-gqks5 2/2 Running 0 13m
  5216. 2020-11-02T12:06:51.8891946Z [cluster2] kube-system pod/weave-net-pntqf 2/2 Running 0 13m
  5217. 2020-11-02T12:06:51.8892834Z [cluster2] kube-system pod/weave-net-prp6w 2/2 Running 0 13m
  5218. 2020-11-02T12:06:51.8894147Z [cluster2] local-path-storage pod/local-path-provisioner-7745554f7f-j7qsb 1/1 Running 0 14m
  5219. 2020-11-02T12:06:51.8895642Z [cluster2] submariner-operator pod/submariner-gateway-whn7f 1/1 Running 0 117s
  5220. 2020-11-02T12:06:51.8897837Z [cluster2] submariner-operator pod/submariner-lighthouse-agent-59556bc578-vx86d 0/1 ImagePullBackOff 0 9m34s
  5221. 2020-11-02T12:06:51.8900072Z [cluster2] submariner-operator pod/submariner-lighthouse-coredns-557485fbc-554kl 0/1 ImagePullBackOff 0 9m33s
  5222. 2020-11-02T12:06:51.8902180Z [cluster2] submariner-operator pod/submariner-lighthouse-coredns-557485fbc-fmn68 0/1 ImagePullBackOff 0 9m33s
  5223. 2020-11-02T12:06:51.8904732Z [cluster2] submariner-operator pod/submariner-operator-6675977db7-kqscw 1/1 Running 0 9m45s
  5224. 2020-11-02T12:06:51.8906379Z [cluster2] submariner-operator pod/submariner-routeagent-8x2cc 1/1 Running 0 9m34s
  5225. 2020-11-02T12:06:51.8907763Z [cluster2] submariner-operator pod/submariner-routeagent-ftrjn 1/1 Running 0 9m34s
  5226. 2020-11-02T12:06:51.8908951Z [cluster2] submariner-operator pod/submariner-routeagent-swbc2 1/1 Running 0 9m34s
  5227. 2020-11-02T12:06:51.8909720Z [cluster2]
  5228. 2020-11-02T12:06:51.8910382Z [cluster2] NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
  5229. 2020-11-02T12:06:51.8911253Z [cluster2] default service/kubernetes ClusterIP 100.2.0.1 <none> 443/TCP 14m
  5230. 2020-11-02T12:06:51.8912106Z [cluster2] kube-system service/kube-dns ClusterIP 100.2.0.10 <none> 53/UDP,53/TCP,9153/TCP 14m
  5231. 2020-11-02T12:06:51.8913730Z [cluster2] submariner-operator service/submariner-lighthouse-coredns ClusterIP 100.2.106.211 <none> 53/UDP 9m32s
  5232. 2020-11-02T12:06:51.8915134Z [cluster2] submariner-operator service/submariner-operator-metrics ClusterIP 100.2.61.14 <none> 8383/TCP,8686/TCP 9m35s
  5233. 2020-11-02T12:06:51.8915837Z [cluster2]
  5234. 2020-11-02T12:06:51.8917022Z [cluster2] NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
  5235. 2020-11-02T12:06:51.8918171Z [cluster2] kube-system daemonset.apps/kube-proxy 3 3 3 3 3 beta.kubernetes.io/os=linux 14m
  5236. 2020-11-02T12:06:51.8975410Z [submariner]$ [cluster2] kubectl get pods -A
  5237. 2020-11-02T12:06:51.8991781Z [submariner]$ [cluster2] tail -n +2
  5238. 2020-11-02T12:06:51.9000609Z [submariner]$ [cluster2] kubectl get pods -A
  5239. 2020-11-02T12:06:51.9013093Z [submariner]$ [cluster2] command kubectl --context=cluster2 get pods -A
  5240. 2020-11-02T12:06:51.9019130Z [submariner]$ [cluster2] grep -v Running
  5241. 2020-11-02T12:06:51.9032731Z [submariner]$ [cluster2] sed s/ */;/g
  5242. 2020-11-02T12:06:51.9045753Z [submariner]$ [cluster2] kubectl --context=cluster2 get pods -A
  5243. 2020-11-02T12:06:52.1768599Z [submariner]$ [cluster2] ns=submariner-operator
  5244. 2020-11-02T12:06:52.1787195Z [submariner]$ [cluster2] cut -f1 -d;
  5245. 2020-11-02T12:06:52.1841497Z [submariner]$ [cluster2] name=submariner-lighthouse-agent-59556bc578-vx86d
  5246. 2020-11-02T12:06:52.1858882Z [submariner]$ [cluster2] cut -f2 -d;
  5247. 2020-11-02T12:06:52.1882121Z [submariner]$ [cluster2] kubectl -n submariner-operator describe pod submariner-lighthouse-agent-59556bc578-vx86d
  5248. 2020-11-02T12:06:52.1894411Z [submariner]$ [cluster2] kubectl -n submariner-operator describe pod submariner-lighthouse-agent-59556bc578-vx86d
  5249. 2020-11-02T12:06:52.1903002Z [submariner]$ [cluster2] command kubectl --context=cluster2 -n submariner-operator describe pod submariner-lighthouse-agent-59556bc578-vx86d
  5250. 2020-11-02T12:06:52.1913325Z [submariner]$ [cluster2] kubectl --context=cluster2 -n submariner-operator describe pod submariner-lighthouse-agent-59556bc578-vx86d
  5251. 2020-11-02T12:06:52.4880784Z [cluster2] kube-system daemonset.apps/weave-net 3 3 3 3 3 <none> 13m
  5252. 2020-11-02T12:06:52.4882111Z [cluster2] submariner-operator daemonset.apps/submariner-gateway 1 1 1 1 1 submariner.io/gateway=true 9m34s
  5253. 2020-11-02T12:06:52.4883259Z [cluster2] submariner-operator daemonset.apps/submariner-routeagent 3 3 3 3 3 <none> 9m34s
  5254. 2020-11-02T12:06:52.4883808Z [cluster2]
  5255. 2020-11-02T12:06:52.4885304Z [cluster2] NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
  5256. 2020-11-02T12:06:52.4886200Z [cluster2] kube-system deployment.apps/coredns 2/2 2 2 14m
  5257. 2020-11-02T12:06:52.4887306Z [cluster2] local-path-storage deployment.apps/local-path-provisioner 1/1 1 1 14m
  5258. 2020-11-02T12:06:52.4888719Z [cluster2] submariner-operator deployment.apps/submariner-lighthouse-agent 0/1 1 0 9m34s
  5259. 2020-11-02T12:06:52.4890046Z [cluster2] submariner-operator deployment.apps/submariner-lighthouse-coredns 0/2 2 0 9m33s
  5260. 2020-11-02T12:06:52.4891285Z [cluster2] submariner-operator deployment.apps/submariner-operator 1/1 1 1 9m45s
  5261. 2020-11-02T12:06:52.4892053Z [cluster2]
  5262. 2020-11-02T12:06:52.4892547Z [cluster2] NAMESPACE NAME DESIRED CURRENT READY AGE
  5263. 2020-11-02T12:06:52.4893454Z [cluster2] kube-system replicaset.apps/coredns-6955765f44 2 2 2 14m
  5264. 2020-11-02T12:06:52.4894616Z [cluster2] local-path-storage replicaset.apps/local-path-provisioner-7745554f7f 1 1 1 14m
  5265. 2020-11-02T12:06:52.4896094Z [cluster2] submariner-operator replicaset.apps/submariner-lighthouse-agent-59556bc578 1 1 0 9m34s
  5266. 2020-11-02T12:06:52.4897701Z [cluster2] submariner-operator replicaset.apps/submariner-lighthouse-coredns-557485fbc 2 2 0 9m33s
  5267. 2020-11-02T12:06:52.4899396Z [cluster2] submariner-operator replicaset.apps/submariner-operator-6675977db7 1 1 1 9m45s
  5268. 2020-11-02T12:06:52.4900929Z [cluster2] ======================= submariner-lighthouse-agent-59556bc578-vx86d - submariner-operator ============================
  5269. 2020-11-02T12:06:52.4902580Z [cluster2] Name: submariner-lighthouse-agent-59556bc578-vx86d
  5270. 2020-11-02T12:06:52.4903921Z [cluster2] Namespace: submariner-operator
  5271. 2020-11-02T12:06:52.4904436Z [cluster2] Priority: 0
  5272. 2020-11-02T12:06:52.4905064Z [cluster2] Node: cluster2-worker2/172.17.0.11
  5273. 2020-11-02T12:06:52.4905561Z [cluster2] Start Time: Mon, 02 Nov 2020 11:57:18 +0000
  5274. 2020-11-02T12:06:52.4906315Z [cluster2] Labels: app=submariner-lighthouse-agent
  5275. 2020-11-02T12:06:52.4907341Z [cluster2] component=submariner-lighthouse
  5276. 2020-11-02T12:06:52.4908409Z [cluster2] pod-template-hash=59556bc578
  5277. 2020-11-02T12:06:52.4908901Z [cluster2] Annotations: <none>
  5278. 2020-11-02T12:06:52.4909492Z [cluster2] Status: Pending
  5279. 2020-11-02T12:06:52.4909856Z [cluster2] IP: 10.2.128.2
  5280. 2020-11-02T12:06:52.4910203Z [cluster2] IPs:
  5281. 2020-11-02T12:06:52.4910563Z [cluster2] IP: 10.2.128.2
  5282. 2020-11-02T12:06:52.4911525Z [cluster2] Controlled By: ReplicaSet/submariner-lighthouse-agent-59556bc578
  5283. 2020-11-02T12:06:52.4912488Z [cluster2] Containers:
  5284. 2020-11-02T12:06:52.4913555Z [cluster2] submariner-lighthouse-agent:
  5285. 2020-11-02T12:06:52.4914137Z [cluster2] Container ID:
  5286. 2020-11-02T12:06:52.4914835Z [cluster2] Image: localhost:5000/lighthouse-agent:local
  5287. 2020-11-02T12:06:52.4915354Z [cluster2] Image ID:
  5288. 2020-11-02T12:06:52.4915728Z [cluster2] Port: <none>
  5289. 2020-11-02T12:06:52.4916112Z [cluster2] Host Port: <none>
  5290. 2020-11-02T12:06:52.4916655Z [cluster2] State: Waiting
  5291. 2020-11-02T12:06:52.4917122Z [cluster2] Reason: ImagePullBackOff
  5292. 2020-11-02T12:06:52.4917573Z [cluster2] Ready: False
  5293. 2020-11-02T12:06:52.4918267Z [cluster2] Restart Count: 0
  5294. 2020-11-02T12:06:52.4918656Z [cluster2] Environment:
  5295. 2020-11-02T12:06:52.4919383Z [cluster2] SUBMARINER_NAMESPACE: submariner-operator
  5296. 2020-11-02T12:06:52.4919993Z [cluster2] SUBMARINER_CLUSTERID: cluster2
  5297. 2020-11-02T12:06:52.4920877Z [cluster2] SUBMARINER_EXCLUDENS: submariner,kube-system,operators
  5298. 2020-11-02T12:06:52.4921893Z [cluster2] SUBMARINER_DEBUG: false
  5299. 2020-11-02T12:06:52.4922577Z [cluster2] SUBMARINER_GLOBALNET_ENABLED: false
  5300. 2020-11-02T12:06:52.4923080Z [cluster2] BROKER_K8S_APISERVER: 172.17.0.9:6443
  5301. 2020-11-02T12:06:52.4966132Z [submariner]$ [cluster2] kubectl -n submariner-operator logs submariner-lighthouse-agent-59556bc578-vx86d
  5302. 2020-11-02T12:06:52.4979035Z [submariner]$ [cluster2] kubectl -n submariner-operator logs submariner-lighthouse-agent-59556bc578-vx86d
  5303. 2020-11-02T12:06:52.5011418Z [submariner]$ [cluster2] command kubectl --context=cluster2 -n submariner-operator logs submariner-lighthouse-agent-59556bc578-vx86d
  5304. 2020-11-02T12:06:53.2276483Z [submariner]$ [cluster2] kubectl --context=cluster2 -n submariner-operator logs submariner-lighthouse-agent-59556bc578-vx86d
  5305. 2020-11-02T12:06:53.2459066Z [cluster2] BROKER_K8S_APISERVERTOKEN: eyJhbGciOiJSUzI1NiIsImtpZCI6InhlWjlWdWVUMTJ5dnpDRDZhc09MU0M3YlJheHl4ZkNMdDhXaW40bDdrdmsifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJzdWJtYXJpbmVyLWs4cy1icm9rZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlY3JldC5uYW1lIjoiY2x1c3Rlci1jbHVzdGVyMi10b2tlbi1scGs3OSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJjbHVzdGVyLWNsdXN0ZXIyIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiY2FhOWYyMTMtN2NlYi00ZTEzLWFmMzMtZDg3N2JjYTJhN2NlIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OnN1Ym1hcmluZXItazhzLWJyb2tlcjpjbHVzdGVyLWNsdXN0ZXIyIn0.muFHaDnVlYdvh3izTTMb-CPZPr0sXjXCcgqftBcJwZrlzzBnBKroEcGwMgHSvRNEznGSK1kaidjn8DI8A68TLtxN4YZFKlbHTNMbtdW_Cn8Dhkr9klp4zFde5eZlQu5mOKGTR75egi15alJl3t63unxDF0iLaKCSRX3fcd4wlvkg3ooqZxkidx4nDw8a9jP5biGKZpxBoIIpUMZqQWQorqjYNu7kfziduPIIZx69Mk_1RXgDyIwChGLDEgz-1-kkKso0zYBxfHYgw9SonpnU0tTWpf94aI8jpI5qB_6QaichRiOX4nI5k4olVBRICSAoHHHdRVh_PWtDv8GDQIH04Q
  5306. 2020-11-02T12:06:53.2631769Z [cluster2] BROKER_K8S_REMOTENAMESPACE: submariner-k8s-broker
  5307. 2020-11-02T12:06:53.3192594Z [cluster2] BROKER_K8S_CA: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJd01URXdNakV4TlRBMU1sb1hEVE13TVRBek1URXhOVEExTWxvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTVUwClVkaVlaV0xudTBjLzRsL1pEVUk2L2dyd3ZMakQ3T2lJczJSSjdNZnNuMXdrb2U3UE1Od2Y4YjQ2Y1MrdkJxR0oKS2xjUHhnTWpkYWtGdndSb3ZjR2JVWHBsdUlLZkNMTWxVKzdYenhuUnc4TTRwaFBndkxCRjhWRzg4WFBJTXpEcgpsT2tZQXdmY2V2L2ZVeWdxdTNBNzJqd3lQNGFodm0rRHcyN1htQ3R5cUloRmVGc1dzOXRBWDdNYk1BYk53Z09ICm82L2VUcEREQ05zc0ZiMUJhaWJrNDJEbm1SUVVkNHdnYSt1UEZPS2oxTEF5LzZaSWExVkJYckJIb2xUVkVnOWYKV3BCZEF5Um5IWis4cUxZeWVtUXQ0RHJrUXBuRy83a3JNcVB5Q2tBRTVONGppTGVvbXpVZlpEWlVCNWxEY1BLMwpuS2JibXhDZkhCQXNqNVhrUWtFQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFLeGUzc3RHbE9PaVIvdFpFUWlBcHRkNE0wOFAKVGIrTDNEVFNLZGlnQjVTNWhFakR6L3RzTHI5NUhQbEMvQXNUeW9yWDV2eGJHeHdsNHVmb0hXRzVtM0xJL3JMUgpNVmFITExVSmwzaWZ0c0R1SXVMaFZMT2pIM3NOelhmcGdFZ0o5Q1VjalhsTTlkMm4wTGlzK24rdy83OTJvREpiClRsMCtxZTdDcDR0eXdNYzN4MWJlZzN5YTFUbEdxaUV1UkRDUFZaWU5wazd3NlpReGN4dUlhRDQ1SW5YVHVqV3gKTHpCdURFSlErQlFkR2VORDc5T1k4SEdyZEYyaG5TdEdBcFMwUUtRWktNZ2ppOEo5MW5VQzNRNFZXT1B4U2Q0LwpCRENHdUJ4RzhXTTEvak5LdCs1MmJWeDEvdGF6VEdrSWhXVnd1UHl2c1IxRk5UWUpDTkRFVkhGT3A3MD0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
  5308. 2020-11-02T12:06:53.3702014Z [cluster2] Mounts:
  5309. 2020-11-02T12:06:53.3703361Z [cluster2] /var/run/secrets/kubernetes.io/serviceaccount from submariner-lighthouse-token-dv4vb (ro)
  5310. 2020-11-02T12:06:53.3708016Z [cluster2] Conditions:
  5311. 2020-11-02T12:06:53.3708537Z [cluster2] Type Status
  5312. 2020-11-02T12:06:53.3709029Z [cluster2] Initialized True
  5313. 2020-11-02T12:06:53.3709752Z [cluster2] Ready False
  5314. 2020-11-02T12:06:53.3710113Z [cluster2] ContainersReady False
  5315. 2020-11-02T12:06:53.3711029Z [cluster2] PodScheduled True
  5316. 2020-11-02T12:06:53.3711343Z [cluster2] Volumes:
  5317. 2020-11-02T12:06:53.3712273Z [cluster2] submariner-lighthouse-token-dv4vb:
  5318. 2020-11-02T12:06:53.3713894Z [cluster2] Type: Secret (a volume populated by a Secret)
  5319. 2020-11-02T12:06:53.3715037Z [cluster2] SecretName: submariner-lighthouse-token-dv4vb
  5320. 2020-11-02T12:06:53.3716580Z [cluster2] Optional: false
  5321. 2020-11-02T12:06:53.3717608Z [cluster2] QoS Class: BestEffort
  5322. 2020-11-02T12:06:53.3727758Z [cluster2] Node-Selectors: <none>
  5323. 2020-11-02T12:06:53.3729268Z [cluster2] Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
  5324. 2020-11-02T12:06:53.3730092Z [cluster2] node.kubernetes.io/unreachable:NoExecute for 300s
  5325. 2020-11-02T12:06:53.3731232Z [cluster2] Events:
  5326. 2020-11-02T12:06:53.3732427Z [cluster2] Type Reason Age From Message
  5327. 2020-11-02T12:06:53.3733084Z [cluster2] ---- ------ ---- ---- -------
  5328. 2020-11-02T12:06:53.3738011Z [cluster2] Normal Scheduled 9m34s default-scheduler Successfully assigned submariner-operator/submariner-lighthouse-agent-59556bc578-vx86d to cluster2-worker2
  5329. 2020-11-02T12:06:53.3740645Z [cluster2] Normal Pulling 7m44s (x4 over 9m32s) kubelet, cluster2-worker2 Pulling image "localhost:5000/lighthouse-agent:local"
  5330. 2020-11-02T12:06:53.3744327Z [cluster2] Warning Failed 7m44s (x4 over 9m7s) kubelet, cluster2-worker2 Failed to pull image "localhost:5000/lighthouse-agent:local": rpc error: code = Unknown desc = failed to pull and unpack image "localhost:5000/lighthouse-agent:local": failed to resolve reference "localhost:5000/lighthouse-agent:local": failed to do request: Head https://localhost:5000/v2/lighthouse-agent/manifests/local: dial tcp 127.0.0.1:5000: connect: connection refused
  5331. 2020-11-02T12:06:53.3746580Z [cluster2] Warning Failed 7m44s (x4 over 9m7s) kubelet, cluster2-worker2 Error: ErrImagePull
  5332. 2020-11-02T12:06:53.3747688Z [cluster2] Normal BackOff 7m17s (x6 over 9m6s) kubelet, cluster2-worker2 Back-off pulling image "localhost:5000/lighthouse-agent:local"
  5333. 2020-11-02T12:06:53.3748784Z [cluster2] Warning Failed 4m30s (x18 over 9m6s) kubelet, cluster2-worker2 Error: ImagePullBackOff
  5334. 2020-11-02T12:06:53.3754307Z Error from server (BadRequest): container "submariner-lighthouse-agent" in pod "submariner-lighthouse-agent-59556bc578-vx86d" is waiting to start: trying and failing to pull image
  5335. 2020-11-02T12:06:53.3763532Z [submariner]$ [cluster2] ns=submariner-operator
  5336. 2020-11-02T12:06:53.3770900Z [submariner]$ [cluster2] cut -f1 -d;
  5337. 2020-11-02T12:06:53.3772591Z [submariner]$ [cluster2] name=submariner-lighthouse-coredns-557485fbc-554kl
  5338. 2020-11-02T12:06:53.3773926Z [submariner]$ [cluster2] cut -f2 -d;
  5339. 2020-11-02T12:06:53.3775800Z [submariner]$ [cluster2] kubectl -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-554kl
  5340. 2020-11-02T12:06:53.3780796Z [submariner]$ [cluster2] kubectl -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-554kl
  5341. 2020-11-02T12:06:53.3784868Z [submariner]$ [cluster2] command kubectl --context=cluster2 -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-554kl
  5342. 2020-11-02T12:06:53.3787658Z [submariner]$ [cluster2] kubectl --context=cluster2 -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-554kl
  5343. 2020-11-02T12:06:53.3790308Z [submariner]$ [cluster2] kubectl -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-554kl
  5344. 2020-11-02T12:06:53.3792665Z [submariner]$ [cluster2] kubectl -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-554kl
  5345. 2020-11-02T12:06:53.3814664Z [submariner]$ [cluster2] command kubectl --context=cluster2 -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-554kl
  5346. 2020-11-02T12:06:53.3816870Z [submariner]$ [cluster2] kubectl --context=cluster2 -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-554kl
  5347. 2020-11-02T12:06:53.5730962Z Error from server (BadRequest): container "submariner-lighthouse-coredns" in pod "submariner-lighthouse-coredns-557485fbc-554kl" is waiting to start: trying and failing to pull image
  5348. 2020-11-02T12:06:53.5797714Z [submariner]$ [cluster2] ns=submariner-operator
  5349. 2020-11-02T12:06:53.5813918Z [submariner]$ [cluster2] cut -f1 -d;
  5350. 2020-11-02T12:06:53.5856211Z [submariner]$ [cluster2] name=submariner-lighthouse-coredns-557485fbc-fmn68
  5351. 2020-11-02T12:06:53.5873717Z [submariner]$ [cluster2] cut -f2 -d;
  5352. 2020-11-02T12:06:53.5903493Z [submariner]$ [cluster2] kubectl -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-fmn68
  5353. 2020-11-02T12:06:53.5913934Z [submariner]$ [cluster2] kubectl -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-fmn68
  5354. 2020-11-02T12:06:53.5928212Z [submariner]$ [cluster2] command kubectl --context=cluster2 -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-fmn68
  5355. 2020-11-02T12:06:53.5940284Z [submariner]$ [cluster2] kubectl --context=cluster2 -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-fmn68
  5356. 2020-11-02T12:06:53.8998437Z [cluster2] ===================== END submariner-lighthouse-agent-59556bc578-vx86d - submariner-operator ==========================
  5357. 2020-11-02T12:06:53.9000422Z [cluster2] ======================= submariner-lighthouse-coredns-557485fbc-554kl - submariner-operator ============================
  5358. 2020-11-02T12:06:53.9001844Z [cluster2] Name: submariner-lighthouse-coredns-557485fbc-554kl
  5359. 2020-11-02T12:06:53.9002794Z [cluster2] Namespace: submariner-operator
  5360. 2020-11-02T12:06:53.9003193Z [cluster2] Priority: 0
  5361. 2020-11-02T12:06:53.9004473Z [cluster2] Node: cluster2-worker/172.17.0.4
  5362. 2020-11-02T12:06:53.9004848Z [cluster2] Start Time: Mon, 02 Nov 2020 11:57:18 +0000
  5363. 2020-11-02T12:06:53.9005512Z [cluster2] Labels: app=submariner-lighthouse-coredns
  5364. 2020-11-02T12:06:53.9006213Z [cluster2] component=submariner-lighthouse
  5365. 2020-11-02T12:06:53.9006794Z [cluster2] pod-template-hash=557485fbc
  5366. 2020-11-02T12:06:53.9007152Z [cluster2] Annotations: <none>
  5367. 2020-11-02T12:06:53.9007461Z [cluster2] Status: Pending
  5368. 2020-11-02T12:06:53.9007713Z [cluster2] IP: 10.2.0.4
  5369. 2020-11-02T12:06:53.9007950Z [cluster2] IPs:
  5370. 2020-11-02T12:06:53.9008195Z [cluster2] IP: 10.2.0.4
  5371. 2020-11-02T12:06:53.9008991Z [cluster2] Controlled By: ReplicaSet/submariner-lighthouse-coredns-557485fbc
  5372. 2020-11-02T12:06:53.9009652Z [cluster2] Containers:
  5373. 2020-11-02T12:06:53.9010232Z [cluster2] submariner-lighthouse-coredns:
  5374. 2020-11-02T12:06:53.9011056Z [cluster2] Container ID:
  5375. 2020-11-02T12:06:53.9011827Z [cluster2] Image: localhost:5000/lighthouse-coredns:local
  5376. 2020-11-02T12:06:53.9012239Z [cluster2] Image ID:
  5377. 2020-11-02T12:06:53.9012527Z [cluster2] Port: <none>
  5378. 2020-11-02T12:06:53.9012830Z [cluster2] Host Port: <none>
  5379. 2020-11-02T12:06:53.9013116Z [cluster2] Args:
  5380. 2020-11-02T12:06:53.9013510Z [cluster2] -conf
  5381. 2020-11-02T12:06:53.9013945Z [cluster2] /etc/coredns/Corefile
  5382. 2020-11-02T12:06:53.9014460Z [cluster2] State: Waiting
  5383. 2020-11-02T12:06:53.9015343Z [cluster2] Reason: ImagePullBackOff
  5384. 2020-11-02T12:06:53.9015859Z [cluster2] Ready: False
  5385. 2020-11-02T12:06:53.9017667Z [cluster2] Restart Count: 0
  5386. 2020-11-02T12:06:53.9018045Z [cluster2] Environment: <none>
  5387. 2020-11-02T12:06:53.9018457Z [cluster2] Mounts:
  5388. 2020-11-02T12:06:53.9019573Z [cluster2] /etc/coredns from config-volume (ro)
  5389. 2020-11-02T12:06:53.9021389Z [cluster2] /var/run/secrets/kubernetes.io/serviceaccount from submariner-lighthouse-token-dv4vb (ro)
  5390. 2020-11-02T12:06:53.9022249Z [cluster2] Conditions:
  5391. 2020-11-02T12:06:53.9022577Z [cluster2] Type Status
  5392. 2020-11-02T12:06:53.9022945Z [cluster2] Initialized True
  5393. 2020-11-02T12:06:53.9023312Z [cluster2] Ready False
  5394. 2020-11-02T12:06:53.9023716Z [cluster2] ContainersReady False
  5395. 2020-11-02T12:06:53.9024154Z [cluster2] PodScheduled True
  5396. 2020-11-02T12:06:53.9024672Z [cluster2] Volumes:
  5397. 2020-11-02T12:06:53.9025583Z [cluster2] config-volume:
  5398. 2020-11-02T12:06:53.9026297Z [cluster2] Type: ConfigMap (a volume populated by a ConfigMap)
  5399. 2020-11-02T12:06:53.9028040Z [cluster2] Name: submariner-lighthouse-coredns
  5400. 2020-11-02T12:06:53.9028619Z [cluster2] Optional: false
  5401. 2020-11-02T12:06:53.9030286Z [cluster2] submariner-lighthouse-token-dv4vb:
  5402. 2020-11-02T12:06:53.9031710Z [cluster2] Type: Secret (a volume populated by a Secret)
  5403. 2020-11-02T12:06:53.9032626Z [cluster2] SecretName: submariner-lighthouse-token-dv4vb
  5404. 2020-11-02T12:06:53.9033500Z [cluster2] Optional: false
  5405. 2020-11-02T12:06:53.9034110Z [cluster2] QoS Class: BestEffort
  5406. 2020-11-02T12:06:53.9034722Z [cluster2] Node-Selectors: <none>
  5407. 2020-11-02T12:06:53.9035482Z [cluster2] Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
  5408. 2020-11-02T12:06:53.9036211Z [cluster2] node.kubernetes.io/unreachable:NoExecute for 300s
  5409. 2020-11-02T12:06:53.9036714Z [cluster2] Events:
  5410. 2020-11-02T12:06:53.9037556Z [cluster2] Type Reason Age From Message
  5411. 2020-11-02T12:06:53.9038074Z [cluster2] ---- ------ ---- ---- -------
  5412. 2020-11-02T12:06:53.9039566Z [cluster2] Normal Scheduled 9m35s default-scheduler Successfully assigned submariner-operator/submariner-lighthouse-coredns-557485fbc-554kl to cluster2-worker
  5413. 2020-11-02T12:06:53.9041550Z [cluster2] Normal Pulling 7m5s (x4 over 9m29s) kubelet, cluster2-worker Pulling image "localhost:5000/lighthouse-coredns:local"
  5414. 2020-11-02T12:06:53.9044715Z [cluster2] Warning Failed 7m5s (x4 over 8m35s) kubelet, cluster2-worker Failed to pull image "localhost:5000/lighthouse-coredns:local": rpc error: code = Unknown desc = failed to pull and unpack image "localhost:5000/lighthouse-coredns:local": failed to resolve reference "localhost:5000/lighthouse-coredns:local": failed to do request: Head https://localhost:5000/v2/lighthouse-coredns/manifests/local: dial tcp 127.0.0.1:5000: connect: connection refused
  5415. 2020-11-02T12:06:53.9046990Z [cluster2] Warning Failed 7m5s (x4 over 8m35s) kubelet, cluster2-worker Error: ErrImagePull
  5416. 2020-11-02T12:06:53.9048081Z [cluster2] Warning Failed 6m40s (x7 over 8m34s) kubelet, cluster2-worker Error: ImagePullBackOff
  5417. 2020-11-02T12:06:53.9049174Z [cluster2] Normal BackOff 4m24s (x16 over 8m34s) kubelet, cluster2-worker Back-off pulling image "localhost:5000/lighthouse-coredns:local"
  5418. 2020-11-02T12:06:53.9050568Z [cluster2] ===================== END submariner-lighthouse-coredns-557485fbc-554kl - submariner-operator ==========================
  5419. 2020-11-02T12:06:53.9052351Z [cluster2] ======================= submariner-lighthouse-coredns-557485fbc-fmn68 - submariner-operator ============================
  5420. 2020-11-02T12:06:53.9075416Z [submariner]$ [cluster2] kubectl -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-fmn68
  5421. 2020-11-02T12:06:53.9082784Z [submariner]$ [cluster2] kubectl -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-fmn68
  5422. 2020-11-02T12:06:53.9095742Z [submariner]$ [cluster2] command kubectl --context=cluster2 -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-fmn68
  5423. 2020-11-02T12:06:53.9111731Z [submariner]$ [cluster2] kubectl --context=cluster2 -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-fmn68
  5424. 2020-11-02T12:06:54.2073303Z Error from server (BadRequest): container "submariner-lighthouse-coredns" in pod "submariner-lighthouse-coredns-557485fbc-fmn68" is waiting to start: trying and failing to pull image
  5425. 2020-11-02T12:06:54.2132541Z [submariner]$ [cluster2] namespace=kube-system
  5426. 2020-11-02T12:06:54.2160742Z [submariner]$ [cluster2] kubectl get pods --selector=k8s-app=kube-proxy -n kube-system -o jsonpath={.items[*].metadata.name}
  5427. 2020-11-02T12:06:54.2173358Z [submariner]$ [cluster2] kubectl get pods --selector=k8s-app=kube-proxy -n kube-system -o jsonpath={.items[*].metadata.name}
  5428. 2020-11-02T12:06:54.2190237Z [submariner]$ [cluster2] command kubectl --context=cluster2 get pods --selector=k8s-app=kube-proxy -n kube-system -o jsonpath={.items[*].metadata.name}
  5429. 2020-11-02T12:06:54.2203756Z [submariner]$ [cluster2] kubectl --context=cluster2 get pods --selector=k8s-app=kube-proxy -n kube-system -o jsonpath={.items[*].metadata.name}
  5430. 2020-11-02T12:06:54.5077557Z [submariner]$ [cluster2] kubectl -n kube-system logs kube-proxy-88gzn
  5431. 2020-11-02T12:06:54.5087665Z [submariner]$ [cluster2] kubectl -n kube-system logs kube-proxy-88gzn
  5432. 2020-11-02T12:06:54.5098290Z [submariner]$ [cluster2] command kubectl --context=cluster2 -n kube-system logs kube-proxy-88gzn
  5433. 2020-11-02T12:06:54.5111484Z [submariner]$ [cluster2] kubectl --context=cluster2 -n kube-system logs kube-proxy-88gzn
  5434. 2020-11-02T12:06:54.8074907Z [cluster2] Name: submariner-lighthouse-coredns-557485fbc-fmn68
  5435. 2020-11-02T12:06:54.8075979Z [cluster2] Namespace: submariner-operator
  5436. 2020-11-02T12:06:54.8076389Z [cluster2] Priority: 0
  5437. 2020-11-02T12:06:54.8076909Z [cluster2] Node: cluster2-worker2/172.17.0.11
  5438. 2020-11-02T12:06:54.8077332Z [cluster2] Start Time: Mon, 02 Nov 2020 11:57:18 +0000
  5439. 2020-11-02T12:06:54.8078010Z [cluster2] Labels: app=submariner-lighthouse-coredns
  5440. 2020-11-02T12:06:54.8079099Z [cluster2] component=submariner-lighthouse
  5441. 2020-11-02T12:06:54.8079803Z [cluster2] pod-template-hash=557485fbc
  5442. 2020-11-02T12:06:54.8080236Z [cluster2] Annotations: <none>
  5443. 2020-11-02T12:06:54.8080615Z [cluster2] Status: Pending
  5444. 2020-11-02T12:06:54.8081280Z [cluster2] IP: 10.2.128.3
  5445. 2020-11-02T12:06:54.8081612Z [cluster2] IPs:
  5446. 2020-11-02T12:06:54.8081903Z [cluster2] IP: 10.2.128.3
  5447. 2020-11-02T12:06:54.8083075Z [cluster2] Controlled By: ReplicaSet/submariner-lighthouse-coredns-557485fbc
  5448. 2020-11-02T12:06:54.8084717Z [cluster2] Containers:
  5449. 2020-11-02T12:06:54.8085411Z [cluster2] submariner-lighthouse-coredns:
  5450. 2020-11-02T12:06:54.8085916Z [cluster2] Container ID:
  5451. 2020-11-02T12:06:54.8086527Z [cluster2] Image: localhost:5000/lighthouse-coredns:local
  5452. 2020-11-02T12:06:54.8086973Z [cluster2] Image ID:
  5453. 2020-11-02T12:06:54.8087262Z [cluster2] Port: <none>
  5454. 2020-11-02T12:06:54.8087561Z [cluster2] Host Port: <none>
  5455. 2020-11-02T12:06:54.8087848Z [cluster2] Args:
  5456. 2020-11-02T12:06:54.8088233Z [cluster2] -conf
  5457. 2020-11-02T12:06:54.8088526Z [cluster2] /etc/coredns/Corefile
  5458. 2020-11-02T12:06:54.8088866Z [cluster2] State: Waiting
  5459. 2020-11-02T12:06:54.8089245Z [cluster2] Reason: ImagePullBackOff
  5460. 2020-11-02T12:06:54.8089616Z [cluster2] Ready: False
  5461. 2020-11-02T12:06:54.8089926Z [cluster2] Restart Count: 0
  5462. 2020-11-02T12:06:54.8090266Z [cluster2] Environment: <none>
  5463. 2020-11-02T12:06:54.8090582Z [cluster2] Mounts:
  5464. 2020-11-02T12:06:54.8091084Z [cluster2] /etc/coredns from config-volume (ro)
  5465. 2020-11-02T12:06:54.8092429Z [cluster2] /var/run/secrets/kubernetes.io/serviceaccount from submariner-lighthouse-token-dv4vb (ro)
  5466. 2020-11-02T12:06:54.8093411Z [cluster2] Conditions:
  5467. 2020-11-02T12:06:54.8093722Z [cluster2] Type Status
  5468. 2020-11-02T12:06:54.8094052Z [cluster2] Initialized True
  5469. 2020-11-02T12:06:54.8094380Z [cluster2] Ready False
  5470. 2020-11-02T12:06:54.8094741Z [cluster2] ContainersReady False
  5471. 2020-11-02T12:06:54.8095280Z [cluster2] PodScheduled True
  5472. 2020-11-02T12:06:54.8095579Z [cluster2] Volumes:
  5473. 2020-11-02T12:06:54.8096059Z [cluster2] config-volume:
  5474. 2020-11-02T12:06:54.8096494Z [cluster2] Type: ConfigMap (a volume populated by a ConfigMap)
  5475. 2020-11-02T12:06:54.8097227Z [cluster2] Name: submariner-lighthouse-coredns
  5476. 2020-11-02T12:06:54.8097715Z [cluster2] Optional: false
  5477. 2020-11-02T12:06:54.8098383Z [cluster2] submariner-lighthouse-token-dv4vb:
  5478. 2020-11-02T12:06:54.8098989Z [cluster2] Type: Secret (a volume populated by a Secret)
  5479. 2020-11-02T12:06:54.8099768Z [cluster2] SecretName: submariner-lighthouse-token-dv4vb
  5480. 2020-11-02T12:06:54.8100320Z [cluster2] Optional: false
  5481. 2020-11-02T12:06:54.8100661Z [cluster2] QoS Class: BestEffort
  5482. 2020-11-02T12:06:54.8101155Z [cluster2] Node-Selectors: <none>
  5483. 2020-11-02T12:06:54.8102033Z [cluster2] Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
  5484. 2020-11-02T12:06:54.8102684Z [cluster2] node.kubernetes.io/unreachable:NoExecute for 300s
  5485. 2020-11-02T12:06:54.8103133Z [cluster2] Events:
  5486. 2020-11-02T12:06:54.8103592Z [cluster2] Type Reason Age From Message
  5487. 2020-11-02T12:06:54.8104329Z [cluster2] ---- ------ ---- ---- -------
  5488. 2020-11-02T12:06:54.8106034Z [cluster2] Normal Scheduled 9m35s default-scheduler Successfully assigned submariner-operator/submariner-lighthouse-coredns-557485fbc-fmn68 to cluster2-worker2
  5489. 2020-11-02T12:06:54.8107893Z [cluster2] Normal Pulling 7m32s (x4 over 9m32s) kubelet, cluster2-worker2 Pulling image "localhost:5000/lighthouse-coredns:local"
  5490. 2020-11-02T12:06:54.8110903Z [cluster2] Warning Failed 7m32s (x4 over 9m7s) kubelet, cluster2-worker2 Failed to pull image "localhost:5000/lighthouse-coredns:local": rpc error: code = Unknown desc = failed to pull and unpack image "localhost:5000/lighthouse-coredns:local": failed to resolve reference "localhost:5000/lighthouse-coredns:local": failed to do request: Head https://localhost:5000/v2/lighthouse-coredns/manifests/local: dial tcp 127.0.0.1:5000: connect: connection refused
  5491. 2020-11-02T12:06:54.8113588Z [cluster2] Warning Failed 7m32s (x4 over 9m7s) kubelet, cluster2-worker2 Error: ErrImagePull
  5492. 2020-11-02T12:06:54.8115174Z [cluster2] Normal BackOff 7m7s (x7 over 9m7s) kubelet, cluster2-worker2 Back-off pulling image "localhost:5000/lighthouse-coredns:local"
  5493. 2020-11-02T12:06:54.8116570Z [cluster2] Warning Failed 4m26s (x18 over 9m7s) kubelet, cluster2-worker2 Error: ImagePullBackOff
  5494. 2020-11-02T12:06:54.8117845Z [cluster2] ===================== END submariner-lighthouse-coredns-557485fbc-fmn68 - submariner-operator ==========================
  5495. 2020-11-02T12:06:54.8119022Z [cluster2] +++++++++++++++++++++: Logs for Pod kube-proxy-88gzn in namespace kube-system :++++++++++++++++++++++
  5496. 2020-11-02T12:06:54.8119626Z [cluster2] W1102 11:52:45.758828 1 server_others.go:323] Unknown proxy mode "", assuming iptables proxy
  5497. 2020-11-02T12:06:54.8120186Z [cluster2] I1102 11:52:45.768906 1 node.go:135] Successfully retrieved node IP: 172.17.0.4
  5498. 2020-11-02T12:06:54.8136976Z [submariner]$ [cluster2] kubectl -n kube-system logs kube-proxy-jm52h
  5499. 2020-11-02T12:06:54.8148161Z [submariner]$ [cluster2] kubectl -n kube-system logs kube-proxy-jm52h
  5500. 2020-11-02T12:06:54.8160675Z [submariner]$ [cluster2] command kubectl --context=cluster2 -n kube-system logs kube-proxy-jm52h
  5501. 2020-11-02T12:06:54.8171934Z [submariner]$ [cluster2] kubectl --context=cluster2 -n kube-system logs kube-proxy-jm52h
  5502. 2020-11-02T12:06:55.1386260Z [submariner]$ [cluster2] kubectl -n kube-system logs kube-proxy-mq6xw
  5503. 2020-11-02T12:06:55.1396271Z [submariner]$ [cluster2] kubectl -n kube-system logs kube-proxy-mq6xw
  5504. 2020-11-02T12:06:55.1408739Z [submariner]$ [cluster2] command kubectl --context=cluster2 -n kube-system logs kube-proxy-mq6xw
  5505. 2020-11-02T12:06:55.1418963Z [submariner]$ [cluster2] kubectl --context=cluster2 -n kube-system logs kube-proxy-mq6xw
  5506. 2020-11-02T12:06:55.4606329Z [submariner]$ [cluster2] namespace=submariner-operator
  5507. 2020-11-02T12:06:55.4621514Z [submariner]$ [cluster2] kubectl get pods --selector=name=submariner-operator -n submariner-operator -o jsonpath={.items[*].metadata.name}
  5508. 2020-11-02T12:06:55.4632756Z [submariner]$ [cluster2] kubectl get pods --selector=name=submariner-operator -n submariner-operator -o jsonpath={.items[*].metadata.name}
  5509. 2020-11-02T12:06:55.4646671Z [submariner]$ [cluster2] command kubectl --context=cluster2 get pods --selector=name=submariner-operator -n submariner-operator -o jsonpath={.items[*].metadata.name}
  5510. 2020-11-02T12:06:55.4672990Z [submariner]$ [cluster2] kubectl --context=cluster2 get pods --selector=name=submariner-operator -n submariner-operator -o jsonpath={.items[*].metadata.name}
  5511. 2020-11-02T12:06:55.7254892Z [cluster2] I1102 11:52:45.768936 1 server_others.go:145] Using iptables Proxier.
  5512. 2020-11-02T12:06:55.7255554Z [cluster2] I1102 11:52:45.769331 1 server.go:571] Version: v1.17.0
  5513. 2020-11-02T12:06:55.7256054Z [cluster2] I1102 11:52:45.769890 1 conntrack.go:52] Setting nf_conntrack_max to 131072
  5514. 2020-11-02T12:06:55.7258029Z [cluster2] I1102 11:52:45.770158 1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_established' to 86400
  5515. 2020-11-02T12:06:55.7259270Z [cluster2] I1102 11:52:45.770329 1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_close_wait' to 3600
  5516. 2020-11-02T12:06:55.7260171Z [cluster2] I1102 11:52:45.777439 1 config.go:313] Starting service config controller
  5517. 2020-11-02T12:06:55.7260965Z [cluster2] I1102 11:52:45.777541 1 shared_informer.go:197] Waiting for caches to sync for service config
  5518. 2020-11-02T12:06:55.7261984Z [cluster2] I1102 11:52:45.777641 1 config.go:131] Starting endpoints config controller
  5519. 2020-11-02T12:06:55.7262589Z [cluster2] I1102 11:52:45.777725 1 shared_informer.go:197] Waiting for caches to sync for endpoints config
  5520. 2020-11-02T12:06:55.7263201Z [cluster2] I1102 11:52:45.877802 1 shared_informer.go:204] Caches are synced for service config
  5521. 2020-11-02T12:06:55.7263803Z [cluster2] I1102 11:52:45.877935 1 shared_informer.go:204] Caches are synced for endpoints config
  5522. 2020-11-02T12:06:55.7264648Z [cluster2] +++++++++++++++++++++: Logs for Pod kube-proxy-jm52h in namespace kube-system :++++++++++++++++++++++
  5523. 2020-11-02T12:06:55.7265281Z [cluster2] W1102 11:52:25.320335 1 server_others.go:323] Unknown proxy mode "", assuming iptables proxy
  5524. 2020-11-02T12:06:55.7265800Z [cluster2] I1102 11:52:25.490645 1 node.go:135] Successfully retrieved node IP: 172.17.0.8
  5525. 2020-11-02T12:06:55.7266329Z [cluster2] I1102 11:52:25.498628 1 server_others.go:145] Using iptables Proxier.
  5526. 2020-11-02T12:06:55.7266800Z [cluster2] I1102 11:52:25.506167 1 server.go:571] Version: v1.17.0
  5527. 2020-11-02T12:06:55.7267307Z [cluster2] I1102 11:52:25.509223 1 conntrack.go:52] Setting nf_conntrack_max to 131072
  5528. 2020-11-02T12:06:55.7268189Z [cluster2] I1102 11:52:25.650897 1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_established' to 86400
  5529. 2020-11-02T12:06:55.7269320Z [cluster2] I1102 11:52:25.651100 1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_close_wait' to 3600
  5530. 2020-11-02T12:06:55.7270152Z [cluster2] I1102 11:52:25.659059 1 config.go:313] Starting service config controller
  5531. 2020-11-02T12:06:55.7270771Z [cluster2] I1102 11:52:25.659075 1 shared_informer.go:197] Waiting for caches to sync for service config
  5532. 2020-11-02T12:06:55.7271890Z [cluster2] I1102 11:52:25.659243 1 config.go:131] Starting endpoints config controller
  5533. 2020-11-02T12:06:55.7272530Z [cluster2] I1102 11:52:25.659254 1 shared_informer.go:197] Waiting for caches to sync for endpoints config
  5534. 2020-11-02T12:06:55.7273191Z [cluster2] I1102 11:52:25.780677 1 shared_informer.go:204] Caches are synced for endpoints config
  5535. 2020-11-02T12:06:55.7273867Z [cluster2] I1102 11:52:25.781107 1 shared_informer.go:204] Caches are synced for service config
  5536. 2020-11-02T12:06:55.7274902Z [cluster2] +++++++++++++++++++++: Logs for Pod kube-proxy-mq6xw in namespace kube-system :++++++++++++++++++++++
  5537. 2020-11-02T12:06:55.7276088Z [cluster2] W1102 11:52:50.625055 1 server_others.go:323] Unknown proxy mode "", assuming iptables proxy
  5538. 2020-11-02T12:06:55.7276688Z [cluster2] I1102 11:52:50.642177 1 node.go:135] Successfully retrieved node IP: 172.17.0.11
  5539. 2020-11-02T12:06:55.7277233Z [cluster2] I1102 11:52:50.642257 1 server_others.go:145] Using iptables Proxier.
  5540. 2020-11-02T12:06:55.7277750Z [cluster2] I1102 11:52:50.642712 1 server.go:571] Version: v1.17.0
  5541. 2020-11-02T12:06:55.7278463Z [cluster2] I1102 11:52:50.643420 1 conntrack.go:52] Setting nf_conntrack_max to 131072
  5542. 2020-11-02T12:06:55.7279966Z [cluster2] I1102 11:52:50.643876 1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_established' to 86400
  5543. 2020-11-02T12:06:55.7281039Z [cluster2] I1102 11:52:50.644216 1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_close_wait' to 3600
  5544. 2020-11-02T12:06:55.7281903Z [cluster2] I1102 11:52:50.648893 1 config.go:131] Starting endpoints config controller
  5545. 2020-11-02T12:06:55.7282525Z [cluster2] I1102 11:52:50.649015 1 shared_informer.go:197] Waiting for caches to sync for endpoints config
  5546. 2020-11-02T12:06:55.7283125Z [cluster2] I1102 11:52:50.649970 1 config.go:313] Starting service config controller
  5547. 2020-11-02T12:06:55.7283742Z [cluster2] I1102 11:52:50.649994 1 shared_informer.go:197] Waiting for caches to sync for service config
  5548. 2020-11-02T12:06:55.7284722Z [cluster2] I1102 11:52:50.763181 1 shared_informer.go:204] Caches are synced for service config
  5549. 2020-11-02T12:06:55.7285343Z [cluster2] I1102 11:52:50.763384 1 shared_informer.go:204] Caches are synced for endpoints config
  5550. 2020-11-02T12:06:55.7285815Z [cluster2] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  5551. 2020-11-02T12:06:55.7297431Z [submariner]$ [cluster2] kubectl -n submariner-operator logs submariner-operator-6675977db7-kqscw
  5552. 2020-11-02T12:06:55.7310666Z [submariner]$ [cluster2] kubectl -n submariner-operator logs submariner-operator-6675977db7-kqscw
  5553. 2020-11-02T12:06:55.7319448Z [submariner]$ [cluster2] command kubectl --context=cluster2 -n submariner-operator logs submariner-operator-6675977db7-kqscw
  5554. 2020-11-02T12:06:55.7332072Z [submariner]$ [cluster2] kubectl --context=cluster2 -n submariner-operator logs submariner-operator-6675977db7-kqscw
  5555. 2020-11-02T12:06:56.0095668Z [cluster2] +++++++++++++++++++++: Logs for Pod submariner-operator-6675977db7-kqscw in namespace submariner-operator :++++++++++++++++++++++
  5556. 2020-11-02T12:06:56.0099841Z [cluster2] {"level":"info","ts":1604318229.7687109,"logger":"cmd","msg":"Go Version: go1.15.2"}
  5557. 2020-11-02T12:06:56.0100685Z [cluster2] {"level":"info","ts":1604318229.7687614,"logger":"cmd","msg":"Go OS/Arch: linux/amd64"}
  5558. 2020-11-02T12:06:56.0103472Z [cluster2] {"level":"info","ts":1604318229.7687705,"logger":"cmd","msg":"Version of operator-sdk: v0.17.0"}
  5559. 2020-11-02T12:06:56.0104483Z [cluster2] {"level":"info","ts":1604318229.7687771,"logger":"cmd","msg":"Submariner operator version: was not built correctly"}
  5560. 2020-11-02T12:06:56.0105358Z [cluster2] {"level":"info","ts":1604318229.7689822,"logger":"leader","msg":"Trying to become the leader."}
  5561. 2020-11-02T12:06:56.0108859Z [cluster2] {"level":"info","ts":1604318230.1582227,"logger":"leader","msg":"No pre-existing lock was found."}
  5562. 2020-11-02T12:06:56.0112943Z [cluster2] {"level":"info","ts":1604318230.1719596,"logger":"leader","msg":"Became the leader."}
  5563. 2020-11-02T12:06:56.0115514Z [cluster2] {"level":"info","ts":1604318230.4758108,"logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":"0.0.0.0:8383"}
  5564. 2020-11-02T12:06:56.0136043Z [cluster2] {"level":"info","ts":1604318230.4767857,"logger":"cmd","msg":"Registering Components."}
  5565. 2020-11-02T12:06:56.0137160Z [cluster2] {"level":"info","ts":1604318230.4770434,"logger":"cmd","msg":"Creating the engine CRDs"}
  5566. 2020-11-02T12:06:56.0137718Z [cluster2] {"level":"info","ts":1604318231.0803795,"logger":"cmd","msg":"Creating the Lighthouse CRDs"}
  5567. 2020-11-02T12:06:56.0138341Z [cluster2] {"level":"info","ts":1604318232.0939777,"logger":"cmd","msg":"The Lighthouse CRDs were updated, restarting..."}
  5568. 2020-11-02T12:06:56.0138916Z [cluster2] {"level":"info","ts":1604318232.1625836,"logger":"cmd","msg":"Go Version: go1.15.2"}
  5569. 2020-11-02T12:06:56.0163107Z [cluster2] {"level":"info","ts":1604318232.162787,"logger":"cmd","msg":"Go OS/Arch: linux/amd64"}
  5570. 2020-11-02T12:06:56.0164664Z [cluster2] {"level":"info","ts":1604318232.16287,"logger":"cmd","msg":"Version of operator-sdk: v0.17.0"}
  5571. 2020-11-02T12:06:56.0165463Z [cluster2] {"level":"info","ts":1604318232.1629686,"logger":"cmd","msg":"Submariner operator version: was not built correctly"}
  5572. 2020-11-02T12:06:56.0166425Z [cluster2] {"level":"info","ts":1604318232.1631854,"logger":"leader","msg":"Trying to become the leader."}
  5573. 2020-11-02T12:06:56.0167265Z [cluster2] {"level":"info","ts":1604318233.036755,"logger":"leader","msg":"Found existing lock with my name. I was likely restarted."}
  5574. 2020-11-02T12:06:56.0168036Z [cluster2] {"level":"info","ts":1604318233.0367908,"logger":"leader","msg":"Continuing as the leader."}
  5575. 2020-11-02T12:06:56.0169466Z [cluster2] {"level":"info","ts":1604318233.540287,"logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":"0.0.0.0:8383"}
  5576. 2020-11-02T12:06:56.0170456Z [cluster2] {"level":"info","ts":1604318233.5406744,"logger":"cmd","msg":"Registering Components."}
  5577. 2020-11-02T12:06:56.0171193Z [cluster2] {"level":"info","ts":1604318233.540931,"logger":"cmd","msg":"Creating the engine CRDs"}
  5578. 2020-11-02T12:06:56.0171847Z [cluster2] {"level":"info","ts":1604318234.1421566,"logger":"cmd","msg":"Creating the Lighthouse CRDs"}
  5579. 2020-11-02T12:06:56.0189233Z [cluster2] {"level":"info","ts":1604318236.8820443,"logger":"metrics","msg":"Metrics Service object created","Service.Name":"submariner-operator-metrics","Service.Namespace":"submariner-operator"}
  5580. 2020-11-02T12:06:56.0192127Z [cluster2] {"level":"info","ts":1604318237.3837,"logger":"cmd","msg":"Could not create ServiceMonitor object","error":"no ServiceMonitor registered with the API"}
  5581. 2020-11-02T12:06:56.0193892Z [cluster2] {"level":"info","ts":1604318237.3838005,"logger":"cmd","msg":"Install prometheus-operator in your cluster to create ServiceMonitor objects","error":"no ServiceMonitor registered with the API"}
  5582. 2020-11-02T12:06:56.0198230Z [cluster2] {"level":"info","ts":1604318237.383864,"logger":"cmd","msg":"Starting the Cmd."}
  5583. 2020-11-02T12:06:56.0200862Z [cluster2] {"level":"info","ts":1604318237.3841653,"logger":"controller-runtime.manager","msg":"starting metrics server","path":"/metrics"}
  5584. 2020-11-02T12:06:56.0202939Z [cluster2] {"level":"info","ts":1604318237.3843696,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"servicediscovery-controller","source":"kind source: /, Kind="}
  5585. 2020-11-02T12:06:56.0208567Z [cluster2] {"level":"info","ts":1604318237.3848088,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"submariner-controller","source":"kind source: /, Kind="}
  5586. 2020-11-02T12:06:56.0210321Z [cluster2] {"level":"info","ts":1604318237.4849188,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"servicediscovery-controller","source":"kind source: /, Kind="}
  5587. 2020-11-02T12:06:56.0212211Z [cluster2] {"level":"info","ts":1604318237.4851623,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"submariner-controller","source":"kind source: /, Kind="}
  5588. 2020-11-02T12:06:56.0214471Z [cluster2] {"level":"info","ts":1604318237.5854006,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"submariner-controller","source":"kind source: /, Kind="}
  5589. 2020-11-02T12:06:56.0216024Z [cluster2] {"level":"info","ts":1604318237.5854168,"logger":"controller-runtime.controller","msg":"Starting Controller","controller":"servicediscovery-controller"}
  5590. 2020-11-02T12:06:56.0217497Z [cluster2] {"level":"info","ts":1604318237.6861446,"logger":"controller-runtime.controller","msg":"Starting Controller","controller":"submariner-controller"}
  5591. 2020-11-02T12:06:56.0219181Z [cluster2] {"level":"info","ts":1604318237.6861954,"logger":"controller-runtime.controller","msg":"Starting workers","controller":"submariner-controller","worker count":1}
  5592. 2020-11-02T12:06:56.0220640Z [cluster2] {"level":"info","ts":1604318237.686308,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5593. 2020-11-02T12:06:56.0222321Z [cluster2] {"level":"info","ts":1604318237.686407,"logger":"controller-runtime.controller","msg":"Starting workers","controller":"servicediscovery-controller","worker count":1}
  5594. 2020-11-02T12:06:56.0223444Z [cluster2] {"level":"info","ts":1604318237.7205145,"logger":"controller_submariner","msg":"Cluster network discovered"}
  5595. 2020-11-02T12:06:56.0224766Z [cluster2] {"level":"info","ts":1604318237.7205408,"logger":"controller_submariner","msg":"Discovered K8s network details","plugin":"weave-net","clusterCIDRs":["10.2.0.0/16"],"serviceCIDRs":["100.2.0.0/16"]}
  5596. 2020-11-02T12:06:56.0225627Z [cluster2] {"level":"info","ts":1604318237.7205572,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5597. 2020-11-02T12:06:56.0226332Z [cluster2] {"level":"info","ts":1604318237.7205615,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5598. 2020-11-02T12:06:56.0228125Z [cluster2] {"level":"info","ts":1604318237.7321188,"logger":"controller_submariner","msg":"Created a new DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5599. 2020-11-02T12:06:56.0230360Z [cluster2] {"level":"info","ts":1604318237.7405398,"logger":"controller_submariner","msg":"Created a new DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5600. 2020-11-02T12:06:56.0232429Z [cluster2] {"level":"info","ts":1604318237.9224005,"logger":"controller_submariner","msg":"Created Service Discovery CR","Request.Namespace":"submariner-operator","Request.Name":"submariner","Namespace":"submariner-operator","Name":"service-discovery"}
  5601. 2020-11-02T12:06:56.0234270Z [cluster2] {"level":"info","ts":1604318237.9234354,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5602. 2020-11-02T12:06:56.0235898Z [cluster2] {"level":"info","ts":1604318237.9305098,"logger":"controller_servicediscovery","msg":"Reconciling ServiceDiscovery","Request.Namespace":"submariner-operator","Request.Name":"service-discovery"}
  5603. 2020-11-02T12:06:56.0237178Z [cluster2] {"level":"info","ts":1604318237.9583037,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5604. 2020-11-02T12:06:56.0237881Z [cluster2] {"level":"info","ts":1604318237.9583328,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5605. 2020-11-02T12:06:56.0239688Z [cluster2] {"level":"info","ts":1604318237.9671462,"logger":"controller_servicediscovery","msg":"Created a new Deployment","Request.Namespace":"submariner-operator","Request.Name":"service-discovery","Deployment.Namespace":"submariner-operator","Deployment.Name":"submariner-lighthouse-agent"}
  5606. 2020-11-02T12:06:56.0245935Z [cluster2] {"level":"info","ts":1604318237.9829388,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5607. 2020-11-02T12:06:56.0249212Z [cluster2] {"level":"info","ts":1604318238.033454,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5608. 2020-11-02T12:06:56.0252132Z [cluster2] {"level":"info","ts":1604318238.0879323,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5609. 2020-11-02T12:06:56.0280868Z [cluster2] {"level":"info","ts":1604318238.1362453,"logger":"controller_servicediscovery","msg":"Created a new ConfigMap","Request.Namespace":"submariner-operator","Request.Name":"service-discovery","ConfigMap.Namespace":"submariner-operator","ConfigMap.Name":"submariner-lighthouse-coredns"}
  5610. 2020-11-02T12:06:56.0282407Z [cluster2] {"level":"info","ts":1604318238.1702645,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5611. 2020-11-02T12:06:56.0283456Z [cluster2] {"level":"info","ts":1604318238.170295,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5612. 2020-11-02T12:06:56.0285818Z [cluster2] {"level":"info","ts":1604318238.2008798,"logger":"controller_servicediscovery","msg":"Created a new Deployment","Request.Namespace":"submariner-operator","Request.Name":"service-discovery","Deployment.Namespace":"submariner-operator","Deployment.Name":"submariner-lighthouse-coredns"}
  5613. 2020-11-02T12:06:56.0288448Z [cluster2] {"level":"info","ts":1604318238.2995784,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5614. 2020-11-02T12:06:56.0291232Z [cluster2] {"level":"info","ts":1604318238.5157404,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5615. 2020-11-02T12:06:56.0293696Z [cluster2] {"level":"info","ts":1604318239.1984978,"logger":"controller_servicediscovery","msg":"Created a new Service","Request.Namespace":"submariner-operator","Request.Name":"service-discovery","Service.Namespace":"submariner-operator","Service.Name":"submariner-lighthouse-coredns"}
  5616. 2020-11-02T12:06:56.0296084Z [cluster2] {"level":"info","ts":1604318239.2962017,"logger":"controller_servicediscovery","msg":"coredns configmap does not have lighthouse configuration hence creating","Request.Namespace":"submariner-operator","Request.Name":"service-discovery"}
  5617. 2020-11-02T12:06:56.0299271Z [cluster2] {"level":"info","ts":1604318239.2962468,"logger":"controller_servicediscovery","msg":"Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n forward . 100.2.106.211\n}\n#lighthouse-end\n.:53 {\n errors\n health {\n lameduck 5s\n }\n ready\n kubernetes cluster2.local in-addr.arpa ip6.arpa {\n pods insecure\n fallthrough in-addr.arpa ip6.arpa\n ttl 30\n }\n prometheus :9153\n forward . /etc/resolv.conf\n cache 30\n loop\n reload\n loadbalance\n}\n"}
  5618. 2020-11-02T12:06:56.0302032Z [cluster2] {"level":"info","ts":1604318239.3631818,"logger":"controller_servicediscovery","msg":"Reconciling ServiceDiscovery","Request.Namespace":"submariner-operator","Request.Name":"service-discovery"}
  5619. 2020-11-02T12:06:56.0304807Z [cluster2] {"level":"info","ts":1604318239.7104967,"logger":"controller_servicediscovery","msg":"Updated existing Deployment","Request.Namespace":"submariner-operator","Request.Name":"service-discovery","Deployment.Namespace":"submariner-operator","Deployment.Name":"submariner-lighthouse-agent"}
  5620. 2020-11-02T12:06:56.0308109Z [cluster2] {"level":"info","ts":1604318239.8026316,"logger":"controller_servicediscovery","msg":"Updated existing Deployment","Request.Namespace":"submariner-operator","Request.Name":"service-discovery","Deployment.Namespace":"submariner-operator","Deployment.Name":"submariner-lighthouse-coredns"}
  5621. 2020-11-02T12:06:56.0310395Z [cluster2] {"level":"info","ts":1604318239.9087284,"logger":"controller_servicediscovery","msg":"coredns configmap has lighthouse configuration hence updating","Request.Namespace":"submariner-operator","Request.Name":"service-discovery"}
  5622. 2020-11-02T12:06:56.0313618Z [cluster2] {"level":"info","ts":1604318239.9087877,"logger":"controller_servicediscovery","msg":"Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n forward . 100.2.106.211\n}\n#lighthouse-end\n.:53 {\n errors\n health {\n lameduck 5s\n }\n ready\n kubernetes cluster2.local in-addr.arpa ip6.arpa {\n pods insecure\n fallthrough in-addr.arpa ip6.arpa\n ttl 30\n }\n prometheus :9153\n forward . /etc/resolv.conf\n cache 30\n loop\n reload\n loadbalance\n}\n\n"}
  5623. 2020-11-02T12:06:56.0316241Z [cluster2] {"level":"info","ts":1604318285.6041644,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5624. 2020-11-02T12:06:56.0317207Z [cluster2] {"level":"info","ts":1604318286.940363,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5625. 2020-11-02T12:06:56.0318061Z [cluster2] {"level":"info","ts":1604318286.940407,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5626. 2020-11-02T12:06:56.0319803Z [cluster2] {"level":"info","ts":1604318287.0829077,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5627. 2020-11-02T12:06:56.0322073Z [cluster2] {"level":"info","ts":1604318287.124731,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5628. 2020-11-02T12:06:56.0323775Z [cluster2] {"level":"info","ts":1604318287.2328327,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5629. 2020-11-02T12:06:56.0331331Z [cluster2] {"level":"error","ts":1604318287.2939565,"logger":"controller-runtime.controller","msg":"Reconciler error","controller":"submariner-controller","name":"submariner","namespace":"submariner-operator","error":"Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\tsubmariner-operator/vendor/github.com/go-logr/zapr/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsubmariner-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:248\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsubmariner-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:222\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\tsubmariner-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:201\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\tsubmariner-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\tsubmariner-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\tsubmariner-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"}
  5630. 2020-11-02T12:06:56.0337831Z [cluster2] {"level":"info","ts":1604318288.2947772,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5631. 2020-11-02T12:06:56.0338793Z [cluster2] {"level":"info","ts":1604318288.3103418,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5632. 2020-11-02T12:06:56.0339496Z [cluster2] {"level":"info","ts":1604318288.3103614,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5633. 2020-11-02T12:06:56.0341408Z [cluster2] {"level":"info","ts":1604318288.3202338,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5634. 2020-11-02T12:06:56.0343632Z [cluster2] {"level":"info","ts":1604318288.3248756,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5635. 2020-11-02T12:06:56.0345770Z [cluster2] {"level":"info","ts":1604318288.3520143,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5636. 2020-11-02T12:06:56.0348402Z [cluster2] {"level":"info","ts":1604318288.378756,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5637. 2020-11-02T12:06:56.0349318Z [cluster2] {"level":"info","ts":1604318288.378779,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5638. 2020-11-02T12:06:56.0351049Z [cluster2] {"level":"info","ts":1604318288.3883424,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5639. 2020-11-02T12:06:56.0353477Z [cluster2] {"level":"info","ts":1604318288.3952444,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5640. 2020-11-02T12:06:56.0355509Z [cluster2] {"level":"info","ts":1604318295.9329782,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5641. 2020-11-02T12:06:56.0356489Z [cluster2] {"level":"info","ts":1604318295.9645233,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5642. 2020-11-02T12:06:56.0357195Z [cluster2] {"level":"info","ts":1604318295.9645631,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5643. 2020-11-02T12:06:56.0358972Z [cluster2] {"level":"info","ts":1604318295.971916,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5644. 2020-11-02T12:06:56.0361775Z [cluster2] {"level":"info","ts":1604318296.008935,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5645. 2020-11-02T12:06:56.0363808Z [cluster2] {"level":"info","ts":1604318296.246312,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5646. 2020-11-02T12:06:56.0370887Z [cluster2] {"level":"info","ts":1604318296.5087566,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5647. 2020-11-02T12:06:56.0371873Z [cluster2] {"level":"info","ts":1604318296.5087867,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5648. 2020-11-02T12:06:56.0373823Z [cluster2] {"level":"info","ts":1604318296.5145016,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5649. 2020-11-02T12:06:56.0377228Z [cluster2] {"level":"info","ts":1604318296.5175915,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5650. 2020-11-02T12:06:56.0381032Z [cluster2] {"level":"info","ts":1604318301.0087547,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5651. 2020-11-02T12:06:56.0382168Z [cluster2] {"level":"info","ts":1604318301.01652,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5652. 2020-11-02T12:06:56.0382863Z [cluster2] {"level":"info","ts":1604318301.0165365,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5653. 2020-11-02T12:06:56.0385465Z [cluster2] {"level":"info","ts":1604318301.0259519,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5654. 2020-11-02T12:06:56.0388524Z [cluster2] {"level":"info","ts":1604318301.092624,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5655. 2020-11-02T12:06:56.0390940Z [cluster2] {"level":"info","ts":1604318302.814153,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5656. 2020-11-02T12:06:56.0392744Z [cluster2] {"level":"info","ts":1604318302.822411,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5657. 2020-11-02T12:06:56.0393565Z [cluster2] {"level":"info","ts":1604318302.822436,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5658. 2020-11-02T12:06:56.0395481Z [cluster2] {"level":"info","ts":1604318302.827071,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5659. 2020-11-02T12:06:56.0399237Z [cluster2] {"level":"info","ts":1604318302.829382,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5660. 2020-11-02T12:06:56.0401349Z [cluster2] {"level":"info","ts":1604318302.8400416,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5661. 2020-11-02T12:06:56.0402283Z [cluster2] {"level":"info","ts":1604318302.8615198,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5662. 2020-11-02T12:06:56.0402970Z [cluster2] {"level":"info","ts":1604318302.8615572,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5663. 2020-11-02T12:06:56.0405060Z [cluster2] {"level":"info","ts":1604318302.8728065,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5664. 2020-11-02T12:06:56.0407658Z [cluster2] {"level":"info","ts":1604318302.87828,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5665. 2020-11-02T12:06:56.0410320Z [cluster2] {"level":"info","ts":1604318306.0316827,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5666. 2020-11-02T12:06:56.0411838Z [cluster2] {"level":"info","ts":1604318306.0370908,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5667. 2020-11-02T12:06:56.0413324Z [cluster2] {"level":"info","ts":1604318306.0371077,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5668. 2020-11-02T12:06:56.0415227Z [cluster2] {"level":"info","ts":1604318306.0408075,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5669. 2020-11-02T12:06:56.0417865Z [cluster2] {"level":"info","ts":1604318306.0437155,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5670. 2020-11-02T12:06:56.0420034Z [cluster2] {"level":"info","ts":1604318306.0497377,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5671. 2020-11-02T12:06:56.0420975Z [cluster2] {"level":"info","ts":1604318306.05442,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5672. 2020-11-02T12:06:56.0421650Z [cluster2] {"level":"info","ts":1604318306.0544403,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5673. 2020-11-02T12:06:56.0425022Z [cluster2] {"level":"info","ts":1604318306.0581093,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5674. 2020-11-02T12:06:56.0428139Z [cluster2] {"level":"info","ts":1604318306.0608041,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5675. 2020-11-02T12:06:56.0430514Z [cluster2] {"level":"info","ts":1604318311.0441508,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5676. 2020-11-02T12:06:56.0431769Z [cluster2] {"level":"info","ts":1604318311.0517037,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5677. 2020-11-02T12:06:56.0432572Z [cluster2] {"level":"info","ts":1604318311.05173,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5678. 2020-11-02T12:06:56.0435920Z [cluster2] {"level":"info","ts":1604318311.0561366,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5679. 2020-11-02T12:06:56.0439195Z [cluster2] {"level":"info","ts":1604318311.0592642,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5680. 2020-11-02T12:06:56.0440953Z [cluster2] {"level":"info","ts":1604318311.0663598,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5681. 2020-11-02T12:06:56.0441874Z [cluster2] {"level":"info","ts":1604318311.0714755,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5682. 2020-11-02T12:06:56.0442724Z [cluster2] {"level":"info","ts":1604318311.0714955,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5683. 2020-11-02T12:06:56.0444930Z [cluster2] {"level":"info","ts":1604318311.075032,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5684. 2020-11-02T12:06:56.0447211Z [cluster2] {"level":"info","ts":1604318311.077434,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5685. 2020-11-02T12:06:56.0449117Z [cluster2] {"level":"info","ts":1604318316.0847528,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5686. 2020-11-02T12:06:56.0450031Z [cluster2] {"level":"info","ts":1604318316.0951912,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5687. 2020-11-02T12:06:56.0451078Z [cluster2] {"level":"info","ts":1604318316.09521,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5688. 2020-11-02T12:06:56.0452841Z [cluster2] {"level":"info","ts":1604318316.1020455,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5689. 2020-11-02T12:06:56.0459532Z [cluster2] {"level":"info","ts":1604318316.1061268,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5690. 2020-11-02T12:06:56.0461865Z [cluster2] {"level":"info","ts":1604318321.099738,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5691. 2020-11-02T12:06:56.0462813Z [cluster2] {"level":"info","ts":1604318321.1062489,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5692. 2020-11-02T12:06:56.0463499Z [cluster2] {"level":"info","ts":1604318321.1062756,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5693. 2020-11-02T12:06:56.0465792Z [cluster2] {"level":"info","ts":1604318321.1102285,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5694. 2020-11-02T12:06:56.0468631Z [cluster2] {"level":"info","ts":1604318321.1143878,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5695. 2020-11-02T12:06:56.0470513Z [cluster2] {"level":"info","ts":1604318326.129755,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5696. 2020-11-02T12:06:56.0471658Z [cluster2] {"level":"info","ts":1604318326.1434755,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5697. 2020-11-02T12:06:56.0472510Z [cluster2] {"level":"info","ts":1604318326.1435614,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5698. 2020-11-02T12:06:56.0474069Z [cluster2] {"level":"info","ts":1604318326.1476934,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5699. 2020-11-02T12:06:56.0476689Z [cluster2] {"level":"info","ts":1604318326.1538608,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5700. 2020-11-02T12:06:56.0479025Z [cluster2] {"level":"info","ts":1604318331.1490126,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5701. 2020-11-02T12:06:56.0480380Z [cluster2] {"level":"info","ts":1604318331.1575196,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5702. 2020-11-02T12:06:56.0481069Z [cluster2] {"level":"info","ts":1604318331.1575747,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5703. 2020-11-02T12:06:56.0482671Z [cluster2] {"level":"info","ts":1604318331.1618383,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5704. 2020-11-02T12:06:56.0486996Z [cluster2] {"level":"info","ts":1604318331.1663823,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5705. 2020-11-02T12:06:56.0489178Z [cluster2] {"level":"info","ts":1604318336.1739268,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5706. 2020-11-02T12:06:56.0490124Z [cluster2] {"level":"info","ts":1604318336.1838048,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5707. 2020-11-02T12:06:56.0490804Z [cluster2] {"level":"info","ts":1604318336.1838267,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5708. 2020-11-02T12:06:56.0492662Z [cluster2] {"level":"info","ts":1604318336.1949935,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5709. 2020-11-02T12:06:56.0497362Z [cluster2] {"level":"info","ts":1604318336.1979795,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5710. 2020-11-02T12:06:56.0499436Z [cluster2] {"level":"info","ts":1604318341.2000613,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5711. 2020-11-02T12:06:56.0500861Z [cluster2] {"level":"info","ts":1604318341.216953,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5712. 2020-11-02T12:06:56.0501708Z [cluster2] {"level":"info","ts":1604318341.2170157,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5713. 2020-11-02T12:06:56.0503751Z [cluster2] {"level":"info","ts":1604318341.2214396,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5714. 2020-11-02T12:06:56.0507231Z [cluster2] {"level":"info","ts":1604318341.2274563,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5715. 2020-11-02T12:06:56.0509865Z [cluster2] {"level":"info","ts":1604318346.2431395,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5716. 2020-11-02T12:06:56.0510963Z [cluster2] {"level":"info","ts":1604318346.250662,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5717. 2020-11-02T12:06:56.0511802Z [cluster2] {"level":"info","ts":1604318346.250702,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5718. 2020-11-02T12:06:56.0514752Z [cluster2] {"level":"info","ts":1604318346.270007,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5719. 2020-11-02T12:06:56.0517711Z [cluster2] {"level":"info","ts":1604318346.300458,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5720. 2020-11-02T12:06:56.0519614Z [cluster2] {"level":"info","ts":1604318351.2708654,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5721. 2020-11-02T12:06:56.0520517Z [cluster2] {"level":"info","ts":1604318351.2863832,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5722. 2020-11-02T12:06:56.0521170Z [cluster2] {"level":"info","ts":1604318351.2864096,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5723. 2020-11-02T12:06:56.0523235Z [cluster2] {"level":"info","ts":1604318351.2917852,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5724. 2020-11-02T12:06:56.0526017Z [cluster2] {"level":"info","ts":1604318351.2954109,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5725. 2020-11-02T12:06:56.0527910Z [cluster2] {"level":"info","ts":1604318356.2989821,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5726. 2020-11-02T12:06:56.0528812Z [cluster2] {"level":"info","ts":1604318356.3084774,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5727. 2020-11-02T12:06:56.0529475Z [cluster2] {"level":"info","ts":1604318356.3085175,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5728. 2020-11-02T12:06:56.0531529Z [cluster2] {"level":"info","ts":1604318356.313817,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5729. 2020-11-02T12:06:56.0533727Z [cluster2] {"level":"info","ts":1604318356.3175597,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5730. 2020-11-02T12:06:56.0535653Z [cluster2] {"level":"info","ts":1604318361.3154306,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5731. 2020-11-02T12:06:56.0536759Z [cluster2] {"level":"info","ts":1604318361.328801,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5732. 2020-11-02T12:06:56.0537769Z [cluster2] {"level":"info","ts":1604318361.3288362,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5733. 2020-11-02T12:06:56.0541370Z [cluster2] {"level":"info","ts":1604318361.336221,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5734. 2020-11-02T12:06:56.0543714Z [cluster2] {"level":"info","ts":1604318361.3419888,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5735. 2020-11-02T12:06:56.0545879Z [cluster2] {"level":"info","ts":1604318366.3598309,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5736. 2020-11-02T12:06:56.0546805Z [cluster2] {"level":"info","ts":1604318366.3739598,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5737. 2020-11-02T12:06:56.0547635Z [cluster2] {"level":"info","ts":1604318366.3739798,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5738. 2020-11-02T12:06:56.0549152Z [cluster2] {"level":"info","ts":1604318366.3798127,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5739. 2020-11-02T12:06:56.0551248Z [cluster2] {"level":"info","ts":1604318366.3840258,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5740. 2020-11-02T12:06:56.0553030Z [cluster2] {"level":"info","ts":1604318371.3831253,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5741. 2020-11-02T12:06:56.0553936Z [cluster2] {"level":"info","ts":1604318371.3933756,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5742. 2020-11-02T12:06:56.0554787Z [cluster2] {"level":"info","ts":1604318371.3934202,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5743. 2020-11-02T12:06:56.0556377Z [cluster2] {"level":"info","ts":1604318371.4049966,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5744. 2020-11-02T12:06:56.0558654Z [cluster2] {"level":"info","ts":1604318371.4180794,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5745. 2020-11-02T12:06:56.0560356Z [cluster2] {"level":"info","ts":1604318376.4114182,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5746. 2020-11-02T12:06:56.0561243Z [cluster2] {"level":"info","ts":1604318376.6085272,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5747. 2020-11-02T12:06:56.0561901Z [cluster2] {"level":"info","ts":1604318376.608573,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5748. 2020-11-02T12:06:56.0563410Z [cluster2] {"level":"info","ts":1604318376.632163,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5749. 2020-11-02T12:06:56.0566563Z [cluster2] {"level":"info","ts":1604318376.636222,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5750. 2020-11-02T12:06:56.0568440Z [cluster2] {"level":"info","ts":1604318381.639602,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5751. 2020-11-02T12:06:56.0569327Z [cluster2] {"level":"info","ts":1604318381.64883,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5752. 2020-11-02T12:06:56.0570111Z [cluster2] {"level":"info","ts":1604318381.6488464,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5753. 2020-11-02T12:06:56.0572319Z [cluster2] {"level":"info","ts":1604318381.6539946,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5754. 2020-11-02T12:06:56.0574690Z [cluster2] {"level":"info","ts":1604318381.6577628,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5755. 2020-11-02T12:06:56.0576438Z [cluster2] {"level":"info","ts":1604318386.6717463,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5756. 2020-11-02T12:06:56.0577746Z [cluster2] {"level":"info","ts":1604318386.6803725,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5757. 2020-11-02T12:06:56.0578570Z [cluster2] {"level":"info","ts":1604318386.6804378,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5758. 2020-11-02T12:06:56.0583845Z [cluster2] {"level":"info","ts":1604318386.6874518,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5759. 2020-11-02T12:06:56.0586583Z [cluster2] {"level":"info","ts":1604318386.6915927,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5760. 2020-11-02T12:06:56.0588913Z [cluster2] {"level":"info","ts":1604318391.7125306,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5761. 2020-11-02T12:06:56.0590074Z [cluster2] {"level":"info","ts":1604318391.7185655,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5762. 2020-11-02T12:06:56.0590786Z [cluster2] {"level":"info","ts":1604318391.7185895,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5763. 2020-11-02T12:06:56.0592733Z [cluster2] {"level":"info","ts":1604318391.725111,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5764. 2020-11-02T12:06:56.0595478Z [cluster2] {"level":"info","ts":1604318391.7286391,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5765. 2020-11-02T12:06:56.0598701Z [cluster2] {"level":"info","ts":1604318396.732688,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5766. 2020-11-02T12:06:56.0600181Z [cluster2] {"level":"info","ts":1604318396.7400784,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5767. 2020-11-02T12:06:56.0600888Z [cluster2] {"level":"info","ts":1604318396.7400963,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5768. 2020-11-02T12:06:56.0603726Z [cluster2] {"level":"info","ts":1604318396.7433941,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5769. 2020-11-02T12:06:56.0608549Z [cluster2] {"level":"info","ts":1604318396.7463562,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5770. 2020-11-02T12:06:56.0611182Z [cluster2] {"level":"info","ts":1604318401.7693715,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5771. 2020-11-02T12:06:56.0613970Z [cluster2] {"level":"info","ts":1604318401.7841163,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5772. 2020-11-02T12:06:56.0614797Z [cluster2] {"level":"info","ts":1604318401.7841678,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5773. 2020-11-02T12:06:56.0617437Z [cluster2] {"level":"info","ts":1604318401.7929869,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5774. 2020-11-02T12:06:56.0620680Z [cluster2] {"level":"info","ts":1604318401.7995265,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5775. 2020-11-02T12:06:56.0623174Z [cluster2] {"level":"info","ts":1604318406.7833855,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5776. 2020-11-02T12:06:56.0624106Z [cluster2] {"level":"info","ts":1604318406.7885756,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5777. 2020-11-02T12:06:56.0625328Z [cluster2] {"level":"info","ts":1604318406.7885947,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5778. 2020-11-02T12:06:56.0627102Z [cluster2] {"level":"info","ts":1604318406.7923248,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5779. 2020-11-02T12:06:56.0629478Z [cluster2] {"level":"info","ts":1604318406.7953732,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5780. 2020-11-02T12:06:56.0631224Z [cluster2] {"level":"info","ts":1604318411.8022847,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5781. 2020-11-02T12:06:56.0632141Z [cluster2] {"level":"info","ts":1604318411.8070514,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5782. 2020-11-02T12:06:56.0633180Z [cluster2] {"level":"info","ts":1604318411.807073,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5783. 2020-11-02T12:06:56.0634934Z [cluster2] {"level":"info","ts":1604318411.8102715,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5784. 2020-11-02T12:06:56.0637161Z [cluster2] {"level":"info","ts":1604318411.8128157,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5785. 2020-11-02T12:06:56.0639191Z [cluster2] {"level":"info","ts":1604318416.814719,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5786. 2020-11-02T12:06:56.0640185Z [cluster2] {"level":"info","ts":1604318416.819983,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5787. 2020-11-02T12:06:56.0640842Z [cluster2] {"level":"info","ts":1604318416.8200166,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5788. 2020-11-02T12:06:56.0642571Z [cluster2] {"level":"info","ts":1604318416.823737,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5789. 2020-11-02T12:06:56.0645116Z [cluster2] {"level":"info","ts":1604318416.8273637,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5790. 2020-11-02T12:06:56.0647445Z [cluster2] {"level":"info","ts":1604318421.8263118,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5791. 2020-11-02T12:06:56.0648370Z [cluster2] {"level":"info","ts":1604318421.8310814,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5792. 2020-11-02T12:06:56.0649144Z [cluster2] {"level":"info","ts":1604318421.8311038,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5793. 2020-11-02T12:06:56.0650745Z [cluster2] {"level":"info","ts":1604318421.834179,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5794. 2020-11-02T12:06:56.0653779Z [cluster2] {"level":"info","ts":1604318421.836984,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5795. 2020-11-02T12:06:56.0656771Z [cluster2] {"level":"info","ts":1604318426.839383,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5796. 2020-11-02T12:06:56.0657893Z [cluster2] {"level":"info","ts":1604318426.845886,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5797. 2020-11-02T12:06:56.0658574Z [cluster2] {"level":"info","ts":1604318426.8459086,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5798. 2020-11-02T12:06:56.0661545Z [cluster2] {"level":"info","ts":1604318426.852875,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5799. 2020-11-02T12:06:56.0665511Z [cluster2] {"level":"info","ts":1604318426.8592248,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5800. 2020-11-02T12:06:56.0668247Z [cluster2] {"level":"info","ts":1604318431.864197,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5801. 2020-11-02T12:06:56.0669449Z [cluster2] {"level":"info","ts":1604318431.873085,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5802. 2020-11-02T12:06:56.0670281Z [cluster2] {"level":"info","ts":1604318431.8731048,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5803. 2020-11-02T12:06:56.0672084Z [cluster2] {"level":"info","ts":1604318431.876541,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5804. 2020-11-02T12:06:56.0675758Z [cluster2] {"level":"info","ts":1604318431.8808937,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5805. 2020-11-02T12:06:56.0677713Z [cluster2] {"level":"info","ts":1604318436.9261434,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5806. 2020-11-02T12:06:56.0678696Z [cluster2] {"level":"info","ts":1604318436.9522429,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5807. 2020-11-02T12:06:56.0679422Z [cluster2] {"level":"info","ts":1604318436.9563127,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5808. 2020-11-02T12:06:56.0681080Z [cluster2] {"level":"info","ts":1604318436.9702277,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5809. 2020-11-02T12:06:56.0683799Z [cluster2] {"level":"info","ts":1604318436.9801407,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5810. 2020-11-02T12:06:56.0687620Z [cluster2] {"level":"info","ts":1604318441.9534786,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5811. 2020-11-02T12:06:56.0688776Z [cluster2] {"level":"info","ts":1604318441.9593425,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5812. 2020-11-02T12:06:56.0689477Z [cluster2] {"level":"info","ts":1604318441.9593596,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5813. 2020-11-02T12:06:56.0692758Z [cluster2] {"level":"info","ts":1604318441.962653,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5814. 2020-11-02T12:06:56.0696466Z [cluster2] {"level":"info","ts":1604318441.9657502,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5815. 2020-11-02T12:06:56.0698593Z [cluster2] {"level":"info","ts":1604318446.981309,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5816. 2020-11-02T12:06:56.0699766Z [cluster2] {"level":"info","ts":1604318446.9877727,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5817. 2020-11-02T12:06:56.0700624Z [cluster2] {"level":"info","ts":1604318446.9877903,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5818. 2020-11-02T12:06:56.0702186Z [cluster2] {"level":"info","ts":1604318447.0010087,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5819. 2020-11-02T12:06:56.0705501Z [cluster2] {"level":"info","ts":1604318447.0062165,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5820. 2020-11-02T12:06:56.0707751Z [cluster2] {"level":"info","ts":1604318452.0224931,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5821. 2020-11-02T12:06:56.0709160Z [cluster2] {"level":"info","ts":1604318452.02814,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5822. 2020-11-02T12:06:56.0709914Z [cluster2] {"level":"info","ts":1604318452.0281637,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5823. 2020-11-02T12:06:56.0712108Z [cluster2] {"level":"info","ts":1604318452.031573,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5824. 2020-11-02T12:06:56.0714269Z [cluster2] {"level":"info","ts":1604318452.0341642,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5825. 2020-11-02T12:06:56.0716209Z [cluster2] {"level":"info","ts":1604318457.1261694,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5826. 2020-11-02T12:06:56.0717376Z [cluster2] {"level":"info","ts":1604318457.1441407,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5827. 2020-11-02T12:06:56.0718398Z [cluster2] {"level":"info","ts":1604318457.144168,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5828. 2020-11-02T12:06:56.0720148Z [cluster2] {"level":"info","ts":1604318457.1551118,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5829. 2020-11-02T12:06:56.0723789Z [cluster2] {"level":"info","ts":1604318457.1572971,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5830. 2020-11-02T12:06:56.0727322Z [cluster2] {"level":"info","ts":1604318462.1502419,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5831. 2020-11-02T12:06:56.0729374Z [cluster2] {"level":"info","ts":1604318462.1599991,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5832. 2020-11-02T12:06:56.0730128Z [cluster2] {"level":"info","ts":1604318462.1600246,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5833. 2020-11-02T12:06:56.0732210Z [cluster2] {"level":"info","ts":1604318462.1645834,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5834. 2020-11-02T12:06:56.0735137Z [cluster2] {"level":"info","ts":1604318462.1680193,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5835. 2020-11-02T12:06:56.0738828Z [cluster2] {"level":"info","ts":1604318467.1733534,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5836. 2020-11-02T12:06:56.0740252Z [cluster2] {"level":"info","ts":1604318467.1783476,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5837. 2020-11-02T12:06:56.0741290Z [cluster2] {"level":"info","ts":1604318467.1783683,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5838. 2020-11-02T12:06:56.0742977Z [cluster2] {"level":"info","ts":1604318467.181538,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5839. 2020-11-02T12:06:56.0745286Z [cluster2] {"level":"info","ts":1604318467.1845741,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5840. 2020-11-02T12:06:56.0747590Z [cluster2] {"level":"info","ts":1604318472.1927404,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5841. 2020-11-02T12:06:56.0748491Z [cluster2] {"level":"info","ts":1604318472.2019007,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5842. 2020-11-02T12:06:56.0749147Z [cluster2] {"level":"info","ts":1604318472.2019196,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5843. 2020-11-02T12:06:56.0750859Z [cluster2] {"level":"info","ts":1604318472.2096677,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5844. 2020-11-02T12:06:56.0753130Z [cluster2] {"level":"info","ts":1604318472.2138948,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5845. 2020-11-02T12:06:56.0755086Z [cluster2] {"level":"info","ts":1604318474.6434555,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5846. 2020-11-02T12:06:56.0755976Z [cluster2] {"level":"info","ts":1604318474.6493413,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5847. 2020-11-02T12:06:56.0756637Z [cluster2] {"level":"info","ts":1604318474.6493628,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5848. 2020-11-02T12:06:56.0758329Z [cluster2] {"level":"info","ts":1604318474.6530566,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5849. 2020-11-02T12:06:56.0760622Z [cluster2] {"level":"info","ts":1604318474.6557236,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5850. 2020-11-02T12:06:56.0762446Z [cluster2] {"level":"info","ts":1604318474.6611974,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5851. 2020-11-02T12:06:56.0763311Z [cluster2] {"level":"info","ts":1604318474.6647744,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5852. 2020-11-02T12:06:56.0764092Z [cluster2] {"level":"info","ts":1604318474.6647985,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5853. 2020-11-02T12:06:56.0765793Z [cluster2] {"level":"info","ts":1604318474.6679327,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5854. 2020-11-02T12:06:56.0768244Z [cluster2] {"level":"info","ts":1604318474.6704311,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5855. 2020-11-02T12:06:56.0770736Z [cluster2] {"level":"info","ts":1604318476.1392298,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5856. 2020-11-02T12:06:56.0772420Z [cluster2] {"level":"info","ts":1604318476.1470664,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5857. 2020-11-02T12:06:56.0773069Z [cluster2] {"level":"info","ts":1604318476.1470914,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5858. 2020-11-02T12:06:56.0774778Z [cluster2] {"level":"info","ts":1604318476.1504478,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5859. 2020-11-02T12:06:56.0777590Z [cluster2] {"level":"info","ts":1604318476.1532385,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5860. 2020-11-02T12:06:56.0780364Z [cluster2] {"level":"info","ts":1604318476.1603146,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5861. 2020-11-02T12:06:56.0781666Z [cluster2] {"level":"info","ts":1604318476.1687531,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5862. 2020-11-02T12:06:56.0782647Z [cluster2] {"level":"info","ts":1604318476.1687696,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5863. 2020-11-02T12:06:56.0784884Z [cluster2] {"level":"info","ts":1604318476.1747887,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5864. 2020-11-02T12:06:56.0787123Z [cluster2] {"level":"info","ts":1604318476.1782014,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5865. 2020-11-02T12:06:56.0789540Z [cluster2] {"level":"info","ts":1604318487.0379586,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5866. 2020-11-02T12:06:56.0790489Z [cluster2] {"level":"info","ts":1604318487.0442076,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5867. 2020-11-02T12:06:56.0791184Z [cluster2] {"level":"info","ts":1604318487.0442429,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5868. 2020-11-02T12:06:56.0792933Z [cluster2] {"level":"info","ts":1604318487.0490859,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5869. 2020-11-02T12:06:56.0795713Z [cluster2] {"level":"info","ts":1604318487.054039,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5870. 2020-11-02T12:06:56.0797612Z [cluster2] {"level":"info","ts":1604318487.0656226,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5871. 2020-11-02T12:06:56.0798690Z [cluster2] {"level":"info","ts":1604318487.0741544,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5872. 2020-11-02T12:06:56.0799553Z [cluster2] {"level":"info","ts":1604318487.0741746,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5873. 2020-11-02T12:06:56.0801359Z [cluster2] {"level":"info","ts":1604318487.0795074,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5874. 2020-11-02T12:06:56.0805713Z [cluster2] {"level":"info","ts":1604318487.0850422,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5875. 2020-11-02T12:06:56.0807927Z [cluster2] {"level":"info","ts":1604318487.1513472,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5876. 2020-11-02T12:06:56.0808822Z [cluster2] {"level":"info","ts":1604318487.1567314,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5877. 2020-11-02T12:06:56.0809484Z [cluster2] {"level":"info","ts":1604318487.1567647,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5878. 2020-11-02T12:06:56.0811008Z [cluster2] {"level":"info","ts":1604318487.164374,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5879. 2020-11-02T12:06:56.0813233Z [cluster2] {"level":"info","ts":1604318487.1697643,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5880. 2020-11-02T12:06:56.0815173Z [cluster2] {"level":"info","ts":1604318487.179326,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5881. 2020-11-02T12:06:56.0816095Z [cluster2] {"level":"info","ts":1604318487.1948109,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5882. 2020-11-02T12:06:56.0816974Z [cluster2] {"level":"info","ts":1604318487.1948307,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5883. 2020-11-02T12:06:56.0818977Z [cluster2] {"level":"info","ts":1604318487.2006176,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5884. 2020-11-02T12:06:56.0821499Z [cluster2] {"level":"info","ts":1604318487.2040248,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5885. 2020-11-02T12:06:56.0823187Z [cluster2] {"level":"info","ts":1604318492.0507772,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5886. 2020-11-02T12:06:56.0824268Z [cluster2] {"level":"info","ts":1604318492.054882,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5887. 2020-11-02T12:06:56.0824944Z [cluster2] {"level":"info","ts":1604318492.0549042,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5888. 2020-11-02T12:06:56.0826503Z [cluster2] {"level":"info","ts":1604318492.058821,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5889. 2020-11-02T12:06:56.0829267Z [cluster2] {"level":"info","ts":1604318492.062938,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5890. 2020-11-02T12:06:56.0831675Z [cluster2] {"level":"info","ts":1604318497.0585873,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5891. 2020-11-02T12:06:56.0832828Z [cluster2] {"level":"info","ts":1604318497.0624697,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5892. 2020-11-02T12:06:56.0834077Z [cluster2] {"level":"info","ts":1604318497.0625093,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5893. 2020-11-02T12:06:56.0836109Z [cluster2] {"level":"info","ts":1604318497.0657501,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5894. 2020-11-02T12:06:56.0838703Z [cluster2] {"level":"info","ts":1604318497.068518,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5895. 2020-11-02T12:06:56.0840452Z [cluster2] {"level":"info","ts":1604318497.0758984,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5896. 2020-11-02T12:06:56.0841462Z [cluster2] {"level":"info","ts":1604318497.0814207,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5897. 2020-11-02T12:06:56.0842129Z [cluster2] {"level":"info","ts":1604318497.0814743,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5898. 2020-11-02T12:06:56.0844129Z [cluster2] {"level":"info","ts":1604318497.085167,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5899. 2020-11-02T12:06:56.0846760Z [cluster2] {"level":"info","ts":1604318497.0889406,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5900. 2020-11-02T12:06:56.0848702Z [cluster2] {"level":"info","ts":1604318502.0784574,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5901. 2020-11-02T12:06:56.0849965Z [cluster2] {"level":"info","ts":1604318502.0838156,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5902. 2020-11-02T12:06:56.0850668Z [cluster2] {"level":"info","ts":1604318502.0838397,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5903. 2020-11-02T12:06:56.0852503Z [cluster2] {"level":"info","ts":1604318502.0876544,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5904. 2020-11-02T12:06:56.0854924Z [cluster2] {"level":"info","ts":1604318502.0991516,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5905. 2020-11-02T12:06:56.0857155Z [cluster2] {"level":"info","ts":1604318507.091275,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5906. 2020-11-02T12:06:56.0858494Z [cluster2] {"level":"info","ts":1604318507.097697,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5907. 2020-11-02T12:06:56.0859213Z [cluster2] {"level":"info","ts":1604318507.0977206,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5908. 2020-11-02T12:06:56.0861519Z [cluster2] {"level":"info","ts":1604318507.1012843,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5909. 2020-11-02T12:06:56.0865285Z [cluster2] {"level":"info","ts":1604318507.1042125,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5910. 2020-11-02T12:06:56.0867176Z [cluster2] {"level":"info","ts":1604318512.1030016,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5911. 2020-11-02T12:06:56.0868308Z [cluster2] {"level":"info","ts":1604318512.1142976,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5912. 2020-11-02T12:06:56.0869014Z [cluster2] {"level":"info","ts":1604318512.1143317,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5913. 2020-11-02T12:06:56.0870887Z [cluster2] {"level":"info","ts":1604318512.126818,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5914. 2020-11-02T12:06:56.0873092Z [cluster2] {"level":"info","ts":1604318512.129473,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5915. 2020-11-02T12:06:56.0875234Z [cluster2] {"level":"info","ts":1604318517.128043,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5916. 2020-11-02T12:06:56.0876210Z [cluster2] {"level":"info","ts":1604318517.1339386,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5917. 2020-11-02T12:06:56.0876936Z [cluster2] {"level":"info","ts":1604318517.1339612,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5918. 2020-11-02T12:06:56.0880395Z [cluster2] {"level":"info","ts":1604318517.137874,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5919. 2020-11-02T12:06:56.0883256Z [cluster2] {"level":"info","ts":1604318517.1410396,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5920. 2020-11-02T12:06:56.0886176Z [cluster2] {"level":"info","ts":1604318522.1396723,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5921. 2020-11-02T12:06:56.0887213Z [cluster2] {"level":"info","ts":1604318522.146776,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5922. 2020-11-02T12:06:56.0888124Z [cluster2] {"level":"info","ts":1604318522.1468084,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5923. 2020-11-02T12:06:56.0890485Z [cluster2] {"level":"info","ts":1604318522.1528304,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5924. 2020-11-02T12:06:56.0931306Z [cluster2] {"level":"info","ts":1604318522.1575432,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5925. 2020-11-02T12:06:56.0933616Z [cluster2] {"level":"info","ts":1604318527.1522374,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5926. 2020-11-02T12:06:56.0934992Z [cluster2] {"level":"info","ts":1604318527.1563585,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5927. 2020-11-02T12:06:56.0935675Z [cluster2] {"level":"info","ts":1604318527.1563778,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5928. 2020-11-02T12:06:56.0937819Z [cluster2] {"level":"info","ts":1604318527.160632,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5929. 2020-11-02T12:06:56.0940536Z [cluster2] {"level":"info","ts":1604318527.163556,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5930. 2020-11-02T12:06:56.0942638Z [cluster2] {"level":"info","ts":1604318532.1649573,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5931. 2020-11-02T12:06:56.0944037Z [cluster2] {"level":"info","ts":1604318532.171514,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5932. 2020-11-02T12:06:56.0944932Z [cluster2] {"level":"info","ts":1604318532.1715357,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5933. 2020-11-02T12:06:56.0946826Z [cluster2] {"level":"info","ts":1604318532.1763625,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5934. 2020-11-02T12:06:56.0949892Z [cluster2] {"level":"info","ts":1604318532.1789436,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5935. 2020-11-02T12:06:56.0951589Z [cluster2] {"level":"info","ts":1604318537.183994,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5936. 2020-11-02T12:06:56.0952482Z [cluster2] {"level":"info","ts":1604318537.190527,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5937. 2020-11-02T12:06:56.0953326Z [cluster2] {"level":"info","ts":1604318537.1905499,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5938. 2020-11-02T12:06:56.0955178Z [cluster2] {"level":"info","ts":1604318537.1943746,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5939. 2020-11-02T12:06:56.0957278Z [cluster2] {"level":"info","ts":1604318537.1971526,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5940. 2020-11-02T12:06:56.0959416Z [cluster2] {"level":"info","ts":1604318542.1962874,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5941. 2020-11-02T12:06:56.0960365Z [cluster2] {"level":"info","ts":1604318542.2038355,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5942. 2020-11-02T12:06:56.0961360Z [cluster2] {"level":"info","ts":1604318542.2038877,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5943. 2020-11-02T12:06:56.0984451Z [cluster2] {"level":"info","ts":1604318542.2080884,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5944. 2020-11-02T12:06:56.0987022Z [cluster2] {"level":"info","ts":1604318542.2105334,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5945. 2020-11-02T12:06:56.0988912Z [cluster2] {"level":"info","ts":1604318547.210081,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5946. 2020-11-02T12:06:56.0989974Z [cluster2] {"level":"info","ts":1604318547.2159314,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5947. 2020-11-02T12:06:56.0990653Z [cluster2] {"level":"info","ts":1604318547.2159488,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5948. 2020-11-02T12:06:56.0992350Z [cluster2] {"level":"info","ts":1604318547.2208295,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5949. 2020-11-02T12:06:56.0994644Z [cluster2] {"level":"info","ts":1604318547.223642,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5950. 2020-11-02T12:06:56.0996344Z [cluster2] {"level":"info","ts":1604318552.2282915,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5951. 2020-11-02T12:06:56.0997249Z [cluster2] {"level":"info","ts":1604318552.2357976,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5952. 2020-11-02T12:06:56.0997909Z [cluster2] {"level":"info","ts":1604318552.2358146,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5953. 2020-11-02T12:06:56.0999424Z [cluster2] {"level":"info","ts":1604318552.2396564,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5954. 2020-11-02T12:06:56.1001702Z [cluster2] {"level":"info","ts":1604318552.2426288,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5955. 2020-11-02T12:06:56.1003451Z [cluster2] {"level":"info","ts":1604318557.243989,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5956. 2020-11-02T12:06:56.1004579Z [cluster2] {"level":"info","ts":1604318557.2515872,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5957. 2020-11-02T12:06:56.1005266Z [cluster2] {"level":"info","ts":1604318557.25161,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5958. 2020-11-02T12:06:56.1006855Z [cluster2] {"level":"info","ts":1604318557.2552,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5959. 2020-11-02T12:06:56.1009123Z [cluster2] {"level":"info","ts":1604318557.2583103,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5960. 2020-11-02T12:06:56.1011642Z [cluster2] {"level":"info","ts":1604318562.2662613,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5961. 2020-11-02T12:06:56.1012569Z [cluster2] {"level":"info","ts":1604318562.2718549,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5962. 2020-11-02T12:06:56.1013247Z [cluster2] {"level":"info","ts":1604318562.2718747,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5963. 2020-11-02T12:06:56.1015111Z [cluster2] {"level":"info","ts":1604318562.2753592,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5964. 2020-11-02T12:06:56.1017387Z [cluster2] {"level":"info","ts":1604318562.278427,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5965. 2020-11-02T12:06:56.1019347Z [cluster2] {"level":"info","ts":1604318567.2811809,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5966. 2020-11-02T12:06:56.1020392Z [cluster2] {"level":"info","ts":1604318567.2880874,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5967. 2020-11-02T12:06:56.1021030Z [cluster2] {"level":"info","ts":1604318567.2881064,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5968. 2020-11-02T12:06:56.1022527Z [cluster2] {"level":"info","ts":1604318567.291996,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5969. 2020-11-02T12:06:56.1024734Z [cluster2] {"level":"info","ts":1604318567.2950864,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5970. 2020-11-02T12:06:56.1026431Z [cluster2] {"level":"info","ts":1604318572.293237,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5971. 2020-11-02T12:06:56.1027320Z [cluster2] {"level":"info","ts":1604318572.2982233,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5972. 2020-11-02T12:06:56.1028125Z [cluster2] {"level":"info","ts":1604318572.2982569,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5973. 2020-11-02T12:06:56.1029583Z [cluster2] {"level":"info","ts":1604318572.301741,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5974. 2020-11-02T12:06:56.1031619Z [cluster2] {"level":"info","ts":1604318572.3047633,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5975. 2020-11-02T12:06:56.1033480Z [cluster2] {"level":"info","ts":1604318577.3077836,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5976. 2020-11-02T12:06:56.1043470Z [cluster2] {"level":"info","ts":1604318577.3144512,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5977. 2020-11-02T12:06:56.1044441Z [cluster2] {"level":"info","ts":1604318577.314523,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5978. 2020-11-02T12:06:56.1046634Z [cluster2] {"level":"info","ts":1604318577.318493,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5979. 2020-11-02T12:06:56.1050072Z [cluster2] {"level":"info","ts":1604318577.321176,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5980. 2020-11-02T12:06:56.1052577Z [cluster2] {"level":"info","ts":1604318582.3249705,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5981. 2020-11-02T12:06:56.1053765Z [cluster2] {"level":"info","ts":1604318582.3299987,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5982. 2020-11-02T12:06:56.1055025Z [cluster2] {"level":"info","ts":1604318582.3300216,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5983. 2020-11-02T12:06:56.1057065Z [cluster2] {"level":"info","ts":1604318582.3331516,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5984. 2020-11-02T12:06:56.1059823Z [cluster2] {"level":"info","ts":1604318582.3359227,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5985. 2020-11-02T12:06:56.1061641Z [cluster2] {"level":"info","ts":1604318587.3370643,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5986. 2020-11-02T12:06:56.1062759Z [cluster2] {"level":"info","ts":1604318587.3438668,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5987. 2020-11-02T12:06:56.1083593Z [cluster2] {"level":"info","ts":1604318587.3439054,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5988. 2020-11-02T12:06:56.1086002Z [cluster2] {"level":"info","ts":1604318587.347123,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5989. 2020-11-02T12:06:56.1088489Z [cluster2] {"level":"info","ts":1604318587.3500443,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5990. 2020-11-02T12:06:56.1090312Z [cluster2] {"level":"info","ts":1604318592.360225,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5991. 2020-11-02T12:06:56.1091447Z [cluster2] {"level":"info","ts":1604318592.3749676,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5992. 2020-11-02T12:06:56.1092182Z [cluster2] {"level":"info","ts":1604318592.3749897,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5993. 2020-11-02T12:06:56.1094368Z [cluster2] {"level":"info","ts":1604318592.3796895,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5994. 2020-11-02T12:06:56.1096887Z [cluster2] {"level":"info","ts":1604318592.3824396,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  5995. 2020-11-02T12:06:56.1098944Z [cluster2] {"level":"info","ts":1604318597.3812504,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  5996. 2020-11-02T12:06:56.1100103Z [cluster2] {"level":"info","ts":1604318597.3872397,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  5997. 2020-11-02T12:06:56.1100824Z [cluster2] {"level":"info","ts":1604318597.3872628,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  5998. 2020-11-02T12:06:56.1102826Z [cluster2] {"level":"info","ts":1604318597.3906882,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  5999. 2020-11-02T12:06:56.1106333Z [cluster2] {"level":"info","ts":1604318597.3931649,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6000. 2020-11-02T12:06:56.1108243Z [cluster2] {"level":"info","ts":1604318602.392939,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6001. 2020-11-02T12:06:56.1109396Z [cluster2] {"level":"info","ts":1604318602.3987408,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6002. 2020-11-02T12:06:56.1110096Z [cluster2] {"level":"info","ts":1604318602.3987641,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6003. 2020-11-02T12:06:56.1112094Z [cluster2] {"level":"info","ts":1604318602.402304,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6004. 2020-11-02T12:06:56.1114657Z [cluster2] {"level":"info","ts":1604318602.4062395,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6005. 2020-11-02T12:06:56.1116641Z [cluster2] {"level":"info","ts":1604318607.4207113,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6006. 2020-11-02T12:06:56.1117744Z [cluster2] {"level":"info","ts":1604318607.4263535,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6007. 2020-11-02T12:06:56.1118419Z [cluster2] {"level":"info","ts":1604318607.4263902,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6008. 2020-11-02T12:06:56.1143742Z [cluster2] {"level":"info","ts":1604318607.430132,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6009. 2020-11-02T12:06:56.1146247Z [cluster2] {"level":"info","ts":1604318607.4329524,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6010. 2020-11-02T12:06:56.1148162Z [cluster2] {"level":"info","ts":1604318612.4343913,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6011. 2020-11-02T12:06:56.1149062Z [cluster2] {"level":"info","ts":1604318612.4414108,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6012. 2020-11-02T12:06:56.1150019Z [cluster2] {"level":"info","ts":1604318612.4414701,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6013. 2020-11-02T12:06:56.1151567Z [cluster2] {"level":"info","ts":1604318612.4444716,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6014. 2020-11-02T12:06:56.1153868Z [cluster2] {"level":"info","ts":1604318612.447216,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6015. 2020-11-02T12:06:56.1155576Z [cluster2] {"level":"info","ts":1604318617.4485521,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6016. 2020-11-02T12:06:56.1156476Z [cluster2] {"level":"info","ts":1604318617.4594667,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6017. 2020-11-02T12:06:56.1157121Z [cluster2] {"level":"info","ts":1604318617.4594846,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6018. 2020-11-02T12:06:56.1158738Z [cluster2] {"level":"info","ts":1604318617.4671614,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6019. 2020-11-02T12:06:56.1160874Z [cluster2] {"level":"info","ts":1604318617.4694118,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6020. 2020-11-02T12:06:56.1162587Z [cluster2] {"level":"info","ts":1604318622.4696164,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6021. 2020-11-02T12:06:56.1163471Z [cluster2] {"level":"info","ts":1604318622.475874,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6022. 2020-11-02T12:06:56.1164264Z [cluster2] {"level":"info","ts":1604318622.4758947,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6023. 2020-11-02T12:06:56.1166000Z [cluster2] {"level":"info","ts":1604318622.4789238,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6024. 2020-11-02T12:06:56.1168156Z [cluster2] {"level":"info","ts":1604318622.4830453,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6025. 2020-11-02T12:06:56.1203015Z [cluster2] {"level":"info","ts":1604318627.483611,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6026. 2020-11-02T12:06:56.1205996Z [cluster2] {"level":"info","ts":1604318627.4907422,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6027. 2020-11-02T12:06:56.1206854Z [cluster2] {"level":"info","ts":1604318627.4907622,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6028. 2020-11-02T12:06:56.1209853Z [cluster2] {"level":"info","ts":1604318627.4945884,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6029. 2020-11-02T12:06:56.1212808Z [cluster2] {"level":"info","ts":1604318627.4997823,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6030. 2020-11-02T12:06:56.1215199Z [cluster2] {"level":"info","ts":1604318632.4972007,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6031. 2020-11-02T12:06:56.1216094Z [cluster2] {"level":"info","ts":1604318632.503844,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6032. 2020-11-02T12:06:56.1216755Z [cluster2] {"level":"info","ts":1604318632.5038676,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6033. 2020-11-02T12:06:56.1218272Z [cluster2] {"level":"info","ts":1604318632.50752,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6034. 2020-11-02T12:06:56.1220738Z [cluster2] {"level":"info","ts":1604318632.5110502,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6035. 2020-11-02T12:06:56.1222847Z [cluster2] {"level":"info","ts":1604318637.5112233,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6036. 2020-11-02T12:06:56.1223922Z [cluster2] {"level":"info","ts":1604318637.517421,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6037. 2020-11-02T12:06:56.1224629Z [cluster2] {"level":"info","ts":1604318637.5174453,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6038. 2020-11-02T12:06:56.1226288Z [cluster2] {"level":"info","ts":1604318637.5206175,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6039. 2020-11-02T12:06:56.1228635Z [cluster2] {"level":"info","ts":1604318637.5234554,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6040. 2020-11-02T12:06:56.1230468Z [cluster2] {"level":"info","ts":1604318642.5294297,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6041. 2020-11-02T12:06:56.1231417Z [cluster2] {"level":"info","ts":1604318642.5376332,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6042. 2020-11-02T12:06:56.1232117Z [cluster2] {"level":"info","ts":1604318642.5376542,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6043. 2020-11-02T12:06:56.1233718Z [cluster2] {"level":"info","ts":1604318642.5410213,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6044. 2020-11-02T12:06:56.1236138Z [cluster2] {"level":"info","ts":1604318642.5452147,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6045. 2020-11-02T12:06:56.1238273Z [cluster2] {"level":"info","ts":1604318647.5452912,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6046. 2020-11-02T12:06:56.1239221Z [cluster2] {"level":"info","ts":1604318647.5513785,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6047. 2020-11-02T12:06:56.1240033Z [cluster2] {"level":"info","ts":1604318647.5514128,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6048. 2020-11-02T12:06:56.1242365Z [cluster2] {"level":"info","ts":1604318647.554646,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6049. 2020-11-02T12:06:56.1254139Z [cluster2] {"level":"info","ts":1604318647.5580668,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6050. 2020-11-02T12:06:56.1275603Z [cluster2] {"level":"info","ts":1604318652.558559,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6051. 2020-11-02T12:06:56.1276793Z [cluster2] {"level":"info","ts":1604318652.563178,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6052. 2020-11-02T12:06:56.1277499Z [cluster2] {"level":"info","ts":1604318652.5632002,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6053. 2020-11-02T12:06:56.1279600Z [cluster2] {"level":"info","ts":1604318652.5664778,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6054. 2020-11-02T12:06:56.1281831Z [cluster2] {"level":"info","ts":1604318652.5694087,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6055. 2020-11-02T12:06:56.1284234Z [cluster2] {"level":"info","ts":1604318657.571349,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6056. 2020-11-02T12:06:56.1285419Z [cluster2] {"level":"info","ts":1604318657.577237,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6057. 2020-11-02T12:06:56.1286496Z [cluster2] {"level":"info","ts":1604318657.5772572,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6058. 2020-11-02T12:06:56.1288355Z [cluster2] {"level":"info","ts":1604318657.5800772,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6059. 2020-11-02T12:06:56.1290785Z [cluster2] {"level":"info","ts":1604318657.5826166,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6060. 2020-11-02T12:06:56.1292761Z [cluster2] {"level":"info","ts":1604318662.581996,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6061. 2020-11-02T12:06:56.1293932Z [cluster2] {"level":"info","ts":1604318662.5877328,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6062. 2020-11-02T12:06:56.1294653Z [cluster2] {"level":"info","ts":1604318662.587751,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6063. 2020-11-02T12:06:56.1296797Z [cluster2] {"level":"info","ts":1604318662.5915363,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6064. 2020-11-02T12:06:56.1299177Z [cluster2] {"level":"info","ts":1604318662.5939476,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6065. 2020-11-02T12:06:56.1301106Z [cluster2] {"level":"info","ts":1604318667.5970268,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6066. 2020-11-02T12:06:56.1302032Z [cluster2] {"level":"info","ts":1604318667.6034563,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6067. 2020-11-02T12:06:56.1302711Z [cluster2] {"level":"info","ts":1604318667.6034775,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6068. 2020-11-02T12:06:56.1304733Z [cluster2] {"level":"info","ts":1604318667.6070948,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6069. 2020-11-02T12:06:56.1307278Z [cluster2] {"level":"info","ts":1604318667.6100006,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6070. 2020-11-02T12:06:56.1309073Z [cluster2] {"level":"info","ts":1604318672.6150508,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6071. 2020-11-02T12:06:56.1309996Z [cluster2] {"level":"info","ts":1604318672.620814,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6072. 2020-11-02T12:06:56.1310678Z [cluster2] {"level":"info","ts":1604318672.6208327,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6073. 2020-11-02T12:06:56.1317244Z [cluster2] {"level":"info","ts":1604318672.625618,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6074. 2020-11-02T12:06:56.1319844Z [cluster2] {"level":"info","ts":1604318672.6283486,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6075. 2020-11-02T12:06:56.1321804Z [cluster2] {"level":"info","ts":1604318677.6296334,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6076. 2020-11-02T12:06:56.1322737Z [cluster2] {"level":"info","ts":1604318677.634682,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6077. 2020-11-02T12:06:56.1323841Z [cluster2] {"level":"info","ts":1604318677.6347048,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6078. 2020-11-02T12:06:56.1325726Z [cluster2] {"level":"info","ts":1604318677.6382263,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6079. 2020-11-02T12:06:56.1328086Z [cluster2] {"level":"info","ts":1604318677.6408148,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6080. 2020-11-02T12:06:56.1329833Z [cluster2] {"level":"info","ts":1604318682.6431375,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6081. 2020-11-02T12:06:56.1330753Z [cluster2] {"level":"info","ts":1604318682.6498837,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6082. 2020-11-02T12:06:56.1331604Z [cluster2] {"level":"info","ts":1604318682.6499047,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6083. 2020-11-02T12:06:56.1333617Z [cluster2] {"level":"info","ts":1604318682.657563,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6084. 2020-11-02T12:06:56.1335925Z [cluster2] {"level":"info","ts":1604318682.660953,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6085. 2020-11-02T12:06:56.1338176Z [cluster2] {"level":"info","ts":1604318687.6550744,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6086. 2020-11-02T12:06:56.1339195Z [cluster2] {"level":"info","ts":1604318687.6595902,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6087. 2020-11-02T12:06:56.1340339Z [cluster2] {"level":"info","ts":1604318687.659606,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6088. 2020-11-02T12:06:56.1342491Z [cluster2] {"level":"info","ts":1604318687.6635952,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6089. 2020-11-02T12:06:56.1345114Z [cluster2] {"level":"info","ts":1604318687.6655812,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6090. 2020-11-02T12:06:56.1348673Z [cluster2] {"level":"info","ts":1604318692.6713357,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6091. 2020-11-02T12:06:56.1349664Z [cluster2] {"level":"info","ts":1604318692.6808522,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6092. 2020-11-02T12:06:56.1350365Z [cluster2] {"level":"info","ts":1604318692.6808684,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6093. 2020-11-02T12:06:56.1352232Z [cluster2] {"level":"info","ts":1604318692.6910768,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6094. 2020-11-02T12:06:56.1355235Z [cluster2] {"level":"info","ts":1604318692.6934457,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6095. 2020-11-02T12:06:56.1357519Z [cluster2] {"level":"info","ts":1604318694.9848084,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6096. 2020-11-02T12:06:56.1358502Z [cluster2] {"level":"info","ts":1604318695.0017438,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6097. 2020-11-02T12:06:56.1359226Z [cluster2] {"level":"info","ts":1604318695.0017655,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6098. 2020-11-02T12:06:56.1361618Z [cluster2] {"level":"info","ts":1604318695.0058556,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6099. 2020-11-02T12:06:56.1364836Z [cluster2] {"level":"info","ts":1604318695.0169835,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6100. 2020-11-02T12:06:56.1367473Z [cluster2] {"level":"info","ts":1604318695.0232012,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6101. 2020-11-02T12:06:56.1368524Z [cluster2] {"level":"info","ts":1604318695.0279605,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6102. 2020-11-02T12:06:56.1369296Z [cluster2] {"level":"info","ts":1604318695.0279806,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6103. 2020-11-02T12:06:56.1371940Z [cluster2] {"level":"info","ts":1604318695.031241,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6104. 2020-11-02T12:06:56.1375060Z [cluster2] {"level":"info","ts":1604318695.034302,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6105. 2020-11-02T12:06:56.1377862Z [cluster2] {"level":"info","ts":1604318697.6985366,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6106. 2020-11-02T12:06:56.1378992Z [cluster2] {"level":"info","ts":1604318697.7106273,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6107. 2020-11-02T12:06:56.1379823Z [cluster2] {"level":"info","ts":1604318697.7106514,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6108. 2020-11-02T12:06:56.1382027Z [cluster2] {"level":"info","ts":1604318697.7148466,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6109. 2020-11-02T12:06:56.1384763Z [cluster2] {"level":"info","ts":1604318697.7179253,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6110. 2020-11-02T12:06:56.1387121Z [cluster2] {"level":"info","ts":1604318703.3171954,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6111. 2020-11-02T12:06:56.1388496Z [cluster2] {"level":"info","ts":1604318703.3314712,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6112. 2020-11-02T12:06:56.1389278Z [cluster2] {"level":"info","ts":1604318703.3314998,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6113. 2020-11-02T12:06:56.1391503Z [cluster2] {"level":"info","ts":1604318703.3365333,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6114. 2020-11-02T12:06:56.1394738Z [cluster2] {"level":"info","ts":1604318703.3397894,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6115. 2020-11-02T12:06:56.1397221Z [cluster2] {"level":"info","ts":1604318706.0160208,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6116. 2020-11-02T12:06:56.1398700Z [cluster2] {"level":"info","ts":1604318706.0222712,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6117. 2020-11-02T12:06:56.1399633Z [cluster2] {"level":"info","ts":1604318706.0222936,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6118. 2020-11-02T12:06:56.1401676Z [cluster2] {"level":"info","ts":1604318706.026208,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6119. 2020-11-02T12:06:56.1405023Z [cluster2] {"level":"info","ts":1604318706.030344,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6120. 2020-11-02T12:06:56.1407053Z [cluster2] {"level":"info","ts":1604318706.036819,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6121. 2020-11-02T12:06:56.1408004Z [cluster2] {"level":"info","ts":1604318706.042335,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6122. 2020-11-02T12:06:56.1408811Z [cluster2] {"level":"info","ts":1604318706.042359,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6123. 2020-11-02T12:06:56.1410667Z [cluster2] {"level":"info","ts":1604318706.0468369,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6124. 2020-11-02T12:06:56.1414827Z [cluster2] {"level":"info","ts":1604318706.049955,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6125. 2020-11-02T12:06:56.1417305Z [cluster2] {"level":"info","ts":1604318706.744079,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6126. 2020-11-02T12:06:56.1418327Z [cluster2] {"level":"info","ts":1604318706.7494626,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6127. 2020-11-02T12:06:56.1419075Z [cluster2] {"level":"info","ts":1604318706.7494853,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6128. 2020-11-02T12:06:56.1420923Z [cluster2] {"level":"info","ts":1604318706.7544372,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6129. 2020-11-02T12:06:56.1423226Z [cluster2] {"level":"info","ts":1604318706.756975,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6130. 2020-11-02T12:06:56.1425096Z [cluster2] {"level":"info","ts":1604318706.7656577,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6131. 2020-11-02T12:06:56.1426073Z [cluster2] {"level":"info","ts":1604318706.7704513,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6132. 2020-11-02T12:06:56.1426795Z [cluster2] {"level":"info","ts":1604318706.7704709,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6133. 2020-11-02T12:06:56.1428454Z [cluster2] {"level":"info","ts":1604318706.7731993,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6134. 2020-11-02T12:06:56.1432199Z [cluster2] {"level":"info","ts":1604318706.7754838,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6135. 2020-11-02T12:06:56.1436215Z [cluster2] {"level":"info","ts":1604318708.339315,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6136. 2020-11-02T12:06:56.1437192Z [cluster2] {"level":"info","ts":1604318708.3449004,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6137. 2020-11-02T12:06:56.1438088Z [cluster2] {"level":"info","ts":1604318708.344922,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6138. 2020-11-02T12:06:56.1440217Z [cluster2] {"level":"info","ts":1604318708.3486843,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6139. 2020-11-02T12:06:56.1445726Z [cluster2] {"level":"info","ts":1604318708.3511493,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6140. 2020-11-02T12:06:56.1448456Z [cluster2] {"level":"info","ts":1604318710.1400878,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6141. 2020-11-02T12:06:56.1449819Z [cluster2] {"level":"info","ts":1604318710.151062,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6142. 2020-11-02T12:06:56.1450898Z [cluster2] {"level":"info","ts":1604318710.1510825,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6143. 2020-11-02T12:06:56.1453214Z [cluster2] {"level":"info","ts":1604318710.15729,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6144. 2020-11-02T12:06:56.1458472Z [cluster2] {"level":"info","ts":1604318710.1599567,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6145. 2020-11-02T12:06:56.1461851Z [cluster2] {"level":"info","ts":1604318710.1650844,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6146. 2020-11-02T12:06:56.1469242Z [cluster2] {"level":"error","ts":1604318710.1686635,"logger":"controller-runtime.controller","msg":"Reconciler error","controller":"submariner-controller","name":"submariner","namespace":"submariner-operator","error":"Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\tsubmariner-operator/vendor/github.com/go-logr/zapr/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsubmariner-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:248\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsubmariner-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:222\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\tsubmariner-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:201\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\tsubmariner-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\tsubmariner-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\tsubmariner-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"}
  6147. 2020-11-02T12:06:56.1475941Z [cluster2] {"level":"info","ts":1604318711.1689603,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6148. 2020-11-02T12:06:56.1476908Z [cluster2] {"level":"info","ts":1604318711.1743574,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6149. 2020-11-02T12:06:56.1477781Z [cluster2] {"level":"info","ts":1604318711.174378,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6150. 2020-11-02T12:06:56.1479351Z [cluster2] {"level":"info","ts":1604318711.1778152,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6151. 2020-11-02T12:06:56.1481828Z [cluster2] {"level":"info","ts":1604318711.1799695,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6152. 2020-11-02T12:06:56.1483682Z [cluster2] {"level":"info","ts":1604318712.4626205,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6153. 2020-11-02T12:06:56.1484793Z [cluster2] {"level":"info","ts":1604318712.4683251,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6154. 2020-11-02T12:06:56.1485498Z [cluster2] {"level":"info","ts":1604318712.4683492,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6155. 2020-11-02T12:06:56.1489043Z [cluster2] {"level":"info","ts":1604318712.4715028,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6156. 2020-11-02T12:06:56.1491230Z [cluster2] {"level":"info","ts":1604318712.474006,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6157. 2020-11-02T12:06:56.1493416Z [cluster2] {"level":"info","ts":1604318712.4795456,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6158. 2020-11-02T12:06:56.1494399Z [cluster2] {"level":"info","ts":1604318712.4833899,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6159. 2020-11-02T12:06:56.1495125Z [cluster2] {"level":"info","ts":1604318712.4834223,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6160. 2020-11-02T12:06:56.1497402Z [cluster2] {"level":"info","ts":1604318712.4868526,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6161. 2020-11-02T12:06:56.1499696Z [cluster2] {"level":"info","ts":1604318712.4893498,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6162. 2020-11-02T12:06:56.1502539Z [cluster2] {"level":"info","ts":1604318716.0308375,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6163. 2020-11-02T12:06:56.1503633Z [cluster2] {"level":"info","ts":1604318716.0348082,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6164. 2020-11-02T12:06:56.1504316Z [cluster2] {"level":"info","ts":1604318716.0348222,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6165. 2020-11-02T12:06:56.1506127Z [cluster2] {"level":"info","ts":1604318716.0378582,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6166. 2020-11-02T12:06:56.1508352Z [cluster2] {"level":"info","ts":1604318716.043639,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6167. 2020-11-02T12:06:56.1510165Z [cluster2] {"level":"info","ts":1604318721.0358243,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6168. 2020-11-02T12:06:56.1511113Z [cluster2] {"level":"info","ts":1604318721.040549,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6169. 2020-11-02T12:06:56.1511900Z [cluster2] {"level":"info","ts":1604318721.0405867,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6170. 2020-11-02T12:06:56.1515553Z [cluster2] {"level":"info","ts":1604318721.0440724,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6171. 2020-11-02T12:06:56.1518999Z [cluster2] {"level":"info","ts":1604318721.0465887,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6172. 2020-11-02T12:06:56.1521516Z [cluster2] {"level":"info","ts":1604318726.051027,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6173. 2020-11-02T12:06:56.1522476Z [cluster2] {"level":"info","ts":1604318726.0626585,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6174. 2020-11-02T12:06:56.1523179Z [cluster2] {"level":"info","ts":1604318726.0626998,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6175. 2020-11-02T12:06:56.1526494Z [cluster2] {"level":"info","ts":1604318726.0674675,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6176. 2020-11-02T12:06:56.1529057Z [cluster2] {"level":"info","ts":1604318726.071954,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6177. 2020-11-02T12:06:56.1531286Z [cluster2] {"level":"info","ts":1604318726.0871184,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6178. 2020-11-02T12:06:56.1532646Z [cluster2] {"level":"info","ts":1604318726.09356,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6179. 2020-11-02T12:06:56.1533353Z [cluster2] {"level":"info","ts":1604318726.0935826,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6180. 2020-11-02T12:06:56.1535019Z [cluster2] {"level":"info","ts":1604318726.0982525,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6181. 2020-11-02T12:06:56.1537467Z [cluster2] {"level":"info","ts":1604318726.1065319,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6182. 2020-11-02T12:06:56.1543525Z [cluster2] {"level":"info","ts":1604318731.0872695,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6183. 2020-11-02T12:06:56.1545754Z [cluster2] {"level":"info","ts":1604318731.116796,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6184. 2020-11-02T12:06:56.1547361Z [cluster2] {"level":"info","ts":1604318731.116818,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6185. 2020-11-02T12:06:56.1550402Z [cluster2] {"level":"info","ts":1604318731.1302195,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6186. 2020-11-02T12:06:56.1553622Z [cluster2] {"level":"info","ts":1604318731.1333103,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6187. 2020-11-02T12:06:56.1555845Z [cluster2] {"level":"info","ts":1604318731.142837,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6188. 2020-11-02T12:06:56.1556796Z [cluster2] {"level":"info","ts":1604318731.1575923,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6189. 2020-11-02T12:06:56.1557503Z [cluster2] {"level":"info","ts":1604318731.157612,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6190. 2020-11-02T12:06:56.1559113Z [cluster2] {"level":"info","ts":1604318731.1664052,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6191. 2020-11-02T12:06:56.1561964Z [cluster2] {"level":"info","ts":1604318731.1740005,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6192. 2020-11-02T12:06:56.1563776Z [cluster2] {"level":"info","ts":1604318736.1271899,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6193. 2020-11-02T12:06:56.1565794Z [cluster2] {"level":"info","ts":1604318736.1337109,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6194. 2020-11-02T12:06:56.1566514Z [cluster2] {"level":"info","ts":1604318736.1337268,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6195. 2020-11-02T12:06:56.1568169Z [cluster2] {"level":"info","ts":1604318736.1393838,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6196. 2020-11-02T12:06:56.1570450Z [cluster2] {"level":"info","ts":1604318736.1449757,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6197. 2020-11-02T12:06:56.1575060Z [cluster2] {"level":"info","ts":1604318736.1504407,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6198. 2020-11-02T12:06:56.1576212Z [cluster2] {"level":"info","ts":1604318736.1546173,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6199. 2020-11-02T12:06:56.1576920Z [cluster2] {"level":"info","ts":1604318736.1546378,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6200. 2020-11-02T12:06:56.1578760Z [cluster2] {"level":"info","ts":1604318736.1576364,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6201. 2020-11-02T12:06:56.1580918Z [cluster2] {"level":"info","ts":1604318736.1627843,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6202. 2020-11-02T12:06:56.1583685Z [cluster2] {"level":"info","ts":1604318741.1493263,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6203. 2020-11-02T12:06:56.1585104Z [cluster2] {"level":"info","ts":1604318741.1552114,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6204. 2020-11-02T12:06:56.1586052Z [cluster2] {"level":"info","ts":1604318741.1552274,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6205. 2020-11-02T12:06:56.1587792Z [cluster2] {"level":"info","ts":1604318741.1611595,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6206. 2020-11-02T12:06:56.1590164Z [cluster2] {"level":"info","ts":1604318741.1640513,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6207. 2020-11-02T12:06:56.1591927Z [cluster2] {"level":"info","ts":1604318746.1648192,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6208. 2020-11-02T12:06:56.1593051Z [cluster2] {"level":"info","ts":1604318746.170268,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6209. 2020-11-02T12:06:56.1593755Z [cluster2] {"level":"info","ts":1604318746.1703072,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6210. 2020-11-02T12:06:56.1595358Z [cluster2] {"level":"info","ts":1604318746.1734104,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6211. 2020-11-02T12:06:56.1597864Z [cluster2] {"level":"info","ts":1604318746.178742,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6212. 2020-11-02T12:06:56.1602560Z [cluster2] {"level":"info","ts":1604318751.174934,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6213. 2020-11-02T12:06:56.1603530Z [cluster2] {"level":"info","ts":1604318751.18256,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6214. 2020-11-02T12:06:56.1604937Z [cluster2] {"level":"info","ts":1604318751.1825826,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6215. 2020-11-02T12:06:56.1606766Z [cluster2] {"level":"info","ts":1604318751.1857164,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6216. 2020-11-02T12:06:56.1609525Z [cluster2] {"level":"info","ts":1604318751.1879692,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6217. 2020-11-02T12:06:56.1611291Z [cluster2] {"level":"info","ts":1604318756.1921399,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6218. 2020-11-02T12:06:56.1612390Z [cluster2] {"level":"info","ts":1604318756.2014086,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6219. 2020-11-02T12:06:56.1613090Z [cluster2] {"level":"info","ts":1604318756.2014248,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6220. 2020-11-02T12:06:56.1614703Z [cluster2] {"level":"info","ts":1604318756.2089307,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6221. 2020-11-02T12:06:56.1617368Z [cluster2] {"level":"info","ts":1604318756.2232006,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6222. 2020-11-02T12:06:56.1619219Z [cluster2] {"level":"info","ts":1604318761.2180862,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6223. 2020-11-02T12:06:56.1621180Z [cluster2] {"level":"info","ts":1604318761.222922,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6224. 2020-11-02T12:06:56.1622109Z [cluster2] {"level":"info","ts":1604318761.2229464,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6225. 2020-11-02T12:06:56.1623916Z [cluster2] {"level":"info","ts":1604318761.2265825,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6226. 2020-11-02T12:06:56.1627037Z [cluster2] {"level":"info","ts":1604318761.2304082,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6227. 2020-11-02T12:06:56.1629305Z [cluster2] {"level":"info","ts":1604318766.2302835,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6228. 2020-11-02T12:06:56.1630254Z [cluster2] {"level":"info","ts":1604318766.2365544,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6229. 2020-11-02T12:06:56.1630932Z [cluster2] {"level":"info","ts":1604318766.2365785,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6230. 2020-11-02T12:06:56.1632548Z [cluster2] {"level":"info","ts":1604318766.2398813,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6231. 2020-11-02T12:06:56.1634701Z [cluster2] {"level":"info","ts":1604318766.2428324,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6232. 2020-11-02T12:06:56.1636445Z [cluster2] {"level":"info","ts":1604318771.2433636,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6233. 2020-11-02T12:06:56.1637686Z [cluster2] {"level":"info","ts":1604318771.2509255,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6234. 2020-11-02T12:06:56.1638392Z [cluster2] {"level":"info","ts":1604318771.2509494,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6235. 2020-11-02T12:06:56.1640034Z [cluster2] {"level":"info","ts":1604318771.2548926,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6236. 2020-11-02T12:06:56.1642340Z [cluster2] {"level":"info","ts":1604318771.257463,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6237. 2020-11-02T12:06:56.1644226Z [cluster2] {"level":"info","ts":1604318776.2582483,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6238. 2020-11-02T12:06:56.1645245Z [cluster2] {"level":"info","ts":1604318776.2663507,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6239. 2020-11-02T12:06:56.1645935Z [cluster2] {"level":"info","ts":1604318776.2663934,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6240. 2020-11-02T12:06:56.1647535Z [cluster2] {"level":"info","ts":1604318776.2696931,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6241. 2020-11-02T12:06:56.1649699Z [cluster2] {"level":"info","ts":1604318776.2729032,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6242. 2020-11-02T12:06:56.1653127Z [cluster2] {"level":"info","ts":1604318781.2746532,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6243. 2020-11-02T12:06:56.1654070Z [cluster2] {"level":"info","ts":1604318781.2815576,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6244. 2020-11-02T12:06:56.1654752Z [cluster2] {"level":"info","ts":1604318781.281573,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6245. 2020-11-02T12:06:56.1657289Z [cluster2] {"level":"info","ts":1604318781.2848887,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6246. 2020-11-02T12:06:56.1659515Z [cluster2] {"level":"info","ts":1604318781.2870696,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6247. 2020-11-02T12:06:56.1661887Z [cluster2] {"level":"info","ts":1604318786.290589,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6248. 2020-11-02T12:06:56.1664484Z [cluster2] {"level":"info","ts":1604318786.296098,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6249. 2020-11-02T12:06:56.1665531Z [cluster2] {"level":"info","ts":1604318786.2961175,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6250. 2020-11-02T12:06:56.1667410Z [cluster2] {"level":"info","ts":1604318786.3005016,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6251. 2020-11-02T12:06:56.1669613Z [cluster2] {"level":"info","ts":1604318786.3048854,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6252. 2020-11-02T12:06:56.1671368Z [cluster2] {"level":"info","ts":1604318791.306212,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6253. 2020-11-02T12:06:56.1672292Z [cluster2] {"level":"info","ts":1604318791.3114204,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6254. 2020-11-02T12:06:56.1673724Z [cluster2] {"level":"info","ts":1604318791.3114455,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6255. 2020-11-02T12:06:56.1675929Z [cluster2] {"level":"info","ts":1604318791.315114,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6256. 2020-11-02T12:06:56.1679538Z [cluster2] {"level":"info","ts":1604318791.3177454,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6257. 2020-11-02T12:06:56.1681326Z [cluster2] {"level":"info","ts":1604318796.3217704,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6258. 2020-11-02T12:06:56.1682226Z [cluster2] {"level":"info","ts":1604318796.3280776,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6259. 2020-11-02T12:06:56.1682915Z [cluster2] {"level":"info","ts":1604318796.3280997,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6260. 2020-11-02T12:06:56.1684602Z [cluster2] {"level":"info","ts":1604318796.334935,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6261. 2020-11-02T12:06:56.1686791Z [cluster2] {"level":"info","ts":1604318796.3396528,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6262. 2020-11-02T12:06:56.1688544Z [cluster2] {"level":"info","ts":1604318801.3417323,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6263. 2020-11-02T12:06:56.1691585Z [submariner]$ [cluster2] namespace=submariner-operator
  6264. 2020-11-02T12:06:56.1692888Z [submariner]$ [cluster2] kubectl get pods --selector=app=submariner-globalnet -n submariner-operator -o jsonpath={.items[*].metadata.name}
  6265. 2020-11-02T12:06:56.1694329Z [submariner]$ [cluster2] kubectl get pods --selector=app=submariner-globalnet -n submariner-operator -o jsonpath={.items[*].metadata.name}
  6266. 2020-11-02T12:06:56.1695851Z [submariner]$ [cluster2] command kubectl --context=cluster2 get pods --selector=app=submariner-globalnet -n submariner-operator -o jsonpath={.items[*].metadata.name}
  6267. 2020-11-02T12:06:56.1697916Z [submariner]$ [cluster2] kubectl --context=cluster2 get pods --selector=app=submariner-globalnet -n submariner-operator -o jsonpath={.items[*].metadata.name}
  6268. 2020-11-02T12:06:56.3156230Z [submariner]$ [cluster2] kubectl get pods --selector=app=submariner-engine -n submariner-operator -o jsonpath={.items[*].metadata.name}
  6269. 2020-11-02T12:06:56.3164470Z [submariner]$ [cluster2] kubectl get pods --selector=app=submariner-engine -n submariner-operator -o jsonpath={.items[*].metadata.name}
  6270. 2020-11-02T12:06:56.3176799Z [submariner]$ [cluster2] command kubectl --context=cluster2 get pods --selector=app=submariner-engine -n submariner-operator -o jsonpath={.items[*].metadata.name}
  6271. 2020-11-02T12:06:56.3189900Z [submariner]$ [cluster2] kubectl --context=cluster2 get pods --selector=app=submariner-engine -n submariner-operator -o jsonpath={.items[*].metadata.name}
  6272. 2020-11-02T12:06:56.5991953Z [submariner]$ [cluster2] kubectl -n submariner-operator logs submariner-gateway-whn7f
  6273. 2020-11-02T12:06:56.6001873Z [submariner]$ [cluster2] kubectl -n submariner-operator logs submariner-gateway-whn7f
  6274. 2020-11-02T12:06:56.6013491Z [submariner]$ [cluster2] command kubectl --context=cluster2 -n submariner-operator logs submariner-gateway-whn7f
  6275. 2020-11-02T12:06:56.6024311Z [submariner]$ [cluster2] kubectl --context=cluster2 -n submariner-operator logs submariner-gateway-whn7f
  6276. 2020-11-02T12:06:56.9048747Z [cluster2] {"level":"info","ts":1604318801.3621247,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6277. 2020-11-02T12:06:56.9049854Z [cluster2] {"level":"info","ts":1604318801.362157,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6278. 2020-11-02T12:06:56.9052048Z [cluster2] {"level":"info","ts":1604318801.3685064,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6279. 2020-11-02T12:06:56.9054798Z [cluster2] {"level":"info","ts":1604318801.3734314,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6280. 2020-11-02T12:06:56.9057884Z [cluster2] {"level":"info","ts":1604318806.3702765,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6281. 2020-11-02T12:06:56.9058895Z [cluster2] {"level":"info","ts":1604318806.3772438,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6282. 2020-11-02T12:06:56.9059619Z [cluster2] {"level":"info","ts":1604318806.3772614,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6283. 2020-11-02T12:06:56.9061462Z [cluster2] {"level":"info","ts":1604318806.3818681,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6284. 2020-11-02T12:06:56.9064401Z [cluster2] {"level":"info","ts":1604318806.3861766,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6285. 2020-11-02T12:06:56.9066275Z [cluster2] {"level":"info","ts":1604318811.40488,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  6286. 2020-11-02T12:06:56.9067594Z [cluster2] {"level":"info","ts":1604318811.4117918,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.2.0.0/16"}
  6287. 2020-11-02T12:06:56.9068305Z [cluster2] {"level":"info","ts":1604318811.411813,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.2.0.0/16"}
  6288. 2020-11-02T12:06:56.9069984Z [cluster2] {"level":"info","ts":1604318811.4170644,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  6289. 2020-11-02T12:06:56.9072467Z [cluster2] {"level":"info","ts":1604318811.4205146,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  6290. 2020-11-02T12:06:56.9073934Z [cluster2] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  6291. 2020-11-02T12:06:56.9074214Z [cluster2] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  6292. 2020-11-02T12:06:56.9075555Z [cluster2] +++++++++++++++++++++: Logs for Pod submariner-gateway-whn7f in namespace submariner-operator :++++++++++++++++++++++
  6293. 2020-11-02T12:06:56.9076366Z [cluster2] + trap 'exit 1' SIGTERM SIGINT
  6294. 2020-11-02T12:06:56.9076929Z [cluster2] + export CHARON_PID_FILE=/var/run/charon.pid
  6295. 2020-11-02T12:06:56.9077355Z [cluster2] + CHARON_PID_FILE=/var/run/charon.pid
  6296. 2020-11-02T12:06:56.9077985Z [cluster2] + rm -f /var/run/charon.pid
  6297. 2020-11-02T12:06:56.9078345Z [cluster2] + SUBMARINER_VERBOSITY=2
  6298. 2020-11-02T12:06:56.9078796Z [cluster2] + '[' false == true ']'
  6299. 2020-11-02T12:06:56.9079183Z [cluster2] + DEBUG=-v=2
  6300. 2020-11-02T12:06:56.9079583Z [cluster2] + mkdir -p /etc/ipsec
  6301. 2020-11-02T12:06:56.9080385Z [cluster2] + sysctl -w net.ipv4.conf.all.send_redirects=0
  6302. 2020-11-02T12:06:56.9081176Z [cluster2] + export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/libexec/strongswan
  6303. 2020-11-02T12:06:56.9081817Z [cluster2] + PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/libexec/strongswan
  6304. 2020-11-02T12:06:56.9082699Z [cluster2] + for f in iptables-save iptables
  6305. 2020-11-02T12:06:56.9083417Z [cluster2] ++ find_iptables_on_host iptables-save
  6306. 2020-11-02T12:06:56.9084152Z [cluster2] ++ chroot /host test -x /usr/sbin/iptables-save
  6307. 2020-11-02T12:06:56.9084639Z [cluster2] net.ipv4.conf.all.send_redirects = 0
  6308. 2020-11-02T12:06:56.9085221Z [cluster2] ++ echo /usr/sbin
  6309. 2020-11-02T12:06:56.9085636Z [cluster2] ++ return
  6310. 2020-11-02T12:06:56.9087618Z [cluster2] iptables-save is present on the host at /usr/sbin/iptables-save
  6311. 2020-11-02T12:06:56.9088073Z [cluster2] + location=/usr/sbin
  6312. 2020-11-02T12:06:56.9097691Z [cluster2] + '[' /usr/sbin '!=' unknown ']'
  6313. 2020-11-02T12:06:56.9098340Z [cluster2] + echo 'iptables-save is present on the host at /usr/sbin/iptables-save'
  6314. 2020-11-02T12:06:56.9099401Z [cluster2] + sed 's!@@PATH@@!/usr/sbin!' /usr/sbin/iptables-wrapper.in
  6315. 2020-11-02T12:06:56.9100050Z [cluster2] + for f in iptables-save iptables
  6316. 2020-11-02T12:06:56.9100433Z [cluster2] ++ find_iptables_on_host iptables
  6317. 2020-11-02T12:06:56.9100946Z [cluster2] ++ chroot /host test -x /usr/sbin/iptables
  6318. 2020-11-02T12:06:56.9101276Z [cluster2] ++ echo /usr/sbin
  6319. 2020-11-02T12:06:56.9101537Z [cluster2] ++ return
  6320. 2020-11-02T12:06:56.9101795Z [cluster2] + location=/usr/sbin
  6321. 2020-11-02T12:06:56.9102229Z [cluster2] + '[' /usr/sbin '!=' unknown ']'
  6322. 2020-11-02T12:06:56.9102835Z [cluster2] + echo 'iptables is present on the host at /usr/sbin/iptables'
  6323. 2020-11-02T12:06:56.9104008Z [cluster2] + sed 's!@@PATH@@!/usr/sbin!' /usr/sbin/iptables-wrapper.in
  6324. 2020-11-02T12:06:56.9104521Z [cluster2] iptables is present on the host at /usr/sbin/iptables
  6325. 2020-11-02T12:06:56.9105400Z [cluster2] + exec submariner-engine -v=2 -alsologtostderr
  6326. 2020-11-02T12:06:56.9106106Z [cluster2] I1102 12:05:05.993356 1 main.go:70] Starting the submariner gateway engine
  6327. 2020-11-02T12:06:56.9107060Z [cluster2] W1102 12:05:05.993783 1 client_config.go:543] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
  6328. 2020-11-02T12:06:56.9107720Z [cluster2] I1102 12:05:05.994101 1 main.go:95] Creating the cable engine
  6329. 2020-11-02T12:06:56.9108148Z [cluster2] I1102 12:05:05.994247 1 syncer.go:61] CableEngine syncer started
  6330. 2020-11-02T12:06:56.9109030Z [cluster2] I1102 12:05:12.002758 1 main.go:252] Gateway leader election config values: main.leaderConfig{LeaseDuration:10, RenewDeadline:5, RetryPeriod:2}
  6331. 2020-11-02T12:06:56.9110506Z [cluster2] I1102 12:05:12.002923 1 main.go:269] Using namespace "submariner-operator" for the leader election lock
  6332. 2020-11-02T12:06:56.9111821Z [cluster2] I1102 12:05:12.002952 1 leaderelection.go:242] attempting to acquire leader lease submariner-operator/submariner-engine-lock...
  6333. 2020-11-02T12:06:56.9113235Z [cluster2] I1102 12:05:23.322666 1 leaderelection.go:252] successfully acquired lease submariner-operator/submariner-engine-lock
  6334. 2020-11-02T12:06:56.9115797Z [cluster2] I1102 12:05:23.322756 1 event.go:281] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"submariner-operator", Name:"submariner-engine-lock", UID:"fc72ba2c-23ac-47f9-92ab-c98b4618a329", APIVersion:"v1", ResourceVersion:"3701", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' cluster2-worker2-submariner-engine became leader
  6335. 2020-11-02T12:06:56.9117442Z [cluster2] I1102 12:05:23.322780 1 main.go:144] Creating the datastore syncer
  6336. 2020-11-02T12:06:56.9117885Z [cluster2] I1102 12:05:23.323017 1 libreswan.go:347] Starting Pluto
  6337. 2020-11-02T12:06:56.9118283Z [cluster2] Initializing NSS database
  6338. 2020-11-02T12:06:56.9118580Z [cluster2]
  6339. 2020-11-02T12:06:56.9118870Z [cluster2] nflog ipsec capture disabled
  6340. 2020-11-02T12:06:56.9119502Z [cluster2] I1102 12:05:24.324434 1 cableengine.go:71] CableEngine controller started, driver: "libreswan"
  6341. 2020-11-02T12:06:56.9120147Z [cluster2] I1102 12:05:24.324473 1 datastoresyncer.go:60] Starting the datastore syncer
  6342. 2020-11-02T12:06:56.9120677Z [cluster2] I1102 12:05:24.324839 1 tunnel.go:22] Starting the tunnel controller
  6343. 2020-11-02T12:06:56.9121414Z [cluster2] I1102 12:05:24.325419 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  6344. 2020-11-02T12:06:56.9122314Z [cluster2] I1102 12:05:24.325463 1 resource_syncer.go:180] Starting syncer "Tunnel controller"
  6345. 2020-11-02T12:06:56.9122929Z [cluster2] I1102 12:05:24.325468 1 resource_syncer.go:193] Syncer "Tunnel controller" waiting for informer cache to sync
  6346. 2020-11-02T12:06:56.9123523Z [cluster2] I1102 12:05:24.426044 1 resource_syncer.go:202] Syncer "Tunnel controller" started
  6347. 2020-11-02T12:06:56.9128901Z [cluster2] I1102 12:05:24.426096 1 resource_syncer.go:264] Syncer "Tunnel controller" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:01:40Z", "generation":1, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-operator", "resourceVersion":"2845", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"8e4ce5be-6521-4f69-bef3-cde34ed638a9"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  6348. 2020-11-02T12:06:56.9138568Z [cluster2] I1102 12:05:24.426511 1 tunnel.go:56] Tunnel controller processing added or updated submariner Endpoint object: &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster2-submariner-cable-cluster2-172-17-0-4", GenerateName:"", Namespace:"submariner-operator", SelfLink:"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", UID:"8e4ce5be-6521-4f69-bef3-cde34ed638a9", ResourceVersion:"2845", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63739915300, loc:(*time.Location)(0x2020aa0)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster2", CableName:"submariner-cable-cluster2-172-17-0-4", Hostname:"cluster2-worker", Subnets:[]string{"100.2.0.0/16", "10.2.0.0/16"}, PrivateIP:"172.17.0.4", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string(nil)}}
  6349. 2020-11-02T12:06:56.9142970Z [cluster2] I1102 12:05:24.426568 1 cableengine.go:92] Not installing cable for local cluster
  6350. 2020-11-02T12:06:56.9144762Z [cluster2] I1102 12:05:24.426574 1 tunnel.go:68] Tunnel controller successfully installed Endpoint cable submariner-cable-cluster2-172-17-0-4 in the engine
  6351. 2020-11-02T12:06:56.9146306Z [cluster2] I1102 12:05:24.426580 1 resource_syncer.go:352] Syncer "Tunnel controller": transform function returned nil - not syncing - requeue: false
  6352. 2020-11-02T12:06:56.9151824Z [cluster2] I1102 12:05:24.426591 1 resource_syncer.go:264] Syncer "Tunnel controller" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3-submariner-cable-cluster3-172-17-0-7", "namespace":"submariner-operator", "resourceVersion":"1772", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster3-submariner-cable-cluster3-172-17-0-7", "uid":"dfbc100c-1c97-4e89-95c6-f4d8406c55aa"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster3-172-17-0-7", "cluster_id":"cluster3", "hostname":"cluster3-worker", "nat_enabled":false, "private_ip":"172.17.0.7", "public_ip":"", "subnets":[]interface {}{"100.3.0.0/16", "10.3.0.0/16"}}}}
  6353. 2020-11-02T12:06:56.9162519Z [cluster2] I1102 12:05:24.426918 1 tunnel.go:56] Tunnel controller processing added or updated submariner Endpoint object: &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster3-submariner-cable-cluster3-172-17-0-7", GenerateName:"", Namespace:"submariner-operator", SelfLink:"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster3-submariner-cable-cluster3-172-17-0-7", UID:"dfbc100c-1c97-4e89-95c6-f4d8406c55aa", ResourceVersion:"1772", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63739915104, loc:(*time.Location)(0x2020aa0)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"submariner-io/clusterID":"cluster3"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster3", CableName:"submariner-cable-cluster3-172-17-0-7", Hostname:"cluster3-worker", Subnets:[]string{"100.3.0.0/16", "10.3.0.0/16"}, PrivateIP:"172.17.0.7", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string(nil)}}
  6354. 2020-11-02T12:06:56.9173646Z [cluster2] I1102 12:05:24.426998 1 cableengine.go:101] Installing Endpoint cable "submariner-cable-cluster3-172-17-0-7"
  6355. 2020-11-02T12:06:56.9174651Z [cluster2] I1102 12:05:24.427006 1 libreswan.go:164] Active connections: []
  6356. 2020-11-02T12:06:56.9175165Z [cluster2] 002 listening for IKE messages
  6357. 2020-11-02T12:06:56.9175602Z [cluster2] 002 forgetting secrets
  6358. 2020-11-02T12:06:56.9176077Z [cluster2] 002 loading secrets from "/etc/ipsec.secrets"
  6359. 2020-11-02T12:06:56.9176958Z [cluster2] 002 loading secrets from "/etc/ipsec.d/submariner.secrets"
  6360. 2020-11-02T12:06:56.9178878Z [cluster2] I1102 12:05:24.428927 1 libreswan.go:245] Creating connection(s) for {{cluster3 submariner-cable-cluster3-172-17-0-7 cluster3-worker [100.3.0.0/16 10.3.0.0/16] 172.17.0.7 false libreswan map[]}}
  6361. 2020-11-02T12:06:56.9180568Z [cluster2] I1102 12:05:24.429089 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster3-172-17-0-7-0-0 --id 172.17.0.11 --host 172.17.0.11 --client 172.17.0.11/32 --to --id 172.17.0.7 --host 172.17.0.7 --client 172.17.0.7/32]
  6362. 2020-11-02T12:06:56.9182162Z [cluster2] 002 added connection description "submariner-cable-cluster3-172-17-0-7-0-0"
  6363. 2020-11-02T12:06:56.9183331Z [cluster2] 181 "submariner-cable-cluster3-172-17-0-7-0-0" #1: initiating IKEv2 IKE SA
  6364. 2020-11-02T12:06:56.9184838Z [cluster2] I1102 12:05:24.474282 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster3-172-17-0-7-0-1 --id 172.17.0.11 --host 172.17.0.11 --client 172.17.0.11/32 --to --id 172.17.0.7 --host 172.17.0.7 --client 100.3.0.0/16]
  6365. 2020-11-02T12:06:56.9186645Z [cluster2] 002 added connection description "submariner-cable-cluster3-172-17-0-7-0-1"
  6366. 2020-11-02T12:06:56.9188353Z [cluster2] 000 "submariner-cable-cluster3-172-17-0-7-0-1": queuing pending IPsec SA negotiating with 172.17.0.7 IKE SA #1 "submariner-cable-cluster3-172-17-0-7-0-0"
  6367. 2020-11-02T12:06:56.9190424Z [cluster2] I1102 12:05:24.514289 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster3-172-17-0-7-0-2 --id 172.17.0.11 --host 172.17.0.11 --client 172.17.0.11/32 --to --id 172.17.0.7 --host 172.17.0.7 --client 10.3.0.0/16]
  6368. 2020-11-02T12:06:56.9191763Z [cluster2] 002 added connection description "submariner-cable-cluster3-172-17-0-7-0-2"
  6369. 2020-11-02T12:06:56.9193486Z [cluster2] 000 "submariner-cable-cluster3-172-17-0-7-0-2": queuing pending IPsec SA negotiating with 172.17.0.7 IKE SA #1 "submariner-cable-cluster3-172-17-0-7-0-0"
  6370. 2020-11-02T12:06:56.9196173Z [cluster2] I1102 12:05:24.552435 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster3-172-17-0-7-1-0 --id 172.17.0.11 --host 172.17.0.11 --client 100.2.0.0/16 --to --id 172.17.0.7 --host 172.17.0.7 --client 172.17.0.7/32]
  6371. 2020-11-02T12:06:56.9197586Z [cluster2] 002 added connection description "submariner-cable-cluster3-172-17-0-7-1-0"
  6372. 2020-11-02T12:06:56.9214921Z [cluster2] 000 "submariner-cable-cluster3-172-17-0-7-1-0": queuing pending IPsec SA negotiating with 172.17.0.7 IKE SA #1 "submariner-cable-cluster3-172-17-0-7-0-0"
  6373. 2020-11-02T12:06:56.9217401Z [cluster2] I1102 12:05:24.555415 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster3-172-17-0-7-1-1 --id 172.17.0.11 --host 172.17.0.11 --client 100.2.0.0/16 --to --id 172.17.0.7 --host 172.17.0.7 --client 100.3.0.0/16]
  6374. 2020-11-02T12:06:56.9219595Z [cluster2] 002 added connection description "submariner-cable-cluster3-172-17-0-7-1-1"
  6375. 2020-11-02T12:06:56.9221424Z [cluster2] 000 "submariner-cable-cluster3-172-17-0-7-1-1": queuing pending IPsec SA negotiating with 172.17.0.7 IKE SA #1 "submariner-cable-cluster3-172-17-0-7-0-0"
  6376. 2020-11-02T12:06:56.9223335Z [cluster2] I1102 12:05:24.558786 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster3-172-17-0-7-1-2 --id 172.17.0.11 --host 172.17.0.11 --client 100.2.0.0/16 --to --id 172.17.0.7 --host 172.17.0.7 --client 10.3.0.0/16]
  6377. 2020-11-02T12:06:56.9224697Z [cluster2] 002 added connection description "submariner-cable-cluster3-172-17-0-7-1-2"
  6378. 2020-11-02T12:06:56.9226681Z [cluster2] 000 "submariner-cable-cluster3-172-17-0-7-1-2": queuing pending IPsec SA negotiating with 172.17.0.7 IKE SA #1 "submariner-cable-cluster3-172-17-0-7-0-0"
  6379. 2020-11-02T12:06:56.9228907Z [cluster2] I1102 12:05:24.561705 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster3-172-17-0-7-2-0 --id 172.17.0.11 --host 172.17.0.11 --client 10.2.0.0/16 --to --id 172.17.0.7 --host 172.17.0.7 --client 172.17.0.7/32]
  6380. 2020-11-02T12:06:56.9230380Z [cluster2] 002 added connection description "submariner-cable-cluster3-172-17-0-7-2-0"
  6381. 2020-11-02T12:06:56.9232008Z [cluster2] 000 "submariner-cable-cluster3-172-17-0-7-2-0": queuing pending IPsec SA negotiating with 172.17.0.7 IKE SA #1 "submariner-cable-cluster3-172-17-0-7-0-0"
  6382. 2020-11-02T12:06:56.9244532Z [cluster2] I1102 12:05:24.564944 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster3-172-17-0-7-2-1 --id 172.17.0.11 --host 172.17.0.11 --client 10.2.0.0/16 --to --id 172.17.0.7 --host 172.17.0.7 --client 100.3.0.0/16]
  6383. 2020-11-02T12:06:56.9246533Z [cluster2] 002 added connection description "submariner-cable-cluster3-172-17-0-7-2-1"
  6384. 2020-11-02T12:06:56.9248368Z [cluster2] 000 "submariner-cable-cluster3-172-17-0-7-2-1": queuing pending IPsec SA negotiating with 172.17.0.7 IKE SA #1 "submariner-cable-cluster3-172-17-0-7-0-0"
  6385. 2020-11-02T12:06:56.9250634Z [cluster2] I1102 12:05:24.568862 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster3-172-17-0-7-2-2 --id 172.17.0.11 --host 172.17.0.11 --client 10.2.0.0/16 --to --id 172.17.0.7 --host 172.17.0.7 --client 10.3.0.0/16]
  6386. 2020-11-02T12:06:56.9252067Z [cluster2] 002 added connection description "submariner-cable-cluster3-172-17-0-7-2-2"
  6387. 2020-11-02T12:06:56.9253904Z [cluster2] 000 "submariner-cable-cluster3-172-17-0-7-2-2": queuing pending IPsec SA negotiating with 172.17.0.7 IKE SA #1 "submariner-cable-cluster3-172-17-0-7-0-0"
  6388. 2020-11-02T12:06:56.9255807Z [cluster2] I1102 12:05:24.571674 1 cableengine.go:150] Successfully installed Endpoint cable "submariner-cable-cluster3-172-17-0-7" with remote IP 172.17.0.7
  6389. 2020-11-02T12:06:56.9257401Z [cluster2] I1102 12:05:24.571701 1 tunnel.go:68] Tunnel controller successfully installed Endpoint cable submariner-cable-cluster3-172-17-0-7 in the engine
  6390. 2020-11-02T12:06:56.9258862Z [cluster2] I1102 12:05:24.571708 1 resource_syncer.go:352] Syncer "Tunnel controller": transform function returned nil - not syncing - requeue: false
  6391. 2020-11-02T12:06:56.9263381Z [cluster2] I1102 12:05:24.571725 1 resource_syncer.go:264] Syncer "Tunnel controller" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1-submariner-cable-cluster1-172-17-0-6", "namespace":"submariner-operator", "resourceVersion":"1789", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster1-submariner-cable-cluster1-172-17-0-6", "uid":"61f6c320-ac18-45ee-bb2d-00a06d34aab0"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster1-172-17-0-6", "cluster_id":"cluster1", "hostname":"cluster1-worker", "nat_enabled":false, "private_ip":"172.17.0.6", "public_ip":"", "subnets":[]interface {}{"100.1.0.0/16", "10.1.0.0/16"}}}}
  6392. 2020-11-02T12:06:56.9272347Z [cluster2] I1102 12:05:24.571918 1 tunnel.go:56] Tunnel controller processing added or updated submariner Endpoint object: &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster1-submariner-cable-cluster1-172-17-0-6", GenerateName:"", Namespace:"submariner-operator", SelfLink:"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster1-submariner-cable-cluster1-172-17-0-6", UID:"61f6c320-ac18-45ee-bb2d-00a06d34aab0", ResourceVersion:"1789", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63739915107, loc:(*time.Location)(0x2020aa0)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"submariner-io/clusterID":"cluster1"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster1", CableName:"submariner-cable-cluster1-172-17-0-6", Hostname:"cluster1-worker", Subnets:[]string{"100.1.0.0/16", "10.1.0.0/16"}, PrivateIP:"172.17.0.6", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string(nil)}}
  6393. 2020-11-02T12:06:56.9277534Z [cluster2] I1102 12:05:24.571968 1 cableengine.go:101] Installing Endpoint cable "submariner-cable-cluster1-172-17-0-6"
  6394. 2020-11-02T12:06:56.9279221Z [cluster2] I1102 12:05:24.571975 1 libreswan.go:164] Active connections: [submariner-cable-cluster3-172-17-0-7]
  6395. 2020-11-02T12:06:56.9280555Z [cluster2] I1102 12:05:24.572949 1 libreswan.go:149] Connection "submariner-cable-cluster3-172-17-0-7" not found in active connections obtained from whack: []
  6396. 2020-11-02T12:06:56.9281357Z [cluster2] 002 listening for IKE messages
  6397. 2020-11-02T12:06:56.9281706Z [cluster2] 002 forgetting secrets
  6398. 2020-11-02T12:06:56.9282096Z [cluster2] 002 loading secrets from "/etc/ipsec.secrets"
  6399. 2020-11-02T12:06:56.9282600Z [cluster2] 002 loading secrets from "/etc/ipsec.d/submariner.secrets"
  6400. 2020-11-02T12:06:56.9284374Z [cluster2] I1102 12:05:24.574785 1 libreswan.go:245] Creating connection(s) for {{cluster1 submariner-cable-cluster1-172-17-0-6 cluster1-worker [100.1.0.0/16 10.1.0.0/16] 172.17.0.6 false libreswan map[]}}
  6401. 2020-11-02T12:06:56.9286948Z [cluster2] I1102 12:05:24.574814 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster1-172-17-0-6-0-0 --id 172.17.0.11 --host 172.17.0.11 --client 172.17.0.11/32 --to --id 172.17.0.6 --host 172.17.0.6 --client 172.17.0.6/32]
  6402. 2020-11-02T12:06:56.9288448Z [cluster2] 002 added connection description "submariner-cable-cluster1-172-17-0-6-0-0"
  6403. 2020-11-02T12:06:56.9290170Z [cluster2] 181 "submariner-cable-cluster1-172-17-0-6-0-0" #2: initiating IKEv2 IKE SA
  6404. 2020-11-02T12:06:56.9291837Z [cluster2] I1102 12:05:24.611424 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster1-172-17-0-6-0-1 --id 172.17.0.11 --host 172.17.0.11 --client 172.17.0.11/32 --to --id 172.17.0.6 --host 172.17.0.6 --client 100.1.0.0/16]
  6405. 2020-11-02T12:06:56.9293209Z [cluster2] 002 added connection description "submariner-cable-cluster1-172-17-0-6-0-1"
  6406. 2020-11-02T12:06:56.9294806Z [cluster2] 000 "submariner-cable-cluster1-172-17-0-6-0-1": queuing pending IPsec SA negotiating with 172.17.0.6 IKE SA #2 "submariner-cable-cluster1-172-17-0-6-0-0"
  6407. 2020-11-02T12:06:56.9296978Z [cluster2] I1102 12:05:24.650315 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster1-172-17-0-6-0-2 --id 172.17.0.11 --host 172.17.0.11 --client 172.17.0.11/32 --to --id 172.17.0.6 --host 172.17.0.6 --client 10.1.0.0/16]
  6408. 2020-11-02T12:06:56.9298509Z [cluster2] 002 added connection description "submariner-cable-cluster1-172-17-0-6-0-2"
  6409. 2020-11-02T12:06:56.9302530Z [cluster2] 000 "submariner-cable-cluster1-172-17-0-6-0-2": queuing pending IPsec SA negotiating with 172.17.0.6 IKE SA #2 "submariner-cable-cluster1-172-17-0-6-0-0"
  6410. 2020-11-02T12:06:56.9304671Z [cluster2] I1102 12:05:24.686589 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster1-172-17-0-6-1-0 --id 172.17.0.11 --host 172.17.0.11 --client 100.2.0.0/16 --to --id 172.17.0.6 --host 172.17.0.6 --client 172.17.0.6/32]
  6411. 2020-11-02T12:06:56.9306649Z [cluster2] 002 added connection description "submariner-cable-cluster1-172-17-0-6-1-0"
  6412. 2020-11-02T12:06:56.9308368Z [cluster2] 000 "submariner-cable-cluster1-172-17-0-6-1-0": queuing pending IPsec SA negotiating with 172.17.0.6 IKE SA #2 "submariner-cable-cluster1-172-17-0-6-0-0"
  6413. 2020-11-02T12:06:56.9310401Z [cluster2] I1102 12:05:24.689729 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster1-172-17-0-6-1-1 --id 172.17.0.11 --host 172.17.0.11 --client 100.2.0.0/16 --to --id 172.17.0.6 --host 172.17.0.6 --client 100.1.0.0/16]
  6414. 2020-11-02T12:06:56.9311740Z [cluster2] 002 added connection description "submariner-cable-cluster1-172-17-0-6-1-1"
  6415. 2020-11-02T12:06:56.9313446Z [cluster2] 000 "submariner-cable-cluster1-172-17-0-6-1-1": queuing pending IPsec SA negotiating with 172.17.0.6 IKE SA #2 "submariner-cable-cluster1-172-17-0-6-0-0"
  6416. 2020-11-02T12:06:56.9316010Z [cluster2] I1102 12:05:24.692956 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster1-172-17-0-6-1-2 --id 172.17.0.11 --host 172.17.0.11 --client 100.2.0.0/16 --to --id 172.17.0.6 --host 172.17.0.6 --client 10.1.0.0/16]
  6417. 2020-11-02T12:06:56.9317620Z [cluster2] 002 added connection description "submariner-cable-cluster1-172-17-0-6-1-2"
  6418. 2020-11-02T12:06:56.9319395Z [cluster2] 000 "submariner-cable-cluster1-172-17-0-6-1-2": queuing pending IPsec SA negotiating with 172.17.0.6 IKE SA #2 "submariner-cable-cluster1-172-17-0-6-0-0"
  6419. 2020-11-02T12:06:56.9321259Z [cluster2] I1102 12:05:24.696103 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster1-172-17-0-6-2-0 --id 172.17.0.11 --host 172.17.0.11 --client 10.2.0.0/16 --to --id 172.17.0.6 --host 172.17.0.6 --client 172.17.0.6/32]
  6420. 2020-11-02T12:06:56.9322596Z [cluster2] 002 added connection description "submariner-cable-cluster1-172-17-0-6-2-0"
  6421. 2020-11-02T12:06:56.9324700Z [cluster2] 000 "submariner-cable-cluster1-172-17-0-6-2-0": queuing pending IPsec SA negotiating with 172.17.0.6 IKE SA #2 "submariner-cable-cluster1-172-17-0-6-0-0"
  6422. 2020-11-02T12:06:56.9326861Z [cluster2] I1102 12:05:24.700059 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster1-172-17-0-6-2-1 --id 172.17.0.11 --host 172.17.0.11 --client 10.2.0.0/16 --to --id 172.17.0.6 --host 172.17.0.6 --client 100.1.0.0/16]
  6423. 2020-11-02T12:06:56.9334151Z [submariner]$ [cluster2] kubectl get pods --selector=app=submariner-routeagent -n submariner-operator -o jsonpath={.items[*].metadata.name}
  6424. 2020-11-02T12:06:56.9335851Z [submariner]$ [cluster2] kubectl get pods --selector=app=submariner-routeagent -n submariner-operator -o jsonpath={.items[*].metadata.name}
  6425. 2020-11-02T12:06:56.9339658Z [submariner]$ [cluster2] command kubectl --context=cluster2 get pods --selector=app=submariner-routeagent -n submariner-operator -o jsonpath={.items[*].metadata.name}
  6426. 2020-11-02T12:06:56.9361505Z [submariner]$ [cluster2] kubectl --context=cluster2 get pods --selector=app=submariner-routeagent -n submariner-operator -o jsonpath={.items[*].metadata.name}
  6427. 2020-11-02T12:06:56.9363237Z [cluster2] 002 added connection description "submariner-cable-cluster1-172-17-0-6-2-1"
  6428. 2020-11-02T12:06:56.9369937Z [cluster2] 000 "submariner-cable-cluster1-172-17-0-6-2-1": queuing pending IPsec SA negotiating with 172.17.0.6 IKE SA #2 "submariner-cable-cluster1-172-17-0-6-0-0"
  6429. 2020-11-02T12:06:56.9372841Z [cluster2] I1102 12:05:24.704536 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster1-172-17-0-6-2-2 --id 172.17.0.11 --host 172.17.0.11 --client 10.2.0.0/16 --to --id 172.17.0.6 --host 172.17.0.6 --client 10.1.0.0/16]
  6430. 2020-11-02T12:06:56.9374672Z [cluster2] 002 added connection description "submariner-cable-cluster1-172-17-0-6-2-2"
  6431. 2020-11-02T12:06:56.9377870Z [cluster2] 000 "submariner-cable-cluster1-172-17-0-6-2-2": queuing pending IPsec SA negotiating with 172.17.0.6 IKE SA #2 "submariner-cable-cluster1-172-17-0-6-0-0"
  6432. 2020-11-02T12:06:56.9380483Z [cluster2] I1102 12:05:24.707575 1 cableengine.go:150] Successfully installed Endpoint cable "submariner-cable-cluster1-172-17-0-6" with remote IP 172.17.0.6
  6433. 2020-11-02T12:06:56.9382322Z [cluster2] I1102 12:05:24.707593 1 tunnel.go:68] Tunnel controller successfully installed Endpoint cable submariner-cable-cluster1-172-17-0-6 in the engine
  6434. 2020-11-02T12:06:56.9386385Z [cluster2] I1102 12:05:24.707600 1 resource_syncer.go:352] Syncer "Tunnel controller": transform function returned nil - not syncing - requeue: false
  6435. 2020-11-02T12:06:56.9388228Z [cluster2] I1102 12:05:26.041750 1 libreswan.go:149] Connection "submariner-cable-cluster3-172-17-0-7" not found in active connections obtained from whack: []
  6436. 2020-11-02T12:06:56.9391678Z [cluster2] I1102 12:05:26.041776 1 libreswan.go:149] Connection "submariner-cable-cluster1-172-17-0-6" not found in active connections obtained from whack: []
  6437. 2020-11-02T12:06:56.9394238Z [cluster2] I1102 12:05:30.330809 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  6438. 2020-11-02T12:06:56.9395857Z [cluster2] I1102 12:05:30.368818 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  6439. 2020-11-02T12:06:56.9397531Z [cluster2] I1102 12:05:30.375372 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  6440. 2020-11-02T12:06:56.9399154Z [cluster2] I1102 12:05:30.376129 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  6441. 2020-11-02T12:06:56.9400302Z [cluster2] I1102 12:05:30.376297 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  6442. 2020-11-02T12:06:56.9401950Z [cluster2] I1102 12:05:30.376454 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  6443. 2020-11-02T12:06:56.9402817Z [cluster2] I1102 12:05:30.376485 1 datastoresyncer.go:80] Starting the broker syncer
  6444. 2020-11-02T12:06:56.9405972Z [cluster2] I1102 12:05:30.376496 1 resource_syncer.go:180] Starting syncer "local -> broker for *v1.Cluster"
  6445. 2020-11-02T12:06:56.9407748Z [cluster2] I1102 12:05:30.376501 1 resource_syncer.go:193] Syncer "local -> broker for *v1.Cluster" waiting for informer cache to sync
  6446. 2020-11-02T12:06:56.9409397Z [cluster2] I1102 12:05:30.476608 1 resource_syncer.go:202] Syncer "local -> broker for *v1.Cluster" started
  6447. 2020-11-02T12:06:56.9410455Z [cluster2] I1102 12:05:30.476633 1 resource_syncer.go:180] Starting syncer "broker -> local for *v1.Cluster"
  6448. 2020-11-02T12:06:56.9411295Z [cluster2] I1102 12:05:30.476640 1 resource_syncer.go:202] Syncer "broker -> local for *v1.Cluster" started
  6449. 2020-11-02T12:06:56.9412142Z [cluster2] I1102 12:05:30.476643 1 resource_syncer.go:180] Starting syncer "local -> broker for *v1.Endpoint"
  6450. 2020-11-02T12:06:56.9413563Z [cluster2] I1102 12:05:30.476648 1 resource_syncer.go:193] Syncer "local -> broker for *v1.Endpoint" waiting for informer cache to sync
  6451. 2020-11-02T12:06:56.9417584Z [cluster2] I1102 12:05:30.476690 1 resource_syncer.go:264] Syncer "local -> broker for *v1.Cluster" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1", "namespace":"submariner-operator", "resourceVersion":"1788", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/clusters/cluster1", "uid":"09111cbd-212d-4207-afb6-79ba2235ca35"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.1.0.0/16"}, "cluster_id":"cluster1", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.1.0.0/16"}}}}
  6452. 2020-11-02T12:06:56.9422176Z [cluster2] I1102 12:05:30.476836 1 resource_syncer.go:437] Found cluster ID label "cluster1" - not syncing resource "cluster1"
  6453. 2020-11-02T12:06:56.9425944Z [cluster2] I1102 12:05:30.476856 1 resource_syncer.go:264] Syncer "local -> broker for *v1.Cluster" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:19Z", "generation":1, "name":"cluster2", "namespace":"submariner-operator", "resourceVersion":"1740", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/clusters/cluster2", "uid":"de0c4f27-25fa-4146-a40b-f270fb0060f4"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.2.0.0/16"}, "cluster_id":"cluster2", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.2.0.0/16"}}}}
  6454. 2020-11-02T12:06:56.9432705Z [cluster2] I1102 12:05:30.476914 1 resource_syncer.go:276] Syncing resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:19Z", "generation":1, "name":"cluster2", "namespace":"submariner-operator", "resourceVersion":"1740", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/clusters/cluster2", "uid":"de0c4f27-25fa-4146-a40b-f270fb0060f4"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.2.0.0/16"}, "cluster_id":"cluster2", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.2.0.0/16"}}}}
  6455. 2020-11-02T12:06:56.9438653Z [cluster2] I1102 12:05:30.476951 1 federator.go:42] In Distribute for &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:19Z", "generation":1, "name":"cluster2", "namespace":"submariner-operator", "resourceVersion":"1740", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/clusters/cluster2", "uid":"de0c4f27-25fa-4146-a40b-f270fb0060f4"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.2.0.0/16"}, "cluster_id":"cluster2", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.2.0.0/16"}}}}
  6456. 2020-11-02T12:06:56.9441918Z [cluster2] I1102 12:05:30.477040 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  6457. 2020-11-02T12:06:56.9443628Z [cluster2] I1102 12:05:30.479302 1 resource_syncer.go:285] Syncer "local -> broker for *v1.Cluster" successfully synced "submariner-operator/cluster2"
  6458. 2020-11-02T12:06:56.9447387Z [cluster2] I1102 12:05:30.479404 1 resource_syncer.go:264] Syncer "local -> broker for *v1.Cluster" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3", "namespace":"submariner-operator", "resourceVersion":"1769", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/clusters/cluster3", "uid":"38bd0686-b9d9-4e39-87f8-f9ed30627382"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.3.0.0/16"}, "cluster_id":"cluster3", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.3.0.0/16"}}}}
  6459. 2020-11-02T12:06:56.9450548Z [cluster2] I1102 12:05:30.479605 1 resource_syncer.go:437] Found cluster ID label "cluster3" - not syncing resource "cluster3"
  6460. 2020-11-02T12:06:56.9454987Z [cluster2] I1102 12:05:30.484294 1 resource_syncer.go:264] Syncer "broker -> local for *v1.Cluster" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1", "namespace":"submariner-k8s-broker", "resourceVersion":"1741", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/clusters/cluster1", "uid":"025b70bc-43a5-4d06-b4ff-a733ee10d897"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.1.0.0/16"}, "cluster_id":"cluster1", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.1.0.0/16"}}}}
  6461. 2020-11-02T12:06:56.9460875Z [cluster2] I1102 12:05:30.484378 1 resource_syncer.go:276] Syncing resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1", "namespace":"submariner-k8s-broker", "resourceVersion":"1741", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/clusters/cluster1", "uid":"025b70bc-43a5-4d06-b4ff-a733ee10d897"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.1.0.0/16"}, "cluster_id":"cluster1", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.1.0.0/16"}}}}
  6462. 2020-11-02T12:06:56.9466528Z [cluster2] I1102 12:05:30.484457 1 federator.go:42] In Distribute for &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1", "namespace":"submariner-k8s-broker", "resourceVersion":"1741", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/clusters/cluster1", "uid":"025b70bc-43a5-4d06-b4ff-a733ee10d897"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.1.0.0/16"}, "cluster_id":"cluster1", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.1.0.0/16"}}}}
  6463. 2020-11-02T12:06:56.9469141Z [cluster2] I1102 12:05:30.484519 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  6464. 2020-11-02T12:06:56.9471173Z [cluster2] I1102 12:05:30.487466 1 resource_syncer.go:285] Syncer "broker -> local for *v1.Cluster" successfully synced "submariner-k8s-broker/cluster1"
  6465. 2020-11-02T12:06:56.9475170Z [cluster2] I1102 12:05:30.487491 1 resource_syncer.go:264] Syncer "broker -> local for *v1.Cluster" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:19Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2", "namespace":"submariner-k8s-broker", "resourceVersion":"1679", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/clusters/cluster2", "uid":"4ede1863-af75-4242-84c7-e67b484c3c22"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.2.0.0/16"}, "cluster_id":"cluster2", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.2.0.0/16"}}}}
  6466. 2020-11-02T12:06:56.9478193Z [cluster2] I1102 12:05:30.487569 1 resource_syncer.go:443] Cluster ID label "cluster2" not present or matches local cluster ID "cluster2" - not syncing resource "cluster2"
  6467. 2020-11-02T12:06:56.9482618Z [cluster2] I1102 12:05:30.487647 1 resource_syncer.go:264] Syncer "broker -> local for *v1.Cluster" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3", "namespace":"submariner-k8s-broker", "resourceVersion":"1721", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/clusters/cluster3", "uid":"b0da8c3d-b10e-4fc0-b95e-e145f3a95836"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.3.0.0/16"}, "cluster_id":"cluster3", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.3.0.0/16"}}}}
  6468. 2020-11-02T12:06:56.9494441Z [cluster2] I1102 12:05:30.487724 1 resource_syncer.go:276] Syncing resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3", "namespace":"submariner-k8s-broker", "resourceVersion":"1721", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/clusters/cluster3", "uid":"b0da8c3d-b10e-4fc0-b95e-e145f3a95836"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.3.0.0/16"}, "cluster_id":"cluster3", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.3.0.0/16"}}}}
  6469. 2020-11-02T12:06:56.9503845Z [cluster2] I1102 12:05:30.487805 1 federator.go:42] In Distribute for &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3", "namespace":"submariner-k8s-broker", "resourceVersion":"1721", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/clusters/cluster3", "uid":"b0da8c3d-b10e-4fc0-b95e-e145f3a95836"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.3.0.0/16"}, "cluster_id":"cluster3", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.3.0.0/16"}}}}
  6470. 2020-11-02T12:06:56.9507028Z [cluster2] I1102 12:05:30.487955 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  6471. 2020-11-02T12:06:56.9508287Z [cluster2] I1102 12:05:30.490083 1 resource_syncer.go:285] Syncer "broker -> local for *v1.Cluster" successfully synced "submariner-k8s-broker/cluster3"
  6472. 2020-11-02T12:06:56.9509420Z [cluster2] I1102 12:05:30.576885 1 resource_syncer.go:202] Syncer "local -> broker for *v1.Endpoint" started
  6473. 2020-11-02T12:06:56.9511029Z [cluster2] I1102 12:05:30.577042 1 resource_syncer.go:180] Starting syncer "broker -> local for *v1.Endpoint"
  6474. 2020-11-02T12:06:56.9512107Z [cluster2] I1102 12:05:30.577063 1 resource_syncer.go:202] Syncer "broker -> local for *v1.Endpoint" started
  6475. 2020-11-02T12:06:56.9512794Z [cluster2] I1102 12:05:30.577070 1 datastoresyncer.go:108] Ensuring we are the only endpoint active for this cluster
  6476. 2020-11-02T12:06:56.9517837Z [cluster2] I1102 12:05:30.576954 1 resource_syncer.go:264] Syncer "local -> broker for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:01:40Z", "generation":1, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-operator", "resourceVersion":"2845", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"8e4ce5be-6521-4f69-bef3-cde34ed638a9"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  6477. 2020-11-02T12:06:56.9527813Z [cluster2] I1102 12:05:30.577103 1 resource_syncer.go:276] Syncing resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:01:40Z", "generation":1, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-operator", "resourceVersion":"2845", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"8e4ce5be-6521-4f69-bef3-cde34ed638a9"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  6478. 2020-11-02T12:06:56.9550384Z [cluster2] I1102 12:05:30.577139 1 federator.go:42] In Distribute for &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:01:40Z", "generation":1, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-operator", "resourceVersion":"2845", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"8e4ce5be-6521-4f69-bef3-cde34ed638a9"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  6479. 2020-11-02T12:06:56.9554527Z [cluster2] I1102 12:05:30.577207 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  6480. 2020-11-02T12:06:56.9555572Z [cluster2] I1102 12:05:30.578063 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  6481. 2020-11-02T12:06:56.9560631Z [cluster2] I1102 12:05:30.578085 1 federator.go:73] Deleting resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:01:40Z", "generation":1, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-operator", "resourceVersion":"2845", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"8e4ce5be-6521-4f69-bef3-cde34ed638a9"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  6482. 2020-11-02T12:06:56.9567665Z [cluster2] I1102 12:05:30.581609 1 resource_syncer.go:285] Syncer "local -> broker for *v1.Endpoint" successfully synced "submariner-operator/cluster2-submariner-cable-cluster2-172-17-0-4"
  6483. 2020-11-02T12:06:56.9575778Z [cluster2] I1102 12:05:30.581745 1 resource_syncer.go:264] Syncer "local -> broker for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3-submariner-cable-cluster3-172-17-0-7", "namespace":"submariner-operator", "resourceVersion":"1772", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster3-submariner-cable-cluster3-172-17-0-7", "uid":"dfbc100c-1c97-4e89-95c6-f4d8406c55aa"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster3-172-17-0-7", "cluster_id":"cluster3", "hostname":"cluster3-worker", "nat_enabled":false, "private_ip":"172.17.0.7", "public_ip":"", "subnets":[]interface {}{"100.3.0.0/16", "10.3.0.0/16"}}}}
  6484. 2020-11-02T12:06:56.9580842Z [cluster2] I1102 12:05:30.581865 1 resource_syncer.go:437] Found cluster ID label "cluster3" - not syncing resource "cluster3-submariner-cable-cluster3-172-17-0-7"
  6485. 2020-11-02T12:06:56.9586489Z [cluster2] I1102 12:05:30.581883 1 resource_syncer.go:264] Syncer "local -> broker for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1-submariner-cable-cluster1-172-17-0-6", "namespace":"submariner-operator", "resourceVersion":"1789", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster1-submariner-cable-cluster1-172-17-0-6", "uid":"61f6c320-ac18-45ee-bb2d-00a06d34aab0"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster1-172-17-0-6", "cluster_id":"cluster1", "hostname":"cluster1-worker", "nat_enabled":false, "private_ip":"172.17.0.6", "public_ip":"", "subnets":[]interface {}{"100.1.0.0/16", "10.1.0.0/16"}}}}
  6486. 2020-11-02T12:06:56.9591559Z [cluster2] I1102 12:05:30.581918 1 resource_syncer.go:437] Found cluster ID label "cluster1" - not syncing resource "cluster1-submariner-cable-cluster1-172-17-0-6"
  6487. 2020-11-02T12:06:56.9598243Z [cluster2] I1102 12:05:30.587552 1 resource_syncer.go:264] Syncer "broker -> local for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:01:40Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-k8s-broker", "resourceVersion":"2629", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"bb7e9be5-2d33-4f1d-947e-4a46c6904408"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  6488. 2020-11-02T12:06:56.9609254Z [cluster2] I1102 12:05:30.587790 1 resource_syncer.go:443] Cluster ID label "cluster2" not present or matches local cluster ID "cluster2" - not syncing resource "cluster2-submariner-cable-cluster2-172-17-0-4"
  6489. 2020-11-02T12:06:56.9616148Z [cluster2] I1102 12:05:30.587882 1 resource_syncer.go:264] Syncer "broker -> local for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3-submariner-cable-cluster3-172-17-0-7", "namespace":"submariner-k8s-broker", "resourceVersion":"1722", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster3-submariner-cable-cluster3-172-17-0-7", "uid":"f4e3ecdd-5302-4901-bfde-9d6483becbcb"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster3-172-17-0-7", "cluster_id":"cluster3", "hostname":"cluster3-worker", "nat_enabled":false, "private_ip":"172.17.0.7", "public_ip":"", "subnets":[]interface {}{"100.3.0.0/16", "10.3.0.0/16"}}}}
  6490. 2020-11-02T12:06:56.9625147Z [cluster2] I1102 12:05:30.588040 1 resource_syncer.go:276] Syncing resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3-submariner-cable-cluster3-172-17-0-7", "namespace":"submariner-k8s-broker", "resourceVersion":"1722", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster3-submariner-cable-cluster3-172-17-0-7", "uid":"f4e3ecdd-5302-4901-bfde-9d6483becbcb"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster3-172-17-0-7", "cluster_id":"cluster3", "hostname":"cluster3-worker", "nat_enabled":false, "private_ip":"172.17.0.7", "public_ip":"", "subnets":[]interface {}{"100.3.0.0/16", "10.3.0.0/16"}}}}
  6491. 2020-11-02T12:06:56.9634900Z [cluster2] I1102 12:05:30.588328 1 federator.go:42] In Distribute for &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3-submariner-cable-cluster3-172-17-0-7", "namespace":"submariner-k8s-broker", "resourceVersion":"1722", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster3-submariner-cable-cluster3-172-17-0-7", "uid":"f4e3ecdd-5302-4901-bfde-9d6483becbcb"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster3-172-17-0-7", "cluster_id":"cluster3", "hostname":"cluster3-worker", "nat_enabled":false, "private_ip":"172.17.0.7", "public_ip":"", "subnets":[]interface {}{"100.3.0.0/16", "10.3.0.0/16"}}}}
  6492. 2020-11-02T12:06:56.9639132Z [cluster2] I1102 12:05:30.588722 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  6493. 2020-11-02T12:06:56.9644805Z [cluster2] I1102 12:05:30.591448 1 resource_syncer.go:294] Syncer "Tunnel controller" informed of deleted resource "submariner-operator/cluster2-submariner-cable-cluster2-172-17-0-4"
  6494. 2020-11-02T12:06:56.9655844Z [cluster2] I1102 12:05:30.591804 1 tunnel.go:74] Tunnel controller processing removed submariner Endpoint object: &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster2-submariner-cable-cluster2-172-17-0-4", GenerateName:"", Namespace:"submariner-operator", SelfLink:"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", UID:"8e4ce5be-6521-4f69-bef3-cde34ed638a9", ResourceVersion:"3728", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63739915300, loc:(*time.Location)(0x2020aa0)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster2", CableName:"submariner-cable-cluster2-172-17-0-4", Hostname:"cluster2-worker", Subnets:[]string{"100.2.0.0/16", "10.2.0.0/16"}, PrivateIP:"172.17.0.4", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string(nil)}}
  6495. 2020-11-02T12:06:56.9670397Z [cluster2] I1102 12:05:30.591991 1 cableengine.go:166] Removing Endpoint cable "submariner-cable-cluster2-172-17-0-4"
  6496. 2020-11-02T12:06:56.9674533Z [cluster2] I1102 12:05:30.592122 1 libreswan.go:300] Deleting connection to {{cluster2 submariner-cable-cluster2-172-17-0-4 cluster2-worker [100.2.0.0/16 10.2.0.0/16] 172.17.0.4 false libreswan map[]}}
  6497. 2020-11-02T12:06:56.9676888Z [cluster2] I1102 12:05:30.592365 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-0-0]
  6498. 2020-11-02T12:06:56.9678834Z [cluster2] I1102 12:05:30.592537 1 resource_syncer.go:294] Syncer "local -> broker for *v1.Endpoint" informed of deleted resource "submariner-operator/cluster2-submariner-cable-cluster2-172-17-0-4"
  6499. 2020-11-02T12:06:56.9686941Z [cluster2] I1102 12:05:30.592895 1 resource_syncer.go:315] Syncer "local -> broker for *v1.Endpoint" deleting resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:01:40Z", "generation":1, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-operator", "resourceVersion":"3728", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"8e4ce5be-6521-4f69-bef3-cde34ed638a9"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  6500. 2020-11-02T12:06:56.9692269Z [cluster2] I1102 12:05:30.593083 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  6501. 2020-11-02T12:06:56.9697530Z [cluster2] I1102 12:05:30.593223 1 federator.go:73] Deleting resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:01:40Z", "generation":1, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-k8s-broker", "resourceVersion":"3728", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"8e4ce5be-6521-4f69-bef3-cde34ed638a9"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  6502. 2020-11-02T12:06:56.9703256Z [cluster2] I1102 12:05:30.599263 1 datastoresyncer.go:136] Successfully deleted existing submariner Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  6503. 2020-11-02T12:06:56.9705102Z [cluster2] I1102 12:05:30.599383 1 datastoresyncer.go:143] Creating local submariner Cluster: types.SubmarinerCluster{ID:"cluster2", Spec:v1.ClusterSpec{ClusterID:"cluster2", ColorCodes:[]string{"blue"}, ServiceCIDR:[]string{"100.2.0.0/16"}, ClusterCIDR:[]string{"10.2.0.0/16"}, GlobalCIDR:[]string{}}}
  6504. 2020-11-02T12:06:56.9709056Z [cluster2] I1102 12:05:30.599558 1 federator.go:42] In Distribute for &v1.Cluster{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster2", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.ClusterSpec{ClusterID:"cluster2", ColorCodes:[]string{"blue"}, ServiceCIDR:[]string{"100.2.0.0/16"}, ClusterCIDR:[]string{"10.2.0.0/16"}, GlobalCIDR:[]string{}}}
  6505. 2020-11-02T12:06:56.9712854Z [cluster2] I1102 12:05:30.599724 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  6506. 2020-11-02T12:06:56.9715914Z [cluster2] I1102 12:05:30.600733 1 resource_syncer.go:330] Syncer "local -> broker for *v1.Endpoint" successfully deleted "submariner-operator/cluster2-submariner-cable-cluster2-172-17-0-4"
  6507. 2020-11-02T12:06:56.9717832Z [cluster2] I1102 12:05:30.602912 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-0-1]
  6508. 2020-11-02T12:06:56.9719908Z [cluster2] I1102 12:05:30.604549 1 resource_syncer.go:285] Syncer "broker -> local for *v1.Endpoint" successfully synced "submariner-k8s-broker/cluster3-submariner-cable-cluster3-172-17-0-7"
  6509. 2020-11-02T12:06:56.9726596Z [cluster2] I1102 12:05:30.604674 1 resource_syncer.go:264] Syncer "broker -> local for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1-submariner-cable-cluster1-172-17-0-6", "namespace":"submariner-k8s-broker", "resourceVersion":"1743", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster1-submariner-cable-cluster1-172-17-0-6", "uid":"b8932e09-5864-498e-b04f-0147ed950e52"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster1-172-17-0-6", "cluster_id":"cluster1", "hostname":"cluster1-worker", "nat_enabled":false, "private_ip":"172.17.0.6", "public_ip":"", "subnets":[]interface {}{"100.1.0.0/16", "10.1.0.0/16"}}}}
  6510. 2020-11-02T12:06:56.9735974Z [cluster2] I1102 12:05:30.604843 1 resource_syncer.go:276] Syncing resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1-submariner-cable-cluster1-172-17-0-6", "namespace":"submariner-k8s-broker", "resourceVersion":"1743", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster1-submariner-cable-cluster1-172-17-0-6", "uid":"b8932e09-5864-498e-b04f-0147ed950e52"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster1-172-17-0-6", "cluster_id":"cluster1", "hostname":"cluster1-worker", "nat_enabled":false, "private_ip":"172.17.0.6", "public_ip":"", "subnets":[]interface {}{"100.1.0.0/16", "10.1.0.0/16"}}}}
  6511. 2020-11-02T12:06:56.9744909Z [cluster2] I1102 12:05:30.604992 1 federator.go:42] In Distribute for &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1-submariner-cable-cluster1-172-17-0-6", "namespace":"submariner-k8s-broker", "resourceVersion":"1743", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster1-submariner-cable-cluster1-172-17-0-6", "uid":"b8932e09-5864-498e-b04f-0147ed950e52"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster1-172-17-0-6", "cluster_id":"cluster1", "hostname":"cluster1-worker", "nat_enabled":false, "private_ip":"172.17.0.6", "public_ip":"", "subnets":[]interface {}{"100.1.0.0/16", "10.1.0.0/16"}}}}
  6512. 2020-11-02T12:06:56.9749971Z [cluster2] I1102 12:05:30.605131 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  6513. 2020-11-02T12:06:56.9752251Z [cluster2] I1102 12:05:30.605513 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-0-2]
  6514. 2020-11-02T12:06:56.9753760Z [cluster2] I1102 12:05:30.618967 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-1-0]
  6515. 2020-11-02T12:06:56.9756391Z [cluster2] I1102 12:05:30.627381 1 resource_syncer.go:285] Syncer "broker -> local for *v1.Endpoint" successfully synced "submariner-k8s-broker/cluster1-submariner-cable-cluster1-172-17-0-6"
  6516. 2020-11-02T12:06:56.9758673Z [cluster2] I1102 12:05:30.627461 1 resource_syncer.go:294] Syncer "broker -> local for *v1.Endpoint" informed of deleted resource "submariner-k8s-broker/cluster2-submariner-cable-cluster2-172-17-0-4"
  6517. 2020-11-02T12:06:56.9760907Z [cluster2] I1102 12:05:30.627507 1 resource_syncer.go:443] Cluster ID label "cluster2" not present or matches local cluster ID "cluster2" - not syncing resource "cluster2-submariner-cable-cluster2-172-17-0-4"
  6518. 2020-11-02T12:06:56.9762855Z [cluster2] I1102 12:05:30.631951 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-1-1]
  6519. 2020-11-02T12:06:56.9766565Z [cluster2] I1102 12:05:30.633142 1 datastoresyncer.go:156] Creating local submariner Endpoint: types.SubmarinerEndpoint{Spec:v1.EndpointSpec{ClusterID:"cluster2", CableName:"submariner-cable-cluster2-172-17-0-11", Hostname:"cluster2-worker2", Subnets:[]string{"100.2.0.0/16", "10.2.0.0/16"}, PrivateIP:"172.17.0.11", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string{}}}
  6520. 2020-11-02T12:06:56.9772932Z [cluster2] I1102 12:05:30.633260 1 federator.go:42] In Distribute for &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster2-submariner-cable-cluster2-172-17-0-11", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster2", CableName:"submariner-cable-cluster2-172-17-0-11", Hostname:"cluster2-worker2", Subnets:[]string{"100.2.0.0/16", "10.2.0.0/16"}, PrivateIP:"172.17.0.11", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string{}}}
  6521. 2020-11-02T12:06:56.9778430Z [cluster2] I1102 12:05:30.633409 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  6522. 2020-11-02T12:06:56.9780747Z [cluster2] I1102 12:05:30.634400 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-1-2]
  6523. 2020-11-02T12:06:56.9782236Z [cluster2] I1102 12:05:30.637052 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-2-0]
  6524. 2020-11-02T12:06:56.9783703Z [cluster2] I1102 12:05:30.637881 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-2-1]
  6525. 2020-11-02T12:06:56.9788055Z [cluster2] I1102 12:05:30.640178 1 create_or_update.go:32] Creating resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"name":"cluster2-submariner-cable-cluster2-172-17-0-11", "namespace":"submariner-operator"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-11", "cluster_id":"cluster2", "hostname":"cluster2-worker2", "nat_enabled":false, "private_ip":"172.17.0.11", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  6526. 2020-11-02T12:06:56.9791492Z [cluster2] I1102 12:05:30.642695 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-2-2]
  6527. 2020-11-02T12:06:56.9793456Z [cluster2] I1102 12:05:30.643586 1 cableengine.go:176] Successfully removed Endpoint cable "submariner-cable-cluster2-172-17-0-4"
  6528. 2020-11-02T12:06:56.9795165Z [cluster2] I1102 12:05:30.643686 1 tunnel.go:85] Tunnel controller successfully removed Endpoint cable submariner-cable-cluster2-172-17-0-4 from the engine
  6529. 2020-11-02T12:06:56.9796583Z [cluster2] I1102 12:05:30.643781 1 resource_syncer.go:352] Syncer "Tunnel controller": transform function returned nil - not syncing - requeue: false
  6530. 2020-11-02T12:06:56.9803090Z [cluster2] I1102 12:05:30.650438 1 resource_syncer.go:264] Syncer "Tunnel controller" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:05:30Z", "generation":1, "name":"cluster2-submariner-cable-cluster2-172-17-0-11", "namespace":"submariner-operator", "resourceVersion":"3729", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-11", "uid":"4dbea653-9ed8-4f2e-9fd3-5ab655122bcd"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-11", "cluster_id":"cluster2", "hostname":"cluster2-worker2", "nat_enabled":false, "private_ip":"172.17.0.11", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  6531. 2020-11-02T12:06:56.9812807Z [cluster2] I1102 12:05:30.650593 1 tunnel.go:56] Tunnel controller processing added or updated submariner Endpoint object: &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster2-submariner-cable-cluster2-172-17-0-11", GenerateName:"", Namespace:"submariner-operator", SelfLink:"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-11", UID:"4dbea653-9ed8-4f2e-9fd3-5ab655122bcd", ResourceVersion:"3729", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63739915530, loc:(*time.Location)(0x2020aa0)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster2", CableName:"submariner-cable-cluster2-172-17-0-11", Hostname:"cluster2-worker2", Subnets:[]string{"100.2.0.0/16", "10.2.0.0/16"}, PrivateIP:"172.17.0.11", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string(nil)}}
  6532. 2020-11-02T12:06:56.9818265Z [cluster2] I1102 12:05:30.650755 1 cableengine.go:92] Not installing cable for local cluster
  6533. 2020-11-02T12:06:56.9820012Z [cluster2] I1102 12:05:30.650847 1 tunnel.go:68] Tunnel controller successfully installed Endpoint cable submariner-cable-cluster2-172-17-0-11 in the engine
  6534. 2020-11-02T12:06:56.9821435Z [cluster2] I1102 12:05:30.650931 1 resource_syncer.go:352] Syncer "Tunnel controller": transform function returned nil - not syncing - requeue: false
  6535. 2020-11-02T12:06:56.9826521Z [cluster2] I1102 12:05:30.652189 1 resource_syncer.go:264] Syncer "local -> broker for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:05:30Z", "generation":1, "name":"cluster2-submariner-cable-cluster2-172-17-0-11", "namespace":"submariner-operator", "resourceVersion":"3729", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-11", "uid":"4dbea653-9ed8-4f2e-9fd3-5ab655122bcd"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-11", "cluster_id":"cluster2", "hostname":"cluster2-worker2", "nat_enabled":false, "private_ip":"172.17.0.11", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  6536. 2020-11-02T12:06:56.9835385Z [cluster2] I1102 12:05:30.652253 1 resource_syncer.go:276] Syncing resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:05:30Z", "generation":1, "name":"cluster2-submariner-cable-cluster2-172-17-0-11", "namespace":"submariner-operator", "resourceVersion":"3729", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-11", "uid":"4dbea653-9ed8-4f2e-9fd3-5ab655122bcd"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-11", "cluster_id":"cluster2", "hostname":"cluster2-worker2", "nat_enabled":false, "private_ip":"172.17.0.11", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  6537. 2020-11-02T12:06:56.9844061Z [cluster2] I1102 12:05:30.652294 1 federator.go:42] In Distribute for &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:05:30Z", "generation":1, "name":"cluster2-submariner-cable-cluster2-172-17-0-11", "namespace":"submariner-operator", "resourceVersion":"3729", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-11", "uid":"4dbea653-9ed8-4f2e-9fd3-5ab655122bcd"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-11", "cluster_id":"cluster2", "hostname":"cluster2-worker2", "nat_enabled":false, "private_ip":"172.17.0.11", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  6538. 2020-11-02T12:06:56.9848072Z [cluster2] I1102 12:05:30.652353 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  6539. 2020-11-02T12:06:56.9848966Z [cluster2] I1102 12:05:30.653218 1 datastoresyncer.go:99] Datastore syncer started
  6540. 2020-11-02T12:06:56.9854900Z [cluster2] I1102 12:05:30.654510 1 create_or_update.go:32] Creating resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-11", "namespace":"submariner-k8s-broker"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-11", "cluster_id":"cluster2", "hostname":"cluster2-worker2", "nat_enabled":false, "private_ip":"172.17.0.11", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  6541. 2020-11-02T12:06:56.9859041Z [cluster2] I1102 12:05:30.665274 1 resource_syncer.go:285] Syncer "local -> broker for *v1.Endpoint" successfully synced "submariner-operator/cluster2-submariner-cable-cluster2-172-17-0-11"
  6542. 2020-11-02T12:06:57.2329021Z [submariner]$ [cluster2] kubectl -n submariner-operator logs submariner-routeagent-8x2cc
  6543. 2020-11-02T12:06:57.2338141Z [submariner]$ [cluster2] kubectl -n submariner-operator logs submariner-routeagent-8x2cc
  6544. 2020-11-02T12:06:57.2380487Z [submariner]$ [cluster2] command kubectl --context=cluster2 -n submariner-operator logs submariner-routeagent-8x2cc
  6545. 2020-11-02T12:06:57.2391887Z [submariner]$ [cluster2] kubectl --context=cluster2 -n submariner-operator logs submariner-routeagent-8x2cc
  6546. 2020-11-02T12:06:57.5706958Z [cluster2] I1102 12:05:30.667950 1 resource_syncer.go:264] Syncer "broker -> local for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:05:30Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-11", "namespace":"submariner-k8s-broker", "resourceVersion":"3455", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-11", "uid":"59c1f9b2-d440-4647-9bdd-1450a56598e9"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-11", "cluster_id":"cluster2", "hostname":"cluster2-worker2", "nat_enabled":false, "private_ip":"172.17.0.11", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  6547. 2020-11-02T12:06:57.5711783Z [cluster2] I1102 12:05:30.668019 1 resource_syncer.go:443] Cluster ID label "cluster2" not present or matches local cluster ID "cluster2" - not syncing resource "cluster2-submariner-cable-cluster2-172-17-0-11"
  6548. 2020-11-02T12:06:57.5713904Z [cluster2] I1102 12:05:31.061040 1 libreswan.go:149] Connection "submariner-cable-cluster1-172-17-0-6" not found in active connections obtained from whack: [submariner-cable-cluster3-172-17-0-7-0-0]
  6549. 2020-11-02T12:06:57.5714995Z [cluster2] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  6550. 2020-11-02T12:06:57.5715972Z [cluster2] +++++++++++++++++++++: Logs for Pod submariner-routeagent-8x2cc in namespace submariner-operator :++++++++++++++++++++++
  6551. 2020-11-02T12:06:57.5716760Z [cluster2] + trap 'exit 1' SIGTERM SIGINT
  6552. 2020-11-02T12:06:57.5717122Z [cluster2] + SUBMARINER_VERBOSITY=2
  6553. 2020-11-02T12:06:57.5717573Z [cluster2] + '[' false == true ']'
  6554. 2020-11-02T12:06:57.5717959Z [cluster2] + DEBUG=-v=2
  6555. 2020-11-02T12:06:57.5718426Z [cluster2] + for f in iptables-save iptables
  6556. 2020-11-02T12:06:57.5718973Z [cluster2] ++ find_iptables_on_host iptables-save
  6557. 2020-11-02T12:06:57.5719548Z [cluster2] ++ chroot /host test -x /usr/sbin/iptables-save
  6558. 2020-11-02T12:06:57.5720208Z [cluster2] iptables-save is present on the host at /usr/sbin/iptables-save
  6559. 2020-11-02T12:06:57.5720619Z [cluster2] ++ echo /usr/sbin
  6560. 2020-11-02T12:06:57.5720888Z [cluster2] ++ return
  6561. 2020-11-02T12:06:57.5721174Z [cluster2] + location=/usr/sbin
  6562. 2020-11-02T12:06:57.5721620Z [cluster2] + '[' /usr/sbin '!=' unknown ']'
  6563. 2020-11-02T12:06:57.5722701Z [cluster2] + echo 'iptables-save is present on the host at /usr/sbin/iptables-save'
  6564. 2020-11-02T12:06:57.5723427Z [cluster2] + sed 's!@@PATH@@!/usr/sbin!' /usr/sbin/iptables-wrapper.in
  6565. 2020-11-02T12:06:57.5724170Z [cluster2] + for f in iptables-save iptables
  6566. 2020-11-02T12:06:57.5724568Z [cluster2] ++ find_iptables_on_host iptables
  6567. 2020-11-02T12:06:57.5725108Z [cluster2] ++ chroot /host test -x /usr/sbin/iptables
  6568. 2020-11-02T12:06:57.5725447Z [cluster2] ++ echo /usr/sbin
  6569. 2020-11-02T12:06:57.5725718Z [cluster2] ++ return
  6570. 2020-11-02T12:06:57.5726002Z [cluster2] + location=/usr/sbin
  6571. 2020-11-02T12:06:57.5726450Z [cluster2] + '[' /usr/sbin '!=' unknown ']'
  6572. 2020-11-02T12:06:57.5727009Z [cluster2] + echo 'iptables is present on the host at /usr/sbin/iptables'
  6573. 2020-11-02T12:06:57.5727673Z [cluster2] + sed 's!@@PATH@@!/usr/sbin!' /usr/sbin/iptables-wrapper.in
  6574. 2020-11-02T12:06:57.5728172Z [cluster2] iptables is present on the host at /usr/sbin/iptables
  6575. 2020-11-02T12:06:57.5728852Z [cluster2] + exec submariner-route-agent -v=2 -alsologtostderr
  6576. 2020-11-02T12:06:57.5729642Z [cluster2] I1102 11:58:22.419530 1 main.go:47] Starting submariner-route-agent
  6577. 2020-11-02T12:06:57.5730619Z [cluster2] W1102 11:58:22.419850 1 client_config.go:543] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
  6578. 2020-11-02T12:06:57.5731684Z [cluster2] I1102 11:58:22.424171 1 route.go:192] Starting Route Controller. ClusterID: cluster2, localClusterCIDR: [10.2.0.0/16], localServiceCIDR: [100.2.0.0/16]
  6579. 2020-11-02T12:06:57.5732444Z [cluster2] I1102 11:58:22.430846 1 route.go:196] Waiting for Endpoint informer caches to sync.
  6580. 2020-11-02T12:06:57.5733506Z [cluster2] I1102 11:58:22.458920 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.11
  6581. 2020-11-02T12:06:57.5734326Z [cluster2] I1102 11:58:22.458939 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.4
  6582. 2020-11-02T12:06:57.5736247Z [cluster2] I1102 11:58:22.458949 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.8
  6583. 2020-11-02T12:06:57.5736947Z [cluster2] I1102 11:58:22.531275 1 driver.go:37] Interface "lo" has "127.0.0.1" address
  6584. 2020-11-02T12:06:57.5737499Z [cluster2] I1102 11:58:22.531405 1 driver.go:37] Interface "weave" has "10.2.0.1" address
  6585. 2020-11-02T12:06:57.5738159Z [cluster2] I1102 11:58:22.531412 1 driver.go:42] Found CNI Interface "weave" that has IP "10.2.0.1" from ClusterCIDR "10.2.0.0/16"
  6586. 2020-11-02T12:06:57.5738965Z [cluster2] I1102 11:58:22.531564 1 driver.go:60] Successfully configured rp_filter to loose mode(2) on cniInterface "weave"
  6587. 2020-11-02T12:06:57.5740249Z [cluster2] I1102 11:58:22.538184 1 iptables.go:22] Install/ensure SUBMARINER-POSTROUTING chain exists
  6588. 2020-11-02T12:06:57.5741471Z [cluster2] I1102 11:58:22.558945 1 iptables.go:28] Insert SUBMARINER-POSTROUTING rule that has rules for inter-cluster traffic
  6589. 2020-11-02T12:06:57.5742443Z [cluster2] I1102 11:58:22.569638 1 iptables.go:34] Install/ensure SUBMARINER-INPUT chain exists
  6590. 2020-11-02T12:06:57.5763181Z [cluster2] I1102 11:58:22.592035 1 iptables.go:45] Allow VxLAN incoming traffic in SUBMARINER-INPUT Chain
  6591. 2020-11-02T12:06:57.5764654Z [cluster2] I1102 11:58:22.600231 1 iptables.go:53] Insert rule to allow traffic over vx-submariner interface in FORWARDing Chain
  6592. 2020-11-02T12:06:57.5765767Z [cluster2] I1102 11:58:22.609674 1 iptables.go:65] Installing rule for host network to remote cluster communication: -s 240.0.0.0/8 -o vx-submariner -j SNAT --to 10.2.0.1
  6593. 2020-11-02T12:06:57.5766726Z [cluster2] I1102 11:58:22.624690 1 route.go:254] In cluster2, podIP of submariner-route-agent[0] is 172.17.0.4
  6594. 2020-11-02T12:06:57.5767579Z [cluster2] I1102 11:58:22.624744 1 route.go:254] In cluster2, podIP of submariner-route-agent[1] is 172.17.0.8
  6595. 2020-11-02T12:06:57.5768432Z [cluster2] I1102 11:58:22.624751 1 route.go:254] In cluster2, podIP of submariner-route-agent[2] is 172.17.0.11
  6596. 2020-11-02T12:06:57.5769613Z [cluster2] I1102 11:58:22.624755 1 route.go:268] Hostname is "cluster2-worker" and routeAgentNodeName is "cluster2-worker"
  6597. 2020-11-02T12:06:57.5770590Z [cluster2] I1102 11:58:22.636689 1 driver.go:87] Successfully annotated node "cluster2-worker" with cniIfaceIP "10.2.0.1"
  6598. 2020-11-02T12:06:57.5771288Z [cluster2] I1102 11:58:22.636714 1 route.go:286] Route agent workers started
  6599. 2020-11-02T12:06:57.5772552Z [cluster2] I1102 11:58:22.636929 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  6600. 2020-11-02T12:06:57.5773804Z [cluster2] I1102 11:58:22.637701 1 route.go:756] Processing route {Ifindex: 8 Dst: <nil> Src: <nil> Gw: 172.17.0.1 Flags: [] Table: 254}
  6601. 2020-11-02T12:06:57.5777242Z [cluster2] I1102 11:58:22.637753 1 route.go:759] Found nil gw or dst
  6602. 2020-11-02T12:06:57.5779083Z [cluster2] I1102 11:58:22.637758 1 route.go:756] Processing route {Ifindex: 4 Dst: 10.2.0.0/16 Src: 10.2.0.1 Gw: <nil> Flags: [] Table: 254}
  6603. 2020-11-02T12:06:57.5779608Z [cluster2] I1102 11:58:22.637772 1 route.go:759] Found nil gw or dst
  6604. 2020-11-02T12:06:57.5780258Z [cluster2] I1102 11:58:22.637775 1 route.go:756] Processing route {Ifindex: 8 Dst: 172.17.0.0/16 Src: 172.17.0.4 Gw: <nil> Flags: [] Table: 254}
  6605. 2020-11-02T12:06:57.5782085Z [cluster2] I1102 11:58:22.637781 1 route.go:759] Found nil gw or dst
  6606. 2020-11-02T12:06:57.5782582Z [cluster2] I1102 11:58:22.637788 1 route.go:582] This route agent is running on the active gateway node
  6607. 2020-11-02T12:06:57.5783457Z [cluster2] I1102 11:58:22.637800 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  6608. 2020-11-02T12:06:57.5785747Z [cluster2] I1102 11:58:22.636947 1 route.go:507] Processing submariner-route-agent pod "submariner-routeagent-swbc2" with host IP "172.17.0.11", pod IP "172.17.0.11"
  6609. 2020-11-02T12:06:57.5786783Z [cluster2] I1102 11:58:22.638692 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.4 00:00:00:00:00:00
  6610. 2020-11-02T12:06:57.5788459Z [cluster2] I1102 11:58:22.638742 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.8 00:00:00:00:00:00
  6611. 2020-11-02T12:06:57.5788987Z [cluster2] I1102 11:58:22.638912 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.11 00:00:00:00:00:00
  6612. 2020-11-02T12:06:57.5790995Z [cluster2] I1102 11:58:22.638983 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  6613. 2020-11-02T12:06:57.5791612Z [cluster2] I1102 11:58:22.639491 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.11 00:00:00:00:00:00
  6614. 2020-11-02T12:06:57.5793750Z [cluster2] I1102 11:58:22.639522 1 route.go:524] FDB entry added on the Gateway node's vxlan iface for route-agent pod "submariner-routeagent-swbc2", IP "172.17.0.11"
  6615. 2020-11-02T12:06:57.5796359Z [cluster2] I1102 11:58:22.639589 1 route.go:531] Successfully processed submariner-route-agent pod "submariner-routeagent-swbc2"
  6616. 2020-11-02T12:06:57.5805526Z [cluster2] I1102 11:58:22.639705 1 route.go:507] Processing submariner-route-agent pod "submariner-routeagent-8x2cc" with host IP "172.17.0.4", pod IP "172.17.0.4"
  6617. 2020-11-02T12:06:57.5806389Z [cluster2] I1102 11:58:22.639741 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.4 00:00:00:00:00:00
  6618. 2020-11-02T12:06:57.5807852Z [cluster2] I1102 11:58:22.639749 1 route.go:524] FDB entry added on the Gateway node's vxlan iface for route-agent pod "submariner-routeagent-8x2cc", IP "172.17.0.4"
  6619. 2020-11-02T12:06:57.5809154Z [cluster2] I1102 11:58:22.639766 1 route.go:531] Successfully processed submariner-route-agent pod "submariner-routeagent-8x2cc"
  6620. 2020-11-02T12:06:57.5810776Z [cluster2] I1102 11:58:22.639772 1 route.go:507] Processing submariner-route-agent pod "submariner-routeagent-ftrjn" with host IP "172.17.0.8", pod IP "172.17.0.8"
  6621. 2020-11-02T12:06:57.5811948Z [cluster2] I1102 11:58:22.639817 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.8 00:00:00:00:00:00
  6622. 2020-11-02T12:06:57.5813246Z [cluster2] I1102 11:58:22.639998 1 route.go:524] FDB entry added on the Gateway node's vxlan iface for route-agent pod "submariner-routeagent-ftrjn", IP "172.17.0.8"
  6623. 2020-11-02T12:06:57.5814696Z [cluster2] I1102 11:58:22.640009 1 route.go:531] Successfully processed submariner-route-agent pod "submariner-routeagent-ftrjn"
  6624. 2020-11-02T12:06:57.5816267Z [cluster2] I1102 11:58:24.184510 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  6625. 2020-11-02T12:06:57.5817571Z [cluster2] I1102 11:58:24.215567 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.2.0.0/16 -d 100.3.0.0/16 -j ACCEPT
  6626. 2020-11-02T12:06:57.5818451Z [cluster2] I1102 11:58:24.324291 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 100.3.0.0/16 -d 10.2.0.0/16 -j ACCEPT
  6627. 2020-11-02T12:06:57.5819342Z [cluster2] I1102 11:58:24.376771 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.2.0.0/16 -d 10.3.0.0/16 -j ACCEPT
  6628. 2020-11-02T12:06:57.5820213Z [cluster2] I1102 11:58:24.398437 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 10.3.0.0/16 -d 10.2.0.0/16 -j ACCEPT
  6629. 2020-11-02T12:06:57.5821647Z [cluster2] I1102 11:58:27.573169 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  6630. 2020-11-02T12:06:57.5822992Z [cluster2] I1102 11:58:27.590366 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.2.0.0/16 -d 100.1.0.0/16 -j ACCEPT
  6631. 2020-11-02T12:06:57.5823870Z [cluster2] I1102 11:58:27.725245 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 100.1.0.0/16 -d 10.2.0.0/16 -j ACCEPT
  6632. 2020-11-02T12:06:57.5824739Z [cluster2] I1102 11:58:27.816203 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.2.0.0/16 -d 10.1.0.0/16 -j ACCEPT
  6633. 2020-11-02T12:06:57.5825621Z [cluster2] I1102 11:58:27.835583 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 10.1.0.0/16 -d 10.2.0.0/16 -j ACCEPT
  6634. 2020-11-02T12:06:57.5826980Z [cluster2] I1102 11:58:52.459773 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  6635. 2020-11-02T12:06:57.5828067Z [cluster2] I1102 11:58:52.460023 1 route.go:756] Processing route {Ifindex: 16 Dst: 240.0.0.0/8 Src: 240.17.0.4 Gw: <nil> Flags: [] Table: 254}
  6636. 2020-11-02T12:06:57.5828548Z [cluster2] I1102 11:58:52.460058 1 route.go:759] Found nil gw or dst
  6637. 2020-11-02T12:06:57.5829012Z [cluster2] I1102 11:58:52.460064 1 route.go:582] This route agent is running on the active gateway node
  6638. 2020-11-02T12:06:57.5829817Z [cluster2] I1102 11:58:52.460071 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  6639. 2020-11-02T12:06:57.5830465Z [cluster2] I1102 11:58:52.460254 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  6640. 2020-11-02T12:06:57.5831054Z [cluster2] I1102 11:58:52.460316 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.11 00:00:00:00:00:00
  6641. 2020-11-02T12:06:57.5831577Z [cluster2] I1102 11:58:52.460347 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.4 00:00:00:00:00:00
  6642. 2020-11-02T12:06:57.5832116Z [cluster2] I1102 11:58:52.460372 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.8 00:00:00:00:00:00
  6643. 2020-11-02T12:06:57.5833146Z [cluster2] I1102 11:58:52.460544 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  6644. 2020-11-02T12:06:57.5834750Z [cluster2] I1102 11:58:52.460630 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  6645. 2020-11-02T12:06:57.5836794Z [cluster2] I1102 11:58:52.460646 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  6646. 2020-11-02T12:06:57.5839169Z [cluster2] I1102 11:59:22.463637 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  6647. 2020-11-02T12:06:57.5840255Z [cluster2] I1102 11:59:22.463902 1 route.go:756] Processing route {Ifindex: 16 Dst: 240.0.0.0/8 Src: 240.17.0.4 Gw: <nil> Flags: [] Table: 254}
  6648. 2020-11-02T12:06:57.5840736Z [cluster2] I1102 11:59:22.463938 1 route.go:759] Found nil gw or dst
  6649. 2020-11-02T12:06:57.5841376Z [cluster2] I1102 11:59:22.463946 1 route.go:582] This route agent is running on the active gateway node
  6650. 2020-11-02T12:06:57.5842214Z [cluster2] I1102 11:59:22.463953 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  6651. 2020-11-02T12:06:57.5842855Z [cluster2] I1102 11:59:22.464112 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  6652. 2020-11-02T12:06:57.5843443Z [cluster2] I1102 11:59:22.464190 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.4 00:00:00:00:00:00
  6653. 2020-11-02T12:06:57.5844347Z [cluster2] I1102 11:59:22.464231 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.8 00:00:00:00:00:00
  6654. 2020-11-02T12:06:57.5845001Z [cluster2] I1102 11:59:22.464266 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.11 00:00:00:00:00:00
  6655. 2020-11-02T12:06:57.5845957Z [cluster2] I1102 11:59:22.464351 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  6656. 2020-11-02T12:06:57.5847313Z [cluster2] I1102 11:59:22.464541 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  6657. 2020-11-02T12:06:57.5849075Z [cluster2] I1102 11:59:22.464570 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  6658. 2020-11-02T12:06:57.5850845Z [cluster2] I1102 11:59:52.464104 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  6659. 2020-11-02T12:06:57.5852610Z [cluster2] I1102 11:59:52.464161 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  6660. 2020-11-02T12:06:57.5853681Z [cluster2] I1102 11:59:52.464438 1 route.go:756] Processing route {Ifindex: 16 Dst: 240.0.0.0/8 Src: 240.17.0.4 Gw: <nil> Flags: [] Table: 254}
  6661. 2020-11-02T12:06:57.5854164Z [cluster2] I1102 11:59:52.464629 1 route.go:759] Found nil gw or dst
  6662. 2020-11-02T12:06:57.5854627Z [cluster2] I1102 11:59:52.464643 1 route.go:582] This route agent is running on the active gateway node
  6663. 2020-11-02T12:06:57.5855589Z [cluster2] I1102 11:59:52.464650 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  6664. 2020-11-02T12:06:57.5856822Z [cluster2] I1102 11:59:52.464792 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  6665. 2020-11-02T12:06:57.5857430Z [cluster2] I1102 11:59:52.464837 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.4 00:00:00:00:00:00
  6666. 2020-11-02T12:06:57.5857975Z [cluster2] I1102 11:59:52.464875 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.8 00:00:00:00:00:00
  6667. 2020-11-02T12:06:57.5858514Z [cluster2] I1102 11:59:52.464906 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.11 00:00:00:00:00:00
  6668. 2020-11-02T12:06:57.5859499Z [cluster2] I1102 11:59:52.465031 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  6669. 2020-11-02T12:06:57.5860853Z [cluster2] I1102 11:59:52.465088 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  6670. 2020-11-02T12:06:57.5862997Z [cluster2] I1102 12:00:22.464351 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  6671. 2020-11-02T12:06:57.5864772Z [cluster2] I1102 12:00:22.464526 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  6672. 2020-11-02T12:06:57.5866552Z [cluster2] I1102 12:00:22.464544 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  6673. 2020-11-02T12:06:57.5890195Z [cluster2] I1102 12:00:22.464789 1 route.go:756] Processing route {Ifindex: 16 Dst: 240.0.0.0/8 Src: 240.17.0.4 Gw: <nil> Flags: [] Table: 254}
  6674. 2020-11-02T12:06:57.5890682Z [cluster2] I1102 12:00:22.464810 1 route.go:759] Found nil gw or dst
  6675. 2020-11-02T12:06:57.5891134Z [cluster2] I1102 12:00:22.464817 1 route.go:582] This route agent is running on the active gateway node
  6676. 2020-11-02T12:06:57.5893328Z [cluster2] I1102 12:00:22.464824 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  6677. 2020-11-02T12:06:57.5894322Z [cluster2] I1102 12:00:22.464991 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  6678. 2020-11-02T12:06:57.5895373Z [cluster2] I1102 12:00:22.465029 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.11 00:00:00:00:00:00
  6679. 2020-11-02T12:06:57.5896307Z [cluster2] I1102 12:00:22.465060 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.4 00:00:00:00:00:00
  6680. 2020-11-02T12:06:57.5897371Z [cluster2] I1102 12:00:22.465090 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.8 00:00:00:00:00:00
  6681. 2020-11-02T12:06:57.5898266Z [cluster2] I1102 12:00:22.465175 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  6682. 2020-11-02T12:06:57.5899643Z [cluster2] I1102 12:00:52.464263 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  6683. 2020-11-02T12:06:57.5901013Z [cluster2] I1102 12:00:52.464716 1 route.go:756] Processing route {Ifindex: 16 Dst: 240.0.0.0/8 Src: 240.17.0.4 Gw: <nil> Flags: [] Table: 254}
  6684. 2020-11-02T12:06:57.5901485Z [cluster2] I1102 12:00:52.464756 1 route.go:759] Found nil gw or dst
  6685. 2020-11-02T12:06:57.5901937Z [cluster2] I1102 12:00:52.464764 1 route.go:582] This route agent is running on the active gateway node
  6686. 2020-11-02T12:06:57.5902750Z [cluster2] I1102 12:00:52.464771 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  6687. 2020-11-02T12:06:57.5903371Z [cluster2] I1102 12:00:52.464940 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  6688. 2020-11-02T12:06:57.5903937Z [cluster2] I1102 12:00:52.464982 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.4 00:00:00:00:00:00
  6689. 2020-11-02T12:06:57.5904464Z [cluster2] I1102 12:00:52.465009 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.8 00:00:00:00:00:00
  6690. 2020-11-02T12:06:57.5904966Z [cluster2] I1102 12:00:52.465032 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.11 00:00:00:00:00:00
  6691. 2020-11-02T12:06:57.5905767Z [cluster2] I1102 12:00:52.465093 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  6692. 2020-11-02T12:06:57.5907209Z [cluster2] I1102 12:00:52.465150 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  6693. 2020-11-02T12:06:57.5909166Z [cluster2] I1102 12:00:52.465164 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  6694. 2020-11-02T12:06:57.5911052Z [cluster2] I1102 12:01:22.464359 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  6695. 2020-11-02T12:06:57.5912557Z [cluster2] I1102 12:01:22.464741 1 route.go:756] Processing route {Ifindex: 16 Dst: 240.0.0.0/8 Src: 240.17.0.4 Gw: <nil> Flags: [] Table: 254}
  6696. 2020-11-02T12:06:57.5913035Z [cluster2] I1102 12:01:22.464777 1 route.go:759] Found nil gw or dst
  6697. 2020-11-02T12:06:57.5913490Z [cluster2] I1102 12:01:22.464786 1 route.go:582] This route agent is running on the active gateway node
  6698. 2020-11-02T12:06:57.5914339Z [cluster2] I1102 12:01:22.464793 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  6699. 2020-11-02T12:06:57.5914964Z [cluster2] I1102 12:01:22.464940 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  6700. 2020-11-02T12:06:57.5915531Z [cluster2] I1102 12:01:22.464989 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.4 00:00:00:00:00:00
  6701. 2020-11-02T12:06:57.5916180Z [cluster2] I1102 12:01:22.465032 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.8 00:00:00:00:00:00
  6702. 2020-11-02T12:06:57.5916713Z [cluster2] I1102 12:01:22.465077 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.11 00:00:00:00:00:00
  6703. 2020-11-02T12:06:57.5917531Z [cluster2] I1102 12:01:22.465141 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  6704. 2020-11-02T12:06:57.5918930Z [cluster2] I1102 12:01:22.465271 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  6705. 2020-11-02T12:06:57.5920835Z [cluster2] I1102 12:01:22.465285 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  6706. 2020-11-02T12:06:57.5925756Z [cluster2] I1102 12:01:40.270990 1 route.go:682] Informed of removed endpoint: {"metadata":{"name":"cluster2-submariner-cable-cluster2-172-17-0-4","namespace":"submariner-operator","selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4","uid":"9b8708cb-6f04-49db-ab5a-2c50a1a2b639","resourceVersion":"2844","generation":1,"creationTimestamp":"2020-11-02T11:58:19Z"},"spec":{"cluster_id":"cluster2","cable_name":"submariner-cable-cluster2-172-17-0-4","hostname":"cluster2-worker","subnets":["100.2.0.0/16","10.2.0.0/16"],"private_ip":"172.17.0.4","public_ip":"","nat_enabled":false,"backend":"libreswan"}}
  6707. 2020-11-02T12:06:57.5929603Z [cluster2] I1102 12:01:40.271024 1 route.go:701] Local Gateway Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with IP 172.17.0.4 was removed: deleting vxlan interface
  6708. 2020-11-02T12:06:57.5930670Z [cluster2] I1102 12:01:40.271215 1 route.go:756] Processing route {Ifindex: 16 Dst: 240.0.0.0/8 Src: 240.17.0.4 Gw: <nil> Flags: [] Table: 254}
  6709. 2020-11-02T12:06:57.5931506Z [cluster2] I1102 12:01:40.271252 1 route.go:759] Found nil gw or dst
  6710. 2020-11-02T12:06:57.5932967Z [cluster2] I1102 12:01:40.321349 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  6711. 2020-11-02T12:06:57.5934267Z [cluster2] I1102 12:01:40.324505 1 route.go:756] Processing route {Ifindex: 8 Dst: <nil> Src: <nil> Gw: 172.17.0.1 Flags: [] Table: 254}
  6712. 2020-11-02T12:06:57.5934790Z [cluster2] I1102 12:01:40.324548 1 route.go:759] Found nil gw or dst
  6713. 2020-11-02T12:06:57.5935309Z [cluster2] I1102 12:01:40.324552 1 route.go:756] Processing route {Ifindex: 4 Dst: 10.2.0.0/16 Src: 10.2.0.1 Gw: <nil> Flags: [] Table: 254}
  6714. 2020-11-02T12:06:57.5936132Z [cluster2] I1102 12:01:40.324561 1 route.go:759] Found nil gw or dst
  6715. 2020-11-02T12:06:57.5936615Z [cluster2] I1102 12:01:40.324564 1 route.go:756] Processing route {Ifindex: 8 Dst: 172.17.0.0/16 Src: 172.17.0.4 Gw: <nil> Flags: [] Table: 254}
  6716. 2020-11-02T12:06:57.5937098Z [cluster2] I1102 12:01:40.324570 1 route.go:759] Found nil gw or dst
  6717. 2020-11-02T12:06:57.5937701Z [cluster2] I1102 12:01:40.324576 1 route.go:582] This route agent is running on the active gateway node
  6718. 2020-11-02T12:06:57.5938550Z [cluster2] I1102 12:01:40.324582 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  6719. 2020-11-02T12:06:57.5939157Z [cluster2] I1102 12:01:40.325990 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.4 00:00:00:00:00:00
  6720. 2020-11-02T12:06:57.5939851Z [cluster2] I1102 12:01:40.326029 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.8 00:00:00:00:00:00
  6721. 2020-11-02T12:06:57.5940357Z [cluster2] I1102 12:01:40.326054 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.11 00:00:00:00:00:00
  6722. 2020-11-02T12:06:57.5941304Z [cluster2] I1102 12:01:40.326105 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  6723. 2020-11-02T12:06:57.5942621Z [cluster2] I1102 12:01:52.464558 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  6724. 2020-11-02T12:06:57.5944361Z [cluster2] I1102 12:01:52.464663 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  6725. 2020-11-02T12:06:57.5945685Z [cluster2] I1102 12:01:52.464890 1 route.go:756] Processing route {Ifindex: 27 Dst: 240.0.0.0/8 Src: 240.17.0.4 Gw: <nil> Flags: [] Table: 254}
  6726. 2020-11-02T12:06:57.5946182Z [cluster2] I1102 12:01:52.464936 1 route.go:759] Found nil gw or dst
  6727. 2020-11-02T12:06:57.5946642Z [cluster2] I1102 12:01:52.464943 1 route.go:582] This route agent is running on the active gateway node
  6728. 2020-11-02T12:06:57.5947479Z [cluster2] I1102 12:01:52.464951 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  6729. 2020-11-02T12:06:57.5948124Z [cluster2] I1102 12:01:52.465127 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  6730. 2020-11-02T12:06:57.5948713Z [cluster2] I1102 12:01:52.465178 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.8 00:00:00:00:00:00
  6731. 2020-11-02T12:06:57.5949253Z [cluster2] I1102 12:01:52.465207 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.11 00:00:00:00:00:00
  6732. 2020-11-02T12:06:57.5949945Z [cluster2] I1102 12:01:52.465230 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.4 00:00:00:00:00:00
  6733. 2020-11-02T12:06:57.5950749Z [cluster2] I1102 12:01:52.465284 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  6734. 2020-11-02T12:06:57.5952059Z [cluster2] I1102 12:01:52.465374 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  6735. 2020-11-02T12:06:57.5953786Z [cluster2] I1102 12:02:22.464633 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  6736. 2020-11-02T12:06:57.5954841Z [cluster2] I1102 12:02:22.464838 1 route.go:756] Processing route {Ifindex: 27 Dst: 240.0.0.0/8 Src: 240.17.0.4 Gw: <nil> Flags: [] Table: 254}
  6737. 2020-11-02T12:06:57.5955312Z [cluster2] I1102 12:02:22.464870 1 route.go:759] Found nil gw or dst
  6738. 2020-11-02T12:06:57.5955766Z [cluster2] I1102 12:02:22.464878 1 route.go:582] This route agent is running on the active gateway node
  6739. 2020-11-02T12:06:57.5956546Z [cluster2] I1102 12:02:22.464885 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  6740. 2020-11-02T12:06:57.5957166Z [cluster2] I1102 12:02:22.465029 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  6741. 2020-11-02T12:06:57.5957733Z [cluster2] I1102 12:02:22.465092 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.11 00:00:00:00:00:00
  6742. 2020-11-02T12:06:57.5958425Z [cluster2] I1102 12:02:22.465133 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.4 00:00:00:00:00:00
  6743. 2020-11-02T12:06:57.5959039Z [cluster2] I1102 12:02:22.465167 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.8 00:00:00:00:00:00
  6744. 2020-11-02T12:06:57.5959880Z [cluster2] I1102 12:02:22.465240 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  6745. 2020-11-02T12:06:57.5977675Z [cluster2] I1102 12:02:22.465310 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  6746. 2020-11-02T12:06:57.5980180Z [cluster2] I1102 12:02:22.465333 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  6747. 2020-11-02T12:06:57.5981989Z [cluster2] I1102 12:02:52.464840 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  6748. 2020-11-02T12:06:57.5983270Z [cluster2] I1102 12:02:52.465140 1 route.go:756] Processing route {Ifindex: 27 Dst: 240.0.0.0/8 Src: 240.17.0.4 Gw: <nil> Flags: [] Table: 254}
  6749. 2020-11-02T12:06:57.5983956Z [cluster2] I1102 12:02:52.465192 1 route.go:759] Found nil gw or dst
  6750. 2020-11-02T12:06:57.5984733Z [cluster2] I1102 12:02:52.465200 1 route.go:582] This route agent is running on the active gateway node
  6751. 2020-11-02T12:06:57.5985767Z [cluster2] I1102 12:02:52.465207 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  6752. 2020-11-02T12:06:57.5986453Z [cluster2] I1102 12:02:52.465351 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  6753. 2020-11-02T12:06:57.5987360Z [cluster2] I1102 12:02:52.465389 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.4 00:00:00:00:00:00
  6754. 2020-11-02T12:06:57.5987888Z [cluster2] I1102 12:02:52.465418 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.8 00:00:00:00:00:00
  6755. 2020-11-02T12:06:57.5988416Z [cluster2] I1102 12:02:52.465443 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.11 00:00:00:00:00:00
  6756. 2020-11-02T12:06:57.5989253Z [cluster2] I1102 12:02:52.465513 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  6757. 2020-11-02T12:06:57.5990576Z [cluster2] I1102 12:02:52.465579 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  6758. 2020-11-02T12:06:57.5992282Z [cluster2] I1102 12:02:52.465595 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  6759. 2020-11-02T12:06:57.5994418Z [cluster2] I1102 12:03:22.464975 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  6760. 2020-11-02T12:06:57.5997736Z [cluster2] I1102 12:03:22.465203 1 route.go:756] Processing route {Ifindex: 27 Dst: 240.0.0.0/8 Src: 240.17.0.4 Gw: <nil> Flags: [] Table: 254}
  6761. 2020-11-02T12:06:57.5998257Z [cluster2] I1102 12:03:22.465236 1 route.go:759] Found nil gw or dst
  6762. 2020-11-02T12:06:57.5998706Z [cluster2] I1102 12:03:22.465244 1 route.go:582] This route agent is running on the active gateway node
  6763. 2020-11-02T12:06:57.5999614Z [cluster2] I1102 12:03:22.465252 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  6764. 2020-11-02T12:06:57.6000401Z [cluster2] I1102 12:03:22.465382 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  6765. 2020-11-02T12:06:57.6000989Z [cluster2] I1102 12:03:22.465430 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.4 00:00:00:00:00:00
  6766. 2020-11-02T12:06:57.6001528Z [cluster2] I1102 12:03:22.465484 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.8 00:00:00:00:00:00
  6767. 2020-11-02T12:06:57.6002069Z [cluster2] I1102 12:03:22.465534 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.11 00:00:00:00:00:00
  6768. 2020-11-02T12:06:57.6003267Z [cluster2] I1102 12:03:22.465595 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  6769. 2020-11-02T12:06:57.6005429Z [cluster2] I1102 12:03:22.465666 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  6770. 2020-11-02T12:06:57.6007385Z [cluster2] I1102 12:03:22.465690 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  6771. 2020-11-02T12:06:57.6009111Z [cluster2] I1102 12:03:52.465125 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  6772. 2020-11-02T12:06:57.6010151Z [cluster2] I1102 12:03:52.465340 1 route.go:756] Processing route {Ifindex: 27 Dst: 240.0.0.0/8 Src: 240.17.0.4 Gw: <nil> Flags: [] Table: 254}
  6773. 2020-11-02T12:06:57.6010625Z [cluster2] I1102 12:03:52.465364 1 route.go:759] Found nil gw or dst
  6774. 2020-11-02T12:06:57.6011074Z [cluster2] I1102 12:03:52.465370 1 route.go:582] This route agent is running on the active gateway node
  6775. 2020-11-02T12:06:57.6011952Z [cluster2] I1102 12:03:52.465376 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  6776. 2020-11-02T12:06:57.6012589Z [cluster2] I1102 12:03:52.465544 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  6777. 2020-11-02T12:06:57.6013338Z [cluster2] I1102 12:03:52.465582 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.8 00:00:00:00:00:00
  6778. 2020-11-02T12:06:57.6014084Z [cluster2] I1102 12:03:52.465610 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.11 00:00:00:00:00:00
  6779. 2020-11-02T12:06:57.6014620Z [cluster2] I1102 12:03:52.465651 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.4 00:00:00:00:00:00
  6780. 2020-11-02T12:06:57.6015502Z [cluster2] I1102 12:03:52.465710 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  6781. 2020-11-02T12:06:57.6017237Z [cluster2] I1102 12:03:52.465770 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  6782. 2020-11-02T12:06:57.6019003Z [cluster2] I1102 12:03:52.465784 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  6783. 2020-11-02T12:06:57.6020892Z [cluster2] I1102 12:04:22.465295 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  6784. 2020-11-02T12:06:57.6022611Z [cluster2] I1102 12:04:22.465334 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  6785. 2020-11-02T12:06:57.6024063Z [cluster2] I1102 12:04:22.465512 1 route.go:756] Processing route {Ifindex: 27 Dst: 240.0.0.0/8 Src: 240.17.0.4 Gw: <nil> Flags: [] Table: 254}
  6786. 2020-11-02T12:06:57.6024563Z [cluster2] I1102 12:04:22.465550 1 route.go:759] Found nil gw or dst
  6787. 2020-11-02T12:06:57.6033293Z [cluster2] I1102 12:04:22.465558 1 route.go:582] This route agent is running on the active gateway node
  6788. 2020-11-02T12:06:57.6042096Z [cluster2] I1102 12:04:22.465564 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  6789. 2020-11-02T12:06:57.6042797Z [cluster2] I1102 12:04:22.465708 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  6790. 2020-11-02T12:06:57.6043759Z [cluster2] I1102 12:04:22.465752 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.4 00:00:00:00:00:00
  6791. 2020-11-02T12:06:57.6044499Z [cluster2] I1102 12:04:22.465788 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.8 00:00:00:00:00:00
  6792. 2020-11-02T12:06:57.6045276Z [cluster2] I1102 12:04:22.465819 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.11 00:00:00:00:00:00
  6793. 2020-11-02T12:06:57.6046304Z [cluster2] I1102 12:04:22.465868 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  6794. 2020-11-02T12:06:57.6049165Z [cluster2] I1102 12:04:22.465931 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  6795. 2020-11-02T12:06:57.6051461Z [cluster2] I1102 12:04:52.465465 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  6796. 2020-11-02T12:06:57.6052750Z [cluster2] I1102 12:04:52.465773 1 route.go:756] Processing route {Ifindex: 27 Dst: 240.0.0.0/8 Src: 240.17.0.4 Gw: <nil> Flags: [] Table: 254}
  6797. 2020-11-02T12:06:57.6053326Z [cluster2] I1102 12:04:52.465803 1 route.go:759] Found nil gw or dst
  6798. 2020-11-02T12:06:57.6053867Z [cluster2] I1102 12:04:52.465811 1 route.go:582] This route agent is running on the active gateway node
  6799. 2020-11-02T12:06:57.6054832Z [cluster2] I1102 12:04:52.465824 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  6800. 2020-11-02T12:06:57.6055751Z [cluster2] I1102 12:04:52.465946 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  6801. 2020-11-02T12:06:57.6056943Z [cluster2] I1102 12:04:52.466013 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.11 00:00:00:00:00:00
  6802. 2020-11-02T12:06:57.6057847Z [cluster2] I1102 12:04:52.466082 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.4 00:00:00:00:00:00
  6803. 2020-11-02T12:06:57.6058380Z [cluster2] I1102 12:04:52.466114 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.8 00:00:00:00:00:00
  6804. 2020-11-02T12:06:57.6059422Z [cluster2] I1102 12:04:52.466169 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  6805. 2020-11-02T12:06:57.6066249Z [cluster2] I1102 12:04:52.466239 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  6806. 2020-11-02T12:06:57.6068636Z [cluster2] I1102 12:04:52.466296 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  6807. 2020-11-02T12:06:57.6080771Z [cluster2] I1102 12:05:22.465452 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  6808. 2020-11-02T12:06:57.6081840Z [cluster2] I1102 12:05:22.465672 1 route.go:756] Processing route {Ifindex: 27 Dst: 240.0.0.0/8 Src: 240.17.0.4 Gw: <nil> Flags: [] Table: 254}
  6809. 2020-11-02T12:06:57.6082313Z [cluster2] I1102 12:05:22.465703 1 route.go:759] Found nil gw or dst
  6810. 2020-11-02T12:06:57.6082960Z [cluster2] I1102 12:05:22.465710 1 route.go:582] This route agent is running on the active gateway node
  6811. 2020-11-02T12:06:57.6084370Z [cluster2] I1102 12:05:22.465717 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  6812. 2020-11-02T12:06:57.6085065Z [cluster2] I1102 12:05:22.465852 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  6813. 2020-11-02T12:06:57.6085846Z [cluster2] I1102 12:05:22.465895 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.4 00:00:00:00:00:00
  6814. 2020-11-02T12:06:57.6086596Z [cluster2] I1102 12:05:22.465930 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.8 00:00:00:00:00:00
  6815. 2020-11-02T12:06:57.6087551Z [cluster2] I1102 12:05:22.465962 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.11 00:00:00:00:00:00
  6816. 2020-11-02T12:06:57.6088658Z [cluster2] I1102 12:05:22.466046 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  6817. 2020-11-02T12:06:57.6090709Z [cluster2] I1102 12:05:22.466197 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  6818. 2020-11-02T12:06:57.6093231Z [cluster2] I1102 12:05:22.466223 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  6819. 2020-11-02T12:06:57.6098222Z [cluster2] I1102 12:05:30.586692 1 route.go:682] Informed of removed endpoint: {"metadata":{"name":"cluster2-submariner-cable-cluster2-172-17-0-4","namespace":"submariner-operator","selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4","uid":"8e4ce5be-6521-4f69-bef3-cde34ed638a9","resourceVersion":"3728","generation":1,"creationTimestamp":"2020-11-02T12:01:40Z"},"spec":{"cluster_id":"cluster2","cable_name":"submariner-cable-cluster2-172-17-0-4","hostname":"cluster2-worker","subnets":["100.2.0.0/16","10.2.0.0/16"],"private_ip":"172.17.0.4","public_ip":"","nat_enabled":false,"backend":"libreswan"}}
  6820. 2020-11-02T12:06:57.6102556Z [cluster2] I1102 12:05:30.586720 1 route.go:701] Local Gateway Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with IP 172.17.0.4 was removed: deleting vxlan interface
  6821. 2020-11-02T12:06:57.6104091Z [cluster2] I1102 12:05:30.586945 1 route.go:756] Processing route {Ifindex: 27 Dst: 240.0.0.0/8 Src: 240.17.0.4 Gw: <nil> Flags: [] Table: 254}
  6822. 2020-11-02T12:06:57.6104804Z [cluster2] I1102 12:05:30.586973 1 route.go:759] Found nil gw or dst
  6823. 2020-11-02T12:06:57.6107087Z [cluster2] I1102 12:05:30.690399 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-11 with IP 172.17.0.11, Host cluster2-worker2 for this cluster
  6824. 2020-11-02T12:06:57.6108689Z [cluster2] I1102 12:05:30.690428 1 route.go:612] This route agent is running on a non-gateway/non-active node
  6825. 2020-11-02T12:06:57.6109604Z [cluster2] I1102 12:05:30.690449 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.11
  6826. 2020-11-02T12:06:57.6110253Z [cluster2] I1102 12:05:30.702299 1 xfrm_cleanup_handler.go:32] Cleaning up 62 XFRM policies
  6827. 2020-11-02T12:06:57.6111211Z [cluster2] I1102 12:05:30.702331 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 172.17.0.6/32, Src: 100.2.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir out, Priority: 2088894, Index: 1057, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.6, Src: 172.17.0.4, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4011}]}
  6828. 2020-11-02T12:06:57.6112803Z [cluster2] I1102 12:05:30.702513 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.2.0.0/16, Src: 172.17.0.6/32, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir fwd, Priority: 2088894, Index: 2490, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.6, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4011}]}
  6829. 2020-11-02T12:06:57.6114649Z [cluster2] I1102 12:05:30.702603 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.2.0.0/16, Src: 172.17.0.6/32, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir in, Priority: 2088894, Index: 2480, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.6, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4011}]}
  6830. 2020-11-02T12:06:57.6116130Z [cluster2] I1102 12:05:30.702662 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.3.0.0/16, Src: 172.17.0.4/32, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir out, Priority: 2080734, Index: 345, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.7, Src: 172.17.0.4, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x402d}]}
  6831. 2020-11-02T12:06:57.6117570Z [cluster2] I1102 12:05:30.702704 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 172.17.0.4/32, Src: 100.3.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir fwd, Priority: 2080734, Index: 2458, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.7, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x402d}]}
  6832. 2020-11-02T12:06:57.6118915Z [cluster2] I1102 12:05:30.702756 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 172.17.0.4/32, Src: 100.3.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir in, Priority: 2080734, Index: 2448, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.7, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x402d}]}
  6833. 2020-11-02T12:06:57.6120135Z [cluster2] I1102 12:05:30.702797 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 172.17.0.6/32, Src: 10.2.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir out, Priority: 2088894, Index: 1097, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.6, Src: 172.17.0.4, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x401d}]}
  6834. 2020-11-02T12:06:57.6121476Z [cluster2] I1102 12:05:30.702851 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 10.2.0.0/16, Src: 172.17.0.6/32, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir fwd, Priority: 2088894, Index: 2426, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.6, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x401d}]}
  6835. 2020-11-02T12:06:57.6122737Z [cluster2] I1102 12:05:30.702888 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 10.2.0.0/16, Src: 172.17.0.6/32, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir in, Priority: 2088894, Index: 2416, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.6, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x401d}]}
  6836. 2020-11-02T12:06:57.6124743Z [cluster2] I1102 12:05:30.702943 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.3.0.0/16, Src: 100.2.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir out, Priority: 2088926, Index: 505, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.7, Src: 172.17.0.4, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4039}]}
  6837. 2020-11-02T12:06:57.6126078Z [cluster2] I1102 12:05:30.702994 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.2.0.0/16, Src: 100.3.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir fwd, Priority: 2088926, Index: 2394, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.7, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4039}]}
  6838. 2020-11-02T12:06:57.6127651Z [cluster2] I1102 12:05:30.703033 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.2.0.0/16, Src: 100.3.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir in, Priority: 2088926, Index: 2384, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.7, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4039}]}
  6839. 2020-11-02T12:06:57.6128824Z [cluster2] I1102 12:05:30.703086 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.1.0.0/16, Src: 10.2.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir out, Priority: 2088926, Index: 1105, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.6, Src: 172.17.0.4, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4021}]}
  6840. 2020-11-02T12:06:57.6130008Z [cluster2] I1102 12:05:30.703125 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 10.2.0.0/16, Src: 100.1.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir fwd, Priority: 2088926, Index: 1442, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.6, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4021}]}
  6841. 2020-11-02T12:06:57.6150240Z [cluster2] I1102 12:05:30.703178 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 10.2.0.0/16, Src: 100.1.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir in, Priority: 2088926, Index: 1432, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.6, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4021}]}
  6842. 2020-11-02T12:06:57.6151484Z [cluster2] I1102 12:05:30.703215 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.3.0.0/16, Src: 10.2.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir out, Priority: 2088926, Index: 561, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.7, Src: 172.17.0.4, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4045}]}
  6843. 2020-11-02T12:06:57.6152893Z [cluster2] I1102 12:05:30.703289 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 10.2.0.0/16, Src: 100.3.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir fwd, Priority: 2088926, Index: 2330, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.7, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4045}]}
  6844. 2020-11-02T12:06:57.6154316Z [cluster2] I1102 12:05:30.703345 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 10.2.0.0/16, Src: 100.3.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir in, Priority: 2088926, Index: 2320, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.7, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4045}]}
  6845. 2020-11-02T12:06:57.6155578Z [cluster2] I1102 12:05:30.703454 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 10.1.0.0/16, Src: 172.17.0.4/32, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir out, Priority: 2080734, Index: 961, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.6, Src: 172.17.0.4, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x400d}]}
  6846. 2020-11-02T12:06:57.6157431Z [cluster2] I1102 12:05:30.703553 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 172.17.0.4/32, Src: 10.1.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir fwd, Priority: 2080734, Index: 2314, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.6, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x400d}]}
  6847. 2020-11-02T12:06:57.6158661Z [cluster2] I1102 12:05:30.704562 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 172.17.0.4/32, Src: 10.1.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir in, Priority: 2080734, Index: 2296, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.6, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x400d}]}
  6848. 2020-11-02T12:06:57.6159879Z [cluster2] I1102 12:05:30.704647 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 10.3.0.0/16, Src: 172.17.0.4/32, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir out, Priority: 2080734, Index: 441, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.7, Src: 172.17.0.4, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4031}]}
  6849. 2020-11-02T12:06:57.6161250Z [cluster2] I1102 12:05:30.704746 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 172.17.0.4/32, Src: 10.3.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir fwd, Priority: 2080734, Index: 2282, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.7, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4031}]}
  6850. 2020-11-02T12:06:57.6162426Z [cluster2] I1102 12:05:30.704830 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 172.17.0.4/32, Src: 10.3.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir in, Priority: 2080734, Index: 2272, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.7, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4031}]}
  6851. 2020-11-02T12:06:57.6164268Z [cluster2] I1102 12:05:30.704890 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 172.17.0.7/32, Src: 172.17.0.4/32, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir out, Priority: 2080702, Index: 329, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.7, Src: 172.17.0.4, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4029}]}
  6852. 2020-11-02T12:06:57.6165542Z [cluster2] I1102 12:05:30.704950 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.1.0.0/16, Src: 172.17.0.4/32, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir out, Priority: 2080734, Index: 897, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.6, Src: 172.17.0.4, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4009}]}
  6853. 2020-11-02T12:06:57.6166964Z [cluster2] I1102 12:05:30.704992 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 172.17.0.4/32, Src: 100.1.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir fwd, Priority: 2080734, Index: 2250, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.6, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4009}]}
  6854. 2020-11-02T12:06:57.6207504Z [cluster2] I1102 12:05:30.705048 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 172.17.0.4/32, Src: 100.1.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir in, Priority: 2080734, Index: 2240, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.6, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4009}]}
  6855. 2020-11-02T12:06:57.6209478Z [cluster2] I1102 12:05:30.705092 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 172.17.0.6/32, Src: 172.17.0.4/32, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir out, Priority: 2080702, Index: 809, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.6, Src: 172.17.0.4, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4005}]}
  6856. 2020-11-02T12:06:57.6210908Z [cluster2] I1102 12:05:30.705202 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 172.17.0.7/32, Src: 100.2.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir out, Priority: 2088894, Index: 497, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.7, Src: 172.17.0.4, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4035}]}
  6857. 2020-11-02T12:06:57.6212159Z [cluster2] I1102 12:05:30.705259 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.2.0.0/16, Src: 172.17.0.7/32, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir fwd, Priority: 2088894, Index: 2218, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.7, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4035}]}
  6858. 2020-11-02T12:06:57.6214016Z [cluster2] I1102 12:05:30.705358 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.2.0.0/16, Src: 172.17.0.7/32, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir in, Priority: 2088894, Index: 2208, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.7, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4035}]}
  6859. 2020-11-02T12:06:57.6215425Z [cluster2] I1102 12:05:30.705398 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.1.0.0/16, Src: 100.2.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir out, Priority: 2088926, Index: 1065, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.6, Src: 172.17.0.4, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4015}]}
  6860. 2020-11-02T12:06:57.6217512Z [cluster2] I1102 12:05:30.705434 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.2.0.0/16, Src: 100.1.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir fwd, Priority: 2088926, Index: 2186, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.6, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4015}]}
  6861. 2020-11-02T12:06:57.6218809Z [cluster2] I1102 12:05:30.705497 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.2.0.0/16, Src: 100.1.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir in, Priority: 2088926, Index: 2176, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.6, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4015}]}
  6862. 2020-11-02T12:06:57.6220350Z [cluster2] I1102 12:05:30.705552 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 172.17.0.7/32, Src: 10.2.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir out, Priority: 2088894, Index: 537, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.7, Src: 172.17.0.4, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4041}]}
  6863. 2020-11-02T12:06:57.6221607Z [cluster2] I1102 12:05:30.705589 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 10.2.0.0/16, Src: 172.17.0.7/32, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir fwd, Priority: 2088894, Index: 2154, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.7, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4041}]}
  6864. 2020-11-02T12:06:57.6223269Z [cluster2] I1102 12:05:30.705700 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 10.2.0.0/16, Src: 172.17.0.7/32, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir in, Priority: 2088894, Index: 2144, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.7, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4041}]}
  6865. 2020-11-02T12:06:57.6224512Z [cluster2] I1102 12:05:30.705741 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 10.1.0.0/16, Src: 100.2.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir out, Priority: 2088926, Index: 1073, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.6, Src: 172.17.0.4, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4019}]}
  6866. 2020-11-02T12:06:57.6226706Z [cluster2] I1102 12:05:30.705775 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.2.0.0/16, Src: 10.1.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir fwd, Priority: 2088926, Index: 2122, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.6, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4019}]}
  6867. 2020-11-02T12:06:57.6228580Z [cluster2] I1102 12:05:30.705829 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.2.0.0/16, Src: 10.1.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir in, Priority: 2088926, Index: 2112, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.6, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4019}]}
  6868. 2020-11-02T12:06:57.6230020Z [cluster2] I1102 12:05:30.705864 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 172.17.0.4/32, Src: 172.17.0.7/32, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir fwd, Priority: 2080702, Index: 2090, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.7, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4029}]}
  6869. 2020-11-02T12:06:57.6231287Z [cluster2] I1102 12:05:30.705899 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 172.17.0.4/32, Src: 172.17.0.7/32, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir in, Priority: 2080702, Index: 2080, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.7, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4029}]}
  6870. 2020-11-02T12:06:57.6232573Z [cluster2] I1102 12:05:30.705932 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 10.3.0.0/16, Src: 100.2.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir out, Priority: 2088926, Index: 513, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.7, Src: 172.17.0.4, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x403d}]}
  6871. 2020-11-02T12:06:57.6233951Z [cluster2] I1102 12:05:30.705965 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.2.0.0/16, Src: 10.3.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir fwd, Priority: 2088926, Index: 2074, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.7, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x403d}]}
  6872. 2020-11-02T12:06:57.6235222Z [cluster2] I1102 12:05:30.706000 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.2.0.0/16, Src: 10.3.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir in, Priority: 2088926, Index: 2064, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.7, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x403d}]}
  6873. 2020-11-02T12:06:57.6236674Z [cluster2] I1102 12:05:30.706033 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 10.1.0.0/16, Src: 10.2.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir out, Priority: 2088926, Index: 1113, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.6, Src: 172.17.0.4, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4025}]}
  6874. 2020-11-02T12:06:57.6238647Z [cluster2] I1102 12:05:30.706461 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 10.2.0.0/16, Src: 10.1.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir fwd, Priority: 2088926, Index: 2026, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.6, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4025}]}
  6875. 2020-11-02T12:06:57.6240154Z [cluster2] I1102 12:05:30.706651 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 10.2.0.0/16, Src: 10.1.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir in, Priority: 2088926, Index: 2016, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.6, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4025}]}
  6876. 2020-11-02T12:06:57.6241547Z [cluster2] I1102 12:05:30.706812 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 172.17.0.4/32, Src: 172.17.0.6/32, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir fwd, Priority: 2080702, Index: 2010, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.6, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4005}]}
  6877. 2020-11-02T12:06:57.6242961Z [cluster2] I1102 12:05:30.708477 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 172.17.0.4/32, Src: 172.17.0.6/32, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir in, Priority: 2080702, Index: 2000, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.6, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4005}]}
  6878. 2020-11-02T12:06:57.6265121Z [cluster2] I1102 12:05:30.708671 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 10.3.0.0/16, Src: 10.2.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir out, Priority: 2088926, Index: 569, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.7, Src: 172.17.0.4, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4049}]}
  6879. 2020-11-02T12:06:57.6268110Z [cluster2] I1102 12:05:30.708847 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 10.2.0.0/16, Src: 10.3.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir fwd, Priority: 2088926, Index: 746, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.7, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4049}]}
  6880. 2020-11-02T12:06:57.6269938Z [cluster2] I1102 12:05:30.711465 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 10.2.0.0/16, Src: 10.3.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir in, Priority: 2088926, Index: 736, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.7, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4049}]}
  6881. 2020-11-02T12:06:57.6271583Z [cluster2] I1102 12:05:30.711528 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.2.0.0/16, Src: 10.3.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir fwd, Priority: 2088926, Index: 650, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.7, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x403d}]}
  6882. 2020-11-02T12:06:57.6272970Z [cluster2] I1102 12:05:30.711567 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.2.0.0/16, Src: 10.3.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir in, Priority: 2088926, Index: 640, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.7, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x403d}]}
  6883. 2020-11-02T12:06:57.6274246Z [cluster2] I1102 12:05:30.711604 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.2.0.0/16, Src: 100.3.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir fwd, Priority: 2088926, Index: 618, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.7, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4039}]}
  6884. 2020-11-02T12:06:57.6275886Z [cluster2] I1102 12:05:30.711649 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.2.0.0/16, Src: 100.3.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir in, Priority: 2088926, Index: 608, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.7, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4039}]}
  6885. 2020-11-02T12:06:57.6277366Z [cluster2] I1102 12:05:30.711682 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.2.0.0/16, Src: 10.1.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir fwd, Priority: 2088926, Index: 1306, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.6, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4019}]}
  6886. 2020-11-02T12:06:57.6279194Z [cluster2] I1102 12:05:30.711718 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.2.0.0/16, Src: 10.1.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir in, Priority: 2088926, Index: 1296, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.6, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4019}]}
  6887. 2020-11-02T12:06:57.6280451Z [cluster2] I1102 12:05:30.711752 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.2.0.0/16, Src: 100.1.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir fwd, Priority: 2088926, Index: 1242, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.6, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4015}]}
  6888. 2020-11-02T12:06:57.6281709Z [cluster2] I1102 12:05:30.711784 1 xfrm_cleanup_handler.go:36] Deleting XFRM policy {Dst: 100.2.0.0/16, Src: 100.1.0.0/16, Proto: 0, DstPort: 0, SrcPort: 0, Dir: dir in, Priority: 2088926, Index: 1232, Action: allow, Ifindex: 0, Ifid: 0, Mark: <nil>, Tmpls: [{Dst: 172.17.0.4, Src: 172.17.0.6, Proto: esp, Mode: tunnel, Spi: 0x0, Reqid: 0x4015}]}
  6889. 2020-11-02T12:06:57.6282648Z [cluster2] I1102 12:05:30.711822 1 route.go:771] Reconciling routes to gw: 240.17.0.11
  6890. 2020-11-02T12:06:57.6283203Z [cluster2] I1102 12:05:30.711916 1 route.go:787] Processing route {Ifindex: 28 Dst: 240.0.0.0/8 Src: 240.17.0.4 Gw: <nil> Flags: [] Table: 254}
  6891. 2020-11-02T12:06:57.6283842Z [cluster2] I1102 12:05:30.711930 1 route.go:790] Found nil gw or dst
  6892. 2020-11-02T12:06:57.6285775Z [cluster2] I1102 12:05:30.712086 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11"
  6893. 2020-11-02T12:06:57.6287980Z [cluster2] I1102 12:05:52.465829 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  6894. 2020-11-02T12:06:57.6290331Z [cluster2] I1102 12:05:52.465903 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-11 with IP 172.17.0.11, Host cluster2-worker2 for this cluster
  6895. 2020-11-02T12:06:57.6291742Z [cluster2] I1102 12:05:52.465914 1 route.go:612] This route agent is running on a non-gateway/non-active node
  6896. 2020-11-02T12:06:57.6292660Z [cluster2] I1102 12:05:52.465924 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.11
  6897. 2020-11-02T12:06:57.6293339Z [cluster2] I1102 12:05:52.466218 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  6898. 2020-11-02T12:06:57.6294042Z [cluster2] I1102 12:05:52.469921 1 route.go:771] Reconciling routes to gw: 240.17.0.11
  6899. 2020-11-02T12:06:57.6294608Z [cluster2] I1102 12:05:52.470078 1 route.go:787] Processing route {Ifindex: 28 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  6900. 2020-11-02T12:06:57.6295292Z [cluster2] I1102 12:05:52.470113 1 route.go:793] Found route {Ifindex: 28 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  6901. 2020-11-02T12:06:57.6296200Z [cluster2] I1102 12:05:52.470124 1 route.go:787] Processing route {Ifindex: 28 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  6902. 2020-11-02T12:06:57.6297077Z [cluster2] I1102 12:05:52.470133 1 route.go:793] Found route {Ifindex: 28 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  6903. 2020-11-02T12:06:57.6297797Z [cluster2] I1102 12:05:52.470141 1 route.go:787] Processing route {Ifindex: 28 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  6904. 2020-11-02T12:06:57.6298518Z [cluster2] I1102 12:05:52.470149 1 route.go:793] Found route {Ifindex: 28 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  6905. 2020-11-02T12:06:57.6299505Z [cluster2] I1102 12:05:52.470158 1 route.go:787] Processing route {Ifindex: 28 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  6906. 2020-11-02T12:06:57.6300174Z [cluster2] I1102 12:05:52.470173 1 route.go:793] Found route {Ifindex: 28 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  6907. 2020-11-02T12:06:57.6300843Z [cluster2] I1102 12:05:52.470181 1 route.go:787] Processing route {Ifindex: 28 Dst: 240.0.0.0/8 Src: 240.17.0.4 Gw: <nil> Flags: [] Table: 254}
  6908. 2020-11-02T12:06:57.6301355Z [cluster2] I1102 12:05:52.470189 1 route.go:790] Found nil gw or dst
  6909. 2020-11-02T12:06:57.6301818Z [cluster2] I1102 12:05:52.470252 1 route.go:829] Found equivalent route, not adding
  6910. 2020-11-02T12:06:57.6302338Z [cluster2] I1102 12:05:52.470262 1 route.go:829] Found equivalent route, not adding
  6911. 2020-11-02T12:06:57.6302856Z [cluster2] I1102 12:05:52.470273 1 route.go:829] Found equivalent route, not adding
  6912. 2020-11-02T12:06:57.6303370Z [cluster2] I1102 12:05:52.470281 1 route.go:829] Found equivalent route, not adding
  6913. 2020-11-02T12:06:57.6304689Z [cluster2] I1102 12:05:52.470314 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11"
  6914. 2020-11-02T12:06:57.6306849Z [cluster2] I1102 12:05:52.470331 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  6915. 2020-11-02T12:06:57.6309009Z [cluster2] I1102 12:06:22.465813 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-11 with IP 172.17.0.11, Host cluster2-worker2 for this cluster
  6916. 2020-11-02T12:06:57.6310399Z [cluster2] I1102 12:06:22.465841 1 route.go:612] This route agent is running on a non-gateway/non-active node
  6917. 2020-11-02T12:06:57.6311307Z [cluster2] I1102 12:06:22.465851 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.11
  6918. 2020-11-02T12:06:57.6312124Z [cluster2] I1102 12:06:22.466072 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  6919. 2020-11-02T12:06:57.6312702Z [cluster2] I1102 12:06:22.470074 1 route.go:771] Reconciling routes to gw: 240.17.0.11
  6920. 2020-11-02T12:06:57.6313271Z [cluster2] I1102 12:06:22.470295 1 route.go:787] Processing route {Ifindex: 28 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  6921. 2020-11-02T12:06:57.6313946Z [cluster2] I1102 12:06:22.470328 1 route.go:793] Found route {Ifindex: 28 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  6922. 2020-11-02T12:06:57.6314616Z [cluster2] I1102 12:06:22.470340 1 route.go:787] Processing route {Ifindex: 28 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  6923. 2020-11-02T12:06:57.6315285Z [cluster2] I1102 12:06:22.470357 1 route.go:793] Found route {Ifindex: 28 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  6924. 2020-11-02T12:06:57.6316216Z [cluster2] I1102 12:06:22.470373 1 route.go:787] Processing route {Ifindex: 28 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  6925. 2020-11-02T12:06:57.6316910Z [cluster2] I1102 12:06:22.470383 1 route.go:793] Found route {Ifindex: 28 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  6926. 2020-11-02T12:06:57.6317601Z [cluster2] I1102 12:06:22.470397 1 route.go:787] Processing route {Ifindex: 28 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  6927. 2020-11-02T12:06:57.6318292Z [cluster2] I1102 12:06:22.470405 1 route.go:793] Found route {Ifindex: 28 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  6928. 2020-11-02T12:06:57.6319134Z [cluster2] I1102 12:06:22.470413 1 route.go:787] Processing route {Ifindex: 28 Dst: 240.0.0.0/8 Src: 240.17.0.4 Gw: <nil> Flags: [] Table: 254}
  6929. 2020-11-02T12:06:57.6319738Z [cluster2] I1102 12:06:22.470426 1 route.go:790] Found nil gw or dst
  6930. 2020-11-02T12:06:57.6320185Z [cluster2] I1102 12:06:22.470497 1 route.go:829] Found equivalent route, not adding
  6931. 2020-11-02T12:06:57.6320683Z [cluster2] I1102 12:06:22.470514 1 route.go:829] Found equivalent route, not adding
  6932. 2020-11-02T12:06:57.6321185Z [cluster2] I1102 12:06:22.470542 1 route.go:829] Found equivalent route, not adding
  6933. 2020-11-02T12:06:57.6321679Z [cluster2] I1102 12:06:22.470555 1 route.go:829] Found equivalent route, not adding
  6934. 2020-11-02T12:06:57.6322928Z [cluster2] I1102 12:06:22.470587 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11"
  6935. 2020-11-02T12:06:58.2318858Z [submariner]$ [cluster2] kubectl -n submariner-operator logs submariner-routeagent-ftrjn
  6936. 2020-11-02T12:06:58.2320485Z [cluster2] I1102 12:06:22.470611 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  6937. 2020-11-02T12:06:58.2322331Z [cluster2] I1102 12:06:22.470629 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  6938. 2020-11-02T12:06:58.2324870Z [cluster2] I1102 12:06:52.466069 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-11 with IP 172.17.0.11, Host cluster2-worker2 for this cluster
  6939. 2020-11-02T12:06:58.2326359Z [cluster2] I1102 12:06:52.466120 1 route.go:612] This route agent is running on a non-gateway/non-active node
  6940. 2020-11-02T12:06:58.2327271Z [cluster2] I1102 12:06:52.466131 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.11
  6941. 2020-11-02T12:06:58.2328304Z [cluster2] I1102 12:06:52.466346 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  6942. 2020-11-02T12:06:58.2328893Z [cluster2] I1102 12:06:52.470631 1 route.go:771] Reconciling routes to gw: 240.17.0.11
  6943. 2020-11-02T12:06:58.2329456Z [cluster2] I1102 12:06:52.470798 1 route.go:787] Processing route {Ifindex: 28 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  6944. 2020-11-02T12:06:58.2330136Z [cluster2] I1102 12:06:52.470820 1 route.go:793] Found route {Ifindex: 28 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  6945. 2020-11-02T12:06:58.2330809Z [cluster2] I1102 12:06:52.470828 1 route.go:787] Processing route {Ifindex: 28 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  6946. 2020-11-02T12:06:58.2331462Z [cluster2] I1102 12:06:52.470836 1 route.go:793] Found route {Ifindex: 28 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  6947. 2020-11-02T12:06:58.2332127Z [cluster2] I1102 12:06:52.470843 1 route.go:787] Processing route {Ifindex: 28 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  6948. 2020-11-02T12:06:58.2332869Z [cluster2] I1102 12:06:52.470851 1 route.go:793] Found route {Ifindex: 28 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  6949. 2020-11-02T12:06:58.2333542Z [cluster2] I1102 12:06:52.470857 1 route.go:787] Processing route {Ifindex: 28 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  6950. 2020-11-02T12:06:58.2334213Z [cluster2] I1102 12:06:52.470864 1 route.go:793] Found route {Ifindex: 28 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  6951. 2020-11-02T12:06:58.2334881Z [cluster2] I1102 12:06:52.470870 1 route.go:787] Processing route {Ifindex: 28 Dst: 240.0.0.0/8 Src: 240.17.0.4 Gw: <nil> Flags: [] Table: 254}
  6952. 2020-11-02T12:06:58.2335394Z [cluster2] I1102 12:06:52.470876 1 route.go:790] Found nil gw or dst
  6953. 2020-11-02T12:06:58.2335853Z [cluster2] I1102 12:06:52.470936 1 route.go:829] Found equivalent route, not adding
  6954. 2020-11-02T12:06:58.2336510Z [cluster2] I1102 12:06:52.470944 1 route.go:829] Found equivalent route, not adding
  6955. 2020-11-02T12:06:58.2337020Z [cluster2] I1102 12:06:52.470952 1 route.go:829] Found equivalent route, not adding
  6956. 2020-11-02T12:06:58.2337529Z [cluster2] I1102 12:06:52.470959 1 route.go:829] Found equivalent route, not adding
  6957. 2020-11-02T12:06:58.2338842Z [cluster2] I1102 12:06:52.471004 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11"
  6958. 2020-11-02T12:06:58.2340677Z [cluster2] I1102 12:06:52.471018 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  6959. 2020-11-02T12:06:58.2342546Z [cluster2] I1102 12:06:52.471030 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  6960. 2020-11-02T12:06:58.2344055Z [cluster2] +++++++++++++++++++++: Logs for Pod submariner-routeagent-ftrjn in namespace submariner-operator :++++++++++++++++++++++
  6961. 2020-11-02T12:06:58.2344873Z [cluster2] + trap 'exit 1' SIGTERM SIGINT
  6962. 2020-11-02T12:06:58.2345241Z [cluster2] + SUBMARINER_VERBOSITY=2
  6963. 2020-11-02T12:06:58.2345718Z [cluster2] + '[' false == true ']'
  6964. 2020-11-02T12:06:58.2346207Z [cluster2] + DEBUG=-v=2
  6965. 2020-11-02T12:06:58.2346742Z [cluster2] + for f in iptables-save iptables
  6966. 2020-11-02T12:06:58.2347319Z [cluster2] ++ find_iptables_on_host iptables-save
  6967. 2020-11-02T12:06:58.2348086Z [cluster2] ++ chroot /host test -x /usr/sbin/iptables-save
  6968. 2020-11-02T12:06:58.2348477Z [cluster2] ++ echo /usr/sbin
  6969. 2020-11-02T12:06:58.2348762Z [cluster2] ++ return
  6970. 2020-11-02T12:06:58.2349048Z [cluster2] + location=/usr/sbin
  6971. 2020-11-02T12:06:58.2349543Z [cluster2] + '[' /usr/sbin '!=' unknown ']'
  6972. 2020-11-02T12:06:58.2350204Z [cluster2] + echo 'iptables-save is present on the host at /usr/sbin/iptables-save'
  6973. 2020-11-02T12:06:58.2350979Z [cluster2] + sed 's!@@PATH@@!/usr/sbin!' /usr/sbin/iptables-wrapper.in
  6974. 2020-11-02T12:06:58.2351724Z [cluster2] iptables-save is present on the host at /usr/sbin/iptables-save
  6975. 2020-11-02T12:06:58.2352378Z [cluster2] + for f in iptables-save iptables
  6976. 2020-11-02T12:06:58.2352794Z [cluster2] ++ find_iptables_on_host iptables
  6977. 2020-11-02T12:06:58.2353493Z [cluster2] ++ chroot /host test -x /usr/sbin/iptables
  6978. 2020-11-02T12:06:58.2353835Z [cluster2] ++ echo /usr/sbin
  6979. 2020-11-02T12:06:58.2354119Z [cluster2] ++ return
  6980. 2020-11-02T12:06:58.2354420Z [cluster2] + location=/usr/sbin
  6981. 2020-11-02T12:06:58.2355058Z [cluster2] + '[' /usr/sbin '!=' unknown ']'
  6982. 2020-11-02T12:06:58.2355851Z [cluster2] + echo 'iptables is present on the host at /usr/sbin/iptables'
  6983. 2020-11-02T12:06:58.2356555Z [cluster2] + sed 's!@@PATH@@!/usr/sbin!' /usr/sbin/iptables-wrapper.in
  6984. 2020-11-02T12:06:58.2357092Z [cluster2] iptables is present on the host at /usr/sbin/iptables
  6985. 2020-11-02T12:06:58.2357941Z [cluster2] + exec submariner-route-agent -v=2 -alsologtostderr
  6986. 2020-11-02T12:06:58.2359003Z [cluster2] I1102 11:58:05.340609 1 main.go:47] Starting submariner-route-agent
  6987. 2020-11-02T12:06:58.2360002Z [cluster2] W1102 11:58:05.362118 1 client_config.go:543] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
  6988. 2020-11-02T12:06:58.2360959Z [cluster2] I1102 11:58:05.364260 1 route.go:192] Starting Route Controller. ClusterID: cluster2, localClusterCIDR: [10.2.0.0/16], localServiceCIDR: [100.2.0.0/16]
  6989. 2020-11-02T12:06:58.2361723Z [cluster2] I1102 11:58:05.364337 1 route.go:196] Waiting for Endpoint informer caches to sync.
  6990. 2020-11-02T12:06:58.2362501Z [cluster2] I1102 11:58:05.405590 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.11
  6991. 2020-11-02T12:06:58.2363302Z [cluster2] I1102 11:58:05.405725 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.4
  6992. 2020-11-02T12:06:58.2369818Z [cluster2] I1102 11:58:05.405799 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.8
  6993. 2020-11-02T12:06:58.2370680Z [cluster2] I1102 11:58:05.469252 1 driver.go:37] Interface "lo" has "127.0.0.1" address
  6994. 2020-11-02T12:06:58.2371159Z [cluster2] I1102 11:58:05.469523 1 driver.go:37] Interface "weave" has "10.2.224.0" address
  6995. 2020-11-02T12:06:58.2371733Z [cluster2] I1102 11:58:05.470078 1 driver.go:42] Found CNI Interface "weave" that has IP "10.2.224.0" from ClusterCIDR "10.2.0.0/16"
  6996. 2020-11-02T12:06:58.2372435Z [cluster2] I1102 11:58:05.470204 1 driver.go:60] Successfully configured rp_filter to loose mode(2) on cniInterface "weave"
  6997. 2020-11-02T12:06:58.2373512Z [cluster2] I1102 11:58:05.479394 1 iptables.go:22] Install/ensure SUBMARINER-POSTROUTING chain exists
  6998. 2020-11-02T12:06:58.2374556Z [cluster2] I1102 11:58:05.552233 1 iptables.go:28] Insert SUBMARINER-POSTROUTING rule that has rules for inter-cluster traffic
  6999. 2020-11-02T12:06:58.2375708Z [cluster2] I1102 11:58:05.561985 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.11
  7000. 2020-11-02T12:06:58.2376567Z [cluster2] I1102 11:58:05.570474 1 iptables.go:34] Install/ensure SUBMARINER-INPUT chain exists
  7001. 2020-11-02T12:06:58.2377463Z [cluster2] I1102 11:58:05.695595 1 iptables.go:45] Allow VxLAN incoming traffic in SUBMARINER-INPUT Chain
  7002. 2020-11-02T12:06:58.2378844Z [cluster2] I1102 11:58:05.707794 1 iptables.go:53] Insert rule to allow traffic over vx-submariner interface in FORWARDing Chain
  7003. 2020-11-02T12:06:58.2379948Z [cluster2] I1102 11:58:05.718788 1 iptables.go:65] Installing rule for host network to remote cluster communication: -s 240.0.0.0/8 -o vx-submariner -j SNAT --to 10.2.224.0
  7004. 2020-11-02T12:06:58.2380919Z [cluster2] I1102 11:58:07.152068 1 route.go:254] In cluster2, podIP of submariner-route-agent[0] is 172.17.0.4
  7005. 2020-11-02T12:06:58.2381792Z [cluster2] I1102 11:58:07.152327 1 route.go:254] In cluster2, podIP of submariner-route-agent[1] is 172.17.0.8
  7006. 2020-11-02T12:06:58.2382661Z [cluster2] I1102 11:58:07.161683 1 route.go:254] In cluster2, podIP of submariner-route-agent[2] is 172.17.0.11
  7007. 2020-11-02T12:06:58.2383757Z [cluster2] I1102 11:58:07.162458 1 route.go:268] Hostname is "cluster2-control-plane" and routeAgentNodeName is "cluster2-control-plane"
  7008. 2020-11-02T12:06:58.2385322Z [cluster2] I1102 11:58:07.420329 1 driver.go:87] Successfully annotated node "cluster2-control-plane" with cniIfaceIP "10.2.224.0"
  7009. 2020-11-02T12:06:58.2386011Z [cluster2] I1102 11:58:07.420489 1 route.go:286] Route agent workers started
  7010. 2020-11-02T12:06:58.2387096Z [cluster2] I1102 11:58:07.420560 1 route.go:507] Processing submariner-route-agent pod "submariner-routeagent-swbc2" with host IP "172.17.0.11", pod IP "172.17.0.11"
  7011. 2020-11-02T12:06:58.2388635Z [cluster2] I1102 11:58:07.420569 1 route.go:531] Successfully processed submariner-route-agent pod "submariner-routeagent-swbc2"
  7012. 2020-11-02T12:06:58.2389962Z [cluster2] I1102 11:58:07.420578 1 route.go:507] Processing submariner-route-agent pod "submariner-routeagent-8x2cc" with host IP "172.17.0.4", pod IP "172.17.0.4"
  7013. 2020-11-02T12:06:58.2391260Z [cluster2] I1102 11:58:07.420612 1 route.go:531] Successfully processed submariner-route-agent pod "submariner-routeagent-8x2cc"
  7014. 2020-11-02T12:06:58.2392913Z [cluster2] I1102 11:58:07.420619 1 route.go:507] Processing submariner-route-agent pod "submariner-routeagent-ftrjn" with host IP "172.17.0.8", pod IP "172.17.0.8"
  7015. 2020-11-02T12:06:58.2394268Z [cluster2] I1102 11:58:07.420624 1 route.go:531] Successfully processed submariner-route-agent pod "submariner-routeagent-ftrjn"
  7016. 2020-11-02T12:06:58.2396307Z [cluster2] I1102 11:58:19.435538 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7017. 2020-11-02T12:06:58.2397716Z [cluster2] I1102 11:58:19.439218 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7018. 2020-11-02T12:06:58.2398779Z [cluster2] I1102 11:58:19.439281 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7019. 2020-11-02T12:06:58.2399463Z [cluster2] I1102 11:58:19.663247 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7020. 2020-11-02T12:06:58.2400006Z [cluster2] I1102 11:58:19.663391 1 route.go:787] Processing route {Ifindex: 11 Dst: 240.0.0.0/8 Src: 240.17.0.8 Gw: <nil> Flags: [] Table: 254}
  7021. 2020-11-02T12:06:58.2401110Z [cluster2] I1102 11:58:19.663427 1 route.go:790] Found nil gw or dst
  7022. 2020-11-02T12:06:58.2402624Z [cluster2] I1102 11:58:19.663496 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7023. 2020-11-02T12:06:58.2413636Z [cluster2] I1102 11:58:24.199191 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7024. 2020-11-02T12:06:58.2415787Z [cluster2] I1102 11:58:24.227651 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.2.0.0/16 -d 100.3.0.0/16 -j ACCEPT
  7025. 2020-11-02T12:06:58.2417031Z [cluster2] I1102 11:58:24.252928 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 100.3.0.0/16 -d 10.2.0.0/16 -j ACCEPT
  7026. 2020-11-02T12:06:58.2418150Z [cluster2] I1102 11:58:24.333668 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.2.0.0/16 -d 10.3.0.0/16 -j ACCEPT
  7027. 2020-11-02T12:06:58.2419520Z [cluster2] I1102 11:58:24.342174 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 10.3.0.0/16 -d 10.2.0.0/16 -j ACCEPT
  7028. 2020-11-02T12:06:58.2420943Z [cluster2] I1102 11:58:27.574604 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7029. 2020-11-02T12:06:58.2432329Z [cluster2] I1102 11:58:27.623121 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.2.0.0/16 -d 100.1.0.0/16 -j ACCEPT
  7030. 2020-11-02T12:06:58.2433511Z [cluster2] I1102 11:58:27.713296 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 100.1.0.0/16 -d 10.2.0.0/16 -j ACCEPT
  7031. 2020-11-02T12:06:58.2434429Z [cluster2] I1102 11:58:27.750789 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.2.0.0/16 -d 10.1.0.0/16 -j ACCEPT
  7032. 2020-11-02T12:06:58.2435628Z [cluster2] I1102 11:58:27.772504 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 10.1.0.0/16 -d 10.2.0.0/16 -j ACCEPT
  7033. 2020-11-02T12:06:58.2437085Z [cluster2] I1102 11:58:35.393916 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7034. 2020-11-02T12:06:58.2438711Z [cluster2] I1102 11:58:35.394031 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7035. 2020-11-02T12:06:58.2439578Z [cluster2] I1102 11:58:35.394059 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7036. 2020-11-02T12:06:58.2440255Z [cluster2] I1102 11:58:35.394318 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7037. 2020-11-02T12:06:58.2440816Z [cluster2] I1102 11:58:35.517525 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7038. 2020-11-02T12:06:58.2441359Z [cluster2] I1102 11:58:35.517677 1 route.go:787] Processing route {Ifindex: 11 Dst: 240.0.0.0/8 Src: 240.17.0.8 Gw: <nil> Flags: [] Table: 254}
  7039. 2020-11-02T12:06:58.2441858Z [cluster2] I1102 11:58:35.517699 1 route.go:790] Found nil gw or dst
  7040. 2020-11-02T12:06:58.2442991Z [cluster2] I1102 11:58:35.522117 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7041. 2020-11-02T12:06:58.2462844Z [cluster2] I1102 11:58:35.522143 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7042. 2020-11-02T12:06:58.2464836Z [cluster2] I1102 11:58:35.522155 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7043. 2020-11-02T12:06:58.2466898Z [cluster2] I1102 11:59:05.394169 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7044. 2020-11-02T12:06:58.2468225Z [cluster2] I1102 11:59:05.394206 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7045. 2020-11-02T12:06:58.2469096Z [cluster2] I1102 11:59:05.394217 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7046. 2020-11-02T12:06:58.2469875Z [cluster2] I1102 11:59:05.394426 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7047. 2020-11-02T12:06:58.2470437Z [cluster2] I1102 11:59:05.400315 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7048. 2020-11-02T12:06:58.2470981Z [cluster2] I1102 11:59:05.400610 1 route.go:787] Processing route {Ifindex: 11 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7049. 2020-11-02T12:06:58.2471642Z [cluster2] I1102 11:59:05.400708 1 route.go:793] Found route {Ifindex: 11 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7050. 2020-11-02T12:06:58.2472391Z [cluster2] I1102 11:59:05.400794 1 route.go:787] Processing route {Ifindex: 11 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7051. 2020-11-02T12:06:58.2473036Z [cluster2] I1102 11:59:05.400850 1 route.go:793] Found route {Ifindex: 11 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7052. 2020-11-02T12:06:58.2473689Z [cluster2] I1102 11:59:05.400959 1 route.go:787] Processing route {Ifindex: 11 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7053. 2020-11-02T12:06:58.2474341Z [cluster2] I1102 11:59:05.400993 1 route.go:793] Found route {Ifindex: 11 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7054. 2020-11-02T12:06:58.2474995Z [cluster2] I1102 11:59:05.401063 1 route.go:787] Processing route {Ifindex: 11 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7055. 2020-11-02T12:06:58.2476028Z [cluster2] I1102 11:59:05.401115 1 route.go:793] Found route {Ifindex: 11 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7056. 2020-11-02T12:06:58.2476679Z [cluster2] I1102 11:59:05.401181 1 route.go:787] Processing route {Ifindex: 11 Dst: 240.0.0.0/8 Src: 240.17.0.8 Gw: <nil> Flags: [] Table: 254}
  7057. 2020-11-02T12:06:58.2477176Z [cluster2] I1102 11:59:05.401215 1 route.go:790] Found nil gw or dst
  7058. 2020-11-02T12:06:58.2477622Z [cluster2] I1102 11:59:05.401354 1 route.go:829] Found equivalent route, not adding
  7059. 2020-11-02T12:06:58.2478286Z [cluster2] I1102 11:59:05.401449 1 route.go:829] Found equivalent route, not adding
  7060. 2020-11-02T12:06:58.2478765Z [cluster2] I1102 11:59:05.401498 1 route.go:829] Found equivalent route, not adding
  7061. 2020-11-02T12:06:58.2479248Z [cluster2] I1102 11:59:05.401536 1 route.go:829] Found equivalent route, not adding
  7062. 2020-11-02T12:06:58.2480460Z [cluster2] I1102 11:59:05.401616 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7063. 2020-11-02T12:06:58.2482170Z [cluster2] I1102 11:59:05.401694 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7064. 2020-11-02T12:06:58.2484417Z [cluster2] I1102 11:59:05.401804 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7065. 2020-11-02T12:06:58.2486537Z [cluster2] I1102 11:59:35.394178 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7066. 2020-11-02T12:06:58.2487850Z [cluster2] I1102 11:59:35.394381 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7067. 2020-11-02T12:06:58.2489011Z [cluster2] I1102 11:59:35.394393 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7068. 2020-11-02T12:06:58.2489645Z [cluster2] I1102 11:59:35.395153 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7069. 2020-11-02T12:06:58.2490193Z [cluster2] I1102 11:59:35.411424 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7070. 2020-11-02T12:06:58.2490717Z [cluster2] I1102 11:59:35.411830 1 route.go:787] Processing route {Ifindex: 11 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7071. 2020-11-02T12:06:58.2491343Z [cluster2] I1102 11:59:35.411894 1 route.go:793] Found route {Ifindex: 11 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7072. 2020-11-02T12:06:58.2491975Z [cluster2] I1102 11:59:35.411920 1 route.go:787] Processing route {Ifindex: 11 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7073. 2020-11-02T12:06:58.2492605Z [cluster2] I1102 11:59:35.411943 1 route.go:793] Found route {Ifindex: 11 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7074. 2020-11-02T12:06:58.2493313Z [cluster2] I1102 11:59:35.411997 1 route.go:787] Processing route {Ifindex: 11 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7075. 2020-11-02T12:06:58.2493953Z [cluster2] I1102 11:59:35.412036 1 route.go:793] Found route {Ifindex: 11 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7076. 2020-11-02T12:06:58.2494993Z [cluster2] I1102 11:59:35.412075 1 route.go:787] Processing route {Ifindex: 11 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7077. 2020-11-02T12:06:58.2495845Z [cluster2] I1102 11:59:35.412119 1 route.go:793] Found route {Ifindex: 11 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7078. 2020-11-02T12:06:58.2496492Z [cluster2] I1102 11:59:35.412165 1 route.go:787] Processing route {Ifindex: 11 Dst: 240.0.0.0/8 Src: 240.17.0.8 Gw: <nil> Flags: [] Table: 254}
  7079. 2020-11-02T12:06:58.2497355Z [cluster2] I1102 11:59:35.412213 1 route.go:790] Found nil gw or dst
  7080. 2020-11-02T12:06:58.2507480Z [cluster2] I1102 11:59:35.412457 1 route.go:829] Found equivalent route, not adding
  7081. 2020-11-02T12:06:58.2508002Z [cluster2] I1102 11:59:35.412769 1 route.go:829] Found equivalent route, not adding
  7082. 2020-11-02T12:06:58.2508753Z [cluster2] I1102 11:59:35.412883 1 route.go:829] Found equivalent route, not adding
  7083. 2020-11-02T12:06:58.2509239Z [cluster2] I1102 11:59:35.413003 1 route.go:829] Found equivalent route, not adding
  7084. 2020-11-02T12:06:58.2510514Z [cluster2] I1102 11:59:35.413625 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7085. 2020-11-02T12:06:58.2512360Z [cluster2] I1102 11:59:35.414406 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7086. 2020-11-02T12:06:58.2514128Z [cluster2] I1102 11:59:35.415120 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7087. 2020-11-02T12:06:58.2516016Z [cluster2] I1102 12:00:05.394281 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7088. 2020-11-02T12:06:58.2518007Z [cluster2] I1102 12:00:05.394553 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7089. 2020-11-02T12:06:58.2519414Z [cluster2] I1102 12:00:05.394703 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7090. 2020-11-02T12:06:58.2520576Z [cluster2] I1102 12:00:05.394810 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7091. 2020-11-02T12:06:58.2521376Z [cluster2] I1102 12:00:05.395347 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7092. 2020-11-02T12:06:58.2521900Z [cluster2] I1102 12:00:05.399242 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7093. 2020-11-02T12:06:58.2522414Z [cluster2] I1102 12:00:05.399404 1 route.go:787] Processing route {Ifindex: 11 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7094. 2020-11-02T12:06:58.2523024Z [cluster2] I1102 12:00:05.399438 1 route.go:793] Found route {Ifindex: 11 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7095. 2020-11-02T12:06:58.2523633Z [cluster2] I1102 12:00:05.399449 1 route.go:787] Processing route {Ifindex: 11 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7096. 2020-11-02T12:06:58.2524356Z [cluster2] I1102 12:00:05.399457 1 route.go:793] Found route {Ifindex: 11 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7097. 2020-11-02T12:06:58.2524972Z [cluster2] I1102 12:00:05.399465 1 route.go:787] Processing route {Ifindex: 11 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7098. 2020-11-02T12:06:58.2525662Z [cluster2] I1102 12:00:05.399473 1 route.go:793] Found route {Ifindex: 11 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7099. 2020-11-02T12:06:58.2526282Z [cluster2] I1102 12:00:05.399486 1 route.go:787] Processing route {Ifindex: 11 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7100. 2020-11-02T12:06:58.2542589Z [cluster2] I1102 12:00:05.399495 1 route.go:793] Found route {Ifindex: 11 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7101. 2020-11-02T12:06:58.2543231Z [cluster2] I1102 12:00:05.399509 1 route.go:787] Processing route {Ifindex: 11 Dst: 240.0.0.0/8 Src: 240.17.0.8 Gw: <nil> Flags: [] Table: 254}
  7102. 2020-11-02T12:06:58.2543709Z [cluster2] I1102 12:00:05.399517 1 route.go:790] Found nil gw or dst
  7103. 2020-11-02T12:06:58.2544126Z [cluster2] I1102 12:00:05.399580 1 route.go:829] Found equivalent route, not adding
  7104. 2020-11-02T12:06:58.2544581Z [cluster2] I1102 12:00:05.399598 1 route.go:829] Found equivalent route, not adding
  7105. 2020-11-02T12:06:58.2545053Z [cluster2] I1102 12:00:05.399608 1 route.go:829] Found equivalent route, not adding
  7106. 2020-11-02T12:06:58.2545523Z [cluster2] I1102 12:00:05.399620 1 route.go:829] Found equivalent route, not adding
  7107. 2020-11-02T12:06:58.2546959Z [cluster2] I1102 12:00:05.399653 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7108. 2020-11-02T12:06:58.2548934Z [cluster2] I1102 12:00:05.399679 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7109. 2020-11-02T12:06:58.2551444Z [cluster2] I1102 12:00:35.444461 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7110. 2020-11-02T12:06:58.2553467Z [cluster2] I1102 12:00:35.444896 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7111. 2020-11-02T12:06:58.2554777Z [cluster2] I1102 12:00:35.444934 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7112. 2020-11-02T12:06:58.2555642Z [cluster2] I1102 12:00:35.445152 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7113. 2020-11-02T12:06:58.2556311Z [cluster2] I1102 12:00:35.446400 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7114. 2020-11-02T12:06:58.2557360Z [cluster2] I1102 12:00:35.471333 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7115. 2020-11-02T12:06:58.2558337Z [cluster2] I1102 12:00:35.472482 1 route.go:787] Processing route {Ifindex: 11 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7116. 2020-11-02T12:06:58.2558990Z [cluster2] I1102 12:00:35.483485 1 route.go:793] Found route {Ifindex: 11 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7117. 2020-11-02T12:06:58.2559643Z [cluster2] I1102 12:00:35.483670 1 route.go:787] Processing route {Ifindex: 11 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7118. 2020-11-02T12:06:58.2560299Z [cluster2] I1102 12:00:35.483727 1 route.go:793] Found route {Ifindex: 11 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7119. 2020-11-02T12:06:58.2561081Z [cluster2] I1102 12:00:35.483908 1 route.go:787] Processing route {Ifindex: 11 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7120. 2020-11-02T12:06:58.2561690Z [cluster2] I1102 12:00:35.483972 1 route.go:793] Found route {Ifindex: 11 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7121. 2020-11-02T12:06:58.2562303Z [cluster2] I1102 12:00:35.484015 1 route.go:787] Processing route {Ifindex: 11 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7122. 2020-11-02T12:06:58.2562992Z [cluster2] I1102 12:00:35.488812 1 route.go:793] Found route {Ifindex: 11 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7123. 2020-11-02T12:06:58.2563589Z [cluster2] I1102 12:00:35.488947 1 route.go:787] Processing route {Ifindex: 11 Dst: 240.0.0.0/8 Src: 240.17.0.8 Gw: <nil> Flags: [] Table: 254}
  7124. 2020-11-02T12:06:58.2585833Z [cluster2] I1102 12:00:35.489848 1 route.go:790] Found nil gw or dst
  7125. 2020-11-02T12:06:58.2586406Z [cluster2] I1102 12:00:35.493300 1 route.go:829] Found equivalent route, not adding
  7126. 2020-11-02T12:06:58.2587295Z [cluster2] I1102 12:00:35.493937 1 route.go:829] Found equivalent route, not adding
  7127. 2020-11-02T12:06:58.2587807Z [cluster2] I1102 12:00:35.494109 1 route.go:829] Found equivalent route, not adding
  7128. 2020-11-02T12:06:58.2588302Z [cluster2] I1102 12:00:35.494661 1 route.go:829] Found equivalent route, not adding
  7129. 2020-11-02T12:06:58.2589641Z [cluster2] I1102 12:00:35.495016 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7130. 2020-11-02T12:06:58.2591494Z [cluster2] I1102 12:00:35.495600 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7131. 2020-11-02T12:06:58.2593418Z [cluster2] I1102 12:01:05.425036 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7132. 2020-11-02T12:06:58.2594825Z [cluster2] I1102 12:01:05.425244 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7133. 2020-11-02T12:06:58.2595833Z [cluster2] I1102 12:01:05.425324 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7134. 2020-11-02T12:06:58.2596458Z [cluster2] I1102 12:01:05.427520 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7135. 2020-11-02T12:06:58.2597390Z [cluster2] I1102 12:01:05.431336 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7136. 2020-11-02T12:06:58.2597937Z [cluster2] I1102 12:01:05.431539 1 route.go:787] Processing route {Ifindex: 11 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7137. 2020-11-02T12:06:58.2598575Z [cluster2] I1102 12:01:05.431570 1 route.go:793] Found route {Ifindex: 11 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7138. 2020-11-02T12:06:58.2599222Z [cluster2] I1102 12:01:05.431580 1 route.go:787] Processing route {Ifindex: 11 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7139. 2020-11-02T12:06:58.2600032Z [cluster2] I1102 12:01:05.431605 1 route.go:793] Found route {Ifindex: 11 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7140. 2020-11-02T12:06:58.2600808Z [cluster2] I1102 12:01:05.431659 1 route.go:787] Processing route {Ifindex: 11 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7141. 2020-11-02T12:06:58.2601418Z [cluster2] I1102 12:01:05.431683 1 route.go:793] Found route {Ifindex: 11 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7142. 2020-11-02T12:06:58.2602024Z [cluster2] I1102 12:01:05.431702 1 route.go:787] Processing route {Ifindex: 11 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7143. 2020-11-02T12:06:58.2602931Z [cluster2] I1102 12:01:05.431743 1 route.go:793] Found route {Ifindex: 11 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7144. 2020-11-02T12:06:58.2603558Z [cluster2] I1102 12:01:05.431847 1 route.go:787] Processing route {Ifindex: 11 Dst: 240.0.0.0/8 Src: 240.17.0.8 Gw: <nil> Flags: [] Table: 254}
  7145. 2020-11-02T12:06:58.2604122Z [cluster2] I1102 12:01:05.432063 1 route.go:790] Found nil gw or dst
  7146. 2020-11-02T12:06:58.2604551Z [cluster2] I1102 12:01:05.432242 1 route.go:829] Found equivalent route, not adding
  7147. 2020-11-02T12:06:58.2605114Z [cluster2] I1102 12:01:05.432266 1 route.go:829] Found equivalent route, not adding
  7148. 2020-11-02T12:06:58.2605750Z [cluster2] I1102 12:01:05.432276 1 route.go:829] Found equivalent route, not adding
  7149. 2020-11-02T12:06:58.2606210Z [cluster2] I1102 12:01:05.432284 1 route.go:829] Found equivalent route, not adding
  7150. 2020-11-02T12:06:58.2607791Z [cluster2] I1102 12:01:05.432548 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7151. 2020-11-02T12:06:58.2609541Z [cluster2] I1102 12:01:05.432579 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7152. 2020-11-02T12:06:58.2613199Z [cluster2] I1102 12:01:05.432773 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7153. 2020-11-02T12:06:58.2615490Z [cluster2] I1102 12:01:35.425240 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7154. 2020-11-02T12:06:58.2617661Z [cluster2] I1102 12:01:35.425324 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7155. 2020-11-02T12:06:58.2618972Z [cluster2] I1102 12:01:35.425333 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7156. 2020-11-02T12:06:58.2619820Z [cluster2] I1102 12:01:35.425341 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7157. 2020-11-02T12:06:58.2620642Z [cluster2] I1102 12:01:35.426142 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7158. 2020-11-02T12:06:58.2621166Z [cluster2] I1102 12:01:35.431650 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7159. 2020-11-02T12:06:58.2621678Z [cluster2] I1102 12:01:35.432005 1 route.go:787] Processing route {Ifindex: 11 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7160. 2020-11-02T12:06:58.2622289Z [cluster2] I1102 12:01:35.433304 1 route.go:793] Found route {Ifindex: 11 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7161. 2020-11-02T12:06:58.2639469Z [cluster2] I1102 12:01:35.433442 1 route.go:787] Processing route {Ifindex: 11 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7162. 2020-11-02T12:06:58.2640567Z [cluster2] I1102 12:01:35.433473 1 route.go:793] Found route {Ifindex: 11 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7163. 2020-11-02T12:06:58.2641344Z [cluster2] I1102 12:01:35.433483 1 route.go:787] Processing route {Ifindex: 11 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7164. 2020-11-02T12:06:58.2642021Z [cluster2] I1102 12:01:35.433614 1 route.go:793] Found route {Ifindex: 11 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7165. 2020-11-02T12:06:58.2642637Z [cluster2] I1102 12:01:35.433621 1 route.go:787] Processing route {Ifindex: 11 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7166. 2020-11-02T12:06:58.2643243Z [cluster2] I1102 12:01:35.433628 1 route.go:793] Found route {Ifindex: 11 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7167. 2020-11-02T12:06:58.2644635Z [cluster2] I1102 12:01:35.433635 1 route.go:787] Processing route {Ifindex: 11 Dst: 240.0.0.0/8 Src: 240.17.0.8 Gw: <nil> Flags: [] Table: 254}
  7168. 2020-11-02T12:06:58.2645256Z [cluster2] I1102 12:01:35.433670 1 route.go:790] Found nil gw or dst
  7169. 2020-11-02T12:06:58.2645677Z [cluster2] I1102 12:01:35.433866 1 route.go:829] Found equivalent route, not adding
  7170. 2020-11-02T12:06:58.2646149Z [cluster2] I1102 12:01:35.434008 1 route.go:829] Found equivalent route, not adding
  7171. 2020-11-02T12:06:58.2646902Z [cluster2] I1102 12:01:35.434069 1 route.go:829] Found equivalent route, not adding
  7172. 2020-11-02T12:06:58.2647644Z [cluster2] I1102 12:01:35.434092 1 route.go:829] Found equivalent route, not adding
  7173. 2020-11-02T12:06:58.2648963Z [cluster2] I1102 12:01:35.434457 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7174. 2020-11-02T12:06:58.2651016Z [cluster2] I1102 12:01:35.434677 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7175. 2020-11-02T12:06:58.2655285Z [cluster2] I1102 12:01:40.266809 1 route.go:682] Informed of removed endpoint: {"metadata":{"name":"cluster2-submariner-cable-cluster2-172-17-0-4","namespace":"submariner-operator","selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4","uid":"9b8708cb-6f04-49db-ab5a-2c50a1a2b639","resourceVersion":"2844","generation":1,"creationTimestamp":"2020-11-02T11:58:19Z"},"spec":{"cluster_id":"cluster2","cable_name":"submariner-cable-cluster2-172-17-0-4","hostname":"cluster2-worker","subnets":["100.2.0.0/16","10.2.0.0/16"],"private_ip":"172.17.0.4","public_ip":"","nat_enabled":false,"backend":"libreswan"}}
  7176. 2020-11-02T12:06:58.2660782Z [cluster2] I1102 12:01:40.266831 1 route.go:711] Local non-gateway Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with IP "172.17.0.4" was removed: deleting vxlan interface
  7177. 2020-11-02T12:06:58.2662896Z [cluster2] I1102 12:01:40.322152 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7178. 2020-11-02T12:06:58.2664153Z [cluster2] I1102 12:01:40.322185 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7179. 2020-11-02T12:06:58.2664965Z [cluster2] I1102 12:01:40.322195 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7180. 2020-11-02T12:06:58.2665512Z [cluster2] I1102 12:01:40.342126 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7181. 2020-11-02T12:06:58.2666024Z [cluster2] I1102 12:01:40.348621 1 route.go:787] Processing route {Ifindex: 12 Dst: 240.0.0.0/8 Src: 240.17.0.8 Gw: <nil> Flags: [] Table: 254}
  7182. 2020-11-02T12:06:58.2666492Z [cluster2] I1102 12:01:40.354355 1 route.go:790] Found nil gw or dst
  7183. 2020-11-02T12:06:58.2667949Z [cluster2] I1102 12:01:40.358952 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7184. 2020-11-02T12:06:58.2669714Z [cluster2] I1102 12:02:05.425982 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7185. 2020-11-02T12:06:58.2671306Z [cluster2] I1102 12:02:05.426211 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7186. 2020-11-02T12:06:58.2672118Z [cluster2] I1102 12:02:05.426315 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7187. 2020-11-02T12:06:58.2672752Z [cluster2] I1102 12:02:05.426623 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7188. 2020-11-02T12:06:58.2673276Z [cluster2] I1102 12:02:05.433381 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7189. 2020-11-02T12:06:58.2673781Z [cluster2] I1102 12:02:05.433627 1 route.go:787] Processing route {Ifindex: 12 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7190. 2020-11-02T12:06:58.2674389Z [cluster2] I1102 12:02:05.433664 1 route.go:793] Found route {Ifindex: 12 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7191. 2020-11-02T12:06:58.2675001Z [cluster2] I1102 12:02:05.433674 1 route.go:787] Processing route {Ifindex: 12 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7192. 2020-11-02T12:06:58.2675612Z [cluster2] I1102 12:02:05.433780 1 route.go:793] Found route {Ifindex: 12 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7193. 2020-11-02T12:06:58.2676293Z [cluster2] I1102 12:02:05.433803 1 route.go:787] Processing route {Ifindex: 12 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7194. 2020-11-02T12:06:58.2677551Z [cluster2] I1102 12:02:05.433813 1 route.go:793] Found route {Ifindex: 12 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7195. 2020-11-02T12:06:58.2679136Z [cluster2] I1102 12:02:05.434008 1 route.go:787] Processing route {Ifindex: 12 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7196. 2020-11-02T12:06:58.2685834Z [cluster2] I1102 12:02:05.434032 1 route.go:793] Found route {Ifindex: 12 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7197. 2020-11-02T12:06:58.2691674Z [cluster2] I1102 12:02:05.434041 1 route.go:787] Processing route {Ifindex: 12 Dst: 240.0.0.0/8 Src: 240.17.0.8 Gw: <nil> Flags: [] Table: 254}
  7198. 2020-11-02T12:06:58.2692180Z [cluster2] I1102 12:02:05.434049 1 route.go:790] Found nil gw or dst
  7199. 2020-11-02T12:06:58.2692612Z [cluster2] I1102 12:02:05.434144 1 route.go:829] Found equivalent route, not adding
  7200. 2020-11-02T12:06:58.2693099Z [cluster2] I1102 12:02:05.434163 1 route.go:829] Found equivalent route, not adding
  7201. 2020-11-02T12:06:58.2693565Z [cluster2] I1102 12:02:05.434187 1 route.go:829] Found equivalent route, not adding
  7202. 2020-11-02T12:06:58.2694228Z [cluster2] I1102 12:02:05.434195 1 route.go:829] Found equivalent route, not adding
  7203. 2020-11-02T12:06:58.2696086Z [cluster2] I1102 12:02:05.434309 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7204. 2020-11-02T12:06:58.2699234Z [cluster2] I1102 12:02:05.434336 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7205. 2020-11-02T12:06:58.2701552Z [cluster2] I1102 12:02:05.434416 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7206. 2020-11-02T12:06:58.2703642Z [cluster2] I1102 12:02:35.426076 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7207. 2020-11-02T12:06:58.2705055Z [cluster2] I1102 12:02:35.429706 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7208. 2020-11-02T12:06:58.2707003Z [cluster2] I1102 12:02:35.430831 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7209. 2020-11-02T12:06:58.2708497Z [cluster2] I1102 12:02:35.432047 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7210. 2020-11-02T12:06:58.2709118Z [cluster2] I1102 12:02:35.435401 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7211. 2020-11-02T12:06:58.2709725Z [cluster2] I1102 12:02:35.435533 1 route.go:787] Processing route {Ifindex: 12 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7212. 2020-11-02T12:06:58.2710756Z [cluster2] I1102 12:02:35.435563 1 route.go:793] Found route {Ifindex: 12 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7213. 2020-11-02T12:06:58.2711428Z [cluster2] I1102 12:02:35.435572 1 route.go:787] Processing route {Ifindex: 12 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7214. 2020-11-02T12:06:58.2712333Z [cluster2] I1102 12:02:35.435579 1 route.go:793] Found route {Ifindex: 12 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7215. 2020-11-02T12:06:58.2712990Z [cluster2] I1102 12:02:35.435586 1 route.go:787] Processing route {Ifindex: 12 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7216. 2020-11-02T12:06:58.2713710Z [cluster2] I1102 12:02:35.435622 1 route.go:793] Found route {Ifindex: 12 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7217. 2020-11-02T12:06:58.2714369Z [cluster2] I1102 12:02:35.435644 1 route.go:787] Processing route {Ifindex: 12 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7218. 2020-11-02T12:06:58.2715063Z [cluster2] I1102 12:02:35.435664 1 route.go:793] Found route {Ifindex: 12 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7219. 2020-11-02T12:06:58.2715707Z [cluster2] I1102 12:02:35.435682 1 route.go:787] Processing route {Ifindex: 12 Dst: 240.0.0.0/8 Src: 240.17.0.8 Gw: <nil> Flags: [] Table: 254}
  7220. 2020-11-02T12:06:58.2716204Z [cluster2] I1102 12:02:35.435714 1 route.go:790] Found nil gw or dst
  7221. 2020-11-02T12:06:58.2716829Z [cluster2] I1102 12:02:35.435871 1 route.go:829] Found equivalent route, not adding
  7222. 2020-11-02T12:06:58.2717513Z [cluster2] I1102 12:02:35.435890 1 route.go:829] Found equivalent route, not adding
  7223. 2020-11-02T12:06:58.2718118Z [cluster2] I1102 12:02:35.435899 1 route.go:829] Found equivalent route, not adding
  7224. 2020-11-02T12:06:58.2718652Z [cluster2] I1102 12:02:35.435907 1 route.go:829] Found equivalent route, not adding
  7225. 2020-11-02T12:06:58.2720038Z [cluster2] I1102 12:02:35.435981 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7226. 2020-11-02T12:06:58.2723217Z [cluster2] I1102 12:02:35.436005 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7227. 2020-11-02T12:06:58.2725433Z [cluster2] I1102 12:02:35.436017 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7228. 2020-11-02T12:06:58.2728591Z [cluster2] I1102 12:03:05.426216 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7229. 2020-11-02T12:06:58.2731462Z [cluster2] I1102 12:03:05.426346 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7230. 2020-11-02T12:06:58.2733303Z [cluster2] I1102 12:03:05.426380 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7231. 2020-11-02T12:06:58.2734614Z [cluster2] I1102 12:03:05.426425 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7232. 2020-11-02T12:06:58.2736017Z [cluster2] I1102 12:03:05.426452 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7233. 2020-11-02T12:06:58.2737215Z [cluster2] I1102 12:03:05.427162 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7234. 2020-11-02T12:06:58.2737816Z [cluster2] I1102 12:03:05.436061 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7235. 2020-11-02T12:06:58.2738401Z [cluster2] I1102 12:03:05.436599 1 route.go:787] Processing route {Ifindex: 12 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7236. 2020-11-02T12:06:58.2739098Z [cluster2] I1102 12:03:05.436635 1 route.go:793] Found route {Ifindex: 12 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7237. 2020-11-02T12:06:58.2740836Z [cluster2] I1102 12:03:05.436663 1 route.go:787] Processing route {Ifindex: 12 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7238. 2020-11-02T12:06:58.2749700Z [cluster2] I1102 12:03:05.436672 1 route.go:793] Found route {Ifindex: 12 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7239. 2020-11-02T12:06:58.2751131Z [cluster2] I1102 12:03:05.436697 1 route.go:787] Processing route {Ifindex: 12 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7240. 2020-11-02T12:06:58.2752094Z [cluster2] I1102 12:03:05.436909 1 route.go:793] Found route {Ifindex: 12 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7241. 2020-11-02T12:06:58.2752766Z [cluster2] I1102 12:03:05.437045 1 route.go:787] Processing route {Ifindex: 12 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7242. 2020-11-02T12:06:58.2753399Z [cluster2] I1102 12:03:05.437070 1 route.go:793] Found route {Ifindex: 12 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7243. 2020-11-02T12:06:58.2754198Z [cluster2] I1102 12:03:05.437081 1 route.go:787] Processing route {Ifindex: 12 Dst: 240.0.0.0/8 Src: 240.17.0.8 Gw: <nil> Flags: [] Table: 254}
  7244. 2020-11-02T12:06:58.2755157Z [cluster2] I1102 12:03:05.437088 1 route.go:790] Found nil gw or dst
  7245. 2020-11-02T12:06:58.2755734Z [cluster2] I1102 12:03:05.437382 1 route.go:829] Found equivalent route, not adding
  7246. 2020-11-02T12:06:58.2756210Z [cluster2] I1102 12:03:05.437402 1 route.go:829] Found equivalent route, not adding
  7247. 2020-11-02T12:06:58.2756863Z [cluster2] I1102 12:03:05.437411 1 route.go:829] Found equivalent route, not adding
  7248. 2020-11-02T12:06:58.2757534Z [cluster2] I1102 12:03:05.437419 1 route.go:829] Found equivalent route, not adding
  7249. 2020-11-02T12:06:58.2758860Z [cluster2] I1102 12:03:05.437576 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7250. 2020-11-02T12:06:58.2760757Z [cluster2] I1102 12:03:35.426312 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7251. 2020-11-02T12:06:58.2762460Z [cluster2] I1102 12:03:35.426454 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7252. 2020-11-02T12:06:58.2764363Z [cluster2] I1102 12:03:35.426486 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7253. 2020-11-02T12:06:58.2765646Z [cluster2] I1102 12:03:35.426504 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7254. 2020-11-02T12:06:58.2769266Z [cluster2] I1102 12:03:35.426564 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7255. 2020-11-02T12:06:58.2770054Z [cluster2] I1102 12:03:35.426843 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7256. 2020-11-02T12:06:58.2771281Z [cluster2] I1102 12:03:35.430394 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7257. 2020-11-02T12:06:58.2771934Z [cluster2] I1102 12:03:35.430560 1 route.go:787] Processing route {Ifindex: 12 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7258. 2020-11-02T12:06:58.2773856Z [cluster2] I1102 12:03:35.430621 1 route.go:793] Found route {Ifindex: 12 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7259. 2020-11-02T12:06:58.2774757Z [cluster2] I1102 12:03:35.430651 1 route.go:787] Processing route {Ifindex: 12 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7260. 2020-11-02T12:06:58.2776060Z [cluster2] I1102 12:03:35.430659 1 route.go:793] Found route {Ifindex: 12 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7261. 2020-11-02T12:06:58.2777767Z [cluster2] I1102 12:03:35.430712 1 route.go:787] Processing route {Ifindex: 12 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7262. 2020-11-02T12:06:58.2778643Z [cluster2] I1102 12:03:35.430740 1 route.go:793] Found route {Ifindex: 12 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7263. 2020-11-02T12:06:58.2779495Z [cluster2] I1102 12:03:35.430858 1 route.go:787] Processing route {Ifindex: 12 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7264. 2020-11-02T12:06:58.2781023Z [cluster2] I1102 12:03:35.430893 1 route.go:793] Found route {Ifindex: 12 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7265. 2020-11-02T12:06:58.2782659Z [cluster2] I1102 12:03:35.430971 1 route.go:787] Processing route {Ifindex: 12 Dst: 240.0.0.0/8 Src: 240.17.0.8 Gw: <nil> Flags: [] Table: 254}
  7266. 2020-11-02T12:06:58.2783275Z [cluster2] I1102 12:03:35.431032 1 route.go:790] Found nil gw or dst
  7267. 2020-11-02T12:06:58.2783804Z [cluster2] I1102 12:03:35.431589 1 route.go:829] Found equivalent route, not adding
  7268. 2020-11-02T12:06:58.2784762Z [cluster2] I1102 12:03:35.431607 1 route.go:829] Found equivalent route, not adding
  7269. 2020-11-02T12:06:58.2785508Z [cluster2] I1102 12:03:35.431617 1 route.go:829] Found equivalent route, not adding
  7270. 2020-11-02T12:06:58.2786255Z [cluster2] I1102 12:03:35.431624 1 route.go:829] Found equivalent route, not adding
  7271. 2020-11-02T12:06:58.2787776Z [cluster2] I1102 12:03:35.431849 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7272. 2020-11-02T12:06:58.2789460Z [cluster2] I1102 12:04:05.426530 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7273. 2020-11-02T12:06:58.2790693Z [cluster2] I1102 12:04:05.426602 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7274. 2020-11-02T12:06:58.2791488Z [cluster2] I1102 12:04:05.426625 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7275. 2020-11-02T12:06:58.2792118Z [cluster2] I1102 12:04:05.426940 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7276. 2020-11-02T12:06:58.2792647Z [cluster2] I1102 12:04:05.431341 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7277. 2020-11-02T12:06:58.2793157Z [cluster2] I1102 12:04:05.431655 1 route.go:787] Processing route {Ifindex: 12 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7278. 2020-11-02T12:06:58.2793773Z [cluster2] I1102 12:04:05.431687 1 route.go:793] Found route {Ifindex: 12 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7279. 2020-11-02T12:06:58.2794380Z [cluster2] I1102 12:04:05.431711 1 route.go:787] Processing route {Ifindex: 12 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7280. 2020-11-02T12:06:58.2794989Z [cluster2] I1102 12:04:05.431989 1 route.go:793] Found route {Ifindex: 12 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7281. 2020-11-02T12:06:58.2795598Z [cluster2] I1102 12:04:05.432013 1 route.go:787] Processing route {Ifindex: 12 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7282. 2020-11-02T12:06:58.2796338Z [cluster2] I1102 12:04:05.432022 1 route.go:793] Found route {Ifindex: 12 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7283. 2020-11-02T12:06:58.2796948Z [cluster2] I1102 12:04:05.432171 1 route.go:787] Processing route {Ifindex: 12 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7284. 2020-11-02T12:06:58.2797551Z [cluster2] I1102 12:04:05.432230 1 route.go:793] Found route {Ifindex: 12 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7285. 2020-11-02T12:06:58.2798156Z [cluster2] I1102 12:04:05.432252 1 route.go:787] Processing route {Ifindex: 12 Dst: 240.0.0.0/8 Src: 240.17.0.8 Gw: <nil> Flags: [] Table: 254}
  7286. 2020-11-02T12:06:58.2798616Z [cluster2] I1102 12:04:05.432259 1 route.go:790] Found nil gw or dst
  7287. 2020-11-02T12:06:58.2799036Z [cluster2] I1102 12:04:05.432619 1 route.go:829] Found equivalent route, not adding
  7288. 2020-11-02T12:06:58.2799506Z [cluster2] I1102 12:04:05.432643 1 route.go:829] Found equivalent route, not adding
  7289. 2020-11-02T12:06:58.2799969Z [cluster2] I1102 12:04:05.432653 1 route.go:829] Found equivalent route, not adding
  7290. 2020-11-02T12:06:58.2800491Z [cluster2] I1102 12:04:05.432662 1 route.go:829] Found equivalent route, not adding
  7291. 2020-11-02T12:06:58.2801641Z [cluster2] I1102 12:04:05.432900 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7292. 2020-11-02T12:06:58.2803286Z [cluster2] I1102 12:04:05.433036 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7293. 2020-11-02T12:06:58.2805134Z [cluster2] I1102 12:04:05.433062 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7294. 2020-11-02T12:06:58.2806835Z [cluster2] I1102 12:04:35.426689 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7295. 2020-11-02T12:06:58.2808531Z [cluster2] I1102 12:04:35.426765 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7296. 2020-11-02T12:06:58.2810644Z [cluster2] I1102 12:04:35.426774 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7297. 2020-11-02T12:06:58.2812165Z [cluster2] I1102 12:04:35.426815 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7298. 2020-11-02T12:06:58.2813057Z [cluster2] I1102 12:04:35.426891 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7299. 2020-11-02T12:06:58.2814093Z [cluster2] I1102 12:04:35.427194 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7300. 2020-11-02T12:06:58.2814631Z [cluster2] I1102 12:04:35.430484 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7301. 2020-11-02T12:06:58.2815829Z [cluster2] I1102 12:04:35.430710 1 route.go:787] Processing route {Ifindex: 12 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7302. 2020-11-02T12:06:58.2816781Z [cluster2] I1102 12:04:35.430743 1 route.go:793] Found route {Ifindex: 12 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7303. 2020-11-02T12:06:58.2817432Z [cluster2] I1102 12:04:35.430775 1 route.go:787] Processing route {Ifindex: 12 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7304. 2020-11-02T12:06:58.2818084Z [cluster2] I1102 12:04:35.430799 1 route.go:793] Found route {Ifindex: 12 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7305. 2020-11-02T12:06:58.2818886Z [cluster2] I1102 12:04:35.430806 1 route.go:787] Processing route {Ifindex: 12 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7306. 2020-11-02T12:06:58.2819531Z [cluster2] I1102 12:04:35.430814 1 route.go:793] Found route {Ifindex: 12 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7307. 2020-11-02T12:06:58.2820315Z [cluster2] I1102 12:04:35.430820 1 route.go:787] Processing route {Ifindex: 12 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7308. 2020-11-02T12:06:58.2820916Z [cluster2] I1102 12:04:35.430890 1 route.go:793] Found route {Ifindex: 12 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7309. 2020-11-02T12:06:58.2821516Z [cluster2] I1102 12:04:35.430910 1 route.go:787] Processing route {Ifindex: 12 Dst: 240.0.0.0/8 Src: 240.17.0.8 Gw: <nil> Flags: [] Table: 254}
  7310. 2020-11-02T12:06:58.2821978Z [cluster2] I1102 12:04:35.430918 1 route.go:790] Found nil gw or dst
  7311. 2020-11-02T12:06:58.2822383Z [cluster2] I1102 12:04:35.431036 1 route.go:829] Found equivalent route, not adding
  7312. 2020-11-02T12:06:58.2822850Z [cluster2] I1102 12:04:35.431054 1 route.go:829] Found equivalent route, not adding
  7313. 2020-11-02T12:06:58.2823314Z [cluster2] I1102 12:04:35.431064 1 route.go:829] Found equivalent route, not adding
  7314. 2020-11-02T12:06:58.2823843Z [cluster2] I1102 12:04:35.431107 1 route.go:829] Found equivalent route, not adding
  7315. 2020-11-02T12:06:58.2825053Z [cluster2] I1102 12:04:35.431196 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7316. 2020-11-02T12:06:58.2826717Z [cluster2] I1102 12:05:05.426735 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7317. 2020-11-02T12:06:58.2827945Z [cluster2] I1102 12:05:05.426781 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7318. 2020-11-02T12:06:58.2828761Z [cluster2] I1102 12:05:05.426792 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7319. 2020-11-02T12:06:58.2829388Z [cluster2] I1102 12:05:05.427013 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7320. 2020-11-02T12:06:58.2829918Z [cluster2] I1102 12:05:05.672550 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7321. 2020-11-02T12:06:58.2830423Z [cluster2] I1102 12:05:05.672945 1 route.go:787] Processing route {Ifindex: 12 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7322. 2020-11-02T12:06:58.2831033Z [cluster2] I1102 12:05:05.673062 1 route.go:793] Found route {Ifindex: 12 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7323. 2020-11-02T12:06:58.2831638Z [cluster2] I1102 12:05:05.673137 1 route.go:787] Processing route {Ifindex: 12 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7324. 2020-11-02T12:06:58.2832248Z [cluster2] I1102 12:05:05.673180 1 route.go:793] Found route {Ifindex: 12 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7325. 2020-11-02T12:06:58.2832855Z [cluster2] I1102 12:05:05.673275 1 route.go:787] Processing route {Ifindex: 12 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7326. 2020-11-02T12:06:58.2833464Z [cluster2] I1102 12:05:05.673322 1 route.go:793] Found route {Ifindex: 12 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7327. 2020-11-02T12:06:58.2834068Z [cluster2] I1102 12:05:05.673439 1 route.go:787] Processing route {Ifindex: 12 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7328. 2020-11-02T12:06:58.2834652Z [cluster2] I1102 12:05:05.673532 1 route.go:793] Found route {Ifindex: 12 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7329. 2020-11-02T12:06:58.2835340Z [cluster2] I1102 12:05:05.673606 1 route.go:787] Processing route {Ifindex: 12 Dst: 240.0.0.0/8 Src: 240.17.0.8 Gw: <nil> Flags: [] Table: 254}
  7330. 2020-11-02T12:06:58.2835803Z [cluster2] I1102 12:05:05.673693 1 route.go:790] Found nil gw or dst
  7331. 2020-11-02T12:06:58.2836222Z [cluster2] I1102 12:05:05.673830 1 route.go:829] Found equivalent route, not adding
  7332. 2020-11-02T12:06:58.2836693Z [cluster2] I1102 12:05:05.673958 1 route.go:829] Found equivalent route, not adding
  7333. 2020-11-02T12:06:58.2837154Z [cluster2] I1102 12:05:05.674148 1 route.go:829] Found equivalent route, not adding
  7334. 2020-11-02T12:06:58.2837613Z [cluster2] I1102 12:05:05.674331 1 route.go:829] Found equivalent route, not adding
  7335. 2020-11-02T12:06:58.2838761Z [cluster2] I1102 12:05:05.674723 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7336. 2020-11-02T12:06:58.2840402Z [cluster2] I1102 12:05:05.674984 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7337. 2020-11-02T12:06:58.2842677Z [cluster2] I1102 12:05:05.675201 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7338. 2020-11-02T12:06:58.2846858Z [cluster2] I1102 12:05:30.589728 1 route.go:682] Informed of removed endpoint: {"metadata":{"name":"cluster2-submariner-cable-cluster2-172-17-0-4","namespace":"submariner-operator","selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4","uid":"8e4ce5be-6521-4f69-bef3-cde34ed638a9","resourceVersion":"3728","generation":1,"creationTimestamp":"2020-11-02T12:01:40Z"},"spec":{"cluster_id":"cluster2","cable_name":"submariner-cable-cluster2-172-17-0-4","hostname":"cluster2-worker","subnets":["100.2.0.0/16","10.2.0.0/16"],"private_ip":"172.17.0.4","public_ip":"","nat_enabled":false,"backend":"libreswan"}}
  7339. 2020-11-02T12:06:58.2850367Z [cluster2] I1102 12:05:30.589811 1 route.go:711] Local non-gateway Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with IP "172.17.0.4" was removed: deleting vxlan interface
  7340. 2020-11-02T12:06:58.2852128Z [cluster2] I1102 12:05:30.689776 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-11 with IP 172.17.0.11, Host cluster2-worker2 for this cluster
  7341. 2020-11-02T12:06:58.2853386Z [cluster2] I1102 12:05:30.689871 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7342. 2020-11-02T12:06:58.2854196Z [cluster2] I1102 12:05:30.689897 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.11
  7343. 2020-11-02T12:06:58.2854741Z [cluster2] I1102 12:05:30.701345 1 route.go:771] Reconciling routes to gw: 240.17.0.11
  7344. 2020-11-02T12:06:58.2855249Z [cluster2] I1102 12:05:30.701591 1 route.go:787] Processing route {Ifindex: 13 Dst: 240.0.0.0/8 Src: 240.17.0.8 Gw: <nil> Flags: [] Table: 254}
  7345. 2020-11-02T12:06:58.2855721Z [cluster2] I1102 12:05:30.701655 1 route.go:790] Found nil gw or dst
  7346. 2020-11-02T12:06:58.2856979Z [cluster2] I1102 12:05:30.701911 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11"
  7347. 2020-11-02T12:06:58.2859617Z [cluster2] I1102 12:05:35.427065 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-11 with IP 172.17.0.11, Host cluster2-worker2 for this cluster
  7348. 2020-11-02T12:06:58.2861207Z [cluster2] I1102 12:05:35.427168 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7349. 2020-11-02T12:06:58.2862029Z [cluster2] I1102 12:05:35.427195 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.11
  7350. 2020-11-02T12:06:58.2862659Z [cluster2] I1102 12:05:35.427462 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7351. 2020-11-02T12:06:58.2863184Z [cluster2] I1102 12:05:35.431120 1 route.go:771] Reconciling routes to gw: 240.17.0.11
  7352. 2020-11-02T12:06:58.2863813Z [cluster2] I1102 12:05:35.431368 1 route.go:787] Processing route {Ifindex: 13 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  7353. 2020-11-02T12:06:58.2864431Z [cluster2] I1102 12:05:35.431399 1 route.go:793] Found route {Ifindex: 13 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  7354. 2020-11-02T12:06:58.2865038Z [cluster2] I1102 12:05:35.431409 1 route.go:787] Processing route {Ifindex: 13 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  7355. 2020-11-02T12:06:58.2865627Z [cluster2] I1102 12:05:35.431506 1 route.go:793] Found route {Ifindex: 13 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  7356. 2020-11-02T12:06:58.2866230Z [cluster2] I1102 12:05:35.431666 1 route.go:787] Processing route {Ifindex: 13 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  7357. 2020-11-02T12:06:58.2867196Z [cluster2] I1102 12:05:35.431692 1 route.go:793] Found route {Ifindex: 13 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  7358. 2020-11-02T12:06:58.2867944Z [cluster2] I1102 12:05:35.431701 1 route.go:787] Processing route {Ifindex: 13 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  7359. 2020-11-02T12:06:58.2868637Z [cluster2] I1102 12:05:35.431708 1 route.go:793] Found route {Ifindex: 13 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  7360. 2020-11-02T12:06:58.2869253Z [cluster2] I1102 12:05:35.431716 1 route.go:787] Processing route {Ifindex: 13 Dst: 240.0.0.0/8 Src: 240.17.0.8 Gw: <nil> Flags: [] Table: 254}
  7361. 2020-11-02T12:06:58.2869716Z [cluster2] I1102 12:05:35.431723 1 route.go:790] Found nil gw or dst
  7362. 2020-11-02T12:06:58.2870134Z [cluster2] I1102 12:05:35.431972 1 route.go:829] Found equivalent route, not adding
  7363. 2020-11-02T12:06:58.2870608Z [cluster2] I1102 12:05:35.431994 1 route.go:829] Found equivalent route, not adding
  7364. 2020-11-02T12:06:58.2871069Z [cluster2] I1102 12:05:35.432003 1 route.go:829] Found equivalent route, not adding
  7365. 2020-11-02T12:06:58.2871530Z [cluster2] I1102 12:05:35.432010 1 route.go:829] Found equivalent route, not adding
  7366. 2020-11-02T12:06:58.2872737Z [cluster2] I1102 12:05:35.432072 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11"
  7367. 2020-11-02T12:06:58.2874391Z [cluster2] I1102 12:05:35.432090 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7368. 2020-11-02T12:06:58.2876530Z [cluster2] I1102 12:05:35.432102 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7369. 2020-11-02T12:06:58.2878902Z [cluster2] I1102 12:06:05.428273 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-11 with IP 172.17.0.11, Host cluster2-worker2 for this cluster
  7370. 2020-11-02T12:06:58.2880371Z [cluster2] I1102 12:06:05.428302 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7371. 2020-11-02T12:06:58.2881189Z [cluster2] I1102 12:06:05.428315 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.11
  7372. 2020-11-02T12:06:58.2881818Z [cluster2] I1102 12:06:05.428613 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7373. 2020-11-02T12:06:58.2882343Z [cluster2] I1102 12:06:05.441047 1 route.go:771] Reconciling routes to gw: 240.17.0.11
  7374. 2020-11-02T12:06:58.2882850Z [cluster2] I1102 12:06:05.442762 1 route.go:787] Processing route {Ifindex: 13 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  7375. 2020-11-02T12:06:58.2883445Z [cluster2] I1102 12:06:05.442807 1 route.go:793] Found route {Ifindex: 13 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  7376. 2020-11-02T12:06:58.2884283Z [cluster2] I1102 12:06:05.442818 1 route.go:787] Processing route {Ifindex: 13 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  7377. 2020-11-02T12:06:58.2884896Z [cluster2] I1102 12:06:05.442829 1 route.go:793] Found route {Ifindex: 13 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  7378. 2020-11-02T12:06:58.2885507Z [cluster2] I1102 12:06:05.442837 1 route.go:787] Processing route {Ifindex: 13 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  7379. 2020-11-02T12:06:58.2886112Z [cluster2] I1102 12:06:05.442845 1 route.go:793] Found route {Ifindex: 13 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  7380. 2020-11-02T12:06:58.2886717Z [cluster2] I1102 12:06:05.442857 1 route.go:787] Processing route {Ifindex: 13 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  7381. 2020-11-02T12:06:58.2887324Z [cluster2] I1102 12:06:05.442866 1 route.go:793] Found route {Ifindex: 13 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  7382. 2020-11-02T12:06:58.2887996Z [cluster2] I1102 12:06:05.442880 1 route.go:787] Processing route {Ifindex: 13 Dst: 240.0.0.0/8 Src: 240.17.0.8 Gw: <nil> Flags: [] Table: 254}
  7383. 2020-11-02T12:06:58.2888465Z [cluster2] I1102 12:06:05.442890 1 route.go:790] Found nil gw or dst
  7384. 2020-11-02T12:06:58.2888880Z [cluster2] I1102 12:06:05.442955 1 route.go:829] Found equivalent route, not adding
  7385. 2020-11-02T12:06:58.2889662Z [cluster2] I1102 12:06:05.442972 1 route.go:829] Found equivalent route, not adding
  7386. 2020-11-02T12:06:58.2890251Z [cluster2] I1102 12:06:05.442982 1 route.go:829] Found equivalent route, not adding
  7387. 2020-11-02T12:06:58.2890722Z [cluster2] I1102 12:06:05.442997 1 route.go:829] Found equivalent route, not adding
  7388. 2020-11-02T12:06:58.2892132Z [cluster2] I1102 12:06:05.443030 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11"
  7389. 2020-11-02T12:06:58.2893836Z [cluster2] I1102 12:06:05.443054 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7390. 2020-11-02T12:06:58.2896393Z [cluster2] I1102 12:06:05.443079 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7391. 2020-11-02T12:06:58.2898815Z [cluster2] I1102 12:06:35.428139 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7392. 2020-11-02T12:06:58.2902025Z [cluster2] I1102 12:06:35.428180 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7393. 2020-11-02T12:06:58.2904259Z [cluster2] I1102 12:06:35.428190 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-11 with IP 172.17.0.11, Host cluster2-worker2 for this cluster
  7394. 2020-11-02T12:06:58.2905796Z [cluster2] I1102 12:06:35.428341 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7395. 2020-11-02T12:06:58.2906796Z [cluster2] I1102 12:06:35.428501 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.11
  7396. 2020-11-02T12:06:58.2908000Z [cluster2] I1102 12:06:35.428794 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7397. 2020-11-02T12:06:58.2908810Z [cluster2] I1102 12:06:35.432105 1 route.go:771] Reconciling routes to gw: 240.17.0.11
  7398. 2020-11-02T12:06:58.2910068Z [cluster2] I1102 12:06:35.432314 1 route.go:787] Processing route {Ifindex: 13 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  7399. 2020-11-02T12:06:58.2911161Z [cluster2] I1102 12:06:35.432346 1 route.go:793] Found route {Ifindex: 13 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  7400. 2020-11-02T12:06:58.2911940Z [cluster2] I1102 12:06:35.432357 1 route.go:787] Processing route {Ifindex: 13 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  7401. 2020-11-02T12:06:58.2913578Z [cluster2] I1102 12:06:35.432366 1 route.go:793] Found route {Ifindex: 13 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  7402. 2020-11-02T12:06:58.2914397Z [cluster2] I1102 12:06:35.432373 1 route.go:787] Processing route {Ifindex: 13 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  7403. 2020-11-02T12:06:58.2915765Z [cluster2] I1102 12:06:35.432476 1 route.go:793] Found route {Ifindex: 13 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  7404. 2020-11-02T12:06:58.2916631Z [cluster2] I1102 12:06:35.432555 1 route.go:787] Processing route {Ifindex: 13 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254}
  7405. 2020-11-02T12:06:58.2917264Z [cluster2] I1102 12:06:35.432891 1 route.go:793] Found route {Ifindex: 13 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.11 Flags: [] Table: 254} with gw 240.17.0.11 already installed
  7406. 2020-11-02T12:06:58.2917975Z [cluster2] I1102 12:06:35.432969 1 route.go:787] Processing route {Ifindex: 13 Dst: 240.0.0.0/8 Src: 240.17.0.8 Gw: <nil> Flags: [] Table: 254}
  7407. 2020-11-02T12:06:58.2918450Z [cluster2] I1102 12:06:35.433077 1 route.go:790] Found nil gw or dst
  7408. 2020-11-02T12:06:58.2918868Z [cluster2] I1102 12:06:35.433439 1 route.go:829] Found equivalent route, not adding
  7409. 2020-11-02T12:06:58.2919342Z [cluster2] I1102 12:06:35.433477 1 route.go:829] Found equivalent route, not adding
  7410. 2020-11-02T12:06:58.2919808Z [cluster2] I1102 12:06:35.433514 1 route.go:829] Found equivalent route, not adding
  7411. 2020-11-02T12:06:58.2920847Z [cluster2] I1102 12:06:35.433572 1 route.go:829] Found equivalent route, not adding
  7412. 2020-11-02T12:06:58.2922280Z [cluster2] I1102 12:06:35.433684 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11"
  7413. 2020-11-02T12:06:58.2926139Z [cluster2] +++++++++++++++++++++: Logs for Pod submariner-routeagent-swbc2 in namespace submariner-operator :++++++++++++++++++++++
  7414. 2020-11-02T12:06:58.2927583Z [cluster2] + trap 'exit 1' SIGTERM SIGINT
  7415. 2020-11-02T12:06:58.2928100Z [cluster2] + SUBMARINER_VERBOSITY=2
  7416. 2020-11-02T12:06:58.2928539Z [cluster2] + '[' false == true ']'
  7417. 2020-11-02T12:06:58.2928907Z [cluster2] + DEBUG=-v=2
  7418. 2020-11-02T12:06:58.2929352Z [cluster2] + for f in iptables-save iptables
  7419. 2020-11-02T12:06:58.2929881Z [cluster2] ++ find_iptables_on_host iptables-save
  7420. 2020-11-02T12:06:58.2930423Z [cluster2] ++ chroot /host test -x /usr/sbin/iptables-save
  7421. 2020-11-02T12:06:58.2930775Z [cluster2] ++ echo /usr/sbin
  7422. 2020-11-02T12:06:58.2931034Z [cluster2] ++ return
  7423. 2020-11-02T12:06:58.2931317Z [cluster2] + location=/usr/sbin
  7424. 2020-11-02T12:06:58.2931755Z [cluster2] + '[' /usr/sbin '!=' unknown ']'
  7425. 2020-11-02T12:06:58.2932349Z [cluster2] + echo 'iptables-save is present on the host at /usr/sbin/iptables-save'
  7426. 2020-11-02T12:06:58.2933039Z [cluster2] + sed 's!@@PATH@@!/usr/sbin!' /usr/sbin/iptables-wrapper.in
  7427. 2020-11-02T12:06:58.2933900Z [cluster2] iptables-save is present on the host at /usr/sbin/iptables-save
  7428. 2020-11-02T12:06:58.2934501Z [cluster2] + for f in iptables-save iptables
  7429. 2020-11-02T12:06:58.2934893Z [cluster2] ++ find_iptables_on_host iptables
  7430. 2020-11-02T12:06:58.2935412Z [cluster2] ++ chroot /host test -x /usr/sbin/iptables
  7431. 2020-11-02T12:06:58.2935748Z [cluster2] ++ echo /usr/sbin
  7432. 2020-11-02T12:06:58.2936122Z [cluster2] iptables is present on the host at /usr/sbin/iptables
  7433. 2020-11-02T12:06:58.2936478Z [cluster2] ++ return
  7434. 2020-11-02T12:06:58.2937151Z [cluster2] + location=/usr/sbin
  7435. 2020-11-02T12:06:58.2937592Z [cluster2] + '[' /usr/sbin '!=' unknown ']'
  7436. 2020-11-02T12:06:58.2939188Z [cluster2] + echo 'iptables is present on the host at /usr/sbin/iptables'
  7437. 2020-11-02T12:06:58.2939923Z [cluster2] + sed 's!@@PATH@@!/usr/sbin!' /usr/sbin/iptables-wrapper.in
  7438. 2020-11-02T12:06:58.2940793Z [cluster2] + exec submariner-route-agent -v=2 -alsologtostderr
  7439. 2020-11-02T12:06:58.2941545Z [cluster2] I1102 11:58:04.798993 1 main.go:47] Starting submariner-route-agent
  7440. 2020-11-02T12:06:58.2942502Z [cluster2] W1102 11:58:05.121321 1 client_config.go:543] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
  7441. 2020-11-02T12:06:58.2943398Z [cluster2] I1102 11:58:05.125639 1 route.go:192] Starting Route Controller. ClusterID: cluster2, localClusterCIDR: [10.2.0.0/16], localServiceCIDR: [100.2.0.0/16]
  7442. 2020-11-02T12:06:58.2944769Z [cluster2] I1102 11:58:05.125679 1 route.go:196] Waiting for Endpoint informer caches to sync.
  7443. 2020-11-02T12:06:58.2945731Z [cluster2] I1102 11:58:05.241995 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.8
  7444. 2020-11-02T12:06:58.2947230Z [cluster2] I1102 11:58:05.242025 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.11
  7445. 2020-11-02T12:06:58.2948051Z [cluster2] I1102 11:58:05.242054 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.4
  7446. 2020-11-02T12:06:58.2948934Z [cluster2] I1102 11:58:05.287540 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.8
  7447. 2020-11-02T12:06:58.2949461Z [cluster2] I1102 11:58:05.327727 1 driver.go:37] Interface "lo" has "127.0.0.1" address
  7448. 2020-11-02T12:06:58.2949892Z [cluster2] I1102 11:58:05.328373 1 driver.go:37] Interface "weave" has "10.2.128.0" address
  7449. 2020-11-02T12:06:58.2950426Z [cluster2] I1102 11:58:05.330356 1 driver.go:42] Found CNI Interface "weave" that has IP "10.2.128.0" from ClusterCIDR "10.2.0.0/16"
  7450. 2020-11-02T12:06:58.2951079Z [cluster2] I1102 11:58:05.331083 1 driver.go:60] Successfully configured rp_filter to loose mode(2) on cniInterface "weave"
  7451. 2020-11-02T12:06:58.2952549Z [cluster2] I1102 11:58:05.406459 1 iptables.go:22] Install/ensure SUBMARINER-POSTROUTING chain exists
  7452. 2020-11-02T12:06:58.2953779Z [cluster2] I1102 11:58:05.520954 1 iptables.go:28] Insert SUBMARINER-POSTROUTING rule that has rules for inter-cluster traffic
  7453. 2020-11-02T12:06:58.2954696Z [cluster2] I1102 11:58:05.537460 1 iptables.go:34] Install/ensure SUBMARINER-INPUT chain exists
  7454. 2020-11-02T12:06:58.2955479Z [cluster2] I1102 11:58:05.545562 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.11
  7455. 2020-11-02T12:06:58.2956273Z [cluster2] I1102 11:58:05.644966 1 iptables.go:45] Allow VxLAN incoming traffic in SUBMARINER-INPUT Chain
  7456. 2020-11-02T12:06:58.2957165Z [cluster2] I1102 11:58:05.664711 1 iptables.go:53] Insert rule to allow traffic over vx-submariner interface in FORWARDing Chain
  7457. 2020-11-02T12:06:58.2958200Z [cluster2] I1102 11:58:05.683847 1 iptables.go:65] Installing rule for host network to remote cluster communication: -s 240.0.0.0/8 -o vx-submariner -j SNAT --to 10.2.128.0
  7458. 2020-11-02T12:06:58.2959338Z [cluster2] I1102 11:58:07.096831 1 route.go:254] In cluster2, podIP of submariner-route-agent[0] is 172.17.0.4
  7459. 2020-11-02T12:06:58.2960218Z [cluster2] I1102 11:58:07.096884 1 route.go:254] In cluster2, podIP of submariner-route-agent[1] is 172.17.0.8
  7460. 2020-11-02T12:06:58.2961099Z [cluster2] I1102 11:58:07.096889 1 route.go:254] In cluster2, podIP of submariner-route-agent[2] is 172.17.0.11
  7461. 2020-11-02T12:06:58.2962094Z [cluster2] I1102 11:58:07.096893 1 route.go:268] Hostname is "cluster2-worker2" and routeAgentNodeName is "cluster2-worker2"
  7462. 2020-11-02T12:06:58.2963105Z [cluster2] I1102 11:58:07.285886 1 driver.go:87] Successfully annotated node "cluster2-worker2" with cniIfaceIP "10.2.128.0"
  7463. 2020-11-02T12:06:58.2964777Z [cluster2] I1102 11:58:07.285916 1 route.go:286] Route agent workers started
  7464. 2020-11-02T12:06:58.2966532Z [cluster2] I1102 11:58:07.285943 1 route.go:507] Processing submariner-route-agent pod "submariner-routeagent-ftrjn" with host IP "172.17.0.8", pod IP "172.17.0.8"
  7465. 2020-11-02T12:06:58.2968375Z [cluster2] I1102 11:58:07.285950 1 route.go:531] Successfully processed submariner-route-agent pod "submariner-routeagent-ftrjn"
  7466. 2020-11-02T12:06:58.2970292Z [cluster2] I1102 11:58:07.285956 1 route.go:507] Processing submariner-route-agent pod "submariner-routeagent-swbc2" with host IP "172.17.0.11", pod IP "172.17.0.11"
  7467. 2020-11-02T12:06:58.2971786Z [cluster2] I1102 11:58:07.285960 1 route.go:531] Successfully processed submariner-route-agent pod "submariner-routeagent-swbc2"
  7468. 2020-11-02T12:06:58.2973162Z [cluster2] I1102 11:58:07.285965 1 route.go:507] Processing submariner-route-agent pod "submariner-routeagent-8x2cc" with host IP "172.17.0.4", pod IP "172.17.0.4"
  7469. 2020-11-02T12:06:58.2974494Z [cluster2] I1102 11:58:07.285969 1 route.go:531] Successfully processed submariner-route-agent pod "submariner-routeagent-8x2cc"
  7470. 2020-11-02T12:06:58.2976815Z [cluster2] I1102 11:58:19.435802 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7471. 2020-11-02T12:06:58.2978166Z [cluster2] I1102 11:58:19.435840 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7472. 2020-11-02T12:06:58.2979126Z [cluster2] I1102 11:58:19.435851 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7473. 2020-11-02T12:06:58.2979697Z [cluster2] I1102 11:58:19.702921 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7474. 2020-11-02T12:06:58.2980224Z [cluster2] I1102 11:58:19.703183 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.11 Gw: <nil> Flags: [] Table: 254}
  7475. 2020-11-02T12:06:58.2980705Z [cluster2] I1102 11:58:19.703212 1 route.go:790] Found nil gw or dst
  7476. 2020-11-02T12:06:58.2981837Z [cluster2] I1102 11:58:19.705769 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7477. 2020-11-02T12:06:58.2983540Z [cluster2] I1102 11:58:24.184004 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7478. 2020-11-02T12:06:58.2985299Z [cluster2] I1102 11:58:24.198803 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.2.0.0/16 -d 100.3.0.0/16 -j ACCEPT
  7479. 2020-11-02T12:06:58.2986399Z [cluster2] I1102 11:58:24.234816 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 100.3.0.0/16 -d 10.2.0.0/16 -j ACCEPT
  7480. 2020-11-02T12:06:58.2987283Z [cluster2] I1102 11:58:24.284120 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.2.0.0/16 -d 10.3.0.0/16 -j ACCEPT
  7481. 2020-11-02T12:06:58.2988343Z [cluster2] I1102 11:58:24.309176 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 10.3.0.0/16 -d 10.2.0.0/16 -j ACCEPT
  7482. 2020-11-02T12:06:58.2990624Z [cluster2] I1102 11:58:27.573183 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7483. 2020-11-02T12:06:58.2992245Z [cluster2] I1102 11:58:27.599407 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.2.0.0/16 -d 100.1.0.0/16 -j ACCEPT
  7484. 2020-11-02T12:06:58.2994005Z [cluster2] I1102 11:58:27.617536 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 100.1.0.0/16 -d 10.2.0.0/16 -j ACCEPT
  7485. 2020-11-02T12:06:58.2995098Z [cluster2] I1102 11:58:27.661398 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.2.0.0/16 -d 10.1.0.0/16 -j ACCEPT
  7486. 2020-11-02T12:06:58.2995983Z [cluster2] I1102 11:58:27.712632 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 10.1.0.0/16 -d 10.2.0.0/16 -j ACCEPT
  7487. 2020-11-02T12:06:58.2997340Z [cluster2] I1102 11:58:35.230445 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7488. 2020-11-02T12:06:58.2998782Z [cluster2] I1102 11:58:35.230473 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7489. 2020-11-02T12:06:58.2999627Z [cluster2] I1102 11:58:35.230501 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7490. 2020-11-02T12:06:58.3000277Z [cluster2] I1102 11:58:35.230865 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7491. 2020-11-02T12:06:58.3000818Z [cluster2] I1102 11:58:35.235988 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7492. 2020-11-02T12:06:58.3001341Z [cluster2] I1102 11:58:35.236320 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.11 Gw: <nil> Flags: [] Table: 254}
  7493. 2020-11-02T12:06:58.3001820Z [cluster2] I1102 11:58:35.236521 1 route.go:790] Found nil gw or dst
  7494. 2020-11-02T12:06:58.3002921Z [cluster2] I1102 11:58:35.236871 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7495. 2020-11-02T12:06:58.3004822Z [cluster2] I1102 11:58:35.237005 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7496. 2020-11-02T12:06:58.3006734Z [cluster2] I1102 11:58:35.237124 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7497. 2020-11-02T12:06:58.3008556Z [cluster2] I1102 11:59:05.196940 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7498. 2020-11-02T12:06:58.3009815Z [cluster2] I1102 11:59:05.196990 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7499. 2020-11-02T12:06:58.3010662Z [cluster2] I1102 11:59:05.197001 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7500. 2020-11-02T12:06:58.3011315Z [cluster2] I1102 11:59:05.197288 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7501. 2020-11-02T12:06:58.3011857Z [cluster2] I1102 11:59:05.201341 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7502. 2020-11-02T12:06:58.3012383Z [cluster2] I1102 11:59:05.201521 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7503. 2020-11-02T12:06:58.3013065Z [cluster2] I1102 11:59:05.201555 1 route.go:793] Found route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7504. 2020-11-02T12:06:58.3013690Z [cluster2] I1102 11:59:05.201565 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7505. 2020-11-02T12:06:58.3014321Z [cluster2] I1102 11:59:05.201579 1 route.go:793] Found route {Ifindex: 15 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7506. 2020-11-02T12:06:58.3014951Z [cluster2] I1102 11:59:05.201587 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7507. 2020-11-02T12:06:58.3015581Z [cluster2] I1102 11:59:05.201602 1 route.go:793] Found route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7508. 2020-11-02T12:06:58.3016576Z [cluster2] I1102 11:59:05.201610 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7509. 2020-11-02T12:06:58.3017238Z [cluster2] I1102 11:59:05.201623 1 route.go:793] Found route {Ifindex: 15 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7510. 2020-11-02T12:06:58.3017899Z [cluster2] I1102 11:59:05.201632 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.11 Gw: <nil> Flags: [] Table: 254}
  7511. 2020-11-02T12:06:58.3018403Z [cluster2] I1102 11:59:05.201645 1 route.go:790] Found nil gw or dst
  7512. 2020-11-02T12:06:58.3018962Z [cluster2] I1102 11:59:05.201755 1 route.go:829] Found equivalent route, not adding
  7513. 2020-11-02T12:06:58.3019478Z [cluster2] I1102 11:59:05.201825 1 route.go:829] Found equivalent route, not adding
  7514. 2020-11-02T12:06:58.3019991Z [cluster2] I1102 11:59:05.201842 1 route.go:829] Found equivalent route, not adding
  7515. 2020-11-02T12:06:58.3020792Z [cluster2] I1102 11:59:05.201852 1 route.go:829] Found equivalent route, not adding
  7516. 2020-11-02T12:06:58.3021990Z [cluster2] I1102 11:59:05.201903 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7517. 2020-11-02T12:06:58.3023696Z [cluster2] I1102 11:59:05.201925 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7518. 2020-11-02T12:06:58.3025452Z [cluster2] I1102 11:59:05.201943 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7519. 2020-11-02T12:06:58.3027234Z [cluster2] I1102 11:59:35.197942 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7520. 2020-11-02T12:06:58.3028579Z [cluster2] I1102 11:59:35.198002 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7521. 2020-11-02T12:06:58.3029464Z [cluster2] I1102 11:59:35.198013 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7522. 2020-11-02T12:06:58.3030112Z [cluster2] I1102 11:59:35.198217 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7523. 2020-11-02T12:06:58.3030651Z [cluster2] I1102 11:59:35.202517 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7524. 2020-11-02T12:06:58.3031173Z [cluster2] I1102 11:59:35.202679 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7525. 2020-11-02T12:06:58.3031806Z [cluster2] I1102 11:59:35.202716 1 route.go:793] Found route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7526. 2020-11-02T12:06:58.3032437Z [cluster2] I1102 11:59:35.202727 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7527. 2020-11-02T12:06:58.3033062Z [cluster2] I1102 11:59:35.202735 1 route.go:793] Found route {Ifindex: 15 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7528. 2020-11-02T12:06:58.3033688Z [cluster2] I1102 11:59:35.202763 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7529. 2020-11-02T12:06:58.3034315Z [cluster2] I1102 11:59:35.202780 1 route.go:793] Found route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7530. 2020-11-02T12:06:58.3034942Z [cluster2] I1102 11:59:35.202791 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7531. 2020-11-02T12:06:58.3035560Z [cluster2] I1102 11:59:35.202805 1 route.go:793] Found route {Ifindex: 15 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7532. 2020-11-02T12:06:58.3036181Z [cluster2] I1102 11:59:35.202814 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.11 Gw: <nil> Flags: [] Table: 254}
  7533. 2020-11-02T12:06:58.3036654Z [cluster2] I1102 11:59:35.202823 1 route.go:790] Found nil gw or dst
  7534. 2020-11-02T12:06:58.3037068Z [cluster2] I1102 11:59:35.202886 1 route.go:829] Found equivalent route, not adding
  7535. 2020-11-02T12:06:58.3037789Z [cluster2] I1102 11:59:35.202917 1 route.go:829] Found equivalent route, not adding
  7536. 2020-11-02T12:06:58.3038499Z [cluster2] I1102 11:59:35.202932 1 route.go:829] Found equivalent route, not adding
  7537. 2020-11-02T12:06:58.3039280Z [cluster2] I1102 11:59:35.202946 1 route.go:829] Found equivalent route, not adding
  7538. 2020-11-02T12:06:58.3040714Z [cluster2] I1102 11:59:35.202978 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7539. 2020-11-02T12:06:58.3042551Z [cluster2] I1102 11:59:35.203003 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7540. 2020-11-02T12:06:58.3044982Z [cluster2] I1102 11:59:35.203016 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7541. 2020-11-02T12:06:58.3047047Z [cluster2] I1102 12:00:05.199754 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7542. 2020-11-02T12:06:58.3048403Z [cluster2] I1102 12:00:05.199800 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7543. 2020-11-02T12:06:58.3049315Z [cluster2] I1102 12:00:05.199811 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7544. 2020-11-02T12:06:58.3050326Z [cluster2] I1102 12:00:05.200022 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7545. 2020-11-02T12:06:58.3051141Z [cluster2] I1102 12:00:05.206100 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7546. 2020-11-02T12:06:58.3051876Z [cluster2] I1102 12:00:05.206854 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7547. 2020-11-02T12:06:58.3052549Z [cluster2] I1102 12:00:05.206905 1 route.go:793] Found route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7548. 2020-11-02T12:06:58.3053575Z [cluster2] I1102 12:00:05.206916 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7549. 2020-11-02T12:06:58.3054298Z [cluster2] I1102 12:00:05.206925 1 route.go:793] Found route {Ifindex: 15 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7550. 2020-11-02T12:06:58.3055310Z [cluster2] I1102 12:00:05.206932 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7551. 2020-11-02T12:06:58.3056643Z [cluster2] I1102 12:00:05.206940 1 route.go:793] Found route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7552. 2020-11-02T12:06:58.3057474Z [cluster2] I1102 12:00:05.206948 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7553. 2020-11-02T12:06:58.3058125Z [cluster2] I1102 12:00:05.207171 1 route.go:793] Found route {Ifindex: 15 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7554. 2020-11-02T12:06:58.3058774Z [cluster2] I1102 12:00:05.207185 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.11 Gw: <nil> Flags: [] Table: 254}
  7555. 2020-11-02T12:06:58.3059270Z [cluster2] I1102 12:00:05.207193 1 route.go:790] Found nil gw or dst
  7556. 2020-11-02T12:06:58.3059869Z [cluster2] I1102 12:00:05.207307 1 route.go:829] Found equivalent route, not adding
  7557. 2020-11-02T12:06:58.3060351Z [cluster2] I1102 12:00:05.207354 1 route.go:829] Found equivalent route, not adding
  7558. 2020-11-02T12:06:58.3061344Z [cluster2] I1102 12:00:05.207365 1 route.go:829] Found equivalent route, not adding
  7559. 2020-11-02T12:06:58.3062228Z [cluster2] I1102 12:00:05.207373 1 route.go:829] Found equivalent route, not adding
  7560. 2020-11-02T12:06:58.3063561Z [cluster2] I1102 12:00:05.207441 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7561. 2020-11-02T12:06:58.3065499Z [cluster2] I1102 12:00:05.207605 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7562. 2020-11-02T12:06:58.3067825Z [cluster2] I1102 12:00:05.207623 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7563. 2020-11-02T12:06:58.3069732Z [cluster2] I1102 12:00:35.199870 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7564. 2020-11-02T12:06:58.3072250Z [cluster2] I1102 12:00:35.199923 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7565. 2020-11-02T12:06:58.3073542Z [cluster2] I1102 12:00:35.199933 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7566. 2020-11-02T12:06:58.3074283Z [cluster2] I1102 12:00:35.200125 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7567. 2020-11-02T12:06:58.3075203Z [cluster2] I1102 12:00:35.203323 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7568. 2020-11-02T12:06:58.3076049Z [cluster2] I1102 12:00:35.203473 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7569. 2020-11-02T12:06:58.3077244Z [cluster2] I1102 12:00:35.203495 1 route.go:793] Found route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7570. 2020-11-02T12:06:58.3077890Z [cluster2] I1102 12:00:35.203503 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7571. 2020-11-02T12:06:58.3078517Z [cluster2] I1102 12:00:35.203510 1 route.go:793] Found route {Ifindex: 15 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7572. 2020-11-02T12:06:58.3079888Z [cluster2] I1102 12:00:35.203516 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7573. 2020-11-02T12:06:58.3081438Z [cluster2] I1102 12:00:35.203524 1 route.go:793] Found route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7574. 2020-11-02T12:06:58.3082208Z [cluster2] I1102 12:00:35.203530 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7575. 2020-11-02T12:06:58.3083382Z [cluster2] I1102 12:00:35.203536 1 route.go:793] Found route {Ifindex: 15 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7576. 2020-11-02T12:06:58.3084628Z [cluster2] I1102 12:00:35.203542 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.11 Gw: <nil> Flags: [] Table: 254}
  7577. 2020-11-02T12:06:58.3085105Z [cluster2] I1102 12:00:35.203563 1 route.go:790] Found nil gw or dst
  7578. 2020-11-02T12:06:58.3085532Z [cluster2] I1102 12:00:35.203631 1 route.go:829] Found equivalent route, not adding
  7579. 2020-11-02T12:06:58.3086024Z [cluster2] I1102 12:00:35.203655 1 route.go:829] Found equivalent route, not adding
  7580. 2020-11-02T12:06:58.3086504Z [cluster2] I1102 12:00:35.203671 1 route.go:829] Found equivalent route, not adding
  7581. 2020-11-02T12:06:58.3086983Z [cluster2] I1102 12:00:35.203678 1 route.go:829] Found equivalent route, not adding
  7582. 2020-11-02T12:06:58.3088274Z [cluster2] I1102 12:00:35.203705 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7583. 2020-11-02T12:06:58.3092957Z [cluster2] I1102 12:00:35.203719 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7584. 2020-11-02T12:06:58.3095365Z [cluster2] I1102 12:00:35.203730 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7585. 2020-11-02T12:06:58.3097708Z [cluster2] I1102 12:01:05.200772 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7586. 2020-11-02T12:06:58.3099029Z [cluster2] I1102 12:01:05.200825 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7587. 2020-11-02T12:06:58.3100066Z [cluster2] I1102 12:01:05.200836 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7588. 2020-11-02T12:06:58.3101416Z [cluster2] I1102 12:01:05.201114 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7589. 2020-11-02T12:06:58.3103432Z [cluster2] I1102 12:01:05.206025 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7590. 2020-11-02T12:06:58.3104041Z [cluster2] I1102 12:01:05.206213 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7591. 2020-11-02T12:06:58.3104763Z [cluster2] I1102 12:01:05.206263 1 route.go:793] Found route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7592. 2020-11-02T12:06:58.3106066Z [cluster2] I1102 12:01:05.206274 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7593. 2020-11-02T12:06:58.3106812Z [cluster2] I1102 12:01:05.206282 1 route.go:793] Found route {Ifindex: 15 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7594. 2020-11-02T12:06:58.3107457Z [cluster2] I1102 12:01:05.206297 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7595. 2020-11-02T12:06:58.3108088Z [cluster2] I1102 12:01:05.206306 1 route.go:793] Found route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7596. 2020-11-02T12:06:58.3108709Z [cluster2] I1102 12:01:05.206321 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7597. 2020-11-02T12:06:58.3109644Z [cluster2] I1102 12:01:05.206330 1 route.go:793] Found route {Ifindex: 15 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7598. 2020-11-02T12:06:58.3110410Z [cluster2] I1102 12:01:05.206347 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.11 Gw: <nil> Flags: [] Table: 254}
  7599. 2020-11-02T12:06:58.3112495Z [cluster2] I1102 12:01:05.206356 1 route.go:790] Found nil gw or dst
  7600. 2020-11-02T12:06:58.3112969Z [cluster2] I1102 12:01:05.206438 1 route.go:829] Found equivalent route, not adding
  7601. 2020-11-02T12:06:58.3113486Z [cluster2] I1102 12:01:05.206457 1 route.go:829] Found equivalent route, not adding
  7602. 2020-11-02T12:06:58.3113981Z [cluster2] I1102 12:01:05.206466 1 route.go:829] Found equivalent route, not adding
  7603. 2020-11-02T12:06:58.3116534Z [cluster2] I1102 12:01:05.206480 1 route.go:829] Found equivalent route, not adding
  7604. 2020-11-02T12:06:58.3118702Z [cluster2] I1102 12:01:05.206524 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7605. 2020-11-02T12:06:58.3120794Z [cluster2] I1102 12:01:05.206549 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7606. 2020-11-02T12:06:58.3124201Z [cluster2] I1102 12:01:05.206568 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7607. 2020-11-02T12:06:58.3127981Z [cluster2] I1102 12:01:35.200182 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7608. 2020-11-02T12:06:58.3129353Z [cluster2] I1102 12:01:35.200208 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7609. 2020-11-02T12:06:58.3132054Z [cluster2] I1102 12:01:35.200235 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7610. 2020-11-02T12:06:58.3133040Z [cluster2] I1102 12:01:35.200510 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7611. 2020-11-02T12:06:58.3134334Z [cluster2] I1102 12:01:35.203689 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7612. 2020-11-02T12:06:58.3135110Z [cluster2] I1102 12:01:35.203849 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7613. 2020-11-02T12:06:58.3136005Z [cluster2] I1102 12:01:35.203870 1 route.go:793] Found route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7614. 2020-11-02T12:06:58.3136727Z [cluster2] I1102 12:01:35.203878 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7615. 2020-11-02T12:06:58.3137445Z [cluster2] I1102 12:01:35.203885 1 route.go:793] Found route {Ifindex: 15 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7616. 2020-11-02T12:06:58.3138170Z [cluster2] I1102 12:01:35.203892 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7617. 2020-11-02T12:06:58.3139026Z [cluster2] I1102 12:01:35.203899 1 route.go:793] Found route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7618. 2020-11-02T12:06:58.3140064Z [cluster2] I1102 12:01:35.203905 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7619. 2020-11-02T12:06:58.3140918Z [cluster2] I1102 12:01:35.203913 1 route.go:793] Found route {Ifindex: 15 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7620. 2020-11-02T12:06:58.3142373Z [cluster2] I1102 12:01:35.203919 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.11 Gw: <nil> Flags: [] Table: 254}
  7621. 2020-11-02T12:06:58.3144208Z [cluster2] I1102 12:01:35.203926 1 route.go:790] Found nil gw or dst
  7622. 2020-11-02T12:06:58.3144737Z [cluster2] I1102 12:01:35.203998 1 route.go:829] Found equivalent route, not adding
  7623. 2020-11-02T12:06:58.3145492Z [cluster2] I1102 12:01:35.204008 1 route.go:829] Found equivalent route, not adding
  7624. 2020-11-02T12:06:58.3146069Z [cluster2] I1102 12:01:35.204015 1 route.go:829] Found equivalent route, not adding
  7625. 2020-11-02T12:06:58.3146638Z [cluster2] I1102 12:01:35.204025 1 route.go:829] Found equivalent route, not adding
  7626. 2020-11-02T12:06:58.3149439Z [cluster2] I1102 12:01:35.204051 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7627. 2020-11-02T12:06:58.3151988Z [cluster2] I1102 12:01:35.204081 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7628. 2020-11-02T12:06:58.3155921Z [cluster2] I1102 12:01:35.204091 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7629. 2020-11-02T12:06:58.3161283Z [cluster2] I1102 12:01:40.266110 1 route.go:682] Informed of removed endpoint: {"metadata":{"name":"cluster2-submariner-cable-cluster2-172-17-0-4","namespace":"submariner-operator","selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4","uid":"9b8708cb-6f04-49db-ab5a-2c50a1a2b639","resourceVersion":"2844","generation":1,"creationTimestamp":"2020-11-02T11:58:19Z"},"spec":{"cluster_id":"cluster2","cable_name":"submariner-cable-cluster2-172-17-0-4","hostname":"cluster2-worker","subnets":["100.2.0.0/16","10.2.0.0/16"],"private_ip":"172.17.0.4","public_ip":"","nat_enabled":false,"backend":"libreswan"}}
  7630. 2020-11-02T12:06:58.3167849Z [cluster2] I1102 12:01:40.266129 1 route.go:711] Local non-gateway Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with IP "172.17.0.4" was removed: deleting vxlan interface
  7631. 2020-11-02T12:06:58.3172421Z [cluster2] I1102 12:01:40.321198 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7632. 2020-11-02T12:06:58.3174318Z [cluster2] I1102 12:01:40.321252 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7633. 2020-11-02T12:06:58.3175433Z [cluster2] I1102 12:01:40.321262 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7634. 2020-11-02T12:06:58.3176029Z [cluster2] I1102 12:01:40.325599 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7635. 2020-11-02T12:06:58.3176593Z [cluster2] I1102 12:01:40.327836 1 route.go:787] Processing route {Ifindex: 25 Dst: 240.0.0.0/8 Src: 240.17.0.11 Gw: <nil> Flags: [] Table: 254}
  7636. 2020-11-02T12:06:58.3177762Z [cluster2] I1102 12:01:40.327871 1 route.go:790] Found nil gw or dst
  7637. 2020-11-02T12:06:58.3179383Z [cluster2] I1102 12:01:40.328031 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7638. 2020-11-02T12:06:58.3181735Z [cluster2] I1102 12:02:05.200564 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7639. 2020-11-02T12:06:58.3183993Z [cluster2] I1102 12:02:05.200603 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7640. 2020-11-02T12:06:58.3185786Z [cluster2] I1102 12:02:05.200611 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7641. 2020-11-02T12:06:58.3186796Z [cluster2] I1102 12:02:05.200620 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7642. 2020-11-02T12:06:58.3187571Z [cluster2] I1102 12:02:05.200839 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7643. 2020-11-02T12:06:58.3188213Z [cluster2] I1102 12:02:05.204644 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7644. 2020-11-02T12:06:58.3189155Z [cluster2] I1102 12:02:05.204790 1 route.go:787] Processing route {Ifindex: 25 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7645. 2020-11-02T12:06:58.3189913Z [cluster2] I1102 12:02:05.204812 1 route.go:793] Found route {Ifindex: 25 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7646. 2020-11-02T12:06:58.3191651Z [cluster2] I1102 12:02:05.204820 1 route.go:787] Processing route {Ifindex: 25 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7647. 2020-11-02T12:06:58.3193046Z [cluster2] I1102 12:02:05.204828 1 route.go:793] Found route {Ifindex: 25 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7648. 2020-11-02T12:06:58.3194531Z [cluster2] I1102 12:02:05.204834 1 route.go:787] Processing route {Ifindex: 25 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7649. 2020-11-02T12:06:58.3195203Z [cluster2] I1102 12:02:05.204841 1 route.go:793] Found route {Ifindex: 25 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7650. 2020-11-02T12:06:58.3196482Z [cluster2] I1102 12:02:05.204847 1 route.go:787] Processing route {Ifindex: 25 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7651. 2020-11-02T12:06:58.3197224Z [cluster2] I1102 12:02:05.204853 1 route.go:793] Found route {Ifindex: 25 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7652. 2020-11-02T12:06:58.3197959Z [cluster2] I1102 12:02:05.204859 1 route.go:787] Processing route {Ifindex: 25 Dst: 240.0.0.0/8 Src: 240.17.0.11 Gw: <nil> Flags: [] Table: 254}
  7653. 2020-11-02T12:06:58.3198523Z [cluster2] I1102 12:02:05.204865 1 route.go:790] Found nil gw or dst
  7654. 2020-11-02T12:06:58.3199168Z [cluster2] I1102 12:02:05.204940 1 route.go:829] Found equivalent route, not adding
  7655. 2020-11-02T12:06:58.3199882Z [cluster2] I1102 12:02:05.204948 1 route.go:829] Found equivalent route, not adding
  7656. 2020-11-02T12:06:58.3200622Z [cluster2] I1102 12:02:05.205005 1 route.go:829] Found equivalent route, not adding
  7657. 2020-11-02T12:06:58.3201319Z [cluster2] I1102 12:02:05.205013 1 route.go:829] Found equivalent route, not adding
  7658. 2020-11-02T12:06:58.3202943Z [cluster2] I1102 12:02:05.205046 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7659. 2020-11-02T12:06:58.3205684Z [cluster2] I1102 12:02:05.205077 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7660. 2020-11-02T12:06:58.3207654Z [cluster2] I1102 12:02:35.200624 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7661. 2020-11-02T12:06:58.3209035Z [cluster2] I1102 12:02:35.200684 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7662. 2020-11-02T12:06:58.3209942Z [cluster2] I1102 12:02:35.200696 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7663. 2020-11-02T12:06:58.3210742Z [cluster2] I1102 12:02:35.201134 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7664. 2020-11-02T12:06:58.3212098Z [cluster2] I1102 12:02:35.206315 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7665. 2020-11-02T12:06:58.3212857Z [cluster2] I1102 12:02:35.206484 1 route.go:787] Processing route {Ifindex: 25 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7666. 2020-11-02T12:06:58.3213590Z [cluster2] I1102 12:02:35.206524 1 route.go:793] Found route {Ifindex: 25 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7667. 2020-11-02T12:06:58.3214316Z [cluster2] I1102 12:02:35.206534 1 route.go:787] Processing route {Ifindex: 25 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7668. 2020-11-02T12:06:58.3215330Z [cluster2] I1102 12:02:35.206542 1 route.go:793] Found route {Ifindex: 25 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7669. 2020-11-02T12:06:58.3216007Z [cluster2] I1102 12:02:35.206550 1 route.go:787] Processing route {Ifindex: 25 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7670. 2020-11-02T12:06:58.3216676Z [cluster2] I1102 12:02:35.206558 1 route.go:793] Found route {Ifindex: 25 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7671. 2020-11-02T12:06:58.3217341Z [cluster2] I1102 12:02:35.206565 1 route.go:787] Processing route {Ifindex: 25 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7672. 2020-11-02T12:06:58.3218010Z [cluster2] I1102 12:02:35.206572 1 route.go:793] Found route {Ifindex: 25 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7673. 2020-11-02T12:06:58.3218863Z [cluster2] I1102 12:02:35.206578 1 route.go:787] Processing route {Ifindex: 25 Dst: 240.0.0.0/8 Src: 240.17.0.11 Gw: <nil> Flags: [] Table: 254}
  7674. 2020-11-02T12:06:58.3219559Z [cluster2] I1102 12:02:35.206598 1 route.go:790] Found nil gw or dst
  7675. 2020-11-02T12:06:58.3220051Z [cluster2] I1102 12:02:35.206658 1 route.go:829] Found equivalent route, not adding
  7676. 2020-11-02T12:06:58.3220602Z [cluster2] I1102 12:02:35.206683 1 route.go:829] Found equivalent route, not adding
  7677. 2020-11-02T12:06:58.3221147Z [cluster2] I1102 12:02:35.206692 1 route.go:829] Found equivalent route, not adding
  7678. 2020-11-02T12:06:58.3221694Z [cluster2] I1102 12:02:35.206699 1 route.go:829] Found equivalent route, not adding
  7679. 2020-11-02T12:06:58.3223109Z [cluster2] I1102 12:02:35.206754 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7680. 2020-11-02T12:06:58.3225467Z [cluster2] I1102 12:02:35.206773 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7681. 2020-11-02T12:06:58.3228624Z [cluster2] I1102 12:02:35.206785 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7682. 2020-11-02T12:06:58.3230927Z [cluster2] I1102 12:03:05.200662 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7683. 2020-11-02T12:06:58.3234207Z [cluster2] I1102 12:03:05.200688 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7684. 2020-11-02T12:06:58.3236960Z [cluster2] I1102 12:03:05.200697 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7685. 2020-11-02T12:06:58.3237966Z [cluster2] I1102 12:03:05.200868 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7686. 2020-11-02T12:06:58.3238610Z [cluster2] I1102 12:03:05.204169 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7687. 2020-11-02T12:06:58.3239692Z [cluster2] I1102 12:03:05.204282 1 route.go:787] Processing route {Ifindex: 25 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7688. 2020-11-02T12:06:58.3240878Z [cluster2] I1102 12:03:05.204302 1 route.go:793] Found route {Ifindex: 25 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7689. 2020-11-02T12:06:58.3241724Z [cluster2] I1102 12:03:05.204309 1 route.go:787] Processing route {Ifindex: 25 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7690. 2020-11-02T12:06:58.3242588Z [cluster2] I1102 12:03:05.204317 1 route.go:793] Found route {Ifindex: 25 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7691. 2020-11-02T12:06:58.3243271Z [cluster2] I1102 12:03:05.204323 1 route.go:787] Processing route {Ifindex: 25 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7692. 2020-11-02T12:06:58.3244977Z [cluster2] I1102 12:03:05.204330 1 route.go:793] Found route {Ifindex: 25 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7693. 2020-11-02T12:06:58.3246048Z [cluster2] I1102 12:03:05.204336 1 route.go:787] Processing route {Ifindex: 25 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7694. 2020-11-02T12:06:58.3247411Z [cluster2] I1102 12:03:05.204342 1 route.go:793] Found route {Ifindex: 25 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7695. 2020-11-02T12:06:58.3248230Z [cluster2] I1102 12:03:05.204348 1 route.go:787] Processing route {Ifindex: 25 Dst: 240.0.0.0/8 Src: 240.17.0.11 Gw: <nil> Flags: [] Table: 254}
  7696. 2020-11-02T12:06:58.3248929Z [cluster2] I1102 12:03:05.204354 1 route.go:790] Found nil gw or dst
  7697. 2020-11-02T12:06:58.3249408Z [cluster2] I1102 12:03:05.204493 1 route.go:829] Found equivalent route, not adding
  7698. 2020-11-02T12:06:58.3249943Z [cluster2] I1102 12:03:05.204502 1 route.go:829] Found equivalent route, not adding
  7699. 2020-11-02T12:06:58.3250475Z [cluster2] I1102 12:03:05.204511 1 route.go:829] Found equivalent route, not adding
  7700. 2020-11-02T12:06:58.3251161Z [cluster2] I1102 12:03:05.204535 1 route.go:829] Found equivalent route, not adding
  7701. 2020-11-02T12:06:58.3253719Z [cluster2] I1102 12:03:05.204645 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7702. 2020-11-02T12:06:58.3257010Z [cluster2] I1102 12:03:05.204662 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7703. 2020-11-02T12:06:58.3259351Z [cluster2] I1102 12:03:05.204674 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7704. 2020-11-02T12:06:58.3262590Z [cluster2] I1102 12:03:35.200897 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7705. 2020-11-02T12:06:58.3264067Z [cluster2] I1102 12:03:35.200928 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7706. 2020-11-02T12:06:58.3264975Z [cluster2] I1102 12:03:35.200938 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7707. 2020-11-02T12:06:58.3265666Z [cluster2] I1102 12:03:35.201229 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7708. 2020-11-02T12:06:58.3266243Z [cluster2] I1102 12:03:35.204891 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7709. 2020-11-02T12:06:58.3266808Z [cluster2] I1102 12:03:35.205063 1 route.go:787] Processing route {Ifindex: 25 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7710. 2020-11-02T12:06:58.3267465Z [cluster2] I1102 12:03:35.205092 1 route.go:793] Found route {Ifindex: 25 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7711. 2020-11-02T12:06:58.3268749Z [cluster2] I1102 12:03:35.205101 1 route.go:787] Processing route {Ifindex: 25 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7712. 2020-11-02T12:06:58.3269460Z [cluster2] I1102 12:03:35.205109 1 route.go:793] Found route {Ifindex: 25 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7713. 2020-11-02T12:06:58.3270136Z [cluster2] I1102 12:03:35.205116 1 route.go:787] Processing route {Ifindex: 25 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7714. 2020-11-02T12:06:58.3270806Z [cluster2] I1102 12:03:35.205131 1 route.go:793] Found route {Ifindex: 25 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7715. 2020-11-02T12:06:58.3271482Z [cluster2] I1102 12:03:35.205138 1 route.go:787] Processing route {Ifindex: 25 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7716. 2020-11-02T12:06:58.3272149Z [cluster2] I1102 12:03:35.205145 1 route.go:793] Found route {Ifindex: 25 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7717. 2020-11-02T12:06:58.3272812Z [cluster2] I1102 12:03:35.205153 1 route.go:787] Processing route {Ifindex: 25 Dst: 240.0.0.0/8 Src: 240.17.0.11 Gw: <nil> Flags: [] Table: 254}
  7718. 2020-11-02T12:06:58.3273317Z [cluster2] I1102 12:03:35.205160 1 route.go:790] Found nil gw or dst
  7719. 2020-11-02T12:06:58.3273776Z [cluster2] I1102 12:03:35.205294 1 route.go:829] Found equivalent route, not adding
  7720. 2020-11-02T12:06:58.3274292Z [cluster2] I1102 12:03:35.205304 1 route.go:829] Found equivalent route, not adding
  7721. 2020-11-02T12:06:58.3274950Z [cluster2] I1102 12:03:35.205312 1 route.go:829] Found equivalent route, not adding
  7722. 2020-11-02T12:06:58.3275665Z [cluster2] I1102 12:03:35.205321 1 route.go:829] Found equivalent route, not adding
  7723. 2020-11-02T12:06:58.3276991Z [cluster2] I1102 12:03:35.205352 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7724. 2020-11-02T12:06:58.3279640Z [cluster2] I1102 12:03:35.205368 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7725. 2020-11-02T12:06:58.3282068Z [cluster2] I1102 12:03:35.205380 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7726. 2020-11-02T12:06:58.3284969Z [cluster2] I1102 12:04:05.201220 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7727. 2020-11-02T12:06:58.3286726Z [cluster2] I1102 12:04:05.201249 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7728. 2020-11-02T12:06:58.3287620Z [cluster2] I1102 12:04:05.201259 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7729. 2020-11-02T12:06:58.3288892Z [cluster2] I1102 12:04:05.201462 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7730. 2020-11-02T12:06:58.3289663Z [cluster2] I1102 12:04:05.204629 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7731. 2020-11-02T12:06:58.3290224Z [cluster2] I1102 12:04:05.204791 1 route.go:787] Processing route {Ifindex: 25 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7732. 2020-11-02T12:06:58.3290901Z [cluster2] I1102 12:04:05.204822 1 route.go:793] Found route {Ifindex: 25 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7733. 2020-11-02T12:06:58.3292054Z [cluster2] I1102 12:04:05.204831 1 route.go:787] Processing route {Ifindex: 25 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7734. 2020-11-02T12:06:58.3292908Z [cluster2] I1102 12:04:05.204843 1 route.go:793] Found route {Ifindex: 25 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7735. 2020-11-02T12:06:58.3293705Z [cluster2] I1102 12:04:05.204850 1 route.go:787] Processing route {Ifindex: 25 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7736. 2020-11-02T12:06:58.3294871Z [cluster2] I1102 12:04:05.204858 1 route.go:793] Found route {Ifindex: 25 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7737. 2020-11-02T12:06:58.3296517Z [cluster2] I1102 12:04:05.204871 1 route.go:787] Processing route {Ifindex: 25 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7738. 2020-11-02T12:06:58.3297199Z [cluster2] I1102 12:04:05.204879 1 route.go:793] Found route {Ifindex: 25 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7739. 2020-11-02T12:06:58.3297873Z [cluster2] I1102 12:04:05.204893 1 route.go:787] Processing route {Ifindex: 25 Dst: 240.0.0.0/8 Src: 240.17.0.11 Gw: <nil> Flags: [] Table: 254}
  7740. 2020-11-02T12:06:58.3298378Z [cluster2] I1102 12:04:05.204917 1 route.go:790] Found nil gw or dst
  7741. 2020-11-02T12:06:58.3298846Z [cluster2] I1102 12:04:05.204972 1 route.go:829] Found equivalent route, not adding
  7742. 2020-11-02T12:06:58.3299361Z [cluster2] I1102 12:04:05.204989 1 route.go:829] Found equivalent route, not adding
  7743. 2020-11-02T12:06:58.3299871Z [cluster2] I1102 12:04:05.204997 1 route.go:829] Found equivalent route, not adding
  7744. 2020-11-02T12:06:58.3300366Z [cluster2] I1102 12:04:05.205010 1 route.go:829] Found equivalent route, not adding
  7745. 2020-11-02T12:06:58.3302044Z [cluster2] I1102 12:04:05.205039 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7746. 2020-11-02T12:06:58.3304166Z [cluster2] I1102 12:04:05.205061 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7747. 2020-11-02T12:06:58.3306374Z [cluster2] I1102 12:04:05.205078 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7748. 2020-11-02T12:06:58.3308198Z [cluster2] I1102 12:04:35.201329 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7749. 2020-11-02T12:06:58.3310036Z [cluster2] I1102 12:04:35.201390 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7750. 2020-11-02T12:06:58.3312189Z [cluster2] I1102 12:04:35.201405 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7751. 2020-11-02T12:06:58.3313295Z [cluster2] I1102 12:04:35.201414 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7752. 2020-11-02T12:06:58.3314129Z [cluster2] I1102 12:04:35.201654 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7753. 2020-11-02T12:06:58.3314671Z [cluster2] I1102 12:04:35.205220 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7754. 2020-11-02T12:06:58.3315201Z [cluster2] I1102 12:04:35.205412 1 route.go:787] Processing route {Ifindex: 25 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7755. 2020-11-02T12:06:58.3315830Z [cluster2] I1102 12:04:35.205442 1 route.go:793] Found route {Ifindex: 25 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7756. 2020-11-02T12:06:58.3316459Z [cluster2] I1102 12:04:35.205454 1 route.go:787] Processing route {Ifindex: 25 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7757. 2020-11-02T12:06:58.3317089Z [cluster2] I1102 12:04:35.205461 1 route.go:793] Found route {Ifindex: 25 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7758. 2020-11-02T12:06:58.3317720Z [cluster2] I1102 12:04:35.205469 1 route.go:787] Processing route {Ifindex: 25 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7759. 2020-11-02T12:06:58.3318426Z [cluster2] I1102 12:04:35.205476 1 route.go:793] Found route {Ifindex: 25 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7760. 2020-11-02T12:06:58.3319064Z [cluster2] I1102 12:04:35.205483 1 route.go:787] Processing route {Ifindex: 25 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7761. 2020-11-02T12:06:58.3320429Z [cluster2] I1102 12:04:35.205503 1 route.go:793] Found route {Ifindex: 25 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7762. 2020-11-02T12:06:58.3321065Z [cluster2] I1102 12:04:35.205512 1 route.go:787] Processing route {Ifindex: 25 Dst: 240.0.0.0/8 Src: 240.17.0.11 Gw: <nil> Flags: [] Table: 254}
  7763. 2020-11-02T12:06:58.3321551Z [cluster2] I1102 12:04:35.205519 1 route.go:790] Found nil gw or dst
  7764. 2020-11-02T12:06:58.3321981Z [cluster2] I1102 12:04:35.205596 1 route.go:829] Found equivalent route, not adding
  7765. 2020-11-02T12:06:58.3322454Z [cluster2] I1102 12:04:35.205627 1 route.go:829] Found equivalent route, not adding
  7766. 2020-11-02T12:06:58.3323120Z [cluster2] I1102 12:04:35.205661 1 route.go:829] Found equivalent route, not adding
  7767. 2020-11-02T12:06:58.3323792Z [cluster2] I1102 12:04:35.205669 1 route.go:829] Found equivalent route, not adding
  7768. 2020-11-02T12:06:58.3327247Z [cluster2] I1102 12:04:35.205738 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7769. 2020-11-02T12:06:58.3329573Z [cluster2] I1102 12:04:35.205754 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7770. 2020-11-02T12:06:58.3333041Z [cluster2] I1102 12:05:05.203332 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-4 with IP 172.17.0.4, Host cluster2-worker for this cluster
  7771. 2020-11-02T12:06:58.3334599Z [cluster2] I1102 12:05:05.203368 1 route.go:612] This route agent is running on a non-gateway/non-active node
  7772. 2020-11-02T12:06:58.3336217Z [cluster2] I1102 12:05:05.203378 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.4
  7773. 2020-11-02T12:06:58.3336882Z [cluster2] I1102 12:05:05.203614 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7774. 2020-11-02T12:06:58.3337424Z [cluster2] I1102 12:05:05.248835 1 route.go:771] Reconciling routes to gw: 240.17.0.4
  7775. 2020-11-02T12:06:58.3337944Z [cluster2] I1102 12:05:05.248984 1 route.go:787] Processing route {Ifindex: 25 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7776. 2020-11-02T12:06:58.3338709Z [cluster2] I1102 12:05:05.249017 1 route.go:793] Found route {Ifindex: 25 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7777. 2020-11-02T12:06:58.3339339Z [cluster2] I1102 12:05:05.249044 1 route.go:787] Processing route {Ifindex: 25 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7778. 2020-11-02T12:06:58.3339971Z [cluster2] I1102 12:05:05.249058 1 route.go:793] Found route {Ifindex: 25 Dst: 10.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7779. 2020-11-02T12:06:58.3340598Z [cluster2] I1102 12:05:05.249065 1 route.go:787] Processing route {Ifindex: 25 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7780. 2020-11-02T12:06:58.3341592Z [cluster2] I1102 12:05:05.249090 1 route.go:793] Found route {Ifindex: 25 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7781. 2020-11-02T12:06:58.3342416Z [cluster2] I1102 12:05:05.249097 1 route.go:787] Processing route {Ifindex: 25 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254}
  7782. 2020-11-02T12:06:58.3344363Z [cluster2] I1102 12:05:05.249105 1 route.go:793] Found route {Ifindex: 25 Dst: 100.3.0.0/16 Src: <nil> Gw: 240.17.0.4 Flags: [] Table: 254} with gw 240.17.0.4 already installed
  7783. 2020-11-02T12:06:58.3345680Z [cluster2] I1102 12:05:05.249112 1 route.go:787] Processing route {Ifindex: 25 Dst: 240.0.0.0/8 Src: 240.17.0.11 Gw: <nil> Flags: [] Table: 254}
  7784. 2020-11-02T12:06:58.3346222Z [cluster2] I1102 12:05:05.249118 1 route.go:790] Found nil gw or dst
  7785. 2020-11-02T12:06:58.3346695Z [cluster2] I1102 12:05:05.249201 1 route.go:829] Found equivalent route, not adding
  7786. 2020-11-02T12:06:58.3347714Z [cluster2] I1102 12:05:05.249222 1 route.go:829] Found equivalent route, not adding
  7787. 2020-11-02T12:06:58.3348224Z [cluster2] I1102 12:05:05.249234 1 route.go:829] Found equivalent route, not adding
  7788. 2020-11-02T12:06:58.3350069Z [cluster2] I1102 12:05:05.249241 1 route.go:829] Found equivalent route, not adding
  7789. 2020-11-02T12:06:58.3351560Z [cluster2] I1102 12:05:05.249273 1 route.go:647] Successfully processed local Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4"
  7790. 2020-11-02T12:06:58.3354466Z [cluster2] I1102 12:05:05.249292 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7791. 2020-11-02T12:06:58.3361604Z [cluster2] I1102 12:05:05.249304 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7792. 2020-11-02T12:06:58.3366374Z [cluster2] I1102 12:05:30.589186 1 route.go:682] Informed of removed endpoint: {"metadata":{"name":"cluster2-submariner-cable-cluster2-172-17-0-4","namespace":"submariner-operator","selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4","uid":"8e4ce5be-6521-4f69-bef3-cde34ed638a9","resourceVersion":"3728","generation":1,"creationTimestamp":"2020-11-02T12:01:40Z"},"spec":{"cluster_id":"cluster2","cable_name":"submariner-cable-cluster2-172-17-0-4","hostname":"cluster2-worker","subnets":["100.2.0.0/16","10.2.0.0/16"],"private_ip":"172.17.0.4","public_ip":"","nat_enabled":false,"backend":"libreswan"}}
  7793. 2020-11-02T12:06:58.3370679Z [cluster2] I1102 12:05:30.589217 1 route.go:711] Local non-gateway Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with IP "172.17.0.4" was removed: deleting vxlan interface
  7794. 2020-11-02T12:06:58.3372631Z [cluster2] I1102 12:05:30.662604 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-11 with IP 172.17.0.11, Host cluster2-worker2 for this cluster
  7795. 2020-11-02T12:06:58.3373775Z [cluster2] I1102 12:05:30.662914 1 route.go:756] Processing route {Ifindex: 22 Dst: <nil> Src: <nil> Gw: 172.17.0.1 Flags: [] Table: 254}
  7796. 2020-11-02T12:06:58.3374440Z [cluster2] I1102 12:05:30.663027 1 route.go:759] Found nil gw or dst
  7797. 2020-11-02T12:06:58.3374944Z [cluster2] I1102 12:05:30.663090 1 route.go:756] Processing route {Ifindex: 4 Dst: 10.2.0.0/16 Src: 10.2.128.0 Gw: <nil> Flags: [] Table: 254}
  7798. 2020-11-02T12:06:58.3375438Z [cluster2] I1102 12:05:30.663147 1 route.go:759] Found nil gw or dst
  7799. 2020-11-02T12:06:58.3376096Z [cluster2] I1102 12:05:30.663172 1 route.go:756] Processing route {Ifindex: 22 Dst: 172.17.0.0/16 Src: 172.17.0.11 Gw: <nil> Flags: [] Table: 254}
  7800. 2020-11-02T12:06:58.3376575Z [cluster2] I1102 12:05:30.663233 1 route.go:759] Found nil gw or dst
  7801. 2020-11-02T12:06:58.3377040Z [cluster2] I1102 12:05:30.663267 1 route.go:582] This route agent is running on the active gateway node
  7802. 2020-11-02T12:06:58.3377887Z [cluster2] I1102 12:05:30.663285 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  7803. 2020-11-02T12:06:58.3378483Z [cluster2] I1102 12:05:30.663690 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.4 00:00:00:00:00:00
  7804. 2020-11-02T12:06:58.3379027Z [cluster2] I1102 12:05:30.663862 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.8 00:00:00:00:00:00
  7805. 2020-11-02T12:06:58.3379567Z [cluster2] I1102 12:05:30.663945 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.11 00:00:00:00:00:00
  7806. 2020-11-02T12:06:58.3380467Z [cluster2] I1102 12:05:30.664066 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  7807. 2020-11-02T12:06:58.3381881Z [cluster2] I1102 12:05:35.201762 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-11 with IP 172.17.0.11, Host cluster2-worker2 for this cluster
  7808. 2020-11-02T12:06:58.3382976Z [cluster2] I1102 12:05:35.201980 1 route.go:756] Processing route {Ifindex: 26 Dst: 240.0.0.0/8 Src: 240.17.0.11 Gw: <nil> Flags: [] Table: 254}
  7809. 2020-11-02T12:06:58.3383707Z [cluster2] I1102 12:05:35.202002 1 route.go:759] Found nil gw or dst
  7810. 2020-11-02T12:06:58.3384192Z [cluster2] I1102 12:05:35.202009 1 route.go:582] This route agent is running on the active gateway node
  7811. 2020-11-02T12:06:58.3385023Z [cluster2] I1102 12:05:35.202015 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  7812. 2020-11-02T12:06:58.3385689Z [cluster2] I1102 12:05:35.202172 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7813. 2020-11-02T12:06:58.3386293Z [cluster2] I1102 12:05:35.202206 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.4 00:00:00:00:00:00
  7814. 2020-11-02T12:06:58.3387002Z [cluster2] I1102 12:05:35.202235 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.8 00:00:00:00:00:00
  7815. 2020-11-02T12:06:58.3387951Z [cluster2] I1102 12:05:35.202257 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.11 00:00:00:00:00:00
  7816. 2020-11-02T12:06:58.3388830Z [cluster2] I1102 12:05:35.202318 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  7817. 2020-11-02T12:06:58.3390470Z [cluster2] I1102 12:05:35.202392 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7818. 2020-11-02T12:06:58.3393782Z [cluster2] I1102 12:05:35.202408 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7819. 2020-11-02T12:06:58.3396499Z [cluster2] I1102 12:06:05.201951 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-11 with IP 172.17.0.11, Host cluster2-worker2 for this cluster
  7820. 2020-11-02T12:06:58.3397599Z [cluster2] I1102 12:06:05.202237 1 route.go:756] Processing route {Ifindex: 26 Dst: 240.0.0.0/8 Src: 240.17.0.11 Gw: <nil> Flags: [] Table: 254}
  7821. 2020-11-02T12:06:58.3398082Z [cluster2] I1102 12:06:05.202282 1 route.go:759] Found nil gw or dst
  7822. 2020-11-02T12:06:58.3398541Z [cluster2] I1102 12:06:05.202290 1 route.go:582] This route agent is running on the active gateway node
  7823. 2020-11-02T12:06:58.3399519Z [cluster2] I1102 12:06:05.202297 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  7824. 2020-11-02T12:06:58.3400161Z [cluster2] I1102 12:06:05.202420 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7825. 2020-11-02T12:06:58.3400758Z [cluster2] I1102 12:06:05.202478 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.8 00:00:00:00:00:00
  7826. 2020-11-02T12:06:58.3401281Z [cluster2] I1102 12:06:05.202521 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.11 00:00:00:00:00:00
  7827. 2020-11-02T12:06:58.3401815Z [cluster2] I1102 12:06:05.202554 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.4 00:00:00:00:00:00
  7828. 2020-11-02T12:06:58.3403039Z [cluster2] I1102 12:06:05.202618 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  7829. 2020-11-02T12:06:58.3405243Z [cluster2] I1102 12:06:05.202681 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7830. 2020-11-02T12:06:58.3407875Z [cluster2] I1102 12:06:05.202700 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7831. 2020-11-02T12:06:58.3410075Z [cluster2] I1102 12:06:35.201910 1 route.go:570] Processing local Endpoint cluster2-submariner-cable-cluster2-172-17-0-11 with IP 172.17.0.11, Host cluster2-worker2 for this cluster
  7832. 2020-11-02T12:06:58.3411187Z [cluster2] I1102 12:06:35.202259 1 route.go:756] Processing route {Ifindex: 26 Dst: 240.0.0.0/8 Src: 240.17.0.11 Gw: <nil> Flags: [] Table: 254}
  7833. 2020-11-02T12:06:58.3411671Z [cluster2] I1102 12:06:35.202286 1 route.go:759] Found nil gw or dst
  7834. 2020-11-02T12:06:58.3412132Z [cluster2] I1102 12:06:35.202293 1 route.go:582] This route agent is running on the active gateway node
  7835. 2020-11-02T12:06:58.3413330Z [cluster2] I1102 12:06:35.202300 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  7836. 2020-11-02T12:06:58.3414012Z [cluster2] I1102 12:06:35.202466 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  7837. 2020-11-02T12:06:58.3414643Z [cluster2] I1102 12:06:35.202503 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.11 00:00:00:00:00:00
  7838. 2020-11-02T12:06:58.5187443Z [submariner]$ [cluster2] kubectl -n submariner-operator logs submariner-routeagent-ftrjn
  7839. 2020-11-02T12:06:58.5189158Z [submariner]$ [cluster2] command kubectl --context=cluster2 -n submariner-operator logs submariner-routeagent-ftrjn
  7840. 2020-11-02T12:06:58.5190471Z [submariner]$ [cluster2] kubectl --context=cluster2 -n submariner-operator logs submariner-routeagent-ftrjn
  7841. 2020-11-02T12:06:58.5191895Z [submariner]$ [cluster2] kubectl -n submariner-operator logs submariner-routeagent-swbc2
  7842. 2020-11-02T12:06:58.5193321Z [submariner]$ [cluster2] kubectl -n submariner-operator logs submariner-routeagent-swbc2
  7843. 2020-11-02T12:06:58.5194791Z [submariner]$ [cluster2] command kubectl --context=cluster2 -n submariner-operator logs submariner-routeagent-swbc2
  7844. 2020-11-02T12:06:58.5196429Z [submariner]$ [cluster2] kubectl --context=cluster2 -n submariner-operator logs submariner-routeagent-swbc2
  7845. 2020-11-02T12:06:58.5197269Z [submariner]$ [cluster2] subctl show all
  7846. 2020-11-02T12:06:58.5197731Z [cluster2] I1102 12:06:35.202533 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.4 00:00:00:00:00:00
  7847. 2020-11-02T12:06:58.5198439Z [cluster2] I1102 12:06:35.202566 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.8 00:00:00:00:00:00
  7848. 2020-11-02T12:06:58.5199471Z [cluster2] I1102 12:06:35.202625 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  7849. 2020-11-02T12:06:58.5201088Z [cluster2] I1102 12:06:35.202717 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7" with subnets [100.3.0.0/16 10.3.0.0/16]
  7850. 2020-11-02T12:06:58.5203433Z [cluster2] I1102 12:06:35.202748 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  7851. 2020-11-02T12:06:58.5204949Z [cluster2] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  7852. 2020-11-02T12:06:58.5205206Z [cluster2]
  7853. 2020-11-02T12:06:58.5205530Z [cluster2] Showing information for cluster "cluster2":
  7854. 2020-11-02T12:06:58.5206074Z [cluster2] Discovered network details:
  7855. 2020-11-02T12:06:58.5206657Z [cluster2] Network plugin: weave-net
  7856. 2020-11-02T12:06:58.5206991Z [cluster2] Service CIDRs: [100.2.0.0/16]
  7857. 2020-11-02T12:06:58.5207295Z [cluster2] Cluster CIDRs: [10.2.0.0/16]
  7858. 2020-11-02T12:06:58.5207555Z [cluster2]
  7859. 2020-11-02T12:06:58.5207912Z [cluster2] CLUSTER ID ENDPOINT IP PUBLIC IP CABLE DRIVER TYPE
  7860. 2020-11-02T12:06:58.5208548Z [cluster2] cluster2 172.17.0.11 libreswan local
  7861. 2020-11-02T12:06:58.5209189Z [cluster2] cluster3 172.17.0.7 libreswan remote
  7862. 2020-11-02T12:06:58.5209758Z [cluster2] cluster1 172.17.0.6 libreswan remote
  7863. 2020-11-02T12:06:58.5210109Z [cluster2]
  7864. 2020-11-02T12:06:58.5210513Z [cluster2] GATEWAY CLUSTER REMOTE IP CABLE DRIVER SUBNETS STATUS
  7865. 2020-11-02T12:06:58.5211567Z [cluster2] cluster3-worker cluster3 172.17.0.7 libreswan 100.3.0.0/16, 10.3.0.0/16 connected
  7866. 2020-11-02T12:06:58.5212549Z [cluster2] cluster1-worker cluster1 172.17.0.6 libreswan 100.1.0.0/16, 10.1.0.0/16 connected
  7867. 2020-11-02T12:06:58.5212967Z [cluster2]
  7868. 2020-11-02T12:06:58.5213267Z [cluster2] NODE HA STATUS SUMMARY
  7869. 2020-11-02T12:06:58.5213936Z [cluster2] cluster2-worker2 active All connections (2) are established
  7870. 2020-11-02T12:06:58.5214360Z [cluster2]
  7871. 2020-11-02T12:06:58.5214865Z [cluster2] COMPONENT REPOSITORY VERSION
  7872. 2020-11-02T12:06:58.5215303Z [cluster2] submariner localhost:5000 local
  7873. 2020-11-02T12:06:58.5215947Z [cluster2] submariner-operator localhost:5000 local
  7874. 2020-11-02T12:06:58.5216642Z [cluster2] service-discovery localhost:5000 local
  7875. 2020-11-02T12:06:58.5217012Z [cluster2]
  7876. 2020-11-02T12:06:58.5217353Z [cluster2] Showing information for cluster "cluster3":
  7877. 2020-11-02T12:06:58.5217771Z [cluster2] Discovered network details:
  7878. 2020-11-02T12:06:58.5218450Z [cluster2] Network plugin: weave-net
  7879. 2020-11-02T12:06:58.5218973Z [cluster2] Service CIDRs: [100.3.0.0/16]
  7880. 2020-11-02T12:06:58.5219295Z [cluster2] Cluster CIDRs: [10.3.0.0/16]
  7881. 2020-11-02T12:06:58.5219574Z [cluster2]
  7882. 2020-11-02T12:06:58.5219933Z [cluster2] CLUSTER ID ENDPOINT IP PUBLIC IP CABLE DRIVER TYPE
  7883. 2020-11-02T12:06:58.5220406Z [cluster2] cluster3 172.17.0.7 libreswan local
  7884. 2020-11-02T12:06:58.5220834Z [cluster2] cluster1 172.17.0.6 libreswan remote
  7885. 2020-11-02T12:06:58.5221270Z [cluster2] cluster2 172.17.0.11 libreswan remote
  7886. 2020-11-02T12:06:58.5221710Z [cluster2]
  7887. 2020-11-02T12:06:58.5222254Z [cluster2] GATEWAY CLUSTER REMOTE IP CABLE DRIVER SUBNETS STATUS
  7888. 2020-11-02T12:06:58.5223071Z [cluster2] cluster1-worker cluster1 172.17.0.6 libreswan 100.1.0.0/16, 10.1.0.0/16 connected
  7889. 2020-11-02T12:06:58.5223877Z [cluster2] cluster2-worker2 cluster2 172.17.0.11 libreswan 100.2.0.0/16, 10.2.0.0/16 connected
  7890. 2020-11-02T12:06:58.5224274Z [cluster2]
  7891. 2020-11-02T12:06:58.5224562Z [cluster2] NODE HA STATUS SUMMARY
  7892. 2020-11-02T12:06:58.5225201Z [cluster2] cluster3-worker active All connections (2) are established
  7893. 2020-11-02T12:06:58.5225585Z [cluster2]
  7894. 2020-11-02T12:06:58.5766899Z [cluster2] COMPONENT REPOSITORY VERSION
  7895. 2020-11-02T12:06:58.5767482Z [cluster2] submariner localhost:5000 local
  7896. 2020-11-02T12:06:58.5768617Z [cluster2] submariner-operator localhost:5000 local
  7897. 2020-11-02T12:06:58.5769511Z [cluster2] service-discovery localhost:5000 local
  7898. 2020-11-02T12:06:58.5770053Z [cluster2]
  7899. 2020-11-02T12:06:58.5770387Z [cluster2] Showing information for cluster "cluster1":
  7900. 2020-11-02T12:06:58.5770784Z [cluster2] Discovered network details:
  7901. 2020-11-02T12:06:58.5771221Z [cluster2] Network plugin: generic
  7902. 2020-11-02T12:06:58.5771546Z [cluster2] Service CIDRs: [100.1.0.0/16]
  7903. 2020-11-02T12:06:58.5771840Z [cluster2] Cluster CIDRs: [10.1.0.0/16]
  7904. 2020-11-02T12:06:58.5772093Z [cluster2]
  7905. 2020-11-02T12:06:58.5772438Z [cluster2] CLUSTER ID ENDPOINT IP PUBLIC IP CABLE DRIVER TYPE
  7906. 2020-11-02T12:06:58.5772865Z [cluster2] cluster1 172.17.0.6 libreswan local
  7907. 2020-11-02T12:06:58.5773260Z [cluster2] cluster3 172.17.0.7 libreswan remote
  7908. 2020-11-02T12:06:58.5773655Z [cluster2] cluster2 172.17.0.11 libreswan remote
  7909. 2020-11-02T12:06:58.5773959Z [cluster2]
  7910. 2020-11-02T12:06:58.5774326Z [cluster2] GATEWAY CLUSTER REMOTE IP CABLE DRIVER SUBNETS STATUS
  7911. 2020-11-02T12:06:58.5775169Z [cluster2] cluster3-worker cluster3 172.17.0.7 libreswan 100.3.0.0/16, 10.3.0.0/16 connected
  7912. 2020-11-02T12:06:58.5775951Z [cluster2] cluster2-worker2 cluster2 172.17.0.11 libreswan 100.2.0.0/16, 10.2.0.0/16 connected
  7913. 2020-11-02T12:06:58.5776349Z [cluster2]
  7914. 2020-11-02T12:06:58.5776630Z [cluster2] NODE HA STATUS SUMMARY
  7915. 2020-11-02T12:06:58.5777253Z [cluster2] cluster1-worker active All connections (2) are established
  7916. 2020-11-02T12:06:58.5777642Z [cluster2]
  7917. 2020-11-02T12:06:58.5777951Z [cluster2] COMPONENT REPOSITORY VERSION
  7918. 2020-11-02T12:06:58.5778373Z [cluster2] submariner localhost:5000 local
  7919. 2020-11-02T12:06:58.5779022Z [cluster2] submariner-operator localhost:5000 local
  7920. 2020-11-02T12:06:58.5779896Z [cluster2] service-discovery localhost:5000 local
  7921. 2020-11-02T12:06:58.5780519Z [cluster2] ===================== END Post mortem cluster2 =====================
  7922. 2020-11-02T12:06:58.5801974Z [submariner]$ [cluster3] post_analyze
  7923. 2020-11-02T12:06:58.5814148Z [submariner]$ [cluster3] sed s/^/[cluster3] /
  7924. 2020-11-02T12:06:58.5819780Z [submariner]$ [cluster3] post_analyze
  7925. 2020-11-02T12:06:58.5836460Z [submariner]$ [cluster3] kubectl get all --all-namespaces
  7926. 2020-11-02T12:06:58.5847467Z [submariner]$ [cluster3] kubectl get all --all-namespaces
  7927. 2020-11-02T12:06:58.5857847Z [submariner]$ [cluster3] command kubectl --context=cluster3 get all --all-namespaces
  7928. 2020-11-02T12:06:58.5867015Z [submariner]$ [cluster3] kubectl --context=cluster3 get all --all-namespaces
  7929. 2020-11-02T12:06:59.0541783Z [cluster3] ======================= Post mortem cluster3 =======================
  7930. 2020-11-02T12:06:59.0542400Z [cluster3] NAMESPACE NAME READY STATUS RESTARTS AGE
  7931. 2020-11-02T12:06:59.0543684Z [cluster3] kube-system pod/coredns-6955765f44-dqmql 1/1 Running 0 14m
  7932. 2020-11-02T12:06:59.0544776Z [cluster3] kube-system pod/coredns-6955765f44-zpsrs 1/1 Running 0 14m
  7933. 2020-11-02T12:06:59.0545651Z [cluster3] kube-system pod/etcd-cluster3-control-plane 1/1 Running 0 14m
  7934. 2020-11-02T12:06:59.0546955Z [cluster3] kube-system pod/kube-apiserver-cluster3-control-plane 1/1 Running 0 14m
  7935. 2020-11-02T12:06:59.0548571Z [cluster3] kube-system pod/kube-controller-manager-cluster3-control-plane 1/1 Running 0 14m
  7936. 2020-11-02T12:06:59.0549718Z [cluster3] kube-system pod/kube-proxy-2wq74 1/1 Running 0 14m
  7937. 2020-11-02T12:06:59.0550551Z [cluster3] kube-system pod/kube-proxy-rxshl 1/1 Running 0 14m
  7938. 2020-11-02T12:06:59.0551762Z [cluster3] kube-system pod/kube-proxy-wzr7l 1/1 Running 0 14m
  7939. 2020-11-02T12:06:59.0552967Z [cluster3] kube-system pod/kube-scheduler-cluster3-control-plane 1/1 Running 0 14m
  7940. 2020-11-02T12:06:59.0553994Z [cluster3] kube-system pod/weave-net-65xmt 2/2 Running 0 14m
  7941. 2020-11-02T12:06:59.0554925Z [cluster3] kube-system pod/weave-net-br9qq 2/2 Running 0 14m
  7942. 2020-11-02T12:06:59.0555678Z [cluster3] kube-system pod/weave-net-xbnsr 2/2 Running 0 14m
  7943. 2020-11-02T12:06:59.0556705Z [cluster3] local-path-storage pod/local-path-provisioner-7745554f7f-qdfz7 1/1 Running 0 14m
  7944. 2020-11-02T12:06:59.0557902Z [cluster3] submariner-operator pod/submariner-gateway-wmr57 1/1 Running 0 9m40s
  7945. 2020-11-02T12:06:59.0559299Z [cluster3] submariner-operator pod/submariner-lighthouse-agent-6f867f79c6-txzdd 0/1 ImagePullBackOff 0 9m37s
  7946. 2020-11-02T12:06:59.0561026Z [cluster3] submariner-operator pod/submariner-lighthouse-coredns-557485fbc-6rr6h 0/1 ImagePullBackOff 0 9m36s
  7947. 2020-11-02T12:06:59.0563074Z [cluster3] submariner-operator pod/submariner-lighthouse-coredns-557485fbc-l2c6c 0/1 ImagePullBackOff 0 9m36s
  7948. 2020-11-02T12:06:59.0565036Z [cluster3] submariner-operator pod/submariner-operator-6675977db7-kh9fb 1/1 Running 0 9m53s
  7949. 2020-11-02T12:06:59.0566247Z [cluster3] submariner-operator pod/submariner-routeagent-46pd2 1/1 Running 0 9m37s
  7950. 2020-11-02T12:06:59.0567360Z [cluster3] submariner-operator pod/submariner-routeagent-szklb 1/1 Running 0 9m38s
  7951. 2020-11-02T12:06:59.0568679Z [cluster3] submariner-operator pod/submariner-routeagent-wvgxr 1/1 Running 0 9m37s
  7952. 2020-11-02T12:06:59.0569256Z [cluster3]
  7953. 2020-11-02T12:06:59.0569891Z [cluster3] NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
  7954. 2020-11-02T12:06:59.0570479Z [cluster3] default service/kubernetes ClusterIP 100.3.0.1 <none> 443/TCP 14m
  7955. 2020-11-02T12:06:59.0571305Z [cluster3] kube-system service/kube-dns ClusterIP 100.3.0.10 <none> 53/UDP,53/TCP,9153/TCP 14m
  7956. 2020-11-02T12:06:59.0572577Z [cluster3] submariner-operator service/submariner-lighthouse-coredns ClusterIP 100.3.80.105 <none> 53/UDP 9m34s
  7957. 2020-11-02T12:06:59.0573889Z [cluster3] submariner-operator service/submariner-operator-metrics ClusterIP 100.3.33.8 <none> 8383/TCP,8686/TCP 9m41s
  7958. 2020-11-02T12:06:59.0574700Z [cluster3]
  7959. 2020-11-02T12:06:59.0575410Z [cluster3] NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
  7960. 2020-11-02T12:06:59.0576862Z [cluster3] kube-system daemonset.apps/kube-proxy 3 3 3 3 3 beta.kubernetes.io/os=linux 14m
  7961. 2020-11-02T12:06:59.0639335Z [submariner]$ [cluster3] kubectl get pods -A
  7962. 2020-11-02T12:06:59.0654638Z [submariner]$ [cluster3] tail -n +2
  7963. 2020-11-02T12:06:59.0655280Z [submariner]$ [cluster3] kubectl get pods -A
  7964. 2020-11-02T12:06:59.0668613Z [submariner]$ [cluster3] command kubectl --context=cluster3 get pods -A
  7965. 2020-11-02T12:06:59.0680395Z [submariner]$ [cluster3] grep -v Running
  7966. 2020-11-02T12:06:59.0686598Z [submariner]$ [cluster3] kubectl --context=cluster3 get pods -A
  7967. 2020-11-02T12:06:59.0708259Z [submariner]$ [cluster3] sed s/ */;/g
  7968. 2020-11-02T12:06:59.3573490Z [submariner]$ [cluster3] ns=submariner-operator
  7969. 2020-11-02T12:06:59.3592919Z [submariner]$ [cluster3] cut -f1 -d;
  7970. 2020-11-02T12:06:59.3654402Z [submariner]$ [cluster3] name=submariner-lighthouse-agent-6f867f79c6-txzdd
  7971. 2020-11-02T12:06:59.3673131Z [submariner]$ [cluster3] cut -f2 -d;
  7972. 2020-11-02T12:06:59.3699010Z [submariner]$ [cluster3] kubectl -n submariner-operator describe pod submariner-lighthouse-agent-6f867f79c6-txzdd
  7973. 2020-11-02T12:06:59.3712994Z [submariner]$ [cluster3] kubectl -n submariner-operator describe pod submariner-lighthouse-agent-6f867f79c6-txzdd
  7974. 2020-11-02T12:06:59.3729091Z [submariner]$ [cluster3] command kubectl --context=cluster3 -n submariner-operator describe pod submariner-lighthouse-agent-6f867f79c6-txzdd
  7975. 2020-11-02T12:06:59.3741825Z [submariner]$ [cluster3] kubectl --context=cluster3 -n submariner-operator describe pod submariner-lighthouse-agent-6f867f79c6-txzdd
  7976. 2020-11-02T12:06:59.6812966Z [cluster3] kube-system daemonset.apps/weave-net 3 3 3 3 3 <none> 14m
  7977. 2020-11-02T12:06:59.6814352Z [cluster3] submariner-operator daemonset.apps/submariner-gateway 1 1 1 1 1 submariner.io/gateway=true 9m40s
  7978. 2020-11-02T12:06:59.6815522Z [cluster3] submariner-operator daemonset.apps/submariner-routeagent 3 3 3 3 3 <none> 9m38s
  7979. 2020-11-02T12:06:59.6816071Z [cluster3]
  7980. 2020-11-02T12:06:59.6816980Z [cluster3] NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
  7981. 2020-11-02T12:06:59.6818029Z [cluster3] kube-system deployment.apps/coredns 2/2 2 2 14m
  7982. 2020-11-02T12:06:59.6819068Z [cluster3] local-path-storage deployment.apps/local-path-provisioner 1/1 1 1 14m
  7983. 2020-11-02T12:06:59.6820654Z [cluster3] submariner-operator deployment.apps/submariner-lighthouse-agent 0/1 1 0 9m37s
  7984. 2020-11-02T12:06:59.6821831Z [cluster3] submariner-operator deployment.apps/submariner-lighthouse-coredns 0/2 2 0 9m37s
  7985. 2020-11-02T12:06:59.6823217Z [cluster3] submariner-operator deployment.apps/submariner-operator 1/1 1 1 9m53s
  7986. 2020-11-02T12:06:59.6823889Z [cluster3]
  7987. 2020-11-02T12:06:59.6824241Z [cluster3] NAMESPACE NAME DESIRED CURRENT READY AGE
  7988. 2020-11-02T12:06:59.6824967Z [cluster3] kube-system replicaset.apps/coredns-6955765f44 2 2 2 14m
  7989. 2020-11-02T12:06:59.6826140Z [cluster3] local-path-storage replicaset.apps/local-path-provisioner-7745554f7f 1 1 1 14m
  7990. 2020-11-02T12:06:59.6827778Z [cluster3] submariner-operator replicaset.apps/submariner-lighthouse-agent-6f867f79c6 1 1 0 9m37s
  7991. 2020-11-02T12:06:59.6830896Z [cluster3] submariner-operator replicaset.apps/submariner-lighthouse-coredns-557485fbc 2 2 0 9m36s
  7992. 2020-11-02T12:06:59.6832986Z [cluster3] submariner-operator replicaset.apps/submariner-operator-6675977db7 1 1 1 9m53s
  7993. 2020-11-02T12:06:59.6834901Z [cluster3] ======================= submariner-lighthouse-agent-6f867f79c6-txzdd - submariner-operator ============================
  7994. 2020-11-02T12:06:59.6836433Z [cluster3] Name: submariner-lighthouse-agent-6f867f79c6-txzdd
  7995. 2020-11-02T12:06:59.6837481Z [cluster3] Namespace: submariner-operator
  7996. 2020-11-02T12:06:59.6837870Z [cluster3] Priority: 0
  7997. 2020-11-02T12:06:59.6838516Z [cluster3] Node: cluster3-worker2/172.17.0.5
  7998. 2020-11-02T12:06:59.6838903Z [cluster3] Start Time: Mon, 02 Nov 2020 11:57:23 +0000
  7999. 2020-11-02T12:06:59.6839508Z [cluster3] Labels: app=submariner-lighthouse-agent
  8000. 2020-11-02T12:06:59.6840185Z [cluster3] component=submariner-lighthouse
  8001. 2020-11-02T12:06:59.6840767Z [cluster3] pod-template-hash=6f867f79c6
  8002. 2020-11-02T12:06:59.6841132Z [cluster3] Annotations: <none>
  8003. 2020-11-02T12:06:59.6841424Z [cluster3] Status: Pending
  8004. 2020-11-02T12:06:59.6841700Z [cluster3] IP: 10.3.128.1
  8005. 2020-11-02T12:06:59.6841938Z [cluster3] IPs:
  8006. 2020-11-02T12:06:59.6842174Z [cluster3] IP: 10.3.128.1
  8007. 2020-11-02T12:06:59.6842954Z [cluster3] Controlled By: ReplicaSet/submariner-lighthouse-agent-6f867f79c6
  8008. 2020-11-02T12:06:59.6843782Z [cluster3] Containers:
  8009. 2020-11-02T12:06:59.6844938Z [cluster3] submariner-lighthouse-agent:
  8010. 2020-11-02T12:06:59.6845387Z [cluster3] Container ID:
  8011. 2020-11-02T12:06:59.6845923Z [cluster3] Image: localhost:5000/lighthouse-agent:local
  8012. 2020-11-02T12:06:59.6846309Z [cluster3] Image ID:
  8013. 2020-11-02T12:06:59.6846740Z [cluster3] Port: <none>
  8014. 2020-11-02T12:06:59.6847026Z [cluster3] Host Port: <none>
  8015. 2020-11-02T12:06:59.6847313Z [cluster3] State: Waiting
  8016. 2020-11-02T12:06:59.6847662Z [cluster3] Reason: ImagePullBackOff
  8017. 2020-11-02T12:06:59.6848001Z [cluster3] Ready: False
  8018. 2020-11-02T12:06:59.6848268Z [cluster3] Restart Count: 0
  8019. 2020-11-02T12:06:59.6848571Z [cluster3] Environment:
  8020. 2020-11-02T12:06:59.6849160Z [cluster3] SUBMARINER_NAMESPACE: submariner-operator
  8021. 2020-11-02T12:06:59.6849824Z [cluster3] SUBMARINER_CLUSTERID: cluster3
  8022. 2020-11-02T12:06:59.6850763Z [cluster3] SUBMARINER_EXCLUDENS: submariner,kube-system,operators
  8023. 2020-11-02T12:06:59.6851340Z [cluster3] SUBMARINER_DEBUG: false
  8024. 2020-11-02T12:06:59.6852082Z [cluster3] SUBMARINER_GLOBALNET_ENABLED: false
  8025. 2020-11-02T12:06:59.6852489Z [cluster3] BROKER_K8S_APISERVER: 172.17.0.9:6443
  8026. 2020-11-02T12:06:59.6869277Z [submariner]$ [cluster3] kubectl -n submariner-operator logs submariner-lighthouse-agent-6f867f79c6-txzdd
  8027. 2020-11-02T12:06:59.6882910Z [submariner]$ [cluster3] kubectl -n submariner-operator logs submariner-lighthouse-agent-6f867f79c6-txzdd
  8028. 2020-11-02T12:06:59.6893705Z [submariner]$ [cluster3] command kubectl --context=cluster3 -n submariner-operator logs submariner-lighthouse-agent-6f867f79c6-txzdd
  8029. 2020-11-02T12:06:59.6904018Z [submariner]$ [cluster3] kubectl --context=cluster3 -n submariner-operator logs submariner-lighthouse-agent-6f867f79c6-txzdd
  8030. 2020-11-02T12:06:59.9908670Z Error from server (BadRequest): container "submariner-lighthouse-agent" in pod "submariner-lighthouse-agent-6f867f79c6-txzdd" is waiting to start: trying and failing to pull image
  8031. 2020-11-02T12:07:00.0101850Z [cluster3] BROKER_K8S_APISERVERTOKEN: eyJhbGciOiJSUzI1NiIsImtpZCI6InhlWjlWdWVUMTJ5dnpDRDZhc09MU0M3YlJheHl4ZkNMdDhXaW40bDdrdmsifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJzdWJtYXJpbmVyLWs4cy1icm9rZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlY3JldC5uYW1lIjoiY2x1c3Rlci1jbHVzdGVyMy10b2tlbi02ajR2bSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJjbHVzdGVyLWNsdXN0ZXIzIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiMzdjNzBmY2UtMDE4OS00ODFhLWJhMjAtZTgxNGQ5OTQzMzc3Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OnN1Ym1hcmluZXItazhzLWJyb2tlcjpjbHVzdGVyLWNsdXN0ZXIzIn0.C80_nkO_0GnffIYxVbbfSZGpP8Lr_y3_EBtqtCFVupdd8jJxmBNM0KZj6YUvUG_58lC9mTEIJY85FBncSz5IWsKwwqwKIKdh0xys5UxVdYHZ2sPqhzdEkpxQTFKo38Z0wnpsIuDKOSztCnGFQnwqmkbcCTTQ6-EQ4Z6mesMI1S1C9PK5VoBCnSjunV6Ay-IcnHIOEWJOUblIHXEwedSUdcLLX9fih8IapvVVFa30-Auat4zVRVPMOuUtKMKp6I55bbpH-zfS1bZBGx_T6AtB2AEXq3VJcX1o08iOJzU3ndTLCaBJ2MMxRqpSh71fLiYOmKcagciITUUZieYdlpByhA
  8032. 2020-11-02T12:07:00.0236163Z [cluster3] BROKER_K8S_REMOTENAMESPACE: submariner-k8s-broker
  8033. 2020-11-02T12:07:00.0724891Z [cluster3] BROKER_K8S_CA: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJd01URXdNakV4TlRBMU1sb1hEVE13TVRBek1URXhOVEExTWxvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTVUwClVkaVlaV0xudTBjLzRsL1pEVUk2L2dyd3ZMakQ3T2lJczJSSjdNZnNuMXdrb2U3UE1Od2Y4YjQ2Y1MrdkJxR0oKS2xjUHhnTWpkYWtGdndSb3ZjR2JVWHBsdUlLZkNMTWxVKzdYenhuUnc4TTRwaFBndkxCRjhWRzg4WFBJTXpEcgpsT2tZQXdmY2V2L2ZVeWdxdTNBNzJqd3lQNGFodm0rRHcyN1htQ3R5cUloRmVGc1dzOXRBWDdNYk1BYk53Z09ICm82L2VUcEREQ05zc0ZiMUJhaWJrNDJEbm1SUVVkNHdnYSt1UEZPS2oxTEF5LzZaSWExVkJYckJIb2xUVkVnOWYKV3BCZEF5Um5IWis4cUxZeWVtUXQ0RHJrUXBuRy83a3JNcVB5Q2tBRTVONGppTGVvbXpVZlpEWlVCNWxEY1BLMwpuS2JibXhDZkhCQXNqNVhrUWtFQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFLeGUzc3RHbE9PaVIvdFpFUWlBcHRkNE0wOFAKVGIrTDNEVFNLZGlnQjVTNWhFakR6L3RzTHI5NUhQbEMvQXNUeW9yWDV2eGJHeHdsNHVmb0hXRzVtM0xJL3JMUgpNVmFITExVSmwzaWZ0c0R1SXVMaFZMT2pIM3NOelhmcGdFZ0o5Q1VjalhsTTlkMm4wTGlzK24rdy83OTJvREpiClRsMCtxZTdDcDR0eXdNYzN4MWJlZzN5YTFUbEdxaUV1UkRDUFZaWU5wazd3NlpReGN4dUlhRDQ1SW5YVHVqV3gKTHpCdURFSlErQlFkR2VORDc5T1k4SEdyZEYyaG5TdEdBcFMwUUtRWktNZ2ppOEo5MW5VQzNRNFZXT1B4U2Q0LwpCRENHdUJ4RzhXTTEvak5LdCs1MmJWeDEvdGF6VEdrSWhXVnd1UHl2c1IxRk5UWUpDTkRFVkhGT3A3MD0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
  8034. 2020-11-02T12:07:00.1261601Z [submariner]$ [cluster3] ns=submariner-operator
  8035. 2020-11-02T12:07:00.1262248Z [submariner]$ [cluster3] cut -f1 -d;
  8036. 2020-11-02T12:07:00.1263564Z [submariner]$ [cluster3] name=submariner-lighthouse-coredns-557485fbc-6rr6h
  8037. 2020-11-02T12:07:00.1265243Z [submariner]$ [cluster3] cut -f2 -d;
  8038. 2020-11-02T12:07:00.1271284Z [submariner]$ [cluster3] kubectl -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-6rr6h
  8039. 2020-11-02T12:07:00.1273142Z [submariner]$ [cluster3] kubectl -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-6rr6h
  8040. 2020-11-02T12:07:00.1274939Z [submariner]$ [cluster3] command kubectl --context=cluster3 -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-6rr6h
  8041. 2020-11-02T12:07:00.1277715Z [submariner]$ [cluster3] kubectl --context=cluster3 -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-6rr6h
  8042. 2020-11-02T12:07:00.1282672Z [cluster3] Mounts:
  8043. 2020-11-02T12:07:00.1284323Z [cluster3] /var/run/secrets/kubernetes.io/serviceaccount from submariner-lighthouse-token-8vtzj (ro)
  8044. 2020-11-02T12:07:00.1285220Z [cluster3] Conditions:
  8045. 2020-11-02T12:07:00.1285658Z [cluster3] Type Status
  8046. 2020-11-02T12:07:00.1285994Z [cluster3] Initialized True
  8047. 2020-11-02T12:07:00.1286443Z [cluster3] Ready False
  8048. 2020-11-02T12:07:00.1286802Z [cluster3] ContainersReady False
  8049. 2020-11-02T12:07:00.1287502Z [cluster3] PodScheduled True
  8050. 2020-11-02T12:07:00.1287811Z [cluster3] Volumes:
  8051. 2020-11-02T12:07:00.1288718Z [cluster3] submariner-lighthouse-token-8vtzj:
  8052. 2020-11-02T12:07:00.1296032Z [cluster3] Type: Secret (a volume populated by a Secret)
  8053. 2020-11-02T12:07:00.1297139Z [cluster3] SecretName: submariner-lighthouse-token-8vtzj
  8054. 2020-11-02T12:07:00.1297720Z [cluster3] Optional: false
  8055. 2020-11-02T12:07:00.1298056Z [cluster3] QoS Class: BestEffort
  8056. 2020-11-02T12:07:00.1298562Z [cluster3] Node-Selectors: <none>
  8057. 2020-11-02T12:07:00.1299403Z [cluster3] Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
  8058. 2020-11-02T12:07:00.1300062Z [cluster3] node.kubernetes.io/unreachable:NoExecute for 300s
  8059. 2020-11-02T12:07:00.1300894Z [cluster3] Events:
  8060. 2020-11-02T12:07:00.1302416Z [cluster3] Type Reason Age From Message
  8061. 2020-11-02T12:07:00.1303225Z [cluster3] ---- ------ ---- ---- -------
  8062. 2020-11-02T12:07:00.1304608Z [cluster3] Normal Scheduled 9m36s default-scheduler Successfully assigned submariner-operator/submariner-lighthouse-agent-6f867f79c6-txzdd to cluster3-worker2
  8063. 2020-11-02T12:07:00.1307324Z [cluster3] Normal Pulling 7m43s (x4 over 9m32s) kubelet, cluster3-worker2 Pulling image "localhost:5000/lighthouse-agent:local"
  8064. 2020-11-02T12:07:00.1310325Z [cluster3] Warning Failed 7m43s (x4 over 9m3s) kubelet, cluster3-worker2 Failed to pull image "localhost:5000/lighthouse-agent:local": rpc error: code = Unknown desc = failed to pull and unpack image "localhost:5000/lighthouse-agent:local": failed to resolve reference "localhost:5000/lighthouse-agent:local": failed to do request: Head https://localhost:5000/v2/lighthouse-agent/manifests/local: dial tcp 127.0.0.1:5000: connect: connection refused
  8065. 2020-11-02T12:07:00.1312182Z [cluster3] Warning Failed 7m43s (x4 over 9m3s) kubelet, cluster3-worker2 Error: ErrImagePull
  8066. 2020-11-02T12:07:00.1313079Z [cluster3] Warning Failed 7m4s (x7 over 9m2s) kubelet, cluster3-worker2 Error: ImagePullBackOff
  8067. 2020-11-02T12:07:00.1314137Z [cluster3] Normal BackOff 4m24s (x18 over 9m2s) kubelet, cluster3-worker2 Back-off pulling image "localhost:5000/lighthouse-agent:local"
  8068. 2020-11-02T12:07:00.3501155Z [submariner]$ [cluster3] kubectl -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-6rr6h
  8069. 2020-11-02T12:07:00.3511567Z [submariner]$ [cluster3] kubectl -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-6rr6h
  8070. 2020-11-02T12:07:00.3527201Z [submariner]$ [cluster3] command kubectl --context=cluster3 -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-6rr6h
  8071. 2020-11-02T12:07:00.3540522Z [submariner]$ [cluster3] kubectl --context=cluster3 -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-6rr6h
  8072. 2020-11-02T12:07:00.6408611Z Error from server (BadRequest): container "submariner-lighthouse-coredns" in pod "submariner-lighthouse-coredns-557485fbc-6rr6h" is waiting to start: trying and failing to pull image
  8073. 2020-11-02T12:07:00.6482431Z [submariner]$ [cluster3] ns=submariner-operator
  8074. 2020-11-02T12:07:00.6498004Z [submariner]$ [cluster3] cut -f1 -d;
  8075. 2020-11-02T12:07:00.6545215Z [submariner]$ [cluster3] name=submariner-lighthouse-coredns-557485fbc-l2c6c
  8076. 2020-11-02T12:07:00.6559517Z [submariner]$ [cluster3] cut -f2 -d;
  8077. 2020-11-02T12:07:00.6587602Z [submariner]$ [cluster3] kubectl -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-l2c6c
  8078. 2020-11-02T12:07:00.6604785Z [submariner]$ [cluster3] kubectl -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-l2c6c
  8079. 2020-11-02T12:07:00.6629548Z [submariner]$ [cluster3] command kubectl --context=cluster3 -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-l2c6c
  8080. 2020-11-02T12:07:00.6641969Z [submariner]$ [cluster3] kubectl --context=cluster3 -n submariner-operator describe pod submariner-lighthouse-coredns-557485fbc-l2c6c
  8081. 2020-11-02T12:07:00.9882141Z [cluster3] ===================== END submariner-lighthouse-agent-6f867f79c6-txzdd - submariner-operator ==========================
  8082. 2020-11-02T12:07:00.9885287Z [cluster3] ======================= submariner-lighthouse-coredns-557485fbc-6rr6h - submariner-operator ============================
  8083. 2020-11-02T12:07:00.9887228Z [cluster3] Name: submariner-lighthouse-coredns-557485fbc-6rr6h
  8084. 2020-11-02T12:07:00.9888541Z [cluster3] Namespace: submariner-operator
  8085. 2020-11-02T12:07:00.9889573Z [cluster3] Priority: 0
  8086. 2020-11-02T12:07:00.9890764Z [cluster3] Node: cluster3-worker2/172.17.0.5
  8087. 2020-11-02T12:07:00.9891408Z [cluster3] Start Time: Mon, 02 Nov 2020 11:57:24 +0000
  8088. 2020-11-02T12:07:00.9893146Z [cluster3] Labels: app=submariner-lighthouse-coredns
  8089. 2020-11-02T12:07:00.9894423Z [cluster3] component=submariner-lighthouse
  8090. 2020-11-02T12:07:00.9895357Z [cluster3] pod-template-hash=557485fbc
  8091. 2020-11-02T12:07:00.9895969Z [cluster3] Annotations: <none>
  8092. 2020-11-02T12:07:00.9896488Z [cluster3] Status: Pending
  8093. 2020-11-02T12:07:00.9897076Z [cluster3] IP: 10.3.128.2
  8094. 2020-11-02T12:07:00.9897521Z [cluster3] IPs:
  8095. 2020-11-02T12:07:00.9897964Z [cluster3] IP: 10.3.128.2
  8096. 2020-11-02T12:07:00.9899139Z [cluster3] Controlled By: ReplicaSet/submariner-lighthouse-coredns-557485fbc
  8097. 2020-11-02T12:07:00.9900213Z [cluster3] Containers:
  8098. 2020-11-02T12:07:00.9901178Z [cluster3] submariner-lighthouse-coredns:
  8099. 2020-11-02T12:07:00.9901959Z [cluster3] Container ID:
  8100. 2020-11-02T12:07:00.9902850Z [cluster3] Image: localhost:5000/lighthouse-coredns:local
  8101. 2020-11-02T12:07:00.9903522Z [cluster3] Image ID:
  8102. 2020-11-02T12:07:00.9904002Z [cluster3] Port: <none>
  8103. 2020-11-02T12:07:00.9904490Z [cluster3] Host Port: <none>
  8104. 2020-11-02T12:07:00.9905116Z [cluster3] Args:
  8105. 2020-11-02T12:07:00.9905765Z [cluster3] -conf
  8106. 2020-11-02T12:07:00.9906231Z [cluster3] /etc/coredns/Corefile
  8107. 2020-11-02T12:07:00.9906769Z [cluster3] State: Waiting
  8108. 2020-11-02T12:07:00.9907344Z [cluster3] Reason: ImagePullBackOff
  8109. 2020-11-02T12:07:00.9907892Z [cluster3] Ready: False
  8110. 2020-11-02T12:07:01.0056143Z [cluster3] Restart Count: 0
  8111. 2020-11-02T12:07:01.0059527Z [cluster3] Environment: <none>
  8112. 2020-11-02T12:07:01.0068451Z [cluster3] Mounts:
  8113. 2020-11-02T12:07:01.0069536Z [cluster3] /etc/coredns from config-volume (ro)
  8114. 2020-11-02T12:07:01.0070661Z [cluster3] /var/run/secrets/kubernetes.io/serviceaccount from submariner-lighthouse-token-8vtzj (ro)
  8115. 2020-11-02T12:07:01.0071487Z [cluster3] Conditions:
  8116. 2020-11-02T12:07:01.0072046Z [cluster3] Type Status
  8117. 2020-11-02T12:07:01.0072575Z [cluster3] Initialized True
  8118. 2020-11-02T12:07:01.0073215Z [cluster3] Ready False
  8119. 2020-11-02T12:07:01.0074029Z [cluster3] ContainersReady False
  8120. 2020-11-02T12:07:01.0074673Z [cluster3] PodScheduled True
  8121. 2020-11-02T12:07:01.0075146Z [cluster3] Volumes:
  8122. 2020-11-02T12:07:01.0075781Z [cluster3] config-volume:
  8123. 2020-11-02T12:07:01.0090242Z [cluster3] Type: ConfigMap (a volume populated by a ConfigMap)
  8124. 2020-11-02T12:07:01.0091525Z [cluster3] Name: submariner-lighthouse-coredns
  8125. 2020-11-02T12:07:01.0092297Z [cluster3] Optional: false
  8126. 2020-11-02T12:07:01.0093240Z [cluster3] submariner-lighthouse-token-8vtzj:
  8127. 2020-11-02T12:07:01.0094082Z [cluster3] Type: Secret (a volume populated by a Secret)
  8128. 2020-11-02T12:07:01.0113650Z [cluster3] SecretName: submariner-lighthouse-token-8vtzj
  8129. 2020-11-02T12:07:01.0117892Z [submariner]$ [cluster3] kubectl -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-l2c6c
  8130. 2020-11-02T12:07:01.0120880Z [submariner]$ [cluster3] kubectl -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-l2c6c
  8131. 2020-11-02T12:07:01.0124570Z [submariner]$ [cluster3] command kubectl --context=cluster3 -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-l2c6c
  8132. 2020-11-02T12:07:01.0127642Z [submariner]$ [cluster3] kubectl --context=cluster3 -n submariner-operator logs submariner-lighthouse-coredns-557485fbc-l2c6c
  8133. 2020-11-02T12:07:01.0128805Z [cluster3] Optional: false
  8134. 2020-11-02T12:07:01.0129463Z [cluster3] QoS Class: BestEffort
  8135. 2020-11-02T12:07:01.0130295Z [cluster3] Node-Selectors: <none>
  8136. 2020-11-02T12:07:01.0131257Z [cluster3] Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
  8137. 2020-11-02T12:07:01.0132355Z [cluster3] node.kubernetes.io/unreachable:NoExecute for 300s
  8138. 2020-11-02T12:07:01.0133026Z [cluster3] Events:
  8139. 2020-11-02T12:07:01.0133520Z [cluster3] Type Reason Age From Message
  8140. 2020-11-02T12:07:01.0134290Z [cluster3] ---- ------ ---- ---- -------
  8141. 2020-11-02T12:07:01.0136943Z [cluster3] Normal Scheduled 9m36s default-scheduler Successfully assigned submariner-operator/submariner-lighthouse-coredns-557485fbc-6rr6h to cluster3-worker2
  8142. 2020-11-02T12:07:01.0138849Z [cluster3] Normal Pulling 7m29s (x4 over 9m33s) kubelet, cluster3-worker2 Pulling image "localhost:5000/lighthouse-coredns:local"
  8143. 2020-11-02T12:07:01.0141629Z [cluster3] Warning Failed 7m29s (x4 over 9m3s) kubelet, cluster3-worker2 Failed to pull image "localhost:5000/lighthouse-coredns:local": rpc error: code = Unknown desc = failed to pull and unpack image "localhost:5000/lighthouse-coredns:local": failed to resolve reference "localhost:5000/lighthouse-coredns:local": failed to do request: Head https://localhost:5000/v2/lighthouse-coredns/manifests/local: dial tcp 127.0.0.1:5000: connect: connection refused
  8144. 2020-11-02T12:07:01.0144330Z [cluster3] Warning Failed 7m29s (x4 over 9m3s) kubelet, cluster3-worker2 Error: ErrImagePull
  8145. 2020-11-02T12:07:01.0145773Z [cluster3] Normal BackOff 7m17s (x6 over 9m3s) kubelet, cluster3-worker2 Back-off pulling image "localhost:5000/lighthouse-coredns:local"
  8146. 2020-11-02T12:07:01.0147310Z [cluster3] Warning Failed 4m25s (x18 over 9m3s) kubelet, cluster3-worker2 Error: ImagePullBackOff
  8147. 2020-11-02T12:07:01.0148771Z [cluster3] ===================== END submariner-lighthouse-coredns-557485fbc-6rr6h - submariner-operator ==========================
  8148. 2020-11-02T12:07:01.0150845Z [cluster3] ======================= submariner-lighthouse-coredns-557485fbc-l2c6c - submariner-operator ============================
  8149. 2020-11-02T12:07:01.3084362Z Error from server (BadRequest): container "submariner-lighthouse-coredns" in pod "submariner-lighthouse-coredns-557485fbc-l2c6c" is waiting to start: trying and failing to pull image
  8150. 2020-11-02T12:07:01.3137769Z [submariner]$ [cluster3] namespace=kube-system
  8151. 2020-11-02T12:07:01.3154564Z [submariner]$ [cluster3] kubectl get pods --selector=k8s-app=kube-proxy -n kube-system -o jsonpath={.items[*].metadata.name}
  8152. 2020-11-02T12:07:01.3167264Z [submariner]$ [cluster3] kubectl get pods --selector=k8s-app=kube-proxy -n kube-system -o jsonpath={.items[*].metadata.name}
  8153. 2020-11-02T12:07:01.3180516Z [submariner]$ [cluster3] command kubectl --context=cluster3 get pods --selector=k8s-app=kube-proxy -n kube-system -o jsonpath={.items[*].metadata.name}
  8154. 2020-11-02T12:07:01.3195039Z [submariner]$ [cluster3] kubectl --context=cluster3 get pods --selector=k8s-app=kube-proxy -n kube-system -o jsonpath={.items[*].metadata.name}
  8155. 2020-11-02T12:07:01.6371397Z [submariner]$ [cluster3] kubectl -n kube-system logs kube-proxy-2wq74
  8156. 2020-11-02T12:07:01.6378970Z [submariner]$ [cluster3] kubectl -n kube-system logs kube-proxy-2wq74
  8157. 2020-11-02T12:07:01.6393833Z [submariner]$ [cluster3] command kubectl --context=cluster3 -n kube-system logs kube-proxy-2wq74
  8158. 2020-11-02T12:07:01.6406924Z [submariner]$ [cluster3] kubectl --context=cluster3 -n kube-system logs kube-proxy-2wq74
  8159. 2020-11-02T12:07:01.9574229Z [cluster3] Name: submariner-lighthouse-coredns-557485fbc-l2c6c
  8160. 2020-11-02T12:07:01.9575432Z [cluster3] Namespace: submariner-operator
  8161. 2020-11-02T12:07:01.9575833Z [cluster3] Priority: 0
  8162. 2020-11-02T12:07:01.9576323Z [cluster3] Node: cluster3-worker/172.17.0.7
  8163. 2020-11-02T12:07:01.9576728Z [cluster3] Start Time: Mon, 02 Nov 2020 11:57:25 +0000
  8164. 2020-11-02T12:07:01.9577861Z [cluster3] Labels: app=submariner-lighthouse-coredns
  8165. 2020-11-02T12:07:01.9578616Z [cluster3] component=submariner-lighthouse
  8166. 2020-11-02T12:07:01.9579491Z [cluster3] pod-template-hash=557485fbc
  8167. 2020-11-02T12:07:01.9580084Z [cluster3] Annotations: <none>
  8168. 2020-11-02T12:07:01.9580923Z [cluster3] Status: Pending
  8169. 2020-11-02T12:07:01.9581636Z [cluster3] IP: 10.3.192.2
  8170. 2020-11-02T12:07:01.9582356Z [cluster3] IPs:
  8171. 2020-11-02T12:07:01.9583085Z [cluster3] IP: 10.3.192.2
  8172. 2020-11-02T12:07:01.9584395Z [cluster3] Controlled By: ReplicaSet/submariner-lighthouse-coredns-557485fbc
  8173. 2020-11-02T12:07:01.9585077Z [cluster3] Containers:
  8174. 2020-11-02T12:07:01.9585688Z [cluster3] submariner-lighthouse-coredns:
  8175. 2020-11-02T12:07:01.9586171Z [cluster3] Container ID:
  8176. 2020-11-02T12:07:01.9586760Z [cluster3] Image: localhost:5000/lighthouse-coredns:local
  8177. 2020-11-02T12:07:01.9587164Z [cluster3] Image ID:
  8178. 2020-11-02T12:07:01.9587438Z [cluster3] Port: <none>
  8179. 2020-11-02T12:07:01.9587733Z [cluster3] Host Port: <none>
  8180. 2020-11-02T12:07:01.9588305Z [cluster3] Args:
  8181. 2020-11-02T12:07:01.9589166Z [cluster3] -conf
  8182. 2020-11-02T12:07:01.9589930Z [cluster3] /etc/coredns/Corefile
  8183. 2020-11-02T12:07:01.9590987Z [cluster3] State: Waiting
  8184. 2020-11-02T12:07:01.9591772Z [cluster3] Reason: ImagePullBackOff
  8185. 2020-11-02T12:07:01.9592610Z [cluster3] Ready: False
  8186. 2020-11-02T12:07:01.9593367Z [cluster3] Restart Count: 0
  8187. 2020-11-02T12:07:01.9594248Z [cluster3] Environment: <none>
  8188. 2020-11-02T12:07:01.9595024Z [cluster3] Mounts:
  8189. 2020-11-02T12:07:01.9596174Z [cluster3] /etc/coredns from config-volume (ro)
  8190. 2020-11-02T12:07:01.9597382Z [cluster3] /var/run/secrets/kubernetes.io/serviceaccount from submariner-lighthouse-token-8vtzj (ro)
  8191. 2020-11-02T12:07:01.9598101Z [cluster3] Conditions:
  8192. 2020-11-02T12:07:01.9598402Z [cluster3] Type Status
  8193. 2020-11-02T12:07:01.9598721Z [cluster3] Initialized True
  8194. 2020-11-02T12:07:01.9599024Z [cluster3] Ready False
  8195. 2020-11-02T12:07:01.9599366Z [cluster3] ContainersReady False
  8196. 2020-11-02T12:07:01.9599984Z [cluster3] PodScheduled True
  8197. 2020-11-02T12:07:01.9601147Z [cluster3] Volumes:
  8198. 2020-11-02T12:07:01.9601662Z [cluster3] config-volume:
  8199. 2020-11-02T12:07:01.9602100Z [cluster3] Type: ConfigMap (a volume populated by a ConfigMap)
  8200. 2020-11-02T12:07:01.9603345Z [cluster3] Name: submariner-lighthouse-coredns
  8201. 2020-11-02T12:07:01.9604446Z [cluster3] Optional: false
  8202. 2020-11-02T12:07:01.9605151Z [cluster3] submariner-lighthouse-token-8vtzj:
  8203. 2020-11-02T12:07:01.9606072Z [cluster3] Type: Secret (a volume populated by a Secret)
  8204. 2020-11-02T12:07:01.9607768Z [cluster3] SecretName: submariner-lighthouse-token-8vtzj
  8205. 2020-11-02T12:07:01.9608341Z [cluster3] Optional: false
  8206. 2020-11-02T12:07:01.9608677Z [cluster3] QoS Class: BestEffort
  8207. 2020-11-02T12:07:01.9609172Z [cluster3] Node-Selectors: <none>
  8208. 2020-11-02T12:07:01.9610371Z [cluster3] Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
  8209. 2020-11-02T12:07:01.9611511Z [cluster3] node.kubernetes.io/unreachable:NoExecute for 300s
  8210. 2020-11-02T12:07:01.9612161Z [cluster3] Events:
  8211. 2020-11-02T12:07:01.9612519Z [cluster3] Type Reason Age From Message
  8212. 2020-11-02T12:07:01.9613081Z [cluster3] ---- ------ ---- ---- -------
  8213. 2020-11-02T12:07:01.9614780Z [cluster3] Normal Scheduled 9m35s default-scheduler Successfully assigned submariner-operator/submariner-lighthouse-coredns-557485fbc-l2c6c to cluster3-worker
  8214. 2020-11-02T12:07:01.9616549Z [cluster3] Normal Pulling 7m17s (x4 over 9m33s) kubelet, cluster3-worker Pulling image "localhost:5000/lighthouse-coredns:local"
  8215. 2020-11-02T12:07:01.9618961Z [cluster3] Warning Failed 7m17s (x4 over 8m43s) kubelet, cluster3-worker Failed to pull image "localhost:5000/lighthouse-coredns:local": rpc error: code = Unknown desc = failed to pull and unpack image "localhost:5000/lighthouse-coredns:local": failed to resolve reference "localhost:5000/lighthouse-coredns:local": failed to do request: Head https://localhost:5000/v2/lighthouse-coredns/manifests/local: dial tcp 127.0.0.1:5000: connect: connection refused
  8216. 2020-11-02T12:07:01.9621125Z [cluster3] Warning Failed 7m17s (x4 over 8m43s) kubelet, cluster3-worker Error: ErrImagePull
  8217. 2020-11-02T12:07:01.9622045Z [cluster3] Warning Failed 6m48s (x7 over 8m43s) kubelet, cluster3-worker Error: ImagePullBackOff
  8218. 2020-11-02T12:07:01.9623497Z [cluster3] Normal BackOff 4m23s (x17 over 8m43s) kubelet, cluster3-worker Back-off pulling image "localhost:5000/lighthouse-coredns:local"
  8219. 2020-11-02T12:07:01.9624898Z [cluster3] ===================== END submariner-lighthouse-coredns-557485fbc-l2c6c - submariner-operator ==========================
  8220. 2020-11-02T12:07:01.9626078Z [cluster3] +++++++++++++++++++++: Logs for Pod kube-proxy-2wq74 in namespace kube-system :++++++++++++++++++++++
  8221. 2020-11-02T12:07:01.9626694Z [cluster3] W1102 11:52:33.935040 1 server_others.go:323] Unknown proxy mode "", assuming iptables proxy
  8222. 2020-11-02T12:07:01.9627405Z [cluster3] I1102 11:52:33.944191 1 node.go:135] Successfully retrieved node IP: 172.17.0.10
  8223. 2020-11-02T12:07:01.9652162Z [submariner]$ [cluster3] kubectl -n kube-system logs kube-proxy-rxshl
  8224. 2020-11-02T12:07:01.9664555Z [submariner]$ [cluster3] kubectl -n kube-system logs kube-proxy-rxshl
  8225. 2020-11-02T12:07:01.9674492Z [submariner]$ [cluster3] command kubectl --context=cluster3 -n kube-system logs kube-proxy-rxshl
  8226. 2020-11-02T12:07:01.9684577Z [submariner]$ [cluster3] kubectl --context=cluster3 -n kube-system logs kube-proxy-rxshl
  8227. 2020-11-02T12:07:02.2788340Z [submariner]$ [cluster3] kubectl -n kube-system logs kube-proxy-wzr7l
  8228. 2020-11-02T12:07:02.2797090Z [submariner]$ [cluster3] kubectl -n kube-system logs kube-proxy-wzr7l
  8229. 2020-11-02T12:07:02.2806502Z [submariner]$ [cluster3] command kubectl --context=cluster3 -n kube-system logs kube-proxy-wzr7l
  8230. 2020-11-02T12:07:02.2817506Z [submariner]$ [cluster3] kubectl --context=cluster3 -n kube-system logs kube-proxy-wzr7l
  8231. 2020-11-02T12:07:02.5841255Z [submariner]$ [cluster3] namespace=submariner-operator
  8232. 2020-11-02T12:07:02.5860302Z [submariner]$ [cluster3] kubectl get pods --selector=name=submariner-operator -n submariner-operator -o jsonpath={.items[*].metadata.name}
  8233. 2020-11-02T12:07:02.5871092Z [submariner]$ [cluster3] kubectl get pods --selector=name=submariner-operator -n submariner-operator -o jsonpath={.items[*].metadata.name}
  8234. 2020-11-02T12:07:02.5882258Z [submariner]$ [cluster3] command kubectl --context=cluster3 get pods --selector=name=submariner-operator -n submariner-operator -o jsonpath={.items[*].metadata.name}
  8235. 2020-11-02T12:07:02.5895163Z [submariner]$ [cluster3] kubectl --context=cluster3 get pods --selector=name=submariner-operator -n submariner-operator -o jsonpath={.items[*].metadata.name}
  8236. 2020-11-02T12:07:02.8769656Z [cluster3] I1102 11:52:33.944219 1 server_others.go:145] Using iptables Proxier.
  8237. 2020-11-02T12:07:02.8770147Z [cluster3] I1102 11:52:33.945433 1 server.go:571] Version: v1.17.0
  8238. 2020-11-02T12:07:02.8770594Z [cluster3] I1102 11:52:33.945826 1 conntrack.go:52] Setting nf_conntrack_max to 131072
  8239. 2020-11-02T12:07:02.8771817Z [cluster3] I1102 11:52:33.952425 1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_established' to 86400
  8240. 2020-11-02T12:07:02.8772723Z [cluster3] I1102 11:52:33.952555 1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_close_wait' to 3600
  8241. 2020-11-02T12:07:02.8773346Z [cluster3] I1102 11:52:33.953369 1 config.go:131] Starting endpoints config controller
  8242. 2020-11-02T12:07:02.8773902Z [cluster3] I1102 11:52:33.953395 1 shared_informer.go:197] Waiting for caches to sync for endpoints config
  8243. 2020-11-02T12:07:02.8774460Z [cluster3] I1102 11:52:33.954516 1 config.go:313] Starting service config controller
  8244. 2020-11-02T12:07:02.8775452Z [cluster3] I1102 11:52:33.954558 1 shared_informer.go:197] Waiting for caches to sync for service config
  8245. 2020-11-02T12:07:02.8776049Z [cluster3] I1102 11:52:34.054060 1 shared_informer.go:204] Caches are synced for endpoints config
  8246. 2020-11-02T12:07:02.8776608Z [cluster3] I1102 11:52:34.054672 1 shared_informer.go:204] Caches are synced for service config
  8247. 2020-11-02T12:07:02.8777589Z [cluster3] +++++++++++++++++++++: Logs for Pod kube-proxy-rxshl in namespace kube-system :++++++++++++++++++++++
  8248. 2020-11-02T12:07:02.8778184Z [cluster3] W1102 11:53:02.167036 1 server_others.go:323] Unknown proxy mode "", assuming iptables proxy
  8249. 2020-11-02T12:07:02.8778728Z [cluster3] I1102 11:53:02.190861 1 node.go:135] Successfully retrieved node IP: 172.17.0.7
  8250. 2020-11-02T12:07:02.8779450Z [cluster3] I1102 11:53:02.190905 1 server_others.go:145] Using iptables Proxier.
  8251. 2020-11-02T12:07:02.8779874Z [cluster3] I1102 11:53:02.191120 1 server.go:571] Version: v1.17.0
  8252. 2020-11-02T12:07:02.8780304Z [cluster3] I1102 11:53:02.191541 1 conntrack.go:52] Setting nf_conntrack_max to 131072
  8253. 2020-11-02T12:07:02.8781123Z [cluster3] I1102 11:53:02.191678 1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_established' to 86400
  8254. 2020-11-02T12:07:02.8781972Z [cluster3] I1102 11:53:02.191740 1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_close_wait' to 3600
  8255. 2020-11-02T12:07:02.8782548Z [cluster3] I1102 11:53:02.192003 1 config.go:313] Starting service config controller
  8256. 2020-11-02T12:07:02.8783082Z [cluster3] I1102 11:53:02.192010 1 shared_informer.go:197] Waiting for caches to sync for service config
  8257. 2020-11-02T12:07:02.8783630Z [cluster3] I1102 11:53:02.193926 1 config.go:131] Starting endpoints config controller
  8258. 2020-11-02T12:07:02.8784173Z [cluster3] I1102 11:53:02.193939 1 shared_informer.go:197] Waiting for caches to sync for endpoints config
  8259. 2020-11-02T12:07:02.8784732Z [cluster3] I1102 11:53:02.293271 1 shared_informer.go:204] Caches are synced for service config
  8260. 2020-11-02T12:07:02.8785483Z [cluster3] I1102 11:53:02.294074 1 shared_informer.go:204] Caches are synced for endpoints config
  8261. 2020-11-02T12:07:02.8786456Z [cluster3] +++++++++++++++++++++: Logs for Pod kube-proxy-wzr7l in namespace kube-system :++++++++++++++++++++++
  8262. 2020-11-02T12:07:02.8787252Z [cluster3] W1102 11:52:59.783711 1 server_others.go:323] Unknown proxy mode "", assuming iptables proxy
  8263. 2020-11-02T12:07:02.8787821Z [cluster3] I1102 11:52:59.860483 1 node.go:135] Successfully retrieved node IP: 172.17.0.5
  8264. 2020-11-02T12:07:02.8788682Z [cluster3] I1102 11:52:59.860679 1 server_others.go:145] Using iptables Proxier.
  8265. 2020-11-02T12:07:02.8790053Z [cluster3] I1102 11:52:59.861030 1 server.go:571] Version: v1.17.0
  8266. 2020-11-02T12:07:02.8790484Z [cluster3] I1102 11:52:59.861681 1 conntrack.go:52] Setting nf_conntrack_max to 131072
  8267. 2020-11-02T12:07:02.8791379Z [cluster3] I1102 11:52:59.863881 1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_established' to 86400
  8268. 2020-11-02T12:07:02.8792481Z [cluster3] I1102 11:52:59.864112 1 conntrack.go:100] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_close_wait' to 3600
  8269. 2020-11-02T12:07:02.8793247Z [cluster3] I1102 11:52:59.884304 1 config.go:313] Starting service config controller
  8270. 2020-11-02T12:07:02.8793795Z [cluster3] I1102 11:52:59.916521 1 shared_informer.go:197] Waiting for caches to sync for service config
  8271. 2020-11-02T12:07:02.8794526Z [cluster3] I1102 11:52:59.890645 1 config.go:131] Starting endpoints config controller
  8272. 2020-11-02T12:07:02.8795286Z [cluster3] I1102 11:52:59.917967 1 shared_informer.go:204] Caches are synced for service config
  8273. 2020-11-02T12:07:02.8795864Z [cluster3] I1102 11:52:59.918099 1 shared_informer.go:197] Waiting for caches to sync for endpoints config
  8274. 2020-11-02T12:07:02.8796448Z [cluster3] I1102 11:52:59.918366 1 shared_informer.go:204] Caches are synced for endpoints config
  8275. 2020-11-02T12:07:02.8796953Z [cluster3] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  8276. 2020-11-02T12:07:02.8906001Z [submariner]$ [cluster3] kubectl -n submariner-operator logs submariner-operator-6675977db7-kh9fb
  8277. 2020-11-02T12:07:02.8930008Z [submariner]$ [cluster3] kubectl -n submariner-operator logs submariner-operator-6675977db7-kh9fb
  8278. 2020-11-02T12:07:02.8947962Z [submariner]$ [cluster3] command kubectl --context=cluster3 -n submariner-operator logs submariner-operator-6675977db7-kh9fb
  8279. 2020-11-02T12:07:02.8999125Z [submariner]$ [cluster3] kubectl --context=cluster3 -n submariner-operator logs submariner-operator-6675977db7-kh9fb
  8280. 2020-11-02T12:07:03.3219961Z [cluster3] +++++++++++++++++++++: Logs for Pod submariner-operator-6675977db7-kh9fb in namespace submariner-operator :++++++++++++++++++++++
  8281. 2020-11-02T12:07:03.3221172Z [cluster3] {"level":"info","ts":1604318231.7320597,"logger":"cmd","msg":"Go Version: go1.15.2"}
  8282. 2020-11-02T12:07:03.3221687Z [cluster3] {"level":"info","ts":1604318231.7323356,"logger":"cmd","msg":"Go OS/Arch: linux/amd64"}
  8283. 2020-11-02T12:07:03.3222438Z [cluster3] {"level":"info","ts":1604318231.7326367,"logger":"cmd","msg":"Version of operator-sdk: v0.17.0"}
  8284. 2020-11-02T12:07:03.3223068Z [cluster3] {"level":"info","ts":1604318231.732775,"logger":"cmd","msg":"Submariner operator version: was not built correctly"}
  8285. 2020-11-02T12:07:03.3223867Z [cluster3] {"level":"info","ts":1604318231.7336745,"logger":"leader","msg":"Trying to become the leader."}
  8286. 2020-11-02T12:07:03.3224642Z [cluster3] {"level":"info","ts":1604318232.066588,"logger":"leader","msg":"No pre-existing lock was found."}
  8287. 2020-11-02T12:07:03.3225216Z [cluster3] {"level":"info","ts":1604318232.0713959,"logger":"leader","msg":"Became the leader."}
  8288. 2020-11-02T12:07:03.3226194Z [cluster3] {"level":"info","ts":1604318232.3752158,"logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":"0.0.0.0:8383"}
  8289. 2020-11-02T12:07:03.3226968Z [cluster3] {"level":"info","ts":1604318232.3759744,"logger":"cmd","msg":"Registering Components."}
  8290. 2020-11-02T12:07:03.3227524Z [cluster3] {"level":"info","ts":1604318232.3761156,"logger":"cmd","msg":"Creating the engine CRDs"}
  8291. 2020-11-02T12:07:03.3228055Z [cluster3] {"level":"info","ts":1604318232.5438848,"logger":"cmd","msg":"Creating the Lighthouse CRDs"}
  8292. 2020-11-02T12:07:03.3228693Z [cluster3] {"level":"info","ts":1604318234.959075,"logger":"cmd","msg":"The Lighthouse CRDs were updated, restarting..."}
  8293. 2020-11-02T12:07:03.3229281Z [cluster3] {"level":"info","ts":1604318235.0692887,"logger":"cmd","msg":"Go Version: go1.15.2"}
  8294. 2020-11-02T12:07:03.3230009Z [cluster3] {"level":"info","ts":1604318235.0693266,"logger":"cmd","msg":"Go OS/Arch: linux/amd64"}
  8295. 2020-11-02T12:07:03.3230797Z [cluster3] {"level":"info","ts":1604318235.069335,"logger":"cmd","msg":"Version of operator-sdk: v0.17.0"}
  8296. 2020-11-02T12:07:03.3231446Z [cluster3] {"level":"info","ts":1604318235.0693433,"logger":"cmd","msg":"Submariner operator version: was not built correctly"}
  8297. 2020-11-02T12:07:03.3232083Z [cluster3] {"level":"info","ts":1604318235.0694935,"logger":"leader","msg":"Trying to become the leader."}
  8298. 2020-11-02T12:07:03.3232721Z [cluster3] {"level":"info","ts":1604318235.59346,"logger":"leader","msg":"Found existing lock with my name. I was likely restarted."}
  8299. 2020-11-02T12:07:03.3233504Z [cluster3] {"level":"info","ts":1604318235.5934918,"logger":"leader","msg":"Continuing as the leader."}
  8300. 2020-11-02T12:07:03.3234692Z [cluster3] {"level":"info","ts":1604318236.1417854,"logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":"0.0.0.0:8383"}
  8301. 2020-11-02T12:07:03.3235448Z [cluster3] {"level":"info","ts":1604318236.143121,"logger":"cmd","msg":"Registering Components."}
  8302. 2020-11-02T12:07:03.3236698Z [cluster3] {"level":"info","ts":1604318236.1436362,"logger":"cmd","msg":"Creating the engine CRDs"}
  8303. 2020-11-02T12:07:03.3237564Z [cluster3] {"level":"info","ts":1604318236.2337923,"logger":"cmd","msg":"Creating the Lighthouse CRDs"}
  8304. 2020-11-02T12:07:03.3239454Z [cluster3] {"level":"info","ts":1604318238.420306,"logger":"metrics","msg":"Metrics Service object created","Service.Name":"submariner-operator-metrics","Service.Namespace":"submariner-operator"}
  8305. 2020-11-02T12:07:03.3240707Z [cluster3] {"level":"info","ts":1604318238.9286237,"logger":"cmd","msg":"Could not create ServiceMonitor object","error":"no ServiceMonitor registered with the API"}
  8306. 2020-11-02T12:07:03.3242105Z [cluster3] {"level":"info","ts":1604318238.9288406,"logger":"cmd","msg":"Install prometheus-operator in your cluster to create ServiceMonitor objects","error":"no ServiceMonitor registered with the API"}
  8307. 2020-11-02T12:07:03.3242994Z [cluster3] {"level":"info","ts":1604318238.9289875,"logger":"cmd","msg":"Starting the Cmd."}
  8308. 2020-11-02T12:07:03.3250188Z [cluster3] {"level":"info","ts":1604318238.9293458,"logger":"controller-runtime.manager","msg":"starting metrics server","path":"/metrics"}
  8309. 2020-11-02T12:07:03.3251774Z [cluster3] {"level":"info","ts":1604318238.9294846,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"servicediscovery-controller","source":"kind source: /, Kind="}
  8310. 2020-11-02T12:07:03.3253662Z [cluster3] {"level":"info","ts":1604318238.929791,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"submariner-controller","source":"kind source: /, Kind="}
  8311. 2020-11-02T12:07:03.3255275Z [cluster3] {"level":"info","ts":1604318239.0345616,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"submariner-controller","source":"kind source: /, Kind="}
  8312. 2020-11-02T12:07:03.3322394Z [cluster3] {"level":"info","ts":1604318239.0348313,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"servicediscovery-controller","source":"kind source: /, Kind="}
  8313. 2020-11-02T12:07:03.3325528Z [cluster3] {"level":"info","ts":1604318239.1349416,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"submariner-controller","source":"kind source: /, Kind="}
  8314. 2020-11-02T12:07:03.3327145Z [cluster3] {"level":"info","ts":1604318239.1358545,"logger":"controller-runtime.controller","msg":"Starting Controller","controller":"servicediscovery-controller"}
  8315. 2020-11-02T12:07:03.3328694Z [cluster3] {"level":"info","ts":1604318239.3354456,"logger":"controller-runtime.controller","msg":"Starting Controller","controller":"submariner-controller"}
  8316. 2020-11-02T12:07:03.3330114Z [cluster3] {"level":"info","ts":1604318239.3354933,"logger":"controller-runtime.controller","msg":"Starting workers","controller":"submariner-controller","worker count":1}
  8317. 2020-11-02T12:07:03.3331771Z [cluster3] {"level":"info","ts":1604318239.3356154,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8318. 2020-11-02T12:07:03.3333439Z [cluster3] {"level":"info","ts":1604318239.3361726,"logger":"controller-runtime.controller","msg":"Starting workers","controller":"servicediscovery-controller","worker count":1}
  8319. 2020-11-02T12:07:03.3334665Z [cluster3] {"level":"info","ts":1604318239.649486,"logger":"controller_submariner","msg":"Cluster network discovered"}
  8320. 2020-11-02T12:07:03.3335764Z [cluster3] {"level":"info","ts":1604318239.6497412,"logger":"controller_submariner","msg":"Discovered K8s network details","plugin":"weave-net","clusterCIDRs":["10.3.0.0/16"],"serviceCIDRs":["100.3.0.0/16"]}
  8321. 2020-11-02T12:07:03.3336618Z [cluster3] {"level":"info","ts":1604318239.649865,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8322. 2020-11-02T12:07:03.3337462Z [cluster3] {"level":"info","ts":1604318239.6499476,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8323. 2020-11-02T12:07:03.3339109Z [cluster3] {"level":"info","ts":1604318239.6968443,"logger":"controller_submariner","msg":"Created a new DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8324. 2020-11-02T12:07:03.3347659Z [cluster3] {"level":"error","ts":1604318239.6976025,"logger":"controller-runtime.controller","msg":"Reconciler error","controller":"submariner-controller","name":"submariner","namespace":"submariner-operator","error":"error creating or updating DaemonSet submariner-operator/submariner-gateway: DaemonSet.apps \"submariner-gateway\" not found","errorVerbose":"DaemonSet.apps \"submariner-gateway\" not found\nerror creating or updating DaemonSet submariner-operator/submariner-gateway","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\tsubmariner-operator/vendor/github.com/go-logr/zapr/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsubmariner-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:248\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsubmariner-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:222\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\tsubmariner-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:201\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\tsubmariner-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\tsubmariner-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\tsubmariner-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"}
  8325. 2020-11-02T12:07:03.3354317Z [cluster3] {"level":"info","ts":1604318240.6979506,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8326. 2020-11-02T12:07:03.3355340Z [cluster3] {"level":"info","ts":1604318241.2541478,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8327. 2020-11-02T12:07:03.3356093Z [cluster3] {"level":"info","ts":1604318241.2541745,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8328. 2020-11-02T12:07:03.3357913Z [cluster3] {"level":"info","ts":1604318241.498376,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8329. 2020-11-02T12:07:03.3360114Z [cluster3] {"level":"info","ts":1604318241.5031617,"logger":"controller_submariner","msg":"Created a new DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8330. 2020-11-02T12:07:03.3362172Z [cluster3] {"level":"info","ts":1604318242.4562364,"logger":"controller_servicediscovery","msg":"Reconciling ServiceDiscovery","Request.Namespace":"submariner-operator","Request.Name":"service-discovery"}
  8331. 2020-11-02T12:07:03.3364558Z [cluster3] {"level":"info","ts":1604318242.4580684,"logger":"controller_submariner","msg":"Created Service Discovery CR","Request.Namespace":"submariner-operator","Request.Name":"submariner","Namespace":"submariner-operator","Name":"service-discovery"}
  8332. 2020-11-02T12:07:03.3366354Z [cluster3] {"level":"info","ts":1604318242.4581075,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8333. 2020-11-02T12:07:03.3368596Z [cluster3] {"level":"info","ts":1604318242.520544,"logger":"controller_servicediscovery","msg":"Created a new Deployment","Request.Namespace":"submariner-operator","Request.Name":"service-discovery","Deployment.Namespace":"submariner-operator","Deployment.Name":"submariner-lighthouse-agent"}
  8334. 2020-11-02T12:07:03.3370090Z [cluster3] {"level":"info","ts":1604318242.8777332,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8335. 2020-11-02T12:07:03.3370892Z [cluster3] {"level":"info","ts":1604318242.8778007,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8336. 2020-11-02T12:07:03.3373177Z [cluster3] {"level":"info","ts":1604318242.8890216,"logger":"controller_servicediscovery","msg":"Created a new ConfigMap","Request.Namespace":"submariner-operator","Request.Name":"service-discovery","ConfigMap.Namespace":"submariner-operator","ConfigMap.Name":"submariner-lighthouse-coredns"}
  8337. 2020-11-02T12:07:03.3376927Z [cluster3] {"level":"info","ts":1604318243.6443326,"logger":"controller_servicediscovery","msg":"Created a new Deployment","Request.Namespace":"submariner-operator","Request.Name":"service-discovery","Deployment.Namespace":"submariner-operator","Deployment.Name":"submariner-lighthouse-coredns"}
  8338. 2020-11-02T12:07:03.3379681Z [cluster3] {"level":"info","ts":1604318243.7274864,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8339. 2020-11-02T12:07:03.3381902Z [cluster3] {"level":"info","ts":1604318244.4632316,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8340. 2020-11-02T12:07:03.3384477Z [cluster3] {"level":"info","ts":1604318245.269389,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8341. 2020-11-02T12:07:03.3386769Z [cluster3] {"level":"info","ts":1604318245.5239599,"logger":"controller_servicediscovery","msg":"Created a new Service","Request.Namespace":"submariner-operator","Request.Name":"service-discovery","Service.Namespace":"submariner-operator","Service.Name":"submariner-lighthouse-coredns"}
  8342. 2020-11-02T12:07:03.3442823Z [cluster3] {"level":"error","ts":1604318245.8229198,"logger":"controller-runtime.controller","msg":"Reconciler error","controller":"submariner-controller","name":"submariner","namespace":"submariner-operator","error":"Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\tsubmariner-operator/vendor/github.com/go-logr/zapr/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsubmariner-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:248\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsubmariner-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:222\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\tsubmariner-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:201\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\tsubmariner-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:152\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\tsubmariner-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\tsubmariner-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"}
  8343. 2020-11-02T12:07:03.3450688Z [cluster3] {"level":"info","ts":1604318245.9501235,"logger":"controller_servicediscovery","msg":"coredns configmap does not have lighthouse configuration hence creating","Request.Namespace":"submariner-operator","Request.Name":"service-discovery"}
  8344. 2020-11-02T12:07:03.3454415Z [cluster3] {"level":"info","ts":1604318245.9501698,"logger":"controller_servicediscovery","msg":"Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n forward . 100.3.80.105\n}\n#lighthouse-end\n.:53 {\n errors\n health {\n lameduck 5s\n }\n ready\n kubernetes cluster3.local in-addr.arpa ip6.arpa {\n pods insecure\n fallthrough in-addr.arpa ip6.arpa\n ttl 30\n }\n prometheus :9153\n forward . /etc/resolv.conf\n cache 30\n loop\n reload\n loadbalance\n}\n"}
  8345. 2020-11-02T12:07:03.3457971Z [cluster3] {"level":"info","ts":1604318246.2646072,"logger":"controller_servicediscovery","msg":"Reconciling ServiceDiscovery","Request.Namespace":"submariner-operator","Request.Name":"service-discovery"}
  8346. 2020-11-02T12:07:03.3459717Z [cluster3] {"level":"info","ts":1604318246.848525,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8347. 2020-11-02T12:07:03.3462220Z [cluster3] {"level":"info","ts":1604318246.8789291,"logger":"controller_servicediscovery","msg":"Updated existing Deployment","Request.Namespace":"submariner-operator","Request.Name":"service-discovery","Deployment.Namespace":"submariner-operator","Deployment.Name":"submariner-lighthouse-agent"}
  8348. 2020-11-02T12:07:03.3465464Z [cluster3] {"level":"info","ts":1604318248.6313872,"logger":"controller_servicediscovery","msg":"Updated existing Deployment","Request.Namespace":"submariner-operator","Request.Name":"service-discovery","Deployment.Namespace":"submariner-operator","Deployment.Name":"submariner-lighthouse-coredns"}
  8349. 2020-11-02T12:07:03.3467088Z [cluster3] {"level":"info","ts":1604318248.722003,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8350. 2020-11-02T12:07:03.3467839Z [cluster3] {"level":"info","ts":1604318248.7220354,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8351. 2020-11-02T12:07:03.3469365Z [cluster3] {"level":"info","ts":1604318248.8769066,"logger":"controller_servicediscovery","msg":"coredns configmap has lighthouse configuration hence updating","Request.Namespace":"submariner-operator","Request.Name":"service-discovery"}
  8352. 2020-11-02T12:07:03.3472268Z [cluster3] {"level":"info","ts":1604318248.8769906,"logger":"controller_servicediscovery","msg":"Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n forward . 100.3.80.105\n}\n#lighthouse-end\n.:53 {\n errors\n health {\n lameduck 5s\n }\n ready\n kubernetes cluster3.local in-addr.arpa ip6.arpa {\n pods insecure\n fallthrough in-addr.arpa ip6.arpa\n ttl 30\n }\n prometheus :9153\n forward . /etc/resolv.conf\n cache 30\n loop\n reload\n loadbalance\n}\n\n"}
  8353. 2020-11-02T12:07:03.3475640Z [cluster3] {"level":"info","ts":1604318248.9659324,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8354. 2020-11-02T12:07:03.3477908Z [cluster3] {"level":"info","ts":1604318249.0570924,"logger":"controller_servicediscovery","msg":"Reconciling ServiceDiscovery","Request.Namespace":"submariner-operator","Request.Name":"service-discovery"}
  8355. 2020-11-02T12:07:03.3480181Z [cluster3] {"level":"info","ts":1604318249.4094949,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8356. 2020-11-02T12:07:03.3482920Z [cluster3] {"level":"info","ts":1604318249.4144855,"logger":"controller_servicediscovery","msg":"Updated existing Deployment","Request.Namespace":"submariner-operator","Request.Name":"service-discovery","Deployment.Namespace":"submariner-operator","Deployment.Name":"submariner-lighthouse-agent"}
  8357. 2020-11-02T12:07:03.3486220Z [cluster3] {"level":"info","ts":1604318249.4184477,"logger":"controller_servicediscovery","msg":"Updated existing Deployment","Request.Namespace":"submariner-operator","Request.Name":"service-discovery","Deployment.Namespace":"submariner-operator","Deployment.Name":"submariner-lighthouse-coredns"}
  8358. 2020-11-02T12:07:03.3488730Z [cluster3] {"level":"info","ts":1604318249.4206953,"logger":"controller_servicediscovery","msg":"coredns configmap has lighthouse configuration hence updating","Request.Namespace":"submariner-operator","Request.Name":"service-discovery"}
  8359. 2020-11-02T12:07:03.3491678Z [cluster3] {"level":"info","ts":1604318249.4207275,"logger":"controller_servicediscovery","msg":"Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n forward . 100.3.80.105\n}\n#lighthouse-end\n.:53 {\n errors\n health {\n lameduck 5s\n }\n ready\n kubernetes cluster3.local in-addr.arpa ip6.arpa {\n pods insecure\n fallthrough in-addr.arpa ip6.arpa\n ttl 30\n }\n prometheus :9153\n forward . /etc/resolv.conf\n cache 30\n loop\n reload\n loadbalance\n}\n\n\n"}
  8360. 2020-11-02T12:07:03.3494874Z [cluster3] {"level":"info","ts":1604318284.6778312,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8361. 2020-11-02T12:07:03.3496078Z [cluster3] {"level":"info","ts":1604318285.3464305,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8362. 2020-11-02T12:07:03.3496977Z [cluster3] {"level":"info","ts":1604318285.3464756,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8363. 2020-11-02T12:07:03.3498650Z [cluster3] {"level":"info","ts":1604318285.7981303,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8364. 2020-11-02T12:07:03.3501181Z [cluster3] {"level":"info","ts":1604318287.056949,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8365. 2020-11-02T12:07:03.3503951Z [cluster3] {"level":"info","ts":1604318287.1901417,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8366. 2020-11-02T12:07:03.3505140Z [cluster3] {"level":"info","ts":1604318287.2646239,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8367. 2020-11-02T12:07:03.3505889Z [cluster3] {"level":"info","ts":1604318287.2646608,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8368. 2020-11-02T12:07:03.3507604Z [cluster3] {"level":"info","ts":1604318287.309102,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8369. 2020-11-02T12:07:03.3510108Z [cluster3] {"level":"info","ts":1604318287.3254545,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8370. 2020-11-02T12:07:03.3588747Z [cluster3] {"level":"info","ts":1604318287.3769119,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8371. 2020-11-02T12:07:03.3589862Z [cluster3] {"level":"info","ts":1604318287.4291518,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8372. 2020-11-02T12:07:03.3590677Z [cluster3] {"level":"info","ts":1604318287.4291763,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8373. 2020-11-02T12:07:03.3592678Z [cluster3] {"level":"info","ts":1604318287.4444904,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8374. 2020-11-02T12:07:03.3597512Z [cluster3] {"level":"info","ts":1604318287.4499512,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8375. 2020-11-02T12:07:03.3599679Z [cluster3] {"level":"info","ts":1604318287.8774893,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8376. 2020-11-02T12:07:03.3602903Z [cluster3] {"level":"info","ts":1604318287.9259255,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8377. 2020-11-02T12:07:03.3603683Z [cluster3] {"level":"info","ts":1604318287.9260685,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8378. 2020-11-02T12:07:03.3605816Z [cluster3] {"level":"info","ts":1604318287.937959,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8379. 2020-11-02T12:07:03.3611119Z [cluster3] {"level":"info","ts":1604318287.9489796,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8380. 2020-11-02T12:07:03.3618239Z [cluster3] {"level":"info","ts":1604318288.0029488,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8381. 2020-11-02T12:07:03.3619295Z [cluster3] {"level":"info","ts":1604318288.0215895,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8382. 2020-11-02T12:07:03.3620420Z [cluster3] {"level":"info","ts":1604318288.02161,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8383. 2020-11-02T12:07:03.3624692Z [cluster3] {"level":"info","ts":1604318288.0298135,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8384. 2020-11-02T12:07:03.3629227Z [cluster3] {"level":"info","ts":1604318288.0326047,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8385. 2020-11-02T12:07:03.3631991Z [cluster3] {"level":"info","ts":1604318292.895422,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8386. 2020-11-02T12:07:03.3635854Z [cluster3] {"level":"info","ts":1604318293.2367415,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8387. 2020-11-02T12:07:03.3636598Z [cluster3] {"level":"info","ts":1604318293.2369273,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8388. 2020-11-02T12:07:03.3640421Z [cluster3] {"level":"info","ts":1604318293.2633636,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8389. 2020-11-02T12:07:03.3644529Z [cluster3] {"level":"info","ts":1604318293.2722275,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8390. 2020-11-02T12:07:03.3646712Z [cluster3] {"level":"info","ts":1604318299.3166595,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8391. 2020-11-02T12:07:03.3649688Z [cluster3] {"level":"info","ts":1604318299.4173782,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8392. 2020-11-02T12:07:03.3650665Z [cluster3] {"level":"info","ts":1604318299.4175382,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8393. 2020-11-02T12:07:03.3653842Z [cluster3] {"level":"info","ts":1604318299.4281409,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8394. 2020-11-02T12:07:03.3658760Z [cluster3] {"level":"info","ts":1604318299.4333487,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8395. 2020-11-02T12:07:03.3661349Z [cluster3] {"level":"info","ts":1604318299.457999,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8396. 2020-11-02T12:07:03.3664216Z [cluster3] {"level":"info","ts":1604318299.506833,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8397. 2020-11-02T12:07:03.3665004Z [cluster3] {"level":"info","ts":1604318299.5070405,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8398. 2020-11-02T12:07:03.3666874Z [cluster3] {"level":"info","ts":1604318299.656237,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8399. 2020-11-02T12:07:03.3671694Z [cluster3] {"level":"info","ts":1604318299.6607869,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8400. 2020-11-02T12:07:03.3674786Z [cluster3] {"level":"info","ts":1604318302.87831,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8401. 2020-11-02T12:07:03.3677013Z [cluster3] {"level":"info","ts":1604318302.8990295,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8402. 2020-11-02T12:07:03.3678014Z [cluster3] {"level":"info","ts":1604318302.8990653,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8403. 2020-11-02T12:07:03.3680544Z [cluster3] {"level":"info","ts":1604318302.9089303,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8404. 2020-11-02T12:07:03.3683454Z [cluster3] {"level":"info","ts":1604318302.9122007,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8405. 2020-11-02T12:07:03.3685812Z [cluster3] {"level":"info","ts":1604318302.918632,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8406. 2020-11-02T12:07:03.3687192Z [cluster3] {"level":"info","ts":1604318302.923793,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8407. 2020-11-02T12:07:03.3687965Z [cluster3] {"level":"info","ts":1604318302.9238164,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8408. 2020-11-02T12:07:03.3689745Z [cluster3] {"level":"info","ts":1604318302.9272525,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8409. 2020-11-02T12:07:03.3692301Z [cluster3] {"level":"info","ts":1604318302.9300413,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8410. 2020-11-02T12:07:03.3694448Z [cluster3] {"level":"info","ts":1604318304.5224242,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8411. 2020-11-02T12:07:03.3695459Z [cluster3] {"level":"info","ts":1604318304.5296469,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8412. 2020-11-02T12:07:03.3696217Z [cluster3] {"level":"info","ts":1604318304.529664,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8413. 2020-11-02T12:07:03.3717718Z [cluster3] {"level":"info","ts":1604318304.5409021,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8414. 2020-11-02T12:07:03.3720530Z [cluster3] {"level":"info","ts":1604318304.5492127,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8415. 2020-11-02T12:07:03.3722852Z [cluster3] {"level":"info","ts":1604318304.5596838,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8416. 2020-11-02T12:07:03.3724080Z [cluster3] {"level":"info","ts":1604318304.563984,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8417. 2020-11-02T12:07:03.3724845Z [cluster3] {"level":"info","ts":1604318304.564106,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8418. 2020-11-02T12:07:03.3726605Z [cluster3] {"level":"info","ts":1604318304.5689409,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8419. 2020-11-02T12:07:03.3728980Z [cluster3] {"level":"info","ts":1604318304.5724936,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8420. 2020-11-02T12:07:03.3731060Z [cluster3] {"level":"info","ts":1604318309.543307,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8421. 2020-11-02T12:07:03.3733046Z [cluster3] {"level":"info","ts":1604318309.5490627,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8422. 2020-11-02T12:07:03.3734126Z [cluster3] {"level":"info","ts":1604318309.5490835,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8423. 2020-11-02T12:07:03.3735983Z [cluster3] {"level":"info","ts":1604318309.552246,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8424. 2020-11-02T12:07:03.3738288Z [cluster3] {"level":"info","ts":1604318309.5550456,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8425. 2020-11-02T12:07:03.3740161Z [cluster3] {"level":"info","ts":1604318309.5610106,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8426. 2020-11-02T12:07:03.3741268Z [cluster3] {"level":"info","ts":1604318309.5653698,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8427. 2020-11-02T12:07:03.3742004Z [cluster3] {"level":"info","ts":1604318309.56539,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8428. 2020-11-02T12:07:03.3743695Z [cluster3] {"level":"info","ts":1604318309.5694797,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8429. 2020-11-02T12:07:03.3745988Z [cluster3] {"level":"info","ts":1604318309.572518,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8430. 2020-11-02T12:07:03.3747868Z [cluster3] {"level":"info","ts":1604318314.5808225,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8431. 2020-11-02T12:07:03.3748845Z [cluster3] {"level":"info","ts":1604318314.5920022,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8432. 2020-11-02T12:07:03.3749548Z [cluster3] {"level":"info","ts":1604318314.5920653,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8433. 2020-11-02T12:07:03.3751558Z [cluster3] {"level":"info","ts":1604318314.598834,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8434. 2020-11-02T12:07:03.3755307Z [cluster3] {"level":"info","ts":1604318314.6051164,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8435. 2020-11-02T12:07:03.3757123Z [cluster3] {"level":"info","ts":1604318319.6055672,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8436. 2020-11-02T12:07:03.3758059Z [cluster3] {"level":"info","ts":1604318319.6100876,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8437. 2020-11-02T12:07:03.3758754Z [cluster3] {"level":"info","ts":1604318319.6101081,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8438. 2020-11-02T12:07:03.3760857Z [cluster3] {"level":"info","ts":1604318319.6139934,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8439. 2020-11-02T12:07:03.3763763Z [cluster3] {"level":"info","ts":1604318319.6169415,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8440. 2020-11-02T12:07:03.3768022Z [cluster3] {"level":"info","ts":1604318324.622186,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8441. 2020-11-02T12:07:03.3769164Z [cluster3] {"level":"info","ts":1604318324.631163,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8442. 2020-11-02T12:07:03.3774456Z [cluster3] {"level":"info","ts":1604318324.6311998,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8443. 2020-11-02T12:07:03.3776244Z [cluster3] {"level":"info","ts":1604318324.637385,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8444. 2020-11-02T12:07:03.3779601Z [cluster3] {"level":"info","ts":1604318324.65232,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8445. 2020-11-02T12:07:03.3787602Z [cluster3] {"level":"info","ts":1604318329.700157,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8446. 2020-11-02T12:07:03.3788761Z [cluster3] {"level":"info","ts":1604318329.7125685,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8447. 2020-11-02T12:07:03.3789513Z [cluster3] {"level":"info","ts":1604318329.712592,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8448. 2020-11-02T12:07:03.3791380Z [cluster3] {"level":"info","ts":1604318329.7170188,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8449. 2020-11-02T12:07:03.3794428Z [cluster3] {"level":"info","ts":1604318329.7237284,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8450. 2020-11-02T12:07:03.3796248Z [cluster3] {"level":"info","ts":1604318334.7478354,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8451. 2020-11-02T12:07:03.3804486Z [cluster3] {"level":"info","ts":1604318334.7576368,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8452. 2020-11-02T12:07:03.3805261Z [cluster3] {"level":"info","ts":1604318334.757661,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8453. 2020-11-02T12:07:03.3807171Z [cluster3] {"level":"info","ts":1604318334.7716177,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8454. 2020-11-02T12:07:03.3809449Z [cluster3] {"level":"info","ts":1604318334.7853549,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8455. 2020-11-02T12:07:03.3811507Z [cluster3] {"level":"info","ts":1604318339.7983408,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8456. 2020-11-02T12:07:03.3812891Z [cluster3] {"level":"info","ts":1604318339.8096085,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8457. 2020-11-02T12:07:03.3813650Z [cluster3] {"level":"info","ts":1604318339.809627,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8458. 2020-11-02T12:07:03.3815441Z [cluster3] {"level":"info","ts":1604318339.814302,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8459. 2020-11-02T12:07:03.3817662Z [cluster3] {"level":"info","ts":1604318339.8196056,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8460. 2020-11-02T12:07:03.3819470Z [cluster3] {"level":"info","ts":1604318344.8261447,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8461. 2020-11-02T12:07:03.3820503Z [cluster3] {"level":"info","ts":1604318344.8375618,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8462. 2020-11-02T12:07:03.3821217Z [cluster3] {"level":"info","ts":1604318344.8375785,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8463. 2020-11-02T12:07:03.3823368Z [cluster3] {"level":"info","ts":1604318344.8462176,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8464. 2020-11-02T12:07:03.3826039Z [cluster3] {"level":"info","ts":1604318344.8589187,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8465. 2020-11-02T12:07:03.3827841Z [cluster3] {"level":"info","ts":1604318349.9037387,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8466. 2020-11-02T12:07:03.3828791Z [cluster3] {"level":"info","ts":1604318349.9147675,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8467. 2020-11-02T12:07:03.3829740Z [cluster3] {"level":"info","ts":1604318349.914786,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8468. 2020-11-02T12:07:03.3832812Z [cluster3] {"level":"info","ts":1604318349.923768,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8469. 2020-11-02T12:07:03.3835634Z [cluster3] {"level":"info","ts":1604318349.9298975,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8470. 2020-11-02T12:07:03.3838152Z [cluster3] {"level":"info","ts":1604318354.9294786,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8471. 2020-11-02T12:07:03.3839071Z [cluster3] {"level":"info","ts":1604318354.9493713,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8472. 2020-11-02T12:07:03.3839924Z [cluster3] {"level":"info","ts":1604318354.9493992,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8473. 2020-11-02T12:07:03.3844350Z [cluster3] {"level":"info","ts":1604318354.957238,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8474. 2020-11-02T12:07:03.3849529Z [cluster3] {"level":"info","ts":1604318354.9603577,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8475. 2020-11-02T12:07:03.3852628Z [cluster3] {"level":"info","ts":1604318360.0892503,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8476. 2020-11-02T12:07:03.3853762Z [cluster3] {"level":"info","ts":1604318360.103144,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8477. 2020-11-02T12:07:03.3854477Z [cluster3] {"level":"info","ts":1604318360.1031735,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8478. 2020-11-02T12:07:03.3856363Z [cluster3] {"level":"info","ts":1604318360.1090891,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8479. 2020-11-02T12:07:03.3858805Z [cluster3] {"level":"info","ts":1604318360.1230683,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8480. 2020-11-02T12:07:03.3862114Z [cluster3] {"level":"info","ts":1604318365.1511214,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8481. 2020-11-02T12:07:03.3865943Z [cluster3] {"level":"info","ts":1604318365.1562028,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8482. 2020-11-02T12:07:03.3866657Z [cluster3] {"level":"info","ts":1604318365.1562362,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8483. 2020-11-02T12:07:03.3870556Z [cluster3] {"level":"info","ts":1604318365.166826,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8484. 2020-11-02T12:07:03.3873133Z [cluster3] {"level":"info","ts":1604318365.1709778,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8485. 2020-11-02T12:07:03.3875013Z [cluster3] {"level":"info","ts":1604318370.176489,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8486. 2020-11-02T12:07:03.3880831Z [cluster3] {"level":"info","ts":1604318370.1824746,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8487. 2020-11-02T12:07:03.3881943Z [cluster3] {"level":"info","ts":1604318370.1824915,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8488. 2020-11-02T12:07:03.3883711Z [cluster3] {"level":"info","ts":1604318370.1943161,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8489. 2020-11-02T12:07:03.3886218Z [cluster3] {"level":"info","ts":1604318370.2006137,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8490. 2020-11-02T12:07:03.3891402Z [cluster3] {"level":"info","ts":1604318375.1977327,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8491. 2020-11-02T12:07:03.3893233Z [cluster3] {"level":"info","ts":1604318375.2114942,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8492. 2020-11-02T12:07:03.3894494Z [cluster3] {"level":"info","ts":1604318375.211518,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8493. 2020-11-02T12:07:03.3896470Z [cluster3] {"level":"info","ts":1604318375.2192898,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8494. 2020-11-02T12:07:03.3933757Z [cluster3] {"level":"info","ts":1604318375.223376,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8495. 2020-11-02T12:07:03.3938262Z [cluster3] {"level":"info","ts":1604318380.2377985,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8496. 2020-11-02T12:07:03.3939665Z [cluster3] {"level":"info","ts":1604318380.2540357,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8497. 2020-11-02T12:07:03.3940505Z [cluster3] {"level":"info","ts":1604318380.2540627,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8498. 2020-11-02T12:07:03.3942886Z [cluster3] {"level":"info","ts":1604318380.2596667,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8499. 2020-11-02T12:07:03.3957710Z [cluster3] {"level":"info","ts":1604318380.2799478,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8500. 2020-11-02T12:07:03.3964897Z [submariner]$ [cluster3] namespace=submariner-operator
  8501. 2020-11-02T12:07:03.3966277Z [submariner]$ [cluster3] kubectl get pods --selector=app=submariner-globalnet -n submariner-operator -o jsonpath={.items[*].metadata.name}
  8502. 2020-11-02T12:07:03.3967635Z [submariner]$ [cluster3] kubectl get pods --selector=app=submariner-globalnet -n submariner-operator -o jsonpath={.items[*].metadata.name}
  8503. 2020-11-02T12:07:03.3969297Z [submariner]$ [cluster3] command kubectl --context=cluster3 get pods --selector=app=submariner-globalnet -n submariner-operator -o jsonpath={.items[*].metadata.name}
  8504. 2020-11-02T12:07:03.3970975Z [submariner]$ [cluster3] kubectl --context=cluster3 get pods --selector=app=submariner-globalnet -n submariner-operator -o jsonpath={.items[*].metadata.name}
  8505. 2020-11-02T12:07:03.3972578Z [cluster3] {"level":"info","ts":1604318385.2550676,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8506. 2020-11-02T12:07:03.3973535Z [cluster3] {"level":"info","ts":1604318385.2722368,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8507. 2020-11-02T12:07:03.3974248Z [cluster3] {"level":"info","ts":1604318385.2722733,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8508. 2020-11-02T12:07:03.3975860Z [cluster3] {"level":"info","ts":1604318385.2760317,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8509. 2020-11-02T12:07:03.3978496Z [cluster3] {"level":"info","ts":1604318385.280604,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8510. 2020-11-02T12:07:03.3980369Z [cluster3] {"level":"info","ts":1604318390.2964172,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8511. 2020-11-02T12:07:03.3981489Z [cluster3] {"level":"info","ts":1604318390.3057122,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8512. 2020-11-02T12:07:03.3982190Z [cluster3] {"level":"info","ts":1604318390.3057296,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8513. 2020-11-02T12:07:03.3983808Z [cluster3] {"level":"info","ts":1604318390.3124554,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8514. 2020-11-02T12:07:03.3986143Z [cluster3] {"level":"info","ts":1604318390.3165238,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8515. 2020-11-02T12:07:03.3988174Z [cluster3] {"level":"info","ts":1604318395.3374505,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8516. 2020-11-02T12:07:03.3989329Z [cluster3] {"level":"info","ts":1604318395.3477483,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8517. 2020-11-02T12:07:03.3990074Z [cluster3] {"level":"info","ts":1604318395.3477652,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8518. 2020-11-02T12:07:03.3992029Z [cluster3] {"level":"info","ts":1604318395.3533256,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8519. 2020-11-02T12:07:03.3994234Z [cluster3] {"level":"info","ts":1604318395.3621197,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8520. 2020-11-02T12:07:03.3995998Z [cluster3] {"level":"info","ts":1604318400.3720717,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8521. 2020-11-02T12:07:03.3996922Z [cluster3] {"level":"info","ts":1604318400.3783982,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8522. 2020-11-02T12:07:03.3997606Z [cluster3] {"level":"info","ts":1604318400.3784192,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8523. 2020-11-02T12:07:03.3999409Z [cluster3] {"level":"info","ts":1604318400.3818352,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8524. 2020-11-02T12:07:03.4001759Z [cluster3] {"level":"info","ts":1604318400.384935,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8525. 2020-11-02T12:07:03.4004385Z [cluster3] {"level":"info","ts":1604318405.3874254,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8526. 2020-11-02T12:07:03.4006084Z [cluster3] {"level":"info","ts":1604318405.393691,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8527. 2020-11-02T12:07:03.4006766Z [cluster3] {"level":"info","ts":1604318405.3937173,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8528. 2020-11-02T12:07:03.4009011Z [cluster3] {"level":"info","ts":1604318405.4014409,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8529. 2020-11-02T12:07:03.4011382Z [cluster3] {"level":"info","ts":1604318405.4040709,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8530. 2020-11-02T12:07:03.4013144Z [cluster3] {"level":"info","ts":1604318410.3971324,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8531. 2020-11-02T12:07:03.4014497Z [cluster3] {"level":"info","ts":1604318410.4027379,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8532. 2020-11-02T12:07:03.4015479Z [cluster3] {"level":"info","ts":1604318410.4027543,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8533. 2020-11-02T12:07:03.4018410Z [cluster3] {"level":"info","ts":1604318410.4074383,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8534. 2020-11-02T12:07:03.4024078Z [cluster3] {"level":"info","ts":1604318410.410189,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8535. 2020-11-02T12:07:03.4027762Z [cluster3] {"level":"info","ts":1604318415.4121928,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8536. 2020-11-02T12:07:03.4028898Z [cluster3] {"level":"info","ts":1604318415.4191475,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8537. 2020-11-02T12:07:03.4029733Z [cluster3] {"level":"info","ts":1604318415.4191682,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8538. 2020-11-02T12:07:03.4032472Z [cluster3] {"level":"info","ts":1604318415.4223418,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8539. 2020-11-02T12:07:03.4035484Z [cluster3] {"level":"info","ts":1604318415.4250822,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8540. 2020-11-02T12:07:03.4038778Z [cluster3] {"level":"info","ts":1604318420.4462276,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8541. 2020-11-02T12:07:03.4039921Z [cluster3] {"level":"info","ts":1604318420.4714289,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8542. 2020-11-02T12:07:03.4040739Z [cluster3] {"level":"info","ts":1604318420.471459,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8543. 2020-11-02T12:07:03.4043778Z [cluster3] {"level":"info","ts":1604318420.4792945,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8544. 2020-11-02T12:07:03.4063526Z [cluster3] {"level":"info","ts":1604318420.4833248,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8545. 2020-11-02T12:07:03.4065837Z [cluster3] {"level":"info","ts":1604318425.485242,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8546. 2020-11-02T12:07:03.4066812Z [cluster3] {"level":"info","ts":1604318425.4942675,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8547. 2020-11-02T12:07:03.4067887Z [cluster3] {"level":"info","ts":1604318425.494288,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8548. 2020-11-02T12:07:03.4070417Z [cluster3] {"level":"info","ts":1604318425.4982626,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8549. 2020-11-02T12:07:03.4073175Z [cluster3] {"level":"info","ts":1604318425.5015926,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8550. 2020-11-02T12:07:03.4075091Z [cluster3] {"level":"info","ts":1604318430.4997818,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8551. 2020-11-02T12:07:03.4076074Z [cluster3] {"level":"info","ts":1604318430.5057237,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8552. 2020-11-02T12:07:03.4076797Z [cluster3] {"level":"info","ts":1604318430.5057425,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8553. 2020-11-02T12:07:03.4078477Z [cluster3] {"level":"info","ts":1604318430.5112321,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8554. 2020-11-02T12:07:03.4080780Z [cluster3] {"level":"info","ts":1604318430.515889,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8555. 2020-11-02T12:07:03.4082644Z [cluster3] {"level":"info","ts":1604318435.5611677,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8556. 2020-11-02T12:07:03.4083623Z [cluster3] {"level":"info","ts":1604318435.5830889,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8557. 2020-11-02T12:07:03.4084459Z [cluster3] {"level":"info","ts":1604318435.5831192,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8558. 2020-11-02T12:07:03.4086144Z [cluster3] {"level":"info","ts":1604318435.5992322,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8559. 2020-11-02T12:07:03.4088433Z [cluster3] {"level":"info","ts":1604318435.6170652,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8560. 2020-11-02T12:07:03.4090295Z [cluster3] {"level":"info","ts":1604318440.597523,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8561. 2020-11-02T12:07:03.4091385Z [cluster3] {"level":"info","ts":1604318440.6031394,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8562. 2020-11-02T12:07:03.4092272Z [cluster3] {"level":"info","ts":1604318440.6031559,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8563. 2020-11-02T12:07:03.4094110Z [cluster3] {"level":"info","ts":1604318440.6070368,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8564. 2020-11-02T12:07:03.4096558Z [cluster3] {"level":"info","ts":1604318440.6096904,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8565. 2020-11-02T12:07:03.4098380Z [cluster3] {"level":"info","ts":1604318445.6137414,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8566. 2020-11-02T12:07:03.4099414Z [cluster3] {"level":"info","ts":1604318445.620118,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8567. 2020-11-02T12:07:03.4100124Z [cluster3] {"level":"info","ts":1604318445.6201587,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8568. 2020-11-02T12:07:03.4101756Z [cluster3] {"level":"info","ts":1604318445.624708,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8569. 2020-11-02T12:07:03.4104139Z [cluster3] {"level":"info","ts":1604318445.627953,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8570. 2020-11-02T12:07:03.4106010Z [cluster3] {"level":"info","ts":1604318450.6273756,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8571. 2020-11-02T12:07:03.4106989Z [cluster3] {"level":"info","ts":1604318450.6325107,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8572. 2020-11-02T12:07:03.4108228Z [cluster3] {"level":"info","ts":1604318450.6325653,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8573. 2020-11-02T12:07:03.4109947Z [cluster3] {"level":"info","ts":1604318450.6363583,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8574. 2020-11-02T12:07:03.4112290Z [cluster3] {"level":"info","ts":1604318450.6390128,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8575. 2020-11-02T12:07:03.4114814Z [cluster3] {"level":"info","ts":1604318455.652402,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8576. 2020-11-02T12:07:03.4116181Z [cluster3] {"level":"info","ts":1604318455.6592054,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8577. 2020-11-02T12:07:03.4117079Z [cluster3] {"level":"info","ts":1604318455.6592286,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8578. 2020-11-02T12:07:03.4119357Z [cluster3] {"level":"info","ts":1604318455.6670547,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8579. 2020-11-02T12:07:03.4121661Z [cluster3] {"level":"info","ts":1604318455.671687,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8580. 2020-11-02T12:07:03.4123404Z [cluster3] {"level":"info","ts":1604318460.6721225,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8581. 2020-11-02T12:07:03.4132928Z [cluster3] {"level":"info","ts":1604318460.6786635,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8582. 2020-11-02T12:07:03.4133891Z [cluster3] {"level":"info","ts":1604318460.6786816,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8583. 2020-11-02T12:07:03.4135836Z [cluster3] {"level":"info","ts":1604318460.6820595,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8584. 2020-11-02T12:07:03.4138216Z [cluster3] {"level":"info","ts":1604318460.685998,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8585. 2020-11-02T12:07:03.4141675Z [cluster3] {"level":"info","ts":1604318465.687875,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8586. 2020-11-02T12:07:03.4142638Z [cluster3] {"level":"info","ts":1604318465.6932838,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8587. 2020-11-02T12:07:03.4143349Z [cluster3] {"level":"info","ts":1604318465.6933308,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8588. 2020-11-02T12:07:03.4144961Z [cluster3] {"level":"info","ts":1604318465.7044716,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8589. 2020-11-02T12:07:03.4147181Z [cluster3] {"level":"info","ts":1604318465.7110527,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8590. 2020-11-02T12:07:03.4148988Z [cluster3] {"level":"info","ts":1604318470.7161126,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8591. 2020-11-02T12:07:03.4149947Z [cluster3] {"level":"info","ts":1604318470.7273648,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8592. 2020-11-02T12:07:03.4150650Z [cluster3] {"level":"info","ts":1604318470.727383,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8593. 2020-11-02T12:07:03.4152254Z [cluster3] {"level":"info","ts":1604318470.7331562,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8594. 2020-11-02T12:07:03.4154480Z [cluster3] {"level":"info","ts":1604318470.7397027,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8595. 2020-11-02T12:07:03.4158160Z [cluster3] {"level":"info","ts":1604318475.746165,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8596. 2020-11-02T12:07:03.4160092Z [cluster3] {"level":"info","ts":1604318475.763427,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8597. 2020-11-02T12:07:03.4161375Z [cluster3] {"level":"info","ts":1604318475.763448,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8598. 2020-11-02T12:07:03.4163548Z [cluster3] {"level":"info","ts":1604318475.767257,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8599. 2020-11-02T12:07:03.4165909Z [cluster3] {"level":"info","ts":1604318475.7750604,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8600. 2020-11-02T12:07:03.4167670Z [cluster3] {"level":"info","ts":1604318480.7677767,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8601. 2020-11-02T12:07:03.4168700Z [cluster3] {"level":"info","ts":1604318480.7741187,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8602. 2020-11-02T12:07:03.4169687Z [cluster3] {"level":"info","ts":1604318480.774141,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8603. 2020-11-02T12:07:03.4171599Z [cluster3] {"level":"info","ts":1604318480.7776973,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8604. 2020-11-02T12:07:03.4178621Z [cluster3] {"level":"info","ts":1604318480.7805293,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8605. 2020-11-02T12:07:03.4181338Z [cluster3] {"level":"info","ts":1604318485.783046,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8606. 2020-11-02T12:07:03.4182360Z [cluster3] {"level":"info","ts":1604318485.7929258,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8607. 2020-11-02T12:07:03.4183120Z [cluster3] {"level":"info","ts":1604318485.7929413,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8608. 2020-11-02T12:07:03.4185054Z [cluster3] {"level":"info","ts":1604318485.7980275,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8609. 2020-11-02T12:07:03.4189301Z [cluster3] {"level":"info","ts":1604318485.8063304,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8610. 2020-11-02T12:07:03.4191659Z [cluster3] {"level":"info","ts":1604318490.8028638,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8611. 2020-11-02T12:07:03.4192714Z [cluster3] {"level":"info","ts":1604318490.8079915,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8612. 2020-11-02T12:07:03.4193764Z [cluster3] {"level":"info","ts":1604318490.8080225,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8613. 2020-11-02T12:07:03.4195372Z [cluster3] {"level":"info","ts":1604318490.8109467,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8614. 2020-11-02T12:07:03.4197849Z [cluster3] {"level":"info","ts":1604318490.8136246,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8615. 2020-11-02T12:07:03.4200004Z [cluster3] {"level":"info","ts":1604318495.8151145,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8616. 2020-11-02T12:07:03.4201197Z [cluster3] {"level":"info","ts":1604318495.8247733,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8617. 2020-11-02T12:07:03.4202230Z [cluster3] {"level":"info","ts":1604318495.8247912,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8618. 2020-11-02T12:07:03.4204707Z [cluster3] {"level":"info","ts":1604318495.8281522,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8619. 2020-11-02T12:07:03.4207365Z [cluster3] {"level":"info","ts":1604318495.8310041,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8620. 2020-11-02T12:07:03.4209378Z [cluster3] {"level":"info","ts":1604318501.091232,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8621. 2020-11-02T12:07:03.4210506Z [cluster3] {"level":"info","ts":1604318501.0966237,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8622. 2020-11-02T12:07:03.4211235Z [cluster3] {"level":"info","ts":1604318501.0966444,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8623. 2020-11-02T12:07:03.4213421Z [cluster3] {"level":"info","ts":1604318501.1009662,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8624. 2020-11-02T12:07:03.4216262Z [cluster3] {"level":"info","ts":1604318501.104811,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8625. 2020-11-02T12:07:03.4218266Z [cluster3] {"level":"info","ts":1604318501.1155906,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8626. 2020-11-02T12:07:03.4219218Z [cluster3] {"level":"info","ts":1604318501.1242738,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8627. 2020-11-02T12:07:03.4220130Z [cluster3] {"level":"info","ts":1604318501.1242933,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8628. 2020-11-02T12:07:03.4221780Z [cluster3] {"level":"info","ts":1604318501.1285772,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8629. 2020-11-02T12:07:03.4224063Z [cluster3] {"level":"info","ts":1604318501.1335437,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8630. 2020-11-02T12:07:03.4228609Z [cluster3] {"level":"info","ts":1604318506.0986233,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8631. 2020-11-02T12:07:03.4229625Z [cluster3] {"level":"info","ts":1604318506.1050217,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8632. 2020-11-02T12:07:03.4230785Z [cluster3] {"level":"info","ts":1604318506.1050458,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8633. 2020-11-02T12:07:03.4232837Z [cluster3] {"level":"info","ts":1604318506.1139,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8634. 2020-11-02T12:07:03.4235636Z [cluster3] {"level":"info","ts":1604318506.1170015,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8635. 2020-11-02T12:07:03.4237525Z [cluster3] {"level":"info","ts":1604318506.1247745,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8636. 2020-11-02T12:07:03.4238647Z [cluster3] {"level":"info","ts":1604318506.1289177,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8637. 2020-11-02T12:07:03.4239392Z [cluster3] {"level":"info","ts":1604318506.1289349,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8638. 2020-11-02T12:07:03.4262808Z [cluster3] {"level":"info","ts":1604318506.1320677,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8639. 2020-11-02T12:07:03.4265404Z [cluster3] {"level":"info","ts":1604318506.1351235,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8640. 2020-11-02T12:07:03.4267242Z [cluster3] {"level":"info","ts":1604318511.110713,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8641. 2020-11-02T12:07:03.4268206Z [cluster3] {"level":"info","ts":1604318511.1174333,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8642. 2020-11-02T12:07:03.4268907Z [cluster3] {"level":"info","ts":1604318511.117466,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8643. 2020-11-02T12:07:03.4270653Z [cluster3] {"level":"info","ts":1604318511.1213417,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8644. 2020-11-02T12:07:03.4272812Z [cluster3] {"level":"info","ts":1604318511.124255,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8645. 2020-11-02T12:07:03.4274561Z [cluster3] {"level":"info","ts":1604318516.1266613,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8646. 2020-11-02T12:07:03.4275476Z [cluster3] {"level":"info","ts":1604318516.132021,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8647. 2020-11-02T12:07:03.4276156Z [cluster3] {"level":"info","ts":1604318516.1328082,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8648. 2020-11-02T12:07:03.4277913Z [cluster3] {"level":"info","ts":1604318516.1364505,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8649. 2020-11-02T12:07:03.4280065Z [cluster3] {"level":"info","ts":1604318516.1389787,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8650. 2020-11-02T12:07:03.4281813Z [cluster3] {"level":"info","ts":1604318521.1405742,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8651. 2020-11-02T12:07:03.4282727Z [cluster3] {"level":"info","ts":1604318521.1490047,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8652. 2020-11-02T12:07:03.4283408Z [cluster3] {"level":"info","ts":1604318521.1490219,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8653. 2020-11-02T12:07:03.4285304Z [cluster3] {"level":"info","ts":1604318521.1527066,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8654. 2020-11-02T12:07:03.4288589Z [cluster3] {"level":"info","ts":1604318521.1551843,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8655. 2020-11-02T12:07:03.4291341Z [cluster3] {"level":"info","ts":1604318526.1588733,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8656. 2020-11-02T12:07:03.4292308Z [cluster3] {"level":"info","ts":1604318526.1650486,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8657. 2020-11-02T12:07:03.4293012Z [cluster3] {"level":"info","ts":1604318526.1650703,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8658. 2020-11-02T12:07:03.4294732Z [cluster3] {"level":"info","ts":1604318526.1698184,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8659. 2020-11-02T12:07:03.4297274Z [cluster3] {"level":"info","ts":1604318526.1727521,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8660. 2020-11-02T12:07:03.4299080Z [cluster3] {"level":"info","ts":1604318531.1682656,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8661. 2020-11-02T12:07:03.4300751Z [cluster3] {"level":"info","ts":1604318531.1744952,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8662. 2020-11-02T12:07:03.4305561Z [cluster3] {"level":"info","ts":1604318531.174519,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8663. 2020-11-02T12:07:03.4309241Z [cluster3] {"level":"info","ts":1604318531.1778107,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8664. 2020-11-02T12:07:03.4311516Z [cluster3] {"level":"info","ts":1604318531.1809351,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8665. 2020-11-02T12:07:03.4314204Z [cluster3] {"level":"info","ts":1604318536.1810954,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8666. 2020-11-02T12:07:03.4315152Z [cluster3] {"level":"info","ts":1604318536.187191,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8667. 2020-11-02T12:07:03.4315840Z [cluster3] {"level":"info","ts":1604318536.1872137,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8668. 2020-11-02T12:07:03.4317399Z [cluster3] {"level":"info","ts":1604318536.1915169,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8669. 2020-11-02T12:07:03.4319552Z [cluster3] {"level":"info","ts":1604318536.1948164,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8670. 2020-11-02T12:07:03.4321428Z [cluster3] {"level":"info","ts":1604318541.1987882,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8671. 2020-11-02T12:07:03.4322373Z [cluster3] {"level":"info","ts":1604318541.2032957,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8672. 2020-11-02T12:07:03.4323056Z [cluster3] {"level":"info","ts":1604318541.2033215,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8673. 2020-11-02T12:07:03.4324792Z [cluster3] {"level":"info","ts":1604318541.2069967,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8674. 2020-11-02T12:07:03.4326996Z [cluster3] {"level":"info","ts":1604318541.209134,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8675. 2020-11-02T12:07:03.4328744Z [cluster3] {"level":"info","ts":1604318546.2157366,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8676. 2020-11-02T12:07:03.4330708Z [cluster3] {"level":"info","ts":1604318546.22795,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8677. 2020-11-02T12:07:03.4331552Z [cluster3] {"level":"info","ts":1604318546.227979,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8678. 2020-11-02T12:07:03.4333976Z [cluster3] {"level":"info","ts":1604318546.2338917,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8679. 2020-11-02T12:07:03.4336147Z [cluster3] {"level":"info","ts":1604318546.2373211,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8680. 2020-11-02T12:07:03.4337900Z [cluster3] {"level":"info","ts":1604318551.2399278,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8681. 2020-11-02T12:07:03.4338999Z [cluster3] {"level":"info","ts":1604318551.245122,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8682. 2020-11-02T12:07:03.4340296Z [cluster3] {"level":"info","ts":1604318551.2451792,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8683. 2020-11-02T12:07:03.4342081Z [cluster3] {"level":"info","ts":1604318551.2488813,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8684. 2020-11-02T12:07:03.4345647Z [cluster3] {"level":"info","ts":1604318551.2512817,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8685. 2020-11-02T12:07:03.4348290Z [cluster3] {"level":"info","ts":1604318556.2526112,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8686. 2020-11-02T12:07:03.4349248Z [cluster3] {"level":"info","ts":1604318556.2591436,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8687. 2020-11-02T12:07:03.4349961Z [cluster3] {"level":"info","ts":1604318556.2591698,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8688. 2020-11-02T12:07:03.4351692Z [cluster3] {"level":"info","ts":1604318556.2629793,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8689. 2020-11-02T12:07:03.4354082Z [cluster3] {"level":"info","ts":1604318556.2655976,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8690. 2020-11-02T12:07:03.4355824Z [cluster3] {"level":"info","ts":1604318561.2655106,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8691. 2020-11-02T12:07:03.4357275Z [cluster3] {"level":"info","ts":1604318561.2709777,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8692. 2020-11-02T12:07:03.4358113Z [cluster3] {"level":"info","ts":1604318561.2709997,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8693. 2020-11-02T12:07:03.4359778Z [cluster3] {"level":"info","ts":1604318561.2748296,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8694. 2020-11-02T12:07:03.4361934Z [cluster3] {"level":"info","ts":1604318561.2776551,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8695. 2020-11-02T12:07:03.4363683Z [cluster3] {"level":"info","ts":1604318566.2803645,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8696. 2020-11-02T12:07:03.4364705Z [cluster3] {"level":"info","ts":1604318566.285141,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8697. 2020-11-02T12:07:03.4365387Z [cluster3] {"level":"info","ts":1604318566.2851646,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8698. 2020-11-02T12:07:03.4367363Z [cluster3] {"level":"info","ts":1604318566.2884815,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8699. 2020-11-02T12:07:03.4370509Z [cluster3] {"level":"info","ts":1604318566.2926815,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8700. 2020-11-02T12:07:03.4372535Z [cluster3] {"level":"info","ts":1604318571.2907467,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8701. 2020-11-02T12:07:03.4373819Z [cluster3] {"level":"info","ts":1604318571.2960029,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8702. 2020-11-02T12:07:03.4374500Z [cluster3] {"level":"info","ts":1604318571.2960205,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8703. 2020-11-02T12:07:03.4376052Z [cluster3] {"level":"info","ts":1604318571.299842,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8704. 2020-11-02T12:07:03.4378619Z [cluster3] {"level":"info","ts":1604318571.3027387,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8705. 2020-11-02T12:07:03.4381093Z [cluster3] {"level":"info","ts":1604318576.3034,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8706. 2020-11-02T12:07:03.4382453Z [cluster3] {"level":"info","ts":1604318576.3095307,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8707. 2020-11-02T12:07:03.4383380Z [cluster3] {"level":"info","ts":1604318576.309554,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8708. 2020-11-02T12:07:03.4385205Z [cluster3] {"level":"info","ts":1604318576.3151913,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8709. 2020-11-02T12:07:03.4388480Z [cluster3] {"level":"info","ts":1604318576.3183928,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8710. 2020-11-02T12:07:03.4391415Z [cluster3] {"level":"info","ts":1604318581.3155298,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8711. 2020-11-02T12:07:03.4392400Z [cluster3] {"level":"info","ts":1604318581.33248,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8712. 2020-11-02T12:07:03.4393118Z [cluster3] {"level":"info","ts":1604318581.332498,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8713. 2020-11-02T12:07:03.4394925Z [cluster3] {"level":"info","ts":1604318581.33906,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8714. 2020-11-02T12:07:03.4397681Z [cluster3] {"level":"info","ts":1604318581.3477278,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8715. 2020-11-02T12:07:03.4400306Z [cluster3] {"level":"info","ts":1604318586.3388987,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8716. 2020-11-02T12:07:03.4401289Z [cluster3] {"level":"info","ts":1604318586.3437665,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8717. 2020-11-02T12:07:03.4402136Z [cluster3] {"level":"info","ts":1604318586.3437839,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8718. 2020-11-02T12:07:03.4404432Z [cluster3] {"level":"info","ts":1604318586.3474152,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8719. 2020-11-02T12:07:03.4406887Z [cluster3] {"level":"info","ts":1604318586.350586,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8720. 2020-11-02T12:07:03.4408751Z [cluster3] {"level":"info","ts":1604318591.3576262,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8721. 2020-11-02T12:07:03.4410261Z [cluster3] {"level":"info","ts":1604318591.3649778,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8722. 2020-11-02T12:07:03.4410989Z [cluster3] {"level":"info","ts":1604318591.3650024,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8723. 2020-11-02T12:07:03.4412745Z [cluster3] {"level":"info","ts":1604318591.368315,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8724. 2020-11-02T12:07:03.4415184Z [cluster3] {"level":"info","ts":1604318591.3713772,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8725. 2020-11-02T12:07:03.4417295Z [cluster3] {"level":"info","ts":1604318596.3736057,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8726. 2020-11-02T12:07:03.4418285Z [cluster3] {"level":"info","ts":1604318596.378842,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8727. 2020-11-02T12:07:03.4419010Z [cluster3] {"level":"info","ts":1604318596.3788598,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8728. 2020-11-02T12:07:03.4420897Z [cluster3] {"level":"info","ts":1604318596.3834088,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8729. 2020-11-02T12:07:03.4423367Z [cluster3] {"level":"info","ts":1604318596.3866491,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8730. 2020-11-02T12:07:03.4425230Z [cluster3] {"level":"info","ts":1604318601.3894382,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8731. 2020-11-02T12:07:03.4426209Z [cluster3] {"level":"info","ts":1604318601.3999584,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8732. 2020-11-02T12:07:03.4426929Z [cluster3] {"level":"info","ts":1604318601.3999786,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8733. 2020-11-02T12:07:03.4428582Z [cluster3] {"level":"info","ts":1604318601.4042575,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8734. 2020-11-02T12:07:03.4432130Z [cluster3] {"level":"info","ts":1604318601.407113,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8735. 2020-11-02T12:07:03.4434711Z [cluster3] {"level":"info","ts":1604318606.4057143,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8736. 2020-11-02T12:07:03.4435703Z [cluster3] {"level":"info","ts":1604318606.4106977,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8737. 2020-11-02T12:07:03.4436414Z [cluster3] {"level":"info","ts":1604318606.4107382,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8738. 2020-11-02T12:07:03.4438070Z [cluster3] {"level":"info","ts":1604318606.4147472,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8739. 2020-11-02T12:07:03.4440720Z [cluster3] {"level":"info","ts":1604318606.4173849,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8740. 2020-11-02T12:07:03.4443038Z [cluster3] {"level":"info","ts":1604318611.4158235,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8741. 2020-11-02T12:07:03.4444537Z [cluster3] {"level":"info","ts":1604318611.429011,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8742. 2020-11-02T12:07:03.4445343Z [cluster3] {"level":"info","ts":1604318611.4290557,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8743. 2020-11-02T12:07:03.4447286Z [cluster3] {"level":"info","ts":1604318611.4351661,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8744. 2020-11-02T12:07:03.4449679Z [cluster3] {"level":"info","ts":1604318611.4500158,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8745. 2020-11-02T12:07:03.4451608Z [cluster3] {"level":"info","ts":1604318616.4366148,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8746. 2020-11-02T12:07:03.4452617Z [cluster3] {"level":"info","ts":1604318616.441799,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8747. 2020-11-02T12:07:03.4453359Z [cluster3] {"level":"info","ts":1604318616.4418163,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8748. 2020-11-02T12:07:03.4455779Z [cluster3] {"level":"info","ts":1604318616.4445763,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8749. 2020-11-02T12:07:03.4458144Z [cluster3] {"level":"info","ts":1604318616.447339,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8750. 2020-11-02T12:07:03.4460009Z [cluster3] {"level":"info","ts":1604318621.4475143,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8751. 2020-11-02T12:07:03.4460984Z [cluster3] {"level":"info","ts":1604318621.453502,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8752. 2020-11-02T12:07:03.4461843Z [cluster3] {"level":"info","ts":1604318621.4535167,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8753. 2020-11-02T12:07:03.4463898Z [cluster3] {"level":"info","ts":1604318621.4635441,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8754. 2020-11-02T12:07:03.4466199Z [cluster3] {"level":"info","ts":1604318621.4667587,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8755. 2020-11-02T12:07:03.4468074Z [cluster3] {"level":"info","ts":1604318626.470206,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8756. 2020-11-02T12:07:03.4469064Z [cluster3] {"level":"info","ts":1604318626.4746234,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8757. 2020-11-02T12:07:03.4469786Z [cluster3] {"level":"info","ts":1604318626.4746575,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8758. 2020-11-02T12:07:03.4471527Z [cluster3] {"level":"info","ts":1604318626.4784184,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8759. 2020-11-02T12:07:03.4475266Z [cluster3] {"level":"info","ts":1604318626.481181,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8760. 2020-11-02T12:07:03.4477092Z [cluster3] {"level":"info","ts":1604318631.4853687,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8761. 2020-11-02T12:07:03.4478058Z [cluster3] {"level":"info","ts":1604318631.4906287,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8762. 2020-11-02T12:07:03.4478761Z [cluster3] {"level":"info","ts":1604318631.4906516,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8763. 2020-11-02T12:07:03.4481775Z [cluster3] {"level":"info","ts":1604318631.4949775,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8764. 2020-11-02T12:07:03.4484226Z [cluster3] {"level":"info","ts":1604318631.4985597,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8765. 2020-11-02T12:07:03.4487074Z [cluster3] {"level":"info","ts":1604318636.502961,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8766. 2020-11-02T12:07:03.4488252Z [cluster3] {"level":"info","ts":1604318636.5116682,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8767. 2020-11-02T12:07:03.4489087Z [cluster3] {"level":"info","ts":1604318636.5116832,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8768. 2020-11-02T12:07:03.4492006Z [cluster3] {"level":"info","ts":1604318636.5159187,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8769. 2020-11-02T12:07:03.4495150Z [cluster3] {"level":"info","ts":1604318636.5190952,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8770. 2020-11-02T12:07:03.4497434Z [cluster3] {"level":"info","ts":1604318641.5269372,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8771. 2020-11-02T12:07:03.4501057Z [cluster3] {"level":"info","ts":1604318641.5354335,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8772. 2020-11-02T12:07:03.4502574Z [cluster3] {"level":"info","ts":1604318641.535459,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8773. 2020-11-02T12:07:03.4505958Z [cluster3] {"level":"info","ts":1604318641.5390728,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8774. 2020-11-02T12:07:03.4510853Z [cluster3] {"level":"info","ts":1604318641.5429008,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8775. 2020-11-02T12:07:03.4514354Z [cluster3] {"level":"info","ts":1604318646.5463452,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8776. 2020-11-02T12:07:03.4516059Z [cluster3] {"level":"info","ts":1604318646.5611207,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8777. 2020-11-02T12:07:03.4516968Z [cluster3] {"level":"info","ts":1604318646.5611858,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8778. 2020-11-02T12:07:03.4518861Z [cluster3] {"level":"info","ts":1604318646.5749688,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8779. 2020-11-02T12:07:03.4521423Z [cluster3] {"level":"info","ts":1604318646.5813613,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8780. 2020-11-02T12:07:03.4523501Z [cluster3] {"level":"info","ts":1604318651.5650144,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8781. 2020-11-02T12:07:03.4524745Z [cluster3] {"level":"info","ts":1604318651.5708117,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8782. 2020-11-02T12:07:03.4526171Z [cluster3] {"level":"info","ts":1604318651.5708294,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8783. 2020-11-02T12:07:03.4528163Z [cluster3] {"level":"info","ts":1604318651.574619,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8784. 2020-11-02T12:07:03.4531316Z [cluster3] {"level":"info","ts":1604318651.5784297,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8785. 2020-11-02T12:07:03.4533795Z [cluster3] {"level":"info","ts":1604318656.5781791,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8786. 2020-11-02T12:07:03.4535318Z [cluster3] {"level":"info","ts":1604318656.5845037,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8787. 2020-11-02T12:07:03.4536835Z [cluster3] {"level":"info","ts":1604318656.5845292,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8788. 2020-11-02T12:07:03.4539615Z [cluster3] {"level":"info","ts":1604318656.588505,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8789. 2020-11-02T12:07:03.4542984Z [cluster3] {"level":"info","ts":1604318656.5910103,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8790. 2020-11-02T12:07:03.4545172Z [cluster3] {"level":"info","ts":1604318661.595834,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8791. 2020-11-02T12:07:03.4546910Z [cluster3] {"level":"info","ts":1604318661.6015575,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8792. 2020-11-02T12:07:03.4548479Z [cluster3] {"level":"info","ts":1604318661.601584,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8793. 2020-11-02T12:07:03.4550810Z [cluster3] {"level":"info","ts":1604318661.605079,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8794. 2020-11-02T12:07:03.4553837Z [cluster3] {"level":"info","ts":1604318661.6075802,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8795. 2020-11-02T12:07:03.4557101Z [cluster3] {"level":"info","ts":1604318666.6096287,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8796. 2020-11-02T12:07:03.4558332Z [cluster3] {"level":"info","ts":1604318666.615064,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8797. 2020-11-02T12:07:03.4559202Z [cluster3] {"level":"info","ts":1604318666.6150851,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8798. 2020-11-02T12:07:03.4562103Z [cluster3] {"level":"info","ts":1604318666.6182058,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8799. 2020-11-02T12:07:03.4596284Z [cluster3] {"level":"info","ts":1604318666.6233933,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8800. 2020-11-02T12:07:03.4601919Z [cluster3] {"level":"info","ts":1604318671.622153,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8801. 2020-11-02T12:07:03.4602936Z [cluster3] {"level":"info","ts":1604318671.628335,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8802. 2020-11-02T12:07:03.4603806Z [cluster3] {"level":"info","ts":1604318671.6283696,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8803. 2020-11-02T12:07:03.4605631Z [cluster3] {"level":"info","ts":1604318671.639553,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8804. 2020-11-02T12:07:03.4608461Z [cluster3] {"level":"info","ts":1604318671.6421778,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8805. 2020-11-02T12:07:03.4610666Z [cluster3] {"level":"info","ts":1604318676.6318488,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8806. 2020-11-02T12:07:03.4611650Z [cluster3] {"level":"info","ts":1604318676.63736,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8807. 2020-11-02T12:07:03.4612375Z [cluster3] {"level":"info","ts":1604318676.637383,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8808. 2020-11-02T12:07:03.4614164Z [cluster3] {"level":"info","ts":1604318676.6405673,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8809. 2020-11-02T12:07:03.4616481Z [cluster3] {"level":"info","ts":1604318676.643621,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8810. 2020-11-02T12:07:03.4618322Z [cluster3] {"level":"info","ts":1604318681.6433792,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8811. 2020-11-02T12:07:03.4619266Z [cluster3] {"level":"info","ts":1604318681.6492856,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8812. 2020-11-02T12:07:03.4620301Z [cluster3] {"level":"info","ts":1604318681.649304,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8813. 2020-11-02T12:07:03.4621912Z [cluster3] {"level":"info","ts":1604318681.653231,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8814. 2020-11-02T12:07:03.4624132Z [cluster3] {"level":"info","ts":1604318681.655859,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8815. 2020-11-02T12:07:03.4627919Z [cluster3] {"level":"info","ts":1604318686.655631,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8816. 2020-11-02T12:07:03.4631249Z [cluster3] {"level":"info","ts":1604318686.6609988,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8817. 2020-11-02T12:07:03.4632005Z [cluster3] {"level":"info","ts":1604318686.6610188,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8818. 2020-11-02T12:07:03.4633934Z [cluster3] {"level":"info","ts":1604318686.6641703,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8819. 2020-11-02T12:07:03.4636501Z [cluster3] {"level":"info","ts":1604318686.6670172,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8820. 2020-11-02T12:07:03.4638548Z [cluster3] {"level":"info","ts":1604318691.673552,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8821. 2020-11-02T12:07:03.4640247Z [cluster3] {"level":"info","ts":1604318691.6805677,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8822. 2020-11-02T12:07:03.4640951Z [cluster3] {"level":"info","ts":1604318691.6806128,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8823. 2020-11-02T12:07:03.4644084Z [cluster3] {"level":"info","ts":1604318691.684164,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8824. 2020-11-02T12:07:03.4646845Z [cluster3] {"level":"info","ts":1604318691.6871479,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8825. 2020-11-02T12:07:03.4648911Z [cluster3] {"level":"info","ts":1604318696.702064,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8826. 2020-11-02T12:07:03.4650214Z [cluster3] {"level":"info","ts":1604318696.9714198,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8827. 2020-11-02T12:07:03.4651226Z [cluster3] {"level":"info","ts":1604318696.9714496,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8828. 2020-11-02T12:07:03.4652881Z [cluster3] {"level":"info","ts":1604318696.988146,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8829. 2020-11-02T12:07:03.4655192Z [cluster3] {"level":"info","ts":1604318697.0004714,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8830. 2020-11-02T12:07:03.4658708Z [cluster3] {"level":"info","ts":1604318701.9781222,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8831. 2020-11-02T12:07:03.4664484Z [cluster3] {"level":"info","ts":1604318701.9910321,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8832. 2020-11-02T12:07:03.4665821Z [cluster3] {"level":"info","ts":1604318701.9910607,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8833. 2020-11-02T12:07:03.4669434Z [cluster3] {"level":"info","ts":1604318701.9959486,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8834. 2020-11-02T12:07:03.4674841Z [cluster3] {"level":"info","ts":1604318701.999382,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8835. 2020-11-02T12:07:03.4678458Z [cluster3] {"level":"info","ts":1604318707.0158856,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8836. 2020-11-02T12:07:03.4681720Z [cluster3] {"level":"info","ts":1604318707.0325763,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8837. 2020-11-02T12:07:03.4685551Z [cluster3] {"level":"info","ts":1604318707.032603,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8838. 2020-11-02T12:07:03.4695931Z [cluster3] {"level":"info","ts":1604318707.0363808,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8839. 2020-11-02T12:07:03.4699374Z [cluster3] {"level":"info","ts":1604318707.0387557,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8840. 2020-11-02T12:07:03.4701264Z [cluster3] {"level":"info","ts":1604318712.025492,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8841. 2020-11-02T12:07:03.4702253Z [cluster3] {"level":"info","ts":1604318712.0308824,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8842. 2020-11-02T12:07:03.4702978Z [cluster3] {"level":"info","ts":1604318712.0309198,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8843. 2020-11-02T12:07:03.4704850Z [cluster3] {"level":"info","ts":1604318712.0339184,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8844. 2020-11-02T12:07:03.4707330Z [cluster3] {"level":"info","ts":1604318712.0363512,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8845. 2020-11-02T12:07:03.4709299Z [cluster3] {"level":"info","ts":1604318717.0407379,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8846. 2020-11-02T12:07:03.4710318Z [cluster3] {"level":"info","ts":1604318717.0458853,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8847. 2020-11-02T12:07:03.4711068Z [cluster3] {"level":"info","ts":1604318717.0459054,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8848. 2020-11-02T12:07:03.4713177Z [cluster3] {"level":"info","ts":1604318717.049474,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8849. 2020-11-02T12:07:03.4715731Z [cluster3] {"level":"info","ts":1604318717.052321,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8850. 2020-11-02T12:07:03.4718227Z [cluster3] {"level":"info","ts":1604318722.0518277,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8851. 2020-11-02T12:07:03.4719247Z [cluster3] {"level":"info","ts":1604318722.06024,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8852. 2020-11-02T12:07:03.4720153Z [cluster3] {"level":"info","ts":1604318722.0602577,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8853. 2020-11-02T12:07:03.4721814Z [cluster3] {"level":"info","ts":1604318722.0640004,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8854. 2020-11-02T12:07:03.4724726Z [cluster3] {"level":"info","ts":1604318722.0679815,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8855. 2020-11-02T12:07:03.4738428Z [cluster3] {"level":"info","ts":1604318727.0623972,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8856. 2020-11-02T12:07:03.4740185Z [cluster3] {"level":"info","ts":1604318727.0675626,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8857. 2020-11-02T12:07:03.4740927Z [cluster3] {"level":"info","ts":1604318727.0675886,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8858. 2020-11-02T12:07:03.4742815Z [cluster3] {"level":"info","ts":1604318727.070578,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8859. 2020-11-02T12:07:03.4745232Z [cluster3] {"level":"info","ts":1604318727.0738027,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8860. 2020-11-02T12:07:03.4747520Z [cluster3] {"level":"info","ts":1604318732.0759037,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8861. 2020-11-02T12:07:03.4748532Z [cluster3] {"level":"info","ts":1604318732.0813658,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8862. 2020-11-02T12:07:03.4749386Z [cluster3] {"level":"info","ts":1604318732.0813894,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8863. 2020-11-02T12:07:03.4751146Z [cluster3] {"level":"info","ts":1604318732.084772,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8864. 2020-11-02T12:07:03.4753529Z [cluster3] {"level":"info","ts":1604318732.0878038,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8865. 2020-11-02T12:07:03.4755883Z [cluster3] {"level":"info","ts":1604318732.0952206,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8866. 2020-11-02T12:07:03.4759197Z [cluster3] {"level":"info","ts":1604318732.1157758,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8867. 2020-11-02T12:07:03.4760283Z [cluster3] {"level":"info","ts":1604318732.1157959,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8868. 2020-11-02T12:07:03.4762058Z [cluster3] {"level":"info","ts":1604318732.1260889,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8869. 2020-11-02T12:07:03.4765125Z [cluster3] {"level":"info","ts":1604318732.1291056,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8870. 2020-11-02T12:07:03.4767671Z [cluster3] {"level":"info","ts":1604318737.0890214,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8871. 2020-11-02T12:07:03.4768725Z [cluster3] {"level":"info","ts":1604318737.0953944,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8872. 2020-11-02T12:07:03.4769503Z [cluster3] {"level":"info","ts":1604318737.0954168,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8873. 2020-11-02T12:07:03.4771396Z [cluster3] {"level":"info","ts":1604318737.0994952,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8874. 2020-11-02T12:07:03.4774847Z [cluster3] {"level":"info","ts":1604318737.101937,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8875. 2020-11-02T12:07:03.4777274Z [cluster3] {"level":"info","ts":1604318742.101721,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8876. 2020-11-02T12:07:03.4778295Z [cluster3] {"level":"info","ts":1604318742.1122253,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8877. 2020-11-02T12:07:03.4779048Z [cluster3] {"level":"info","ts":1604318742.1122468,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8878. 2020-11-02T12:07:03.4780778Z [cluster3] {"level":"info","ts":1604318742.1167662,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8879. 2020-11-02T12:07:03.4784480Z [cluster3] {"level":"info","ts":1604318742.122273,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8880. 2020-11-02T12:07:03.4786481Z [cluster3] {"level":"info","ts":1604318747.122255,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8881. 2020-11-02T12:07:03.4787504Z [cluster3] {"level":"info","ts":1604318747.1286666,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8882. 2020-11-02T12:07:03.4788260Z [cluster3] {"level":"info","ts":1604318747.1286893,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8883. 2020-11-02T12:07:03.4790176Z [cluster3] {"level":"info","ts":1604318747.13219,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8884. 2020-11-02T12:07:03.4792557Z [cluster3] {"level":"info","ts":1604318747.1353257,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8885. 2020-11-02T12:07:03.4794628Z [cluster3] {"level":"info","ts":1604318752.1349552,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8886. 2020-11-02T12:07:03.4795809Z [cluster3] {"level":"info","ts":1604318752.1408658,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8887. 2020-11-02T12:07:03.4796555Z [cluster3] {"level":"info","ts":1604318752.1408901,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8888. 2020-11-02T12:07:03.4798808Z [cluster3] {"level":"info","ts":1604318752.1441324,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8889. 2020-11-02T12:07:03.4801471Z [cluster3] {"level":"info","ts":1604318752.147424,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8890. 2020-11-02T12:07:03.4803792Z [cluster3] {"level":"info","ts":1604318757.1786814,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8891. 2020-11-02T12:07:03.4805051Z [cluster3] {"level":"info","ts":1604318757.1919801,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8892. 2020-11-02T12:07:03.4805805Z [cluster3] {"level":"info","ts":1604318757.1920023,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8893. 2020-11-02T12:07:03.4807758Z [cluster3] {"level":"info","ts":1604318757.1985648,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8894. 2020-11-02T12:07:03.4811368Z [cluster3] {"level":"info","ts":1604318757.2026248,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8895. 2020-11-02T12:07:03.4813907Z [cluster3] {"level":"info","ts":1604318762.2021601,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8896. 2020-11-02T12:07:03.4815504Z [cluster3] {"level":"info","ts":1604318762.2086322,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8897. 2020-11-02T12:07:03.4816289Z [cluster3] {"level":"info","ts":1604318762.2086482,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8898. 2020-11-02T12:07:03.4818084Z [cluster3] {"level":"info","ts":1604318762.213035,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8899. 2020-11-02T12:07:03.4820615Z [cluster3] {"level":"info","ts":1604318762.2164922,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8900. 2020-11-02T12:07:03.4822557Z [cluster3] {"level":"info","ts":1604318767.2225587,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8901. 2020-11-02T12:07:03.4823565Z [cluster3] {"level":"info","ts":1604318767.2327032,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8902. 2020-11-02T12:07:03.4824306Z [cluster3] {"level":"info","ts":1604318767.2327235,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8903. 2020-11-02T12:07:03.4826011Z [cluster3] {"level":"info","ts":1604318767.2385757,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8904. 2020-11-02T12:07:03.4828378Z [cluster3] {"level":"info","ts":1604318767.2423027,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8905. 2020-11-02T12:07:03.4830305Z [cluster3] {"level":"info","ts":1604318772.237094,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8906. 2020-11-02T12:07:03.4831561Z [cluster3] {"level":"info","ts":1604318772.2428436,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8907. 2020-11-02T12:07:03.4832476Z [cluster3] {"level":"info","ts":1604318772.2428672,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8908. 2020-11-02T12:07:03.4834179Z [cluster3] {"level":"info","ts":1604318772.2530591,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8909. 2020-11-02T12:07:03.4836674Z [cluster3] {"level":"info","ts":1604318772.2560165,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8910. 2020-11-02T12:07:03.4838925Z [cluster3] {"level":"info","ts":1604318777.2545574,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8911. 2020-11-02T12:07:03.4839933Z [cluster3] {"level":"info","ts":1604318777.2666008,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8912. 2020-11-02T12:07:03.4841071Z [cluster3] {"level":"info","ts":1604318777.2666209,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8913. 2020-11-02T12:07:03.4843401Z [cluster3] {"level":"info","ts":1604318777.2706196,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8914. 2020-11-02T12:07:03.4847204Z [cluster3] {"level":"info","ts":1604318777.2740405,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8915. 2020-11-02T12:07:03.4849577Z [cluster3] {"level":"info","ts":1604318782.27192,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8916. 2020-11-02T12:07:03.4851670Z [cluster3] {"level":"info","ts":1604318782.2806537,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8917. 2020-11-02T12:07:03.4852537Z [cluster3] {"level":"info","ts":1604318782.280691,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8918. 2020-11-02T12:07:03.4854786Z [cluster3] {"level":"info","ts":1604318782.2864537,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8919. 2020-11-02T12:07:03.4862649Z [cluster3] {"level":"info","ts":1604318782.289818,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8920. 2020-11-02T12:07:03.4864866Z [cluster3] {"level":"info","ts":1604318787.2859983,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8921. 2020-11-02T12:07:03.4865915Z [cluster3] {"level":"info","ts":1604318787.300145,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8922. 2020-11-02T12:07:03.4866671Z [cluster3] {"level":"info","ts":1604318787.300161,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8923. 2020-11-02T12:07:03.4868412Z [cluster3] {"level":"info","ts":1604318787.3046527,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8924. 2020-11-02T12:07:03.4873646Z [cluster3] {"level":"info","ts":1604318787.3161209,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8925. 2020-11-02T12:07:03.4876298Z [cluster3] {"level":"info","ts":1604318792.3010085,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8926. 2020-11-02T12:07:03.4877320Z [cluster3] {"level":"info","ts":1604318792.311018,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8927. 2020-11-02T12:07:03.4878083Z [cluster3] {"level":"info","ts":1604318792.3110354,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8928. 2020-11-02T12:07:03.4879884Z [cluster3] {"level":"info","ts":1604318792.3346171,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8929. 2020-11-02T12:07:03.4882254Z [cluster3] {"level":"info","ts":1604318792.3430705,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8930. 2020-11-02T12:07:03.4884365Z [cluster3] {"level":"info","ts":1604318797.3133464,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8931. 2020-11-02T12:07:03.4885494Z [cluster3] {"level":"info","ts":1604318797.3195357,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8932. 2020-11-02T12:07:03.4886257Z [cluster3] {"level":"info","ts":1604318797.3195682,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8933. 2020-11-02T12:07:03.4888019Z [cluster3] {"level":"info","ts":1604318797.3251722,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8934. 2020-11-02T12:07:03.4892321Z [cluster3] {"level":"info","ts":1604318797.3291306,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8935. 2020-11-02T12:07:03.4894724Z [cluster3] {"level":"info","ts":1604318802.3334074,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8936. 2020-11-02T12:07:03.4895740Z [cluster3] {"level":"info","ts":1604318802.3395817,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8937. 2020-11-02T12:07:03.4896493Z [cluster3] {"level":"info","ts":1604318802.3396022,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8938. 2020-11-02T12:07:03.4898438Z [cluster3] {"level":"info","ts":1604318802.34783,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8939. 2020-11-02T12:07:03.4901418Z [cluster3] {"level":"info","ts":1604318802.353987,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8940. 2020-11-02T12:07:03.4903485Z [cluster3] {"level":"info","ts":1604318807.3537846,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8941. 2020-11-02T12:07:03.4904714Z [cluster3] {"level":"info","ts":1604318807.36441,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8942. 2020-11-02T12:07:03.4905492Z [cluster3] {"level":"info","ts":1604318807.364449,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8943. 2020-11-02T12:07:03.4907455Z [cluster3] {"level":"info","ts":1604318807.3681242,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8944. 2020-11-02T12:07:03.4909931Z [cluster3] {"level":"info","ts":1604318807.3717222,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8945. 2020-11-02T12:07:03.4913357Z [cluster3] {"level":"info","ts":1604318812.3733017,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8946. 2020-11-02T12:07:03.4914840Z [cluster3] {"level":"info","ts":1604318812.381004,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8947. 2020-11-02T12:07:03.4915631Z [cluster3] {"level":"info","ts":1604318812.3810184,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8948. 2020-11-02T12:07:03.4917596Z [cluster3] {"level":"info","ts":1604318812.3854694,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8949. 2020-11-02T12:07:03.4920286Z [cluster3] {"level":"info","ts":1604318812.3902245,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8950. 2020-11-02T12:07:03.7943768Z [submariner]$ [cluster3] kubectl get pods --selector=app=submariner-engine -n submariner-operator -o jsonpath={.items[*].metadata.name}
  8951. 2020-11-02T12:07:03.7955629Z [submariner]$ [cluster3] kubectl get pods --selector=app=submariner-engine -n submariner-operator -o jsonpath={.items[*].metadata.name}
  8952. 2020-11-02T12:07:03.7966923Z [submariner]$ [cluster3] command kubectl --context=cluster3 get pods --selector=app=submariner-engine -n submariner-operator -o jsonpath={.items[*].metadata.name}
  8953. 2020-11-02T12:07:03.7979316Z [submariner]$ [cluster3] kubectl --context=cluster3 get pods --selector=app=submariner-engine -n submariner-operator -o jsonpath={.items[*].metadata.name}
  8954. 2020-11-02T12:07:04.0861064Z [submariner]$ [cluster3] kubectl -n submariner-operator logs submariner-gateway-wmr57
  8955. 2020-11-02T12:07:04.0873635Z [submariner]$ [cluster3] kubectl -n submariner-operator logs submariner-gateway-wmr57
  8956. 2020-11-02T12:07:04.0885770Z [submariner]$ [cluster3] command kubectl --context=cluster3 -n submariner-operator logs submariner-gateway-wmr57
  8957. 2020-11-02T12:07:04.0901163Z [submariner]$ [cluster3] kubectl --context=cluster3 -n submariner-operator logs submariner-gateway-wmr57
  8958. 2020-11-02T12:07:04.3857176Z [cluster3] {"level":"info","ts":1604318817.4017918,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8959. 2020-11-02T12:07:04.3858612Z [cluster3] {"level":"info","ts":1604318817.4189124,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8960. 2020-11-02T12:07:04.3859729Z [cluster3] {"level":"info","ts":1604318817.418956,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8961. 2020-11-02T12:07:04.3862549Z [cluster3] {"level":"info","ts":1604318817.428496,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8962. 2020-11-02T12:07:04.3865751Z [cluster3] {"level":"info","ts":1604318817.4827302,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8963. 2020-11-02T12:07:04.3867982Z [cluster3] {"level":"info","ts":1604318822.5055606,"logger":"controller_submariner","msg":"Reconciling Submariner","Request.Namespace":"submariner-operator","Request.Name":"submariner"}
  8964. 2020-11-02T12:07:04.3869471Z [cluster3] {"level":"info","ts":1604318822.517391,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Cluster","CIDR":"10.3.0.0/16"}
  8965. 2020-11-02T12:07:04.3870488Z [cluster3] {"level":"info","ts":1604318822.5174103,"logger":"controller_submariner","msg":"Using detected CIDR","type":"Service","CIDR":"100.3.0.0/16"}
  8966. 2020-11-02T12:07:04.3872373Z [cluster3] {"level":"info","ts":1604318822.520943,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-gateway"}
  8967. 2020-11-02T12:07:04.3875248Z [cluster3] {"level":"info","ts":1604318822.52515,"logger":"controller_submariner","msg":"Updated existing DaemonSet","Request.Namespace":"submariner-operator","Request.Name":"submariner","DaemonSet.Namespace":"submariner-operator","DaemonSet.Name":"submariner-routeagent"}
  8968. 2020-11-02T12:07:04.3876593Z [cluster3] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  8969. 2020-11-02T12:07:04.3877003Z [cluster3] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  8970. 2020-11-02T12:07:04.3877913Z [cluster3] +++++++++++++++++++++: Logs for Pod submariner-gateway-wmr57 in namespace submariner-operator :++++++++++++++++++++++
  8971. 2020-11-02T12:07:04.3878978Z [cluster3] + trap 'exit 1' SIGTERM SIGINT
  8972. 2020-11-02T12:07:04.3879682Z [cluster3] + export CHARON_PID_FILE=/var/run/charon.pid
  8973. 2020-11-02T12:07:04.3880373Z [cluster3] + CHARON_PID_FILE=/var/run/charon.pid
  8974. 2020-11-02T12:07:04.3881029Z [cluster3] + rm -f /var/run/charon.pid
  8975. 2020-11-02T12:07:04.3881542Z [cluster3] + SUBMARINER_VERBOSITY=2
  8976. 2020-11-02T12:07:04.3882173Z [cluster3] + '[' false == true ']'
  8977. 2020-11-02T12:07:04.3882959Z [cluster3] + DEBUG=-v=2
  8978. 2020-11-02T12:07:04.3883819Z [cluster3] + mkdir -p /etc/ipsec
  8979. 2020-11-02T12:07:04.3885000Z [cluster3] + sysctl -w net.ipv4.conf.all.send_redirects=0
  8980. 2020-11-02T12:07:04.3885963Z [cluster3] + export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/libexec/strongswan
  8981. 2020-11-02T12:07:04.3886648Z [cluster3] + PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/libexec/strongswan
  8982. 2020-11-02T12:07:04.3887391Z [cluster3] + for f in iptables-save iptables
  8983. 2020-11-02T12:07:04.3888082Z [cluster3] ++ find_iptables_on_host iptables-save
  8984. 2020-11-02T12:07:04.3888940Z [cluster3] ++ chroot /host test -x /usr/sbin/iptables-save
  8985. 2020-11-02T12:07:04.3889732Z [cluster3] net.ipv4.conf.all.send_redirects = 0
  8986. 2020-11-02T12:07:04.3890393Z [cluster3] ++ echo /usr/sbin
  8987. 2020-11-02T12:07:04.3891245Z [cluster3] iptables-save is present on the host at /usr/sbin/iptables-save
  8988. 2020-11-02T12:07:04.3891799Z [cluster3] ++ return
  8989. 2020-11-02T12:07:04.3892299Z [cluster3] + location=/usr/sbin
  8990. 2020-11-02T12:07:04.3893085Z [cluster3] + '[' /usr/sbin '!=' unknown ']'
  8991. 2020-11-02T12:07:04.3894238Z [cluster3] + echo 'iptables-save is present on the host at /usr/sbin/iptables-save'
  8992. 2020-11-02T12:07:04.3895096Z [cluster3] + sed 's!@@PATH@@!/usr/sbin!' /usr/sbin/iptables-wrapper.in
  8993. 2020-11-02T12:07:04.3895986Z [cluster3] + for f in iptables-save iptables
  8994. 2020-11-02T12:07:04.3896474Z [cluster3] ++ find_iptables_on_host iptables
  8995. 2020-11-02T12:07:04.3897272Z [cluster3] ++ chroot /host test -x /usr/sbin/iptables
  8996. 2020-11-02T12:07:04.3897728Z [cluster3] ++ echo /usr/sbin
  8997. 2020-11-02T12:07:04.3898082Z [cluster3] ++ return
  8998. 2020-11-02T12:07:04.3898527Z [cluster3] iptables is present on the host at /usr/sbin/iptables
  8999. 2020-11-02T12:07:04.3899180Z [cluster3] + location=/usr/sbin
  9000. 2020-11-02T12:07:04.3900128Z [cluster3] + '[' /usr/sbin '!=' unknown ']'
  9001. 2020-11-02T12:07:04.3901223Z [cluster3] + echo 'iptables is present on the host at /usr/sbin/iptables'
  9002. 2020-11-02T12:07:04.3902076Z [cluster3] + sed 's!@@PATH@@!/usr/sbin!' /usr/sbin/iptables-wrapper.in
  9003. 2020-11-02T12:07:04.3903196Z [cluster3] + exec submariner-engine -v=2 -alsologtostderr
  9004. 2020-11-02T12:07:04.3960976Z [cluster3] I1102 11:58:07.618224 1 main.go:70] Starting the submariner gateway engine
  9005. 2020-11-02T12:07:04.3962591Z [cluster3] W1102 11:58:07.750917 1 client_config.go:543] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
  9006. 2020-11-02T12:07:04.3963741Z [cluster3] I1102 11:58:07.751402 1 main.go:95] Creating the cable engine
  9007. 2020-11-02T12:07:04.3977461Z [cluster3] I1102 11:58:07.751673 1 syncer.go:61] CableEngine syncer started
  9008. 2020-11-02T12:07:04.3978230Z [cluster3] I1102 11:58:13.756027 1 main.go:252] Gateway leader election config values: main.leaderConfig{LeaseDuration:10, RenewDeadline:5, RetryPeriod:2}
  9009. 2020-11-02T12:07:04.3981141Z [cluster3] I1102 11:58:13.756525 1 main.go:269] Using namespace "submariner-operator" for the leader election lock
  9010. 2020-11-02T12:07:04.3983125Z [cluster3] I1102 11:58:13.756626 1 leaderelection.go:242] attempting to acquire leader lease submariner-operator/submariner-engine-lock...
  9011. 2020-11-02T12:07:04.3984988Z [cluster3] I1102 11:58:13.803054 1 leaderelection.go:252] successfully acquired lease submariner-operator/submariner-engine-lock
  9012. 2020-11-02T12:07:04.3987607Z [cluster3] I1102 11:58:13.803115 1 event.go:281] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"submariner-operator", Name:"submariner-engine-lock", UID:"1d18f962-46fd-4763-9814-968dad447566", APIVersion:"v1", ResourceVersion:"1695", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' cluster3-worker-submariner-engine became leader
  9013. 2020-11-02T12:07:04.3989817Z [cluster3] I1102 11:58:13.803181 1 main.go:144] Creating the datastore syncer
  9014. 2020-11-02T12:07:04.3990935Z [cluster3] I1102 11:58:13.803465 1 libreswan.go:347] Starting Pluto
  9015. 2020-11-02T12:07:04.3991605Z [cluster3] Initializing NSS database
  9016. 2020-11-02T12:07:04.3992557Z [cluster3]
  9017. 2020-11-02T12:07:04.3993474Z [cluster3] nflog ipsec capture disabled
  9018. 2020-11-02T12:07:04.3994273Z [cluster3] I1102 11:58:17.811718 1 cableengine.go:71] CableEngine controller started, driver: "libreswan"
  9019. 2020-11-02T12:07:04.3995208Z [cluster3] I1102 11:58:17.811760 1 datastoresyncer.go:60] Starting the datastore syncer
  9020. 2020-11-02T12:07:04.3996172Z [cluster3] I1102 11:58:17.811863 1 tunnel.go:22] Starting the tunnel controller
  9021. 2020-11-02T12:07:04.3997090Z [cluster3] I1102 11:58:17.816847 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  9022. 2020-11-02T12:07:04.3998078Z [cluster3] I1102 11:58:17.816976 1 resource_syncer.go:180] Starting syncer "Tunnel controller"
  9023. 2020-11-02T12:07:04.3998948Z [cluster3] I1102 11:58:17.816985 1 resource_syncer.go:193] Syncer "Tunnel controller" waiting for informer cache to sync
  9024. 2020-11-02T12:07:04.4000053Z [cluster3] I1102 11:58:17.917208 1 resource_syncer.go:202] Syncer "Tunnel controller" started
  9025. 2020-11-02T12:07:04.4001319Z [cluster3] I1102 11:58:23.819515 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  9026. 2020-11-02T12:07:04.4003068Z [cluster3] I1102 11:58:23.862614 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  9027. 2020-11-02T12:07:04.4005236Z [cluster3] I1102 11:58:23.869537 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  9028. 2020-11-02T12:07:04.4006812Z [cluster3] I1102 11:58:23.869652 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  9029. 2020-11-02T12:07:04.4008874Z [cluster3] I1102 11:58:23.869743 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  9030. 2020-11-02T12:07:04.4010751Z [cluster3] I1102 11:58:23.869814 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  9031. 2020-11-02T12:07:04.4018574Z [cluster3] I1102 11:58:23.869835 1 datastoresyncer.go:80] Starting the broker syncer
  9032. 2020-11-02T12:07:04.4020031Z [cluster3] I1102 11:58:23.869861 1 resource_syncer.go:180] Starting syncer "local -> broker for *v1.Cluster"
  9033. 2020-11-02T12:07:04.4022142Z [cluster3] I1102 11:58:23.869866 1 resource_syncer.go:193] Syncer "local -> broker for *v1.Cluster" waiting for informer cache to sync
  9034. 2020-11-02T12:07:04.4024006Z [cluster3] I1102 11:58:23.970056 1 resource_syncer.go:202] Syncer "local -> broker for *v1.Cluster" started
  9035. 2020-11-02T12:07:04.4026281Z [cluster3] I1102 11:58:23.970082 1 resource_syncer.go:180] Starting syncer "broker -> local for *v1.Cluster"
  9036. 2020-11-02T12:07:04.4028557Z [cluster3] I1102 11:58:23.970112 1 resource_syncer.go:202] Syncer "broker -> local for *v1.Cluster" started
  9037. 2020-11-02T12:07:04.4030495Z [cluster3] I1102 11:58:23.970117 1 resource_syncer.go:180] Starting syncer "local -> broker for *v1.Endpoint"
  9038. 2020-11-02T12:07:04.4032006Z [cluster3] I1102 11:58:23.970122 1 resource_syncer.go:193] Syncer "local -> broker for *v1.Endpoint" waiting for informer cache to sync
  9039. 2020-11-02T12:07:04.4037998Z [cluster3] I1102 11:58:23.977206 1 resource_syncer.go:264] Syncer "broker -> local for *v1.Cluster" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:19Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2", "namespace":"submariner-k8s-broker", "resourceVersion":"1679", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/clusters/cluster2", "uid":"4ede1863-af75-4242-84c7-e67b484c3c22"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.2.0.0/16"}, "cluster_id":"cluster2", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.2.0.0/16"}}}}
  9040. 2020-11-02T12:07:04.4060971Z [cluster3] I1102 11:58:23.977299 1 resource_syncer.go:276] Syncing resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:19Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2", "namespace":"submariner-k8s-broker", "resourceVersion":"1679", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/clusters/cluster2", "uid":"4ede1863-af75-4242-84c7-e67b484c3c22"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.2.0.0/16"}, "cluster_id":"cluster2", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.2.0.0/16"}}}}
  9041. 2020-11-02T12:07:04.4065924Z [cluster3] I1102 11:58:23.977486 1 federator.go:42] In Distribute for &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:19Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2", "namespace":"submariner-k8s-broker", "resourceVersion":"1679", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/clusters/cluster2", "uid":"4ede1863-af75-4242-84c7-e67b484c3c22"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.2.0.0/16"}, "cluster_id":"cluster2", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.2.0.0/16"}}}}
  9042. 2020-11-02T12:07:04.4070021Z [cluster3] I1102 11:58:23.977598 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  9043. 2020-11-02T12:07:04.4074973Z [cluster3] I1102 11:58:23.979925 1 create_or_update.go:32] Creating resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2", "namespace":"submariner-operator"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.2.0.0/16"}, "cluster_id":"cluster2", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.2.0.0/16"}}}}
  9044. 2020-11-02T12:07:04.4077146Z [cluster3] I1102 11:58:23.983659 1 resource_syncer.go:285] Syncer "broker -> local for *v1.Cluster" successfully synced "submariner-k8s-broker/cluster2"
  9045. 2020-11-02T12:07:04.4081274Z [cluster3] I1102 11:58:23.989051 1 resource_syncer.go:264] Syncer "local -> broker for *v1.Cluster" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:23Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2", "namespace":"submariner-operator", "resourceVersion":"1743", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/clusters/cluster2", "uid":"163415fa-a671-4f3b-8d57-df32d2854d1e"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.2.0.0/16"}, "cluster_id":"cluster2", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.2.0.0/16"}}}}
  9046. 2020-11-02T12:07:04.4085532Z [cluster3] I1102 11:58:23.989135 1 resource_syncer.go:437] Found cluster ID label "cluster2" - not syncing resource "cluster2"
  9047. 2020-11-02T12:07:04.4086752Z [cluster3] I1102 11:58:24.070330 1 resource_syncer.go:202] Syncer "local -> broker for *v1.Endpoint" started
  9048. 2020-11-02T12:07:04.4087637Z [cluster3] I1102 11:58:24.070356 1 resource_syncer.go:180] Starting syncer "broker -> local for *v1.Endpoint"
  9049. 2020-11-02T12:07:04.4088443Z [cluster3] I1102 11:58:24.070362 1 resource_syncer.go:202] Syncer "broker -> local for *v1.Endpoint" started
  9050. 2020-11-02T12:07:04.4089102Z [cluster3] I1102 11:58:24.070368 1 datastoresyncer.go:108] Ensuring we are the only endpoint active for this cluster
  9051. 2020-11-02T12:07:04.4090343Z [cluster3] I1102 11:58:24.070379 1 datastoresyncer.go:143] Creating local submariner Cluster: types.SubmarinerCluster{ID:"cluster3", Spec:v1.ClusterSpec{ClusterID:"cluster3", ColorCodes:[]string{"blue"}, ServiceCIDR:[]string{"100.3.0.0/16"}, ClusterCIDR:[]string{"10.3.0.0/16"}, GlobalCIDR:[]string{}}}
  9052. 2020-11-02T12:07:04.4094988Z [cluster3] I1102 11:58:24.070620 1 federator.go:42] In Distribute for &v1.Cluster{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster3", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.ClusterSpec{ClusterID:"cluster3", ColorCodes:[]string{"blue"}, ServiceCIDR:[]string{"100.3.0.0/16"}, ClusterCIDR:[]string{"10.3.0.0/16"}, GlobalCIDR:[]string{}}}
  9053. 2020-11-02T12:07:04.4143353Z [cluster3] I1102 11:58:24.071663 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  9054. 2020-11-02T12:07:04.4146647Z [cluster3] I1102 11:58:24.074625 1 create_or_update.go:32] Creating resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"name":"cluster3", "namespace":"submariner-operator"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.3.0.0/16"}, "cluster_id":"cluster3", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.3.0.0/16"}}}}
  9055. 2020-11-02T12:07:04.4150304Z [cluster3] I1102 11:58:24.077409 1 datastoresyncer.go:156] Creating local submariner Endpoint: types.SubmarinerEndpoint{Spec:v1.EndpointSpec{ClusterID:"cluster3", CableName:"submariner-cable-cluster3-172-17-0-7", Hostname:"cluster3-worker", Subnets:[]string{"100.3.0.0/16", "10.3.0.0/16"}, PrivateIP:"172.17.0.7", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string{}}}
  9056. 2020-11-02T12:07:04.4155584Z [cluster3] I1102 11:58:24.077526 1 federator.go:42] In Distribute for &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster3-submariner-cable-cluster3-172-17-0-7", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster3", CableName:"submariner-cable-cluster3-172-17-0-7", Hostname:"cluster3-worker", Subnets:[]string{"100.3.0.0/16", "10.3.0.0/16"}, PrivateIP:"172.17.0.7", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string{}}}
  9057. 2020-11-02T12:07:04.4159579Z [cluster3] I1102 11:58:24.077611 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  9058. 2020-11-02T12:07:04.4165993Z [cluster3] I1102 11:58:24.077916 1 resource_syncer.go:264] Syncer "broker -> local for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:19Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-k8s-broker", "resourceVersion":"1681", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"2d4d0eb4-8d26-4d4c-92fe-e2b47a023c8a"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  9059. 2020-11-02T12:07:04.4175098Z [cluster3] I1102 11:58:24.077980 1 resource_syncer.go:276] Syncing resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:19Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-k8s-broker", "resourceVersion":"1681", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"2d4d0eb4-8d26-4d4c-92fe-e2b47a023c8a"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  9060. 2020-11-02T12:07:04.4183822Z [cluster3] I1102 11:58:24.078023 1 federator.go:42] In Distribute for &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:19Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-k8s-broker", "resourceVersion":"1681", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"2d4d0eb4-8d26-4d4c-92fe-e2b47a023c8a"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  9061. 2020-11-02T12:07:04.4187728Z [cluster3] I1102 11:58:24.078110 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  9062. 2020-11-02T12:07:04.4191373Z [cluster3] I1102 11:58:24.079566 1 create_or_update.go:32] Creating resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-operator"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  9063. 2020-11-02T12:07:04.4196354Z [cluster3] I1102 11:58:24.080096 1 create_or_update.go:32] Creating resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"name":"cluster3-submariner-cable-cluster3-172-17-0-7", "namespace":"submariner-operator"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster3-172-17-0-7", "cluster_id":"cluster3", "hostname":"cluster3-worker", "nat_enabled":false, "private_ip":"172.17.0.7", "public_ip":"", "subnets":[]interface {}{"100.3.0.0/16", "10.3.0.0/16"}}}}
  9064. 2020-11-02T12:07:04.4203743Z [cluster3] I1102 11:58:24.082987 1 resource_syncer.go:264] Syncer "local -> broker for *v1.Cluster" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "name":"cluster3", "namespace":"submariner-operator", "resourceVersion":"1744", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/clusters/cluster3", "uid":"06200b8a-e697-473a-8cf3-ac5994a4aa05"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.3.0.0/16"}, "cluster_id":"cluster3", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.3.0.0/16"}}}}
  9065. 2020-11-02T12:07:04.4208571Z [cluster3] I1102 11:58:24.083061 1 resource_syncer.go:276] Syncing resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "name":"cluster3", "namespace":"submariner-operator", "resourceVersion":"1744", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/clusters/cluster3", "uid":"06200b8a-e697-473a-8cf3-ac5994a4aa05"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.3.0.0/16"}, "cluster_id":"cluster3", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.3.0.0/16"}}}}
  9066. 2020-11-02T12:07:04.4213381Z [cluster3] I1102 11:58:24.083086 1 federator.go:42] In Distribute for &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "name":"cluster3", "namespace":"submariner-operator", "resourceVersion":"1744", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/clusters/cluster3", "uid":"06200b8a-e697-473a-8cf3-ac5994a4aa05"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.3.0.0/16"}, "cluster_id":"cluster3", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.3.0.0/16"}}}}
  9067. 2020-11-02T12:07:04.4218399Z [cluster3] I1102 11:58:24.083132 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  9068. 2020-11-02T12:07:04.4220367Z [cluster3] I1102 11:58:24.083937 1 resource_syncer.go:285] Syncer "broker -> local for *v1.Endpoint" successfully synced "submariner-k8s-broker/cluster2-submariner-cable-cluster2-172-17-0-4"
  9069. 2020-11-02T12:07:04.4221581Z [cluster3] I1102 11:58:24.084822 1 datastoresyncer.go:99] Datastore syncer started
  9070. 2020-11-02T12:07:04.4224426Z [cluster3] I1102 11:58:24.085565 1 create_or_update.go:32] Creating resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3", "namespace":"submariner-k8s-broker"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.3.0.0/16"}, "cluster_id":"cluster3", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.3.0.0/16"}}}}
  9071. 2020-11-02T12:07:04.4228627Z [cluster3] I1102 11:58:24.091768 1 resource_syncer.go:285] Syncer "local -> broker for *v1.Cluster" successfully synced "submariner-operator/cluster3"
  9072. 2020-11-02T12:07:04.4234694Z [cluster3] I1102 11:58:24.094469 1 resource_syncer.go:264] Syncer "local -> broker for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-operator", "resourceVersion":"1745", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"8c63de0a-c225-4c4f-b602-4d0ec35f7bae"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  9073. 2020-11-02T12:07:04.4239028Z [cluster3] I1102 11:58:24.094653 1 resource_syncer.go:437] Found cluster ID label "cluster2" - not syncing resource "cluster2-submariner-cable-cluster2-172-17-0-4"
  9074. 2020-11-02T12:07:04.4246241Z [cluster3] I1102 11:58:24.094683 1 resource_syncer.go:264] Syncer "local -> broker for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "name":"cluster3-submariner-cable-cluster3-172-17-0-7", "namespace":"submariner-operator", "resourceVersion":"1746", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster3-submariner-cable-cluster3-172-17-0-7", "uid":"0b8d5ecd-a1b2-4456-b720-c0dd679e2b57"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster3-172-17-0-7", "cluster_id":"cluster3", "hostname":"cluster3-worker", "nat_enabled":false, "private_ip":"172.17.0.7", "public_ip":"", "subnets":[]interface {}{"100.3.0.0/16", "10.3.0.0/16"}}}}
  9075. 2020-11-02T12:07:04.4255411Z [cluster3] I1102 11:58:24.094775 1 resource_syncer.go:276] Syncing resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "name":"cluster3-submariner-cable-cluster3-172-17-0-7", "namespace":"submariner-operator", "resourceVersion":"1746", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster3-submariner-cable-cluster3-172-17-0-7", "uid":"0b8d5ecd-a1b2-4456-b720-c0dd679e2b57"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster3-172-17-0-7", "cluster_id":"cluster3", "hostname":"cluster3-worker", "nat_enabled":false, "private_ip":"172.17.0.7", "public_ip":"", "subnets":[]interface {}{"100.3.0.0/16", "10.3.0.0/16"}}}}
  9076. 2020-11-02T12:07:04.4264085Z [cluster3] I1102 11:58:24.094822 1 federator.go:42] In Distribute for &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "name":"cluster3-submariner-cable-cluster3-172-17-0-7", "namespace":"submariner-operator", "resourceVersion":"1746", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster3-submariner-cable-cluster3-172-17-0-7", "uid":"0b8d5ecd-a1b2-4456-b720-c0dd679e2b57"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster3-172-17-0-7", "cluster_id":"cluster3", "hostname":"cluster3-worker", "nat_enabled":false, "private_ip":"172.17.0.7", "public_ip":"", "subnets":[]interface {}{"100.3.0.0/16", "10.3.0.0/16"}}}}
  9077. 2020-11-02T12:07:04.4267869Z [cluster3] I1102 11:58:24.094974 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  9078. 2020-11-02T12:07:04.4273737Z [cluster3] I1102 11:58:24.094999 1 resource_syncer.go:264] Syncer "Tunnel controller" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-operator", "resourceVersion":"1745", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"8c63de0a-c225-4c4f-b602-4d0ec35f7bae"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  9079. 2020-11-02T12:07:04.4283729Z [cluster3] I1102 11:58:24.095180 1 tunnel.go:56] Tunnel controller processing added or updated submariner Endpoint object: &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster2-submariner-cable-cluster2-172-17-0-4", GenerateName:"", Namespace:"submariner-operator", SelfLink:"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", UID:"8c63de0a-c225-4c4f-b602-4d0ec35f7bae", ResourceVersion:"1745", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63739915104, loc:(*time.Location)(0x2020aa0)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"submariner-io/clusterID":"cluster2"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster2", CableName:"submariner-cable-cluster2-172-17-0-4", Hostname:"cluster2-worker", Subnets:[]string{"100.2.0.0/16", "10.2.0.0/16"}, PrivateIP:"172.17.0.4", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string(nil)}}
  9080. 2020-11-02T12:07:04.4289131Z [cluster3] I1102 11:58:24.095228 1 cableengine.go:101] Installing Endpoint cable "submariner-cable-cluster2-172-17-0-4"
  9081. 2020-11-02T12:07:04.4289903Z [cluster3] I1102 11:58:24.095235 1 libreswan.go:164] Active connections: []
  9082. 2020-11-02T12:07:04.4293848Z [cluster3] I1102 11:58:24.096148 1 resource_syncer.go:264] Syncer "broker -> local for *v1.Cluster" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3", "namespace":"submariner-k8s-broker", "resourceVersion":"1721", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/clusters/cluster3", "uid":"b0da8c3d-b10e-4fc0-b95e-e145f3a95836"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.3.0.0/16"}, "cluster_id":"cluster3", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.3.0.0/16"}}}}
  9083. 2020-11-02T12:07:04.4296970Z [cluster3] I1102 11:58:24.103273 1 resource_syncer.go:443] Cluster ID label "cluster3" not present or matches local cluster ID "cluster3" - not syncing resource "cluster3"
  9084. 2020-11-02T12:07:04.4301615Z [cluster3] I1102 11:58:24.128034 1 create_or_update.go:32] Creating resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3-submariner-cable-cluster3-172-17-0-7", "namespace":"submariner-k8s-broker"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster3-172-17-0-7", "cluster_id":"cluster3", "hostname":"cluster3-worker", "nat_enabled":false, "private_ip":"172.17.0.7", "public_ip":"", "subnets":[]interface {}{"100.3.0.0/16", "10.3.0.0/16"}}}}
  9085. 2020-11-02T12:07:04.4304601Z [cluster3] 002 listening for IKE messages
  9086. 2020-11-02T12:07:04.4304953Z [cluster3] 002 forgetting secrets
  9087. 2020-11-02T12:07:04.4305345Z [cluster3] 002 loading secrets from "/etc/ipsec.secrets"
  9088. 2020-11-02T12:07:04.4305861Z [cluster3] 002 loading secrets from "/etc/ipsec.d/submariner.secrets"
  9089. 2020-11-02T12:07:04.4307366Z [cluster3] I1102 11:58:24.140125 1 libreswan.go:245] Creating connection(s) for {{cluster2 submariner-cable-cluster2-172-17-0-4 cluster2-worker [100.2.0.0/16 10.2.0.0/16] 172.17.0.4 false libreswan map[]}}
  9090. 2020-11-02T12:07:04.4309030Z [cluster3] I1102 11:58:24.141614 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-0-0 --id 172.17.0.7 --host 172.17.0.7 --client 172.17.0.7/32 --to --id 172.17.0.4 --host 172.17.0.4 --client 172.17.0.4/32]
  9091. 2020-11-02T12:07:04.4311238Z [cluster3] I1102 11:58:24.145364 1 resource_syncer.go:285] Syncer "local -> broker for *v1.Endpoint" successfully synced "submariner-operator/cluster3-submariner-cable-cluster3-172-17-0-7"
  9092. 2020-11-02T12:07:04.4312860Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-4-0-0"
  9093. 2020-11-02T12:07:04.4317686Z [cluster3] I1102 11:58:24.152300 1 resource_syncer.go:264] Syncer "broker -> local for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster3"}, "name":"cluster3-submariner-cable-cluster3-172-17-0-7", "namespace":"submariner-k8s-broker", "resourceVersion":"1722", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster3-submariner-cable-cluster3-172-17-0-7", "uid":"f4e3ecdd-5302-4901-bfde-9d6483becbcb"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster3-172-17-0-7", "cluster_id":"cluster3", "hostname":"cluster3-worker", "nat_enabled":false, "private_ip":"172.17.0.7", "public_ip":"", "subnets":[]interface {}{"100.3.0.0/16", "10.3.0.0/16"}}}}
  9094. 2020-11-02T12:07:04.4323424Z [cluster3] I1102 11:58:24.152682 1 resource_syncer.go:443] Cluster ID label "cluster3" not present or matches local cluster ID "cluster3" - not syncing resource "cluster3-submariner-cable-cluster3-172-17-0-7"
  9095. 2020-11-02T12:07:04.4325089Z [cluster3] 181 "submariner-cable-cluster2-172-17-0-4-0-0" #1: initiating IKEv2 IKE SA
  9096. 2020-11-02T12:07:04.4326962Z [cluster3] I1102 11:58:24.381791 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-0-1 --id 172.17.0.7 --host 172.17.0.7 --client 172.17.0.7/32 --to --id 172.17.0.4 --host 172.17.0.4 --client 100.2.0.0/16]
  9097. 2020-11-02T12:07:04.4328496Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-4-0-1"
  9098. 2020-11-02T12:07:04.4331849Z [cluster3] 000 "submariner-cable-cluster2-172-17-0-4-0-1": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #1 "submariner-cable-cluster2-172-17-0-4-0-0"
  9099. 2020-11-02T12:07:04.4335031Z [cluster3] I1102 11:58:24.440650 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-0-2 --id 172.17.0.7 --host 172.17.0.7 --client 172.17.0.7/32 --to --id 172.17.0.4 --host 172.17.0.4 --client 10.2.0.0/16]
  9100. 2020-11-02T12:07:04.4336402Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-4-0-2"
  9101. 2020-11-02T12:07:04.4338138Z [cluster3] 000 "submariner-cable-cluster2-172-17-0-4-0-2": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #1 "submariner-cable-cluster2-172-17-0-4-0-0"
  9102. 2020-11-02T12:07:04.4340033Z [cluster3] I1102 11:58:24.487806 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-1-0 --id 172.17.0.7 --host 172.17.0.7 --client 100.3.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 172.17.0.4/32]
  9103. 2020-11-02T12:07:04.4341587Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-4-1-0"
  9104. 2020-11-02T12:07:04.4343226Z [cluster3] 000 "submariner-cable-cluster2-172-17-0-4-1-0": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #1 "submariner-cable-cluster2-172-17-0-4-0-0"
  9105. 2020-11-02T12:07:04.4346009Z [cluster3] I1102 11:58:24.491170 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-1-1 --id 172.17.0.7 --host 172.17.0.7 --client 100.3.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 100.2.0.0/16]
  9106. 2020-11-02T12:07:04.4347413Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-4-1-1"
  9107. 2020-11-02T12:07:04.4348999Z [cluster3] 000 "submariner-cable-cluster2-172-17-0-4-1-1": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #1 "submariner-cable-cluster2-172-17-0-4-0-0"
  9108. 2020-11-02T12:07:04.4351028Z [cluster3] I1102 11:58:24.494019 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-1-2 --id 172.17.0.7 --host 172.17.0.7 --client 100.3.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 10.2.0.0/16]
  9109. 2020-11-02T12:07:04.4353191Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-4-1-2"
  9110. 2020-11-02T12:07:04.4355158Z [cluster3] 000 "submariner-cable-cluster2-172-17-0-4-1-2": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #1 "submariner-cable-cluster2-172-17-0-4-0-0"
  9111. 2020-11-02T12:07:04.4357279Z [cluster3] I1102 11:58:24.497165 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-2-0 --id 172.17.0.7 --host 172.17.0.7 --client 10.3.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 172.17.0.4/32]
  9112. 2020-11-02T12:07:04.4358782Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-4-2-0"
  9113. 2020-11-02T12:07:04.4360753Z [cluster3] 000 "submariner-cable-cluster2-172-17-0-4-2-0": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #1 "submariner-cable-cluster2-172-17-0-4-0-0"
  9114. 2020-11-02T12:07:04.4362709Z [cluster3] I1102 11:58:24.499842 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-2-1 --id 172.17.0.7 --host 172.17.0.7 --client 10.3.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 100.2.0.0/16]
  9115. 2020-11-02T12:07:04.4366177Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-4-2-1"
  9116. 2020-11-02T12:07:04.4368210Z [cluster3] 000 "submariner-cable-cluster2-172-17-0-4-2-1": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #1 "submariner-cable-cluster2-172-17-0-4-0-0"
  9117. 2020-11-02T12:07:04.4370689Z [cluster3] I1102 11:58:24.506383 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-2-2 --id 172.17.0.7 --host 172.17.0.7 --client 10.3.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 10.2.0.0/16]
  9118. 2020-11-02T12:07:04.4372194Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-4-2-2"
  9119. 2020-11-02T12:07:04.4374021Z [cluster3] 000 "submariner-cable-cluster2-172-17-0-4-2-2": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #1 "submariner-cable-cluster2-172-17-0-4-0-0"
  9120. 2020-11-02T12:07:04.4376358Z [cluster3] I1102 11:58:24.511331 1 cableengine.go:150] Successfully installed Endpoint cable "submariner-cable-cluster2-172-17-0-4" with remote IP 172.17.0.4
  9121. 2020-11-02T12:07:04.4379507Z [cluster3] I1102 11:58:24.511475 1 tunnel.go:68] Tunnel controller successfully installed Endpoint cable submariner-cable-cluster2-172-17-0-4 in the engine
  9122. 2020-11-02T12:07:04.4381312Z [cluster3] I1102 11:58:24.511497 1 resource_syncer.go:352] Syncer "Tunnel controller": transform function returned nil - not syncing - requeue: false
  9123. 2020-11-02T12:07:04.4387324Z [cluster3] I1102 11:58:24.511714 1 resource_syncer.go:264] Syncer "Tunnel controller" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:24Z", "generation":1, "name":"cluster3-submariner-cable-cluster3-172-17-0-7", "namespace":"submariner-operator", "resourceVersion":"1746", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster3-submariner-cable-cluster3-172-17-0-7", "uid":"0b8d5ecd-a1b2-4456-b720-c0dd679e2b57"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster3-172-17-0-7", "cluster_id":"cluster3", "hostname":"cluster3-worker", "nat_enabled":false, "private_ip":"172.17.0.7", "public_ip":"", "subnets":[]interface {}{"100.3.0.0/16", "10.3.0.0/16"}}}}
  9124. 2020-11-02T12:07:04.4399243Z [cluster3] I1102 11:58:24.512086 1 tunnel.go:56] Tunnel controller processing added or updated submariner Endpoint object: &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster3-submariner-cable-cluster3-172-17-0-7", GenerateName:"", Namespace:"submariner-operator", SelfLink:"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster3-submariner-cable-cluster3-172-17-0-7", UID:"0b8d5ecd-a1b2-4456-b720-c0dd679e2b57", ResourceVersion:"1746", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63739915104, loc:(*time.Location)(0x2020aa0)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster3", CableName:"submariner-cable-cluster3-172-17-0-7", Hostname:"cluster3-worker", Subnets:[]string{"100.3.0.0/16", "10.3.0.0/16"}, PrivateIP:"172.17.0.7", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string(nil)}}
  9125. 2020-11-02T12:07:04.4404530Z [cluster3] I1102 11:58:24.512132 1 cableengine.go:92] Not installing cable for local cluster
  9126. 2020-11-02T12:07:04.4405799Z [cluster3] I1102 11:58:24.512138 1 tunnel.go:68] Tunnel controller successfully installed Endpoint cable submariner-cable-cluster3-172-17-0-7 in the engine
  9127. 2020-11-02T12:07:04.4407452Z [cluster3] I1102 11:58:24.512151 1 resource_syncer.go:352] Syncer "Tunnel controller": transform function returned nil - not syncing - requeue: false
  9128. 2020-11-02T12:07:04.4408722Z [cluster3] I1102 11:58:24.516175 1 libreswan.go:149] Connection "submariner-cable-cluster2-172-17-0-4" not found in active connections obtained from whack: []
  9129. 2020-11-02T12:07:04.4412773Z [cluster3] I1102 11:58:27.361607 1 resource_syncer.go:264] Syncer "broker -> local for *v1.Cluster" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1", "namespace":"submariner-k8s-broker", "resourceVersion":"1741", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/clusters/cluster1", "uid":"025b70bc-43a5-4d06-b4ff-a733ee10d897"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.1.0.0/16"}, "cluster_id":"cluster1", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.1.0.0/16"}}}}
  9130. 2020-11-02T12:07:04.4419287Z [cluster3] I1102 11:58:27.361680 1 resource_syncer.go:276] Syncing resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1", "namespace":"submariner-k8s-broker", "resourceVersion":"1741", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/clusters/cluster1", "uid":"025b70bc-43a5-4d06-b4ff-a733ee10d897"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.1.0.0/16"}, "cluster_id":"cluster1", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.1.0.0/16"}}}}
  9131. 2020-11-02T12:07:04.4425352Z [cluster3] I1102 11:58:27.361738 1 federator.go:42] In Distribute for &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1", "namespace":"submariner-k8s-broker", "resourceVersion":"1741", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/clusters/cluster1", "uid":"025b70bc-43a5-4d06-b4ff-a733ee10d897"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.1.0.0/16"}, "cluster_id":"cluster1", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.1.0.0/16"}}}}
  9132. 2020-11-02T12:07:04.4428164Z [cluster3] I1102 11:58:27.361816 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"clusters"}
  9133. 2020-11-02T12:07:04.4430856Z [cluster3] I1102 11:58:27.365486 1 create_or_update.go:32] Creating resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1", "namespace":"submariner-operator"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.1.0.0/16"}, "cluster_id":"cluster1", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.1.0.0/16"}}}}
  9134. 2020-11-02T12:07:04.4433277Z [cluster3] I1102 11:58:27.370443 1 resource_syncer.go:285] Syncer "broker -> local for *v1.Cluster" successfully synced "submariner-k8s-broker/cluster1"
  9135. 2020-11-02T12:07:04.4437492Z [cluster3] I1102 11:58:27.375136 1 resource_syncer.go:264] Syncer "local -> broker for *v1.Cluster" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Cluster", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1", "namespace":"submariner-operator", "resourceVersion":"1765", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/clusters/cluster1", "uid":"84a8fe7e-a559-4daf-90fb-988c32372f28"}, "spec":map[string]interface {}{"cluster_cidr":[]interface {}{"10.1.0.0/16"}, "cluster_id":"cluster1", "color_codes":[]interface {}{"blue"}, "global_cidr":[]interface {}{}, "service_cidr":[]interface {}{"100.1.0.0/16"}}}}
  9136. 2020-11-02T12:07:04.4440872Z [cluster3] I1102 11:58:27.375196 1 resource_syncer.go:437] Found cluster ID label "cluster1" - not syncing resource "cluster1"
  9137. 2020-11-02T12:07:04.4446269Z [cluster3] I1102 11:58:27.557367 1 resource_syncer.go:264] Syncer "broker -> local for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1-submariner-cable-cluster1-172-17-0-6", "namespace":"submariner-k8s-broker", "resourceVersion":"1743", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster1-submariner-cable-cluster1-172-17-0-6", "uid":"b8932e09-5864-498e-b04f-0147ed950e52"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster1-172-17-0-6", "cluster_id":"cluster1", "hostname":"cluster1-worker", "nat_enabled":false, "private_ip":"172.17.0.6", "public_ip":"", "subnets":[]interface {}{"100.1.0.0/16", "10.1.0.0/16"}}}}
  9138. 2020-11-02T12:07:04.4455436Z [cluster3] I1102 11:58:27.557456 1 resource_syncer.go:276] Syncing resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1-submariner-cable-cluster1-172-17-0-6", "namespace":"submariner-k8s-broker", "resourceVersion":"1743", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster1-submariner-cable-cluster1-172-17-0-6", "uid":"b8932e09-5864-498e-b04f-0147ed950e52"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster1-172-17-0-6", "cluster_id":"cluster1", "hostname":"cluster1-worker", "nat_enabled":false, "private_ip":"172.17.0.6", "public_ip":"", "subnets":[]interface {}{"100.1.0.0/16", "10.1.0.0/16"}}}}
  9139. 2020-11-02T12:07:04.4464301Z [cluster3] I1102 11:58:27.557488 1 federator.go:42] In Distribute for &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1-submariner-cable-cluster1-172-17-0-6", "namespace":"submariner-k8s-broker", "resourceVersion":"1743", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster1-submariner-cable-cluster1-172-17-0-6", "uid":"b8932e09-5864-498e-b04f-0147ed950e52"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster1-172-17-0-6", "cluster_id":"cluster1", "hostname":"cluster1-worker", "nat_enabled":false, "private_ip":"172.17.0.6", "public_ip":"", "subnets":[]interface {}{"100.1.0.0/16", "10.1.0.0/16"}}}}
  9140. 2020-11-02T12:07:04.4468680Z [cluster3] I1102 11:58:27.557549 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  9141. 2020-11-02T12:07:04.4472390Z [cluster3] I1102 11:58:27.561190 1 create_or_update.go:32] Creating resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1-submariner-cable-cluster1-172-17-0-6", "namespace":"submariner-operator"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster1-172-17-0-6", "cluster_id":"cluster1", "hostname":"cluster1-worker", "nat_enabled":false, "private_ip":"172.17.0.6", "public_ip":"", "subnets":[]interface {}{"100.1.0.0/16", "10.1.0.0/16"}}}}
  9142. 2020-11-02T12:07:04.4477477Z [cluster3] I1102 11:58:27.566111 1 resource_syncer.go:285] Syncer "broker -> local for *v1.Endpoint" successfully synced "submariner-k8s-broker/cluster1-submariner-cable-cluster1-172-17-0-6"
  9143. 2020-11-02T12:07:04.4482854Z [cluster3] I1102 11:58:27.569159 1 resource_syncer.go:264] Syncer "Tunnel controller" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1-submariner-cable-cluster1-172-17-0-6", "namespace":"submariner-operator", "resourceVersion":"1766", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster1-submariner-cable-cluster1-172-17-0-6", "uid":"f7fe0848-0c40-4b23-b292-631b9b9b97ac"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster1-172-17-0-6", "cluster_id":"cluster1", "hostname":"cluster1-worker", "nat_enabled":false, "private_ip":"172.17.0.6", "public_ip":"", "subnets":[]interface {}{"100.1.0.0/16", "10.1.0.0/16"}}}}
  9144. 2020-11-02T12:07:04.4496655Z [cluster3] I1102 11:58:27.569348 1 tunnel.go:56] Tunnel controller processing added or updated submariner Endpoint object: &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster1-submariner-cable-cluster1-172-17-0-6", GenerateName:"", Namespace:"submariner-operator", SelfLink:"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster1-submariner-cable-cluster1-172-17-0-6", UID:"f7fe0848-0c40-4b23-b292-631b9b9b97ac", ResourceVersion:"1766", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63739915107, loc:(*time.Location)(0x2020aa0)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"submariner-io/clusterID":"cluster1"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster1", CableName:"submariner-cable-cluster1-172-17-0-6", Hostname:"cluster1-worker", Subnets:[]string{"100.1.0.0/16", "10.1.0.0/16"}, PrivateIP:"172.17.0.6", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string(nil)}}
  9145. 2020-11-02T12:07:04.4503375Z [cluster3] I1102 11:58:27.569385 1 cableengine.go:101] Installing Endpoint cable "submariner-cable-cluster1-172-17-0-6"
  9146. 2020-11-02T12:07:04.4505146Z [cluster3] I1102 11:58:27.569392 1 libreswan.go:164] Active connections: [submariner-cable-cluster2-172-17-0-4]
  9147. 2020-11-02T12:07:04.4520386Z [cluster3] I1102 11:58:27.570922 1 resource_syncer.go:264] Syncer "local -> broker for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:27Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster1"}, "name":"cluster1-submariner-cable-cluster1-172-17-0-6", "namespace":"submariner-operator", "resourceVersion":"1766", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster1-submariner-cable-cluster1-172-17-0-6", "uid":"f7fe0848-0c40-4b23-b292-631b9b9b97ac"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster1-172-17-0-6", "cluster_id":"cluster1", "hostname":"cluster1-worker", "nat_enabled":false, "private_ip":"172.17.0.6", "public_ip":"", "subnets":[]interface {}{"100.1.0.0/16", "10.1.0.0/16"}}}}
  9148. 2020-11-02T12:07:04.4528665Z [cluster3] I1102 11:58:27.571015 1 resource_syncer.go:437] Found cluster ID label "cluster1" - not syncing resource "cluster1-submariner-cable-cluster1-172-17-0-6"
  9149. 2020-11-02T12:07:04.4529828Z [cluster3] 002 listening for IKE messages
  9150. 2020-11-02T12:07:04.4530666Z [cluster3] 002 adding interface vx-submariner/vx-submariner (esp-hw-offload not supported by kernel) 240.17.0.7:500
  9151. 2020-11-02T12:07:04.4531687Z [cluster3] 002 adding interface vx-submariner/vx-submariner 240.17.0.7:4500
  9152. 2020-11-02T12:07:04.4532582Z [cluster3] 002 forgetting secrets
  9153. 2020-11-02T12:07:04.4533393Z [cluster3] 002 loading secrets from "/etc/ipsec.secrets"
  9154. 2020-11-02T12:07:04.4535719Z [cluster3] 002 loading secrets from "/etc/ipsec.d/submariner.secrets"
  9155. 2020-11-02T12:07:04.4537465Z [cluster3] I1102 11:58:27.597800 1 libreswan.go:245] Creating connection(s) for {{cluster1 submariner-cable-cluster1-172-17-0-6 cluster1-worker [100.1.0.0/16 10.1.0.0/16] 172.17.0.6 false libreswan map[]}}
  9156. 2020-11-02T12:07:04.4539381Z [cluster3] I1102 11:58:27.597828 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster1-172-17-0-6-0-0 --id 172.17.0.7 --host 172.17.0.7 --client 172.17.0.7/32 --to --id 172.17.0.6 --host 172.17.0.6 --client 172.17.0.6/32]
  9157. 2020-11-02T12:07:04.4542154Z [cluster3] 002 added connection description "submariner-cable-cluster1-172-17-0-6-0-0"
  9158. 2020-11-02T12:07:04.4544105Z [cluster3] 181 "submariner-cable-cluster1-172-17-0-6-0-0" #21: initiating IKEv2 IKE SA
  9159. 2020-11-02T12:07:04.4548524Z [cluster3] I1102 11:58:27.819533 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster1-172-17-0-6-0-1 --id 172.17.0.7 --host 172.17.0.7 --client 172.17.0.7/32 --to --id 172.17.0.6 --host 172.17.0.6 --client 100.1.0.0/16]
  9160. 2020-11-02T12:07:04.4550096Z [cluster3] 002 added connection description "submariner-cable-cluster1-172-17-0-6-0-1"
  9161. 2020-11-02T12:07:04.4553701Z [cluster3] 000 "submariner-cable-cluster1-172-17-0-6-0-1": queuing pending IPsec SA negotiating with 172.17.0.6 IKE SA #21 "submariner-cable-cluster1-172-17-0-6-0-0"
  9162. 2020-11-02T12:07:04.4557231Z [cluster3] I1102 11:58:27.873828 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster1-172-17-0-6-0-2 --id 172.17.0.7 --host 172.17.0.7 --client 172.17.0.7/32 --to --id 172.17.0.6 --host 172.17.0.6 --client 10.1.0.0/16]
  9163. 2020-11-02T12:07:04.4559170Z [cluster3] 002 added connection description "submariner-cable-cluster1-172-17-0-6-0-2"
  9164. 2020-11-02T12:07:04.4562045Z [cluster3] 000 "submariner-cable-cluster1-172-17-0-6-0-2": queuing pending IPsec SA negotiating with 172.17.0.6 IKE SA #21 "submariner-cable-cluster1-172-17-0-6-0-0"
  9165. 2020-11-02T12:07:04.4564369Z [cluster3] I1102 11:58:27.922965 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster1-172-17-0-6-1-0 --id 172.17.0.7 --host 172.17.0.7 --client 100.3.0.0/16 --to --id 172.17.0.6 --host 172.17.0.6 --client 172.17.0.6/32]
  9166. 2020-11-02T12:07:04.4566522Z [cluster3] 002 added connection description "submariner-cable-cluster1-172-17-0-6-1-0"
  9167. 2020-11-02T12:07:04.4568257Z [cluster3] 000 "submariner-cable-cluster1-172-17-0-6-1-0": queuing pending IPsec SA negotiating with 172.17.0.6 IKE SA #21 "submariner-cable-cluster1-172-17-0-6-0-0"
  9168. 2020-11-02T12:07:04.4570756Z [cluster3] I1102 11:58:27.931714 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster1-172-17-0-6-1-1 --id 172.17.0.7 --host 172.17.0.7 --client 100.3.0.0/16 --to --id 172.17.0.6 --host 172.17.0.6 --client 100.1.0.0/16]
  9169. 2020-11-02T12:07:04.4572226Z [cluster3] 002 added connection description "submariner-cable-cluster1-172-17-0-6-1-1"
  9170. 2020-11-02T12:07:04.4575530Z [cluster3] 000 "submariner-cable-cluster1-172-17-0-6-1-1": queuing pending IPsec SA negotiating with 172.17.0.6 IKE SA #21 "submariner-cable-cluster1-172-17-0-6-0-0"
  9171. 2020-11-02T12:07:04.4577890Z [cluster3] I1102 11:58:27.942694 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster1-172-17-0-6-1-2 --id 172.17.0.7 --host 172.17.0.7 --client 100.3.0.0/16 --to --id 172.17.0.6 --host 172.17.0.6 --client 10.1.0.0/16]
  9172. 2020-11-02T12:07:04.4579320Z [cluster3] 002 added connection description "submariner-cable-cluster1-172-17-0-6-1-2"
  9173. 2020-11-02T12:07:04.4580993Z [cluster3] 000 "submariner-cable-cluster1-172-17-0-6-1-2": queuing pending IPsec SA negotiating with 172.17.0.6 IKE SA #21 "submariner-cable-cluster1-172-17-0-6-0-0"
  9174. 2020-11-02T12:07:04.4582950Z [cluster3] I1102 11:58:27.951497 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster1-172-17-0-6-2-0 --id 172.17.0.7 --host 172.17.0.7 --client 10.3.0.0/16 --to --id 172.17.0.6 --host 172.17.0.6 --client 172.17.0.6/32]
  9175. 2020-11-02T12:07:04.4584898Z [cluster3] 002 added connection description "submariner-cable-cluster1-172-17-0-6-2-0"
  9176. 2020-11-02T12:07:04.4588469Z [cluster3] 000 "submariner-cable-cluster1-172-17-0-6-2-0": queuing pending IPsec SA negotiating with 172.17.0.6 IKE SA #21 "submariner-cable-cluster1-172-17-0-6-0-0"
  9177. 2020-11-02T12:07:04.4591085Z [cluster3] I1102 11:58:27.956085 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster1-172-17-0-6-2-1 --id 172.17.0.7 --host 172.17.0.7 --client 10.3.0.0/16 --to --id 172.17.0.6 --host 172.17.0.6 --client 100.1.0.0/16]
  9178. 2020-11-02T12:07:04.4592688Z [cluster3] 002 added connection description "submariner-cable-cluster1-172-17-0-6-2-1"
  9179. 2020-11-02T12:07:04.4594510Z [cluster3] 000 "submariner-cable-cluster1-172-17-0-6-2-1": queuing pending IPsec SA negotiating with 172.17.0.6 IKE SA #21 "submariner-cable-cluster1-172-17-0-6-0-0"
  9180. 2020-11-02T12:07:04.4596582Z [cluster3] I1102 11:58:27.959855 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster1-172-17-0-6-2-2 --id 172.17.0.7 --host 172.17.0.7 --client 10.3.0.0/16 --to --id 172.17.0.6 --host 172.17.0.6 --client 10.1.0.0/16]
  9181. 2020-11-02T12:07:04.4598009Z [cluster3] 002 added connection description "submariner-cable-cluster1-172-17-0-6-2-2"
  9182. 2020-11-02T12:07:04.4600586Z [cluster3] 000 "submariner-cable-cluster1-172-17-0-6-2-2": queuing pending IPsec SA negotiating with 172.17.0.6 IKE SA #21 "submariner-cable-cluster1-172-17-0-6-0-0"
  9183. 2020-11-02T12:07:04.4603152Z [cluster3] I1102 11:58:27.981291 1 cableengine.go:150] Successfully installed Endpoint cable "submariner-cable-cluster1-172-17-0-6" with remote IP 172.17.0.6
  9184. 2020-11-02T12:07:04.4605218Z [cluster3] I1102 11:58:27.981404 1 tunnel.go:68] Tunnel controller successfully installed Endpoint cable submariner-cable-cluster1-172-17-0-6 in the engine
  9185. 2020-11-02T12:07:04.4606743Z [cluster3] I1102 11:58:27.981536 1 resource_syncer.go:352] Syncer "Tunnel controller": transform function returned nil - not syncing - requeue: false
  9186. 2020-11-02T12:07:04.4609472Z [cluster3] I1102 12:01:40.295454 1 resource_syncer.go:294] Syncer "broker -> local for *v1.Endpoint" informed of deleted resource "submariner-k8s-broker/cluster2-submariner-cable-cluster2-172-17-0-4"
  9187. 2020-11-02T12:07:04.4614847Z [cluster3] I1102 12:01:40.295482 1 resource_syncer.go:315] Syncer "broker -> local for *v1.Endpoint" deleting resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:19Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-k8s-broker", "resourceVersion":"2627", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"2d4d0eb4-8d26-4d4c-92fe-e2b47a023c8a"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  9188. 2020-11-02T12:07:04.4619727Z [cluster3] I1102 12:01:40.295595 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  9189. 2020-11-02T12:07:04.4624914Z [cluster3] I1102 12:01:40.295623 1 federator.go:73] Deleting resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T11:58:19Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-operator", "resourceVersion":"2627", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"2d4d0eb4-8d26-4d4c-92fe-e2b47a023c8a"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  9190. 2020-11-02T12:07:04.4630377Z [cluster3] I1102 12:01:40.305773 1 resource_syncer.go:330] Syncer "broker -> local for *v1.Endpoint" successfully deleted "submariner-k8s-broker/cluster2-submariner-cable-cluster2-172-17-0-4"
  9191. 2020-11-02T12:07:04.4632430Z [cluster3] I1102 12:01:40.306195 1 resource_syncer.go:294] Syncer "Tunnel controller" informed of deleted resource "submariner-operator/cluster2-submariner-cable-cluster2-172-17-0-4"
  9192. 2020-11-02T12:07:04.4640563Z [cluster3] I1102 12:01:40.306354 1 tunnel.go:74] Tunnel controller processing removed submariner Endpoint object: &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster2-submariner-cable-cluster2-172-17-0-4", GenerateName:"", Namespace:"submariner-operator", SelfLink:"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", UID:"8c63de0a-c225-4c4f-b602-4d0ec35f7bae", ResourceVersion:"2815", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63739915104, loc:(*time.Location)(0x2020aa0)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"submariner-io/clusterID":"cluster2"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster2", CableName:"submariner-cable-cluster2-172-17-0-4", Hostname:"cluster2-worker", Subnets:[]string{"100.2.0.0/16", "10.2.0.0/16"}, PrivateIP:"172.17.0.4", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string(nil)}}
  9193. 2020-11-02T12:07:04.4652231Z [cluster3] I1102 12:01:40.306391 1 cableengine.go:166] Removing Endpoint cable "submariner-cable-cluster2-172-17-0-4"
  9194. 2020-11-02T12:07:04.4653903Z [cluster3] I1102 12:01:40.306400 1 libreswan.go:300] Deleting connection to {{cluster2 submariner-cable-cluster2-172-17-0-4 cluster2-worker [100.2.0.0/16 10.2.0.0/16] 172.17.0.4 false libreswan map[]}}
  9195. 2020-11-02T12:07:04.4656340Z [cluster3] I1102 12:01:40.306411 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-0-0]
  9196. 2020-11-02T12:07:04.4658281Z [cluster3] I1102 12:01:40.307211 1 resource_syncer.go:294] Syncer "local -> broker for *v1.Endpoint" informed of deleted resource "submariner-operator/cluster2-submariner-cable-cluster2-172-17-0-4"
  9197. 2020-11-02T12:07:04.4660498Z [cluster3] I1102 12:01:40.307231 1 resource_syncer.go:437] Found cluster ID label "cluster2" - not syncing resource "cluster2-submariner-cable-cluster2-172-17-0-4"
  9198. 2020-11-02T12:07:04.4662706Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-0-0": terminating SAs using this connection
  9199. 2020-11-02T12:07:04.4664642Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-0-0": IKE SA is shared - only terminating IPsec SA
  9200. 2020-11-02T12:07:04.4666519Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-0-0" #34: deleting state (STATE_V2_IPSEC_R) aged 5.818s and sending notification
  9201. 2020-11-02T12:07:04.4667796Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-0-0" #34: ESP traffic information: in=0B out=0B
  9202. 2020-11-02T12:07:04.4672572Z [cluster3] I1102 12:01:40.343595 1 resource_syncer.go:264] Syncer "broker -> local for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:01:40Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-k8s-broker", "resourceVersion":"2629", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"bb7e9be5-2d33-4f1d-947e-4a46c6904408"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  9203. 2020-11-02T12:07:04.4679864Z [cluster3] I1102 12:01:40.343714 1 resource_syncer.go:276] Syncing resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:01:40Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-k8s-broker", "resourceVersion":"2629", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"bb7e9be5-2d33-4f1d-947e-4a46c6904408"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  9204. 2020-11-02T12:07:04.4687142Z [cluster3] I1102 12:01:40.343777 1 federator.go:42] In Distribute for &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:01:40Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-k8s-broker", "resourceVersion":"2629", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"bb7e9be5-2d33-4f1d-947e-4a46c6904408"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  9205. 2020-11-02T12:07:04.4691483Z [cluster3] I1102 12:01:40.343835 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  9206. 2020-11-02T12:07:04.4694883Z [cluster3] I1102 12:01:40.355215 1 create_or_update.go:32] Creating resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-operator"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  9207. 2020-11-02T12:07:04.4698950Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-0-0" #13: deleting state (STATE_V2_IPSEC_R) aged 195.745s and sending notification
  9208. 2020-11-02T12:07:04.4700767Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-0-0" #13: ESP traffic information: in=0B out=0B
  9209. 2020-11-02T12:07:04.4702099Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-0-0" #3: deleting state (STATE_V2_IPSEC_I) aged 195.921s and sending notification
  9210. 2020-11-02T12:07:04.4703548Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-0-0" #3: ESP traffic information: in=0B out=0B
  9211. 2020-11-02T12:07:04.4704844Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-0-0" #1: deleting state (STATE_PARENT_I3) aged 195.979s and sending notification
  9212. 2020-11-02T12:07:04.4706808Z [cluster3] 003 "submariner-cable-cluster2-172-17-0-4-0-0" #1: expire pending child #11 STATE_V2_CREATE_I0 of connection "submariner-cable-cluster2-172-17-0-4-0-1" - the parent is going away
  9213. 2020-11-02T12:07:04.4709726Z [cluster3] 003 "submariner-cable-cluster2-172-17-0-4-0-0" #1: expire pending child #10 STATE_V2_CREATE_I0 of connection "submariner-cable-cluster2-172-17-0-4-0-2" - the parent is going away
  9214. 2020-11-02T12:07:04.4712308Z [cluster3] 003 "submariner-cable-cluster2-172-17-0-4-0-0" #1: expire pending child #9 STATE_V2_CREATE_I0 of connection "submariner-cable-cluster2-172-17-0-4-1-0" - the parent is going away
  9215. 2020-11-02T12:07:04.4714550Z [cluster3] 003 "submariner-cable-cluster2-172-17-0-4-0-0" #1: expire pending child #8 STATE_V2_CREATE_I0 of connection "submariner-cable-cluster2-172-17-0-4-1-1" - the parent is going away
  9216. 2020-11-02T12:07:04.4716819Z [cluster3] 003 "submariner-cable-cluster2-172-17-0-4-0-0" #1: expire pending child #7 STATE_V2_CREATE_I0 of connection "submariner-cable-cluster2-172-17-0-4-1-2" - the parent is going away
  9217. 2020-11-02T12:07:04.4719064Z [cluster3] 003 "submariner-cable-cluster2-172-17-0-4-0-0" #1: expire pending child #6 STATE_V2_CREATE_I0 of connection "submariner-cable-cluster2-172-17-0-4-2-0" - the parent is going away
  9218. 2020-11-02T12:07:04.4721310Z [cluster3] 003 "submariner-cable-cluster2-172-17-0-4-0-0" #1: expire pending child #5 STATE_V2_CREATE_I0 of connection "submariner-cable-cluster2-172-17-0-4-2-1" - the parent is going away
  9219. 2020-11-02T12:07:04.4722951Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-0-1": terminating SAs using this connection
  9220. 2020-11-02T12:07:04.4724542Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-0-1" #37: deleting state (STATE_V2_IPSEC_R) aged 5.801s and sending notification
  9221. 2020-11-02T12:07:04.4726029Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-0-1" #37: ESP traffic information: in=0B out=0B
  9222. 2020-11-02T12:07:04.4727318Z [cluster3] I1102 12:01:40.361987 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-0-1]
  9223. 2020-11-02T12:07:04.4729149Z [cluster3] I1102 12:01:40.375983 1 resource_syncer.go:285] Syncer "broker -> local for *v1.Endpoint" successfully synced "submariner-k8s-broker/cluster2-submariner-cable-cluster2-172-17-0-4"
  9224. 2020-11-02T12:07:04.4735356Z [cluster3] I1102 12:01:40.380597 1 resource_syncer.go:264] Syncer "local -> broker for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:01:40Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-operator", "resourceVersion":"2817", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"94d2ede3-9111-44d5-a192-56c2299fcfe7"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  9225. 2020-11-02T12:07:04.4742158Z [cluster3] I1102 12:01:40.380706 1 resource_syncer.go:437] Found cluster ID label "cluster2" - not syncing resource "cluster2-submariner-cable-cluster2-172-17-0-4"
  9226. 2020-11-02T12:07:04.4743866Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-0-1" #15: deleting state (STATE_V2_IPSEC_R) aged 195.758s and sending notification
  9227. 2020-11-02T12:07:04.4746226Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-0-1" #15: ESP traffic information: in=0B out=0B
  9228. 2020-11-02T12:07:04.4747729Z [cluster3] I1102 12:01:40.400320 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-0-2]
  9229. 2020-11-02T12:07:04.4750173Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-0-2": terminating SAs using this connection
  9230. 2020-11-02T12:07:04.4751580Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-0-2" #40: deleting state (STATE_V2_IPSEC_R) aged 5.791s and sending notification
  9231. 2020-11-02T12:07:04.4753066Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-0-2" #40: ESP traffic information: in=0B out=0B
  9232. 2020-11-02T12:07:04.4755728Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-0-2" #18: deleting state (STATE_V2_IPSEC_R) aged 195.704s and sending notification
  9233. 2020-11-02T12:07:04.4757321Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-0-2" #18: ESP traffic information: in=0B out=0B
  9234. 2020-11-02T12:07:04.4758699Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-1-0": terminating SAs using this connection
  9235. 2020-11-02T12:07:04.4760002Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-1-0" #35: deleting state (STATE_V2_IPSEC_R) aged 5.936s and sending notification
  9236. 2020-11-02T12:07:04.4761469Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-1-0" #35: ESP traffic information: in=0B out=0B
  9237. 2020-11-02T12:07:04.4762763Z [cluster3] I1102 12:01:40.438072 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-1-0]
  9238. 2020-11-02T12:07:04.4765433Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-1-0" #12: deleting state (STATE_V2_IPSEC_R) aged 195.909s and sending notification
  9239. 2020-11-02T12:07:04.4766792Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-1-0" #12: ESP traffic information: in=0B out=0B
  9240. 2020-11-02T12:07:04.4768238Z [cluster3] I1102 12:01:40.469848 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-1-1]
  9241. 2020-11-02T12:07:04.4769466Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-1-1": terminating SAs using this connection
  9242. 2020-11-02T12:07:04.4770960Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-1-1" #38: deleting state (STATE_V2_IPSEC_R) aged 5.889s and sending notification
  9243. 2020-11-02T12:07:04.4772252Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-1-1" #38: ESP traffic information: in=0B out=0B
  9244. 2020-11-02T12:07:04.4773745Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-1-1" #16: deleting state (STATE_V2_IPSEC_R) aged 195.827s and sending notification
  9245. 2020-11-02T12:07:04.4775174Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-1-1" #16: ESP traffic information: in=0B out=0B
  9246. 2020-11-02T12:07:04.4776471Z [cluster3] I1102 12:01:40.513804 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-1-2]
  9247. 2020-11-02T12:07:04.4777741Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-1-2": terminating SAs using this connection
  9248. 2020-11-02T12:07:04.4779205Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-1-2" #41: deleting state (STATE_V2_IPSEC_R) aged 5.887s and sending notification
  9249. 2020-11-02T12:07:04.4780633Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-1-2" #41: ESP traffic information: in=0B out=0B
  9250. 2020-11-02T12:07:04.4781959Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-1-2" #19: deleting state (STATE_V2_IPSEC_R) aged 195.797s and sending notification
  9251. 2020-11-02T12:07:04.4784454Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-1-2" #19: ESP traffic information: in=10KB out=10KB
  9252. 2020-11-02T12:07:04.4786097Z [cluster3] I1102 12:01:40.579467 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-2-0]
  9253. 2020-11-02T12:07:04.4787601Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-2-0": terminating SAs using this connection
  9254. 2020-11-02T12:07:04.4789458Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-2-0" #36: deleting state (STATE_V2_IPSEC_R) aged 6.031s and sending notification
  9255. 2020-11-02T12:07:04.4794333Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-2-0" #36: ESP traffic information: in=0B out=0B
  9256. 2020-11-02T12:07:04.4795893Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-2-0" #14: deleting state (STATE_V2_IPSEC_R) aged 195.997s and sending notification
  9257. 2020-11-02T12:07:04.4797189Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-2-0" #14: ESP traffic information: in=0B out=0B
  9258. 2020-11-02T12:07:04.4799362Z [cluster3] I1102 12:01:40.656701 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-2-1]
  9259. 2020-11-02T12:07:04.4801329Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-2-1": terminating SAs using this connection
  9260. 2020-11-02T12:07:04.4803568Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-2-1" #39: deleting state (STATE_V2_IPSEC_R) aged 6.062s and sending notification
  9261. 2020-11-02T12:07:04.4805761Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-2-1" #39: ESP traffic information: in=0B out=0B
  9262. 2020-11-02T12:07:04.4807098Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-2-1" #17: deleting state (STATE_V2_IPSEC_R) aged 195.968s and sending notification
  9263. 2020-11-02T12:07:04.4808545Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-2-1" #17: ESP traffic information: in=454B out=397B
  9264. 2020-11-02T12:07:04.4809802Z [cluster3] I1102 12:01:40.728593 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-2-2]
  9265. 2020-11-02T12:07:04.4811224Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-2-2": terminating SAs using this connection
  9266. 2020-11-02T12:07:04.4812567Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-2-2" #42: deleting state (STATE_V2_IPSEC_R) aged 6.085s and sending notification
  9267. 2020-11-02T12:07:04.4814856Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-2-2" #42: ESP traffic information: in=0B out=0B
  9268. 2020-11-02T12:07:04.4816902Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-2-2" #20: deleting state (STATE_V2_IPSEC_R) aged 195.929s and sending notification
  9269. 2020-11-02T12:07:04.4818254Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-2-2" #20: ESP traffic information: in=14KB out=13KB
  9270. 2020-11-02T12:07:04.4819748Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-2-2" #33: deleting state (STATE_PARENT_R2) aged 6.304s and sending notification
  9271. 2020-11-02T12:07:04.4821470Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-2-2" #2: deleting state (STATE_PARENT_R2) aged 196.319s and sending notification
  9272. 2020-11-02T12:07:04.4822864Z [cluster3] I1102 12:01:40.791579 1 cableengine.go:176] Successfully removed Endpoint cable "submariner-cable-cluster2-172-17-0-4"
  9273. 2020-11-02T12:07:04.4824304Z [cluster3] I1102 12:01:40.791809 1 tunnel.go:85] Tunnel controller successfully removed Endpoint cable submariner-cable-cluster2-172-17-0-4 from the engine
  9274. 2020-11-02T12:07:04.4825761Z [cluster3] I1102 12:01:40.791997 1 resource_syncer.go:352] Syncer "Tunnel controller": transform function returned nil - not syncing - requeue: false
  9275. 2020-11-02T12:07:04.4833100Z [cluster3] I1102 12:01:40.792156 1 resource_syncer.go:264] Syncer "Tunnel controller" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:01:40Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-operator", "resourceVersion":"2817", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"94d2ede3-9111-44d5-a192-56c2299fcfe7"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  9276. 2020-11-02T12:07:04.4844226Z [cluster3] I1102 12:01:40.792708 1 tunnel.go:56] Tunnel controller processing added or updated submariner Endpoint object: &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster2-submariner-cable-cluster2-172-17-0-4", GenerateName:"", Namespace:"submariner-operator", SelfLink:"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", UID:"94d2ede3-9111-44d5-a192-56c2299fcfe7", ResourceVersion:"2817", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63739915300, loc:(*time.Location)(0x2020aa0)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"submariner-io/clusterID":"cluster2"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster2", CableName:"submariner-cable-cluster2-172-17-0-4", Hostname:"cluster2-worker", Subnets:[]string{"100.2.0.0/16", "10.2.0.0/16"}, PrivateIP:"172.17.0.4", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string(nil)}}
  9277. 2020-11-02T12:07:04.4850470Z [cluster3] I1102 12:01:40.792974 1 cableengine.go:101] Installing Endpoint cable "submariner-cable-cluster2-172-17-0-4"
  9278. 2020-11-02T12:07:04.4851774Z [cluster3] I1102 12:01:40.793054 1 libreswan.go:164] Active connections: [submariner-cable-cluster1-172-17-0-6]
  9279. 2020-11-02T12:07:04.4853937Z [cluster3] I1102 12:01:40.797116 1 libreswan.go:245] Creating connection(s) for {{cluster2 submariner-cable-cluster2-172-17-0-4 cluster2-worker [100.2.0.0/16 10.2.0.0/16] 172.17.0.4 false libreswan map[]}}
  9280. 2020-11-02T12:07:04.4856387Z [cluster3] I1102 12:01:40.797176 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-0-0 --id 172.17.0.7 --host 172.17.0.7 --client 172.17.0.7/32 --to --id 172.17.0.4 --host 172.17.0.4 --client 172.17.0.4/32]
  9281. 2020-11-02T12:07:04.4857793Z [cluster3] 002 listening for IKE messages
  9282. 2020-11-02T12:07:04.4858340Z [cluster3] 002 forgetting secrets
  9283. 2020-11-02T12:07:04.4858767Z [cluster3] 002 loading secrets from "/etc/ipsec.secrets"
  9284. 2020-11-02T12:07:04.4859329Z [cluster3] 002 loading secrets from "/etc/ipsec.d/submariner.secrets"
  9285. 2020-11-02T12:07:04.4860575Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-4-0-0"
  9286. 2020-11-02T12:07:04.4861825Z [cluster3] 181 "submariner-cable-cluster2-172-17-0-4-0-0" #43: initiating IKEv2 IKE SA
  9287. 2020-11-02T12:07:04.4863556Z [cluster3] I1102 12:01:40.870894 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-0-1 --id 172.17.0.7 --host 172.17.0.7 --client 172.17.0.7/32 --to --id 172.17.0.4 --host 172.17.0.4 --client 100.2.0.0/16]
  9288. 2020-11-02T12:07:04.4865320Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-4-0-1"
  9289. 2020-11-02T12:07:04.4867524Z [cluster3] 000 "submariner-cable-cluster2-172-17-0-4-0-1": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #43 "submariner-cable-cluster2-172-17-0-4-0-0"
  9290. 2020-11-02T12:07:04.4869810Z [cluster3] I1102 12:01:40.961191 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-0-2 --id 172.17.0.7 --host 172.17.0.7 --client 172.17.0.7/32 --to --id 172.17.0.4 --host 172.17.0.4 --client 10.2.0.0/16]
  9291. 2020-11-02T12:07:04.4871564Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-4-0-2"
  9292. 2020-11-02T12:07:04.4875481Z [cluster3] 000 "submariner-cable-cluster2-172-17-0-4-0-2": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #43 "submariner-cable-cluster2-172-17-0-4-0-0"
  9293. 2020-11-02T12:07:04.4878236Z [cluster3] I1102 12:01:41.056320 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-1-0 --id 172.17.0.7 --host 172.17.0.7 --client 100.3.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 172.17.0.4/32]
  9294. 2020-11-02T12:07:04.4880464Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-4-1-0"
  9295. 2020-11-02T12:07:04.4882173Z [cluster3] 000 "submariner-cable-cluster2-172-17-0-4-1-0": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #43 "submariner-cable-cluster2-172-17-0-4-0-0"
  9296. 2020-11-02T12:07:04.4884258Z [cluster3] I1102 12:01:41.062437 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-1-1 --id 172.17.0.7 --host 172.17.0.7 --client 100.3.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 100.2.0.0/16]
  9297. 2020-11-02T12:07:04.4885727Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-4-1-1"
  9298. 2020-11-02T12:07:04.4887550Z [cluster3] 000 "submariner-cable-cluster2-172-17-0-4-1-1": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #43 "submariner-cable-cluster2-172-17-0-4-0-0"
  9299. 2020-11-02T12:07:04.4889945Z [cluster3] I1102 12:01:41.066991 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-1-2 --id 172.17.0.7 --host 172.17.0.7 --client 100.3.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 10.2.0.0/16]
  9300. 2020-11-02T12:07:04.4891537Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-4-1-2"
  9301. 2020-11-02T12:07:04.4893223Z [cluster3] 000 "submariner-cable-cluster2-172-17-0-4-1-2": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #43 "submariner-cable-cluster2-172-17-0-4-0-0"
  9302. 2020-11-02T12:07:04.4896082Z [cluster3] I1102 12:01:41.070095 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-2-0 --id 172.17.0.7 --host 172.17.0.7 --client 10.3.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 172.17.0.4/32]
  9303. 2020-11-02T12:07:04.4898609Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-4-2-0"
  9304. 2020-11-02T12:07:04.4900640Z [cluster3] 000 "submariner-cable-cluster2-172-17-0-4-2-0": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #43 "submariner-cable-cluster2-172-17-0-4-0-0"
  9305. 2020-11-02T12:07:04.4902985Z [cluster3] I1102 12:01:41.073357 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-2-1 --id 172.17.0.7 --host 172.17.0.7 --client 10.3.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 100.2.0.0/16]
  9306. 2020-11-02T12:07:04.4905059Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-4-2-1"
  9307. 2020-11-02T12:07:04.4910514Z [cluster3] 000 "submariner-cable-cluster2-172-17-0-4-2-1": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #43 "submariner-cable-cluster2-172-17-0-4-0-0"
  9308. 2020-11-02T12:07:04.4913491Z [cluster3] I1102 12:01:41.078103 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-4-2-2 --id 172.17.0.7 --host 172.17.0.7 --client 10.3.0.0/16 --to --id 172.17.0.4 --host 172.17.0.4 --client 10.2.0.0/16]
  9309. 2020-11-02T12:07:04.4914926Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-4-2-2"
  9310. 2020-11-02T12:07:04.4916565Z [cluster3] 000 "submariner-cable-cluster2-172-17-0-4-2-2": queuing pending IPsec SA negotiating with 172.17.0.4 IKE SA #43 "submariner-cable-cluster2-172-17-0-4-0-0"
  9311. 2020-11-02T12:07:04.4918297Z [cluster3] I1102 12:01:41.080945 1 cableengine.go:150] Successfully installed Endpoint cable "submariner-cable-cluster2-172-17-0-4" with remote IP 172.17.0.4
  9312. 2020-11-02T12:07:04.4919975Z [cluster3] I1102 12:01:41.081100 1 tunnel.go:68] Tunnel controller successfully installed Endpoint cable submariner-cable-cluster2-172-17-0-4 in the engine
  9313. 2020-11-02T12:07:04.4921459Z [cluster3] I1102 12:01:41.081328 1 resource_syncer.go:352] Syncer "Tunnel controller": transform function returned nil - not syncing - requeue: false
  9314. 2020-11-02T12:07:04.4926222Z [cluster3] I1102 12:01:41.082759 1 libreswan.go:149] Connection "submariner-cable-cluster2-172-17-0-4" not found in active connections obtained from whack: [submariner-cable-cluster1-172-17-0-6-2-1 submariner-cable-cluster1-172-17-0-6-0-1 submariner-cable-cluster1-172-17-0-6-0-2 submariner-cable-cluster1-172-17-0-6-1-0 submariner-cable-cluster1-172-17-0-6-1-1 submariner-cable-cluster1-172-17-0-6-1-2 submariner-cable-cluster1-172-17-0-6-2-0 submariner-cable-cluster1-172-17-0-6-0-0 submariner-cable-cluster1-172-17-0-6-2-2]
  9315. 2020-11-02T12:07:04.4931093Z [cluster3] I1102 12:05:30.605451 1 resource_syncer.go:294] Syncer "broker -> local for *v1.Endpoint" informed of deleted resource "submariner-k8s-broker/cluster2-submariner-cable-cluster2-172-17-0-4"
  9316. 2020-11-02T12:07:04.4936919Z [cluster3] I1102 12:05:30.605567 1 resource_syncer.go:315] Syncer "broker -> local for *v1.Endpoint" deleting resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:01:40Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-k8s-broker", "resourceVersion":"3453", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"bb7e9be5-2d33-4f1d-947e-4a46c6904408"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  9317. 2020-11-02T12:07:04.4941270Z [cluster3] I1102 12:05:30.605804 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  9318. 2020-11-02T12:07:04.4945897Z [cluster3] I1102 12:05:30.605817 1 federator.go:73] Deleting resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:01:40Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-4", "namespace":"submariner-operator", "resourceVersion":"3453", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", "uid":"bb7e9be5-2d33-4f1d-947e-4a46c6904408"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-4", "cluster_id":"cluster2", "hostname":"cluster2-worker", "nat_enabled":false, "private_ip":"172.17.0.4", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  9319. 2020-11-02T12:07:04.4950272Z [cluster3] I1102 12:05:30.620959 1 resource_syncer.go:294] Syncer "Tunnel controller" informed of deleted resource "submariner-operator/cluster2-submariner-cable-cluster2-172-17-0-4"
  9320. 2020-11-02T12:07:04.4958073Z [cluster3] I1102 12:05:30.621166 1 tunnel.go:74] Tunnel controller processing removed submariner Endpoint object: &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster2-submariner-cable-cluster2-172-17-0-4", GenerateName:"", Namespace:"submariner-operator", SelfLink:"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4", UID:"94d2ede3-9111-44d5-a192-56c2299fcfe7", ResourceVersion:"3680", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63739915300, loc:(*time.Location)(0x2020aa0)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"submariner-io/clusterID":"cluster2"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster2", CableName:"submariner-cable-cluster2-172-17-0-4", Hostname:"cluster2-worker", Subnets:[]string{"100.2.0.0/16", "10.2.0.0/16"}, PrivateIP:"172.17.0.4", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string(nil)}}
  9321. 2020-11-02T12:07:04.4964942Z [cluster3] I1102 12:05:30.621254 1 cableengine.go:166] Removing Endpoint cable "submariner-cable-cluster2-172-17-0-4"
  9322. 2020-11-02T12:07:04.4966515Z [cluster3] I1102 12:05:30.621265 1 libreswan.go:300] Deleting connection to {{cluster2 submariner-cable-cluster2-172-17-0-4 cluster2-worker [100.2.0.0/16 10.2.0.0/16] 172.17.0.4 false libreswan map[]}}
  9323. 2020-11-02T12:07:04.4968216Z [cluster3] I1102 12:05:30.621277 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-0-0]
  9324. 2020-11-02T12:07:04.4970166Z [cluster3] I1102 12:05:30.622032 1 resource_syncer.go:330] Syncer "broker -> local for *v1.Endpoint" successfully deleted "submariner-k8s-broker/cluster2-submariner-cable-cluster2-172-17-0-4"
  9325. 2020-11-02T12:07:04.4973180Z [cluster3] I1102 12:05:30.622234 1 resource_syncer.go:294] Syncer "local -> broker for *v1.Endpoint" informed of deleted resource "submariner-operator/cluster2-submariner-cable-cluster2-172-17-0-4"
  9326. 2020-11-02T12:07:04.4975242Z [cluster3] I1102 12:05:30.622247 1 resource_syncer.go:437] Found cluster ID label "cluster2" - not syncing resource "cluster2-submariner-cable-cluster2-172-17-0-4"
  9327. 2020-11-02T12:07:04.4978447Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-0-0": terminating SAs using this connection
  9328. 2020-11-02T12:07:04.4981156Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-0-0": IKE SA is shared - only terminating IPsec SA
  9329. 2020-11-02T12:07:04.4982815Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-0-0" #53: deleting state (STATE_V2_IPSEC_I) aged 229.438s and sending notification
  9330. 2020-11-02T12:07:04.4984539Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-0-0" #53: ESP traffic information: in=0B out=0B
  9331. 2020-11-02T12:07:04.4986093Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-0-0" #54: deleting state (STATE_V2_IPSEC_R) aged 229.441s and sending notification
  9332. 2020-11-02T12:07:04.4987437Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-0-0" #54: ESP traffic information: in=0B out=0B
  9333. 2020-11-02T12:07:04.4988977Z [cluster3] I1102 12:05:30.646060 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-0-1]
  9334. 2020-11-02T12:07:04.4990438Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-0-0" #43: deleting state (STATE_PARENT_I3) aged 229.776s and sending notification
  9335. 2020-11-02T12:07:04.4991991Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-0-1": terminating SAs using this connection
  9336. 2020-11-02T12:07:04.4993592Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-0-1" #52: deleting state (STATE_V2_IPSEC_I) aged 229.464s and NOT sending notification
  9337. 2020-11-02T12:07:04.4994989Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-0-1" #52: ESP traffic information: in=0B out=0B
  9338. 2020-11-02T12:07:04.5000121Z [cluster3] I1102 12:05:30.669113 1 resource_syncer.go:264] Syncer "broker -> local for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:05:30Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-11", "namespace":"submariner-k8s-broker", "resourceVersion":"3455", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-11", "uid":"59c1f9b2-d440-4647-9bdd-1450a56598e9"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-11", "cluster_id":"cluster2", "hostname":"cluster2-worker2", "nat_enabled":false, "private_ip":"172.17.0.11", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  9339. 2020-11-02T12:07:04.5017033Z [cluster3] I1102 12:05:30.669229 1 resource_syncer.go:276] Syncing resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:05:30Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-11", "namespace":"submariner-k8s-broker", "resourceVersion":"3455", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-11", "uid":"59c1f9b2-d440-4647-9bdd-1450a56598e9"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-11", "cluster_id":"cluster2", "hostname":"cluster2-worker2", "nat_enabled":false, "private_ip":"172.17.0.11", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  9340. 2020-11-02T12:07:04.5029165Z [cluster3] I1102 12:05:30.669460 1 federator.go:42] In Distribute for &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:05:30Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-11", "namespace":"submariner-k8s-broker", "resourceVersion":"3455", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-k8s-broker/endpoints/cluster2-submariner-cable-cluster2-172-17-0-11", "uid":"59c1f9b2-d440-4647-9bdd-1450a56598e9"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-11", "cluster_id":"cluster2", "hostname":"cluster2-worker2", "nat_enabled":false, "private_ip":"172.17.0.11", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  9341. 2020-11-02T12:07:04.5033356Z [cluster3] I1102 12:05:30.669577 1 helpers.go:75] Found schema.GroupVersionResource{Group:"submariner.io", Version:"v1", Resource:"endpoints"}
  9342. 2020-11-02T12:07:04.5037112Z [cluster3] I1102 12:05:30.671792 1 create_or_update.go:32] Creating resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-11", "namespace":"submariner-operator"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-11", "cluster_id":"cluster2", "hostname":"cluster2-worker2", "nat_enabled":false, "private_ip":"172.17.0.11", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  9343. 2020-11-02T12:07:04.5040911Z [cluster3] I1102 12:05:30.679413 1 resource_syncer.go:285] Syncer "broker -> local for *v1.Endpoint" successfully synced "submariner-k8s-broker/cluster2-submariner-cable-cluster2-172-17-0-11"
  9344. 2020-11-02T12:07:04.5042688Z [cluster3] I1102 12:05:30.682341 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-0-2]
  9345. 2020-11-02T12:07:04.5049133Z [cluster3] I1102 12:05:30.685289 1 resource_syncer.go:264] Syncer "local -> broker for *v1.Endpoint" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:05:30Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-11", "namespace":"submariner-operator", "resourceVersion":"3681", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-11", "uid":"a0fa1fb5-d7af-4699-b15c-f9559eb39523"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-11", "cluster_id":"cluster2", "hostname":"cluster2-worker2", "nat_enabled":false, "private_ip":"172.17.0.11", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  9346. 2020-11-02T12:07:04.5054477Z [cluster3] I1102 12:05:30.685454 1 resource_syncer.go:437] Found cluster ID label "cluster2" - not syncing resource "cluster2-submariner-cable-cluster2-172-17-0-11"
  9347. 2020-11-02T12:07:04.5056141Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-0-2": terminating SAs using this connection
  9348. 2020-11-02T12:07:04.5057510Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-0-2" #51: deleting state (STATE_V2_IPSEC_I) aged 229.511s and NOT sending notification
  9349. 2020-11-02T12:07:04.5059182Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-0-2" #51: ESP traffic information: in=0B out=0B
  9350. 2020-11-02T12:07:04.5060972Z [cluster3] I1102 12:05:30.720279 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-1-0]
  9351. 2020-11-02T12:07:04.5062817Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-1-0": terminating SAs using this connection
  9352. 2020-11-02T12:07:04.5065259Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-1-0" #50: deleting state (STATE_V2_IPSEC_I) aged 229.537s and NOT sending notification
  9353. 2020-11-02T12:07:04.5066898Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-1-0" #50: ESP traffic information: in=0B out=0B
  9354. 2020-11-02T12:07:04.5068419Z [cluster3] I1102 12:05:30.739195 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-1-1]
  9355. 2020-11-02T12:07:04.5069914Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-1-1": terminating SAs using this connection
  9356. 2020-11-02T12:07:04.5071852Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-1-1" #49: deleting state (STATE_V2_IPSEC_I) aged 229.556s and NOT sending notification
  9357. 2020-11-02T12:07:04.5075629Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-1-1" #49: ESP traffic information: in=0B out=0B
  9358. 2020-11-02T12:07:04.5077262Z [cluster3] I1102 12:05:30.756679 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-1-2]
  9359. 2020-11-02T12:07:04.5078535Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-1-2": terminating SAs using this connection
  9360. 2020-11-02T12:07:04.5080903Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-1-2" #48: deleting state (STATE_V2_IPSEC_I) aged 229.575s and NOT sending notification
  9361. 2020-11-02T12:07:04.5082461Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-1-2" #48: ESP traffic information: in=0B out=0B
  9362. 2020-11-02T12:07:04.5083976Z [cluster3] I1102 12:05:30.778641 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-2-0]
  9363. 2020-11-02T12:07:04.5085257Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-2-0": terminating SAs using this connection
  9364. 2020-11-02T12:07:04.5088377Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-2-0" #47: deleting state (STATE_V2_IPSEC_I) aged 229.595s and NOT sending notification
  9365. 2020-11-02T12:07:04.5090045Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-2-0" #47: ESP traffic information: in=0B out=0B
  9366. 2020-11-02T12:07:04.5091992Z [cluster3] I1102 12:05:30.824490 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-2-1]
  9367. 2020-11-02T12:07:04.5093558Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-2-1": terminating SAs using this connection
  9368. 2020-11-02T12:07:04.5096002Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-2-1" #46: deleting state (STATE_V2_IPSEC_I) aged 229.641s and NOT sending notification
  9369. 2020-11-02T12:07:04.5098159Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-2-1" #46: ESP traffic information: in=0B out=0B
  9370. 2020-11-02T12:07:04.5099611Z [cluster3] I1102 12:05:30.877848 1 libreswan.go:312] Whacking with [--delete --name submariner-cable-cluster2-172-17-0-4-2-2]
  9371. 2020-11-02T12:07:04.5101279Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-2-2": terminating SAs using this connection
  9372. 2020-11-02T12:07:04.5103078Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-2-2" #45: deleting state (STATE_V2_IPSEC_I) aged 229.747s and NOT sending notification
  9373. 2020-11-02T12:07:04.5105495Z [cluster3] 005 "submariner-cable-cluster2-172-17-0-4-2-2" #45: ESP traffic information: in=0B out=2KB
  9374. 2020-11-02T12:07:04.5107256Z [cluster3] 002 "submariner-cable-cluster2-172-17-0-4-2-2" #44: deleting state (STATE_PARENT_R2) aged 229.772s and sending notification
  9375. 2020-11-02T12:07:04.5109419Z [cluster3] I1102 12:05:30.927378 1 cableengine.go:176] Successfully removed Endpoint cable "submariner-cable-cluster2-172-17-0-4"
  9376. 2020-11-02T12:07:04.5112523Z [cluster3] I1102 12:05:30.927572 1 tunnel.go:85] Tunnel controller successfully removed Endpoint cable submariner-cable-cluster2-172-17-0-4 from the engine
  9377. 2020-11-02T12:07:04.5115299Z [cluster3] I1102 12:05:30.927686 1 resource_syncer.go:352] Syncer "Tunnel controller": transform function returned nil - not syncing - requeue: false
  9378. 2020-11-02T12:07:04.5120047Z [cluster3] I1102 12:05:30.927801 1 resource_syncer.go:264] Syncer "Tunnel controller" retrieved created resource: &unstructured.Unstructured{Object:map[string]interface {}{"apiVersion":"submariner.io/v1", "kind":"Endpoint", "metadata":map[string]interface {}{"creationTimestamp":"2020-11-02T12:05:30Z", "generation":1, "labels":map[string]interface {}{"submariner-io/clusterID":"cluster2"}, "name":"cluster2-submariner-cable-cluster2-172-17-0-11", "namespace":"submariner-operator", "resourceVersion":"3681", "selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-11", "uid":"a0fa1fb5-d7af-4699-b15c-f9559eb39523"}, "spec":map[string]interface {}{"backend":"libreswan", "cable_name":"submariner-cable-cluster2-172-17-0-11", "cluster_id":"cluster2", "hostname":"cluster2-worker2", "nat_enabled":false, "private_ip":"172.17.0.11", "public_ip":"", "subnets":[]interface {}{"100.2.0.0/16", "10.2.0.0/16"}}}}
  9379. 2020-11-02T12:07:04.5129609Z [cluster3] I1102 12:05:30.928142 1 tunnel.go:56] Tunnel controller processing added or updated submariner Endpoint object: &v1.Endpoint{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"cluster2-submariner-cable-cluster2-172-17-0-11", GenerateName:"", Namespace:"submariner-operator", SelfLink:"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-11", UID:"a0fa1fb5-d7af-4699-b15c-f9559eb39523", ResourceVersion:"3681", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63739915530, loc:(*time.Location)(0x2020aa0)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"submariner-io/clusterID":"cluster2"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.EndpointSpec{ClusterID:"cluster2", CableName:"submariner-cable-cluster2-172-17-0-11", Hostname:"cluster2-worker2", Subnets:[]string{"100.2.0.0/16", "10.2.0.0/16"}, PrivateIP:"172.17.0.11", PublicIP:"", NATEnabled:false, Backend:"libreswan", BackendConfig:map[string]string(nil)}}
  9380. 2020-11-02T12:07:04.5135303Z [cluster3] I1102 12:05:30.928279 1 cableengine.go:101] Installing Endpoint cable "submariner-cable-cluster2-172-17-0-11"
  9381. 2020-11-02T12:07:04.5137669Z [cluster3] I1102 12:05:30.928455 1 libreswan.go:164] Active connections: [submariner-cable-cluster1-172-17-0-6]
  9382. 2020-11-02T12:07:04.5138408Z [cluster3] 002 listening for IKE messages
  9383. 2020-11-02T12:07:04.5138762Z [cluster3] 002 forgetting secrets
  9384. 2020-11-02T12:07:04.5139294Z [cluster3] 002 loading secrets from "/etc/ipsec.secrets"
  9385. 2020-11-02T12:07:04.5139820Z [cluster3] 002 loading secrets from "/etc/ipsec.d/submariner.secrets"
  9386. 2020-11-02T12:07:04.5141870Z [cluster3] I1102 12:05:30.935773 1 libreswan.go:245] Creating connection(s) for {{cluster2 submariner-cable-cluster2-172-17-0-11 cluster2-worker2 [100.2.0.0/16 10.2.0.0/16] 172.17.0.11 false libreswan map[]}}
  9387. 2020-11-02T12:07:04.5143977Z [cluster3] I1102 12:05:30.935807 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-11-0-0 --id 172.17.0.7 --host 172.17.0.7 --client 172.17.0.7/32 --to --id 172.17.0.11 --host 172.17.0.11 --client 172.17.0.11/32]
  9388. 2020-11-02T12:07:04.5145566Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-11-0-0"
  9389. 2020-11-02T12:07:04.5146791Z [cluster3] 181 "submariner-cable-cluster2-172-17-0-11-0-0" #56: initiating IKEv2 IKE SA
  9390. 2020-11-02T12:07:04.5148471Z [cluster3] I1102 12:05:30.989012 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-11-0-1 --id 172.17.0.7 --host 172.17.0.7 --client 172.17.0.7/32 --to --id 172.17.0.11 --host 172.17.0.11 --client 100.2.0.0/16]
  9391. 2020-11-02T12:07:04.5149844Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-11-0-1"
  9392. 2020-11-02T12:07:04.5151736Z [cluster3] I1102 12:05:31.063878 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-11-0-2 --id 172.17.0.7 --host 172.17.0.7 --client 172.17.0.7/32 --to --id 172.17.0.11 --host 172.17.0.11 --client 10.2.0.0/16]
  9393. 2020-11-02T12:07:04.5154064Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-11-0-2"
  9394. 2020-11-02T12:07:04.5155600Z [cluster3] I1102 12:05:31.169085 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-11-1-0 --id 172.17.0.7 --host 172.17.0.7 --client 100.3.0.0/16 --to --id 172.17.0.11 --host 172.17.0.11 --client 172.17.0.11/32]
  9395. 2020-11-02T12:07:04.5156973Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-11-1-0"
  9396. 2020-11-02T12:07:04.5158181Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-11-1-1"
  9397. 2020-11-02T12:07:04.5159705Z [cluster3] I1102 12:05:31.180927 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-11-1-1 --id 172.17.0.7 --host 172.17.0.7 --client 100.3.0.0/16 --to --id 172.17.0.11 --host 172.17.0.11 --client 100.2.0.0/16]
  9398. 2020-11-02T12:07:04.5161631Z [cluster3] I1102 12:05:31.191970 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-11-1-2 --id 172.17.0.7 --host 172.17.0.7 --client 100.3.0.0/16 --to --id 172.17.0.11 --host 172.17.0.11 --client 10.2.0.0/16]
  9399. 2020-11-02T12:07:04.5165213Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-11-1-2"
  9400. 2020-11-02T12:07:04.5167036Z [cluster3] I1102 12:05:31.195767 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-11-2-0 --id 172.17.0.7 --host 172.17.0.7 --client 10.3.0.0/16 --to --id 172.17.0.11 --host 172.17.0.11 --client 172.17.0.11/32]
  9401. 2020-11-02T12:07:04.5168761Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-11-2-0"
  9402. 2020-11-02T12:07:04.5171384Z [submariner]$ [cluster3] kubectl get pods --selector=app=submariner-routeagent -n submariner-operator -o jsonpath={.items[*].metadata.name}
  9403. 2020-11-02T12:07:04.5172888Z [submariner]$ [cluster3] kubectl get pods --selector=app=submariner-routeagent -n submariner-operator -o jsonpath={.items[*].metadata.name}
  9404. 2020-11-02T12:07:04.5174451Z [submariner]$ [cluster3] command kubectl --context=cluster3 get pods --selector=app=submariner-routeagent -n submariner-operator -o jsonpath={.items[*].metadata.name}
  9405. 2020-11-02T12:07:04.5175976Z [submariner]$ [cluster3] kubectl --context=cluster3 get pods --selector=app=submariner-routeagent -n submariner-operator -o jsonpath={.items[*].metadata.name}
  9406. 2020-11-02T12:07:04.7008893Z [submariner]$ [cluster3] kubectl -n submariner-operator logs submariner-routeagent-46pd2
  9407. 2020-11-02T12:07:04.7018303Z [submariner]$ [cluster3] kubectl -n submariner-operator logs submariner-routeagent-46pd2
  9408. 2020-11-02T12:07:04.7032785Z [submariner]$ [cluster3] command kubectl --context=cluster3 -n submariner-operator logs submariner-routeagent-46pd2
  9409. 2020-11-02T12:07:04.7051110Z [submariner]$ [cluster3] kubectl --context=cluster3 -n submariner-operator logs submariner-routeagent-46pd2
  9410. 2020-11-02T12:07:05.0272623Z [cluster3] I1102 12:05:31.219034 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-11-2-1 --id 172.17.0.7 --host 172.17.0.7 --client 10.3.0.0/16 --to --id 172.17.0.11 --host 172.17.0.11 --client 100.2.0.0/16]
  9411. 2020-11-02T12:07:05.0274136Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-11-2-1"
  9412. 2020-11-02T12:07:05.0276384Z [cluster3] I1102 12:05:31.222781 1 libreswan.go:273] Executing whack with args: [--psk --encrypt --name submariner-cable-cluster2-172-17-0-11-2-2 --id 172.17.0.7 --host 172.17.0.7 --client 10.3.0.0/16 --to --id 172.17.0.11 --host 172.17.0.11 --client 10.2.0.0/16]
  9413. 2020-11-02T12:07:05.0277926Z [cluster3] 002 added connection description "submariner-cable-cluster2-172-17-0-11-2-2"
  9414. 2020-11-02T12:07:05.0279351Z [cluster3] I1102 12:05:31.228173 1 cableengine.go:150] Successfully installed Endpoint cable "submariner-cable-cluster2-172-17-0-11" with remote IP 172.17.0.11
  9415. 2020-11-02T12:07:05.0280939Z [cluster3] I1102 12:05:31.228193 1 tunnel.go:68] Tunnel controller successfully installed Endpoint cable submariner-cable-cluster2-172-17-0-11 in the engine
  9416. 2020-11-02T12:07:05.0282375Z [cluster3] I1102 12:05:31.228200 1 resource_syncer.go:352] Syncer "Tunnel controller": transform function returned nil - not syncing - requeue: false
  9417. 2020-11-02T12:07:05.0283126Z [cluster3] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  9418. 2020-11-02T12:07:05.0284111Z [cluster3] +++++++++++++++++++++: Logs for Pod submariner-routeagent-46pd2 in namespace submariner-operator :++++++++++++++++++++++
  9419. 2020-11-02T12:07:05.0285327Z [cluster3] + trap 'exit 1' SIGTERM SIGINT
  9420. 2020-11-02T12:07:05.0286220Z [cluster3] + SUBMARINER_VERBOSITY=2
  9421. 2020-11-02T12:07:05.0286647Z [cluster3] + '[' false == true ']'
  9422. 2020-11-02T12:07:05.0287002Z [cluster3] + DEBUG=-v=2
  9423. 2020-11-02T12:07:05.0287435Z [cluster3] + for f in iptables-save iptables
  9424. 2020-11-02T12:07:05.0287949Z [cluster3] ++ find_iptables_on_host iptables-save
  9425. 2020-11-02T12:07:05.0288642Z [cluster3] ++ chroot /host test -x /usr/sbin/iptables-save
  9426. 2020-11-02T12:07:05.0288993Z [cluster3] ++ echo /usr/sbin
  9427. 2020-11-02T12:07:05.0289426Z [cluster3] ++ return
  9428. 2020-11-02T12:07:05.0289814Z [cluster3] + location=/usr/sbin
  9429. 2020-11-02T12:07:05.0290351Z [cluster3] + '[' /usr/sbin '!=' unknown ']'
  9430. 2020-11-02T12:07:05.0291140Z [cluster3] + echo 'iptables-save is present on the host at /usr/sbin/iptables-save'
  9431. 2020-11-02T12:07:05.0291877Z [cluster3] + sed 's!@@PATH@@!/usr/sbin!' /usr/sbin/iptables-wrapper.in
  9432. 2020-11-02T12:07:05.0292780Z [cluster3] iptables-save is present on the host at /usr/sbin/iptables-save
  9433. 2020-11-02T12:07:05.0293509Z [cluster3] iptables is present on the host at /usr/sbin/iptables
  9434. 2020-11-02T12:07:05.0294278Z [cluster3] + for f in iptables-save iptables
  9435. 2020-11-02T12:07:05.0294834Z [cluster3] ++ find_iptables_on_host iptables
  9436. 2020-11-02T12:07:05.0295355Z [cluster3] ++ chroot /host test -x /usr/sbin/iptables
  9437. 2020-11-02T12:07:05.0295843Z [cluster3] ++ echo /usr/sbin
  9438. 2020-11-02T12:07:05.0296104Z [cluster3] ++ return
  9439. 2020-11-02T12:07:05.0296380Z [cluster3] + location=/usr/sbin
  9440. 2020-11-02T12:07:05.0296808Z [cluster3] + '[' /usr/sbin '!=' unknown ']'
  9441. 2020-11-02T12:07:05.0297325Z [cluster3] + echo 'iptables is present on the host at /usr/sbin/iptables'
  9442. 2020-11-02T12:07:05.0298326Z [cluster3] + sed 's!@@PATH@@!/usr/sbin!' /usr/sbin/iptables-wrapper.in
  9443. 2020-11-02T12:07:05.0299035Z [cluster3] + exec submariner-route-agent -v=2 -alsologtostderr
  9444. 2020-11-02T12:07:05.0299784Z [cluster3] I1102 11:58:04.720320 1 main.go:47] Starting submariner-route-agent
  9445. 2020-11-02T12:07:05.0300833Z [cluster3] W1102 11:58:04.807592 1 client_config.go:543] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
  9446. 2020-11-02T12:07:05.0301946Z [cluster3] I1102 11:58:04.810189 1 route.go:192] Starting Route Controller. ClusterID: cluster3, localClusterCIDR: [10.3.0.0/16], localServiceCIDR: [100.3.0.0/16]
  9447. 2020-11-02T12:07:05.0302867Z [cluster3] I1102 11:58:04.810381 1 route.go:196] Waiting for Endpoint informer caches to sync.
  9448. 2020-11-02T12:07:05.0303677Z [cluster3] I1102 11:58:04.883531 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.10
  9449. 2020-11-02T12:07:05.0304923Z [cluster3] I1102 11:58:04.883840 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.7
  9450. 2020-11-02T12:07:05.0305703Z [cluster3] I1102 11:58:04.884031 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.5
  9451. 2020-11-02T12:07:05.0307034Z [cluster3] I1102 11:58:04.910917 1 driver.go:37] Interface "lo" has "127.0.0.1" address
  9452. 2020-11-02T12:07:05.0308119Z [cluster3] I1102 11:58:04.911189 1 driver.go:37] Interface "weave" has "10.3.128.0" address
  9453. 2020-11-02T12:07:05.0308735Z [cluster3] I1102 11:58:04.911287 1 driver.go:42] Found CNI Interface "weave" that has IP "10.3.128.0" from ClusterCIDR "10.3.0.0/16"
  9454. 2020-11-02T12:07:05.0309490Z [cluster3] I1102 11:58:04.911490 1 driver.go:60] Successfully configured rp_filter to loose mode(2) on cniInterface "weave"
  9455. 2020-11-02T12:07:05.0312623Z [cluster3] I1102 11:58:04.917052 1 iptables.go:22] Install/ensure SUBMARINER-POSTROUTING chain exists
  9456. 2020-11-02T12:07:05.0313729Z [cluster3] I1102 11:58:05.322763 1 iptables.go:28] Insert SUBMARINER-POSTROUTING rule that has rules for inter-cluster traffic
  9457. 2020-11-02T12:07:05.0315074Z [cluster3] I1102 11:58:05.359302 1 iptables.go:34] Install/ensure SUBMARINER-INPUT chain exists
  9458. 2020-11-02T12:07:05.0349594Z [cluster3] I1102 11:58:05.632283 1 iptables.go:45] Allow VxLAN incoming traffic in SUBMARINER-INPUT Chain
  9459. 2020-11-02T12:07:05.0351112Z [cluster3] I1102 11:58:05.659805 1 iptables.go:53] Insert rule to allow traffic over vx-submariner interface in FORWARDing Chain
  9460. 2020-11-02T12:07:05.0352287Z [cluster3] I1102 11:58:05.681500 1 iptables.go:65] Installing rule for host network to remote cluster communication: -s 240.0.0.0/8 -o vx-submariner -j SNAT --to 10.3.128.0
  9461. 2020-11-02T12:07:05.0353235Z [cluster3] I1102 11:58:05.787021 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.10
  9462. 2020-11-02T12:07:05.0354102Z [cluster3] I1102 11:58:07.037180 1 route.go:254] In cluster3, podIP of submariner-route-agent[0] is 172.17.0.5
  9463. 2020-11-02T12:07:05.0355020Z [cluster3] I1102 11:58:07.037204 1 route.go:254] In cluster3, podIP of submariner-route-agent[1] is 172.17.0.10
  9464. 2020-11-02T12:07:05.0355923Z [cluster3] I1102 11:58:07.037210 1 route.go:254] In cluster3, podIP of submariner-route-agent[2] is 172.17.0.7
  9465. 2020-11-02T12:07:05.0357113Z [cluster3] I1102 11:58:07.037214 1 route.go:268] Hostname is "cluster3-worker2" and routeAgentNodeName is "cluster3-worker2"
  9466. 2020-11-02T12:07:05.0358179Z [cluster3] I1102 11:58:07.255689 1 driver.go:87] Successfully annotated node "cluster3-worker2" with cniIfaceIP "10.3.128.0"
  9467. 2020-11-02T12:07:05.0358803Z [cluster3] I1102 11:58:07.255719 1 route.go:286] Route agent workers started
  9468. 2020-11-02T12:07:05.0359856Z [cluster3] I1102 11:58:07.255769 1 route.go:507] Processing submariner-route-agent pod "submariner-routeagent-szklb" with host IP "172.17.0.10", pod IP "172.17.0.10"
  9469. 2020-11-02T12:07:05.0361465Z [cluster3] I1102 11:58:07.255799 1 route.go:531] Successfully processed submariner-route-agent pod "submariner-routeagent-szklb"
  9470. 2020-11-02T12:07:05.0363069Z [cluster3] I1102 11:58:07.255823 1 route.go:507] Processing submariner-route-agent pod "submariner-routeagent-wvgxr" with host IP "172.17.0.7", pod IP "172.17.0.7"
  9471. 2020-11-02T12:07:05.0364542Z [cluster3] I1102 11:58:07.255828 1 route.go:531] Successfully processed submariner-route-agent pod "submariner-routeagent-wvgxr"
  9472. 2020-11-02T12:07:05.0365922Z [cluster3] I1102 11:58:07.255833 1 route.go:507] Processing submariner-route-agent pod "submariner-routeagent-46pd2" with host IP "172.17.0.5", pod IP "172.17.0.5"
  9473. 2020-11-02T12:07:05.0367217Z [cluster3] I1102 11:58:07.255838 1 route.go:531] Successfully processed submariner-route-agent pod "submariner-routeagent-46pd2"
  9474. 2020-11-02T12:07:05.0368773Z [cluster3] I1102 11:58:24.092872 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  9475. 2020-11-02T12:07:05.0370207Z [cluster3] I1102 11:58:24.103560 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.3.0.0/16 -d 100.2.0.0/16 -j ACCEPT
  9476. 2020-11-02T12:07:05.0371100Z [cluster3] I1102 11:58:24.120958 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 100.2.0.0/16 -d 10.3.0.0/16 -j ACCEPT
  9477. 2020-11-02T12:07:05.0372108Z [cluster3] I1102 11:58:24.177000 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.3.0.0/16 -d 10.2.0.0/16 -j ACCEPT
  9478. 2020-11-02T12:07:05.0373097Z [cluster3] I1102 11:58:24.226403 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 10.2.0.0/16 -d 10.3.0.0/16 -j ACCEPT
  9479. 2020-11-02T12:07:05.0374449Z [cluster3] I1102 11:58:24.253628 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  9480. 2020-11-02T12:07:05.0375723Z [cluster3] I1102 11:58:24.253650 1 route.go:612] This route agent is running on a non-gateway/non-active node
  9481. 2020-11-02T12:07:05.0376570Z [cluster3] I1102 11:58:24.253661 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  9482. 2020-11-02T12:07:05.0377303Z [cluster3] I1102 11:58:24.280537 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  9483. 2020-11-02T12:07:05.0378063Z [cluster3] I1102 11:58:24.305662 1 route.go:787] Processing route {Ifindex: 14 Dst: 240.0.0.0/8 Src: 240.17.0.5 Gw: <nil> Flags: [] Table: 254}
  9484. 2020-11-02T12:07:05.0378919Z [cluster3] I1102 11:58:24.305726 1 route.go:790] Found nil gw or dst
  9485. 2020-11-02T12:07:05.0380101Z [cluster3] I1102 11:58:24.305935 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  9486. 2020-11-02T12:07:05.0382410Z [cluster3] I1102 11:58:27.570259 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  9487. 2020-11-02T12:07:05.0383798Z [cluster3] I1102 11:58:27.580492 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.3.0.0/16 -d 100.1.0.0/16 -j ACCEPT
  9488. 2020-11-02T12:07:05.0385062Z [cluster3] I1102 11:58:27.601278 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 100.1.0.0/16 -d 10.3.0.0/16 -j ACCEPT
  9489. 2020-11-02T12:07:05.0386885Z [cluster3] I1102 11:58:27.641942 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.3.0.0/16 -d 10.1.0.0/16 -j ACCEPT
  9490. 2020-11-02T12:07:05.0387755Z [cluster3] I1102 11:58:27.670693 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 10.1.0.0/16 -d 10.3.0.0/16 -j ACCEPT
  9491. 2020-11-02T12:07:05.0389043Z [cluster3] I1102 11:58:34.871667 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  9492. 2020-11-02T12:07:05.0390739Z [cluster3] I1102 11:58:34.871715 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  9493. 2020-11-02T12:07:05.0392898Z [cluster3] I1102 11:58:34.871725 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  9494. 2020-11-02T12:07:05.0394167Z [cluster3] I1102 11:58:34.871733 1 route.go:612] This route agent is running on a non-gateway/non-active node
  9495. 2020-11-02T12:07:05.0395158Z [cluster3] I1102 11:58:34.871741 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  9496. 2020-11-02T12:07:05.0395796Z [cluster3] I1102 11:58:34.872008 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  9497. 2020-11-02T12:07:05.0396511Z [cluster3] I1102 11:58:34.877032 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  9498. 2020-11-02T12:07:05.0397024Z [cluster3] I1102 11:58:34.877159 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9499. 2020-11-02T12:07:05.0397646Z [cluster3] I1102 11:58:34.877190 1 route.go:793] Found route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9500. 2020-11-02T12:07:05.0398264Z [cluster3] I1102 11:58:34.877201 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9501. 2020-11-02T12:07:05.0398975Z [cluster3] I1102 11:58:34.877214 1 route.go:793] Found route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9502. 2020-11-02T12:07:05.0399599Z [cluster3] I1102 11:58:34.877221 1 route.go:787] Processing route {Ifindex: 14 Dst: 240.0.0.0/8 Src: 240.17.0.5 Gw: <nil> Flags: [] Table: 254}
  9503. 2020-11-02T12:07:05.0400065Z [cluster3] I1102 11:58:34.877234 1 route.go:790] Found nil gw or dst
  9504. 2020-11-02T12:07:05.0400486Z [cluster3] I1102 11:58:34.877288 1 route.go:829] Found equivalent route, not adding
  9505. 2020-11-02T12:07:05.0400959Z [cluster3] I1102 11:58:34.877303 1 route.go:829] Found equivalent route, not adding
  9506. 2020-11-02T12:07:05.0402345Z [cluster3] I1102 11:58:34.877393 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  9507. 2020-11-02T12:07:05.0404190Z [cluster3] I1102 11:59:04.871821 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  9508. 2020-11-02T12:07:05.0406522Z [cluster3] I1102 11:59:04.871860 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  9509. 2020-11-02T12:07:05.0407781Z [cluster3] I1102 11:59:04.871891 1 route.go:612] This route agent is running on a non-gateway/non-active node
  9510. 2020-11-02T12:07:05.0408571Z [cluster3] I1102 11:59:04.871899 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  9511. 2020-11-02T12:07:05.0409191Z [cluster3] I1102 11:59:04.872149 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  9512. 2020-11-02T12:07:05.0409704Z [cluster3] I1102 11:59:04.876774 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  9513. 2020-11-02T12:07:05.0410197Z [cluster3] I1102 11:59:04.878594 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9514. 2020-11-02T12:07:05.0410795Z [cluster3] I1102 11:59:04.878622 1 route.go:793] Found route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9515. 2020-11-02T12:07:05.0411803Z [cluster3] I1102 11:59:04.878631 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9516. 2020-11-02T12:07:05.0412644Z [cluster3] I1102 11:59:04.878651 1 route.go:793] Found route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9517. 2020-11-02T12:07:05.0413455Z [cluster3] I1102 11:59:04.878659 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9518. 2020-11-02T12:07:05.0414130Z [cluster3] I1102 11:59:04.878668 1 route.go:793] Found route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9519. 2020-11-02T12:07:05.0414952Z [cluster3] I1102 11:59:04.878675 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9520. 2020-11-02T12:07:05.0415703Z [cluster3] I1102 11:59:04.878683 1 route.go:793] Found route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9521. 2020-11-02T12:07:05.0416290Z [cluster3] I1102 11:59:04.878690 1 route.go:787] Processing route {Ifindex: 14 Dst: 240.0.0.0/8 Src: 240.17.0.5 Gw: <nil> Flags: [] Table: 254}
  9522. 2020-11-02T12:07:05.0416740Z [cluster3] I1102 11:59:04.878697 1 route.go:790] Found nil gw or dst
  9523. 2020-11-02T12:07:05.0417152Z [cluster3] I1102 11:59:04.878773 1 route.go:829] Found equivalent route, not adding
  9524. 2020-11-02T12:07:05.0417595Z [cluster3] I1102 11:59:04.878782 1 route.go:829] Found equivalent route, not adding
  9525. 2020-11-02T12:07:05.0418050Z [cluster3] I1102 11:59:04.878790 1 route.go:829] Found equivalent route, not adding
  9526. 2020-11-02T12:07:05.0418574Z [cluster3] I1102 11:59:04.878797 1 route.go:829] Found equivalent route, not adding
  9527. 2020-11-02T12:07:05.0419853Z [cluster3] I1102 11:59:04.878827 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  9528. 2020-11-02T12:07:05.0421828Z [cluster3] I1102 11:59:04.878842 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  9529. 2020-11-02T12:07:05.0427543Z [cluster3] I1102 11:59:34.871909 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  9530. 2020-11-02T12:07:05.0432275Z [cluster3] I1102 11:59:34.871948 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  9531. 2020-11-02T12:07:05.0440653Z [cluster3] I1102 11:59:34.871957 1 route.go:612] This route agent is running on a non-gateway/non-active node
  9532. 2020-11-02T12:07:05.0441764Z [cluster3] I1102 11:59:34.871966 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  9533. 2020-11-02T12:07:05.0450851Z [cluster3] I1102 11:59:34.872157 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  9534. 2020-11-02T12:07:05.0452074Z [cluster3] I1102 11:59:34.876533 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  9535. 2020-11-02T12:07:05.0452953Z [cluster3] I1102 11:59:34.876682 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9536. 2020-11-02T12:07:05.0454318Z [cluster3] I1102 11:59:34.876704 1 route.go:793] Found route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9537. 2020-11-02T12:07:05.0455429Z [cluster3] I1102 11:59:34.876714 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9538. 2020-11-02T12:07:05.0456064Z [cluster3] I1102 11:59:34.876721 1 route.go:793] Found route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9539. 2020-11-02T12:07:05.0456779Z [cluster3] I1102 11:59:34.876729 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9540. 2020-11-02T12:07:05.0457629Z [cluster3] I1102 11:59:34.876743 1 route.go:793] Found route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9541. 2020-11-02T12:07:05.0458428Z [cluster3] I1102 11:59:34.876751 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9542. 2020-11-02T12:07:05.0459054Z [cluster3] I1102 11:59:34.876759 1 route.go:793] Found route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9543. 2020-11-02T12:07:05.0459843Z [cluster3] I1102 11:59:34.876766 1 route.go:787] Processing route {Ifindex: 14 Dst: 240.0.0.0/8 Src: 240.17.0.5 Gw: <nil> Flags: [] Table: 254}
  9544. 2020-11-02T12:07:05.0460513Z [cluster3] I1102 11:59:34.876773 1 route.go:790] Found nil gw or dst
  9545. 2020-11-02T12:07:05.0461152Z [cluster3] I1102 11:59:34.876831 1 route.go:829] Found equivalent route, not adding
  9546. 2020-11-02T12:07:05.0461836Z [cluster3] I1102 11:59:34.876840 1 route.go:829] Found equivalent route, not adding
  9547. 2020-11-02T12:07:05.0462374Z [cluster3] I1102 11:59:34.876848 1 route.go:829] Found equivalent route, not adding
  9548. 2020-11-02T12:07:05.0462910Z [cluster3] I1102 11:59:34.876854 1 route.go:829] Found equivalent route, not adding
  9549. 2020-11-02T12:07:05.0464969Z [cluster3] I1102 11:59:34.876883 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  9550. 2020-11-02T12:07:05.0467149Z [cluster3] I1102 11:59:34.876897 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  9551. 2020-11-02T12:07:05.0471321Z [cluster3] I1102 12:00:04.871960 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  9552. 2020-11-02T12:07:05.0473885Z [cluster3] I1102 12:00:04.871998 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  9553. 2020-11-02T12:07:05.0475389Z [cluster3] I1102 12:00:04.872007 1 route.go:612] This route agent is running on a non-gateway/non-active node
  9554. 2020-11-02T12:07:05.0476277Z [cluster3] I1102 12:00:04.872016 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  9555. 2020-11-02T12:07:05.0477088Z [cluster3] I1102 12:00:04.872204 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  9556. 2020-11-02T12:07:05.0477831Z [cluster3] I1102 12:00:04.875456 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  9557. 2020-11-02T12:07:05.0478747Z [cluster3] I1102 12:00:04.875586 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9558. 2020-11-02T12:07:05.0480046Z [cluster3] I1102 12:00:04.875661 1 route.go:793] Found route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9559. 2020-11-02T12:07:05.0481495Z [cluster3] I1102 12:00:04.875685 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9560. 2020-11-02T12:07:05.0482396Z [cluster3] I1102 12:00:04.875696 1 route.go:793] Found route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9561. 2020-11-02T12:07:05.0483058Z [cluster3] I1102 12:00:04.875728 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9562. 2020-11-02T12:07:05.0484015Z [cluster3] I1102 12:00:04.875736 1 route.go:793] Found route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9563. 2020-11-02T12:07:05.0484855Z [cluster3] I1102 12:00:04.875744 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9564. 2020-11-02T12:07:05.0485497Z [cluster3] I1102 12:00:04.875751 1 route.go:793] Found route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9565. 2020-11-02T12:07:05.0486308Z [cluster3] I1102 12:00:04.875760 1 route.go:787] Processing route {Ifindex: 14 Dst: 240.0.0.0/8 Src: 240.17.0.5 Gw: <nil> Flags: [] Table: 254}
  9566. 2020-11-02T12:07:05.0486885Z [cluster3] I1102 12:00:04.875769 1 route.go:790] Found nil gw or dst
  9567. 2020-11-02T12:07:05.0487332Z [cluster3] I1102 12:00:04.875833 1 route.go:829] Found equivalent route, not adding
  9568. 2020-11-02T12:07:05.0487843Z [cluster3] I1102 12:00:04.875864 1 route.go:829] Found equivalent route, not adding
  9569. 2020-11-02T12:07:05.0488465Z [cluster3] I1102 12:00:04.875872 1 route.go:829] Found equivalent route, not adding
  9570. 2020-11-02T12:07:05.0489124Z [cluster3] I1102 12:00:04.875881 1 route.go:829] Found equivalent route, not adding
  9571. 2020-11-02T12:07:05.0491236Z [cluster3] I1102 12:00:04.875919 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  9572. 2020-11-02T12:07:05.0493079Z [cluster3] I1102 12:00:04.875957 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  9573. 2020-11-02T12:07:05.0496658Z [cluster3] I1102 12:00:34.872302 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  9574. 2020-11-02T12:07:05.0498812Z [cluster3] I1102 12:00:34.872341 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  9575. 2020-11-02T12:07:05.0500144Z [cluster3] I1102 12:00:34.872370 1 route.go:612] This route agent is running on a non-gateway/non-active node
  9576. 2020-11-02T12:07:05.0500986Z [cluster3] I1102 12:00:34.872378 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  9577. 2020-11-02T12:07:05.0501622Z [cluster3] I1102 12:00:34.872785 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  9578. 2020-11-02T12:07:05.0502174Z [cluster3] I1102 12:00:34.876000 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  9579. 2020-11-02T12:07:05.0502700Z [cluster3] I1102 12:00:34.876200 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9580. 2020-11-02T12:07:05.0503701Z [cluster3] I1102 12:00:34.876261 1 route.go:793] Found route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9581. 2020-11-02T12:07:05.0504379Z [cluster3] I1102 12:00:34.876286 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9582. 2020-11-02T12:07:05.0505048Z [cluster3] I1102 12:00:34.876293 1 route.go:793] Found route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9583. 2020-11-02T12:07:05.0505864Z [cluster3] I1102 12:00:34.876299 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9584. 2020-11-02T12:07:05.0506695Z [cluster3] I1102 12:00:34.876306 1 route.go:793] Found route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9585. 2020-11-02T12:07:05.0507666Z [cluster3] I1102 12:00:34.876313 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9586. 2020-11-02T12:07:05.0508290Z [cluster3] I1102 12:00:34.876318 1 route.go:793] Found route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9587. 2020-11-02T12:07:05.0509082Z [cluster3] I1102 12:00:34.876325 1 route.go:787] Processing route {Ifindex: 14 Dst: 240.0.0.0/8 Src: 240.17.0.5 Gw: <nil> Flags: [] Table: 254}
  9588. 2020-11-02T12:07:05.0509577Z [cluster3] I1102 12:00:34.876330 1 route.go:790] Found nil gw or dst
  9589. 2020-11-02T12:07:05.0510024Z [cluster3] I1102 12:00:34.876443 1 route.go:829] Found equivalent route, not adding
  9590. 2020-11-02T12:07:05.0510625Z [cluster3] I1102 12:00:34.876453 1 route.go:829] Found equivalent route, not adding
  9591. 2020-11-02T12:07:05.0511121Z [cluster3] I1102 12:00:34.876460 1 route.go:829] Found equivalent route, not adding
  9592. 2020-11-02T12:07:05.0511617Z [cluster3] I1102 12:00:34.876467 1 route.go:829] Found equivalent route, not adding
  9593. 2020-11-02T12:07:05.0512850Z [cluster3] I1102 12:00:34.876497 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  9594. 2020-11-02T12:07:05.0514829Z [cluster3] I1102 12:00:34.876509 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  9595. 2020-11-02T12:07:05.0516801Z [cluster3] I1102 12:01:04.872506 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  9596. 2020-11-02T12:07:05.0518642Z [cluster3] I1102 12:01:04.872555 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  9597. 2020-11-02T12:07:05.0519953Z [cluster3] I1102 12:01:04.872569 1 route.go:612] This route agent is running on a non-gateway/non-active node
  9598. 2020-11-02T12:07:05.0521057Z [cluster3] I1102 12:01:04.872577 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  9599. 2020-11-02T12:07:05.0521718Z [cluster3] I1102 12:01:04.872838 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  9600. 2020-11-02T12:07:05.0522262Z [cluster3] I1102 12:01:04.876506 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  9601. 2020-11-02T12:07:05.0522785Z [cluster3] I1102 12:01:04.876701 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9602. 2020-11-02T12:07:05.0523417Z [cluster3] I1102 12:01:04.876735 1 route.go:793] Found route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9603. 2020-11-02T12:07:05.0524206Z [cluster3] I1102 12:01:04.876744 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9604. 2020-11-02T12:07:05.0524847Z [cluster3] I1102 12:01:04.876759 1 route.go:793] Found route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9605. 2020-11-02T12:07:05.0525475Z [cluster3] I1102 12:01:04.876872 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9606. 2020-11-02T12:07:05.0526104Z [cluster3] I1102 12:01:04.876896 1 route.go:793] Found route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9607. 2020-11-02T12:07:05.0526733Z [cluster3] I1102 12:01:04.876904 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9608. 2020-11-02T12:07:05.0527359Z [cluster3] I1102 12:01:04.876926 1 route.go:793] Found route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9609. 2020-11-02T12:07:05.0527985Z [cluster3] I1102 12:01:04.876935 1 route.go:787] Processing route {Ifindex: 14 Dst: 240.0.0.0/8 Src: 240.17.0.5 Gw: <nil> Flags: [] Table: 254}
  9610. 2020-11-02T12:07:05.0528464Z [cluster3] I1102 12:01:04.876964 1 route.go:790] Found nil gw or dst
  9611. 2020-11-02T12:07:05.0528896Z [cluster3] I1102 12:01:04.877045 1 route.go:829] Found equivalent route, not adding
  9612. 2020-11-02T12:07:05.0529367Z [cluster3] I1102 12:01:04.877057 1 route.go:829] Found equivalent route, not adding
  9613. 2020-11-02T12:07:05.0529850Z [cluster3] I1102 12:01:04.877065 1 route.go:829] Found equivalent route, not adding
  9614. 2020-11-02T12:07:05.0530337Z [cluster3] I1102 12:01:04.877073 1 route.go:829] Found equivalent route, not adding
  9615. 2020-11-02T12:07:05.0531644Z [cluster3] I1102 12:01:04.877104 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  9616. 2020-11-02T12:07:05.0533350Z [cluster3] I1102 12:01:04.877118 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  9617. 2020-11-02T12:07:05.0535352Z [cluster3] I1102 12:01:34.872813 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  9618. 2020-11-02T12:07:05.0537306Z [cluster3] I1102 12:01:34.872865 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  9619. 2020-11-02T12:07:05.0538743Z [cluster3] I1102 12:01:34.872875 1 route.go:612] This route agent is running on a non-gateway/non-active node
  9620. 2020-11-02T12:07:05.0540181Z [cluster3] I1102 12:01:34.872885 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  9621. 2020-11-02T12:07:05.0540874Z [cluster3] I1102 12:01:34.873188 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  9622. 2020-11-02T12:07:05.0541454Z [cluster3] I1102 12:01:34.877129 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  9623. 2020-11-02T12:07:05.0542426Z [cluster3] I1102 12:01:34.877326 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9624. 2020-11-02T12:07:05.0543228Z [cluster3] I1102 12:01:34.877373 1 route.go:793] Found route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9625. 2020-11-02T12:07:05.0544034Z [cluster3] I1102 12:01:34.877381 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9626. 2020-11-02T12:07:05.0544660Z [cluster3] I1102 12:01:34.877387 1 route.go:793] Found route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9627. 2020-11-02T12:07:05.0545296Z [cluster3] I1102 12:01:34.877393 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9628. 2020-11-02T12:07:05.0545925Z [cluster3] I1102 12:01:34.877400 1 route.go:793] Found route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9629. 2020-11-02T12:07:05.0546553Z [cluster3] I1102 12:01:34.877406 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9630. 2020-11-02T12:07:05.0547309Z [cluster3] I1102 12:01:34.877413 1 route.go:793] Found route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9631. 2020-11-02T12:07:05.0547913Z [cluster3] I1102 12:01:34.877419 1 route.go:787] Processing route {Ifindex: 14 Dst: 240.0.0.0/8 Src: 240.17.0.5 Gw: <nil> Flags: [] Table: 254}
  9632. 2020-11-02T12:07:05.0548383Z [cluster3] I1102 12:01:34.877425 1 route.go:790] Found nil gw or dst
  9633. 2020-11-02T12:07:05.0548803Z [cluster3] I1102 12:01:34.877496 1 route.go:829] Found equivalent route, not adding
  9634. 2020-11-02T12:07:05.0549280Z [cluster3] I1102 12:01:34.877504 1 route.go:829] Found equivalent route, not adding
  9635. 2020-11-02T12:07:05.0549985Z [cluster3] I1102 12:01:34.877511 1 route.go:829] Found equivalent route, not adding
  9636. 2020-11-02T12:07:05.0550466Z [cluster3] I1102 12:01:34.877517 1 route.go:829] Found equivalent route, not adding
  9637. 2020-11-02T12:07:05.0551669Z [cluster3] I1102 12:01:34.877603 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  9638. 2020-11-02T12:07:05.0553366Z [cluster3] I1102 12:01:34.877618 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  9639. 2020-11-02T12:07:05.0557574Z [cluster3] I1102 12:01:40.300830 1 route.go:682] Informed of removed endpoint: {"metadata":{"name":"cluster2-submariner-cable-cluster2-172-17-0-4","namespace":"submariner-operator","selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4","uid":"8c63de0a-c225-4c4f-b602-4d0ec35f7bae","resourceVersion":"2815","generation":1,"creationTimestamp":"2020-11-02T11:58:24Z","labels":{"submariner-io/clusterID":"cluster2"}},"spec":{"cluster_id":"cluster2","cable_name":"submariner-cable-cluster2-172-17-0-4","hostname":"cluster2-worker","subnets":["100.2.0.0/16","10.2.0.0/16"],"private_ip":"172.17.0.4","public_ip":"","nat_enabled":false,"backend":"libreswan"}}
  9640. 2020-11-02T12:07:05.0562723Z [cluster3] I1102 12:01:40.378835 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  9641. 2020-11-02T12:07:05.0564562Z [cluster3] I1102 12:02:04.872807 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  9642. 2020-11-02T12:07:05.0566437Z [cluster3] I1102 12:02:04.872843 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  9643. 2020-11-02T12:07:05.0567731Z [cluster3] I1102 12:02:04.872852 1 route.go:612] This route agent is running on a non-gateway/non-active node
  9644. 2020-11-02T12:07:05.0568548Z [cluster3] I1102 12:02:04.872862 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  9645. 2020-11-02T12:07:05.0569178Z [cluster3] I1102 12:02:04.873052 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  9646. 2020-11-02T12:07:05.0569931Z [cluster3] I1102 12:02:04.876308 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  9647. 2020-11-02T12:07:05.0570461Z [cluster3] I1102 12:02:04.876527 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9648. 2020-11-02T12:07:05.0572395Z [cluster3] I1102 12:02:04.876561 1 route.go:793] Found route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9649. 2020-11-02T12:07:05.0573212Z [cluster3] I1102 12:02:04.876571 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9650. 2020-11-02T12:07:05.0573828Z [cluster3] I1102 12:02:04.876579 1 route.go:793] Found route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9651. 2020-11-02T12:07:05.0574440Z [cluster3] I1102 12:02:04.876593 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9652. 2020-11-02T12:07:05.0575053Z [cluster3] I1102 12:02:04.876603 1 route.go:793] Found route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9653. 2020-11-02T12:07:05.0575669Z [cluster3] I1102 12:02:04.876616 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9654. 2020-11-02T12:07:05.0576278Z [cluster3] I1102 12:02:04.876625 1 route.go:793] Found route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9655. 2020-11-02T12:07:05.0576882Z [cluster3] I1102 12:02:04.876638 1 route.go:787] Processing route {Ifindex: 14 Dst: 240.0.0.0/8 Src: 240.17.0.5 Gw: <nil> Flags: [] Table: 254}
  9656. 2020-11-02T12:07:05.0577346Z [cluster3] I1102 12:02:04.876646 1 route.go:790] Found nil gw or dst
  9657. 2020-11-02T12:07:05.0577765Z [cluster3] I1102 12:02:04.876720 1 route.go:829] Found equivalent route, not adding
  9658. 2020-11-02T12:07:05.0578239Z [cluster3] I1102 12:02:04.876736 1 route.go:829] Found equivalent route, not adding
  9659. 2020-11-02T12:07:05.0578827Z [cluster3] I1102 12:02:04.876745 1 route.go:829] Found equivalent route, not adding
  9660. 2020-11-02T12:07:05.0579977Z [cluster3] I1102 12:02:04.876753 1 route.go:829] Found equivalent route, not adding
  9661. 2020-11-02T12:07:05.0581570Z [cluster3] I1102 12:02:04.876785 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  9662. 2020-11-02T12:07:05.0583796Z [cluster3] I1102 12:02:04.876807 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  9663. 2020-11-02T12:07:05.0586653Z [cluster3] I1102 12:02:34.872984 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  9664. 2020-11-02T12:07:05.0589046Z [cluster3] I1102 12:02:34.873021 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  9665. 2020-11-02T12:07:05.0591345Z [cluster3] I1102 12:02:34.873030 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  9666. 2020-11-02T12:07:05.0593006Z [cluster3] I1102 12:02:34.873043 1 route.go:612] This route agent is running on a non-gateway/non-active node
  9667. 2020-11-02T12:07:05.0594070Z [cluster3] I1102 12:02:34.873051 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  9668. 2020-11-02T12:07:05.0594845Z [cluster3] I1102 12:02:34.873311 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  9669. 2020-11-02T12:07:05.0595491Z [cluster3] I1102 12:02:34.876765 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  9670. 2020-11-02T12:07:05.0596176Z [cluster3] I1102 12:02:34.876908 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9671. 2020-11-02T12:07:05.0597118Z [cluster3] I1102 12:02:34.876929 1 route.go:793] Found route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9672. 2020-11-02T12:07:05.0598186Z [cluster3] I1102 12:02:34.876938 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9673. 2020-11-02T12:07:05.0599076Z [cluster3] I1102 12:02:34.876968 1 route.go:793] Found route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9674. 2020-11-02T12:07:05.0599799Z [cluster3] I1102 12:02:34.876974 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9675. 2020-11-02T12:07:05.0601236Z [cluster3] I1102 12:02:34.876981 1 route.go:793] Found route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9676. 2020-11-02T12:07:05.0601912Z [cluster3] I1102 12:02:34.876987 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9677. 2020-11-02T12:07:05.0603816Z [cluster3] I1102 12:02:34.877015 1 route.go:793] Found route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9678. 2020-11-02T12:07:05.0605867Z [cluster3] I1102 12:02:34.877021 1 route.go:787] Processing route {Ifindex: 14 Dst: 240.0.0.0/8 Src: 240.17.0.5 Gw: <nil> Flags: [] Table: 254}
  9679. 2020-11-02T12:07:05.0606337Z [cluster3] I1102 12:02:34.877028 1 route.go:790] Found nil gw or dst
  9680. 2020-11-02T12:07:05.0606924Z [cluster3] I1102 12:02:34.877169 1 route.go:829] Found equivalent route, not adding
  9681. 2020-11-02T12:07:05.0607397Z [cluster3] I1102 12:02:34.877178 1 route.go:829] Found equivalent route, not adding
  9682. 2020-11-02T12:07:05.0607865Z [cluster3] I1102 12:02:34.877189 1 route.go:829] Found equivalent route, not adding
  9683. 2020-11-02T12:07:05.0608473Z [cluster3] I1102 12:02:34.877196 1 route.go:829] Found equivalent route, not adding
  9684. 2020-11-02T12:07:05.0610122Z [cluster3] I1102 12:02:34.877226 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  9685. 2020-11-02T12:07:05.0612104Z [cluster3] I1102 12:03:04.873273 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  9686. 2020-11-02T12:07:05.0614375Z [cluster3] I1102 12:03:04.873310 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  9687. 2020-11-02T12:07:05.0616451Z [cluster3] I1102 12:03:04.873319 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  9688. 2020-11-02T12:07:05.0618085Z [cluster3] I1102 12:03:04.873327 1 route.go:612] This route agent is running on a non-gateway/non-active node
  9689. 2020-11-02T12:07:05.0619121Z [cluster3] I1102 12:03:04.873346 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  9690. 2020-11-02T12:07:05.0620336Z [cluster3] I1102 12:03:04.873559 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  9691. 2020-11-02T12:07:05.0621128Z [cluster3] I1102 12:03:04.877004 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  9692. 2020-11-02T12:07:05.0621666Z [cluster3] I1102 12:03:04.877300 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9693. 2020-11-02T12:07:05.0622301Z [cluster3] I1102 12:03:04.877340 1 route.go:793] Found route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9694. 2020-11-02T12:07:05.0623065Z [cluster3] I1102 12:03:04.877348 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9695. 2020-11-02T12:07:05.0623680Z [cluster3] I1102 12:03:04.877355 1 route.go:793] Found route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9696. 2020-11-02T12:07:05.0624288Z [cluster3] I1102 12:03:04.877362 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9697. 2020-11-02T12:07:05.0624899Z [cluster3] I1102 12:03:04.877369 1 route.go:793] Found route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9698. 2020-11-02T12:07:05.0625508Z [cluster3] I1102 12:03:04.877415 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9699. 2020-11-02T12:07:05.0626114Z [cluster3] I1102 12:03:04.877531 1 route.go:793] Found route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9700. 2020-11-02T12:07:05.0626722Z [cluster3] I1102 12:03:04.877543 1 route.go:787] Processing route {Ifindex: 14 Dst: 240.0.0.0/8 Src: 240.17.0.5 Gw: <nil> Flags: [] Table: 254}
  9701. 2020-11-02T12:07:05.0627184Z [cluster3] I1102 12:03:04.877552 1 route.go:790] Found nil gw or dst
  9702. 2020-11-02T12:07:05.0627602Z [cluster3] I1102 12:03:04.877706 1 route.go:829] Found equivalent route, not adding
  9703. 2020-11-02T12:07:05.0628077Z [cluster3] I1102 12:03:04.877733 1 route.go:829] Found equivalent route, not adding
  9704. 2020-11-02T12:07:05.0628545Z [cluster3] I1102 12:03:04.877741 1 route.go:829] Found equivalent route, not adding
  9705. 2020-11-02T12:07:05.0629009Z [cluster3] I1102 12:03:04.877747 1 route.go:829] Found equivalent route, not adding
  9706. 2020-11-02T12:07:05.0633104Z [cluster3] I1102 12:03:04.877924 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  9707. 2020-11-02T12:07:05.0636263Z [cluster3] I1102 12:03:34.873285 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  9708. 2020-11-02T12:07:05.0640631Z [cluster3] I1102 12:03:34.873354 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  9709. 2020-11-02T12:07:05.0642486Z [cluster3] I1102 12:03:34.873363 1 route.go:612] This route agent is running on a non-gateway/non-active node
  9710. 2020-11-02T12:07:05.0644162Z [cluster3] I1102 12:03:34.873371 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  9711. 2020-11-02T12:07:05.0644812Z [cluster3] I1102 12:03:34.873560 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  9712. 2020-11-02T12:07:05.0645359Z [cluster3] I1102 12:03:34.876728 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  9713. 2020-11-02T12:07:05.0645889Z [cluster3] I1102 12:03:34.876917 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9714. 2020-11-02T12:07:05.0646745Z [cluster3] I1102 12:03:34.876960 1 route.go:793] Found route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9715. 2020-11-02T12:07:05.0647684Z [cluster3] I1102 12:03:34.876970 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9716. 2020-11-02T12:07:05.0648353Z [cluster3] I1102 12:03:34.876977 1 route.go:793] Found route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9717. 2020-11-02T12:07:05.0648983Z [cluster3] I1102 12:03:34.876985 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9718. 2020-11-02T12:07:05.0649819Z [cluster3] I1102 12:03:34.876997 1 route.go:793] Found route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9719. 2020-11-02T12:07:05.0650471Z [cluster3] I1102 12:03:34.877004 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9720. 2020-11-02T12:07:05.0651123Z [cluster3] I1102 12:03:34.877012 1 route.go:793] Found route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9721. 2020-11-02T12:07:05.0651768Z [cluster3] I1102 12:03:34.877018 1 route.go:787] Processing route {Ifindex: 14 Dst: 240.0.0.0/8 Src: 240.17.0.5 Gw: <nil> Flags: [] Table: 254}
  9722. 2020-11-02T12:07:05.0652264Z [cluster3] I1102 12:03:34.877025 1 route.go:790] Found nil gw or dst
  9723. 2020-11-02T12:07:05.0652707Z [cluster3] I1102 12:03:34.877098 1 route.go:829] Found equivalent route, not adding
  9724. 2020-11-02T12:07:05.0653209Z [cluster3] I1102 12:03:34.877107 1 route.go:829] Found equivalent route, not adding
  9725. 2020-11-02T12:07:05.0653702Z [cluster3] I1102 12:03:34.877115 1 route.go:829] Found equivalent route, not adding
  9726. 2020-11-02T12:07:05.0654199Z [cluster3] I1102 12:03:34.877122 1 route.go:829] Found equivalent route, not adding
  9727. 2020-11-02T12:07:05.0656230Z [cluster3] I1102 12:03:34.877150 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  9728. 2020-11-02T12:07:05.0658531Z [cluster3] I1102 12:03:34.877162 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  9729. 2020-11-02T12:07:05.0661177Z [cluster3] I1102 12:04:04.873523 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  9730. 2020-11-02T12:07:05.0662718Z [cluster3] I1102 12:04:04.873548 1 route.go:612] This route agent is running on a non-gateway/non-active node
  9731. 2020-11-02T12:07:05.0663560Z [cluster3] I1102 12:04:04.873557 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  9732. 2020-11-02T12:07:05.0664358Z [cluster3] I1102 12:04:04.873759 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  9733. 2020-11-02T12:07:05.0664901Z [cluster3] I1102 12:04:04.876907 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  9734. 2020-11-02T12:07:05.0665431Z [cluster3] I1102 12:04:04.877086 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9735. 2020-11-02T12:07:05.0666062Z [cluster3] I1102 12:04:04.877118 1 route.go:793] Found route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9736. 2020-11-02T12:07:05.0667289Z [cluster3] I1102 12:04:04.877145 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9737. 2020-11-02T12:07:05.0667946Z [cluster3] I1102 12:04:04.877154 1 route.go:793] Found route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9738. 2020-11-02T12:07:05.0668586Z [cluster3] I1102 12:04:04.877190 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9739. 2020-11-02T12:07:05.0669309Z [cluster3] I1102 12:04:04.877206 1 route.go:793] Found route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9740. 2020-11-02T12:07:05.0669948Z [cluster3] I1102 12:04:04.877213 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9741. 2020-11-02T12:07:05.0670572Z [cluster3] I1102 12:04:04.877226 1 route.go:793] Found route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9742. 2020-11-02T12:07:05.0671196Z [cluster3] I1102 12:04:04.877233 1 route.go:787] Processing route {Ifindex: 14 Dst: 240.0.0.0/8 Src: 240.17.0.5 Gw: <nil> Flags: [] Table: 254}
  9743. 2020-11-02T12:07:05.0671679Z [cluster3] I1102 12:04:04.877246 1 route.go:790] Found nil gw or dst
  9744. 2020-11-02T12:07:05.0672111Z [cluster3] I1102 12:04:04.877334 1 route.go:829] Found equivalent route, not adding
  9745. 2020-11-02T12:07:05.0672998Z [cluster3] I1102 12:04:04.877351 1 route.go:829] Found equivalent route, not adding
  9746. 2020-11-02T12:07:05.0673637Z [cluster3] I1102 12:04:04.877360 1 route.go:829] Found equivalent route, not adding
  9747. 2020-11-02T12:07:05.0674755Z [submariner]$ [cluster3] kubectl -n submariner-operator logs submariner-routeagent-szklb
  9748. 2020-11-02T12:07:05.0675866Z [submariner]$ [cluster3] kubectl -n submariner-operator logs submariner-routeagent-szklb
  9749. 2020-11-02T12:07:05.0677057Z [submariner]$ [cluster3] command kubectl --context=cluster3 -n submariner-operator logs submariner-routeagent-szklb
  9750. 2020-11-02T12:07:05.0678298Z [submariner]$ [cluster3] kubectl --context=cluster3 -n submariner-operator logs submariner-routeagent-szklb
  9751. 2020-11-02T12:07:05.0679260Z [cluster3] I1102 12:04:04.877367 1 route.go:829] Found equivalent route, not adding
  9752. 2020-11-02T12:07:05.0680584Z [cluster3] I1102 12:04:04.877396 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  9753. 2020-11-02T12:07:05.0682282Z [cluster3] I1102 12:04:04.877417 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  9754. 2020-11-02T12:07:05.0690041Z [cluster3] I1102 12:04:04.877436 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  9755. 2020-11-02T12:07:05.0695779Z [cluster3] I1102 12:04:34.873590 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  9756. 2020-11-02T12:07:05.0697849Z [cluster3] I1102 12:04:34.873629 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  9757. 2020-11-02T12:07:05.0700299Z [cluster3] I1102 12:04:34.873642 1 route.go:612] This route agent is running on a non-gateway/non-active node
  9758. 2020-11-02T12:07:05.0701233Z [cluster3] I1102 12:04:34.873650 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  9759. 2020-11-02T12:07:05.0703650Z [cluster3] I1102 12:04:34.873844 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  9760. 2020-11-02T12:07:05.0704207Z [cluster3] I1102 12:04:34.876616 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  9761. 2020-11-02T12:07:05.0705073Z [cluster3] I1102 12:04:34.876762 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9762. 2020-11-02T12:07:05.0705707Z [cluster3] I1102 12:04:34.876790 1 route.go:793] Found route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9763. 2020-11-02T12:07:05.0706907Z [cluster3] I1102 12:04:34.876798 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9764. 2020-11-02T12:07:05.0707870Z [cluster3] I1102 12:04:34.876823 1 route.go:793] Found route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9765. 2020-11-02T12:07:05.0708695Z [cluster3] I1102 12:04:34.876829 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9766. 2020-11-02T12:07:05.0709732Z [cluster3] I1102 12:04:34.876837 1 route.go:793] Found route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9767. 2020-11-02T12:07:05.0710385Z [cluster3] I1102 12:04:34.876844 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9768. 2020-11-02T12:07:05.0711040Z [cluster3] I1102 12:04:34.876851 1 route.go:793] Found route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9769. 2020-11-02T12:07:05.0711669Z [cluster3] I1102 12:04:34.876857 1 route.go:787] Processing route {Ifindex: 14 Dst: 240.0.0.0/8 Src: 240.17.0.5 Gw: <nil> Flags: [] Table: 254}
  9770. 2020-11-02T12:07:05.0712165Z [cluster3] I1102 12:04:34.876863 1 route.go:790] Found nil gw or dst
  9771. 2020-11-02T12:07:05.0712767Z [cluster3] I1102 12:04:34.876932 1 route.go:829] Found equivalent route, not adding
  9772. 2020-11-02T12:07:05.0713257Z [cluster3] I1102 12:04:34.876941 1 route.go:829] Found equivalent route, not adding
  9773. 2020-11-02T12:07:05.0713736Z [cluster3] I1102 12:04:34.876947 1 route.go:829] Found equivalent route, not adding
  9774. 2020-11-02T12:07:05.0714214Z [cluster3] I1102 12:04:34.876955 1 route.go:829] Found equivalent route, not adding
  9775. 2020-11-02T12:07:05.0715711Z [cluster3] I1102 12:04:34.876982 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  9776. 2020-11-02T12:07:05.0717683Z [cluster3] I1102 12:04:34.876994 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  9777. 2020-11-02T12:07:05.0720391Z [cluster3] I1102 12:05:04.873799 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  9778. 2020-11-02T12:07:05.0723223Z [cluster3] I1102 12:05:04.873835 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  9779. 2020-11-02T12:07:05.0725031Z [cluster3] I1102 12:05:04.873869 1 route.go:612] This route agent is running on a non-gateway/non-active node
  9780. 2020-11-02T12:07:05.0725877Z [cluster3] I1102 12:05:04.873877 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  9781. 2020-11-02T12:07:05.0727135Z [cluster3] I1102 12:05:04.874117 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  9782. 2020-11-02T12:07:05.0727695Z [cluster3] I1102 12:05:04.906036 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  9783. 2020-11-02T12:07:05.0728231Z [cluster3] I1102 12:05:04.906615 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9784. 2020-11-02T12:07:05.0728846Z [cluster3] I1102 12:05:04.906660 1 route.go:793] Found route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9785. 2020-11-02T12:07:05.0729475Z [cluster3] I1102 12:05:04.906671 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9786. 2020-11-02T12:07:05.0730102Z [cluster3] I1102 12:05:04.906679 1 route.go:793] Found route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9787. 2020-11-02T12:07:05.0730916Z [cluster3] I1102 12:05:04.906686 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9788. 2020-11-02T12:07:05.0731820Z [cluster3] I1102 12:05:04.906713 1 route.go:793] Found route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9789. 2020-11-02T12:07:05.0733000Z [cluster3] I1102 12:05:04.906720 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9790. 2020-11-02T12:07:05.0733695Z [cluster3] I1102 12:05:04.906726 1 route.go:793] Found route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9791. 2020-11-02T12:07:05.0734342Z [cluster3] I1102 12:05:04.906733 1 route.go:787] Processing route {Ifindex: 14 Dst: 240.0.0.0/8 Src: 240.17.0.5 Gw: <nil> Flags: [] Table: 254}
  9792. 2020-11-02T12:07:05.0735117Z [cluster3] I1102 12:05:04.906740 1 route.go:790] Found nil gw or dst
  9793. 2020-11-02T12:07:05.0736221Z [cluster3] I1102 12:05:04.906868 1 route.go:829] Found equivalent route, not adding
  9794. 2020-11-02T12:07:05.0736810Z [cluster3] I1102 12:05:04.906877 1 route.go:829] Found equivalent route, not adding
  9795. 2020-11-02T12:07:05.0737390Z [cluster3] I1102 12:05:04.906886 1 route.go:829] Found equivalent route, not adding
  9796. 2020-11-02T12:07:05.0737963Z [cluster3] I1102 12:05:04.906895 1 route.go:829] Found equivalent route, not adding
  9797. 2020-11-02T12:07:05.0739491Z [cluster3] I1102 12:05:04.906928 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  9798. 2020-11-02T12:07:05.0742539Z [cluster3] I1102 12:05:04.906949 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  9799. 2020-11-02T12:07:05.0747698Z [cluster3] I1102 12:05:30.618024 1 route.go:682] Informed of removed endpoint: {"metadata":{"name":"cluster2-submariner-cable-cluster2-172-17-0-4","namespace":"submariner-operator","selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4","uid":"94d2ede3-9111-44d5-a192-56c2299fcfe7","resourceVersion":"3680","generation":1,"creationTimestamp":"2020-11-02T12:01:40Z","labels":{"submariner-io/clusterID":"cluster2"}},"spec":{"cluster_id":"cluster2","cable_name":"submariner-cable-cluster2-172-17-0-4","hostname":"cluster2-worker","subnets":["100.2.0.0/16","10.2.0.0/16"],"private_ip":"172.17.0.4","public_ip":"","nat_enabled":false,"backend":"libreswan"}}
  9800. 2020-11-02T12:07:05.0753028Z [cluster3] I1102 12:05:30.683619 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11" with subnets [100.2.0.0/16 10.2.0.0/16]
  9801. 2020-11-02T12:07:05.0755707Z [cluster3] I1102 12:05:34.874339 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11" with subnets [100.2.0.0/16 10.2.0.0/16]
  9802. 2020-11-02T12:07:05.0758500Z [cluster3] I1102 12:05:34.874376 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  9803. 2020-11-02T12:07:05.0760323Z [cluster3] I1102 12:05:34.874386 1 route.go:612] This route agent is running on a non-gateway/non-active node
  9804. 2020-11-02T12:07:05.0761570Z [cluster3] I1102 12:05:34.874394 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  9805. 2020-11-02T12:07:05.0762263Z [cluster3] I1102 12:05:34.874569 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  9806. 2020-11-02T12:07:05.0763033Z [cluster3] I1102 12:05:34.877629 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  9807. 2020-11-02T12:07:05.0763768Z [cluster3] I1102 12:05:34.877797 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9808. 2020-11-02T12:07:05.0764966Z [cluster3] I1102 12:05:34.877822 1 route.go:793] Found route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9809. 2020-11-02T12:07:05.0766802Z [cluster3] I1102 12:05:34.877830 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9810. 2020-11-02T12:07:05.0767714Z [cluster3] I1102 12:05:34.877838 1 route.go:793] Found route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9811. 2020-11-02T12:07:05.0768665Z [cluster3] I1102 12:05:34.877845 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9812. 2020-11-02T12:07:05.0769340Z [cluster3] I1102 12:05:34.877853 1 route.go:793] Found route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9813. 2020-11-02T12:07:05.0771787Z [cluster3] I1102 12:05:34.877859 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9814. 2020-11-02T12:07:05.0772622Z [cluster3] I1102 12:05:34.877866 1 route.go:793] Found route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9815. 2020-11-02T12:07:05.0773441Z [cluster3] I1102 12:05:34.877872 1 route.go:787] Processing route {Ifindex: 14 Dst: 240.0.0.0/8 Src: 240.17.0.5 Gw: <nil> Flags: [] Table: 254}
  9816. 2020-11-02T12:07:05.0774093Z [cluster3] I1102 12:05:34.877879 1 route.go:790] Found nil gw or dst
  9817. 2020-11-02T12:07:05.0774849Z [cluster3] I1102 12:05:34.877945 1 route.go:829] Found equivalent route, not adding
  9818. 2020-11-02T12:07:05.0775522Z [cluster3] I1102 12:05:34.877953 1 route.go:829] Found equivalent route, not adding
  9819. 2020-11-02T12:07:05.0776335Z [cluster3] I1102 12:05:34.877961 1 route.go:829] Found equivalent route, not adding
  9820. 2020-11-02T12:07:05.0776870Z [cluster3] I1102 12:05:34.877969 1 route.go:829] Found equivalent route, not adding
  9821. 2020-11-02T12:07:05.0791016Z [cluster3] I1102 12:05:34.878014 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  9822. 2020-11-02T12:07:05.0807305Z [cluster3] I1102 12:05:34.878048 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  9823. 2020-11-02T12:07:05.0809307Z [cluster3] I1102 12:06:04.874349 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11" with subnets [100.2.0.0/16 10.2.0.0/16]
  9824. 2020-11-02T12:07:05.0811286Z [cluster3] I1102 12:06:04.874386 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  9825. 2020-11-02T12:07:05.0812934Z [cluster3] I1102 12:06:04.874396 1 route.go:612] This route agent is running on a non-gateway/non-active node
  9826. 2020-11-02T12:07:05.0813869Z [cluster3] I1102 12:06:04.874404 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  9827. 2020-11-02T12:07:05.0814603Z [cluster3] I1102 12:06:04.874597 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  9828. 2020-11-02T12:07:05.0815387Z [cluster3] I1102 12:06:04.877936 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  9829. 2020-11-02T12:07:05.0816148Z [cluster3] I1102 12:06:04.878139 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9830. 2020-11-02T12:07:05.0816949Z [cluster3] I1102 12:06:04.878180 1 route.go:793] Found route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9831. 2020-11-02T12:07:05.0817649Z [cluster3] I1102 12:06:04.878190 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9832. 2020-11-02T12:07:05.0818355Z [cluster3] I1102 12:06:04.878199 1 route.go:793] Found route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9833. 2020-11-02T12:07:05.0819937Z [cluster3] I1102 12:06:04.878206 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9834. 2020-11-02T12:07:05.0820704Z [cluster3] I1102 12:06:04.878214 1 route.go:793] Found route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9835. 2020-11-02T12:07:05.0821375Z [cluster3] I1102 12:06:04.878221 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9836. 2020-11-02T12:07:05.0822043Z [cluster3] I1102 12:06:04.878230 1 route.go:793] Found route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9837. 2020-11-02T12:07:05.0822723Z [cluster3] I1102 12:06:04.878236 1 route.go:787] Processing route {Ifindex: 14 Dst: 240.0.0.0/8 Src: 240.17.0.5 Gw: <nil> Flags: [] Table: 254}
  9838. 2020-11-02T12:07:05.0823234Z [cluster3] I1102 12:06:04.878243 1 route.go:790] Found nil gw or dst
  9839. 2020-11-02T12:07:05.0823701Z [cluster3] I1102 12:06:04.878324 1 route.go:829] Found equivalent route, not adding
  9840. 2020-11-02T12:07:05.0824224Z [cluster3] I1102 12:06:04.878333 1 route.go:829] Found equivalent route, not adding
  9841. 2020-11-02T12:07:05.0824736Z [cluster3] I1102 12:06:04.878341 1 route.go:829] Found equivalent route, not adding
  9842. 2020-11-02T12:07:05.0825453Z [cluster3] I1102 12:06:04.878348 1 route.go:829] Found equivalent route, not adding
  9843. 2020-11-02T12:07:05.0827216Z [cluster3] I1102 12:06:04.878378 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  9844. 2020-11-02T12:07:05.0829405Z [cluster3] I1102 12:06:04.878392 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  9845. 2020-11-02T12:07:05.0831285Z [cluster3] I1102 12:06:34.875738 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11" with subnets [100.2.0.0/16 10.2.0.0/16]
  9846. 2020-11-02T12:07:05.0833973Z [cluster3] I1102 12:06:34.875780 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  9847. 2020-11-02T12:07:05.0835490Z [cluster3] I1102 12:06:34.875790 1 route.go:612] This route agent is running on a non-gateway/non-active node
  9848. 2020-11-02T12:07:05.0836419Z [cluster3] I1102 12:06:34.875798 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  9849. 2020-11-02T12:07:05.0837139Z [cluster3] I1102 12:06:34.876044 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  9850. 2020-11-02T12:07:05.0837884Z [cluster3] I1102 12:06:34.879131 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  9851. 2020-11-02T12:07:05.0838467Z [cluster3] I1102 12:06:34.879292 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9852. 2020-11-02T12:07:05.0839175Z [cluster3] I1102 12:06:34.879323 1 route.go:793] Found route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9853. 2020-11-02T12:07:05.0839873Z [cluster3] I1102 12:06:34.879332 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9854. 2020-11-02T12:07:05.0840569Z [cluster3] I1102 12:06:34.879340 1 route.go:793] Found route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9855. 2020-11-02T12:07:05.0841266Z [cluster3] I1102 12:06:34.879347 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9856. 2020-11-02T12:07:05.0842590Z [cluster3] I1102 12:06:34.879354 1 route.go:793] Found route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9857. 2020-11-02T12:07:05.0843373Z [cluster3] I1102 12:06:34.879366 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9858. 2020-11-02T12:07:05.0844463Z [cluster3] I1102 12:06:34.879373 1 route.go:793] Found route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9859. 2020-11-02T12:07:05.0845156Z [cluster3] I1102 12:06:34.879379 1 route.go:787] Processing route {Ifindex: 14 Dst: 240.0.0.0/8 Src: 240.17.0.5 Gw: <nil> Flags: [] Table: 254}
  9860. 2020-11-02T12:07:05.0845669Z [cluster3] I1102 12:06:34.879386 1 route.go:790] Found nil gw or dst
  9861. 2020-11-02T12:07:05.0846142Z [cluster3] I1102 12:06:34.879461 1 route.go:829] Found equivalent route, not adding
  9862. 2020-11-02T12:07:05.7342408Z [submariner]$ [cluster3] kubectl -n submariner-operator logs submariner-routeagent-wvgxr
  9863. 2020-11-02T12:07:05.7343519Z [cluster3] I1102 12:06:34.879469 1 route.go:829] Found equivalent route, not adding
  9864. 2020-11-02T12:07:05.7344901Z [submariner]$ [cluster3] kubectl -n submariner-operator logs submariner-routeagent-wvgxr
  9865. 2020-11-02T12:07:05.7346868Z [submariner]$ [cluster3] command kubectl --context=cluster3 -n submariner-operator logs submariner-routeagent-wvgxr
  9866. 2020-11-02T12:07:05.7348211Z [submariner]$ [cluster3] kubectl --context=cluster3 -n submariner-operator logs submariner-routeagent-wvgxr
  9867. 2020-11-02T12:07:05.7349265Z [submariner]$ [cluster3] subctl show all
  9868. 2020-11-02T12:07:05.7349890Z [cluster3] I1102 12:06:34.879478 1 route.go:829] Found equivalent route, not adding
  9869. 2020-11-02T12:07:05.7350497Z [cluster3] I1102 12:06:34.879485 1 route.go:829] Found equivalent route, not adding
  9870. 2020-11-02T12:07:05.7352039Z [cluster3] I1102 12:06:34.879514 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  9871. 2020-11-02T12:07:05.7354640Z [cluster3] I1102 12:06:34.879528 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  9872. 2020-11-02T12:07:05.7357314Z [cluster3] I1102 12:07:04.875726 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11" with subnets [100.2.0.0/16 10.2.0.0/16]
  9873. 2020-11-02T12:07:05.7359821Z [cluster3] I1102 12:07:04.875771 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  9874. 2020-11-02T12:07:05.7361944Z [cluster3] I1102 12:07:04.875784 1 route.go:612] This route agent is running on a non-gateway/non-active node
  9875. 2020-11-02T12:07:05.7363383Z [cluster3] I1102 12:07:04.875792 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  9876. 2020-11-02T12:07:05.7374175Z [cluster3] I1102 12:07:04.876057 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  9877. 2020-11-02T12:07:05.7374791Z [cluster3] I1102 12:07:04.885778 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  9878. 2020-11-02T12:07:05.7375361Z [cluster3] I1102 12:07:04.885970 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9879. 2020-11-02T12:07:05.7376406Z [cluster3] I1102 12:07:04.885992 1 route.go:793] Found route {Ifindex: 14 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9880. 2020-11-02T12:07:05.7377314Z [cluster3] I1102 12:07:04.886001 1 route.go:787] Processing route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9881. 2020-11-02T12:07:05.7378043Z [cluster3] I1102 12:07:04.886254 1 route.go:793] Found route {Ifindex: 14 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9882. 2020-11-02T12:07:05.7378768Z [cluster3] I1102 12:07:04.886264 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9883. 2020-11-02T12:07:05.7380736Z [cluster3] I1102 12:07:04.886274 1 route.go:793] Found route {Ifindex: 14 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9884. 2020-11-02T12:07:05.7381470Z [cluster3] I1102 12:07:04.886300 1 route.go:787] Processing route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9885. 2020-11-02T12:07:05.7382163Z [cluster3] I1102 12:07:04.886310 1 route.go:793] Found route {Ifindex: 14 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9886. 2020-11-02T12:07:05.7383020Z [cluster3] I1102 12:07:04.886318 1 route.go:787] Processing route {Ifindex: 14 Dst: 240.0.0.0/8 Src: 240.17.0.5 Gw: <nil> Flags: [] Table: 254}
  9887. 2020-11-02T12:07:05.7383681Z [cluster3] I1102 12:07:04.886326 1 route.go:790] Found nil gw or dst
  9888. 2020-11-02T12:07:05.7384128Z [cluster3] I1102 12:07:04.886498 1 route.go:829] Found equivalent route, not adding
  9889. 2020-11-02T12:07:05.7384802Z [cluster3] I1102 12:07:04.886510 1 route.go:829] Found equivalent route, not adding
  9890. 2020-11-02T12:07:05.7385320Z [cluster3] I1102 12:07:04.886535 1 route.go:829] Found equivalent route, not adding
  9891. 2020-11-02T12:07:05.7385836Z [cluster3] I1102 12:07:04.886542 1 route.go:829] Found equivalent route, not adding
  9892. 2020-11-02T12:07:05.7387235Z [cluster3] I1102 12:07:04.886592 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  9893. 2020-11-02T12:07:05.7389407Z [cluster3] I1102 12:07:04.886607 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  9894. 2020-11-02T12:07:05.7390923Z [cluster3] +++++++++++++++++++++: Logs for Pod submariner-routeagent-szklb in namespace submariner-operator :++++++++++++++++++++++
  9895. 2020-11-02T12:07:05.7391742Z [cluster3] + trap 'exit 1' SIGTERM SIGINT
  9896. 2020-11-02T12:07:05.7392131Z [cluster3] + SUBMARINER_VERBOSITY=2
  9897. 2020-11-02T12:07:05.7392801Z [cluster3] + '[' false == true ']'
  9898. 2020-11-02T12:07:05.7393222Z [cluster3] + DEBUG=-v=2
  9899. 2020-11-02T12:07:05.7393715Z [cluster3] + for f in iptables-save iptables
  9900. 2020-11-02T12:07:05.7394319Z [cluster3] ++ find_iptables_on_host iptables-save
  9901. 2020-11-02T12:07:05.7394955Z [cluster3] ++ chroot /host test -x /usr/sbin/iptables-save
  9902. 2020-11-02T12:07:05.7395510Z [cluster3] ++ echo /usr/sbin
  9903. 2020-11-02T12:07:05.7395798Z [cluster3] ++ return
  9904. 2020-11-02T12:07:05.7396399Z [cluster3] iptables-save is present on the host at /usr/sbin/iptables-save
  9905. 2020-11-02T12:07:05.7396873Z [cluster3] + location=/usr/sbin
  9906. 2020-11-02T12:07:05.7397506Z [cluster3] + '[' /usr/sbin '!=' unknown ']'
  9907. 2020-11-02T12:07:05.7398145Z [cluster3] + echo 'iptables-save is present on the host at /usr/sbin/iptables-save'
  9908. 2020-11-02T12:07:05.7398908Z [cluster3] + sed 's!@@PATH@@!/usr/sbin!' /usr/sbin/iptables-wrapper.in
  9909. 2020-11-02T12:07:05.7399544Z [cluster3] + for f in iptables-save iptables
  9910. 2020-11-02T12:07:05.7399968Z [cluster3] ++ find_iptables_on_host iptables
  9911. 2020-11-02T12:07:05.7400524Z [cluster3] ++ chroot /host test -x /usr/sbin/iptables
  9912. 2020-11-02T12:07:05.7400885Z [cluster3] ++ echo /usr/sbin
  9913. 2020-11-02T12:07:05.7401284Z [cluster3] iptables is present on the host at /usr/sbin/iptables
  9914. 2020-11-02T12:07:05.7401669Z [cluster3] ++ return
  9915. 2020-11-02T12:07:05.7401953Z [cluster3] + location=/usr/sbin
  9916. 2020-11-02T12:07:05.7402437Z [cluster3] + '[' /usr/sbin '!=' unknown ']'
  9917. 2020-11-02T12:07:05.7403027Z [cluster3] + echo 'iptables is present on the host at /usr/sbin/iptables'
  9918. 2020-11-02T12:07:05.7403796Z [cluster3] + sed 's!@@PATH@@!/usr/sbin!' /usr/sbin/iptables-wrapper.in
  9919. 2020-11-02T12:07:05.7404785Z [cluster3] + exec submariner-route-agent -v=2 -alsologtostderr
  9920. 2020-11-02T12:07:05.7405617Z [cluster3] I1102 11:58:06.271461 1 main.go:47] Starting submariner-route-agent
  9921. 2020-11-02T12:07:05.7406749Z [cluster3] W1102 11:58:06.361017 1 client_config.go:543] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
  9922. 2020-11-02T12:07:05.7407749Z [cluster3] I1102 11:58:06.368825 1 route.go:192] Starting Route Controller. ClusterID: cluster3, localClusterCIDR: [10.3.0.0/16], localServiceCIDR: [100.3.0.0/16]
  9923. 2020-11-02T12:07:05.7408541Z [cluster3] I1102 11:58:06.368959 1 route.go:196] Waiting for Endpoint informer caches to sync.
  9924. 2020-11-02T12:07:05.7409379Z [cluster3] I1102 11:58:06.392227 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.10
  9925. 2020-11-02T12:07:05.7410212Z [cluster3] I1102 11:58:06.392257 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.7
  9926. 2020-11-02T12:07:05.7411196Z [cluster3] I1102 11:58:06.392264 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.5
  9927. 2020-11-02T12:07:05.7411760Z [cluster3] I1102 11:58:06.469423 1 driver.go:37] Interface "lo" has "127.0.0.1" address
  9928. 2020-11-02T12:07:05.7412252Z [cluster3] I1102 11:58:06.469644 1 driver.go:37] Interface "weave" has "10.3.0.1" address
  9929. 2020-11-02T12:07:05.7412847Z [cluster3] I1102 11:58:06.469665 1 driver.go:42] Found CNI Interface "weave" that has IP "10.3.0.1" from ClusterCIDR "10.3.0.0/16"
  9930. 2020-11-02T12:07:05.7413573Z [cluster3] I1102 11:58:06.469782 1 driver.go:60] Successfully configured rp_filter to loose mode(2) on cniInterface "weave"
  9931. 2020-11-02T12:07:05.7414565Z [cluster3] I1102 11:58:06.610191 1 iptables.go:22] Install/ensure SUBMARINER-POSTROUTING chain exists
  9932. 2020-11-02T12:07:05.7415643Z [cluster3] I1102 11:58:06.625123 1 iptables.go:28] Insert SUBMARINER-POSTROUTING rule that has rules for inter-cluster traffic
  9933. 2020-11-02T12:07:05.7416786Z [cluster3] I1102 11:58:06.632669 1 iptables.go:34] Install/ensure SUBMARINER-INPUT chain exists
  9934. 2020-11-02T12:07:05.7417648Z [cluster3] I1102 11:58:06.652512 1 iptables.go:45] Allow VxLAN incoming traffic in SUBMARINER-INPUT Chain
  9935. 2020-11-02T12:07:05.7418601Z [cluster3] I1102 11:58:06.660891 1 iptables.go:53] Insert rule to allow traffic over vx-submariner interface in FORWARDing Chain
  9936. 2020-11-02T12:07:05.7419684Z [cluster3] I1102 11:58:06.669094 1 iptables.go:65] Installing rule for host network to remote cluster communication: -s 240.0.0.0/8 -o vx-submariner -j SNAT --to 10.3.0.1
  9937. 2020-11-02T12:07:05.7420683Z [cluster3] I1102 11:58:07.118148 1 route.go:254] In cluster3, podIP of submariner-route-agent[0] is 172.17.0.5
  9938. 2020-11-02T12:07:05.7421587Z [cluster3] I1102 11:58:07.119223 1 route.go:254] In cluster3, podIP of submariner-route-agent[1] is 172.17.0.10
  9939. 2020-11-02T12:07:05.7422483Z [cluster3] I1102 11:58:07.130973 1 route.go:254] In cluster3, podIP of submariner-route-agent[2] is 172.17.0.7
  9940. 2020-11-02T12:07:05.7423743Z [cluster3] I1102 11:58:07.131098 1 route.go:268] Hostname is "cluster3-control-plane" and routeAgentNodeName is "cluster3-control-plane"
  9941. 2020-11-02T12:07:05.7424952Z [cluster3] I1102 11:58:07.390019 1 driver.go:87] Successfully annotated node "cluster3-control-plane" with cniIfaceIP "10.3.0.1"
  9942. 2020-11-02T12:07:05.7425628Z [cluster3] I1102 11:58:07.390207 1 route.go:286] Route agent workers started
  9943. 2020-11-02T12:07:05.7426679Z [cluster3] I1102 11:58:07.390334 1 route.go:507] Processing submariner-route-agent pod "submariner-routeagent-szklb" with host IP "172.17.0.10", pod IP "172.17.0.10"
  9944. 2020-11-02T12:07:05.7428017Z [cluster3] I1102 11:58:07.390610 1 route.go:531] Successfully processed submariner-route-agent pod "submariner-routeagent-szklb"
  9945. 2020-11-02T12:07:05.7429385Z [cluster3] I1102 11:58:07.390781 1 route.go:507] Processing submariner-route-agent pod "submariner-routeagent-wvgxr" with host IP "172.17.0.7", pod IP "172.17.0.7"
  9946. 2020-11-02T12:07:05.7430732Z [cluster3] I1102 11:58:07.390941 1 route.go:531] Successfully processed submariner-route-agent pod "submariner-routeagent-wvgxr"
  9947. 2020-11-02T12:07:05.7432103Z [cluster3] I1102 11:58:07.391083 1 route.go:507] Processing submariner-route-agent pod "submariner-routeagent-46pd2" with host IP "172.17.0.5", pod IP "172.17.0.5"
  9948. 2020-11-02T12:07:05.7433515Z [cluster3] I1102 11:58:07.391170 1 route.go:531] Successfully processed submariner-route-agent pod "submariner-routeagent-46pd2"
  9949. 2020-11-02T12:07:05.7435206Z [cluster3] I1102 11:58:24.121190 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  9950. 2020-11-02T12:07:05.7436702Z [cluster3] I1102 11:58:24.141049 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.3.0.0/16 -d 100.2.0.0/16 -j ACCEPT
  9951. 2020-11-02T12:07:05.7437568Z [cluster3] I1102 11:58:24.159424 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 100.2.0.0/16 -d 10.3.0.0/16 -j ACCEPT
  9952. 2020-11-02T12:07:05.7438448Z [cluster3] I1102 11:58:24.174658 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.3.0.0/16 -d 10.2.0.0/16 -j ACCEPT
  9953. 2020-11-02T12:07:05.7439312Z [cluster3] I1102 11:58:24.236219 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 10.2.0.0/16 -d 10.3.0.0/16 -j ACCEPT
  9954. 2020-11-02T12:07:05.7440662Z [cluster3] I1102 11:58:24.281569 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  9955. 2020-11-02T12:07:05.7441935Z [cluster3] I1102 11:58:24.281592 1 route.go:612] This route agent is running on a non-gateway/non-active node
  9956. 2020-11-02T12:07:05.7442770Z [cluster3] I1102 11:58:24.281601 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  9957. 2020-11-02T12:07:05.7443332Z [cluster3] I1102 11:58:24.319424 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  9958. 2020-11-02T12:07:05.7445116Z [cluster3] I1102 11:58:24.319682 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.10 Gw: <nil> Flags: [] Table: 254}
  9959. 2020-11-02T12:07:05.7445649Z [cluster3] I1102 11:58:24.319722 1 route.go:790] Found nil gw or dst
  9960. 2020-11-02T12:07:05.7494907Z [cluster3] I1102 11:58:24.319962 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  9961. 2020-11-02T12:07:05.7497638Z [cluster3] I1102 11:58:27.571947 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  9962. 2020-11-02T12:07:05.7499362Z [cluster3] I1102 11:58:27.607264 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.3.0.0/16 -d 100.1.0.0/16 -j ACCEPT
  9963. 2020-11-02T12:07:05.7500735Z [cluster3] I1102 11:58:27.645039 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 100.1.0.0/16 -d 10.3.0.0/16 -j ACCEPT
  9964. 2020-11-02T12:07:05.7502761Z [cluster3] I1102 11:58:27.702423 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.3.0.0/16 -d 10.1.0.0/16 -j ACCEPT
  9965. 2020-11-02T12:07:05.7503902Z [cluster3] I1102 11:58:27.730954 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 10.1.0.0/16 -d 10.3.0.0/16 -j ACCEPT
  9966. 2020-11-02T12:07:05.7505574Z [cluster3] I1102 11:58:36.387007 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  9967. 2020-11-02T12:07:05.7507845Z [cluster3] I1102 11:58:36.387057 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  9968. 2020-11-02T12:07:05.7509936Z [cluster3] I1102 11:58:36.387084 1 route.go:612] This route agent is running on a non-gateway/non-active node
  9969. 2020-11-02T12:07:05.7511419Z [cluster3] I1102 11:58:36.387117 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  9970. 2020-11-02T12:07:05.7512344Z [cluster3] I1102 11:58:36.387612 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  9971. 2020-11-02T12:07:05.7513291Z [cluster3] I1102 11:58:36.393354 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  9972. 2020-11-02T12:07:05.7513999Z [cluster3] I1102 11:58:36.393534 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9973. 2020-11-02T12:07:05.7514859Z [cluster3] I1102 11:58:36.393691 1 route.go:793] Found route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9974. 2020-11-02T12:07:05.7515538Z [cluster3] I1102 11:58:36.393716 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9975. 2020-11-02T12:07:05.7516215Z [cluster3] I1102 11:58:36.393766 1 route.go:793] Found route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9976. 2020-11-02T12:07:05.7516895Z [cluster3] I1102 11:58:36.393816 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.10 Gw: <nil> Flags: [] Table: 254}
  9977. 2020-11-02T12:07:05.7517393Z [cluster3] I1102 11:58:36.393928 1 route.go:790] Found nil gw or dst
  9978. 2020-11-02T12:07:05.7517859Z [cluster3] I1102 11:58:36.394165 1 route.go:829] Found equivalent route, not adding
  9979. 2020-11-02T12:07:05.7518899Z [cluster3] I1102 11:58:36.394181 1 route.go:829] Found equivalent route, not adding
  9980. 2020-11-02T12:07:05.7520752Z [cluster3] I1102 11:58:36.394243 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  9981. 2020-11-02T12:07:05.7522989Z [cluster3] I1102 11:58:36.394267 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  9982. 2020-11-02T12:07:05.7547789Z [cluster3] I1102 11:59:06.387120 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  9983. 2020-11-02T12:07:05.7549260Z [cluster3] I1102 11:59:06.387153 1 route.go:612] This route agent is running on a non-gateway/non-active node
  9984. 2020-11-02T12:07:05.7550200Z [cluster3] I1102 11:59:06.387163 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  9985. 2020-11-02T12:07:05.7550923Z [cluster3] I1102 11:59:06.387662 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  9986. 2020-11-02T12:07:05.7551524Z [cluster3] I1102 11:59:06.405080 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  9987. 2020-11-02T12:07:05.7552110Z [cluster3] I1102 11:59:06.405469 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9988. 2020-11-02T12:07:05.7553008Z [cluster3] I1102 11:59:06.405637 1 route.go:793] Found route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9989. 2020-11-02T12:07:05.7553709Z [cluster3] I1102 11:59:06.405809 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9990. 2020-11-02T12:07:05.7554565Z [cluster3] I1102 11:59:06.405913 1 route.go:793] Found route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9991. 2020-11-02T12:07:05.7555241Z [cluster3] I1102 11:59:06.406008 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9992. 2020-11-02T12:07:05.7555915Z [cluster3] I1102 11:59:06.406084 1 route.go:793] Found route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9993. 2020-11-02T12:07:05.7556584Z [cluster3] I1102 11:59:06.406164 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  9994. 2020-11-02T12:07:05.7557257Z [cluster3] I1102 11:59:06.406234 1 route.go:793] Found route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  9995. 2020-11-02T12:07:05.7557981Z [cluster3] I1102 11:59:06.406340 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.10 Gw: <nil> Flags: [] Table: 254}
  9996. 2020-11-02T12:07:05.7558500Z [cluster3] I1102 11:59:06.409860 1 route.go:790] Found nil gw or dst
  9997. 2020-11-02T12:07:05.7558963Z [cluster3] I1102 11:59:06.410124 1 route.go:829] Found equivalent route, not adding
  9998. 2020-11-02T12:07:05.7559644Z [cluster3] I1102 11:59:06.410265 1 route.go:829] Found equivalent route, not adding
  9999. 2020-11-02T12:07:05.7560160Z [cluster3] I1102 11:59:06.410357 1 route.go:829] Found equivalent route, not adding
  10000. 2020-11-02T12:07:05.7560684Z [cluster3] I1102 11:59:06.410495 1 route.go:829] Found equivalent route, not adding
  10001. 2020-11-02T12:07:05.7561993Z [cluster3] I1102 11:59:06.410718 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  10002. 2020-11-02T12:07:05.7564513Z [cluster3] I1102 11:59:06.410868 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10003. 2020-11-02T12:07:05.7566517Z [cluster3] I1102 11:59:06.410969 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10004. 2020-11-02T12:07:05.7568389Z [cluster3] I1102 11:59:36.387214 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10005. 2020-11-02T12:07:05.7570281Z [cluster3] I1102 11:59:36.387476 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10006. 2020-11-02T12:07:05.7571825Z [cluster3] I1102 11:59:36.387606 1 route.go:612] This route agent is running on a non-gateway/non-active node
  10007. 2020-11-02T12:07:05.7572730Z [cluster3] I1102 11:59:36.387755 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  10008. 2020-11-02T12:07:05.7573424Z [cluster3] I1102 11:59:36.388128 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10009. 2020-11-02T12:07:05.7574003Z [cluster3] I1102 11:59:36.392135 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  10010. 2020-11-02T12:07:05.7574696Z [cluster3] I1102 11:59:36.392474 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10011. 2020-11-02T12:07:05.7575353Z [cluster3] I1102 11:59:36.398594 1 route.go:793] Found route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10012. 2020-11-02T12:07:05.7576154Z [cluster3] I1102 11:59:36.398772 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10013. 2020-11-02T12:07:05.7576815Z [cluster3] I1102 11:59:36.398925 1 route.go:793] Found route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10014. 2020-11-02T12:07:05.7577477Z [cluster3] I1102 11:59:36.399089 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10015. 2020-11-02T12:07:05.7578125Z [cluster3] I1102 11:59:36.399206 1 route.go:793] Found route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10016. 2020-11-02T12:07:05.7578775Z [cluster3] I1102 11:59:36.399353 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10017. 2020-11-02T12:07:05.7579426Z [cluster3] I1102 11:59:36.399588 1 route.go:793] Found route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10018. 2020-11-02T12:07:05.7580068Z [cluster3] I1102 11:59:36.399738 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.10 Gw: <nil> Flags: [] Table: 254}
  10019. 2020-11-02T12:07:05.7580632Z [cluster3] I1102 11:59:36.399846 1 route.go:790] Found nil gw or dst
  10020. 2020-11-02T12:07:05.7581085Z [cluster3] I1102 11:59:36.400086 1 route.go:829] Found equivalent route, not adding
  10021. 2020-11-02T12:07:05.7581587Z [cluster3] I1102 11:59:36.400215 1 route.go:829] Found equivalent route, not adding
  10022. 2020-11-02T12:07:05.7582088Z [cluster3] I1102 11:59:36.400307 1 route.go:829] Found equivalent route, not adding
  10023. 2020-11-02T12:07:05.7582584Z [cluster3] I1102 11:59:36.400445 1 route.go:829] Found equivalent route, not adding
  10024. 2020-11-02T12:07:05.7583970Z [cluster3] I1102 11:59:36.400738 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  10025. 2020-11-02T12:07:05.7585763Z [cluster3] I1102 11:59:36.400898 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10026. 2020-11-02T12:07:05.7587748Z [cluster3] I1102 12:00:06.387435 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10027. 2020-11-02T12:07:05.7589581Z [cluster3] I1102 12:00:06.387476 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10028. 2020-11-02T12:07:05.7590889Z [cluster3] I1102 12:00:06.387486 1 route.go:612] This route agent is running on a non-gateway/non-active node
  10029. 2020-11-02T12:07:05.7591740Z [cluster3] I1102 12:00:06.387494 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  10030. 2020-11-02T12:07:05.7592518Z [cluster3] I1102 12:00:06.388211 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10031. 2020-11-02T12:07:05.7593080Z [cluster3] I1102 12:00:06.394000 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  10032. 2020-11-02T12:07:05.7593628Z [cluster3] I1102 12:00:06.394221 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10033. 2020-11-02T12:07:05.7594282Z [cluster3] I1102 12:00:06.394270 1 route.go:793] Found route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10034. 2020-11-02T12:07:05.7594933Z [cluster3] I1102 12:00:06.394340 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10035. 2020-11-02T12:07:05.7595747Z [cluster3] I1102 12:00:06.394363 1 route.go:793] Found route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10036. 2020-11-02T12:07:05.7596487Z [cluster3] I1102 12:00:06.394371 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10037. 2020-11-02T12:07:05.7597121Z [cluster3] I1102 12:00:06.394380 1 route.go:793] Found route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10038. 2020-11-02T12:07:05.7597745Z [cluster3] I1102 12:00:06.394387 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10039. 2020-11-02T12:07:05.7598368Z [cluster3] I1102 12:00:06.394559 1 route.go:793] Found route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10040. 2020-11-02T12:07:05.7598989Z [cluster3] I1102 12:00:06.394580 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.10 Gw: <nil> Flags: [] Table: 254}
  10041. 2020-11-02T12:07:05.7599468Z [cluster3] I1102 12:00:06.394605 1 route.go:790] Found nil gw or dst
  10042. 2020-11-02T12:07:05.7599899Z [cluster3] I1102 12:00:06.394805 1 route.go:829] Found equivalent route, not adding
  10043. 2020-11-02T12:07:05.7600383Z [cluster3] I1102 12:00:06.394962 1 route.go:829] Found equivalent route, not adding
  10044. 2020-11-02T12:07:05.7600926Z [cluster3] I1102 12:00:06.395143 1 route.go:829] Found equivalent route, not adding
  10045. 2020-11-02T12:07:05.7601400Z [cluster3] I1102 12:00:06.395163 1 route.go:829] Found equivalent route, not adding
  10046. 2020-11-02T12:07:05.7602612Z [cluster3] I1102 12:00:06.395296 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  10047. 2020-11-02T12:07:05.7605375Z [cluster3] I1102 12:00:06.395324 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10048. 2020-11-02T12:07:05.7607191Z [cluster3] I1102 12:00:36.395313 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10049. 2020-11-02T12:07:05.7609027Z [cluster3] I1102 12:00:36.395370 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10050. 2020-11-02T12:07:05.7610341Z [cluster3] I1102 12:00:36.395388 1 route.go:612] This route agent is running on a non-gateway/non-active node
  10051. 2020-11-02T12:07:05.7612036Z [cluster3] I1102 12:00:36.395397 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  10052. 2020-11-02T12:07:05.7612819Z [cluster3] I1102 12:00:36.395588 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10053. 2020-11-02T12:07:05.7613464Z [cluster3] I1102 12:00:36.527212 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  10054. 2020-11-02T12:07:05.7614092Z [cluster3] I1102 12:00:36.536518 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10055. 2020-11-02T12:07:05.7614856Z [cluster3] I1102 12:00:36.539016 1 route.go:793] Found route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10056. 2020-11-02T12:07:05.7615607Z [cluster3] I1102 12:00:36.539251 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10057. 2020-11-02T12:07:05.7616357Z [cluster3] I1102 12:00:36.539558 1 route.go:793] Found route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10058. 2020-11-02T12:07:05.7617103Z [cluster3] I1102 12:00:36.540495 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10059. 2020-11-02T12:07:05.7659149Z [cluster3] I1102 12:00:36.540995 1 route.go:793] Found route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10060. 2020-11-02T12:07:05.7659961Z [cluster3] I1102 12:00:36.541471 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10061. 2020-11-02T12:07:05.7660587Z [cluster3] I1102 12:00:36.544078 1 route.go:793] Found route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10062. 2020-11-02T12:07:05.7661198Z [cluster3] I1102 12:00:36.544885 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.10 Gw: <nil> Flags: [] Table: 254}
  10063. 2020-11-02T12:07:05.7661665Z [cluster3] I1102 12:00:36.545319 1 route.go:790] Found nil gw or dst
  10064. 2020-11-02T12:07:05.7662090Z [cluster3] I1102 12:00:36.546723 1 route.go:829] Found equivalent route, not adding
  10065. 2020-11-02T12:07:05.7662551Z [cluster3] I1102 12:00:36.546917 1 route.go:829] Found equivalent route, not adding
  10066. 2020-11-02T12:07:05.7663207Z [cluster3] I1102 12:00:36.547518 1 route.go:829] Found equivalent route, not adding
  10067. 2020-11-02T12:07:05.7663919Z [cluster3] I1102 12:00:36.548265 1 route.go:829] Found equivalent route, not adding
  10068. 2020-11-02T12:07:05.7665770Z [cluster3] I1102 12:00:36.548474 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  10069. 2020-11-02T12:07:05.7667906Z [cluster3] I1102 12:00:36.549355 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10070. 2020-11-02T12:07:05.7670042Z [cluster3] I1102 12:01:06.391873 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10071. 2020-11-02T12:07:05.7671437Z [cluster3] I1102 12:01:06.391921 1 route.go:612] This route agent is running on a non-gateway/non-active node
  10072. 2020-11-02T12:07:05.7672279Z [cluster3] I1102 12:01:06.391930 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  10073. 2020-11-02T12:07:05.7672944Z [cluster3] I1102 12:01:06.392210 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10074. 2020-11-02T12:07:05.7673703Z [cluster3] I1102 12:01:06.396432 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  10075. 2020-11-02T12:07:05.7674413Z [cluster3] I1102 12:01:06.396690 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10076. 2020-11-02T12:07:05.7675059Z [cluster3] I1102 12:01:06.396724 1 route.go:793] Found route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10077. 2020-11-02T12:07:05.7675693Z [cluster3] I1102 12:01:06.396733 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10078. 2020-11-02T12:07:05.7676325Z [cluster3] I1102 12:01:06.396741 1 route.go:793] Found route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10079. 2020-11-02T12:07:05.7676961Z [cluster3] I1102 12:01:06.396754 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10080. 2020-11-02T12:07:05.7677595Z [cluster3] I1102 12:01:06.396762 1 route.go:793] Found route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10081. 2020-11-02T12:07:05.7678222Z [cluster3] I1102 12:01:06.396775 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10082. 2020-11-02T12:07:05.7678830Z [cluster3] I1102 12:01:06.396801 1 route.go:793] Found route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10083. 2020-11-02T12:07:05.7679453Z [cluster3] I1102 12:01:06.396808 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.10 Gw: <nil> Flags: [] Table: 254}
  10084. 2020-11-02T12:07:05.7680044Z [cluster3] I1102 12:01:06.396823 1 route.go:790] Found nil gw or dst
  10085. 2020-11-02T12:07:05.7680473Z [cluster3] I1102 12:01:06.396902 1 route.go:829] Found equivalent route, not adding
  10086. 2020-11-02T12:07:05.7681129Z [cluster3] I1102 12:01:06.396919 1 route.go:829] Found equivalent route, not adding
  10087. 2020-11-02T12:07:05.7681601Z [cluster3] I1102 12:01:06.396926 1 route.go:829] Found equivalent route, not adding
  10088. 2020-11-02T12:07:05.7682069Z [cluster3] I1102 12:01:06.396934 1 route.go:829] Found equivalent route, not adding
  10089. 2020-11-02T12:07:05.7683813Z [cluster3] I1102 12:01:06.396971 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  10090. 2020-11-02T12:07:05.7686036Z [cluster3] I1102 12:01:06.396998 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10091. 2020-11-02T12:07:05.7687920Z [cluster3] I1102 12:01:06.397023 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10092. 2020-11-02T12:07:05.7689719Z [cluster3] I1102 12:01:36.392033 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10093. 2020-11-02T12:07:05.7691483Z [cluster3] I1102 12:01:36.392070 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10094. 2020-11-02T12:07:05.7692908Z [cluster3] I1102 12:01:36.392234 1 route.go:612] This route agent is running on a non-gateway/non-active node
  10095. 2020-11-02T12:07:05.7694138Z [cluster3] I1102 12:01:36.392269 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  10096. 2020-11-02T12:07:05.7694963Z [cluster3] I1102 12:01:36.392572 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10097. 2020-11-02T12:07:05.7699121Z [cluster3] I1102 12:01:36.396827 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  10098. 2020-11-02T12:07:05.7699635Z [cluster3] I1102 12:01:36.397038 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10099. 2020-11-02T12:07:05.7700265Z [cluster3] I1102 12:01:36.397148 1 route.go:793] Found route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10100. 2020-11-02T12:07:05.7700883Z [cluster3] I1102 12:01:36.397171 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10101. 2020-11-02T12:07:05.7701579Z [cluster3] I1102 12:01:36.397180 1 route.go:793] Found route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10102. 2020-11-02T12:07:05.7702200Z [cluster3] I1102 12:01:36.397187 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10103. 2020-11-02T12:07:05.7702810Z [cluster3] I1102 12:01:36.397194 1 route.go:793] Found route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10104. 2020-11-02T12:07:05.7703816Z [cluster3] I1102 12:01:36.397200 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10105. 2020-11-02T12:07:05.7704615Z [cluster3] I1102 12:01:36.397207 1 route.go:793] Found route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10106. 2020-11-02T12:07:05.7705219Z [cluster3] I1102 12:01:36.397312 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.10 Gw: <nil> Flags: [] Table: 254}
  10107. 2020-11-02T12:07:05.7705677Z [cluster3] I1102 12:01:36.397322 1 route.go:790] Found nil gw or dst
  10108. 2020-11-02T12:07:05.7706258Z [cluster3] I1102 12:01:36.397631 1 route.go:829] Found equivalent route, not adding
  10109. 2020-11-02T12:07:05.7706732Z [cluster3] I1102 12:01:36.397652 1 route.go:829] Found equivalent route, not adding
  10110. 2020-11-02T12:07:05.7707948Z [cluster3] I1102 12:01:36.397661 1 route.go:829] Found equivalent route, not adding
  10111. 2020-11-02T12:07:05.7708834Z [cluster3] I1102 12:01:36.397668 1 route.go:829] Found equivalent route, not adding
  10112. 2020-11-02T12:07:05.7710185Z [cluster3] I1102 12:01:36.397701 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  10113. 2020-11-02T12:07:05.7712618Z [cluster3] I1102 12:01:36.397826 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10114. 2020-11-02T12:07:05.7717852Z [cluster3] I1102 12:01:40.309384 1 route.go:682] Informed of removed endpoint: {"metadata":{"name":"cluster2-submariner-cable-cluster2-172-17-0-4","namespace":"submariner-operator","selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4","uid":"8c63de0a-c225-4c4f-b602-4d0ec35f7bae","resourceVersion":"2815","generation":1,"creationTimestamp":"2020-11-02T11:58:24Z","labels":{"submariner-io/clusterID":"cluster2"}},"spec":{"cluster_id":"cluster2","cable_name":"submariner-cable-cluster2-172-17-0-4","hostname":"cluster2-worker","subnets":["100.2.0.0/16","10.2.0.0/16"],"private_ip":"172.17.0.4","public_ip":"","nat_enabled":false,"backend":"libreswan"}}
  10115. 2020-11-02T12:07:05.7725816Z [cluster3] I1102 12:01:40.385445 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10116. 2020-11-02T12:07:05.7728142Z [cluster3] I1102 12:02:06.392429 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10117. 2020-11-02T12:07:05.7729934Z [cluster3] I1102 12:02:06.392485 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10118. 2020-11-02T12:07:05.7731264Z [cluster3] I1102 12:02:06.392494 1 route.go:612] This route agent is running on a non-gateway/non-active node
  10119. 2020-11-02T12:07:05.7732112Z [cluster3] I1102 12:02:06.392503 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  10120. 2020-11-02T12:07:05.7732769Z [cluster3] I1102 12:02:06.392900 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10121. 2020-11-02T12:07:05.7737355Z [cluster3] I1102 12:02:06.397354 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  10122. 2020-11-02T12:07:05.7738156Z [cluster3] I1102 12:02:06.397533 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10123. 2020-11-02T12:07:05.7739404Z [cluster3] I1102 12:02:06.397563 1 route.go:793] Found route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10124. 2020-11-02T12:07:05.7740448Z [cluster3] I1102 12:02:06.397573 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10125. 2020-11-02T12:07:05.7744022Z [cluster3] I1102 12:02:06.397580 1 route.go:793] Found route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10126. 2020-11-02T12:07:05.7744667Z [cluster3] I1102 12:02:06.397616 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10127. 2020-11-02T12:07:05.7745283Z [cluster3] I1102 12:02:06.397637 1 route.go:793] Found route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10128. 2020-11-02T12:07:05.7745896Z [cluster3] I1102 12:02:06.397669 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10129. 2020-11-02T12:07:05.7746656Z [cluster3] I1102 12:02:06.397710 1 route.go:793] Found route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10130. 2020-11-02T12:07:05.7747271Z [cluster3] I1102 12:02:06.397742 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.10 Gw: <nil> Flags: [] Table: 254}
  10131. 2020-11-02T12:07:05.7747720Z [cluster3] I1102 12:02:06.397770 1 route.go:790] Found nil gw or dst
  10132. 2020-11-02T12:07:05.7748140Z [cluster3] I1102 12:02:06.397861 1 route.go:829] Found equivalent route, not adding
  10133. 2020-11-02T12:07:05.7748614Z [cluster3] I1102 12:02:06.397915 1 route.go:829] Found equivalent route, not adding
  10134. 2020-11-02T12:07:05.7749080Z [cluster3] I1102 12:02:06.397955 1 route.go:829] Found equivalent route, not adding
  10135. 2020-11-02T12:07:05.7749543Z [cluster3] I1102 12:02:06.397986 1 route.go:829] Found equivalent route, not adding
  10136. 2020-11-02T12:07:05.7756008Z [cluster3] I1102 12:02:06.398049 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  10137. 2020-11-02T12:07:05.7758472Z [cluster3] I1102 12:02:06.398073 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10138. 2020-11-02T12:07:05.7760243Z [cluster3] I1102 12:02:36.392476 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10139. 2020-11-02T12:07:05.7822195Z [cluster3] I1102 12:02:36.392514 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10140. 2020-11-02T12:07:05.7824137Z [cluster3] I1102 12:02:36.392525 1 route.go:612] This route agent is running on a non-gateway/non-active node
  10141. 2020-11-02T12:07:05.7825528Z [cluster3] I1102 12:02:36.392535 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  10142. 2020-11-02T12:07:05.7826182Z [cluster3] I1102 12:02:36.392792 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10143. 2020-11-02T12:07:05.7826897Z [cluster3] I1102 12:02:36.397042 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  10144. 2020-11-02T12:07:05.7827440Z [cluster3] I1102 12:02:36.397282 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10145. 2020-11-02T12:07:05.7828097Z [cluster3] I1102 12:02:36.397323 1 route.go:793] Found route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10146. 2020-11-02T12:07:05.7828730Z [cluster3] I1102 12:02:36.397332 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10147. 2020-11-02T12:07:05.7829389Z [cluster3] I1102 12:02:36.397446 1 route.go:793] Found route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10148. 2020-11-02T12:07:05.7830086Z [cluster3] I1102 12:02:36.397555 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10149. 2020-11-02T12:07:05.7830740Z [cluster3] I1102 12:02:36.397686 1 route.go:793] Found route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10150. 2020-11-02T12:07:05.7831392Z [cluster3] I1102 12:02:36.397741 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10151. 2020-11-02T12:07:05.7832041Z [cluster3] I1102 12:02:36.397750 1 route.go:793] Found route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10152. 2020-11-02T12:07:05.7832816Z [cluster3] I1102 12:02:36.397757 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.10 Gw: <nil> Flags: [] Table: 254}
  10153. 2020-11-02T12:07:05.7833522Z [cluster3] I1102 12:02:36.397765 1 route.go:790] Found nil gw or dst
  10154. 2020-11-02T12:07:05.7834148Z [cluster3] I1102 12:02:36.397904 1 route.go:829] Found equivalent route, not adding
  10155. 2020-11-02T12:07:05.7834830Z [cluster3] I1102 12:02:36.397926 1 route.go:829] Found equivalent route, not adding
  10156. 2020-11-02T12:07:05.7835330Z [cluster3] I1102 12:02:36.397935 1 route.go:829] Found equivalent route, not adding
  10157. 2020-11-02T12:07:05.7835823Z [cluster3] I1102 12:02:36.398003 1 route.go:829] Found equivalent route, not adding
  10158. 2020-11-02T12:07:05.7837237Z [cluster3] I1102 12:02:36.398123 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  10159. 2020-11-02T12:07:05.7838934Z [cluster3] I1102 12:02:36.398150 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10160. 2020-11-02T12:07:05.7840690Z [cluster3] I1102 12:03:06.392522 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10161. 2020-11-02T12:07:05.7842703Z [cluster3] I1102 12:03:06.392563 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10162. 2020-11-02T12:07:05.7845259Z [cluster3] I1102 12:03:06.392572 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10163. 2020-11-02T12:07:05.7846541Z [cluster3] I1102 12:03:06.392585 1 route.go:612] This route agent is running on a non-gateway/non-active node
  10164. 2020-11-02T12:07:05.7847383Z [cluster3] I1102 12:03:06.392750 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  10165. 2020-11-02T12:07:05.7848026Z [cluster3] I1102 12:03:06.393051 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10166. 2020-11-02T12:07:05.7848565Z [cluster3] I1102 12:03:06.396913 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  10167. 2020-11-02T12:07:05.7849098Z [cluster3] I1102 12:03:06.397166 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10168. 2020-11-02T12:07:05.7849733Z [cluster3] I1102 12:03:06.397198 1 route.go:793] Found route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10169. 2020-11-02T12:07:05.7850366Z [cluster3] I1102 12:03:06.397209 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10170. 2020-11-02T12:07:05.7850997Z [cluster3] I1102 12:03:06.397300 1 route.go:793] Found route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10171. 2020-11-02T12:07:05.7851632Z [cluster3] I1102 12:03:06.397320 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10172. 2020-11-02T12:07:05.7852266Z [cluster3] I1102 12:03:06.397329 1 route.go:793] Found route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10173. 2020-11-02T12:07:05.7852895Z [cluster3] I1102 12:03:06.397335 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10174. 2020-11-02T12:07:05.7853897Z [cluster3] I1102 12:03:06.397500 1 route.go:793] Found route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10175. 2020-11-02T12:07:05.7854702Z [cluster3] I1102 12:03:06.397522 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.10 Gw: <nil> Flags: [] Table: 254}
  10176. 2020-11-02T12:07:05.7855304Z [cluster3] I1102 12:03:06.397530 1 route.go:790] Found nil gw or dst
  10177. 2020-11-02T12:07:05.7855734Z [cluster3] I1102 12:03:06.397655 1 route.go:829] Found equivalent route, not adding
  10178. 2020-11-02T12:07:05.7856219Z [cluster3] I1102 12:03:06.397693 1 route.go:829] Found equivalent route, not adding
  10179. 2020-11-02T12:07:05.7856854Z [cluster3] I1102 12:03:06.397701 1 route.go:829] Found equivalent route, not adding
  10180. 2020-11-02T12:07:05.7857317Z [cluster3] I1102 12:03:06.397797 1 route.go:829] Found equivalent route, not adding
  10181. 2020-11-02T12:07:05.7858468Z [cluster3] I1102 12:03:06.397882 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  10182. 2020-11-02T12:07:05.7860135Z [cluster3] I1102 12:03:36.393438 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10183. 2020-11-02T12:07:05.7861358Z [cluster3] I1102 12:03:36.393689 1 route.go:612] This route agent is running on a non-gateway/non-active node
  10184. 2020-11-02T12:07:05.7862335Z [cluster3] I1102 12:03:36.393719 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  10185. 2020-11-02T12:07:05.7863228Z [cluster3] I1102 12:03:36.393974 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10186. 2020-11-02T12:07:05.7864059Z [cluster3] I1102 12:03:36.397252 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  10187. 2020-11-02T12:07:05.7864915Z [cluster3] I1102 12:03:36.397469 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10188. 2020-11-02T12:07:05.7865527Z [cluster3] I1102 12:03:36.397555 1 route.go:793] Found route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10189. 2020-11-02T12:07:05.7866135Z [cluster3] I1102 12:03:36.397639 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10190. 2020-11-02T12:07:05.7866752Z [cluster3] I1102 12:03:36.397684 1 route.go:793] Found route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10191. 2020-11-02T12:07:05.7867365Z [cluster3] I1102 12:03:36.397727 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10192. 2020-11-02T12:07:05.7867974Z [cluster3] I1102 12:03:36.397769 1 route.go:793] Found route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10193. 2020-11-02T12:07:05.7868577Z [cluster3] I1102 12:03:36.397793 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10194. 2020-11-02T12:07:05.7869179Z [cluster3] I1102 12:03:36.397813 1 route.go:793] Found route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10195. 2020-11-02T12:07:05.7869784Z [cluster3] I1102 12:03:36.397846 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.10 Gw: <nil> Flags: [] Table: 254}
  10196. 2020-11-02T12:07:05.7870245Z [cluster3] I1102 12:03:36.397906 1 route.go:790] Found nil gw or dst
  10197. 2020-11-02T12:07:05.7870661Z [cluster3] I1102 12:03:36.398057 1 route.go:829] Found equivalent route, not adding
  10198. 2020-11-02T12:07:05.7871119Z [cluster3] I1102 12:03:36.398102 1 route.go:829] Found equivalent route, not adding
  10199. 2020-11-02T12:07:05.7871641Z [cluster3] I1102 12:03:36.398134 1 route.go:829] Found equivalent route, not adding
  10200. 2020-11-02T12:07:05.7872112Z [cluster3] I1102 12:03:36.398155 1 route.go:829] Found equivalent route, not adding
  10201. 2020-11-02T12:07:05.7873680Z [cluster3] I1102 12:03:36.398212 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  10202. 2020-11-02T12:07:05.7875524Z [cluster3] I1102 12:03:36.399193 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10203. 2020-11-02T12:07:05.7877479Z [cluster3] I1102 12:03:36.399263 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10204. 2020-11-02T12:07:05.7879129Z [cluster3] I1102 12:04:06.393513 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10205. 2020-11-02T12:07:05.7880797Z [cluster3] I1102 12:04:06.393584 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10206. 2020-11-02T12:07:05.7882245Z [cluster3] I1102 12:04:06.393593 1 route.go:612] This route agent is running on a non-gateway/non-active node
  10207. 2020-11-02T12:07:05.7883239Z [cluster3] I1102 12:04:06.393748 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  10208. 2020-11-02T12:07:05.7915880Z [cluster3] I1102 12:04:06.394052 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10209. 2020-11-02T12:07:05.7916781Z [cluster3] I1102 12:04:06.407190 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  10210. 2020-11-02T12:07:05.7942778Z [cluster3] I1102 12:04:06.407427 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10211. 2020-11-02T12:07:05.7943957Z [cluster3] I1102 12:04:06.407491 1 route.go:793] Found route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10212. 2020-11-02T12:07:05.7944835Z [cluster3] I1102 12:04:06.407517 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10213. 2020-11-02T12:07:05.7945804Z [cluster3] I1102 12:04:06.407652 1 route.go:793] Found route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10214. 2020-11-02T12:07:05.7946480Z [cluster3] I1102 12:04:06.407689 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10215. 2020-11-02T12:07:05.7947291Z [cluster3] I1102 12:04:06.407735 1 route.go:793] Found route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10216. 2020-11-02T12:07:05.7947924Z [cluster3] I1102 12:04:06.407781 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10217. 2020-11-02T12:07:05.7948554Z [cluster3] I1102 12:04:06.407814 1 route.go:793] Found route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10218. 2020-11-02T12:07:05.7949164Z [cluster3] I1102 12:04:06.407860 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.10 Gw: <nil> Flags: [] Table: 254}
  10219. 2020-11-02T12:07:05.7949648Z [cluster3] I1102 12:04:06.407895 1 route.go:790] Found nil gw or dst
  10220. 2020-11-02T12:07:05.7950083Z [cluster3] I1102 12:04:06.408062 1 route.go:829] Found equivalent route, not adding
  10221. 2020-11-02T12:07:05.7950570Z [cluster3] I1102 12:04:06.408131 1 route.go:829] Found equivalent route, not adding
  10222. 2020-11-02T12:07:05.7951054Z [cluster3] I1102 12:04:06.408156 1 route.go:829] Found equivalent route, not adding
  10223. 2020-11-02T12:07:05.7951555Z [cluster3] I1102 12:04:06.408198 1 route.go:829] Found equivalent route, not adding
  10224. 2020-11-02T12:07:05.7952883Z [cluster3] I1102 12:04:06.408287 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  10225. 2020-11-02T12:07:05.7954997Z [cluster3] I1102 12:04:06.408327 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10226. 2020-11-02T12:07:05.7957091Z [cluster3] I1102 12:04:36.393664 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10227. 2020-11-02T12:07:05.7958822Z [cluster3] I1102 12:04:36.393725 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10228. 2020-11-02T12:07:05.7960064Z [cluster3] I1102 12:04:36.394049 1 route.go:612] This route agent is running on a non-gateway/non-active node
  10229. 2020-11-02T12:07:05.7960880Z [cluster3] I1102 12:04:36.394072 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  10230. 2020-11-02T12:07:05.7961511Z [cluster3] I1102 12:04:36.394416 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10231. 2020-11-02T12:07:05.7962036Z [cluster3] I1102 12:04:36.397530 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  10232. 2020-11-02T12:07:05.7962553Z [cluster3] I1102 12:04:36.397754 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10233. 2020-11-02T12:07:05.7963332Z [cluster3] I1102 12:04:36.397784 1 route.go:793] Found route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10234. 2020-11-02T12:07:05.7965291Z [cluster3] I1102 12:04:36.397793 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10235. 2020-11-02T12:07:05.7965935Z [cluster3] I1102 12:04:36.397880 1 route.go:793] Found route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10236. 2020-11-02T12:07:05.7966553Z [cluster3] I1102 12:04:36.397890 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10237. 2020-11-02T12:07:05.7967261Z [cluster3] I1102 12:04:36.397898 1 route.go:793] Found route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10238. 2020-11-02T12:07:05.7967882Z [cluster3] I1102 12:04:36.397945 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10239. 2020-11-02T12:07:05.7968490Z [cluster3] I1102 12:04:36.397978 1 route.go:793] Found route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10240. 2020-11-02T12:07:05.7969095Z [cluster3] I1102 12:04:36.397986 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.10 Gw: <nil> Flags: [] Table: 254}
  10241. 2020-11-02T12:07:05.7969559Z [cluster3] I1102 12:04:36.397993 1 route.go:790] Found nil gw or dst
  10242. 2020-11-02T12:07:05.7969977Z [cluster3] I1102 12:04:36.398188 1 route.go:829] Found equivalent route, not adding
  10243. 2020-11-02T12:07:05.7970449Z [cluster3] I1102 12:04:36.398208 1 route.go:829] Found equivalent route, not adding
  10244. 2020-11-02T12:07:05.7970919Z [cluster3] I1102 12:04:36.398216 1 route.go:829] Found equivalent route, not adding
  10245. 2020-11-02T12:07:05.7971384Z [cluster3] I1102 12:04:36.398301 1 route.go:829] Found equivalent route, not adding
  10246. 2020-11-02T12:07:05.7972619Z [cluster3] I1102 12:04:36.398414 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  10247. 2020-11-02T12:07:05.7974838Z [cluster3] I1102 12:04:36.398439 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10248. 2020-11-02T12:07:05.7976542Z [cluster3] I1102 12:05:06.393769 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10249. 2020-11-02T12:07:05.7978458Z [cluster3] I1102 12:05:06.394019 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10250. 2020-11-02T12:07:05.7980047Z [cluster3] I1102 12:05:06.394116 1 route.go:612] This route agent is running on a non-gateway/non-active node
  10251. 2020-11-02T12:07:05.7981005Z [cluster3] I1102 12:05:06.394168 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  10252. 2020-11-02T12:07:05.7981843Z [cluster3] I1102 12:05:06.394565 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10253. 2020-11-02T12:07:05.7982368Z [cluster3] I1102 12:05:06.401030 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  10254. 2020-11-02T12:07:05.7982879Z [cluster3] I1102 12:05:06.401967 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10255. 2020-11-02T12:07:05.7983872Z [cluster3] I1102 12:05:06.402029 1 route.go:793] Found route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10256. 2020-11-02T12:07:05.7984662Z [cluster3] I1102 12:05:06.402039 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10257. 2020-11-02T12:07:05.7985469Z [cluster3] I1102 12:05:06.402046 1 route.go:793] Found route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10258. 2020-11-02T12:07:05.7986167Z [cluster3] I1102 12:05:06.402240 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10259. 2020-11-02T12:07:05.7986789Z [cluster3] I1102 12:05:06.402348 1 route.go:793] Found route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10260. 2020-11-02T12:07:05.7987398Z [cluster3] I1102 12:05:06.402415 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10261. 2020-11-02T12:07:05.7988002Z [cluster3] I1102 12:05:06.402482 1 route.go:793] Found route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10262. 2020-11-02T12:07:05.7988609Z [cluster3] I1102 12:05:06.402566 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.10 Gw: <nil> Flags: [] Table: 254}
  10263. 2020-11-02T12:07:05.7989068Z [cluster3] I1102 12:05:06.402649 1 route.go:790] Found nil gw or dst
  10264. 2020-11-02T12:07:05.7989490Z [cluster3] I1102 12:05:06.402769 1 route.go:829] Found equivalent route, not adding
  10265. 2020-11-02T12:07:05.7989960Z [cluster3] I1102 12:05:06.402841 1 route.go:829] Found equivalent route, not adding
  10266. 2020-11-02T12:07:05.7990427Z [cluster3] I1102 12:05:06.402905 1 route.go:829] Found equivalent route, not adding
  10267. 2020-11-02T12:07:05.7990893Z [cluster3] I1102 12:05:06.402966 1 route.go:829] Found equivalent route, not adding
  10268. 2020-11-02T12:07:05.8021920Z [cluster3] I1102 12:05:06.403056 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  10269. 2020-11-02T12:07:05.8024143Z [cluster3] I1102 12:05:06.403215 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10270. 2020-11-02T12:07:05.8029141Z [cluster3] I1102 12:05:30.623904 1 route.go:682] Informed of removed endpoint: {"metadata":{"name":"cluster2-submariner-cable-cluster2-172-17-0-4","namespace":"submariner-operator","selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4","uid":"94d2ede3-9111-44d5-a192-56c2299fcfe7","resourceVersion":"3680","generation":1,"creationTimestamp":"2020-11-02T12:01:40Z","labels":{"submariner-io/clusterID":"cluster2"}},"spec":{"cluster_id":"cluster2","cable_name":"submariner-cable-cluster2-172-17-0-4","hostname":"cluster2-worker","subnets":["100.2.0.0/16","10.2.0.0/16"],"private_ip":"172.17.0.4","public_ip":"","nat_enabled":false,"backend":"libreswan"}}
  10271. 2020-11-02T12:07:05.8033031Z [cluster3] I1102 12:05:30.684279 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11" with subnets [100.2.0.0/16 10.2.0.0/16]
  10272. 2020-11-02T12:07:05.8035401Z [cluster3] I1102 12:05:36.393956 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10273. 2020-11-02T12:07:05.8037637Z [cluster3] I1102 12:05:36.394010 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11" with subnets [100.2.0.0/16 10.2.0.0/16]
  10274. 2020-11-02T12:07:05.8039714Z [cluster3] I1102 12:05:36.394021 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10275. 2020-11-02T12:07:05.8040940Z [cluster3] I1102 12:05:36.394037 1 route.go:612] This route agent is running on a non-gateway/non-active node
  10276. 2020-11-02T12:07:05.8041761Z [cluster3] I1102 12:05:36.394047 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  10277. 2020-11-02T12:07:05.8042400Z [cluster3] I1102 12:05:36.394413 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10278. 2020-11-02T12:07:05.8043207Z [cluster3] I1102 12:05:36.397803 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  10279. 2020-11-02T12:07:05.8044612Z [cluster3] I1102 12:05:36.398105 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10280. 2020-11-02T12:07:05.8045719Z [cluster3] I1102 12:05:36.398155 1 route.go:793] Found route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10281. 2020-11-02T12:07:05.8046569Z [cluster3] I1102 12:05:36.398165 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10282. 2020-11-02T12:07:05.8047365Z [cluster3] I1102 12:05:36.398262 1 route.go:793] Found route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10283. 2020-11-02T12:07:05.8047998Z [cluster3] I1102 12:05:36.398279 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10284. 2020-11-02T12:07:05.8048637Z [cluster3] I1102 12:05:36.398287 1 route.go:793] Found route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10285. 2020-11-02T12:07:05.8049270Z [cluster3] I1102 12:05:36.398295 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10286. 2020-11-02T12:07:05.8049900Z [cluster3] I1102 12:05:36.398302 1 route.go:793] Found route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10287. 2020-11-02T12:07:05.8050529Z [cluster3] I1102 12:05:36.398309 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.10 Gw: <nil> Flags: [] Table: 254}
  10288. 2020-11-02T12:07:05.8051014Z [cluster3] I1102 12:05:36.398315 1 route.go:790] Found nil gw or dst
  10289. 2020-11-02T12:07:05.8051446Z [cluster3] I1102 12:05:36.398499 1 route.go:829] Found equivalent route, not adding
  10290. 2020-11-02T12:07:05.8051936Z [cluster3] I1102 12:05:36.398617 1 route.go:829] Found equivalent route, not adding
  10291. 2020-11-02T12:07:05.8052420Z [cluster3] I1102 12:05:36.398774 1 route.go:829] Found equivalent route, not adding
  10292. 2020-11-02T12:07:05.8052900Z [cluster3] I1102 12:05:36.398859 1 route.go:829] Found equivalent route, not adding
  10293. 2020-11-02T12:07:05.8054166Z [cluster3] I1102 12:05:36.398968 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  10294. 2020-11-02T12:07:05.8086254Z [cluster3] I1102 12:06:06.394079 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11" with subnets [100.2.0.0/16 10.2.0.0/16]
  10295. 2020-11-02T12:07:05.8089359Z [cluster3] I1102 12:06:06.394116 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10296. 2020-11-02T12:07:05.8090706Z [cluster3] I1102 12:06:06.394125 1 route.go:612] This route agent is running on a non-gateway/non-active node
  10297. 2020-11-02T12:07:05.8092000Z [cluster3] I1102 12:06:06.394133 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  10298. 2020-11-02T12:07:05.8092975Z [cluster3] I1102 12:06:06.394389 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10299. 2020-11-02T12:07:05.8093581Z [cluster3] I1102 12:06:06.397832 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  10300. 2020-11-02T12:07:05.8094186Z [cluster3] I1102 12:06:06.398178 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10301. 2020-11-02T12:07:05.8095420Z [cluster3] I1102 12:06:06.398210 1 route.go:793] Found route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10302. 2020-11-02T12:07:05.8096322Z [cluster3] I1102 12:06:06.398244 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10303. 2020-11-02T12:07:05.8097164Z [cluster3] I1102 12:06:06.398362 1 route.go:793] Found route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10304. 2020-11-02T12:07:05.8097900Z [cluster3] I1102 12:06:06.398384 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10305. 2020-11-02T12:07:05.8099084Z [cluster3] I1102 12:06:06.398393 1 route.go:793] Found route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10306. 2020-11-02T12:07:05.8099760Z [cluster3] I1102 12:06:06.398416 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10307. 2020-11-02T12:07:05.8100426Z [cluster3] I1102 12:06:06.398434 1 route.go:793] Found route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10308. 2020-11-02T12:07:05.8101101Z [cluster3] I1102 12:06:06.398441 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.10 Gw: <nil> Flags: [] Table: 254}
  10309. 2020-11-02T12:07:05.8101683Z [cluster3] I1102 12:06:06.398450 1 route.go:790] Found nil gw or dst
  10310. 2020-11-02T12:07:05.8102144Z [cluster3] I1102 12:06:06.398553 1 route.go:829] Found equivalent route, not adding
  10311. 2020-11-02T12:07:05.8102868Z [cluster3] I1102 12:06:06.398585 1 route.go:829] Found equivalent route, not adding
  10312. 2020-11-02T12:07:05.8103400Z [cluster3] I1102 12:06:06.398595 1 route.go:829] Found equivalent route, not adding
  10313. 2020-11-02T12:07:05.8103934Z [cluster3] I1102 12:06:06.398602 1 route.go:829] Found equivalent route, not adding
  10314. 2020-11-02T12:07:05.8105733Z [cluster3] I1102 12:06:06.398637 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  10315. 2020-11-02T12:07:05.8107760Z [cluster3] I1102 12:06:06.398780 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10316. 2020-11-02T12:07:05.8109824Z [cluster3] I1102 12:06:36.394243 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11" with subnets [100.2.0.0/16 10.2.0.0/16]
  10317. 2020-11-02T12:07:05.8111875Z [cluster3] I1102 12:06:36.394481 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10318. 2020-11-02T12:07:05.8113243Z [cluster3] I1102 12:06:36.394609 1 route.go:612] This route agent is running on a non-gateway/non-active node
  10319. 2020-11-02T12:07:05.8114288Z [cluster3] I1102 12:06:36.394633 1 route.go:623] Creating the vxlan interface vx-submariner with gateway node IP 172.17.0.7
  10320. 2020-11-02T12:07:05.8114981Z [cluster3] I1102 12:06:36.395114 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10321. 2020-11-02T12:07:05.8115770Z [cluster3] I1102 12:06:36.398577 1 route.go:771] Reconciling routes to gw: 240.17.0.7
  10322. 2020-11-02T12:07:05.8116667Z [cluster3] I1102 12:06:36.398923 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10323. 2020-11-02T12:07:05.8117363Z [cluster3] I1102 12:06:36.398961 1 route.go:793] Found route {Ifindex: 15 Dst: 10.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10324. 2020-11-02T12:07:05.8118060Z [cluster3] I1102 12:06:36.398972 1 route.go:787] Processing route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10325. 2020-11-02T12:07:05.8118913Z [cluster3] I1102 12:06:36.398981 1 route.go:793] Found route {Ifindex: 15 Dst: 10.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10326. 2020-11-02T12:07:05.8119588Z [cluster3] I1102 12:06:36.398989 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10327. 2020-11-02T12:07:05.8120341Z [cluster3] I1102 12:06:36.399254 1 route.go:793] Found route {Ifindex: 15 Dst: 100.1.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10328. 2020-11-02T12:07:05.8121028Z [cluster3] I1102 12:06:36.399404 1 route.go:787] Processing route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254}
  10329. 2020-11-02T12:07:05.8121697Z [cluster3] I1102 12:06:36.399484 1 route.go:793] Found route {Ifindex: 15 Dst: 100.2.0.0/16 Src: <nil> Gw: 240.17.0.7 Flags: [] Table: 254} with gw 240.17.0.7 already installed
  10330. 2020-11-02T12:07:05.8122555Z [cluster3] I1102 12:06:36.399571 1 route.go:787] Processing route {Ifindex: 15 Dst: 240.0.0.0/8 Src: 240.17.0.10 Gw: <nil> Flags: [] Table: 254}
  10331. 2020-11-02T12:07:05.8123085Z [cluster3] I1102 12:06:36.399643 1 route.go:790] Found nil gw or dst
  10332. 2020-11-02T12:07:05.8123561Z [cluster3] I1102 12:06:36.399772 1 route.go:829] Found equivalent route, not adding
  10333. 2020-11-02T12:07:05.8124727Z [cluster3] I1102 12:06:36.399888 1 route.go:829] Found equivalent route, not adding
  10334. 2020-11-02T12:07:05.8125459Z [cluster3] I1102 12:06:36.399964 1 route.go:829] Found equivalent route, not adding
  10335. 2020-11-02T12:07:05.8126009Z [cluster3] I1102 12:06:36.400032 1 route.go:829] Found equivalent route, not adding
  10336. 2020-11-02T12:07:05.8127608Z [cluster3] I1102 12:06:36.400138 1 route.go:647] Successfully processed local Endpoint "cluster3-submariner-cable-cluster3-172-17-0-7"
  10337. 2020-11-02T12:07:05.8129835Z [cluster3] I1102 12:06:36.400165 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10338. 2020-11-02T12:07:05.8131401Z [cluster3] +++++++++++++++++++++: Logs for Pod submariner-routeagent-wvgxr in namespace submariner-operator :++++++++++++++++++++++
  10339. 2020-11-02T12:07:05.8132402Z [cluster3] + trap 'exit 1' SIGTERM SIGINT
  10340. 2020-11-02T12:07:05.8132847Z [cluster3] + SUBMARINER_VERBOSITY=2
  10341. 2020-11-02T12:07:05.8133468Z [cluster3] + '[' false == true ']'
  10342. 2020-11-02T12:07:05.8133895Z [cluster3] + DEBUG=-v=2
  10343. 2020-11-02T12:07:05.8134793Z [cluster3] + for f in iptables-save iptables
  10344. 2020-11-02T12:07:05.8135758Z [cluster3] ++ find_iptables_on_host iptables-save
  10345. 2020-11-02T12:07:05.8136441Z [cluster3] ++ chroot /host test -x /usr/sbin/iptables-save
  10346. 2020-11-02T12:07:05.8136875Z [cluster3] ++ echo /usr/sbin
  10347. 2020-11-02T12:07:05.8137194Z [cluster3] ++ return
  10348. 2020-11-02T12:07:05.8137533Z [cluster3] + location=/usr/sbin
  10349. 2020-11-02T12:07:05.8138360Z [cluster3] + '[' /usr/sbin '!=' unknown ']'
  10350. 2020-11-02T12:07:05.8139200Z [cluster3] + echo 'iptables-save is present on the host at /usr/sbin/iptables-save'
  10351. 2020-11-02T12:07:05.8139992Z [cluster3] + sed 's!@@PATH@@!/usr/sbin!' /usr/sbin/iptables-wrapper.in
  10352. 2020-11-02T12:07:05.8140747Z [cluster3] iptables-save is present on the host at /usr/sbin/iptables-save
  10353. 2020-11-02T12:07:05.8141440Z [cluster3] + for f in iptables-save iptables
  10354. 2020-11-02T12:07:05.8141879Z [cluster3] ++ find_iptables_on_host iptables
  10355. 2020-11-02T12:07:05.8142450Z [cluster3] ++ chroot /host test -x /usr/sbin/iptables
  10356. 2020-11-02T12:07:05.8142809Z [cluster3] ++ echo /usr/sbin
  10357. 2020-11-02T12:07:05.8143104Z [cluster3] ++ return
  10358. 2020-11-02T12:07:05.8143505Z [cluster3] iptables is present on the host at /usr/sbin/iptables
  10359. 2020-11-02T12:07:05.8143934Z [cluster3] + location=/usr/sbin
  10360. 2020-11-02T12:07:05.8144425Z [cluster3] + '[' /usr/sbin '!=' unknown ']'
  10361. 2020-11-02T12:07:05.8145033Z [cluster3] + echo 'iptables is present on the host at /usr/sbin/iptables'
  10362. 2020-11-02T12:07:05.8145767Z [cluster3] + sed 's!@@PATH@@!/usr/sbin!' /usr/sbin/iptables-wrapper.in
  10363. 2020-11-02T12:07:05.8146565Z [cluster3] + exec submariner-route-agent -v=2 -alsologtostderr
  10364. 2020-11-02T12:07:05.8147404Z [cluster3] I1102 11:58:22.358673 1 main.go:47] Starting submariner-route-agent
  10365. 2020-11-02T12:07:05.8191063Z [cluster3] W1102 11:58:22.359053 1 client_config.go:543] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
  10366. 2020-11-02T12:07:05.8192114Z [cluster3] I1102 11:58:22.361065 1 route.go:192] Starting Route Controller. ClusterID: cluster3, localClusterCIDR: [10.3.0.0/16], localServiceCIDR: [100.3.0.0/16]
  10367. 2020-11-02T12:07:05.8192912Z [cluster3] I1102 11:58:22.361653 1 route.go:196] Waiting for Endpoint informer caches to sync.
  10368. 2020-11-02T12:07:05.8193804Z [cluster3] I1102 11:58:22.433816 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.5
  10369. 2020-11-02T12:07:05.8194642Z [cluster3] I1102 11:58:22.434822 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.10
  10370. 2020-11-02T12:07:05.8195485Z [cluster3] I1102 11:58:22.435111 1 route.go:656] Enqueueing sm-route-agent-pod event, ip: 172.17.0.7
  10371. 2020-11-02T12:07:05.8196048Z [cluster3] I1102 11:58:22.462095 1 driver.go:37] Interface "lo" has "127.0.0.1" address
  10372. 2020-11-02T12:07:05.8196547Z [cluster3] I1102 11:58:22.462186 1 driver.go:37] Interface "weave" has "10.3.192.0" address
  10373. 2020-11-02T12:07:05.8197135Z [cluster3] I1102 11:58:22.462193 1 driver.go:42] Found CNI Interface "weave" that has IP "10.3.192.0" from ClusterCIDR "10.3.0.0/16"
  10374. 2020-11-02T12:07:05.8197861Z [cluster3] I1102 11:58:22.462257 1 driver.go:60] Successfully configured rp_filter to loose mode(2) on cniInterface "weave"
  10375. 2020-11-02T12:07:05.8198849Z [cluster3] I1102 11:58:22.470700 1 iptables.go:22] Install/ensure SUBMARINER-POSTROUTING chain exists
  10376. 2020-11-02T12:07:05.8200014Z [cluster3] I1102 11:58:22.490037 1 iptables.go:28] Insert SUBMARINER-POSTROUTING rule that has rules for inter-cluster traffic
  10377. 2020-11-02T12:07:05.8201007Z [cluster3] I1102 11:58:22.504540 1 iptables.go:34] Install/ensure SUBMARINER-INPUT chain exists
  10378. 2020-11-02T12:07:05.8201887Z [cluster3] I1102 11:58:22.528446 1 iptables.go:45] Allow VxLAN incoming traffic in SUBMARINER-INPUT Chain
  10379. 2020-11-02T12:07:05.8203023Z [cluster3] I1102 11:58:22.537515 1 iptables.go:53] Insert rule to allow traffic over vx-submariner interface in FORWARDing Chain
  10380. 2020-11-02T12:07:05.8204357Z [cluster3] I1102 11:58:22.548956 1 iptables.go:65] Installing rule for host network to remote cluster communication: -s 240.0.0.0/8 -o vx-submariner -j SNAT --to 10.3.192.0
  10381. 2020-11-02T12:07:05.8205417Z [cluster3] I1102 11:58:22.564611 1 route.go:254] In cluster3, podIP of submariner-route-agent[0] is 172.17.0.5
  10382. 2020-11-02T12:07:05.8206318Z [cluster3] I1102 11:58:22.564761 1 route.go:254] In cluster3, podIP of submariner-route-agent[1] is 172.17.0.10
  10383. 2020-11-02T12:07:05.8207217Z [cluster3] I1102 11:58:22.564927 1 route.go:254] In cluster3, podIP of submariner-route-agent[2] is 172.17.0.7
  10384. 2020-11-02T12:07:05.8208384Z [cluster3] I1102 11:58:22.565034 1 route.go:268] Hostname is "cluster3-worker" and routeAgentNodeName is "cluster3-worker"
  10385. 2020-11-02T12:07:05.8209418Z [cluster3] I1102 11:58:22.580050 1 driver.go:87] Successfully annotated node "cluster3-worker" with cniIfaceIP "10.3.192.0"
  10386. 2020-11-02T12:07:05.8210033Z [cluster3] I1102 11:58:22.580216 1 route.go:286] Route agent workers started
  10387. 2020-11-02T12:07:05.8211080Z [cluster3] I1102 11:58:22.580370 1 route.go:507] Processing submariner-route-agent pod "submariner-routeagent-46pd2" with host IP "172.17.0.5", pod IP "172.17.0.5"
  10388. 2020-11-02T12:07:05.8212418Z [cluster3] I1102 11:58:22.580600 1 route.go:531] Successfully processed submariner-route-agent pod "submariner-routeagent-46pd2"
  10389. 2020-11-02T12:07:05.8213789Z [cluster3] I1102 11:58:22.580746 1 route.go:507] Processing submariner-route-agent pod "submariner-routeagent-szklb" with host IP "172.17.0.10", pod IP "172.17.0.10"
  10390. 2020-11-02T12:07:05.8215136Z [cluster3] I1102 11:58:22.580846 1 route.go:531] Successfully processed submariner-route-agent pod "submariner-routeagent-szklb"
  10391. 2020-11-02T12:07:05.8216500Z [cluster3] I1102 11:58:22.580995 1 route.go:507] Processing submariner-route-agent pod "submariner-routeagent-wvgxr" with host IP "172.17.0.7", pod IP "172.17.0.7"
  10392. 2020-11-02T12:07:05.8218074Z [cluster3] I1102 11:58:22.581090 1 route.go:531] Successfully processed submariner-route-agent pod "submariner-routeagent-wvgxr"
  10393. 2020-11-02T12:07:05.8219670Z [cluster3] I1102 11:58:24.091367 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10394. 2020-11-02T12:07:05.8220968Z [cluster3] I1102 11:58:24.114031 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.3.0.0/16 -d 100.2.0.0/16 -j ACCEPT
  10395. 2020-11-02T12:07:05.8221827Z [cluster3] I1102 11:58:24.169260 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 100.2.0.0/16 -d 10.3.0.0/16 -j ACCEPT
  10396. 2020-11-02T12:07:05.8222701Z [cluster3] I1102 11:58:24.272910 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.3.0.0/16 -d 10.2.0.0/16 -j ACCEPT
  10397. 2020-11-02T12:07:05.8223570Z [cluster3] I1102 11:58:24.347881 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 10.2.0.0/16 -d 10.3.0.0/16 -j ACCEPT
  10398. 2020-11-02T12:07:05.8224923Z [cluster3] I1102 11:58:24.363077 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10399. 2020-11-02T12:07:05.8226007Z [cluster3] I1102 11:58:24.363356 1 route.go:756] Processing route {Ifindex: 14 Dst: <nil> Src: <nil> Gw: 172.17.0.1 Flags: [] Table: 254}
  10400. 2020-11-02T12:07:05.8226498Z [cluster3] I1102 11:58:24.363462 1 route.go:759] Found nil gw or dst
  10401. 2020-11-02T12:07:05.8226985Z [cluster3] I1102 11:58:24.363467 1 route.go:756] Processing route {Ifindex: 4 Dst: 10.3.0.0/16 Src: 10.3.192.0 Gw: <nil> Flags: [] Table: 254}
  10402. 2020-11-02T12:07:05.8227469Z [cluster3] I1102 11:58:24.363498 1 route.go:759] Found nil gw or dst
  10403. 2020-11-02T12:07:05.8227955Z [cluster3] I1102 11:58:24.363504 1 route.go:756] Processing route {Ifindex: 14 Dst: 172.17.0.0/16 Src: 172.17.0.7 Gw: <nil> Flags: [] Table: 254}
  10404. 2020-11-02T12:07:05.8228440Z [cluster3] I1102 11:58:24.363600 1 route.go:759] Found nil gw or dst
  10405. 2020-11-02T12:07:05.8228902Z [cluster3] I1102 11:58:24.363644 1 route.go:582] This route agent is running on the active gateway node
  10406. 2020-11-02T12:07:05.8229765Z [cluster3] I1102 11:58:24.363650 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  10407. 2020-11-02T12:07:05.8230369Z [cluster3] I1102 11:58:24.364486 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.5 00:00:00:00:00:00
  10408. 2020-11-02T12:07:05.8230910Z [cluster3] I1102 11:58:24.364545 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.10 00:00:00:00:00:00
  10409. 2020-11-02T12:07:05.8231555Z [cluster3] I1102 11:58:24.364571 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.7 00:00:00:00:00:00
  10410. 2020-11-02T12:07:05.8232388Z [cluster3] I1102 11:58:24.364617 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  10411. 2020-11-02T12:07:05.8233741Z [cluster3] I1102 11:58:27.569372 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10412. 2020-11-02T12:07:05.8235034Z [cluster3] I1102 11:58:27.588757 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.3.0.0/16 -d 100.1.0.0/16 -j ACCEPT
  10413. 2020-11-02T12:07:05.8236832Z [cluster3] I1102 11:58:27.681981 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 100.1.0.0/16 -d 10.3.0.0/16 -j ACCEPT
  10414. 2020-11-02T12:07:05.8237883Z [cluster3] I1102 11:58:27.762076 1 iptables.go:83] Installing iptables rule for outgoing traffic: -s 10.3.0.0/16 -d 10.1.0.0/16 -j ACCEPT
  10415. 2020-11-02T12:07:05.8239241Z [cluster3] I1102 11:58:27.798192 1 iptables.go:91] Installing iptables rule for incoming traffic: -s 10.1.0.0/16 -d 10.3.0.0/16 -j ACCEPT
  10416. 2020-11-02T12:07:05.8240860Z [cluster3] I1102 11:58:52.402537 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10417. 2020-11-02T12:07:05.8243408Z [cluster3] I1102 11:58:52.402587 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10418. 2020-11-02T12:07:05.8244627Z [cluster3] I1102 11:58:52.402836 1 route.go:756] Processing route {Ifindex: 13 Dst: 240.0.0.0/8 Src: 240.17.0.7 Gw: <nil> Flags: [] Table: 254}
  10419. 2020-11-02T12:07:05.8245111Z [cluster3] I1102 11:58:52.402870 1 route.go:759] Found nil gw or dst
  10420. 2020-11-02T12:07:05.8245777Z [cluster3] I1102 11:58:52.402878 1 route.go:582] This route agent is running on the active gateway node
  10421. 2020-11-02T12:07:05.8246629Z [cluster3] I1102 11:58:52.402885 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  10422. 2020-11-02T12:07:05.8247294Z [cluster3] I1102 11:58:52.403067 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10423. 2020-11-02T12:07:05.8247906Z [cluster3] I1102 11:58:52.403142 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.5 00:00:00:00:00:00
  10424. 2020-11-02T12:07:05.8248468Z [cluster3] I1102 11:58:52.403179 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.10 00:00:00:00:00:00
  10425. 2020-11-02T12:07:05.8249023Z [cluster3] I1102 11:58:52.403232 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.7 00:00:00:00:00:00
  10426. 2020-11-02T12:07:05.8249858Z [cluster3] I1102 11:58:52.403314 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  10427. 2020-11-02T12:07:05.8251234Z [cluster3] I1102 11:58:52.403391 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10428. 2020-11-02T12:07:05.8253066Z [cluster3] I1102 11:59:22.402807 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10429. 2020-11-02T12:07:05.8254184Z [cluster3] I1102 11:59:22.403473 1 route.go:756] Processing route {Ifindex: 13 Dst: 240.0.0.0/8 Src: 240.17.0.7 Gw: <nil> Flags: [] Table: 254}
  10430. 2020-11-02T12:07:05.8254680Z [cluster3] I1102 11:59:22.403550 1 route.go:759] Found nil gw or dst
  10431. 2020-11-02T12:07:05.8255157Z [cluster3] I1102 11:59:22.403559 1 route.go:582] This route agent is running on the active gateway node
  10432. 2020-11-02T12:07:05.8255982Z [cluster3] I1102 11:59:22.403566 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  10433. 2020-11-02T12:07:05.8256764Z [cluster3] I1102 11:59:22.403775 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10434. 2020-11-02T12:07:05.8257371Z [cluster3] I1102 11:59:22.403824 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.5 00:00:00:00:00:00
  10435. 2020-11-02T12:07:05.8257932Z [cluster3] I1102 11:59:22.403868 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.10 00:00:00:00:00:00
  10436. 2020-11-02T12:07:05.8258486Z [cluster3] I1102 11:59:22.403904 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.7 00:00:00:00:00:00
  10437. 2020-11-02T12:07:05.8259354Z [cluster3] I1102 11:59:22.403985 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  10438. 2020-11-02T12:07:05.8260871Z [cluster3] I1102 11:59:22.404075 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10439. 2020-11-02T12:07:05.8262612Z [cluster3] I1102 11:59:22.404109 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10440. 2020-11-02T12:07:05.8264359Z [cluster3] I1102 11:59:52.402942 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10441. 2020-11-02T12:07:05.8266204Z [cluster3] I1102 11:59:52.403006 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10442. 2020-11-02T12:07:05.8267296Z [cluster3] I1102 11:59:52.403320 1 route.go:756] Processing route {Ifindex: 13 Dst: 240.0.0.0/8 Src: 240.17.0.7 Gw: <nil> Flags: [] Table: 254}
  10443. 2020-11-02T12:07:05.8267777Z [cluster3] I1102 11:59:52.403341 1 route.go:759] Found nil gw or dst
  10444. 2020-11-02T12:07:05.8268244Z [cluster3] I1102 11:59:52.403348 1 route.go:582] This route agent is running on the active gateway node
  10445. 2020-11-02T12:07:05.8269052Z [cluster3] I1102 11:59:52.403355 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  10446. 2020-11-02T12:07:05.8269692Z [cluster3] I1102 11:59:52.403483 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10447. 2020-11-02T12:07:05.8270284Z [cluster3] I1102 11:59:52.403525 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.7 00:00:00:00:00:00
  10448. 2020-11-02T12:07:05.8270823Z [cluster3] I1102 11:59:52.403552 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.5 00:00:00:00:00:00
  10449. 2020-11-02T12:07:05.8271362Z [cluster3] I1102 11:59:52.403573 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.10 00:00:00:00:00:00
  10450. 2020-11-02T12:07:05.8272170Z [cluster3] I1102 11:59:52.403632 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  10451. 2020-11-02T12:07:05.8273509Z [cluster3] I1102 11:59:52.403737 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10452. 2020-11-02T12:07:05.8275254Z [cluster3] I1102 12:00:22.403140 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10453. 2020-11-02T12:07:05.8277024Z [cluster3] I1102 12:00:22.403179 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10454. 2020-11-02T12:07:05.8278100Z [cluster3] I1102 12:00:22.403369 1 route.go:756] Processing route {Ifindex: 13 Dst: 240.0.0.0/8 Src: 240.17.0.7 Gw: <nil> Flags: [] Table: 254}
  10455. 2020-11-02T12:07:05.8278581Z [cluster3] I1102 12:00:22.403414 1 route.go:759] Found nil gw or dst
  10456. 2020-11-02T12:07:05.8279043Z [cluster3] I1102 12:00:22.403420 1 route.go:582] This route agent is running on the active gateway node
  10457. 2020-11-02T12:07:05.8279982Z [cluster3] I1102 12:00:22.403427 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  10458. 2020-11-02T12:07:05.8280694Z [cluster3] I1102 12:00:22.403552 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10459. 2020-11-02T12:07:05.8281261Z [cluster3] I1102 12:00:22.403591 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.5 00:00:00:00:00:00
  10460. 2020-11-02T12:07:05.8281772Z [cluster3] I1102 12:00:22.403619 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.10 00:00:00:00:00:00
  10461. 2020-11-02T12:07:05.8282294Z [cluster3] I1102 12:00:22.403643 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.7 00:00:00:00:00:00
  10462. 2020-11-02T12:07:05.8283099Z [cluster3] I1102 12:00:22.403698 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  10463. 2020-11-02T12:07:05.8288425Z [cluster3] I1102 12:00:22.403771 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10464. 2020-11-02T12:07:05.8290346Z [cluster3] I1102 12:00:52.403452 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10465. 2020-11-02T12:07:05.8291553Z [cluster3] I1102 12:00:52.403979 1 route.go:756] Processing route {Ifindex: 13 Dst: 240.0.0.0/8 Src: 240.17.0.7 Gw: <nil> Flags: [] Table: 254}
  10466. 2020-11-02T12:07:05.8292044Z [cluster3] I1102 12:00:52.404020 1 route.go:759] Found nil gw or dst
  10467. 2020-11-02T12:07:05.8292890Z [cluster3] I1102 12:00:52.404044 1 route.go:582] This route agent is running on the active gateway node
  10468. 2020-11-02T12:07:05.8293756Z [cluster3] I1102 12:00:52.404053 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  10469. 2020-11-02T12:07:05.8294421Z [cluster3] I1102 12:00:52.404373 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10470. 2020-11-02T12:07:05.8295323Z [cluster3] I1102 12:00:52.404482 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.5 00:00:00:00:00:00
  10471. 2020-11-02T12:07:05.8295855Z [cluster3] I1102 12:00:52.404515 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.10 00:00:00:00:00:00
  10472. 2020-11-02T12:07:05.8296382Z [cluster3] I1102 12:00:52.404540 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.7 00:00:00:00:00:00
  10473. 2020-11-02T12:07:05.8297171Z [cluster3] I1102 12:00:52.404620 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  10474. 2020-11-02T12:07:05.8298473Z [cluster3] I1102 12:00:52.404775 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10475. 2020-11-02T12:07:05.8300302Z [cluster3] I1102 12:00:52.404807 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10476. 2020-11-02T12:07:05.8301951Z [cluster3] I1102 12:01:22.403552 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10477. 2020-11-02T12:07:05.8303875Z [cluster3] I1102 12:01:22.403591 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10478. 2020-11-02T12:07:05.8304923Z [cluster3] I1102 12:01:22.403765 1 route.go:756] Processing route {Ifindex: 13 Dst: 240.0.0.0/8 Src: 240.17.0.7 Gw: <nil> Flags: [] Table: 254}
  10479. 2020-11-02T12:07:05.8305377Z [cluster3] I1102 12:01:22.403794 1 route.go:759] Found nil gw or dst
  10480. 2020-11-02T12:07:05.8305970Z [cluster3] I1102 12:01:22.403801 1 route.go:582] This route agent is running on the active gateway node
  10481. 2020-11-02T12:07:05.8306896Z [cluster3] I1102 12:01:22.403807 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  10482. 2020-11-02T12:07:05.8307694Z [cluster3] I1102 12:01:22.403939 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10483. 2020-11-02T12:07:05.8308393Z [cluster3] I1102 12:01:22.404004 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.7 00:00:00:00:00:00
  10484. 2020-11-02T12:07:05.8308931Z [cluster3] I1102 12:01:22.404042 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.5 00:00:00:00:00:00
  10485. 2020-11-02T12:07:05.8309474Z [cluster3] I1102 12:01:22.404074 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.10 00:00:00:00:00:00
  10486. 2020-11-02T12:07:05.8310455Z [cluster3] I1102 12:01:22.404144 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  10487. 2020-11-02T12:07:05.8312416Z [cluster3] I1102 12:01:22.404263 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10488. 2020-11-02T12:07:05.8317039Z [cluster3] I1102 12:01:40.301214 1 route.go:682] Informed of removed endpoint: {"metadata":{"name":"cluster2-submariner-cable-cluster2-172-17-0-4","namespace":"submariner-operator","selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4","uid":"8c63de0a-c225-4c4f-b602-4d0ec35f7bae","resourceVersion":"2815","generation":1,"creationTimestamp":"2020-11-02T11:58:24Z","labels":{"submariner-io/clusterID":"cluster2"}},"spec":{"cluster_id":"cluster2","cable_name":"submariner-cable-cluster2-172-17-0-4","hostname":"cluster2-worker","subnets":["100.2.0.0/16","10.2.0.0/16"],"private_ip":"172.17.0.4","public_ip":"","nat_enabled":false,"backend":"libreswan"}}
  10489. 2020-11-02T12:07:05.8320665Z [cluster3] I1102 12:01:40.381663 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10490. 2020-11-02T12:07:05.8322465Z [cluster3] I1102 12:01:52.403804 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10491. 2020-11-02T12:07:05.8323754Z [cluster3] I1102 12:01:52.404058 1 route.go:756] Processing route {Ifindex: 13 Dst: 240.0.0.0/8 Src: 240.17.0.7 Gw: <nil> Flags: [] Table: 254}
  10492. 2020-11-02T12:07:05.8324344Z [cluster3] I1102 12:01:52.404090 1 route.go:759] Found nil gw or dst
  10493. 2020-11-02T12:07:05.8324798Z [cluster3] I1102 12:01:52.404163 1 route.go:582] This route agent is running on the active gateway node
  10494. 2020-11-02T12:07:05.8325602Z [cluster3] I1102 12:01:52.404176 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  10495. 2020-11-02T12:07:05.8326509Z [cluster3] I1102 12:01:52.404337 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10496. 2020-11-02T12:07:05.8327464Z [cluster3] I1102 12:01:52.404444 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.5 00:00:00:00:00:00
  10497. 2020-11-02T12:07:05.8327990Z [cluster3] I1102 12:01:52.404497 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.10 00:00:00:00:00:00
  10498. 2020-11-02T12:07:05.8328516Z [cluster3] I1102 12:01:52.404532 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.7 00:00:00:00:00:00
  10499. 2020-11-02T12:07:05.8329311Z [cluster3] I1102 12:01:52.404580 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  10500. 2020-11-02T12:07:05.8330614Z [cluster3] I1102 12:01:52.404672 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10501. 2020-11-02T12:07:05.8332949Z [cluster3] I1102 12:01:52.404695 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10502. 2020-11-02T12:07:05.8334686Z [cluster3] I1102 12:02:22.403912 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10503. 2020-11-02T12:07:05.8336847Z [cluster3] I1102 12:02:22.403958 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10504. 2020-11-02T12:07:05.8338519Z [cluster3] I1102 12:02:22.403968 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10505. 2020-11-02T12:07:05.8339533Z [cluster3] I1102 12:02:22.404181 1 route.go:756] Processing route {Ifindex: 13 Dst: 240.0.0.0/8 Src: 240.17.0.7 Gw: <nil> Flags: [] Table: 254}
  10506. 2020-11-02T12:07:05.8340139Z [cluster3] I1102 12:02:22.404213 1 route.go:759] Found nil gw or dst
  10507. 2020-11-02T12:07:05.8340565Z [cluster3] I1102 12:02:22.404220 1 route.go:582] This route agent is running on the active gateway node
  10508. 2020-11-02T12:07:05.8341276Z [cluster3] I1102 12:02:22.404227 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  10509. 2020-11-02T12:07:05.8341867Z [cluster3] I1102 12:02:22.404476 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10510. 2020-11-02T12:07:05.8342757Z [cluster3] I1102 12:02:22.404526 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.5 00:00:00:00:00:00
  10511. 2020-11-02T12:07:05.8343362Z [cluster3] I1102 12:02:22.404567 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.10 00:00:00:00:00:00
  10512. 2020-11-02T12:07:05.8343893Z [cluster3] I1102 12:02:22.404600 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.7 00:00:00:00:00:00
  10513. 2020-11-02T12:07:05.8344705Z [cluster3] I1102 12:02:22.404673 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  10514. 2020-11-02T12:07:05.8346133Z [cluster3] I1102 12:02:52.404089 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10515. 2020-11-02T12:07:05.8347948Z [cluster3] I1102 12:02:52.404151 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10516. 2020-11-02T12:07:05.8348964Z [cluster3] I1102 12:02:52.404362 1 route.go:756] Processing route {Ifindex: 13 Dst: 240.0.0.0/8 Src: 240.17.0.7 Gw: <nil> Flags: [] Table: 254}
  10517. 2020-11-02T12:07:05.8349418Z [cluster3] I1102 12:02:52.404459 1 route.go:759] Found nil gw or dst
  10518. 2020-11-02T12:07:05.8350002Z [cluster3] I1102 12:02:52.404473 1 route.go:582] This route agent is running on the active gateway node
  10519. 2020-11-02T12:07:05.8350732Z [cluster3] I1102 12:02:52.404480 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  10520. 2020-11-02T12:07:05.8351314Z [cluster3] I1102 12:02:52.404640 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10521. 2020-11-02T12:07:05.8351847Z [cluster3] I1102 12:02:52.404681 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.5 00:00:00:00:00:00
  10522. 2020-11-02T12:07:05.8352712Z [cluster3] I1102 12:02:52.404721 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.10 00:00:00:00:00:00
  10523. 2020-11-02T12:07:05.8353217Z [cluster3] I1102 12:02:52.404754 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.7 00:00:00:00:00:00
  10524. 2020-11-02T12:07:05.8354009Z [cluster3] I1102 12:02:52.404807 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  10525. 2020-11-02T12:07:05.8355311Z [cluster3] I1102 12:02:52.404874 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10526. 2020-11-02T12:07:05.8360720Z [cluster3] I1102 12:03:22.404231 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10527. 2020-11-02T12:07:05.8363018Z [cluster3] I1102 12:03:22.404276 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10528. 2020-11-02T12:07:05.8365094Z [cluster3] I1102 12:03:22.404286 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10529. 2020-11-02T12:07:05.8366310Z [cluster3] I1102 12:03:22.404532 1 route.go:756] Processing route {Ifindex: 13 Dst: 240.0.0.0/8 Src: 240.17.0.7 Gw: <nil> Flags: [] Table: 254}
  10530. 2020-11-02T12:07:05.8366931Z [cluster3] I1102 12:03:22.404582 1 route.go:759] Found nil gw or dst
  10531. 2020-11-02T12:07:05.8367367Z [cluster3] I1102 12:03:22.404589 1 route.go:582] This route agent is running on the active gateway node
  10532. 2020-11-02T12:07:05.8368119Z [cluster3] I1102 12:03:22.404596 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  10533. 2020-11-02T12:07:05.8368726Z [cluster3] I1102 12:03:22.404726 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10534. 2020-11-02T12:07:05.8369285Z [cluster3] I1102 12:03:22.404773 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.5 00:00:00:00:00:00
  10535. 2020-11-02T12:07:05.8370772Z [cluster3] I1102 12:03:22.404813 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.10 00:00:00:00:00:00
  10536. 2020-11-02T12:07:05.8371421Z [cluster3] I1102 12:03:22.404848 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.7 00:00:00:00:00:00
  10537. 2020-11-02T12:07:05.8372455Z [cluster3] I1102 12:03:22.404935 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  10538. 2020-11-02T12:07:05.8373956Z [cluster3] I1102 12:03:52.404281 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10539. 2020-11-02T12:07:05.8375829Z [cluster3] I1102 12:03:52.404329 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10540. 2020-11-02T12:07:05.8377006Z [cluster3] I1102 12:03:52.404701 1 route.go:756] Processing route {Ifindex: 13 Dst: 240.0.0.0/8 Src: 240.17.0.7 Gw: <nil> Flags: [] Table: 254}
  10541. 2020-11-02T12:07:05.8377447Z [cluster3] I1102 12:03:52.404734 1 route.go:759] Found nil gw or dst
  10542. 2020-11-02T12:07:05.8377887Z [cluster3] I1102 12:03:52.404742 1 route.go:582] This route agent is running on the active gateway node
  10543. 2020-11-02T12:07:05.8378789Z [cluster3] I1102 12:03:52.404749 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  10544. 2020-11-02T12:07:05.8379406Z [cluster3] I1102 12:03:52.404905 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10545. 2020-11-02T12:07:05.8380106Z [cluster3] I1102 12:03:52.404950 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.5 00:00:00:00:00:00
  10546. 2020-11-02T12:07:05.8380600Z [cluster3] I1102 12:03:52.404996 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.10 00:00:00:00:00:00
  10547. 2020-11-02T12:07:05.8381098Z [cluster3] I1102 12:03:52.405035 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.7 00:00:00:00:00:00
  10548. 2020-11-02T12:07:05.8381868Z [cluster3] I1102 12:03:52.405093 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  10549. 2020-11-02T12:07:05.8383098Z [cluster3] I1102 12:03:52.405164 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10550. 2020-11-02T12:07:05.8384690Z [cluster3] I1102 12:04:22.404740 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10551. 2020-11-02T12:07:05.8386309Z [cluster3] I1102 12:04:22.404779 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10552. 2020-11-02T12:07:05.8387607Z [cluster3] I1102 12:04:22.404968 1 route.go:756] Processing route {Ifindex: 13 Dst: 240.0.0.0/8 Src: 240.17.0.7 Gw: <nil> Flags: [] Table: 254}
  10553. 2020-11-02T12:07:05.8388063Z [cluster3] I1102 12:04:22.404994 1 route.go:759] Found nil gw or dst
  10554. 2020-11-02T12:07:05.8388498Z [cluster3] I1102 12:04:22.405001 1 route.go:582] This route agent is running on the active gateway node
  10555. 2020-11-02T12:07:05.8389279Z [cluster3] I1102 12:04:22.405007 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  10556. 2020-11-02T12:07:05.8390019Z [cluster3] I1102 12:04:22.405158 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10557. 2020-11-02T12:07:05.8390553Z [cluster3] I1102 12:04:22.405205 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.5 00:00:00:00:00:00
  10558. 2020-11-02T12:07:05.8391041Z [cluster3] I1102 12:04:22.405265 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.10 00:00:00:00:00:00
  10559. 2020-11-02T12:07:05.8391538Z [cluster3] I1102 12:04:22.405290 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.7 00:00:00:00:00:00
  10560. 2020-11-02T12:07:05.8393414Z [cluster3] I1102 12:04:22.405350 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  10561. 2020-11-02T12:07:05.8395189Z [cluster3] I1102 12:04:22.405451 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10562. 2020-11-02T12:07:05.8397994Z [cluster3] I1102 12:04:52.404893 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10563. 2020-11-02T12:07:05.8399657Z [cluster3] I1102 12:04:52.404938 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10564. 2020-11-02T12:07:05.8400796Z [cluster3] I1102 12:04:52.405133 1 route.go:756] Processing route {Ifindex: 13 Dst: 240.0.0.0/8 Src: 240.17.0.7 Gw: <nil> Flags: [] Table: 254}
  10565. 2020-11-02T12:07:05.8401241Z [cluster3] I1102 12:04:52.405163 1 route.go:759] Found nil gw or dst
  10566. 2020-11-02T12:07:05.8401663Z [cluster3] I1102 12:04:52.405170 1 route.go:582] This route agent is running on the active gateway node
  10567. 2020-11-02T12:07:05.8402402Z [cluster3] I1102 12:04:52.405176 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  10568. 2020-11-02T12:07:05.8402988Z [cluster3] I1102 12:04:52.405311 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10569. 2020-11-02T12:07:05.8404028Z [cluster3] I1102 12:04:52.405355 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.5 00:00:00:00:00:00
  10570. 2020-11-02T12:07:05.8404563Z [cluster3] I1102 12:04:52.405392 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.10 00:00:00:00:00:00
  10571. 2020-11-02T12:07:05.8405086Z [cluster3] I1102 12:04:52.405423 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.7 00:00:00:00:00:00
  10572. 2020-11-02T12:07:05.8405897Z [cluster3] I1102 12:04:52.405478 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  10573. 2020-11-02T12:07:05.8407345Z [cluster3] I1102 12:04:52.405537 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10574. 2020-11-02T12:07:05.8408990Z [cluster3] I1102 12:05:22.404954 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-4" with subnets [100.2.0.0/16 10.2.0.0/16]
  10575. 2020-11-02T12:07:05.8410785Z [cluster3] I1102 12:05:22.405010 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10576. 2020-11-02T12:07:05.8411773Z [cluster3] I1102 12:05:22.405285 1 route.go:756] Processing route {Ifindex: 13 Dst: 240.0.0.0/8 Src: 240.17.0.7 Gw: <nil> Flags: [] Table: 254}
  10577. 2020-11-02T12:07:05.8412532Z [cluster3] I1102 12:05:22.405337 1 route.go:759] Found nil gw or dst
  10578. 2020-11-02T12:07:05.8412966Z [cluster3] I1102 12:05:22.405346 1 route.go:582] This route agent is running on the active gateway node
  10579. 2020-11-02T12:07:05.8414271Z [cluster3] I1102 12:05:22.405353 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  10580. 2020-11-02T12:07:05.8414917Z [cluster3] I1102 12:05:22.405488 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10581. 2020-11-02T12:07:05.8415504Z [cluster3] I1102 12:05:22.405555 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.7 00:00:00:00:00:00
  10582. 2020-11-02T12:07:05.8416193Z [cluster3] I1102 12:05:22.405591 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.5 00:00:00:00:00:00
  10583. 2020-11-02T12:07:05.8416883Z [cluster3] I1102 12:05:22.405614 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.10 00:00:00:00:00:00
  10584. 2020-11-02T12:07:05.8418026Z [cluster3] I1102 12:05:22.405679 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  10585. 2020-11-02T12:07:05.8419320Z [cluster3] I1102 12:05:22.405734 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10586. 2020-11-02T12:07:05.8424346Z [cluster3] I1102 12:05:30.615957 1 route.go:682] Informed of removed endpoint: {"metadata":{"name":"cluster2-submariner-cable-cluster2-172-17-0-4","namespace":"submariner-operator","selfLink":"/apis/submariner.io/v1/namespaces/submariner-operator/endpoints/cluster2-submariner-cable-cluster2-172-17-0-4","uid":"94d2ede3-9111-44d5-a192-56c2299fcfe7","resourceVersion":"3680","generation":1,"creationTimestamp":"2020-11-02T12:01:40Z","labels":{"submariner-io/clusterID":"cluster2"}},"spec":{"cluster_id":"cluster2","cable_name":"submariner-cable-cluster2-172-17-0-4","hostname":"cluster2-worker","subnets":["100.2.0.0/16","10.2.0.0/16"],"private_ip":"172.17.0.4","public_ip":"","nat_enabled":false,"backend":"libreswan"}}
  10587. 2020-11-02T12:07:05.8429068Z [cluster3] I1102 12:05:30.683887 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11" with subnets [100.2.0.0/16 10.2.0.0/16]
  10588. 2020-11-02T12:07:05.8430864Z [cluster3] I1102 12:05:52.405290 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11" with subnets [100.2.0.0/16 10.2.0.0/16]
  10589. 2020-11-02T12:07:05.8432497Z [cluster3] I1102 12:05:52.405333 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10590. 2020-11-02T12:07:05.8433897Z [cluster3] I1102 12:05:52.405659 1 route.go:756] Processing route {Ifindex: 13 Dst: 240.0.0.0/8 Src: 240.17.0.7 Gw: <nil> Flags: [] Table: 254}
  10591. 2020-11-02T12:07:05.8434368Z [cluster3] I1102 12:05:52.405697 1 route.go:759] Found nil gw or dst
  10592. 2020-11-02T12:07:05.8434824Z [cluster3] I1102 12:05:52.405705 1 route.go:582] This route agent is running on the active gateway node
  10593. 2020-11-02T12:07:05.8435596Z [cluster3] I1102 12:05:52.405748 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  10594. 2020-11-02T12:07:05.8436222Z [cluster3] I1102 12:05:52.405973 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10595. 2020-11-02T12:07:05.8437121Z [cluster3] I1102 12:05:52.406060 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.5 00:00:00:00:00:00
  10596. 2020-11-02T12:07:05.8437633Z [cluster3] I1102 12:05:52.406126 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.10 00:00:00:00:00:00
  10597. 2020-11-02T12:07:05.8438141Z [cluster3] I1102 12:05:52.406182 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.7 00:00:00:00:00:00
  10598. 2020-11-02T12:07:05.8438904Z [cluster3] I1102 12:05:52.406293 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  10599. 2020-11-02T12:07:05.8440439Z [cluster3] I1102 12:05:52.406395 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10600. 2020-11-02T12:07:05.8442033Z [cluster3] I1102 12:06:22.405303 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10601. 2020-11-02T12:07:05.8448248Z [cluster3] I1102 12:06:22.405403 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11" with subnets [100.2.0.0/16 10.2.0.0/16]
  10602. 2020-11-02T12:07:05.8450658Z [cluster3] I1102 12:06:22.405416 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10603. 2020-11-02T12:07:05.8452038Z [cluster3] I1102 12:06:22.405656 1 route.go:756] Processing route {Ifindex: 13 Dst: 240.0.0.0/8 Src: 240.17.0.7 Gw: <nil> Flags: [] Table: 254}
  10604. 2020-11-02T12:07:05.8452679Z [cluster3] I1102 12:06:22.405693 1 route.go:759] Found nil gw or dst
  10605. 2020-11-02T12:07:05.8454326Z [cluster3] I1102 12:06:22.405700 1 route.go:582] This route agent is running on the active gateway node
  10606. 2020-11-02T12:07:05.8455366Z [cluster3] I1102 12:06:22.405706 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  10607. 2020-11-02T12:07:05.8456175Z [cluster3] I1102 12:06:22.405830 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10608. 2020-11-02T12:07:05.8456933Z [cluster3] I1102 12:06:22.405864 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.5 00:00:00:00:00:00
  10609. 2020-11-02T12:07:05.8457951Z [cluster3] I1102 12:06:22.405892 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.10 00:00:00:00:00:00
  10610. 2020-11-02T12:07:05.8458476Z [cluster3] I1102 12:06:22.405918 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.7 00:00:00:00:00:00
  10611. 2020-11-02T12:07:05.8459310Z [cluster3] I1102 12:06:22.406046 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  10612. 2020-11-02T12:07:05.8460627Z [cluster3] I1102 12:06:52.405591 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster2-submariner-cable-cluster2-172-17-0-11" with subnets [100.2.0.0/16 10.2.0.0/16]
  10613. 2020-11-02T12:07:05.8462580Z [cluster3] I1102 12:06:52.405640 1 route.go:570] Processing local Endpoint cluster3-submariner-cable-cluster3-172-17-0-7 with IP 172.17.0.7, Host cluster3-worker for this cluster
  10614. 2020-11-02T12:07:05.8463976Z [cluster3] I1102 12:06:52.405807 1 route.go:756] Processing route {Ifindex: 13 Dst: 240.0.0.0/8 Src: 240.17.0.7 Gw: <nil> Flags: [] Table: 254}
  10615. 2020-11-02T12:07:05.8464444Z [cluster3] I1102 12:06:52.405837 1 route.go:759] Found nil gw or dst
  10616. 2020-11-02T12:07:05.8464893Z [cluster3] I1102 12:06:52.405844 1 route.go:582] This route agent is running on the active gateway node
  10617. 2020-11-02T12:07:05.8465673Z [cluster3] I1102 12:06:52.405851 1 route.go:590] Creating the vxlan interface: vx-submariner on the gateway node
  10618. 2020-11-02T12:07:05.8466279Z [cluster3] I1102 12:06:52.406000 1 vxlan.go:61] VxLAN interface already exists with same configuration.
  10619. 2020-11-02T12:07:05.8467069Z [cluster3] I1102 12:06:52.406042 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.7 00:00:00:00:00:00
  10620. 2020-11-02T12:07:05.8467578Z [cluster3] I1102 12:06:52.406073 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.5 00:00:00:00:00:00
  10621. 2020-11-02T12:07:05.8468085Z [cluster3] I1102 12:06:52.406105 1 vxlan.go:159] Successfully added the bridge fdb entry 172.17.0.10 00:00:00:00:00:00
  10622. 2020-11-02T12:07:05.8468851Z [cluster3] I1102 12:06:52.406155 1 route.go:464] Successfully configured rp_filter to loose mode(2) on vx-submariner
  10623. 2020-11-02T12:07:05.8470260Z [cluster3] I1102 12:06:52.406215 1 route.go:558] Updating iptable rules for remote cluster's Endpoint "cluster1-submariner-cable-cluster1-172-17-0-6" with subnets [100.1.0.0/16 10.1.0.0/16]
  10624. 2020-11-02T12:07:06.0548494Z [cluster3] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  10625. 2020-11-02T12:07:06.0549197Z [cluster3]
  10626. 2020-11-02T12:07:06.0549580Z [cluster3] Showing information for cluster "cluster2":
  10627. 2020-11-02T12:07:06.0550073Z [cluster3] Discovered network details:
  10628. 2020-11-02T12:07:06.0551087Z [cluster3] Network plugin: weave-net
  10629. 2020-11-02T12:07:06.0551463Z [cluster3] Service CIDRs: [100.2.0.0/16]
  10630. 2020-11-02T12:07:06.0551953Z [cluster3] Cluster CIDRs: [10.2.0.0/16]
  10631. 2020-11-02T12:07:06.0552213Z [cluster3]
  10632. 2020-11-02T12:07:06.0552831Z [cluster3] CLUSTER ID ENDPOINT IP PUBLIC IP CABLE DRIVER TYPE
  10633. 2020-11-02T12:07:06.0553325Z [cluster3] cluster2 172.17.0.11 libreswan local
  10634. 2020-11-02T12:07:06.0553784Z [cluster3] cluster3 172.17.0.7 libreswan remote
  10635. 2020-11-02T12:07:06.0554411Z [cluster3] cluster1 172.17.0.6 libreswan remote
  10636. 2020-11-02T12:07:06.0554765Z [cluster3]
  10637. 2020-11-02T12:07:06.0555690Z [cluster3] GATEWAY CLUSTER REMOTE IP CABLE DRIVER SUBNETS STATUS
  10638. 2020-11-02T12:07:06.0556633Z [cluster3] cluster3-worker cluster3 172.17.0.7 libreswan 100.3.0.0/16, 10.3.0.0/16 connected
  10639. 2020-11-02T12:07:06.0557845Z [cluster3] cluster1-worker cluster1 172.17.0.6 libreswan 100.1.0.0/16, 10.1.0.0/16 connected
  10640. 2020-11-02T12:07:06.0558270Z [cluster3]
  10641. 2020-11-02T12:07:06.0558576Z [cluster3] NODE HA STATUS SUMMARY
  10642. 2020-11-02T12:07:06.0559615Z [cluster3] cluster2-worker2 active All connections (2) are established
  10643. 2020-11-02T12:07:06.0560197Z [cluster3]
  10644. 2020-11-02T12:07:06.0560536Z [cluster3] COMPONENT REPOSITORY VERSION
  10645. 2020-11-02T12:07:06.0560988Z [cluster3] submariner localhost:5000 local
  10646. 2020-11-02T12:07:06.0561852Z [cluster3] submariner-operator localhost:5000 local
  10647. 2020-11-02T12:07:06.0562915Z [cluster3] service-discovery localhost:5000 local
  10648. 2020-11-02T12:07:06.0563301Z [cluster3]
  10649. 2020-11-02T12:07:06.0563658Z [cluster3] Showing information for cluster "cluster3":
  10650. 2020-11-02T12:07:06.0564736Z [cluster3] Discovered network details:
  10651. 2020-11-02T12:07:06.0565325Z [cluster3] Network plugin: weave-net
  10652. 2020-11-02T12:07:06.0565833Z [cluster3] Service CIDRs: [100.3.0.0/16]
  10653. 2020-11-02T12:07:06.0566154Z [cluster3] Cluster CIDRs: [10.3.0.0/16]
  10654. 2020-11-02T12:07:06.0566427Z [cluster3]
  10655. 2020-11-02T12:07:06.0566802Z [cluster3] CLUSTER ID ENDPOINT IP PUBLIC IP CABLE DRIVER TYPE
  10656. 2020-11-02T12:07:06.0567282Z [cluster3] cluster3 172.17.0.7 libreswan local
  10657. 2020-11-02T12:07:06.0567710Z [cluster3] cluster1 172.17.0.6 libreswan remote
  10658. 2020-11-02T12:07:06.0568140Z [cluster3] cluster2 172.17.0.11 libreswan remote
  10659. 2020-11-02T12:07:06.0568472Z [cluster3]
  10660. 2020-11-02T12:07:06.0568874Z [cluster3] GATEWAY CLUSTER REMOTE IP CABLE DRIVER SUBNETS STATUS
  10661. 2020-11-02T12:07:06.0569933Z [cluster3] cluster1-worker cluster1 172.17.0.6 libreswan 100.1.0.0/16, 10.1.0.0/16 connected
  10662. 2020-11-02T12:07:06.0570758Z [cluster3] cluster2-worker2 cluster2 172.17.0.11 libreswan 100.2.0.0/16, 10.2.0.0/16 connected
  10663. 2020-11-02T12:07:06.0571311Z [cluster3]
  10664. 2020-11-02T12:07:06.0571786Z [cluster3] NODE HA STATUS SUMMARY
  10665. 2020-11-02T12:07:06.0572650Z [cluster3] cluster3-worker active All connections (2) are established
  10666. 2020-11-02T12:07:06.0573076Z [cluster3]
  10667. 2020-11-02T12:07:06.0573427Z [cluster3] COMPONENT REPOSITORY VERSION
  10668. 2020-11-02T12:07:06.0573886Z [cluster3] submariner localhost:5000 local
  10669. 2020-11-02T12:07:06.0574771Z [cluster3] submariner-operator localhost:5000 local
  10670. 2020-11-02T12:07:06.0575542Z [cluster3] service-discovery localhost:5000 local
  10671. 2020-11-02T12:07:06.0576106Z [cluster3]
  10672. 2020-11-02T12:07:06.0576445Z [cluster3] Showing information for cluster "cluster1":
  10673. 2020-11-02T12:07:06.0576988Z [cluster3] Discovered network details:
  10674. 2020-11-02T12:07:06.0577384Z [cluster3] Network plugin: generic
  10675. 2020-11-02T12:07:06.0577876Z [cluster3] Service CIDRs: [100.1.0.0/16]
  10676. 2020-11-02T12:07:06.0578187Z [cluster3] Cluster CIDRs: [10.1.0.0/16]
  10677. 2020-11-02T12:07:06.0578453Z [cluster3]
  10678. 2020-11-02T12:07:06.0725113Z [cluster3] CLUSTER ID ENDPOINT IP PUBLIC IP CABLE DRIVER TYPE
  10679. 2020-11-02T12:07:06.0725672Z [cluster3] cluster1 172.17.0.6 libreswan local
  10680. 2020-11-02T12:07:06.0726082Z [cluster3] cluster3 172.17.0.7 libreswan remote
  10681. 2020-11-02T12:07:06.0726513Z [cluster3] cluster2 172.17.0.11 libreswan remote
  10682. 2020-11-02T12:07:06.0726810Z [cluster3]
  10683. 2020-11-02T12:07:06.0727204Z [cluster3] GATEWAY CLUSTER REMOTE IP CABLE DRIVER SUBNETS STATUS
  10684. 2020-11-02T12:07:06.0728378Z [cluster3] cluster3-worker cluster3 172.17.0.7 libreswan 100.3.0.0/16, 10.3.0.0/16 connected
  10685. 2020-11-02T12:07:06.0729580Z [cluster3] cluster2-worker2 cluster2 172.17.0.11 libreswan 100.2.0.0/16, 10.2.0.0/16 connected
  10686. 2020-11-02T12:07:06.0729980Z [cluster3]
  10687. 2020-11-02T12:07:06.0730270Z [cluster3] NODE HA STATUS SUMMARY
  10688. 2020-11-02T12:07:06.0730921Z [cluster3] cluster1-worker active All connections (2) are established
  10689. 2020-11-02T12:07:06.0731324Z [cluster3]
  10690. 2020-11-02T12:07:06.0731657Z [cluster3] COMPONENT REPOSITORY VERSION
  10691. 2020-11-02T12:07:06.0732095Z [cluster3] submariner localhost:5000 local
  10692. 2020-11-02T12:07:06.0732907Z [cluster3] submariner-operator localhost:5000 local
  10693. 2020-11-02T12:07:06.0733580Z [cluster3] service-discovery localhost:5000 local
  10694. 2020-11-02T12:07:06.0733988Z [cluster3] ===================== END Post mortem cluster3 =====================
  10695. 2020-11-02T12:07:06.0753630Z [submariner]$ make post-mortem
  10696. 2020-11-02T12:07:06.4585238Z Post job cleanup.
  10697. 2020-11-02T12:07:06.9605944Z [command]/usr/bin/git version
  10698. 2020-11-02T12:07:06.9738414Z git version 2.29.0
  10699. 2020-11-02T12:07:06.9821997Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
  10700. 2020-11-02T12:07:06.9901555Z [command]/usr/bin/git submodule foreach --recursive git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :
  10701. 2020-11-02T12:07:07.0435002Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
  10702. 2020-11-02T12:07:07.0482029Z http.https://github.com/.extraheader
  10703. 2020-11-02T12:07:07.0502751Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader
  10704. 2020-11-02T12:07:07.0548217Z [command]/usr/bin/git submodule foreach --recursive git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :
  10705. 2020-11-02T12:07:07.0992863Z Cleaning up orphan processes
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!