API tools faq
paste
Advertisement
Guest User

VCLD log for VCL Reinstall

a guest
May 29th, 2013
47
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 363.58 KB | None | 0 0
raw download clone embed print report
  1. 2013-05-28 13:55:21|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:55:21
  2. 2013-05-28 13:55:26|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:55:26
  3. 2013-05-28 13:55:31|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:55:31
  4. 2013-05-28 13:55:31|31074|3243:3604|reinstall|utils.pm:reservation_being_processed(8638)|computerloadlog 'begin' entry does NOT exist for reservation 3604
  5. 2013-05-28 13:55:31|31074|3243:3604|reinstall|utils.pm:run_command(8710)|executed command: pgrep -fl 'vcld [0-9]+:3604 ', pid: 12863, exit status: 1, output:
  6. 2013-05-28 13:55:31|31074|3243:3604|reinstall|utils.pm:is_management_node_process_running(8943)|process is NOT running, identifier: 'vcld [0-9]+:3604 '
  7. 2013-05-28 13:55:31|31074|3243:3604|reinstall|utils.pm:reservation_being_processed(8659)|reservation is NOT currently being processed
  8. 2013-05-28 13:55:31|31074|3243:3604|reinstall|vcld:main(284)|reservation 3604 is NOT already being processed
  9. 2013-05-28 13:55:31|31074|3243:3604|reinstall|utils.pm:get_management_node_info(5456)|retrieving current management node info for 'ecco-adm3.sharcnet.ca' from database, cached data is stale: 91 seconds old
  10. 2013-05-28 13:55:31|31074|3243:3604|reinstall|utils.pm:get_management_node_info(5603)|retrieved management node info: 'ecco-adm3.sharcnet.ca' (ecco-adm3)
  11. 2013-05-28 13:55:31|31074|3243:3604|reinstall|utils.pm:get_connect_method_info(10063)|attempting to retrieve connect method info:
  12. |31074|3243:3604|reinstall| imagerevision: 174 - linux-TestImage2-16-v2
  13. |31074|3243:3604|reinstall| OS: 36 - vmwarelinux
  14. |31074|3243:3604|reinstall| OS type: 2 - linux
  15. 2013-05-28 13:55:31|31074|3243:3604|reinstall|utils.pm:get_connect_method_info(10127)|ssh: connectmethodid=1, OStypeid=2, OSid=NULL, imagerevisionid=NULL, disabled=0
  16. 2013-05-28 13:55:31|31074|3243:3604|reinstall|utils.pm:get_connect_method_info(10127)|Xvnc: connectmethodid=4, OStypeid=NULL, OSid=NULL, imagerevisionid=174, disabled=0
  17. 2013-05-28 13:55:31|31074|3243:3604|reinstall|utils.pm:get_user_info(7558)|non-standalone affiliation found for user nsasherr:
  18. |31074|3243:3604|reinstall| user affiliation: sharcnet
  19. |31074|3243:3604|reinstall| not standalone list: sharcnet
  20. 2013-05-28 13:55:31|31074|3243:3604|reinstall|utils.pm:get_user_info(7558)|non-standalone affiliation found for user nsasherr:
  21. |31074|3243:3604|reinstall| user affiliation: sharcnet
  22. |31074|3243:3604|reinstall| not standalone list: sharcnet
  23. 2013-05-28 13:55:31|31074|3243:3604|reinstall|utils.pm:get_connect_method_info(10063)|attempting to retrieve connect method info:
  24. |31074|3243:3604|reinstall| imagerevision: 174 - linux-TestImage2-16-v2
  25. |31074|3243:3604|reinstall| OS: 36 - vmwarelinux
  26. |31074|3243:3604|reinstall| OS type: 2 - linux
  27. 2013-05-28 13:55:31|31074|3243:3604|reinstall|utils.pm:get_connect_method_info(10127)|ssh: connectmethodid=1, OStypeid=2, OSid=NULL, imagerevisionid=NULL, disabled=0
  28. 2013-05-28 13:55:31|31074|3243:3604|reinstall|utils.pm:get_connect_method_info(10127)|Xvnc: connectmethodid=4, OStypeid=NULL, OSid=NULL, imagerevisionid=174, disabled=0
  29. 2013-05-28 13:55:31|31074|3243:3604|reinstall|utils.pm:get_user_info(7558)|non-standalone affiliation found for user nsasherr:
  30. |31074|3243:3604|reinstall| user affiliation: sharcnet
  31. |31074|3243:3604|reinstall| not standalone list: sharcnet
  32. 2013-05-28 13:55:31|31074|3243:3604|reinstall|utils.pm:get_user_info(7558)|non-standalone affiliation found for user nsasherr:
  33. |31074|3243:3604|reinstall| user affiliation: sharcnet
  34. |31074|3243:3604|reinstall| not standalone list: sharcnet
  35. 2013-05-28 13:55:31|31074|3243:3604|reinstall|vcld:main(289)|retrieved request information from database
  36. 2013-05-28 13:55:31|31074|3243:3604|reinstall|DataStructure.pm:is_parent_reservation(1004)|returning true: parent reservation ID for this request: 3604
  37. 2013-05-28 13:55:31|31074|3243:3604|reinstall|utils.pm:update_request_state(1545)|request 3243 state updated to: pending, laststate to: reinstall
  38. 2013-05-28 13:55:31|31074|3243:3604|reinstall|utils.pm:insertloadlog(3665)|inserted computer=18, begin, beginning to process, state is reinstall
  39. 2013-05-28 13:55:31|31074|3243:3604|reinstall|vcld:make_new_child(512)|request will be processed by new.pm
  40. 2013-05-28 13:55:31|31074|3243:3604|reinstall|vcld:make_new_child(520)|loaded VCL::new module
  41. 2013-05-28 13:55:31|31074|3243:3604|reinstall|vcld:make_new_child(544)|current number of forked kids: 1
  42. 2013-05-28 13:55:31|12878|3243:3604|reinstall|vcld:make_new_child(560)|vcld environment variable set to 0 for this process
  43. 2013-05-28 13:55:31|12878|3243:3604|reinstall|DataStructure.pm:_initialize(643)|image ID argument was specified: noimage, DataStructure object will contain image information for the production imagerevision of this image
  44. 2013-05-28 13:55:31|31074|3243:3605|reinstall|utils.pm:reservation_being_processed(8638)|computerloadlog 'begin' entry does NOT exist for reservation 3605
  45. 2013-05-28 13:55:31|12878|3243:3604|reinstall|DataStructure.pm:_initialize(674)|retrieved data for imagerevision ID: 1
  46. 2013-05-28 13:55:31|12878|3243:3604|reinstall|DataStructure.pm:_initialize(690)|retrieved data for image ID: 1
  47. 2013-05-28 13:55:31|31074|3243:3605|reinstall|utils.pm:run_command(8710)|executed command: pgrep -fl 'vcld [0-9]+:3605 ', pid: 12879, exit status: 1, output:
  48. 2013-05-28 13:55:31|31074|3243:3605|reinstall|utils.pm:is_management_node_process_running(8943)|process is NOT running, identifier: 'vcld [0-9]+:3605 '
  49. 2013-05-28 13:55:31|31074|3243:3605|reinstall|utils.pm:reservation_being_processed(8659)|reservation is NOT currently being processed
  50. 2013-05-28 13:55:31|31074|3243:3605|reinstall|vcld:main(284)|reservation 3605 is NOT already being processed
  51. 2013-05-28 13:55:31|31074|3243:3605|reinstall|utils.pm:get_connect_method_info(10063)|attempting to retrieve connect method info:
  52. |31074|3243:3605|reinstall| imagerevision: 174 - linux-TestImage2-16-v2
  53. |31074|3243:3605|reinstall| OS: 36 - vmwarelinux
  54. |31074|3243:3605|reinstall| OS type: 2 - linux
  55. 2013-05-28 13:55:31|31074|3243:3605|reinstall|utils.pm:get_connect_method_info(10127)|ssh: connectmethodid=1, OStypeid=2, OSid=NULL, imagerevisionid=NULL, disabled=0
  56. 2013-05-28 13:55:31|31074|3243:3605|reinstall|utils.pm:get_connect_method_info(10127)|Xvnc: connectmethodid=4, OStypeid=NULL, OSid=NULL, imagerevisionid=174, disabled=0
  57. 2013-05-28 13:55:31|31074|3243:3605|reinstall|utils.pm:get_user_info(7558)|non-standalone affiliation found for user nsasherr:
  58. |31074|3243:3605|reinstall| user affiliation: sharcnet
  59. |31074|3243:3605|reinstall| not standalone list: sharcnet
  60. 2013-05-28 13:55:31|31074|3243:3605|reinstall|utils.pm:get_user_info(7558)|non-standalone affiliation found for user nsasherr:
  61. |31074|3243:3605|reinstall| user affiliation: sharcnet
  62. |31074|3243:3605|reinstall| not standalone list: sharcnet
  63. 2013-05-28 13:55:31|31074|3243:3605|reinstall|utils.pm:get_connect_method_info(10063)|attempting to retrieve connect method info:
  64. |31074|3243:3605|reinstall| imagerevision: 174 - linux-TestImage2-16-v2
  65. |31074|3243:3605|reinstall| OS: 36 - vmwarelinux
  66. |31074|3243:3605|reinstall| OS type: 2 - linux
  67. 2013-05-28 13:55:31|31074|3243:3605|reinstall|utils.pm:get_connect_method_info(10127)|ssh: connectmethodid=1, OStypeid=2, OSid=NULL, imagerevisionid=NULL, disabled=0
  68. 2013-05-28 13:55:31|31074|3243:3605|reinstall|utils.pm:get_connect_method_info(10127)|Xvnc: connectmethodid=4, OStypeid=NULL, OSid=NULL, imagerevisionid=174, disabled=0
  69. 2013-05-28 13:55:31|31074|3243:3605|reinstall|utils.pm:get_user_info(7558)|non-standalone affiliation found for user nsasherr:
  70. |31074|3243:3605|reinstall| user affiliation: sharcnet
  71. |31074|3243:3605|reinstall| not standalone list: sharcnet
  72. 2013-05-28 13:55:31|31074|3243:3605|reinstall|utils.pm:get_user_info(7558)|non-standalone affiliation found for user nsasherr:
  73. |31074|3243:3605|reinstall| user affiliation: sharcnet
  74. |31074|3243:3605|reinstall| not standalone list: sharcnet
  75. 2013-05-28 13:55:31|31074|3243:3605|reinstall|vcld:main(289)|retrieved request information from database
  76. 2013-05-28 13:55:31|31074|3243:3605|reinstall|DataStructure.pm:is_parent_reservation(1008)|returning false: parent reservation ID for this request: 3604
  77. 2013-05-28 13:55:31|31074|3243:3605|reinstall|vcld:main(335)|child reservation: request state NOT updated to pending
  78. 2013-05-28 13:55:31|12878|3243:3604|reinstall|Module.pm:create_mn_os_object(376)|VCL::Module::OS::Linux::ManagementNode module loaded
  79. 2013-05-28 13:55:31|12878|3243:3604|reinstall|Module.pm:new(196)|VCL::Module::OS::Linux::ManagementNode object created for image noimage, address: 3c49308
  80. 2013-05-28 13:55:31|12878|3243:3604|reinstall|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{0}{computer}{hostname}, data identifier: computer_hostname, data:
  81. |12878|3243:3604|reinstall| : "ecco-adm3.sharcnet.ca"
  82. 2013-05-28 13:55:31|12878|3243:3604|reinstall|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{0}{computer}{NODENAME}, data identifier: computer_node_name, data:
  83. |12878|3243:3604|reinstall| : "ecco-adm3"
  84. 2013-05-28 13:55:31|12878|3243:3604|reinstall|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{0}{computer}{SHORTNAME}, data identifier: computer_short_name, data:
  85. |12878|3243:3604|reinstall| : "ecco-adm3"
  86. 2013-05-28 13:55:31|12878|3243:3604|reinstall|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{0}{computer}{IPaddress}, data identifier: computer_ip_address, data:
  87. |12878|3243:3604|reinstall| : "10.18.20.4"
  88. 2013-05-28 13:55:31|12878|3243:3604|reinstall|Module.pm:create_mn_os_object(381)|VCL::Module::OS::Linux::ManagementNode OS object created, address: 3c49308
  89. 2013-05-28 13:55:31|12878|3243:3604|reinstall|Module.pm:new(192)|VCL::new object created for state reinstall, address: 334d010
  90. 2013-05-28 13:55:31|12878|3243:3604|reinstall|State.pm:initialize(88)|obtained a database handle for this state process, stored as $ENV{dbh}
  91. 2013-05-28 13:55:31|31074|3243:3605|reinstall|utils.pm:insertloadlog(3665)|inserted computer=40, begin, beginning to process, state is reinstall
  92. 2013-05-28 13:55:31|31074|3243:3605|reinstall|vcld:make_new_child(512)|request will be processed by new.pm
  93. 2013-05-28 13:55:31|31074|3243:3605|reinstall|vcld:make_new_child(520)|loaded VCL::new module
  94. 2013-05-28 13:55:31|31074|3243:3605|reinstall|vcld:make_new_child(544)|current number of forked kids: 2
  95. 2013-05-28 13:55:31|12889|3243:3605|reinstall|vcld:make_new_child(560)|vcld environment variable set to 0 for this process
  96. 2013-05-28 13:55:31|12889|3243:3605|reinstall|DataStructure.pm:_initialize(643)|image ID argument was specified: noimage, DataStructure object will contain image information for the production imagerevision of this image
  97. 2013-05-28 13:55:31|12889|3243:3605|reinstall|DataStructure.pm:_initialize(674)|retrieved data for imagerevision ID: 1
  98. 2013-05-28 13:55:31|12889|3243:3605|reinstall|DataStructure.pm:_initialize(690)|retrieved data for image ID: 1
  99. 2013-05-28 13:55:31|12878|3243:3604|reinstall|State.pm:check_image_os(771)|no corrections need to be made, not an imaging request, returning 1
  100. 2013-05-28 13:55:31|12878|3243:3604|reinstall|DataStructure.pm:is_parent_reservation(1004)|returning true: parent reservation ID for this request: 3604
  101. 2013-05-28 13:55:31|12878|3243:3604|reinstall|utils.pm:rename_vcld_process(6756)|reservation count: 2
  102. 2013-05-28 13:55:31|12878|3243:3604|reinstall|utils.pm:rename_vcld_process(6775)|PARENTIMAGE: 1
  103. 2013-05-28 13:55:31|12878|3243:3604|reinstall|utils.pm:rename_vcld_process(6776)|SUBIMAGE: 0
  104. 2013-05-28 13:55:31|12878|3243:3604|reinstall|utils.pm:rename_vcld_process(6794)|renamed process to 'vcld 3243:3604 reinstall ecco-vm-x10>ecco-vmhost-x3 linux-TestImage2-16-v2 nsasherr (cluster=parent)'
  105. 2013-05-28 13:55:31|12878|3243:3604|reinstall|DataStructure.pm:is_parent_reservation(1004)|returning true: parent reservation ID for this request: 3604
  106. 2013-05-28 13:55:31|12878|3243:3604|reinstall|DataStructure.pm:is_parent_reservation(1004)|returning true: parent reservation ID for this request: 3604
  107. 2013-05-28 13:55:31|12878|3243:3604|reinstall|Module.pm:create_os_object(308)|VCL::Module::OS::Linux module loaded
  108. 2013-05-28 13:55:31|12878|3243:3604|reinstall|Module.pm:new(196)|VCL::Module::OS::Linux object created for image linux-TestImage2-16-v2, address: 3c4bbf8
  109. 2013-05-28 13:55:31|12878|3243:3604|reinstall|Module.pm:create_os_object(314)|VCL::Module::OS::Linux OS object created, address: 3c4bbf8
  110. 2013-05-28 13:55:31|12878|3243:3604|reinstall|State.pm:initialize(125)|computer is a VM, attempting to create VM host OS object
  111. 2013-05-28 13:55:31|12878|3243:3604|reinstall|DataStructure.pm:_initialize(619)|computer ID argument was specified, retrieving data for computer ID: 21
  112. 2013-05-28 13:55:31|12878|3243:3604|reinstall|DataStructure.pm:_initialize(643)|image ID argument was specified: 1, DataStructure object will contain image information for the production imagerevision of this image
  113. 2013-05-28 13:55:31|12878|3243:3604|reinstall|DataStructure.pm:_initialize(674)|retrieved data for imagerevision ID: 1
  114. 2013-05-28 13:55:31|12878|3243:3604|reinstall|DataStructure.pm:_initialize(690)|retrieved data for image ID: 1
  115. 2013-05-28 13:55:31|12878|3243:3604|reinstall|Module.pm:create_vmhost_os_object(454)|attempting to load VM host OS module: VCL::Module::OS::Linux::UnixLab (image: 1)
  116. 2013-05-28 13:55:31|12878|3243:3604|reinstall|Module.pm:create_vmhost_os_object(460)|VM host OS module loaded: VCL::Module::OS::Linux::UnixLab
  117. 2013-05-28 13:55:31|12878|3243:3604|reinstall|Module.pm:new(196)|VCL::Module::OS::Linux::UnixLab object created for image noimage, address: 3c4bd60
  118. 2013-05-28 13:55:31|12878|3243:3604|reinstall|Module.pm:create_vmhost_os_object(465)|VCL::Module::OS::Linux::UnixLab OS object created, address: 3c4bd60
  119. 2013-05-28 13:55:31|12878|3243:3604|reinstall|Module.pm:create_provisioning_object(519)|VCL::Module::Provisioning::libvirt module loaded
  120. 2013-05-28 13:55:31|12878|3243:3604|reinstall|Module.pm:new(200)|VCL::Module::Provisioning::libvirt object created for computer ecco-vm-x10, address: 3c4c108
  121. 2013-05-28 13:55:31|12878|3243:3604|reinstall|libvirt.pm:initialize(99)|libvirt driver module directory path: /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/libvirt
  122. 2013-05-28 13:55:31|12878|3243:3604|reinstall|OS.pm:find_files(2917)|attempting to find files on ecco-adm3, base directory path: '/usr/local/vcl/bin/../lib/VCL/Module/Provisioning/libvirt/', pattern: *.pm, command: /usr/bin/find "/usr/local/vcl/bin/../lib/VCL/Module/Provisioning/libvirt/" -type f -iname "*.pm"
  123. 2013-05-28 13:55:31|12878|3243:3604|reinstall|OS.pm:find_files(2945)|files found: 1, base directory: '/usr/local/vcl/bin/../lib/VCL/Module/Provisioning/libvirt/', pattern: '*.pm'
  124. |12878|3243:3604|reinstall| command: '/usr/bin/find "/usr/local/vcl/bin/../lib/VCL/Module/Provisioning/libvirt/" -type f -iname "*.pm"'
  125. 2013-05-28 13:55:31|12878|3243:3604|reinstall|Module.pm:new(200)|VCL::Module::Provisioning::libvirt::KVM object created for computer ecco-vm-x10, address: 3e63320
  126. 2013-05-28 13:55:31|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  127. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'which virsh' 2>&1
  128. 2013-05-28 13:55:31|12889|3243:3605|reinstall|Module.pm:create_mn_os_object(376)|VCL::Module::OS::Linux::ManagementNode module loaded
  129. 2013-05-28 13:55:31|12889|3243:3605|reinstall|Module.pm:new(196)|VCL::Module::OS::Linux::ManagementNode object created for image noimage, address: 3c48d88
  130. 2013-05-28 13:55:31|12889|3243:3605|reinstall|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{0}{computer}{hostname}, data identifier: computer_hostname, data:
  131. |12889|3243:3605|reinstall| : "ecco-adm3.sharcnet.ca"
  132. 2013-05-28 13:55:31|12889|3243:3605|reinstall|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{0}{computer}{NODENAME}, data identifier: computer_node_name, data:
  133. |12889|3243:3605|reinstall| : "ecco-adm3"
  134. 2013-05-28 13:55:31|12889|3243:3605|reinstall|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{0}{computer}{SHORTNAME}, data identifier: computer_short_name, data:
  135. |12889|3243:3605|reinstall| : "ecco-adm3"
  136. 2013-05-28 13:55:31|12889|3243:3605|reinstall|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{0}{computer}{IPaddress}, data identifier: computer_ip_address, data:
  137. |12889|3243:3605|reinstall| : "10.18.20.4"
  138. 2013-05-28 13:55:31|12889|3243:3605|reinstall|Module.pm:create_mn_os_object(381)|VCL::Module::OS::Linux::ManagementNode OS object created, address: 3c48d88
  139. 2013-05-28 13:55:31|12889|3243:3605|reinstall|Module.pm:new(192)|VCL::new object created for state reinstall, address: 36a8430
  140. 2013-05-28 13:55:31|12889|3243:3605|reinstall|State.pm:initialize(88)|obtained a database handle for this state process, stored as $ENV{dbh}
  141. 2013-05-28 13:55:31|12889|3243:3605|reinstall|State.pm:check_image_os(771)|no corrections need to be made, not an imaging request, returning 1
  142. 2013-05-28 13:55:31|12889|3243:3605|reinstall|DataStructure.pm:is_parent_reservation(1008)|returning false: parent reservation ID for this request: 3604
  143. 2013-05-28 13:55:31|12889|3243:3605|reinstall|utils.pm:rename_vcld_process(6756)|reservation count: 2
  144. 2013-05-28 13:55:31|12889|3243:3605|reinstall|utils.pm:rename_vcld_process(6775)|PARENTIMAGE: 0
  145. 2013-05-28 13:55:31|12889|3243:3605|reinstall|utils.pm:rename_vcld_process(6776)|SUBIMAGE: 1
  146. 2013-05-28 13:55:31|12889|3243:3605|reinstall|utils.pm:rename_vcld_process(6794)|renamed process to 'vcld 3243:3605 reinstall ecco-vm-x26>ecco-vmhost-x7 linux-TestImage2-16-v2 nsasherr (cluster=child)'
  147. 2013-05-28 13:55:31|12889|3243:3605|reinstall|DataStructure.pm:is_parent_reservation(1008)|returning false: parent reservation ID for this request: 3604
  148. 2013-05-28 13:55:31|12889|3243:3605|reinstall|DataStructure.pm:is_parent_reservation(1008)|returning false: parent reservation ID for this request: 3604
  149. 2013-05-28 13:55:31|12889|3243:3605|reinstall|Module.pm:create_os_object(308)|VCL::Module::OS::Linux module loaded
  150. 2013-05-28 13:55:31|12889|3243:3605|reinstall|Module.pm:new(196)|VCL::Module::OS::Linux object created for image linux-TestImage2-16-v2, address: 3c4b7f8
  151. 2013-05-28 13:55:31|12889|3243:3605|reinstall|Module.pm:create_os_object(314)|VCL::Module::OS::Linux OS object created, address: 3c4b7f8
  152. 2013-05-28 13:55:31|12889|3243:3605|reinstall|State.pm:initialize(125)|computer is a VM, attempting to create VM host OS object
  153. 2013-05-28 13:55:31|12889|3243:3605|reinstall|DataStructure.pm:_initialize(619)|computer ID argument was specified, retrieving data for computer ID: 24
  154. 2013-05-28 13:55:31|12889|3243:3605|reinstall|DataStructure.pm:_initialize(643)|image ID argument was specified: 1, DataStructure object will contain image information for the production imagerevision of this image
  155. 2013-05-28 13:55:31|12889|3243:3605|reinstall|DataStructure.pm:_initialize(674)|retrieved data for imagerevision ID: 1
  156. 2013-05-28 13:55:31|12889|3243:3605|reinstall|DataStructure.pm:_initialize(690)|retrieved data for image ID: 1
  157. 2013-05-28 13:55:31|12889|3243:3605|reinstall|Module.pm:create_vmhost_os_object(454)|attempting to load VM host OS module: VCL::Module::OS::Linux::UnixLab (image: 1)
  158. 2013-05-28 13:55:31|12889|3243:3605|reinstall|Module.pm:create_vmhost_os_object(460)|VM host OS module loaded: VCL::Module::OS::Linux::UnixLab
  159. 2013-05-28 13:55:31|12889|3243:3605|reinstall|Module.pm:new(196)|VCL::Module::OS::Linux::UnixLab object created for image noimage, address: 3c4b960
  160. 2013-05-28 13:55:31|12889|3243:3605|reinstall|Module.pm:create_vmhost_os_object(465)|VCL::Module::OS::Linux::UnixLab OS object created, address: 3c4b960
  161. 2013-05-28 13:55:32|12889|3243:3605|reinstall|Module.pm:create_provisioning_object(519)|VCL::Module::Provisioning::libvirt module loaded
  162. 2013-05-28 13:55:32|12889|3243:3605|reinstall|Module.pm:new(200)|VCL::Module::Provisioning::libvirt object created for computer ecco-vm-x26, address: 3c78880
  163. 2013-05-28 13:55:32|12889|3243:3605|reinstall|libvirt.pm:initialize(99)|libvirt driver module directory path: /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/libvirt
  164. 2013-05-28 13:55:32|12889|3243:3605|reinstall|OS.pm:find_files(2917)|attempting to find files on ecco-adm3, base directory path: '/usr/local/vcl/bin/../lib/VCL/Module/Provisioning/libvirt/', pattern: *.pm, command: /usr/bin/find "/usr/local/vcl/bin/../lib/VCL/Module/Provisioning/libvirt/" -type f -iname "*.pm"
  165. 2013-05-28 13:55:32|12889|3243:3605|reinstall|OS.pm:find_files(2945)|files found: 1, base directory: '/usr/local/vcl/bin/../lib/VCL/Module/Provisioning/libvirt/', pattern: '*.pm'
  166. |12889|3243:3605|reinstall| command: '/usr/bin/find "/usr/local/vcl/bin/../lib/VCL/Module/Provisioning/libvirt/" -type f -iname "*.pm"'
  167. 2013-05-28 13:55:32|12889|3243:3605|reinstall|Module.pm:new(200)|VCL::Module::Provisioning::libvirt::KVM object created for computer ecco-vm-x26, address: 3e65e30
  168. 2013-05-28 13:55:32|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x7:
  169. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x7 'which virsh' 2>&1
  170. 2013-05-28 13:55:32|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  171. |12878|3243:3604|reinstall| /usr/bin/virsh
  172. 2013-05-28 13:55:32|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x3, returning (0, "/usr/bin/virsh")
  173. 2013-05-28 13:55:32|12878|3243:3604|reinstall|KVM.pm:initialize(104)|verified 'virsh' command is available on ecco-vmhost-x3
  174. 2013-05-28 13:55:32|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  175. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'which qemu-img' 2>&1
  176. 2013-05-28 13:55:32|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  177. |12889|3243:3605|reinstall| /usr/bin/virsh
  178. 2013-05-28 13:55:32|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x7, returning (0, "/usr/bin/virsh")
  179. 2013-05-28 13:55:32|12889|3243:3605|reinstall|KVM.pm:initialize(104)|verified 'virsh' command is available on ecco-vmhost-x7
  180. 2013-05-28 13:55:32|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x7:
  181. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x7 'which qemu-img' 2>&1
  182. 2013-05-28 13:55:32|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  183. |12878|3243:3604|reinstall| /usr/bin/qemu-img
  184. 2013-05-28 13:55:32|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x3, returning (0, "/usr/bin/qemu-img")
  185. 2013-05-28 13:55:32|12878|3243:3604|reinstall|KVM.pm:initialize(104)|verified 'qemu-img' command is available on ecco-vmhost-x3
  186. 2013-05-28 13:55:32|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  187. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'which virt-win-reg' 2>&1
  188. 2013-05-28 13:55:32|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  189. |12889|3243:3605|reinstall| /usr/bin/qemu-img
  190. 2013-05-28 13:55:32|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x7, returning (0, "/usr/bin/qemu-img")
  191. 2013-05-28 13:55:32|12889|3243:3605|reinstall|KVM.pm:initialize(104)|verified 'qemu-img' command is available on ecco-vmhost-x7
  192. 2013-05-28 13:55:32|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x7:
  193. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x7 'which virt-win-reg' 2>&1
  194. 2013-05-28 13:55:32|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  195. |12878|3243:3604|reinstall| /usr/bin/virt-win-reg
  196. 2013-05-28 13:55:32|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x3, returning (0, "/usr/bin/virt-win-reg")
  197. 2013-05-28 13:55:32|12878|3243:3604|reinstall|KVM.pm:initialize(104)|verified 'virt-win-reg' command is available on ecco-vmhost-x3
  198. 2013-05-28 13:55:32|12878|3243:3604|reinstall|KVM.pm:initialize(113)|KVM driver module successfully initialized to control ecco-vmhost-x3
  199. 2013-05-28 13:55:32|12878|3243:3604|reinstall|libvirt.pm:initialize(119)|libvirt KVM driver object created and initialized to control ecco-vmhost-x3
  200. 2013-05-28 13:55:32|12878|3243:3604|reinstall|libvirt.pm:initialize(139)|VCL::Module::Provisioning::libvirt provisioning module initialized
  201. 2013-05-28 13:55:32|12878|3243:3604|reinstall|Module.pm:create_provisioning_object(525)|VCL::Module::Provisioning::libvirt provisioner object created for ecco-vm-x10, address: 3c4c108
  202. 2013-05-28 13:55:32|12878|3243:3604|reinstall|State.pm:initialize(154)|returning 1
  203. 2013-05-28 13:55:32|12878|3243:3604|reinstall|vcld:make_new_child(570)|VCL::new object created and initialized
  204. 2013-05-28 13:55:32|12878|3243:3604|reinstall|DataStructure.pm:is_parent_reservation(1004)|returning true: parent reservation ID for this request: 3604
  205. 2013-05-28 13:55:32|12878|3243:3604|reinstall|DataStructure.pm:get_computer_state_name(2436)|attempting to retrieve current state of computer ecco-vm-x10 from the database
  206. 2013-05-28 13:55:32|12878|3243:3604|reinstall|DataStructure.pm:get_computer_state_name(2467)|retrieved current state of computer ecco-vm-x10 from the database: inuse
  207. 2013-05-28 13:55:32|12878|3243:3604|reinstall|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{3604}{computer}{state}{name}, data identifier: computer_state_name, data:
  208. |12878|3243:3604|reinstall| : "inuse"
  209. 2013-05-28 13:55:32|12878|3243:3604|reinstall|DataStructure.pm:get_computer_state_name(2436)|attempting to retrieve current state of computer ecco-vm-x10 from the database
  210. 2013-05-28 13:55:32|12878|3243:3604|reinstall|DataStructure.pm:get_computer_state_name(2467)|retrieved current state of computer ecco-vm-x10 from the database: inuse
  211. 2013-05-28 13:55:32|12878|3243:3604|reinstall|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{3604}{computer}{state}{name}, data identifier: computer_state_name, data:
  212. |12878|3243:3604|reinstall| : "inuse"
  213. 2013-05-28 13:55:32|12878|3243:3604|reinstall|new.pm:computer_not_being_used(761)|retrieving info for reservations assigned to ecco-vm-x10
  214. 2013-05-28 13:55:32|12878|3243:3604|reinstall|utils.pm:get_connect_method_info(10063)|attempting to retrieve connect method info:
  215. |12878|3243:3604|reinstall| imagerevision: 174 - linux-TestImage2-16-v2
  216. |12878|3243:3604|reinstall| OS: 36 - vmwarelinux
  217. |12878|3243:3604|reinstall| OS type: 2 - linux
  218. 2013-05-28 13:55:32|12878|3243:3604|reinstall|utils.pm:get_connect_method_info(10127)|ssh: connectmethodid=1, OStypeid=2, OSid=NULL, imagerevisionid=NULL, disabled=0
  219. 2013-05-28 13:55:32|12878|3243:3604|reinstall|utils.pm:get_connect_method_info(10127)|Xvnc: connectmethodid=4, OStypeid=NULL, OSid=NULL, imagerevisionid=174, disabled=0
  220. 2013-05-28 13:55:32|12878|3243:3604|reinstall|utils.pm:get_user_info(7558)|non-standalone affiliation found for user nsasherr:
  221. |12878|3243:3604|reinstall| user affiliation: sharcnet
  222. |12878|3243:3604|reinstall| not standalone list: sharcnet
  223. 2013-05-28 13:55:32|12878|3243:3604|reinstall|utils.pm:get_user_info(7558)|non-standalone affiliation found for user nsasherr:
  224. |12878|3243:3604|reinstall| user affiliation: sharcnet
  225. |12878|3243:3604|reinstall| not standalone list: sharcnet
  226. 2013-05-28 13:55:32|12878|3243:3604|reinstall|utils.pm:get_connect_method_info(10063)|attempting to retrieve connect method info:
  227. |12878|3243:3604|reinstall| imagerevision: 174 - linux-TestImage2-16-v2
  228. |12878|3243:3604|reinstall| OS: 36 - vmwarelinux
  229. |12878|3243:3604|reinstall| OS type: 2 - linux
  230. 2013-05-28 13:55:32|12878|3243:3604|reinstall|utils.pm:get_connect_method_info(10127)|ssh: connectmethodid=1, OStypeid=2, OSid=NULL, imagerevisionid=NULL, disabled=0
  231. 2013-05-28 13:55:32|12878|3243:3604|reinstall|utils.pm:get_connect_method_info(10127)|Xvnc: connectmethodid=4, OStypeid=NULL, OSid=NULL, imagerevisionid=174, disabled=0
  232. 2013-05-28 13:55:32|12878|3243:3604|reinstall|utils.pm:get_user_info(7558)|non-standalone affiliation found for user nsasherr:
  233. |12878|3243:3604|reinstall| user affiliation: sharcnet
  234. |12878|3243:3604|reinstall| not standalone list: sharcnet
  235. 2013-05-28 13:55:32|12878|3243:3604|reinstall|utils.pm:get_user_info(7558)|non-standalone affiliation found for user nsasherr:
  236. |12878|3243:3604|reinstall| user affiliation: sharcnet
  237. |12878|3243:3604|reinstall| not standalone list: sharcnet
  238. 2013-05-28 13:55:32|12878|3243:3604|reinstall|utils.pm:get_request_by_computerid(5991)|retrieved info and DataStructure object for 3243:3604
  239. 2013-05-28 13:55:32|12878|3243:3604|reinstall|new.pm:computer_not_being_used(774)|ecco-vm-x10 is not assigned to any other reservations
  240. 2013-05-28 13:55:32|12878|3243:3604|reinstall|new.pm:process(127)|ecco-vm-x10 is not being used
  241. 2013-05-28 13:55:32|12878|3243:3604|reinstall|DataStructure.pm:get_computer_state_name(2436)|attempting to retrieve current state of computer ecco-vm-x10 from the database
  242. 2013-05-28 13:55:32|12878|3243:3604|reinstall|DataStructure.pm:get_computer_state_name(2467)|retrieved current state of computer ecco-vm-x10 from the database: inuse
  243. 2013-05-28 13:55:32|12878|3243:3604|reinstall|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{3604}{computer}{state}{name}, data identifier: computer_state_name, data:
  244. |12878|3243:3604|reinstall| : "inuse"
  245. 2013-05-28 13:55:32|12878|3243:3604|reinstall|new.pm:reload_image(530)|calling VCL::Module::Provisioning::libvirt->node_status()
  246. 2013-05-28 13:55:32|12878|3243:3604|reinstall|utils.pm:insertloadlog(3665)|inserted computer=18, statuscheck, checking status of node
  247. 2013-05-28 13:55:32|12878|3243:3604|reinstall|Provisioning.pm:node_status(162)|attempting to check the status of computer ecco-vm-x10, image: linux-TestImage2-16-v2 imagerevision_id; 174
  248. 2013-05-28 13:55:32|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  249. |12889|3243:3605|reinstall| /usr/bin/virt-win-reg
  250. 2013-05-28 13:55:32|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x7, returning (0, "/usr/bin/virt-win-reg")
  251. 2013-05-28 13:55:32|12889|3243:3605|reinstall|KVM.pm:initialize(104)|verified 'virt-win-reg' command is available on ecco-vmhost-x7
  252. 2013-05-28 13:55:32|12889|3243:3605|reinstall|KVM.pm:initialize(113)|KVM driver module successfully initialized to control ecco-vmhost-x7
  253. 2013-05-28 13:55:32|12889|3243:3605|reinstall|libvirt.pm:initialize(119)|libvirt KVM driver object created and initialized to control ecco-vmhost-x7
  254. 2013-05-28 13:55:32|12889|3243:3605|reinstall|libvirt.pm:initialize(139)|VCL::Module::Provisioning::libvirt provisioning module initialized
  255. 2013-05-28 13:55:32|12889|3243:3605|reinstall|Module.pm:create_provisioning_object(525)|VCL::Module::Provisioning::libvirt provisioner object created for ecco-vm-x26, address: 3c78880
  256. 2013-05-28 13:55:32|12889|3243:3605|reinstall|State.pm:initialize(154)|returning 1
  257. 2013-05-28 13:55:32|12889|3243:3605|reinstall|vcld:make_new_child(570)|VCL::new object created and initialized
  258. 2013-05-28 13:55:32|12889|3243:3605|reinstall|DataStructure.pm:is_parent_reservation(1008)|returning false: parent reservation ID for this request: 3604
  259. 2013-05-28 13:55:32|12889|3243:3605|reinstall|DataStructure.pm:get_computer_state_name(2436)|attempting to retrieve current state of computer ecco-vm-x26 from the database
  260. 2013-05-28 13:55:32|12889|3243:3605|reinstall|DataStructure.pm:get_computer_state_name(2467)|retrieved current state of computer ecco-vm-x26 from the database: inuse
  261. 2013-05-28 13:55:32|12889|3243:3605|reinstall|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{3605}{computer}{state}{name}, data identifier: computer_state_name, data:
  262. |12889|3243:3605|reinstall| : "inuse"
  263. 2013-05-28 13:55:32|12889|3243:3605|reinstall|DataStructure.pm:get_computer_state_name(2436)|attempting to retrieve current state of computer ecco-vm-x26 from the database
  264. 2013-05-28 13:55:32|12889|3243:3605|reinstall|DataStructure.pm:get_computer_state_name(2467)|retrieved current state of computer ecco-vm-x26 from the database: inuse
  265. 2013-05-28 13:55:32|12889|3243:3605|reinstall|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{3605}{computer}{state}{name}, data identifier: computer_state_name, data:
  266. |12889|3243:3605|reinstall| : "inuse"
  267. 2013-05-28 13:55:32|12889|3243:3605|reinstall|new.pm:computer_not_being_used(761)|retrieving info for reservations assigned to ecco-vm-x26
  268. 2013-05-28 13:55:32|12889|3243:3605|reinstall|utils.pm:get_connect_method_info(10063)|attempting to retrieve connect method info:
  269. |12889|3243:3605|reinstall| imagerevision: 174 - linux-TestImage2-16-v2
  270. |12889|3243:3605|reinstall| OS: 36 - vmwarelinux
  271. |12889|3243:3605|reinstall| OS type: 2 - linux
  272. 2013-05-28 13:55:32|12889|3243:3605|reinstall|utils.pm:get_connect_method_info(10127)|ssh: connectmethodid=1, OStypeid=2, OSid=NULL, imagerevisionid=NULL, disabled=0
  273. 2013-05-28 13:55:32|12889|3243:3605|reinstall|utils.pm:get_connect_method_info(10127)|Xvnc: connectmethodid=4, OStypeid=NULL, OSid=NULL, imagerevisionid=174, disabled=0
  274. 2013-05-28 13:55:32|12889|3243:3605|reinstall|utils.pm:get_user_info(7558)|non-standalone affiliation found for user nsasherr:
  275. |12889|3243:3605|reinstall| user affiliation: sharcnet
  276. |12889|3243:3605|reinstall| not standalone list: sharcnet
  277. 2013-05-28 13:55:32|12889|3243:3605|reinstall|utils.pm:get_user_info(7558)|non-standalone affiliation found for user nsasherr:
  278. |12889|3243:3605|reinstall| user affiliation: sharcnet
  279. |12889|3243:3605|reinstall| not standalone list: sharcnet
  280. 2013-05-28 13:55:32|12889|3243:3605|reinstall|utils.pm:get_connect_method_info(10063)|attempting to retrieve connect method info:
  281. |12889|3243:3605|reinstall| imagerevision: 174 - linux-TestImage2-16-v2
  282. |12889|3243:3605|reinstall| OS: 36 - vmwarelinux
  283. |12889|3243:3605|reinstall| OS type: 2 - linux
  284. 2013-05-28 13:55:32|12889|3243:3605|reinstall|utils.pm:get_connect_method_info(10127)|ssh: connectmethodid=1, OStypeid=2, OSid=NULL, imagerevisionid=NULL, disabled=0
  285. 2013-05-28 13:55:32|12889|3243:3605|reinstall|utils.pm:get_connect_method_info(10127)|Xvnc: connectmethodid=4, OStypeid=NULL, OSid=NULL, imagerevisionid=174, disabled=0
  286. 2013-05-28 13:55:32|12889|3243:3605|reinstall|utils.pm:get_user_info(7558)|non-standalone affiliation found for user nsasherr:
  287. |12889|3243:3605|reinstall| user affiliation: sharcnet
  288. |12889|3243:3605|reinstall| not standalone list: sharcnet
  289. 2013-05-28 13:55:32|12889|3243:3605|reinstall|utils.pm:get_user_info(7558)|non-standalone affiliation found for user nsasherr:
  290. |12889|3243:3605|reinstall| user affiliation: sharcnet
  291. |12889|3243:3605|reinstall| not standalone list: sharcnet
  292. 2013-05-28 13:55:32|12889|3243:3605|reinstall|utils.pm:get_request_by_computerid(5991)|retrieved info and DataStructure object for 3243:3605
  293. 2013-05-28 13:55:32|12889|3243:3605|reinstall|new.pm:computer_not_being_used(774)|ecco-vm-x26 is not assigned to any other reservations
  294. 2013-05-28 13:55:32|12889|3243:3605|reinstall|new.pm:process(127)|ecco-vm-x26 is not being used
  295. 2013-05-28 13:55:32|12889|3243:3605|reinstall|DataStructure.pm:get_computer_state_name(2436)|attempting to retrieve current state of computer ecco-vm-x26 from the database
  296. 2013-05-28 13:55:32|12889|3243:3605|reinstall|DataStructure.pm:get_computer_state_name(2467)|retrieved current state of computer ecco-vm-x26 from the database: inuse
  297. 2013-05-28 13:55:32|12889|3243:3605|reinstall|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{3605}{computer}{state}{name}, data identifier: computer_state_name, data:
  298. |12889|3243:3605|reinstall| : "inuse"
  299. 2013-05-28 13:55:32|12889|3243:3605|reinstall|new.pm:reload_image(530)|calling VCL::Module::Provisioning::libvirt->node_status()
  300. 2013-05-28 13:55:32|12889|3243:3605|reinstall|utils.pm:insertloadlog(3665)|inserted computer=40, statuscheck, checking status of node
  301. 2013-05-28 13:55:32|12889|3243:3605|reinstall|Provisioning.pm:node_status(162)|attempting to check the status of computer ecco-vm-x26, image: linux-TestImage2-16-v2 imagerevision_id; 174
  302. 2013-05-28 13:55:32|12878|3243:3604|reinstall|OS.pm:is_ssh_responding(678)|ecco-vm-x10 is responding to SSH, port 22: open, port 24: closed
  303. 2013-05-28 13:55:32|12878|3243:3604|reinstall|Provisioning.pm:node_status(173)|ecco-vm-x10 is responding to SSH
  304. 2013-05-28 13:55:32|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  305. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'cat ~/currentimage.txt' 2>&1
  306. 2013-05-28 13:55:32|12889|3243:3605|reinstall|OS.pm:is_ssh_responding(678)|ecco-vm-x26 is responding to SSH, port 22: open, port 24: closed
  307. 2013-05-28 13:55:32|12889|3243:3605|reinstall|Provisioning.pm:node_status(173)|ecco-vm-x26 is responding to SSH
  308. 2013-05-28 13:55:32|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  309. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'cat ~/currentimage.txt' 2>&1
  310. 2013-05-28 13:55:32|12878|3243:3604|reinstall|OS.pm:execute(1994)|executed command: 'cat ~/currentimage.txt', exit status: 0, output:
  311. |12878|3243:3604|reinstall| linux-TestImage2-16-v2
  312. |12878|3243:3604|reinstall| id=16
  313. |12878|3243:3604|reinstall| prettyname=Test Image 2
  314. |12878|3243:3604|reinstall| imagerevision_id=174
  315. |12878|3243:3604|reinstall| imagerevision_datecreated=2013-05-27 13:17:23
  316. |12878|3243:3604|reinstall| computer_id=10
  317. |12878|3243:3604|reinstall| computer_hostname=ecco-vm-x5
  318. |12878|3243:3604|reinstall| vcld_post_load=success (Tue May 28 13:51:09 2013)
  319. 2013-05-28 13:55:32|12878|3243:3604|reinstall|OS.pm:get_currentimage_txt_contents(276)|retrieved currentimage.txt contents from ecco-vm-x10:
  320. |12878|3243:3604|reinstall| linux-TestImage2-16-v2
  321. |12878|3243:3604|reinstall| id=16
  322. |12878|3243:3604|reinstall| prettyname=Test Image 2
  323. |12878|3243:3604|reinstall| imagerevision_id=174
  324. |12878|3243:3604|reinstall| imagerevision_datecreated=2013-05-27 13:17:23
  325. |12878|3243:3604|reinstall| computer_id=10
  326. |12878|3243:3604|reinstall| computer_hostname=ecco-vm-x5
  327. |12878|3243:3604|reinstall| vcld_post_load=success (Tue May 28 13:51:09 2013)
  328. 2013-05-28 13:55:32|12878|3243:3604|reinstall|OS.pm:get_current_image_info(331)|retrieved currentimage.txt contents from ecco-vm-x10
  329. 2013-05-28 13:55:32|12878|3243:3604|reinstall|OS.pm:get_current_image_info(341)|user selected content of image currently loaded on ecco-vm-x10: linux-TestImage2-16-v2
  330. 2013-05-28 13:55:32|12878|3243:3604|reinstall|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{3604}{computer}{currentimage}, data identifier: computer_currentimage_data, data:
  331. |12878|3243:3604|reinstall| : {
  332. |12878|3243:3604|reinstall| : "IDENTITY" => "/etc/vcl/vcl.key",
  333. |12878|3243:3604|reinstall| : "OS" => {
  334. |12878|3243:3604|reinstall| : "OStype" => {
  335. |12878|3243:3604|reinstall| : "id" => 2,
  336. |12878|3243:3604|reinstall| : "name" => "linux"
  337. |12878|3243:3604|reinstall| : },
  338. |12878|3243:3604|reinstall| : "id" => 36,
  339. |12878|3243:3604|reinstall| : "installtype" => "vmware",
  340. |12878|3243:3604|reinstall| : "module" => {
  341. |12878|3243:3604|reinstall| : "description" => "",
  342. |12878|3243:3604|reinstall| : "id" => 5,
  343. |12878|3243:3604|reinstall| : "name" => "os_linux",
  344. |12878|3243:3604|reinstall| : "perlpackage" => "VCL::Module::OS::Linux",
  345. |12878|3243:3604|reinstall| : "prettyname" => "Linux OS Module"
  346. |12878|3243:3604|reinstall| : },
  347. |12878|3243:3604|reinstall| : "moduleid" => 5,
  348. |12878|3243:3604|reinstall| : "name" => "vmwarelinux",
  349. |12878|3243:3604|reinstall| : "prettyname" => "Generic Linux (VMware)",
  350. |12878|3243:3604|reinstall| : "sourcepath" => "vmware_images",
  351. |12878|3243:3604|reinstall| : "type" => "linux"
  352. |12878|3243:3604|reinstall| : },
  353. |12878|3243:3604|reinstall| : "OSid" => 36,
  354. |12878|3243:3604|reinstall| : "architecture" => "x86_64",
  355. |12878|3243:3604|reinstall| : "basedoffrevisionid" => 127,
  356. |12878|3243:3604|reinstall| : "deleted" => 0,
  357. |12878|3243:3604|reinstall| : "description" => "Test Image 2",
  358. |12878|3243:3604|reinstall| : "forcheckout" => 1,
  359. |12878|3243:3604|reinstall| : "id" => 16,
  360. |12878|3243:3604|reinstall| : "imagemeta" => {
  361. |12878|3243:3604|reinstall| : "architecture" => "",
  362. |12878|3243:3604|reinstall| : "checkuser" => 0,
  363. |12878|3243:3604|reinstall| : "id" => 16,
  364. |12878|3243:3604|reinstall| : "postoption" => "",
  365. |12878|3243:3604|reinstall| : "rootaccess" => 0,
  366. |12878|3243:3604|reinstall| : "subimages" => 1,
  367. |12878|3243:3604|reinstall| : "sysprep" => 0,
  368. |12878|3243:3604|reinstall| : "usergroupid" => ""
  369. |12878|3243:3604|reinstall| : },
  370. |12878|3243:3604|reinstall| : "imagemetaid" => 16,
  371. |12878|3243:3604|reinstall| : "imagetype" => {
  372. |12878|3243:3604|reinstall| : "id" => 7,
  373. |12878|3243:3604|reinstall| : "name" => "qcow2"
  374. |12878|3243:3604|reinstall| : },
  375. |12878|3243:3604|reinstall| : "imagetypeid" => 7,
  376. |12878|3243:3604|reinstall| : "lastupdate" => "2013-05-27 13:17:23",
  377. |12878|3243:3604|reinstall| : "maxconcurrent" => undef,
  378. |12878|3243:3604|reinstall| : "maxinitialtime" => 0,
  379. |12878|3243:3604|reinstall| : "minnetwork" => 10,
  380. |12878|3243:3604|reinstall| : "minprocnumber" => 2,
  381. |12878|3243:3604|reinstall| : "minprocspeed" => 500,
  382. |12878|3243:3604|reinstall| : "minram" => 8191,
  383. |12878|3243:3604|reinstall| : "name" => "linux-TestImage2-16-v2",
  384. |12878|3243:3604|reinstall| : "owner" => {
  385. |12878|3243:3604|reinstall| : "IMid" => "",
  386. |12878|3243:3604|reinstall| : "IMtype" => {
  387. |12878|3243:3604|reinstall| : "name" => "none"
  388. |12878|3243:3604|reinstall| : },
  389. |12878|3243:3604|reinstall| : "IMtypeid" => 1,
  390. |12878|3243:3604|reinstall| : "STANDALONE" => 1,
  391. |12878|3243:3604|reinstall| : "adminlevel" => {
  392. |12878|3243:3604|reinstall| : "name" => "developer"
  393. |12878|3243:3604|reinstall| : },
  394. |12878|3243:3604|reinstall| : "adminlevelid" => 3,
  395. |12878|3243:3604|reinstall| : "affiliation" => {
  396. |12878|3243:3604|reinstall| : "dataUpdateText" => "",
  397. |12878|3243:3604|reinstall| : "helpaddress" => "nobody\@example.com",
  398. |12878|3243:3604|reinstall| : "name" => "Local",
  399. |12878|3243:3604|reinstall| : "shibname" => undef,
  400. |12878|3243:3604|reinstall| : "shibonly" => 0,
  401. |12878|3243:3604|reinstall| : "sitewwwaddress" => "http://cwiki.apache.org/VCL"
  402. |12878|3243:3604|reinstall| : },
  403. |12878|3243:3604|reinstall| : "affiliationid" => 1,
  404. |12878|3243:3604|reinstall| : "audiomode" => "local",
  405. |12878|3243:3604|reinstall| : "bpp" => 16,
  406. |12878|3243:3604|reinstall| : "email" => "vcladmin\@sharcnet.ca",
  407. |12878|3243:3604|reinstall| : "emailnotices" => 0,
  408. |12878|3243:3604|reinstall| : "firstname" => "vcl",
  409. |12878|3243:3604|reinstall| : "height" => 768,
  410. |12878|3243:3604|reinstall| : "id" => 1,
  411. |12878|3243:3604|reinstall| : "lastname" => "admin",
  412. |12878|3243:3604|reinstall| : "lastupdated" => "2007-05-17 09:58:39",
  413. |12878|3243:3604|reinstall| : "localauth" => {
  414. |12878|3243:3604|reinstall| : "lastupdated" => "2007-05-17 09:56:01",
  415. |12878|3243:3604|reinstall| : "lockedout" => 0,
  416. |12878|3243:3604|reinstall| : "passhash" => "e3d58029c003867cba100e43137f8814f3fe5de2",
  417. |12878|3243:3604|reinstall| : "salt" => "QwkCHLpY"
  418. |12878|3243:3604|reinstall| : },
  419. |12878|3243:3604|reinstall| : "mapdrives" => 1,
  420. |12878|3243:3604|reinstall| : "mapprinters" => 1,
  421. |12878|3243:3604|reinstall| : "mapserial" => 1,
  422. |12878|3243:3604|reinstall| : "preferredname" => "vcl",
  423. |12878|3243:3604|reinstall| : "showallgroups" => 1,
  424. |12878|3243:3604|reinstall| : "uid" => 1000,
  425. |12878|3243:3604|reinstall| : "unityid" => "admin",
  426. |12878|3243:3604|reinstall| : "validated" => 1,
  427. |12878|3243:3604|reinstall| : "width" => 1024
  428. |12878|3243:3604|reinstall| : },
  429. |12878|3243:3604|reinstall| : "ownerid" => 1,
  430. |12878|3243:3604|reinstall| : "platform" => {
  431. |12878|3243:3604|reinstall| : "id" => 1,
  432. |12878|3243:3604|reinstall| : "name" => "x86_64"
  433. |12878|3243:3604|reinstall| : },
  434. |12878|3243:3604|reinstall| : "platformid" => 1,
  435. |12878|3243:3604|reinstall| : "prettyname" => "Test Image 2",
  436. |12878|3243:3604|reinstall| : "project" => "vcl",
  437. |12878|3243:3604|reinstall| : "reloadtime" => 20,
  438. |12878|3243:3604|reinstall| : "size" => 3336,
  439. |12878|3243:3604|reinstall| : "test" => 1,
  440. |12878|3243:3604|reinstall| : "usage" => ""
  441. |12878|3243:3604|reinstall| : }
  442. 2013-05-28 13:55:32|12878|3243:3604|reinstall|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{3604}{computer}{currentimagerevision}, data identifier: computer_currentimagerevision_data, data:
  443. |12878|3243:3604|reinstall| : {
  444. |12878|3243:3604|reinstall| : "autocaptured" => 0,
  445. |12878|3243:3604|reinstall| : "comments" => "Fix for using the 'mark' module on a child node of a cluster.",
  446. |12878|3243:3604|reinstall| : "datecreated" => "2013-05-27 13:17:23",
  447. |12878|3243:3604|reinstall| : "datedeleted" => undef,
  448. |12878|3243:3604|reinstall| : "deleted" => 0,
  449. |12878|3243:3604|reinstall| : "id" => 174,
  450. |12878|3243:3604|reinstall| : "image" => {
  451. |12878|3243:3604|reinstall| : "IDENTITY" => "/etc/vcl/vcl.key",
  452. |12878|3243:3604|reinstall| : "OS" => {
  453. |12878|3243:3604|reinstall| : "OStype" => {
  454. |12878|3243:3604|reinstall| : "id" => 2,
  455. |12878|3243:3604|reinstall| : "name" => "linux"
  456. |12878|3243:3604|reinstall| : },
  457. |12878|3243:3604|reinstall| : "id" => 36,
  458. |12878|3243:3604|reinstall| : "installtype" => "vmware",
  459. |12878|3243:3604|reinstall| : "module" => {
  460. |12878|3243:3604|reinstall| : "description" => "",
  461. |12878|3243:3604|reinstall| : "id" => 5,
  462. |12878|3243:3604|reinstall| : "name" => "os_linux",
  463. |12878|3243:3604|reinstall| : "perlpackage" => "VCL::Module::OS::Linux",
  464. |12878|3243:3604|reinstall| : "prettyname" => "Linux OS Module"
  465. |12878|3243:3604|reinstall| : },
  466. |12878|3243:3604|reinstall| : "moduleid" => 5,
  467. |12878|3243:3604|reinstall| : "name" => "vmwarelinux",
  468. |12878|3243:3604|reinstall| : "prettyname" => "Generic Linux (VMware)",
  469. |12878|3243:3604|reinstall| : "sourcepath" => "vmware_images",
  470. |12878|3243:3604|reinstall| : "type" => "linux"
  471. |12878|3243:3604|reinstall| : },
  472. |12878|3243:3604|reinstall| : "OSid" => 36,
  473. |12878|3243:3604|reinstall| : "architecture" => "x86_64",
  474. |12878|3243:3604|reinstall| : "basedoffrevisionid" => 127,
  475. |12878|3243:3604|reinstall| : "deleted" => 0,
  476. |12878|3243:3604|reinstall| : "description" => "Test Image 2",
  477. |12878|3243:3604|reinstall| : "forcheckout" => 1,
  478. |12878|3243:3604|reinstall| : "id" => 16,
  479. |12878|3243:3604|reinstall| : "imagemeta" => {
  480. |12878|3243:3604|reinstall| : "architecture" => "",
  481. |12878|3243:3604|reinstall| : "checkuser" => 0,
  482. |12878|3243:3604|reinstall| : "id" => 16,
  483. |12878|3243:3604|reinstall| : "postoption" => "",
  484. |12878|3243:3604|reinstall| : "rootaccess" => 0,
  485. |12878|3243:3604|reinstall| : "subimages" => 1,
  486. |12878|3243:3604|reinstall| : "sysprep" => 0,
  487. |12878|3243:3604|reinstall| : "usergroupid" => ""
  488. |12878|3243:3604|reinstall| : },
  489. |12878|3243:3604|reinstall| : "imagemetaid" => 16,
  490. |12878|3243:3604|reinstall| : "imagetype" => {
  491. |12878|3243:3604|reinstall| : "id" => 7,
  492. |12878|3243:3604|reinstall| : "name" => "qcow2"
  493. |12878|3243:3604|reinstall| : },
  494. |12878|3243:3604|reinstall| : "imagetypeid" => 7,
  495. |12878|3243:3604|reinstall| : "lastupdate" => "2013-05-27 13:17:23",
  496. |12878|3243:3604|reinstall| : "maxconcurrent" => undef,
  497. |12878|3243:3604|reinstall| : "maxinitialtime" => 0,
  498. |12878|3243:3604|reinstall| : "minnetwork" => 10,
  499. |12878|3243:3604|reinstall| : "minprocnumber" => 2,
  500. |12878|3243:3604|reinstall| : "minprocspeed" => 500,
  501. |12878|3243:3604|reinstall| : "minram" => 8191,
  502. |12878|3243:3604|reinstall| : "name" => "linux-TestImage2-16-v2",
  503. |12878|3243:3604|reinstall| : "owner" => {
  504. |12878|3243:3604|reinstall| : "IMid" => "",
  505. |12878|3243:3604|reinstall| : "IMtype" => {
  506. |12878|3243:3604|reinstall| : "name" => "none"
  507. |12878|3243:3604|reinstall| : },
  508. |12878|3243:3604|reinstall| : "IMtypeid" => 1,
  509. |12878|3243:3604|reinstall| : "STANDALONE" => 1,
  510. |12878|3243:3604|reinstall| : "adminlevel" => {
  511. |12878|3243:3604|reinstall| : "name" => "developer"
  512. |12878|3243:3604|reinstall| : },
  513. |12878|3243:3604|reinstall| : "adminlevelid" => 3,
  514. |12878|3243:3604|reinstall| : "affiliation" => {
  515. |12878|3243:3604|reinstall| : "dataUpdateText" => "",
  516. |12878|3243:3604|reinstall| : "helpaddress" => "nobody\@example.com",
  517. |12878|3243:3604|reinstall| : "name" => "Local",
  518. |12878|3243:3604|reinstall| : "shibname" => undef,
  519. |12878|3243:3604|reinstall| : "shibonly" => 0,
  520. |12878|3243:3604|reinstall| : "sitewwwaddress" => "http://cwiki.apache.org/VCL"
  521. |12878|3243:3604|reinstall| : },
  522. |12878|3243:3604|reinstall| : "affiliationid" => 1,
  523. |12878|3243:3604|reinstall| : "audiomode" => "local",
  524. |12878|3243:3604|reinstall| : "bpp" => 16,
  525. |12878|3243:3604|reinstall| : "email" => "vcladmin\@sharcnet.ca",
  526. |12878|3243:3604|reinstall| : "emailnotices" => 0,
  527. |12878|3243:3604|reinstall| : "firstname" => "vcl",
  528. |12878|3243:3604|reinstall| : "height" => 768,
  529. |12878|3243:3604|reinstall| : "id" => 1,
  530. |12878|3243:3604|reinstall| : "lastname" => "admin",
  531. |12878|3243:3604|reinstall| : "lastupdated" => "2007-05-17 09:58:39",
  532. |12878|3243:3604|reinstall| : "localauth" => {
  533. |12878|3243:3604|reinstall| : "lastupdated" => "2007-05-17 09:56:01",
  534. |12878|3243:3604|reinstall| : "lockedout" => 0,
  535. |12878|3243:3604|reinstall| : "passhash" => "e3d58029c003867cba100e43137f8814f3fe5de2",
  536. |12878|3243:3604|reinstall| : "salt" => "QwkCHLpY"
  537. |12878|3243:3604|reinstall| : },
  538. |12878|3243:3604|reinstall| : "mapdrives" => 1,
  539. |12878|3243:3604|reinstall| : "mapprinters" => 1,
  540. |12878|3243:3604|reinstall| : "mapserial" => 1,
  541. |12878|3243:3604|reinstall| : "preferredname" => "vcl",
  542. |12878|3243:3604|reinstall| : "showallgroups" => 1,
  543. |12878|3243:3604|reinstall| : "uid" => 1000,
  544. |12878|3243:3604|reinstall| : "unityid" => "admin",
  545. |12878|3243:3604|reinstall| : "validated" => 1,
  546. |12878|3243:3604|reinstall| : "width" => 1024
  547. |12878|3243:3604|reinstall| : },
  548. |12878|3243:3604|reinstall| : "ownerid" => 1,
  549. |12878|3243:3604|reinstall| : "platform" => {
  550. |12878|3243:3604|reinstall| : "id" => 1,
  551. |12878|3243:3604|reinstall| : "name" => "x86_64"
  552. |12878|3243:3604|reinstall| : },
  553. |12878|3243:3604|reinstall| : "platformid" => 1,
  554. |12878|3243:3604|reinstall| : "prettyname" => "Test Image 2",
  555. |12878|3243:3604|reinstall| : "project" => "vcl",
  556. |12878|3243:3604|reinstall| : "reloadtime" => 20,
  557. |12878|3243:3604|reinstall| : "size" => 3336,
  558. |12878|3243:3604|reinstall| : "test" => 1,
  559. |12878|3243:3604|reinstall| : "usage" => ""
  560. |12878|3243:3604|reinstall| : },
  561. |12878|3243:3604|reinstall| : "imageid" => 16,
  562. |12878|3243:3604|reinstall| : "imagename" => "linux-TestImage2-16-v2",
  563. |12878|3243:3604|reinstall| : "production" => 1,
  564. |12878|3243:3604|reinstall| : "revision" => 2,
  565. |12878|3243:3604|reinstall| : "user" => {
  566. |12878|3243:3604|reinstall| : "IMid" => "",
  567. |12878|3243:3604|reinstall| : "IMtype" => {
  568. |12878|3243:3604|reinstall| : "name" => "none"
  569. |12878|3243:3604|reinstall| : },
  570. |12878|3243:3604|reinstall| : "IMtypeid" => 1,
  571. |12878|3243:3604|reinstall| : "STANDALONE" => 1,
  572. |12878|3243:3604|reinstall| : "adminlevel" => {
  573. |12878|3243:3604|reinstall| : "name" => "developer"
  574. |12878|3243:3604|reinstall| : },
  575. |12878|3243:3604|reinstall| : "adminlevelid" => 3,
  576. |12878|3243:3604|reinstall| : "affiliation" => {
  577. |12878|3243:3604|reinstall| : "dataUpdateText" => "",
  578. |12878|3243:3604|reinstall| : "helpaddress" => "nobody\@example.com",
  579. |12878|3243:3604|reinstall| : "name" => "Local",
  580. |12878|3243:3604|reinstall| : "shibname" => undef,
  581. |12878|3243:3604|reinstall| : "shibonly" => 0,
  582. |12878|3243:3604|reinstall| : "sitewwwaddress" => "http://cwiki.apache.org/VCL"
  583. |12878|3243:3604|reinstall| : },
  584. |12878|3243:3604|reinstall| : "affiliationid" => 1,
  585. |12878|3243:3604|reinstall| : "audiomode" => "local",
  586. |12878|3243:3604|reinstall| : "bpp" => 16,
  587. |12878|3243:3604|reinstall| : "email" => "vcladmin\@sharcnet.ca",
  588. |12878|3243:3604|reinstall| : "emailnotices" => 0,
  589. |12878|3243:3604|reinstall| : "firstname" => "vcl",
  590. |12878|3243:3604|reinstall| : "height" => 768,
  591. |12878|3243:3604|reinstall| : "id" => 1,
  592. |12878|3243:3604|reinstall| : "lastname" => "admin",
  593. |12878|3243:3604|reinstall| : "lastupdated" => "2007-05-17 09:58:39",
  594. |12878|3243:3604|reinstall| : "localauth" => {
  595. |12878|3243:3604|reinstall| : "lastupdated" => "2007-05-17 09:56:01",
  596. |12878|3243:3604|reinstall| : "lockedout" => 0,
  597. |12878|3243:3604|reinstall| : "passhash" => "e3d58029c003867cba100e43137f8814f3fe5de2",
  598. |12878|3243:3604|reinstall| : "salt" => "QwkCHLpY"
  599. |12878|3243:3604|reinstall| : },
  600. |12878|3243:3604|reinstall| : "mapdrives" => 1,
  601. |12878|3243:3604|reinstall| : "mapprinters" => 1,
  602. |12878|3243:3604|reinstall| : "mapserial" => 1,
  603. |12878|3243:3604|reinstall| : "preferredname" => "vcl",
  604. |12878|3243:3604|reinstall| : "showallgroups" => 1,
  605. |12878|3243:3604|reinstall| : "uid" => 1000,
  606. |12878|3243:3604|reinstall| : "unityid" => "admin",
  607. |12878|3243:3604|reinstall| : "validated" => 1,
  608. |12878|3243:3604|reinstall| : "width" => 1024
  609. |12878|3243:3604|reinstall| : },
  610. |12878|3243:3604|reinstall| : "userid" => 1
  611. |12878|3243:3604|reinstall| : }
  612. 2013-05-28 13:55:32|12878|3243:3604|reinstall|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{3604}{computer}{currentimage}{vcld_post_load}, data identifier: computer_currentimage_vcld_post_load, data:
  613. |12878|3243:3604|reinstall| : "success (Tue May 28 13:51:09 2013)"
  614. 2013-05-28 13:55:33|12878|3243:3604|reinstall|Provisioning.pm:node_status(198)|currentimage.txt image 174 (linux-TestImage2-16-v2) matches requested image 174 (linux-TestImage2-16-v2) on ecco-vm-x10
  615. 2013-05-28 13:55:33|12878|3243:3604|reinstall|Provisioning.pm:node_status(208)|OS module post_load tasks have been completed on ecco-vm-x10
  616. 2013-05-28 13:55:33|12878|3243:3604|reinstall|Provisioning.pm:node_status(216)|status of ecco-vm-x10: READY
  617. 2013-05-28 13:55:33|12878|3243:3604|reinstall|new.pm:reload_image(545)|node_status returned a hash reference
  618. 2013-05-28 13:55:33|12878|3243:3604|reinstall|new.pm:reload_image(550)|node_status hash reference contains key {status}=READY
  619. 2013-05-28 13:55:33|12878|3243:3604|reinstall|new.pm:reload_image(595)|request state is reinstall, node will be reloaded regardless of status
  620. 2013-05-28 13:55:33|12878|3243:3604|reinstall|new.pm:reload_image(627)|node status is reload, ecco-vm-x10 will be reloaded
  621. 2013-05-28 13:55:33|12878|3243:3604|reinstall|utils.pm:insertloadlog(3665)|inserted computer=18, loadimageblade, ecco-vm-x10 must be reloaded with linux-TestImage2-16-v2
  622. 2013-05-28 13:55:33|12878|3243:3604|reinstall|new.pm:reload_image(641)|calling VCL::Module::Provisioning::libvirt->does_image_exist()
  623. 2013-05-28 13:55:33|12878|3243:3604|reinstall|OS.pm:find_files(2917)|attempting to find files on ecco-vmhost-x3, base directory path: '/images/vm/', pattern: linux-TestImage2-16-v2.*, command: /usr/bin/find "/images/vm/" -type f -iname "linux-TestImage2-16-v2.*"
  624. 2013-05-28 13:55:33|12889|3243:3605|reinstall|OS.pm:execute(1994)|executed command: 'cat ~/currentimage.txt', exit status: 0, output:
  625. |12889|3243:3605|reinstall| linux-TestImage2-16-v2
  626. |12889|3243:3605|reinstall| id=16
  627. |12889|3243:3605|reinstall| prettyname=Test Image 2
  628. |12889|3243:3605|reinstall| imagerevision_id=174
  629. |12889|3243:3605|reinstall| imagerevision_datecreated=2013-05-27 13:17:23
  630. |12889|3243:3605|reinstall| computer_id=10
  631. |12889|3243:3605|reinstall| computer_hostname=ecco-vm-x5
  632. |12889|3243:3605|reinstall| vcld_post_load=success (Tue May 28 13:51:14 2013)
  633. 2013-05-28 13:55:33|12889|3243:3605|reinstall|OS.pm:get_currentimage_txt_contents(276)|retrieved currentimage.txt contents from ecco-vm-x26:
  634. |12889|3243:3605|reinstall| linux-TestImage2-16-v2
  635. |12889|3243:3605|reinstall| id=16
  636. |12889|3243:3605|reinstall| prettyname=Test Image 2
  637. |12889|3243:3605|reinstall| imagerevision_id=174
  638. |12889|3243:3605|reinstall| imagerevision_datecreated=2013-05-27 13:17:23
  639. |12889|3243:3605|reinstall| computer_id=10
  640. |12889|3243:3605|reinstall| computer_hostname=ecco-vm-x5
  641. |12889|3243:3605|reinstall| vcld_post_load=success (Tue May 28 13:51:14 2013)
  642. 2013-05-28 13:55:33|12889|3243:3605|reinstall|OS.pm:get_current_image_info(331)|retrieved currentimage.txt contents from ecco-vm-x26
  643. 2013-05-28 13:55:33|12889|3243:3605|reinstall|OS.pm:get_current_image_info(341)|user selected content of image currently loaded on ecco-vm-x26: linux-TestImage2-16-v2
  644. 2013-05-28 13:55:33|12889|3243:3605|reinstall|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{3605}{computer}{currentimage}, data identifier: computer_currentimage_data, data:
  645. |12889|3243:3605|reinstall| : {
  646. |12889|3243:3605|reinstall| : "IDENTITY" => "/etc/vcl/vcl.key",
  647. |12889|3243:3605|reinstall| : "OS" => {
  648. |12889|3243:3605|reinstall| : "OStype" => {
  649. |12889|3243:3605|reinstall| : "id" => 2,
  650. |12889|3243:3605|reinstall| : "name" => "linux"
  651. |12889|3243:3605|reinstall| : },
  652. |12889|3243:3605|reinstall| : "id" => 36,
  653. |12889|3243:3605|reinstall| : "installtype" => "vmware",
  654. |12889|3243:3605|reinstall| : "module" => {
  655. |12889|3243:3605|reinstall| : "description" => "",
  656. |12889|3243:3605|reinstall| : "id" => 5,
  657. |12889|3243:3605|reinstall| : "name" => "os_linux",
  658. |12889|3243:3605|reinstall| : "perlpackage" => "VCL::Module::OS::Linux",
  659. |12889|3243:3605|reinstall| : "prettyname" => "Linux OS Module"
  660. |12889|3243:3605|reinstall| : },
  661. |12889|3243:3605|reinstall| : "moduleid" => 5,
  662. |12889|3243:3605|reinstall| : "name" => "vmwarelinux",
  663. |12889|3243:3605|reinstall| : "prettyname" => "Generic Linux (VMware)",
  664. |12889|3243:3605|reinstall| : "sourcepath" => "vmware_images",
  665. |12889|3243:3605|reinstall| : "type" => "linux"
  666. |12889|3243:3605|reinstall| : },
  667. |12889|3243:3605|reinstall| : "OSid" => 36,
  668. |12889|3243:3605|reinstall| : "architecture" => "x86_64",
  669. |12889|3243:3605|reinstall| : "basedoffrevisionid" => 127,
  670. |12889|3243:3605|reinstall| : "deleted" => 0,
  671. |12889|3243:3605|reinstall| : "description" => "Test Image 2",
  672. |12889|3243:3605|reinstall| : "forcheckout" => 1,
  673. |12889|3243:3605|reinstall| : "id" => 16,
  674. |12889|3243:3605|reinstall| : "imagemeta" => {
  675. |12889|3243:3605|reinstall| : "architecture" => "",
  676. |12889|3243:3605|reinstall| : "checkuser" => 0,
  677. |12889|3243:3605|reinstall| : "id" => 16,
  678. |12889|3243:3605|reinstall| : "postoption" => "",
  679. |12889|3243:3605|reinstall| : "rootaccess" => 0,
  680. |12889|3243:3605|reinstall| : "subimages" => 1,
  681. |12889|3243:3605|reinstall| : "sysprep" => 0,
  682. |12889|3243:3605|reinstall| : "usergroupid" => ""
  683. |12889|3243:3605|reinstall| : },
  684. |12889|3243:3605|reinstall| : "imagemetaid" => 16,
  685. |12889|3243:3605|reinstall| : "imagetype" => {
  686. |12889|3243:3605|reinstall| : "id" => 7,
  687. |12889|3243:3605|reinstall| : "name" => "qcow2"
  688. |12889|3243:3605|reinstall| : },
  689. |12889|3243:3605|reinstall| : "imagetypeid" => 7,
  690. |12889|3243:3605|reinstall| : "lastupdate" => "2013-05-27 13:17:23",
  691. |12889|3243:3605|reinstall| : "maxconcurrent" => undef,
  692. |12889|3243:3605|reinstall| : "maxinitialtime" => 0,
  693. |12889|3243:3605|reinstall| : "minnetwork" => 10,
  694. |12889|3243:3605|reinstall| : "minprocnumber" => 2,
  695. |12889|3243:3605|reinstall| : "minprocspeed" => 500,
  696. |12889|3243:3605|reinstall| : "minram" => 8191,
  697. |12889|3243:3605|reinstall| : "name" => "linux-TestImage2-16-v2",
  698. |12889|3243:3605|reinstall| : "owner" => {
  699. |12889|3243:3605|reinstall| : "IMid" => "",
  700. |12889|3243:3605|reinstall| : "IMtype" => {
  701. |12889|3243:3605|reinstall| : "name" => "none"
  702. |12889|3243:3605|reinstall| : },
  703. |12889|3243:3605|reinstall| : "IMtypeid" => 1,
  704. |12889|3243:3605|reinstall| : "STANDALONE" => 1,
  705. |12889|3243:3605|reinstall| : "adminlevel" => {
  706. |12889|3243:3605|reinstall| : "name" => "developer"
  707. |12889|3243:3605|reinstall| : },
  708. |12889|3243:3605|reinstall| : "adminlevelid" => 3,
  709. |12889|3243:3605|reinstall| : "affiliation" => {
  710. |12889|3243:3605|reinstall| : "dataUpdateText" => "",
  711. |12889|3243:3605|reinstall| : "helpaddress" => "nobody\@example.com",
  712. |12889|3243:3605|reinstall| : "name" => "Local",
  713. |12889|3243:3605|reinstall| : "shibname" => undef,
  714. |12889|3243:3605|reinstall| : "shibonly" => 0,
  715. |12889|3243:3605|reinstall| : "sitewwwaddress" => "http://cwiki.apache.org/VCL"
  716. |12889|3243:3605|reinstall| : },
  717. |12889|3243:3605|reinstall| : "affiliationid" => 1,
  718. |12889|3243:3605|reinstall| : "audiomode" => "local",
  719. |12889|3243:3605|reinstall| : "bpp" => 16,
  720. |12889|3243:3605|reinstall| : "email" => "vcladmin\@sharcnet.ca",
  721. |12889|3243:3605|reinstall| : "emailnotices" => 0,
  722. |12889|3243:3605|reinstall| : "firstname" => "vcl",
  723. |12889|3243:3605|reinstall| : "height" => 768,
  724. |12889|3243:3605|reinstall| : "id" => 1,
  725. |12889|3243:3605|reinstall| : "lastname" => "admin",
  726. |12889|3243:3605|reinstall| : "lastupdated" => "2007-05-17 09:58:39",
  727. |12889|3243:3605|reinstall| : "localauth" => {
  728. |12889|3243:3605|reinstall| : "lastupdated" => "2007-05-17 09:56:01",
  729. |12889|3243:3605|reinstall| : "lockedout" => 0,
  730. |12889|3243:3605|reinstall| : "passhash" => "e3d58029c003867cba100e43137f8814f3fe5de2",
  731. |12889|3243:3605|reinstall| : "salt" => "QwkCHLpY"
  732. |12889|3243:3605|reinstall| : },
  733. |12889|3243:3605|reinstall| : "mapdrives" => 1,
  734. |12889|3243:3605|reinstall| : "mapprinters" => 1,
  735. |12889|3243:3605|reinstall| : "mapserial" => 1,
  736. |12889|3243:3605|reinstall| : "preferredname" => "vcl",
  737. |12889|3243:3605|reinstall| : "showallgroups" => 1,
  738. |12889|3243:3605|reinstall| : "uid" => 1000,
  739. |12889|3243:3605|reinstall| : "unityid" => "admin",
  740. |12889|3243:3605|reinstall| : "validated" => 1,
  741. |12889|3243:3605|reinstall| : "width" => 1024
  742. |12889|3243:3605|reinstall| : },
  743. |12889|3243:3605|reinstall| : "ownerid" => 1,
  744. |12889|3243:3605|reinstall| : "platform" => {
  745. |12889|3243:3605|reinstall| : "id" => 1,
  746. |12889|3243:3605|reinstall| : "name" => "x86_64"
  747. |12889|3243:3605|reinstall| : },
  748. |12889|3243:3605|reinstall| : "platformid" => 1,
  749. |12889|3243:3605|reinstall| : "prettyname" => "Test Image 2",
  750. |12889|3243:3605|reinstall| : "project" => "vcl",
  751. |12889|3243:3605|reinstall| : "reloadtime" => 20,
  752. |12889|3243:3605|reinstall| : "size" => 3336,
  753. |12889|3243:3605|reinstall| : "test" => 1,
  754. |12889|3243:3605|reinstall| : "usage" => ""
  755. |12889|3243:3605|reinstall| : }
  756. 2013-05-28 13:55:33|12889|3243:3605|reinstall|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{3605}{computer}{currentimagerevision}, data identifier: computer_currentimagerevision_data, data:
  757. |12889|3243:3605|reinstall| : {
  758. |12889|3243:3605|reinstall| : "autocaptured" => 0,
  759. |12889|3243:3605|reinstall| : "comments" => "Fix for using the 'mark' module on a child node of a cluster.",
  760. |12889|3243:3605|reinstall| : "datecreated" => "2013-05-27 13:17:23",
  761. |12889|3243:3605|reinstall| : "datedeleted" => undef,
  762. |12889|3243:3605|reinstall| : "deleted" => 0,
  763. |12889|3243:3605|reinstall| : "id" => 174,
  764. |12889|3243:3605|reinstall| : "image" => {
  765. |12889|3243:3605|reinstall| : "IDENTITY" => "/etc/vcl/vcl.key",
  766. |12889|3243:3605|reinstall| : "OS" => {
  767. |12889|3243:3605|reinstall| : "OStype" => {
  768. |12889|3243:3605|reinstall| : "id" => 2,
  769. |12889|3243:3605|reinstall| : "name" => "linux"
  770. |12889|3243:3605|reinstall| : },
  771. |12889|3243:3605|reinstall| : "id" => 36,
  772. |12889|3243:3605|reinstall| : "installtype" => "vmware",
  773. |12889|3243:3605|reinstall| : "module" => {
  774. |12889|3243:3605|reinstall| : "description" => "",
  775. |12889|3243:3605|reinstall| : "id" => 5,
  776. |12889|3243:3605|reinstall| : "name" => "os_linux",
  777. |12889|3243:3605|reinstall| : "perlpackage" => "VCL::Module::OS::Linux",
  778. |12889|3243:3605|reinstall| : "prettyname" => "Linux OS Module"
  779. |12889|3243:3605|reinstall| : },
  780. |12889|3243:3605|reinstall| : "moduleid" => 5,
  781. |12889|3243:3605|reinstall| : "name" => "vmwarelinux",
  782. |12889|3243:3605|reinstall| : "prettyname" => "Generic Linux (VMware)",
  783. |12889|3243:3605|reinstall| : "sourcepath" => "vmware_images",
  784. |12889|3243:3605|reinstall| : "type" => "linux"
  785. |12889|3243:3605|reinstall| : },
  786. |12889|3243:3605|reinstall| : "OSid" => 36,
  787. |12889|3243:3605|reinstall| : "architecture" => "x86_64",
  788. |12889|3243:3605|reinstall| : "basedoffrevisionid" => 127,
  789. |12889|3243:3605|reinstall| : "deleted" => 0,
  790. |12889|3243:3605|reinstall| : "description" => "Test Image 2",
  791. |12889|3243:3605|reinstall| : "forcheckout" => 1,
  792. |12889|3243:3605|reinstall| : "id" => 16,
  793. |12889|3243:3605|reinstall| : "imagemeta" => {
  794. |12889|3243:3605|reinstall| : "architecture" => "",
  795. |12889|3243:3605|reinstall| : "checkuser" => 0,
  796. |12889|3243:3605|reinstall| : "id" => 16,
  797. |12889|3243:3605|reinstall| : "postoption" => "",
  798. |12889|3243:3605|reinstall| : "rootaccess" => 0,
  799. |12889|3243:3605|reinstall| : "subimages" => 1,
  800. |12889|3243:3605|reinstall| : "sysprep" => 0,
  801. |12889|3243:3605|reinstall| : "usergroupid" => ""
  802. |12889|3243:3605|reinstall| : },
  803. |12889|3243:3605|reinstall| : "imagemetaid" => 16,
  804. |12889|3243:3605|reinstall| : "imagetype" => {
  805. |12889|3243:3605|reinstall| : "id" => 7,
  806. |12889|3243:3605|reinstall| : "name" => "qcow2"
  807. |12889|3243:3605|reinstall| : },
  808. |12889|3243:3605|reinstall| : "imagetypeid" => 7,
  809. |12889|3243:3605|reinstall| : "lastupdate" => "2013-05-27 13:17:23",
  810. |12889|3243:3605|reinstall| : "maxconcurrent" => undef,
  811. |12889|3243:3605|reinstall| : "maxinitialtime" => 0,
  812. |12889|3243:3605|reinstall| : "minnetwork" => 10,
  813. |12889|3243:3605|reinstall| : "minprocnumber" => 2,
  814. |12889|3243:3605|reinstall| : "minprocspeed" => 500,
  815. |12889|3243:3605|reinstall| : "minram" => 8191,
  816. |12889|3243:3605|reinstall| : "name" => "linux-TestImage2-16-v2",
  817. |12889|3243:3605|reinstall| : "owner" => {
  818. |12889|3243:3605|reinstall| : "IMid" => "",
  819. |12889|3243:3605|reinstall| : "IMtype" => {
  820. |12889|3243:3605|reinstall| : "name" => "none"
  821. |12889|3243:3605|reinstall| : },
  822. |12889|3243:3605|reinstall| : "IMtypeid" => 1,
  823. |12889|3243:3605|reinstall| : "STANDALONE" => 1,
  824. |12889|3243:3605|reinstall| : "adminlevel" => {
  825. |12889|3243:3605|reinstall| : "name" => "developer"
  826. |12889|3243:3605|reinstall| : },
  827. |12889|3243:3605|reinstall| : "adminlevelid" => 3,
  828. |12889|3243:3605|reinstall| : "affiliation" => {
  829. |12889|3243:3605|reinstall| : "dataUpdateText" => "",
  830. |12889|3243:3605|reinstall| : "helpaddress" => "nobody\@example.com",
  831. |12889|3243:3605|reinstall| : "name" => "Local",
  832. |12889|3243:3605|reinstall| : "shibname" => undef,
  833. |12889|3243:3605|reinstall| : "shibonly" => 0,
  834. |12889|3243:3605|reinstall| : "sitewwwaddress" => "http://cwiki.apache.org/VCL"
  835. |12889|3243:3605|reinstall| : },
  836. |12889|3243:3605|reinstall| : "affiliationid" => 1,
  837. |12889|3243:3605|reinstall| : "audiomode" => "local",
  838. |12889|3243:3605|reinstall| : "bpp" => 16,
  839. |12889|3243:3605|reinstall| : "email" => "vcladmin\@sharcnet.ca",
  840. |12889|3243:3605|reinstall| : "emailnotices" => 0,
  841. |12889|3243:3605|reinstall| : "firstname" => "vcl",
  842. |12889|3243:3605|reinstall| : "height" => 768,
  843. |12889|3243:3605|reinstall| : "id" => 1,
  844. |12889|3243:3605|reinstall| : "lastname" => "admin",
  845. |12889|3243:3605|reinstall| : "lastupdated" => "2007-05-17 09:58:39",
  846. |12889|3243:3605|reinstall| : "localauth" => {
  847. |12889|3243:3605|reinstall| : "lastupdated" => "2007-05-17 09:56:01",
  848. |12889|3243:3605|reinstall| : "lockedout" => 0,
  849. |12889|3243:3605|reinstall| : "passhash" => "e3d58029c003867cba100e43137f8814f3fe5de2",
  850. |12889|3243:3605|reinstall| : "salt" => "QwkCHLpY"
  851. |12889|3243:3605|reinstall| : },
  852. |12889|3243:3605|reinstall| : "mapdrives" => 1,
  853. |12889|3243:3605|reinstall| : "mapprinters" => 1,
  854. |12889|3243:3605|reinstall| : "mapserial" => 1,
  855. |12889|3243:3605|reinstall| : "preferredname" => "vcl",
  856. |12889|3243:3605|reinstall| : "showallgroups" => 1,
  857. |12889|3243:3605|reinstall| : "uid" => 1000,
  858. |12889|3243:3605|reinstall| : "unityid" => "admin",
  859. |12889|3243:3605|reinstall| : "validated" => 1,
  860. |12889|3243:3605|reinstall| : "width" => 1024
  861. |12889|3243:3605|reinstall| : },
  862. |12889|3243:3605|reinstall| : "ownerid" => 1,
  863. |12889|3243:3605|reinstall| : "platform" => {
  864. |12889|3243:3605|reinstall| : "id" => 1,
  865. |12889|3243:3605|reinstall| : "name" => "x86_64"
  866. |12889|3243:3605|reinstall| : },
  867. |12889|3243:3605|reinstall| : "platformid" => 1,
  868. |12889|3243:3605|reinstall| : "prettyname" => "Test Image 2",
  869. |12889|3243:3605|reinstall| : "project" => "vcl",
  870. |12889|3243:3605|reinstall| : "reloadtime" => 20,
  871. |12889|3243:3605|reinstall| : "size" => 3336,
  872. |12889|3243:3605|reinstall| : "test" => 1,
  873. |12889|3243:3605|reinstall| : "usage" => ""
  874. |12889|3243:3605|reinstall| : },
  875. |12889|3243:3605|reinstall| : "imageid" => 16,
  876. |12889|3243:3605|reinstall| : "imagename" => "linux-TestImage2-16-v2",
  877. |12889|3243:3605|reinstall| : "production" => 1,
  878. |12889|3243:3605|reinstall| : "revision" => 2,
  879. |12889|3243:3605|reinstall| : "user" => {
  880. |12889|3243:3605|reinstall| : "IMid" => "",
  881. |12889|3243:3605|reinstall| : "IMtype" => {
  882. |12889|3243:3605|reinstall| : "name" => "none"
  883. |12889|3243:3605|reinstall| : },
  884. |12889|3243:3605|reinstall| : "IMtypeid" => 1,
  885. |12889|3243:3605|reinstall| : "STANDALONE" => 1,
  886. |12889|3243:3605|reinstall| : "adminlevel" => {
  887. |12889|3243:3605|reinstall| : "name" => "developer"
  888. |12889|3243:3605|reinstall| : },
  889. |12889|3243:3605|reinstall| : "adminlevelid" => 3,
  890. |12889|3243:3605|reinstall| : "affiliation" => {
  891. |12889|3243:3605|reinstall| : "dataUpdateText" => "",
  892. |12889|3243:3605|reinstall| : "helpaddress" => "nobody\@example.com",
  893. |12889|3243:3605|reinstall| : "name" => "Local",
  894. |12889|3243:3605|reinstall| : "shibname" => undef,
  895. |12889|3243:3605|reinstall| : "shibonly" => 0,
  896. |12889|3243:3605|reinstall| : "sitewwwaddress" => "http://cwiki.apache.org/VCL"
  897. |12889|3243:3605|reinstall| : },
  898. |12889|3243:3605|reinstall| : "affiliationid" => 1,
  899. |12889|3243:3605|reinstall| : "audiomode" => "local",
  900. |12889|3243:3605|reinstall| : "bpp" => 16,
  901. |12889|3243:3605|reinstall| : "email" => "vcladmin\@sharcnet.ca",
  902. |12889|3243:3605|reinstall| : "emailnotices" => 0,
  903. |12889|3243:3605|reinstall| : "firstname" => "vcl",
  904. |12889|3243:3605|reinstall| : "height" => 768,
  905. |12889|3243:3605|reinstall| : "id" => 1,
  906. |12889|3243:3605|reinstall| : "lastname" => "admin",
  907. |12889|3243:3605|reinstall| : "lastupdated" => "2007-05-17 09:58:39",
  908. |12889|3243:3605|reinstall| : "localauth" => {
  909. |12889|3243:3605|reinstall| : "lastupdated" => "2007-05-17 09:56:01",
  910. |12889|3243:3605|reinstall| : "lockedout" => 0,
  911. |12889|3243:3605|reinstall| : "passhash" => "e3d58029c003867cba100e43137f8814f3fe5de2",
  912. |12889|3243:3605|reinstall| : "salt" => "QwkCHLpY"
  913. |12889|3243:3605|reinstall| : },
  914. |12889|3243:3605|reinstall| : "mapdrives" => 1,
  915. |12889|3243:3605|reinstall| : "mapprinters" => 1,
  916. |12889|3243:3605|reinstall| : "mapserial" => 1,
  917. |12889|3243:3605|reinstall| : "preferredname" => "vcl",
  918. |12889|3243:3605|reinstall| : "showallgroups" => 1,
  919. |12889|3243:3605|reinstall| : "uid" => 1000,
  920. |12889|3243:3605|reinstall| : "unityid" => "admin",
  921. |12889|3243:3605|reinstall| : "validated" => 1,
  922. |12889|3243:3605|reinstall| : "width" => 1024
  923. |12889|3243:3605|reinstall| : },
  924. |12889|3243:3605|reinstall| : "userid" => 1
  925. |12889|3243:3605|reinstall| : }
  926. 2013-05-28 13:55:33|12889|3243:3605|reinstall|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{3605}{computer}{currentimage}{vcld_post_load}, data identifier: computer_currentimage_vcld_post_load, data:
  927. |12889|3243:3605|reinstall| : "success (Tue May 28 13:51:14 2013)"
  928. 2013-05-28 13:55:33|12889|3243:3605|reinstall|Provisioning.pm:node_status(198)|currentimage.txt image 174 (linux-TestImage2-16-v2) matches requested image 174 (linux-TestImage2-16-v2) on ecco-vm-x26
  929. 2013-05-28 13:55:33|12889|3243:3605|reinstall|Provisioning.pm:node_status(208)|OS module post_load tasks have been completed on ecco-vm-x26
  930. 2013-05-28 13:55:33|12889|3243:3605|reinstall|Provisioning.pm:node_status(216)|status of ecco-vm-x26: READY
  931. 2013-05-28 13:55:33|12889|3243:3605|reinstall|new.pm:reload_image(545)|node_status returned a hash reference
  932. 2013-05-28 13:55:33|12889|3243:3605|reinstall|new.pm:reload_image(550)|node_status hash reference contains key {status}=READY
  933. 2013-05-28 13:55:33|12889|3243:3605|reinstall|new.pm:reload_image(595)|request state is reinstall, node will be reloaded regardless of status
  934. 2013-05-28 13:55:33|12889|3243:3605|reinstall|new.pm:reload_image(627)|node status is reload, ecco-vm-x26 will be reloaded
  935. 2013-05-28 13:55:33|12889|3243:3605|reinstall|utils.pm:insertloadlog(3665)|inserted computer=40, loadimageblade, ecco-vm-x26 must be reloaded with linux-TestImage2-16-v2
  936. 2013-05-28 13:55:33|12889|3243:3605|reinstall|new.pm:reload_image(641)|calling VCL::Module::Provisioning::libvirt->does_image_exist()
  937. 2013-05-28 13:55:33|12889|3243:3605|reinstall|OS.pm:find_files(2917)|attempting to find files on ecco-vmhost-x7, base directory path: '/images/vm/', pattern: linux-TestImage2-16-v2.*, command: /usr/bin/find "/images/vm/" -type f -iname "linux-TestImage2-16-v2.*"
  938. 2013-05-28 13:55:33|12878|3243:3604|reinstall|OS.pm:find_files(2945)|files found: 1, base directory: '/images/vm/', pattern: 'linux-TestImage2-16-v2.*'
  939. |12878|3243:3604|reinstall| command: '/usr/bin/find "/images/vm/" -type f -iname "linux-TestImage2-16-v2.*"'
  940. 2013-05-28 13:55:33|12878|3243:3604|reinstall|libvirt.pm:get_master_image_file_path(1166)|found master image file on ecco-vmhost-x3: /images/vm/linux-TestImage2-16-v2.qcow2
  941. 2013-05-28 13:55:33|12878|3243:3604|reinstall|Module.pm:new(203)|VCL::Module::Semaphore object created, address: 3e96e08
  942. 2013-05-28 13:55:33|12878|3243:3604|reinstall|Module.pm:code_loop_timeout(887)|attempting to open lockfile, maximum of 7200 seconds
  943. 2013-05-28 13:55:33|12878|3243:3604|reinstall|Semaphore.pm:open_lockfile(175)|opened and obtained an exclusive lock on file: /tmp/master-linux-TestImage2-16-v2.semaphore
  944. 2013-05-28 13:55:33|12878|3243:3604|reinstall|Semaphore.pm:open_lockfile(182)|wrote to file: /tmp/master-linux-TestImage2-16-v2.semaphore, contents:
  945. |12878|3243:3604|reinstall| '12878 vcld 3243:3604 reinstall ecco-vm-x10>ecco-vmhost-x3 linux-TestImage2-16-v2 nsasherr (cluster=parent)'
  946. 2013-05-28 13:55:33|12878|3243:3604|reinstall|Module.pm:code_loop_timeout(900)|attempting to open lockfile, code returned true
  947. 2013-05-28 13:55:33|12878|3243:3604|reinstall|Module.pm:get_semaphore(993)|created 'master:linux-TestImage2-16-v2' Semaphore object, memory address: 3e96e08
  948. 2013-05-28 13:55:33|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  949. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'stat /images/vm/linux\-TestImage2\-16\-v2\.qcow2' 2>&1
  950. 2013-05-28 13:55:33|12889|3243:3605|reinstall|OS.pm:find_files(2945)|files found: 1, base directory: '/images/vm/', pattern: 'linux-TestImage2-16-v2.*'
  951. |12889|3243:3605|reinstall| command: '/usr/bin/find "/images/vm/" -type f -iname "linux-TestImage2-16-v2.*"'
  952. 2013-05-28 13:55:33|12889|3243:3605|reinstall|libvirt.pm:get_master_image_file_path(1166)|found master image file on ecco-vmhost-x7: /images/vm/linux-TestImage2-16-v2.qcow2
  953. 2013-05-28 13:55:33|12889|3243:3605|reinstall|Module.pm:new(203)|VCL::Module::Semaphore object created, address: 3e722f8
  954. 2013-05-28 13:55:33|12889|3243:3605|reinstall|Module.pm:code_loop_timeout(887)|attempting to open lockfile, maximum of 7200 seconds
  955. 2013-05-28 13:55:33|12889|3243:3605|reinstall|Semaphore.pm:open_lockfile(188)|unable to obtain exclusive lock on file: /tmp/master-linux-TestImage2-16-v2.semaphore
  956. 2013-05-28 13:55:33|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  957. |12878|3243:3604|reinstall| File: `/images/vm/linux-TestImage2-16-v2.qcow2'
  958. |12878|3243:3604|reinstall| Size: 3498573824 Blocks: 6832920 IO Block: 65536 regular file
  959. |12878|3243:3604|reinstall| Device: 1dh/29d Inode: 122683637 Links: 2
  960. |12878|3243:3604|reinstall| Access: (0644/-rw-r--r--) Uid: ( 99/ nobody) Gid: ( 99/ nobody)
  961. |12878|3243:3604|reinstall| Access: 2013-05-27 14:00:22.909942374 -0400
  962. |12878|3243:3604|reinstall| Modify: 2013-05-27 13:19:17.866852434 -0400
  963. |12878|3243:3604|reinstall| Change: 2013-05-27 14:00:01.634693243 -0400
  964. 2013-05-28 13:55:33|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x3, returning (0, "File: `/images/vm/linux-TestIm...")
  965. 2013-05-28 13:55:33|12878|3243:3604|reinstall|Linux.pm:file_exists(1449)|'/images/vm/linux-TestImage2-16-v2.qcow2' exists on ecco-vmhost-x3, files: 1, directories: 0, links: 0
  966. 2013-05-28 13:55:33|12878|3243:3604|reinstall|libvirt.pm:does_image_exist(527)|linux-TestImage2-16-v2 image exists on ecco-vmhost-x3: /images/vm/linux-TestImage2-16-v2.qcow2
  967. 2013-05-28 13:55:33|12878|3243:3604|reinstall|new.pm:reload_image(644)|linux-TestImage2-16-v2 exists on this management node
  968. 2013-05-28 13:55:33|12878|3243:3604|reinstall|utils.pm:insertloadlog(3665)|inserted computer=18, doesimageexists, confirmed image exists
  969. 2013-05-28 13:55:33|12878|3243:3604|reinstall|Semaphore.pm:release_lockfile(280)|deleted file: /tmp/master-linux-TestImage2-16-v2.semaphore
  970. 2013-05-28 13:55:33|12878|3243:3604|reinstall|Semaphore.pm:DESTROY(377)|destroyed Semaphore object, memory address: 3e96e08
  971. 2013-05-28 13:55:33|12878|3243:3604|reinstall|utils.pm:update_computer_state(1587)|computer 18 state updated to: reloading
  972. 2013-05-28 13:55:33|12878|3243:3604|reinstall|new.pm:reload_image(677)|computer ecco-vm-x10 state set to reloading
  973. 2013-05-28 13:55:33|12878|3243:3604|reinstall|utils.pm:insertloadlog(3665)|inserted computer=18, info, computer state updated to reloading
  974. 2013-05-28 13:55:33|12878|3243:3604|reinstall|new.pm:reload_image(688)|calling VCL::Module::Provisioning::libvirt->load() subroutine
  975. 2013-05-28 13:55:33|12878|3243:3604|reinstall|utils.pm:insertloadlog(3665)|inserted computer=18, info, calling VCL::Module::Provisioning::libvirt->load() subroutine
  976. 2013-05-28 13:55:33|12878|3243:3604|reinstall|libvirt.pm:get_domain_name(914)|constructed domain name: 'ecco-vm-x10:linux-TestImage2-16-v2'
  977. 2013-05-28 13:55:33|12878|3243:3604|reinstall|utils.pm:insertloadlog(3665)|inserted computer=18, doesimageexists, image exists linux-TestImage2-16-v2
  978. 2013-05-28 13:55:33|12878|3243:3604|reinstall|utils.pm:insertloadlog(3665)|inserted computer=18, startload, ecco-vm-x10 linux-TestImage2-16-v2
  979. 2013-05-28 13:55:33|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  980. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'virsh list --all' 2>&1
  981. 2013-05-28 13:55:33|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  982. |12878|3243:3604|reinstall| Id Name State
  983. |12878|3243:3604|reinstall| ----------------------------------------------------
  984. |12878|3243:3604|reinstall| 65 ecco-vm-x9:linux-InfoSphereBigInsights-12-v10 running
  985. |12878|3243:3604|reinstall| 66 ecco-vm-x11:linux-TestImage2-16-v2 running
  986. |12878|3243:3604|reinstall| 67 ecco-vm-x12:linux-TestImage2-16-v2 running
  987. |12878|3243:3604|reinstall| 68 ecco-vm-x10:linux-TestImage2-16-v2 running
  988. 2013-05-28 13:55:33|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x3, returning (0, "Id Name State ----------------...")
  989. 2013-05-28 13:55:33|12878|3243:3604|reinstall|libvirt.pm:get_domain_info(1769)|listed defined domains on ecco-vmhost-x3
  990. |12878|3243:3604|reinstall| command: virsh list --all
  991. |12878|3243:3604|reinstall| output:
  992. |12878|3243:3604|reinstall| Id Name State
  993. |12878|3243:3604|reinstall| ----------------------------------------------------
  994. |12878|3243:3604|reinstall| 65 ecco-vm-x9:linux-InfoSphereBigInsights-12-v10 running
  995. |12878|3243:3604|reinstall| 66 ecco-vm-x11:linux-TestImage2-16-v2 running
  996. |12878|3243:3604|reinstall| 67 ecco-vm-x12:linux-TestImage2-16-v2 running
  997. |12878|3243:3604|reinstall| 68 ecco-vm-x10:linux-TestImage2-16-v2 running
  998. 2013-05-28 13:55:33|12878|3243:3604|reinstall|libvirt.pm:get_domain_info(1791)|retrieved domain info from ecco-vmhost-x3:
  999. |12878|3243:3604|reinstall| 65. ecco-vm-x9:linux-InfoSphereBigInsights-12-v10 (running)
  1000. |12878|3243:3604|reinstall| 66. ecco-vm-x11:linux-TestImage2-16-v2 (running)
  1001. |12878|3243:3604|reinstall| 67. ecco-vm-x12:linux-TestImage2-16-v2 (running)
  1002. |12878|3243:3604|reinstall| 68. ecco-vm-x10:linux-TestImage2-16-v2 (running)
  1003. 2013-05-28 13:55:33|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  1004. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'virsh list --all' 2>&1
  1005. 2013-05-28 13:55:33|12889|3243:3605|reinstall|Semaphore.pm:open_lockfile(231)|lsof did not return any PIDs of processes which prevented an exclusive lock to be obtained, lock may have been released before lsof command was executed
  1006. 2013-05-28 13:55:33|12889|3243:3605|reinstall|Module.pm:code_loop_timeout(909)|attempt 1: attempting to open lockfile (0/7200 elapsed/remaining seconds), sleeping for 15 seconds
  1007. 2013-05-28 13:55:33|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1008. |12878|3243:3604|reinstall| Id Name State
  1009. |12878|3243:3604|reinstall| ----------------------------------------------------
  1010. |12878|3243:3604|reinstall| 65 ecco-vm-x9:linux-InfoSphereBigInsights-12-v10 running
  1011. |12878|3243:3604|reinstall| 66 ecco-vm-x11:linux-TestImage2-16-v2 running
  1012. |12878|3243:3604|reinstall| 67 ecco-vm-x12:linux-TestImage2-16-v2 running
  1013. |12878|3243:3604|reinstall| 68 ecco-vm-x10:linux-TestImage2-16-v2 running
  1014. 2013-05-28 13:55:33|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x3, returning (0, "Id Name State ----------------...")
  1015. 2013-05-28 13:55:33|12878|3243:3604|reinstall|libvirt.pm:get_domain_info(1769)|listed defined domains on ecco-vmhost-x3
  1016. |12878|3243:3604|reinstall| command: virsh list --all
  1017. |12878|3243:3604|reinstall| output:
  1018. |12878|3243:3604|reinstall| Id Name State
  1019. |12878|3243:3604|reinstall| ----------------------------------------------------
  1020. |12878|3243:3604|reinstall| 65 ecco-vm-x9:linux-InfoSphereBigInsights-12-v10 running
  1021. |12878|3243:3604|reinstall| 66 ecco-vm-x11:linux-TestImage2-16-v2 running
  1022. |12878|3243:3604|reinstall| 67 ecco-vm-x12:linux-TestImage2-16-v2 running
  1023. |12878|3243:3604|reinstall| 68 ecco-vm-x10:linux-TestImage2-16-v2 running
  1024. 2013-05-28 13:55:33|12878|3243:3604|reinstall|libvirt.pm:get_domain_info(1791)|retrieved domain info from ecco-vmhost-x3:
  1025. |12878|3243:3604|reinstall| 65. ecco-vm-x9:linux-InfoSphereBigInsights-12-v10 (running)
  1026. |12878|3243:3604|reinstall| 66. ecco-vm-x11:linux-TestImage2-16-v2 (running)
  1027. |12878|3243:3604|reinstall| 67. ecco-vm-x12:linux-TestImage2-16-v2 (running)
  1028. |12878|3243:3604|reinstall| 68. ecco-vm-x10:linux-TestImage2-16-v2 (running)
  1029. 2013-05-28 13:55:33|12878|3243:3604|reinstall|libvirt.pm:domain_exists(2013)|'ecco-vm-x10:linux-TestImage2-16-v2' exists on ecco-vmhost-x3
  1030. 2013-05-28 13:55:33|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  1031. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'virsh list --all' 2>&1
  1032. 2013-05-28 13:55:33|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1033. |12878|3243:3604|reinstall| Id Name State
  1034. |12878|3243:3604|reinstall| ----------------------------------------------------
  1035. |12878|3243:3604|reinstall| 65 ecco-vm-x9:linux-InfoSphereBigInsights-12-v10 running
  1036. |12878|3243:3604|reinstall| 66 ecco-vm-x11:linux-TestImage2-16-v2 running
  1037. |12878|3243:3604|reinstall| 67 ecco-vm-x12:linux-TestImage2-16-v2 running
  1038. |12878|3243:3604|reinstall| 68 ecco-vm-x10:linux-TestImage2-16-v2 running
  1039. 2013-05-28 13:55:33|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x3, returning (0, "Id Name State ----------------...")
  1040. 2013-05-28 13:55:33|12878|3243:3604|reinstall|libvirt.pm:get_domain_info(1769)|listed defined domains on ecco-vmhost-x3
  1041. |12878|3243:3604|reinstall| command: virsh list --all
  1042. |12878|3243:3604|reinstall| output:
  1043. |12878|3243:3604|reinstall| Id Name State
  1044. |12878|3243:3604|reinstall| ----------------------------------------------------
  1045. |12878|3243:3604|reinstall| 65 ecco-vm-x9:linux-InfoSphereBigInsights-12-v10 running
  1046. |12878|3243:3604|reinstall| 66 ecco-vm-x11:linux-TestImage2-16-v2 running
  1047. |12878|3243:3604|reinstall| 67 ecco-vm-x12:linux-TestImage2-16-v2 running
  1048. |12878|3243:3604|reinstall| 68 ecco-vm-x10:linux-TestImage2-16-v2 running
  1049. 2013-05-28 13:55:33|12878|3243:3604|reinstall|libvirt.pm:get_domain_info(1791)|retrieved domain info from ecco-vmhost-x3:
  1050. |12878|3243:3604|reinstall| 65. ecco-vm-x9:linux-InfoSphereBigInsights-12-v10 (running)
  1051. |12878|3243:3604|reinstall| 66. ecco-vm-x11:linux-TestImage2-16-v2 (running)
  1052. |12878|3243:3604|reinstall| 67. ecco-vm-x12:linux-TestImage2-16-v2 (running)
  1053. |12878|3243:3604|reinstall| 68. ecco-vm-x10:linux-TestImage2-16-v2 (running)
  1054. 2013-05-28 13:55:33|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  1055. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'virsh destroy "ecco-vm-x10:linux-TestImage2-16-v2"' 2>&1
  1056. 2013-05-28 13:55:34|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1057. |12878|3243:3604|reinstall| Domain ecco-vm-x10:linux-TestImage2-16-v2 destroyed
  1058. 2013-05-28 13:55:34|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x3, returning (0, "Domain ecco-vm-x10:linux-TestI...")
  1059. 2013-05-28 13:55:34|12878|3243:3604|reinstall|libvirt.pm:power_off(715)|destroyed 'ecco-vm-x10:linux-TestImage2-16-v2' domain on ecco-vmhost-x3
  1060. 2013-05-28 13:55:34|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  1061. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'virsh snapshot-list "ecco-vm-x10:linux-TestImage2-16-v2"' 2>&1
  1062. 2013-05-28 13:55:34|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1063. |12878|3243:3604|reinstall| Name Creation Time State
  1064. |12878|3243:3604|reinstall| ------------------------------------------------------------
  1065. 2013-05-28 13:55:34|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x3, returning (0, "Name Creation Time State -----...")
  1066. 2013-05-28 13:55:34|12878|3243:3604|reinstall|libvirt.pm:get_snapshot_info(2060)|listed snapshots of 'ecco-vm-x10:linux-TestImage2-16-v2' domain on ecco-vmhost-x3
  1067. |12878|3243:3604|reinstall| command: virsh snapshot-list "ecco-vm-x10:linux-TestImage2-16-v2"
  1068. |12878|3243:3604|reinstall| exit status: 0
  1069. |12878|3243:3604|reinstall| output:
  1070. |12878|3243:3604|reinstall| Name Creation Time State
  1071. |12878|3243:3604|reinstall| ------------------------------------------------------------
  1072. 2013-05-28 13:55:34|12878|3243:3604|reinstall|libvirt.pm:get_snapshot_info(2077)|retrieved snapshot info for 'ecco-vm-x10:linux-TestImage2-16-v2' domain on ecco-vmhost-x3:
  1073. |12878|3243:3604|reinstall| : {}
  1074. 2013-05-28 13:55:34|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  1075. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'virsh dumpxml "ecco-vm-x10:linux-TestImage2-16-v2"' 2>&1
  1076. 2013-05-28 13:55:34|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1077. |12878|3243:3604|reinstall| <domain type='kvm'>
  1078. |12878|3243:3604|reinstall| <name>ecco-vm-x10:linux-TestImage2-16-v2</name>
  1079. |12878|3243:3604|reinstall| <uuid>57c2b9ac-fa76-f1f3-c00d-fa730513d859</uuid>
  1080. |12878|3243:3604|reinstall| <description>Test Image 2</description>
  1081. |12878|3243:3604|reinstall| <memory unit='KiB'>8387584</memory>
  1082. |12878|3243:3604|reinstall| <currentMemory unit='KiB'>8387584</currentMemory>
  1083. |12878|3243:3604|reinstall| <vcpu placement='static'>2</vcpu>
  1084. |12878|3243:3604|reinstall| <os>
  1085. |12878|3243:3604|reinstall| <type arch='x86_64' machine='rhel6.3.0'>hvm</type>
  1086. |12878|3243:3604|reinstall| <boot dev='hd'/>
  1087. |12878|3243:3604|reinstall| </os>
  1088. |12878|3243:3604|reinstall| <features>
  1089. |12878|3243:3604|reinstall| <acpi/>
  1090. |12878|3243:3604|reinstall| <apic/>
  1091. |12878|3243:3604|reinstall| </features>
  1092. |12878|3243:3604|reinstall| <cpu>
  1093. |12878|3243:3604|reinstall| <topology sockets='2' cores='2' threads='2'/>
  1094. |12878|3243:3604|reinstall| </cpu>
  1095. |12878|3243:3604|reinstall| <clock offset='utc'/>
  1096. |12878|3243:3604|reinstall| <on_poweroff>preserve</on_poweroff>
  1097. |12878|3243:3604|reinstall| <on_reboot>restart</on_reboot>
  1098. |12878|3243:3604|reinstall| <on_crash>preserve</on_crash>
  1099. |12878|3243:3604|reinstall| <devices>
  1100. |12878|3243:3604|reinstall| <emulator>/usr/libexec/qemu-kvm</emulator>
  1101. |12878|3243:3604|reinstall| <disk type='file' device='disk'>
  1102. |12878|3243:3604|reinstall| <driver name='qemu' type='qcow2' cache='none'/>
  1103. |12878|3243:3604|reinstall| <source file='/var/lib/libvirt/images/ecco-vm-x10_16-v2.qcow2'/>
  1104. |12878|3243:3604|reinstall| <target dev='vda' bus='ide'/>
  1105. |12878|3243:3604|reinstall| <address type='drive' controller='0' bus='0' target='0' unit='0'/>
  1106. |12878|3243:3604|reinstall| </disk>
  1107. |12878|3243:3604|reinstall| <controller type='usb' index='0'>
  1108. |12878|3243:3604|reinstall| <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
  1109. |12878|3243:3604|reinstall| </controller>
  1110. |12878|3243:3604|reinstall| <controller type='ide' index='0'>
  1111. |12878|3243:3604|reinstall| <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
  1112. |12878|3243:3604|reinstall| </controller>
  1113. |12878|3243:3604|reinstall| <interface type='bridge'>
  1114. |12878|3243:3604|reinstall| <mac address='52:54:00:01:00:09'/>
  1115. |12878|3243:3604|reinstall| <source bridge='br0'/>
  1116. |12878|3243:3604|reinstall| <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
  1117. |12878|3243:3604|reinstall| </interface>
  1118. |12878|3243:3604|reinstall| <interface type='bridge'>
  1119. |12878|3243:3604|reinstall| <mac address='52:54:00:01:80:09'/>
  1120. |12878|3243:3604|reinstall| <source bridge='br1'/>
  1121. |12878|3243:3604|reinstall| <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
  1122. |12878|3243:3604|reinstall| </interface>
  1123. |12878|3243:3604|reinstall| <input type='mouse' bus='ps2'/>
  1124. |12878|3243:3604|reinstall| <graphics type='vnc' port='-1' autoport='yes'/>
  1125. |12878|3243:3604|reinstall| <video>
  1126. |12878|3243:3604|reinstall| <model type='cirrus' vram='9216' heads='1'/>
  1127. |12878|3243:3604|reinstall| <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
  1128. |12878|3243:3604|reinstall| </video>
  1129. |12878|3243:3604|reinstall| <memballoon model='virtio'>
  1130. |12878|3243:3604|reinstall| <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
  1131. |12878|3243:3604|reinstall| </memballoon>
  1132. |12878|3243:3604|reinstall| </devices>
  1133. |12878|3243:3604|reinstall| </domain>
  1134. 2013-05-28 13:55:34|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x3, returning (0, "<domain type='kvm'> <name>ecco...")
  1135. 2013-05-28 13:55:34|12878|3243:3604|reinstall|libvirt.pm:get_domain_xml(1869)|retrieved XML definition for 'ecco-vm-x10:linux-TestImage2-16-v2' domain on ecco-vmhost-x3
  1136. 2013-05-28 13:55:34|12878|3243:3604|reinstall|libvirt.pm:delete_domain(1525)|deleting disk assigned to domain: /var/lib/libvirt/images/ecco-vm-x10_16-v2.qcow2
  1137. 2013-05-28 13:55:34|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  1138. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'rm -rfv /var/lib/libvirt/images/ecco\-vm\-x10_16\-v2\.qcow2' 2>&1
  1139. 2013-05-28 13:55:34|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1140. |12878|3243:3604|reinstall| removed `/var/lib/libvirt/images/ecco-vm-x10_16-v2.qcow2'
  1141. 2013-05-28 13:55:34|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x3, returning (0, "removed `/var/lib/libvirt/imag...")
  1142. 2013-05-28 13:55:34|12878|3243:3604|reinstall|Linux.pm:delete_file(1504)|deleted '/var/lib/libvirt/images/ecco-vm-x10_16-v2.qcow2' on ecco-vmhost-x3
  1143. 2013-05-28 13:55:34|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  1144. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'stat /var/lib/libvirt/images/ecco\-vm\-x10_16\-v2\.qcow2' 2>&1
  1145. 2013-05-28 13:55:34|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1146. |12878|3243:3604|reinstall| stat: cannot stat `/var/lib/libvirt/images/ecco-vm-x10_16-v2.qcow2': No such file or directory
  1147. 2013-05-28 13:55:34|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vmhost-x3, command:
  1148. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'stat /var/lib/libvirt/images/ecco\-vm\-x10_16\-v2\.qcow2' 2>&1
  1149. |12878|3243:3604|reinstall| returning (1, "stat: cannot stat `/var/lib/li...")
  1150. 2013-05-28 13:55:34|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  1151. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'virsh undefine "ecco-vm-x10:linux-TestImage2-16-v2" --managed-save --snapshots-metadata' 2>&1
  1152. 2013-05-28 13:55:34|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1153. |12878|3243:3604|reinstall| Domain ecco-vm-x10:linux-TestImage2-16-v2 has been undefined
  1154. 2013-05-28 13:55:34|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x3, returning (0, "Domain ecco-vm-x10:linux-TestI...")
  1155. 2013-05-28 13:55:34|12878|3243:3604|reinstall|libvirt.pm:delete_domain(1549)|undefined 'ecco-vm-x10:linux-TestImage2-16-v2' domain on ecco-vmhost-x3
  1156. 2013-05-28 13:55:34|12878|3243:3604|reinstall|libvirt.pm:delete_domain(1552)|deleted 'ecco-vm-x10:linux-TestImage2-16-v2' domain from ecco-vmhost-x3
  1157. 2013-05-28 13:55:34|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  1158. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'rm -rfv /tmp/vcl/ecco\-vm\-x10_*\.xml' 2>&1
  1159. 2013-05-28 13:55:34|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1160. |12878|3243:3604|reinstall| removed `/tmp/vcl/ecco-vm-x10_16-v2.xml'
  1161. 2013-05-28 13:55:34|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x3, returning (0, "removed `/tmp/vcl/ecco-vm-x10_...")
  1162. 2013-05-28 13:55:34|12878|3243:3604|reinstall|Linux.pm:delete_file(1504)|deleted '/tmp/vcl/ecco-vm-x10_*.xml' on ecco-vmhost-x3
  1163. 2013-05-28 13:55:34|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  1164. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'stat /tmp/vcl/ecco\-vm\-x10_*\.xml' 2>&1
  1165. 2013-05-28 13:55:34|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1166. |12878|3243:3604|reinstall| stat: cannot stat `/tmp/vcl/ecco-vm-x10_*.xml': No such file or directory
  1167. 2013-05-28 13:55:34|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vmhost-x3, command:
  1168. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'stat /tmp/vcl/ecco\-vm\-x10_*\.xml' 2>&1
  1169. |12878|3243:3604|reinstall| returning (1, "stat: cannot stat `/tmp/vcl/ec...")
  1170. 2013-05-28 13:55:34|12878|3243:3604|reinstall|libvirt.pm:delete_existing_domains(1460)|deleted existing domains created for ecco-vm-x10 on ecco-vmhost-x3
  1171. 2013-05-28 13:55:34|12878|3243:3604|reinstall|OS.pm:find_files(2917)|attempting to find files on ecco-vmhost-x3, base directory path: '/images/vm/', pattern: linux-TestImage2-16-v2.*, command: /usr/bin/find "/images/vm/" -type f -iname "linux-TestImage2-16-v2.*"
  1172. 2013-05-28 13:55:35|12878|3243:3604|reinstall|OS.pm:find_files(2945)|files found: 1, base directory: '/images/vm/', pattern: 'linux-TestImage2-16-v2.*'
  1173. |12878|3243:3604|reinstall| command: '/usr/bin/find "/images/vm/" -type f -iname "linux-TestImage2-16-v2.*"'
  1174. 2013-05-28 13:55:35|12878|3243:3604|reinstall|libvirt.pm:get_master_image_file_path(1166)|found master image file on ecco-vmhost-x3: /images/vm/linux-TestImage2-16-v2.qcow2
  1175. 2013-05-28 13:55:35|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  1176. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'stat /images/vm/linux\-TestImage2\-16\-v2\.qcow2' 2>&1
  1177. 2013-05-28 13:55:35|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1178. |12878|3243:3604|reinstall| File: `/images/vm/linux-TestImage2-16-v2.qcow2'
  1179. |12878|3243:3604|reinstall| Size: 3498573824 Blocks: 6832920 IO Block: 65536 regular file
  1180. |12878|3243:3604|reinstall| Device: 1dh/29d Inode: 122683637 Links: 2
  1181. |12878|3243:3604|reinstall| Access: (0644/-rw-r--r--) Uid: ( 99/ nobody) Gid: ( 99/ nobody)
  1182. |12878|3243:3604|reinstall| Access: 2013-05-27 14:00:22.909942374 -0400
  1183. |12878|3243:3604|reinstall| Modify: 2013-05-27 13:19:17.866852434 -0400
  1184. |12878|3243:3604|reinstall| Change: 2013-05-27 14:00:01.634693243 -0400
  1185. 2013-05-28 13:55:35|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x3, returning (0, "File: `/images/vm/linux-TestIm...")
  1186. 2013-05-28 13:55:35|12878|3243:3604|reinstall|Linux.pm:file_exists(1449)|'/images/vm/linux-TestImage2-16-v2.qcow2' exists on ecco-vmhost-x3, files: 1, directories: 0, links: 0
  1187. 2013-05-28 13:55:35|12878|3243:3604|reinstall|KVM.pm:pre_define(192)|master image file exists in the datastore on ecco-vmhost-x3: /images/vm/linux-TestImage2-16-v2.qcow2
  1188. 2013-05-28 13:55:35|12878|3243:3604|reinstall|KVM.pm:create_copy_on_write_image(670)|creating copy on write image on ecco-vmhost-x3
  1189. |12878|3243:3604|reinstall| master disk image: /images/vm/linux-TestImage2-16-v2.qcow2
  1190. |12878|3243:3604|reinstall| copy on write image: /var/lib/libvirt/images/ecco-vm-x10_16-v2.qcow2
  1191. |12878|3243:3604|reinstall| format: qcow2
  1192. 2013-05-28 13:55:35|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  1193. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'qemu-img create -f qcow2 -b "/images/vm/linux-TestImage2-16-v2.qcow2" "/var/lib/libvirt/images/ecco-vm-x10_16-v2.qcow2"' 2>&1
  1194. 2013-05-28 13:55:35|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1195. |12878|3243:3604|reinstall| Formatting '/var/lib/libvirt/images/ecco-vm-x10_16-v2.qcow2', fmt=qcow2 size=32212254720 backing_file='/images/vm/linux-TestImage2-16-v2.qcow2' encryption=off cluster_size=65536
  1196. 2013-05-28 13:55:35|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x3, returning (0, "Formatting '/var/lib/libvirt/i...")
  1197. 2013-05-28 13:55:35|12878|3243:3604|reinstall|KVM.pm:create_copy_on_write_image(682)|created copy on write image: /var/lib/libvirt/images/ecco-vm-x10_16-v2.qcow2, output:
  1198. |12878|3243:3604|reinstall| Formatting '/var/lib/libvirt/images/ecco-vm-x10_16-v2.qcow2', fmt=qcow2 size=32212254720 backing_file='/images/vm/linux-TestImage2-16-v2.qcow2' encryption=off cluster_size=65536
  1199. 2013-05-28 13:55:35|12878|3243:3604|reinstall|utils.pm:insertloadlog(3665)|inserted computer=18, transfervm, performed libvirt driver tasks before defining domain
  1200. 2013-05-28 13:55:35|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  1201. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'ls -d --color=never "/tmp/vcl" 2>&1 || mkdir -p "/tmp/vcl" 2>&1 && ls -d --color=never "/tmp/vcl"' 2>&1
  1202. 2013-05-28 13:55:35|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1203. |12878|3243:3604|reinstall| /tmp/vcl
  1204. |12878|3243:3604|reinstall| /tmp/vcl
  1205. 2013-05-28 13:55:35|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x3, returning (0, "/tmp/vcl /tmp/vcl")
  1206. 2013-05-28 13:55:35|12878|3243:3604|reinstall|Linux.pm:create_directory(1573)|directory already exists on ecco-vmhost-x3: '/tmp/vcl'
  1207. 2013-05-28 13:55:35|12878|3243:3604|reinstall|OS.pm:create_text_file(1851)|created file on ecco-vmhost-x3: /tmp/vcl/ecco-vm-x10_16-v2.xml
  1208. 2013-05-28 13:55:35|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  1209. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'virsh define "/tmp/vcl/ecco-vm-x10_16-v2.xml"' 2>&1
  1210. 2013-05-28 13:55:35|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1211. |12878|3243:3604|reinstall| Domain ecco-vm-x10:linux-TestImage2-16-v2 defined from /tmp/vcl/ecco-vm-x10_16-v2.xml
  1212. 2013-05-28 13:55:35|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x3, returning (0, "Domain ecco-vm-x10:linux-TestI...")
  1213. 2013-05-28 13:55:35|12878|3243:3604|reinstall|libvirt.pm:define_domain(1384)|defined domain on ecco-vmhost-x3, output:
  1214. |12878|3243:3604|reinstall| Domain ecco-vm-x10:linux-TestImage2-16-v2 defined from /tmp/vcl/ecco-vm-x10_16-v2.xml
  1215. 2013-05-28 13:55:35|12878|3243:3604|reinstall|libvirt.pm:define_domain(1400)|parsed domain name from 'virsh define' output: 'ecco-vm-x10:linux-TestImage2-16-v2'
  1216. 2013-05-28 13:55:35|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  1217. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'virsh autostart "ecco-vm-x10:linux-TestImage2-16-v2"' 2>&1
  1218. 2013-05-28 13:55:35|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1219. |12878|3243:3604|reinstall| Domain ecco-vm-x10:linux-TestImage2-16-v2 marked as autostarted
  1220. 2013-05-28 13:55:35|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x3, returning (0, "Domain ecco-vm-x10:linux-TestI...")
  1221. 2013-05-28 13:55:35|12878|3243:3604|reinstall|libvirt.pm:define_domain(1410)|configured 'ecco-vm-x10:linux-TestImage2-16-v2' domain to autostart on ecco-vmhost-x3, output:
  1222. |12878|3243:3604|reinstall| Domain ecco-vm-x10:linux-TestImage2-16-v2 marked as autostarted
  1223. 2013-05-28 13:55:35|12878|3243:3604|reinstall|utils.pm:insertloadlog(3665)|inserted computer=18, vmsetupconfig, defined ecco-vm-x10 domain node ecco-vmhost-x3
  1224. 2013-05-28 13:55:35|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  1225. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'virsh start "ecco-vm-x10:linux-TestImage2-16-v2"' 2>&1
  1226. 2013-05-28 13:55:36|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1227. |12878|3243:3604|reinstall| Domain ecco-vm-x10:linux-TestImage2-16-v2 started
  1228. 2013-05-28 13:55:36|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x3, returning (0, "Domain ecco-vm-x10:linux-TestI...")
  1229. 2013-05-28 13:55:36|12878|3243:3604|reinstall|libvirt.pm:power_on(673)|started 'ecco-vm-x10:linux-TestImage2-16-v2' domain on ecco-vmhost-x3
  1230. 2013-05-28 13:55:36|12878|3243:3604|reinstall|utils.pm:insertloadlog(3665)|inserted computer=18, startvm, powered on ecco-vm-x10 domain node ecco-vmhost-x3
  1231. 2013-05-28 13:55:36|12878|3243:3604|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: 'Kernel IP routing table'
  1232. 2013-05-28 13:55:36|12878|3243:3604|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: 'Destination Gateway Genmask Flags Metric Ref Use Iface'
  1233. 2013-05-28 13:55:36|12878|3243:3604|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '199.241.160.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1'
  1234. 2013-05-28 13:55:36|12878|3243:3604|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '10.18.20.0 0.0.0.0 255.255.254.0 U 0 0 0 eth0'
  1235. 2013-05-28 13:55:36|12878|3243:3604|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '169.254.0.0 0.0.0.0 255.255.0.0 U 1003 0 0 eth0'
  1236. 2013-05-28 13:55:36|12878|3243:3604|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '169.254.0.0 0.0.0.0 255.255.0.0 U 1004 0 0 eth1'
  1237. 2013-05-28 13:55:36|12878|3243:3604|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '10.0.0.0 10.18.21.254 255.0.0.0 UG 0 0 0 eth0'
  1238. 2013-05-28 13:55:36|12878|3243:3604|reinstall|Linux.pm:get_network_configuration(2703)|found default route configured for 'eth1' interface: 199.241.160.1
  1239. 2013-05-28 13:55:36|12878|3243:3604|reinstall|Linux.pm:get_network_configuration(2708)|retrieved network configuration:
  1240. |12878|3243:3604|reinstall| : {
  1241. |12878|3243:3604|reinstall| : "eth0" => {
  1242. |12878|3243:3604|reinstall| : "broadcast_address" => "10.18.21.255",
  1243. |12878|3243:3604|reinstall| : "ip_address" => {
  1244. |12878|3243:3604|reinstall| : "10.18.20.4" => "255.255.254.0"
  1245. |12878|3243:3604|reinstall| : },
  1246. |12878|3243:3604|reinstall| : "name" => "eth0",
  1247. |12878|3243:3604|reinstall| : "physical_address" => "34:40:b5:82:4e:7c"
  1248. |12878|3243:3604|reinstall| : },
  1249. |12878|3243:3604|reinstall| : "eth1" => {
  1250. |12878|3243:3604|reinstall| : "broadcast_address" => "199.241.160.255",
  1251. |12878|3243:3604|reinstall| : "default_gateway" => "199.241.160.1",
  1252. |12878|3243:3604|reinstall| : "ip_address" => {
  1253. |12878|3243:3604|reinstall| : "199.241.160.4" => "255.255.255.0"
  1254. |12878|3243:3604|reinstall| : },
  1255. |12878|3243:3604|reinstall| : "name" => "eth1",
  1256. |12878|3243:3604|reinstall| : "physical_address" => "34:40:b5:82:4e:7e"
  1257. |12878|3243:3604|reinstall| : },
  1258. |12878|3243:3604|reinstall| : "lo" => {
  1259. |12878|3243:3604|reinstall| : "name" => "lo"
  1260. |12878|3243:3604|reinstall| : },
  1261. |12878|3243:3604|reinstall| : "usb0" => {
  1262. |12878|3243:3604|reinstall| : "name" => "usb0",
  1263. |12878|3243:3604|reinstall| : "physical_address" => "36:40:b5:85:4e:7f"
  1264. |12878|3243:3604|reinstall| : }
  1265. |12878|3243:3604|reinstall| : }
  1266. 2013-05-28 13:55:36|12878|3243:3604|reinstall|DataStructure.pm:get_computer_private_ip_address(1641)|attempting to retrieve private IP address for computer: ecco-adm3
  1267. 2013-05-28 13:55:36|12878|3243:3604|reinstall|OS.pm:get_file_contents(1897)|retrieved 146 lines from file on ecco-adm3: '/etc/hosts'
  1268. 2013-05-28 13:55:36|12878|3243:3604|reinstall|DataStructure.pm:get_computer_private_ip_address(1697)|returning IP address from /etc/hosts file: 10.18.20.4
  1269. 2013-05-28 13:55:36|12878|3243:3604|reinstall|OS.pm:get_private_interface_name(1119)|determined private interface name: eth0 (10.18.20.4)
  1270. 2013-05-28 13:55:36|12878|3243:3604|reinstall|OS.pm:get_ip_address(1538)|returning private IP address: 10.18.20.4
  1271. 2013-05-28 13:55:36|12878|3243:3604|reinstall|Linux.pm:post_load(237)|initiating Linux post_load: linux-TestImage2-16-v2 on ecco-vm-x10
  1272. 2013-05-28 13:55:36|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:55:36
  1273. 2013-05-28 13:55:37|12878|3243:3604|reinstall|OS.pm:is_ssh_responding(653)|ecco-vm-x10 is NOT responding to SSH, ports 22 or 24 are both closed
  1274. 2013-05-28 13:55:37|12878|3243:3604|reinstall|OS.pm:wait_for_response(734)|waiting 30 seconds for ecco-vm-x10 to boot
  1275. 2013-05-28 13:55:41|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:55:41
  1276. 2013-05-28 13:55:46|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:55:46
  1277. 2013-05-28 13:55:48|12889|3243:3605|reinstall|Semaphore.pm:open_lockfile(175)|opened and obtained an exclusive lock on file: /tmp/master-linux-TestImage2-16-v2.semaphore
  1278. 2013-05-28 13:55:48|12889|3243:3605|reinstall|Semaphore.pm:open_lockfile(182)|wrote to file: /tmp/master-linux-TestImage2-16-v2.semaphore, contents:
  1279. |12889|3243:3605|reinstall| '12889 vcld 3243:3605 reinstall ecco-vm-x26>ecco-vmhost-x7 linux-TestImage2-16-v2 nsasherr (cluster=child)'
  1280. 2013-05-28 13:55:48|12889|3243:3605|reinstall|Module.pm:code_loop_timeout(900)|attempting to open lockfile, code returned true
  1281. 2013-05-28 13:55:48|12889|3243:3605|reinstall|Module.pm:get_semaphore(993)|created 'master:linux-TestImage2-16-v2' Semaphore object, memory address: 3e722f8
  1282. 2013-05-28 13:55:48|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x7:
  1283. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x7 'stat /images/vm/linux\-TestImage2\-16\-v2\.qcow2' 2>&1
  1284. 2013-05-28 13:55:48|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1285. |12889|3243:3605|reinstall| File: `/images/vm/linux-TestImage2-16-v2.qcow2'
  1286. |12889|3243:3605|reinstall| Size: 3498573824 Blocks: 6832920 IO Block: 65536 regular file
  1287. |12889|3243:3605|reinstall| Device: 1dh/29d Inode: 122683637 Links: 2
  1288. |12889|3243:3605|reinstall| Access: (0644/-rw-r--r--) Uid: ( 99/ nobody) Gid: ( 99/ nobody)
  1289. |12889|3243:3605|reinstall| Access: 2013-05-27 14:00:22.909942374 -0400
  1290. |12889|3243:3605|reinstall| Modify: 2013-05-27 13:19:17.866852434 -0400
  1291. |12889|3243:3605|reinstall| Change: 2013-05-27 14:00:01.634693243 -0400
  1292. 2013-05-28 13:55:48|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x7, returning (0, "File: `/images/vm/linux-TestIm...")
  1293. 2013-05-28 13:55:48|12889|3243:3605|reinstall|Linux.pm:file_exists(1449)|'/images/vm/linux-TestImage2-16-v2.qcow2' exists on ecco-vmhost-x7, files: 1, directories: 0, links: 0
  1294. 2013-05-28 13:55:48|12889|3243:3605|reinstall|libvirt.pm:does_image_exist(527)|linux-TestImage2-16-v2 image exists on ecco-vmhost-x7: /images/vm/linux-TestImage2-16-v2.qcow2
  1295. 2013-05-28 13:55:48|12889|3243:3605|reinstall|new.pm:reload_image(644)|linux-TestImage2-16-v2 exists on this management node
  1296. 2013-05-28 13:55:48|12889|3243:3605|reinstall|utils.pm:insertloadlog(3665)|inserted computer=40, doesimageexists, confirmed image exists
  1297. 2013-05-28 13:55:48|12889|3243:3605|reinstall|Semaphore.pm:release_lockfile(280)|deleted file: /tmp/master-linux-TestImage2-16-v2.semaphore
  1298. 2013-05-28 13:55:48|12889|3243:3605|reinstall|Semaphore.pm:DESTROY(377)|destroyed Semaphore object, memory address: 3e722f8
  1299. 2013-05-28 13:55:48|12889|3243:3605|reinstall|utils.pm:update_computer_state(1587)|computer 40 state updated to: reloading
  1300. 2013-05-28 13:55:48|12889|3243:3605|reinstall|new.pm:reload_image(677)|computer ecco-vm-x26 state set to reloading
  1301. 2013-05-28 13:55:48|12889|3243:3605|reinstall|utils.pm:insertloadlog(3665)|inserted computer=40, info, computer state updated to reloading
  1302. 2013-05-28 13:55:48|12889|3243:3605|reinstall|new.pm:reload_image(688)|calling VCL::Module::Provisioning::libvirt->load() subroutine
  1303. 2013-05-28 13:55:48|12889|3243:3605|reinstall|utils.pm:insertloadlog(3665)|inserted computer=40, info, calling VCL::Module::Provisioning::libvirt->load() subroutine
  1304. 2013-05-28 13:55:48|12889|3243:3605|reinstall|libvirt.pm:get_domain_name(914)|constructed domain name: 'ecco-vm-x26:linux-TestImage2-16-v2'
  1305. 2013-05-28 13:55:48|12889|3243:3605|reinstall|utils.pm:insertloadlog(3665)|inserted computer=40, doesimageexists, image exists linux-TestImage2-16-v2
  1306. 2013-05-28 13:55:48|12889|3243:3605|reinstall|utils.pm:insertloadlog(3665)|inserted computer=40, startload, ecco-vm-x26 linux-TestImage2-16-v2
  1307. 2013-05-28 13:55:48|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x7:
  1308. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x7 'virsh list --all' 2>&1
  1309. 2013-05-28 13:55:48|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1310. |12889|3243:3605|reinstall| Id Name State
  1311. |12889|3243:3605|reinstall| ----------------------------------------------------
  1312. |12889|3243:3605|reinstall| 29 ecco-vm-x28:linux-InfoSphereStreams-9-v12 running
  1313. |12889|3243:3605|reinstall| 42 ecco-vm-x25:linux-TestImage1-14-v5 running
  1314. |12889|3243:3605|reinstall| 52 ecco-vm-x27:linux-TestImage2-16-v2 running
  1315. |12889|3243:3605|reinstall| 53 ecco-vm-x26:linux-TestImage2-16-v2 running
  1316. 2013-05-28 13:55:48|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x7, returning (0, "Id Name State ----------------...")
  1317. 2013-05-28 13:55:48|12889|3243:3605|reinstall|libvirt.pm:get_domain_info(1769)|listed defined domains on ecco-vmhost-x7
  1318. |12889|3243:3605|reinstall| command: virsh list --all
  1319. |12889|3243:3605|reinstall| output:
  1320. |12889|3243:3605|reinstall| Id Name State
  1321. |12889|3243:3605|reinstall| ----------------------------------------------------
  1322. |12889|3243:3605|reinstall| 29 ecco-vm-x28:linux-InfoSphereStreams-9-v12 running
  1323. |12889|3243:3605|reinstall| 42 ecco-vm-x25:linux-TestImage1-14-v5 running
  1324. |12889|3243:3605|reinstall| 52 ecco-vm-x27:linux-TestImage2-16-v2 running
  1325. |12889|3243:3605|reinstall| 53 ecco-vm-x26:linux-TestImage2-16-v2 running
  1326. 2013-05-28 13:55:48|12889|3243:3605|reinstall|libvirt.pm:get_domain_info(1791)|retrieved domain info from ecco-vmhost-x7:
  1327. |12889|3243:3605|reinstall| 29. ecco-vm-x28:linux-InfoSphereStreams-9-v12 (running)
  1328. |12889|3243:3605|reinstall| 42. ecco-vm-x25:linux-TestImage1-14-v5 (running)
  1329. |12889|3243:3605|reinstall| 52. ecco-vm-x27:linux-TestImage2-16-v2 (running)
  1330. |12889|3243:3605|reinstall| 53. ecco-vm-x26:linux-TestImage2-16-v2 (running)
  1331. 2013-05-28 13:55:48|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x7:
  1332. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x7 'virsh list --all' 2>&1
  1333. 2013-05-28 13:55:49|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1334. |12889|3243:3605|reinstall| Id Name State
  1335. |12889|3243:3605|reinstall| ----------------------------------------------------
  1336. |12889|3243:3605|reinstall| 29 ecco-vm-x28:linux-InfoSphereStreams-9-v12 running
  1337. |12889|3243:3605|reinstall| 42 ecco-vm-x25:linux-TestImage1-14-v5 running
  1338. |12889|3243:3605|reinstall| 52 ecco-vm-x27:linux-TestImage2-16-v2 running
  1339. |12889|3243:3605|reinstall| 53 ecco-vm-x26:linux-TestImage2-16-v2 running
  1340. 2013-05-28 13:55:49|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x7, returning (0, "Id Name State ----------------...")
  1341. 2013-05-28 13:55:49|12889|3243:3605|reinstall|libvirt.pm:get_domain_info(1769)|listed defined domains on ecco-vmhost-x7
  1342. |12889|3243:3605|reinstall| command: virsh list --all
  1343. |12889|3243:3605|reinstall| output:
  1344. |12889|3243:3605|reinstall| Id Name State
  1345. |12889|3243:3605|reinstall| ----------------------------------------------------
  1346. |12889|3243:3605|reinstall| 29 ecco-vm-x28:linux-InfoSphereStreams-9-v12 running
  1347. |12889|3243:3605|reinstall| 42 ecco-vm-x25:linux-TestImage1-14-v5 running
  1348. |12889|3243:3605|reinstall| 52 ecco-vm-x27:linux-TestImage2-16-v2 running
  1349. |12889|3243:3605|reinstall| 53 ecco-vm-x26:linux-TestImage2-16-v2 running
  1350. 2013-05-28 13:55:49|12889|3243:3605|reinstall|libvirt.pm:get_domain_info(1791)|retrieved domain info from ecco-vmhost-x7:
  1351. |12889|3243:3605|reinstall| 29. ecco-vm-x28:linux-InfoSphereStreams-9-v12 (running)
  1352. |12889|3243:3605|reinstall| 42. ecco-vm-x25:linux-TestImage1-14-v5 (running)
  1353. |12889|3243:3605|reinstall| 52. ecco-vm-x27:linux-TestImage2-16-v2 (running)
  1354. |12889|3243:3605|reinstall| 53. ecco-vm-x26:linux-TestImage2-16-v2 (running)
  1355. 2013-05-28 13:55:49|12889|3243:3605|reinstall|libvirt.pm:domain_exists(2013)|'ecco-vm-x26:linux-TestImage2-16-v2' exists on ecco-vmhost-x7
  1356. 2013-05-28 13:55:49|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x7:
  1357. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x7 'virsh list --all' 2>&1
  1358. 2013-05-28 13:55:49|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1359. |12889|3243:3605|reinstall| Id Name State
  1360. |12889|3243:3605|reinstall| ----------------------------------------------------
  1361. |12889|3243:3605|reinstall| 29 ecco-vm-x28:linux-InfoSphereStreams-9-v12 running
  1362. |12889|3243:3605|reinstall| 42 ecco-vm-x25:linux-TestImage1-14-v5 running
  1363. |12889|3243:3605|reinstall| 52 ecco-vm-x27:linux-TestImage2-16-v2 running
  1364. |12889|3243:3605|reinstall| 53 ecco-vm-x26:linux-TestImage2-16-v2 running
  1365. 2013-05-28 13:55:49|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x7, returning (0, "Id Name State ----------------...")
  1366. 2013-05-28 13:55:49|12889|3243:3605|reinstall|libvirt.pm:get_domain_info(1769)|listed defined domains on ecco-vmhost-x7
  1367. |12889|3243:3605|reinstall| command: virsh list --all
  1368. |12889|3243:3605|reinstall| output:
  1369. |12889|3243:3605|reinstall| Id Name State
  1370. |12889|3243:3605|reinstall| ----------------------------------------------------
  1371. |12889|3243:3605|reinstall| 29 ecco-vm-x28:linux-InfoSphereStreams-9-v12 running
  1372. |12889|3243:3605|reinstall| 42 ecco-vm-x25:linux-TestImage1-14-v5 running
  1373. |12889|3243:3605|reinstall| 52 ecco-vm-x27:linux-TestImage2-16-v2 running
  1374. |12889|3243:3605|reinstall| 53 ecco-vm-x26:linux-TestImage2-16-v2 running
  1375. 2013-05-28 13:55:49|12889|3243:3605|reinstall|libvirt.pm:get_domain_info(1791)|retrieved domain info from ecco-vmhost-x7:
  1376. |12889|3243:3605|reinstall| 29. ecco-vm-x28:linux-InfoSphereStreams-9-v12 (running)
  1377. |12889|3243:3605|reinstall| 42. ecco-vm-x25:linux-TestImage1-14-v5 (running)
  1378. |12889|3243:3605|reinstall| 52. ecco-vm-x27:linux-TestImage2-16-v2 (running)
  1379. |12889|3243:3605|reinstall| 53. ecco-vm-x26:linux-TestImage2-16-v2 (running)
  1380. 2013-05-28 13:55:49|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x7:
  1381. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x7 'virsh destroy "ecco-vm-x26:linux-TestImage2-16-v2"' 2>&1
  1382. 2013-05-28 13:55:49|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1383. |12889|3243:3605|reinstall| Domain ecco-vm-x26:linux-TestImage2-16-v2 destroyed
  1384. 2013-05-28 13:55:49|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x7, returning (0, "Domain ecco-vm-x26:linux-TestI...")
  1385. 2013-05-28 13:55:49|12889|3243:3605|reinstall|libvirt.pm:power_off(715)|destroyed 'ecco-vm-x26:linux-TestImage2-16-v2' domain on ecco-vmhost-x7
  1386. 2013-05-28 13:55:49|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x7:
  1387. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x7 'virsh snapshot-list "ecco-vm-x26:linux-TestImage2-16-v2"' 2>&1
  1388. 2013-05-28 13:55:49|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1389. |12889|3243:3605|reinstall| Name Creation Time State
  1390. |12889|3243:3605|reinstall| ------------------------------------------------------------
  1391. 2013-05-28 13:55:49|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x7, returning (0, "Name Creation Time State -----...")
  1392. 2013-05-28 13:55:49|12889|3243:3605|reinstall|libvirt.pm:get_snapshot_info(2060)|listed snapshots of 'ecco-vm-x26:linux-TestImage2-16-v2' domain on ecco-vmhost-x7
  1393. |12889|3243:3605|reinstall| command: virsh snapshot-list "ecco-vm-x26:linux-TestImage2-16-v2"
  1394. |12889|3243:3605|reinstall| exit status: 0
  1395. |12889|3243:3605|reinstall| output:
  1396. |12889|3243:3605|reinstall| Name Creation Time State
  1397. |12889|3243:3605|reinstall| ------------------------------------------------------------
  1398. 2013-05-28 13:55:49|12889|3243:3605|reinstall|libvirt.pm:get_snapshot_info(2077)|retrieved snapshot info for 'ecco-vm-x26:linux-TestImage2-16-v2' domain on ecco-vmhost-x7:
  1399. |12889|3243:3605|reinstall| : {}
  1400. 2013-05-28 13:55:49|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x7:
  1401. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x7 'virsh dumpxml "ecco-vm-x26:linux-TestImage2-16-v2"' 2>&1
  1402. 2013-05-28 13:55:49|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1403. |12889|3243:3605|reinstall| <domain type='kvm'>
  1404. |12889|3243:3605|reinstall| <name>ecco-vm-x26:linux-TestImage2-16-v2</name>
  1405. |12889|3243:3605|reinstall| <uuid>45e3b5b4-202e-ca61-cbb8-0a32e51ed143</uuid>
  1406. |12889|3243:3605|reinstall| <description>Test Image 2</description>
  1407. |12889|3243:3605|reinstall| <memory unit='KiB'>8387584</memory>
  1408. |12889|3243:3605|reinstall| <currentMemory unit='KiB'>8387584</currentMemory>
  1409. |12889|3243:3605|reinstall| <vcpu placement='static'>2</vcpu>
  1410. |12889|3243:3605|reinstall| <os>
  1411. |12889|3243:3605|reinstall| <type arch='x86_64' machine='rhel6.3.0'>hvm</type>
  1412. |12889|3243:3605|reinstall| <boot dev='hd'/>
  1413. |12889|3243:3605|reinstall| </os>
  1414. |12889|3243:3605|reinstall| <features>
  1415. |12889|3243:3605|reinstall| <acpi/>
  1416. |12889|3243:3605|reinstall| <apic/>
  1417. |12889|3243:3605|reinstall| </features>
  1418. |12889|3243:3605|reinstall| <cpu>
  1419. |12889|3243:3605|reinstall| <topology sockets='2' cores='2' threads='2'/>
  1420. |12889|3243:3605|reinstall| </cpu>
  1421. |12889|3243:3605|reinstall| <clock offset='utc'/>
  1422. |12889|3243:3605|reinstall| <on_poweroff>preserve</on_poweroff>
  1423. |12889|3243:3605|reinstall| <on_reboot>restart</on_reboot>
  1424. |12889|3243:3605|reinstall| <on_crash>preserve</on_crash>
  1425. |12889|3243:3605|reinstall| <devices>
  1426. |12889|3243:3605|reinstall| <emulator>/usr/libexec/qemu-kvm</emulator>
  1427. |12889|3243:3605|reinstall| <disk type='file' device='disk'>
  1428. |12889|3243:3605|reinstall| <driver name='qemu' type='qcow2' cache='none'/>
  1429. |12889|3243:3605|reinstall| <source file='/var/lib/libvirt/images/ecco-vm-x26_16-v2.qcow2'/>
  1430. |12889|3243:3605|reinstall| <target dev='vda' bus='ide'/>
  1431. |12889|3243:3605|reinstall| <address type='drive' controller='0' bus='0' target='0' unit='0'/>
  1432. |12889|3243:3605|reinstall| </disk>
  1433. |12889|3243:3605|reinstall| <controller type='usb' index='0'>
  1434. |12889|3243:3605|reinstall| <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
  1435. |12889|3243:3605|reinstall| </controller>
  1436. |12889|3243:3605|reinstall| <controller type='ide' index='0'>
  1437. |12889|3243:3605|reinstall| <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
  1438. |12889|3243:3605|reinstall| </controller>
  1439. |12889|3243:3605|reinstall| <interface type='bridge'>
  1440. |12889|3243:3605|reinstall| <mac address='52:54:00:01:00:19'/>
  1441. |12889|3243:3605|reinstall| <source bridge='br0'/>
  1442. |12889|3243:3605|reinstall| <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
  1443. |12889|3243:3605|reinstall| </interface>
  1444. |12889|3243:3605|reinstall| <interface type='bridge'>
  1445. |12889|3243:3605|reinstall| <mac address='52:54:00:01:80:19'/>
  1446. |12889|3243:3605|reinstall| <source bridge='br1'/>
  1447. |12889|3243:3605|reinstall| <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
  1448. |12889|3243:3605|reinstall| </interface>
  1449. |12889|3243:3605|reinstall| <input type='mouse' bus='ps2'/>
  1450. |12889|3243:3605|reinstall| <graphics type='vnc' port='-1' autoport='yes'/>
  1451. |12889|3243:3605|reinstall| <video>
  1452. |12889|3243:3605|reinstall| <model type='cirrus' vram='9216' heads='1'/>
  1453. |12889|3243:3605|reinstall| <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
  1454. |12889|3243:3605|reinstall| </video>
  1455. |12889|3243:3605|reinstall| <memballoon model='virtio'>
  1456. |12889|3243:3605|reinstall| <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
  1457. |12889|3243:3605|reinstall| </memballoon>
  1458. |12889|3243:3605|reinstall| </devices>
  1459. |12889|3243:3605|reinstall| </domain>
  1460. 2013-05-28 13:55:49|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x7, returning (0, "<domain type='kvm'> <name>ecco...")
  1461. 2013-05-28 13:55:49|12889|3243:3605|reinstall|libvirt.pm:get_domain_xml(1869)|retrieved XML definition for 'ecco-vm-x26:linux-TestImage2-16-v2' domain on ecco-vmhost-x7
  1462. 2013-05-28 13:55:49|12889|3243:3605|reinstall|libvirt.pm:delete_domain(1525)|deleting disk assigned to domain: /var/lib/libvirt/images/ecco-vm-x26_16-v2.qcow2
  1463. 2013-05-28 13:55:49|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x7:
  1464. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x7 'rm -rfv /var/lib/libvirt/images/ecco\-vm\-x26_16\-v2\.qcow2' 2>&1
  1465. 2013-05-28 13:55:49|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1466. |12889|3243:3605|reinstall| removed `/var/lib/libvirt/images/ecco-vm-x26_16-v2.qcow2'
  1467. 2013-05-28 13:55:49|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x7, returning (0, "removed `/var/lib/libvirt/imag...")
  1468. 2013-05-28 13:55:49|12889|3243:3605|reinstall|Linux.pm:delete_file(1504)|deleted '/var/lib/libvirt/images/ecco-vm-x26_16-v2.qcow2' on ecco-vmhost-x7
  1469. 2013-05-28 13:55:49|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x7:
  1470. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x7 'stat /var/lib/libvirt/images/ecco\-vm\-x26_16\-v2\.qcow2' 2>&1
  1471. 2013-05-28 13:55:50|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1472. |12889|3243:3605|reinstall| stat: cannot stat `/var/lib/libvirt/images/ecco-vm-x26_16-v2.qcow2': No such file or directory
  1473. 2013-05-28 13:55:50|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vmhost-x7, command:
  1474. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x7 'stat /var/lib/libvirt/images/ecco\-vm\-x26_16\-v2\.qcow2' 2>&1
  1475. |12889|3243:3605|reinstall| returning (1, "stat: cannot stat `/var/lib/li...")
  1476. 2013-05-28 13:55:50|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x7:
  1477. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x7 'virsh undefine "ecco-vm-x26:linux-TestImage2-16-v2" --managed-save --snapshots-metadata' 2>&1
  1478. 2013-05-28 13:55:50|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1479. |12889|3243:3605|reinstall| Domain ecco-vm-x26:linux-TestImage2-16-v2 has been undefined
  1480. 2013-05-28 13:55:50|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x7, returning (0, "Domain ecco-vm-x26:linux-TestI...")
  1481. 2013-05-28 13:55:50|12889|3243:3605|reinstall|libvirt.pm:delete_domain(1549)|undefined 'ecco-vm-x26:linux-TestImage2-16-v2' domain on ecco-vmhost-x7
  1482. 2013-05-28 13:55:50|12889|3243:3605|reinstall|libvirt.pm:delete_domain(1552)|deleted 'ecco-vm-x26:linux-TestImage2-16-v2' domain from ecco-vmhost-x7
  1483. 2013-05-28 13:55:50|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x7:
  1484. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x7 'rm -rfv /tmp/vcl/ecco\-vm\-x26_*\.xml' 2>&1
  1485. 2013-05-28 13:55:50|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1486. |12889|3243:3605|reinstall| removed `/tmp/vcl/ecco-vm-x26_16-v2.xml'
  1487. 2013-05-28 13:55:50|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x7, returning (0, "removed `/tmp/vcl/ecco-vm-x26_...")
  1488. 2013-05-28 13:55:50|12889|3243:3605|reinstall|Linux.pm:delete_file(1504)|deleted '/tmp/vcl/ecco-vm-x26_*.xml' on ecco-vmhost-x7
  1489. 2013-05-28 13:55:50|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x7:
  1490. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x7 'stat /tmp/vcl/ecco\-vm\-x26_*\.xml' 2>&1
  1491. 2013-05-28 13:55:50|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1492. |12889|3243:3605|reinstall| stat: cannot stat `/tmp/vcl/ecco-vm-x26_*.xml': No such file or directory
  1493. 2013-05-28 13:55:50|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vmhost-x7, command:
  1494. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x7 'stat /tmp/vcl/ecco\-vm\-x26_*\.xml' 2>&1
  1495. |12889|3243:3605|reinstall| returning (1, "stat: cannot stat `/tmp/vcl/ec...")
  1496. 2013-05-28 13:55:50|12889|3243:3605|reinstall|libvirt.pm:delete_existing_domains(1460)|deleted existing domains created for ecco-vm-x26 on ecco-vmhost-x7
  1497. 2013-05-28 13:55:50|12889|3243:3605|reinstall|OS.pm:find_files(2917)|attempting to find files on ecco-vmhost-x7, base directory path: '/images/vm/', pattern: linux-TestImage2-16-v2.*, command: /usr/bin/find "/images/vm/" -type f -iname "linux-TestImage2-16-v2.*"
  1498. 2013-05-28 13:55:50|12889|3243:3605|reinstall|OS.pm:find_files(2945)|files found: 1, base directory: '/images/vm/', pattern: 'linux-TestImage2-16-v2.*'
  1499. |12889|3243:3605|reinstall| command: '/usr/bin/find "/images/vm/" -type f -iname "linux-TestImage2-16-v2.*"'
  1500. 2013-05-28 13:55:50|12889|3243:3605|reinstall|libvirt.pm:get_master_image_file_path(1166)|found master image file on ecco-vmhost-x7: /images/vm/linux-TestImage2-16-v2.qcow2
  1501. 2013-05-28 13:55:50|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x7:
  1502. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x7 'stat /images/vm/linux\-TestImage2\-16\-v2\.qcow2' 2>&1
  1503. 2013-05-28 13:55:50|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1504. |12889|3243:3605|reinstall| File: `/images/vm/linux-TestImage2-16-v2.qcow2'
  1505. |12889|3243:3605|reinstall| Size: 3498573824 Blocks: 6832920 IO Block: 65536 regular file
  1506. |12889|3243:3605|reinstall| Device: 1dh/29d Inode: 122683637 Links: 2
  1507. |12889|3243:3605|reinstall| Access: (0644/-rw-r--r--) Uid: ( 99/ nobody) Gid: ( 99/ nobody)
  1508. |12889|3243:3605|reinstall| Access: 2013-05-27 14:00:22.909942374 -0400
  1509. |12889|3243:3605|reinstall| Modify: 2013-05-27 13:19:17.866852434 -0400
  1510. |12889|3243:3605|reinstall| Change: 2013-05-27 14:00:01.634693243 -0400
  1511. 2013-05-28 13:55:50|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x7, returning (0, "File: `/images/vm/linux-TestIm...")
  1512. 2013-05-28 13:55:50|12889|3243:3605|reinstall|Linux.pm:file_exists(1449)|'/images/vm/linux-TestImage2-16-v2.qcow2' exists on ecco-vmhost-x7, files: 1, directories: 0, links: 0
  1513. 2013-05-28 13:55:50|12889|3243:3605|reinstall|KVM.pm:pre_define(192)|master image file exists in the datastore on ecco-vmhost-x7: /images/vm/linux-TestImage2-16-v2.qcow2
  1514. 2013-05-28 13:55:50|12889|3243:3605|reinstall|KVM.pm:create_copy_on_write_image(670)|creating copy on write image on ecco-vmhost-x7
  1515. |12889|3243:3605|reinstall| master disk image: /images/vm/linux-TestImage2-16-v2.qcow2
  1516. |12889|3243:3605|reinstall| copy on write image: /var/lib/libvirt/images/ecco-vm-x26_16-v2.qcow2
  1517. |12889|3243:3605|reinstall| format: qcow2
  1518. 2013-05-28 13:55:50|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x7:
  1519. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x7 'qemu-img create -f qcow2 -b "/images/vm/linux-TestImage2-16-v2.qcow2" "/var/lib/libvirt/images/ecco-vm-x26_16-v2.qcow2"' 2>&1
  1520. 2013-05-28 13:55:50|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1521. |12889|3243:3605|reinstall| Formatting '/var/lib/libvirt/images/ecco-vm-x26_16-v2.qcow2', fmt=qcow2 size=32212254720 backing_file='/images/vm/linux-TestImage2-16-v2.qcow2' encryption=off cluster_size=65536
  1522. 2013-05-28 13:55:50|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x7, returning (0, "Formatting '/var/lib/libvirt/i...")
  1523. 2013-05-28 13:55:50|12889|3243:3605|reinstall|KVM.pm:create_copy_on_write_image(682)|created copy on write image: /var/lib/libvirt/images/ecco-vm-x26_16-v2.qcow2, output:
  1524. |12889|3243:3605|reinstall| Formatting '/var/lib/libvirt/images/ecco-vm-x26_16-v2.qcow2', fmt=qcow2 size=32212254720 backing_file='/images/vm/linux-TestImage2-16-v2.qcow2' encryption=off cluster_size=65536
  1525. 2013-05-28 13:55:50|12889|3243:3605|reinstall|utils.pm:insertloadlog(3665)|inserted computer=40, transfervm, performed libvirt driver tasks before defining domain
  1526. 2013-05-28 13:55:50|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x7:
  1527. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x7 'ls -d --color=never "/tmp/vcl" 2>&1 || mkdir -p "/tmp/vcl" 2>&1 && ls -d --color=never "/tmp/vcl"' 2>&1
  1528. 2013-05-28 13:55:50|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1529. |12889|3243:3605|reinstall| /tmp/vcl
  1530. |12889|3243:3605|reinstall| /tmp/vcl
  1531. 2013-05-28 13:55:50|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x7, returning (0, "/tmp/vcl /tmp/vcl")
  1532. 2013-05-28 13:55:50|12889|3243:3605|reinstall|Linux.pm:create_directory(1573)|directory already exists on ecco-vmhost-x7: '/tmp/vcl'
  1533. 2013-05-28 13:55:50|12889|3243:3605|reinstall|OS.pm:create_text_file(1851)|created file on ecco-vmhost-x7: /tmp/vcl/ecco-vm-x26_16-v2.xml
  1534. 2013-05-28 13:55:50|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x7:
  1535. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x7 'virsh define "/tmp/vcl/ecco-vm-x26_16-v2.xml"' 2>&1
  1536. 2013-05-28 13:55:51|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1537. |12889|3243:3605|reinstall| Domain ecco-vm-x26:linux-TestImage2-16-v2 defined from /tmp/vcl/ecco-vm-x26_16-v2.xml
  1538. 2013-05-28 13:55:51|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x7, returning (0, "Domain ecco-vm-x26:linux-TestI...")
  1539. 2013-05-28 13:55:51|12889|3243:3605|reinstall|libvirt.pm:define_domain(1384)|defined domain on ecco-vmhost-x7, output:
  1540. |12889|3243:3605|reinstall| Domain ecco-vm-x26:linux-TestImage2-16-v2 defined from /tmp/vcl/ecco-vm-x26_16-v2.xml
  1541. 2013-05-28 13:55:51|12889|3243:3605|reinstall|libvirt.pm:define_domain(1400)|parsed domain name from 'virsh define' output: 'ecco-vm-x26:linux-TestImage2-16-v2'
  1542. 2013-05-28 13:55:51|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x7:
  1543. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x7 'virsh autostart "ecco-vm-x26:linux-TestImage2-16-v2"' 2>&1
  1544. 2013-05-28 13:55:51|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1545. |12889|3243:3605|reinstall| Domain ecco-vm-x26:linux-TestImage2-16-v2 marked as autostarted
  1546. 2013-05-28 13:55:51|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x7, returning (0, "Domain ecco-vm-x26:linux-TestI...")
  1547. 2013-05-28 13:55:51|12889|3243:3605|reinstall|libvirt.pm:define_domain(1410)|configured 'ecco-vm-x26:linux-TestImage2-16-v2' domain to autostart on ecco-vmhost-x7, output:
  1548. |12889|3243:3605|reinstall| Domain ecco-vm-x26:linux-TestImage2-16-v2 marked as autostarted
  1549. 2013-05-28 13:55:51|12889|3243:3605|reinstall|utils.pm:insertloadlog(3665)|inserted computer=40, vmsetupconfig, defined ecco-vm-x26 domain node ecco-vmhost-x7
  1550. 2013-05-28 13:55:51|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x7:
  1551. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x7 'virsh start "ecco-vm-x26:linux-TestImage2-16-v2"' 2>&1
  1552. 2013-05-28 13:55:51|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1553. |12889|3243:3605|reinstall| Domain ecco-vm-x26:linux-TestImage2-16-v2 started
  1554. 2013-05-28 13:55:51|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x7, returning (0, "Domain ecco-vm-x26:linux-TestI...")
  1555. 2013-05-28 13:55:51|12889|3243:3605|reinstall|libvirt.pm:power_on(673)|started 'ecco-vm-x26:linux-TestImage2-16-v2' domain on ecco-vmhost-x7
  1556. 2013-05-28 13:55:51|12889|3243:3605|reinstall|utils.pm:insertloadlog(3665)|inserted computer=40, startvm, powered on ecco-vm-x26 domain node ecco-vmhost-x7
  1557. 2013-05-28 13:55:51|12889|3243:3605|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: 'Kernel IP routing table'
  1558. 2013-05-28 13:55:51|12889|3243:3605|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: 'Destination Gateway Genmask Flags Metric Ref Use Iface'
  1559. 2013-05-28 13:55:51|12889|3243:3605|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '199.241.160.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1'
  1560. 2013-05-28 13:55:51|12889|3243:3605|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '10.18.20.0 0.0.0.0 255.255.254.0 U 0 0 0 eth0'
  1561. 2013-05-28 13:55:51|12889|3243:3605|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '169.254.0.0 0.0.0.0 255.255.0.0 U 1003 0 0 eth0'
  1562. 2013-05-28 13:55:51|12889|3243:3605|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '169.254.0.0 0.0.0.0 255.255.0.0 U 1004 0 0 eth1'
  1563. 2013-05-28 13:55:51|12889|3243:3605|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '10.0.0.0 10.18.21.254 255.0.0.0 UG 0 0 0 eth0'
  1564. 2013-05-28 13:55:51|12889|3243:3605|reinstall|Linux.pm:get_network_configuration(2703)|found default route configured for 'eth1' interface: 199.241.160.1
  1565. 2013-05-28 13:55:51|12889|3243:3605|reinstall|Linux.pm:get_network_configuration(2708)|retrieved network configuration:
  1566. |12889|3243:3605|reinstall| : {
  1567. |12889|3243:3605|reinstall| : "eth0" => {
  1568. |12889|3243:3605|reinstall| : "broadcast_address" => "10.18.21.255",
  1569. |12889|3243:3605|reinstall| : "ip_address" => {
  1570. |12889|3243:3605|reinstall| : "10.18.20.4" => "255.255.254.0"
  1571. |12889|3243:3605|reinstall| : },
  1572. |12889|3243:3605|reinstall| : "name" => "eth0",
  1573. |12889|3243:3605|reinstall| : "physical_address" => "34:40:b5:82:4e:7c"
  1574. |12889|3243:3605|reinstall| : },
  1575. |12889|3243:3605|reinstall| : "eth1" => {
  1576. |12889|3243:3605|reinstall| : "broadcast_address" => "199.241.160.255",
  1577. |12889|3243:3605|reinstall| : "default_gateway" => "199.241.160.1",
  1578. |12889|3243:3605|reinstall| : "ip_address" => {
  1579. |12889|3243:3605|reinstall| : "199.241.160.4" => "255.255.255.0"
  1580. |12889|3243:3605|reinstall| : },
  1581. |12889|3243:3605|reinstall| : "name" => "eth1",
  1582. |12889|3243:3605|reinstall| : "physical_address" => "34:40:b5:82:4e:7e"
  1583. |12889|3243:3605|reinstall| : },
  1584. |12889|3243:3605|reinstall| : "lo" => {
  1585. |12889|3243:3605|reinstall| : "name" => "lo"
  1586. |12889|3243:3605|reinstall| : },
  1587. |12889|3243:3605|reinstall| : "usb0" => {
  1588. |12889|3243:3605|reinstall| : "name" => "usb0",
  1589. |12889|3243:3605|reinstall| : "physical_address" => "36:40:b5:85:4e:7f"
  1590. |12889|3243:3605|reinstall| : }
  1591. |12889|3243:3605|reinstall| : }
  1592. 2013-05-28 13:55:51|12889|3243:3605|reinstall|DataStructure.pm:get_computer_private_ip_address(1641)|attempting to retrieve private IP address for computer: ecco-adm3
  1593. 2013-05-28 13:55:51|12889|3243:3605|reinstall|OS.pm:get_file_contents(1897)|retrieved 146 lines from file on ecco-adm3: '/etc/hosts'
  1594. 2013-05-28 13:55:51|12889|3243:3605|reinstall|DataStructure.pm:get_computer_private_ip_address(1697)|returning IP address from /etc/hosts file: 10.18.20.4
  1595. 2013-05-28 13:55:51|12889|3243:3605|reinstall|OS.pm:get_private_interface_name(1119)|determined private interface name: eth0 (10.18.20.4)
  1596. 2013-05-28 13:55:51|12889|3243:3605|reinstall|OS.pm:get_ip_address(1538)|returning private IP address: 10.18.20.4
  1597. 2013-05-28 13:55:51|12889|3243:3605|reinstall|Linux.pm:post_load(237)|initiating Linux post_load: linux-TestImage2-16-v2 on ecco-vm-x26
  1598. 2013-05-28 13:55:52|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:55:51
  1599. 2013-05-28 13:55:52|12889|3243:3605|reinstall|OS.pm:is_ssh_responding(653)|ecco-vm-x26 is NOT responding to SSH, ports 22 or 24 are both closed
  1600. 2013-05-28 13:55:52|12889|3243:3605|reinstall|OS.pm:wait_for_response(734)|waiting 30 seconds for ecco-vm-x26 to boot
  1601. 2013-05-28 13:55:57|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:55:57
  1602. 2013-05-28 13:56:02|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:56:02
  1603. 2013-05-28 13:56:07|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:56:07
  1604. 2013-05-28 13:56:07|12878|3243:3604|reinstall|OS.pm:wait_for_response(736)|waited 30 seconds for ecco-vm-x10 to boot
  1605. 2013-05-28 13:56:07|12878|3243:3604|reinstall|OS.pm:wait_for_response(740)|waiting for ecco-vm-x10 to respond to SSH, maximum of 600 seconds
  1606. 2013-05-28 13:56:07|12878|3243:3604|reinstall|Module.pm:code_loop_timeout(887)|waiting for ecco-vm-x10 to respond to SSH, maximum of 600 seconds
  1607. 2013-05-28 13:56:08|12878|3243:3604|reinstall|OS.pm:is_ssh_responding(678)|ecco-vm-x10 is responding to SSH, port 22: open, port 24: closed
  1608. 2013-05-28 13:56:08|12878|3243:3604|reinstall|Module.pm:code_loop_timeout(900)|waiting for ecco-vm-x10 to respond to SSH, code returned true
  1609. 2013-05-28 13:56:08|12878|3243:3604|reinstall|OS.pm:wait_for_response(750)|ecco-vm-x10 is responding to SSH after 32 seconds
  1610. 2013-05-28 13:56:08|12878|3243:3604|reinstall|OS.pm:update_public_ip_address(941)|IP configuration is set to dynamicDHCP, attempting to retrieve dynamic public IP address from ecco-vm-x10
  1611. 2013-05-28 13:56:08|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1612. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 '/sbin/ifconfig -a' 2>&1
  1613. 2013-05-28 13:56:08|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1614. |12878|3243:3604|reinstall| eth0 Link encap:Ethernet HWaddr 52:54:00:01:00:09
  1615. |12878|3243:3604|reinstall| inet addr:10.18.20.242 Bcast:10.18.21.255 Mask:255.255.254.0
  1616. |12878|3243:3604|reinstall| inet6 addr: fe80::5054:ff:fe01:9/64 Scope:Link
  1617. |12878|3243:3604|reinstall| UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
  1618. |12878|3243:3604|reinstall| RX packets:113 errors:0 dropped:0 overruns:0 frame:0
  1619. |12878|3243:3604|reinstall| TX packets:108 errors:0 dropped:0 overruns:0 carrier:0
  1620. |12878|3243:3604|reinstall| collisions:0 txqueuelen:1000
  1621. |12878|3243:3604|reinstall| RX bytes:18290 (17.8 KiB) TX bytes:14689 (14.3 KiB)
  1622. |12878|3243:3604|reinstall| Interrupt:11 Base address:0xc000
  1623. |12878|3243:3604|reinstall| eth1 Link encap:Ethernet HWaddr 52:54:00:01:80:09
  1624. |12878|3243:3604|reinstall| inet addr:199.241.160.134 Bcast:199.241.160.255 Mask:255.255.255.0
  1625. |12878|3243:3604|reinstall| inet6 addr: fe80::5054:ff:fe01:8009/64 Scope:Link
  1626. |12878|3243:3604|reinstall| UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
  1627. |12878|3243:3604|reinstall| RX packets:16 errors:0 dropped:0 overruns:0 frame:0
  1628. |12878|3243:3604|reinstall| TX packets:23 errors:0 dropped:0 overruns:0 carrier:0
  1629. |12878|3243:3604|reinstall| collisions:0 txqueuelen:1000
  1630. |12878|3243:3604|reinstall| RX bytes:1404 (1.3 KiB) TX bytes:3914 (3.8 KiB)
  1631. |12878|3243:3604|reinstall| Interrupt:11 Base address:0x2000
  1632. |12878|3243:3604|reinstall| lo Link encap:Local Loopback
  1633. |12878|3243:3604|reinstall| inet addr:127.0.0.1 Mask:255.0.0.0
  1634. |12878|3243:3604|reinstall| inet6 addr: ::1/128 Scope:Host
  1635. |12878|3243:3604|reinstall| UP LOOPBACK RUNNING MTU:16436 Metric:1
  1636. |12878|3243:3604|reinstall| RX packets:0 errors:0 dropped:0 overruns:0 frame:0
  1637. |12878|3243:3604|reinstall| TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
  1638. |12878|3243:3604|reinstall| collisions:0 txqueuelen:0
  1639. |12878|3243:3604|reinstall| RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
  1640. 2013-05-28 13:56:08|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "eth0 Link encap:Ethernet HWadd...")
  1641. 2013-05-28 13:56:08|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1642. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 '/sbin/route -n' 2>&1
  1643. 2013-05-28 13:56:08|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1644. |12878|3243:3604|reinstall| Kernel IP routing table
  1645. |12878|3243:3604|reinstall| Destination Gateway Genmask Flags Metric Ref Use Iface
  1646. |12878|3243:3604|reinstall| 199.241.160.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
  1647. |12878|3243:3604|reinstall| 10.18.20.0 0.0.0.0 255.255.254.0 U 0 0 0 eth0
  1648. |12878|3243:3604|reinstall| 169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0
  1649. |12878|3243:3604|reinstall| 169.254.0.0 0.0.0.0 255.255.0.0 U 1003 0 0 eth1
  1650. |12878|3243:3604|reinstall| 10.0.0.0 10.18.21.254 255.0.0.0 UG 0 0 0 eth0
  1651. |12878|3243:3604|reinstall| 0.0.0.0 199.241.160.1 0.0.0.0 UG 0 0 0 eth1
  1652. 2013-05-28 13:56:08|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "Kernel IP routing table Destin...")
  1653. 2013-05-28 13:56:08|12878|3243:3604|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: 'Kernel IP routing table'
  1654. 2013-05-28 13:56:08|12878|3243:3604|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: 'Destination Gateway Genmask Flags Metric Ref Use Iface'
  1655. 2013-05-28 13:56:08|12878|3243:3604|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '199.241.160.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1'
  1656. 2013-05-28 13:56:08|12878|3243:3604|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '10.18.20.0 0.0.0.0 255.255.254.0 U 0 0 0 eth0'
  1657. 2013-05-28 13:56:08|12878|3243:3604|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0'
  1658. 2013-05-28 13:56:08|12878|3243:3604|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '169.254.0.0 0.0.0.0 255.255.0.0 U 1003 0 0 eth1'
  1659. 2013-05-28 13:56:08|12878|3243:3604|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '10.0.0.0 10.18.21.254 255.0.0.0 UG 0 0 0 eth0'
  1660. 2013-05-28 13:56:08|12878|3243:3604|reinstall|Linux.pm:get_network_configuration(2703)|found default route configured for 'eth1' interface: 199.241.160.1
  1661. 2013-05-28 13:56:08|12878|3243:3604|reinstall|Linux.pm:get_network_configuration(2708)|retrieved network configuration:
  1662. |12878|3243:3604|reinstall| : {
  1663. |12878|3243:3604|reinstall| : "eth0" => {
  1664. |12878|3243:3604|reinstall| : "broadcast_address" => "10.18.21.255",
  1665. |12878|3243:3604|reinstall| : "ip_address" => {
  1666. |12878|3243:3604|reinstall| : "10.18.20.242" => "255.255.254.0"
  1667. |12878|3243:3604|reinstall| : },
  1668. |12878|3243:3604|reinstall| : "name" => "eth0",
  1669. |12878|3243:3604|reinstall| : "physical_address" => "52:54:00:01:00:09"
  1670. |12878|3243:3604|reinstall| : },
  1671. |12878|3243:3604|reinstall| : "eth1" => {
  1672. |12878|3243:3604|reinstall| : "broadcast_address" => "199.241.160.255",
  1673. |12878|3243:3604|reinstall| : "default_gateway" => "199.241.160.1",
  1674. |12878|3243:3604|reinstall| : "ip_address" => {
  1675. |12878|3243:3604|reinstall| : "199.241.160.134" => "255.255.255.0"
  1676. |12878|3243:3604|reinstall| : },
  1677. |12878|3243:3604|reinstall| : "name" => "eth1",
  1678. |12878|3243:3604|reinstall| : "physical_address" => "52:54:00:01:80:09"
  1679. |12878|3243:3604|reinstall| : },
  1680. |12878|3243:3604|reinstall| : "lo" => {
  1681. |12878|3243:3604|reinstall| : "name" => "lo"
  1682. |12878|3243:3604|reinstall| : }
  1683. |12878|3243:3604|reinstall| : }
  1684. 2013-05-28 13:56:08|12878|3243:3604|reinstall|DataStructure.pm:get_computer_private_ip_address(1641)|attempting to retrieve private IP address for computer: ecco-vm-x10
  1685. 2013-05-28 13:56:08|12878|3243:3604|reinstall|OS.pm:get_file_contents(1897)|retrieved 146 lines from file on ecco-adm3: '/etc/hosts'
  1686. 2013-05-28 13:56:08|12878|3243:3604|reinstall|DataStructure.pm:get_computer_private_ip_address(1697)|returning IP address from /etc/hosts file: 10.18.20.242
  1687. 2013-05-28 13:56:08|12878|3243:3604|reinstall|OS.pm:get_public_interface_name(1196)|'eth0' could not be the public interface, it is only assigned the private IP address
  1688. 2013-05-28 13:56:08|12878|3243:3604|reinstall|OS.pm:get_public_interface_name(1220)|'eth1' could potententially be public interface, not assigned private IP address
  1689. 2013-05-28 13:56:08|12878|3243:3604|reinstall|OS.pm:get_public_interface_name(1177)|interface 'lo' ignored because its name is 'lo'
  1690. 2013-05-28 13:56:08|12878|3243:3604|reinstall|OS.pm:get_public_interface_name(1238)|determined the public interface name: 'eth1'
  1691. |12878|3243:3604|reinstall| : {
  1692. |12878|3243:3604|reinstall| : "broadcast_address" => "199.241.160.255",
  1693. |12878|3243:3604|reinstall| : "default_gateway" => "199.241.160.1",
  1694. |12878|3243:3604|reinstall| : "ip_address" => {
  1695. |12878|3243:3604|reinstall| : "199.241.160.134" => "255.255.255.0"
  1696. |12878|3243:3604|reinstall| : },
  1697. |12878|3243:3604|reinstall| : "name" => "eth1",
  1698. |12878|3243:3604|reinstall| : "physical_address" => "52:54:00:01:80:09"
  1699. |12878|3243:3604|reinstall| : }
  1700. 2013-05-28 13:56:08|12878|3243:3604|reinstall|OS.pm:get_ip_address(1538)|returning public IP address: 199.241.160.134
  1701. 2013-05-28 13:56:08|12878|3243:3604|reinstall|OS.pm:update_public_ip_address(946)|retrieved public IP address from ecco-vm-x10 using the OS module: 199.241.160.134
  1702. 2013-05-28 13:56:08|12878|3243:3604|reinstall|OS.pm:update_public_ip_address(974)|public IP address in computer table is already correct for ecco-vm-x10: 199.241.160.134
  1703. 2013-05-28 13:56:08|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1704. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'file /etc/rc.local' 2>&1
  1705. 2013-05-28 13:56:08|12878|3243:3604|reinstall|OS.pm:execute(1994)|executed command: 'file /etc/rc.local', exit status: 0, output:
  1706. |12878|3243:3604|reinstall| /etc/rc.local: symbolic link to `rc.d/rc.local'
  1707. 2013-05-28 13:56:08|12878|3243:3604|reinstall|Linux.pm:generate_rc_local(2053)|confirmed /etc/rc.local is symbolic link
  1708. |12878|3243:3604|reinstall| /etc/rc.local: symbolic link to `rc.d/rc.local'
  1709. 2013-05-28 13:56:08|12878|3243:3604|reinstall|utils.pm:run_scp_command(5156)|attempting to copy file via SCP: '/tmp/3243.rc.local' --> 'ecco-vm-x10:/etc/rc.d/rc.local'
  1710. 2013-05-28 13:56:08|12878|3243:3604|reinstall|utils.pm:run_scp_command(5192)|copied file via SCP: '/tmp/3243.rc.local' --> 'ecco-vm-x10:/etc/rc.d/rc.local'
  1711. 2013-05-28 13:56:08|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1712. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'chmod +rx /etc/rc.d/rc.local' 2>&1
  1713. 2013-05-28 13:56:08|12878|3243:3604|reinstall|Linux.pm:generate_rc_local(2127)|executed chmod +rx /etc/rc.d/rc.local, exit status: 0
  1714. 2013-05-28 13:56:08|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1715. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  1716. 2013-05-28 13:56:08|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1717. |12878|3243:3604|reinstall| stat: cannot stat `/bin/systemctl': No such file or directory
  1718. 2013-05-28 13:56:08|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vm-x10, command:
  1719. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  1720. |12878|3243:3604|reinstall| returning (1, "stat: cannot stat `/bin/system...")
  1721. 2013-05-28 13:56:08|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1722. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 '/etc/rc.d/rc.local' 2>&1
  1723. 2013-05-28 13:56:11|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1724. |12878|3243:3604|reinstall| Stopping ext_sshd:[ OK ]
  1725. |12878|3243:3604|reinstall| Stopping sshd: [ OK ]
  1726. |12878|3243:3604|reinstall| Starting sshd: [ OK ]
  1727. |12878|3243:3604|reinstall| Starting ext_sshd: [ OK ]
  1728. 2013-05-28 13:56:11|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "Stopping ext_sshd:[ OK ] Stopp...")
  1729. 2013-05-28 13:56:11|12878|3243:3604|reinstall|Linux.pm:generate_rc_local(2156)|successfully ran /etc/rc.d/rc.local
  1730. 2013-05-28 13:56:11|12878|3243:3604|reinstall|DataStructure.pm:is_variable_set(1779)|variable is NOT set: timesource|ecco-adm3.sharcnet.ca
  1731. 2013-05-28 13:56:11|12878|3243:3604|reinstall|DataStructure.pm:is_variable_set(1791)|variable is set: timesource|global
  1732. 2013-05-28 13:56:11|12878|3243:3604|reinstall|DataStructure.pm:get_variable(1890)|retrieved variable 'timesource|global', serialization: none
  1733. 2013-05-28 13:56:11|12878|3243:3604|reinstall|Linux.pm:sync_date(1068)|time_source is time.nist.gov,time-a.nist.gov,time-b.nist.gov,time.windows.com set for timesource|ecco-adm3.sharcnet.ca
  1734. 2013-05-28 13:56:11|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1735. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'rdate -s time.nist.gov time-a.nist.gov time-b.nist.gov time.windows.com' 2>&1
  1736. 2013-05-28 13:56:11|12878|3243:3604|reinstall|OS.pm:execute(1994)|executed command: 'rdate -s time.nist.gov time-a.nist.gov time-b.nist.gov time.windows.com', exit status: 127, output:
  1737. |12878|3243:3604|reinstall| bash: rdate: command not found
  1738. 2013-05-28 13:56:11|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1739. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'cp /dev/null /etc/ntp/ntpservers;echo time.nist.gov >> /etc/ntp/ntpservers; echo time-a.nist.gov >> /etc/ntp/ntpservers; echo time-b.nist.gov >> /etc/ntp/ntpservers; echo time.windows.com >> /etc/ntp/ntpservers; /etc/init.d/ntpd restart' 2>&1
  1740. 2013-05-28 13:56:11|12878|3243:3604|reinstall|OS.pm:execute(1994)|executed command: 'cp /dev/null /etc/ntp/ntpservers;echo time.nist.gov >> /etc/ntp/ntpservers; echo time-a.nist.gov >> /etc/ntp/ntpservers; echo time-b.nist.gov >> /etc/ntp/ntpservers; echo time.windows.com >> /etc/ntp/ntpservers; /etc/init.d/ntpd restart', exit status: 0, output:
  1741. |12878|3243:3604|reinstall| Shutting down ntpd: [ OK ]
  1742. |12878|3243:3604|reinstall| Starting ntpd: [ OK ]
  1743. 2013-05-28 13:56:11|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1744. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'echo CMGBGEsjHVDeTQA | /usr/bin/passwd -f root --stdin' 2>&1
  1745. 2013-05-28 13:56:11|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1746. |12878|3243:3604|reinstall| Changing password for user root.
  1747. |12878|3243:3604|reinstall| passwd: all authentication tokens updated successfully.
  1748. 2013-05-28 13:56:11|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "Changing password for user roo...")
  1749. 2013-05-28 13:56:11|12878|3243:3604|reinstall|Linux.pm:changepasswd(1133)|changed password for account: root, output:
  1750. |12878|3243:3604|reinstall| Changing password for user root.
  1751. |12878|3243:3604|reinstall| passwd: all authentication tokens updated successfully.
  1752. 2013-05-28 13:56:11|12878|3243:3604|reinstall|Linux.pm:post_load(287)|successfully changed root password on ecco-vm-x10
  1753. 2013-05-28 13:56:11|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1754. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  1755. 2013-05-28 13:56:12|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1756. |12878|3243:3604|reinstall| stat: cannot stat `/bin/systemctl': No such file or directory
  1757. 2013-05-28 13:56:12|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vm-x10, command:
  1758. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  1759. |12878|3243:3604|reinstall| returning (1, "stat: cannot stat `/bin/system...")
  1760. 2013-05-28 13:56:12|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1761. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'service ext_sshd stop' 2>&1
  1762. 2013-05-28 13:56:12|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:56:12
  1763. 2013-05-28 13:56:12|12878|3243:3604|reinstall|Linux.pm:stop_service(3298)|service exists: ext_sshd
  1764. 2013-05-28 13:56:12|12878|3243:3604|reinstall|Linux.pm:post_load(295)|ext sshd stopped on ecco-vm-x10
  1765. 2013-05-28 13:56:12|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1766. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'sed -i -e "/^AllowUsers .*/d" /etc/ssh/external_sshd_config' 2>&1
  1767. 2013-05-28 13:56:12|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1768. 2013-05-28 13:56:12|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "")
  1769. 2013-05-28 13:56:12|12878|3243:3604|reinstall|Linux.pm:post_load(301)|cleared AllowUsers directive from external_sshd_config
  1770. 2013-05-28 13:56:12|12878|3243:3604|reinstall|Linux.pm:post_load(308)|calling clear_private_keys
  1771. 2013-05-28 13:56:12|12878|3243:3604|reinstall|Linux.pm:clear_private_keys(492)|perparing to clear known identity keys
  1772. 2013-05-28 13:56:12|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1773. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 '/bin/rm -f /root/.ssh/id_rsa /root/.ssh/id_rsa.pub' 2>&1
  1774. 2013-05-28 13:56:12|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1775. 2013-05-28 13:56:12|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "")
  1776. 2013-05-28 13:56:12|12878|3243:3604|reinstall|Linux.pm:clear_private_keys(500)|cleared any id_rsa keys from /root/.ssh
  1777. 2013-05-28 13:56:12|12878|3243:3604|reinstall|Linux.pm:post_load(310)|cleared known identity keys
  1778. 2013-05-28 13:56:12|12878|3243:3604|reinstall|OS.pm:get_public_interface_name(1150)|returning public interface name previously retrieved: eth1
  1779. 2013-05-28 13:56:12|12878|3243:3604|reinstall|OS.pm:get_ip_address(1538)|returning public IP address: 199.241.160.134
  1780. 2013-05-28 13:56:12|12878|3243:3604|reinstall|Linux.pm:update_public_hostname(424)|retrieved public IP address of ecco-vm-x10: 199.241.160.134
  1781. 2013-05-28 13:56:12|12878|3243:3604|reinstall|utils.pm:run_command(8710)|executed command: /bin/ipcalc --hostname 199.241.160.134, pid: 16824, exit status: 0, output:
  1782. |12878|3243:3604|reinstall| HOSTNAME=ecco-computer7.sharcnet.ca
  1783. 2013-05-28 13:56:12|12878|3243:3604|reinstall|Linux.pm:update_public_hostname(435)|determined registered public hostname of ecco-vm-x10 (199.241.160.134) by running ipcalc on the management node: 'ecco-computer7.sharcnet.ca'
  1784. 2013-05-28 13:56:12|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1785. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'hostname -v ecco-computer7.sharcnet.ca' 2>&1
  1786. 2013-05-28 13:56:12|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1787. |12878|3243:3604|reinstall| Setting hostname to `ecco-computer7.sharcnet.ca'
  1788. 2013-05-28 13:56:12|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "Setting hostname to `ecco-comp...")
  1789. 2013-05-28 13:56:12|12878|3243:3604|reinstall|Linux.pm:update_public_hostname(467)|set public hostname on ecco-vm-x10 to ecco-computer7.sharcnet.ca
  1790. 2013-05-28 13:56:12|12878|3243:3604|reinstall|Linux.pm:post_load(315)|Updated hostname
  1791. 2013-05-28 13:56:12|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1792. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /etc/init\.d/vcl_post_load' 2>&1
  1793. 2013-05-28 13:56:12|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1794. |12878|3243:3604|reinstall| File: `/etc/init.d/vcl_post_load'
  1795. |12878|3243:3604|reinstall| Size: 43 Blocks: 8 IO Block: 4096 regular file
  1796. |12878|3243:3604|reinstall| Device: 802h/2050d Inode: 7976 Links: 1
  1797. |12878|3243:3604|reinstall| Access: (0700/-rwx------) Uid: ( 0/ root) Gid: ( 0/ root)
  1798. |12878|3243:3604|reinstall| Access: 2013-05-27 08:15:46.167661786 -0400
  1799. |12878|3243:3604|reinstall| Modify: 2013-05-27 08:17:13.367712524 -0400
  1800. |12878|3243:3604|reinstall| Change: 2013-05-27 08:17:13.375594944 -0400
  1801. 2013-05-28 13:56:12|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "File: `/etc/init.d/vcl_post_lo...")
  1802. 2013-05-28 13:56:12|12878|3243:3604|reinstall|Linux.pm:file_exists(1449)|'/etc/init.d/vcl_post_load' exists on ecco-vm-x10, files: 1, directories: 0, links: 0
  1803. 2013-05-28 13:56:12|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1804. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /etc/init\.d/vcl_post_load' 2>&1
  1805. 2013-05-28 13:56:12|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1806. |12878|3243:3604|reinstall| File: `/etc/init.d/vcl_post_load'
  1807. |12878|3243:3604|reinstall| Size: 43 Blocks: 8 IO Block: 4096 regular file
  1808. |12878|3243:3604|reinstall| Device: 802h/2050d Inode: 7976 Links: 1
  1809. |12878|3243:3604|reinstall| Access: (0700/-rwx------) Uid: ( 0/ root) Gid: ( 0/ root)
  1810. |12878|3243:3604|reinstall| Access: 2013-05-27 08:15:46.167661786 -0400
  1811. |12878|3243:3604|reinstall| Modify: 2013-05-27 08:17:13.367712524 -0400
  1812. |12878|3243:3604|reinstall| Change: 2013-05-27 08:17:13.375594944 -0400
  1813. 2013-05-28 13:56:12|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "File: `/etc/init.d/vcl_post_lo...")
  1814. 2013-05-28 13:56:12|12878|3243:3604|reinstall|Linux.pm:file_exists(1449)|'/etc/init.d/vcl_post_load' exists on ecco-vm-x10, files: 1, directories: 0, links: 0
  1815. 2013-05-28 13:56:12|12878|3243:3604|reinstall|Linux.pm:run_script(1348)|script exists: /etc/init.d/vcl_post_load
  1816. 2013-05-28 13:56:12|12878|3243:3604|reinstall|Linux.pm:run_script(1358)|script name: vcl_post_load
  1817. 2013-05-28 13:56:12|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1818. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'ls -d --color=never "/root/VCL/Logs" 2>&1 || mkdir -p "/root/VCL/Logs" 2>&1 && ls -d --color=never "/root/VCL/Logs"' 2>&1
  1819. 2013-05-28 13:56:12|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1820. |12878|3243:3604|reinstall| /root/VCL/Logs
  1821. |12878|3243:3604|reinstall| /root/VCL/Logs
  1822. 2013-05-28 13:56:12|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "/root/VCL/Logs /root/VCL/Logs")
  1823. 2013-05-28 13:56:12|12878|3243:3604|reinstall|Linux.pm:create_directory(1573)|directory already exists on ecco-vm-x10: '/root/VCL/Logs'
  1824. 2013-05-28 13:56:12|12878|3243:3604|reinstall|Linux.pm:run_script(1372)|script log file path: /root/VCL/Logs/vcl_post_load.log
  1825. 2013-05-28 13:56:12|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1826. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'chmod +rx "/etc/init.d/vcl_post_load" && "/etc/init.d/vcl_post_load" >> "/root/VCL/Logs/vcl_post_load.log" 2>&1' 2>&1
  1827. 2013-05-28 13:56:14|12878|3243:3604|reinstall|OS.pm:execute(1994)|executed command: 'chmod +rx "/etc/init.d/vcl_post_load" && "/etc/init.d/vcl_post_load" >> "/root/VCL/Logs/vcl_post_load.log" 2>&1', exit status: 0, output:
  1828. 2013-05-28 13:56:14|12878|3243:3604|reinstall|Linux.pm:run_script(1379)|executed /etc/init.d/vcl_post_load, exit status: 0
  1829. 2013-05-28 13:56:14|12878|3243:3604|reinstall|Linux.pm:post_load(330)|ran /etc/init.d/vcl_post_load
  1830. 2013-05-28 13:56:14|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1831. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  1832. 2013-05-28 13:56:14|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1833. |12878|3243:3604|reinstall| stat: cannot stat `/bin/systemctl': No such file or directory
  1834. 2013-05-28 13:56:14|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vm-x10, command:
  1835. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  1836. |12878|3243:3604|reinstall| returning (1, "stat: cannot stat `/bin/system...")
  1837. 2013-05-28 13:56:14|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1838. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /sbin/chkconfig' 2>&1
  1839. 2013-05-28 13:56:14|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1840. |12878|3243:3604|reinstall| File: `/sbin/chkconfig'
  1841. |12878|3243:3604|reinstall| Size: 31728 Blocks: 64 IO Block: 4096 regular file
  1842. |12878|3243:3604|reinstall| Device: 802h/2050d Inode: 642 Links: 1
  1843. |12878|3243:3604|reinstall| Access: (0755/-rwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
  1844. |12878|3243:3604|reinstall| Access: 2013-05-28 13:58:10.443116161 -0400
  1845. |12878|3243:3604|reinstall| Modify: 2012-06-22 06:09:02.000000000 -0400
  1846. |12878|3243:3604|reinstall| Change: 2012-10-09 15:03:55.140697475 -0400
  1847. 2013-05-28 13:56:14|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "File: `/sbin/chkconfig' Size: ...")
  1848. 2013-05-28 13:56:14|12878|3243:3604|reinstall|Linux.pm:file_exists(1449)|'/sbin/chkconfig' exists on ecco-vm-x10, files: 1, directories: 0, links: 0
  1849. 2013-05-28 13:56:14|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1850. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 '/sbin/chkconfig --list iptables' 2>&1
  1851. 2013-05-28 13:56:15|12878|3243:3604|reinstall|Linux.pm:service_exists(3187)|'iptables' service exists
  1852. 2013-05-28 13:56:15|12878|3243:3604|reinstall|OS.pm:get_ip_address(1538)|returning private IP address: 10.18.20.4
  1853. 2013-05-28 13:56:15|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1854. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  1855. 2013-05-28 13:56:15|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1856. |12878|3243:3604|reinstall| stat: cannot stat `/bin/systemctl': No such file or directory
  1857. 2013-05-28 13:56:15|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vm-x10, command:
  1858. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  1859. |12878|3243:3604|reinstall| returning (1, "stat: cannot stat `/bin/system...")
  1860. 2013-05-28 13:56:15|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1861. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /sbin/chkconfig' 2>&1
  1862. 2013-05-28 13:56:15|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1863. |12878|3243:3604|reinstall| File: `/sbin/chkconfig'
  1864. |12878|3243:3604|reinstall| Size: 31728 Blocks: 64 IO Block: 4096 regular file
  1865. |12878|3243:3604|reinstall| Device: 802h/2050d Inode: 642 Links: 1
  1866. |12878|3243:3604|reinstall| Access: (0755/-rwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
  1867. |12878|3243:3604|reinstall| Access: 2013-05-28 13:58:10.443116161 -0400
  1868. |12878|3243:3604|reinstall| Modify: 2012-06-22 06:09:02.000000000 -0400
  1869. |12878|3243:3604|reinstall| Change: 2012-10-09 15:03:55.140697475 -0400
  1870. 2013-05-28 13:56:15|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "File: `/sbin/chkconfig' Size: ...")
  1871. 2013-05-28 13:56:15|12878|3243:3604|reinstall|Linux.pm:file_exists(1449)|'/sbin/chkconfig' exists on ecco-vm-x10, files: 1, directories: 0, links: 0
  1872. 2013-05-28 13:56:15|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1873. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 '/sbin/chkconfig --list iptables' 2>&1
  1874. 2013-05-28 13:56:15|12878|3243:3604|reinstall|Linux.pm:service_exists(3187)|'iptables' service exists
  1875. 2013-05-28 13:56:15|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1876. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'iptables -L --line-number -n' 2>&1
  1877. 2013-05-28 13:56:15|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1878. |12878|3243:3604|reinstall| Chain INPUT (policy ACCEPT)
  1879. |12878|3243:3604|reinstall| num target prot opt source destination
  1880. |12878|3243:3604|reinstall| 1 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,RELATED,ESTABLISHED tcp dpt:5900
  1881. |12878|3243:3604|reinstall| 2 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,RELATED,ESTABLISHED tcp dpt:22
  1882. |12878|3243:3604|reinstall| 3 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
  1883. |12878|3243:3604|reinstall| 4 ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0
  1884. |12878|3243:3604|reinstall| 5 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
  1885. |12878|3243:3604|reinstall| 6 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
  1886. |12878|3243:3604|reinstall| Chain FORWARD (policy ACCEPT)
  1887. |12878|3243:3604|reinstall| num target prot opt source destination
  1888. |12878|3243:3604|reinstall| 1 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
  1889. |12878|3243:3604|reinstall| Chain OUTPUT (policy ACCEPT)
  1890. |12878|3243:3604|reinstall| num target prot opt source destination
  1891. 2013-05-28 13:56:15|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "Chain INPUT (policy ACCEPT) nu...")
  1892. 2013-05-28 13:56:15|12878|3243:3604|reinstall|Linux.pm:get_firewall_configuration(3992)|output Chain = INPUT
  1893. 2013-05-28 13:56:15|12878|3243:3604|reinstall|Linux.pm:get_firewall_configuration(4008)|output rule: 1, ACCEPT, tcp, 0.0.0.0/0, 0.0.0.0/0, 5900
  1894. 2013-05-28 13:56:15|12878|3243:3604|reinstall|Linux.pm:get_firewall_configuration(4008)|output rule: 2, ACCEPT, tcp, 0.0.0.0/0, 0.0.0.0/0, 22
  1895. 2013-05-28 13:56:17|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:56:17
  1896. 2013-05-28 13:56:17|12878|3243:3604|reinstall|Linux.pm:get_firewall_configuration(3992)|output Chain = FORWARD
  1897. 2013-05-28 13:56:17|12878|3243:3604|reinstall|Linux.pm:get_firewall_configuration(3992)|output Chain = OUTPUT
  1898. 2013-05-28 13:56:17|12878|3243:3604|reinstall|Linux.pm:get_firewall_configuration(4051)|retrieved firewall configuration from ecco-vm-x10:
  1899. |12878|3243:3604|reinstall| : {
  1900. |12878|3243:3604|reinstall| : "FORWARD" => {
  1901. |12878|3243:3604|reinstall| : 1 => {
  1902. |12878|3243:3604|reinstall| : "all" => {
  1903. |12878|3243:3604|reinstall| : "any" => {
  1904. |12878|3243:3604|reinstall| : "destination" => "0.0.0.0/0",
  1905. |12878|3243:3604|reinstall| : "name" => "any",
  1906. |12878|3243:3604|reinstall| : "number" => 1,
  1907. |12878|3243:3604|reinstall| : "scope" => "0.0.0.0/0",
  1908. |12878|3243:3604|reinstall| : "target" => "REJECT"
  1909. |12878|3243:3604|reinstall| : }
  1910. |12878|3243:3604|reinstall| : }
  1911. |12878|3243:3604|reinstall| : }
  1912. |12878|3243:3604|reinstall| : },
  1913. |12878|3243:3604|reinstall| : "INPUT" => {
  1914. |12878|3243:3604|reinstall| : 1 => {
  1915. |12878|3243:3604|reinstall| : "tcp" => {
  1916. |12878|3243:3604|reinstall| : 5900 => {
  1917. |12878|3243:3604|reinstall| : "destination" => "0.0.0.0/0",
  1918. |12878|3243:3604|reinstall| : "name" => 5900,
  1919. |12878|3243:3604|reinstall| : "number" => 1,
  1920. |12878|3243:3604|reinstall| : "scope" => "0.0.0.0/0",
  1921. |12878|3243:3604|reinstall| : "target" => "ACCEPT"
  1922. |12878|3243:3604|reinstall| : }
  1923. |12878|3243:3604|reinstall| : }
  1924. |12878|3243:3604|reinstall| : },
  1925. |12878|3243:3604|reinstall| : 2 => {
  1926. |12878|3243:3604|reinstall| : "tcp" => {
  1927. |12878|3243:3604|reinstall| : 22 => {
  1928. |12878|3243:3604|reinstall| : "destination" => "0.0.0.0/0",
  1929. |12878|3243:3604|reinstall| : "name" => "ssh",
  1930. |12878|3243:3604|reinstall| : "number" => 2,
  1931. |12878|3243:3604|reinstall| : "scope" => "0.0.0.0/0",
  1932. |12878|3243:3604|reinstall| : "target" => "ACCEPT"
  1933. |12878|3243:3604|reinstall| : }
  1934. |12878|3243:3604|reinstall| : }
  1935. |12878|3243:3604|reinstall| : },
  1936. |12878|3243:3604|reinstall| : 3 => {
  1937. |12878|3243:3604|reinstall| : "all" => {
  1938. |12878|3243:3604|reinstall| : "any" => {
  1939. |12878|3243:3604|reinstall| : "destination" => "0.0.0.0/0",
  1940. |12878|3243:3604|reinstall| : "name" => "any",
  1941. |12878|3243:3604|reinstall| : "number" => 3,
  1942. |12878|3243:3604|reinstall| : "scope" => "0.0.0.0/0",
  1943. |12878|3243:3604|reinstall| : "target" => "ACCEPT"
  1944. |12878|3243:3604|reinstall| : }
  1945. |12878|3243:3604|reinstall| : }
  1946. |12878|3243:3604|reinstall| : },
  1947. |12878|3243:3604|reinstall| : 4 => {
  1948. |12878|3243:3604|reinstall| : "icmp" => {
  1949. |12878|3243:3604|reinstall| : "any" => {
  1950. |12878|3243:3604|reinstall| : "destination" => "0.0.0.0/0",
  1951. |12878|3243:3604|reinstall| : "name" => "any",
  1952. |12878|3243:3604|reinstall| : "number" => 4,
  1953. |12878|3243:3604|reinstall| : "scope" => "0.0.0.0/0",
  1954. |12878|3243:3604|reinstall| : "target" => "ACCEPT"
  1955. |12878|3243:3604|reinstall| : }
  1956. |12878|3243:3604|reinstall| : }
  1957. |12878|3243:3604|reinstall| : },
  1958. |12878|3243:3604|reinstall| : 5 => {
  1959. |12878|3243:3604|reinstall| : "all" => {
  1960. |12878|3243:3604|reinstall| : "any" => {
  1961. |12878|3243:3604|reinstall| : "destination" => "0.0.0.0/0",
  1962. |12878|3243:3604|reinstall| : "name" => "any",
  1963. |12878|3243:3604|reinstall| : "number" => 5,
  1964. |12878|3243:3604|reinstall| : "scope" => "0.0.0.0/0",
  1965. |12878|3243:3604|reinstall| : "target" => "ACCEPT"
  1966. |12878|3243:3604|reinstall| : }
  1967. |12878|3243:3604|reinstall| : }
  1968. |12878|3243:3604|reinstall| : },
  1969. |12878|3243:3604|reinstall| : 6 => {
  1970. |12878|3243:3604|reinstall| : "all" => {
  1971. |12878|3243:3604|reinstall| : "any" => {
  1972. |12878|3243:3604|reinstall| : "destination" => "0.0.0.0/0",
  1973. |12878|3243:3604|reinstall| : "name" => "any",
  1974. |12878|3243:3604|reinstall| : "number" => 6,
  1975. |12878|3243:3604|reinstall| : "scope" => "0.0.0.0/0",
  1976. |12878|3243:3604|reinstall| : "target" => "REJECT"
  1977. |12878|3243:3604|reinstall| : }
  1978. |12878|3243:3604|reinstall| : }
  1979. |12878|3243:3604|reinstall| : }
  1980. |12878|3243:3604|reinstall| : }
  1981. |12878|3243:3604|reinstall| : }
  1982. 2013-05-28 13:56:17|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1983. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'ls -d --color=never "/etc/sysconfig" 2>&1 || mkdir -p "/etc/sysconfig" 2>&1 && ls -d --color=never "/etc/sysconfig"' 2>&1
  1984. 2013-05-28 13:56:18|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1985. |12878|3243:3604|reinstall| /etc/sysconfig
  1986. |12878|3243:3604|reinstall| /etc/sysconfig
  1987. 2013-05-28 13:56:18|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "/etc/sysconfig /etc/sysconfig")
  1988. 2013-05-28 13:56:18|12878|3243:3604|reinstall|Linux.pm:create_directory(1573)|directory already exists on ecco-vm-x10: '/etc/sysconfig'
  1989. 2013-05-28 13:56:18|12878|3243:3604|reinstall|OS.pm:copy_file(2781)|attempting to copy file on ecco-vm-x10: '/etc/sysconfig/iptables' -> '/etc/sysconfig/iptables_pre_any'
  1990. 2013-05-28 13:56:18|12878|3243:3604|reinstall|OS.pm:copy_file(2792)|copied file on ecco-vm-x10: '/etc/sysconfig/iptables' --> '/etc/sysconfig/iptables_pre_any'
  1991. 2013-05-28 13:56:18|12878|3243:3604|reinstall|Linux.pm:enable_firewall_port(3703)|backed up original iptables file to: '/etc/sysconfig/iptables_pre_any'
  1992. 2013-05-28 13:56:18|12878|3243:3604|reinstall|Linux.pm:enable_firewall_port(3711)|attempting to execute command on ecco-vm-x10: '/sbin/iptables -I INPUT 1 -m state --state NEW,RELATED,ESTABLISHED -m tcp -p tcp -j ACCEPT -s 10.18.20.4'
  1993. 2013-05-28 13:56:18|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1994. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 '/sbin/iptables -I INPUT 1 -m state --state NEW,RELATED,ESTABLISHED -m tcp -p tcp -j ACCEPT -s 10.18.20.4' 2>&1
  1995. 2013-05-28 13:56:18|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  1996. 2013-05-28 13:56:18|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "")
  1997. 2013-05-28 13:56:18|12878|3243:3604|reinstall|Linux.pm:enable_firewall_port(3713)|executed command on ecco-vm-x10: '/sbin/iptables -I INPUT 1 -m state --state NEW,RELATED,ESTABLISHED -m tcp -p tcp -j ACCEPT -s 10.18.20.4'
  1998. 2013-05-28 13:56:18|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  1999. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 '/sbin/iptables-save > /etc/sysconfig/iptables' 2>&1
  2000. 2013-05-28 13:56:18|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2001. 2013-05-28 13:56:18|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "")
  2002. 2013-05-28 13:56:18|12878|3243:3604|reinstall|Linux.pm:enable_firewall_port(3723)|executed command /sbin/iptables-save > /etc/sysconfig/iptables on ecco-vm-x10
  2003. 2013-05-28 13:56:18|12878|3243:3604|reinstall|Linux.pm:post_load(334)|added MN_Priv_IP 10.18.20.4 to firewall on ecco-vm-x10
  2004. 2013-05-28 13:56:18|12878|3243:3604|reinstall|Linux.pm:activate_interfaces(2561)|attempting to find network interfaces with an active link
  2005. 2013-05-28 13:56:18|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2006. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'ip link' 2>&1
  2007. 2013-05-28 13:56:18|12878|3243:3604|reinstall|OS.pm:execute(1994)|executed command: 'ip link', exit status: 0, output:
  2008. |12878|3243:3604|reinstall| 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
  2009. |12878|3243:3604|reinstall| link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
  2010. |12878|3243:3604|reinstall| 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
  2011. |12878|3243:3604|reinstall| link/ether 52:54:00:01:00:09 brd ff:ff:ff:ff:ff:ff
  2012. |12878|3243:3604|reinstall| 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
  2013. |12878|3243:3604|reinstall| link/ether 52:54:00:01:80:09 brd ff:ff:ff:ff:ff:ff
  2014. 2013-05-28 13:56:18|12878|3243:3604|reinstall|Linux.pm:activate_interfaces(2570)|found interface names:
  2015. |12878|3243:3604|reinstall| eth0
  2016. |12878|3243:3604|reinstall| eth1
  2017. 2013-05-28 13:56:18|12878|3243:3604|reinstall|OS.pm:find_files(2917)|attempting to find files on ecco-vm-x10, base directory path: '/etc/sysconfig/network-scripts/', pattern: ifcfg-eth*, command: /usr/bin/find "/etc/sysconfig/network-scripts/" -type f -iname "ifcfg-eth*"
  2018. 2013-05-28 13:56:18|12878|3243:3604|reinstall|OS.pm:find_files(2945)|files found: 2, base directory: '/etc/sysconfig/network-scripts/', pattern: 'ifcfg-eth*'
  2019. |12878|3243:3604|reinstall| command: '/usr/bin/find "/etc/sysconfig/network-scripts/" -type f -iname "ifcfg-eth*"'
  2020. 2013-05-28 13:56:18|12878|3243:3604|reinstall|Linux.pm:activate_interfaces(2575)|found existing ifcfg-eth* files:
  2021. |12878|3243:3604|reinstall| /etc/sysconfig/network-scripts/ifcfg-eth1
  2022. |12878|3243:3604|reinstall| /etc/sysconfig/network-scripts/ifcfg-eth0
  2023. 2013-05-28 13:56:18|12878|3243:3604|reinstall|Linux.pm:activate_interfaces(2583)|ifcfg file already exists for eth0
  2024. 2013-05-28 13:56:18|12878|3243:3604|reinstall|Linux.pm:activate_interfaces(2583)|ifcfg file already exists for eth1
  2025. 2013-05-28 13:56:18|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2026. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'sed -i -e '/vcld_post_load.*/d' currentimage.txt && echo >> currentimage.txt && echo "vcld_post_load=success (Tue May 28 13:56:18 2013)" >> currentimage.txt && sed -i -e '/^[\s\r\n]*$/d' currentimage.txt' 2>&1
  2027. 2013-05-28 13:56:18|12878|3243:3604|reinstall|OS.pm:set_vcld_post_load_status(1056)|added line to currentimage.txt on ecco-vm-x10: 'vcld_post_load=success (Tue May 28 13:56:18 2013)'
  2028. 2013-05-28 13:56:18|12878|3243:3604|reinstall|utils.pm:insertloadlog(3665)|inserted computer=18, loadimagecomplete, performed OS post-load tasks \'ecco-vm-x10:linux-TestImage2-16-v2\' domain on ecco-vmhost-x3
  2029. 2013-05-28 13:56:18|12878|3243:3604|reinstall|new.pm:reload_image(691)|linux-TestImage2-16-v2 was successfully reloaded on ecco-vm-x10
  2030. 2013-05-28 13:56:18|12878|3243:3604|reinstall|utils.pm:insertloadlog(3665)|inserted computer=18, loadimagecomplete, linux-TestImage2-16-v2 was successfully reloaded on ecco-vm-x10
  2031. 2013-05-28 13:56:18|12878|3243:3604|reinstall|utils.pm:update_currentimage(5685)|updating computer 18: image=16, imagerevision=174
  2032. 2013-05-28 13:56:18|12878|3243:3604|reinstall|utils.pm:update_currentimage(5715)|updated currentimageid and imagerevision id for computer id 18
  2033. 2013-05-28 13:56:18|12878|3243:3604|reinstall|new.pm:reload_image(702)|updated computer table for ecco-vm-x10: currentimageid=16
  2034. 2013-05-28 13:56:18|12878|3243:3604|reinstall|new.pm:reload_image(722)|returning 1
  2035. 2013-05-28 13:56:18|12878|3243:3604|reinstall|new.pm:process(292)|ecco-vm-x10 is loaded with linux-TestImage2-16-v2
  2036. 2013-05-28 13:56:18|12878|3243:3604|reinstall|utils.pm:insertloadlog(3665)|inserted computer=18, nodeready, ecco-vm-x10 is loaded with linux-TestImage2-16-v2
  2037. 2013-05-28 13:56:18|12878|3243:3604|reinstall|utils.pm:insertloadlog(3665)|inserted computer=18, info, cluster based reservation
  2038. 2013-05-28 13:56:18|12878|3243:3604|reinstall|utils.pm:reservations_ready(8577)|all reservations for request 3243 are ready, returning 1:
  2039. |12878|3243:3604|reinstall| reservation 3243:3604: nodeready
  2040. |12878|3243:3604|reinstall| reservation 3243:3605: nodeready
  2041. 2013-05-28 13:56:18|12878|3243:3604|reinstall|new.pm:wait_for_child_reservations(1237)|ready: all child reservations are ready according to computerloadlog, returning 1
  2042. 2013-05-28 13:56:18|12878|3243:3604|reinstall|new.pm:process(351)|done waiting for child reservations, they are all ready
  2043. 2013-05-28 13:56:18|12878|3243:3604|reinstall|new.pm:process(374)|request_state_name= reinstall
  2044. 2013-05-28 13:56:18|12878|3243:3604|reinstall|new.pm:process(386)|ecco-vm-x10 next image is already set to linux-TestImage2-16-v2
  2045. 2013-05-28 13:56:18|12878|3243:3604|reinstall|DataStructure.pm:is_parent_reservation(1004)|returning true: parent reservation ID for this request: 3604
  2046. 2013-05-28 13:56:18|12878|3243:3604|reinstall|new.pm:reserve_computer(955)|user_standalone=0, image OS type=linux
  2047. 2013-05-28 13:56:18|12878|3243:3604|reinstall|OS.pm:get_public_interface_name(1150)|returning public interface name previously retrieved: eth1
  2048. 2013-05-28 13:56:18|12878|3243:3604|reinstall|OS.pm:get_ip_address(1538)|returning public IP address: 199.241.160.134
  2049. 2013-05-28 13:56:18|12878|3243:3604|reinstall|new.pm:confirm_public_ip_address(1325)|retrieved public IP address from ecco-vm-x10 using the OS module: 199.241.160.134
  2050. 2013-05-28 13:56:18|12878|3243:3604|reinstall|utils.pm:insertloadlog(3665)|inserted computer=18, info, node ready adding user account
  2051. 2013-05-28 13:56:18|12878|3243:3604|reinstall|new.pm:reserve_computer(1003)|calling OS module's reserve() subroutine
  2052. 2013-05-28 13:56:18|12878|3243:3604|reinstall|Linux.pm:reserve(931)|Enterered reserve() in the Linux OS module
  2053. 2013-05-28 13:56:18|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2054. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'groupadd vcl' 2>&1
  2055. 2013-05-28 13:56:18|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2056. |12878|3243:3604|reinstall| groupadd: group 'vcl' already exists
  2057. 2013-05-28 13:56:18|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vm-x10, command:
  2058. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'groupadd vcl' 2>&1
  2059. |12878|3243:3604|reinstall| returning (9, "groupadd: group 'vcl' already ...")
  2060. 2013-05-28 13:56:18|12878|3243:3604|reinstall|Linux.pm:add_vcl_usergroup(1272)|successfully added the vcl user group
  2061. 2013-05-28 13:56:18|12878|3243:3604|reinstall|Linux.pm:create_user(2909)|username not provided, pulling from datastructure
  2062.  
  2063. |12878|3243:3604|reinstall| ---- WARNING ----
  2064. |12878|3243:3604|reinstall| 2013-05-28 13:56:18|12878|3243:3604|reinstall|DataStructure.pm:_automethod(825)|corresponding data has not been initialized for get_reservation_password: $self->request_data->{reservation}{3604}{pw}
  2065. |12878|3243:3604|reinstall| ( 0) DataStructure.pm, _automethod (line: 825)
  2066. |12878|3243:3604|reinstall| (-1) Autoload.pm, __ANON__ (line: 80)
  2067. |12878|3243:3604|reinstall| (-2) Linux.pm, create_user (line: 2911)
  2068. |12878|3243:3604|reinstall| (-3) Linux.pm, reserve (line: 943)
  2069. |12878|3243:3604|reinstall| (-4) new.pm, reserve_computer (line: 1004)
  2070. |12878|3243:3604|reinstall| (-5) new.pm, process (line: 408)
  2071.  
  2072. 2013-05-28 13:56:18|12878|3243:3604|reinstall|Linux.pm:create_user(2919)|user_uid not provided, pulling from datastructure
  2073. 2013-05-28 13:56:18|12878|3243:3604|reinstall|Linux.pm:create_user(2924)|user_standalone not provided, pulling from datastructure
  2074. 2013-05-28 13:56:18|12878|3243:3604|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2075. |12878|3243:3604|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 '/usr/sbin/useradd -u 130667 -d /home/nsasherr -m nsasherr -g vcl; chown -R nsasherr /home/nsasherr' 2>&1
  2076. 2013-05-28 13:56:19|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2077. 2013-05-28 13:56:19|12878|3243:3604|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "")
  2078. 2013-05-28 13:56:19|12878|3243:3604|reinstall|new.pm:reserve_computer(1004)|OS module successfully reserved resources for this reservation
  2079. 2013-05-28 13:56:19|12878|3243:3604|reinstall|new.pm:reserve_computer(1114)|nsasherr@sharcnet.ca
  2080. |12878|3243:3604|reinstall| Your reservation was successfully reinstalled and you can proceed to reconnect.
  2081. |12878|3243:3604|reinstall| Please revisit the Current reservations page for any additional information.
  2082. |12878|3243:3604|reinstall| Reservation Information:
  2083. |12878|3243:3604|reinstall| Image Name: Test Image 2
  2084. |12878|3243:3604|reinstall| Image Name: Test Image 2
  2085. |12878|3243:3604|reinstall| -Visit http://vcl.sharcnet.ca/
  2086. |12878|3243:3604|reinstall| -Select "Current Reservations"
  2087. |12878|3243:3604|reinstall| -Click the "Connect" button
  2088. |12878|3243:3604|reinstall| Upon acknowledgement, all of the remaining connection details will be displayed.
  2089. |12878|3243:3604|reinstall| Thank You,
  2090. |12878|3243:3604|reinstall| VCL Team
  2091. |12878|3243:3604|reinstall| ******************************************************************
  2092. |12878|3243:3604|reinstall| This is an automated notice. If you need assistance please respond
  2093. |12878|3243:3604|reinstall| with detailed information on the issue and a help ticket will be
  2094. |12878|3243:3604|reinstall| generated.
  2095. |12878|3243:3604|reinstall| To disable email notices
  2096. |12878|3243:3604|reinstall| -Visit http://vcl.sharcnet.ca/
  2097. |12878|3243:3604|reinstall| -Select User Preferences
  2098. |12878|3243:3604|reinstall| -Select General Preferences
  2099. |12878|3243:3604|reinstall| ******************************************************************
  2100. 2013-05-28 13:56:19|12878|3243:3604|reinstall|new.pm:reserve_computer(1117)|IMTYPE_name= none calling notify_via
  2101. 2013-05-28 13:56:19|12878|3243:3604|reinstall|new.pm:reserve_computer(1187)|updated sublog 1329 for node ecco-vm-x10 IPaddress 199.241.160.134
  2102. 2013-05-28 13:56:19|12878|3243:3604|reinstall|new.pm:process(408)|ecco-vm-x10 with linux-TestImage2-16-v2 successfully reserved
  2103. 2013-05-28 13:56:19|12878|3243:3604|reinstall|utils.pm:insertloadlog(3665)|inserted computer=18, reserved, ecco-vm-x10 successfully reserved with linux-TestImage2-16-v2
  2104. 2013-05-28 13:56:19|12878|3243:3604|reinstall|new.pm:process(420)|inserted computerloadlog entry, load state=reserved
  2105. 2013-05-28 13:56:19|12878|3243:3604|reinstall|utils.pm:update_request_state(1545)|request 3243 state updated to: reserved, laststate to: reinstall
  2106. 2013-05-28 13:56:19|12878|3243:3604|reinstall|new.pm:process(460)|request state set to 'reserved'/'reinstall'
  2107. 2013-05-28 13:56:19|12878|3243:3604|reinstall|utils.pm:insertloadlog(3665)|inserted computer=18, info, ecco-vm-x10 successfully set to with linux-TestImage2-16-v2
  2108. 2013-05-28 13:56:19|12878|3243:3604|reinstall|new.pm:process(489)|inserted computerloadlog entry: ecco-vm-x10 successfully set to with linux-TestImage2-16-v2
  2109. 2013-05-28 13:56:19|12878|3243:3604|reinstall|new.pm:process(496)|exiting
  2110. 2013-05-28 13:56:19|12878|3243:3604|reinstall|utils.pm:delete_computerloadlog_reservation(6396)|removing computerloadlog entries matching loadstate = begin
  2111. 2013-05-28 13:56:21|12878|3243:3604|reinstall|utils.pm:delete_computerloadlog_reservation(6447)|deleted rows from computerloadlog for reservation id=3604
  2112. 2013-05-28 13:56:21|12878|3243:3604|reinstall|State.pm:DESTROY(929)|VCL::new process duration: 50 seconds
  2113. 2013-05-28 13:56:21|31074|vcld:REAPER(723)|VCL process exited for reservation 3604, PID: 12878, signal: CHLD
  2114. 2013-05-28 13:56:21|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:56:21
  2115. 2013-05-28 13:56:21|31074|3243:3604|reserved|utils.pm:reservation_being_processed(8638)|computerloadlog 'begin' entry does NOT exist for reservation 3604
  2116. 2013-05-28 13:56:21|31074|3243:3604|reserved|utils.pm:run_command(8710)|executed command: pgrep -fl 'vcld [0-9]+:3604 ', pid: 17733, exit status: 1, output:
  2117. 2013-05-28 13:56:21|31074|3243:3604|reserved|utils.pm:is_management_node_process_running(8943)|process is NOT running, identifier: 'vcld [0-9]+:3604 '
  2118. 2013-05-28 13:56:21|31074|3243:3604|reserved|utils.pm:reservation_being_processed(8659)|reservation is NOT currently being processed
  2119. 2013-05-28 13:56:21|31074|3243:3604|reserved|vcld:main(284)|reservation 3604 is NOT already being processed
  2120. 2013-05-28 13:56:21|31074|3243:3604|reserved|utils.pm:get_connect_method_info(10063)|attempting to retrieve connect method info:
  2121. |31074|3243:3604|reserved| imagerevision: 174 - linux-TestImage2-16-v2
  2122. |31074|3243:3604|reserved| OS: 36 - vmwarelinux
  2123. |31074|3243:3604|reserved| OS type: 2 - linux
  2124. 2013-05-28 13:56:21|31074|3243:3604|reserved|utils.pm:get_connect_method_info(10127)|ssh: connectmethodid=1, OStypeid=2, OSid=NULL, imagerevisionid=NULL, disabled=0
  2125. 2013-05-28 13:56:21|31074|3243:3604|reserved|utils.pm:get_connect_method_info(10127)|Xvnc: connectmethodid=4, OStypeid=NULL, OSid=NULL, imagerevisionid=174, disabled=0
  2126. 2013-05-28 13:56:21|31074|3243:3604|reserved|utils.pm:get_user_info(7558)|non-standalone affiliation found for user nsasherr:
  2127. |31074|3243:3604|reserved| user affiliation: sharcnet
  2128. |31074|3243:3604|reserved| not standalone list: sharcnet
  2129. 2013-05-28 13:56:21|31074|3243:3604|reserved|utils.pm:get_user_info(7558)|non-standalone affiliation found for user nsasherr:
  2130. |31074|3243:3604|reserved| user affiliation: sharcnet
  2131. |31074|3243:3604|reserved| not standalone list: sharcnet
  2132. 2013-05-28 13:56:21|31074|3243:3604|reserved|utils.pm:get_connect_method_info(10063)|attempting to retrieve connect method info:
  2133. |31074|3243:3604|reserved| imagerevision: 174 - linux-TestImage2-16-v2
  2134. |31074|3243:3604|reserved| OS: 36 - vmwarelinux
  2135. |31074|3243:3604|reserved| OS type: 2 - linux
  2136. 2013-05-28 13:56:21|31074|3243:3604|reserved|utils.pm:get_connect_method_info(10127)|ssh: connectmethodid=1, OStypeid=2, OSid=NULL, imagerevisionid=NULL, disabled=0
  2137. 2013-05-28 13:56:21|31074|3243:3604|reserved|utils.pm:get_connect_method_info(10127)|Xvnc: connectmethodid=4, OStypeid=NULL, OSid=NULL, imagerevisionid=174, disabled=0
  2138. 2013-05-28 13:56:21|31074|3243:3604|reserved|utils.pm:get_user_info(7558)|non-standalone affiliation found for user nsasherr:
  2139. |31074|3243:3604|reserved| user affiliation: sharcnet
  2140. |31074|3243:3604|reserved| not standalone list: sharcnet
  2141. 2013-05-28 13:56:21|31074|3243:3604|reserved|utils.pm:get_user_info(7558)|non-standalone affiliation found for user nsasherr:
  2142. |31074|3243:3604|reserved| user affiliation: sharcnet
  2143. |31074|3243:3604|reserved| not standalone list: sharcnet
  2144. 2013-05-28 13:56:21|31074|3243:3604|reserved|vcld:main(289)|retrieved request information from database
  2145. 2013-05-28 13:56:21|31074|3243:3604|reserved|DataStructure.pm:is_parent_reservation(1004)|returning true: parent reservation ID for this request: 3604
  2146. 2013-05-28 13:56:21|31074|3243:3604|reserved|utils.pm:update_request_state(1545)|request 3243 state updated to: pending, laststate to: reserved
  2147. 2013-05-28 13:56:21|31074|3243:3604|reserved|utils.pm:insertloadlog(3665)|inserted computer=18, begin, beginning to process, state is reserved
  2148. 2013-05-28 13:56:21|31074|3243:3604|reserved|vcld:make_new_child(520)|loaded VCL::reserved module
  2149. 2013-05-28 13:56:21|31074|3243:3604|reserved|vcld:make_new_child(544)|current number of forked kids: 2
  2150. 2013-05-28 13:56:21|17747|3243:3604|reserved|vcld:make_new_child(560)|vcld environment variable set to 0 for this process
  2151. 2013-05-28 13:56:21|17747|3243:3604|reserved|DataStructure.pm:_initialize(643)|image ID argument was specified: noimage, DataStructure object will contain image information for the production imagerevision of this image
  2152. 2013-05-28 13:56:21|31074|3243:3605|reserved|utils.pm:reservation_being_processed(8630)|computerloadlog 'begin' entry exists for reservation
  2153. 2013-05-28 13:56:21|17747|3243:3604|reserved|DataStructure.pm:_initialize(674)|retrieved data for imagerevision ID: 1
  2154. 2013-05-28 13:56:21|17747|3243:3604|reserved|DataStructure.pm:_initialize(690)|retrieved data for image ID: 1
  2155. 2013-05-28 13:56:21|31074|3243:3605|reserved|utils.pm:run_command(8710)|executed command: pgrep -fl 'vcld [0-9]+:3605 ', pid: 17749, exit status: 0, output:
  2156. |31074|3243:3605|reserved| 12889 vcld 3243:3605 reinstall ecco-vm-x26>ecco-vmhost-x7 linux-TestImage2-16-v2 nsasherr (cluster=child)
  2157. 2013-05-28 13:56:21|31074|3243:3605|reserved|utils.pm:is_management_node_process_running(8933)|found matching process: 12889 vcld 3243:3605 reinstall ecco-vm-x26>ecco-vmhost-x7 linux-TestImage2-16-v2 nsasherr (cluster=child)
  2158. 2013-05-28 13:56:21|31074|3243:3605|reserved|utils.pm:is_management_node_process_running(8939)|process is running, identifier: 'vcld [0-9]+:3605 ', returning array containing PIDs: 12889
  2159. 2013-05-28 13:56:21|31074|3243:3605|reserved|utils.pm:reservation_being_processed(8647)|reservation is currently being processed, computerloadlog 'begin' entry exists and running process was found: 12889
  2160.  
  2161. |31074|3243:3605|reserved| ---- WARNING ----
  2162. |31074|3243:3605|reserved| 2013-05-28 13:56:21|31074|3243:3605|reserved|vcld:main(277)|reservation 3605 is already being processed
  2163. |31074|3243:3605|reserved| ( 0) vcld, main (line: 277)
  2164.  
  2165. 2013-05-28 13:56:21|17747|3243:3604|reserved|Module.pm:create_mn_os_object(376)|VCL::Module::OS::Linux::ManagementNode module loaded
  2166. 2013-05-28 13:56:21|17747|3243:3604|reserved|Module.pm:new(196)|VCL::Module::OS::Linux::ManagementNode object created for image noimage, address: 3c49b60
  2167. 2013-05-28 13:56:21|17747|3243:3604|reserved|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{0}{computer}{hostname}, data identifier: computer_hostname, data:
  2168. |17747|3243:3604|reserved| : "ecco-adm3.sharcnet.ca"
  2169. 2013-05-28 13:56:21|17747|3243:3604|reserved|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{0}{computer}{NODENAME}, data identifier: computer_node_name, data:
  2170. |17747|3243:3604|reserved| : "ecco-adm3"
  2171. 2013-05-28 13:56:21|17747|3243:3604|reserved|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{0}{computer}{SHORTNAME}, data identifier: computer_short_name, data:
  2172. |17747|3243:3604|reserved| : "ecco-adm3"
  2173. 2013-05-28 13:56:21|17747|3243:3604|reserved|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{0}{computer}{IPaddress}, data identifier: computer_ip_address, data:
  2174. |17747|3243:3604|reserved| : "10.18.20.4"
  2175. 2013-05-28 13:56:21|17747|3243:3604|reserved|Module.pm:create_mn_os_object(381)|VCL::Module::OS::Linux::ManagementNode OS object created, address: 3c49b60
  2176. 2013-05-28 13:56:21|17747|3243:3604|reserved|Module.pm:new(192)|VCL::reserved object created for state reserved, address: 366c008
  2177. 2013-05-28 13:56:21|17747|3243:3604|reserved|State.pm:initialize(88)|obtained a database handle for this state process, stored as $ENV{dbh}
  2178. 2013-05-28 13:56:21|17747|3243:3604|reserved|State.pm:check_image_os(771)|no corrections need to be made, not an imaging request, returning 1
  2179. 2013-05-28 13:56:21|17747|3243:3604|reserved|DataStructure.pm:is_parent_reservation(1004)|returning true: parent reservation ID for this request: 3604
  2180. 2013-05-28 13:56:21|17747|3243:3604|reserved|utils.pm:rename_vcld_process(6756)|reservation count: 2
  2181. 2013-05-28 13:56:21|17747|3243:3604|reserved|utils.pm:rename_vcld_process(6775)|PARENTIMAGE: 1
  2182. 2013-05-28 13:56:21|17747|3243:3604|reserved|utils.pm:rename_vcld_process(6776)|SUBIMAGE: 0
  2183. 2013-05-28 13:56:21|17747|3243:3604|reserved|utils.pm:rename_vcld_process(6794)|renamed process to 'vcld 3243:3604 reserved ecco-vm-x10>ecco-vmhost-x3 linux-TestImage2-16-v2 nsasherr (cluster=parent)'
  2184. 2013-05-28 13:56:21|17747|3243:3604|reserved|DataStructure.pm:is_parent_reservation(1004)|returning true: parent reservation ID for this request: 3604
  2185. 2013-05-28 13:56:21|17747|3243:3604|reserved|DataStructure.pm:is_parent_reservation(1004)|returning true: parent reservation ID for this request: 3604
  2186. 2013-05-28 13:56:21|17747|3243:3604|reserved|Module.pm:create_os_object(308)|VCL::Module::OS::Linux module loaded
  2187. 2013-05-28 13:56:21|17747|3243:3604|reserved|Module.pm:new(196)|VCL::Module::OS::Linux object created for image linux-TestImage2-16-v2, address: 3c4c310
  2188. 2013-05-28 13:56:21|17747|3243:3604|reserved|Module.pm:create_os_object(314)|VCL::Module::OS::Linux OS object created, address: 3c4c310
  2189. 2013-05-28 13:56:21|17747|3243:3604|reserved|State.pm:initialize(125)|computer is a VM, attempting to create VM host OS object
  2190. 2013-05-28 13:56:21|17747|3243:3604|reserved|DataStructure.pm:_initialize(619)|computer ID argument was specified, retrieving data for computer ID: 21
  2191. 2013-05-28 13:56:21|17747|3243:3604|reserved|DataStructure.pm:_initialize(643)|image ID argument was specified: 1, DataStructure object will contain image information for the production imagerevision of this image
  2192. 2013-05-28 13:56:21|17747|3243:3604|reserved|DataStructure.pm:_initialize(674)|retrieved data for imagerevision ID: 1
  2193. 2013-05-28 13:56:21|17747|3243:3604|reserved|DataStructure.pm:_initialize(690)|retrieved data for image ID: 1
  2194. 2013-05-28 13:56:21|17747|3243:3604|reserved|Module.pm:create_vmhost_os_object(454)|attempting to load VM host OS module: VCL::Module::OS::Linux::UnixLab (image: 1)
  2195. 2013-05-28 13:56:21|17747|3243:3604|reserved|Module.pm:create_vmhost_os_object(460)|VM host OS module loaded: VCL::Module::OS::Linux::UnixLab
  2196. 2013-05-28 13:56:21|17747|3243:3604|reserved|Module.pm:new(196)|VCL::Module::OS::Linux::UnixLab object created for image noimage, address: 3c4c478
  2197. 2013-05-28 13:56:21|17747|3243:3604|reserved|Module.pm:create_vmhost_os_object(465)|VCL::Module::OS::Linux::UnixLab OS object created, address: 3c4c478
  2198. 2013-05-28 13:56:21|17747|3243:3604|reserved|Module.pm:create_provisioning_object(519)|VCL::Module::Provisioning::libvirt module loaded
  2199. 2013-05-28 13:56:21|17747|3243:3604|reserved|Module.pm:new(200)|VCL::Module::Provisioning::libvirt object created for computer ecco-vm-x10, address: 3c4c820
  2200. 2013-05-28 13:56:21|17747|3243:3604|reserved|libvirt.pm:initialize(99)|libvirt driver module directory path: /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/libvirt
  2201. 2013-05-28 13:56:21|17747|3243:3604|reserved|OS.pm:find_files(2917)|attempting to find files on ecco-adm3, base directory path: '/usr/local/vcl/bin/../lib/VCL/Module/Provisioning/libvirt/', pattern: *.pm, command: /usr/bin/find "/usr/local/vcl/bin/../lib/VCL/Module/Provisioning/libvirt/" -type f -iname "*.pm"
  2202. 2013-05-28 13:56:21|17747|3243:3604|reserved|OS.pm:find_files(2945)|files found: 1, base directory: '/usr/local/vcl/bin/../lib/VCL/Module/Provisioning/libvirt/', pattern: '*.pm'
  2203. |17747|3243:3604|reserved| command: '/usr/bin/find "/usr/local/vcl/bin/../lib/VCL/Module/Provisioning/libvirt/" -type f -iname "*.pm"'
  2204. 2013-05-28 13:56:21|17747|3243:3604|reserved|Module.pm:new(200)|VCL::Module::Provisioning::libvirt::KVM object created for computer ecco-vm-x10, address: 3e62a28
  2205. 2013-05-28 13:56:21|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  2206. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'which virsh' 2>&1
  2207. 2013-05-28 13:56:21|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2208. |17747|3243:3604|reserved| /usr/bin/virsh
  2209. 2013-05-28 13:56:21|17747|3243:3604|reserved|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x3, returning (0, "/usr/bin/virsh")
  2210. 2013-05-28 13:56:21|17747|3243:3604|reserved|KVM.pm:initialize(104)|verified 'virsh' command is available on ecco-vmhost-x3
  2211. 2013-05-28 13:56:21|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  2212. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'which qemu-img' 2>&1
  2213. 2013-05-28 13:56:21|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2214. |17747|3243:3604|reserved| /usr/bin/qemu-img
  2215. 2013-05-28 13:56:21|17747|3243:3604|reserved|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x3, returning (0, "/usr/bin/qemu-img")
  2216. 2013-05-28 13:56:21|17747|3243:3604|reserved|KVM.pm:initialize(104)|verified 'qemu-img' command is available on ecco-vmhost-x3
  2217. 2013-05-28 13:56:21|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x3:
  2218. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x3 'which virt-win-reg' 2>&1
  2219. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2220. |17747|3243:3604|reserved| /usr/bin/virt-win-reg
  2221. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x3, returning (0, "/usr/bin/virt-win-reg")
  2222. 2013-05-28 13:56:22|17747|3243:3604|reserved|KVM.pm:initialize(104)|verified 'virt-win-reg' command is available on ecco-vmhost-x3
  2223. 2013-05-28 13:56:22|17747|3243:3604|reserved|KVM.pm:initialize(113)|KVM driver module successfully initialized to control ecco-vmhost-x3
  2224. 2013-05-28 13:56:22|17747|3243:3604|reserved|libvirt.pm:initialize(119)|libvirt KVM driver object created and initialized to control ecco-vmhost-x3
  2225. 2013-05-28 13:56:22|17747|3243:3604|reserved|libvirt.pm:initialize(139)|VCL::Module::Provisioning::libvirt provisioning module initialized
  2226. 2013-05-28 13:56:22|17747|3243:3604|reserved|Module.pm:create_provisioning_object(525)|VCL::Module::Provisioning::libvirt provisioner object created for ecco-vm-x10, address: 3c4c820
  2227. 2013-05-28 13:56:22|17747|3243:3604|reserved|State.pm:initialize(154)|returning 1
  2228. 2013-05-28 13:56:22|17747|3243:3604|reserved|vcld:make_new_child(570)|VCL::reserved object created and initialized
  2229. 2013-05-28 13:56:22|17747|3243:3604|reserved|reserved.pm:process(119)|updated log table, set loaded time to now for id:1329
  2230. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:update_computer_state(1587)|computer 18 state updated to: reserved
  2231. 2013-05-28 13:56:22|17747|3243:3604|reserved|reserved.pm:process(127)|ecco-vm-x10 state set to 'reserved'
  2232. 2013-05-28 13:56:22|17747|3243:3604|reserved|reserved.pm:process(153)|computer info: id=18, type=virtualmachine, hostname=ecco-vm-x10
  2233. 2013-05-28 13:56:22|17747|3243:3604|reserved|reserved.pm:process(154)|user info: login_id id=nsasherr, uid=130667, standalone=0
  2234. 2013-05-28 13:56:22|17747|3243:3604|reserved|reserved.pm:process(155)|imagemeta checkuser set to: 0
  2235. 2013-05-28 13:56:22|17747|3243:3604|reserved|reserved.pm:process(156)|formimaging set to: 0
  2236. 2013-05-28 13:56:22|17747|3243:3604|reserved|reserved.pm:process(167)|begin checking for user acknowledgement
  2237. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:insertloadlog(3665)|inserted computer=18, info, reserved: waiting for user acknowledgement
  2238. 2013-05-28 13:56:22|17747|3243:3604|reserved|DataStructure.pm:get_reservation_remote_ip(1165)|retrieved remote IP for reservation 3604: 129.100.171.120
  2239. 2013-05-28 13:56:22|17747|3243:3604|reserved|reserved.pm:process(185)|user acknowledged, remote IP: 129.100.171.120
  2240. 2013-05-28 13:56:22|17747|3243:3604|reserved|reserved.pm:process(189)|cluster reservation, attempting to populate nodes with cluster_info data
  2241. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:update_cluster_info(8196)|ecco-vm-x10 is_cluster_parent = 1
  2242. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:update_cluster_info(8197)|ecco-vm-x10 is_cluster_child = 0
  2243. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:update_cluster_info(8198)|parent_reservation_id = 3604
  2244. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:update_cluster_info(8203)|writing parent= 199.241.160.134
  2245. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:update_cluster_info(8207)|writing child= 199.241.160.149
  2246. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:run_scp_command(5156)|attempting to copy file via SCP: '/tmp/ecco-vm-x10.cluster_info' --> 'ecco-vm-x10:/etc/cluster_info'
  2247. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:run_scp_command(5192)|copied file via SCP: '/tmp/ecco-vm-x10.cluster_info' --> 'ecco-vm-x10:/etc/cluster_info'
  2248. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:update_cluster_info(8236)|successfully copied cluster_info file to ecco-vm-x10
  2249. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:run_scp_command(5156)|attempting to copy file via SCP: '/tmp/ecco-vm-x10.cluster_info' --> 'ecco-vm-x26:/etc/cluster_info'
  2250. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:run_scp_command(5192)|copied file via SCP: '/tmp/ecco-vm-x10.cluster_info' --> 'ecco-vm-x26:/etc/cluster_info'
  2251. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:update_cluster_info(8236)|successfully copied cluster_info file to ecco-vm-x26
  2252. 2013-05-28 13:56:22|17747|3243:3604|reserved|reserved.pm:process(190)|updated cluster nodes with cluster infomation
  2253. 2013-05-28 13:56:22|17747|3243:3604|reserved|reserved.pm:process(199)|calling VCL::Module::OS::Linux::grant_access() subroutine
  2254. 2013-05-28 13:56:22|17747|3243:3604|reserved|Linux.pm:grant_access(973)|In grant_access routine nsasherr,ecco-vm-x10
  2255. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2256. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'sed -i -e "/^AllowUsers .*/d" /etc/ssh/external_sshd_config' 2>&1
  2257. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2258. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "")
  2259. 2013-05-28 13:56:22|17747|3243:3604|reserved|Linux.pm:grant_access(976)|cleared AllowUsers directive from external_sshd_config
  2260. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2261. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'echo "AllowUsers nsasherr" >> /etc/ssh/external_sshd_config' 2>&1
  2262. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2263. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "")
  2264. 2013-05-28 13:56:22|17747|3243:3604|reserved|Linux.pm:grant_access(984)|added AllowUsers nsasherr to external_sshd_config
  2265. 2013-05-28 13:56:22|17747|3243:3604|reserved|Linux.pm:grant_access(992)|server_request_id= 0
  2266. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2267. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  2268. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2269. |17747|3243:3604|reserved| stat: cannot stat `/bin/systemctl': No such file or directory
  2270. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vm-x10, command:
  2271. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  2272. |17747|3243:3604|reserved| returning (1, "stat: cannot stat `/bin/system...")
  2273. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2274. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'service ext_sshd stop' 2>&1
  2275. 2013-05-28 13:56:22|17747|3243:3604|reserved|Linux.pm:stop_service(3298)|service exists: ext_sshd
  2276. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2277. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  2278. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2279. |17747|3243:3604|reserved| stat: cannot stat `/bin/systemctl': No such file or directory
  2280. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vm-x10, command:
  2281. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  2282. |17747|3243:3604|reserved| returning (1, "stat: cannot stat `/bin/system...")
  2283. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2284. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'service ext_sshd start' 2>&1
  2285. 2013-05-28 13:56:22|12889|3243:3605|reinstall|OS.pm:wait_for_response(736)|waited 30 seconds for ecco-vm-x26 to boot
  2286. 2013-05-28 13:56:22|12889|3243:3605|reinstall|OS.pm:wait_for_response(740)|waiting for ecco-vm-x26 to respond to SSH, maximum of 600 seconds
  2287. 2013-05-28 13:56:22|12889|3243:3605|reinstall|Module.pm:code_loop_timeout(887)|waiting for ecco-vm-x26 to respond to SSH, maximum of 600 seconds
  2288. 2013-05-28 13:56:22|17747|3243:3604|reserved|Linux.pm:start_service(3241)|service exists: ext_sshd
  2289. 2013-05-28 13:56:22|17747|3243:3604|reserved|Linux.pm:grant_access(1016)|started ext_sshd on ecco-vm-x10
  2290. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:get_connect_method_info(10063)|attempting to retrieve connect method info:
  2291. |17747|3243:3604|reserved| imagerevision: 174 - linux-TestImage2-16-v2
  2292. |17747|3243:3604|reserved| OS: 36 - vmwarelinux
  2293. |17747|3243:3604|reserved| OS type: 2 - linux
  2294. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:get_connect_method_info(10127)|ssh: connectmethodid=1, OStypeid=2, OSid=NULL, imagerevisionid=NULL, disabled=0
  2295. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:get_connect_method_info(10127)|Xvnc: connectmethodid=4, OStypeid=NULL, OSid=NULL, imagerevisionid=174, disabled=0
  2296. 2013-05-28 13:56:22|17747|3243:3604|reserved|OS.pm:process_connect_methods(2525)|reservation remote IP address is not defined, connect methods will be available from any IP address
  2297. 2013-05-28 13:56:22|17747|3243:3604|reserved|OS.pm:process_connect_methods(2543)|state = reserved
  2298. 2013-05-28 13:56:22|17747|3243:3604|reserved|OS.pm:process_connect_methods(2592)|checking if 'ext_sshd' service exists for 'ssh' connect method on ecco-vm-x10
  2299. 2013-05-28 13:56:22|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2300. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  2301. 2013-05-28 13:56:23|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2302. |17747|3243:3604|reserved| stat: cannot stat `/bin/systemctl': No such file or directory
  2303. 2013-05-28 13:56:23|17747|3243:3604|reserved|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vm-x10, command:
  2304. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  2305. |17747|3243:3604|reserved| returning (1, "stat: cannot stat `/bin/system...")
  2306. 2013-05-28 13:56:23|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2307. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /sbin/chkconfig' 2>&1
  2308. 2013-05-28 13:56:23|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2309. |17747|3243:3604|reserved| File: `/sbin/chkconfig'
  2310. |17747|3243:3604|reserved| Size: 31728 Blocks: 64 IO Block: 4096 regular file
  2311. |17747|3243:3604|reserved| Device: 802h/2050d Inode: 642 Links: 1
  2312. |17747|3243:3604|reserved| Access: (0755/-rwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
  2313. |17747|3243:3604|reserved| Access: 2013-05-28 13:58:10.443116161 -0400
  2314. |17747|3243:3604|reserved| Modify: 2012-06-22 06:09:02.000000000 -0400
  2315. |17747|3243:3604|reserved| Change: 2012-10-09 15:03:55.140697475 -0400
  2316. 2013-05-28 13:56:23|17747|3243:3604|reserved|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "File: `/sbin/chkconfig' Size: ...")
  2317. 2013-05-28 13:56:23|17747|3243:3604|reserved|Linux.pm:file_exists(1449)|'/sbin/chkconfig' exists on ecco-vm-x10, files: 1, directories: 0, links: 0
  2318. 2013-05-28 13:56:23|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2319. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 '/sbin/chkconfig --list ext_sshd' 2>&1
  2320. 2013-05-28 13:56:23|17747|3243:3604|reserved|Linux.pm:service_exists(3170)|'ext_sshd' service exists but is not referenced in any runlevel: output:
  2321. |17747|3243:3604|reserved| service ext_sshd supports chkconfig, but is not referenced in any runlevel (run 'chkconfig --add ext_sshd')
  2322. 2013-05-28 13:56:23|17747|3243:3604|reserved|OS.pm:process_connect_methods(2594)|attempting to start 'ext_sshd' service for 'ssh' connect method on ecco-vm-x10
  2323. 2013-05-28 13:56:23|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2324. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  2325. 2013-05-28 13:56:23|12889|3243:3605|reinstall|OS.pm:is_ssh_responding(653)|ecco-vm-x26 is NOT responding to SSH, ports 22 or 24 are both closed
  2326. 2013-05-28 13:56:23|12889|3243:3605|reinstall|Module.pm:code_loop_timeout(909)|attempt 1: waiting for ecco-vm-x26 to respond to SSH (1/599 elapsed/remaining seconds), sleeping for 10 seconds
  2327. 2013-05-28 13:56:23|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2328. |17747|3243:3604|reserved| stat: cannot stat `/bin/systemctl': No such file or directory
  2329. 2013-05-28 13:56:23|17747|3243:3604|reserved|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vm-x10, command:
  2330. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  2331. |17747|3243:3604|reserved| returning (1, "stat: cannot stat `/bin/system...")
  2332. 2013-05-28 13:56:23|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2333. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'service ext_sshd start' 2>&1
  2334. 2013-05-28 13:56:23|17747|3243:3604|reserved|Linux.pm:start_service(3241)|service exists: ext_sshd
  2335. 2013-05-28 13:56:23|17747|3243:3604|reserved|OS.pm:process_connect_methods(2596)|'ext_sshd' started on ecco-vm-x10
  2336. 2013-05-28 13:56:23|17747|3243:3604|reserved|OS.pm:process_connect_methods(2624)|attempting to open firewall port 22 on ecco-vm-x10 for 'ssh' connect method
  2337. 2013-05-28 13:56:23|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2338. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  2339. 2013-05-28 13:56:23|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2340. |17747|3243:3604|reserved| stat: cannot stat `/bin/systemctl': No such file or directory
  2341. 2013-05-28 13:56:23|17747|3243:3604|reserved|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vm-x10, command:
  2342. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  2343. |17747|3243:3604|reserved| returning (1, "stat: cannot stat `/bin/system...")
  2344. 2013-05-28 13:56:23|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2345. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /sbin/chkconfig' 2>&1
  2346. 2013-05-28 13:56:23|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2347. |17747|3243:3604|reserved| File: `/sbin/chkconfig'
  2348. |17747|3243:3604|reserved| Size: 31728 Blocks: 64 IO Block: 4096 regular file
  2349. |17747|3243:3604|reserved| Device: 802h/2050d Inode: 642 Links: 1
  2350. |17747|3243:3604|reserved| Access: (0755/-rwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
  2351. |17747|3243:3604|reserved| Access: 2013-05-28 13:58:10.443116161 -0400
  2352. |17747|3243:3604|reserved| Modify: 2012-06-22 06:09:02.000000000 -0400
  2353. |17747|3243:3604|reserved| Change: 2012-10-09 15:03:55.140697475 -0400
  2354. 2013-05-28 13:56:23|17747|3243:3604|reserved|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "File: `/sbin/chkconfig' Size: ...")
  2355. 2013-05-28 13:56:23|17747|3243:3604|reserved|Linux.pm:file_exists(1449)|'/sbin/chkconfig' exists on ecco-vm-x10, files: 1, directories: 0, links: 0
  2356. 2013-05-28 13:56:23|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2357. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 '/sbin/chkconfig --list iptables' 2>&1
  2358. 2013-05-28 13:56:23|17747|3243:3604|reserved|Linux.pm:service_exists(3187)|'iptables' service exists
  2359. 2013-05-28 13:56:23|17747|3243:3604|reserved|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: 'Kernel IP routing table'
  2360. 2013-05-28 13:56:23|17747|3243:3604|reserved|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: 'Destination Gateway Genmask Flags Metric Ref Use Iface'
  2361. 2013-05-28 13:56:23|17747|3243:3604|reserved|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '199.241.160.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1'
  2362. 2013-05-28 13:56:23|17747|3243:3604|reserved|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '10.18.20.0 0.0.0.0 255.255.254.0 U 0 0 0 eth0'
  2363. 2013-05-28 13:56:23|17747|3243:3604|reserved|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '169.254.0.0 0.0.0.0 255.255.0.0 U 1003 0 0 eth0'
  2364. 2013-05-28 13:56:23|17747|3243:3604|reserved|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '169.254.0.0 0.0.0.0 255.255.0.0 U 1004 0 0 eth1'
  2365. 2013-05-28 13:56:23|17747|3243:3604|reserved|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '10.0.0.0 10.18.21.254 255.0.0.0 UG 0 0 0 eth0'
  2366. 2013-05-28 13:56:23|17747|3243:3604|reserved|Linux.pm:get_network_configuration(2703)|found default route configured for 'eth1' interface: 199.241.160.1
  2367. 2013-05-28 13:56:23|17747|3243:3604|reserved|Linux.pm:get_network_configuration(2708)|retrieved network configuration:
  2368. |17747|3243:3604|reserved| : {
  2369. |17747|3243:3604|reserved| : "eth0" => {
  2370. |17747|3243:3604|reserved| : "broadcast_address" => "10.18.21.255",
  2371. |17747|3243:3604|reserved| : "ip_address" => {
  2372. |17747|3243:3604|reserved| : "10.18.20.4" => "255.255.254.0"
  2373. |17747|3243:3604|reserved| : },
  2374. |17747|3243:3604|reserved| : "name" => "eth0",
  2375. |17747|3243:3604|reserved| : "physical_address" => "34:40:b5:82:4e:7c"
  2376. |17747|3243:3604|reserved| : },
  2377. |17747|3243:3604|reserved| : "eth1" => {
  2378. |17747|3243:3604|reserved| : "broadcast_address" => "199.241.160.255",
  2379. |17747|3243:3604|reserved| : "default_gateway" => "199.241.160.1",
  2380. |17747|3243:3604|reserved| : "ip_address" => {
  2381. |17747|3243:3604|reserved| : "199.241.160.4" => "255.255.255.0"
  2382. |17747|3243:3604|reserved| : },
  2383. |17747|3243:3604|reserved| : "name" => "eth1",
  2384. |17747|3243:3604|reserved| : "physical_address" => "34:40:b5:82:4e:7e"
  2385. |17747|3243:3604|reserved| : },
  2386. |17747|3243:3604|reserved| : "lo" => {
  2387. |17747|3243:3604|reserved| : "name" => "lo"
  2388. |17747|3243:3604|reserved| : },
  2389. |17747|3243:3604|reserved| : "usb0" => {
  2390. |17747|3243:3604|reserved| : "name" => "usb0",
  2391. |17747|3243:3604|reserved| : "physical_address" => "36:40:b5:85:4e:7f"
  2392. |17747|3243:3604|reserved| : }
  2393. |17747|3243:3604|reserved| : }
  2394. 2013-05-28 13:56:23|17747|3243:3604|reserved|DataStructure.pm:get_computer_private_ip_address(1641)|attempting to retrieve private IP address for computer: ecco-adm3
  2395. 2013-05-28 13:56:23|17747|3243:3604|reserved|OS.pm:get_file_contents(1897)|retrieved 146 lines from file on ecco-adm3: '/etc/hosts'
  2396. 2013-05-28 13:56:23|17747|3243:3604|reserved|DataStructure.pm:get_computer_private_ip_address(1697)|returning IP address from /etc/hosts file: 10.18.20.4
  2397. 2013-05-28 13:56:23|17747|3243:3604|reserved|OS.pm:get_private_interface_name(1119)|determined private interface name: eth0 (10.18.20.4)
  2398. 2013-05-28 13:56:23|17747|3243:3604|reserved|OS.pm:get_ip_address(1538)|returning private IP address: 10.18.20.4
  2399. 2013-05-28 13:56:23|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2400. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  2401. 2013-05-28 13:56:23|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2402. |17747|3243:3604|reserved| stat: cannot stat `/bin/systemctl': No such file or directory
  2403. 2013-05-28 13:56:23|17747|3243:3604|reserved|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vm-x10, command:
  2404. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  2405. |17747|3243:3604|reserved| returning (1, "stat: cannot stat `/bin/system...")
  2406. 2013-05-28 13:56:23|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2407. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /sbin/chkconfig' 2>&1
  2408. 2013-05-28 13:56:24|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2409. |17747|3243:3604|reserved| File: `/sbin/chkconfig'
  2410. |17747|3243:3604|reserved| Size: 31728 Blocks: 64 IO Block: 4096 regular file
  2411. |17747|3243:3604|reserved| Device: 802h/2050d Inode: 642 Links: 1
  2412. |17747|3243:3604|reserved| Access: (0755/-rwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
  2413. |17747|3243:3604|reserved| Access: 2013-05-28 13:58:10.443116161 -0400
  2414. |17747|3243:3604|reserved| Modify: 2012-06-22 06:09:02.000000000 -0400
  2415. |17747|3243:3604|reserved| Change: 2012-10-09 15:03:55.140697475 -0400
  2416. 2013-05-28 13:56:24|17747|3243:3604|reserved|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "File: `/sbin/chkconfig' Size: ...")
  2417. 2013-05-28 13:56:24|17747|3243:3604|reserved|Linux.pm:file_exists(1449)|'/sbin/chkconfig' exists on ecco-vm-x10, files: 1, directories: 0, links: 0
  2418. 2013-05-28 13:56:24|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2419. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 '/sbin/chkconfig --list iptables' 2>&1
  2420. 2013-05-28 13:56:24|17747|3243:3604|reserved|Linux.pm:service_exists(3187)|'iptables' service exists
  2421. 2013-05-28 13:56:24|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2422. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'iptables -L --line-number -n' 2>&1
  2423. 2013-05-28 13:56:24|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2424. |17747|3243:3604|reserved| Chain INPUT (policy ACCEPT)
  2425. |17747|3243:3604|reserved| num target prot opt source destination
  2426. |17747|3243:3604|reserved| 1 ACCEPT tcp -- 10.18.20.4 0.0.0.0/0 state NEW,RELATED,ESTABLISHED tcp
  2427. |17747|3243:3604|reserved| 2 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,RELATED,ESTABLISHED tcp dpt:5900
  2428. |17747|3243:3604|reserved| 3 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,RELATED,ESTABLISHED tcp dpt:22
  2429. |17747|3243:3604|reserved| 4 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
  2430. |17747|3243:3604|reserved| 5 ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0
  2431. |17747|3243:3604|reserved| 6 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
  2432. |17747|3243:3604|reserved| 7 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
  2433. |17747|3243:3604|reserved| Chain FORWARD (policy ACCEPT)
  2434. |17747|3243:3604|reserved| num target prot opt source destination
  2435. |17747|3243:3604|reserved| 1 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
  2436. |17747|3243:3604|reserved| Chain OUTPUT (policy ACCEPT)
  2437. |17747|3243:3604|reserved| num target prot opt source destination
  2438. 2013-05-28 13:56:24|17747|3243:3604|reserved|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "Chain INPUT (policy ACCEPT) nu...")
  2439. 2013-05-28 13:56:24|17747|3243:3604|reserved|Linux.pm:get_firewall_configuration(3992)|output Chain = INPUT
  2440. 2013-05-28 13:56:24|17747|3243:3604|reserved|Linux.pm:get_firewall_configuration(4008)|output rule: 2, ACCEPT, tcp, 0.0.0.0/0, 0.0.0.0/0, 5900
  2441. 2013-05-28 13:56:24|17747|3243:3604|reserved|Linux.pm:get_firewall_configuration(4008)|output rule: 3, ACCEPT, tcp, 0.0.0.0/0, 0.0.0.0/0, 22
  2442. 2013-05-28 13:56:26|17747|3243:3604|reserved|Linux.pm:get_firewall_configuration(3992)|output Chain = FORWARD
  2443. 2013-05-28 13:56:26|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:56:26
  2444. 2013-05-28 13:56:26|17747|3243:3604|reserved|Linux.pm:get_firewall_configuration(3992)|output Chain = OUTPUT
  2445. 2013-05-28 13:56:26|17747|3243:3604|reserved|Linux.pm:get_firewall_configuration(4051)|retrieved firewall configuration from ecco-vm-x10:
  2446. |17747|3243:3604|reserved| : {
  2447. |17747|3243:3604|reserved| : "FORWARD" => {
  2448. |17747|3243:3604|reserved| : 1 => {
  2449. |17747|3243:3604|reserved| : "all" => {
  2450. |17747|3243:3604|reserved| : "any" => {
  2451. |17747|3243:3604|reserved| : "destination" => "0.0.0.0/0",
  2452. |17747|3243:3604|reserved| : "name" => "any",
  2453. |17747|3243:3604|reserved| : "number" => 1,
  2454. |17747|3243:3604|reserved| : "scope" => "0.0.0.0/0",
  2455. |17747|3243:3604|reserved| : "target" => "REJECT"
  2456. |17747|3243:3604|reserved| : }
  2457. |17747|3243:3604|reserved| : }
  2458. |17747|3243:3604|reserved| : }
  2459. |17747|3243:3604|reserved| : },
  2460. |17747|3243:3604|reserved| : "INPUT" => {
  2461. |17747|3243:3604|reserved| : 1 => {
  2462. |17747|3243:3604|reserved| : "tcp" => {
  2463. |17747|3243:3604|reserved| : "any" => {
  2464. |17747|3243:3604|reserved| : "destination" => "0.0.0.0/0",
  2465. |17747|3243:3604|reserved| : "name" => "any",
  2466. |17747|3243:3604|reserved| : "number" => 1,
  2467. |17747|3243:3604|reserved| : "scope" => "10.18.20.4",
  2468. |17747|3243:3604|reserved| : "target" => "ACCEPT"
  2469. |17747|3243:3604|reserved| : }
  2470. |17747|3243:3604|reserved| : }
  2471. |17747|3243:3604|reserved| : },
  2472. |17747|3243:3604|reserved| : 2 => {
  2473. |17747|3243:3604|reserved| : "tcp" => {
  2474. |17747|3243:3604|reserved| : 5900 => {
  2475. |17747|3243:3604|reserved| : "destination" => "0.0.0.0/0",
  2476. |17747|3243:3604|reserved| : "name" => 5900,
  2477. |17747|3243:3604|reserved| : "number" => 2,
  2478. |17747|3243:3604|reserved| : "scope" => "0.0.0.0/0",
  2479. |17747|3243:3604|reserved| : "target" => "ACCEPT"
  2480. |17747|3243:3604|reserved| : }
  2481. |17747|3243:3604|reserved| : }
  2482. |17747|3243:3604|reserved| : },
  2483. |17747|3243:3604|reserved| : 3 => {
  2484. |17747|3243:3604|reserved| : "tcp" => {
  2485. |17747|3243:3604|reserved| : 22 => {
  2486. |17747|3243:3604|reserved| : "destination" => "0.0.0.0/0",
  2487. |17747|3243:3604|reserved| : "name" => "ssh",
  2488. |17747|3243:3604|reserved| : "number" => 3,
  2489. |17747|3243:3604|reserved| : "scope" => "0.0.0.0/0",
  2490. |17747|3243:3604|reserved| : "target" => "ACCEPT"
  2491. |17747|3243:3604|reserved| : }
  2492. |17747|3243:3604|reserved| : }
  2493. |17747|3243:3604|reserved| : },
  2494. |17747|3243:3604|reserved| : 4 => {
  2495. |17747|3243:3604|reserved| : "all" => {
  2496. |17747|3243:3604|reserved| : "any" => {
  2497. |17747|3243:3604|reserved| : "destination" => "0.0.0.0/0",
  2498. |17747|3243:3604|reserved| : "name" => "any",
  2499. |17747|3243:3604|reserved| : "number" => 4,
  2500. |17747|3243:3604|reserved| : "scope" => "0.0.0.0/0",
  2501. |17747|3243:3604|reserved| : "target" => "ACCEPT"
  2502. |17747|3243:3604|reserved| : }
  2503. |17747|3243:3604|reserved| : }
  2504. |17747|3243:3604|reserved| : },
  2505. |17747|3243:3604|reserved| : 5 => {
  2506. |17747|3243:3604|reserved| : "icmp" => {
  2507. |17747|3243:3604|reserved| : "any" => {
  2508. |17747|3243:3604|reserved| : "destination" => "0.0.0.0/0",
  2509. |17747|3243:3604|reserved| : "name" => "any",
  2510. |17747|3243:3604|reserved| : "number" => 5,
  2511. |17747|3243:3604|reserved| : "scope" => "0.0.0.0/0",
  2512. |17747|3243:3604|reserved| : "target" => "ACCEPT"
  2513. |17747|3243:3604|reserved| : }
  2514. |17747|3243:3604|reserved| : }
  2515. |17747|3243:3604|reserved| : },
  2516. |17747|3243:3604|reserved| : 6 => {
  2517. |17747|3243:3604|reserved| : "all" => {
  2518. |17747|3243:3604|reserved| : "any" => {
  2519. |17747|3243:3604|reserved| : "destination" => "0.0.0.0/0",
  2520. |17747|3243:3604|reserved| : "name" => "any",
  2521. |17747|3243:3604|reserved| : "number" => 6,
  2522. |17747|3243:3604|reserved| : "scope" => "0.0.0.0/0",
  2523. |17747|3243:3604|reserved| : "target" => "ACCEPT"
  2524. |17747|3243:3604|reserved| : }
  2525. |17747|3243:3604|reserved| : }
  2526. |17747|3243:3604|reserved| : },
  2527. |17747|3243:3604|reserved| : 7 => {
  2528. |17747|3243:3604|reserved| : "all" => {
  2529. |17747|3243:3604|reserved| : "any" => {
  2530. |17747|3243:3604|reserved| : "destination" => "0.0.0.0/0",
  2531. |17747|3243:3604|reserved| : "name" => "any",
  2532. |17747|3243:3604|reserved| : "number" => 7,
  2533. |17747|3243:3604|reserved| : "scope" => "0.0.0.0/0",
  2534. |17747|3243:3604|reserved| : "target" => "REJECT"
  2535. |17747|3243:3604|reserved| : }
  2536. |17747|3243:3604|reserved| : }
  2537. |17747|3243:3604|reserved| : }
  2538. |17747|3243:3604|reserved| : }
  2539. |17747|3243:3604|reserved| : }
  2540. 2013-05-28 13:56:26|17747|3243:3604|reserved|Linux.pm:enable_firewall_port(3615)|num= 3 protocol= tcp port= 22 existing_scope= 0.0.0.0/0 existing_name= ssh existing_description= ssh
  2541. 2013-05-28 13:56:26|17747|3243:3604|reserved|Linux.pm:parse_firewall_scope(4182)|parsed firewall scope:
  2542. |17747|3243:3604|reserved| argument: '0.0.0.0/0'
  2543. |17747|3243:3604|reserved| result: '0.0.0.0/0.0.0.0'
  2544. |17747|3243:3604|reserved| IP address ranges:
  2545. |17747|3243:3604|reserved| 0.0.0.0-255.255.255.255
  2546. 2013-05-28 13:56:26|17747|3243:3604|reserved|Linux.pm:enable_firewall_port(3625)|existing firewall opening on ecco-vm-x10 will be replaced:
  2547. |17747|3243:3604|reserved| name: 'ssh'
  2548. |17747|3243:3604|reserved| num: '3'
  2549. |17747|3243:3604|reserved| protocol: tcp
  2550. |17747|3243:3604|reserved| port/type: 22
  2551. |17747|3243:3604|reserved| existing scope: '0.0.0.0/0'
  2552. |17747|3243:3604|reserved| new scope: 0.0.0.0/0.0.0.0
  2553. |17747|3243:3604|reserved| overwrite existing rule: yes
  2554. 2013-05-28 13:56:26|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2555. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'ls -d --color=never "/etc/sysconfig" 2>&1 || mkdir -p "/etc/sysconfig" 2>&1 && ls -d --color=never "/etc/sysconfig"' 2>&1
  2556. 2013-05-28 13:56:27|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2557. |17747|3243:3604|reserved| /etc/sysconfig
  2558. |17747|3243:3604|reserved| /etc/sysconfig
  2559. 2013-05-28 13:56:27|17747|3243:3604|reserved|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "/etc/sysconfig /etc/sysconfig")
  2560. 2013-05-28 13:56:27|17747|3243:3604|reserved|Linux.pm:create_directory(1573)|directory already exists on ecco-vm-x10: '/etc/sysconfig'
  2561. 2013-05-28 13:56:27|17747|3243:3604|reserved|OS.pm:copy_file(2781)|attempting to copy file on ecco-vm-x10: '/etc/sysconfig/iptables' -> '/etc/sysconfig/iptables_pre_22'
  2562. 2013-05-28 13:56:27|17747|3243:3604|reserved|OS.pm:copy_file(2792)|copied file on ecco-vm-x10: '/etc/sysconfig/iptables' --> '/etc/sysconfig/iptables_pre_22'
  2563. 2013-05-28 13:56:27|17747|3243:3604|reserved|Linux.pm:enable_firewall_port(3703)|backed up original iptables file to: '/etc/sysconfig/iptables_pre_22'
  2564. 2013-05-28 13:56:27|17747|3243:3604|reserved|Linux.pm:enable_firewall_port(3711)|attempting to execute command on ecco-vm-x10: 'iptables -D INPUT 3 ; /sbin/iptables -I INPUT 1 -m state --state NEW,RELATED,ESTABLISHED -m tcp -p tcp -j ACCEPT --dport 22 -s 0.0.0.0/0'
  2565. 2013-05-28 13:56:27|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2566. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'iptables -D INPUT 3 ; /sbin/iptables -I INPUT 1 -m state --state NEW,RELATED,ESTABLISHED -m tcp -p tcp -j ACCEPT --dport 22 -s 0.0.0.0/0' 2>&1
  2567. 2013-05-28 13:56:27|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2568. 2013-05-28 13:56:27|17747|3243:3604|reserved|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "")
  2569. 2013-05-28 13:56:27|17747|3243:3604|reserved|Linux.pm:enable_firewall_port(3713)|executed command on ecco-vm-x10: 'iptables -D INPUT 3 ; /sbin/iptables -I INPUT 1 -m state --state NEW,RELATED,ESTABLISHED -m tcp -p tcp -j ACCEPT --dport 22 -s 0.0.0.0/0'
  2570. 2013-05-28 13:56:27|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2571. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 '/sbin/iptables-save > /etc/sysconfig/iptables' 2>&1
  2572. 2013-05-28 13:56:27|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2573. 2013-05-28 13:56:27|17747|3243:3604|reserved|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "")
  2574. 2013-05-28 13:56:27|17747|3243:3604|reserved|Linux.pm:enable_firewall_port(3723)|executed command /sbin/iptables-save > /etc/sysconfig/iptables on ecco-vm-x10
  2575. 2013-05-28 13:56:27|17747|3243:3604|reserved|OS.pm:process_connect_methods(2625)|opened firewall port 22 on ecco-vm-x10 for 0.0.0.0/0 ssh connect method
  2576. 2013-05-28 13:56:27|17747|3243:3604|reserved|OS.pm:process_connect_methods(2592)|checking if '/usr/bin/Xvnc' service exists for 'Xvnc' connect method on ecco-vm-x10
  2577. 2013-05-28 13:56:27|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2578. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  2579. 2013-05-28 13:56:27|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2580. |17747|3243:3604|reserved| stat: cannot stat `/bin/systemctl': No such file or directory
  2581. 2013-05-28 13:56:27|17747|3243:3604|reserved|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vm-x10, command:
  2582. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  2583. |17747|3243:3604|reserved| returning (1, "stat: cannot stat `/bin/system...")
  2584. 2013-05-28 13:56:27|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2585. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /sbin/chkconfig' 2>&1
  2586. 2013-05-28 13:56:27|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2587. |17747|3243:3604|reserved| File: `/sbin/chkconfig'
  2588. |17747|3243:3604|reserved| Size: 31728 Blocks: 64 IO Block: 4096 regular file
  2589. |17747|3243:3604|reserved| Device: 802h/2050d Inode: 642 Links: 1
  2590. |17747|3243:3604|reserved| Access: (0755/-rwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
  2591. |17747|3243:3604|reserved| Access: 2013-05-28 13:58:10.443116161 -0400
  2592. |17747|3243:3604|reserved| Modify: 2012-06-22 06:09:02.000000000 -0400
  2593. |17747|3243:3604|reserved| Change: 2012-10-09 15:03:55.140697475 -0400
  2594. 2013-05-28 13:56:27|17747|3243:3604|reserved|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "File: `/sbin/chkconfig' Size: ...")
  2595. 2013-05-28 13:56:27|17747|3243:3604|reserved|Linux.pm:file_exists(1449)|'/sbin/chkconfig' exists on ecco-vm-x10, files: 1, directories: 0, links: 0
  2596. 2013-05-28 13:56:27|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2597. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 '/sbin/chkconfig --list /usr/bin/Xvnc' 2>&1
  2598. 2013-05-28 13:56:27|17747|3243:3604|reserved|Linux.pm:service_exists(3175)|'/usr/bin/Xvnc' service does not exist on ecco-vm-x10
  2599.  
  2600. |17747|3243:3604|reserved| ---- WARNING ----
  2601. |17747|3243:3604|reserved| 2013-05-28 13:56:27|17747|3243:3604|reserved|OS.pm:process_connect_methods(2604)|'/usr/bin/Xvnc' service for 'Xvnc' connect method does NOT exist on ecco-vm-x10, connect method install script is not defined
  2602. |17747|3243:3604|reserved| ( 0) OS.pm, process_connect_methods (line: 2604)
  2603. |17747|3243:3604|reserved| (-1) Linux.pm, grant_access (line: 1029)
  2604. |17747|3243:3604|reserved| (-2) reserved.pm, process (line: 200)
  2605. |17747|3243:3604|reserved| (-3) vcld, make_new_child (line: 573)
  2606. |17747|3243:3604|reserved| (-4) vcld, main (line: 352)
  2607.  
  2608. 2013-05-28 13:56:27|17747|3243:3604|reserved|OS.pm:process_connect_methods(2624)|attempting to open firewall port 5900 on ecco-vm-x10 for 'Xvnc' connect method
  2609. 2013-05-28 13:56:27|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2610. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  2611. 2013-05-28 13:56:27|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2612. |17747|3243:3604|reserved| stat: cannot stat `/bin/systemctl': No such file or directory
  2613. 2013-05-28 13:56:27|17747|3243:3604|reserved|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vm-x10, command:
  2614. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  2615. |17747|3243:3604|reserved| returning (1, "stat: cannot stat `/bin/system...")
  2616. 2013-05-28 13:56:27|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2617. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /sbin/chkconfig' 2>&1
  2618. 2013-05-28 13:56:27|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2619. |17747|3243:3604|reserved| File: `/sbin/chkconfig'
  2620. |17747|3243:3604|reserved| Size: 31728 Blocks: 64 IO Block: 4096 regular file
  2621. |17747|3243:3604|reserved| Device: 802h/2050d Inode: 642 Links: 1
  2622. |17747|3243:3604|reserved| Access: (0755/-rwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
  2623. |17747|3243:3604|reserved| Access: 2013-05-28 13:58:10.443116161 -0400
  2624. |17747|3243:3604|reserved| Modify: 2012-06-22 06:09:02.000000000 -0400
  2625. |17747|3243:3604|reserved| Change: 2012-10-09 15:03:55.140697475 -0400
  2626. 2013-05-28 13:56:27|17747|3243:3604|reserved|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "File: `/sbin/chkconfig' Size: ...")
  2627. 2013-05-28 13:56:27|17747|3243:3604|reserved|Linux.pm:file_exists(1449)|'/sbin/chkconfig' exists on ecco-vm-x10, files: 1, directories: 0, links: 0
  2628. 2013-05-28 13:56:27|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2629. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 '/sbin/chkconfig --list iptables' 2>&1
  2630. 2013-05-28 13:56:28|17747|3243:3604|reserved|Linux.pm:service_exists(3187)|'iptables' service exists
  2631. 2013-05-28 13:56:28|17747|3243:3604|reserved|OS.pm:get_ip_address(1538)|returning private IP address: 10.18.20.4
  2632. 2013-05-28 13:56:28|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2633. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  2634. 2013-05-28 13:56:28|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2635. |17747|3243:3604|reserved| stat: cannot stat `/bin/systemctl': No such file or directory
  2636. 2013-05-28 13:56:28|17747|3243:3604|reserved|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vm-x10, command:
  2637. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /bin/systemctl' 2>&1
  2638. |17747|3243:3604|reserved| returning (1, "stat: cannot stat `/bin/system...")
  2639. 2013-05-28 13:56:28|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2640. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /sbin/chkconfig' 2>&1
  2641. 2013-05-28 13:56:28|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2642. |17747|3243:3604|reserved| File: `/sbin/chkconfig'
  2643. |17747|3243:3604|reserved| Size: 31728 Blocks: 64 IO Block: 4096 regular file
  2644. |17747|3243:3604|reserved| Device: 802h/2050d Inode: 642 Links: 1
  2645. |17747|3243:3604|reserved| Access: (0755/-rwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
  2646. |17747|3243:3604|reserved| Access: 2013-05-28 13:58:10.443116161 -0400
  2647. |17747|3243:3604|reserved| Modify: 2012-06-22 06:09:02.000000000 -0400
  2648. |17747|3243:3604|reserved| Change: 2012-10-09 15:03:55.140697475 -0400
  2649. 2013-05-28 13:56:28|17747|3243:3604|reserved|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "File: `/sbin/chkconfig' Size: ...")
  2650. 2013-05-28 13:56:28|17747|3243:3604|reserved|Linux.pm:file_exists(1449)|'/sbin/chkconfig' exists on ecco-vm-x10, files: 1, directories: 0, links: 0
  2651. 2013-05-28 13:56:28|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2652. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 '/sbin/chkconfig --list iptables' 2>&1
  2653. 2013-05-28 13:56:28|17747|3243:3604|reserved|Linux.pm:service_exists(3187)|'iptables' service exists
  2654. 2013-05-28 13:56:28|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2655. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'iptables -L --line-number -n' 2>&1
  2656. 2013-05-28 13:56:28|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2657. |17747|3243:3604|reserved| Chain INPUT (policy ACCEPT)
  2658. |17747|3243:3604|reserved| num target prot opt source destination
  2659. |17747|3243:3604|reserved| 1 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,RELATED,ESTABLISHED tcp dpt:22
  2660. |17747|3243:3604|reserved| 2 ACCEPT tcp -- 10.18.20.4 0.0.0.0/0 state NEW,RELATED,ESTABLISHED tcp
  2661. |17747|3243:3604|reserved| 3 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,RELATED,ESTABLISHED tcp dpt:5900
  2662. |17747|3243:3604|reserved| 4 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
  2663. |17747|3243:3604|reserved| 5 ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0
  2664. |17747|3243:3604|reserved| 6 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
  2665. |17747|3243:3604|reserved| 7 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
  2666. |17747|3243:3604|reserved| Chain FORWARD (policy ACCEPT)
  2667. |17747|3243:3604|reserved| num target prot opt source destination
  2668. |17747|3243:3604|reserved| 1 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
  2669. |17747|3243:3604|reserved| Chain OUTPUT (policy ACCEPT)
  2670. |17747|3243:3604|reserved| num target prot opt source destination
  2671. 2013-05-28 13:56:28|17747|3243:3604|reserved|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "Chain INPUT (policy ACCEPT) nu...")
  2672. 2013-05-28 13:56:28|17747|3243:3604|reserved|Linux.pm:get_firewall_configuration(3992)|output Chain = INPUT
  2673. 2013-05-28 13:56:28|17747|3243:3604|reserved|Linux.pm:get_firewall_configuration(4008)|output rule: 1, ACCEPT, tcp, 0.0.0.0/0, 0.0.0.0/0, 22
  2674. 2013-05-28 13:56:29|17747|3243:3604|reserved|Linux.pm:get_firewall_configuration(4008)|output rule: 3, ACCEPT, tcp, 0.0.0.0/0, 0.0.0.0/0, 5900
  2675. 2013-05-28 13:56:30|17747|3243:3604|reserved|Linux.pm:get_firewall_configuration(3992)|output Chain = FORWARD
  2676. 2013-05-28 13:56:31|17747|3243:3604|reserved|Linux.pm:get_firewall_configuration(3992)|output Chain = OUTPUT
  2677. 2013-05-28 13:56:31|17747|3243:3604|reserved|Linux.pm:get_firewall_configuration(4051)|retrieved firewall configuration from ecco-vm-x10:
  2678. |17747|3243:3604|reserved| : {
  2679. |17747|3243:3604|reserved| : "FORWARD" => {
  2680. |17747|3243:3604|reserved| : 1 => {
  2681. |17747|3243:3604|reserved| : "all" => {
  2682. |17747|3243:3604|reserved| : "any" => {
  2683. |17747|3243:3604|reserved| : "destination" => "0.0.0.0/0",
  2684. |17747|3243:3604|reserved| : "name" => "any",
  2685. |17747|3243:3604|reserved| : "number" => 1,
  2686. |17747|3243:3604|reserved| : "scope" => "0.0.0.0/0",
  2687. |17747|3243:3604|reserved| : "target" => "REJECT"
  2688. |17747|3243:3604|reserved| : }
  2689. |17747|3243:3604|reserved| : }
  2690. |17747|3243:3604|reserved| : }
  2691. |17747|3243:3604|reserved| : },
  2692. |17747|3243:3604|reserved| : "INPUT" => {
  2693. |17747|3243:3604|reserved| : 1 => {
  2694. |17747|3243:3604|reserved| : "tcp" => {
  2695. |17747|3243:3604|reserved| : 22 => {
  2696. |17747|3243:3604|reserved| : "destination" => "0.0.0.0/0",
  2697. |17747|3243:3604|reserved| : "name" => "ssh",
  2698. |17747|3243:3604|reserved| : "number" => 1,
  2699. |17747|3243:3604|reserved| : "scope" => "0.0.0.0/0",
  2700. |17747|3243:3604|reserved| : "target" => "ACCEPT"
  2701. |17747|3243:3604|reserved| : }
  2702. |17747|3243:3604|reserved| : }
  2703. |17747|3243:3604|reserved| : },
  2704. |17747|3243:3604|reserved| : 2 => {
  2705. |17747|3243:3604|reserved| : "tcp" => {
  2706. |17747|3243:3604|reserved| : "any" => {
  2707. |17747|3243:3604|reserved| : "destination" => "0.0.0.0/0",
  2708. |17747|3243:3604|reserved| : "name" => "any",
  2709. |17747|3243:3604|reserved| : "number" => 2,
  2710. |17747|3243:3604|reserved| : "scope" => "10.18.20.4",
  2711. |17747|3243:3604|reserved| : "target" => "ACCEPT"
  2712. |17747|3243:3604|reserved| : }
  2713. |17747|3243:3604|reserved| : }
  2714. |17747|3243:3604|reserved| : },
  2715. |17747|3243:3604|reserved| : 3 => {
  2716. |17747|3243:3604|reserved| : "tcp" => {
  2717. |17747|3243:3604|reserved| : 5900 => {
  2718. |17747|3243:3604|reserved| : "destination" => "0.0.0.0/0",
  2719. |17747|3243:3604|reserved| : "name" => 5900,
  2720. |17747|3243:3604|reserved| : "number" => 3,
  2721. |17747|3243:3604|reserved| : "scope" => "0.0.0.0/0",
  2722. |17747|3243:3604|reserved| : "target" => "ACCEPT"
  2723. |17747|3243:3604|reserved| : }
  2724. |17747|3243:3604|reserved| : }
  2725. |17747|3243:3604|reserved| : },
  2726. |17747|3243:3604|reserved| : 4 => {
  2727. |17747|3243:3604|reserved| : "all" => {
  2728. |17747|3243:3604|reserved| : "any" => {
  2729. |17747|3243:3604|reserved| : "destination" => "0.0.0.0/0",
  2730. |17747|3243:3604|reserved| : "name" => "any",
  2731. |17747|3243:3604|reserved| : "number" => 4,
  2732. |17747|3243:3604|reserved| : "scope" => "0.0.0.0/0",
  2733. |17747|3243:3604|reserved| : "target" => "ACCEPT"
  2734. |17747|3243:3604|reserved| : }
  2735. |17747|3243:3604|reserved| : }
  2736. |17747|3243:3604|reserved| : },
  2737. |17747|3243:3604|reserved| : 5 => {
  2738. |17747|3243:3604|reserved| : "icmp" => {
  2739. |17747|3243:3604|reserved| : "any" => {
  2740. |17747|3243:3604|reserved| : "destination" => "0.0.0.0/0",
  2741. |17747|3243:3604|reserved| : "name" => "any",
  2742. |17747|3243:3604|reserved| : "number" => 5,
  2743. |17747|3243:3604|reserved| : "scope" => "0.0.0.0/0",
  2744. |17747|3243:3604|reserved| : "target" => "ACCEPT"
  2745. |17747|3243:3604|reserved| : }
  2746. |17747|3243:3604|reserved| : }
  2747. |17747|3243:3604|reserved| : },
  2748. |17747|3243:3604|reserved| : 6 => {
  2749. |17747|3243:3604|reserved| : "all" => {
  2750. |17747|3243:3604|reserved| : "any" => {
  2751. |17747|3243:3604|reserved| : "destination" => "0.0.0.0/0",
  2752. |17747|3243:3604|reserved| : "name" => "any",
  2753. |17747|3243:3604|reserved| : "number" => 6,
  2754. |17747|3243:3604|reserved| : "scope" => "0.0.0.0/0",
  2755. |17747|3243:3604|reserved| : "target" => "ACCEPT"
  2756. |17747|3243:3604|reserved| : }
  2757. |17747|3243:3604|reserved| : }
  2758. |17747|3243:3604|reserved| : },
  2759. |17747|3243:3604|reserved| : 7 => {
  2760. |17747|3243:3604|reserved| : "all" => {
  2761. |17747|3243:3604|reserved| : "any" => {
  2762. |17747|3243:3604|reserved| : "destination" => "0.0.0.0/0",
  2763. |17747|3243:3604|reserved| : "name" => "any",
  2764. |17747|3243:3604|reserved| : "number" => 7,
  2765. |17747|3243:3604|reserved| : "scope" => "0.0.0.0/0",
  2766. |17747|3243:3604|reserved| : "target" => "REJECT"
  2767. |17747|3243:3604|reserved| : }
  2768. |17747|3243:3604|reserved| : }
  2769. |17747|3243:3604|reserved| : }
  2770. |17747|3243:3604|reserved| : }
  2771. |17747|3243:3604|reserved| : }
  2772. 2013-05-28 13:56:31|17747|3243:3604|reserved|Linux.pm:enable_firewall_port(3615)|num= 3 protocol= tcp port= 5900 existing_scope= 0.0.0.0/0 existing_name= 5900 existing_description= 5900
  2773. 2013-05-28 13:56:31|17747|3243:3604|reserved|Linux.pm:parse_firewall_scope(4182)|parsed firewall scope:
  2774. |17747|3243:3604|reserved| argument: '0.0.0.0/0'
  2775. |17747|3243:3604|reserved| result: '0.0.0.0/0.0.0.0'
  2776. |17747|3243:3604|reserved| IP address ranges:
  2777. |17747|3243:3604|reserved| 0.0.0.0-255.255.255.255
  2778. 2013-05-28 13:56:31|17747|3243:3604|reserved|Linux.pm:enable_firewall_port(3625)|existing firewall opening on ecco-vm-x10 will be replaced:
  2779. |17747|3243:3604|reserved| name: '5900'
  2780. |17747|3243:3604|reserved| num: '3'
  2781. |17747|3243:3604|reserved| protocol: tcp
  2782. |17747|3243:3604|reserved| port/type: 5900
  2783. |17747|3243:3604|reserved| existing scope: '0.0.0.0/0'
  2784. |17747|3243:3604|reserved| new scope: 0.0.0.0/0.0.0.0
  2785. |17747|3243:3604|reserved| overwrite existing rule: yes
  2786. 2013-05-28 13:56:31|17747|3243:3604|reserved|utils.pm:get_management_node_info(5456)|retrieving current management node info for 'ecco-adm3.sharcnet.ca' from database, cached data is stale: 60 seconds old
  2787. 2013-05-28 13:56:31|17747|3243:3604|reserved|utils.pm:get_management_node_info(5603)|retrieved management node info: 'ecco-adm3.sharcnet.ca' (ecco-adm3)
  2788. 2013-05-28 13:56:31|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2789. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'ls -d --color=never "/etc/sysconfig" 2>&1 || mkdir -p "/etc/sysconfig" 2>&1 && ls -d --color=never "/etc/sysconfig"' 2>&1
  2790. 2013-05-28 13:56:31|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2791. |17747|3243:3604|reserved| /etc/sysconfig
  2792. |17747|3243:3604|reserved| /etc/sysconfig
  2793. 2013-05-28 13:56:31|17747|3243:3604|reserved|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "/etc/sysconfig /etc/sysconfig")
  2794. 2013-05-28 13:56:31|17747|3243:3604|reserved|Linux.pm:create_directory(1573)|directory already exists on ecco-vm-x10: '/etc/sysconfig'
  2795. 2013-05-28 13:56:31|17747|3243:3604|reserved|OS.pm:copy_file(2781)|attempting to copy file on ecco-vm-x10: '/etc/sysconfig/iptables' -> '/etc/sysconfig/iptables_pre_5900'
  2796. 2013-05-28 13:56:31|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:56:31
  2797. 2013-05-28 13:56:31|17747|3243:3604|reserved|OS.pm:copy_file(2792)|copied file on ecco-vm-x10: '/etc/sysconfig/iptables' --> '/etc/sysconfig/iptables_pre_5900'
  2798. 2013-05-28 13:56:31|17747|3243:3604|reserved|Linux.pm:enable_firewall_port(3703)|backed up original iptables file to: '/etc/sysconfig/iptables_pre_5900'
  2799. 2013-05-28 13:56:31|17747|3243:3604|reserved|Linux.pm:enable_firewall_port(3711)|attempting to execute command on ecco-vm-x10: 'iptables -D INPUT 3 ; /sbin/iptables -I INPUT 1 -m state --state NEW,RELATED,ESTABLISHED -m tcp -p tcp -j ACCEPT --dport 5900 -s 0.0.0.0/0'
  2800. 2013-05-28 13:56:31|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2801. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'iptables -D INPUT 3 ; /sbin/iptables -I INPUT 1 -m state --state NEW,RELATED,ESTABLISHED -m tcp -p tcp -j ACCEPT --dport 5900 -s 0.0.0.0/0' 2>&1
  2802. 2013-05-28 13:56:31|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2803. 2013-05-28 13:56:31|17747|3243:3604|reserved|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "")
  2804. 2013-05-28 13:56:31|17747|3243:3604|reserved|Linux.pm:enable_firewall_port(3713)|executed command on ecco-vm-x10: 'iptables -D INPUT 3 ; /sbin/iptables -I INPUT 1 -m state --state NEW,RELATED,ESTABLISHED -m tcp -p tcp -j ACCEPT --dport 5900 -s 0.0.0.0/0'
  2805. 2013-05-28 13:56:31|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2806. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 '/sbin/iptables-save > /etc/sysconfig/iptables' 2>&1
  2807. 2013-05-28 13:56:32|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2808. 2013-05-28 13:56:32|17747|3243:3604|reserved|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "")
  2809. 2013-05-28 13:56:32|17747|3243:3604|reserved|Linux.pm:enable_firewall_port(3723)|executed command /sbin/iptables-save > /etc/sysconfig/iptables on ecco-vm-x10
  2810. 2013-05-28 13:56:32|17747|3243:3604|reserved|OS.pm:process_connect_methods(2625)|opened firewall port 5900 on ecco-vm-x10 for 0.0.0.0/0 Xvnc connect method
  2811. 2013-05-28 13:56:32|17747|3243:3604|reserved|Linux.pm:grant_access(1029)|processed connection methods on ecco-vm-x10 setting 0.0.0.0 for all allowed ports
  2812. 2013-05-28 13:56:32|17747|3243:3604|reserved|reserved.pm:process(200)|OS access has been granted on ecco-vm-x10
  2813. 2013-05-28 13:56:32|17747|3243:3604|reserved|reserved.pm:process(215)|VCL::Module::OS::Linux->post_reserve() subroutine exists
  2814. 2013-05-28 13:56:32|17747|3243:3604|reserved|reserved.pm:process(218)|calling VCL::Module::OS::Linux->post_reserve() subroutine
  2815. 2013-05-28 13:56:32|17747|3243:3604|reserved|utils.pm:insertloadlog(3665)|inserted computer=18, info, calling VCL::Module::OS::Linux->post_reserve() subroutine
  2816. 2013-05-28 13:56:32|17747|3243:3604|reserved|Linux.pm:post_reserve(369)|initiating Linux post_reserve: linux-TestImage2-16-v2 on ecco-vm-x10
  2817. 2013-05-28 13:56:32|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2818. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /etc/init\.d/vcl_post_reserve' 2>&1
  2819. 2013-05-28 13:56:32|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2820. |17747|3243:3604|reserved| File: `/etc/init.d/vcl_post_reserve'
  2821. |17747|3243:3604|reserved| Size: 96 Blocks: 8 IO Block: 4096 regular file
  2822. |17747|3243:3604|reserved| Device: 802h/2050d Inode: 7983 Links: 1
  2823. |17747|3243:3604|reserved| Access: (0700/-rwx------) Uid: ( 0/ root) Gid: ( 0/ root)
  2824. |17747|3243:3604|reserved| Access: 2013-05-28 13:58:10.729066722 -0400
  2825. |17747|3243:3604|reserved| Modify: 2013-05-27 08:17:13.370647610 -0400
  2826. |17747|3243:3604|reserved| Change: 2013-05-27 08:17:13.376594496 -0400
  2827. 2013-05-28 13:56:32|17747|3243:3604|reserved|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "File: `/etc/init.d/vcl_post_re...")
  2828. 2013-05-28 13:56:32|17747|3243:3604|reserved|Linux.pm:file_exists(1449)|'/etc/init.d/vcl_post_reserve' exists on ecco-vm-x10, files: 1, directories: 0, links: 0
  2829. 2013-05-28 13:56:32|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2830. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'stat /etc/init\.d/vcl_post_reserve' 2>&1
  2831. 2013-05-28 13:56:33|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2832. |17747|3243:3604|reserved| File: `/etc/init.d/vcl_post_reserve'
  2833. |17747|3243:3604|reserved| Size: 96 Blocks: 8 IO Block: 4096 regular file
  2834. |17747|3243:3604|reserved| Device: 802h/2050d Inode: 7983 Links: 1
  2835. |17747|3243:3604|reserved| Access: (0700/-rwx------) Uid: ( 0/ root) Gid: ( 0/ root)
  2836. |17747|3243:3604|reserved| Access: 2013-05-28 13:58:10.729066722 -0400
  2837. |17747|3243:3604|reserved| Modify: 2013-05-27 08:17:13.370647610 -0400
  2838. |17747|3243:3604|reserved| Change: 2013-05-27 08:17:13.376594496 -0400
  2839. 2013-05-28 13:56:33|17747|3243:3604|reserved|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "File: `/etc/init.d/vcl_post_re...")
  2840. 2013-05-28 13:56:33|17747|3243:3604|reserved|Linux.pm:file_exists(1449)|'/etc/init.d/vcl_post_reserve' exists on ecco-vm-x10, files: 1, directories: 0, links: 0
  2841. 2013-05-28 13:56:33|17747|3243:3604|reserved|Linux.pm:run_script(1348)|script exists: /etc/init.d/vcl_post_reserve
  2842. 2013-05-28 13:56:33|17747|3243:3604|reserved|Linux.pm:run_script(1358)|script name: vcl_post_reserve
  2843. 2013-05-28 13:56:33|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2844. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'ls -d --color=never "/root/VCL/Logs" 2>&1 || mkdir -p "/root/VCL/Logs" 2>&1 && ls -d --color=never "/root/VCL/Logs"' 2>&1
  2845. 2013-05-28 13:56:33|17747|3243:3604|reserved|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2846. |17747|3243:3604|reserved| /root/VCL/Logs
  2847. |17747|3243:3604|reserved| /root/VCL/Logs
  2848. 2013-05-28 13:56:33|17747|3243:3604|reserved|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x10, returning (0, "/root/VCL/Logs /root/VCL/Logs")
  2849. 2013-05-28 13:56:33|17747|3243:3604|reserved|Linux.pm:create_directory(1573)|directory already exists on ecco-vm-x10: '/root/VCL/Logs'
  2850. 2013-05-28 13:56:33|17747|3243:3604|reserved|Linux.pm:run_script(1372)|script log file path: /root/VCL/Logs/vcl_post_reserve.log
  2851. 2013-05-28 13:56:33|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  2852. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'chmod +rx "/etc/init.d/vcl_post_reserve" && "/etc/init.d/vcl_post_reserve" >> "/root/VCL/Logs/vcl_post_reserve.log" 2>&1' 2>&1
  2853. 2013-05-28 13:56:33|12889|3243:3605|reinstall|utils.pm:get_management_node_info(5456)|retrieving current management node info for 'ecco-adm3.sharcnet.ca' from database, cached data is stale: 62 seconds old
  2854. 2013-05-28 13:56:33|12889|3243:3605|reinstall|utils.pm:get_management_node_info(5603)|retrieved management node info: 'ecco-adm3.sharcnet.ca' (ecco-adm3)
  2855. 2013-05-28 13:56:33|12889|3243:3605|reinstall|OS.pm:is_ssh_responding(678)|ecco-vm-x26 is responding to SSH, port 22: open, port 24: closed
  2856. 2013-05-28 13:56:33|12889|3243:3605|reinstall|Module.pm:code_loop_timeout(900)|waiting for ecco-vm-x26 to respond to SSH, code returned true
  2857. 2013-05-28 13:56:33|12889|3243:3605|reinstall|OS.pm:wait_for_response(750)|ecco-vm-x26 is responding to SSH after 42 seconds
  2858. 2013-05-28 13:56:33|12889|3243:3605|reinstall|OS.pm:update_public_ip_address(941)|IP configuration is set to dynamicDHCP, attempting to retrieve dynamic public IP address from ecco-vm-x26
  2859. 2013-05-28 13:56:33|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  2860. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 '/sbin/ifconfig -a' 2>&1
  2861. 2013-05-28 13:56:34|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2862. |12889|3243:3605|reinstall| eth0 Link encap:Ethernet HWaddr 52:54:00:01:00:19
  2863. |12889|3243:3605|reinstall| inet addr:10.18.20.226 Bcast:10.18.21.255 Mask:255.255.254.0
  2864. |12889|3243:3605|reinstall| inet6 addr: fe80::5054:ff:fe01:19/64 Scope:Link
  2865. |12889|3243:3605|reinstall| UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
  2866. |12889|3243:3605|reinstall| RX packets:135 errors:0 dropped:0 overruns:0 frame:0
  2867. |12889|3243:3605|reinstall| TX packets:116 errors:0 dropped:0 overruns:0 carrier:0
  2868. |12889|3243:3605|reinstall| collisions:0 txqueuelen:1000
  2869. |12889|3243:3605|reinstall| RX bytes:19164 (18.7 KiB) TX bytes:15569 (15.2 KiB)
  2870. |12889|3243:3605|reinstall| Interrupt:11 Base address:0xc000
  2871. |12889|3243:3605|reinstall| eth1 Link encap:Ethernet HWaddr 52:54:00:01:80:19
  2872. |12889|3243:3605|reinstall| inet addr:199.241.160.149 Bcast:199.241.160.255 Mask:255.255.255.0
  2873. |12889|3243:3605|reinstall| inet6 addr: fe80::5054:ff:fe01:8019/64 Scope:Link
  2874. |12889|3243:3605|reinstall| UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
  2875. |12889|3243:3605|reinstall| RX packets:22 errors:0 dropped:0 overruns:0 frame:0
  2876. |12889|3243:3605|reinstall| TX packets:29 errors:0 dropped:0 overruns:0 carrier:0
  2877. |12889|3243:3605|reinstall| collisions:0 txqueuelen:1000
  2878. |12889|3243:3605|reinstall| RX bytes:1710 (1.6 KiB) TX bytes:4773 (4.6 KiB)
  2879. |12889|3243:3605|reinstall| Interrupt:11 Base address:0x2000
  2880. |12889|3243:3605|reinstall| lo Link encap:Local Loopback
  2881. |12889|3243:3605|reinstall| inet addr:127.0.0.1 Mask:255.0.0.0
  2882. |12889|3243:3605|reinstall| inet6 addr: ::1/128 Scope:Host
  2883. |12889|3243:3605|reinstall| UP LOOPBACK RUNNING MTU:16436 Metric:1
  2884. |12889|3243:3605|reinstall| RX packets:4 errors:0 dropped:0 overruns:0 frame:0
  2885. |12889|3243:3605|reinstall| TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
  2886. |12889|3243:3605|reinstall| collisions:0 txqueuelen:0
  2887. |12889|3243:3605|reinstall| RX bytes:240 (240.0 b) TX bytes:240 (240.0 b)
  2888. 2013-05-28 13:56:34|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x26, returning (0, "eth0 Link encap:Ethernet HWadd...")
  2889. 2013-05-28 13:56:34|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  2890. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 '/sbin/route -n' 2>&1
  2891. 2013-05-28 13:56:34|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2892. |12889|3243:3605|reinstall| Kernel IP routing table
  2893. |12889|3243:3605|reinstall| Destination Gateway Genmask Flags Metric Ref Use Iface
  2894. |12889|3243:3605|reinstall| 199.241.160.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
  2895. |12889|3243:3605|reinstall| 10.18.20.0 0.0.0.0 255.255.254.0 U 0 0 0 eth0
  2896. |12889|3243:3605|reinstall| 169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0
  2897. |12889|3243:3605|reinstall| 169.254.0.0 0.0.0.0 255.255.0.0 U 1003 0 0 eth1
  2898. |12889|3243:3605|reinstall| 10.0.0.0 10.18.21.254 255.0.0.0 UG 0 0 0 eth0
  2899. |12889|3243:3605|reinstall| 0.0.0.0 199.241.160.1 0.0.0.0 UG 0 0 0 eth1
  2900. 2013-05-28 13:56:34|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x26, returning (0, "Kernel IP routing table Destin...")
  2901. 2013-05-28 13:56:34|12889|3243:3605|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: 'Kernel IP routing table'
  2902. 2013-05-28 13:56:34|12889|3243:3605|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: 'Destination Gateway Genmask Flags Metric Ref Use Iface'
  2903. 2013-05-28 13:56:34|12889|3243:3605|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '199.241.160.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1'
  2904. 2013-05-28 13:56:34|12889|3243:3605|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '10.18.20.0 0.0.0.0 255.255.254.0 U 0 0 0 eth0'
  2905. 2013-05-28 13:56:34|12889|3243:3605|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0'
  2906. 2013-05-28 13:56:34|12889|3243:3605|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '169.254.0.0 0.0.0.0 255.255.0.0 U 1003 0 0 eth1'
  2907. 2013-05-28 13:56:34|12889|3243:3605|reinstall|Linux.pm:get_network_configuration(2692)|route output line does not contain a default gateway: '10.0.0.0 10.18.21.254 255.0.0.0 UG 0 0 0 eth0'
  2908. 2013-05-28 13:56:34|12889|3243:3605|reinstall|Linux.pm:get_network_configuration(2703)|found default route configured for 'eth1' interface: 199.241.160.1
  2909. 2013-05-28 13:56:34|12889|3243:3605|reinstall|Linux.pm:get_network_configuration(2708)|retrieved network configuration:
  2910. |12889|3243:3605|reinstall| : {
  2911. |12889|3243:3605|reinstall| : "eth0" => {
  2912. |12889|3243:3605|reinstall| : "broadcast_address" => "10.18.21.255",
  2913. |12889|3243:3605|reinstall| : "ip_address" => {
  2914. |12889|3243:3605|reinstall| : "10.18.20.226" => "255.255.254.0"
  2915. |12889|3243:3605|reinstall| : },
  2916. |12889|3243:3605|reinstall| : "name" => "eth0",
  2917. |12889|3243:3605|reinstall| : "physical_address" => "52:54:00:01:00:19"
  2918. |12889|3243:3605|reinstall| : },
  2919. |12889|3243:3605|reinstall| : "eth1" => {
  2920. |12889|3243:3605|reinstall| : "broadcast_address" => "199.241.160.255",
  2921. |12889|3243:3605|reinstall| : "default_gateway" => "199.241.160.1",
  2922. |12889|3243:3605|reinstall| : "ip_address" => {
  2923. |12889|3243:3605|reinstall| : "199.241.160.149" => "255.255.255.0"
  2924. |12889|3243:3605|reinstall| : },
  2925. |12889|3243:3605|reinstall| : "name" => "eth1",
  2926. |12889|3243:3605|reinstall| : "physical_address" => "52:54:00:01:80:19"
  2927. |12889|3243:3605|reinstall| : },
  2928. |12889|3243:3605|reinstall| : "lo" => {
  2929. |12889|3243:3605|reinstall| : "name" => "lo"
  2930. |12889|3243:3605|reinstall| : }
  2931. |12889|3243:3605|reinstall| : }
  2932. 2013-05-28 13:56:34|12889|3243:3605|reinstall|DataStructure.pm:get_computer_private_ip_address(1641)|attempting to retrieve private IP address for computer: ecco-vm-x26
  2933. 2013-05-28 13:56:34|12889|3243:3605|reinstall|OS.pm:get_file_contents(1897)|retrieved 146 lines from file on ecco-adm3: '/etc/hosts'
  2934. 2013-05-28 13:56:34|12889|3243:3605|reinstall|DataStructure.pm:get_computer_private_ip_address(1697)|returning IP address from /etc/hosts file: 10.18.20.226
  2935. 2013-05-28 13:56:34|12889|3243:3605|reinstall|OS.pm:get_public_interface_name(1196)|'eth0' could not be the public interface, it is only assigned the private IP address
  2936. 2013-05-28 13:56:34|12889|3243:3605|reinstall|OS.pm:get_public_interface_name(1220)|'eth1' could potententially be public interface, not assigned private IP address
  2937. 2013-05-28 13:56:34|12889|3243:3605|reinstall|OS.pm:get_public_interface_name(1177)|interface 'lo' ignored because its name is 'lo'
  2938. 2013-05-28 13:56:34|12889|3243:3605|reinstall|OS.pm:get_public_interface_name(1238)|determined the public interface name: 'eth1'
  2939. |12889|3243:3605|reinstall| : {
  2940. |12889|3243:3605|reinstall| : "broadcast_address" => "199.241.160.255",
  2941. |12889|3243:3605|reinstall| : "default_gateway" => "199.241.160.1",
  2942. |12889|3243:3605|reinstall| : "ip_address" => {
  2943. |12889|3243:3605|reinstall| : "199.241.160.149" => "255.255.255.0"
  2944. |12889|3243:3605|reinstall| : },
  2945. |12889|3243:3605|reinstall| : "name" => "eth1",
  2946. |12889|3243:3605|reinstall| : "physical_address" => "52:54:00:01:80:19"
  2947. |12889|3243:3605|reinstall| : }
  2948. 2013-05-28 13:56:34|12889|3243:3605|reinstall|OS.pm:get_ip_address(1538)|returning public IP address: 199.241.160.149
  2949. 2013-05-28 13:56:34|12889|3243:3605|reinstall|OS.pm:update_public_ip_address(946)|retrieved public IP address from ecco-vm-x26 using the OS module: 199.241.160.149
  2950. 2013-05-28 13:56:34|12889|3243:3605|reinstall|OS.pm:update_public_ip_address(974)|public IP address in computer table is already correct for ecco-vm-x26: 199.241.160.149
  2951. 2013-05-28 13:56:34|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  2952. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'file /etc/rc.local' 2>&1
  2953. 2013-05-28 13:56:34|12889|3243:3605|reinstall|OS.pm:execute(1994)|executed command: 'file /etc/rc.local', exit status: 0, output:
  2954. |12889|3243:3605|reinstall| /etc/rc.local: symbolic link to `rc.d/rc.local'
  2955. 2013-05-28 13:56:34|12889|3243:3605|reinstall|Linux.pm:generate_rc_local(2053)|confirmed /etc/rc.local is symbolic link
  2956. |12889|3243:3605|reinstall| /etc/rc.local: symbolic link to `rc.d/rc.local'
  2957. 2013-05-28 13:56:34|12889|3243:3605|reinstall|utils.pm:run_scp_command(5156)|attempting to copy file via SCP: '/tmp/3243.rc.local' --> 'ecco-vm-x26:/etc/rc.d/rc.local'
  2958. 2013-05-28 13:56:34|12889|3243:3605|reinstall|utils.pm:run_scp_command(5192)|copied file via SCP: '/tmp/3243.rc.local' --> 'ecco-vm-x26:/etc/rc.d/rc.local'
  2959. 2013-05-28 13:56:34|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  2960. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'chmod +rx /etc/rc.d/rc.local' 2>&1
  2961. 2013-05-28 13:56:34|12889|3243:3605|reinstall|Linux.pm:generate_rc_local(2127)|executed chmod +rx /etc/rc.d/rc.local, exit status: 0
  2962. 2013-05-28 13:56:34|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  2963. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'stat /bin/systemctl' 2>&1
  2964. 2013-05-28 13:56:34|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2965. |12889|3243:3605|reinstall| stat: cannot stat `/bin/systemctl': No such file or directory
  2966. 2013-05-28 13:56:34|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vm-x26, command:
  2967. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'stat /bin/systemctl' 2>&1
  2968. |12889|3243:3605|reinstall| returning (1, "stat: cannot stat `/bin/system...")
  2969. 2013-05-28 13:56:34|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  2970. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 '/etc/rc.d/rc.local' 2>&1
  2971. 2013-05-28 13:56:36|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:56:36
  2972. 2013-05-28 13:56:36|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2973. |12889|3243:3605|reinstall| Stopping ext_sshd:[ OK ]
  2974. |12889|3243:3605|reinstall| Stopping sshd: [ OK ]
  2975. |12889|3243:3605|reinstall| Starting sshd: [ OK ]
  2976. |12889|3243:3605|reinstall| Starting ext_sshd: [ OK ]
  2977. 2013-05-28 13:56:36|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x26, returning (0, "Stopping ext_sshd:[ OK ] Stopp...")
  2978. 2013-05-28 13:56:36|12889|3243:3605|reinstall|Linux.pm:generate_rc_local(2156)|successfully ran /etc/rc.d/rc.local
  2979. 2013-05-28 13:56:36|12889|3243:3605|reinstall|DataStructure.pm:is_variable_set(1779)|variable is NOT set: timesource|ecco-adm3.sharcnet.ca
  2980. 2013-05-28 13:56:36|12889|3243:3605|reinstall|DataStructure.pm:is_variable_set(1791)|variable is set: timesource|global
  2981. 2013-05-28 13:56:36|12889|3243:3605|reinstall|DataStructure.pm:get_variable(1890)|retrieved variable 'timesource|global', serialization: none
  2982. 2013-05-28 13:56:36|12889|3243:3605|reinstall|Linux.pm:sync_date(1068)|time_source is time.nist.gov,time-a.nist.gov,time-b.nist.gov,time.windows.com set for timesource|ecco-adm3.sharcnet.ca
  2983. 2013-05-28 13:56:36|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  2984. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'rdate -s time.nist.gov time-a.nist.gov time-b.nist.gov time.windows.com' 2>&1
  2985. 2013-05-28 13:56:37|12889|3243:3605|reinstall|OS.pm:execute(1994)|executed command: 'rdate -s time.nist.gov time-a.nist.gov time-b.nist.gov time.windows.com', exit status: 127, output:
  2986. |12889|3243:3605|reinstall| bash: rdate: command not found
  2987. 2013-05-28 13:56:37|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  2988. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'cp /dev/null /etc/ntp/ntpservers;echo time.nist.gov >> /etc/ntp/ntpservers; echo time-a.nist.gov >> /etc/ntp/ntpservers; echo time-b.nist.gov >> /etc/ntp/ntpservers; echo time.windows.com >> /etc/ntp/ntpservers; /etc/init.d/ntpd restart' 2>&1
  2989. 2013-05-28 13:56:37|12889|3243:3605|reinstall|OS.pm:execute(1994)|executed command: 'cp /dev/null /etc/ntp/ntpservers;echo time.nist.gov >> /etc/ntp/ntpservers; echo time-a.nist.gov >> /etc/ntp/ntpservers; echo time-b.nist.gov >> /etc/ntp/ntpservers; echo time.windows.com >> /etc/ntp/ntpservers; /etc/init.d/ntpd restart', exit status: 0, output:
  2990. |12889|3243:3605|reinstall| Shutting down ntpd: [ OK ]
  2991. |12889|3243:3605|reinstall| Starting ntpd: [ OK ]
  2992. 2013-05-28 13:56:37|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  2993. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'echo 9Uq2VeWXgYDQQsu | /usr/bin/passwd -f root --stdin' 2>&1
  2994. 2013-05-28 13:56:37|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  2995. |12889|3243:3605|reinstall| Changing password for user root.
  2996. |12889|3243:3605|reinstall| passwd: all authentication tokens updated successfully.
  2997. 2013-05-28 13:56:37|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x26, returning (0, "Changing password for user roo...")
  2998. 2013-05-28 13:56:37|12889|3243:3605|reinstall|Linux.pm:changepasswd(1133)|changed password for account: root, output:
  2999. |12889|3243:3605|reinstall| Changing password for user root.
  3000. |12889|3243:3605|reinstall| passwd: all authentication tokens updated successfully.
  3001. 2013-05-28 13:56:37|12889|3243:3605|reinstall|Linux.pm:post_load(287)|successfully changed root password on ecco-vm-x26
  3002. 2013-05-28 13:56:37|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  3003. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'stat /bin/systemctl' 2>&1
  3004. 2013-05-28 13:56:37|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3005. |12889|3243:3605|reinstall| stat: cannot stat `/bin/systemctl': No such file or directory
  3006. 2013-05-28 13:56:37|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vm-x26, command:
  3007. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'stat /bin/systemctl' 2>&1
  3008. |12889|3243:3605|reinstall| returning (1, "stat: cannot stat `/bin/system...")
  3009. 2013-05-28 13:56:37|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  3010. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'service ext_sshd stop' 2>&1
  3011. 2013-05-28 13:56:37|12889|3243:3605|reinstall|Linux.pm:stop_service(3298)|service exists: ext_sshd
  3012. 2013-05-28 13:56:37|12889|3243:3605|reinstall|Linux.pm:post_load(295)|ext sshd stopped on ecco-vm-x26
  3013. 2013-05-28 13:56:37|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  3014. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'sed -i -e "/^AllowUsers .*/d" /etc/ssh/external_sshd_config' 2>&1
  3015. 2013-05-28 13:56:37|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3016. 2013-05-28 13:56:37|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x26, returning (0, "")
  3017. 2013-05-28 13:56:37|12889|3243:3605|reinstall|Linux.pm:post_load(301)|cleared AllowUsers directive from external_sshd_config
  3018. 2013-05-28 13:56:37|12889|3243:3605|reinstall|Linux.pm:post_load(308)|calling clear_private_keys
  3019. 2013-05-28 13:56:37|12889|3243:3605|reinstall|Linux.pm:clear_private_keys(492)|perparing to clear known identity keys
  3020. 2013-05-28 13:56:37|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  3021. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 '/bin/rm -f /root/.ssh/id_rsa /root/.ssh/id_rsa.pub' 2>&1
  3022. 2013-05-28 13:56:38|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3023. 2013-05-28 13:56:38|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x26, returning (0, "")
  3024. 2013-05-28 13:56:38|12889|3243:3605|reinstall|Linux.pm:clear_private_keys(500)|cleared any id_rsa keys from /root/.ssh
  3025. 2013-05-28 13:56:38|12889|3243:3605|reinstall|Linux.pm:post_load(310)|cleared known identity keys
  3026. 2013-05-28 13:56:38|12889|3243:3605|reinstall|OS.pm:get_public_interface_name(1150)|returning public interface name previously retrieved: eth1
  3027. 2013-05-28 13:56:38|12889|3243:3605|reinstall|OS.pm:get_ip_address(1538)|returning public IP address: 199.241.160.149
  3028. 2013-05-28 13:56:38|12889|3243:3605|reinstall|Linux.pm:update_public_hostname(424)|retrieved public IP address of ecco-vm-x26: 199.241.160.149
  3029. 2013-05-28 13:56:38|12889|3243:3605|reinstall|utils.pm:run_command(8710)|executed command: /bin/ipcalc --hostname 199.241.160.149, pid: 19465, exit status: 0, output:
  3030. |12889|3243:3605|reinstall| HOSTNAME=ecco-computer22.sharcnet.ca
  3031. 2013-05-28 13:56:38|12889|3243:3605|reinstall|Linux.pm:update_public_hostname(435)|determined registered public hostname of ecco-vm-x26 (199.241.160.149) by running ipcalc on the management node: 'ecco-computer22.sharcnet.ca'
  3032. 2013-05-28 13:56:38|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  3033. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'hostname -v ecco-computer22.sharcnet.ca' 2>&1
  3034. 2013-05-28 13:56:38|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3035. |12889|3243:3605|reinstall| Setting hostname to `ecco-computer22.sharcnet.ca'
  3036. 2013-05-28 13:56:38|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x26, returning (0, "Setting hostname to `ecco-comp...")
  3037. 2013-05-28 13:56:38|12889|3243:3605|reinstall|Linux.pm:update_public_hostname(467)|set public hostname on ecco-vm-x26 to ecco-computer22.sharcnet.ca
  3038. 2013-05-28 13:56:38|12889|3243:3605|reinstall|Linux.pm:post_load(315)|Updated hostname
  3039. 2013-05-28 13:56:38|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  3040. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'stat /etc/init\.d/vcl_post_load' 2>&1
  3041. 2013-05-28 13:56:38|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3042. |12889|3243:3605|reinstall| File: `/etc/init.d/vcl_post_load'
  3043. |12889|3243:3605|reinstall| Size: 43 Blocks: 8 IO Block: 4096 regular file
  3044. |12889|3243:3605|reinstall| Device: 802h/2050d Inode: 7976 Links: 1
  3045. |12889|3243:3605|reinstall| Access: (0700/-rwx------) Uid: ( 0/ root) Gid: ( 0/ root)
  3046. |12889|3243:3605|reinstall| Access: 2013-05-27 08:15:46.167661786 -0400
  3047. |12889|3243:3605|reinstall| Modify: 2013-05-27 08:17:13.367712524 -0400
  3048. |12889|3243:3605|reinstall| Change: 2013-05-27 08:17:13.375594944 -0400
  3049. 2013-05-28 13:56:38|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x26, returning (0, "File: `/etc/init.d/vcl_post_lo...")
  3050. 2013-05-28 13:56:38|12889|3243:3605|reinstall|Linux.pm:file_exists(1449)|'/etc/init.d/vcl_post_load' exists on ecco-vm-x26, files: 1, directories: 0, links: 0
  3051. 2013-05-28 13:56:38|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  3052. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'stat /etc/init\.d/vcl_post_load' 2>&1
  3053. 2013-05-28 13:56:38|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3054. |12889|3243:3605|reinstall| File: `/etc/init.d/vcl_post_load'
  3055. |12889|3243:3605|reinstall| Size: 43 Blocks: 8 IO Block: 4096 regular file
  3056. |12889|3243:3605|reinstall| Device: 802h/2050d Inode: 7976 Links: 1
  3057. |12889|3243:3605|reinstall| Access: (0700/-rwx------) Uid: ( 0/ root) Gid: ( 0/ root)
  3058. |12889|3243:3605|reinstall| Access: 2013-05-27 08:15:46.167661786 -0400
  3059. |12889|3243:3605|reinstall| Modify: 2013-05-27 08:17:13.367712524 -0400
  3060. |12889|3243:3605|reinstall| Change: 2013-05-27 08:17:13.375594944 -0400
  3061. 2013-05-28 13:56:38|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x26, returning (0, "File: `/etc/init.d/vcl_post_lo...")
  3062. 2013-05-28 13:56:38|12889|3243:3605|reinstall|Linux.pm:file_exists(1449)|'/etc/init.d/vcl_post_load' exists on ecco-vm-x26, files: 1, directories: 0, links: 0
  3063. 2013-05-28 13:56:38|12889|3243:3605|reinstall|Linux.pm:run_script(1348)|script exists: /etc/init.d/vcl_post_load
  3064. 2013-05-28 13:56:38|12889|3243:3605|reinstall|Linux.pm:run_script(1358)|script name: vcl_post_load
  3065. 2013-05-28 13:56:38|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  3066. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'ls -d --color=never "/root/VCL/Logs" 2>&1 || mkdir -p "/root/VCL/Logs" 2>&1 && ls -d --color=never "/root/VCL/Logs"' 2>&1
  3067. 2013-05-28 13:56:38|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3068. |12889|3243:3605|reinstall| /root/VCL/Logs
  3069. |12889|3243:3605|reinstall| /root/VCL/Logs
  3070. 2013-05-28 13:56:38|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x26, returning (0, "/root/VCL/Logs /root/VCL/Logs")
  3071. 2013-05-28 13:56:38|12889|3243:3605|reinstall|Linux.pm:create_directory(1573)|directory already exists on ecco-vm-x26: '/root/VCL/Logs'
  3072. 2013-05-28 13:56:38|12889|3243:3605|reinstall|Linux.pm:run_script(1372)|script log file path: /root/VCL/Logs/vcl_post_load.log
  3073. 2013-05-28 13:56:38|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  3074. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'chmod +rx "/etc/init.d/vcl_post_load" && "/etc/init.d/vcl_post_load" >> "/root/VCL/Logs/vcl_post_load.log" 2>&1' 2>&1
  3075. 2013-05-28 13:56:38|17747|3243:3604|reserved|OS.pm:execute(1994)|executed command: 'chmod +rx "/etc/init.d/vcl_post_reserve" && "/etc/init.d/vcl_post_reserve" >> "/root/VCL/Logs/vcl_post_reserve.log" 2>&1', exit status: 0, output:
  3076. 2013-05-28 13:56:38|17747|3243:3604|reserved|Linux.pm:run_script(1379)|executed /etc/init.d/vcl_post_reserve, exit status: 0
  3077. 2013-05-28 13:56:38|17747|3243:3604|reserved|Linux.pm:post_reserve(386)|ran /etc/init.d/vcl_post_reserve
  3078. 2013-05-28 13:56:38|17747|3243:3604|reserved|reserved.pm:process(221)|performed OS post_reserve tasks for linux-TestImage2-16-v2 on ecco-vm-x10
  3079. 2013-05-28 13:56:38|17747|3243:3604|reserved|utils.pm:insertloadlog(3665)|inserted computer=18, info, performed OS post_reserve tasks for linux-TestImage2-16-v2 on ecco-vm-x10
  3080. 2013-05-28 13:56:38|17747|3243:3604|reserved|reserved.pm:process(233)|server_request_id = 0
  3081. 2013-05-28 13:56:38|17747|3243:3604|reserved|OS.pm:is_user_connected(2676)|checking for connection by nsasherr on ecco-vm-x10
  3082. 2013-05-28 13:56:38|17747|3243:3604|reserved|DataStructure.pm:get_reservation_remote_ip(1165)|retrieved remote IP for reservation 3604: 129.100.171.120
  3083. 2013-05-28 13:56:38|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  3084. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'netstat -an' 2>&1
  3085. 2013-05-28 13:56:38|17747|3243:3604|reserved|Linux.pm:check_connection_on_port(3350)|checking connections on node ecco-vm-x10 on port 22
  3086. 2013-05-28 13:56:38|17747|3243:3604|reserved|DataStructure.pm:get_reservation_remote_ip(1165)|retrieved remote IP for reservation 3604: 129.100.171.120
  3087. 2013-05-28 13:56:38|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  3088. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'netstat -an' 2>&1
  3089. 2013-05-28 13:56:38|17747|3243:3604|reserved|Linux.pm:check_connection_on_port(3350)|checking connections on node ecco-vm-x10 on port 5900
  3090. 2013-05-28 13:56:38|17747|3243:3604|reserved|OS.pm:is_user_connected(2723)|nsasherr has not connected to ecco-vm-x10 (0/900 seconds elapsed/remaining), sleeping for 15 seconds
  3091. 2013-05-28 13:56:40|12889|3243:3605|reinstall|OS.pm:execute(1994)|executed command: 'chmod +rx "/etc/init.d/vcl_post_load" && "/etc/init.d/vcl_post_load" >> "/root/VCL/Logs/vcl_post_load.log" 2>&1', exit status: 0, output:
  3092. 2013-05-28 13:56:40|12889|3243:3605|reinstall|Linux.pm:run_script(1379)|executed /etc/init.d/vcl_post_load, exit status: 0
  3093. 2013-05-28 13:56:40|12889|3243:3605|reinstall|Linux.pm:post_load(330)|ran /etc/init.d/vcl_post_load
  3094. 2013-05-28 13:56:40|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  3095. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'stat /bin/systemctl' 2>&1
  3096. 2013-05-28 13:56:40|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3097. |12889|3243:3605|reinstall| stat: cannot stat `/bin/systemctl': No such file or directory
  3098. 2013-05-28 13:56:40|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vm-x26, command:
  3099. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'stat /bin/systemctl' 2>&1
  3100. |12889|3243:3605|reinstall| returning (1, "stat: cannot stat `/bin/system...")
  3101. 2013-05-28 13:56:40|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  3102. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'stat /sbin/chkconfig' 2>&1
  3103. 2013-05-28 13:56:40|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3104. |12889|3243:3605|reinstall| File: `/sbin/chkconfig'
  3105. |12889|3243:3605|reinstall| Size: 31728 Blocks: 64 IO Block: 4096 regular file
  3106. |12889|3243:3605|reinstall| Device: 802h/2050d Inode: 642 Links: 1
  3107. |12889|3243:3605|reinstall| Access: (0755/-rwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
  3108. |12889|3243:3605|reinstall| Access: 2013-05-28 13:58:30.965188891 -0400
  3109. |12889|3243:3605|reinstall| Modify: 2012-06-22 06:09:02.000000000 -0400
  3110. |12889|3243:3605|reinstall| Change: 2012-10-09 15:03:55.140697475 -0400
  3111. 2013-05-28 13:56:40|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x26, returning (0, "File: `/sbin/chkconfig' Size: ...")
  3112. 2013-05-28 13:56:40|12889|3243:3605|reinstall|Linux.pm:file_exists(1449)|'/sbin/chkconfig' exists on ecco-vm-x26, files: 1, directories: 0, links: 0
  3113. 2013-05-28 13:56:40|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  3114. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 '/sbin/chkconfig --list iptables' 2>&1
  3115. 2013-05-28 13:56:40|12889|3243:3605|reinstall|Linux.pm:service_exists(3187)|'iptables' service exists
  3116. 2013-05-28 13:56:40|12889|3243:3605|reinstall|OS.pm:get_ip_address(1538)|returning private IP address: 10.18.20.4
  3117. 2013-05-28 13:56:40|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  3118. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'stat /bin/systemctl' 2>&1
  3119. 2013-05-28 13:56:40|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3120. |12889|3243:3605|reinstall| stat: cannot stat `/bin/systemctl': No such file or directory
  3121. 2013-05-28 13:56:40|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vm-x26, command:
  3122. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'stat /bin/systemctl' 2>&1
  3123. |12889|3243:3605|reinstall| returning (1, "stat: cannot stat `/bin/system...")
  3124. 2013-05-28 13:56:40|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  3125. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'stat /sbin/chkconfig' 2>&1
  3126. 2013-05-28 13:56:40|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3127. |12889|3243:3605|reinstall| File: `/sbin/chkconfig'
  3128. |12889|3243:3605|reinstall| Size: 31728 Blocks: 64 IO Block: 4096 regular file
  3129. |12889|3243:3605|reinstall| Device: 802h/2050d Inode: 642 Links: 1
  3130. |12889|3243:3605|reinstall| Access: (0755/-rwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
  3131. |12889|3243:3605|reinstall| Access: 2013-05-28 13:58:30.965188891 -0400
  3132. |12889|3243:3605|reinstall| Modify: 2012-06-22 06:09:02.000000000 -0400
  3133. |12889|3243:3605|reinstall| Change: 2012-10-09 15:03:55.140697475 -0400
  3134. 2013-05-28 13:56:40|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x26, returning (0, "File: `/sbin/chkconfig' Size: ...")
  3135. 2013-05-28 13:56:40|12889|3243:3605|reinstall|Linux.pm:file_exists(1449)|'/sbin/chkconfig' exists on ecco-vm-x26, files: 1, directories: 0, links: 0
  3136. 2013-05-28 13:56:40|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  3137. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 '/sbin/chkconfig --list iptables' 2>&1
  3138. 2013-05-28 13:56:41|12889|3243:3605|reinstall|Linux.pm:service_exists(3187)|'iptables' service exists
  3139. 2013-05-28 13:56:41|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  3140. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'iptables -L --line-number -n' 2>&1
  3141. 2013-05-28 13:56:41|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3142. |12889|3243:3605|reinstall| Chain INPUT (policy ACCEPT)
  3143. |12889|3243:3605|reinstall| num target prot opt source destination
  3144. |12889|3243:3605|reinstall| 1 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,RELATED,ESTABLISHED tcp dpt:5900
  3145. |12889|3243:3605|reinstall| 2 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW,RELATED,ESTABLISHED tcp dpt:22
  3146. |12889|3243:3605|reinstall| 3 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
  3147. |12889|3243:3605|reinstall| 4 ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0
  3148. |12889|3243:3605|reinstall| 5 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
  3149. |12889|3243:3605|reinstall| 6 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
  3150. |12889|3243:3605|reinstall| Chain FORWARD (policy ACCEPT)
  3151. |12889|3243:3605|reinstall| num target prot opt source destination
  3152. |12889|3243:3605|reinstall| 1 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
  3153. |12889|3243:3605|reinstall| Chain OUTPUT (policy ACCEPT)
  3154. |12889|3243:3605|reinstall| num target prot opt source destination
  3155. 2013-05-28 13:56:41|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x26, returning (0, "Chain INPUT (policy ACCEPT) nu...")
  3156. 2013-05-28 13:56:41|12889|3243:3605|reinstall|Linux.pm:get_firewall_configuration(3992)|output Chain = INPUT
  3157. 2013-05-28 13:56:41|12889|3243:3605|reinstall|Linux.pm:get_firewall_configuration(4008)|output rule: 1, ACCEPT, tcp, 0.0.0.0/0, 0.0.0.0/0, 5900
  3158. 2013-05-28 13:56:41|12889|3243:3605|reinstall|Linux.pm:get_firewall_configuration(4008)|output rule: 2, ACCEPT, tcp, 0.0.0.0/0, 0.0.0.0/0, 22
  3159. 2013-05-28 13:56:41|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:56:41
  3160. 2013-05-28 13:56:43|12889|3243:3605|reinstall|Linux.pm:get_firewall_configuration(3992)|output Chain = FORWARD
  3161. 2013-05-28 13:56:43|12889|3243:3605|reinstall|Linux.pm:get_firewall_configuration(3992)|output Chain = OUTPUT
  3162. 2013-05-28 13:56:43|12889|3243:3605|reinstall|Linux.pm:get_firewall_configuration(4051)|retrieved firewall configuration from ecco-vm-x26:
  3163. |12889|3243:3605|reinstall| : {
  3164. |12889|3243:3605|reinstall| : "FORWARD" => {
  3165. |12889|3243:3605|reinstall| : 1 => {
  3166. |12889|3243:3605|reinstall| : "all" => {
  3167. |12889|3243:3605|reinstall| : "any" => {
  3168. |12889|3243:3605|reinstall| : "destination" => "0.0.0.0/0",
  3169. |12889|3243:3605|reinstall| : "name" => "any",
  3170. |12889|3243:3605|reinstall| : "number" => 1,
  3171. |12889|3243:3605|reinstall| : "scope" => "0.0.0.0/0",
  3172. |12889|3243:3605|reinstall| : "target" => "REJECT"
  3173. |12889|3243:3605|reinstall| : }
  3174. |12889|3243:3605|reinstall| : }
  3175. |12889|3243:3605|reinstall| : }
  3176. |12889|3243:3605|reinstall| : },
  3177. |12889|3243:3605|reinstall| : "INPUT" => {
  3178. |12889|3243:3605|reinstall| : 1 => {
  3179. |12889|3243:3605|reinstall| : "tcp" => {
  3180. |12889|3243:3605|reinstall| : 5900 => {
  3181. |12889|3243:3605|reinstall| : "destination" => "0.0.0.0/0",
  3182. |12889|3243:3605|reinstall| : "name" => 5900,
  3183. |12889|3243:3605|reinstall| : "number" => 1,
  3184. |12889|3243:3605|reinstall| : "scope" => "0.0.0.0/0",
  3185. |12889|3243:3605|reinstall| : "target" => "ACCEPT"
  3186. |12889|3243:3605|reinstall| : }
  3187. |12889|3243:3605|reinstall| : }
  3188. |12889|3243:3605|reinstall| : },
  3189. |12889|3243:3605|reinstall| : 2 => {
  3190. |12889|3243:3605|reinstall| : "tcp" => {
  3191. |12889|3243:3605|reinstall| : 22 => {
  3192. |12889|3243:3605|reinstall| : "destination" => "0.0.0.0/0",
  3193. |12889|3243:3605|reinstall| : "name" => "ssh",
  3194. |12889|3243:3605|reinstall| : "number" => 2,
  3195. |12889|3243:3605|reinstall| : "scope" => "0.0.0.0/0",
  3196. |12889|3243:3605|reinstall| : "target" => "ACCEPT"
  3197. |12889|3243:3605|reinstall| : }
  3198. |12889|3243:3605|reinstall| : }
  3199. |12889|3243:3605|reinstall| : },
  3200. |12889|3243:3605|reinstall| : 3 => {
  3201. |12889|3243:3605|reinstall| : "all" => {
  3202. |12889|3243:3605|reinstall| : "any" => {
  3203. |12889|3243:3605|reinstall| : "destination" => "0.0.0.0/0",
  3204. |12889|3243:3605|reinstall| : "name" => "any",
  3205. |12889|3243:3605|reinstall| : "number" => 3,
  3206. |12889|3243:3605|reinstall| : "scope" => "0.0.0.0/0",
  3207. |12889|3243:3605|reinstall| : "target" => "ACCEPT"
  3208. |12889|3243:3605|reinstall| : }
  3209. |12889|3243:3605|reinstall| : }
  3210. |12889|3243:3605|reinstall| : },
  3211. |12889|3243:3605|reinstall| : 4 => {
  3212. |12889|3243:3605|reinstall| : "icmp" => {
  3213. |12889|3243:3605|reinstall| : "any" => {
  3214. |12889|3243:3605|reinstall| : "destination" => "0.0.0.0/0",
  3215. |12889|3243:3605|reinstall| : "name" => "any",
  3216. |12889|3243:3605|reinstall| : "number" => 4,
  3217. |12889|3243:3605|reinstall| : "scope" => "0.0.0.0/0",
  3218. |12889|3243:3605|reinstall| : "target" => "ACCEPT"
  3219. |12889|3243:3605|reinstall| : }
  3220. |12889|3243:3605|reinstall| : }
  3221. |12889|3243:3605|reinstall| : },
  3222. |12889|3243:3605|reinstall| : 5 => {
  3223. |12889|3243:3605|reinstall| : "all" => {
  3224. |12889|3243:3605|reinstall| : "any" => {
  3225. |12889|3243:3605|reinstall| : "destination" => "0.0.0.0/0",
  3226. |12889|3243:3605|reinstall| : "name" => "any",
  3227. |12889|3243:3605|reinstall| : "number" => 5,
  3228. |12889|3243:3605|reinstall| : "scope" => "0.0.0.0/0",
  3229. |12889|3243:3605|reinstall| : "target" => "ACCEPT"
  3230. |12889|3243:3605|reinstall| : }
  3231. |12889|3243:3605|reinstall| : }
  3232. |12889|3243:3605|reinstall| : },
  3233. |12889|3243:3605|reinstall| : 6 => {
  3234. |12889|3243:3605|reinstall| : "all" => {
  3235. |12889|3243:3605|reinstall| : "any" => {
  3236. |12889|3243:3605|reinstall| : "destination" => "0.0.0.0/0",
  3237. |12889|3243:3605|reinstall| : "name" => "any",
  3238. |12889|3243:3605|reinstall| : "number" => 6,
  3239. |12889|3243:3605|reinstall| : "scope" => "0.0.0.0/0",
  3240. |12889|3243:3605|reinstall| : "target" => "REJECT"
  3241. |12889|3243:3605|reinstall| : }
  3242. |12889|3243:3605|reinstall| : }
  3243. |12889|3243:3605|reinstall| : }
  3244. |12889|3243:3605|reinstall| : }
  3245. |12889|3243:3605|reinstall| : }
  3246. 2013-05-28 13:56:43|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  3247. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'ls -d --color=never "/etc/sysconfig" 2>&1 || mkdir -p "/etc/sysconfig" 2>&1 && ls -d --color=never "/etc/sysconfig"' 2>&1
  3248. 2013-05-28 13:56:43|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3249. |12889|3243:3605|reinstall| /etc/sysconfig
  3250. |12889|3243:3605|reinstall| /etc/sysconfig
  3251. 2013-05-28 13:56:43|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x26, returning (0, "/etc/sysconfig /etc/sysconfig")
  3252. 2013-05-28 13:56:43|12889|3243:3605|reinstall|Linux.pm:create_directory(1573)|directory already exists on ecco-vm-x26: '/etc/sysconfig'
  3253. 2013-05-28 13:56:43|12889|3243:3605|reinstall|OS.pm:copy_file(2781)|attempting to copy file on ecco-vm-x26: '/etc/sysconfig/iptables' -> '/etc/sysconfig/iptables_pre_any'
  3254. 2013-05-28 13:56:43|12889|3243:3605|reinstall|OS.pm:copy_file(2792)|copied file on ecco-vm-x26: '/etc/sysconfig/iptables' --> '/etc/sysconfig/iptables_pre_any'
  3255. 2013-05-28 13:56:43|12889|3243:3605|reinstall|Linux.pm:enable_firewall_port(3703)|backed up original iptables file to: '/etc/sysconfig/iptables_pre_any'
  3256. 2013-05-28 13:56:43|12889|3243:3605|reinstall|Linux.pm:enable_firewall_port(3711)|attempting to execute command on ecco-vm-x26: '/sbin/iptables -I INPUT 1 -m state --state NEW,RELATED,ESTABLISHED -m tcp -p tcp -j ACCEPT -s 10.18.20.4'
  3257. 2013-05-28 13:56:43|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  3258. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 '/sbin/iptables -I INPUT 1 -m state --state NEW,RELATED,ESTABLISHED -m tcp -p tcp -j ACCEPT -s 10.18.20.4' 2>&1
  3259. 2013-05-28 13:56:43|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3260. 2013-05-28 13:56:43|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x26, returning (0, "")
  3261. 2013-05-28 13:56:43|12889|3243:3605|reinstall|Linux.pm:enable_firewall_port(3713)|executed command on ecco-vm-x26: '/sbin/iptables -I INPUT 1 -m state --state NEW,RELATED,ESTABLISHED -m tcp -p tcp -j ACCEPT -s 10.18.20.4'
  3262. 2013-05-28 13:56:43|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  3263. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 '/sbin/iptables-save > /etc/sysconfig/iptables' 2>&1
  3264. 2013-05-28 13:56:43|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3265. 2013-05-28 13:56:43|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x26, returning (0, "")
  3266. 2013-05-28 13:56:43|12889|3243:3605|reinstall|Linux.pm:enable_firewall_port(3723)|executed command /sbin/iptables-save > /etc/sysconfig/iptables on ecco-vm-x26
  3267. 2013-05-28 13:56:43|12889|3243:3605|reinstall|Linux.pm:post_load(334)|added MN_Priv_IP 10.18.20.4 to firewall on ecco-vm-x26
  3268. 2013-05-28 13:56:43|12889|3243:3605|reinstall|Linux.pm:activate_interfaces(2561)|attempting to find network interfaces with an active link
  3269. 2013-05-28 13:56:43|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  3270. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'ip link' 2>&1
  3271. 2013-05-28 13:56:44|12889|3243:3605|reinstall|OS.pm:execute(1994)|executed command: 'ip link', exit status: 0, output:
  3272. |12889|3243:3605|reinstall| 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
  3273. |12889|3243:3605|reinstall| link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
  3274. |12889|3243:3605|reinstall| 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
  3275. |12889|3243:3605|reinstall| link/ether 52:54:00:01:00:19 brd ff:ff:ff:ff:ff:ff
  3276. |12889|3243:3605|reinstall| 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
  3277. |12889|3243:3605|reinstall| link/ether 52:54:00:01:80:19 brd ff:ff:ff:ff:ff:ff
  3278. 2013-05-28 13:56:44|12889|3243:3605|reinstall|Linux.pm:activate_interfaces(2570)|found interface names:
  3279. |12889|3243:3605|reinstall| eth0
  3280. |12889|3243:3605|reinstall| eth1
  3281. 2013-05-28 13:56:44|12889|3243:3605|reinstall|OS.pm:find_files(2917)|attempting to find files on ecco-vm-x26, base directory path: '/etc/sysconfig/network-scripts/', pattern: ifcfg-eth*, command: /usr/bin/find "/etc/sysconfig/network-scripts/" -type f -iname "ifcfg-eth*"
  3282. 2013-05-28 13:56:44|12889|3243:3605|reinstall|OS.pm:find_files(2945)|files found: 2, base directory: '/etc/sysconfig/network-scripts/', pattern: 'ifcfg-eth*'
  3283. |12889|3243:3605|reinstall| command: '/usr/bin/find "/etc/sysconfig/network-scripts/" -type f -iname "ifcfg-eth*"'
  3284. 2013-05-28 13:56:44|12889|3243:3605|reinstall|Linux.pm:activate_interfaces(2575)|found existing ifcfg-eth* files:
  3285. |12889|3243:3605|reinstall| /etc/sysconfig/network-scripts/ifcfg-eth1
  3286. |12889|3243:3605|reinstall| /etc/sysconfig/network-scripts/ifcfg-eth0
  3287. 2013-05-28 13:56:44|12889|3243:3605|reinstall|Linux.pm:activate_interfaces(2583)|ifcfg file already exists for eth0
  3288. 2013-05-28 13:56:44|12889|3243:3605|reinstall|Linux.pm:activate_interfaces(2583)|ifcfg file already exists for eth1
  3289. 2013-05-28 13:56:44|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  3290. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'sed -i -e '/vcld_post_load.*/d' currentimage.txt && echo >> currentimage.txt && echo "vcld_post_load=success (Tue May 28 13:56:44 2013)" >> currentimage.txt && sed -i -e '/^[\s\r\n]*$/d' currentimage.txt' 2>&1
  3291. 2013-05-28 13:56:44|12889|3243:3605|reinstall|OS.pm:set_vcld_post_load_status(1056)|added line to currentimage.txt on ecco-vm-x26: 'vcld_post_load=success (Tue May 28 13:56:44 2013)'
  3292. 2013-05-28 13:56:44|12889|3243:3605|reinstall|utils.pm:insertloadlog(3665)|inserted computer=40, loadimagecomplete, performed OS post-load tasks \'ecco-vm-x26:linux-TestImage2-16-v2\' domain on ecco-vmhost-x7
  3293. 2013-05-28 13:56:44|12889|3243:3605|reinstall|new.pm:reload_image(691)|linux-TestImage2-16-v2 was successfully reloaded on ecco-vm-x26
  3294. 2013-05-28 13:56:44|12889|3243:3605|reinstall|utils.pm:insertloadlog(3665)|inserted computer=40, loadimagecomplete, linux-TestImage2-16-v2 was successfully reloaded on ecco-vm-x26
  3295. 2013-05-28 13:56:44|12889|3243:3605|reinstall|utils.pm:update_currentimage(5685)|updating computer 40: image=16, imagerevision=174
  3296. 2013-05-28 13:56:44|12889|3243:3605|reinstall|utils.pm:update_currentimage(5715)|updated currentimageid and imagerevision id for computer id 40
  3297. 2013-05-28 13:56:44|12889|3243:3605|reinstall|new.pm:reload_image(702)|updated computer table for ecco-vm-x26: currentimageid=16
  3298. 2013-05-28 13:56:44|12889|3243:3605|reinstall|new.pm:reload_image(722)|returning 1
  3299. 2013-05-28 13:56:44|12889|3243:3605|reinstall|new.pm:process(292)|ecco-vm-x26 is loaded with linux-TestImage2-16-v2
  3300. 2013-05-28 13:56:44|12889|3243:3605|reinstall|utils.pm:insertloadlog(3665)|inserted computer=40, nodeready, ecco-vm-x26 is loaded with linux-TestImage2-16-v2
  3301. 2013-05-28 13:56:44|12889|3243:3605|reinstall|new.pm:process(374)|request_state_name= reinstall
  3302. 2013-05-28 13:56:44|12889|3243:3605|reinstall|new.pm:process(386)|ecco-vm-x26 next image is already set to linux-TestImage2-16-v2
  3303. 2013-05-28 13:56:44|12889|3243:3605|reinstall|DataStructure.pm:is_parent_reservation(1008)|returning false: parent reservation ID for this request: 3604
  3304. 2013-05-28 13:56:44|12889|3243:3605|reinstall|new.pm:reserve_computer(955)|user_standalone=0, image OS type=linux
  3305. 2013-05-28 13:56:44|12889|3243:3605|reinstall|OS.pm:get_public_interface_name(1150)|returning public interface name previously retrieved: eth1
  3306. 2013-05-28 13:56:44|12889|3243:3605|reinstall|OS.pm:get_ip_address(1538)|returning public IP address: 199.241.160.149
  3307. 2013-05-28 13:56:44|12889|3243:3605|reinstall|new.pm:confirm_public_ip_address(1325)|retrieved public IP address from ecco-vm-x26 using the OS module: 199.241.160.149
  3308. 2013-05-28 13:56:44|12889|3243:3605|reinstall|utils.pm:insertloadlog(3665)|inserted computer=40, info, node ready adding user account
  3309. 2013-05-28 13:56:44|12889|3243:3605|reinstall|new.pm:reserve_computer(1003)|calling OS module's reserve() subroutine
  3310. 2013-05-28 13:56:44|12889|3243:3605|reinstall|Linux.pm:reserve(931)|Enterered reserve() in the Linux OS module
  3311. 2013-05-28 13:56:44|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  3312. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'groupadd vcl' 2>&1
  3313. 2013-05-28 13:56:44|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3314. |12889|3243:3605|reinstall| groupadd: group 'vcl' already exists
  3315. 2013-05-28 13:56:44|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vm-x26, command:
  3316. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 'groupadd vcl' 2>&1
  3317. |12889|3243:3605|reinstall| returning (9, "groupadd: group 'vcl' already ...")
  3318. 2013-05-28 13:56:44|12889|3243:3605|reinstall|Linux.pm:add_vcl_usergroup(1272)|successfully added the vcl user group
  3319. 2013-05-28 13:56:44|12889|3243:3605|reinstall|Linux.pm:create_user(2909)|username not provided, pulling from datastructure
  3320.  
  3321. |12889|3243:3605|reinstall| ---- WARNING ----
  3322. |12889|3243:3605|reinstall| 2013-05-28 13:56:44|12889|3243:3605|reinstall|DataStructure.pm:_automethod(825)|corresponding data has not been initialized for get_reservation_password: $self->request_data->{reservation}{3605}{pw}
  3323. |12889|3243:3605|reinstall| ( 0) DataStructure.pm, _automethod (line: 825)
  3324. |12889|3243:3605|reinstall| (-1) Autoload.pm, __ANON__ (line: 80)
  3325. |12889|3243:3605|reinstall| (-2) Linux.pm, create_user (line: 2911)
  3326. |12889|3243:3605|reinstall| (-3) Linux.pm, reserve (line: 943)
  3327. |12889|3243:3605|reinstall| (-4) new.pm, reserve_computer (line: 1004)
  3328. |12889|3243:3605|reinstall| (-5) new.pm, process (line: 408)
  3329.  
  3330. 2013-05-28 13:56:44|12889|3243:3605|reinstall|Linux.pm:create_user(2919)|user_uid not provided, pulling from datastructure
  3331. 2013-05-28 13:56:44|12889|3243:3605|reinstall|Linux.pm:create_user(2924)|user_standalone not provided, pulling from datastructure
  3332. 2013-05-28 13:56:44|12889|3243:3605|reinstall|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x26:
  3333. |12889|3243:3605|reinstall| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x26 '/usr/sbin/useradd -u 130667 -d /home/nsasherr -m nsasherr -g vcl; chown -R nsasherr /home/nsasherr' 2>&1
  3334. 2013-05-28 13:56:44|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3335. 2013-05-28 13:56:44|12889|3243:3605|reinstall|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x26, returning (0, "")
  3336. 2013-05-28 13:56:44|12889|3243:3605|reinstall|new.pm:reserve_computer(1004)|OS module successfully reserved resources for this reservation
  3337. 2013-05-28 13:56:44|12889|3243:3605|reinstall|new.pm:process(408)|ecco-vm-x26 with linux-TestImage2-16-v2 successfully reserved
  3338. 2013-05-28 13:56:44|12889|3243:3605|reinstall|utils.pm:insertloadlog(3665)|inserted computer=40, reserved, ecco-vm-x26 successfully reserved with linux-TestImage2-16-v2
  3339. 2013-05-28 13:56:44|12889|3243:3605|reinstall|new.pm:process(420)|inserted computerloadlog entry, load state=reserved
  3340. 2013-05-28 13:56:44|12889|3243:3605|reinstall|new.pm:process(469)|this is a child image, request state NOT changed to 'reserved'
  3341. 2013-05-28 13:56:44|12889|3243:3605|reinstall|utils.pm:insertloadlog(3665)|inserted computer=40, info, ecco-vm-x26 successfully set to with linux-TestImage2-16-v2
  3342. 2013-05-28 13:56:44|12889|3243:3605|reinstall|new.pm:process(489)|inserted computerloadlog entry: ecco-vm-x26 successfully set to with linux-TestImage2-16-v2
  3343. 2013-05-28 13:56:44|12889|3243:3605|reinstall|new.pm:process(496)|exiting
  3344. 2013-05-28 13:56:44|12889|3243:3605|reinstall|utils.pm:delete_computerloadlog_reservation(6396)|removing computerloadlog entries matching loadstate = begin
  3345. 2013-05-28 13:56:46|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:56:46
  3346. 2013-05-28 13:56:46|12889|3243:3605|reinstall|utils.pm:delete_computerloadlog_reservation(6447)|deleted rows from computerloadlog for reservation id=3605
  3347. 2013-05-28 13:56:46|12889|3243:3605|reinstall|State.pm:DESTROY(929)|VCL::new process duration: 75 seconds
  3348. 2013-05-28 13:56:46|31074|vcld:REAPER(723)|VCL process exited for reservation 3605, PID: 12889, signal: CHLD
  3349. 2013-05-28 13:56:47|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:56:46
  3350. 2013-05-28 13:56:52|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:56:52
  3351. 2013-05-28 13:56:53|17747|3243:3604|reserved|OS.pm:is_user_connected(2676)|checking for connection by nsasherr on ecco-vm-x10
  3352. 2013-05-28 13:56:53|17747|3243:3604|reserved|DataStructure.pm:get_reservation_remote_ip(1165)|retrieved remote IP for reservation 3604: 129.100.171.120
  3353. 2013-05-28 13:56:53|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  3354. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'netstat -an' 2>&1
  3355. 2013-05-28 13:56:54|17747|3243:3604|reserved|Linux.pm:check_connection_on_port(3350)|checking connections on node ecco-vm-x10 on port 22
  3356. 2013-05-28 13:56:54|17747|3243:3604|reserved|DataStructure.pm:get_reservation_remote_ip(1165)|retrieved remote IP for reservation 3604: 129.100.171.120
  3357. 2013-05-28 13:56:54|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  3358. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'netstat -an' 2>&1
  3359. 2013-05-28 13:56:54|17747|3243:3604|reserved|Linux.pm:check_connection_on_port(3350)|checking connections on node ecco-vm-x10 on port 5900
  3360. 2013-05-28 13:56:54|17747|3243:3604|reserved|OS.pm:is_user_connected(2723)|nsasherr has not connected to ecco-vm-x10 (16/884 seconds elapsed/remaining), sleeping for 15 seconds
  3361. 2013-05-28 13:56:57|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:56:57
  3362. 2013-05-28 13:57:02|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:57:02
  3363. 2013-05-28 13:57:07|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:57:07
  3364. 2013-05-28 13:57:07|31074|3240:3600|inuse|utils.pm:reservation_being_processed(8638)|computerloadlog 'begin' entry does NOT exist for reservation 3600
  3365. 2013-05-28 13:57:07|31074|3240:3600|inuse|utils.pm:run_command(8710)|executed command: pgrep -fl 'vcld [0-9]+:3600 ', pid: 22252, exit status: 1, output:
  3366. 2013-05-28 13:57:07|31074|3240:3600|inuse|utils.pm:is_management_node_process_running(8943)|process is NOT running, identifier: 'vcld [0-9]+:3600 '
  3367. 2013-05-28 13:57:07|31074|3240:3600|inuse|utils.pm:reservation_being_processed(8659)|reservation is NOT currently being processed
  3368. 2013-05-28 13:57:07|31074|3240:3600|inuse|vcld:main(284)|reservation 3600 is NOT already being processed
  3369. 2013-05-28 13:57:07|31074|3240:3600|inuse|utils.pm:get_management_node_info(5456)|retrieving current management node info for 'ecco-adm3.sharcnet.ca' from database, cached data is stale: 96 seconds old
  3370. 2013-05-28 13:57:07|31074|3240:3600|inuse|utils.pm:get_management_node_info(5603)|retrieved management node info: 'ecco-adm3.sharcnet.ca' (ecco-adm3)
  3371. 2013-05-28 13:57:07|31074|3240:3600|inuse|utils.pm:get_connect_method_info(10063)|attempting to retrieve connect method info:
  3372. |31074|3240:3600|inuse| imagerevision: 143 - linux-NumericalComputing-18-v2
  3373. |31074|3240:3600|inuse| OS: 36 - vmwarelinux
  3374. |31074|3240:3600|inuse| OS type: 2 - linux
  3375. 2013-05-28 13:57:07|31074|3240:3600|inuse|utils.pm:get_connect_method_info(10127)|ssh: connectmethodid=1, OStypeid=2, OSid=NULL, imagerevisionid=NULL, disabled=0
  3376. 2013-05-28 13:57:07|31074|3240:3600|inuse|utils.pm:get_connect_method_info(10127)|Xvnc: connectmethodid=4, OStypeid=NULL, OSid=NULL, imagerevisionid=143, disabled=0
  3377. 2013-05-28 13:57:07|31074|3240:3600|inuse|utils.pm:get_connect_method_info(10127)|freenx: connectmethodid=6, OStypeid=NULL, OSid=NULL, imagerevisionid=143, disabled=0
  3378. 2013-05-28 13:57:07|31074|3240:3600|inuse|utils.pm:get_user_info(7558)|non-standalone affiliation found for user zhijie:
  3379. |31074|3240:3600|inuse| user affiliation: sharcnet
  3380. |31074|3240:3600|inuse| not standalone list: sharcnet
  3381. 2013-05-28 13:57:07|31074|3240:3600|inuse|utils.pm:get_user_info(7558)|non-standalone affiliation found for user zhijie:
  3382. |31074|3240:3600|inuse| user affiliation: sharcnet
  3383. |31074|3240:3600|inuse| not standalone list: sharcnet
  3384. 2013-05-28 13:57:07|31074|3240:3600|inuse|utils.pm:get_request_info(4062)|request length > 24 hours, disabling user checks
  3385. 2013-05-28 13:57:07|31074|3240:3600|inuse|vcld:main(289)|retrieved request information from database
  3386. 2013-05-28 13:57:07|31074|3240:3600|inuse|DataStructure.pm:is_parent_reservation(1004)|returning true: parent reservation ID for this request: 3600
  3387. 2013-05-28 13:57:07|31074|3240:3600|inuse|utils.pm:update_request_state(1545)|request 3240 state updated to: pending, laststate to: inuse
  3388. 2013-05-28 13:57:07|31074|3240:3600|inuse|utils.pm:insertloadlog(3665)|inserted computer=38, begin, beginning to process, state is inuse
  3389. 2013-05-28 13:57:07|31074|3240:3600|inuse|vcld:make_new_child(520)|loaded VCL::inuse module
  3390. 2013-05-28 13:57:07|31074|3240:3600|inuse|vcld:make_new_child(544)|current number of forked kids: 2
  3391. 2013-05-28 13:57:07|22263|3240:3600|inuse|vcld:make_new_child(560)|vcld environment variable set to 0 for this process
  3392. 2013-05-28 13:57:07|22263|3240:3600|inuse|DataStructure.pm:_initialize(643)|image ID argument was specified: noimage, DataStructure object will contain image information for the production imagerevision of this image
  3393. 2013-05-28 13:57:07|22263|3240:3600|inuse|DataStructure.pm:_initialize(674)|retrieved data for imagerevision ID: 1
  3394. 2013-05-28 13:57:07|22263|3240:3600|inuse|DataStructure.pm:_initialize(690)|retrieved data for image ID: 1
  3395. 2013-05-28 13:57:07|22263|3240:3600|inuse|Module.pm:create_mn_os_object(376)|VCL::Module::OS::Linux::ManagementNode module loaded
  3396. 2013-05-28 13:57:07|22263|3240:3600|inuse|Module.pm:new(196)|VCL::Module::OS::Linux::ManagementNode object created for image noimage, address: 3c3fdf8
  3397. 2013-05-28 13:57:07|22263|3240:3600|inuse|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{0}{computer}{hostname}, data identifier: computer_hostname, data:
  3398. |22263|3240:3600|inuse| : "ecco-adm3.sharcnet.ca"
  3399. 2013-05-28 13:57:07|22263|3240:3600|inuse|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{0}{computer}{NODENAME}, data identifier: computer_node_name, data:
  3400. |22263|3240:3600|inuse| : "ecco-adm3"
  3401. 2013-05-28 13:57:07|22263|3240:3600|inuse|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{0}{computer}{SHORTNAME}, data identifier: computer_short_name, data:
  3402. |22263|3240:3600|inuse| : "ecco-adm3"
  3403. 2013-05-28 13:57:07|22263|3240:3600|inuse|DataStructure.pm:_automethod(847)|data structure updated, hash path: $self->request_data->{reservation}{0}{computer}{IPaddress}, data identifier: computer_ip_address, data:
  3404. |22263|3240:3600|inuse| : "10.18.20.4"
  3405. 2013-05-28 13:57:07|22263|3240:3600|inuse|Module.pm:create_mn_os_object(381)|VCL::Module::OS::Linux::ManagementNode OS object created, address: 3c3fdf8
  3406. 2013-05-28 13:57:07|22263|3240:3600|inuse|Module.pm:new(192)|VCL::inuse object created for state inuse, address: 3341dc0
  3407. 2013-05-28 13:57:07|22263|3240:3600|inuse|State.pm:initialize(88)|obtained a database handle for this state process, stored as $ENV{dbh}
  3408. 2013-05-28 13:57:07|22263|3240:3600|inuse|State.pm:check_image_os(771)|no corrections need to be made, not an imaging request, returning 1
  3409. 2013-05-28 13:57:07|22263|3240:3600|inuse|DataStructure.pm:is_parent_reservation(1004)|returning true: parent reservation ID for this request: 3600
  3410. 2013-05-28 13:57:07|22263|3240:3600|inuse|utils.pm:rename_vcld_process(6756)|reservation count: 1
  3411. 2013-05-28 13:57:07|22263|3240:3600|inuse|utils.pm:rename_vcld_process(6775)|PARENTIMAGE: 1
  3412. 2013-05-28 13:57:07|22263|3240:3600|inuse|utils.pm:rename_vcld_process(6776)|SUBIMAGE: 0
  3413. 2013-05-28 13:57:07|22263|3240:3600|inuse|utils.pm:rename_vcld_process(6794)|renamed process to 'vcld 3240:3600 inuse ecco-vm-x24>ecco-vmhost-x6 linux-NumericalComputing-18-v2 zhijie'
  3414. 2013-05-28 13:57:07|22263|3240:3600|inuse|DataStructure.pm:is_parent_reservation(1004)|returning true: parent reservation ID for this request: 3600
  3415. 2013-05-28 13:57:07|22263|3240:3600|inuse|DataStructure.pm:is_parent_reservation(1004)|returning true: parent reservation ID for this request: 3600
  3416. 2013-05-28 13:57:07|22263|3240:3600|inuse|Module.pm:create_os_object(308)|VCL::Module::OS::Linux module loaded
  3417. 2013-05-28 13:57:07|22263|3240:3600|inuse|Module.pm:new(196)|VCL::Module::OS::Linux object created for image linux-NumericalComputing-18-v2, address: 3c43310
  3418. 2013-05-28 13:57:07|22263|3240:3600|inuse|Module.pm:create_os_object(314)|VCL::Module::OS::Linux OS object created, address: 3c43310
  3419. 2013-05-28 13:57:07|22263|3240:3600|inuse|State.pm:initialize(125)|computer is a VM, attempting to create VM host OS object
  3420. 2013-05-28 13:57:07|22263|3240:3600|inuse|DataStructure.pm:_initialize(619)|computer ID argument was specified, retrieving data for computer ID: 23
  3421. 2013-05-28 13:57:07|22263|3240:3600|inuse|DataStructure.pm:_initialize(643)|image ID argument was specified: 1, DataStructure object will contain image information for the production imagerevision of this image
  3422. 2013-05-28 13:57:07|22263|3240:3600|inuse|DataStructure.pm:_initialize(674)|retrieved data for imagerevision ID: 1
  3423. 2013-05-28 13:57:07|22263|3240:3600|inuse|DataStructure.pm:_initialize(690)|retrieved data for image ID: 1
  3424. 2013-05-28 13:57:07|22263|3240:3600|inuse|Module.pm:create_vmhost_os_object(454)|attempting to load VM host OS module: VCL::Module::OS::Linux::UnixLab (image: 1)
  3425. 2013-05-28 13:57:07|22263|3240:3600|inuse|Module.pm:create_vmhost_os_object(460)|VM host OS module loaded: VCL::Module::OS::Linux::UnixLab
  3426. 2013-05-28 13:57:07|22263|3240:3600|inuse|Module.pm:new(196)|VCL::Module::OS::Linux::UnixLab object created for image noimage, address: 3c43580
  3427. 2013-05-28 13:57:07|22263|3240:3600|inuse|Module.pm:create_vmhost_os_object(465)|VCL::Module::OS::Linux::UnixLab OS object created, address: 3c43580
  3428. 2013-05-28 13:57:07|22263|3240:3600|inuse|Module.pm:create_provisioning_object(519)|VCL::Module::Provisioning::libvirt module loaded
  3429. 2013-05-28 13:57:07|22263|3240:3600|inuse|Module.pm:new(200)|VCL::Module::Provisioning::libvirt object created for computer ecco-vm-x24, address: 3c43928
  3430. 2013-05-28 13:57:07|22263|3240:3600|inuse|libvirt.pm:initialize(99)|libvirt driver module directory path: /usr/local/vcl/bin/../lib/VCL/Module/Provisioning/libvirt
  3431. 2013-05-28 13:57:07|22263|3240:3600|inuse|OS.pm:find_files(2917)|attempting to find files on ecco-adm3, base directory path: '/usr/local/vcl/bin/../lib/VCL/Module/Provisioning/libvirt/', pattern: *.pm, command: /usr/bin/find "/usr/local/vcl/bin/../lib/VCL/Module/Provisioning/libvirt/" -type f -iname "*.pm"
  3432. 2013-05-28 13:57:07|22263|3240:3600|inuse|OS.pm:find_files(2945)|files found: 1, base directory: '/usr/local/vcl/bin/../lib/VCL/Module/Provisioning/libvirt/', pattern: '*.pm'
  3433. |22263|3240:3600|inuse| command: '/usr/bin/find "/usr/local/vcl/bin/../lib/VCL/Module/Provisioning/libvirt/" -type f -iname "*.pm"'
  3434. 2013-05-28 13:57:07|22263|3240:3600|inuse|Module.pm:new(200)|VCL::Module::Provisioning::libvirt::KVM object created for computer ecco-vm-x24, address: 3e45538
  3435. 2013-05-28 13:57:07|22263|3240:3600|inuse|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x6:
  3436. |22263|3240:3600|inuse| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x6 'which virsh' 2>&1
  3437. 2013-05-28 13:57:07|22263|3240:3600|inuse|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3438. |22263|3240:3600|inuse| /usr/bin/virsh
  3439. 2013-05-28 13:57:07|22263|3240:3600|inuse|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x6, returning (0, "/usr/bin/virsh")
  3440. 2013-05-28 13:57:07|22263|3240:3600|inuse|KVM.pm:initialize(104)|verified 'virsh' command is available on ecco-vmhost-x6
  3441. 2013-05-28 13:57:07|22263|3240:3600|inuse|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x6:
  3442. |22263|3240:3600|inuse| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x6 'which qemu-img' 2>&1
  3443. 2013-05-28 13:57:07|22263|3240:3600|inuse|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3444. |22263|3240:3600|inuse| /usr/bin/qemu-img
  3445. 2013-05-28 13:57:07|22263|3240:3600|inuse|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x6, returning (0, "/usr/bin/qemu-img")
  3446. 2013-05-28 13:57:07|22263|3240:3600|inuse|KVM.pm:initialize(104)|verified 'qemu-img' command is available on ecco-vmhost-x6
  3447. 2013-05-28 13:57:07|22263|3240:3600|inuse|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vmhost-x6:
  3448. |22263|3240:3600|inuse| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vmhost-x6 'which virt-win-reg' 2>&1
  3449. 2013-05-28 13:57:07|22263|3240:3600|inuse|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3450. |22263|3240:3600|inuse| /usr/bin/virt-win-reg
  3451. 2013-05-28 13:57:07|22263|3240:3600|inuse|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vmhost-x6, returning (0, "/usr/bin/virt-win-reg")
  3452. 2013-05-28 13:57:07|22263|3240:3600|inuse|KVM.pm:initialize(104)|verified 'virt-win-reg' command is available on ecco-vmhost-x6
  3453. 2013-05-28 13:57:07|22263|3240:3600|inuse|KVM.pm:initialize(113)|KVM driver module successfully initialized to control ecco-vmhost-x6
  3454. 2013-05-28 13:57:07|22263|3240:3600|inuse|libvirt.pm:initialize(119)|libvirt KVM driver object created and initialized to control ecco-vmhost-x6
  3455. 2013-05-28 13:57:07|22263|3240:3600|inuse|libvirt.pm:initialize(139)|VCL::Module::Provisioning::libvirt provisioning module initialized
  3456. 2013-05-28 13:57:07|22263|3240:3600|inuse|Module.pm:create_provisioning_object(525)|VCL::Module::Provisioning::libvirt provisioner object created for ecco-vm-x24, address: 3c43928
  3457. 2013-05-28 13:57:07|22263|3240:3600|inuse|State.pm:initialize(154)|returning 1
  3458. 2013-05-28 13:57:07|22263|3240:3600|inuse|vcld:make_new_child(570)|VCL::inuse object created and initialized
  3459. 2013-05-28 13:57:07|22263|3240:3600|inuse|DataStructure.pm:get_reservation_remote_ip(1165)|retrieved remote IP for reservation 3600: 142.89.184.196
  3460. 2013-05-28 13:57:07|22263|3240:3600|inuse|DataStructure.pm:is_parent_reservation(1004)|returning true: parent reservation ID for this request: 3600
  3461. 2013-05-28 13:57:07|22263|3240:3600|inuse|inuse.pm:_check_imaging_request(1096)|checking if request is imaging or if forimaging flag has changed
  3462. 2013-05-28 13:57:07|22263|3240:3600|inuse|utils.pm:is_request_imaging(1974)|forimaging=0, currentstate=pending, laststate=inuse
  3463. 2013-05-28 13:57:07|22263|3240:3600|inuse|utils.pm:delete_computerloadlog_reservation(6413)|removing computerloadlog entries NOT matching loadstate = begin
  3464. 2013-05-28 13:57:09|17747|3243:3604|reserved|OS.pm:is_user_connected(2676)|checking for connection by nsasherr on ecco-vm-x10
  3465. 2013-05-28 13:57:09|17747|3243:3604|reserved|DataStructure.pm:get_reservation_remote_ip(1165)|retrieved remote IP for reservation 3604: 129.100.171.120
  3466. 2013-05-28 13:57:09|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  3467. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'netstat -an' 2>&1
  3468. 2013-05-28 13:57:09|17747|3243:3604|reserved|Linux.pm:check_connection_on_port(3350)|checking connections on node ecco-vm-x10 on port 22
  3469. 2013-05-28 13:57:09|17747|3243:3604|reserved|DataStructure.pm:get_reservation_remote_ip(1165)|retrieved remote IP for reservation 3604: 129.100.171.120
  3470. 2013-05-28 13:57:09|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  3471. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'netstat -an' 2>&1
  3472. 2013-05-28 13:57:09|17747|3243:3604|reserved|Linux.pm:check_connection_on_port(3350)|checking connections on node ecco-vm-x10 on port 5900
  3473. 2013-05-28 13:57:09|17747|3243:3604|reserved|OS.pm:is_user_connected(2723)|nsasherr has not connected to ecco-vm-x10 (31/869 seconds elapsed/remaining), sleeping for 15 seconds
  3474. 2013-05-28 13:57:09|22263|3240:3600|inuse|utils.pm:delete_computerloadlog_reservation(6447)|deleted rows from computerloadlog for reservation id=3600
  3475. 2013-05-28 13:57:09|22263|3240:3600|inuse|inuse.pm:process(186)|rows removed from computerloadlog table for reservation 3600
  3476. 2013-05-28 13:57:09|22263|3240:3600|inuse|inuse.pm:process(194)|updated lastcheck time for reservation 3600
  3477. 2013-05-28 13:57:09|22263|3240:3600|inuse|inuse.pm:process(204)|beginning to poll
  3478. 2013-05-28 13:57:10|22263|3240:3600|inuse|OS.pm:is_ssh_responding(678)|ecco-vm-x24 is responding to SSH, port 22: open, port 24: closed
  3479. 2013-05-28 13:57:10|22263|3240:3600|inuse|inuse.pm:process(211)|checking if firewall needs to be updated
  3480. 2013-05-28 13:57:10|22263|3240:3600|inuse|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x24:
  3481. |22263|3240:3600|inuse| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x24 'stat /bin/systemctl' 2>&1
  3482. 2013-05-28 13:57:10|22263|3240:3600|inuse|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3483. |22263|3240:3600|inuse| stat: cannot stat `/bin/systemctl': No such file or directory
  3484. 2013-05-28 13:57:10|22263|3240:3600|inuse|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vm-x24, command:
  3485. |22263|3240:3600|inuse| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x24 'stat /bin/systemctl' 2>&1
  3486. |22263|3240:3600|inuse| returning (1, "stat: cannot stat `/bin/system...")
  3487. 2013-05-28 13:57:10|22263|3240:3600|inuse|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x24:
  3488. |22263|3240:3600|inuse| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x24 'stat /sbin/chkconfig' 2>&1
  3489. 2013-05-28 13:57:10|22263|3240:3600|inuse|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3490. |22263|3240:3600|inuse| File: `/sbin/chkconfig'
  3491. |22263|3240:3600|inuse| Size: 31728 Blocks: 64 IO Block: 4096 regular file
  3492. |22263|3240:3600|inuse| Device: 802h/2050d Inode: 642 Links: 1
  3493. |22263|3240:3600|inuse| Access: (0755/-rwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
  3494. |22263|3240:3600|inuse| Access: 2013-05-28 10:46:24.649933040 -0400
  3495. |22263|3240:3600|inuse| Modify: 2012-06-22 06:09:02.000000000 -0400
  3496. |22263|3240:3600|inuse| Change: 2012-10-09 15:03:55.140697475 -0400
  3497. 2013-05-28 13:57:10|22263|3240:3600|inuse|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x24, returning (0, "File: `/sbin/chkconfig' Size: ...")
  3498. 2013-05-28 13:57:10|22263|3240:3600|inuse|Linux.pm:file_exists(1449)|'/sbin/chkconfig' exists on ecco-vm-x24, files: 1, directories: 0, links: 0
  3499. 2013-05-28 13:57:10|22263|3240:3600|inuse|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x24:
  3500. |22263|3240:3600|inuse| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x24 '/sbin/chkconfig --list iptables' 2>&1
  3501. 2013-05-28 13:57:10|22263|3240:3600|inuse|Linux.pm:service_exists(3187)|'iptables' service exists
  3502. 2013-05-28 13:57:10|22263|3240:3600|inuse|DataStructure.pm:get_reservation_remote_ip(1165)|retrieved remote IP for reservation 3600: 142.89.184.196
  3503. 2013-05-28 13:57:10|22263|3240:3600|inuse|utils.pm:get_connect_method_info(10063)|attempting to retrieve connect method info:
  3504. |22263|3240:3600|inuse| imagerevision: 143 - linux-NumericalComputing-18-v2
  3505. |22263|3240:3600|inuse| OS: 36 - vmwarelinux
  3506. |22263|3240:3600|inuse| OS type: 2 - linux
  3507. 2013-05-28 13:57:10|22263|3240:3600|inuse|utils.pm:get_connect_method_info(10127)|ssh: connectmethodid=1, OStypeid=2, OSid=NULL, imagerevisionid=NULL, disabled=0
  3508. 2013-05-28 13:57:10|22263|3240:3600|inuse|utils.pm:get_connect_method_info(10127)|Xvnc: connectmethodid=4, OStypeid=NULL, OSid=NULL, imagerevisionid=143, disabled=0
  3509. 2013-05-28 13:57:10|22263|3240:3600|inuse|utils.pm:get_connect_method_info(10127)|freenx: connectmethodid=6, OStypeid=NULL, OSid=NULL, imagerevisionid=143, disabled=0
  3510. 2013-05-28 13:57:10|22263|3240:3600|inuse|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x24:
  3511. |22263|3240:3600|inuse| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x24 'stat /bin/systemctl' 2>&1
  3512. 2013-05-28 13:57:10|22263|3240:3600|inuse|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3513. |22263|3240:3600|inuse| stat: cannot stat `/bin/systemctl': No such file or directory
  3514. 2013-05-28 13:57:10|22263|3240:3600|inuse|utils.pm:run_ssh_command(5030)|SSH command executed on ecco-vm-x24, command:
  3515. |22263|3240:3600|inuse| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x24 'stat /bin/systemctl' 2>&1
  3516. |22263|3240:3600|inuse| returning (1, "stat: cannot stat `/bin/system...")
  3517. 2013-05-28 13:57:10|22263|3240:3600|inuse|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x24:
  3518. |22263|3240:3600|inuse| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x24 'stat /sbin/chkconfig' 2>&1
  3519. 2013-05-28 13:57:10|22263|3240:3600|inuse|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3520. |22263|3240:3600|inuse| File: `/sbin/chkconfig'
  3521. |22263|3240:3600|inuse| Size: 31728 Blocks: 64 IO Block: 4096 regular file
  3522. |22263|3240:3600|inuse| Device: 802h/2050d Inode: 642 Links: 1
  3523. |22263|3240:3600|inuse| Access: (0755/-rwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
  3524. |22263|3240:3600|inuse| Access: 2013-05-28 10:46:24.649933040 -0400
  3525. |22263|3240:3600|inuse| Modify: 2012-06-22 06:09:02.000000000 -0400
  3526. |22263|3240:3600|inuse| Change: 2012-10-09 15:03:55.140697475 -0400
  3527. 2013-05-28 13:57:10|22263|3240:3600|inuse|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x24, returning (0, "File: `/sbin/chkconfig' Size: ...")
  3528. 2013-05-28 13:57:10|22263|3240:3600|inuse|Linux.pm:file_exists(1449)|'/sbin/chkconfig' exists on ecco-vm-x24, files: 1, directories: 0, links: 0
  3529. 2013-05-28 13:57:10|22263|3240:3600|inuse|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x24:
  3530. |22263|3240:3600|inuse| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x24 '/sbin/chkconfig --list iptables' 2>&1
  3531. 2013-05-28 13:57:10|22263|3240:3600|inuse|Linux.pm:service_exists(3187)|'iptables' service exists
  3532. 2013-05-28 13:57:10|22263|3240:3600|inuse|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x24:
  3533. |22263|3240:3600|inuse| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x24 'iptables -L --line-number -n' 2>&1
  3534. 2013-05-28 13:57:11|22263|3240:3600|inuse|utils.pm:run_ssh_command(5020)|run_ssh_command output:
  3535. |22263|3240:3600|inuse| Chain INPUT (policy ACCEPT)
  3536. |22263|3240:3600|inuse| num target prot opt source destination
  3537. |22263|3240:3600|inuse| 1 ACCEPT tcp -- 142.89.184.0/24 0.0.0.0/0 state NEW,RELATED,ESTABLISHED tcp dpt:22
  3538. |22263|3240:3600|inuse| 2 ACCEPT tcp -- 142.89.184.0/24 0.0.0.0/0 state NEW,RELATED,ESTABLISHED tcp dpt:5900
  3539. |22263|3240:3600|inuse| 3 ACCEPT tcp -- 10.18.20.4 0.0.0.0/0 state NEW,RELATED,ESTABLISHED tcp
  3540. |22263|3240:3600|inuse| 4 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
  3541. |22263|3240:3600|inuse| 5 ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0
  3542. |22263|3240:3600|inuse| 6 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
  3543. |22263|3240:3600|inuse| 7 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
  3544. |22263|3240:3600|inuse| Chain FORWARD (policy ACCEPT)
  3545. |22263|3240:3600|inuse| num target prot opt source destination
  3546. |22263|3240:3600|inuse| 1 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
  3547. |22263|3240:3600|inuse| Chain OUTPUT (policy ACCEPT)
  3548. |22263|3240:3600|inuse| num target prot opt source destination
  3549. 2013-05-28 13:57:11|22263|3240:3600|inuse|utils.pm:run_ssh_command(5034)|SSH command executed on ecco-vm-x24, returning (0, "Chain INPUT (policy ACCEPT) nu...")
  3550. 2013-05-28 13:57:11|22263|3240:3600|inuse|Linux.pm:get_firewall_configuration(3992)|output Chain = INPUT
  3551. 2013-05-28 13:57:11|22263|3240:3600|inuse|Linux.pm:get_firewall_configuration(4008)|output rule: 1, ACCEPT, tcp, 142.89.184.0/24, 0.0.0.0/0, 22
  3552. 2013-05-28 13:57:11|22263|3240:3600|inuse|Linux.pm:get_firewall_configuration(4008)|output rule: 2, ACCEPT, tcp, 142.89.184.0/24, 0.0.0.0/0, 5900
  3553. 2013-05-28 13:57:12|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:57:12
  3554. 2013-05-28 13:57:13|22263|3240:3600|inuse|Linux.pm:get_firewall_configuration(3992)|output Chain = FORWARD
  3555. 2013-05-28 13:57:13|22263|3240:3600|inuse|Linux.pm:get_firewall_configuration(3992)|output Chain = OUTPUT
  3556. 2013-05-28 13:57:13|22263|3240:3600|inuse|Linux.pm:get_firewall_configuration(4051)|retrieved firewall configuration from ecco-vm-x24:
  3557. |22263|3240:3600|inuse| : {
  3558. |22263|3240:3600|inuse| : "FORWARD" => {
  3559. |22263|3240:3600|inuse| : 1 => {
  3560. |22263|3240:3600|inuse| : "all" => {
  3561. |22263|3240:3600|inuse| : "any" => {
  3562. |22263|3240:3600|inuse| : "destination" => "0.0.0.0/0",
  3563. |22263|3240:3600|inuse| : "name" => "any",
  3564. |22263|3240:3600|inuse| : "number" => 1,
  3565. |22263|3240:3600|inuse| : "scope" => "0.0.0.0/0",
  3566. |22263|3240:3600|inuse| : "target" => "REJECT"
  3567. |22263|3240:3600|inuse| : }
  3568. |22263|3240:3600|inuse| : }
  3569. |22263|3240:3600|inuse| : }
  3570. |22263|3240:3600|inuse| : },
  3571. |22263|3240:3600|inuse| : "INPUT" => {
  3572. |22263|3240:3600|inuse| : 1 => {
  3573. |22263|3240:3600|inuse| : "tcp" => {
  3574. |22263|3240:3600|inuse| : 22 => {
  3575. |22263|3240:3600|inuse| : "destination" => "0.0.0.0/0",
  3576. |22263|3240:3600|inuse| : "name" => "ssh",
  3577. |22263|3240:3600|inuse| : "number" => 1,
  3578. |22263|3240:3600|inuse| : "scope" => "142.89.184.0/24",
  3579. |22263|3240:3600|inuse| : "target" => "ACCEPT"
  3580. |22263|3240:3600|inuse| : }
  3581. |22263|3240:3600|inuse| : }
  3582. |22263|3240:3600|inuse| : },
  3583. |22263|3240:3600|inuse| : 2 => {
  3584. |22263|3240:3600|inuse| : "tcp" => {
  3585. |22263|3240:3600|inuse| : 5900 => {
  3586. |22263|3240:3600|inuse| : "destination" => "0.0.0.0/0",
  3587. |22263|3240:3600|inuse| : "name" => 5900,
  3588. |22263|3240:3600|inuse| : "number" => 2,
  3589. |22263|3240:3600|inuse| : "scope" => "142.89.184.0/24",
  3590. |22263|3240:3600|inuse| : "target" => "ACCEPT"
  3591. |22263|3240:3600|inuse| : }
  3592. |22263|3240:3600|inuse| : }
  3593. |22263|3240:3600|inuse| : },
  3594. |22263|3240:3600|inuse| : 3 => {
  3595. |22263|3240:3600|inuse| : "tcp" => {
  3596. |22263|3240:3600|inuse| : "any" => {
  3597. |22263|3240:3600|inuse| : "destination" => "0.0.0.0/0",
  3598. |22263|3240:3600|inuse| : "name" => "any",
  3599. |22263|3240:3600|inuse| : "number" => 3,
  3600. |22263|3240:3600|inuse| : "scope" => "10.18.20.4",
  3601. |22263|3240:3600|inuse| : "target" => "ACCEPT"
  3602. |22263|3240:3600|inuse| : }
  3603. |22263|3240:3600|inuse| : }
  3604. |22263|3240:3600|inuse| : },
  3605. |22263|3240:3600|inuse| : 4 => {
  3606. |22263|3240:3600|inuse| : "all" => {
  3607. |22263|3240:3600|inuse| : "any" => {
  3608. |22263|3240:3600|inuse| : "destination" => "0.0.0.0/0",
  3609. |22263|3240:3600|inuse| : "name" => "any",
  3610. |22263|3240:3600|inuse| : "number" => 4,
  3611. |22263|3240:3600|inuse| : "scope" => "0.0.0.0/0",
  3612. |22263|3240:3600|inuse| : "target" => "ACCEPT"
  3613. |22263|3240:3600|inuse| : }
  3614. |22263|3240:3600|inuse| : }
  3615. |22263|3240:3600|inuse| : },
  3616. |22263|3240:3600|inuse| : 5 => {
  3617. |22263|3240:3600|inuse| : "icmp" => {
  3618. |22263|3240:3600|inuse| : "any" => {
  3619. |22263|3240:3600|inuse| : "destination" => "0.0.0.0/0",
  3620. |22263|3240:3600|inuse| : "name" => "any",
  3621. |22263|3240:3600|inuse| : "number" => 5,
  3622. |22263|3240:3600|inuse| : "scope" => "0.0.0.0/0",
  3623. |22263|3240:3600|inuse| : "target" => "ACCEPT"
  3624. |22263|3240:3600|inuse| : }
  3625. |22263|3240:3600|inuse| : }
  3626. |22263|3240:3600|inuse| : },
  3627. |22263|3240:3600|inuse| : 6 => {
  3628. |22263|3240:3600|inuse| : "all" => {
  3629. |22263|3240:3600|inuse| : "any" => {
  3630. |22263|3240:3600|inuse| : "destination" => "0.0.0.0/0",
  3631. |22263|3240:3600|inuse| : "name" => "any",
  3632. |22263|3240:3600|inuse| : "number" => 6,
  3633. |22263|3240:3600|inuse| : "scope" => "0.0.0.0/0",
  3634. |22263|3240:3600|inuse| : "target" => "ACCEPT"
  3635. |22263|3240:3600|inuse| : }
  3636. |22263|3240:3600|inuse| : }
  3637. |22263|3240:3600|inuse| : },
  3638. |22263|3240:3600|inuse| : 7 => {
  3639. |22263|3240:3600|inuse| : "all" => {
  3640. |22263|3240:3600|inuse| : "any" => {
  3641. |22263|3240:3600|inuse| : "destination" => "0.0.0.0/0",
  3642. |22263|3240:3600|inuse| : "name" => "any",
  3643. |22263|3240:3600|inuse| : "number" => 7,
  3644. |22263|3240:3600|inuse| : "scope" => "0.0.0.0/0",
  3645. |22263|3240:3600|inuse| : "target" => "REJECT"
  3646. |22263|3240:3600|inuse| : }
  3647. |22263|3240:3600|inuse| : }
  3648. |22263|3240:3600|inuse| : }
  3649. |22263|3240:3600|inuse| : }
  3650. |22263|3240:3600|inuse| : }
  3651. 2013-05-28 13:57:13|22263|3240:3600|inuse|Linux.pm:parse_firewall_scope(4182)|parsed firewall scope:
  3652. |22263|3240:3600|inuse| argument: '142.89.184.0/24'
  3653. |22263|3240:3600|inuse| result: '142.89.184.0/255.255.255.0'
  3654. |22263|3240:3600|inuse| IP address ranges:
  3655. |22263|3240:3600|inuse| 142.89.184.0-142.89.184.255
  3656. 2013-05-28 13:57:13|22263|3240:3600|inuse|Linux.pm:parse_firewall_scope(4182)|parsed firewall scope:
  3657. |22263|3240:3600|inuse| argument: '142.89.184.196,142.89.184.0/24'
  3658. |22263|3240:3600|inuse| result: '142.89.184.0/255.255.255.0'
  3659. |22263|3240:3600|inuse| IP address ranges:
  3660. |22263|3240:3600|inuse| 142.89.184.0-142.89.184.255
  3661. 2013-05-28 13:57:13|22263|3240:3600|inuse|Linux.pm:firewall_compare_update(4270)|firewall is already open on ecco-vm-x24, existing scope matches scope argument:
  3662. |22263|3240:3600|inuse| name: 'ssh'
  3663. |22263|3240:3600|inuse| protocol: tcp
  3664. |22263|3240:3600|inuse| port/type: 22
  3665. |22263|3240:3600|inuse| scope: 142.89.184.0/255.255.255.0
  3666. 2013-05-28 13:57:13|22263|3240:3600|inuse|inuse.pm:process(212)|confirmed firewall scope has been updated
  3667. 2013-05-28 13:57:13|22263|3240:3600|inuse|inuse.pm:process(228)|request checkuser flag not set, skipping user connection check
  3668. 2013-05-28 13:57:13|22263|3240:3600|inuse|inuse.pm:process(231)|imagemeta checkuser flag not set, skipping user connection check
  3669. 2013-05-28 13:57:13|22263|3240:3600|inuse|inuse.pm:process(253)|request duration greater than 24 hours (346500 seconds), checking if endtime notice should be sent to user
  3670. 2013-05-28 13:57:13|22263|3240:3600|inuse|utils.pm:check_endtimenotice_interval(898)|endtime= 2013-06-01 13:15:00 epoch_until_end= 343067
  3671. 2013-05-28 13:57:13|22263|3240:3600|inuse|utils.pm:check_endtimenotice_interval(915)|End Time is in: 0 week(s) 3 day(s) 23 hour(s) 17 min(s) and 47 sec(s)
  3672. 2013-05-28 13:57:13|22263|3240:3600|inuse|inuse.pm:_check_imaging_request(1096)|checking if request is imaging or if forimaging flag has changed
  3673. 2013-05-28 13:57:13|22263|3240:3600|inuse|utils.pm:is_request_imaging(1974)|forimaging=0, currentstate=pending, laststate=inuse
  3674. 2013-05-28 13:57:13|22263|3240:3600|inuse|inuse.pm:process(284)|updated reservation lastcheck time to now
  3675. 2013-05-28 13:57:13|22263|3240:3600|inuse|utils.pm:update_request_state(1545)|request 3240 state updated to: inuse, laststate to: inuse
  3676. 2013-05-28 13:57:13|22263|3240:3600|inuse|inuse.pm:process(291)|request state set back to inuse
  3677. 2013-05-28 13:57:13|22263|3240:3600|inuse|inuse.pm:process(303)|exiting
  3678. 2013-05-28 13:57:13|22263|3240:3600|inuse|utils.pm:delete_computerloadlog_reservation(6396)|removing computerloadlog entries matching loadstate = begin
  3679. 2013-05-28 13:57:15|22263|3240:3600|inuse|utils.pm:delete_computerloadlog_reservation(6447)|deleted rows from computerloadlog for reservation id=3600
  3680. 2013-05-28 13:57:15|22263|3240:3600|inuse|State.pm:DESTROY(929)|VCL::inuse process duration: 8 seconds
  3681. 2013-05-28 13:57:15|31074|vcld:REAPER(723)|VCL process exited for reservation 3600, PID: 22263, signal: CHLD
  3682. 2013-05-28 13:57:15|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:57:15
  3683. 2013-05-28 13:57:21|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:57:20
  3684. 2013-05-28 13:57:24|17747|3243:3604|reserved|OS.pm:is_user_connected(2676)|checking for connection by nsasherr on ecco-vm-x10
  3685. 2013-05-28 13:57:24|17747|3243:3604|reserved|DataStructure.pm:get_reservation_remote_ip(1165)|retrieved remote IP for reservation 3604: 129.100.171.120
  3686. 2013-05-28 13:57:24|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  3687. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'netstat -an' 2>&1
  3688. 2013-05-28 13:57:24|17747|3243:3604|reserved|Linux.pm:check_connection_on_port(3350)|checking connections on node ecco-vm-x10 on port 22
  3689. 2013-05-28 13:57:24|17747|3243:3604|reserved|DataStructure.pm:get_reservation_remote_ip(1165)|retrieved remote IP for reservation 3604: 129.100.171.120
  3690. 2013-05-28 13:57:24|17747|3243:3604|reserved|utils.pm:run_ssh_command(4902)|executing SSH command on ecco-vm-x10:
  3691. |17747|3243:3604|reserved| /usr/bin/ssh -i /etc/vcl/vcl.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectionAttempts=1 -o ConnectTimeout=3 -l root -p 22 -x ecco-vm-x10 'netstat -an' 2>&1
  3692. 2013-05-28 13:57:24|17747|3243:3604|reserved|Linux.pm:check_connection_on_port(3350)|checking connections on node ecco-vm-x10 on port 5900
  3693. 2013-05-28 13:57:24|17747|3243:3604|reserved|OS.pm:is_user_connected(2723)|nsasherr has not connected to ecco-vm-x10 (46/854 seconds elapsed/remaining), sleeping for 15 seconds
  3694. 2013-05-28 13:57:26|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:57:26
  3695. 2013-05-28 13:57:31|31074|vcld:main(167)|lastcheckin time updated for management node 1: 2013-05-28 13:57:31
  3696. ^C
  3697. [root@ecco-adm3 ~]#
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!