API tools faq
paste
Advertisement
Guest User

acs-bootup-log

a guest
Feb 16th, 2016
103
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 22.30 KB | None | 0 0
raw download clone embed print report
  1. 2016-02-16 15:54:07,228 INFO [o.a.c.s.l.CloudStackExtendedLifeCycle] (main:null) Done Configuring CloudStack Components
  2. 2016-02-16 15:54:07,246 INFO [c.c.u.LogUtils] (main:null) log4j configuration found at /etc/cloudstack/management/log4j-cloud.xml
  3. 2016-02-16 15:54:07,795 DEBUG [c.c.u.d.ConnectionConcierge] (Cluster-Heartbeat-1:ctx-46856fac) Registering a database connection for ClusterManagerHeartbeat2
  4. 2016-02-16 15:54:07,837 INFO [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-46856fac) We are good, no orphan management server msid in host table is found
  5. 2016-02-16 15:54:07,837 INFO [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-46856fac) Found 1 inactive management server node based on timestamp
  6. 2016-02-16 15:54:07,837 INFO [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-46856fac) management server node msid: 57177340185274, name: acs-management-1, service ip: 10.97.13.1, version: 4.7.1
  7. 2016-02-16 15:54:07,837 INFO [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-46856fac) Trying to connect to 10.97.13.1
  8. 2016-02-16 15:54:07,860 DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Heartbeat-1:ctx-46856fac) Detected management node joined, id:2, nodeIP:10.97.13.1
  9. 2016-02-16 15:54:09,295 DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Notification-1:ctx-e86a7e1e) Notify management server node join to listeners.
  10. 2016-02-16 15:54:09,296 DEBUG [c.c.c.ClusterManagerImpl] (Cluster-Notification-1:ctx-e86a7e1e) Joining node, IP: 10.97.13.1, msid: 57177340185273
  11. 2016-02-16 15:54:09,296 DEBUG [c.c.a.ClusterAlertAdapter] (Cluster-Notification-1:ctx-e86a7e1e) Receive cluster alert, EventArgs: com.cloud.cluster.ClusterNodeJoinEventArgs
  12. 2016-02-16 15:54:09,296 DEBUG [c.c.a.ClusterAlertAdapter] (Cluster-Notification-1:ctx-e86a7e1e) Handle cluster node join alert, joined node: 10.97.13.1, msidL: 57177340185273
  13. 2016-02-16 15:54:09,296 DEBUG [c.c.a.ClusterAlertAdapter] (Cluster-Notification-1:ctx-e86a7e1e) Management server node 10.97.13.1 is up, send alert
  14. 2016-02-16 15:54:09,299 WARN [o.a.c.alerts] (Cluster-Notification-1:ctx-e86a7e1e) alertType:: 14 // dataCenterId:: 0 // podId:: 0 // clusterId:: null // message:: Management server node 10.97.13.1 is up
  15. 2016-02-16 15:54:09,409 WARN [c.c.c.ClusterManagerImpl] (Cluster-Notification-1:ctx-e86a7e1e) Notifying management server join event took 114 ms
  16. 2016-02-16 15:54:11,303 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-1c185bf4) Resetting hosts suitable for reconnect
  17. 2016-02-16 15:54:11,305 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-1c185bf4) Completed resetting hosts suitable for reconnect
  18. 2016-02-16 15:54:11,305 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-1c185bf4) Acquiring hosts for clusters already owned by this management server
  19. 2016-02-16 15:54:11,306 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-1c185bf4) Completed acquiring hosts for clusters already owned by this management server
  20. 2016-02-16 15:54:11,306 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-1c185bf4) Acquiring hosts for clusters not owned by any management server
  21. 2016-02-16 15:54:11,307 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager Timer:ctx-1c185bf4) Completed acquiring hosts for clusters not owned by any management server
  22. 2016-02-16 15:54:15,992 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-854a1acf) Begin cleanup expired async-jobs
  23. 2016-02-16 15:54:16,012 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-854a1acf) Expunging completed job-10714
  24. 2016-02-16 15:54:16,047 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-854a1acf) Expunging completed job-10715
  25. 2016-02-16 15:54:16,093 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-854a1acf) Expunging completed job-10717
  26. 2016-02-16 15:54:16,196 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-854a1acf) Expunging completed job-10719
  27. 2016-02-16 15:54:16,242 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-854a1acf) Expunging completed job-10722
  28. 2016-02-16 15:54:16,283 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-854a1acf) Expunging completed job-10724
  29. 2016-02-16 15:54:16,317 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-854a1acf) Expunging completed job-10728
  30. 2016-02-16 15:54:16,351 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-854a1acf) Expunging completed job-10731
  31. 2016-02-16 15:54:16,386 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-854a1acf) Expunging completed job-10735
  32. 2016-02-16 15:54:16,420 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-854a1acf) Expunging completed job-10738
  33. 2016-02-16 15:54:16,454 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-854a1acf) Expunging completed job-10742
  34. 2016-02-16 15:54:16,489 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-854a1acf) Expunging completed job-10744
  35. 2016-02-16 15:54:16,523 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-854a1acf) Expunging completed job-10746
  36. 2016-02-16 15:54:16,558 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-854a1acf) Expunging completed job-10748
  37. 2016-02-16 15:54:16,598 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-854a1acf) Expunging completed job-10753
  38. 2016-02-16 15:54:16,632 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-854a1acf) Expunging completed job-10757
  39. 2016-02-16 15:54:16,673 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-854a1acf) End cleanup expired async-jobs
  40. 2016-02-16 15:54:16,804 DEBUG [c.c.n.l.LBHealthCheckManagerImpl] (LBHealthCheck-1:ctx-faa85440) LB HealthCheck Manager is running and getting the updates from LB providers and updating service status
  41. 2016-02-16 15:54:16,917 DEBUG [c.c.n.l.LBHealthCheckManagerImpl] (LBHealthCheck-1:ctx-faa85440) LB HealthCheck Manager is running and getting the updates from LB providers and updating service status
  42. 2016-02-16 15:54:21,622 DEBUG [c.c.s.StatsCollector] (StatsCollector-1:ctx-43c26ea5) HostStatsCollector is running...
  43. 2016-02-16 15:54:21,623 DEBUG [c.c.s.StatsCollector] (StatsCollector-2:ctx-3943d1b4) VmStatsCollector is running...
  44. 2016-02-16 15:54:21,625 DEBUG [c.c.s.StatsCollector] (StatsCollector-3:ctx-44947594) StorageCollector is running...
  45. 2016-02-16 15:54:21,627 DEBUG [c.c.s.StatsCollector] (StatsCollector-4:ctx-f97cf2c4) AutoScaling Monitor is running...
  46. 2016-02-16 15:54:21,641 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (StatsCollector-1:ctx-43c26ea5) Host 1 has switched to another management server, need to update agent map with a forwarding agent attache
  47. 2016-02-16 15:54:21,642 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (StatsCollector-1:ctx-43c26ea5) create forwarding ClusteredAgentAttache for 1
  48. 2016-02-16 15:54:21,654 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru] (StatsCollector-3:ctx-44947594) getCommandHostDelegation: class com.cloud.agent.api.GetStorageStatsCommand
  49. 2016-02-16 15:54:21,654 DEBUG [c.c.h.XenServerGuru] (StatsCollector-3:ctx-44947594) getCommandHostDelegation: class com.cloud.agent.api.GetStorageStatsCommand
  50. 2016-02-16 15:54:21,658 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (StatsCollector-3:ctx-44947594) Host 6 has switched to another management server, need to update agent map with a forwarding agent attache
  51. 2016-02-16 15:54:21,658 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (StatsCollector-3:ctx-44947594) create forwarding ClusteredAgentAttache for 6
  52. 2016-02-16 15:54:21,863 INFO [c.c.a.m.ClusteredAgentManagerImpl] (StatsCollector-1:ctx-43c26ea5) SSL: Handshake done
  53. 2016-02-16 15:54:21,863 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (StatsCollector-1:ctx-43c26ea5) Connection to peer opened: 57177340185274, ip: 10.97.13.1
  54. 2016-02-16 15:54:21,869 DEBUG [c.c.a.m.ClusteredAgentAttache] (StatsCollector-3:ctx-44947594) Seq 6-9007480729717702657: Forwarding null to 57177340185274
  55. 2016-02-16 15:54:21,869 DEBUG [c.c.a.m.ClusteredAgentAttache] (StatsCollector-1:ctx-43c26ea5) Seq 1-4458000681143369729: Forwarding null to 57177340185274
  56. 2016-02-16 15:54:21,870 DEBUG [c.c.a.m.ClusteredAgentAttache] (StatsCollector-2:ctx-3943d1b4) Seq 1-4458000681143369730: Forwarding null to 57177340185274
  57. 2016-02-16 15:54:21,919 DEBUG [c.c.a.m.ClusteredAgentAttache] (AgentManager-Handler-2:null) Seq 6-9007480729717702657: Routing from 57177340185273
  58. 2016-02-16 15:54:21,919 DEBUG [c.c.a.m.ClusteredAgentAttache] (AgentManager-Handler-3:null) Seq 1-4458000681143369729: Routing from 57177340185273
  59. 2016-02-16 15:54:21,919 DEBUG [c.c.a.m.ClusteredAgentAttache] (AgentManager-Handler-2:null) Seq 6-9007480729717702657: Link is closed
  60. 2016-02-16 15:54:21,919 DEBUG [c.c.a.m.ClusteredAgentAttache] (AgentManager-Handler-3:null) Seq 1-4458000681143369729: Link is closed
  61. 2016-02-16 15:54:21,919 DEBUG [c.c.a.m.ClusteredAgentAttache] (AgentManager-Handler-4:null) Seq 1-4458000681143369730: Routing from 57177340185273
  62. 2016-02-16 15:54:21,920 DEBUG [c.c.a.m.ClusteredAgentAttache] (AgentManager-Handler-4:null) Seq 1-4458000681143369730: Link is closed
  63. 2016-02-16 15:54:21,921 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-2:null) Seq 6-9007480729717702657: MgmtId 57177340185273: Req: Resource [Host:6] is unreachable: Host 6: Link is closed
  64. 2016-02-16 15:54:21,921 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-4:null) Seq 1-4458000681143369730: MgmtId 57177340185273: Req: Resource [Host:1] is unreachable: Host 1: Link is closed
  65. 2016-02-16 15:54:21,921 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-3:null) Seq 1-4458000681143369729: MgmtId 57177340185273: Req: Resource [Host:1] is unreachable: Host 1: Link is closed
  66. 2016-02-16 15:54:22,026 INFO [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-2:null) SSL: Handshake done
  67. 2016-02-16 15:54:22,027 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-2:null) Connection to peer opened: 57177340185273, ip: 10.97.13.1
  68. 2016-02-16 15:54:22,027 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-2:null) Seq 6--1: MgmtId 57177340185273: Req: Routing to peer
  69. 2016-02-16 15:54:22,027 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-3:null) Seq 1--1: MgmtId 57177340185273: Req: Routing to peer
  70. 2016-02-16 15:54:22,027 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-4:null) Seq 1--1: MgmtId 57177340185273: Req: Routing to peer
  71. 2016-02-16 15:54:22,075 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-7:null) Seq 1--1: MgmtId 57177340185273: Req: Cancel request received
  72. 2016-02-16 15:54:22,075 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-6:null) Seq 6--1: MgmtId 57177340185273: Req: Cancel request received
  73. 2016-02-16 15:54:22,075 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-8:null) Seq 1--1: MgmtId 57177340185273: Req: Cancel request received
  74. 2016-02-16 15:54:22,076 DEBUG [c.c.a.m.AgentAttache] (AgentManager-Handler-6:null) Seq 6-9007480729717702657: Cancelling.
  75. 2016-02-16 15:54:22,076 DEBUG [c.c.a.m.AgentAttache] (AgentManager-Handler-7:null) Seq 1-4458000681143369730: Cancelling.
  76. 2016-02-16 15:54:22,076 DEBUG [c.c.a.m.AgentAttache] (StatsCollector-3:ctx-44947594) Seq 6-9007480729717702657: Waiting some more time because this is the current command
  77. 2016-02-16 15:54:22,076 DEBUG [c.c.a.m.AgentAttache] (StatsCollector-2:ctx-3943d1b4) Seq 1-4458000681143369730: Waiting some more time because this is the current command
  78. 2016-02-16 15:54:22,076 DEBUG [c.c.a.m.AgentAttache] (AgentManager-Handler-8:null) Seq 1-4458000681143369729: Cancelling.
  79. 2016-02-16 15:54:22,076 DEBUG [c.c.a.m.AgentAttache] (StatsCollector-3:ctx-44947594) Seq 6-9007480729717702657: Waiting some more time because this is the current command
  80. 2016-02-16 15:54:22,076 DEBUG [c.c.a.m.AgentAttache] (StatsCollector-1:ctx-43c26ea5) Seq 1-4458000681143369729: Waiting some more time because this is the current command
  81. 2016-02-16 15:54:22,076 DEBUG [c.c.a.m.AgentAttache] (StatsCollector-2:ctx-3943d1b4) Seq 1-4458000681143369730: Waiting some more time because this is the current command
  82. 2016-02-16 15:54:22,076 DEBUG [c.c.a.m.AgentAttache] (StatsCollector-1:ctx-43c26ea5) Seq 1-4458000681143369729: Waiting some more time because this is the current command
  83. 2016-02-16 15:54:22,076 INFO [c.c.u.e.CSExceptionErrorCode] (StatsCollector-1:ctx-43c26ea5) Could not find exception: com.cloud.exception.OperationTimedoutException in error code list for exceptions
  84. 2016-02-16 15:54:22,076 INFO [c.c.u.e.CSExceptionErrorCode] (StatsCollector-3:ctx-44947594) Could not find exception: com.cloud.exception.OperationTimedoutException in error code list for exceptions
  85. 2016-02-16 15:54:22,076 INFO [c.c.u.e.CSExceptionErrorCode] (StatsCollector-2:ctx-3943d1b4) Could not find exception: com.cloud.exception.OperationTimedoutException in error code list for exceptions
  86. 2016-02-16 15:54:22,077 WARN [c.c.a.m.AgentAttache] (StatsCollector-1:ctx-43c26ea5) Seq 1-4458000681143369729: Timed out on null
  87. 2016-02-16 15:54:22,077 WARN [c.c.a.m.AgentAttache] (StatsCollector-2:ctx-3943d1b4) Seq 1-4458000681143369730: Timed out on null
  88. 2016-02-16 15:54:22,077 WARN [c.c.a.m.AgentAttache] (StatsCollector-3:ctx-44947594) Seq 6-9007480729717702657: Timed out on null
  89. 2016-02-16 15:54:22,077 DEBUG [c.c.a.m.AgentAttache] (StatsCollector-1:ctx-43c26ea5) Seq 1-4458000681143369729: Cancelling.
  90. 2016-02-16 15:54:22,077 DEBUG [c.c.a.m.AgentAttache] (StatsCollector-3:ctx-44947594) Seq 6-9007480729717702657: Cancelling.
  91. 2016-02-16 15:54:22,077 DEBUG [c.c.a.m.AgentAttache] (StatsCollector-2:ctx-3943d1b4) Seq 1-4458000681143369730: Cancelling.
  92. 2016-02-16 15:54:22,077 WARN [c.c.a.m.AgentManagerImpl] (StatsCollector-1:ctx-43c26ea5) Operation timed out: Commands 4458000681143369729 to Host 1 timed out after 3600
  93. 2016-02-16 15:54:22,077 WARN [c.c.a.m.AgentManagerImpl] (StatsCollector-2:ctx-3943d1b4) Operation timed out: Commands 4458000681143369730 to Host 1 timed out after 3600
  94. 2016-02-16 15:54:22,077 DEBUG [o.a.c.s.RemoteHostEndPoint] (StatsCollector-3:ctx-44947594) Failed to send command, due to Agent:6, com.cloud.exception.OperationTimedoutException: Commands 9007480729717702657 to Host 6 timed out after 3600
  95. 2016-02-16 15:54:22,078 WARN [c.c.v.UserVmManagerImpl] (StatsCollector-2:ctx-3943d1b4) Unable to obtain VM statistics.
  96. 2016-02-16 15:54:22,078 WARN [c.c.r.ResourceManagerImpl] (StatsCollector-1:ctx-43c26ea5) Unable to obtain host 1 statistics.
  97. 2016-02-16 15:54:22,078 WARN [c.c.s.StatsCollector] (StatsCollector-1:ctx-43c26ea5) Received invalid host stats for host: 1
  98. 2016-02-16 15:54:22,079 ERROR [c.c.s.StatsCollector] (StatsCollector-3:ctx-44947594) Error trying to retrieve storage stats
  99. com.cloud.utils.exception.CloudRuntimeException: Failed to send command, due to Agent:6, com.cloud.exception.OperationTimedoutException: Commands 9007480729717702657 to Host 6 timed out after 3600
  100. at org.apache.cloudstack.storage.RemoteHostEndPoint.sendMessage(RemoteHostEndPoint.java:133)
  101. at com.cloud.server.StatsCollector$StorageCollector.runInContext(StatsCollector.java:725)
  102. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  103. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  104. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  105. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  106. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  107. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  108. at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
  109. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
  110. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
  111. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  112. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  113. at java.lang.Thread.run(Thread.java:745)
  114. 2016-02-16 15:54:22,085 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (StatsCollector-1:ctx-43c26ea5) Host 3 has switched to another management server, need to update agent map with a forwarding agent attache
  115. 2016-02-16 15:54:22,086 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (StatsCollector-1:ctx-43c26ea5) create forwarding ClusteredAgentAttache for 3
  116. 2016-02-16 15:54:22,088 DEBUG [c.c.a.m.ClusteredAgentAttache] (StatsCollector-1:ctx-43c26ea5) Seq 3-2144839322535198721: Forwarding null to 57177340185274
  117. 2016-02-16 15:54:22,098 DEBUG [c.c.a.m.ClusteredAgentAttache] (AgentManager-Handler-9:null) Seq 3-2144839322535198721: Routing from 57177340185273
  118. 2016-02-16 15:54:22,099 DEBUG [c.c.a.m.ClusteredAgentAttache] (AgentManager-Handler-9:null) Seq 3-2144839322535198721: Link is closed
  119. 2016-02-16 15:54:22,099 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-9:null) Seq 3-2144839322535198721: MgmtId 57177340185273: Req: Resource [Host:3] is unreachable: Host 3: Link is closed
  120. 2016-02-16 15:54:22,099 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-9:null) Seq 3--1: MgmtId 57177340185273: Req: Routing to peer
  121. 2016-02-16 15:54:22,100 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-10:null) Seq 3--1: MgmtId 57177340185273: Req: Cancel request received
  122. 2016-02-16 15:54:22,101 DEBUG [c.c.a.m.AgentAttache] (AgentManager-Handler-10:null) Seq 3-2144839322535198721: Cancelling.
  123. 2016-02-16 15:54:22,101 DEBUG [c.c.a.m.AgentAttache] (StatsCollector-1:ctx-43c26ea5) Seq 3-2144839322535198721: Waiting some more time because this is the current command
  124. 2016-02-16 15:54:22,101 DEBUG [c.c.a.m.AgentAttache] (StatsCollector-1:ctx-43c26ea5) Seq 3-2144839322535198721: Waiting some more time because this is the current command
  125. 2016-02-16 15:54:22,101 INFO [c.c.u.e.CSExceptionErrorCode] (StatsCollector-1:ctx-43c26ea5) Could not find exception: com.cloud.exception.OperationTimedoutException in error code list for exceptions
  126. 2016-02-16 15:54:22,101 WARN [c.c.a.m.AgentAttache] (StatsCollector-1:ctx-43c26ea5) Seq 3-2144839322535198721: Timed out on null
  127. 2016-02-16 15:54:22,102 DEBUG [c.c.a.m.AgentAttache] (StatsCollector-1:ctx-43c26ea5) Seq 3-2144839322535198721: Cancelling.
  128. 2016-02-16 15:54:22,102 WARN [c.c.a.m.AgentManagerImpl] (StatsCollector-1:ctx-43c26ea5) Operation timed out: Commands 2144839322535198721 to Host 3 timed out after 3600
  129. 2016-02-16 15:54:22,102 WARN [c.c.r.ResourceManagerImpl] (StatsCollector-1:ctx-43c26ea5) Unable to obtain host 3 statistics.
  130. 2016-02-16 15:54:22,102 WARN [c.c.s.StatsCollector] (StatsCollector-1:ctx-43c26ea5) Received invalid host stats for host: 3
  131. 2016-02-16 15:54:22,110 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (StatsCollector-1:ctx-43c26ea5) Host 8 has switched to another management server, need to update agent map with a forwarding agent attache
  132. 2016-02-16 15:54:22,110 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (StatsCollector-1:ctx-43c26ea5) create forwarding ClusteredAgentAttache for 8
  133. 2016-02-16 15:54:22,115 DEBUG [c.c.a.m.ClusteredAgentAttache] (StatsCollector-1:ctx-43c26ea5) Seq 8-8092968530384781313: Forwarding null to 57177340185274
  134. 2016-02-16 15:54:22,117 DEBUG [c.c.a.m.ClusteredAgentAttache] (AgentManager-Handler-11:null) Seq 8-8092968530384781313: Routing from 57177340185273
  135. 2016-02-16 15:54:22,118 DEBUG [c.c.a.m.ClusteredAgentAttache] (AgentManager-Handler-11:null) Seq 8-8092968530384781313: Link is closed
  136. 2016-02-16 15:54:22,119 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-11:null) Seq 8-8092968530384781313: MgmtId 57177340185273: Req: Resource [Host:8] is unreachable: Host 8: Link is closed
  137. 2016-02-16 15:54:22,119 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-11:null) Seq 8--1: MgmtId 57177340185273: Req: Routing to peer
  138. 2016-02-16 15:54:22,120 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-12:null) Seq 8--1: MgmtId 57177340185273: Req: Cancel request received
  139. 2016-02-16 15:54:22,122 DEBUG [c.c.a.m.AgentAttache] (AgentManager-Handler-12:null) Seq 8-8092968530384781313: Cancelling.
  140. 2016-02-16 15:54:22,122 DEBUG [c.c.a.m.AgentAttache] (StatsCollector-1:ctx-43c26ea5) Seq 8-8092968530384781313: Waiting some more time because this is the current command
  141. 2016-02-16 15:54:22,122 DEBUG [c.c.a.m.AgentAttache] (StatsCollector-1:ctx-43c26ea5) Seq 8-8092968530384781313: Waiting some more time because this is the current command
  142. 2016-02-16 15:54:22,122 INFO [c.c.u.e.CSExceptionErrorCode] (StatsCollector-1:ctx-43c26ea5) Could not find exception: com.cloud.exception.OperationTimedoutException in error code list for exceptions
  143. 2016-02-16 15:54:22,122 WARN [c.c.a.m.AgentAttache] (StatsCollector-1:ctx-43c26ea5) Seq 8-8092968530384781313: Timed out on null
  144. 2016-02-16 15:54:22,123 DEBUG [c.c.a.m.AgentAttache] (StatsCollector-1:ctx-43c26ea5) Seq 8-8092968530384781313: Cancelling.
  145. 2016-02-16 15:54:22,123 WARN [c.c.a.m.AgentManagerImpl] (StatsCollector-1:ctx-43c26ea5) Operation timed out: Commands 8092968530384781313 to Host 8 timed out after 3600
  146. 2016-02-16 15:54:22,123 WARN [c.c.r.ResourceManagerImpl] (StatsCollector-1:ctx-43c26ea5) Unable to obtain host 8 statistics.
  147. 2016-02-16 15:54:22,123 WARN [c.c.s.StatsCollector] (StatsCollector-1:ctx-43c26ea5) Received invalid host stats for host: 8
  148. 2016-02-16 15:54:22,129 DEBUG [c.c.a.m.ClusteredAgentAttache] (StatsCollector-1:ctx-43c26ea5) Seq 1-4458000681143369731: Forwarding null to 57177340185274
  149. 2016-02-16 15:54:22,130 DEBUG [c.c.a.m.ClusteredAgentAttache] (AgentManager-Handler-13:null) Seq 1-4458000681143369731: Routing from 57177340185273
  150. 2016-02-16 15:54:22,131 DEBUG [c.c.a.m.ClusteredAgentAttache] (AgentManager-Handler-13:null) Seq 1-4458000681143369731: Link is closed
  151. 2016-02-16 15:54:22,131 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-13:null) Seq 1-4458000681143369731: MgmtId 57177340185273: Req: Resource [Host:1] is unreachable: Host 1: Link is closed
  152. 2016-02-16 15:54:22,131 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-13:null) Seq 1--1: MgmtId 57177340185273: Req: Routing to peer
  153. 2016-02-16 15:54:22,132 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-14:null) Seq 1--1: MgmtId 57177340185273: Req: Cancel request received
  154. 2016-02-16 15:54:22,133 DEBUG [c.c.a.m.AgentAttache] (AgentManager-Handler-14:null) Seq 1-4458000681143369731: Cancelling.
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!