API tools faq
paste
juesor

SecStor Fail

juesor
Feb 23rd, 2015
268
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 197.56 KB | None | 0 0
raw download clone embed print report
  1. 2015-02-23 11:54:54,631 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) We are allocating VM, increasing the used capacity of this host:2
  2. 2015-02-23 11:54:54,631 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Current Used CPU: 0 , Free CPU:48000 ,Requested CPU: 500
  3. 2015-02-23 11:54:54,631 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Current Used RAM: 0 , Free RAM:128787136512 ,Requested RAM: 536870912
  4. 2015-02-23 11:54:54,631 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) CPU STATS after allocation: for host: 2, old used: 0, old reserved: 0, actual total: 48000, total with overprovisioning: 48000; new used:500, reserved:0; requested cpu:500,alloc_from_last:false
  5. 2015-02-23 11:54:54,632 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) RAM STATS after allocation: for host: 2, old used: 0, old reserved: 0, total: 128787136512; new used: 536870912, reserved: 0; requested mem: 536870912,alloc_from_last:false
  6. 2015-02-23 11:54:54,664 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) VM is being created in podId: 1
  7. 2015-02-23 11:54:54,671 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Network id=200 is already implemented
  8. 2015-02-23 11:54:54,711 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Network id=202 is already implemented
  9. 2015-02-23 11:54:54,789 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Network id=201 is already implemented
  10. 2015-02-23 11:54:54,825 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Releasing ip address for instance=66
  11. 2015-02-23 11:54:54,840 DEBUG [c.c.n.g.PodBasedNetworkGuru] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Allocated a nic NicProfile[66-17-14e955a3-d9ae-48aa-8dab-7a4e74ceb1c6-10.90.2.206-null for VM[SecondaryStorageVm|s-17-VM]
  12. 2015-02-23 11:54:54,873 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Network id=203 is already implemented
  13. 2015-02-23 11:54:54,929 DEBUG [c.c.n.g.StorageNetworkGuru] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Allocated a storage nic NicProfile[67-17-14e955a3-d9ae-48aa-8dab-7a4e74ceb1c6-10.83.2.204-storage://832 for VM[SecondaryStorageVm|s-17-VM]
  14. 2015-02-23 11:54:54,948 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Checking if we need to prepare 1 volumes for VM[SecondaryStorageVm|s-17-VM]
  15. 2015-02-23 11:54:54,949 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) No need to recreate the volume: Vol[17|vm=17|ROOT], since it already has a pool assigned: 1, adding disk to VM
  16. 2015-02-23 11:54:54,981 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) VmWare hypervisor configured, telling the ssvm to load the PremiumSecondaryStorageResource
  17. 2015-02-23 11:54:54,987 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Boot Args for VM[SecondaryStorageVm|s-17-VM]: template=domP type=secstorage host=10.90.1.21 port=8250 name=s-17-VM zone=1 pod=1 guid=s-17-VM workers=5 resource=com.cloud.storage.resource.PremiumSecondaryStorageResource instance=SecStorage sslcopy=false role=templateProcessor mtu=1500 eth2ip=205.185.158.69 eth2mask=255.255.255.192 gateway=205.185.158.65 public.network.device=eth2 eth0ip=169.254.0.109 eth0mask=255.255.0.0 eth1ip=10.90.2.206 eth1mask=255.255.255.0 mgmtcidr=10.90.1.0/24 localgw=10.90.2.254 private.network.device=eth1 eth3ip=10.83.2.204 eth3mask=255.255.255.0 storageip=10.83.2.204 storagenetmask=255.255.255.0 storagegateway=10.83.2.1 internaldns1=10.90.2.41 internaldns2=10.90.2.42 dns1=208.74.240.5 dns2=208.74.247.245
  18. 2015-02-23 11:54:55,049 DEBUG [c.c.a.t.Request] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Seq 2-8777234198768386141: Sending { Cmd , MgmtId: 345050411715, via: 2(Flex-Xen1.flexhost.local), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StartCommand":{"vm":{"id":17,"name":"s-17-VM","bootloader":"PyGrub","type":"SecondaryStorageVm","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian GNU/Linux 7(64-bit)","platformEmulator":"Debian Wheezy 7.0 (64-bit)","bootArgs":" template=domP type=secstorage host=10.90.1.21 port=8250 name=s-17-VM zone=1 pod=1 guid=s-17-VM workers=5 resource=com.cloud.storage.resource.PremiumSecondaryStorageResource instance=SecStorage sslcopy=false role=templateProcessor mtu=1500 eth2ip=205.185.158.69 eth2mask=255.255.255.192 gateway=205.185.158.65 public.network.device=eth2 eth0ip=169.254.0.109 eth0mask=255.255.0.0 eth1ip=10.90.2.206 eth1mask=255.255.255.0 mgmtcidr=10.90.1.0/24 localgw=10.90.2.254 private.network.device=eth1 eth3ip=10.83.2.204 eth3mask=255.255.255.0 storageip=10.83.2.204 storagenetmask=255.255.255.0 storagegateway=10.83.2.1 internaldns1=10.90.2.41 internaldns2=10.90.2.42 dns1=208.74.240.5 dns2=208.74.247.245","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"cUF2OGhYFGZtCrttlpKhiw==","params":{},"uuid":"e00bed4f-47fe-42cb-830e-4c70a3bc2546","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"73b9fc39-35b0-46eb-bee6-1e52fc4f9b7e","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"FlexSAN1-LUN0","id":1,"poolType":"PreSetup","host":"localhost","path":"/FlexSAN1-LUN0","port":0,"url":"PreSetup://localhost/FlexSAN1-LUN0/?ROLE=Primary&STOREUUID=FlexSAN1-LUN0"}},"name":"ROOT-17","size":2684354560,"path":"850139ef-2fe6-4118-9486-c6486dcc798a","volumeId":17,"vmName":"s-17-VM","accountId":1,"format":"VHD","provisioningType":"THIN","id":17,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"850139ef-2fe6-4118-9486-c6486dcc798a","type":"ROOT","_details":{"managed":"false","storagePort":"0","storageHost":"localhost","volumeSize":"2684354560"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"4af99ea2-0f8d-42df-815c-c524a7847e5a","uuid":"bc1ef7ea-19c0-41f1-827d-fc556a73df48","ip":"205.185.158.69","netmask":"255.255.255.192","gateway":"205.185.158.65","mac":"06:29:0e:00:00:0c","dns1":"208.74.240.5","dns2":"208.74.247.245","broadcastType":"Vlan","type":"Public","broadcastUri":"vlan://1755","isolationUri":"vlan://1755","isSecurityGroupEnabled":false,"name":"GUEST-PUB"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"c4469cb3-9f89-44cc-9e3c-cf7f6cbc0f6e","uuid":"d418b3c2-2122-42ca-93df-5f18ccd5274b","ip":"169.254.0.109","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:00:6d","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"a6398c49-3400-46c6-8bcc-319e276b1425","uuid":"5eaf68eb-a5d3-4d9c-ab71-513926e4d58f","ip":"10.90.2.206","netmask":"255.255.255.0","gateway":"10.90.2.254","mac":"06:84:e6:00:00:06","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"MGMT"},{"deviceId":3,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"5ad17f51-f91a-4111-a699-7961637dce94","uuid":"3c76f7d5-db9c-4a42-8fc7-9663c8ed2728","ip":"10.83.2.204","netmask":"255.255.255.0","gateway":"10.83.2.1","mac":"06:ef:8e:00:00:49","broadcastType":"Storage","type":"Storage","broadcastUri":"storage://832","isSecurityGroupEnabled":false,"name":"SEC-STORE"}]},"hostIp":"10.90.2.111","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.0.109","port":3922,"interval":6,"retries":100,"name":"s-17-VM","wait":0}}] }
  19. 2015-02-23 11:54:55,051 DEBUG [c.c.a.t.Request] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Seq 2-8777234198768386141: Executing: { Cmd , MgmtId: 345050411715, via: 2(Flex-Xen1.flexhost.local), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StartCommand":{"vm":{"id":17,"name":"s-17-VM","bootloader":"PyGrub","type":"SecondaryStorageVm","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian GNU/Linux 7(64-bit)","platformEmulator":"Debian Wheezy 7.0 (64-bit)","bootArgs":" template=domP type=secstorage host=10.90.1.21 port=8250 name=s-17-VM zone=1 pod=1 guid=s-17-VM workers=5 resource=com.cloud.storage.resource.PremiumSecondaryStorageResource instance=SecStorage sslcopy=false role=templateProcessor mtu=1500 eth2ip=205.185.158.69 eth2mask=255.255.255.192 gateway=205.185.158.65 public.network.device=eth2 eth0ip=169.254.0.109 eth0mask=255.255.0.0 eth1ip=10.90.2.206 eth1mask=255.255.255.0 mgmtcidr=10.90.1.0/24 localgw=10.90.2.254 private.network.device=eth1 eth3ip=10.83.2.204 eth3mask=255.255.255.0 storageip=10.83.2.204 storagenetmask=255.255.255.0 storagegateway=10.83.2.1 internaldns1=10.90.2.41 internaldns2=10.90.2.42 dns1=208.74.240.5 dns2=208.74.247.245","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"cUF2OGhYFGZtCrttlpKhiw==","params":{},"uuid":"e00bed4f-47fe-42cb-830e-4c70a3bc2546","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"73b9fc39-35b0-46eb-bee6-1e52fc4f9b7e","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"FlexSAN1-LUN0","id":1,"poolType":"PreSetup","host":"localhost","path":"/FlexSAN1-LUN0","port":0,"url":"PreSetup://localhost/FlexSAN1-LUN0/?ROLE=Primary&STOREUUID=FlexSAN1-LUN0"}},"name":"ROOT-17","size":2684354560,"path":"850139ef-2fe6-4118-9486-c6486dcc798a","volumeId":17,"vmName":"s-17-VM","accountId":1,"format":"VHD","provisioningType":"THIN","id":17,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"850139ef-2fe6-4118-9486-c6486dcc798a","type":"ROOT","_details":{"managed":"false","storagePort":"0","storageHost":"localhost","volumeSize":"2684354560"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"4af99ea2-0f8d-42df-815c-c524a7847e5a","uuid":"bc1ef7ea-19c0-41f1-827d-fc556a73df48","ip":"205.185.158.69","netmask":"255.255.255.192","gateway":"205.185.158.65","mac":"06:29:0e:00:00:0c","dns1":"208.74.240.5","dns2":"208.74.247.245","broadcastType":"Vlan","type":"Public","broadcastUri":"vlan://1755","isolationUri":"vlan://1755","isSecurityGroupEnabled":false,"name":"GUEST-PUB"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"c4469cb3-9f89-44cc-9e3c-cf7f6cbc0f6e","uuid":"d418b3c2-2122-42ca-93df-5f18ccd5274b","ip":"169.254.0.109","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:00:6d","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"a6398c49-3400-46c6-8bcc-319e276b1425","uuid":"5eaf68eb-a5d3-4d9c-ab71-513926e4d58f","ip":"10.90.2.206","netmask":"255.255.255.0","gateway":"10.90.2.254","mac":"06:84:e6:00:00:06","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"MGMT"},{"deviceId":3,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"5ad17f51-f91a-4111-a699-7961637dce94","uuid":"3c76f7d5-db9c-4a42-8fc7-9663c8ed2728","ip":"10.83.2.204","netmask":"255.255.255.0","gateway":"10.83.2.1","mac":"06:ef:8e:00:00:49","broadcastType":"Storage","type":"Storage","broadcastUri":"storage://832","isSecurityGroupEnabled":false,"name":"SEC-STORE"}]},"hostIp":"10.90.2.111","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.0.109","port":3922,"interval":6,"retries":100,"name":"s-17-VM","wait":0}}] }
  20. 2015-02-23 11:54:55,052 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-24:ctx-1a05c11e) Seq 2-8777234198768386141: Executing request
  21. 2015-02-23 11:54:55,059 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) 1. The VM s-17-VM is in Starting state.
  22. 2015-02-23 11:54:55,092 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) Created VM fd9f69ee-8c89-2a3c-4c32-97728b53fe78 for s-17-VM
  23. 2015-02-23 11:54:55,099 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) PV args are -- quiet console=hvc0%template=domP%type=secstorage%host=10.90.1.21%port=8250%name=s-17-VM%zone=1%pod=1%guid=s-17-VM%workers=5%resource=com.cloud.storage.resource.PremiumSecondaryStorageResource%instance=SecStorage%sslcopy=false%role=templateProcessor%mtu=1500%eth2ip=205.185.158.69%eth2mask=255.255.255.192%gateway=205.185.158.65%public.network.device=eth2%eth0ip=169.254.0.109%eth0mask=255.255.0.0%eth1ip=10.90.2.206%eth1mask=255.255.255.0%mgmtcidr=10.90.1.0/24%localgw=10.90.2.254%private.network.device=eth1%eth3ip=10.83.2.204%eth3mask=255.255.255.0%storageip=10.83.2.204%storagenetmask=255.255.255.0%storagegateway=10.83.2.1%internaldns1=10.90.2.41%internaldns2=10.90.2.42%dns1=208.74.240.5%dns2=208.74.247.245
  24. 2015-02-23 11:54:55,136 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) VBD 03c45073-e316-abc4-84bb-83b7f53515d2 created for com.cloud.agent.api.to.DiskTO@755f0a15
  25. 2015-02-23 11:54:55,158 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) Creating VIF for s-17-VM on nic [Nic:Public-205.185.158.69-vlan://1755]
  26. 2015-02-23 11:54:55,161 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) Looking for network named GUEST-PUB
  27. 2015-02-23 11:54:55,172 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) Found a network called GUEST-PUB on host=10.90.2.111; Network=644f87cd-a11e-e94b-f3d1-7e221679ded2; pif=1a83e71c-1576-b3af-ed8a-3a4a7c22dab5
  28. 2015-02-23 11:54:55,179 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) Found a network called VLAN-644f87cd-a11e-e94b-f3d1-7e221679ded2-1755 on host=10.90.2.111; Network=e379b22c-3ac8-a0de-6482-40df647fb16f; pif=40054b40-9467-2221-228e-be34f2d4d46b
  29. 2015-02-23 11:54:55,194 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) Created a vif 4c7a6feb-b7d2-2705-242f-0c00eb70c87c on 2
  30. 2015-02-23 11:54:55,194 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) Creating VIF for s-17-VM on nic [Nic:Control-169.254.0.109-null]
  31. 2015-02-23 11:54:55,215 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) already have a vif on dom0 for link local network
  32. 2015-02-23 11:54:55,407 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) Created a vif 0f23ce84-f7cf-c8c0-84e0-21c6738b9cef on 0
  33. 2015-02-23 11:54:55,407 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) Creating VIF for s-17-VM on nic [Nic:Management-10.90.2.206-null]
  34. 2015-02-23 11:54:55,410 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) Looking for network named MGMT
  35. 2015-02-23 11:54:55,424 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) Created a vif 01fcc790-f9b5-6ae9-2962-67f72e2502fe on 1
  36. 2015-02-23 11:54:55,424 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) Creating VIF for s-17-VM on nic [Nic:Storage-10.83.2.204-storage://832]
  37. 2015-02-23 11:54:55,426 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) Looking for network named SEC-STORE
  38. 2015-02-23 11:54:55,448 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) Found a network called SEC-STORE on host=10.90.2.111; Network=c9754adc-0015-88cf-8b40-b6eceed12b94; pif=aedbbb70-a2ad-174f-96ac-9a4e99597bc1
  39. 2015-02-23 11:54:55,451 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) Creating VLAN 832 on host 10.90.2.111 on device bond1
  40. 2015-02-23 11:54:55,479 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) Catch Exception: class com.xensource.xenapi.Types$PifIsVlan due to You tried to create a VLAN on top of another VLAN - use the underlying physical PIF/bond instead
  41. You tried to create a VLAN on top of another VLAN - use the underlying physical PIF/bond instead
  42. at com.xensource.xenapi.Types.checkResponse(Types.java:2003)
  43. at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
  44. at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:465)
  45. at com.xensource.xenapi.VLAN.create(VLAN.java:363)
  46. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.enableVlanNetwork(CitrixResourceBase.java:4026)
  47. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getNetwork(CitrixResourceBase.java:1043)
  48. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.createVif(CitrixResourceBase.java:1103)
  49. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1780)
  50. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:489)
  51. at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
  52. at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
  53. at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
  54. at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:304)
  55. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  56. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  57. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  58. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  59. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  60. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  61. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  62. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
  63. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
  64. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  65. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  66. at java.lang.Thread.run(Thread.java:745)
  67. 2015-02-23 11:54:55,481 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) Unable to start s-17-VM due to
  68. You tried to create a VLAN on top of another VLAN - use the underlying physical PIF/bond instead
  69. at com.xensource.xenapi.Types.checkResponse(Types.java:2003)
  70. at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
  71. at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:465)
  72. at com.xensource.xenapi.VLAN.create(VLAN.java:363)
  73. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.enableVlanNetwork(CitrixResourceBase.java:4026)
  74. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getNetwork(CitrixResourceBase.java:1043)
  75. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.createVif(CitrixResourceBase.java:1103)
  76. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1780)
  77. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:489)
  78. at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
  79. at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
  80. at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
  81. at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:304)
  82. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  83. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  84. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  85. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  86. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  87. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  88. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  89. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
  90. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
  91. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  92. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  93. at java.lang.Thread.run(Thread.java:745)
  94. 2015-02-23 11:54:55,530 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) Unable to clean up VBD due to
  95. You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
  96. at com.xensource.xenapi.Types.checkResponse(Types.java:693)
  97. at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
  98. at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:465)
  99. at com.xensource.xenapi.VBD.unplug(VBD.java:1109)
  100. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1512)
  101. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1863)
  102. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:489)
  103. at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
  104. at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
  105. at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
  106. at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:304)
  107. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  108. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  109. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  110. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  111. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  112. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  113. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  114. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
  115. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
  116. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  117. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  118. at java.lang.Thread.run(Thread.java:745)
  119. 2015-02-23 11:54:55,536 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) Unable to clean up VBD due to
  120. You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
  121. at com.xensource.xenapi.Types.checkResponse(Types.java:693)
  122. at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
  123. at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:465)
  124. at com.xensource.xenapi.VBD.unplug(VBD.java:1109)
  125. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1512)
  126. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1863)
  127. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:489)
  128. at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
  129. at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
  130. at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
  131. at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:304)
  132. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  133. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  134. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  135. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  136. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  137. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  138. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  139. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
  140. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
  141. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  142. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  143. at java.lang.Thread.run(Thread.java:745)
  144. 2015-02-23 11:54:55,542 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) Unable to cleanup VIF
  145. You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
  146. at com.xensource.xenapi.Types.checkResponse(Types.java:693)
  147. at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
  148. at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:465)
  149. at com.xensource.xenapi.VIF.unplug(VIF.java:921)
  150. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1520)
  151. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1863)
  152. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:489)
  153. at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
  154. at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
  155. at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
  156. at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:304)
  157. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  158. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  159. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  160. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  161. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  162. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  163. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  164. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
  165. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
  166. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  167. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  168. at java.lang.Thread.run(Thread.java:745)
  169. 2015-02-23 11:54:55,549 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) Unable to cleanup VIF
  170. You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
  171. at com.xensource.xenapi.Types.checkResponse(Types.java:693)
  172. at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
  173. at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:465)
  174. at com.xensource.xenapi.VIF.unplug(VIF.java:921)
  175. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1520)
  176. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1863)
  177. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:489)
  178. at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
  179. at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
  180. at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
  181. at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:304)
  182. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  183. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  184. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  185. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  186. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  187. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  188. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  189. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
  190. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
  191. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  192. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  193. at java.lang.Thread.run(Thread.java:745)
  194. 2015-02-23 11:54:55,556 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) Unable to cleanup VIF
  195. You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
  196. at com.xensource.xenapi.Types.checkResponse(Types.java:693)
  197. at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
  198. at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:465)
  199. at com.xensource.xenapi.VIF.unplug(VIF.java:921)
  200. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1520)
  201. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1863)
  202. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:489)
  203. at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
  204. at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
  205. at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
  206. at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:304)
  207. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  208. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  209. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  210. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  211. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  212. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  213. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  214. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
  215. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
  216. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  217. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  218. at java.lang.Thread.run(Thread.java:745)
  219. 2015-02-23 11:54:55,604 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-24:ctx-1a05c11e) The VM is in stopped state, detected problem during startup : s-17-VM
  220. 2015-02-23 11:54:55,604 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-24:ctx-1a05c11e) Seq 2-8777234198768386141: Cancelling because one of the answers is false and it is stop on error.
  221. 2015-02-23 11:54:55,604 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-24:ctx-1a05c11e) Seq 2-8777234198768386141: Response Received:
  222. 2015-02-23 11:54:55,606 DEBUG [c.c.a.t.Request] (DirectAgent-24:ctx-1a05c11e) Seq 2-8777234198768386141: Processing: { Ans: , MgmtId: 345050411715, via: 2, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":17,"name":"s-17-VM","bootloader":"PyGrub","type":"SecondaryStorageVm","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian GNU/Linux 7(64-bit)","platformEmulator":"Debian Wheezy 7.0 (64-bit)","bootArgs":" template=domP type=secstorage host=10.90.1.21 port=8250 name=s-17-VM zone=1 pod=1 guid=s-17-VM workers=5 resource=com.cloud.storage.resource.PremiumSecondaryStorageResource instance=SecStorage sslcopy=false role=templateProcessor mtu=1500 eth2ip=205.185.158.69 eth2mask=255.255.255.192 gateway=205.185.158.65 public.network.device=eth2 eth0ip=169.254.0.109 eth0mask=255.255.0.0 eth1ip=10.90.2.206 eth1mask=255.255.255.0 mgmtcidr=10.90.1.0/24 localgw=10.90.2.254 private.network.device=eth1 eth3ip=10.83.2.204 eth3mask=255.255.255.0 storageip=10.83.2.204 storagenetmask=255.255.255.0 storagegateway=10.83.2.1 internaldns1=10.90.2.41 internaldns2=10.90.2.42 dns1=208.74.240.5 dns2=208.74.247.245","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"cUF2OGhYFGZtCrttlpKhiw==","params":{},"uuid":"e00bed4f-47fe-42cb-830e-4c70a3bc2546","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"73b9fc39-35b0-46eb-bee6-1e52fc4f9b7e","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"FlexSAN1-LUN0","id":1,"poolType":"PreSetup","host":"localhost","path":"/FlexSAN1-LUN0","port":0,"url":"PreSetup://localhost/FlexSAN1-LUN0/?ROLE=Primary&STOREUUID=FlexSAN1-LUN0"}},"name":"ROOT-17","size":2684354560,"path":"850139ef-2fe6-4118-9486-c6486dcc798a","volumeId":17,"vmName":"s-17-VM","accountId":1,"format":"VHD","provisioningType":"THIN","id":17,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"850139ef-2fe6-4118-9486-c6486dcc798a","type":"ROOT","_details":{"managed":"false","storagePort":"0","storageHost":"localhost","volumeSize":"2684354560"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"4af99ea2-0f8d-42df-815c-c524a7847e5a","uuid":"bc1ef7ea-19c0-41f1-827d-fc556a73df48","ip":"205.185.158.69","netmask":"255.255.255.192","gateway":"205.185.158.65","mac":"06:29:0e:00:00:0c","dns1":"208.74.240.5","dns2":"208.74.247.245","broadcastType":"Vlan","type":"Public","broadcastUri":"vlan://1755","isolationUri":"vlan://1755","isSecurityGroupEnabled":false,"name":"GUEST-PUB"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"c4469cb3-9f89-44cc-9e3c-cf7f6cbc0f6e","uuid":"d418b3c2-2122-42ca-93df-5f18ccd5274b","ip":"169.254.0.109","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:00:6d","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"a6398c49-3400-46c6-8bcc-319e276b1425","uuid":"5eaf68eb-a5d3-4d9c-ab71-513926e4d58f","ip":"10.90.2.206","netmask":"255.255.255.0","gateway":"10.90.2.254","mac":"06:84:e6:00:00:06","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"MGMT"},{"deviceId":3,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"5ad17f51-f91a-4111-a699-7961637dce94","uuid":"3c76f7d5-db9c-4a42-8fc7-9663c8ed2728","ip":"10.83.2.204","netmask":"255.255.255.0","gateway":"10.83.2.1","mac":"06:ef:8e:00:00:49","broadcastType":"Storage","type":"Storage","broadcastUri":"storage://832","isSecurityGroupEnabled":false,"name":"SEC-STORE"}]},"_iqnToPath":{},"result":false,"details":"Unable to start s-17-VM due to ","wait":0}}] }
  223. 2015-02-23 11:54:55,606 DEBUG [c.c.a.t.Request] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Seq 2-8777234198768386141: Received: { Ans: , MgmtId: 345050411715, via: 2, Ver: v1, Flags: 10, { StartAnswer } }
  224. 2015-02-23 11:54:55,621 INFO [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Unable to start VM on Host[-2-Routing] due to Unable to start s-17-VM due to
  225. 2015-02-23 11:54:55,650 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Cleaning up resources for the vm VM[SecondaryStorageVm|s-17-VM] in Starting state
  226. 2015-02-23 11:54:55,656 DEBUG [c.c.a.t.Request] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Seq 2-8777234198768386142: Sending { Cmd , MgmtId: 345050411715, via: 2(Flex-Xen1.flexhost.local), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"s-17-VM","wait":0}}] }
  227. 2015-02-23 11:54:55,656 DEBUG [c.c.a.t.Request] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Seq 2-8777234198768386142: Executing: { Cmd , MgmtId: 345050411715, via: 2(Flex-Xen1.flexhost.local), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"s-17-VM","wait":0}}] }
  228. 2015-02-23 11:54:55,656 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-95:ctx-b9b966f1) Seq 2-8777234198768386142: Executing request
  229. 2015-02-23 11:54:55,664 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-95:ctx-b9b966f1) Seq 2-8777234198768386142: Response Received:
  230. 2015-02-23 11:54:55,664 DEBUG [c.c.a.t.Request] (DirectAgent-95:ctx-b9b966f1) Seq 2-8777234198768386142: Processing: { Ans: , MgmtId: 345050411715, via: 2, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StopAnswer":{"result":true,"details":"VM does not exist","wait":0}}] }
  231. 2015-02-23 11:54:55,664 DEBUG [c.c.a.t.Request] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Seq 2-8777234198768386142: Received: { Ans: , MgmtId: 345050411715, via: 2, Ver: v1, Flags: 10, { StopAnswer } }
  232. 2015-02-23 11:54:55,694 DEBUG [c.c.n.g.ControlNetworkGuru] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Released nic: NicProfile[65-17-null-null-null
  233. 2015-02-23 11:54:55,719 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Releasing ip address for reservationId=14e955a3-d9ae-48aa-8dab-7a4e74ceb1c6, instance=66
  234. 2015-02-23 11:54:55,720 DEBUG [c.c.n.g.PodBasedNetworkGuru] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Released nic: NicProfile[66-17-null-null-null
  235. 2015-02-23 11:54:55,745 DEBUG [c.c.n.g.StorageNetworkGuru] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Release an storage ip 10.83.2.204
  236. 2015-02-23 11:54:55,760 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Successfully released network resources for the vm VM[SecondaryStorageVm|s-17-VM]
  237. 2015-02-23 11:54:55,760 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Successfully cleanued up resources for the vm VM[SecondaryStorageVm|s-17-VM] in Starting state
  238. 2015-02-23 11:54:55,765 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Root volume is ready, need to place VM in volume's cluster
  239. 2015-02-23 11:54:55,765 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Vol[17|vm=17|ROOT] is READY, changing deployment plan to use this pool's dcId: 1 , podId: 1 , and clusterId: 1
  240. 2015-02-23 11:54:55,769 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Deploy avoids pods: null, clusters: null, hosts: [1, 2]
  241. 2015-02-23 11:54:55,771 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) DeploymentPlanner allocation algorithm: com.cloud.deploy.FirstFitPlanner@7ace2d66
  242. 2015-02-23 11:54:55,771 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Trying to allocate a host and storage pools from dc:1, pod:1,cluster:1, requested cpu: 500, requested ram: 536870912
  243. 2015-02-23 11:54:55,771 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Is ROOT volume READY (pool already allocated)?: Yes
  244. 2015-02-23 11:54:55,771 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Searching resources only under specified Cluster: 1
  245. 2015-02-23 11:54:55,783 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Checking resources in Cluster: 1 under Pod: 1
  246. 2015-02-23 11:54:55,783 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041 FirstFitRoutingAllocator) Looking for hosts in dc: 1 pod:1 cluster:1
  247. 2015-02-23 11:54:55,787 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041 FirstFitRoutingAllocator) FirstFitAllocator has 2 hosts to check for allocation: [Host[-1-Routing], Host[-2-Routing]]
  248. 2015-02-23 11:54:55,797 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041 FirstFitRoutingAllocator) Found 2 hosts for allocation after prioritization: [Host[-1-Routing], Host[-2-Routing]]
  249. 2015-02-23 11:54:55,797 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041 FirstFitRoutingAllocator) Looking for speed=500Mhz, Ram=512
  250. 2015-02-23 11:54:55,797 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041 FirstFitRoutingAllocator) Host name: Flex-Xen2.flexhost.local, hostId: 1 is in avoid set, skipping this and trying other available hosts
  251. 2015-02-23 11:54:55,797 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041 FirstFitRoutingAllocator) Host name: Flex-Xen1.flexhost.local, hostId: 2 is in avoid set, skipping this and trying other available hosts
  252. 2015-02-23 11:54:55,797 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041 FirstFitRoutingAllocator) Host Allocator returning 0 suitable hosts
  253. 2015-02-23 11:54:55,797 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) No suitable hosts found
  254. 2015-02-23 11:54:55,797 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) No suitable hosts found under this Cluster: 1
  255. 2015-02-23 11:54:55,800 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Could not find suitable Deployment Destination for this VM under any clusters, returning.
  256. 2015-02-23 11:54:55,800 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Searching resources only under specified Cluster: 1
  257. 2015-02-23 11:54:55,801 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) The specified cluster is in avoid set, returning.
  258. 2015-02-23 11:54:55,805 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Deploy avoids pods: null, clusters: [1], hosts: [1, 2]
  259. 2015-02-23 11:54:55,806 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) DeploymentPlanner allocation algorithm: com.cloud.deploy.FirstFitPlanner@7ace2d66
  260. 2015-02-23 11:54:55,806 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Trying to allocate a host and storage pools from dc:1, pod:null,cluster:null, requested cpu: 500, requested ram: 536870912
  261. 2015-02-23 11:54:55,806 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Is ROOT volume READY (pool already allocated)?: No
  262. 2015-02-23 11:54:55,807 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Searching all possible resources under this Zone: 1
  263. 2015-02-23 11:54:55,808 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Listing clusters in order of aggregate capacity, that have (atleast one host with) enough CPU and RAM capacity under this Zone: 1
  264. 2015-02-23 11:54:55,813 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Removing from the clusterId list these clusters from avoid set: [1]
  265. 2015-02-23 11:54:55,816 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) No clusters found after removing disabled clusters and clusters in avoid list, returning.
  266. 2015-02-23 11:54:55,874 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) VM state transitted from :Starting to Stopped with event: OperationFailedvm's original host id: null new host id: null host id before state transition: 2
  267. 2015-02-23 11:54:55,883 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Hosts's actual total CPU: 48000 and CPU after applying overprovisioning: 48000
  268. 2015-02-23 11:54:55,883 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Hosts's actual total RAM: 128787138048 and RAM after applying overprovisioning: 128787136512
  269. 2015-02-23 11:54:55,883 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) release cpu from host: 2, old used: 500,reserved: 0, actual total: 48000, total with overprovisioning: 48000; new used: 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
  270. 2015-02-23 11:54:55,883 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) release mem from host: 2, old used: 536870912,reserved: 0, total: 128787136512; new used: 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
  271. 2015-02-23 11:54:55,900 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Invocation exception, caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[SecondaryStorageVm|s-17-VM]Scope=interface com.cloud.dc.DataCenter; id=1
  272. 2015-02-23 11:54:55,900 INFO [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103 ctx-72716041) Rethrow exception com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[SecondaryStorageVm|s-17-VM]Scope=interface com.cloud.dc.DataCenter; id=1
  273. 2015-02-23 11:54:55,900 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103) Done with run of VM work job: com.cloud.vm.VmWorkStart for VM 17, job origin: 2
  274. 2015-02-23 11:54:55,901 ERROR [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103) Unable to complete AsyncJobVO {id:103, userId: 1, accountId: 1, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo: rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAEXQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 345050411715, completeMsid: null, lastUpdated: null, lastPolled: null, created: Mon Feb 23 11:54:49 CST 2015}, job origin:2
  275. com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[SecondaryStorageVm|s-17-VM]Scope=interface com.cloud.dc.DataCenter; id=1
  276. at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:941)
  277. at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4471)
  278. at sun.reflect.GeneratedMethodAccessor258.invoke(Unknown Source)
  279. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  280. at java.lang.reflect.Method.invoke(Method.java:606)
  281. at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
  282. at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4627)
  283. at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:103)
  284. at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:536)
  285. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  286. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  287. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  288. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  289. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  290. at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:493)
  291. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  292. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  293. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  294. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  295. at java.lang.Thread.run(Thread.java:745)
  296. 2015-02-23 11:54:55,903 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103) Complete async job-103, jobStatus: FAILED, resultCode: 0, result: rO0ABXNyADdjb20uY2xvdWQuZXhjZXB0aW9uLkluc3VmZmljaWVudFNlcnZlckNhcGFjaXR5RXhjZXB0aW9uAAAAAFZNT3ACAAFaABVhZmZpbml0eUdyb3Vwc0FwcGxpZWR4cgAxY29tLmNsb3VkLmV4Y2VwdGlvbi5JbnN1ZmZpY2llbnRDYXBhY2l0eUV4Y2VwdGlvbgAAAABWTU91AgACTAACaWR0ABBMamF2YS9sYW5nL0xvbmc7TAAFc2NvcGV0ABFMamF2YS9sYW5nL0NsYXNzO3hyACJjb20uY2xvdWQuZXhjZXB0aW9uLkNsb3VkRXhjZXB0aW9ueeiOG_HkRMcCAAJMAAtjc0Vycm9yQ29kZXQAE0xqYXZhL2xhbmcvSW50ZWdlcjtMAAZpZExpc3R0ABVMamF2YS91dGlsL0FycmF5TGlzdDt4cgATamF2YS5sYW5nLkV4Y2VwdGlvbtD9Hz4aOxzEAgAAeHIAE2phdmEubGFuZy5UaHJvd2FibGXVxjUnOXe4ywMABEwABWNhdXNldAAVTGphdmEvbGFuZy9UaHJvd2FibGU7TAANZGV0YWlsTWVzc2FnZXQAEkxqYXZhL2xhbmcvU3RyaW5nO1sACnN0YWNrVHJhY2V0AB5bTGphdmEvbGFuZy9TdGFja1RyYWNlRWxlbWVudDtMABRzdXBwcmVzc2VkRXhjZXB0aW9uc3QAEExqYXZhL3V0aWwvTGlzdDt4cHEAfgANdABAVW5hYmxlIHRvIGNyZWF0ZSBhIGRlcGxveW1lbnQgZm9yIFZNW1NlY29uZGFyeVN0b3JhZ2VWbXxzLTE3LVZNXXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAFHNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAKTAAIZmlsZU5hbWVxAH4ACkwACm1ldGhvZE5hbWVxAH4ACnhwAAADrXQAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHNxAH4AEQAAEXdxAH4AE3EAfgAUcQB-ABVzcQB-ABH_____dAAmc3VuLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3IyNThwdAAGaW52b2tlc3EAfgARAAAAK3QAKHN1bi5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGXNxAH4AEQAAAl50ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAZc3EAfgARAAAAa3QAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnNxAH4AEQAAEhNxAH4AE3EAfgAUcQB-ACNzcQB-ABEAAABndAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-ABEAAAIYdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgARAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgARAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4AEQAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ADN0AA9jYWxsV2l0aENvbnRleHRzcQB-ABEAAAA1cQB-ADZxAH4AM3QADnJ1bldpdGhDb250ZXh0c3EAfgARAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAvcQB-ADBzcQB-ABEAAAHtcQB-ACpxAH4AK3EAfgAwc3EAfgARAAAB13QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgA0c3EAfgARAAABBnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4AMHNxAH4AEQAABHl0ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnNxAH4AEQAAAmd0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEVxAH4AMHNxAH4AEQAAAul0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4AMHNyACZqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlTGlzdPwPJTG17I4QAgABTAAEbGlzdHEAfgAMeHIALGphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVDb2xsZWN0aW9uGUIAgMte9x4CAAFMAAFjdAAWTGphdmEvdXRpbC9Db2xsZWN0aW9uO3hwc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAB3BAAAAAB4cQB-AFF4c3IAEWphdmEubGFuZy5JbnRlZ2VyEuKgpPeBhzgCAAFJAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAEO9zcQB-AFAAAAAAdwQAAAAAeHNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cQB-AFMAAAAAAAAAAXZyABdjb20uY2xvdWQuZGMuRGF0YUNlbnRlcrLxCuun53i-AgAAeHAA
  297. 2015-02-23 11:54:55,905 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103) Publish async job-103 complete on message bus
  298. 2015-02-23 11:54:55,905 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103) Wake up jobs related to job-103
  299. 2015-02-23 11:54:55,905 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103) Update db status for job-103
  300. 2015-02-23 11:54:55,907 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103) Wake up jobs joined with job-103 and disjoin all subjobs created from job- 103
  301. 2015-02-23 11:54:55,926 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103) Done executing com.cloud.vm.VmWorkStart for job-103
  302. 2015-02-23 11:54:55,932 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103) Sync queue (65) is currently empty
  303. 2015-02-23 11:54:55,933 INFO [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-65:ctx-8b769553 job-2/job-103) Remove job-103 from job monitoring
  304. 2015-02-23 11:54:55,948 WARN [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-434deb3a) Exception while trying to start secondary storage vm
  305. com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[SecondaryStorageVm|s-17-VM]Scope=interface com.cloud.dc.DataCenter; id=1
  306. at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:941)
  307. at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4471)
  308. at sun.reflect.GeneratedMethodAccessor258.invoke(Unknown Source)
  309. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  310. at java.lang.reflect.Method.invoke(Method.java:606)
  311. at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
  312. at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4627)
  313. at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:103)
  314. at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:536)
  315. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  316. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  317. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  318. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  319. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  320. at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:493)
  321. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  322. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  323. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  324. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  325. at java.lang.Thread.run(Thread.java:745)
  326. 2015-02-23 11:54:55,949 INFO [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-434deb3a) Unable to start secondary storage vm for standby capacity, secStorageVm vm Id : 17, will recycle it and start a new one
  327. 2015-02-23 11:54:55,949 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter] (secstorage-1:ctx-434deb3a) received secondary storage vm alert
  328. 2015-02-23 11:54:55,950 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter] (secstorage-1:ctx-434deb3a) Secondary Storage Vm creation failure, zone: Rushford
  329. 2015-02-23 11:54:55,951 WARN [o.a.c.alerts] (secstorage-1:ctx-434deb3a) alertType:: 19 // dataCenterId:: 1 // podId:: null // clusterId:: null // message:: Secondary Storage Vm creation failure. zone: Rushford, error details: null
  330. 2015-02-23 11:54:55,965 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:ctx-434deb3a) Primary secondary storage is not even started, wait until next turn
  331. 2015-02-23 11:55:00,798 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-2252c7e4) Begin cleanup expired async-jobs
  332. 2015-02-23 11:55:00,807 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-2252c7e4) End cleanup expired async-jobs
  333. 2015-02-23 11:55:10,798 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-7206414c) Begin cleanup expired async-jobs
  334. 2015-02-23 11:55:10,808 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-7206414c) End cleanup expired async-jobs
  335. 2015-02-23 11:55:11,131 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-e5669991) Found 0 routers to update status.
  336. 2015-02-23 11:55:11,133 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:ctx-e5669991) Found 0 networks to update RvR status.
  337. 2015-02-23 11:55:14,712 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgentCronJob-13:ctx-8b98b02f) Ping from 1(Flex-Xen2.flexhost.local)
  338. 2015-02-23 11:55:14,712 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-13:ctx-8b98b02f) Process host VM state report from ping process. host: 1
  339. 2015-02-23 11:55:14,716 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-13:ctx-8b98b02f) Process VM state report. host: 1, number of records in report: 1
  340. 2015-02-23 11:55:14,716 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-13:ctx-8b98b02f) VM state report. host: 1, vm id: 1, power state: PowerOn
  341. 2015-02-23 11:55:14,719 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-13:ctx-8b98b02f) VM power state does not change, skip DB writing. vm id: 1
  342. 2015-02-23 11:55:14,721 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-13:ctx-8b98b02f) Done with process of VM state report. host: 1
  343. 2015-02-23 11:55:17,836 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgentCronJob-26:ctx-0b3c60c3) Seq 1-3355181722391019524: Executing request
  344. 2015-02-23 11:55:17,963 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgentCronJob-26:ctx-0b3c60c3) Seq 1-3355181722391019524: Response Received:
  345. 2015-02-23 11:55:17,964 DEBUG [c.c.a.t.Request] (DirectAgentCronJob-26:ctx-0b3c60c3) Seq 1-3355181722391019524: Processing: { Ans: , MgmtId: 345050411715, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.ClusterVMMetaDataSyncAnswer":{"_clusterId":1,"_vmMetaDatum":{"v-1-VM":"viridian:true;acpi:1;apic:true;pae:true;nx:true"},"_isExecuted":false,"result":true,"wait":0}}] }
  346. 2015-02-23 11:55:19,249 DEBUG [c.c.c.ConsoleProxyManagerImpl] (consoleproxy-1:ctx-ff1c11e3) Zone 1 is ready to launch console proxy
  347. 2015-02-23 11:55:19,779 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-8bce787f) Zone 1 is ready to launch secondary storage VM
  348. 2015-02-23 11:55:19,789 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:ctx-8bce787f) No running secondary storage vms found in datacenter id=1, starting one
  349. 2015-02-23 11:55:19,797 INFO [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-8bce787f) Found a stopped secondary storage vm, bring it up to running pool. secStorageVm vm id : 17
  350. 2015-02-23 11:55:19,810 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (secstorage-1:ctx-8bce787f) Sync job-104 execution on object VmWorkJobQueue.17
  351. 2015-02-23 11:55:20,806 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-014d720a) Execute sync-queue item: SyncQueueItemVO {id:66, queueId: 65, contentType: AsyncJob, contentId: 104, lastProcessMsid: null, lastprocessNumber: null, lastProcessTime: null, created: Mon Feb 23 11:55:19 CST 2015}
  352. 2015-02-23 11:55:20,808 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-014d720a) Schedule queued job-104
  353. 2015-02-23 11:55:20,824 INFO [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104) Add job-104 into job monitoring
  354. 2015-02-23 11:55:20,824 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104) Executing AsyncJobVO {id:104, userId: 1, accountId: 1, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo: rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAEXQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 345050411715, completeMsid: null, lastUpdated: null, lastPolled: null, created: Mon Feb 23 11:55:19 CST 2015}
  355. 2015-02-23 11:55:20,824 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104) Run VM work job: com.cloud.vm.VmWorkStart for VM 17, job origin: 2
  356. 2015-02-23 11:55:20,826 DEBUG [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Execute VM work job: com.cloud.vm.VmWorkStart{"dcId":0,"userId":1,"accountId":1,"vmId":17,"handlerName":"VirtualMachineManagerImpl"}
  357. 2015-02-23 11:55:20,831 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-d576c342) Begin cleanup expired async-jobs
  358. 2015-02-23 11:55:20,839 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) VM state transitted from :Stopped to Starting with event: StartRequestedvm's original host id: null new host id: null host id before state transition: null
  359. 2015-02-23 11:55:20,839 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Successfully transitioned to start state for VM[SecondaryStorageVm|s-17-VM] reservation id = a899c48b-111c-4bbf-9bce-112f87b36da7
  360. 2015-02-23 11:55:20,839 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-d576c342) End cleanup expired async-jobs
  361. 2015-02-23 11:55:20,857 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Trying to deploy VM, vm has dcId: 1 and podId: 1
  362. 2015-02-23 11:55:20,857 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Deploy avoids pods: null, clusters: null, hosts: null
  363. 2015-02-23 11:55:20,862 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Root volume is ready, need to place VM in volume's cluster
  364. 2015-02-23 11:55:20,862 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Vol[17|vm=17|ROOT] is READY, changing deployment plan to use this pool's dcId: 1 , podId: 1 , and clusterId: 1
  365. 2015-02-23 11:55:20,865 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Deploy avoids pods: null, clusters: null, hosts: null
  366. 2015-02-23 11:55:20,867 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) DeploymentPlanner allocation algorithm: com.cloud.deploy.FirstFitPlanner@7ace2d66
  367. 2015-02-23 11:55:20,867 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Trying to allocate a host and storage pools from dc:1, pod:1,cluster:1, requested cpu: 500, requested ram: 536870912
  368. 2015-02-23 11:55:20,867 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Is ROOT volume READY (pool already allocated)?: Yes
  369. 2015-02-23 11:55:20,867 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Searching resources only under specified Cluster: 1
  370. 2015-02-23 11:55:20,878 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Checking resources in Cluster: 1 under Pod: 1
  371. 2015-02-23 11:55:20,878 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Looking for hosts in dc: 1 pod:1 cluster:1
  372. 2015-02-23 11:55:20,882 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) FirstFitAllocator has 2 hosts to check for allocation: [Host[-1-Routing], Host[-2-Routing]]
  373. 2015-02-23 11:55:20,898 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Found 2 hosts for allocation after prioritization: [Host[-1-Routing], Host[-2-Routing]]
  374. 2015-02-23 11:55:20,898 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Looking for speed=500Mhz, Ram=512
  375. 2015-02-23 11:55:20,909 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Host: 1 has cpu capability (cpu:24, speed:2000) to support requested CPU: 1 and requested speed: 500
  376. 2015-02-23 11:55:20,910 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Checking if host: 1 has enough capacity for requested CPU: 500 and requested RAM: 536870912 , cpuOverprovisioningFactor: 1.0
  377. 2015-02-23 11:55:20,913 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Hosts's actual total CPU: 48000 and CPU after applying overprovisioning: 48000
  378. 2015-02-23 11:55:20,913 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Free CPU: 47500 , Requested CPU: 500
  379. 2015-02-23 11:55:20,913 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Free RAM: 127713402880 , Requested RAM: 536870912
  380. 2015-02-23 11:55:20,913 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Host has enough CPU and RAM available
  381. 2015-02-23 11:55:20,913 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) STATS: Can alloc CPU from host: 1, used: 500, reserved: 0, actual total: 48000, total with overprovisioning: 48000; requested cpu:500,alloc_from_last_host?:false ,considerReservedCapacity?: true
  382. 2015-02-23 11:55:20,913 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) STATS: Can alloc MEM from host: 1, used: 1073741824, reserved: 0, total: 128787144704; requested mem: 536870912,alloc_from_last_host?:false ,considerReservedCapacity?: true
  383. 2015-02-23 11:55:20,913 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Found a suitable host, adding to list: 1
  384. 2015-02-23 11:55:20,920 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgentCronJob-7:ctx-74bf4eb5) Ping from 2(Flex-Xen1.flexhost.local)
  385. 2015-02-23 11:55:20,920 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-7:ctx-74bf4eb5) Process host VM state report from ping process. host: 2
  386. 2015-02-23 11:55:20,921 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-7:ctx-74bf4eb5) Process VM state report. host: 2, number of records in report: 0
  387. 2015-02-23 11:55:20,922 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-7:ctx-74bf4eb5) Done with process of VM state report. host: 2
  388. 2015-02-23 11:55:20,924 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Host: 2 has cpu capability (cpu:24, speed:2000) to support requested CPU: 1 and requested speed: 500
  389. 2015-02-23 11:55:20,924 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Checking if host: 2 has enough capacity for requested CPU: 500 and requested RAM: 536870912 , cpuOverprovisioningFactor: 1.0
  390. 2015-02-23 11:55:20,928 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Hosts's actual total CPU: 48000 and CPU after applying overprovisioning: 48000
  391. 2015-02-23 11:55:20,928 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Free CPU: 48000 , Requested CPU: 500
  392. 2015-02-23 11:55:20,928 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Free RAM: 128787136512 , Requested RAM: 536870912
  393. 2015-02-23 11:55:20,928 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Host has enough CPU and RAM available
  394. 2015-02-23 11:55:20,928 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) STATS: Can alloc CPU from host: 2, used: 0, reserved: 0, actual total: 48000, total with overprovisioning: 48000; requested cpu:500,alloc_from_last_host?:false ,considerReservedCapacity?: true
  395. 2015-02-23 11:55:20,928 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) STATS: Can alloc MEM from host: 2, used: 0, reserved: 0, total: 128787136512; requested mem: 536870912,alloc_from_last_host?:false ,considerReservedCapacity?: true
  396. 2015-02-23 11:55:20,928 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Found a suitable host, adding to list: 2
  397. 2015-02-23 11:55:20,928 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Host Allocator returning 2 suitable hosts
  398. 2015-02-23 11:55:20,932 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Checking suitable pools for volume (Id, Type): (17,ROOT)
  399. 2015-02-23 11:55:20,932 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Volume has pool already allocated, checking if pool can be reused, poolId: 1
  400. 2015-02-23 11:55:20,937 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Planner need not allocate a pool for this volume since its READY
  401. 2015-02-23 11:55:20,937 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Trying to find a potenial host and associated storage pools from the suitable host/pool lists for this VM
  402. 2015-02-23 11:55:20,937 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Checking if host: 1 can access any suitable storage pool for volume: ROOT
  403. 2015-02-23 11:55:20,938 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Host: 1 can access pool: 1
  404. 2015-02-23 11:55:20,940 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Found a potential host id: 1 name: Flex-Xen2.flexhost.local and associated storage pools for this VM
  405. 2015-02-23 11:55:20,942 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Returning Deployment Destination: Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))] : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
  406. 2015-02-23 11:55:20,942 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Deployment found - P0=VM[SecondaryStorageVm|s-17-VM], P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))] : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(1)-Storage()]
  407. 2015-02-23 11:55:20,972 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) VM state transitted from :Starting to Starting with event: OperationRetryvm's original host id: null new host id: 1 host id before state transition: null
  408. 2015-02-23 11:55:20,986 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Hosts's actual total CPU: 48000 and CPU after applying overprovisioning: 48000
  409. 2015-02-23 11:55:20,986 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) We are allocating VM, increasing the used capacity of this host:1
  410. 2015-02-23 11:55:20,986 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Current Used CPU: 500 , Free CPU:47500 ,Requested CPU: 500
  411. 2015-02-23 11:55:20,987 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Current Used RAM: 1073741824 , Free RAM:127713402880 ,Requested RAM: 536870912
  412. 2015-02-23 11:55:20,987 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) CPU STATS after allocation: for host: 1, old used: 500, old reserved: 0, actual total: 48000, total with overprovisioning: 48000; new used:1000, reserved:0; requested cpu:500,alloc_from_last:false
  413. 2015-02-23 11:55:20,987 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) RAM STATS after allocation: for host: 1, old used: 1073741824, old reserved: 0, total: 128787144704; new used: 1610612736, reserved: 0; requested mem: 536870912,alloc_from_last:false
  414. 2015-02-23 11:55:21,017 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) VM is being created in podId: 1
  415. 2015-02-23 11:55:21,023 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Network id=200 is already implemented
  416. 2015-02-23 11:55:21,060 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Network id=202 is already implemented
  417. 2015-02-23 11:55:21,121 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Network id=201 is already implemented
  418. 2015-02-23 11:55:21,156 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Releasing ip address for instance=66
  419. 2015-02-23 11:55:21,172 DEBUG [c.c.n.g.PodBasedNetworkGuru] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Allocated a nic NicProfile[66-17-a899c48b-111c-4bbf-9bce-112f87b36da7-10.90.2.210-null for VM[SecondaryStorageVm|s-17-VM]
  420. 2015-02-23 11:55:21,198 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Network id=203 is already implemented
  421. 2015-02-23 11:55:21,248 DEBUG [c.c.n.g.StorageNetworkGuru] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Allocated a storage nic NicProfile[67-17-a899c48b-111c-4bbf-9bce-112f87b36da7-10.83.2.204-storage://832 for VM[SecondaryStorageVm|s-17-VM]
  422. 2015-02-23 11:55:21,266 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Checking if we need to prepare 1 volumes for VM[SecondaryStorageVm|s-17-VM]
  423. 2015-02-23 11:55:21,268 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) No need to recreate the volume: Vol[17|vm=17|ROOT], since it already has a pool assigned: 1, adding disk to VM
  424. 2015-02-23 11:55:21,297 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) VmWare hypervisor configured, telling the ssvm to load the PremiumSecondaryStorageResource
  425. 2015-02-23 11:55:21,302 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Boot Args for VM[SecondaryStorageVm|s-17-VM]: template=domP type=secstorage host=10.90.1.21 port=8250 name=s-17-VM zone=1 pod=1 guid=s-17-VM workers=5 resource=com.cloud.storage.resource.PremiumSecondaryStorageResource instance=SecStorage sslcopy=false role=templateProcessor mtu=1500 eth2ip=205.185.158.69 eth2mask=255.255.255.192 gateway=205.185.158.65 public.network.device=eth2 eth0ip=169.254.1.64 eth0mask=255.255.0.0 eth1ip=10.90.2.210 eth1mask=255.255.255.0 mgmtcidr=10.90.1.0/24 localgw=10.90.2.254 private.network.device=eth1 eth3ip=10.83.2.204 eth3mask=255.255.255.0 storageip=10.83.2.204 storagenetmask=255.255.255.0 storagegateway=10.83.2.1 internaldns1=10.90.2.41 internaldns2=10.90.2.42 dns1=208.74.240.5 dns2=208.74.247.245
  426. 2015-02-23 11:55:21,395 DEBUG [c.c.a.t.Request] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Seq 1-3355181722391019620: Sending { Cmd , MgmtId: 345050411715, via: 1(Flex-Xen2.flexhost.local), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StartCommand":{"vm":{"id":17,"name":"s-17-VM","bootloader":"PyGrub","type":"SecondaryStorageVm","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian GNU/Linux 7(64-bit)","platformEmulator":"Debian Wheezy 7.0 (64-bit)","bootArgs":" template=domP type=secstorage host=10.90.1.21 port=8250 name=s-17-VM zone=1 pod=1 guid=s-17-VM workers=5 resource=com.cloud.storage.resource.PremiumSecondaryStorageResource instance=SecStorage sslcopy=false role=templateProcessor mtu=1500 eth2ip=205.185.158.69 eth2mask=255.255.255.192 gateway=205.185.158.65 public.network.device=eth2 eth0ip=169.254.1.64 eth0mask=255.255.0.0 eth1ip=10.90.2.210 eth1mask=255.255.255.0 mgmtcidr=10.90.1.0/24 localgw=10.90.2.254 private.network.device=eth1 eth3ip=10.83.2.204 eth3mask=255.255.255.0 storageip=10.83.2.204 storagenetmask=255.255.255.0 storagegateway=10.83.2.1 internaldns1=10.90.2.41 internaldns2=10.90.2.42 dns1=208.74.240.5 dns2=208.74.247.245","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"cUF2OGhYFGZtCrttlpKhiw==","params":{},"uuid":"e00bed4f-47fe-42cb-830e-4c70a3bc2546","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"73b9fc39-35b0-46eb-bee6-1e52fc4f9b7e","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"FlexSAN1-LUN0","id":1,"poolType":"PreSetup","host":"localhost","path":"/FlexSAN1-LUN0","port":0,"url":"PreSetup://localhost/FlexSAN1-LUN0/?ROLE=Primary&STOREUUID=FlexSAN1-LUN0"}},"name":"ROOT-17","size":2684354560,"path":"850139ef-2fe6-4118-9486-c6486dcc798a","volumeId":17,"vmName":"s-17-VM","accountId":1,"format":"VHD","provisioningType":"THIN","id":17,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"850139ef-2fe6-4118-9486-c6486dcc798a","type":"ROOT","_details":{"managed":"false","storagePort":"0","storageHost":"localhost","volumeSize":"2684354560"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"4af99ea2-0f8d-42df-815c-c524a7847e5a","uuid":"bc1ef7ea-19c0-41f1-827d-fc556a73df48","ip":"205.185.158.69","netmask":"255.255.255.192","gateway":"205.185.158.65","mac":"06:29:0e:00:00:0c","dns1":"208.74.240.5","dns2":"208.74.247.245","broadcastType":"Vlan","type":"Public","broadcastUri":"vlan://1755","isolationUri":"vlan://1755","isSecurityGroupEnabled":false,"name":"GUEST-PUB"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"c4469cb3-9f89-44cc-9e3c-cf7f6cbc0f6e","uuid":"d418b3c2-2122-42ca-93df-5f18ccd5274b","ip":"169.254.1.64","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:40","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"a6398c49-3400-46c6-8bcc-319e276b1425","uuid":"5eaf68eb-a5d3-4d9c-ab71-513926e4d58f","ip":"10.90.2.210","netmask":"255.255.255.0","gateway":"10.90.2.254","mac":"06:77:fc:00:00:0a","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"MGMT"},{"deviceId":3,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"5ad17f51-f91a-4111-a699-7961637dce94","uuid":"3c76f7d5-db9c-4a42-8fc7-9663c8ed2728","ip":"10.83.2.204","netmask":"255.255.255.0","gateway":"10.83.2.1","mac":"06:b5:ca:00:00:49","broadcastType":"Storage","type":"Storage","broadcastUri":"storage://832","isSecurityGroupEnabled":false,"name":"SEC-STORE"}]},"hostIp":"10.90.2.112","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.1.64","port":3922,"interval":6,"retries":100,"name":"s-17-VM","wait":0}}] }
  427. 2015-02-23 11:55:21,396 DEBUG [c.c.a.t.Request] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Seq 1-3355181722391019620: Executing: { Cmd , MgmtId: 345050411715, via: 1(Flex-Xen2.flexhost.local), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StartCommand":{"vm":{"id":17,"name":"s-17-VM","bootloader":"PyGrub","type":"SecondaryStorageVm","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian GNU/Linux 7(64-bit)","platformEmulator":"Debian Wheezy 7.0 (64-bit)","bootArgs":" template=domP type=secstorage host=10.90.1.21 port=8250 name=s-17-VM zone=1 pod=1 guid=s-17-VM workers=5 resource=com.cloud.storage.resource.PremiumSecondaryStorageResource instance=SecStorage sslcopy=false role=templateProcessor mtu=1500 eth2ip=205.185.158.69 eth2mask=255.255.255.192 gateway=205.185.158.65 public.network.device=eth2 eth0ip=169.254.1.64 eth0mask=255.255.0.0 eth1ip=10.90.2.210 eth1mask=255.255.255.0 mgmtcidr=10.90.1.0/24 localgw=10.90.2.254 private.network.device=eth1 eth3ip=10.83.2.204 eth3mask=255.255.255.0 storageip=10.83.2.204 storagenetmask=255.255.255.0 storagegateway=10.83.2.1 internaldns1=10.90.2.41 internaldns2=10.90.2.42 dns1=208.74.240.5 dns2=208.74.247.245","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"cUF2OGhYFGZtCrttlpKhiw==","params":{},"uuid":"e00bed4f-47fe-42cb-830e-4c70a3bc2546","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"73b9fc39-35b0-46eb-bee6-1e52fc4f9b7e","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"FlexSAN1-LUN0","id":1,"poolType":"PreSetup","host":"localhost","path":"/FlexSAN1-LUN0","port":0,"url":"PreSetup://localhost/FlexSAN1-LUN0/?ROLE=Primary&STOREUUID=FlexSAN1-LUN0"}},"name":"ROOT-17","size":2684354560,"path":"850139ef-2fe6-4118-9486-c6486dcc798a","volumeId":17,"vmName":"s-17-VM","accountId":1,"format":"VHD","provisioningType":"THIN","id":17,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"850139ef-2fe6-4118-9486-c6486dcc798a","type":"ROOT","_details":{"managed":"false","storagePort":"0","storageHost":"localhost","volumeSize":"2684354560"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"4af99ea2-0f8d-42df-815c-c524a7847e5a","uuid":"bc1ef7ea-19c0-41f1-827d-fc556a73df48","ip":"205.185.158.69","netmask":"255.255.255.192","gateway":"205.185.158.65","mac":"06:29:0e:00:00:0c","dns1":"208.74.240.5","dns2":"208.74.247.245","broadcastType":"Vlan","type":"Public","broadcastUri":"vlan://1755","isolationUri":"vlan://1755","isSecurityGroupEnabled":false,"name":"GUEST-PUB"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"c4469cb3-9f89-44cc-9e3c-cf7f6cbc0f6e","uuid":"d418b3c2-2122-42ca-93df-5f18ccd5274b","ip":"169.254.1.64","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:40","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"a6398c49-3400-46c6-8bcc-319e276b1425","uuid":"5eaf68eb-a5d3-4d9c-ab71-513926e4d58f","ip":"10.90.2.210","netmask":"255.255.255.0","gateway":"10.90.2.254","mac":"06:77:fc:00:00:0a","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"MGMT"},{"deviceId":3,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"5ad17f51-f91a-4111-a699-7961637dce94","uuid":"3c76f7d5-db9c-4a42-8fc7-9663c8ed2728","ip":"10.83.2.204","netmask":"255.255.255.0","gateway":"10.83.2.1","mac":"06:b5:ca:00:00:49","broadcastType":"Storage","type":"Storage","broadcastUri":"storage://832","isSecurityGroupEnabled":false,"name":"SEC-STORE"}]},"hostIp":"10.90.2.112","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.1.64","port":3922,"interval":6,"retries":100,"name":"s-17-VM","wait":0}}] }
  428. 2015-02-23 11:55:21,396 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-48:ctx-e5df0748) Seq 1-3355181722391019620: Executing request
  429. 2015-02-23 11:55:21,402 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) 1. The VM s-17-VM is in Starting state.
  430. 2015-02-23 11:55:21,431 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) Created VM 68ef45f1-9dda-862b-c78e-6e5b9443c30e for s-17-VM
  431. 2015-02-23 11:55:21,437 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) PV args are -- quiet console=hvc0%template=domP%type=secstorage%host=10.90.1.21%port=8250%name=s-17-VM%zone=1%pod=1%guid=s-17-VM%workers=5%resource=com.cloud.storage.resource.PremiumSecondaryStorageResource%instance=SecStorage%sslcopy=false%role=templateProcessor%mtu=1500%eth2ip=205.185.158.69%eth2mask=255.255.255.192%gateway=205.185.158.65%public.network.device=eth2%eth0ip=169.254.1.64%eth0mask=255.255.0.0%eth1ip=10.90.2.210%eth1mask=255.255.255.0%mgmtcidr=10.90.1.0/24%localgw=10.90.2.254%private.network.device=eth1%eth3ip=10.83.2.204%eth3mask=255.255.255.0%storageip=10.83.2.204%storagenetmask=255.255.255.0%storagegateway=10.83.2.1%internaldns1=10.90.2.41%internaldns2=10.90.2.42%dns1=208.74.240.5%dns2=208.74.247.245
  432. 2015-02-23 11:55:21,474 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) VBD 4f1fc343-bde4-ea96-9782-f1a074865a29 created for com.cloud.agent.api.to.DiskTO@123474b6
  433. 2015-02-23 11:55:21,497 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) Creating VIF for s-17-VM on nic [Nic:Public-205.185.158.69-vlan://1755]
  434. 2015-02-23 11:55:21,499 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) Looking for network named GUEST-PUB
  435. 2015-02-23 11:55:21,513 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) Found a network called GUEST-PUB on host=10.90.2.112; Network=644f87cd-a11e-e94b-f3d1-7e221679ded2; pif=d9aeaabc-8989-8a9d-9836-8e499abbce58
  436. 2015-02-23 11:55:21,523 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) Found a network called VLAN-644f87cd-a11e-e94b-f3d1-7e221679ded2-1755 on host=10.90.2.112; Network=e379b22c-3ac8-a0de-6482-40df647fb16f; pif=9c0e17e2-ff0a-df28-6445-376cc3c45b28
  437. 2015-02-23 11:55:21,537 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) Created a vif 829a2677-52f0-f25c-62ae-73864f6c82e7 on 2
  438. 2015-02-23 11:55:21,537 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) Creating VIF for s-17-VM on nic [Nic:Control-169.254.1.64-null]
  439. 2015-02-23 11:55:21,556 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) already have a vif on dom0 for link local network
  440. 2015-02-23 11:55:21,730 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) Created a vif ace64a91-68e4-1219-fab0-693319f61584 on 0
  441. 2015-02-23 11:55:21,730 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) Creating VIF for s-17-VM on nic [Nic:Management-10.90.2.210-null]
  442. 2015-02-23 11:55:21,732 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) Looking for network named MGMT
  443. 2015-02-23 11:55:21,745 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) Created a vif aa56c514-63ee-e673-df7d-fe1b8642f5dd on 1
  444. 2015-02-23 11:55:21,745 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) Creating VIF for s-17-VM on nic [Nic:Storage-10.83.2.204-storage://832]
  445. 2015-02-23 11:55:21,747 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) Looking for network named SEC-STORE
  446. 2015-02-23 11:55:21,757 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) Found a network called SEC-STORE on host=10.90.2.112; Network=c9754adc-0015-88cf-8b40-b6eceed12b94; pif=e1f60848-48fc-d77c-ec12-7fff89141754
  447. 2015-02-23 11:55:21,759 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) Creating VLAN 832 on host 10.90.2.112 on device bond1
  448. 2015-02-23 11:55:21,768 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) Catch Exception: class com.xensource.xenapi.Types$PifIsVlan due to You tried to create a VLAN on top of another VLAN - use the underlying physical PIF/bond instead
  449. You tried to create a VLAN on top of another VLAN - use the underlying physical PIF/bond instead
  450. at com.xensource.xenapi.Types.checkResponse(Types.java:2003)
  451. at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
  452. at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:465)
  453. at com.xensource.xenapi.VLAN.create(VLAN.java:363)
  454. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.enableVlanNetwork(CitrixResourceBase.java:4026)
  455. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getNetwork(CitrixResourceBase.java:1043)
  456. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.createVif(CitrixResourceBase.java:1103)
  457. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1780)
  458. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:489)
  459. at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
  460. at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
  461. at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
  462. at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:304)
  463. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  464. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  465. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  466. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  467. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  468. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  469. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  470. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
  471. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
  472. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  473. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  474. at java.lang.Thread.run(Thread.java:745)
  475. 2015-02-23 11:55:21,770 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) Unable to start s-17-VM due to
  476. You tried to create a VLAN on top of another VLAN - use the underlying physical PIF/bond instead
  477. at com.xensource.xenapi.Types.checkResponse(Types.java:2003)
  478. at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
  479. at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:465)
  480. at com.xensource.xenapi.VLAN.create(VLAN.java:363)
  481. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.enableVlanNetwork(CitrixResourceBase.java:4026)
  482. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getNetwork(CitrixResourceBase.java:1043)
  483. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.createVif(CitrixResourceBase.java:1103)
  484. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1780)
  485. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:489)
  486. at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
  487. at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
  488. at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
  489. at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:304)
  490. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  491. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  492. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  493. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  494. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  495. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  496. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  497. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
  498. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
  499. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  500. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  501. at java.lang.Thread.run(Thread.java:745)
  502. 2015-02-23 11:55:21,808 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) Unable to clean up VBD due to
  503. You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
  504. at com.xensource.xenapi.Types.checkResponse(Types.java:693)
  505. at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
  506. at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:465)
  507. at com.xensource.xenapi.VBD.unplug(VBD.java:1109)
  508. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1512)
  509. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1863)
  510. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:489)
  511. at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
  512. at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
  513. at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
  514. at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:304)
  515. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  516. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  517. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  518. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  519. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  520. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  521. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  522. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
  523. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
  524. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  525. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  526. at java.lang.Thread.run(Thread.java:745)
  527. 2015-02-23 11:55:21,813 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) Unable to clean up VBD due to
  528. You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
  529. at com.xensource.xenapi.Types.checkResponse(Types.java:693)
  530. at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
  531. at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:465)
  532. at com.xensource.xenapi.VBD.unplug(VBD.java:1109)
  533. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1512)
  534. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1863)
  535. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:489)
  536. at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
  537. at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
  538. at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
  539. at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:304)
  540. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  541. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  542. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  543. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  544. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  545. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  546. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  547. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
  548. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
  549. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  550. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  551. at java.lang.Thread.run(Thread.java:745)
  552. 2015-02-23 11:55:21,818 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) Unable to cleanup VIF
  553. You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
  554. at com.xensource.xenapi.Types.checkResponse(Types.java:693)
  555. at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
  556. at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:465)
  557. at com.xensource.xenapi.VIF.unplug(VIF.java:921)
  558. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1520)
  559. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1863)
  560. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:489)
  561. at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
  562. at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
  563. at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
  564. at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:304)
  565. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  566. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  567. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  568. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  569. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  570. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  571. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  572. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
  573. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
  574. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  575. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  576. at java.lang.Thread.run(Thread.java:745)
  577. 2015-02-23 11:55:21,822 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) Unable to cleanup VIF
  578. You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
  579. at com.xensource.xenapi.Types.checkResponse(Types.java:693)
  580. at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
  581. at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:465)
  582. at com.xensource.xenapi.VIF.unplug(VIF.java:921)
  583. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1520)
  584. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1863)
  585. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:489)
  586. at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
  587. at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
  588. at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
  589. at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:304)
  590. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  591. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  592. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  593. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  594. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  595. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  596. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  597. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
  598. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
  599. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  600. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  601. at java.lang.Thread.run(Thread.java:745)
  602. 2015-02-23 11:55:21,827 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) Unable to cleanup VIF
  603. You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
  604. at com.xensource.xenapi.Types.checkResponse(Types.java:693)
  605. at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
  606. at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:465)
  607. at com.xensource.xenapi.VIF.unplug(VIF.java:921)
  608. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1520)
  609. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1863)
  610. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:489)
  611. at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
  612. at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
  613. at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
  614. at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:304)
  615. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  616. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  617. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  618. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  619. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  620. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  621. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  622. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
  623. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
  624. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  625. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  626. at java.lang.Thread.run(Thread.java:745)
  627. 2015-02-23 11:55:21,849 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-48:ctx-e5df0748) The VM is in stopped state, detected problem during startup : s-17-VM
  628. 2015-02-23 11:55:21,849 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-48:ctx-e5df0748) Seq 1-3355181722391019620: Cancelling because one of the answers is false and it is stop on error.
  629. 2015-02-23 11:55:21,849 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-48:ctx-e5df0748) Seq 1-3355181722391019620: Response Received:
  630. 2015-02-23 11:55:21,851 DEBUG [c.c.a.t.Request] (DirectAgent-48:ctx-e5df0748) Seq 1-3355181722391019620: Processing: { Ans: , MgmtId: 345050411715, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":17,"name":"s-17-VM","bootloader":"PyGrub","type":"SecondaryStorageVm","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian GNU/Linux 7(64-bit)","platformEmulator":"Debian Wheezy 7.0 (64-bit)","bootArgs":" template=domP type=secstorage host=10.90.1.21 port=8250 name=s-17-VM zone=1 pod=1 guid=s-17-VM workers=5 resource=com.cloud.storage.resource.PremiumSecondaryStorageResource instance=SecStorage sslcopy=false role=templateProcessor mtu=1500 eth2ip=205.185.158.69 eth2mask=255.255.255.192 gateway=205.185.158.65 public.network.device=eth2 eth0ip=169.254.1.64 eth0mask=255.255.0.0 eth1ip=10.90.2.210 eth1mask=255.255.255.0 mgmtcidr=10.90.1.0/24 localgw=10.90.2.254 private.network.device=eth1 eth3ip=10.83.2.204 eth3mask=255.255.255.0 storageip=10.83.2.204 storagenetmask=255.255.255.0 storagegateway=10.83.2.1 internaldns1=10.90.2.41 internaldns2=10.90.2.42 dns1=208.74.240.5 dns2=208.74.247.245","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"cUF2OGhYFGZtCrttlpKhiw==","params":{},"uuid":"e00bed4f-47fe-42cb-830e-4c70a3bc2546","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"73b9fc39-35b0-46eb-bee6-1e52fc4f9b7e","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"FlexSAN1-LUN0","id":1,"poolType":"PreSetup","host":"localhost","path":"/FlexSAN1-LUN0","port":0,"url":"PreSetup://localhost/FlexSAN1-LUN0/?ROLE=Primary&STOREUUID=FlexSAN1-LUN0"}},"name":"ROOT-17","size":2684354560,"path":"850139ef-2fe6-4118-9486-c6486dcc798a","volumeId":17,"vmName":"s-17-VM","accountId":1,"format":"VHD","provisioningType":"THIN","id":17,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"850139ef-2fe6-4118-9486-c6486dcc798a","type":"ROOT","_details":{"managed":"false","storagePort":"0","storageHost":"localhost","volumeSize":"2684354560"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"4af99ea2-0f8d-42df-815c-c524a7847e5a","uuid":"bc1ef7ea-19c0-41f1-827d-fc556a73df48","ip":"205.185.158.69","netmask":"255.255.255.192","gateway":"205.185.158.65","mac":"06:29:0e:00:00:0c","dns1":"208.74.240.5","dns2":"208.74.247.245","broadcastType":"Vlan","type":"Public","broadcastUri":"vlan://1755","isolationUri":"vlan://1755","isSecurityGroupEnabled":false,"name":"GUEST-PUB"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"c4469cb3-9f89-44cc-9e3c-cf7f6cbc0f6e","uuid":"d418b3c2-2122-42ca-93df-5f18ccd5274b","ip":"169.254.1.64","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:01:40","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"a6398c49-3400-46c6-8bcc-319e276b1425","uuid":"5eaf68eb-a5d3-4d9c-ab71-513926e4d58f","ip":"10.90.2.210","netmask":"255.255.255.0","gateway":"10.90.2.254","mac":"06:77:fc:00:00:0a","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"MGMT"},{"deviceId":3,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"5ad17f51-f91a-4111-a699-7961637dce94","uuid":"3c76f7d5-db9c-4a42-8fc7-9663c8ed2728","ip":"10.83.2.204","netmask":"255.255.255.0","gateway":"10.83.2.1","mac":"06:b5:ca:00:00:49","broadcastType":"Storage","type":"Storage","broadcastUri":"storage://832","isSecurityGroupEnabled":false,"name":"SEC-STORE"}]},"_iqnToPath":{},"result":false,"details":"Unable to start s-17-VM due to ","wait":0}}] }
  631. 2015-02-23 11:55:21,851 DEBUG [c.c.a.t.Request] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Seq 1-3355181722391019620: Received: { Ans: , MgmtId: 345050411715, via: 1, Ver: v1, Flags: 10, { StartAnswer } }
  632. 2015-02-23 11:55:21,869 INFO [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Unable to start VM on Host[-1-Routing] due to Unable to start s-17-VM due to
  633. 2015-02-23 11:55:21,883 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Cleaning up resources for the vm VM[SecondaryStorageVm|s-17-VM] in Starting state
  634. 2015-02-23 11:55:21,889 DEBUG [c.c.a.t.Request] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Seq 1-3355181722391019621: Sending { Cmd , MgmtId: 345050411715, via: 1(Flex-Xen2.flexhost.local), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"s-17-VM","wait":0}}] }
  635. 2015-02-23 11:55:21,889 DEBUG [c.c.a.t.Request] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Seq 1-3355181722391019621: Executing: { Cmd , MgmtId: 345050411715, via: 1(Flex-Xen2.flexhost.local), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"s-17-VM","wait":0}}] }
  636. 2015-02-23 11:55:21,890 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-96:ctx-7cfc61ea) Seq 1-3355181722391019621: Executing request
  637. 2015-02-23 11:55:21,900 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-96:ctx-7cfc61ea) Seq 1-3355181722391019621: Response Received:
  638. 2015-02-23 11:55:21,900 DEBUG [c.c.a.t.Request] (DirectAgent-96:ctx-7cfc61ea) Seq 1-3355181722391019621: Processing: { Ans: , MgmtId: 345050411715, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StopAnswer":{"result":true,"details":"VM does not exist","wait":0}}] }
  639. 2015-02-23 11:55:21,900 DEBUG [c.c.a.t.Request] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Seq 1-3355181722391019621: Received: { Ans: , MgmtId: 345050411715, via: 1, Ver: v1, Flags: 10, { StopAnswer } }
  640. 2015-02-23 11:55:21,921 DEBUG [c.c.n.g.ControlNetworkGuru] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Released nic: NicProfile[65-17-null-null-null
  641. 2015-02-23 11:55:21,942 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Releasing ip address for reservationId=a899c48b-111c-4bbf-9bce-112f87b36da7, instance=66
  642. 2015-02-23 11:55:21,942 DEBUG [c.c.n.g.PodBasedNetworkGuru] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Released nic: NicProfile[66-17-null-null-null
  643. 2015-02-23 11:55:21,965 DEBUG [c.c.n.g.StorageNetworkGuru] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Release an storage ip 10.83.2.204
  644. 2015-02-23 11:55:21,977 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Successfully released network resources for the vm VM[SecondaryStorageVm|s-17-VM]
  645. 2015-02-23 11:55:21,978 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Successfully cleanued up resources for the vm VM[SecondaryStorageVm|s-17-VM] in Starting state
  646. 2015-02-23 11:55:21,981 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Root volume is ready, need to place VM in volume's cluster
  647. 2015-02-23 11:55:21,981 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Vol[17|vm=17|ROOT] is READY, changing deployment plan to use this pool's dcId: 1 , podId: 1 , and clusterId: 1
  648. 2015-02-23 11:55:21,984 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Deploy avoids pods: null, clusters: null, hosts: [1]
  649. 2015-02-23 11:55:21,986 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) DeploymentPlanner allocation algorithm: com.cloud.deploy.FirstFitPlanner@7ace2d66
  650. 2015-02-23 11:55:21,986 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Trying to allocate a host and storage pools from dc:1, pod:1,cluster:1, requested cpu: 500, requested ram: 536870912
  651. 2015-02-23 11:55:21,986 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Is ROOT volume READY (pool already allocated)?: Yes
  652. 2015-02-23 11:55:21,986 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Searching resources only under specified Cluster: 1
  653. 2015-02-23 11:55:21,997 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Checking resources in Cluster: 1 under Pod: 1
  654. 2015-02-23 11:55:21,997 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Looking for hosts in dc: 1 pod:1 cluster:1
  655. 2015-02-23 11:55:22,001 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) FirstFitAllocator has 2 hosts to check for allocation: [Host[-2-Routing], Host[-1-Routing]]
  656. 2015-02-23 11:55:22,010 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Found 2 hosts for allocation after prioritization: [Host[-2-Routing], Host[-1-Routing]]
  657. 2015-02-23 11:55:22,010 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Looking for speed=500Mhz, Ram=512
  658. 2015-02-23 11:55:22,021 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Host: 2 has cpu capability (cpu:24, speed:2000) to support requested CPU: 1 and requested speed: 500
  659. 2015-02-23 11:55:22,021 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Checking if host: 2 has enough capacity for requested CPU: 500 and requested RAM: 536870912 , cpuOverprovisioningFactor: 1.0
  660. 2015-02-23 11:55:22,024 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Hosts's actual total CPU: 48000 and CPU after applying overprovisioning: 48000
  661. 2015-02-23 11:55:22,024 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Free CPU: 48000 , Requested CPU: 500
  662. 2015-02-23 11:55:22,024 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Free RAM: 128787136512 , Requested RAM: 536870912
  663. 2015-02-23 11:55:22,024 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Host has enough CPU and RAM available
  664. 2015-02-23 11:55:22,024 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) STATS: Can alloc CPU from host: 2, used: 0, reserved: 0, actual total: 48000, total with overprovisioning: 48000; requested cpu:500,alloc_from_last_host?:false ,considerReservedCapacity?: true
  665. 2015-02-23 11:55:22,024 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) STATS: Can alloc MEM from host: 2, used: 0, reserved: 0, total: 128787136512; requested mem: 536870912,alloc_from_last_host?:false ,considerReservedCapacity?: true
  666. 2015-02-23 11:55:22,024 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Found a suitable host, adding to list: 2
  667. 2015-02-23 11:55:22,024 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Host name: Flex-Xen2.flexhost.local, hostId: 1 is in avoid set, skipping this and trying other available hosts
  668. 2015-02-23 11:55:22,024 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Host Allocator returning 1 suitable hosts
  669. 2015-02-23 11:55:22,028 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Checking suitable pools for volume (Id, Type): (17,ROOT)
  670. 2015-02-23 11:55:22,028 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Volume has pool already allocated, checking if pool can be reused, poolId: 1
  671. 2015-02-23 11:55:22,030 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Planner need not allocate a pool for this volume since its READY
  672. 2015-02-23 11:55:22,030 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Trying to find a potenial host and associated storage pools from the suitable host/pool lists for this VM
  673. 2015-02-23 11:55:22,030 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Checking if host: 2 can access any suitable storage pool for volume: ROOT
  674. 2015-02-23 11:55:22,032 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Host: 2 can access pool: 1
  675. 2015-02-23 11:55:22,033 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Found a potential host id: 2 name: Flex-Xen1.flexhost.local and associated storage pools for this VM
  676. 2015-02-23 11:55:22,035 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Returning Deployment Destination: Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))] : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(2)-Storage()]
  677. 2015-02-23 11:55:22,035 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Deployment found - P0=VM[SecondaryStorageVm|s-17-VM], P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))] : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(2)-Storage()]
  678. 2015-02-23 11:55:22,073 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) VM state transitted from :Starting to Starting with event: OperationRetryvm's original host id: null new host id: 2 host id before state transition: 1
  679. 2015-02-23 11:55:22,083 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Hosts's actual total CPU: 48000 and CPU after applying overprovisioning: 48000
  680. 2015-02-23 11:55:22,083 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Hosts's actual total RAM: 128787145984 and RAM after applying overprovisioning: 128787144704
  681. 2015-02-23 11:55:22,083 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) release cpu from host: 1, old used: 1000,reserved: 0, actual total: 48000, total with overprovisioning: 48000; new used: 500,reserved:0; movedfromreserved: false,moveToReserveredfalse
  682. 2015-02-23 11:55:22,083 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) release mem from host: 1, old used: 1610612736,reserved: 0, total: 128787144704; new used: 1073741824,reserved:0; movedfromreserved: false,moveToReserveredfalse
  683. 2015-02-23 11:55:22,110 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Hosts's actual total CPU: 48000 and CPU after applying overprovisioning: 48000
  684. 2015-02-23 11:55:22,110 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) We are allocating VM, increasing the used capacity of this host:2
  685. 2015-02-23 11:55:22,110 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Current Used CPU: 0 , Free CPU:48000 ,Requested CPU: 500
  686. 2015-02-23 11:55:22,110 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Current Used RAM: 0 , Free RAM:128787136512 ,Requested RAM: 536870912
  687. 2015-02-23 11:55:22,110 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) CPU STATS after allocation: for host: 2, old used: 0, old reserved: 0, actual total: 48000, total with overprovisioning: 48000; new used:500, reserved:0; requested cpu:500,alloc_from_last:false
  688. 2015-02-23 11:55:22,111 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) RAM STATS after allocation: for host: 2, old used: 0, old reserved: 0, total: 128787136512; new used: 536870912, reserved: 0; requested mem: 536870912,alloc_from_last:false
  689. 2015-02-23 11:55:22,124 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) VM is being created in podId: 1
  690. 2015-02-23 11:55:22,130 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Network id=200 is already implemented
  691. 2015-02-23 11:55:22,163 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Network id=202 is already implemented
  692. 2015-02-23 11:55:22,222 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Network id=201 is already implemented
  693. 2015-02-23 11:55:22,257 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Releasing ip address for instance=66
  694. 2015-02-23 11:55:22,274 DEBUG [c.c.n.g.PodBasedNetworkGuru] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Allocated a nic NicProfile[66-17-a899c48b-111c-4bbf-9bce-112f87b36da7-10.90.2.202-null for VM[SecondaryStorageVm|s-17-VM]
  695. 2015-02-23 11:55:22,297 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Network id=203 is already implemented
  696. 2015-02-23 11:55:22,363 DEBUG [c.c.n.g.StorageNetworkGuru] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Allocated a storage nic NicProfile[67-17-a899c48b-111c-4bbf-9bce-112f87b36da7-10.83.2.208-storage://832 for VM[SecondaryStorageVm|s-17-VM]
  697. 2015-02-23 11:55:22,380 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Checking if we need to prepare 1 volumes for VM[SecondaryStorageVm|s-17-VM]
  698. 2015-02-23 11:55:22,381 DEBUG [o.a.c.e.o.VolumeOrchestrator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) No need to recreate the volume: Vol[17|vm=17|ROOT], since it already has a pool assigned: 1, adding disk to VM
  699. 2015-02-23 11:55:22,410 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) VmWare hypervisor configured, telling the ssvm to load the PremiumSecondaryStorageResource
  700. 2015-02-23 11:55:22,415 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Boot Args for VM[SecondaryStorageVm|s-17-VM]: template=domP type=secstorage host=10.90.1.21 port=8250 name=s-17-VM zone=1 pod=1 guid=s-17-VM workers=5 resource=com.cloud.storage.resource.PremiumSecondaryStorageResource instance=SecStorage sslcopy=false role=templateProcessor mtu=1500 eth2ip=205.185.158.69 eth2mask=255.255.255.192 gateway=205.185.158.65 public.network.device=eth2 eth0ip=169.254.3.61 eth0mask=255.255.0.0 eth1ip=10.90.2.202 eth1mask=255.255.255.0 mgmtcidr=10.90.1.0/24 localgw=10.90.2.254 private.network.device=eth1 eth3ip=10.83.2.208 eth3mask=255.255.255.0 storageip=10.83.2.208 storagenetmask=255.255.255.0 storagegateway=10.83.2.1 internaldns1=10.90.2.41 internaldns2=10.90.2.42 dns1=208.74.240.5 dns2=208.74.247.245
  701. 2015-02-23 11:55:22,477 DEBUG [c.c.a.t.Request] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Seq 2-8777234198768386143: Sending { Cmd , MgmtId: 345050411715, via: 2(Flex-Xen1.flexhost.local), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StartCommand":{"vm":{"id":17,"name":"s-17-VM","bootloader":"PyGrub","type":"SecondaryStorageVm","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian GNU/Linux 7(64-bit)","platformEmulator":"Debian Wheezy 7.0 (64-bit)","bootArgs":" template=domP type=secstorage host=10.90.1.21 port=8250 name=s-17-VM zone=1 pod=1 guid=s-17-VM workers=5 resource=com.cloud.storage.resource.PremiumSecondaryStorageResource instance=SecStorage sslcopy=false role=templateProcessor mtu=1500 eth2ip=205.185.158.69 eth2mask=255.255.255.192 gateway=205.185.158.65 public.network.device=eth2 eth0ip=169.254.3.61 eth0mask=255.255.0.0 eth1ip=10.90.2.202 eth1mask=255.255.255.0 mgmtcidr=10.90.1.0/24 localgw=10.90.2.254 private.network.device=eth1 eth3ip=10.83.2.208 eth3mask=255.255.255.0 storageip=10.83.2.208 storagenetmask=255.255.255.0 storagegateway=10.83.2.1 internaldns1=10.90.2.41 internaldns2=10.90.2.42 dns1=208.74.240.5 dns2=208.74.247.245","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"cUF2OGhYFGZtCrttlpKhiw==","params":{},"uuid":"e00bed4f-47fe-42cb-830e-4c70a3bc2546","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"73b9fc39-35b0-46eb-bee6-1e52fc4f9b7e","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"FlexSAN1-LUN0","id":1,"poolType":"PreSetup","host":"localhost","path":"/FlexSAN1-LUN0","port":0,"url":"PreSetup://localhost/FlexSAN1-LUN0/?ROLE=Primary&STOREUUID=FlexSAN1-LUN0"}},"name":"ROOT-17","size":2684354560,"path":"850139ef-2fe6-4118-9486-c6486dcc798a","volumeId":17,"vmName":"s-17-VM","accountId":1,"format":"VHD","provisioningType":"THIN","id":17,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"850139ef-2fe6-4118-9486-c6486dcc798a","type":"ROOT","_details":{"managed":"false","storagePort":"0","storageHost":"localhost","volumeSize":"2684354560"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"4af99ea2-0f8d-42df-815c-c524a7847e5a","uuid":"bc1ef7ea-19c0-41f1-827d-fc556a73df48","ip":"205.185.158.69","netmask":"255.255.255.192","gateway":"205.185.158.65","mac":"06:29:0e:00:00:0c","dns1":"208.74.240.5","dns2":"208.74.247.245","broadcastType":"Vlan","type":"Public","broadcastUri":"vlan://1755","isolationUri":"vlan://1755","isSecurityGroupEnabled":false,"name":"GUEST-PUB"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"c4469cb3-9f89-44cc-9e3c-cf7f6cbc0f6e","uuid":"d418b3c2-2122-42ca-93df-5f18ccd5274b","ip":"169.254.3.61","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:03:3d","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"a6398c49-3400-46c6-8bcc-319e276b1425","uuid":"5eaf68eb-a5d3-4d9c-ab71-513926e4d58f","ip":"10.90.2.202","netmask":"255.255.255.0","gateway":"10.90.2.254","mac":"06:bd:56:00:00:02","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"MGMT"},{"deviceId":3,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"5ad17f51-f91a-4111-a699-7961637dce94","uuid":"3c76f7d5-db9c-4a42-8fc7-9663c8ed2728","ip":"10.83.2.208","netmask":"255.255.255.0","gateway":"10.83.2.1","mac":"06:4f:ce:00:00:4d","broadcastType":"Storage","type":"Storage","broadcastUri":"storage://832","isSecurityGroupEnabled":false,"name":"SEC-STORE"}]},"hostIp":"10.90.2.111","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.3.61","port":3922,"interval":6,"retries":100,"name":"s-17-VM","wait":0}}] }
  702. 2015-02-23 11:55:22,479 DEBUG [c.c.a.t.Request] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Seq 2-8777234198768386143: Executing: { Cmd , MgmtId: 345050411715, via: 2(Flex-Xen1.flexhost.local), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StartCommand":{"vm":{"id":17,"name":"s-17-VM","bootloader":"PyGrub","type":"SecondaryStorageVm","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian GNU/Linux 7(64-bit)","platformEmulator":"Debian Wheezy 7.0 (64-bit)","bootArgs":" template=domP type=secstorage host=10.90.1.21 port=8250 name=s-17-VM zone=1 pod=1 guid=s-17-VM workers=5 resource=com.cloud.storage.resource.PremiumSecondaryStorageResource instance=SecStorage sslcopy=false role=templateProcessor mtu=1500 eth2ip=205.185.158.69 eth2mask=255.255.255.192 gateway=205.185.158.65 public.network.device=eth2 eth0ip=169.254.3.61 eth0mask=255.255.0.0 eth1ip=10.90.2.202 eth1mask=255.255.255.0 mgmtcidr=10.90.1.0/24 localgw=10.90.2.254 private.network.device=eth1 eth3ip=10.83.2.208 eth3mask=255.255.255.0 storageip=10.83.2.208 storagenetmask=255.255.255.0 storagegateway=10.83.2.1 internaldns1=10.90.2.41 internaldns2=10.90.2.42 dns1=208.74.240.5 dns2=208.74.247.245","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"cUF2OGhYFGZtCrttlpKhiw==","params":{},"uuid":"e00bed4f-47fe-42cb-830e-4c70a3bc2546","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"73b9fc39-35b0-46eb-bee6-1e52fc4f9b7e","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"FlexSAN1-LUN0","id":1,"poolType":"PreSetup","host":"localhost","path":"/FlexSAN1-LUN0","port":0,"url":"PreSetup://localhost/FlexSAN1-LUN0/?ROLE=Primary&STOREUUID=FlexSAN1-LUN0"}},"name":"ROOT-17","size":2684354560,"path":"850139ef-2fe6-4118-9486-c6486dcc798a","volumeId":17,"vmName":"s-17-VM","accountId":1,"format":"VHD","provisioningType":"THIN","id":17,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"850139ef-2fe6-4118-9486-c6486dcc798a","type":"ROOT","_details":{"managed":"false","storagePort":"0","storageHost":"localhost","volumeSize":"2684354560"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"4af99ea2-0f8d-42df-815c-c524a7847e5a","uuid":"bc1ef7ea-19c0-41f1-827d-fc556a73df48","ip":"205.185.158.69","netmask":"255.255.255.192","gateway":"205.185.158.65","mac":"06:29:0e:00:00:0c","dns1":"208.74.240.5","dns2":"208.74.247.245","broadcastType":"Vlan","type":"Public","broadcastUri":"vlan://1755","isolationUri":"vlan://1755","isSecurityGroupEnabled":false,"name":"GUEST-PUB"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"c4469cb3-9f89-44cc-9e3c-cf7f6cbc0f6e","uuid":"d418b3c2-2122-42ca-93df-5f18ccd5274b","ip":"169.254.3.61","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:03:3d","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"a6398c49-3400-46c6-8bcc-319e276b1425","uuid":"5eaf68eb-a5d3-4d9c-ab71-513926e4d58f","ip":"10.90.2.202","netmask":"255.255.255.0","gateway":"10.90.2.254","mac":"06:bd:56:00:00:02","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"MGMT"},{"deviceId":3,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"5ad17f51-f91a-4111-a699-7961637dce94","uuid":"3c76f7d5-db9c-4a42-8fc7-9663c8ed2728","ip":"10.83.2.208","netmask":"255.255.255.0","gateway":"10.83.2.1","mac":"06:4f:ce:00:00:4d","broadcastType":"Storage","type":"Storage","broadcastUri":"storage://832","isSecurityGroupEnabled":false,"name":"SEC-STORE"}]},"hostIp":"10.90.2.111","executeInSequence":false,"wait":0}},{"com.cloud.agent.api.check.CheckSshCommand":{"ip":"169.254.3.61","port":3922,"interval":6,"retries":100,"name":"s-17-VM","wait":0}}] }
  703. 2015-02-23 11:55:22,479 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-3:ctx-2bdbccb8) Seq 2-8777234198768386143: Executing request
  704. 2015-02-23 11:55:22,489 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) 1. The VM s-17-VM is in Starting state.
  705. 2015-02-23 11:55:22,521 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) Created VM e8a2f40d-6daa-0432-71bb-29ab1f3e792f for s-17-VM
  706. 2015-02-23 11:55:22,527 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) PV args are -- quiet console=hvc0%template=domP%type=secstorage%host=10.90.1.21%port=8250%name=s-17-VM%zone=1%pod=1%guid=s-17-VM%workers=5%resource=com.cloud.storage.resource.PremiumSecondaryStorageResource%instance=SecStorage%sslcopy=false%role=templateProcessor%mtu=1500%eth2ip=205.185.158.69%eth2mask=255.255.255.192%gateway=205.185.158.65%public.network.device=eth2%eth0ip=169.254.3.61%eth0mask=255.255.0.0%eth1ip=10.90.2.202%eth1mask=255.255.255.0%mgmtcidr=10.90.1.0/24%localgw=10.90.2.254%private.network.device=eth1%eth3ip=10.83.2.208%eth3mask=255.255.255.0%storageip=10.83.2.208%storagenetmask=255.255.255.0%storagegateway=10.83.2.1%internaldns1=10.90.2.41%internaldns2=10.90.2.42%dns1=208.74.240.5%dns2=208.74.247.245
  707. 2015-02-23 11:55:22,559 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) VBD 8c82b7ac-44ca-a0d2-8da5-5be082f41ca2 created for com.cloud.agent.api.to.DiskTO@693dc7bb
  708. 2015-02-23 11:55:22,577 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) Creating VIF for s-17-VM on nic [Nic:Public-205.185.158.69-vlan://1755]
  709. 2015-02-23 11:55:22,579 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) Looking for network named GUEST-PUB
  710. 2015-02-23 11:55:22,588 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) Found a network called GUEST-PUB on host=10.90.2.111; Network=644f87cd-a11e-e94b-f3d1-7e221679ded2; pif=1a83e71c-1576-b3af-ed8a-3a4a7c22dab5
  711. 2015-02-23 11:55:22,598 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) Found a network called VLAN-644f87cd-a11e-e94b-f3d1-7e221679ded2-1755 on host=10.90.2.111; Network=e379b22c-3ac8-a0de-6482-40df647fb16f; pif=40054b40-9467-2221-228e-be34f2d4d46b
  712. 2015-02-23 11:55:22,608 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) Created a vif bd43afce-e5eb-56d5-e3eb-f040b39afea0 on 2
  713. 2015-02-23 11:55:22,608 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) Creating VIF for s-17-VM on nic [Nic:Control-169.254.3.61-null]
  714. 2015-02-23 11:55:22,627 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) already have a vif on dom0 for link local network
  715. 2015-02-23 11:55:22,821 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) Created a vif 342e56cc-a822-97a7-33dc-82a284a8d37c on 0
  716. 2015-02-23 11:55:22,821 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) Creating VIF for s-17-VM on nic [Nic:Management-10.90.2.202-null]
  717. 2015-02-23 11:55:22,823 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) Looking for network named MGMT
  718. 2015-02-23 11:55:22,835 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) Created a vif 2bb11752-8479-332e-78ea-188384ca8304 on 1
  719. 2015-02-23 11:55:22,836 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) Creating VIF for s-17-VM on nic [Nic:Storage-10.83.2.208-storage://832]
  720. 2015-02-23 11:55:22,837 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) Looking for network named SEC-STORE
  721. 2015-02-23 11:55:22,855 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) Found a network called SEC-STORE on host=10.90.2.111; Network=c9754adc-0015-88cf-8b40-b6eceed12b94; pif=aedbbb70-a2ad-174f-96ac-9a4e99597bc1
  722. 2015-02-23 11:55:22,859 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) Creating VLAN 832 on host 10.90.2.111 on device bond1
  723. 2015-02-23 11:55:22,887 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) Catch Exception: class com.xensource.xenapi.Types$PifIsVlan due to You tried to create a VLAN on top of another VLAN - use the underlying physical PIF/bond instead
  724. You tried to create a VLAN on top of another VLAN - use the underlying physical PIF/bond instead
  725. at com.xensource.xenapi.Types.checkResponse(Types.java:2003)
  726. at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
  727. at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:465)
  728. at com.xensource.xenapi.VLAN.create(VLAN.java:363)
  729. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.enableVlanNetwork(CitrixResourceBase.java:4026)
  730. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getNetwork(CitrixResourceBase.java:1043)
  731. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.createVif(CitrixResourceBase.java:1103)
  732. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1780)
  733. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:489)
  734. at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
  735. at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
  736. at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
  737. at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:304)
  738. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  739. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  740. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  741. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  742. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  743. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  744. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  745. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
  746. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
  747. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  748. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  749. at java.lang.Thread.run(Thread.java:745)
  750. 2015-02-23 11:55:22,888 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) Unable to start s-17-VM due to
  751. You tried to create a VLAN on top of another VLAN - use the underlying physical PIF/bond instead
  752. at com.xensource.xenapi.Types.checkResponse(Types.java:2003)
  753. at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
  754. at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:465)
  755. at com.xensource.xenapi.VLAN.create(VLAN.java:363)
  756. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.enableVlanNetwork(CitrixResourceBase.java:4026)
  757. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.getNetwork(CitrixResourceBase.java:1043)
  758. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.createVif(CitrixResourceBase.java:1103)
  759. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1780)
  760. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:489)
  761. at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
  762. at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
  763. at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
  764. at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:304)
  765. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  766. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  767. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  768. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  769. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  770. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  771. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  772. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
  773. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
  774. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  775. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  776. at java.lang.Thread.run(Thread.java:745)
  777. 2015-02-23 11:55:22,930 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) Unable to clean up VBD due to
  778. You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
  779. at com.xensource.xenapi.Types.checkResponse(Types.java:693)
  780. at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
  781. at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:465)
  782. at com.xensource.xenapi.VBD.unplug(VBD.java:1109)
  783. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1512)
  784. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1863)
  785. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:489)
  786. at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
  787. at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
  788. at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
  789. at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:304)
  790. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  791. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  792. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  793. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  794. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  795. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  796. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  797. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
  798. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
  799. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  800. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  801. at java.lang.Thread.run(Thread.java:745)
  802. 2015-02-23 11:55:22,937 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) Unable to clean up VBD due to
  803. You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
  804. at com.xensource.xenapi.Types.checkResponse(Types.java:693)
  805. at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
  806. at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:465)
  807. at com.xensource.xenapi.VBD.unplug(VBD.java:1109)
  808. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1512)
  809. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1863)
  810. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:489)
  811. at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
  812. at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
  813. at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
  814. at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:304)
  815. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  816. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  817. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  818. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  819. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  820. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  821. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  822. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
  823. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
  824. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  825. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  826. at java.lang.Thread.run(Thread.java:745)
  827. 2015-02-23 11:55:22,943 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) Unable to cleanup VIF
  828. You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
  829. at com.xensource.xenapi.Types.checkResponse(Types.java:693)
  830. at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
  831. at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:465)
  832. at com.xensource.xenapi.VIF.unplug(VIF.java:921)
  833. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1520)
  834. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1863)
  835. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:489)
  836. at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
  837. at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
  838. at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
  839. at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:304)
  840. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  841. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  842. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  843. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  844. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  845. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  846. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  847. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
  848. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
  849. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  850. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  851. at java.lang.Thread.run(Thread.java:745)
  852. 2015-02-23 11:55:22,949 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) Unable to cleanup VIF
  853. You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
  854. at com.xensource.xenapi.Types.checkResponse(Types.java:693)
  855. at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
  856. at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:465)
  857. at com.xensource.xenapi.VIF.unplug(VIF.java:921)
  858. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1520)
  859. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1863)
  860. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:489)
  861. at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
  862. at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
  863. at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
  864. at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:304)
  865. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  866. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  867. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  868. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  869. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  870. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  871. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  872. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
  873. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
  874. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  875. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  876. at java.lang.Thread.run(Thread.java:745)
  877. 2015-02-23 11:55:22,958 WARN [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) Unable to cleanup VIF
  878. You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
  879. at com.xensource.xenapi.Types.checkResponse(Types.java:693)
  880. at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
  881. at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:465)
  882. at com.xensource.xenapi.VIF.unplug(VIF.java:921)
  883. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:1520)
  884. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1863)
  885. at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:489)
  886. at com.cloud.hypervisor.xenserver.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:64)
  887. at com.cloud.hypervisor.xenserver.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:87)
  888. at com.cloud.hypervisor.xenserver.resource.XenServer620SP1Resource.executeRequest(XenServer620SP1Resource.java:65)
  889. at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:304)
  890. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  891. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  892. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  893. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  894. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  895. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  896. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  897. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
  898. at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
  899. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  900. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  901. at java.lang.Thread.run(Thread.java:745)
  902. 2015-02-23 11:55:23,006 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-3:ctx-2bdbccb8) The VM is in stopped state, detected problem during startup : s-17-VM
  903. 2015-02-23 11:55:23,007 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-3:ctx-2bdbccb8) Seq 2-8777234198768386143: Cancelling because one of the answers is false and it is stop on error.
  904. 2015-02-23 11:55:23,007 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-3:ctx-2bdbccb8) Seq 2-8777234198768386143: Response Received:
  905. 2015-02-23 11:55:23,008 DEBUG [c.c.a.t.Request] (DirectAgent-3:ctx-2bdbccb8) Seq 2-8777234198768386143: Processing: { Ans: , MgmtId: 345050411715, via: 2, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StartAnswer":{"vm":{"id":17,"name":"s-17-VM","bootloader":"PyGrub","type":"SecondaryStorageVm","cpus":1,"minSpeed":500,"maxSpeed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"Debian GNU/Linux 7(64-bit)","platformEmulator":"Debian Wheezy 7.0 (64-bit)","bootArgs":" template=domP type=secstorage host=10.90.1.21 port=8250 name=s-17-VM zone=1 pod=1 guid=s-17-VM workers=5 resource=com.cloud.storage.resource.PremiumSecondaryStorageResource instance=SecStorage sslcopy=false role=templateProcessor mtu=1500 eth2ip=205.185.158.69 eth2mask=255.255.255.192 gateway=205.185.158.65 public.network.device=eth2 eth0ip=169.254.3.61 eth0mask=255.255.0.0 eth1ip=10.90.2.202 eth1mask=255.255.255.0 mgmtcidr=10.90.1.0/24 localgw=10.90.2.254 private.network.device=eth1 eth3ip=10.83.2.208 eth3mask=255.255.255.0 storageip=10.83.2.208 storagenetmask=255.255.255.0 storagegateway=10.83.2.1 internaldns1=10.90.2.41 internaldns2=10.90.2.42 dns1=208.74.240.5 dns2=208.74.247.245","enableHA":false,"limitCpuUse":false,"enableDynamicallyScaleVm":false,"vncPassword":"cUF2OGhYFGZtCrttlpKhiw==","params":{},"uuid":"e00bed4f-47fe-42cb-830e-4c70a3bc2546","disks":[{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"73b9fc39-35b0-46eb-bee6-1e52fc4f9b7e","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"FlexSAN1-LUN0","id":1,"poolType":"PreSetup","host":"localhost","path":"/FlexSAN1-LUN0","port":0,"url":"PreSetup://localhost/FlexSAN1-LUN0/?ROLE=Primary&STOREUUID=FlexSAN1-LUN0"}},"name":"ROOT-17","size":2684354560,"path":"850139ef-2fe6-4118-9486-c6486dcc798a","volumeId":17,"vmName":"s-17-VM","accountId":1,"format":"VHD","provisioningType":"THIN","id":17,"deviceId":0,"hypervisorType":"XenServer"}},"diskSeq":0,"path":"850139ef-2fe6-4118-9486-c6486dcc798a","type":"ROOT","_details":{"managed":"false","storagePort":"0","storageHost":"localhost","volumeSize":"2684354560"}}],"nics":[{"deviceId":2,"networkRateMbps":-1,"defaultNic":true,"pxeDisable":true,"nicUuid":"4af99ea2-0f8d-42df-815c-c524a7847e5a","uuid":"bc1ef7ea-19c0-41f1-827d-fc556a73df48","ip":"205.185.158.69","netmask":"255.255.255.192","gateway":"205.185.158.65","mac":"06:29:0e:00:00:0c","dns1":"208.74.240.5","dns2":"208.74.247.245","broadcastType":"Vlan","type":"Public","broadcastUri":"vlan://1755","isolationUri":"vlan://1755","isSecurityGroupEnabled":false,"name":"GUEST-PUB"},{"deviceId":0,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"c4469cb3-9f89-44cc-9e3c-cf7f6cbc0f6e","uuid":"d418b3c2-2122-42ca-93df-5f18ccd5274b","ip":"169.254.3.61","netmask":"255.255.0.0","gateway":"169.254.0.1","mac":"0e:00:a9:fe:03:3d","broadcastType":"LinkLocal","type":"Control","isSecurityGroupEnabled":false},{"deviceId":1,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"a6398c49-3400-46c6-8bcc-319e276b1425","uuid":"5eaf68eb-a5d3-4d9c-ab71-513926e4d58f","ip":"10.90.2.202","netmask":"255.255.255.0","gateway":"10.90.2.254","mac":"06:bd:56:00:00:02","broadcastType":"Native","type":"Management","isSecurityGroupEnabled":false,"name":"MGMT"},{"deviceId":3,"networkRateMbps":-1,"defaultNic":false,"pxeDisable":true,"nicUuid":"5ad17f51-f91a-4111-a699-7961637dce94","uuid":"3c76f7d5-db9c-4a42-8fc7-9663c8ed2728","ip":"10.83.2.208","netmask":"255.255.255.0","gateway":"10.83.2.1","mac":"06:4f:ce:00:00:4d","broadcastType":"Storage","type":"Storage","broadcastUri":"storage://832","isSecurityGroupEnabled":false,"name":"SEC-STORE"}]},"_iqnToPath":{},"result":false,"details":"Unable to start s-17-VM due to ","wait":0}}] }
  906. 2015-02-23 11:55:23,008 DEBUG [c.c.a.t.Request] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Seq 2-8777234198768386143: Received: { Ans: , MgmtId: 345050411715, via: 2, Ver: v1, Flags: 10, { StartAnswer } }
  907. 2015-02-23 11:55:23,047 INFO [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Unable to start VM on Host[-2-Routing] due to Unable to start s-17-VM due to
  908. 2015-02-23 11:55:23,093 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Cleaning up resources for the vm VM[SecondaryStorageVm|s-17-VM] in Starting state
  909. 2015-02-23 11:55:23,098 DEBUG [c.c.a.t.Request] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Seq 2-8777234198768386144: Sending { Cmd , MgmtId: 345050411715, via: 2(Flex-Xen1.flexhost.local), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"s-17-VM","wait":0}}] }
  910. 2015-02-23 11:55:23,099 DEBUG [c.c.a.t.Request] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Seq 2-8777234198768386144: Executing: { Cmd , MgmtId: 345050411715, via: 2(Flex-Xen1.flexhost.local), Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"checkBeforeCleanup":false,"vmName":"s-17-VM","wait":0}}] }
  911. 2015-02-23 11:55:23,099 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-97:ctx-ec1a555a) Seq 2-8777234198768386144: Executing request
  912. 2015-02-23 11:55:23,105 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-97:ctx-ec1a555a) Seq 2-8777234198768386144: Response Received:
  913. 2015-02-23 11:55:23,106 DEBUG [c.c.a.t.Request] (DirectAgent-97:ctx-ec1a555a) Seq 2-8777234198768386144: Processing: { Ans: , MgmtId: 345050411715, via: 2, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StopAnswer":{"result":true,"details":"VM does not exist","wait":0}}] }
  914. 2015-02-23 11:55:23,106 DEBUG [c.c.a.t.Request] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Seq 2-8777234198768386144: Received: { Ans: , MgmtId: 345050411715, via: 2, Ver: v1, Flags: 10, { StopAnswer } }
  915. 2015-02-23 11:55:23,145 DEBUG [c.c.n.g.ControlNetworkGuru] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Released nic: NicProfile[65-17-null-null-null
  916. 2015-02-23 11:55:23,204 DEBUG [c.c.d.d.DataCenterIpAddressDaoImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Releasing ip address for reservationId=a899c48b-111c-4bbf-9bce-112f87b36da7, instance=66
  917. 2015-02-23 11:55:23,205 DEBUG [c.c.n.g.PodBasedNetworkGuru] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Released nic: NicProfile[66-17-null-null-null
  918. 2015-02-23 11:55:23,253 DEBUG [c.c.n.g.StorageNetworkGuru] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Release an storage ip 10.83.2.208
  919. 2015-02-23 11:55:23,299 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Successfully released network resources for the vm VM[SecondaryStorageVm|s-17-VM]
  920. 2015-02-23 11:55:23,299 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Successfully cleanued up resources for the vm VM[SecondaryStorageVm|s-17-VM] in Starting state
  921. 2015-02-23 11:55:23,303 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Root volume is ready, need to place VM in volume's cluster
  922. 2015-02-23 11:55:23,303 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Vol[17|vm=17|ROOT] is READY, changing deployment plan to use this pool's dcId: 1 , podId: 1 , and clusterId: 1
  923. 2015-02-23 11:55:23,306 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Deploy avoids pods: null, clusters: null, hosts: [1, 2]
  924. 2015-02-23 11:55:23,308 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) DeploymentPlanner allocation algorithm: com.cloud.deploy.FirstFitPlanner@7ace2d66
  925. 2015-02-23 11:55:23,308 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Trying to allocate a host and storage pools from dc:1, pod:1,cluster:1, requested cpu: 500, requested ram: 536870912
  926. 2015-02-23 11:55:23,308 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Is ROOT volume READY (pool already allocated)?: Yes
  927. 2015-02-23 11:55:23,308 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Searching resources only under specified Cluster: 1
  928. 2015-02-23 11:55:23,319 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Checking resources in Cluster: 1 under Pod: 1
  929. 2015-02-23 11:55:23,319 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Looking for hosts in dc: 1 pod:1 cluster:1
  930. 2015-02-23 11:55:23,324 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) FirstFitAllocator has 2 hosts to check for allocation: [Host[-2-Routing], Host[-1-Routing]]
  931. 2015-02-23 11:55:23,337 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Found 2 hosts for allocation after prioritization: [Host[-2-Routing], Host[-1-Routing]]
  932. 2015-02-23 11:55:23,337 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Looking for speed=500Mhz, Ram=512
  933. 2015-02-23 11:55:23,337 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Host name: Flex-Xen1.flexhost.local, hostId: 2 is in avoid set, skipping this and trying other available hosts
  934. 2015-02-23 11:55:23,337 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Host name: Flex-Xen2.flexhost.local, hostId: 1 is in avoid set, skipping this and trying other available hosts
  935. 2015-02-23 11:55:23,337 DEBUG [c.c.a.m.a.i.FirstFitAllocator] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe FirstFitRoutingAllocator) Host Allocator returning 0 suitable hosts
  936. 2015-02-23 11:55:23,337 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) No suitable hosts found
  937. 2015-02-23 11:55:23,337 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) No suitable hosts found under this Cluster: 1
  938. 2015-02-23 11:55:23,340 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Could not find suitable Deployment Destination for this VM under any clusters, returning.
  939. 2015-02-23 11:55:23,340 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Searching resources only under specified Cluster: 1
  940. 2015-02-23 11:55:23,352 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) The specified cluster is in avoid set, returning.
  941. 2015-02-23 11:55:23,357 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Deploy avoids pods: null, clusters: [1], hosts: [1, 2]
  942. 2015-02-23 11:55:23,362 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) DeploymentPlanner allocation algorithm: com.cloud.deploy.FirstFitPlanner@7ace2d66
  943. 2015-02-23 11:55:23,363 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Trying to allocate a host and storage pools from dc:1, pod:1,cluster:null, requested cpu: 500, requested ram: 536870912
  944. 2015-02-23 11:55:23,363 DEBUG [c.c.d.DeploymentPlanningManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Is ROOT volume READY (pool already allocated)?: No
  945. 2015-02-23 11:55:23,363 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Searching resources only under specified Pod: 1
  946. 2015-02-23 11:55:23,363 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Listing clusters in order of aggregate capacity, that have (atleast one host with) enough CPU and RAM capacity under this Pod: 1
  947. 2015-02-23 11:55:23,369 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Removing from the clusterId list these clusters from avoid set: [1]
  948. 2015-02-23 11:55:23,371 DEBUG [c.c.d.FirstFitPlanner] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) No clusters found after removing disabled clusters and clusters in avoid list, returning.
  949. 2015-02-23 11:55:23,427 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) VM state transitted from :Starting to Stopped with event: OperationFailedvm's original host id: null new host id: null host id before state transition: 2
  950. 2015-02-23 11:55:23,439 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Hosts's actual total CPU: 48000 and CPU after applying overprovisioning: 48000
  951. 2015-02-23 11:55:23,439 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Hosts's actual total RAM: 128787138048 and RAM after applying overprovisioning: 128787136512
  952. 2015-02-23 11:55:23,440 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) release cpu from host: 2, old used: 500,reserved: 0, actual total: 48000, total with overprovisioning: 48000; new used: 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
  953. 2015-02-23 11:55:23,440 DEBUG [c.c.c.CapacityManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) release mem from host: 2, old used: 536870912,reserved: 0, total: 128787136512; new used: 0,reserved:0; movedfromreserved: false,moveToReserveredfalse
  954. 2015-02-23 11:55:23,458 ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Invocation exception, caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[SecondaryStorageVm|s-17-VM]Scope=interface com.cloud.dc.DataCenter; id=1
  955. 2015-02-23 11:55:23,459 INFO [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104 ctx-c74cdafe) Rethrow exception com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[SecondaryStorageVm|s-17-VM]Scope=interface com.cloud.dc.DataCenter; id=1
  956. 2015-02-23 11:55:23,459 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104) Done with run of VM work job: com.cloud.vm.VmWorkStart for VM 17, job origin: 2
  957. 2015-02-23 11:55:23,459 ERROR [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104) Unable to complete AsyncJobVO {id:104, userId: 1, accountId: 1, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo: rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAEXQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcHA, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 345050411715, completeMsid: null, lastUpdated: null, lastPolled: null, created: Mon Feb 23 11:55:19 CST 2015}, job origin:2
  958. com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[SecondaryStorageVm|s-17-VM]Scope=interface com.cloud.dc.DataCenter; id=1
  959. at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:941)
  960. at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4471)
  961. at sun.reflect.GeneratedMethodAccessor258.invoke(Unknown Source)
  962. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  963. at java.lang.reflect.Method.invoke(Method.java:606)
  964. at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
  965. at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4627)
  966. at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:103)
  967. at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:536)
  968. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  969. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  970. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  971. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  972. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  973. at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:493)
  974. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  975. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  976. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  977. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  978. at java.lang.Thread.run(Thread.java:745)
  979. 2015-02-23 11:55:23,461 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104) Complete async job-104, jobStatus: FAILED, resultCode: 0, result: rO0ABXNyADdjb20uY2xvdWQuZXhjZXB0aW9uLkluc3VmZmljaWVudFNlcnZlckNhcGFjaXR5RXhjZXB0aW9uAAAAAFZNT3ACAAFaABVhZmZpbml0eUdyb3Vwc0FwcGxpZWR4cgAxY29tLmNsb3VkLmV4Y2VwdGlvbi5JbnN1ZmZpY2llbnRDYXBhY2l0eUV4Y2VwdGlvbgAAAABWTU91AgACTAACaWR0ABBMamF2YS9sYW5nL0xvbmc7TAAFc2NvcGV0ABFMamF2YS9sYW5nL0NsYXNzO3hyACJjb20uY2xvdWQuZXhjZXB0aW9uLkNsb3VkRXhjZXB0aW9ueeiOG_HkRMcCAAJMAAtjc0Vycm9yQ29kZXQAE0xqYXZhL2xhbmcvSW50ZWdlcjtMAAZpZExpc3R0ABVMamF2YS91dGlsL0FycmF5TGlzdDt4cgATamF2YS5sYW5nLkV4Y2VwdGlvbtD9Hz4aOxzEAgAAeHIAE2phdmEubGFuZy5UaHJvd2FibGXVxjUnOXe4ywMABEwABWNhdXNldAAVTGphdmEvbGFuZy9UaHJvd2FibGU7TAANZGV0YWlsTWVzc2FnZXQAEkxqYXZhL2xhbmcvU3RyaW5nO1sACnN0YWNrVHJhY2V0AB5bTGphdmEvbGFuZy9TdGFja1RyYWNlRWxlbWVudDtMABRzdXBwcmVzc2VkRXhjZXB0aW9uc3QAEExqYXZhL3V0aWwvTGlzdDt4cHEAfgANdABAVW5hYmxlIHRvIGNyZWF0ZSBhIGRlcGxveW1lbnQgZm9yIFZNW1NlY29uZGFyeVN0b3JhZ2VWbXxzLTE3LVZNXXVyAB5bTGphdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudDsCRio8PP0iOQIAAHhwAAAAFHNyABtqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnRhCcWaJjbdhQIABEkACmxpbmVOdW1iZXJMAA5kZWNsYXJpbmdDbGFzc3EAfgAKTAAIZmlsZU5hbWVxAH4ACkwACm1ldGhvZE5hbWVxAH4ACnhwAAADrXQAJmNvbS5jbG91ZC52bS5WaXJ0dWFsTWFjaGluZU1hbmFnZXJJbXBsdAAeVmlydHVhbE1hY2hpbmVNYW5hZ2VySW1wbC5qYXZhdAAQb3JjaGVzdHJhdGVTdGFydHNxAH4AEQAAEXdxAH4AE3EAfgAUcQB-ABVzcQB-ABH_____dAAmc3VuLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3IyNThwdAAGaW52b2tlc3EAfgARAAAAK3QAKHN1bi5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AGXNxAH4AEQAAAl50ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAZc3EAfgARAAAAa3QAImNvbS5jbG91ZC52bS5WbVdvcmtKb2JIYW5kbGVyUHJveHl0ABpWbVdvcmtKb2JIYW5kbGVyUHJveHkuamF2YXQAD2hhbmRsZVZtV29ya0pvYnNxAH4AEQAAEhNxAH4AE3EAfgAUcQB-ACNzcQB-ABEAAABndAAgY29tLmNsb3VkLnZtLlZtV29ya0pvYkRpc3BhdGNoZXJ0ABhWbVdvcmtKb2JEaXNwYXRjaGVyLmphdmF0AAZydW5Kb2JzcQB-ABEAAAIYdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0c3EAfgARAAAAMXQAPm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZSQxdAAbTWFuYWdlZENvbnRleHRSdW5uYWJsZS5qYXZhdAADcnVuc3EAfgARAAAAOHQAQm9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHQkMXQAGkRlZmF1bHRNYW5hZ2VkQ29udGV4dC5qYXZhdAAEY2FsbHNxAH4AEQAAAGd0AEBvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-ADN0AA9jYWxsV2l0aENvbnRleHRzcQB-ABEAAAA1cQB-ADZxAH4AM3QADnJ1bldpdGhDb250ZXh0c3EAfgARAAAALnQAPG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuTWFuYWdlZENvbnRleHRSdW5uYWJsZXEAfgAvcQB-ADBzcQB-ABEAAAHtcQB-ACpxAH4AK3EAfgAwc3EAfgARAAAB13QALmphdmEudXRpbC5jb25jdXJyZW50LkV4ZWN1dG9ycyRSdW5uYWJsZUFkYXB0ZXJ0AA5FeGVjdXRvcnMuamF2YXEAfgA0c3EAfgARAAABBnQAH2phdmEudXRpbC5jb25jdXJyZW50LkZ1dHVyZVRhc2t0AA9GdXR1cmVUYXNrLmphdmFxAH4AMHNxAH4AEQAABHl0ACdqYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3J0ABdUaHJlYWRQb29sRXhlY3V0b3IuamF2YXQACXJ1bldvcmtlcnNxAH4AEQAAAmd0AC5qYXZhLnV0aWwuY29uY3VycmVudC5UaHJlYWRQb29sRXhlY3V0b3IkV29ya2VycQB-AEVxAH4AMHNxAH4AEQAAAul0ABBqYXZhLmxhbmcuVGhyZWFkdAALVGhyZWFkLmphdmFxAH4AMHNyACZqYXZhLnV0aWwuQ29sbGVjdGlvbnMkVW5tb2RpZmlhYmxlTGlzdPwPJTG17I4QAgABTAAEbGlzdHEAfgAMeHIALGphdmEudXRpbC5Db2xsZWN0aW9ucyRVbm1vZGlmaWFibGVDb2xsZWN0aW9uGUIAgMte9x4CAAFMAAFjdAAWTGphdmEvdXRpbC9Db2xsZWN0aW9uO3hwc3IAE2phdmEudXRpbC5BcnJheUxpc3R4gdIdmcdhnQMAAUkABHNpemV4cAAAAAB3BAAAAAB4cQB-AFF4c3IAEWphdmEubGFuZy5JbnRlZ2VyEuKgpPeBhzgCAAFJAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAAAEO9zcQB-AFAAAAAAdwQAAAAAeHNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cQB-AFMAAAAAAAAAAXZyABdjb20uY2xvdWQuZGMuRGF0YUNlbnRlcrLxCuun53i-AgAAeHAA
  980. 2015-02-23 11:55:23,463 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104) Publish async job-104 complete on message bus
  981. 2015-02-23 11:55:23,463 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104) Wake up jobs related to job-104
  982. 2015-02-23 11:55:23,463 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104) Update db status for job-104
  983. 2015-02-23 11:55:23,465 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104) Wake up jobs joined with job-104 and disjoin all subjobs created from job- 104
  984. 2015-02-23 11:55:23,482 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104) Done executing com.cloud.vm.VmWorkStart for job-104
  985. 2015-02-23 11:55:23,488 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104) Sync queue (65) is currently empty
  986. 2015-02-23 11:55:23,489 INFO [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-66:ctx-c4a64e18 job-2/job-104) Remove job-104 from job monitoring
  987. 2015-02-23 11:55:23,507 WARN [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-8bce787f) Exception while trying to start secondary storage vm
  988. com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[SecondaryStorageVm|s-17-VM]Scope=interface com.cloud.dc.DataCenter; id=1
  989. at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:941)
  990. at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4471)
  991. at sun.reflect.GeneratedMethodAccessor258.invoke(Unknown Source)
  992. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  993. at java.lang.reflect.Method.invoke(Method.java:606)
  994. at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
  995. at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4627)
  996. at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:103)
  997. at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:536)
  998. at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
  999. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
  1000. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
  1001. at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
  1002. at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
  1003. at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:493)
  1004. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  1005. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  1006. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  1007. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  1008. at java.lang.Thread.run(Thread.java:745)
  1009. 2015-02-23 11:55:23,508 INFO [o.a.c.s.SecondaryStorageManagerImpl] (secstorage-1:ctx-8bce787f) Unable to start secondary storage vm for standby capacity, secStorageVm vm Id : 17, will recycle it and start a new one
  1010. 2015-02-23 11:55:23,522 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (secstorage-1:ctx-8bce787f) Sync job-105 execution on object VmWorkJobQueue.17
  1011. 2015-02-23 11:55:23,795 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgentCronJob-27:ctx-18eab266) Seq 2-8777234198768386052: Executing request
  1012. 2015-02-23 11:55:23,811 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgentCronJob-27:ctx-18eab266) Seq 2-8777234198768386052: Response Received:
  1013. 2015-02-23 11:55:23,812 DEBUG [c.c.a.t.Request] (DirectAgentCronJob-27:ctx-18eab266) Seq 2-8777234198768386052: Processing: { Ans: , MgmtId: 345050411715, via: 2, Ver: v1, Flags: 10, [{"com.cloud.agent.api.ClusterVMMetaDataSyncAnswer":{"_clusterId":1,"_isExecuted":false,"result":true,"wait":0}}] }
  1014. 2015-02-23 11:55:24,806 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-784e549f) Execute sync-queue item: SyncQueueItemVO {id:67, queueId: 65, contentType: AsyncJob, contentId: 105, lastProcessMsid: null, lastprocessNumber: null, lastProcessTime: null, created: Mon Feb 23 11:55:23 CST 2015}
  1015. 2015-02-23 11:55:24,807 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-784e549f) Schedule queued job-105
  1016. 2015-02-23 11:55:24,821 INFO [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-67:ctx-dcb069dc job-2/job-105) Add job-105 into job monitoring
  1017. 2015-02-23 11:55:24,821 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-67:ctx-dcb069dc job-2/job-105) Executing AsyncJobVO {id:105, userId: 1, accountId: 1, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStop, cmdInfo: rO0ABXNyABdjb20uY2xvdWQudm0uVm1Xb3JrU3RvcALQ4GymiWjjAgABWgAHY2xlYW51cHhyABNjb20uY2xvdWQudm0uVm1Xb3Jrn5m2VvAlZ2sCAARKAAlhY2NvdW50SWRKAAZ1c2VySWRKAAR2bUlkTAALaGFuZGxlck5hbWV0ABJMamF2YS9sYW5nL1N0cmluZzt4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAEXQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwA, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 345050411715, completeMsid: null, lastUpdated: null, lastPolled: null, created: Mon Feb 23 11:55:23 CST 2015}
  1018. 2015-02-23 11:55:24,822 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-67:ctx-dcb069dc job-2/job-105) Run VM work job: com.cloud.vm.VmWorkStop for VM 17, job origin: 2
  1019. 2015-02-23 11:55:24,824 DEBUG [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-67:ctx-dcb069dc job-2/job-105 ctx-8b4f76ac) Execute VM work job: com.cloud.vm.VmWorkStop{"cleanup":false,"userId":1,"accountId":1,"vmId":17,"handlerName":"VirtualMachineManagerImpl"}
  1020. 2015-02-23 11:55:24,829 DEBUG [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-67:ctx-dcb069dc job-2/job-105 ctx-8b4f76ac) VM is already stopped: VM[SecondaryStorageVm|s-17-VM]
  1021. 2015-02-23 11:55:24,829 DEBUG [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-67:ctx-dcb069dc job-2/job-105 ctx-8b4f76ac) Done executing VM work job: com.cloud.vm.VmWorkStop{"cleanup":false,"userId":1,"accountId":1,"vmId":17,"handlerName":"VirtualMachineManagerImpl"}
  1022. 2015-02-23 11:55:24,829 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-67:ctx-dcb069dc job-2/job-105 ctx-8b4f76ac) Complete async job-105, jobStatus: SUCCEEDED, resultCode: 0, result: null
  1023. 2015-02-23 11:55:24,831 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-67:ctx-dcb069dc job-2/job-105 ctx-8b4f76ac) Publish async job-105 complete on message bus
  1024. 2015-02-23 11:55:24,831 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-67:ctx-dcb069dc job-2/job-105 ctx-8b4f76ac) Wake up jobs related to job-105
  1025. 2015-02-23 11:55:24,831 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-67:ctx-dcb069dc job-2/job-105 ctx-8b4f76ac) Update db status for job-105
  1026. 2015-02-23 11:55:24,832 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-67:ctx-dcb069dc job-2/job-105 ctx-8b4f76ac) Wake up jobs joined with job-105 and disjoin all subjobs created from job- 105
  1027. 2015-02-23 11:55:24,854 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-67:ctx-dcb069dc job-2/job-105) Done with run of VM work job: com.cloud.vm.VmWorkStop for VM 17, job origin: 2
  1028. 2015-02-23 11:55:24,854 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-67:ctx-dcb069dc job-2/job-105) Done executing com.cloud.vm.VmWorkStop for job-105
  1029. 2015-02-23 11:55:24,859 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (Work-Job-Executor-67:ctx-dcb069dc job-2/job-105) Sync queue (65) is currently empty
  1030. 2015-02-23 11:55:24,860 INFO [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-67:ctx-dcb069dc job-2/job-105) Remove job-105 from job monitoring
  1031. 2015-02-23 11:55:24,910 DEBUG [c.c.c.CapacityManagerImpl] (secstorage-1:ctx-8bce787f) VM state transitted from :Stopped to Expunging with event: ExpungeOperationvm's original host id: null new host id: null host id before state transition: null
  1032. 2015-02-23 11:55:24,911 DEBUG [c.c.v.VirtualMachineManagerImpl] (secstorage-1:ctx-8bce787f) Destroying vm VM[SecondaryStorageVm|s-17-VM]
  1033. 2015-02-23 11:55:24,911 DEBUG [c.c.v.VirtualMachineManagerImpl] (secstorage-1:ctx-8bce787f) Cleaning up NICS
  1034. 2015-02-23 11:55:24,911 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:ctx-8bce787f) Cleaning network for vm: 17
  1035. 2015-02-23 11:55:24,934 DEBUG [c.c.n.g.PublicNetworkGuru] (secstorage-1:ctx-8bce787f) public network deallocate network: networkId: 200, ip: 205.185.158.69
  1036. 2015-02-23 11:55:24,951 DEBUG [c.c.n.g.PublicNetworkGuru] (secstorage-1:ctx-8bce787f) Deallocated nic: NicProfile[64-17-null-null-null
  1037. 2015-02-23 11:55:24,962 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:ctx-8bce787f) Removed nic id=64
  1038. 2015-02-23 11:55:24,964 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:ctx-8bce787f) Revoving nic secondary ip entry ...
  1039. 2015-02-23 11:55:24,991 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:ctx-8bce787f) Removed nic id=65
  1040. 2015-02-23 11:55:24,993 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:ctx-8bce787f) Revoving nic secondary ip entry ...
  1041. 2015-02-23 11:55:25,028 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:ctx-8bce787f) Removed nic id=66
  1042. 2015-02-23 11:55:25,029 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:ctx-8bce787f) Revoving nic secondary ip entry ...
  1043. 2015-02-23 11:55:25,079 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:ctx-8bce787f) Removed nic id=67
  1044. 2015-02-23 11:55:25,080 DEBUG [o.a.c.e.o.NetworkOrchestrator] (secstorage-1:ctx-8bce787f) Revoving nic secondary ip entry ...
  1045. 2015-02-23 11:55:25,081 DEBUG [c.c.v.VirtualMachineManagerImpl] (secstorage-1:ctx-8bce787f) Cleaning up hypervisor data structures (ex. SRs in XenServer) for managed storage
  1046. 2015-02-23 11:55:25,085 DEBUG [o.a.c.e.o.VolumeOrchestrator] (secstorage-1:ctx-8bce787f) Cleaning storage for vm: 17
  1047. 2015-02-23 11:55:25,165 DEBUG [c.c.a.t.Request] (secstorage-1:ctx-8bce787f) Seq 1-3355181722391019622: Sending { Cmd , MgmtId: 345050411715, via: 1(Flex-Xen2.flexhost.local), Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"73b9fc39-35b0-46eb-bee6-1e52fc4f9b7e","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"FlexSAN1-LUN0","id":1,"poolType":"PreSetup","host":"localhost","path":"/FlexSAN1-LUN0","port":0,"url":"PreSetup://localhost/FlexSAN1-LUN0/?ROLE=Primary&STOREUUID=FlexSAN1-LUN0"}},"name":"ROOT-17","size":2684354560,"path":"850139ef-2fe6-4118-9486-c6486dcc798a","volumeId":17,"vmName":"s-17-VM","accountId":1,"format":"VHD","provisioningType":"THIN","id":17,"deviceId":0,"hypervisorType":"XenServer"}},"wait":0}}] }
  1048. 2015-02-23 11:55:25,165 DEBUG [c.c.a.t.Request] (secstorage-1:ctx-8bce787f) Seq 1-3355181722391019622: Executing: { Cmd , MgmtId: 345050411715, via: 1(Flex-Xen2.flexhost.local), Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.DeleteCommand":{"data":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"73b9fc39-35b0-46eb-bee6-1e52fc4f9b7e","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"FlexSAN1-LUN0","id":1,"poolType":"PreSetup","host":"localhost","path":"/FlexSAN1-LUN0","port":0,"url":"PreSetup://localhost/FlexSAN1-LUN0/?ROLE=Primary&STOREUUID=FlexSAN1-LUN0"}},"name":"ROOT-17","size":2684354560,"path":"850139ef-2fe6-4118-9486-c6486dcc798a","volumeId":17,"vmName":"s-17-VM","accountId":1,"format":"VHD","provisioningType":"THIN","id":17,"deviceId":0,"hypervisorType":"XenServer"}},"wait":0}}] }
  1049. 2015-02-23 11:55:25,165 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-49:ctx-93489689) Seq 1-3355181722391019622: Executing request
  1050. 2015-02-23 11:55:25,791 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-49:ctx-93489689) Seq 1-3355181722391019622: Response Received:
  1051. 2015-02-23 11:55:25,791 DEBUG [c.c.a.t.Request] (DirectAgent-49:ctx-93489689) Seq 1-3355181722391019622: Processing: { Ans: , MgmtId: 345050411715, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
  1052. 2015-02-23 11:55:25,791 DEBUG [c.c.a.t.Request] (secstorage-1:ctx-8bce787f) Seq 1-3355181722391019622: Received: { Ans: , MgmtId: 345050411715, via: 1, Ver: v1, Flags: 10, { Answer } }
  1053. 2015-02-23 11:55:25,813 INFO [o.a.c.s.v.VolumeServiceImpl] (secstorage-1:ctx-8bce787f) Volume 17 is not referred anywhere, remove it from volumes table
  1054. 2015-02-23 11:55:25,858 DEBUG [c.c.v.VirtualMachineManagerImpl] (secstorage-1:ctx-8bce787f) Expunged VM[SecondaryStorageVm|s-17-VM]
  1055. 2015-02-23 11:55:25,874 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter] (secstorage-1:ctx-8bce787f) received secondary storage vm alert
  1056. 2015-02-23 11:55:25,874 DEBUG [c.c.a.SecondaryStorageVmAlertAdapter] (secstorage-1:ctx-8bce787f) Secondary Storage Vm creation failure, zone: Rushford
  1057. 2015-02-23 11:55:25,875 WARN [o.a.c.alerts] (secstorage-1:ctx-8bce787f) alertType:: 19 // dataCenterId:: 1 // podId:: null // clusterId:: null // message:: Secondary Storage Vm creation failure. zone: Rushford, error details: null
  1058. 2015-02-23 11:55:25,908 INFO [o.a.c.s.PremiumSecondaryStorageManagerImpl] (secstorage-1:ctx-8bce787f) Primary secondary storage is not even started, wait until next turn
  1059. 2015-02-23 11:55:26,495 DEBUG [c.c.s.StatsCollector] (StatsCollector-2:ctx-78bc7e4e) AutoScaling Monitor is running...
  1060. 2015-02-23 11:55:26,587 DEBUG [c.c.s.StatsCollector] (StatsCollector-3:ctx-9ffab05f) VmStatsCollector is running...
  1061. 2015-02-23 11:55:28,329 DEBUG [c.c.s.StatsCollector] (StatsCollector-1:ctx-a4768db3) HostStatsCollector is running...
  1062. 2015-02-23 11:55:28,342 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-98:ctx-677851d6) Seq 1-3355181722391019623: Executing request
  1063. 2015-02-23 11:55:28,356 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-98:ctx-677851d6) Seq 1-3355181722391019623: Response Received:
  1064. 2015-02-23 11:55:28,356 DEBUG [c.c.a.t.Request] (StatsCollector-1:ctx-a4768db3) Seq 1-3355181722391019623: Received: { Ans: , MgmtId: 345050411715, via: 1, Ver: v1, Flags: 10, { GetHostStatsAnswer } }
  1065. 2015-02-23 11:55:28,366 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-25:ctx-ffcb9b06) Seq 2-8777234198768386145: Executing request
  1066. 2015-02-23 11:55:28,403 DEBUG [c.c.a.m.DirectAgentAttache] (DirectAgent-25:ctx-ffcb9b06) Seq 2-8777234198768386145: Response Received:
  1067. 2015-02-23 11:55:28,403 DEBUG [c.c.a.t.Request] (StatsCollector-1:ctx-a4768db3) Seq 2-8777234198768386145: Received: { Ans: , MgmtId: 345050411715, via: 2, Ver: v1, Flags: 10, { GetHostStatsAnswer } }
  1068. 2015-02-23 11:55:30,798 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-52d180e9) Begin cleanup expired async-jobs
  1069. 2015-02-23 11:55:30,807 INFO [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-52d180e9) End cleanup expired async-jobs
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!