API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Sep 15th, 2019
300
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 70.23 KB | None | 0 0
raw download clone embed print report
  1. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:03Z [DEBUG] No container health metrics to report
  2. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:03Z [DEBUG] Received message of type: AckPublishMetric
  3. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:03Z [DEBUG] Received AckPublishMetric from tcs
  4. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:23Z [DEBUG] No container health metrics to report
  5. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:23Z [DEBUG] Received message of type: AckPublishMetric
  6. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:23Z [DEBUG] Received AckPublishMetric from tcs
  7. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:41Z [DEBUG] Received message of type: AttachTaskNetworkInterfacesMessage
  8. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:41Z [DEBUG] ACS activity occurred
  9. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:41Z [INFO] Handling ENI attachment: arn:aws:ecs:us-east-1:123456789012:attachment/d35c5ba2-e3a1-42c1-8aaa-96f95d879e37
  10. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:41Z [INFO] Starting ENI ack timer with duration=2m59.99998802s, ENI Attachment: task=arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c;attachment=arn:aws:ecs:us-east-1:123456789012:attachment/d35c5ba2-e3a1-42c1-8aaa-96f95d879e37;attachmentType=task-eni;attachmentSent=false;mac=0a:05:89:1b:0e:0a;status=NONE;expiresAt=2019-09-15 13:23:41.227378897 +0000 UTC m=+458.248372456
  11. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:41Z [INFO] Adding task eni attachment info for task 'arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c' to state, attachment=arn:aws:ecs:us-east-1:123456789012:attachment/d35c5ba2-e3a1-42c1-8aaa-96f95d879e37 mac=0a:05:89:1b:0e:0a
  12. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:41Z [INFO] Saving state! module="statemanager"
  13. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:42Z [DEBUG] Udev watcher event-handler: add interface: eth1
  14. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:42Z [INFO] Emitting task ENI attached event for: ENI Attachment: task=arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c;attachment=arn:aws:ecs:us-east-1:123456789012:attachment/d35c5ba2-e3a1-42c1-8aaa-96f95d879e37;attachmentType=task-eni;attachmentSent=false;mac=0a:05:89:1b:0e:0a;status=ATTACHED;expiresAt=2019-09-15 13:23:41.227378897 +0000 UTC m=+458.248372456
  15. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:42Z [DEBUG] TaskHandler: collecting events for new task; event: TaskChange: [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c -> NONE, ENI Attachment: task=arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c;attachment=arn:aws:ecs:us-east-1:123456789012:attachment/d35c5ba2-e3a1-42c1-8aaa-96f95d879e37;attachmentType=task-eni;attachmentSent=false;mac=0a:05:89:1b:0e:0a;status=ATTACHED;expiresAt=2019-09-15 13:23:41.227378897 +0000 UTC m=+458.248372456] sent: false; events: Task event list [taskARN: arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, sending: false, createdAt: 2019-09-15 13:20:42.104489552 +0000 UTC m=+279.125483116]
  16. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:42Z [INFO] TaskHandler: Adding event: TaskChange: [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c -> NONE, ENI Attachment: task=arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c;attachment=arn:aws:ecs:us-east-1:123456789012:attachment/d35c5ba2-e3a1-42c1-8aaa-96f95d879e37;attachmentType=task-eni;attachmentSent=false;mac=0a:05:89:1b:0e:0a;status=ATTACHED;expiresAt=2019-09-15 13:23:41.227378897 +0000 UTC m=+458.248372456] sent: false
  17. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:42Z [DEBUG] TaskHandler: Waiting on semaphore to send events...
  18. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:42Z [DEBUG] TaskHandler: Acquiring lock for sending event...
  19. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:42Z [DEBUG] TaskHandler: Acquired lock, processing event list: : Task event list [taskARN: arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, sending: true, createdAt: 2019-09-15 13:20:42.104489552 +0000 UTC m=+279.125483116]
  20. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:42Z [INFO] TaskHandler: Sending task attachment change: TaskChange: [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c -> NONE, ENI Attachment: task=arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c;attachment=arn:aws:ecs:us-east-1:123456789012:attachment/d35c5ba2-e3a1-42c1-8aaa-96f95d879e37;attachmentType=task-eni;attachmentSent=false;mac=0a:05:89:1b:0e:0a;status=ATTACHED;expiresAt=2019-09-15 13:23:41.227378897 +0000 UTC m=+458.248372456] sent: false
  21. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:42Z [DEBUG] TaskHandler: Submitted task state change: TaskChange: [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c -> NONE, ENI Attachment: task=arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c;attachment=arn:aws:ecs:us-east-1:123456789012:attachment/d35c5ba2-e3a1-42c1-8aaa-96f95d879e37;attachmentType=task-eni;attachmentSent=true;mac=0a:05:89:1b:0e:0a;status=ATTACHED;expiresAt=2019-09-15 13:23:41.227378897 +0000 UTC m=+458.248372456] sent: true
  22. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:42Z [DEBUG] TaskHandler: Removed the last element, no longer sending
  23. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:43Z [DEBUG] No container health metrics to report
  24. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:43Z [DEBUG] Received message of type: AckPublishMetric
  25. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:43Z [DEBUG] Received AckPublishMetric from tcs
  26. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Received message of type: PayloadMessage
  27. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] ACS activity occurred
  28. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Received payload message, message id: 6c1140f4-c9ea-4011-a28e-f21c77015500
  29. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Updating task: [dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->NONE),app-application123 (NONE->NONE),~internal~ecs~pause (NONE->NONE), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]]
  30. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Acking credentials message: {
  31. CredentialsId: "226d776a-6cba-43c3-9fe0-ab5c4c3e4e30",
  32. Expiration: "2019-09-15T19:20:47Z",
  33. MessageId: "6c1140f4-c9ea-4011-a28e-f21c77015500"
  34. dev-cluster/ecs-agents i-12345678901234567 }
  35. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] api/task: Updating task's known status, task: dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (NONE->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]
  36. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] api/task: Container with earliest known container is [nginx-application123(123456789012.dkr.ecr.us-east-1.amazonaws.com/nginx-application123:development-716b4e55dd3235f6548d645af9e463e744d3785f) (NONE->RUNNING)] for task: dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (NONE->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]
  37. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Updating task: [dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (NONE->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]]
  38. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: unable to create state change event for container [nginx-application123]: create container state change event api: status not recognized by ECS: NONE
  39. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: unable to create state change event for container [app-application123]: create container state change event api: status not recognized by ECS: NONE
  40. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: unable to create state change event for container [~internal~ecs~pause]: create container state change event api: status not recognized by ECS: NONE
  41. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: unable to create task state change event []: create task state change event api: status not recognized by ECS: NONE
  42. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: waiting for any previous stops to complete. Sequence number: 19873
  43. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Acking credentials message: {
  44. CredentialsId: "226d776a-6cba-43c3-9fe0-ab5c4c3e4e30",
  45. Expiration: "2019-09-15T19:20:47Z",
  46. MessageId: "6c1140f4-c9ea-4011-a28e-f21c77015500"
  47. dev-cluster/ecs-agents i-12345678901234567 }
  48. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Acking payload message id: 6c1140f4-c9ea-4011-a28e-f21c77015500
  49. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: waiting for event for task
  50. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: no longer waiting
  51. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: wait over; ready to move towards status: RUNNING
  52. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: task not steady state or terminal; progressing it
  53. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: progressing containers and resources in task
  54. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: can't apply state to container [nginx-application123] yet due to unresolved dependencies: dependency graph: failed to resolve the container ordering dependency [app-application123(123456789012.dkr.ecr.us-east-1.amazonaws.com/application123:development-716b4e55dd3235f6548d645af9e463e744d3785f) (NONE->RUNNING)] for target [nginx-application123(123456789012.dkr.ecr.us-east-1.amazonaws.com/nginx-application123:development-716b4e55dd3235f6548d645af9e463e744d3785f) (NONE->RUNNING)]
  55. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: can't apply state to container [app-application123] yet due to unresolved dependencies: dependency graph: dependency on containers not resolved
  56. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: waiting for event for task
  57. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Task engine [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: transitioned container [~internal~ecs~pause] to [PULLED]
  58. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: got container [~internal~ecs~pause] event: [PULLED]
  59. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: handling container change [{PULLED { <nil> [] <nil> [] map[] 0001-01-01 00:00:00 +0000 UTC 0001-01-01 00:00:00 +0000 UTC 0001-01-01 00:00:00 +0000 UTC {UNKNOWN <nil> 0 } <nil>} ContainerStatusChangeEvent}] for container [~internal~ecs~pause]
  60. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: sending container change event to tcs, container: [~internal~ecs~pause()], status: PULLED
  61. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: unable to create state change event for container [~internal~ecs~pause]: create container state change event api: status not recognized by ECS: PULLED
  62. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] api/task: Updating task's known status, task: dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (PULLED->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]
  63. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] api/task: Container with earliest known container is [nginx-application123(123456789012.dkr.ecr.us-east-1.amazonaws.com/nginx-application123:development-716b4e55dd3235f6548d645af9e463e744d3785f) (NONE->RUNNING)] for task: dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (PULLED->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]
  64. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Updating task: [dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (PULLED->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]]
  65. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: received non-transition events
  66. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] api/task: Updating task's known status, task: dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (PULLED->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]
  67. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] api/task: Container with earliest known container is [nginx-application123(123456789012.dkr.ecr.us-east-1.amazonaws.com/nginx-application123:development-716b4e55dd3235f6548d645af9e463e744d3785f) (NONE->RUNNING)] for task: dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (PULLED->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]
  68. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Updating task: [dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (PULLED->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]]
  69. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: task not steady state or terminal; progressing it
  70. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: progressing containers and resources in task
  71. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: can't apply state to container [nginx-application123] yet due to unresolved dependencies: dependency graph: failed to resolve the container ordering dependency [app-application123(123456789012.dkr.ecr.us-east-1.amazonaws.com/application123:development-716b4e55dd3235f6548d645af9e463e744d3785f) (NONE->RUNNING)] for target [nginx-application123(123456789012.dkr.ecr.us-east-1.amazonaws.com/nginx-application123:development-716b4e55dd3235f6548d645af9e463e744d3785f) (NONE->RUNNING)]
  72. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: can't apply state to container [app-application123] yet due to unresolved dependencies: dependency graph: dependency on containers not resolved
  73. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: waiting for event for task
  74. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Task engine [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: creating container: ~internal~ecs~pause
  75. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Converting CPU shares to allowed minimum of 2 for task arn: [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c] and cpu shares: 0
  76. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Task engine [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: created container name mapping for task: ~internal~ecs~pause -> ecs-dev-application123-api-td-360-internalecspause-b6f389d4a89b82f64700
  77. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Saving state! module="statemanager"
  78. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Cgroup resource [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: setting up cgroup at: /ecs/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c
  79. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Checking existence of cgroup: /ecs/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c
  80. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Event stream ContainerChange received events, broadcasting to listeners...
  81. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Ignoring event for container, id: , status: 1
  82. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Creating cgroup /ecs/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c
  83. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: transitioned resource [cgroup] to [CREATED]
  84. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: got resource [cgroup] event: [CREATED]
  85. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: received non-transition events
  86. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] api/task: Updating task's known status, task: dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (PULLED->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]
  87. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] api/task: Container with earliest known container is [nginx-application123(123456789012.dkr.ecr.us-east-1.amazonaws.com/nginx-application123:development-716b4e55dd3235f6548d645af9e463e744d3785f) (NONE->RUNNING)] for task: dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (PULLED->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]
  88. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Updating task: [dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (PULLED->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]]
  89. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: task not steady state or terminal; progressing it
  90. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: progressing containers and resources in task
  91. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: resource [cgroup] has already transitioned to or beyond the desired status CREATED; current known is CREATED
  92. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: can't apply state to container [nginx-application123] yet due to unresolved dependencies: dependency graph: failed to resolve the container ordering dependency [app-application123(123456789012.dkr.ecr.us-east-1.amazonaws.com/application123:development-716b4e55dd3235f6548d645af9e463e744d3785f) (NONE->RUNNING)] for target [nginx-application123(123456789012.dkr.ecr.us-east-1.amazonaws.com/nginx-application123:development-716b4e55dd3235f6548d645af9e463e744d3785f) (NONE->RUNNING)]
  93. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: can't apply state to container [app-application123] yet due to unresolved dependencies: dependency graph: dependency on containers not resolved
  94. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: waiting for event for task
  95. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] DockerGoClient: got event from docker daemon: &{create d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8 amazon/amazon-ecs-pause:0.1.0 container create {d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8 map[com.amazonaws.ecs.cluster:dev-cluster com.amazonaws.ecs.container-name:~internal~ecs~pause com.amazonaws.ecs.task-arn:arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c com.amazonaws.ecs.task-definition-family:dev-application123-api-td com.amazonaws.ecs.task-definition-version:360 image:amazon/amazon-ecs-pause:0.1.0 name:ecs-dev-application123-api-td-360-internalecspause-b6f389d4a89b82f64700]} local 1568553647 1568553647817335542}
  96. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Task engine: handling a docker event: Status: CREATED, DockerID: d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8, health: UNKNOWN
  97. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Task engine: event for container [d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8] not managed, unable to map container id to task
  98. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Task engine [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: created docker container for task: ~internal~ecs~pause -> d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8
  99. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Task engine [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: created docker container for task: ~internal~ecs~pause -> d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8, took 70.324473ms
  100. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Task engine [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: transitioned container [~internal~ecs~pause] to [CREATED]
  101. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: got container [~internal~ecs~pause] event: [CREATED]
  102. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: handling container change [{CREATED {d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8 <nil> [] <nil> [] map[com.amazonaws.ecs.cluster:dev-cluster com.amazonaws.ecs.container-name:~internal~ecs~pause com.amazonaws.ecs.task-arn:arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c com.amazonaws.ecs.task-definition-family:dev-application123-api-td com.amazonaws.ecs.task-definition-version:360] 2019-09-15 13:20:47.750972232 +0000 UTC 0001-01-01 00:00:00 +0000 UTC 0001-01-01 00:00:00 +0000 UTC {UNKNOWN <nil> 0 } none 0xc000650200} ContainerStatusChangeEvent}] for container [~internal~ecs~pause]
  103. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: sending container change event to tcs, container: [~internal~ecs~pause(d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8)], status: CREATED
  104. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: unable to create state change event for container [~internal~ecs~pause]: create container state change event api: status not recognized by ECS: CREATED
  105. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] api/task: Updating task's known status, task: dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (CREATED->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]
  106. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] api/task: Container with earliest known container is [nginx-application123(123456789012.dkr.ecr.us-east-1.amazonaws.com/nginx-application123:development-716b4e55dd3235f6548d645af9e463e744d3785f) (NONE->RUNNING)] for task: dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (CREATED->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]
  107. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Updating task: [dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (CREATED->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]]
  108. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: received non-transition events
  109. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] api/task: Updating task's known status, task: dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (CREATED->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]
  110. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] api/task: Container with earliest known container is [nginx-application123(123456789012.dkr.ecr.us-east-1.amazonaws.com/nginx-application123:development-716b4e55dd3235f6548d645af9e463e744d3785f) (NONE->RUNNING)] for task: dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (CREATED->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]
  111. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Updating task: [dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (CREATED->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]]
  112. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: task not steady state or terminal; progressing it
  113. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: progressing containers and resources in task
  114. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: resource [cgroup] has already transitioned to or beyond the desired status CREATED; current known is CREATED
  115. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: can't apply state to container [nginx-application123] yet due to unresolved dependencies: dependency graph: failed to resolve the container ordering dependency [app-application123(123456789012.dkr.ecr.us-east-1.amazonaws.com/application123:development-716b4e55dd3235f6548d645af9e463e744d3785f) (NONE->RUNNING)] for target [nginx-application123(123456789012.dkr.ecr.us-east-1.amazonaws.com/nginx-application123:development-716b4e55dd3235f6548d645af9e463e744d3785f) (NONE->RUNNING)]
  116. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: can't apply state to container [app-application123] yet due to unresolved dependencies: dependency graph: dependency on containers not resolved
  117. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: waiting for event for task
  118. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [INFO] Task engine [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: starting container: ~internal~ecs~pause
  119. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Event stream ContainerChange received events, broadcasting to listeners...
  120. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:47Z [DEBUG] Ignoring event for container, id: d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8, status: 2
  121. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] DockerGoClient: got event from docker daemon: &{start d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8 amazon/amazon-ecs-pause:0.1.0 container start {d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8 map[com.amazonaws.ecs.cluster:dev-cluster com.amazonaws.ecs.container-name:~internal~ecs~pause com.amazonaws.ecs.task-arn:arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c com.amazonaws.ecs.task-definition-family:dev-application123-api-td com.amazonaws.ecs.task-definition-version:360 image:amazon/amazon-ecs-pause:0.1.0 name:ecs-dev-application123-api-td-360-internalecspause-b6f389d4a89b82f64700]} local 1568553648 1568553648115871418}
  122. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] Task engine [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: started docker container for task: ~internal~ecs~pause -> d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8, took 299.198289ms
  123. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Task engine [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: transitioned container [~internal~ecs~pause] to [RUNNING]
  124. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: got container [~internal~ecs~pause] event: [RUNNING]
  125. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: handling container change [{RUNNING {d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8 <nil> [] <nil> [] map[com.amazonaws.ecs.cluster:dev-cluster com.amazonaws.ecs.container-name:~internal~ecs~pause com.amazonaws.ecs.task-arn:arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c com.amazonaws.ecs.task-definition-family:dev-application123-api-td com.amazonaws.ecs.task-definition-version:360] 2019-09-15 13:20:47.750972232 +0000 UTC 2019-09-15 13:20:48.108339562 +0000 UTC 0001-01-01 00:00:00 +0000 UTC {UNKNOWN <nil> 0 } none 0xc00011b700} ContainerStatusChangeEvent}] for container [~internal~ecs~pause]
  126. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: sending container change event to tcs, container: [~internal~ecs~pause(d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8)], status: RUNNING
  127. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: unable to create state change event for container [~internal~ecs~pause]: create container state change event api: status not recognized by ECS: RUNNING
  128. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] api/task: Updating task's known status, task: dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (RUNNING->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]
  129. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] api/task: Container with earliest known container is [nginx-application123(123456789012.dkr.ecr.us-east-1.amazonaws.com/nginx-application123:development-716b4e55dd3235f6548d645af9e463e744d3785f) (NONE->RUNNING)] for task: dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (RUNNING->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]
  130. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Updating task: [dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (RUNNING->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]]
  131. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: received non-transition events
  132. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] api/task: Updating task's known status, task: dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (RUNNING->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]
  133. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] api/task: Container with earliest known container is [nginx-application123(123456789012.dkr.ecr.us-east-1.amazonaws.com/nginx-application123:development-716b4e55dd3235f6548d645af9e463e744d3785f) (NONE->RUNNING)] for task: dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (RUNNING->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]
  134. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Updating task: [dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (RUNNING->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]]
  135. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: task not steady state or terminal; progressing it
  136. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: progressing containers and resources in task
  137. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: resource [cgroup] has already transitioned to or beyond the desired status CREATED; current known is CREATED
  138. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: can't apply state to container [nginx-application123] yet due to unresolved dependencies: dependency graph: failed to resolve the container ordering dependency [app-application123(123456789012.dkr.ecr.us-east-1.amazonaws.com/application123:development-716b4e55dd3235f6548d645af9e463e744d3785f) (NONE->RUNNING)] for target [nginx-application123(123456789012.dkr.ecr.us-east-1.amazonaws.com/nginx-application123:development-716b4e55dd3235f6548d645af9e463e744d3785f) (NONE->RUNNING)]
  139. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: can't apply state to container [app-application123] yet due to unresolved dependencies: dependency graph: dependency on containers not resolved
  140. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: waiting for event for task
  141. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] Task engine [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: setting up container resources for container [~internal~ecs~pause]
  142. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Event stream ContainerChange received events, broadcasting to listeners...
  143. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Adding container to stats watch list, id: d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8, task: arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c
  144. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Collecting stats for container d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8
  145. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Task engine: handling a docker event: Status: RUNNING, DockerID: d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8, health: UNKNOWN, Labels: map[com.amazonaws.ecs.cluster:dev-cluster com.amazonaws.ecs.container-name:~internal~ecs~pause com.amazonaws.ecs.task-arn:arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c com.amazonaws.ecs.task-definition-family:dev-application123-api-td com.amazonaws.ecs.task-definition-version:360], CreatedAt: 2019-09-15 13:20:47.750972232 +0000 UTC, StartedAt: 2019-09-15 13:20:48.108339562 +0000 UTC
  146. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Task engine [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: writing docker event to the task: Status: RUNNING, DockerID: d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8, health: UNKNOWN, Labels: map[com.amazonaws.ecs.cluster:dev-cluster com.amazonaws.ecs.container-name:~internal~ecs~pause com.amazonaws.ecs.task-arn:arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c com.amazonaws.ecs.task-definition-family:dev-application123-api-td com.amazonaws.ecs.task-definition-version:360], CreatedAt: 2019-09-15 13:20:47.750972232 +0000 UTC, StartedAt: 2019-09-15 13:20:48.108339562 +0000 UTC
  147. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Task engine [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: wrote docker event to the task: Status: RUNNING, DockerID: d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8, health: UNKNOWN, Labels: map[com.amazonaws.ecs.cluster:dev-cluster com.amazonaws.ecs.container-name:~internal~ecs~pause com.amazonaws.ecs.task-arn:arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c com.amazonaws.ecs.task-definition-family:dev-application123-api-td com.amazonaws.ecs.task-definition-version:360], CreatedAt: 2019-09-15 13:20:47.750972232 +0000 UTC, StartedAt: 2019-09-15 13:20:48.108339562 +0000 UTC
  148. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: got container [~internal~ecs~pause] event: [RUNNING]
  149. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: handling container change [{RUNNING {d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8 <nil> [] <nil> [] map[com.amazonaws.ecs.cluster:dev-cluster com.amazonaws.ecs.container-name:~internal~ecs~pause com.amazonaws.ecs.task-arn:arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c com.amazonaws.ecs.task-definition-family:dev-application123-api-td com.amazonaws.ecs.task-definition-version:360] 2019-09-15 13:20:47.750972232 +0000 UTC 2019-09-15 13:20:48.108339562 +0000 UTC 0001-01-01 00:00:00 +0000 UTC {UNKNOWN <nil> 0 } none 0xc0001ce200} ContainerStatusChangeEvent}] for container [~internal~ecs~pause]
  150. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: redundant container state change. ~internal~ecs~pause to RUNNING, but already RUNNING
  151. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: received non-transition events
  152. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] api/task: Updating task's known status, task: dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (RUNNING->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]
  153. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] api/task: Container with earliest known container is [nginx-application123(123456789012.dkr.ecr.us-east-1.amazonaws.com/nginx-application123:development-716b4e55dd3235f6548d645af9e463e744d3785f) (NONE->RUNNING)] for task: dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (RUNNING->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]
  154. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Updating task: [dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (RUNNING->RESOURCES_PROVISIONED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]]
  155. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: task not steady state or terminal; progressing it
  156. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: progressing containers and resources in task
  157. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: resource [cgroup] has already transitioned to or beyond the desired status CREATED; current known is CREATED
  158. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: can't apply state to container [nginx-application123] yet due to unresolved dependencies: dependency graph: failed to resolve the container ordering dependency [app-application123(123456789012.dkr.ecr.us-east-1.amazonaws.com/application123:development-716b4e55dd3235f6548d645af9e463e744d3785f) (NONE->RUNNING)] for target [nginx-application123(123456789012.dkr.ecr.us-east-1.amazonaws.com/nginx-application123:development-716b4e55dd3235f6548d645af9e463e744d3785f) (NONE->RUNNING)]
  159. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: can't apply state to container [app-application123] yet due to unresolved dependencies: dependency graph: dependency on containers not resolved
  160. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: waiting for event for task
  161. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] [ECSCNI] Starting ENI (eni-02ce4ecc66dc7a812) setup in the the container namespace: d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8
  162. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] [ECSCNI] ENI setup done: Interfaces:[{Name:eth1 Mac:0a:05:89:1b:0e:0a Sandbox:}], IP:[{Version:4 Interface:<nil> Address:{IP:10.3.11.0 Mask:ffffff00} Gateway:<nil>}], DNS:{Nameservers:[] Domain: Search:[] Options:[]}
  163. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Udev watcher event handler: ignoring event for invalid network device: add@/devices/virtual/net/veth0552e99a
  164. dev-cluster/ecs-agents i-12345678901234567 ACTION=add
  165. dev-cluster/ecs-agents i-12345678901234567 DEVPATH=/devices/virtual/net/veth0552e99a
  166. dev-cluster/ecs-agents i-12345678901234567 SUBSYSTEM=net
  167. dev-cluster/ecs-agents i-12345678901234567 INTERFACE=veth0552e99a
  168. dev-cluster/ecs-agents i-12345678901234567 IFINDEX=8188
  169. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [ERROR] Task engine [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: unable to configure pause container namespace: cni setup: invoke bridge plugin failed: bridge ipam ADD: failed to execute plugin: ecs-ipam: getIPV4AddressFromDB commands: failed to get available ip from the db: getAvailableIP ipstore: failed to find available ip addresses in the subnet
  170. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] Task engine [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: error transitioning container [~internal~ecs~pause] to [RESOURCES_PROVISIONED]: container resource provisioning: failed to setup network namespace: cni setup: invoke bridge plugin failed: bridge ipam ADD: failed to execute plugin: ecs-ipam: getIPV4AddressFromDB commands: failed to get available ip from the db: getAvailableIP ipstore: failed to find available ip addresses in the subnet
  171. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: got container [~internal~ecs~pause] event: [RESOURCES_PROVISIONED]
  172. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: handling container change [{RESOURCES_PROVISIONED {d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8 <nil> [] container resource provisioning: failed to setup network namespace: cni setup: invoke bridge plugin failed: bridge ipam ADD: failed to execute plugin: ecs-ipam: getIPV4AddressFromDB commands: failed to get available ip from the db: getAvailableIP ipstore: failed to find available ip addresses in the subnet [] map[] 0001-01-01 00:00:00 +0000 UTC 0001-01-01 00:00:00 +0000 UTC 0001-01-01 00:00:00 +0000 UTC {UNKNOWN <nil> 0 } <nil>} ContainerStatusChangeEvent}] for container [~internal~ecs~pause]
  173. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [WARN] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: error starting/provisioning container[~internal~ecs~pause]; marking its desired status as STOPPED: container resource provisioning: failed to setup network namespace: cni setup: invoke bridge plugin failed: bridge ipam ADD: failed to execute plugin: ecs-ipam: getIPV4AddressFromDB commands: failed to get available ip from the db: getAvailableIP ipstore: failed to find available ip addresses in the subnet
  174. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: received non-transition events
  175. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] api/task: Updating task's known status, task: dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (RUNNING->STOPPED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]
  176. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] api/task: Container with earliest known container is [nginx-application123(123456789012.dkr.ecr.us-east-1.amazonaws.com/nginx-application123:development-716b4e55dd3235f6548d645af9e463e744d3785f) (NONE->RUNNING)] for task: dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (RUNNING->STOPPED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]
  177. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Updating task: [dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (RUNNING->STOPPED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]]
  178. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] api/task: Updating task desired status to stopped because of container: [~internal~ecs~pause]; task: [dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->RUNNING) Containers: [nginx-application123 (NONE->RUNNING),app-application123 (NONE->RUNNING),~internal~ecs~pause (RUNNING->STOPPED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]]
  179. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: task not steady state or terminal; progressing it
  180. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: progressing containers and resources in task
  181. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: can't apply state to container [nginx-application123] yet due to unresolved dependencies: dependency graph: dependency on containers not resolved
  182. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: can't apply state to container [app-application123] yet due to unresolved dependencies: dependency graph: dependency on containers not resolved
  183. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: waiting for event for task
  184. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] Task engine [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: stopping container [~internal~ecs~pause]
  185. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] Task engine [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: cleaning up the network namespace
  186. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: got resource [cgroup] event: [REMOVED]
  187. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: received non-transition events
  188. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] api/task: Updating task's known status, task: dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->STOPPED) Containers: [nginx-application123 (NONE->STOPPED),app-application123 (NONE->STOPPED),~internal~ecs~pause (RUNNING->STOPPED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]
  189. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] api/task: Container with earliest known container is [nginx-application123(123456789012.dkr.ecr.us-east-1.amazonaws.com/nginx-application123:development-716b4e55dd3235f6548d645af9e463e744d3785f) (NONE->STOPPED)] for task: dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->STOPPED) Containers: [nginx-application123 (NONE->STOPPED),app-application123 (NONE->STOPPED),~internal~ecs~pause (RUNNING->STOPPED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]
  190. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Updating task: [dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->STOPPED) Containers: [nginx-application123 (NONE->STOPPED),app-application123 (NONE->STOPPED),~internal~ecs~pause (RUNNING->STOPPED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]]
  191. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] api/task: Updating task desired status to stopped because of container: [nginx-application123]; task: [dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->STOPPED) Containers: [nginx-application123 (NONE->STOPPED),app-application123 (NONE->STOPPED),~internal~ecs~pause (RUNNING->STOPPED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]]
  192. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] api/task: Updating task desired status to stopped because of container: [app-application123]; task: [dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->STOPPED) Containers: [nginx-application123 (NONE->STOPPED),app-application123 (NONE->STOPPED),~internal~ecs~pause (RUNNING->STOPPED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]]
  193. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] api/task: Updating task desired status to stopped because of container: [~internal~ecs~pause]; task: [dev-application123-api-td:360 arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c, TaskStatus: (NONE->STOPPED) Containers: [nginx-application123 (NONE->STOPPED),app-application123 (NONE->STOPPED),~internal~ecs~pause (RUNNING->STOPPED), ENI: [eni id:eni-02ce4ecc66dc7a812, mac: 0a:05:89:1b:0e:0a, hostname: ip-10-3-11-128.ec2.internal, ipv4addresses: [10.3.11.128], ipv6addresses: [], dns: [], dns search: [], gateway ipv4: [10.3.11.1/24][ ,VLan ID: [], TrunkInterfaceMacAddress: []]]]]
  194. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: task not steady state or terminal; progressing it
  195. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: progressing containers and resources in task
  196. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: resource [cgroup] has already transitioned to or beyond the desired status REMOVED; current known is REMOVED
  197. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: can't apply state to container [nginx-application123] yet due to unresolved dependencies: dependency graph: dependency on containers not resolved
  198. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: can't apply state to container [app-application123] yet due to unresolved dependencies: dependency graph: dependency on containers not resolved
  199. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] Managed task [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: waiting for event for task
  200. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] [ECSCNI] Starting clean up the container namespace: d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8
  201. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] [ECSCNI] bridge cleanup done: d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8
  202. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] [ECSCNI] container namespace cleanup done: d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8
  203. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [INFO] Task engine [arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c]: cleaned pause container network namespace
  204. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] DockerGoClient: got event from docker daemon: &{die d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8 amazon/amazon-ecs-pause:0.1.0 container die {d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8 map[com.amazonaws.ecs.cluster:dev-cluster com.amazonaws.ecs.container-name:~internal~ecs~pause com.amazonaws.ecs.task-arn:arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c com.amazonaws.ecs.task-definition-family:dev-application123-api-td com.amazonaws.ecs.task-definition-version:360 exitCode:0 image:amazon/amazon-ecs-pause:0.1.0 name:ecs-dev-application123-api-td-360-internalecspause-b6f389d4a89b82f64700]} local 1568553648 1568553648368146427}
  205. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] Udev watcher event-handler: add interface: dev8187
  206. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [WARN] Udev watcher event-handler: unable to send state change: udev watcher send ENI state change: eni status already sent: ENI Attachment: task=arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c;attachment=arn:aws:ecs:us-east-1:123456789012:attachment/d35c5ba2-e3a1-42c1-8aaa-96f95d879e37;attachmentType=task-eni;attachmentSent=true;mac=0a:05:89:1b:0e:0a;status=ATTACHED;expiresAt=2019-09-15 13:23:41.227378897 +0000 UTC m=+458.248372456
  207. dev-cluster/ecs-agents i-12345678901234567 2019-09-15T13:20:48Z [DEBUG] DockerGoClient: got event from docker daemon: &{stop d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8 amazon/amazon-ecs-pause:0.1.0 container stop {d493a4d4a8b7331bce92f798d195009cd38278bce7cbb5e2bb6cc49d32e219e8 map[com.amazonaws.ecs.cluster:dev-cluster com.amazonaws.ecs.container-name:~internal~ecs~pause com.amazonaws.ecs.task-arn:arn:aws:ecs:us-east-1:123456789012:task/ea1d6e4b-ff9f-4e0a-b77a-1698721faa5c com.amazonaws.ecs.task-definition-family:dev-application123-api-td com.amazonaws.ecs.task-definition-version:360 image:amazon/amazon-ecs-pause:0.1.0 name:ecs-dev-application123-api-td-360-internalecspause-b6f389d4a89b82f64700]} local 1568553648 1568553648475442372}
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!