API tools faq
paste
Advertisement
log2akshat

Could not connect to ZooKeeper

log2akshat
Sep 6th, 2018
152
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 396.66 KB | None | 0 0
raw download clone embed print report
  1. ==> /opt/zimbra/log/mailbox.log <==
  2. 2018-09-06 13:26:30,921 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=41962;soapId=4e502178;] misc - delegated access: doc=ModifyAdminSavedSearches, authenticated account=admin@zmc.com, target account=admin@zmc.com
  3. 2018-09-06 13:26:30,923 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=41962;soapId=4e502178;] soap - ModifyAdminSavedSearchesRequest elapsed=2
  4. 2018-09-06 13:26:30,938 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53648;soapId=4e502179;] misc - delegated access: doc=GetAdminSavedSearches, authenticated account=admin@zmc.com, target account=admin@zmc.com
  5. 2018-09-06 13:26:30,938 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53648;soapId=4e502179;] soap - GetAdminSavedSearchesRequest elapsed=0
  6. 2018-09-06 13:26:31,068 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42004;soapId=4e50217a;] soap - AuthRequest elapsed=1
  7. 2018-09-06 13:26:31,093 WARN [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53690;soapId=4e50217b;] event - Shutdown called for Event Notifier Executor; initiating shutdown sequence...
  8. 2018-09-06 13:26:31,093 INFO [EventLogger-Worker-Thread-3] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@d58437c
  9. 2018-09-06 13:26:31,093 INFO [EventLogger-Worker-Thread-2] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@17f52aa5
  10. 2018-09-06 13:26:31,093 INFO [EventLogger-Worker-Thread-0] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@7b829f20
  11. 2018-09-06 13:26:31,093 INFO [EventLogger-Worker-Thread-1] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@fc8da44
  12. 2018-09-06 13:26:31,093 INFO [EventLogger-Worker-Thread-4] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@457c4e46
  13. 2018-09-06 13:26:31,093 INFO [EventLogger-Worker-Thread-5] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@7d6fc8aa
  14. 2018-09-06 13:26:31,093 INFO [EventLogger-Worker-Thread-7] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@6ae67c40
  15. 2018-09-06 13:26:31,093 INFO [EventLogger-Worker-Thread-9] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@13335d9
  16. 2018-09-06 13:26:31,093 INFO [EventLogger-Worker-Thread-8] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@6b5c3562
  17. 2018-09-06 13:26:31,093 INFO [EventLogger-Worker-Thread-6] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@4bae907d
  18. 2018-09-06 13:26:31,094 INFO [EventLogger-Worker-Thread-3] [] ZooKeeper - Session: 0x0 closed
  19. 2018-09-06 13:26:31,094 INFO [EventLogger-Worker-Thread-3-EventThread] [] ClientCnxn - EventThread shut down
  20. 2018-09-06 13:26:31,094 ERROR [EventLogger-Worker-Thread-3] [] event - unable to send 1 events to Solr backend
  21. com.zimbra.common.service.ServiceException: system failure: unexpected error running command com.zimbra.cs.index.solr.SolrUtils$4
  22. ExceptionId:EventLogger-Worker-Thread-3:1536240391094:a4a5f9c6e5824c5f
  23. Code:service.FAILURE
  24. at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:288)
  25. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:39)
  26. at com.zimbra.cs.index.solr.SolrUtils.executeCloudRequestWithRetry(SolrUtils.java:320)
  27. at com.zimbra.cs.index.solr.SolrCloudHelper.executeUpdateRequest(SolrCloudHelper.java:42)
  28. at com.zimbra.cs.index.solr.SolrRequestHelper.executeUpdate(SolrRequestHelper.java:48)
  29. at com.zimbra.cs.event.logger.SolrEventCallback.execute(SolrEventCallback.java:34)
  30. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.send(BatchingEventLogger.java:126)
  31. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.access$000(BatchingEventLogger.java:103)
  32. at com.zimbra.cs.event.logger.BatchingEventLogger$1.onRemoval(BatchingEventLogger.java:53)
  33. at com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:2013)
  34. at com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3511)
  35. at com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3487)
  36. at com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3292)
  37. at com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
  38. at com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4829)
  39. at com.zimbra.cs.event.logger.BatchingEventLogger.sendAllBatched(BatchingEventLogger.java:136)
  40. at com.zimbra.cs.event.logger.BatchingEventLogger.shutdown(BatchingEventLogger.java:79)
  41. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.shutdownEventLogHandlers(EventLogger.java:244)
  42. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.run(EventLogger.java:215)
  43. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  44. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  45. at java.lang.Thread.run(Thread.java:748)
  46. Caused by: org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 within 10000 ms
  47. at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:182)
  48. at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:119)
  49. at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:109)
  50. at org.apache.solr.common.cloud.ZkStateReader.<init>(ZkStateReader.java:277)
  51. at org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:155)
  52. at org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:397)
  53. at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:826)
  54. at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
  55. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
  56. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
  57. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:299)
  58. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:295)
  59. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:22)
  60. ... 20 more
  61. Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 within 10000 ms
  62. at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:232)
  63. at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:174)
  64. ... 32 more
  65. 2018-09-06 13:26:31,095 INFO [EventLogger-Worker-Thread-4-SendThread(b3ecdace375e.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server b3ecdace375e.zm-docker_default/10.0.0.7:2181. Will not attempt to authenticate using SASL (unknown error)
  66. 2018-09-06 13:26:31,095 INFO [EventLogger-Worker-Thread-5-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181. Will not attempt to authenticate using SASL (unknown error)
  67. 2018-09-06 13:26:31,095 INFO [EventLogger-Worker-Thread-2-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181. Will not attempt to authenticate using SASL (unknown error)
  68. 2018-09-06 13:26:31,095 INFO [EventLogger-Worker-Thread-3-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181. Will not attempt to authenticate using SASL (unknown error)
  69. 2018-09-06 13:26:31,096 INFO [EventLogger-Worker-Thread-4-SendThread(b3ecdace375e.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to b3ecdace375e.zm-docker_default/10.0.0.7:2181, initiating session
  70. 2018-09-06 13:26:31,096 INFO [EventLogger-Worker-Thread-2-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, initiating session
  71. 2018-09-06 13:26:31,096 INFO [EventLogger-Worker-Thread-5-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, initiating session
  72. 2018-09-06 13:26:31,097 INFO [EventLogger-Worker-Thread-1-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181. Will not attempt to authenticate using SASL (unknown error)
  73. 2018-09-06 13:26:31,097 INFO [EventLogger-Worker-Thread-0-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181. Will not attempt to authenticate using SASL (unknown error)
  74. 2018-09-06 13:26:31,097 INFO [EventLogger-Worker-Thread-7-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181. Will not attempt to authenticate using SASL (unknown error)
  75. 2018-09-06 13:26:31,097 INFO [EventLogger-Worker-Thread-3-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, initiating session
  76. 2018-09-06 13:26:31,097 INFO [EventLogger-Worker-Thread-8-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181. Will not attempt to authenticate using SASL (unknown error)
  77. 2018-09-06 13:26:31,097 INFO [EventLogger-Worker-Thread-1-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, initiating session
  78. 2018-09-06 13:26:31,097 INFO [EventLogger-Worker-Thread-7-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181, initiating session
  79. 2018-09-06 13:26:31,097 INFO [EventLogger-Worker-Thread-6-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181. Will not attempt to authenticate using SASL (unknown error)
  80. 2018-09-06 13:26:31,097 INFO [EventLogger-Worker-Thread-9-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181. Will not attempt to authenticate using SASL (unknown error)
  81. 2018-09-06 13:26:31,097 INFO [EventLogger-Worker-Thread-0-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, initiating session
  82. 2018-09-06 13:26:31,098 INFO [EventLogger-Worker-Thread-9-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, initiating session
  83. 2018-09-06 13:26:31,098 INFO [EventLogger-Worker-Thread-6-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181, initiating session
  84. 2018-09-06 13:26:31,098 INFO [EventLogger-Worker-Thread-8-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181, initiating session
  85. 2018-09-06 13:26:31,099 INFO [EventLogger-Worker-Thread-2-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, sessionid = 0x20000d48aa80025, negotiated timeout = 10000
  86. 2018-09-06 13:26:31,099 INFO [EventLogger-Worker-Thread-5-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, sessionid = 0x10000d48b750026, negotiated timeout = 10000
  87. 2018-09-06 13:26:31,099 INFO [zkConnectionManagerCallback-303-thread-1] [] ConnectionManager - zkClient has connected
  88. 2018-09-06 13:26:31,099 INFO [zkConnectionManagerCallback-313-thread-1] [] ConnectionManager - zkClient has connected
  89. 2018-09-06 13:26:31,099 INFO [EventLogger-Worker-Thread-4-SendThread(b3ecdace375e.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server b3ecdace375e.zm-docker_default/10.0.0.7:2181, sessionid = 0x20000d48aa80026, negotiated timeout = 10000
  90. 2018-09-06 13:26:31,101 INFO [EventLogger-Worker-Thread-1-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, sessionid = 0x20000d48aa80027, negotiated timeout = 10000
  91. 2018-09-06 13:26:31,101 INFO [zkConnectionManagerCallback-299-thread-1] [] ConnectionManager - zkClient has connected
  92. 2018-09-06 13:26:31,101 INFO [zkConnectionManagerCallback-309-thread-1] [] ConnectionManager - zkClient has connected
  93. 2018-09-06 13:26:31,102 INFO [EventLogger-Worker-Thread-7-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181, sessionid = 0x30000d48aad001f, negotiated timeout = 10000
  94. 2018-09-06 13:26:31,102 INFO [EventLogger-Worker-Thread-0-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, sessionid = 0x10000d48b750027, negotiated timeout = 10000
  95. 2018-09-06 13:26:31,102 INFO [EventLogger-Worker-Thread-9-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, sessionid = 0x20000d48aa80028, negotiated timeout = 10000
  96. 2018-09-06 13:26:31,102 INFO [zkConnectionManagerCallback-310-thread-1] [] ConnectionManager - zkClient has connected
  97. 2018-09-06 13:26:31,103 INFO [EventLogger-Worker-Thread-5] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  98. 2018-09-06 13:26:31,103 INFO [zkConnectionManagerCallback-304-thread-1] [] ConnectionManager - zkClient has connected
  99. 2018-09-06 13:26:31,103 INFO [EventLogger-Worker-Thread-8-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181, sessionid = 0x30000d48aad0021, negotiated timeout = 10000
  100. 2018-09-06 13:26:31,102 INFO [EventLogger-Worker-Thread-6-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181, sessionid = 0x30000d48aad0020, negotiated timeout = 10000
  101. 2018-09-06 13:26:31,103 INFO [zkConnectionManagerCallback-316-thread-1] [] ConnectionManager - zkClient has connected
  102. 2018-09-06 13:26:31,105 INFO [zkConnectionManagerCallback-319-thread-1] [] ConnectionManager - zkClient has connected
  103. 2018-09-06 13:26:31,105 INFO [zkConnectionManagerCallback-302-thread-1] [] ConnectionManager - zkClient has connected
  104. 2018-09-06 13:26:31,106 INFO [EventLogger-Worker-Thread-0] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  105. 2018-09-06 13:26:31,106 INFO [EventLogger-Worker-Thread-4] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  106. 2018-09-06 13:26:31,106 INFO [EventLogger-Worker-Thread-1] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  107. 2018-09-06 13:26:31,106 INFO [EventLogger-Worker-Thread-5] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  108. 2018-09-06 13:26:31,107 INFO [EventLogger-Worker-Thread-2] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  109. 2018-09-06 13:26:31,107 INFO [EventLogger-Worker-Thread-9] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  110. 2018-09-06 13:26:31,107 INFO [EventLogger-Worker-Thread-8] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  111. 2018-09-06 13:26:31,107 INFO [EventLogger-Worker-Thread-6] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  112. 2018-09-06 13:26:31,107 INFO [EventLogger-Worker-Thread-4] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  113. 2018-09-06 13:26:31,107 INFO [EventLogger-Worker-Thread-0] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  114. 2018-09-06 13:26:31,107 INFO [EventLogger-Worker-Thread-7] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  115. 2018-09-06 13:26:31,109 INFO [EventLogger-Worker-Thread-1] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  116. 2018-09-06 13:26:31,109 INFO [EventLogger-Worker-Thread-2] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  117. 2018-09-06 13:26:31,109 INFO [EventLogger-Worker-Thread-9] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  118. 2018-09-06 13:26:31,109 INFO [EventLogger-Worker-Thread-6] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  119. 2018-09-06 13:26:31,109 INFO [EventLogger-Worker-Thread-7] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  120. 2018-09-06 13:26:31,110 INFO [EventLogger-Worker-Thread-8] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  121. 2018-09-06 13:26:31,124 INFO [EventLogger-Worker-Thread-6] [] ZooKeeper - Session: 0x30000d48aad0020 closed
  122. 2018-09-06 13:26:31,124 INFO [EventLogger-Worker-Thread-6-EventThread] [] ClientCnxn - EventThread shut down
  123. 2018-09-06 13:26:31,125 INFO [EventLogger-Worker-Thread-5] [] ZooKeeper - Session: 0x10000d48b750026 closed
  124. 2018-09-06 13:26:31,125 INFO [EventLogger-Worker-Thread-7] [] ZooKeeper - Session: 0x30000d48aad001f closed
  125. 2018-09-06 13:26:31,125 INFO [EventLogger-Worker-Thread-7-EventThread] [] ClientCnxn - EventThread shut down
  126. 2018-09-06 13:26:31,125 INFO [EventLogger-Worker-Thread-8] [] ZooKeeper - Session: 0x30000d48aad0021 closed
  127. 2018-09-06 13:26:31,125 INFO [EventLogger-Worker-Thread-8-EventThread] [] ClientCnxn - EventThread shut down
  128. 2018-09-06 13:26:31,126 INFO [EventLogger-Worker-Thread-5-EventThread] [] ClientCnxn - EventThread shut down
  129. 2018-09-06 13:26:31,126 INFO [EventLogger-Worker-Thread-9] [] ZooKeeper - Session: 0x20000d48aa80028 closed
  130. 2018-09-06 13:26:31,126 INFO [EventLogger-Worker-Thread-9-EventThread] [] ClientCnxn - EventThread shut down
  131. 2018-09-06 13:26:31,133 INFO [EventLogger-Worker-Thread-4] [] ZooKeeper - Session: 0x20000d48aa80026 closed
  132. 2018-09-06 13:26:31,133 INFO [EventLogger-Worker-Thread-4-EventThread] [] ClientCnxn - EventThread shut down
  133. 2018-09-06 13:26:31,137 INFO [EventLogger-Worker-Thread-0] [] ZooKeeper - Session: 0x10000d48b750027 closed
  134. 2018-09-06 13:26:31,137 INFO [EventLogger-Worker-Thread-0-EventThread] [] ClientCnxn - EventThread shut down
  135. 2018-09-06 13:26:31,149 INFO [EventLogger-Worker-Thread-1] [] ZooKeeper - Session: 0x20000d48aa80027 closed
  136. 2018-09-06 13:26:31,149 INFO [EventLogger-Worker-Thread-1-EventThread] [] ClientCnxn - EventThread shut down
  137. 2018-09-06 13:26:31,150 INFO [EventLogger-Worker-Thread-2] [] ZooKeeper - Session: 0x20000d48aa80025 closed
  138. 2018-09-06 13:26:31,150 INFO [EventLogger-Worker-Thread-2-EventThread] [] ClientCnxn - EventThread shut down
  139. 2018-09-06 13:26:31,150 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53690;soapId=4e50217b;] event - Event Notifier Executor shutdown was successful
  140. 2018-09-06 13:26:31,150 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53690;soapId=4e50217b;] event - Event Queue Size 0
  141. 2018-09-06 13:26:31,150 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53690;soapId=4e50217b;] event - Starting Event Notifier Logger with 10 threads; initial event queue size is 0
  142. 2018-09-06 13:26:31,152 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53690;soapId=4e50217b;] soap - CreateCosRequest elapsed=67
  143. 2018-09-06 13:26:31,232 WARN [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42056;soapId=4e50217c;] event - Shutdown called for Event Notifier Executor; initiating shutdown sequence...
  144. 2018-09-06 13:26:31,232 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42056;soapId=4e50217c;] event - Event Notifier Executor shutdown was successful
  145. 2018-09-06 13:26:31,232 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42056;soapId=4e50217c;] event - Event Queue Size 0
  146. 2018-09-06 13:26:31,232 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42056;soapId=4e50217c;] event - Starting Event Notifier Logger with 10 threads; initial event queue size is 0
  147. 2018-09-06 13:26:31,234 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42056;soapId=4e50217c;] soap - CreateCosRequest elapsed=10
  148. 2018-09-06 13:26:31,257 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53742;soapId=4e50217d;] soap - GetCosRequest elapsed=2
  149. 2018-09-06 13:26:31,346 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42066;soapId=4e50217e;] soap - ModifyCosRequest elapsed=6
  150. 2018-09-06 13:26:31,368 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53752;soapId=4e50217f;] soap - AuthRequest elapsed=0
  151. 2018-09-06 13:26:31,442 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42074;soapId=4e502180;] soap - AuthRequest elapsed=1
  152. 2018-09-06 13:26:31,464 WARN [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53760;soapId=4e502181;] event - Shutdown called for Event Notifier Executor; initiating shutdown sequence...
  153. 2018-09-06 13:26:31,464 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53760;soapId=4e502181;] event - Event Notifier Executor shutdown was successful
  154. 2018-09-06 13:26:31,464 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53760;soapId=4e502181;] event - Event Queue Size 0
  155. 2018-09-06 13:26:31,464 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53760;soapId=4e502181;] event - Starting Event Notifier Logger with 10 threads; initial event queue size is 0
  156. 2018-09-06 13:26:31,466 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53760;soapId=4e502181;] soap - CreateCosRequest elapsed=10
  157. 2018-09-06 13:26:31,548 WARN [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42082;soapId=4e502182;] event - Shutdown called for Event Notifier Executor; initiating shutdown sequence...
  158. 2018-09-06 13:26:31,549 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42082;soapId=4e502182;] event - Event Notifier Executor shutdown was successful
  159. 2018-09-06 13:26:31,549 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42082;soapId=4e502182;] event - Event Queue Size 0
  160. 2018-09-06 13:26:31,549 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42082;soapId=4e502182;] event - Starting Event Notifier Logger with 10 threads; initial event queue size is 0
  161. 2018-09-06 13:26:31,551 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42082;soapId=4e502182;] soap - CreateCosRequest elapsed=10
  162. 2018-09-06 13:26:31,571 INFO [qtp788117692-1540:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53768;soapId=4e502183;] soap - ModifyDelegatedAdminConstraintsRequest elapsed=3
  163. 2018-09-06 13:26:31,652 INFO [qtp788117692-1540:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42090;soapId=4e502184;] soap - GetAllCosRequest elapsed=35
  164.  
  165. ==> /opt/zimbra/log/audit.log <==
  166. 2018-09-06 13:26:31,067 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42004;soapId=4e50217a;] security - cmd=AdminAuth; account=admin@zmc.com;
  167. 2018-09-06 13:26:31,068 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42004;soapId=4e50217a;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  168. 2018-09-06 13:26:31,152 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53690;soapId=4e50217b;] security - cmd=CreateCos; name=cos153624039094327;
  169. 2018-09-06 13:26:31,233 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42056;soapId=4e50217c;] security - cmd=CreateCos; name=cos153624039094328;
  170. 2018-09-06 13:26:31,345 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42066;soapId=4e50217e;] security - cmd=ModifyCos; name=cos153624039094329; zimbraMailQuota=20;
  171. 2018-09-06 13:26:31,368 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53752;soapId=4e50217f;] security - cmd=AdminAuth; account=admin@zmc.com;
  172. 2018-09-06 13:26:31,368 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53752;soapId=4e50217f;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  173. 2018-09-06 13:26:31,441 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42074;soapId=4e502180;] security - cmd=AdminAuth; account=admin@zmc.com;
  174. 2018-09-06 13:26:31,442 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42074;soapId=4e502180;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  175. 2018-09-06 13:26:31,465 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53760;soapId=4e502181;] security - cmd=CreateCos; name=cos153624039094431;
  176. 2018-09-06 13:26:31,550 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42082;soapId=4e502182;] security - cmd=CreateCos; name=cos153624039094432;
  177. 2018-09-06 13:26:31,571 INFO [qtp788117692-1540:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53768;soapId=4e502183;] security - cmd=ModifyDelegatedAdminConstraints; name=cos153624039094432; zimbraConstraint=zimbraMailQuota:min=20971520:max=524288000;
  178.  
  179. ==> /opt/zimbra/log/mailbox.log <==
  180. 2018-09-06 13:26:30,959 INFO [qtp788117692-1565:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=41966;soapId=4faf0c55;] soap - AuthRequest elapsed=1
  181. 2018-09-06 13:26:30,984 WARN [qtp788117692-1565:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53652;soapId=4faf0c56;] event - Shutdown called for Event Notifier Executor; initiating shutdown sequence...
  182. 2018-09-06 13:26:30,984 INFO [EventLogger-Worker-Thread-0] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@7c2cf62
  183. 2018-09-06 13:26:30,984 INFO [EventLogger-Worker-Thread-7] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@387e9bd2
  184. 2018-09-06 13:26:30,984 INFO [EventLogger-Worker-Thread-5] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@5b76b6cd
  185. 2018-09-06 13:26:30,984 INFO [EventLogger-Worker-Thread-1] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@75fdb59f
  186. 2018-09-06 13:26:30,984 INFO [EventLogger-Worker-Thread-8] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@e413254
  187. 2018-09-06 13:26:30,984 ERROR [EventLogger-Worker-Thread-2] [] event - unable to send 1 events to Solr backend
  188. com.zimbra.common.service.ServiceException: system failure: unexpected error running command com.zimbra.cs.index.solr.SolrUtils$4
  189. ExceptionId:EventLogger-Worker-Thread-2:1536240390984:33d5d0aceb48fe53
  190. Code:service.FAILURE
  191. at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:288)
  192. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:39)
  193. at com.zimbra.cs.index.solr.SolrUtils.executeCloudRequestWithRetry(SolrUtils.java:320)
  194. at com.zimbra.cs.index.solr.SolrCloudHelper.executeUpdateRequest(SolrCloudHelper.java:42)
  195. at com.zimbra.cs.index.solr.SolrRequestHelper.executeUpdate(SolrRequestHelper.java:48)
  196. at com.zimbra.cs.event.logger.SolrEventCallback.execute(SolrEventCallback.java:34)
  197. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.send(BatchingEventLogger.java:126)
  198. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.access$000(BatchingEventLogger.java:103)
  199. at com.zimbra.cs.event.logger.BatchingEventLogger$1.onRemoval(BatchingEventLogger.java:53)
  200. at com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:2013)
  201. at com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3511)
  202. at com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3487)
  203. at com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3292)
  204. at com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
  205. at com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4829)
  206. at com.zimbra.cs.event.logger.BatchingEventLogger.sendAllBatched(BatchingEventLogger.java:136)
  207. at com.zimbra.cs.event.logger.BatchingEventLogger.shutdown(BatchingEventLogger.java:79)
  208. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.shutdownEventLogHandlers(EventLogger.java:244)
  209. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.run(EventLogger.java:215)
  210. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  211. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  212. at java.lang.Thread.run(Thread.java:748)
  213. Caused by: org.apache.solr.common.SolrException: Could not load collection from ZK: event_index_data
  214. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1126)
  215. at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
  216. at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1205)
  217. at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:848)
  218. at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
  219. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
  220. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
  221. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:299)
  222. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:295)
  223. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:22)
  224. ... 20 more
  225. Caused by: java.lang.InterruptedException
  226. at java.lang.Object.wait(Native Method)
  227. at java.lang.Object.wait(Object.java:502)
  228. at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1309)
  229. at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1149)
  230. at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
  231. at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
  232. at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
  233. at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1135)
  234. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1121)
  235. ... 29 more
  236. 2018-09-06 13:26:30,984 ERROR [EventLogger-Worker-Thread-3] [] event - unable to send 1 events to Solr backend
  237. com.zimbra.common.service.ServiceException: system failure: unexpected error running command com.zimbra.cs.index.solr.SolrUtils$4
  238. ExceptionId:EventLogger-Worker-Thread-3:1536240390984:33d5d0aceb48fe53
  239. Code:service.FAILURE
  240. at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:288)
  241. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:39)
  242. at com.zimbra.cs.index.solr.SolrUtils.executeCloudRequestWithRetry(SolrUtils.java:320)
  243. at com.zimbra.cs.index.solr.SolrCloudHelper.executeUpdateRequest(SolrCloudHelper.java:42)
  244. at com.zimbra.cs.index.solr.SolrRequestHelper.executeUpdate(SolrRequestHelper.java:48)
  245. at com.zimbra.cs.event.logger.SolrEventCallback.execute(SolrEventCallback.java:34)
  246. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.send(BatchingEventLogger.java:126)
  247. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.access$000(BatchingEventLogger.java:103)
  248. at com.zimbra.cs.event.logger.BatchingEventLogger$1.onRemoval(BatchingEventLogger.java:53)
  249. at com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:2013)
  250. at com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3511)
  251. at com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3487)
  252. at com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3292)
  253. at com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
  254. at com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4829)
  255. at com.zimbra.cs.event.logger.BatchingEventLogger.sendAllBatched(BatchingEventLogger.java:136)
  256. at com.zimbra.cs.event.logger.BatchingEventLogger.shutdown(BatchingEventLogger.java:79)
  257. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.shutdownEventLogHandlers(EventLogger.java:244)
  258. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.run(EventLogger.java:215)
  259. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  260. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  261. at java.lang.Thread.run(Thread.java:748)
  262. Caused by: org.apache.solr.common.SolrException: Could not load collection from ZK: event_index_data
  263. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1126)
  264. at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
  265. at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1205)
  266. at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:848)
  267. at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
  268. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
  269. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
  270. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:299)
  271. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:295)
  272. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:22)
  273. ... 20 more
  274. Caused by: java.lang.InterruptedException
  275. at java.lang.Object.wait(Native Method)
  276. at java.lang.Object.wait(Object.java:502)
  277. at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1309)
  278. at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1149)
  279. at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
  280. at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
  281. at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
  282. at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1135)
  283. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1121)
  284. ... 29 more
  285. 2018-09-06 13:26:30,985 ERROR [EventLogger-Worker-Thread-2] [] event - unable to send 1 events to Solr backend
  286. com.zimbra.common.service.ServiceException: system failure: unexpected error running command com.zimbra.cs.index.solr.SolrUtils$4
  287. ExceptionId:EventLogger-Worker-Thread-2:1536240390985:33d5d0aceb48fe53
  288. Code:service.FAILURE
  289. at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:288)
  290. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:39)
  291. at com.zimbra.cs.index.solr.SolrUtils.executeCloudRequestWithRetry(SolrUtils.java:320)
  292. at com.zimbra.cs.index.solr.SolrCloudHelper.executeUpdateRequest(SolrCloudHelper.java:42)
  293. at com.zimbra.cs.index.solr.SolrRequestHelper.executeUpdate(SolrRequestHelper.java:48)
  294. at com.zimbra.cs.event.logger.SolrEventCallback.execute(SolrEventCallback.java:34)
  295. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.send(BatchingEventLogger.java:126)
  296. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.access$000(BatchingEventLogger.java:103)
  297. at com.zimbra.cs.event.logger.BatchingEventLogger$1.onRemoval(BatchingEventLogger.java:53)
  298. at com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:2013)
  299. at com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3511)
  300. at com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3487)
  301. at com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3292)
  302. at com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
  303. at com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4829)
  304. at com.zimbra.cs.event.logger.BatchingEventLogger.sendAllBatched(BatchingEventLogger.java:136)
  305. at com.zimbra.cs.event.logger.BatchingEventLogger.shutdown(BatchingEventLogger.java:79)
  306. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.shutdownEventLogHandlers(EventLogger.java:244)
  307. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.run(EventLogger.java:215)
  308. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  309. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  310. at java.lang.Thread.run(Thread.java:748)
  311. Caused by: org.apache.solr.common.SolrException: Could not load collection from ZK: event_index_data
  312. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1126)
  313. at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
  314. at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1205)
  315. at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:848)
  316. at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
  317. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
  318. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
  319. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:299)
  320. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:295)
  321. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:22)
  322. ... 20 more
  323. Caused by: java.lang.InterruptedException
  324. at java.lang.Object.wait(Native Method)
  325. at java.lang.Object.wait(Object.java:502)
  326. at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1309)
  327. at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1149)
  328. at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
  329. at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
  330. at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
  331. at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1135)
  332. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1121)
  333. ... 29 more
  334. 2018-09-06 13:26:30,985 ERROR [EventLogger-Worker-Thread-4] [] event - unable to send 1 events to Solr backend
  335. com.zimbra.common.service.ServiceException: system failure: unexpected error running command com.zimbra.cs.index.solr.SolrUtils$4
  336. ExceptionId:EventLogger-Worker-Thread-4:1536240390985:33d5d0aceb48fe53
  337. Code:service.FAILURE
  338. at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:288)
  339. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:39)
  340. at com.zimbra.cs.index.solr.SolrUtils.executeCloudRequestWithRetry(SolrUtils.java:320)
  341. at com.zimbra.cs.index.solr.SolrCloudHelper.executeUpdateRequest(SolrCloudHelper.java:42)
  342. at com.zimbra.cs.index.solr.SolrRequestHelper.executeUpdate(SolrRequestHelper.java:48)
  343. at com.zimbra.cs.event.logger.SolrEventCallback.execute(SolrEventCallback.java:34)
  344. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.send(BatchingEventLogger.java:126)
  345. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.access$000(BatchingEventLogger.java:103)
  346. at com.zimbra.cs.event.logger.BatchingEventLogger$1.onRemoval(BatchingEventLogger.java:53)
  347. at com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:2013)
  348. at com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3511)
  349. at com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3487)
  350. at com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3292)
  351. at com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
  352. at com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4829)
  353. at com.zimbra.cs.event.logger.BatchingEventLogger.sendAllBatched(BatchingEventLogger.java:136)
  354. at com.zimbra.cs.event.logger.BatchingEventLogger.shutdown(BatchingEventLogger.java:79)
  355. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.shutdownEventLogHandlers(EventLogger.java:244)
  356. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.run(EventLogger.java:215)
  357. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  358. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  359. at java.lang.Thread.run(Thread.java:748)
  360. Caused by: org.apache.solr.common.SolrException: Could not load collection from ZK: event_index_data
  361. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1126)
  362. at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
  363. at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1205)
  364. at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:848)
  365. at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
  366. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
  367. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
  368. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:299)
  369. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:295)
  370. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:22)
  371. ... 20 more
  372. Caused by: java.lang.InterruptedException
  373. at java.lang.Object.wait(Native Method)
  374. at java.lang.Object.wait(Object.java:502)
  375. at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1309)
  376. at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1149)
  377. at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
  378. at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
  379. at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
  380. at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1135)
  381. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1121)
  382. ... 29 more
  383. 2018-09-06 13:26:30,985 ERROR [EventLogger-Worker-Thread-6] [] event - unable to send 2 events to Solr backend
  384. com.zimbra.common.service.ServiceException: system failure: unexpected error running command com.zimbra.cs.index.solr.SolrUtils$4
  385. ExceptionId:EventLogger-Worker-Thread-6:1536240390985:33d5d0aceb48fe53
  386. Code:service.FAILURE
  387. at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:288)
  388. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:39)
  389. at com.zimbra.cs.index.solr.SolrUtils.executeCloudRequestWithRetry(SolrUtils.java:320)
  390. at com.zimbra.cs.index.solr.SolrCloudHelper.executeUpdateRequest(SolrCloudHelper.java:42)
  391. at com.zimbra.cs.index.solr.SolrRequestHelper.executeUpdate(SolrRequestHelper.java:48)
  392. at com.zimbra.cs.event.logger.SolrEventCallback.execute(SolrEventCallback.java:34)
  393. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.send(BatchingEventLogger.java:126)
  394. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.access$000(BatchingEventLogger.java:103)
  395. at com.zimbra.cs.event.logger.BatchingEventLogger$1.onRemoval(BatchingEventLogger.java:53)
  396. at com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:2013)
  397. at com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3511)
  398. at com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3487)
  399. at com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3292)
  400. at com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
  401. at com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4829)
  402. at com.zimbra.cs.event.logger.BatchingEventLogger.sendAllBatched(BatchingEventLogger.java:136)
  403. at com.zimbra.cs.event.logger.BatchingEventLogger.shutdown(BatchingEventLogger.java:79)
  404. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.shutdownEventLogHandlers(EventLogger.java:244)
  405. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.run(EventLogger.java:215)
  406. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  407. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  408. at java.lang.Thread.run(Thread.java:748)
  409. Caused by: org.apache.solr.common.SolrException: Could not load collection from ZK: event_index_data
  410. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1126)
  411. at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
  412. at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1205)
  413. at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:848)
  414. at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
  415. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
  416. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
  417. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:299)
  418. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:295)
  419. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:22)
  420. ... 20 more
  421. Caused by: java.lang.InterruptedException
  422. at java.lang.Object.wait(Native Method)
  423. at java.lang.Object.wait(Object.java:502)
  424. at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1309)
  425. at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1149)
  426. at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
  427. at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
  428. at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
  429. at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1135)
  430. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1121)
  431. ... 29 more
  432. 2018-09-06 13:26:30,984 ERROR [EventLogger-Worker-Thread-9] [] event - unable to send 2 events to Solr backend
  433. com.zimbra.common.service.ServiceException: system failure: unexpected error running command com.zimbra.cs.index.solr.SolrUtils$4
  434. ExceptionId:EventLogger-Worker-Thread-9:1536240390984:33d5d0aceb48fe53
  435. Code:service.FAILURE
  436. at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:288)
  437. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:39)
  438. at com.zimbra.cs.index.solr.SolrUtils.executeCloudRequestWithRetry(SolrUtils.java:320)
  439. at com.zimbra.cs.index.solr.SolrCloudHelper.executeUpdateRequest(SolrCloudHelper.java:42)
  440. at com.zimbra.cs.index.solr.SolrRequestHelper.executeUpdate(SolrRequestHelper.java:48)
  441. at com.zimbra.cs.event.logger.SolrEventCallback.execute(SolrEventCallback.java:34)
  442. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.send(BatchingEventLogger.java:126)
  443. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.access$000(BatchingEventLogger.java:103)
  444. at com.zimbra.cs.event.logger.BatchingEventLogger$1.onRemoval(BatchingEventLogger.java:53)
  445. at com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:2013)
  446. at com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3511)
  447. at com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3487)
  448. at com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3292)
  449. at com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
  450. at com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4829)
  451. at com.zimbra.cs.event.logger.BatchingEventLogger.sendAllBatched(BatchingEventLogger.java:136)
  452. at com.zimbra.cs.event.logger.BatchingEventLogger.shutdown(BatchingEventLogger.java:79)
  453. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.shutdownEventLogHandlers(EventLogger.java:244)
  454. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.run(EventLogger.java:215)
  455. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  456. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  457. at java.lang.Thread.run(Thread.java:748)
  458. Caused by: org.apache.solr.common.SolrException: Could not load collection from ZK: event_index_data
  459. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1126)
  460. at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
  461. at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1205)
  462. at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:848)
  463. at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
  464. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
  465. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
  466. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:299)
  467. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:295)
  468. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:22)
  469. ... 20 more
  470. Caused by: java.lang.InterruptedException
  471. at java.lang.Object.wait(Native Method)
  472. at java.lang.Object.wait(Object.java:502)
  473. at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1309)
  474. at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1149)
  475. at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
  476. at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
  477. at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
  478. at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1135)
  479. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1121)
  480. ... 29 more
  481. 2018-09-06 13:26:30,989 INFO [EventLogger-Worker-Thread-7-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181. Will not attempt to authenticate using SASL (unknown error)
  482. 2018-09-06 13:26:30,989 INFO [EventLogger-Worker-Thread-1-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181. Will not attempt to authenticate using SASL (unknown error)
  483. 2018-09-06 13:26:30,989 ERROR [EventLogger-Worker-Thread-2] [] event - unable to send 2 events to Solr backend
  484. com.zimbra.common.service.ServiceException: system failure: unexpected error running command com.zimbra.cs.index.solr.SolrUtils$4
  485. ExceptionId:EventLogger-Worker-Thread-2:1536240390988:33d5d0aceb48fe53
  486. Code:service.FAILURE
  487. at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:288)
  488. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:39)
  489. at com.zimbra.cs.index.solr.SolrUtils.executeCloudRequestWithRetry(SolrUtils.java:320)
  490. at com.zimbra.cs.index.solr.SolrCloudHelper.executeUpdateRequest(SolrCloudHelper.java:42)
  491. at com.zimbra.cs.index.solr.SolrRequestHelper.executeUpdate(SolrRequestHelper.java:48)
  492. at com.zimbra.cs.event.logger.SolrEventCallback.execute(SolrEventCallback.java:34)
  493. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.send(BatchingEventLogger.java:126)
  494. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.access$000(BatchingEventLogger.java:103)
  495. at com.zimbra.cs.event.logger.BatchingEventLogger$1.onRemoval(BatchingEventLogger.java:53)
  496. at com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:2013)
  497. at com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3511)
  498. at com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3487)
  499. at com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3292)
  500. at com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
  501. at com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4829)
  502. at com.zimbra.cs.event.logger.BatchingEventLogger.sendAllBatched(BatchingEventLogger.java:136)
  503. at com.zimbra.cs.event.logger.BatchingEventLogger.shutdown(BatchingEventLogger.java:79)
  504. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.shutdownEventLogHandlers(EventLogger.java:244)
  505. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.run(EventLogger.java:215)
  506. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  507. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  508. at java.lang.Thread.run(Thread.java:748)
  509. Caused by: org.apache.solr.common.SolrException: Could not load collection from ZK: event_index_data
  510. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1126)
  511. at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
  512. at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1205)
  513. at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:848)
  514. at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
  515. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
  516. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
  517. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:299)
  518. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:295)
  519. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:22)
  520. ... 20 more
  521. Caused by: java.lang.InterruptedException
  522. at java.lang.Object.wait(Native Method)
  523. at java.lang.Object.wait(Object.java:502)
  524. at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1309)
  525. at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1149)
  526. at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
  527. at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
  528. at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
  529. at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1135)
  530. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1121)
  531. ... 29 more
  532. 2018-09-06 13:26:30,990 ERROR [EventLogger-Worker-Thread-6] [] event - unable to send 1 events to Solr backend
  533. com.zimbra.common.service.ServiceException: system failure: unexpected error running command com.zimbra.cs.index.solr.SolrUtils$4
  534. ExceptionId:EventLogger-Worker-Thread-6:1536240390990:33d5d0aceb48fe53
  535. Code:service.FAILURE
  536. at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:288)
  537. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:39)
  538. at com.zimbra.cs.index.solr.SolrUtils.executeCloudRequestWithRetry(SolrUtils.java:320)
  539. at com.zimbra.cs.index.solr.SolrCloudHelper.executeUpdateRequest(SolrCloudHelper.java:42)
  540. at com.zimbra.cs.index.solr.SolrRequestHelper.executeUpdate(SolrRequestHelper.java:48)
  541. at com.zimbra.cs.event.logger.SolrEventCallback.execute(SolrEventCallback.java:34)
  542. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.send(BatchingEventLogger.java:126)
  543. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.access$000(BatchingEventLogger.java:103)
  544. at com.zimbra.cs.event.logger.BatchingEventLogger$1.onRemoval(BatchingEventLogger.java:53)
  545. at com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:2013)
  546. at com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3511)
  547. at com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3487)
  548. at com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3292)
  549. at com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
  550. at com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4829)
  551. at com.zimbra.cs.event.logger.BatchingEventLogger.sendAllBatched(BatchingEventLogger.java:136)
  552. at com.zimbra.cs.event.logger.BatchingEventLogger.shutdown(BatchingEventLogger.java:79)
  553. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.shutdownEventLogHandlers(EventLogger.java:244)
  554. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.run(EventLogger.java:215)
  555. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  556. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  557. at java.lang.Thread.run(Thread.java:748)
  558. Caused by: org.apache.solr.common.SolrException: Could not load collection from ZK: event_index_data
  559. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1126)
  560. at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
  561. at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1205)
  562. at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:848)
  563. at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
  564. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
  565. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
  566. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:299)
  567. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:295)
  568. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:22)
  569. ... 20 more
  570. Caused by: java.lang.InterruptedException
  571. at java.lang.Object.wait(Native Method)
  572. at java.lang.Object.wait(Object.java:502)
  573. at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1309)
  574. at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1149)
  575. at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
  576. at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
  577. at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
  578. at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1135)
  579. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1121)
  580. ... 29 more
  581. 2018-09-06 13:26:30,988 ERROR [EventLogger-Worker-Thread-4] [] event - unable to send 1 events to Solr backend
  582. com.zimbra.common.service.ServiceException: system failure: unexpected error running command com.zimbra.cs.index.solr.SolrUtils$4
  583. ExceptionId:EventLogger-Worker-Thread-4:1536240390988:33d5d0aceb48fe53
  584. Code:service.FAILURE
  585. at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:288)
  586. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:39)
  587. at com.zimbra.cs.index.solr.SolrUtils.executeCloudRequestWithRetry(SolrUtils.java:320)
  588. at com.zimbra.cs.index.solr.SolrCloudHelper.executeUpdateRequest(SolrCloudHelper.java:42)
  589. at com.zimbra.cs.index.solr.SolrRequestHelper.executeUpdate(SolrRequestHelper.java:48)
  590. at com.zimbra.cs.event.logger.SolrEventCallback.execute(SolrEventCallback.java:34)
  591. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.send(BatchingEventLogger.java:126)
  592. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.access$000(BatchingEventLogger.java:103)
  593. at com.zimbra.cs.event.logger.BatchingEventLogger$1.onRemoval(BatchingEventLogger.java:53)
  594. at com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:2013)
  595. at com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3511)
  596. at com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3487)
  597. at com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3292)
  598. at com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
  599. at com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4829)
  600. at com.zimbra.cs.event.logger.BatchingEventLogger.sendAllBatched(BatchingEventLogger.java:136)
  601. at com.zimbra.cs.event.logger.BatchingEventLogger.shutdown(BatchingEventLogger.java:79)
  602. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.shutdownEventLogHandlers(EventLogger.java:244)
  603. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.run(EventLogger.java:215)
  604. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  605. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  606. at java.lang.Thread.run(Thread.java:748)
  607. Caused by: org.apache.solr.common.SolrException: Could not load collection from ZK: event_index_data
  608. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1126)
  609. at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
  610. at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1205)
  611. at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:848)
  612. at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
  613. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
  614. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
  615. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:299)
  616. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:295)
  617. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:22)
  618. ... 20 more
  619. Caused by: java.lang.InterruptedException
  620. at java.lang.Object.wait(Native Method)
  621. at java.lang.Object.wait(Object.java:502)
  622. at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1309)
  623. at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1149)
  624. at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
  625. at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
  626. at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
  627. at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1135)
  628. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1121)
  629. ... 29 more
  630. 2018-09-06 13:26:30,988 INFO [EventLogger-Worker-Thread-8-SendThread(dcc73dce5ecb.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server dcc73dce5ecb.zm-docker_default/10.0.0.17:2181. Will not attempt to authenticate using SASL (unknown error)
  631. 2018-09-06 13:26:30,988 INFO [EventLogger-Worker-Thread-0-EventThread] [] ClientCnxn - EventThread shut down
  632. 2018-09-06 13:26:30,988 INFO [EventLogger-Worker-Thread-0] [] ZooKeeper - Session: 0x0 closed
  633. 2018-09-06 13:26:30,987 INFO [EventLogger-Worker-Thread-0-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181. Will not attempt to authenticate using SASL (unknown error)
  634. 2018-09-06 13:26:30,987 INFO [EventLogger-Worker-Thread-5-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181. Will not attempt to authenticate using SASL (unknown error)
  635. 2018-09-06 13:26:30,991 INFO [EventLogger-Worker-Thread-8-SendThread(dcc73dce5ecb.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to dcc73dce5ecb.zm-docker_default/10.0.0.17:2181, initiating session
  636. 2018-09-06 13:26:30,985 ERROR [EventLogger-Worker-Thread-3] [] event - unable to send 1 events to Solr backend
  637. com.zimbra.common.service.ServiceException: system failure: unexpected error running command com.zimbra.cs.index.solr.SolrUtils$4
  638. ExceptionId:EventLogger-Worker-Thread-3:1536240390985:33d5d0aceb48fe53
  639. Code:service.FAILURE
  640. at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:288)
  641. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:39)
  642. at com.zimbra.cs.index.solr.SolrUtils.executeCloudRequestWithRetry(SolrUtils.java:320)
  643. at com.zimbra.cs.index.solr.SolrCloudHelper.executeUpdateRequest(SolrCloudHelper.java:42)
  644. at com.zimbra.cs.index.solr.SolrRequestHelper.executeUpdate(SolrRequestHelper.java:48)
  645. at com.zimbra.cs.event.logger.SolrEventCallback.execute(SolrEventCallback.java:34)
  646. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.send(BatchingEventLogger.java:126)
  647. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.access$000(BatchingEventLogger.java:103)
  648. at com.zimbra.cs.event.logger.BatchingEventLogger$1.onRemoval(BatchingEventLogger.java:53)
  649. at com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:2013)
  650. at com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3511)
  651. at com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3487)
  652. at com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3292)
  653. at com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
  654. at com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4829)
  655. at com.zimbra.cs.event.logger.BatchingEventLogger.sendAllBatched(BatchingEventLogger.java:136)
  656. at com.zimbra.cs.event.logger.BatchingEventLogger.shutdown(BatchingEventLogger.java:79)
  657. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.shutdownEventLogHandlers(EventLogger.java:244)
  658. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.run(EventLogger.java:215)
  659. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  660. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  661. at java.lang.Thread.run(Thread.java:748)
  662. Caused by: org.apache.solr.common.SolrException: Could not load collection from ZK: event_index_data
  663. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1126)
  664. at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
  665. at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1205)
  666. at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:848)
  667. at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
  668. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
  669. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
  670. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:299)
  671. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:295)
  672. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:22)
  673. ... 20 more
  674. Caused by: java.lang.InterruptedException
  675. at java.lang.Object.wait(Native Method)
  676. at java.lang.Object.wait(Object.java:502)
  677. at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1309)
  678. at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1149)
  679. at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
  680. at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
  681. at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
  682. at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1135)
  683. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1121)
  684. ... 29 more
  685. 2018-09-06 13:26:30,991 INFO [EventLogger-Worker-Thread-5-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, initiating session
  686. 2018-09-06 13:26:30,991 INFO [EventLogger-Worker-Thread-0-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, initiating session
  687. 2018-09-06 13:26:30,991 ERROR [EventLogger-Worker-Thread-4] [] event - unable to send 1 events to Solr backend
  688. com.zimbra.common.service.ServiceException: system failure: unexpected error running command com.zimbra.cs.index.solr.SolrUtils$4
  689. ExceptionId:EventLogger-Worker-Thread-4:1536240390991:33d5d0aceb48fe53
  690. Code:service.FAILURE
  691. at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:288)
  692. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:39)
  693. at com.zimbra.cs.index.solr.SolrUtils.executeCloudRequestWithRetry(SolrUtils.java:320)
  694. at com.zimbra.cs.index.solr.SolrCloudHelper.executeUpdateRequest(SolrCloudHelper.java:42)
  695. at com.zimbra.cs.index.solr.SolrRequestHelper.executeUpdate(SolrRequestHelper.java:48)
  696. at com.zimbra.cs.event.logger.SolrEventCallback.execute(SolrEventCallback.java:34)
  697. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.send(BatchingEventLogger.java:126)
  698. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.access$000(BatchingEventLogger.java:103)
  699. at com.zimbra.cs.event.logger.BatchingEventLogger$1.onRemoval(BatchingEventLogger.java:53)
  700. at com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:2013)
  701. at com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3511)
  702. at com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3487)
  703. at com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3292)
  704. at com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
  705. at com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4829)
  706. at com.zimbra.cs.event.logger.BatchingEventLogger.sendAllBatched(BatchingEventLogger.java:136)
  707. at com.zimbra.cs.event.logger.BatchingEventLogger.shutdown(BatchingEventLogger.java:79)
  708. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.shutdownEventLogHandlers(EventLogger.java:244)
  709. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.run(EventLogger.java:215)
  710. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  711. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  712. at java.lang.Thread.run(Thread.java:748)
  713. Caused by: org.apache.solr.common.SolrException: Could not load collection from ZK: event_index_data
  714. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1126)
  715. at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
  716. at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1205)
  717. at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:848)
  718. at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
  719. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
  720. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
  721. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:299)
  722. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:295)
  723. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:22)
  724. ... 20 more
  725. Caused by: java.lang.InterruptedException
  726. at java.lang.Object.wait(Native Method)
  727. at java.lang.Object.wait(Object.java:502)
  728. at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1309)
  729. at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1149)
  730. at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
  731. at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
  732. at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
  733. at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1135)
  734. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1121)
  735. ... 29 more
  736. 2018-09-06 13:26:30,991 ERROR [EventLogger-Worker-Thread-0] [] event - unable to send 1 events to Solr backend
  737. com.zimbra.common.service.ServiceException: system failure: unexpected error running command com.zimbra.cs.index.solr.SolrUtils$4
  738. ExceptionId:EventLogger-Worker-Thread-0:1536240390991:33d5d0aceb48fe53
  739. Code:service.FAILURE
  740. at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:288)
  741. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:39)
  742. at com.zimbra.cs.index.solr.SolrUtils.executeCloudRequestWithRetry(SolrUtils.java:320)
  743. at com.zimbra.cs.index.solr.SolrCloudHelper.executeUpdateRequest(SolrCloudHelper.java:42)
  744. at com.zimbra.cs.index.solr.SolrRequestHelper.executeUpdate(SolrRequestHelper.java:48)
  745. at com.zimbra.cs.event.logger.SolrEventCallback.execute(SolrEventCallback.java:34)
  746. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.send(BatchingEventLogger.java:126)
  747. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.access$000(BatchingEventLogger.java:103)
  748. at com.zimbra.cs.event.logger.BatchingEventLogger$1.onRemoval(BatchingEventLogger.java:53)
  749. at com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:2013)
  750. at com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3511)
  751. at com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3487)
  752. at com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3292)
  753. at com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
  754. at com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4829)
  755. at com.zimbra.cs.event.logger.BatchingEventLogger.sendAllBatched(BatchingEventLogger.java:136)
  756. at com.zimbra.cs.event.logger.BatchingEventLogger.shutdown(BatchingEventLogger.java:79)
  757. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.shutdownEventLogHandlers(EventLogger.java:244)
  758. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.run(EventLogger.java:215)
  759. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  760. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  761. at java.lang.Thread.run(Thread.java:748)
  762. Caused by: org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 within 10000 ms
  763. at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:182)
  764. at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:119)
  765. at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:109)
  766. at org.apache.solr.common.cloud.ZkStateReader.<init>(ZkStateReader.java:277)
  767. at org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:155)
  768. at org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:397)
  769. at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:826)
  770. at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
  771. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
  772. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
  773. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:299)
  774. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:295)
  775. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:22)
  776. ... 20 more
  777. Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 within 10000 ms
  778. at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:232)
  779. at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:174)
  780. ... 32 more
  781. 2018-09-06 13:26:30,990 ERROR [EventLogger-Worker-Thread-6] [] event - unable to send 1 events to Solr backend
  782. com.zimbra.common.service.ServiceException: system failure: unexpected error running command com.zimbra.cs.index.solr.SolrUtils$4
  783. ExceptionId:EventLogger-Worker-Thread-6:1536240390990:33d5d0aceb48fe53
  784. Code:service.FAILURE
  785. at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:288)
  786. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:39)
  787. at com.zimbra.cs.index.solr.SolrUtils.executeCloudRequestWithRetry(SolrUtils.java:320)
  788. at com.zimbra.cs.index.solr.SolrCloudHelper.executeUpdateRequest(SolrCloudHelper.java:42)
  789. at com.zimbra.cs.index.solr.SolrRequestHelper.executeUpdate(SolrRequestHelper.java:48)
  790. at com.zimbra.cs.event.logger.SolrEventCallback.execute(SolrEventCallback.java:34)
  791. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.send(BatchingEventLogger.java:126)
  792. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.access$000(BatchingEventLogger.java:103)
  793. at com.zimbra.cs.event.logger.BatchingEventLogger$1.onRemoval(BatchingEventLogger.java:53)
  794. at com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:2013)
  795. at com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3511)
  796. at com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3487)
  797. at com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3292)
  798. at com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
  799. at com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4829)
  800. at com.zimbra.cs.event.logger.BatchingEventLogger.sendAllBatched(BatchingEventLogger.java:136)
  801. at com.zimbra.cs.event.logger.BatchingEventLogger.shutdown(BatchingEventLogger.java:79)
  802. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.shutdownEventLogHandlers(EventLogger.java:244)
  803. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.run(EventLogger.java:215)
  804. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  805. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  806. at java.lang.Thread.run(Thread.java:748)
  807. Caused by: org.apache.solr.common.SolrException: Could not load collection from ZK: event_index_data
  808. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1126)
  809. at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
  810. at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1205)
  811. at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:848)
  812. at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
  813. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
  814. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
  815. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:299)
  816. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:295)
  817. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:22)
  818. ... 20 more
  819. Caused by: java.lang.InterruptedException
  820. at java.lang.Object.wait(Native Method)
  821. at java.lang.Object.wait(Object.java:502)
  822. at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1309)
  823. at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1149)
  824. at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
  825. at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
  826. at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
  827. at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1135)
  828. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1121)
  829. ... 29 more
  830. 2018-09-06 13:26:30,990 INFO [EventLogger-Worker-Thread-2-EventThread] [] ClientCnxn - EventThread shut down
  831. 2018-09-06 13:26:30,990 INFO [EventLogger-Worker-Thread-2] [] ZooKeeper - Session: 0x10000d48b75001d closed
  832. 2018-09-06 13:26:30,994 INFO [EventLogger-Worker-Thread-6] [] ZooKeeper - Session: 0x20000d48aa80021 closed
  833. 2018-09-06 13:26:30,994 INFO [EventLogger-Worker-Thread-6-EventThread] [] ClientCnxn - EventThread shut down
  834. 2018-09-06 13:26:30,990 INFO [EventLogger-Worker-Thread-7-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, initiating session
  835. 2018-09-06 13:26:30,990 INFO [EventLogger-Worker-Thread-1-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, initiating session
  836. 2018-09-06 13:26:30,990 ERROR [EventLogger-Worker-Thread-9] [] event - unable to send 1 events to Solr backend
  837. com.zimbra.common.service.ServiceException: system failure: unexpected error running command com.zimbra.cs.index.solr.SolrUtils$4
  838. ExceptionId:EventLogger-Worker-Thread-9:1536240390990:33d5d0aceb48fe53
  839. Code:service.FAILURE
  840. at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:288)
  841. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:39)
  842. at com.zimbra.cs.index.solr.SolrUtils.executeCloudRequestWithRetry(SolrUtils.java:320)
  843. at com.zimbra.cs.index.solr.SolrCloudHelper.executeUpdateRequest(SolrCloudHelper.java:42)
  844. at com.zimbra.cs.index.solr.SolrRequestHelper.executeUpdate(SolrRequestHelper.java:48)
  845. at com.zimbra.cs.event.logger.SolrEventCallback.execute(SolrEventCallback.java:34)
  846. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.send(BatchingEventLogger.java:126)
  847. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.access$000(BatchingEventLogger.java:103)
  848. at com.zimbra.cs.event.logger.BatchingEventLogger$1.onRemoval(BatchingEventLogger.java:53)
  849. at com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:2013)
  850. at com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3511)
  851. at com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3487)
  852. at com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3292)
  853. at com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
  854. at com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4829)
  855. at com.zimbra.cs.event.logger.BatchingEventLogger.sendAllBatched(BatchingEventLogger.java:136)
  856. at com.zimbra.cs.event.logger.BatchingEventLogger.shutdown(BatchingEventLogger.java:79)
  857. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.shutdownEventLogHandlers(EventLogger.java:244)
  858. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.run(EventLogger.java:215)
  859. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  860. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  861. at java.lang.Thread.run(Thread.java:748)
  862. Caused by: org.apache.solr.common.SolrException: Could not load collection from ZK: event_index_data
  863. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1126)
  864. at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
  865. at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1205)
  866. at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:848)
  867. at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
  868. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
  869. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
  870. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:299)
  871. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:295)
  872. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:22)
  873. ... 20 more
  874. Caused by: java.lang.InterruptedException
  875. at java.lang.Object.wait(Native Method)
  876. at java.lang.Object.wait(Object.java:502)
  877. at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1309)
  878. at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1149)
  879. at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
  880. at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
  881. at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
  882. at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1135)
  883. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1121)
  884. ... 29 more
  885. 2018-09-06 13:26:30,994 INFO [EventLogger-Worker-Thread-0] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@26d03a16
  886. 2018-09-06 13:26:30,994 ERROR [EventLogger-Worker-Thread-4] [] event - unable to send 1 events to Solr backend
  887. com.zimbra.common.service.ServiceException: system failure: unexpected error running command com.zimbra.cs.index.solr.SolrUtils$4
  888. ExceptionId:EventLogger-Worker-Thread-4:1536240390994:33d5d0aceb48fe53
  889. Code:service.FAILURE
  890. at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:288)
  891. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:39)
  892. at com.zimbra.cs.index.solr.SolrUtils.executeCloudRequestWithRetry(SolrUtils.java:320)
  893. at com.zimbra.cs.index.solr.SolrCloudHelper.executeUpdateRequest(SolrCloudHelper.java:42)
  894. at com.zimbra.cs.index.solr.SolrRequestHelper.executeUpdate(SolrRequestHelper.java:48)
  895. at com.zimbra.cs.event.logger.SolrEventCallback.execute(SolrEventCallback.java:34)
  896. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.send(BatchingEventLogger.java:126)
  897. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.access$000(BatchingEventLogger.java:103)
  898. at com.zimbra.cs.event.logger.BatchingEventLogger$1.onRemoval(BatchingEventLogger.java:53)
  899. at com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:2013)
  900. at com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3511)
  901. at com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3487)
  902. at com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3292)
  903. at com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
  904. at com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4829)
  905. at com.zimbra.cs.event.logger.BatchingEventLogger.sendAllBatched(BatchingEventLogger.java:136)
  906. at com.zimbra.cs.event.logger.BatchingEventLogger.shutdown(BatchingEventLogger.java:79)
  907. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.shutdownEventLogHandlers(EventLogger.java:244)
  908. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.run(EventLogger.java:215)
  909. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  910. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  911. at java.lang.Thread.run(Thread.java:748)
  912. Caused by: org.apache.solr.common.SolrException: Could not load collection from ZK: event_index_data
  913. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1126)
  914. at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
  915. at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1205)
  916. at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:848)
  917. at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
  918. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
  919. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
  920. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:299)
  921. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:295)
  922. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:22)
  923. ... 20 more
  924. Caused by: java.lang.InterruptedException
  925. at java.lang.Object.wait(Native Method)
  926. at java.lang.Object.wait(Object.java:502)
  927. at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1309)
  928. at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1149)
  929. at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
  930. at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
  931. at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
  932. at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1135)
  933. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1121)
  934. ... 29 more
  935. 2018-09-06 13:26:30,994 INFO [EventLogger-Worker-Thread-5-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, sessionid = 0x10000d48b750024, negotiated timeout = 10000
  936. 2018-09-06 13:26:30,992 INFO [EventLogger-Worker-Thread-8-SendThread(dcc73dce5ecb.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server dcc73dce5ecb.zm-docker_default/10.0.0.17:2181, sessionid = 0x10000d48b750023, negotiated timeout = 10000
  937. 2018-09-06 13:26:30,992 ERROR [EventLogger-Worker-Thread-3] [] event - unable to send 1 events to Solr backend
  938. com.zimbra.common.service.ServiceException: system failure: unexpected error running command com.zimbra.cs.index.solr.SolrUtils$4
  939. ExceptionId:EventLogger-Worker-Thread-3:1536240390992:33d5d0aceb48fe53
  940. Code:service.FAILURE
  941. at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:288)
  942. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:39)
  943. at com.zimbra.cs.index.solr.SolrUtils.executeCloudRequestWithRetry(SolrUtils.java:320)
  944. at com.zimbra.cs.index.solr.SolrCloudHelper.executeUpdateRequest(SolrCloudHelper.java:42)
  945. at com.zimbra.cs.index.solr.SolrRequestHelper.executeUpdate(SolrRequestHelper.java:48)
  946. at com.zimbra.cs.event.logger.SolrEventCallback.execute(SolrEventCallback.java:34)
  947. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.send(BatchingEventLogger.java:126)
  948. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.access$000(BatchingEventLogger.java:103)
  949. at com.zimbra.cs.event.logger.BatchingEventLogger$1.onRemoval(BatchingEventLogger.java:53)
  950. at com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:2013)
  951. at com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3511)
  952. at com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3487)
  953. at com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3292)
  954. at com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
  955. at com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4829)
  956. at com.zimbra.cs.event.logger.BatchingEventLogger.sendAllBatched(BatchingEventLogger.java:136)
  957. at com.zimbra.cs.event.logger.BatchingEventLogger.shutdown(BatchingEventLogger.java:79)
  958. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.shutdownEventLogHandlers(EventLogger.java:244)
  959. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.run(EventLogger.java:215)
  960. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  961. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  962. at java.lang.Thread.run(Thread.java:748)
  963. Caused by: org.apache.solr.common.SolrException: Could not load collection from ZK: event_index_data
  964. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1126)
  965. at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
  966. at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1205)
  967. at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:848)
  968. at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
  969. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
  970. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
  971. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:299)
  972. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:295)
  973. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:22)
  974. ... 20 more
  975. Caused by: java.lang.InterruptedException
  976. at java.lang.Object.wait(Native Method)
  977. at java.lang.Object.wait(Object.java:502)
  978. at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1309)
  979. at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1149)
  980. at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
  981. at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
  982. at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
  983. at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1135)
  984. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1121)
  985. ... 29 more
  986. 2018-09-06 13:26:30,996 INFO [zkConnectionManagerCallback-217-thread-1] [] ConnectionManager - zkClient has connected
  987. 2018-09-06 13:26:30,996 INFO [zkConnectionManagerCallback-214-thread-1] [] ConnectionManager - zkClient has connected
  988. 2018-09-06 13:26:30,996 INFO [EventLogger-Worker-Thread-4-EventThread] [] ClientCnxn - EventThread shut down
  989. 2018-09-06 13:26:30,996 INFO [EventLogger-Worker-Thread-4] [] ZooKeeper - Session: 0x20000d48aa80022 closed
  990. 2018-09-06 13:26:30,995 ERROR [EventLogger-Worker-Thread-9] [] event - unable to send 1 events to Solr backend
  991. com.zimbra.common.service.ServiceException: system failure: unexpected error running command com.zimbra.cs.index.solr.SolrUtils$4
  992. ExceptionId:EventLogger-Worker-Thread-9:1536240390995:33d5d0aceb48fe53
  993. Code:service.FAILURE
  994. at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:288)
  995. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:39)
  996. at com.zimbra.cs.index.solr.SolrUtils.executeCloudRequestWithRetry(SolrUtils.java:320)
  997. at com.zimbra.cs.index.solr.SolrCloudHelper.executeUpdateRequest(SolrCloudHelper.java:42)
  998. at com.zimbra.cs.index.solr.SolrRequestHelper.executeUpdate(SolrRequestHelper.java:48)
  999. at com.zimbra.cs.event.logger.SolrEventCallback.execute(SolrEventCallback.java:34)
  1000. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.send(BatchingEventLogger.java:126)
  1001. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.access$000(BatchingEventLogger.java:103)
  1002. at com.zimbra.cs.event.logger.BatchingEventLogger$1.onRemoval(BatchingEventLogger.java:53)
  1003. at com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:2013)
  1004. at com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3511)
  1005. at com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3487)
  1006. at com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3292)
  1007. at com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
  1008. at com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4829)
  1009. at com.zimbra.cs.event.logger.BatchingEventLogger.sendAllBatched(BatchingEventLogger.java:136)
  1010. at com.zimbra.cs.event.logger.BatchingEventLogger.shutdown(BatchingEventLogger.java:79)
  1011. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.shutdownEventLogHandlers(EventLogger.java:244)
  1012. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.run(EventLogger.java:215)
  1013. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  1014. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  1015. at java.lang.Thread.run(Thread.java:748)
  1016. Caused by: org.apache.solr.common.SolrException: Could not load collection from ZK: event_index_data
  1017. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1126)
  1018. at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
  1019. at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1205)
  1020. at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:848)
  1021. at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
  1022. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
  1023. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
  1024. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:299)
  1025. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:295)
  1026. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:22)
  1027. ... 20 more
  1028. Caused by: java.lang.InterruptedException
  1029. at java.lang.Object.wait(Native Method)
  1030. at java.lang.Object.wait(Object.java:502)
  1031. at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1309)
  1032. at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1149)
  1033. at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
  1034. at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
  1035. at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
  1036. at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1135)
  1037. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1121)
  1038. ... 29 more
  1039. 2018-09-06 13:26:30,999 INFO [EventLogger-Worker-Thread-5] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  1040. 2018-09-06 13:26:30,999 INFO [EventLogger-Worker-Thread-8] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  1041. 2018-09-06 13:26:31,001 INFO [EventLogger-Worker-Thread-9-EventThread] [] ClientCnxn - EventThread shut down
  1042. 2018-09-06 13:26:30,998 ERROR [EventLogger-Worker-Thread-3] [] event - unable to send 1 events to Solr backend
  1043. com.zimbra.common.service.ServiceException: system failure: unexpected error running command com.zimbra.cs.index.solr.SolrUtils$4
  1044. ExceptionId:EventLogger-Worker-Thread-3:1536240390998:33d5d0aceb48fe53
  1045. Code:service.FAILURE
  1046. at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:288)
  1047. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:39)
  1048. at com.zimbra.cs.index.solr.SolrUtils.executeCloudRequestWithRetry(SolrUtils.java:320)
  1049. at com.zimbra.cs.index.solr.SolrCloudHelper.executeUpdateRequest(SolrCloudHelper.java:42)
  1050. at com.zimbra.cs.index.solr.SolrRequestHelper.executeUpdate(SolrRequestHelper.java:48)
  1051. at com.zimbra.cs.event.logger.SolrEventCallback.execute(SolrEventCallback.java:34)
  1052. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.send(BatchingEventLogger.java:126)
  1053. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.access$000(BatchingEventLogger.java:103)
  1054. at com.zimbra.cs.event.logger.BatchingEventLogger$1.onRemoval(BatchingEventLogger.java:53)
  1055. at com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:2013)
  1056. at com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3511)
  1057. at com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3487)
  1058. at com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3292)
  1059. at com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
  1060. at com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4829)
  1061. at com.zimbra.cs.event.logger.BatchingEventLogger.sendAllBatched(BatchingEventLogger.java:136)
  1062. at com.zimbra.cs.event.logger.BatchingEventLogger.shutdown(BatchingEventLogger.java:79)
  1063. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.shutdownEventLogHandlers(EventLogger.java:244)
  1064. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.run(EventLogger.java:215)
  1065. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  1066. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  1067. at java.lang.Thread.run(Thread.java:748)
  1068. Caused by: org.apache.solr.common.SolrException: Could not load collection from ZK: event_index_data
  1069. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1126)
  1070. at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
  1071. at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1205)
  1072. at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:848)
  1073. at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
  1074. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
  1075. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
  1076. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:299)
  1077. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:295)
  1078. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:22)
  1079. ... 20 more
  1080. Caused by: java.lang.InterruptedException
  1081. at java.lang.Object.wait(Native Method)
  1082. at java.lang.Object.wait(Object.java:502)
  1083. at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1309)
  1084. at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1149)
  1085. at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
  1086. at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
  1087. at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
  1088. at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1135)
  1089. at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1121)
  1090. ... 29 more
  1091. 2018-09-06 13:26:30,997 INFO [EventLogger-Worker-Thread-1-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, sessionid = 0x20000d48aa80024, negotiated timeout = 10000
  1092. 2018-09-06 13:26:30,996 INFO [EventLogger-Worker-Thread-7-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, sessionid = 0x10000d48b750025, negotiated timeout = 10000
  1093. 2018-09-06 13:26:31,001 INFO [EventLogger-Worker-Thread-3] [] ZooKeeper - Session: 0x10000d48b75001e closed
  1094. 2018-09-06 13:26:30,996 INFO [EventLogger-Worker-Thread-0-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181. Will not attempt to authenticate using SASL (unknown error)
  1095. 2018-09-06 13:26:31,001 INFO [EventLogger-Worker-Thread-3-EventThread] [] ClientCnxn - EventThread shut down
  1096. 2018-09-06 13:26:31,001 INFO [EventLogger-Worker-Thread-9] [] ZooKeeper - Session: 0x10000d48b75001c closed
  1097. 2018-09-06 13:26:31,001 INFO [EventLogger-Worker-Thread-0-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181, initiating session
  1098. 2018-09-06 13:26:31,002 INFO [zkConnectionManagerCallback-212-thread-1] [] ConnectionManager - zkClient has connected
  1099. 2018-09-06 13:26:31,002 INFO [zkConnectionManagerCallback-211-thread-1] [] ConnectionManager - zkClient has connected
  1100. 2018-09-06 13:26:31,004 INFO [EventLogger-Worker-Thread-1] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  1101. 2018-09-06 13:26:31,005 INFO [EventLogger-Worker-Thread-1] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  1102. 2018-09-06 13:26:31,005 INFO [EventLogger-Worker-Thread-7] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  1103. 2018-09-06 13:26:31,005 INFO [EventLogger-Worker-Thread-5] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  1104. 2018-09-06 13:26:31,005 INFO [EventLogger-Worker-Thread-8] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  1105. 2018-09-06 13:26:31,005 INFO [EventLogger-Worker-Thread-0-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181, sessionid = 0x30000d48aad001e, negotiated timeout = 10000
  1106. 2018-09-06 13:26:31,006 INFO [zkConnectionManagerCallback-220-thread-1] [] ConnectionManager - zkClient has connected
  1107. 2018-09-06 13:26:31,008 INFO [EventLogger-Worker-Thread-0] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  1108. 2018-09-06 13:26:31,008 INFO [EventLogger-Worker-Thread-7] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  1109. 2018-09-06 13:26:31,009 INFO [EventLogger-Worker-Thread-0] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  1110. 2018-09-06 13:26:31,031 INFO [EventLogger-Worker-Thread-7] [] ZooKeeper - Session: 0x10000d48b750025 closed
  1111. 2018-09-06 13:26:31,031 INFO [EventLogger-Worker-Thread-7-EventThread] [] ClientCnxn - EventThread shut down
  1112. 2018-09-06 13:26:31,032 INFO [EventLogger-Worker-Thread-0] [] ZooKeeper - Session: 0x30000d48aad001e closed
  1113. 2018-09-06 13:26:31,032 INFO [EventLogger-Worker-Thread-0-EventThread] [] ClientCnxn - EventThread shut down
  1114. 2018-09-06 13:26:31,034 INFO [EventLogger-Worker-Thread-1] [] ZooKeeper - Session: 0x20000d48aa80024 closed
  1115. 2018-09-06 13:26:31,034 INFO [EventLogger-Worker-Thread-1-EventThread] [] ClientCnxn - EventThread shut down
  1116. 2018-09-06 13:26:31,040 INFO [EventLogger-Worker-Thread-5] [] ZooKeeper - Session: 0x10000d48b750024 closed
  1117. 2018-09-06 13:26:31,040 INFO [EventLogger-Worker-Thread-5-EventThread] [] ClientCnxn - EventThread shut down
  1118. 2018-09-06 13:26:31,041 INFO [EventLogger-Worker-Thread-8] [] ZooKeeper - Session: 0x10000d48b750023 closed
  1119. 2018-09-06 13:26:31,041 INFO [EventLogger-Worker-Thread-8-EventThread] [] ClientCnxn - EventThread shut down
  1120. 2018-09-06 13:26:31,045 INFO [qtp788117692-1565:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53652;soapId=4faf0c56;] event - Event Notifier Executor shutdown was successful
  1121. 2018-09-06 13:26:31,045 INFO [qtp788117692-1565:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53652;soapId=4faf0c56;] event - Event Queue Size 0
  1122. 2018-09-06 13:26:31,045 INFO [qtp788117692-1565:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53652;soapId=4faf0c56;] event - Starting Event Notifier Logger with 10 threads; initial event queue size is 0
  1123. 2018-09-06 13:26:31,047 INFO [qtp788117692-1565:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53652;soapId=4faf0c56;] soap - CreateCosRequest elapsed=71
  1124. 2018-09-06 13:26:31,184 WARN [qtp788117692-1565:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42052;soapId=4faf0c57;] event - Shutdown called for Event Notifier Executor; initiating shutdown sequence...
  1125. 2018-09-06 13:26:31,185 INFO [qtp788117692-1565:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42052;soapId=4faf0c57;] event - Event Notifier Executor shutdown was successful
  1126. 2018-09-06 13:26:31,185 INFO [qtp788117692-1565:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42052;soapId=4faf0c57;] event - Event Queue Size 0
  1127. 2018-09-06 13:26:31,185 INFO [qtp788117692-1565:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42052;soapId=4faf0c57;] event - Starting Event Notifier Logger with 10 threads; initial event queue size is 0
  1128. 2018-09-06 13:26:31,187 INFO [qtp788117692-1565:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42052;soapId=4faf0c57;] soap - CopyCosRequest elapsed=12
  1129. 2018-09-06 13:26:31,206 INFO [qtp788117692-1536:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53738;soapId=4faf0c58;] soap - AuthRequest elapsed=0
  1130. 2018-09-06 13:26:31,287 INFO [qtp788117692-1559:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42060;soapId=4faf0c59;] soap - AuthRequest elapsed=0
  1131. 2018-09-06 13:26:31,313 WARN [qtp788117692-1541:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53748;soapId=4faf0c5a;] event - Shutdown called for Event Notifier Executor; initiating shutdown sequence...
  1132. 2018-09-06 13:26:31,313 INFO [qtp788117692-1541:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53748;soapId=4faf0c5a;] event - Event Notifier Executor shutdown was successful
  1133. 2018-09-06 13:26:31,313 INFO [qtp788117692-1541:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53748;soapId=4faf0c5a;] event - Event Queue Size 0
  1134. 2018-09-06 13:26:31,313 INFO [qtp788117692-1541:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53748;soapId=4faf0c5a;] event - Starting Event Notifier Logger with 10 threads; initial event queue size is 0
  1135. 2018-09-06 13:26:31,316 INFO [qtp788117692-1541:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53748;soapId=4faf0c5a;] soap - CreateCosRequest elapsed=10
  1136. 2018-09-06 13:26:31,398 WARN [qtp788117692-1558:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42070;soapId=4faf0c5b;] event - Shutdown called for Event Notifier Executor; initiating shutdown sequence...
  1137. 2018-09-06 13:26:31,398 INFO [qtp788117692-1558:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42070;soapId=4faf0c5b;] event - Event Notifier Executor shutdown was successful
  1138. 2018-09-06 13:26:31,398 INFO [qtp788117692-1558:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42070;soapId=4faf0c5b;] event - Event Queue Size 0
  1139. 2018-09-06 13:26:31,398 INFO [qtp788117692-1558:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42070;soapId=4faf0c5b;] event - Starting Event Notifier Logger with 10 threads; initial event queue size is 0
  1140. 2018-09-06 13:26:31,401 INFO [qtp788117692-1558:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42070;soapId=4faf0c5b;] soap - CreateCosRequest elapsed=13
  1141. 2018-09-06 13:26:31,425 INFO [qtp788117692-1563:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53756;soapId=4faf0c5c;] soap - RenameCosRequest elapsed=4
  1142. 2018-09-06 13:26:31,512 INFO [qtp788117692-1535:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42078;soapId=4faf0c5d;] soap - DeleteCosRequest elapsed=3
  1143. 2018-09-06 13:26:31,526 INFO [qtp788117692-1536:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53764;soapId=4faf0c5e;] soap - AuthRequest elapsed=0
  1144. 2018-09-06 13:26:31,587 INFO [qtp788117692-1547:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42086;soapId=4faf0c5f;] soap - GetDelegatedAdminConstraintsRequest elapsed=2
  1145. 2018-09-06 13:26:31,603 INFO [qtp788117692-1541:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53772;soapId=4faf0c60;] soap - AuthRequest elapsed=1
  1146.  
  1147. ==> /opt/zimbra/log/audit.log <==
  1148. 2018-09-06 13:26:30,958 INFO [qtp788117692-1565:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=41966;soapId=4faf0c55;] security - cmd=AdminAuth; account=admin@zmc.com;
  1149. 2018-09-06 13:26:30,959 INFO [qtp788117692-1565:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=41966;soapId=4faf0c55;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  1150. 2018-09-06 13:26:31,046 INFO [qtp788117692-1565:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53652;soapId=4faf0c56;] security - cmd=CreateCos; name=cos153624039094224;
  1151. 2018-09-06 13:26:31,186 INFO [qtp788117692-1565:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42052;soapId=4faf0c57;] security - cmd=CopyCos; name=cos153624039094325; cos= cba54f5d-9ee5-4001-85aa-687a98dafda3;
  1152. 2018-09-06 13:26:31,206 INFO [qtp788117692-1536:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53738;soapId=4faf0c58;] security - cmd=AdminAuth; account=admin@zmc.com;
  1153. 2018-09-06 13:26:31,206 INFO [qtp788117692-1536:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53738;soapId=4faf0c58;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  1154. 2018-09-06 13:26:31,287 INFO [qtp788117692-1559:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42060;soapId=4faf0c59;] security - cmd=AdminAuth; account=admin@zmc.com;
  1155. 2018-09-06 13:26:31,287 INFO [qtp788117692-1559:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42060;soapId=4faf0c59;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  1156. 2018-09-06 13:26:31,315 INFO [qtp788117692-1541:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53748;soapId=4faf0c5a;] security - cmd=CreateCos; name=cos153624039094329;
  1157. 2018-09-06 13:26:31,401 INFO [qtp788117692-1558:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42070;soapId=4faf0c5b;] security - cmd=CreateCos; name=cos153624039094330;
  1158. 2018-09-06 13:26:31,423 INFO [qtp788117692-1563:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53756;soapId=4faf0c5c;] security - cmd=RenameCos; name=cos153624039094330; newName=cos153624039094326;
  1159. 2018-09-06 13:26:31,512 INFO [qtp788117692-1535:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42078;soapId=4faf0c5d;] security - cmd=DeleteCos; name=cos153624039094431; id=9785e4b0-57a4-4a9b-bf32-9295a2029b9d;
  1160. 2018-09-06 13:26:31,526 INFO [qtp788117692-1536:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53764;soapId=4faf0c5e;] security - cmd=AdminAuth; account=admin@zmc.com;
  1161. 2018-09-06 13:26:31,526 INFO [qtp788117692-1536:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53764;soapId=4faf0c5e;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  1162. 2018-09-06 13:26:31,602 INFO [qtp788117692-1541:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53772;soapId=4faf0c60;] security - cmd=AdminAuth; account=admin@zmc.com;
  1163. 2018-09-06 13:26:31,602 INFO [qtp788117692-1541:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=53772;soapId=4faf0c60;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  1164.  
  1165.  
  1166.  
  1167.  
  1168. […]
  1169.  
  1170.  
  1171.  
  1172. ==> /opt/zimbra/log/mailbox.log <==
  1173. 2018-09-06 13:27:44,530 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=54244;soapId=4e5021a5;] soap - AuthRequest elapsed=2
  1174. 2018-09-06 13:27:44,560 WARN [qtp788117692-1794:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42562;soapId=4e5021a6;] event - Shutdown called for Event Notifier Executor; initiating shutdown sequence...
  1175. 2018-09-06 13:27:44,560 INFO [EventLogger-Worker-Thread-9] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@fa6cdc6
  1176. 2018-09-06 13:27:44,560 INFO [EventLogger-Worker-Thread-7] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@4194abbe
  1177. 2018-09-06 13:27:44,560 INFO [EventLogger-Worker-Thread-0] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@e74382
  1178. 2018-09-06 13:27:44,560 INFO [EventLogger-Worker-Thread-5] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@6216dddf
  1179. 2018-09-06 13:27:44,560 INFO [EventLogger-Worker-Thread-2] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@189c0ab7
  1180. 2018-09-06 13:27:44,560 INFO [EventLogger-Worker-Thread-1] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@352452e
  1181. 2018-09-06 13:27:44,560 INFO [EventLogger-Worker-Thread-6] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@7866c246
  1182. 2018-09-06 13:27:44,560 INFO [EventLogger-Worker-Thread-8] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@2263f2b8
  1183. 2018-09-06 13:27:44,560 INFO [EventLogger-Worker-Thread-3] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@124713a4
  1184. 2018-09-06 13:27:44,560 INFO [EventLogger-Worker-Thread-4] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@ec2e49
  1185. 2018-09-06 13:27:44,563 INFO [EventLogger-Worker-Thread-0-SendThread(b3ecdace375e.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server b3ecdace375e.zm-docker_default/10.0.0.7:2181. Will not attempt to authenticate using SASL (unknown error)
  1186. 2018-09-06 13:27:44,563 INFO [EventLogger-Worker-Thread-5-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181. Will not attempt to authenticate using SASL (unknown error)
  1187. 2018-09-06 13:27:44,563 INFO [EventLogger-Worker-Thread-5-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, initiating session
  1188. 2018-09-06 13:27:44,563 INFO [EventLogger-Worker-Thread-0-SendThread(b3ecdace375e.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to b3ecdace375e.zm-docker_default/10.0.0.7:2181, initiating session
  1189. 2018-09-06 13:27:44,564 INFO [EventLogger-Worker-Thread-5-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, sessionid = 0x20000d48aa80029, negotiated timeout = 10000
  1190. 2018-09-06 13:27:44,564 INFO [EventLogger-Worker-Thread-8-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181. Will not attempt to authenticate using SASL (unknown error)
  1191. 2018-09-06 13:27:44,564 INFO [EventLogger-Worker-Thread-3-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181. Will not attempt to authenticate using SASL (unknown error)
  1192. 2018-09-06 13:27:44,565 INFO [EventLogger-Worker-Thread-4-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181. Will not attempt to authenticate using SASL (unknown error)
  1193. 2018-09-06 13:27:44,565 INFO [EventLogger-Worker-Thread-8-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, initiating session
  1194. 2018-09-06 13:27:44,565 INFO [EventLogger-Worker-Thread-4-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, initiating session
  1195. 2018-09-06 13:27:44,565 INFO [EventLogger-Worker-Thread-1-SendThread(b3ecdace375e.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server b3ecdace375e.zm-docker_default/10.0.0.7:2181. Will not attempt to authenticate using SASL (unknown error)
  1196. 2018-09-06 13:27:44,565 INFO [EventLogger-Worker-Thread-7-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181. Will not attempt to authenticate using SASL (unknown error)
  1197. 2018-09-06 13:27:44,565 INFO [EventLogger-Worker-Thread-1-SendThread(b3ecdace375e.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to b3ecdace375e.zm-docker_default/10.0.0.7:2181, initiating session
  1198. 2018-09-06 13:27:44,565 INFO [EventLogger-Worker-Thread-7-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181, initiating session
  1199. 2018-09-06 13:27:44,569 INFO [EventLogger-Worker-Thread-2-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181. Will not attempt to authenticate using SASL (unknown error)
  1200. 2018-09-06 13:27:44,569 INFO [EventLogger-Worker-Thread-4-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, sessionid = 0x10000d48b750028, negotiated timeout = 10000
  1201. 2018-09-06 13:27:44,569 INFO [EventLogger-Worker-Thread-1-SendThread(b3ecdace375e.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server b3ecdace375e.zm-docker_default/10.0.0.7:2181, sessionid = 0x20000d48aa8002c, negotiated timeout = 10000
  1202. 2018-09-06 13:27:44,569 INFO [EventLogger-Worker-Thread-0-SendThread(b3ecdace375e.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server b3ecdace375e.zm-docker_default/10.0.0.7:2181, sessionid = 0x20000d48aa8002b, negotiated timeout = 10000
  1203. 2018-09-06 13:27:44,569 INFO [EventLogger-Worker-Thread-7-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181, sessionid = 0x30000d48aad0022, negotiated timeout = 10000
  1204. 2018-09-06 13:27:44,569 INFO [EventLogger-Worker-Thread-3-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, initiating session
  1205. 2018-09-06 13:27:44,569 INFO [EventLogger-Worker-Thread-8-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, sessionid = 0x20000d48aa8002a, negotiated timeout = 10000
  1206. 2018-09-06 13:27:44,570 INFO [EventLogger-Worker-Thread-6-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181. Will not attempt to authenticate using SASL (unknown error)
  1207. 2018-09-06 13:27:44,570 INFO [zkConnectionManagerCallback-369-thread-1] [] ConnectionManager - zkClient has connected
  1208. 2018-09-06 13:27:44,570 INFO [EventLogger-Worker-Thread-2-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, initiating session
  1209. 2018-09-06 13:27:44,570 INFO [zkConnectionManagerCallback-389-thread-1] [] ConnectionManager - zkClient has connected
  1210. 2018-09-06 13:27:44,570 INFO [EventLogger-Worker-Thread-6-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, initiating session
  1211. 2018-09-06 13:27:44,570 INFO [zkConnectionManagerCallback-373-thread-1] [] ConnectionManager - zkClient has connected
  1212. 2018-09-06 13:27:44,571 INFO [zkConnectionManagerCallback-382-thread-1] [] ConnectionManager - zkClient has connected
  1213. 2018-09-06 13:27:44,572 INFO [EventLogger-Worker-Thread-3-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, sessionid = 0x10000d48b750029, negotiated timeout = 10000
  1214. 2018-09-06 13:27:44,572 INFO [EventLogger-Worker-Thread-2-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, sessionid = 0x10000d48b75002a, negotiated timeout = 10000
  1215. 2018-09-06 13:27:44,572 INFO [zkConnectionManagerCallback-374-thread-1] [] ConnectionManager - zkClient has connected
  1216. 2018-09-06 13:27:44,572 INFO [EventLogger-Worker-Thread-9-SendThread(dcc73dce5ecb.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server dcc73dce5ecb.zm-docker_default/10.0.0.17:2181. Will not attempt to authenticate using SASL (unknown error)
  1217. 2018-09-06 13:27:44,572 INFO [zkConnectionManagerCallback-387-thread-1] [] ConnectionManager - zkClient has connected
  1218. 2018-09-06 13:27:44,572 INFO [EventLogger-Worker-Thread-9] [] ZooKeeper - Session: 0x0 closed
  1219. 2018-09-06 13:27:44,572 INFO [EventLogger-Worker-Thread-9-EventThread] [] ClientCnxn - EventThread shut down
  1220. 2018-09-06 13:27:44,572 INFO [zkConnectionManagerCallback-383-thread-1] [] ConnectionManager - zkClient has connected
  1221. 2018-09-06 13:27:44,572 ERROR [EventLogger-Worker-Thread-9] [] event - unable to send 1 events to Solr backend
  1222. com.zimbra.common.service.ServiceException: system failure: unexpected error running command com.zimbra.cs.index.solr.SolrUtils$4
  1223. ExceptionId:EventLogger-Worker-Thread-9:1536240464572:a4a5f9c6e5824c5f
  1224. Code:service.FAILURE
  1225. at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:288)
  1226. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:39)
  1227. at com.zimbra.cs.index.solr.SolrUtils.executeCloudRequestWithRetry(SolrUtils.java:320)
  1228. at com.zimbra.cs.index.solr.SolrCloudHelper.executeUpdateRequest(SolrCloudHelper.java:42)
  1229. at com.zimbra.cs.index.solr.SolrRequestHelper.executeUpdate(SolrRequestHelper.java:48)
  1230. at com.zimbra.cs.event.logger.SolrEventCallback.execute(SolrEventCallback.java:34)
  1231. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.send(BatchingEventLogger.java:126)
  1232. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.access$000(BatchingEventLogger.java:103)
  1233. at com.zimbra.cs.event.logger.BatchingEventLogger$1.onRemoval(BatchingEventLogger.java:53)
  1234. at com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:2013)
  1235. at com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3511)
  1236. at com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3487)
  1237. at com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3292)
  1238. at com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
  1239. at com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4829)
  1240. at com.zimbra.cs.event.logger.BatchingEventLogger.sendAllBatched(BatchingEventLogger.java:136)
  1241. at com.zimbra.cs.event.logger.BatchingEventLogger.shutdown(BatchingEventLogger.java:79)
  1242. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.shutdownEventLogHandlers(EventLogger.java:244)
  1243. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.run(EventLogger.java:215)
  1244. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  1245. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  1246. at java.lang.Thread.run(Thread.java:748)
  1247. Caused by: org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 within 10000 ms
  1248. at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:182)
  1249. at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:119)
  1250. at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:109)
  1251. at org.apache.solr.common.cloud.ZkStateReader.<init>(ZkStateReader.java:277)
  1252. at org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:155)
  1253. at org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:397)
  1254. at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:826)
  1255. at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
  1256. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
  1257. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
  1258. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:299)
  1259. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:295)
  1260. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:22)
  1261. ... 20 more
  1262. Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 within 10000 ms
  1263. at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:232)
  1264. at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:174)
  1265. ... 32 more
  1266. 2018-09-06 13:27:44,573 INFO [EventLogger-Worker-Thread-6-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, sessionid = 0x20000d48aa8002d, negotiated timeout = 10000
  1267. 2018-09-06 13:27:44,573 INFO [EventLogger-Worker-Thread-9-SendThread(dcc73dce5ecb.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to dcc73dce5ecb.zm-docker_default/10.0.0.17:2181, initiating session
  1268. 2018-09-06 13:27:44,573 INFO [zkConnectionManagerCallback-372-thread-1] [] ConnectionManager - zkClient has connected
  1269. 2018-09-06 13:27:44,576 INFO [EventLogger-Worker-Thread-4] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  1270. 2018-09-06 13:27:44,576 INFO [zkConnectionManagerCallback-377-thread-1] [] ConnectionManager - zkClient has connected
  1271. 2018-09-06 13:27:44,576 INFO [EventLogger-Worker-Thread-3] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  1272. 2018-09-06 13:27:44,576 INFO [EventLogger-Worker-Thread-5] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  1273. 2018-09-06 13:27:44,578 INFO [EventLogger-Worker-Thread-7] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  1274. 2018-09-06 13:27:44,579 INFO [EventLogger-Worker-Thread-8] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  1275. 2018-09-06 13:27:44,579 INFO [EventLogger-Worker-Thread-1] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  1276. 2018-09-06 13:27:44,579 INFO [EventLogger-Worker-Thread-0] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  1277. 2018-09-06 13:27:44,579 INFO [EventLogger-Worker-Thread-2] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  1278. 2018-09-06 13:27:44,579 INFO [EventLogger-Worker-Thread-6] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  1279. 2018-09-06 13:27:44,579 INFO [EventLogger-Worker-Thread-3] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  1280. 2018-09-06 13:27:44,579 INFO [EventLogger-Worker-Thread-8] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  1281. 2018-09-06 13:27:44,579 INFO [EventLogger-Worker-Thread-4] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  1282. 2018-09-06 13:27:44,579 INFO [EventLogger-Worker-Thread-1] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  1283. 2018-09-06 13:27:44,580 INFO [EventLogger-Worker-Thread-5] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  1284. 2018-09-06 13:27:44,580 INFO [EventLogger-Worker-Thread-2] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  1285. 2018-09-06 13:27:44,580 INFO [EventLogger-Worker-Thread-7] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  1286. 2018-09-06 13:27:44,580 INFO [EventLogger-Worker-Thread-0] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  1287. 2018-09-06 13:27:44,581 INFO [EventLogger-Worker-Thread-6] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  1288. 2018-09-06 13:27:44,602 INFO [EventLogger-Worker-Thread-6] [] ZooKeeper - Session: 0x20000d48aa8002d closed
  1289. 2018-09-06 13:27:44,602 INFO [EventLogger-Worker-Thread-6-EventThread] [] ClientCnxn - EventThread shut down
  1290. 2018-09-06 13:27:44,604 INFO [EventLogger-Worker-Thread-8] [] ZooKeeper - Session: 0x20000d48aa8002a closed
  1291. 2018-09-06 13:27:44,604 INFO [EventLogger-Worker-Thread-8-EventThread] [] ClientCnxn - EventThread shut down
  1292. 2018-09-06 13:27:44,605 INFO [EventLogger-Worker-Thread-3] [] ZooKeeper - Session: 0x10000d48b750029 closed
  1293. 2018-09-06 13:27:44,605 INFO [EventLogger-Worker-Thread-7] [] ZooKeeper - Session: 0x30000d48aad0022 closed
  1294. 2018-09-06 13:27:44,605 INFO [EventLogger-Worker-Thread-3-EventThread] [] ClientCnxn - EventThread shut down
  1295. 2018-09-06 13:27:44,605 INFO [EventLogger-Worker-Thread-7-EventThread] [] ClientCnxn - EventThread shut down
  1296. 2018-09-06 13:27:44,606 INFO [EventLogger-Worker-Thread-0] [] ZooKeeper - Session: 0x20000d48aa8002b closed
  1297. 2018-09-06 13:27:44,606 INFO [EventLogger-Worker-Thread-0-EventThread] [] ClientCnxn - EventThread shut down
  1298. 2018-09-06 13:27:44,607 INFO [EventLogger-Worker-Thread-1] [] ZooKeeper - Session: 0x20000d48aa8002c closed
  1299. 2018-09-06 13:27:44,607 INFO [EventLogger-Worker-Thread-4] [] ZooKeeper - Session: 0x10000d48b750028 closed
  1300. 2018-09-06 13:27:44,607 INFO [EventLogger-Worker-Thread-1-EventThread] [] ClientCnxn - EventThread shut down
  1301. 2018-09-06 13:27:44,607 INFO [EventLogger-Worker-Thread-4-EventThread] [] ClientCnxn - EventThread shut down
  1302. 2018-09-06 13:27:44,607 INFO [EventLogger-Worker-Thread-5] [] ZooKeeper - Session: 0x20000d48aa80029 closed
  1303. 2018-09-06 13:27:44,607 INFO [EventLogger-Worker-Thread-5-EventThread] [] ClientCnxn - EventThread shut down
  1304. 2018-09-06 13:27:44,614 INFO [EventLogger-Worker-Thread-2] [] ZooKeeper - Session: 0x10000d48b75002a closed
  1305. 2018-09-06 13:27:44,614 INFO [EventLogger-Worker-Thread-2-EventThread] [] ClientCnxn - EventThread shut down
  1306. 2018-09-06 13:27:44,615 INFO [qtp788117692-1794:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42562;soapId=4e5021a6;] event - Event Notifier Executor shutdown was successful
  1307. 2018-09-06 13:27:44,615 INFO [qtp788117692-1794:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42562;soapId=4e5021a6;] event - Event Queue Size 0
  1308. 2018-09-06 13:27:44,615 INFO [qtp788117692-1794:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42562;soapId=4e5021a6;] event - Starting Event Notifier Logger with 10 threads; initial event queue size is 0
  1309. 2018-09-06 13:27:44,618 INFO [qtp788117692-1794:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42562;soapId=4e5021a6;] soap - CreateCosRequest elapsed=68
  1310.  
  1311. ==> /opt/zimbra/log/audit.log <==
  1312. 2018-09-06 13:27:44,528 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=54244;soapId=4e5021a5;] security - cmd=AdminAuth; account=admin@zmc.com;
  1313. 2018-09-06 13:27:44,529 INFO [qtp788117692-1661:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=54244;soapId=4e5021a5;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  1314. 2018-09-06 13:27:44,617 INFO [qtp788117692-1794:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42562;soapId=4e5021a6;] security - cmd=CreateCos; name=cos1536240391917.41; zimbraFeatureCalendarReminderDeviceEmailEnabled=TRUE;
  1315.  
  1316. ==> /opt/zimbra/log/mailbox.log <==
  1317. 2018-09-06 13:27:44,497 INFO [qtp788117692-1473:https:https://zmc-proxy/service/soap/] [name=test1536240391916.33@zmc.com;mid=127;ip=10.0.0.20;port=60010;soapId=4faf0c82;] soap - SearchRequest elapsed=41
  1318. 2018-09-06 13:27:44,514 INFO [qtp788117692-1473:https:https://zmc-proxy/service/soap/] [name=test1536240391916.33@zmc.com;mid=127;ip=10.0.0.19;port=41308;soapId=4faf0c83;] soap - VerifyCodeRequest elapsed=1
  1319. 2018-09-06 13:27:44,646 INFO [qtp788117692-1683:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=54278;soapId=4faf0c84;] soap - CreateAccountRequest elapsed=6
  1320.  
  1321. ==> /opt/zimbra/log/audit.log <==
  1322. 2018-09-06 13:27:44,645 INFO [qtp788117692-1683:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=54278;soapId=4faf0c84;] security - cmd=CreateAccount; name=test1536240391916.36@zmc.com; zimbraCalendarReminderDeviceEmail=test1536240391916.36@zmc.com; zimbraCOSId=334fd78a-5197-48c9-8dee-367fa6244ca4;
  1323.  
  1324. ==> /opt/zimbra/log/mailbox.log <==
  1325. 2018-09-06 13:27:47,712 INFO [qtp788117692-1576:https:https://zmc-proxy/service/soap/] [name=test1536240391916.33@zmc.com;ip=10.0.0.19;port=41356;soapId=4e5021a7;] soap - AuthRequest elapsed=1
  1326. 2018-09-06 13:27:47,746 INFO [qtp788117692-1540:https:https://zmc-proxy/service/soap/] [name=test1536240391916.33@zmc.com;mid=127;ip=10.0.0.20;port=60062;soapId=4e5021a8;] calendar - <CreateCalendarItem> folderId=10, subject="Subject of meeting1536240403138.46", UID=dc8c432f-aed7-4916-8d19-b81c1f727423
  1327.  
  1328.  
  1329.  
  1330. […]
  1331.  
  1332.  
  1333.  
  1334. ==> /opt/zimbra/log/mailbox.log <==
  1335. 2018-09-06 13:27:54,166 INFO [qtp788117692-1848:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42638;soapId=4e5021ab;] misc - delegated access: doc=DeleteCalendarResource, authenticated account=admin@zmc.com, target account=resource2.1536240468061.52@zmc.com
  1336. 2018-09-06 13:27:54,256 INFO [qtp788117692-1848:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42638;soapId=4e5021ab;] mailbox - Creating mailbox with id 129 and group id 29 for resource2.1536240468061.52@zmc.com.
  1337. 2018-09-06 13:27:54,260 INFO [qtp788117692-1848:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42638;soapId=4e5021ab;] cache - initializing folder and tag caches for mailbox 129
  1338. 2018-09-06 13:27:54,492 INFO [qtp788117692-1848:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42638;soapId=4e5021ab;] mailbox - beginning listening on Redis notification channel NOTIFICATION-CHANNEL-29
  1339. 2018-09-06 13:27:54,492 INFO [qtp788117692-1848:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42638;soapId=4e5021ab;] mailbox - added account d4b0aa8a-015b-4dfc-8c6b-b56f828649fd to Redis notification channel NOTIFICATION-CHANNEL-29
  1340. 2018-09-06 13:27:54,493 INFO [redisson-3-6] [] mailbox - handling notification. accountId=d4b0aa8a-015b-4dfc-8c6b-b56f828649fd, changeId=1, channel=NOTIFICATION-CHANNEL-29
  1341. 2018-09-06 13:27:54,497 INFO [qtp788117692-1848:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42638;soapId=4e5021ab;] mbxmgr - Mailbox 129 account d4b0aa8a-015b-4dfc-8c6b-b56f828649fd CREATED
  1342. 2018-09-06 13:27:54,500 INFO [qtp788117692-1848:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42638;soapId=4e5021ab;] mailbox - Putting mailbox 129 under maintenance.
  1343. 2018-09-06 13:27:54,503 INFO [qtp788117692-1848:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42638;soapId=4e5021ab;] mailbox - clearing contents of mailbox 129, group 29
  1344. 2018-09-06 13:27:54,607 INFO [qtp788117692-1848:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42638;soapId=4e5021ab;] mbxmgr - Mailbox for account d4b0aa8a-015b-4dfc-8c6b-b56f828649fd DELETED
  1345. 2018-09-06 13:27:54,615 INFO [qtp788117692-1848:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42638;soapId=4e5021ab;] event - deleting events for account d4b0aa8a-015b-4dfc-8c6b-b56f828649fd
  1346. 2018-09-06 13:27:54,648 ERROR [redisson-3-6] [] mailbox - unable to deserialize notifications for accountId=d4b0aa8a-015b-4dfc-8c6b-b56f828649fd, changeId=1, channel=NOTIFICATION-CHANNEL-29
  1347. com.zimbra.cs.mailbox.MailServiceException$MailboxInMaintenanceException: mailbox in maintenance mode: 129
  1348. ExceptionId:redisson-3-6:1536240474647:a4a5f9c6e5824c5f
  1349. Code:mail.MAINTENANCE Arg:(mboxId, IID, "129")
  1350. at com.zimbra.cs.mailbox.MailServiceException.MAINTENANCE(MailServiceException.java:187)
  1351. at com.zimbra.cs.mailbox.Mailbox$MailboxTransaction.<init>(Mailbox.java:9810)
  1352. at com.zimbra.cs.mailbox.Mailbox$MailboxTransaction.<init>(Mailbox.java:9752)
  1353. at com.zimbra.cs.mailbox.Mailbox.mailboxReadTransaction(Mailbox.java:9724)
  1354. at com.zimbra.cs.mailbox.Mailbox.getItemById(Mailbox.java:2478)
  1355. at com.zimbra.cs.mailbox.Mailbox.getItemById(Mailbox.java:2473)
  1356. at com.zimbra.cs.mailbox.Mailbox.getFolderById(Mailbox.java:3611)
  1357. at com.zimbra.cs.session.PendingLocalModifications.fromSnapshot(PendingLocalModifications.java:346)
  1358. at com.zimbra.cs.mailbox.RedisPubSub$NotificationChannel.onMessage(RedisPubSub.java:158)
  1359. at com.zimbra.cs.mailbox.RedisPubSub$NotificationChannel.onMessage(RedisPubSub.java:102)
  1360. at org.redisson.PubSubMessageListener.onMessage(PubSubMessageListener.java:76)
  1361. at org.redisson.client.RedisPubSubConnection.onMessage(RedisPubSubConnection.java:81)
  1362. at org.redisson.client.handler.CommandPubSubDecoder$2.run(CommandPubSubDecoder.java:136)
  1363. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  1364. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  1365. at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
  1366. at java.lang.Thread.run(Thread.java:748)
  1367. 2018-09-06 13:27:54,649 INFO [qtp788117692-1848:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42638;soapId=4e5021ab;] soap - DeleteCalendarResourceRequest elapsed=484
  1368. 2018-09-06 13:27:54,666 INFO [qtp788117692-1795:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=54324;soapId=4e5021ac;] soap - AuthRequest elapsed=1
  1369.  
  1370.  
  1371. […]
  1372.  
  1373.  
  1374. ==> /opt/zimbra/log/mailbox.log <==
  1375. 2018-09-06 13:28:34,874 INFO [qtp788117692-1796:https:https://zmc-proxy/service/soap/] [name=test.1536240493305.69@zmc.com;ip=10.0.0.19;port=41558;soapId=4e5021c5;] mailbox - Creating mailbox with id 135 and group id 35 for test.1536240493305.69@zmc.com.
  1376. 2018-09-06 13:28:34,877 INFO [qtp788117692-1796:https:https://zmc-proxy/service/soap/] [name=test.1536240493305.69@zmc.com;ip=10.0.0.19;port=41558;soapId=4e5021c5;] cache - initializing folder and tag caches for mailbox 135
  1377. 2018-09-06 13:28:35,096 INFO [qtp788117692-1796:https:https://zmc-proxy/service/soap/] [name=test.1536240493305.69@zmc.com;ip=10.0.0.19;port=41558;soapId=4e5021c5;] mailbox - beginning listening on Redis notification channel NOTIFICATION-CHANNEL-35
  1378. 2018-09-06 13:28:35,096 INFO [qtp788117692-1796:https:https://zmc-proxy/service/soap/] [name=test.1536240493305.69@zmc.com;ip=10.0.0.19;port=41558;soapId=4e5021c5;] mailbox - added account 76da9416-3587-4e0c-925a-93437e968c24 to Redis notification channel NOTIFICATION-CHANNEL-35
  1379. 2018-09-06 13:28:35,097 INFO [redisson-3-32] [] mailbox - handling notification. accountId=76da9416-3587-4e0c-925a-93437e968c24, changeId=1, channel=NOTIFICATION-CHANNEL-35
  1380. 2018-09-06 13:28:35,098 INFO [qtp788117692-1796:https:https://zmc-proxy/service/soap/] [name=test.1536240493305.69@zmc.com;ip=10.0.0.19;port=41558;soapId=4e5021c5;] mbxmgr - Mailbox 135 account 76da9416-3587-4e0c-925a-93437e968c24 CREATED
  1381. 2018-09-06 13:28:35,104 INFO [qtp788117692-1796:https:https://zmc-proxy/service/soap/] [name=test.1536240493305.69@zmc.com;mid=135;ip=10.0.0.19;port=41558;soapId=4e5021c5;] mailop - adding contact email.1536240514765.86@domain.com: id=257, folderId=7, folderName=Contacts.
  1382. 2018-09-06 13:28:35,248 INFO [qtp788117692-1796:https:https://zmc-proxy/service/soap/] [name=test.1536240493305.69@zmc.com;mid=135;ip=10.0.0.19;port=41558;soapId=4e5021c5;] soap - CreateContactRequest elapsed=462
  1383. 2018-09-06 13:28:35,266 INFO [redisson-3-32] [] mailbox - handling notification. accountId=76da9416-3587-4e0c-925a-93437e968c24, changeId=2, channel=NOTIFICATION-CHANNEL-35
  1384. 2018-09-06 13:28:35,406 WARN [qtp788117692-1577:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=54502;soapId=4e5021c6;] event - Shutdown called for Event Notifier Executor; initiating shutdown sequence...
  1385. 2018-09-06 13:28:35,406 INFO [EventLogger-Worker-Thread-0] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@5fb458fe
  1386. 2018-09-06 13:28:35,407 INFO [EventLogger-Worker-Thread-0] [] ZooKeeper - Session: 0x0 closed
  1387. 2018-09-06 13:28:35,407 INFO [EventLogger-Worker-Thread-0-EventThread] [] ClientCnxn - EventThread shut down
  1388. 2018-09-06 13:28:35,407 ERROR [EventLogger-Worker-Thread-0] [] event - unable to send 1 events to Solr backend
  1389. com.zimbra.common.service.ServiceException: system failure: unexpected error running command com.zimbra.cs.index.solr.SolrUtils$4
  1390. ExceptionId:EventLogger-Worker-Thread-0:1536240515407:a4a5f9c6e5824c5f
  1391. Code:service.FAILURE
  1392. at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:288)
  1393. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:39)
  1394. at com.zimbra.cs.index.solr.SolrUtils.executeCloudRequestWithRetry(SolrUtils.java:320)
  1395. at com.zimbra.cs.index.solr.SolrCloudHelper.executeUpdateRequest(SolrCloudHelper.java:42)
  1396. at com.zimbra.cs.index.solr.SolrRequestHelper.executeUpdate(SolrRequestHelper.java:48)
  1397. at com.zimbra.cs.event.logger.SolrEventCallback.execute(SolrEventCallback.java:34)
  1398. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.send(BatchingEventLogger.java:126)
  1399. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.access$000(BatchingEventLogger.java:103)
  1400. at com.zimbra.cs.event.logger.BatchingEventLogger$1.onRemoval(BatchingEventLogger.java:53)
  1401. at com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:2013)
  1402. at com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3511)
  1403. at com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3487)
  1404. at com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3292)
  1405. at com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
  1406. at com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4829)
  1407. at com.zimbra.cs.event.logger.BatchingEventLogger.sendAllBatched(BatchingEventLogger.java:136)
  1408. at com.zimbra.cs.event.logger.BatchingEventLogger.shutdown(BatchingEventLogger.java:79)
  1409. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.shutdownEventLogHandlers(EventLogger.java:244)
  1410. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.run(EventLogger.java:215)
  1411. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  1412. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  1413. at java.lang.Thread.run(Thread.java:748)
  1414. Caused by: org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 within 10000 ms
  1415. at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:182)
  1416. at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:119)
  1417. at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:109)
  1418. at org.apache.solr.common.cloud.ZkStateReader.<init>(ZkStateReader.java:277)
  1419. at org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:155)
  1420. at org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:397)
  1421. at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:826)
  1422. at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
  1423. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
  1424. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
  1425. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:299)
  1426. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:295)
  1427. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:22)
  1428. ... 20 more
  1429. Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 within 10000 ms
  1430. at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:232)
  1431. at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:174)
  1432. ... 32 more
  1433. 2018-09-06 13:28:35,408 INFO [EventLogger-Worker-Thread-0-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181. Will not attempt to authenticate using SASL (unknown error)
  1434. 2018-09-06 13:28:35,408 INFO [qtp788117692-1577:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=54502;soapId=4e5021c6;] event - Event Notifier Executor shutdown was successful
  1435. 2018-09-06 13:28:35,408 INFO [qtp788117692-1577:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=54502;soapId=4e5021c6;] event - Event Queue Size 0
  1436. 2018-09-06 13:28:35,408 INFO [qtp788117692-1577:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=54502;soapId=4e5021c6;] event - Starting Event Notifier Logger with 10 threads; initial event queue size is 0
  1437. 2018-09-06 13:28:35,409 INFO [EventLogger-Worker-Thread-0-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181, initiating session
  1438. 2018-09-06 13:28:35,411 INFO [qtp788117692-1577:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=54502;soapId=4e5021c6;] soap - CreateCosRequest elapsed=13
  1439. 2018-09-06 13:28:35,434 INFO [qtp788117692-1659:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=42824;soapId=4e5021c7;] soap - CreateAccountRequest elapsed=4
  1440.  
  1441.  
  1442.  
  1443.  
  1444.  
  1445. […]
  1446.  
  1447.  
  1448.  
  1449.  
  1450.  
  1451. ==> /opt/zimbra/log/mailbox.log <==
  1452. 2018-09-06 13:35:19,033 INFO [qtp788117692-1700:https:https://zmc-proxy/service/soap/] [name=account.1536240915960214@zmc.com;ip=10.0.0.19;port=42726;soapId=4faf0d38;] soap - AuthRequest elapsed=1
  1453. 2018-09-06 13:35:19,130 INFO [qtp788117692-1561:https:https://zmc-proxy/service/soap/] [name=account.1536240915960214@zmc.com;ip=10.0.0.20;port=33202;soapId=4faf0d39;] mailbox - Creating mailbox with id 160 and group id 60 for account.1536240915960214@zmc.com.
  1454. 2018-09-06 13:35:19,133 INFO [qtp788117692-1561:https:https://zmc-proxy/service/soap/] [name=account.1536240915960214@zmc.com;ip=10.0.0.20;port=33202;soapId=4faf0d39;] cache - initializing folder and tag caches for mailbox 160
  1455. 2018-09-06 13:35:19,354 INFO [qtp788117692-1561:https:https://zmc-proxy/service/soap/] [name=account.1536240915960214@zmc.com;ip=10.0.0.20;port=33202;soapId=4faf0d39;] mailbox - beginning listening on Redis notification channel NOTIFICATION-CHANNEL-60
  1456. 2018-09-06 13:35:19,354 INFO [qtp788117692-1561:https:https://zmc-proxy/service/soap/] [name=account.1536240915960214@zmc.com;ip=10.0.0.20;port=33202;soapId=4faf0d39;] mailbox - added account 8935c3f3-70a2-4d30-9805-abd7d49aa8e7 to Redis notification channel NOTIFICATION-CHANNEL-60
  1457. 2018-09-06 13:35:19,355 INFO [redisson-3-13] [] mailbox - handling notification. accountId=8935c3f3-70a2-4d30-9805-abd7d49aa8e7, changeId=1, channel=NOTIFICATION-CHANNEL-60
  1458. 2018-09-06 13:35:19,355 INFO [qtp788117692-1561:https:https://zmc-proxy/service/soap/] [name=account.1536240915960214@zmc.com;ip=10.0.0.20;port=33202;soapId=4faf0d39;] mbxmgr - Mailbox 160 account 8935c3f3-70a2-4d30-9805-abd7d49aa8e7 CREATED
  1459. 2018-09-06 13:35:19,514 INFO [qtp788117692-1561:https:https://zmc-proxy/service/soap/] [name=account.1536240915960214@zmc.com;mid=160;ip=10.0.0.20;port=33202;soapId=4faf0d39;] smtp - Sending message to MTA at zmc-mta: Message-ID=<669216526.464.1536240919357.JavaMail.zimbra@zmc.com>, replyType=r, sender=account.1536240915960214@zmc.com, nrcpts=1, to=account.1536240915960215@zmc.com
  1460. 2018-09-06 13:35:19,514 INFO [EventLogger-Worker-Thread-8] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@2b53dca4
  1461. 2018-09-06 13:35:19,516 INFO [EventLogger-Worker-Thread-8-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181. Will not attempt to authenticate using SASL (unknown error)
  1462. 2018-09-06 13:35:19,517 INFO [EventLogger-Worker-Thread-8-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, initiating session
  1463. 2018-09-06 13:35:19,518 INFO [EventLogger-Worker-Thread-8-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, sessionid = 0x10000d48b75002b, negotiated timeout = 10000
  1464. 2018-09-06 13:35:19,519 INFO [zkConnectionManagerCallback-263-thread-1] [] ConnectionManager - zkClient has connected
  1465. 2018-09-06 13:35:19,521 INFO [redisson-3-13] [] mailbox - handling notification. accountId=8935c3f3-70a2-4d30-9805-abd7d49aa8e7, changeId=2, channel=NOTIFICATION-CHANNEL-60
  1466. 2018-09-06 13:35:19,521 INFO [EventLogger-Worker-Thread-8] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  1467. 2018-09-06 13:35:19,521 INFO [EventLogger-Worker-Thread-8] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  1468. 2018-09-06 13:35:19,534 INFO [redisson-3-20] [] mailbox - handling notification. accountId=8935c3f3-70a2-4d30-9805-abd7d49aa8e7, changeId=3, channel=NOTIFICATION-CHANNEL-60
  1469. 2018-09-06 13:35:19,540 INFO [redisson-3-29] [] mailbox - handling notification. accountId=8935c3f3-70a2-4d30-9805-abd7d49aa8e7, changeId=4, channel=NOTIFICATION-CHANNEL-60
  1470. 2018-09-06 13:35:19,543 INFO [qtp788117692-1561:https:https://zmc-proxy/service/soap/] [name=account.1536240915960214@zmc.com;mid=160;ip=10.0.0.20;port=33202;soapId=4faf0d39;] mailop - adding contact account.1536240915960215@zmc.com: id=258, folderId=13, folderName=Emailed Contacts.
  1471. 2018-09-06 13:35:19,679 INFO [redisson-3-4] [] mailbox - handling notification. accountId=8935c3f3-70a2-4d30-9805-abd7d49aa8e7, changeId=5, channel=NOTIFICATION-CHANNEL-60
  1472. 2018-09-06 13:35:19,679 INFO [qtp788117692-1561:https:https://zmc-proxy/service/soap/] [name=account.1536240915960214@zmc.com;mid=160;ip=10.0.0.20;port=33202;soapId=4faf0d39;] soap - SendMsgRequest elapsed=624
  1473.  
  1474. ==> /opt/zimbra/log/audit.log <==
  1475. 2018-09-06 13:35:19,033 INFO [qtp788117692-1700:https:https://zmc-proxy/service/soap/] [name=account.1536240915960214@zmc.com;ip=10.0.0.19;port=42726;soapId=4faf0d38;] security - cmd=Auth; account=account.1536240915960214@zmc.com; protocol=soap;
  1476.  
  1477. ==> /opt/zimbra/log/mailbox.log <==
  1478. 2018-09-06 13:35:21,457 INFO [MailboxPurge] [name=test.1536239362712.10@zmc.com;mid=65;] purge - Purging messages.
  1479.  
  1480. ==> /opt/zimbra/log/mailbox.log <==
  1481. 2018-09-06 13:35:22,779 INFO [MailboxPurge] [mid=70;] mbxmgr - Mailbox 70 account e3ffe96b-5035-4c5b-a363-2b6f9d153f0b LOADED
  1482. 2018-09-06 13:35:22,779 INFO [MailboxPurge] [name=test.1536239511200.64@zmc.com;mid=70;] purge - Purging messages.
  1483. 2018-09-06 13:35:22,781 INFO [MailboxPurge] [name=test.1536239511200.64@zmc.com;mid=70;] cache - initializing folder and tag caches for mailbox 70
  1484. 2018-09-06 13:35:22,765 INFO [qtp788117692-1743:https:https://zmc-proxy/service/soap/] [name=account.1536240915960216@zmc.com;ip=10.0.0.19;port=42752;soapId=4faf0d3a;] soap - AuthRequest elapsed=2
  1485.  
  1486. ==> /opt/zimbra/log/audit.log <==
  1487. 2018-09-06 13:35:22,764 INFO [qtp788117692-1743:https:https://zmc-proxy/service/soap/] [name=account.1536240915960216@zmc.com;ip=10.0.0.19;port=42752;soapId=4faf0d3a;] security - cmd=Auth; account=account.1536240915960216@zmc.com; protocol=soap;
  1488. 2018-09-06 13:35:22,883 INFO [qtp788117692-1886:https:https://zmc-proxy/service/soap/] [name=account.1536240915960216@zmc.com;ip=10.0.0.20;port=33228;soapId=4e50225d;] mailbox - Creating mailbox with id 161 and group id 61 for account.1536240915960216@zmc.com.
  1489. 2018-09-06 13:35:22,886 INFO [qtp788117692-1886:https:https://zmc-proxy/service/soap/] [name=account.1536240915960216@zmc.com;ip=10.0.0.20;port=33228;soapId=4e50225d;] cache - initializing folder and tag caches for mailbox 161
  1490. 2018-09-06 13:35:23,115 INFO [qtp788117692-1886:https:https://zmc-proxy/service/soap/] [name=account.1536240915960216@zmc.com;ip=10.0.0.20;port=33228;soapId=4e50225d;] mailbox - added account 52243954-52fa-42be-9158-b031567f7641 to Redis notification channel NOTIFICATION-CHANNEL-61
  1491. 2018-09-06 13:35:23,116 INFO [redisson-3-1] [] mailbox - handling notification. accountId=52243954-52fa-42be-9158-b031567f7641, changeId=1, channel=NOTIFICATION-CHANNEL-61
  1492. 2018-09-06 13:35:23,116 INFO [qtp788117692-1886:https:https://zmc-proxy/service/soap/] [name=account.1536240915960216@zmc.com;ip=10.0.0.20;port=33228;soapId=4e50225d;] mbxmgr - Mailbox 161 account 52243954-52fa-42be-9158-b031567f7641 CREATED
  1493. 2018-09-06 13:35:23,146 INFO [qtp788117692-1886:https:https://zmc-proxy/service/soap/] [name=account.1536240915960216@zmc.com;mid=161;ip=10.0.0.20;port=33228;soapId=4e50225d;] soap - GetFolderRequest elapsed=359
  1494. 2018-09-06 13:35:23,385 INFO [redisson-3-19] [] mailbox - handling notification. accountId=52243954-52fa-42be-9158-b031567f7641, changeId=2, channel=NOTIFICATION-CHANNEL-61
  1495. 2018-09-06 13:35:23,417 INFO [qtp788117692-1897:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44014;soapId=4e50225e;] soap - CreateAccountRequest elapsed=4
  1496.  
  1497. ==> /opt/zimbra/log/audit.log <==
  1498. 2018-09-06 13:35:23,416 INFO [qtp788117692-1897:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44014;soapId=4e50225e;] security - cmd=CreateAccount; name=account.1536240915960217@zmc.com;
  1499.  
  1500. ==> /opt/zimbra/log/mailbox.log <==
  1501. 2018-09-06 13:35:23,176 INFO [qtp788117692-1743:https:https://zmc-proxy/service/soap/] [name=account.1536240915960216@zmc.com;ip=10.0.0.19;port=42760;] mbxmgr - Mailbox 161 account 52243954-52fa-42be-9158-b031567f7641 LOADED
  1502. 2018-09-06 13:35:23,178 INFO [qtp788117692-1743:https:https://zmc-proxy/service/soap/] [name=account.1536240915960216@zmc.com;mid=161;ip=10.0.0.19;port=42760;soapId=4faf0d3b;] cache - initializing folder and tag caches for mailbox 161
  1503. 2018-09-06 13:35:23,384 INFO [notifyMboxListeners-1743] [] mailbox - beginning listening on Redis notification channel NOTIFICATION-CHANNEL-61
  1504. 2018-09-06 13:35:23,384 INFO [notifyMboxListeners-1743] [] mailbox - added account 52243954-52fa-42be-9158-b031567f7641 to Redis notification channel NOTIFICATION-CHANNEL-61
  1505. 2018-09-06 13:35:23,385 INFO [qtp788117692-1743:https:https://zmc-proxy/service/soap/] [name=account.1536240915960216@zmc.com;mid=161;ip=10.0.0.19;port=42760;soapId=4faf0d3b;] soap - AddMsgRequest elapsed=209
  1506. 2018-09-06 13:35:23,385 INFO [redisson-3-21] [] mailbox - handling notification. accountId=52243954-52fa-42be-9158-b031567f7641, changeId=2, channel=NOTIFICATION-CHANNEL-61
  1507. 2018-09-06 13:35:23,399 INFO [qtp788117692-1743:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55696;soapId=4faf0d3c;] soap - AuthRequest elapsed=0
  1508.  
  1509. ==> /opt/zimbra/log/audit.log <==
  1510. 2018-09-06 13:35:23,399 INFO [qtp788117692-1743:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55696;soapId=4faf0d3c;] security - cmd=AdminAuth; account=admin@zmc.com;
  1511. 2018-09-06 13:35:23,399 INFO [qtp788117692-1743:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55696;soapId=4faf0d3c;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  1512.  
  1513. ==> /opt/zimbra/log/mailbox.log <==
  1514. 2018-09-06 13:35:26,437 INFO [qtp788117692-1869:https:https://zmc-proxy/service/soap/] [name=account.1536240915960217@zmc.com;ip=10.0.0.20;port=33242;soapId=4e50225f;] soap - AuthRequest elapsed=2
  1515. 2018-09-06 13:35:26,551 INFO [qtp788117692-1916:https:https://zmc-proxy/service/soap/] [name=account.1536240915960217@zmc.com;ip=10.0.0.19;port=42774;soapId=4e502260;] mailbox - Creating mailbox with id 162 and group id 62 for account.1536240915960217@zmc.com.
  1516. 2018-09-06 13:35:26,555 INFO [qtp788117692-1916:https:https://zmc-proxy/service/soap/] [name=account.1536240915960217@zmc.com;ip=10.0.0.19;port=42774;soapId=4e502260;] cache - initializing folder and tag caches for mailbox 162
  1517.  
  1518. ==> /opt/zimbra/log/audit.log <==
  1519. 2018-09-06 13:35:26,436 INFO [qtp788117692-1869:https:https://zmc-proxy/service/soap/] [name=account.1536240915960217@zmc.com;ip=10.0.0.20;port=33242;soapId=4e50225f;] security - cmd=Auth; account=account.1536240915960217@zmc.com; protocol=soap;
  1520.  
  1521. ==> /opt/zimbra/log/mailbox.log <==
  1522. 2018-09-06 13:35:26,800 INFO [qtp788117692-1916:https:https://zmc-proxy/service/soap/] [name=account.1536240915960217@zmc.com;ip=10.0.0.19;port=42774;soapId=4e502260;] mailbox - added account 8edf7138-07ac-44e0-9c49-e2036eaf3a0b to Redis notification channel NOTIFICATION-CHANNEL-62
  1523. 2018-09-06 13:35:26,800 INFO [redisson-3-30] [] mailbox - handling notification. accountId=8edf7138-07ac-44e0-9c49-e2036eaf3a0b, changeId=1, channel=NOTIFICATION-CHANNEL-62
  1524. 2018-09-06 13:35:26,803 INFO [qtp788117692-1916:https:https://zmc-proxy/service/soap/] [name=account.1536240915960217@zmc.com;ip=10.0.0.19;port=42774;soapId=4e502260;] mbxmgr - Mailbox 162 account 8edf7138-07ac-44e0-9c49-e2036eaf3a0b CREATED
  1525. 2018-09-06 13:35:27,035 INFO [qtp788117692-1916:https:https://zmc-proxy/service/soap/] [name=account.1536240915960217@zmc.com;mid=162;ip=10.0.0.19;port=42774;soapId=4e502260;] soap - SaveDraftRequest elapsed=574
  1526. 2018-09-06 13:35:27,044 INFO [redisson-3-30] [] mailbox - handling notification. accountId=8edf7138-07ac-44e0-9c49-e2036eaf3a0b, changeId=2, channel=NOTIFICATION-CHANNEL-62
  1527. 2018-09-06 13:35:27,046 INFO [redisson-3-30] [] mailbox - handling notification. accountId=8edf7138-07ac-44e0-9c49-e2036eaf3a0b, changeId=3, channel=NOTIFICATION-CHANNEL-62
  1528. 2018-09-06 13:35:27,098 INFO [qtp788117692-1874:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55714;soapId=4e502261;] soap - CreateAccountRequest elapsed=4
  1529.  
  1530. ==> /opt/zimbra/log/audit.log <==
  1531. 2018-09-06 13:35:27,097 INFO [qtp788117692-1874:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55714;soapId=4e502261;] security - cmd=CreateAccount; name=account.1536240915960219@zmc.com;
  1532.  
  1533. ==> /opt/zimbra/log/mailbox.log <==
  1534. 2018-09-06 13:35:27,054 INFO [qtp788117692-1755:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55710;soapId=4faf0d3d;] soap - AuthRequest elapsed=1
  1535. 2018-09-06 13:35:27,076 INFO [qtp788117692-1733:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44028;soapId=4faf0d3e;] soap - CreateAccountRequest elapsed=5
  1536.  
  1537. ==> /opt/zimbra/log/audit.log <==
  1538. 2018-09-06 13:35:27,053 INFO [qtp788117692-1755:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55710;soapId=4faf0d3d;] security - cmd=AdminAuth; account=admin@zmc.com;
  1539. 2018-09-06 13:35:27,054 INFO [qtp788117692-1755:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55710;soapId=4faf0d3d;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  1540. 2018-09-06 13:35:27,075 INFO [qtp788117692-1733:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44028;soapId=4faf0d3e;] security - cmd=CreateAccount; name=account.1536240915960218@zmc.com;
  1541.  
  1542. ==> /opt/zimbra/log/mailbox.log <==
  1543. 2018-09-06 13:35:28,022 INFO [LmtpServer-5] [ip=10.0.0.35;] mailbox - Creating mailbox with id 163 and group id 63 for account.1536240915960215@zmc.com.
  1544. 2018-09-06 13:35:28,025 INFO [LmtpServer-5] [ip=10.0.0.35;] cache - initializing folder and tag caches for mailbox 163
  1545. 2018-09-06 13:35:28,258 INFO [LmtpServer-5] [ip=10.0.0.35;] mailbox - added account de37e45c-e318-4d46-879e-538afc5920a9 to Redis notification channel NOTIFICATION-CHANNEL-63
  1546. 2018-09-06 13:35:28,258 INFO [redisson-3-15] [] mailbox - handling notification. accountId=de37e45c-e318-4d46-879e-538afc5920a9, changeId=1, channel=NOTIFICATION-CHANNEL-63
  1547. 2018-09-06 13:35:28,259 INFO [LmtpServer-5] [ip=10.0.0.35;] mbxmgr - Mailbox 163 account de37e45c-e318-4d46-879e-538afc5920a9 CREATED
  1548. 2018-09-06 13:35:28,259 INFO [LmtpServer-5] [ip=10.0.0.35;] lmtp - Delivering message: size=1941 bytes, nrcpts=1, sender=account.1536240915960214@zmc.com, msgid=<669216526.464.1536240919357.JavaMail.zimbra@zmc.com>
  1549. 2018-09-06 13:35:28,450 INFO [redisson-3-15] [] mailbox - handling notification. accountId=de37e45c-e318-4d46-879e-538afc5920a9, changeId=2, channel=NOTIFICATION-CHANNEL-63
  1550. 2018-09-06 13:35:30,166 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=account.1536240915960218@zmc.com;ip=10.0.0.19;port=42790;soapId=4e502262;] soap - AuthRequest elapsed=2
  1551. 2018-09-06 13:35:30,278 INFO [qtp788117692-1906:https:https://zmc-proxy/service/soap/] [name=account.1536240915960218@zmc.com;ip=10.0.0.20;port=33266;soapId=4e502263;] mailbox - Creating mailbox with id 164 and group id 64 for account.1536240915960218@zmc.com.
  1552. 2018-09-06 13:35:30,281 INFO [qtp788117692-1906:https:https://zmc-proxy/service/soap/] [name=account.1536240915960218@zmc.com;ip=10.0.0.20;port=33266;soapId=4e502263;] cache - initializing folder and tag caches for mailbox 164
  1553. 2018-09-06 13:35:30,512 INFO [qtp788117692-1906:https:https://zmc-proxy/service/soap/] [name=account.1536240915960218@zmc.com;ip=10.0.0.20;port=33266;soapId=4e502263;] mailbox - added account f88eaebf-1180-4b19-98c1-43a432f78d22 to Redis notification channel NOTIFICATION-CHANNEL-64
  1554. 2018-09-06 13:35:30,513 INFO [redisson-3-23] [] mailbox - handling notification. accountId=f88eaebf-1180-4b19-98c1-43a432f78d22, changeId=1, channel=NOTIFICATION-CHANNEL-64
  1555. 2018-09-06 13:35:30,514 INFO [qtp788117692-1906:https:https://zmc-proxy/service/soap/] [name=account.1536240915960218@zmc.com;ip=10.0.0.20;port=33266;soapId=4e502263;] mbxmgr - Mailbox 164 account f88eaebf-1180-4b19-98c1-43a432f78d22 CREATED
  1556. 2018-09-06 13:35:30,706 INFO [qtp788117692-1906:https:https://zmc-proxy/service/soap/] [name=account.1536240915960218@zmc.com;mid=164;ip=10.0.0.20;port=33266;soapId=4e502263;] smtp - Sending message to MTA at zmc-mta: Message-ID=<42434506.431.1536240930515.JavaMail.zimbra@zmc.com>, replyType=r, sender=account.1536240915960218@zmc.com, nrcpts=1, to=account.1536240915960219@zmc.com
  1557. 2018-09-06 13:35:30,709 INFO [redisson-3-23] [] mailbox - handling notification. accountId=f88eaebf-1180-4b19-98c1-43a432f78d22, changeId=2, channel=NOTIFICATION-CHANNEL-64
  1558. 2018-09-06 13:35:30,724 INFO [redisson-3-27] [] mailbox - handling notification. accountId=f88eaebf-1180-4b19-98c1-43a432f78d22, changeId=3, channel=NOTIFICATION-CHANNEL-64
  1559. 2018-09-06 13:35:30,730 INFO [redisson-3-1] [] mailbox - handling notification. accountId=f88eaebf-1180-4b19-98c1-43a432f78d22, changeId=4, channel=NOTIFICATION-CHANNEL-64
  1560. 2018-09-06 13:35:30,734 INFO [qtp788117692-1906:https:https://zmc-proxy/service/soap/] [name=account.1536240915960218@zmc.com;mid=164;ip=10.0.0.20;port=33266;soapId=4e502263;] mailop - adding contact account.1536240915960219@zmc.com: id=258, folderId=13, folderName=Emailed Contacts.
  1561.  
  1562. ==> /opt/zimbra/log/audit.log <==
  1563. 2018-09-06 13:35:30,165 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=account.1536240915960218@zmc.com;ip=10.0.0.19;port=42790;soapId=4e502262;] security - cmd=Auth; account=account.1536240915960218@zmc.com; protocol=soap;
  1564.  
  1565. ==> /opt/zimbra/log/mailbox.log <==
  1566. 2018-09-06 13:35:30,942 INFO [qtp788117692-1745:https:https://zmc-proxy/service/soap/] [name=account.1536240915960218@zmc.com;ip=10.0.0.20;port=33282;] mbxmgr - Mailbox 164 account f88eaebf-1180-4b19-98c1-43a432f78d22 LOADED
  1567. 2018-09-06 13:35:30,943 INFO [qtp788117692-1745:https:https://zmc-proxy/service/soap/] [name=account.1536240915960218@zmc.com;mid=164;ip=10.0.0.20;port=33282;soapId=4faf0d3f;] cache - initializing folder and tag caches for mailbox 164
  1568.  
  1569. ==> /opt/zimbra/log/mailbox.log <==
  1570. 2018-09-06 13:35:30,893 INFO [redisson-3-7] [] mailbox - handling notification. accountId=f88eaebf-1180-4b19-98c1-43a432f78d22, changeId=5, channel=NOTIFICATION-CHANNEL-64
  1571. 2018-09-06 13:35:30,893 INFO [qtp788117692-1906:https:https://zmc-proxy/service/soap/] [name=account.1536240915960218@zmc.com;mid=164;ip=10.0.0.20;port=33266;soapId=4e502263;] soap - SendMsgRequest elapsed=705
  1572. 2018-09-06 13:35:30,921 INFO [qtp788117692-1849:https:https://zmc-proxy/service/soap/] [name=account.1536240915960218@zmc.com;mid=164;ip=10.0.0.19;port=42808;soapId=4e502264;] soap - GetMsgRequest elapsed=4
  1573. 2018-09-06 13:35:31,157 INFO [redisson-3-25] [] mailbox - handling notification. accountId=f88eaebf-1180-4b19-98c1-43a432f78d22, changeId=6, channel=NOTIFICATION-CHANNEL-64
  1574. 2018-09-06 13:35:31,274 INFO [qtp788117692-1897:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55748;soapId=4e502265;] soap - CreateAccountRequest elapsed=4
  1575. 2018-09-06 13:35:31,296 INFO [qtp788117692-1891:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44066;soapId=4e502266;] soap - CreateAccountRequest elapsed=4
  1576.  
  1577. ==> /opt/zimbra/log/audit.log <==
  1578. 2018-09-06 13:35:31,273 INFO [qtp788117692-1897:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55748;soapId=4e502265;] security - cmd=CreateAccount; name=account.1536240915960220@zmc.com;
  1579. 2018-09-06 13:35:31,295 INFO [qtp788117692-1891:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44066;soapId=4e502266;] security - cmd=CreateAccount; name=account.1536240915960221@zmc.com;
  1580. 2018-09-06 13:35:31,021 INFO [qtp788117692-1745:https:https://zmc-proxy/service/soap/] [name=account.1536240915960218@zmc.com;mid=164;ip=10.0.0.20;port=33282;soapId=4faf0d3f;] mailop - Deleting Message (id=257).
  1581. 2018-09-06 13:35:31,156 INFO [qtp788117692-1745:https:https://zmc-proxy/service/soap/] [name=account.1536240915960218@zmc.com;mid=164;ip=10.0.0.20;port=33282;soapId=4faf0d3f;] mailbox - added account f88eaebf-1180-4b19-98c1-43a432f78d22 to Redis notification channel NOTIFICATION-CHANNEL-64
  1582. 2018-09-06 13:35:31,157 INFO [redisson-3-31] [] mailbox - handling notification. accountId=f88eaebf-1180-4b19-98c1-43a432f78d22, changeId=6, channel=NOTIFICATION-CHANNEL-64
  1583. 2018-09-06 13:35:31,172 INFO [qtp788117692-1745:https:https://zmc-proxy/service/soap/] [name=account.1536240915960218@zmc.com;mid=164;ip=10.0.0.20;port=33282;soapId=4faf0d3f;] soap - MsgActionRequest elapsed=230
  1584. 2018-09-06 13:35:31,223 INFO [qtp788117692-1695:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44062;soapId=4faf0d40;] soap - AuthRequest elapsed=1
  1585.  
  1586. ==> /opt/zimbra/log/audit.log <==
  1587. 2018-09-06 13:35:31,222 INFO [qtp788117692-1695:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44062;soapId=4faf0d40;] security - cmd=AdminAuth; account=admin@zmc.com;
  1588. 2018-09-06 13:35:31,222 INFO [qtp788117692-1695:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44062;soapId=4faf0d40;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  1589.  
  1590. ==> /opt/zimbra/log/mailbox.log <==
  1591. 2018-09-06 13:35:34,433 INFO [qtp788117692-1928:https:https://zmc-proxy/service/soap/] [name=account.1536240915960220@zmc.com;ip=10.0.0.19;port=42830;soapId=4e502267;] mailbox - Creating mailbox with id 165 and group id 65 for account.1536240915960220@zmc.com.
  1592. 2018-09-06 13:35:34,436 INFO [qtp788117692-1928:https:https://zmc-proxy/service/soap/] [name=account.1536240915960220@zmc.com;ip=10.0.0.19;port=42830;soapId=4e502267;] cache - initializing folder and tag caches for mailbox 165
  1593. 2018-09-06 13:35:34,705 INFO [qtp788117692-1928:https:https://zmc-proxy/service/soap/] [name=account.1536240915960220@zmc.com;ip=10.0.0.19;port=42830;soapId=4e502267;] mailbox - beginning listening on Redis notification channel NOTIFICATION-CHANNEL-65
  1594. 2018-09-06 13:35:34,705 INFO [qtp788117692-1928:https:https://zmc-proxy/service/soap/] [name=account.1536240915960220@zmc.com;ip=10.0.0.19;port=42830;soapId=4e502267;] mailbox - added account ba6372c0-5695-47e1-bdd7-f6e666f03ff5 to Redis notification channel NOTIFICATION-CHANNEL-65
  1595. 2018-09-06 13:35:34,706 INFO [redisson-3-16] [] mailbox - handling notification. accountId=ba6372c0-5695-47e1-bdd7-f6e666f03ff5, changeId=1, channel=NOTIFICATION-CHANNEL-65
  1596. 2018-09-06 13:35:34,707 INFO [qtp788117692-1928:https:https://zmc-proxy/service/soap/] [name=account.1536240915960220@zmc.com;ip=10.0.0.19;port=42830;soapId=4e502267;] mbxmgr - Mailbox 165 account ba6372c0-5695-47e1-bdd7-f6e666f03ff5 CREATED
  1597.  
  1598. ==> /opt/zimbra/log/mailbox.log <==
  1599. 2018-09-06 13:35:34,316 INFO [qtp788117692-1541:https:https://zmc-proxy/service/soap/] [name=account.1536240915960220@zmc.com;ip=10.0.0.20;port=33298;soapId=4faf0d41;] soap - AuthRequest elapsed=1
  1600.  
  1601. ==> /opt/zimbra/log/audit.log <==
  1602. 2018-09-06 13:35:34,316 INFO [qtp788117692-1541:https:https://zmc-proxy/service/soap/] [name=account.1536240915960220@zmc.com;ip=10.0.0.20;port=33298;soapId=4faf0d41;] security - cmd=Auth; account=account.1536240915960220@zmc.com; protocol=soap;
  1603. 2018-09-06 13:35:34,924 INFO [qtp788117692-1928:https:https://zmc-proxy/service/soap/] [name=account.1536240915960220@zmc.com;mid=165;ip=10.0.0.19;port=42830;soapId=4e502267;] smtp - Sending message to MTA at zmc-mta: Message-ID=<508998178.432.1536240934709.JavaMail.zimbra@zmc.com>, replyType=r, sender=account.1536240915960220@zmc.com, nrcpts=1, to=account.1536240915960221@zmc.com
  1604. 2018-09-06 13:35:34,930 INFO [redisson-3-16] [] mailbox - handling notification. accountId=ba6372c0-5695-47e1-bdd7-f6e666f03ff5, changeId=2, channel=NOTIFICATION-CHANNEL-65
  1605. 2018-09-06 13:35:34,945 INFO [redisson-3-22] [] mailbox - handling notification. accountId=ba6372c0-5695-47e1-bdd7-f6e666f03ff5, changeId=3, channel=NOTIFICATION-CHANNEL-65
  1606. 2018-09-06 13:35:34,953 INFO [redisson-3-15] [] mailbox - handling notification. accountId=ba6372c0-5695-47e1-bdd7-f6e666f03ff5, changeId=4, channel=NOTIFICATION-CHANNEL-65
  1607. 2018-09-06 13:35:34,956 INFO [qtp788117692-1928:https:https://zmc-proxy/service/soap/] [name=account.1536240915960220@zmc.com;mid=165;ip=10.0.0.19;port=42830;soapId=4e502267;] mailop - adding contact account.1536240915960221@zmc.com: id=258, folderId=13, folderName=Emailed Contacts.
  1608. 2018-09-06 13:35:35,113 INFO [redisson-3-15] [] mailbox - handling notification. accountId=ba6372c0-5695-47e1-bdd7-f6e666f03ff5, changeId=5, channel=NOTIFICATION-CHANNEL-65
  1609. 2018-09-06 13:35:35,113 INFO [qtp788117692-1928:https:https://zmc-proxy/service/soap/] [name=account.1536240915960220@zmc.com;mid=165;ip=10.0.0.19;port=42830;soapId=4e502267;] soap - SendMsgRequest elapsed=772
  1610. 2018-09-06 13:35:35,130 INFO [qtp788117692-1945:https:https://zmc-proxy/service/soap/] [name=account.1536240915960221@zmc.com;ip=10.0.0.20;port=33316;soapId=4e502268;] soap - AuthRequest elapsed=2
  1611. 2018-09-06 13:35:35,245 INFO [qtp788117692-1945:https:https://zmc-proxy/service/soap/] [name=account.1536240915960221@zmc.com;ip=10.0.0.19;port=42848;soapId=4e502269;] mailbox - Creating mailbox with id 166 and group id 66 for account.1536240915960221@zmc.com.
  1612. 2018-09-06 13:35:35,248 INFO [qtp788117692-1945:https:https://zmc-proxy/service/soap/] [name=account.1536240915960221@zmc.com;ip=10.0.0.19;port=42848;soapId=4e502269;] cache - initializing folder and tag caches for mailbox 166
  1613. 2018-09-06 13:35:35,465 INFO [qtp788117692-1945:https:https://zmc-proxy/service/soap/] [name=account.1536240915960221@zmc.com;ip=10.0.0.19;port=42848;soapId=4e502269;] mailbox - beginning listening on Redis notification channel NOTIFICATION-CHANNEL-66
  1614. 2018-09-06 13:35:35,465 INFO [qtp788117692-1945:https:https://zmc-proxy/service/soap/] [name=account.1536240915960221@zmc.com;ip=10.0.0.19;port=42848;soapId=4e502269;] mailbox - added account 6aa2ad67-f114-46f7-8bd8-591a3a9c88cc to Redis notification channel NOTIFICATION-CHANNEL-66
  1615. 2018-09-06 13:35:35,465 INFO [redisson-3-4] [] mailbox - handling notification. accountId=6aa2ad67-f114-46f7-8bd8-591a3a9c88cc, changeId=1, channel=NOTIFICATION-CHANNEL-66
  1616. 2018-09-06 13:35:35,466 INFO [qtp788117692-1945:https:https://zmc-proxy/service/soap/] [name=account.1536240915960221@zmc.com;ip=10.0.0.19;port=42848;soapId=4e502269;] mbxmgr - Mailbox 166 account 6aa2ad67-f114-46f7-8bd8-591a3a9c88cc CREATED
  1617. 2018-09-06 13:35:35,493 INFO [qtp788117692-1945:https:https://zmc-proxy/service/soap/] [name=account.1536240915960221@zmc.com;mid=166;ip=10.0.0.19;port=42848;soapId=4e502269;] soap - SearchRequest elapsed=338
  1618. 2018-09-06 13:35:35,546 INFO [qtp788117692-1945:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55788;soapId=4e50226a;] soap - CreateAccountRequest elapsed=4
  1619.  
  1620. ==> /opt/zimbra/log/audit.log <==
  1621. 2018-09-06 13:35:35,130 INFO [qtp788117692-1945:https:https://zmc-proxy/service/soap/] [name=account.1536240915960221@zmc.com;ip=10.0.0.20;port=33316;soapId=4e502268;] security - cmd=Auth; account=account.1536240915960221@zmc.com; protocol=soap;
  1622. 2018-09-06 13:35:35,545 INFO [qtp788117692-1945:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55788;soapId=4e50226a;] security - cmd=CreateAccount; name=account.1536240915960223@zmc.com;
  1623.  
  1624. ==> /opt/zimbra/log/mailbox.log <==
  1625. 2018-09-06 13:35:35,508 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55784;soapId=4faf0d42;] soap - AuthRequest elapsed=1
  1626. 2018-09-06 13:35:35,525 INFO [qtp788117692-1717:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44102;soapId=4faf0d43;] soap - CreateAccountRequest elapsed=4
  1627.  
  1628. ==> /opt/zimbra/log/audit.log <==
  1629. 2018-09-06 13:35:35,507 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55784;soapId=4faf0d42;] security - cmd=AdminAuth; account=admin@zmc.com;
  1630. 2018-09-06 13:35:35,508 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55784;soapId=4faf0d42;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  1631. 2018-09-06 13:35:35,524 INFO [qtp788117692-1717:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44102;soapId=4faf0d43;] security - cmd=CreateAccount; name=account.1536240915960222@zmc.com;
  1632.  
  1633. ==> /opt/zimbra/log/mailbox.log <==
  1634. 2018-09-06 13:35:38,627 INFO [qtp788117692-1945:https:https://zmc-proxy/service/soap/] [name=account.1536240915960222@zmc.com;ip=10.0.0.19;port=42860;soapId=4e50226b;] soap - AuthRequest elapsed=2
  1635. 2018-09-06 13:35:38,733 INFO [qtp788117692-1945:https:https://zmc-proxy/service/soap/] [name=account.1536240915960222@zmc.com;ip=10.0.0.20;port=33336;soapId=4e50226c;] mailbox - Creating mailbox with id 167 and group id 67 for account.1536240915960222@zmc.com.
  1636. 2018-09-06 13:35:38,736 INFO [qtp788117692-1945:https:https://zmc-proxy/service/soap/] [name=account.1536240915960222@zmc.com;ip=10.0.0.20;port=33336;soapId=4e50226c;] cache - initializing folder and tag caches for mailbox 167
  1637.  
  1638. ==> /opt/zimbra/log/audit.log <==
  1639. 2018-09-06 13:35:38,626 INFO [qtp788117692-1945:https:https://zmc-proxy/service/soap/] [name=account.1536240915960222@zmc.com;ip=10.0.0.19;port=42860;soapId=4e50226b;] security - cmd=Auth; account=account.1536240915960222@zmc.com; protocol=soap;
  1640.  
  1641. ==> /opt/zimbra/log/mailbox.log <==
  1642. 2018-09-06 13:35:38,948 INFO [qtp788117692-1945:https:https://zmc-proxy/service/soap/] [name=account.1536240915960222@zmc.com;ip=10.0.0.20;port=33336;soapId=4e50226c;] mailbox - added account c970b92c-b109-4104-a8cf-32741e90a2e8 to Redis notification channel NOTIFICATION-CHANNEL-67
  1643. 2018-09-06 13:35:38,949 INFO [redisson-3-19] [] mailbox - handling notification. accountId=c970b92c-b109-4104-a8cf-32741e90a2e8, changeId=1, channel=NOTIFICATION-CHANNEL-67
  1644. 2018-09-06 13:35:38,949 INFO [qtp788117692-1945:https:https://zmc-proxy/service/soap/] [name=account.1536240915960222@zmc.com;ip=10.0.0.20;port=33336;soapId=4e50226c;] mbxmgr - Mailbox 167 account c970b92c-b109-4104-a8cf-32741e90a2e8 CREATED
  1645. 2018-09-06 13:35:39,124 INFO [qtp788117692-1945:https:https://zmc-proxy/service/soap/] [name=account.1536240915960222@zmc.com;mid=167;ip=10.0.0.20;port=33336;soapId=4e50226c;] smtp - Sending message to MTA at zmc-mta: Message-ID=<2124774574.433.1536240938951.JavaMail.zimbra@zmc.com>, replyType=r, sender=account.1536240915960222@zmc.com, nrcpts=1, to=account.1536240915960223@zmc.com
  1646. 2018-09-06 13:35:39,128 INFO [redisson-3-19] [] mailbox - handling notification. accountId=c970b92c-b109-4104-a8cf-32741e90a2e8, changeId=2, channel=NOTIFICATION-CHANNEL-67
  1647. 2018-09-06 13:35:39,141 INFO [redisson-3-12] [] mailbox - handling notification. accountId=c970b92c-b109-4104-a8cf-32741e90a2e8, changeId=3, channel=NOTIFICATION-CHANNEL-67
  1648. 2018-09-06 13:35:39,148 INFO [redisson-3-3] [] mailbox - handling notification. accountId=c970b92c-b109-4104-a8cf-32741e90a2e8, changeId=4, channel=NOTIFICATION-CHANNEL-67
  1649. 2018-09-06 13:35:39,152 INFO [qtp788117692-1945:https:https://zmc-proxy/service/soap/] [name=account.1536240915960222@zmc.com;mid=167;ip=10.0.0.20;port=33336;soapId=4e50226c;] mailop - adding contact account.1536240915960223@zmc.com: id=258, folderId=13, folderName=Emailed Contacts.
  1650. 2018-09-06 13:35:39,307 INFO [redisson-3-14] [] mailbox - handling notification. accountId=c970b92c-b109-4104-a8cf-32741e90a2e8, changeId=5, channel=NOTIFICATION-CHANNEL-67
  1651. 2018-09-06 13:35:39,307 INFO [qtp788117692-1945:https:https://zmc-proxy/service/soap/] [name=account.1536240915960222@zmc.com;mid=167;ip=10.0.0.20;port=33336;soapId=4e50226c;] soap - SendMsgRequest elapsed=659
  1652. 2018-09-06 13:35:39,338 INFO [qtp788117692-1905:https:https://zmc-proxy/service/soap/] [name=account.1536240915960222@zmc.com;mid=167;ip=10.0.0.19;port=42882;soapId=4e50226d;] mailop - Deleting Message (id=257).
  1653. 2018-09-06 13:35:39,497 INFO [redisson-3-2] [] mailbox - handling notification. accountId=c970b92c-b109-4104-a8cf-32741e90a2e8, changeId=6, channel=NOTIFICATION-CHANNEL-67
  1654. 2018-09-06 13:35:39,512 INFO [qtp788117692-1905:https:https://zmc-proxy/service/soap/] [name=account.1536240915960222@zmc.com;mid=167;ip=10.0.0.19;port=42882;soapId=4e50226d;] soap - MsgActionRequest elapsed=181
  1655.  
  1656. ==> /opt/zimbra/log/mailbox.log <==
  1657. 2018-09-06 13:35:39,143 INFO [LmtpServer-5] [ip=10.0.0.35;] mailbox - Creating mailbox with id 168 and group id 68 for account.1536240915960219@zmc.com.
  1658. 2018-09-06 13:35:39,146 INFO [LmtpServer-5] [ip=10.0.0.35;] cache - initializing folder and tag caches for mailbox 168
  1659. 2018-09-06 13:35:39,384 INFO [LmtpServer-5] [ip=10.0.0.35;] mailbox - added account 4ab81c6c-b56c-48c9-8564-16bbf3b31c28 to Redis notification channel NOTIFICATION-CHANNEL-68
  1660. 2018-09-06 13:35:39,385 INFO [redisson-3-27] [] mailbox - handling notification. accountId=4ab81c6c-b56c-48c9-8564-16bbf3b31c28, changeId=1, channel=NOTIFICATION-CHANNEL-68
  1661. 2018-09-06 13:35:39,386 INFO [LmtpServer-5] [ip=10.0.0.35;] mbxmgr - Mailbox 168 account 4ab81c6c-b56c-48c9-8564-16bbf3b31c28 CREATED
  1662. 2018-09-06 13:35:39,387 INFO [LmtpServer-5] [ip=10.0.0.35;] lmtp - Delivering message: size=1903 bytes, nrcpts=1, sender=account.1536240915960218@zmc.com, msgid=<42434506.431.1536240930515.JavaMail.zimbra@zmc.com>
  1663. 2018-09-06 13:35:39,528 INFO [qtp788117692-1729:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55818;soapId=4faf0d44;] soap - AuthRequest elapsed=1
  1664. 2018-09-06 13:35:39,585 INFO [qtp788117692-1755:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44140;soapId=4faf0d45;] soap - CreateAccountRequest elapsed=4
  1665. 2018-09-06 13:35:39,635 INFO [EventLogger-Worker-Thread-9] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@4e2011a1
  1666. 2018-09-06 13:35:39,636 INFO [EventLogger-Worker-Thread-9-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181. Will not attempt to authenticate using SASL (unknown error)
  1667. 2018-09-06 13:35:39,637 INFO [EventLogger-Worker-Thread-9-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, initiating session
  1668. 2018-09-06 13:35:39,638 INFO [EventLogger-Worker-Thread-9-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, sessionid = 0x20000d48aa8002e, negotiated timeout = 10000
  1669. 2018-09-06 13:35:39,639 INFO [zkConnectionManagerCallback-266-thread-1] [] ConnectionManager - zkClient has connected
  1670. 2018-09-06 13:35:39,640 INFO [EventLogger-Worker-Thread-9] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  1671. 2018-09-06 13:35:39,641 INFO [EventLogger-Worker-Thread-9] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  1672. 2018-09-06 13:35:39,654 INFO [redisson-3-27] [] mailbox - handling notification. accountId=4ab81c6c-b56c-48c9-8564-16bbf3b31c28, changeId=2, channel=NOTIFICATION-CHANNEL-68
  1673.  
  1674. ==> /opt/zimbra/log/audit.log <==
  1675. 2018-09-06 13:35:39,527 INFO [qtp788117692-1729:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55818;soapId=4faf0d44;] security - cmd=AdminAuth; account=admin@zmc.com;
  1676. 2018-09-06 13:35:39,528 INFO [qtp788117692-1729:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55818;soapId=4faf0d44;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  1677. 2018-09-06 13:35:39,585 INFO [qtp788117692-1755:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44140;soapId=4faf0d45;] security - cmd=CreateAccount; name=account.1536240915960226@zmc.com;
  1678. 2018-09-06 13:35:42,608 INFO [qtp788117692-1905:https:https://zmc-proxy/service/soap/] [name=account.1536240915960226@zmc.com;ip=10.0.0.20;port=33370;soapId=4e50226e;] soap - AuthRequest elapsed=2
  1679.  
  1680. ==> /opt/zimbra/log/audit.log <==
  1681. 2018-09-06 13:35:42,607 INFO [qtp788117692-1905:https:https://zmc-proxy/service/soap/] [name=account.1536240915960226@zmc.com;ip=10.0.0.20;port=33370;soapId=4e50226e;] security - cmd=Auth; account=account.1536240915960226@zmc.com; protocol=soap;
  1682.  
  1683. ==> /var/log/zimbra.log <==
  1684. Sep 6 13:35:42 zmc-mailbox zmconfigd[1703]: Fetching All configs
  1685.  
  1686. ==> /opt/zimbra/log/mailbox.log <==
  1687. 2018-09-06 13:35:42,724 INFO [qtp788117692-1745:https:https://zmc-proxy/service/soap/] [name=account.1536240915960226@zmc.com;ip=10.0.0.19;port=42902;soapId=4faf0d46;] mailbox - Creating mailbox with id 169 and group id 69 for account.1536240915960226@zmc.com.
  1688. 2018-09-06 13:35:42,727 INFO [qtp788117692-1745:https:https://zmc-proxy/service/soap/] [name=account.1536240915960226@zmc.com;ip=10.0.0.19;port=42902;soapId=4faf0d46;] cache - initializing folder and tag caches for mailbox 169
  1689. 2018-09-06 13:35:42,985 INFO [qtp788117692-1745:https:https://zmc-proxy/service/soap/] [name=account.1536240915960226@zmc.com;ip=10.0.0.19;port=42902;soapId=4faf0d46;] mailbox - beginning listening on Redis notification channel NOTIFICATION-CHANNEL-69
  1690. 2018-09-06 13:35:42,985 INFO [qtp788117692-1745:https:https://zmc-proxy/service/soap/] [name=account.1536240915960226@zmc.com;ip=10.0.0.19;port=42902;soapId=4faf0d46;] mailbox - added account 5b4b670a-6814-45c0-872a-55ebc0aacaab to Redis notification channel NOTIFICATION-CHANNEL-69
  1691. 2018-09-06 13:35:42,986 INFO [redisson-3-8] [] mailbox - handling notification. accountId=5b4b670a-6814-45c0-872a-55ebc0aacaab, changeId=1, channel=NOTIFICATION-CHANNEL-69
  1692. 2018-09-06 13:35:42,987 INFO [qtp788117692-1745:https:https://zmc-proxy/service/soap/] [name=account.1536240915960226@zmc.com;ip=10.0.0.19;port=42902;soapId=4faf0d46;] mbxmgr - Mailbox 169 account 5b4b670a-6814-45c0-872a-55ebc0aacaab CREATED
  1693.  
  1694. ==> /var/log/zimbra.log <==
  1695. Sep 6 13:35:42 zmc-mailbox zmconfigd[1705]: Fetching All configs
  1696. Sep 6 13:35:42 zmc-mailbox zmconfigd[1705]: All configs fetched in 0.03 seconds
  1697. Sep 6 13:35:42 zmc-mailbox zmconfigd[1705]: Command not defined for imapd
  1698.  
  1699. ==> /opt/zimbra/log/mailbox.log <==
  1700. 2018-09-06 13:35:43,495 INFO [LmtpServer-5] [ip=10.0.0.35;] lmtp - Delivering message: size=1941 bytes, nrcpts=1, sender=account.1536240915960220@zmc.com, msgid=<508998178.432.1536240934709.JavaMail.zimbra@zmc.com>
  1701. 2018-09-06 13:35:43,496 INFO [qtp788117692-1903:https:https://zmc-proxy/service/soap/] [name=account.1536240915960226@zmc.com;ip=10.0.0.20;port=33394;] mbxmgr - Mailbox 169 account 5b4b670a-6814-45c0-872a-55ebc0aacaab LOADED
  1702. 2018-09-06 13:35:43,498 INFO [qtp788117692-1903:https:https://zmc-proxy/service/soap/] [name=account.1536240915960226@zmc.com;mid=169;ip=10.0.0.20;port=33394;soapId=4e50226f;] cache - initializing folder and tag caches for mailbox 169
  1703. 2018-09-06 13:35:43,569 INFO [qtp788117692-1903:https:https://zmc-proxy/service/soap/] [name=account.1536240915960226@zmc.com;mid=169;ip=10.0.0.20;port=33394;soapId=4e50226f;] soap - GetMsgMetadataRequest elapsed=73
  1704. 2018-09-06 13:35:43,584 INFO [qtp788117692-1884:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44174;soapId=4e502270;] soap - AuthRequest elapsed=0
  1705. 2018-09-06 13:35:43,673 INFO [redisson-3-8] [] mailbox - handling notification. accountId=6aa2ad67-f114-46f7-8bd8-591a3a9c88cc, changeId=3, channel=NOTIFICATION-CHANNEL-66
  1706.  
  1707. ==> /opt/zimbra/log/audit.log <==
  1708. 2018-09-06 13:35:43,584 INFO [qtp788117692-1884:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44174;soapId=4e502270;] security - cmd=AdminAuth; account=admin@zmc.com;
  1709. 2018-09-06 13:35:43,584 INFO [qtp788117692-1884:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44174;soapId=4e502270;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  1710.  
  1711. ==> /var/log/zimbra.log <==
  1712. Sep 6 13:35:42 zmc-mailbox zmconfigd[1703]: All configs fetched in 0.03 seconds
  1713. Sep 6 13:35:42 zmc-mailbox zmconfigd[1703]: Command not defined for imapd
  1714.  
  1715. ==> /opt/zimbra/log/mailbox.log <==
  1716. 2018-09-06 13:35:43,172 INFO [qtp788117692-1745:https:https://zmc-proxy/service/soap/] [name=account.1536240915960226@zmc.com;mid=169;ip=10.0.0.19;port=42902;soapId=4faf0d46;] smtp - Sending message to MTA at zmc-mta: Message-ID=<1193072669.465.1536240942989.JavaMail.zimbra@zmc.com>, replyType=r, sender=account.1536240915960226@zmc.com, nrcpts=1, to=account.1536240915960226@zmc.com
  1717. 2018-09-06 13:35:43,172 INFO [EventLogger-Worker-Thread-0] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@4b50729e
  1718. 2018-09-06 13:35:43,173 INFO [EventLogger-Worker-Thread-0-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181. Will not attempt to authenticate using SASL (unknown error)
  1719. 2018-09-06 13:35:43,173 INFO [EventLogger-Worker-Thread-0-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, initiating session
  1720. 2018-09-06 13:35:43,174 INFO [redisson-3-8] [] mailbox - handling notification. accountId=5b4b670a-6814-45c0-872a-55ebc0aacaab, changeId=2, channel=NOTIFICATION-CHANNEL-69
  1721. 2018-09-06 13:35:43,175 INFO [EventLogger-Worker-Thread-0-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, sessionid = 0x10000d48b75002c, negotiated timeout = 10000
  1722. 2018-09-06 13:35:43,176 INFO [zkConnectionManagerCallback-269-thread-1] [] ConnectionManager - zkClient has connected
  1723. 2018-09-06 13:35:43,179 INFO [EventLogger-Worker-Thread-0] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  1724. 2018-09-06 13:35:43,180 INFO [EventLogger-Worker-Thread-0] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  1725. 2018-09-06 13:35:43,191 INFO [redisson-3-24] [] mailbox - handling notification. accountId=5b4b670a-6814-45c0-872a-55ebc0aacaab, changeId=3, channel=NOTIFICATION-CHANNEL-69
  1726. 2018-09-06 13:35:43,198 INFO [redisson-3-5] [] mailbox - handling notification. accountId=5b4b670a-6814-45c0-872a-55ebc0aacaab, changeId=4, channel=NOTIFICATION-CHANNEL-69
  1727. 2018-09-06 13:35:43,202 INFO [qtp788117692-1745:https:https://zmc-proxy/service/soap/] [name=account.1536240915960226@zmc.com;mid=169;ip=10.0.0.19;port=42902;soapId=4faf0d46;] mailop - adding contact account.1536240915960226@zmc.com: id=258, folderId=13, folderName=Emailed Contacts.
  1728. 2018-09-06 13:35:43,445 INFO [redisson-3-7] [] mailbox - handling notification. accountId=5b4b670a-6814-45c0-872a-55ebc0aacaab, changeId=5, channel=NOTIFICATION-CHANNEL-69
  1729. 2018-09-06 13:35:43,445 INFO [qtp788117692-1745:https:https://zmc-proxy/service/soap/] [name=account.1536240915960226@zmc.com;mid=169;ip=10.0.0.19;port=42902;soapId=4faf0d46;] soap - SendMsgRequest elapsed=813
  1730. 2018-09-06 13:35:43,635 INFO [qtp788117692-1541:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55860;soapId=4faf0d47;] soap - CreateAccountRequest elapsed=4
  1731. 2018-09-06 13:35:43,671 INFO [qtp788117692-1700:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44178;soapId=4faf0d48;] soap - CreateAccountRequest elapsed=4
  1732.  
  1733. ==> /opt/zimbra/log/audit.log <==
  1734. 2018-09-06 13:35:43,634 INFO [qtp788117692-1541:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55860;soapId=4faf0d47;] security - cmd=CreateAccount; name=account.1536240915960224@zmc.com;
  1735. 2018-09-06 13:35:43,670 INFO [qtp788117692-1700:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44178;soapId=4faf0d48;] security - cmd=CreateAccount; name=account.1536240915960225@zmc.com;
  1736.  
  1737. ==> /var/log/zimbra.log <==
  1738. Sep 6 13:35:43 zmc-mailbox zmconfigd[1705]: Watchdog: skipping service antivirus. Service not yet available for restarts.
  1739. Sep 6 13:35:43 zmc-mailbox zmconfigd[1705]: All rewrite threads completed in 0.00 sec
  1740. Sep 6 13:35:43 zmc-mailbox zmconfigd[1705]: All restarts completed in 0.00 sec
  1741. Sep 6 13:35:43 zmc-mailbox zmconfigd[1703]: Watchdog: skipping service antivirus. Service not yet available for restarts.
  1742. Sep 6 13:35:43 zmc-mailbox zmconfigd[1703]: All rewrite threads completed in 0.00 sec
  1743. Sep 6 13:35:43 zmc-mailbox zmconfigd[1703]: All restarts completed in 0.00 sec
  1744.  
  1745. ==> /opt/zimbra/log/mailbox.log <==
  1746. 2018-09-06 13:35:46,691 INFO [qtp788117692-1916:https:https://zmc-proxy/service/soap/] [name=account.1536240915960224@zmc.com;ip=10.0.0.20;port=33406;soapId=4e502271;] soap - AuthRequest elapsed=2
  1747.  
  1748. ==> /opt/zimbra/log/audit.log <==
  1749. 2018-09-06 13:35:46,690 INFO [qtp788117692-1916:https:https://zmc-proxy/service/soap/] [name=account.1536240915960224@zmc.com;ip=10.0.0.20;port=33406;soapId=4e502271;] security - cmd=Auth; account=account.1536240915960224@zmc.com; protocol=soap;
  1750.  
  1751. ==> /opt/zimbra/log/mailbox.log <==
  1752. 2018-09-06 13:35:46,800 INFO [qtp788117692-1723:https:https://zmc-proxy/service/soap/] [name=account.1536240915960224@zmc.com;ip=10.0.0.19;port=42938;soapId=4faf0d49;] mailbox - Creating mailbox with id 170 and group id 70 for account.1536240915960224@zmc.com.
  1753. 2018-09-06 13:35:46,804 INFO [qtp788117692-1723:https:https://zmc-proxy/service/soap/] [name=account.1536240915960224@zmc.com;ip=10.0.0.19;port=42938;soapId=4faf0d49;] cache - initializing folder and tag caches for mailbox 170
  1754.  
  1755. ==> /opt/zimbra/log/mailbox.log <==
  1756. 2018-09-06 13:35:47,420 INFO [qtp788117692-1915:https:https://zmc-proxy/service/soap/] [name=account.1536240915960224@zmc.com;ip=10.0.0.20;port=33424;] mbxmgr - Mailbox 170 account 286cbe22-f300-42c4-8a78-3eb232b978b1 LOADED
  1757. 2018-09-06 13:35:47,422 INFO [qtp788117692-1915:https:https://zmc-proxy/service/soap/] [name=account.1536240915960224@zmc.com;mid=170;ip=10.0.0.20;port=33424;soapId=4e502272;] cache - initializing folder and tag caches for mailbox 170
  1758. 2018-09-06 13:35:47,537 INFO [qtp788117692-1915:https:https://zmc-proxy/service/soap/] [name=account.1536240915960224@zmc.com;mid=170;ip=10.0.0.20;port=33424;soapId=4e502272;] mailbox - beginning listening on Redis notification channel NOTIFICATION-CHANNEL-70
  1759. 2018-09-06 13:35:47,538 INFO [qtp788117692-1915:https:https://zmc-proxy/service/soap/] [name=account.1536240915960224@zmc.com;mid=170;ip=10.0.0.20;port=33424;soapId=4e502272;] mailbox - added account 286cbe22-f300-42c4-8a78-3eb232b978b1 to Redis notification channel NOTIFICATION-CHANNEL-70
  1760. 2018-09-06 13:35:47,538 INFO [qtp788117692-1915:https:https://zmc-proxy/service/soap/] [name=account.1536240915960224@zmc.com;mid=170;ip=10.0.0.20;port=33424;soapId=4e502272;] soap - ItemActionRequest elapsed=118
  1761. 2018-09-06 13:35:47,539 INFO [redisson-3-32] [] mailbox - handling notification. accountId=286cbe22-f300-42c4-8a78-3eb232b978b1, changeId=6, channel=NOTIFICATION-CHANNEL-70
  1762. 2018-09-06 13:35:47,555 INFO [qtp788117692-1906:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44204;soapId=4e502273;] soap - AuthRequest elapsed=1
  1763.  
  1764. ==> /opt/zimbra/log/audit.log <==
  1765. 2018-09-06 13:35:47,554 INFO [qtp788117692-1906:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44204;soapId=4e502273;] security - cmd=AdminAuth; account=admin@zmc.com;
  1766. 2018-09-06 13:35:47,554 INFO [qtp788117692-1906:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44204;soapId=4e502273;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  1767. 2018-09-06 13:35:47,024 INFO [qtp788117692-1723:https:https://zmc-proxy/service/soap/] [name=account.1536240915960224@zmc.com;ip=10.0.0.19;port=42938;soapId=4faf0d49;] mailbox - added account 286cbe22-f300-42c4-8a78-3eb232b978b1 to Redis notification channel NOTIFICATION-CHANNEL-70
  1768. 2018-09-06 13:35:47,024 INFO [redisson-3-19] [] mailbox - handling notification. accountId=286cbe22-f300-42c4-8a78-3eb232b978b1, changeId=1, channel=NOTIFICATION-CHANNEL-70
  1769. 2018-09-06 13:35:47,025 INFO [qtp788117692-1723:https:https://zmc-proxy/service/soap/] [name=account.1536240915960224@zmc.com;ip=10.0.0.19;port=42938;soapId=4faf0d49;] mbxmgr - Mailbox 170 account 286cbe22-f300-42c4-8a78-3eb232b978b1 CREATED
  1770. 2018-09-06 13:35:47,212 INFO [qtp788117692-1723:https:https://zmc-proxy/service/soap/] [name=account.1536240915960224@zmc.com;mid=170;ip=10.0.0.19;port=42938;soapId=4faf0d49;] smtp - Sending message to MTA at zmc-mta: Message-ID=<1701883999.466.1536240947027.JavaMail.zimbra@zmc.com>, replyType=r, sender=account.1536240915960224@zmc.com, nrcpts=1, to=account.1536240915960225@zmc.com
  1771. 2018-09-06 13:35:47,216 INFO [redisson-3-19] [] mailbox - handling notification. accountId=286cbe22-f300-42c4-8a78-3eb232b978b1, changeId=2, channel=NOTIFICATION-CHANNEL-70
  1772. 2018-09-06 13:35:47,228 INFO [redisson-3-17] [] mailbox - handling notification. accountId=286cbe22-f300-42c4-8a78-3eb232b978b1, changeId=3, channel=NOTIFICATION-CHANNEL-70
  1773. 2018-09-06 13:35:47,234 INFO [redisson-3-12] [] mailbox - handling notification. accountId=286cbe22-f300-42c4-8a78-3eb232b978b1, changeId=4, channel=NOTIFICATION-CHANNEL-70
  1774. 2018-09-06 13:35:47,237 INFO [qtp788117692-1723:https:https://zmc-proxy/service/soap/] [name=account.1536240915960224@zmc.com;mid=170;ip=10.0.0.19;port=42938;soapId=4faf0d49;] mailop - adding contact account.1536240915960225@zmc.com: id=258, folderId=13, folderName=Emailed Contacts.
  1775. 2018-09-06 13:35:47,388 INFO [redisson-3-23] [] mailbox - handling notification. accountId=286cbe22-f300-42c4-8a78-3eb232b978b1, changeId=5, channel=NOTIFICATION-CHANNEL-70
  1776. 2018-09-06 13:35:47,388 INFO [qtp788117692-1723:https:https://zmc-proxy/service/soap/] [name=account.1536240915960224@zmc.com;mid=170;ip=10.0.0.19;port=42938;soapId=4faf0d49;] soap - SendMsgRequest elapsed=673
  1777. 2018-09-06 13:35:47,538 INFO [redisson-3-21] [] mailbox - handling notification. accountId=286cbe22-f300-42c4-8a78-3eb232b978b1, changeId=6, channel=NOTIFICATION-CHANNEL-70
  1778. 2018-09-06 13:35:47,578 INFO [LmtpServer-5] [ip=10.0.0.35;] mailbox - Creating mailbox with id 171 and group id 71 for account.1536240915960223@zmc.com.
  1779. 2018-09-06 13:35:47,580 INFO [LmtpServer-5] [ip=10.0.0.35;] cache - initializing folder and tag caches for mailbox 171
  1780. 2018-09-06 13:35:47,810 INFO [LmtpServer-5] [ip=10.0.0.35;] mailbox - added account e2183048-7d8e-4ed2-9c16-823060bc400b to Redis notification channel NOTIFICATION-CHANNEL-71
  1781. 2018-09-06 13:35:47,811 INFO [redisson-3-27] [] mailbox - handling notification. accountId=e2183048-7d8e-4ed2-9c16-823060bc400b, changeId=1, channel=NOTIFICATION-CHANNEL-71
  1782. 2018-09-06 13:35:47,812 INFO [LmtpServer-5] [ip=10.0.0.35;] mbxmgr - Mailbox 171 account e2183048-7d8e-4ed2-9c16-823060bc400b CREATED
  1783. 2018-09-06 13:35:47,812 INFO [LmtpServer-5] [ip=10.0.0.35;] lmtp - Delivering message: size=1942 bytes, nrcpts=1, sender=account.1536240915960222@zmc.com, msgid=<2124774574.433.1536240938951.JavaMail.zimbra@zmc.com>
  1784. 2018-09-06 13:35:47,816 INFO [qtp788117692-1743:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55890;soapId=4faf0d4a;] soap - CreateAccountRequest elapsed=5
  1785. 2018-09-06 13:35:47,981 INFO [EventLogger-Worker-Thread-2] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@4ddd7e1a
  1786. 2018-09-06 13:35:47,982 INFO [EventLogger-Worker-Thread-2-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181. Will not attempt to authenticate using SASL (unknown error)
  1787. 2018-09-06 13:35:47,983 INFO [EventLogger-Worker-Thread-2-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, initiating session
  1788. 2018-09-06 13:35:47,984 INFO [EventLogger-Worker-Thread-2-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, sessionid = 0x10000d48b75002d, negotiated timeout = 10000
  1789. 2018-09-06 13:35:47,985 INFO [zkConnectionManagerCallback-272-thread-1] [] ConnectionManager - zkClient has connected
  1790. 2018-09-06 13:35:47,986 INFO [EventLogger-Worker-Thread-2] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  1791. 2018-09-06 13:35:47,987 INFO [EventLogger-Worker-Thread-2] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  1792. 2018-09-06 13:35:48,002 INFO [redisson-3-27] [] mailbox - handling notification. accountId=e2183048-7d8e-4ed2-9c16-823060bc400b, changeId=2, channel=NOTIFICATION-CHANNEL-71
  1793.  
  1794. ==> /opt/zimbra/log/audit.log <==
  1795. 2018-09-06 13:35:47,815 INFO [qtp788117692-1743:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55890;soapId=4faf0d4a;] security - cmd=CreateAccount; name=account.1536240915960227@zmc.com;
  1796.  
  1797. ==> /opt/zimbra/log/mailbox.log <==
  1798. 2018-09-06 13:35:50,836 INFO [qtp788117692-1743:https:https://zmc-proxy/service/soap/] [name=account.1536240915960227@zmc.com;ip=10.0.0.19;port=42964;soapId=4faf0d4b;] soap - AuthRequest elapsed=1
  1799. 2018-09-06 13:35:50,946 INFO [qtp788117692-1740:https:https://zmc-proxy/service/soap/] [name=account.1536240915960227@zmc.com;ip=10.0.0.20;port=33440;soapId=4faf0d4c;] mailbox - Creating mailbox with id 172 and group id 72 for account.1536240915960227@zmc.com.
  1800. 2018-09-06 13:35:50,949 INFO [qtp788117692-1740:https:https://zmc-proxy/service/soap/] [name=account.1536240915960227@zmc.com;ip=10.0.0.20;port=33440;soapId=4faf0d4c;] cache - initializing folder and tag caches for mailbox 172
  1801.  
  1802. ==> /opt/zimbra/log/audit.log <==
  1803. 2018-09-06 13:35:50,836 INFO [qtp788117692-1743:https:https://zmc-proxy/service/soap/] [name=account.1536240915960227@zmc.com;ip=10.0.0.19;port=42964;soapId=4faf0d4b;] security - cmd=Auth; account=account.1536240915960227@zmc.com; protocol=soap;
  1804.  
  1805. ==> /opt/zimbra/log/mailbox.log <==
  1806. 2018-09-06 13:35:51,697 INFO [qtp788117692-1886:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55924;soapId=4e502274;] soap - AuthRequest elapsed=0
  1807. 2018-09-06 13:35:51,708 INFO [LmtpServer-5] [ip=10.0.0.35;] lmtp - Delivering message: size=1942 bytes, nrcpts=1, sender=account.1536240915960226@zmc.com, msgid=<1193072669.465.1536240942989.JavaMail.zimbra@zmc.com>
  1808. 2018-09-06 13:35:51,751 INFO [qtp788117692-1897:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44242;soapId=4e502275;] soap - CreateAccountRequest elapsed=5
  1809.  
  1810. ==> /opt/zimbra/log/audit.log <==
  1811. 2018-09-06 13:35:51,697 INFO [qtp788117692-1886:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55924;soapId=4e502274;] security - cmd=AdminAuth; account=admin@zmc.com;
  1812. 2018-09-06 13:35:51,697 INFO [qtp788117692-1886:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55924;soapId=4e502274;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  1813. 2018-09-06 13:35:51,750 INFO [qtp788117692-1897:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44242;soapId=4e502275;] security - cmd=CreateAccount; name=account.1536240915960228@zmc.com;
  1814.  
  1815. ==> /opt/zimbra/log/mailbox.log <==
  1816. 2018-09-06 13:35:51,185 INFO [qtp788117692-1740:https:https://zmc-proxy/service/soap/] [name=account.1536240915960227@zmc.com;ip=10.0.0.20;port=33440;soapId=4faf0d4c;] mailbox - beginning listening on Redis notification channel NOTIFICATION-CHANNEL-72
  1817. 2018-09-06 13:35:51,185 INFO [qtp788117692-1740:https:https://zmc-proxy/service/soap/] [name=account.1536240915960227@zmc.com;ip=10.0.0.20;port=33440;soapId=4faf0d4c;] mailbox - added account 8e21eaa4-68a4-499a-ab01-11447866ba0e to Redis notification channel NOTIFICATION-CHANNEL-72
  1818. 2018-09-06 13:35:51,186 INFO [redisson-3-3] [] mailbox - handling notification. accountId=8e21eaa4-68a4-499a-ab01-11447866ba0e, changeId=1, channel=NOTIFICATION-CHANNEL-72
  1819. 2018-09-06 13:35:51,187 INFO [qtp788117692-1740:https:https://zmc-proxy/service/soap/] [name=account.1536240915960227@zmc.com;ip=10.0.0.20;port=33440;soapId=4faf0d4c;] mbxmgr - Mailbox 172 account 8e21eaa4-68a4-499a-ab01-11447866ba0e CREATED
  1820. 2018-09-06 13:35:51,373 INFO [qtp788117692-1740:https:https://zmc-proxy/service/soap/] [name=account.1536240915960227@zmc.com;mid=172;ip=10.0.0.20;port=33440;soapId=4faf0d4c;] smtp - Sending message to MTA at zmc-mta: Message-ID=<178412476.467.1536240951188.JavaMail.zimbra@zmc.com>, replyType=r, sender=account.1536240915960227@zmc.com, nrcpts=1, to=account.1536240915960227@zmc.com
  1821. 2018-09-06 13:35:51,374 INFO [EventLogger-Worker-Thread-3] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@53c7b56
  1822. 2018-09-06 13:35:51,375 INFO [EventLogger-Worker-Thread-3-SendThread(dcc73dce5ecb.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server dcc73dce5ecb.zm-docker_default/10.0.0.17:2181. Will not attempt to authenticate using SASL (unknown error)
  1823. 2018-09-06 13:35:51,375 INFO [EventLogger-Worker-Thread-3-SendThread(dcc73dce5ecb.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to dcc73dce5ecb.zm-docker_default/10.0.0.17:2181, initiating session
  1824. 2018-09-06 13:35:51,377 INFO [EventLogger-Worker-Thread-3-SendThread(dcc73dce5ecb.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server dcc73dce5ecb.zm-docker_default/10.0.0.17:2181, sessionid = 0x10000d48b75002e, negotiated timeout = 10000
  1825. 2018-09-06 13:35:51,377 INFO [zkConnectionManagerCallback-275-thread-1] [] ConnectionManager - zkClient has connected
  1826. 2018-09-06 13:35:51,379 INFO [EventLogger-Worker-Thread-3] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  1827. 2018-09-06 13:35:51,379 INFO [redisson-3-3] [] mailbox - handling notification. accountId=8e21eaa4-68a4-499a-ab01-11447866ba0e, changeId=2, channel=NOTIFICATION-CHANNEL-72
  1828. 2018-09-06 13:35:51,380 INFO [EventLogger-Worker-Thread-3] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  1829. 2018-09-06 13:35:51,392 INFO [redisson-3-5] [] mailbox - handling notification. accountId=8e21eaa4-68a4-499a-ab01-11447866ba0e, changeId=3, channel=NOTIFICATION-CHANNEL-72
  1830. 2018-09-06 13:35:51,398 INFO [redisson-3-6] [] mailbox - handling notification. accountId=8e21eaa4-68a4-499a-ab01-11447866ba0e, changeId=4, channel=NOTIFICATION-CHANNEL-72
  1831. 2018-09-06 13:35:51,402 INFO [qtp788117692-1740:https:https://zmc-proxy/service/soap/] [name=account.1536240915960227@zmc.com;mid=172;ip=10.0.0.20;port=33440;soapId=4faf0d4c;] mailop - adding contact account.1536240915960227@zmc.com: id=258, folderId=13, folderName=Emailed Contacts.
  1832. 2018-09-06 13:35:51,561 INFO [redisson-3-13] [] mailbox - handling notification. accountId=8e21eaa4-68a4-499a-ab01-11447866ba0e, changeId=5, channel=NOTIFICATION-CHANNEL-72
  1833. 2018-09-06 13:35:51,561 INFO [qtp788117692-1740:https:https://zmc-proxy/service/soap/] [name=account.1536240915960227@zmc.com;mid=172;ip=10.0.0.20;port=33440;soapId=4faf0d4c;] soap - SendMsgRequest elapsed=702
  1834. 2018-09-06 13:35:51,683 INFO [qtp788117692-1695:https:https://zmc-proxy/service/soap/] [name=account.1536240915960227@zmc.com;mid=172;ip=10.0.0.19;port=42984;soapId=4faf0d4d;] soap - SendDeliveryReportRequest elapsed=67
  1835. 2018-09-06 13:35:51,683 INFO [redisson-3-20] [] mailbox - handling notification. accountId=8e21eaa4-68a4-499a-ab01-11447866ba0e, changeId=6, channel=NOTIFICATION-CHANNEL-72
  1836. 2018-09-06 13:35:51,883 INFO [redisson-3-19] [] mailbox - handling notification. accountId=5b4b670a-6814-45c0-872a-55ebc0aacaab, changeId=6, channel=NOTIFICATION-CHANNEL-69
  1837.  
  1838. ==> /opt/zimbra/log/mailbox.log <==
  1839. 2018-09-06 13:35:51,882 INFO [notifyMboxListeners-1783] [] mailbox - added account 5b4b670a-6814-45c0-872a-55ebc0aacaab to Redis notification channel NOTIFICATION-CHANNEL-69
  1840. 2018-09-06 13:35:51,883 INFO [redisson-3-28] [] mailbox - handling notification. accountId=5b4b670a-6814-45c0-872a-55ebc0aacaab, changeId=6, channel=NOTIFICATION-CHANNEL-69
  1841. 2018-09-06 13:35:54,771 INFO [qtp788117692-1745:https:https://zmc-proxy/service/soap/] [name=account.1536240915960228@zmc.com;ip=10.0.0.20;port=33470;soapId=4faf0d4e;] soap - AuthRequest elapsed=1
  1842.  
  1843. ==> /opt/zimbra/log/audit.log <==
  1844. 2018-09-06 13:35:54,771 INFO [qtp788117692-1745:https:https://zmc-proxy/service/soap/] [name=account.1536240915960228@zmc.com;ip=10.0.0.20;port=33470;soapId=4faf0d4e;] security - cmd=Auth; account=account.1536240915960228@zmc.com; protocol=soap;
  1845. 2018-09-06 13:35:54,882 INFO [qtp788117692-1874:https:https://zmc-proxy/service/soap/] [name=account.1536240915960228@zmc.com;ip=10.0.0.19;port=43002;soapId=4e502276;] mailbox - Creating mailbox with id 173 and group id 73 for account.1536240915960228@zmc.com.
  1846. 2018-09-06 13:35:54,886 INFO [qtp788117692-1874:https:https://zmc-proxy/service/soap/] [name=account.1536240915960228@zmc.com;ip=10.0.0.19;port=43002;soapId=4e502276;] cache - initializing folder and tag caches for mailbox 173
  1847. 2018-09-06 13:35:55,129 INFO [qtp788117692-1874:https:https://zmc-proxy/service/soap/] [name=account.1536240915960228@zmc.com;ip=10.0.0.19;port=43002;soapId=4e502276;] mailbox - added account 9b437bf0-1aba-4e46-a073-7d653aaad984 to Redis notification channel NOTIFICATION-CHANNEL-73
  1848. 2018-09-06 13:35:55,130 INFO [redisson-3-14] [] mailbox - handling notification. accountId=9b437bf0-1aba-4e46-a073-7d653aaad984, changeId=1, channel=NOTIFICATION-CHANNEL-73
  1849. 2018-09-06 13:35:55,131 INFO [qtp788117692-1874:https:https://zmc-proxy/service/soap/] [name=account.1536240915960228@zmc.com;ip=10.0.0.19;port=43002;soapId=4e502276;] mbxmgr - Mailbox 173 account 9b437bf0-1aba-4e46-a073-7d653aaad984 CREATED
  1850. 2018-09-06 13:35:55,160 INFO [qtp788117692-1874:https:https://zmc-proxy/service/soap/] [name=account.1536240915960228@zmc.com;mid=173;ip=10.0.0.19;port=43002;soapId=4e502276;] soap - GetFolderRequest elapsed=366
  1851. 2018-09-06 13:35:55,441 INFO [redisson-3-5] [] mailbox - handling notification. accountId=9b437bf0-1aba-4e46-a073-7d653aaad984, changeId=2, channel=NOTIFICATION-CHANNEL-73
  1852. 2018-09-06 13:35:55,462 WARN [qtp788117692-1905:https:https://zmc-proxy/service/soap/] [name=account.1536240915960228@zmc.com;mid=173;ip=10.0.0.20;port=33482;soapId=4e502277;] SoapEngine - handler exception
  1853. java.lang.NumberFormatException: null
  1854. at java.lang.Integer.parseInt(Integer.java:542)
  1855. at java.lang.Short.parseShort(Short.java:118)
  1856. at java.lang.Short.valueOf(Short.java:174)
  1857. at java.lang.Short.valueOf(Short.java:200)
  1858. at com.zimbra.cs.store.file.FileBlobStore.getMailboxBlob(FileBlobStore.java:353)
  1859. at com.zimbra.cs.store.StoreManager.getMailboxBlob(StoreManager.java:347)
  1860. at com.zimbra.cs.store.StoreManager.getMailboxBlob(StoreManager.java:372)
  1861. at com.zimbra.cs.mailbox.MailItem.getBlob(MailItem.java:1459)
  1862. at com.zimbra.cs.service.mail.ParseMimeMessage.attachDocument(ParseMimeMessage.java:826)
  1863. at com.zimbra.cs.service.mail.ParseMimeMessage.attachDocument(ParseMimeMessage.java:787)
  1864. at com.zimbra.cs.service.mail.ParseMimeMessage.handleAttachments(ParseMimeMessage.java:509)
  1865. at com.zimbra.cs.service.mail.ParseMimeMessage.parseMimeMsgSoap(ParseMimeMessage.java:384)
  1866. at com.zimbra.cs.service.mail.ParseMimeMessage.parseMimeMsgSoap(ParseMimeMessage.java:213)
  1867. at com.zimbra.cs.service.mail.SaveDraft.handle(SaveDraft.java:114)
  1868. at com.zimbra.soap.SoapEngine.dispatchRequest(SoapEngine.java:656)
  1869. at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:501)
  1870. at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:288)
  1871. at com.zimbra.soap.SoapServlet.doWork(SoapServlet.java:303)
  1872. at com.zimbra.soap.SoapServlet.doPost(SoapServlet.java:213)
  1873. at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  1874. at com.zimbra.cs.servlet.ZimbraServlet.service(ZimbraServlet.java:211)
  1875. at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
  1876. at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:821)
  1877. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1685)
  1878. at com.zimbra.cs.servlet.CsrfFilter.doFilter(CsrfFilter.java:169)
  1879. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  1880. at com.zimbra.cs.servlet.RequestStringFilter.doFilter(RequestStringFilter.java:54)
  1881. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  1882. at com.zimbra.cs.servlet.SetHeaderFilter.doFilter(SetHeaderFilter.java:59)
  1883. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  1884. at com.zimbra.cs.servlet.ETagHeaderFilter.doFilter(ETagHeaderFilter.java:47)
  1885. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  1886. at com.zimbra.cs.servlet.ContextPathBasedThreadPoolBalancerFilter.doFilter(ContextPathBasedThreadPoolBalancerFilter.java:107)
  1887. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  1888. at com.zimbra.cs.servlet.ZimbraQoSFilter.doFilter(ZimbraQoSFilter.java:125)
  1889. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  1890. at com.zimbra.cs.servlet.ZimbraInvalidLoginFilter.doFilter(ZimbraInvalidLoginFilter.java:117)
  1891. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  1892. at org.eclipse.jetty.servlets.DoSFilter.doFilterChain(DoSFilter.java:473)
  1893. at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:318)
  1894. at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:288)
  1895. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  1896. at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
  1897. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
  1898. at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
  1899. at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
  1900. at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1158)
  1901. at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
  1902. at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
  1903. at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1090)
  1904. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
  1905. at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
  1906. at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:109)
  1907. at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119)
  1908. at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:318)
  1909. at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:437)
  1910. at org.eclipse.jetty.server.handler.DebugHandler.handle(DebugHandler.java:84)
  1911. at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119)
  1912. at org.eclipse.jetty.server.Server.handle(Server.java:517)
  1913. at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:306)
  1914. at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:242)
  1915. at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261)
  1916. at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
  1917. at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:192)
  1918. at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261)
  1919. at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
  1920. at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:75)
  1921. at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:213)
  1922. at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:147)
  1923. at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
  1924. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1925. at java.lang.Thread.run(Thread.java:748)
  1926. 2018-09-06 13:35:55,462 INFO [qtp788117692-1905:https:https://zmc-proxy/service/soap/] [name=account.1536240915960228@zmc.com;mid=173;ip=10.0.0.20;port=33482;soapId=4e502277;] soap - SaveDraftRequest elapsed=5
  1927. 2018-09-06 13:35:55,480 WARN [qtp788117692-1891:https:https://zmc-proxy/service/soap/] [name=account.1536240915960228@zmc.com;mid=173;ip=10.0.0.19;port=43014;soapId=4e502278;] SoapEngine - handler exception
  1928. com.zimbra.common.service.ServiceException: invalid request: malformed item ID: UNSET
  1929. ExceptionId:qtp788117692-1891:https:https://zmc-proxy/service/soap/:1536240955480:a4a5f9c6e5824c5f
  1930. Code:service.INVALID_REQUEST
  1931. at com.zimbra.common.service.ServiceException.INVALID_REQUEST(ServiceException.java:295)
  1932. at com.zimbra.common.mailbox.ItemIdentifier.<init>(ItemIdentifier.java:83)
  1933. at com.zimbra.cs.service.util.ItemId.<init>(ItemId.java:88)
  1934. at com.zimbra.cs.service.util.ItemId.<init>(ItemId.java:80)
  1935. at com.zimbra.cs.service.mail.ParseMimeMessage.handleAttachments(ParseMimeMessage.java:493)
  1936. at com.zimbra.cs.service.mail.ParseMimeMessage.parseMimeMsgSoap(ParseMimeMessage.java:384)
  1937. at com.zimbra.cs.service.mail.ParseMimeMessage.parseMimeMsgSoap(ParseMimeMessage.java:206)
  1938. at com.zimbra.cs.service.mail.SendMsg.handle(SendMsg.java:179)
  1939. at com.zimbra.cs.service.mail.SendMsg.handle(SendMsg.java:108)
  1940. at com.zimbra.soap.SoapEngine.dispatchRequest(SoapEngine.java:656)
  1941. at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:501)
  1942. at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:288)
  1943. at com.zimbra.soap.SoapServlet.doWork(SoapServlet.java:303)
  1944. at com.zimbra.soap.SoapServlet.doPost(SoapServlet.java:213)
  1945. at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  1946. at com.zimbra.cs.servlet.ZimbraServlet.service(ZimbraServlet.java:211)
  1947. at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
  1948. at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:821)
  1949. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1685)
  1950. at com.zimbra.cs.servlet.CsrfFilter.doFilter(CsrfFilter.java:169)
  1951. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  1952. at com.zimbra.cs.servlet.RequestStringFilter.doFilter(RequestStringFilter.java:54)
  1953. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  1954. at com.zimbra.cs.servlet.SetHeaderFilter.doFilter(SetHeaderFilter.java:59)
  1955. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  1956. at com.zimbra.cs.servlet.ETagHeaderFilter.doFilter(ETagHeaderFilter.java:47)
  1957. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  1958. at com.zimbra.cs.servlet.ContextPathBasedThreadPoolBalancerFilter.doFilter(ContextPathBasedThreadPoolBalancerFilter.java:107)
  1959. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  1960. at com.zimbra.cs.servlet.ZimbraQoSFilter.doFilter(ZimbraQoSFilter.java:125)
  1961. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  1962. at com.zimbra.cs.servlet.ZimbraInvalidLoginFilter.doFilter(ZimbraInvalidLoginFilter.java:117)
  1963. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  1964. at org.eclipse.jetty.servlets.DoSFilter.doFilterChain(DoSFilter.java:473)
  1965. at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:318)
  1966. at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:288)
  1967. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  1968. at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
  1969. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
  1970. at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
  1971. at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
  1972. at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1158)
  1973. at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
  1974. at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
  1975. at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1090)
  1976. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
  1977. at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
  1978. at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:109)
  1979. at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119)
  1980. at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:318)
  1981. at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:437)
  1982. at org.eclipse.jetty.server.handler.DebugHandler.handle(DebugHandler.java:84)
  1983. at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119)
  1984. at org.eclipse.jetty.server.Server.handle(Server.java:517)
  1985. at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:306)
  1986. at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:242)
  1987. at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261)
  1988. at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
  1989. at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:192)
  1990. at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261)
  1991. at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
  1992. at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:75)
  1993. at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:213)
  1994. at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:147)
  1995. at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
  1996. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  1997. at java.lang.Thread.run(Thread.java:748)
  1998. Caused by: java.lang.NumberFormatException: For input string: "UNSET"
  1999. at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
  2000. at java.lang.Integer.parseInt(Integer.java:580)
  2001. at java.lang.Integer.parseInt(Integer.java:615)
  2002. at com.zimbra.common.mailbox.ItemIdentifier.<init>(ItemIdentifier.java:81)
  2003. ... 65 more
  2004. 2018-09-06 13:35:55,480 INFO [qtp788117692-1891:https:https://zmc-proxy/service/soap/] [name=account.1536240915960228@zmc.com;mid=173;ip=10.0.0.19;port=43014;soapId=4e502278;] soap - SendMsgRequest elapsed=3
  2005. 2018-09-06 13:35:55,532 INFO [qtp788117692-1884:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55956;soapId=4e502279;] soap - CreateAccountRequest elapsed=4
  2006.  
  2007. ==> /opt/zimbra/log/audit.log <==
  2008. 2018-09-06 13:35:55,531 INFO [qtp788117692-1884:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55956;soapId=4e502279;] security - cmd=CreateAccount; name=test1536240955501.233@zmc.com;
  2009.  
  2010. ==> /opt/zimbra/log/mailbox.log <==
  2011. 2018-09-06 13:35:55,188 INFO [qtp788117692-1717:https:https://zmc-proxy/service/upload?fmt=raw] [name=account.1536240915960228@zmc.com;ip=10.0.0.20;port=33478;ua=Jakarta Commons-HttpClient/3.1;] mbxmgr - Mailbox 173 account 9b437bf0-1aba-4e46-a073-7d653aaad984 LOADED
  2012. 2018-09-06 13:35:55,189 INFO [qtp788117692-1717:https:https://zmc-proxy/service/upload?fmt=raw] [name=account.1536240915960228@zmc.com;mid=173;ip=10.0.0.20;port=33478;ua=Jakarta Commons-HttpClient/3.1;] FileUploadServlet - Received multipart: Upload: { accountId=9b437bf0-1aba-4e46-a073-7d653aaad984, time=Thu Sep 06 13:35:55 UTC 2018, size=2576, uploadId=d5f0f26e-04d6-463a-8d2e-352647ebeaab:24deaf6f-3116-49b5-8a46-0cfd46f7d1d4, name=contact1.txt, path=/opt/zimbra/data/tmp/upload/upload_9b437bf0-1aba-4e46-a073-7d653aaad984_d5f0f26e-04d6-463a-8d2e-352647ebeaab:24deaf6f-3116-49b5-8a46-0cfd46f7d1d4.tmp }
  2013. 2018-09-06 13:35:55,208 INFO [qtp788117692-1778:https:https://zmc-proxy/service/soap/] [name=account.1536240915960228@zmc.com;mid=173;ip=10.0.0.19;port=43008;soapId=4faf0d4f;] cache - initializing folder and tag caches for mailbox 173
  2014. 2018-09-06 13:35:55,277 INFO [qtp788117692-1778:https:https://zmc-proxy/service/soap/] [name=account.1536240915960228@zmc.com;mid=173;ip=10.0.0.19;port=43008;soapId=4faf0d4f;] mailbox - uploadId=d5f0f26e-04d6-463a-8d2e-352647ebeaab:24deaf6f-3116-49b5-8a46-0cfd46f7d1d4
  2015. 2018-09-06 13:35:55,285 INFO [qtp788117692-1778:https:https://zmc-proxy/service/soap/] [name=account.1536240915960228@zmc.com;mid=173;ip=10.0.0.19;port=43008;soapId=4faf0d4f;] mailop - Adding Document contact1.txt: id=257, folderId=16, folderName=Briefcase
  2016. 2018-09-06 13:35:55,440 INFO [qtp788117692-1778:https:https://zmc-proxy/service/soap/] [name=account.1536240915960228@zmc.com;mid=173;ip=10.0.0.19;port=43008;soapId=4faf0d4f;] mailbox - added account 9b437bf0-1aba-4e46-a073-7d653aaad984 to Redis notification channel NOTIFICATION-CHANNEL-73
  2017. 2018-09-06 13:35:55,441 INFO [redisson-3-11] [] mailbox - handling notification. accountId=9b437bf0-1aba-4e46-a073-7d653aaad984, changeId=2, channel=NOTIFICATION-CHANNEL-73
  2018. 2018-09-06 13:35:55,441 INFO [qtp788117692-1778:https:https://zmc-proxy/service/soap/] [name=account.1536240915960228@zmc.com;mid=173;ip=10.0.0.19;port=43008;soapId=4faf0d4f;] soap - SaveDocumentRequest elapsed=235
  2019. 2018-09-06 13:35:55,497 WARN [qtp788117692-1743:https:https://zmc-proxy/service/soap/] [name=account.1536240915960228@zmc.com;mid=173;ip=10.0.0.20;port=33490;soapId=4faf0d50;] SoapEngine - handler exception
  2020. com.zimbra.common.service.ServiceException: invalid request: malformed item ID: UNSET
  2021. ExceptionId:qtp788117692-1743:https:https://zmc-proxy/service/soap/:1536240955497:33d5d0aceb48fe53
  2022. Code:service.INVALID_REQUEST
  2023. at com.zimbra.common.service.ServiceException.INVALID_REQUEST(ServiceException.java:295)
  2024. at com.zimbra.common.mailbox.ItemIdentifier.<init>(ItemIdentifier.java:83)
  2025. at com.zimbra.cs.service.util.ItemId.<init>(ItemId.java:88)
  2026. at com.zimbra.cs.service.util.ItemId.<init>(ItemId.java:80)
  2027. at com.zimbra.cs.service.mail.MailDocumentHandler.proxyIfNecessary(MailDocumentHandler.java:58)
  2028. at com.zimbra.soap.SoapEngine.dispatchRequest(SoapEngine.java:647)
  2029. at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:501)
  2030. at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:288)
  2031. at com.zimbra.soap.SoapServlet.doWork(SoapServlet.java:303)
  2032. at com.zimbra.soap.SoapServlet.doPost(SoapServlet.java:213)
  2033. at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  2034. at com.zimbra.cs.servlet.ZimbraServlet.service(ZimbraServlet.java:211)
  2035. at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
  2036. at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:821)
  2037. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1685)
  2038. at com.zimbra.cs.servlet.CsrfFilter.doFilter(CsrfFilter.java:169)
  2039. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2040. at com.zimbra.cs.servlet.RequestStringFilter.doFilter(RequestStringFilter.java:54)
  2041. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2042. at com.zimbra.cs.servlet.SetHeaderFilter.doFilter(SetHeaderFilter.java:59)
  2043. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2044. at com.zimbra.cs.servlet.ETagHeaderFilter.doFilter(ETagHeaderFilter.java:47)
  2045. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2046. at com.zimbra.cs.servlet.ContextPathBasedThreadPoolBalancerFilter.doFilter(ContextPathBasedThreadPoolBalancerFilter.java:107)
  2047. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2048. at com.zimbra.cs.servlet.ZimbraQoSFilter.doFilter(ZimbraQoSFilter.java:125)
  2049. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2050. at com.zimbra.cs.servlet.ZimbraInvalidLoginFilter.doFilter(ZimbraInvalidLoginFilter.java:117)
  2051. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2052. at org.eclipse.jetty.servlets.DoSFilter.doFilterChain(DoSFilter.java:473)
  2053. at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:318)
  2054. at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:288)
  2055. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2056. at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
  2057. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
  2058. at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
  2059. at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
  2060. at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1158)
  2061. at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
  2062. at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
  2063. at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1090)
  2064. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
  2065. at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
  2066. at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:109)
  2067. at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119)
  2068. at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:318)
  2069. at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:437)
  2070. at org.eclipse.jetty.server.handler.DebugHandler.handle(DebugHandler.java:84)
  2071. at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119)
  2072. at org.eclipse.jetty.server.Server.handle(Server.java:517)
  2073. at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:306)
  2074. at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:242)
  2075. at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261)
  2076. at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
  2077. at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:192)
  2078. at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261)
  2079. at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
  2080. at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:75)
  2081. at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:213)
  2082. at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:147)
  2083. at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
  2084. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  2085. at java.lang.Thread.run(Thread.java:748)
  2086. Caused by: java.lang.NumberFormatException: For input string: "UNSET"
  2087. at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
  2088. at java.lang.Integer.parseInt(Integer.java:580)
  2089. at java.lang.Integer.parseInt(Integer.java:615)
  2090. at com.zimbra.common.mailbox.ItemIdentifier.<init>(ItemIdentifier.java:81)
  2091. ... 61 more
  2092. 2018-09-06 13:35:55,497 INFO [qtp788117692-1743:https:https://zmc-proxy/service/soap/] [name=account.1536240915960228@zmc.com;mid=173;ip=10.0.0.20;port=33490;soapId=4faf0d50;] soap - RemoveAttachmentsRequest elapsed=1
  2093. 2018-09-06 13:35:55,514 INFO [qtp788117692-1561:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44270;soapId=4faf0d51;] soap - AuthRequest elapsed=1
  2094. 2018-09-06 13:35:55,656 INFO [LmtpServer-5] [ip=10.0.0.35;] mailbox - Creating mailbox with id 174 and group id 74 for account.1536240915960225@zmc.com.
  2095. 2018-09-06 13:35:55,659 INFO [LmtpServer-5] [ip=10.0.0.35;] cache - initializing folder and tag caches for mailbox 174
  2096. 2018-09-06 13:35:55,885 INFO [LmtpServer-5] [ip=10.0.0.35;] mailbox - added account 5dd21c14-5ab8-47d4-a2cf-d571ca2264db to Redis notification channel NOTIFICATION-CHANNEL-74
  2097. 2018-09-06 13:35:55,886 INFO [redisson-3-10] [] mailbox - handling notification. accountId=5dd21c14-5ab8-47d4-a2cf-d571ca2264db, changeId=1, channel=NOTIFICATION-CHANNEL-74
  2098. 2018-09-06 13:35:55,887 INFO [LmtpServer-5] [ip=10.0.0.35;] mbxmgr - Mailbox 174 account 5dd21c14-5ab8-47d4-a2cf-d571ca2264db CREATED
  2099. 2018-09-06 13:35:55,887 INFO [LmtpServer-5] [ip=10.0.0.35;] lmtp - Delivering message: size=1942 bytes, nrcpts=1, sender=account.1536240915960224@zmc.com, msgid=<1701883999.466.1536240947027.JavaMail.zimbra@zmc.com>
  2100.  
  2101. ==> /opt/zimbra/log/audit.log <==
  2102. 2018-09-06 13:35:55,513 INFO [qtp788117692-1561:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44270;soapId=4faf0d51;] security - cmd=AdminAuth; account=admin@zmc.com;
  2103. 2018-09-06 13:35:55,513 INFO [qtp788117692-1561:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44270;soapId=4faf0d51;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2104.  
  2105. ==> /opt/zimbra/log/mailbox.log <==
  2106. 2018-09-06 13:35:56,049 INFO [EventLogger-Worker-Thread-4] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@308b549a
  2107. 2018-09-06 13:35:56,051 INFO [EventLogger-Worker-Thread-4-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181. Will not attempt to authenticate using SASL (unknown error)
  2108. 2018-09-06 13:35:56,051 INFO [EventLogger-Worker-Thread-4-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181, initiating session
  2109. 2018-09-06 13:35:56,053 INFO [EventLogger-Worker-Thread-4-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181, sessionid = 0x30000d48aad0023, negotiated timeout = 10000
  2110. 2018-09-06 13:35:56,053 INFO [zkConnectionManagerCallback-278-thread-1] [] ConnectionManager - zkClient has connected
  2111. 2018-09-06 13:35:56,054 INFO [EventLogger-Worker-Thread-4] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  2112. 2018-09-06 13:35:56,055 INFO [EventLogger-Worker-Thread-4] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  2113. 2018-09-06 13:35:56,067 INFO [redisson-3-10] [] mailbox - handling notification. accountId=5dd21c14-5ab8-47d4-a2cf-d571ca2264db, changeId=2, channel=NOTIFICATION-CHANNEL-74
  2114.  
  2115. ==> /opt/zimbra/log/mailbox.log <==
  2116. 2018-09-06 13:35:58,551 INFO [qtp788117692-1903:https:https://zmc-proxy/service/soap/] [name=test1536240955501.233@zmc.com;ip=10.0.0.19;port=43030;soapId=4e50227a;] soap - AuthRequest elapsed=1
  2117. 2018-09-06 13:35:58,664 INFO [qtp788117692-1945:https:https://zmc-proxy/service/soap/] [name=test1536240955501.233@zmc.com;ip=10.0.0.20;port=33506;soapId=4e50227b;] mailbox - Creating mailbox with id 175 and group id 75 for test1536240955501.233@zmc.com.
  2118. 2018-09-06 13:35:58,667 INFO [qtp788117692-1945:https:https://zmc-proxy/service/soap/] [name=test1536240955501.233@zmc.com;ip=10.0.0.20;port=33506;soapId=4e50227b;] cache - initializing folder and tag caches for mailbox 175
  2119.  
  2120. ==> /opt/zimbra/log/audit.log <==
  2121. 2018-09-06 13:35:58,551 INFO [qtp788117692-1903:https:https://zmc-proxy/service/soap/] [name=test1536240955501.233@zmc.com;ip=10.0.0.19;port=43030;soapId=4e50227a;] security - cmd=Auth; account=test1536240955501.233@zmc.com; protocol=soap;
  2122.  
  2123. ==> /opt/zimbra/log/mailbox.log <==
  2124. 2018-09-06 13:35:58,885 INFO [qtp788117692-1945:https:https://zmc-proxy/service/soap/] [name=test1536240955501.233@zmc.com;ip=10.0.0.20;port=33506;soapId=4e50227b;] mailbox - added account c7f661e7-2676-41aa-9674-0357785543cf to Redis notification channel NOTIFICATION-CHANNEL-75
  2125. 2018-09-06 13:35:58,886 INFO [redisson-3-2] [] mailbox - handling notification. accountId=c7f661e7-2676-41aa-9674-0357785543cf, changeId=1, channel=NOTIFICATION-CHANNEL-75
  2126. 2018-09-06 13:35:58,886 INFO [qtp788117692-1945:https:https://zmc-proxy/service/soap/] [name=test1536240955501.233@zmc.com;ip=10.0.0.20;port=33506;soapId=4e50227b;] mbxmgr - Mailbox 175 account c7f661e7-2676-41aa-9674-0357785543cf CREATED
  2127. 2018-09-06 13:35:59,046 INFO [qtp788117692-1945:https:https://zmc-proxy/service/soap/] [name=test1536240955501.233@zmc.com;mid=175;ip=10.0.0.20;port=33506;soapId=4e50227b;] soap - AddMsgRequest elapsed=473
  2128. 2018-09-06 13:35:59,058 INFO [redisson-3-2] [] mailbox - handling notification. accountId=c7f661e7-2676-41aa-9674-0357785543cf, changeId=2, channel=NOTIFICATION-CHANNEL-75
  2129. 2018-09-06 13:35:59,157 INFO [qtp788117692-1874:https:https://zmc-proxy/service/soap/] [name=test1536240955501.233@zmc.com;mid=175;ip=10.0.0.19;port=43038;soapId=4e50227c;] soap - SetCustomMetadataRequest elapsed=54
  2130. 2018-09-06 13:35:59,157 INFO [redisson-3-31] [] mailbox - handling notification. accountId=c7f661e7-2676-41aa-9674-0357785543cf, changeId=3, channel=NOTIFICATION-CHANNEL-75
  2131. 2018-09-06 13:35:59,735 INFO [LmtpServer-5] [ip=10.0.0.35;] mbxmgr - Mailbox 172 account 8e21eaa4-68a4-499a-ab01-11447866ba0e LOADED
  2132. 2018-09-06 13:35:59,736 INFO [LmtpServer-5] [ip=10.0.0.35;] lmtp - Delivering message: size=1941 bytes, nrcpts=1, sender=account.1536240915960227@zmc.com, msgid=<178412476.467.1536240951188.JavaMail.zimbra@zmc.com>
  2133. 2018-09-06 13:35:59,737 INFO [LmtpServer-5] [name=account.1536240915960227@zmc.com;mid=172;ip=10.0.0.35;] cache - initializing folder and tag caches for mailbox 172
  2134. 2018-09-06 13:35:59,172 INFO [qtp788117692-1561:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55974;soapId=4faf0d52;] soap - AuthRequest elapsed=1
  2135. 2018-09-06 13:35:59,190 INFO [qtp788117692-1561:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44292;soapId=4faf0d53;] soap - CreateAccountRequest elapsed=4
  2136. 2018-09-06 13:35:59,959 INFO [redisson-3-31] [] mailbox - handling notification. accountId=8e21eaa4-68a4-499a-ab01-11447866ba0e, changeId=7, channel=NOTIFICATION-CHANNEL-72
  2137.  
  2138. ==> /opt/zimbra/log/audit.log <==
  2139. 2018-09-06 13:35:59,171 INFO [qtp788117692-1561:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55974;soapId=4faf0d52;] security - cmd=AdminAuth; account=admin@zmc.com;
  2140. 2018-09-06 13:35:59,171 INFO [qtp788117692-1561:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55974;soapId=4faf0d52;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2141. 2018-09-06 13:35:59,188 INFO [qtp788117692-1561:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44292;soapId=4faf0d53;] security - cmd=CreateAccount; name=test1536240955501.234@zmc.com;
  2142. 2018-09-06 13:35:59,958 INFO [notifyMboxListeners-1783] [] mailbox - added account 8e21eaa4-68a4-499a-ab01-11447866ba0e to Redis notification channel NOTIFICATION-CHANNEL-72
  2143. 2018-09-06 13:35:59,959 INFO [redisson-3-23] [] mailbox - handling notification. accountId=8e21eaa4-68a4-499a-ab01-11447866ba0e, changeId=7, channel=NOTIFICATION-CHANNEL-72
  2144. 2018-09-06 13:36:02,208 INFO [qtp788117692-1869:https:https://zmc-proxy/service/soap/] [name=test1536240955501.234@zmc.com;ip=10.0.0.20;port=33524;soapId=4e50227d;] soap - AuthRequest elapsed=1
  2145. 2018-09-06 13:36:02,713 INFO [qtp788117692-1916:https:https://zmc-proxy/service/soap/] [name=test1536240955501.234@zmc.com;ip=10.0.0.20;port=33532;] mbxmgr - Mailbox 176 account 8e417c1a-7f07-44eb-bf60-7f3e2fea527b LOADED
  2146. 2018-09-06 13:36:02,715 INFO [qtp788117692-1916:https:https://zmc-proxy/service/soap/] [name=test1536240955501.234@zmc.com;mid=176;ip=10.0.0.20;port=33532;soapId=4e50227e;] cache - initializing folder and tag caches for mailbox 176
  2147.  
  2148. ==> /opt/zimbra/log/audit.log <==
  2149. 2018-09-06 13:36:02,207 INFO [qtp788117692-1869:https:https://zmc-proxy/service/soap/] [name=test1536240955501.234@zmc.com;ip=10.0.0.20;port=33524;soapId=4e50227d;] security - cmd=Auth; account=test1536240955501.234@zmc.com; protocol=soap;
  2150.  
  2151. ==> /opt/zimbra/log/mailbox.log <==
  2152. 2018-09-06 13:36:02,313 INFO [qtp788117692-1561:https:https://zmc-proxy/service/soap/] [name=test1536240955501.234@zmc.com;ip=10.0.0.19;port=43056;soapId=4faf0d54;] mailbox - Creating mailbox with id 176 and group id 76 for test1536240955501.234@zmc.com.
  2153. 2018-09-06 13:36:02,316 INFO [qtp788117692-1561:https:https://zmc-proxy/service/soap/] [name=test1536240955501.234@zmc.com;ip=10.0.0.19;port=43056;soapId=4faf0d54;] cache - initializing folder and tag caches for mailbox 176
  2154. 2018-09-06 13:36:02,530 INFO [qtp788117692-1561:https:https://zmc-proxy/service/soap/] [name=test1536240955501.234@zmc.com;ip=10.0.0.19;port=43056;soapId=4faf0d54;] mailbox - added account 8e417c1a-7f07-44eb-bf60-7f3e2fea527b to Redis notification channel NOTIFICATION-CHANNEL-76
  2155. 2018-09-06 13:36:02,531 INFO [redisson-3-5] [] mailbox - handling notification. accountId=8e417c1a-7f07-44eb-bf60-7f3e2fea527b, changeId=1, channel=NOTIFICATION-CHANNEL-76
  2156. 2018-09-06 13:36:02,531 INFO [qtp788117692-1561:https:https://zmc-proxy/service/soap/] [name=test1536240955501.234@zmc.com;ip=10.0.0.19;port=43056;soapId=4faf0d54;] mbxmgr - Mailbox 176 account 8e417c1a-7f07-44eb-bf60-7f3e2fea527b CREATED
  2157. 2018-09-06 13:36:02,686 INFO [qtp788117692-1561:https:https://zmc-proxy/service/soap/] [name=test1536240955501.234@zmc.com;mid=176;ip=10.0.0.19;port=43056;soapId=4faf0d54;] soap - AddMsgRequest elapsed=457
  2158. 2018-09-06 13:36:02,698 INFO [redisson-3-5] [] mailbox - handling notification. accountId=8e417c1a-7f07-44eb-bf60-7f3e2fea527b, changeId=2, channel=NOTIFICATION-CHANNEL-76
  2159. 2018-09-06 13:36:02,826 INFO [redisson-3-20] [] mailbox - handling notification. accountId=8e417c1a-7f07-44eb-bf60-7f3e2fea527b, changeId=3, channel=NOTIFICATION-CHANNEL-76
  2160. 2018-09-06 13:36:02,866 INFO [qtp788117692-1742:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55998;soapId=4faf0d55;] soap - AuthRequest elapsed=1
  2161. 2018-09-06 13:36:02,919 INFO [qtp788117692-1700:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44316;soapId=4faf0d56;] soap - CreateAccountRequest elapsed=4
  2162.  
  2163. ==> /opt/zimbra/log/audit.log <==
  2164. 2018-09-06 13:36:02,866 INFO [qtp788117692-1742:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55998;soapId=4faf0d55;] security - cmd=AdminAuth; account=admin@zmc.com;
  2165. 2018-09-06 13:36:02,866 INFO [qtp788117692-1742:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=55998;soapId=4faf0d55;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2166. 2018-09-06 13:36:02,918 INFO [qtp788117692-1700:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44316;soapId=4faf0d56;] security - cmd=CreateAccount; name=test1536240955501.235@zmc.com;
  2167.  
  2168. ==> /opt/zimbra/log/mailbox.log <==
  2169. 2018-09-06 13:36:02,826 INFO [qtp788117692-1916:https:https://zmc-proxy/service/soap/] [name=test1536240955501.234@zmc.com;mid=176;ip=10.0.0.20;port=33532;soapId=4e50227e;] mailbox - beginning listening on Redis notification channel NOTIFICATION-CHANNEL-76
  2170. 2018-09-06 13:36:02,826 INFO [qtp788117692-1916:https:https://zmc-proxy/service/soap/] [name=test1536240955501.234@zmc.com;mid=176;ip=10.0.0.20;port=33532;soapId=4e50227e;] mailbox - added account 8e417c1a-7f07-44eb-bf60-7f3e2fea527b to Redis notification channel NOTIFICATION-CHANNEL-76
  2171. 2018-09-06 13:36:02,826 INFO [qtp788117692-1916:https:https://zmc-proxy/service/soap/] [name=test1536240955501.234@zmc.com;mid=176;ip=10.0.0.20;port=33532;soapId=4e50227e;] soap - SetCustomMetadataRequest elapsed=113
  2172. 2018-09-06 13:36:02,826 INFO [redisson-3-7] [] mailbox - handling notification. accountId=8e417c1a-7f07-44eb-bf60-7f3e2fea527b, changeId=3, channel=NOTIFICATION-CHANNEL-76
  2173. 2018-09-06 13:36:02,852 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=test1536240955501.234@zmc.com;mid=176;ip=10.0.0.19;port=43062;soapId=4e50227f;] soap - GetCustomMetadataRequest elapsed=2
  2174. 2018-09-06 13:36:05,937 INFO [qtp788117692-1903:https:https://zmc-proxy/service/soap/] [name=test1536240955501.235@zmc.com;ip=10.0.0.20;port=33544;soapId=4e502280;] soap - AuthRequest elapsed=1
  2175. 2018-09-06 13:36:06,310 INFO [qtp788117692-1906:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44330;soapId=4e502281;] soap - CreateAccountRequest elapsed=4
  2176.  
  2177. ==> /opt/zimbra/log/audit.log <==
  2178. 2018-09-06 13:36:05,937 INFO [qtp788117692-1903:https:https://zmc-proxy/service/soap/] [name=test1536240955501.235@zmc.com;ip=10.0.0.20;port=33544;soapId=4e502280;] security - cmd=Auth; account=test1536240955501.235@zmc.com; protocol=soap;
  2179. 2018-09-06 13:36:06,309 INFO [qtp788117692-1906:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44330;soapId=4e502281;] security - cmd=CreateAccount; name=test1536240955501.236@zmc.com;
  2180.  
  2181. ==> /opt/zimbra/log/mailbox.log <==
  2182. 2018-09-06 13:36:06,047 INFO [qtp788117692-1743:https:https://zmc-proxy/service/soap/] [name=test1536240955501.235@zmc.com;ip=10.0.0.19;port=43076;soapId=4faf0d57;] mailbox - Creating mailbox with id 177 and group id 77 for test1536240955501.235@zmc.com.
  2183. 2018-09-06 13:36:06,050 INFO [qtp788117692-1743:https:https://zmc-proxy/service/soap/] [name=test1536240955501.235@zmc.com;ip=10.0.0.19;port=43076;soapId=4faf0d57;] cache - initializing folder and tag caches for mailbox 177
  2184. 2018-09-06 13:36:06,267 INFO [qtp788117692-1743:https:https://zmc-proxy/service/soap/] [name=test1536240955501.235@zmc.com;ip=10.0.0.19;port=43076;soapId=4faf0d57;] mailbox - beginning listening on Redis notification channel NOTIFICATION-CHANNEL-77
  2185. 2018-09-06 13:36:06,267 INFO [qtp788117692-1743:https:https://zmc-proxy/service/soap/] [name=test1536240955501.235@zmc.com;ip=10.0.0.19;port=43076;soapId=4faf0d57;] mailbox - added account ec440f3a-8f16-4013-bcf0-cfac93ab06bc to Redis notification channel NOTIFICATION-CHANNEL-77
  2186. 2018-09-06 13:36:06,268 INFO [redisson-3-29] [] mailbox - handling notification. accountId=ec440f3a-8f16-4013-bcf0-cfac93ab06bc, changeId=1, channel=NOTIFICATION-CHANNEL-77
  2187. 2018-09-06 13:36:06,269 INFO [qtp788117692-1743:https:https://zmc-proxy/service/soap/] [name=test1536240955501.235@zmc.com;ip=10.0.0.19;port=43076;soapId=4faf0d57;] mbxmgr - Mailbox 177 account ec440f3a-8f16-4013-bcf0-cfac93ab06bc CREATED
  2188. 2018-09-06 13:36:06,275 INFO [qtp788117692-1743:https:https://zmc-proxy/service/soap/] [name=test1536240955501.235@zmc.com;mid=177;ip=10.0.0.19;port=43076;soapId=4faf0d57;] soap - SetMailboxMetadataRequest elapsed=315
  2189. 2018-09-06 13:36:06,291 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56012;soapId=4faf0d58;] soap - AuthRequest elapsed=1
  2190. 2018-09-06 13:36:06,296 INFO [redisson-3-29] [] mailbox - handling notification. accountId=ec440f3a-8f16-4013-bcf0-cfac93ab06bc, changeId=2, channel=NOTIFICATION-CHANNEL-77
  2191.  
  2192. ==> /opt/zimbra/log/audit.log <==
  2193. 2018-09-06 13:36:06,290 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56012;soapId=4faf0d58;] security - cmd=AdminAuth; account=admin@zmc.com;
  2194. 2018-09-06 13:36:06,291 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56012;soapId=4faf0d58;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2195.  
  2196. ==> /opt/zimbra/log/mailbox.log <==
  2197. 2018-09-06 13:36:09,327 INFO [qtp788117692-1915:https:https://zmc-proxy/service/soap/] [name=test1536240955501.236@zmc.com;ip=10.0.0.20;port=33556;soapId=4e502282;] soap - AuthRequest elapsed=1
  2198. 2018-09-06 13:36:09,434 INFO [qtp788117692-1874:https:https://zmc-proxy/service/soap/] [name=test1536240955501.236@zmc.com;ip=10.0.0.19;port=43088;soapId=4e502283;] mailbox - Creating mailbox with id 178 and group id 78 for test1536240955501.236@zmc.com.
  2199. 2018-09-06 13:36:09,437 INFO [qtp788117692-1874:https:https://zmc-proxy/service/soap/] [name=test1536240955501.236@zmc.com;ip=10.0.0.19;port=43088;soapId=4e502283;] cache - initializing folder and tag caches for mailbox 178
  2200. 2018-09-06 13:36:09,651 INFO [qtp788117692-1874:https:https://zmc-proxy/service/soap/] [name=test1536240955501.236@zmc.com;ip=10.0.0.19;port=43088;soapId=4e502283;] mailbox - beginning listening on Redis notification channel NOTIFICATION-CHANNEL-78
  2201. 2018-09-06 13:36:09,652 INFO [qtp788117692-1874:https:https://zmc-proxy/service/soap/] [name=test1536240955501.236@zmc.com;ip=10.0.0.19;port=43088;soapId=4e502283;] mailbox - added account 947efc84-ffb9-4f97-860a-9405514fcb95 to Redis notification channel NOTIFICATION-CHANNEL-78
  2202. 2018-09-06 13:36:09,652 INFO [redisson-3-25] [] mailbox - handling notification. accountId=947efc84-ffb9-4f97-860a-9405514fcb95, changeId=1, channel=NOTIFICATION-CHANNEL-78
  2203. 2018-09-06 13:36:09,653 INFO [qtp788117692-1874:https:https://zmc-proxy/service/soap/] [name=test1536240955501.236@zmc.com;ip=10.0.0.19;port=43088;soapId=4e502283;] mbxmgr - Mailbox 178 account 947efc84-ffb9-4f97-860a-9405514fcb95 CREATED
  2204. 2018-09-06 13:36:09,663 INFO [qtp788117692-1874:https:https://zmc-proxy/service/soap/] [name=test1536240955501.236@zmc.com;mid=178;ip=10.0.0.19;port=43088;soapId=4e502283;] soap - SetMailboxMetadataRequest elapsed=314
  2205. 2018-09-06 13:36:09,682 INFO [redisson-3-25] [] mailbox - handling notification. accountId=947efc84-ffb9-4f97-860a-9405514fcb95, changeId=2, channel=NOTIFICATION-CHANNEL-78
  2206. 2018-09-06 13:36:09,691 INFO [qtp788117692-1905:https:https://zmc-proxy/service/soap/] [name=test1536240955501.236@zmc.com;mid=178;ip=10.0.0.20;port=33564;soapId=4e502284;] soap - GetMailboxMetadataRequest elapsed=3
  2207.  
  2208. ==> /opt/zimbra/log/audit.log <==
  2209. 2018-09-06 13:36:09,327 INFO [qtp788117692-1915:https:https://zmc-proxy/service/soap/] [name=test1536240955501.236@zmc.com;ip=10.0.0.20;port=33556;soapId=4e502282;] security - cmd=Auth; account=test1536240955501.236@zmc.com; protocol=soap;
  2210.  
  2211. ==> /opt/zimbra/log/mailbox.log <==
  2212. 2018-09-06 13:36:09,705 INFO [qtp788117692-1778:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44344;soapId=4faf0d59;] soap - AuthRequest elapsed=1
  2213. 2018-09-06 13:36:09,759 INFO [qtp788117692-1729:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56030;soapId=4faf0d5a;] soap - CreateAccountRequest elapsed=4
  2214.  
  2215. ==> /opt/zimbra/log/audit.log <==
  2216. 2018-09-06 13:36:09,704 INFO [qtp788117692-1778:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44344;soapId=4faf0d59;] security - cmd=AdminAuth; account=admin@zmc.com;
  2217. 2018-09-06 13:36:09,705 INFO [qtp788117692-1778:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44344;soapId=4faf0d59;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2218. 2018-09-06 13:36:09,758 INFO [qtp788117692-1729:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56030;soapId=4faf0d5a;] security - cmd=CreateAccount; name=test1536240955501.237@zmc.com;
  2219.  
  2220. ==> /opt/zimbra/log/mailbox.log <==
  2221. 2018-09-06 13:36:12,777 INFO [qtp788117692-1916:https:https://zmc-proxy/service/soap/] [name=test1536240955501.237@zmc.com;ip=10.0.0.19;port=43102;soapId=4e502285;] soap - AuthRequest elapsed=1
  2222.  
  2223. ==> /opt/zimbra/log/audit.log <==
  2224. 2018-09-06 13:36:12,777 INFO [qtp788117692-1916:https:https://zmc-proxy/service/soap/] [name=test1536240955501.237@zmc.com;ip=10.0.0.19;port=43102;soapId=4e502285;] security - cmd=Auth; account=test1536240955501.237@zmc.com; protocol=soap;
  2225.  
  2226. ==> /opt/zimbra/log/mailbox.log <==
  2227. 2018-09-06 13:36:12,883 INFO [qtp788117692-1793:https:https://zmc-proxy/service/soap/] [name=test1536240955501.237@zmc.com;ip=10.0.0.20;port=33578;soapId=4faf0d5b;] mailbox - Creating mailbox with id 179 and group id 79 for test1536240955501.237@zmc.com.
  2228. 2018-09-06 13:36:12,886 INFO [qtp788117692-1793:https:https://zmc-proxy/service/soap/] [name=test1536240955501.237@zmc.com;ip=10.0.0.20;port=33578;soapId=4faf0d5b;] cache - initializing folder and tag caches for mailbox 179
  2229.  
  2230. ==> /opt/zimbra/log/mailbox.log <==
  2231. 2018-09-06 13:36:13,131 INFO [qtp788117692-1934:https:https://zmc-proxy/service/soap/] [name=test1536240955501.237@zmc.com;ip=10.0.0.19;port=43110;] mbxmgr - Mailbox 179 account 6dc5002d-a930-457b-bc8f-bdba4d16db8e LOADED
  2232. 2018-09-06 13:36:13,132 INFO [qtp788117692-1934:https:https://zmc-proxy/service/soap/] [name=test1536240955501.237@zmc.com;mid=179;ip=10.0.0.19;port=43110;soapId=4e502286;] cache - initializing folder and tag caches for mailbox 179
  2233. 2018-09-06 13:36:13,198 INFO [qtp788117692-1934:https:https://zmc-proxy/service/soap/] [name=test1536240955501.237@zmc.com;mid=179;ip=10.0.0.19;port=43110;soapId=4e502286;] mailbox - added account 6dc5002d-a930-457b-bc8f-bdba4d16db8e to Redis notification channel NOTIFICATION-CHANNEL-79
  2234. 2018-09-06 13:36:13,198 INFO [qtp788117692-1934:https:https://zmc-proxy/service/soap/] [name=test1536240955501.237@zmc.com;mid=179;ip=10.0.0.19;port=43110;soapId=4e502286;] soap - ModifyMailboxMetadataRequest elapsed=67
  2235. 2018-09-06 13:36:13,198 INFO [redisson-3-5] [] mailbox - handling notification. accountId=6dc5002d-a930-457b-bc8f-bdba4d16db8e, changeId=3, channel=NOTIFICATION-CHANNEL-79
  2236. 2018-09-06 13:36:13,215 INFO [qtp788117692-1906:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56046;soapId=4e502287;] soap - AuthRequest elapsed=1
  2237. 2018-09-06 13:36:13,331 INFO [qtp788117692-1921:https:https://zmc-proxy/service/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=43118;soapId=4e502288;] soap - NoOpRequest elapsed=0
  2238. 2018-09-06 13:36:13,344 INFO [qtp788117692-1874:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56054;soapId=4e502289;] soap - AuthRequest elapsed=0
  2239.  
  2240. ==> /opt/zimbra/log/audit.log <==
  2241. 2018-09-06 13:36:13,214 INFO [qtp788117692-1906:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56046;soapId=4e502287;] security - cmd=AdminAuth; account=admin@zmc.com;
  2242. 2018-09-06 13:36:13,215 INFO [qtp788117692-1906:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56046;soapId=4e502287;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2243. 2018-09-06 13:36:13,344 INFO [qtp788117692-1874:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56054;soapId=4e502289;] security - cmd=AdminAuth; account=admin@zmc.com;
  2244. 2018-09-06 13:36:13,344 INFO [qtp788117692-1874:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56054;soapId=4e502289;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2245. 2018-09-06 13:36:13,094 INFO [qtp788117692-1793:https:https://zmc-proxy/service/soap/] [name=test1536240955501.237@zmc.com;ip=10.0.0.20;port=33578;soapId=4faf0d5b;] mailbox - added account 6dc5002d-a930-457b-bc8f-bdba4d16db8e to Redis notification channel NOTIFICATION-CHANNEL-79
  2246. 2018-09-06 13:36:13,095 INFO [redisson-3-14] [] mailbox - handling notification. accountId=6dc5002d-a930-457b-bc8f-bdba4d16db8e, changeId=1, channel=NOTIFICATION-CHANNEL-79
  2247. 2018-09-06 13:36:13,096 INFO [qtp788117692-1793:https:https://zmc-proxy/service/soap/] [name=test1536240955501.237@zmc.com;ip=10.0.0.20;port=33578;soapId=4faf0d5b;] mbxmgr - Mailbox 179 account 6dc5002d-a930-457b-bc8f-bdba4d16db8e CREATED
  2248. 2018-09-06 13:36:13,102 INFO [qtp788117692-1793:https:https://zmc-proxy/service/soap/] [name=test1536240955501.237@zmc.com;mid=179;ip=10.0.0.20;port=33578;soapId=4faf0d5b;] soap - SetMailboxMetadataRequest elapsed=301
  2249. 2018-09-06 13:36:13,117 INFO [redisson-3-14] [] mailbox - handling notification. accountId=6dc5002d-a930-457b-bc8f-bdba4d16db8e, changeId=2, channel=NOTIFICATION-CHANNEL-79
  2250. 2018-09-06 13:36:13,198 INFO [redisson-3-10] [] mailbox - handling notification. accountId=6dc5002d-a930-457b-bc8f-bdba4d16db8e, changeId=3, channel=NOTIFICATION-CHANNEL-79
  2251. 2018-09-06 13:36:13,266 INFO [qtp788117692-1755:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44364;soapId=4faf0d5c;] soap - AuthRequest elapsed=0
  2252. 2018-09-06 13:36:13,280 INFO [qtp788117692-1742:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56050;soapId=4faf0d5d;] misc - delegated access: doc=GetAccount, authenticated account=admin@zmc.com, target account=admin@zmc.com
  2253. 2018-09-06 13:36:13,280 INFO [qtp788117692-1742:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56050;soapId=4faf0d5d;] misc - delegated access: doc=GetAccount, authenticated account=admin@zmc.com, target account=admin@zmc.com
  2254. 2018-09-06 13:36:13,281 INFO [qtp788117692-1742:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56050;soapId=4faf0d5d;] soap - GetAccountRequest elapsed=1
  2255. 2018-09-06 13:36:13,361 INFO [qtp788117692-1743:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44372;soapId=4faf0d5e;] soap - CreateAccountRequest elapsed=4
  2256.  
  2257. ==> /opt/zimbra/log/audit.log <==
  2258. 2018-09-06 13:36:13,266 INFO [qtp788117692-1755:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44364;soapId=4faf0d5c;] security - cmd=AdminAuth; account=admin@zmc.com;
  2259. 2018-09-06 13:36:13,266 INFO [qtp788117692-1755:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44364;soapId=4faf0d5c;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2260. 2018-09-06 13:36:13,360 INFO [qtp788117692-1743:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44372;soapId=4faf0d5e;] security - cmd=CreateAccount; name=test.1536240973333.258@zmc.com; zimbraNotes=test of zimbraPasswordMustChange=TRUE; zimbraPasswordMustChange=TRUE;
  2261.  
  2262. ==> /opt/zimbra/log/mailbox.log <==
  2263. 2018-09-06 13:36:14,841 INFO [Timer-Zimbra] [] FileUploadServlet - Removed 1 expired file uploads; 0 pending file uploads
  2264.  
  2265. ==> /opt/zimbra/log/mailbox.log <==
  2266. 2018-09-06 13:36:15,469 INFO [Timer-Zimbra] [] FileUploadServlet - 2 pending file uploads
  2267. 2018-09-06 13:36:16,394 INFO [qtp788117692-1886:https:https://zmc-proxy/service/soap/] [ip=10.0.0.19;port=43130;soapId=4e50228a;] soap - ChangePasswordRequest elapsed=3
  2268. 2018-09-06 13:36:16,379 WARN [qtp788117692-1745:https:https://zmc-proxy/service/soap/] [name=test.1536240973333.258@zmc.com;ip=10.0.0.20;port=33600;soapId=4faf0d5f;] SoapEngine - handler exception
  2269. com.zimbra.cs.account.AccountServiceException: must change password
  2270. ExceptionId:qtp788117692-1745:https:https://zmc-proxy/service/soap/:1536240976379:33d5d0aceb48fe53
  2271. Code:account.CHANGE_PASSWORD
  2272. at com.zimbra.cs.account.AccountServiceException.CHANGE_PASSWORD(AccountServiceException.java:167)
  2273. at com.zimbra.cs.account.ldap.LdapProvisioning.authAccount(LdapProvisioning.java:5369)
  2274. at com.zimbra.cs.account.ldap.LdapProvisioning.authAccount(LdapProvisioning.java:5327)
  2275. at com.zimbra.cs.service.account.Auth.handle(Auth.java:317)
  2276. at com.zimbra.soap.SoapEngine.dispatchRequest(SoapEngine.java:656)
  2277. at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:501)
  2278. at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:288)
  2279. at com.zimbra.soap.SoapServlet.doWork(SoapServlet.java:303)
  2280. at com.zimbra.soap.SoapServlet.doPost(SoapServlet.java:213)
  2281. at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  2282. at com.zimbra.cs.servlet.ZimbraServlet.service(ZimbraServlet.java:211)
  2283. at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
  2284. at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:821)
  2285. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1685)
  2286. at com.zimbra.cs.servlet.CsrfFilter.doFilter(CsrfFilter.java:169)
  2287. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2288. at com.zimbra.cs.servlet.RequestStringFilter.doFilter(RequestStringFilter.java:54)
  2289. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2290. at com.zimbra.cs.servlet.SetHeaderFilter.doFilter(SetHeaderFilter.java:59)
  2291. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2292. at com.zimbra.cs.servlet.ETagHeaderFilter.doFilter(ETagHeaderFilter.java:47)
  2293. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2294. at com.zimbra.cs.servlet.ContextPathBasedThreadPoolBalancerFilter.doFilter(ContextPathBasedThreadPoolBalancerFilter.java:107)
  2295. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2296. at com.zimbra.cs.servlet.ZimbraQoSFilter.doFilter(ZimbraQoSFilter.java:116)
  2297. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2298. at com.zimbra.cs.servlet.ZimbraInvalidLoginFilter.doFilter(ZimbraInvalidLoginFilter.java:117)
  2299. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2300. at org.eclipse.jetty.servlets.DoSFilter.doFilterChain(DoSFilter.java:473)
  2301. at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:318)
  2302. at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:288)
  2303. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2304. at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
  2305. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
  2306. at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
  2307. at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
  2308. at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1158)
  2309. at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
  2310. at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
  2311. at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1090)
  2312. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
  2313. at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
  2314. at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:109)
  2315. at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119)
  2316. at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:318)
  2317. at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:437)
  2318. at org.eclipse.jetty.server.handler.DebugHandler.handle(DebugHandler.java:84)
  2319. at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119)
  2320. at org.eclipse.jetty.server.Server.handle(Server.java:517)
  2321. at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:306)
  2322. at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:242)
  2323. at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261)
  2324. at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
  2325. at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:192)
  2326. at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261)
  2327. at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
  2328. at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:75)
  2329. at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:213)
  2330. at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:147)
  2331. at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
  2332. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  2333. at java.lang.Thread.run(Thread.java:748)
  2334. 2018-09-06 13:36:16,379 INFO [qtp788117692-1745:https:https://zmc-proxy/service/soap/] [name=test.1536240973333.258@zmc.com;ip=10.0.0.20;port=33600;soapId=4faf0d5f;] soap - AuthRequest elapsed=1
  2335.  
  2336. ==> /opt/zimbra/log/audit.log <==
  2337. 2018-09-06 13:36:16,379 WARN [qtp788117692-1745:https:https://zmc-proxy/service/soap/] [name=test.1536240973333.258@zmc.com;ip=10.0.0.20;port=33600;soapId=4faf0d5f;] security - cmd=Auth; account=test.1536240973333.258@zmc.com; protocol=soap; error=must change password;
  2338.  
  2339. ==> /opt/zimbra/log/mailbox.log <==
  2340. 2018-09-06 13:36:21,546 INFO [MailboxPurge] [name=test.1536239362715.20@zmc.com;mid=66;] purge - Purging messages.
  2341. 2018-09-06 13:36:23,124 INFO [MailboxPurge] [name=test.1536239511201.65@zmc.com;mid=71;] purge - Purging messages.
  2342. 2018-09-06 13:36:26,409 INFO [qtp788117692-1905:https:https://zmc-proxy/service/soap/] [name=test.1536240973333.258@zmc.com;ip=10.0.0.20;port=33614;soapId=4e50228b;] soap - AuthRequest elapsed=1
  2343.  
  2344. ==> /opt/zimbra/log/audit.log <==
  2345. 2018-09-06 13:36:26,409 INFO [qtp788117692-1905:https:https://zmc-proxy/service/soap/] [name=test.1536240973333.258@zmc.com;ip=10.0.0.20;port=33614;soapId=4e50228b;] security - cmd=Auth; account=test.1536240973333.258@zmc.com; protocol=soap;
  2346. 2018-09-06 13:36:26,425 INFO [qtp788117692-1778:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44394;soapId=4faf0d60;] soap - AuthRequest elapsed=0
  2347. 2018-09-06 13:36:26,443 INFO [qtp788117692-1745:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56080;soapId=4faf0d61;] soap - CreateAccountRequest elapsed=4
  2348.  
  2349. ==> /opt/zimbra/log/audit.log <==
  2350. 2018-09-06 13:36:26,425 INFO [qtp788117692-1778:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44394;soapId=4faf0d60;] security - cmd=AdminAuth; account=admin@zmc.com;
  2351. 2018-09-06 13:36:26,425 INFO [qtp788117692-1778:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44394;soapId=4faf0d60;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2352. 2018-09-06 13:36:26,442 INFO [qtp788117692-1745:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56080;soapId=4faf0d61;] security - cmd=CreateAccount; name=account11536240986412259@zmc.com;
  2353.  
  2354. ==> /opt/zimbra/log/mailbox.log <==
  2355. 2018-09-06 13:36:29,463 INFO [qtp788117692-1958:https:https://zmc-proxy/service/soap/] [name=account11536240986412259@zmc.com;ip=10.0.0.19;port=43150;soapId=4e50228c;] soap - AuthRequest elapsed=1
  2356. 2018-09-06 13:36:29,553 INFO [qtp788117692-1953:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56092;soapId=4e50228d;] soap - CreateAccountRequest elapsed=4
  2357.  
  2358. ==> /opt/zimbra/log/audit.log <==
  2359. 2018-09-06 13:36:29,463 INFO [qtp788117692-1958:https:https://zmc-proxy/service/soap/] [name=account11536240986412259@zmc.com;ip=10.0.0.19;port=43150;soapId=4e50228c;] security - cmd=Auth; account=account11536240986412259@zmc.com; protocol=soap;
  2360. 2018-09-06 13:36:29,552 INFO [qtp788117692-1953:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56092;soapId=4e50228d;] security - cmd=CreateAccount; name=account21536240986412260@zmc.com;
  2361.  
  2362. ==> /opt/zimbra/log/mailbox.log <==
  2363. 2018-09-06 13:36:29,488 INFO [qtp788117692-1541:https:https://zmc-proxy/service/soap/] [name=account11536240986412259@zmc.com;ip=10.0.0.20;port=33626;soapId=4faf0d62;] soap - GrantPermissionRequest elapsed=1
  2364. 2018-09-06 13:36:29,534 INFO [qtp788117692-1693:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44406;soapId=4faf0d63;] soap - AuthRequest elapsed=0
  2365.  
  2366. ==> /opt/zimbra/log/audit.log <==
  2367. 2018-09-06 13:36:29,534 INFO [qtp788117692-1693:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44406;soapId=4faf0d63;] security - cmd=AdminAuth; account=admin@zmc.com;
  2368. 2018-09-06 13:36:29,534 INFO [qtp788117692-1693:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44406;soapId=4faf0d63;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2369.  
  2370. ==> /opt/zimbra/log/mailbox.log <==
  2371. 2018-09-06 13:36:32,572 INFO [qtp788117692-1891:https:https://zmc-proxy/service/soap/] [name=account21536240986412260@zmc.com;ip=10.0.0.19;port=43164;soapId=4e50228e;] soap - AuthRequest elapsed=1
  2372. 2018-09-06 13:36:32,597 INFO [qtp788117692-1906:https:https://zmc-proxy/service/soap/] [name=account21536240986412260@zmc.com;ip=10.0.0.20;port=33640;soapId=4e50228f;] soap - GrantPermissionRequest elapsed=2
  2373. 2018-09-06 13:36:32,620 INFO [qtp788117692-1874:https:https://zmc-proxy/service/soap/] [name=account21536240986412260@zmc.com;ip=10.0.0.19;port=43172;soapId=4e502290;] soap - GetPermissionRequest elapsed=0
  2374.  
  2375. ==> /opt/zimbra/log/audit.log <==
  2376. 2018-09-06 13:36:32,571 INFO [qtp788117692-1891:https:https://zmc-proxy/service/soap/] [name=account21536240986412260@zmc.com;ip=10.0.0.19;port=43164;soapId=4e50228e;] security - cmd=Auth; account=account21536240986412260@zmc.com; protocol=soap;
  2377.  
  2378. ==> /opt/zimbra/log/mailbox.log <==
  2379. 2018-09-06 13:36:32,634 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56108;soapId=4faf0d64;] soap - AuthRequest elapsed=1
  2380. 2018-09-06 13:36:32,652 INFO [qtp788117692-1743:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44426;soapId=4faf0d65;] soap - CreateAccountRequest elapsed=4
  2381.  
  2382. ==> /opt/zimbra/log/audit.log <==
  2383. 2018-09-06 13:36:32,633 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56108;soapId=4faf0d64;] security - cmd=AdminAuth; account=admin@zmc.com;
  2384. 2018-09-06 13:36:32,634 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56108;soapId=4faf0d64;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2385. 2018-09-06 13:36:32,651 INFO [qtp788117692-1743:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44426;soapId=4faf0d65;] security - cmd=CreateAccount; name=account31536240986412261@zmc.com;
  2386.  
  2387. ==> /opt/zimbra/log/mailbox.log <==
  2388. 2018-09-06 13:36:35,704 INFO [qtp788117692-1957:https:https://zmc-proxy/service/soap/] [name=account31536240986412261@zmc.com;ip=10.0.0.20;port=33652;soapId=4e502291;] soap - AuthRequest elapsed=1
  2389. 2018-09-06 13:36:35,755 INFO [qtp788117692-1959:https:https://zmc-proxy/service/soap/] [name=account31536240986412261@zmc.com;ip=10.0.0.20;port=33660;soapId=4e502292;] soap - RevokePermissionRequest elapsed=1
  2390. 2018-09-06 13:36:35,772 INFO [qtp788117692-1953:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44440;soapId=4e502293;] soap - AuthRequest elapsed=1
  2391.  
  2392. ==> /opt/zimbra/log/audit.log <==
  2393. 2018-09-06 13:36:35,704 INFO [qtp788117692-1957:https:https://zmc-proxy/service/soap/] [name=account31536240986412261@zmc.com;ip=10.0.0.20;port=33652;soapId=4e502291;] security - cmd=Auth; account=account31536240986412261@zmc.com; protocol=soap;
  2394. 2018-09-06 13:36:35,771 INFO [qtp788117692-1953:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44440;soapId=4e502293;] security - cmd=AdminAuth; account=admin@zmc.com;
  2395. 2018-09-06 13:36:35,772 INFO [qtp788117692-1953:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44440;soapId=4e502293;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2396.  
  2397. ==> /opt/zimbra/log/mailbox.log <==
  2398. 2018-09-06 13:36:35,730 INFO [qtp788117692-1740:https:https://zmc-proxy/service/soap/] [name=account31536240986412261@zmc.com;ip=10.0.0.19;port=43184;soapId=4faf0d66;] soap - GrantPermissionRequest elapsed=1
  2399. 2018-09-06 13:36:35,792 INFO [qtp788117692-1758:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56126;soapId=4faf0d67;] soap - CreateAccountRequest elapsed=4
  2400. 2018-09-06 13:36:35,821 INFO [qtp788117692-1778:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44446;soapId=4faf0d68;] soap - CreateAccountRequest elapsed=4
  2401.  
  2402. ==> /opt/zimbra/log/audit.log <==
  2403. 2018-09-06 13:36:35,791 INFO [qtp788117692-1758:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56126;soapId=4faf0d67;] security - cmd=CreateAccount; name=account41536240986412262@zmc.com;
  2404. 2018-09-06 13:36:35,820 INFO [qtp788117692-1778:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44446;soapId=4faf0d68;] security - cmd=CreateAccount; name=account51536240986413263@zmc.com;
  2405.  
  2406. ==> /opt/zimbra/log/mailbox.log <==
  2407. 2018-09-06 13:36:38,945 INFO [qtp788117692-1541:https:https://zmc-proxy/service/soap/] [name=account41536240986412262@zmc.com;ip=10.0.0.19;port=43208;soapId=4faf0d69;] soap - GrantPermissionRequest elapsed=1
  2408. 2018-09-06 13:36:38,960 INFO [qtp788117692-1700:https:https://zmc-proxy/service/soap/] [name=account51536240986413263@zmc.com;ip=10.0.0.20;port=33682;soapId=4faf0d6a;] soap - AuthRequest elapsed=2
  2409.  
  2410. ==> /opt/zimbra/log/audit.log <==
  2411. 2018-09-06 13:36:38,959 INFO [qtp788117692-1700:https:https://zmc-proxy/service/soap/] [name=account51536240986413263@zmc.com;ip=10.0.0.20;port=33682;soapId=4faf0d6a;] security - cmd=Auth; account=account51536240986413263@zmc.com; protocol=soap;
  2412.  
  2413. ==> /opt/zimbra/log/mailbox.log <==
  2414. 2018-09-06 13:36:38,887 INFO [qtp788117692-1897:https:https://zmc-proxy/service/soap/] [name=account41536240986412262@zmc.com;ip=10.0.0.20;port=33676;soapId=4e502294;] soap - AuthRequest elapsed=2
  2415. 2018-09-06 13:36:39,031 INFO [qtp788117692-1916:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56150;soapId=4e502295;] soap - AuthRequest elapsed=1
  2416. 2018-09-06 13:36:39,048 INFO [qtp788117692-1934:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44468;soapId=4e502296;] soap - CreateAccountRequest elapsed=4
  2417.  
  2418. ==> /opt/zimbra/log/audit.log <==
  2419. 2018-09-06 13:36:38,886 INFO [qtp788117692-1897:https:https://zmc-proxy/service/soap/] [name=account41536240986412262@zmc.com;ip=10.0.0.20;port=33676;soapId=4e502294;] security - cmd=Auth; account=account41536240986412262@zmc.com; protocol=soap;
  2420. 2018-09-06 13:36:39,030 INFO [qtp788117692-1916:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56150;soapId=4e502295;] security - cmd=AdminAuth; account=admin@zmc.com;
  2421. 2018-09-06 13:36:39,031 INFO [qtp788117692-1916:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56150;soapId=4e502295;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2422. 2018-09-06 13:36:39,047 INFO [qtp788117692-1934:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44468;soapId=4e502296;] security - cmd=CreateAccount; name=test.1536240999018.264@zmc.com;
  2423.  
  2424. ==> /opt/zimbra/log/mailbox.log <==
  2425. 2018-09-06 13:36:39,015 INFO [qtp788117692-1742:https:https://zmc-proxy/service/soap/] [name=account51536240986413263@zmc.com;ip=10.0.0.19;port=43214;soapId=4faf0d6b;] soap - CheckPermissionRequest elapsed=0
  2426.  
  2427. ==> /opt/zimbra/log/mailbox.log <==
  2428. 2018-09-06 13:36:42,091 INFO [qtp788117692-1921:https:https://zmc-proxy/service/soap/] [name=test.1536240999018.264@zmc.com;ip=10.0.0.19;port=43228;soapId=4e502297;] soap - ModifyPrefsRequest elapsed=2
  2429. 2018-09-06 13:36:42,179 INFO [qtp788117692-1958:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56170;soapId=4e502298;] soap - CreateAccountRequest elapsed=4
  2430.  
  2431. ==> /opt/zimbra/log/audit.log <==
  2432. 2018-09-06 13:36:42,178 INFO [qtp788117692-1958:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56170;soapId=4e502298;] security - cmd=CreateAccount; name=wiki1.1536241002149.265@zmc.com;
  2433. 2018-09-06 13:36:42,067 INFO [qtp788117692-1745:https:https://zmc-proxy/service/soap/] [name=test.1536240999018.264@zmc.com;ip=10.0.0.20;port=33696;soapId=4faf0d6c;] soap - AuthRequest elapsed=1
  2434. 2018-09-06 13:36:42,147 INFO [qtp788117692-1801:https:https://zmc-proxy/service/soap/] [name=test.1536240999018.264@zmc.com;ip=10.0.0.20;port=33704;soapId=4faf0d6d;] soap - GetPrefsRequest elapsed=0
  2435. 2018-09-06 13:36:42,162 INFO [qtp788117692-1778:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44484;soapId=4faf0d6e;] soap - AuthRequest elapsed=1
  2436.  
  2437. ==> /opt/zimbra/log/audit.log <==
  2438. 2018-09-06 13:36:42,067 INFO [qtp788117692-1745:https:https://zmc-proxy/service/soap/] [name=test.1536240999018.264@zmc.com;ip=10.0.0.20;port=33696;soapId=4faf0d6c;] security - cmd=Auth; account=test.1536240999018.264@zmc.com; protocol=soap;
  2439. 2018-09-06 13:36:42,161 INFO [qtp788117692-1778:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44484;soapId=4faf0d6e;] security - cmd=AdminAuth; account=admin@zmc.com;
  2440. 2018-09-06 13:36:42,161 INFO [qtp788117692-1778:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44484;soapId=4faf0d6e;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2441.  
  2442. ==> /var/log/zimbra.log <==
  2443. Sep 6 13:36:43 zmc-mailbox zmconfigd[1705]: Fetching All configs
  2444. Sep 6 13:36:43 zmc-mailbox zmconfigd[1705]: All configs fetched in 0.03 seconds
  2445. Sep 6 13:36:43 zmc-mailbox zmconfigd[1705]: Command not defined for imapd
  2446. Sep 6 13:36:43 zmc-mailbox zmconfigd[1705]: Watchdog: skipping service antivirus. Service not yet available for restarts.
  2447. Sep 6 13:36:44 zmc-mailbox zmconfigd[1705]: All rewrite threads completed in 0.00 sec
  2448. Sep 6 13:36:44 zmc-mailbox zmconfigd[1705]: All restarts completed in 0.00 sec
  2449.  
  2450. ==> /var/log/zimbra.log <==
  2451. Sep 6 13:36:43 zmc-mailbox zmconfigd[1703]: Fetching All configs
  2452. Sep 6 13:36:43 zmc-mailbox zmconfigd[1703]: All configs fetched in 0.03 seconds
  2453. Sep 6 13:36:43 zmc-mailbox zmconfigd[1703]: Command not defined for imapd
  2454. Sep 6 13:36:44 zmc-mailbox zmconfigd[1703]: Watchdog: skipping service antivirus. Service not yet available for restarts.
  2455. Sep 6 13:36:44 zmc-mailbox zmconfigd[1703]: All rewrite threads completed in 0.00 sec
  2456. Sep 6 13:36:44 zmc-mailbox zmconfigd[1703]: All restarts completed in 0.00 sec
  2457.  
  2458. ==> /opt/zimbra/log/mailbox.log <==
  2459. 2018-09-06 13:36:45,198 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=wiki1.1536241002149.265@zmc.com;ip=10.0.0.19;port=43240;soapId=4e502299;] soap - AuthRequest elapsed=2
  2460. 2018-09-06 13:36:45,316 INFO [qtp788117692-1874:https:https://zmc-proxy/service/soap/] [name=wiki1.1536241002149.265@zmc.com;ip=10.0.0.19;port=43246;soapId=4e50229a;] mailbox - Creating mailbox with id 180 and group id 80 for wiki1.1536241002149.265@zmc.com.
  2461. 2018-09-06 13:36:45,319 INFO [qtp788117692-1874:https:https://zmc-proxy/service/soap/] [name=wiki1.1536241002149.265@zmc.com;ip=10.0.0.19;port=43246;soapId=4e50229a;] cache - initializing folder and tag caches for mailbox 180
  2462. 2018-09-06 13:36:45,531 INFO [qtp788117692-1874:https:https://zmc-proxy/service/soap/] [name=wiki1.1536241002149.265@zmc.com;ip=10.0.0.19;port=43246;soapId=4e50229a;] mailbox - added account 37d1b71f-5d67-4832-99e9-4924a1394101 to Redis notification channel NOTIFICATION-CHANNEL-80
  2463. 2018-09-06 13:36:45,532 INFO [redisson-3-16] [] mailbox - handling notification. accountId=37d1b71f-5d67-4832-99e9-4924a1394101, changeId=1, channel=NOTIFICATION-CHANNEL-80
  2464. 2018-09-06 13:36:45,535 INFO [qtp788117692-1874:https:https://zmc-proxy/service/soap/] [name=wiki1.1536241002149.265@zmc.com;ip=10.0.0.19;port=43246;soapId=4e50229a;] mbxmgr - Mailbox 180 account 37d1b71f-5d67-4832-99e9-4924a1394101 CREATED
  2465. 2018-09-06 13:36:45,562 INFO [qtp788117692-1874:https:https://zmc-proxy/service/soap/] [name=wiki1.1536241002149.265@zmc.com;mid=180;ip=10.0.0.19;port=43246;soapId=4e50229a;] soap - GetFolderRequest elapsed=329
  2466.  
  2467. ==> /opt/zimbra/log/audit.log <==
  2468. 2018-09-06 13:36:45,197 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=wiki1.1536241002149.265@zmc.com;ip=10.0.0.19;port=43240;soapId=4e502299;] security - cmd=Auth; account=wiki1.1536241002149.265@zmc.com; protocol=soap;
  2469.  
  2470. ==> /opt/zimbra/log/mailbox.log <==
  2471. 2018-09-06 13:36:45,212 INFO [qtp788117692-1561:https:https://zmc-proxy/service/soap/] [name=wiki1.1536241002149.265@zmc.com;ip=10.0.0.20;port=33714;soapId=4faf0d6f;] soap - AuthRequest elapsed=1
  2472. 2018-09-06 13:36:45,589 INFO [qtp788117692-1693:https:https://zmc-proxy/service/upload?fmt=raw] [name=wiki1.1536241002149.265@zmc.com;ip=10.0.0.20;port=33722;ua=Jakarta Commons-HttpClient/3.1;] mbxmgr - Mailbox 180 account 37d1b71f-5d67-4832-99e9-4924a1394101 LOADED
  2473. 2018-09-06 13:36:45,590 INFO [qtp788117692-1693:https:https://zmc-proxy/service/upload?fmt=raw] [name=wiki1.1536241002149.265@zmc.com;mid=180;ip=10.0.0.20;port=33722;ua=Jakarta Commons-HttpClient/3.1;] FileUploadServlet - Received multipart: Upload: { accountId=37d1b71f-5d67-4832-99e9-4924a1394101, time=Thu Sep 06 13:36:45 UTC 2018, size=2576, uploadId=d5f0f26e-04d6-463a-8d2e-352647ebeaab:4dfbd285-50dd-47ed-95b9-3e7701a9f015, name=contact1.txt, path=/opt/zimbra/data/tmp/upload/upload_37d1b71f-5d67-4832-99e9-4924a1394101_d5f0f26e-04d6-463a-8d2e-352647ebeaab:4dfbd285-50dd-47ed-95b9-3e7701a9f015.tmp }
  2474. 2018-09-06 13:36:45,608 INFO [qtp788117692-1803:https:https://zmc-proxy/service/soap/] [name=wiki1.1536241002149.265@zmc.com;mid=180;ip=10.0.0.19;port=43252;soapId=4faf0d70;] cache - initializing folder and tag caches for mailbox 180
  2475. 2018-09-06 13:36:45,666 INFO [qtp788117692-1803:https:https://zmc-proxy/service/soap/] [name=wiki1.1536241002149.265@zmc.com;mid=180;ip=10.0.0.19;port=43252;soapId=4faf0d70;] mailbox - uploadId=d5f0f26e-04d6-463a-8d2e-352647ebeaab:4dfbd285-50dd-47ed-95b9-3e7701a9f015
  2476. 2018-09-06 13:36:45,675 INFO [qtp788117692-1803:https:https://zmc-proxy/service/soap/] [name=wiki1.1536241002149.265@zmc.com;mid=180;ip=10.0.0.19;port=43252;soapId=4faf0d70;] mailop - Adding Document contact1.txt: id=257, folderId=16, folderName=Briefcase
  2477. 2018-09-06 13:36:45,823 INFO [qtp788117692-1803:https:https://zmc-proxy/service/soap/] [name=wiki1.1536241002149.265@zmc.com;mid=180;ip=10.0.0.19;port=43252;soapId=4faf0d70;] mailbox - beginning listening on Redis notification channel NOTIFICATION-CHANNEL-80
  2478. 2018-09-06 13:36:45,823 INFO [qtp788117692-1803:https:https://zmc-proxy/service/soap/] [name=wiki1.1536241002149.265@zmc.com;mid=180;ip=10.0.0.19;port=43252;soapId=4faf0d70;] mailbox - added account 37d1b71f-5d67-4832-99e9-4924a1394101 to Redis notification channel NOTIFICATION-CHANNEL-80
  2479. 2018-09-06 13:36:45,823 INFO [redisson-3-7] [] mailbox - handling notification. accountId=37d1b71f-5d67-4832-99e9-4924a1394101, changeId=2, channel=NOTIFICATION-CHANNEL-80
  2480. 2018-09-06 13:36:45,824 INFO [qtp788117692-1803:https:https://zmc-proxy/service/soap/] [name=wiki1.1536241002149.265@zmc.com;mid=180;ip=10.0.0.19;port=43252;soapId=4faf0d70;] soap - SaveDocumentRequest elapsed=218
  2481. 2018-09-06 13:36:45,879 INFO [qtp788117692-1804:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56194;soapId=4faf0d71;] soap - CreateAccountRequest elapsed=5
  2482.  
  2483. ==> /opt/zimbra/log/audit.log <==
  2484. 2018-09-06 13:36:45,212 INFO [qtp788117692-1561:https:https://zmc-proxy/service/soap/] [name=wiki1.1536241002149.265@zmc.com;ip=10.0.0.20;port=33714;soapId=4faf0d6f;] security - cmd=Auth; account=wiki1.1536241002149.265@zmc.com; protocol=soap;
  2485. 2018-09-06 13:36:45,878 INFO [qtp788117692-1804:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56194;soapId=4faf0d71;] security - cmd=CreateAccount; name=account1536241005845267@zmc.com;
  2486.  
  2487. ==> /opt/zimbra/log/mailbox.log <==
  2488. 2018-09-06 13:36:45,824 INFO [redisson-3-13] [] mailbox - handling notification. accountId=37d1b71f-5d67-4832-99e9-4924a1394101, changeId=2, channel=NOTIFICATION-CHANNEL-80
  2489. 2018-09-06 13:36:45,843 INFO [qtp788117692-1957:https:https://zmc-proxy/service/soap/] [name=wiki1.1536241002149.265@zmc.com;mid=180;ip=10.0.0.20;port=33728;soapId=4e50229b;] soap - PurgeRevisionRequest elapsed=4
  2490. 2018-09-06 13:36:45,857 INFO [qtp788117692-1897:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44508;soapId=4e50229c;] soap - AuthRequest elapsed=0
  2491.  
  2492. ==> /opt/zimbra/log/audit.log <==
  2493. 2018-09-06 13:36:45,857 INFO [qtp788117692-1897:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44508;soapId=4e50229c;] security - cmd=AdminAuth; account=admin@zmc.com;
  2494. 2018-09-06 13:36:45,857 INFO [qtp788117692-1897:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44508;soapId=4e50229c;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2495.  
  2496. ==> /opt/zimbra/log/mailbox.log <==
  2497. 2018-09-06 13:36:48,898 INFO [qtp788117692-1804:https:https://zmc-proxy/service/soap/] [name=account1536241005845267@zmc.com;ip=10.0.0.19;port=43264;soapId=4faf0d72;] soap - AuthRequest elapsed=1
  2498. 2018-09-06 13:36:49,006 INFO [qtp788117692-1804:https:https://zmc-proxy/service/soap/] [name=account1536241005845267@zmc.com;ip=10.0.0.20;port=33740;soapId=4faf0d73;] mailbox - Creating mailbox with id 181 and group id 81 for account1536241005845267@zmc.com.
  2499. 2018-09-06 13:36:49,009 INFO [qtp788117692-1804:https:https://zmc-proxy/service/soap/] [name=account1536241005845267@zmc.com;ip=10.0.0.20;port=33740;soapId=4faf0d73;] cache - initializing folder and tag caches for mailbox 181
  2500.  
  2501. ==> /opt/zimbra/log/audit.log <==
  2502. 2018-09-06 13:36:48,897 INFO [qtp788117692-1804:https:https://zmc-proxy/service/soap/] [name=account1536241005845267@zmc.com;ip=10.0.0.19;port=43264;soapId=4faf0d72;] security - cmd=Auth; account=account1536241005845267@zmc.com; protocol=soap;
  2503.  
  2504. ==> /opt/zimbra/log/mailbox.log <==
  2505. 2018-09-06 13:36:49,253 INFO [qtp788117692-1869:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44520;soapId=4e50229d;] soap - AuthRequest elapsed=1
  2506. 2018-09-06 13:36:49,269 INFO [qtp788117692-1903:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56206;soapId=4e50229e;] soap - CreateDomainRequest elapsed=4
  2507.  
  2508. ==> /opt/zimbra/log/audit.log <==
  2509. 2018-09-06 13:36:49,252 INFO [qtp788117692-1869:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44520;soapId=4e50229d;] security - cmd=AdminAuth; account=admin@zmc.com;
  2510. 2018-09-06 13:36:49,252 INFO [qtp788117692-1869:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44520;soapId=4e50229d;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2511. 2018-09-06 13:36:49,269 INFO [qtp788117692-1903:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56206;soapId=4e50229e;] security - cmd=CreateDomain; name=right.2681536241009240; zimbraDomainStatus=active; zimbraSMIMELdapAttribute=; zimbraSMIMELdapSearchBase=; zimbraDomainType=local; zimbraSMIMELdapStartTlsEnabled=; zimbraSMIMELdapURL=; zimbraSMIMELdapFilter=; zimbraSMIMELdapBindDn=; zimbraSMIMELdapBindPassword=;
  2512.  
  2513. ==> /opt/zimbra/log/mailbox.log <==
  2514. 2018-09-06 13:36:49,223 INFO [qtp788117692-1804:https:https://zmc-proxy/service/soap/] [name=account1536241005845267@zmc.com;ip=10.0.0.20;port=33740;soapId=4faf0d73;] mailbox - added account 6391e368-4eca-4073-a4e4-fd468d90a010 to Redis notification channel NOTIFICATION-CHANNEL-81
  2515. 2018-09-06 13:36:49,224 INFO [redisson-3-9] [] mailbox - handling notification. accountId=6391e368-4eca-4073-a4e4-fd468d90a010, changeId=1, channel=NOTIFICATION-CHANNEL-81
  2516. 2018-09-06 13:36:49,225 INFO [qtp788117692-1804:https:https://zmc-proxy/service/soap/] [name=account1536241005845267@zmc.com;ip=10.0.0.20;port=33740;soapId=4faf0d73;] mbxmgr - Mailbox 181 account 6391e368-4eca-4073-a4e4-fd468d90a010 CREATED
  2517. 2018-09-06 13:36:49,237 INFO [qtp788117692-1804:https:https://zmc-proxy/service/soap/] [name=account1536241005845267@zmc.com;ip=10.0.0.20;port=33740;soapId=4faf0d73;] soap - RankingActionRequest elapsed=318
  2518. 2018-09-06 13:36:49,249 INFO [redisson-3-9] [] mailbox - handling notification. accountId=6391e368-4eca-4073-a4e4-fd468d90a010, changeId=2, channel=NOTIFICATION-CHANNEL-81
  2519. 2018-09-06 13:36:49,250 INFO [redisson-3-9] [] mailbox - handling notification. accountId=6391e368-4eca-4073-a4e4-fd468d90a010, changeId=3, channel=NOTIFICATION-CHANNEL-81
  2520. 2018-09-06 13:36:49,288 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44524;soapId=4faf0d74;] soap - CreateAccountRequest elapsed=5
  2521.  
  2522. ==> /opt/zimbra/log/audit.log <==
  2523. 2018-09-06 13:36:49,287 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44524;soapId=4faf0d74;] security - cmd=CreateAccount; name=admin1.2741536241009240@right.2681536241009240; zimbraIsDelegatedAdminAccount=TRUE;
  2524.  
  2525. ==> /opt/zimbra/log/mailbox.log <==
  2526. 2018-09-06 13:36:52,323 INFO [qtp788117692-1916:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44532;soapId=4e50229f;] soap - AuthRequest elapsed=1
  2527. 2018-09-06 13:36:52,340 INFO [qtp788117692-1906:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56218;soapId=4e5022a0;] soap - CreateDomainRequest elapsed=4
  2528.  
  2529. ==> /opt/zimbra/log/audit.log <==
  2530. 2018-09-06 13:36:52,322 INFO [qtp788117692-1916:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44532;soapId=4e50229f;] security - cmd=AdminAuth; account=admin@zmc.com;
  2531. 2018-09-06 13:36:52,322 INFO [qtp788117692-1916:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44532;soapId=4e50229f;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2532. 2018-09-06 13:36:52,339 INFO [qtp788117692-1906:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56218;soapId=4e5022a0;] security - cmd=CreateDomain; name=right.2691536241009240; zimbraDomainStatus=active; zimbraSMIMELdapAttribute=; zimbraSMIMELdapSearchBase=; zimbraDomainType=local; zimbraSMIMELdapStartTlsEnabled=; zimbraSMIMELdapURL=; zimbraSMIMELdapFilter=; zimbraSMIMELdapBindDn=; zimbraSMIMELdapBindPassword=;
  2533.  
  2534. ==> /opt/zimbra/log/mailbox.log <==
  2535. 2018-09-06 13:36:52,308 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56214;soapId=4faf0d75;] soap - GrantRightRequest elapsed=2
  2536. 2018-09-06 13:36:52,361 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44536;soapId=4faf0d76;] soap - CreateAccountRequest elapsed=4
  2537.  
  2538. ==> /opt/zimbra/log/audit.log <==
  2539. 2018-09-06 13:36:52,360 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44536;soapId=4faf0d76;] security - cmd=CreateAccount; name=admin2.2751536241009240@right.2691536241009240; zimbraIsDelegatedAdminAccount=TRUE;
  2540.  
  2541. ==> /opt/zimbra/log/mailbox.log <==
  2542. 2018-09-06 13:36:55,400 INFO [qtp788117692-1959:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin2.2751536241009240@right.2691536241009240;ip=10.0.0.19;port=44542;soapId=4e5022a1;] soap - AuthRequest elapsed=2
  2543. 2018-09-06 13:36:55,415 INFO [qtp788117692-1959:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin2.2751536241009240@right.2691536241009240;ip=10.0.0.20;port=56228;soapId=4e5022a2;] soap - GetRightRequest elapsed=1
  2544. 2018-09-06 13:36:55,461 INFO [qtp788117692-1959:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44550;soapId=4e5022a3;] soap - CreateAccountRequest elapsed=4
  2545.  
  2546. ==> /opt/zimbra/log/audit.log <==
  2547. 2018-09-06 13:36:55,399 INFO [qtp788117692-1959:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin2.2751536241009240@right.2691536241009240;ip=10.0.0.19;port=44542;soapId=4e5022a1;] security - cmd=AdminAuth; account=admin2.2751536241009240@right.2691536241009240;
  2548. 2018-09-06 13:36:55,399 INFO [qtp788117692-1959:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin2.2751536241009240@right.2691536241009240;ip=10.0.0.19;port=44542;soapId=4e5022a1;] security - cmd=Auth; account=admin2.2751536241009240@right.2691536241009240; protocol=soap;
  2549. 2018-09-06 13:36:55,460 INFO [qtp788117692-1959:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44550;soapId=4e5022a3;] security - cmd=CreateAccount; name=admin1.2761536241009240@right.2701536241009240; zimbraIsDelegatedAdminAccount=TRUE;
  2550.  
  2551. ==> /opt/zimbra/log/mailbox.log <==
  2552. 2018-09-06 13:36:55,385 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56224;soapId=4faf0d77;] soap - GrantRightRequest elapsed=2
  2553. 2018-09-06 13:36:55,427 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44546;soapId=4faf0d78;] soap - AuthRequest elapsed=0
  2554. 2018-09-06 13:36:55,443 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56232;soapId=4faf0d79;] soap - CreateDomainRequest elapsed=3
  2555.  
  2556. ==> /opt/zimbra/log/audit.log <==
  2557. 2018-09-06 13:36:55,427 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44546;soapId=4faf0d78;] security - cmd=AdminAuth; account=admin@zmc.com;
  2558. 2018-09-06 13:36:55,427 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44546;soapId=4faf0d78;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2559. 2018-09-06 13:36:55,443 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56232;soapId=4faf0d79;] security - cmd=CreateDomain; name=right.2701536241009240; zimbraDomainStatus=active; zimbraSMIMELdapAttribute=; zimbraSMIMELdapSearchBase=; zimbraDomainType=local; zimbraSMIMELdapStartTlsEnabled=; zimbraSMIMELdapURL=; zimbraSMIMELdapFilter=; zimbraSMIMELdapBindDn=; zimbraSMIMELdapBindPassword=;
  2560.  
  2561. ==> /opt/zimbra/log/mailbox.log <==
  2562. 2018-09-06 13:36:58,480 INFO [qtp788117692-1959:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56240;soapId=4e5022a4;] soap - GrantRightRequest elapsed=2
  2563. 2018-09-06 13:36:58,535 INFO [qtp788117692-1959:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44562;soapId=4e5022a5;] soap - CreateDomainRequest elapsed=4
  2564. 2018-09-06 13:36:58,553 INFO [qtp788117692-1959:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56248;soapId=4e5022a6;] soap - CreateAccountRequest elapsed=4
  2565.  
  2566. ==> /opt/zimbra/log/audit.log <==
  2567. 2018-09-06 13:36:58,535 INFO [qtp788117692-1959:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44562;soapId=4e5022a5;] security - cmd=CreateDomain; name=right.2711536241009240; zimbraDomainStatus=active; zimbraSMIMELdapAttribute=; zimbraSMIMELdapSearchBase=; zimbraDomainType=local; zimbraSMIMELdapStartTlsEnabled=; zimbraSMIMELdapURL=; zimbraSMIMELdapFilter=; zimbraSMIMELdapBindDn=; zimbraSMIMELdapBindPassword=;
  2568. 2018-09-06 13:36:58,552 INFO [qtp788117692-1959:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56248;soapId=4e5022a6;] security - cmd=CreateAccount; name=admin2.2771536241009240@right.2711536241009240; zimbraIsDelegatedAdminAccount=TRUE;
  2569.  
  2570. ==> /opt/zimbra/log/mailbox.log <==
  2571. 2018-09-06 13:36:58,504 INFO [qtp788117692-1803:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44558;soapId=4faf0d7a;] soap - RevokeRightRequest elapsed=2
  2572. 2018-09-06 13:36:58,517 INFO [qtp788117692-1803:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56244;soapId=4faf0d7b;] soap - AuthRequest elapsed=1
  2573.  
  2574. ==> /opt/zimbra/log/audit.log <==
  2575. 2018-09-06 13:36:58,516 INFO [qtp788117692-1803:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56244;soapId=4faf0d7b;] security - cmd=AdminAuth; account=admin@zmc.com;
  2576. 2018-09-06 13:36:58,517 INFO [qtp788117692-1803:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56244;soapId=4faf0d7b;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2577.  
  2578. ==> /opt/zimbra/log/mailbox.log <==
  2579. 2018-09-06 13:37:01,603 INFO [qtp788117692-1959:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44572;soapId=4e5022a7;] soap - AuthRequest elapsed=1
  2580. 2018-09-06 13:37:01,623 INFO [qtp788117692-1916:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56258;soapId=4e5022a8;] soap - CreateDomainRequest elapsed=4
  2581.  
  2582. ==> /opt/zimbra/log/audit.log <==
  2583. 2018-09-06 13:37:01,602 INFO [qtp788117692-1959:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44572;soapId=4e5022a7;] security - cmd=AdminAuth; account=admin@zmc.com;
  2584. 2018-09-06 13:37:01,603 INFO [qtp788117692-1959:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44572;soapId=4e5022a7;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2585. 2018-09-06 13:37:01,623 INFO [qtp788117692-1916:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56258;soapId=4e5022a8;] security - cmd=CreateDomain; name=right.2721536241009240; zimbraDomainStatus=active; zimbraSMIMELdapAttribute=; zimbraSMIMELdapSearchBase=; zimbraDomainType=local; zimbraSMIMELdapStartTlsEnabled=; zimbraSMIMELdapURL=; zimbraSMIMELdapFilter=; zimbraSMIMELdapBindDn=; zimbraSMIMELdapBindPassword=;
  2586.  
  2587. ==> /opt/zimbra/log/mailbox.log <==
  2588. 2018-09-06 13:37:01,573 INFO [qtp788117692-1803:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44568;soapId=4faf0d7c;] soap - GrantRightRequest elapsed=3
  2589. 2018-09-06 13:37:01,590 INFO [qtp788117692-1805:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56254;soapId=4faf0d7d;] soap - CheckRightRequest elapsed=2
  2590. 2018-09-06 13:37:01,641 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44576;soapId=4faf0d7e;] soap - CreateAccountRequest elapsed=4
  2591.  
  2592. ==> /opt/zimbra/log/audit.log <==
  2593. 2018-09-06 13:37:01,640 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44576;soapId=4faf0d7e;] security - cmd=CreateAccount; name=admin1.2781536241009240@right.2721536241009240; zimbraIsDelegatedAdminAccount=TRUE;
  2594.  
  2595. ==> /opt/zimbra/log/mailbox.log <==
  2596. 2018-09-06 13:37:04,678 INFO [qtp788117692-1903:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44584;soapId=4e5022a9;] soap - GetEffectiveRightsRequest elapsed=2
  2597. 2018-09-06 13:37:04,691 INFO [qtp788117692-1869:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56270;soapId=4e5022aa;] soap - AuthRequest elapsed=1
  2598.  
  2599. ==> /opt/zimbra/log/audit.log <==
  2600. 2018-09-06 13:37:04,690 INFO [qtp788117692-1869:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56270;soapId=4e5022aa;] security - cmd=AdminAuth; account=admin@zmc.com;
  2601. 2018-09-06 13:37:04,690 INFO [qtp788117692-1869:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56270;soapId=4e5022aa;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2602.  
  2603. ==> /opt/zimbra/log/mailbox.log <==
  2604. 2018-09-06 13:37:04,661 INFO [qtp788117692-1804:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56266;soapId=4faf0d7f;] soap - GrantRightRequest elapsed=2
  2605. 2018-09-06 13:37:04,709 INFO [qtp788117692-1743:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44588;soapId=4faf0d80;] soap - CreateDomainRequest elapsed=3
  2606. 2018-09-06 13:37:04,727 INFO [qtp788117692-1758:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56274;soapId=4faf0d81;] soap - CreateAccountRequest elapsed=3
  2607.  
  2608. ==> /opt/zimbra/log/audit.log <==
  2609. 2018-09-06 13:37:04,709 INFO [qtp788117692-1743:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44588;soapId=4faf0d80;] security - cmd=CreateDomain; name=right.2731536241009240; zimbraDomainStatus=active; zimbraSMIMELdapAttribute=; zimbraSMIMELdapSearchBase=; zimbraDomainType=local; zimbraSMIMELdapStartTlsEnabled=; zimbraSMIMELdapURL=; zimbraSMIMELdapFilter=; zimbraSMIMELdapBindDn=; zimbraSMIMELdapBindPassword=;
  2610. 2018-09-06 13:37:04,727 INFO [qtp788117692-1758:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56274;soapId=4faf0d81;] security - cmd=CreateAccount; name=admin1.2781536241009240@right.2731536241009240; zimbraIsDelegatedAdminAccount=TRUE;
  2611.  
  2612. ==> /opt/zimbra/log/mailbox.log <==
  2613. 2018-09-06 13:37:07,749 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44594;soapId=4e5022ab;] soap - GrantRightRequest elapsed=3
  2614. 2018-09-06 13:37:07,768 INFO [qtp788117692-1962:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56280;soapId=4e5022ac;] soap - GetAllEffectiveRightsRequest elapsed=4
  2615.  
  2616. ==> /opt/zimbra/log/mailbox.log <==
  2617. 2018-09-06 13:37:07,782 INFO [qtp788117692-1803:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44598;soapId=4faf0d82;] soap - AuthRequest elapsed=1
  2618. 2018-09-06 13:37:07,798 WARN [qtp788117692-1805:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56284;soapId=4faf0d83;] misc - no resource bundle for base name ZsMsgRights can be found, (locale=manageAccountArchives)
  2619. java.util.MissingResourceException: Can't find resource for bundle java.util.PropertyResourceBundle, key manageAccountArchives
  2620. at java.util.ResourceBundle.getObject(ResourceBundle.java:450)
  2621. at java.util.ResourceBundle.getString(ResourceBundle.java:407)
  2622. at com.zimbra.common.util.L10nUtil.getMessage(L10nUtil.java:371)
  2623. at com.zimbra.cs.account.accesscontrol.RightCommand.rightToXML(RightCommand.java:1567)
  2624. at com.zimbra.cs.service.admin.GetAllRights.handle(GetAllRights.java:48)
  2625. at com.zimbra.soap.SoapEngine.dispatchRequest(SoapEngine.java:656)
  2626. at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:501)
  2627. at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:288)
  2628. at com.zimbra.soap.SoapServlet.doWork(SoapServlet.java:303)
  2629. at com.zimbra.soap.SoapServlet.doPost(SoapServlet.java:213)
  2630. at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  2631. at com.zimbra.cs.servlet.ZimbraServlet.service(ZimbraServlet.java:211)
  2632. at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
  2633. at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:821)
  2634. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1685)
  2635. at com.zimbra.cs.servlet.CsrfFilter.doFilter(CsrfFilter.java:169)
  2636. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2637. at com.zimbra.cs.servlet.RequestStringFilter.doFilter(RequestStringFilter.java:54)
  2638. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2639. at com.zimbra.cs.servlet.SetHeaderFilter.doFilter(SetHeaderFilter.java:59)
  2640. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2641. at com.zimbra.cs.servlet.ETagHeaderFilter.doFilter(ETagHeaderFilter.java:47)
  2642. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2643. at com.zimbra.cs.servlet.ContextPathBasedThreadPoolBalancerFilter.doFilter(ContextPathBasedThreadPoolBalancerFilter.java:107)
  2644. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2645. at com.zimbra.cs.servlet.ZimbraQoSFilter.doFilter(ZimbraQoSFilter.java:116)
  2646. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2647. at com.zimbra.cs.servlet.ZimbraInvalidLoginFilter.doFilter(ZimbraInvalidLoginFilter.java:117)
  2648. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2649. at org.eclipse.jetty.servlets.DoSFilter.doFilterChain(DoSFilter.java:473)
  2650. at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:318)
  2651. at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:288)
  2652. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2653. at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
  2654. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
  2655. at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
  2656. at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
  2657. at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1158)
  2658. at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
  2659. at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
  2660. at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1090)
  2661. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
  2662. at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
  2663. at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:109)
  2664. at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119)
  2665. at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:318)
  2666. at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:437)
  2667. at org.eclipse.jetty.server.handler.DebugHandler.handle(DebugHandler.java:84)
  2668. at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119)
  2669. at org.eclipse.jetty.server.Server.handle(Server.java:517)
  2670. at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:306)
  2671. at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:242)
  2672. at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261)
  2673. at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
  2674. at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:192)
  2675. at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261)
  2676. at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
  2677. at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:75)
  2678. at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:213)
  2679. at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:147)
  2680. at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
  2681. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  2682. at java.lang.Thread.run(Thread.java:748)
  2683. 2018-09-06 13:37:07,798 WARN [qtp788117692-1805:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56284;soapId=4faf0d83;] misc - no resource bundle for base name ZsMsgRights can be found, (locale=manageCalendarResourceArchives)
  2684. java.util.MissingResourceException: Can't find resource for bundle java.util.PropertyResourceBundle, key manageCalendarResourceArchives
  2685. at java.util.ResourceBundle.getObject(ResourceBundle.java:450)
  2686. at java.util.ResourceBundle.getString(ResourceBundle.java:407)
  2687. at com.zimbra.common.util.L10nUtil.getMessage(L10nUtil.java:371)
  2688. at com.zimbra.cs.account.accesscontrol.RightCommand.rightToXML(RightCommand.java:1567)
  2689. at com.zimbra.cs.service.admin.GetAllRights.handle(GetAllRights.java:48)
  2690. at com.zimbra.soap.SoapEngine.dispatchRequest(SoapEngine.java:656)
  2691. at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:501)
  2692. at com.zimbra.soap.SoapEngine.dispatch(SoapEngine.java:288)
  2693. at com.zimbra.soap.SoapServlet.doWork(SoapServlet.java:303)
  2694. at com.zimbra.soap.SoapServlet.doPost(SoapServlet.java:213)
  2695. at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
  2696. at com.zimbra.cs.servlet.ZimbraServlet.service(ZimbraServlet.java:211)
  2697. at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
  2698. at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:821)
  2699. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1685)
  2700. at com.zimbra.cs.servlet.CsrfFilter.doFilter(CsrfFilter.java:169)
  2701. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2702. at com.zimbra.cs.servlet.RequestStringFilter.doFilter(RequestStringFilter.java:54)
  2703. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2704. at com.zimbra.cs.servlet.SetHeaderFilter.doFilter(SetHeaderFilter.java:59)
  2705. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2706. at com.zimbra.cs.servlet.ETagHeaderFilter.doFilter(ETagHeaderFilter.java:47)
  2707. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2708. at com.zimbra.cs.servlet.ContextPathBasedThreadPoolBalancerFilter.doFilter(ContextPathBasedThreadPoolBalancerFilter.java:107)
  2709. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2710. at com.zimbra.cs.servlet.ZimbraQoSFilter.doFilter(ZimbraQoSFilter.java:116)
  2711. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2712. at com.zimbra.cs.servlet.ZimbraInvalidLoginFilter.doFilter(ZimbraInvalidLoginFilter.java:117)
  2713. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2714. at org.eclipse.jetty.servlets.DoSFilter.doFilterChain(DoSFilter.java:473)
  2715. at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:318)
  2716. at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:288)
  2717. at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
  2718. at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
  2719. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
  2720. at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
  2721. at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
  2722. at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1158)
  2723. at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
  2724. at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
  2725. at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1090)
  2726. at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
  2727. at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
  2728. at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:109)
  2729. at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119)
  2730. at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:318)
  2731. at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:437)
  2732. at org.eclipse.jetty.server.handler.DebugHandler.handle(DebugHandler.java:84)
  2733. at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119)
  2734. at org.eclipse.jetty.server.Server.handle(Server.java:517)
  2735. at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:306)
  2736. at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:242)
  2737. at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261)
  2738. at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
  2739. at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:192)
  2740. at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261)
  2741. at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
  2742. at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:75)
  2743. at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:213)
  2744. at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:147)
  2745. at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
  2746. at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
  2747. at java.lang.Thread.run(Thread.java:748)
  2748. 2018-09-06 13:37:07,799 INFO [qtp788117692-1805:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56284;soapId=4faf0d83;] soap - GetAllRightsRequest elapsed=3
  2749. 2018-09-06 13:37:07,925 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44608;soapId=4faf0d84;] soap - CreateAccountRequest elapsed=4
  2750.  
  2751. ==> /opt/zimbra/log/audit.log <==
  2752. 2018-09-06 13:37:07,781 INFO [qtp788117692-1803:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44598;soapId=4faf0d82;] security - cmd=AdminAuth; account=admin@zmc.com;
  2753. 2018-09-06 13:37:07,782 INFO [qtp788117692-1803:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44598;soapId=4faf0d82;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2754. 2018-09-06 13:37:07,925 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44608;soapId=4faf0d84;] security - cmd=CreateAccount; name=searchdir.1536241027875283@domain1536241027875282.com;
  2755. 2018-09-06 13:37:07,890 INFO [qtp788117692-1959:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44604;soapId=4e5022ad;] soap - AuthRequest elapsed=0
  2756. 2018-09-06 13:37:07,907 INFO [qtp788117692-1916:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56290;soapId=4e5022ae;] soap - CreateDomainRequest elapsed=4
  2757.  
  2758. ==> /opt/zimbra/log/audit.log <==
  2759. 2018-09-06 13:37:07,890 INFO [qtp788117692-1959:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44604;soapId=4e5022ad;] security - cmd=AdminAuth; account=admin@zmc.com;
  2760. 2018-09-06 13:37:07,890 INFO [qtp788117692-1959:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44604;soapId=4e5022ad;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2761. 2018-09-06 13:37:07,907 INFO [qtp788117692-1916:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56290;soapId=4e5022ae;] security - cmd=CreateDomain; name=domain1536241027875282.com; zimbraDomainStatus=active; zimbraSMIMELdapAttribute=; zimbraSMIMELdapSearchBase=; zimbraDomainType=local; zimbraSMIMELdapStartTlsEnabled=; zimbraSMIMELdapURL=; zimbraSMIMELdapFilter=; zimbraSMIMELdapBindDn=; zimbraSMIMELdapBindPassword=;
  2762.  
  2763. ==> /opt/zimbra/log/mailbox.log <==
  2764. 2018-09-06 13:37:10,951 INFO [qtp788117692-1804:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56296;soapId=4faf0d85;] soap - SearchDirectoryRequest elapsed=7
  2765.  
  2766. ==> /opt/zimbra/log/mailbox.log <==
  2767. 2018-09-06 13:37:10,966 INFO [qtp788117692-1903:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44614;soapId=4e5022af;] soap - AuthRequest elapsed=1
  2768. 2018-09-06 13:37:10,986 WARN [qtp788117692-1869:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56300;soapId=4e5022b0;] event - Shutdown called for Event Notifier Executor; initiating shutdown sequence...
  2769. 2018-09-06 13:37:10,986 INFO [EventLogger-Worker-Thread-6] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@36d63041
  2770. 2018-09-06 13:37:10,986 INFO [EventLogger-Worker-Thread-7] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@3dffb1a4
  2771. 2018-09-06 13:37:10,986 INFO [EventLogger-Worker-Thread-3] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@458bf9e6
  2772. 2018-09-06 13:37:10,986 INFO [EventLogger-Worker-Thread-5] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@81609e7
  2773. 2018-09-06 13:37:10,986 INFO [EventLogger-Worker-Thread-8] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@6cb20066
  2774. 2018-09-06 13:37:10,986 INFO [EventLogger-Worker-Thread-1] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@1fb87ecf
  2775. 2018-09-06 13:37:10,986 INFO [EventLogger-Worker-Thread-4] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@4b1115bf
  2776. 2018-09-06 13:37:10,986 INFO [EventLogger-Worker-Thread-2] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@476cc10b
  2777. 2018-09-06 13:37:10,986 INFO [EventLogger-Worker-Thread-9] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@3edfd6cf
  2778. 2018-09-06 13:37:10,986 INFO [EventLogger-Worker-Thread-0] [] ZooKeeper - Initiating client connection, connectString=zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.SolrZkClient$1@5a180483
  2779. 2018-09-06 13:37:10,988 INFO [EventLogger-Worker-Thread-6-SendThread(65e3c8013cdc.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server 65e3c8013cdc.zm-docker_default/10.0.0.11:2181. Will not attempt to authenticate using SASL (unknown error)
  2780. 2018-09-06 13:37:10,988 INFO [EventLogger-Worker-Thread-6] [] ZooKeeper - Session: 0x0 closed
  2781. 2018-09-06 13:37:10,988 INFO [EventLogger-Worker-Thread-6-EventThread] [] ClientCnxn - EventThread shut down
  2782. 2018-09-06 13:37:10,988 ERROR [EventLogger-Worker-Thread-6] [] event - unable to send 1 events to Solr backend
  2783. com.zimbra.common.service.ServiceException: system failure: unexpected error running command com.zimbra.cs.index.solr.SolrUtils$4
  2784. ExceptionId:EventLogger-Worker-Thread-6:1536241030988:a4a5f9c6e5824c5f
  2785. Code:service.FAILURE
  2786. at com.zimbra.common.service.ServiceException.FAILURE(ServiceException.java:288)
  2787. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:39)
  2788. at com.zimbra.cs.index.solr.SolrUtils.executeCloudRequestWithRetry(SolrUtils.java:320)
  2789. at com.zimbra.cs.index.solr.SolrCloudHelper.executeUpdateRequest(SolrCloudHelper.java:42)
  2790. at com.zimbra.cs.index.solr.SolrRequestHelper.executeUpdate(SolrRequestHelper.java:48)
  2791. at com.zimbra.cs.event.logger.SolrEventCallback.execute(SolrEventCallback.java:34)
  2792. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.send(BatchingEventLogger.java:126)
  2793. at com.zimbra.cs.event.logger.BatchingEventLogger$BatchedEvents.access$000(BatchingEventLogger.java:103)
  2794. at com.zimbra.cs.event.logger.BatchingEventLogger$1.onRemoval(BatchingEventLogger.java:53)
  2795. at com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:2013)
  2796. at com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3511)
  2797. at com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3487)
  2798. at com.google.common.cache.LocalCache$Segment.clear(LocalCache.java:3292)
  2799. at com.google.common.cache.LocalCache.clear(LocalCache.java:4270)
  2800. at com.google.common.cache.LocalCache$LocalManualCache.invalidateAll(LocalCache.java:4829)
  2801. at com.zimbra.cs.event.logger.BatchingEventLogger.sendAllBatched(BatchingEventLogger.java:136)
  2802. at com.zimbra.cs.event.logger.BatchingEventLogger.shutdown(BatchingEventLogger.java:79)
  2803. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.shutdownEventLogHandlers(EventLogger.java:244)
  2804. at com.zimbra.cs.event.logger.EventLogger$EventNotifier.run(EventLogger.java:215)
  2805. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  2806. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  2807. at java.lang.Thread.run(Thread.java:748)
  2808. Caused by: org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 within 10000 ms
  2809. at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:182)
  2810. at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:119)
  2811. at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:109)
  2812. at org.apache.solr.common.cloud.ZkStateReader.<init>(ZkStateReader.java:277)
  2813. at org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:155)
  2814. at org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:397)
  2815. at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:826)
  2816. at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
  2817. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
  2818. at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
  2819. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:299)
  2820. at com.zimbra.cs.index.solr.SolrUtils$4.execute(SolrUtils.java:295)
  2821. at com.zimbra.cs.util.RetryUtil$RequestWithRetry.execute(RetryUtil.java:22)
  2822. ... 20 more
  2823. Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 within 10000 ms
  2824. at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:232)
  2825. at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:174)
  2826. ... 32 more
  2827. 2018-09-06 13:37:10,991 INFO [EventLogger-Worker-Thread-9-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181. Will not attempt to authenticate using SASL (unknown error)
  2828. 2018-09-06 13:37:10,991 INFO [EventLogger-Worker-Thread-3-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181. Will not attempt to authenticate using SASL (unknown error)
  2829. 2018-09-06 13:37:10,992 INFO [EventLogger-Worker-Thread-7-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181. Will not attempt to authenticate using SASL (unknown error)
  2830. 2018-09-06 13:37:10,991 INFO [EventLogger-Worker-Thread-6-SendThread(65e3c8013cdc.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to 65e3c8013cdc.zm-docker_default/10.0.0.11:2181, initiating session
  2831. 2018-09-06 13:37:10,991 INFO [EventLogger-Worker-Thread-1-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181. Will not attempt to authenticate using SASL (unknown error)
  2832. 2018-09-06 13:37:10,992 INFO [EventLogger-Worker-Thread-3-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, initiating session
  2833. 2018-09-06 13:37:10,992 INFO [EventLogger-Worker-Thread-9-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181, initiating session
  2834. 2018-09-06 13:37:10,991 INFO [EventLogger-Worker-Thread-0-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181. Will not attempt to authenticate using SASL (unknown error)
  2835. 2018-09-06 13:37:10,992 INFO [EventLogger-Worker-Thread-7-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181, initiating session
  2836. 2018-09-06 13:37:10,992 INFO [EventLogger-Worker-Thread-5-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181. Will not attempt to authenticate using SASL (unknown error)
  2837. 2018-09-06 13:37:10,993 INFO [EventLogger-Worker-Thread-2-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181. Will not attempt to authenticate using SASL (unknown error)
  2838. 2018-09-06 13:37:10,992 INFO [EventLogger-Worker-Thread-8-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181. Will not attempt to authenticate using SASL (unknown error)
  2839. 2018-09-06 13:37:10,992 INFO [EventLogger-Worker-Thread-4-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Opening socket connection to server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181. Will not attempt to authenticate using SASL (unknown error)
  2840. 2018-09-06 13:37:10,992 INFO [EventLogger-Worker-Thread-1-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, initiating session
  2841. 2018-09-06 13:37:10,993 INFO [EventLogger-Worker-Thread-5-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, initiating session
  2842. 2018-09-06 13:37:10,993 INFO [EventLogger-Worker-Thread-2-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, initiating session
  2843. 2018-09-06 13:37:10,993 INFO [EventLogger-Worker-Thread-0-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181, initiating session
  2844. 2018-09-06 13:37:10,993 INFO [EventLogger-Worker-Thread-4-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, initiating session
  2845. 2018-09-06 13:37:10,994 INFO [EventLogger-Worker-Thread-8-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Socket connection established to zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, initiating session
  2846. 2018-09-06 13:37:10,994 INFO [EventLogger-Worker-Thread-3-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, sessionid = 0x20000d48aa8002f, negotiated timeout = 10000
  2847. 2018-09-06 13:37:10,994 INFO [EventLogger-Worker-Thread-9-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181, sessionid = 0x30000d48aad0024, negotiated timeout = 10000
  2848. 2018-09-06 13:37:10,995 INFO [EventLogger-Worker-Thread-7-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181, sessionid = 0x30000d48aad0025, negotiated timeout = 10000
  2849. 2018-09-06 13:37:10,995 INFO [EventLogger-Worker-Thread-5-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, sessionid = 0x10000d48b75002f, negotiated timeout = 10000
  2850. 2018-09-06 13:37:10,995 INFO [zkConnectionManagerCallback-440-thread-1] [] ConnectionManager - zkClient has connected
  2851. 2018-09-06 13:37:10,995 INFO [zkConnectionManagerCallback-428-thread-1] [] ConnectionManager - zkClient has connected
  2852. 2018-09-06 13:37:10,996 INFO [EventLogger-Worker-Thread-4-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, sessionid = 0x20000d48aa80031, negotiated timeout = 10000
  2853. 2018-09-06 13:37:10,996 INFO [zkConnectionManagerCallback-442-thread-1] [] ConnectionManager - zkClient has connected
  2854. 2018-09-06 13:37:10,996 INFO [EventLogger-Worker-Thread-2-SendThread(zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper1.1.o3oydvx8pmpe2mvm9528ytquy.zm-docker_default/10.0.0.17:2181, sessionid = 0x10000d48b750030, negotiated timeout = 10000
  2855. 2018-09-06 13:37:10,997 INFO [zkConnectionManagerCallback-435-thread-1] [] ConnectionManager - zkClient has connected
  2856. 2018-09-06 13:37:10,996 INFO [EventLogger-Worker-Thread-1-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, sessionid = 0x20000d48aa80030, negotiated timeout = 10000
  2857. 2018-09-06 13:37:10,997 INFO [zkConnectionManagerCallback-436-thread-1] [] ConnectionManager - zkClient has connected
  2858. 2018-09-06 13:37:10,998 INFO [zkConnectionManagerCallback-437-thread-1] [] ConnectionManager - zkClient has connected
  2859. 2018-09-06 13:37:10,998 INFO [zkConnectionManagerCallback-425-thread-1] [] ConnectionManager - zkClient has connected
  2860. 2018-09-06 13:37:10,998 INFO [EventLogger-Worker-Thread-8-SendThread(zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper2.1.70qp7xtekqofebjr45uvj8wt8.zm-docker_default/10.0.0.7:2181, sessionid = 0x20000d48aa80032, negotiated timeout = 10000
  2861. 2018-09-06 13:37:10,998 INFO [EventLogger-Worker-Thread-0-SendThread(zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default:2181)] [] ClientCnxn - Session establishment complete on server zm-docker_zmc-zookeeper3.1.v2dvkuhcb9o72himbtv870oew.zm-docker_default/10.0.0.11:2181, sessionid = 0x30000d48aad0026, negotiated timeout = 10000
  2862. 2018-09-06 13:37:11,000 INFO [EventLogger-Worker-Thread-1] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  2863. 2018-09-06 13:37:11,000 INFO [EventLogger-Worker-Thread-4] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  2864. 2018-09-06 13:37:11,000 INFO [EventLogger-Worker-Thread-9] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  2865. 2018-09-06 13:37:11,000 INFO [EventLogger-Worker-Thread-5] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  2866. 2018-09-06 13:37:11,000 INFO [EventLogger-Worker-Thread-2] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  2867. 2018-09-06 13:37:11,000 INFO [EventLogger-Worker-Thread-7] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  2868. 2018-09-06 13:37:11,002 INFO [zkConnectionManagerCallback-434-thread-1] [] ConnectionManager - zkClient has connected
  2869. 2018-09-06 13:37:11,000 INFO [EventLogger-Worker-Thread-3] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  2870. 2018-09-06 13:37:10,999 INFO [zkConnectionManagerCallback-426-thread-1] [] ConnectionManager - zkClient has connected
  2871. 2018-09-06 13:37:11,004 INFO [EventLogger-Worker-Thread-0] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  2872. 2018-09-06 13:37:11,005 INFO [EventLogger-Worker-Thread-2] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  2873. 2018-09-06 13:37:11,005 INFO [EventLogger-Worker-Thread-1] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  2874. 2018-09-06 13:37:11,005 INFO [EventLogger-Worker-Thread-5] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  2875. 2018-09-06 13:37:11,005 INFO [EventLogger-Worker-Thread-9] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  2876. 2018-09-06 13:37:11,005 INFO [EventLogger-Worker-Thread-7] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  2877. 2018-09-06 13:37:11,005 INFO [EventLogger-Worker-Thread-4] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  2878. 2018-09-06 13:37:11,006 INFO [EventLogger-Worker-Thread-3] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  2879. 2018-09-06 13:37:11,006 INFO [EventLogger-Worker-Thread-8] [] ZkStateReader - Updated live nodes from ZooKeeper... (0) -> (4)
  2880. 2018-09-06 13:37:11,006 INFO [EventLogger-Worker-Thread-0] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  2881. 2018-09-06 13:37:11,006 INFO [EventLogger-Worker-Thread-8] [] ZkClientClusterStateProvider - Cluster at zmc-zookeeper1:2181,zmc-zookeeper2:2181,zmc-zookeeper3:2181 ready
  2882. 2018-09-06 13:37:11,017 INFO [EventLogger-Worker-Thread-2] [] ZooKeeper - Session: 0x10000d48b750030 closed
  2883. 2018-09-06 13:37:11,017 INFO [EventLogger-Worker-Thread-2-EventThread] [] ClientCnxn - EventThread shut down
  2884. 2018-09-06 13:37:11,019 INFO [EventLogger-Worker-Thread-9] [] ZooKeeper - Session: 0x30000d48aad0024 closed
  2885. 2018-09-06 13:37:11,019 INFO [EventLogger-Worker-Thread-9-EventThread] [] ClientCnxn - EventThread shut down
  2886. 2018-09-06 13:37:11,020 INFO [EventLogger-Worker-Thread-3] [] ZooKeeper - Session: 0x20000d48aa8002f closed
  2887. 2018-09-06 13:37:11,020 INFO [EventLogger-Worker-Thread-3-EventThread] [] ClientCnxn - EventThread shut down
  2888. 2018-09-06 13:37:11,021 INFO [EventLogger-Worker-Thread-1] [] ZooKeeper - Session: 0x20000d48aa80030 closed
  2889. 2018-09-06 13:37:11,021 INFO [EventLogger-Worker-Thread-8] [] ZooKeeper - Session: 0x20000d48aa80032 closed
  2890. 2018-09-06 13:37:11,021 INFO [EventLogger-Worker-Thread-1-EventThread] [] ClientCnxn - EventThread shut down
  2891. 2018-09-06 13:37:11,021 INFO [EventLogger-Worker-Thread-8-EventThread] [] ClientCnxn - EventThread shut down
  2892. 2018-09-06 13:37:11,022 INFO [EventLogger-Worker-Thread-4] [] ZooKeeper - Session: 0x20000d48aa80031 closed
  2893. 2018-09-06 13:37:11,022 INFO [EventLogger-Worker-Thread-4-EventThread] [] ClientCnxn - EventThread shut down
  2894. 2018-09-06 13:37:11,023 INFO [EventLogger-Worker-Thread-0] [] ZooKeeper - Session: 0x30000d48aad0026 closed
  2895. 2018-09-06 13:37:11,023 INFO [EventLogger-Worker-Thread-0-EventThread] [] ClientCnxn - EventThread shut down
  2896. 2018-09-06 13:37:11,026 INFO [EventLogger-Worker-Thread-5] [] ZooKeeper - Session: 0x10000d48b75002f closed
  2897. 2018-09-06 13:37:11,026 INFO [EventLogger-Worker-Thread-5-EventThread] [] ClientCnxn - EventThread shut down
  2898. 2018-09-06 13:37:11,028 INFO [EventLogger-Worker-Thread-7] [] ZooKeeper - Session: 0x30000d48aad0025 closed
  2899. 2018-09-06 13:37:11,028 INFO [EventLogger-Worker-Thread-7-EventThread] [] ClientCnxn - EventThread shut down
  2900. 2018-09-06 13:37:11,029 INFO [qtp788117692-1869:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56300;soapId=4e5022b0;] event - Event Notifier Executor shutdown was successful
  2901. 2018-09-06 13:37:11,029 INFO [qtp788117692-1869:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56300;soapId=4e5022b0;] event - Event Queue Size 0
  2902. 2018-09-06 13:37:11,029 INFO [qtp788117692-1869:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56300;soapId=4e5022b0;] event - Starting Event Notifier Logger with 10 threads; initial event queue size is 0
  2903. 2018-09-06 13:37:11,031 INFO [qtp788117692-1869:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56300;soapId=4e5022b0;] soap - CreateCosRequest elapsed=53
  2904.  
  2905. ==> /opt/zimbra/log/audit.log <==
  2906. 2018-09-06 13:37:10,965 INFO [qtp788117692-1903:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44614;soapId=4e5022af;] security - cmd=AdminAuth; account=admin@zmc.com;
  2907. 2018-09-06 13:37:10,965 INFO [qtp788117692-1903:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44614;soapId=4e5022af;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2908. 2018-09-06 13:37:11,030 INFO [qtp788117692-1869:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56300;soapId=4e5022b0;] security - cmd=CreateCos; name=cos1536241030967287; zimbraBatchedIndexingSize=0;
  2909. 2018-09-06 13:37:11,060 INFO [qtp788117692-1802:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44666;soapId=4faf0d86;] soap - CreateAccountRequest elapsed=5
  2910.  
  2911. ==> /opt/zimbra/log/audit.log <==
  2912. 2018-09-06 13:37:11,059 INFO [qtp788117692-1802:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44666;soapId=4faf0d86;] security - cmd=CreateAccount; name=acct284.1536241030954@zmc.com; zimbraCOSId=66aaa6b4-2945-4ba9-b7b5-35cefbae84a3;
  2913.  
  2914. ==> /opt/zimbra/log/mailbox.log <==
  2915. 2018-09-06 13:37:14,158 INFO [LmtpServer-5] [ip=10.0.0.37;] mailbox - Creating mailbox with id 182 and group id 82 for acct284.1536241030954@zmc.com.
  2916. 2018-09-06 13:37:14,161 INFO [LmtpServer-5] [ip=10.0.0.37;] cache - initializing folder and tag caches for mailbox 182
  2917. 2018-09-06 13:37:14,374 INFO [LmtpServer-5] [ip=10.0.0.37;] mailbox - added account 574946b2-f060-4b15-9405-de79d746b20b to Redis notification channel NOTIFICATION-CHANNEL-82
  2918. 2018-09-06 13:37:14,374 INFO [redisson-3-32] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=1, channel=NOTIFICATION-CHANNEL-82
  2919. 2018-09-06 13:37:14,375 INFO [LmtpServer-5] [ip=10.0.0.37;] mbxmgr - Mailbox 182 account 574946b2-f060-4b15-9405-de79d746b20b CREATED
  2920. 2018-09-06 13:37:14,375 INFO [LmtpServer-5] [ip=10.0.0.37;] lmtp - Delivering message: size=unspecified, nrcpts=1, sender=foo@foo.com, msgid=
  2921. 2018-09-06 13:37:14,607 INFO [redisson-3-32] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=2, channel=NOTIFICATION-CHANNEL-82
  2922. 2018-09-06 13:37:14,757 INFO [redisson-3-32] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=3, channel=NOTIFICATION-CHANNEL-82
  2923. 2018-09-06 13:37:14,764 INFO [LmtpServer-5] [ip=10.0.0.37;] lmtp - Delivering message: size=unspecified, nrcpts=1, sender=foo@foo.com, msgid=
  2924.  
  2925. ==> /opt/zimbra/log/mailbox.log <==
  2926. 2018-09-06 13:37:14,546 INFO [LmtpServer-5] [ip=10.0.0.37;] mbxmgr - Mailbox 182 account 574946b2-f060-4b15-9405-de79d746b20b LOADED
  2927. 2018-09-06 13:37:14,547 INFO [LmtpServer-5] [ip=10.0.0.37;] lmtp - Delivering message: size=unspecified, nrcpts=1, sender=foo@foo.com, msgid=
  2928. 2018-09-06 13:37:14,549 INFO [LmtpServer-5] [name=acct284.1536241030954@zmc.com;mid=182;ip=10.0.0.37;] cache - initializing folder and tag caches for mailbox 182
  2929. 2018-09-06 13:37:14,751 INFO [notifyMboxListeners-1401] [] mailbox - added account 574946b2-f060-4b15-9405-de79d746b20b to Redis notification channel NOTIFICATION-CHANNEL-82
  2930. 2018-09-06 13:37:14,751 INFO [redisson-3-32] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=3, channel=NOTIFICATION-CHANNEL-82
  2931. 2018-09-06 13:37:14,912 INFO [redisson-3-28] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=4, channel=NOTIFICATION-CHANNEL-82
  2932. 2018-09-06 13:37:14,924 INFO [LmtpServer-5] [ip=10.0.0.37;] lmtp - Delivering message: size=unspecified, nrcpts=1, sender=foo@foo.com, msgid=
  2933. 2018-09-06 13:37:14,912 INFO [redisson-3-14] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=4, channel=NOTIFICATION-CHANNEL-82
  2934. 2018-09-06 13:37:15,078 INFO [redisson-3-2] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=5, channel=NOTIFICATION-CHANNEL-82
  2935. 2018-09-06 13:37:15,164 INFO [LmtpServer-5] [ip=10.0.0.37;] lmtp - Delivering message: size=unspecified, nrcpts=1, sender=foo@foo.com, msgid=
  2936. 2018-09-06 13:37:15,328 INFO [redisson-3-1] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=6, channel=NOTIFICATION-CHANNEL-82
  2937. 2018-09-06 13:37:15,469 INFO [Timer-Zimbra] [] FileUploadServlet - 2 pending file uploads
  2938. 2018-09-06 13:37:15,483 INFO [redisson-3-11] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=7, channel=NOTIFICATION-CHANNEL-82
  2939. 2018-09-06 13:37:15,493 INFO [LmtpServer-5] [ip=10.0.0.37;] lmtp - Delivering message: size=unspecified, nrcpts=1, sender=foo@foo.com, msgid=
  2940. 2018-09-06 13:37:15,643 INFO [redisson-3-28] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=8, channel=NOTIFICATION-CHANNEL-82
  2941. 2018-09-06 13:37:15,802 INFO [redisson-3-13] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=9, channel=NOTIFICATION-CHANNEL-82
  2942. 2018-09-06 13:37:15,078 INFO [redisson-3-24] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=5, channel=NOTIFICATION-CHANNEL-82
  2943. 2018-09-06 13:37:15,328 INFO [redisson-3-15] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=6, channel=NOTIFICATION-CHANNEL-82
  2944. 2018-09-06 13:37:15,339 INFO [LmtpServer-5] [ip=10.0.0.37;] lmtp - Delivering message: size=unspecified, nrcpts=1, sender=foo@foo.com, msgid=
  2945. 2018-09-06 13:37:15,483 INFO [redisson-3-5] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=7, channel=NOTIFICATION-CHANNEL-82
  2946. 2018-09-06 13:37:15,643 INFO [redisson-3-21] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=8, channel=NOTIFICATION-CHANNEL-82
  2947. 2018-09-06 13:37:15,655 INFO [LmtpServer-5] [ip=10.0.0.37;] lmtp - Delivering message: size=unspecified, nrcpts=1, sender=foo@foo.com, msgid=
  2948. 2018-09-06 13:37:15,802 INFO [redisson-3-8] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=9, channel=NOTIFICATION-CHANNEL-82
  2949. 2018-09-06 13:37:15,968 INFO [redisson-3-10] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=10, channel=NOTIFICATION-CHANNEL-82
  2950. 2018-09-06 13:37:15,979 INFO [LmtpServer-5] [ip=10.0.0.37;] lmtp - Delivering message: size=unspecified, nrcpts=1, sender=foo@foo.com, msgid=
  2951. 2018-09-06 13:37:15,814 INFO [LmtpServer-5] [ip=10.0.0.37;] lmtp - Delivering message: size=unspecified, nrcpts=1, sender=foo@foo.com, msgid=
  2952. 2018-09-06 13:37:15,968 INFO [redisson-3-8] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=10, channel=NOTIFICATION-CHANNEL-82
  2953. 2018-09-06 13:37:16,134 INFO [redisson-3-26] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=11, channel=NOTIFICATION-CHANNEL-82
  2954. 2018-09-06 13:37:16,146 INFO [LmtpServer-5] [ip=10.0.0.37;] lmtp - Delivering message: size=unspecified, nrcpts=1, sender=foo@foo.com, msgid=
  2955. 2018-09-06 13:37:16,294 INFO [redisson-3-24] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=12, channel=NOTIFICATION-CHANNEL-82
  2956. 2018-09-06 13:37:16,452 INFO [redisson-3-22] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=13, channel=NOTIFICATION-CHANNEL-82
  2957. 2018-09-06 13:37:16,464 INFO [LmtpServer-5] [ip=10.0.0.37;] lmtp - Delivering message: size=unspecified, nrcpts=1, sender=foo@foo.com, msgid=
  2958. 2018-09-06 13:37:16,619 INFO [redisson-3-31] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=14, channel=NOTIFICATION-CHANNEL-82
  2959. 2018-09-06 13:37:16,726 INFO [qtp788117692-1906:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56390;soapId=4e5022b1;] soap - AuthRequest elapsed=1
  2960. 2018-09-06 13:37:16,739 INFO [qtp788117692-1962:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44708;soapId=4e5022b2;] soap - GetSessionsRequest elapsed=0
  2961. 2018-09-06 13:37:16,777 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56398;soapId=4e5022b3;] soap - AuthRequest elapsed=1
  2962. 2018-09-06 13:37:16,794 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44716;soapId=4e5022b4;] soap - CreateAccountRequest elapsed=4
  2963.  
  2964. ==> /opt/zimbra/log/audit.log <==
  2965. 2018-09-06 13:37:16,725 INFO [qtp788117692-1906:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56390;soapId=4e5022b1;] security - cmd=AdminAuth; account=admin@zmc.com;
  2966. 2018-09-06 13:37:16,726 INFO [qtp788117692-1906:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56390;soapId=4e5022b1;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2967. 2018-09-06 13:37:16,776 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56398;soapId=4e5022b3;] security - cmd=AdminAuth; account=admin@zmc.com;
  2968. 2018-09-06 13:37:16,776 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56398;soapId=4e5022b3;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2969. 2018-09-06 13:37:16,793 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44716;soapId=4e5022b4;] security - cmd=CreateAccount; name=test1536241036680288@zmc.com;
  2970. 2018-09-06 13:37:16,134 INFO [redisson-3-13] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=11, channel=NOTIFICATION-CHANNEL-82
  2971. 2018-09-06 13:37:16,294 INFO [redisson-3-5] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=12, channel=NOTIFICATION-CHANNEL-82
  2972. 2018-09-06 13:37:16,305 INFO [LmtpServer-5] [ip=10.0.0.37;] lmtp - Delivering message: size=unspecified, nrcpts=1, sender=foo@foo.com, msgid=
  2973. 2018-09-06 13:37:16,452 INFO [redisson-3-21] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=13, channel=NOTIFICATION-CHANNEL-82
  2974. 2018-09-06 13:37:16,619 INFO [redisson-3-3] [] mailbox - handling notification. accountId=574946b2-f060-4b15-9405-de79d746b20b, changeId=14, channel=NOTIFICATION-CHANNEL-82
  2975. 2018-09-06 13:37:16,643 INFO [qtp788117692-1758:https:https://zmc-proxy/service/soap/] [name=acct284.1536241030954@zmc.com;ip=10.0.0.20;port=33922;soapId=4faf0d87;] soap - AuthRequest elapsed=1
  2976. 2018-09-06 13:37:16,677 INFO [qtp788117692-1743:https:https://zmc-proxy/service/soap/] [name=acct284.1536241030954@zmc.com;mid=182;ip=10.0.0.19;port=43454;soapId=4faf0d88;] soap - BrowseRequest elapsed=11
  2977. 2018-09-06 13:37:16,752 INFO [qtp788117692-1742:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56394;soapId=4faf0d89;] soap - AuthRequest elapsed=1
  2978. 2018-09-06 13:37:16,765 INFO [qtp788117692-1803:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44712;soapId=4faf0d8a;] soap - DumpSessionsRequest elapsed=0
  2979.  
  2980. ==> /opt/zimbra/log/audit.log <==
  2981. 2018-09-06 13:37:16,643 INFO [qtp788117692-1758:https:https://zmc-proxy/service/soap/] [name=acct284.1536241030954@zmc.com;ip=10.0.0.20;port=33922;soapId=4faf0d87;] security - cmd=Auth; account=acct284.1536241030954@zmc.com; protocol=soap;
  2982. 2018-09-06 13:37:16,751 INFO [qtp788117692-1742:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56394;soapId=4faf0d89;] security - cmd=AdminAuth; account=admin@zmc.com;
  2983. 2018-09-06 13:37:16,752 INFO [qtp788117692-1742:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56394;soapId=4faf0d89;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  2984.  
  2985. ==> /opt/zimbra/log/mailbox.log <==
  2986. 2018-09-06 13:37:19,812 INFO [qtp788117692-1805:https:https://zmc-proxy/service/soap/] [name=test1536241036680288@zmc.com;ip=10.0.0.20;port=33942;soapId=4faf0d8b;] soap - AuthRequest elapsed=1
  2987. 2018-09-06 13:37:19,865 INFO [qtp788117692-1755:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44730;soapId=4faf0d8c;] soap - CreateAccountRequest elapsed=4
  2988.  
  2989. ==> /opt/zimbra/log/audit.log <==
  2990. 2018-09-06 13:37:19,812 INFO [qtp788117692-1805:https:https://zmc-proxy/service/soap/] [name=test1536241036680288@zmc.com;ip=10.0.0.20;port=33942;soapId=4faf0d8b;] security - cmd=Auth; account=test1536241036680288@zmc.com; protocol=soap;
  2991. 2018-09-06 13:37:19,864 INFO [qtp788117692-1755:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44730;soapId=4faf0d8c;] security - cmd=CreateAccount; name=right1536241039836290@zmc.com;
  2992.  
  2993. ==> /opt/zimbra/log/mailbox.log <==
  2994. 2018-09-06 13:37:19,833 INFO [qtp788117692-1957:https:https://zmc-proxy/service/soap/] [name=test1536241036680288@zmc.com;ip=10.0.0.19;port=43476;soapId=4e5022b5;] soap - EndSessionRequest elapsed=0
  2995. 2018-09-06 13:37:19,848 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56412;soapId=4e5022b6;] soap - AuthRequest elapsed=1
  2996.  
  2997. ==> /opt/zimbra/log/audit.log <==
  2998. 2018-09-06 13:37:19,847 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56412;soapId=4e5022b6;] security - cmd=AdminAuth; account=admin@zmc.com;
  2999. 2018-09-06 13:37:19,847 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56412;soapId=4e5022b6;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  3000.  
  3001. ==> /opt/zimbra/log/mailbox.log <==
  3002. 2018-09-06 13:37:21,644 INFO [MailboxPurge] [name=test1536239411146.33@zmc.com;mid=67;] purge - Purging messages.
  3003. 2018-09-06 13:37:22,883 INFO [qtp788117692-1723:https:https://zmc-proxy/service/soap/] [name=right1536241039836290@zmc.com;ip=10.0.0.20;port=33956;soapId=4faf0d8d;] soap - AuthRequest elapsed=1
  3004. 2018-09-06 13:37:22,906 INFO [qtp788117692-1541:https:https://zmc-proxy/service/soap/] [name=right1536241039836290@zmc.com;ip=10.0.0.19;port=43488;soapId=4faf0d8e;] soap - CreateSignatureRequest elapsed=1
  3005.  
  3006. ==> /opt/zimbra/log/audit.log <==
  3007. 2018-09-06 13:37:22,883 INFO [qtp788117692-1723:https:https://zmc-proxy/service/soap/] [name=right1536241039836290@zmc.com;ip=10.0.0.20;port=33956;soapId=4faf0d8d;] security - cmd=Auth; account=right1536241039836290@zmc.com; protocol=soap;
  3008.  
  3009. ==> /opt/zimbra/log/mailbox.log <==
  3010. 2018-09-06 13:37:22,919 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56424;soapId=4e5022b7;] soap - AuthRequest elapsed=1
  3011. 2018-09-06 13:37:22,936 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44742;soapId=4e5022b8;] soap - CreateAccountRequest elapsed=5
  3012. 2018-09-06 13:37:23,223 INFO [MailboxPurge] [name=test.1536239511201.66@zmc.com;mid=72;] purge - Purging messages.
  3013.  
  3014. ==> /opt/zimbra/log/audit.log <==
  3015. 2018-09-06 13:37:22,918 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56424;soapId=4e5022b7;] security - cmd=AdminAuth; account=admin@zmc.com;
  3016. 2018-09-06 13:37:22,919 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56424;soapId=4e5022b7;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  3017. 2018-09-06 13:37:22,935 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44742;soapId=4e5022b8;] security - cmd=CreateAccount; name=right1536241039836291@zmc.com;
  3018.  
  3019. ==> /opt/zimbra/log/mailbox.log <==
  3020. 2018-09-06 13:37:25,989 INFO [qtp788117692-1812:https:https://zmc-proxy/service/soap/] [name=right1536241039836291@zmc.com;ip=10.0.0.20;port=33970;soapId=4faf0d8f;] soap - AuthRequest elapsed=1
  3021.  
  3022. ==> /opt/zimbra/log/audit.log <==
  3023. 2018-09-06 13:37:25,989 INFO [qtp788117692-1812:https:https://zmc-proxy/service/soap/] [name=right1536241039836291@zmc.com;ip=10.0.0.20;port=33970;soapId=4faf0d8f;] security - cmd=Auth; account=right1536241039836291@zmc.com; protocol=soap;
  3024.  
  3025. ==> /opt/zimbra/log/mailbox.log <==
  3026. 2018-09-06 13:37:26,013 INFO [qtp788117692-1957:https:https://zmc-proxy/service/soap/] [name=right1536241039836291@zmc.com;ip=10.0.0.19;port=43502;soapId=4e5022b9;] soap - CreateSignatureRequest elapsed=1
  3027. 2018-09-06 13:37:26,066 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56444;soapId=4e5022ba;] soap - CreateAccountRequest elapsed=4
  3028.  
  3029. ==> /opt/zimbra/log/audit.log <==
  3030. 2018-09-06 13:37:26,065 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56444;soapId=4e5022ba;] security - cmd=CreateAccount; name=right1536241039836289@zmc.com;
  3031.  
  3032. ==> /opt/zimbra/log/mailbox.log <==
  3033. 2018-09-06 13:37:26,036 INFO [qtp788117692-1740:https:https://zmc-proxy/service/soap/] [name=right1536241039836291@zmc.com;ip=10.0.0.20;port=33978;soapId=4faf0d90;] soap - ModifySignatureRequest elapsed=1
  3034. 2018-09-06 13:37:26,050 INFO [qtp788117692-1803:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44758;soapId=4faf0d91;] soap - AuthRequest elapsed=1
  3035.  
  3036. ==> /opt/zimbra/log/audit.log <==
  3037. 2018-09-06 13:37:26,049 INFO [qtp788117692-1803:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44758;soapId=4faf0d91;] security - cmd=AdminAuth; account=admin@zmc.com;
  3038. 2018-09-06 13:37:26,049 INFO [qtp788117692-1803:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44758;soapId=4faf0d91;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  3039.  
  3040. ==> /opt/zimbra/log/mailbox.log <==
  3041. 2018-09-06 13:37:29,118 INFO [qtp788117692-1957:https:https://zmc-proxy/service/soap/] [name=right1536241039836289@zmc.com;ip=10.0.0.19;port=43514;soapId=4e5022bb;] soap - AuthRequest elapsed=1
  3042. 2018-09-06 13:37:29,145 INFO [qtp788117692-2020:https:https://zmc-proxy/service/soap/] [name=right1536241039836289@zmc.com;ip=10.0.0.20;port=33990;soapId=4e5022bc;] soap - CreateSignatureRequest elapsed=2
  3043. 2018-09-06 13:37:29,166 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=right1536241039836289@zmc.com;ip=10.0.0.19;port=43522;soapId=4e5022bd;] soap - GetSignaturesRequest elapsed=1
  3044.  
  3045. ==> /opt/zimbra/log/audit.log <==
  3046. 2018-09-06 13:37:29,118 INFO [qtp788117692-1957:https:https://zmc-proxy/service/soap/] [name=right1536241039836289@zmc.com;ip=10.0.0.19;port=43514;soapId=4e5022bb;] security - cmd=Auth; account=right1536241039836289@zmc.com; protocol=soap;
  3047.  
  3048. ==> /opt/zimbra/log/mailbox.log <==
  3049. 2018-09-06 13:37:29,178 INFO [qtp788117692-1561:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56458;soapId=4faf0d92;] soap - AuthRequest elapsed=0
  3050. 2018-09-06 13:37:29,195 INFO [qtp788117692-1743:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44776;soapId=4faf0d93;] soap - CreateAccountRequest elapsed=4
  3051.  
  3052. ==> /opt/zimbra/log/audit.log <==
  3053. 2018-09-06 13:37:29,178 INFO [qtp788117692-1561:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56458;soapId=4faf0d92;] security - cmd=AdminAuth; account=admin@zmc.com;
  3054. 2018-09-06 13:37:29,178 INFO [qtp788117692-1561:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56458;soapId=4faf0d92;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  3055. 2018-09-06 13:37:29,194 INFO [qtp788117692-1743:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44776;soapId=4faf0d93;] security - cmd=CreateAccount; name=right1536241039836292@zmc.com;
  3056.  
  3057. ==> /opt/zimbra/log/mailbox.log <==
  3058. 2018-09-06 13:37:32,212 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=right1536241039836292@zmc.com;ip=10.0.0.20;port=34004;soapId=4e5022be;] soap - AuthRequest elapsed=1
  3059. 2018-09-06 13:37:32,258 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=right1536241039836292@zmc.com;ip=10.0.0.20;port=34012;soapId=4e5022bf;] soap - DeleteSignatureRequest elapsed=2
  3060. 2018-09-06 13:37:32,272 INFO [qtp788117692-1884:https:https://zmc-proxy:7071/service/admin/soap/] [ip=10.0.0.19;port=44792;soapId=4e5022c0;] SoapEngine - no valid authtoken present: cannot dispatch request
  3061. 2018-09-06 13:37:32,272 INFO [qtp788117692-1884:https:https://zmc-proxy:7071/service/admin/soap/] [ip=10.0.0.19;port=44792;soapId=4e5022c0;] soap - PingRequest elapsed=1
  3062.  
  3063. ==> /opt/zimbra/log/audit.log <==
  3064. 2018-09-06 13:37:32,212 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=right1536241039836292@zmc.com;ip=10.0.0.20;port=34004;soapId=4e5022be;] security - cmd=Auth; account=right1536241039836292@zmc.com; protocol=soap;
  3065.  
  3066. ==> /opt/zimbra/log/mailbox.log <==
  3067. 2018-09-06 13:37:32,236 INFO [qtp788117692-1742:https:https://zmc-proxy/service/soap/] [name=right1536241039836292@zmc.com;ip=10.0.0.19;port=43536;soapId=4faf0d94;] soap - CreateSignatureRequest elapsed=2
  3068. 2018-09-06 13:37:32,284 INFO [qtp788117692-1805:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56478;soapId=4faf0d95;] soap - AuthRequest elapsed=1
  3069. 2018-09-06 13:37:32,301 INFO [qtp788117692-1693:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44796;soapId=4faf0d96;] soap - CreateAccountRequest elapsed=4
  3070.  
  3071. ==> /opt/zimbra/log/audit.log <==
  3072. 2018-09-06 13:37:32,283 INFO [qtp788117692-1805:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56478;soapId=4faf0d95;] security - cmd=AdminAuth; account=admin@zmc.com;
  3073. 2018-09-06 13:37:32,284 INFO [qtp788117692-1805:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56478;soapId=4faf0d95;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  3074. 2018-09-06 13:37:32,300 INFO [qtp788117692-1693:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44796;soapId=4faf0d96;] security - cmd=CreateAccount; name=test1536241052261.302@zmc.com;
  3075.  
  3076. ==> /opt/zimbra/log/mailbox.log <==
  3077. 2018-09-06 13:37:35,354 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=test1536241052261.302@zmc.com;ip=10.0.0.20;port=34022;soapId=4e5022c1;] soap - AuthRequest elapsed=1
  3078. 2018-09-06 13:37:35,760 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=test1536241052261.302@zmc.com;ip=10.0.0.20;port=34030;] mbxmgr - Mailbox 183 account 61597766-b584-4cd5-b5c5-19509b9da1d4 LOADED
  3079. 2018-09-06 13:37:35,764 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=test1536241052261.302@zmc.com;mid=183;ip=10.0.0.20;port=34030;soapId=4e5022c2;] cache - initializing folder and tag caches for mailbox 183
  3080.  
  3081. ==> /opt/zimbra/log/audit.log <==
  3082. 2018-09-06 13:37:35,354 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=test1536241052261.302@zmc.com;ip=10.0.0.20;port=34022;soapId=4e5022c1;] security - cmd=Auth; account=test1536241052261.302@zmc.com; protocol=soap;
  3083.  
  3084. ==> /opt/zimbra/log/mailbox.log <==
  3085. 2018-09-06 13:37:35,494 INFO [qtp788117692-1740:https:https://zmc-proxy/service/soap/] [name=test1536241052261.302@zmc.com;ip=10.0.0.19;port=43554;soapId=4faf0d97;] mailbox - Creating mailbox with id 183 and group id 83 for test1536241052261.302@zmc.com.
  3086. 2018-09-06 13:37:35,497 INFO [qtp788117692-1740:https:https://zmc-proxy/service/soap/] [name=test1536241052261.302@zmc.com;ip=10.0.0.19;port=43554;soapId=4faf0d97;] cache - initializing folder and tag caches for mailbox 183
  3087. 2018-09-06 13:37:35,705 INFO [qtp788117692-1740:https:https://zmc-proxy/service/soap/] [name=test1536241052261.302@zmc.com;ip=10.0.0.19;port=43554;soapId=4faf0d97;] mailbox - added account 61597766-b584-4cd5-b5c5-19509b9da1d4 to Redis notification channel NOTIFICATION-CHANNEL-83
  3088. 2018-09-06 13:37:35,706 INFO [redisson-3-6] [] mailbox - handling notification. accountId=61597766-b584-4cd5-b5c5-19509b9da1d4, changeId=1, channel=NOTIFICATION-CHANNEL-83
  3089. 2018-09-06 13:37:35,707 INFO [qtp788117692-1740:https:https://zmc-proxy/service/soap/] [name=test1536241052261.302@zmc.com;ip=10.0.0.19;port=43554;soapId=4faf0d97;] mbxmgr - Mailbox 183 account 61597766-b584-4cd5-b5c5-19509b9da1d4 CREATED
  3090. 2018-09-06 13:37:35,733 INFO [qtp788117692-1740:https:https://zmc-proxy/service/soap/] [name=test1536241052261.302@zmc.com;mid=183;ip=10.0.0.19;port=43554;soapId=4faf0d97;] soap - GetFolderRequest elapsed=325
  3091. 2018-09-06 13:37:35,900 INFO [redisson-3-10] [] mailbox - handling notification. accountId=61597766-b584-4cd5-b5c5-19509b9da1d4, changeId=2, channel=NOTIFICATION-CHANNEL-83
  3092. 2018-09-06 13:37:35,967 INFO [redisson-3-15] [] mailbox - handling notification. accountId=61597766-b584-4cd5-b5c5-19509b9da1d4, changeId=3, channel=NOTIFICATION-CHANNEL-83
  3093. 2018-09-06 13:37:36,013 INFO [qtp788117692-1812:https:https://zmc-proxy/service/soap/] [name=test1536241052261.302@zmc.com;mid=183;ip=10.0.0.20;port=34036;soapId=4faf0d98;] soap - GetFolderRequest elapsed=27
  3094.  
  3095. ==> /opt/zimbra/log/mailbox.log <==
  3096. 2018-09-06 13:37:35,826 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=test1536241052261.302@zmc.com;mid=183;ip=10.0.0.20;port=34030;soapId=4e5022c2;] calendar - <CreateCalendarItem> folderId=10, subject="Subject of meeting1536241052261.303", UID=60b5b310-399d-4277-a238-8b371b21be95
  3097. 2018-09-06 13:37:35,832 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=test1536241052261.302@zmc.com;mid=183;ip=10.0.0.20;port=34030;soapId=4e5022c2;] calendar - Adding CalendarItem: id=258, Message-ID="<347980600.455.1536241055763.JavaMail.zimbra@zmc.com>", folderId=10, subject="Subject of meeting1536241052261.303", UID=60b5b310-399d-4277-a238-8b371b21be95
  3098. 2018-09-06 13:37:35,899 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=test1536241052261.302@zmc.com;mid=183;ip=10.0.0.20;port=34030;soapId=4e5022c2;] mailbox - added account 61597766-b584-4cd5-b5c5-19509b9da1d4 to Redis notification channel NOTIFICATION-CHANNEL-83
  3099. 2018-09-06 13:37:35,900 INFO [redisson-3-6] [] mailbox - handling notification. accountId=61597766-b584-4cd5-b5c5-19509b9da1d4, changeId=2, channel=NOTIFICATION-CHANNEL-83
  3100. 2018-09-06 13:37:35,900 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=test1536241052261.302@zmc.com;mid=183;ip=10.0.0.20;port=34030;soapId=4e5022c2;] soap - CreateAppointmentRequest elapsed=140
  3101. 2018-09-06 13:37:35,914 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=test1536241052261.302@zmc.com;mid=183;ip=10.0.0.19;port=43562;soapId=4e5022c3;] calendar - <SnoozeCalendarItemAlarm> id=61597766-b584-4cd5-b5c5-19509b9da1d4:258,until=1536241355902
  3102. 2018-09-06 13:37:35,967 INFO [redisson-3-7] [] mailbox - handling notification. accountId=61597766-b584-4cd5-b5c5-19509b9da1d4, changeId=3, channel=NOTIFICATION-CHANNEL-83
  3103. 2018-09-06 13:37:35,971 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=test1536241052261.302@zmc.com;mid=183;ip=10.0.0.19;port=43562;soapId=4e5022c3;] soap - SnoozeCalendarItemAlarmRequest elapsed=57
  3104. 2018-09-06 13:37:36,183 INFO [redisson-3-8] [] mailbox - handling notification. accountId=61597766-b584-4cd5-b5c5-19509b9da1d4, changeId=4, channel=NOTIFICATION-CHANNEL-83
  3105. 2018-09-06 13:37:36,197 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=test1536241052261.302@zmc.com;mid=183;ip=10.0.0.20;port=34040;soapId=4e5022c4;] calendar - <SnoozeCalendarItemAlarm> id=61597766-b584-4cd5-b5c5-19509b9da1d4:260,until=1536262656184
  3106. 2018-09-06 13:37:36,249 INFO [redisson-3-17] [] mailbox - handling notification. accountId=61597766-b584-4cd5-b5c5-19509b9da1d4, changeId=5, channel=NOTIFICATION-CHANNEL-83
  3107. 2018-09-06 13:37:36,253 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=test1536241052261.302@zmc.com;mid=183;ip=10.0.0.20;port=34040;soapId=4e5022c4;] soap - SnoozeCalendarItemAlarmRequest elapsed=56
  3108. 2018-09-06 13:37:36,269 INFO [qtp788117692-1884:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44820;soapId=4e5022c5;] soap - AuthRequest elapsed=1
  3109.  
  3110. ==> /opt/zimbra/log/audit.log <==
  3111. 2018-09-06 13:37:36,268 INFO [qtp788117692-1884:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44820;soapId=4e5022c5;] security - cmd=AdminAuth; account=admin@zmc.com;
  3112. 2018-09-06 13:37:36,268 INFO [qtp788117692-1884:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44820;soapId=4e5022c5;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  3113. 2018-09-06 13:37:36,036 INFO [qtp788117692-1541:https:https://zmc-proxy/service/soap/] [name=test1536241052261.302@zmc.com;mid=183;ip=10.0.0.19;port=43566;soapId=4faf0d99;] calendar - <CreateCalendarItem> folderId=15, subject="Subject of task1536241052261.306", UID=82a0ce55-d6aa-4eee-ba13-9fb69e3bd9b1
  3114. 2018-09-06 13:37:36,040 INFO [qtp788117692-1541:https:https://zmc-proxy/service/soap/] [name=test1536241052261.302@zmc.com;mid=183;ip=10.0.0.19;port=43566;soapId=4faf0d99;] calendar - Adding CalendarItem: id=260, Message-ID="<517732224.469.1536241056035.JavaMail.zimbra@zmc.com>", folderId=15, subject="Subject of task1536241052261.306", UID=82a0ce55-d6aa-4eee-ba13-9fb69e3bd9b1
  3115. 2018-09-06 13:37:36,182 INFO [qtp788117692-1541:https:https://zmc-proxy/service/soap/] [name=test1536241052261.302@zmc.com;mid=183;ip=10.0.0.19;port=43566;soapId=4faf0d99;] soap - CreateTaskRequest elapsed=152
  3116. 2018-09-06 13:37:36,183 INFO [redisson-3-20] [] mailbox - handling notification. accountId=61597766-b584-4cd5-b5c5-19509b9da1d4, changeId=4, channel=NOTIFICATION-CHANNEL-83
  3117. 2018-09-06 13:37:36,250 INFO [redisson-3-17] [] mailbox - handling notification. accountId=61597766-b584-4cd5-b5c5-19509b9da1d4, changeId=5, channel=NOTIFICATION-CHANNEL-83
  3118. 2018-09-06 13:37:36,323 INFO [qtp788117692-1758:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56506;soapId=4faf0d9a;] soap - CreateAccountRequest elapsed=3
  3119.  
  3120. ==> /opt/zimbra/log/audit.log <==
  3121. 2018-09-06 13:37:36,322 INFO [qtp788117692-1758:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56506;soapId=4faf0d9a;] security - cmd=CreateAccount; name=test.1536241056256.309@zmc.com;
  3122.  
  3123. ==> /opt/zimbra/log/mailbox.log <==
  3124. 2018-09-06 13:37:39,744 INFO [qtp788117692-1957:https:https://zmc-proxy/service/soap/] [name=test.1536241056256.309@zmc.com;ip=10.0.0.19;port=43586;soapId=4e5022c6;] soap - AuthRequest elapsed=1
  3125. 2018-09-06 13:37:39,749 INFO [qtp788117692-1957:https:https://zmc-proxy/service/soap/] [name=test.1536241056256.309@zmc.com;ip=10.0.0.19;port=43586;soapId=4e5022c6;] mbxmgr - Mailbox 184 account 4c2bd23e-faa1-404f-8f59-ff06dd82374f LOADED
  3126. 2018-09-06 13:37:39,764 INFO [qtp788117692-1906:https:https://zmc-proxy/service/soap/] [name=test.1536241056256.309@zmc.com;mid=184;ip=10.0.0.20;port=34060;soapId=4e5022c7;] cache - initializing folder and tag caches for mailbox 184
  3127.  
  3128. ==> /opt/zimbra/log/audit.log <==
  3129. 2018-09-06 13:37:39,744 INFO [qtp788117692-1957:https:https://zmc-proxy/service/soap/] [name=test.1536241056256.309@zmc.com;ip=10.0.0.19;port=43586;soapId=4e5022c6;] security - cmd=Auth; account=test.1536241056256.309@zmc.com; protocol=soap;
  3130.  
  3131. ==> /opt/zimbra/log/mailbox.log <==
  3132. 2018-09-06 13:37:39,342 INFO [qtp788117692-1804:https:https://zmc-proxy/service/soap/] [name=test.1536241056256.309@zmc.com;ip=10.0.0.19;port=43580;soapId=4faf0d9b;] soap - AuthRequest elapsed=1
  3133. 2018-09-06 13:37:39,460 INFO [qtp788117692-1815:https:https://zmc-proxy/service/soap/] [name=test.1536241056256.309@zmc.com;ip=10.0.0.20;port=34056;soapId=4faf0d9c;] mailbox - Creating mailbox with id 184 and group id 84 for test.1536241056256.309@zmc.com.
  3134. 2018-09-06 13:37:39,463 INFO [qtp788117692-1815:https:https://zmc-proxy/service/soap/] [name=test.1536241056256.309@zmc.com;ip=10.0.0.20;port=34056;soapId=4faf0d9c;] cache - initializing folder and tag caches for mailbox 184
  3135. 2018-09-06 13:37:39,700 INFO [qtp788117692-1815:https:https://zmc-proxy/service/soap/] [name=test.1536241056256.309@zmc.com;ip=10.0.0.20;port=34056;soapId=4faf0d9c;] mailbox - added account 4c2bd23e-faa1-404f-8f59-ff06dd82374f to Redis notification channel NOTIFICATION-CHANNEL-84
  3136. 2018-09-06 13:37:39,701 INFO [redisson-3-12] [] mailbox - handling notification. accountId=4c2bd23e-faa1-404f-8f59-ff06dd82374f, changeId=1, channel=NOTIFICATION-CHANNEL-84
  3137. 2018-09-06 13:37:39,702 INFO [qtp788117692-1815:https:https://zmc-proxy/service/soap/] [name=test.1536241056256.309@zmc.com;ip=10.0.0.20;port=34056;soapId=4faf0d9c;] mbxmgr - Mailbox 184 account 4c2bd23e-faa1-404f-8f59-ff06dd82374f CREATED
  3138. 2018-09-06 13:37:39,704 INFO [qtp788117692-1815:https:https://zmc-proxy/service/soap/] [name=test.1536241056256.309@zmc.com;mid=184;ip=10.0.0.20;port=34056;soapId=4faf0d9c;] mailop - Adding Tag tag1536241056256310: id=257.
  3139. 2018-09-06 13:37:39,729 INFO [qtp788117692-1815:https:https://zmc-proxy/service/soap/] [name=test.1536241056256.309@zmc.com;mid=184;ip=10.0.0.20;port=34056;soapId=4faf0d9c;] soap - CreateTagRequest elapsed=354
  3140. 2018-09-06 13:37:39,746 INFO [redisson-3-12] [] mailbox - handling notification. accountId=4c2bd23e-faa1-404f-8f59-ff06dd82374f, changeId=2, channel=NOTIFICATION-CHANNEL-84
  3141. 2018-09-06 13:37:39,853 INFO [redisson-3-29] [] mailbox - handling notification. accountId=4c2bd23e-faa1-404f-8f59-ff06dd82374f, changeId=3, channel=NOTIFICATION-CHANNEL-84
  3142. 2018-09-06 13:37:39,893 INFO [qtp788117692-1803:https:https://zmc-proxy/service/soap/] [name=test.1536241056256.309@zmc.com;ip=10.0.0.20;port=34068;soapId=4faf0d9d;] soap - AuthRequest elapsed=1
  3143. 2018-09-06 13:37:39,909 INFO [qtp788117692-1723:https:https://zmc-proxy/service/soap/] [name=test.1536241056256.309@zmc.com;mid=184;ip=10.0.0.19;port=43598;soapId=4faf0d9e;] mailop - Adding Tag tag1536241056256312: id=259.
  3144. 2018-09-06 13:37:39,935 INFO [redisson-3-11] [] mailbox - handling notification. accountId=4c2bd23e-faa1-404f-8f59-ff06dd82374f, changeId=4, channel=NOTIFICATION-CHANNEL-84
  3145. 2018-09-06 13:37:39,935 INFO [qtp788117692-1723:https:https://zmc-proxy/service/soap/] [name=test.1536241056256.309@zmc.com;mid=184;ip=10.0.0.19;port=43598;soapId=4faf0d9e;] soap - CreateTagRequest elapsed=28
  3146. 2018-09-06 13:37:39,987 INFO [redisson-3-18] [] mailbox - handling notification. accountId=4c2bd23e-faa1-404f-8f59-ff06dd82374f, changeId=5, channel=NOTIFICATION-CHANNEL-84
  3147. 2018-09-06 13:37:40,016 INFO [qtp788117692-1541:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56538;soapId=4faf0d9f;] soap - GetVersionInfoRequest elapsed=1
  3148.  
  3149. ==> /opt/zimbra/log/audit.log <==
  3150. 2018-09-06 13:37:39,342 INFO [qtp788117692-1804:https:https://zmc-proxy/service/soap/] [name=test.1536241056256.309@zmc.com;ip=10.0.0.19;port=43580;soapId=4faf0d9b;] security - cmd=Auth; account=test.1536241056256.309@zmc.com; protocol=soap;
  3151. 2018-09-06 13:37:39,893 INFO [qtp788117692-1803:https:https://zmc-proxy/service/soap/] [name=test.1536241056256.309@zmc.com;ip=10.0.0.20;port=34068;soapId=4faf0d9d;] security - cmd=Auth; account=test.1536241056256.309@zmc.com; protocol=soap;
  3152.  
  3153. ==> /opt/zimbra/log/mailbox.log <==
  3154. 2018-09-06 13:37:39,827 INFO [qtp788117692-1906:https:https://zmc-proxy/service/soap/] [name=test.1536241056256.309@zmc.com;mid=184;ip=10.0.0.20;port=34060;soapId=4e5022c7;] mailop - Adding Tag tag1536241056256311: id=258.
  3155. 2018-09-06 13:37:39,852 INFO [qtp788117692-1906:https:https://zmc-proxy/service/soap/] [name=test.1536241056256.309@zmc.com;mid=184;ip=10.0.0.20;port=34060;soapId=4e5022c7;] mailbox - added account 4c2bd23e-faa1-404f-8f59-ff06dd82374f to Redis notification channel NOTIFICATION-CHANNEL-84
  3156. 2018-09-06 13:37:39,853 INFO [redisson-3-12] [] mailbox - handling notification. accountId=4c2bd23e-faa1-404f-8f59-ff06dd82374f, changeId=3, channel=NOTIFICATION-CHANNEL-84
  3157. 2018-09-06 13:37:39,853 INFO [qtp788117692-1906:https:https://zmc-proxy/service/soap/] [name=test.1536241056256.309@zmc.com;mid=184;ip=10.0.0.20;port=34060;soapId=4e5022c7;] soap - CreateTagRequest elapsed=90
  3158. 2018-09-06 13:37:39,878 INFO [qtp788117692-1957:https:https://zmc-proxy/service/soap/] [name=test.1536241056256.309@zmc.com;mid=184;ip=10.0.0.19;port=43594;soapId=4e5022c8;] soap - GetTagRequest elapsed=3
  3159. 2018-09-06 13:37:39,935 INFO [redisson-3-26] [] mailbox - handling notification. accountId=4c2bd23e-faa1-404f-8f59-ff06dd82374f, changeId=4, channel=NOTIFICATION-CHANNEL-84
  3160. 2018-09-06 13:37:39,987 INFO [qtp788117692-1957:https:https://zmc-proxy/service/soap/] [name=test.1536241056256.309@zmc.com;mid=184;ip=10.0.0.20;port=34072;soapId=4e5022c9;] filter - Updating filter rules that reference tag1536241056256312.
  3161. 2018-09-06 13:37:39,987 INFO [qtp788117692-1957:https:https://zmc-proxy/service/soap/] [name=test.1536241056256.309@zmc.com;mid=184;ip=10.0.0.20;port=34072;soapId=4e5022c9;] soap - TagActionRequest elapsed=38
  3162. 2018-09-06 13:37:39,987 INFO [redisson-3-30] [] mailbox - handling notification. accountId=4c2bd23e-faa1-404f-8f59-ff06dd82374f, changeId=5, channel=NOTIFICATION-CHANNEL-84
  3163. 2018-09-06 13:37:40,002 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44852;soapId=4e5022ca;] soap - AuthRequest elapsed=1
  3164. 2018-09-06 13:37:40,044 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56542;soapId=4e5022cb;] misc - delegated access: doc=GetAccount, authenticated account=admin@zmc.com, target account=admin@zmc.com
  3165. 2018-09-06 13:37:40,044 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56542;soapId=4e5022cb;] misc - delegated access: doc=GetAccount, authenticated account=admin@zmc.com, target account=admin@zmc.com
  3166. 2018-09-06 13:37:40,045 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56542;soapId=4e5022cb;] soap - GetAccountRequest elapsed=1
  3167. 2018-09-06 13:37:40,061 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44860;soapId=4e5022cc;] soap - AuthRequest elapsed=0
  3168. 2018-09-06 13:37:40,116 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56550;soapId=4e5022cd;] soap - CreateAccountRequest elapsed=4
  3169.  
  3170. ==> /opt/zimbra/log/audit.log <==
  3171. 2018-09-06 13:37:40,001 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44852;soapId=4e5022ca;] security - cmd=AdminAuth; account=admin@zmc.com;
  3172. 2018-09-06 13:37:40,001 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44852;soapId=4e5022ca;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  3173. 2018-09-06 13:37:40,061 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44860;soapId=4e5022cc;] security - cmd=AdminAuth; account=admin@zmc.com;
  3174. 2018-09-06 13:37:40,061 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44860;soapId=4e5022cc;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  3175. 2018-09-06 13:37:40,115 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56550;soapId=4e5022cd;] security - cmd=CreateAccount; name=account2.1536241060064.324@zmc.com;
  3176.  
  3177. ==> /opt/zimbra/log/mailbox.log <==
  3178. 2018-09-06 13:37:40,031 INFO [qtp788117692-1561:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44856;soapId=4faf0da0;] soap - AuthRequest elapsed=1
  3179. 2018-09-06 13:37:40,077 INFO [qtp788117692-1755:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56546;soapId=4faf0da1;] soap - AuthRequest elapsed=1
  3180. 2018-09-06 13:37:40,095 INFO [qtp788117692-1805:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44864;soapId=4faf0da2;] soap - CreateAccountRequest elapsed=4
  3181.  
  3182. ==> /opt/zimbra/log/audit.log <==
  3183. 2018-09-06 13:37:40,030 INFO [qtp788117692-1561:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44856;soapId=4faf0da0;] security - cmd=AdminAuth; account=admin@zmc.com;
  3184. 2018-09-06 13:37:40,031 INFO [qtp788117692-1561:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44856;soapId=4faf0da0;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  3185. 2018-09-06 13:37:40,076 INFO [qtp788117692-1755:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56546;soapId=4faf0da1;] security - cmd=AdminAuth; account=admin@zmc.com;
  3186. 2018-09-06 13:37:40,076 INFO [qtp788117692-1755:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56546;soapId=4faf0da1;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  3187. 2018-09-06 13:37:40,094 INFO [qtp788117692-1805:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44864;soapId=4faf0da2;] security - cmd=CreateAccount; name=account1.1536241060064.323@zmc.com;
  3188.  
  3189. ==> /opt/zimbra/log/mailbox.log <==
  3190. 2018-09-06 13:37:43,134 INFO [qtp788117692-1957:https:https://zmc-proxy/service/soap/] [name=account1.1536241060064.323@zmc.com;ip=10.0.0.19;port=43620;soapId=4e5022ce;] soap - AuthRequest elapsed=2
  3191. 2018-09-06 13:37:43,156 INFO [qtp788117692-1957:https:https://zmc-proxy/service/soap/] [name=account1.1536241060064.323@zmc.com;ip=10.0.0.20;port=34096;soapId=4e5022cf;] soap - ModifyWhiteBlackListRequest elapsed=1
  3192. 2018-09-06 13:37:43,243 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44880;soapId=4e5022d0;] soap - CreateAccountRequest elapsed=4
  3193.  
  3194. ==> /opt/zimbra/log/audit.log <==
  3195. 2018-09-06 13:37:43,133 INFO [qtp788117692-1957:https:https://zmc-proxy/service/soap/] [name=account1.1536241060064.323@zmc.com;ip=10.0.0.19;port=43620;soapId=4e5022ce;] security - cmd=Auth; account=account1.1536241060064.323@zmc.com; protocol=soap;
  3196. 2018-09-06 13:37:43,242 INFO [qtp788117692-1957:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44880;soapId=4e5022d0;] security - cmd=CreateAccount; name=account4.1536241060064.326@zmc.com;
  3197.  
  3198. ==> /opt/zimbra/log/mailbox.log <==
  3199. 2018-09-06 13:37:43,204 INFO [qtp788117692-1813:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44876;soapId=4faf0da3;] soap - AuthRequest elapsed=0
  3200. 2018-09-06 13:37:43,222 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56562;soapId=4faf0da4;] soap - CreateAccountRequest elapsed=4
  3201.  
  3202. ==> /opt/zimbra/log/audit.log <==
  3203. 2018-09-06 13:37:43,204 INFO [qtp788117692-1813:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44876;soapId=4faf0da3;] security - cmd=AdminAuth; account=admin@zmc.com;
  3204. 2018-09-06 13:37:43,204 INFO [qtp788117692-1813:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.19;port=44876;soapId=4faf0da3;] security - cmd=Auth; account=admin@zmc.com; protocol=soap;
  3205. 2018-09-06 13:37:43,221 INFO [qtp788117692-1723:https:https://zmc-proxy:7071/service/admin/soap/] [name=admin@zmc.com;ip=10.0.0.20;port=56562;soapId=4faf0da4;] security - cmd=CreateAccount; name=account3.1536241060064.325@zmc.com;
  3206.  
  3207. ==> /var/log/zimbra.log <==
  3208. Sep 6 13:37:44 zmc-mailbox zmconfigd[1705]: Fetching All configs
  3209.  
  3210. ==> /var/log/zimbra.log <==
  3211. Sep 6 13:37:44 zmc-mailbox zmconfigd[1703]: Fetching All configs
  3212. Sep 6 13:37:44 zmc-mailbox zmconfigd[1703]: All configs fetched in 0.03 seconds
  3213. Sep 6 13:37:44 zmc-mailbox zmconfigd[1703]: Command not defined for imapd
  3214. Sep 6 13:37:44 zmc-mailbox zmconfigd[1705]: All configs fetched in 0.02 seconds
  3215. Sep 6 13:37:44 zmc-mailbox zmconfigd[1705]: Command not defined for imapd
  3216. Sep 6 13:37:44 zmc-mailbox zmconfigd[1705]: Watchdog: skipping service antivirus. Service not yet available for restarts.
  3217. Sep 6 13:37:44 zmc-mailbox zmconfigd[1705]: All rewrite threads completed in 0.00 sec
  3218. Sep 6 13:37:44 zmc-mailbox zmconfigd[1705]: All restarts completed in 0.00 sec
  3219. Sep 6 13:37:45 zmc-mailbox zmconfigd[1703]: Watchdog: skipping service antivirus. Service not yet available for restarts.
  3220. Sep 6 13:37:45 zmc-mailbox zmconfigd[1703]: All rewrite threads completed in 0.00 sec
  3221. Sep 6 13:37:45 zmc-mailbox zmconfigd[1703]: All restarts completed in 0.00 sec
  3222.  
  3223. ==> /opt/zimbra/log/mailbox.log <==
  3224. 2018-09-06 13:37:46,315 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=account3.1536241060064.325@zmc.com;ip=10.0.0.20;port=34108;soapId=4e5022d1;] soap - AuthRequest elapsed=1
  3225. 2018-09-06 13:37:46,372 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=account3.1536241060064.325@zmc.com;ip=10.0.0.19;port=43640;soapId=4e5022d2;] soap - ModifyWhiteBlackListRequest elapsed=1
  3226. 2018-09-06 13:37:46,393 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=account3.1536241060064.325@zmc.com;ip=10.0.0.20;port=34116;soapId=4e5022d3;] soap - GetWhiteBlackListRequest elapsed=0
  3227.  
  3228. ==> /opt/zimbra/log/audit.log <==
  3229. 2018-09-06 13:37:46,314 INFO [qtp788117692-1884:https:https://zmc-proxy/service/soap/] [name=account3.1536241060064.325@zmc.com;ip=10.0.0.20;port=34108;soapId=4e5022d1;] security - cmd=Auth; account=account3.1536241060064.325@zmc.com; protocol=soap;
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!