API tools faq
paste
Advertisement
Guest User

node2

a guest
Jul 2nd, 2015
292
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 121.91 KB | None | 0 0
raw download clone embed print report
  1. Jul 02 2015 14:28:57 GMT: INFO (as): (as.c::373) <><><><><><><><><><> Aerospike Community Edition build 3.5.14 <><><><><><><><><><>
  2. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) # Aerospike database configuration file.
  3. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104)
  4. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) service {
  5. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) user root
  6. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) group root
  7. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) paxos-single-replica-limit 1 # Number of nodes where the replica count is automatically reduced to 1.
  8. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) pidfile /var/run/aerospike/asd.pid
  9. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) service-threads 4
  10. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) transaction-queues 4
  11. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) transaction-threads-per-queue 4
  12. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) proto-fd-max 15000
  13. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) }
  14. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104)
  15. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) logging {
  16. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) # Log file must be an absolute path.
  17. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) file /var/log/aerospike/aerospike.log {
  18. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) context any info
  19. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) }
  20. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) }
  21. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104)
  22. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) network {
  23. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) service {
  24. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) address any
  25. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) port 3000
  26. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) access-address 192.168.1.11 virtual
  27. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) network-interface-name eth0
  28. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) }
  29. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104)
  30. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) heartbeat {
  31. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) mode mesh
  32. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) address any
  33. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) port 3002
  34. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104)
  35. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) interface-address 192.168.1.11
  36. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104)
  37. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) # To use unicast-mesh heartbeats, remove the 3 lines above, and see
  38. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) # aerospike_mesh.conf for alternative.
  39. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) mesh-seed-address-port 192.168.1.10 3002
  40. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) interval 150
  41. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) timeout 10
  42. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) }
  43. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104)
  44. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) fabric {
  45. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) address 10.0.2.15
  46. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) port 3001
  47. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) }
  48. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104)
  49. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) info {
  50. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) port 3003
  51. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) }
  52. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) }
  53. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104)
  54. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) #namespace test {
  55. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) # replication-factor 2
  56. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) # memory-size 4G
  57. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) # default-ttl 30d # 30 days, use 0 to never expire/evict.
  58. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) #
  59. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) # storage-engine memory
  60. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) #}
  61. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104)
  62. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) namespace test {
  63. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) replication-factor 2
  64. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) memory-size 1G
  65. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) default-ttl 0 # 30 days, use 0 to never expire/evict.
  66. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104)
  67. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) #storage-engine memory
  68. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104)
  69. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) # To use file storage backing, comment out the line above and use the
  70. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) # following lines instead.
  71. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) storage-engine device {
  72. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) file /opt/aerospike/data/test.dat
  73. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) filesize 1G
  74. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) data-in-memory true # Store data in memory in addition to file.
  75. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) }
  76. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3104) }
  77. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3122) system file descriptor limit: 100000, proto-fd-max: 15000
  78. Jul 02 2015 14:28:57 GMT: INFO (cf:misc): (id.c::119) Node ip: 10.0.2.15
  79. Jul 02 2015 14:28:57 GMT: INFO (cf:misc): (id.c::265) Heartbeat address for mesh: 0.0.0.0
  80. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3178) Rack Aware mode not enabled
  81. Jul 02 2015 14:28:57 GMT: INFO (config): (cfg.c::3181) Node id bb9ca5f3a270008
  82. Jul 02 2015 14:28:57 GMT: INFO (namespace): (namespace_cold.c::101) ns test beginning COLD start
  83. Jul 02 2015 14:28:57 GMT: INFO (drv_ssd): (drv_ssd.c::4210) opened file /opt/aerospike/data/test.dat: usable size 1073741824
  84. Jul 02 2015 14:28:57 GMT: INFO (drv_ssd): (drv_ssd.c::1086) /opt/aerospike/data/test.dat has 1024 wblocks of size 1048576
  85. Jul 02 2015 14:28:57 GMT: INFO (drv_ssd): (drv_ssd.c::3589) device /opt/aerospike/data/test.dat: reading device to load index
  86. Jul 02 2015 14:28:57 GMT: INFO (drv_ssd): (drv_ssd.c::3594) In TID 8461: Using arena #150 for loading data for namespace "test"
  87. Jul 02 2015 14:28:57 GMT: INFO (drv_ssd): (drv_ssd.c::3615) device /opt/aerospike/data/test.dat: read complete: UNIQUE 0 (REPLACED 0) (GEN 0) (EXPIRED 0) (MAX-TTL 0) records
  88. Jul 02 2015 14:28:57 GMT: INFO (drv_ssd): (drv_ssd.c::1051) ns test loading free & defrag queues
  89. Jul 02 2015 14:28:57 GMT: INFO (drv_ssd): (drv_ssd.c::985) /opt/aerospike/data/test.dat init defrag profile: 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
  90. Jul 02 2015 14:28:57 GMT: INFO (drv_ssd): (drv_ssd.c::1075) /opt/aerospike/data/test.dat init wblock free-q 1023, defrag-q 0
  91. Jul 02 2015 14:28:57 GMT: INFO (drv_ssd): (drv_ssd.c::2768) ns test starting device maintenance threads
  92. Jul 02 2015 14:28:57 GMT: INFO (as): (as.c::410) initializing services...
  93. Jul 02 2015 14:28:57 GMT: INFO (drv_ssd): (drv_ssd.c::1840) ns test starting write worker threads
  94. Jul 02 2015 14:28:57 GMT: INFO (drv_ssd): (drv_ssd.c::902) ns test starting defrag threads
  95. Jul 02 2015 14:28:57 GMT: INFO (tsvc): (thr_tsvc.c::916) shared queues: 4 queues with 4 threads each
  96. Jul 02 2015 14:28:57 GMT: INFO (hb): (hb.c::2459) heartbeat socket initialization
  97. Jul 02 2015 14:28:57 GMT: INFO (hb): (hb.c::2473) initializing mesh heartbeat socket : 0.0.0.0:3002
  98. Jul 02 2015 14:28:57 GMT: INFO (info): (thr_info.c::5276) static external network definition
  99. Jul 02 2015 14:28:57 GMT: INFO (paxos): (paxos.c::3127) partitions from storage: total 4096 found 0 lost(set) 4096 lost(unset) 0
  100. Jul 02 2015 14:28:57 GMT: INFO (partition): (partition.c::4011) {test} 4096 partitions: found 4096 absent, 0 stored
  101. Jul 02 2015 14:28:57 GMT: INFO (paxos): (paxos.c::3131) Paxos service ignited: bb9ca5f3a270008
  102. Jul 02 2015 14:28:57 GMT: INFO (scan): (thr_tscan.c::2118) started 32 threads
  103. Jul 02 2015 14:28:57 GMT: INFO (batch): (thr_batch.c::342) Initialize 4 batch worker threads.
  104. Jul 02 2015 14:28:57 GMT: INFO (drv_ssd): (drv_ssd.c::4568) {test} floor set at 45 wblocks per device
  105. Jul 02 2015 14:29:00 GMT: INFO (paxos): (paxos.c::3212) listening for other nodes (max 3000 milliseconds) ...
  106. Jul 02 2015 14:29:00 GMT: INFO (hb): (hb.c::1979) connecting to remote heartbeat service at 192.168.1.10:3002
  107. Jul 02 2015 14:29:00 GMT: INFO (hb): (hb.c::1055) initiated connection to mesh host at 192.168.1.10:3002 (192.168.1.10:3002) via socket 61 from 10.0.2.15:35331
  108. Jul 02 2015 14:29:00 GMT: INFO (paxos): (paxos.c::3220) ... other node(s) detected - node will operate in a multi-node cluster
  109. Jul 02 2015 14:29:00 GMT: INFO (paxos): (paxos.c::3189) paxos supervisor thread started
  110. Jul 02 2015 14:29:00 GMT: INFO (nsup): (thr_nsup.c::1289) namespace supervisor started
  111. Jul 02 2015 14:29:00 GMT: INFO (ldt): (thr_nsup.c::1246) LDT supervisor started
  112. Jul 02 2015 14:29:00 GMT: INFO (demarshal): (thr_demarshal.c::255) Saved original JEMalloc arena #8 for thr_demarshal()
  113. Jul 02 2015 14:29:00 GMT: INFO (demarshal): (thr_demarshal.c::283) Service started: socket 3000
  114. Jul 02 2015 14:29:00 GMT: INFO (hb): (hb.c::2388) new heartbeat received: bb97abaa1270008 principal node is bb97abaa1270008
  115. Jul 02 2015 14:29:00 GMT: INFO (fabric): (fabric.c::1818) fabric: node bb97abaa1270008 arrived
  116. Jul 02 2015 14:29:00 GMT: INFO (paxos): (paxos.c::1931) as_paxos_spark establishing transaction [1.0]@bb9ca5f3a270008 ClSz = 1 ; # change = 1 : ADD bb97abaa1270008;
  117. Jul 02 2015 14:29:00 GMT: INFO (partition): (partition.c::392) DISALLOW MIGRATIONS
  118. Jul 02 2015 14:29:00 GMT: INFO (paxos): (paxos.c::1439) sending sync message to bb97abaa1270008
  119. Jul 02 2015 14:29:00 GMT: INFO (paxos): (paxos.c::1448) SUCCESSION [1.0]@bb9ca5f3a270008: bb9ca5f3a270008 bb97abaa1270008
  120. Jul 02 2015 14:29:00 GMT: INFO (partition): (partition.c::392) DISALLOW MIGRATIONS
  121. Jul 02 2015 14:29:00 GMT: INFO (paxos): (paxos.c::2869) SUCCESSION [1.0]@bb9ca5f3a270008*: bb9ca5f3a270008 bb97abaa1270008
  122. Jul 02 2015 14:29:00 GMT: INFO (paxos): (paxos.c::2880) node bb9ca5f3a270008 is now principal pro tempore
  123. Jul 02 2015 14:29:00 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb9ca5f3a270008
  124. Jul 02 2015 14:29:00 GMT: INFO (paxos): (paxos.c::2900) Received paxos partition sync request from bb9ca5f3a270008
  125. Jul 02 2015 14:29:00 GMT: INFO (hb): (hb.c::2388) new heartbeat received: bb9ca5f3a270008 principal node is bb9ca5f3a270008
  126. Jul 02 2015 14:29:00 GMT: INFO (fabric): (fabric.c::1818) fabric: node bb9ca5f3a270008 arrived
  127. Jul 02 2015 14:29:01 GMT: INFO (info): (thr_info.c::983) tip command executed: params host=192.168.1.10;port=3002
  128. Jul 02 2015 14:29:01 GMT: INFO (demarshal): (thr_demarshal.c::255) Saved original JEMalloc arena #13 for thr_demarshal()
  129. Jul 02 2015 14:29:01 GMT: INFO (demarshal): (thr_demarshal.c::255) Saved original JEMalloc arena #14 for thr_demarshal()
  130. Jul 02 2015 14:29:01 GMT: INFO (demarshal): (thr_demarshal.c::255) Saved original JEMalloc arena #15 for thr_demarshal()
  131. Jul 02 2015 14:29:02 GMT: INFO (hb): (hb.c::2189) hb considers expiring: now 14992170 last 14990667
  132. Jul 02 2015 14:29:02 GMT: INFO (hb): (hb.c::2202) hb expires but fabric says DEAD: node bb9ca5f3a270008
  133. Jul 02 2015 14:29:02 GMT: WARNING (hb): (hb.c::516) as_hb_nodes_discovered_hash_shutdown_conn node bb9ca5f3a270008 not found
  134. Jul 02 2015 14:29:02 GMT: INFO (hb): (hb.c::2382) removing node on heartbeat failure: bb9ca5f3a270008
  135. Jul 02 2015 14:29:02 GMT: INFO (fabric): (fabric.c::1824) fabric: node bb9ca5f3a270008 departed
  136. Jul 02 2015 14:29:02 GMT: INFO (fabric): (fabric.c::1734) fabric disconnecting node: bb9ca5f3a270008
  137. Jul 02 2015 14:29:02 GMT: INFO (paxos): (paxos.c::2104) departed node bb9ca5f3a270008 is no longer principal pro tempore
  138. Jul 02 2015 14:29:02 GMT: INFO (demarshal): (thr_demarshal.c::792) Waiting to spawn demarshal threads ...
  139. Jul 02 2015 14:29:02 GMT: INFO (demarshal): (thr_demarshal.c::795) Started 4 Demarshal Threads
  140. Jul 02 2015 14:29:02 GMT: INFO (as): (as.c::450) service ready: soon there will be cake!
  141. Jul 02 2015 14:29:05 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  142. Jul 02 2015 14:29:05 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  143. Jul 02 2015 14:29:05 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  144. Jul 02 2015 14:29:05 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  145. Jul 02 2015 14:29:07 GMT: INFO (info): (thr_info.c::983) tip command executed: params host=192.168.1.10;port=3002
  146. Jul 02 2015 14:29:10 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  147. Jul 02 2015 14:29:10 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  148. Jul 02 2015 14:29:10 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  149. Jul 02 2015 14:29:10 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  150. Jul 02 2015 14:29:12 GMT: INFO (info): (thr_info.c::4796) system memory: free 1946364kb ( 94 percent free )
  151. Jul 02 2015 14:29:12 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  152. Jul 02 2015 14:29:12 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  153. Jul 02 2015 14:29:12 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  154. Jul 02 2015 14:29:12 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 2, 2) : hb (2, 2, 0) : fab (16, 18, 2)
  155. Jul 02 2015 14:29:12 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 168
  156. Jul 02 2015 14:29:12 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 114 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 1 efd 1 efa 0 um 0 mcf 0 rc 0
  157. Jul 02 2015 14:29:12 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  158. Jul 02 2015 14:29:12 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  159. Jul 02 2015 14:29:12 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  160. Jul 02 2015 14:29:12 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  161. Jul 02 2015 14:29:12 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  162. Jul 02 2015 14:29:12 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  163. Jul 02 2015 14:29:12 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  164. Jul 02 2015 14:29:12 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  165. Jul 02 2015 14:29:12 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  166. Jul 02 2015 14:29:12 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  167. Jul 02 2015 14:29:15 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  168. Jul 02 2015 14:29:15 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  169. Jul 02 2015 14:29:15 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  170. Jul 02 2015 14:29:15 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  171. Jul 02 2015 14:29:15 GMT: INFO (hb): (hb.c::2319) HB node bb97abaa1270008 in different cluster - succession lists don't match
  172. Jul 02 2015 14:29:15 GMT: INFO (hb): (hb.c::2388) new heartbeat received: bb9ca5f3a270008 principal node is bb9ca5f3a270008
  173. Jul 02 2015 14:29:15 GMT: INFO (fabric): (fabric.c::1818) fabric: node bb9ca5f3a270008 arrived
  174. Jul 02 2015 14:29:15 GMT: INFO (paxos): (paxos.c::1093) Node bb9ca5f3a270008 revived
  175. Jul 02 2015 14:29:17 GMT: INFO (hb): (hb.c::2189) hb considers expiring: now 15007255 last 15005653
  176. Jul 02 2015 14:29:17 GMT: INFO (hb): (hb.c::2202) hb expires but fabric says DEAD: node bb9ca5f3a270008
  177. Jul 02 2015 14:29:17 GMT: WARNING (hb): (hb.c::516) as_hb_nodes_discovered_hash_shutdown_conn node bb9ca5f3a270008 not found
  178. Jul 02 2015 14:29:17 GMT: INFO (hb): (hb.c::2382) removing node on heartbeat failure: bb9ca5f3a270008
  179. Jul 02 2015 14:29:17 GMT: INFO (fabric): (fabric.c::1824) fabric: node bb9ca5f3a270008 departed
  180. Jul 02 2015 14:29:17 GMT: INFO (fabric): (fabric.c::1734) fabric disconnecting node: bb9ca5f3a270008
  181. Jul 02 2015 14:29:17 GMT: INFO (drv_ssd): (drv_ssd.c::2536) device /opt/aerospike/data/test.dat: used 0, contig-free 1023M (1023 wblocks), swb-free 0, n-w 0, w-q 0 w-tot 0 (0.0/s), defrag-q 0 defrag-tot 0 (0.0/s)
  182. Jul 02 2015 14:29:20 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  183. Jul 02 2015 14:29:20 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  184. Jul 02 2015 14:29:20 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  185. Jul 02 2015 14:29:20 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  186. Jul 02 2015 14:29:22 GMT: INFO (info): (thr_info.c::4796) system memory: free 1942640kb ( 94 percent free )
  187. Jul 02 2015 14:29:22 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  188. Jul 02 2015 14:29:22 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  189. Jul 02 2015 14:29:22 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  190. Jul 02 2015 14:29:22 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 2, 2) : hb (2, 2, 0) : fab (17, 20, 3)
  191. Jul 02 2015 14:29:22 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 298
  192. Jul 02 2015 14:29:22 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 211 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 2 efd 2 efa 0 um 0 mcf 0 rc 0
  193. Jul 02 2015 14:29:22 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  194. Jul 02 2015 14:29:22 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  195. Jul 02 2015 14:29:22 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  196. Jul 02 2015 14:29:22 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  197. Jul 02 2015 14:29:22 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  198. Jul 02 2015 14:29:22 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  199. Jul 02 2015 14:29:22 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  200. Jul 02 2015 14:29:22 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  201. Jul 02 2015 14:29:22 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  202. Jul 02 2015 14:29:22 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  203. Jul 02 2015 14:29:25 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  204. Jul 02 2015 14:29:25 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  205. Jul 02 2015 14:29:25 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  206. Jul 02 2015 14:29:25 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  207. Jul 02 2015 14:29:30 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  208. Jul 02 2015 14:29:30 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  209. Jul 02 2015 14:29:30 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  210. Jul 02 2015 14:29:30 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  211. Jul 02 2015 14:29:30 GMT: INFO (hb): (hb.c::2388) new heartbeat received: bb9ca5f3a270008 principal node is bb9ca5f3a270008
  212. Jul 02 2015 14:29:30 GMT: INFO (fabric): (fabric.c::1818) fabric: node bb9ca5f3a270008 arrived
  213. Jul 02 2015 14:29:30 GMT: INFO (paxos): (paxos.c::1093) Node bb9ca5f3a270008 revived
  214. Jul 02 2015 14:29:30 GMT: INFO (hb): (hb.c::2319) HB node bb97abaa1270008 in different cluster - succession lists don't match
  215. Jul 02 2015 14:29:32 GMT: INFO (hb): (hb.c::2189) hb considers expiring: now 15022200 last 15020653
  216. Jul 02 2015 14:29:32 GMT: INFO (hb): (hb.c::2202) hb expires but fabric says DEAD: node bb9ca5f3a270008
  217. Jul 02 2015 14:29:32 GMT: WARNING (hb): (hb.c::516) as_hb_nodes_discovered_hash_shutdown_conn node bb9ca5f3a270008 not found
  218. Jul 02 2015 14:29:32 GMT: INFO (hb): (hb.c::2382) removing node on heartbeat failure: bb9ca5f3a270008
  219. Jul 02 2015 14:29:32 GMT: INFO (fabric): (fabric.c::1824) fabric: node bb9ca5f3a270008 departed
  220. Jul 02 2015 14:29:32 GMT: INFO (fabric): (fabric.c::1734) fabric disconnecting node: bb9ca5f3a270008
  221. Jul 02 2015 14:29:32 GMT: INFO (info): (thr_info.c::4796) system memory: free 1938764kb ( 94 percent free )
  222. Jul 02 2015 14:29:32 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  223. Jul 02 2015 14:29:32 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  224. Jul 02 2015 14:29:32 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  225. Jul 02 2015 14:29:32 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 2, 2) : hb (2, 2, 0) : fab (18, 22, 4)
  226. Jul 02 2015 14:29:32 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 426
  227. Jul 02 2015 14:29:32 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 302 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 3 efd 3 efa 0 um 0 mcf 0 rc 0
  228. Jul 02 2015 14:29:32 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  229. Jul 02 2015 14:29:32 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  230. Jul 02 2015 14:29:32 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  231. Jul 02 2015 14:29:32 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  232. Jul 02 2015 14:29:32 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  233. Jul 02 2015 14:29:32 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  234. Jul 02 2015 14:29:32 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  235. Jul 02 2015 14:29:32 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  236. Jul 02 2015 14:29:32 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  237. Jul 02 2015 14:29:32 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  238. Jul 02 2015 14:29:35 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  239. Jul 02 2015 14:29:35 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  240. Jul 02 2015 14:29:35 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  241. Jul 02 2015 14:29:35 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  242. Jul 02 2015 14:29:37 GMT: INFO (drv_ssd): (drv_ssd.c::2536) device /opt/aerospike/data/test.dat: used 0, contig-free 1023M (1023 wblocks), swb-free 0, n-w 0, w-q 0 w-tot 0 (0.0/s), defrag-q 0 defrag-tot 0 (0.0/s)
  243. Jul 02 2015 14:29:40 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  244. Jul 02 2015 14:29:40 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  245. Jul 02 2015 14:29:40 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  246. Jul 02 2015 14:29:40 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  247. Jul 02 2015 14:29:42 GMT: INFO (info): (thr_info.c::4796) system memory: free 1938956kb ( 94 percent free )
  248. Jul 02 2015 14:29:42 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  249. Jul 02 2015 14:29:42 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  250. Jul 02 2015 14:29:42 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  251. Jul 02 2015 14:29:42 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 2, 2) : hb (2, 2, 0) : fab (16, 22, 6)
  252. Jul 02 2015 14:29:42 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 554
  253. Jul 02 2015 14:29:42 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 402 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 3 efd 3 efa 0 um 0 mcf 0 rc 0
  254. Jul 02 2015 14:29:42 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  255. Jul 02 2015 14:29:42 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  256. Jul 02 2015 14:29:42 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  257. Jul 02 2015 14:29:42 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  258. Jul 02 2015 14:29:42 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  259. Jul 02 2015 14:29:42 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  260. Jul 02 2015 14:29:42 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  261. Jul 02 2015 14:29:42 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  262. Jul 02 2015 14:29:42 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  263. Jul 02 2015 14:29:42 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  264. Jul 02 2015 14:29:45 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  265. Jul 02 2015 14:29:45 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  266. Jul 02 2015 14:29:45 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  267. Jul 02 2015 14:29:45 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  268. Jul 02 2015 14:29:45 GMT: INFO (hb): (hb.c::2388) new heartbeat received: bb9ca5f3a270008 principal node is bb9ca5f3a270008
  269. Jul 02 2015 14:29:45 GMT: INFO (fabric): (fabric.c::1818) fabric: node bb9ca5f3a270008 arrived
  270. Jul 02 2015 14:29:45 GMT: INFO (paxos): (paxos.c::1093) Node bb9ca5f3a270008 revived
  271. Jul 02 2015 14:29:45 GMT: INFO (hb): (hb.c::2319) HB node bb97abaa1270008 in different cluster - succession lists don't match
  272. Jul 02 2015 14:29:47 GMT: INFO (hb): (hb.c::2189) hb considers expiring: now 15037284 last 15035658
  273. Jul 02 2015 14:29:47 GMT: INFO (hb): (hb.c::2202) hb expires but fabric says DEAD: node bb9ca5f3a270008
  274. Jul 02 2015 14:29:47 GMT: WARNING (hb): (hb.c::516) as_hb_nodes_discovered_hash_shutdown_conn node bb9ca5f3a270008 not found
  275. Jul 02 2015 14:29:47 GMT: INFO (hb): (hb.c::2382) removing node on heartbeat failure: bb9ca5f3a270008
  276. Jul 02 2015 14:29:47 GMT: INFO (fabric): (fabric.c::1824) fabric: node bb9ca5f3a270008 departed
  277. Jul 02 2015 14:29:47 GMT: INFO (fabric): (fabric.c::1734) fabric disconnecting node: bb9ca5f3a270008
  278. Jul 02 2015 14:29:50 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  279. Jul 02 2015 14:29:50 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  280. Jul 02 2015 14:29:50 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  281. Jul 02 2015 14:29:50 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  282. Jul 02 2015 14:29:52 GMT: INFO (info): (thr_info.c::4796) system memory: free 1935240kb ( 94 percent free )
  283. Jul 02 2015 14:29:52 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  284. Jul 02 2015 14:29:52 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  285. Jul 02 2015 14:29:52 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  286. Jul 02 2015 14:29:52 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 2, 2) : hb (2, 2, 0) : fab (17, 24, 7)
  287. Jul 02 2015 14:29:52 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 684
  288. Jul 02 2015 14:29:52 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 491 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 4 efd 4 efa 0 um 0 mcf 0 rc 0
  289. Jul 02 2015 14:29:52 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  290. Jul 02 2015 14:29:52 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  291. Jul 02 2015 14:29:52 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  292. Jul 02 2015 14:29:52 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  293. Jul 02 2015 14:29:52 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  294. Jul 02 2015 14:29:52 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  295. Jul 02 2015 14:29:52 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  296. Jul 02 2015 14:29:52 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  297. Jul 02 2015 14:29:52 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  298. Jul 02 2015 14:29:52 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  299. Jul 02 2015 14:29:55 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  300. Jul 02 2015 14:29:55 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  301. Jul 02 2015 14:29:55 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  302. Jul 02 2015 14:29:55 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  303. Jul 02 2015 14:29:57 GMT: INFO (drv_ssd): (drv_ssd.c::2536) device /opt/aerospike/data/test.dat: used 0, contig-free 1023M (1023 wblocks), swb-free 0, n-w 0, w-q 0 w-tot 0 (0.0/s), defrag-q 0 defrag-tot 0 (0.0/s)
  304. Jul 02 2015 14:30:00 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  305. Jul 02 2015 14:30:00 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  306. Jul 02 2015 14:30:00 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  307. Jul 02 2015 14:30:00 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  308. Jul 02 2015 14:30:00 GMT: INFO (hb): (hb.c::2388) new heartbeat received: bb9ca5f3a270008 principal node is bb9ca5f3a270008
  309. Jul 02 2015 14:30:00 GMT: INFO (fabric): (fabric.c::1818) fabric: node bb9ca5f3a270008 arrived
  310. Jul 02 2015 14:30:00 GMT: INFO (paxos): (paxos.c::1093) Node bb9ca5f3a270008 revived
  311. Jul 02 2015 14:30:00 GMT: INFO (hb): (hb.c::2319) HB node bb97abaa1270008 in different cluster - succession lists don't match
  312. Jul 02 2015 14:30:02 GMT: INFO (hb): (hb.c::2189) hb considers expiring: now 15052239 last 15050657
  313. Jul 02 2015 14:30:02 GMT: INFO (hb): (hb.c::2202) hb expires but fabric says DEAD: node bb9ca5f3a270008
  314. Jul 02 2015 14:30:02 GMT: WARNING (hb): (hb.c::516) as_hb_nodes_discovered_hash_shutdown_conn node bb9ca5f3a270008 not found
  315. Jul 02 2015 14:30:02 GMT: INFO (hb): (hb.c::2382) removing node on heartbeat failure: bb9ca5f3a270008
  316. Jul 02 2015 14:30:02 GMT: INFO (fabric): (fabric.c::1824) fabric: node bb9ca5f3a270008 departed
  317. Jul 02 2015 14:30:02 GMT: INFO (fabric): (fabric.c::1734) fabric disconnecting node: bb9ca5f3a270008
  318. Jul 02 2015 14:30:02 GMT: INFO (info): (thr_info.c::4796) system memory: free 1929356kb ( 94 percent free )
  319. Jul 02 2015 14:30:02 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  320. Jul 02 2015 14:30:02 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  321. Jul 02 2015 14:30:02 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  322. Jul 02 2015 14:30:02 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 2, 2) : hb (2, 2, 0) : fab (18, 26, 8)
  323. Jul 02 2015 14:30:02 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 812
  324. Jul 02 2015 14:30:02 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 588 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 5 efd 5 efa 0 um 0 mcf 0 rc 0
  325. Jul 02 2015 14:30:02 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  326. Jul 02 2015 14:30:02 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  327. Jul 02 2015 14:30:02 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  328. Jul 02 2015 14:30:02 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  329. Jul 02 2015 14:30:02 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  330. Jul 02 2015 14:30:02 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  331. Jul 02 2015 14:30:02 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  332. Jul 02 2015 14:30:02 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  333. Jul 02 2015 14:30:02 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  334. Jul 02 2015 14:30:02 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  335. Jul 02 2015 14:30:05 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  336. Jul 02 2015 14:30:05 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  337. Jul 02 2015 14:30:05 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  338. Jul 02 2015 14:30:05 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  339. Jul 02 2015 14:30:10 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  340. Jul 02 2015 14:30:10 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  341. Jul 02 2015 14:30:10 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  342. Jul 02 2015 14:30:10 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  343. Jul 02 2015 14:30:12 GMT: INFO (info): (thr_info.c::4796) system memory: free 1929464kb ( 94 percent free )
  344. Jul 02 2015 14:30:12 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  345. Jul 02 2015 14:30:12 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  346. Jul 02 2015 14:30:12 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  347. Jul 02 2015 14:30:12 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 2, 2) : hb (2, 2, 0) : fab (16, 26, 10)
  348. Jul 02 2015 14:30:12 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 940
  349. Jul 02 2015 14:30:12 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 678 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 5 efd 5 efa 0 um 0 mcf 0 rc 0
  350. Jul 02 2015 14:30:12 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  351. Jul 02 2015 14:30:12 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  352. Jul 02 2015 14:30:12 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  353. Jul 02 2015 14:30:12 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  354. Jul 02 2015 14:30:12 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  355. Jul 02 2015 14:30:12 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  356. Jul 02 2015 14:30:12 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  357. Jul 02 2015 14:30:12 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  358. Jul 02 2015 14:30:12 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  359. Jul 02 2015 14:30:12 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  360. Jul 02 2015 14:30:15 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  361. Jul 02 2015 14:30:15 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  362. Jul 02 2015 14:30:15 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  363. Jul 02 2015 14:30:15 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  364. Jul 02 2015 14:30:15 GMT: INFO (hb): (hb.c::2388) new heartbeat received: bb9ca5f3a270008 principal node is bb9ca5f3a270008
  365. Jul 02 2015 14:30:15 GMT: INFO (fabric): (fabric.c::1818) fabric: node bb9ca5f3a270008 arrived
  366. Jul 02 2015 14:30:15 GMT: INFO (paxos): (paxos.c::1093) Node bb9ca5f3a270008 revived
  367. Jul 02 2015 14:30:16 GMT: INFO (hb): (hb.c::2319) HB node bb97abaa1270008 in different cluster - succession lists don't match
  368. Jul 02 2015 14:30:17 GMT: INFO (hb): (hb.c::2189) hb considers expiring: now 15067197 last 15065663
  369. Jul 02 2015 14:30:17 GMT: INFO (hb): (hb.c::2202) hb expires but fabric says DEAD: node bb9ca5f3a270008
  370. Jul 02 2015 14:30:17 GMT: WARNING (hb): (hb.c::516) as_hb_nodes_discovered_hash_shutdown_conn node bb9ca5f3a270008 not found
  371. Jul 02 2015 14:30:17 GMT: INFO (hb): (hb.c::2382) removing node on heartbeat failure: bb9ca5f3a270008
  372. Jul 02 2015 14:30:17 GMT: INFO (fabric): (fabric.c::1824) fabric: node bb9ca5f3a270008 departed
  373. Jul 02 2015 14:30:17 GMT: INFO (fabric): (fabric.c::1734) fabric disconnecting node: bb9ca5f3a270008
  374. Jul 02 2015 14:30:17 GMT: INFO (drv_ssd): (drv_ssd.c::2536) device /opt/aerospike/data/test.dat: used 0, contig-free 1023M (1023 wblocks), swb-free 0, n-w 0, w-q 0 w-tot 0 (0.0/s), defrag-q 0 defrag-tot 0 (0.0/s)
  375. Jul 02 2015 14:30:20 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  376. Jul 02 2015 14:30:20 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  377. Jul 02 2015 14:30:20 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  378. Jul 02 2015 14:30:20 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  379. Jul 02 2015 14:30:22 GMT: INFO (info): (thr_info.c::4796) system memory: free 1925748kb ( 93 percent free )
  380. Jul 02 2015 14:30:22 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  381. Jul 02 2015 14:30:22 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  382. Jul 02 2015 14:30:22 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  383. Jul 02 2015 14:30:22 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 2, 2) : hb (2, 2, 0) : fab (17, 28, 11)
  384. Jul 02 2015 14:30:22 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 1068
  385. Jul 02 2015 14:30:22 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 771 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 6 efd 6 efa 0 um 0 mcf 0 rc 0
  386. Jul 02 2015 14:30:22 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  387. Jul 02 2015 14:30:22 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  388. Jul 02 2015 14:30:22 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  389. Jul 02 2015 14:30:22 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  390. Jul 02 2015 14:30:22 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  391. Jul 02 2015 14:30:22 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  392. Jul 02 2015 14:30:22 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  393. Jul 02 2015 14:30:22 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  394. Jul 02 2015 14:30:22 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  395. Jul 02 2015 14:30:22 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  396. Jul 02 2015 14:30:25 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  397. Jul 02 2015 14:30:25 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  398. Jul 02 2015 14:30:25 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  399. Jul 02 2015 14:30:25 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  400. Jul 02 2015 14:30:30 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  401. Jul 02 2015 14:30:30 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  402. Jul 02 2015 14:30:30 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  403. Jul 02 2015 14:30:30 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  404. Jul 02 2015 14:30:30 GMT: INFO (hb): (hb.c::2388) new heartbeat received: bb9ca5f3a270008 principal node is bb9ca5f3a270008
  405. Jul 02 2015 14:30:30 GMT: INFO (fabric): (fabric.c::1818) fabric: node bb9ca5f3a270008 arrived
  406. Jul 02 2015 14:30:30 GMT: INFO (paxos): (paxos.c::1093) Node bb9ca5f3a270008 revived
  407. Jul 02 2015 14:30:31 GMT: INFO (hb): (hb.c::2319) HB node bb97abaa1270008 in different cluster - succession lists don't match
  408. Jul 02 2015 14:30:32 GMT: INFO (hb): (hb.c::2189) hb considers expiring: now 15082283 last 15080662
  409. Jul 02 2015 14:30:32 GMT: INFO (hb): (hb.c::2202) hb expires but fabric says DEAD: node bb9ca5f3a270008
  410. Jul 02 2015 14:30:32 GMT: WARNING (hb): (hb.c::516) as_hb_nodes_discovered_hash_shutdown_conn node bb9ca5f3a270008 not found
  411. Jul 02 2015 14:30:32 GMT: INFO (hb): (hb.c::2382) removing node on heartbeat failure: bb9ca5f3a270008
  412. Jul 02 2015 14:30:32 GMT: INFO (fabric): (fabric.c::1824) fabric: node bb9ca5f3a270008 departed
  413. Jul 02 2015 14:30:32 GMT: INFO (fabric): (fabric.c::1734) fabric disconnecting node: bb9ca5f3a270008
  414. Jul 02 2015 14:30:32 GMT: INFO (info): (thr_info.c::4796) system memory: free 1921912kb ( 93 percent free )
  415. Jul 02 2015 14:30:32 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  416. Jul 02 2015 14:30:32 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  417. Jul 02 2015 14:30:32 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  418. Jul 02 2015 14:30:32 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 2, 2) : hb (2, 2, 0) : fab (18, 30, 12)
  419. Jul 02 2015 14:30:32 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 1196
  420. Jul 02 2015 14:30:32 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 865 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 7 efd 7 efa 0 um 0 mcf 0 rc 0
  421. Jul 02 2015 14:30:32 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  422. Jul 02 2015 14:30:32 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  423. Jul 02 2015 14:30:32 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  424. Jul 02 2015 14:30:32 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  425. Jul 02 2015 14:30:32 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  426. Jul 02 2015 14:30:32 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  427. Jul 02 2015 14:30:32 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  428. Jul 02 2015 14:30:32 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  429. Jul 02 2015 14:30:32 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  430. Jul 02 2015 14:30:32 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  431. Jul 02 2015 14:30:35 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  432. Jul 02 2015 14:30:35 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  433. Jul 02 2015 14:30:35 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  434. Jul 02 2015 14:30:35 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  435. Jul 02 2015 14:30:37 GMT: INFO (drv_ssd): (drv_ssd.c::2536) device /opt/aerospike/data/test.dat: used 0, contig-free 1023M (1023 wblocks), swb-free 0, n-w 0, w-q 0 w-tot 0 (0.0/s), defrag-q 0 defrag-tot 0 (0.0/s)
  436. Jul 02 2015 14:30:40 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  437. Jul 02 2015 14:30:40 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  438. Jul 02 2015 14:30:40 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  439. Jul 02 2015 14:30:40 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  440. Jul 02 2015 14:30:42 GMT: INFO (info): (thr_info.c::4796) system memory: free 1922020kb ( 93 percent free )
  441. Jul 02 2015 14:30:42 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  442. Jul 02 2015 14:30:42 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  443. Jul 02 2015 14:30:42 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  444. Jul 02 2015 14:30:42 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 2, 2) : hb (2, 2, 0) : fab (16, 30, 14)
  445. Jul 02 2015 14:30:42 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 1326
  446. Jul 02 2015 14:30:42 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 968 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 7 efd 7 efa 0 um 0 mcf 0 rc 0
  447. Jul 02 2015 14:30:42 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  448. Jul 02 2015 14:30:42 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  449. Jul 02 2015 14:30:42 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  450. Jul 02 2015 14:30:42 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  451. Jul 02 2015 14:30:42 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  452. Jul 02 2015 14:30:42 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  453. Jul 02 2015 14:30:42 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  454. Jul 02 2015 14:30:42 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  455. Jul 02 2015 14:30:42 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  456. Jul 02 2015 14:30:42 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  457. Jul 02 2015 14:30:45 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  458. Jul 02 2015 14:30:45 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  459. Jul 02 2015 14:30:45 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  460. Jul 02 2015 14:30:45 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  461. Jul 02 2015 14:30:45 GMT: INFO (hb): (hb.c::2388) new heartbeat received: bb9ca5f3a270008 principal node is bb9ca5f3a270008
  462. Jul 02 2015 14:30:45 GMT: INFO (fabric): (fabric.c::1818) fabric: node bb9ca5f3a270008 arrived
  463. Jul 02 2015 14:30:45 GMT: INFO (paxos): (paxos.c::1093) Node bb9ca5f3a270008 revived
  464. Jul 02 2015 14:30:46 GMT: INFO (hb): (hb.c::2319) HB node bb97abaa1270008 in different cluster - succession lists don't match
  465. Jul 02 2015 14:30:47 GMT: INFO (hb): (hb.c::2189) hb considers expiring: now 15097244 last 15095663
  466. Jul 02 2015 14:30:47 GMT: INFO (hb): (hb.c::2202) hb expires but fabric says DEAD: node bb9ca5f3a270008
  467. Jul 02 2015 14:30:47 GMT: WARNING (hb): (hb.c::516) as_hb_nodes_discovered_hash_shutdown_conn node bb9ca5f3a270008 not found
  468. Jul 02 2015 14:30:47 GMT: INFO (hb): (hb.c::2382) removing node on heartbeat failure: bb9ca5f3a270008
  469. Jul 02 2015 14:30:47 GMT: INFO (fabric): (fabric.c::1824) fabric: node bb9ca5f3a270008 departed
  470. Jul 02 2015 14:30:47 GMT: INFO (fabric): (fabric.c::1734) fabric disconnecting node: bb9ca5f3a270008
  471. Jul 02 2015 14:30:50 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  472. Jul 02 2015 14:30:50 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  473. Jul 02 2015 14:30:50 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  474. Jul 02 2015 14:30:50 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  475. Jul 02 2015 14:30:52 GMT: INFO (info): (thr_info.c::4796) system memory: free 1918296kb ( 93 percent free )
  476. Jul 02 2015 14:30:52 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  477. Jul 02 2015 14:30:52 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  478. Jul 02 2015 14:30:52 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  479. Jul 02 2015 14:30:52 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 2, 2) : hb (2, 2, 0) : fab (17, 32, 15)
  480. Jul 02 2015 14:30:52 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 1454
  481. Jul 02 2015 14:30:52 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 1063 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 8 efd 8 efa 0 um 0 mcf 0 rc 0
  482. Jul 02 2015 14:30:52 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  483. Jul 02 2015 14:30:52 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  484. Jul 02 2015 14:30:52 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  485. Jul 02 2015 14:30:52 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  486. Jul 02 2015 14:30:52 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  487. Jul 02 2015 14:30:52 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  488. Jul 02 2015 14:30:52 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  489. Jul 02 2015 14:30:52 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  490. Jul 02 2015 14:30:52 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  491. Jul 02 2015 14:30:52 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  492. Jul 02 2015 14:30:55 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  493. Jul 02 2015 14:30:55 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  494. Jul 02 2015 14:30:55 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  495. Jul 02 2015 14:30:55 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  496. Jul 02 2015 14:30:57 GMT: INFO (drv_ssd): (drv_ssd.c::2536) device /opt/aerospike/data/test.dat: used 0, contig-free 1023M (1023 wblocks), swb-free 0, n-w 0, w-q 0 w-tot 0 (0.0/s), defrag-q 0 defrag-tot 0 (0.0/s)
  497. Jul 02 2015 14:31:00 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  498. Jul 02 2015 14:31:00 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  499. Jul 02 2015 14:31:00 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  500. Jul 02 2015 14:31:00 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  501. Jul 02 2015 14:31:00 GMT: INFO (hb): (hb.c::2388) new heartbeat received: bb9ca5f3a270008 principal node is bb9ca5f3a270008
  502. Jul 02 2015 14:31:00 GMT: INFO (fabric): (fabric.c::1818) fabric: node bb9ca5f3a270008 arrived
  503. Jul 02 2015 14:31:00 GMT: INFO (paxos): (paxos.c::1093) Node bb9ca5f3a270008 revived
  504. Jul 02 2015 14:31:00 GMT: INFO (nsup): (thr_nsup.c::1319) {test} nsup start
  505. Jul 02 2015 14:31:00 GMT: INFO (nsup): (thr_nsup.c::1237) {test} Records: 0, 0 0-vt, 0(0) expired, 0(0) evicted, 0(0) set deletes, 0(0) set evicted. Evict ttls: 0,0,0.000. Waits: 0,0,0. Total time: 2 ms
  506. Jul 02 2015 14:31:01 GMT: INFO (hb): (hb.c::2319) HB node bb97abaa1270008 in different cluster - succession lists don't match
  507. Jul 02 2015 14:31:02 GMT: INFO (hb): (hb.c::2189) hb considers expiring: now 15112179 last 15110665
  508. Jul 02 2015 14:31:02 GMT: INFO (hb): (hb.c::2202) hb expires but fabric says DEAD: node bb9ca5f3a270008
  509. Jul 02 2015 14:31:02 GMT: WARNING (hb): (hb.c::516) as_hb_nodes_discovered_hash_shutdown_conn node bb9ca5f3a270008 not found
  510. Jul 02 2015 14:31:02 GMT: INFO (hb): (hb.c::2382) removing node on heartbeat failure: bb9ca5f3a270008
  511. Jul 02 2015 14:31:02 GMT: INFO (fabric): (fabric.c::1824) fabric: node bb9ca5f3a270008 departed
  512. Jul 02 2015 14:31:02 GMT: INFO (fabric): (fabric.c::1734) fabric disconnecting node: bb9ca5f3a270008
  513. Jul 02 2015 14:31:02 GMT: INFO (info): (thr_info.c::4796) system memory: free 1916476kb ( 93 percent free )
  514. Jul 02 2015 14:31:02 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  515. Jul 02 2015 14:31:02 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  516. Jul 02 2015 14:31:02 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  517. Jul 02 2015 14:31:02 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 2, 2) : hb (2, 2, 0) : fab (18, 34, 16)
  518. Jul 02 2015 14:31:02 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 1586
  519. Jul 02 2015 14:31:02 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 1172 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 9 efd 9 efa 0 um 0 mcf 0 rc 0
  520. Jul 02 2015 14:31:02 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  521. Jul 02 2015 14:31:02 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  522. Jul 02 2015 14:31:02 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  523. Jul 02 2015 14:31:02 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  524. Jul 02 2015 14:31:02 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  525. Jul 02 2015 14:31:02 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  526. Jul 02 2015 14:31:02 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  527. Jul 02 2015 14:31:02 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  528. Jul 02 2015 14:31:02 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  529. Jul 02 2015 14:31:02 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  530. Jul 02 2015 14:31:05 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  531. Jul 02 2015 14:31:05 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  532. Jul 02 2015 14:31:05 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  533. Jul 02 2015 14:31:05 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  534. Jul 02 2015 14:31:10 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  535. Jul 02 2015 14:31:10 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  536. Jul 02 2015 14:31:10 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  537. Jul 02 2015 14:31:10 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  538. Jul 02 2015 14:31:12 GMT: INFO (info): (thr_info.c::4796) system memory: free 1916668kb ( 93 percent free )
  539. Jul 02 2015 14:31:12 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  540. Jul 02 2015 14:31:12 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  541. Jul 02 2015 14:31:12 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  542. Jul 02 2015 14:31:12 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 2, 2) : hb (2, 2, 0) : fab (16, 34, 18)
  543. Jul 02 2015 14:31:12 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 1716
  544. Jul 02 2015 14:31:12 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 1265 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 9 efd 9 efa 0 um 0 mcf 0 rc 0
  545. Jul 02 2015 14:31:12 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  546. Jul 02 2015 14:31:12 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  547. Jul 02 2015 14:31:12 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  548. Jul 02 2015 14:31:12 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  549. Jul 02 2015 14:31:12 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  550. Jul 02 2015 14:31:12 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  551. Jul 02 2015 14:31:12 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  552. Jul 02 2015 14:31:12 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  553. Jul 02 2015 14:31:12 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  554. Jul 02 2015 14:31:12 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  555. Jul 02 2015 14:31:15 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  556. Jul 02 2015 14:31:15 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  557. Jul 02 2015 14:31:15 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  558. Jul 02 2015 14:31:15 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  559. Jul 02 2015 14:31:15 GMT: INFO (hb): (hb.c::2388) new heartbeat received: bb9ca5f3a270008 principal node is bb9ca5f3a270008
  560. Jul 02 2015 14:31:15 GMT: INFO (fabric): (fabric.c::1818) fabric: node bb9ca5f3a270008 arrived
  561. Jul 02 2015 14:31:15 GMT: INFO (paxos): (paxos.c::1093) Node bb9ca5f3a270008 revived
  562. Jul 02 2015 14:31:16 GMT: INFO (hb): (hb.c::2319) HB node bb97abaa1270008 in different cluster - succession lists don't match
  563. Jul 02 2015 14:31:17 GMT: INFO (hb): (hb.c::2189) hb considers expiring: now 15127280 last 15125672
  564. Jul 02 2015 14:31:17 GMT: INFO (hb): (hb.c::2202) hb expires but fabric says DEAD: node bb9ca5f3a270008
  565. Jul 02 2015 14:31:17 GMT: WARNING (hb): (hb.c::516) as_hb_nodes_discovered_hash_shutdown_conn node bb9ca5f3a270008 not found
  566. Jul 02 2015 14:31:17 GMT: INFO (hb): (hb.c::2382) removing node on heartbeat failure: bb9ca5f3a270008
  567. Jul 02 2015 14:31:17 GMT: INFO (fabric): (fabric.c::1824) fabric: node bb9ca5f3a270008 departed
  568. Jul 02 2015 14:31:17 GMT: INFO (fabric): (fabric.c::1734) fabric disconnecting node: bb9ca5f3a270008
  569. Jul 02 2015 14:31:17 GMT: INFO (drv_ssd): (drv_ssd.c::2536) device /opt/aerospike/data/test.dat: used 0, contig-free 1023M (1023 wblocks), swb-free 0, n-w 0, w-q 0 w-tot 0 (0.0/s), defrag-q 0 defrag-tot 0 (0.0/s)
  570. Jul 02 2015 14:31:20 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  571. Jul 02 2015 14:31:20 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  572. Jul 02 2015 14:31:20 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  573. Jul 02 2015 14:31:20 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  574. Jul 02 2015 14:31:22 GMT: INFO (info): (thr_info.c::4796) system memory: free 1904852kb ( 92 percent free )
  575. Jul 02 2015 14:31:22 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  576. Jul 02 2015 14:31:22 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  577. Jul 02 2015 14:31:22 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  578. Jul 02 2015 14:31:22 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 5, 5) : hb (2, 2, 0) : fab (17, 36, 19)
  579. Jul 02 2015 14:31:22 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 1846
  580. Jul 02 2015 14:31:22 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 1362 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 10 efd 10 efa 0 um 0 mcf 0 rc 0
  581. Jul 02 2015 14:31:22 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  582. Jul 02 2015 14:31:22 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  583. Jul 02 2015 14:31:22 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  584. Jul 02 2015 14:31:22 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  585. Jul 02 2015 14:31:22 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  586. Jul 02 2015 14:31:22 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  587. Jul 02 2015 14:31:22 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  588. Jul 02 2015 14:31:22 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  589. Jul 02 2015 14:31:22 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  590. Jul 02 2015 14:31:22 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  591. Jul 02 2015 14:31:25 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  592. Jul 02 2015 14:31:25 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  593. Jul 02 2015 14:31:25 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  594. Jul 02 2015 14:31:25 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  595. Jul 02 2015 14:31:30 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  596. Jul 02 2015 14:31:30 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  597. Jul 02 2015 14:31:30 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  598. Jul 02 2015 14:31:30 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  599. Jul 02 2015 14:31:30 GMT: INFO (hb): (hb.c::2388) new heartbeat received: bb9ca5f3a270008 principal node is bb9ca5f3a270008
  600. Jul 02 2015 14:31:30 GMT: INFO (fabric): (fabric.c::1818) fabric: node bb9ca5f3a270008 arrived
  601. Jul 02 2015 14:31:30 GMT: INFO (paxos): (paxos.c::1093) Node bb9ca5f3a270008 revived
  602. Jul 02 2015 14:31:31 GMT: INFO (hb): (hb.c::2319) HB node bb97abaa1270008 in different cluster - succession lists don't match
  603. Jul 02 2015 14:31:32 GMT: INFO (hb): (hb.c::2189) hb considers expiring: now 15142215 last 15140675
  604. Jul 02 2015 14:31:32 GMT: INFO (hb): (hb.c::2202) hb expires but fabric says DEAD: node bb9ca5f3a270008
  605. Jul 02 2015 14:31:32 GMT: WARNING (hb): (hb.c::516) as_hb_nodes_discovered_hash_shutdown_conn node bb9ca5f3a270008 not found
  606. Jul 02 2015 14:31:32 GMT: INFO (hb): (hb.c::2382) removing node on heartbeat failure: bb9ca5f3a270008
  607. Jul 02 2015 14:31:32 GMT: INFO (fabric): (fabric.c::1824) fabric: node bb9ca5f3a270008 departed
  608. Jul 02 2015 14:31:32 GMT: INFO (fabric): (fabric.c::1734) fabric disconnecting node: bb9ca5f3a270008
  609. Jul 02 2015 14:31:32 GMT: INFO (info): (thr_info.c::4796) system memory: free 1904688kb ( 92 percent free )
  610. Jul 02 2015 14:31:32 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  611. Jul 02 2015 14:31:32 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  612. Jul 02 2015 14:31:32 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  613. Jul 02 2015 14:31:32 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 5, 5) : hb (2, 2, 0) : fab (18, 38, 20)
  614. Jul 02 2015 14:31:32 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 1976
  615. Jul 02 2015 14:31:32 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 1464 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 11 efd 11 efa 0 um 0 mcf 0 rc 0
  616. Jul 02 2015 14:31:32 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  617. Jul 02 2015 14:31:32 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  618. Jul 02 2015 14:31:32 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  619. Jul 02 2015 14:31:32 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  620. Jul 02 2015 14:31:32 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  621. Jul 02 2015 14:31:32 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  622. Jul 02 2015 14:31:32 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  623. Jul 02 2015 14:31:32 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  624. Jul 02 2015 14:31:32 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  625. Jul 02 2015 14:31:32 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  626. Jul 02 2015 14:31:35 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  627. Jul 02 2015 14:31:35 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  628. Jul 02 2015 14:31:35 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  629. Jul 02 2015 14:31:35 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  630. Jul 02 2015 14:31:37 GMT: INFO (drv_ssd): (drv_ssd.c::2536) device /opt/aerospike/data/test.dat: used 0, contig-free 1023M (1023 wblocks), swb-free 0, n-w 0, w-q 0 w-tot 0 (0.0/s), defrag-q 0 defrag-tot 0 (0.0/s)
  631. Jul 02 2015 14:31:40 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  632. Jul 02 2015 14:31:40 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  633. Jul 02 2015 14:31:40 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  634. Jul 02 2015 14:31:40 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  635. Jul 02 2015 14:31:42 GMT: INFO (info): (thr_info.c::4796) system memory: free 1904864kb ( 92 percent free )
  636. Jul 02 2015 14:31:42 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  637. Jul 02 2015 14:31:42 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  638. Jul 02 2015 14:31:42 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  639. Jul 02 2015 14:31:42 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 5, 5) : hb (2, 2, 0) : fab (16, 38, 22)
  640. Jul 02 2015 14:31:42 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 2108
  641. Jul 02 2015 14:31:42 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 1564 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 11 efd 11 efa 0 um 0 mcf 0 rc 0
  642. Jul 02 2015 14:31:42 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  643. Jul 02 2015 14:31:42 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  644. Jul 02 2015 14:31:42 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  645. Jul 02 2015 14:31:42 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  646. Jul 02 2015 14:31:42 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  647. Jul 02 2015 14:31:42 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  648. Jul 02 2015 14:31:42 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  649. Jul 02 2015 14:31:42 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  650. Jul 02 2015 14:31:42 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  651. Jul 02 2015 14:31:42 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  652. Jul 02 2015 14:31:45 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  653. Jul 02 2015 14:31:45 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  654. Jul 02 2015 14:31:45 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  655. Jul 02 2015 14:31:45 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  656. Jul 02 2015 14:31:45 GMT: INFO (hb): (hb.c::2388) new heartbeat received: bb9ca5f3a270008 principal node is bb9ca5f3a270008
  657. Jul 02 2015 14:31:45 GMT: INFO (fabric): (fabric.c::1818) fabric: node bb9ca5f3a270008 arrived
  658. Jul 02 2015 14:31:45 GMT: INFO (paxos): (paxos.c::1093) Node bb9ca5f3a270008 revived
  659. Jul 02 2015 14:31:46 GMT: INFO (hb): (hb.c::2319) HB node bb97abaa1270008 in different cluster - succession lists don't match
  660. Jul 02 2015 14:31:47 GMT: INFO (hb): (hb.c::2189) hb considers expiring: now 15157309 last 15155677
  661. Jul 02 2015 14:31:47 GMT: INFO (hb): (hb.c::2202) hb expires but fabric says DEAD: node bb9ca5f3a270008
  662. Jul 02 2015 14:31:47 GMT: WARNING (hb): (hb.c::516) as_hb_nodes_discovered_hash_shutdown_conn node bb9ca5f3a270008 not found
  663. Jul 02 2015 14:31:47 GMT: INFO (hb): (hb.c::2382) removing node on heartbeat failure: bb9ca5f3a270008
  664. Jul 02 2015 14:31:47 GMT: INFO (fabric): (fabric.c::1824) fabric: node bb9ca5f3a270008 departed
  665. Jul 02 2015 14:31:47 GMT: INFO (fabric): (fabric.c::1734) fabric disconnecting node: bb9ca5f3a270008
  666. Jul 02 2015 14:31:50 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  667. Jul 02 2015 14:31:50 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  668. Jul 02 2015 14:31:50 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  669. Jul 02 2015 14:31:50 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  670. Jul 02 2015 14:31:52 GMT: INFO (info): (thr_info.c::4796) system memory: free 1904844kb ( 92 percent free )
  671. Jul 02 2015 14:31:52 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  672. Jul 02 2015 14:31:52 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  673. Jul 02 2015 14:31:52 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  674. Jul 02 2015 14:31:52 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 5, 5) : hb (2, 2, 0) : fab (17, 40, 23)
  675. Jul 02 2015 14:31:52 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 2238
  676. Jul 02 2015 14:31:52 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 1667 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 12 efd 12 efa 0 um 0 mcf 0 rc 0
  677. Jul 02 2015 14:31:52 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  678. Jul 02 2015 14:31:52 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  679. Jul 02 2015 14:31:52 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  680. Jul 02 2015 14:31:52 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  681. Jul 02 2015 14:31:52 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  682. Jul 02 2015 14:31:52 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  683. Jul 02 2015 14:31:52 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  684. Jul 02 2015 14:31:52 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  685. Jul 02 2015 14:31:52 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  686. Jul 02 2015 14:31:52 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  687. Jul 02 2015 14:31:55 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  688. Jul 02 2015 14:31:55 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  689. Jul 02 2015 14:31:55 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  690. Jul 02 2015 14:31:55 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  691. Jul 02 2015 14:31:57 GMT: INFO (drv_ssd): (drv_ssd.c::2536) device /opt/aerospike/data/test.dat: used 0, contig-free 1023M (1023 wblocks), swb-free 0, n-w 0, w-q 0 w-tot 0 (0.0/s), defrag-q 0 defrag-tot 0 (0.0/s)
  692. Jul 02 2015 14:32:00 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  693. Jul 02 2015 14:32:00 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  694. Jul 02 2015 14:32:00 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  695. Jul 02 2015 14:32:00 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  696. Jul 02 2015 14:32:00 GMT: INFO (hb): (hb.c::2388) new heartbeat received: bb9ca5f3a270008 principal node is bb9ca5f3a270008
  697. Jul 02 2015 14:32:00 GMT: INFO (fabric): (fabric.c::1818) fabric: node bb9ca5f3a270008 arrived
  698. Jul 02 2015 14:32:00 GMT: INFO (paxos): (paxos.c::1093) Node bb9ca5f3a270008 revived
  699. Jul 02 2015 14:32:01 GMT: INFO (hb): (hb.c::2319) HB node bb97abaa1270008 in different cluster - succession lists don't match
  700. Jul 02 2015 14:32:02 GMT: INFO (hb): (hb.c::2189) hb considers expiring: now 15172256 last 15170679
  701. Jul 02 2015 14:32:02 GMT: INFO (hb): (hb.c::2202) hb expires but fabric says DEAD: node bb9ca5f3a270008
  702. Jul 02 2015 14:32:02 GMT: WARNING (hb): (hb.c::516) as_hb_nodes_discovered_hash_shutdown_conn node bb9ca5f3a270008 not found
  703. Jul 02 2015 14:32:02 GMT: INFO (hb): (hb.c::2382) removing node on heartbeat failure: bb9ca5f3a270008
  704. Jul 02 2015 14:32:02 GMT: INFO (fabric): (fabric.c::1824) fabric: node bb9ca5f3a270008 departed
  705. Jul 02 2015 14:32:02 GMT: INFO (fabric): (fabric.c::1734) fabric disconnecting node: bb9ca5f3a270008
  706. Jul 02 2015 14:32:02 GMT: INFO (info): (thr_info.c::4796) system memory: free 1912432kb ( 93 percent free )
  707. Jul 02 2015 14:32:02 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  708. Jul 02 2015 14:32:02 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  709. Jul 02 2015 14:32:02 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  710. Jul 02 2015 14:32:02 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 5, 5) : hb (2, 2, 0) : fab (18, 42, 24)
  711. Jul 02 2015 14:32:02 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 2368
  712. Jul 02 2015 14:32:02 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 1776 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 13 efd 13 efa 0 um 0 mcf 0 rc 0
  713. Jul 02 2015 14:32:02 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  714. Jul 02 2015 14:32:02 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  715. Jul 02 2015 14:32:02 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  716. Jul 02 2015 14:32:02 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  717. Jul 02 2015 14:32:02 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  718. Jul 02 2015 14:32:02 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  719. Jul 02 2015 14:32:02 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  720. Jul 02 2015 14:32:02 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  721. Jul 02 2015 14:32:02 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  722. Jul 02 2015 14:32:02 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  723. Jul 02 2015 14:32:05 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  724. Jul 02 2015 14:32:05 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  725. Jul 02 2015 14:32:05 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  726. Jul 02 2015 14:32:05 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  727. Jul 02 2015 14:32:10 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  728. Jul 02 2015 14:32:10 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  729. Jul 02 2015 14:32:10 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  730. Jul 02 2015 14:32:10 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  731. Jul 02 2015 14:32:12 GMT: INFO (info): (thr_info.c::4796) system memory: free 1904920kb ( 92 percent free )
  732. Jul 02 2015 14:32:12 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  733. Jul 02 2015 14:32:12 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  734. Jul 02 2015 14:32:12 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  735. Jul 02 2015 14:32:12 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 8, 8) : hb (2, 2, 0) : fab (16, 42, 26)
  736. Jul 02 2015 14:32:12 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 2498
  737. Jul 02 2015 14:32:12 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 1876 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 13 efd 13 efa 0 um 0 mcf 0 rc 0
  738. Jul 02 2015 14:32:12 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  739. Jul 02 2015 14:32:12 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  740. Jul 02 2015 14:32:12 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  741. Jul 02 2015 14:32:12 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  742. Jul 02 2015 14:32:12 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  743. Jul 02 2015 14:32:12 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  744. Jul 02 2015 14:32:12 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  745. Jul 02 2015 14:32:12 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  746. Jul 02 2015 14:32:12 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  747. Jul 02 2015 14:32:12 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  748. Jul 02 2015 14:32:15 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  749. Jul 02 2015 14:32:15 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  750. Jul 02 2015 14:32:15 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  751. Jul 02 2015 14:32:15 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  752. Jul 02 2015 14:32:15 GMT: INFO (hb): (hb.c::2388) new heartbeat received: bb9ca5f3a270008 principal node is bb9ca5f3a270008
  753. Jul 02 2015 14:32:15 GMT: INFO (fabric): (fabric.c::1818) fabric: node bb9ca5f3a270008 arrived
  754. Jul 02 2015 14:32:15 GMT: INFO (paxos): (paxos.c::1093) Node bb9ca5f3a270008 revived
  755. Jul 02 2015 14:32:16 GMT: INFO (hb): (hb.c::2319) HB node bb97abaa1270008 in different cluster - succession lists don't match
  756. Jul 02 2015 14:32:17 GMT: INFO (hb): (hb.c::2189) hb considers expiring: now 15187194 last 15185682
  757. Jul 02 2015 14:32:17 GMT: INFO (hb): (hb.c::2202) hb expires but fabric says DEAD: node bb9ca5f3a270008
  758. Jul 02 2015 14:32:17 GMT: WARNING (hb): (hb.c::516) as_hb_nodes_discovered_hash_shutdown_conn node bb9ca5f3a270008 not found
  759. Jul 02 2015 14:32:17 GMT: INFO (hb): (hb.c::2382) removing node on heartbeat failure: bb9ca5f3a270008
  760. Jul 02 2015 14:32:17 GMT: INFO (fabric): (fabric.c::1824) fabric: node bb9ca5f3a270008 departed
  761. Jul 02 2015 14:32:17 GMT: INFO (fabric): (fabric.c::1734) fabric disconnecting node: bb9ca5f3a270008
  762. Jul 02 2015 14:32:17 GMT: INFO (drv_ssd): (drv_ssd.c::2536) device /opt/aerospike/data/test.dat: used 0, contig-free 1023M (1023 wblocks), swb-free 0, n-w 0, w-q 0 w-tot 0 (0.0/s), defrag-q 0 defrag-tot 0 (0.0/s)
  763. Jul 02 2015 14:32:20 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  764. Jul 02 2015 14:32:20 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  765. Jul 02 2015 14:32:20 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  766. Jul 02 2015 14:32:20 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  767. Jul 02 2015 14:32:22 GMT: INFO (info): (thr_info.c::4796) system memory: free 1904884kb ( 92 percent free )
  768. Jul 02 2015 14:32:22 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  769. Jul 02 2015 14:32:22 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  770. Jul 02 2015 14:32:22 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  771. Jul 02 2015 14:32:22 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 8, 8) : hb (2, 2, 0) : fab (17, 44, 27)
  772. Jul 02 2015 14:32:22 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 2630
  773. Jul 02 2015 14:32:22 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 1975 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 14 efd 14 efa 0 um 0 mcf 0 rc 0
  774. Jul 02 2015 14:32:22 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  775. Jul 02 2015 14:32:22 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  776. Jul 02 2015 14:32:22 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  777. Jul 02 2015 14:32:22 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  778. Jul 02 2015 14:32:22 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  779. Jul 02 2015 14:32:22 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  780. Jul 02 2015 14:32:22 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  781. Jul 02 2015 14:32:22 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  782. Jul 02 2015 14:32:22 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  783. Jul 02 2015 14:32:22 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  784. Jul 02 2015 14:32:25 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  785. Jul 02 2015 14:32:25 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  786. Jul 02 2015 14:32:25 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  787. Jul 02 2015 14:32:25 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  788. Jul 02 2015 14:32:30 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  789. Jul 02 2015 14:32:30 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  790. Jul 02 2015 14:32:30 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  791. Jul 02 2015 14:32:30 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  792. Jul 02 2015 14:32:30 GMT: INFO (hb): (hb.c::2388) new heartbeat received: bb9ca5f3a270008 principal node is bb9ca5f3a270008
  793. Jul 02 2015 14:32:30 GMT: INFO (fabric): (fabric.c::1818) fabric: node bb9ca5f3a270008 arrived
  794. Jul 02 2015 14:32:30 GMT: INFO (paxos): (paxos.c::1093) Node bb9ca5f3a270008 revived
  795. Jul 02 2015 14:32:31 GMT: INFO (hb): (hb.c::2319) HB node bb97abaa1270008 in different cluster - succession lists don't match
  796. Jul 02 2015 14:32:32 GMT: INFO (hb): (hb.c::2189) hb considers expiring: now 15202280 last 15200683
  797. Jul 02 2015 14:32:32 GMT: INFO (hb): (hb.c::2202) hb expires but fabric says DEAD: node bb9ca5f3a270008
  798. Jul 02 2015 14:32:32 GMT: WARNING (hb): (hb.c::516) as_hb_nodes_discovered_hash_shutdown_conn node bb9ca5f3a270008 not found
  799. Jul 02 2015 14:32:32 GMT: INFO (hb): (hb.c::2382) removing node on heartbeat failure: bb9ca5f3a270008
  800. Jul 02 2015 14:32:32 GMT: INFO (fabric): (fabric.c::1824) fabric: node bb9ca5f3a270008 departed
  801. Jul 02 2015 14:32:32 GMT: INFO (fabric): (fabric.c::1734) fabric disconnecting node: bb9ca5f3a270008
  802. Jul 02 2015 14:32:32 GMT: INFO (info): (thr_info.c::4796) system memory: free 1904796kb ( 92 percent free )
  803. Jul 02 2015 14:32:32 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  804. Jul 02 2015 14:32:32 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  805. Jul 02 2015 14:32:32 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  806. Jul 02 2015 14:32:32 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 8, 8) : hb (2, 2, 0) : fab (18, 46, 28)
  807. Jul 02 2015 14:32:32 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 2760
  808. Jul 02 2015 14:32:32 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 2071 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 15 efd 15 efa 0 um 0 mcf 0 rc 0
  809. Jul 02 2015 14:32:32 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  810. Jul 02 2015 14:32:32 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  811. Jul 02 2015 14:32:32 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  812. Jul 02 2015 14:32:32 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  813. Jul 02 2015 14:32:32 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  814. Jul 02 2015 14:32:32 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  815. Jul 02 2015 14:32:32 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  816. Jul 02 2015 14:32:32 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  817. Jul 02 2015 14:32:32 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  818. Jul 02 2015 14:32:32 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  819. Jul 02 2015 14:32:35 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  820. Jul 02 2015 14:32:35 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  821. Jul 02 2015 14:32:35 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  822. Jul 02 2015 14:32:35 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  823. Jul 02 2015 14:32:37 GMT: INFO (drv_ssd): (drv_ssd.c::2536) device /opt/aerospike/data/test.dat: used 0, contig-free 1023M (1023 wblocks), swb-free 0, n-w 0, w-q 0 w-tot 0 (0.0/s), defrag-q 0 defrag-tot 0 (0.0/s)
  824. Jul 02 2015 14:32:40 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  825. Jul 02 2015 14:32:40 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  826. Jul 02 2015 14:32:40 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  827. Jul 02 2015 14:32:40 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  828. Jul 02 2015 14:32:42 GMT: INFO (info): (thr_info.c::4796) system memory: free 1904928kb ( 92 percent free )
  829. Jul 02 2015 14:32:42 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  830. Jul 02 2015 14:32:42 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  831. Jul 02 2015 14:32:42 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  832. Jul 02 2015 14:32:42 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 8, 8) : hb (2, 2, 0) : fab (16, 46, 30)
  833. Jul 02 2015 14:32:42 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 2890
  834. Jul 02 2015 14:32:42 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 2170 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 15 efd 15 efa 0 um 0 mcf 0 rc 0
  835. Jul 02 2015 14:32:42 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  836. Jul 02 2015 14:32:42 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  837. Jul 02 2015 14:32:42 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  838. Jul 02 2015 14:32:42 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  839. Jul 02 2015 14:32:42 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  840. Jul 02 2015 14:32:42 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  841. Jul 02 2015 14:32:42 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  842. Jul 02 2015 14:32:42 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  843. Jul 02 2015 14:32:42 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  844. Jul 02 2015 14:32:42 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  845. Jul 02 2015 14:32:45 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  846. Jul 02 2015 14:32:45 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  847. Jul 02 2015 14:32:45 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  848. Jul 02 2015 14:32:45 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  849. Jul 02 2015 14:32:45 GMT: INFO (hb): (hb.c::2388) new heartbeat received: bb9ca5f3a270008 principal node is bb9ca5f3a270008
  850. Jul 02 2015 14:32:45 GMT: INFO (fabric): (fabric.c::1818) fabric: node bb9ca5f3a270008 arrived
  851. Jul 02 2015 14:32:45 GMT: INFO (paxos): (paxos.c::1093) Node bb9ca5f3a270008 revived
  852. Jul 02 2015 14:32:46 GMT: INFO (hb): (hb.c::2319) HB node bb97abaa1270008 in different cluster - succession lists don't match
  853. Jul 02 2015 14:32:47 GMT: INFO (hb): (hb.c::2189) hb considers expiring: now 15217223 last 15215684
  854. Jul 02 2015 14:32:47 GMT: INFO (hb): (hb.c::2202) hb expires but fabric says DEAD: node bb9ca5f3a270008
  855. Jul 02 2015 14:32:47 GMT: WARNING (hb): (hb.c::516) as_hb_nodes_discovered_hash_shutdown_conn node bb9ca5f3a270008 not found
  856. Jul 02 2015 14:32:47 GMT: INFO (hb): (hb.c::2382) removing node on heartbeat failure: bb9ca5f3a270008
  857. Jul 02 2015 14:32:47 GMT: INFO (fabric): (fabric.c::1824) fabric: node bb9ca5f3a270008 departed
  858. Jul 02 2015 14:32:47 GMT: INFO (fabric): (fabric.c::1734) fabric disconnecting node: bb9ca5f3a270008
  859. Jul 02 2015 14:32:50 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  860. Jul 02 2015 14:32:50 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  861. Jul 02 2015 14:32:50 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  862. Jul 02 2015 14:32:50 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  863. Jul 02 2015 14:32:52 GMT: INFO (info): (thr_info.c::4796) system memory: free 1904916kb ( 92 percent free )
  864. Jul 02 2015 14:32:52 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  865. Jul 02 2015 14:32:52 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  866. Jul 02 2015 14:32:52 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  867. Jul 02 2015 14:32:52 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 8, 8) : hb (2, 2, 0) : fab (17, 48, 31)
  868. Jul 02 2015 14:32:52 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 3020
  869. Jul 02 2015 14:32:52 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 2278 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 16 efd 16 efa 0 um 0 mcf 0 rc 0
  870. Jul 02 2015 14:32:52 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  871. Jul 02 2015 14:32:52 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  872. Jul 02 2015 14:32:52 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  873. Jul 02 2015 14:32:52 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  874. Jul 02 2015 14:32:52 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  875. Jul 02 2015 14:32:52 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  876. Jul 02 2015 14:32:52 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  877. Jul 02 2015 14:32:52 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  878. Jul 02 2015 14:32:52 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  879. Jul 02 2015 14:32:52 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  880. Jul 02 2015 14:32:55 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  881. Jul 02 2015 14:32:55 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  882. Jul 02 2015 14:32:55 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  883. Jul 02 2015 14:32:55 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  884. Jul 02 2015 14:32:57 GMT: INFO (drv_ssd): (drv_ssd.c::2536) device /opt/aerospike/data/test.dat: used 0, contig-free 1023M (1023 wblocks), swb-free 0, n-w 0, w-q 0 w-tot 0 (0.0/s), defrag-q 0 defrag-tot 0 (0.0/s)
  885. Jul 02 2015 14:33:00 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  886. Jul 02 2015 14:33:00 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  887. Jul 02 2015 14:33:00 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  888. Jul 02 2015 14:33:00 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  889. Jul 02 2015 14:33:00 GMT: INFO (hb): (hb.c::2388) new heartbeat received: bb9ca5f3a270008 principal node is bb9ca5f3a270008
  890. Jul 02 2015 14:33:00 GMT: INFO (fabric): (fabric.c::1818) fabric: node bb9ca5f3a270008 arrived
  891. Jul 02 2015 14:33:00 GMT: INFO (paxos): (paxos.c::1093) Node bb9ca5f3a270008 revived
  892. Jul 02 2015 14:33:00 GMT: INFO (nsup): (thr_nsup.c::1319) {test} nsup start
  893. Jul 02 2015 14:33:00 GMT: INFO (nsup): (thr_nsup.c::1237) {test} Records: 0, 0 0-vt, 0(0) expired, 0(0) evicted, 0(0) set deletes, 0(0) set evicted. Evict ttls: 0,0,0.000. Waits: 0,0,0. Total time: 3 ms
  894. Jul 02 2015 14:33:02 GMT: INFO (hb): (hb.c::2319) HB node bb97abaa1270008 in different cluster - succession lists don't match
  895. Jul 02 2015 14:33:02 GMT: INFO (hb): (hb.c::2189) hb considers expiring: now 15232289 last 15230685
  896. Jul 02 2015 14:33:02 GMT: INFO (hb): (hb.c::2202) hb expires but fabric says DEAD: node bb9ca5f3a270008
  897. Jul 02 2015 14:33:02 GMT: WARNING (hb): (hb.c::516) as_hb_nodes_discovered_hash_shutdown_conn node bb9ca5f3a270008 not found
  898. Jul 02 2015 14:33:02 GMT: INFO (hb): (hb.c::2382) removing node on heartbeat failure: bb9ca5f3a270008
  899. Jul 02 2015 14:33:02 GMT: INFO (fabric): (fabric.c::1824) fabric: node bb9ca5f3a270008 departed
  900. Jul 02 2015 14:33:02 GMT: INFO (fabric): (fabric.c::1734) fabric disconnecting node: bb9ca5f3a270008
  901. Jul 02 2015 14:33:02 GMT: INFO (info): (thr_info.c::4796) system memory: free 1912452kb ( 93 percent free )
  902. Jul 02 2015 14:33:02 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  903. Jul 02 2015 14:33:02 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  904. Jul 02 2015 14:33:02 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  905. Jul 02 2015 14:33:02 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 8, 8) : hb (2, 2, 0) : fab (18, 50, 32)
  906. Jul 02 2015 14:33:02 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 3152
  907. Jul 02 2015 14:33:02 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 2387 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 17 efd 17 efa 0 um 0 mcf 0 rc 0
  908. Jul 02 2015 14:33:02 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  909. Jul 02 2015 14:33:02 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  910. Jul 02 2015 14:33:02 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  911. Jul 02 2015 14:33:02 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  912. Jul 02 2015 14:33:02 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  913. Jul 02 2015 14:33:02 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  914. Jul 02 2015 14:33:02 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  915. Jul 02 2015 14:33:02 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  916. Jul 02 2015 14:33:02 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  917. Jul 02 2015 14:33:02 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  918. Jul 02 2015 14:33:05 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  919. Jul 02 2015 14:33:05 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  920. Jul 02 2015 14:33:05 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  921. Jul 02 2015 14:33:05 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  922. Jul 02 2015 14:33:10 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  923. Jul 02 2015 14:33:10 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  924. Jul 02 2015 14:33:10 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  925. Jul 02 2015 14:33:10 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  926. Jul 02 2015 14:33:12 GMT: INFO (info): (thr_info.c::4796) system memory: free 1912644kb ( 93 percent free )
  927. Jul 02 2015 14:33:12 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  928. Jul 02 2015 14:33:12 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  929. Jul 02 2015 14:33:12 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  930. Jul 02 2015 14:33:12 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 8, 8) : hb (2, 2, 0) : fab (16, 50, 34)
  931. Jul 02 2015 14:33:12 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 3282
  932. Jul 02 2015 14:33:12 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 2485 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 17 efd 17 efa 0 um 0 mcf 0 rc 0
  933. Jul 02 2015 14:33:12 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  934. Jul 02 2015 14:33:12 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  935. Jul 02 2015 14:33:12 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  936. Jul 02 2015 14:33:12 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  937. Jul 02 2015 14:33:12 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  938. Jul 02 2015 14:33:12 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  939. Jul 02 2015 14:33:12 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  940. Jul 02 2015 14:33:12 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  941. Jul 02 2015 14:33:12 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  942. Jul 02 2015 14:33:12 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  943. Jul 02 2015 14:33:15 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  944. Jul 02 2015 14:33:15 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  945. Jul 02 2015 14:33:15 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  946. Jul 02 2015 14:33:15 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  947. Jul 02 2015 14:33:15 GMT: INFO (hb): (hb.c::2388) new heartbeat received: bb9ca5f3a270008 principal node is bb9ca5f3a270008
  948. Jul 02 2015 14:33:15 GMT: INFO (fabric): (fabric.c::1818) fabric: node bb9ca5f3a270008 arrived
  949. Jul 02 2015 14:33:15 GMT: INFO (paxos): (paxos.c::1093) Node bb9ca5f3a270008 revived
  950. Jul 02 2015 14:33:17 GMT: INFO (hb): (hb.c::2189) hb considers expiring: now 15247249 last 15245688
  951. Jul 02 2015 14:33:17 GMT: INFO (hb): (hb.c::2202) hb expires but fabric says DEAD: node bb9ca5f3a270008
  952. Jul 02 2015 14:33:17 GMT: WARNING (hb): (hb.c::516) as_hb_nodes_discovered_hash_shutdown_conn node bb9ca5f3a270008 not found
  953. Jul 02 2015 14:33:17 GMT: INFO (hb): (hb.c::2319) HB node bb97abaa1270008 in different cluster - succession lists don't match
  954. Jul 02 2015 14:33:17 GMT: INFO (hb): (hb.c::2382) removing node on heartbeat failure: bb9ca5f3a270008
  955. Jul 02 2015 14:33:17 GMT: INFO (fabric): (fabric.c::1824) fabric: node bb9ca5f3a270008 departed
  956. Jul 02 2015 14:33:17 GMT: INFO (fabric): (fabric.c::1734) fabric disconnecting node: bb9ca5f3a270008
  957. Jul 02 2015 14:33:17 GMT: INFO (drv_ssd): (drv_ssd.c::2536) device /opt/aerospike/data/test.dat: used 0, contig-free 1023M (1023 wblocks), swb-free 0, n-w 0, w-q 0 w-tot 0 (0.0/s), defrag-q 0 defrag-tot 0 (0.0/s)
  958. Jul 02 2015 14:33:20 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  959. Jul 02 2015 14:33:20 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  960. Jul 02 2015 14:33:20 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  961. Jul 02 2015 14:33:20 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  962. Jul 02 2015 14:33:22 GMT: INFO (info): (thr_info.c::4796) system memory: free 1912624kb ( 93 percent free )
  963. Jul 02 2015 14:33:22 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  964. Jul 02 2015 14:33:22 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  965. Jul 02 2015 14:33:22 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
  966. Jul 02 2015 14:33:22 GMT: INFO (info): (thr_info.c::4828) trans_in_progress: wr 0 prox 0 wait 0 ::: q 0 ::: bq 0 ::: iq 0 ::: dq 0 : fds - proto (0, 8, 8) : hb (2, 2, 0) : fab (17, 52, 35)
  967. Jul 02 2015 14:33:22 GMT: INFO (info): (thr_info.c::4830) heartbeat_received: self 0 : foreign 3412
  968. Jul 02 2015 14:33:22 GMT: INFO (info): (thr_info.c::4831) heartbeat_stats: bt 0 bf 2590 nt 0 ni 0 nn 0 nnir 0 nal 0 sf1 0 sf2 0 sf3 0 sf4 0 sf5 0 sf6 0 mrf 0 eh 18 efd 18 efa 0 um 0 mcf 0 rc 0
  969. Jul 02 2015 14:33:22 GMT: INFO (info): (thr_info.c::4844) tree_counts: nsup 0 scan 0 batch 0 dup 0 wprocess 0 migrx 0 migtx 0 ssdr 0 ssdw 0 rw 0
  970. Jul 02 2015 14:33:22 GMT: INFO (info): (thr_info.c::4860) namespace test: disk inuse: 0 memory inuse: 0 (bytes) sindex memory inuse: 0 (bytes) avail pct 99
  971. Jul 02 2015 14:33:22 GMT: INFO (info): (thr_info.c::4905) partitions: actual 0 sync 0 desync 0 zombie 0 wait 0 absent 4096
  972. Jul 02 2015 14:33:22 GMT: INFO (info): (hist.c::137) histogram dump: reads (0 total) msec
  973. Jul 02 2015 14:33:22 GMT: INFO (info): (hist.c::137) histogram dump: writes_master (0 total) msec
  974. Jul 02 2015 14:33:22 GMT: INFO (info): (hist.c::137) histogram dump: proxy (0 total) msec
  975. Jul 02 2015 14:33:22 GMT: INFO (info): (hist.c::137) histogram dump: writes_reply (0 total) msec
  976. Jul 02 2015 14:33:22 GMT: INFO (info): (hist.c::137) histogram dump: udf (0 total) msec
  977. Jul 02 2015 14:33:22 GMT: INFO (info): (hist.c::137) histogram dump: query (0 total) msec
  978. Jul 02 2015 14:33:22 GMT: INFO (info): (hist.c::137) histogram dump: query_rec_count (0 total) count
  979. Jul 02 2015 14:33:25 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  980. Jul 02 2015 14:33:25 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  981. Jul 02 2015 14:33:25 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  982. Jul 02 2015 14:33:25 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  983. Jul 02 2015 14:33:30 GMT: INFO (paxos): (paxos.c::2367) Cluster Integrity Check: Detected succession list discrepancy between node bb97abaa1270008 and self bb9ca5f3a270008
  984. Jul 02 2015 14:33:30 GMT: INFO (paxos): (paxos.c::2412) CLUSTER INTEGRITY FAULT. [Phase 1 of 2] To fix, issue this command across all nodes: dun:nodes=bb97abaa1270008
  985. Jul 02 2015 14:33:30 GMT: INFO (paxos): (paxos.c::2519) as_paxos_retransmit_check: node bb9ca5f3a270008 retransmitting partition sync request to principal bb97abaa1270008 ...
  986. Jul 02 2015 14:33:30 GMT: INFO (paxos): (paxos.c::2229) Sent partition sync request to node bb97abaa1270008
  987. Jul 02 2015 14:33:30 GMT: INFO (hb): (hb.c::2388) new heartbeat received: bb9ca5f3a270008 principal node is bb9ca5f3a270008
  988. Jul 02 2015 14:33:30 GMT: INFO (fabric): (fabric.c::1818) fabric: node bb9ca5f3a270008 arrived
  989. Jul 02 2015 14:33:30 GMT: INFO (paxos): (paxos.c::1093) Node bb9ca5f3a270008 revived
  990. Jul 02 2015 14:33:32 GMT: INFO (hb): (hb.c::2189) hb considers expiring: now 15262327 last 15260688
  991. Jul 02 2015 14:33:32 GMT: INFO (hb): (hb.c::2202) hb expires but fabric says DEAD: node bb9ca5f3a270008
  992. Jul 02 2015 14:33:32 GMT: WARNING (hb): (hb.c::516) as_hb_nodes_discovered_hash_shutdown_conn node bb9ca5f3a270008 not found
  993. Jul 02 2015 14:33:32 GMT: INFO (hb): (hb.c::2319) HB node bb97abaa1270008 in different cluster - succession lists don't match
  994. Jul 02 2015 14:33:32 GMT: INFO (hb): (hb.c::2382) removing node on heartbeat failure: bb9ca5f3a270008
  995. Jul 02 2015 14:33:32 GMT: INFO (fabric): (fabric.c::1824) fabric: node bb9ca5f3a270008 departed
  996. Jul 02 2015 14:33:32 GMT: INFO (fabric): (fabric.c::1734) fabric disconnecting node: bb9ca5f3a270008
  997. Jul 02 2015 14:33:32 GMT: INFO (info): (thr_info.c::4796) system memory: free 1912452kb ( 93 percent free )
  998. Jul 02 2015 14:33:32 GMT: INFO (info): (thr_info.c::4804) migrates in progress ( 0 , 0 ) ::: ClusterSize 1 ::: objects 0 ::: sub_objects 0
  999. Jul 02 2015 14:33:32 GMT: INFO (info): (thr_info.c::4812) rec refs 0 ::: rec locks 0 ::: trees 0 ::: wr reqs 0 ::: mig tx 0 ::: mig rx 0
  1000. Jul 02 2015 14:33:32 GMT: INFO (info): (thr_info.c::4818) replica errs :: null 0 non-null 0 ::: sync copy errs :: node 0 :: master 0
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!