API tools faq
paste
Advertisement
Guest User

gearmand crash

a guest
Apr 5th, 2013
97
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 15.61 KB | None | 0 0
raw download clone embed print report
  1. INFO 2013-04-03 03:06:52.361142 [ main ] Gear unpack
  2. DEBUG 2013-04-03 03:06:52.361147 [ main ] GEAR length: 0 gearmand_command_t: GEARMAN_COMMAND_PRE_SLEEP -> libgearman-server/plugins/protocol/gear/protocol.cc:248
  3. DEBUG 2013-04-03 03:06:52.361152 [ main ] Received PRE_SLEEP 10.0.0.8:11271625 -> libgearman-server/thread.cc:318
  4. DEBUG 2013-04-03 03:06:52.361157 [ main ] 10.0.0.8:36410 packet command PRE_SLEEP -> libgearman-server/server.cc:98
  5. INFO 2013-04-03 03:06:52.361162 [ main ] Gear unpack
  6. DEBUG 2013-04-03 03:06:52.361167 [ main ] GEAR length: 0 gearmand_command_t: GEARMAN_COMMAND_GRAB_JOB_ALL -> libgearman-server/plugins/protocol/gear/protocol.cc:248
  7. DEBUG 2013-04-03 03:06:52.361172 [ main ] Received GEARMAN_COMMAND_GRAB_JOB_ALL 10.0.0.8:11271625 -> libgearman-server/thread.cc:318
  8. DEBUG 2013-04-03 03:06:52.361177 [ main ] 10.0.0.8:36410 packet command GEARMAN_COMMAND_GRAB_JOB_ALL -> libgearman-server/server.cc:98
  9. DEBUG 2013-04-03 03:06:52.361183 [ main ] 10.0.0.8:36410 Watching POLLIN -> libgearman-server/gearmand_thread.cc:151
  10. DEBUG 2013-04-03 03:06:52.361189 [ main ] GEAR length: 0 gearmand_command_t: GEARMAN_COMMAND_NO_JOB -> libgearman-server/plugins/protocol/gear/protocol.cc:274
  11. DEBUG 2013-04-03 03:06:52.361199 [ main ] send() 12 bytes to peer 10.0.0.8:36410 -> libgearman-server/io.cc:275
  12. DEBUG 2013-04-03 03:06:52.361205 [ main ] Sent NO_JOB to 10.0.0.8:11271625 -> libgearman-server/thread.cc:365
  13. DEBUG 2013-04-03 03:06:52.361213 [ main ] 10.0.0.8:36411 Ready POLLIN -> libgearman-server/gearmand_con.cc:118
  14. DEBUG 2013-04-03 03:06:52.361223 [ main ] read 24 bytes -> libgearman-server/io.cc:696
  15. INFO 2013-04-03 03:06:52.361229 [ main ] Gear unpack
  16. DEBUG 2013-04-03 03:06:52.361233 [ main ] GEAR length: 0 gearmand_command_t: GEARMAN_COMMAND_PRE_SLEEP -> libgearman-server/plugins/protocol/gear/protocol.cc:248
  17. DEBUG 2013-04-03 03:06:52.361238 [ main ] Received PRE_SLEEP 10.0.0.8:11290073 -> libgearman-server/thread.cc:318
  18. DEBUG 2013-04-03 03:06:52.361243 [ main ] 10.0.0.8:36411 packet command PRE_SLEEP -> libgearman-server/server.cc:98
  19. INFO 2013-04-03 03:06:52.361249 [ main ] Gear unpack
  20. DEBUG 2013-04-03 03:06:52.361253 [ main ] GEAR length: 0 gearmand_command_t: GEARMAN_COMMAND_GRAB_JOB_ALL -> libgearman-server/plugins/protocol/gear/protocol.cc:248
  21. DEBUG 2013-04-03 03:06:52.361258 [ main ] Received GEARMAN_COMMAND_GRAB_JOB_ALL 10.0.0.8:11290073 -> libgearman-server/thread.cc:318
  22. DEBUG 2013-04-03 03:06:52.361263 [ main ] 10.0.0.8:36411 packet command GEARMAN_COMMAND_GRAB_JOB_ALL -> libgearman-server/server.cc:98
  23. DEBUG 2013-04-03 03:06:52.361270 [ main ] 10.0.0.8:36411 Watching POLLIN -> libgearman-server/gearmand_thread.cc:151
  24. DEBUG 2013-04-03 03:06:52.361276 [ main ] GEAR length: 0 gearmand_command_t: GEARMAN_COMMAND_NO_JOB -> libgearman-server/plugins/protocol/gear/protocol.cc:274
  25. DEBUG 2013-04-03 03:06:52.361286 [ main ] send() 12 bytes to peer 10.0.0.8:36411 -> libgearman-server/io.cc:275
  26. DEBUG 2013-04-03 03:06:52.361291 [ main ] Sent NO_JOB to 10.0.0.8:11290073 -> libgearman-server/thread.cc:365
  27. DEBUG 2013-04-03 03:06:52.361297 [ main ] 10.0.0.10:53761 Ready POLLIN -> libgearman-server/gearmand_con.cc:118
  28. DEBUG 2013-04-03 03:06:52.361304 [ main ] read 72 bytes -> libgearman-server/io.cc:696
  29. INFO 2013-04-03 03:06:52.361309 [ main ] Gear unpack
  30. DEBUG 2013-04-03 03:06:52.361314 [ main ] GEAR length: 25 gearmand_command_t: GEARMAN_COMMAND_WORK_EXCEPTION -> libgearman-server/plugins/protocol/gear/protocol.cc:248
  31. DEBUG 2013-04-03 03:06:52.361319 [ main ] Received WORK_EXCEPTION 10.0.0.10:11062249 -> libgearman-server/thread.cc:318
  32. DEBUG 2013-04-03 03:06:52.361324 [ main ] 10.0.0.10:53761 packet command WORK_EXCEPTION -> libgearman-server/server.cc:98
  33.  
  34. Begin stack trace, frames found: 11
  35. ?0 0x4180f3 in /usr/local/sbin/gearmand() [0x4180f3]
  36. ?1 0x4094a5 in /usr/local/sbin/gearmand() [0x4094a5]
  37. ?2 0x7f3833549cb0 in /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f3833549cb0]
  38. ?3 0x421fb8 in /usr/local/sbin/gearmand() [0x421fb8]
  39. ?4 0x423aad in /usr/local/sbin/gearmand() [0x423aad]
  40. ?5 0x41d509 in /usr/local/sbin/gearmand() [0x41d509]
  41. #6 0x7f38342ed94c in event_base_loop at /usr/lib/libevent-2.0.so.5
  42. ?7 0x41a900 in /usr/local/sbin/gearmand() [0x41a900]
  43. ?8 0x408c5c in /usr/local/sbin/gearmand() [0x408c5c]
  44. #9 0x7f383319c76d in __libc_start_main at /lib/x86_64-linux-gnu/libc.so.6
  45. ?10 0x40926d in /usr/local/sbin/gearmand() [0x40926d]
  46. root@wwwd:/home#
  47. root@wwwd:/home#
  48. root@wwwd:/home#
  49. root@wwwd:/home#
  50. root@wwwd:/home# /usr/local/sbin/gearmand --pid-file=/var/run/gearman/gearmand.pid --user=gearman --backlog=32 --job-retries=0 --port=4730 --listen=10.0.0.10 --threads=0 --file-descriptors=1024 --worker-wakeup=0 --round-robin -q libsqlite3 --libsqlite3-db=/tmp/gearman.sqlite3 --verbose DEBUG
  51. /usr/local/sbin/gearmand: Could not open log file "/usr/local/var/log/gearmand.log", from "/home/", switching to stderr. (No such file or directory)
  52. INFO 2013-04-03 03:07:09.361994 [ main ] Initializing libsqlite3 module
  53. DEBUG 2013-04-03 03:07:09.362074 [ main ] sqlite open: /tmp/gearman.sqlite3 -> libgearman-server/plugins/queue/sqlite/instance.cc:221
  54. DEBUG 2013-04-03 03:07:09.362604 [ main ] sqlite query: SELECT when_to_run FROM gearman_queue -> libgearman-server/plugins/queue/sqlite/instance.cc:124
  55. DEBUG 2013-04-03 03:07:09.362642 [ main ] sqlite query: DELETE FROM gearman_queue WHERE unique_key=? and function_name=? -> libgearman-server/plugins/queue/sqlite/instance.cc:124
  56. DEBUG 2013-04-03 03:07:09.362680 [ main ] sqlite query: INSERT OR REPLACE INTO gearman_queue (priority, unique_key, function_name, data, when_to_run) VALUES (?,?,?,?,?) -> libgearman-server/plugins/queue/sqlite/instance.cc:124
  57. DEBUG 2013-04-03 03:07:09.362719 [ main ] sqlite query: SELECT unique_key,function_name,priority,data,when_to_run FROM gearman_queue -> libgearman-server/plugins/queue/sqlite/instance.cc:124
  58. INFO 2013-04-03 03:07:09.362750 [ main ] Initializing Gear
  59. INFO 2013-04-03 03:07:09.362823 [ main ] Starting up(6575), verbose set to DEBUG
  60. DEBUG 2013-04-03 03:07:09.363436 [ main ] Method for libevent: epoll -> libgearman-server/gearmand.cc:331
  61. DEBUG 2013-04-03 03:07:09.363489 [ main ] Trying to listen on 10.0.0.10:4730 -> libgearman-server/gearmand.cc:571
  62. INFO 2013-04-03 03:07:09.363531 [ main ] Listening on 10.0.0.10:4730 (9)
  63. DEBUG 2013-04-03 03:07:09.363546 [ main ] Creating wakeup pipe -> libgearman-server/gearmand.cc:824
  64. DEBUG 2013-04-03 03:07:09.363563 [ main ] Creating 0 threads -> libgearman-server/gearmand.cc:345
  65. DEBUG 2013-04-03 03:07:09.363582 [ main ] Creating IO thread wakeup pipe -> libgearman-server/gearmand_thread.cc:477
  66. DEBUG 2013-04-03 03:07:09.363612 [ main ] replaying queue: begin -> libgearman-server/gearmand.cc:358
  67. INFO 2013-04-03 03:07:09.363625 [ main ] sqlite replay start
  68. DEBUG 2013-04-03 03:07:09.363669 [ main ] sqlite replay: unique_key: , function_name: heartbeat_0_sdb_worker_9 -> libgearman-server/plugins/queue/sqlite/instance.cc:572
  69. DEBUG 2013-04-03 03:07:09.363688 [ main ] Comparing queue 0 to limit 0 for priority 1 -> libgearman-server/job.cc:174
  70. .....removed about 30 lines here ...
  71. DEBUG 2013-04-03 03:07:09.376354 [ main ] Comparing queue 0 to limit 0 for priority 1 -> libgearman-server/job.cc:174
  72. DEBUG 2013-04-03 03:07:09.376369 [ main ] sqlite replay: unique_key: , function_name: heartbeat_1_sdb_worker_d -> libgearman-server/plugins/queue/sqlite/instance.cc:572
  73. DEBUG 2013-04-03 03:07:09.376380 [ main ] Comparing queue 0 to limit 0 for priority 1 -> libgearman-server/job.cc:174
  74. DEBUG 2013-04-03 03:07:09.376392 [ main ] sqlite replay: unique_key: , function_name: heartbeat_1_sdb_worker_e -> libgearman-server/plugins/queue/sqlite/instance.cc:572
  75. DEBUG 2013-04-03 03:07:09.376403 [ main ] Comparing queue 0 to limit 0 for priority 1 -> libgearman-server/job.cc:174
  76. DEBUG 2013-04-03 03:07:09.376415 [ main ] sqlite replay: unique_key: , function_name: heartbeat_1_sdb_worker_f -> libgearman-server/plugins/queue/sqlite/instance.cc:572
  77. DEBUG 2013-04-03 03:07:09.376428 [ main ] Comparing queue 0 to limit 0 for priority 1 -> libgearman-server/job.cc:174
  78. DEBUG 2013-04-03 03:07:09.376452 [ main ] replaying queue: end -> libgearman-server/gearmand.cc:364
  79. INFO 2013-04-03 03:07:09.376466 [ main ] Adding event for listening socket (9)
  80. DEBUG 2013-04-03 03:07:09.376484 [ main ] Adding event for wakeup pipe -> libgearman-server/gearmand.cc:877
  81. DEBUG 2013-04-03 03:07:09.376499 [ main ] Entering main event loop -> libgearman-server/gearmand.cc:373
  82. DEBUG 2013-04-03 03:07:09.380292 [ main ] accept() 14 -> libgearman-server/gearmand.cc:791
  83. INFO 2013-04-03 03:07:09.380307 [ main ] Accepted connection from 10.0.0.4:45924
  84. DEBUG 2013-04-03 03:07:09.380326 [ main ] setsockopt() 14 -> libgearman-server/io.cc:818
  85. DEBUG 2013-04-03 03:07:09.380342 [ main ] 10.0.0.4:45924 Watching POLLIN -> libgearman-server/gearmand_thread.cc:151
  86. INFO 2013-04-03 03:07:09.380359 [ main ] Gear connection made
  87. DEBUG 2013-04-03 03:07:09.380377 [ main ] accept() 15 -> libgearman-server/gearmand.cc:791
  88. INFO 2013-04-03 03:07:09.380390 [ main ] Accepted connection from 10.0.0.4:45925
  89. DEBUG 2013-04-03 03:07:09.380406 [ main ] setsockopt() 15 -> libgearman-server/io.cc:818
  90. DEBUG 2013-04-03 03:07:09.380422 [ main ] 10.0.0.4:45925 Watching POLLIN -> libgearman-server/gearmand_thread.cc:151
  91. INFO 2013-04-03 03:07:09.380432 [ main ] Gear connection made
  92. DEBUG 2013-04-03 03:07:09.380442 [ main ] 10.0.0.4:45924 Ready POLLIN -> libgearman-server/gearmand_con.cc:118
  93. DEBUG 2013-04-03 03:07:09.380465 [ main ] read 22 bytes -> libgearman-server/io.cc:696
  94. INFO 2013-04-03 03:07:09.380478 [ main ] Gear unpack
  95. DEBUG 2013-04-03 03:07:09.380497 [ main ] GEAR length: 0 gearmand_command_t: GEARMAN_COMMAND_OPTION_REQ -> libgearman-server/plugins/protocol/gear/protocol.cc:248
  96. DEBUG 2013-04-03 03:07:09.380510 [ main ] Received OPTION_REQ 10.0.0.4:9147609 -> libgearman-server/thread.cc:318
  97. DEBUG 2013-04-03 03:07:09.380524 [ main ] 10.0.0.4:45924 packet command OPTION_REQ -> libgearman-server/server.cc:98
  98. DEBUG 2013-04-03 03:07:09.380544 [ main ] 'exceptions' -> libgearman-server/server.cc:485
  99. DEBUG 2013-04-03 03:07:09.380562 [ main ] 10.0.0.4:45924 Watching POLLIN -> libgearman-server/gearmand_thread.cc:151
  100. DEBUG 2013-04-03 03:07:09.380577 [ main ] GEAR length: 0 gearmand_command_t: GEARMAN_COMMAND_OPTION_RES -> libgearman-server/plugins/protocol/gear/protocol.cc:274
  101. DEBUG 2013-04-03 03:07:09.380600 [ main ] send() 22 bytes to peer 10.0.0.4:45924 -> libgearman-server/io.cc:275
  102. DEBUG 2013-04-03 03:07:09.380615 [ main ] Sent OPTION_RES to 10.0.0.4:9147609 -> libgearman-server/thread.cc:365
  103. DEBUG 2013-04-03 03:07:09.380631 [ main ] 10.0.0.4:45925 Ready POLLIN -> libgearman-server/gearmand_con.cc:118
  104. DEBUG 2013-04-03 03:07:09.380652 [ main ] read 22 bytes -> libgearman-server/io.cc:696
  105. INFO 2013-04-03 03:07:09.380664 [ main ] Gear unpack
  106. DEBUG 2013-04-03 03:07:09.380675 [ main ] GEAR length: 0 gearmand_command_t: GEARMAN_COMMAND_OPTION_REQ -> libgearman-server/plugins/protocol/gear/protocol.cc:248
  107. DEBUG 2013-04-03 03:07:09.380688 [ main ] Received OPTION_REQ 10.0.0.4:9166057 -> libgearman-server/thread.cc:318
  108. DEBUG 2013-04-03 03:07:09.380698 [ main ] 10.0.0.4:45925 packet command OPTION_REQ -> libgearman-server/server.cc:98
  109. DEBUG 2013-04-03 03:07:09.380711 [ main ] 'exceptions' -> libgearman-server/server.cc:485
  110. DEBUG 2013-04-03 03:07:09.380723 [ main ] 10.0.0.4:45925 Watching POLLIN -> libgearman-server/gearmand_thread.cc:151
  111. DEBUG 2013-04-03 03:07:09.380735 [ main ] GEAR length: 0 gearmand_command_t: GEARMAN_COMMAND_OPTION_RES -> libgearman-server/plugins/protocol/gear/protocol.cc:274
  112. DEBUG 2013-04-03 03:07:09.380753 [ main ] send() 22 bytes to peer 10.0.0.4:45925 -> libgearman-server/io.cc:275
  113. DEBUG 2013-04-03 03:07:09.380765 [ main ] Sent OPTION_RES to 10.0.0.4:9166057 -> libgearman-server/thread.cc:365
  114. DEBUG 2013-04-03 03:07:09.381325 [ main ] accept() 16 -> libgearman-server/gearmand.cc:791
  115. INFO 2013-04-03 03:07:09.381339 [ main ] Accepted connection from 10.0.0.4:45926
  116. DEBUG 2013-04-03 03:07:09.381354 [ main ] setsockopt() 16 -> libgearman-server/io.cc:818
  117.  
  118. .... ~10k lines removed, but less than 1 second elapsed....
  119.  
  120. DEBUG 2013-04-03 03:07:10.159416 [ main ] Sent NO_JOB to 10.0.0.8:9260313 -> libgearman-server/thread.cc:365
  121. DEBUG 2013-04-03 03:07:10.159424 [ main ] 10.0.0.10:55565 Ready POLLIN -> libgearman-server/gearmand_con.cc:118
  122. DEBUG 2013-04-03 03:07:10.159438 [ main ] read 24 bytes -> libgearman-server/io.cc:696
  123. INFO 2013-04-03 03:07:10.159443 [ main ] Gear unpack
  124. DEBUG 2013-04-03 03:07:10.159448 [ main ] GEAR length: 0 gearmand_command_t: GEARMAN_COMMAND_PRE_SLEEP -> libgearman-server/plugins/protocol/gear/protocol.cc:248
  125. DEBUG 2013-04-03 03:07:10.159453 [ main ] Received PRE_SLEEP 10.0.0.10:9354025 -> libgearman-server/thread.cc:318
  126. DEBUG 2013-04-03 03:07:10.159458 [ main ] 10.0.0.10:55565 packet command PRE_SLEEP -> libgearman-server/server.cc:98
  127. INFO 2013-04-03 03:07:10.159463 [ main ] Gear unpack
  128. DEBUG 2013-04-03 03:07:10.159468 [ main ] GEAR length: 0 gearmand_command_t: GEARMAN_COMMAND_GRAB_JOB_ALL -> libgearman-server/plugins/protocol/gear/protocol.cc:248
  129. DEBUG 2013-04-03 03:07:10.159473 [ main ] Received GEARMAN_COMMAND_GRAB_JOB_ALL 10.0.0.10:9354025 -> libgearman-server/thread.cc:318
  130. DEBUG 2013-04-03 03:07:10.159478 [ main ] 10.0.0.10:55565 packet command GEARMAN_COMMAND_GRAB_JOB_ALL -> libgearman-server/server.cc:98
  131. DEBUG 2013-04-03 03:07:10.159484 [ main ] 10.0.0.10:55565 Watching POLLIN -> libgearman-server/gearmand_thread.cc:151
  132. DEBUG 2013-04-03 03:07:10.159490 [ main ] GEAR length: 0 gearmand_command_t: GEARMAN_COMMAND_NO_JOB -> libgearman-server/plugins/protocol/gear/protocol.cc:274
  133. DEBUG 2013-04-03 03:07:10.159503 [ main ] send() 12 bytes to peer 10.0.0.10:55565 -> libgearman-server/io.cc:275
  134. DEBUG 2013-04-03 03:07:10.159509 [ main ] Sent NO_JOB to 10.0.0.10:9354025 -> libgearman-server/thread.cc:365
  135. DEBUG 2013-04-03 03:07:10.164516 [ main ] 10.0.0.10:55566 Ready POLLIN -> libgearman-server/gearmand_con.cc:118
  136. DEBUG 2013-04-03 03:07:10.164529 [ main ] read 72 bytes -> libgearman-server/io.cc:696
  137. INFO 2013-04-03 03:07:10.164535 [ main ] Gear unpack
  138. DEBUG 2013-04-03 03:07:10.164540 [ main ] GEAR length: 25 gearmand_command_t: GEARMAN_COMMAND_WORK_EXCEPTION -> libgearman-server/plugins/protocol/gear/protocol.cc:248
  139. DEBUG 2013-04-03 03:07:10.164546 [ main ] Received WORK_EXCEPTION 10.0.0.10:9780089 -> libgearman-server/thread.cc:318
  140. DEBUG 2013-04-03 03:07:10.164551 [ main ] 10.0.0.10:55566 packet command WORK_EXCEPTION -> libgearman-server/server.cc:98
  141.  
  142. Begin stack trace, frames found: 11
  143. ?0 0x4180f3 in /usr/local/sbin/gearmand() [0x4180f3]
  144. ?1 0x4094a5 in /usr/local/sbin/gearmand() [0x4094a5]
  145. ?2 0x7f3066c39cb0 in /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f3066c39cb0]
  146. ?3 0x421fb8 in /usr/local/sbin/gearmand() [0x421fb8]
  147. ?4 0x423aad in /usr/local/sbin/gearmand() [0x423aad]
  148. ?5 0x41d509 in /usr/local/sbin/gearmand() [0x41d509]
  149. #6 0x7f30679dd94c in event_base_loop at /usr/lib/libevent-2.0.so.5
  150. ?7 0x41a900 in /usr/local/sbin/gearmand() [0x41a900]
  151. ?8 0x408c5c in /usr/local/sbin/gearmand() [0x408c5c]
  152. #9 0x7f306688c76d in __libc_start_main at /lib/x86_64-linux-gnu/libc.so.6
  153. ?10 0x40926d in /usr/local/sbin/gearmand() [0x40926d]
  154. root@wwwd:/home/#
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!