API tools faq
paste
Guest User

Untitled

a guest
Dec 15th, 2018
104
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 16.34 KB | None | 0 0
raw download clone embed print report
  1. # this is on the PROBED machine (10.3.0.128)
  2.  
  3. # glusterd --debug
  4. [2011-09-12 19:50:23.259984] D [glusterfsd.c:374:get_volfp] 0-glusterfsd: loading volume file /etc/glusterfs/glusterd.vol
  5. [2011-09-12 19:50:23.260757] D [xlator.c:1302:xlator_dynload] 0-xlator: dlsym(reconfigure) on /opt/glusterfs/3.2.3/lib64/glusterfs/3.2.3/xlator/mgmt/glusterd.so: undefined symbol: reconfigure -- neglecting
  6. [2011-09-12 19:50:23.260779] D [xlator.c:1308:xlator_dynload] 0-xlator: dlsym(validate_options) on /opt/glusterfs/3.2.3/lib64/glusterfs/3.2.3/xlator/mgmt/glusterd.so: undefined symbol: validate_options -- neglecting
  7. [2011-09-12 19:50:23.260899] I [glusterd.c:564:init] 0-management: Using /etc/glusterd as working directory
  8. [2011-09-12 19:50:23.260943] D [glusterd.c:242:glusterd_rpcsvc_options_build] 0-: listen-backlog value: 128
  9. [2011-09-12 19:50:23.261317] D [rpcsvc.c:1771:rpcsvc_init] 0-rpc-service: RPC service inited.
  10. [2011-09-12 19:50:23.261336] D [rpcsvc.c:1568:rpcsvc_program_register] 0-rpc-service: New program registered: GF-DUMP, Num: 123451501, Ver: 1, Port: 0
  11. [2011-09-12 19:50:23.261366] D [rpc-transport.c:672:rpc_transport_load] 0-rpc-transport: attempt to load file /opt/glusterfs/3.2.3/lib64/glusterfs/3.2.3/rpc-transport/socket.so
  12. [2011-09-12 19:50:23.261526] D [rpc-transport.c:97:__volume_option_value_validate] 0-socket.management: no range check required for 'option transport.socket.listen-backlog 128'
  13. [2011-09-12 19:50:23.261547] D [rpc-transport.c:97:__volume_option_value_validate] 0-socket.management: no range check required for 'option transport.socket.keepalive-interval 2'
  14. [2011-09-12 19:50:23.261561] D [rpc-transport.c:97:__volume_option_value_validate] 0-socket.management: no range check required for 'option transport.socket.keepalive-time 10'
  15. [2011-09-12 19:50:23.261591] D [name.c:552:server_fill_address_family] 0-socket.management: option address-family not specified, defaulting to inet/inet6
  16. [2011-09-12 19:50:23.261776] D [rpc-transport.c:672:rpc_transport_load] 0-rpc-transport: attempt to load file /opt/glusterfs/3.2.3/lib64/glusterfs/3.2.3/rpc-transport/rdma.so
  17. libibverbs: Fatal: couldn't read uverbs ABI version.
  18. [2011-09-12 19:50:23.262103] C [rdma.c:3941:rdma_init] 0-rpc-transport/rdma: No IB devices found
  19. [2011-09-12 19:50:23.262375] E [rdma.c:4812:init] 0-rdma.management: Failed to initialize IB Device
  20. [2011-09-12 19:50:23.262395] E [rpc-transport.c:741:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed
  21. [2011-09-12 19:50:23.262406] W [rpcsvc.c:1288:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed
  22. [2011-09-12 19:50:23.262418] D [rpcsvc.c:1568:rpcsvc_program_register] 0-rpc-service: New program registered: GlusterD0.0.1, Num: 1298433, Ver: 1, Port: 0
  23. [2011-09-12 19:50:23.262430] D [rpcsvc.c:1568:rpcsvc_program_register] 0-rpc-service: New program registered: GlusterD svc cli, Num: 1238463, Ver: 1, Port: 0
  24. [2011-09-12 19:50:23.262442] D [rpcsvc.c:1568:rpcsvc_program_register] 0-rpc-service: New program registered: GlusterD svc mgmt, Num: 1238433, Ver: 1, Port: 0
  25. [2011-09-12 19:50:23.262453] D [rpcsvc.c:1568:rpcsvc_program_register] 0-rpc-service: New program registered: Gluster Portmap, Num: 34123456, Ver: 1, Port: 0
  26. [2011-09-12 19:50:23.262465] D [rpcsvc.c:1568:rpcsvc_program_register] 0-rpc-service: New program registered: GlusterFS Handshake, Num: 14398633, Ver: 1, Port: 0
  27. [2011-09-12 19:50:23.262486] D [glusterd-utils.c:3157:glusterd_sm_tr_log_init] 0-: returning 0
  28. [2011-09-12 19:50:23.262517] D [glusterd-store.c:1134:glusterd_store_handle_new] 0-: Returning 0
  29. [2011-09-12 19:50:23.262532] D [glusterd-store.c:1152:glusterd_store_handle_retrieve] 0-: Returning 0
  30. [2011-09-12 19:50:23.262571] D [glusterd-store.c:1038:glusterd_store_retrieve_value] 0-: key UUID read
  31. [2011-09-12 19:50:23.262583] D [glusterd-store.c:1041:glusterd_store_retrieve_value] 0-: key UUID found
  32. [2011-09-12 19:50:23.262603] D [glusterd-store.c:1269:glusterd_retrieve_uuid] 0-: Returning 0
  33. [2011-09-12 19:50:23.262623] I [glusterd.c:88:glusterd_uuid_init] 0-glusterd: retrieved UUID: 6af2cc62-7707-4ee5-a1cc-b9c6258de43e
  34. sh: /opt/glusterfs/3.2.3/local/libexec//glusterfs/gsyncd: No such file or directory
  35. [2011-09-12 19:50:23.266522] D [glusterd.c:302:glusterd_check_gsync_present] 0-: Returning -1
  36. [2011-09-12 19:50:23.266604] D [glusterd-store.c:1904:glusterd_store_retrieve_volumes] 0-: Returning with 0
  37. [2011-09-12 19:50:23.266644] D [glusterd-store.c:1134:glusterd_store_handle_new] 0-: Returning 0
  38. [2011-09-12 19:50:23.266656] D [glusterd-store.c:1152:glusterd_store_handle_retrieve] 0-: Returning 0
  39. [2011-09-12 19:50:23.266684] D [glusterd-store.c:1315:glusterd_store_iter_new] 0-: Returning with 0
  40. [2011-09-12 19:50:23.266708] D [glusterd-store.c:1433:glusterd_store_iter_get_next] 0-: Returning with 0
  41. [2011-09-12 19:50:23.266723] D [glusterd-store.c:1433:glusterd_store_iter_get_next] 0-: Returning with 0
  42. [2011-09-12 19:50:23.266736] D [glusterd-store.c:1433:glusterd_store_iter_get_next] 0-: Returning with 0
  43. [2011-09-12 19:50:23.266749] D [glusterd-store.c:1433:glusterd_store_iter_get_next] 0-: Returning with -1
  44. [2011-09-12 19:50:23.266775] D [glusterd-utils.c:3157:glusterd_sm_tr_log_init] 0-: returning 0
  45. [2011-09-12 19:50:23.266788] D [glusterd-utils.c:3246:glusterd_peerinfo_new] 0-: returning 0
  46. [2011-09-12 19:50:23.266827] D [glusterd-handler.c:3328:glusterd_transport_inet_keepalive_options_build] 0-glusterd: Returning 0
  47. [2011-09-12 19:50:23.268244] D [rpc-clnt.c:914:rpc_clnt_connection_init] 0-management: defaulting frame-timeout to 30mins
  48. [2011-09-12 19:50:23.268272] D [rpc-transport.c:672:rpc_transport_load] 0-rpc-transport: attempt to load file /opt/glusterfs/3.2.3/lib64/glusterfs/3.2.3/rpc-transport/socket.so
  49. [2011-09-12 19:50:23.268312] D [rpc-transport.c:97:__volume_option_value_validate] 0-management: no range check required for 'option transport.socket.keepalive-time 10'
  50. [2011-09-12 19:50:23.268331] D [rpc-transport.c:97:__volume_option_value_validate] 0-management: no range check required for 'option transport.socket.keepalive-interval 2'
  51. [2011-09-12 19:50:23.268348] D [rpc-transport.c:97:__volume_option_value_validate] 0-management: no range check required for 'option remote-port 24007'
  52. [2011-09-12 19:50:23.268604] D [common-utils.c:151:gf_resolve_ip6] 0-resolver: returning ip-10.3.0.127 (port-24007) for hostname: 10.3.0.127 and port: 24007
  53. [2011-09-12 19:50:23.268769] D [glusterd-handler.c:3282:glusterd_rpc_create] 0-: returning 0
  54. [2011-09-12 19:50:23.268785] I [glusterd-handler.c:3404:glusterd_friend_add] 0-glusterd: connect returned 0
  55. [2011-09-12 19:50:23.268796] D [glusterd-store.c:2252:glusterd_store_retrieve_peers] 0-: Returning with 0
  56. [2011-09-12 19:50:23.268803] D [glusterd-store.c:2282:glusterd_resolve_all_bricks] 0-: Returning with 0
  57. [2011-09-12 19:50:23.268958] D [glusterd-store.c:2309:glusterd_restore] 0-: Returning 0
  58. Given volfile:
  59. +------------------------------------------------------------------------------+
  60. 1: volume management
  61. 2: type mgmt/glusterd
  62. 3: option working-directory /etc/glusterd
  63. 4: option transport-type socket,rdma
  64. 5: option transport.socket.keepalive-time 10
  65. 6: option transport.socket.keepalive-interval 2
  66. 7: end-volume
  67. 8:
  68.  
  69. +------------------------------------------------------------------------------+
  70. [2011-09-12 19:50:23.270022] D [glusterd-handler.c:3854:glusterd_peer_rpc_notify] 0-management: got RPC_CLNT_CONNECT
  71. [2011-09-12 19:50:23.270663] D [glusterd-handshake.c:308:glusterd_set_clnt_mgmt_program] 0-: GF-DUMP (123451501:1) not supported
  72. [2011-09-12 19:50:23.270686] I [glusterd-handshake.c:317:glusterd_set_clnt_mgmt_program] 0-: Using Program glusterd clnt mgmt, Num (1238433), Version (1)
  73. [2011-09-12 19:50:33.784762] D [socket.c:193:__socket_rwv] 0-management: EOF from peer 10.3.0.127:24007
  74. [2011-09-12 19:50:33.784828] W [socket.c:1494:__socket_proto_state_machine] 0-management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.3.0.127:24007)
  75. [2011-09-12 19:50:33.784846] D [socket.c:1768:socket_event_handler] 0-transport: disconnecting now
  76. [2011-09-12 19:50:33.784961] D [glusterd-handler.c:3867:glusterd_peer_rpc_notify] 0-management: got RPC_CLNT_DISCONNECT
  77. [2011-09-12 19:50:33.784983] D [socket.c:193:__socket_rwv] 0-socket.management: EOF from peer 10.3.0.127:1020
  78. [2011-09-12 19:50:33.784997] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.3.0.127:1020)
  79. [2011-09-12 19:50:33.785010] D [socket.c:1768:socket_event_handler] 0-transport: disconnecting now
  80. [2011-09-12 19:50:35.269991] D [common-utils.c:151:gf_resolve_ip6] 0-resolver: returning ip-10.3.0.127 (port-24007) for hostname: 10.3.0.127 and port: 24007
  81. [2011-09-12 19:50:38.267331] E [socket.c:1685:socket_connect_finish] 0-management: connection to 10.3.0.127:24007 failed (Connection refused)
  82. [2011-09-12 19:50:38.267368] D [socket.c:289:__socket_disconnect] 0-management: shutdown() returned -1. Transport endpoint is not connected
  83. [2011-09-12 19:50:38.267380] D [glusterd-handler.c:3867:glusterd_peer_rpc_notify] 0-management: got RPC_CLNT_DISCONNECT
  84. [2011-09-12 19:50:38.267394] D [socket.c:193:__socket_rwv] 0-management: EOF from peer 10.3.0.127:24007
  85. [2011-09-12 19:50:38.267401] D [socket.c:1494:__socket_proto_state_machine] 0-management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.3.0.127:24007)
  86. [2011-09-12 19:50:38.267408] D [socket.c:1768:socket_event_handler] 0-transport: disconnecting now
  87. [2011-09-12 19:50:38.267431] D [glusterd-handler.c:3867:glusterd_peer_rpc_notify] 0-management: got RPC_CLNT_DISCONNECT
  88. [2011-09-12 19:50:38.270618] D [common-utils.c:151:gf_resolve_ip6] 0-resolver: returning ip-10.3.0.127 (port-24007) for hostname: 10.3.0.127 and port: 24007
  89. [2011-09-12 19:50:38.271134] D [socket.c:289:__socket_disconnect] 0-management: shutdown() returned -1. Transport endpoint is not connected
  90. [2011-09-12 19:50:38.271149] D [glusterd-handler.c:3867:glusterd_peer_rpc_notify] 0-management: got RPC_CLNT_DISCONNECT
  91. [2011-09-12 19:50:38.271157] D [socket.c:193:__socket_rwv] 0-management: EOF from peer 10.3.0.127:24007
  92. [2011-09-12 19:50:38.271164] D [socket.c:1494:__socket_proto_state_machine] 0-management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.3.0.127:24007)
  93. [2011-09-12 19:50:38.271170] D [socket.c:1768:socket_event_handler] 0-transport: disconnecting now
  94. [2011-09-12 19:50:38.271182] D [glusterd-handler.c:3867:glusterd_peer_rpc_notify] 0-management: got RPC_CLNT_DISCONNECT
  95. [2011-09-12 19:50:41.270969] D [common-utils.c:151:gf_resolve_ip6] 0-resolver: returning ip-10.3.0.127 (port-24007) for hostname: 10.3.0.127 and port: 24007
  96. [2011-09-12 19:50:41.271391] D [socket.c:289:__socket_disconnect] 0-management: shutdown() returned -1. Transport endpoint is not connected
  97. [2011-09-12 19:50:41.271408] D [glusterd-handler.c:3867:glusterd_peer_rpc_notify] 0-management: got RPC_CLNT_DISCONNECT
  98. [2011-09-12 19:50:41.271422] D [socket.c:193:__socket_rwv] 0-management: EOF from peer 10.3.0.127:24007
  99. [2011-09-12 19:50:41.271428] D [socket.c:1494:__socket_proto_state_machine] 0-management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.3.0.127:24007)
  100. [2011-09-12 19:50:41.271435] D [socket.c:1768:socket_event_handler] 0-transport: disconnecting now
  101. [2011-09-12 19:50:41.271450] D [glusterd-handler.c:3867:glusterd_peer_rpc_notify] 0-management: got RPC_CLNT_DISCONNECT
  102. ^C[2011-09-12 19:50:41.699285] W [glusterfsd.c:727:cleanup_and_exit] (-->/opt/glusterfs/3.2.3/lib64/libglusterfs.so.0(+0x3f360) [0x7f984b34d360] (-->/lib64/libc.so.6(epoll_wait+0x33) [0x7f984a837ed3] (-->/lib64/libc.so.6(+0x329e0) [0x7f984a7959e0]))) 0-: received signum (2), shutting down
  103. [2011-09-12 19:50:41.699329] D [glusterfsd-mgmt.c:1076:glusterfs_mgmt_pmap_signout] 0-fsd-mgmt: portmapper signout arguments not given
  104.  
  105.  
  106. # this is on the PROBING machine (the one that segfaults)
  107.  
  108. # gluster peer probe 10.3.0.128
  109. Connection failed. Please check if gluster daemon is operational.
  110.  
  111. # the log is:
  112.  
  113. [2011-09-12 19:50:13.18700] I [glusterd.c:564:init] 0-management: Using /etc/glusterd as working directory
  114. [2011-09-12 19:50:13.19515] C [rdma.c:3941:rdma_init] 0-rpc-transport/rdma: No IB devices found
  115. [2011-09-12 19:50:13.19768] E [rdma.c:4812:init] 0-rdma.management: Failed to initialize IB Device
  116. [2011-09-12 19:50:13.19787] E [rpc-transport.c:741:rpc_transport_load] 0-rpc-transport: 'rdma' initialization failed
  117. [2011-09-12 19:50:13.19797] W [rpcsvc.c:1288:rpcsvc_transport_create] 0-rpc-service: cannot create listener, initing the transport failed
  118. [2011-09-12 19:50:13.19824] E [glusterd-store.c:1147:glusterd_store_handle_retrieve] 0-glusterd: Unable to retrieve store handle for /etc/glusterd/glusterd.info, error: No such file or directory
  119. [2011-09-12 19:50:13.19833] E [glusterd-store.c:1248:glusterd_retrieve_uuid] 0-: Unable to get store handle!
  120. [2011-09-12 19:50:13.19863] I [glusterd.c:96:glusterd_uuid_init] 0-glusterd: generated UUID: f41cc69f-40f4-4d9b-833d-7f00e726e81b
  121. Given volfile:
  122. +------------------------------------------------------------------------------+
  123. 1: volume management
  124. 2: type mgmt/glusterd
  125. 3: option working-directory /etc/glusterd
  126. 4: option transport-type socket,rdma
  127. 5: option transport.socket.keepalive-time 10
  128. 6: option transport.socket.keepalive-interval 2
  129. 7: end-volume
  130. 8:
  131.  
  132. +------------------------------------------------------------------------------+
  133. [2011-09-12 19:50:15.603827] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (10.3.0.128:1023)
  134. [2011-09-12 19:50:32.898849] I [glusterd-handler.c:750:glusterd_handle_cli_list_friends] 0-glusterd: Received cli list req
  135. [2011-09-12 19:50:32.900181] W [socket.c:1494:__socket_proto_state_machine] 0-socket.management: reading from socket failed. Error (Transport endpoint is not connected), peer (127.0.0.1:1023)
  136. [2011-09-12 19:50:34.973877] I [glusterd-handler.c:623:glusterd_handle_cli_probe] 0-glusterd: Received CLI probe req 10.3.0.128 24007
  137. [2011-09-12 19:50:34.975956] I [glusterd-handler.c:391:glusterd_friend_find] 0-glusterd: Unable to find hostname: 10.3.0.128
  138. [2011-09-12 19:50:34.975983] I [glusterd-handler.c:3422:glusterd_probe_begin] 0-glusterd: Unable to find peerinfo for host: 10.3.0.128 (24007)
  139. [2011-09-12 19:50:34.979775] I [glusterd-handler.c:3404:glusterd_friend_add] 0-glusterd: connect returned 0
  140. [2011-09-12 19:50:34.980315] I [glusterd-handshake.c:317:glusterd_set_clnt_mgmt_program] 0-: Using Program glusterd clnt mgmt, Num (1238433), Version (1)
  141. pending frames:
  142.  
  143. patchset: git://git.gluster.com/glusterfs.git
  144. signal received: 11
  145. time of crash: 2011-09-12 19:50:34
  146. configuration details:
  147. argp 1
  148. backtrace 1
  149. dlfcn 1
  150. fdatasync 1
  151. libpthread 1
  152. llistxattr 1
  153. setfsid 1
  154. spinlock 1
  155. epoll.h 1
  156. xattr.h 1
  157. st_atim.tv_nsec 1
  158. package-string: glusterfs 3.2.3
  159. /lib64/libc.so.6(+0x329e0)[0x7f8c3a1399e0]
  160. /opt/glusterfs/3.2.3/lib64/libgfrpc.so.0(rpc_transport_connect+0xc)[0x7f8c3aaa5c5c]
  161. /opt/glusterfs/3.2.3/lib64/libgfrpc.so.0(rpc_clnt_submit+0x498)[0x7f8c3aaab848]
  162. /opt/glusterfs/3.2.3/lib64/glusterfs/3.2.3/xlator/mgmt/glusterd.so(glusterd_submit_request+0x15b)[0x7f8c3892e35b]
  163. /opt/glusterfs/3.2.3/lib64/glusterfs/3.2.3/xlator/mgmt/glusterd.so(glusterd3_1_probe+0x175)[0x7f8c38934a75]
  164. /opt/glusterfs/3.2.3/lib64/glusterfs/3.2.3/xlator/mgmt/glusterd.so(+0x300e1)[0x7f8c3890f0e1]
  165. /opt/glusterfs/3.2.3/lib64/glusterfs/3.2.3/xlator/mgmt/glusterd.so(glusterd_friend_sm+0x175)[0x7f8c3890f265]
  166. /opt/glusterfs/3.2.3/lib64/glusterfs/3.2.3/xlator/mgmt/glusterd.so(glusterd_peer_dump_version_cbk+0x3b1)[0x7f8c38942691]
  167. /opt/glusterfs/3.2.3/lib64/libgfrpc.so.0(rpc_clnt_handle_reply+0xa4)[0x7f8c3aaaa9f4]
  168. /opt/glusterfs/3.2.3/lib64/libgfrpc.so.0(rpc_clnt_notify+0xc8)[0x7f8c3aaaac38]
  169. /opt/glusterfs/3.2.3/lib64/libgfrpc.so.0(rpc_transport_notify+0x27)[0x7f8c3aaa5ae7]
  170. /opt/glusterfs/3.2.3/lib64/glusterfs/3.2.3/rpc-transport/socket.so(socket_event_poll_in+0x3f)[0x7f8c386d77df]
  171. /opt/glusterfs/3.2.3/lib64/glusterfs/3.2.3/rpc-transport/socket.so(socket_event_handler+0x148)[0x7f8c386d7948]
  172. /opt/glusterfs/3.2.3/lib64/libglusterfs.so.0(+0x3f427)[0x7f8c3acf1427]
  173. /opt/glusterfs/3.2.3/sbin/glusterd(main+0x25a)[0x4063ca]
  174. /lib64/libc.so.6(__libc_start_main+0xe6)[0x7f8c3a125bc6]
  175. /opt/glusterfs/3.2.3/sbin/glusterd[0x403bf9]
  176. ---------
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!