[2019-05-08 09:17:36.820168] I [glusterd-utils.c:6312:glusterd_brick_start] 0-management: starting a fresh brick process for brick /data/nvme1n1
[2019-05-08 09:17:36.936897] I [MSGID: 106142] [glusterd-pmap.c:290:pmap_registry_bind] 0-pmap: adding brick /data/nvme1n1 on port 49152
[2019-05-08 09:17:36.937437] I [rpc-clnt.c:1005:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2019-05-08 09:17:36.937694] I [glusterd-utils.c:6312:glusterd_brick_start] 0-management: starting a fresh brick process for brick /data/nvme2n1
[2019-05-08 09:17:36.980696] I [MSGID: 106142] [glusterd-pmap.c:290:pmap_registry_bind] 0-pmap: adding brick /data/nvme2n1 on port 49153
[2019-05-08 09:17:36.981256] I [rpc-clnt.c:1005:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2019-05-08 09:17:36.981565] I [glusterd-utils.c:6312:glusterd_brick_start] 0-management: starting a fresh brick process for brick /data/nvme3n1
[2019-05-08 09:17:37.022751] I [MSGID: 106142] [glusterd-pmap.c:290:pmap_registry_bind] 0-pmap: adding brick /data/nvme3n1 on port 49154
[2019-05-08 09:17:37.023338] I [rpc-clnt.c:1005:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2019-05-08 09:17:37.023638] I [glusterd-utils.c:6312:glusterd_brick_start] 0-management: starting a fresh brick process for brick /data/nvme4n1
[2019-05-08 09:17:37.061940] I [MSGID: 106142] [glusterd-pmap.c:290:pmap_registry_bind] 0-pmap: adding brick /data/nvme4n1 on port 49155
[2019-05-08 09:17:37.062475] I [rpc-clnt.c:1005:rpc_clnt_connection_init] 0-management: setting frame-timeout to 600
[2019-05-08 09:17:37.145519] E [mem-pool.c:351:__gf_free] (-->/usr/lib64/glusterfs/6.1/xlator/mgmt/glusterd.so(+0x4d9ba) [0x7f5d5b3499ba] -->/usr/lib64/glusterfs/6.1/xlator/mgmt/glusterd.so(+0x4d75d) [0x7f5d5b34975d] -->/lib64/libglusterfs.so.0(__gf_free+0x24e) [0x7f5d672203fe] ) 0-: Assertion failed: mem_acct->rec[header->type].size >= header->size
[2019-05-08 09:17:37.145822] I [rpc-clnt.c:1005:rpc_clnt_connection_init] 0-snapd: setting frame-timeout to 600
[2019-05-08 09:17:37.146202] I [rpc-clnt.c:1005:rpc_clnt_connection_init] 0-gfproxyd: setting frame-timeout to 600
[2019-05-08 09:17:37.146635] I [rpc-clnt.c:1005:rpc_clnt_connection_init] 0-nfs: setting frame-timeout to 600
[2019-05-08 09:17:37.146857] I [MSGID: 106131] [glusterd-proc-mgmt.c:86:glusterd_proc_stop] 0-management: nfs already stopped
[2019-05-08 09:17:37.146913] I [MSGID: 106568] [glusterd-svc-mgmt.c:253:glusterd_svc_stop] 0-management: nfs service is stopped
[2019-05-08 09:17:37.146950] I [MSGID: 106599] [glusterd-nfs-svc.c:81:glusterd_nfssvc_manager] 0-management: nfs/server.so xlator is not installed
[2019-05-08 09:17:37.147052] I [rpc-clnt.c:1005:rpc_clnt_connection_init] 0-glustershd: setting frame-timeout to 600
[2019-05-08 09:17:37.148860] I [MSGID: 106131] [glusterd-proc-mgmt.c:86:glusterd_proc_stop] 0-management: glustershd already stopped
[2019-05-08 09:17:37.148910] I [MSGID: 106568] [glusterd-svc-mgmt.c:253:glusterd_svc_stop] 0-management: glustershd service is stopped
[2019-05-08 09:17:37.148961] I [MSGID: 106567] [glusterd-svc-mgmt.c:220:glusterd_svc_start] 0-management: Starting glustershd service
[2019-05-08 09:17:38.152360] I [rpc-clnt.c:1005:rpc_clnt_connection_init] 0-quotad: setting frame-timeout to 600
[2019-05-08 09:17:38.152942] I [rpc-clnt.c:1005:rpc_clnt_connection_init] 0-bitd: setting frame-timeout to 600
[2019-05-08 09:17:38.153266] I [MSGID: 106131] [glusterd-proc-mgmt.c:86:glusterd_proc_stop] 0-management: bitd already stopped
[2019-05-08 09:17:38.153309] I [MSGID: 106568] [glusterd-svc-mgmt.c:253:glusterd_svc_stop] 0-management: bitd service is stopped
[2019-05-08 09:17:38.153366] I [rpc-clnt.c:1005:rpc_clnt_connection_init] 0-scrub: setting frame-timeout to 600
[2019-05-08 09:17:38.153686] I [MSGID: 106131] [glusterd-proc-mgmt.c:86:glusterd_proc_stop] 0-management: scrub already stopped
[2019-05-08 09:17:38.153721] I [MSGID: 106568] [glusterd-svc-mgmt.c:253:glusterd_svc_stop] 0-management: scrub service is stopped
pending frames:
frame : type(0) op(0)
frame : type(0) op(0)
patchset: git://git.gluster.org/glusterfs.git
signal received: 11
time of crash:
2019-05-08 09:17:38
configuration details:
argp 1
backtrace 1
dlfcn 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 6.1
/lib64/libglusterfs.so.0(+0x26db0)[0x7f5d671f3db0]
/lib64/libglusterfs.so.0(gf_print_trace+0x334)[0x7f5d671fe7b4]
/lib64/libc.so.6(+0x36280)[0x7f5d65833280]
/lib64/libglusterfs.so.0(iobref_ref+0xd)[0x7f5d6722505d]
/usr/lib64/glusterfs/6.1/rpc-transport/rdma.so(gf_rdma_ioq_new+0x21e)[0x7f5d58c928de]
/usr/lib64/glusterfs/6.1/rpc-transport/rdma.so(gf_rdma_submit_reply+0x75)[0x7f5d58c92c35]
/lib64/libgfrpc.so.0(+0x66d2)[0x7f5d66f956d2]
/lib64/libgfrpc.so.0(rpcsvc_submit_generic+0x180)[0x7f5d66f97380]
/lib64/libgfrpc.so.0(+0x89f3)[0x7f5d66f979f3]
/lib64/libgfrpc.so.0(+0x95e5)[0x7f5d66f985e5]
/lib64/libgfrpc.so.0(+0x995b)[0x7f5d66f9895b]
/lib64/libgfrpc.so.0(rpc_transport_notify+0x23)[0x7f5d66f9a9f3]
/usr/lib64/glusterfs/6.1/rpc-transport/rdma.so(gf_rdma_pollin_notify+0xb6)[0x7f5d58c93c06]
/usr/lib64/glusterfs/6.1/rpc-transport/rdma.so(gf_rdma_process_recv+0x2e9)[0x7f5d58c94369]
/usr/lib64/glusterfs/6.1/rpc-transport/rdma.so(+0xc6b2)[0x7f5d58c946b2]
/lib64/libpthread.so.0(+0x7dd5)[0x7f5d66033dd5]
/lib64/libc.so.6(clone+0x6d)[0x7f5d658faead]
---------