[2019-03-05 06:06:13.499827] W [socket.c:719:__socket_rwv] 0-glusterfs: readv on 78.47.202.50:24007 failed (No data available)
[2019-03-05 06:06:13.499883] I [glusterfsd-mgmt.c:2424:mgmt_rpc_notify] 0-glusterfsd-mgmt: disconnected from remote-host: gluster1
[2019-03-05 06:06:13.549775] I [MSGID: 115036] [server.c:469:server_rpc_notify] 0-persistent-server: disconnecting connection from CTX_ID:b0673da6-c472-407c-be24-98b4909cc7e0-GRAPH_ID:0-PID:889-HOST:gluster1-PC_NAME:persistent-client-0-RECON_NO:-0
[2019-03-05 06:06:13.550273] I [MSGID: 101055] [client_t.c:435:gf_client_unref] 0-persistent-server: Shutting down connection CTX_ID:b0673da6-c472-407c-be24-98b4909cc7e0-GRAPH_ID:0-PID:889-HOST:gluster1-PC_NAME:persistent-client-0-RECON_NO:-0
[2019-03-05 06:06:23.782417] I [glusterfsd-mgmt.c:2005:mgmt_getspec_cbk] 0-glusterfs: No change in volfile,continuing
[2019-03-05 06:06:25.905118] I [addr.c:54:compare_addr_and_update] 0-/gluster/md3/persistent: allowed = "*", received addr = "EXTERNAL-IP"
[2019-03-05 06:06:25.905180] I [login.c:110:gf_auth] 0-auth/login: allowed user names: 9695945f-59bf-40b7-8d71-b35ec56652df
[2019-03-05 06:06:25.905201] I [MSGID: 115029] [server-handshake.c:537:server_setvolume] 0-persistent-server: accepted client from CTX_ID:b9993336-e501-49da-97ac-4bd621859512-GRAPH_ID:0-PID:17670-HOST:gluster1-PC_NAME:persistent-client-0-RECON_NO:-0 (version: 5.4)
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 18854 times between [2019-03-05 06:06:13.385624] and [2019-03-05 06:08:09.188968]
[2019-03-05 06:08:09.903525] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
[2019-03-05 06:08:12.816192] I [glusterfsd-mgmt.c:2444:mgmt_rpc_notify] 0-glusterfsd-mgmt: Exhausted all volfile servers
[2019-03-05 06:08:12.846761] I [MSGID: 115036] [server.c:469:server_rpc_notify] 0-persistent-server: disconnecting connection from CTX_ID:b9993336-e501-49da-97ac-4bd621859512-GRAPH_ID:0-PID:17670-HOST:gluster1-PC_NAME:persistent-client-0-RECON_NO:-0
[2019-03-05 06:08:12.847233] I [MSGID: 101055] [client_t.c:435:gf_client_unref] 0-persistent-server: Shutting down connection CTX_ID:b9993336-e501-49da-97ac-4bd621859512-GRAPH_ID:0-PID:17670-HOST:gluster1-PC_NAME:persistent-client-0-RECON_NO:-0
[2019-03-05 06:08:23.795427] I [glusterfsd-mgmt.c:2005:mgmt_getspec_cbk] 0-glusterfs: No change in volfile,continuing
[2019-03-05 06:08:24.901340] I [addr.c:54:compare_addr_and_update] 0-/gluster/md3/persistent: allowed = "*", received addr = "EXTERNAL-IP"
[2019-03-05 06:08:24.901383] I [login.c:110:gf_auth] 0-auth/login: allowed user names: 9695945f-59bf-40b7-8d71-b35ec56652df
[2019-03-05 06:08:24.901402] I [MSGID: 115029] [server-handshake.c:537:server_setvolume] 0-persistent-server: accepted client from CTX_ID:93600008-b90a-4ba6-b14b-ab104a6ec9fb-GRAPH_ID:0-PID:18059-HOST:gluster1-PC_NAME:persistent-client-0-RECON_NO:-0 (version: 5.4)
[2019-03-05 06:08:40.726355] I [MSGID: 115036] [server.c:469:server_rpc_notify] 0-persistent-server: disconnecting connection from CTX_ID:93600008-b90a-4ba6-b14b-ab104a6ec9fb-GRAPH_ID:0-PID:18059-HOST:gluster1-PC_NAME:persistent-client-0-RECON_NO:-0
[2019-03-05 06:08:40.726728] I [MSGID: 101055] [client_t.c:435:gf_client_unref] 0-persistent-server: Shutting down connection CTX_ID:93600008-b90a-4ba6-b14b-ab104a6ec9fb-GRAPH_ID:0-PID:18059-HOST:gluster1-PC_NAME:persistent-client-0-RECON_NO:-0
The message "E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler" repeated 2609 times between [2019-03-05 06:08:09.903525] and [2019-03-05 06:08:42.275766]
[2019-03-05 06:08:43.082823] W [glusterfsd.c:1500:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x74a4) [0x7f5b54ab64a4] -->/usr/sbin/glusterfsd(glusterfs_sigwaiter+0xfd) [0x5566ba93c91d] -->/usr/sbin/glusterfsd(cleanup_and_exit+0x54) [0x5566ba93c774] ) 0-: received signum (15), shutting down
[2019-03-05 06:08:43.113687] E [socket.c:3466:socket_connect] 0-glusterfs: connection attempt on EXTERNAL-IP:24007 failed, (Network is unreachable)
[2019-03-05 06:08:43.113720] W [rpc-clnt.c:1683:rpc_clnt_submit] 0-glusterfs: error returned while attempting to connect to host:(null), port:0
[2019-03-05 06:08:43.113731] I [socket.c:3710:socket_submit_outgoing_msg] 0-glusterfs: not connected (priv->connected = 0)
[2019-03-05 06:08:43.113739] W [rpc-clnt.c:1695:rpc_clnt_submit] 0-glusterfs: failed to submit rpc-request (unique: 0, XID: 0xd Program: Gluster Portmap, ProgVers: 1, Proc: 5) to rpc-transport (glusterfs)
[2019-03-05 06:08:43.113798] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler