API tools faq
paste
wgseligman

Nevis log file excerpt

wgseligman
Mar 2nd, 2012
181
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 25.65 KB | None | 0 0
raw download clone embed print report
  1. Mar 2 17:17:00 orestes-tb cib: [2997]: info: crm_get_peer: Node hypatia-tb.nevis.columbia.edu now has id: 1
  2. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: update_dc: Set DC to hypatia-tb.nevis.columbia.edu (3.0.5)
  3. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: update_attrd: Connecting to attrd...
  4. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: do_state_transition: State transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_j
  5. oin_finalize_respond ]
  6. Mar 2 17:17:00 orestes-tb attrd: [2999]: notice: attrd_local_callback: Sending full refresh (origin=crmd)
  7. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: erase_xpath_callback: Deletion of "//node_state[@uname='orestes-tb.nevis.columbia.edu']/transient_attributes":
  8. ok (rc=0)
  9. Mar 2 17:17:00 orestes-tb lrmd: [2998]: notice: lrmd_rsc_new(): No lrm_rprovider field in message
  10. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=32:0:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=Clvmd:0_monitor_0 )
  11. Mar 2 17:17:00 orestes-tb lrmd: [2998]: info: rsc:Clvmd:0:2: probe
  12. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=33:0:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=AdminDrbd:1_monitor_0 )
  13. Mar 2 17:17:00 orestes-tb lrmd: [2998]: info: rsc:AdminDrbd:1:3: probe
  14. Mar 2 17:17:00 orestes-tb lrmd: [2998]: notice: lrmd_rsc_new(): No lrm_rprovider field in message
  15. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=34:0:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=Gfs2:0_monitor_0 )
  16. Mar 2 17:17:00 orestes-tb lrmd: [2998]: info: rsc:Gfs2:0:4: probe
  17. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=35:0:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=CronAmbientTemperature_monitor_0
  18. )
  19. Mar 2 17:17:00 orestes-tb lrmd: [2998]: info: rsc:CronAmbientTemperature:5: probeMar 2 17:17:00 orestes-tb lrmd: [2998]: notice: lrmd_rsc_new(): No lrm_rprovider field in message
  20. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=36:0:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=StonithHypatia_monitor_0 )Mar 2 17:17:00 orestes-tb lrmd: [2998]: notice: lrmd_rsc_new(): No lrm_rprovider field in message
  21. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=37:0:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=StonithOrestes_monitor_0 )
  22. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=38:0:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=SymlinkDhcpdConf_monitor_0 )
  23. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=39:0:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=SymlinkSysconfigDhcpd_monitor_0 )
  24. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=40:0:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=SymlinkDhcpdLeases_monitor_0 )
  25. Mar 2 17:17:00 orestes-tb lrmd: [2998]: notice: lrmd_rsc_new(): No lrm_rprovider field in message
  26. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=41:0:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=Dhcpd_monitor_0 )
  27. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=42:0:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=SymlinkUsrShareCups:0_monitor_0 )
  28. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=43:0:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=SymlinkCupsdConf:0_monitor_0 )
  29. Mar 2 17:17:00 orestes-tb lrmd: [2998]: notice: lrmd_rsc_new(): No lrm_rprovider field in message
  30. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=44:0:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=Cups:0_monitor_0 )
  31. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=45:0:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=ClusterIP:0_monitor_0 )
  32. Mar 2 17:17:00 orestes-tb lrmd: [2998]: info: rsc:ClusterIP:0:15: probe
  33. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=46:0:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=ClusterIPLocal:0_monitor_0 )
  34. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation Gfs2:0_monitor_0 (call=4, rc=7, cib-update=8, confirmed=true) not running
  35. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=47:0:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=ClusterIPSandbox:0_monitor_0 )
  36. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=48:0:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=ClusterIP:1_monitor_0 )
  37. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation Clvmd:0_monitor_0 (call=2, rc=7, cib-update=9, confirmed=true) not running
  38. Mar 2 17:17:00 orestes-tb symlink[3024]: WARNING: /var/nevis/etc/cron.d/ambient-temperature does not exist!
  39. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=49:0:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=ClusterIPLocal:1_monitor_0 )
  40. Mar 2 17:17:00 orestes-tb lrmd: [2998]: info: rsc:ClusterIPLocal:1:19: probe
  41. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation CronAmbientTemperature_monitor_0 (call=5, rc=7, cib-update=10, confirmed=true) not running
  42. Mar 2 17:17:00 orestes-tb crm_attribute: [3127]: info: Invoked: crm_attribute -N orestes-tb.nevis.columbia.edu -n master-AdminDrbd:1 -l reboot -D
  43. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation AdminDrbd:1_monitor_0 (call=3, rc=7, cib-update=11, confirmed=true) not running
  44. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation ClusterIPLocal:1_monitor_0 (call=19, rc=7, cib-update=12, confirmed=true) not running
  45. Mar 2 17:17:00 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation ClusterIP:0_monitor_0 (call=15, rc=7, cib-update=13, confirmed=true) not running
  46. Mar 2 17:17:01 orestes-tb lrmd: [2998]: info: rsc:StonithHypatia:6: probe
  47. Mar 2 17:17:01 orestes-tb lrmd: [2998]: info: rsc:StonithOrestes:7: probe
  48. Mar 2 17:17:01 orestes-tb lrmd: [2998]: info: rsc:SymlinkDhcpdConf:8: probe
  49. Mar 2 17:17:01 orestes-tb stonith-ng: [2996]: notice: stonith_device_action: Device StonithHypatia not found
  50. Mar 2 17:17:01 orestes-tb stonith-ng: [2996]: info: stonith_command: Processed st_execute from lrmd: rc=-12
  51. Mar 2 17:17:01 orestes-tb stonith-ng: [2996]: notice: stonith_device_action: Device StonithOrestes not found
  52. Mar 2 17:17:01 orestes-tb stonith-ng: [2996]: info: stonith_command: Processed st_execute from lrmd: rc=-12
  53. Mar 2 17:17:01 orestes-tb lrmd: [2998]: info: rsc:SymlinkSysconfigDhcpd:9: probe
  54. Mar 2 17:17:01 orestes-tb lrmd: [2998]: info: rsc:SymlinkDhcpdLeases:10: probe
  55. Mar 2 17:17:01 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation StonithHypatia_monitor_0 (call=6, rc=7, cib-update=14, confirmed=true) not running
  56. Mar 2 17:17:01 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation StonithOrestes_monitor_0 (call=7, rc=7, cib-update=15, confirmed=true) not running
  57. Mar 2 17:17:01 orestes-tb lrmd: [2998]: info: rsc:Cups:0:14: probe
  58. Mar 2 17:17:01 orestes-tb symlink[3158]: WARNING: /var/nevis/etc/dhcpd.conf does not exist!
  59. Mar 2 17:17:01 orestes-tb symlink[3160]: WARNING: /var/nevis/dhcpd does not exist!
  60. Mar 2 17:17:01 orestes-tb symlink[3159]: WARNING: /var/nevis/etc/sysconfig/dhcpd does not exist!
  61. Mar 2 17:17:01 orestes-tb lrmd: [2998]: info: rsc:ClusterIPSandbox:0:17: probe
  62. Mar 2 17:17:01 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation SymlinkDhcpdConf_monitor_0 (call=8, rc=7, cib-update=16, confirmed=true) not running
  63. Mar 2 17:17:01 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation SymlinkSysconfigDhcpd_monitor_0 (call=9, rc=7, cib-update=17, confirmed=true) not running
  64. Mar 2 17:17:01 orestes-tb lrmd: [2998]: info: rsc:ClusterIP:1:18: probe
  65. Mar 2 17:17:01 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation SymlinkDhcpdLeases_monitor_0 (call=10, rc=7, cib-update=18, confirmed=true) not running
  66. Mar 2 17:17:01 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation Cups:0_monitor_0 (call=14, rc=7, cib-update=19, confirmed=true) not running
  67. Mar 2 17:17:01 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation ClusterIPSandbox:0_monitor_0 (call=17, rc=7, cib-update=20, confirmed=true) not running
  68. Mar 2 17:17:01 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation ClusterIP:1_monitor_0 (call=18, rc=7, cib-update=21, confirmed=true) not running
  69. Mar 2 17:17:02 orestes-tb lrmd: [2998]: info: rsc:Dhcpd:11: probe
  70. Mar 2 17:17:02 orestes-tb lrmd: [2998]: info: rsc:SymlinkUsrShareCups:0:12: probe
  71. Mar 2 17:17:02 orestes-tb lrmd: [2998]: info: rsc:SymlinkCupsdConf:0:13: probe
  72. Mar 2 17:17:02 orestes-tb lrmd: [2998]: info: rsc:ClusterIPLocal:0:16: probe
  73. Mar 2 17:17:02 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation Dhcpd_monitor_0 (call=11, rc=7, cib-update=22, confirmed=true) not running
  74. Mar 2 17:17:02 orestes-tb symlink[3278]: WARNING: /var/nevis/share/cups does not exist!
  75. Mar 2 17:17:02 orestes-tb symlink[3279]: WARNING: /var/nevis/etc/cups/cupsd.conf does not exist!
  76. Mar 2 17:17:02 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation SymlinkUsrShareCups:0_monitor_0 (call=12, rc=7, cib-update=23, confirmed=true) not running
  77. Mar 2 17:17:02 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation SymlinkCupsdConf:0_monitor_0 (call=13, rc=7, cib-update=24, confirmed=true) not running
  78. Mar 2 17:17:02 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation ClusterIPLocal:0_monitor_0 (call=16, rc=7, cib-update=25, confirmed=true) not running
  79. Mar 2 17:17:02 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=68:1:0:81f50bd7-02f3-422f-a577-8928f23483a6 op=StonithHypatia_start_0 )
  80. Mar 2 17:17:02 orestes-tb lrmd: [2998]: info: rsc:StonithHypatia:20: start
  81. Mar 2 17:17:02 orestes-tb stonith-ng: [2996]: info: stonith_device_register: Added 'StonithHypatia' to the device list (1 active devices)
  82. Mar 2 17:17:02 orestes-tb stonith-ng: [2996]: info: stonith_command: Processed st_device_register from lrmd: rc=0
  83. Mar 2 17:17:02 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=14:1:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=ClusterIPSandbox:1_monitor_0 )
  84. Mar 2 17:17:02 orestes-tb lrmd: [2998]: info: rsc:ClusterIPSandbox:1:21: probe
  85. Mar 2 17:17:02 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=15:1:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=SymlinkEtcLibvirt:0_monitor_0 )
  86. Mar 2 17:17:02 orestes-tb lrmd: [2998]: info: rsc:SymlinkEtcLibvirt:0:22: probe
  87. Mar 2 17:17:02 orestes-tb lrmd: [2998]: notice: lrmd_rsc_new(): No lrm_rprovider field in message
  88. Mar 2 17:17:02 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=16:1:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=Libvirtd:0_monitor_0 )
  89. Mar 2 17:17:02 orestes-tb lrmd: [2998]: info: rsc:Libvirtd:0:23: probe
  90. Mar 2 17:17:02 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=17:1:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=SymlinkEtcHosts:0_monitor_0 )
  91. Mar 2 17:17:02 orestes-tb lrmd: [2998]: info: rsc:SymlinkEtcHosts:0:24: probe
  92. Mar 2 17:17:02 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=18:1:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=SymlinkNtpConf:0_monitor_0 )
  93. Mar 2 17:17:02 orestes-tb lrmd: [2998]: notice: lrmd_rsc_new(): No lrm_rprovider field in message
  94. Mar 2 17:17:02 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=19:1:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=Ntpd:0_monitor_0 )
  95. Mar 2 17:17:02 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=20:1:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=SymlinkTftp:0_monitor_0 )
  96. Mar 2 17:17:02 orestes-tb lrmd: [2998]: notice: lrmd_rsc_new(): No lrm_rprovider field in message
  97. Mar 2 17:17:02 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=21:1:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=Xinetd:0_monitor_0 )
  98. Mar 2 17:17:02 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=22:1:7:81f50bd7-02f3-422f-a577-8928f23483a6 op=ExportMail:0_monitor_0 )
  99. Mar 2 17:17:02 orestes-tb lrmd: [2998]: info: stonith_api_device_metadata: looking up fence_nut/redhat metadata
  100. Mar 2 17:17:02 orestes-tb symlink[3346]: WARNING: /var/nevis/etc/libvirt does not exist!
  101. Mar 2 17:17:02 orestes-tb symlink[3348]: WARNING: /var/nevis/etc/hosts does not exist!
  102. Mar 2 17:17:02 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation StonithHypatia_start_0 (call=20, rc=0, cib-update=26, confirmed=true) ok
  103. Mar 2 17:17:02 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation SymlinkEtcLibvirt:0_monitor_0 (call=22, rc=7, cib-update=27, confirmed=true) not running
  104. Mar 2 17:17:02 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation SymlinkEtcHosts:0_monitor_0 (call=24, rc=7, cib-update=28, confirmed=true) not running
  105. Mar 2 17:17:02 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation Libvirtd:0_monitor_0 (call=23, rc=7, cib-update=29, confirmed=true) not running
  106. Mar 2 17:17:02 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation ClusterIPSandbox:1_monitor_0 (call=21, rc=7, cib-update=30, confirmed=true) not running
  107. Mar 2 17:17:02 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=34:1:0:81f50bd7-02f3-422f-a577-8928f23483a6 op=AdminDrbd:1_start_0 )
  108. Mar 2 17:17:02 orestes-tb lrmd: [2998]: info: rsc:AdminDrbd:1:30: start
  109. Mar 2 17:17:03 orestes-tb lrmd: [2998]: info: RA output: (AdminDrbd:1:start:stdout) allow-two-primaries;
  110. Mar 2 17:17:03 orestes-tb kernel: drbd: initialized. Version: 8.3.12 (api:88/proto:86-96)
  111. Mar 2 17:17:03 orestes-tb kernel: drbd: GIT-hash: e2a8ef4656be026bbae540305fcb998a5991090f build by root@hypatia-tb.nevis.columbia.edu, 2012-02-28 18:01:34
  112. Mar 2 17:17:03 orestes-tb kernel: drbd: registered as block device major 147
  113. Mar 2 17:17:03 orestes-tb kernel: drbd: minor_table @ 0xffff88021c1917c0
  114. Mar 2 17:17:03 orestes-tb lrmd: [2998]: info: RA output: (AdminDrbd:1:start:stdout)
  115. Mar 2 17:17:03 orestes-tb lrmd: [2998]: info: RA output: (AdminDrbd:1:start:stdout)
  116. Mar 2 17:17:03 orestes-tb lrmd: [2998]: info: RA output: (AdminDrbd:1:start:stdout)
  117. Mar 2 17:17:03 orestes-tb kernel: block drbd0: Starting worker thread (from cqueue [3441])
  118. Mar 2 17:17:03 orestes-tb kernel: block drbd0: disk( Diskless -> Attaching )
  119. Mar 2 17:17:03 orestes-tb kernel: block drbd0: Found 57 transactions (57 active extents) in activity log.
  120. Mar 2 17:17:03 orestes-tb kernel: block drbd0: Method to ensure write ordering: barrier
  121. Mar 2 17:17:03 orestes-tb kernel: block drbd0: max BIO size = 130560
  122. Mar 2 17:17:03 orestes-tb kernel: block drbd0: Adjusting my ra_pages to backing device's (32 -> 768)
  123. Mar 2 17:17:03 orestes-tb kernel: block drbd0: drbd_bm_resize called with capacity == 5611549368
  124. Mar 2 17:17:03 orestes-tb kernel: block drbd0: resync bitmap: bits=701443671 words=10960058 pages=21407
  125. Mar 2 17:17:03 orestes-tb kernel: block drbd0: size = 2676 GB (2805774684 KB)
  126. Mar 2 17:17:03 orestes-tb lrmd: [2998]: info: rsc:SymlinkNtpConf:0:25: probe
  127. Mar 2 17:17:03 orestes-tb lrmd: [2998]: info: rsc:Ntpd:0:26: probe
  128. Mar 2 17:17:03 orestes-tb lrmd: [2998]: info: rsc:SymlinkTftp:0:27: probe
  129. Mar 2 17:17:03 orestes-tb symlink[3467]: WARNING: /var/nevis/etc/ntp.conf does not exist!
  130. Mar 2 17:17:03 orestes-tb symlink[3469]: WARNING: /var/nevis/etc/xinetd.d/tftp does not exist!
  131. Mar 2 17:17:03 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation SymlinkTftp:0_monitor_0 (call=27, rc=7, cib-update=31, confirmed=true) not running
  132. Mar 2 17:17:03 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation SymlinkNtpConf:0_monitor_0 (call=25, rc=7, cib-update=32, confirmed=true) not running
  133. Mar 2 17:17:03 orestes-tb kernel: block drbd0: bitmap READ of 21407 pages took 631 jiffies
  134. Mar 2 17:17:04 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation Ntpd:0_monitor_0 (call=26, rc=7, cib-update=33, confirmed=true) not running
  135. Mar 2 17:17:04 orestes-tb kernel: block drbd0: recounting of set bits took additional 95 jiffies
  136. Mar 2 17:17:04 orestes-tb kernel: block drbd0: 0 KB (0 bits) marked out-of-sync by on disk bit-map.
  137. Mar 2 17:17:04 orestes-tb kernel: block drbd0: Marked additional 52 MB as out-of-sync based on AL.
  138. Mar 2 17:17:04 orestes-tb kernel: block drbd0: bitmap WRITE of 0 pages took 0 jiffies
  139. Mar 2 17:17:04 orestes-tb kernel: block drbd0: 52 MB (13312 bits) marked out-of-sync by on disk bit-map.
  140. Mar 2 17:17:04 orestes-tb kernel: block drbd0: disk( Attaching -> Consistent )
  141. Mar 2 17:17:04 orestes-tb kernel: block drbd0: attached to UUIDs 489E63EA01E0F333:0000000000000000:B758FBBC40B2620B:B757FBBC40B2620B
  142. Mar 2 17:17:04 orestes-tb lrmd: [2998]: info: RA output: (AdminDrbd:1:start:stdout)
  143. Mar 2 17:17:04 orestes-tb kernel: block drbd0: conn( StandAlone -> Unconnected )
  144. Mar 2 17:17:04 orestes-tb kernel: block drbd0: Starting receiver thread (from drbd0_worker [3463])
  145. Mar 2 17:17:04 orestes-tb kernel: block drbd0: receiver (re)started
  146. Mar 2 17:17:04 orestes-tb kernel: block drbd0: conn( Unconnected -> WFConnection )
  147. Mar 2 17:17:04 orestes-tb lrmd: [2998]: info: RA output: (AdminDrbd:1:start:stdout)
  148. Mar 2 17:17:04 orestes-tb attrd: [2999]: notice: attrd_trigger_update: Sending flush op to all hosts for: master-AdminDrbd:1 (5)
  149. Mar 2 17:17:04 orestes-tb attrd: [2999]: notice: attrd_perform_update: Sent update 5: master-AdminDrbd:1=5
  150. Mar 2 17:17:04 orestes-tb lrmd: [2998]: info: RA output: (AdminDrbd:1:start:stdout)
  151. Mar 2 17:17:04 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation AdminDrbd:1_start_0 (call=30, rc=0, cib-update=34, confirmed=true) ok
  152. Mar 2 17:17:04 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=208:1:0:81f50bd7-02f3-422f-a577-8928f23483a6 op=AdminDrbd:1_notify_0 )
  153. Mar 2 17:17:04 orestes-tb lrmd: [2998]: info: rsc:AdminDrbd:1:31: notify
  154. Mar 2 17:17:04 orestes-tb lrmd: [2998]: info: RA output: (AdminDrbd:1:notify:stdout)
  155. Mar 2 17:17:04 orestes-tb crmd: [3001]: info: send_direct_ack: ACK'ing resource op AdminDrbd:1_notify_0 from 208:1:0:81f50bd7-02f3-422f-a577-8928f23483a6: lrm_invoke-lrmd-1330726624-5
  156. Mar 2 17:17:04 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation AdminDrbd:1_notify_0 (call=31, rc=0, cib-update=0, confirmed=true) ok
  157. Mar 2 17:17:04 orestes-tb kernel: block drbd0: Handshake successful: Agreed network protocol version 96
  158. Mar 2 17:17:04 orestes-tb kernel: block drbd0: conn( WFConnection -> WFReportParams )
  159. Mar 2 17:17:04 orestes-tb kernel: block drbd0: Starting asender thread (from drbd0_receiver [3506])
  160. Mar 2 17:17:04 orestes-tb kernel: block drbd0: data-integrity-alg: <not-used>
  161. Mar 2 17:17:04 orestes-tb kernel: block drbd0: drbd_sync_handshake:
  162. Mar 2 17:17:04 orestes-tb kernel: block drbd0: self 489E63EA01E0F332:0000000000000000:B758FBBC40B2620B:B757FBBC40B2620B bits:13312 flags:0
  163. Mar 2 17:17:04 orestes-tb kernel: block drbd0: peer 489E63EA01E0F332:0000000000000000:B758FBBC40B2620A:B757FBBC40B2620B bits:301056 flags:2
  164. Mar 2 17:17:04 orestes-tb kernel: block drbd0: uuid_compare()=-1 by rule 40
  165. Mar 2 17:17:04 orestes-tb kernel: block drbd0: peer( Unknown -> Secondary ) conn( WFReportParams -> WFBitMapT ) disk( Consistent -> Outdated ) pdsk( DUnknown -> UpToDate )
  166. Mar 2 17:17:04 orestes-tb lrmd: [2998]: info: rsc:Xinetd:0:28: probe
  167. Mar 2 17:17:04 orestes-tb lrmd: [2998]: info: rsc:ExportMail:0:29: probe
  168. Mar 2 17:17:04 orestes-tb logger: exportfs monitor: rc=1 directory=/mail clientspec=franklin.nevis.columbia.edu
  169. Mar 2 17:17:04 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation Xinetd:0_monitor_0 (call=28, rc=7, cib-update=35, confirmed=true) not running
  170. Mar 2 17:17:06 orestes-tb kernel: block drbd0: conn( WFBitMapT -> WFSyncUUID )
  171. Mar 2 17:17:06 orestes-tb kernel: block drbd0: updated sync uuid E65812AAD07BB10A:0000000000000000:B758FBBC40B2620B:B757FBBC40B2620B
  172. Mar 2 17:17:06 orestes-tb kernel: block drbd0: helper command: /sbin/drbdadm before-resync-target minor-0
  173. Mar 2 17:17:06 orestes-tb kernel: block drbd0: helper command: /sbin/drbdadm before-resync-target minor-0 exit code 0 (0x0)
  174. Mar 2 17:17:06 orestes-tb kernel: block drbd0: conn( WFSyncUUID -> SyncTarget ) disk( Outdated -> Inconsistent )
  175. Mar 2 17:17:06 orestes-tb kernel: block drbd0: Began resync as SyncTarget (will sync 1232896 KB [308224 bits set]).
  176. Mar 2 17:17:24 orestes-tb lrmd: [2998]: WARN: ExportMail:0:monitor process (PID 3558) timed out (try 1). Killing with signal SIGTERM (15).
  177. Mar 2 17:17:24 orestes-tb lrmd: [2998]: WARN: operation monitor[29] on ocf::exportfs::ExportMail:0 for client 3001, its parameters: clientspec=[franklin.nevis.columbia.edu] CRM_meta_clone=[0] fsid=[30] CRM_meta_clone_node_max=[1] CRM_meta_clone_max=[2] CRM_meta_notify=[false] crm_feature_set=[3.0.5] CRM_meta_globally_unique=[false] options=[rw,no_root_squash,async] directory=[/mail] CRM_meta_timeout=[20000] : pid [3558] timed out
  178. Mar 2 17:17:24 orestes-tb crmd: [3001]: ERROR: process_lrm_event: LRM operation ExportMail:0_monitor_0 (29) Timed Out (timeout=20000ms)
  179. Mar 2 17:17:24 orestes-tb attrd: [2999]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
  180. Mar 2 17:17:24 orestes-tb attrd: [2999]: notice: attrd_perform_update: Sent update 8: probe_complete=true
  181. Mar 2 17:17:24 orestes-tb attrd: [2999]: notice: attrd_perform_update: Sent update 11: probe_complete=true
  182. Mar 2 17:17:24 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=214:2:0:81f50bd7-02f3-422f-a577-8928f23483a6 op=AdminDrbd:1_notify_0 )
  183. Mar 2 17:17:24 orestes-tb lrmd: [2998]: info: rsc:AdminDrbd:1:32: notify
  184. Mar 2 17:17:24 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=2:2:0:81f50bd7-02f3-422f-a577-8928f23483a6 op=ExportMail:0_stop_0 )
  185. Mar 2 17:17:24 orestes-tb lrmd: [2998]: info: rsc:ExportMail:0:33: stop
  186. Mar 2 17:17:25 orestes-tb logger: exportfs stop: directory=/mail clientspec=franklin.nevis.columbia.edu
  187. Mar 2 17:17:25 orestes-tb logger: exportfs monitor: rc=1 directory=/mail clientspec=franklin.nevis.columbia.edu
  188. Mar 2 17:17:25 orestes-tb crmd: [3001]: info: send_direct_ack: ACK'ing resource op AdminDrbd:1_notify_0 from 214:2:0:81f50bd7-02f3-422f-a577-8928f23483a6: lrm_invoke-lrmd-1330726645-6
  189. Mar 2 17:17:25 orestes-tb crmd: [3001]: info: process_lrm_event: LRM operation AdminDrbd:1_notify_0 (call=32, rc=0, cib-update=0, confirmed=true) ok
  190. Mar 2 17:17:25 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=23:2:0:81f50bd7-02f3-422f-a577-8928f23483a6 op=AdminDrbd:1_promote_0 )
  191. Mar 2 17:17:25 orestes-tb lrmd: [2998]: info: rsc:AdminDrbd:1:34: promote
  192. Mar 2 17:17:25 orestes-tb lrmd: [2998]: info: RA output: (AdminDrbd:1:promote:stdout) allow-two-primaries;
  193. Mar 2 17:17:44 orestes-tb lrmd: [2998]: WARN: ExportMail:0:stop process (PID 3576) timed out (try 1). Killing with signal SIGTERM (15).
  194. Mar 2 17:17:44 orestes-tb lrmd: [2998]: WARN: operation stop[33] on ocf::exportfs::ExportMail:0 for client 3001, its parameters: clientspec=[franklin.nevis.columbia.edu] CRM_meta_clone=[0] fsid=[30] CRM_meta_clone_node_max=[1] CRM_meta_clone_max=[2] CRM_meta_notify=[false] crm_feature_set=[3.0.5] CRM_meta_globally_unique=[false] options=[rw,no_root_squash,async] directory=[/mail] CRM_meta_timeout=[20000] : pid [3576] timed out
  195. Mar 2 17:17:44 orestes-tb crmd: [3001]: ERROR: process_lrm_event: LRM operation ExportMail:0_stop_0 (33) Timed Out (timeout=20000ms)
  196. Mar 2 17:17:45 orestes-tb attrd: [2999]: notice: attrd_ais_dispatch: Update relayed from hypatia-tb.nevis.columbia.edu
  197. Mar 2 17:17:45 orestes-tb attrd: [2999]: notice: attrd_trigger_update: Sending flush op to all hosts for: fail-count-ExportMail:0 (INFINITY)
  198. Mar 2 17:17:45 orestes-tb attrd: [2999]: notice: attrd_perform_update: Sent update 16: fail-count-ExportMail:0=INFINITY
  199. Mar 2 17:17:45 orestes-tb attrd: [2999]: notice: attrd_ais_dispatch: Update relayed from hypatia-tb.nevis.columbia.edu
  200. Mar 2 17:17:45 orestes-tb attrd: [2999]: notice: attrd_trigger_update: Sending flush op to all hosts for: last-failure-ExportMail:0 (1330726664)
  201. Mar 2 17:17:45 orestes-tb attrd: [2999]: notice: attrd_perform_update: Sent update 19: last-failure-ExportMail:0=1330726664
  202. Mar 2 17:17:45 orestes-tb lrmd: [2998]: WARN: AdminDrbd:1:promote process (PID 3607) timed out (try 1). Killing with signal SIGTERM (15).
  203. Mar 2 17:17:45 orestes-tb lrmd: [2998]: WARN: operation promote[34] on ocf::drbd::AdminDrbd:1 for client 3001, its parameters: CRM_meta_clone=[1] CRM_meta_notify_slave_resource=[AdminDrbd:0 AdminDrbd:1 ] CRM_meta_notify_active_resource=[ ] CRM_meta_notify_demote_uname=[ ] drbd_resource=[admin] CRM_meta_notify_start_resource=[ ] CRM_meta_master_node_max=[1] CRM_meta_notify_stop_resource=[ ] CRM_meta_notify_master_resource=[ ] CRM_meta_clone_node_max=[1] CRM_meta_clone_max=[2] CRM_meta_notify=[true] CRM_meta_notify_master_uname=[ ] CRM_meta_master_max=[: pid [3607] timed out
  204. Mar 2 17:17:45 orestes-tb crmd: [3001]: ERROR: process_lrm_event: LRM operation AdminDrbd:1_promote_0 (34) Timed Out (timeout=20000ms)
  205. Mar 2 17:17:45 orestes-tb crmd: [3001]: info: do_lrm_rsc_op: Performing key=215:2:0:81f50bd7-02f3-422f-a577-8928f23483a6 op=AdminDrbd:1_notify_0 )
  206. Mar 2 17:17:45 orestes-tb lrmd: [2998]: info: rsc:AdminDrbd:1:35: notify
  207. Mar 2 17:17:45 orestes-tb attrd: [2999]: notice: attrd_ais_dispatch: Update relayed from hypatia-tb.nevis.columbia.edu
  208. Mar 2 17:17:45 orestes-tb attrd: [2999]: notice: attrd_trigger_update: Sending flush op to all hosts for: fail-count-AdminDrbd:1 (1)
  209. Mar 2 17:17:45 orestes-tb attrd: [2999]: notice: attrd_perform_update: Sent update 22: fail-count-AdminDrbd:1=1
  210. Mar 2 17:17:45 orestes-tb attrd: [2999]: notice: attrd_ais_dispatch: Update relayed from hypatia-tb.nevis.columbia.edu
  211. Mar 2 17:17:45 orestes-tb attrd: [2999]: notice: attrd_trigger_update: Sending flush op to all hosts for: last-failure-AdminDrbd:1 (1330726664)
  212. Mar 2 17:17:45 orestes-tb attrd: [2999]: notice: attrd_perform_update: Sent update 25: last-failure-AdminDrbd:1=1330726664
  213. Mar 2 17:17:58 orestes-tb kernel: block drbd0: role( Secondary -> Primary )
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!