Starting the OSD manually

root@glusterfs4:~# /usr/bin/ceph-osd --cluster=ceph -i 0 --osd-data=/data --osd-journal=/data/journal -d --debug_ms 99
2013-11-20 19:02:14.703072 7f1fb473e780  0 ceph version 0.67.4 (ad85b8bfafea6232d64cb7ba76a8b6e8252fa0c7), process ceph-osd, pid 3355
starting osd.0 at :/0 osd_data /data /data/journal
2013-11-20 19:02:14.703411 7f1fb473e780 10 -- :/0 rank.bind :/0
2013-11-20 19:02:14.703423 7f1fb473e780 10 accepter.accepter.bind
2013-11-20 19:02:14.703433 7f1fb473e780 10 accepter.accepter.bind bound on random port 0.0.0.0:6800/0
2013-11-20 19:02:14.703442 7f1fb473e780 10 accepter.accepter.bind bound to 0.0.0.0:6800/0
2013-11-20 19:02:14.703452 7f1fb473e780  1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6800/3355 need_addr=1
2013-11-20 19:02:14.703459 7f1fb473e780 10 -- :/0 rank.bind :/0
2013-11-20 19:02:14.703461 7f1fb473e780 10 accepter.accepter.bind
2013-11-20 19:02:14.703469 7f1fb473e780 10 accepter.accepter.bind bound on random port 0.0.0.0:6801/0
2013-11-20 19:02:14.703472 7f1fb473e780 10 accepter.accepter.bind bound to 0.0.0.0:6801/0
2013-11-20 19:02:14.703477 7f1fb473e780  1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6801/3355 need_addr=1
2013-11-20 19:02:14.703480 7f1fb473e780 10 -- :/0 rank.bind :/0
2013-11-20 19:02:14.703481 7f1fb473e780 10 accepter.accepter.bind
2013-11-20 19:02:14.703485 7f1fb473e780 10 accepter.accepter.bind bound on random port 0.0.0.0:6802/0
2013-11-20 19:02:14.703487 7f1fb473e780 10 accepter.accepter.bind bound to 0.0.0.0:6802/0
2013-11-20 19:02:14.703495 7f1fb473e780  1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6802/3355 need_addr=1
2013-11-20 19:02:14.703497 7f1fb473e780 10 -- :/0 rank.bind :/0
2013-11-20 19:02:14.703499 7f1fb473e780 10 accepter.accepter.bind
2013-11-20 19:02:14.703503 7f1fb473e780 10 accepter.accepter.bind bound on random port 0.0.0.0:6803/0
2013-11-20 19:02:14.703505 7f1fb473e780 10 accepter.accepter.bind bound to 0.0.0.0:6803/0
2013-11-20 19:02:14.703510 7f1fb473e780  1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6803/3355 need_addr=1
2013-11-20 19:02:14.705689 7f1fb473e780  1 filestore(/data) mount detected xfs
2013-11-20 19:02:14.705697 7f1fb473e780  1 filestore(/data)  disabling 'filestore replica fadvise' due to known issues with fadvise(DONTNEED) on xfs
2013-11-20 19:02:14.707582 7f1fb473e780  0 filestore(/data) mount FIEMAP ioctl is supported and appears to work
2013-11-20 19:02:14.707589 7f1fb473e780  0 filestore(/data) mount FIEMAP ioctl is disabled via 'filestore fiemap' config option
2013-11-20 19:02:14.707794 7f1fb473e780  0 filestore(/data) mount did NOT detect btrfs
2013-11-20 19:02:14.708272 7f1fb473e780  0 filestore(/data) mount syncfs(2) syscall fully supported (by glibc and kernel)
2013-11-20 19:02:14.708334 7f1fb473e780  0 filestore(/data) mount found snaps <>
2013-11-20 19:02:14.709365 7f1fb473e780  0 filestore(/data) mount: enabling WRITEAHEAD journal mode: btrfs not detected
2013-11-20 19:02:14.709480 7f1fb473e780 -1 journal FileJournal::_open: disabling aio for non-block journal.  Use journal_force_aio to force use of aio anyway
2013-11-20 19:02:14.709493 7f1fb473e780  1 journal _open /data/journal fd 19: 1073741824 bytes, block size 4096 bytes, directio = 1, aio = 0
2013-11-20 19:02:14.709867 7f1fb473e780  1 journal _open /data/journal fd 19: 1073741824 bytes, block size 4096 bytes, directio = 1, aio = 0
2013-11-20 19:02:14.710420 7f1fb473e780  1 journal close /data/journal

ceph.log

2013-11-20 19:00:54.407271 mon.0 10.2.16.220:6789/0 47 : [INF] pgmap v21: 192 pgs: 192 creating; 0 bytes data, 2114 MB used, 4091 GB / 4094 GB avail
2013-11-20 19:00:57.912244 mon.0 10.2.16.220:6789/0 48 : [INF] pgmap v22: 192 pgs: 192 creating; 0 bytes data, 2114 MB used, 4091 GB / 4094 GB avail
2013-11-20 19:02:03.160155 mon.0 10.2.16.220:6789/0 49 : [INF] osd.0 marked itself down
2013-11-20 19:02:03.214126 mon.0 10.2.16.220:6789/0 50 : [INF] osdmap e11: 2 osds: 1 up, 2 in
2013-11-20 19:02:03.220799 mon.0 10.2.16.220:6789/0 51 : [INF] pgmap v23: 192 pgs: 192 creating; 0 bytes data, 2114 MB used, 4091 GB / 4094 GB avail
2013-11-20 19:02:14.783270 mon.0 10.2.16.220:6789/0 52 : [INF] osd.0 10.2.16.221:6800/3355 boot
2013-11-20 19:02:14.783427 mon.0 10.2.16.220:6789/0 53 : [INF] osdmap e12: 2 osds: 2 up, 2 in
2013-11-20 19:02:14.785661 mon.0 10.2.16.220:6789/0 54 : [INF] pgmap v24: 192 pgs: 192 creating; 0 bytes data, 2114 MB used, 4091 GB / 4094 GB avail
2013-11-20 19:02:20.775893 mon.0 10.2.16.220:6789/0 55 : [INF] pgmap v25: 192 pgs: 192 creating; 0 bytes data, 2114 MB used, 4091 GB / 4094 GB avail


Same thing on the other node