daily pastebin goal
80%
SHARE
TWEET

Untitled

a guest Nov 14th, 2018 98 Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. I found stack trace for the segfault:
  2. ```
  3. #0  0x00002aaaab037706 in __memcpy_ssse3_back () from /usr/lib64/libc.so.6                                                                                         [298/337]
  4. #1  0x00002aab2c6b2f2c in MPIDI_CH3U_Buffer_copy () from /opt/mvapich2/gnu/6.3/2.2/lib/libmpi.so.12
  5. #2  0x00002aab2c6ded44 in MPIDI_Isend_self () from /opt/mvapich2/gnu/6.3/2.2/lib/libmpi.so.12
  6. #3  0x00002aab2c6d8b18 in MPID_Isend () from /opt/mvapich2/gnu/6.3/2.2/lib/libmpi.so.12
  7. #4  0x00002aab2c692d0b in MPIC_Isend () from /opt/mvapich2/gnu/6.3/2.2/lib/libmpi.so.12
  8. #5  0x00002aab2c494fca in MPIR_Alltoall_Scatter_dest_MV2 () from /opt/mvapich2/gnu/6.3/2.2/lib/libmpi.so.12
  9. #6  0x00002aab2c496f4e in MPIR_Alltoall_MV2 () from /opt/mvapich2/gnu/6.3/2.2/lib/libmpi.so.12
  10. #7  0x00002aab2c447ca9 in MPIR_Alltoall_impl () from /opt/mvapich2/gnu/6.3/2.2/lib/libmpi.so.12
  11. #8  0x00002aab2c448373 in PMPI_Alltoall () from /opt/mvapich2/gnu/6.3/2.2/lib/libmpi.so.12
  12. #9  0x00002aaab553742f in __pyx_pf_6mpi4py_3MPI_4Comm_96Alltoall (__pyx_v_self=0x2aaab564a8a0, __pyx_v_recvbuf=<optimized out>, __pyx_v_sendbuf=<optimized out>) at src/mpi4
  13. py.MPI.c:107259
  14. #10 __pyx_pw_6mpi4py_3MPI_4Comm_97Alltoall (__pyx_v_self=0x2aaab564a8a0, __pyx_args=<optimized out>, __pyx_kwds=<optimized out>) at src/mpi4py.MPI.c:41665
  15. #11 0x00005555556673d4 in _PyCFunction_FastCallDict () at /tmp/build/80754af9/python_1540319457073/work/Objects/methodobject.c:231
  16. #12 0x00005555556f0c1c in call_function () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:4837
  17. #13 0x000055555571371a in _PyEval_EvalFrameDefault () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:3335
  18. #14 0x00005555556ead8b in _PyFunction_FastCall (globals=<optimized out>, nargs=8, args=<optimized out>, co=<optimized out>) at /tmp/build/80754af9/python_1540319457073/work
  19. /Python/ceval.c:4919
  20. #15 fast_function () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:4954
  21. #16 0x00005555556f0cf5 in call_function () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:4858
  22. #17 0x000055555571371a in _PyEval_EvalFrameDefault () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:3335
  23. #18 0x00005555556ead8b in _PyFunction_FastCall (globals=<optimized out>, nargs=2, args=<optimized out>, co=<optimized out>) at /tmp/build/80754af9/python_154031945[316/337]
  24. /Python/ceval.c:4919
  25. #19 fast_function () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:4954
  26. #20 0x00005555556f0cf5 in call_function () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:4858                                                    [319/337]
  27. #21 0x000055555571371a in _PyEval_EvalFrameDefault () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:3335
  28. #22 0x00005555556ea124 in _PyEval_EvalCodeWithName () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:4166
  29. #23 0x00005555556eb49c in _PyFunction_FastCallDict () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:5070
  30. #24 0x000055555566779f in _PyObject_FastCallDict () at /tmp/build/80754af9/python_1540319457073/work/Objects/abstract.c:2310
  31. #25 0x000055555566c303 in _PyObject_Call_Prepend () at /tmp/build/80754af9/python_1540319457073/work/Objects/abstract.c:2373
  32. #26 0x00005555556671de in PyObject_Call () at /tmp/build/80754af9/python_1540319457073/work/Objects/abstract.c:2261
  33. #27 0x0000555555714d9a in do_call_core (kwdict=0x0, callargs=0x2aaab79ab318, func=0x2aaab56e2cc8) at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:5106
  34. #28 _PyEval_EvalFrameDefault () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:3404
  35. #29 0x00005555556ead8b in _PyFunction_FastCall (globals=<optimized out>, nargs=1, args=<optimized out>, co=<optimized out>) at /tmp/build/80754af9/python_1540319457073/work
  36. /Python/ceval.c:4919
  37. #30 fast_function () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:4954
  38. #31 0x00005555556f0cf5 in call_function () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:4858
  39. #32 0x000055555571371a in _PyEval_EvalFrameDefault () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:3335
  40. #33 0x00005555556ead8b in _PyFunction_FastCall (globals=<optimized out>, nargs=1, args=<optimized out>, co=<optimized out>) at /tmp/build/80754af9/python_1540319457073/work
  41. /Python/ceval.c:4919
  42. #34 fast_function () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:4954
  43. #35 0x00005555556f0cf5 in call_function () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:4858
  44. #36 0x000055555571371a in _PyEval_EvalFrameDefault () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:3335
  45. #37 0x00005555556ea42e in _PyEval_EvalCodeWithName () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:4166
  46. #38 0x00005555556eafc1 in fast_function () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:4978
  47. #39 0x00005555556f0cf5 in call_function () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:4858
  48. #40 0x000055555571371a in _PyEval_EvalFrameDefault () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:3335                                         [336/337]
  49. #41 0x00005555556ead8b in _PyFunction_FastCall (globals=<optimized out>, nargs=0, args=<optimized out>, co=<optimized out>) at /tmp/build/80754af9/python_1540319457073/work
  50. /Python/ceval.c:4919
  51. #42 fast_function () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:4954
  52. #43 0x00005555556f0cf5 in call_function () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:4858
  53. #44 0x000055555571371a in _PyEval_EvalFrameDefault () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:3335
  54. #45 0x00005555556ebad9 in _PyEval_EvalCodeWithName (qualname=0x0, name=<optimized out>, closure=0x0, kwdefs=0x0, defcount=0, defs=0x0, kwstep=2, kwcount=<optimized out>, kw
  55. args=0x0, kwnames=0x0, argcount=0, args=0x0, locals=0x2aaaaab963f0, globals=0x2aaaaab963f0, _co=0x2aaab211ced0) at /tmp/build/80754af9/python_1540319457073/work/Python/ceva
  56. l.c:4166
  57. #46 PyEval_EvalCodeEx () at /tmp/build/80754af9/python_1540319457073/work/Python/ceval.c:4187
  58. #47 0x00005555556ec87c in PyEval_EvalCode (co=co@entry=0x2aaab211ced0, globals=globals@entry=0x2aaaaab963f0, locals=locals@entry=0x2aaaaab963f0) at /tmp/build/80754af9/pyth
  59. on_1540319457073/work/Python/ceval.c:731
  60. #48 0x000055555576d074 in run_mod () at /tmp/build/80754af9/python_1540319457073/work/Python/pythonrun.c:1025
  61. #49 0x000055555576d471 in PyRun_FileExFlags () at /tmp/build/80754af9/python_1540319457073/work/Python/pythonrun.c:978
  62. #50 0x000055555576d673 in PyRun_SimpleFileExFlags () at /tmp/build/80754af9/python_1540319457073/work/Python/pythonrun.c:419
  63. #51 0x000055555576d77d in PyRun_AnyFileExFlags () at /tmp/build/80754af9/python_1540319457073/work/Python/pythonrun.c:81
  64. #52 0x00005555557711b0 in run_file (p_cf=0x7fffffffb80c, filename=0x5555558ac3e0 L"train_imagenet.py", fp=0x555555962930) at /tmp/build/80754af9/python_1540319457073/work/M
  65. odules/main.c:340
  66. #53 Py_Main () at /tmp/build/80754af9/python_1540319457073/work/Modules/main.c:811
  67. #54 0x0000555555638b4e in main () at /tmp/build/80754af9/python_1540319457073/work/Programs/python.c:69
  68. #55 0x00002aaaaaf0cc05 in __libc_start_main () from /usr/lib64/libc.so.6
  69. #56 0x000055555571a1a8 in _start () at ../sysdeps/x86_64/elf/start.S:103
RAW Paste Data
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy. OK, I Understand
 
Top