cat work/app-20150713133347-0000/17/hs_err_pid11865.log
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 4294967296 bytes for committing reserved memory.
# Possible reasons:
#   The system is out of physical RAM or swap space
#   In 32 bit mode, the process size limit was hit
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Use 64 bit Java on a 64 bit OS
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (os_linux.cpp:2673), pid=11865, tid=139639313053440
#
# JRE version:  (8.0_45-b13) (build )
# Java VM: OpenJDK 64-Bit Server VM (25.45-b02 mixed mode linux-amd64 compressed oops)
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#

---------------  T H R E A D  ---------------

Current thread (0x00007f0048009000):  JavaThread "Unknown thread" [_thread_in_vm, id=11869, stack(0x00007f004f96b000,0x00007f004fa6c000)]

Stack: [0x00007f004f96b000,0x00007f004fa6c000],  sp=0x00007f004fa6a3b0,  free space=1020k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0xa1d6fe]
V  [libjvm.so+0x4d1239]
V  [libjvm.so+0x891553]
V  [libjvm.so+0x8918f9]
V  [libjvm.so+0x88a83a]
V  [libjvm.so+0x8f683b]
V  [libjvm.so+0x8e6dd9]
V  [libjvm.so+0x2ca37e]
V  [libjvm.so+0x8b0577]
V  [libjvm.so+0x9e8ca1]
V  [libjvm.so+0x9ea1fe]
V  [libjvm.so+0x5fb7b5]
V  [libjvm.so+0x9d0d07]
V  [libjvm.so+0x65f6f1]  JNI_CreateJavaVM+0x71
C  [libjli.so+0x3407]
C  [libpthread.so.0+0x79d1]


---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )

Other Threads:

=>0x00007f0048009000 (exited) JavaThread "Unknown thread" [_thread_in_vm, id=11869, stack(0x00007f004f96b000,0x00007f004fa6c000)]

VM state:not at safepoint (not fully initialized)

VM Mutex/Monitor currently owned by a thread: None

GC Heap History (0 events):
No events

Deoptimization events (0 events):
No events

Internal exceptions (0 events):
No events

Events (0 events):
No events


Dynamic libraries:
00400000-00401000 r-xp 00000000 fd:00 273632                             /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.45-28.b13.el6_6.x86_64/jre/bin/java
00600000-00602000 rw-p 00000000 fd:00 273632                             /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.45-28.b13.el6_6.x86_64/jre/bin/java
0108e000-010af000 rw-p 00000000 00:00 0                                  [heap]
740000000-7c0000000 rw-p 00000000 00:00 0
3a3da00000-3a3da20000 r-xp 00000000 fd:00 66185                          /lib64/ld-2.12.so
3a3dc1f000-3a3dc20000 r--p 0001f000 fd:00 66185                          /lib64/ld-2.12.so
3a3dc20000-3a3dc21000 rw-p 00020000 fd:00 66185                          /lib64/ld-2.12.so
3a3dc21000-3a3dc22000 rw-p 00000000 00:00 0
3a3de00000-3a3de02000 r-xp 00000000 fd:00 66191                          /lib64/libdl-2.12.so
3a3de02000-3a3e002000 ---p 00002000 fd:00 66191                          /lib64/libdl-2.12.so
3a3e002000-3a3e003000 r--p 00002000 fd:00 66191                          /lib64/libdl-2.12.so
3a3e003000-3a3e004000 rw-p 00003000 fd:00 66191                          /lib64/libdl-2.12.so
3a3e200000-3a3e38b000 r-xp 00000000 fd:00 66186                          /lib64/libc-2.12.so
3a3e38b000-3a3e58a000 ---p 0018b000 fd:00 66186                          /lib64/libc-2.12.so
3a3e58a000-3a3e58e000 r--p 0018a000 fd:00 66186                          /lib64/libc-2.12.so
3a3e58e000-3a3e58f000 rw-p 0018e000 fd:00 66186                          /lib64/libc-2.12.so
3a3e58f000-3a3e594000 rw-p 00000000 00:00 0
3a3e600000-3a3e617000 r-xp 00000000 fd:00 66187                          /lib64/libpthread-2.12.so
3a3e617000-3a3e817000 ---p 00017000 fd:00 66187                          /lib64/libpthread-2.12.so
3a3e817000-3a3e818000 r--p 00017000 fd:00 66187                          /lib64/libpthread-2.12.so
3a3e818000-3a3e819000 rw-p 00018000 fd:00 66187                          /lib64/libpthread-2.12.so
3a3e819000-3a3e81d000 rw-p 00000000 00:00 0
3a3ea00000-3a3ea0d000 r-xp 00000000 fd:00 273763                         /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.45-28.b13.el6_6.x86_64/jre/lib/amd64/jli/libjli.so
3a3ea0d000-3a3ec0d000 ---p 0000d000 fd:00 273763                         /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.45-28.b13.el6_6.x86_64/jre/lib/amd64/jli/libjli.so
3a3ec0d000-3a3ec0e000 rw-p 0000d000 fd:00 273763                         /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.45-28.b13.el6_6.x86_64/jre/lib/amd64/jli/libjli.so
3a3ee00000-3a3ee15000 r-xp 00000000 fd:00 66188                          /lib64/libz.so.1.2.3
3a3ee15000-3a3f014000 ---p 00015000 fd:00 66188                          /lib64/libz.so.1.2.3
3a3f014000-3a3f015000 r--p 00014000 fd:00 66188                          /lib64/libz.so.1.2.3
3a3f015000-3a3f016000 rw-p 00015000 fd:00 66188                          /lib64/libz.so.1.2.3
3a3f200000-3a3f283000 r-xp 00000000 fd:00 66193                          /lib64/libm-2.12.so
3a3f283000-3a3f482000 ---p 00083000 fd:00 66193                          /lib64/libm-2.12.so
3a3f482000-3a3f483000 r--p 00082000 fd:00 66193                          /lib64/libm-2.12.so
3a3f483000-3a3f484000 rw-p 00083000 fd:00 66193                          /lib64/libm-2.12.so
3a49200000-3a49216000 r-xp 00000000 fd:00 29359                          /lib64/libgcc_s-4.4.7-20120601.so.1
3a49216000-3a49415000 ---p 00016000 fd:00 29359                          /lib64/libgcc_s-4.4.7-20120601.so.1
3a49415000-3a49416000 rw-p 00015000 fd:00 29359                          /lib64/libgcc_s-4.4.7-20120601.so.1
3a4aa00000-3a4aae8000 r-xp 00000000 fd:00 66267                          /usr/lib64/libstdc++.so.6.0.13
3a4aae8000-3a4ace8000 ---p 000e8000 fd:00 66267                          /usr/lib64/libstdc++.so.6.0.13
3a4ace8000-3a4acef000 r--p 000e8000 fd:00 66267                          /usr/lib64/libstdc++.so.6.0.13
3a4acef000-3a4acf1000 rw-p 000ef000 fd:00 66267                          /usr/lib64/libstdc++.so.6.0.13
3a4acf1000-3a4ad06000 rw-p 00000000 00:00 0
7f0039000000-7f0039270000 rwxp 00000000 00:00 0
7f0039270000-7f0048000000 ---p 00000000 00:00 0
7f0048000000-7f004802f000 rw-p 00000000 00:00 0
7f004802f000-7f004c000000 ---p 00000000 00:00 0
7f004df47000-7f004e747000 ---p 00000000 00:00 0
7f004e747000-7f004eb52000 rw-p 00000000 00:00 0
7f004eb52000-7f004ef08000 ---p 00000000 00:00 0
7f004ef08000-7f004ef10000 r-xp 00000000 fd:00 273665                     /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.45-28.b13.el6_6.x86_64/jre/lib/amd64/libzip.so
7f004ef10000-7f004f10f000 ---p 00008000 fd:00 273665                     /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.45-28.b13.el6_6.x86_64/jre/lib/amd64/libzip.so
7f004f10f000-7f004f110000 rw-p 00007000 fd:00 273665                     /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.45-28.b13.el6_6.x86_64/jre/lib/amd64/libzip.so
7f004f110000-7f004f11c000 r-xp 00000000 fd:00 28741                      /lib64/libnss_files-2.12.so
7f004f11c000-7f004f31c000 ---p 0000c000 fd:00 28741                      /lib64/libnss_files-2.12.so
7f004f31c000-7f004f31d000 r--p 0000c000 fd:00 28741                      /lib64/libnss_files-2.12.so
7f004f31d000-7f004f31e000 rw-p 0000d000 fd:00 28741                      /lib64/libnss_files-2.12.so
7f004f327000-7f004f329000 rw-p 00000000 00:00 0
7f004f329000-7f004f353000 r-xp 00000000 fd:00 273648                     /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.45-28.b13.el6_6.x86_64/jre/lib/amd64/libjava.so
7f004f353000-7f004f552000 ---p 0002a000 fd:00 273648                     /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.45-28.b13.el6_6.x86_64/jre/lib/amd64/libjava.so
7f004f552000-7f004f554000 rw-p 00029000 fd:00 273648                     /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.45-28.b13.el6_6.x86_64/jre/lib/amd64/libjava.so
7f004f554000-7f004f561000 r-xp 00000000 fd:00 273664                     /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.45-28.b13.el6_6.x86_64/jre/lib/amd64/libverify.so
7f004f561000-7f004f761000 ---p 0000d000 fd:00 273664                     /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.45-28.b13.el6_6.x86_64/jre/lib/amd64/libverify.so
7f004f761000-7f004f763000 rw-p 0000d000 fd:00 273664                     /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.45-28.b13.el6_6.x86_64/jre/lib/amd64/libverify.so
7f004f763000-7f004f76a000 r-xp 00000000 fd:00 66201                      /lib64/librt-2.12.so
7f004f76a000-7f004f969000 ---p 00007000 fd:00 66201                      /lib64/librt-2.12.so
7f004f969000-7f004f96a000 r--p 00006000 fd:00 66201                      /lib64/librt-2.12.so
7f004f96a000-7f004f96b000 rw-p 00007000 fd:00 66201                      /lib64/librt-2.12.so
7f004f96b000-7f004f96e000 ---p 00000000 00:00 0
7f004f96e000-7f004fa6c000 rw-p 00000000 00:00 0
7f004fa6c000-7f0050681000 r-xp 00000000 fd:00 273669                     /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.45-28.b13.el6_6.x86_64/jre/lib/amd64/server/libjvm.so
7f0050681000-7f0050880000 ---p 00c15000 fd:00 273669                     /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.45-28.b13.el6_6.x86_64/jre/lib/amd64/server/libjvm.so
7f0050880000-7f005094e000 rw-p 00c14000 fd:00 273669                     /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.45-28.b13.el6_6.x86_64/jre/lib/amd64/server/libjvm.so
7f005094e000-7f0050982000 rw-p 00000000 00:00 0
7f0050982000-7f005098a000 rw-s 00000000 fd:00 398090                     /tmp/hsperfdata_root/11865
7f005098a000-7f005098b000 rw-p 00000000 00:00 0
7f005098b000-7f005098c000 r--p 00000000 00:00 0
7f005098c000-7f005098d000 rw-p 00000000 00:00 0
7fff638c2000-7fff638d7000 rw-p 00000000 00:00 0                          [stack]
7fff63972000-7fff63973000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]

VM Arguments:
jvm_args: -Xms6144M -Xmx6144M -Dspark.driver.port=34732
java_command: org.apache.spark.executor.CoarseGrainedExecutorBackend --driver-url akka.tcp://[email protected]:34732/user/CoarseGrainedScheduler --executor-id 17 --hostname 192.168.122.142 --cores 4 --app-id app-20150713133347-0000 --worker-url akka.tcp://[email protected]:38487/user/Worker
java_class_path (initial): /home/spark/spark-1.4.0-bin-hadoop2.4/sbin/../conf/:/home/spark/spark-1.4.0-bin-hadoop2.4/lib/spark-assembly-1.4.0-hadoop2.4.0.jar:/home/spark/spark-1.4.0-bin-hadoop2.4/lib/datanucleus-api-jdo-3.2.6.jar:/home/spark/spark-1.4.0-bin-hadoop2.4/lib/datanucleus-rdbms-3.2.9.jar:/home/spark/spark-1.4.0-bin-hadoop2.4/lib/datanucleus-core-3.2.10.jar
Launcher Type: SUN_STANDARD

Environment Variables:
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
SHELL=/bin/bash

Signal Handlers:
SIGSEGV: [libjvm.so+0xa1e0b0], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGBUS: [libjvm.so+0xa1e0b0], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGFPE: [libjvm.so+0x88df30], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGPIPE: [libjvm.so+0x88df30], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGXFSZ: [libjvm.so+0x88df30], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGILL: [libjvm.so+0x88df30], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGUSR1: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGUSR2: [libjvm.so+0x88f420], sa_mask[0]=00100000000000000000000000000000, sa_flags=SA_RESTART|SA_SIGINFO
SIGHUP: SIG_IGN, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGINT: SIG_IGN, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGTERM: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGQUIT: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none


---------------  S Y S T E M  ---------------

OS:CentOS release 6.5 (Final)

uname:Linux 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013 x86_64
libc:glibc 2.12 NPTL 2.12
rlimit: STACK 10240k, CORE 0k, NPROC 30489, NOFILE 4096, AS infinity
load average:9.72 6.79 3.57

/proc/meminfo:
MemTotal:        3923392 kB
MemFree:          168688 kB
Buffers:            3908 kB
Cached:           142520 kB
SwapCached:       373920 kB
Active:          2667384 kB
Inactive:         936972 kB
Active(anon):    2651864 kB
Inactive(anon):   806096 kB
Active(file):      15520 kB
Inactive(file):   130876 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:       4128760 kB
SwapFree:        2667288 kB
Dirty:             91036 kB
Writeback:         57268 kB
AnonPages:       3320032 kB
Mapped:            14516 kB
Shmem:                 0 kB
Slab:              80624 kB
SReclaimable:      16096 kB
SUnreclaim:        64528 kB
KernelStack:        3440 kB
PageTables:        19756 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     6090456 kB
Committed_AS:   10090680 kB
VmallocTotal:   34359738367 kB
VmallocUsed:       23740 kB
VmallocChunk:   34359712156 kB
HardwareCorrupted:     0 kB
AnonHugePages:   1814528 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        8180 kB
DirectMap2M:     4186112 kB


CPU:total 5 (1 cores per cpu, 1 threads per core) family 6 model 13 stepping 3, cmov, cx8, fxsr, mmx, sse, sse2, sse3, tsc

/proc/cpuinfo:
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 13
model name      : QEMU Virtual CPU version (cpu64-rhel6)
stepping        : 3
cpu MHz         : 2094.950
cache size      : 4096 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 4
wp              : yes
flags           : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm unfair_spinlock pni cx16 hypervisor lahf_lm
bogomips        : 4189.90
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 13
model name      : QEMU Virtual CPU version (cpu64-rhel6)
stepping        : 3
cpu MHz         : 2094.950
cache size      : 4096 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 4
wp              : yes
flags           : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm unfair_spinlock pni cx16 hypervisor lahf_lm
bogomips        : 4189.90
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor       : 2
vendor_id       : GenuineIntel
cpu family      : 6
model           : 13
model name      : QEMU Virtual CPU version (cpu64-rhel6)
stepping        : 3
cpu MHz         : 2094.950
cache size      : 4096 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 4
wp              : yes
flags           : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm unfair_spinlock pni cx16 hypervisor lahf_lm
bogomips        : 4189.90
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor       : 3
vendor_id       : GenuineIntel
cpu family      : 6
model           : 13
model name      : QEMU Virtual CPU version (cpu64-rhel6)
stepping        : 3
cpu MHz         : 2094.950
cache size      : 4096 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 4
wp              : yes
flags           : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm unfair_spinlock pni cx16 hypervisor lahf_lm
bogomips        : 4189.90
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor       : 4
vendor_id       : GenuineIntel
cpu family      : 6
model           : 13
model name      : QEMU Virtual CPU version (cpu64-rhel6)
stepping        : 3
cpu MHz         : 2094.950
cache size      : 4096 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 4
wp              : yes
flags           : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm unfair_spinlock pni cx16 hypervisor lahf_lm
bogomips        : 4189.90
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:



Memory: 4k page, physical 3923392k(168688k free), swap 4128760k(2667288k free)

vm_info: OpenJDK 64-Bit Server VM (25.45-b02) for linux-amd64 JRE (1.8.0_45-b13), built on Apr 15 2015 01:11:29 by "mockbuild" with gcc 4.4.7 20120313 (Red Hat 4.4.7-11)

time: Mon Jul 13 15:48:55 2015