Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- Apache Pig version 0.10.0 (r1328203) compiled Apr 19 2012, 22:54:12
- Logging error messages to: /tmp/hadoop_pig_logs/520767955_log.txt
- Connecting to hadoop file system at: [redacted]
- Connecting to map-reduce job tracker at: [redacted]
- 2012-07-26T13:49:31.724-0700: [GC [PSYoungGen: 49248K->1487K(56192K)] 49248K->1487K(184576K), 0.0043850 secs] [Times: user=0.04 sys=0.01, real=0.00 secs]
- 2012-07-26T13:49:34.627-0700: [GC [PSYoungGen: 49679K->2752K(56192K)] 49679K->2752K(184576K), 0.0037310 secs] [Times: user=0.03 sys=0.00, real=0.00 secs]
- 2012-07-26T13:49:37.942-0700: [GC [PSYoungGen: 50944K->2768K(56192K)] 50944K->2768K(184576K), 0.0031950 secs] [Times: user=0.00 sys=0.02, real=0.01 secs]
- 2012-07-26T13:49:41.367-0700: [GC [PSYoungGen: 50960K->2688K(49792K)] 50960K->2688K(178176K), 0.0035940 secs] [Times: user=0.03 sys=0.01, real=0.00 secs]
- 2012-07-26T13:49:45.064-0700: [GC [PSYoungGen: 49728K->2768K(48704K)] 49728K->2768K(177088K), 0.0031790 secs] [Times: user=0.01 sys=0.02, real=0.00 secs]
- 2012-07-26T13:49:48.226-0700: [GC [PSYoungGen: 48656K->1696K(46528K)] 48656K->2905K(174912K), 0.0042270 secs] [Times: user=0.02 sys=0.00, real=0.00 secs]
- 2012-07-26T13:49:51.608-0700: [GC [PSYoungGen: 46496K->736K(45504K)] 47705K->2910K(173888K), 0.0010140 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
- 2012-07-26T13:49:54.829-0700: [GC [PSYoungGen: 44512K->640K(43392K)] 46686K->3084K(171776K), 0.0011540 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
- [...]
- 2012-07-26T13:58:28.748-0700: [GC [PSYoungGen: 6304K->288K(6528K)] 134082K->128295K(134912K), 0.0006140 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
- 2012-07-26T13:58:29.186-0700: [GC [PSYoungGen: 6048K->288K(6080K)] 134055K->128479K(134464K), 0.0004790 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T13:58:29.186-0700: [Full GC [PSYoungGen: 288K->0K(6080K)] [ParOldGen: 128191K->3216K(49664K)] 128479K->3216K(55744K) [PSPermGen: 17744K->17732K(35776K)], 0.0493240 secs] [Times: user=0.10 sys=0.03, real=0.05 secs]
- 2012-07-26T13:58:29.566-0700: [GC [PSYoungGen: 5760K->608K(6400K)] 8976K->3824K(56064K), 0.0005210 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T13:58:30.056-0700: [GC [PSYoungGen: 6304K->288K(6464K)] 9520K->3672K(56128K), 0.0003630 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- [...]
- 2012-07-26T14:00:02.318-0700: [GC [PSYoungGen: 5344K->352K(5312K)] 54202K->49594K(54976K), 0.0005090 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T14:00:02.691-0700: [GC [PSYoungGen: 5152K->320K(5376K)] 54394K->49730K(55040K), 0.0003860 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T14:00:02.691-0700: [Full GC [PSYoungGen: 320K->0K(5376K)] [ParOldGen: 49410K->3210K(47616K)] 49730K->3210K(52992K) [PSPermGen: 17756K->17750K(39936K)], 0.0354630 secs] [Times: user=0.10 sys=0.00, real=0.04 secs]
- 2012-07-26T14:00:03.048-0700: [GC [PSYoungGen: 4800K->352K(5376K)] 8010K->3562K(52992K), 0.0005060 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T14:00:03.322-0700: [GC [PSYoungGen: 5152K->416K(5376K)] 8362K->3834K(52992K), 0.0004670 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- [...]
- 2012-07-26T14:01:36.612-0700: [GC [PSYoungGen: 6048K->544K(6528K)] 52974K->47638K(54144K), 0.0006460 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
- 2012-07-26T14:01:37.131-0700: [GC [PSYoungGen: 6304K->320K(6528K)] 53398K->47590K(54144K), 0.0003770 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T14:01:37.131-0700: [Full GC [PSYoungGen: 320K->0K(6528K)] [ParOldGen: 47270K->2953K(45440K)] 47590K->2953K(51968K) [PSPermGen: 17776K->17770K(38784K)], 0.0523720 secs] [Times: user=0.21 sys=0.01, real=0.06 secs]
- 2012-07-26T14:01:37.673-0700: [GC [PSYoungGen: 5760K->320K(6080K)] 8713K->3273K(51520K), 0.0003320 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T14:01:38.058-0700: [GC [PSYoungGen: 6080K->512K(6400K)] 9033K->3633K(51840K), 0.0005760 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
- [...]
- 2012-07-26T14:03:00.637-0700: [GC [PSYoungGen: 5248K->448K(5632K)] 49968K->45328K(51072K), 0.0006090 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T14:03:00.997-0700: [GC [PSYoungGen: 5440K->352K(5632K)] 50320K->45536K(51072K), 0.0004940 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T14:03:00.997-0700: [Full GC [PSYoungGen: 352K->0K(5632K)] [ParOldGen: 45184K->2803K(43456K)] 45536K->2803K(49088K) [PSPermGen: 17805K->17799K(37248K)], 0.0321290 secs] [Times: user=0.05 sys=0.03, real=0.03 secs]
- 2012-07-26T14:03:01.448-0700: [GC [PSYoungGen: 4992K->320K(5504K)] 7795K->3123K(48960K), 0.0003480 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
- 2012-07-26T14:03:01.837-0700: [GC [PSYoungGen: 5312K->288K(5568K)] 8115K->3251K(49024K), 0.0021230 secs] [Times: user=0.01 sys=0.01, real=0.00 secs]
- [...]
- 2012-07-26T14:03:57.243-0700: [GC [PSYoungGen: 4352K->288K(4544K)] 38243K->34315K(48000K), 0.0003870 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T14:03:57.521-0700: [GC [PSYoungGen: 4320K->288K(4608K)] 38347K->34451K(48064K), 0.0003970 secs] [Times: user=0.00 sys=0.01, real=0.00 secs]
- Encountered Warning IMPLICIT_CAST_TO_LONG 1 time(s).
- 2012-07-26T14:03:57.682-0700: [GC [PSYoungGen: 4318K->384K(4608K)] 38481K->34691K(48064K), 0.0005720 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T14:03:57.863-0700: [GC [PSYoungGen: 4416K->480K(4608K)] 38723K->34923K(48064K), 0.0004430 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- [...]
- 2012-07-26T14:04:10.542-0700: [GC [PSYoungGen: 4192K->384K(4416K)] 46963K->43291K(47872K), 0.0006810 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T14:04:10.780-0700: [GC [PSYoungGen: 4288K->352K(4352K)] 47195K->43507K(47808K), 0.0004940 secs] [Times: user=0.00 sys=0.01, real=0.00 secs]
- 2012-07-26T14:04:10.780-0700: [Full GC [PSYoungGen: 352K->0K(4352K)] [ParOldGen: 43155K->2808K(41664K)] 43507K->2808K(46016K) [PSPermGen: 17979K->17974K(35712K)], 0.0320490 secs] [Times: user=0.07 sys=0.00, real=0.03 secs]
- 2012-07-26T14:04:11.120-0700: [GC [PSYoungGen: 3904K->256K(4416K)] 6712K->3064K(46080K), 0.0058790 secs] [Times: user=0.04 sys=0.00, real=0.01 secs]
- 2012-07-26T14:04:11.391-0700: [GC [PSYoungGen: 4160K->288K(4416K)] 6968K->3248K(46080K), 0.0003120 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- [...]
- 2012-07-26T14:04:31.823-0700: [GC [PSYoungGen: 4000K->288K(4224K)] 22088K->18496K(45888K), 0.0003440 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
- 2012-07-26T14:04:32.087-0700: [GC [PSYoungGen: 3936K->288K(4224K)] 22144K->18624K(45888K), 0.0003550 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- Pig features used in the script: REPLICATED_JOIN,GROUP_BY,FILTER
- 2012-07-26T14:04:32.326-0700: [GC [PSYoungGen: 3936K->352K(4224K)] 22272K->18808K(45888K), 0.0005750 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T14:04:32.496-0700: [GC [PSYoungGen: 4000K->384K(4224K)] 22456K->19008K(45888K), 0.0003990 secs] [Times: user=0.00 sys=0.01, real=0.00 secs]
- [...]
- 2012-07-26T14:05:01.789-0700: [GC [PSYoungGen: 3584K->352K(3584K)] 44584K->41616K(45248K), 0.0004500 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T14:05:02.034-0700: [GC [PSYoungGen: 3488K->288K(3648K)] 44752K->41728K(45312K), 0.0003950 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
- 2012-07-26T14:05:02.034-0700: [Full GC [PSYoungGen: 288K->0K(3648K)] [ParOldGen: 41440K->2990K(40064K)] 41728K->2990K(43712K) [PSPermGen: 18320K->18320K(34048K)], 0.0327790 secs] [Times: user=0.09 sys=0.00, real=0.03 secs]
- 2012-07-26T14:05:02.297-0700: [GC [PSYoungGen: 3136K->288K(3648K)] 6126K->3278K(43712K), 0.0003130 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T14:05:02.501-0700: [GC [PSYoungGen: 3424K->256K(3648K)] 6414K->3366K(43712K), 0.0003650 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- [...]
- 2012-07-26T14:05:49.470-0700: [GC [PSYoungGen: 2880K->320K(2944K)] 42430K->40086K(43008K), 0.0004520 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T14:05:49.683-0700: [GC [PSYoungGen: 2816K->256K(2944K)] 42582K->40230K(43008K), 0.0003910 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T14:05:49.683-0700: [Full GC [PSYoungGen: 256K->0K(2944K)] [ParOldGen: 39974K->2979K(38336K)] 40230K->2979K(41280K) [PSPermGen: 18329K->18329K(32384K)], 0.0327790 secs] [Times: user=0.10 sys=0.00, real=0.04 secs]
- 2012-07-26T14:05:49.922-0700: [GC [PSYoungGen: 2496K->288K(2944K)] 5475K->3267K(41280K), 0.0002870 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T14:05:50.098-0700: [GC [PSYoungGen: 2784K->288K(2944K)] 5763K->3363K(41280K), 0.0003540 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- [...]
- 2012-07-26T14:06:34.746-0700: [GC [PSYoungGen: 2848K->320K(3008K)] 40787K->38347K(41344K), 0.0007780 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T14:06:34.828-0700: [GC [PSYoungGen: 2880K->352K(3008K)] 40907K->38523K(41344K), 0.0005860 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T14:06:34.828-0700: [Full GC [PSYoungGen: 352K->0K(3008K)] [ParOldGen: 38171K->2927K(36672K)] 38523K->2927K(39680K) [PSPermGen: 18395K->18394K(30784K)], 0.0323760 secs] [Times: user=0.07 sys=0.01, real=0.03 secs]
- 2012-07-26T14:06:35.044-0700: [GC [PSYoungGen: 2560K->288K(3008K)] 5487K->3215K(39680K), 0.0003350 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T14:06:35.234-0700: [GC [PSYoungGen: 2848K->288K(3008K)] 5775K->3311K(39680K), 0.0003460 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- [...]
- 2012-07-26T14:06:57.102-0700: [GC [PSYoungGen: 2528K->320K(2624K)] 22391K->20279K(39296K), 0.0003410 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T14:06:57.179-0700: [GC [PSYoungGen: 2560K->320K(2624K)] 22519K->20367K(39296K), 0.0003430 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
- File concatenation threshold: 100 optimistic? false
- 2012-07-26T14:06:57.212-0700: [GC [PSYoungGen: 2560K->352K(2624K)] 22607K->20535K(39296K), 0.0013220 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
- Total input paths to process : 1
- Total input paths to process : 1
- Total input paths (combined) to process : 1
- number of input files: 1
- MR plan size before optimization: 6
- Merged 0 out of total 3 MR operators.
- MR plan size after optimization: 6
- 2012-07-26T14:06:57.255-0700: [GC [PSYoungGen: 2592K->288K(2624K)] 22775K->20591K(39296K), 0.0003540 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- Reducer is to run in accumulative mode.
- Reducer is to run in accumulative mode.
- Pig script settings are added to the job
- mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
- creating jar file Job6883078213189934987.jar
- 2012-07-26T14:06:57.305-0700: [GC [PSYoungGen: 2528K->352K(3008K)] 22831K->20735K(39680K), 0.0019370 secs] [Times: user=0.01 sys=0.01, real=0.00 secs]
- 2012-07-26T14:06:57.482-0700: [GC [PSYoungGen: 2976K->352K(3072K)] 23359K->20863K(39744K), 0.0013430 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
- [...]
- 2012-07-26T14:06:59.944-0700: [GC [PSYoungGen: 3712K->384K(3776K)] 26829K->23861K(40448K), 0.0005960 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
- 2012-07-26T14:07:00.192-0700: [GC [PSYoungGen: 3584K->416K(3776K)] 27061K->24141K(40448K), 0.0006460 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
- jar file Job6883078213189934987.jar created
- Setting up single store job
- 2012-07-26T14:07:00.621-0700: [GC [PSYoungGen: 3616K->432K(3776K)] 27341K->24406K(40448K), 0.0012990 secs] [Times: user=0.01 sys=0.01, real=0.00 secs]
- 1 map-reduce job(s) waiting for submission.
- 2012-07-26T14:07:00.701-0700: [GC [PSYoungGen: 3619K->575K(3776K)] 27593K->25349K(40448K), 0.0007660 secs] [Times: user=0.00 sys=0.01, real=0.00 secs]
- 2012-07-26T14:07:00.841-0700: [GC [PSYoungGen: 3731K->608K(4160K)] 28505K->25717K(40832K), 0.0004560 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T14:07:00.963-0700: [GC [PSYoungGen: 3800K->576K(3840K)] 28910K->25685K(40512K), 0.0004400 secs] [Times: user=0.00 sys=0.00, real=0.01 secs]
- 2012-07-26T14:07:01.078-0700: [GC [PSYoungGen: 3743K->576K(4096K)] 28853K->25693K(40768K), 0.0003710 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 0% complete
- Total input paths to process : 1
- Total input paths to process : 1
- Total input paths (combined) to process : 1
- 2012-07-26T14:07:01.306-0700: [GC [PSYoungGen: 3776K->288K(4096K)] 28893K->25405K(40768K), 0.0005180 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
- HadoopJobId: job_201207252114_0175
- More information at: [redacted]
- 8% complete
- 16% complete
- Pig script settings are added to the job
- mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
- creating jar file Job1572388749387658611.jar
- 2012-07-26T14:07:28.170-0700: [GC [PSYoungGen: 3488K->352K(4096K)] 28605K->25658K(40768K), 0.0006150 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T14:07:28.304-0700: [GC [PSYoungGen: 3551K->320K(4096K)] 28857K->25778K(40768K), 0.0004440 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
- [...]
- 2012-07-26T14:07:30.594-0700: [GC [PSYoungGen: 3708K->416K(3776K)] 31757K->28840K(40448K), 0.0006270 secs] [Times: user=0.02 sys=0.00, real=0.00 secs]
- 2012-07-26T14:07:30.880-0700: [GC [PSYoungGen: 3616K->416K(3840K)] 32040K->29072K(40512K), 0.0005900 secs] [Times: user=0.00 sys=0.01, real=0.00 secs]
- jar file Job1572388749387658611.jar created
- Setting up single store job
- 2012-07-26T14:07:31.263-0700: [GC [PSYoungGen: 3616K->477K(3840K)] 32272K->29334K(40512K), 0.0015660 secs] [Times: user=0.02 sys=0.00, real=0.01 secs]
- 1 map-reduce job(s) waiting for submission.
- 2012-07-26T14:07:31.627-0700: [GC [PSYoungGen: 3667K->639K(3840K)] 32524K->30211K(40512K), 0.0006380 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
- 2012-07-26T14:07:31.814-0700: [GC [PSYoungGen: 3818K->928K(4032K)] 33390K->31051K(40704K), 0.0005600 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
- [...]
- 2012-07-26T14:07:32.983-0700: [GC [PSYoungGen: 5184K->736K(5376K)] 39896K->35912K(42048K), 0.0007490 secs] [Times: user=0.01 sys=0.00, real=0.01 secs]
- 2012-07-26T14:07:33.050-0700: [GC [PSYoungGen: 5280K->736K(6272K)] 40456K->36526K(42944K), 0.0007880 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
- Total input paths to process : 7174
- Total input paths to process : 7174
- 2012-07-26T14:07:33.226-0700: [GC [PSYoungGen: 6176K->766K(6336K)] 41966K->37148K(43008K), 0.0008810 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
- 2012-07-26T14:07:33.227-0700: [Full GC [PSYoungGen: 766K->0K(6336K)] [ParOldGen: 36382K->9543K(44416K)] 37148K->9543K(50752K) [PSPermGen: 21700K->21692K(33024K)], 0.0601860 secs] [Times: user=0.25 sys=0.00, real=0.06 secs]
- 2012-07-26T14:07:33.918-0700: [GC [PSYoungGen: 5440K->288K(5760K)] 14983K->9831K(50176K), 0.0003760 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- 2012-07-26T14:07:34.431-0700: [GC [PSYoungGen: 5728K->320K(5760K)] 15271K->10063K(50176K), 0.0003940 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
- [...]
- 2012-07-26T14:07:40.136-0700: [GC [PSYoungGen: 5728K->352K(6208K)] 17389K->12213K(50624K), 0.0005090 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
- 2012-07-26T14:07:40.559-0700: [GC [PSYoungGen: 5792K->320K(6208K)] 17653K->12333K(50624K), 0.0004710 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- Total input paths (combined) to process : 6285
- 2012-07-26T14:07:40.819-0700: [GC [PSYoungGen: 5760K->736K(6208K)] 17773K->13109K(50624K), 0.0008760 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
- 2012-07-26T14:07:41.151-0700: [GC [PSYoungGen: 6176K->640K(6336K)] 18549K->13726K(50752K), 0.0007830 secs] [Times: user=0.00 sys=0.01, real=0.00 secs]
- [...]
- 2012-07-26T14:07:43.707-0700: [GC [PSYoungGen: 5984K->640K(6336K)] 19462K->14190K(50752K), 0.0004370 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
- 2012-07-26T14:07:44.031-0700: [GC [PSYoungGen: 6016K->672K(6336K)] 19566K->14294K(50752K), 0.0004320 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
- HadoopJobId: job_201207252114_0176
- More information at: [redacted]
- 2012-07-26T14:10:44.499-0700: [GC [PSYoungGen: 6048K->352K(6336K)] 19670K->14038K(50752K), 0.0007020 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]
- 2012-07-26T14:16:15.226-0700: [GC [PSYoungGen: 5728K->192K(6336K)] 19414K->13942K(50752K), 0.0003540 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
- [...]
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement