Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- grunt> ec = LOAD 'cassandra://abc/ec' USING CassandraStorage();
- grunt> DESCRIBE ec;
- ec: {key: chararray,columns: {(name: (),value: long)}}
- grunt> DUMP ec;
- 2013-07-19 12:22:25,951 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
- 2013-07-19 12:22:25,954 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
- 2013-07-19 12:22:25,956 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
- 2013-07-19 12:22:25,956 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
- 2013-07-19 12:22:25,959 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
- 2013-07-19 12:22:25,959 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
- 2013-07-19 12:22:25,970 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job
- 2013-07-19 12:22:25,976 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission.
- 2013-07-19 12:22:25,982 [Thread-15] WARN org.apache.hadoop.mapred.JobClient - No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
- 2013-07-19 12:22:26,001 [Thread-15] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1
- 2013-07-19 12:22:26,057 [Thread-15] WARN org.apache.hadoop.conf.Configuration - file:/tmp/hadoop-konrad/mapred/local/localRunner/job_local823738536_0003.xml:a attempt to override final parameter: mapred.system.dir; Ignoring.
- 2013-07-19 12:22:26,057 [Thread-15] WARN org.apache.hadoop.conf.Configuration - file:/tmp/hadoop-konrad/mapred/local/localRunner/job_local823738536_0003.xml:a attempt to override final parameter: mapred.local.dir; Ignoring.
- 2013-07-19 12:22:26,057 [Thread-15] WARN org.apache.hadoop.conf.Configuration - file:/tmp/hadoop-konrad/mapred/local/localRunner/job_local823738536_0003.xml:a attempt to override final parameter: mapred.job.tracker.http.address; Ignoring.
- 2013-07-19 12:22:26,059 [Thread-15] WARN org.apache.hadoop.conf.Configuration - file:/tmp/hadoop-konrad/mapred/local/localRunner/job_local823738536_0003.xml:a attempt to override final parameter: fs.default.name; Ignoring.
- 2013-07-19 12:22:26,059 [Thread-15] WARN org.apache.hadoop.conf.Configuration - file:/tmp/hadoop-konrad/mapred/local/localRunner/job_local823738536_0003.xml:a attempt to override final parameter: mapred.jobtracker.maxtasks.per.job; Ignoring.
- 2013-07-19 12:22:26,059 [Thread-15] WARN org.apache.hadoop.conf.Configuration - file:/tmp/hadoop-konrad/mapred/local/localRunner/job_local823738536_0003.xml:a attempt to override final parameter: mapred.job.tracker; Ignoring.
- 2013-07-19 12:22:26,061 [Thread-16] INFO org.apache.hadoop.mapred.LocalJobRunner - Waiting for map tasks
- 2013-07-19 12:22:26,061 [pool-6-thread-1] INFO org.apache.hadoop.mapred.LocalJobRunner - Starting task: attempt_local823738536_0003_m_000000_0
- 2013-07-19 12:22:26,078 [pool-6-thread-1] INFO org.apache.hadoop.mapred.Task - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@2b52b6f5
- 2013-07-19 12:22:26,079 [pool-6-thread-1] INFO org.apache.hadoop.mapred.MapTask - Processing split: Number of splits :2
- Total Length = 0
- Input split[0]:
- Length = 0
- Locations:
- localhost
- -----------------------
- Input split[1]:
- Length = 0
- Locations:
- localhost
- -----------------------
- 2013-07-19 12:22:26,081 [pool-6-thread-1] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader - Current split being processed ColumnFamilySplit((-2169904116074813774, '-1] @[localhost])
- 2013-07-19 12:22:26,105 [Thread-16] INFO org.apache.hadoop.mapred.LocalJobRunner - Map task executor complete.
- 2013-07-19 12:22:26,106 [Thread-16] WARN org.apache.hadoop.mapred.FileOutputCommitter - Output path is null in cleanup
- 2013-07-19 12:22:26,108 [Thread-16] WARN org.apache.hadoop.mapred.LocalJobRunner - job_local823738536_0003
- java.lang.Exception: java.lang.IndexOutOfBoundsException
- at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:354)
- Caused by: java.lang.IndexOutOfBoundsException
- at java.nio.Buffer.checkIndex(Buffer.java:518)
- at java.nio.HeapByteBuffer.getLong(HeapByteBuffer.java:391)
- at org.apache.cassandra.utils.ByteBufferUtil.toLong(ByteBufferUtil.java:421)
- at org.apache.cassandra.cql.jdbc.JdbcLong.compose(JdbcLong.java:94)
- at org.apache.cassandra.db.marshal.LongType.compose(LongType.java:34)
- at org.apache.cassandra.db.marshal.LongType.compose(LongType.java:26)
- at org.apache.cassandra.hadoop.pig.AbstractCassandraStorage.columnToTuple(AbstractCassandraStorage.java:129)
- at org.apache.cassandra.hadoop.pig.CassandraStorage.getNext(CassandraStorage.java:243)
- at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:194)
- at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:531)
- at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
- at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
- at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
- at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
- at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:223)
- at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
- at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
- at java.util.concurrent.FutureTask.run(FutureTask.java:138)
- at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
- at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
- at java.lang.Thread.run(Thread.java:662)
- 2013-07-19 12:22:26,477 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_local823738536_0003
- 2013-07-19 12:22:26,477 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete
- 2013-07-19 12:22:31,485 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - job job_local823738536_0003 has failed! Stop running all dependent jobs
- 2013-07-19 12:22:31,485 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
- 2013-07-19 12:22:31,486 [main] ERROR org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
- 2013-07-19 12:22:31,486 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Detected Local mode. Stats reported below may be incomplete
- 2013-07-19 12:22:31,486 [main] INFO org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
- HadoopVersion PigVersion UserId StartedAt FinishedAt Features
- 1.2.0 0.10.0 konrad 2013-07-19 12:22:25 2013-07-19 12:22:31 UNKNOWN
- Failed!
- Failed Jobs:
- JobId Alias Feature Message Outputs
- job_local823738536_0003 ec MAP_ONLY Message: Job failed! Error - NA file:/tmp/temp1312341768/tmp-1393597214,
- Input(s):
- Failed to read data from "cassandra://abc/ec"
- Output(s):
- Failed to produce result in "file:/tmp/temp1312341768/tmp-1393597214"
- Job DAG:
- job_local823738536_0003
- 2013-07-19 12:22:31,487 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!
- 2013-07-19 12:22:31,489 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias ec
- Details at logfile: /home/konrad/pig_1374228132069.log
- grunt> 2013-07-19 12:22:32,066 [communication thread] INFO org.apache.hadoop.mapred.LocalJobRunner -
- from cqlsh:
- CREATE TABLE ec (
- item_id text PRIMARY KEY,
- gr_1 counter,
- gr_10 counter,
- gr_11 counter,
- gr_2 counter,
- gr_3 counter,
- gr_4 counter,
- gr_5 counter,
- gr_6 counter,
- gr_7 counter,
- gr_8 counter,
- gr_9 counter
- ) WITH
- bloom_filter_fp_chance=0.010000 AND
- caching='KEYS_ONLY' AND
- comment='' AND
- dclocal_read_repair_chance=0.000000 AND
- gc_grace_seconds=864000 AND
- read_repair_chance=0.100000 AND
- replicate_on_write='true' AND
- populate_io_cache_on_flush='false' AND
- compaction={'class': 'SizeTieredCompactionStrategy'} AND
- compression={'sstable_compression': 'SnappyCompressor'};
- CREATE INDEX ec_gr_10_idx ON ec (gr_10);
- CREATE INDEX ec_gr_11_idx ON ec (gr_11);
- CREATE INDEX ec_gr_2_idx ON ec (gr_2);
- CREATE INDEX ec_gr_3_idx ON ec (gr_3);
- CREATE INDEX ec_gr_4_idx ON ec (gr_4);
- CREATE INDEX ec_gr_5_idx ON ec (gr_5);
- CREATE INDEX ec_gr_6_idx ON ec (gr_6);
- CREATE INDEX ec_gr_7_idx ON ec (gr_7);
- CREATE INDEX ec_gr_8_idx ON ec (gr_8);
- CREATE INDEX ec_gr_9_idx ON ec (gr_9);
- item_id | gr_1 | gr_10 | gr_11 | gr_2 | gr_3 | gr_4 | gr_5 | gr_6 | gr_7 | gr_8 | gr_9
- ---------+------+-------+-------+------+------+------+------+------+------+------+------
- 3 | 2 | null | null | 1 | 1 | 1 | 2 | 1 | 1 | 1 | null
- 2 | 3 | 2 | null | 1 | 3 | 1 | 1 | 2 | 2 | 3 | 3
- 1 | 1 | 2 | null | 2 | null | 2 | 1 | 1 | 1 | null | 1
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement