daily pastebin goal
75%
SHARE
TWEET

Untitled

a guest Jan 28th, 2016 96 Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. SolrIndexWriter
  2.     solr.server.type : Type of SolrServer to communicate with (default 'http' however options include 'cloud', 'lb' and 'concurrent')
  3.     solr.server.url : URL of the Solr instance (mandatory)
  4.     solr.zookeeper.url : URL of the Zookeeper URL (mandatory if 'cloud' value for solr.server.type)
  5.     solr.loadbalance.urls : Comma-separated string of Solr server strings to be used (madatory if 'lb' value for solr.server.type)
  6.     solr.mapping.file : name of the mapping file for fields (default solrindex-mapping.xml)
  7.     solr.commit.size : buffer size when sending to Solr (default 1000)
  8.     solr.auth : use authentication (default false)
  9.     solr.auth.username : username for authentication
  10.     solr.auth.password : password for authentication
  11.  
  12.  
  13. 2016-01-28 02:49:41,422 INFO  indexer.IndexerMapReduce - IndexerMapReduce: crawldb: nutchweb/crawldb
  14. 2016-01-28 02:49:41,425 INFO  indexer.IndexerMapReduce - IndexerMapReduce: linkdb: nutchweb/linkdb
  15. 2016-01-28 02:49:41,425 INFO  indexer.IndexerMapReduce - IndexerMapReduces: adding segment: nutchweb/segments/20160127234706
  16. 2016-01-28 02:49:41,652 WARN  util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  17. 2016-01-28 02:49:42,586 WARN  conf.Configuration - file:/tmp/hadoop-micky/mapred/staging/micky810285982/.staging/job_local810285982_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
  18. 2016-01-28 02:49:42,587 WARN  conf.Configuration - file:/tmp/hadoop-micky/mapred/staging/micky810285982/.staging/job_local810285982_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
  19. 2016-01-28 02:49:42,751 WARN  conf.Configuration - file:/tmp/hadoop-micky/mapred/local/localRunner/micky/job_local810285982_0001/job_local810285982_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
  20. 2016-01-28 02:49:42,752 WARN  conf.Configuration - file:/tmp/hadoop-micky/mapred/local/localRunner/micky/job_local810285982_0001/job_local810285982_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
  21. 2016-01-28 02:49:43,342 INFO  anchor.AnchorIndexingFilter - Anchor deduplication is: off
  22. 2016-01-28 02:49:49,230 INFO  indexer.IndexWriters - Adding org.apache.nutch.indexwriter.solr.SolrIndexWriter
  23. 2016-01-28 02:49:50,627 INFO  solr.SolrMappingReader - source: content dest: content
  24. 2016-01-28 02:49:50,627 INFO  solr.SolrMappingReader - source: title dest: title
  25. 2016-01-28 02:49:50,627 INFO  solr.SolrMappingReader - source: host dest: host
  26. 2016-01-28 02:49:50,627 INFO  solr.SolrMappingReader - source: segment dest: segment
  27. 2016-01-28 02:49:50,627 INFO  solr.SolrMappingReader - source: boost dest: boost
  28. 2016-01-28 02:49:50,627 INFO  solr.SolrMappingReader - source: digest dest: digest
  29. 2016-01-28 02:49:50,627 INFO  solr.SolrMappingReader - source: tstamp dest: tstamp
  30. 2016-01-28 02:49:50,959 INFO  solr.SolrIndexWriter - Indexing 250 documents
  31. 2016-01-28 02:49:50,960 INFO  solr.SolrIndexWriter - Deleting 0 documents
  32. 2016-01-28 02:49:54,346 INFO  solr.SolrIndexWriter - Indexing 250 documents
  33. 2016-01-28 02:50:06,471 WARN  mapred.LocalJobRunner - job_local810285982_0001
  34. java.lang.Exception: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Exception writing document id http://nutch.apache.org/apidocs/apidocs-1.1/overview-tree.html to the index; possible analysis error.
  35.     at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
  36.     at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:529)
  37. Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Exception writing document id http://nutch.apache.org/apidocs/apidocs-1.1/overview-tree.html to the index; possible analysis error.
  38.     at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:552)
  39.     at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
  40.     at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
  41.     at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:124)
  42.     at org.apache.nutch.indexwriter.solr.SolrIndexWriter.write(SolrIndexWriter.java:134)
  43.     at org.apache.nutch.indexer.IndexWriters.write(IndexWriters.java:85)
  44.     at org.apache.nutch.indexer.IndexerOutputFormat$1.write(IndexerOutputFormat.java:50)
  45.     at org.apache.nutch.indexer.IndexerOutputFormat$1.write(IndexerOutputFormat.java:41)
  46.     at org.apache.hadoop.mapred.ReduceTask$OldTrackingRecordWriter.write(ReduceTask.java:493)
  47.     at org.apache.hadoop.mapred.ReduceTask$3.collect(ReduceTask.java:422)
  48.     at org.apache.nutch.indexer.IndexerMapReduce.reduce(IndexerMapReduce.java:356)
  49.     at org.apache.nutch.indexer.IndexerMapReduce.reduce(IndexerMapReduce.java:56)
  50.     at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:444)
  51.     at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
  52.     at org.apache.hadoop.mapred.LocalJobRunner$Job$ReduceTaskRunnable.run(LocalJobRunner.java:319)
  53.     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  54.     at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  55.     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  56.     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  57.     at java.lang.Thread.run(Thread.java:745)
  58. 2016-01-28 02:50:07,330 ERROR indexer.IndexingJob - Indexer: java.io.IOException: Job failed!
  59.     at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:836)
  60.     at org.apache.nutch.indexer.IndexingJob.index(IndexingJob.java:145)
  61.     at org.apache.nutch.indexer.IndexingJob.run(IndexingJob.java:222)
  62.     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
  63.     at org.apache.nutch.indexer.IndexingJob.main(IndexingJob.java:231)
RAW Paste Data
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy. OK, I Understand
 
Top