API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Apr 10th, 2017
107
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 9.06 KB | None | 0 0
raw download clone embed print report
  1. $ sqoop export -fs local -jt local -D 'mapreduce.application.framework.path=/usr/hdp/2.5.0.0-1245/hadoop/mapreduce.tar.gz' --connect jdbc:mysql://172.52.21.64:3306/cf_ae07c762_41a9_4b46_af6c_a29ecb050204 --username username --password password --table test3 --export-dir file:///home/username/folder/test3.csv
  2.  
  3. 17/04/10 09:36:28 INFO mapreduce.Job: The url to track the job: http://localhost:8080/
  4. 17/04/10 09:36:28 INFO mapreduce.Job: Running job: job_local2136897360_0001
  5. 17/04/10 09:36:28 INFO mapred.LocalJobRunner: OutputCommitter set in config null
  6. 17/04/10 09:36:28 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.sqoop.mapreduce.NullOutputCommitter
  7. 17/04/10 09:36:28 INFO mapred.LocalJobRunner: Waiting for map tasks
  8. 17/04/10 09:36:28 INFO mapred.LocalJobRunner: Starting task: attempt_local2136897360_0001_m_000000_0
  9. 17/04/10 09:36:28 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]
  10. 17/04/10 09:36:28 INFO mapred.MapTask: Processing split: Paths:/home/username/folder/test3.csv:36+7,/home/username/folder/test3.csv:43+8
  11. 17/04/10 09:36:28 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false
  12. 17/04/10 09:36:28 INFO mapred.LocalJobRunner:
  13. 17/04/10 09:36:28 INFO mapred.Task: Task:attempt_local2136897360_0001_m_000000_0 is done. And is in the process of committing
  14. 17/04/10 09:36:28 INFO mapred.LocalJobRunner: map
  15. 17/04/10 09:36:28 INFO mapred.Task: Task 'attempt_local2136897360_0001_m_000000_0' done.
  16. 17/04/10 09:36:28 INFO mapred.LocalJobRunner: Finishing task: attempt_local2136897360_0001_m_000000_0
  17. 17/04/10 09:36:28 INFO mapred.LocalJobRunner: Starting task: attempt_local2136897360_0001_m_000001_0
  18. 17/04/10 09:36:28 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]
  19. 17/04/10 09:36:28 INFO mapred.MapTask: Processing split: Paths:/home/username/folder/test3.csv:0+12
  20. 17/04/10 09:36:28 ERROR mapreduce.TextExportMapper:
  21. 17/04/10 09:36:28 ERROR mapreduce.TextExportMapper: Exception raised during data export
  22. 17/04/10 09:36:28 ERROR mapreduce.TextExportMapper:
  23. 17/04/10 09:36:28 ERROR mapreduce.TextExportMapper: Exception:
  24. java.lang.RuntimeException: Can't parse input data: 'id'
  25. at test3.__loadFromFields(test3.java:316)
  26. at test3.parse(test3.java:254)
  27. at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:89)
  28. at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
  29. at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
  30. at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
  31. at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
  32. at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
  33. at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
  34. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  35. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  36. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  37. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  38. at java.lang.Thread.run(Thread.java:745)
  39. Caused by: java.lang.NumberFormatException: For input string: "id"
  40. at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
  41. at java.lang.Integer.parseInt(Integer.java:492)
  42. at java.lang.Integer.valueOf(Integer.java:582)
  43. at test3.__loadFromFields(test3.java:303)
  44. ... 13 more
  45. 17/04/10 09:36:28 ERROR mapreduce.TextExportMapper: Dumping data is not allowed by default, please run the job with -Dorg.apache.sqoop.export.text.dump_data_on_error=true to get corrupted line.
  46. 17/04/10 09:36:28 ERROR mapreduce.TextExportMapper: On input file: file:/home/username/folder/test3.csv
  47. 17/04/10 09:36:28 ERROR mapreduce.TextExportMapper: At position 0
  48. 17/04/10 09:36:28 ERROR mapreduce.TextExportMapper:
  49. 17/04/10 09:36:28 ERROR mapreduce.TextExportMapper: Currently processing split:
  50. 17/04/10 09:36:28 ERROR mapreduce.TextExportMapper: Paths:/home/username/folder/test3.csv:0+12
  51. 17/04/10 09:36:28 ERROR mapreduce.TextExportMapper:
  52. 17/04/10 09:36:28 ERROR mapreduce.TextExportMapper: This issue might not necessarily be caused by current input
  53. 17/04/10 09:36:28 ERROR mapreduce.TextExportMapper: due to the batching nature of export.
  54. 17/04/10 09:36:28 ERROR mapreduce.TextExportMapper:
  55. 17/04/10 09:36:28 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false
  56. 17/04/10 09:36:28 INFO mapred.LocalJobRunner: Starting task: attempt_local2136897360_0001_m_000002_0
  57. 17/04/10 09:36:28 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]
  58. 17/04/10 09:36:28 INFO mapred.MapTask: Processing split: Paths:/home/username/folder/test3.csv:12+12
  59. 17/04/10 09:36:28 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false
  60. 17/04/10 09:36:28 INFO mapred.LocalJobRunner:
  61. 17/04/10 09:36:28 INFO mapred.Task: Task:attempt_local2136897360_0001_m_000002_0 is done. And is in the process of committing
  62. 17/04/10 09:36:28 INFO mapred.LocalJobRunner: map
  63. 17/04/10 09:36:28 INFO mapred.Task: Task 'attempt_local2136897360_0001_m_000002_0' done.
  64. 17/04/10 09:36:28 INFO mapred.LocalJobRunner: Finishing task: attempt_local2136897360_0001_m_000002_0
  65. 17/04/10 09:36:28 INFO mapred.LocalJobRunner: Starting task: attempt_local2136897360_0001_m_000003_0
  66. 17/04/10 09:36:28 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]
  67. 17/04/10 09:36:28 INFO mapred.MapTask: Processing split: Paths:/home/username/folder/test3.csv:24+12
  68. 17/04/10 09:36:28 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false
  69. 17/04/10 09:36:28 INFO mapred.LocalJobRunner:
  70. 17/04/10 09:36:28 INFO mapred.Task: Task:attempt_local2136897360_0001_m_000003_0 is done. And is in the process of committing
  71. 17/04/10 09:36:28 INFO mapred.LocalJobRunner: map
  72. 17/04/10 09:36:28 INFO mapred.Task: Task 'attempt_local2136897360_0001_m_000003_0' done.
  73. 17/04/10 09:36:28 INFO mapred.LocalJobRunner: Finishing task: attempt_local2136897360_0001_m_000003_0
  74. 17/04/10 09:36:28 INFO mapred.LocalJobRunner: map task executor complete.
  75. 17/04/10 09:36:28 WARN mapred.LocalJobRunner: job_local2136897360_0001
  76. java.lang.Exception: java.io.IOException: Can't export data, please check failed map task logs
  77. at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
  78. at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
  79. Caused by: java.io.IOException: Can't export data, please check failed map task logs
  80. at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:122)
  81. at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
  82. at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
  83. at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
  84. at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
  85. at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
  86. at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
  87. at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  88. at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  89. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  90. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  91. at java.lang.Thread.run(Thread.java:745)
  92. Caused by: java.lang.RuntimeException: Can't parse input data: 'id'
  93. at test3.__loadFromFields(test3.java:316)
  94. at test3.parse(test3.java:254)
  95. at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:89)
  96. ... 11 more
  97. Caused by: java.lang.NumberFormatException: For input string: "id"
  98. at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
  99. at java.lang.Integer.parseInt(Integer.java:492)
  100. at java.lang.Integer.valueOf(Integer.java:582)
  101. at test3.__loadFromFields(test3.java:303)
  102. ... 13 more
  103. 17/04/10 09:36:29 INFO mapreduce.Job: Job job_local2136897360_0001 running in uber mode : false
  104. 17/04/10 09:36:29 INFO mapreduce.Job: map 100% reduce 0%
  105. 17/04/10 09:36:29 INFO mapreduce.Job: Job job_local2136897360_0001 failed with state FAILED due to: NA
  106. 17/04/10 09:36:29 INFO mapreduce.Job: Counters: 15
  107. File System Counters
  108. FILE: Number of bytes read=673345391
  109. FILE: Number of bytes written=679694703
  110. FILE: Number of read operations=0
  111. FILE: Number of large read operations=0
  112. FILE: Number of write operations=0
  113. Map-Reduce Framework
  114. Map input records=2
  115. Map output records=2
  116. Input split bytes=388
  117. Spilled Records=0
  118. Failed Shuffles=0
  119. Merged Map outputs=0
  120. GC time elapsed (ms)=0
  121. Total committed heap usage (bytes)=2805989376
  122. File Input Format Counters
  123. Bytes Read=0
  124. File Output Format Counters
  125. Bytes Written=0
  126. 17/04/10 09:36:29 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 5.4541 seconds (0 bytes/sec)
  127. 17/04/10 09:36:29 INFO mapreduce.ExportJobBase: Exported 2 records.
  128. 17/04/10 09:36:29 ERROR mapreduce.ExportJobBase: Export job failed!
  129. 17/04/10 09:36:29 ERROR tool.ExportTool: Error during export: Export job failed!
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!