Want more features on Pastebin? Sign Up, it's FREE!
Guest

MySQL to Hive Import

By: a guest on Jul 3rd, 2013  |  syntax: None  |  size: 5.15 KB  |  views: 39  |  expires: Never
download  |  raw  |  embed  |  report abuse  |  print
This paste has a previous version, view the difference. Text below is selected. Please press Ctrl+C to copy to your clipboard. (⌘+C on Mac)
  1. ssk01:~/siddharth/tools/sqoop-1.4.3.bin__hadoop-1.0.0 # ./bin/sqoop import --connect jdbc:mysql://localhost/ClassicModels -table Customers -m 1 --hive-home /root/siddharth/tools/hive-0.11.0-bin --hive-import
  2.  
  3. Warning: /usr/lib/hbase does not exist! HBase imports will fail.
  4. Please set $HBASE_HOME to the root of your HBase installation.
  5. 13/07/03 05:09:09 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
  6. 13/07/03 05:09:09 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
  7. 13/07/03 05:09:09 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
  8. 13/07/03 05:09:09 INFO tool.CodeGenTool: Beginning code generation
  9. 13/07/03 05:09:10 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `Customers` AS t LIMIT 1
  10. 13/07/03 05:09:10 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `Customers` AS t LIMIT 1
  11. 13/07/03 05:09:10 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /root/siddharth/tools/hadoop-1.1.2
  12. Note: /tmp/sqoop-root/compile/e72df40aa8eb926787fa71e1e06e5838/Customers.java uses or overrides a deprecated API.
  13. Note: Recompile with -Xlint:deprecation for details.
  14. 13/07/03 05:09:10 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/e72df40aa8eb926787fa71e1e06e5838/Customers.jar
  15. 13/07/03 05:09:10 WARN manager.MySQLManager: It looks like you are importing from mysql.
  16. 13/07/03 05:09:10 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
  17. 13/07/03 05:09:10 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
  18. 13/07/03 05:09:10 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
  19. 13/07/03 05:09:10 INFO mapreduce.ImportJobBase: Beginning import of Customers
  20. 13/07/03 05:09:13 INFO mapred.JobClient: Running job: job_201306240345_0043
  21. 13/07/03 05:09:14 INFO mapred.JobClient:  map 0% reduce 0%
  22. 13/07/03 05:09:20 INFO mapred.JobClient:  map 100% reduce 0%
  23. 13/07/03 05:09:21 INFO mapred.JobClient: Job complete: job_201306240345_0043
  24. 13/07/03 05:09:21 INFO mapred.JobClient: Counters: 18
  25. 13/07/03 05:09:21 INFO mapred.JobClient:   Job Counters
  26. 13/07/03 05:09:21 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=5991
  27. 13/07/03 05:09:21 INFO mapred.JobClient:     Total time spent by all reduces waiting after reserving slots (ms)=0
  28. 13/07/03 05:09:21 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving slots (ms)=0
  29. 13/07/03 05:09:21 INFO mapred.JobClient:     Launched map tasks=1
  30. 13/07/03 05:09:21 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
  31. 13/07/03 05:09:21 INFO mapred.JobClient:   File Output Format Counters
  32. 13/07/03 05:09:21 INFO mapred.JobClient:     Bytes Written=15556
  33. 13/07/03 05:09:21 INFO mapred.JobClient:   FileSystemCounters
  34. 13/07/03 05:09:21 INFO mapred.JobClient:     HDFS_BYTES_READ=87
  35. 13/07/03 05:09:21 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=61093
  36. 13/07/03 05:09:21 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=15556
  37. 13/07/03 05:09:21 INFO mapred.JobClient:   File Input Format Counters
  38. 13/07/03 05:09:21 INFO mapred.JobClient:     Bytes Read=0
  39. 13/07/03 05:09:21 INFO mapred.JobClient:   Map-Reduce Framework
  40. 13/07/03 05:09:21 INFO mapred.JobClient:     Map input records=122
  41. 13/07/03 05:09:21 INFO mapred.JobClient:     Physical memory (bytes) snapshot=99495936
  42. 13/07/03 05:09:21 INFO mapred.JobClient:     Spilled Records=0
  43. 13/07/03 05:09:21 INFO mapred.JobClient:     CPU time spent (ms)=690
  44. 13/07/03 05:09:21 INFO mapred.JobClient:     Total committed heap usage (bytes)=52297728
  45. 13/07/03 05:09:21 INFO mapred.JobClient:     Virtual memory (bytes) snapshot=521707520
  46. 13/07/03 05:09:21 INFO mapred.JobClient:     Map output records=122
  47. 13/07/03 05:09:21 INFO mapred.JobClient:     SPLIT_RAW_BYTES=87
  48. 13/07/03 05:09:21 INFO mapreduce.ImportJobBase: Transferred 15.1914 KB in 10.6546 seconds (1.4258 KB/sec)
  49. 13/07/03 05:09:21 INFO mapreduce.ImportJobBase: Retrieved 122 records.
  50. 13/07/03 05:09:21 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `Customers` AS t LIMIT 1
  51. 13/07/03 05:09:21 INFO hive.HiveImport: Removing temporary files from import process: hdfs://localhost:9000/user/root/Customers/_logs
  52. 13/07/03 05:09:21 INFO hive.HiveImport: Loading uploaded data into Hive
  53. 13/07/03 05:09:23 INFO hive.HiveImport:
  54. 13/07/03 05:09:23 INFO hive.HiveImport: Logging initialized using configuration in jar:file:/root/siddharth/tools/hive-0.11.0-bin/lib/hive-common-0.11.0.jar!/hive-log4j.properties
  55. 13/07/03 05:09:23 INFO hive.HiveImport: Hive history file=/tmp/root/hive_job_log_root_22980@ssk01.vxindia.veritas.com_201307030509_1500912543.txt
  56. 13/07/03 05:09:29 INFO hive.HiveImport: OK
  57. 13/07/03 05:09:29 INFO hive.HiveImport: Time taken: 6.037 seconds
  58. 13/07/03 05:09:29 INFO hive.HiveImport: Loading data to table default.customers
  59. 13/07/03 05:09:30 INFO hive.HiveImport: Table default.customers stats: [num_partitions: 0, num_files: 2, num_rows: 0, total_size: 15556, raw_data_size: 0]
  60. 13/07/03 05:09:30 INFO hive.HiveImport: OK
  61. 13/07/03 05:09:30 INFO hive.HiveImport: Time taken: 0.74 seconds
  62. 13/07/03 05:09:30 INFO hive.HiveImport: Hive import complete.
  63. 13/07/03 05:09:30 INFO hive.HiveImport: Export directory is empty, removing it.
clone this paste RAW Paste Data