API tools faq
paste
Guest User

Untitled

a guest
Jul 26th, 2016
166
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 9.39 KB | None | 0 0
raw download clone embed print report
  1. 2016-07-26 18:06:52,459 INFO [Thread-8]: compactor.Initiator (Initiator.java:run(89)) - Checking to see if we should compact default.data_aaa.dt=20160726
  2. 2016-07-26 18:06:52,459 DEBUG [Thread-8]: metastore.ObjectStore (ObjectStore.java:debugLog(6848)) - Open transaction: count = 1, isActive = true at:
  3. org.apache.hadoop.hive.metastore.ObjectStore.getTable(ObjectStore.java:936)
  4. 2016-07-26 18:06:52,460 DEBUG [Thread-8]: metastore.ObjectStore (ObjectStore.java:debugLog(6848)) - Open transaction: count = 2, isActive = true at:
  5. org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:999)
  6. 2016-07-26 18:06:52,463 DEBUG [Thread-8]: metastore.ObjectStore (ObjectStore.java:debugLog(6848)) - Commit transaction: count = 1, isactive true at:
  7. org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:1007)
  8. 2016-07-26 18:06:52,469 DEBUG [Thread-8]: metastore.ObjectStore (ObjectStore.java:debugLog(6848)) - Commit transaction: count = 0, isactive true at:
  9. org.apache.hadoop.hive.metastore.ObjectStore.getTable(ObjectStore.java:938)
  10. 2016-07-26 18:06:52,470 DEBUG [Thread-8]: metastore.ObjectStore (ObjectStore.java:debugLog(6848)) - Open transaction: count = 1, isActive = true at:
  11. org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.start(ObjectStore.java:2417)
  12. 2016-07-26 18:06:52,472 DEBUG [Thread-8]: metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:timingTrace(804)) - Direct SQL query in 0.09325ms + 0.006401ms, the query is [SET @@session.sql_mode=ANSI_QUOTES]
  13. 2016-07-26 18:06:52,474 DEBUG [Thread-8]: metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:timingTrace(804)) - Direct SQL query in 0.643516ms + 0.227728ms, the query is [select "PARTITIONS"."PART_ID", "SDS"."SD_ID", "SDS"."CD_ID", "SERDES"."SERDE_ID", "PARTITIONS"."CREATE_TIME", "PARTITIONS"."LAST_ACCESS_TIME", "SDS"."INPUT_FORMAT", "SDS"."IS_COMPRESSED", "SDS"."IS_STOREDASSUBDIRECTORIES", "SDS"."LOCATION", "SDS"."NUM_BUCKETS", "SDS"."OUTPUT_FORMAT", "SERDES"."NAME", "SERDES"."SLIB" from "PARTITIONS" left outer join "SDS" on "PARTITIONS"."SD_ID" = "SDS"."SD_ID" left outer join "SERDES" on "SDS"."SERDE_ID" = "SERDES"."SERDE_ID" where "PART_ID" in (105929) order by "PART_NAME" asc]
  14. 2016-07-26 18:06:52,475 DEBUG [Thread-8]: metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:timingTrace(804)) - Direct SQL query in 0.511778ms + 0.094395ms, the query is [select "PART_ID", "PARAM_KEY", "PARAM_VALUE" from "PARTITION_PARAMS" where "PART_ID" in (105929) and "PARAM_KEY" is not null order by "PART_ID" asc]
  15. 2016-07-26 18:06:52,475 DEBUG [Thread-8]: metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:timingTrace(804)) - Direct SQL query in 0.442281ms + 0.056765ms, the query is [select "PART_ID", "PART_KEY_VAL" from "PARTITION_KEY_VALS" where "PART_ID" in (105929) and "INTEGER_IDX" >= 0 order by "PART_ID" asc, "INTEGER_IDX" asc]
  16. 2016-07-26 18:06:52,476 DEBUG [Thread-8]: metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:timingTrace(804)) - Direct SQL query in 0.366498ms + 0.033421ms, the query is [select "SD_ID", "PARAM_KEY", "PARAM_VALUE" from "SD_PARAMS" where "SD_ID" in (106109) and "PARAM_KEY" is not null order by "SD_ID" asc]
  17. 2016-07-26 18:06:52,476 DEBUG [Thread-8]: metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:timingTrace(804)) - Direct SQL query in 0.353232ms + 0.031343ms, the query is [select "SD_ID", "COLUMN_NAME", "SORT_COLS"."ORDER" from "SORT_COLS" where "SD_ID" in (106109) and "INTEGER_IDX" >= 0 order by "SD_ID" asc, "INTEGER_IDX" asc]
  18. 2016-07-26 18:06:52,477 DEBUG [Thread-8]: metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:timingTrace(804)) - Direct SQL query in 0.361442ms + 0.05459ms, the query is [select "SD_ID", "BUCKET_COL_NAME" from "BUCKETING_COLS" where "SD_ID" in (106109) and "INTEGER_IDX" >= 0 order by "SD_ID" asc, "INTEGER_IDX" asc]
  19. 2016-07-26 18:06:52,477 DEBUG [Thread-8]: metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:timingTrace(804)) - Direct SQL query in 0.333516ms + 0.030909ms, the query is [select "SD_ID", "SKEWED_COL_NAME" from "SKEWED_COL_NAMES" where "SD_ID" in (106109) and "INTEGER_IDX" >= 0 order by "SD_ID" asc, "INTEGER_IDX" asc]
  20. 2016-07-26 18:06:52,478 DEBUG [Thread-8]: metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:timingTrace(804)) - Direct SQL query in 0.648632ms + 0.23802ms, the query is [select "CD_ID", "COMMENT", "COLUMN_NAME", "TYPE_NAME" from "COLUMNS_V2" where "CD_ID" in (220) and "INTEGER_IDX" >= 0 order by "CD_ID" asc, "INTEGER_IDX" asc]
  21. 2016-07-26 18:06:52,479 DEBUG [Thread-8]: metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:timingTrace(804)) - Direct SQL query in 0.452062ms + 0.055236ms, the query is [select "SERDE_ID", "PARAM_KEY", "PARAM_VALUE" from "SERDE_PARAMS" where "SERDE_ID" in (106109) and "PARAM_KEY" is not null order by "SERDE_ID" asc]
  22. 2016-07-26 18:06:52,479 DEBUG [Thread-8]: metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:timingTrace(804)) - Direct SQL query in 0.440613ms + 6.367183ms, the query is [select "PARTITIONS"."PART_ID" from "PARTITIONS" inner join "TBLS" on "PARTITIONS"."TBL_ID" = "TBLS"."TBL_ID" and "TBLS"."TBL_NAME" = ? inner join "DBS" on "TBLS"."DB_ID" = "DBS"."DB_ID" and "DBS"."NAME" = ? where "PARTITIONS"."PART_NAME" in (?)]
  23. 2016-07-26 18:06:52,479 DEBUG [Thread-8]: metastore.ObjectStore (ObjectStore.java:debugLog(6848)) - Commit transaction: count = 0, isactive true at:
  24. org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.commit(ObjectStore.java:2467)
  25. 2016-07-26 18:06:52,480 DEBUG [Thread-8]: metastore.ObjectStore (ObjectStore.java:commit(2469)) - 1 entries retrieved using SQL in 9.378114ms
  26. 2016-07-26 18:06:52,480 DEBUG [Thread-8]: compactor.CompactorThread (CompactorThread.java:findUserToRunAs(153)) - Determining who to run the job as.
  27. 2016-07-26 18:06:52,481 DEBUG [IPC Parameter Sending Thread #0]: ipc.Client (Client.java:run(1035)) - IPC Client (1970774753) connection to sorm-master01.msk.mts.ru/172.16.12.1:8020 from hive sending #6
  28. 2016-07-26 18:06:52,483 DEBUG [IPC Client (1970774753) connection to sorm-master01.msk.mts.ru/172.16.12.1:8020 from hive]: ipc.Client (Client.java:receiveRpcResponse(1092)) - IPC Client (1970774753) connection to sorm-master01.msk.mts.ru/172.16.12.1:8020 from hive got value #6
  29. 2016-07-26 18:06:52,483 DEBUG [Thread-8]: ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(250)) - Call: getFileInfo took 3ms
  30. 2016-07-26 18:06:52,483 DEBUG [Thread-8]: compactor.CompactorThread (CompactorThread.java:findUserToRunAs(158)) - Running job as storm
  31. 2016-07-26 18:06:52,483 INFO [Thread-8]: compactor.Initiator (Initiator.java:checkForCompaction(208)) - Going to initiate as user storm
  32. 2016-07-26 18:06:52,484 DEBUG [Thread-8]: security.UserGroupInformation (UserGroupInformation.java:logPrivilegedAction(1681)) - PrivilegedAction as:storm (auth:PROXY) via hive (auth:SIMPLE) from:org.apache.hadoop.hive.ql.txn.compactor.Initiator.checkForCompaction(Initiator.java:211)
  33. 2016-07-26 18:06:52,485 DEBUG [Thread-8]: hdfs.BlockReaderLocal (DFSClient.java:<init>(457)) - dfs.client.use.legacy.blockreader.local = false
  34. 2016-07-26 18:06:52,485 DEBUG [Thread-8]: hdfs.BlockReaderLocal (DFSClient.java:<init>(460)) - dfs.client.read.shortcircuit = true
  35. 2016-07-26 18:06:52,485 DEBUG [Thread-8]: hdfs.BlockReaderLocal (DFSClient.java:<init>(463)) - dfs.client.domain.socket.data.traffic = false
  36. 2016-07-26 18:06:52,485 DEBUG [Thread-8]: hdfs.BlockReaderLocal (DFSClient.java:<init>(466)) - dfs.domain.socket.path = /var/lib/hadoop-hdfs/dn_socket
  37. 2016-07-26 18:06:52,486 DEBUG [Thread-8]: retry.RetryUtils (RetryUtils.java:getDefaultRetryPolicy(75)) - multipleLinearRandomRetry = null
  38. 2016-07-26 18:06:52,486 DEBUG [Thread-8]: ipc.Client (ClientCache.java:getClient(63)) - getting client out of cache: org.apache.hadoop.ipc.Client@6a006276
  39. 2016-07-26 18:06:52,486 DEBUG [Thread-8]: sasl.DataTransferSaslUtil (DataTransferSaslUtil.java:getSaslPropertiesResolver(183)) - DataTransferProtocol not using SaslPropertiesResolver, no QOP found in configuration for dfs.data.transfer.protection
  40. 2016-07-26 18:06:52,487 DEBUG [Thread-8]: ipc.Client (Client.java:<init>(432)) - The ping interval is 60000 ms.
  41. 2016-07-26 18:06:52,488 DEBUG [Thread-8]: ipc.Client (Client.java:setupIOstreams(702)) - Connecting to sorm-master01.msk.mts.ru/172.16.12.1:8020
  42. 2016-07-26 18:06:52,489 DEBUG [IPC Client (1970774753) connection to sorm-master01.msk.mts.ru/172.16.12.1:8020 from storm]: ipc.Client (Client.java:run(972)) - IPC Client (1970774753) connection to sorm-master01.msk.mts.ru/172.16.12.1:8020 from storm: starting, having connections 5
  43. 2016-07-26 18:06:52,489 DEBUG [IPC Parameter Sending Thread #0]: ipc.Client (Client.java:run(1035)) - IPC Client (1970774753) connection to sorm-master01.msk.mts.ru/172.16.12.1:8020 from storm sending #7
  44. 2016-07-26 18:06:52,490 DEBUG [IPC Client (1970774753) connection to sorm-master01.msk.mts.ru/172.16.12.1:8020 from storm]: ipc.Client (Client.java:receiveRpcResponse(1092)) - IPC Client (1970774753) connection to sorm-master01.msk.mts.ru/172.16.12.1:8020 from storm got value #7
  45. 2016-07-26 18:06:52,490 DEBUG [Thread-8]: ipc.ProtobufRpcEngine (ProtobufRpcEngine.java:invoke(250)) - Call: getListing took 3ms
  46. 2016-07-26 18:06:52,496 DEBUG [Thread-8]: io.AcidUtils (AcidUtils.java:getAcidState(432)) - in directory hdfs://sorm-master01.msk.mts.ru:8020/apps/hive/warehouse/data_aaa/dt=20160726 base = null deltas = 0
  47. 2016-07-26 18:06:52,496 DEBUG [Thread-8]: compactor.Initiator (Initiator.java:determineCompactionType(271)) - delta size: 0 base size: 0 threshold: 0.1 will major compact: false
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!