API tools faq
paste
Guest User

explain_plan_mr

a guest
Mar 20th, 2017
80
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 11.01 KB | None | 0 0
raw download clone embed print report
  1. Explain
  2. STAGE DEPENDENCIES:
  3. Stage-1 is a root stage
  4. Stage-0 depends on stages: Stage-1
  5. Stage-2 depends on stages: Stage-0
  6.  
  7. STAGE PLANS:
  8. Stage: Stage-1
  9. Map Reduce
  10. Map Operator Tree:
  11. TableScan
  12. alias: test_tez_input
  13. filterExpr: ((to_date(start_date_time) <= 2016-11-20) and ((to_date(end_date_time) > 2016-11-20) or end_date_time is null)) (type: boolean)
  14. Statistics: Num rows: 3925863392 Data size: 514735215147 Basic stats: COMPLETE Column stats: NONE
  15. Filter Operator
  16. predicate: ((to_date(start_date_time) <= 2016-11-20) and ((to_date(end_date_time) > 2016-11-20) or end_date_time is null)) (type: boolean)
  17. Statistics: Num rows: 1090517608 Data size: 142982004090 Basic stats: COMPLETE Column stats: NONE
  18. Select Operator
  19. expressions: l_id (type: string), psla (type: int), fsp (type: double), lis (type: boolean), aver (type: string), asub (type: string), iid (type: string)
  20. outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
  21. Statistics: Num rows: 1090517608 Data size: 142982004090 Basic stats: COMPLETE Column stats: NONE
  22. Union
  23. Statistics: Num rows: 4362070432 Data size: 571928016360 Basic stats: COMPLETE Column stats: NONE
  24. Group By Operator
  25. aggregations: count(DISTINCT _col0), count(DISTINCT _col6)
  26. keys: _col1 (type: int), _col2 (type: double), _col3 (type: boolean), _col4 (type: string), _col5 (type: string), _col0 (type: string), _col6 (type: string)
  27. mode: hash
  28. outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
  29. Statistics: Num rows: 4362070432 Data size: 571928016360 Basic stats: COMPLETE Column stats: NONE
  30. Reduce Output Operator
  31. key expressions: _col0 (type: int), _col1 (type: double), _col2 (type: boolean), _col3 (type: string), _col4 (type: string), _col5 (type: string), _col6 (type: string)
  32. sort order: +++++++
  33. Map-reduce partition columns: _col0 (type: int), _col1 (type: double), _col2 (type: boolean), _col3 (type: string), _col4 (type: string)
  34. Statistics: Num rows: 4362070432 Data size: 571928016360 Basic stats: COMPLETE Column stats: NONE
  35. TableScan
  36. alias: test_tez_input
  37. filterExpr: ((to_date(start_date_time) <= 2016-11-20) and ((to_date(end_date_time) > 2016-11-20) or end_date_time is null)) (type: boolean)
  38. Statistics: Num rows: 3925863392 Data size: 514735215147 Basic stats: COMPLETE Column stats: NONE
  39. Filter Operator
  40. predicate: ((to_date(start_date_time) <= 2016-11-20) and ((to_date(end_date_time) > 2016-11-20) or end_date_time is null)) (type: boolean)
  41. Statistics: Num rows: 1090517608 Data size: 142982004090 Basic stats: COMPLETE Column stats: NONE
  42. Select Operator
  43. expressions: l_id (type: string), psla (type: int), fsp (type: double), lis (type: boolean), aver (type: string), asub (type: string), iid (type: string)
  44. outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
  45. Statistics: Num rows: 1090517608 Data size: 142982004090 Basic stats: COMPLETE Column stats: NONE
  46. Union
  47. Statistics: Num rows: 4362070432 Data size: 571928016360 Basic stats: COMPLETE Column stats: NONE
  48. Group By Operator
  49. aggregations: count(DISTINCT _col0), count(DISTINCT _col6)
  50. keys: _col1 (type: int), _col2 (type: double), _col3 (type: boolean), _col4 (type: string), _col5 (type: string), _col0 (type: string), _col6 (type: string)
  51. mode: hash
  52. outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
  53. Statistics: Num rows: 4362070432 Data size: 571928016360 Basic stats: COMPLETE Column stats: NONE
  54. Reduce Output Operator
  55. key expressions: _col0 (type: int), _col1 (type: double), _col2 (type: boolean), _col3 (type: string), _col4 (type: string), _col5 (type: string), _col6 (type: string)
  56. sort order: +++++++
  57. Map-reduce partition columns: _col0 (type: int), _col1 (type: double), _col2 (type: boolean), _col3 (type: string), _col4 (type: string)
  58. Statistics: Num rows: 4362070432 Data size: 571928016360 Basic stats: COMPLETE Column stats: NONE
  59. TableScan
  60. alias: test_tez_input
  61. filterExpr: ((to_date(start_date_time) <= 2016-11-20) and ((to_date(end_date_time) > 2016-11-20) or end_date_time is null)) (type: boolean)
  62. Statistics: Num rows: 3925863392 Data size: 514735215147 Basic stats: COMPLETE Column stats: NONE
  63. Filter Operator
  64. predicate: ((to_date(start_date_time) <= 2016-11-20) and ((to_date(end_date_time) > 2016-11-20) or end_date_time is null)) (type: boolean)
  65. Statistics: Num rows: 1090517608 Data size: 142982004090 Basic stats: COMPLETE Column stats: NONE
  66. Select Operator
  67. expressions: l_id (type: string), psla (type: int), fsp (type: double), lis (type: boolean), aver (type: string), asub (type: string), iid (type: string)
  68. outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
  69. Statistics: Num rows: 1090517608 Data size: 142982004090 Basic stats: COMPLETE Column stats: NONE
  70. Union
  71. Statistics: Num rows: 4362070432 Data size: 571928016360 Basic stats: COMPLETE Column stats: NONE
  72. Group By Operator
  73. aggregations: count(DISTINCT _col0), count(DISTINCT _col6)
  74. keys: _col1 (type: int), _col2 (type: double), _col3 (type: boolean), _col4 (type: string), _col5 (type: string), _col0 (type: string), _col6 (type: string)
  75. mode: hash
  76. outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
  77. Statistics: Num rows: 4362070432 Data size: 571928016360 Basic stats: COMPLETE Column stats: NONE
  78. Reduce Output Operator
  79. key expressions: _col0 (type: int), _col1 (type: double), _col2 (type: boolean), _col3 (type: string), _col4 (type: string), _col5 (type: string), _col6 (type: string)
  80. sort order: +++++++
  81. Map-reduce partition columns: _col0 (type: int), _col1 (type: double), _col2 (type: boolean), _col3 (type: string), _col4 (type: string)
  82. Statistics: Num rows: 4362070432 Data size: 571928016360 Basic stats: COMPLETE Column stats: NONE
  83. TableScan
  84. alias: test_tez_input
  85. filterExpr: ((to_date(start_date_time) <= 2016-11-20) and ((to_date(end_date_time) > 2016-11-20) or end_date_time is null)) (type: boolean)
  86. Statistics: Num rows: 3925863392 Data size: 514735215147 Basic stats: COMPLETE Column stats: NONE
  87. Filter Operator
  88. predicate: ((to_date(start_date_time) <= 2016-11-20) and ((to_date(end_date_time) > 2016-11-20) or end_date_time is null)) (type: boolean)
  89. Statistics: Num rows: 1090517608 Data size: 142982004090 Basic stats: COMPLETE Column stats: NONE
  90. Select Operator
  91. expressions: l_id (type: string), psla (type: int), fsp (type: double), lis (type: boolean), aver (type: string), asub (type: string), iid (type: string)
  92. outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
  93. Statistics: Num rows: 1090517608 Data size: 142982004090 Basic stats: COMPLETE Column stats: NONE
  94. Union
  95. Statistics: Num rows: 4362070432 Data size: 571928016360 Basic stats: COMPLETE Column stats: NONE
  96. Group By Operator
  97. aggregations: count(DISTINCT _col0), count(DISTINCT _col6)
  98. keys: _col1 (type: int), _col2 (type: double), _col3 (type: boolean), _col4 (type: string), _col5 (type: string), _col0 (type: string), _col6 (type: string)
  99. mode: hash
  100. outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6, _col7, _col8
  101. Statistics: Num rows: 4362070432 Data size: 571928016360 Basic stats: COMPLETE Column stats: NONE
  102. Reduce Output Operator
  103. key expressions: _col0 (type: int), _col1 (type: double), _col2 (type: boolean), _col3 (type: string), _col4 (type: string), _col5 (type: string), _col6 (type: string)
  104. sort order: +++++++
  105. Map-reduce partition columns: _col0 (type: int), _col1 (type: double), _col2 (type: boolean), _col3 (type: string), _col4 (type: string)
  106. Statistics: Num rows: 4362070432 Data size: 571928016360 Basic stats: COMPLETE Column stats: NONE
  107. Reduce Operator Tree:
  108. Group By Operator
  109. aggregations: count(DISTINCT KEY._col5:0._col0), count(DISTINCT KEY._col5:1._col0)
  110. keys: KEY._col0 (type: int), KEY._col1 (type: double), KEY._col2 (type: boolean), KEY._col3 (type: string), KEY._col4 (type: string)
  111. mode: mergepartial
  112. outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
  113. Statistics: Num rows: 2181035216 Data size: 285964008180 Basic stats: COMPLETE Column stats: NONE
  114. Select Operator
  115. expressions: _col3 (type: string), _col4 (type: string), '2016-11-20 00:00:00' (type: string), _col5 (type: bigint), CASE WHEN (((_col0 >= 0) and (_col1 > 0.0) and (_col2 = '1'))) THEN (_col5) ELSE (0) END (type: bigint), _col6 (type: bigint), CASE WHEN (((_col0 >= 0) and (_col1 > 0.0) and (_col2 = '1'))) THEN (_col6) ELSE (0) END (type: bigint)
  116. outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5, _col6
  117. Statistics: Num rows: 2181035216 Data size: 285964008180 Basic stats: COMPLETE Column stats: NONE
  118. File Output Operator
  119. compressed: false
  120. Statistics: Num rows: 2181035216 Data size: 285964008180 Basic stats: COMPLETE Column stats: NONE
  121. table:
  122. input format: org.apache.hadoop.mapred.TextInputFormat
  123. output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
  124. serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
  125. name: test.test_tez_multiunion
  126.  
  127. Stage: Stage-0
  128. Move Operator
  129. tables:
  130. replace: true
  131. table:
  132. input format: org.apache.hadoop.mapred.TextInputFormat
  133. output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
  134. serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
  135. name: test.test_tez_multiunion
  136.  
  137. Stage: Stage-2
  138. Stats-Aggr Operator
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!