API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Feb 11th, 2020
37
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 79.34 KB | None | 0 0
raw download clone embed print report
  1.  
  2.  
  3. Query (id=9242530ef412061f:1d82eaeb00000000):
  4. Summary:
  5. Session ID: 944fff761bf09762:bdfd68a4f8283fbf
  6. Session Type: HIVESERVER2
  7. HiveServer2 Protocol Version: V6
  8. Start Time: 2020-02-11 13:48:10.117340000
  9. End Time: 2020-02-11 13:48:13.336679000
  10. Query Type: DDL
  11. Query State: FINISHED
  12. Query Status: OK
  13. Impala Version: impalad version 3.2.0-cdh6.3.0 RELEASE (build 495397101e5807c701df71ea288f4815d69c2c8a)
  14. User: nmali
  15. Connected User: nmali
  16. Delegated User:
  17. Network Address: ::ffff:172.17.250.66:36614
  18. Default Db: alerts
  19. Sql Statement: create table y5 AS
  20. SELECT
  21. CAST(orig_cust_id as INT) as orig_cust_id,
  22. CAST(dest_npa as INT) as dest_npa,
  23. CAST(dest_nxx as INT) as dest_nxx,
  24. dest_state,
  25. dest_ocn,
  26. orig_trunk_grp_nm as orig_trunk_group_name,
  27. orig_product,
  28. jurisdiction_state as jurisdiction,
  29. sum(cost) as cost,
  30. sum(revenue) as revenue,
  31. sum(conv_seconds) as duration,
  32. call_date_time_hours,
  33. call_date_time_day,
  34. call_date_time_month,
  35. call_date_time_year
  36. FROM call_detail_records.raw
  37. WHERE
  38. call_date_time_day=day(DAYS_SUB(now(), 1)) AND
  39. call_date_time_month=month(DAYS_SUB(now(), 1)) AND
  40. call_date_time_year=year(DAYS_SUB(now(), 1)) AND
  41. orig_cust_id='25'
  42. GROUP BY
  43. orig_cust_id,
  44. dest_npa,
  45. dest_nxx,
  46. dest_state,
  47. dest_ocn,
  48. orig_trunk_grp_nm,
  49. orig_product,
  50. jurisdiction_state,
  51. call_date_time_hours,
  52. call_date_time_day,
  53. call_date_time_month,
  54. call_date_time_year
  55. Coordinator: itchil804.inteliquent.com:22000
  56. Query Options (set by configuration): QUERY_TIMEOUT_S=300,IDLE_SESSION_TIMEOUT=900
  57. Query Options (set by configuration and planner): QUERY_TIMEOUT_S=300,MT_DOP=0,IDLE_SESSION_TIMEOUT=900
  58. DDL Type: CREATE_TABLE_AS_SELECT
  59. Plan:
  60. ----------------
  61. Max Per-Host Resource Reservation: Memory=116.00MB Threads=3
  62. Per-Host Resource Estimates: Memory=696MB
  63. WARNING: The following tables are missing relevant table and/or column statistics.
  64. call_detail_records.raw
  65. Analyzed query: SELECT CAST(orig_cust_id AS INT) orig_cust_id, CAST(dest_npa AS
  66. INT) dest_npa, CAST(dest_nxx AS INT) dest_nxx, dest_state, dest_ocn,
  67. orig_trunk_grp_nm orig_trunk_group_name, orig_product, jurisdiction_state
  68. jurisdiction, sum(cost) cost, sum(revenue) revenue, sum(conv_seconds) duration,
  69. call_date_time_hours, call_date_time_day, call_date_time_month,
  70. call_date_time_year FROM call_detail_records.raw WHERE call_date_time_day =
  71. CAST(10 AS INT) AND call_date_time_month = CAST(2 AS INT) AND
  72. call_date_time_year = CAST(2020 AS INT) AND orig_cust_id = '25' GROUP BY
  73. CAST(orig_cust_id AS INT), CAST(dest_npa AS INT), CAST(dest_nxx AS INT),
  74. dest_state, dest_ocn, orig_trunk_grp_nm, orig_product, jurisdiction_state,
  75. call_date_time_hours, call_date_time_day, call_date_time_month,
  76. call_date_time_year
  77.  
  78. F01:PLAN FRAGMENT [HASH(CAST(orig_cust_id AS INT),CAST(dest_npa AS INT),CAST(dest_nxx AS INT),dest_state,dest_ocn,orig_trunk_grp_nm,orig_product,jurisdiction_state,call_date_time_hours,call_date_time_day,call_date_time_month,call_date_time_year)] hosts=3 instances=3
  79. | Per-Host Resources: mem-estimate=128.43MB mem-reservation=34.00MB thread-reservation=1
  80. WRITE TO HDFS [alerts.y5, OVERWRITE=false]
  81. | partitions=1
  82. | mem-estimate=100.00KB mem-reservation=0B thread-reservation=0
  83. |
  84. 03:AGGREGATE [FINALIZE]
  85. | output: sum:merge(cost), sum:merge(revenue), sum:merge(conv_seconds)
  86. | group by: CAST(orig_cust_id AS INT), CAST(dest_npa AS INT), CAST(dest_nxx AS INT), dest_state, dest_ocn, orig_trunk_grp_nm, orig_product, jurisdiction_state, call_date_time_hours, call_date_time_day, call_date_time_month, call_date_time_year
  87. | mem-estimate=128.00MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
  88. | tuple-ids=1 row-size=144B cardinality=unavailable
  89. | in pipelines: 03(GETNEXT), 00(OPEN)
  90. |
  91. 02:EXCHANGE [HASH(CAST(orig_cust_id AS INT),CAST(dest_npa AS INT),CAST(dest_nxx AS INT),dest_state,dest_ocn,orig_trunk_grp_nm,orig_product,jurisdiction_state,call_date_time_hours,call_date_time_day,call_date_time_month,call_date_time_year)]
  92. | mem-estimate=443.95KB mem-reservation=0B thread-reservation=0
  93. | tuple-ids=1 row-size=144B cardinality=unavailable
  94. | in pipelines: 00(GETNEXT)
  95. |
  96. F00:PLAN FRAGMENT [RANDOM] hosts=3 instances=3
  97. Per-Host Resources: mem-estimate=568.00MB mem-reservation=82.00MB thread-reservation=2
  98. 01:AGGREGATE [STREAMING]
  99. | output: sum(cost), sum(revenue), sum(conv_seconds)
  100. | group by: CAST(orig_cust_id AS INT), CAST(dest_npa AS INT), CAST(dest_nxx AS INT), dest_state, dest_ocn, orig_trunk_grp_nm, orig_product, jurisdiction_state, call_date_time_hours, call_date_time_day, call_date_time_month, call_date_time_year
  101. | mem-estimate=128.00MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
  102. | tuple-ids=1 row-size=144B cardinality=unavailable
  103. | in pipelines: 00(GETNEXT)
  104. |
  105. 00:SCAN HDFS [call_detail_records.raw, RANDOM]
  106. partition predicates: call_date_time_day = CAST(10 AS INT), call_date_time_month = CAST(2 AS INT), call_date_time_year = CAST(2020 AS INT)
  107. partitions=24/631 files=478 size=14.48GB
  108. predicates: orig_cust_id = '25'
  109. stored statistics:
  110. table: rows=unavailable size=unavailable
  111. partitions: 0/24 rows=unavailable
  112. columns missing stats: orig_cust_id, dest_npa, dest_nxx, dest_state, dest_ocn, orig_trunk_grp_nm, orig_product, jurisdiction_state, cost, revenue, conv_seconds
  113. extrapolated-rows=disabled max-scan-range-rows=unavailable
  114. parquet statistics predicates: orig_cust_id = '25'
  115. parquet dictionary predicates: orig_cust_id = '25'
  116. mem-estimate=440.00MB mem-reservation=48.00MB thread-reservation=1
  117. tuple-ids=0 row-size=128B cardinality=unavailable
  118. in pipelines: 00(GETNEXT)
  119. ----------------
  120. Estimated Per-Host Mem: 730263504
  121. Tables Missing Stats: call_detail_records.raw
  122. Per Host Min Memory Reservation: itchil804.inteliquent.com:22000(116.00 MB) itchil805.inteliquent.com:22000(116.00 MB) itchil806.inteliquent.com:22000(116.00 MB)
  123. Request Pool: root.nmali
  124. Admission result: Admitted immediately
  125. Cluster Memory Admitted: 2.04 GB
  126. ExecSummary:
  127. Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail
  128. --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  129. F01:HDFS WRITER 3 803.750ms 867.374ms 50.00 KB 100.00 KB
  130. 03:AGGREGATE 3 311.296ms 335.526ms 821.57K -1 74.11 MB 128.00 MB FINALIZE
  131. 02:EXCHANGE 3 21.720ms 22.442ms 1.09M -1 1.23 MB 443.95 KB HASH(CAST(orig_cust_id AS INT),CAST(dest_npa AS INT),CAST(dest_nxx AS INT),dest_state,dest_ocn,orig_trunk_grp_nm,orig_product,jurisdiction_state,call_date_time_hours,call_date_time_day,call_date_time_month,call_date_time_year)
  132. F00:EXCHANGE SENDER 3 331.632ms 420.582ms 86.95 KB 0
  133. 01:AGGREGATE 3 960.047ms 1s366ms 1.09M -1 116.09 MB 128.00 MB STREAMING
  134. 00:SCAN HDFS 3 52.015ms 62.367ms 4.47M -1 3.15 GB 440.00 MB call_detail_records.raw
  135. Errors: Snappy: GetUncompressedLength failed
  136. File 'hdfs://itchil800.inteliquent.com:8020/user/inteliquent/data/cdr/database/call_date_time_year=2020/call_date_time_month=2/call_date_time_day=10/call_date_time_hours=16/part-00192-7937e39f-9e89-4247-8561-f0491d26c970.c000.snappy.parquet' is corrupt: error decoding value of type DECIMAL(12,12) at offset 1766 (1 of 286 similar)
  137. Query Compilation: 71.330ms
  138. - Metadata of all 1 tables cached: 1.327ms (1.327ms)
  139. - Analysis finished: 67.962ms (66.635ms)
  140. - Value transfer graph computed: 67.999ms (36.825us)
  141. - Single node plan created: 69.569ms (1.570ms)
  142. - Runtime filters computed: 69.746ms (176.729us)
  143. - Distributed plan created: 69.782ms (36.701us)
  144. - Lineage info computed: 69.894ms (111.476us)
  145. - Planning finished: 71.330ms (1.436ms)
  146. Query Timeline: 45s629ms
  147. - Query submitted: 115.761us (115.761us)
  148. - Planning finished: 73.784ms (73.668ms)
  149. - Submit for admission: 160.789ms (87.005ms)
  150. - Completed admission: 161.184ms (394.556us)
  151. - Ready to start on 3 backends: 162.253ms (1.069ms)
  152. - All 3 execution backends (6 fragment instances) started: 168.746ms (6.492ms)
  153. - Last row fetched: 3s219ms (3s050ms)
  154. - Released admission control resources: 3s219ms (81.694us)
  155. - DML data written: 3s221ms (1.924ms)
  156. - DML Metastore update finished: 3s320ms (99.025ms)
  157. - Rows available: 3s320ms (404.296us)
  158. - First row fetched: 5s189ms (1s869ms)
  159. - AdmissionControlTimeSinceLastUpdate: 35.000ms
  160. - ComputeScanRangeAssignmentTimer: 1.957ms
  161. Frontend:
  162. ImpalaServer:
  163. - CatalogOpExecTimer: 82.915ms
  164. - ClientFetchWaitTimer: 1s869ms
  165. - MetastoreUpdateTimer: 99.371ms
  166. - RowMaterializationTimer: 0.000ns
  167. Execution Profile 9242530ef412061f:1d82eaeb00000000:(Total: 3s058ms, non-child: 0.000ns, % non-child: 0.00%)
  168. Number of filters: 0
  169. Filter routing table:
  170. ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled
  171. -------------------------------------------------------------------------------------------------------------------
  172. Backend startup latencies: Count: 3, min / max: 3ms / 5ms, 25th %-ile: 3ms, 50th %-ile: 4ms, 75th %-ile: 4ms, 90th %-ile: 5ms, 95th %-ile: 5ms, 99.9th %-ile: 5ms
  173. Slowest backend to start up: itchil805.inteliquent.com:22000
  174. Per Node Peak Memory Usage: itchil804.inteliquent.com:22000(3.26 GB) itchil805.inteliquent.com:22000(2.95 GB) itchil806.inteliquent.com:22000(2.84 GB)
  175. Per Node Bytes Read: itchil804.inteliquent.com:22000(2.16 GB) itchil805.inteliquent.com:22000(1.79 GB) itchil806.inteliquent.com:22000(1.61 GB)
  176. Per Node User Time: itchil804.inteliquent.com:22000(6s780ms) itchil805.inteliquent.com:22000(6s098ms) itchil806.inteliquent.com:22000(3s917ms)
  177. Per Node System Time: itchil804.inteliquent.com:22000(594.456ms) itchil805.inteliquent.com:22000(324.952ms) itchil806.inteliquent.com:22000(317.868ms)
  178. DML Stats:
  179. Partition: Default
  180. NumModifiedRows: 821568
  181. BytesWritten: 78.54 MB
  182. - ExchangeScanRatio: 0.01
  183. - FiltersReceived: 0 (0)
  184. - FinalizationTimer: 6.587ms
  185. - FileDeletionTimer: 1.685ms
  186. - FileMoveTimer: 1.992ms
  187. - Overwrite/PartitionCreationTimer: 1.544ms
  188. - InnerNodeSelectivityRatio: 0.00
  189. - NumBackends: 3 (3)
  190. - NumFragmentInstances: 6 (6)
  191. - NumFragments: 2 (2)
  192. - TotalBytesRead: 5.56 GB (5967308785)
  193. - TotalBytesSent: 84.03 MB (88110575)
  194. - TotalCpuTime: 18s034ms
  195. - TotalInnerBytesSent: 0
  196. - TotalScanBytesSent: 84.03 MB (88110575)
  197. Per Node Profiles:
  198. itchil804.inteliquent.com:22000:
  199. - ScratchBytesRead: 0
  200. - ScratchBytesWritten: 0
  201. - ScratchFileUsedBytes: 0
  202. - ScratchReads: 0 (0)
  203. - ScratchWrites: 0 (0)
  204. - TotalEncryptionTime: 0.000ns
  205. - TotalReadBlockTime: 0.000ns
  206. itchil805.inteliquent.com:22000:
  207. - ScratchBytesRead: 0
  208. - ScratchBytesWritten: 0
  209. - ScratchFileUsedBytes: 0
  210. - ScratchReads: 0 (0)
  211. - ScratchWrites: 0 (0)
  212. - TotalEncryptionTime: 0.000ns
  213. - TotalReadBlockTime: 0.000ns
  214. itchil806.inteliquent.com:22000:
  215. - ScratchBytesRead: 0
  216. - ScratchBytesWritten: 0
  217. - ScratchFileUsedBytes: 0
  218. - ScratchReads: 0 (0)
  219. - ScratchWrites: 0 (0)
  220. - TotalEncryptionTime: 0.000ns
  221. - TotalReadBlockTime: 0.000ns
  222. Averaged Fragment F01:(Total: 2s976ms, non-child: 838.441us, % non-child: 0.03%)
  223. split sizes: min: 0, max: 0, avg: 0, stddev: 0
  224. completion times: min:2s920ms max:3s044ms mean: 2s980ms stddev:50.650ms
  225. execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec
  226. num instances: 3
  227. - AverageThreadTokens: 1.00
  228. - BloomFilterBytes: 0
  229. - ExchangeScanRatio: 0.00
  230. - PeakMemoryUsage: 74.96 MB (78604310)
  231. - PeakReservation: 74.00 MB (77594624)
  232. - PeakUsedReservation: 0
  233. - PerHostPeakMemUsage: 3.01 GB (3236270346)
  234. - RowsProduced: 273.86K (273856)
  235. - TotalNetworkReceiveTime: 1s676ms
  236. - TotalNetworkSendTime: 0.000ns
  237. - TotalStorageWaitTime: 0.000ns
  238. - TotalThreadsInvoluntaryContextSwitches: 3 (3)
  239. - TotalThreadsTotalWallClockTime: 2s975ms
  240. - TotalThreadsSysTime: 59.687ms
  241. - TotalThreadsUserTime: 1s250ms
  242. - TotalThreadsVoluntaryContextSwitches: 2.13K (2130)
  243. Buffer pool:
  244. - AllocTime: 0.000ns
  245. - CumulativeAllocationBytes: 0
  246. - CumulativeAllocations: 0 (0)
  247. - PeakReservation: 0
  248. - PeakUnpinnedBytes: 0
  249. - PeakUsedReservation: 0
  250. - ReadIoBytes: 0
  251. - ReadIoOps: 0 (0)
  252. - ReadIoWaitTime: 0.000ns
  253. - ReservationLimit: 0
  254. - SystemAllocTime: 0.000ns
  255. - WriteIoBytes: 0
  256. - WriteIoOps: 0 (0)
  257. - WriteIoWaitTime: 0.000ns
  258. Fragment Instance Lifecycle Timings:
  259. - ExecTime: 815.218ms
  260. - ExecTreeExecTime: 11.342ms
  261. - OpenTime: 2s160ms
  262. - ExecTreeOpenTime: 1s988ms
  263. - PrepareTime: 790.606us
  264. - ExecTreePrepareTime: 332.996us
  265. HdfsTableSink:(Total: 803.750ms, non-child: 803.750ms, % non-child: 100.00%)
  266. - BytesWritten: 26.18 MB (27452425)
  267. - CompressTimer: 0.000ns
  268. - EncodeTimer: 733.243ms
  269. - FilesCreated: 1 (1)
  270. - FinalizePartitionFileTimer: 6.726ms
  271. - HdfsWriteTimer: 51.731ms
  272. - PartitionsCreated: 1 (1)
  273. - PeakMemoryUsage: 50.00 KB (51200)
  274. - RowsInserted: 273.86K (273856)
  275. - TmpFileCreateTimer: 10.308ms
  276. AGGREGATION_NODE (id=3):(Total: 2s000ms, non-child: 311.296ms, % non-child: 15.56%)
  277. - PeakMemoryUsage: 74.11 MB (77713536)
  278. - RowsReturned: 273.86K (273856)
  279. - RowsReturnedRate: 136.89 K/sec
  280. GroupingAggregator 0:
  281. - BuildTime: 296.855ms
  282. - GetResultsTime: 10.707ms
  283. - HTResizeTime: 16.626ms
  284. - HashBuckets: 524.29K (524288)
  285. - LargestPartitionPercent: 4 (4)
  286. - MaxPartitionLevel: 0 (0)
  287. - NumRepartitions: 0 (0)
  288. - PartitionsCreated: 16 (16)
  289. - PeakMemoryUsage: 74.11 MB (77709440)
  290. - PeakReservation: 74.00 MB (77594624)
  291. - PeakUsedReservation: 0
  292. - RowsRepartitioned: 0 (0)
  293. - RowsReturned: 273.86K (273856)
  294. - SpilledPartitions: 0 (0)
  295. Buffer pool:
  296. - AllocTime: 19.763ms
  297. - CumulativeAllocationBytes: 74.00 MB (77594624)
  298. - CumulativeAllocations: 37 (37)
  299. - PeakReservation: 74.00 MB (77594624)
  300. - PeakUnpinnedBytes: 0
  301. - PeakUsedReservation: 74.00 MB (77594624)
  302. - ReadIoBytes: 0
  303. - ReadIoOps: 0 (0)
  304. - ReadIoWaitTime: 0.000ns
  305. - SystemAllocTime: 19.699ms
  306. - WriteIoBytes: 0
  307. - WriteIoOps: 0 (0)
  308. - WriteIoWaitTime: 0.000ns
  309. EXCHANGE_NODE (id=2):(Total: 1s689ms, non-child: 1s689ms, % non-child: 100.00%)
  310. - ConvertRowBatchTime: 9.603ms
  311. - PeakMemoryUsage: 856.00 KB (876544)
  312. - RowsReturned: 363.24K (363242)
  313. - RowsReturnedRate: 215.06 K/sec
  314. Buffer pool:
  315. - AllocTime: 8.410ms
  316. - CumulativeAllocationBytes: 112.63 MB (118098602)
  317. - CumulativeAllocations: 5.77K (5768)
  318. - PeakReservation: 856.00 KB (876544)
  319. - PeakUnpinnedBytes: 0
  320. - PeakUsedReservation: 856.00 KB (876544)
  321. - ReadIoBytes: 0
  322. - ReadIoOps: 0 (0)
  323. - ReadIoWaitTime: 0.000ns
  324. - SystemAllocTime: 3.992ms
  325. - WriteIoBytes: 0
  326. - WriteIoOps: 0 (0)
  327. - WriteIoWaitTime: 0.000ns
  328. Dequeue:
  329. - FirstBatchWaitTime: 693.464ms
  330. - TotalBytesDequeued: 61.04 MB (64005906)
  331. - TotalGetBatchTime: 1s675ms
  332. - DataWaitTime: 1s667ms
  333. Enqueue:
  334. - DeserializeRowBatchTime: 195.032ms
  335. - TotalBatchesEnqueued: 2.88K (2884)
  336. - TotalBatchesReceived: 2.88K (2884)
  337. - TotalBytesReceived: 28.01 MB (29370191)
  338. - TotalEarlySenders: 0 (0)
  339. - TotalEosReceived: 3 (3)
  340. - TotalHasDeferredRPCsTime: 0.000ns
  341. - TotalRPCsDeferred: 0 (0)
  342. CodeGen:(Total: 171.044ms, non-child: 171.044ms, % non-child: 100.00%)
  343. - CodegenInvoluntaryContextSwitches: 0 (0)
  344. - CodegenTotalWallClockTime: 171.032ms
  345. - CodegenSysTime: 1.317ms
  346. - CodegenUserTime: 169.358ms
  347. - CodegenVoluntaryContextSwitches: 0 (0)
  348. - CompileTime: 41.286ms
  349. - IrGenerationTime: 3.596ms
  350. - LoadTime: 0.000ns
  351. - ModuleBitcodeSize: 2.28 MB (2393476)
  352. - NumFunctions: 66 (66)
  353. - NumInstructions: 1.84K (1837)
  354. - OptimizationTime: 101.053ms
  355. - PeakMemoryUsage: 918.50 KB (940544)
  356. - PrepareTime: 24.181ms
  357. Fragment F01:
  358. Instance 9242530ef412061f:1d82eaeb00000003 (host=itchil806.inteliquent.com:22000):(Total: 3s039ms, non-child: 891.903us, % non-child: 0.03%)
  359. Last report received time: 2020-02-11 13:48:13.329
  360. Fragment Instance Lifecycle Event Timeline: 3s040ms
  361. - Prepare Finished: 1.279ms (1.279ms)
  362. - Open Finished: 2s160ms (2s159ms)
  363. - First Batch Produced: 2s161ms (149.925us)
  364. - First Batch Sent: 2s183ms (22.460ms)
  365. - ExecInternal Finished: 3s040ms (857.304ms)
  366. - MemoryUsage (500.000ms): 122.02 KB, 34.85 MB, 36.85 MB, 64.85 MB, 56.17 MB, 34.17 MB
  367. - ThreadUsage (500.000ms): 1, 1, 1, 1, 1, 1
  368. - AverageThreadTokens: 1.00
  369. - BloomFilterBytes: 0
  370. - ExchangeScanRatio: 0.00
  371. - PeakMemoryUsage: 74.85 MB (78481430)
  372. - PeakReservation: 74.00 MB (77594624)
  373. - PeakUsedReservation: 0
  374. - PerHostPeakMemUsage: 2.84 GB (3046039782)
  375. - RowsProduced: 273.94K (273945)
  376. - TotalNetworkReceiveTime: 1s683ms
  377. - TotalNetworkSendTime: 0.000ns
  378. - TotalStorageWaitTime: 0.000ns
  379. - TotalThreadsInvoluntaryContextSwitches: 4 (4)
  380. - TotalThreadsTotalWallClockTime: 3s038ms
  381. - TotalThreadsSysTime: 49.209ms
  382. - TotalThreadsUserTime: 1s304ms
  383. - TotalThreadsVoluntaryContextSwitches: 2.15K (2154)
  384. Buffer pool:
  385. - AllocTime: 0.000ns
  386. - CumulativeAllocationBytes: 0
  387. - CumulativeAllocations: 0 (0)
  388. - PeakReservation: 0
  389. - PeakUnpinnedBytes: 0
  390. - PeakUsedReservation: 0
  391. - ReadIoBytes: 0
  392. - ReadIoOps: 0 (0)
  393. - ReadIoWaitTime: 0.000ns
  394. - ReservationLimit: 0
  395. - SystemAllocTime: 0.000ns
  396. - WriteIoBytes: 0
  397. - WriteIoOps: 0 (0)
  398. - WriteIoWaitTime: 0.000ns
  399. Fragment Instance Lifecycle Timings:
  400. - ExecTime: 879.414ms
  401. - ExecTreeExecTime: 11.888ms
  402. - OpenTime: 2s159ms
  403. - ExecTreeOpenTime: 1s992ms
  404. - PrepareTime: 822.820us
  405. - ExecTreePrepareTime: 358.297us
  406. HdfsTableSink:(Total: 867.374ms, non-child: 867.374ms, % non-child: 100.00%)
  407. - BytesWritten: 26.19 MB (27460238)
  408. - CompressTimer: 0.000ns
  409. - EncodeTimer: 776.984ms
  410. - FilesCreated: 1 (1)
  411. - FinalizePartitionFileTimer: 7.127ms
  412. - HdfsWriteTimer: 70.327ms
  413. - PartitionsCreated: 1 (1)
  414. - PeakMemoryUsage: 50.00 KB (51200)
  415. - RowsInserted: 273.94K (273945)
  416. - TmpFileCreateTimer: 10.918ms
  417. AGGREGATION_NODE (id=3):(Total: 2s004ms, non-child: 308.419ms, % non-child: 15.39%)
  418. Node Lifecycle Event Timeline: 3s040ms
  419. - Open Started: 168.453ms (168.453ms)
  420. - Open Finished: 2s160ms (1s992ms)
  421. - First Batch Requested: 2s160ms (41.328us)
  422. - First Batch Returned: 2s161ms (144.213us)
  423. - Last Batch Returned: 3s033ms (872.110ms)
  424. - Closed: 3s040ms (7.545ms)
  425. - PeakMemoryUsage: 74.11 MB (77713536)
  426. - RowsReturned: 273.94K (273945)
  427. - RowsReturnedRate: 136.66 K/sec
  428. GroupingAggregator 0:
  429. ExecOption: Codegen Enabled
  430. - BuildTime: 293.691ms
  431. - GetResultsTime: 11.181ms
  432. - HTResizeTime: 13.450ms
  433. - HashBuckets: 524.29K (524288)
  434. - LargestPartitionPercent: 4 (4)
  435. - MaxPartitionLevel: 0 (0)
  436. - NumRepartitions: 0 (0)
  437. - PartitionsCreated: 16 (16)
  438. - PeakMemoryUsage: 74.11 MB (77709440)
  439. - PeakReservation: 74.00 MB (77594624)
  440. - PeakUsedReservation: 0
  441. - RowsRepartitioned: 0 (0)
  442. - RowsReturned: 273.94K (273945)
  443. - SpilledPartitions: 0 (0)
  444. Buffer pool:
  445. - AllocTime: 9.967ms
  446. - CumulativeAllocationBytes: 74.00 MB (77594624)
  447. - CumulativeAllocations: 37 (37)
  448. - PeakReservation: 74.00 MB (77594624)
  449. - PeakUnpinnedBytes: 0
  450. - PeakUsedReservation: 74.00 MB (77594624)
  451. - ReadIoBytes: 0
  452. - ReadIoOps: 0 (0)
  453. - ReadIoWaitTime: 0.000ns
  454. - SystemAllocTime: 9.916ms
  455. - WriteIoBytes: 0
  456. - WriteIoOps: 0 (0)
  457. - WriteIoWaitTime: 0.000ns
  458. EXCHANGE_NODE (id=2):(Total: 1s696ms, non-child: 21.941ms, % non-child: 1.29%)
  459. Node Lifecycle Event Timeline: 2s160ms
  460. - Open Started: 168.454ms (168.454ms)
  461. - Open Finished: 865.838ms (697.383ms)
  462. - First Batch Requested: 866.647ms (808.658us)
  463. - First Batch Returned: 869.325ms (2.678ms)
  464. - Last Batch Returned: 2s160ms (1s290ms)
  465. - Closed: 2s160ms (656.593us)
  466. - ConvertRowBatchTime: 9.712ms
  467. - PeakMemoryUsage: 736.00 KB (753664)
  468. - RowsReturned: 363.16K (363164)
  469. - RowsReturnedRate: 214.11 K/sec
  470. Buffer pool:
  471. - AllocTime: 6.480ms
  472. - CumulativeAllocationBytes: 112.62 MB (118087680)
  473. - CumulativeAllocations: 5.77K (5768)
  474. - PeakReservation: 736.00 KB (753664)
  475. - PeakUnpinnedBytes: 0
  476. - PeakUsedReservation: 736.00 KB (753664)
  477. - ReadIoBytes: 0
  478. - ReadIoOps: 0 (0)
  479. - ReadIoWaitTime: 0.000ns
  480. - SystemAllocTime: 1.700ms
  481. - WriteIoBytes: 0
  482. - WriteIoOps: 0 (0)
  483. - WriteIoWaitTime: 0.000ns
  484. Dequeue:
  485. - BytesDequeued (500.000ms): 0, 5.40 MB, 16.93 MB, 49.47 MB
  486. - FirstBatchWaitTime: 697.372ms
  487. - TotalBytesDequeued: 61.03 MB (63991761)
  488. - TotalGetBatchTime: 1s682ms
  489. - DataWaitTime: 1s674ms
  490. Enqueue:
  491. - BytesReceived (500.000ms): 0, 2.47 MB, 7.74 MB, 22.69 MB
  492. - DeferredQueueSize (500.000ms): 0, 0, 0, 0
  493. - DispatchTime: (Avg: 30.611us ; Min: 20.355us ; Max: 129.704us ; Number of samples: 2884)
  494. - DeserializeRowBatchTime: 205.349ms
  495. - TotalBatchesEnqueued: 2.88K (2884)
  496. - TotalBatchesReceived: 2.88K (2884)
  497. - TotalBytesReceived: 28.00 MB (29355584)
  498. - TotalEarlySenders: 0 (0)
  499. - TotalEosReceived: 3 (3)
  500. - TotalHasDeferredRPCsTime: 0.000ns
  501. - TotalRPCsDeferred: 0 (0)
  502. CodeGen:(Total: 167.071ms, non-child: 167.071ms, % non-child: 100.00%)
  503. - CodegenInvoluntaryContextSwitches: 0 (0)
  504. - CodegenTotalWallClockTime: 167.059ms
  505. - CodegenSysTime: 0.000ns
  506. - CodegenUserTime: 167.493ms
  507. - CodegenVoluntaryContextSwitches: 0 (0)
  508. - CompileTime: 40.242ms
  509. - IrGenerationTime: 3.406ms
  510. - LoadTime: 0.000ns
  511. - ModuleBitcodeSize: 2.28 MB (2393476)
  512. - NumFunctions: 66 (66)
  513. - NumInstructions: 1.84K (1837)
  514. - OptimizationTime: 99.493ms
  515. - PeakMemoryUsage: 918.50 KB (940544)
  516. - PrepareTime: 23.006ms
  517. Instance 9242530ef412061f:1d82eaeb00000005 (host=itchil805.inteliquent.com:22000):(Total: 2s972ms, non-child: 828.537us, % non-child: 0.03%)
  518. Last report received time: 2020-02-11 13:48:13.262
  519. Fragment Instance Lifecycle Event Timeline: 2s972ms
  520. - Prepare Finished: 1.296ms (1.296ms)
  521. - Open Finished: 2s161ms (2s160ms)
  522. - First Batch Produced: 2s161ms (110.087us)
  523. - First Batch Sent: 2s179ms (17.661ms)
  524. - ExecInternal Finished: 2s972ms (793.898ms)
  525. - MemoryUsage (500.000ms): 122.02 KB, 34.61 MB, 36.61 MB, 58.69 MB, 54.17 MB, 34.17 MB
  526. - ThreadUsage (500.000ms): 1, 1, 1, 1, 1, 1
  527. - AverageThreadTokens: 1.00
  528. - BloomFilterBytes: 0
  529. - ExchangeScanRatio: 0.00
  530. - PeakMemoryUsage: 74.69 MB (78317590)
  531. - PeakReservation: 74.00 MB (77594624)
  532. - PeakUsedReservation: 0
  533. - PerHostPeakMemUsage: 2.95 GB (3163636375)
  534. - RowsProduced: 273.90K (273899)
  535. - TotalNetworkReceiveTime: 1s693ms
  536. - TotalNetworkSendTime: 0.000ns
  537. - TotalStorageWaitTime: 0.000ns
  538. - TotalThreadsInvoluntaryContextSwitches: 3 (3)
  539. - TotalThreadsTotalWallClockTime: 2s971ms
  540. - TotalThreadsSysTime: 51.495ms
  541. - TotalThreadsUserTime: 1s247ms
  542. - TotalThreadsVoluntaryContextSwitches: 2.20K (2196)
  543. Buffer pool:
  544. - AllocTime: 0.000ns
  545. - CumulativeAllocationBytes: 0
  546. - CumulativeAllocations: 0 (0)
  547. - PeakReservation: 0
  548. - PeakUnpinnedBytes: 0
  549. - PeakUsedReservation: 0
  550. - ReadIoBytes: 0
  551. - ReadIoOps: 0 (0)
  552. - ReadIoWaitTime: 0.000ns
  553. - ReservationLimit: 0
  554. - SystemAllocTime: 0.000ns
  555. - WriteIoBytes: 0
  556. - WriteIoOps: 0 (0)
  557. - WriteIoWaitTime: 0.000ns
  558. Fragment Instance Lifecycle Timings:
  559. - ExecTime: 811.222ms
  560. - ExecTreeExecTime: 11.113ms
  561. - OpenTime: 2s160ms
  562. - ExecTreeOpenTime: 1s985ms
  563. - PrepareTime: 770.979us
  564. - ExecTreePrepareTime: 323.386us
  565. HdfsTableSink:(Total: 799.982ms, non-child: 799.982ms, % non-child: 100.00%)
  566. - BytesWritten: 26.19 MB (27457726)
  567. - CompressTimer: 0.000ns
  568. - EncodeTimer: 740.714ms
  569. - FilesCreated: 1 (1)
  570. - FinalizePartitionFileTimer: 6.210ms
  571. - HdfsWriteTimer: 41.486ms
  572. - PartitionsCreated: 1 (1)
  573. - PeakMemoryUsage: 50.00 KB (51200)
  574. - RowsInserted: 273.90K (273899)
  575. - TmpFileCreateTimer: 10.009ms
  576. AGGREGATION_NODE (id=3):(Total: 1s996ms, non-child: 289.943ms, % non-child: 14.52%)
  577. Node Lifecycle Event Timeline: 2s972ms
  578. - Open Started: 175.778ms (175.778ms)
  579. - Open Finished: 2s161ms (1s985ms)
  580. - First Batch Requested: 2s161ms (37.171us)
  581. - First Batch Returned: 2s161ms (106.268us)
  582. - Last Batch Returned: 2s966ms (804.881ms)
  583. - Closed: 2s972ms (6.376ms)
  584. - PeakMemoryUsage: 74.11 MB (77713536)
  585. - RowsReturned: 273.90K (273899)
  586. - RowsReturnedRate: 137.16 K/sec
  587. GroupingAggregator 0:
  588. ExecOption: Codegen Enabled
  589. - BuildTime: 275.701ms
  590. - GetResultsTime: 10.530ms
  591. - HTResizeTime: 13.512ms
  592. - HashBuckets: 524.29K (524288)
  593. - LargestPartitionPercent: 4 (4)
  594. - MaxPartitionLevel: 0 (0)
  595. - NumRepartitions: 0 (0)
  596. - PartitionsCreated: 16 (16)
  597. - PeakMemoryUsage: 74.11 MB (77709440)
  598. - PeakReservation: 74.00 MB (77594624)
  599. - PeakUsedReservation: 0
  600. - RowsRepartitioned: 0 (0)
  601. - RowsReturned: 273.90K (273899)
  602. - SpilledPartitions: 0 (0)
  603. Buffer pool:
  604. - AllocTime: 2.165ms
  605. - CumulativeAllocationBytes: 74.00 MB (77594624)
  606. - CumulativeAllocations: 37 (37)
  607. - PeakReservation: 74.00 MB (77594624)
  608. - PeakUnpinnedBytes: 0
  609. - PeakUsedReservation: 74.00 MB (77594624)
  610. - ReadIoBytes: 0
  611. - ReadIoOps: 0 (0)
  612. - ReadIoWaitTime: 0.000ns
  613. - SystemAllocTime: 2.100ms
  614. - WriteIoBytes: 0
  615. - WriteIoOps: 0 (0)
  616. - WriteIoWaitTime: 0.000ns
  617. EXCHANGE_NODE (id=2):(Total: 1s706ms, non-child: 22.442ms, % non-child: 1.31%)
  618. Node Lifecycle Event Timeline: 2s161ms
  619. - Open Started: 175.780ms (175.780ms)
  620. - Open Finished: 865.373ms (689.593ms)
  621. - First Batch Requested: 866.406ms (1.032ms)
  622. - First Batch Returned: 868.543ms (2.137ms)
  623. - Last Batch Returned: 2s160ms (1s292ms)
  624. - Closed: 2s161ms (552.108us)
  625. - ConvertRowBatchTime: 10.307ms
  626. - PeakMemoryUsage: 576.00 KB (589824)
  627. - RowsReturned: 363.19K (363185)
  628. - RowsReturnedRate: 212.77 K/sec
  629. Buffer pool:
  630. - AllocTime: 4.381ms
  631. - CumulativeAllocationBytes: 112.60 MB (118071296)
  632. - CumulativeAllocations: 5.77K (5766)
  633. - PeakReservation: 576.00 KB (589824)
  634. - PeakUnpinnedBytes: 0
  635. - PeakUsedReservation: 576.00 KB (589824)
  636. - ReadIoBytes: 0
  637. - ReadIoOps: 0 (0)
  638. - ReadIoWaitTime: 0.000ns
  639. - SystemAllocTime: 74.897us
  640. - WriteIoBytes: 0
  641. - WriteIoOps: 0 (0)
  642. - WriteIoWaitTime: 0.000ns
  643. Dequeue:
  644. - BytesDequeued (500.000ms): 0, 3.43 MB, 14.98 MB, 47.61 MB
  645. - FirstBatchWaitTime: 689.575ms
  646. - TotalBytesDequeued: 61.03 MB (63996502)
  647. - TotalGetBatchTime: 1s692ms
  648. - DataWaitTime: 1s684ms
  649. Enqueue:
  650. - BytesReceived (500.000ms): 0, 1.58 MB, 6.86 MB, 21.86 MB
  651. - DeferredQueueSize (500.000ms): 0, 0, 0, 0
  652. - DispatchTime: (Avg: 24.038us ; Min: 12.147us ; Max: 107.232us ; Number of samples: 2883)
  653. - DeserializeRowBatchTime: 194.098ms
  654. - TotalBatchesEnqueued: 2.88K (2883)
  655. - TotalBatchesReceived: 2.88K (2883)
  656. - TotalBytesReceived: 28.01 MB (29368148)
  657. - TotalEarlySenders: 0 (0)
  658. - TotalEosReceived: 3 (3)
  659. - TotalHasDeferredRPCsTime: 0.000ns
  660. - TotalRPCsDeferred: 0 (0)
  661. CodeGen:(Total: 174.377ms, non-child: 174.377ms, % non-child: 100.00%)
  662. - CodegenInvoluntaryContextSwitches: 0 (0)
  663. - CodegenTotalWallClockTime: 174.364ms
  664. - CodegenSysTime: 3.952ms
  665. - CodegenUserTime: 169.965ms
  666. - CodegenVoluntaryContextSwitches: 0 (0)
  667. - CompileTime: 41.954ms
  668. - IrGenerationTime: 3.777ms
  669. - LoadTime: 0.000ns
  670. - ModuleBitcodeSize: 2.28 MB (2393476)
  671. - NumFunctions: 66 (66)
  672. - NumInstructions: 1.84K (1837)
  673. - OptimizationTime: 100.752ms
  674. - PeakMemoryUsage: 918.50 KB (940544)
  675. - PrepareTime: 26.924ms
  676. Instance 9242530ef412061f:1d82eaeb00000004 (host=itchil804.inteliquent.com:22000):(Total: 2s916ms, non-child: 794.882us, % non-child: 0.03%)
  677. Last report received time: 2020-02-11 13:48:13.205
  678. Fragment Instance Lifecycle Event Timeline: 2s917ms
  679. - Prepare Finished: 1.152ms (1.152ms)
  680. - Open Finished: 2s161ms (2s160ms)
  681. - First Batch Produced: 2s162ms (147.405us)
  682. - First Batch Sent: 2s175ms (13.045ms)
  683. - ExecInternal Finished: 2s917ms (742.321ms)
  684. - MemoryUsage (500.000ms): 122.02 KB, 122.02 KB, 36.65 MB, 43.00 MB, 70.17 MB, 34.17 MB
  685. - ThreadUsage (500.000ms): 1, 1, 1, 1, 1, 1
  686. - AverageThreadTokens: 1.00
  687. - BloomFilterBytes: 0
  688. - ExchangeScanRatio: 0.00
  689. - PeakMemoryUsage: 75.35 MB (79013910)
  690. - PeakReservation: 74.00 MB (77594624)
  691. - PeakUsedReservation: 0
  692. - PerHostPeakMemUsage: 3.26 GB (3499134881)
  693. - RowsProduced: 273.72K (273724)
  694. - TotalNetworkReceiveTime: 1s653ms
  695. - TotalNetworkSendTime: 0.000ns
  696. - TotalStorageWaitTime: 0.000ns
  697. - TotalThreadsInvoluntaryContextSwitches: 3 (3)
  698. - TotalThreadsTotalWallClockTime: 2s915ms
  699. - TotalThreadsSysTime: 78.359ms
  700. - TotalThreadsUserTime: 1s200ms
  701. - TotalThreadsVoluntaryContextSwitches: 2.04K (2041)
  702. Buffer pool:
  703. - AllocTime: 0.000ns
  704. - CumulativeAllocationBytes: 0
  705. - CumulativeAllocations: 0 (0)
  706. - PeakReservation: 0
  707. - PeakUnpinnedBytes: 0
  708. - PeakUsedReservation: 0
  709. - ReadIoBytes: 0
  710. - ReadIoOps: 0 (0)
  711. - ReadIoWaitTime: 0.000ns
  712. - ReservationLimit: 0
  713. - SystemAllocTime: 0.000ns
  714. - WriteIoBytes: 0
  715. - WriteIoOps: 0 (0)
  716. - WriteIoWaitTime: 0.000ns
  717. Fragment Instance Lifecycle Timings:
  718. - ExecTime: 755.018ms
  719. - ExecTreeExecTime: 11.023ms
  720. - OpenTime: 2s160ms
  721. - ExecTreeOpenTime: 1s988ms
  722. - PrepareTime: 778.020us
  723. - ExecTreePrepareTime: 317.306us
  724. HdfsTableSink:(Total: 743.893ms, non-child: 743.893ms, % non-child: 100.00%)
  725. - BytesWritten: 26.17 MB (27439311)
  726. - CompressTimer: 0.000ns
  727. - EncodeTimer: 682.031ms
  728. - FilesCreated: 1 (1)
  729. - FinalizePartitionFileTimer: 6.841ms
  730. - HdfsWriteTimer: 43.381ms
  731. - PartitionsCreated: 1 (1)
  732. - PeakMemoryUsage: 50.00 KB (51200)
  733. - RowsInserted: 273.72K (273724)
  734. - TmpFileCreateTimer: 9.999ms
  735. AGGREGATION_NODE (id=3):(Total: 2s000ms, non-child: 335.526ms, % non-child: 16.77%)
  736. Node Lifecycle Event Timeline: 2s917ms
  737. - Open Started: 172.922ms (172.922ms)
  738. - Open Finished: 2s161ms (1s988ms)
  739. - First Batch Requested: 2s161ms (37.789us)
  740. - First Batch Returned: 2s162ms (142.295us)
  741. - Last Batch Returned: 2s910ms (747.998ms)
  742. - Closed: 2s917ms (7.239ms)
  743. - PeakMemoryUsage: 74.11 MB (77713536)
  744. - RowsReturned: 273.72K (273724)
  745. - RowsReturnedRate: 136.85 K/sec
  746. GroupingAggregator 0:
  747. ExecOption: Codegen Enabled
  748. - BuildTime: 321.172ms
  749. - GetResultsTime: 10.409ms
  750. - HTResizeTime: 22.917ms
  751. - HashBuckets: 524.29K (524288)
  752. - LargestPartitionPercent: 4 (4)
  753. - MaxPartitionLevel: 0 (0)
  754. - NumRepartitions: 0 (0)
  755. - PartitionsCreated: 16 (16)
  756. - PeakMemoryUsage: 74.11 MB (77709440)
  757. - PeakReservation: 74.00 MB (77594624)
  758. - PeakUsedReservation: 0
  759. - RowsRepartitioned: 0 (0)
  760. - RowsReturned: 273.72K (273724)
  761. - SpilledPartitions: 0 (0)
  762. Buffer pool:
  763. - AllocTime: 47.156ms
  764. - CumulativeAllocationBytes: 74.00 MB (77594624)
  765. - CumulativeAllocations: 37 (37)
  766. - PeakReservation: 74.00 MB (77594624)
  767. - PeakUnpinnedBytes: 0
  768. - PeakUsedReservation: 74.00 MB (77594624)
  769. - ReadIoBytes: 0
  770. - ReadIoOps: 0 (0)
  771. - ReadIoWaitTime: 0.000ns
  772. - SystemAllocTime: 47.082ms
  773. - WriteIoBytes: 0
  774. - WriteIoOps: 0 (0)
  775. - WriteIoWaitTime: 0.000ns
  776. EXCHANGE_NODE (id=2):(Total: 1s664ms, non-child: 20.777ms, % non-child: 1.25%)
  777. Node Lifecycle Event Timeline: 2s161ms
  778. - Open Started: 172.924ms (172.924ms)
  779. - Open Finished: 866.380ms (693.456ms)
  780. - First Batch Requested: 867.441ms (1.060ms)
  781. - First Batch Returned: 869.699ms (2.258ms)
  782. - Last Batch Returned: 2s160ms (1s291ms)
  783. - Closed: 2s161ms (898.446us)
  784. - ConvertRowBatchTime: 8.791ms
  785. - PeakMemoryUsage: 1.23 MB (1286144)
  786. - RowsReturned: 363.38K (363379)
  787. - RowsReturnedRate: 218.28 K/sec
  788. Buffer pool:
  789. - AllocTime: 14.370ms
  790. - CumulativeAllocationBytes: 112.66 MB (118136832)
  791. - CumulativeAllocations: 5.77K (5772)
  792. - PeakReservation: 1.23 MB (1286144)
  793. - PeakUnpinnedBytes: 0
  794. - PeakUsedReservation: 1.23 MB (1286144)
  795. - ReadIoBytes: 0
  796. - ReadIoOps: 0 (0)
  797. - ReadIoWaitTime: 0.000ns
  798. - SystemAllocTime: 10.203ms
  799. - WriteIoBytes: 0
  800. - WriteIoOps: 0 (0)
  801. - WriteIoWaitTime: 0.000ns
  802. Dequeue:
  803. - BytesDequeued (500.000ms): 0, 0, 12.51 MB, 32.70 MB
  804. - FirstBatchWaitTime: 693.444ms
  805. - TotalBytesDequeued: 61.06 MB (64029457)
  806. - TotalGetBatchTime: 1s652ms
  807. - DataWaitTime: 1s643ms
  808. Enqueue:
  809. - BytesReceived (500.000ms): 0, 0, 5.71 MB, 15.13 MB
  810. - DeferredQueueSize (500.000ms): 0, 0, 0, 0
  811. - DispatchTime: (Avg: 24.319us ; Min: 8.576us ; Max: 151.609us ; Number of samples: 2886)
  812. - DeserializeRowBatchTime: 185.649ms
  813. - TotalBatchesEnqueued: 2.89K (2886)
  814. - TotalBatchesReceived: 2.89K (2886)
  815. - TotalBytesReceived: 28.03 MB (29386843)
  816. - TotalEarlySenders: 0 (0)
  817. - TotalEosReceived: 3 (3)
  818. - TotalHasDeferredRPCsTime: 0.000ns
  819. - TotalRPCsDeferred: 0 (0)
  820. CodeGen:(Total: 171.684ms, non-child: 171.684ms, % non-child: 100.00%)
  821. - CodegenInvoluntaryContextSwitches: 1 (1)
  822. - CodegenTotalWallClockTime: 171.673ms
  823. - CodegenSysTime: 0.000ns
  824. - CodegenUserTime: 170.616ms
  825. - CodegenVoluntaryContextSwitches: 0 (0)
  826. - CompileTime: 41.662ms
  827. - IrGenerationTime: 3.604ms
  828. - LoadTime: 0.000ns
  829. - ModuleBitcodeSize: 2.28 MB (2393476)
  830. - NumFunctions: 66 (66)
  831. - NumInstructions: 1.84K (1837)
  832. - OptimizationTime: 102.912ms
  833. - PeakMemoryUsage: 918.50 KB (940544)
  834. - PrepareTime: 22.612ms
  835. Averaged Fragment F00:(Total: 1s676ms, non-child: 1.080ms, % non-child: 0.06%)
  836. split sizes: min: 4.61 GB, max: 5.26 GB, avg: 4.83 GB, stddev: 311.98 MB
  837. completion times: min:2s920ms max:3s044ms mean: 2s980ms stddev:50.652ms
  838. execution rates: min:1.51 GB/sec max:1.80 GB/sec mean:1.62 GB/sec stddev:130.32 MB/sec
  839. num instances: 3
  840. - AverageThreadTokens: 18.78
  841. - BloomFilterBytes: 0
  842. - ExchangeScanRatio: 0.01
  843. - PeakMemoryUsage: 2.98 GB (3198737993)
  844. - PeakReservation: 2.86 GB (3068133376)
  845. - PeakUsedReservation: 0
  846. - PerHostPeakMemUsage: 3.01 GB (3236270346)
  847. - RowsProduced: 363.24K (363242)
  848. - TotalNetworkReceiveTime: 0.000ns
  849. - TotalNetworkSendTime: 11.104ms
  850. - TotalStorageWaitTime: 4s645ms
  851. - TotalThreadsInvoluntaryContextSwitches: 17 (17)
  852. - TotalThreadsTotalWallClockTime: 29s661ms
  853. - TotalThreadsSysTime: 352.737ms
  854. - TotalThreadsUserTime: 4s348ms
  855. - TotalThreadsVoluntaryContextSwitches: 4.20K (4204)
  856. Buffer pool:
  857. - AllocTime: 0.000ns
  858. - CumulativeAllocationBytes: 0
  859. - CumulativeAllocations: 0 (0)
  860. - PeakReservation: 0
  861. - PeakUnpinnedBytes: 0
  862. - PeakUsedReservation: 0
  863. - ReadIoBytes: 0
  864. - ReadIoOps: 0 (0)
  865. - ReadIoWaitTime: 0.000ns
  866. - ReservationLimit: 0
  867. - SystemAllocTime: 0.000ns
  868. - WriteIoBytes: 0
  869. - WriteIoOps: 0 (0)
  870. - WriteIoWaitTime: 0.000ns
  871. Fragment Instance Lifecycle Timings:
  872. - ExecTime: 1s352ms
  873. - ExecTreeExecTime: 1s009ms
  874. - OpenTime: 321.365ms
  875. - ExecTreeOpenTime: 403.603us
  876. - PrepareTime: 2.613ms
  877. - ExecTreePrepareTime: 1.899ms
  878. KrpcDataStreamSender (dst_id=2):(Total: 342.999ms, non-child: 342.999ms, % non-child: 100.00%)
  879. - EosSent: 3 (3)
  880. - PeakMemoryUsage: 86.95 KB (89040)
  881. - RowsSent: 363.24K (363242)
  882. - RpcFailure: 0 (0)
  883. - RpcRetry: 0 (0)
  884. - SerializeBatchTime: 178.712ms
  885. - TotalBytesSent: 28.01 MB (29370191)
  886. - UncompressedRowBatchSize: 61.04 MB (64005906)
  887. AGGREGATION_NODE (id=1):(Total: 1s012ms, non-child: 960.047ms, % non-child: 94.86%)
  888. - PeakMemoryUsage: 102.44 MB (107417901)
  889. - RowsReturned: 363.24K (363242)
  890. - RowsReturnedRate: 370.53 K/sec
  891. GroupingAggregator 0:
  892. - BuildTime: 0.000ns
  893. - GetResultsTime: 16.037ms
  894. - HTResizeTime: 24.890ms
  895. - HashBuckets: 873.81K (873813)
  896. - LargestPartitionPercent: 1 (1)
  897. - PartitionsCreated: 16 (16)
  898. - PeakMemoryUsage: 100.77 MB (105667370)
  899. - PeakReservation: 100.67 MB (105556650)
  900. - PeakUsedReservation: 0
  901. - ReductionFactorEstimate: 3.90
  902. - ReductionFactorThresholdToExpand: 2.00
  903. - RowsPassedThrough: 0 (0)
  904. - RowsReturned: 363.24K (363242)
  905. - StreamingTime: 829.816ms
  906. Buffer pool:
  907. - AllocTime: 32.864ms
  908. - CumulativeAllocationBytes: 100.67 MB (105556650)
  909. - CumulativeAllocations: 50 (50)
  910. - PeakReservation: 100.67 MB (105556650)
  911. - PeakUnpinnedBytes: 0
  912. - PeakUsedReservation: 100.67 MB (105556650)
  913. - ReadIoBytes: 0
  914. - ReadIoOps: 0 (0)
  915. - ReadIoWaitTime: 0.000ns
  916. - SystemAllocTime: 32.799ms
  917. - WriteIoBytes: 0
  918. - WriteIoOps: 0 (0)
  919. - WriteIoWaitTime: 0.000ns
  920. HDFS_SCAN_NODE (id=0):(Total: 52.015ms, non-child: 52.015ms, % non-child: 100.00%)
  921. - AverageHdfsReadThreadConcurrency: 2.67
  922. - AverageScannerThreadConcurrency: 32.00
  923. - BytesRead: 1.85 GB (1989102928)
  924. - BytesReadDataNodeCache: 0
  925. - BytesReadLocal: 1.85 GB (1989102928)
  926. - BytesReadRemoteUnexpected: 0
  927. - BytesReadShortCircuit: 1.85 GB (1989102928)
  928. - CachedFileHandlesHitCount: 1.52K (1517)
  929. - CachedFileHandlesMissCount: 0 (0)
  930. - CollectionItemsRead: 0 (0)
  931. - DataCacheHitBytes: 0
  932. - DataCacheHitCount: 0 (0)
  933. - DataCacheMissBytes: 0
  934. - DataCacheMissCount: 0 (0)
  935. - DataCachePartialHitCount: 0 (0)
  936. - DecompressionTime: 773.288ms
  937. - MaterializeTupleTime(*): 2s108ms
  938. - MaxCompressedTextFileLength: 0
  939. - NumColumns: 11 (11)
  940. - NumDictFilteredRowGroups: 30 (30)
  941. - NumDisksAccessed: 8 (8)
  942. - NumRowGroups: 163 (163)
  943. - NumScannerThreadMemUnavailable: 0 (0)
  944. - NumScannerThreadReservationsDenied: 0 (0)
  945. - NumScannerThreadsStarted: 32 (32)
  946. - NumScannersWithNoReads: 0 (0)
  947. - NumStatsFilteredRowGroups: 23 (23)
  948. - PeakMemoryUsage: 2.92 GB (3132809789)
  949. - PeakScannerThreadConcurrency: 32 (32)
  950. - PerReadThreadRawHdfsThroughput: 927.36 MB/sec
  951. - RemoteScanRanges: 0 (0)
  952. - RowBatchBytesEnqueued: 1.03 GB (1108820087)
  953. - RowBatchQueueGetWaitTime: 29.965ms
  954. - RowBatchQueuePeakMemoryUsage: 39.96 MB (41900202)
  955. - RowBatchQueuePutWaitTime: 19s566ms
  956. - RowBatchesEnqueued: 1.84K (1845)
  957. - RowsRead: 9.80M (9798354)
  958. - RowsReturned: 1.49M (1490587)
  959. - RowsReturnedRate: 29.05 M/sec
  960. - ScanRangesComplete: 160 (160)
  961. - ScannerIoWaitTime: 4s645ms
  962. - ScannerThreadWorklessLoops: 0 (0)
  963. - ScannerThreadsInvoluntaryContextSwitches: 15 (15)
  964. - ScannerThreadsTotalWallClockTime: 27s987ms
  965. - ScannerThreadsSysTime: 298.357ms
  966. - ScannerThreadsUserTime: 2s855ms
  967. - ScannerThreadsVoluntaryContextSwitches: 3.91K (3910)
  968. - TotalRawHdfsOpenFileTime(*): 3.819ms
  969. - TotalRawHdfsReadTime(*): 2s080ms
  970. - TotalReadThroughput: 1.21 GB/sec
  971. Buffer pool:
  972. - AllocTime: 239.675ms
  973. - CumulativeAllocationBytes: 2.12 GB (2276928170)
  974. - CumulativeAllocations: 1.52K (1518)
  975. - PeakReservation: 2.80 GB (3003121664)
  976. - PeakUnpinnedBytes: 0
  977. - PeakUsedReservation: 1.56 GB (1672208384)
  978. - ReadIoBytes: 0
  979. - ReadIoOps: 0 (0)
  980. - ReadIoWaitTime: 0.000ns
  981. - SystemAllocTime: 237.600ms
  982. - WriteIoBytes: 0
  983. - WriteIoOps: 0 (0)
  984. - WriteIoWaitTime: 0.000ns
  985. CodeGen:(Total: 320.851ms, non-child: 320.851ms, % non-child: 100.00%)
  986. - CodegenInvoluntaryContextSwitches: 1 (1)
  987. - CodegenTotalWallClockTime: 320.839ms
  988. - CodegenSysTime: 1.330ms
  989. - CodegenUserTime: 319.556ms
  990. - CodegenVoluntaryContextSwitches: 0 (0)
  991. - CompileTime: 91.190ms
  992. - IrGenerationTime: 7.506ms
  993. - LoadTime: 0.000ns
  994. - ModuleBitcodeSize: 2.28 MB (2393476)
  995. - NumFunctions: 113 (113)
  996. - NumInstructions: 3.79K (3795)
  997. - OptimizationTime: 198.354ms
  998. - PeakMemoryUsage: 1.85 MB (1943040)
  999. - PrepareTime: 22.593ms
  1000. Fragment F00:
  1001. Instance 9242530ef412061f:1d82eaeb00000001 (host=itchil804.inteliquent.com:22000):(Total: 2s161ms, non-child: 1.169ms, % non-child: 0.05%)
  1002. Last report received time: 2020-02-11 13:48:13.204
  1003. Hdfs split stats (<volume id>:<# splits>/<split lengths>): 2:27/93.28 MB 7:25/743.47 MB 3:22/1.39 GB 4:25/209.13 MB 6:30/463.04 MB 5:28/281.73 MB 0:26/1.03 GB 1:28/1.09 GB
  1004. Fragment Instance Lifecycle Event Timeline: 2s162ms
  1005. - Prepare Finished: 4.023ms (4.023ms)
  1006. - Open Finished: 323.663ms (319.639ms)
  1007. - First Batch Produced: 1s711ms (1s387ms)
  1008. - First Batch Sent: 1s711ms (3.875us)
  1009. - ExecInternal Finished: 2s162ms (450.658ms)
  1010. - MemoryUsage (500.000ms): 1.97 MB, 2.61 GB, 3.08 GB, 114.19 MB
  1011. - ThreadUsage (500.000ms): 1, 33, 33, 1
  1012. - AverageThreadTokens: 17.00
  1013. - BloomFilterBytes: 0
  1014. - ExchangeScanRatio: 0.02
  1015. - PeakMemoryUsage: 3.22 GB (3460704139)
  1016. - PeakReservation: 3.08 GB (3310354432)
  1017. - PeakUsedReservation: 0
  1018. - PerHostPeakMemUsage: 3.26 GB (3499134881)
  1019. - RowsProduced: 449.98K (449977)
  1020. - TotalNetworkReceiveTime: 0.000ns
  1021. - TotalNetworkSendTime: 8.339ms
  1022. - TotalStorageWaitTime: 4s865ms
  1023. - TotalThreadsInvoluntaryContextSwitches: 24 (24)
  1024. - TotalThreadsTotalWallClockTime: 42s592ms
  1025. - TotalThreadsSysTime: 516.097ms
  1026. - TotalThreadsUserTime: 5s580ms
  1027. - TotalThreadsVoluntaryContextSwitches: 5.26K (5256)
  1028. Buffer pool:
  1029. - AllocTime: 0.000ns
  1030. - CumulativeAllocationBytes: 0
  1031. - CumulativeAllocations: 0 (0)
  1032. - PeakReservation: 0
  1033. - PeakUnpinnedBytes: 0
  1034. - PeakUsedReservation: 0
  1035. - ReadIoBytes: 0
  1036. - ReadIoOps: 0 (0)
  1037. - ReadIoWaitTime: 0.000ns
  1038. - ReservationLimit: 0
  1039. - SystemAllocTime: 0.000ns
  1040. - WriteIoBytes: 0
  1041. - WriteIoOps: 0 (0)
  1042. - WriteIoWaitTime: 0.000ns
  1043. Fragment Instance Lifecycle Timings:
  1044. - ExecTime: 1s838ms
  1045. - ExecTreeExecTime: 1s408ms
  1046. - OpenTime: 319.639ms
  1047. - ExecTreeOpenTime: 389.830us
  1048. - PrepareTime: 3.341ms
  1049. - ExecTreePrepareTime: 2.564ms
  1050. KrpcDataStreamSender (dst_id=2):(Total: 429.259ms, non-child: 420.582ms, % non-child: 97.98%)
  1051. ExecOption: Hash Partitioned Sender Codegen Enabled
  1052. - BytesSent (500.000ms): 0, 0, 0, 985.71 KB
  1053. - NetworkThroughput: (Avg: 63.34 MB/sec ; Min: 892.26 KB/sec ; Max: 95.37 MB/sec ; Number of samples: 3573)
  1054. - EosSent: 3 (3)
  1055. - PeakMemoryUsage: 86.95 KB (89040)
  1056. - RowsSent: 449.98K (449977)
  1057. - RpcFailure: 0 (0)
  1058. - RpcRetry: 0 (0)
  1059. - SerializeBatchTime: 226.692ms
  1060. - TotalBytesSent: 34.68 MB (36362646)
  1061. - UncompressedRowBatchSize: 75.62 MB (79294731)
  1062. AGGREGATION_NODE (id=1):(Total: 1s411ms, non-child: 1s366ms, % non-child: 96.82%)
  1063. Node Lifecycle Event Timeline: 2s161ms
  1064. - Open Started: 323.264ms (323.264ms)
  1065. - Open Finished: 323.650ms (385.736us)
  1066. - First Batch Requested: 323.665ms (15.179us)
  1067. - First Batch Returned: 1s711ms (1s387ms)
  1068. - Last Batch Returned: 2s159ms (448.333ms)
  1069. - Closed: 2s161ms (2.127ms)
  1070. - PeakMemoryUsage: 116.09 MB (121732897)
  1071. - RowsReturned: 449.98K (449977)
  1072. - RowsReturnedRate: 318.78 K/sec
  1073. GroupingAggregator 0:
  1074. ExecOption: Streaming Preaggregation, Codegen Enabled
  1075. - BuildTime: 0.000ns
  1076. - GetResultsTime: 20.512ms
  1077. - HTResizeTime: 43.784ms
  1078. - HashBuckets: 1.05M (1048576)
  1079. - LargestPartitionPercent: 1 (1)
  1080. - PartitionsCreated: 16 (16)
  1081. - PeakMemoryUsage: 114.11 MB (119648384)
  1082. - PeakReservation: 114.00 MB (119537664)
  1083. - PeakUsedReservation: 0
  1084. - ReductionFactorEstimate: 4.11
  1085. - ReductionFactorThresholdToExpand: 2.00
  1086. - RowsPassedThrough: 0 (0)
  1087. - RowsReturned: 449.98K (449977)
  1088. - StreamingTime: 1s102ms
  1089. Buffer pool:
  1090. - AllocTime: 64.404ms
  1091. - CumulativeAllocationBytes: 114.00 MB (119537664)
  1092. - CumulativeAllocations: 57 (57)
  1093. - PeakReservation: 114.00 MB (119537664)
  1094. - PeakUnpinnedBytes: 0
  1095. - PeakUsedReservation: 114.00 MB (119537664)
  1096. - ReadIoBytes: 0
  1097. - ReadIoOps: 0 (0)
  1098. - ReadIoWaitTime: 0.000ns
  1099. - SystemAllocTime: 64.338ms
  1100. - WriteIoBytes: 0
  1101. - WriteIoOps: 0 (0)
  1102. - WriteIoWaitTime: 0.000ns
  1103. HDFS_SCAN_NODE (id=0):(Total: 44.877ms, non-child: 44.877ms, % non-child: 100.00%)
  1104. Hdfs split stats (<volume id>:<# splits>/<split lengths>): 2:27/93.28 MB 7:25/743.47 MB 3:22/1.39 GB 4:25/209.13 MB 6:30/463.04 MB 5:28/281.73 MB 0:26/1.03 GB 1:28/1.09 GB
  1105. ExecOption: PARQUET Codegen Enabled, Codegen enabled: 211 out of 211
  1106. Hdfs Read Thread Concurrency Bucket: 0:0% 1:50% 2:50% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0%
  1107. File Formats: PARQUET/NONE:28 PARQUET/SNAPPY:183
  1108. Node Lifecycle Event Timeline: 1s711ms
  1109. - Open Started: 323.267ms (323.267ms)
  1110. - Open Finished: 323.417ms (149.919us)
  1111. - First Batch Requested: 323.667ms (250.119us)
  1112. - First Batch Returned: 331.119ms (7.452ms)
  1113. - Last Batch Returned: 1s710ms (1s379ms)
  1114. - Closed: 1s711ms (452.403us)
  1115. - BytesRead (500.000ms): 0, 1.12 GB, 1.97 GB
  1116. - FooterProcessingTime: (Avg: 7.490ms ; Min: 123.707us ; Max: 146.576ms ; Number of samples: 211)
  1117. - InitialRangeActualReservation: (Avg: 52.55 MB (55102420) ; Min: 48.00 MB (50331648) ; Max: 240.00 MB (251658240) ; Number of samples: 211)
  1118. - InitialRangeIdealReservation: (Avg: 71.81 KB (73533) ; Min: 8.00 KB (8192) ; Max: 128.00 KB (131072) ; Number of samples: 211)
  1119. - ParquetCompressedBytesReadPerColumn: (Avg: 20.74 MB (21751208) ; Min: 3.68 MB (3862392) ; Max: 53.51 MB (56113954) ; Number of samples: 11)
  1120. - ParquetCompressedPageSize: (Avg: 96.51 KB (98828) ; Min: 8.00 B (8) ; Max: 714.22 KB (731361) ; Number of samples: 2421)
  1121. - ParquetRowGroupActualReservation: (Avg: 66.87 MB (70115153) ; Min: 48.00 MB (50331648) ; Max: 256.00 MB (268435456) ; Number of samples: 188)
  1122. - ParquetRowGroupIdealReservation: (Avg: 14.45 MB (15150058) ; Min: 88.00 KB (90112) ; Max: 256.00 MB (268435456) ; Number of samples: 188)
  1123. - ParquetUncompressedBytesReadPerColumn: (Avg: 29.71 MB (31151832) ; Min: 3.68 MB (3860778) ; Max: 91.61 MB (96063186) ; Number of samples: 11)
  1124. - ParquetUncompressedPageSize: (Avg: 138.22 KB (141540) ; Min: 6.00 B (6) ; Max: 1.00 MB (1048578) ; Number of samples: 2421)
  1125. - AverageHdfsReadThreadConcurrency: 1.50
  1126. - AverageScannerThreadConcurrency: 32.00
  1127. - BytesRead: 2.16 GB (2318073684)
  1128. - BytesReadDataNodeCache: 0
  1129. - BytesReadLocal: 2.16 GB (2318073684)
  1130. - BytesReadRemoteUnexpected: 0
  1131. - BytesReadShortCircuit: 2.16 GB (2318073684)
  1132. - CachedFileHandlesHitCount: 2.02K (2021)
  1133. - CachedFileHandlesMissCount: 0 (0)
  1134. - CollectionItemsRead: 0 (0)
  1135. - DataCacheHitBytes: 0
  1136. - DataCacheHitCount: 0 (0)
  1137. - DataCacheMissBytes: 0
  1138. - DataCacheMissCount: 0 (0)
  1139. - DataCachePartialHitCount: 0 (0)
  1140. - DecompressionTime: 1s023ms
  1141. - MaterializeTupleTime(*): 2s942ms
  1142. - MaxCompressedTextFileLength: 0
  1143. - NumColumns: 11 (11)
  1144. - NumDictFilteredRowGroups: 40 (40)
  1145. - NumDisksAccessed: 9 (9)
  1146. - NumRowGroups: 216 (216)
  1147. - NumScannerThreadMemUnavailable: 0 (0)
  1148. - NumScannerThreadReservationsDenied: 0 (0)
  1149. - NumScannerThreadsStarted: 32 (32)
  1150. - NumScannersWithNoReads: 0 (0)
  1151. - NumStatsFilteredRowGroups: 28 (28)
  1152. - PeakMemoryUsage: 3.15 GB (3382118779)
  1153. - PeakScannerThreadConcurrency: 32 (32)
  1154. - PerReadThreadRawHdfsThroughput: 833.55 MB/sec
  1155. - RemoteScanRanges: 0 (0)
  1156. - RowBatchBytesEnqueued: 1.30 GB (1395284167)
  1157. - RowBatchQueueGetWaitTime: 17.288ms
  1158. - RowBatchQueuePeakMemoryUsage: 43.26 MB (45365253)
  1159. - RowBatchQueuePutWaitTime: 30s374ms
  1160. - RowBatchesEnqueued: 2.35K (2350)
  1161. - RowsRead: 13.16M (13159520)
  1162. - RowsReturned: 1.88M (1881977)
  1163. - RowsReturnedRate: 41.94 M/sec
  1164. - ScanRangesComplete: 211 (211)
  1165. - ScannerIoWaitTime: 4s865ms
  1166. - ScannerThreadWorklessLoops: 0 (0)
  1167. - ScannerThreadsInvoluntaryContextSwitches: 22 (22)
  1168. - ScannerThreadsTotalWallClockTime: 40s435ms
  1169. - ScannerThreadsSysTime: 424.634ms
  1170. - ScannerThreadsUserTime: 3s742ms
  1171. - ScannerThreadsVoluntaryContextSwitches: 4.92K (4919)
  1172. - TotalRawHdfsOpenFileTime(*): 5.163ms
  1173. - TotalRawHdfsReadTime(*): 2s652ms
  1174. - TotalReadThroughput: 1.31 GB/sec
  1175. Buffer pool:
  1176. - AllocTime: 443.500ms
  1177. - CumulativeAllocationBytes: 2.49 GB (2676826112)
  1178. - CumulativeAllocations: 2.02K (2021)
  1179. - PeakReservation: 3.01 GB (3232759808)
  1180. - PeakUnpinnedBytes: 0
  1181. - PeakUsedReservation: 1.67 GB (1789730816)
  1182. - ReadIoBytes: 0
  1183. - ReadIoOps: 0 (0)
  1184. - ReadIoWaitTime: 0.000ns
  1185. - SystemAllocTime: 440.782ms
  1186. - WriteIoBytes: 0
  1187. - WriteIoOps: 0 (0)
  1188. - WriteIoWaitTime: 0.000ns
  1189. CodeGen:(Total: 319.147ms, non-child: 319.147ms, % non-child: 100.00%)
  1190. - CodegenInvoluntaryContextSwitches: 0 (0)
  1191. - CodegenTotalWallClockTime: 319.136ms
  1192. - CodegenSysTime: 0.000ns
  1193. - CodegenUserTime: 319.464ms
  1194. - CodegenVoluntaryContextSwitches: 0 (0)
  1195. - CompileTime: 90.335ms
  1196. - IrGenerationTime: 7.723ms
  1197. - LoadTime: 0.000ns
  1198. - ModuleBitcodeSize: 2.28 MB (2393476)
  1199. - NumFunctions: 113 (113)
  1200. - NumInstructions: 3.79K (3795)
  1201. - OptimizationTime: 198.178ms
  1202. - PeakMemoryUsage: 1.85 MB (1943040)
  1203. - PrepareTime: 21.825ms
  1204. Instance 9242530ef412061f:1d82eaeb00000002 (host=itchil805.inteliquent.com:22000):(Total: 1s796ms, non-child: 1.147ms, % non-child: 0.06%)
  1205. Last report received time: 2020-02-11 13:48:13.261
  1206. Hdfs split stats (<volume id>:<# splits>/<split lengths>): 7:16/1.01 GB 0:20/248.59 MB 1:20/215.93 MB 3:20/137.59 MB 4:24/975.47 MB 2:22/647.65 MB 5:16/1.11 GB 6:20/318.79 MB
  1207. Fragment Instance Lifecycle Event Timeline: 1s797ms
  1208. - Prepare Finished: 3.184ms (3.184ms)
  1209. - Open Finished: 325.788ms (322.603ms)
  1210. - First Batch Produced: 1s377ms (1s052ms)
  1211. - First Batch Sent: 1s377ms (4.952us)
  1212. - ExecInternal Finished: 1s797ms (419.540ms)
  1213. - MemoryUsage (500.000ms): 2.17 GB, 2.65 GB, 108.16 MB
  1214. - ThreadUsage (500.000ms): 33, 33, 1
  1215. - AverageThreadTokens: 22.33
  1216. - BloomFilterBytes: 0
  1217. - ExchangeScanRatio: 0.02
  1218. - PeakMemoryUsage: 2.91 GB (3125246593)
  1219. - PeakReservation: 2.78 GB (2986344448)
  1220. - PeakUsedReservation: 0
  1221. - PerHostPeakMemUsage: 2.95 GB (3163636375)
  1222. - RowsProduced: 416.39K (416389)
  1223. - TotalNetworkReceiveTime: 0.000ns
  1224. - TotalNetworkSendTime: 14.264ms
  1225. - TotalStorageWaitTime: 3s774ms
  1226. - TotalThreadsInvoluntaryContextSwitches: 15 (15)
  1227. - TotalThreadsTotalWallClockTime: 31s558ms
  1228. - TotalThreadsSysTime: 273.457ms
  1229. - TotalThreadsUserTime: 4s850ms
  1230. - TotalThreadsVoluntaryContextSwitches: 4.44K (4444)
  1231. Buffer pool:
  1232. - AllocTime: 0.000ns
  1233. - CumulativeAllocationBytes: 0
  1234. - CumulativeAllocations: 0 (0)
  1235. - PeakReservation: 0
  1236. - PeakUnpinnedBytes: 0
  1237. - PeakUsedReservation: 0
  1238. - ReadIoBytes: 0
  1239. - ReadIoOps: 0 (0)
  1240. - ReadIoWaitTime: 0.000ns
  1241. - ReservationLimit: 0
  1242. - SystemAllocTime: 0.000ns
  1243. - WriteIoBytes: 0
  1244. - WriteIoOps: 0 (0)
  1245. - WriteIoWaitTime: 0.000ns
  1246. Fragment Instance Lifecycle Timings:
  1247. - ExecTime: 1s471ms
  1248. - ExecTreeExecTime: 1s070ms
  1249. - OpenTime: 322.596ms
  1250. - ExecTreeOpenTime: 413.072us
  1251. - PrepareTime: 2.484ms
  1252. - ExecTreePrepareTime: 1.799ms
  1253. KrpcDataStreamSender (dst_id=2):(Total: 400.627ms, non-child: 386.070ms, % non-child: 96.37%)
  1254. ExecOption: Hash Partitioned Sender Codegen Enabled
  1255. - BytesSent (500.000ms): 0, 0, 3.42 MB
  1256. - NetworkThroughput: (Avg: 61.76 MB/sec ; Min: 16.56 MB/sec ; Max: 96.46 MB/sec ; Number of samples: 3306)
  1257. - EosSent: 3 (3)
  1258. - PeakMemoryUsage: 86.95 KB (89040)
  1259. - RowsSent: 416.39K (416389)
  1260. - RpcFailure: 0 (0)
  1261. - RpcRetry: 0 (0)
  1262. - SerializeBatchTime: 202.813ms
  1263. - TotalBytesSent: 32.22 MB (33789950)
  1264. - UncompressedRowBatchSize: 69.98 MB (73379836)
  1265. AGGREGATION_NODE (id=1):(Total: 1s072ms, non-child: 1s010ms, % non-child: 94.19%)
  1266. Node Lifecycle Event Timeline: 1s797ms
  1267. - Open Started: 325.359ms (325.359ms)
  1268. - Open Finished: 325.768ms (409.825us)
  1269. - First Batch Requested: 325.789ms (20.640us)
  1270. - First Batch Returned: 1s377ms (1s052ms)
  1271. - Last Batch Returned: 1s795ms (417.564ms)
  1272. - Closed: 1s797ms (1.843ms)
  1273. - PeakMemoryUsage: 115.68 MB (121297896)
  1274. - RowsReturned: 416.39K (416389)
  1275. - RowsReturnedRate: 388.22 K/sec
  1276. GroupingAggregator 0:
  1277. ExecOption: Streaming Preaggregation, Codegen Enabled
  1278. - BuildTime: 0.000ns
  1279. - GetResultsTime: 17.858ms
  1280. - HTResizeTime: 21.133ms
  1281. - HashBuckets: 1.05M (1048576)
  1282. - LargestPartitionPercent: 1 (1)
  1283. - PartitionsCreated: 16 (16)
  1284. - PeakMemoryUsage: 114.11 MB (119648384)
  1285. - PeakReservation: 114.00 MB (119537664)
  1286. - PeakUsedReservation: 0
  1287. - ReductionFactorEstimate: 4.16
  1288. - ReductionFactorThresholdToExpand: 2.00
  1289. - RowsPassedThrough: 0 (0)
  1290. - RowsReturned: 416.39K (416389)
  1291. - StreamingTime: 945.927ms
  1292. Buffer pool:
  1293. - AllocTime: 17.357ms
  1294. - CumulativeAllocationBytes: 114.00 MB (119537664)
  1295. - CumulativeAllocations: 57 (57)
  1296. - PeakReservation: 114.00 MB (119537664)
  1297. - PeakUnpinnedBytes: 0
  1298. - PeakUsedReservation: 114.00 MB (119537664)
  1299. - ReadIoBytes: 0
  1300. - ReadIoOps: 0 (0)
  1301. - ReadIoWaitTime: 0.000ns
  1302. - SystemAllocTime: 17.272ms
  1303. - WriteIoBytes: 0
  1304. - WriteIoOps: 0 (0)
  1305. - WriteIoWaitTime: 0.000ns
  1306. HDFS_SCAN_NODE (id=0):(Total: 62.367ms, non-child: 62.367ms, % non-child: 100.00%)
  1307. Hdfs split stats (<volume id>:<# splits>/<split lengths>): 7:16/1.01 GB 0:20/248.59 MB 1:20/215.93 MB 3:20/137.59 MB 4:24/975.47 MB 2:22/647.65 MB 5:16/1.11 GB 6:20/318.79 MB
  1308. ExecOption: PARQUET Codegen Enabled, Codegen enabled: 158 out of 158
  1309. Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:50% 3:0% 4:0% 5:50% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0%
  1310. File Formats: PARQUET/NONE:24 PARQUET/SNAPPY:134
  1311. Node Lifecycle Event Timeline: 1s377ms
  1312. - Open Started: 325.361ms (325.361ms)
  1313. - Open Finished: 325.490ms (129.077us)
  1314. - First Batch Requested: 325.791ms (300.656us)
  1315. - First Batch Returned: 334.084ms (8.292ms)
  1316. - Last Batch Returned: 1s377ms (1s043ms)
  1317. - Closed: 1s377ms (749.487us)
  1318. - BytesRead (500.000ms): 421.21 MB, 1.27 GB
  1319. - FooterProcessingTime: (Avg: 7.342ms ; Min: 175.113us ; Max: 78.692ms ; Number of samples: 158)
  1320. - InitialRangeActualReservation: (Avg: 52.42 MB (54963964) ; Min: 48.00 MB (50331648) ; Max: 180.00 MB (188743680) ; Number of samples: 158)
  1321. - InitialRangeIdealReservation: (Avg: 74.18 KB (75957) ; Min: 8.00 KB (8192) ; Max: 128.00 KB (131072) ; Number of samples: 158)
  1322. - ParquetCompressedBytesReadPerColumn: (Avg: 17.23 MB (18071873) ; Min: 3.01 MB (3161022) ; Max: 44.58 MB (46749999) ; Number of samples: 11)
  1323. - ParquetCompressedPageSize: (Avg: 104.15 KB (106647) ; Min: 6.00 B (6) ; Max: 713.52 KB (730645) ; Number of samples: 1864)
  1324. - ParquetRowGroupActualReservation: (Avg: 68.63 MB (71967014) ; Min: 48.00 MB (50331648) ; Max: 256.00 MB (268435456) ; Number of samples: 139)
  1325. - ParquetRowGroupIdealReservation: (Avg: 16.16 MB (16948187) ; Min: 88.00 KB (90112) ; Max: 256.00 MB (268435456) ; Number of samples: 139)
  1326. - ParquetUncompressedBytesReadPerColumn: (Avg: 24.25 MB (25424148) ; Min: 3.01 MB (3159756) ; Max: 74.60 MB (78228456) ; Number of samples: 11)
  1327. - ParquetUncompressedPageSize: (Avg: 146.52 KB (150035) ; Min: 8.00 B (8) ; Max: 1.00 MB (1048578) ; Number of samples: 1864)
  1328. - AverageHdfsReadThreadConcurrency: 3.50
  1329. - AverageScannerThreadConcurrency: 32.00
  1330. - BytesRead: 1.79 GB (1924847666)
  1331. - BytesReadDataNodeCache: 0
  1332. - BytesReadLocal: 1.79 GB (1924847666)
  1333. - BytesReadRemoteUnexpected: 0
  1334. - BytesReadShortCircuit: 1.79 GB (1924847666)
  1335. - CachedFileHandlesHitCount: 1.55K (1546)
  1336. - CachedFileHandlesMissCount: 0 (0)
  1337. - CollectionItemsRead: 0 (0)
  1338. - DataCacheHitBytes: 0
  1339. - DataCacheHitCount: 0 (0)
  1340. - DataCacheMissBytes: 0
  1341. - DataCacheMissCount: 0 (0)
  1342. - DataCachePartialHitCount: 0 (0)
  1343. - DecompressionTime: 784.071ms
  1344. - MaterializeTupleTime(*): 2s190ms
  1345. - MaxCompressedTextFileLength: 0
  1346. - NumColumns: 11 (11)
  1347. - NumDictFilteredRowGroups: 25 (25)
  1348. - NumDisksAccessed: 9 (9)
  1349. - NumRowGroups: 163 (163)
  1350. - NumScannerThreadMemUnavailable: 0 (0)
  1351. - NumScannerThreadReservationsDenied: 0 (0)
  1352. - NumScannerThreadsStarted: 32 (32)
  1353. - NumScannersWithNoReads: 0 (0)
  1354. - NumStatsFilteredRowGroups: 24 (24)
  1355. - PeakMemoryUsage: 2.84 GB (3046943857)
  1356. - PeakScannerThreadConcurrency: 32 (32)
  1357. - PerReadThreadRawHdfsThroughput: 926.00 MB/sec
  1358. - RemoteScanRanges: 0 (0)
  1359. - RowBatchBytesEnqueued: 1.18 GB (1265026461)
  1360. - RowBatchQueueGetWaitTime: 38.580ms
  1361. - RowBatchQueuePeakMemoryUsage: 41.88 MB (43911448)
  1362. - RowBatchQueuePutWaitTime: 22s271ms
  1363. - RowBatchesEnqueued: 2.11K (2107)
  1364. - RowsRead: 10.73M (10733650)
  1365. - RowsReturned: 1.76M (1760973)
  1366. - RowsReturnedRate: 28.24 M/sec
  1367. - ScanRangesComplete: 158 (158)
  1368. - ScannerIoWaitTime: 3s774ms
  1369. - ScannerThreadWorklessLoops: 0 (0)
  1370. - ScannerThreadsInvoluntaryContextSwitches: 13 (13)
  1371. - ScannerThreadsTotalWallClockTime: 29s764ms
  1372. - ScannerThreadsSysTime: 229.882ms
  1373. - ScannerThreadsUserTime: 3s176ms
  1374. - ScannerThreadsVoluntaryContextSwitches: 4.17K (4167)
  1375. - TotalRawHdfsOpenFileTime(*): 4.007ms
  1376. - TotalRawHdfsReadTime(*): 1s982ms
  1377. - TotalReadThroughput: 1.27 GB/sec
  1378. Buffer pool:
  1379. - AllocTime: 158.286ms
  1380. - CumulativeAllocationBytes: 2.07 GB (2224603136)
  1381. - CumulativeAllocations: 1.55K (1546)
  1382. - PeakReservation: 2.71 GB (2908749824)
  1383. - PeakUnpinnedBytes: 0
  1384. - PeakUsedReservation: 1.37 GB (1468489728)
  1385. - ReadIoBytes: 0
  1386. - ReadIoOps: 0 (0)
  1387. - ReadIoWaitTime: 0.000ns
  1388. - SystemAllocTime: 156.239ms
  1389. - WriteIoBytes: 0
  1390. - WriteIoOps: 0 (0)
  1391. - WriteIoWaitTime: 0.000ns
  1392. CodeGen:(Total: 322.074ms, non-child: 322.074ms, % non-child: 100.00%)
  1393. - CodegenInvoluntaryContextSwitches: 1 (1)
  1394. - CodegenTotalWallClockTime: 322.061ms
  1395. - CodegenSysTime: 0.000ns
  1396. - CodegenUserTime: 319.981ms
  1397. - CodegenVoluntaryContextSwitches: 0 (0)
  1398. - CompileTime: 91.899ms
  1399. - IrGenerationTime: 7.521ms
  1400. - LoadTime: 0.000ns
  1401. - ModuleBitcodeSize: 2.28 MB (2393476)
  1402. - NumFunctions: 113 (113)
  1403. - NumInstructions: 3.79K (3795)
  1404. - OptimizationTime: 198.163ms
  1405. - PeakMemoryUsage: 1.85 MB (1943040)
  1406. - PrepareTime: 23.104ms
  1407. Instance 9242530ef412061f:1d82eaeb00000000 (host=itchil806.inteliquent.com:22000):(Total: 1s073ms, non-child: 923.076us, % non-child: 0.09%)
  1408. Last report received time: 2020-02-11 13:48:13.329
  1409. Hdfs split stats (<volume id>:<# splits>/<split lengths>): 6:11/18.95 MB 2:8/216.70 MB 1:15/1.03 GB 7:14/5.87 MB 5:16/1.32 GB 4:11/115.64 MB 0:17/1.03 GB 3:19/901.70 MB
  1410. Fragment Instance Lifecycle Event Timeline: 1s074ms
  1411. - Prepare Finished: 2.723ms (2.723ms)
  1412. - Open Finished: 324.582ms (321.859ms)
  1413. - First Batch Produced: 864.968ms (540.385ms)
  1414. - First Batch Sent: 864.971ms (2.603us)
  1415. - ExecInternal Finished: 1s074ms (209.414ms)
  1416. - MemoryUsage (500.000ms): 2.56 GB, 48.18 MB
  1417. - ThreadUsage (500.000ms): 33, 1
  1418. - AverageThreadTokens: 17.00
  1419. - BloomFilterBytes: 0
  1420. - ExchangeScanRatio: 0.01
  1421. - PeakMemoryUsage: 2.80 GB (3010263248)
  1422. - PeakReservation: 2.71 GB (2907701248)
  1423. - PeakUsedReservation: 0
  1424. - PerHostPeakMemUsage: 2.84 GB (3046039782)
  1425. - RowsProduced: 223.36K (223362)
  1426. - TotalNetworkReceiveTime: 0.000ns
  1427. - TotalNetworkSendTime: 10.709ms
  1428. - TotalStorageWaitTime: 5s294ms
  1429. - TotalThreadsInvoluntaryContextSwitches: 14 (14)
  1430. - TotalThreadsTotalWallClockTime: 14s832ms
  1431. - TotalThreadsSysTime: 268.659ms
  1432. - TotalThreadsUserTime: 2s613ms
  1433. - TotalThreadsVoluntaryContextSwitches: 2.91K (2913)
  1434. Buffer pool:
  1435. - AllocTime: 0.000ns
  1436. - CumulativeAllocationBytes: 0
  1437. - CumulativeAllocations: 0 (0)
  1438. - PeakReservation: 0
  1439. - PeakUnpinnedBytes: 0
  1440. - PeakUsedReservation: 0
  1441. - ReadIoBytes: 0
  1442. - ReadIoOps: 0 (0)
  1443. - ReadIoWaitTime: 0.000ns
  1444. - ReservationLimit: 0
  1445. - SystemAllocTime: 0.000ns
  1446. - WriteIoBytes: 0
  1447. - WriteIoOps: 0 (0)
  1448. - WriteIoWaitTime: 0.000ns
  1449. Fragment Instance Lifecycle Timings:
  1450. - ExecTime: 749.478ms
  1451. - ExecTreeExecTime: 550.374ms
  1452. - OpenTime: 321.859ms
  1453. - ExecTreeOpenTime: 407.909us
  1454. - PrepareTime: 2.012ms
  1455. - ExecTreePrepareTime: 1.335ms
  1456. KrpcDataStreamSender (dst_id=2):(Total: 199.111ms, non-child: 188.243ms, % non-child: 94.54%)
  1457. ExecOption: Hash Partitioned Sender Codegen Enabled
  1458. - BytesSent (500.000ms): 0, 7.41 MB
  1459. - NetworkThroughput: (Avg: 74.13 MB/sec ; Min: 20.22 MB/sec ; Max: 104.29 MB/sec ; Number of samples: 1774)
  1460. - EosSent: 3 (3)
  1461. - PeakMemoryUsage: 86.95 KB (89040)
  1462. - RowsSent: 223.36K (223362)
  1463. - RpcFailure: 0 (0)
  1464. - RpcRetry: 0 (0)
  1465. - SerializeBatchTime: 106.631ms
  1466. - TotalBytesSent: 17.13 MB (17957979)
  1467. - UncompressedRowBatchSize: 37.52 MB (39343153)
  1468. AGGREGATION_NODE (id=1):(Total: 552.060ms, non-child: 503.258ms, % non-child: 91.16%)
  1469. Node Lifecycle Event Timeline: 1s074ms
  1470. - Open Started: 324.166ms (324.166ms)
  1471. - Open Finished: 324.571ms (404.324us)
  1472. - First Batch Requested: 324.584ms (13.446us)
  1473. - First Batch Returned: 864.966ms (540.382ms)
  1474. - Last Batch Returned: 1s072ms (207.907ms)
  1475. - Closed: 1s074ms (1.345ms)
  1476. - PeakMemoryUsage: 75.55 MB (79222912)
  1477. - RowsReturned: 223.36K (223362)
  1478. - RowsReturnedRate: 404.60 K/sec
  1479. GroupingAggregator 0:
  1480. ExecOption: Streaming Preaggregation, Codegen Enabled
  1481. - BuildTime: 0.000ns
  1482. - GetResultsTime: 9.741ms
  1483. - HTResizeTime: 9.755ms
  1484. - HashBuckets: 524.29K (524288)
  1485. - LargestPartitionPercent: 1 (1)
  1486. - PartitionsCreated: 16 (16)
  1487. - PeakMemoryUsage: 74.11 MB (77705344)
  1488. - PeakReservation: 74.00 MB (77594624)
  1489. - PeakUsedReservation: 0
  1490. - ReductionFactorEstimate: 3.44
  1491. - ReductionFactorThresholdToExpand: 2.00
  1492. - RowsPassedThrough: 0 (0)
  1493. - RowsReturned: 223.36K (223362)
  1494. - StreamingTime: 440.944ms
  1495. Buffer pool:
  1496. - AllocTime: 16.830ms
  1497. - CumulativeAllocationBytes: 74.00 MB (77594624)
  1498. - CumulativeAllocations: 37 (37)
  1499. - PeakReservation: 74.00 MB (77594624)
  1500. - PeakUnpinnedBytes: 0
  1501. - PeakUsedReservation: 74.00 MB (77594624)
  1502. - ReadIoBytes: 0
  1503. - ReadIoOps: 0 (0)
  1504. - ReadIoWaitTime: 0.000ns
  1505. - SystemAllocTime: 16.788ms
  1506. - WriteIoBytes: 0
  1507. - WriteIoOps: 0 (0)
  1508. - WriteIoWaitTime: 0.000ns
  1509. HDFS_SCAN_NODE (id=0):(Total: 48.802ms, non-child: 48.802ms, % non-child: 100.00%)
  1510. Hdfs split stats (<volume id>:<# splits>/<split lengths>): 6:11/18.95 MB 2:8/216.70 MB 1:15/1.03 GB 7:14/5.87 MB 5:16/1.32 GB 4:11/115.64 MB 0:17/1.03 GB 3:19/901.70 MB
  1511. ExecOption: PARQUET Codegen Enabled, Codegen enabled: 111 out of 111
  1512. Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:100% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0%
  1513. File Formats: PARQUET/NONE:17 PARQUET/SNAPPY:94
  1514. Node Lifecycle Event Timeline: 864.939ms
  1515. - Open Started: 324.169ms (324.169ms)
  1516. - Open Finished: 324.278ms (108.966us)
  1517. - First Batch Requested: 324.586ms (308.390us)
  1518. - First Batch Returned: 331.344ms (6.757ms)
  1519. - Last Batch Returned: 864.426ms (533.081ms)
  1520. - Closed: 864.939ms (513.700us)
  1521. - BytesRead (500.000ms): 530.47 MB
  1522. - FooterProcessingTime: (Avg: 20.898ms ; Min: 182.237us ; Max: 123.909ms ; Number of samples: 111)
  1523. - InitialRangeActualReservation: (Avg: 48.00 MB (50331648) ; Min: 48.00 MB (50331648) ; Max: 48.00 MB (50331648) ; Number of samples: 111)
  1524. - InitialRangeIdealReservation: (Avg: 66.45 KB (68045) ; Min: 8.00 KB (8192) ; Max: 128.00 KB (131072) ; Number of samples: 111)
  1525. - ParquetCompressedBytesReadPerColumn: (Avg: 9.47 MB (9932384) ; Min: 1.59 MB (1670780) ; Max: 25.01 MB (26221532) ; Number of samples: 11)
  1526. - ParquetCompressedPageSize: (Avg: 100.66 KB (103071) ; Min: 10.00 B (10) ; Max: 699.96 KB (716755) ; Number of samples: 1060)
  1527. - ParquetRowGroupActualReservation: (Avg: 60.76 MB (63706569) ; Min: 48.00 MB (50331648) ; Max: 248.00 MB (260046848) ; Number of samples: 94)
  1528. - ParquetRowGroupIdealReservation: (Avg: 20.58 MB (21581388) ; Min: 88.00 KB (90112) ; Max: 248.00 MB (260046848) ; Number of samples: 94)
  1529. - ParquetUncompressedBytesReadPerColumn: (Avg: 13.27 MB (13916439) ; Min: 1.59 MB (1670099) ; Max: 42.81 MB (44890320) ; Number of samples: 11)
  1530. - ParquetUncompressedPageSize: (Avg: 141.03 KB (144415) ; Min: 8.00 B (8) ; Max: 1.00 MB (1048578) ; Number of samples: 1060)
  1531. - AverageHdfsReadThreadConcurrency: 3.00
  1532. - AverageScannerThreadConcurrency: 32.00
  1533. - BytesRead: 1.61 GB (1724387435)
  1534. - BytesReadDataNodeCache: 0
  1535. - BytesReadLocal: 1.61 GB (1724387435)
  1536. - BytesReadRemoteUnexpected: 0
  1537. - BytesReadShortCircuit: 1.61 GB (1724387435)
  1538. - CachedFileHandlesHitCount: 986 (986)
  1539. - CachedFileHandlesMissCount: 0 (0)
  1540. - CollectionItemsRead: 0 (0)
  1541. - DataCacheHitBytes: 0
  1542. - DataCacheHitCount: 0 (0)
  1543. - DataCacheMissBytes: 0
  1544. - DataCacheMissCount: 0 (0)
  1545. - DataCachePartialHitCount: 0 (0)
  1546. - DecompressionTime: 511.988ms
  1547. - MaterializeTupleTime(*): 1s191ms
  1548. - MaxCompressedTextFileLength: 0
  1549. - NumColumns: 11 (11)
  1550. - NumDictFilteredRowGroups: 27 (27)
  1551. - NumDisksAccessed: 8 (8)
  1552. - NumRowGroups: 111 (111)
  1553. - NumScannerThreadMemUnavailable: 0 (0)
  1554. - NumScannerThreadReservationsDenied: 0 (0)
  1555. - NumScannerThreadsStarted: 32 (32)
  1556. - NumScannersWithNoReads: 0 (0)
  1557. - NumStatsFilteredRowGroups: 17 (17)
  1558. - PeakMemoryUsage: 2.77 GB (2969366732)
  1559. - PeakScannerThreadConcurrency: 32 (32)
  1560. - PerReadThreadRawHdfsThroughput: 1022.54 MB/sec
  1561. - RemoteScanRanges: 0 (0)
  1562. - RowBatchBytesEnqueued: 635.29 MB (666149633)
  1563. - RowBatchQueueGetWaitTime: 34.028ms
  1564. - RowBatchQueuePeakMemoryUsage: 34.74 MB (36423906)
  1565. - RowBatchQueuePutWaitTime: 6s053ms
  1566. - RowBatchesEnqueued: 1.08K (1080)
  1567. - RowsRead: 5.50M (5501892)
  1568. - RowsReturned: 828.81K (828813)
  1569. - RowsReturnedRate: 16.98 M/sec
  1570. - ScanRangesComplete: 111 (111)
  1571. - ScannerIoWaitTime: 5s294ms
  1572. - ScannerThreadWorklessLoops: 0 (0)
  1573. - ScannerThreadsInvoluntaryContextSwitches: 11 (11)
  1574. - ScannerThreadsTotalWallClockTime: 13s761ms
  1575. - ScannerThreadsSysTime: 240.555ms
  1576. - ScannerThreadsUserTime: 1s645ms
  1577. - ScannerThreadsVoluntaryContextSwitches: 2.65K (2646)
  1578. - TotalRawHdfsOpenFileTime(*): 2.288ms
  1579. - TotalRawHdfsReadTime(*): 1s608ms
  1580. - TotalReadThroughput: 1.04 GB/sec
  1581. Buffer pool:
  1582. - AllocTime: 117.240ms
  1583. - CumulativeAllocationBytes: 1.80 GB (1929355264)
  1584. - CumulativeAllocations: 988 (988)
  1585. - PeakReservation: 2.67 GB (2867855360)
  1586. - PeakUnpinnedBytes: 0
  1587. - PeakUsedReservation: 1.64 GB (1758404608)
  1588. - ReadIoBytes: 0
  1589. - ReadIoOps: 0 (0)
  1590. - ReadIoWaitTime: 0.000ns
  1591. - SystemAllocTime: 115.780ms
  1592. - WriteIoBytes: 0
  1593. - WriteIoOps: 0 (0)
  1594. - WriteIoWaitTime: 0.000ns
  1595. CodeGen:(Total: 321.333ms, non-child: 321.333ms, % non-child: 100.00%)
  1596. - CodegenInvoluntaryContextSwitches: 2 (2)
  1597. - CodegenTotalWallClockTime: 321.319ms
  1598. - CodegenSysTime: 3.990ms
  1599. - CodegenUserTime: 319.223ms
  1600. - CodegenVoluntaryContextSwitches: 0 (0)
  1601. - CompileTime: 91.337ms
  1602. - IrGenerationTime: 7.272ms
  1603. - LoadTime: 0.000ns
  1604. - ModuleBitcodeSize: 2.28 MB (2393476)
  1605. - NumFunctions: 113 (113)
  1606. - NumInstructions: 3.79K (3795)
  1607. - OptimizationTime: 198.720ms
  1608. - PeakMemoryUsage: 1.85 MB (1943040)
  1609. - PrepareTime: 22.849ms
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!