Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- Query (id=434987288e39baec:45101a3900000000):
- Summary:
- Session ID: 944fff761bf09762:bdfd68a4f8283fbf
- Session Type: HIVESERVER2
- HiveServer2 Protocol Version: V6
- Start Time: 2020-02-11 13:41:51.790478000
- End Time: 2020-02-11 13:42:04.630228000
- Query Type: QUERY
- Query State: FINISHED
- Query Status: OK
- Impala Version: impalad version 3.2.0-cdh6.3.0 RELEASE (build 495397101e5807c701df71ea288f4815d69c2c8a)
- User: nmali
- Connected User: nmali
- Delegated User:
- Network Address: ::ffff:172.17.250.66:36614
- Default Db: alerts
- Sql Statement: SELECT count(*) FROM (
- SELECT
- CAST(orig_cust_id as INT) as orig_cust_id,
- CAST(dest_npa as INT) as dest_npa,
- CAST(dest_nxx as INT) as dest_nxx,
- dest_state,
- dest_ocn,
- orig_trunk_grp_nm as orig_trunk_group_name,
- orig_product,
- jurisdiction_state as jurisdiction,
- sum(cost) as cost,
- sum(revenue) as revenue,
- sum(conv_seconds) as duration,
- call_date_time_hours,
- call_date_time_day,
- call_date_time_month,
- call_date_time_year
- FROM call_detail_records.raw
- WHERE
- call_date_time_day=day(DAYS_SUB(now(), 1)) AND
- call_date_time_month=month(DAYS_SUB(now(), 1)) AND
- call_date_time_year=year(DAYS_SUB(now(), 1)) AND
- orig_cust_id='25'
- GROUP BY
- orig_cust_id,
- dest_npa,
- dest_nxx,
- dest_state,
- dest_ocn,
- orig_trunk_grp_nm,
- orig_product,
- jurisdiction_state,
- call_date_time_hours,
- call_date_time_day,
- call_date_time_month,
- call_date_time_year
- ) a
- Coordinator: itchil804.inteliquent.com:22000
- Query Options (set by configuration): QUERY_TIMEOUT_S=300,IDLE_SESSION_TIMEOUT=900
- Query Options (set by configuration and planner): QUERY_TIMEOUT_S=300,MT_DOP=0,IDLE_SESSION_TIMEOUT=900
- Plan:
- ----------------
- Max Per-Host Resource Reservation: Memory=100.00MB Threads=4
- Per-Host Resource Estimates: Memory=596MB
- WARNING: The following tables are missing relevant table and/or column statistics.
- call_detail_records.raw
- Analyzed query: SELECT count(*) FROM (SELECT CAST(orig_cust_id AS INT)
- orig_cust_id, CAST(dest_npa AS INT) dest_npa, CAST(dest_nxx AS INT) dest_nxx,
- dest_state, dest_ocn, orig_trunk_grp_nm orig_trunk_group_name, orig_product,
- jurisdiction_state jurisdiction, sum(cost) cost, sum(revenue) revenue,
- sum(conv_seconds) duration, call_date_time_hours, call_date_time_day,
- call_date_time_month, call_date_time_year FROM call_detail_records.raw WHERE
- call_date_time_day = CAST(10 AS INT) AND call_date_time_month = CAST(2 AS INT)
- AND call_date_time_year = CAST(2020 AS INT) AND orig_cust_id = '25' GROUP BY
- CAST(orig_cust_id AS INT), CAST(dest_npa AS INT), CAST(dest_nxx AS INT),
- dest_state, dest_ocn, orig_trunk_grp_nm, orig_product, jurisdiction_state,
- call_date_time_hours, call_date_time_day, call_date_time_month,
- call_date_time_year) a
- F02:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
- | Per-Host Resources: mem-estimate=10.02MB mem-reservation=0B thread-reservation=1
- PLAN-ROOT SINK
- | mem-estimate=0B mem-reservation=0B thread-reservation=0
- |
- 06:AGGREGATE [FINALIZE]
- | output: count:merge(*)
- | mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
- | tuple-ids=3 row-size=8B cardinality=1
- | in pipelines: 06(GETNEXT), 02(OPEN)
- |
- 05:EXCHANGE [UNPARTITIONED]
- | mem-estimate=16.00KB mem-reservation=0B thread-reservation=0
- | tuple-ids=3 row-size=8B cardinality=1
- | in pipelines: 02(GETNEXT)
- |
- F01:PLAN FRAGMENT [HASH(CAST(orig_cust_id AS INT),CAST(dest_npa AS INT),CAST(dest_nxx AS INT),dest_state,dest_ocn,orig_trunk_grp_nm,orig_product,jurisdiction_state,call_date_time_hours,call_date_time_day,call_date_time_month,call_date_time_year)] hosts=3 instances=3
- Per-Host Resources: mem-estimate=138.00MB mem-reservation=34.00MB thread-reservation=1
- 02:AGGREGATE
- | output: count(*)
- | mem-estimate=10.00MB mem-reservation=0B spill-buffer=2.00MB thread-reservation=0
- | tuple-ids=3 row-size=8B cardinality=1
- | in pipelines: 02(GETNEXT), 04(OPEN)
- |
- 04:AGGREGATE [FINALIZE]
- | group by: CAST(orig_cust_id AS INT), CAST(dest_npa AS INT), CAST(dest_nxx AS INT), dest_state, dest_ocn, orig_trunk_grp_nm, orig_product, jurisdiction_state, call_date_time_hours, call_date_time_day, call_date_time_month, call_date_time_year
- | mem-estimate=128.00MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
- | tuple-ids=1 row-size=104B cardinality=unavailable
- | in pipelines: 04(GETNEXT), 00(OPEN)
- |
- 03:EXCHANGE [HASH(CAST(orig_cust_id AS INT),CAST(dest_npa AS INT),CAST(dest_nxx AS INT),dest_state,dest_ocn,orig_trunk_grp_nm,orig_product,jurisdiction_state,call_date_time_hours,call_date_time_day,call_date_time_month,call_date_time_year)]
- | mem-estimate=323.97KB mem-reservation=0B thread-reservation=0
- | tuple-ids=1 row-size=104B cardinality=unavailable
- | in pipelines: 00(GETNEXT)
- |
- F00:PLAN FRAGMENT [RANDOM] hosts=3 instances=3
- Per-Host Resources: mem-estimate=448.00MB mem-reservation=66.00MB thread-reservation=2
- 01:AGGREGATE [STREAMING]
- | group by: CAST(orig_cust_id AS INT), CAST(dest_npa AS INT), CAST(dest_nxx AS INT), dest_state, dest_ocn, orig_trunk_grp_nm, orig_product, jurisdiction_state, call_date_time_hours, call_date_time_day, call_date_time_month, call_date_time_year
- | mem-estimate=128.00MB mem-reservation=34.00MB spill-buffer=2.00MB thread-reservation=0
- | tuple-ids=1 row-size=104B cardinality=unavailable
- | in pipelines: 00(GETNEXT)
- |
- 00:SCAN HDFS [call_detail_records.raw, RANDOM]
- partition predicates: call_date_time_day = CAST(10 AS INT), call_date_time_month = CAST(2 AS INT), call_date_time_year = CAST(2020 AS INT)
- partitions=24/631 files=478 size=14.48GB
- predicates: orig_cust_id = '25'
- stored statistics:
- table: rows=unavailable size=unavailable
- partitions: 0/24 rows=unavailable
- columns missing stats: orig_cust_id, dest_npa, dest_nxx, dest_state, dest_ocn, orig_trunk_grp_nm, orig_product, jurisdiction_state, cost, revenue, conv_seconds
- extrapolated-rows=disabled max-scan-range-rows=unavailable
- parquet statistics predicates: orig_cust_id = '25'
- parquet dictionary predicates: orig_cust_id = '25'
- mem-estimate=320.00MB mem-reservation=32.00MB thread-reservation=1
- tuple-ids=0 row-size=104B cardinality=unavailable
- in pipelines: 00(GETNEXT)
- ----------------
- Estimated Per-Host Mem: 624967680
- Tables Missing Stats: call_detail_records.raw
- Per Host Min Memory Reservation: itchil804.inteliquent.com:22000(100.00 MB) itchil805.inteliquent.com:22000(100.00 MB) itchil806.inteliquent.com:22000(100.00 MB)
- Request Pool: root.nmali
- Admission result: Admitted immediately
- Cluster Memory Admitted: 1.75 GB
- ExecSummary:
- Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail
- -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
- F02:ROOT 1 38.528us 38.528us 0 0
- 06:AGGREGATE 1 419.131us 419.131us 1 1 16.00 KB 10.00 MB FINALIZE
- 05:EXCHANGE 1 53.043us 53.043us 3 1 32.00 KB 16.00 KB UNPARTITIONED
- F01:EXCHANGE SENDER 3 140.015us 193.994us 16.00 KB 0
- 02:AGGREGATE 3 1.143ms 1.160ms 3 1 16.00 KB 10.00 MB
- 04:AGGREGATE 3 898.648ms 954.214ms 3.46M -1 192.11 MB 128.00 MB FINALIZE
- 03:EXCHANGE 3 69.093ms 71.785ms 5.11M -1 1.09 MB 323.97 KB HASH(CAST(orig_cust_id AS INT),CAST(dest_npa AS INT),CAST(dest_nxx AS INT),dest_state,dest_ocn,orig_trunk_grp_nm,orig_product,jurisdiction_state,call_date_time_hours,call_date_time_day,call_date_time_month,call_date_time_year)
- F00:EXCHANGE SENDER 3 1s226ms 1s333ms 88.27 KB 0
- 01:AGGREGATE 3 6s900ms 7s759ms 5.11M -1 321.54 MB 128.00 MB STREAMING
- 00:SCAN HDFS 3 221.143ms 297.334ms 54.04M -1 2.02 GB 320.00 MB call_detail_records.raw
- Errors:
- Query Compilation: 23.051ms
- - Metadata of all 1 tables cached: 2.680ms (2.680ms)
- - Analysis finished: 10.512ms (7.831ms)
- - Value transfer graph computed: 10.668ms (156.378us)
- - Single node plan created: 19.915ms (9.247ms)
- - Runtime filters computed: 20.189ms (273.752us)
- - Distributed plan created: 20.260ms (70.978us)
- - Lineage info computed: 20.393ms (132.543us)
- - Planning finished: 23.051ms (2.658ms)
- Query Timeline: 2m24s
- - Query submitted: 118.576us (118.576us)
- - Planning finished: 26.581ms (26.462ms)
- - Submit for admission: 29.364ms (2.782ms)
- - Completed admission: 29.676ms (311.984us)
- - Ready to start on 3 backends: 31.032ms (1.355ms)
- - All 3 execution backends (7 fragment instances) started: 38.421ms (7.389ms)
- - Rows available: 9s787ms (9s749ms)
- - First row fetched: 11s498ms (1s710ms)
- - Last row fetched: 12s839ms (1s341ms)
- - Released admission control resources: 12s839ms (141.351us)
- - AdmissionControlTimeSinceLastUpdate: 83.000ms
- - ComputeScanRangeAssignmentTimer: 774.145us
- Frontend:
- ImpalaServer:
- - ClientFetchWaitTimer: 3s051ms
- - RowMaterializationTimer: 3.938ms
- Execution Profile 434987288e39baec:45101a3900000000:(Total: 9s760ms, non-child: 0.000ns, % non-child: 0.00%)
- Number of filters: 0
- Filter routing table:
- ID Src. Node Tgt. Node(s) Target type Partition filter Pending (Expected) First arrived Completed Enabled
- -------------------------------------------------------------------------------------------------------------------
- Backend startup latencies: Count: 3, min / max: 4ms / 5ms, 25th %-ile: 4ms, 50th %-ile: 5ms, 75th %-ile: 5ms, 90th %-ile: 5ms, 95th %-ile: 5ms, 99.9th %-ile: 5ms
- Slowest backend to start up: itchil805.inteliquent.com:22000
- Per Node Peak Memory Usage: itchil804.inteliquent.com:22000(2.17 GB) itchil805.inteliquent.com:22000(2.00 GB) itchil806.inteliquent.com:22000(1.92 GB)
- Per Node Bytes Read: itchil804.inteliquent.com:22000(1.48 GB) itchil805.inteliquent.com:22000(1.30 GB) itchil806.inteliquent.com:22000(1.30 GB)
- Per Node User Time: itchil804.inteliquent.com:22000(28s887ms) itchil805.inteliquent.com:22000(23s195ms) itchil806.inteliquent.com:22000(23s017ms)
- Per Node System Time: itchil804.inteliquent.com:22000(1s206ms) itchil805.inteliquent.com:22000(931.985ms) itchil806.inteliquent.com:22000(825.762ms)
- - ExchangeScanRatio: 0.07
- - FiltersReceived: 0 (0)
- - FinalizationTimer: 0.000ns
- - InnerNodeSelectivityRatio: 0.00
- - NumBackends: 3 (3)
- - NumFragmentInstances: 7 (7)
- - NumFragments: 3 (3)
- - TotalBytesRead: 4.08 GB (4385496404)
- - TotalBytesSent: 294.32 MB (308617202)
- - TotalCpuTime: 1m18s
- - TotalInnerBytesSent: 36.00 B (36)
- - TotalScanBytesSent: 294.32 MB (308617166)
- Per Node Profiles:
- itchil804.inteliquent.com:22000:
- - ScratchBytesRead: 0
- - ScratchBytesWritten: 0
- - ScratchFileUsedBytes: 0
- - ScratchReads: 0 (0)
- - ScratchWrites: 0 (0)
- - TotalEncryptionTime: 0.000ns
- - TotalReadBlockTime: 0.000ns
- itchil805.inteliquent.com:22000:
- - ScratchBytesRead: 0
- - ScratchBytesWritten: 0
- - ScratchFileUsedBytes: 0
- - ScratchReads: 0 (0)
- - ScratchWrites: 0 (0)
- - TotalEncryptionTime: 0.000ns
- - TotalReadBlockTime: 0.000ns
- itchil806.inteliquent.com:22000:
- - ScratchBytesRead: 0
- - ScratchBytesWritten: 0
- - ScratchFileUsedBytes: 0
- - ScratchReads: 0 (0)
- - ScratchWrites: 0 (0)
- - TotalEncryptionTime: 0.000ns
- - TotalReadBlockTime: 0.000ns
- Averaged Fragment F02:(Total: 11s461ms, non-child: 656.174us, % non-child: 0.01%)
- split sizes: min: 0, max: 0, avg: 0, stddev: 0
- completion times: min:11s463ms max:11s463ms mean: 11s463ms stddev:0.000ns
- execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec
- num instances: 1
- - AverageThreadTokens: 0.00
- - BloomFilterBytes: 0
- - ExchangeScanRatio: 0.00
- - PeakMemoryUsage: 56.00 KB (57344)
- - PeakReservation: 0
- - PeakUsedReservation: 0
- - PerHostPeakMemUsage: 2.17 GB (2325567844)
- - RowsProduced: 1 (1)
- - TotalNetworkReceiveTime: 9s722ms
- - TotalNetworkSendTime: 0.000ns
- - TotalStorageWaitTime: 0.000ns
- - TotalThreadsInvoluntaryContextSwitches: 0 (0)
- - TotalThreadsTotalWallClockTime: 11s460ms
- - TotalThreadsSysTime: 0.000ns
- - TotalThreadsUserTime: 28.181ms
- - TotalThreadsVoluntaryContextSwitches: 5 (5)
- Buffer pool:
- - AllocTime: 0.000ns
- - CumulativeAllocationBytes: 0
- - CumulativeAllocations: 0 (0)
- - PeakReservation: 0
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 0
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - ReservationLimit: 0
- - SystemAllocTime: 0.000ns
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- Fragment Instance Lifecycle Timings:
- - ExecTime: 1s710ms
- - ExecTreeExecTime: 13.981us
- - OpenTime: 9s749ms
- - ExecTreeOpenTime: 9s722ms
- - PrepareTime: 614.622us
- - ExecTreePrepareTime: 246.124us
- PLAN_ROOT_SINK:(Total: 1s710ms, non-child: 1s710ms, % non-child: 100.00%)
- - PeakMemoryUsage: 0
- AGGREGATION_NODE (id=6):(Total: 9s722ms, non-child: 419.131us, % non-child: 0.00%)
- - PeakMemoryUsage: 16.00 KB (16384)
- - RowsReturned: 1 (1)
- - RowsReturnedRate: 0
- NonGroupingAggregator 0:
- - BuildTime: 6.488us
- - PeakMemoryUsage: 8.00 KB (8192)
- - RowsReturned: 1 (1)
- EXCHANGE_NODE (id=5):(Total: 9s722ms, non-child: 9s722ms, % non-child: 100.00%)
- - ConvertRowBatchTime: 5.451us
- - PeakMemoryUsage: 32.00 KB (32768)
- - RowsReturned: 3 (3)
- - RowsReturnedRate: 0
- Buffer pool:
- - AllocTime: 10.847us
- - CumulativeAllocationBytes: 48.00 KB (49152)
- - CumulativeAllocations: 6 (6)
- - PeakReservation: 32.00 KB (32768)
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 32.00 KB (32768)
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - SystemAllocTime: 0.000ns
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- Dequeue:
- - FirstBatchWaitTime: 9s719ms
- - TotalBytesDequeued: 48.00 B (48)
- - TotalGetBatchTime: 9s722ms
- - DataWaitTime: 9s722ms
- Enqueue:
- - DeserializeRowBatchTime: 40.204us
- - TotalBatchesEnqueued: 3 (3)
- - TotalBatchesReceived: 3 (3)
- - TotalBytesReceived: 36.00 B (36)
- - TotalEarlySenders: 0 (0)
- - TotalEosReceived: 3 (3)
- - TotalHasDeferredRPCsTime: 0.000ns
- - TotalRPCsDeferred: 0 (0)
- CodeGen:(Total: 27.063ms, non-child: 27.063ms, % non-child: 100.00%)
- - CodegenInvoluntaryContextSwitches: 0 (0)
- - CodegenTotalWallClockTime: 27.040ms
- - CodegenSysTime: 0.000ns
- - CodegenUserTime: 23.853ms
- - CodegenVoluntaryContextSwitches: 0 (0)
- - CompileTime: 0.000ns
- - IrGenerationTime: 21.336us
- - LoadTime: 0.000ns
- - ModuleBitcodeSize: 2.28 MB (2393476)
- - NumFunctions: 0 (0)
- - NumInstructions: 0 (0)
- - OptimizationTime: 0.000ns
- - PeakMemoryUsage: 0
- - PrepareTime: 27.031ms
- Coordinator Fragment F02:
- Instance 434987288e39baec:45101a3900000000 (host=itchil804.inteliquent.com:22000):(Total: 11s461ms, non-child: 656.174us, % non-child: 0.01%)
- Last report received time: 2020-02-11 13:42:03.290
- Fragment Instance Lifecycle Event Timeline: 11s461ms
- - Prepare Finished: 1.171ms (1.171ms)
- - Open Finished: 9s751ms (9s749ms)
- - First Batch Produced: 9s751ms (16.303us)
- - First Batch Sent: 11s461ms (1s710ms)
- - ExecInternal Finished: 11s461ms (145.994us)
- - MemoryUsage (500.000ms): 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 12.00 KB, 16.00 KB, 16.00 KB, 16.00 KB, 16.00 KB
- - AverageThreadTokens: 0.00
- - BloomFilterBytes: 0
- - ExchangeScanRatio: 0.00
- - PeakMemoryUsage: 56.00 KB (57344)
- - PeakReservation: 0
- - PeakUsedReservation: 0
- - PerHostPeakMemUsage: 2.17 GB (2325567844)
- - RowsProduced: 1 (1)
- - TotalNetworkReceiveTime: 9s722ms
- - TotalNetworkSendTime: 0.000ns
- - TotalStorageWaitTime: 0.000ns
- - TotalThreadsInvoluntaryContextSwitches: 0 (0)
- - TotalThreadsTotalWallClockTime: 11s460ms
- - TotalThreadsSysTime: 0.000ns
- - TotalThreadsUserTime: 28.181ms
- - TotalThreadsVoluntaryContextSwitches: 5 (5)
- Buffer pool:
- - AllocTime: 0.000ns
- - CumulativeAllocationBytes: 0
- - CumulativeAllocations: 0 (0)
- - PeakReservation: 0
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 0
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - ReservationLimit: 0
- - SystemAllocTime: 0.000ns
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- Fragment Instance Lifecycle Timings:
- - ExecTime: 1s710ms
- - ExecTreeExecTime: 13.981us
- - OpenTime: 9s749ms
- - ExecTreeOpenTime: 9s722ms
- - PrepareTime: 614.622us
- - ExecTreePrepareTime: 246.124us
- PLAN_ROOT_SINK:(Total: 1s710ms, non-child: 38.528us, % non-child: 0.00%)
- - PeakMemoryUsage: 0
- AGGREGATION_NODE (id=6):(Total: 9s722ms, non-child: 419.131us, % non-child: 0.00%)
- Node Lifecycle Event Timeline: 11s461ms
- - Open Started: 28.405ms (28.405ms)
- - Open Finished: 9s750ms (9s722ms)
- - First Batch Requested: 9s751ms (48.797us)
- - First Batch Returned: 9s751ms (9.059us)
- - Last Batch Returned: 9s751ms (882.000ns)
- - Closed: 11s461ms (1s710ms)
- - PeakMemoryUsage: 16.00 KB (16384)
- - RowsReturned: 1 (1)
- - RowsReturnedRate: 0
- NonGroupingAggregator 0:
- - BuildTime: 6.488us
- - PeakMemoryUsage: 8.00 KB (8192)
- - RowsReturned: 1 (1)
- EXCHANGE_NODE (id=5):(Total: 9s722ms, non-child: 53.043us, % non-child: 0.00%)
- Node Lifecycle Event Timeline: 9s750ms
- - Open Started: 28.406ms (28.406ms)
- - Open Finished: 9s747ms (9s719ms)
- - First Batch Requested: 9s747ms (23.876us)
- - First Batch Returned: 9s750ms (3.156ms)
- - Last Batch Returned: 9s750ms (1.147us)
- - Closed: 9s750ms (137.191us)
- - ConvertRowBatchTime: 5.451us
- - PeakMemoryUsage: 32.00 KB (32768)
- - RowsReturned: 3 (3)
- - RowsReturnedRate: 0
- Buffer pool:
- - AllocTime: 10.847us
- - CumulativeAllocationBytes: 48.00 KB (49152)
- - CumulativeAllocations: 6 (6)
- - PeakReservation: 32.00 KB (32768)
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 32.00 KB (32768)
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - SystemAllocTime: 0.000ns
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- Dequeue:
- - BytesDequeued (500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- - FirstBatchWaitTime: 9s719ms
- - TotalBytesDequeued: 48.00 B (48)
- - TotalGetBatchTime: 9s722ms
- - DataWaitTime: 9s722ms
- Enqueue:
- - BytesReceived (500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- - DeferredQueueSize (500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- - DispatchTime: (Avg: 105.892us ; Min: 91.295us ; Max: 116.186us ; Number of samples: 3)
- - DeserializeRowBatchTime: 40.204us
- - TotalBatchesEnqueued: 3 (3)
- - TotalBatchesReceived: 3 (3)
- - TotalBytesReceived: 36.00 B (36)
- - TotalEarlySenders: 0 (0)
- - TotalEosReceived: 3 (3)
- - TotalHasDeferredRPCsTime: 0.000ns
- - TotalRPCsDeferred: 0 (0)
- CodeGen:(Total: 27.063ms, non-child: 27.063ms, % non-child: 100.00%)
- - CodegenInvoluntaryContextSwitches: 0 (0)
- - CodegenTotalWallClockTime: 27.040ms
- - CodegenSysTime: 0.000ns
- - CodegenUserTime: 23.853ms
- - CodegenVoluntaryContextSwitches: 0 (0)
- - CompileTime: 0.000ns
- - IrGenerationTime: 21.336us
- - LoadTime: 0.000ns
- - ModuleBitcodeSize: 2.28 MB (2393476)
- - NumFunctions: 0 (0)
- - NumInstructions: 0 (0)
- - OptimizationTime: 0.000ns
- - PeakMemoryUsage: 0
- - PrepareTime: 27.031ms
- Averaged Fragment F01:(Total: 9s747ms, non-child: 840.589us, % non-child: 0.01%)
- split sizes: min: 0, max: 0, avg: 0, stddev: 0
- completion times: min:9s750ms max:10s012ms mean: 9s838ms stddev:123.148ms
- execution rates: min:0.00 /sec max:0.00 /sec mean:0.00 /sec stddev:0.00 /sec
- num instances: 3
- - AverageThreadTokens: 1.00
- - BloomFilterBytes: 0
- - ExchangeScanRatio: 0.00
- - PeakMemoryUsage: 193.11 MB (202490902)
- - PeakReservation: 192.00 MB (201326592)
- - PeakUsedReservation: 0
- - PerHostPeakMemUsage: 2.03 GB (2176278826)
- - RowsProduced: 1 (1)
- - TotalNetworkReceiveTime: 8s646ms
- - TotalNetworkSendTime: 801.044us
- - TotalStorageWaitTime: 0.000ns
- - TotalThreadsInvoluntaryContextSwitches: 3 (3)
- - TotalThreadsTotalWallClockTime: 9s745ms
- - TotalThreadsSysTime: 87.213ms
- - TotalThreadsUserTime: 1s103ms
- - TotalThreadsVoluntaryContextSwitches: 7.00K (6995)
- Buffer pool:
- - AllocTime: 0.000ns
- - CumulativeAllocationBytes: 0
- - CumulativeAllocations: 0 (0)
- - PeakReservation: 0
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 0
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - ReservationLimit: 0
- - SystemAllocTime: 0.000ns
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- Fragment Instance Lifecycle Timings:
- - ExecTime: 868.414us
- - ExecTreeExecTime: 3.766us
- - OpenTime: 9s744ms
- - ExecTreeOpenTime: 9s588ms
- - PrepareTime: 1.294ms
- - ExecTreePrepareTime: 682.041us
- KrpcDataStreamSender (dst_id=5):(Total: 941.745us, non-child: 941.745us, % non-child: 100.00%)
- - EosSent: 1 (1)
- - PeakMemoryUsage: 16.00 KB (16384)
- - RowsSent: 1 (1)
- - RpcFailure: 0 (0)
- - RpcRetry: 0 (0)
- - SerializeBatchTime: 7.304us
- - TotalBytesSent: 12.00 B (12)
- - UncompressedRowBatchSize: 16.00 B (16)
- AGGREGATION_NODE (id=2):(Total: 9s589ms, non-child: 1.143ms, % non-child: 0.01%)
- - PeakMemoryUsage: 16.00 KB (16384)
- - RowsReturned: 1 (1)
- - RowsReturnedRate: 0
- NonGroupingAggregator 0:
- - BuildTime: 644.082us
- - PeakMemoryUsage: 8.00 KB (8192)
- - RowsReturned: 1 (1)
- AGGREGATION_NODE (id=4):(Total: 9s587ms, non-child: 898.648ms, % non-child: 9.37%)
- - PeakMemoryUsage: 192.11 MB (201441408)
- - RowsReturned: 1.15M (1153006)
- - RowsReturnedRate: 120.25 K/sec
- GroupingAggregator 0:
- - BuildTime: 857.723ms
- - GetResultsTime: 33.524ms
- - HTResizeTime: 58.997ms
- - HashBuckets: 2.10M (2097152)
- - LargestPartitionPercent: 4 (4)
- - MaxPartitionLevel: 0 (0)
- - NumRepartitions: 0 (0)
- - PartitionsCreated: 16 (16)
- - PeakMemoryUsage: 192.11 MB (201437312)
- - PeakReservation: 192.00 MB (201326592)
- - PeakUsedReservation: 0
- - RowsRepartitioned: 0 (0)
- - RowsReturned: 1.15M (1153006)
- - SpilledPartitions: 0 (0)
- Buffer pool:
- - AllocTime: 21.993ms
- - CumulativeAllocationBytes: 208.00 MB (218103808)
- - CumulativeAllocations: 104 (104)
- - PeakReservation: 192.00 MB (201326592)
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 192.00 MB (201326592)
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - SystemAllocTime: 21.840ms
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- EXCHANGE_NODE (id=3):(Total: 8s689ms, non-child: 8s689ms, % non-child: 100.00%)
- - ConvertRowBatchTime: 34.148ms
- - PeakMemoryUsage: 992.00 KB (1015808)
- - RowsReturned: 1.70M (1702077)
- - RowsReturnedRate: 195.89 K/sec
- Buffer pool:
- - AllocTime: 17.911ms
- - CumulativeAllocationBytes: 365.33 MB (383074304)
- - CumulativeAllocations: 18.71K (18706)
- - PeakReservation: 992.00 KB (1015808)
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 992.00 KB (1015808)
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - SystemAllocTime: 2.384ms
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- Dequeue:
- - FirstBatchWaitTime: 6s669ms
- - TotalBytesDequeued: 221.77 MB (232547548)
- - TotalGetBatchTime: 8s644ms
- - DataWaitTime: 8s620ms
- Enqueue:
- - DeserializeRowBatchTime: 671.310ms
- - TotalBatchesEnqueued: 9.35K (9353)
- - TotalBatchesReceived: 9.35K (9353)
- - TotalBytesReceived: 98.11 MB (102872388)
- - TotalEarlySenders: 0 (0)
- - TotalEosReceived: 3 (3)
- - TotalHasDeferredRPCsTime: 0.000ns
- - TotalRPCsDeferred: 0 (0)
- CodeGen:(Total: 156.328ms, non-child: 156.328ms, % non-child: 100.00%)
- - CodegenInvoluntaryContextSwitches: 0 (0)
- - CodegenTotalWallClockTime: 156.316ms
- - CodegenSysTime: 3.852ms
- - CodegenUserTime: 152.786ms
- - CodegenVoluntaryContextSwitches: 0 (0)
- - CompileTime: 38.029ms
- - IrGenerationTime: 3.661ms
- - LoadTime: 0.000ns
- - ModuleBitcodeSize: 2.28 MB (2393476)
- - NumFunctions: 57 (57)
- - NumInstructions: 1.63K (1626)
- - OptimizationTime: 88.224ms
- - PeakMemoryUsage: 813.00 KB (832512)
- - PrepareTime: 25.545ms
- Fragment F01:
- Instance 434987288e39baec:45101a3900000006 (host=itchil805.inteliquent.com:22000):(Total: 9s748ms, non-child: 966.160us, % non-child: 0.01%)
- Last report received time: 2020-02-11 13:42:01.581
- Fragment Instance Lifecycle Event Timeline: 9s749ms
- - Prepare Finished: 1.955ms (1.955ms)
- - Open Finished: 9s747ms (9s746ms)
- - First Batch Produced: 9s747ms (5.194us)
- - First Batch Sent: 9s748ms (28.907us)
- - ExecInternal Finished: 9s749ms (1.042ms)
- - MemoryUsage (500.000ms): 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 52.53 MB, 108.92 MB, 161.23 MB, 165.23 MB, 181.23 MB, 193.23 MB
- - ThreadUsage (500.000ms): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1
- - AverageThreadTokens: 1.00
- - BloomFilterBytes: 0
- - ExchangeScanRatio: 0.00
- - PeakMemoryUsage: 193.23 MB (202613782)
- - PeakReservation: 192.00 MB (201326592)
- - PeakUsedReservation: 0
- - PerHostPeakMemUsage: 2.00 GB (2144411750)
- - RowsProduced: 1 (1)
- - TotalNetworkReceiveTime: 8s650ms
- - TotalNetworkSendTime: 854.560us
- - TotalStorageWaitTime: 0.000ns
- - TotalThreadsInvoluntaryContextSwitches: 7 (7)
- - TotalThreadsTotalWallClockTime: 9s746ms
- - TotalThreadsSysTime: 76.204ms
- - TotalThreadsUserTime: 1s113ms
- - TotalThreadsVoluntaryContextSwitches: 7.08K (7076)
- Buffer pool:
- - AllocTime: 0.000ns
- - CumulativeAllocationBytes: 0
- - CumulativeAllocations: 0 (0)
- - PeakReservation: 0
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 0
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - ReservationLimit: 0
- - SystemAllocTime: 0.000ns
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- Fragment Instance Lifecycle Timings:
- - ExecTime: 922.107us
- - ExecTreeExecTime: 4.160us
- - OpenTime: 9s746ms
- - ExecTreeOpenTime: 9s590ms
- - PrepareTime: 1.302ms
- - ExecTreePrepareTime: 643.950us
- KrpcDataStreamSender (dst_id=5):(Total: 973.453us, non-child: 118.024us, % non-child: 12.12%)
- ExecOption: Unpartitioned Sender Codegen Disabled: not needed
- - BytesSent (500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- - NetworkThroughput: (Avg: 25.88 KB/sec ; Min: 25.88 KB/sec ; Max: 25.88 KB/sec ; Number of samples: 1)
- - EosSent: 1 (1)
- - PeakMemoryUsage: 16.00 KB (16384)
- - RowsSent: 1 (1)
- - RpcFailure: 0 (0)
- - RpcRetry: 0 (0)
- - SerializeBatchTime: 6.180us
- - TotalBytesSent: 12.00 B (12)
- - UncompressedRowBatchSize: 16.00 B (16)
- AGGREGATION_NODE (id=2):(Total: 9s590ms, non-child: 1.134ms, % non-child: 0.01%)
- Node Lifecycle Event Timeline: 9s748ms
- - Open Started: 157.879ms (157.879ms)
- - Open Finished: 9s747ms (9s590ms)
- - First Batch Requested: 9s747ms (24.105us)
- - First Batch Returned: 9s747ms (2.445us)
- - Last Batch Returned: 9s747ms (348.000ns)
- - Closed: 9s748ms (934.683us)
- - PeakMemoryUsage: 16.00 KB (16384)
- - RowsReturned: 1 (1)
- - RowsReturnedRate: 0
- NonGroupingAggregator 0:
- ExecOption: Codegen Enabled
- - BuildTime: 657.443us
- - PeakMemoryUsage: 8.00 KB (8192)
- - RowsReturned: 1 (1)
- AGGREGATION_NODE (id=4):(Total: 9s589ms, non-child: 895.905ms, % non-child: 9.34%)
- Node Lifecycle Event Timeline: 9s747ms
- - Open Started: 157.880ms (157.880ms)
- - Open Finished: 9s711ms (9s554ms)
- - First Batch Requested: 9s711ms (13.807us)
- - First Batch Returned: 9s711ms (84.999us)
- - Last Batch Returned: 9s747ms (35.932ms)
- - Closed: 9s747ms (54.060us)
- - PeakMemoryUsage: 192.11 MB (201441408)
- - RowsReturned: 1.15M (1154681)
- - RowsReturnedRate: 120.41 K/sec
- GroupingAggregator 0:
- ExecOption: Codegen Enabled
- - BuildTime: 854.089ms
- - GetResultsTime: 34.583ms
- - HTResizeTime: 64.639ms
- - HashBuckets: 2.10M (2097152)
- - LargestPartitionPercent: 4 (4)
- - MaxPartitionLevel: 0 (0)
- - NumRepartitions: 0 (0)
- - PartitionsCreated: 16 (16)
- - PeakMemoryUsage: 192.11 MB (201437312)
- - PeakReservation: 192.00 MB (201326592)
- - PeakUsedReservation: 0
- - RowsRepartitioned: 0 (0)
- - RowsReturned: 1.15M (1154681)
- - SpilledPartitions: 0 (0)
- Buffer pool:
- - AllocTime: 19.627ms
- - CumulativeAllocationBytes: 208.00 MB (218103808)
- - CumulativeAllocations: 104 (104)
- - PeakReservation: 192.00 MB (201326592)
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 192.00 MB (201326592)
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - SystemAllocTime: 19.486ms
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- EXCHANGE_NODE (id=3):(Total: 8s693ms, non-child: 69.571ms, % non-child: 0.80%)
- Node Lifecycle Event Timeline: 9s711ms
- - Open Started: 157.881ms (157.881ms)
- - Open Finished: 6s827ms (6s669ms)
- - First Batch Requested: 6s827ms (507.809us)
- - First Batch Returned: 6s830ms (2.104ms)
- - Last Batch Returned: 9s711ms (2s881ms)
- - Closed: 9s711ms (334.097us)
- - ConvertRowBatchTime: 34.530ms
- - PeakMemoryUsage: 1.09 MB (1138688)
- - RowsReturned: 1.70M (1703171)
- - RowsReturnedRate: 195.91 K/sec
- Buffer pool:
- - AllocTime: 16.925ms
- - CumulativeAllocationBytes: 365.57 MB (383328256)
- - CumulativeAllocations: 18.72K (18720)
- - PeakReservation: 1.09 MB (1138688)
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 1.09 MB (1138688)
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - SystemAllocTime: 1.646ms
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- Dequeue:
- - BytesDequeued (500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 29.10 MB, 86.75 MB, 139.44 MB, 162.79 MB, 192.01 MB, 221.27 MB
- - FirstBatchWaitTime: 6s669ms
- - TotalBytesDequeued: 221.92 MB (232700420)
- - TotalGetBatchTime: 8s648ms
- - DataWaitTime: 8s624ms
- Enqueue:
- - BytesReceived (500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 12.90 MB, 38.41 MB, 61.74 MB, 72.07 MB, 84.97 MB, 97.90 MB
- - DeferredQueueSize (500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- - DispatchTime: (Avg: 26.238us ; Min: 13.959us ; Max: 180.908us ; Number of samples: 9360)
- - DeserializeRowBatchTime: 671.705ms
- - TotalBatchesEnqueued: 9.36K (9360)
- - TotalBatchesReceived: 9.36K (9360)
- - TotalBytesReceived: 98.17 MB (102942688)
- - TotalEarlySenders: 0 (0)
- - TotalEosReceived: 3 (3)
- - TotalHasDeferredRPCsTime: 0.000ns
- - TotalRPCsDeferred: 0 (0)
- CodeGen:(Total: 155.712ms, non-child: 155.712ms, % non-child: 100.00%)
- - CodegenInvoluntaryContextSwitches: 0 (0)
- - CodegenTotalWallClockTime: 155.700ms
- - CodegenSysTime: 3.962ms
- - CodegenUserTime: 150.589ms
- - CodegenVoluntaryContextSwitches: 0 (0)
- - CompileTime: 38.069ms
- - IrGenerationTime: 3.796ms
- - LoadTime: 0.000ns
- - ModuleBitcodeSize: 2.28 MB (2393476)
- - NumFunctions: 57 (57)
- - NumInstructions: 1.63K (1626)
- - OptimizationTime: 88.107ms
- - PeakMemoryUsage: 813.00 KB (832512)
- - PrepareTime: 24.849ms
- Instance 434987288e39baec:45101a3900000005 (host=itchil804.inteliquent.com:22000):(Total: 9s747ms, non-child: 665.820us, % non-child: 0.01%)
- Last report received time: 2020-02-11 13:42:01.839
- Fragment Instance Lifecycle Event Timeline: 9s748ms
- - Prepare Finished: 2.011ms (2.011ms)
- - Open Finished: 9s747ms (9s745ms)
- - First Batch Produced: 9s747ms (4.810us)
- - First Batch Sent: 9s747ms (32.231us)
- - ExecInternal Finished: 9s748ms (804.987us)
- - MemoryUsage (500.000ms): 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 36.53 MB, 77.15 MB, 135.15 MB, 161.15 MB, 175.15 MB, 187.15 MB
- - ThreadUsage (500.000ms): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1
- - AverageThreadTokens: 1.00
- - BloomFilterBytes: 0
- - ExchangeScanRatio: 0.00
- - PeakMemoryUsage: 193.15 MB (202531862)
- - PeakReservation: 192.00 MB (201326592)
- - PeakUsedReservation: 0
- - PerHostPeakMemUsage: 2.17 GB (2325567844)
- - RowsProduced: 1 (1)
- - TotalNetworkReceiveTime: 8s704ms
- - TotalNetworkSendTime: 631.237us
- - TotalStorageWaitTime: 0.000ns
- - TotalThreadsInvoluntaryContextSwitches: 2 (2)
- - TotalThreadsTotalWallClockTime: 9s745ms
- - TotalThreadsSysTime: 103.776ms
- - TotalThreadsUserTime: 1s022ms
- - TotalThreadsVoluntaryContextSwitches: 7.24K (7244)
- Buffer pool:
- - AllocTime: 0.000ns
- - CumulativeAllocationBytes: 0
- - CumulativeAllocations: 0 (0)
- - PeakReservation: 0
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 0
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - ReservationLimit: 0
- - SystemAllocTime: 0.000ns
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- Fragment Instance Lifecycle Timings:
- - ExecTime: 702.930us
- - ExecTreeExecTime: 3.598us
- - OpenTime: 9s745ms
- - ExecTreeOpenTime: 9s590ms
- - PrepareTime: 1.176ms
- - ExecTreePrepareTime: 624.957us
- KrpcDataStreamSender (dst_id=5):(Total: 825.773us, non-child: 193.994us, % non-child: 23.49%)
- ExecOption: Unpartitioned Sender Codegen Disabled: not needed
- - BytesSent (500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- - NetworkThroughput: (Avg: 40.17 KB/sec ; Min: 40.17 KB/sec ; Max: 40.17 KB/sec ; Number of samples: 1)
- - EosSent: 1 (1)
- - PeakMemoryUsage: 16.00 KB (16384)
- - RowsSent: 1 (1)
- - RpcFailure: 0 (0)
- - RpcRetry: 0 (0)
- - SerializeBatchTime: 7.636us
- - TotalBytesSent: 12.00 B (12)
- - UncompressedRowBatchSize: 16.00 B (16)
- AGGREGATION_NODE (id=2):(Total: 9s590ms, non-child: 1.160ms, % non-child: 0.01%)
- Node Lifecycle Event Timeline: 9s747ms
- - Open Started: 157.101ms (157.101ms)
- - Open Finished: 9s747ms (9s590ms)
- - First Batch Requested: 9s747ms (22.844us)
- - First Batch Returned: 9s747ms (1.905us)
- - Last Batch Returned: 9s747ms (118.000ns)
- - Closed: 9s747ms (734.099us)
- - PeakMemoryUsage: 16.00 KB (16384)
- - RowsReturned: 1 (1)
- - RowsReturnedRate: 0
- NonGroupingAggregator 0:
- ExecOption: Codegen Enabled
- - BuildTime: 623.687us
- - PeakMemoryUsage: 8.00 KB (8192)
- - RowsReturned: 1 (1)
- AGGREGATION_NODE (id=4):(Total: 9s589ms, non-child: 845.825ms, % non-child: 8.82%)
- Node Lifecycle Event Timeline: 9s747ms
- - Open Started: 157.102ms (157.102ms)
- - Open Finished: 9s713ms (9s556ms)
- - First Batch Requested: 9s713ms (11.429us)
- - First Batch Returned: 9s713ms (56.535us)
- - Last Batch Returned: 9s747ms (33.773ms)
- - Closed: 9s747ms (120.131us)
- - PeakMemoryUsage: 192.11 MB (201441408)
- - RowsReturned: 1.15M (1152636)
- - RowsReturnedRate: 120.20 K/sec
- GroupingAggregator 0:
- ExecOption: Codegen Enabled
- - BuildTime: 806.089ms
- - GetResultsTime: 32.459ms
- - HTResizeTime: 50.989ms
- - HashBuckets: 2.10M (2097152)
- - LargestPartitionPercent: 4 (4)
- - MaxPartitionLevel: 0 (0)
- - NumRepartitions: 0 (0)
- - PartitionsCreated: 16 (16)
- - PeakMemoryUsage: 192.11 MB (201437312)
- - PeakReservation: 192.00 MB (201326592)
- - PeakUsedReservation: 0
- - RowsRepartitioned: 0 (0)
- - RowsReturned: 1.15M (1152636)
- - SpilledPartitions: 0 (0)
- Buffer pool:
- - AllocTime: 27.779ms
- - CumulativeAllocationBytes: 208.00 MB (218103808)
- - CumulativeAllocations: 104 (104)
- - PeakReservation: 192.00 MB (201326592)
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 192.00 MB (201326592)
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - SystemAllocTime: 27.619ms
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- EXCHANGE_NODE (id=3):(Total: 8s743ms, non-child: 65.924ms, % non-child: 0.75%)
- Node Lifecycle Event Timeline: 9s713ms
- - Open Started: 157.103ms (157.103ms)
- - Open Finished: 6s829ms (6s672ms)
- - First Batch Requested: 6s830ms (784.435us)
- - First Batch Returned: 6s832ms (1.924ms)
- - Last Batch Returned: 9s712ms (2s880ms)
- - Closed: 9s713ms (505.500us)
- - ConvertRowBatchTime: 31.041ms
- - PeakMemoryUsage: 1.01 MB (1056768)
- - RowsReturned: 1.70M (1701728)
- - RowsReturnedRate: 194.62 K/sec
- Buffer pool:
- - AllocTime: 19.958ms
- - CumulativeAllocationBytes: 365.25 MB (382992384)
- - CumulativeAllocations: 18.70K (18702)
- - PeakReservation: 1.01 MB (1056768)
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 1.01 MB (1056768)
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - SystemAllocTime: 3.997ms
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- Dequeue:
- - BytesDequeued (500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 5.05 MB, 56.44 MB, 114.14 MB, 147.32 MB, 176.53 MB, 206.10 MB
- - FirstBatchWaitTime: 6s672ms
- - TotalBytesDequeued: 221.73 MB (232499480)
- - TotalGetBatchTime: 8s701ms
- - DataWaitTime: 8s677ms
- Enqueue:
- - BytesReceived (500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2.24 MB, 25.01 MB, 50.53 MB, 65.23 MB, 78.12 MB, 91.18 MB
- - DeferredQueueSize (500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- - DispatchTime: (Avg: 24.631us ; Min: 8.775us ; Max: 219.984us ; Number of samples: 9351)
- - DeserializeRowBatchTime: 683.958ms
- - TotalBatchesEnqueued: 9.35K (9351)
- - TotalBatchesReceived: 9.35K (9351)
- - TotalBytesReceived: 98.08 MB (102848549)
- - TotalEarlySenders: 0 (0)
- - TotalEosReceived: 3 (3)
- - TotalHasDeferredRPCsTime: 0.000ns
- - TotalRPCsDeferred: 0 (0)
- CodeGen:(Total: 154.949ms, non-child: 154.949ms, % non-child: 100.00%)
- - CodegenInvoluntaryContextSwitches: 0 (0)
- - CodegenTotalWallClockTime: 154.939ms
- - CodegenSysTime: 7.594ms
- - CodegenUserTime: 148.226ms
- - CodegenVoluntaryContextSwitches: 0 (0)
- - CompileTime: 38.064ms
- - IrGenerationTime: 3.264ms
- - LoadTime: 0.000ns
- - ModuleBitcodeSize: 2.28 MB (2393476)
- - NumFunctions: 57 (57)
- - NumInstructions: 1.63K (1626)
- - OptimizationTime: 87.573ms
- - PeakMemoryUsage: 813.00 KB (832512)
- - PrepareTime: 25.098ms
- Instance 434987288e39baec:45101a3900000004 (host=itchil806.inteliquent.com:22000):(Total: 9s745ms, non-child: 889.784us, % non-child: 0.01%)
- Last report received time: 2020-02-11 13:42:01.579
- Fragment Instance Lifecycle Event Timeline: 9s746ms
- - Prepare Finished: 2.025ms (2.025ms)
- - Open Finished: 9s745ms (9s743ms)
- - First Batch Produced: 9s745ms (4.893us)
- - First Batch Sent: 9s745ms (33.456us)
- - ExecInternal Finished: 9s746ms (1.333ms)
- - MemoryUsage (500.000ms): 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 137.02 KB, 52.61 MB, 106.84 MB, 160.95 MB, 164.95 MB, 182.95 MB, 108.15 MB
- - ThreadUsage (500.000ms): 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1
- - AverageThreadTokens: 1.00
- - BloomFilterBytes: 0
- - ExchangeScanRatio: 0.00
- - PeakMemoryUsage: 192.95 MB (202327062)
- - PeakReservation: 192.00 MB (201326592)
- - PeakUsedReservation: 0
- - PerHostPeakMemUsage: 1.92 GB (2058856884)
- - RowsProduced: 1 (1)
- - TotalNetworkReceiveTime: 8s584ms
- - TotalNetworkSendTime: 917.335us
- - TotalStorageWaitTime: 0.000ns
- - TotalThreadsInvoluntaryContextSwitches: 2 (2)
- - TotalThreadsTotalWallClockTime: 9s744ms
- - TotalThreadsSysTime: 81.660ms
- - TotalThreadsUserTime: 1s175ms
- - TotalThreadsVoluntaryContextSwitches: 6.67K (6665)
- Buffer pool:
- - AllocTime: 0.000ns
- - CumulativeAllocationBytes: 0
- - CumulativeAllocations: 0 (0)
- - PeakReservation: 0
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 0
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - ReservationLimit: 0
- - SystemAllocTime: 0.000ns
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- Fragment Instance Lifecycle Timings:
- - ExecTime: 980.205us
- - ExecTreeExecTime: 3.542us
- - OpenTime: 9s743ms
- - ExecTreeOpenTime: 9s584ms
- - PrepareTime: 1.404ms
- - ExecTreePrepareTime: 777.217us
- KrpcDataStreamSender (dst_id=5):(Total: 1.026ms, non-child: 108.027us, % non-child: 10.53%)
- ExecOption: Unpartitioned Sender Codegen Disabled: not needed
- - BytesSent (500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- - NetworkThroughput: (Avg: 24.21 KB/sec ; Min: 24.21 KB/sec ; Max: 24.21 KB/sec ; Number of samples: 1)
- - EosSent: 1 (1)
- - PeakMemoryUsage: 16.00 KB (16384)
- - RowsSent: 1 (1)
- - RpcFailure: 0 (0)
- - RpcRetry: 0 (0)
- - SerializeBatchTime: 8.096us
- - TotalBytesSent: 12.00 B (12)
- - UncompressedRowBatchSize: 16.00 B (16)
- AGGREGATION_NODE (id=2):(Total: 9s585ms, non-child: 1.136ms, % non-child: 0.01%)
- Node Lifecycle Event Timeline: 9s746ms
- - Open Started: 160.531ms (160.531ms)
- - Open Finished: 9s745ms (9s584ms)
- - First Batch Requested: 9s745ms (22.466us)
- - First Batch Returned: 9s745ms (2.236us)
- - Last Batch Returned: 9s745ms (289.000ns)
- - Closed: 9s746ms (997.955us)
- - PeakMemoryUsage: 16.00 KB (16384)
- - RowsReturned: 1 (1)
- - RowsReturnedRate: 0
- NonGroupingAggregator 0:
- ExecOption: Codegen Enabled
- - BuildTime: 651.117us
- - PeakMemoryUsage: 8.00 KB (8192)
- - RowsReturned: 1 (1)
- AGGREGATION_NODE (id=4):(Total: 9s584ms, non-child: 954.214ms, % non-child: 9.96%)
- Node Lifecycle Event Timeline: 9s745ms
- - Open Started: 160.532ms (160.532ms)
- - Open Finished: 9s710ms (9s550ms)
- - First Batch Requested: 9s710ms (15.557us)
- - First Batch Returned: 9s710ms (78.560us)
- - Last Batch Returned: 9s745ms (34.804ms)
- - Closed: 9s745ms (55.318us)
- - PeakMemoryUsage: 192.11 MB (201441408)
- - RowsReturned: 1.15M (1151702)
- - RowsReturnedRate: 120.16 K/sec
- GroupingAggregator 0:
- ExecOption: Codegen Enabled
- - BuildTime: 912.992ms
- - GetResultsTime: 33.529ms
- - HTResizeTime: 61.362ms
- - HashBuckets: 2.10M (2097152)
- - LargestPartitionPercent: 4 (4)
- - MaxPartitionLevel: 0 (0)
- - NumRepartitions: 0 (0)
- - PartitionsCreated: 16 (16)
- - PeakMemoryUsage: 192.11 MB (201437312)
- - PeakReservation: 192.00 MB (201326592)
- - PeakUsedReservation: 0
- - RowsRepartitioned: 0 (0)
- - RowsReturned: 1.15M (1151702)
- - SpilledPartitions: 0 (0)
- Buffer pool:
- - AllocTime: 18.572ms
- - CumulativeAllocationBytes: 208.00 MB (218103808)
- - CumulativeAllocations: 104 (104)
- - PeakReservation: 192.00 MB (201326592)
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 192.00 MB (201326592)
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - SystemAllocTime: 18.414ms
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- EXCHANGE_NODE (id=3):(Total: 8s630ms, non-child: 71.785ms, % non-child: 0.83%)
- Node Lifecycle Event Timeline: 9s710ms
- - Open Started: 160.533ms (160.533ms)
- - Open Finished: 6s827ms (6s667ms)
- - First Batch Requested: 6s828ms (809.173us)
- - First Batch Returned: 6s830ms (1.902ms)
- - Last Batch Returned: 9s710ms (2s879ms)
- - Closed: 9s710ms (382.620us)
- - ConvertRowBatchTime: 36.872ms
- - PeakMemoryUsage: 832.00 KB (851968)
- - RowsReturned: 1.70M (1701332)
- - RowsReturnedRate: 197.13 K/sec
- Buffer pool:
- - AllocTime: 16.852ms
- - CumulativeAllocationBytes: 365.16 MB (382902272)
- - CumulativeAllocations: 18.70K (18698)
- - PeakReservation: 832.00 KB (851968)
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 832.00 KB (851968)
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - SystemAllocTime: 1.509ms
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- Dequeue:
- - BytesDequeued (500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 32.53 MB, 90.11 MB, 139.33 MB, 164.42 MB, 193.59 MB
- - FirstBatchWaitTime: 6s667ms
- - TotalBytesDequeued: 221.67 MB (232442744)
- - TotalGetBatchTime: 8s581ms
- - DataWaitTime: 8s558ms
- Enqueue:
- - BytesReceived (500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 14.41 MB, 39.90 MB, 61.69 MB, 72.77 MB, 85.66 MB
- - DeferredQueueSize (500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
- - DispatchTime: (Avg: 28.844us ; Min: 10.155us ; Max: 301.511us ; Number of samples: 9349)
- - DeserializeRowBatchTime: 658.267ms
- - TotalBatchesEnqueued: 9.35K (9349)
- - TotalBatchesReceived: 9.35K (9349)
- - TotalBytesReceived: 98.06 MB (102825929)
- - TotalEarlySenders: 0 (0)
- - TotalEosReceived: 3 (3)
- - TotalHasDeferredRPCsTime: 0.000ns
- - TotalRPCsDeferred: 0 (0)
- CodeGen:(Total: 158.322ms, non-child: 158.322ms, % non-child: 100.00%)
- - CodegenInvoluntaryContextSwitches: 0 (0)
- - CodegenTotalWallClockTime: 158.309ms
- - CodegenSysTime: 0.000ns
- - CodegenUserTime: 159.543ms
- - CodegenVoluntaryContextSwitches: 0 (0)
- - CompileTime: 37.953ms
- - IrGenerationTime: 3.923ms
- - LoadTime: 0.000ns
- - ModuleBitcodeSize: 2.28 MB (2393476)
- - NumFunctions: 57 (57)
- - NumInstructions: 1.63K (1626)
- - OptimizationTime: 88.991ms
- - PeakMemoryUsage: 813.00 KB (832512)
- - PrepareTime: 26.689ms
- Averaged Fragment F00:(Total: 8s670ms, non-child: 1.908ms, % non-child: 0.02%)
- split sizes: min: 4.61 GB, max: 5.26 GB, avg: 4.83 GB, stddev: 311.98 MB
- completion times: min:9s750ms max:10s012ms mean: 9s838ms stddev:123.069ms
- execution rates: min:484.12 MB/sec max:537.61 MB/sec mean:501.96 MB/sec stddev:25.21 MB/sec
- num instances: 3
- - AverageThreadTokens: 10.71
- - BloomFilterBytes: 0
- - ExchangeScanRatio: 0.07
- - PeakMemoryUsage: 1.99 GB (2140482836)
- - PeakReservation: 1.92 GB (2061849941)
- - PeakUsedReservation: 0
- - PerHostPeakMemUsage: 2.03 GB (2176278826)
- - RowsProduced: 1.70M (1702077)
- - TotalNetworkReceiveTime: 0.000ns
- - TotalNetworkSendTime: 13.343ms
- - TotalStorageWaitTime: 8s959ms
- - TotalThreadsInvoluntaryContextSwitches: 64 (64)
- - TotalThreadsTotalWallClockTime: 1m33s
- - TotalThreadsSysTime: 900.711ms
- - TotalThreadsUserTime: 23s920ms
- - TotalThreadsVoluntaryContextSwitches: 19.38K (19379)
- Buffer pool:
- - AllocTime: 0.000ns
- - CumulativeAllocationBytes: 0
- - CumulativeAllocations: 0 (0)
- - PeakReservation: 0
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 0
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - ReservationLimit: 0
- - SystemAllocTime: 0.000ns
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- Fragment Instance Lifecycle Timings:
- - ExecTime: 8s360ms
- - ExecTreeExecTime: 7s119ms
- - OpenTime: 306.424ms
- - ExecTreeOpenTime: 495.525us
- - PrepareTime: 3.356ms
- - ExecTreePrepareTime: 2.605ms
- KrpcDataStreamSender (dst_id=3):(Total: 1s240ms, non-child: 1s240ms, % non-child: 100.00%)
- - EosSent: 3 (3)
- - PeakMemoryUsage: 88.27 KB (90384)
- - RowsSent: 1.70M (1702077)
- - RpcFailure: 0 (0)
- - RpcRetry: 0 (0)
- - SerializeBatchTime: 678.173ms
- - TotalBytesSent: 98.11 MB (102872388)
- - UncompressedRowBatchSize: 221.77 MB (232547548)
- AGGREGATION_NODE (id=1):(Total: 7s121ms, non-child: 6s900ms, % non-child: 96.89%)
- - PeakMemoryUsage: 279.09 MB (292651914)
- - RowsReturned: 1.70M (1702077)
- - RowsReturnedRate: 239.38 K/sec
- GroupingAggregator 0:
- - BuildTime: 0.000ns
- - GetResultsTime: 47.917ms
- - HTResizeTime: 73.215ms
- - HashBuckets: 3.50M (3495253)
- - LargestPartitionPercent: 0 (0)
- - PartitionsCreated: 16 (16)
- - PeakMemoryUsage: 277.44 MB (290911701)
- - PeakReservation: 277.33 MB (290805077)
- - PeakUsedReservation: 0
- - ReductionFactorEstimate: 7.70
- - ReductionFactorThresholdToExpand: 2.00
- - RowsPassedThrough: 0 (0)
- - RowsReturned: 1.70M (1702077)
- - StreamingTime: 6s336ms
- Buffer pool:
- - AllocTime: 14.013ms
- - CumulativeAllocationBytes: 316.67 MB (332049066)
- - CumulativeAllocations: 147 (147)
- - PeakReservation: 277.33 MB (290805077)
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 277.33 MB (290805077)
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - SystemAllocTime: 13.854ms
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- HDFS_SCAN_NODE (id=0):(Total: 221.143ms, non-child: 221.143ms, % non-child: 100.00%)
- - AverageHdfsReadThreadConcurrency: 0.49
- - AverageScannerThreadConcurrency: 11.78
- - BytesRead: 1.36 GB (1461832134)
- - BytesReadDataNodeCache: 0
- - BytesReadLocal: 1.36 GB (1461832134)
- - BytesReadRemoteUnexpected: 0
- - BytesReadShortCircuit: 1.36 GB (1461832134)
- - CachedFileHandlesHitCount: 1.04K (1038)
- - CachedFileHandlesMissCount: 138 (138)
- - CollectionItemsRead: 0 (0)
- - DataCacheHitBytes: 0
- - DataCacheHitCount: 0 (0)
- - DataCacheMissBytes: 0
- - DataCacheMissCount: 0 (0)
- - DataCachePartialHitCount: 0 (0)
- - DecompressionTime: 975.586ms
- - MaterializeTupleTime(*): 14s357ms
- - MaxCompressedTextFileLength: 0
- - NumColumns: 8 (8)
- - NumDictFilteredRowGroups: 30 (30)
- - NumDisksAccessed: 8 (8)
- - NumRowGroups: 163 (163)
- - NumScannerThreadMemUnavailable: 0 (0)
- - NumScannerThreadReservationsDenied: 0 (0)
- - NumScannerThreadsStarted: 31 (31)
- - NumScannersWithNoReads: 0 (0)
- - NumStatsFilteredRowGroups: 23 (23)
- - PeakMemoryUsage: 1.90 GB (2045374139)
- - PeakScannerThreadConcurrency: 31 (31)
- - PerReadThreadRawHdfsThroughput: 478.41 MB/sec
- - RemoteScanRanges: 0 (0)
- - RowBatchBytesEnqueued: 13.42 GB (14410423370)
- - RowBatchQueueGetWaitTime: 124.121ms
- - RowBatchQueuePeakMemoryUsage: 37.14 MB (38945348)
- - RowBatchQueuePutWaitTime: 59s334ms
- - RowBatchesEnqueued: 17.98K (17976)
- - RowsRead: 173.34M (173336970)
- - RowsReturned: 18.01M (18014920)
- - RowsReturnedRate: 87.52 M/sec
- - ScanRangesComplete: 160 (160)
- - ScannerIoWaitTime: 8s959ms
- - ScannerThreadWorklessLoops: 0 (0)
- - ScannerThreadsInvoluntaryContextSwitches: 54 (54)
- - ScannerThreadsTotalWallClockTime: 1m24s
- - ScannerThreadsSysTime: 336.371ms
- - ScannerThreadsUserTime: 15s955ms
- - ScannerThreadsVoluntaryContextSwitches: 18.62K (18617)
- - TotalRawHdfsOpenFileTime(*): 299.732ms
- - TotalRawHdfsReadTime(*): 3s359ms
- - TotalReadThroughput: 189.96 MB/sec
- Buffer pool:
- - AllocTime: 47.276ms
- - CumulativeAllocationBytes: 1.34 GB (1439490048)
- - CumulativeAllocations: 1.14K (1141)
- - PeakReservation: 1.83 GB (1967478101)
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 1.25 GB (1345410389)
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - SystemAllocTime: 45.536ms
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- CodeGen:(Total: 305.784ms, non-child: 305.784ms, % non-child: 100.00%)
- - CodegenInvoluntaryContextSwitches: 1 (1)
- - CodegenTotalWallClockTime: 305.772ms
- - CodegenSysTime: 4.551ms
- - CodegenUserTime: 300.508ms
- - CodegenVoluntaryContextSwitches: 0 (0)
- - CompileTime: 86.383ms
- - IrGenerationTime: 6.873ms
- - LoadTime: 0.000ns
- - ModuleBitcodeSize: 2.28 MB (2393476)
- - NumFunctions: 102 (102)
- - NumInstructions: 3.39K (3387)
- - OptimizationTime: 186.402ms
- - PeakMemoryUsage: 1.65 MB (1734144)
- - PrepareTime: 25.121ms
- Fragment F00:
- Instance 434987288e39baec:45101a3900000002 (host=itchil804.inteliquent.com:22000):(Total: 9s712ms, non-child: 1.949ms, % non-child: 0.02%)
- Last report received time: 2020-02-11 13:42:01.839
- Hdfs split stats (<volume id>:<# splits>/<split lengths>): 2:27/93.28 MB 7:25/743.47 MB 3:22/1.39 GB 4:25/209.13 MB 6:30/463.04 MB 5:28/281.73 MB 0:26/1.03 GB 1:28/1.09 GB
- Fragment Instance Lifecycle Event Timeline: 9s713ms
- - Prepare Finished: 5.040ms (5.040ms)
- - Open Finished: 304.795ms (299.754ms)
- - First Batch Produced: 8s302ms (7s997ms)
- - First Batch Sent: 8s302ms (3.281us)
- - ExecInternal Finished: 9s713ms (1s411ms)
- - MemoryUsage (500.000ms): 1.64 GB, 1.72 GB, 1.97 GB, 2.03 GB, 1.74 GB, 1.60 GB, 1.48 GB, 1.48 GB, 1.51 GB, 1.37 GB, 1.30 GB, 1.34 GB, 1.11 GB, 1.11 GB, 865.51 MB, 508.31 MB, 300.17 MB, 180.18 MB, 80.21 MB
- - ThreadUsage (500.000ms): 32, 32, 32, 32, 19, 15, 11, 11, 11, 9, 8, 8, 6, 6, 4, 2, 1, 1, 1
- - AverageThreadTokens: 12.68
- - BloomFilterBytes: 0
- - ExchangeScanRatio: 0.07
- - PeakMemoryUsage: 2.13 GB (2289763662)
- - PeakReservation: 2.06 GB (2208301056)
- - PeakUsedReservation: 0
- - PerHostPeakMemUsage: 2.17 GB (2325567844)
- - RowsProduced: 1.90M (1897875)
- - TotalNetworkReceiveTime: 0.000ns
- - TotalNetworkSendTime: 19.912ms
- - TotalStorageWaitTime: 15s211ms
- - TotalThreadsInvoluntaryContextSwitches: 89 (89)
- - TotalThreadsTotalWallClockTime: 1m56s
- - TotalThreadsSysTime: 1s102ms
- - TotalThreadsUserTime: 27s837ms
- - TotalThreadsVoluntaryContextSwitches: 22.23K (22229)
- Buffer pool:
- - AllocTime: 0.000ns
- - CumulativeAllocationBytes: 0
- - CumulativeAllocations: 0 (0)
- - PeakReservation: 0
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 0
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - ReservationLimit: 0
- - SystemAllocTime: 0.000ns
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- Fragment Instance Lifecycle Timings:
- - ExecTime: 9s408ms
- - ExecTreeExecTime: 8s054ms
- - OpenTime: 299.752ms
- - ExecTreeOpenTime: 470.708us
- - PrepareTime: 3.932ms
- - ExecTreePrepareTime: 3.153ms
- KrpcDataStreamSender (dst_id=3):(Total: 1s354ms, non-child: 1s333ms, % non-child: 98.46%)
- ExecOption: Hash Partitioned Sender Codegen Enabled
- - BytesSent (500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10.66 MB, 49.34 MB, 88.50 MB
- - NetworkThroughput: (Avg: 62.29 MB/sec ; Min: 11.34 MB/sec ; Max: 119.52 MB/sec ; Number of samples: 10429)
- - EosSent: 3 (3)
- - PeakMemoryUsage: 88.27 KB (90384)
- - RowsSent: 1.90M (1897875)
- - RpcFailure: 0 (0)
- - RpcRetry: 0 (0)
- - SerializeBatchTime: 751.840ms
- - TotalBytesSent: 109.23 MB (114535422)
- - UncompressedRowBatchSize: 247.27 MB (259285196)
- AGGREGATION_NODE (id=1):(Total: 8s057ms, non-child: 7s759ms, % non-child: 96.31%)
- Node Lifecycle Event Timeline: 9s713ms
- - Open Started: 304.313ms (304.313ms)
- - Open Finished: 304.779ms (466.185us)
- - First Batch Requested: 304.796ms (16.811us)
- - First Batch Returned: 8s302ms (7s997ms)
- - Last Batch Returned: 9s711ms (1s409ms)
- - Closed: 9s713ms (2.089ms)
- - PeakMemoryUsage: 321.54 MB (337162282)
- - RowsReturned: 1.90M (1897875)
- - RowsReturnedRate: 235.55 K/sec
- GroupingAggregator 0:
- ExecOption: Streaming Preaggregation, Codegen Enabled
- - BuildTime: 0.000ns
- - GetResultsTime: 55.603ms
- - HTResizeTime: 97.615ms
- - HashBuckets: 4.19M (4194304)
- - LargestPartitionPercent: 0 (0)
- - PartitionsCreated: 16 (16)
- - PeakMemoryUsage: 320.10 MB (335650944)
- - PeakReservation: 320.00 MB (335544320)
- - PeakUsedReservation: 0
- - ReductionFactorEstimate: 8.35
- - ReductionFactorThresholdToExpand: 2.00
- - RowsPassedThrough: 0 (0)
- - RowsReturned: 1.90M (1897875)
- - StreamingTime: 7s112ms
- Buffer pool:
- - AllocTime: 15.506ms
- - CumulativeAllocationBytes: 370.00 MB (387973120)
- - CumulativeAllocations: 169 (169)
- - PeakReservation: 320.00 MB (335544320)
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 320.00 MB (335544320)
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - SystemAllocTime: 15.316ms
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- HDFS_SCAN_NODE (id=0):(Total: 297.334ms, non-child: 297.334ms, % non-child: 100.00%)
- Hdfs split stats (<volume id>:<# splits>/<split lengths>): 2:27/93.28 MB 7:25/743.47 MB 3:22/1.39 GB 4:25/209.13 MB 6:30/463.04 MB 5:28/281.73 MB 0:26/1.03 GB 1:28/1.09 GB
- ExecOption: PARQUET Codegen Enabled, Codegen enabled: 211 out of 211
- Hdfs Read Thread Concurrency Bucket: 0:75% 1:0% 2:18.75% 3:0% 4:0% 5:6.25% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0%
- File Formats: PARQUET/NONE:28 PARQUET/SNAPPY:183
- Node Lifecycle Event Timeline: 8s302ms
- - Open Started: 304.316ms (304.316ms)
- - Open Finished: 304.466ms (149.783us)
- - First Batch Requested: 304.798ms (332.430us)
- - First Batch Returned: 328.473ms (23.675ms)
- - Last Batch Returned: 8s302ms (7s973ms)
- - Closed: 8s302ms (366.206us)
- - BytesRead (500.000ms): 206.54 MB, 724.60 MB, 996.93 MB, 1.21 GB, 1.33 GB, 1.33 GB, 1.33 GB, 1.33 GB, 1.36 GB, 1.40 GB, 1.41 GB, 1.42 GB, 1.43 GB, 1.46 GB, 1.47 GB, 1.48 GB
- - FooterProcessingTime: (Avg: 34.210ms ; Min: 668.107us ; Max: 525.336ms ; Number of samples: 211)
- - InitialRangeActualReservation: (Avg: 32.00 MB (33554432) ; Min: 32.00 MB (33554432) ; Max: 32.00 MB (33554432) ; Number of samples: 211)
- - InitialRangeIdealReservation: (Avg: 71.81 KB (73533) ; Min: 8.00 KB (8192) ; Max: 128.00 KB (131072) ; Number of samples: 211)
- - ParquetCompressedBytesReadPerColumn: (Avg: 187.35 MB (196448795) ; Min: 45.50 MB (47712830) ; Max: 278.76 MB (292300027) ; Number of samples: 8)
- - ParquetCompressedPageSize: (Avg: 99.91 KB (102310) ; Min: 10.00 B (10) ; Max: 192.31 KB (196924) ; Number of samples: 15361)
- - ParquetRowGroupActualReservation: (Avg: 41.36 MB (43370888) ; Min: 32.00 MB (33554432) ; Max: 184.00 MB (192937984) ; Number of samples: 188)
- - ParquetRowGroupIdealReservation: (Avg: 9.07 MB (9513395) ; Min: 64.00 KB (65536) ; Max: 184.00 MB (192937984) ; Number of samples: 188)
- - ParquetUncompressedBytesReadPerColumn: (Avg: 189.69 MB (198903044) ; Min: 45.49 MB (47695936) ; Max: 283.66 MB (297439301) ; Number of samples: 8)
- - ParquetUncompressedPageSize: (Avg: 101.16 KB (103588) ; Min: 8.00 B (8) ; Max: 192.28 KB (196890) ; Number of samples: 15361)
- - AverageHdfsReadThreadConcurrency: 0.69
- - AverageScannerThreadConcurrency: 13.88
- - BytesRead: 1.48 GB (1589431105)
- - BytesReadDataNodeCache: 0
- - BytesReadLocal: 1.48 GB (1589431105)
- - BytesReadRemoteUnexpected: 0
- - BytesReadShortCircuit: 1.48 GB (1589431105)
- - CachedFileHandlesHitCount: 1.35K (1352)
- - CachedFileHandlesMissCount: 196 (196)
- - CollectionItemsRead: 0 (0)
- - DataCacheHitBytes: 0
- - DataCacheHitCount: 0 (0)
- - DataCacheMissBytes: 0
- - DataCacheMissCount: 0 (0)
- - DataCachePartialHitCount: 0 (0)
- - DecompressionTime: 1s210ms
- - MaterializeTupleTime(*): 17s043ms
- - MaxCompressedTextFileLength: 0
- - NumColumns: 8 (8)
- - NumDictFilteredRowGroups: 40 (40)
- - NumDisksAccessed: 9 (9)
- - NumRowGroups: 216 (216)
- - NumScannerThreadMemUnavailable: 0 (0)
- - NumScannerThreadReservationsDenied: 0 (0)
- - NumScannerThreadsStarted: 31 (31)
- - NumScannersWithNoReads: 0 (0)
- - NumStatsFilteredRowGroups: 28 (28)
- - PeakMemoryUsage: 2.02 GB (2169608057)
- - PeakScannerThreadConcurrency: 31 (31)
- - PerReadThreadRawHdfsThroughput: 312.46 MB/sec
- - RemoteScanRanges: 0 (0)
- - RowBatchBytesEnqueued: 14.88 GB (15976722465)
- - RowBatchQueueGetWaitTime: 186.345ms
- - RowBatchQueuePeakMemoryUsage: 38.42 MB (40287438)
- - RowBatchQueuePutWaitTime: 1m11s
- - RowBatchesEnqueued: 20.36K (20363)
- - RowsRead: 189.05M (189051711)
- - RowsReturned: 20.33M (20332212)
- - RowsReturnedRate: 68.38 M/sec
- - ScanRangesComplete: 211 (211)
- - ScannerIoWaitTime: 15s211ms
- - ScannerThreadWorklessLoops: 0 (0)
- - ScannerThreadsInvoluntaryContextSwitches: 76 (76)
- - ScannerThreadsTotalWallClockTime: 1m46s
- - ScannerThreadsSysTime: 464.562ms
- - ScannerThreadsUserTime: 18s973ms
- - ScannerThreadsVoluntaryContextSwitches: 21.05K (21055)
- - TotalRawHdfsOpenFileTime(*): 396.581ms
- - TotalRawHdfsReadTime(*): 4s851ms
- - TotalReadThroughput: 189.24 MB/sec
- Buffer pool:
- - AllocTime: 48.626ms
- - CumulativeAllocationBytes: 1.55 GB (1665302528)
- - CumulativeAllocations: 1.52K (1520)
- - PeakReservation: 1.95 GB (2088763392)
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 1.41 GB (1512538112)
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - SystemAllocTime: 46.369ms
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- CodeGen:(Total: 299.131ms, non-child: 299.131ms, % non-child: 100.00%)
- - CodegenInvoluntaryContextSwitches: 0 (0)
- - CodegenTotalWallClockTime: 299.120ms
- - CodegenSysTime: 3.996ms
- - CodegenUserTime: 291.997ms
- - CodegenVoluntaryContextSwitches: 0 (0)
- - CompileTime: 85.669ms
- - IrGenerationTime: 6.711ms
- - LoadTime: 0.000ns
- - ModuleBitcodeSize: 2.28 MB (2393476)
- - NumFunctions: 102 (102)
- - NumInstructions: 3.39K (3387)
- - OptimizationTime: 180.688ms
- - PeakMemoryUsage: 1.65 MB (1734144)
- - PrepareTime: 25.022ms
- Instance 434987288e39baec:45101a3900000003 (host=itchil805.inteliquent.com:22000):(Total: 8s187ms, non-child: 2.039ms, % non-child: 0.02%)
- Last report received time: 2020-02-11 13:42:01.580
- Hdfs split stats (<volume id>:<# splits>/<split lengths>): 7:16/1.01 GB 0:20/248.59 MB 1:20/215.93 MB 3:20/137.59 MB 4:24/975.47 MB 2:22/647.65 MB 5:16/1.11 GB 6:20/318.79 MB
- Fragment Instance Lifecycle Event Timeline: 8s188ms
- - Prepare Finished: 4.322ms (4.322ms)
- - Open Finished: 308.287ms (303.965ms)
- - First Batch Produced: 6s826ms (6s517ms)
- - First Batch Sent: 6s826ms (22.520us)
- - ExecInternal Finished: 8s188ms (1s361ms)
- - MemoryUsage (500.000ms): 1.77 MB, 1.63 GB, 1.79 GB, 1.93 GB, 1.45 GB, 1.31 GB, 1.29 GB, 1.30 GB, 1.16 GB, 898.50 MB, 779.20 MB, 842.82 MB, 843.83 MB, 476.34 MB, 216.19 MB, 126.21 MB
- - ThreadUsage (500.000ms): 1, 33, 33, 32, 17, 12, 11, 11, 8, 5, 4, 4, 4, 2, 1, 1
- - AverageThreadTokens: 11.19
- - BloomFilterBytes: 0
- - ExchangeScanRatio: 0.07
- - PeakMemoryUsage: 1.96 GB (2108619856)
- - PeakReservation: 1.88 GB (2021654528)
- - PeakUsedReservation: 0
- - PerHostPeakMemUsage: 2.00 GB (2144411750)
- - RowsProduced: 1.71M (1713711)
- - TotalNetworkReceiveTime: 0.000ns
- - TotalNetworkSendTime: 12.850ms
- - TotalStorageWaitTime: 3s616ms
- - TotalThreadsInvoluntaryContextSwitches: 52 (52)
- - TotalThreadsTotalWallClockTime: 1m33s
- - TotalThreadsSysTime: 855.781ms
- - TotalThreadsUserTime: 22s082ms
- - TotalThreadsVoluntaryContextSwitches: 18.03K (18031)
- Buffer pool:
- - AllocTime: 0.000ns
- - CumulativeAllocationBytes: 0
- - CumulativeAllocations: 0 (0)
- - PeakReservation: 0
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 0
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - ReservationLimit: 0
- - SystemAllocTime: 0.000ns
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- Fragment Instance Lifecycle Timings:
- - ExecTime: 7s879ms
- - ExecTreeExecTime: 6s575ms
- - OpenTime: 303.964ms
- - ExecTreeOpenTime: 515.602us
- - PrepareTime: 3.454ms
- - ExecTreePrepareTime: 2.646ms
- KrpcDataStreamSender (dst_id=3):(Total: 1s303ms, non-child: 1s289ms, % non-child: 98.94%)
- ExecOption: Hash Partitioned Sender Codegen Enabled
- - BytesSent (500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 24.82 MB, 61.80 MB
- - NetworkThroughput: (Avg: 61.54 MB/sec ; Min: 8.90 MB/sec ; Max: 111.30 MB/sec ; Number of samples: 9417)
- - EosSent: 3 (3)
- - PeakMemoryUsage: 88.27 KB (90384)
- - RowsSent: 1.71M (1713711)
- - RpcFailure: 0 (0)
- - RpcRetry: 0 (0)
- - SerializeBatchTime: 708.086ms
- - TotalBytesSent: 99.06 MB (103866828)
- - UncompressedRowBatchSize: 223.26 MB (234100915)
- AGGREGATION_NODE (id=1):(Total: 6s578ms, non-child: 6s354ms, % non-child: 96.59%)
- Node Lifecycle Event Timeline: 8s187ms
- - Open Started: 307.762ms (307.762ms)
- - Open Finished: 308.275ms (512.640us)
- - First Batch Requested: 308.289ms (14.087us)
- - First Batch Returned: 6s826ms (6s517ms)
- - Last Batch Returned: 8s185ms (1s359ms)
- - Closed: 8s187ms (2.076ms)
- - PeakMemoryUsage: 289.87 MB (303951618)
- - RowsReturned: 1.71M (1713711)
- - RowsReturnedRate: 260.49 K/sec
- GroupingAggregator 0:
- ExecOption: Streaming Preaggregation, Codegen Enabled
- - BuildTime: 0.000ns
- - GetResultsTime: 57.212ms
- - HTResizeTime: 85.321ms
- - HashBuckets: 4.19M (4194304)
- - LargestPartitionPercent: 0 (0)
- - PartitionsCreated: 16 (16)
- - PeakMemoryUsage: 288.10 MB (302096512)
- - PeakReservation: 288.00 MB (301989888)
- - PeakUsedReservation: 0
- - ReductionFactorEstimate: 8.49
- - ReductionFactorThresholdToExpand: 2.00
- - RowsPassedThrough: 0 (0)
- - RowsReturned: 1.71M (1713711)
- - StreamingTime: 5s848ms
- Buffer pool:
- - AllocTime: 17.684ms
- - CumulativeAllocationBytes: 338.00 MB (354418688)
- - CumulativeAllocations: 153 (153)
- - PeakReservation: 288.00 MB (301989888)
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 288.00 MB (301989888)
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - SystemAllocTime: 17.524ms
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- HDFS_SCAN_NODE (id=0):(Total: 224.452ms, non-child: 224.452ms, % non-child: 100.00%)
- Hdfs split stats (<volume id>:<# splits>/<split lengths>): 7:16/1.01 GB 0:20/248.59 MB 1:20/215.93 MB 3:20/137.59 MB 4:24/975.47 MB 2:22/647.65 MB 5:16/1.11 GB 6:20/318.79 MB
- ExecOption: PARQUET Codegen Enabled, Codegen enabled: 158 out of 158
- Hdfs Read Thread Concurrency Bucket: 0:84.62% 1:7.692% 2:7.692% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0%
- File Formats: PARQUET/NONE:24 PARQUET/SNAPPY:134
- Node Lifecycle Event Timeline: 6s826ms
- - Open Started: 307.765ms (307.765ms)
- - Open Finished: 307.900ms (135.202us)
- - First Batch Requested: 308.291ms (390.971us)
- - First Batch Returned: 323.436ms (15.144ms)
- - Last Batch Returned: 6s825ms (6s502ms)
- - Closed: 6s826ms (360.938us)
- - BytesRead (500.000ms): 0, 678.14 MB, 920.50 MB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.10 GB, 1.13 GB, 1.20 GB, 1.24 GB, 1.29 GB, 1.30 GB, 1.30 GB
- - FooterProcessingTime: (Avg: 8.579ms ; Min: 720.806us ; Max: 71.456ms ; Number of samples: 158)
- - InitialRangeActualReservation: (Avg: 32.00 MB (33554432) ; Min: 32.00 MB (33554432) ; Max: 32.00 MB (33554432) ; Number of samples: 158)
- - InitialRangeIdealReservation: (Avg: 74.18 KB (75957) ; Min: 8.00 KB (8192) ; Max: 128.00 KB (131072) ; Number of samples: 158)
- - ParquetCompressedBytesReadPerColumn: (Avg: 164.67 MB (172668959) ; Min: 39.91 MB (41846921) ; Max: 249.59 MB (261709972) ; Number of samples: 8)
- - ParquetCompressedPageSize: (Avg: 101.21 KB (103635) ; Min: 10.00 B (10) ; Max: 192.32 KB (196939) ; Number of samples: 13329)
- - ParquetRowGroupActualReservation: (Avg: 43.15 MB (45247185) ; Min: 32.00 MB (33554432) ; Max: 184.00 MB (192937984) ; Number of samples: 139)
- - ParquetRowGroupIdealReservation: (Avg: 10.13 MB (10622902) ; Min: 64.00 KB (65536) ; Max: 184.00 MB (192937984) ; Number of samples: 139)
- - ParquetUncompressedBytesReadPerColumn: (Avg: 166.75 MB (174854074) ; Min: 39.89 MB (41832210) ; Max: 252.05 MB (264295961) ; Number of samples: 8)
- - ParquetUncompressedPageSize: (Avg: 102.49 KB (104946) ; Min: 8.00 B (8) ; Max: 192.28 KB (196890) ; Number of samples: 13329)
- - AverageHdfsReadThreadConcurrency: 0.23
- - AverageScannerThreadConcurrency: 12.54
- - BytesRead: 1.30 GB (1395709086)
- - BytesReadDataNodeCache: 0
- - BytesReadLocal: 1.30 GB (1395709086)
- - BytesReadRemoteUnexpected: 0
- - BytesReadShortCircuit: 1.30 GB (1395709086)
- - CachedFileHandlesHitCount: 1.08K (1080)
- - CachedFileHandlesMissCount: 113 (113)
- - CollectionItemsRead: 0 (0)
- - DataCacheHitBytes: 0
- - DataCacheHitCount: 0 (0)
- - DataCacheMissBytes: 0
- - DataCacheMissCount: 0 (0)
- - DataCachePartialHitCount: 0 (0)
- - DecompressionTime: 954.463ms
- - MaterializeTupleTime(*): 13s068ms
- - MaxCompressedTextFileLength: 0
- - NumColumns: 8 (8)
- - NumDictFilteredRowGroups: 25 (25)
- - NumDisksAccessed: 9 (9)
- - NumRowGroups: 163 (163)
- - NumScannerThreadMemUnavailable: 0 (0)
- - NumScannerThreadReservationsDenied: 0 (0)
- - NumScannerThreadsStarted: 32 (32)
- - NumScannersWithNoReads: 0 (0)
- - NumStatsFilteredRowGroups: 24 (24)
- - PeakMemoryUsage: 1.85 GB (1988328687)
- - PeakScannerThreadConcurrency: 32 (32)
- - PerReadThreadRawHdfsThroughput: 730.42 MB/sec
- - RemoteScanRanges: 0 (0)
- - RowBatchBytesEnqueued: 12.20 GB (13101039531)
- - RowBatchQueueGetWaitTime: 133.627ms
- - RowBatchQueuePeakMemoryUsage: 39.19 MB (41094586)
- - RowBatchQueuePutWaitTime: 1m6s
- - RowBatchesEnqueued: 16.82K (16815)
- - RowsRead: 165.82M (165821581)
- - RowsReturned: 16.83M (16830795)
- - RowsReturnedRate: 74.99 M/sec
- - ScanRangesComplete: 158 (158)
- - ScannerIoWaitTime: 3s616ms
- - ScannerThreadWorklessLoops: 0 (0)
- - ScannerThreadsInvoluntaryContextSwitches: 44 (44)
- - ScannerThreadsTotalWallClockTime: 1m25s
- - ScannerThreadsSysTime: 299.232ms
- - ScannerThreadsUserTime: 14s601ms
- - ScannerThreadsVoluntaryContextSwitches: 17.21K (17207)
- - TotalRawHdfsOpenFileTime(*): 254.492ms
- - TotalRawHdfsReadTime(*): 1s822ms
- - TotalReadThroughput: 189.87 MB/sec
- Buffer pool:
- - AllocTime: 43.390ms
- - CumulativeAllocationBytes: 1.30 GB (1395785728)
- - CumulativeAllocations: 1.16K (1159)
- - PeakReservation: 1.77 GB (1902116864)
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 1.21 GB (1299546112)
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - SystemAllocTime: 41.672ms
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- CodeGen:(Total: 303.288ms, non-child: 303.288ms, % non-child: 100.00%)
- - CodegenInvoluntaryContextSwitches: 1 (1)
- - CodegenTotalWallClockTime: 303.275ms
- - CodegenSysTime: 1.679ms
- - CodegenUserTime: 302.317ms
- - CodegenVoluntaryContextSwitches: 0 (0)
- - CompileTime: 86.690ms
- - IrGenerationTime: 7.270ms
- - LoadTime: 0.000ns
- - ModuleBitcodeSize: 2.28 MB (2393476)
- - NumFunctions: 102 (102)
- - NumInstructions: 3.39K (3387)
- - OptimizationTime: 183.189ms
- - PeakMemoryUsage: 1.65 MB (1734144)
- - PrepareTime: 25.043ms
- Instance 434987288e39baec:45101a3900000001 (host=itchil806.inteliquent.com:22000):(Total: 8s111ms, non-child: 1.737ms, % non-child: 0.02%)
- Last report received time: 2020-02-11 13:42:01.579
- Hdfs split stats (<volume id>:<# splits>/<split lengths>): 6:11/18.95 MB 2:8/216.70 MB 1:15/1.03 GB 7:14/5.87 MB 5:16/1.32 GB 4:11/115.64 MB 0:17/1.03 GB 3:19/901.70 MB
- Fragment Instance Lifecycle Event Timeline: 8s112ms
- - Prepare Finished: 3.483ms (3.483ms)
- - Open Finished: 319.039ms (315.556ms)
- - First Batch Produced: 7s014ms (6s695ms)
- - First Batch Sent: 7s014ms (3.142us)
- - ExecInternal Finished: 8s112ms (1s097ms)
- - MemoryUsage (500.000ms): 1.77 MB, 1.81 GB, 1.45 GB, 1.34 GB, 1.21 GB, 1.22 GB, 1.12 GB, 1.15 GB, 1.05 GB, 1.05 GB, 919.69 MB, 952.05 MB, 953.66 MB, 781.52 MB, 182.19 MB, 84.18 MB
- - ThreadUsage (500.000ms): 1, 33, 18, 14, 10, 9, 7, 7, 6, 6, 5, 5, 5, 4, 1, 1
- - AverageThreadTokens: 8.25
- - BloomFilterBytes: 0
- - ExchangeScanRatio: 0.06
- - PeakMemoryUsage: 1.88 GB (2023064990)
- - PeakReservation: 1.82 GB (1955594240)
- - PeakUsedReservation: 0
- - PerHostPeakMemUsage: 1.92 GB (2058856884)
- - RowsProduced: 1.49M (1494645)
- - TotalNetworkReceiveTime: 0.000ns
- - TotalNetworkSendTime: 7.267ms
- - TotalStorageWaitTime: 8s048ms
- - TotalThreadsInvoluntaryContextSwitches: 51 (51)
- - TotalThreadsTotalWallClockTime: 1m9s
- - TotalThreadsSysTime: 744.102ms
- - TotalThreadsUserTime: 21s841ms
- - TotalThreadsVoluntaryContextSwitches: 17.88K (17877)
- Buffer pool:
- - AllocTime: 0.000ns
- - CumulativeAllocationBytes: 0
- - CumulativeAllocations: 0 (0)
- - PeakReservation: 0
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 0
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - ReservationLimit: 0
- - SystemAllocTime: 0.000ns
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- Fragment Instance Lifecycle Timings:
- - ExecTime: 7s793ms
- - ExecTreeExecTime: 6s727ms
- - OpenTime: 315.556ms
- - ExecTreeOpenTime: 500.264us
- - PrepareTime: 2.683ms
- - ExecTreePrepareTime: 2.016ms
- KrpcDataStreamSender (dst_id=3):(Total: 1s065ms, non-child: 1s057ms, % non-child: 99.24%)
- ExecOption: Hash Partitioned Sender Codegen Enabled
- - BytesSent (500.000ms): 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 16.27 MB, 55.58 MB
- - NetworkThroughput: (Avg: 80.38 MB/sec ; Min: 12.08 MB/sec ; Max: 156.91 MB/sec ; Number of samples: 8214)
- - EosSent: 3 (3)
- - PeakMemoryUsage: 88.27 KB (90384)
- - RowsSent: 1.49M (1494645)
- - RpcFailure: 0 (0)
- - RpcRetry: 0 (0)
- - SerializeBatchTime: 574.594ms
- - TotalBytesSent: 86.04 MB (90214916)
- - UncompressedRowBatchSize: 194.79 MB (204256533)
- AGGREGATION_NODE (id=1):(Total: 6s729ms, non-child: 6s587ms, % non-child: 97.90%)
- Node Lifecycle Event Timeline: 8s112ms
- - Open Started: 318.529ms (318.529ms)
- - Open Finished: 319.027ms (497.671us)
- - First Batch Requested: 319.040ms (13.165us)
- - First Batch Returned: 7s014ms (6s695ms)
- - Last Batch Returned: 8s110ms (1s095ms)
- - Closed: 8s112ms (1.848ms)
- - PeakMemoryUsage: 225.87 MB (236841842)
- - RowsReturned: 1.49M (1494645)
- - RowsReturnedRate: 222.11 K/sec
- GroupingAggregator 0:
- ExecOption: Streaming Preaggregation, Codegen Enabled
- - BuildTime: 0.000ns
- - GetResultsTime: 30.937ms
- - HTResizeTime: 36.710ms
- - HashBuckets: 2.10M (2097152)
- - LargestPartitionPercent: 0 (0)
- - PartitionsCreated: 16 (16)
- - PeakMemoryUsage: 224.10 MB (234987648)
- - PeakReservation: 224.00 MB (234881024)
- - PeakUsedReservation: 0
- - ReductionFactorEstimate: 6.25
- - ReductionFactorThresholdToExpand: 2.00
- - RowsPassedThrough: 0 (0)
- - RowsReturned: 1.49M (1494645)
- - StreamingTime: 6s048ms
- Buffer pool:
- - AllocTime: 8.849ms
- - CumulativeAllocationBytes: 242.00 MB (253755392)
- - CumulativeAllocations: 121 (121)
- - PeakReservation: 224.00 MB (234881024)
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 224.00 MB (234881024)
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - SystemAllocTime: 8.721ms
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- HDFS_SCAN_NODE (id=0):(Total: 141.643ms, non-child: 141.643ms, % non-child: 100.00%)
- Hdfs split stats (<volume id>:<# splits>/<split lengths>): 6:11/18.95 MB 2:8/216.70 MB 1:15/1.03 GB 7:14/5.87 MB 5:16/1.32 GB 4:11/115.64 MB 0:17/1.03 GB 3:19/901.70 MB
- ExecOption: PARQUET Codegen Enabled, Codegen enabled: 111 out of 111
- Hdfs Read Thread Concurrency Bucket: 0:76.92% 1:7.692% 2:7.692% 3:0% 4:7.692% 5:0% 6:0% 7:0% 8:0% 9:0% 10:0% 11:0% 12:0% 13:0% 14:0%
- File Formats: PARQUET/NONE:17 PARQUET/SNAPPY:94
- Node Lifecycle Event Timeline: 7s014ms
- - Open Started: 318.532ms (318.532ms)
- - Open Finished: 318.631ms (99.380us)
- - First Batch Requested: 319.042ms (411.175us)
- - First Batch Returned: 344.258ms (25.216ms)
- - Last Batch Returned: 7s014ms (6s669ms)
- - Closed: 7s014ms (512.941us)
- - BytesRead (500.000ms): 0, 970.24 MB, 1.05 GB, 1.05 GB, 1.05 GB, 1.05 GB, 1.07 GB, 1.19 GB, 1.21 GB, 1.26 GB, 1.29 GB, 1.30 GB, 1.30 GB, 1.30 GB
- - FooterProcessingTime: (Avg: 26.418ms ; Min: 1.911ms ; Max: 127.681ms ; Number of samples: 111)
- - InitialRangeActualReservation: (Avg: 32.00 MB (33554432) ; Min: 32.00 MB (33554432) ; Max: 32.00 MB (33554432) ; Number of samples: 111)
- - InitialRangeIdealReservation: (Avg: 66.45 KB (68045) ; Min: 8.00 KB (8192) ; Max: 128.00 KB (131072) ; Number of samples: 111)
- - ParquetCompressedBytesReadPerColumn: (Avg: 165.88 MB (173940457) ; Min: 39.75 MB (41685700) ; Max: 251.41 MB (263620723) ; Number of samples: 8)
- - ParquetCompressedPageSize: (Avg: 103.61 KB (106101) ; Min: 10.00 B (10) ; Max: 192.31 KB (196930) ; Number of samples: 13115)
- - ParquetRowGroupActualReservation: (Avg: 40.50 MB (42467328) ; Min: 32.00 MB (33554432) ; Max: 176.00 MB (184549376) ; Number of samples: 94)
- - ParquetRowGroupIdealReservation: (Avg: 13.32 MB (13970845) ; Min: 64.00 KB (65536) ; Max: 176.00 MB (184549376) ; Number of samples: 94)
- - ParquetUncompressedBytesReadPerColumn: (Avg: 166.75 MB (174852906) ; Min: 39.74 MB (41671154) ; Max: 252.39 MB (264652421) ; Number of samples: 8)
- - ParquetUncompressedPageSize: (Avg: 104.16 KB (106658) ; Min: 8.00 B (8) ; Max: 192.28 KB (196890) ; Number of samples: 13115)
- - AverageHdfsReadThreadConcurrency: 0.54
- - AverageScannerThreadConcurrency: 8.92
- - BytesRead: 1.30 GB (1400356213)
- - BytesReadDataNodeCache: 0
- - BytesReadLocal: 1.30 GB (1400356213)
- - BytesReadRemoteUnexpected: 0
- - BytesReadShortCircuit: 1.30 GB (1400356213)
- - CachedFileHandlesHitCount: 682 (682)
- - CachedFileHandlesMissCount: 107 (107)
- - CollectionItemsRead: 0 (0)
- - DataCacheHitBytes: 0
- - DataCacheHitCount: 0 (0)
- - DataCacheMissBytes: 0
- - DataCacheMissCount: 0 (0)
- - DataCachePartialHitCount: 0 (0)
- - DecompressionTime: 761.586ms
- - MaterializeTupleTime(*): 12s961ms
- - MaxCompressedTextFileLength: 0
- - NumColumns: 8 (8)
- - NumDictFilteredRowGroups: 27 (27)
- - NumDisksAccessed: 8 (8)
- - NumRowGroups: 111 (111)
- - NumScannerThreadMemUnavailable: 0 (0)
- - NumScannerThreadReservationsDenied: 0 (0)
- - NumScannerThreadsStarted: 32 (32)
- - NumScannersWithNoReads: 0 (0)
- - NumStatsFilteredRowGroups: 17 (17)
- - PeakMemoryUsage: 1.84 GB (1978185673)
- - PeakScannerThreadConcurrency: 32 (32)
- - PerReadThreadRawHdfsThroughput: 392.34 MB/sec
- - RemoteScanRanges: 0 (0)
- - RowBatchBytesEnqueued: 13.18 GB (14153508115)
- - RowBatchQueueGetWaitTime: 52.391ms
- - RowBatchQueuePeakMemoryUsage: 33.81 MB (35454022)
- - RowBatchQueuePutWaitTime: 39s316ms
- - RowBatchesEnqueued: 16.75K (16750)
- - RowsRead: 165.14M (165137619)
- - RowsReturned: 16.88M (16881754)
- - RowsReturnedRate: 119.18 M/sec
- - ScanRangesComplete: 111 (111)
- - ScannerIoWaitTime: 8s048ms
- - ScannerThreadWorklessLoops: 0 (0)
- - ScannerThreadsInvoluntaryContextSwitches: 44 (44)
- - ScannerThreadsTotalWallClockTime: 1m1s
- - ScannerThreadsSysTime: 245.320ms
- - ScannerThreadsUserTime: 14s292ms
- - ScannerThreadsVoluntaryContextSwitches: 17.59K (17590)
- - TotalRawHdfsOpenFileTime(*): 248.123ms
- - TotalRawHdfsReadTime(*): 3s403ms
- - TotalReadThroughput: 190.78 MB/sec
- Buffer pool:
- - AllocTime: 49.813ms
- - CumulativeAllocationBytes: 1.17 GB (1257381888)
- - CumulativeAllocations: 746 (746)
- - PeakReservation: 1.78 GB (1911554048)
- - PeakUnpinnedBytes: 0
- - PeakUsedReservation: 1.14 GB (1224146944)
- - ReadIoBytes: 0
- - ReadIoOps: 0 (0)
- - ReadIoWaitTime: 0.000ns
- - SystemAllocTime: 48.568ms
- - WriteIoBytes: 0
- - WriteIoOps: 0 (0)
- - WriteIoWaitTime: 0.000ns
- CodeGen:(Total: 314.933ms, non-child: 314.933ms, % non-child: 100.00%)
- - CodegenInvoluntaryContextSwitches: 2 (2)
- - CodegenTotalWallClockTime: 314.922ms
- - CodegenSysTime: 7.979ms
- - CodegenUserTime: 307.211ms
- - CodegenVoluntaryContextSwitches: 1 (1)
- - CompileTime: 86.789ms
- - IrGenerationTime: 6.638ms
- - LoadTime: 0.000ns
- - ModuleBitcodeSize: 2.28 MB (2393476)
- - NumFunctions: 102 (102)
- - NumInstructions: 3.39K (3387)
- - OptimizationTime: 195.328ms
- - PeakMemoryUsage: 1.65 MB (1734144)
- - PrepareTime: 25.297ms
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement