Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- Query (id=af4d1f47c62b76b4:d88072cab3fa4197):
- - AsyncTotalTime: 0ns
- Summary:
- Session ID: f14532f5f56fd776:64e25774df90d9b4
- Session Type: BEESWAX
- Start Time: 2014-05-30 15:40:13.268075000
- End Time:
- Query Type: DML
- Query State: CREATED
- Query Status: OK
- Impala Version: impalad version 1.3.1-cdh4 RELEASE (build 907481bf45b248a7bb3bb077d54831a71f484e5f)
- User: root
- Network Address: ::ffff:10.20.11.117:59244
- Default Db: db
- Sql Statement: insert INTO TABLE t2 PARTITION(dt) SELECT * FROM t WHERE dt='2014-05-27-00'
- Plan:
- ----------------
- Estimated Per-Host Requirements: Memory=3.00GB VCores=1
- WARNING: The following tables are missing relevant table and/or column statistics.
- db.t
- F00:PLAN FRAGMENT [PARTITION=RANDOM]
- WRITE TO HDFS [db.t2, OVERWRITE=false, PARTITION-KEYS=(t.dt)]
- | partitions=1
- | hosts=3 per-host-mem=6.01KB
- |
- 00:SCAN HDFS [db.t, PARTITION=RANDOM]
- partitions=1/1 size=9.91GB
- table stats: 0 rows total
- columns missing stats: col1, col2, col3, col4, col5, col6, col7, col8, col9, col10, col11, col12, col13, col14, col15, col16, col17, col18, col19, col20, col21, col22, col23, col24, col25, col26, col27, col28, col29, col30, col31, col32, col33, col34, col35, col36, col37, col38, col39, col40, col41, col42, col43, col44, col45, col46, col47, col48, col49, col50, col51, col52, col53, col54, col55, col56, col57, col58, col59, col60, col61, col62, col63, col64, col65, col66, col67, col68, col69, col70, col71, col72, col73, col74, col75, col76, col77, col78, col79, col80, col81, col82, col83, col84, col85, col86, col87, col88, col89, col90, col91, col92, col93, col94, col95, col96, col97, col98, col99, col100, col101, col102, col103, col104, col105, col106, col107, col108, col109, col110, col111, col112, col113, col114, col115, col116, col117, col118, col119, col120, col121, col122, col123, col124, col125, col126, col127, col128, col129, col130, col131, col132, col133, col134, col135, col136, col137, col138, col139, col140, col141, col142, col143, col144, col145, col146, col147, col148, col149, col150, col151, col152, col153, col154, col155, col156, col157, col158, col159, col160, col161, col162, col163, col164, col165, col166, col167, col168, col169, col170, col171, col172, col173, col174, col175, col176, col177, col178, col179, col180, col181, col182, col183, col184, col185, col186, col187, col188, col189, col190, col191, col192, col193, col194, col195, col196, col197, col198, col199, col200, col201, col202, col203, col204, col205, col206, col207, col208, col209, col210, col211, col212, col213, col214, col215, col216, col217, col218, col219, col220, col221, col222, col223, col224, col225, col226, col227, col228, col229, col230, col231, col232, col233, col234, col235, col236, col237, col238, col239, col240, col241, col242, col243, col244, col245, col246, col247, col248, col249, col250, col251, col252, col253, col254, col255, col256, col257, col258, col259, col260, col261, col262, col263, col264, col265, col266, col267, col268, col269, col270, col271, col272, col273, col274, col275, col276, col277, col278, col279, col280, col281, col282, col283, col284, col285, col286, col287, col288, col289, col290, col291, col292, col293, col294, col295, col296, col297, col298, col299, col300, col301, col302, col303, col304, col305, col306, col307, col308, col309, col310, col311, col312, col313, col314, col315, col316, col317, col318, col319, col320, col321, col322, col323, col324, col325, col326, col327, col328, col329, col330, col331, col332, col333, col334, col335, col336, col337, col338, col339, col340, col341, col342, col343, col344, col345, col346, col347, col348, col349, col350, col351, col352, col353, col354, col355, col356, col357, col358, col359, col360, col361, col362, col363, col364, col365, col366, col367, col368, col369, col370, col371, col372, col373, col374, col375, col376, col377, col378, col379, col380, col381, col382, col383, col384, col385, col386, col387, col388, col389, col390, col391, col392, col393, col394, col395, col396, col397, col398, col399, col400, col401, col402, col403, col404, col405, col406, col407, col408, col409
- hosts=3 per-host-mem=3.00GB
- tuple-ids=0 row-size=6.01KB cardinality=0
- ----------------
- Estimated Per-Host Mem: 3221231622
- Estimated Per-Host VCores: 1
- Tables Missing Stats: db.t
- Request Pool: default-pool
- Query Timeline: 45m50s
- - Start execution: 3.223ms (3.223ms)
- - Planning finished: 59.459ms (56.236ms)
- - AsyncTotalTime: 0ns
- ImpalaServer:
- - AsyncTotalTime: 0ns
- - ClientFetchWaitTimer: 0ns
- - RowMaterializationTimer: 0ns
- Execution Profile af4d1f47c62b76b4:d88072cab3fa4197:(Total: 172.575ms, non-child: 0ns, % non-child: 0.00%)
- - AsyncTotalTime: 0ns
- - FinalizationTimer: 0ns
- Averaged Fragment 0:(Total: 15m50s, non-child: 0ns, % non-child: 0.00%)
- split sizes: min: 3.30 GB, max: 3.30 GB, avg: 3.30 GB, stddev: 579.09 KB
- - AverageThreadTokens: 8.94
- - PeakMemoryUsage: 1.41 GB
- - PrepareTime: 904.889us
- - RowsProduced: 11.76M (11759217)
- - TotalCpuTime: 2h14m
- - TotalNetworkReceiveTime: 0ns
- - TotalNetworkSendTime: 0ns
- - TotalStorageWaitTime: 1m58s
- CodeGen:(Total: 119.852ms, non-child: 119.852ms, % non-child: 100.00%)
- - CodegenTime: 0ns
- - CompileTime: 107.582ms
- - LoadTime: 11.202ms
- - ModuleFileSize: 105.51 KB
- HdfsTableSink:(Total: 15m49s, non-child: 15m49s, % non-child: 100.00%)
- - BytesWritten: 943.56 MB
- - EncodeTimer: 15m16s
- - FinalizePartitionFileTimer: 28s942ms
- - HdfsWriteTimer: 19s193ms
- - PeakMemoryUsage: 490.60 MB
- - RowsInserted: 11.39M (11386118)
- - TmpFileCreateTimer: 52.937ms
- HDFS_SCAN_NODE (id=0):(Total: 1s488ms, non-child: 1s488ms, % non-child: 100.00%)
- - AverageHdfsReadThreadConcurrency: 0.00
- - AverageScannerThreadConcurrency: 7.95
- - BytesRead: 2.29 GB
- - BytesReadDataNodeCache: 0
- - BytesReadLocal: 2.21 GB
- - BytesReadShortCircuit: 2.21 GB
- - DecompressionTime: 0ns
- - NumColumns: 0
- - NumDisksAccessed: 4
- - NumScannerThreadsStarted: 8
- - PeakMemoryUsage: 958.72 MB
- - PerReadThreadRawHdfsThroughput: 1.53 GB/sec
- - RowsRead: 11.78M (11778854)
- - RowsReturned: 11.76M (11759217)
- - RowsReturnedRate: 7.77 M/sec
- - ScanRangesComplete: 98
- - ScannerThreadsInvoluntaryContextSwitches: 30.67K (30666)
- - ScannerThreadsTotalWallClockTime: 2h
- - MaterializeTupleTime(*): 1h57m
- - ScannerThreadsSysTime: 453.262ms
- - ScannerThreadsUserTime: 3m21s
- - ScannerThreadsVoluntaryContextSwitches: 88.60K (88603)
- - TotalRawHdfsReadTime(*): 1s409ms
- - TotalReadThroughput: 1.68 MB/sec
- Fragment 0:
- - AsyncTotalTime: 0ns
- Instance af4d1f47c62b76b4:d88072cab3fa419a (host=SOMEHOST3:22000):(Total: 23m36s, non-child: 0ns, % non-child: 0.00%)
- Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:13/307.42 MB 1:13/325.22 MB 2:11/262.74 MB 3:10/229.46 MB 4:30/716.74 MB 5:41/946.83 MB 6:30/593.54 MB
- MemoryUsage(32s000ms): 868.04 MB, 987.55 MB, 1.01 GB, 1.04 GB, 1.15 GB, 1.21 GB, 1.25 GB, 1.21 GB, 1.27 GB, 1.50 GB, 1.47 GB, 1.49 GB, 1.47 GB, 1.49 GB, 1.49 GB, 1.46 GB, 1.47 GB, 1.48 GB, 1.48 GB, 1.50 GB, 1.46 GB, 1.51 GB, 1.48 GB, 1.48 GB, 1.48 GB, 1.45 GB, 1.49 GB, 1.49 GB, 1.47 GB, 1.50 GB, 1.45 GB, 1.50 GB, 1.51 GB, 1.49 GB, 1.51 GB, 1.44 GB, 1.48 GB, 1.48 GB, 1.48 GB, 1.49 GB, 1.46 GB, 1.48 GB, 1.50 GB, 1.39 GB
- ThreadUsage(32s000ms): 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 6
- - AsyncTotalTime: 0ns
- - AverageThreadTokens: 8.90
- - PeakMemoryUsage: 1.70 GB
- - PrepareTime: 761.852us
- - RowsProduced: 17.10M (17098222)
- - TotalCpuTime: 3h27m
- - TotalNetworkReceiveTime: 0ns
- - TotalNetworkSendTime: 0ns
- - TotalStorageWaitTime: 2m51s
- CodeGen:(Total: 105.281ms, non-child: 105.281ms, % non-child: 100.00%)
- - AsyncTotalTime: 0ns
- - CodegenTime: 0ns
- - CompileTime: 94.999ms
- - LoadTime: 9.413ms
- - ModuleFileSize: 105.51 KB
- HdfsTableSink:(Total: 23m34s, non-child: 23m34s, % non-child: 100.00%)
- - AsyncTotalTime: 0ns
- - BytesWritten: 1.38 GB
- - EncodeTimer: 22m46s
- - FinalizePartitionFileTimer: 28s036ms
- - HdfsWriteTimer: 27s852ms
- - PeakMemoryUsage: 639.93 MB
- - RowsInserted: 17.10M (17098222)
- - TmpFileCreateTimer: 119.715ms
- HDFS_SCAN_NODE (id=0):(Total: 1s983ms, non-child: 1s983ms, % non-child: 100.00%)
- Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:13/307.42 MB 1:13/325.22 MB 2:11/262.74 MB 3:10/229.46 MB 4:30/716.74 MB 5:41/946.83 MB 6:30/593.54 MB
- Hdfs Read Thread Concurrency Bucket: 0:99.93% 1:0.07077% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0%
- File Formats: PARQUET/NONE:60532
- ExecOption: Codegen enabled: 0 out of 148
- BytesRead(32s000ms): 115.89 MB, 152.97 MB, 223.85 MB, 323.57 MB, 348.70 MB, 422.45 MB, 523.20 MB, 552.07 MB, 624.55 MB, 719.87 MB, 769.28 MB, 861.50 MB, 962.17 MB, 1010.68 MB, 1.10 GB, 1.15 GB, 1.22 GB, 1.34 GB, 1.37 GB, 1.49 GB, 1.54 GB, 1.62 GB, 1.74 GB, 1.77 GB, 1.87 GB, 1.94 GB, 2.00 GB, 2.12 GB, 2.16 GB, 2.26 GB, 2.34 GB, 2.39 GB, 2.50 GB, 2.55 GB, 2.64 GB, 2.73 GB, 2.79 GB, 2.88 GB, 2.95 GB, 3.02 GB, 3.11 GB, 3.18 GB, 3.26 GB, 3.30 GB
- - AsyncTotalTime: 0ns
- - AverageHdfsReadThreadConcurrency: 0.00
- - AverageScannerThreadConcurrency: 7.92
- - BytesRead: 3.32 GB
- - BytesReadDataNodeCache: 0
- - BytesReadLocal: 3.32 GB
- - BytesReadShortCircuit: 3.32 GB
- - DecompressionTime: 0ns
- - NumColumns: 0
- - NumDisksAccessed: 7
- - NumScannerThreadsStarted: 8
- - PeakMemoryUsage: 1.07 GB
- - PerReadThreadRawHdfsThroughput: 1.72 GB/sec
- - RowsRead: 17.10M (17098222)
- - RowsReturned: 17.10M (17098222)
- - RowsReturnedRate: 8.62 M/sec
- - ScanRangesComplete: 148
- - ScannerThreadsInvoluntaryContextSwitches: 45.08K (45083)
- - ScannerThreadsTotalWallClockTime: 3h6m
- - MaterializeTupleTime(*): 3h
- - ScannerThreadsSysTime: 673.893ms
- - ScannerThreadsUserTime: 5m
- - ScannerThreadsVoluntaryContextSwitches: 133.15K (133145)
- - TotalRawHdfsReadTime(*): 1s925ms
- - TotalReadThroughput: 2.41 MB/sec
- Instance af4d1f47c62b76b4:d88072cab3fa4198 (host=SOMEHOST2:22000):(Total: 1m32s, non-child: 0ns, % non-child: 0.00%)
- Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:31/709.29 MB 1:23/502.28 MB 2:34/797.81 MB 3:26/574.11 MB 4:34/799.07 MB
- MemoryUsage(1m4s): 769.72 MB, 838.56 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB, 830.72 MB
- ThreadUsage(1m4s): 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9
- - AsyncTotalTime: 0ns
- - AverageThreadTokens: 9.03
- - PeakMemoryUsage: 930.11 MB
- - PrepareTime: 1.195ms
- - RowsProduced: 1.12M (1119297)
- - TotalCpuTime: 0ns
- - TotalNetworkReceiveTime: 0ns
- - TotalNetworkSendTime: 0ns
- - TotalStorageWaitTime: 18s984ms
- CodeGen:(Total: 163.407ms, non-child: 163.407ms, % non-child: 100.00%)
- - AsyncTotalTime: 0ns
- - CodegenTime: 0ns
- - CompileTime: 149.239ms
- - LoadTime: 12.719ms
- - ModuleFileSize: 105.51 KB
- HdfsTableSink:(Total: 1m31s, non-child: 1m31s, % non-child: 100.00%)
- - AsyncTotalTime: 0ns
- - BytesWritten: 4.00 B
- - EncodeTimer: 1m29s
- - HdfsWriteTimer: 0ns
- - PeakMemoryUsage: 191.93 MB
- - RowsInserted: 0
- - TmpFileCreateTimer: 7.651ms
- HDFS_SCAN_NODE (id=0):(Total: 1s245ms, non-child: 1s245ms, % non-child: 100.00%)
- Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:31/709.29 MB 1:23/502.28 MB 2:34/797.81 MB 3:26/574.11 MB 4:34/799.07 MB
- BytesRead(1m4s): 142.42 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB, 233.01 MB
- - AsyncTotalTime: 0ns
- - AverageHdfsReadThreadConcurrency: 0.00
- - AverageScannerThreadConcurrency: 8.03
- - BytesRead: 233.01 MB
- - BytesReadDataNodeCache: 0
- - BytesReadLocal: 0
- - BytesReadShortCircuit: 0
- - DecompressionTime: 0ns
- - NumColumns: 0
- - NumDisksAccessed: 0
- - NumScannerThreadsStarted: 9
- - PeakMemoryUsage: 760.21 MB
- - PerReadThreadRawHdfsThroughput: 1.30 GB/sec
- - RowsRead: 1.18M (1178208)
- - RowsReturned: 1.12M (1119297)
- - RowsReturnedRate: 898.43 K/sec
- - ScanRangesComplete: 1
- - ScannerThreadsInvoluntaryContextSwitches: 0
- - ScannerThreadsTotalWallClockTime: 0ns
- - MaterializeTupleTime(*): 1m29s
- - ScannerThreadsSysTime: 0ns
- - ScannerThreadsUserTime: 0ns
- - ScannerThreadsVoluntaryContextSwitches: 0
- - TotalRawHdfsReadTime(*): 175.602ms
- - TotalReadThroughput: 86.81 KB/sec
- Instance af4d1f47c62b76b4:d88072cab3fa4199 (host=SOMEHOST4:22000):(Total: 22m22s, non-child: 0ns, % non-child: 0.00%)
- Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:14/285.90 MB 1:12/296.74 MB 2:14/336.06 MB 3:10/226.44 MB 4:34/808.60 MB 5:32/691.37 MB 6:31/736.07 MB
- MemoryUsage(32s000ms): 910.23 MB, 1013.02 MB, 1.05 GB, 1.18 GB, 1.24 GB, 1.22 GB, 1.41 GB, 1.45 GB, 1.46 GB, 1.50 GB, 1.48 GB, 1.49 GB, 1.46 GB, 1.48 GB, 1.50 GB, 1.47 GB, 1.50 GB, 1.45 GB, 1.49 GB, 1.53 GB, 1.48 GB, 1.52 GB, 1.47 GB, 1.49 GB, 1.51 GB, 1.46 GB, 1.49 GB, 1.50 GB, 1.49 GB, 1.51 GB, 1.48 GB, 1.50 GB, 1.51 GB, 1.48 GB, 1.52 GB, 1.49 GB, 1.50 GB, 1.52 GB, 1.46 GB, 1.46 GB, 1.41 GB
- ThreadUsage(32s000ms): 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 8
- - AsyncTotalTime: 0ns
- - AverageThreadTokens: 8.87
- - PeakMemoryUsage: 1.62 GB
- - PrepareTime: 757.359us
- - RowsProduced: 17.06M (17060133)
- - TotalCpuTime: 3h15m
- - TotalNetworkReceiveTime: 0ns
- - TotalNetworkSendTime: 0ns
- - TotalStorageWaitTime: 2m45s
- CodeGen:(Total: 90.869ms, non-child: 90.869ms, % non-child: 100.00%)
- - AsyncTotalTime: 0ns
- - CodegenTime: 0ns
- - CompileTime: 78.508ms
- - LoadTime: 11.475ms
- - ModuleFileSize: 105.51 KB
- HdfsTableSink:(Total: 22m21s, non-child: 22m21s, % non-child: 100.00%)
- - AsyncTotalTime: 0ns
- - BytesWritten: 1.38 GB
- - EncodeTimer: 21m32s
- - FinalizePartitionFileTimer: 29s849ms
- - HdfsWriteTimer: 29s726ms
- - PeakMemoryUsage: 639.93 MB
- - RowsInserted: 17.06M (17060133)
- - TmpFileCreateTimer: 31.444ms
- HDFS_SCAN_NODE (id=0):(Total: 1s236ms, non-child: 1s236ms, % non-child: 100.00%)
- Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:14/285.90 MB 1:12/296.74 MB 2:14/336.06 MB 3:10/226.44 MB 4:34/808.60 MB 5:32/691.37 MB 6:31/736.07 MB
- Hdfs Read Thread Concurrency Bucket: 0:99.63% 1:0.3736% 2:0% 3:0% 4:0% 5:0% 6:0% 7:0% 8:0%
- File Formats: PARQUET/NONE:60123
- ExecOption: Codegen enabled: 0 out of 147
- BytesRead(32s000ms): 121.30 MB, 199.02 MB, 317.71 MB, 379.78 MB, 506.43 MB, 574.23 MB, 681.94 MB, 757.79 MB, 821.30 MB, 949.04 MB, 983.30 MB, 1.06 GB, 1.14 GB, 1.19 GB, 1.31 GB, 1.35 GB, 1.44 GB, 1.54 GB, 1.58 GB, 1.70 GB, 1.75 GB, 1.83 GB, 1.93 GB, 1.97 GB, 2.07 GB, 2.14 GB, 2.21 GB, 2.31 GB, 2.37 GB, 2.44 GB, 2.54 GB, 2.59 GB, 2.69 GB, 2.75 GB, 2.83 GB, 2.93 GB, 2.98 GB, 3.07 GB, 3.13 GB, 3.20 GB, 3.28 GB
- - AsyncTotalTime: 0ns
- - AverageHdfsReadThreadConcurrency: 0.00
- - AverageScannerThreadConcurrency: 7.90
- - BytesRead: 3.32 GB
- - BytesReadDataNodeCache: 0
- - BytesReadLocal: 3.32 GB
- - BytesReadShortCircuit: 3.32 GB
- - DecompressionTime: 0ns
- - NumColumns: 0
- - NumDisksAccessed: 7
- - NumScannerThreadsStarted: 8
- - PeakMemoryUsage: 1017.33 MB
- - PerReadThreadRawHdfsThroughput: 1.56 GB/sec
- - RowsRead: 17.06M (17060133)
- - RowsReturned: 17.06M (17060133)
- - RowsReturnedRate: 13.80 M/sec
- - ScanRangesComplete: 147
- - ScannerThreadsInvoluntaryContextSwitches: 46.91K (46915)
- - ScannerThreadsTotalWallClockTime: 2h56m
- - MaterializeTupleTime(*): 2h50m
- - ScannerThreadsSysTime: 685.893ms
- - ScannerThreadsUserTime: 5m5s
- - ScannerThreadsVoluntaryContextSwitches: 132.66K (132664)
- - TotalRawHdfsReadTime(*): 2s126ms
- - TotalReadThroughput: 2.54 MB/sec
Advertisement
Add Comment
Please, Sign In to add comment
