Advertisement
lorenzo_gatto

Spark query parquet toDebugString

Jun 15th, 2015
538
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 2.93 KB | None | 0 0
  1. (1) SchemaRDD[95] at RDD at SchemaRDD.scala:108
  2. == Query Plan ==
  3. == Physical Plan ==
  4. InsertIntoHiveTable (MetastoreRelation tpch_100gb_parquet, q3_shipping_priority, None), Map(), true
  5. TakeOrdered 10, [revenue#278 DESC,o_orderdate#295 ASC]
  6. Aggregate false, [l_orderkey#300,o_orderdate#295,o_shippriority#298], [l_orderkey#300,SUM(PartialSum#317) AS revenue#278,o_orderdate#295,o_shippriority#298]
  7. Exchange (HashPartitioning [l_orderkey#300,o_orderdate#295,o_shippriority#298], 200)
  8. Aggregate true, [l_orderkey#300,o_orderdate#295,o_shippriority#298], [l_orderkey#300,o_orderdate#295,o_shippriority#298,SUM((l_extendedprice#305 * (1.0 - l_discount#306))) AS PartialSum#317]
  9. Project [o_orderdate#295,l_discount#306,o_shippriority#298,l_orderkey#300,l_extendedprice#305]
  10. ShuffledHashJoin [o_orderkey#291], [l_orderkey#300], BuildRight
  11. Exchange (HashPartitioning [o_orderkey#291], 200)
  12. Project [o_orderdate#295,o_orderkey#291,o_shippriority#298]
  13. ShuffledHashJoin [c_custkey#283], [o_custkey#292], BuildLeft
  14. Exchange (HashPartitioning [c_custkey#283], 200)
  15. !OutputFaker [c_custkey#283]
  16. Project [c_custkey#318]
  17. Filter (c_mktsegment#324 = BUILDING)
  18. ParquetTableScan [c_custkey#318,c_mktsegment#324], (ParquetRelation hdfs://isi-bigcluster1.csr.unibo.it:8020/user/hive/warehouse/tpch_100gb_parquet.db/customer, Some(Configuration: core-default.xml, core-site.xml, yarn-default.xml, yarn-site.xml, mapred-default.xml, mapred-site.xml, hdfs-default.xml, hdfs-site.xml), org.apache.spark.sql.hive.HiveContext@743f2543, []), []
  19. Exchange (HashPartitioning [o_custkey#292], 200)
  20. !OutputFaker [o_orderdate#295,o_custkey#292,o_orderkey#291,o_shippriority#298]
  21. Filter (o_orderdate#330 < 1995-03-15)
  22. ParquetTableScan [o_orderdate#330,o_custkey#327,o_orderkey#326,o_shippriority#333], (ParquetRelation hdfs://isi-bigcluster1.csr.unibo.it:8020/user/hive/warehouse/tpch_100gb_parquet.db/orders, Some(Configuration: core-default.xml, core-site.xml, yarn-default.xml, yarn-site.xml, mapred-default.xml, mapred-site.xml, hdfs-default.xml, hdfs-site.xml), org.apache.spark.sql.hive.HiveContext@743f2543, []), []
  23. Exchange (HashPartitioning [l_orderkey#300], 200)
  24. !OutputFaker [l_extendedprice#305,l_orderkey#300,l_discount#306]
  25. Project [l_extendedprice#340,l_orderkey#335,l_discount#341]
  26. Filter (l_shipdate#345 > 1995-03-15)
  27. ParquetTableScan [l_extendedprice#340,l_orderkey#335,l_discount#341,l_shipdate#345], (ParquetRelation hdfs://isi-bigcluster1.csr.unibo.it:8020/user/hive/warehouse/tpch_100gb_parquet.db/lineitem, Some(Configuration: core-default.xml, core-site.xml, yarn-default.xml, yarn-site.xml, mapred-default.xml, mapred-site.xml, hdfs-default.xml, hdfs-site.xml), org.apache.spark.sql.hive.HiveContext@743f2543, []), [] []
  28. | ParallelCollectionRDD[143] at parallelize at commands.scala:46 []
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement