Advertisement
Guest User

Untitled

a guest
Apr 24th, 2017
781
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.71 KB | None | 0 0
  1. from pyspark.sql import HiveContext
  2. Query=""" select dt
  3. from default.content_publisher_events_log
  4. where dt between '20170415' and '20170419'
  5. """
  6. hive_context = HiveContext(sc)
  7. user_data = hive_context.sql(Query)
  8. user_data.count()
  9. 0 #that's the result
  10.  
  11. >>> sqlContext.sql("show tables").show()
  12. +--------+--------------------+-----------+
  13. |database| tableName|isTemporary|
  14. +--------+--------------------+-----------+
  15. | default|content_publisher...| false|
  16. | default| feed_installer_log| false|
  17. | default|keyword_based_ads...| false|
  18. | default|search_providers_log| false|
  19. +--------+--------------------+-----------+
  20.  
  21. >>> user_data.printSchema()
  22. root
  23. |-- dt: string (nullable = true)
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement