Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- spark = SparkSession
- .builder
- .appName("Cloudant Spark SQL Example in Python using temp tables")
- .config("cloudant.host","ACCOUNT.cloudant.com")
- .config("cloudant.username", "USERNAME")
- .config("cloudant.password","PASSWORD")
- .getOrCreate()
- # Loading temp table from Cloudant db
- df = spark.sql(" CREATE TEMPORARY TABLE test USING
- org.apache.bahir.cloudant OPTIONS ( database 'test')")
- df.printSchema()
- spark-submit --packages org.apache.bahir:spark-sql-cloudant_2.11:2.1.1 --conf spark.cloudant.host=ACCOUNT.cloudant.com --conf spark.cloudant.username=rizwan --conf spark.cloudant.password=mypassword ./couch-spark.py
- :: problems summary ::
- :::: WARNINGS
- module not found: org.apache.bahir#spark-sql-cloudant_2.11;2.1.1
- ==== local-m2-cache: tried
- file:/C:/Users/rrahman006/.m2/repository/org/apache/bahir/spark-sql-cl
- oudant_2.11/2.1.1/spark-sql-cloudant_2.11-2.1.1.pom
- -- artifact org.apache.bahir#spark-sql-cloudant_2.11;2.1.1!spark-sql-c
- loudant_2.11.jar:
- file:/C:/Users/rrahman006/.m2/repository/org/apache/bahir/spark-sql-cl
- oudant_2.11/2.1.1/spark-sql-cloudant_2.11-2.1.1.jar
- ==== local-ivy-cache: tried
- C:Usersrrahman006.ivy2localorg.apache.bahirspark-sql-cloudant_2.
- 112.1.1ivysivy.xml
- ==== central: tried
- https://repo1.maven.org/maven2/org/apache/bahir/spark-sql-cloudant_2.1
- 1/2.1.1/spark-sql-cloudant_2.11-2.1.1.pom
- -- artifact org.apache.bahir#spark-sql-cloudant_2.11;2.1.1!spark-sql-c
- loudant_2.11.jar:
- https://repo1.maven.org/maven2/org/apache/bahir/spark-sql-cloudant_2.1
- 1/2.1.1/spark-sql-cloudant_2.11-2.1.1.jar
- ==== spark-packages: tried
- http://dl.bintray.com/spark-packages/maven/org/apache/bahir/spark-sql-
- cloudant_2.11/2.1.1/spark-sql-cloudant_2.11-2.1.1.pom
- -- artifact org.apache.bahir#spark-sql-cloudant_2.11;2.1.1!spark-sql-c
- loudant_2.11.jar:
- http://dl.bintray.com/spark-packages/maven/org/apache/bahir/spark-sql-
- cloudant_2.11/2.1.1/spark-sql-cloudant_2.11-2.1.1.jar
- ::::::::::::::::::::::::::::::::::::::::::::::
- :: UNRESOLVED DEPENDENCIES ::
- ::::::::::::::::::::::::::::::::::::::::::::::
- :: org.apache.bahir#spark-sql-cloudant_2.11;2.1.1: not found
- ::::::::::::::::::::::::::::::::::::::::::::::
- :::: ERRORS
- Server access error at url https://repo1.maven.org/maven2/org/apache/bah
- ir/spark-sql-cloudant_2.11/2.1.1/spark-sql-cloudant_2.11-2.1.1.pom (javax.net.ss
- l.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path bu
- ilding failed: sun.security.provider.certpath.SunCertPathBuilderException: unabl
- e to find valid certification path to requested target)
- Server access error at url https://repo1.maven.org/maven2/org/apache/bah
- ir/spark-sql-cloudant_2.11/2.1.1/spark-sql-cloudant_2.11-2.1.1.jar (javax.net.ss
- l.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path bu
- ilding failed: sun.security.provider.certpath.SunCertPathBuilderException: unabl
- e to find valid certification path to requested target)
- :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
- Exception in thread "main" java.lang.RuntimeException: [unresolved dependency: o
- rg.apache.bahir#spark-sql-cloudant_2.11;2.1.1: not found]
- at org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(Spa
- rkSubmit.scala:1068)
- at org.apache.spark.deploy.SparkSubmit$.prepareSubmitEnvironment(SparkSu
- bmit.scala:287)
- at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:154)
- at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
- at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Add Comment
Please, Sign In to add comment