Advertisement
Guest User

Untitled

a guest
Aug 29th, 2017
157
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 2.10 KB | None | 0 0
  1. error: missing or invalid dependency detected while loading class file 'QualifiedTableName.class'.
  2. Could not access type AnyRef in package scala,
  3. because it (or its dependencies) are missing. Check your build definition for
  4. missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
  5. A full rebuild may help if 'QualifiedTableName.class' was compiled against an incompatible version of scala.
  6. error: missing or invalid dependency detected while loading class file 'FunctionIdentifier.class'.
  7. Could not access type AnyRef in package scala,
  8. because it (or its dependencies) are missing. Check your build definition for
  9. missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
  10. A full rebuild may help if 'FunctionIdentifier.class' was compiled against an incompatible version of scala.
  11. error: missing or invalid dependency detected while loading class file 'DefinedByConstructorParams.class'.
  12. ...
  13.  
  14. %AddDeps org.postgresql postgresql 42.1.4 --transitive
  15. val props = new java.util.Properties();
  16. props.setProperty("driver","org.postgresql.Driver");
  17. val df = spark.read.jdbc(url = "jdbc:postgresql://postgresql/database?user=user&password=password",
  18. table = "table", predicates = Array("1=1"), connectionProperties = props)
  19. df.show()
  20.  
  21. RUN wget https://d3kbcqa49mib13.cloudfront.net/spark-2.2.0-bin-hadoop2.7.tgz && tar -zxf spark-2.2.0-bin-hadoop2.7.tgz && chmod -R og+rw /opt/spark-2.2.0-bin-hadoop2.7 && chown -R a1414.a1414 /opt/spark-2.2.0-bin-hadoop2.7
  22. RUN (curl https://bintray.com/sbt/rpm/rpm > /etc/yum.repos.d/bintray-sbt-rpm.repo)
  23. RUN yum -y install --nogpgcheck sbt
  24. RUN (unset http_proxy; unset https_proxy; yum -y install --nogpgcheck java-1.8.0-openjdk-devel.i686)
  25. RUN (git clone https://github.com/apache/incubator-toree && cd incubator-toree && make clean release APACHE_SPARK_VERSION=2.2.0 ; exit 0)
  26. RUN (. /opt/rh/rh-python35/enable; cd /opt/incubator-toree/dist/toree-pip ;python setup.py install)
  27. RUN (. /opt/rh/rh-python35/enable; jupyter toree install --spark_home=/opt/spark-2.2.0-bin-hadoop2.7 --interpreters=Scala)
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement