Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- error: missing or invalid dependency detected while loading class file 'QualifiedTableName.class'.
- Could not access type AnyRef in package scala,
- because it (or its dependencies) are missing. Check your build definition for
- missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
- A full rebuild may help if 'QualifiedTableName.class' was compiled against an incompatible version of scala.
- error: missing or invalid dependency detected while loading class file 'FunctionIdentifier.class'.
- Could not access type AnyRef in package scala,
- because it (or its dependencies) are missing. Check your build definition for
- missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)
- A full rebuild may help if 'FunctionIdentifier.class' was compiled against an incompatible version of scala.
- error: missing or invalid dependency detected while loading class file 'DefinedByConstructorParams.class'.
- ...
- %AddDeps org.postgresql postgresql 42.1.4 --transitive
- val props = new java.util.Properties();
- props.setProperty("driver","org.postgresql.Driver");
- val df = spark.read.jdbc(url = "jdbc:postgresql://postgresql/database?user=user&password=password",
- table = "table", predicates = Array("1=1"), connectionProperties = props)
- df.show()
- RUN wget https://d3kbcqa49mib13.cloudfront.net/spark-2.2.0-bin-hadoop2.7.tgz && tar -zxf spark-2.2.0-bin-hadoop2.7.tgz && chmod -R og+rw /opt/spark-2.2.0-bin-hadoop2.7 && chown -R a1414.a1414 /opt/spark-2.2.0-bin-hadoop2.7
- RUN (curl https://bintray.com/sbt/rpm/rpm > /etc/yum.repos.d/bintray-sbt-rpm.repo)
- RUN yum -y install --nogpgcheck sbt
- RUN (unset http_proxy; unset https_proxy; yum -y install --nogpgcheck java-1.8.0-openjdk-devel.i686)
- RUN (git clone https://github.com/apache/incubator-toree && cd incubator-toree && make clean release APACHE_SPARK_VERSION=2.2.0 ; exit 0)
- RUN (. /opt/rh/rh-python35/enable; cd /opt/incubator-toree/dist/toree-pip ;python setup.py install)
- RUN (. /opt/rh/rh-python35/enable; jupyter toree install --spark_home=/opt/spark-2.2.0-bin-hadoop2.7 --interpreters=Scala)
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement