spark diocan conf

Premetto che Jupiter era gia installato sulla macchina.

Fondamentalmente ho seguito questa guida:
https://medium.com/@josemarcialportilla/getting-spark-python-and-jupyter-notebook-running-on-amazon-ec2-dec599e1c297

ho solo cambiato qualche cosina, jupyter non e' installato come servizio, quindi forse nel tuo caso forse dovrai cambiare un po' di cose, in questo caso io lo avvio a mano prima di lavorarci (per vari motivi... certamente non per laziness :P)

altri pacchetti che ho installato:


sudo apt-get install ipython3
sudo ipython3 kernel install
ipython3 kernel install # prova questo se il primo non va
sudo pip3 install py4j findspark


Ti allego i miei file di conf:
######################################################################
# FILE: ~/.jupyter/jupyter_notebook_config.py
######################################################################

c = get_config()

# Notebook config this is where you saved your pem cert
c.NotebookApp.certfile = u'/home/agalloni/certs/mycert.pem'
# Run on all IP addresses of your instance
c.NotebookApp.ip = '*'
# Don't open browser by default
c.NotebookApp.open_browser = False
# Fix port to 8888
c.NotebookApp.port = 8888

#######################################################################

######################################################################
# FILE: ~/.bashrc
######################################################################

# ho attaccato i seguenti export alla fine del file

export PYSPARK_PYTHON=python3
export SPARK_HOME=/opt/spark
export PATH=$SPARK_HOME/bin:$PATH
export PYTHONPATH=$SPARK_HOME/python:$PYTHONPATH
#############################################################################