Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- hdfs dfs -ls /user/hduser/input
- Found 1 items
- -rw-r--r-- 1 me me 1356604 2016-12-10 10:30 /user/hduser/input/salaries.csv
- sudo chmod a+x work/hadoop/top_salaries.py
- python2 top_salaries.py -r local salaries.csv
- ~work/hadoop$ python2 top_salaries.py -r hadoop hdfs:///user/hduser/input/salaries.csv
- No configs found; falling back on auto-configuration
- Looking for hadoop binary in $PATH...
- Found hadoop binary: /usr/local/hadoop/bin/hadoop
- STDERR: ls: Call From Notebook-PC/127.0.1.1 to localhost:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
- Traceback (most recent call last):
- File "top_salaries.py", line 37, in <module>
- salarymax.run()
- File "/usr/local/lib/python2.7/dist-packages/mrjob/job.py", line 429, in run
- mr_job.execute()
- File "/usr/local/lib/python2.7/dist-packages/mrjob/job.py", line 447, in execute
- super(MRJob, self).execute()
- File "/usr/local/lib/python2.7/dist-packages/mrjob/launch.py", line 158, in execute
- self.run_job()
- File "/usr/local/lib/python2.7/dist-packages/mrjob/launch.py", line 228, in run_job
- runner.run()
- File "/usr/local/lib/python2.7/dist-packages/mrjob/runner.py", line 481, in run
- self._run()
- File "/usr/local/lib/python2.7/dist-packages/mrjob/hadoop.py", line 332, in _run
- self._check_input_exists()
- File "/usr/local/lib/python2.7/dist-packages/mrjob/hadoop.py", line 346, in _check_input_exists
- if not self.fs.exists(path):
- File "/usr/local/lib/python2.7/dist-packages/mrjob/fs/composite.py", line 79, in exists
- return self._do_action('exists', path_glob)
- File "/usr/local/lib/python2.7/dist-packages/mrjob/fs/composite.py", line 63, in _do_action
- raise first_exception
- IOError: Could not check path hdfs:///user/hduser/input/salaries.csv
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement