Advertisement
Guest User

Untitled

a guest
Dec 10th, 2016
65
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 1.83 KB | None | 0 0
  1. hdfs dfs -ls /user/hduser/input
  2.  
  3. Found 1 items
  4. -rw-r--r-- 1 me me 1356604 2016-12-10 10:30 /user/hduser/input/salaries.csv
  5.  
  6. sudo chmod a+x work/hadoop/top_salaries.py
  7.  
  8. python2 top_salaries.py -r local salaries.csv
  9.  
  10. ~work/hadoop$ python2 top_salaries.py -r hadoop hdfs:///user/hduser/input/salaries.csv
  11.  
  12. No configs found; falling back on auto-configuration
  13. Looking for hadoop binary in $PATH...
  14. Found hadoop binary: /usr/local/hadoop/bin/hadoop
  15. STDERR: ls: Call From Notebook-PC/127.0.1.1 to localhost:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
  16. Traceback (most recent call last):
  17. File "top_salaries.py", line 37, in <module>
  18. salarymax.run()
  19. File "/usr/local/lib/python2.7/dist-packages/mrjob/job.py", line 429, in run
  20. mr_job.execute()
  21. File "/usr/local/lib/python2.7/dist-packages/mrjob/job.py", line 447, in execute
  22. super(MRJob, self).execute()
  23. File "/usr/local/lib/python2.7/dist-packages/mrjob/launch.py", line 158, in execute
  24. self.run_job()
  25. File "/usr/local/lib/python2.7/dist-packages/mrjob/launch.py", line 228, in run_job
  26. runner.run()
  27. File "/usr/local/lib/python2.7/dist-packages/mrjob/runner.py", line 481, in run
  28. self._run()
  29. File "/usr/local/lib/python2.7/dist-packages/mrjob/hadoop.py", line 332, in _run
  30. self._check_input_exists()
  31. File "/usr/local/lib/python2.7/dist-packages/mrjob/hadoop.py", line 346, in _check_input_exists
  32. if not self.fs.exists(path):
  33. File "/usr/local/lib/python2.7/dist-packages/mrjob/fs/composite.py", line 79, in exists
  34. return self._do_action('exists', path_glob)
  35. File "/usr/local/lib/python2.7/dist-packages/mrjob/fs/composite.py", line 63, in _do_action
  36. raise first_exception
  37. IOError: Could not check path hdfs:///user/hduser/input/salaries.csv
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement