josephxsxn

Spark Design Rules

Apr 18th, 2017
108
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  • Number of Partitions Less then or Equal to Cores
    • Smaller partitions are more scheduling overhead but less to retry when failed
    • Better schedule to handle speculative execution
    • Lets you monitor progress rather then staying at 0%
Add Comment
Please, Sign In to add comment