Advertisement
Guest User

Untitled

a guest
Mar 23rd, 2019
80
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.45 KB | None | 0 0
  1. # dask only version of this problem
  2.  
  3. shape = (100000, 100, 200)
  4. chunks = (50, 100, 200)
  5. data = dsa.random.random(shape, chunks=chunks)
  6. data
  7.  
  8. data_rc = data.rechunk((1000, 1, 200))
  9. data_rc
  10.  
  11. da.data.ravel
  12.  
  13. from time import sleep
  14.  
  15. def my_custom_function(f):
  16. # do some computations
  17. g = f**2 + 2*np.sin(f)
  18. sleep(0.01)
  19. # and reduce shape
  20. return g.ravel()[::15]
  21.  
  22. data_rc.map_blocks(my_custom_function, drop_axis=[1, 2], new_axis=[1], chunks=(1, 15))
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement