Guest User

Untitled

a guest
Mar 22nd, 2018
101
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.43 KB | None | 0 0
  1. if (path.endsWith(".zip")) {
  2. sc.binaryFiles(path, minPartitions) .flatMap {
  3. case (name: String, content: PortableDataStream) => val zis = new ZipInputStream(content.open) // this solution works only for single file in the zip
  4. val entry = zis.getNextEntry
  5. val br = new BufferedReader(new InputStreamReader(zis))
  6. Stream.continually(br.readLine()).takeWhile(_ != null) } }
  7. else {
  8. sc.textFile(path, minPartitions)
  9. }
Add Comment
Please, Sign In to add comment