mitrakov

FS2 CSV->TXT parser

Apr 24th, 2020
806
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. import java.nio.file.{Paths, StandardOpenOption}
  2. import fs2._
  3. import cats.effect.{Blocker, ExitCode, IO, IOApp}
  4. import cats.implicits.toFunctorOps
  5.  
  6. object Hello extends IOApp {
  7.   override def run(args: List[String]): IO[ExitCode] = {
  8.     def program: Stream[IO, Unit] = {
  9.       Stream.resource(Blocker[IO]) flatMap { blocker =>
  10.         val stream = io.file.readAll[IO](Paths.get("/Users/macbook1/a.csv"), blocker, 4096)
  11.         stream
  12.           .through(text.utf8Decode)          // decode bytes as UTF-8
  13.           .through(text.lines)               // get lines removing CR/LF separators
  14.           .drop(1)                           // drop CSV header
  15.           .filter(_.nonEmpty)                // remove empty lines
  16.           .map(_.replaceAll("""['"]""", "")) // remove all ' and "
  17.           .intersperse(",")                  // add commas
  18.           .through(text.utf8Encode)          // encode back to bytes
  19.           .through(io.file.writeAll(Paths.get("/Users/macbook1/b.txt"), blocker, Seq(StandardOpenOption.TRUNCATE_EXISTING)))
  20.       }
  21.     }
  22.  
  23.     program.compile.drain.as(ExitCode.Success)
  24.   }
  25. }
RAW Paste Data

Adblocker detected! Please consider disabling it...

We've detected AdBlock Plus or some other adblocking software preventing Pastebin.com from fully loading.

We don't have any obnoxious sound, or popup ads, we actively block these annoying types of ads!

Please add Pastebin.com to your ad blocker whitelist or disable your adblocking software.

×