Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- DataFrame dataframe =
- hiveContext.createDataFrame(rowData, schema);
- dataframe.write().format("orc").options(new HashMap(){
- {
- put("orc.compress","SNAPPY");
- put("hive.exec.orc.default.compress","SNAPPY");
- put("orc.compress.size","524288");
- put("hive.exec.orc.default.buffer.size","524288");
- put("hive.exec.orc.compression.strategy", "COMPRESSION");
- }
- }).save("spark_orc_output");
- Compression: ZLIB
- Compression size: 262144
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement