API tools faq
paste
Advertisement
Guest User

Untitled

a guest
Nov 5th, 2019
114
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 55.62 KB | None | 0 0
raw download clone embed print report
  1. =~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2019.11.05 08:18:19 =~=~=~=~=~=~=~=~=~=~=~=
  2.  
  3. SLF4J: Class path contains multiple SLF4J bindings.
  4. SLF4J: Found binding in [jar:file:/opt/Spark/extraLibraries/slf4j-nop-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
  5. SLF4J: Found binding in [jar:file:/opt/Spark/spark-2.4.3-bin-hadoop2.7/jars/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class]
  6. SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
  7. SLF4J: Actual binding is of type [org.slf4j.helpers.NOPLoggerFactory]
  8. ---- CREATING SPARK Session:
  9. warehouseLocation:/data2/spark-warehouse
  10. +------------+--------+--------------------+----+------------+
  11. | fwSerial|panosver| csvpath|size|afterProcess|
  12. +------------+--------+--------------------+----+------------+
  13. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4895| Delete|
  14. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4701| Delete|
  15. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4465| Delete|
  16. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4701| Delete|
  17. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|5028| Delete|
  18. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2366| Delete|
  19. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2038| Delete|
  20. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4854| Delete|
  21. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2796| Delete|
  22. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4465| Delete|
  23. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|5264| Delete|
  24. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4414| Delete|
  25. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4721| Delete|
  26. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4946| Delete|
  27. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2376| Delete|
  28. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4885| Delete|
  29. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2622| Delete|
  30. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|5039| Delete|
  31. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4885| Delete|
  32. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2499| Delete|
  33. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4742| Delete|
  34. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2468| Delete|
  35. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4660| Delete|
  36. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4946| Delete|
  37. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2335| Delete|
  38. +------------+--------+--------------------+----+------------+
  39.  
  40. Memory: 1160m
  41. LogCollector&Compacter called with the following parameters:
  42. Parameters for execution
  43. Master[processes]:............ local[1]
  44. Available RAM (MB):........... 1187840
  45. User:......................... admin
  46. debug:........................ false
  47. Parameters for Job Connections
  48. Task ID:...................... 2106
  49. My IP:........................ 10.4.23.43
  50. Expedition IP:................ 10.4.23.43:3306
  51. Time Zone:.................... Europe/Helsinki
  52. dbUser (dbPassword):.......... root (************)
  53. projectName:.................. demo
  54. Parameters for Data Sources
  55. App Categories (source):........ (Expedition)
  56. CSV Files Path:................./tmp/1572897807_traffic_files.csv
  57. Parquet output path:.......... file:///myLogs/connections.parquet
  58. Temporary folder:............. /data2
  59. ---- AppID DB LOAD:
  60. Application Categories loading...
  61. Application Categories loaded
  62.  
  63. +------------+--------+--------------------+----+------------+--------+---+---------------+
  64. | fwSerial|panosver| csvpath|size|afterProcess| grouped|row|accumulatedSize|
  65. +------------+--------+--------------------+----+------------+--------+---+---------------+
  66. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4895| Delete|grouping| 1| 4895.0|
  67. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4701| Delete|grouping| 2| 9596.0|
  68. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4465| Delete|grouping| 3| 14061.0|
  69. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4701| Delete|grouping| 4| 18762.0|
  70. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|5028| Delete|grouping| 5| 23790.0|
  71. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2366| Delete|grouping| 6| 26156.0|
  72. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2038| Delete|grouping| 7| 28194.0|
  73. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4854| Delete|grouping| 8| 33048.0|
  74. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2796| Delete|grouping| 9| 35844.0|
  75. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4465| Delete|grouping| 10| 40309.0|
  76. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|5264| Delete|grouping| 11| 45573.0|
  77. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4414| Delete|grouping| 12| 49987.0|
  78. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4721| Delete|grouping| 13| 54708.0|
  79. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4946| Delete|grouping| 14| 59654.0|
  80. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2376| Delete|grouping| 15| 62030.0|
  81. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4885| Delete|grouping| 16| 66915.0|
  82. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2622| Delete|grouping| 17| 69537.0|
  83. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|5039| Delete|grouping| 18| 74576.0|
  84. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4885| Delete|grouping| 19| 79461.0|
  85. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2499| Delete|grouping| 20| 81960.0|
  86. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4742| Delete|grouping| 21| 86702.0|
  87. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2468| Delete|grouping| 22| 89170.0|
  88. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4660| Delete|grouping| 23| 93830.0|
  89. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4946| Delete|grouping| 24| 98776.0|
  90. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2335| Delete|grouping| 25| 101111.0|
  91. +------------+--------+--------------------+----+------------+--------+---+---------------+
  92.  
  93. Selection criteria: 0 < accumulatedSize and accumulatedSize <= 1187840
  94. Processing from lowLimit:0 to highLimit:1187840 with StepLine:1187840
  95. Few logs can fit in this batch:25
  96. 8.1.0:/myLogs/PCC-CORP-PA1_traffic_2019_10_25_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_11_04_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_31_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_27_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_22_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_30_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_29_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_15_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_16_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_11_02_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_21_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_18_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_20_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_12_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_11_03_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_11_01_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_17_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_14_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_26_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_13_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_11_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_24_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_28_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_19_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_23_last_calendar_day.csv
  97. Logs of format 7.1.x NOT found
  98. Logs of format 8.0.2 NOT found
  99. Logs of format 8.1.0-beta17 NOT found
  100. Logs of format 8.1.0 found
  101. Logs of format 9.0.0 NOT found
  102. Logs of format 9.1.0-beta NOT found
  103. Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 595 in stage 188.0 failed 1 times, most recent failure: Lost task 595.0 in stage 188.0 (TID 6943, localhost, executor driver): com.esotericsoftware.kryo.KryoException: java.io.IOException: No space left on device
  104. Serialization trace:
  105. buffers (org.apache.spark.sql.execution.columnar.CachedBatch)
  106. at com.esotericsoftware.kryo.io.Output.flush(Output.java:188)
  107. at com.esotericsoftware.kryo.io.Output.require(Output.java:164)
  108. at com.esotericsoftware.kryo.io.Output.writeBytes(Output.java:251)
  109. at com.esotericsoftware.kryo.io.Output.writeBytes(Output.java:237)
  110. at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ByteArraySerializer.write(DefaultArraySerializers.java:49)
  111. at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ByteArraySerializer.write(DefaultArraySerializers.java:38)
  112. at com.esotericsoftware.kryo.Kryo.writeObjectOrNull(Kryo.java:629)
  113. at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:332)
  114. at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:302)
  115. at com.esotericsoftware.kryo.Kryo.writeObjectOrNull(Kryo.java:629)
  116. at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:86)
  117. at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:508)
  118. at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:651)
  119. at org.apache.spark.serializer.KryoSerializationStream.writeObject(KryoSerializer.scala:241)
  120. at org.apache.spark.serializer.SerializationStream.writeAll(Serializer.scala:140)
  121. at org.apache.spark.serializer.SerializerManager.dataSerializeStream(SerializerManager.scala:174)
  122. at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1$$anonfun$apply$7.apply(BlockManager.scala:1174)
  123. at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1$$anonfun$apply$7.apply(BlockManager.scala:1172)
  124. at org.apache.spark.storage.DiskStore.put(DiskStore.scala:69)
  125. at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:1172)
  126. at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:1156)
  127. at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:1091)
  128. at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1156)
  129. at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:882)
  130. at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:335)
  131. at org.apache.spark.rdd.RDD.iterator(RDD.scala:286)
  132. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  133. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
  134. at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
  135. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  136. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
  137. at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
  138. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  139. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
  140. at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
  141. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  142. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
  143. at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
  144. at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
  145. at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
  146. at org.apache.spark.scheduler.Task.run(Task.scala:121)
  147. at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
  148. at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
  149. at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
  150. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  151. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  152. at java.lang.Thread.run(Thread.java:748)
  153. Caused by: java.io.IOException: No space left on device
  154. at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
  155. at sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:60)
  156. at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
  157. at sun.nio.ch.IOUtil.write(IOUtil.java:65)
  158. at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:211)
  159. at org.apache.spark.storage.CountingWritableChannel.write(DiskStore.scala:332)
  160. at java.nio.channels.Channels.writeFullyImpl(Channels.java:78)
  161. at java.nio.channels.Channels.writeFully(Channels.java:101)
  162. at java.nio.channels.Channels.access$000(Channels.java:61)
  163. at java.nio.channels.Channels$1.write(Channels.java:174)
  164. at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
  165. at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
  166. at com.esotericsoftware.kryo.io.Output.flush(Output.java:186)
  167. ... 46 more
  168.  
  169. Driver stacktrace:
  170. at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1889)
  171. at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1877)
  172. at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1876)
  173. at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
  174. at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
  175. at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1876)
  176. at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
  177. at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
  178. at scala.Option.foreach(Option.scala:257)
  179. at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:926)
  180. at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2110)
  181. at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2059)
  182. at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2048)
  183. at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
  184. at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:737)
  185. at org.apache.spark.SparkContext.runJob(SparkContext.scala:2061)
  186. at org.apache.spark.SparkContext.runJob(SparkContext.scala:2082)
  187. at org.apache.spark.SparkContext.runJob(SparkContext.scala:2101)
  188. at org.apache.spark.SparkContext.runJob(SparkContext.scala:2126)
  189. at org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:945)
  190. at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
  191. at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
  192. at org.apache.spark.rdd.RDD.withScope(RDD.scala:363)
  193. at org.apache.spark.rdd.RDD.collect(RDD.scala:944)
  194. at com.paloaltonetworks.tbd.LogCollectorCompacter$.processSubFiles(LogCollectorCompacter.scala:1355)
  195. at com.paloaltonetworks.tbd.LogCollectorCompacter$.main(LogCollectorCompacter.scala:471)
  196. at com.paloaltonetworks.tbd.LogCollectorCompacter.main(LogCollectorCompacter.scala)
  197. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  198. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  199. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  200. at java.lang.reflect.Method.invoke(Method.java:498)
  201. at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
  202. at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:849)
  203. at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
  204. at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
  205. at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
  206. at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
  207. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
  208. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  209. Caused by: com.esotericsoftware.kryo.KryoException: java.io.IOException: No space left on device
  210. Serialization trace:
  211. buffers (org.apache.spark.sql.execution.columnar.CachedBatch)
  212. at com.esotericsoftware.kryo.io.Output.flush(Output.java:188)
  213. at com.esotericsoftware.kryo.io.Output.require(Output.java:164)
  214. at com.esotericsoftware.kryo.io.Output.writeBytes(Output.java:251)
  215. at com.esotericsoftware.kryo.io.Output.writeBytes(Output.java:237)
  216. at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ByteArraySerializer.write(DefaultArraySerializers.java:49)
  217. at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ByteArraySerializer.write(DefaultArraySerializers.java:38)
  218. at com.esotericsoftware.kryo.Kryo.writeObjectOrNull(Kryo.java:629)
  219. at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:332)
  220. at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:302)
  221. at com.esotericsoftware.kryo.Kryo.writeObjectOrNull(Kryo.java:629)
  222. at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:86)
  223. at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:508)
  224. at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:651)
  225. at org.apache.spark.serializer.KryoSerializationStream.writeObject(KryoSerializer.scala:241)
  226. at org.apache.spark.serializer.SerializationStream.writeAll(Serializer.scala:140)
  227. at org.apache.spark.serializer.SerializerManager.dataSerializeStream(SerializerManager.scala:174)
  228. at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1$$anonfun$apply$7.apply(BlockManager.scala:1174)
  229. at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1$$anonfun$apply$7.apply(BlockManager.scala:1172)
  230. at org.apache.spark.storage.DiskStore.put(DiskStore.scala:69)
  231. at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:1172)
  232. at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:1156)
  233. at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:1091)
  234. at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1156)
  235. at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:882)
  236. at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:335)
  237. at org.apache.spark.rdd.RDD.iterator(RDD.scala:286)
  238. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  239. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
  240. at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
  241. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  242. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
  243. at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
  244. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  245. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
  246. at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
  247. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  248. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
  249. at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
  250. at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
  251. at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
  252. at org.apache.spark.scheduler.Task.run(Task.scala:121)
  253. at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
  254. at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
  255. at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
  256. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  257. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  258. at java.lang.Thread.run(Thread.java:748)
  259. Caused by: java.io.IOException: No space left on device
  260. at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
  261. at sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:60)
  262. at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
  263. at sun.nio.ch.IOUtil.write(IOUtil.java:65)
  264. at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:211)
  265. at org.apache.spark.storage.CountingWritableChannel.write(DiskStore.scala:332)
  266. at java.nio.channels.Channels.writeFullyImpl(Channels.java:78)
  267. at java.nio.channels.Channels.writeFully(Channels.java:101)
  268. at java.nio.channels.Channels.access$000(Channels.java:61)
  269. at java.nio.channels.Channels$1.write(Channels.java:174)
  270. at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
  271. at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
  272. at com.esotericsoftware.kryo.io.Output.flush(Output.java:186)
  273. ... 46 more
  274. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  275. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  276. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  277. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  278. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  279. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  280. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  281. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  282. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  283. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  284. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  285. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  286. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  287. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  288. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  289. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  290. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  291. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  292. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  293. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  294. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  295. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  296. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  297. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  298. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  299. SLF4J: Class path contains multiple SLF4J bindings.
  300. SLF4J: Found binding in [jar:file:/opt/Spark/extraLibraries/slf4j-nop-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
  301. SLF4J: Found binding in [jar:file:/opt/Spark/spark-2.4.3-bin-hadoop2.7/jars/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class]
  302. SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
  303. SLF4J: Actual binding is of type [org.slf4j.helpers.NOPLoggerFactory]
  304. ---- CREATING SPARK Session:
  305. warehouseLocation:/data2/spark-warehouse
  306. +------------+--------+--------------------+----+------------+
  307. | fwSerial|panosver| csvpath|size|afterProcess|
  308. +------------+--------+--------------------+----+------------+
  309. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4895| Delete|
  310. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4701| Delete|
  311. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4465| Delete|
  312. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4701| Delete|
  313. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|5028| Delete|
  314. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2366| Delete|
  315. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2038| Delete|
  316. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4854| Delete|
  317. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2796| Delete|
  318. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4465| Delete|
  319. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|5264| Delete|
  320. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4414| Delete|
  321. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4721| Delete|
  322. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4946| Delete|
  323. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2376| Delete|
  324. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4885| Delete|
  325. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2622| Delete|
  326. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|5039| Delete|
  327. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4885| Delete|
  328. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2499| Delete|
  329. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4742| Delete|
  330. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2468| Delete|
  331. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4660| Delete|
  332. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4946| Delete|
  333. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2335| Delete|
  334. +------------+--------+--------------------+----+------------+
  335.  
  336. Memory: 1160m
  337. LogCollector&Compacter called with the following parameters:
  338. Parameters for execution
  339. Master[processes]:............ local[1]
  340. Available RAM (MB):........... 1187840
  341. User:......................... admin
  342. debug:........................ false
  343. Parameters for Job Connections
  344. Task ID:...................... 2107
  345. My IP:........................ 10.4.23.43
  346. Expedition IP:................ 10.4.23.43:3306
  347. Time Zone:.................... Europe/Helsinki
  348. dbUser (dbPassword):.......... root (************)
  349. projectName:.................. demo
  350. Parameters for Data Sources
  351. App Categories (source):........ (Expedition)
  352. CSV Files Path:................./tmp/1572904912_traffic_files.csv
  353. Parquet output path:.......... file:///myLogs/connections.parquet
  354. Temporary folder:............. /data2
  355. ---- AppID DB LOAD:
  356. Application Categories loading...
  357. Application Categories loaded
  358.  
  359. +------------+--------+--------------------+----+------------+--------+---+---------------+
  360. | fwSerial|panosver| csvpath|size|afterProcess| grouped|row|accumulatedSize|
  361. +------------+--------+--------------------+----+------------+--------+---+---------------+
  362. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4895| Delete|grouping| 1| 4895.0|
  363. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4701| Delete|grouping| 2| 9596.0|
  364. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4465| Delete|grouping| 3| 14061.0|
  365. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4701| Delete|grouping| 4| 18762.0|
  366. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|5028| Delete|grouping| 5| 23790.0|
  367. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2366| Delete|grouping| 6| 26156.0|
  368. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2038| Delete|grouping| 7| 28194.0|
  369. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4854| Delete|grouping| 8| 33048.0|
  370. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2796| Delete|grouping| 9| 35844.0|
  371. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4465| Delete|grouping| 10| 40309.0|
  372. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|5264| Delete|grouping| 11| 45573.0|
  373. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4414| Delete|grouping| 12| 49987.0|
  374. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4721| Delete|grouping| 13| 54708.0|
  375. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4946| Delete|grouping| 14| 59654.0|
  376. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2376| Delete|grouping| 15| 62030.0|
  377. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4885| Delete|grouping| 16| 66915.0|
  378. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2622| Delete|grouping| 17| 69537.0|
  379. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|5039| Delete|grouping| 18| 74576.0|
  380. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4885| Delete|grouping| 19| 79461.0|
  381. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2499| Delete|grouping| 20| 81960.0|
  382. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4742| Delete|grouping| 21| 86702.0|
  383. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2468| Delete|grouping| 22| 89170.0|
  384. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4660| Delete|grouping| 23| 93830.0|
  385. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4946| Delete|grouping| 24| 98776.0|
  386. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2335| Delete|grouping| 25| 101111.0|
  387. +------------+--------+--------------------+----+------------+--------+---+---------------+
  388.  
  389. Selection criteria: 0 < accumulatedSize and accumulatedSize <= 1187840
  390. Processing from lowLimit:0 to highLimit:1187840 with StepLine:1187840
  391. Few logs can fit in this batch:25
  392. 8.1.0:/myLogs/PCC-CORP-PA1_traffic_2019_10_25_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_11_04_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_31_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_27_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_22_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_30_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_29_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_15_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_16_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_11_02_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_21_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_18_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_20_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_12_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_11_03_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_11_01_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_17_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_14_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_26_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_13_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_11_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_24_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_28_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_19_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_23_last_calendar_day.csv
  393. Logs of format 7.1.x NOT found
  394. Logs of format 8.0.2 NOT found
  395. Logs of format 8.1.0-beta17 NOT found
  396. Logs of format 8.1.0 found
  397. Logs of format 9.0.0 NOT found
  398. Logs of format 9.1.0-beta NOT found
  399. Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 595 in stage 188.0 failed 1 times, most recent failure: Lost task 595.0 in stage 188.0 (TID 6943, localhost, executor driver): com.esotericsoftware.kryo.KryoException: java.io.IOException: No space left on device
  400. Serialization trace:
  401. buffers (org.apache.spark.sql.execution.columnar.CachedBatch)
  402. at com.esotericsoftware.kryo.io.Output.flush(Output.java:188)
  403. at com.esotericsoftware.kryo.io.Output.require(Output.java:164)
  404. at com.esotericsoftware.kryo.io.Output.writeBytes(Output.java:251)
  405. at com.esotericsoftware.kryo.io.Output.writeBytes(Output.java:237)
  406. at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ByteArraySerializer.write(DefaultArraySerializers.java:49)
  407. at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ByteArraySerializer.write(DefaultArraySerializers.java:38)
  408. at com.esotericsoftware.kryo.Kryo.writeObjectOrNull(Kryo.java:629)
  409. at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:332)
  410. at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:302)
  411. at com.esotericsoftware.kryo.Kryo.writeObjectOrNull(Kryo.java:629)
  412. at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:86)
  413. at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:508)
  414. at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:651)
  415. at org.apache.spark.serializer.KryoSerializationStream.writeObject(KryoSerializer.scala:241)
  416. at org.apache.spark.serializer.SerializationStream.writeAll(Serializer.scala:140)
  417. at org.apache.spark.serializer.SerializerManager.dataSerializeStream(SerializerManager.scala:174)
  418. at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1$$anonfun$apply$7.apply(BlockManager.scala:1174)
  419. at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1$$anonfun$apply$7.apply(BlockManager.scala:1172)
  420. at org.apache.spark.storage.DiskStore.put(DiskStore.scala:69)
  421. at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:1172)
  422. at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:1156)
  423. at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:1091)
  424. at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1156)
  425. at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:882)
  426. at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:335)
  427. at org.apache.spark.rdd.RDD.iterator(RDD.scala:286)
  428. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  429. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
  430. at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
  431. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  432. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
  433. at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
  434. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  435. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
  436. at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
  437. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  438. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
  439. at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
  440. at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
  441. at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
  442. at org.apache.spark.scheduler.Task.run(Task.scala:121)
  443. at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
  444. at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
  445. at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
  446. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  447. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  448. at java.lang.Thread.run(Thread.java:748)
  449. Caused by: java.io.IOException: No space left on device
  450. at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
  451. at sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:60)
  452. at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
  453. at sun.nio.ch.IOUtil.write(IOUtil.java:65)
  454. at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:211)
  455. at org.apache.spark.storage.CountingWritableChannel.write(DiskStore.scala:332)
  456. at java.nio.channels.Channels.writeFullyImpl(Channels.java:78)
  457. at java.nio.channels.Channels.writeFully(Channels.java:101)
  458. at java.nio.channels.Channels.access$000(Channels.java:61)
  459. at java.nio.channels.Channels$1.write(Channels.java:174)
  460. at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
  461. at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
  462. at com.esotericsoftware.kryo.io.Output.flush(Output.java:186)
  463. ... 46 more
  464.  
  465. Driver stacktrace:
  466. at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1889)
  467. at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1877)
  468. at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1876)
  469. at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
  470. at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
  471. at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1876)
  472. at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
  473. at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:926)
  474. at scala.Option.foreach(Option.scala:257)
  475. at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:926)
  476. at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2110)
  477. at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2059)
  478. at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2048)
  479. at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
  480. at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:737)
  481. at org.apache.spark.SparkContext.runJob(SparkContext.scala:2061)
  482. at org.apache.spark.SparkContext.runJob(SparkContext.scala:2082)
  483. at org.apache.spark.SparkContext.runJob(SparkContext.scala:2101)
  484. at org.apache.spark.SparkContext.runJob(SparkContext.scala:2126)
  485. at org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:945)
  486. at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
  487. at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
  488. at org.apache.spark.rdd.RDD.withScope(RDD.scala:363)
  489. at org.apache.spark.rdd.RDD.collect(RDD.scala:944)
  490. at com.paloaltonetworks.tbd.LogCollectorCompacter$.processSubFiles(LogCollectorCompacter.scala:1355)
  491. at com.paloaltonetworks.tbd.LogCollectorCompacter$.main(LogCollectorCompacter.scala:471)
  492. at com.paloaltonetworks.tbd.LogCollectorCompacter.main(LogCollectorCompacter.scala)
  493. at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  494. at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  495. at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  496. at java.lang.reflect.Method.invoke(Method.java:498)
  497. at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
  498. at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:849)
  499. at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
  500. at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
  501. at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
  502. at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
  503. at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
  504. at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
  505. Caused by: com.esotericsoftware.kryo.KryoException: java.io.IOException: No space left on device
  506. Serialization trace:
  507. buffers (org.apache.spark.sql.execution.columnar.CachedBatch)
  508. at com.esotericsoftware.kryo.io.Output.flush(Output.java:188)
  509. at com.esotericsoftware.kryo.io.Output.require(Output.java:164)
  510. at com.esotericsoftware.kryo.io.Output.writeBytes(Output.java:251)
  511. at com.esotericsoftware.kryo.io.Output.writeBytes(Output.java:237)
  512. at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ByteArraySerializer.write(DefaultArraySerializers.java:49)
  513. at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ByteArraySerializer.write(DefaultArraySerializers.java:38)
  514. at com.esotericsoftware.kryo.Kryo.writeObjectOrNull(Kryo.java:629)
  515. at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:332)
  516. at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:302)
  517. at com.esotericsoftware.kryo.Kryo.writeObjectOrNull(Kryo.java:629)
  518. at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:86)
  519. at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:508)
  520. at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:651)
  521. at org.apache.spark.serializer.KryoSerializationStream.writeObject(KryoSerializer.scala:241)
  522. at org.apache.spark.serializer.SerializationStream.writeAll(Serializer.scala:140)
  523. at org.apache.spark.serializer.SerializerManager.dataSerializeStream(SerializerManager.scala:174)
  524. at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1$$anonfun$apply$7.apply(BlockManager.scala:1174)
  525. at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1$$anonfun$apply$7.apply(BlockManager.scala:1172)
  526. at org.apache.spark.storage.DiskStore.put(DiskStore.scala:69)
  527. at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:1172)
  528. at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:1156)
  529. at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:1091)
  530. at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1156)
  531. at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:882)
  532. at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:335)
  533. at org.apache.spark.rdd.RDD.iterator(RDD.scala:286)
  534. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  535. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
  536. at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
  537. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  538. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
  539. at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
  540. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  541. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
  542. at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
  543. at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
  544. at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
  545. at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
  546. at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
  547. at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
  548. at org.apache.spark.scheduler.Task.run(Task.scala:121)
  549. at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
  550. at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
  551. at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
  552. at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  553. at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  554. at java.lang.Thread.run(Thread.java:748)
  555. Caused by: java.io.IOException: No space left on device
  556. at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
  557. at sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:60)
  558. at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
  559. at sun.nio.ch.IOUtil.write(IOUtil.java:65)
  560. at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:211)
  561. at org.apache.spark.storage.CountingWritableChannel.write(DiskStore.scala:332)
  562. at java.nio.channels.Channels.writeFullyImpl(Channels.java:78)
  563. at java.nio.channels.Channels.writeFully(Channels.java:101)
  564. at java.nio.channels.Channels.access$000(Channels.java:61)
  565. at java.nio.channels.Channels$1.write(Channels.java:174)
  566. at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
  567. at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
  568. at com.esotericsoftware.kryo.io.Output.flush(Output.java:186)
  569. ... 46 more
  570. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  571. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  572. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  573. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  574. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  575. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  576. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  577. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  578. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  579. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  580. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  581. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  582. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  583. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  584. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  585. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  586. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  587. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  588. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  589. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  590. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  591. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  592. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  593. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  594. PHP Notice: Trying to get property of non-object in /var/www/html/API/convertCSVtoParquet_afterProcess.php on line 22
  595. SLF4J: Class path contains multiple SLF4J bindings.
  596. SLF4J: Found binding in [jar:file:/opt/Spark/extraLibraries/slf4j-nop-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
  597. SLF4J: Found binding in [jar:file:/opt/Spark/spark-2.4.3-bin-hadoop2.7/jars/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class]
  598. SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
  599. SLF4J: Actual binding is of type [org.slf4j.helpers.NOPLoggerFactory]
  600. ---- CREATING SPARK Session:
  601. warehouseLocation:/data2/spark-warehouse
  602. +------------+--------+--------------------+----+------------+
  603. | fwSerial|panosver| csvpath|size|afterProcess|
  604. +------------+--------+--------------------+----+------------+
  605. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4895| Delete|
  606. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4701| Delete|
  607. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4465| Delete|
  608. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4701| Delete|
  609. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|5028| Delete|
  610. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2366| Delete|
  611. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2038| Delete|
  612. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4854| Delete|
  613. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2796| Delete|
  614. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4465| Delete|
  615. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|5264| Delete|
  616. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4414| Delete|
  617. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4721| Delete|
  618. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4946| Delete|
  619. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2376| Delete|
  620. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4885| Delete|
  621. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2622| Delete|
  622. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|5039| Delete|
  623. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4885| Delete|
  624. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2499| Delete|
  625. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4875| Delete|
  626. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4742| Delete|
  627. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2468| Delete|
  628. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4660| Delete|
  629. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4946| Delete|
  630. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2335| Delete|
  631. +------------+--------+--------------------+----+------------+
  632.  
  633. Memory: 1160m
  634. LogCollector&Compacter called with the following parameters:
  635. Parameters for execution
  636. Master[processes]:............ local[1]
  637. Available RAM (MB):........... 1187840
  638. User:......................... admin
  639. debug:........................ false
  640. Parameters for Job Connections
  641. Task ID:...................... 2110
  642. My IP:........................ 10.4.23.43
  643. Expedition IP:................ 10.4.23.43:3306
  644. Time Zone:.................... Europe/Helsinki
  645. dbUser (dbPassword):.......... root (************)
  646. projectName:.................. demo
  647. Parameters for Data Sources
  648. App Categories (source):........ (Expedition)
  649. CSV Files Path:................./tmp/1572963212_traffic_files.csv
  650. Parquet output path:.......... file:///myLogs/connections.parquet
  651. Temporary folder:............. /data2
  652. ---- AppID DB LOAD:
  653. Application Categories loading...
  654. Application Categories loaded
  655.  
  656. +------------+--------+--------------------+----+------------+--------+---+---------------+
  657. | fwSerial|panosver| csvpath|size|afterProcess| grouped|row|accumulatedSize|
  658. +------------+--------+--------------------+----+------------+--------+---+---------------+
  659. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4895| Delete|grouping| 1| 4895.0|
  660. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4701| Delete|grouping| 2| 9596.0|
  661. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4465| Delete|grouping| 3| 14061.0|
  662. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4701| Delete|grouping| 4| 18762.0|
  663. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|5028| Delete|grouping| 5| 23790.0|
  664. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2366| Delete|grouping| 6| 26156.0|
  665. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2038| Delete|grouping| 7| 28194.0|
  666. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4854| Delete|grouping| 8| 33048.0|
  667. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2796| Delete|grouping| 9| 35844.0|
  668. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4465| Delete|grouping| 10| 40309.0|
  669. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|5264| Delete|grouping| 11| 45573.0|
  670. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4414| Delete|grouping| 12| 49987.0|
  671. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4721| Delete|grouping| 13| 54708.0|
  672. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4946| Delete|grouping| 14| 59654.0|
  673. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2376| Delete|grouping| 15| 62030.0|
  674. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4885| Delete|grouping| 16| 66915.0|
  675. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2622| Delete|grouping| 17| 69537.0|
  676. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|5039| Delete|grouping| 18| 74576.0|
  677. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4885| Delete|grouping| 19| 79461.0|
  678. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2499| Delete|grouping| 20| 81960.0|
  679. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4875| Delete|grouping| 21| 86835.0|
  680. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4742| Delete|grouping| 22| 91577.0|
  681. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2468| Delete|grouping| 23| 94045.0|
  682. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4660| Delete|grouping| 24| 98705.0|
  683. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|4946| Delete|grouping| 25| 103651.0|
  684. |001801019445| 8.1.0|/myLogs/PCC-CORP-...|2335| Delete|grouping| 26| 105986.0|
  685. +------------+--------+--------------------+----+------------+--------+---+---------------+
  686.  
  687. Selection criteria: 0 < accumulatedSize and accumulatedSize <= 1187840
  688. Processing from lowLimit:0 to highLimit:1187840 with StepLine:1187840
  689. Few logs can fit in this batch:26
  690. 8.1.0:/myLogs/PCC-CORP-PA1_traffic_2019_10_25_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_11_04_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_31_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_27_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_22_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_30_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_29_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_15_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_16_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_11_02_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_21_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_18_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_20_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_12_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_11_03_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_11_01_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_17_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_14_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_11_05_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_26_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_13_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_11_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_24_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_28_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_19_last_calendar_day.csv,/myLogs/PCC-CORP-PA1_traffic_2019_10_23_last_calendar_day.csv
  691. Logs of format 7.1.x NOT found
  692. Logs of format 8.0.2 NOT found
  693. Logs of format 8.1.0-beta17 NOT found
  694. Logs of format 8.1.0 found
  695. Logs of format 9.0.0 NOT found
  696. Logs of format 9.1.0-beta NOT found
  697. ]0;expedition@Expedition: ~expedition@Expedition:~$
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!