API tools faq
paste
Guest User

driver failed recovery

a guest
Oct 29th, 2015
70
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 47.90 KB | None | 0 0
raw download clone embed print report
  1. ...
  2.  
  3. 2015-10-18 14:18:30,154 [Driver] INFO org.apache.spark.streaming.CheckpointReader - Checkpoint files found: hdfs://root/checkpoint/checkpoint-1445170800000,hdfs://root/checkpoint/checkpoint-1445170780000,hdfs://root/checkpoint/checkpoint-1445170760000,hdfs://root/checkpoint/checkpoint-1445170740000,hdfs://root/checkpoint/checkpoint-1445170720000,hdfs://root/checkpoint/checkpoint-1445170700000,hdfs://root/checkpoint/checkpoint-1445170680000,hdfs://root/checkpoint/checkpoint-1445170660000,hdfs://root/checkpoint/checkpoint-1445170640000,hdfs://root/checkpoint/checkpoint-1445170620000
  4. 2015-10-18 14:18:30,155 [Driver] INFO org.apache.spark.streaming.CheckpointReader - Attempting to load checkpoint from file hdfs://root/checkpoint/checkpoint-1445170800000
  5. 2015-10-18 14:18:30,430 [Driver] INFO org.apache.spark.streaming.Checkpoint - Checkpoint for time 1445170800000 ms validated
  6. 2015-10-18 14:18:30,431 [Driver] INFO org.apache.spark.streaming.CheckpointReader - Checkpoint successfully loaded from file hdfs://root/checkpoint/checkpoint-1445170800000
  7. 2015-10-18 14:18:30,431 [Driver] INFO org.apache.spark.streaming.CheckpointReader - Checkpoint was generated at time 1445170800000 ms
  8. 2015-10-18 14:18:36,028 [Driver] INFO org.apache.spark.streaming.DStreamGraph - Restoring checkpoint data
  9. 2015-10-18 14:18:36,028 [Driver] INFO org.apache.spark.streaming.dstream.ForEachDStream - Restoring checkpoint data
  10. 2015-10-18 14:18:36,029 [Driver] INFO org.apache.spark.streaming.dstream.UnionDStream - Restoring checkpoint data
  11. 2015-10-18 14:18:36,029 [Driver] INFO org.apache.spark.streaming.dstream.FlatMappedDStream - Restoring checkpoint data
  12. 2015-10-18 14:18:36,029 [Driver] INFO org.apache.spark.streaming.dstream.FilteredDStream - Restoring checkpoint data
  13. 2015-10-18 14:18:36,029 [Driver] INFO org.apache.spark.streaming.dstream.FlatMapValuedDStream - Restoring checkpoint data
  14. 2015-10-18 14:18:36,029 [Driver] INFO org.apache.spark.streaming.dstream.StateDStream - Restoring checkpoint data
  15. 2015-10-18 14:18:36,029 [Driver] INFO org.apache.spark.streaming.dstream.DStreamCheckpointData - Restoring checkpointed RDD for time 1445170260000 ms from file 'hdfs://root/checkpoint/4c53b3c0-ec8c-43d1-b859-9f0e42ffcbdb/rdd-282878'
  16. 2015-10-18 14:18:36,314 [Driver] INFO org.apache.spark.SparkContext - Created broadcast 0 from checkpointFile at DStreamCheckpointData.scala:112
  17. 2015-10-18 14:18:36,317 [Driver] INFO org.apache.spark.streaming.dstream.DStreamCheckpointData - Restoring checkpointed RDD for time 1445170360000 ms from file 'hdfs://root/checkpoint/4c53b3c0-ec8c-43d1-b859-9f0e42ffcbdb/rdd-283039'
  18. 2015-10-18 14:18:36,337 [Driver] INFO org.apache.spark.SparkContext - Created broadcast 1 from checkpointFile at DStreamCheckpointData.scala:112
  19. 2015-10-18 14:18:36,338 [Driver] INFO org.apache.spark.streaming.dstream.TransformedDStream - Restoring checkpoint data
  20. 2015-10-18 14:18:36,338 [Driver] INFO org.apache.spark.streaming.dstream.MappedDStream - Restoring checkpoint data
  21. 2015-10-18 14:18:36,338 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream - Restoring checkpoint data
  22. 2015-10-18 14:18:36,440 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170180000 ms [(events,26,66377161,66377193), (events,0,66389136,66389168), (events,29,66379223,66379259), (events,18,66382974,66383001), (events,17,66453580,66453626), (events,9,66388003,66388037), (events,23,66354829,66354858), (events,1,66384954,66384991), (events,22,66390939,66390970), (events,28,66378274,66378309), (events,11,66398920,66398943), (events,25,66396362,66396395), (events,33,66385295,66385339), (events,27,66384972,66385010), (events,7,66392743,66392774), (events,21,66385861,66385885), (events,10,66461669,66461704), (events,8,66387649,66387686), (events,2,66386911,66386940), (events,4,66395213,66395255), (events,24,66464895,66464923), (events,16,66468866,66468905), (events,31,66457615,66457644), (events,15,66397487,66397526), (events,6,66391641,66391666), (events,35,66383046,66383076), (events,19,66363890,66363931), (events,14,66387023,66387050), (events,12,66367170,66367206), (events,32,66382753,66382784), (events,30,66376347,66376378), (events,3,66465945,66465981), (events,5,66369157,66369188), (events,20,66387839,66387875), (events,13,66378191,66378223), (events,34,66399785,66399819)]
  23. 2015-10-18 14:18:36,444 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170200000 ms [(events,26,66377193,66377230), (events,0,66389168,66389210), (events,29,66379259,66379288), (events,18,66383001,66383027), (events,17,66453626,66453660), (events,9,66388037,66388070), (events,23,66354858,66354887), (events,1,66384991,66385026), (events,22,66390970,66391004), (events,28,66378309,66378344), (events,11,66398943,66398977), (events,25,66396395,66396429), (events,33,66385339,66385378), (events,27,66385010,66385047), (events,7,66392774,66392811), (events,21,66385885,66385925), (events,10,66461704,66461736), (events,8,66387686,66387718), (events,2,66386940,66386980), (events,4,66395255,66395298), (events,24,66464923,66464964), (events,16,66468905,66468939), (events,31,66457644,66457678), (events,15,66397526,66397555), (events,6,66391666,66391690), (events,35,66383076,66383099), (events,19,66363931,66363959), (events,14,66387050,66387082), (events,12,66367206,66367230), (events,32,66382784,66382819), (events,30,66376378,66376412), (events,3,66465981,66466014), (events,5,66369188,66369228), (events,20,66387875,66387909), (events,13,66378223,66378250), (events,34,66399819,66399845)]
  24. 2015-10-18 14:18:36,444 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170220000 ms [(events,26,66377230,66377255), (events,0,66389210,66389242), (events,29,66379288,66379308), (events,18,66383027,66383063), (events,17,66453660,66453689), (events,9,66388070,66388108), (events,23,66354887,66354924), (events,1,66385026,66385061), (events,22,66391004,66391040), (events,28,66378344,66378379), (events,11,66398977,66399009), (events,25,66396429,66396464), (events,33,66385378,66385412), (events,27,66385047,66385090), (events,7,66392811,66392840), (events,21,66385925,66385955), (events,10,66461736,66461772), (events,8,66387718,66387759), (events,2,66386980,66387011), (events,4,66395298,66395325), (events,24,66464964,66464987), (events,16,66468939,66468983), (events,31,66457678,66457722), (events,15,66397555,66397588), (events,6,66391690,66391720), (events,35,66383099,66383134), (events,19,66363959,66363995), (events,14,66387082,66387113), (events,12,66367230,66367260), (events,32,66382819,66382848), (events,30,66376412,66376445), (events,3,66466014,66466057), (events,5,66369228,66369259), (events,20,66387909,66387950), (events,13,66378250,66378282), (events,34,66399845,66399869)]
  25. 2015-10-18 14:18:36,444 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170240000 ms [(events,26,66377255,66377290), (events,0,66389242,66389274), (events,29,66379308,66379341), (events,18,66383063,66383092), (events,17,66453689,66453727), (events,9,66388108,66388130), (events,23,66354924,66354962), (events,1,66385061,66385103), (events,22,66391040,66391070), (events,28,66378379,66378413), (events,11,66399009,66399034), (events,25,66396464,66396503), (events,33,66385412,66385444), (events,27,66385090,66385124), (events,7,66392840,66392878), (events,21,66385955,66385997), (events,10,66461772,66461795), (events,8,66387759,66387798), (events,2,66387011,66387043), (events,4,66395325,66395362), (events,24,66464987,66465014), (events,16,66468983,66469006), (events,31,66457722,66457770), (events,15,66397588,66397615), (events,6,66391720,66391758), (events,35,66383134,66383165), (events,19,66363995,66364028), (events,14,66387113,66387146), (events,12,66367260,66367293), (events,32,66382848,66382887), (events,30,66376445,66376471), (events,3,66466057,66466087), (events,5,66369259,66369296), (events,20,66387950,66387982), (events,13,66378282,66378318), (events,34,66399869,66399902)]
  26. 2015-10-18 14:18:36,445 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170260000 ms [(events,26,66377290,66377323), (events,0,66389274,66389300), (events,29,66379341,66379370), (events,18,66383092,66383127), (events,17,66453727,66453762), (events,9,66388130,66388165), (events,23,66354962,66354991), (events,1,66385103,66385132), (events,22,66391070,66391113), (events,28,66378413,66378441), (events,11,66399034,66399070), (events,25,66396503,66396538), (events,33,66385444,66385478), (events,27,66385124,66385146), (events,7,66392878,66392913), (events,21,66385997,66386023), (events,10,66461795,66461832), (events,8,66387798,66387827), (events,2,66387043,66387082), (events,4,66395362,66395402), (events,24,66465014,66465048), (events,16,66469006,66469027), (events,31,66457770,66457811), (events,15,66397615,66397653), (events,6,66391758,66391794), (events,35,66383165,66383205), (events,19,66364028,66364061), (events,14,66387146,66387174), (events,12,66367293,66367338), (events,32,66382887,66382914), (events,30,66376471,66376512), (events,3,66466087,66466115), (events,5,66369296,66369331), (events,20,66387982,66388017), (events,13,66378318,66378351), (events,34,66399902,66399932)]
  27. 2015-10-18 14:18:36,445 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170280000 ms [(events,26,66377323,66377349), (events,0,66389300,66389336), (events,29,66379370,66379402), (events,18,66383127,66383165), (events,17,66453762,66453785), (events,9,66388165,66388207), (events,23,66354991,66355022), (events,1,66385132,66385167), (events,22,66391113,66391150), (events,28,66378441,66378481), (events,11,66399070,66399108), (events,25,66396538,66396566), (events,33,66385478,66385512), (events,27,66385146,66385171), (events,7,66392913,66392936), (events,21,66386023,66386049), (events,10,66461832,66461870), (events,8,66387827,66387861), (events,2,66387082,66387114), (events,4,66395402,66395431), (events,24,66465048,66465096), (events,16,66469027,66469069), (events,31,66457811,66457847), (events,15,66397653,66397677), (events,6,66391794,66391823), (events,35,66383205,66383230), (events,19,66364061,66364094), (events,14,66387174,66387222), (events,12,66367338,66367370), (events,32,66382914,66382951), (events,30,66376512,66376541), (events,3,66466115,66466154), (events,5,66369331,66369354), (events,20,66388017,66388054), (events,13,66378351,66378385), (events,34,66399932,66399969)]
  28. 2015-10-18 14:18:36,446 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170300000 ms [(events,26,66377349,66377383), (events,0,66389336,66389362), (events,29,66379402,66379441), (events,18,66383165,66383200), (events,17,66453785,66453814), (events,9,66388207,66388234), (events,23,66355022,66355056), (events,1,66385167,66385209), (events,22,66391150,66391186), (events,28,66378481,66378511), (events,11,66399108,66399140), (events,25,66396566,66396591), (events,33,66385512,66385550), (events,27,66385171,66385199), (events,7,66392936,66392971), (events,21,66386049,66386073), (events,10,66461870,66461906), (events,8,66387861,66387895), (events,2,66387114,66387151), (events,4,66395431,66395457), (events,24,66465096,66465138), (events,16,66469069,66469110), (events,31,66457847,66457874), (events,15,66397677,66397715), (events,6,66391823,66391864), (events,35,66383230,66383259), (events,19,66364094,66364123), (events,14,66387222,66387252), (events,12,66367370,66367396), (events,32,66382951,66382986), (events,30,66376541,66376585), (events,3,66466154,66466184), (events,5,66369354,66369399), (events,20,66388054,66388083), (events,13,66378385,66378426), (events,34,66399969,66399995)]
  29. 2015-10-18 14:18:36,446 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170320000 ms [(events,26,66377383,66377420), (events,0,66389362,66389392), (events,29,66379441,66379496), (events,18,66383200,66383242), (events,17,66453814,66453860), (events,9,66388234,66388286), (events,23,66355056,66355097), (events,1,66385209,66385253), (events,22,66391186,66391218), (events,28,66378511,66378546), (events,11,66399140,66399186), (events,25,66396591,66396630), (events,33,66385550,66385596), (events,27,66385199,66385239), (events,7,66392971,66393015), (events,21,66386073,66386110), (events,10,66461906,66461933), (events,8,66387895,66387944), (events,2,66387151,66387198), (events,4,66395457,66395491), (events,24,66465138,66465174), (events,16,66469110,66469152), (events,31,66457874,66457926), (events,15,66397715,66397753), (events,6,66391864,66391907), (events,35,66383259,66383313), (events,19,66364123,66364165), (events,14,66387252,66387288), (events,12,66367396,66367430), (events,32,66382986,66383034), (events,30,66376585,66376631), (events,3,66466184,66466221), (events,5,66369399,66369436), (events,20,66388083,66388124), (events,13,66378426,66378477), (events,34,66399995,66400035)]
  30. 2015-10-18 14:18:36,447 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170340000 ms [(events,26,66377420,66377448), (events,0,66389392,66389419), (events,29,66379496,66379545), (events,18,66383242,66383275), (events,17,66453860,66453893), (events,9,66388286,66388320), (events,23,66355097,66355143), (events,1,66385253,66385303), (events,22,66391218,66391258), (events,28,66378546,66378596), (events,11,66399186,66399242), (events,25,66396630,66396663), (events,33,66385596,66385636), (events,27,66385239,66385271), (events,7,66393015,66393056), (events,21,66386110,66386143), (events,10,66461933,66461972), (events,8,66387944,66387980), (events,2,66387198,66387234), (events,4,66395491,66395531), (events,24,66465174,66465224), (events,16,66469152,66469192), (events,31,66457926,66457957), (events,15,66397753,66397781), (events,6,66391907,66391949), (events,35,66383313,66383350), (events,19,66364165,66364204), (events,14,66387288,66387329), (events,12,66367430,66367469), (events,32,66383034,66383064), (events,30,66376631,66376675), (events,3,66466221,66466257), (events,5,66369436,66369475), (events,20,66388124,66388164), (events,13,66378477,66378531), (events,34,66400035,66400069)]
  31. 2015-10-18 14:18:36,448 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170360000 ms [(events,26,66377448,66377504), (events,0,66389419,66389475), (events,29,66379545,66379590), (events,18,66383275,66383333), (events,17,66453893,66453932), (events,9,66388320,66388375), (events,23,66355143,66355190), (events,1,66385303,66385358), (events,22,66391258,66391317), (events,28,66378596,66378651), (events,11,66399242,66399283), (events,25,66396663,66396712), (events,33,66385636,66385683), (events,27,66385271,66385311), (events,7,66393056,66393118), (events,21,66386143,66386192), (events,10,66461972,66462027), (events,8,66387980,66388039), (events,2,66387234,66387280), (events,4,66395531,66395588), (events,24,66465224,66465276), (events,16,66469192,66469250), (events,31,66457957,66458027), (events,15,66397781,66397836), (events,6,66391949,66392008), (events,35,66383350,66383400), (events,19,66364204,66364250), (events,14,66387329,66387390), (events,12,66367469,66367526), (events,32,66383064,66383118), (events,30,66376675,66376720), (events,3,66466257,66466309), (events,5,66369475,66369529), (events,20,66388164,66388209), (events,13,66378531,66378589), (events,34,66400069,66400123)]
  32. 2015-10-18 14:18:36,448 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170380000 ms [(events,26,66377504,66377566), (events,0,66389475,66389542), (events,29,66379590,66379666), (events,18,66383333,66383387), (events,17,66453932,66454008), (events,9,66388375,66388446), (events,23,66355190,66355268), (events,1,66385358,66385436), (events,22,66391317,66391390), (events,28,66378651,66378731), (events,11,66399283,66399347), (events,25,66396712,66396775), (events,33,66385683,66385757), (events,27,66385311,66385395), (events,7,66393118,66393184), (events,21,66386192,66386252), (events,10,66462027,66462103), (events,8,66388039,66388102), (events,2,66387280,66387352), (events,4,66395588,66395654), (events,24,66465276,66465350), (events,16,66469250,66469323), (events,31,66458027,66458090), (events,15,66397836,66397896), (events,6,66392008,66392073), (events,35,66383400,66383474), (events,19,66364250,66364327), (events,14,66387390,66387469), (events,12,66367526,66367598), (events,32,66383118,66383176), (events,30,66376720,66376790), (events,3,66466309,66466379), (events,5,66369529,66369600), (events,20,66388209,66388274), (events,13,66378589,66378648), (events,34,66400123,66400190)]
  33. 2015-10-18 14:18:36,449 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170400000 ms [(events,26,66377566,66377577), (events,0,66389542,66389562), (events,29,66379666,66379682), (events,18,66383387,66383403), (events,17,66454008,66454020), (events,9,66388446,66388451), (events,23,66355268,66355289), (events,1,66385436,66385451), (events,22,66391390,66391402), (events,28,66378731,66378749), (events,11,66399347,66399368), (events,25,66396775,66396783), (events,33,66385757,66385778), (events,27,66385395,66385405), (events,7,66393184,66393191), (events,21,66386252,66386267), (events,10,66462103,66462120), (events,8,66388102,66388111), (events,2,66387352,66387364), (events,4,66395654,66395667), (events,24,66465350,66465365), (events,16,66469323,66469332), (events,31,66458090,66458108), (events,15,66397896,66397906), (events,6,66392073,66392086), (events,35,66383474,66383483), (events,19,66364327,66364346), (events,14,66387469,66387493), (events,12,66367598,66367613), (events,32,66383176,66383186), (events,30,66376790,66376807), (events,3,66466379,66466391), (events,5,66369600,66369613), (events,20,66388274,66388285), (events,13,66378648,66378663), (events,34,66400190,66400201)]
  34. 2015-10-18 14:18:36,449 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170420000 ms [(events,26,66377577,66377623), (events,0,66389562,66389596), (events,29,66379682,66379727), (events,18,66383403,66383457), (events,17,66454020,66454064), (events,9,66388451,66388496), (events,23,66355289,66355348), (events,1,66385451,66385514), (events,22,66391402,66391453), (events,28,66378749,66378790), (events,11,66399368,66399426), (events,25,66396783,66396832), (events,33,66385778,66385833), (events,27,66385405,66385447), (events,7,66393191,66393240), (events,21,66386267,66386315), (events,10,66462120,66462176), (events,8,66388111,66388156), (events,2,66387364,66387405), (events,4,66395667,66395705), (events,24,66465365,66465411), (events,16,66469332,66469389), (events,31,66458108,66458153), (events,15,66397906,66397951), (events,6,66392086,66392140), (events,35,66383483,66383538), (events,19,66364346,66364388), (events,14,66387493,66387552), (events,12,66367613,66367676), (events,32,66383186,66383242), (events,30,66376807,66376858), (events,3,66466391,66466451), (events,5,66369613,66369669), (events,20,66388285,66388337), (events,13,66378663,66378701), (events,34,66400201,66400259)]
  35. 2015-10-18 14:18:36,449 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170440000 ms [(events,26,66377623,66377664), (events,0,66389596,66389660), (events,29,66379727,66379777), (events,18,66383457,66383509), (events,17,66454064,66454113), (events,9,66388496,66388542), (events,23,66355348,66355401), (events,1,66385514,66385566), (events,22,66391453,66391516), (events,28,66378790,66378848), (events,11,66399426,66399484), (events,25,66396832,66396878), (events,33,66385833,66385884), (events,27,66385447,66385508), (events,7,66393240,66393286), (events,21,66386315,66386346), (events,10,66462176,66462228), (events,8,66388156,66388196), (events,2,66387405,66387452), (events,4,66395705,66395756), (events,24,66465411,66465451), (events,16,66469389,66469419), (events,31,66458153,66458208), (events,15,66397951,66398003), (events,6,66392140,66392187), (events,35,66383538,66383593), (events,19,66364388,66364446), (events,14,66387552,66387600), (events,12,66367676,66367724), (events,32,66383242,66383297), (events,30,66376858,66376915), (events,3,66466451,66466503), (events,5,66369669,66369723), (events,20,66388337,66388383), (events,13,66378701,66378747), (events,34,66400259,66400305)]
  36. 2015-10-18 14:18:36,450 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170460000 ms [(events,26,66377664,66377694), (events,0,66389660,66389703), (events,29,66379777,66379815), (events,18,66383509,66383537), (events,17,66454113,66454154), (events,9,66388542,66388586), (events,23,66355401,66355436), (events,1,66385566,66385594), (events,22,66391516,66391553), (events,28,66378848,66378875), (events,11,66399484,66399512), (events,25,66396878,66396905), (events,33,66385884,66385915), (events,27,66385508,66385548), (events,7,66393286,66393310), (events,21,66386346,66386381), (events,10,66462228,66462264), (events,8,66388196,66388225), (events,2,66387452,66387491), (events,4,66395756,66395787), (events,24,66465451,66465487), (events,16,66469419,66469456), (events,31,66458208,66458236), (events,15,66398003,66398043), (events,6,66392187,66392217), (events,35,66383593,66383631), (events,19,66364446,66364475), (events,14,66387600,66387625), (events,12,66367724,66367754), (events,32,66383297,66383335), (events,30,66376915,66376942), (events,3,66466503,66466538), (events,5,66369723,66369758), (events,20,66388383,66388423), (events,13,66378747,66378774), (events,34,66400305,66400339)]
  37. 2015-10-18 14:18:36,450 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170480000 ms [(events,26,66377694,66377724), (events,0,66389703,66389737), (events,29,66379815,66379849), (events,18,66383537,66383566), (events,17,66454154,66454190), (events,9,66388586,66388608), (events,23,66355436,66355460), (events,1,66385594,66385630), (events,22,66391553,66391583), (events,28,66378875,66378909), (events,11,66399512,66399542), (events,25,66396905,66396936), (events,33,66385915,66385950), (events,27,66385548,66385587), (events,7,66393310,66393352), (events,21,66386381,66386421), (events,10,66462264,66462298), (events,8,66388225,66388252), (events,2,66387491,66387525), (events,4,66395787,66395826), (events,24,66465487,66465523), (events,16,66469456,66469485), (events,31,66458236,66458278), (events,15,66398043,66398088), (events,6,66392217,66392252), (events,35,66383631,66383662), (events,19,66364475,66364503), (events,14,66387625,66387651), (events,12,66367754,66367787), (events,32,66383335,66383376), (events,30,66376942,66376968), (events,3,66466538,66466562), (events,5,66369758,66369798), (events,20,66388423,66388453), (events,13,66378774,66378809), (events,34,66400339,66400378)]
  38. 2015-10-18 14:18:36,451 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170500000 ms [(events,26,66377724,66377755), (events,0,66389737,66389767), (events,29,66379849,66379889), (events,18,66383566,66383593), (events,17,66454190,66454235), (events,9,66388608,66388645), (events,23,66355460,66355497), (events,1,66385630,66385659), (events,22,66391583,66391617), (events,28,66378909,66378944), (events,11,66399542,66399570), (events,25,66396936,66396966), (events,33,66385950,66385980), (events,27,66385587,66385628), (events,7,66393352,66393385), (events,21,66386421,66386460), (events,10,66462298,66462335), (events,8,66388252,66388286), (events,2,66387525,66387566), (events,4,66395826,66395855), (events,24,66465523,66465558), (events,16,66469485,66469513), (events,31,66458278,66458306), (events,15,66398088,66398133), (events,6,66392252,66392281), (events,35,66383662,66383698), (events,19,66364503,66364530), (events,14,66387651,66387679), (events,12,66367787,66367815), (events,32,66383376,66383408), (events,30,66376968,66377010), (events,3,66466562,66466587), (events,5,66369798,66369834), (events,20,66388453,66388492), (events,13,66378809,66378841), (events,34,66400378,66400401)]
  39. 2015-10-18 14:18:36,451 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170520000 ms [(events,26,66377755,66377795), (events,0,66389767,66389799), (events,29,66379889,66379926), (events,18,66383593,66383628), (events,17,66454235,66454267), (events,9,66388645,66388678), (events,23,66355497,66355532), (events,1,66385659,66385688), (events,22,66391617,66391654), (events,28,66378944,66378983), (events,11,66399570,66399609), (events,25,66396966,66397005), (events,33,66385980,66386007), (events,27,66385628,66385652), (events,7,66393385,66393422), (events,21,66386460,66386493), (events,10,66462335,66462373), (events,8,66388286,66388312), (events,2,66387566,66387590), (events,4,66395855,66395894), (events,24,66465558,66465590), (events,16,66469513,66469548), (events,31,66458306,66458343), (events,15,66398133,66398165), (events,6,66392281,66392320), (events,35,66383698,66383739), (events,19,66364530,66364571), (events,14,66387679,66387716), (events,12,66367815,66367856), (events,32,66383408,66383446), (events,30,66377010,66377043), (events,3,66466587,66466616), (events,5,66369834,66369871), (events,20,66388492,66388530), (events,13,66378841,66378874), (events,34,66400401,66400430)]
  40. 2015-10-18 14:18:36,452 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170540000 ms [(events,26,66377795,66377835), (events,0,66389799,66389828), (events,29,66379926,66379968), (events,18,66383628,66383665), (events,17,66454267,66454311), (events,9,66388678,66388702), (events,23,66355532,66355564), (events,1,66385688,66385715), (events,22,66391654,66391683), (events,28,66378983,66379017), (events,11,66399609,66399653), (events,25,66397005,66397041), (events,33,66386007,66386046), (events,27,66385652,66385681), (events,7,66393422,66393455), (events,21,66386493,66386522), (events,10,66462373,66462402), (events,8,66388312,66388341), (events,2,66387590,66387618), (events,4,66395894,66395932), (events,24,66465590,66465625), (events,16,66469548,66469578), (events,31,66458343,66458377), (events,15,66398165,66398188), (events,6,66392320,66392339), (events,35,66383739,66383766), (events,19,66364571,66364598), (events,14,66387716,66387738), (events,12,66367856,66367883), (events,32,66383446,66383481), (events,30,66377043,66377077), (events,3,66466616,66466641), (events,5,66369871,66369907), (events,20,66388530,66388566), (events,13,66378874,66378911), (events,34,66400430,66400464)]
  41. 2015-10-18 14:18:36,452 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170560000 ms [(events,26,66377835,66377867), (events,0,66389828,66389864), (events,29,66379968,66379992), (events,18,66383665,66383696), (events,17,66454311,66454345), (events,9,66388702,66388731), (events,23,66355564,66355603), (events,1,66385715,66385750), (events,22,66391683,66391716), (events,28,66379017,66379047), (events,11,66399653,66399678), (events,25,66397041,66397071), (events,33,66386046,66386071), (events,27,66385681,66385718), (events,7,66393455,66393487), (events,21,66386522,66386557), (events,10,66462402,66462438), (events,8,66388341,66388374), (events,2,66387618,66387650), (events,4,66395932,66395967), (events,24,66465625,66465664), (events,16,66469578,66469612), (events,31,66458377,66458403), (events,15,66398188,66398219), (events,6,66392339,66392372), (events,35,66383766,66383802), (events,19,66364598,66364635), (events,14,66387738,66387766), (events,12,66367883,66367910), (events,32,66383481,66383534), (events,30,66377077,66377109), (events,3,66466641,66466682), (events,5,66369907,66369939), (events,20,66388566,66388602), (events,13,66378911,66378948), (events,34,66400464,66400499)]
  42. 2015-10-18 14:18:36,453 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170580000 ms [(events,26,66377867,66377909), (events,0,66389864,66389903), (events,29,66379992,66380022), (events,18,66383696,66383724), (events,17,66454345,66454379), (events,9,66388731,66388759), (events,23,66355603,66355642), (events,1,66385750,66385775), (events,22,66391716,66391746), (events,28,66379047,66379070), (events,11,66399678,66399714), (events,25,66397071,66397109), (events,33,66386071,66386104), (events,27,66385718,66385754), (events,7,66393487,66393526), (events,21,66386557,66386592), (events,10,66462438,66462474), (events,8,66388374,66388400), (events,2,66387650,66387678), (events,4,66395967,66396000), (events,24,66465664,66465696), (events,16,66469612,66469647), (events,31,66458403,66458432), (events,15,66398219,66398249), (events,6,66392372,66392403), (events,35,66383802,66383840), (events,19,66364635,66364673), (events,14,66387766,66387792), (events,12,66367910,66367949), (events,32,66383534,66383572), (events,30,66377109,66377145), (events,3,66466682,66466714), (events,5,66369939,66369971), (events,20,66388602,66388636), (events,13,66378948,66378982), (events,34,66400499,66400537)]
  43. 2015-10-18 14:18:36,453 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170600000 ms [(events,26,66377909,66377941), (events,0,66389903,66389934), (events,29,66380022,66380050), (events,18,66383724,66383767), (events,17,66454379,66454420), (events,9,66388759,66388792), (events,23,66355642,66355680), (events,1,66385775,66385811), (events,22,66391746,66391769), (events,28,66379070,66379097), (events,11,66399714,66399741), (events,25,66397109,66397146), (events,33,66386104,66386136), (events,27,66385754,66385789), (events,7,66393526,66393563), (events,21,66386592,66386620), (events,10,66462474,66462501), (events,8,66388400,66388438), (events,2,66387678,66387708), (events,4,66396000,66396021), (events,24,66465696,66465731), (events,16,66469647,66469684), (events,31,66458432,66458462), (events,15,66398249,66398285), (events,6,66392403,66392452), (events,35,66383840,66383882), (events,19,66364673,66364699), (events,14,66387792,66387817), (events,12,66367949,66367974), (events,32,66383572,66383614), (events,30,66377145,66377186), (events,3,66466714,66466742), (events,5,66369971,66370004), (events,20,66388636,66388667), (events,13,66378982,66379017), (events,34,66400537,66400578)]
  44. 2015-10-18 14:18:36,454 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170620000 ms [(events,26,66377941,66377965), (events,0,66389934,66389962), (events,29,66380050,66380077), (events,18,66383767,66383801), (events,17,66454420,66454453), (events,9,66388792,66388827), (events,23,66355680,66355717), (events,1,66385811,66385843), (events,22,66391769,66391803), (events,28,66379097,66379139), (events,11,66399741,66399775), (events,25,66397146,66397185), (events,33,66386136,66386174), (events,27,66385789,66385831), (events,7,66393563,66393596), (events,21,66386620,66386652), (events,10,66462501,66462533), (events,8,66388438,66388475), (events,2,66387708,66387750), (events,4,66396021,66396049), (events,24,66465731,66465772), (events,16,66469684,66469720), (events,31,66458462,66458497), (events,15,66398285,66398310), (events,6,66392452,66392489), (events,35,66383882,66383913), (events,19,66364699,66364730), (events,14,66387817,66387843), (events,12,66367974,66368011), (events,32,66383614,66383645), (events,30,66377186,66377219), (events,3,66466742,66466776), (events,5,66370004,66370033), (events,20,66388667,66388696), (events,13,66379017,66379047), (events,34,66400578,66400610)]
  45. 2015-10-18 14:18:36,454 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170640000 ms [(events,26,66377965,66378033), (events,0,66389962,66390015), (events,29,66380077,66380121), (events,18,66383801,66383865), (events,17,66454453,66454518), (events,9,66388827,66388875), (events,23,66355717,66355775), (events,1,66385843,66385914), (events,22,66391803,66391858), (events,28,66379139,66379191), (events,11,66399775,66399835), (events,25,66397185,66397240), (events,33,66386174,66386239), (events,27,66385831,66385896), (events,7,66393596,66393651), (events,21,66386652,66386714), (events,10,66462533,66462596), (events,8,66388475,66388540), (events,2,66387750,66387809), (events,4,66396049,66396113), (events,24,66465772,66465837), (events,16,66469720,66469787), (events,31,66458497,66458560), (events,15,66398310,66398374), (events,6,66392489,66392563), (events,35,66383913,66383974), (events,19,66364730,66364785), (events,14,66387843,66387906), (events,12,66368011,66368096), (events,32,66383645,66383707), (events,30,66377219,66377271), (events,3,66466776,66466827), (events,5,66370033,66370097), (events,20,66388696,66388750), (events,13,66379047,66379107), (events,34,66400610,66400679)]
  46. 2015-10-18 14:18:36,454 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170660000 ms [(events,26,66378033,66378053), (events,0,66390015,66390025), (events,29,66380121,66380136), (events,18,66383865,66383879), (events,17,66454518,66454538), (events,9,66388875,66388891), (events,23,66355775,66355787), (events,1,66385914,66385929), (events,22,66391858,66391871), (events,28,66379191,66379206), (events,11,66399835,66399854), (events,25,66397240,66397257), (events,33,66386239,66386254), (events,27,66385896,66385904), (events,7,66393651,66393661), (events,21,66386714,66386731), (events,10,66462596,66462613), (events,8,66388540,66388562), (events,2,66387809,66387826), (events,4,66396113,66396132), (events,24,66465837,66465860), (events,16,66469787,66469805), (events,31,66458560,66458581), (events,15,66398374,66398394), (events,6,66392563,66392583), (events,35,66383974,66383992), (events,19,66364785,66364801), (events,14,66387906,66387926), (events,12,66368096,66368110), (events,32,66383707,66383721), (events,30,66377271,66377286), (events,3,66466827,66466849), (events,5,66370097,66370119), (events,20,66388750,66388771), (events,13,66379107,66379119), (events,34,66400679,66400692)]
  47. 2015-10-18 14:18:36,455 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170680000 ms [(events,26,66378053,66378098), (events,0,66390025,66390058), (events,29,66380136,66380177), (events,18,66383879,66383923), (events,17,66454538,66454582), (events,9,66388891,66388932), (events,23,66355787,66355818), (events,1,66385929,66385967), (events,22,66391871,66391914), (events,28,66379206,66379254), (events,11,66399854,66399902), (events,25,66397257,66397297), (events,33,66386254,66386287), (events,27,66385904,66385945), (events,7,66393661,66393708), (events,21,66386731,66386780), (events,10,66462613,66462659), (events,8,66388562,66388607), (events,2,66387826,66387853), (events,4,66396132,66396173), (events,24,66465860,66465911), (events,16,66469805,66469851), (events,31,66458581,66458623), (events,15,66398394,66398436), (events,6,66392583,66392633), (events,35,66383992,66384028), (events,19,66364801,66364840), (events,14,66387926,66387965), (events,12,66368110,66368156), (events,32,66383721,66383761), (events,30,66377286,66377330), (events,3,66466849,66466891), (events,5,66370119,66370152), (events,20,66388771,66388815), (events,13,66379119,66379157), (events,34,66400692,66400735)]
  48. 2015-10-18 14:18:36,455 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170700000 ms [(events,26,66378098,66378159), (events,0,66390058,66390115), (events,29,66380177,66380235), (events,18,66383923,66383981), (events,17,66454582,66454645), (events,9,66388932,66389010), (events,23,66355818,66355886), (events,1,66385967,66386028), (events,22,66391914,66391984), (events,28,66379254,66379306), (events,11,66399902,66399971), (events,25,66397297,66397369), (events,33,66386287,66386354), (events,27,66385945,66386010), (events,7,66393708,66393769), (events,21,66386780,66386835), (events,10,66462659,66462729), (events,8,66388607,66388670), (events,2,66387853,66387915), (events,4,66396173,66396248), (events,24,66465911,66465958), (events,16,66469851,66469907), (events,31,66458623,66458685), (events,15,66398436,66398502), (events,6,66392633,66392702), (events,35,66384028,66384099), (events,19,66364840,66364901), (events,14,66387965,66388035), (events,12,66368156,66368216), (events,32,66383761,66383820), (events,30,66377330,66377400), (events,3,66466891,66466948), (events,5,66370152,66370216), (events,20,66388815,66388885), (events,13,66379157,66379216), (events,34,66400735,66400803)]
  49. 2015-10-18 14:18:36,456 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170720000 ms [(events,26,66378159,66378191), (events,0,66390115,66390153), (events,29,66380235,66380260), (events,18,66383981,66384013), (events,17,66454645,66454678), (events,9,66389010,66389045), (events,23,66355886,66355916), (events,1,66386028,66386062), (events,22,66391984,66392017), (events,28,66379306,66379345), (events,11,66399971,66400012), (events,25,66397369,66397408), (events,33,66386354,66386385), (events,27,66386010,66386047), (events,7,66393769,66393808), (events,21,66386835,66386865), (events,10,66462729,66462763), (events,8,66388670,66388699), (events,2,66387915,66387947), (events,4,66396248,66396286), (events,24,66465958,66465997), (events,16,66469907,66469942), (events,31,66458685,66458729), (events,15,66398502,66398529), (events,6,66392702,66392727), (events,35,66384099,66384130), (events,19,66364901,66364945), (events,14,66388035,66388059), (events,12,66368216,66368250), (events,32,66383820,66383858), (events,30,66377400,66377436), (events,3,66466948,66466981), (events,5,66370216,66370246), (events,20,66388885,66388930), (events,13,66379216,66379259), (events,34,66400803,66400832)]
  50. 2015-10-18 14:18:36,456 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170740000 ms [(events,26,66378191,66378237), (events,0,66390153,66390196), (events,29,66380260,66380297), (events,18,66384013,66384065), (events,17,66454678,66454708), (events,9,66389045,66389082), (events,23,66355916,66355963), (events,1,66386062,66386103), (events,22,66392017,66392064), (events,28,66379345,66379382), (events,11,66400012,66400052), (events,25,66397408,66397452), (events,33,66386385,66386427), (events,27,66386047,66386093), (events,7,66393808,66393861), (events,21,66386865,66386908), (events,10,66462763,66462815), (events,8,66388699,66388754), (events,2,66387947,66387999), (events,4,66396286,66396326), (events,24,66465997,66466039), (events,16,66469942,66469985), (events,31,66458729,66458770), (events,15,66398529,66398563), (events,6,66392727,66392775), (events,35,66384130,66384174), (events,19,66364945,66364985), (events,14,66388059,66388107), (events,12,66368250,66368286), (events,32,66383858,66383908), (events,30,66377436,66377478), (events,3,66466981,66467029), (events,5,66370246,66370285), (events,20,66388930,66388982), (events,13,66379259,66379292), (events,34,66400832,66400876)]
  51. 2015-10-18 14:18:36,456 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170760000 ms [(events,26,66378237,66378276), (events,0,66390196,66390247), (events,29,66380297,66380356), (events,18,66384065,66384124), (events,17,66454708,66454759), (events,9,66389082,66389133), (events,23,66355963,66356012), (events,1,66386103,66386165), (events,22,66392064,66392105), (events,28,66379382,66379437), (events,11,66400052,66400112), (events,25,66397452,66397502), (events,33,66386427,66386493), (events,27,66386093,66386156), (events,7,66393861,66393917), (events,21,66386908,66386947), (events,10,66462815,66462874), (events,8,66388754,66388806), (events,2,66387999,66388055), (events,4,66396326,66396378), (events,24,66466039,66466092), (events,16,66469985,66470050), (events,31,66458770,66458833), (events,15,66398563,66398623), (events,6,66392775,66392838), (events,35,66384174,66384230), (events,19,66364985,66365034), (events,14,66388107,66388176), (events,12,66368286,66368340), (events,32,66383908,66383979), (events,30,66377478,66377525), (events,3,66467029,66467083), (events,5,66370285,66370339), (events,20,66388982,66389041), (events,13,66379292,66379359), (events,34,66400876,66400922)]
  52. 2015-10-18 14:18:36,457 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170780000 ms [(events,26,66378276,66378308), (events,0,66390247,66390285), (events,29,66380356,66380385), (events,18,66384124,66384164), (events,17,66454759,66454786), (events,9,66389133,66389169), (events,23,66356012,66356051), (events,1,66386165,66386184), (events,22,66392105,66392135), (events,28,66379437,66379469), (events,11,66400112,66400143), (events,25,66397502,66397538), (events,33,66386493,66386518), (events,27,66386156,66386184), (events,7,66393917,66393947), (events,21,66386947,66386977), (events,10,66462874,66462912), (events,8,66388806,66388831), (events,2,66388055,66388095), (events,4,66396378,66396412), (events,24,66466092,66466132), (events,16,66470050,66470091), (events,31,66458833,66458867), (events,15,66398623,66398662), (events,6,66392838,66392883), (events,35,66384230,66384263), (events,19,66365034,66365059), (events,14,66388176,66388211), (events,12,66368340,66368374), (events,32,66383979,66384009), (events,30,66377525,66377555), (events,3,66467083,66467114), (events,5,66370339,66370380), (events,20,66389041,66389083), (events,13,66379359,66379390), (events,34,66400922,66400952)]
  53. 2015-10-18 14:18:36,457 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream$DirectKafkaInputDStreamCheckpointData - Restoring KafkaRDD for time 1445170800000 ms [(events,26,66378308,66378347), (events,0,66390285,66390313), (events,29,66380385,66380431), (events,18,66384164,66384196), (events,17,66454786,66454816), (events,9,66389169,66389205), (events,23,66356051,66356081), (events,1,66386184,66386210), (events,22,66392135,66392172), (events,28,66379469,66379513), (events,11,66400143,66400175), (events,25,66397538,66397567), (events,33,66386518,66386555), (events,27,66386184,66386219), (events,7,66393947,66393978), (events,21,66386977,66387011), (events,10,66462912,66462958), (events,8,66388831,66388874), (events,2,66388095,66388134), (events,4,66396412,66396445), (events,24,66466132,66466165), (events,16,66470091,66470122), (events,31,66458867,66458910), (events,15,66398662,66398699), (events,6,66392883,66392918), (events,35,66384263,66384310), (events,19,66365059,66365094), (events,14,66388211,66388248), (events,12,66368374,66368411), (events,32,66384009,66384054), (events,30,66377555,66377596), (events,3,66467114,66467149), (events,5,66370380,66370414), (events,20,66389083,66389114), (events,13,66379390,66379425), (events,34,66400952,66400989)]
  54. 2015-10-18 14:18:36,458 [Driver] INFO org.apache.spark.streaming.kafka.DirectKafkaInputDStream - Restored checkpoint data
  55. 2015-10-18 14:18:36,458 [Driver] INFO org.apache.spark.streaming.dstream.MappedDStream - Restored checkpoint data
  56. 2015-10-18 14:18:36,458 [Driver] INFO org.apache.spark.streaming.dstream.TransformedDStream - Restored checkpoint data
  57. 2015-10-18 14:18:36,458 [Driver] INFO org.apache.spark.streaming.dstream.StateDStream - Restored checkpoint data
  58. 2015-10-18 14:18:36,458 [Driver] INFO org.apache.spark.streaming.dstream.FlatMapValuedDStream - Restored checkpoint data
  59. 2015-10-18 14:18:36,458 [Driver] INFO org.apache.spark.streaming.dstream.FilteredDStream - Restored checkpoint data
  60. 2015-10-18 14:18:36,458 [Driver] INFO org.apache.spark.streaming.dstream.FlatMappedDStream - Restored checkpoint data
  61. 2015-10-18 14:18:36,458 [Driver] INFO org.apache.spark.streaming.dstream.FlatMappedDStream - Restoring checkpoint data
  62. 2015-10-18 14:18:36,458 [Driver] INFO org.apache.spark.streaming.dstream.FlatMappedDStream - Restoring checkpoint data
  63. 2015-10-18 14:18:36,458 [Driver] INFO org.apache.spark.streaming.dstream.StateDStream - Restoring checkpoint data
  64. 2015-10-18 14:18:36,458 [Driver] INFO org.apache.spark.streaming.dstream.DStreamCheckpointData - Restoring checkpointed RDD for time 1445170340000 ms from file 'hdfs://root/checkpoint/4c53b3c0-ec8c-43d1-b859-9f0e42ffcbdb/rdd-283015'
  65. 2015-10-18 14:18:36,478 [Driver] INFO org.apache.spark.SparkContext - Created broadcast 2 from checkpointFile at DStreamCheckpointData.scala:112
  66. 2015-10-18 14:18:36,480 [Driver] ERROR org.apache.spark.deploy.yarn.ApplicationMaster - User class threw exception: java.lang.IllegalArgumentException: requirement failed: Checkpoint directory does not exist: hdfs://root/checkpoint/4c53b3c0-ec8c-43d1-b859-9f0e42ffcbdb/rdd-283015
  67. java.lang.IllegalArgumentException: requirement failed: Checkpoint directory does not exist: hdfs://root/checkpoint/4c53b3c0-ec8c-43d1-b859-9f0e42ffcbdb/rdd-283015
  68. at org.apache.spark.rdd.ReliableCheckpointRDD.<init>(ReliableCheckpointRDD.scala:45)
  69. at org.apache.spark.SparkContext$$anonfun$checkpointFile$1.apply(SparkContext.scala:1227)
  70. at org.apache.spark.SparkContext$$anonfun$checkpointFile$1.apply(SparkContext.scala:1227)
  71. at org.apache.spark.SparkContext.checkpointFile(SparkContext.scala:1226)
  72. at org.apache.spark.streaming.dstream.DStreamCheckpointData$$anonfun$restore$1.apply(DStreamCheckpointData.scala:112)
  73. at org.apache.spark.streaming.dstream.DStreamCheckpointData$$anonfun$restore$1.apply(DStreamCheckpointData.scala:109)
  74. at org.apache.spark.streaming.dstream.DStreamCheckpointData.restore(DStreamCheckpointData.scala:109)
  75. at org.apache.spark.streaming.dstream.DStream.restoreCheckpointData(DStream.scala:487)
  76. at org.apache.spark.streaming.dstream.DStream$$anonfun$restoreCheckpointData$2.apply(DStream.scala:488)
  77. at org.apache.spark.streaming.dstream.DStream$$anonfun$restoreCheckpointData$2.apply(DStream.scala:488)
  78. at org.apache.spark.streaming.dstream.DStream.restoreCheckpointData(DStream.scala:488)
  79. at org.apache.spark.streaming.dstream.DStream$$anonfun$restoreCheckpointData$2.apply(DStream.scala:488)
  80. at org.apache.spark.streaming.dstream.DStream$$anonfun$restoreCheckpointData$2.apply(DStream.scala:488)
  81. at org.apache.spark.streaming.dstream.DStream.restoreCheckpointData(DStream.scala:488)
  82. at org.apache.spark.streaming.dstream.DStream$$anonfun$restoreCheckpointData$2.apply(DStream.scala:488)
  83. at org.apache.spark.streaming.dstream.DStream$$anonfun$restoreCheckpointData$2.apply(DStream.scala:488)
  84. at org.apache.spark.streaming.dstream.DStream.restoreCheckpointData(DStream.scala:488)
  85. at org.apache.spark.streaming.dstream.DStream$$anonfun$restoreCheckpointData$2.apply(DStream.scala:488)
  86. at org.apache.spark.streaming.dstream.DStream$$anonfun$restoreCheckpointData$2.apply(DStream.scala:488)
  87. at org.apache.spark.streaming.dstream.DStream.restoreCheckpointData(DStream.scala:488)
  88. at org.apache.spark.streaming.DStreamGraph$$anonfun$restoreCheckpointData$2.apply(DStreamGraph.scala:153)
  89. at org.apache.spark.streaming.DStreamGraph$$anonfun$restoreCheckpointData$2.apply(DStreamGraph.scala:153)
  90. at org.apache.spark.streaming.DStreamGraph.restoreCheckpointData(DStreamGraph.scala:153)
  91. 2015-10-18 14:18:36,484 [Driver] INFO org.apache.spark.deploy.yarn.ApplicationMaster - Final app status: FAILED, exitCode: 15, (reason: User class threw exception: java.lang.IllegalArgumentException: requirement failed: Checkpoint directory does not exist: hdfs://root/checkpoint/4c53b3c0-ec8c-43d1-b859-9f0e42ffcbdb/rdd-283015)
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!