API tools faq
paste
Guest User

Samza Bootstrap Bug

a guest
Jan 4th, 2015
214
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 11.59 KB | None | 0 0
raw download clone embed print report
  1. 2015-01-04 14:58:36 BootstrappingChooser [DEBUG] task[] ssp[] offset[] Registering stream partition with offset: SystemStreamPartition [kafka, ops._raw.web_parsed, 0], null
  2. 2015-01-04 14:58:36 BootstrappingChooser [TRACE] task[] ssp[] offset[] Check UPCOMING offset null against null for SystemStreamPartition [kafka, ops._raw.web_parsed, 0].
  3. 2015-01-04 14:58:36 BootstrappingChooser [DEBUG] task[] ssp[] offset[] Registering stream partition with offset: SystemStreamPartition [kafka, ops.web.parse_rules__ck_4_module, 0], null
  4. 2015-01-04 14:58:36 BootstrappingChooser [TRACE] task[] ssp[] offset[] Check UPCOMING offset null against 0 for SystemStreamPartition [kafka, ops.web.parse_rules__ck_4_module, 0].
  5. 2015-01-04 14:58:36 BootstrappingChooser [DEBUG] task[] ssp[] offset[] Registering stream partition with offset: SystemStreamPartition [kafka, ops._raw.web_parsed, 1], null
  6. 2015-01-04 14:58:36 BootstrappingChooser [TRACE] task[] ssp[] offset[] Check UPCOMING offset null against null for SystemStreamPartition [kafka, ops._raw.web_parsed, 1].
  7. 2015-01-04 14:58:36 BootstrappingChooser [DEBUG] task[] ssp[] offset[] Registering stream partition with offset: SystemStreamPartition [kafka, ops.web.parse_rules__ck_4_module, 1], null
  8. 2015-01-04 14:58:36 BootstrappingChooser [TRACE] task[] ssp[] offset[] Check UPCOMING offset null against 0 for SystemStreamPartition [kafka, ops.web.parse_rules__ck_4_module, 1].
  9. 2015-01-04 14:58:36 BootstrappingChooser [DEBUG] task[] ssp[] offset[] Registering stream partition with offset: SystemStreamPartition [kafka, ops._raw.web_parsed, 2], null
  10. 2015-01-04 14:58:36 BootstrappingChooser [TRACE] task[] ssp[] offset[] Check UPCOMING offset null against null for SystemStreamPartition [kafka, ops._raw.web_parsed, 2].
  11. 2015-01-04 14:58:36 BootstrappingChooser [DEBUG] task[] ssp[] offset[] Registering stream partition with offset: SystemStreamPartition [kafka, ops.web.parse_rules__ck_4_module, 2], null
  12. 2015-01-04 14:58:36 BootstrappingChooser [TRACE] task[] ssp[] offset[] Check UPCOMING offset null against 0 for SystemStreamPartition [kafka, ops.web.parse_rules__ck_4_module, 2].
  13. 2015-01-04 14:58:36 BootstrappingChooser [DEBUG] task[] ssp[] offset[] Registering stream partition with offset: SystemStreamPartition [kafka, ops._raw.web_parsed, 3], 0
  14. 2015-01-04 14:58:36 BootstrappingChooser [TRACE] task[] ssp[] offset[] Check UPCOMING offset 0 against null for SystemStreamPartition [kafka, ops._raw.web_parsed, 3].
  15. 2015-01-04 14:58:36 BootstrappingChooser [DEBUG] task[] ssp[] offset[] Registering stream partition with offset: SystemStreamPartition [kafka, ops.web.parse_rules__ck_4_module, 3], null
  16. 2015-01-04 14:58:36 BootstrappingChooser [TRACE] task[] ssp[] offset[] Check UPCOMING offset null against 0 for SystemStreamPartition [kafka, ops.web.parse_rules__ck_4_module, 3].
  17.  
  18. ...
  19.  
  20. 2015-01-04 14:58:36 BootstrappingChooser [DEBUG] task[] ssp[] offset[] Starting bootstrapping chooser with bootstrap metadata: Map(SystemStream [system=kafka, stream=ops.web.parse_rules__ck_4_module] -> SystemStreamMetadata [streamName=ops.web.parse_rules__ck_4_module, partitionMetadata={Partition [partition=0]=SystemStreamPartitionMetadata [oldestOffset=null, newestOffset=null, upcomingOffset=0], Partition [partition=3]=SystemStreamPartitionMetadata [oldestOffset=null, newestOffset=null, upcomingOffset=0], Partition [partition=2]=SystemStreamPartitionMetadata [oldestOffset=null, newestOffset=null, upcomingOffset=0], Partition [partition=1]=SystemStreamPartitionMetadata [oldestOffset=null, newestOffset=null, upcomingOffset=0]}])
  21. 2015-01-04 14:58:36 BootstrappingChooser [INFO] task[] ssp[] offset[] Got lagging partition counts for bootstrap streams: Map(SystemStream [system=kafka, stream=ops.web.parse_rules__ck_4_module] -> 4)
  22. 2015-01-04 14:58:36 MetricsRegistryMap [DEBUG] task[] ssp[] offset[] Adding new gauge org.apache.samza.system.chooser.BootstrappingChooserMetrics lagging-batch-streams 4.
  23. 2015-01-04 14:58:36 JmxReporter [DEBUG] task[] ssp[] offset[] Resolved name for org.apache.samza.system.chooser.BootstrappingChooserMetrics, lagging-batch-streams, local-process-container to: org.apache.samza.system.chooser.BootstrappingChooserMetrics:type=local-process-container,name=lagging-batch-streams
  24. 2015-01-04 14:58:36 JmxReporter [DEBUG] task[] ssp[] offset[] Registering MBean for org.apache.samza.system.chooser.BootstrappingChooserMetrics:type=local-process-container,name=lagging-batch-streams.
  25. 2015-01-04 14:58:36 MetricsRegistryMap [DEBUG] task[] ssp[] offset[] Adding new gauge org.apache.samza.system.chooser.BootstrappingChooserMetrics kafka-ops.web.parse_rules__ck_4_module-lagging-partitions 4.
  26. 2015-01-04 14:58:36 JmxReporter [DEBUG] task[] ssp[] offset[] Resolved name for org.apache.samza.system.chooser.BootstrappingChooserMetrics, kafka-ops.web.parse_rules__ck_4_module-lagging-partitions, local-process-container to: org.apache.samza.system.chooser.BootstrappingChooserMetrics:type=local-process-container,name=kafka-ops.web.parse_rules__ck_4_module-lagging-partitions
  27. 2015-01-04 14:58:36 JmxReporter [DEBUG] task[] ssp[] offset[] Registering MBean for org.apache.samza.system.chooser.BootstrappingChooserMetrics:type=local-process-container,name=kafka-ops.web.parse_rules__ck_4_module-lagging-partitions.
  28. 2015-01-04 14:58:36 TieredPriorityChooser [INFO] task[] ssp[] offset[] Starting priority chooser with priorities: Map(SystemStream [system=kafka, stream=ops.web.parse_rules__ck_4_module] -> 2147483647, SystemStream [system=kafka, stream=ops._raw.web_parsed] -> 0)
  29. 2015-01-04 14:58:36 TieredPriorityChooser [INFO] task[] ssp[] offset[] Priority chooser has a default chooser: org.apache.samza.system.chooser.RoundRobinChooser@707f7d2b
  30. 2015-01-04 14:58:36 MetricsRegistryMap [DEBUG] task[] ssp[] offset[] Adding new gauge org.apache.samza.system.chooser.RoundRobinChooserMetrics buffered-messages 0.
  31. 2015-01-04 14:58:36 JmxReporter [DEBUG] task[] ssp[] offset[] Resolved name for org.apache.samza.system.chooser.RoundRobinChooserMetrics, buffered-messages, local-process-container to: org.apache.samza.system.chooser.RoundRobinChooserMetrics:type=local-process-container,name=buffered-messages
  32. 2015-01-04 14:58:36 JmxReporter [DEBUG] task[] ssp[] offset[] Registering MBean for org.apache.samza.system.chooser.RoundRobinChooserMetrics:type=local-process-container,name=buffered-messages.
  33. 2015-01-04 14:58:36 MetricsRegistryMap [DEBUG] task[] ssp[] offset[] Adding new gauge org.apache.samza.system.chooser.RoundRobinChooserMetrics buffered-messages 0.
  34. 2015-01-04 14:58:36 JmxReporter [DEBUG] task[] ssp[] offset[] Resolved name for org.apache.samza.system.chooser.RoundRobinChooserMetrics, buffered-messages, local-process-container to: org.apache.samza.system.chooser.RoundRobinChooserMetrics:type=local-process-container,name=buffered-messages
  35. 2015-01-04 14:58:36 MetricsRegistryMap [DEBUG] task[] ssp[] offset[] Adding new gauge org.apache.samza.system.chooser.RoundRobinChooserMetrics buffered-messages 0.
  36. 2015-01-04 14:58:36 JmxReporter [DEBUG] task[] ssp[] offset[] Resolved name for org.apache.samza.system.chooser.RoundRobinChooserMetrics, buffered-messages, local-process-container to: org.apache.samza.system.chooser.RoundRobinChooserMetrics:type=local-process-container,name=buffered-messages
  37. 2015-01-04 14:58:36 SystemConsumers [TRACE] task[] ssp[] offset[] Refreshing chooser with new messages.
  38. 2015-01-04 14:58:36 SystemConsumers [TRACE] task[] ssp[] offset[] Polling system consumer: kafka
  39. 2015-01-04 14:58:36 SystemConsumers [TRACE] task[] ssp[] offset[] Getting fetch map for system: kafka
  40. 2015-01-04 14:58:36 SystemConsumers [TRACE] task[] ssp[] offset[] Fetching: [SystemStreamPartition [kafka, ops._raw.web_parsed, 3], SystemStreamPartition [kafka, ops._raw.web_parsed, 1], SystemStreamPartition [kafka, ops._raw.web_parsed, 2], SystemStreamPartition [kafka, ops._raw.web_parsed, 0], SystemStreamPartition [kafka, ops.web.parse_rules__ck_4_module, 3], SystemStreamPartition [kafka, ops.web.parse_rules__ck_4_module, 2], SystemStreamPartition [kafka, ops.web.parse_rules__ck_4_module, 1], SystemStreamPartition [kafka, ops.web.parse_rules__ck_4_module, 0]]
  41. 2015-01-04 14:58:36 SystemConsumers [TRACE] task[] ssp[] offset[] Got incoming message envelopes: {SystemStreamPartition [kafka, ops._raw.web_parsed, 3]=[IncomingMessageEnvelope [systemStreamPartition=SystemStreamPartition [kafka, ops._raw.web_parsed, 3], offset=0, key=null, message=[B@587b8882], IncomingMessageEnvelope [systemStreamPartition=SystemStreamPartition [kafka, ops._raw.web_parsed, 3], offset=1, key=null, message=[B@163f3c30], IncomingMessageEnvelope [systemStreamPartition=SystemStreamPartition [kafka, ops._raw.web_parsed, 3], offset=2, key=null, message=[B@11887eb9], IncomingMessageEnvelope [systemStreamPartition=SystemStreamPartition [kafka, ops._raw.web_parsed, 3], offset=3, key=null, message=[B@1837acad]]}
  42. 2015-01-04 14:58:36 TieredPriorityChooser [TRACE] task[] ssp[] offset[] Got prioritized chooser for stream: SystemStream [system=kafka, stream=ops._raw.web_parsed]
  43. 2015-01-04 14:58:36 SamzaContainer [INFO] task[] ssp[] offset[] Entering run loop.
  44. 2015-01-04 14:58:36 RunLoop [TRACE] task[] ssp[] offset[] Attempting to choose a message to process.
  45. 2015-01-04 14:58:36 BootstrappingChooser [TRACE] task[] ssp[] offset[] Blocking wrapped.chooser since bootstrapping is not done, but not all streams have messages available.
  46. 2015-01-04 14:58:36 SystemConsumers [TRACE] task[] ssp[] offset[] Chooser returned null.
  47. 2015-01-04 14:58:36 SystemConsumers [TRACE] task[] ssp[] offset[] Refreshing chooser with new messages.
  48. 2015-01-04 14:58:36 SystemConsumers [TRACE] task[] ssp[] offset[] Polling system consumer: kafka
  49. 2015-01-04 14:58:36 SystemConsumers [TRACE] task[] ssp[] offset[] Getting fetch map for system: kafka
  50. 2015-01-04 14:58:36 SystemConsumers [TRACE] task[] ssp[] offset[] Fetching: [SystemStreamPartition [kafka, ops._raw.web_parsed, 1], SystemStreamPartition [kafka, ops._raw.web_parsed, 2], SystemStreamPartition [kafka, ops._raw.web_parsed, 0], SystemStreamPartition [kafka, ops.web.parse_rules__ck_4_module, 3], SystemStreamPartition [kafka, ops.web.parse_rules__ck_4_module, 2], SystemStreamPartition [kafka, ops.web.parse_rules__ck_4_module, 1], SystemStreamPartition [kafka, ops.web.parse_rules__ck_4_module, 0]]
  51. 2015-01-04 14:58:36 SystemConsumers [TRACE] task[] ssp[] offset[] Got incoming message envelopes: {}
  52.  
  53. ...
  54.  
  55. 2015-01-04 14:59:24 SystemConsumers [TRACE] task[] ssp[] offset[] Chooser returned null.
  56. 2015-01-04 14:59:24 SystemConsumers [TRACE] task[] ssp[] offset[] Refreshing chooser with new messages.
  57. 2015-01-04 14:59:24 SystemConsumers [TRACE] task[] ssp[] offset[] Polling system consumer: kafka
  58. 2015-01-04 14:59:24 SystemConsumers [TRACE] task[] ssp[] offset[] Getting fetch map for system: kafka
  59. 2015-01-04 14:59:24 SystemConsumers [TRACE] task[] ssp[] offset[] Fetching: [SystemStreamPartition [kafka, ops._raw.web_parsed, 1], SystemStreamPartition [kafka, ops._raw.web_parsed, 2], SystemStreamPartition [kafka, ops._raw.web_parsed, 0], SystemStreamPartition [kafka, ops.web.parse_rules__ck_4_module, 3], SystemStreamPartition [kafka, ops.web.parse_rules__ck_4_module, 2], SystemStreamPartition [kafka, ops.web.parse_rules__ck_4_module, 1], SystemStreamPartition [kafka, ops.web.parse_rules__ck_4_module, 0]]
  60. 2015-01-04 14:59:24 SystemConsumers [TRACE] task[] ssp[] offset[] Got incoming message envelopes: {}
  61. 2015-01-04 14:59:24 RunLoop [TRACE] task[] ssp[] offset[] No incoming message envelope was available.
  62. 2015-01-04 14:59:24 RunLoop [TRACE] task[] ssp[] offset[] Attempting to choose a message to process.
  63. 2015-01-04 14:59:24 BootstrappingChooser [TRACE] task[] ssp[] offset[] Blocking wrapped.chooser since bootstrapping is not done, but not all streams have messages available.
Advertisement
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!