API tools faq
paste
Advertisement
Guest User

Flume run Log

a guest
Jul 12th, 2012
36
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 6.90 KB | None | 0 0
raw download clone embed print report
  1. 2012-07-12 17:11:49,608 INFO lifecycle.LifecycleSupervisor: Starting lifecycle supervisor 1
  2. 2012-07-12 17:11:49,611 INFO node.FlumeNode: Flume node starting - agent1
  3. 2012-07-12 17:11:49,615 INFO nodemanager.DefaultLogicalNodeManager: Node manager starting
  4. 2012-07-12 17:11:49,615 INFO lifecycle.LifecycleSupervisor: Starting lifecycle supervisor 12
  5. 2012-07-12 17:11:49,616 INFO properties.PropertiesFileConfigurationProvider: Configuration provider starting
  6. 2012-07-12 17:11:49,616 DEBUG nodemanager.DefaultLogicalNodeManager: Node manager started
  7. 2012-07-12 17:11:49,619 DEBUG properties.PropertiesFileConfigurationProvider: Configuration provider started
  8. 2012-07-12 17:11:49,619 DEBUG properties.PropertiesFileConfigurationProvider: Checking file:flume-example.cfg for changes
  9. 2012-07-12 17:11:49,619 INFO properties.PropertiesFileConfigurationProvider: Reloading configuration file:flume-example.cfg
  10. 2012-07-12 17:11:49,628 INFO conf.FlumeConfiguration: Processing:hdfs-spool-sink
  11. 2012-07-12 17:11:49,628 DEBUG conf.FlumeConfiguration: Created context for hdfs-spool-sink: type
  12. 2012-07-12 17:11:49,628 INFO conf.FlumeConfiguration: Processing:hdfs-spool-sink
  13. 2012-07-12 17:11:49,628 INFO conf.FlumeConfiguration: Processing:hdfs-sink
  14. 2012-07-12 17:11:49,628 DEBUG conf.FlumeConfiguration: Created context for hdfs-sink: channel
  15. 2012-07-12 17:11:49,628 INFO conf.FlumeConfiguration: Processing:avro-spool-sink
  16. 2012-07-12 17:11:49,628 DEBUG conf.FlumeConfiguration: Created context for avro-spool-sink: channel
  17. 2012-07-12 17:11:49,628 INFO conf.FlumeConfiguration: Processing:hdfs-sink
  18. 2012-07-12 17:11:49,628 INFO conf.FlumeConfiguration: Processing:hdfs-sink
  19. 2012-07-12 17:11:49,628 INFO conf.FlumeConfiguration: Processing:avro-spool-sink
  20. 2012-07-12 17:11:49,628 INFO conf.FlumeConfiguration: Processing:hdfs-sink
  21. 2012-07-12 17:11:49,628 INFO conf.FlumeConfiguration: Processing:hdfs-spool-sink
  22. 2012-07-12 17:11:49,628 INFO conf.FlumeConfiguration: Processing:avro-spool-sink
  23. 2012-07-12 17:11:49,628 INFO conf.FlumeConfiguration: Processing:hdfs-spool-sink
  24. 2012-07-12 17:11:49,628 INFO conf.FlumeConfiguration: Processing:avro-spool-sink
  25. 2012-07-12 17:11:49,629 INFO conf.FlumeConfiguration: Processing:hdfs-spool-sink
  26. 2012-07-12 17:11:49,629 INFO conf.FlumeConfiguration: Processing:avro-spool-sink
  27. 2012-07-12 17:11:49,629 INFO conf.FlumeConfiguration: Added sinks: hdfs-spool-sink avro-spool-sink hdfs-sink Agent: agent1
  28. 2012-07-12 17:11:49,629 INFO conf.FlumeConfiguration: Processing:hdfs-sink
  29. 2012-07-12 17:11:49,629 DEBUG conf.FlumeConfiguration: Starting validation of configuration for agent: agent1, initial-configuration: AgentConfiguration[agent1]
  30. SOURCES: {avro-source2={ parameters:{port=41419, channels=spoolchannel, type=avro, bind=0.0.0.0} }, seq-source={ parameters:{channels=mainchannel, type=seq} }}
  31. CHANNELS: {mainchannel={ parameters:{type=memory} }, spoolchannel={ parameters:{type=file} }}
  32. SINKS: {avro-spool-sink={ parameters:{port=41419, batch-size=100, hostname=0.0.0.0, type=avro, channel=mainchannel} }, hdfs-sink={ parameters:{hdfs.fileType=datastream, hdfs.path=hdfs://localhost, hdfs.filePrefix=flume-inder3-data/, type=hdfs, channel=mainchannel} }, hdfs-spool-sink={ parameters:{hdfs.fileType=datastream, hdfs.path=hdfs://localhost, hdfs.filePrefix=flume-inder3-data/, type=hdfs, channel=spoolchannel} }}
  33.  
  34. 2012-07-12 17:11:49,636 DEBUG conf.FlumeConfiguration: Created channel mainchannel
  35. 2012-07-12 17:11:49,638 DEBUG conf.FlumeConfiguration: Created channel spoolchannel
  36. 2012-07-12 17:11:49,664 DEBUG conf.FlumeConfiguration: Creating sink: avro-spool-sink using AVRO
  37. 2012-07-12 17:11:49,668 DEBUG conf.FlumeConfiguration: Creating sink: hdfs-sink using HDFS
  38. 2012-07-12 17:11:49,671 DEBUG conf.FlumeConfiguration: Creating sink: hdfs-spool-sink using HDFS
  39. 2012-07-12 17:11:49,705 DEBUG conf.FlumeConfiguration: Post validation configuration for agent1
  40. AgentConfiguration created without Configuration stubs for which only basic syntactical validation was performed[agent1]
  41. SOURCES: {avro-source2={ parameters:{port=41419, channels=spoolchannel, type=avro, bind=0.0.0.0} }, seq-source={ parameters:{channels=mainchannel, type=seq} }}
  42. CHANNELS: {mainchannel={ parameters:{type=memory} }, spoolchannel={ parameters:{type=file} }}
  43. 2012-07-12 17:11:49,636 DEBUG conf.FlumeConfiguration: Created channel mainchannel
  44. 2012-07-12 17:11:49,638 DEBUG conf.FlumeConfiguration: Created channel spoolchannel
  45. 2012-07-12 17:11:49,664 DEBUG conf.FlumeConfiguration: Creating sink: avro-spool-sink using AVRO
  46. 2012-07-12 17:11:49,668 DEBUG conf.FlumeConfiguration: Creating sink: hdfs-sink using HDFS
  47. 2012-07-12 17:11:49,671 DEBUG conf.FlumeConfiguration: Creating sink: hdfs-spool-sink using HDFS
  48. 2012-07-12 17:11:49,705 DEBUG conf.FlumeConfiguration: Post validation configuration for agent1
  49. AgentConfiguration created without Configuration stubs for which only basic syntactical validation was performed[agent1]
  50. SOURCES: {avro-source2={ parameters:{port=41419, channels=spoolchannel, type=avro, bind=0.0.0.0} }, seq-source={ parameters:{channels=mainchannel, type=seq} }}
  51. CHANNELS: {mainchannel={ parameters:{type=memory} }, spoolchannel={ parameters:{type=file} }}
  52. SINKS: {avro-spool-sink={ parameters:{port=41419, batch-size=100, hostname=0.0.0.0, type=avro, channel=mainchannel} }, hdfs-sink={ parameters:{hdfs.fileType=datastream, hdfs.path=hdfs://localhost, hdfs.filePrefix=flume-inder3-data/, type=hdfs, channel=mainchannel} }, hdfs-spool-sink={ parameters:{hdfs.fileType=datastream, hdfs.path=hdfs://localhost, hdfs.filePrefix=flume-inder3-data/, type=hdfs, channel=spoolchannel} }}
  53.  
  54. 2012-07-12 17:11:49,705 DEBUG conf.FlumeConfiguration: Channels:mainchannel spoolchannel
  55.  
  56. 2012-07-12 17:11:49,705 DEBUG conf.FlumeConfiguration: Sinks avro-spool-sink hdfs-sink hdfs-spool-sink
  57.  
  58. 2012-07-12 17:11:49,705 DEBUG conf.FlumeConfiguration: Sources avro-source2 seq-source
  59.  
  60. 2012-07-12 17:11:49,705 INFO conf.FlumeConfiguration: Post-validation flume configuration contains configuration for agents: [agent1]
  61. 2012-07-12 17:11:49,705 INFO properties.PropertiesFileConfigurationProvider: Creating channels
  62. 2012-07-12 17:11:49,705 DEBUG channel.DefaultChannelFactory: Creating instance of channel mainchannel type memory
  63. 2012-07-12 17:11:49,712 INFO properties.PropertiesFileConfigurationProvider: created channel mainchannel
  64. 2012-07-12 17:11:49,712 DEBUG channel.DefaultChannelFactory: Creating instance of channel spoolchannel type file
  65. 2012-07-12 17:11:49,715 INFO properties.PropertiesFileConfigurationProvider: created channel spoolchannel
  66. 2012-07-12 17:11:49,715 DEBUG source.DefaultSourceFactory: Creating instance of source avro-source2, type avro
  67. 2012-07-12 17:11:49,736 DEBUG source.DefaultSourceFactory: Creating instance of source seq-source, type seq
  68. 2012-07-12 17:11:49,736 INFO sink.DefaultSinkFactory: Creating instance of sink: avro-spool-sink, type: avro
  69. 2012-07-12 17:11:49,739 INFO sink.DefaultSinkFactory: Creating instance of sink: hdfs-sink, type: hdfs
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!