API tools faq
paste
Advertisement
Guest User

Untitled

a guest
May 24th, 2016
113
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 9.95 KB | None | 0 0
raw download clone embed print report
  1. $ cat conf/agent.conf
  2. cygnusagent.sources = http-source
  3. cygnusagent.sinks = ckan-sink hdfs-sink
  4. cygnusagent.channels = ckan-channel hdfs-channel
  5.  
  6. cygnusagent.sources.http-source.channels = ckan-channel hdfs-channel
  7. cygnusagent.sources.http-source.type = org.apache.flume.source.http.HTTPSource
  8. cygnusagent.sources.http-source.port = 5050
  9. cygnusagent.sources.http-source.handler = com.telefonica.iot.cygnus.handlers.OrionRestHandler
  10. cygnusagent.sources.http-source.handler.notification_target = /notify
  11. cygnusagent.sources.http-source.handler.events_ttl = 2
  12. cygnusagent.sources.http-source.interceptors = ts gi
  13. cygnusagent.sources.http-source.interceptors.ts.type = timestamp
  14. cygnusagent.sources.http-source.interceptors.gi.type = com.telefonica.iot.cygnus.interceptors.GroupingInterceptor$Builder
  15. cygnusagent.sources.http-source.interceptors.gi.grouping_rules_conf_file = /usr/cygnus/conf/grouping_rules.conf
  16.  
  17. cygnusagent.channels.ckan-channel.type = memory
  18. cygnusagent.channels.ckan-channel.capacity = 1000
  19. cygnusagent.channels.ckan-channel.transactionCapacity = 100
  20.  
  21. cygnusagent.sinks.ckan-sink.channel = ckan-channel
  22. cygnusagent.sinks.ckan-sink.type = com.telefonica.iot.cygnus.sinks.OrionCKANSink
  23. cygnusagent.sinks.ckan-sink.enable_grouping = false
  24. cygnusagent.sinks.ckan-sink.api_key = XXXXXXXXXX
  25. cygnusagent.sinks.ckan-sink.ckan_host = data.lab.fiware.org
  26. cygnusagent.sinks.ckan-sink.ckan_port = 443
  27. cygnusagent.sinks.ckan-sink.orion_url = localhost:1026
  28. cygnusagent.sinks.ckan-sink.attr_persistence = row
  29. cygnusagent.sinks.ckan-sink.ssl = true
  30. cygnusagent.sinks.ckan-sink.batch_size = 1
  31. cygnusagent.sinks.ckan-sink.batch_timeout = 10
  32.  
  33. cygnusagent.sinks.hdfs-sink.type = com.telefonica.iot.cygnus.sinks.OrionHDFSSink
  34. cygnusagent.sinks.hdfs-sink.channel = hdfs-channel
  35. cygnusagent.sinks.hdfs-sink.enable_grouping = false
  36. cygnusagent.sinks.hdfs-sink.hdfs_host = cosmos.lab.fiware.org
  37. cygnusagent.sinks.hdfs-sink.hdfs_port = 14000
  38. cygnusagent.sinks.hdfs-sink.hdfs_username = XXXXX
  39. cygnusagent.sinks.hdfs-sink.hdfs_password = XXX
  40. cygnusagent.sinks.hdfs-sink.oauth2_token = XXXXXXXXX
  41. cygnusagent.sinks.hdfs-sink.file_format = json-column
  42.  
  43.  
  44. cygnusagent.channels.hdfs-channel.type = memory
  45. cygnusagent.channels.hdfs-channel.capacity = 1000
  46. cygnusagent.channels.hdfs-channel.transactionCapacity = 100
  47. $
  48. $
  49. $ /usr/cygnus/bin/cygnus-flume-ng agent —conf /usr/cygnus/conf -f /usr/cygnus/conf/agent.conf -n cygnusagent -Dflume.root.logger=DEBUG,console -p 8085
  50. Warning: No configuration directory set! Use --conf <dir> to override.
  51. Warning: JAVA_HOME is not set!
  52. + exec /usr/bin/java -Xmx20m -Dflume.root.logger=DEBUG,console -cp '/usr/cygnus/lib/*:/usr/cygnus/plugins.d/cygnus/lib/*:/usr/cygnus/plugins.d/cygnus/libext/*' -Djava.library.path= com.telefonica.iot.cygnus.nodes.CygnusApplication $'\342\200\224conf' /usr/cygnus/conf -f /usr/cygnus/conf/agent.conf -n cygnusagent -p 8085
  53. SLF4J: Class path contains multiple SLF4J bindings.
  54. SLF4J: Found binding in [jar:file:/usr/cygnus/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
  55. SLF4J: Found binding in [jar:file:/usr/cygnus/plugins.d/cygnus/lib/cygnus-0.13.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
  56. SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
  57. 16/05/24 13:02:38 INFO nodes.CygnusApplication: Starting Cygnus application
  58. 16/05/24 13:02:38 INFO nodes.CygnusApplication: Waiting for valid Flume components references...
  59. 16/05/24 13:02:38 INFO node.PollingPropertiesFileConfigurationProvider: Configuration provider starting
  60. 16/05/24 13:02:38 INFO node.PollingPropertiesFileConfigurationProvider: Reloading configuration file:/usr/cygnus/conf/agent.conf
  61. 16/05/24 13:02:38 INFO conf.FlumeConfiguration: Processing:ckan-sink
  62. 16/05/24 13:02:38 INFO conf.FlumeConfiguration: Processing:hdfs-sink
  63. 16/05/24 13:02:38 INFO conf.FlumeConfiguration: Processing:ckan-sink
  64. 16/05/24 13:02:38 INFO conf.FlumeConfiguration: Processing:ckan-sink
  65. 16/05/24 13:02:38 INFO conf.FlumeConfiguration: Processing:ckan-sink
  66. 16/05/24 13:02:38 INFO conf.FlumeConfiguration: Processing:hdfs-sink
  67. 16/05/24 13:02:38 INFO conf.FlumeConfiguration: Processing:ckan-sink
  68. 16/05/24 13:02:38 INFO conf.FlumeConfiguration: Processing:hdfs-sink
  69. 16/05/24 13:02:38 INFO conf.FlumeConfiguration: Added sinks: ckan-sink hdfs-sink Agent: cygnusagent
  70. 16/05/24 13:02:38 INFO conf.FlumeConfiguration: Processing:ckan-sink
  71. 16/05/24 13:02:38 INFO conf.FlumeConfiguration: Processing:ckan-sink
  72. 16/05/24 13:02:38 INFO conf.FlumeConfiguration: Processing:hdfs-sink
  73. 16/05/24 13:02:38 INFO conf.FlumeConfiguration: Processing:hdfs-sink
  74. 16/05/24 13:02:38 INFO conf.FlumeConfiguration: Processing:ckan-sink
  75. 16/05/24 13:02:38 INFO conf.FlumeConfiguration: Processing:ckan-sink
  76. 16/05/24 13:02:38 INFO conf.FlumeConfiguration: Processing:hdfs-sink
  77. 16/05/24 13:02:38 INFO conf.FlumeConfiguration: Processing:hdfs-sink
  78. 16/05/24 13:02:38 INFO conf.FlumeConfiguration: Processing:ckan-sink
  79. 16/05/24 13:02:38 INFO conf.FlumeConfiguration: Processing:hdfs-sink
  80. 16/05/24 13:02:38 INFO conf.FlumeConfiguration: Processing:hdfs-sink
  81. 16/05/24 13:02:38 INFO conf.FlumeConfiguration: Processing:ckan-sink
  82. 16/05/24 13:02:38 INFO conf.FlumeConfiguration: Post-validation flume configuration contains configuration for agents: [cygnusagent]
  83. 16/05/24 13:02:38 INFO node.AbstractConfigurationProvider: Creating channels
  84. 16/05/24 13:02:38 INFO channel.DefaultChannelFactory: Creating instance of channel ckan-channel type memory
  85. 16/05/24 13:02:38 INFO node.AbstractConfigurationProvider: Created channel ckan-channel
  86. 16/05/24 13:02:38 INFO channel.DefaultChannelFactory: Creating instance of channel hdfs-channel type memory
  87. 16/05/24 13:02:38 INFO node.AbstractConfigurationProvider: Created channel hdfs-channel
  88. 16/05/24 13:02:38 INFO source.DefaultSourceFactory: Creating instance of source http-source, type org.apache.flume.source.http.HTTPSource
  89. 16/05/24 13:02:38 INFO handlers.OrionRestHandler: Cygnus version (0.13.0.0c6765fe9e089bfb5820aa64b8edb30b9058fe0e)
  90. 16/05/24 13:02:38 INFO handlers.OrionRestHandler: Startup completed
  91. 16/05/24 13:02:38 INFO sink.DefaultSinkFactory: Creating instance of sink: ckan-sink, type: com.telefonica.iot.cygnus.sinks.OrionCKANSink
  92. 16/05/24 13:02:38 INFO sink.DefaultSinkFactory: Creating instance of sink: hdfs-sink, type: com.telefonica.iot.cygnus.sinks.OrionHDFSSink
  93. 16/05/24 13:02:38 INFO node.AbstractConfigurationProvider: Channel ckan-channel connected to [http-source, ckan-sink]
  94. 16/05/24 13:02:38 INFO node.AbstractConfigurationProvider: Channel hdfs-channel connected to [http-source, hdfs-sink]
  95. 16/05/24 13:02:38 INFO node.Application: Starting new configuration:{ sourceRunners:{http-source=EventDrivenSourceRunner: { source:org.apache.flume.source.http.HTTPSource{name:http-source,state:IDLE} }} sinkRunners:{ckan-sink=SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@4d3e20dc counterGroup:{ name:null counters:{} } }, hdfs-sink=SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@29df7322 counterGroup:{ name:null counters:{} } }} channels:{ckan-channel=org.apache.flume.channel.MemoryChannel{name: ckan-channel}, hdfs-channel=org.apache.flume.channel.MemoryChannel{name: hdfs-channel}} }
  96. 16/05/24 13:02:38 INFO node.Application: Starting Channel ckan-channel
  97. 16/05/24 13:02:38 INFO node.Application: Starting Channel hdfs-channel
  98. 16/05/24 13:02:38 INFO instrumentation.MonitoredCounterGroup: Monitoried counter group for type: CHANNEL, name: ckan-channel, registered successfully.
  99. 16/05/24 13:02:38 INFO instrumentation.MonitoredCounterGroup: Monitoried counter group for type: CHANNEL, name: hdfs-channel, registered successfully.
  100. 16/05/24 13:02:38 INFO instrumentation.MonitoredCounterGroup: Component type: CHANNEL, name: ckan-channel started
  101. 16/05/24 13:02:38 INFO instrumentation.MonitoredCounterGroup: Component type: CHANNEL, name: hdfs-channel started
  102. 16/05/24 13:02:38 INFO node.Application: Starting Sink ckan-sink
  103. 16/05/24 13:02:38 INFO node.Application: Starting Sink hdfs-sink
  104. 16/05/24 13:02:38 INFO node.Application: Starting Source http-source
  105. 16/05/24 13:02:38 INFO interceptors.GroupingRules: Grouping rules read: { "grouping_rules": [ ]}
  106. 16/05/24 13:02:38 INFO interceptors.GroupingRules: Grouping rules syntax is OK
  107. 16/05/24 13:02:38 INFO interceptors.GroupingRules: Grouping rules regex'es have been compiled
  108. 16/05/24 13:02:38 INFO interceptors.GroupingRules: Grouping rules read: { "grouping_rules": [ ]}
  109. 16/05/24 13:02:38 INFO interceptors.GroupingRules: Grouping rules syntax is OK
  110. 16/05/24 13:02:38 INFO interceptors.GroupingRules: Grouping rules regex'es have been compiled
  111. 16/05/24 13:02:38 INFO mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
  112. 16/05/24 13:02:38 INFO http.HttpClientFactory: Setting max total connections (500)
  113. 16/05/24 13:02:38 INFO http.HttpClientFactory: Setting default max connections per route (100)
  114. 16/05/24 13:02:38 INFO mortbay.log: jetty-6.1.26
  115. 16/05/24 13:02:38 INFO http.HttpClientFactory: Setting max total connections (500)
  116. 16/05/24 13:02:38 INFO http.HttpClientFactory: Setting default max connections per route (100)
  117. 16/05/24 13:02:38 INFO sinks.OrionSink: [ckan-sink] Startup completed
  118. 16/05/24 13:02:38 INFO mortbay.log: Started SocketConnector@0.0.0.0:5050
  119. 16/05/24 13:02:38 INFO instrumentation.MonitoredCounterGroup: Monitoried counter group for type: SOURCE, name: http-source, registered successfully.
  120. 16/05/24 13:02:38 INFO instrumentation.MonitoredCounterGroup: Component type: SOURCE, name: http-source started
  121. 16/05/24 13:02:38 INFO http.HttpClientFactory: Setting max total connections (500)
  122. 16/05/24 13:02:38 INFO http.HttpClientFactory: Setting default max connections per route (100)
  123. 16/05/24 13:02:38 INFO sinks.OrionSink: [hdfs-sink] Startup completed
  124. 16/05/24 13:02:39 INFO nodes.CygnusApplication: Starting a Jetty server listening on port 8085 (Management Interface)
  125. 16/05/24 13:02:39 INFO mortbay.log: jetty-6.1.26
  126. 16/05/24 13:02:39 INFO mortbay.log: Started SocketConnector@0.0.0.0:8085
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!