Advertisement
Guest User

Untitled

a guest
Jan 16th, 2014
268
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 54.78 KB | None | 0 0
  1. 01-16-2014 14:06:58.197 INFO dispatchRunner - initing LicenseMgr in search process: nonPro=0
  2. 01-16-2014 14:06:58.197 INFO LicenseMgr - Initing LicenseMgr
  3. 01-16-2014 14:06:58.197 INFO LMConfig - serverName=master1 guid=6D635ECF-3A04-46EE-A7A2-E944BEA1897C
  4. 01-16-2014 14:06:58.199 INFO LMConfig - connection_timeout=30
  5. 01-16-2014 14:06:58.199 INFO LMConfig - send_timeout=30
  6. 01-16-2014 14:06:58.199 INFO LMConfig - receive_timeout=30
  7. 01-16-2014 14:06:58.199 INFO LMConfig - squash_threshold=2000
  8. 01-16-2014 14:06:58.199 INFO LicenseMgr - Initing LicenseMgr runContext_splunkd=false
  9. 01-16-2014 14:06:58.199 INFO LMStackMgr - closing stack mgr
  10. 01-16-2014 14:06:58.199 INFO LMSlaveInfo - all slaves cleared
  11. 01-16-2014 14:06:58.202 INFO LMStack - Added type=download-trial license, from file=enttrial.lic, to stack=download-trial of group=Trial
  12. 01-16-2014 14:06:58.202 INFO LMStackMgr - created stack='download-trial'
  13. 01-16-2014 14:06:58.202 INFO LMStackMgr - have to auto-set active stack group='Trial' reason='invalid/missing group id' gidStr='' oldGid=Invalid
  14. 01-16-2014 14:06:58.202 INFO LMStackMgr - added pool auto_generated_pool_download-trial to stack download-trial
  15. 01-16-2014 14:06:58.202 INFO LMStackMgr - added pool auto_generated_pool_forwarder to stack forwarder
  16. 01-16-2014 14:06:58.202 INFO LMStackMgr - added pool auto_generated_pool_free to stack free
  17. 01-16-2014 14:06:58.202 INFO LMStackMgr - init completed [6D635ECF-3A04-46EE-A7A2-E944BEA1897C,Trial,runContext_splunkd=false]
  18. 01-16-2014 14:06:58.202 INFO LicenseMgr - StackMgr init complete...
  19. 01-16-2014 14:06:58.202 INFO LMTracker - this is not splunkd, will perform partial init
  20. 01-16-2014 14:06:58.202 INFO LMTracker - Setting feature=Auth state=ENABLED (featureStatus=1)
  21. 01-16-2014 14:06:58.202 INFO LMTracker - Setting feature=FwdData state=ENABLED (featureStatus=1)
  22. 01-16-2014 14:06:58.202 INFO LMTracker - Setting feature=RcvData state=ENABLED (featureStatus=1)
  23. 01-16-2014 14:06:58.202 INFO LMTracker - Setting feature=DistSearch state=ENABLED (featureStatus=1)
  24. 01-16-2014 14:06:58.202 INFO LMTracker - Setting feature=RcvSearch state=ENABLED (featureStatus=1)
  25. 01-16-2014 14:06:58.202 INFO LMTracker - Setting feature=ScheduledSearch state=ENABLED (featureStatus=1)
  26. 01-16-2014 14:06:58.202 INFO LMTracker - Setting feature=Alerting state=ENABLED (featureStatus=1)
  27. 01-16-2014 14:06:58.202 INFO LMTracker - Setting feature=DeployClient state=ENABLED (featureStatus=1)
  28. 01-16-2014 14:06:58.202 INFO LMTracker - Setting feature=DeployServer state=ENABLED (featureStatus=1)
  29. 01-16-2014 14:06:58.203 INFO LMTracker - Setting feature=SplunkWeb state=ENABLED (featureStatus=1)
  30. 01-16-2014 14:06:58.203 INFO LMTracker - Setting feature=SyslogOutputProcessor state=ENABLED (featureStatus=1)
  31. 01-16-2014 14:06:58.203 INFO LMTracker - Setting feature=SigningProcessor state=ENABLED (featureStatus=1)
  32. 01-16-2014 14:06:58.203 INFO LMTracker - Setting feature=LocalSearch state=ENABLED (featureStatus=1)
  33. 01-16-2014 14:06:58.203 INFO LicenseMgr - Tracker init complete...
  34. 01-16-2014 14:06:58.207 INFO AdminManager - added factory for admin handler: 'licenses'
  35. 01-16-2014 14:06:58.207 INFO AdminManager - added factory for admin handler: 'pools'
  36. 01-16-2014 14:06:58.207 INFO AdminManager - added factory for admin handler: 'stacks'
  37. 01-16-2014 14:06:58.207 INFO AdminManager - added factory for admin handler: 'groups'
  38. 01-16-2014 14:06:58.207 INFO AdminManager - added factory for admin handler: 'slaves'
  39. 01-16-2014 14:06:58.207 INFO AdminManager - added factory for admin handler: 'localslave'
  40. 01-16-2014 14:06:58.207 INFO AdminManager - added factory for admin handler: 'licensermessages'
  41. 01-16-2014 14:06:58.207 INFO PipelineComponent - registering timer callback name=HTTPAuthManager:timeoutCallback callback=0xcb4e10 arg=0x7f51d6e7de48
  42. 01-16-2014 14:06:58.207 INFO loader - Splunkd starting (build 189883).
  43. 01-16-2014 14:06:58.207 INFO loader - System info: Linux, master1, 3.2.0-51-generic, #77-Ubuntu SMP Wed Jul 24 20:18:19 UTC 2013, x86_64.
  44. 01-16-2014 14:06:58.207 INFO loader - Detected 4 (virtual) CPUs and 16048MB RAM
  45. 01-16-2014 14:06:58.207 INFO loader - Maximum number of threads (approximate): 8024
  46. 01-16-2014 14:06:58.207 INFO loader - Arguments are: "search" "--id=1389877618.324" "--maxbuckets=300" "--ttl=600" "--maxout=500000" "--maxtime=8640000" "--lookups=1" "--reduce_freq=10" "--rf=*" "--user=admin" "--pro" "--roles=admin:power:user"
  47. 01-16-2014 14:06:58.207 INFO loader - Getting search configuration data from: /home/lmcnise/splunk/etc/modules/parsing/config.xml
  48. 01-16-2014 14:06:58.208 INFO BundlesSetup - Setup stats for /home/lmcnise/splunk/etc: wallclock_elapsed_msec=14, cpu_time_used=0.012, shared_services_generation=2, shared_services_population=1
  49. 01-16-2014 14:06:58.211 INFO UserManagerPro - Load authentication: forcing roles="admin, power, user"
  50. 01-16-2014 14:06:58.212 INFO UserManager - Setting user context: admin
  51. 01-16-2014 14:06:58.212 INFO UserManager - Done setting user context: NULL -> admin
  52. 01-16-2014 14:06:58.213 INFO dispatchRunner - search context: user="admin", app="search", bs-pathname="/home/lmcnise/splunk/etc"
  53. 01-16-2014 14:06:58.216 INFO IndexProcessor - Initializing: readonly=true reloading=false
  54. 01-16-2014 14:06:58.220 INFO HotDBManager - idx=_audit Setting hot mgr params: maxHotSpanSecs=7776000 snapBucketTimespans=false maxHotBuckets=3 maxDataSizeBytes=786432000 quarantinePastSecs=77760000 quarantineFutureSecs=2592000
  55. 01-16-2014 14:06:58.221 INFO AuditTrailManager - audit stanza does not exist in audit.conf - no signing will take place
  56. 01-16-2014 14:06:58.221 INFO databasePartitionPolicy - idx=_audit Initialized, params='[300,60,188697600,,,,786432000,5,true,500000,5,5,true,6,0,_blocksignature,7776000,1000000,0,3,77760000,2592000,131072,25,0,15,0,0,-1,18446744073709551615,2592000,true,60000,300000,false,2000]' isSlave=false needApplyDeleteJournal=false
  57. 01-16-2014 14:06:58.221 INFO HotDBManager - idx=_blocksignature Setting hot mgr params: maxHotSpanSecs=7776000 snapBucketTimespans=false maxHotBuckets=3 maxDataSizeBytes=1048576000 quarantinePastSecs=77760000 quarantineFutureSecs=2592000
  58. 01-16-2014 14:06:58.221 INFO AuditTrailManager - audit stanza does not exist in audit.conf - no signing will take place
  59. 01-16-2014 14:06:58.221 INFO databasePartitionPolicy - idx=_blocksignature Initialized, params='[300,60,0,,,,1048576000,5,true,0,5,5,true,6,0,_blocksignature,7776000,1000000,0,3,77760000,2592000,131072,25,0,15,0,0,-1,18446744073709551615,2592000,true,60000,300000,false,2000]' isSlave=false needApplyDeleteJournal=false
  60. 01-16-2014 14:06:58.221 INFO HotDBManager - idx=_internal Setting hot mgr params: maxHotSpanSecs=7776000 snapBucketTimespans=false maxHotBuckets=3 maxDataSizeBytes=104857600 quarantinePastSecs=77760000 quarantineFutureSecs=2592000
  61. 01-16-2014 14:06:58.221 INFO AuditTrailManager - audit stanza does not exist in audit.conf - no signing will take place
  62. 01-16-2014 14:06:58.221 INFO databasePartitionPolicy - idx=_internal Initialized, params='[300,60,2592000,,,,104857600,5,true,500000,5,5,true,6,0,_blocksignature,7776000,1000000,0,3,77760000,2592000,131072,25,0,15,0,0,-1,18446744073709551615,2592000,true,60000,300000,false,2000]' isSlave=false needApplyDeleteJournal=false
  63. 01-16-2014 14:06:58.221 INFO HotDBManager - idx=_thefishbucket Setting hot mgr params: maxHotSpanSecs=7776000 snapBucketTimespans=false maxHotBuckets=3 maxDataSizeBytes=524288000 quarantinePastSecs=77760000 quarantineFutureSecs=2592000
  64. 01-16-2014 14:06:58.221 INFO AuditTrailManager - audit stanza does not exist in audit.conf - no signing will take place
  65. 01-16-2014 14:06:58.221 INFO databasePartitionPolicy - idx=_thefishbucket Initialized, params='[300,60,2419200,,,,524288000,5,true,500000,5,5,true,6,0,_blocksignature,7776000,1000000,0,3,77760000,2592000,131072,25,0,15,0,0,-1,18446744073709551615,2592000,true,60000,300000,false,2000]' isSlave=false needApplyDeleteJournal=false
  66. 01-16-2014 14:06:58.221 INFO HotDBManager - idx=history Setting hot mgr params: maxHotSpanSecs=7776000 snapBucketTimespans=false maxHotBuckets=3 maxDataSizeBytes=10485760 quarantinePastSecs=77760000 quarantineFutureSecs=2592000
  67. 01-16-2014 14:06:58.221 INFO AuditTrailManager - audit stanza does not exist in audit.conf - no signing will take place
  68. 01-16-2014 14:06:58.221 INFO databasePartitionPolicy - idx=history Initialized, params='[300,60,604800,,,,10485760,5,true,500000,5,5,true,6,0,_blocksignature,7776000,1000000,0,3,77760000,2592000,131072,25,0,15,0,0,-1,18446744073709551615,2592000,true,60000,300000,false,2000]' isSlave=false needApplyDeleteJournal=false
  69. 01-16-2014 14:06:58.221 INFO HotDBManager - idx=main Setting hot mgr params: maxHotSpanSecs=7776000 snapBucketTimespans=false maxHotBuckets=10 maxDataSizeBytes=10737418240 quarantinePastSecs=77760000 quarantineFutureSecs=2592000
  70. 01-16-2014 14:06:58.221 INFO AuditTrailManager - audit stanza does not exist in audit.conf - no signing will take place
  71. 01-16-2014 14:06:58.221 INFO databasePartitionPolicy - idx=main Initialized, params='[300,60,188697600,,,,10737418240,5,true,500000,20,5,true,6,0,_blocksignature,7776000,1000000,86400,10,77760000,2592000,131072,25,0,15,0,0,-1,18446744073709551615,2592000,true,60000,300000,false,2000]' isSlave=false needApplyDeleteJournal=false
  72. 01-16-2014 14:06:58.221 INFO HotDBManager - idx=summary Setting hot mgr params: maxHotSpanSecs=7776000 snapBucketTimespans=false maxHotBuckets=3 maxDataSizeBytes=786432000 quarantinePastSecs=77760000 quarantineFutureSecs=2592000
  73. 01-16-2014 14:06:58.221 INFO AuditTrailManager - audit stanza does not exist in audit.conf - no signing will take place
  74. 01-16-2014 14:06:58.221 INFO databasePartitionPolicy - idx=summary Initialized, params='[300,60,188697600,,,,786432000,5,true,500000,5,5,true,6,0,_blocksignature,7776000,1000000,0,3,77760000,2592000,131072,25,0,15,0,0,-1,18446744073709551615,2592000,true,60000,300000,false,2000]' isSlave=false needApplyDeleteJournal=false
  75. 01-16-2014 14:06:58.221 INFO SearchParser - PARSING: search index=demo-open-day-index source=/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv source="/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv"
  76. 01-16-2014 14:06:58.222 INFO ISplunkDispatch - Not running in splunkd. Bundle replication not triggered.
  77. 01-16-2014 14:06:58.247 INFO UserManager - Setting user context: admin
  78. 01-16-2014 14:06:58.248 INFO UserManager - Done setting user context: NULL -> admin
  79. 01-16-2014 14:06:58.283 INFO SearchOperator:kv - name=EXTRACT-collection,category,object, can_use_re2=0, regex: collection=\"?(?P<collection>[^\"\n]+)\"?\ncategory=\"?(?P<category>[^\"\n]+)\"?\nobject=\"?(?P<object>[^\"\n]+)\"?\n
  80. 01-16-2014 14:06:58.283 INFO SearchOperator:kv - name=EXTRACT-GUID, can_use_re2=0, regex: (?i)(?!=\w)(?:objectguid|guid)\s*=\s*(?<guid_lookup>[\w\-]+)
  81. 01-16-2014 14:06:58.283 INFO SearchOperator:kv - name=EXTRACT-SID, can_use_re2=0, regex: objectSid\s*=\s*(?<sid_lookup>\S+)
  82. 01-16-2014 14:06:58.286 INFO SearchOperator:kv - name=ad-kv, can_use_re2=0, regex: (?<_KEY_1>[\w-]+)=(?<_VAL_1>[^\r\n]*)
  83. 01-16-2014 14:06:58.288 INFO SearchOperator:kv - name=access-extractions, can_use_re2=0, regex: ^(?P<clientip>\S+)\s++(?P<ident>\S+)\s++(?P<user>\S+)\s++\[(?<req_time>[^\]]*+)\]\s++"\s*+(?P<method>[^\s"]++)?(?:\s++(?<uri>(?:(?<uri_domain>\w++://[^/\s"]++))?+(?<uri_path>(?:/++(?<root>(?:\\"|[^\s\?/"])++)/++)?(?:(?:\\"|[^\s\?/"])*+/++)*(?<file>[^\s\?/]+)?)(?:\?(?<uri_query>[^\s]*))?)(?:\s++(?P<version>[^\s"]++))*)?\s*+"\s++(?P<status>\S+)\s++(?P<bytes>\S+)(?:\s++"(?<referer>(?:(?<referer_domain>\w++://[^/\s"]++))?+[^"]*+)"(?:\s++"(?<useragent>[^"]*+)"(?:\s++"(?<cookie>[^"]*+)")?+)?+)?(?P<other>.*)
  84. 01-16-2014 14:06:58.289 INFO SearchOperator:kv - name=syslog-extractions, can_use_re2=0, regex: \s([^\s\[]+)(?:\[(\d+)\])?:\s
  85. 01-16-2014 14:06:58.290 INFO SearchOperator:kv - name=db2, can_use_re2=0, regex: ([A-Z]+) *: (.*?)(?=\n|$| +[A-Z]+ *:)
  86. 01-16-2014 14:06:58.290 INFO SearchOperator:kv - name=EXTRACT-extract_spent, can_use_re2=0, regex: (?<spent>\d+)ms$
  87. 01-16-2014 14:06:58.291 INFO SearchOperator:kv - name=EXTRACT-1, can_use_re2=0, regex: (?<_KEY_1>\S+)::(?<_VAL_1>\S+)
  88. 01-16-2014 14:06:58.293 INFO SearchOperator:kv - name=bracket-space, can_use_re2=0, regex: \[(\S+) (.*?)\]
  89. 01-16-2014 14:06:58.295 INFO SearchOperator:kv - name=sendmail-extractions, can_use_re2=0, regex: sendmail\[(\d+)\]: (\w+):
  90. 01-16-2014 14:06:58.295 INFO SearchOperator:kv - name=tcpdump-endpoints, can_use_re2=0, regex: (\d+\.\d+\.\d+\.\d+):(\d+) -> (\d+\.\d+\.\d+\.\d+):(\d+)
  91. 01-16-2014 14:06:58.295 INFO SearchOperator:kv - name=colon-kv, can_use_re2=0, regex: (?<= )([A-Za-z]+): ?((0x[A-F\d]+)|\d+)(?= |\n|$)
  92. 01-16-2014 14:06:58.307 INFO SearchOperator:kv - name=wel-message, can_use_re2=0, regex: (?sm)^(?<_pre_msg>.+)\nMessage=(?<Message>.+)$
  93. 01-16-2014 14:06:58.307 INFO SearchOperator:kv - name=wel-col-kv, can_use_re2=0, regex: \n([^:\n\r]+):[ \t]++([^\n]*)
  94. 01-16-2014 14:06:58.308 INFO SearchOperator:kv - name=EXTRACT-useragent, can_use_re2=0, regex: userAgent=(?P<browser>[^ (]+)
  95. 01-16-2014 14:06:58.308 INFO SearchOperator:kv - name=splunk-service-extractions, can_use_re2=0, regex: (?i)^(?:[^ ]* ){2}(?P<log_level>[^\s]*)\s+\[(?P<requestid>\w+)]\s+(?P<component>[^ ]+):(?P<line>\d+) - (?P<message>.+)
  96. 01-16-2014 14:06:58.308 INFO SearchOperator:kv - name=EXTRACT-fields, can_use_re2=0, regex: (?i)^(?:[^ ]* ){2}(?:[+\-]\d+ )?(?P<log_level>[^ ]*)\s+(?P<component>[^ ]+) - (?P<message>.+)
  97. 01-16-2014 14:06:58.308 INFO SearchOperator:kv - name=extract_spent, can_use_re2=0, regex: (?P<spent>\d+)ms$
  98. 01-16-2014 14:06:58.309 INFO SearchOperator:kv - name=weblogic-code, can_use_re2=0, regex: <BEA-([0-9]+)>
  99. 01-16-2014 14:06:58.310 INFO SearchOperator:kv - name=colon-line, can_use_re2=0, regex: ^(\w+)\s*:[ \t]*(.*?)$
  100. 01-16-2014 14:06:58.310 INFO SearchOperator:kv - name=was-trlog-code, can_use_re2=0, regex: ] ([a-fA-F0-9]{8})
  101. 01-16-2014 14:06:58.311 INFO UnifiedSearch - base lispy: [ AND index::demo-open-day-index source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv ]
  102. 01-16-2014 14:06:58.312 INFO DispatchThread - BatchMode: allowBatchMode: 0, conf(1): 1, timeline/Status buckets(0):300, realtime(0):0, report pipe empty(0):1, reqTimeOrder(0):0, summarize(0):0, statefulStreaming(0):0
  103. 01-16-2014 14:06:58.312 INFO DispatchThread - Storing only 1000 events per timeline buckets due to limits.conf max_events_per_bucket setting.
  104. 01-16-2014 14:06:58.312 INFO DispatchThread - required fields list = *,_bkt,_cd,_si,host,index,linecount,source,sourcetype,splunk_server
  105. 01-16-2014 14:06:58.312 INFO SearchParser - PARSING: fields keepcolorder=t "*" "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server"
  106. 01-16-2014 14:06:58.312 INFO ProviderQueue - Stream search: litsearch index=demo-open-day-index source=/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv source="/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv" | fields keepcolorder=t "*" "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server"
  107. 01-16-2014 14:06:58.313 INFO ProviderQueue - Search proc: index=demo-open-day-index source=/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv source="/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv"
  108. 01-16-2014 14:06:58.313 INFO ProviderQueue - Search after asserting splunk_server=local: ((index="demo-open-day-index") (source="/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv") (source="/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv"))
  109. 01-16-2014 14:06:58.313 INFO ProviderQueue - Found referenced index, name=demo-open-day-index, provider=Stor
  110. 01-16-2014 14:06:58.314 INFO ProviderQueue - Stream search: litsearch index=demo-open-day-index source=/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv source="/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv" | fields keepcolorder=t "*" "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server"
  111. 01-16-2014 14:06:58.314 INFO ProviderQueue - round_robin=1
  112. 01-16-2014 14:06:58.373 INFO ProviderQueue - Found bundlePath=/home/lmcnise/splunk/var/run/master1-1389774711.bundle
  113. 01-16-2014 14:06:58.373 INFO ProviderQueue - Creating external result provider=Stor, with index.count=1, search="search (index="demo-open-day-index" ((source="/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv") (source="/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv"))) | fields keepcolorder=t "*" "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server""
  114. 01-16-2014 14:06:58.373 INFO SearchParser - PARSING: stdin | search (index="demo-open-day-index" ((source="/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv") (source="/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv"))) | fields keepcolorder=t "*" "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server"
  115. 01-16-2014 14:06:58.395 INFO SearchParser - PARSING: typer | tags
  116. 01-16-2014 14:06:58.397 INFO FastTyper - found nodes count: comparisons=6, unique_comparisons=5, terms=4, unique_terms=4, phrases=12, unique_phrases=12, total leaves=22
  117. 01-16-2014 14:06:58.412 INFO SearchOperator:stdin - setting _need_timestamp_fields=1, required time field name=date_second
  118. 01-16-2014 14:06:58.412 INFO SearchOperator:stdin - required fields list = *,Message,_bkt,_cd,_si,_time,host,index,linecount,source,sourcetype,splunk_server
  119. 01-16-2014 14:06:58.412 INFO ResultProvider - turning off streaming mode for provider name=Stor
  120. 01-16-2014 14:06:58.412 INFO ResultProvider - provider=Stor, mode.config=report, mode.search=mixed
  121. 01-16-2014 14:06:58.413 INFO ERP.Stor - Starting: /home/lmcnise/splunk/bin/jars/sudobash /usr/lib/hadoop/bin/hadoop jar "/home/lmcnise/splunk/bin/jars/SplunkMR-s6.0-hy2.0.jar" "com.splunk.mr.SplunkMR"
  122. 01-16-2014 14:06:58.413 INFO ProviderQueue - using single thread to setup ResultProviders
  123. 01-16-2014 14:06:58.413 INFO UserManager - Setting user context: admin
  124. 01-16-2014 14:06:58.413 INFO UserManager - Setting user context: admin
  125. 01-16-2014 14:06:58.414 INFO UserManager - Done setting user context: NULL -> admin
  126. 01-16-2014 14:06:58.414 INFO UserManager - Done setting user context: admin -> admin
  127. 01-16-2014 14:06:58.414 INFO ResultProvider - Creating result provider for peer: local
  128. 01-16-2014 14:06:58.414 INFO SearchParser - PARSING: litsearch index=demo-open-day-index source=/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv source="/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv" | fields keepcolorder=t "*" "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server"
  129. 01-16-2014 14:06:58.437 INFO SearchParser - PARSING: typer | tags
  130. 01-16-2014 14:06:58.439 INFO FastTyper - found nodes count: comparisons=6, unique_comparisons=5, terms=4, unique_terms=4, phrases=12, unique_phrases=12, total leaves=22
  131. 01-16-2014 14:06:58.446 INFO ResultProvider - Successfully created result provider for peer: local in 0.033000 seconds
  132. 01-16-2014 14:06:58.502 INFO UserManager - Unwound user context: admin -> admin
  133. 01-16-2014 14:06:58.502 INFO UserManager - Setting user context: admin
  134. 01-16-2014 14:06:58.502 INFO UserManager - Done setting user context: NULL -> admin
  135. 01-16-2014 14:06:58.502 INFO DispatchThread - Disk quota = 10485760000
  136. 01-16-2014 14:06:58.504 INFO UserManager - Unwound user context: admin -> NULL
  137. 01-16-2014 14:06:58.504 INFO ProviderQueue - Round Robin Threaded ProviderQueue: done reading from peer 'master1'
  138. 01-16-2014 14:06:59.594 WARN ERP.Stor - Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
  139. 01-16-2014 14:06:59.642 INFO ERP.Stor - SplunkMR$SearchHandler - Reduce search: null
  140. 01-16-2014 14:06:59.642 INFO ERP.Stor - SplunkMR$SearchHandler - Search mode: mixed
  141. 01-16-2014 14:07:00.476 INFO ERP.Stor - SplunkMR$SearchHandler - Created filesystem object, elapsed_ms=834
  142. 01-16-2014 14:07:00.486 INFO ERP.Stor - DispatchReaper - Starting. Settings: hdfs.path=/user/splunk/splunkMR/dispatch, local.path=/home/lmcnise/splunk/var/run/splunk/dispatch
  143. 01-16-2014 14:07:00.572 INFO ERP.Stor - VirtualIndex - listStatus started, vix.name=demo-open-day-index ...
  144. 01-16-2014 14:07:00.584 INFO ERP.Stor - AbstractService - Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
  145. 01-16-2014 14:07:00.676 INFO ERP.Stor - AbstractService - Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
  146. 01-16-2014 14:07:00.677 INFO ERP.Stor - DispatchReaper - Deleted HDFS dispatch dir=hdfs://master1:8020/user/splunk/splunkMR/dispatch/1389875809.318, local dispatch dir=/home/lmcnise/splunk/var/run/splunk/dispatch/1389875809.318 does not exist
  147. 01-16-2014 14:07:00.677 INFO ERP.Stor - DispatchReaper - Finished, scanned=4, deleted=1, errors=0
  148. 01-16-2014 14:07:00.782 INFO ERP.Stor - VirtualIndex - listStatus done, vix.name=demo-open-day-index, files.total=1, files.time.filtered=0, files.search.filtered=0, files.mr=1, elapsed=210ms
  149. 01-16-2014 14:07:00.783 INFO ERP.Stor - JobSubmitter - finished called: _splits.count=81, total.count=81
  150. 01-16-2014 14:07:00.783 INFO ERP.Stor - JobSubmitter - creating new job, curent split.count=81
  151. 01-16-2014 14:07:00.783 INFO ERP.Stor - SplunkMR$SearchHandler - state=start, set up the required env in HDFS for MR spawning search
  152. 01-16-2014 14:07:00.787 WARN ERP.Stor - SplunkMR$SplunkBaseMapper - Could not create preprocessor object, will try the next one ... class=com.splunk.mr.input.ValueAvroRecordReader, message=File path does not match regex to use this record reader, name=com.splunk.mr.input.ValueAvroRecordReader, path=hdfs://master1:8020/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv, regex=\.avro$.
  153. 01-16-2014 14:07:00.825 INFO ERP.Stor - SplunkMR$SplunkBaseMapper - using class=com.splunk.mr.input.SplunkLineRecordReader to process split=hdfs://master1:8020/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv:0+67108864
  154. 01-16-2014 14:07:00.825 WARN ERP.Stor - ResourceMgrDelegate - getBlacklistedTrackers - Not implemented yet
  155. 01-16-2014 14:07:00.826 INFO ERP.Stor - ClusterInfoLogger - Hadoop cluster spec: provider=Stor, tasktrackers=27, map_inuse=1, map_slots=270, reduce_inuse=1, reduce_slots=54
  156. 01-16-2014 14:07:00.864 INFO ERP.Stor - SplunkMR$SearchHandler - state=done, set up the required env in HDFS for MR spawning search
  157. 01-16-2014 14:07:00.864 INFO ERP.Stor - FileBasedHeartBeat - starting hearbeat thread, path=/user/splunk/splunkMR/dispatch/1389877618.324, interval=1000, _startIndex=0, _index=0
  158. 01-16-2014 14:07:00.890 INFO ERP.Stor - JobSubmitter - submitting new job, name=SPLK_master1_1389877618.324_0
  159. 01-16-2014 14:07:00.891 INFO ERP.Stor - AsyncMRJob - AsyncMRJob job.name=SPLK_master1_1389877618.324_0 running ...
  160. 01-16-2014 14:07:00.891 INFO ERP.Stor - AsyncMRJob - Submitting job.name=SPLK_master1_1389877618.324_0 ...
  161. 01-16-2014 14:07:00.924 INFO ERP.Stor - AbstractService - Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
  162. 01-16-2014 14:07:00.925 INFO ERP.Stor - AbstractService - Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
  163. 01-16-2014 14:07:01.006 INFO SearchOperator:stdin - Initializing from configuration
  164. 01-16-2014 14:07:01.008 INFO PipelineComponent - registering timer callback name=MetricsManager:probeandreport callback=0xb07120 arg=0x7f51d2c2c400
  165. 01-16-2014 14:07:01.008 INFO PipelineComponent - registering timer callback name=triggerCollection callback=0xb4e230 arg=0x7f51d2c4c180
  166. 01-16-2014 14:07:01.008 INFO LineBreakingProcessor - Initializing
  167. 01-16-2014 14:07:01.008 INFO regexExtractionProcessor - Initializing
  168. 01-16-2014 14:07:01.010 INFO PipelineComponent - Launching the pipelines.
  169. 01-16-2014 14:07:01.010 INFO SearchOperator:stdin - setting up new preview state and writer ...
  170. 01-16-2014 14:07:01.010 INFO pipeline - Registering metrics callback for: Pipeline:vix
  171. 01-16-2014 14:07:01.015 INFO UserManager - Setting user context: splunk-system-user
  172. 01-16-2014 14:07:01.015 INFO UserManager - Done setting user context: NULL -> splunk-system-user
  173. 01-16-2014 14:07:01.016 INFO UserManager - Unwound user context: splunk-system-user -> NULL
  174. 01-16-2014 14:07:01.016 INFO SearchOperator:stdin - started writer thread, conf=source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv| ...
  175. 01-16-2014 14:07:01.016 INFO UTF8Processor - Converting using CHARSET="UTF-8" for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
  176. 01-16-2014 14:07:01.016 INFO LineBreakingProcessor - Using truncation length 10000 for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
  177. 01-16-2014 14:07:01.016 INFO LineBreakingProcessor - Using lookbehind 100 for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
  178. 01-16-2014 14:07:01.016 INFO DateParserVerbose - Setting maxDaysAgo=1825 and maxDaysHence=7
  179. 01-16-2014 14:07:01.016 INFO AggregatorMiningProcessor - Setting up line merging apparatus for: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  180. 01-16-2014 14:07:01.018 INFO DateParserVerbose - Setting maxDaysAgo=2000 and maxDaysHence=2
  181. 01-16-2014 14:07:01.019 WARN DateParserVerbose - A possible timestamp match (Mon Feb 15 07:27:31 1999) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  182. 01-16-2014 14:07:01.019 WARN DateParserVerbose - A possible timestamp match (Mon Jul 7 11:38:56 2014) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  183. 01-16-2014 14:07:01.019 WARN DateParserVerbose - A possible timestamp match (Wed Feb 24 03:23:20 1999) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  184. 01-16-2014 14:07:01.020 WARN SearchOperator:kv - host is an indexed field, ignoring TOKENIZER
  185. 01-16-2014 14:07:01.020 WARN SearchOperator:kv - index is an indexed field, ignoring TOKENIZER
  186. 01-16-2014 14:07:01.020 WARN SearchOperator:kv - linecount is an indexed field, ignoring TOKENIZER
  187. 01-16-2014 14:07:01.020 WARN SearchOperator:kv - source is an indexed field, ignoring TOKENIZER
  188. 01-16-2014 14:07:01.020 WARN SearchOperator:kv - sourcetype is an indexed field, ignoring TOKENIZER
  189. 01-16-2014 14:07:01.020 WARN SearchOperator:kv - splunk_server is an indexed field, ignoring TOKENIZER
  190. 01-16-2014 14:07:01.020 WARN DateParserVerbose - A possible timestamp match (Wed Sep 9 10:23:34 1998) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  191. 01-16-2014 14:07:01.020 WARN SearchOperator:kv - host is an indexed field, ignoring TOKENIZER
  192. 01-16-2014 14:07:01.021 WARN DateParserVerbose - A possible timestamp match (Fri Aug 18 02:56:08 2000) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  193. 01-16-2014 14:07:01.021 WARN SearchOperator:kv - index is an indexed field, ignoring TOKENIZER
  194. 01-16-2014 14:07:01.021 WARN SearchOperator:kv - linecount is an indexed field, ignoring TOKENIZER
  195. 01-16-2014 14:07:01.021 WARN SearchOperator:kv - source is an indexed field, ignoring TOKENIZER
  196. 01-16-2014 14:07:01.021 WARN SearchOperator:kv - sourcetype is an indexed field, ignoring TOKENIZER
  197. 01-16-2014 14:07:01.021 WARN DateParserVerbose - A possible timestamp match (Wed Sep 9 10:29:25 1998) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  198. 01-16-2014 14:07:01.021 WARN DateParserVerbose - A possible timestamp match (Wed Sep 9 10:29:25 1998) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  199. 01-16-2014 14:07:01.021 WARN SearchOperator:kv - splunk_server is an indexed field, ignoring TOKENIZER
  200. 01-16-2014 14:07:01.021 WARN DateParserVerbose - A possible timestamp match (Thu Aug 31 16:10:00 2000) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  201. 01-16-2014 14:07:01.021 WARN DateParserVerbose - A possible timestamp match (Wed Feb 24 03:23:20 1999) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  202. 01-16-2014 14:07:01.021 WARN DateParserVerbose - A possible timestamp match (Wed Feb 24 03:23:20 1999) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  203. 01-16-2014 14:07:01.033 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Tue Jan 6 01:50:00 2009). Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  204. 01-16-2014 14:07:01.037 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Tue Jan 6 01:50:00 2009). Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  205. 01-16-2014 14:07:01.037 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Tue Jan 6 01:50:00 2009). Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  206. 01-16-2014 14:07:01.037 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Tue Jan 6 01:50:00 2009). Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  207. 01-16-2014 14:07:01.037 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Tue Jan 6 01:50:00 2009). Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  208. 01-16-2014 14:07:01.038 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Tue Jan 6 01:50:00 2009). Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  209. 01-16-2014 14:07:01.038 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Tue Jan 6 01:50:00 2009). Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  210. 01-16-2014 14:07:01.038 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Tue Jan 6 01:50:00 2009). Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  211. 01-16-2014 14:07:01.038 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Tue Jan 6 01:50:00 2009). Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  212. 01-16-2014 14:07:01.038 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Tue Jan 6 01:50:00 2009). Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  213. 01-16-2014 14:07:01.777 INFO ERP.Stor - JobSubmitter - number of splits:81
  214. 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.jar is deprecated. Instead, use mapreduce.job.jar
  215. 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
  216. 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.job.reduce.memory.mb is deprecated. Instead, use mapreduce.reduce.memory.mb
  217. 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
  218. 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class
  219. 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
  220. 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.job.name is deprecated. Instead, use mapreduce.job.name
  221. 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
  222. 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class
  223. 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
  224. 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
  225. 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
  226. 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
  227. 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.job.queue.name is deprecated. Instead, use mapreduce.job.queuename
  228. 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class
  229. 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.job.reuse.jvm.num.tasks is deprecated. Instead, use mapreduce.job.jvm.numtasks
  230. 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.job.map.memory.mb is deprecated. Instead, use mapreduce.map.memory.mb
  231. 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
  232. 01-16-2014 14:07:01.777 INFO ERP.Stor - JobSubmitter - Submitting tokens for job: job_1389874731460_0009
  233. 01-16-2014 14:07:01.778 INFO DispatchThread - Generating results preview took 1 ms
  234. 01-16-2014 14:07:01.901 INFO ERP.Stor - YarnClientImpl - Submitted application application_1389874731460_0009 to ResourceManager at master1/192.168.1.1:8032
  235. 01-16-2014 14:07:02.040 WARN SearchOperator:kv - date_hour is an indexed field, ignoring TOKENIZER
  236. 01-16-2014 14:07:02.040 WARN SearchOperator:kv - date_mday is an indexed field, ignoring TOKENIZER
  237. 01-16-2014 14:07:02.040 WARN SearchOperator:kv - date_minute is an indexed field, ignoring TOKENIZER
  238. 01-16-2014 14:07:02.040 WARN SearchOperator:kv - date_month is an indexed field, ignoring TOKENIZER
  239. 01-16-2014 14:07:02.040 WARN SearchOperator:kv - date_second is an indexed field, ignoring TOKENIZER
  240. 01-16-2014 14:07:02.041 WARN SearchOperator:kv - date_wday is an indexed field, ignoring TOKENIZER
  241. 01-16-2014 14:07:02.041 WARN SearchOperator:kv - date_year is an indexed field, ignoring TOKENIZER
  242. 01-16-2014 14:07:02.041 WARN SearchOperator:kv - date_zone is an indexed field, ignoring TOKENIZER
  243. 01-16-2014 14:07:02.041 WARN SearchOperator:kv - timeendpos is an indexed field, ignoring TOKENIZER
  244. 01-16-2014 14:07:02.041 WARN SearchOperator:kv - timestartpos is an indexed field, ignoring TOKENIZER
  245. 01-16-2014 14:07:02.263 WARN SearchOperator:kv - date_hour is an indexed field, ignoring TOKENIZER
  246. 01-16-2014 14:07:02.263 WARN SearchOperator:kv - date_mday is an indexed field, ignoring TOKENIZER
  247. 01-16-2014 14:07:02.263 WARN SearchOperator:kv - date_minute is an indexed field, ignoring TOKENIZER
  248. 01-16-2014 14:07:02.263 WARN SearchOperator:kv - date_month is an indexed field, ignoring TOKENIZER
  249. 01-16-2014 14:07:02.263 WARN SearchOperator:kv - date_second is an indexed field, ignoring TOKENIZER
  250. 01-16-2014 14:07:02.263 WARN SearchOperator:kv - date_wday is an indexed field, ignoring TOKENIZER
  251. 01-16-2014 14:07:02.263 WARN SearchOperator:kv - date_year is an indexed field, ignoring TOKENIZER
  252. 01-16-2014 14:07:02.263 WARN SearchOperator:kv - date_zone is an indexed field, ignoring TOKENIZER
  253. 01-16-2014 14:07:02.263 WARN SearchOperator:kv - timeendpos is an indexed field, ignoring TOKENIZER
  254. 01-16-2014 14:07:02.263 WARN SearchOperator:kv - timestartpos is an indexed field, ignoring TOKENIZER
  255. 01-16-2014 14:07:02.434 INFO ERP.Stor - Job - The url to track the job: http://master1:8088/proxy/application_1389874731460_0009/
  256. 01-16-2014 14:07:02.434 INFO ERP.Stor - AsyncMRJob - Done submitting job.name=SPLK_master1_1389877618.324_0, url=http://master1:8088/proxy/application_1389874731460_0009/
  257. 01-16-2014 14:07:02.434 INFO ERP.Stor - AbstractService - Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
  258. 01-16-2014 14:07:02.434 INFO ERP.Stor - AbstractService - Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
  259. 01-16-2014 14:07:02.434 INFO ERP.Stor - JobSubmitter - waiting for job to complete job.name=SPLK_master1_1389877618.324_0
  260. 01-16-2014 14:07:03.647 INFO DispatchThread - Generating results preview took 1 ms
  261. 01-16-2014 14:07:05.095 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
  262. 01-16-2014 14:07:05.475 INFO DispatchThread - Generating results preview took 1 ms
  263. 01-16-2014 14:07:06.340 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
  264. 01-16-2014 14:07:06.872 INFO DispatchThread - Generating results preview took 1 ms
  265. 01-16-2014 14:07:08.427 INFO DispatchThread - Generating results preview took 1 ms
  266. 01-16-2014 14:07:08.867 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
  267. 01-16-2014 14:07:10.684 INFO DispatchThread - Generating results preview took 1 ms
  268. 01-16-2014 14:07:11.326 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
  269. 01-16-2014 14:07:11.353 WARN ERP.Stor - SplunkMR$SplunkBaseMapper - Could not create preprocessor object, will try the next one ... class=com.splunk.mr.input.ValueAvroRecordReader, message=File path does not match regex to use this record reader, name=com.splunk.mr.input.ValueAvroRecordReader, path=hdfs://master1:8020/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv, regex=\.avro$.
  270. 01-16-2014 14:07:11.364 INFO ERP.Stor - SplunkMR$SplunkBaseMapper - using class=com.splunk.mr.input.SplunkLineRecordReader to process split=hdfs://master1:8020/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv:67108864+67108864
  271. 01-16-2014 14:07:11.402 INFO SearchOperator:stdin - StdinDataProvider::finish called
  272. 01-16-2014 14:07:11.509 INFO AggregatorMiningProcessor - Got done message for: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  273. 01-16-2014 14:07:12.048 INFO DispatchThread - Generating results preview took 1 ms
  274. 01-16-2014 14:07:13.084 INFO DispatchThread - Generating results preview took 1 ms
  275. 01-16-2014 14:07:15.626 INFO SearchOperator:stdin - preview completed, resetting state and writer ...
  276. 01-16-2014 14:07:15.778 INFO SearchOperator:stdin - setting up new preview state and writer ...
  277. 01-16-2014 14:07:15.779 INFO UserManager - Setting user context: splunk-system-user
  278. 01-16-2014 14:07:15.779 INFO UserManager - Done setting user context: NULL -> splunk-system-user
  279. 01-16-2014 14:07:15.779 INFO UserManager - Unwound user context: splunk-system-user -> NULL
  280. 01-16-2014 14:07:15.779 INFO SearchOperator:stdin - started writer thread, conf=source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv| ...
  281. 01-16-2014 14:07:15.779 INFO UTF8Processor - Converting using CHARSET="UTF-8" for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
  282. 01-16-2014 14:07:15.779 INFO LineBreakingProcessor - Using truncation length 10000 for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
  283. 01-16-2014 14:07:15.779 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
  284. 01-16-2014 14:07:15.779 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
  285. 01-16-2014 14:07:15.779 INFO LineBreakingProcessor - Using lookbehind 100 for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
  286. 01-16-2014 14:07:15.779 INFO DateParserVerbose - Setting maxDaysAgo=1825 and maxDaysHence=7
  287. 01-16-2014 14:07:15.779 INFO AggregatorMiningProcessor - Setting up line merging apparatus for: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  288. 01-16-2014 14:07:15.779 INFO DateParserVerbose - Setting maxDaysAgo=2000 and maxDaysHence=2
  289. 01-16-2014 14:07:15.814 INFO DispatchThread - Generating results preview took 1 ms
  290. 01-16-2014 14:07:17.385 INFO DispatchThread - Generating results preview took 1 ms
  291. 01-16-2014 14:07:17.564 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
  292. 01-16-2014 14:07:18.457 INFO DispatchThread - Generating results preview took 1 ms
  293. 01-16-2014 14:07:19.013 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
  294. 01-16-2014 14:07:20.384 INFO DispatchThread - Generating results preview took 1 ms
  295. 01-16-2014 14:07:21.251 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
  296. 01-16-2014 14:07:21.650 INFO DispatchThread - Generating results preview took 1 ms
  297. 01-16-2014 14:07:22.701 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
  298. 01-16-2014 14:07:23.124 INFO DispatchThread - Generating results preview took 1 ms
  299. 01-16-2014 14:07:24.154 WARN ERP.Stor - SplunkMR$SplunkBaseMapper - Could not create preprocessor object, will try the next one ... class=com.splunk.mr.input.ValueAvroRecordReader, message=File path does not match regex to use this record reader, name=com.splunk.mr.input.ValueAvroRecordReader, path=hdfs://master1:8020/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv, regex=\.avro$.
  300. 01-16-2014 14:07:24.164 INFO ERP.Stor - SplunkMR$SplunkBaseMapper - using class=com.splunk.mr.input.SplunkLineRecordReader to process split=hdfs://master1:8020/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv:134217728+67108864
  301. 01-16-2014 14:07:26.003 INFO SearchOperator:stdin - StdinDataProvider::finish called
  302. 01-16-2014 14:07:26.003 INFO AggregatorMiningProcessor - Got done message for: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  303. 01-16-2014 14:07:26.422 INFO DispatchThread - Generating results preview took 1 ms
  304. 01-16-2014 14:07:26.840 INFO SearchOperator:stdin - preview completed, resetting state and writer ...
  305. 01-16-2014 14:07:26.840 INFO SearchOperator:stdin - setting up new preview state and writer ...
  306. 01-16-2014 14:07:26.840 INFO UserManager - Setting user context: splunk-system-user
  307. 01-16-2014 14:07:26.840 INFO UserManager - Done setting user context: NULL -> splunk-system-user
  308. 01-16-2014 14:07:26.840 INFO UserManager - Unwound user context: splunk-system-user -> NULL
  309. 01-16-2014 14:07:26.840 INFO SearchOperator:stdin - started writer thread, conf=source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv| ...
  310. 01-16-2014 14:07:26.840 INFO UTF8Processor - Converting using CHARSET="UTF-8" for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
  311. 01-16-2014 14:07:26.840 INFO LineBreakingProcessor - Using truncation length 10000 for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
  312. 01-16-2014 14:07:26.841 INFO LineBreakingProcessor - Using lookbehind 100 for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
  313. 01-16-2014 14:07:26.841 INFO DateParserVerbose - Setting maxDaysAgo=1825 and maxDaysHence=7
  314. 01-16-2014 14:07:26.841 INFO AggregatorMiningProcessor - Setting up line merging apparatus for: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  315. 01-16-2014 14:07:26.841 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
  316. 01-16-2014 14:07:26.841 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
  317. 01-16-2014 14:07:26.841 INFO DateParserVerbose - Setting maxDaysAgo=2000 and maxDaysHence=2
  318. 01-16-2014 14:07:28.044 INFO DispatchThread - Generating results preview took 1 ms
  319. 01-16-2014 14:07:28.957 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
  320. 01-16-2014 14:07:29.656 INFO DispatchThread - Generating results preview took 1 ms
  321. 01-16-2014 14:07:30.758 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
  322. 01-16-2014 14:07:31.721 INFO DispatchThread - Generating results preview took 1 ms
  323. 01-16-2014 14:07:33.459 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 1.23%, Reduce: 0.00%
  324. 01-16-2014 14:07:33.779 INFO DispatchThread - Generating results preview took 1 ms
  325. 01-16-2014 14:07:35.266 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 1.23%, Reduce: 0.00%
  326. 01-16-2014 14:07:35.266 INFO ERP.Stor - JobSubmitter - job.nameSPLK_master1_1389877618.324_0, Successful tasks: 0
  327. 01-16-2014 14:07:35.266 INFO ERP.Stor - JobSubmitter - job.nameSPLK_master1_1389877618.324_0, Failed or killed tasks: 1
  328. 01-16-2014 14:07:35.266 INFO ERP.Stor - JobSubmitter - time spent writing map errors, elapsed_ms=160
  329. 01-16-2014 14:07:35.333 WARN ERP.Stor - SplunkMR$SplunkBaseMapper - Could not create preprocessor object, will try the next one ... class=com.splunk.mr.input.ValueAvroRecordReader, message=File path does not match regex to use this record reader, name=com.splunk.mr.input.ValueAvroRecordReader, path=hdfs://master1:8020/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv, regex=\.avro$.
  330. 01-16-2014 14:07:35.355 INFO ERP.Stor - SplunkMR$SplunkBaseMapper - using class=com.splunk.mr.input.SplunkLineRecordReader to process split=hdfs://master1:8020/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv:201326592+67108864
  331. 01-16-2014 14:07:35.366 INFO SearchOperator:stdin - StdinDataProvider::finish called
  332. 01-16-2014 14:07:35.819 INFO AggregatorMiningProcessor - Got done message for: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  333. 01-16-2014 14:07:35.849 INFO DispatchThread - Generating results preview took 1 ms
  334. 01-16-2014 14:07:37.387 INFO SearchOperator:stdin - preview completed, resetting state and writer ...
  335. 01-16-2014 14:07:37.387 INFO SearchOperator:stdin - setting up new preview state and writer ...
  336. 01-16-2014 14:07:37.387 INFO UserManager - Setting user context: splunk-system-user
  337. 01-16-2014 14:07:37.387 INFO UserManager - Done setting user context: NULL -> splunk-system-user
  338. 01-16-2014 14:07:37.387 INFO UserManager - Unwound user context: splunk-system-user -> NULL
  339. 01-16-2014 14:07:37.387 INFO SearchOperator:stdin - started writer thread, conf=source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv| ...
  340. 01-16-2014 14:07:37.388 INFO UTF8Processor - Converting using CHARSET="UTF-8" for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
  341. 01-16-2014 14:07:37.388 INFO ERP.Stor - FileBasedHeartBeat - stopping hearbeat thread, path=/user/splunk/splunkMR/dispatch/1389877618.324, interval=1000, _startIndex=0, _index=33
  342. 01-16-2014 14:07:37.388 INFO LineBreakingProcessor - Using truncation length 10000 for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
  343. 01-16-2014 14:07:37.388 INFO LineBreakingProcessor - Using lookbehind 100 for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
  344. 01-16-2014 14:07:37.388 INFO DateParserVerbose - Setting maxDaysAgo=1825 and maxDaysHence=7
  345. 01-16-2014 14:07:37.388 INFO AggregatorMiningProcessor - Setting up line merging apparatus for: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  346. 01-16-2014 14:07:37.388 INFO DateParserVerbose - Setting maxDaysAgo=2000 and maxDaysHence=2
  347. 01-16-2014 14:07:37.391 INFO SearchOperator:stdin - StdinDataProvider::finish called
  348. 01-16-2014 14:07:37.429 INFO AggregatorMiningProcessor - Got done message for: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
  349. 01-16-2014 14:07:37.657 INFO SearchOperator:stdin - preview completed, resetting state and writer ...
  350. 01-16-2014 14:07:37.657 INFO SearchOperator:stdin - total preview wait time: 14ms
  351. 01-16-2014 14:07:37.657 ERROR ERP.Stor - SplunkMR - Error while waiting for MapReduce job to complete, job_id=[!http://master1:8088/proxy/application_1389874731460_0009/ job_1389874731460_0009], state=FAILED, reason=Task failed task_1389874731460_0009_m_000021
  352. 01-16-2014 14:07:37.657 ERROR ERP.Stor - Job failed as tasks failed. failedMaps:1 failedReduces:0
  353. 01-16-2014 14:07:37.657 ERROR ERP.Stor - java.io.IOException: Error while waiting for MapReduce job to complete, job_id=[!http://master1:8088/proxy/application_1389874731460_0009/ job_1389874731460_0009], state=FAILED, reason=Task failed task_1389874731460_0009_m_000021
  354. 01-16-2014 14:07:37.657 ERROR ERP.Stor - Job failed as tasks failed. failedMaps:1 failedReduces:0
  355. 01-16-2014 14:07:37.657 ERROR ERP.Stor - at com.splunk.mr.JobSubmitter.waitForCurrentJobToComplete(JobSubmitter.java:321)
  356. 01-16-2014 14:07:37.657 ERROR ERP.Stor - at com.splunk.mr.JobSubmitter.waitForCompletion(JobSubmitter.java:117)
  357. 01-16-2014 14:07:37.657 ERROR ERP.Stor - at com.splunk.mr.SplunkMR$SearchHandler.executeMapReduce(SplunkMR.java:1209)
  358. 01-16-2014 14:07:37.657 ERROR ERP.Stor - at com.splunk.mr.SplunkMR$SearchHandler.executeImpl(SplunkMR.java:1163)
  359. 01-16-2014 14:07:37.657 ERROR ERP.Stor - at com.splunk.mr.SplunkMR$SearchHandler.execute(SplunkMR.java:1086)
  360. 01-16-2014 14:07:37.657 ERROR ERP.Stor - at com.splunk.mr.SplunkMR.runImpl(SplunkMR.java:1381)
  361. 01-16-2014 14:07:37.657 ERROR ERP.Stor - at com.splunk.mr.SplunkMR.run(SplunkMR.java:1223)
  362. 01-16-2014 14:07:37.657 ERROR ERP.Stor - at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
  363. 01-16-2014 14:07:37.657 ERROR ERP.Stor - at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
  364. 01-16-2014 14:07:37.657 ERROR ERP.Stor - at com.splunk.mr.SplunkMR.main(SplunkMR.java:1393)
  365. 01-16-2014 14:07:37.657 ERROR ERP.Stor - at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  366. 01-16-2014 14:07:37.657 ERROR ERP.Stor - at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  367. 01-16-2014 14:07:37.657 ERROR ERP.Stor - at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  368. 01-16-2014 14:07:37.657 ERROR ERP.Stor - at java.lang.reflect.Method.invoke(Method.java:597)
  369. 01-16-2014 14:07:37.657 ERROR ERP.Stor - at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
  370. 01-16-2014 14:07:37.663 ERROR ChunkedOutputStreamReader - Invalid header line="63112697,\N,"240084603079537","467190002277634","4540591209",\N,"N/A","","10370",14,28-JAN-12,"Delivered direct",28-JAN-12,1,0,"Securitas-10370",12,"MS_Legacy",1,"M",28-JAN-12,"",766589,177403303,343"
  371. 01-16-2014 14:07:37.663 ERROR ResultProvider - Error in 'ChunkedOutputStreamReader': Invalid header line="63112697,\N,"240084603079537","467190002277634","4540591209",\N,"N/A","","10370",14,28-JAN-12,"Delivered direct",28-JAN-12,1,0,"Securitas-10370",12,"MS_Legacy",1,"M",28-JAN-12,"",766589,177403303,343"
  372. 01-16-2014 14:07:37.663 INFO UserManager - Unwound user context: admin -> NULL
  373. 01-16-2014 14:07:37.673 INFO ProviderQueue - Round Robin Threaded ProviderQueue: done reading from peer 'Stor'
  374. 01-16-2014 14:07:37.730 INFO UserManager - Unwound user context: admin -> NULL
  375. 01-16-2014 14:07:37.732 INFO UserManager - Setting user context: admin
  376. 01-16-2014 14:07:37.732 INFO UserManager - Done setting user context: NULL -> admin
  377. 01-16-2014 14:07:37.732 INFO UserManager - Unwound user context: admin -> NULL
  378. 01-16-2014 14:07:37.749 INFO DispatchManager - DispatchManager::dispatchHasFinished(id='1389877618.324', username='admin')
  379. 01-16-2014 14:07:37.758 INFO UserManager - Unwound user context: admin -> NULL
  380. 01-16-2014 14:07:37.759 INFO ShutdownHandler - Shutting down splunkd
  381. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_Begin"
  382. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_Thruput"
  383. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_TcpInput"
  384. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_TcpOutput"
  385. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_UdpInput"
  386. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_FifoInput"
  387. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_WinEventLogInput"
  388. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_Scheduler"
  389. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_Tailing"
  390. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_SyslogOutput"
  391. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_HTTPOutput"
  392. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_TailingXP"
  393. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_BatchReader"
  394. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_ArchiveAndOneshot"
  395. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_AuditTrailManager"
  396. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_AuditTrailQueueServiceThread"
  397. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_FSChangeMonitor"
  398. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_FSChangeManagerProcessor"
  399. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_HttpClientPollingThread"
  400. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_AsyncQueuedMessageDispatcherThread"
  401. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_OfflineFlusher"
  402. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_Slave"
  403. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_SlaveSearch"
  404. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_Select"
  405. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_Database1"
  406. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_LoadLDAPUsers"
  407. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_MetricsManager"
  408. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_Pipeline"
  409. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_Queue"
  410. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_Exec"
  411. 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_CallbackRunner"
  412. 01-16-2014 14:07:37.759 INFO ShutdownHandler - Shutdown complete in 291 microseconds
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement