Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- 01-16-2014 14:06:58.197 INFO dispatchRunner - initing LicenseMgr in search process: nonPro=0
- 01-16-2014 14:06:58.197 INFO LicenseMgr - Initing LicenseMgr
- 01-16-2014 14:06:58.197 INFO LMConfig - serverName=master1 guid=6D635ECF-3A04-46EE-A7A2-E944BEA1897C
- 01-16-2014 14:06:58.199 INFO LMConfig - connection_timeout=30
- 01-16-2014 14:06:58.199 INFO LMConfig - send_timeout=30
- 01-16-2014 14:06:58.199 INFO LMConfig - receive_timeout=30
- 01-16-2014 14:06:58.199 INFO LMConfig - squash_threshold=2000
- 01-16-2014 14:06:58.199 INFO LicenseMgr - Initing LicenseMgr runContext_splunkd=false
- 01-16-2014 14:06:58.199 INFO LMStackMgr - closing stack mgr
- 01-16-2014 14:06:58.199 INFO LMSlaveInfo - all slaves cleared
- 01-16-2014 14:06:58.202 INFO LMStack - Added type=download-trial license, from file=enttrial.lic, to stack=download-trial of group=Trial
- 01-16-2014 14:06:58.202 INFO LMStackMgr - created stack='download-trial'
- 01-16-2014 14:06:58.202 INFO LMStackMgr - have to auto-set active stack group='Trial' reason='invalid/missing group id' gidStr='' oldGid=Invalid
- 01-16-2014 14:06:58.202 INFO LMStackMgr - added pool auto_generated_pool_download-trial to stack download-trial
- 01-16-2014 14:06:58.202 INFO LMStackMgr - added pool auto_generated_pool_forwarder to stack forwarder
- 01-16-2014 14:06:58.202 INFO LMStackMgr - added pool auto_generated_pool_free to stack free
- 01-16-2014 14:06:58.202 INFO LMStackMgr - init completed [6D635ECF-3A04-46EE-A7A2-E944BEA1897C,Trial,runContext_splunkd=false]
- 01-16-2014 14:06:58.202 INFO LicenseMgr - StackMgr init complete...
- 01-16-2014 14:06:58.202 INFO LMTracker - this is not splunkd, will perform partial init
- 01-16-2014 14:06:58.202 INFO LMTracker - Setting feature=Auth state=ENABLED (featureStatus=1)
- 01-16-2014 14:06:58.202 INFO LMTracker - Setting feature=FwdData state=ENABLED (featureStatus=1)
- 01-16-2014 14:06:58.202 INFO LMTracker - Setting feature=RcvData state=ENABLED (featureStatus=1)
- 01-16-2014 14:06:58.202 INFO LMTracker - Setting feature=DistSearch state=ENABLED (featureStatus=1)
- 01-16-2014 14:06:58.202 INFO LMTracker - Setting feature=RcvSearch state=ENABLED (featureStatus=1)
- 01-16-2014 14:06:58.202 INFO LMTracker - Setting feature=ScheduledSearch state=ENABLED (featureStatus=1)
- 01-16-2014 14:06:58.202 INFO LMTracker - Setting feature=Alerting state=ENABLED (featureStatus=1)
- 01-16-2014 14:06:58.202 INFO LMTracker - Setting feature=DeployClient state=ENABLED (featureStatus=1)
- 01-16-2014 14:06:58.202 INFO LMTracker - Setting feature=DeployServer state=ENABLED (featureStatus=1)
- 01-16-2014 14:06:58.203 INFO LMTracker - Setting feature=SplunkWeb state=ENABLED (featureStatus=1)
- 01-16-2014 14:06:58.203 INFO LMTracker - Setting feature=SyslogOutputProcessor state=ENABLED (featureStatus=1)
- 01-16-2014 14:06:58.203 INFO LMTracker - Setting feature=SigningProcessor state=ENABLED (featureStatus=1)
- 01-16-2014 14:06:58.203 INFO LMTracker - Setting feature=LocalSearch state=ENABLED (featureStatus=1)
- 01-16-2014 14:06:58.203 INFO LicenseMgr - Tracker init complete...
- 01-16-2014 14:06:58.207 INFO AdminManager - added factory for admin handler: 'licenses'
- 01-16-2014 14:06:58.207 INFO AdminManager - added factory for admin handler: 'pools'
- 01-16-2014 14:06:58.207 INFO AdminManager - added factory for admin handler: 'stacks'
- 01-16-2014 14:06:58.207 INFO AdminManager - added factory for admin handler: 'groups'
- 01-16-2014 14:06:58.207 INFO AdminManager - added factory for admin handler: 'slaves'
- 01-16-2014 14:06:58.207 INFO AdminManager - added factory for admin handler: 'localslave'
- 01-16-2014 14:06:58.207 INFO AdminManager - added factory for admin handler: 'licensermessages'
- 01-16-2014 14:06:58.207 INFO PipelineComponent - registering timer callback name=HTTPAuthManager:timeoutCallback callback=0xcb4e10 arg=0x7f51d6e7de48
- 01-16-2014 14:06:58.207 INFO loader - Splunkd starting (build 189883).
- 01-16-2014 14:06:58.207 INFO loader - System info: Linux, master1, 3.2.0-51-generic, #77-Ubuntu SMP Wed Jul 24 20:18:19 UTC 2013, x86_64.
- 01-16-2014 14:06:58.207 INFO loader - Detected 4 (virtual) CPUs and 16048MB RAM
- 01-16-2014 14:06:58.207 INFO loader - Maximum number of threads (approximate): 8024
- 01-16-2014 14:06:58.207 INFO loader - Arguments are: "search" "--id=1389877618.324" "--maxbuckets=300" "--ttl=600" "--maxout=500000" "--maxtime=8640000" "--lookups=1" "--reduce_freq=10" "--rf=*" "--user=admin" "--pro" "--roles=admin:power:user"
- 01-16-2014 14:06:58.207 INFO loader - Getting search configuration data from: /home/lmcnise/splunk/etc/modules/parsing/config.xml
- 01-16-2014 14:06:58.208 INFO BundlesSetup - Setup stats for /home/lmcnise/splunk/etc: wallclock_elapsed_msec=14, cpu_time_used=0.012, shared_services_generation=2, shared_services_population=1
- 01-16-2014 14:06:58.211 INFO UserManagerPro - Load authentication: forcing roles="admin, power, user"
- 01-16-2014 14:06:58.212 INFO UserManager - Setting user context: admin
- 01-16-2014 14:06:58.212 INFO UserManager - Done setting user context: NULL -> admin
- 01-16-2014 14:06:58.213 INFO dispatchRunner - search context: user="admin", app="search", bs-pathname="/home/lmcnise/splunk/etc"
- 01-16-2014 14:06:58.216 INFO IndexProcessor - Initializing: readonly=true reloading=false
- 01-16-2014 14:06:58.220 INFO HotDBManager - idx=_audit Setting hot mgr params: maxHotSpanSecs=7776000 snapBucketTimespans=false maxHotBuckets=3 maxDataSizeBytes=786432000 quarantinePastSecs=77760000 quarantineFutureSecs=2592000
- 01-16-2014 14:06:58.221 INFO AuditTrailManager - audit stanza does not exist in audit.conf - no signing will take place
- 01-16-2014 14:06:58.221 INFO databasePartitionPolicy - idx=_audit Initialized, params='[300,60,188697600,,,,786432000,5,true,500000,5,5,true,6,0,_blocksignature,7776000,1000000,0,3,77760000,2592000,131072,25,0,15,0,0,-1,18446744073709551615,2592000,true,60000,300000,false,2000]' isSlave=false needApplyDeleteJournal=false
- 01-16-2014 14:06:58.221 INFO HotDBManager - idx=_blocksignature Setting hot mgr params: maxHotSpanSecs=7776000 snapBucketTimespans=false maxHotBuckets=3 maxDataSizeBytes=1048576000 quarantinePastSecs=77760000 quarantineFutureSecs=2592000
- 01-16-2014 14:06:58.221 INFO AuditTrailManager - audit stanza does not exist in audit.conf - no signing will take place
- 01-16-2014 14:06:58.221 INFO databasePartitionPolicy - idx=_blocksignature Initialized, params='[300,60,0,,,,1048576000,5,true,0,5,5,true,6,0,_blocksignature,7776000,1000000,0,3,77760000,2592000,131072,25,0,15,0,0,-1,18446744073709551615,2592000,true,60000,300000,false,2000]' isSlave=false needApplyDeleteJournal=false
- 01-16-2014 14:06:58.221 INFO HotDBManager - idx=_internal Setting hot mgr params: maxHotSpanSecs=7776000 snapBucketTimespans=false maxHotBuckets=3 maxDataSizeBytes=104857600 quarantinePastSecs=77760000 quarantineFutureSecs=2592000
- 01-16-2014 14:06:58.221 INFO AuditTrailManager - audit stanza does not exist in audit.conf - no signing will take place
- 01-16-2014 14:06:58.221 INFO databasePartitionPolicy - idx=_internal Initialized, params='[300,60,2592000,,,,104857600,5,true,500000,5,5,true,6,0,_blocksignature,7776000,1000000,0,3,77760000,2592000,131072,25,0,15,0,0,-1,18446744073709551615,2592000,true,60000,300000,false,2000]' isSlave=false needApplyDeleteJournal=false
- 01-16-2014 14:06:58.221 INFO HotDBManager - idx=_thefishbucket Setting hot mgr params: maxHotSpanSecs=7776000 snapBucketTimespans=false maxHotBuckets=3 maxDataSizeBytes=524288000 quarantinePastSecs=77760000 quarantineFutureSecs=2592000
- 01-16-2014 14:06:58.221 INFO AuditTrailManager - audit stanza does not exist in audit.conf - no signing will take place
- 01-16-2014 14:06:58.221 INFO databasePartitionPolicy - idx=_thefishbucket Initialized, params='[300,60,2419200,,,,524288000,5,true,500000,5,5,true,6,0,_blocksignature,7776000,1000000,0,3,77760000,2592000,131072,25,0,15,0,0,-1,18446744073709551615,2592000,true,60000,300000,false,2000]' isSlave=false needApplyDeleteJournal=false
- 01-16-2014 14:06:58.221 INFO HotDBManager - idx=history Setting hot mgr params: maxHotSpanSecs=7776000 snapBucketTimespans=false maxHotBuckets=3 maxDataSizeBytes=10485760 quarantinePastSecs=77760000 quarantineFutureSecs=2592000
- 01-16-2014 14:06:58.221 INFO AuditTrailManager - audit stanza does not exist in audit.conf - no signing will take place
- 01-16-2014 14:06:58.221 INFO databasePartitionPolicy - idx=history Initialized, params='[300,60,604800,,,,10485760,5,true,500000,5,5,true,6,0,_blocksignature,7776000,1000000,0,3,77760000,2592000,131072,25,0,15,0,0,-1,18446744073709551615,2592000,true,60000,300000,false,2000]' isSlave=false needApplyDeleteJournal=false
- 01-16-2014 14:06:58.221 INFO HotDBManager - idx=main Setting hot mgr params: maxHotSpanSecs=7776000 snapBucketTimespans=false maxHotBuckets=10 maxDataSizeBytes=10737418240 quarantinePastSecs=77760000 quarantineFutureSecs=2592000
- 01-16-2014 14:06:58.221 INFO AuditTrailManager - audit stanza does not exist in audit.conf - no signing will take place
- 01-16-2014 14:06:58.221 INFO databasePartitionPolicy - idx=main Initialized, params='[300,60,188697600,,,,10737418240,5,true,500000,20,5,true,6,0,_blocksignature,7776000,1000000,86400,10,77760000,2592000,131072,25,0,15,0,0,-1,18446744073709551615,2592000,true,60000,300000,false,2000]' isSlave=false needApplyDeleteJournal=false
- 01-16-2014 14:06:58.221 INFO HotDBManager - idx=summary Setting hot mgr params: maxHotSpanSecs=7776000 snapBucketTimespans=false maxHotBuckets=3 maxDataSizeBytes=786432000 quarantinePastSecs=77760000 quarantineFutureSecs=2592000
- 01-16-2014 14:06:58.221 INFO AuditTrailManager - audit stanza does not exist in audit.conf - no signing will take place
- 01-16-2014 14:06:58.221 INFO databasePartitionPolicy - idx=summary Initialized, params='[300,60,188697600,,,,786432000,5,true,500000,5,5,true,6,0,_blocksignature,7776000,1000000,0,3,77760000,2592000,131072,25,0,15,0,0,-1,18446744073709551615,2592000,true,60000,300000,false,2000]' isSlave=false needApplyDeleteJournal=false
- 01-16-2014 14:06:58.221 INFO SearchParser - PARSING: search index=demo-open-day-index source=/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv source="/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv"
- 01-16-2014 14:06:58.222 INFO ISplunkDispatch - Not running in splunkd. Bundle replication not triggered.
- 01-16-2014 14:06:58.247 INFO UserManager - Setting user context: admin
- 01-16-2014 14:06:58.248 INFO UserManager - Done setting user context: NULL -> admin
- 01-16-2014 14:06:58.283 INFO SearchOperator:kv - name=EXTRACT-collection,category,object, can_use_re2=0, regex: collection=\"?(?P<collection>[^\"\n]+)\"?\ncategory=\"?(?P<category>[^\"\n]+)\"?\nobject=\"?(?P<object>[^\"\n]+)\"?\n
- 01-16-2014 14:06:58.283 INFO SearchOperator:kv - name=EXTRACT-GUID, can_use_re2=0, regex: (?i)(?!=\w)(?:objectguid|guid)\s*=\s*(?<guid_lookup>[\w\-]+)
- 01-16-2014 14:06:58.283 INFO SearchOperator:kv - name=EXTRACT-SID, can_use_re2=0, regex: objectSid\s*=\s*(?<sid_lookup>\S+)
- 01-16-2014 14:06:58.286 INFO SearchOperator:kv - name=ad-kv, can_use_re2=0, regex: (?<_KEY_1>[\w-]+)=(?<_VAL_1>[^\r\n]*)
- 01-16-2014 14:06:58.288 INFO SearchOperator:kv - name=access-extractions, can_use_re2=0, regex: ^(?P<clientip>\S+)\s++(?P<ident>\S+)\s++(?P<user>\S+)\s++\[(?<req_time>[^\]]*+)\]\s++"\s*+(?P<method>[^\s"]++)?(?:\s++(?<uri>(?:(?<uri_domain>\w++://[^/\s"]++))?+(?<uri_path>(?:/++(?<root>(?:\\"|[^\s\?/"])++)/++)?(?:(?:\\"|[^\s\?/"])*+/++)*(?<file>[^\s\?/]+)?)(?:\?(?<uri_query>[^\s]*))?)(?:\s++(?P<version>[^\s"]++))*)?\s*+"\s++(?P<status>\S+)\s++(?P<bytes>\S+)(?:\s++"(?<referer>(?:(?<referer_domain>\w++://[^/\s"]++))?+[^"]*+)"(?:\s++"(?<useragent>[^"]*+)"(?:\s++"(?<cookie>[^"]*+)")?+)?+)?(?P<other>.*)
- 01-16-2014 14:06:58.289 INFO SearchOperator:kv - name=syslog-extractions, can_use_re2=0, regex: \s([^\s\[]+)(?:\[(\d+)\])?:\s
- 01-16-2014 14:06:58.290 INFO SearchOperator:kv - name=db2, can_use_re2=0, regex: ([A-Z]+) *: (.*?)(?=\n|$| +[A-Z]+ *:)
- 01-16-2014 14:06:58.290 INFO SearchOperator:kv - name=EXTRACT-extract_spent, can_use_re2=0, regex: (?<spent>\d+)ms$
- 01-16-2014 14:06:58.291 INFO SearchOperator:kv - name=EXTRACT-1, can_use_re2=0, regex: (?<_KEY_1>\S+)::(?<_VAL_1>\S+)
- 01-16-2014 14:06:58.293 INFO SearchOperator:kv - name=bracket-space, can_use_re2=0, regex: \[(\S+) (.*?)\]
- 01-16-2014 14:06:58.295 INFO SearchOperator:kv - name=sendmail-extractions, can_use_re2=0, regex: sendmail\[(\d+)\]: (\w+):
- 01-16-2014 14:06:58.295 INFO SearchOperator:kv - name=tcpdump-endpoints, can_use_re2=0, regex: (\d+\.\d+\.\d+\.\d+):(\d+) -> (\d+\.\d+\.\d+\.\d+):(\d+)
- 01-16-2014 14:06:58.295 INFO SearchOperator:kv - name=colon-kv, can_use_re2=0, regex: (?<= )([A-Za-z]+): ?((0x[A-F\d]+)|\d+)(?= |\n|$)
- 01-16-2014 14:06:58.307 INFO SearchOperator:kv - name=wel-message, can_use_re2=0, regex: (?sm)^(?<_pre_msg>.+)\nMessage=(?<Message>.+)$
- 01-16-2014 14:06:58.307 INFO SearchOperator:kv - name=wel-col-kv, can_use_re2=0, regex: \n([^:\n\r]+):[ \t]++([^\n]*)
- 01-16-2014 14:06:58.308 INFO SearchOperator:kv - name=EXTRACT-useragent, can_use_re2=0, regex: userAgent=(?P<browser>[^ (]+)
- 01-16-2014 14:06:58.308 INFO SearchOperator:kv - name=splunk-service-extractions, can_use_re2=0, regex: (?i)^(?:[^ ]* ){2}(?P<log_level>[^\s]*)\s+\[(?P<requestid>\w+)]\s+(?P<component>[^ ]+):(?P<line>\d+) - (?P<message>.+)
- 01-16-2014 14:06:58.308 INFO SearchOperator:kv - name=EXTRACT-fields, can_use_re2=0, regex: (?i)^(?:[^ ]* ){2}(?:[+\-]\d+ )?(?P<log_level>[^ ]*)\s+(?P<component>[^ ]+) - (?P<message>.+)
- 01-16-2014 14:06:58.308 INFO SearchOperator:kv - name=extract_spent, can_use_re2=0, regex: (?P<spent>\d+)ms$
- 01-16-2014 14:06:58.309 INFO SearchOperator:kv - name=weblogic-code, can_use_re2=0, regex: <BEA-([0-9]+)>
- 01-16-2014 14:06:58.310 INFO SearchOperator:kv - name=colon-line, can_use_re2=0, regex: ^(\w+)\s*:[ \t]*(.*?)$
- 01-16-2014 14:06:58.310 INFO SearchOperator:kv - name=was-trlog-code, can_use_re2=0, regex: ] ([a-fA-F0-9]{8})
- 01-16-2014 14:06:58.311 INFO UnifiedSearch - base lispy: [ AND index::demo-open-day-index source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv ]
- 01-16-2014 14:06:58.312 INFO DispatchThread - BatchMode: allowBatchMode: 0, conf(1): 1, timeline/Status buckets(0):300, realtime(0):0, report pipe empty(0):1, reqTimeOrder(0):0, summarize(0):0, statefulStreaming(0):0
- 01-16-2014 14:06:58.312 INFO DispatchThread - Storing only 1000 events per timeline buckets due to limits.conf max_events_per_bucket setting.
- 01-16-2014 14:06:58.312 INFO DispatchThread - required fields list = *,_bkt,_cd,_si,host,index,linecount,source,sourcetype,splunk_server
- 01-16-2014 14:06:58.312 INFO SearchParser - PARSING: fields keepcolorder=t "*" "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server"
- 01-16-2014 14:06:58.312 INFO ProviderQueue - Stream search: litsearch index=demo-open-day-index source=/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv source="/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv" | fields keepcolorder=t "*" "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server"
- 01-16-2014 14:06:58.313 INFO ProviderQueue - Search proc: index=demo-open-day-index source=/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv source="/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv"
- 01-16-2014 14:06:58.313 INFO ProviderQueue - Search after asserting splunk_server=local: ((index="demo-open-day-index") (source="/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv") (source="/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv"))
- 01-16-2014 14:06:58.313 INFO ProviderQueue - Found referenced index, name=demo-open-day-index, provider=Stor
- 01-16-2014 14:06:58.314 INFO ProviderQueue - Stream search: litsearch index=demo-open-day-index source=/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv source="/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv" | fields keepcolorder=t "*" "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server"
- 01-16-2014 14:06:58.314 INFO ProviderQueue - round_robin=1
- 01-16-2014 14:06:58.373 INFO ProviderQueue - Found bundlePath=/home/lmcnise/splunk/var/run/master1-1389774711.bundle
- 01-16-2014 14:06:58.373 INFO ProviderQueue - Creating external result provider=Stor, with index.count=1, search="search (index="demo-open-day-index" ((source="/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv") (source="/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv"))) | fields keepcolorder=t "*" "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server""
- 01-16-2014 14:06:58.373 INFO SearchParser - PARSING: stdin | search (index="demo-open-day-index" ((source="/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv") (source="/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv"))) | fields keepcolorder=t "*" "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server"
- 01-16-2014 14:06:58.395 INFO SearchParser - PARSING: typer | tags
- 01-16-2014 14:06:58.397 INFO FastTyper - found nodes count: comparisons=6, unique_comparisons=5, terms=4, unique_terms=4, phrases=12, unique_phrases=12, total leaves=22
- 01-16-2014 14:06:58.412 INFO SearchOperator:stdin - setting _need_timestamp_fields=1, required time field name=date_second
- 01-16-2014 14:06:58.412 INFO SearchOperator:stdin - required fields list = *,Message,_bkt,_cd,_si,_time,host,index,linecount,source,sourcetype,splunk_server
- 01-16-2014 14:06:58.412 INFO ResultProvider - turning off streaming mode for provider name=Stor
- 01-16-2014 14:06:58.412 INFO ResultProvider - provider=Stor, mode.config=report, mode.search=mixed
- 01-16-2014 14:06:58.413 INFO ERP.Stor - Starting: /home/lmcnise/splunk/bin/jars/sudobash /usr/lib/hadoop/bin/hadoop jar "/home/lmcnise/splunk/bin/jars/SplunkMR-s6.0-hy2.0.jar" "com.splunk.mr.SplunkMR"
- 01-16-2014 14:06:58.413 INFO ProviderQueue - using single thread to setup ResultProviders
- 01-16-2014 14:06:58.413 INFO UserManager - Setting user context: admin
- 01-16-2014 14:06:58.413 INFO UserManager - Setting user context: admin
- 01-16-2014 14:06:58.414 INFO UserManager - Done setting user context: NULL -> admin
- 01-16-2014 14:06:58.414 INFO UserManager - Done setting user context: admin -> admin
- 01-16-2014 14:06:58.414 INFO ResultProvider - Creating result provider for peer: local
- 01-16-2014 14:06:58.414 INFO SearchParser - PARSING: litsearch index=demo-open-day-index source=/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv source="/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv" | fields keepcolorder=t "*" "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server"
- 01-16-2014 14:06:58.437 INFO SearchParser - PARSING: typer | tags
- 01-16-2014 14:06:58.439 INFO FastTyper - found nodes count: comparisons=6, unique_comparisons=5, terms=4, unique_terms=4, phrases=12, unique_phrases=12, total leaves=22
- 01-16-2014 14:06:58.446 INFO ResultProvider - Successfully created result provider for peer: local in 0.033000 seconds
- 01-16-2014 14:06:58.502 INFO UserManager - Unwound user context: admin -> admin
- 01-16-2014 14:06:58.502 INFO UserManager - Setting user context: admin
- 01-16-2014 14:06:58.502 INFO UserManager - Done setting user context: NULL -> admin
- 01-16-2014 14:06:58.502 INFO DispatchThread - Disk quota = 10485760000
- 01-16-2014 14:06:58.504 INFO UserManager - Unwound user context: admin -> NULL
- 01-16-2014 14:06:58.504 INFO ProviderQueue - Round Robin Threaded ProviderQueue: done reading from peer 'master1'
- 01-16-2014 14:06:59.594 WARN ERP.Stor - Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
- 01-16-2014 14:06:59.642 INFO ERP.Stor - SplunkMR$SearchHandler - Reduce search: null
- 01-16-2014 14:06:59.642 INFO ERP.Stor - SplunkMR$SearchHandler - Search mode: mixed
- 01-16-2014 14:07:00.476 INFO ERP.Stor - SplunkMR$SearchHandler - Created filesystem object, elapsed_ms=834
- 01-16-2014 14:07:00.486 INFO ERP.Stor - DispatchReaper - Starting. Settings: hdfs.path=/user/splunk/splunkMR/dispatch, local.path=/home/lmcnise/splunk/var/run/splunk/dispatch
- 01-16-2014 14:07:00.572 INFO ERP.Stor - VirtualIndex - listStatus started, vix.name=demo-open-day-index ...
- 01-16-2014 14:07:00.584 INFO ERP.Stor - AbstractService - Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
- 01-16-2014 14:07:00.676 INFO ERP.Stor - AbstractService - Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
- 01-16-2014 14:07:00.677 INFO ERP.Stor - DispatchReaper - Deleted HDFS dispatch dir=hdfs://master1:8020/user/splunk/splunkMR/dispatch/1389875809.318, local dispatch dir=/home/lmcnise/splunk/var/run/splunk/dispatch/1389875809.318 does not exist
- 01-16-2014 14:07:00.677 INFO ERP.Stor - DispatchReaper - Finished, scanned=4, deleted=1, errors=0
- 01-16-2014 14:07:00.782 INFO ERP.Stor - VirtualIndex - listStatus done, vix.name=demo-open-day-index, files.total=1, files.time.filtered=0, files.search.filtered=0, files.mr=1, elapsed=210ms
- 01-16-2014 14:07:00.783 INFO ERP.Stor - JobSubmitter - finished called: _splits.count=81, total.count=81
- 01-16-2014 14:07:00.783 INFO ERP.Stor - JobSubmitter - creating new job, curent split.count=81
- 01-16-2014 14:07:00.783 INFO ERP.Stor - SplunkMR$SearchHandler - state=start, set up the required env in HDFS for MR spawning search
- 01-16-2014 14:07:00.787 WARN ERP.Stor - SplunkMR$SplunkBaseMapper - Could not create preprocessor object, will try the next one ... class=com.splunk.mr.input.ValueAvroRecordReader, message=File path does not match regex to use this record reader, name=com.splunk.mr.input.ValueAvroRecordReader, path=hdfs://master1:8020/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv, regex=\.avro$.
- 01-16-2014 14:07:00.825 INFO ERP.Stor - SplunkMR$SplunkBaseMapper - using class=com.splunk.mr.input.SplunkLineRecordReader to process split=hdfs://master1:8020/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv:0+67108864
- 01-16-2014 14:07:00.825 WARN ERP.Stor - ResourceMgrDelegate - getBlacklistedTrackers - Not implemented yet
- 01-16-2014 14:07:00.826 INFO ERP.Stor - ClusterInfoLogger - Hadoop cluster spec: provider=Stor, tasktrackers=27, map_inuse=1, map_slots=270, reduce_inuse=1, reduce_slots=54
- 01-16-2014 14:07:00.864 INFO ERP.Stor - SplunkMR$SearchHandler - state=done, set up the required env in HDFS for MR spawning search
- 01-16-2014 14:07:00.864 INFO ERP.Stor - FileBasedHeartBeat - starting hearbeat thread, path=/user/splunk/splunkMR/dispatch/1389877618.324, interval=1000, _startIndex=0, _index=0
- 01-16-2014 14:07:00.890 INFO ERP.Stor - JobSubmitter - submitting new job, name=SPLK_master1_1389877618.324_0
- 01-16-2014 14:07:00.891 INFO ERP.Stor - AsyncMRJob - AsyncMRJob job.name=SPLK_master1_1389877618.324_0 running ...
- 01-16-2014 14:07:00.891 INFO ERP.Stor - AsyncMRJob - Submitting job.name=SPLK_master1_1389877618.324_0 ...
- 01-16-2014 14:07:00.924 INFO ERP.Stor - AbstractService - Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
- 01-16-2014 14:07:00.925 INFO ERP.Stor - AbstractService - Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
- 01-16-2014 14:07:01.006 INFO SearchOperator:stdin - Initializing from configuration
- 01-16-2014 14:07:01.008 INFO PipelineComponent - registering timer callback name=MetricsManager:probeandreport callback=0xb07120 arg=0x7f51d2c2c400
- 01-16-2014 14:07:01.008 INFO PipelineComponent - registering timer callback name=triggerCollection callback=0xb4e230 arg=0x7f51d2c4c180
- 01-16-2014 14:07:01.008 INFO LineBreakingProcessor - Initializing
- 01-16-2014 14:07:01.008 INFO regexExtractionProcessor - Initializing
- 01-16-2014 14:07:01.010 INFO PipelineComponent - Launching the pipelines.
- 01-16-2014 14:07:01.010 INFO SearchOperator:stdin - setting up new preview state and writer ...
- 01-16-2014 14:07:01.010 INFO pipeline - Registering metrics callback for: Pipeline:vix
- 01-16-2014 14:07:01.015 INFO UserManager - Setting user context: splunk-system-user
- 01-16-2014 14:07:01.015 INFO UserManager - Done setting user context: NULL -> splunk-system-user
- 01-16-2014 14:07:01.016 INFO UserManager - Unwound user context: splunk-system-user -> NULL
- 01-16-2014 14:07:01.016 INFO SearchOperator:stdin - started writer thread, conf=source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv| ...
- 01-16-2014 14:07:01.016 INFO UTF8Processor - Converting using CHARSET="UTF-8" for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
- 01-16-2014 14:07:01.016 INFO LineBreakingProcessor - Using truncation length 10000 for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
- 01-16-2014 14:07:01.016 INFO LineBreakingProcessor - Using lookbehind 100 for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
- 01-16-2014 14:07:01.016 INFO DateParserVerbose - Setting maxDaysAgo=1825 and maxDaysHence=7
- 01-16-2014 14:07:01.016 INFO AggregatorMiningProcessor - Setting up line merging apparatus for: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:01.018 INFO DateParserVerbose - Setting maxDaysAgo=2000 and maxDaysHence=2
- 01-16-2014 14:07:01.019 WARN DateParserVerbose - A possible timestamp match (Mon Feb 15 07:27:31 1999) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:01.019 WARN DateParserVerbose - A possible timestamp match (Mon Jul 7 11:38:56 2014) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:01.019 WARN DateParserVerbose - A possible timestamp match (Wed Feb 24 03:23:20 1999) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:01.020 WARN SearchOperator:kv - host is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:01.020 WARN SearchOperator:kv - index is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:01.020 WARN SearchOperator:kv - linecount is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:01.020 WARN SearchOperator:kv - source is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:01.020 WARN SearchOperator:kv - sourcetype is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:01.020 WARN SearchOperator:kv - splunk_server is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:01.020 WARN DateParserVerbose - A possible timestamp match (Wed Sep 9 10:23:34 1998) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:01.020 WARN SearchOperator:kv - host is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:01.021 WARN DateParserVerbose - A possible timestamp match (Fri Aug 18 02:56:08 2000) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:01.021 WARN SearchOperator:kv - index is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:01.021 WARN SearchOperator:kv - linecount is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:01.021 WARN SearchOperator:kv - source is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:01.021 WARN SearchOperator:kv - sourcetype is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:01.021 WARN DateParserVerbose - A possible timestamp match (Wed Sep 9 10:29:25 1998) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:01.021 WARN DateParserVerbose - A possible timestamp match (Wed Sep 9 10:29:25 1998) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:01.021 WARN SearchOperator:kv - splunk_server is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:01.021 WARN DateParserVerbose - A possible timestamp match (Thu Aug 31 16:10:00 2000) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:01.021 WARN DateParserVerbose - A possible timestamp match (Wed Feb 24 03:23:20 1999) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:01.021 WARN DateParserVerbose - A possible timestamp match (Wed Feb 24 03:23:20 1999) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:01.033 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Tue Jan 6 01:50:00 2009). Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:01.037 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Tue Jan 6 01:50:00 2009). Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:01.037 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Tue Jan 6 01:50:00 2009). Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:01.037 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Tue Jan 6 01:50:00 2009). Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:01.037 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Tue Jan 6 01:50:00 2009). Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:01.038 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Tue Jan 6 01:50:00 2009). Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:01.038 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Tue Jan 6 01:50:00 2009). Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:01.038 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Tue Jan 6 01:50:00 2009). Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:01.038 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Tue Jan 6 01:50:00 2009). Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:01.038 WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Tue Jan 6 01:50:00 2009). Context: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:01.777 INFO ERP.Stor - JobSubmitter - number of splits:81
- 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.jar is deprecated. Instead, use mapreduce.job.jar
- 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
- 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.job.reduce.memory.mb is deprecated. Instead, use mapreduce.reduce.memory.mb
- 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
- 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class
- 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
- 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.job.name is deprecated. Instead, use mapreduce.job.name
- 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
- 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class
- 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
- 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
- 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapreduce.outputformat.class is deprecated. Instead, use mapreduce.job.outputformat.class
- 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
- 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.job.queue.name is deprecated. Instead, use mapreduce.job.queuename
- 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class
- 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.job.reuse.jvm.num.tasks is deprecated. Instead, use mapreduce.job.jvm.numtasks
- 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.job.map.memory.mb is deprecated. Instead, use mapreduce.map.memory.mb
- 01-16-2014 14:07:01.777 WARN ERP.Stor - Configuration - mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
- 01-16-2014 14:07:01.777 INFO ERP.Stor - JobSubmitter - Submitting tokens for job: job_1389874731460_0009
- 01-16-2014 14:07:01.778 INFO DispatchThread - Generating results preview took 1 ms
- 01-16-2014 14:07:01.901 INFO ERP.Stor - YarnClientImpl - Submitted application application_1389874731460_0009 to ResourceManager at master1/192.168.1.1:8032
- 01-16-2014 14:07:02.040 WARN SearchOperator:kv - date_hour is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:02.040 WARN SearchOperator:kv - date_mday is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:02.040 WARN SearchOperator:kv - date_minute is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:02.040 WARN SearchOperator:kv - date_month is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:02.040 WARN SearchOperator:kv - date_second is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:02.041 WARN SearchOperator:kv - date_wday is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:02.041 WARN SearchOperator:kv - date_year is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:02.041 WARN SearchOperator:kv - date_zone is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:02.041 WARN SearchOperator:kv - timeendpos is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:02.041 WARN SearchOperator:kv - timestartpos is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:02.263 WARN SearchOperator:kv - date_hour is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:02.263 WARN SearchOperator:kv - date_mday is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:02.263 WARN SearchOperator:kv - date_minute is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:02.263 WARN SearchOperator:kv - date_month is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:02.263 WARN SearchOperator:kv - date_second is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:02.263 WARN SearchOperator:kv - date_wday is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:02.263 WARN SearchOperator:kv - date_year is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:02.263 WARN SearchOperator:kv - date_zone is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:02.263 WARN SearchOperator:kv - timeendpos is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:02.263 WARN SearchOperator:kv - timestartpos is an indexed field, ignoring TOKENIZER
- 01-16-2014 14:07:02.434 INFO ERP.Stor - Job - The url to track the job: http://master1:8088/proxy/application_1389874731460_0009/
- 01-16-2014 14:07:02.434 INFO ERP.Stor - AsyncMRJob - Done submitting job.name=SPLK_master1_1389877618.324_0, url=http://master1:8088/proxy/application_1389874731460_0009/
- 01-16-2014 14:07:02.434 INFO ERP.Stor - AbstractService - Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
- 01-16-2014 14:07:02.434 INFO ERP.Stor - AbstractService - Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
- 01-16-2014 14:07:02.434 INFO ERP.Stor - JobSubmitter - waiting for job to complete job.name=SPLK_master1_1389877618.324_0
- 01-16-2014 14:07:03.647 INFO DispatchThread - Generating results preview took 1 ms
- 01-16-2014 14:07:05.095 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
- 01-16-2014 14:07:05.475 INFO DispatchThread - Generating results preview took 1 ms
- 01-16-2014 14:07:06.340 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
- 01-16-2014 14:07:06.872 INFO DispatchThread - Generating results preview took 1 ms
- 01-16-2014 14:07:08.427 INFO DispatchThread - Generating results preview took 1 ms
- 01-16-2014 14:07:08.867 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
- 01-16-2014 14:07:10.684 INFO DispatchThread - Generating results preview took 1 ms
- 01-16-2014 14:07:11.326 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
- 01-16-2014 14:07:11.353 WARN ERP.Stor - SplunkMR$SplunkBaseMapper - Could not create preprocessor object, will try the next one ... class=com.splunk.mr.input.ValueAvroRecordReader, message=File path does not match regex to use this record reader, name=com.splunk.mr.input.ValueAvroRecordReader, path=hdfs://master1:8020/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv, regex=\.avro$.
- 01-16-2014 14:07:11.364 INFO ERP.Stor - SplunkMR$SplunkBaseMapper - using class=com.splunk.mr.input.SplunkLineRecordReader to process split=hdfs://master1:8020/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv:67108864+67108864
- 01-16-2014 14:07:11.402 INFO SearchOperator:stdin - StdinDataProvider::finish called
- 01-16-2014 14:07:11.509 INFO AggregatorMiningProcessor - Got done message for: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:12.048 INFO DispatchThread - Generating results preview took 1 ms
- 01-16-2014 14:07:13.084 INFO DispatchThread - Generating results preview took 1 ms
- 01-16-2014 14:07:15.626 INFO SearchOperator:stdin - preview completed, resetting state and writer ...
- 01-16-2014 14:07:15.778 INFO SearchOperator:stdin - setting up new preview state and writer ...
- 01-16-2014 14:07:15.779 INFO UserManager - Setting user context: splunk-system-user
- 01-16-2014 14:07:15.779 INFO UserManager - Done setting user context: NULL -> splunk-system-user
- 01-16-2014 14:07:15.779 INFO UserManager - Unwound user context: splunk-system-user -> NULL
- 01-16-2014 14:07:15.779 INFO SearchOperator:stdin - started writer thread, conf=source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv| ...
- 01-16-2014 14:07:15.779 INFO UTF8Processor - Converting using CHARSET="UTF-8" for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
- 01-16-2014 14:07:15.779 INFO LineBreakingProcessor - Using truncation length 10000 for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
- 01-16-2014 14:07:15.779 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
- 01-16-2014 14:07:15.779 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
- 01-16-2014 14:07:15.779 INFO LineBreakingProcessor - Using lookbehind 100 for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
- 01-16-2014 14:07:15.779 INFO DateParserVerbose - Setting maxDaysAgo=1825 and maxDaysHence=7
- 01-16-2014 14:07:15.779 INFO AggregatorMiningProcessor - Setting up line merging apparatus for: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:15.779 INFO DateParserVerbose - Setting maxDaysAgo=2000 and maxDaysHence=2
- 01-16-2014 14:07:15.814 INFO DispatchThread - Generating results preview took 1 ms
- 01-16-2014 14:07:17.385 INFO DispatchThread - Generating results preview took 1 ms
- 01-16-2014 14:07:17.564 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
- 01-16-2014 14:07:18.457 INFO DispatchThread - Generating results preview took 1 ms
- 01-16-2014 14:07:19.013 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
- 01-16-2014 14:07:20.384 INFO DispatchThread - Generating results preview took 1 ms
- 01-16-2014 14:07:21.251 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
- 01-16-2014 14:07:21.650 INFO DispatchThread - Generating results preview took 1 ms
- 01-16-2014 14:07:22.701 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
- 01-16-2014 14:07:23.124 INFO DispatchThread - Generating results preview took 1 ms
- 01-16-2014 14:07:24.154 WARN ERP.Stor - SplunkMR$SplunkBaseMapper - Could not create preprocessor object, will try the next one ... class=com.splunk.mr.input.ValueAvroRecordReader, message=File path does not match regex to use this record reader, name=com.splunk.mr.input.ValueAvroRecordReader, path=hdfs://master1:8020/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv, regex=\.avro$.
- 01-16-2014 14:07:24.164 INFO ERP.Stor - SplunkMR$SplunkBaseMapper - using class=com.splunk.mr.input.SplunkLineRecordReader to process split=hdfs://master1:8020/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv:134217728+67108864
- 01-16-2014 14:07:26.003 INFO SearchOperator:stdin - StdinDataProvider::finish called
- 01-16-2014 14:07:26.003 INFO AggregatorMiningProcessor - Got done message for: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:26.422 INFO DispatchThread - Generating results preview took 1 ms
- 01-16-2014 14:07:26.840 INFO SearchOperator:stdin - preview completed, resetting state and writer ...
- 01-16-2014 14:07:26.840 INFO SearchOperator:stdin - setting up new preview state and writer ...
- 01-16-2014 14:07:26.840 INFO UserManager - Setting user context: splunk-system-user
- 01-16-2014 14:07:26.840 INFO UserManager - Done setting user context: NULL -> splunk-system-user
- 01-16-2014 14:07:26.840 INFO UserManager - Unwound user context: splunk-system-user -> NULL
- 01-16-2014 14:07:26.840 INFO SearchOperator:stdin - started writer thread, conf=source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv| ...
- 01-16-2014 14:07:26.840 INFO UTF8Processor - Converting using CHARSET="UTF-8" for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
- 01-16-2014 14:07:26.840 INFO LineBreakingProcessor - Using truncation length 10000 for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
- 01-16-2014 14:07:26.841 INFO LineBreakingProcessor - Using lookbehind 100 for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
- 01-16-2014 14:07:26.841 INFO DateParserVerbose - Setting maxDaysAgo=1825 and maxDaysHence=7
- 01-16-2014 14:07:26.841 INFO AggregatorMiningProcessor - Setting up line merging apparatus for: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:26.841 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
- 01-16-2014 14:07:26.841 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
- 01-16-2014 14:07:26.841 INFO DateParserVerbose - Setting maxDaysAgo=2000 and maxDaysHence=2
- 01-16-2014 14:07:28.044 INFO DispatchThread - Generating results preview took 1 ms
- 01-16-2014 14:07:28.957 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
- 01-16-2014 14:07:29.656 INFO DispatchThread - Generating results preview took 1 ms
- 01-16-2014 14:07:30.758 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 0.00%, Reduce: 0.00%
- 01-16-2014 14:07:31.721 INFO DispatchThread - Generating results preview took 1 ms
- 01-16-2014 14:07:33.459 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 1.23%, Reduce: 0.00%
- 01-16-2014 14:07:33.779 INFO DispatchThread - Generating results preview took 1 ms
- 01-16-2014 14:07:35.266 INFO ERP.Stor - JobSubmitter - JobId=job_1389874731460_0009, Map: 1.23%, Reduce: 0.00%
- 01-16-2014 14:07:35.266 INFO ERP.Stor - JobSubmitter - job.nameSPLK_master1_1389877618.324_0, Successful tasks: 0
- 01-16-2014 14:07:35.266 INFO ERP.Stor - JobSubmitter - job.nameSPLK_master1_1389877618.324_0, Failed or killed tasks: 1
- 01-16-2014 14:07:35.266 INFO ERP.Stor - JobSubmitter - time spent writing map errors, elapsed_ms=160
- 01-16-2014 14:07:35.333 WARN ERP.Stor - SplunkMR$SplunkBaseMapper - Could not create preprocessor object, will try the next one ... class=com.splunk.mr.input.ValueAvroRecordReader, message=File path does not match regex to use this record reader, name=com.splunk.mr.input.ValueAvroRecordReader, path=hdfs://master1:8020/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv, regex=\.avro$.
- 01-16-2014 14:07:35.355 INFO ERP.Stor - SplunkMR$SplunkBaseMapper - using class=com.splunk.mr.input.SplunkLineRecordReader to process split=hdfs://master1:8020/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv:201326592+67108864
- 01-16-2014 14:07:35.366 INFO SearchOperator:stdin - StdinDataProvider::finish called
- 01-16-2014 14:07:35.819 INFO AggregatorMiningProcessor - Got done message for: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:35.849 INFO DispatchThread - Generating results preview took 1 ms
- 01-16-2014 14:07:37.387 INFO SearchOperator:stdin - preview completed, resetting state and writer ...
- 01-16-2014 14:07:37.387 INFO SearchOperator:stdin - setting up new preview state and writer ...
- 01-16-2014 14:07:37.387 INFO UserManager - Setting user context: splunk-system-user
- 01-16-2014 14:07:37.387 INFO UserManager - Done setting user context: NULL -> splunk-system-user
- 01-16-2014 14:07:37.387 INFO UserManager - Unwound user context: splunk-system-user -> NULL
- 01-16-2014 14:07:37.387 INFO SearchOperator:stdin - started writer thread, conf=source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv| ...
- 01-16-2014 14:07:37.388 INFO UTF8Processor - Converting using CHARSET="UTF-8" for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
- 01-16-2014 14:07:37.388 INFO ERP.Stor - FileBasedHeartBeat - stopping hearbeat thread, path=/user/splunk/splunkMR/dispatch/1389877618.324, interval=1000, _startIndex=0, _index=33
- 01-16-2014 14:07:37.388 INFO LineBreakingProcessor - Using truncation length 10000 for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
- 01-16-2014 14:07:37.388 INFO LineBreakingProcessor - Using lookbehind 100 for conf "source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|"
- 01-16-2014 14:07:37.388 INFO DateParserVerbose - Setting maxDaysAgo=1825 and maxDaysHence=7
- 01-16-2014 14:07:37.388 INFO AggregatorMiningProcessor - Setting up line merging apparatus for: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:37.388 INFO DateParserVerbose - Setting maxDaysAgo=2000 and maxDaysHence=2
- 01-16-2014 14:07:37.391 INFO SearchOperator:stdin - StdinDataProvider::finish called
- 01-16-2014 14:07:37.429 INFO AggregatorMiningProcessor - Got done message for: source::/user/lmcnise/demo-er-jan21/sms_jan-feb-mar-12.csv|host::master1|csv|
- 01-16-2014 14:07:37.657 INFO SearchOperator:stdin - preview completed, resetting state and writer ...
- 01-16-2014 14:07:37.657 INFO SearchOperator:stdin - total preview wait time: 14ms
- 01-16-2014 14:07:37.657 ERROR ERP.Stor - SplunkMR - Error while waiting for MapReduce job to complete, job_id=[!http://master1:8088/proxy/application_1389874731460_0009/ job_1389874731460_0009], state=FAILED, reason=Task failed task_1389874731460_0009_m_000021
- 01-16-2014 14:07:37.657 ERROR ERP.Stor - Job failed as tasks failed. failedMaps:1 failedReduces:0
- 01-16-2014 14:07:37.657 ERROR ERP.Stor - java.io.IOException: Error while waiting for MapReduce job to complete, job_id=[!http://master1:8088/proxy/application_1389874731460_0009/ job_1389874731460_0009], state=FAILED, reason=Task failed task_1389874731460_0009_m_000021
- 01-16-2014 14:07:37.657 ERROR ERP.Stor - Job failed as tasks failed. failedMaps:1 failedReduces:0
- 01-16-2014 14:07:37.657 ERROR ERP.Stor - at com.splunk.mr.JobSubmitter.waitForCurrentJobToComplete(JobSubmitter.java:321)
- 01-16-2014 14:07:37.657 ERROR ERP.Stor - at com.splunk.mr.JobSubmitter.waitForCompletion(JobSubmitter.java:117)
- 01-16-2014 14:07:37.657 ERROR ERP.Stor - at com.splunk.mr.SplunkMR$SearchHandler.executeMapReduce(SplunkMR.java:1209)
- 01-16-2014 14:07:37.657 ERROR ERP.Stor - at com.splunk.mr.SplunkMR$SearchHandler.executeImpl(SplunkMR.java:1163)
- 01-16-2014 14:07:37.657 ERROR ERP.Stor - at com.splunk.mr.SplunkMR$SearchHandler.execute(SplunkMR.java:1086)
- 01-16-2014 14:07:37.657 ERROR ERP.Stor - at com.splunk.mr.SplunkMR.runImpl(SplunkMR.java:1381)
- 01-16-2014 14:07:37.657 ERROR ERP.Stor - at com.splunk.mr.SplunkMR.run(SplunkMR.java:1223)
- 01-16-2014 14:07:37.657 ERROR ERP.Stor - at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
- 01-16-2014 14:07:37.657 ERROR ERP.Stor - at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
- 01-16-2014 14:07:37.657 ERROR ERP.Stor - at com.splunk.mr.SplunkMR.main(SplunkMR.java:1393)
- 01-16-2014 14:07:37.657 ERROR ERP.Stor - at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
- 01-16-2014 14:07:37.657 ERROR ERP.Stor - at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
- 01-16-2014 14:07:37.657 ERROR ERP.Stor - at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
- 01-16-2014 14:07:37.657 ERROR ERP.Stor - at java.lang.reflect.Method.invoke(Method.java:597)
- 01-16-2014 14:07:37.657 ERROR ERP.Stor - at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
- 01-16-2014 14:07:37.663 ERROR ChunkedOutputStreamReader - Invalid header line="63112697,\N,"240084603079537","467190002277634","4540591209",\N,"N/A","","10370",14,28-JAN-12,"Delivered direct",28-JAN-12,1,0,"Securitas-10370",12,"MS_Legacy",1,"M",28-JAN-12,"",766589,177403303,343"
- 01-16-2014 14:07:37.663 ERROR ResultProvider - Error in 'ChunkedOutputStreamReader': Invalid header line="63112697,\N,"240084603079537","467190002277634","4540591209",\N,"N/A","","10370",14,28-JAN-12,"Delivered direct",28-JAN-12,1,0,"Securitas-10370",12,"MS_Legacy",1,"M",28-JAN-12,"",766589,177403303,343"
- 01-16-2014 14:07:37.663 INFO UserManager - Unwound user context: admin -> NULL
- 01-16-2014 14:07:37.673 INFO ProviderQueue - Round Robin Threaded ProviderQueue: done reading from peer 'Stor'
- 01-16-2014 14:07:37.730 INFO UserManager - Unwound user context: admin -> NULL
- 01-16-2014 14:07:37.732 INFO UserManager - Setting user context: admin
- 01-16-2014 14:07:37.732 INFO UserManager - Done setting user context: NULL -> admin
- 01-16-2014 14:07:37.732 INFO UserManager - Unwound user context: admin -> NULL
- 01-16-2014 14:07:37.749 INFO DispatchManager - DispatchManager::dispatchHasFinished(id='1389877618.324', username='admin')
- 01-16-2014 14:07:37.758 INFO UserManager - Unwound user context: admin -> NULL
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - Shutting down splunkd
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_Begin"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_Thruput"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_TcpInput"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_TcpOutput"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_UdpInput"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_FifoInput"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_WinEventLogInput"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_Scheduler"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_Tailing"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_SyslogOutput"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_HTTPOutput"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_TailingXP"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_BatchReader"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_ArchiveAndOneshot"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_AuditTrailManager"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_AuditTrailQueueServiceThread"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_FSChangeMonitor"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_FSChangeManagerProcessor"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_HttpClientPollingThread"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_AsyncQueuedMessageDispatcherThread"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_OfflineFlusher"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_Slave"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_SlaveSearch"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_Select"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_Database1"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_LoadLDAPUsers"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_MetricsManager"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_Pipeline"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_Queue"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_Exec"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - shutting down level "ShutdownLevel_CallbackRunner"
- 01-16-2014 14:07:37.759 INFO ShutdownHandler - Shutdown complete in 291 microseconds
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement