Installation Steps:

•	Copy file from local to location where you installed hadoop ( cp ~/Desktop/POC/apache-flume-1.6.0-bin.tar.gz  /opt)
•	Run sudo tar -xzvf apache-flume-1.6.0-bin.tar.gz
•	Now go to folder flume-1.6.0-bin/conf( cd apache-flume-1.6.0-bin/conf folder)
•	Rename the flume-conf.properties.template file (sudo mv flume-conf.properties.template flume-conf.properties)
•	Now rename flume-env.sh.template (sudo mv flume-env.sh.template flume-env.sh)
•	Add this line export JAVA_HOME=/usr/lib/jvm/jdk1.7.0_51
•	Now go to ./bashrc file and write these lines:
export FLUME_HOME=/opt/apache-flume-1.6.0-bin
export PATH=$PATH:$FLUME_HOME/bin
export CLASSPATH=$CLASSPATH:$FLUME_HOME/lib/*
•	Run . ~/.baschrc
•	Run ./flume-ng
After successfully installed Help prompt of flume will get

Moving Data into Flume:

•	Go to apache-flume-1.6.0-bin/conf folder
•	Create one file datasource.conf and write following lines:

agent1.channels = c1
agent1.sources = r1
agent1.sinks = k1

#source configuration
agent1.sources.r1.type = spooldir
agent1.sources.r1.channels = c1
agent1.sources.r1.spoolDir = /home/ubuntu/flume_source/
agent1.sources.r1.fileHeader = true

 #channel configuration
agent1.channels.c1.type = memory

 #sink configuration
agent1.sinks.k1.channel = c1
agent1.sinks.k1.type = hdfs
agent1.sinks.k1.hdfs.path = hdfs://localhost:9000/flume_sink
agent1.sinks.k1.hdfs.serializer = Text
agent1.sinks.k1.hdfs.writeFormat = Text
agent1.sinks.k1.hdfs.fileType = DataStream
agent1.sinks.k1.hdfs.serializer.appendNewline=true
agent1.sinks.k1.hdfs.rollSize = 6700000000 #This parameter decides number of files going to HDFS system.This is in MB.
agent1.sinks.k1.hdfs.rollCount = 0
agent1.sinks.k1.hdfs.rollInterval = 30
agent1.sinks.k1.hdfs.batchSize = 100


•	Create folder /home/ubuntu/flume_source/ on your local and put the log files there.
•	Now run the following command:

flume-ng agent  --conf ./conf/ -f /opt/apache-flume-1.6.0-bin/conf/datasource.conf Dflume.root.logger=DEBUG,console -n agent1

Once it’s done check following:
•	hadoop fs -cat /flume_sink/FlumeData.1453279735239 |wc –l