环境：两台虚拟机（ 每台都有flume）
第一台slave作为消息的产生者
第二台master作为消息的接收者 IP（192.168.83.133）
原理：通过监听slave中文件的变化，获取变化信息，发送到另一台电脑。
1、slave上配置conf里面的example.conf(标红的注意下)
#Name the components on this agent a1.sources = r1 a1.sinks = k1 a1.channels = c1 # Describe/configure the source a1.sources.r1.type = exec a1.sources.r1.command = tail -F /home/qq/pp/data/test.log a1.sources.r1.channels = c1 # Describe the sink ##sink端的avro是一个数据发送者 a1.sinks = k1 ##type设置成avro来设置发消息 a1.sinks.k1.type = avro a1.sinks.k1.channel = c1 ##下沉到hadoop2这台机器 a1.sinks.k1.hostname = 192.168.83.133 ##下沉到mini2中的44444 a1.sinks.k1.port = 44444 a1.sinks.k1.batch-size = 2 # Use a channel which buffers events in memory a1.channels.c1.type = memory a1.channels.c1.capacity = 1000 a1.channels.c1.transactionCapacity = 100 # Bind the source and sink to the channel a1.sources.r1.channels = c1 a1.sinks.k1.channel = c1
2、master上配置conf里面的example.conf(标红的注意下)
# Name the components on this agent a1.sources = r1 a1.sinks = k1 a1.channels = c1 # Describe/configure the source ##source中的avro组件是一个接收者服务 a1.sources.r1.type = avro a1.sources.r1.channels = c1 a1.sources.r1.bind = 0.0.0.0 a1.sources.r1.port = 44444 # Describe the sink a1.sinks.k1.type = logger # Use a channel which buffers events in memory a1.channels.c1.type = memory a1.channels.c1.capacity = 1000 a1.channels.c1.transactionCapacity = 100 # Bind the source and sink to the channel a1.sources.r1.channels = c1 a1.sinks.k1.channel = c1
3、向监听文件写入字符串(程序循环写入，不用手动修改test.log文件了)
[root@s1 # cd /home/qq/pp/data [root@s1 home/qq/pp/data# while true > do > echo "fdssdgfgf" >> test.log
> sleep 1
> done
4、查看上面的程序是否执行
#cd /home/qq/pp/data
#tail -f test.log
显示如下即可：
5、打开消息接收者master的flume
进入flume安装目录，执行如下语句
bin/flume-ng agent -c conf -f conf/example.conf -n a1 -Dflume.root.logger=INFO,console
现在回打印出一些信息
6、启动slave的flume
进入flume安装目录，执行如下语句
bin/flume-ng agent -c conf -f conf/example.conf -n a1 -Dflume.root.logger=INFO,console
7、查看5）中的窗口会显示插入的字符串
参考：https://blog.csdn.net/tototuzuoquan/article/details/73203241
原文：https://www.cnblogs.com/51python/p/10961371.html