python3.6hdfs的使用
https://blog.csdn.net/qq_29863961/article/details/80291654
https://pypi.org/ 官网直接搜索hdfs就好
学习flume,下载地址
http://archive.apache.org/dist/flume/1.7.0/
170-bin.tar.gz版本
然后解压配置到hadoop_home目录下
之后在~/.profile文件配置环境变量,bin
export PATH=$PATH:$HADOOP_HOME/sbin
export HIVE_HOME=/home/hadoop/hadoop_home/apache-hive-2.3.4-bin
source ~/.profile
之后在解压后的包下的conf(即配置里找env.sh(没有就找类似的那个cp一个flume-env.sh文件))把javajdk的环境变量路径加进去即
export JAVA_HOME=/home/hadoop/hadoop_home/jdk1.8.0_181
然后输入flume-ng即可运行(在bin下)
下面是cmd里没有Telnet解决,在程序和功能里找,打开就好
把telnet客户端选中即可
在conf/配置目录下,新建文件telnet.conf
内容如下(注释不需要加)
#Name the components on this agent
a1.sources = r1
a1.sinks = k1
a1.channels = c1
#Describe/configure the source
a1.sources.r1.type = netcat
a1.sources.r1.bind = localhost
a1.sources.r1.port = 44444
#Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
#Describe the sink
a1.sinks.k1.type = logger
#Bind the source and sink to the channel
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
开启44444(即flume所在端口)
flume-ng agent --conf conf/ --name a1 --conf-file telnet.conf -Dflume.root.logger==INFO,console
记住,要么是在conf/目录下运行,要么加上所在目录,默认在当时所在目录下,相对路径
然后新打开一个shell窗口,判断44444端口是否打开
netstat -anpt |grep 44444
之后打开44444端口发送内容
telnet localhost 44444
python执行telnet代码连接(python3)
https://www.cnblogs.com/lsdb/p/9258964.html
https://docs.python.org/3/library/telnetlib.html
原文:https://www.cnblogs.com/pursue339/p/10510100.html