logstash如何从redis拉出日志并写入es？

input {

redis {

batch_count => 1

host => "sz-a-xxxxxredis01-redis-xen.xxxxx.com"

type => "syslog"

data_type => "list"

key => "logstash:syslog-log"

}

redis {

batch_count => 1

host => "sz-a-xxxxxredis01-redis-xen.xxxxx.com"

type => "log4j-json-web"

data_type => "list"

key => "logstash:xxxxxweb-web-log"

}

redis {

batch_count => 1

host => "sz-a-xxxxxredis01-redis-xen.xxxxx.com"

type => "log4j-json-tomcat"

data_type => "list"

key => "logstash:xxxxxweb-tomcat-log"

}

}

output{

if [type] == "syslog" {

elasticsearch { hosts => ["sz-a-xxxxxes01-es-hw.xxxxx.com"]

index => "dev-system-logs-%{+YYYY.MM.dd}"

document_type => "system-logs" }

}

if [type] == "log4j-json-web" {

elasticsearch { hosts => ["sz-a-xxxxxes01-es-hw.xxxxx.com"]

index => "dev-xxxxxweb-master-logs-%{+YYYY.MM.dd}"

document_type => "%{class}" }

}

if [type] == "log4j-json-tomcat" {

elasticsearch { hosts => ["sz-a-xxxxxes01-es-hw.xxxxx.com"]

index => "dev-xxxxxweb-master-logs-%{+YYYY.MM.dd}"

document_type => "tomcat_logs" }

}

}

以上配置中还涉及到了rsyslog日志的传输，借鉴了另一位大神的配置，他的博客地址：http://dl528888.blog.51cto.com

总结

本文中最后其实少了kibana效果展示，是因为展示图还在完善中，在后面的记述中，会展示一些kibana呈现的效果。

ELKstack-基于java工程tomcat应用日志处理过程-02

原文：http://vekergu.blog.51cto.com/9966832/1787459