将Logstash中的时间戳时区转换为输出索引名称
在我的场景中,Logstash接收的syslog行的“时间戳”是UTC,我们在Elasticsearch输出中使用事件“ timestamp”:
output { elasticsearch {
embedded => false
host => localhost
port => 9200
protocol => http
cluster => 'elasticsearch'
index => "syslog-%{+YYYY.MM.dd}"
}
}
我的问题是,在UTC午夜,Logstash会在一天结束之前在时区外(GMT-4 => America /
Montreal)将日志发送到其他索引,并且由于“时间戳记”,因此在20h(8h PM)之后索引没有日志就是UTC。
我们已经完成了围绕时区的转换工作,但是性能明显下降:
filter { mutate {
add_field => {
# Create a new field with string value of the UTC event date
"timestamp_zoned" => "%{@timestamp}"
}
}
date {
# Parse UTC string value and convert it to my timezone into a new field
match => [ "timestamp_zoned", "yyyy-MM-dd HH:mm:ss Z" ]
timezone => "America/Montreal"
locale => "en"
remove_field => [ "timestamp_zoned" ]
target => "timestamp_zoned_obj"
}
ruby {
# Output the zoned date to a new field
code => "event['index_day'] = event['timestamp_zoned_obj'].strftime('%Y.%m.%d')"
remove_field => [ "timestamp_zoned_obj" ]
}
}
output {
elasticsearch {
embedded => false
host => localhost
port => 9200
protocol => http
cluster => 'elasticsearch'
# Use of the string value
index => "syslog-%{index_day}"
}
}
有没有一种方法可以优化此配置?
回答:
这是优化配置,请尝试并测试性能。
您无需使用mutate
和date
插件。ruby
直接使用插件。
input { stdin {
}
}
filter {
ruby {
code => "
event['index_day'] = event['@timestamp'].localtime.strftime('%Y.%m.%d')
"
}
}
output {
stdout { codec => rubydebug }
}
输出示例:
{ "message" => "test",
"@version" => "1",
"@timestamp" => "2015-03-30T05:27:06.310Z",
"host" => "BEN_LIM",
"index_day" => "2015.03.29"
}
以上是 将Logstash中的时间戳时区转换为输出索引名称 的全部内容, 来源链接: utcz.com/qa/427860.html