将Logstash中的时间戳时区转换为输出索引名称

在我的场景中,Logstash接收的syslog行的“时间戳”是UTC,我们在Elasticsearch输出中使用事件“ timestamp”:

output {

elasticsearch {

embedded => false

host => localhost

port => 9200

protocol => http

cluster => 'elasticsearch'

index => "syslog-%{+YYYY.MM.dd}"

}

}

我的问题是,在UTC午夜,Logstash会在一天结束之前在时区外(GMT-4 => America /

Montreal)将日志发送到其他索引,并且由于“时间戳记”,因此在20h(8h PM)之后索引没有日志就是UTC。

我们已经完成了围绕时区的转换工作,但是性能明显下降:

filter {

mutate {

add_field => {

# Create a new field with string value of the UTC event date

"timestamp_zoned" => "%{@timestamp}"

}

}

date {

# Parse UTC string value and convert it to my timezone into a new field

match => [ "timestamp_zoned", "yyyy-MM-dd HH:mm:ss Z" ]

timezone => "America/Montreal"

locale => "en"

remove_field => [ "timestamp_zoned" ]

target => "timestamp_zoned_obj"

}

ruby {

# Output the zoned date to a new field

code => "event['index_day'] = event['timestamp_zoned_obj'].strftime('%Y.%m.%d')"

remove_field => [ "timestamp_zoned_obj" ]

}

}

output {

elasticsearch {

embedded => false

host => localhost

port => 9200

protocol => http

cluster => 'elasticsearch'

# Use of the string value

index => "syslog-%{index_day}"

}

}

有没有一种方法可以优化此配置?

回答:

这是优化配置,请尝试并测试性能。

您无需使用mutatedate插件。ruby直接使用插件。

input {

stdin {

}

}

filter {

ruby {

code => "

event['index_day'] = event['@timestamp'].localtime.strftime('%Y.%m.%d')

"

}

}

output {

stdout { codec => rubydebug }

}

输出示例:

{

"message" => "test",

"@version" => "1",

"@timestamp" => "2015-03-30T05:27:06.310Z",

"host" => "BEN_LIM",

"index_day" => "2015.03.29"

}

以上是 将Logstash中的时间戳时区转换为输出索引名称 的全部内容, 来源链接: utcz.com/qa/427860.html

回到顶部