#logstash

linjan: cant i just create a timestamp_indexed field on the indexers and same for shippers?

rofl____: add_field => [ "received_at", "%{@timestamp}" ]

Before you parse your log entry's timestamp

bjorn_: i want to see the timestamp for when the shipper and the indexer processes the entry

shipper sends to redis

indexer pulls from redis

What's your shipper?

logstash with udp/tcp inputs

not all entries has timestamp in message

My suggestion above equals "now()"

isnt @timestamp the timestamp from the message if any?

Not until you date parse it. Until then, it's the timestamp when Logstash starts working on the log entry, afaik.

If you don't provide any kind of timestamp, Logstash will add @timestamp all by itself.

https://www.elastic.co/guide/en/logstash/curren...: "In the absence of this filter, logstash will choose a timestamp based on the first time it sees the event (at input time) [...]"

to me it would be easier just to add received_at_indexer and received_at_shipper with now()

is that possible?

have to use ruby codec?

Probably.

Xylakant: sorry for bothering you again, where i can get source for windows filebeat and, maybe, build scripts?

filebeat sources are part of the beats repository https://github.com/elastic/beats

bjorn_: https://gist.githubusercontent.com/roflmao/4399...

:)

whats the status for ES _timestamp mapping function?

deprecated?

the versioning for ES is not easy to follow

heh

right

should I see fields created by grok in an 'stdout' output?

jb: yes

thought so, thanks!

hm, I have an instance where the grok is passing, but the fields aren't getting added.

this works on another logstash instance, so I'm not sure what is wrong.

only difference is that the input for this instance is 'beats'

torrancew, it was an IPv6 issue -_-

jb: that's rather unlikely. It's probably an error in your config somewhere.

if I have 10 match => lines in a grok filter, and I jam an "add_tag" after the 6th one, will it just add that tag to that specifiy match? or does that apply to all of the matches in the stanza?

keith4: add_tag is applied if the grok{} succeeds.

so, if *any* pattern matches.

And I don't think you can have 10 match => lines

A single match line can contain multiple fields and for each field multiple patterns, but afaik you're restricted to one match parameter per grok stanza