#logstash

are there any additional requirements than kibana/es/logstash to increment values from logs? i.e bytes, latency etc

In http://logstash.net/docs/1.4.1/tutorials/metric... they are using statsd as output ?

<http://tinyurl.com/p5c3p84>; (at logstash.net)

If using statds as output as per example above, its meant to be shipped from statds to graphite, not kibana?

moapa look at the metrics filter if you want them to stay in kibana / ES (http://logstash.net/docs/1.4.1/filters/metrics)

Title: logstash - open source log management (at logstash.net)

MONTHDAY stopped working as soon as June started for me. so it worked while it was two digits but 1 and 2 couldn't be parsed. MONTHDAY (?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])

that regex is over my head, maybe I should just use \d+?

the input data starts like this <166>Jun 2 09:23:29 and my pattern for that is <\d+>%{MONTH} %{MONTHDAY} %{TIME}

Hello World

my grok rules work fine in grokdebug.herokuapp.com but i get _grokparsefailure tags

should your grok match every single line?

Don_E: Thanks!

stemid: hi, i've set match rules on a keyword ( certified unique ) which add a tag, then a condition to work on the tagged message

stemid: you mean i should set a break_on_match => yes where i add the tag ?

Jarth73: try it. I just know that grokparsefailure tags could just mean that some line did not match, or some part did not match.

while the grok in general still works.

I'm a newbie myself, started using logstash a few weeks ago.

break_on_match should default to true ?

stemid: about the same time as me then

stemid: break_on_match is on by default

grokdebug never fails, not sure if it will display if a grokparsefailure occurs, i figure it blanks then

Don_E: hey, is not not so by default ? docs say yes

Default value is true

also use add_field not add_tag. are you getting the new field you want added?

hmmm, could it be i get a _grokparsefailure tag on ALL messages because the filter fails one time ?

stemid: i do not want to add a field ( which contain values ) i want to add a tag for referral

ok I thought tags were being replaced by fields.

you can do conditionals on fields

by now i'm quite sure this fails continuously because one entry is not matching for the rule

stemid: i'm reasonably sure the conditional part works as expected :)

stemid: but it's a good idea to add a tag when it's processed it as well actually, just for testing

Jarth73 add tags on every grok match to see where it fails

Don_E: that's what i'm doing :)

Don_E: how are you ?

stemid: now i get your remark, it seems indeed the conditional on [tag] == "tagname" is not working

I only do conditionals on fields because I got the impression that tags were being phased out.

stemid: uhm, i've not seen any 'deprecated' notification yet

stemid: if i get what they serve that would not be beneficial either

Hmm. how to deal with log lines containing different amount of spaces & tabs between words?

for example

%{TIMESTAMP} Hello John, How are you?

%{TIMESTAMP} Hello John, How are you?

Im using %{GREEDYDATA} right now, but i get _grokparsefailure

for some rows

moapa: I've been forced to use \s* or \s*

sometimes

ugh, sorry I mean \s?

depends on how many spaces

but GREEDYDATA should cover that

you're probably getting grokparsefailure for other reasons

moapa: same here, apparently my _grokparsefailure are due to pattern to data match failures

!argh! it is a space in the input

stemid: when a data is shown it is not prepended by a zero if it is only a single digit day

stemid: so my pattern should read %{MONTH}\s*%{MONTHDAY} instead of having just the two fields separated by a space

once more it works in grokdebuggger but not in logstash grok

Don_E: i don't get it why it keeps failing after i add the tag, the tag is added when there's a match on a keyword in the message

Don_E: still, the tags are correctly added but always get a _grokparsefailure

Don_E: i've set remove_tag => [ "_grokparsefailure" ]

but this seems to just delay adding the _grok... fail tag