#logstash

rrix: can you update your config gist?

yeah, sec

torrancew: https://gist.github.com/rrix/f79b689e73da4d01ca3d I added another file tot his gist

Title: gist:f79b689e73da4d01ca3d (at gist.github.com)

just fiddling around with it

Yeah, looks like my grok filter might've been bad

grokdebug thinks so at least

I don't use the "singles" option, but I don't know of any reason a filter that doesn't match would cause you to not get output data at all

it should come through, just without the fields parsed out

I found this filter randomly on the internet, so I'm not married to it

but you'er right

also, tiny optimization suggestion: rather than dropping "ts" in a discrete mutate, use "remove_field" in the date filter itself

that way it only removes it if it successfully parses it

which means that if it doesn't, you still have the field around to figure out what's wrong

oh, here's one:

port 9300 on a default ES install is NOT an HTTP port

9200 is

if you're running in the "nodes" or "transport" protocol IIRC, 9300 is what you want - but for http, you want 9200 AFAIK

rrix: ^^^^

No dice there either

torrancew: My stdout output doesn't even appaer to be hit

interesting. What LS version, rrix?

also, are you sure something is sending live graphite data to LS at the moment/

1.4.1

and I'm sending data using netcat and telnet right now

whenever I jiggle something

random metrics that one of my sensu metrics dumps out

kk

I just appended a sample to that gist

kk

rrix: you remembering to put the \n at the end?

Yup

ah, I see you are

hmmm, I'll try to repro in a moment if I can finagle the time

alright

I'm about at the end of my work day, I may shelve this and think on it a bit

US PDT?

Yup

same here

nice nice

rrix: I've got a bit of a nuts week ($dayjob has a big engineering deadline friday)

ouch

but *knock on wood* I'm ahead of schedule so far, so I should be able to help you dig into this a bit

heh, I'm in the same state

rest of the org is scrambling towards a release and i'm out in front of it on a research project

:)

I'm not completely in front of it, but most of the items left after today could be done by lunch tomorrow

we had an ELK cluster before launch, but I deprecated it since no one was using the dashboards... bringing it back now and better than ever

woot!

thanks for the poking and prodding torrancew, I'll be idling around here again asking silly questions :)

rrix: that's what we're here for! :)

redis vs rabbitMQ ? is there a clear answer???

anyone?

didn't know someone wrote a nodejs implementation of logstash: https://github.com/bpaquet/node-logstash

Title: bpaquet/node-logstash · GitHub (at github.com)

also, with python-logstash-formatter, do I need to do extra special things on the logstash side? Sometimes I get something like "org.elasticsearch.index.mapper.MapperParsingException: failed to parse [@fields.args]"