#logstash

awesomeness. Thanks guys.

np

ok last couple of issues. :)

I'm passing a field via lumberjack

--field app=foobar

actually restarting logstash to test that field again but the other thing I am seeing is some of my rails logs being cut off in the middle.

multiline seems to be working 95% of the time but every once in a while I get a log that isn't parsed properly by multiline

and subsequently turns into a grokparse error because it's tyring to grok half a log entry

To answer my own question there it looks like our rails logs are not always ordered properly with multiple process in rails 3.2. wtf.

And here is a helpful SO thread on making rails 3.2 logs sane again: http://stackoverflow.com/questions/11561846/rai...

<http://goo.gl/dTfdJ>; (at stackoverflow.com)

evening gents <3

Scub: hi.

pleased to meet you, mortini :)

um, hi.

whack: do you have some examples are large scale deployments?

of

I recall Mozilla had a pretty big install, pretty significant events/sec

once I finish deploying here I'll be doing >100000 events/sec

for mozilla I'm not sure what their event rates were estimated at

but I'm pretty sure their cluster could do at least 500k events/sec

estimated with math, that is, not actually observed

phrawzty is heading that up

hi all, does anyone know the state of logstash+rabbitmq/es/river? Namely, is it considered production-ready and the preferred method for doing logstash <> ES flows?

thanks, I just found your scalex deck

I'll quote that

plinde_: it should work, but I don't know who uses it.

it's been around for years

whack, running into few issues using it where upwards of 75% of our logs are not making it from RabbitMQ into ES

just wanted to see if its advised to go that route or if elasticsearch output is 'advised'

i realize there are a lot of variables in the equation (ES version, RMQ version, ES River plugin version, etc).

plinde_: I try to know as little as possible about rabbitmq, so I may not be able to help

are you certain it's makign it into rabbitmq?

and can you pastebin your config?

yes, we tried using amqp/rabbitmq output and all the logs end up in the queue (counts match). starting a river from ES to that queue ends up with all the messages not being ack'd by rabbitmq.

no idea :\

no worries, we are fairly pleased with performance of using elasticsearch output

we've been pondering if river might be a preferred alternative

given the issues, we'll probably stick with what we've got

preference doesn't matter.

does it work for you, does it support your requirements, etc.

agreed there, maybe preference is the wrong word. recommended

do what works, if you find performance problems or want to experiment, then try something else :)

but if it ain't broken, don't fix it ;)

whack, no argument there, appreciate the input

np :)

in general, elasticsearch and elasticsearch_http have the best performance

rabbitmq performs probably the worst of all the transports logstash supports due to the library we use

so I'd expect the river output to perform worse than the two alternatives (es or es_http) right now

woo 30% perf improvement in date filter

cha ching

whack: was going to ask you yesterday, what are your thoughts using nodes for logstash configs, instead of defining types, similar to puppet

not sure what you mean?

you mean using specific servers to process certain kinds of logs?

i mean instead of defining type for every input, filter, output (when necessary)

encapsualting them in a node (which would be equivalent to a type)

can you explain further? not grasping it yet ;)

new event v1 perf improvements are looking pretty good

2x faster throughput doing generator->null output

so you have node 'logstash:type' { inputs {} filters {} outputs {}}

ahh

you mean grouping things more visually

yea

:)