#logstash

the 'elasticsearch_http' output doesn't depend on a specific elasticsearch version

When might support for 0.90 be added? (we run a few ES clusters, once for LS and it would be nice to have the same)

whack: afair bodik did some extensive stress testing and the two outputs had similar performance

indeed.. _http was a bit slower ( not that much )

also i did not know how to make the embeeded one to use the aws zone awareness plugin which was a must for me , so i just gave up on it

Kubes__: 0.90 isn't released yet, iirc. If you need 0.90, use elasticsearch_http

whack: 0.90 has gone stable

piavlo: I observed only small differences, elasticsearch_http was 15% slower with a single node. I haven't tested multiple clients on multinode-cluster

ahh cool

whack: only 2 days ago :p lol

whack: maybe do 1.1.11 with ES 0.90 then

hmm, yeah

1.1.11 is pretty much code-complete

one last windows bug to fix

then I'm branching 1.2

okay :-)

how far are you with the new event layout?

really need to get codecs in as well as some other important improvements that have been waiting far too long ;)

Electrical: that's the nice thing about the new event layout, most of it is deleting code ;)

no tangible progress yet though

hehe okay. if i can help give me a shout.

whack: was that with elasticsearch + 1 ES node? or just elasticsearch in embedded mode?

piavlo: independent ES node

Electrical: once I fork 1.2, I'd love your help purging all the old deprecated stuff

whack: okay. give me a shout when its time.

woot

whack: so 1 node with logstash+elasticsearch + 1 ES node vs 1 node with logstash+elasticsearch_http+ES(no data mode) + 1 ES node

or

so 1 node with logstash+elasticsearch + 1 ES node vs 1 node with logstash+elasticsearch_http+ 1 ES node

always 1 ES node, separate from logstash

tested the same exact scenario varying only "elasticsearch" vs "elasticsearch_http" plugins

https://github.com/jordansissel/experiments/tre... <--

<http://goo.gl/iKxzS>; (at github.com)

yes but on the logstash node did you run ES(no data mode) or directly spoke to remote ES node from logstash?

I don't understand the question

I suspect you are asking me how it was configured, but I don't know *what* configuration you are asking me for m.

https://github.com/jordansissel/experiments/blo... is the elasticsearch output I used, for example.

<http://goo.gl/HVAkW>; (at github.com)

the equivalent elasticsearch_http configuration was used.

whack: Can you think of a reason why 1.1.10 would error on startup with a could not load FFI Provider, but 1.1.9 starts fine?

rmelissari: what os?

Solaris

could be missing solaris support, or the java tmpdir isn't writable?

Wouldn't those show up with 1.1.9?

I don't know ;)

I have been running 1.1.9 for months, just tried going to 1.1.10. I can pastebin the error for you.

yeah please

Whack: http://pastebin.com/X6xkUje5 See if anything jumps out at you.

Title: logstash error - Pastebin.com (at pastebin.com)

If I have my logstash agent > redis > logstash server…. Should I also do eventlog > nxlog json > redis > logstash server or just eventlog > nxlog json > logstash?

Best practice to have everything incoming go through redis before the server?

jonconley: doing everything through redis lets you more easily restart/upgrade/configure logstash

in that redis can buffer a bit while logstash is restarting or whatnot

whack: I was afraid you would use logic against me. ;) thx

jonconley: haha ;)

jonconley: Imean, it's totally up to you

whatever is easiest to operate is what you should do :)

whack: I appreciate your attempt there.

hehe :)

holy shit someone *still* hitting the leap second bug.

I did first time I tried to use logstash :P

jonconley: is nxlog solid for windows -> logstash (or redis first)

jonconley: i've got a bunch of windows boxes i need to get adding myself

when using the statsd output, can i configure in essence multiple output config blocks for different types of logs i want to create metrics for?

jmartin: yes you can.

jmartin: I am just starting to incorporate the windows side. I've been following a couple bloggers and githubbers that have been doing it for a while though and they haven't had any issues.

Electrical: excellent, thx

jonconley: good deal

whack: want me to modify the PR for hipchat?

Electrical: yeah, trigger_notify?

I'll merge it into 1.1.11

so you can get a new puppet module out

sounds like a plan.

lemme know when you have, I'll mergeificate

I think the last bug is the graphite crap?

i think so. not completely followed that discussion

I'll try to reproduce it

New news from newjiraissues: Evan Ravenelle created LOGSTASH-968 - Graphite plugin no longer works in daily dev build logstash-1.1.10.dev-monolithic.jar <https://logstash.jira.com/browse/LOGSTASH-968>;

does anyone know why im getting a 'source_host' namespace in graphite from this config ? https://gist.github.com/hugespoon/5497819 basically its 'stats -> HOSTNAME -> source_host -> event'

Title: Indexer.conf (at gist.github.com)

when i want it to be 'stats -> HOSTNAME -> event'

which it actually seems to be updating

whack: https://github.com/logstash/logstash/pull/467

Title: rename notify to trigger_notify by electrical · Pull Request #467 · logstash/logstash · GitHub (at github.com)