#logstash

untergeek: my json data was like this { type : "myAppLog" , data : "myData" }

the type in the input was not overriden the one in the data, so the output never pop

i had to use that output elasticsearch_http{ type => "myAppLog"} for output to effectively be indexed by ES

oh ho ho! look what I found: https://github.com/elasticsearch/elasticsearch/...

<http://tinyurl.com/pnp6kfq>; (at github.com)

veeerrrrrryyyyy interesting setting

Time to increase the hell out of that

for some reason s3 input is writing temporary data (downloading logs) to /tmp. it is filling up /tmp by doing this. i dont see a way to set the tmp dir to another location in s3 input. java.io.tmpdir=$LS_HOME/tmp. LS_HOME=/opt/logstash. /opt/logstash/tmp is symlinked to location on bug partition

s/bug/big/

looks like i'm gonna have to increase size of /tmp. lame...

maseda: look at the code for the s3 input?

see what it does?

avleen: havent seen it. i imagine that it downloads logs to /tmp and then reads them into ES.

maseda: it does Dir.mktmpdir in ruby. So whatever ruby uses for tmpdir

it might look for $TMP_DIR or something

avleen: word thx

but yeah, it does download it in order to process it.

hope that helps finding what you need :)

avleen: totally, man. you rock ;-)

np :)

avleen, how many cores are you using on your ES nodes?

anyone can help with template ?

in the elasticsearch_http , in set template ield with a path to a json but it doesn't seem to be loaded

*field

bummer. gilhoum left before I could answer him. He was using a deprecated type and should have been using conditionals.

xBabyJesus: apparently not enough. I'm pretty sure i've managed to break everything.

My logstash is set to use 10 ES workers now. And it still spawns over 1000 ES output threads

and overwhelms the cluster

maybe whack has some ideas

Hey guys I just installed 1.3.3, trying to start logstash-web and I'm getting "logstash not configured to start, please edit /etc/default/logstash to enable". Perusing the chan logs from awhile ago, I set START=true in that file, but same error. Anything else I should try?

Ah, nevermind, I got it. Have to edit /etc/default/logstash-web (might be good to have the error message be that instead ;))

akat: open a bug report? :) logstash.jira.com

i shall

it shall be done

Hi guys. I've followed the normal centralised setup. I've configured the indexer to push to elasticsearch and to statsd. Now I need to drop some messages, but only for the statsd output. I can't see how to acheive that.

New news from newjiraissues: Allyson Katch created LOGSTASH-1846 - logstash-web requests edit of /etc/default/logstash on first start, should be /etc/default/logstash-web <https://logstash.jira.com/browse/LOGSTASH-1846>;

barthoda: you probably want to use a filter so you only send messages to statsd that match say a tag (or do not match a tag)

<http://tinyurl.com/m8cg9jm>; (at logstash.net)

sorry, conditional, not filter

sagarC: thanks, that looks like what I need

2.9.6 aus wheezy

ECHAN

New news from newjiraissues: akat created LOGSTASH-1846 - logstash-web requests edit of /etc/default/logstash on first start, should be /etc/default/logstash-web <https://logstash.jira.com/browse/LOGSTASH-1846>;