#logstash

eskp: Check out the index api on the ES site, it'll tell you how to delete things.

nicholasf: There should be a source and dest field for translate

source is field => and destination is destination =>, I think you can set them to the same, but you have to set replace=>true (I think that's right.. check the docs to be sure)

ronnocol: hrm, this is what IM trying to use

Title: gist:9244783 (at gist.github.com)

I cant see destination

add to the translate plugin destination => .. http://logstash.net/docs/1.3.3/filters/translat...

<http://tinyurl.com/lucw5tb>; (at logstash.net)

oh, replace is call override; so also set override => true

thanks mate, trying it

ronnocol: grr, it's still not sticking - https://gist.github.com/nicholasf/9244783

Title: gist:9244783 (at gist.github.com)

nicholasf: try quoting the numbers in the dictionary

mp___: nice idea, trying that

mp___: thanks, that did it!

nicholasf: Here's my translate:

<http://tinyurl.com/nbcm7bm>; (at gist.github.com)

cheers guys :)

ronnocol: got it, thanks

nicholasf: np. had similar issue when I couldn't do status:[400 TO 499] in ES: turned out it was storing it as a string and not an int

didn't realise ES / logstash was typed up until that point

mp___: do you have email working ? that's the next thing I want to crack

naw

how do I make logstash log when it throws away lines? I have a lot of these, feeding into redis.. http://pastie.org/8800093

Title: #8800093 - Pastie (at pastie.org)

but when the other logstash daemon picks them up from redis, they just disappear

and nothing is logged :(

FiXion: could you paste your config?

lieter: http://pastie.org/8800208

Title: #8800208 - Pastie (at pastie.org)

FiXion: is your logstash connected to the ES cluster? because you haven't configured the clustername

lieter: currently it's just one node - and it sends it to that.

I'm going to switch to elasticsearch_http

but the logs simply doesn't show up in ES

and I can see they reach redis

I also had an incident where logstash simply repeatedly crashed, upon receiving a faulty json (I believe it was missing a few fields)

I had to clear the queue

unfortunately I had to fix it quickly - and forgot to save evidence :(

I wonder is that has to do with the codec options, I keep those default wiht a redis input and ES output and don't have any issues

the json pastie is missing @timestamp.. I think that's why its throwing it out. I'd like it to log that though :(

lieter: mine aren't default?

but IMHO logs are like money..(or email :) they should never be thrown away.. at most deferred - or atleast logged as "lost"

FiXion: not sure.. but I don't think so