#logstash

hi all, just wanted to put on the record that I fixed those problems.

turns out elasticsearch was running out of memory and it wasn't picking up my ES_HEAP_SIZE var

exported it from the right place and everything is working great now... :)

whack: do you know if there is any pending fix for https://logstash.jira.com/browse/LOGSTASH-1328 ? I've seem to hit the issue too

Title: [LOGSTASH-1328] Encoding issue with JSON causing elasticsearch to reset the connection - logstash.jira.com (at logstash.jira.com)

Jippi: did you try applying the patch in that issue--did it fix the problem?

sastraxi: I'm using the jar, not possible(?) to patch

there is a guy saying it fixes it though

if you head over to the logstash github there are building instructions there to build the jar from sources

so if it's urgent that might be your best bet

Jippi: out of curiosity, what's generating the invalid JSON?

sastraxi: no idea, even in debug mode ES just spew out the stack trace and nothing more

the json that is logged to file looks good though

i mean, where is it coming from?

passes and all

are you generating the JSON yourself, or is the source a different logging tool, or...

phps json_encode => logstash udp => es

can you pastie an example json that triggers the error, your logstash conf, and the errors from ES/logstash?

sastraxi: i tried to update ES to 0.90.5 - and so far it works

its any json though, got tons of different formats, and almost none gets saved

are you using the elasticsearch or elasticsearch_http output plugin?

Jippi: sup, what are you doing

savant: http version

savant: fightning a bug =)

sastraxi: the upgrade to es 0.90.5 have seem to fix it

maybe the improved chunked transfer too

Jippi: good luck, nice that it's fixed but that'd be make so paranoid

oh heh

*make me

sastraxi: gonna let it run over night and see if it was just lucky

its 3:30am, time to sleep

well, fml, the error is back =)

lol

gonna try a custom jar tomorrow

we're still using 1.1 at work

fun

hey, it problem works better =P

Hi, I have a problem. I am using Logstash 1.1.13, and I have an agent passing a @message field and a message field. I have tried using match => [ "message", "grok pattern" ] but it keeps using the @message field

Is there a way to just get the "message" field?

(newbie!) example conf for 'file' input just to pick up existing file (using start_position => beginning)? Or how to debug my attempt ... not getting any output from file {...} though good from stdin {...}