#logstash

that what most of us do i think

yeah

i am early enough in my testing that i can just nuke the data and start over, having grok explcitly store them as ints

How can I check which data is in a elastic index and if maps to a custom mapping template?

I am getting error on queries: Failed to query elasticsearch for previous event {:query=>"exim_msg_id:\"1XBWUN-0007Hw-Fz,1XBWUN-0007Hw-Fz\" AND exim_msg_state:\"received\"",

Would this be the correct usage of multiline in filter? http://pastebin.com/EfFBaYBA

Title: [Ruby] # SAML if [type] == "saml" { multiline { pattern => "^\s" - Pastebin.com (at pastebin.com)

mak0_reactor - the add_tag is not needed...LS will auto add this tag when it discovers a mutltiline msg

mrlesmithjr: OK

b#ew

logstash is automatically switching codecs from json to json_line on an input, is there any way to prevent that?

mikeface_: can you give an example? I don't think it should do that, so it would be useful to see the situation where it's occurring. Here's an example of the "json" codec vs the "json_lines" codec. Make sure to clone the repo and then run it

whoops,here's the link: https://gist.github.com/shurane/27c8d7d1da685f9...

<http://tinyurl.com/muxrr5r>; (at gist.github.com)

:message=>"Automatically switching from json to json_lines codec", :plugin=>"tcp", :level=>:info, :file=>"logstash/inputs/base.rb", :line=>"133"}

shoerain: the tcp input only has two lines, port and codec

how can i manually delete logs

thumpba: rm </path/to/logfile>

I'm experiencing some weird behavior. After awhile the logstash server stops reading certain log files sent via logstash-forwarder

If I bounce the logstash process, the logs get picked up again. Anything to look at?

logstash's own logfile?

what is the input methong on your logstash.conf?

No this is just from a handful of other clients

mako_reactor: just a sec, I will double check

mako_reactor: input is lumberjack

Anyone familiar with the elasticsearch puppet module for LogStash? I'm having some trouble understanding how to use it to generate conf files.

pjanzen: where would that path be..ive used curator to delete logs but it hasnt freed up any space

Nevermind, I broke hiera, and that probably explains my problems!

thumpba: From what are you trying to delete log files?

logstash i was running curator but it wasn't deleting the logs i was seeing

thumpba: you want to clear indices? install the kopf plugin really handy (I started with ELK at 10:30 NL time today) :)

thumpba: from here http://logstash.net/docs/1.4.0.beta1/tutorials/...: bin/plugin -install lmenezes/elasticsearch-kopf

<http://tinyurl.com/lz6taca>; (at logstash.net)

mikeface_: hm I see, do you think you can still give a nexample?

Has there been a new drop of the logstash forwarder recently? While testing I found out that is the SSL certs are wrong it can cause the indexer to crash

I was wondering if this is somethign known and if ithas been fixed in a newer version

_Bryan_: not afaik :(

avleen; Hmm...ok...any idea how I can protect against someone brining up a forwarder with invlaid SSL certs and trying to connect and crashing my indexers/

_Bryan_: i do it by controling all my forwarders from chef, so they all get the same cert and config

that's all i got right now, other than fixin the bug itself :)

right...but if somoene else builds on to connect which it shoudl get denied without the right SSL certs...how would you prevent them from crashing you indexer

hehe.

I may have to dig in for that one..and ix the bug

err fixthe bug

oh i see what you mean yeah :)