#logstash

magnumbonum: ah, i've yet to use kibana

Anyway: what is the semantic meaning of the first attribute of the match directive? "message" in my example? match match => {"message", "%{SYSLOGLINE}"}

Jarth: its the best

tried ? https://grokdebug.herokuapp.com/

Title: Grok Debugger (at grokdebug.herokuapp.com)

oh yeah

magnumbonum: i've written what you're trying to to differently, i use (test %{GREEDYMESSAGE:messagefield}|test2 %{GREEDYMESSAGE:messagefield}|....)

that is how i worked out the specific matches. The question is how the "message" attribute is constructed. When logstash eats the SYSLOGLINE statement, it correctly strips of the "header" from the syslogmessage, which is already added to the correct fields. My hope was that I could use several match statements to pick out more data, but still get the entire syslog message in "message" for easier log comprehension by the analyst.

Jarth: hmm lets see if I understand...

you mean GREEDYDATA?

ehr, yes, it's 9am here

;)

the only part which fails is detecting the variable part in the message

magnumbonum: might it not be more preferable to just write your own extended patterns ?

yeah, might be. It's just that regexpes have a tendency to become "write once, understand never" for me ;-)

I was hoping for the multiple matches feature, which seems to be more straight forward for understanding what is configured.

Thanks a bunch Jarth, I will explore some more...

magnumbonum: regex complexity is overrated, just use what you know, or document what you used in something you don't throw away easily

Title: Regular expression visualizer (at regexvisualizer.apphb.com)

magnumbonum: i only use a regex to have patterns replaced, detected, also to have OR conditional filters

I think what I want to do is to *not* have conditional filters - I want AND filters. First pick out the specific fields I am interested in, such as username or IP. Then store the entire message line.

What about using individual grok statements? First eat the entire message, store it properly. Then in an individual grok statement eat the specifc? Will give it a try.

magnumbonum: afraid i'm too much of a n00b at logstash & grok to be of any real assistance

i only use the (..|..|..|) on the part that is not getting indexed

Anyone here exporting/archiving logs for "logtime storage"?

cleaning up is easy with curator imho.. But now I'm looking to export/archive as txt.gzip or such..

Jarth: figured out. I looked at the SYSLOGPAMSESSION which worked and does what I want, modified that and I am good to go!

magnumbonum: great, will you upload the pattern as an extension to github M

?

sure. The patterns need a little work before they are ready for prime time... Any pointers on how to go about it? Fork it and push request?

magnumbonum: clueless really, subscribe and upload for merge ?

magnumbonum: ask on this list what to do ? :)

hi!

i need some help with the imap plugin ... it causes to trash the logstash process

anyone running this plugin?

message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::IMAP host=>\"xxx.bla.net\", user=>\"blabla\", content_type=>\"text/plain\">\n Error: undefined method `encode' for nil:NilClass", :level=>:error}

Hi guys

I have some problems : I want to match some tags in the output section of logstash : I want to send a mail when there is a notification tag in the element. How can I do this ? I tried if "notification" in [tags] , if [tags] == "notification" , but no results... Help me Please

gist/pastebin your config