#logstash

can mutate filter be used for trimming fields with whitespaces

leading/trailing whitespace that is

does anyone know how to set up an ssl cert on windows? no matter what it keeps dropping the connection but doesnt say why

this is using the lumberjack output plugin

would logstash 1.5 use a different ssh protocol than 1.4.2?

a different ssh protocol?

echelon: you could use the gsub setting of the mutate filter to strip whitespace

ralphm: cool, thanks

echelon: something like: gsub => ["field", "^\s+", "", "field", "\s+$",""]

ah

echelon: or maybe even ["field", "^\s+(.*)\s+$","\1"]

(untested)

waltertv: how are you observing that it drops the connection?

whack: i keep getting "OpenSSL::SSL::SSLError: An existing connection was forcibly closed by the remote host"

i'm just setting up another server to run in debug mode to see if i get a better message on the other end

waltertv: can you link to your logstash configs?

whack: https://gist.github.com/cameronevans/7a8933b2cb... and https://gist.github.com/cameronevans/dec7069159... thanks

<http://tinyurl.com/neepsmx>; (at gist.github.com)

waltertv: can you link to your other logstash config also? The one that has a lumberjack input

whack: yup right here https://gist.github.com/cameronevans/dec7069159...

<http://tinyurl.com/o8t6agy>; (at gist.github.com)

waltertv: what version of logstash on both?

whack: 1.5 on the windows server and 1.4.2 on ours

waltertv: that error you get "connection was forcibly" indicates that the lumberjack input closed the connection (or logstash died, or a firewall destroyed the connection)

waltertv: can you try 1.5 on both?

I'm not sure what is causing your error at this time though

without seeing what the input side is doing to cause the closure

hello all - I am trying to parse a static log file that has date in the format of [05/May/2015:10:01:32 -0500] - I wrote a custom pattern that parses it into "datetime" - but when I try to match it into be the date with match => ["datetime", "dd/MMM/YYY:HH:mm:ss Z"] i get an error

whack: hmm okay. with it pointing at one of our servers in debug mode i can see the logs getting through but not the fields added by mutate... that would cause the logs to get stuck in the mulitline filter i believe

still don't know what that connection error is about

i'll try 1.5 on both now

thanks for your help, really appreciate it

does anyone know what would be the right syntax for that date format would be? (the error is The error reported is:

Illegal pattern component: t"

deruke: that should work, can you link to your full config?

whack: yes, i can do that - one min

whack: think it might just be this issue... i hope that there is a workaround https://github.com/logstash-plugins/logstash-ou...

<http://tinyurl.com/ph6bs6y>; (at github.com)

whack: you can see that fields are being stripped here if you like https://gist.github.com/cameronevans/e414948803...

<http://tinyurl.com/oaqjvxg>; (at gist.github.com)

whack: http://pastebin.com/sVWY9Jfu

Title: logstash_snipet_01 - Pastebin.com (at pastebin.com)

oh, right

waltertv: set codec => json on both your lumberjack output and input

whack: sweet thanks, i'll try that now

deruke: the error you get is confusing, sorry

deruke: your two lines setting match will combine to mean: match => ["datetime", "MMM dd HH:mm:ss", "datetime", "dd/MMM/YYY:HH:mm:ss Z"]

deruke: you'll want to just have one 'match' line in your date config

awesome - thanks!

match => [ "datetime", "MMM dd HH:mm:ss", "dd/MMM/YYY:HH:mm:ss Z" ]

^^ will work

does anyone have any experience with windows servers? i fixed this error last week on another server but forget how i did it... running logstash from the command line throws "windows can't open this file: environment.rb"

might have had something to do with JAVA_HOME?

deploying on windows has been a nightmare haha

folks - I'm logging (and filtering!) from logstash into an SNS topic

but I'm wondering - any way to *not* send a message

instead, to force a topic to have specific message contents

my issue is that the SNS topic details that logstash emits are a json message that is backslash escaped

looking at the logstash code for the SNS outputter - it seems obvious that I *could* format it somehow

but nothing that I understand explains message formatting :)