#logstash

if you are logging data within the request

like "new user signup"

that sort of thing

whack: http://pastebin.com/Bv3xvtyX

Title: Aug 30 11:36:43 bertrand amavis[23044]: (23044-08-9) Passed CLEAN, LOCAL [IPv6:: - Pastebin.com (at pastebin.com)

savant: ahha, yes

what format for timestamp?

I use the following in php

date('c')

savant: ISO8601

whack: pls ignore.. I clipped out some data.. I'll make a new one

whack: thanks

BaM`: but basically you should be able to use multiline to have messages starting with '...' merge with the previous

that's the problem though - they don't actually start with '...'

there's the standard syslog junk in front of each line

right, do it in two steps

one, use grok to parse out the parts of the message

call the actual content part something like "HURRAYFORCHEESE"

or something appropriate

multiline on that field perhaps? Not sure if that's allowed

that's what I'm wondering :/

but you can take your captured field (that is the actual message, not just syslog headers) and use it as @message in the worst case

so grok, mutate to replace @mesage with the 'real message'

then multiline

hm, that sounds like it will work, except I'd rather leave @message unmolested, for reference

alternately

you could use multiline to say

lines starting with TIMESTAMP HOST PROGRAM: (whatever) \.\.\. get mreged upwards

in multiline, when you give it a pattern, does it remove it from the result?

oh - will it use grok type tags? or do I need some regex for that?

multiline patterns can be grok patterns for easier stuff

pattern => "%{SYSLOGBASE} \([^)]+\) \.\.\."

oh in that case - that sounds easy

grok type tags?

it doesn't remove anything (multiline)

%{SYSLOGBASE}, etc

oh, so I'll have the dots stuffed in there and some syslog stuff in the middle of the recipients?

I might go have a read up and see if amavisd has an option to not be a dick and stop splitting log lines

anyone know of a way to get kibana terms panel to not display the broken up names? I have stuff like "application-akka.actor.default-dispatcher" in a field and it splits it at the dashes and dots

professoruss: I need to figure that out too

doing it manually sounds like a bad idea

totally

it might involve the analyzer ES is using

but I don't understand ES enough yet to say for sure

likewise