#logstash

Hey everyone. What's the best way to send windows event logs to a logstash server?

I've seen some about nxlog

what are some other options?

nxlog is probably the best

Ok thats cool. It seems like a strong choice

nxlog is probably the best bet - just watch out, IIRC they still enable SSLv3 (not sure you can disable it, either :/)

oh, looks like they fixed it

Oh nice

Im gonna have to learn it a bit

What is the proper way to GROK this value? It's the URI Query string from an IIS log: d=p6ed21QCM_0B4-6FKW-yJKL_khaQJL-sfsmf&t=6355844

what do you want to break out of it

Nothing, I just want to capture the entire value into a key

It's contained within a tab-delimited line

maybe greedydata?

have you tried using the grok debgger?

Title: Grok Debugger (at grokdebug.herokuapp.com)

I have yeah, greedydata looks like it grabs the remainder of the line though, this value is in the middle

define the other fields, then you can just take that one

whats wrong with this syntax? if [type] == "ocsplog" or [type] == "ejbcalog" {

looks ok, what are you seeing

{:timestamp=>"2015-04-28T21:43:42.228000-0400", :message=>"Error: Expected one of #, } at line 86, column 47 (byte 2931) after filter {\n if [type] == \"rsyslog\" {\n\tgrok { match => [ \"message\", \"%RSYSLOGBASE\" ]"}

Title: Paste #XUL - Apache Paste Bucket (at apaste.info)

having a hard time finding the syntax error

warkolm: Thanks, looks like that works. Does greedydata just take the remaining un-matched contents of the line?

I have a syslog listener, it seems LS is able to index the data without me creating a pattern, I want to use a custom pattern which i have created, but i think the default one is getting in the way. How do i tell LS to not run syslog events through %{SYSLOGBASE} before hitting my filter

Socket-: you can override the %{SYSLOGBASE} pattern by providing your own with the same name

...or any pattern for that matter

when i do that i get a grok parse error

possible issue with your pattern then?

i'll share

almost done with paste

Title: #26811 CentOS Pastebin (at pastebin.centos.org)

BaM`: see anything?

pls hold...

a couple of things stand out

you need %{RSYSLOGBASE} not %RSYSLOGBASE in your grok match

ohh, thats huge

thanks, i didnt see that

also your custom pattern is never going to match just the "message" field - it looks like it's trying to match an entire syslog line

e.g., it's starting with %{SYSLOGTIMESTAMP}

but you're trying to match "pam_unix(su:session): session opened for user root by dan(uid=0)"

i think i need to match the entire line

some of my syslog events look like this

timestamp and program etc have already been matched somewhere earlier

<77>Apr 28 16:22:44 nix run-parts(/etc/cron.daily)[7980] finished prelink

ah yep

how do i match the line instead?

well that matches for the sample you just gave me

or what do you think is the best way to handle these and normal syslog data

you can either give them different types and let them hit different conditions

yeah, it matches and seems to index everything right, but it still has grok parse tag on it

or you can just have multiple match patterns inside one grok

if you don't mind the extra overhead of some failing

Im not sure i follow you, can you break that down differently?

if it reaches the end of a grok without mathing anything, it will get a _grokparsefail

ok

do you have different syslog stuff coming from different sources?

they all come from rsyslog remote forwarder

ah I see

do you know which hosts are sending which syslog formats?

actually, I have a better plan

pls hold

all using: RSYSLOG_TraditionalFileFormat

are some in JSON format already though?

your custom message snippet suggests that

im not sure what causes that

i think it's auto done with the input maybe?

nah - there are things like severity and facility in there

so it's either already tagged up when it reaches logstash, or it's hitting another filter somewhere

so, here is a raw syslog event: Apr 27 19:03:34 nix yum[54088]: Updated: dracut-004-356.el6_6.1.noarch