#logstash

hi everyone, anybody want to help a fresh-to-grok newb?

You just preempted my own grok commentary.

I'll hold my comment. What's up?

what i'm trying to do should be quite simple, i've got a named dns server log which offers external and internal lookups. I'd like to separate internal hosts (FQDN ends in internal.net) from all others

Many ways to do it using grok. One is to use grok to grab just the FQDN and then use a couple if statements to switch which one you're working on.

Like I have a pretty standard grok match for syslogs that grabs the severity as a field. match => { "message" => "<%{POSINT:syslog_pri}>(%{SYSLOGTIMESTAMP:syslog_timestamp} )?(%{SYSLOGHOST:syslog_hostname} )?%{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }

Then I have some if checks that add a field for different severities.

if([syslog_severity] == "1") { mutate { add_field => [ "severity", "100" ] } }

In that if I could put in more grok statements, geo lookups, mutates... anything.

that looks really useful; how do i match whether that query ends in internal.net ? (two line example log here http://pastebin.ca/3708085 )

Title: pastebin - Unnamed - post number 3708085 (at pastebin.ca)

Like for iptables firewall logs I have it do a cidr filter looking and add a tag for inbound and outbound connections.

(that query = 'that query' in the dns log)

=> "client %{IP:src_ip}#%{NUMBER:src_port}: query: %{DATA:query_string} IN %{WORD:query_type}"

Something like that probably?

Something that might help: http://grokconstructor.appspot.com/do/match

yup i'm using that at the mo

Then you can use a grok match on the query_string value.