#logstash

is it possible from the udp output module to send only a specific field?

sds__: Perhaps if you clone the event and cut it down

volter: interesting, didn't know about the clone module

can I use wildcards in field matching?

if [fields][type] == "syslog-*"

Looks like maybe this will work: if [fields][type] =~ /^syslog/

gimmic_: I think there's no globbing, now. Regex certainly works.

sigh, i've never got filebeats to come through correctly, datestamp is when it was imported, whole thing is just 'message'... just web server logs, would think it would be easy. Not sure to try and actually make it work or upgrade and start over, think I'm a version behind

Which has pretty much been my experience with ELK every time... and I started with LSF and other older things...

hi great people! Im learning how to read the slow-query csv file in my ELK stack, what do you recommend to make it right?

tgodar: how would filebeat know what time your log entry is from, vs using the ingest time?

mariadb slow query file sorry

rastro, I thought at some point I'd told it to treat the thing as an apache log file

I'll try setting up the filter again see how it goes. I'm on 2.3.4 now?

probably should look at upgrading, 5th time is the charm

no lack of opportuntiies to upgrade, for sure...

well I'd have abandoned if it was not improved each time also...

but yeah, I'm in a pissy mood today ;)

volter: know if regex is going to be significantly more computationally expensive?

I'm moving from a direct match ( == ) to a regex match =~ /^syslog/

so it'll check against every event

I wonder what the performance hit will be..

doing the same thing on my output now

never use regex for what you can accomplish with a string function is usually a good rule of thumb

gimmic_: but stuff like anchoring your regexp will help, and being concise...

another option would be to have a bunch more redis inputs pulling from multiple keys

gimmic_: that solves your regexp problem? must have missed a redis upgrade :)

hmm?

I mean i could split my log types out earlier using filebeat sending to multiple redis keys(queues)

then on my LS input, have a bunch of different redis inputs

gimmic_: while tgodar is 1000% correct about string vs regexp, you'd probably need a bunch of regexps to really require a rearchitecture like that.

alright. Just more worried about events-per-second rate

if it is parsing 50k EPS, how much of an impact moving from string-to-regex I'll see

but the deployment is under heavy development so I don't really have a baseline anyway

gimmic_: Well, you can benchmark that on a purpose-created instance, I guess.

Yeah, I'll add it to the list of to-dos which have a probable timeframe of never

If you are doing _any_ parsing later on, the cost is probably insignificant here.

Well, I guess that could be set up in less than 15 minutes.

But sure, time is time.

mostly dissects

I have one single regex to kill a specific message, but everything else for this dataset is a dissect

now, the rest of the logstash instance is still running some heavier regex on other logtypes