#logstash

humm

i gtg for now. thanks for the input, although hardly anything of that was new. if anyone knows who to contact regarding this (we are willing to pay for this), please drop me a msg.

<-

I'll take it from here

kleind: you could always take Support from ES ;-) hehe

they told me to buy more hardware.......

and never really listened to the problem description

kleind: really? hmm

also they don't have any hands-on support on short notice

we don't do hands on support no ( Disclaimer; i work at Elastic )

I've contacted elastic for some local support/consulting (germany) and they sent me some info on an official company who does official elastic support and consulting

I'll be visiting the ELK workshop next week in amsterdam. Will have to do for now, probably

SkaveRat: ah okay :-)

Yeah, we use partners for the consulting stuff indeed.

they'd do some in-house consulting, but not before may, which is a little late for us

Ah i see. okay

The workshop might give some insight indeed.

i do wish there was better documentation for performance tuning the stack

Issue is that it very depends on the use case... for example website search is completely different pattern ( mainly searching ) then for logging ( mainly indexing )

our LS nodes do about 2k doc/s which is better than they were and cpu is about 50% used, but i would love to squeeze out more

I hope at some point we can work on that kind of documentation.

the biggest problem we currently see is, that LS can't remotely handle the data we throw at it. we now log about 2k logs/s, which a single machine can easily handle, but we plan on pushing about 50k+ logs in the future, which is really annoying if we need to set up a giant cluster just for parsing the logs

for use we are alomst 50%/50%

we have a cluster now, and planning on about 60k/s

with peaks much more than that

SkaveRat: if i read back correctly you are using Redis as in input right? is that queue filled up or empty at most times?

like 3x

electrical, while we benchmark, we reset the (in-memory) queue to 8million logs each time, to get reproducable results. in production it's pretty much empty, just a shorttime buffer

SkaveRat: okay. that means that LS is processing as fast as its receiving if its near 0 items in the queue.

did you see any times the queue size increased

?

well, in production we push only about 2k logs into it, while we can handle a lot more than that. that's the reason it's empty

hm, so the irc input seems to only connect to the last channel in the channels list

cittatva: really? that's a horrible bug.

it's possibly that I'm doing it wrong, I guess? I'll post my config...

cittatva: okay

Title: only last channel is joined, logstash irc plugin - Pastebin.com (at pastebin.com)

cittatva: config looks good yeah. weird stuff

can you create a bug report on the github repo?

will do - i'll poke at it a little first and see if I can figure out what's going on

okay thanks!

I'm trying to use gsub to remove the middle part of a log line which has been joined together using the multiline filter (I want to remove the end of the first log line and the start of the second log line) but it doesn't seem to work how I would expect it to.... as far as I can see the regex should work, but it doesn't replace anything when a line goes through. I have put the config along with a message after

it has gone through the filter: http://pastebin.com/Eh6ntJQW

Title: gsub doesn't remove the string I expect it should - Pastebin.com (at pastebin.com)

can anyone advise whether the order in that config wouldn't run through as I wrote it?

Rumbles: what if you only do \.\.\. in the gsub? instead all the extra's you have in there? that works in my ruby gsub try

so replace \.\.\..*\.\.\. with just \.\.\. ?

electrical, is there any way of debugging the LS pipeline? To see in which step it takes the longest and optimize it

Rumbles: yeah

that would just replace the ...'s and not the bit of text in the middle though wouldn't it ?

SkaveRat: i'm afraid not. we have no internal metrics at the moment :-(

Rumbles: correct

1 min

electrical using just \.\.\. the ...'s are removed...

Rumbles: hehe ;-)

I used https://regex101.com/ to test my regex should match the input.... :/

Title: Online regex tester and debugger: JavaScript, Python, PHP, and PCRE (at regex101.com)

when I copy it there doesn't appear to be a new line in there....

and there shouldn't be as it's gone through multiline...

ARGH

there is a new line

right, so after the 2 lines go through multiline filter there is a newline after the first set of ...'s :/

lol :-(

strip all newlines :p

I thought it should be after it had gone through multiline....

but tbh if this works I don't care

^_^

hehe

yeah that works.... thanks electrical....

Rumbles: np :-)

once again... fix one issue... another one pops it's head up :)

anyone have any tips on finding out why json failed to parse? every other message that goes through multiline fails on the json parse :/

Rumbles: json filter i assume?

yeah

can you gist/pastebin config + log + example line? can do some local testing

1 mo

k

here you go electrical http://pastebin.com/geCxkdxb

Title: untangle NGFW filter with json parse failure and sucess egs - Pastebin.com (at pastebin.com)

I've put in an entry that fails to pass the json parser straight after the filter, after that I have also put an example of one which passed the json parse filter successfully

Good Morning