#logstash

Hi!

It's possible to run Logstash on Tomcat, JBoss or Glassfish?

(ver 1.3.2)

mikess: we never tried that.

Seems that my problem was that I used the dns filter (for some unknown reason, since i use the hostname set in the syslog messages).

morning logstash-folk

morning poolski

how's your gubbins working?

ok... I got the RabbitMQ layer in this weekend

2 node cluster with mirroring... doing about 3K msg/s

really liking kopf as well

kopf?

elasticsearch-kopf

^^^

I've already reached the point I'm snapshotting off to S3 and clearing out the oldest indices because of space

ooh nice

yeah... I'm hitting that point and wondering how best to go about archiving old indexes

q

I suspect if I can get them to give me more servers not only will it make it more robust but we'll be able to retain more

well, quite

I've been tweaking the knobs on how many threads/workers/etc to have running to keep up

we're not running LS forwarders locally. just have the logs being sent over the wire... so far only 3 nodes receiving the logs and feeding RMQ... 2 nodes processing RMQ and filtering before indexing to ES

how do you load balance across your receivers?

just round-robin TCP LB?

{noformat}

null

{noformat}

ack... wrong buffer ;)

<http://tinyurl.com/qyn6vlg>; (at www.dropbox.com)

why rmq over redis?

poolski: the inbound LS nodes that receive are behind a F5 LB VIP ... the RMQ cluster is also behind an F5 LB VIP as is ES cluster

lol you got too much money :P

because we already had RMQ within our environment but not redis

ah ok

last I checked, F5s were expensive sumbitches

we also had puppet ready to deploy it

ain't my money :) but I have been given plenty of resources

was it much of a pain to configure RMQ?

lol, yeah, sound slike

poolski: with puppet it was stupid simple

I need to puppetise my LS nodes

:(

only piece of the config I had to do was enable the HA policy

aside from the nodes sending their logs which was setup manually... the entire LS stack (LS+RMQ+ES) was configured via puppet

oh and Kibana as well

only 45 more minutes and I can head off to bed

which LS puppet module did you use?

poolski: the one from the the elasticsearch puppet-logstash module from the puppet forge... along with the puppet-elasticsearch module

I then just wrote a wrapper module to handled our custom config and call any defines

Cyis, cool - last I checked it it wasn't terribly good...

still haven't upgraded to LS 1.4 yet though

hi all. Should i be worried that logstash is constantly writing 'Registrar received 100 events" to /var/log/messages, when /var/log/messages is one of the files it's supposed to read? I guess it is a bit of a a waste, but not an infinite loop

zeroXten, I drop those messages at the indexer

if [type] == "syslog" and [message] =~ "Registrar received" { drop { } }

or something like that

modify to taste

poolski: what wasn't very good? the puppet modules?

poolski: yeah, good idea

I might be thinking of the logstash-forwarder module

man, that's pretty spammy.. i think i'm just going to /dev/null logstash-forwarder logs, not even let them get to system

*syslog

electrical, latest version of LS won't start: Starting Logstash Daemon:start-stop-daemon: unable to stat (No such file or directory) (already running)

poolski working fine for me

how to do start LS ?

do you

fcking keyboard..

or brain ..

/etc/init.d/logstash start

debian initscipt

is it starting LS using ../bin/logstash ?

it's not creating the pidfile and yet it thinks it's running

Hi all

I can't understand why it's not working one only ONE host

Is there a way to "require" an external .jar file in plugin .rb?? Got always "cannot load such file", no matter where i put it or what rights i gave.

vali, yes

In jruby console everything is working fine

i'm working on logstash-forwarder & trying to get the "message" to be just the actual log message itself, but i can't seem to do it