#logstash

cgroup: http://build.logstash.net/view/logstash/job/log... <-- that may not be the exact link you need but should ptu you on the right path

<http://tinyurl.com/d9ll9br>; (at build.logstash.net)

this may be better: http://build.logstash.net/view/logstash/job/log...

<http://tinyurl.com/kygy68g>; (at build.logstash.net)

thanks for the pointer, I'll go search for the stable build in there

rashidkpc: there?

rashidkpc: i was just thinking maybe adding something like the inquisitor plugin analyzer/tokenizer tester to kibana would alleviate some of the bugreports about terms panels getting split on '-' etc.

rashidkpc: https://github.com/polyfractal/elasticsearch-in...

<http://tinyurl.com/bwgxqsm>; (at github.com)

Even though the Lucene search syntax is fairly straightforward, I can't seem to get accurate results returned from my search query

or at least some way of showing to end users how each field is split by the analyzer, maybe it could even be something as easy as mouse over on the field picker or something

If I search for "foo bar", I will get "foo bar" results, but then I'll get results that have "foo" and some that have "bar" and some that have neither!

It's frustrating because I'm trying to demo this to a dev that needs something better than tail + grep

tail + grep + cut/awk usually does it for me :)

It's too early to troll right now lol

surge_: i guess you need to specify what fields you want those keywords to appear in?

and maybe use filters instead of just the search bar

I do, but it's just flat out not working properly. I havent set up logstash to filter out this particular message he's looking for

so unfortunately i have to use the @message field

but can you not add 2 filters and search for * still?

field "must" - field:message - query"foo"

and another for query"bar"

Yeah, I guess I just thought

if i typed @message:"123 456" that I would get ONLY results with a message field that has EXACTLY that

and not like

123

and then 456

and then some with nothing

-___-

yeah man i dunno

i've more than once told my devs to use the proper @fields.username:"whatever"

I'll try stacking the search queries like you said

instead of just searching for whatever and then they complain why they get all this other nonsense in the results

Right

I mean I would use that subfield if I could but I can't in this case =/

like so

What version is that?

Is that still kibana?

I guess I should upgrade. I'm using "Kibana 3 milestone pre-3"

I don't suppose anyone has an "add_field" line they use for SonicWall logs? :)

whack: Hey, was curious if you can give me a quick run down of how to use the "add_field" for SonicWall Logs.

surge_: i'm running head from 2 weeks back or so

Yeah I just looked at the ES blog and saw all the new features. Gonna upgrade now

one question: is lumberjack still considered "experimental"? I'm looking for a lightweight shipper supporting authn/authz for ~600 nodes.

tenaglia, i was going to ask the same thing. Looking over doing centralized logstash and logstash itself has such a big footprint as a shipper

don't know what the next best solution is

i'm not using it in production, but I've heard that people are

I use beaver

cool the SSH tunnel option

Maior: does it "resume" consuming a log file when restarted?

I used he python script on github that provides for data retention. I removed all but 90 days worth of data .. after which the log stash indexer was barfing unable to index and elastic search started to stack trace. Are you aware of any sort of problem with that script that might cause the database to corrupt? Ive since moved that entire set of data off to the side .. and restarted everything and Im not stack tracing and log

stash is indexing properly. Im curious if theres any good way to load that data back in slowly?

tenaglia: yes

but I've lost 90 days worth of logs in the meantime

Maior: this is something lumberjack doesn't do actually

(at least it's in the "future functional features")

for how many boxes are you using it?

Current lumberjack fpm build seems to miss the libs, which causes it to fail