#logstash

Hmm, it's hard to tell. There is a lot of data being output here.

I see a discover file.

but at the end it says, "/filewatch/watch.rb"

anyone have any ideas on how to use lumberjack without an ssl cert/key ?

whack: so it looks like it's working, but it cannot find the file.

whack: I'm not sure how to limit the debug to only show me where the sincedb is.

whack: I have this setup and running as the "Standalone" version for right now.

nevermind. i think i found it.

anyone else have any ideas where the sincedb maybe on a windows machine?

Blizzard_: ok so, I think you're down a rathole here.

Blizzard_: I'm guessing you're *not* appending to that file.

beeraydee: lumberjack cannot be used without ssl.

whack: correct, it's a static file.

using lumberjack without certificates or validation defeats the purpose of the S in SSL ;)

whack: I wanted logstash to import it to see what the output looks like in logstach

Blizzard_: yeah agreed, this is weird

Blizzard_: I'm not near a windows workstation at the moment, but this should work

Blizzard_: can you pastebin your debug log output from logstash?

maybe I can figure out what's going on from that

whack: there is a ton of output on the cmd line

Blizzard_: yes, can you pastebin it?

whack: so it's not easy to see where that file is.

Sure.

I'll dig through it ;)

whack: here you are - http://pastebin.com/YHPxUzW7

Title: de][87KOHMyFQA6eFqfhKXZRcQ][inet[/10.26.83.109:9301]]{client=true, data=false}]) - Pastebin.com (at pastebin.com)

whack: keep in mind, windows CMD only has a short buffer.

Blizzard_: you should be able to capture the output though; can't oyu?

for sure you can in powershell

whack: I can try.

whack: I don't remember the right line to add.

ok, progress!

36m_discover_file_glob: C:\logstash\test_log_file\*: glob is: []

that log message means that no files were found matching that path

which could mean a few things

1) logstash doesn't have permission to read it somehow

2) there's a bug

I can try to walk you through a few things to help identify the cause

run 'java -jar logstash.jar irb' from the command line

(replace logstash.jar with whatever your actual jar file name is)

'irb' is a ruby shell and we can try to reproduce your problem from there

Ok

nyou should see a little text prompt like ">> "

or something

brb, cleaning up the kid from lunch

err, not yet, false alarm heh

Can you try this:

p Dir.glob('C:\logstash\test_log_file\*')

Ok, I have it running as you said.

what's the output if you run that above 'p' command?

p Dir.glob('C:\logstash\test_log_file\*')

would someone mind looking at my config file? I'm having trouble importing data when I try to use the date filter: https://gist.github.com/bswinnerton/e11b9c9252e...

It shows

<http://tinyurl.com/n5nllat>; (at gist.github.com)

[}

=> []

irb(main):002:0>

ou can paste the output here for this stuff; just to make it simpler ;)

whenever I try to use that line, it doesn't import any data, no _grokparse error or anything, just no importing of data

Blizzard_: ok that's sorta what I expected. Next step:

Blizzard_: p Dir.new('C\logstash\test_log_file\').to_a

Title: Bark At The Moon (Full Album With Bonus Tracks) - YouTube (at www.youtube.com)

brooks_: what error do you get?

oh duh you just said that.

whack: absolutely nothing =/ nothing in the logs, even when I output to stdout, it appears the log entry gets parsed into JSON but just flat out doesn't display in kibana

whack: I got this output = irb(main):003:1'

brooks_: "it appears" how are you observing that?

oh, stdout

man, my brain today. Sorry ;)

whack: no worries : )

Blizzard_: sounds like you made a typo.. or I did.

Blizzard_: yeah I made a typo

Blizzard_: break out (control C) and restart the process

or just contrl-c might drop you at the >> prompt again

Correct me if I am wrong, but Dir.mkdir does not create directories recursively?

Blizzard_: p Dir.new('C\logstash\test_log_file').to_a

mveitas: correct

Then a small PR to use FileUtils.mkdir_p "/opt/logstash/S3_temp" is in order. If you don't have that directory created the S3 output plugin fails

whack: should it be p Dir.new('C:\logstash\test_log_file').to_a

whack: ?

Blizzard_: yeah sorry I can't type today. Yes.

whack: I got this output = irb(main):002:1'

confusing

hmm

it thinks you have an open quotation or parenthesis for some reason

Does anyone know why a multiline filter add_tag parameter would be added to an event even though the multiline pattern defined does not match ? http://pastebin.com/ZUuwq6uh

Title: Multiline filter add_tag problem - Pastebin.com (at pastebin.com)

avleen: you said you had speed problems with lots of fields, how many is lots? What number did you drop to ti get better performance?

gster: I think that was a bug filed a short while ago; I think it's fixed now

whack: I was able to get the whole output with debug finally.

whack: let me get that over to you.

gster: what version are you using?

whack: Ok make sense then, I am usinf 1.1.13

gster: ahh, yeah that was fixed in... one sec

whack: I got the output. We needed to use double quotes.

gster: that was fixed in the latest release

whack: I got an output of "No such file or directory"

Blizzard_: so, "C:\\logstash\\test_whatever" ?

you'll have to escape the \ if you use double quoets

Blizzard_: p Dir.new("C\\logstash\\test_log_file").to_a

whack: Great. Thank you.

agaldfjladfjks I can't type today