#logstash

Guest36640: that would help :)

echelon: where the hell are you finding this?

Title: logstash - Pastebin.com (at pastebin.com)

Thanks Rastro - this is driving us all nuts....

Guest36640: unexpected token at '## ....

torrancew: vendor/kibana/app/app.js

you should never, under any circumstances, edit that file

it's comppiled

Guest36640: next line: json/ext/Parser.java

generated from simpler code that you won't find in vendor

Guest36640: we were just complaining about json's lack of comment support!

again, you should honestly take 5 minutes, disable ls-web, install nginx and point a docroot at kibana - it will save you so much pain, and win you so much flexibility (and quite possibly, initial loads of kibana will be faster)

I don't know why it worked or didn't work when you did what you did, because that's an unintended action, that we considered impossible until recently (you'd have to explode the jar in past releases)

we're hanging out in non-determinism land

Guest36640: oops, maybe i got lost in your stacktrace.

Guest36640: "LoadErrror: no such file to load -- jmx4r"

I ran ls-web for about 6 months, due to laziness

and I could not believe how foolish I had been, when I finally deployed it on its own

(separating it also allows you to sanely secure it)

on Ubuntu 14.04.1 LTS with logstash 1.4.2-1-2c0f5a1 (part of my ELK stack), if I run service logstash start, I get no data in the Kibana dashboard, but if I run /opt/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf, I do get data. Can someone help me out?

yes, we cant understand that line

I think we don't adequately understand what this plugin can and cannot do.

Habitual: what is the init script actually running?

Habitual: couple o fthings: 1) Ubuntu uses upstart, and logstash ships both upstart and sysv scripts (not the typical upstart wrappers) - try "start logstash", "stop logstash", "restart logstash" for a bit, see how they work

the othe rthing to check is permissions

sorry for the typos, laggy network

I have a jmx json file at that path that I have been told to use to grab performance data about logstash - but thats what I get...

Guest36640: how did you install logstas-contrib?

I just took the jxm.rb file and dropped it into the input folder (windows)...

aha

yeah, you need to install logstash-contrib in full

There are posts saying that it hadn't been installed properly

it has other dependencies on that package

rastro: You'll have to clarifiy that Q. it's the stock init script supplied by the .deb.

We noticed those

But as we are on Windows and using Logstash 1.4.2 it seemed moot?

Guest36640: highly unlikely

contrib is not intended to be cherry-picked from

there are embedded gems and such in there, that some plugins depend on

and that's exactly what you're seeing

jmx4r == jmx 4 ruby

start logstash

logstash start/running, process 25942 and lsof -p 25942 is empty.

typo

lsof -p 25942 is empty

Habitual: time to check the logstash logs, permissions on those logs, adn the user pid 25942 is running as

Guest36640: yup, you need jmx4r installed the way LS is expecting it.

Okay. Damn.... sorry guys.... you know what I'm going to ask now eh? How??

Guest36640: on windows, not sure, but I think we have a wrapper script for it in bin/ ?

Habitual: the script may be providing flags that are different than what you're setting on the command line. comparing those might help. also, as always, listen to torrancew.

I thought those instructions were Mac/Unix only....

ah Okay...

Not sure - Not a windows user :/

I have to go now to pick up my wee one,,, will try from home later. THANK YOU VERY MUCH!!

logstash logstash 0 Oct 22 06:34 logstash.log

np, and good luck!

so empty log file, Habitual -- what user do you run it as from the CLI?

the manual method with -f I'm running as root.

ok, so question - you listening on any ports < 1024?

torrancew: there's several listening ports < 1024 but java isn't one of them.

I know that's not what's /happenin/g

but what did you configure?

I have a logstash.conf in /opt/logstash/ and /etc/logstash/conf.d and they identical.

Habitual: what I'm getting at is, that if you configured LS to listen to any ports < 1024, it won't work as the logstash user

try running your manual command as that user, to see what is happening, pehraps

how come I have a tag=false on my messages ?

lofic: not enough data

would need to see configs, possibly sample messages, logs, etc

torrancew: is su - logstash -c "/opt/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf" sufficient?

maybe, but why not just get a login shell, then run it?

why half-hog it, I say :)

OK I polish the config and paste

well, the deb did this: logstash:x:999:999:LogStash Service User:/var/lib/logstash:/sbin/nologin

ah

I can fix that easily.

then just go straight for a sudo -u or something

no, don't fix that

k

Let's find the problem first

if it winds up being that, let's know, not guess

sudo -u logstash barkls with "usage" output.

barks, woof

sudo -u logstash /opt/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf ?

(-c isn't valid for sudo )

running sudo -u logstash /opt/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf now

k

milestone message...

those are normal

yes, but data usually follows. emphasis on "usually"

tick tock. only the milestone usual 'banner'.

perhaps add some -v to the command

also, we're getting well into the "can you paste your config" territory

I can pastie with the best of 'em.

as we're at a point where I don't know what you normally see, or what you should see, I'm just kind of waiting to see if anything explodes

torrancew: http://pastie.org/private/1xji6f83j33ugfwepvjyya

Title: Private Paste - Pastie (at pastie.org)

Habitual: can I get some version of the config? (Sanitizing hostnames or whatever is ok, but try to preserve context - changing 14 different strings to 'foo' will just confuse me greatly)

Title: Private Paste - Pastie (at pastie.org)

Title: input { file { type => "syslog" path => "/var/log/secure" } } - Pastebin.com (at pastebin.com)

hello. i’m trying to use the file output with logrotate. https://gist.github.com/tphummel/b952ba08706960...

<http://tinyurl.com/k9pxbk5>; (at gist.github.com)

Habitual: these files live, and have data coming in currently?

.../kibana/web/web.log does, dorkblog is 0 bytes

is there a signal i can send to logstash to close the file descriptor on postrotate?

lofic: misuse of tag_on_failure

<http://tinyurl.com/ko886fy>; (at logstash.net)

it's an array of tags to apply when the grok fails

so the grok failed, and you got "false" added

tphummel: You can bounce it, or go copytruncate

no HUP or any other clever signal stuff I know of works, though

yes but the grok seems to success : it adds the custom tags