#logstash

Randm, what was it?

I had two configs with it

so it would connect twice

turns out I need to setup RELP though

the ELB will listen on that port, even if logstash isn't running

and rsyslog will write its data

and then the ELB will close that eventually

logstash behind ELB?

does RELP's lack of SSL discourage anyone?

you can do TLS with relp

in v7

DPP: yes, logstash behind an ELB.

(in a vpc)

but, as the relp plugin sucks..

so for an ES cluster - there can only be 1 master right?

yes

that role can move around

what's the best way to move it around? can curl commands do it?

shutdown the master?

which would allow an 'elligble' one to come up yes?

they'd call an election and figure it out

why?

trying to figure out the best way to move a master around from 1 colo to another if we were to flip live colos to a different location

ah, I don't know about the rack awareness stuff

you'd have to read the docs or ask in #elasticsearch

cool, ty

do we have a good pattern for filtering the results of an ES query, changing mapping, then storing in some temp index? or is there something like that in the works?

<http://tinyurl.com/mkbrwee>; (at github.com)

zounese: isn't that more of an ES question?

fayesal: yes. i suppose it is, but my solution involved logstash. i wanted to see if anyone else came to the same conclusion.

Interesting idea, I've not done it. zounese

fayesal: thanks anyways. i'll try asking in #elasticsearch.

zounese: gl

hi, is there a way I can make kibana look for elasticsearch on localhost:9200 instead of mydomainname.9200?

nicholasf: yes, but that's probably not what you want

kibana runs in your browser

torrancew: right of course

so hardcodign localhost would have it look to the user using kibana's workstation

yes

are you running kibana via the jar file?

I dont want to open my elasticsearch port to the world

yes, I wrote a simple shell script around it

*nod*

ok, so you cannot confiugre the kibana the jar contains

torrancew: should I install kibana another way? Just trying to find docs

but if you deploy it externally (which is frankly saner, anyway, as kibana doesn't need a fat app server to serve it up - it's just html+js+css)

then you can edit that path

ah right

it defaults to "http://" + location of current window + ":9200"

so, in that case, Id maintain my own kibana copy

and Id edit the elastic search path

yeah, you'd just deploy it to a web server, like apache or nginx

gothca

thanks torrancew

is there anybody that could help me with parsing the following log further (my input is syslog, but I am also passing apache log lines as the message, and would like to further process that) https://gist.github.com/pvwoods/24d2604ef2dbc09...

<http://tinyurl.com/pawyj3z>; (at gist.github.com)

is that just a standard apache access log message?

yes Nam, but I seem to be misunderstanding how I should have my log stash filter set up

if it's common logs you just need a grok filter like this .... grok { match => [ "message", "%{COMMONAPACHELOG}" ] }

the regex to parse it is already made

there is also COMBINEDAPACHELOG

oh shucks :/

so I could have something like

if [program] == "my-apache-log-stuff" {

then have that grok match

I will give that a shot

yup, should work

I am currently having a problem with my logstash indexer obtaining some information from redis an processing it but then stalling for like 20 seconds before it does anything

i talked to the elasticsearch support line briefly and the guy mentioned it was somewhat a known problem and that he knew about some way to fix it, but said a support subscription was necessary to proceed further

is anyone aware of this problem and how I can fix it?

our company is looking to get a support subscription at some point, but management doesn't really want to until this is working, whic doesn't really make a lot of sense, but that's management for you