#logstash

Is logstash-forwarder being merged into the logstash-contrib repo? The v1.4.0rc1 tag in the forwarder repo looks very like master on logstash-contrib, but I can’t find any mentions of forwarder in the contrib repo.

lukewaite: the tag was a mistake. Its not being merged with contrib ;-)

Thanks!

Np

hello guys

i need keylogger

hi scharf

hi doggmatic19

i need shells bro

do u have?

all, im running a single node ES cluster atm and due to some disk space issues which spiraled, ive had to restart ES. but now ive got tons of UNASSIGNED shards :( https://gist.github.com/hugespoon/c788e865e5328...

<http://tinyurl.com/kuaye2f>; (at gist.github.com)

is there any way to recover this without deleting my data?

such as force reassignment to local node?

hugespoon: is the count of unassigned shards decrementing? it should be automatically recovering them.

lukewaite: it does at first…then just stops at 821

do you know for sure that they were all assigned previously? if you are on a single-node setup and you had indicies which required a replica (I think the default is 1 master, 1 replica) you would have “unassigned” shards on a 1:1 basis

lukewaite: hrm, i am not

im now also getting maintained yellow status

whereas previously it would go yellow then back to red

red would probably be during the recovery of the master shards, then it would maintain yellow if there were any shards missing replicas

hugespoon: I’d take a look at it in head if possible: http://cl.ly/image/1E2S2l3g1a1o http://mobz.github.io/elasticsearch-head/

Title: Image 2014-03-30 at 12.19.04 PM.png (at cl.ly)

lukewaite: yeah, ive been looking at it through bigdesk, have head installed, but neither are being very responsive atm :-/

can you see if the unassigned shards are just the replicas in either?

lukewaite: cant atm…im just going to shut down things and give it a quick restart so ican get some info before i start shoving data at it again

lukewaite: ok, ive restarted, plugins are responding again…it "looks" like the unassigned shards are replicas

but now im back to just plain red status

820 total shards, 409 active…so it seems like one shard and/or its replica is now flaking

hugespoon: you can update the existing indicies to require 0 replicas which would allow you to go back to green: http://www.elasticsearch.org/guide/en/elasticse...

<http://tinyurl.com/qa84muw>; (at www.elasticsearch.org)

hugespoon: and then you can add a mapping for your logstash indicies that would create new ones with 0 replicas: https://gist.github.com/deverton/2970285

Title: Logstash Elasticsearch Template (at gist.github.com)

hugespoon: as to your single shard not recovering, not sure what you might to to try to resolve it - I think it’s a bit beyond my depth at that point

hrm it might just have no data

let me see

i am the brother to wikkileaks

lukewaite: hrm so ive got it back to yellow seemingly…but its now not responding when im trying to send in data…but LS doesnt seem to be blocking on the output

maybe its just overwhelmed atm by a backlog?

lukewaite: aha! a concurrent issue…hit max open files too

too many open indices i guess

well now i can get more hardware and make this a real cluster

hugespoon: nice!

yeah, thankfully its not 'production' but its nice to be able to save most of the data

So I have my logstash config reading data out of /proc for my parallel filesystem, I had my keys the way I want using rubydebug, I installed carbon, and carbon-cache logs show a connection from the host runing logstash, but I don't see any whisper files being created,

is there a way to debug the logstash output {} block for antyhing other than stdout?

brockp: id check the carbon logs and the logstash logs

it should be telling you if its blocking

in logstash

All I see in the carbon logs is: MetricLineReceiver connection with 10.164.5.164:56756 established

brockp: you can also turn on -v for a short bit on logstash

which is the IP of the logstash source

and it will firehose information

into the log

Yeah I just ran it with --debug whoops

hah yeah

ahhh typo status != stats ;)

anyone know a good syslog agent for windows server?

acassio: I don't but why not just connect nativly to the event log?

acassio: logstash-forwarder and/or nxlog

I did not know that I can connect nativly... I'll search about that. thanks

Title: logstash - open source log management (at logstash.net)

Never used it,