#logstash

RSN

if you use haproxy with elasticsearch it's likely you are doing something very /very/ wrong. Although it's also possible you have a crappy client driver that you disabled the cluster abilities on

so it maybe the lesser of worse

also don't forget to deploy freenas to help scale logstash ;)

Damm: there is nothing wrong with a proxy in front of elasticsearch.

a dumb proxy? sure is

Damm: Wolland was even reporting remarkable speed improvements from putting a proxy in front of a cluster of nodes.

writing to a deadless node that's non master

will likely be just as performant as haproxy

s/deadless/diskless

gchristensen, basically my concern is is the data right? I've had scenario's where elasticsearch clustering has dropped the ball

lost a few bits of a shard without an issue

and then your replicas are off

so my paranoia about writing to 1 is basically that the clustering protocol isn't perfect in elasticsearch and multiple writers I would suspect not all events would be recorded properly on all nodes in high load

single point fails you know where to look and start picking up

multiple points of failure ... lots more places to clean up

clients don't handle failovers that well

we had divergent data when it should be 'clustered' and sharing the data properly

nothing worse when you find out that lie is really a lie

divergent data is just a fact of life when replicating and dealing with multi-master

it happens

yep but when the data does not converge (or clean up)

it becomes a hanging chad over your neck

imo

can't speak to that, never had that experience

sometimes companies try to punish people unjustly for an error

you working in north korea?

Wolland, nope. But I've been sued for a product going down and being the Sysadmin on staff

and had to defend myself

wow, that's messed up

sued by who?

local startup who puts a lot of devices in school buses

client of the company you work for?

which is how I almost landed in jail

because I disrupted a public service

they were grasping at straws trying to punish me

interesting

Wolland, remember the bridge collapse in the US back in 2007?

our devices were in many vehicles during the outage

and that's when the outage happened

so lots of government entities were depending on us when a coworker sabotaged a service

and brought us down

sabotaged on purpose?

wow

likely but he wasn't on charges

but I admit after reading this book that's very Marxist I realized that the greatest asset a company can ever have

is it's working staff

(or it's slaves)

sometimes the most untapped into asset and if you can make them do your bidding for less

win