#mcollective

dhollinger: if it works, use it :)

Volcane: o/

Volcane: any chance you an do a new release of puppet-choria soonish?

yeah, i have a few things on my todo list and will get a bunch of releases out soon

mainly tons of things to test https://github.com/orgs/choria-io/projects/3

mhm can a project be secret? I get a 404

really? hmm

let me see

wow didnt even know its possible lol

visible now

ahh

thanks

will get a release out soon, snowed under writing a bloody cron daemon ugh

\o/

why do you write a cron daemon

Do I actually want to know

it'll run just RPCs

ah

today people cron mco rpc

its a fuckup of note

:D

all sorts of issues, loooong running rpcs cron'd every minute etc

so building a web ui to create jobs and targets DCs

where i can have things like a worker pool per DC and can be defensive - dont schedule jobs thats still running from previous invocation etc

and doesnt require people have shell access

this is distributed over 30 DCs with 3 instances per DC in a HA group

I thought about something similar, backed into foreman. Mostly because it's a working UI and nobody like to deal with UI

replies fro the fleet stream back over the choria adapter framework to a central point where it gets reported centrally

so choria brokers gets replies, rewrites them into NATS Stream or something like Kafka

from there i use another choria project to replicate all the replies to a central DC so its resilient to DCs becoming unroutable etc

yeah luckily we already have a big web ui for all sorts of systems needs so it goes in there

ah nice

so i get to pull a bunch out of this soon to open source

the HA framework which sits ontop of the choria brokers

and each of these crond runs a management backplane so i can pause/resume jobs on it if ever there's trouble - also extract running config etc

this management backplane connects to the choria network so i can do `mco rpc cron pause -W dc=DC1` or whatever

ah sweet

and this will become like a 5 line embed into a go app

whole thing - backplane, scheduler, HA - all fits in 25MB RAM :)

way more efficient than the ruby stuff :D

yeah

also means I had to write a choria client + RPC client in o

go

which is already open just need to announce

and its AMAZING

50k nodes full RPC round trip around 1 second

discovers full node set in < 800ms

I saw the demo vids from you on youtube, pretty amazing

had to make NATS a lot faster

now its kind of too fast - it considers the ruby code a slow consumer

but i think i can keep it working around 50k nodes which is about my goal for the ruby stuff

bastelfreak: other thing i want to fix before this round of releases is this https://github.com/choria-io/packager/issues/1#... can you think of any ubuntu work i am missing there?

I use mcollective with multiple of your agents on ubuntu1.04. I don't know at the moment if I already pull in anything from your packagecloud repos, but if I do, it works

ok, then it should be easy stuff, the ubuntu build for the go daemon already seem to work, so thats easy

yep

i put that in a 1804 repo already on packagecloud

https://github.com/choria-io/go-backplane#readme <-- the embedable management backplane

Volcane: so this basically allows my go application to register itself to the broker and then it can be controlled from an mco client?

so orchestration for services?

yes

like this cron i mentioned

we run it in all these DCs and multiple instances per DC etc

i can never get to say a REST interface to manage it

but i can make them all talk out to a management collective where i can use discovery to target one DC, all DCs etc and manage them nicely

mhm

so we develop java microservices with the spring framework and have this: http://www.baeldung.com/spring-boot-actuators#b...

multiple endpoints to reload the service, stop it, get prometheus facts, health data, stuff like that

We thought about a proper way to orchestrate the services

yes, eventually i imagine i will put a bunch of this stuff in this too

same idea

maybe it is a good idea to deploy the backplane as a sidecar service next to the actual microservice, and it will translate calls from choria to the rest interface and vice versa

yeah you can do that too - fine idea if you already have osmething like this

there is always this anoying gap between good ideas and timeslots to implement them :D

yeah :)

sidecar is basically traditional mco and thats perfectly fine - this embedded thing might be nicer to make a very targetted side car with no execute options etc

but generally yeah thats the mco model

i am looking toward systems without mco, or iot etc

yep

for a seperation of concerns, a dedicated side car might be better

so independet people can work and release it

well, in theory

thats what this does

nice

so -W name=/DC1/ would hit all these

for the moment we just need circuit breakers, I'll add more stuff in time :)

but having built this damn thing 10 times over i got sick of it and extracted it into a standalone reusable package

ah yeah

and all slightly different, now i can standardise