#mcollective

Hello everyone, I have a problem, that I'm not able to trace the server, which is running middleware and federation broker

however other servers in this federation works fine

worked last week didnt it?

it worked in testing environment

now I deployed it to prod and it's almost working

so what happens in logs?

as far as I see, client nats receives the message and sends it back to federation

'client' is where you run mco

I'm running mco trace from server, where nats and federation broker is running

i dont know, you're seeing stuff, i dont see the same stuff, best share what you see

OK, one second

https://pastebin.com/1JurGu8f this trace is correct, however when I want to trace 'federation_broker+client', mcollective service is not even receiving the message

'client' is where you run 'mco foo'

clients dont send messages etc

so using client there is really confusing

i mean clients dont reply to messages

your mcollectived are 'servers'

and the end point of a trace is not 'client_nats_broker' its the mcollective server on some node

i get the need to anonymise stuff - but then you have to do so in a way that does not make it impossible to follow

Maybe I'll just replace it with foo and bar, prepare more logs and then ping you Volcane

Volcane: https://pastebin.com/9TMXMejU

and mcollective log from foo.testing/

are you still running as root?

nope, running as regular user

also note, not federated message works, so when I set CHORIA_FED_COLLECTIVE=, it's working

about mcollective log from foo.testing. there is nothing

put it in debug

Of course, sorry

ok so when like that, and you do your trace, do you see anything in the logs?

or even just when mco ping with the federation configured

No, I see nothing, just pluginmanager

with stats

if you take a look at nats logs, you see that in first example, it forwards? the message

in not working fragment, there is much less logs

it publishes but the mcollectived never gets it somehow i'd say

the lines 74-75 is the federation broker publishing it to the mcollectived

I see "->> [PING]" and "<<- [PONG]", and lines you mentioned have only "->>", so I thought nats receives the message, but it's not sending it

or it is other way round that nats is sending PINGs ?

https://pastebin.com/9TMXMejU lines 74 and 75

I think I figured out what's wrong

I was missing federation_middleware_hosts in plugin.d/choria.cfg

however in docs, it's federated_middleware_hosts

the problem was that middleware_hosts was pointing to bar.testing, instead of foo.testing

and federation_middleware_hosts was missing...

ah :( pls send a PR or make a issue about the doc showing wrong key

I'll do both :)

anyway, thanks for help

Volcane: I created both issue and PR. As I said last week, description to each config parameter would save me a lot of time today

But I'm happy that it's all working properly now

thanks :) merged already