#celery

Alright, I'm a bit lost in the documentation and I'm not entirely sure how to work with Celery. I have a webapp from which I want to send tasks to subscribers (task executed once by one subscriber). How do I pass configuration from a dictionary? (All I see is from an object)

Hey, did something change recently with celery's logging? I just pulled master and celery beat doesn't seem to be logging anything at all.

Does anyone have recent experience using celery with SQS?

asksol: librabbitmq doesnt have any kind of shared memory stuff right?

These logs show our celery clients show a single point of time after which all queue declares never reply.

They also show the uwsgi root pid connecting to celery which is pretty weird

│·····················································

I suspect it could be related to this

We've seen that error before too

I think I might need to try and make a reproduction in plain C :(

alexs: by shared memory you mean shm?

asksol: Yeah

Anything really

64 client procs all stop being able to declare stuff at the same time :-/

well it does share memory, it uses globals and static variables

did you check the rabbit logs?

I havent yet

It usually only effects 1 of 4 machines at a time

if they all stop at the same it would be more likely to suspect the network

THey continue to publish things

I have noticed many reports of librabbitmq hanging due to network issues though

they, as in the same connections declaring queues?

Yes

how would they declare queues and publish at the same time?

Other processes continue to publish

but as soon as those other processes need to declare a new queue they hang

well, publish is not synchronous

declare needs a reply, publish doesn't

I see

The send buffer would fill up...

if the network was totally broken

and/or they won't get a reply

publish would block or fail if the buffer was full

Weirdest thing in rabbit logs is a few "unexpected frame" errors.

right, assuming there's no router reading the buffer

There's no intermediary

"expected content header for class 60, got non content header frame instead",

'basic.publish'}

still, the only difference between them is the rpc reply, otherwise the same code

That's from one our affected hosts actually

OK so if this is a network problem, how do we work around that?

There's no enqueue timeout afaik