#celery

so I have a problem

I set the CELERY_DEFAULT_QUEUE option to a queue, and then unset it

but now all tasks go to that queue

the set up is something like this: I have a machine running `celery multi start ...` that serves as the 'default' queue (i dont have a -Q option set)

and then I have another machine running `celery multi start ... -Q my_queue`, which I only want to run specific tasks from

previously, it was all working as expected: if I specified the queue using send_task(queue='my_queue'), then the task would execute on that queue, and all other tasks would go to the default queue

now, even after having unset the CELERY_DEFAULT_QUEUE option, tasks get routed to BOTH the default queue and my_queue

using redis as a backend... any reason that this might be happening?

hello. is there a doc page or something explaining how the 'pidbox' queue is used?

igors: not that I know of. What do you want to know about it?

VooDooNOFX: I'm having a issue using a beanstalkd backend, it only consumes jobs every 10 seconds. It timeouts waiting on that 'pidbox' queue

bobbyr_: can you show me your celeryconfig.py for this server?

turns out it's a limitation of the beanstalkd backend

it's doesn't do multiplexed i/o

igors: pidbox is used for message broadcasting in celery control. I've never used beanstalk though.

does beanstalkd support fanout exchanges?

VooDooNOFX: i'm not sure, we just use the basic, I'm not really an expert...

igors: https://github.com/kr/beanstalkd/issues/152 the issue was closed as WONTFIX.

So no, it doesn't support it.

I guess we'll just switch the backend for something else, no real reason to keep using beanstalkd

yep, that's the one I found

there is nobody maintaining the beanstalkd backend

Interestingly, in that PR, they claim "beanstalk isn't a real message queue".

Is this amazon by any chance? (just curious)

igors: try setting CELERY_ENABLE_REMOTE_CONTROL = False

It should disable the use of the pidbox queue entirely, but leave you in a state that you can't use celery control's sub-commands at all

VooDooNOFX: cool, good to know. thanks!

igors: however, I'd just use a real message queue (like rabbit) if I were you.

Hi. Working on a conversion pipeline and “kind of successfully” got in touch with celery. Currently I have a major issue in my workflow so that blocking behaviour is introduced. Any comments on my attempts to use apply_async & link http://pastebin.com/MSgdHXX3 ? I’m trying to replace the line create_square.s(res.get(), id, pipeline).apply_async(link=g) with something more suitable. Also the line of code is currently relying on CELERY

i have a GroupResult on which i call .revoke(terminate=False), but that only kills some of the tasks. is there a condition that i've missed?

*terminate=False. in fact i've also tried many different signals too. TERM, INT, QUIT, KILL and nothing seems to affect the task, even though it's not stuck. i do get an error if i send an invalid signal however, which means that the message does get through

*terminate=True.

hm, nevermind, i think a library in the code that's running in the worker handles some signals. although i don't see how it would interact with SIGKILL