#celery

so on this page: http://docs.celeryproject.org/en/latest/django/... under:

Relative Imports

its missing a "be" in "you have to be consistant"

y

I want to run an example in python, but the celery docs are a little confusing. Where can I find a complete example with clients and server that works?

I have celery in ~/Programs/python2.7/lib/python2.7/site-packages/celery, but when I run "python demo.py" I get the error "ImportError: No module named celery". I have python installed in my $HOME. What is wrong with my installation?

xeon123: does "python -m celery" work?

chance: thanks, fixed

asksol: fixed

I want that several hosts exchange data between them. How many brokers I need?

host1 <---> host2

xeon123: everyone talks to the same broker

and if I have 3 hosts interconnected in the same broker, I can send a task to a specific host?

you can if you create a dedicated queue

e.g. celery worker -n myname -Q celery,myname

task.apply_async(args, kwargs, queue='myname')

asksol: you around?

does setting a rate limit on a task cause other tasks to back up behind it?

even if there are spare workers?

dpn`: not for messages on the queue

asksol: ok doh.. I've got a lot of those ;)

inside the worker it depends on prefetch limits, e.g. if the prefetch count is 10 it can only hold 10 tasks at a time, so if all of those are rate limited

it will receive the next message on the queue even if idle

yeah I've got a few hundred thousand ahead of these :P

urm

it will *not* receive the next message on the queue even if idle

curse me whoring the concurrency to speed up imports

asksol: right, no worries.. i suspected as much

asksol: one quickie while I've got you... memory use ballooning with big chords..

I've disabled the chord for now.. but was wondering if I'm doing something stupid with memory or if it's something you suspect might be a real issue

(ie i'm using chord for a really long chain and i'm just stupid for doing so)

asksol: did you try the datagrams?

udp is useless for this since udp sockets are always writable

dpn`: not sure, but how big is it?

asksol: a few hundred thousand items potentially

dpn`: that is probably not a good idea at this point

asksol: yeah, but you don't need to check for writeable

you just write

asksol: ok no problems.. they are shiny though :)

if the worker is available the it will read it

otherwise other workers can read it

dpn`: have some ideas to make very large chords work, but no time yet

ionelmc: if we don't know if it's writable or not we have to do scheduling manually

asksol: ok no worries.. i should have a dig into it so I at least know what you're talking about when it comes to impl details

i.e. have to choose the socket to write to

asksol: http://i.imgur.com/sEWZC0O.png

i didn't try it tho :P

if this works then the OS will balance

but there are multiple sockets, not just one