#celery

I'm getting an import error from within one of my installed django apps when I attempt to run a celery worker.

The setup is a duplicate of

"No module named models."

hello

I have been trying to use Celery with Django to enable asynchronous execution of database updates. I have tried Celery with both RabbitMQ and Redis, and the combination (former for the broker and later for storing results) of them. One thing I noticed is that the latency (difference between the request for a task is placed and time at which gets picked up for execution) is significant - in the order of few hundred milliseconds (~ 1 sec).

That too, this is happening on a machine with substantial resources (8 Intel(R) Atom(TM) CPU C2750 @ 2.40GHz and 32 GB RAM). Moreover, there were not many other processes running at the same time.

Any advice for reducing this latency?

hi, i am using celery with postgres as message queue and have workes on two different hosts. it seems like some tasks are picked up by two workers at the same time, any idea?

mrud: that can easily happen if you use the database transports, they are made for testing purposes...

see http://docs.celeryproject.org/en/latest/getting... (limitations)

can probably be fixed with some locking if you want to look into it

asksol: no, now i have a reason to move ;)

may not be that difficult now that Django has select_for_update and friends, the transport is based on the old django-queue-service project

hm i will probably switch to redis

also postgres has features that could be used to implement queues

but yeah, redis will be a lot less effort

maybe i have a look

btw. i really like celery, used it before on multiple occasions and it is rather nice and simple

thanks :)