#celery

celery/celery#3678 (master - e3d1993 : Ask Solem): The build was broken.

Change view : https://github.com/celery/celery/compare/739f6c...

Build details : https://travis-ci.org/celery/celery/builds/1481...

celery/celery#3679 (master - 80c9bd3 : Ask Solem): The build was fixed.

Change view : https://github.com/celery/celery/compare/e3d199...

Build details : https://travis-ci.org/celery/celery/builds/1481...

After recently reinstalling the development branch off of github, I keep running into import errors.

Like for example:

eb_1 | File "/app/user/github_scraper/worker.py", line 15, in <module> web_1 | worker = celery.Celery('worker', broker=os.environ['CLOUDAMQP_URL']) web_1 | File "/app/.heroku/python/lib/python3.5/site-packages/celery/five.py", line 148, in __getattr__ web_1 | module = __import__(self._object_origins[name], None, None, [name]) web_1 | ImportError: No module named 'celery.app'

Sorry, here it is formatted: http://pastebin.com/2SpGCSFx

is it possible to add info about a worker during registration? For instance, I would like to know the total of RAM, CPU free space etc. this way, I could split large tasks more efficiently

hey there, I am using celery workers to consume from dynamically created queues

whenever a new queue is created, I send an event to all workers and create new consumers

everything works fine

except that when workers consume from ~1k queues

processing rate drops dramatically

can’t find anything about consumer limitations in documentation

anyone ever experienced something similar?

tbarbugli: which broker?

rabbit

tbarbugli: are you 100% sure consuming rate drops (i.e. blame celery) or producing rate drops (i.e. blame your producers or rabbitmq)?

here’s what i saw happening a few times

a bunch of workers connected to 1k queues

all queues are empty except for one with 100k messages

a couple of workers consuming only the queue with lot of msgs

ack rate for the workers consuming only 1 queue was much higher

something like 100 times higher

tbarbugli: is there a chance you can attach to the celery processes with `strace -p` and inspect what they're doing exactly, when listening for 1k queues?

and create the issue with this information

i will try that