#celery

Im trying to rate limit the backend generation of thumbnails on my site, but I still want to requests for those images to behave synchronously. Can I do this with celery?

__joshua___: you can send the task and wait for it to complete, yes

so just AsyncResult.get

and ignore the documentation that says don't do it?

asksol: figured it out, it was actually a global variable that was getting set by the task that ran before it

asksol: nothing to do with celery :)

yeah that would save a lot of time

__joshua___: a task is not allowed to wait for a subtask, but the client can very well wait

yeah, its working well

i have an email task after a report task.... it polls the report task state then returns. Am i at risk of deadlock? it only does a .get IF the state is finished

how do I debug?

my task isn't working

tasks don't get into the queue

nothing happens when I do $$ redis-cli monitor

asksol: before, I was using the auto-create missing queues option, and specifying the name of the queue in -Q worked fine. Then I defined all of the queues and suddenly -Q doesn't limit a worker to the listed queue(s)

hello there, celery n00b here. I'm wondering what is the correct way to enforce a global rate limit among multiple workers? From reading the docs it is NOT task.rate_limit, but instead to enforce it at the queue level. I'm unsure how to actually do this, though.

using RabbitMQ as broker.

EncodeError: <email.message.Message instance at 0x2988830> is not JSON serializable

how do I json this, celery apparently is trying to pickle some big dicts

what does delay() do?

I see a task being pushed into redis but nothing is happening

frege: that is essentially exactly what delay() does. Under the hood it is just the apply_async() function, which fires off a message containing the task name to be run and its argumetns.

the message gets put into the queue.

in order for something to happen, you have to have a worker process running and pointed at the same broker/queue.

xkillac4: I thought it's running

smart-attach-daemon = /var/run/uwsgi/celery.pid celery -A tasks worker

how do I point it at the right broker?

frege: the broker argument to Celery() in tasks.py

Celery('tasks', broker='redis://localhost') or whatever

I'm also a noob, just went thru the tutorial for the first time earlier today

one thing

when you run task.delay()

xkillac4: is there a web interface for celery?

make sure you use exactly the import line shown in the tutorial

I don't think so, but like I said I'm pretty new

yeah I did that

it's pushing shit into redis

probably bcause of DEBUG=True

try running a worker without uwsgi maybe

I did, it didn't send email

[2014-07-28 23:59:09,111: INFO/MainProcess] Task shared.tasks.send_email[2108b557-3099-458c-a882-80ee062a028c] succeeded in 0.00488159499946s: None

hmm

but nothing happened

so then the task is being received and processed, it sounds like... how sure are you of your email sending code