Do I really need a chain in order to execute tasks sequentially (next one comes after the previous one finishes?)
I ned to execute t1, t2, t3, t4 (...) in that order, and collect the results
Celery complains when I call .get() inside a task so I do not know how to solve this problem otherwise...
the results are completely unrelated...
t2 does not depend on t1's output
chillaranand
Dejan: why are calling .get() inside a task?
Vathau joined the channel
italorossi joined the channel
Vathau
Hi everyone, I'm trying to use platforms.set_process_title but it doesn't seem to work. Is there a way to disable the celery proc name change so I can use setproctitle? I start my worker using celery.start(argv=['celery', 'worker', '-l', 'info', '-Q', self.options.queue_name, '-n', self.options.worker_name]).
italorossi has quit
alekibango
Dejan: its bad idea to call get inside of task. really that task is blocking .... and waits for another task to finish... it may even lead to mutual lock, every worker will wait for another...
Dejan: if you want to run task from a task, do not wait for results
let the other task do something with its result, ie run third task
italorossi joined the channel
domino14_ joined the channel
italorossi has quit
chillaranand has quit
domino14_ joined the channel
italorossi joined the channel
italorossi has quit
Dejan
I do understand all that, but I am really fin if task is blocked while waiting for result from other tasks
my tasks are executed inside different processes, so I do not worry about problems that are common in other execution models
s/fin/fine/
bomb-on joined the channel
italorossi joined the channel
lllama joined the channel
lllama
hello all. Has anyone got a recommendation for a broker and results backend for local dev? Was looking to use sqlite but I see that this are no more since 4.
italorossi has quit
mihaj has quit
alekibango
Dejan: how many workers do you have? how many concurrent jobs are you doing? you might get surprised how fast you cant get into problems :)
s/cant get /can get/
Dejan: also, it will slow down the workflow a lot
Dejan
Is there a way to execute a group() sequentially?
one-by-one?
alekibango
dejan: if you can reimagine your tasks in a way they do not need ordering... it will work better
every attempt to make lock, synchronization or ordering in distributed system takes huge, tremendous impact on effectivity.
Dejan
look mate, i am not stupid, i understand all you are saying
alekibango
i am not saying you are stupid, just trying to help from my experience ;)
Dejan
in this particular case, i have to wait for the task to finish before i do the next...