#celery

[travis-ci] celery/celery#2022 (master - ca4c9b0 : Ask Solem): The build was fixed.

[travis-ci] Change view : https://github.com/celery/celery/compare/f05a95...

[travis-ci] Build details : http://travis-ci.org/celery/celery/builds/27496514

that's funny

heya

still havent managed to fix my issue

asksol: you around? someone told me you might be ableto help

what issue?

asksol: I have celery running production,using django celery and redis as my borker

*broker

we have a couple of periodic tasks, and when I change the code on those tasks or even the crontab settings to change the periods there are no changes

even when the beat is not running and after restarting the workers several times

and deleting the beat file

and deleting the pyc files

it's a mystery, and I am sure I am doing something stupid but I can't figure what

Gnoze5: do you restart the worker too, right ?

yeah

Gnoze5: do you restart the beat too, right ?

and Ive made sure several tims that no worker or beat was running

yep

maybe some ghost processes ?

the more interesting symptom is that even when the beat is NOT running the tasks are still running

(the periodic tasks that is)

maybe some ghost processes ?

I dont know how to get rid of them..

do you use signals ?

yep

but not in the context of these tasks

ok

in my experience

if something goes wrong in a signal handler

e.g. an unhandled exception

is there the risk that the worker crashes

or something that classifies it as ghost

[travis-ci] celery/celery#2024 (3.1 - 48aaecb : Ask Solem): The build passed.

[travis-ci] Change view : https://github.com/celery/celery/compare/1aec45...

[travis-ci] Build details : http://travis-ci.org/celery/celery/builds/27502732

my advice is: kill all processes: rabbit, worker, beat

and edit your code in a way that can handle exceptions in signals

and then try

but handle exceptions for signals in general?

in signals, try to handle all sort of exception

don't let propagate

hey is anyone available to chat high level about celery

im trying to figure out how jobs are sent and executed

ie

if the client defines a task and then shecdules it and then it runs on a remote worker

does that worker need to have the same directory structure and module set?

or is that code streamed to the worker from the client?

does that worker need to have the same directory structure and module set? -> yes

bdonnahue: do you have a local function. All is needed by this local function is needed by the remote function too (worker)

ok that makes sense

my other question is: how does celery handle limited resources ie. how does the scheduler work. is there a priority system for tasks?

Gnoze5: sounds like you have duplicate beat process running if the tasks are sent periodically when you stop beat. Always use a --pidfile argument (with an absolute path)

bdonnahue: i'm linking to you a good doc resource

asksol: but I cant see another beat process running

bdonnahue: http://celery.readthedocs.org/en/latest/faq.htm...

bdonnahue: read ^

Gnoze5: I understand, but it explains your systems perfectly and there is no alternative explanation that I can think of

your symptoms

asksol: right now my beat is dead for example and the tasks are still running, how do I stop that madness?

so look harder for a stray beat process

hm

wait

argh

there is a -B option which embeds the beat in the worker process right?

silver fix. thanks for the post. So the docs lead me to believe that cellery does not support it regardless of messaging platform