#celery

Hello, what is the difference between apply_async and delay() ??

what if I want one task to be automatically added after a different task? Should I call the second task in my first task??

usr123: delay is a bit more pythonic in that you can pass args/kwargs to the task naturally. The tradeoff is it doesn't support additional arguments that affect how the task runs.

Compare the signatures of http://docs.celeryproject.org/en/latest/referen... and http://docs.celeryproject.org/en/latest/referen...

There's a variety of ways to call tasks together. In your case a simple callback would probably suffice. http://docs.celeryproject.org/en/latest/usergui...

add.apply_async((2, 2), link=add.s(16)), what is .s(16) here? What is .s used for?

usr123: It's a signature, scroll back up to the top of that page. It's a way to define a task to call (with arguments) without actually calling it.

Useful for passing around to arguments that expect a task.

mattmcc: So I could use an 's' as well to call with arguements????

Well, consider the example at http://docs.celeryproject.org/en/latest/usergui...

Here, other_task will be called if the add task succeeds.

They couldn't do link=other_task() because that would call it immediately. And just passing the function reference, link=other_task wouldn't let you specify arguments for the task.

josuebc: my datetime issue turned out to be a celery issue (which has now been patched)

Thanks for your help last week

I've been playing around with celery for some time now. It's awesome.

My problem right now is, that I don't know how I should handle the 'retrying' of tasks, in a non blocking way.

Would it be smart to put a list of "tasks to do" in a separate database?

.. I'm afraid I will end up with many workers doing the same task

you can pin workers to queues

and balance that way

fleetfox: is using "import inspect; active_task = i.active()", a good way to prevent duplicated tasks, according to you?

before starting with my tasks, i could see if the same task is already running.

.. alternatively: I make a special table in my DB, which holds: currently processing tasks.

why do you want to do that, that makes little sense to me

Thanks

I'm building a system to process RSS feeds

(everytime a new article comes in, I want to scan it)

Perhaps my job is still running, and I would scan the article another time. (I wanna stop that)

So my plan is: use inspect(), to check if an job is already running with that *specific* url, so I dont scan an article 2x.

good plan?

(apologies if this is all too stupid, i've never done anything like this before)

not a good plan

run a dedicated queue and one worker

it can only do on task a time

wow

thanks!

as to making sure it's unique http://docs.celeryproject.org/en/latest/tutoria...

or is the whole problem about making them unique?

each page is different; I'm sure the jobs are unique in that sense

and why don't you want to run multiple jobs concurrently?

ahh.. damn, i have no experience with all this stuff. put i'm gonna fix it

thanks for pointing me to the cookbook

my worry is about "losing data" so to say

but its probably a foobar worry, i'm just gonna implement it

you should try describing your problem better since it sounds like XY

I have run celery via a third party launcher for some years. I now want to get rid of this and just run "vanilla" celery. I have created a "celery_config.py" file which defines my application and beat schedule, what is the correct way to then launch/run the beat scheduler with the provided setting files? I have something like this: https://gist.github.com/crooksey/2faebf0570f4cd...

celert -A celery_config ?

I tried: celery -A celery_config beat

And it ignored my settings, and gave an output for a different broker etc as specified in the config file

Does my setup look correct?

i've just glanced over it

read this

Yea I see that and it makes sense

I just had to split my file with a app.config_from_object('celeryconfig')