#celery

right, I can try this

then again, maybe you just want to configure a output file in your init script

in celery init script ?

there must be something there

celery log by himself, only things like "logging.error("some error")" aren't working

and these kind of calls are inside methods called by task

what's the best way to get celery to log any task exceptions/errors to sentry when using djcelery?

there aren't that many ways to start with

least invasive is to just use sentry's logging handlers

Zac12: or appenlight.com

Zac12: https://appenlight.com/page/python/celery-excep...

like this

it will also use python logging if you want

many thanks to all, I'll try these solutions

Ergo: what's app enlight? an alternative to sentry?

Zac12: sentry + new relic

those are really horrible examples there

according to asksol they were ok

they should at least encourage the user to make a decorator for the tasks, instead of riddling the task code with unrelated code

ionelmc: there is nothing related to tasks there

those are signals

ionelmc: and keep in mind that task_retry signal was added on my request

oh oops, i only looked at the last example

so all older i think pre 3.0 celery instances need to use it this way

but that last example still can use a decorator

ionelmc: have an example?

im following celery docs on this

also for some reason when using task-retry signal - i couldn't retrieve local frame vars

so this seems most proof way of doing it

but im all open for improvements

Ergo: http://paste.ubuntu.com/6843371/

now that i look at it it's much uglier :)

ionelmc: initial plan was to actually make

client.ext.register_celery_signals()

and register success/fail tasks there

cant remember why i didn't do it though now

ionelmc: like what?

I can give more details but I need to know what ::p

hr: any traceback ?

very short and not helpful to me

In [65]: result.traceback

Out[65]: u'Traceback (most recent call last):\n File "/home/stefan.berder/.virtualenvs/ledapei_web/local/lib/python2.7/site-packages/celery/backends/base.py", line 521, in on_chord_part_return\n culprit = next(deps._failed_join_report())\nStopIteration: TimeoutError(\'Operation timed out (3.0)\',)\n'

it means the results do not exist in the backend

ok, I had to deal with something else for a bit. Back, and now looking into getting access to the "hub" or "timer" from my Consumer bootstep. The docs (http://docs.celeryproject.org/en/latest/usergui...) say that "Consumer" has a .hub, but "your bootstep must require the Hub bootstep to use this"

when I have "class BatchConsumerStep(bootsteps.ConsumerStep):\n requires = ('Hub', )" I get "ImportError: No module named Hub" from kombu/utils/__init__.py", line 91, in symbol_by_name

the bootsteps.ConsumerStep already has "requires = ('Connection', )", but if I replicate that in my own class, I get the same import error

maular: is that in the example

?

asksol: I based my bootstep from the Consumer bootstep example (http://docs.celeryproject.org/en/latest/usergui...), but then I needed a timer to do the flushing

I had a threading.Timer, and found this interfered with the signalling (ie the timer thread would need to wake, before the process would restart with SIG_HUP)

so ionelmc recommended I use the "hub" instead (similar to how the celery.contrib.batches.Batches does, although it's accessing a consumer.timer directly)

maular: I cannot see it says 'Connection' there?E

you must use the fqdn: 'celery.worker.consumer:Connection'

or import it and use requires = (Connection, )

you must not use threads in a worker when the event loop is used

ahh, that could be it. Sorry no, the "requires = (Connection,)" is from the bootsteps.ConsumerStep source directly

the worker has a timer component for this purpose anyway

weird, that is wrong then, but it must have worked at some point

you can just override the attribute in your subclass though

consumer.timer.call_after(10.0, fun, *args)

I don't think you need to require the Hub in a consumer bootstep

hub is a worker component, and that is already fully started when consumer bootsteps are loaded

using consumer.hub means that it will not work when the threads-mode is used

using consumer.timer means it will work both in the event loop and with threads

I don't have consumer.timer... is that because my consumer is a kombu.Consumer?

yeah, not kombu.Consumer you need to celery.worker.consumer.Consumer object

need the*

ok I'll use that... I have no desire for a kombu.Consumer, just following the example :)

ConsumerStep.start(consumer)

consumer is the object you want

i.e. the bootstep methods are called with the parent object as first argument

I have considered renaming celery.worker.consumer:Consumer but have not figured a better name

asksol: Processor or WorkConsumer ?

With early acknowledgement, is the task lost if the worker crashes in the middle of execution?

wait, I'm confused. If I subclass celery.bootsteps.ConsumerStep, I need to implement get_consumers. This should create a celery.worker.consumer.Consumer instance, which needs to be passed the Celery app, and a timer...

gp: yes

Is there any way to recover from that without using acks_late?

Going to give redis a go, but I need to avoid visiblity_timeout issues so I don't think I can use acks_late

asksol: I have a question, I try to subclass Task, and override __call__, but super(MyTask, self).__call__() not works as expected (decorate Task.__call__ with monkeypatching not works too)

yeah, I'm stuck with this ConsumerStep... to be honest, looking at the celery.worker.consumer.Consumer source, it looks like I can just implement a custom "strategy" to handle the tasks (buffering without ack), and then I'm already in a consumer with a timer...?

Is it possible to manually acknowledge tasks?

asksol: how does it not exist in the backend?

any special reason for this happening?

and a timeout error doesn't seem so consistent with no data available...

gp: there was this question on irc in the past

hr: it must have been removed in the mean time

hr: .join waits for the results to be available, so time out is suitable

asksol: is that possible that it would happen in a group/chain?

asksol: this is the task that is failing

In [66]: task

Out[66]: (foobar.tasks.img_resize(708, 'big_square'), foobar.tasks.img_resize(708, 'article_thumb')) | notification.tasks.activity_create('ArticleAddActivity', owner=1, article=709)

the activity_create never happens because of this

hr: only chord waits for results

that's the strange thing about that exception

asksol: are there any other fds except sentinel, inqW_fd, outqR_fd, synqR_fd and synqW_fd that might be registered in the hub ?

for redis/memcache it would mean that the counter was full, but the results were not actually stored

I have no chords in there

if you have a group chained with another task then you have a chord

ok

how could I check this counter?

the key is chord-unlock-{group_id}

ionelmc: not for the pool