#celery

didn't help

(I restarted flower and reloaded the page after restarting celery worker -E, just in case)

mgedmin: it should be somewhere round here http://docs.celeryproject.org/en/latest/usergui...

mgedmin: what broker?

rabbitmq, I think

amqp://...@localhost

are you sure flower connects to same vhost?

the Tasks tab shows real tasks

the Workers tab shows realistic Completed Tasks numbers that change when I reload the page

ook

what doesn't it show again?

https://github.com/mher/flower/issues/324 has screenshots and the JavaScript traceback

I don't see any output on the terminal where I ran 'celery flower'

mgedmin: looks like a bug in flower

try different release

or browser

:/

yeah

stop using internet explorer :p

anyway thank you for pointing me towards celery worker -E

now at least I can see things happening in 'celery events'

we've all been there :)

Hi all!

Anyone using 3.1 with Python3.4.x?

Has anyone experienced any problems?

I have two celery apps in my django project and I'd like to control to which app handles which @shared_tasks.

It seems a bit arbitrary which app handles which task now and I'd like to specify which app will take the task on execution, or on definition of the app, but I can't find how to do this.

Does anybody here have experience with multiple apps and shared tasks?

'ello! Anyone aboot?

rabbitmq-plugins enable rabbitmq_management -> one of the four graphs now works, the rest remain empty

Hi, i am having an issue with celery 3.1.17 django 1.7 an redis as a broker. I don't understand why task duplicates with time. But the django task didn't created them. I had to add a db checkup to see if the task was created or just appeared from nowhere

could it be a problem with my settings ?

hello
i have run into problems with celery

at some point it starts to "choke" with tasks

e.g. now i have 1800 tasks in "reserved" state

so the lag between a task being "received" and "started" can be as large as several minutes

which makes my app non-functional

those it creates task from nowhere ?

how can i go about increasing celery throughput?

because i have the same problem

currently i have one node with --concurrency at 20

but it only runs 8-10 tasks at any given moment

not 20 as i'd expect

Austriker: not in my case, those tasks aren't from nowhere, i just want them to be processed faster

most of those tasks fall into two categories duration-wise: less-than-a-second long and 3-to-15 seconds long

i set prefetch multiplier to 128

enabled transience for most queues

of which i have 5

i run about 100 of 3-to-15 seconds tasks in 10 minutes

and one short task a second (it's a scheduled task)

plus some other rarer tasks

help! timezones! aaugh

flower emits this on startup: "Substantial drift from ... 7200 seconds [orig: 11:50, recv: 09:50]"

CELERY_TIMEZONE and CELERY_ENABLE_UTC settings do not appear in git grep, so I assume default settings are used (UTC)

my timezone is UTC+02, which is 7200 seconds (what a coincidence!)

so I set TZ=UTC and restarted flower

"Substantial drift .... 3600 seconds"

wat

I suspect this may be why my task completion etc. graphs are empty

does anybody understand how celery and flower use timezones?

can anyone say what I'm doing wrong?

I am also experiencing this MySQL server has gone away issue after restart mentioned here: https://groups.google.com/forum/#!searchin/cele...

I am wondering how I go about closing database connections at the start of a new worker process

is there a hook for doing this in celery?

robsmoniker: you can use custom task class like so http://celery.readthedocs.org/en/latest/usergui...

dunno what is the best practice here though

That hooks it into a specific task though, I ideally want to reset the database connections at the start of the worker

robsmoniker: celery already does connection management

robsmoniker: do you use CONN_MAX_AGE or something strange?

yea

CONN_MAX_AGE is set to 1200

Helloooo! How are you people?

going crazy because timezones

uff! i hate! timezones u.u

Are u using Django?

yes

Wich is your problem?

but my current pain is caused by flower (I think?): https://github.com/mher/flower/issues/324#issue...

afaict the "Current drift" message comes from celery.events.state

You can use rabbitmq monitor if flower is not working..

sometimes I get in the mood where I want to improve things for everyone, and not just me

ionelmc: I am consider hooking Worker.on_start method to close connections does this sound sensible?

ok, Worker.event receives two timestamps

robsmoniker: there's a signal for that

but there's also a fix for the connmaxage issue in 3.1

pip install https://github.com/celery/celery/archive/3.1.zip

now all i need is asksol making a pypi release :-)

asksol: pong

mgedmin: i'm not so sure now about the default

best to check with celery inspect

are you saying the documentation lies to me? :O?