#celery

mike_k: overhead to setup rmq? It's amazingly easy to setup, thanks to erlang/otp and docker. But still, there are hosted rmqs available, like https://www.cloudamqp.com/

mike_k: redis is even worse than sqs. It does not provide any guarantees so you _will_ loose tasks once

and it wasn't designed to work as a message broker at all

malinoff, true, Redis is something I'd like to avoid. We've had a discussion regarding rabbitmq internally and it was declined, so I just have to face it. SQS has zero maintenance cost though.

mike_k: any specific reasons?

God just use a hosted rmq - also zero maintenance costs

malinoff, I have no exact reason for that as I wasn't involved in the original discussion (something about yet another stack for devops to maintain, IIRC). Anyway, this seems to be an off-topic.

there is nothing as easy as redis...

that is the reason we picked it

mike_k: some questions have no answer except "Don't do it". You have only one reason to use SQS - because it's hosted. cloudamqp provides hosted rabbitmq. No reason to use SQS anymore.

Dejan: have you ever tried to understand how redis as a message queue works? Have you set it up to not loose tasks?

since we do everything on AWS we may switch to SQS too

malinoff, see the above - I actually get too many tasks :)

Dejan: the above? What exactly?

(the SO URL I pasted)

I actually never saw the task lost to be honest

but I trust you it happens

in our case it is fine

the problem we have now is much worse than losing a task

because all our tasks are long-running

and if something is missing, we re-run it

Dejan: can you reproduce this behaviour locally, with redis and 3 workers (running with different names)?

malinoff, i could not reproduce it when I wanted

but it did happen in my local (testing) environment

I basically run 3 LX containers with the same configuration as in production/staging

the inspect active on SO is from production

Dejan: we can't really help if you can't reproduce it reliably

I know ... I hoped for some answer from people who encountered the same problem before

hi!

I've been away for awhile (personal reasons....) and today i wanted to clear the issue with pull #3433

[ celery/celery - Distributed Task Queue (development branch) | ★ 5944 stars | 233 open issues ]

And because it has been quite a long time... I'm a bit lost

so if someone has a few minutes to spare to guide me it would be neat

Mulugruntz: you've been asked to do a proper pull request (via a rebase). Have you did it?

malinoff: well, I've been trying to do it for half an hour... but I think my local repo is broken

I'm cloning to start anew

(from my remote)

Mulugruntz: once you're done with it, create a new pull request and somebody will review it

ok

malinoff: The thing is.... my changes already are in master

well.... except my name in the contributors.txt, everything has already been integrated a long time ago

[ celery/celery - Distributed Task Queue (development branch) | ★ 5944 stars | 233 open issues ]

[ celery/celery - Distributed Task Queue (development branch) | ★ 5944 stars | 233 open issues ]

so I'm not sure was is actually asked of me tbh

i have a script that calls a task in celery passing an object inherited from dict (sometask.delay(obj_inherited_dict)). when the task is going to be executed, obj_inherited_dict is a dict and not the original object i passed. what am i doing wrong?

the serializer is pickle for task and result

Hello. I had a couple of questions about Celery internals. I am trying to get the name of the current queue inside a bound task. What's the best way to accomplish this? I can get access to the routing key and the exchange, but for the life of me can't find a way to get the name of the queue.

i have a script that calls a task in celery passing an object inherited from dict (sometask.delay(obj_inherited_dict)). when the task is going to be executed, obj_inherited_dict is a dict and not the original object i passed. what am i doing wrong? the serializer is pickle for task and result and i'm using celery-4.0.2