#celery

The great feature removal has begun :)

I think this should be the policy from now on, if there are too many unresolved issues related to a feature with no actionable plan or funding to fix: we remove it swiftly

The same with issues. No issue should stay open for years

the_drow, I'm here now but I guess you aren't. Why are you not conforming to my timezone ;)

asksol: are you still here?

If you kill canvases, we riot!

.oO(Now starting to call asksol the George RR Martin of software.)

TauPan: lol, canvases in no danger :)

;)

found a possible SQS bug. if you retry a task which isn't run on the default queue, the retry doesn't get sent to any queue - you have to explicitly set the queue name

anyone hit that?

asksol, ping?

jbkkd, Could be. SQS isn't one of the most well maintained brokers

the_drow: hey

PM :)

jbkkd: is this in master or 3.1?

RedisLabs officially agreed to support and maintain the Redis broker. https://github.com/celery/celery/issues/3274#is...

Join the discussion

Sorry to be negative, but I only see "we could" and "we're willing" here. I've learned to be careful when I see words like that.

let them fork and make a proper PR first

Yes, but it would be crazy to commit to it, I think it's a serious proposal

TauPan, asksol: I can dedicate a few hours weekly for it personally, but if this requires a more concentrated effort to begin with, of a few weeks full time like someone mentioned in the github discussion, we'll divert one of our python devs to it as an initial effort, since I'm tied up with the redis modules effort. but we are serious :)

dvirsky_: Thanks :) We welcome our new friends at Redis Labs

I will keep make try to figure out the list of important issues tomorrow, but now I have to get some sleep if I shall wake up before lunch

As we do use a redis broker, it's good to know that it will be maintained. I apologize for being a bit paranoid. Also I'm just a mere celery user. I Didn't mean to speak out of turn here.

urm-make try. definitely zzzz

TauPan: I don't think anyone had an any issue with the statement :)

asksol: in 3.1

jbkkd: SQS rewritten in master: async faster and better. It uses the "anon exchange" for routing by default so no routing table is required, and I think that may resolve your issue.

interesting. how well tested is the master? I'm launching a new project so willing to take a risk here

dvirsky_, That's great! I'll try to finally provide a hiredis implementation that works on (recent) PyPy implementations

That way when we'll expose the async sockets from hiredis we'll be able to provide them both for CPython and PyPY

asksol, Good night!

the_drow, if we're just doing some pubsub, it might not be worth using hiredis at all

not sure, I need to do a full context switch to it to have smarter opinions :)

Having stuff written in C is always better in CPython.

It's not always the case with PyPy though

exactly

I've been programming mostly in go and C in the past 1.5 years, all those problems seem like a relic of a very primitive past :) though I love python

Hehe

I'm in Ruby right now :(

Hi guys, I need some help regarding batch celery

if we are using a single queue and there are multiple tasks, then both the tasks will write to same queue, then at the worker end, the worker function reads the batched args, is it possible in any case that one function will read the batch of given size that might contain tasks of other function

any help is deeply appreciated

Are you talking about canvas?

@the_drow are you talking to me?

yeh

no, I'm using something like this

@celery.task( base=Batches, flush_every=batch_size, flush_interval=flush_interval, acks_late=True )

@celery.task(base=Batches, flush_every=batch_size, flush_interval=flush_interval, acks_late=True)

please paste somewhere else