#amara

read and cry

i guess it's time write celery for humans

kennethreitz style

I think that's RQ

sjl: yep

i still think amqp > python pickle

pickle is a serialization format, amqp is a messaging format...

honza: i thought celery still pickled to rabbit

yeah

yeah

what

pickle is like JSON or EDN

amqp is like the Redis protocol

hm

or 0mq protocol

you need one of each

right

RQ uses pickle + redis

celery uses pickle + (whatever protocol your backend uses)

sjl: plus shit

i just feel like amqp is more of a standard and other tools play nicely with it --- but the same is probably true of the redis protocol these days

ok, you convinced me

honza: the original designer of AMQP has called it "fatally flawed"

er

"fundamentally flawed and unfixable"

same thing

welp

heh

wtf

ok, since we're talking... what about rabbitmq vs redis

for pubsub

i don't think rabbit does that

also, all of this shit is why celery's guts are so complex and buggy: http://docs.celeryproject.org/en/latest/usergui...

ehazlett: it can

er wait yeah i think you can use fanout

yeah

sjl: yeah

honza: redis is great until the single server you're running it on breaks

exactly

honza: it's music easier to write something that works on one machine like redis than to write something that works as a cluster like rabbitmq

rabbitmq seems to be more resilient or something

*much

I can't fucking type

heh

right

rabbit is designed to work in a distributed fashion from the start, and it supports a lot more than the basic pub/sub stuff redis provides

zeromq is another option, though much more mind-bendy

yeah

i always thought zeromq was a nice abstraction and you could put whatever you wanted under it

but i'm not even a real programme so what do i know

honestly if Python had something like Clojure's (future) and used mostly immutable data structures I'd just use that for celery-like tasks

sjl: +10

shoving tasks over the network is too much pain for too little benefit

all: back

serialize, send over a wire, sit in a queue, deserialize -- in exchange you get... to maintain more things?

just let the app servers do the work. if they start getting too busy, spin up more of them.

but since Python threading is a joke that's not necessarily a good option

plus you have to have a copy of the entire repo on the worker node

sjl - what is the current state of the staging branch?

I'm only asking because the entire webdriver_testing directory seems to be MIA

janeted: staging is now in the "before the big DMR merge" state

janeted: did you get the emails I sent out? I'm not sure where qa@univseralsubtitles.org goes

i did - just wasn't clear how that would affect things

and so when you do the big merge, then all the changes will be back?

janeted: basically all the main DMR code is on dev

if you need to do anything related to the DMR it should go on dev

ok - was just feeling a little scared

and I'll handle making sure dev gets into staging at the appropriate time

got it - thanks

all: django 1.5 is out https://www.djangoproject.com/weblog/2013/feb/2...

aww sheeit

nice

we could ditch our CustomUser crap

yeah

score

or rather, plug it in properly instead of the hacked in way we do it now

you mean the hacked way in which every django app on earth does it?

hehe

specifically userprofiles was always a mess

and felt very second-rate

yep

that was a thing that i always complained

oh neat, the verbatim template tag is in core now

wat? you want more than a name and an email for your users?! nevar!

ooo

that's hot

ehazlett: staging is still running pre-tern migrations, btw, so far it looks good

sjl: awesome

about 30% done with them

I'm getting good at this process with all the practice, lol

probably a good thing

all: anyone deploying to prod in the next hour or so?

arthurdebert: not

arthurdebert: not me

arthurdebert: not me

arthurdebert: i

tks folks

niiiiice

timezone conversions built-int

AST : 15:54:12 AST-0400

Sao Paulo : 16:54:12 BRT-0300

EST : 14:54:12 EST-0500

UTC : 19:54:12 UTC+0000

PST : 11:54:12 PST-0800

that's awesome

lol

nicksergeant: that is awesome

sjl: a subtitles.pipeline question for you: do you have any thoughts on how the visibility attribute should be inherited? right now it seems to ignore previous states

sjl: ie when you add a new sv, it doesn't look at the parent, it just passes the attribute down the chain

sjl: should that just be entirely on the caller of add_subtitles()?

honza won't that make things more opaque, as in the caller should know what is doing

i'm just thinking out loud

honza yeah me too,

honza: yes, the caller is responsible for deciding what to do with visibility

k

Task complete: demo:amara,prod-test proxy_user:arthur deploy

sjl is there a ballpark for when staging will be on DRM code? I'm just trying to coordinate with partners when they'll be able to do work on staging again

sjl as in "later today" "tomorrow" or "later this week"

arthurdebert: however long it takes the tern migrations to run, which I'll hopefully be able to start before I have to leave for my bass lesson. usually takes 3-4 days for those I think, so Monday is probably a safe bet?

sjl k, that sounds like a plan

honza the last production error on the api logger is from jan 3rd . does this sound right to you? https://sentry.amara.org/amaraproduction/?p=2&a...

arthurdebert: seems suspicious

honza hm… udacity is reporting getting 500 errors on a video resource request, can't find anything on sentry

arthurdebert: it's probably on the root logger

honza can't find it there neither

arthurdebert: hmmm

arthurdebert: is it happening right now? sometimes those messages are late coming in

honza it happend a couple of hours ago

arthurdebert: getting timeouts on sentry, ugh

honza arg

honza: yeah large queries result in timeouts

software is hard

honza software is too brittle

honza: wow that sounded mean

honza: sorry -- i didn't mean it that way

:)

honza: i just meant sentry is a pita sometimes

ehazlett: huh? i don't get it :()

:)

honza: ah sorry - i re-read it and thought it sounded mean :)

honza: disregard then