#amara

2) is there a good way for me to look at the webstats and see what API calls people have been using?

nicholasreville: ^^ this means that when we do the metadata code, it's just a delpoy no migrations needed and not much downtime

daughter is crying going to take her for a walk, but I'll be back soon

bendk: sounds good

all: back but internet is a little wonky

bendk: migrations finished but still crashed on the same duplicate key error

bendk: i think another migration wont hurt at this point

bendk: as for stats on the api i would have to collect them. we have analytics but they dont catch the api. i have to use the raw logs for that.

raw logs would actually be fine

nicholasreville: okay can we get terri on a call to try to figure out what to do?

ehazlett: what's a good way for me to check out that data?

I want to run some sql queries on it to better understand what the issue is

bendk: ill send you creds

bendk: sure

ehazlett: did you send the creds?

bendk: let us know whenever you're ready

okay, give me 5 more minutes

ready

bendk: sorry - notification fail - one sec

bendk: DM'g

deanjansen: so there are a few times where the data changes, but the only thing that changed was the team_id, from TED to TED translate

for 3 subtitle versions

is it still okay to drop that data?

ehazlett: bendk: how are we looking on migration stuff?

nicholasreville: bendk is looking at the database now to see the state of the DB

should be done in a few minutes

k

deanjansen: you around?

nicholasreville: got one part done, now wehave another issue

bendk: hey what's up?

should we only have 1 billing record per video language?

even if there are multiple versions on it?

bendk: yes, that's correct

ok, well that's another database constraint error

let me look into it more

bendk: did you just skype me?

yes

i'm here now

talking about billing again

oh fun

bendk: 0157 migrations still running...

nicholasreville: janeted ^ fyi... (on test)

ehazlett: how long has that been going for?

there's definitely a better way to implement that oen

one

bendk: nicholasreville 0157 migrations still running...

ehazlett: does that mean it's going well?

nicholasreville: it means it's going :)

but the fact that it's not erroring is a good thing

ehazlett: is migration 157 the last one?

157 - Fix widget-breaking CSS - https://unisubs.sifterapp.com/issue/157

janeted: for the videos app yes. not sure how many others

wow - ok

janeted: i'm guessing it's close

phew - how many hrs?

4.5 total without this one included (have to wait until it's finished to get the count)

and no other problems after the billing stuff, ehazlett?

janeted: so far no but we removed a bunch of data

bendk: done

bendk: nicholasreville total migration time looks like 7 hours

this last migration (videos 0157 took 160 minutes)

bendk: nicholasreville and the app at least starts and returns the index

ehazlett: wow that's long

I'm pretty sure I can re-write some of those migrations to work faster

but I'm not sure if it's a good idea

nicholasreville: what do you think?

wow, ok

so that's 7 hours of downtime? and how confident are we that it would work?

bendk: ehazlett ^

I can't say

and nothing we can do to add raw capacity?

So the other route would be that ehazlett tells me which migrations were on the slow side, then I rewrite them to make them faster

I think if I wrote 0157 using raw SQL, it wouldn't take very long at all

it's just a join and an update

I'm assuming the actions table has a lot of rows, but still

yeah not sure -- we could at least try

i don't want to migrate until i do this again with steps that work successfully out of the box

i really don't want to be manually fixing migrations during the downtime

bendk: nicholasreville ^

wellllllllllll, hmm

so the choice is:

1. do a 7 hour downtime

2. do some speedups, then do a full test of that, then do the migration

right?

ehazlett: bendk ^

the other side of that is that it worked because I stepped in in the middle and ran some DB queries

so I can cut-n-paste them into the migration and things *should* work the same, but that's slightly risky

so you are saying someone needs to be watching while it goes anyway, is that right?

oh i see

bendk: roughly how much faster would it be able to get?

i wouldn't hold you to an estimate, just ballpark

I would guess 10 minutes

if we had other migrations that were that long, then I think it would definitely be worth it to rewrite them

if it's only that one, then we have to way the time improvement with another round of testing

stepping away for a bit, i'll be back to check on the chat

nicholasreville: correct

ehazlett: when ben says 10 minutes that means it goes from 160 minutes to 10 minutes? so it saves almost 3 hours?

160 - Fix Add Another Translation and Ask Friend to Translate buttons on Finished dialog - https://unisubs.sifterapp.com/issue/160

that's what I think

nicholasreville: yeah if that works

will save 2.5 hours

bendk: well, ok, let's do that

worst case we have to wait until tomorrow, right?

but 7 hours is just too much downtime i think

ehazlett: I think we can change that python code to just to:

db.execute("UPDATE video_action va SET new_language_id = (SELECT new_language_id FROM videos_subtitlelanguage sl WHERE sl.id = va.subtitle_id)")

along those lines

pretty sure that will result in a huge speedup, but correct me if you think otherwise

ehazlett: do you know if there were other queries that took more than say, 30 minutes?

s/queries/migrations/

bendk: hm

bendk: yeah i think that 0127 was a while

bendk: other than that i didn't log that close

ehazlett: ping?

DarrenBridenbeck: hey

ehazlett: i know you've got a big night ahead of ya, but have you had a chance to take a look at that email?

DarrenBridenbeck: well i'm not migrating at this point (we don't have a reproducible successful migration plan yet)

DarrenBridenbeck: yeah i encoded it -- let me see if it's finished

ehazlett: you are awesome

all: taking off for a while ; will be aroun dlater