#hypothes.is

<wyan> Morning. I'll be offline part of the morning to do a couple errands.

<robertknight> Morning :slightly_smiling_face:

<robertknight> @seanh: When you're around, is https://github.com/hypothesis/h/pull/3525 still WIP? Your last comment on there suggests that it shouldn't be any more

<seanh> robertknight: Not here this morning but I think that is still WIP at least until I discuss it with nick. Even though it seems to run "fast" (not really) on my laptop, it may be best to figure out a way to batch the queries still

<sheetaluk> Hey @robertknight are you around?

<robertknight> @sheetaluk: Yes, busyish but around

<sheetaluk> Good morning :)

<robertknight> Morning :slightly_smiling_face:

<chdorner> hello everybody :wave: just heading out to run some errands and lunch, back in a bit

<sheetaluk> Hello!

<wyan> @robertknight: for now what I'm going for is to add the direct link to the metadata returned by `extractDocumentMetadata()`, that way I can use it when creating the link. I was wondering how to create the direct links while running locally…

<wyan> should they be generated every time I save the annotation?

<robertknight> @wyan: The links are generated on-demand by the H service if the bouncer URL is configured

<robertknight> See `links.py`

<wyan> oh I see. So if I build the dev extension with bouncer support, I should be getting DLs when I save an annotation.

<robertknight> No, its the Hypothesis _service_ which generates the direct links, not the extension

<robertknight> Setting the `BOUNCER_URL` env var before running the H service should do the trick, I think.

<wyan> :+1:

<robertknight> (See `config.py`)

<robertknight> Does the Hypothesis `/api` route as described at http://h.readthedocs.io/en/latest/api/ adhere to any particular standard for hypermedia links?

<slackbot> Reminder: we have a demo coming up! maybe you can demo someone else’s work.

<seanh> Afternoon all

<nick> :wave:

<lenazun> Morning :sunny:

<nick> uhhh, is anybody else's Chrome extension v0.30.0 crashing, hard on local PDFs?

Action: robertknight tests

<nick> I'm trying to extract the error messages from Chrome, but it seems Chrome's "Collect errors" is broken, too

<robertknight> A local build of the extension from recent master works. Just trying prod 0.30.0 now

<robertknight> Prod v0.30 build works

<nick> It's crashing out hard for me in an empty profile.

<robertknight> I first tried without 'Allow access to local file URLs' checked and got the expected helpful error page, then tried again

<robertknight> Will try an empty profile

<nick> I'm using 53.0.2774.3 dev, so it might be Chrome that's broken.

<robertknight> Or rather, I'll do that just after the standup call

<nick> I wish I could get at the flipping traceback...

<robertknight> @nick: Can you reproduce with a local extension build?

<robertknight> I've never seen that balloon before. Perhaps it means that the process in which the extension is running crashed? So not a JS error.

<robertknight> Yep, had a look for references to the above message in the Chromium source and it gets called when the process hosting the background page crashes - https://cs.chromium.org/chromium/src/chrome/bro...

<robertknight> ( I absolutely <3 stuff like https://cs.chromium.org>; and <https://dxr.mozilla.org )

<robertknight> So yep it is a Chrome bug

<nick> @robertknight: ok, I'll work on the assumption it's bleeding-edge Chrome that's the problem for now ;)

<wyan> @robertknight: when you have a minute, I'd appreciate your input on https://github.com/hypothesis/h/pull/3550

<robertknight> @wyan: Just having a read through

<robertknight> Given that `extractDocumentMetadata()` is only used in one place and the result is then immediately passed to `documentDomain()` and `documentTitle()` I think it would be clearer if all the functions were just merged into one that took an annotation as input and returned an object containing the title text, title link and domain text.

<lenazun> @here reminder that this upcoming sprint is only 1 week long. we’ll start the next one on July 5th to kick off our (still unnamed) project!

<chdorner> @seanh and @nick: can we talk about the data integrity of our document data tomorrow? I'm currently writing the data migration for the new httpx uri normalisation and running into unique constraint errors for both document uris and meta. Sean already found a bug and worked on a migration for potentially duplicated document_uri data (which might fix my document_uri issue), but there's also weird things happenin

<nick> @chdorner: :+1: definitely

<nick> @chdorner: FWIW I'll probably be in the office tomorrow if you fancy getting out of the house

<chdorner> yeah, would be good to get out the house after last week. see you there then

<nick> DDog is going to the vet with Lucy for the day :(

<chdorner> uhm, I might have to rethink my plan then ;)

<nick> ...harsh

<chdorner> I'll be there don't worry :slightly_smiling_face:

<chdorner> alright, I'm off. have a nice evening everybody :wave:

<seanh> Yeah I'd like to discuss that

<seanh> Some frontend tests failing on master for me

<lenazun> @seanh: is it the time tests, by any chance?

<seanh> No, it's `selections-test.j`. Four failing tests to do with mouse ups and when it should emit selections

<seanh> Are these failing for anyone else? They're passing on Travis apparently

<robertknight> @seanh: Console output?

<robertknight> Also, do those tests fail when run exclusively on their own or only when run in the context of the whole suite? You can run them on their own with "gulp test-watch-app --grep selections'

<seanh> robertknight: console output ^

<seanh> They do also fail when run on their own

<seanh> I take it these aren't failing for anyone else

<seanh> I wonder if this is because I've got PhantomJS 1.9.8?

<seanh> Hmm, don't think so

<seanh> nick: How long is it acceptable to lock the db tables for at a time? Timing it in my laptop. What should I be aiming for? Also, I take it's okay if instead of locking the db for 30s at once I lock it for, say, 1 second at a time 30 times in a row? i.e. I'm just breaking it up into separate transactions, not actually making it faster overall

<seanh> All of the time seems to be in the "change nulls to empty strings" bit, which I guess is the only bit that actually inserts new data (as opposed to deleting)

<nick> Making it faster overall isn't the goal. Being cooperative with requests from the web application is the goal.

<nick> And don't forget that it's only the update query itself that's going to take a lock on the table. So doing stuff in Python or doing selects is fine.

<nick> Ideally you'd choose your chunk size so you lock for less than 100ms at a time.

<nick> 1s is still a long time for a web request to be blocked.

<seanh> Wait

<seanh> Only update locks, select and delete don't?

<seanh> Ok, 100ms

<seanh> The point being that other web app transactions can be slotted in between my transactions, I assume

<nick> Here's the Postgres documentation on locking: https://www.postgresql.org/docs/9.4/static/expl...

<nick> Table 13-2 is what you want to look at.

<nick> And the descriptions above it.

<nick> SELECT takes a different kind of lock to UPDATE and DELETE.

<nick> And yes, the point is that when your transaction commits, any blocked queries in web application transactions can then execute.

<seanh> Ok, this is interesting stuff