#hypothes.is

robertknight: I published v4 of dom-anchor-text-position with your changes

I decided that since it only affects fromRange() and not toRange() the performance hit is not likely to be such a problem, and definitely not for Hypothesis.

i.e. it won't affect anchoring speed on page load

I'll try to get dom-anchor-fragment out this week just to complete the symmetry of API changes. Thanks for your help!

<robertknight> Thanks @tilgovi - I'll integrate those changes with Hypothesis later today or next week.

<robertknight> > I decided that since it only affects fromRange() and not toRange() the performance hit is not likely to be such a problem, and definitely not for Hypothesis. Good point.

<robertknight> Morning all :sun_with_face: - I'm going to be out for much of the morning, but I'll be getting on with a client release later.

<slackbot> Reminder: Standup in seven minutes! Grab a cup of tea and head over to https://hangouts.google.com/hangouts/_/hypothes...

@hslack how can I check it?

<slackbot> Reminder: Standup time! https://hangouts.google.com/hangouts/_/hypothes...

<robertknight> @meysam - ElasticSearch can be queried via HTTP. IIRC using the default configuration it will be exposed on port 9200, so you can do:

<robertknight> ``` curl 'http://localhost:9200/_search' ```

<robertknight> When I run this command, I'll see the annotation "documents" that are stored in the search index. These are in JSON format and look somewhat like the data returned by Hypothesis' search API

@hsalck : it returns a JSON :{"took":10,"timed_out":false,"_shards":{"total":15,"successful":15,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]}}

<robertknight> @meysam - Ah indeed, that suggests that the annotations never got indexed in the first place.

@hslack : so whats the solution? whats wrong in my build?

<robertknight> Can you paste the output you see when the dev server starts up? ie. When you run `make dev`?

<robertknight> Also, can you get the output of http://localhost:9200/_mapping , post it somewhere (eg. a Gist) and then paste a link to that here?

<robertknight> `/_mapping` returns the schema for the search index

<robertknight> Hmm... ``` 18:38:08 worker | /home/meysam/h/.venv/local/lib/python2.7/site-packages/celery/platforms.py:812: RuntimeWarning: You are running the worker with superuser privileges, ``` You're not running `sudo make dev` are you?

<robertknight> I can't immediately see that it would cause the problem, but the dev server should be run with normal user rather than superuser priviledges

no I just make dev, but with root privileges (sud su)

<robertknight> When you create an annotation using the client, do you see any log output appear in the terminal where the dev server is running?

no, nothing

without sudo su (root privileges) celery returns permission error : 18:58:52 beat | celery.platforms.LockFailed: [Errno 13] Permission denied: '/home/meysam/h/celerybeat.pid'

<robertknight> That's because you ran `make dev` _with_ root privileges in the past, so there will be a number of files floating around that can only be read/written by root.

<robertknight> Just remove those

just fixed permission issue. but still no success of fetching annotation

recently I see this error in server terminal log: 19:03:40 web | /home/meysam/h/.venv/local/lib/python2.7/site-packages/sqlalchemy/sql/sqltypes.py:219: SAWarning: Unicode type received non-unicode bind param value '__world__'. (this warning may be suppressed after 10 occurrences)

is it relevant?

<robertknight> I have seen that myself, so I don't think it is a problem

<robertknight> There is a command to rebuild the search index: ``` ./bin/hypothesis --dev search reindex ```

<robertknight> Running that locally however I see `RuntimeError: cannot reindex if current index is not aliased`. I think that may be because I created by local search index a long time ago before some recent work was done on reindexing.

As you said it got an error :raise RuntimeError("We don't support managing aliases that "

RuntimeError: We don't support managing aliases that point to multiple indices at the moment!

@hslack : it's now more complicated!

<robertknight> What is the output of `curl http://localhost:9200/`?

<robertknight> Okay, that's somewhat newer than the version of ES I'm using (1.4.4)

it's based on documents docker link

<robertknight> Right, something to check is whether the task that is actually supposed to index the documents gets run when an annotation is added. There might be a quicker way to do this, but one way is: 1. Kill the dev server 2. Apply the change below 3. Run the dev server and once it has started up, try adding another annotation

<robertknight> ```diff diff --git a/h/tasks/indexer.py b/h/tasks/indexer.py index 1dd2e3f..5dd6f91 100644 --- a/h/tasks/indexer.py +++ b/h/tasks/indexer.py @@ -10,6 +10,7 @@ from memex.search.index import delete @celery.task def add_annotation(id_): + print 'Indexing annotation {}'.format(id_) annotation = storage.fetch_annotation(celery.request.db, id_) if annotation: index(celery.request.es, a

<robertknight> When you add or update an annotation, you should see `Indexing annotation ID` printed out in the dev server's output

would you please paste it as snipet? it got syntax error

<robertknight> It is a diff. Add the line `print 'Indexing annotation {}'.format(id_)` underneath `def add_annotation` in `h/tasks/indexer.py`

<robertknight> @tilgovi - `dom-anchor-text-quote` still depends on dom-anchor-text-position v3. Since it uses d-a-t-p's fromRange function, it will need to be updated as well to function in IE/Edge

<robertknight> (Thanks for publishing the d-a-t-p changes btw!)

@hslack : it returns : [2017-01-05 19:48:49,954: WARNING/Worker-1] Indexing annotation pECGWNNiEeaq6vcycccnJA

while add a new annotation

<robertknight> OK, and if you `curl 'http://localhost:9200/_search' ` that still returns no results?

again : {"took":4,"timed_out":false,"_shards":{"total":15,"successful":15,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]}}

<robertknight> OK, I'm going to take a few minutes to get ES updated to the current version linked from the documentation.

<robertknight> @dwhly asked: > I’m noticing that the intended behavior where we switch the tab view to Page Notes by default when there are no annotations isn’t live. Is that already recorded as an issue somewhere? I can't remember where the discussion around that got to. I don't suppose your memory about this is better than mine @sheetaluk ?

<dwhly> Oh, here: https://github.com/hypothesis/client/issues/53

<sheetaluk> Hmmm, let me think

<sheetaluk> maybe the same link that dan just posted

<robertknight> OK, I'm going to get back to where I was above for the moment, since it looks like we've got an existing issue.

<sheetaluk> k

<dwhly> This one might be nice to tackle prior to the launch of tabs ^

<dwhly> At least the headline on the issue.

<robertknight> We've already launched tabs. It is just the Orphans tab that is not yet released to all.

<dwhly> Ah, right. I forget what normal folks can see sometimes.

<sheetaluk> @dwhly if could add a comment against that ticket in reply to the scenarios I described in my comment that would be super

<dwhly> Just doing so! :slightly_smiling_face:

<robertknight> @meysam - So I've just removed my ES index and recreated it using the latest ES Docker image and I was able to successfully run the reindex command.

<robertknight> So, I think you should be able to try the same steps:

<robertknight> 1. `docker ps` and copy the container ID for the elasticsearch container

<robertknight> 2. Run `docker stop ES container ID>` followed by `docker rm <ES container ID`

<robertknight> 3. Create a new ES container: `docker run -d --name elasticsearch -p 9200:9200 -p 9300:9300 nickstenning/elasticsearch-icu`

<robertknight> 4. Reindex annotations: `./bin/hypothesis --dev search reindex`

<robertknight> (And again, all of this as your normal user, not root)

<robertknight> My highly non-specific diagnosis is that something got screwed up with the configuration of your ES container.

@hslack : it just works finally. cheers up! thank you very much. as I understand the problem was about being in root user. when dockers and make commands are as normal user everything works perfect and also reindexing command makes it work!

<robertknight> You're welcome :slightly_smiling_face:

@hslack : its now running in localhost. would you please guide me how to configure it to be accessible remotely (probable with nginx)

<robertknight> The minimal step to make the dev server accessible remotely is to change the `host` setting in `conf/development-app.ini` from "localhost" to 0.0.0.0 .

<robertknight> The nginx configuration that the public service at https://hypothes.is uses is not in a public repo, but I'm guessing it probably isn't that hard to set up nginx in front of the dev server as a starting point.

<robertknight> See the `Dockerfile` for the commands that are used to launch the web server in production.