#readthedocs

what's readthedocs-build?

so, readthedocs-build is meant to be an encapsulation of the build process

at least eventually

so you can configure and run, locally, the same processes that we run

which locally, would mostly be a wrapper around sphinx

that seems to be a requirement listed in pip.txt

RTD is meant to extend this so that we can ensure the same process is being used

so should I be looking to modify that one instead?

so far, it's mostly for configuration

right now, it's a two step process, rtd-build and RTD both need patches, as rtd-build mostly just does config and config validation

I can't get pandas to install on py3.3

any version

it always tries to install the latest numpy

:(

stupid scientific stack

pandas .19 looks like it doesn't do 3.3

yes but no version will work

they ALL want to download the latest numpy

even prerelease versions

let me try something

does pandas 0.17 work? looks to have a cp33 wheel

not sure what the pinned version of numpy is there though

ok I think I got it

I need to install numpy/scipy first, then pandas and matplotlib in a separate batch

pandas has a braindead setup.py it seems

do I even have to mention the fact that it installs nose as a run-time dependency?

whoever made this needs a spanking

agj: do you happen to know why it installs auxlib?

agronholm: believe that is a conda thing

conda requires it for the bootstrap to work...?

it'd be weird

huh.

agj: ok so I recommend modifying the code to do "virtualenv -ppython%s %s' % (interpreter_version, envname) or similar

actually

-p shouldn't be necessary

because it should already do "pyenv global %s' % interpreter_version

if you really want to support arbitrary minor versions, that will complicate things somewhat

as opposed to "%(version)s -mvirtualenv" ?

yeah

i believe dstufft may have recommended calling virtualenv in the fashion we are now

that's the venv module which is not available on older pythons

and on some pythons it installs a totally useless virtualenv

on the other hand you could install the same software on each and every python but what's the point?

i remember running into many issues with our set up and running the correct virtualenv/pip, but not finding traces of that atm

with pyenv you just set whatever version you want as the current and then off you go

i wouldn't want to make RTD aware of pyenv at all, it should just be using the binary name

which seems like it should be possible

it might take specifying all versions are available globally at the same time

ie, for local development, most of my development paths have pyenv local py27,py35 set

agj: if the developer chooses not to use virtualenv but specifies a python version, wouldn't you want that to be what "python" runs?

what is the appeal of virtualenvs anyway when you're already running in a container?

yeah, so it might take altering the commands run a bit

if we ever decide to drop non-docker use, there isn't much of a reason, really

i suppose it does make storing the python path outside of docker

that is, we mount a path in the container and virtualenv and the build process write to that path

so virtualenvs live between builds

might be difficult to replicate otherwise

I see

but if virtualenv is not used, is there any point in that?

I'm trying to think of a use case where virtualenv would be necessary

agj: you could cache ~/.cache instead

yeah, we're doing pip caching similarly

having a virtualenv already installed with deps does speed up builds

theoretically, we could cache all the non-virtualenv pythonpaths and propagate that between builds, but that would be a bit of work to get right

virtualenv does make this easier

you could just save the container after the build is done

and reuse it next time

this is certainly an option

i don't know if we're at the point we want to rely on docker yet

are you using other types of containers?

(virtualenv does not count)

not in prod, but the downstream of users that are using rtd might not be using any

currently, RTD is able to run without docker in the same way it does

i think for now, virtualenv isn't presenting any problems, so the benefit might be rather low

agj: http://paste.ofcode.org/7wAkBd7WeTks3bC4iBtmMF

I think I'm more or less done with this Dockerfile

you can handle any tweaks you need

strange comment on line 54

cool, starting a build of it now

better: http://paste.ofcode.org/8FrQs57Q6TKDEV3UEMPbXJ

(just moved the comment)

hm, might need to add m2crypto to the 2.7 pyenv

if you decide to keep the pyenv installed one

I would prefer to do it like this to prevent any accidental differences between the different py versions

aye, probably makes sense

oops, apt deps changed again, i know what i'm doing for the next 2h :)

yup

m2crypto probably isn't a big deal anyway

it will get installed if it's ever needed

not sure why that's special cased in the first place

yeah, not sure on some of that, there are some historical dependencies in there

oops, build image ate my disk and crashed my vm, and docker hub doesn't appreciate being sent an image that's 4.3G, so need to shuffle around some things on the image first

the larger apt-get step seems to be the 4.3G image