#techempower-fwbm

!m jyenTE

You're doing good work, jyenTE!

;)

oh no... you're documenting my text!?

NSA!!!

DID SOMEONE CALL MY NAME!>??

oh thanks!

!seen alex-techempower

hmm

[d__d]: seen alex-techempower

Yes, I saw alex-techempower just now.

alex-techempower said: "[d__d]: seen alex-techempower"

[d__d]: seen NSA

Sorry, I haven't seen NSA.

hmm...

Do you guys see this in rich technicolor?!

brb - You don't want to know.

ha ha

my client has this huge script collection I never saw before

Bender: If it ain't black and white, peck, scratch and bite.

ƃuıllɐɟ sı ʎʞs ǝɥʇ

haha

[FrameworkBenchmarks] hamiltont opened pull request #948: Adding Header utility class (master...header) http://git.io/98TaLw

dude baller

travis picked it up

It's in the build queue

nice

apparently when it's done building it, travis will update the PR with build status too

although like all of our statuses will probably be something like "build cancelled" :-/

hamiltont: I'm seeing a lot of red :(

also, why is flask installing java?

interesting, this job ran for exactly 10k lines of log output: https://travis-ci.org/TechEmpower/FrameworkBenc...

I wonder if travis kills a job if it has too many logs

Yea. I honestly believe that's how bad it is at the moment. Some of that is, I believe, the fault of gh#915 not being tested fully. I should not have set the default to pertest

Separate test installation directories and environments: https://github.com/TechEmpower/FrameworkBenchma...

yea :(

The second major reason things are failing is that they hardcoded the project root to ~/FrameworkBenchmarks in their setup.py files

Realistically, these are both the same problem

ok, we'll take the travis results with a grain of salt

Tests should never ever hardcode, but we need to go through each framework and ensure that's the case

If you would like to switch back from pertest to unified as the default I'm in favor of that

I think it makes more sense as a default option anyway

It was a mistake to set pertest as default...the only people that need pertest would be the kind of people that would read all the options quite a few times anyways

Until we get travis to 100%, it's not fully useful

yea

how would I wind up doing that?

getting to 100%?

no, changing it from unified to pertest

is that in the travis.yml?

it's in run-tests.py I believe

run-ci.py just uses the default

why is that default?

why is pertest default?

that's what gh#915 set it to

Separate test installation directories and environments: https://github.com/TechEmpower/FrameworkBenchma...

When it introduced --install-strategy

I chose a default of pertest

[FrameworkBenchmarks] aschneider-techempower pushed 1 new commit to master: http://git.io/4A3ibw

FrameworkBenchmarks/master ab74843 Alexander Schneider: [disable ci] Update default install strategy

Perfect, thanks

wait

no

why is it defaulting to unified now?

that seems wrnog

just like that spelling

TE-msmith: pertest was breaking travis

why?

and besides, it increases the installation process length tremendously

you can still install individual tests without the entire thing

but it won't install the entire thing for each individual test

Most people probably want unified

i'm confused

isn't unified just every pertest?

Look at what --install-strategy does

no

look at the wonderful documentation that hamiltont wrote!

I wrote docs?

oh sweet

yay docs

hamiltont: you don't remember?

not really

hah

I do now

just forgot a minute ago

anyways

don't think flask is install java

documentation!/

someone calls sudo apt-get upgrade somewhere I think

flask is upgrading the java that's already installed

ah

okay... i disagree with this implementation

I think that we want to guarantee isolation

I think you do when you're actually running the round

i agree that having to download/install these things multiple times is a source of pain, but I thought we had a plan in mind where we would have a local proxy to combat that

Very much agree - but you're not loosing the pertest option

It's just not the default

if you want it, just use --install-strategy pertest right?

hmm...

although that would then break every framework that currently has hardcoded their installation root

which, as we can see from travis, is a lot

TE-msmith: that's the main issue

So sooner or later if you want to use pertest, we will have to fix these projects

(frameworks)

I think that ultimately, I want to completely do away with unified in favor of pertest

i don't even think unified should be an option moving forward

TE-msmith: I can get matt on that once he's set up

the use-case I'm aimed at with this is people who want to contribute a new framework test from scratch - they should be instructed to do VERY little (ease of entry is hard preesntly)

if you like?

maybe

That's not a big deal to me, I always use pertest anyway. If that's what your team wants, feel free to just delete the --install-strategy option entirely

question - if this is a blocker on travis, is it because unified runs across multiple tests?

not at all

i.e. php-test1 and php-test2 are run separately, but they are referencing the same php?

No no

not at all

okay

what's the issue

they all run on different VMs

hardcoded installation roots?

Yup

Exactly

let me find an example if I can

can you link me an example?

i'm clearly just missing it

look at django's setup.py

line 7 https://github.com/TechEmpower/FrameworkBenchma...

This hard codes the installation directory

a lot of frameworks are doing this

because the variables didn't exist before gh#920 was merged in

Use python logging framework consistently: https://github.com/TechEmpower/FrameworkBenchma...

err..

sorry, wrong pull

the one with FWROOT, IROOT, etc

The hard-coded installation path would be correct if you ran this test with --install-stragey unified

but it would be wrong if you ran with --install-strategy pertest

TE-msmith: I started matt on fixing that up

I'll probably bring him in here once the install is done

alex-techempower: can I ask what you asked him to do?

I've been going through setup.py files one by one and fixing

don't want to duplicate work

how do you fix it?

hamiltont: oh, cool

does one*

yea, that's what I was planning on asking him to do

but if you're already doing it, we can do something else

Help would be amazing actually