#techempower-fwbm

TE-msmith: alex-techempower: gh#932

Adding some documentation: https://github.com/TechEmpower/FrameworkBenchma...

Fantastic! We really appreciate your work on this. Thank you.

yea...doing docs is annoyingly harder than just writing code :-P

haha

agreed

anywhoo it's done and should help people conform as things go forward

definitely

that central readme is getting huge. Needs to be split off at some point

probably. it is nice to have it all in one file though. in the past people got confused about where to find instructions.

yea, without a frameworks/ directory in the root it's unclear where you would split it off into

so for now one biggie seems best

but would be good to eventually have a readme for running the system, and a readme for adding a framework

Yes, that seems like a sensible split.

hamiltont: awesome

we could probably have a table of contents at least

to make it easier to digest

additionally, it would be helpful to see how to install only what is required for a given test

hamiltont: was this intentional? https://github.com/TechEmpower/FrameworkBenchma...

there's an onion install in the installs directory

s/install/setup

TE-msmith: https://github.com/TechEmpower/FrameworkBenchma... describes how to install software for only one test

Under the "Using Install server flag" section

alex-techempower: Yes - the originall installer.py has a section to install onion, so I just made it a dependency that others could depend on

hamiltont: ah, well, the onion framework didn't depend on onion

if you wish, you could just move everything from onion.sh to onion/install.sh

that was the main issue

I fixed htat

oh ha ha ok

I figured there would be a few where fw_depends was wrong

nothing too bad

also, thanks for the documentation

!m hamiltont

You're doing good work, hamiltont!

No worries ;-)

if you have time in the remainder of today, feedback on gh#920 would be useful

Use python logging framework consistently: https://github.com/TechEmpower/FrameworkBenchma...

is there any reason you wanted it today specifically? We're going to be processing PRs throughout the next week

no reason

I'm just building stuff based off it, so if there are issues with the approach I"d like to know before I go too far afield

hamiltont: I'll admit that I'm a little bit gunshy about merging it right now because it makes a pretty big change to the python files - I was thinking I may just postpone it to round 11

What are you building off of it?

oh yea, it should probably be delayed until after 10

Sorry, thought you had already drawn the line for that

We've drawn the line for accepting pull requests, but we're still merging existing PRs

If you still have 10 stuff to do then just ignore 920 for now

ah ok

yea nevermind then

I'm interested to hear what you're building off of it

though

Sure - I'm refactoring how the setup.py module is loaded. Currently I have a lovely implementation of some shell utility functions

For example, if you use this "new" approach to access the shell from setup.py, you get soemthing like this: https://github.com/hamiltont/FrameworkBenchmark...

and once I've committed the latest changes, the sh_async function is going to be slickkk - it will automatically watch for output from the asynchronous process and send that output to the console and to the log file

nice

Same for using the "sh" function - output is automatically sent to both the log file and to the python logger. If you use the default log level then you see no extra output on the console, but if you bump up to --log DEBUG then your console shows you stuff like "go/setup.py: go run: Launching something"

e.g. in my latest stuff, it can show you exactly what commands in what files caused what outpu

It's really frigging great for debugging stuff

that's definitely not in the scope for round 10 though, but I think that'll be really really nice for round 11 and for helping people build their own tests

yea. I've also got a branch that's working out travis-ci integration

I've got it roughly "working", but it needs some more tweaks to be finalized

See https://travis-ci.org/hamiltont/FrameworkBenchm... for a small example

It basically verifies what tests are working

holy cow

that's amazing

Yea...I was really tired of making big changes and not knowing what worked/didnt

aren't there limitations for travis? what if we need something like racket that takes forever to build?

Yea, I thought that would be a problem. So far it hasn't, but if I run into it I'll just disable using travis on those tests

Their "limits" are quite generous though TBH

90 minutes per test verification

yea

Oh, I added this too, but it's still a bit of a shoddy setup

interesting

I know that brianhauer and TE-msmith will be very happy about the travis-ci thing

Yea, once it's fully working it will be a darn lifesaver.

travis would be amazing - it would really reduce our workload

and allow us to get through PRs faster

At some point we should probably add a "toolset/run-travis-tests.py" though that attempts to only verify frameworks that have had code changes

But that's future stuff at this point

Their API is quite good though, so it should be easy to be a bit more polite w.r.t their freely-provided workers

well hamiltont, I'll tell you what, I've got until thursday to merge PRs, and it looks like I'll be done long before then. If I finish, I'll take a crack at gh#920

Use python logging framework consistently: https://github.com/TechEmpower/FrameworkBenchma...

no promises though

sounds good, it's nbd if you don't have time

I have only tested it with the go framework, so perhaps it's actually smarter to wait until R11 in case there are unexpected edge cases

It's modifying the arguments being sent to setup.py files

So it needs a lot of testing

all right

hey hamiltont, mind if I pick your brain for a minute?

I'm trying to merge https://github.com/TechEmpower/FrameworkBenchma..., and the setup script is failing

though when I run the commands manually, they seem to work just fine

here's the log: https://gist.github.com/aschneider-techempower/...

here's a slightly more descriptive log: https://gist.github.com/aschneider-techempower/...

why would that error occur in the script, but not when installing manually?

alex-techempower: The top of your script reports saving "leda-0.5.1.tar.gz.1"

Is that what you were trying to download (e.g. wget http://sergeyzavadski.github.io/leda/releases/l...)

hamiltont: I'm not sure

the guy posted an update

Perhpas add the -O option to wget to ensure you're putting your downloaded tar.gz in the right place

I'm going to try it

you may have an older version being untarred (instead of the newer one)

I can give it a shot

that's all i see right off, I'm going offline for a while but I'll look more lat4er if i have time

thanks

I'll tag you on the issue if there's an issue