#dat

this is how scientists share datasets right now http://www.ncbi.nlm.nih.gov/sra?linkname=biopro...

(in rnaseq land at least)

(ref https://monsterbashseq.wordpress.com/2016/04/07...)

ogd: nice find

"There have been several challenges so far in working with this public data set."

its via blahah

and of course theyre using binder :)

ogd: right now im thinking through refactoring the progress code in dat so they're streams

karissa: mafintosh has thoughts on streams for progress

ogd: and merging 'download' and 'join' functions, cause they're the same

ogd: mafintosh cool

ogd: mafintosh any good examples in the wild?

ogd: mafintosh or maybe not a stream but an event emitter?

i think hes a fan of static properties

like how it is now? sorry i dont follow

karissa: yea ee is the way to go for that imo

mafintosh: ok

karissa: with the progress at a property

*as

mafintosh: ah ok

i see

using streams you might run into memory issues

since all streams have to be drained or they'll buffer everything

that makes sense

tidying a dataset, makes me think of how we can help people automate this with data pipelines.. http://rpubs.com/dgrtwo/tidying-enron

karissa: oh ya have you read the "Tidy Data" paper by hadley wickham?

no i haven't

karissa: http://vita.had.co.nz/papers/tidy-data.pdf

ogd: afk for a bit, feel free to review/expand on o/

ogd: just pushed a commit

[dat] karissa pushed 1 new commit to rc2: https://git.io/vVX3g

dat/rc2 613744d karissa: add event emitters instead of return objects for stats

ogd: also made join/download the same function

maxogden/dat - add event emitters instead of return objects for stats - failed - https://travis-ci.org/maxogden/dat/builds/12157...

ogd: just sort of did a few cleanup things, too. i feel like the next step is to see if we can move all of the scanning/adding stuff into the daemon and get that stat output through the auto-daemon stream

ogd: so the cli will listen on the daemon's stream and those will be the stat objects

ogd: let me know if it sounds off base

ogd: bbl

karissa: +1 on merging download and join

mafintosh: getting on that call the other day with you was super helpful!