#dat

wa7son: epic hack: https://gist.github.com/mafintosh/2d2a7c8503b21...

ya’ll seen http://papaparse.com/?

todrobbins: wow that page is very .... interestingly marketed

todrobbins: also it is likely slower than npmjs.org/csv-parser b/c of this line https://github.com/mholt/PapaParse/blob/master/...

todrobbins: but they dont have benchmarks so i dunno

todrobbins: readAsText causes a binary -> utf8 decode which is really slow and can be avoided if your parser is binary

mafintosh: will you add me to csv-parser

ogd: done

ogd: welcome home!

:D

already rode a tandem bike to a coffee shop

back in my normal rhythm

me and wa7son where hacking at coffee shops until late last night so we're still going strong :)

epic streak :)

ogd: i have an idea for blob metadata replication i wanna show you. wanna do screenhero?

ya

mafintosh: sorry i only have 1 bar on my hotspot

ogd: np - so basically my idea is to add a sublevel like interface to level-dat

when you put some data into a subset it gets a `subset: "name-of-subset"` when replicating

so the idea is that you would do a subset for blobs and store all blob metadata there - that way it gets replicated through the changes feed (which is awesome) BUT it also allows you to a meta subset where you put the schema

that way if you update the schema it would also be replicated using the changes feed - then we can always guarantee that you'll have the latest schema

i can make an example if that's more clear maybe

what does " BUT it also allows you to a meta subset where you put the schema" mean

i think you accidentally a word

mafintosh: oh you probably meant to say 'have'

mafintosh: so we would have 3 subsets by default?

mafintosh: can this be used for easily adding abritrary user-defined secondary indexes too

yes i meant have (my brain forgets words all the time sry)

yes i guess we would have three indexes by defaults (default, blobs, meta)

but you can add as many as you want basically - level-dat doesn't care

mafintosh: so theres still only 1 changes feed

yes

mafintosh: seems good to me

mafintosh: so its basically a sublevel that has the versioning semantics and gets added to the global db changes feed