#dat

<feed metadata><fs metadata><fs metadata>...

where feed metadata describes what kind of feed this is etc. this might make upgrading the feed in the future easier

i'm gonna think about this a bit

feed metadata block could in theory contain things like 'author', 'publish-data' etc

publish-date i mean

version

mafintosh: makes sense to me

mafintosh: that will be directly translatable to a cloud storage dump of a dat too

karissa: yea i think its a good idea. wondering how to structure it for maximum forward comptibility

maybe something as simple as, https://gist.github.com/mafintosh/19af0df5d4521...

mafintosh: sounds good to me.

mafintosh: feels like http ;)

karissa: lol best part about http

there would only be one header though

so i guess its more like a head

ogd: curious to hear your thoughts on o/

it could also just be a json object

mafintosh: question regarding the current storage architecture

karissa: shoot

mafintosh: if each directory has its own leveldb, does that hinder hyperdrive deduplicating data downloads if i move the folder or download a similar version into a separate folder?

karissa: yep

karissa: the 'each dir has a db' is a tmp hack

mafintosh: is there any way we can prevent that bug we were trying to fix without introducing this bug?

mafintosh: do you know what might be the ideal fix?

karissa: not easily unfortunately

karissa: an high level rpc layer

mafintosh: ok. so that would fix two bugs :)

karissa: yea exactly

karissa: i'm thinking we can spec that out soon

mafintosh: k. i can create an issue for it on the repo

mafintosh: i want to start using the issue tracker, after we delete 95% of these :)

karissa: also the deduplicator isn't perfect yet as well.

karissa: so plenty of room for improvement

cool

karissa: +1 on issue

[dat] karissa opened issue #437: high-level rpc layer https://git.io/vgYrO

i actually made this a while a go

mafintosh: nice does that load balance?

pfraze: no its picks a master

mafintosh: just for failover

pfraze: yea exactly

cool

[dat] joehand opened issue #438: create subdirectories when non-existent path is specified https://git.io/vgYFi

mafintosh: karissa i had to partition and add the disk google gave us to the fstab but its working now

look at me im doing ops

ogd: lol yay

mafintosh: i like the header idea, we could use it for filesize

ogd: as proto or json?

mafintosh: oh i thought it was gonna be json in the proto bytes field

mafintosh: but im cool with json too, it wont be big

mafintosh: prob better to be human readable

mafintosh: especially since its the first thing

ogd: right. i would wanna reserve a type key in the json doc though

ogd: that had info about the feed type

mafintosh: oh ya thats fine

wheather or not its file data or tabular data etc

cool

ogd: thats our dat.json file basically

mafintosh: oh yea

mafintosh: having some issues with psy maybe you can help me with

mafintosh: can you log into the google server and do screen -x

mafintosh: ah nvm its this bug https://github.com/substack/psy/issues/11

mafintosh: seems psy is broken

hmmm

substack: i downgraded to psy@1.6.0 just now and it works as expected

substack: on 1.7.0 it was creating tons of duplicate processes of both psy and my process

substack: did the cli api change between those versions (e.g. am i using it wrong now) or do you think it could be a bug?

it's a bug

substack: ill see if i can write a test case

substack: and also check if it happens on mac too

substack: yep happens on mac

that would be great

ogd: was biking

substack: ok https://github.com/substack/psy/pull/13

nice

I'll see if I can whip up a solution

[dat] maxogden tagged v8.0.8 at b55dcb4: https://git.io/vgO3M