#dat

mafintosh: so now that hypercores are each their own feed rather than a set of feeds, how do you reckon hypercore-archiver's design will change?

karissa: mafintosh started some hyperdrive-http sleep stuff (https://github.com/mafintosh/hyperdrive-http/tr...). Haven't quite looked through it all but may be useful for the datproject.org sleep stuff.

jhand: ok cool. thanks. i looked at that but its all hyperdrive, the cool thing about hypercore-archiver is it's using one swarm and combines the feeds into one hypercore instance so i was hoping we could continue that

karissa: on hypercore-archiver we can still have the one main feed that tracks all the other feeds, storage will just be decoupled a bit.

oh ya ya

forgot about that magic

karissa: it is the same design as always

aka, they hve always been their own feed

merged to master, https://github.com/mafintosh/hyperdrive

gonna write some tests now

(scriptjs) @mafintosh good morning. Was hoping you could answer my question. I am looking for an appropriate method for collaboration on a dat. Perhaps an answer could make for a good faq since this is something I believe would want to use dat for.

@scriptjs hi!

(scriptjs) hey ya :)

dat is only single writer right now so that's the basic collab model

(scriptjs) Yes, I realize it is a single writer. Given that scenario it is not appropriate to share your key.

yea you should not share your private key unless you coordinate writes

like a classic master election system

there is multi writer stuff planned at some point

distributed multi writer is what i meant

(scriptjs) @mafintosh I have seen an couple of things over some time that could be a solution so was interested in their merit. The first is forking a dat so each collaborator maintains their own copy and then each writer listens to each others feed.

yea i like that one a lot

(scriptjs) @mafintosh I know bearker is doing a fork for example but there is no formal api for one that I am aware of

i like the fork idea cause it means no expansion to core dat for now

aka, the ecosystem can solve it :)

(scriptjs) @mafintosh Yes, that is what I am hoping to do but hopefully in a way that is just a package

that'd be really cool yea

@scriptjs i *just* merged hyperdrive 8 to master

and that has support for unix permissions

to answer your original question

(scriptjs) Sweet

(scriptjs) I was reading through it a bit last night

it is just implementing the node fs api

it has like 95% of it so far. aim to cover 100%

(scriptjs) @mafintosh I was a bit mixed up with my implementation of hypercore and hypercore-protocol since the changes

yea those changed quite a bit

(scriptjs) Trying to bring something up to date

but wont change again

aka, they are super stable now

might be some api tweaks here and there nothing major

*but notthing major

(scriptjs) great! so the first thing was the key on creation of the database before and now it appears null

you just need to wait for ready to fire now

i think i mention that in the readme

if not let me know :)

(scriptjs) Hehe, yeah I am just working through it. It was late last night and I started my changes

nice

@scriptjs new hypercore is *a lot* faster than the old one btw

up to 100x haha

(scriptjs) wow

(scriptjs) Thats pretty sweet

(scriptjs) Good work

(scriptjs) :)

you'll get gigabytes/s out of it now

under optimal conditions

aka, network will be your main bottleneck

(scriptjs) i like that a lot

(scriptjs) :)

(scriptjs) For me this collaboration thing is important for buiding things with the apis. There are many wonderful packages

ya

looking forward to see what you come up with!

(scriptjs) @mafintosh the first thing I want to try today is just exchanging messages between collaborators with the new hypercore and hypercore-protocol

scriptjs: if you do that please post a link to it because it's something i've been wondering about doing as well

(scriptjs) @aaaaaaaa____ Will see how far I get but I think some of this needs to be fleshed out so there is package or at least a solid approach that can be endorsed as being a credible way of doing this.

currently testing hyperdrive v8 with mafintosh - getting 20mb/s on a connection that's already doing 20mb/s on a direct HTTP download from s3

my jaw is dragging on the floor

(scriptjs) @blahblah woo hoo!

(scriptjs) is there a gist to test it

@scriptjs npm install -g dat-next-next

I don't even understand how hyperdrive can be doing the same speed encrypted as direct http from s3 is doing with 20 concurrent requests

disclamer, everything might break at anytime using dat-next-next

its my playground

(scriptjs) kk

blahah: I think the trick is to make network IO the bottleneck C:

yoshuawuyts: :)

I think it's more like mafintosh is one of the scientists running the simulated universe we're all in, and this is one of his tests of the boundaries of our perceived reality

in fact, this would explain the whole dat project

it's a way to harvest all the data collected from this universe simulation

blahah: nope, Dat is not involved in testing the fabric of reality. We're just a bunch of JS programmers building modules

you are just a bunch of JS programmers building modules

what's the latest best way to connect to a discovery swarm

still hyperdiscovery?

mafintosh: what does filter mode mean in dat-next-next?

btw, congrats :) \o/ and so on

blahah: ya, thats my next massive refactor target btw

TheLink: it means only download/share a specific file/folder from the dat

ok

(scriptjs) @mafintosh Can you advise why dat has no explicit fork api. Shoud it?

@scriptjs cause we have enough moving parts right now basically

once everything is super stable i might look into it

although i think once i get diffing across dats working that'll solve most of the use cases tvh

tbh

(scriptjs) @mafintosh Sure. Thanks. I know beaker makes use of this so was wondering why it was not official

mafintosh: https://gist.github.com/beingalink/ff726d451158... I'm doing it wrong :(

TheLink: lol i forgot to update the readme

cause i was in a hurry to show blahah cool demos