#dat

I'm very ok with an answer that says "this month, we thing X, Y and Z, will become the best practice, but they still need a, b and c" - that would give me a reduced set to work with, and help out on

for my usage, i'm expecting to need my own registry (i think this is a yellow pages, and doesn't store or serve data), a multi-dat server, with some http UI, and then individual dat containers to sync data

I'm unsure how best to manage the secrets for updating dats, so i'm thinking to just use a private git repo for now - but i kinda like the idea of a dat that contains all that info

for eg, https://datbase.org/ doesn't help _me_ understand where that site / service fits into the dat landscape

and at a later point, the docs tell me to run a dat server, I should use hypercored - https://docs.datproject.org/server

mmm, that same page talks about a registry, but i'm failing to see it then tell me how i can run my own - or where that code is

oh, hypercloud, hashbase, dathttpd - more maybes !

mmm, or not - they don't do `dat publish` ?

SvenDowideit: there are lots of options. i've been using hypercored for most of my archiving needs

yeah - i see there are lots of options - the trouble is i keep failing to see info on what the similarities / differences in the options are

it's still pretty early days for many of these projects... the documentation will come - there is an active project to improve the documentation / specification of the most important parts

and given the "know-nothing" state i'm at, that makes it hard to evaluate where to start

does hypercored do `dat publish` ?

i kinda would like a workflow that i can show off to the data producers here, what won't require me to edit a cfg file every time they make a new one

mmm, i could make the feeds file a dat, and give everyone the secrets needed to update the feed?

i've never used the public registry stuff

seems like a killer feature of the `dat` cmdline tool

for the 'pixel pusher' project i worker on, we modified hypercored so that clients could submit feeds and it archives everything it receives

"submit" as in using a webUI?

in the case of pixelpusherd, we made a dummy hypercore that joined the swarm for discovery ... then we can give the clients the key for that. when they connect, they add a little 'user data' to the connection in which we pass to the daemon the feed we want to archive. it's sort of dumb, but it works

so very kinda like making the feeds file a dat that everyone can append to

the only real problem is that you don't want to leak the 'key' of the pixelpusherd onto the internet, because then anybody could store anything in it. it's supposed to be something of a secret

yup

but it could probably work pretty well 'behind the firewall', or via an authenticated discovery mechanism

last year, i made a hacked up version of hypercored that would get it's feed from a hyperdrive it subscribes too. it works okayish for keeping a fleet of hypercored machines in sync. i only have to update the feed file in one place, and then sync it

yeah - that was almost exactly my first thought too

in production, i'd want a lot more 'devops' support features. but it works as a proof-of-concept

so long as you have versioned access to the feeds dat - and potentially, it could be driven from a git repo

last year, i was experiencing a lot of issues where some peers couldn't see other peers. i'd like to see more visibility as to whether or not things really did sync

it just feels more UXy to support `dat register/publish`

yeah, that should work

mmm, there's a docker-compose file in the datbase project - but its "interesting"

i actually haven't tried the public dat registry before ... i don't think it was open before

i'm guessing it isn't providing long term peering though, right?

@karissa @jhand I'm willing to help make docs pr's and probably code some too - but I think i need a bit more lead-in info, wrt what the plan is, and how the components should fit together

tbh, its not clear to me what each of the (too many) sites do, and dont do

@karissa I'm very curious to know what port 25 is used for - https://github.com/datproject/datBase/commit/03...

it kinda suggests that datbase is acting as an smtp server

SvenDowideit: docker i think

SvenDowideit: needed that to be open for us to send emails

shoudln't be

ports you expose in Docker are for allowing the container to listen - and to send email, you don't need that

SvenDowideit: oh nice, good to know. you can open a PR if you like

will do - I'm basically trying to work out how to use this - but as i rambled about above

i can't find enough info to tell me what the parts do / are intended to do

weirdly, running the datproject/datbase image fails for me with lots of npm errors

SvenDowideit: that's really good feedback. jhand had been the one to most recently change the marketing and main website for datBase

SvenDowideit: so feel free to direct more feedback his way, unfortunately I don't have time atm to put significant effort into docs and such

(martinheidegger) karissa: Out of curiosity, what are you working on these days?

martinheidegger i'm working on peer to peer offline mapping apps!

OH - so also mildly relevant to my work needs :)

(using hyper*/dat stack, naturally)

(martinheidegger) Nice: I am working on GIS stuff too!

(martinheidegger) (though not mapping)

(martinheidegger) Will attend the next FossGiS sprint in Bonn.

(martinheidegger) Just tried out mateo, interesting!

(martinheidegger) It doesn't create dat's yet ;)

(martinheidegger) To "open"

martinheidegger didn't know that! cool, what exactly?

martinheidegger hehe yeah, dat is used more as the underlying tech and not as part of the ui

(martinheidegger) I saw using it as hyperlog, though "connect usb" - come on: that screams dat right there.

(martinheidegger) I am working on a pointcloud distribution system, with a lot more user-generated data (into Terabytes)

(martinheidegger) Trying to figure out distributed indexing and "thumbnailing" of the data.

(martinheidegger) https://github.com/dotloom

ok, @jhand and others - https://github.com/datproject/datBase/pull/628 is about as far as i've gotten

hello! i have a short, quick question

if I create a dat, is it public or private?

I assume the hash/address is made available in the dat network so other devices can connect

but I'm not sure if the hash/address is available for anyone to see, as well

I'm not sure either - i think the answer is that its public, but undiscoverable/unknown - in that someone would need to guess the address

oh, hadn't spotted that page. thanks for the link!

yeah - there's a tone of pages - i'm just trying to get started figuring out what does what :)

> "Only someone with the key can download data for Dat."

I can see that a test dat that I created is still available on datbase.org, even if no peers are connected/serving the data

yeah - but isn't that because you told datbase about your data (ie, the key)

well, yes, I browsed the test dat using the website

so... if I had shut down the clients before checking the website, then datbase would have returned a 404 instead?

i very much think so

yes, that seems to be the case -- just tested it

tomboy, SvenDowideit: the hash is basically the secret information that allow some{body,thing} to access your dat. If you keep it to yourself nobody can guess it. As soon as you give it to some{body,thing} it is now a share secret

but still, only the peeps knowing the hash can access the data. Other peeps still can guess it

yeah - but it gets muddy, because you can browse to the dat - which is sharing it, but that's not entirely made explicit to the new user

SvenDowideit: true

basically - there's a large amount of assumed knowledge that newbs (like me) don't have a source for

it'll be fun to work on the UX & docs to make what's happening more obvious

maybe that could be an aditionnal question in the list. Like "Does browsing to my dat make it public?"

I agree sven im highly interested in projects like this

like the port numbers used by the second dat client

Because i want to make a protocol for discover by organizing decentralized tags

gogogo: https://github.com/datproject/docs/ :)

Im trying to understand so i can combine the technology with a tag protocol for discover

Y

It be nice if i found someone to consult

Or more experienced in the file sharing side to design with

(martinheidegger) raziellight: one reason why dat has a doc problem is because of the unfortunate name. `dat` finds so many things non-dat-related that its hard to use google. Maybe we need to come up with a second keyword that makes it better to google?

Does dat desktop allow for shared folders that are write- and readable for all peers? Or is it just readable?

Nils[m]: all dats are writeable only by the person that creates it right now.

Thanks, jhand . And is it possible to have peers mirror some data to make it available without being able to decrypt the data?

Are writable dats on the roadmap?

Nils[m]: yes, you can proxy data without knowing what it is. I'm not sure how that is implemented but mafintosh has been talking about working on that.

Nils[m]: and yes, writable dats are coming via hyperdb: https://github.com/mafintosh/hyperdb