#dat

/

      • vmx joined the channel
      • emilbayes
        sam[m]: Here's the part in the libsodium docs why the shared secret should be hashed with the two pk's to produce a secure shared secret: https://download.libsodium.org/doc/advanced/sca...
      • vmx joined the channel
      • discopatrick joined the channel
      • charlesbrandt joined the channel
      • charlesbrandt has quit
      • vmx joined the channel
      • charlesbrandt joined the channel
      • charlesbrandt has quit
      • charlesbrandt joined the channel
      • Zerata has quit
      • ZerataX joined the channel
      • charlesbrandt has quit
      • charlesbrandt joined the channel
      • charlesbrandt has quit
      • charlesbrandt joined the channel
      • charlesbrandt has quit
      • vmx has quit
      • soyuka_ joined the channel
      • charlesbrandt joined the channel
      • soyuka_
        exit
      • soyuka_ has quit
      • soyuka_ joined the channel
      • charlesbrandt has quit
      • Mitar: you there?
      • charlesbrandt joined the channel
      • charlesbrandt has quit
      • charlesbrandt joined the channel
      • rho joined the channel
      • kernkern has quit
      • ZerataX has quit
      • ZerataX joined the channel
      • vmx joined the channel
      • Mitar
        I am here
      • soyuka
      • tyrannus
        I am trying to use dat to mostly all our research data transfer needs. But I have stumble on 2 problems:
      • 1. Firewall issues. Question: are there any plans to extend the protocol to work over, say HTTPS? That will make things easier
      • 2. Internal-to-internal access: two nodes in our campus network will not be able to communicate because they try to use the public IP (the exit IP) and the campus rules disallow use of public IPs for internal communication (10.x to 10.x should be used). I suspect that this will be quite common elsewhere...
      • jhand
        tyrannus: yep we've seen those similar issues at other universities
      • tyrannus: we're working on http provider (https://github.com/datproject/dat-http) and have also floated the idea of proxy peers for places that disallow internal connections
      • tyrannus: (If you ther person who emailed us at hi@dat, apologies we're slow to getting back - still interested in chatting though! we'll try to get back soon)
      • tyrannus
        jhand: in our case internal communications _are_ allowed but we have to use 10.x addresses. Looking at `dat doctor` it seems that the protocol tries to discover a public IP (which, of course, makes sense e most cases)
      • jhand: yes, its me. No rush
      • jhand
        oh ok interesting, I wonder if we could attempt both
      • tyrannus
        jhand: is dat-http supported by dat protocol discovery automatically? If so, I will proceed to install it
      • jhand
        tyrannus: its not finished yet and no =)
      • but the idea is that you will be able to provide a http source if p2p fails totally
      • tyrannus
        It would be great if that would be built-in in the protocol - no need for user intervention. Lots of places will require it
      • jhand
        yea definitely!
      • tyrannus
        I suspect, for example most US Federal Government networks are a good example where this will apply
      • For our internal-to-internal users, were are spinning up an outside AWS machine to act as a bridge for internal communications. Not optimal,but works
      • i.e. with a dat-server
      • iml_ joined the channel
      • jhand
        nice, glad you were able to get that working
      • TheLink
        jhand: does it make sense to create a feature request ticket for dat-cli regarding dat-daemon integration?
      • jhand
        TheLink: mmm sure but may not happen for a bit
      • TheLink
        ok
      • kernkern joined the channel
      • soyuka_
        TheLink: IMO it's a good idea do decouple dat-desktop, dat-cli behind a daemon that'd continuously share dats. See https://github.com/dat-land/dat-desktop/issues/434 for my initial thoughts (and a related PR I think).
      • Though, they've been nice thoughts/ideas/improvements in dat-daemon these past day (support websockets + advanced RPC on dats) and things might move a lot before it's kinda stable haha
      • days*
      • TheLink
        soyuka_: I agree that the daemon should be an integrated solution for everything dat
      • guess it makes sense that it's a separate thing that other dat operators can register with
      • soyuka_ has quit
      • dat-gitter
        (martinheidegger) soyuka_ I generally like the idea of the daemon running in the background. for DAT desktop there is quite a litany of more features important.
      • (martinheidegger) i.e. We need to have access to some data of a dat before we decide to store it.
      • (martinheidegger) (preview before actually showing it)
      • (martinheidegger) I am currently studying dat/hyperdrive/etc. to figure out a good way to display the status of replication per peer + version + file. In order to show: what versions are available and what files in what version is replicated.
      • Mitar
        if I create multiple dat repositories which have some files the same, does dat deduplicate across dat repositories?
      • jhand
        Mitar: no, currently each Dat is independent
      • they never know about each other
      • Mitar
        so even in the network, it does not deduplicate anything?
      • jhand
        Mitar: no because each dat is it's own set of peers, peers never connect across different dats
      • Mitar
        hm, ipfs is then better in this regard, but I see that this is a different design
      • are there any plans to have collections of dats?
      • fabrice_d
        the local dat store could dedupe I guess
      • Mitar
        so we have 1000 datasets we would like to seed, and many of them are different permutations (test, train, etc.) of same data
      • jhand
        Mitar: ya that could be a future optimization to share dats
      • Mitar: when our handling of many small files improves, you could also put that into one dat - since you can sync subsets. But we need some better tooling around syncing subsets of data
      • vmx has quit
      • Mitar
        but then datbase will display one huge dataset
      • instead of multiple small ones?
      • even now I have two layers of datasets
      • jhand
        ah yes, not a great way to handle submodule type Dats right now
      • Mitar
        I have a top-level dat dataset, which then include 3 directories: full dataset, train, test
      • and I add then top-level as one dat, and those three directories as each one dat
      • dat-gitter
        (martinheidegger) jhand: At the moment the `tree` doesn't store the sha-hashes for each file. Without that it seems impossible to deduplicate?!
      • (martinheidegger) (in hyperdrive)
      • jhand
        yea it'd be at the chunk level i think, not file
      • dat-gitter
        (martinheidegger) (i.e. to not re-download a file you know exists locally)
      • jhand
        but could also use content-addressed storage to dedup i think
      • that'd just be at the local level though
      • Mitar: nice, that makes sense! good use case
      • millette
        frabrunelle, you around? Would you be interested in talking about dat at http://jido2018.waglo.com/ March 3rd for OpenDataDay? (Montréal)
      • dat-gitter
        (martinheidegger) My thinking might be rudimentary but if the tree would store the sha hashes and that sha was downloaded somewhere else it could just deduplicate the download - no?
      • (martinheidegger) Might be useful if some people create multiple dats for the same data?!
      • (martinheidegger) Too much overhead?
      • frabrunelle
        millette: I will likely be in Toronto at that time, so I wouldn't be available to talk about Dat. But I might have been interested otherwise! Thanks for asking :-)
      • millette
        frabrunelle, can you recommend anyone else around? Also, I'm pushing this tweet https://twitter.com/RoLLodeQc/status/9660473268...
      • Maybe anarcat I'm sure he's an expert by now :-)
      • anarcat
        hmm?
      • an expert in what?
      • millette
        frabrunelle, merci :-)
      • anarcat, dat, of course
      • anarcat
        millette: nope
      • kernkern has quit
      • tyrannus
        Is Bunsen browser dead? I wanted to support them on my dat-detection library, but got no answer...
      • dat-gitter
        (martinheidegger) jhand: in lieu of an answer: https://github.com/mafintosh/hyperdrive/issues/203
      • shama joined the channel
      • kernkern joined the channel
      • iml_ has quit
      • iml_ joined the channel
      • kernkern has quit
      • fsteff joined the channel
      • fsteff has quit