#ipfs

/

      • ajray__ joined the channel
      • wscott_ joined the channel
      • wscott has quit
      • ajray_ has quit
      • ajray__ is now known as ajray_
      • jbenet
        hey whyrusleeping around? want to try add/cat across our machines?
      • flavioribeiro joined the channel
      • pfraze joined the channel
      • flavioribeiro has quit
      • whyrusleeping
        going to a movie atm, but im totally game when i get back
      • jbenet: did you see my question about how to differentiate between a directory and a file with multiple blocks?
      • jbenet
        whyrusleeping yeah great question. Unfortunately, unix fs made dirs and files fundamentally distinct, so we have to hack around it. (editorially, i think this is one of the few things the web changed from unix fs correctly (i.e. any path is both a file and directory).)
      • I think TRTTD is to specify it in the data segment of the object.
      • i want to design some data expressions to generate the data
      • for example, a way to express "data = concat the data of every object linked"
      • (which can apply recursively)
      • but can do more interesting things like, "links[0].data * 1000 || links[1].data" for 1000 the first linked object concat the second
      • basically, a pure function that generates the data from other pieces, getting closer to the kolmogorov complexity of the data.
      • --- anyway, not important right now.
      • for now, we can just have a special data value that means concat all the data of the linked objects.
      • --- I need to survey the literature on this. many of these simple, pure data generation languages already exist, so can see if one fits before designing one.
      • amiller, if there's something new here, this may be one of the research topics we can pursue o/
      • chas, you might be interested too. o/
      • pfraze has quit
      • whyrusleeping
        jbenet: what if we had a flag on the nodes that specified that the Data was actually a function of sorts?
      • jbenet: also, using 'ipfs mount XXX' im not sure how to make it connect to any other peers
      • we need to do a few things
      • first: we need to announce what values we have locally
      • that requires being able to read through the datastore and getting a list of keys from it
      • second, we need a way of bootstrapping the network
      • third, i need to stop drinking so much so i stop forgetting my third point...
      • oh! we need to figure out ipns, so we can make things a tad more user friendly
      • jbenet
        whyrusleeping: on "flag on the nodes" data can be self-describing. best this way, rather than trying to figure out exhaustively how to identify formats, etc. (we can even make raw bytes have a specific prefix or something)
      • on "connecting to other peers" yeah we need a set of well known DHT boostrap nodes.
      • bootstrap*
      • owen1 has left the channel
      • (DHTs always need some bootstrapping for discover)
      • discovery*
      • on reading datastore: yep-- datastore used to have a Query call (in the python version it does) for this purpose. we can add it (or call leveldb directly, if we write an ipfs-specific Datastore instance)
      • guybrush joined the channel
      • flavioribeiro joined the channel
      • frafra has quit
      • frafra joined the channel
      • pfraze joined the channel
      • fiatjaf joined the channel
      • cgervang joined the channel
      • cgervang has quit
      • whyrusleeping
        jbenet: my worry is this: what if i want to have a file that contains a valid 'data description function'
      • jbenet
        Then you encode it as data. Self-description doesn't have to be ambiguous
      • whyrusleeping
        im still slightly confused... how would i tell the difference between my file and a file that has indirect blocks?
      • jbenet
        Oh, i'm suggesting that instead of having a flag as a separate field in the obejct, the data value starts with the field as a prefix (that way it is of variable length, it can be dumped as is with the raw data, or interpreted and the final data computed). meaning, "just data" would also have a field (maybe 0x01 or something). unsure, this may be the wrong
      • way to do it
      • (i think we agree on the need for such a flag/field, just where to put is is the distinction).
      • whyrusleeping ^
      • whyrusleeping
        ah! okay, were on the same page now, lol
      • thank you
      • jbenet
        :D
      • i tend to favor such fields within the data because it emphasizes self-description. meaning that if you copy and move the data, you always bring the field (no need for an extra logical step). _something_'s type should be transmitted with the _something_ always. self-description helps.
      • whyrusleeping
        yeah, i totally agree with you.
      • jbenet
        the cool part is when we can reference type decoders within ipfs itself -- relevant thread: https://twitter.com/juanbenet/status/4782468035... (scroll up)
      • whyrusleeping
        oooOooo... okay, thats neat
      • domanic joined the channel
      • domanic
        jbenet, feross yo whats up
      • jbenet
        domanic: yo man, hacking today on ipfs. not sure if feross is back yet-- he probably should be back today or tonight
      • whyrusleeping
        jbenet: what are you working on?
      • jbenet
        would be great to finish the namespace today
      • whyrusleeping
        namespace ~= ipns?
      • jbenet
        also, sinking deeply into your bitswap and checking it, etc.
      • yeah ipns
      • whyrusleeping
        sweet
      • right now im working on pub/priv key stuff for generating peer ID's and identifying with the network
      • jbenet
        sweet!
      • whyrusleeping
        im putting off the rabin fingerprinting stuff until i find some good source code
      • btc joined the channel
      • jbenet
        yeah sounds good. no word yet from dm.
      • btc
        hello
      • jbenet
        whyrusleeping hey do we currently have cases where we have a multiaddr, but not the peer.ID ?
      • (the corresponding peer it)
      • id*
      • domanic
        jbenet, okay I'm gonna head down to your neck of the woods this evening
      • jbenet
        domanic: sounds good. note it in the email thread so dc/feross are aware.
      • whyrusleeping
        jbenet: we have such a situation
      • but thats generally only in bootstrapping situations
      • jbenet
        whyrusleeping: when does it happen? is it when we receive an incoming connection that hasn't identified itself?
      • (i.e. before it sends it pk)?
      • whyrusleeping
        no, only when we are saying "connect to a peer at this address"
      • jbenet
        when does that happen?
      • whyrusleeping
        when we first start the dht
      • jbenet
        ahhh ok
      • whyrusleeping
        the initial connection to a new peer doesnt generally have an id
      • jbenet
        so if we just store both addresses and pks then we're fine?
      • err addresses and peer.IDs
      • whyrusleeping
        yeah
      • jbenet
        (bootstrap info MUST have the pk too because otherwise you can't trust it)
      • whyrusleeping
        that would be secure
      • jbenet
        err node.ID
      • whyrusleeping
        just need the ID because its a has of the PK
      • jbenet
        err peer.ID**
      • yep
      • exactly
      • domanic has quit
      • ok, so other than that, + the case i mentioned above, that's it, right?
      • whyrusleeping
        yeap
      • jbenet
        ok sweet.
      • random Go Q: if we send to a channel, and then die, and nothing else is hanging on to the channel, it still properly gets garbage collected, right?
      • (i.e. no need to read to the end)
      • (die = goroutine ends)
      • whyrusleeping
        yeap
      • as long as the channel is buffered
      • if the channel isnt buffered, then the goroutine wont end
      • jbenet
        right
      • whyrusleeping
        yeap, it will be properly gc'ed
      • jbenet
        seems like in that case the gc could still find and kill the goroutine-- but *shrug*
      • ... maybe ...
      • whyrusleeping
        yeah... that seems probable... but probably also difficult
      • i dont think it does that
      • jbenet
        whyrusleeping that you know if, is there a reason to leak out from the swarm module which multiaddr is being used? (rather than exposing an interface like `Dial(peer.ID) error` that tries all relevant addresses)
      • (suppose that the swarm is aware of all peer.ID -> peer.Peer (with addresses) mappings)
      • whyrusleeping
        if that is the case, then nobody but the swarm should ever care about multiaddrs
      • to do that, we need to refactor a bit of stuff
      • jbenet
        yeah, am cleaning things up a bit as i get to understand them better -- and, routing will care about retrieving multiaddrs, but not using them directly.
      • flavioribeiro has quit
      • whyrusleeping
        basically does what msgio does for us
      • jbenet
        yeah, and a bit more: handling request/responses (MessageListener)
      • unsure how ready it is.
      • whyrusleeping
        hmm... dropping it in wouldnt be too difficult of a change if we decide we want it
      • jbenet
        whyrusleeping: right now, do we ever listen for more than one response? on the MessageListener?
      • whyrusleeping
        i think we do... i have the functionality there for t
      • jbenet
        ah i see it: routing.go#381
      • As is now, vulnerable to peers sending multiple responses.
      • whyrusleeping
        true...
      • jbenet
        probably safer to send each peer a different msg ID.
      • (i'll fix it-- cleaning up a few things)
      • whyrusleeping
        alright, thanks!
      • jbenet
        whyrusleeping is there a way of selecting over a slice of channels?
      • (variable length)
      • whyrusleeping
        nope
      • sadly
      • btc
        ?http://play.golang.org/p/8zwvSk4kjx
      • whyrusleeping
        yeap
      • thats a little more complicated than it needs to be though
      • jbenet
        oh man, need to use reflection? sad face.
      • btc
        golang-nuts suggests listening on one channel
      • is it feasible in the case you're thinking of?
      • whyrusleeping
        yeah, i would just spawn a goroutine for each channel, listen on it, and have each of those send on one 'central' channel
      • jbenet
        also, whyrusleeping, the GenerateMessageID uses 16 bytes-- meant 16 bits or bytes? 128 bits much more secure-- but security best done via listening on a (peer.ID, responseID) pair. The person we're talking to can always do whatever it wants with its responses-- nothing we can do about that, but forcing the pair prevents others from randomly guessing to
      • affect our comm.
      • m3s o/ discussion on chan slices.