#dat

mafintosh: not sure what you mean sorry

mafintosh: do you get what im saying about the 'checkout last state before write started' thing

ogd: yup!

ogd: i'm talking about auto merging

mafintosh: ok so when import2 finishes

mafintosh: assuming import1 has finished already

we might have to explore what optimistic merges look like later. they'd be a feature afterward i think

mafintosh: then couldnt the import2 process be like 'hey import1 finished. lets try and make it appear to the user that we never forked in the first place'

mafintosh: and they do a diff of import1's branch and their (import2's) branch

ogd: imo thats too magic

are you always going to be sure that is the case, though?

ogd: i mean - you are doing a concurrent write in the first place

would it break down on shared drives?

karissa: not sure what case you are talking about

are you sure that import1 and import2 aren't conflicting and/or done by the same person

ogd: which would create forks if it was done on multiple machines

mafintosh: the output of import2 will either be 'finished import, automerged into master' or 'finished import, created new branch abc123'

karissa: well thats what the diff tells us

ogd: would you do the same thing if the forks happened on two different machines and we replicated?

mafintosh: i wouldn't.

mafintosh: no because the other machine doesnt have write privileges on my computer :)

i dont feel strongly

i am just trying to think through the options

yup!

its a lot to think about, and a lot to implement.

i do think we need to be able to detect if a write is happening in the middle of another write

ogd: +1

ogd: start->batch->...->end solves that i agree

my initial instinct is to at least be able to do that and tell the user what the consequences are of that write if they are to complete the action

and the start->batch->end sounds like a good idea

karissa: so like when import2 starts it would prompt and say 'there is an import happening write now, your import will be put into a new fork, is that ok?'?

ogd: but detecting that that write is "dead" is a bit hard to impl because of multiprocess

ogd: yeah. that'd be nice to have for beta at least, if its possible. gonna go to sweetbar brb

ogd: since we would have to message all processes if any of them are currently writing a specific stream etc

mafintosh: well now im not sure when we need to detect a 'dead import

ogd: if dat crashes?

yea it crashes

ogd: so you have start->batch->batch on disk with no end

you open a new process, start a new import

it just goes in a new branch

ogd: and you'd have two heads?

yea to the new import it appears just like the first import (that is dead) is still happening

i forgot that it would "rollback" to the last start node - that part makes sense

ah cool

im not sure what you do to fix it though

ok - you're convincing me

its kinda like having a dirty working directory in git

ogd: when we add a undo api for a "remove head" api you'd do that

you need to stash or 'checkout .' to revert

or reset --hard

ah yea

if we record time when we write the 'start

ogd: basically the user knows that the import failed

then we can show 'Import has been running for 38 minutes' in ' dat status'

mafintosh: yea

ogd: "end" should store the message

as well

mafintosh: the commit message?

ogd: yea

yea agreed

ogd: or maybe we add that to the start node?

hmm actually yea im not sure

ogd: hmm well you want to checkout the end node - thats the important one actually

ogd: since that describes the previous state

i need to go get more coffee afk

ogd, karissa: btw try running npm install level

no more compilation \o/

mafintosh: nice it worked

ogd: i like the start->batch->end idea

ogd: we probably shouldn't replicate a branch without an end node as well

mafintosh: yea true

mafintosh: purgatory branch

hehe

ogd: so that is basically a dat commit

ogd: brb reallocting

ogd: whats a better word that commit for this thing? :)

ogd: mafintosh: we might just want to use commit if its exactly the same as git

ogd: mafintosh i guess there's no staging area..

karissa: its staging as long as you don't have the end node inserted

*as long as you don't have the end node

yea like if you do a writestream, the writestream is 'staging' until it finishes

ogd: mafintosh ah right. but that state probably wont be very common, as most writes will finish

mafintosh: what does reallocating mean!?

jlord: i was moving cafes

Ohhh relocating

haha

lol i thought it said reallocating

whats the difference?

relocating reallocating

allocate means to reserve

ah got it, lol

sounds kinda robotic lol

but i guess it makes sense.. you are reallocating space for yourself to occupy

karissa: maybe its not just a write stream - maybe its just a commit you're working on :)

allocate: distribute (resources or duties) for a particular purpose

karissa: add some data, then some files, then other stuff and then you "close" the commit after a few days with the end node

jlord, ogd: its one of those english words i've learned from movies

mafintosh: haha nice

mor

is a word I learned from tv

omfordele

is a word i just looked up

rød grød med fløde :)

mafintosh: ah i see

roh goh muh fohhh

mafintosh: kinda seems like a writestream should have its own start/end

mafintosh: and a commit should have just an end

mafintosh: dunno for sure though

i think its hard to tell without some clear examples

are you guys wanting to prioritize this or are you just musing for the future?

i think what we have now works ok as long as you don't do concurrent writes so this is future stuff

toss them in a queue?

ogd: mafintosh: okay cool. it'd be nice if one of us could write up some of this discussion from today on an issue so we have a reference for later/other people can weigh in

substack: thats basically what we're gonna do

substack: (except the queue will be graph nodes with a single parent)

ogd, karissa this is basically a transaction api actually

yea thats what i was thinking earlier

you cover a write stream in a transaction - then you add a tag/commit node with a friendly message

and if you are in the middle of a transaction and start a new one you'd checkout the transaction start node first

ogd, karissa this is good a simple - i like it

mafintosh: i think tag/commit is separate though

mafintosh: and we shouldnt add it now

mafintosh: but we should add the transaction thing

i'll do the transaction api

ogd, karissa whats missing for doing the beta release?

mafintosh: i gotta finish making the cli match the docs, and add tests

mafintosh: are these done? https://github.com/maxogden/dat/issues/317

karissa: #9 was the above discussion :)

mafintosh: oh okay