#criu

/

      • ilackarms joined the channel
      • boucher has quit
      • nspiridonov_ joined the channel
      • ilackarms has quit
      • ilackarms joined the channel
      • obsrwr_home joined the channel
      • ilackarms has quit
      • boucher joined the channel
      • boucher has quit
      • cyrillos joined the channel
      • ldu4_away is now known as ldu4
      • kir joined the channel
      • ilackarms joined the channel
      • ilackarms has quit
      • kir has quit
      • katerina_ joined the channel
      • uvgroovy has quit
      • juliank has quit
      • uvgroovy joined the channel
      • juliank joined the channel
      • boucher joined the channel
      • ilackarms joined the channel
      • boucher has quit
      • ilackarms has quit
      • ilackarms joined the channel
      • ilackarms has quit
      • boucher joined the channel
      • katerina_
        hi again :) Well, I am currently trying to C/R a libvirt lxc container that is running /bin/sh as init; So I imagine that I ll need --shell-job option here. Anyway, when running the exact same command from terminal checkpoint and restore finish successfully but when I run them from inside libvirt code, I get the same error as the other time "tty: Standard stream is not a terminal,...
      • ...aborting". And indeed, the libvirt process from which I am running criu from, has stdin pointing to /dev/null for example. What should I do here ?
      • Ah, forgot to mention that the error appears on restore.
      • obsrwr_home has quit
      • Well, I'll reformulate the question. When running criu restore from inside a process whose stdin is NOT a tty, can we avoid the "tty: Standard stream is not a terminal" somehow ?
      • boucher
        katerina_: is your container actually a terminal?
      • katerina_
        Yup :)
      • boucher
        so you want to checkpoint an interactive terminal and then restore it non interactively
      • this doesn’t work in docker either fwiw
      • you end up with the same problem
      • cyrillos
        katerina_: the only option is only to use external fd, so criu won't check for tty being really tty
      • katerina_
        boucher: I dont understad why restoring from a process like that one I describe cannot let the container have a /dev/pts device attached.
      • cyrillos: external fd for what?
      • cyrillos
        for terminal you're going to restore stdin/out
      • wait a minute
      • boucher
        external-fd just lets you swap the file descriptor with another right
      • tych0
        boucher: you mean inherit-fd?
      • boucher
        right, sorry
      • katerina_
        well check, these are the fd for the child I am trying to restore and it's parent when they were running. https://paste.ubuntu.com/18241808/
      • cyrillos
        boucher: yeah
      • tych0
        boucher: yeah, if you exec criu with a particular fd open (that fd will have to be opened with no O_CLOEXEC), then yaah
      • cyrillos
      • when restore happens and no -j passed we will try to connect to existing slaves, but you have them already prepared by a caller
      • katerina_
        Well, to understand the things, when run from terminal restore succeeds, because the bash running criu restore has stdin pointing to a tty
      • But when I run it from inside libvirt, it fails, because, the parent of criu restore process, as seen in the pastebin, has df 0 pointing to dev/null
      • And the suggestion is to use an --external-fd on dump, but for which fd?
      • cyrillos
        gimme some time
      • gonna take a look
      • katerina_
        Sure, thanks.
      • cyrillos
        katerina_: who creates this /dev/null?
      • libvirt?
      • katerina_
        Yeah I guess.
      • this process is libvirt-lxc, that is created when starting a container.
      • I am trying to run criu restore from inside that process.
      • cyrillos
        do you know why they change the standart streams into null?
      • katerina_
        I have no idea.
      • cyrillos
        tych0: how you do preapre devices on restore?
      • tych0
        cyrillos: i thin kthe problem is that we restore cgroup properties by one big write
      • cyrillos
        i mean you should have provide slave peers as well, right?
      • tych0
        cyrillos: but the kernel only takes whats up to the first \n
      • cyrillos: oh, you mean for the login thing
      • cyrillos
        tych0: I think you mean the bug I replied :)
      • yeah
      • boucher has quit
      • tych0: how you migrate terminals?
      • iirc there were some things to setup in lxc config
      • no?
      • tych0
        cyrillos: yes, but we got rid of most of those
      • cyrillos
        and how you do it now?
      • tych0
        cyrillos: sec
      • cyrillos
        sure
      • katerina_: if environment get changed (just like in you case) most likely special hacks needed, so we need to figure out why the hell terminal peer converted into /dev/null
      • tych0
      • are the relevant bits
      • oh, i guess that's for /dev/console, though
      • cyrillos
        ah
      • you prepare --inherit-fd
      • tych0
        yes, but that should be only for the /dev/console fd i think
      • but maybe not :)
      • cyrillos
        looks like console only
      • hmm
      • katerina_
        cyrillos: I cxan surely find out about the /dev/null thing, but what do you mean the environment changes?
      • I mean, when the container is dumped, it has the exact same process as parent that has stdin pointing to /dev/null.
      • cyrillos
        katerina_: i mean when you dump container (or whatever) you have slave peers opened by a process (0,1,2 points to /dev/pty/0), then you start restore and here because you pass -j option we're testing if stdin is a terminal we can inherit from, but instead of terminal we meet null device and can't proceed. I think it's a change in environment
      • iow on dump we have stdin/out/err pointing to /dev/pts/0, but on restore it start pointing to /dev/null, and we simply don't know how to proceed
      • I think we can workaround it if we pass inherit fd and no -j option on restore
      • but the question is which --inherit-fd syntax should be there
      • also it's unclean why libvirt-lxc provides /dev/null instead of some terminal
      • katerina_
        I dont know, but I dont find it weird. libvirt-lxc is just the parent process of the container, why it should need stdin after all ?
      • cyrillos
        katerina_: this libvirt-lxc then executes criu restore, right?
      • katerina_
        Yeap.
      • cyrillos
        letme think...
      • katerina_
        To be accurate I call clone from inside this libvirt-lxc process. And in the cloned process I call criu restore.
      • cyrillos
        yup, i see
      • the first ls ouput is from the container which just started using libvirt?
      • katerina_
        Yup
      • cyrillos
        and where the master peer belong?
      • if /dev/pts/0 opened someone have to carry master peer of this pair
      • katerina_
        What do you mean by "if /dev/pts/0 opened someone"?
      • Well, you want that ptm device I guess.
      • cyrillos
        look, /dev/pts/0 device will only appear when some process opened /dev/ptmx
      • yes
      • katerina_
        let me see
      • cyrillos
        in vz containers we don't pass -j option for c/r as far as I remember
      • both ends must belong to the container
      • katerina_
        Yeah I get it.
      • Well check
      • cyrillos
        from what I see now -- without criu modification you won't be able to restore, we need a terminal to inherit session from
      • katerina_
        I can see /dev/pts/0 in the container lsof output
      • cyrillos
        yea
      • katerina_
        And the last entry in the parent's lsof output is that /dev/ptmx
      • cyrillos
        looks like parent opened /dev/ptmx
      • ok, so if i understand correctly, the process which starts the container, opened /dev/ptmx, then clone'd process opened /dev/pts/0
      • katerina_
        I guess that's correct.
      • ldu4 is now known as ldu4_away
      • Well, if I do the same thing before restoring? I mean let the libvirt-lxc open the /dev/ptmx to get the master end, and somehow pass the slave pair to the criu restore?
      • cyrillos
        now on restore you're tryint to create child process only which has a reference to /dev/pts/0, so you pass -j option to inherit the session, but in turn on restore libvirt-lxc closes all terminals and provide /dev/null back instead, right?
      • i guess so
      • katerina_
        Right, but when dumping libvirt-lxc had stdin pointing to /dev/null as well.
      • uvgroovy has quit
      • cyrillos
        yes, because on dump we don't need to inherit from anything
      • uvgroovy joined the channel
      • katerina_
        Right.
      • cyrillos
        the -j option is rather exception
      • the "right thing" is to have both peers inside container
      • initially -j option has been added to migrate top|htop|mutt from terminal to terminal, and it was assumed that stdin/out/err are valid terminals to inherit session from
      • you know, we can modify criu to take stdin device from something passed in command line
      • but i think it won't help much
      • katerina_
        Okay. Maybe it was wrong using -j option from start.
      • But how should I use --external in that case?
      • cyrillos
        lets look into external fd syntax :)
      • katerina_
        If i get right, --external gets as args, the end that lives outside the container, right?
      • cyrillos
        yup
      • katerina_
        So we 'll need to pass for start that /dev/ptmx
      • which is indeed a tty,
      • cyrillos
        i'm thinking ;)
      • katerina_: lets figure out what we need: will container carries both ends or only one of them?
      • katerina_
        I guess its better to leave it as it was.
      • cyrillos
        where only one peer belong into container?
      • katerina_
        The master end on the libvirt-lxc
      • cyrillos
        aha
      • katerina_
        But I have no opinion on that. Just thinking there should be some reason they did it that way