编程知识 cdmana.com

Docker container hot migration technology based on criu


Preface

Recently, I have been studying docker Container heat transfer , I have consulted a lot of relevant materials on the Internet , And refer to the example on the Internet , There are many problems in this process , For example, follow a lot of tutorials , In the end, it didn't achieve the expected effect . So I wrote a summary here , It is also the consolidation of their own learning .

technology :Checkpoint/Restore

One 、 The local environment

1. System environment

Centos7.6 $ lsb_release -a                                           
LSB Version:	:core-4.1-amd64:core-4.1-noarch
Distributor ID:	CentOS
Description:	CentOS Linux release 7.8.2003 (Core)
Release:	7.8.2003
Codename:	Core

 Actually measured  CentOS Linux release 7.9.2009 It's OK 

2. Kernel version

Centos7.6 $ uname -a
Linux bogon 5.10.2-1.el7.elrepo.x86_64 #1 SMP Sun Dec 20 09:53:23 EST 2020 x86_64 

 Actually measured  3.10 The version is OK 

3.Docker edition

Centos7.6 $ docker version
Client:
 Version:      17.06.0-ce
 API version:  1.30
 Go version:   go1.8.3
 Git commit:   02c1d87
 Built:        Fri Jun 23 21:20:36 2017
 OS/Arch:      linux/amd64
 
Server:
 Version:      17.06.0-ce
 API version:  1.30 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   02c1d87
 Built:        Fri Jun 23 21:21:56 2017
 OS/Arch:      linux/amd64
 Experimental: true
 
 Actually measured  17.12.1-ce no way 

4.CRIU edition

Centos7.6 $ criu -V
Version: 3.12

 Insert picture description here

here CRIU There is a warning that the memory snapshot is not available , It's better to solve it , However, it does not affect the realization of this thermal migration .

5.daemon.json

Centos7.6 $ cat /etc/docker/daemon.json 
{
   
    
"registry-mirrors": ["https://docker.mirrors.ustc.edu.cn"] ,
"experimental": true
}

The above environments are used in this implementation , Other environments have not been verified except as mentioned above .( Software installation or kernel update can be self access to information )

Docker Unloading and reloading
Centos Update kernel

restart docker service

systemctl daemon-reload
systemctl restart docker

Two 、 The container itself migrates

First use bash Script to verify

// Create a container 
docker run -d --name looper2 --security-opt seccomp:unconfined busybox /bin/sh -c 'i=0; while true; do echo $i; i=$(expr $i + 1); sleep 1; done'

 Insert picture description here

// New checkpoint 
docker checkpoint create looper2 checkpoint1

 Insert picture description here
The state of the container changes from Up Turn into Exited
 Insert picture description here

At this point, the program has run to the current state

 Insert picture description here

// Restore checkpoint 
docker start --checkpoint checkpoint1 looper2

After restoration, the log can be renewed to the last print , In line with expectations .
 Insert picture description here
 Insert picture description here

3、 ... and 、 Cross container migration

At this point, create a container with the same command looper2-clone, Pause immediately after creation , Avoid writing too many logs

docker run -d --name looper2-clone --security-opt seccomp:unconfined busybox /bin/sh -c 'i=0; while true; do echo $i; i=$(expr $i + 1); sleep 1; done'
docker stop looper2-clone

 Insert picture description here
 Insert picture description here
The container looper2 The checkpoint file for , copy to looper2-clone, otherwise looper2-clone If the checkpoint is not found, the migration will fail
 Insert picture description here


// transfer 
docker start --checkpoint checkpoint1 looper2-clone

At this point, the log jumps directly to checkpoint1 Where checkpoints run , Then keep running , In line with expectations .
 Insert picture description here

Four 、 Multiple process runtime container migration

Create a container first , Then I'll write a few programs in the container , To make it easier to see if hot migration is in effect , I always use the way of writing files to observe the implementation .

docker run -id --security-opt=seccomp:unconfined --name test -h test test:1222 /usr/sbin/init

/root/app.c

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>

int main()
{
   
    
    int i = 0;
    int j = 0;
    char buff[128] = {
   
    '\0'};
    char tmp[128] = {
   
    '\0'};
    pid_t fpid;
    fpid=fork();
    if (fpid == 0) {
   
    
        while(j < 100000)
        {
   
    
            memset(tmp,0,sizeof(tmp));
            sprintf(tmp,"echo \"fork:%d\">>/root/1",j);
            system(tmp);
            j++;
            sleep(1.5);
        }
    }
    while(i < 200000)
    {
   
    
        memset(buff,0,sizeof(buff));
        sprintf(buff,"echo \"count:%d\">>/root/1",i);
        system(buff);
        i++;
        sleep(1);
    }
    return 0;
}

compile :gcc -o app app.c

test.sh

#/bin/bash
/root/app &

for a in {
   
    1..1000000}
do
    echo "count:$a">>/root/2
    sleep 1
done

/etc/rc.local

#!/bin/bash
# THIS FILE IS ADDED FOR COMPATIBILITY PURPOSES
#
# It is highly advisable to create own systemd services or udev rules
# to run scripts during boot instead of using this file.
#
# In contrast to previous versions due to parallel execution during boot
# this script will NOT be run after all other services.
#
# Please note that you must run 'chmod +x /etc/rc.d/rc.local' to ensure
# that this script will be executed during boot.

touch /var/lock/subsys/local

sleep 1
/root/test.sh &
/bin/bash

exit 0

chmod +x /etc/rc.d/rc.local

// Submit to mirror image 
docker commit -p test test:1122

First, delete all the running containers , Convenient observation

docker kill $(docker ps -q)
docker rm $(docker ps -a)"

Start hot container migration

// Container to create 
docker run -id --security-opt=seccomp:unconfined --name test -h test test:1222 /etc/rc.local

Go into the container and see how it works :
 Insert picture description here

// Create checkpoints 
docker checkpoint create test checkpoint2

 Insert picture description here
In practice, because the program is still running before the checkpoint is created , All the pictures above are for reference only

// transfer 
docker start --checkpoint checkpoint2 test

 Insert picture description here
 Insert picture description here

After migration, the program can continue to execute the last checkpoint , In line with expectations .

Roll back

We can continue to run the program in the container for a period of time after the migration , And then use the checkpoint again checkpoint2 Migration , In this way, the running state of the program can be rolled back .

docker start --checkpoint checkpoint2 test

Take a break

docker stop test
// Re migrate 
docker start --checkpoint checkpoint2 test

At this point, the program running state :
 Insert picture description here
 Insert picture description here
 Insert picture description here
After migration, the container implements the function of rollback , The running state of the program is rolled back to the previous running state , In line with expectations .



thus , Basic realization of container heat transfer . In container process acceleration , The process fallback in the container meets the requirements .

I also verified the cross host container hot migration , Similar to the above cross container migration , You need to copy the checkpoint file to another host's directory where the checkpoint is stored
You need to ensure that the two host container environments , The host environment is the same , Avoid other problems

版权声明
本文为[osc_ 1ajf1srl]所创,转载请带上原文链接,感谢
https://cdmana.com/2020/12/20201225100106185w.html

Scroll to Top