从ARM镜像的三种方式看统一镜像构建

Java虚拟机所提出的“一处编译、处处运行”的机制使得Java得到了极大的发展，不依赖与底层的操作系统，在JVM层进行解耦，同一份Java字节码在不同平台的运行真正实现了跨操作系统的需求，从此Java应用基本上默认被认为跨操作系统可用的。这种思路显然对于容器的发展具有同样的借鉴作用，在前面的文章曾介绍过使用QEMU在x86体系架构的机器上进行ARM镜像构建的方式，这篇文章以ARM镜像的三种构建方式为例对镜像的统一构建进行展开。

统一构建的背景

在ARM体系结构的硬件环境中安装Docker，然后构建并运行ARM的镜像自然非常简单，但是如果没有ARM环境，只有x86的机器又想做这件事情。

方式1: 使用特定的ARM硬件

在特定的ARM硬件设备上安装Docker并进行构建和运行，这没有任何问题，但是如果这样的话不同的体系结构仅仅构建就需要在各自不同的硬件环境上进行管理，维护起来非常费力。

方式2: 使用QEMU

使用QEMU进行仿真模拟，使用QEMU作为中间的适配层，不同的体系架构的镜像构建和运行可以通过QEMU来进行统一模拟，详细可参看：

• https://liumiaocn.blog.csdn.net/article/details/109305857

方式3: 使用Docker的新特性buildx

buildx是Docker下一代的构建特性，从19.03开始已经处于实验特性阶段，缺省处于关闭状态，打开此特性的方式可参看如下内容：

• https://liumiaocn.blog.csdn.net/article/details/109385231

buildx简介

buildx是在BuildKit基础上提供的构建功能扩展的模块，所以它有个x。除了目前docker build的功能之外，buildx还提供如下功能支持：

• 和docker build类似的使用方式
• 包含BuildKit的相关功能
• 提供多构建器实例支持
• 多节点跨平台镜像构建支持
• 组合构建支持
• 支持bake功能
• 容器构建支持（Docker和Kubernetes）

支持的体系结构

buildx所支持的体系结构类别如下所示，使用docker buildx ls命令即可查看

liumiaocn:~ liumiao$ docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS  PLATFORMS
default * docker                  
  default default         running linux/amd64, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6
liumiaocn:~ liumiao$

常见问题与对应

• multiple platforms feature is currently not supported for docker driver

liumiaocn:go liumiao$ docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7,linux/arm/v6 -t liumiaocn/godemo/multiarch .
WARN[0000] invalid non-bool value for BUILDX_NO_DEFAULT_LOAD:  
multiple platforms feature is currently not supported for docker driver. Please switch to a different driver (eg. "docker buildx create --use")
liumiaocn:go liumiao$

对应方法也提示的非常清楚了，执行docker buildx create --use命令即可。

多体系架构镜像构建示例

这里以方式3为例，对多体系架构方式下的镜像构建进行演示，比如某go语言应用可能既需要跑在x86的环境中，又需要跑在ARM体系建构的环境中，而使用buildx则可以一次性的构建多种，并直接推送至docker镜像库DockerHub上。

事前准备

事前准备如下Dockerfile用于构建一个基于Alpine的go语言应用，类似与Spring Boot的demo应用，在8080端口返回一段消息。

liumiaocn:go liumiao$ ls
Dockerfile		basic-web-hello.go
liumiaocn:go liumiao$ cat Dockerfile 
ARG  GOLANG_VER=1.15.3
ARG  ALPINE_VER=3.12
FROM golang:${GOLANG_VER}-alpine${ALPINE_VER} as go-builder-1.15.3

WORKDIR /build

COPY basic-web-hello.go /build
RUN  cd /build && go build -o http-server basic-web-hello.go

FROM alpine:${ALPINE_VER}

WORKDIR /target

COPY --from=go-builder-1.15.3 /build/http-server /target/

EXPOSE  8080
ENTRYPOINT ["/target/http-server"]
liumiaocn:go liumiao$

go语言的示例应用也非常简单，详细如下所示：

liumiaocn:go liumiao$ cat basic-web-hello.go package main import "fmt" import "net/http" func Hello(response http.ResponseWriter, request *http.Request) { fmt.Fprintf(response, "Hello, LiuMiao, Welcome to go web programming...\n") } func main() { http.HandleFunc("/", Hello) http.ListenAndServe(":8080", nil) } liumiaocn:go liumiao$

因为此处示例中会将一次构建出来的多种镜像推送至DockerHub，所以此处会事先进行docker login的操作

liumiaocn:go liumiao$ docker login
Authenticating with existing credentials...
Login Succeeded
liumiaocn:go liumiao$

步骤1: 创建一个buildx的构建器

liumiaocn:go liumiao$ docker buildx create --use --name multiarhbuilder
multiarhbuilder
liumiaocn:go liumiao$

步骤2: 查看构建器

liumiaocn:go liumiao$ docker buildx inspect --bootstrap
Name:   multiarhbuilder
Driver: docker-container

Nodes:
Name:      multiarhbuilder0
Endpoint:  unix:///var/run/docker.sock
Status:    running
Platforms: linux/amd64, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6
liumiaocn:go liumiao$

确认此时buildx ls状态

liumiaocn:go liumiao$ docker buildx ls
NAME/NODE          DRIVER/ENDPOINT             STATUS  PLATFORMS
multiarhbuilder *  docker-container                    
  multiarhbuilder0 unix:///var/run/docker.sock running linux/amd64, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6
default            docker                              
  default          default                     running linux/amd64, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6
liumiaocn:go liumiao$

步骤3: 构建并推送

构建并推送至镜像库

liumiaocn:go liumiao$ docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7,linux/arm/v6 -t liumiaocn/demo-multiarch --push .
WARN[0000] invalid non-bool value for BUILDX_NO_DEFAULT_LOAD:  
[+] Building 50.5s (40/40) FINISHED                                                                                                     
 => [internal] load build definition from Dockerfile                                                                               0.0s
 => => transferring dockerfile: 32B                                                                                                0.0s
 => [internal] load .dockerignore                                                                                                  0.0s
 => => transferring context: 2B                                                                                                    0.0s
 => [linux/arm/v7 internal] load metadata for docker.io/library/alpine:3.12                                                        5.2s
 => [linux/amd64 internal] load metadata for docker.io/library/alpine:3.12                                                         5.3s
 => [linux/amd64 internal] load metadata for docker.io/library/golang:1.15.3-alpine3.12                                            5.7s
 => [linux/arm/v7 internal] load metadata for docker.io/library/golang:1.15.3-alpine3.12                                           5.3s
 => [linux/arm/v6 internal] load metadata for docker.io/library/alpine:3.12                                                        5.5s
 => [linux/arm64 internal] load metadata for docker.io/library/alpine:3.12                                                         5.4s
 => [linux/arm64 internal] load metadata for docker.io/library/golang:1.15.3-alpine3.12                                            5.5s
 => [linux/arm/v6 internal] load metadata for docker.io/library/golang:1.15.3-alpine3.12                                           5.4s
 => [internal] load build context                                                                                                  0.0s
 => => transferring context: 40B                                                                                                   0.0s
 => [linux/arm/v6 stage-1 1/3] FROM docker.io/library/alpine:3.12@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7  0.0s
 => [linux/arm/v6 go-builder-1.15.3 1/4] FROM docker.io/library/golang:1.15.3-alpine3.12@sha256:df0119b970c8e5e9f0f5c40f6b55edddf  0.0s
 => [linux/arm64 go-builder-1.15.3 1/4] FROM docker.io/library/golang:1.15.3-alpine3.12@sha256:df0119b970c8e5e9f0f5c40f6b55edddf6  0.0s
 => [linux/arm64 stage-1 1/3] FROM docker.io/library/alpine:3.12@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e78  0.0s
 => => resolve docker.io/library/alpine:3.12@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a               0.0s
 => [linux/arm/v7 go-builder-1.15.3 1/4] FROM docker.io/library/golang:1.15.3-alpine3.12@sha256:df0119b970c8e5e9f0f5c40f6b55edddf  0.0s
 => => resolve docker.io/library/golang:1.15.3-alpine3.12@sha256:df0119b970c8e5e9f0f5c40f6b55edddf616bab2b911927ebc3b361c469ea29c  0.0s
 => [linux/arm/v7 stage-1 1/3] FROM docker.io/library/alpine:3.12@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7  0.0s
 => => resolve docker.io/library/alpine:3.12@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a               0.0s
 => [linux/amd64 go-builder-1.15.3 1/4] FROM docker.io/library/golang:1.15.3-alpine3.12@sha256:df0119b970c8e5e9f0f5c40f6b55edddf6  0.0s
 => => resolve docker.io/library/golang:1.15.3-alpine3.12@sha256:df0119b970c8e5e9f0f5c40f6b55edddf616bab2b911927ebc3b361c469ea29c  0.0s
 => [linux/amd64 stage-1 1/3] FROM docker.io/library/alpine:3.12@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e78  0.0s
 => => resolve docker.io/library/alpine:3.12@sha256:c0e9560cda118f9ec63ddefb4a173a2b2a0347082d7dff7dc14272e7841a5b5a               0.0s
 => CACHED [linux/arm64 stage-1 2/3] WORKDIR /target                                                                               0.0s
 => CACHED [linux/arm64 go-builder-1.15.3 2/4] WORKDIR /build                                                                      0.0s
 => CACHED [linux/arm64 go-builder-1.15.3 3/4] COPY basic-web-hello.go /build                                                      0.0s
 => CACHED [linux/arm64 go-builder-1.15.3 4/4] RUN  cd /build && go build -o http-server basic-web-hello.go                        0.0s
 => CACHED [linux/arm64 stage-1 3/3] COPY --from=go-builder-1.15.3 /build/http-server /target/                                     0.0s
 => CACHED [linux/arm/v7 stage-1 2/3] WORKDIR /target                                                                              0.0s
 => CACHED [linux/arm/v7 go-builder-1.15.3 2/4] WORKDIR /build                                                                     0.0s
 => CACHED [linux/arm/v7 go-builder-1.15.3 3/4] COPY basic-web-hello.go /build                                                     0.0s
 => CACHED [linux/arm/v7 go-builder-1.15.3 4/4] RUN  cd /build && go build -o http-server basic-web-hello.go                       0.0s
 => CACHED [linux/arm/v7 stage-1 3/3] COPY --from=go-builder-1.15.3 /build/http-server /target/                                    0.0s
 => CACHED [linux/amd64 stage-1 2/3] WORKDIR /target                                                                               0.0s
 => CACHED [linux/amd64 go-builder-1.15.3 2/4] WORKDIR /build                                                                      0.0s
 => CACHED [linux/amd64 go-builder-1.15.3 3/4] COPY basic-web-hello.go /build                                                      0.0s
 => CACHED [linux/amd64 go-builder-1.15.3 4/4] RUN  cd /build && go build -o http-server basic-web-hello.go                        0.0s
 => CACHED [linux/amd64 stage-1 3/3] COPY --from=go-builder-1.15.3 /build/http-server /target/                                     0.0s
 => CACHED [linux/arm/v6 stage-1 2/3] WORKDIR /target                                                                              0.0s
 => CACHED [linux/arm/v6 go-builder-1.15.3 2/4] WORKDIR /build                                                                     0.0s
 => CACHED [linux/arm/v6 go-builder-1.15.3 3/4] COPY basic-web-hello.go /build                                                     0.0s
 => CACHED [linux/arm/v6 go-builder-1.15.3 4/4] RUN  cd /build && go build -o http-server basic-web-hello.go                       0.0s
 => CACHED [linux/arm/v6 stage-1 3/3] COPY --from=go-builder-1.15.3 /build/http-server /target/                                    0.0s
 => exporting to image                                                                                                            44.8s
 => => exporting layers                                                                                                            0.0s
 => => exporting manifest sha256:302f1cf75ffe9246805815e3005b9297d2c4cb7397e597486be7b41dd8da33fa                                  0.0s
 => => exporting config sha256:cd860084c606487866ad9b506d8c3f01503994c9ce324abc881cd8ad25d8ed25                                    0.0s
 => => exporting manifest sha256:86eb799aaf21514df1ff585911ccd298896ae1b3bb5cc91c10526626794d386e                                  0.0s
 => => exporting config sha256:0ad87475cb499ba63b0e1dd21dc49ce58db242ac4d7ef694fc9b0921c01b42aa                                    0.0s
 => => exporting manifest sha256:f2925920211e74da20c633e68aaa1a27a39baf0c78de77f1e09dfc9ac567fa0f                                  0.0s
 => => exporting config sha256:b308baa8074e6aff5791d3412be2e6009883d2a10d59bcba8a189bfd0bc1a4d2                                    0.0s
 => => exporting manifest sha256:e21a86bd0382a8641ad9594432429f41e40883528dda90197a6da019334bd6f8                                  0.0s
 => => exporting config sha256:30841366c1dc914a3024c20f265b0dd84bddc02d360bd0c64ea71426e4216ae1                                    0.0s
 => => exporting manifest list sha256:ede8b685cf5a4a8265bd4beb8432189a40f9e13c9220647b7737dd0e3e466e7d                             0.0s
 => => pushing layers                                                                                                             25.0s
 => => pushing manifest for docker.io/liumiaocn/demo-multiarch:latest                                                             19.6s
liumiaocn:go liumiao$

注：此处因为之前执行过一次没有push，所以上述日志中会有CACHED的内容。另外此处Docker Hub支持这种方式的推送，并不意味着所有镜像私库都会无缝支持，以Harbor为例，Harbor 2.0.0的版本才对此特性有所支持。

步骤4: DockerHub查看

查看DockerHub上的推送的镜像可以看到，一次性有多个体系结构的镜像

步骤5: 本地确认

在步骤3中在目前的版本中，进行构建和推送之后被定实际上并没有镜像留下，而推送之后使用docker pull所获得的镜像也会根据当前环境进行选取和拉取相应的体系架构的镜像，执行日志如下所示

liumiaocn:go liumiao$ docker images |grep demo-multiarch
liumiaocn:go liumiao$ docker pull liumiaocn/demo-multiarch:latest
latest: Pulling from liumiaocn/demo-multiarch
188c0c94c7c5: Already exists 
996a6a4793a6: Pull complete 
954105e4b4c8: Pull complete 
Digest: sha256:ede8b685cf5a4a8265bd4beb8432189a40f9e13c9220647b7737dd0e3e466e7d
Status: Downloaded newer image for liumiaocn/demo-multiarch:latest
docker.io/liumiaocn/demo-multiarch:latest
liumiaocn:go liumiao$ 
liumiaocn:go liumiao$ docker images |grep demo-multiarch
liumiaocn/demo-multiarch                                             latest                          cd860084c606        12 hours ago        12MB
liumiaocn:go liumiao$

• 确认容器的体系架构 可以看到当前环境为64位的x86体系架构，拉取的镜像也相同（手头没有ARM环境，有设备的可以验证一下）

liumiaocn:go liumiao$ uname -m
x86_64
liumiaocn:go liumiao$ docker run --name multiarch --rm -d -p 8080:8080 liumiaocn/demo-multiarch
bec67c653a36cbf04d2757018aea2b8b401a138523ab525cd7cd79d8135fa577
liumiaocn:go liumiao$ docker exec -it multiarch sh
/target # uname -m
x86_64
/target #

然后在别的终端访问8080端口，可以正常确认到信息的返回如下

liumiaocn:go liumiao$ curl http://localhost:8080
Hello, LiuMiao, Welcome to go web programming...
liumiaocn:go liumiao$

总结

随着技术的不断推进，容器化在越来越多的场景中得到了广泛地应用。统一的镜像构建的整体方案也已经呼之欲出，但是不同的体系架构是否能够在市场中存活足够长的时间，是否能够对其提供相应的支持和生态，这些非技术性的因素会起到什么样作用还是不确定的，可以边走边看。

参考内容

https://docs.docker.com/buildx/working-with-buildx/ https://bbs.huaweicloud.com/blogs/155274