利用docker+qemu搭建arm下cgo的编译环境

发表于 分类于

平时在写go程序过程中难免会写一些调用c/c++库的程序，go语言结合c/c++的代码在编译时往往非常令人头痛。下面以gocv库为例讲一下如何借助docker和qemu实现arm编译环境的搭建。

go语言原生支持交叉编译，但因为在程序中使用了gocv库，需要实现cgo的交叉编译。这一部分我实现了两种编译方式。第一种是通过虚拟机实现原生编译，第二种是配置交叉编译，本文我们讲一下借助docker和qemu实现的虚拟机容器原生编译，下一篇文章来讲交叉编译。

gocv对opencv进行了一次封装，所以如果想要编译arm版的程序，那必须搞定arm版的opencv库的编译。

采用docker+qemu搭建编译环境的原理是这样的：qemu项目中有一个qemu-user-static子项目https://github.com/multiarch/qemu-user-static 它可以实现让我们在x86_64架构的机器上通过qemu虚拟机来运行其他架构的容器。这样只要在这个虚拟机里把opencv编译好那就可以实现对应架构程序的原生编译了。本方法只有一个缺点：慢！

首先是注册qemu-user-static：

1
docker run --rm --privileged multiarch/qemu-user-static:register

这个命令只要运行一次就可以了，之后在运行异构镜像的时候就会自动调用qemu进行虚拟。

接下来只要写好Dockerfile，然后耐心地等待镜像构建就行了。

这里我贴一下我打包armhf和arm64架构镜像的Dockerfile。这两个Dockerfile几乎是一样的，除了架构不同。
armhf：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
FROM arm32v7/ubuntu

LABEL maintainer="Jiangda He. mail: hejiangda@gmail.com"

ENV OPENCV_VERSION=4.5.5

ENV BUILD="unzip \
    wget \
    build-essential \
    cmake \
    curl \
    git \
    libgtk2.0-dev \
    pkg-config \
    libavcodec-dev \
    libavformat-dev \
    libswscale-dev \
    libtbb2 \
    libtbb-dev \
    libjpeg-dev \
    libpng-dev \
    libtiff-dev \
    libdc1394-dev"

ENV TZ=Asia/Shanghai
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN apt-get update && apt-get install -y --no-install-recommends ${BUILD}

RUN mkdir /tmp/opencv && \
    cd /tmp/opencv && \
    wget -O opencv.zip https://github.com/opencv/opencv/archive/${OPENCV_VERSION}.zip && \
    unzip opencv.zip && \
    wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/${OPENCV_VERSION}.zip && \
    unzip opencv_contrib.zip && \
    mkdir /tmp/opencv/opencv-${OPENCV_VERSION}/build
RUN cmake --version

RUN cd /tmp/opencv/opencv-${OPENCV_VERSION}/build && \
    cmake \
    -D CMAKE_BUILD_TYPE=RELEASE \
    -D CMAKE_INSTALL_PREFIX=/usr/local \
    -D OPENCV_EXTRA_MODULES_PATH=/tmp/opencv/opencv_contrib-${OPENCV_VERSION}/modules \
    -D WITH_FFMPEG=OFF \
    -D WITH_GTK=OFF \
    -D WITH_1394=OFF \
    -D WITH_TIFF=OFF \
    -D BUILD_PROTOBUF=ON \
    -D INSTALL_C_EXAMPLES=NO \
    -D INSTALL_PYTHON_EXAMPLES=NO \
    -D BUILD_ANDROID_EXAMPLES=NO \
    -D BUILD_DOCS=NO \
    -D BUILD_TESTS=NO \
    -D BUILD_PERF_TESTS=NO \
    -D BUILD_EXAMPLES=NO \
    -D BUILD_opencv_java=NO \
    -D BUILD_opencv_python=NO \
    -D BUILD_opencv_python2=NO \
    -D BUILD_opencv_python3=NO \
    -D BUILD_SHARED_LIBS=NO \
    -D OPENCV_GENERATE_PKGCONFIG=YES .. && \
    make -j1 && \
    make install && \
    cd && rm -rf /tmp/opencv

ENV CGO_CPPFLAGS -I/usr/local/include/opencv4
ENV PKG_CONFIG_PATH /usr/local/lib/pkgconfig
ENV LD_LIBRARY_PATH /usr/local/lib
ENV CGO_CXXFLAGS "--std=c++1z"
ENV CGO_LDFLAGS "--static -L/usr/local/lib -L/usr/local/lib/opencv4/3rdparty -lopencv_gapi -lopencv_stitching -lopencv_aruco -lopencv_bgsegm -lopencv_bioinspired -lopencv_ccalib -lopencv_dnn_objdetect -lopencv_dnn_superres -lopencv_dpm -lopencv_highgui -lopencv_face -lopencv_freetype -lopencv_fuzzy -lopencv_hfs -lopencv_img_hash -lopencv_intensity_transform -lopencv_line_descriptor -lopencv_mcc -lopencv_quality -lopencv_rapid -lopencv_reg -lopencv_rgbd -lopencv_saliency -lopencv_stereo -lopencv_structured_light -lopencv_phase_unwrapping -lopencv_superres -lopencv_optflow -lopencv_surface_matching -lopencv_tracking -lopencv_datasets -lopencv_text -lopencv_dnn -lopencv_plot -lopencv_videostab -lopencv_videoio -lopencv_xfeatures2d -lopencv_shape -lopencv_ml -lopencv_ximgproc -lopencv_video -lopencv_xobjdetect -lopencv_objdetect -lopencv_calib3d -lopencv_imgcodecs -lopencv_features2d -lopencv_flann -lopencv_xphoto -lopencv_photo -lopencv_imgproc -lopencv_core -littnotify -llibprotobuf -llibwebp -llibopenjp2 -lIlmImf -lquirc -lade -ljpeg -lpng -lz -ltiff -lfreetype -lharfbuzz -ljbig -llzma -lm -lpthread -lrt -lc -ldl"

WORKDIR /tmp
RUN wget https://go.dev/dl/go1.18.linux-armv6l.tar.gz
RUN rm -rf /usr/local/go && tar -C /usr/local -xzf go1.18.linux-armv6l.tar.gz
ENV PATH $PATH:/usr/local/go/bin
RUN go version

arm64:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
FROM arm64v8/ubuntu

LABEL maintainer="Jiangda He. mail: hejiangda@gmail.com"

ENV OPENCV_VERSION=4.5.5

ENV BUILD="unzip \
    wget \
    build-essential \
    cmake \
    curl \
    git \
    libgtk2.0-dev \
    pkg-config \
    libavcodec-dev \
    libavformat-dev \
    libswscale-dev \
    libtbb2 \
    libtbb-dev \
    libjpeg-dev \
    libpng-dev \
    libtiff-dev \
    libdc1394-dev"

ENV TZ=Asia/Shanghai
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN apt-get update && apt-get install -y --no-install-recommends ${BUILD}

RUN mkdir /tmp/opencv && \
    cd /tmp/opencv && \
    wget -O opencv.zip https://github.com/opencv/opencv/archive/${OPENCV_VERSION}.zip && \
    unzip opencv.zip && \
    wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/${OPENCV_VERSION}.zip && \
    unzip opencv_contrib.zip && \
    mkdir /tmp/opencv/opencv-${OPENCV_VERSION}/build
RUN cmake --version

RUN cd /tmp/opencv/opencv-${OPENCV_VERSION}/build && \
    cmake \
    -D CMAKE_BUILD_TYPE=RELEASE \
    -D CMAKE_INSTALL_PREFIX=/usr/local \
    -D OPENCV_EXTRA_MODULES_PATH=/tmp/opencv/opencv_contrib-${OPENCV_VERSION}/modules \
    -D WITH_FFMPEG=OFF \
    -D WITH_GTK=OFF \
    -D WITH_1394=OFF \
    -D WITH_TIFF=OFF \
    -D BUILD_PROTOBUF=ON \
    -D INSTALL_C_EXAMPLES=NO \
    -D INSTALL_PYTHON_EXAMPLES=NO \
    -D BUILD_ANDROID_EXAMPLES=NO \
    -D BUILD_DOCS=NO \
    -D BUILD_TESTS=NO \
    -D BUILD_PERF_TESTS=NO \
    -D BUILD_EXAMPLES=NO \
    -D BUILD_opencv_java=NO \
    -D BUILD_opencv_python=NO \
    -D BUILD_opencv_python2=NO \
    -D BUILD_opencv_python3=NO \
    -D BUILD_SHARED_LIBS=NO \
    -D OPENCV_GENERATE_PKGCONFIG=YES .. && \
    make -j1 && \
    make install && \
    cd && rm -rf /tmp/opencv

ENV CGO_CPPFLAGS -I/usr/local/include/opencv4
ENV PKG_CONFIG_PATH /usr/local/lib/pkgconfig
ENV LD_LIBRARY_PATH /usr/local/lib
ENV CGO_CXXFLAGS "--std=c++1z"

ENV CGO_LDFLAGS "--static -L/usr/local/lib -L/usr/local/lib/opencv4/3rdparty -lopencv_gapi -lopencv_stitching -lopencv_aruco -lopencv_bgsegm -lopencv_bioinspired -lopencv_ccalib -lopencv_dnn_objdetect -lopencv_dnn_superres -lopencv_dpm -lopencv_highgui -lopencv_face -lopencv_freetype -lopencv_fuzzy -lopencv_hfs -lopencv_img_hash -lopencv_intensity_transform -lopencv_line_descriptor -lopencv_mcc -lopencv_quality -lopencv_rapid -lopencv_reg -lopencv_rgbd -lopencv_saliency -lopencv_stereo -lopencv_structured_light -lopencv_phase_unwrapping -lopencv_superres -lopencv_optflow -lopencv_surface_matching -lopencv_tracking -lopencv_datasets -lopencv_text -lopencv_dnn -lopencv_plot -lopencv_videostab -lopencv_videoio -lopencv_xfeatures2d -lopencv_shape -lopencv_ml -lopencv_ximgproc -lopencv_video -lopencv_xobjdetect -lopencv_objdetect -lopencv_calib3d -lopencv_imgcodecs -lopencv_features2d -lopencv_flann -lopencv_xphoto -lopencv_photo -lopencv_imgproc -lopencv_core -littnotify -llibprotobuf -llibwebp -llibopenjp2 -lIlmImf -lquirc -lade -ljpeg -lpng -lz -ltiff -lfreetype -lharfbuzz -ljbig -llzma -lm -lpthread -lrt -lc -ldl"

WORKDIR /tmp
RUN wget https://go.dev/dl/go1.18.linux-arm64.tar.gz
RUN rm -rf /usr/local/go && tar -C /usr/local -xzf go1.18.linux-arm64.tar.gz
ENV PATH $PATH:/usr/local/go/bin
RUN go version

构建好docker镜像后就可以用这两个镜像来打包了，在本地调试运行的话可以先运行相应的镜像然后把源代码拷到容器中，再运行go build就行了。最后用docker cp命令把编译结果复制出来就好了。

江达小记