[Crash-utility] [PATCH 2/5] ci: build the primary arches on ubuntu-26.04 through a reusable workflow

Move the per-arch build into a reusable workflow that can also be dispatched for a single architecture, and build the primary group on ubuntu-26.04: x86_64, x86, aarch64, riscv64 and loongarch64. 26.04 is the first Ubuntu with a loongarch64 cross toolchain, and it is the image ubuntu-latest switches to in November 2026. x86_64 is a plain native build, the other four use the archive's cross toolchains. The embedded gdb refuses to configure without GMP 4.2+ and MPFR 3.1+. Use the archive's libgmp-dev/libmpfr-dev packages where they exist (amd64, i386, arm64, ppc64el, s390x, riscv64), adding a ports.ubuntu.com source that mirrors the image's own suites and keeps every host architecture (26.04 ships amd64v3) enabled. When apt refuses the install because syncing the multiarch versions would downgrade the host, unpack the target .debs instead and point /usr/include and /usr/lib at the copied files. For the architectures the archive does not cover (powerpc64, mips, alpha, sparc64, loongarch64) build gmp 6.3.0 / mpfr 4.2.1 from sha256-verified tarballs, with CFLAGS=-O2 -std=gnu17 because gcc 15 defaults to gnu23 and GMP's configure probes reject it. The apt suite comes from /etc/os-release and the runner image is an input, so other groups can use a different base. The built binary is checked against a probe object from the same compiler, and the compiler version is logged. Also: fail-fast disabled, per-arch concurrency, job timeouts, read-only token permissions, explicit -y/--no-install-recommends apt calls (except for the cross toolchain, whose target libc is only Recommended), and dispatch input validation for a subset of arches. Signed-off-by: Dave Young <yangrr.2009(a)tsinghua.org.cn&gt; Assisted-by: LLM --- .github/workflows/ci-arch-build.yml | 282 ++++++++++++++++++++++++++++ .github/workflows/ci-build.yml | 146 +++++++------- 2 files changed, 345 insertions(+), 83 deletions(-) create mode 100644 .github/workflows/ci-arch-build.yml diff --git a/.github/workflows/ci-arch-build.yml b/.github/workflows/ci-arch-build.yml new file mode 100644 index 000000000000..00820e75bdaf --- /dev/null +++ b/.github/workflows/ci-arch-build.yml @@ -0,0 +1,282 @@ +name: CI build (one arch) + +# Cross compiles crash for a single architecture, or builds it natively when the +# target is the runner's own architecture. Used by the primary and secondary +# matrix workflows, and dispatchable directly to build just one arch. +# +# gmp/mpfr come from the Ubuntu archive when the target architecture has +# packages there (amd64, i386, arm64, ppc64el, s390x, riscv64) and are built +# from source otherwise (powerpc64, mips, alpha, sparc64, loongarch64). +# loongarch64 needs a 26.04 runner: the ubuntu-24.04 archive has no +# loongarch64 cross toolchain. +on: + workflow_call: + inputs: + arch: + description: 'Architecture to cross compile for' + required: true + type: string + runner: + description: 'Runner image label' + required: false + type: string + default: ubuntu-24.04 + workflow_dispatch: + inputs: + arch: + description: 'Architecture to cross compile for' + required: true + type: choice + default: x86_64 + options: + - x86_64 + - x86 + - aarch64 + - riscv64 + - s390x + - powerpc64 + - mips + - alpha + - sparc64 + - powerpc64le + - loongarch64 + runner: + description: 'Runner image label' + required: false + type: choice + default: ubuntu-24.04 + options: + - ubuntu-24.04 + - ubuntu-26.04 + +permissions: + contents: read + +env: + DEBIAN_FRONTEND: noninteractive + +jobs: + build: + name: ${{ inputs.arch }} + runs-on: ${{ inputs.runner }} + timeout-minutes: 45 + # per arch, so the primary and secondary groups never cancel each other + concurrency: + group: ${{ github.workflow }}-${{ github.ref }}-${{ inputs.arch }} + cancel-in-progress: true + steps: + - name: Checkout + uses: actions/checkout@v4 + + # The embedded gdb refuses to configure without GMP >= 4.2 and MPFR >= 3.1, + # so either the Ubuntu multiarch packages or a source build is used. + - name: Set Environment + env: + ARCH: ${{ inputs.arch }} + run: | + # GNU_ARCH cross compiler triplet, also crash's CROSS_COMPILE + # DEB_ARCH dpkg architecture of the target gmp/mpfr packages + # MA_ARCH multiarch tuple the packages install their files under + case $ARCH in + x86) GNU_ARCH=i686-linux-gnu; DEB_ARCH=i386; MA_ARCH=i386-linux-gnu ;; + # the runner is amd64, so x86_64 is a plain native build + x86_64) GNU_ARCH=; DEB_ARCH=amd64; MA_ARCH=x86_64-linux-gnu ;; + aarch64) GNU_ARCH=aarch64-linux-gnu; DEB_ARCH=arm64; MA_ARCH=aarch64-linux-gnu ;; + riscv64) GNU_ARCH=riscv64-linux-gnu; DEB_ARCH=riscv64; MA_ARCH=riscv64-linux-gnu ;; + s390x) GNU_ARCH=s390x-linux-gnu; DEB_ARCH=s390x; MA_ARCH=s390x-linux-gnu ;; + # ppc64el is an Ubuntu architecture: archive packages exist, but the + # Debian cross triplet is powerpc64le-linux-gnu + powerpc64le) GNU_ARCH="$ARCH-linux-gnu"; DEB_ARCH=ppc64el; MA_ARCH="$ARCH-linux-gnu" ;; + loongarch64) GNU_ARCH="$ARCH-linux-gnu"; DEB_ARCH=; MA_ARCH= ;; + *) GNU_ARCH="$ARCH-linux-gnu"; DEB_ARCH=; MA_ARCH= ;; + esac + + { + echo "CROSS_COMPILE=$GNU_ARCH" + echo "DEB_ARCH=$DEB_ARCH" + echo "MA_ARCH=$MA_ARCH" + if [ -n "$GNU_ARCH" ]; then + echo "CROSS_PKGS=gcc-$GNU_ARCH g++-$GNU_ARCH" + fi + } >> "$GITHUB_ENV" + + - name: Install deps for crash-utility building + run: | + sudo apt-get update + sudo apt-get install -y --no-install-recommends \ + make gcc g++ bison flex texinfo wget patch tar build-essential \ + autoconf automake libncurses-dev zlib1g-dev + + # no --no-install-recommends here: gcc-<triplet> only *recommends* the + # target libc (libc6-dev-<target>-cross), which the cross build needs + - name: Install cross compile tool pkgs + if: env.CROSS_PKGS != '' + run: | + sudo apt-get install -y ${{ env.CROSS_PKGS }} + + # amd64, i386, arm64, s390x and riscv64 have target packages in the Ubuntu + # archive; powerpc64 is big-endian (only little-endian ppc64el exists) and + # mips, alpha and sparc64 are not Ubuntu architectures at all. + - name: Install gmp/mpfr from the Ubuntu archive + if: env.DEB_ARCH != '' + run: | + sources=/etc/apt/sources.list.d/ubuntu.sources + keyring=/usr/share/keyrings/ubuntu-archive-keyring.gpg + + # Use the suite this runner actually is, and mirror the exact suite list + # the image uses (including -security): multiarch packages have to match + # the versions already installed, otherwise apt wants to downgrade them. + suite=$(. /etc/os-release && printf '%s' "${VERSION_CODENAME:-}") + if [ -z "$suite" ]; then + echo "cannot determine the Ubuntu suite" >&2 + exit 1 + fi + # union of every suite the image uses (they are split across stanzas, + # so -security would be missed by looking at the first line only) + if [ -f "$sources" ]; then + suites=$( { sed -n 's/^Suites: *//p' "$sources"; echo "$suite $suite-updates"; } | + tr ' ' '\n' | awk 'NF && !seen[$0]++' | paste -sd' ' - ) + else + suites="$suite $suite-updates" + fi + + if [ "$DEB_ARCH" != amd64 ]; then + sudo dpkg --add-architecture "$DEB_ARCH" + + if [ -f "$sources" ]; then + keyring=$(sed -n 's/^Signed-By: //p' "$sources" | head -n 1) + # Keep every host architecture the image enabled (26.04 ships + # amd64v3 next to amd64): restricting this to amd64 drops those + # indexes and makes apt want to downgrade the installed builds. + # The target architecture comes from the foreign source below. + host_archs=$( { dpkg --print-architecture; dpkg --print-foreign-architectures; } | + grep -vx "$DEB_ARCH" | paste -sd' ' - ) + if ! grep -q '^Architectures:' "$sources"; then + sudo sed -i "/^Types: deb\$/a Architectures: $host_archs" "$sources" + fi + fi + + # i386 is served by archive.ubuntu.com, the remaining architectures + # only by ports.ubuntu.com + case $DEB_ARCH in + i386) mirror=http://archive.ubuntu.com/ubuntu/ ;; + *) mirror=http://ports.ubuntu.com/ubuntu-ports/ ;; + esac + printf 'Types: deb\nURIs: %s\nSuites: %s\nComponents: main universe\nArchitectures: %s\nSigned-By: %s\n' \ + "$mirror" "$suites" "$DEB_ARCH" "$keyring" | + sudo tee /etc/apt/sources.list.d/ubuntu-foreign.sources > /dev/null + + sudo apt-get update + fi + + # crash configures the embedded gdb without --with-gmp/--with-mpfr, so + # the files have to be visible in the compiler's default search paths; + # multiarch packages install them under /usr/{include,lib}/<tuple>. + link_from() { + root=${1%/} + # gmp.h is per-architecture (usr/include/<tuple>/), mpfr.h is shared + # (usr/include/), so look in both places + for header in gmp.h mpfr.h; do + src= + for dir in "$root/usr/include/$MA_ARCH" "$root/usr/include"; do + if [ -e "$dir/$header" ]; then + src=$dir/$header + break + fi + done + if [ -n "$src" ] && [ "$src" != "/usr/include/$header" ]; then + sudo ln -sf "$src" "/usr/include/$header" + fi + done + for library in libgmp.so libmpfr.so; do + src= + for dir in "$root/usr/lib/$MA_ARCH" "$root/usr/lib"; do + if [ -e "$dir/$library" ]; then + src=$dir/$library + break + fi + done + if [ -n "$src" ] && [ "$src" != "/usr/lib/$library" ]; then + sudo ln -sf "$src" "/usr/lib/$library" + fi + done + } + + # Prefer a normal install, but apt refuses it when keeping the multiarch + # versions in sync would downgrade the host's amd64v3 packages. In that + # case unpack the two libraries instead: linking only needs the header + # and the shared object, not an installed package. + if sudo apt-get install -y "libgmp-dev:$DEB_ARCH" "libmpfr-dev:$DEB_ARCH"; then + link_from / + else + echo "apt refused the install, unpacking the packages instead" + work=$(mktemp -d) + ( + cd "$work" + apt-get download "libgmp10:$DEB_ARCH" "libgmp-dev:$DEB_ARCH" \ + "libmpfr6:$DEB_ARCH" "libmpfr-dev:$DEB_ARCH" + for deb in ./*.deb; do dpkg-deb -x "$deb" root; done + ) + link_from "$work/root" + fi + + - name: Build gmp from source + if: env.DEB_ARCH == '' + run: | + wget -q --tries=3 --timeout=30 https://ftp.gnu.org/gnu/gmp/gmp-6.3.0.tar.xz + echo "a3c2b80201b89e68616f4ad30bc66aee4927c3ce50e33929ca819d5c43538898 gmp-6.3.0.tar.xz" | sha256sum -c - + tar -xJf gmp-6.3.0.tar.xz + cd gmp-6.3.0 + # gcc 15 defaults to -std=gnu23 and GMP 6.3.0's configure probes still + # use implicit-int style code, so pin the dialect for the build + CFLAGS="-O2 -std=gnu17" ./configure --host="$CROSS_COMPILE" --prefix=/usr + make -j"$(nproc)" + sudo make install + + - name: Build mpfr from source + if: env.DEB_ARCH == '' + run: | + wget -q --tries=3 --timeout=30 https://ftp.gnu.org/gnu/mpfr/mpfr-4.2.1.tar.xz + echo "277807353a6726978996945af13e52829e3abd7a9a5b7fb2793894e18f1fcbb2 mpfr-4.2.1.tar.xz" | sha256sum -c - + tar -xJf mpfr-4.2.1.tar.xz + cd mpfr-4.2.1 + CFLAGS="-O2 -std=gnu17" ./configure --host="$CROSS_COMPILE" --prefix=/usr + make -j"$(nproc)" + sudo make install + + - name: Build crash + run: | + args=() + if [ -n "${CROSS_COMPILE:-}" ]; then + args+=("CROSS_COMPILE=$CROSS_COMPILE-") + fi + make "${args[@]}" -j"$(nproc)" warn + + - name: Check the built binary + run: | + if [ -n "${CROSS_COMPILE:-}" ]; then + compiler="$CROSS_COMPILE-gcc" + else + compiler=gcc + fi + file crash + "$compiler" --version | head -n 1 + printf 'int main(void) { return 0; }\n' > probe.c + "$compiler" -c probe.c -o probe.o + for field in 'Class:' 'Data:' 'Machine:'; do + expected=$(readelf -h probe.o | sed -n "s/^ *$field *//p") + actual=$(readelf -h crash | sed -n "s/^ *$field *//p") + if [ "$expected" != "$actual" ]; then + echo "crash has $field '$actual', the compiler targets '$expected'" + exit 1 + fi + done + rm -f probe.c probe.o + + - name: Clean up for compiling + run: | + args=() + if [ -n "${CROSS_COMPILE:-}" ]; then + args+=("CROSS_COMPILE=$CROSS_COMPILE-") + fi + make "${args[@]}" -j"$(nproc)" clean diff --git a/.github/workflows/ci-build.yml b/.github/workflows/ci-build.yml index 62cb9fef9c29..d94f5562710a 100644 --- a/.github/workflows/ci-build.yml +++ b/.github/workflows/ci-build.yml @@ -1,92 +1,72 @@ name: Build +# Primary architectures: cross compiled on every push to master, on +# ubuntu-26.04, the first Ubuntu with a loongarch64 cross toolchain (and the +# image ubuntu-latest switches to in November 2026). Dispatch with an "arches" +# list to build a subset, or run "CI build (one arch)" for a single +# architecture. on: push: - branches: [ "master" ] + branches: [ master ] + workflow_dispatch: + inputs: + arches: + description: 'Comma-separated subset (empty = all): x86_64 x86 aarch64 riscv64 loongarch64' + required: false + default: '' + workflow_call: + inputs: + arches: + description: 'Comma-separated subset (empty = all)' + required: false + type: string + default: '' -jobs: - build: - name: Build - runs-on: ubuntu-24.04 - strategy: - matrix: - arch: - - x86_64 - - aarch64 - - s390x - - powerpc64 - - x86 - - riscv64 - - mips - - alpha - - sparc64 +permissions: + contents: read +jobs: + plan: + name: Plan + runs-on: ubuntu-latest + timeout-minutes: 5 + outputs: + matrix: ${{ steps.select.outputs.matrix }} steps: - - name: Checkout - uses: actions/checkout@v4 - - - name: Set Environment - env: - ARCH: ${{ matrix.arch }} - run: | - case $ARCH in - x86) GNU_ARCH="i686-linux-gnu" ;; - x86_64) CROSS_COMPILER_PKG="gcc-x86-64-linux-gnu" - CROSS_COMPILER_PKG+=" g++-x86-64-linux-gnu" - GNU_ARCH="$ARCH-linux-gnu" ;; - *) GNU_ARCH="$ARCH-linux-gnu" ;; - esac - - if [ -n "$GNU_ARCH" ]; then - if [ -z "$CROSS_COMPILER_PKG" ]; then - CROSS_COMPILER_PKG="gcc-$GNU_ARCH" - CROSS_COMPILER_PKG+=" g++-$GNU_ARCH" + - name: Select arches + id: select + env: + ALL_ARCHES: x86_64 x86 aarch64 riscv64 loongarch64 + REQUESTED: ${{ inputs.arches }} + run: | + set -eu + if [ -z "$REQUESTED" ]; then + selected=$ALL_ARCHES + else + selected= + for arch in $(echo "$REQUESTED" | tr ',' ' '); do + case " $ALL_ARCHES " in + *" $arch "*) selected="$selected $arch" ;; + *) echo "Unknown architecture '$arch'; valid: $ALL_ARCHES" >&2 + exit 1 ;; + esac + done fi - EXTRA_PKGS+=" $CROSS_COMPILER_PKG" - CROSS_COMPILE="$GNU_ARCH" - fi - - echo "EXTRA_PKGS=$EXTRA_PKGS" >> $GITHUB_ENV - echo "CROSS_COMPILE=$CROSS_COMPILE" >> $GITHUB_ENV - - - name: Install deps for crash-utility building - run: | - sudo apt-get update - sudo apt-get install make gcc g++ bison flex texinfo wget patch tar build-essential libc-dev autoconf automake libncurses-dev - - - name: Install corss compile tool pkgs - if: env.EXTRA_PKGS != '' - run: | - sudo apt-get update - sudo apt-get -q=2 install ${{ env.EXTRA_PKGS }} + if [ -z "${selected// /}" ]; then + echo "No architectures selected" >&2 + exit 1 + fi + matrix=$(printf '%s\n' $selected | jq -R . | jq -s -c '{arch: .}') + echo "matrix=$matrix" + echo "matrix=$matrix" >> "$GITHUB_OUTPUT" - - name: Cross compile gmp lib - run: | - mkdir $GITHUB_WORKSPACE/libtools - wget https://gcc.gnu.org/pub/gcc/infrastructure/gmp-6.2.1.tar.bz2 - tar -jxvf gmp-6.2.1.tar.bz2 - cd gmp-6.2.1 - ./configure --host=${{ env.CROSS_COMPILE }} --prefix=$GITHUB_WORKSPACE/libtools - make -j`nproc` - sudo make install - cd .. - - name: Cross compile mpfr lib - run: | - wget https://gcc.gnu.org/pub/gcc/infrastructure/mpfr-4.1.0.tar.bz2 - tar -jxvf mpfr-4.1.0.tar.bz2 - cd mpfr-4.1.0 - ./configure --host=${{ env.CROSS_COMPILE }} --prefix=$GITHUB_WORKSPACE/libtools --with-gmp=$GITHUB_WORKSPACE/libtools - make -j`nproc` - sudo make install - cd .. - - name: Cross compile crash-utility - run: | - sudo cp $GITHUB_WORKSPACE/libtools/include/* /usr/include/ - sudo cp -r $GITHUB_WORKSPACE/libtools/lib/* /usr/lib/ - make CROSS_COMPILE=${{ env.CROSS_COMPILE }}- -j`nproc` warn - - name: Checking and Clean up - run: | - echo "Run command: file crash ..." - file crash - echo "Clean up for compiling ..." - make CROSS_COMPILE=${{ env.CROSS_COMPILE }}- -j`nproc` clean + build: + name: ${{ matrix.arch }} + needs: plan + strategy: + fail-fast: false + matrix: ${{ fromJson(needs.plan.outputs.matrix) }} + uses: ./.github/workflows/ci-arch-build.yml + with: + arch: ${{ matrix.arch }} + runner: ubuntu-26.04 -- 2.51.1