Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(build): static build might require zstd lib #4059

Merged
merged 3 commits into from
May 21, 2024
Merged

fix(build): static build might require zstd lib #4059

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e40109f
fix(build): static build might require zstd lib
geyslan May 21, 2024
656df9a
chore: install required bpftool from btfhub
geyslan May 21, 2024
ac9bdb1
chore: temp fix for install libzstd on AMIs
geyslan May 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/actions/build-dependencies/action.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ runs:
run: |
sudo apt-get update
sudo apt-get install -y bsdutils build-essential pkgconf
sudo apt-get install -y zlib1g-dev libelf-dev
sudo apt-get install -y zlib1g-dev libelf-dev libzstd-dev
sudo apt-get install -y software-properties-common
shell: bash
- name: Install Golang
Expand Down
11 changes: 9 additions & 2 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ CMD_CONTROLLER_GEN ?= controller-gen

LIB_ELF ?= libelf
LIB_ZLIB ?= zlib
LIB_ZSTD ?= libzstd

define pkg_config
$(CMD_PKGCONFIG) --libs $(1)
Expand Down Expand Up @@ -172,6 +173,7 @@ env:
@echo ---------------------------------------
@echo "LIB_ELF $(LIB_ELF)"
@echo "LIB_ZLIB $(LIB_ZLIB)"
@echo "LIB_ZSTD $(LIB_ZSTD)"
@echo ---------------------------------------
@echo "VERSION $(VERSION)"
@echo "LAST_GIT_TAG $(LAST_GIT_TAG)"
Expand Down Expand Up @@ -316,7 +318,7 @@ $(OUTPUT_DIR)/btfhub:
#

LIBBPF_CFLAGS = "-fPIC"
LIBBPF_LDLAGS =
LIBBPF_LDFLAGS =
LIBBPF_SRC = ./3rdparty/libbpf/src

$(OUTPUT_DIR)/libbpf/libbpf.a: \
Expand Down Expand Up @@ -392,7 +394,10 @@ TRACEE_SRC_DIRS = ./cmd/ ./pkg/ ./signatures/
TRACEE_SRC = $(shell find $(TRACEE_SRC_DIRS) -type f -name '*.go' ! -name '*_test.go')

CUSTOM_CGO_CFLAGS = "-I$(abspath $(OUTPUT_DIR)/libbpf)"
CUSTOM_CGO_LDFLAGS = "$(shell $(call pkg_config, $(LIB_ELF))) $(shell $(call pkg_config, $(LIB_ZLIB))) $(abspath $(OUTPUT_DIR)/libbpf/libbpf.a)"
CUSTOM_CGO_LDFLAGS = "$(shell $(call pkg_config, $(LIB_ELF))) \
$(shell $(call pkg_config, $(LIB_ZLIB))) \
$(shell $(call pkg_config, $(LIB_ZSTD))) \
$(abspath $(OUTPUT_DIR)/libbpf/libbpf.a)"

GO_ENV_EBPF =
GO_ENV_EBPF += GOOS=linux
Expand Down Expand Up @@ -439,6 +444,7 @@ $(OUTPUT_DIR)/tracee: \
| .checkver_$(CMD_GO) \
.checklib_$(LIB_ELF) \
.checklib_$(LIB_ZLIB) \
.checklib_$(LIB_ZSTD) \
btfhub \
signatures
#
Expand Down Expand Up @@ -472,6 +478,7 @@ $(OUTPUT_DIR)/tracee-ebpf: \
| .checkver_$(CMD_GO) \
.checklib_$(LIB_ELF) \
.checklib_$(LIB_ZLIB) \
.checklib_$(LIB_ZSTD) \
btfhub
#
$(MAKE) $(OUTPUT_DIR)/btfhub
Expand Down
2 changes: 1 addition & 1 deletion Vagrantfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ Vagrant.configure("2") do |config|
ln -s "$path" "${path%-*}"
done

apt-get install --yes zlib1g-dev libelf-dev
apt-get install --yes zlib1g-dev libelf-dev libzstd-dev
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we discussed @rscampos, the setup of the bpftool in Vagrantfile must be done in a further PR (since it requires a system upgrade). Thanks for noticing that. 👍🏼

Copy link
Collaborator

@rscampos rscampos May 21, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll upgrade Vagrantfile in order use compiled version of bpftool :)

apt-get install --yes protobuf-compiler
apt-get install --yes linux-tools-"$(uname -r)" ||
apt-get install --yes linux-tools-generic
Expand Down
10 changes: 3 additions & 7 deletions builder/Dockerfile.alpine-tracee-container
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ USER root
RUN apk --no-cache update && \
apk --no-cache add coreutils && \
apk --no-cache add sudo curl && \
apk --no-cache add libelf zlib && \
apk --no-cache add libelf zlib zstd && \
apk --no-cache add libc6-compat

# install OPA
Expand Down Expand Up @@ -47,6 +47,7 @@ RUN apk --no-cache update && \
apk --no-cache add elfutils-dev && \
apk --no-cache add libelf-static && \
apk --no-cache add zlib-static && \
apk --no-cache add zstd-static && \
rm -f /usr/bin/cc && \
rm -f /usr/bin/clang && \
rm -f /usr/bin/clang++ && \
Expand Down Expand Up @@ -85,12 +86,7 @@ RUN apk --no-cache update && \
RUN cd /tmp && \
git clone https://github.com/aquasecurity/btfhub.git && \
cd ./btfhub && \
git submodule update --init --recursive 3rdparty/bpftool && \
cd ./3rdparty/bpftool && \
make -C src clean && \
make -C src all && \
cp ./src/bpftool /usr/sbin/bpftool && \
make -C src clean
./3rdparty/bpftool.sh

#
# tracee-make
Expand Down
8 changes: 8 additions & 0 deletions builder/Dockerfile.alpine-tracee-make
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ RUN apk --no-cache update && \
apk --no-cache add elfutils-dev && \
apk --no-cache add libelf-static && \
apk --no-cache add zlib-static && \
apk --no-cache add zstd-static && \
rm -f /usr/bin/cc && \
rm -f /usr/bin/clang && \
rm -f /usr/bin/clang++ && \
Expand Down Expand Up @@ -52,6 +53,13 @@ RUN apk --no-cache update && \
ln -s /usr/lib/llvm14/bin/llvm-readelf /usr/bin/llvm-readelf && \
ln -s /usr/lib/llvm14/bin/opt /usr/bin/opt

# install bpftool from btfhub

RUN cd /tmp && \
git clone https://github.com/aquasecurity/btfhub.git && \
cd ./btfhub && \
./3rdparty/bpftool.sh

# install OPA

RUN altarch=$(uname -m | sed 's:x86_64:amd64:g' | sed 's:aarch64:arm64:g') && \
Expand Down
8 changes: 8 additions & 0 deletions builder/Dockerfile.ubuntu-tracee-make
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,17 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
apt-get install -y linux-headers-generic && \
apt-get install -y libelf-dev && \
apt-get install -y zlib1g-dev && \
apt-get install -y libzstd-dev && \
update-alternatives --install /usr/bin/clang clang /usr/bin/clang-12 130 --slave /usr/bin/clang++ clang++ /usr/bin/clang++-12 --slave /usr/bin/llc llc /usr/bin/llc-12 --slave /usr/bin/clang-format clang-format /usr/bin/clang-format-12 --slave /usr/bin/clangd clangd /usr/bin/clangd-12 && \
update-alternatives --install /usr/bin/clang clang /usr/bin/clang-14 140 --slave /usr/bin/clang++ clang++ /usr/bin/clang++-14 --slave /usr/bin/llc llc /usr/bin/llc-14 --slave /usr/bin/clang-format clang-format /usr/bin/clang-format-14 --slave /usr/bin/clangd clangd /usr/bin/clangd-14

# install bpftool from btfhub

RUN cd /tmp && \
git clone https://github.com/aquasecurity/btfhub.git && \
cd ./btfhub && \
./3rdparty/bpftool.sh

# install OPA

RUN altarch=$(uname -m | sed 's:x86_64:amd64:g' | sed 's:aarch64:arm64:g') && \
Expand Down
5 changes: 3 additions & 2 deletions docs/contributing/building/building.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,11 @@

1. **clang** && **llvm** (12, 13 or 14)
1. **golang** (1.21)
1. **libelf** and **libelf-dev**
1. **libelf** and **libelf-dev**
(or elfutils-libelf and elfutils-libelf-devel)
1. **zlib1g** and **zlib1g-dev**
1. **zlib1g** and **zlib1g-dev**
(or zlib and zlib-devel)
1. **libzstd-dev** for static build (libelf linkage)
1. **clang-format-12** (specific version) for `fix-fmt`

> You might take a look at the following files to understand how to have a
Expand Down
19 changes: 19 additions & 0 deletions tests/e2e-install-deps.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,22 @@ remove_golang_os_packages() {
apt-get --purge autoremove -y
}

install_libzstd_os_packages() {
case $ID in
"ubuntu")
wait_for_apt_locks
apt-get install -y libzstd-dev
;;
"almalinux")
yum install -y libzstd-devel
;;
*)
echo "Unsupported OS: $ID"
exit 1
;;
esac
}

Comment on lines +267 to +282
Copy link
Member Author

@geyslan geyslan May 21, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All this would be unnecessary if we opted for an easier way of updating the AMIs bunch in a single shot. Anyway, after its future upgrade, those steps should be commented/removed.

# Main logic.

# Note: I left commented out the commands that would (re)install clang-14. This
Expand Down Expand Up @@ -328,3 +344,6 @@ if [[ $ID == "almalinux" ]]; then
remove_golang_usr_bin_files
install_golang_from_github
fi

# for static builds libelf might require libzstd
install_libzstd_os_packages
Loading