Skip to content

Latest commit

 

History

History
515 lines (398 loc) · 16.5 KB

INSTALL.md

File metadata and controls

515 lines (398 loc) · 16.5 KB
Raw

bpftrace Install

Linux Kernel Requirements

It is recommended that you are running a Linux 4.9 kernel or higher. Some tools may work on older kernels, but these old kernels are no longer tested. To explain this requirement, these are the kernel versions where major features were added:

  • 4.1 - kprobes
  • 4.3 - uprobes
  • 4.6 - stack traces, count and hist builtins (use PERCPU maps for accuracy and efficiency)
  • 4.7 - tracepoints
  • 4.9 - timers/profiling

Minor improvements have been added in later kernels, so newer than 4.9 is preferred.

Your kernel also needs to be built with the following options:

CONFIG_BPF=y
CONFIG_BPF_SYSCALL=y
CONFIG_BPF_JIT=y
CONFIG_HAVE_EBPF_JIT=y
CONFIG_BPF_EVENTS=y
CONFIG_FTRACE_SYSCALLS=y
CONFIG_FUNCTION_TRACER=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_DYNAMIC_FTRACE=y
CONFIG_HAVE_KPROBES=y
CONFIG_KPROBES=y
CONFIG_KPROBE_EVENTS=y
CONFIG_ARCH_SUPPORTS_UPROBES=y
CONFIG_UPROBES=y
CONFIG_UPROBE_EVENTS=y
CONFIG_DEBUG_FS=y

This can be verified by running the check_kernel_features script from the scripts directory.

Package install

Ubuntu packages

sudo apt-get install -y bpftrace

Should work on Ubuntu 19.04 and later.

On Ubuntu 16.04 and later, bpftrace is also available as a snap package (https://snapcraft.io/bpftrace), however, the snap provides extremely limited file permissions so the --devmode option should be specified on installation in order avoid file access issues.

sudo snap install --devmode bpftrace
sudo snap connect bpftrace:system-trace

The snap package also currently has issues with uprobes (#829).

Fedora package

For Fedora 28 (and later), bpftrace is already included in the official repo. Just install the package with dnf.

sudo dnf install -y bpftrace

Gentoo package

On Gentoo, bpftrace is included in the official repo. The package can be installed with emerge.

sudo emerge -av bpftrace

Debian package

Is available and tracked here.

openSUSE package

Is available and tracked here.

CentOS package

A build maintained by @fbs can be found here.

Arch package

In Arch Linux, bpftrace is available in the official repositories.

sudo pacman -S bpftrace

Alpine package

bpftrace is available in Alpine's official community repository:

sudo apk add bpftrace

To install tools and documentation:

sudo apk add bpftrace-doc bpftrace-tools bpftrace-tools-doc

Docker images

Each push to master will result in a docker image being built and pushed to the quay.io container hosting service. This publishes the docker embedded build linked to glibc, packaged in a minimal ubuntu container.

This allows for such an invocation of bpftrace:

$ docker run -ti -v /usr/src:/usr/src:ro \
       -v /lib/modules/:/lib/modules:ro \
       -v /sys/kernel/debug/:/sys/kernel/debug:rw \
       --net=host --pid=host --privileged \
       quay.io/iovisor/bpftrace:latest \
       tcplife.bt
Attaching 3 probes...
PID   COMM       LADDR           LPORT RADDR           RPORT TX_KB RX_KB MS

The following tags are published for all builds:

  • quay.io/iovisor/bpftrace:${GIT_SHA}-${TYPE_TAG}- eg 69149e94952db2eea579ad40e15fbc67c7b810d5-vanilla_llvm_clang_glibc2.27
  • quay.io/iovisor/bpftrace:${GIT_REF}-${TYPE_TAG}- eg master-vanilla_llvm_clang_glibc2.23 or v0.9.5-vanilla_llvm_clang_glibc2.23

If the build is on the master branch, it also publishes these additional tags:

  • quay.io/iovisor/bpftrace:${GIT_REF}- eg master or v0.9.5
  • quay.io/iovisor/bpftrace:${GIT_SHA}- eg 69149e94952db2eea579ad40e15fbc67c7b810d5
  • quay.io/iovisor/bpftrace:latest

If the build type name ends with _edge, and EDGE=ON is set, and the build is on master, these tags are not pushed, and instead the edge tag is pushed:

  • quay.io/iovisor/bpftrace:edge

This :edge build is likely less stable than :latest or tagged revisions, but builds against bcc master and the latest LLVM supported by bpftrace. The principal goal of the :edge build is to help detect integration issues early, and make all latest features available, but that may also make it less stable for day-to-day or production use.

If using floating tagged images, such as branch tags, :latest, or :edge or :master, it may be necessary to run docker pull explicitly, to ensure that the tag is updated.

The full list of tags can be used to search for tags, and the history of all tags is recorded on quay.io, and the distributed images are regularly scanned by a vulnerability scanner.

Copying bpftrace binary from docker

As docker builds produce a bpftrace binary on every push to master, they also allow for a convenient way to distribute bpftrace binaries. The only software requirement to run bpftrace is a version of glibc that is the same or newer as what it was built at.

For this reason, an older glibc - 2.23 is provided for all builds, it can be pulled with:

docker pull quay.io/iovisor/bpftrace:master-vanilla_llvm_clang_glibc2.23

To copy the binary out of bpftrace in the current directory:

$ docker run -v $(pwd):/output quay.io/iovisor/bpftrace:master-vanilla_llvm_clang_glibc2.23 \
  /bin/bash -c "cp /usr/bin/bpftrace /output"
$ ./bpftrace -V
v0.9.4

bpftrace currently links to glibc 2.27 from Ubuntu Bionic by default, though this should be portably to any glibc-based OS, such as Fedora, Chromium OS, etc.

Kernel headers install

Usually kernels headers can be installed from a system package manager. In some cases though, this may not be an option, and headers aren't easily available. For instance, the default docker desktop (as of writing ships with kernel 4.19 which supports bpf), benefits from this, as does Chromium OS and other lightweight Linux distributions.

Newer kernels may have the IKHEADERS option, or support btf - in which case there is no need to build these headers as the kernel provides this. For older kernels, and on distributions where headers may not be available, this script provides a generic means to get bpftrace kernel headers:

#!/bin/bash

set -e

KERNEL_VERSION="${KERNEL_VERSION:-$(uname -r)}"
kernel_version="$(echo "${KERNEL_VERSION}" | awk -vFS=- '{ print $1 }')"
major_version="$(echo "${KERNEL_VERSION}" | awk -vFS=. '{ print $1 }')"

apt-get install -y build-essential bc curl flex bison libelf-dev

mkdir -p /usr/src/linux
curl -sL "https://www.kernel.org/pub/linux/kernel/v${major_version}.x/linux-$kernel_version.tar.gz" \
  | tar --strip-components=1 -xzf - -C /usr/src/linux
cd /usr/src/linux
zcat /proc/config.gz > .config
make ARCH=x86 oldconfig
make ARCH=x86 prepare
mkdir -p /lib/modules/$(uname -r)
ln -sf /usr/src/linux /lib/modules/$(uname -r)/source
ln -sf /usr/src/linux /lib/modules/$(uname -r)/build

Building bpftrace

Vendored libraries

The bpftrace tree include bcc and libbpf submodules. Use git submodule init && git submodule update --recursive to initialize them (after having cloned the repo) or use git clone --recurse-submodules https://github.com/iovisor/bpftrace when first cloning the bpftrace repo.

By default the build system will look for these libraries as built in the bpftrace tree. If you want to use the system ones, pass -DUSE_SYSTEM_BPF_BCC=1 to your cmake invocation.

Ubuntu

Due to the kernel requirements Ubuntu 18.04 or newer is highly recommended.

18.04 and 18.10

The versions of bcc currently available in Ubuntu 18.04 (Bionic) and 18.10 (Cosmic) do not have all the requirements for building bpftrace so building bcc first is required. The instructions for building bcc can be found here. The build dependencies listed below are also required for bcc so install those first.

Make sure bcc works by testing some of the shipped tools before proceeding. It might be required to ldconfig to update the linker.

19.04 and newer

The version of bcc available in Ubuntu 19.04 (Disco) is new enough so compilation is not required, install with:

sudo apt-get install -y libbpfcc-dev

Building bpftrace

sudo apt-get update
sudo apt-get install -y \
  bison \
  cmake \
  flex \
  g++ \
  git \
  libelf-dev \
  zlib1g-dev \
  libfl-dev \
  systemtap-sdt-dev \
  binutils-dev \
  libcereal-dev \
  llvm-dev \
  llvm-runtime \
  libclang-dev \
  clang \
  libpcap-dev \
  libgtest-dev \
  libgmock-dev \
  asciidoctor \
  libdw-dev \
  pahole
git clone https://github.com/iovisor/bpftrace --recurse-submodules
mkdir bpftrace/build; cd bpftrace/build;
../build-libs.sh
cmake -DCMAKE_BUILD_TYPE=Release ..
make -j8
sudo make install

The bpftrace binary will be in installed in /usr/local/bin/bpftrace, and tools in /usr/local/share/bpftrace/tools. You can change the install location using an argument to cmake, where the default is -DCMAKE_INSTALL_PREFIX=/usr/local.

Fedora

You'll want the newest kernel possible (see kernel requirements), eg, by using Fedora 28 or newer.

sudo dnf install -y bison \
  flex \
  cmake \
  make \
  git \
  gcc-c++ \
  xxd \
  libffi-devel \
  elfutils-libelf-devel \
  elfutils-devel \
  zlib-devel \
  xz-devel \
  llvm-devel \
  clang-devel \
  bcc-devel \
  systemtap-sdt-devel \
  binutils-devel \
  libbpf-devel \
  libpcap-devel \
  gtest-devel \
  gmock-devel \
  cereal-devel \
  asciidoctor \
  dwarves
git clone https://github.com/iovisor/bpftrace --recurse-submodules
cd bpftrace
mkdir build; cd build
../build-libs.sh
cmake -DCMAKE_BUILD_TYPE=Release ..
make -j8
sudo make install

The bpftrace binary will be in installed in /usr/local/bin/bpftrace, and tools in /usr/local/share/bpftrace/tools. You can change the install location using an argument to cmake, where the default is -DCMAKE_INSTALL_PREFIX=/usr/local.

Amazon Linux

In the future the install should be yum install bpftrace. Right now (16-Oct-2018), however, three dependencies need updating in the Amazon Linux repositories (llvm, libtinfo, bison), and bpftrace itself needs to be packaged. The current workaround is to build the three dependencies manually, as well as bpftrace. It's not fun, but it is doable, and will only get better as Amazon updates things.

sudo bash
builddir=/media/ephemeral0	# change to suit your system: needs about 2 Gbytes free

# dependencies
yum install git cmake3 gcc64-c++.x86_64 bison flex

# llvm
cd $builddir
wget http://releases.llvm.org/6.0.0/clang+llvm-6.0.0-x86_64-linux-gnu-Fedora27.tar.xz
tar xf clang*
(cd clang* && sudo cp -R * /usr/local/)
cp -p /usr/lib64/llvm6.0/lib/libLLVM-6.0.so /usr/lib64/libLLVM.so

# libtinfo.so.6 (comes from ncurses)
cd $builddir
wget ftp://ftp.gnu.org/gnu/ncurses/ncurses-6.0.tar.gz
tar xvf ncurses-6.0.tar.gz
cd ncurses-6.0
./configure --with-shared --with-termlib
make -j8
make install

# bison
cd $builddir
wget http://ftp.gnu.org/gnu/bison/bison-3.1.tar.gz
tar xf bison*
cd bison*
./configure
make -j4
make install

# bpftrace
cd $builddir
git clone https://github.com/iovisor/bpftrace
cd bpftrace
mkdir build; cd build
../build-libs.sh
cmake3 ..
make -j8
make install
echo /usr/local/lib >> /etc/ld.so.conf
ldconfig -v

The bpftrace binary will be in installed in /usr/local/bin/bpftrace, and tools in /usr/local/share/bpftrace/tools. You may need to add /usr/local/bin to your $PATH. You can also change the install location using an argument to cmake, where the default is -DCMAKE_INSTALL_PREFIX=/usr/local.

Using Docker

There are currently problems with bpftrace string comparisons when using the Docker build. The regular build is recommended for now.

Building inside a Docker container will produce a statically linked bpftrace executable.

./build.sh

There are some more fine-grained options if you find yourself building bpftrace a lot:

  • ./build-docker-image.sh - builds just the bpftrace-builder Docker image
  • ./build-debug.sh - builds bpftrace with debugging information (requires ./build-docker-image.sh to have already been run)
  • ./build-release.sh - builds bpftrace in a release configuration (requires ./build-docker-image.sh to have already been run)

./build.sh is equivalent to ./build-docker-image.sh && ./build-release.sh

Generic build process

Use specific OS build sections listed earlier if available (Ubuntu, Docker).

Requirements

  • A C++ compiler
  • CMake
  • Flex
  • Bison
  • LLVM & Clang 10.0+ development packages
  • LibElf
  • Binutils development package
  • Libcereal
  • Kernel requirements described earlier
  • Libpcap

The bpftrace repository contains submodules for appropriate libbpf and bcc versions, which will be used unless -DUSE_SYSTEM_BPF_BCC=1 is passed to cmake.

If using USE_SYSTEM_BPF_BCC, the current required versions are:

  • BCC development package - v0.25.0
  • libbpf - v0.8.1

Compilation

git clone https://github.com/iovisor/bpftrace
mkdir -p bpftrace/build
cd bpftrace/build
../build-libs.sh
cmake -DCMAKE_BUILD_TYPE=Release ../
make

If using system bcc and libbpf libraries, it would look like this:

git clone https://github.com/iovisor/bpftrace
mkdir -p bpftrace/build
cd bpftrace/build
cmake -DUSE_SYSTEM_BPF_BCC=1 -DCMAKE_BUILD_TYPE=Release ../
make

By default bpftrace will be built as a dynamically linked executable (except for vendored libraries, if used). If a statically linked executable would be preferred and your system has the required libraries installed, the CMake option -DSTATIC_LINKING:BOOL=ON can be used. Building bpftrace using the alpine Docker image below will result in a statically linked executable, and additional flags allow for compiling and statically linking the dependencies of bpftrace, see the embedded build docs for more about this type of build. A debug build of bpftrace can be set up with cmake -DCMAKE_BUILD_TYPE=Debug ../.

The latest version of Google Test will be downloaded on each build. To speed up builds and only download its source on the first run, use the CMake option -DOFFLINE_BUILDS:BOOL=ON.

To test that the build works, you can try running the test suite, and a one-liner:

./tests/bpftrace_test
./src/bpftrace -e 'kprobe:do_nanosleep { printf("sleep by %s\n", comm); }'

Disable Lockdown

From the original patch set description:

This patchset introduces an optional kernel lockdown feature, intended to strengthen the boundary between UID 0 and the kernel. When enabled, various pieces of kernel functionality are restricted. Applications that rely on low-level access to either hardware or the kernel may cease working as a result - therefore this should not be enabled without appropriate evaluation beforehand.

The majority of mainstream distributions have been carrying variants of this patchset for many years now, so there's value in providing a doesn't meet every distribution requirement, but gets us much closer to not requiring external patches.

When lockdown is enabled and set to 'confidentiality' all methods that can extract confidential data from the kernel are blocked. This means that:

  • kprobes are blocked
  • tracefs access is blocked
  • probe_read and probe_read_str are blocked

which makes it impossible for bpftrace to function.

There are a few ways to disable lockdown.

  1. Disable secure boot in UEFI.
  2. Disable validation using mokutil, run the following command, reboot and follow the prompt.
$ sudo mokutil --disable-validation
  1. Use the SysRQ+x key combination to temporarily lift lockdown (until next boot)

Note that you may encounter kernel lockdown error if you install bpftrace via snap incorrectly. Please refer to Ubuntu for more details regrading how to use snap to install bpftrace.