ld.lld: error: undefined symbol: __chkstk or __rt__blahblah

Problem

E.g.

ld.lld: error: undefined symbol: __rt_udiv64
>>> referenced by sndmix.cpp:122
>>>               .libs/sndmix.o:(_muldiv(long, long, long))
>>> referenced by sndmix.cpp:173
>>>               .libs/sndmix.o:(_muldivr(long, long, long))
>>> referenced by sndmix.cpp:122
>>>               .libs/sndmix.o:(CSoundFile::FadeSong(unsigned int))
>>> referenced 4 more times

ld.lld: error: undefined symbol: __rt_sdiv
>>> referenced by sndmix.cpp:213
>>>               .libs/sndmix.o:(CSoundFile::FadeSong(unsigned int))
>>> referenced by sndmix.cpp:214
>>>               .libs/sndmix.o:(CSoundFile::FadeSong(unsigned int))
>>> referenced by sndmix.cpp:213
>>>               .libs/sndmix.o:(CSoundFile::Read(void*, unsigned int))
>>> referenced 48 more times

Other commonly missing symbols include __chkstk, __memcpy_chk/__memset_chk and like.

Analysis

The undefined symbol diagnostic typically indicates a missing dependency encountered in "no-undefined" linkage mode. When a shared library or an executable is produced, every symbol (unless it's declared weak, i.e. optional) referenced in the program needs to be defined in the same binary unit or imported from a known source. That source must be presented to the linker, and it's usually a dynamic library obtained from a system SDK (example being MinGW itself) or built by some other package.

In some cases, however, the symbol reference (e.g. a procedure call) is not encountered in the code being compiled but inserted by the compiler itself (two opening underscores in the name suggest, by a C/C++ convention, some kind of special treatment). For instance, integer division support varies between architectures, so the compiler would emit either a single div instruction (when targeting an advanced processor) or a call to an implementing div routine in the compiler runtime library (when targeting a less advanced one); and 32-bit architectures aren't considered particularly advanced today. Therefore the runtime library contains an entire integer division framework.

The following find command issued at MXE root displays CLang compiler library builds for various CPU architectures:

$ find usr/lib/clang/ -name '*builtins*'
usr/lib/clang/14.0.0/include/builtins.h
usr/lib/clang/14.0.0/include/__clang_cuda_complex_builtins.h
usr/lib/clang/14.0.0/lib/windows/libclang_rt.builtins-i386.a
usr/lib/clang/14.0.0/lib/windows/libclang_rt.builtins-arm.a
usr/lib/clang/14.0.0/lib/windows/libclang_rt.builtins-x86_64.a
usr/lib/clang/14.0.0/lib/windows/libclang_rt.builtins-aarch64.a

As one can see, the runtime library is CPU and OS specific.

The runtime library is typically provided silently, by a private agreement between the compiler and the linker. However, the project configuration can (and may) request a different runtime library, e.g. the GNU runtime. In this case the procedure calls are still generated, but there is no implementing symbol. A normal long-term solution would begin with finding out why the particular component requests a particular runtime. Until that analysis is carried out, there is a quick hackaround.

Solution

Merging two runtime libraries (GNU and CLang) in the same build is considered harmless by at least some CLang developers, but generally, mixing two low-level libraries implementing similar or overlapping sets of APIs is a recipe for disaster. Fortunately, static libraries are simply object archives. We can extract a particular object and link with our code by passing its name as a linker argument.

In the example above the name of the first missing builtin is __rt_udiv64. Let's examine the runtime library to find the object that defines it:

usr/bin/armv7-w64-mingw32-objdump -t usr/lib/clang/14.0.0/lib/windows/libclang_rt.builtins-arm.a | grep -B 20 __rt_udiv64

-t means "display the symbol table" (we don't need -C — "demangle" — because we are looking for a symbol name as the linker, rather than the C++ code, sees it). -B 20 means "show 20 lines before the substring occurrence" (most often enough in practice, but if you are struggling to find an implementation, increase the number at the cost of output verbosity). If we omit -B for brevity, we would see a few lines like this:

[ 6](sec  1)(fl 0x00)(ty  20)(scl   2) (nx 0) 0x00000000 __rt_udiv64
[ 9](sec  0)(fl 0x00)(ty   0)(scl   2) (nx 0) 0x00000000 __rt_udiv64
[11](sec  0)(fl 0x00)(ty   0)(scl   2) (nx 0) 0x00000000 __rt_udiv64

The one with symbol type 20 (ty 20) is the implementation. With -B we see that it belongs to object… tadam…

usr/lib/clang/14.0.0/lib/windows/libclang_rt.builtins-arm.a(aeabi_uldivmod.S.obj):      file format coff-arm

SYMBOL TABLE:
[ 0](sec  1)(fl 0x00)(ty   0)(scl   3) (nx 1) 0x00000000 .text
AUX scnlen 0x20 nreloc 1 nlnno 0 checksum 0x5543e273 assoc 1 comdat 0
[ 2](sec  2)(fl 0x00)(ty   0)(scl   3) (nx 1) 0x00000000 .data
AUX scnlen 0x0 nreloc 0 nlnno 0 checksum 0x0 assoc 2 comdat 0
[ 4](sec  3)(fl 0x00)(ty   0)(scl   3) (nx 1) 0x00000000 .bss
AUX scnlen 0x0 nreloc 0 nlnno 0 checksum 0x0 assoc 3 comdat 0
[ 6](sec  1)(fl 0x00)(ty  20)(scl   2) (nx 0) 0x00000000 __rt_udiv64

The (initial) fix would be to add the following LDFLAGS to the component $(MAKE) call (adding to ./configure seems redundant; adding to $(MAKE) install, if the latter is invoked separately, is redundant by definition):

LDFLAGS='`$(MXE_INTRINSIC_SH) aeabi_uldivmod.S.obj`'

This is not the end of the story if the procedure in question calls another one residing in a different object file. In that case, we will see another "undefined symbol" message. A repeated search produces the missing dependency object, udivmoddi4.c.obj.

Division builtins

A (seemingly) complete Bash brace enumeration (remember, component build recipes are Bash scripts!) for division would be something like

LDFLAGS='`$(MXE_INTRINSIC_SH) {{aeabi_u{i,l}divmod,udivmodsi4}.S,udivmoddi4.c}.obj`'

It includes signed and unsigned representations, int (32-bit) and long (64-bit) words, the entry point and the facility method.

The __chkstk builtin

On Windows, __chkstk is embedded into procedure calls if the procedure's stack frame (the total stack "slice" its saved/spilled registers and local variables occupy) is larger than a single memory page, and is a way of interaction with the virtual memory dispatcher in the NT kernel. The thread stack size on Windows is dynamic rather than preallocated; it grows on demand. A so-called guard page of nonreadable and nonwritable memory precedes it. When the userspace tries to access the memory range marked as a guard page, a page fault interrupt occurs; control is the transferred to the kernel — that, in turn, allocates real memory in place of the guard page and moves the guard page further down. However, if, as the stack grows down, the callee accesses memory below the guard page (skipping it), this logic is broken. To fix it, __chkstk (inserted by the compiler, who is aware of both the frame size and the page size, and compares the two) "probes" (iteratively accesses) memory in the to-be procedure frame, each time stepping down by a single page.

The __chkstk builtin is provided by a single object: chkstk.S.obj. Put it in the MXE_INTRINSIC_SH arguments alongside the other builtins you need.

__memcpy_chk and __memset_chk

These builtins come into play when the code (e.g. implementing high-grade security) is compiled in FORTIFY_SOURCE mode. They don't come from the compiler runtime library, but from -lssp, which is provided by MinGW. Therefore the extra linker flag would simply be -lssp, without MXE_INTRINSIC_SH.

Implementation notes

MXE_INTRINSIC_SH is defined in the root Makefile and implemented in ./mxe.intrinsic.sh. It outputs the name of a *.lo (libtool object metadata) file that describes the extracted object as position-independent (which is of utter concern to libtool).

Validity

The information in this article was true as of commit 68cd8de8e5f92983099daba0f153508f41a5f875 ("Merge pull request #13").