add shared library support #338

dicej · 2023-07-28T17:06:47Z

This adds support for building WASI shared libraries per https://github.com/WebAssembly/tool-conventions/blob/main/DynamicLinking.md.

For the time being, the goal is to allow "pseudo-dynamic" linking using the Component Model per
https://github.com/WebAssembly/component-model/blob/main/design/mvp/examples/SharedEverythingDynamicLinking.md. This requires all libraries to be available when the component is created, but still allows runtime symbol resolution via dlopen/dlsym backed by a static lookup table. This is sufficient to support Python native extensions, for example. A complete demo using wit-component is available at https://github.com/dicej/component-linking-demo.

This requires https://reviews.llvm.org/D153293, which we will need to backport to LLVM 16 until 17 is released, hence the llvm-D153293-backport.patch file.

.gitmodules

Makefile

yamt · 2023-09-29T02:11:21Z

i prefer not to release a binary with __wasm_call_ctors.
cf. dicej/component-linking-demo#3

dicej · 2023-09-29T20:03:37Z

I've rebased onto main and removed the patch. @yamt does that address your concern regarding __wasm_call_ctors?

src/llvm-project

yamt · 2023-09-30T03:25:49Z

I've rebased onto main and removed the patch. @yamt does that address your concern regarding __wasm_call_ctors?

no.
the simplest fix would be to include https://reviews.llvm.org/D156205, which is unfortunately not included in LLVM 17.

yamt · 2023-09-30T03:46:28Z

wasi-sdk.cmake

@@ -38,3 +38,5 @@ set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
 set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
 set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
 set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
+
+set(CMAKE_POSITION_INDEPENDENT_CODE On)


i'm not sure how these cmake files are supposed to be used wrt shared library. can you explain a bit?
is it intended to always build pic even if shared lib is not used? why?

I think I added CMAKE_POSITION_INDEPENDENT_CODE to LIBCXX_CMAKE_FLAGS in the Makefile, but it wasn't working (i.e. libc++.so and libc++abi.so still contained position-dependent code), so I added the lines to wasi-sdk.cmake and wasi-sdk-pthread.cmake. However, I just tried moving it back to Makefile, it that works fine now, so I'll push an update.

dicej · 2023-10-03T19:42:03Z

I've rebased onto main and removed the patch. @yamt does that address your concern regarding __wasm_call_ctors?

no. the simplest fix would be to include https://reviews.llvm.org/D156205, which is unfortunately not included in LLVM 17.

How does this plan sound to you?

Create a new wasi-libc PR to remove the -nostdlib parameter from libc.so link command (i.e. let wasm-ld link in crt-reactor.o).
Add a llvm-D156205-backport.patch to this PR and apply it as part of the build/llvm.BUILT target.

sbc100 · 2023-10-03T20:15:15Z

I've rebased onto main and removed the patch. @yamt does that address your concern regarding __wasm_call_ctors?

no. the simplest fix would be to include https://reviews.llvm.org/D156205, which is unfortunately not included in LLVM 17.

Its possible you could get that patch backported to the 17 branch.. it seems very low risk.

How does this plan sound to you?

Create a new wasi-libc PR to remove the -nostdlib parameter from libc.so link command (i.e. let wasm-ld link in crt-reactor.o).

Add a llvm-D156205-backport.patch to this PR and apply it as part of the build/llvm.BUILT target.

dicej · 2023-10-03T20:43:28Z

More generally, I wonder if we should follow the Rust project's example and create a WebAssembly/llvm-project fork as a home for backports (cf. https://github.com/rust-lang/llvm-project). I do agree that backporting upstream is ideal whenever possible, but it would be nice to have a fallback option that doesn't involve applying patches at build time.

sbc100 · 2023-10-03T20:55:24Z

More generally, I wonder if we should follow the Rust project's example and create a WebAssembly/llvm-project fork as a home for backports (cf. https://github.com/rust-lang/llvm-project). I do agree that backporting upstream is ideal whenever possible, but it would be nice to have a fallback option that doesn't involve applying patches at build time.

Personally, I think the fact that we build wasi-sdk directly from upstream llvm is a really nice feature. It means the sdk is truly just a vanilla clang. Nothing more and nothing less.

wasi-libc also supports a range of stable llvm releases which is really nice. Folks can take wasi-libc and use it with their own pre-built llvm binaries.

If wasi-sdk started to diverge from vanilla llvm I think it could cause confusion and mean that such users would not have the same experience as wasi-sdk users.

yamt · 2023-10-06T06:15:48Z

Its possible you could get that patch backported to the 17 branch.. it seems very low risk.

i tried to make a backport request, following https://www.llvm.org/docs/GitHub.html#backporting-fixes-to-the-release-branches.
but i couldn't create an issue associated to the milestone.
does it require some special permission?

dicej · 2023-10-06T17:37:45Z

FYI, I've opened WebAssembly/wasi-libc#440 to remove the -nostdlib flag from the libc.so link command. Once that's merged, and the LLVM patch is backported, I can update both submodules in this PR.

tschneidereit · 2023-11-09T16:42:47Z

@sbc100 do you have any advice on the issue @yamt ran into in trying to backport the patch to the 17 branch?

sbc100 · 2023-11-09T17:01:18Z

@sbc100 do you have any advice on the issue @yamt ran into in trying to backport the patch to the 17 branch?

I don't think I've tried to backport anything since the switch to github, but it seems unlikely that they wouldn't accept request from external folks.

@yamt can you link to the issue you created and I can see if I can mark it correctly?

yamt · 2023-11-10T02:47:42Z

@sbc100 do you have any advice on the issue @yamt ran into in trying to backport the patch to the 17 branch?

I don't think I've tried to backport anything since the switch to github, but it seems unlikely that they wouldn't accept request from external folks.

@yamt can you link to the issue you created and I can see if I can mark it correctly?

llvm/llvm-project#68385
llvm/llvm-project#68386
llvm/llvm-project#68387
(multiple issues just because i made a few attempts)

sbc100 · 2023-11-10T03:18:15Z

Looks like I was able to add the Milestone to llvm/llvm-project#68387

yamt · 2023-11-10T04:52:13Z

Looks like I was able to add the Milestone to llvm/llvm-project#68387

thank you

dicej · 2023-11-16T14:54:18Z

@sbc100 In case you missed it, looks like this is waiting on confirmation from you: llvm/llvm-project-release-prs#773 (comment)

dicej · 2023-11-26T16:24:49Z

I've updated both the wasi-libc and llvm-project submodules (including llvm-D156205), so I think this is ready to merge.

dicej · 2023-11-29T14:20:08Z

Looks like the CI failure was due to a spurious network issue. Would someone who has the appropriate permissions please re-run the failed job?

sbc100 · 2023-11-29T16:50:48Z

Can we do the llvm update as a separate PR maybe?

sbc100 · 2023-11-29T16:51:13Z

Restarted failed CI jobs

yamt · 2023-12-07T08:09:34Z

Can we do the llvm update as a separate PR maybe?

@dicej do you want to make a separate PR?
or, i can reopen and update #359

dicej · 2023-12-07T16:04:47Z

Can we do the llvm update as a separate PR maybe?

@dicej do you want to make a separate PR? or, i can reopen and update #359

Done: #362

abrown · 2023-12-08T18:34:36Z

Ok, it seems to me that we just need to remove the LLVM submodule update from this PR (since it happened in #362) and then this should be good to go?

This adds support for building WASI shared libraries per https://github.com/WebAssembly/tool-conventions/blob/main/DynamicLinking.md. For the time being, the goal is to allow "pseudo-dynamic" linking using the Component Model per https://github.com/WebAssembly/component-model/blob/main/design/mvp/examples/SharedEverythingDynamicLinking.md. This requires all libraries to be available when the component is created, but still allows runtime symbol resolution via `dlopen`/`dlsym` backed by a static lookup table. This is sufficient to support Python native extensions, for example. A complete demo using `wit-component` is available at https://github.com/dicej/component-linking-demo. This requires https://reviews.llvm.org/D153293, which we will need to backport to LLVM 16 we're ready to upgrade to LLVM 17, hence the llvm-D153293-backport.patch file. Signed-off-by: Joel Dice <[email protected]>

This is where I originally meant the directive to go, but it didn't seem to work the first time I tried it, hence the added lines in wasi-sdk.cmake and wasi-sdk-pthread.cmake. However, it seems to work fine now, so I'm switching it back to be consistent with other similar CMake flags. Signed-off-by: Joel Dice <[email protected]>

Signed-off-by: Joel Dice <[email protected]>

dicej · 2023-12-08T19:12:27Z

I just force-pushed a rebase onto main.

sbc100

lgtm now!

(If I had my druthers to wasm-libc updated would also land separately..)

sbc100 · 2023-12-11T21:10:09Z

Makefile

 	$(MAKE) -C $(ROOT_DIR)/src/wasi-libc \
 		CC=$(BUILD_PREFIX)/bin/clang \
 		AR=$(BUILD_PREFIX)/bin/llvm-ar \
 		NM=$(BUILD_PREFIX)/bin/llvm-nm \
 		SYSROOT=$(BUILD_PREFIX)/share/wasi-sysroot \
+		BUILTINS_LIB=$(BUILD_PREFIX)/lib/clang/$(CLANG_VERSION)/lib/wasi/libclang_rt.builtins-wasm32.a \


Why is BUILTINS_LIB now needed where it wasn't before?

It's needed for the libc.so link step to succeed, but it looks like WebAssembly/wasi-libc#442 has removed the need to specify it explicitly, so I'll remove it.

sbc100 · 2023-12-11T21:10:23Z

.gitmodules

@@ -3,7 +3,7 @@
 	url = https://github.com/llvm/llvm-project
 [submodule "src/wasi-libc"]
 	path = src/wasi-libc
-	url = https://github.com/CraneStation/wasi-libc
+	url = https://github.com/WebAssembly/wasi-libc


Seems unrelated.. but a good change.

As of WebAssembly/wasi-libc#442, these are determined automatically. Signed-off-by: Joel Dice <[email protected]>

dicej force-pushed the dynamic-linking branch from e5b8dd8 to e288a08 Compare July 28, 2023 17:07

dicej commented Jul 28, 2023

View reviewed changes

.gitmodules Outdated Show resolved Hide resolved

TerrorJack reviewed Jul 28, 2023

View reviewed changes

Makefile Outdated Show resolved Hide resolved

dicej mentioned this pull request Jul 28, 2023

wasi-libc upstreaming? dicej/component-linking-demo#4

Open

dicej commented Jul 28, 2023

View reviewed changes

Makefile Outdated Show resolved Hide resolved

This was referenced Aug 22, 2023

Upgrade Clang in order to allow using the -shared option #199

Closed

relocation R_WASM_MEMORY_ADDR_LEB cannot be used against symbol errno; recompile with -fPIC #209

Closed

yamt mentioned this pull request Sep 28, 2023

release request #348

Closed

dicej force-pushed the dynamic-linking branch from 8f9aa2e to f014077 Compare September 28, 2023 17:14

dicej marked this pull request as ready for review September 28, 2023 17:15

dicej force-pushed the dynamic-linking branch from f014077 to f16150e Compare September 29, 2023 19:58

dicej force-pushed the dynamic-linking branch 2 times, most recently from c8876db to e6e8ec4 Compare September 29, 2023 20:08

abrown reviewed Sep 29, 2023

View reviewed changes

src/llvm-project Outdated Show resolved Hide resolved

dicej force-pushed the dynamic-linking branch from e6e8ec4 to fdd022f Compare September 30, 2023 01:56

yamt reviewed Sep 30, 2023

View reviewed changes

dicej mentioned this pull request Nov 8, 2023

add stubs for dlopen, dlsym, etc. WebAssembly/wasi-libc#443

Merged

yamt mentioned this pull request Nov 29, 2023

Bump llvm version to 17.0.4 #359

Closed

dicej added 3 commits December 8, 2023 12:11

update wasi-libc to main and llvm-project to release/17.x

3e0a45a

Signed-off-by: Joel Dice <[email protected]>

dicej force-pushed the dynamic-linking branch from b9743ab to 3e0a45a Compare December 8, 2023 19:11

sbc100 approved these changes Dec 11, 2023

View reviewed changes

remove explicit BUILTINS_LIB values

64cd0c0

As of WebAssembly/wasi-libc#442, these are determined automatically. Signed-off-by: Joel Dice <[email protected]>

sbc100 approved these changes Dec 11, 2023

View reviewed changes

abrown merged commit a35b453 into WebAssembly:main Dec 12, 2023
5 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add shared library support #338

add shared library support #338

dicej commented Jul 28, 2023

yamt commented Sep 29, 2023

dicej commented Sep 29, 2023

yamt commented Sep 30, 2023

yamt Sep 30, 2023 •

edited

Loading

dicej Oct 3, 2023 •

edited

Loading

dicej commented Oct 3, 2023

sbc100 commented Oct 3, 2023

dicej commented Oct 3, 2023

sbc100 commented Oct 3, 2023

yamt commented Oct 6, 2023

dicej commented Oct 6, 2023

tschneidereit commented Nov 9, 2023

sbc100 commented Nov 9, 2023

yamt commented Nov 10, 2023

sbc100 commented Nov 10, 2023

yamt commented Nov 10, 2023

dicej commented Nov 16, 2023

dicej commented Nov 26, 2023

dicej commented Nov 29, 2023

sbc100 commented Nov 29, 2023

sbc100 commented Nov 29, 2023

yamt commented Dec 7, 2023

dicej commented Dec 7, 2023

abrown commented Dec 8, 2023

dicej commented Dec 8, 2023

sbc100 left a comment

sbc100 Dec 11, 2023

dicej Dec 11, 2023

sbc100 Dec 11, 2023

add shared library support #338

add shared library support #338

Conversation

dicej commented Jul 28, 2023

yamt commented Sep 29, 2023

dicej commented Sep 29, 2023

yamt commented Sep 30, 2023

yamt Sep 30, 2023 • edited Loading

Choose a reason for hiding this comment

dicej Oct 3, 2023 • edited Loading

Choose a reason for hiding this comment

dicej commented Oct 3, 2023

sbc100 commented Oct 3, 2023

dicej commented Oct 3, 2023

sbc100 commented Oct 3, 2023

yamt commented Oct 6, 2023

dicej commented Oct 6, 2023

tschneidereit commented Nov 9, 2023

sbc100 commented Nov 9, 2023

yamt commented Nov 10, 2023

sbc100 commented Nov 10, 2023

yamt commented Nov 10, 2023

dicej commented Nov 16, 2023

dicej commented Nov 26, 2023

dicej commented Nov 29, 2023

sbc100 commented Nov 29, 2023

sbc100 commented Nov 29, 2023

yamt commented Dec 7, 2023

dicej commented Dec 7, 2023

abrown commented Dec 8, 2023

dicej commented Dec 8, 2023

sbc100 left a comment

Choose a reason for hiding this comment

sbc100 Dec 11, 2023

Choose a reason for hiding this comment

dicej Dec 11, 2023

Choose a reason for hiding this comment

sbc100 Dec 11, 2023

Choose a reason for hiding this comment

yamt Sep 30, 2023 •

edited

Loading

dicej Oct 3, 2023 •

edited

Loading