Add a compiler intrinsic to back `bigint_helper_methods` #133663

scottmcm · 2024-11-30T07:00:22Z

This adds a new carrying_mul_add intrinsic, to implement wide_mul and carrying_mul.

It has fallback MIR for all types -- including u128, which isn't currently supported on nightly -- so that it'll continue to work on all backends, including CTFE.

Then it's overridden in cg_llvm to use wider intermediate types, including i256 for u128::carrying_mul.

rustbot · 2024-11-30T07:00:31Z

r? @ibraheemdev

rustbot has assigned @ibraheemdev.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

rustbot · 2024-11-30T07:00:33Z

Some changes occurred to the intrinsics. Make sure the CTFE / Miri interpreter
gets adapted for the changes, if necessary.

cc @rust-lang/miri, @rust-lang/wg-const-eval

library/core/src/intrinsics/fallback.rs

clarfonthey · 2024-11-30T15:32:11Z

Will take a look at this later, but I like that you managed to make fallback intrinsics work here when I couldn't quite get them to work for #132195.

compiler/rustc_codegen_llvm/src/intrinsic.rs

bors · 2024-12-02T15:53:04Z

☔ The latest upstream changes (presumably #133728) made this pull request unmergeable. Please resolve the merge conflicts.

clarfonthey · 2024-12-03T04:34:44Z

Also just to clarify my opinion on where this stands relative to #132195: I think that this PR just adding the intrinsic is okay, although I would like the intrinsic to also handle the signed case, even if it's only used at the moment for the unsigned case. That way, my PR can just update the library methods to use the intrinsic, without having to worry about the slow implementation.

Although the signed version of the library methods will likely return an unsigned word always for the least-significant / high-order part, it vastly simplifies the intrinsic to just have the same type for the arguments and return value, so, I would prefer that we do this specifically for the intrinsic. Since it doesn't actually affect the returned bits, I think that it's okay to just cast half of it unsigned on the library side. You can see how I did this here in a previous (closed) PR.

Otherwise, LGTM. Also, while the codegen test should suffice, if you want to copy over the library tests I added in my PR to verify that the intrinsic works correctly, feel free. I specifically added a bunch of edge cases with MAX and MIN as operands to ensure that things work correctly.

scottmcm · 2024-12-07T11:23:06Z

Success! No more "oh my this is hacky", just a couple of impl consts with fairly normal code.

it vastly simplifies the intrinsic to just have the same type for the arguments and return value

It didn't, actually. Once the fallback was working, the implementation in cg_llvm was super-easy -- after all, at that level the signed and unsigned types are the same anyway.

compiler/rustc_codegen_llvm/src/intrinsic.rs

clarfonthey · 2024-12-07T23:40:35Z

compiler/rustc_codegen_llvm/src/intrinsic.rs

+                let high = self.trunc(high, narrow_llty);
+
+                let pair_llty = self.type_struct(&[narrow_llty, narrow_llty], false);
+                let pair = self.const_poison(pair_llty);


Just comparing to my version, this doesn't really matter that much, but const_undef feels slightly more accurate here just semantically. Not actually sure if this affects codegen though.

poison is always preferred where possible, because it doesn't have the same complications as undef. (undef << 1) & 1 is guaranteed to be 0, but (poison << 1) & 1 is still poison. You'll also see that both what we emit for tuples and what the optimizer does for tuples both use poison for this: https://rust.godbolt.org/z/WTnW4GPE6

scottmcm · 2024-12-08T05:20:39Z

library/core/src/intrinsics/mod.rs

+/// => (2²ⁿ - 2ⁿ⁺¹ + 1) + (2ⁿ⁺¹ - 2)
+/// => 2²ⁿ - 1
+///
+/// For `iN`, the upper bound is MIN * MIN + MAX + MAX => 2²ⁿ⁻² + 2ⁿ - 2,


Pop Quiz: after a double-width isize::MIN * isize::MIN (which is the largest possible positive result from a multiplication), how many more isize::MAXs can you add to the result before it would need to carry to triple-width?

Answer: isize::MAX + 2 of them!

(The carrying_mul_add really feels so much nicer for unsigned types, since uN::MAX * uN::MAX + uN::MAX + uN::MAX == u2N::MAX -- no more space available after that.)

ibraheemdev · 2024-12-13T21:25:02Z

This is outside my territory so I'm going to pass this along. r? libs

bors · 2024-12-20T01:50:04Z

☔ The latest upstream changes (presumably #134516) made this pull request unmergeable. Please resolve the merge conflicts.

scottmcm · 2024-12-20T08:29:28Z

Since I'm not sure if the random-reroll notifies, friendly ping here @Amanieu

Amanieu · 2024-12-26T18:22:25Z

Is it intentional that this intrinsic is only ever called with addend == 0 ? Are there plans to use the addend in other places in the future? If not then the intrinsic could probably be simplified.

Otherwise LGTM

clarfonthey · 2024-12-26T18:33:30Z

Yes; once this is merged I'll be rebasing #132195 on top of it which does add methods that use this to its full extent. I think that scottmcm just wanted to lower the scope of this PR to just the intrinsic, and not any library changes.

scottmcm · 2024-12-26T23:33:57Z

Yup, what @clarfonthey said -- this takes 4 arguments because MAX * MAX + MAX + MAX is the most that fits in unsigned without overflowing further, and while I originally didn't think it was needed, since a = x * y only needs the 3-argument version, the version that's a += x * y needs both "carries" -- one for the internal carry in the multiplication and one to add in the original a part.

So I'm happy to let them take over for how to actually expose it to stable :)

@bors r=Amanieu

bors · 2024-12-26T23:34:00Z

📌 Commit 0c1cc9e has been approved by Amanieu

It is now in the queue for this repository.

bors · 2024-12-27T06:45:44Z

⌛ Testing commit 0c1cc9e with merge 2822f19...

Add a compiler intrinsic to back `bigint_helper_methods` cc rust-lang#85532 This adds a new `carrying_mul_add` intrinsic, to implement `wide_mul` and `carrying_mul`. It has fallback MIR for all types -- including `u128`, which isn't currently supported on nightly -- so that it'll continue to work on all backends, including CTFE. Then it's overridden in `cg_llvm` to use wider intermediate types, including `i256` for `u128::carrying_mul`.

bors · 2024-12-27T07:39:57Z

💔 Test failed - checks-actions

Including implementing it for `u128`, so it can be defined in `uint_impl!`. This way it works for all backends, including CTFE.

scottmcm · 2024-12-27T18:41:29Z

Fix was just changing some i64s to {{i32|i64}}s in GEPs.

@bors r=Amanieu

bors · 2024-12-27T18:41:31Z

📌 Commit 4669c0d has been approved by Amanieu

It is now in the queue for this repository.

…iaskrgr Rollup of 6 pull requests Successful merges: - rust-lang#133663 (Add a compiler intrinsic to back `bigint_helper_methods`) - rust-lang#134798 (Make `ty::Error` implement all auto traits) - rust-lang#134808 (compiletest: Remove empty 'expected' files when blessing) - rust-lang#134809 (Add `--no-capture`/`--nocapture` as bootstrap arguments) - rust-lang#134826 (Add spastorino to users_on_vacation) - rust-lang#134828 (Add clubby789 back to bootstrap review rotation) r? `@ghost` `@rustbot` modify labels: rollup

Rollup merge of rust-lang#133663 - scottmcm:carrying_mul_add, r=Amanieu Add a compiler intrinsic to back `bigint_helper_methods` cc rust-lang#85532 This adds a new `carrying_mul_add` intrinsic, to implement `wide_mul` and `carrying_mul`. It has fallback MIR for all types -- including `u128`, which isn't currently supported on nightly -- so that it'll continue to work on all backends, including CTFE. Then it's overridden in `cg_llvm` to use wider intermediate types, including `i256` for `u128::carrying_mul`.

rustbot assigned ibraheemdev Nov 30, 2024

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Nov 30, 2024

scottmcm force-pushed the carrying_mul_add branch from f52a909 to 09a8f68 Compare November 30, 2024 07:14

scottmcm mentioned this pull request Nov 30, 2024

Tracking Issue for bigint helper methods #85532

Open

9 tasks

RalfJung mentioned this pull request Nov 30, 2024

Add intrinsics for bigint helper methods #131566

Closed

scottmcm commented Nov 30, 2024

View reviewed changes

library/core/src/intrinsics/fallback.rs Outdated Show resolved Hide resolved

scottmcm force-pushed the carrying_mul_add branch from 09a8f68 to acbeaa2 Compare December 2, 2024 00:17

clarfonthey reviewed Dec 2, 2024

View reviewed changes

compiler/rustc_codegen_llvm/src/intrinsic.rs Outdated Show resolved Hide resolved

scottmcm force-pushed the carrying_mul_add branch from acbeaa2 to c67e5f4 Compare December 7, 2024 11:18

This comment has been minimized.

Sign in to view

scottmcm force-pushed the carrying_mul_add branch from c67e5f4 to 5d91169 Compare December 7, 2024 11:57

This comment has been minimized.

Sign in to view

clarfonthey reviewed Dec 7, 2024

View reviewed changes

compiler/rustc_codegen_llvm/src/intrinsic.rs Outdated Show resolved Hide resolved

clarfonthey reviewed Dec 7, 2024

View reviewed changes

scottmcm force-pushed the carrying_mul_add branch 2 times, most recently from ee18903 to f636b64 Compare December 8, 2024 02:23

scottmcm commented Dec 8, 2024

View reviewed changes

rustbot assigned Amanieu and unassigned ibraheemdev Dec 13, 2024

scottmcm force-pushed the carrying_mul_add branch from f636b64 to 0c1cc9e Compare December 20, 2024 05:37

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 26, 2024

This comment has been minimized.

Sign in to view

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Dec 27, 2024

scottmcm added 2 commits December 27, 2024 08:17

Move {widening, carrying}_mul to an intrinsic with fallback MIR

2c0c912

Including implementing it for `u128`, so it can be defined in `uint_impl!`. This way it works for all backends, including CTFE.

Override carrying_mul_add in cg_llvm

4669c0d

scottmcm force-pushed the carrying_mul_add branch from 0c1cc9e to 4669c0d Compare December 27, 2024 16:18

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 27, 2024

matthiaskrgr mentioned this pull request Dec 27, 2024

Rollup of 6 pull requests #134830

Merged

bors merged commit 95e66ff into rust-lang:master Dec 27, 2024
6 checks passed

rustbot added this to the 1.85.0 milestone Dec 27, 2024

clarfonthey mentioned this pull request Dec 28, 2024

Tidy up bigint multiplication methods #132195

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a compiler intrinsic to back `bigint_helper_methods` #133663

Add a compiler intrinsic to back `bigint_helper_methods` #133663

scottmcm commented Nov 30, 2024

rustbot commented Nov 30, 2024

rustbot commented Nov 30, 2024

clarfonthey commented Nov 30, 2024

bors commented Dec 2, 2024

clarfonthey commented Dec 3, 2024

scottmcm commented Dec 7, 2024

This comment has been minimized.

This comment has been minimized.

clarfonthey Dec 7, 2024

scottmcm Dec 8, 2024

scottmcm Dec 8, 2024 •

edited

Loading

ibraheemdev commented Dec 13, 2024

bors commented Dec 20, 2024

scottmcm commented Dec 20, 2024

Amanieu commented Dec 26, 2024

clarfonthey commented Dec 26, 2024

scottmcm commented Dec 26, 2024

bors commented Dec 26, 2024

bors commented Dec 27, 2024

This comment has been minimized.

bors commented Dec 27, 2024

scottmcm commented Dec 27, 2024

bors commented Dec 27, 2024

Add a compiler intrinsic to back bigint_helper_methods #133663

Add a compiler intrinsic to back bigint_helper_methods #133663

Conversation

scottmcm commented Nov 30, 2024

rustbot commented Nov 30, 2024

rustbot commented Nov 30, 2024

clarfonthey commented Nov 30, 2024

bors commented Dec 2, 2024

clarfonthey commented Dec 3, 2024

scottmcm commented Dec 7, 2024

This comment has been minimized.

This comment has been minimized.

clarfonthey Dec 7, 2024

Choose a reason for hiding this comment

scottmcm Dec 8, 2024

Choose a reason for hiding this comment

scottmcm Dec 8, 2024 • edited Loading

Choose a reason for hiding this comment

ibraheemdev commented Dec 13, 2024

bors commented Dec 20, 2024

scottmcm commented Dec 20, 2024

Amanieu commented Dec 26, 2024

clarfonthey commented Dec 26, 2024

scottmcm commented Dec 26, 2024

bors commented Dec 26, 2024

bors commented Dec 27, 2024

This comment has been minimized.

bors commented Dec 27, 2024

scottmcm commented Dec 27, 2024

bors commented Dec 27, 2024

Add a compiler intrinsic to back `bigint_helper_methods` #133663

Add a compiler intrinsic to back `bigint_helper_methods` #133663

scottmcm Dec 8, 2024 •

edited

Loading