test large overflows and underflows for EnsureIntegerInRange

[Hecate2]

The correctness of OpCode.AND depends on the representation of BigInteger in bytes, and I am really suspicious whether AND is correct. Here are some examples of bytes representations.
https://learn.microsoft.com/en-us/dotnet/api/system.numerics.biginteger.tobytearray?view=net-8.0
MOD is safe, no matter how BigInteger is implemented in bytes.

[Jim8y]

Need unit test to demonstrate the problem.

[Hecate2]

Need unit test to demonstrate the problem.

I have not come up with any test case to demonstrate that AND is wrong, but I will try it in 24 hrs

[vncoelho]

For BigIntenger Negative values it may be the error if I understood correctly

[shargon]

We should ensure that is well covered with the current UT

[Hecate2]

In tests I found that AND works well, while MOD is wrong. In C#, (-1 mod 5) returns -1. In other languages the result is often 4 😰

[nan01ab]

Expression x AND mask is not equal to x MOD (mask + 1) when x is negative, so MOD cannot replace AND here?
For expample:

var x = new BigInteger(-3);
var y = new BigInteger(255);


Console.WriteLine($"mod {x % (y+1)}, and {x & y}");

x = new BigInteger(3);
y = new BigInteger(255);

Console.WriteLine($"mod {x % (y+1)}, and {x & y}");

Output:

mod -3, and 253
mod 3, and 3

[nan01ab]

In tests I found that AND works well, while MOD is wrong. In C#, (-1 mod 5) returns -1. In other languages the result is often 4 😰

Other languages are Python? C, Java, Rust etc. have the same semantics as C#.

[Hecate2]

My fault. OpCode.AND is always correct in tests. I will add tests for large overflows and underflows.