integer parsing updates

[rmanaloto-tastytrade]

Out of curiosity, what's the reasoning for building your own implementation of number parsing logic vs using other 3rd party libs like fast_loat, ryu or dragonbox?

I'm just curious if you see inefficiencies in those libraries that you are improving on?

[stephenberry]

We were using a slightly optimized version as is found in yyjson. However, that doesn’t have optimizations for integer types like uint8_t, int16_t, etc. We can get better performance and less binary by making versions for these. Also, in some cases performance has been doubled versus the yyjson approach. And, with less binary. Performance is particularly better for short integers.

Also, by default we want to reject decimal numbers and have users opt in to decimal conversions, so that users make a conscious choice about data loss.

We found the yyjson approach faster than fast_float. And, this new approach is even faster.

I’m planning to have a decimal parsing version that is also much faster.

We use dragon box for floating point serialization. It doesn’t do reading.

The ryu algorithm was not designed to handle positive exponents, which are very useful for scientific integer inputs. And, we found this approach faster. It also isn't JSON conformant.

I remember testing a very fast implementation that used an enormous table for character pairs. My preference is to keep the binary smaller.

[rmanaloto-tastytrade]

thanks for the quick reply
I'm always impressed with the optimizaitions you seem to find with this library. So just wanted to get more insights on your thinking regarding integer/floating point parsing/serialization/deserialization

Regarding, precision loss, we've been using this decimal library to guarantee no data loss up to a certain decimal point