Add weejson-jsoniter-scala

[htmldoug]

Summary

Adds basic integration for the very fast jsoniter-scala parser.

Benchmarks

• My benchmarks on doppelganger JSON files with the parsers integrated into weepickle (possibly suboptimally)
• Official jsoniter-scala benchmarks using the JSON specialized macros instead of the weepickle macros

Optional Jar

Based on performance alone, I'd love to have this parser as weepickle's default UTF-8 implementation, but:

1. I'm not giving up jackson-core integration (particularly for dataformats-binary and dataformats-text).
2. Having two default parser dependencies is slightly "wat"
3. I'm not sure if jsoniter-scala v3 will use the same package names and cause us dependency hell. Last major version release was in Nov 2019 and did not change package names.)

The README suggests as a feature:

Support of shading to another package for locking on a particular released version

We may want to go this route in the future, but for now, I'm adding the integration as an optional jar primarily for use by non-libraries at the absolute root of the dependency tree.

TODO

• parse floats less horribly
• update readme

[htmldoug]

@plokhotnyuk, I seem to be stuck here. Do you have any suggestions to achieve better performance than my hacky regex given the failed attempts in the comment above? Specifically, is there anything in JsonReader that can help me that I might have overlooked?

[plokhotnyuk]

@htmldoug You can use some counters to calculate number of valid chars like in the following code snippet (BEWARE: it is not tested)

    private def parseNumber[J](
      in: JsonReader,
      v: Visitor[_, J]
    ): J = {
      in.setMark()
      var b = in.nextByte()
      var digits, index = 0
      var decIndex, expIndex = -1
      if (b == '-') {
        b = in.nextByte()
        index += 1
      }
      try {
        digits -= index
        while (b >= '0' && b <= '9') {
          b = in.nextByte()
          index += 1
        }
        digits += index
        if (b == '.') {
          decIndex = index
          b = in.nextByte()
          index += 1
        }
        digits -= index
        while (b >= '0' && b <= '9') {
          b = in.nextByte()
          index += 1
        }
        digits += index
        if ((b | 0x20) == 'e') {
          expIndex = index
          b = in.nextByte()
          index += 1
          if (b == '-' || b == '+') {
            b = in.nextByte()
            index += 1
          }
          while (b >= '0' && b <= '9') {
            b = in.nextByte()
            index += 1
          }
        }
      } catch {
        case _: JsonReaderException => // ignore the end of input error for now
      } finally in.rollbackToMark()
      if ((decIndex & expIndex) == -1) {
        if (digits < 19) v.visitInt64(in.readLong())
        else {
          val x = in.readBigInt(null)
          if (x.bitLength < 64) v.visitInt64(x.longValue)
          else v.visitFloat64StringParts(x.toString, -1, -1)
        }
      } else {
        val cs = new String(in.readRawValAsBytes(), StandardCharsets.US_ASCII)
        require(cs.length - 1 == index, "invalid number")
        v.visitFloat64String(cs)
      }
    }

[htmldoug]

Ooh, comparing the walked length vs the string.length is smart. I'll run with this implementation tomorrow. Thank you, @plokhotnyuk!

I tried something similar previously, but it wasn't giving me the throughput I expected. I just realized that most of the time is spent in endOfInputError > appendHexDump. "3.14 " parses 3x faster than "3.14"! I wonder if something similar could also be happening in BigDecimalReading, BigIntReading, etc.

[plokhotnyuk]

@htmldoug You can parse an array of numbers to make your benchmark more realistic. I bet nobody use JSON parsers to read a single number.

[plokhotnyuk]

@htmldoug I've added a missing method for checking of remaining bytes in the reader, you can now use it to parse numbers without need to catch end of input errors, like here.