LibWeb: Make named character references more spec-compliant and effic…

…ient

There are two changes happening here: a correctness fix, and an
optimization. In theory they are unrelated, but the optimization
actually paves the way for the correctness fix.

Before this commit, the HTML tokenizer would attempt to look for named
character references by checking from after the `&` until the end of
m_decoded_input, which meant that it was unable to recognize things like
named character references that are inserted via `document.write` one
byte at a time. For example, if `∉` was written one-byte-at-a-time
with `document.write`, then the tokenizer would only check against `n`
since that's all that would exist at the time of the check and therefore
erroneously conclude that it was an invalid named character reference.

This commit modifies the approach taken for named character reference
matching by using a trie-like structure (specifically, a deterministic
acyclic finite state automaton or DAFSA), which allows for efficiently
matching one-character-at-a-time and therefore it is able to pick up
matching where it left off after each code point is consumed.

Note: Because it's possible for a partial match to not actually develop
into a full match (e.g. `&notindo` which could lead to `⋵̸`),
some backtracking is performed after-the-fact in order to only consume
the code points within the longest match found (e.g. `&notindo` would
backtrack back to `&not`).

With this new approach, `document.write` being called one-byte-at-a-time
is handled correctly, which allows for passing more WPT tests, with the
most directly relevant tests being
`/html/syntax/parsing/html5lib_entities01.html`
and
`/html/syntax/parsing/html5lib_entities02.html`
when run with `?run_type=write_single`. Additionally, the implementation
now better conforms to the language of the spec (and resolves a FIXME)
because exactly the matched characters are consumed and nothing more, so
SWITCH_TO is able to be used as the spec says instead of RECONSUME_IN.

The new approach is also an optimization:

- Instead of a linear search using `starts_with`, the usage of a DAFSA
  means that it is always aware of which characters can lead to a match
  at any given point, and will bail out whenever a match is no longer
  possible.
- The DAFSA is able to take advantage of the note in the section
  `13.5 Named character references` that says "This list is static and
  will not be expanded or changed in the future." and tailor its Node
  struct accordingly to tightly pack each node's data into 32-bits.
  Together with the inherent DAFSA property of redundant node
  deduplication, the amount of data stored for named character reference
  matching is minimized.

In my testing:

- A benchmark tokenizing an arbitrary set of HTML test files was about
  1.23x faster (2070ms to 1682ms).
- A benchmark tokenizing a file with tens of thousands of named
  character references mixed in with truncated named character
  references and arbitrary ASCII characters/ampersands runs about 8x
  faster (758ms to 93ms).
- The size of `liblagom-web.so` was reduced by 94.96KiB.

Some technical details:

A DAFSA (deterministic acyclic finite state automaton) is essentially a
trie flattened into an array, but it also uses techniques to minimize
redundant nodes. This provides fast lookups while minimizing the
required data size, but normally does not allow for associating data
related to each word. However, by adding a count of the number of
possible words from each node, it becomes possible to also use it to
achieve minimal perfect hashing for the set of words (which allows going
from word -> unique index as well as unique index -> word). This allows
us to store a second array of data so that the DAFSA can be used as a
lookup for e.g. the associated code points.

Libraries/LibWeb/CMakeLists.txt

@@ -866,6 +866,8 @@ invoke_generator(
 
 generate_css_implementation()
 
+generate_html_implementation()
+
 invoke_generator(
     "WebGLRenderingContextImpl.cpp"
     Lagom::GenerateWebGLRenderingContext
@@ -909,6 +911,7 @@ set(GENERATED_SOURCES
     WebGL/WebGLRenderingContextImpl.cpp
     Worker/WebWorkerClientEndpoint.h
     Worker/WebWorkerServerEndpoint.h
+    HTML/Parser/NamedCharacterReferences.cpp
 )
 
 serenity_lib(LibWeb web)