Tokenizer: moved test-only tokenize()
to SimpleTokenizer
/ several cleanups
#13090
Job | Run time |
---|---|
5s | |
2s | |
7s |
tokenize()
to SimpleTokenizer
/ several cleanups
#13090
Job | Run time |
---|---|
5s | |
2s | |
7s |