Tokenizer: moved test-only tokenize()
to SimpleTokenizer
/ several cleanups
#13089
Job | Run time |
---|---|
7m 26s | |
3m 14s | |
10m 40s |
tokenize()
to SimpleTokenizer
/ several cleanups
#13089
Job | Run time |
---|---|
7m 26s | |
3m 14s | |
10m 40s |