Tokenizer: moved test-only tokenize()
to SimpleTokenizer
/ several cleanups
#8738
Job | Run time |
---|---|
2m 45s | |
2m 45s |
tokenize()
to SimpleTokenizer
/ several cleanups
#8738
Job | Run time |
---|---|
2m 45s | |
2m 45s |