Tokenizer: moved test-only tokenize()
to SimpleTokenizer
/ several cleanups
#3973
Job | Run time |
---|---|
1m 37s | |
1m 37s |
tokenize()
to SimpleTokenizer
/ several cleanups
#3973
Job | Run time |
---|---|
1m 37s | |
1m 37s |