Tokenizer: moved test-only tokenize()
to SimpleTokenizer
/ severa…
#14793
Job | Run time |
---|---|
6m 1s | |
4m 34s | |
2m 21s | |
5m 16s | |
2m 8s | |
5m 15s | |
2m 1s | |
1m 51s | |
29m 27s |
tokenize()
to SimpleTokenizer
/ severa…
#14793
Job | Run time |
---|---|
6m 1s | |
4m 34s | |
2m 21s | |
5m 16s | |
2m 8s | |
5m 15s | |
2m 1s | |
1m 51s | |
29m 27s |