6b4c86b35a
now it will return each cjk character as a separate token this will pair well with a cjk bigram filter for indexing |
||
---|---|---|
.. | ||
regexp_tokenizer_test.go | ||
regexp_tokenizer.go |
6b4c86b35a
now it will return each cjk character as a separate token this will pair well with a cjk bigram filter for indexing |
||
---|---|---|
.. | ||
regexp_tokenizer_test.go | ||
regexp_tokenizer.go |