![]() now it will return each cjk character as a separate token this will pair well with a cjk bigram filter for indexing |
||
---|---|---|
.. | ||
regexp_tokenizer_test.go | ||
regexp_tokenizer.go |
![]() now it will return each cjk character as a separate token this will pair well with a cjk bigram filter for indexing |
||
---|---|---|
.. | ||
regexp_tokenizer_test.go | ||
regexp_tokenizer.go |