2 Commits

Author	SHA1	Message	Date
Marty Schoch	1dc466a800	modified token filters to avoid creating new token stream ... often the result stream was the same length, so can reuse the existing token stream also, in cases where a new stream was required, set capacity to the length of the input stream. most output stream are at least as long as the input, so this may avoid some subsequent resizing	2014-09-23 18:41:32 -04:00
Marty Schoch	1a1cf32a86	introducing cjk_bigram filter and cjk analyzer ... closes #34	2014-09-11 10:39:05 -04:00