21408e49eb
added thai analyzer test
2014-08-14 16:39:37 -04:00
599ef6edce
added swedish analyzer test
2014-08-14 16:12:48 -04:00
64255e3eb9
added russian analyzer test
2014-08-14 16:11:23 -04:00
8896de2039
added romanian analyzer test
2014-08-14 16:06:17 -04:00
c2937b4b81
added portuguese analyzer test
...
discrepencies found, logged in #70
failing tests commented out for now
2014-08-14 16:04:29 -04:00
81a9d325a2
added norwegian analyzer test
2014-08-14 16:01:03 -04:00
a3a97a09d3
added dutch analyzer test
2014-08-14 15:59:39 -04:00
6714d5d765
added italian analyzer test
...
discrepencies found between us and lucene, documented in #69
failing tests commented out for now
2014-08-14 15:56:47 -04:00
b9c0477762
added hungarian analyzer test
2014-08-14 15:51:55 -04:00
6a9f8e85ae
added french analyzer test
...
many discrepencies noted, opened issue #68 to track this
failing tests commented out for now
2014-08-14 15:48:32 -04:00
f6f17c7a9e
added finish analyzer test
2014-08-14 15:27:45 -04:00
80d7c4f870
added persian analyzer test
2014-08-14 15:24:42 -04:00
2ef7c80c92
added spanish analyzer test
2014-08-14 14:44:46 -04:00
4398aab723
added sorani analyzer test
2014-08-14 14:42:36 -04:00
b22941ee37
added test for danish anlyzer
2014-08-14 14:36:24 -04:00
8c9997f1e2
added test for german analyzer
2014-08-14 14:33:30 -04:00
6a951b9372
added analyzer test for english
2014-08-14 14:28:24 -04:00
c526a38369
major refactor of analysis files, now wired up to registry
...
ultimately this is make it more convenient for us to wire up
different elements of the analysis pipeline, without having to
preload everything into memory before we need it
separately the index layer now has a mechanism for storing
internal key/value pairs. this is expected to be used to
store the mapping, and possibly other pieces of data by the
top layer, but not exposed to the user at the top.
2014-08-13 21:14:47 -04:00
3481ec9cef
added hindi stemmer
...
closes #40
2014-08-11 22:29:47 -04:00
c65f7415ff
added hindi normalizer
...
closes #64
2014-08-11 19:51:47 -04:00
cd0e3fd85b
added german normalizer
...
updated german analyzer to use this normalizer
closes #65
2014-08-11 19:25:37 -04:00
a4707ebb4e
configured zero width non joiner char filter, and persian analyzer
2014-08-11 18:57:04 -04:00
4ccd69ed45
added arabic normalizer
...
closes #63
2014-08-11 18:35:35 -04:00
73b252f6a6
added persian normalizer
...
closes #67
2014-08-11 18:15:41 -04:00
e5d4e6f1e4
refactored index layer to support batch operations
...
this change was then exposed at the higher levels
also the beer-sample app was upgraded to index in batches of 100
by default. this yieled an indexing speed up from 27s to 16s.
closes #57
2014-08-11 16:27:18 -04:00
cac707b5b7
upgraded beer-search to index in background
...
this allows the app to be usable while indexing takes place
also prints out indexing performace stats to console
2014-08-11 13:20:32 -04:00
42895649de
further streamlined the API
...
introduced concept of byte array converters
right now only wired up to top-level index mapping
allowing the removal of the JSON methods, now at the top level
we default to parsing []byte as JSON, override if thats not
the behavior you want.
future enhancements will allow use of these byte array converters
to control how byte arrays are handled elsewhere in documents
this would allow for handing binary attachments, etc in the future
closes #59
2014-08-11 12:47:29 -04:00
7bbaa8ecd5
added support for returning facet results with requests
...
supports terms, numeric ranges, and date ranges
closes #14
2014-08-11 11:03:29 -04:00
e21b7f4436
added sorani normalizer and stemmer, now have analyzer
...
closes #43
2014-08-08 09:38:28 -04:00
ef35ea1985
added czech stop word list
...
closes #36
2014-08-07 22:32:49 -04:00
964b87f76e
added rune tokenizer
...
not used directly right now, but basis for other simple tokenizers
2014-08-07 22:14:26 -04:00
a3ac85c0de
added prefix search to beer-search example app
2014-08-07 13:46:34 -04:00
292af78b9e
implemented prefix search
...
closes #4
2014-08-07 13:45:39 -04:00
b16c1d7f79
changed term row encoding
...
previously we used the format:
't' <utf-8 term> <byte separator> <16-bit field id> <utf-8 docID> <byte separator>
now we have moved the field before the term, resulting in:
't' <16-bit field id> <utf-8 term> <byte separator> <utf-8 docID> <byte separator>
this means now instead of all fields with the same term being grouped together
all terms within the same field are grouped together
this allows us to enumerate the terms used with a field
this allows us to implement prefix search, and possibly improve numeric range queries
2014-08-07 09:39:04 -04:00
0e54fbd8da
added keyword marker filter
...
updated stemmer filter to not stem tokens marked as keyword
closes #48
2014-08-07 08:13:00 -04:00
c19270108c
added ngram and edge ngram token filters
...
closes #46 and closes #47
2014-08-06 22:11:42 -04:00
9a777aaa80
added token truncate filter
...
closes #49
2014-08-06 20:39:42 -04:00
0441c6bef6
refactored names, removing Term from things that were more general
...
closes #60
2014-08-06 20:03:41 -04:00
f69838d670
added support for boost with ^boostval in syntax query
...
closes #51
also found/fixed major bug in scoring, closes #61
2014-08-06 19:36:23 -04:00
41d4f67ee2
fix storing/retrieving numeric and date fields
...
also includes new ability to request stored fields be returned with results
closes #55 and closes #56 and closes #58
2014-08-06 13:52:20 -04:00
d84187fd24
added apostrophe filter to improve turkish analyzer
...
closes #27
2014-08-06 08:50:00 -04:00
649a4999a1
fix broken test
2014-08-06 08:42:57 -04:00
da26a24031
removed optional element, and fixed took time formatting
2014-08-06 08:25:42 -04:00
893efa670e
added some more explicit mappings
2014-08-06 08:25:10 -04:00
78da6fd65d
added support for a default field
...
this works at the config and index mapping levels
2014-08-06 08:23:29 -04:00
79ab2b9b3d
added unicode normalization filter
2014-08-04 21:59:57 -04:00
2c0bf23fac
added elision filter
...
defined article word maps for french, italian, irish and catalan
defined elision filters for these same languages
updated analyers for french and italian to use this new filter
irish and catalan still depend on other missing pieces
closes #25
2014-08-03 19:17:35 -04:00
0960cab0ae
refactored StopWordsMap into WordMap so it can be reused
...
the ElisionFilter will need a word list of articles and plan to reuse this
2014-08-03 17:46:35 -04:00
1e5cc5c89f
fix issues identified by go vet
2014-08-03 17:21:41 -04:00
41ee1028c9
added date range query to beer-search sample app
2014-08-03 17:20:00 -04:00
Marty Schoch