Marty Schoch
aaa686869c
Merge pull request #484 from mschoch/blevecheck
...
add the bleve check tool
2016-10-22 19:41:25 -04:00
Marty Schoch
9174872ba2
add the bleve check tool
...
bleve check was a consistency checking tool originally developed
as a part of cbft. currently it checks that the term dictionary
counts match the number of postings for the term. in the future
additional checks could be added. this tool has been back
ported to bleve as we've now adopted a single common tool for
both cbft and bleve.
2016-10-22 06:11:50 -04:00
Marty Schoch
09fc57c5db
Merge pull request #482 from steveyen/MB-21377
...
revert optimization that trims search-disjunction child searchers
2016-10-22 05:50:55 -04:00
Rob McColl
12c404aec0
Update kvstore.go
2016-10-19 10:31:00 -04:00
Rob McColl
87c7e9915e
KVStoreConstructur doc
2016-10-19 10:28:44 -04:00
Rob McColl
414d21a541
Add comment about JSON serialization of kvconfig
2016-10-19 10:19:14 -04:00
Steve Yen
58c3b5c9b8
revert optimization that trims search-disjunction child searchers
...
This commit reverts a previous optimization attempt 3f588cd4a
that
tried to trim or shrink the array of child searchers in a
search-disjunction.
Although I am not sure why at the moment, that optimization
incorrectly broke higher level boolean queries, but reverting so that
functionality is restored.
2016-10-18 14:38:34 -07:00
Marty Schoch
bfec4f23c0
Merge pull request #481 from robmccoll/patch-1
...
Fix NumDeletes doc copy/paste err s/Merge/Delete/g
2016-10-17 18:26:11 +01:00
Rob McColl
2b26218591
Fix NumDeletes doc copy/paste err s/Merge/Delete/g
2016-10-17 12:42:21 -04:00
Marty Schoch
77b79a2684
Merge pull request #466 from steveyen/optimize-fieldDict-reader-with-prealloc
...
Optimize upside-down's field dict reader with preallocated objects
2016-10-13 14:09:54 +02:00
Marty Schoch
5c7a2264a2
Merge pull request #473 from steveyen/reuse-incrementBytes-in-moss-kv-integration
...
reuse incrementBytes() in moss KV store integration
2016-10-13 14:03:46 +02:00
Marty Schoch
cee18d302e
Merge pull request #475 from steveyen/phrase-searcher-simplifications-dry
...
some simplification / DRY for phrase searcher
2016-10-12 23:07:35 +02:00
Marty Schoch
ac02b206e5
Merge pull request #476 from steveyen/optimize-fuzzy-searcher-prefixTerm-loop
...
end fuzzy searcher prefixTerm construction loop early
2016-10-12 23:04:50 +02:00
Marty Schoch
8b409d7ab9
Merge pull request #477 from steveyen/fix-BleveQueryType-json-marshaling
...
fix BleveQueryTime json marshaling with double-quoting
2016-10-12 23:04:08 +02:00
Steve Yen
32e459f6b6
fix BleveQueryTime json marshaling with double-quoting
...
See also MB-21322 found by Mihir Kamdar.
2016-10-12 11:39:08 -07:00
Steve Yen
1a994ce2a7
end fuzzy searcher prefixTerm construction loop early
2016-10-12 09:51:36 -07:00
Steve Yen
6a38fa3719
go fmt
2016-10-12 09:39:43 -07:00
Steve Yen
62e6f1f648
reuse incrementBytes() in moss KV store integration
...
In this commit, I saw that there was a simple incrementBytes()
implementation elsewhere in bleve that seemed simpler than using the
big int package.
Edge case note: if the input bytes would overflow in incrementBytes(),
such as with an input of [0xff 0xff 0xff], it returns nil. moss then
treats a nil endKeyExclusive iterator param as a logical
"higher-than-topmost" key, which produces the prefix iteration
behavior that we want for this edge situation.
2016-10-12 09:34:36 -07:00
Steve Yen
8230a7195f
some simplification / DRY for phrase searcher
2016-10-12 09:26:31 -07:00
Steve Yen
01fb59d293
optimize upside-down DictionaryRow for fewer parsing alloc's
2016-10-12 09:22:50 -07:00
Steve Yen
2d72b542c0
optimize upside-down FieldDict reader with prealloc'ed objects
...
As part of this commit, there's also a newly added
Dictionaryrow.parseDictionaryK() helper method.
2016-10-12 09:18:58 -07:00
Marty Schoch
d026a44230
Merge pull request #474 from steveyen/simplify-AddLocation
...
simplify TermLocationMap.AddLocation()
2016-10-12 14:20:50 +02:00
Marty Schoch
bddc064069
Merge pull request #471 from steveyen/remove-extra-indirection-LevenshteinDistance
...
removed extra level of pointer indirection from LevenshteinDistance()'s params
2016-10-12 14:05:34 +02:00
Marty Schoch
4160fb296f
Merge pull request #470 from daschl/sigma
...
Address special unicode sigma at end of term when lowercasing.
2016-10-12 14:03:17 +02:00
Marty Schoch
483f06ef5b
Merge pull request #467 from steveyen/optimize-disjunction-searcher-shrink-children
...
optimize disjunction searcher to trim child searchers array earlier
2016-10-12 14:00:19 +02:00
Marty Schoch
b76cbc805e
Merge pull request #465 from steveyen/cleanup-when-PrefixSearcher-error
...
close resources when we encounter an error on PrefixSearcher initialization
2016-10-12 13:39:28 +02:00
Marty Schoch
4e16818656
Merge pull request #464 from steveyen/check-FieldDictPrefix-err
...
check for error when prefix searcher starts a FieldDictPrefix reader
2016-10-12 13:36:04 +02:00
Marty Schoch
155827aeef
Merge pull request #462 from steveyen/master
...
log slow queries only when Config.SlowSearchLogThreshold > 0
2016-10-12 13:34:15 +02:00
Steve Yen
e72c8be353
simplify TermLocationMap.AddLocation()
2016-10-11 12:15:28 -07:00
Steve Yen
b6c97ddbfe
removed extra ptr indirection from LevenshteinDistance
2016-10-11 08:49:10 -07:00
Michael Nitschinger
7e656dad32
Address special unicode sigma at end of term when lowercasing.
...
Σ maps to σ, except at the end of a word where it maps to ς.
This is the only conditional (contextual) but language-independent
mapping in unicode.
2016-10-11 12:37:08 +02:00
Marty Schoch
586c6ee1a3
Merge pull request #469 from daschl/optim-lowercase
...
Skip already lowercased runes on transformation.
2016-10-11 12:12:21 +02:00
Michael Nitschinger
ff35d75aa4
Skip already lowercased runes on transformation.
...
The LowerCaseFilter works on the original slice to avoid allocations,
so skipping already lowercased runes avoids unnecessary work.
benchmark old ns/op new ns/op delta
BenchmarkLowerCaseFilter-8 1302 815 -37.40%
2016-10-11 12:03:26 +02:00
Steve Yen
3f588cd4ae
optimize disjunction searcher to trim child searchers array earlier
...
Disjunction searchers are used heavily by higher-level searchers, like
prefix searchers. In that case, a disjunction searcher might have
many thousands of child searchers.
This commit adds an optimization to close each child term searcher as
soon as a child searcher is finished and remove it from the
disjunction searcher's children.
2016-10-10 22:47:11 -07:00
Steve Yen
535b746b41
close resources when error on PrefixSearcher initialization
2016-10-10 17:29:59 -07:00
Steve Yen
2a022830f0
check FieldDictPrefix err result in prefix searcher
2016-10-10 15:35:54 -07:00
Steve Yen
21b3d592b8
log slow queries only when Config.SlowSearchLogThreshold > 0
2016-10-10 11:34:32 -07:00
Marty Schoch
de0c26718d
Merge pull request #461 from bcampbell/master
...
Settle on default fuzziness of 1 (for now)
2016-10-04 08:36:02 -04:00
Ben Campbell
11f18333fb
Settle on default fuzziness of 1 (for now)
...
see https://groups.google.com/d/msg/bleve/vkVxnLMlXow/5qM1jL0ZEgAJ
2016-10-04 15:00:50 +13:00
Marty Schoch
dfc78ca725
simplify, per gofmt -s recommendation
2016-10-02 12:14:53 -04:00
Marty Schoch
2f48d7fb02
fix misspellings
2016-10-02 12:11:15 -04:00
Marty Schoch
2dc2130633
additional golint cleanups
2016-10-02 12:00:01 -04:00
Marty Schoch
efb1ea7e64
fix golint comment
2016-10-02 11:56:37 -04:00
Marty Schoch
8e784c362b
another golint suggestions
2016-10-02 11:54:04 -04:00
Marty Schoch
abeca559cd
don't export unnecessary method
2016-10-02 11:50:58 -04:00
Marty Schoch
1b4ee737e0
more golint fixes
2016-10-02 11:46:27 -04:00
Marty Schoch
ce572091eb
additional golint cleanups
2016-10-02 11:44:34 -04:00
Marty Schoch
ee6b698edb
rename variable with _
2016-10-02 11:32:46 -04:00
Marty Schoch
667371dbec
more golint simplifications
2016-10-02 11:30:58 -04:00
Marty Schoch
c36eb74ead
address some golint suggestions
2016-10-02 11:14:09 -04:00