0
0
Commit Graph

861 Commits

Author SHA1 Message Date
Marty Schoch
d7292ed891 add support for gathering stats via map for easier consumption 2016-03-07 18:37:46 -05:00
Marty Schoch
e00577f265 change registry cache implementation to allow concurrent use
previously we just used a Go builtin map
this was not safe for concurrent read/write and upon upgrading
to Go 1.6 we were notified of the problem

fixes #349
2016-03-07 16:05:34 -05:00
Marty Schoch
e51f4d5450 changing async test strategy, was failing in go 1.6 2016-03-07 09:39:20 -05:00
Marty Schoch
5badbfdb0e allow running integration tests on alternate kvstore 2016-03-07 08:40:15 -05:00
Marty Schoch
23a323bc9d add support for numPlainTextBytesIndexed metric 2016-03-05 14:05:08 -05:00
Marty Schoch
81780f97d0 add term search stats 2016-03-05 07:50:25 -05:00
Marty Schoch
147debaa12 expose metrics and moss stats wrapping underlying stats as well 2016-03-04 13:43:39 -05:00
Marty Schoch
9d2f424a88 Merge pull request #348 from steveyen/moss-MaxPreMergerBatches-renamed
moss option MaxPreMergerBatches renamed
2016-03-03 15:03:49 -05:00
Steve Yen
f6d1bd2c87 moss option MaxPreMergerBatches renamed 2016-03-03 11:18:30 -08:00
Marty Schoch
b057dacfe4 Merge pull request #347 from mschoch/context
introduce ability for searches to timeout or be cancelled
2016-03-02 18:04:54 -05:00
Marty Schoch
0b2380d9bf introduce ability for searches to timeout or be cancelled
our implementation uses: golang.org/x/net/context

New method SearchInContext() allows the user to run a search
in the provided context.  If that context is cancelled or
exceeds its deadline Bleve will attempt to stop and return
as soon as possible.  This is a *best effort* attempt at this
time and may *not* be in a timely manner.  If the caller must
return very near the timeout, the call should also be wrapped
in a goroutine.

The IndexAlias implementation is affected in a slightly more
complex way.  In order to return partial results when a timeout
occurs on some indexes, the timeout is strictly enforced, and
at the moment this does introduce an additional goroutine.

The Bleve implementation honoring the context is currently
very course-grained.  Specifically we check the Done() channel
between each DocumentMatch produced during the search.  In the
future we will propogate the context deeper into the internals
of Bleve, and this will allow finer-grained timeout behavior.
2016-03-02 17:30:21 -05:00
Marty Schoch
f1a3a7a221 Merge pull request #346 from steveyen/MB-18441
MB-18441 - moss lower-level iterator starts  positioned on current
2016-03-02 08:40:07 -05:00
Steve Yen
7d67d89a9c MB-18441 - moss lower-level iterator starts positioned on current
The iterator starts off positioned so that Current() is correct, so
invoking Next() right off the bat was incorrect.
2016-03-01 21:45:48 -08:00
Marty Schoch
758cb54952 Merge pull request #345 from korzonek/master
codebeat badge
2016-03-01 08:13:26 -05:00
Tomasz Korzeniowski
a866c0cbf9 codebeat badge
Is it fine to add codebeat badge to README? 

codebeat is automated code review tool for Swift, Ruby & Go that helps get instant feedback on code quality. 

"Quick wins" suggested by codebeat could be a nice candidate for a pull request and help other developers become contributors.

FYI. To be fully open and honest. I'm co-founder of codebeat.
2016-03-01 13:51:12 +01:00
Marty Schoch
2527651c6d fix broken identification of analyzer for field name 2016-02-26 17:33:39 -05:00
Marty Schoch
265ab24b6e Merge pull request #339 from steveyen/WIP-moss
integrate index/store/moss KV store
2016-02-26 09:36:49 -05:00
Marty Schoch
a5bb81e19b fix highlighting bug with array values 2016-02-24 14:06:47 -05:00
Marty Schoch
6c988de5b5 fix date facet merging for searches on index aliases
previously we incorrectly identified matching buckets by
comparing string pointers.  this worked in the unit test
but not in real applications since the strings result from
date parsing inside the facet collector, and are therefore
different pointers
2016-02-23 15:33:07 -05:00
Marty Schoch
496fd365fd fix broken test expectations 2016-02-23 13:05:16 -05:00
Marty Schoch
55c43ae6c2 make DocCount() on IndexAlias consistent with partial results
it will now return the partial count of nodes that did
respond without error
2016-02-23 12:54:55 -05:00
Marty Schoch
5408083ab5 from JSON parsing regexp/wildcard queries defaulted to boost of 0
having boost of 0 led to invalid scores of NaN
added integration test for wildcard query
added ability to run single integration test at a time
added assertion that scoare is not NaN/+Inf/-Inf
2016-02-23 09:22:39 -05:00
Marty Schoch
214b67ad66 SearchResult now includes a Status section
the Status section can report on the number of total/fail/success
indexes when querying across multiple indexes through IndexAlias

Further, searching an IndexAlias will now return partial results,
the burden is on the caller to check the number of failed
indexes and decide how to handle this situation.
2016-02-22 16:50:40 -05:00
Steve Yen
a29dd25a48 upside_down dict row value size accounts for large uvarint's
This is somewhat unlikely, but if a term is (incredibly) popular, its
uvarint count value representation might go beyond 8 bytes.

Some KVStore implementations (like forestdb) provide a BatchEx cgo
optimization that depends on proper preallocated counting, so this
change provides a proper worst-case estimate based on the max-unvarint
of 10 bytes instead of the previously incorrect 8 bytes.
2016-02-22 11:52:51 -08:00
Steve Yen
dd1718fa78 index/store/moss uses AllocMerge() instead of Merge()
Performance optimization.  Before this change, by using Merge()
instead of AllocMerge(), moss's internal batch buf's would be
wastefully, dramatically grown during append()'s to a mis-sized buf.
2016-02-22 11:48:02 -08:00
Steve Yen
ea1a52464d more index/store/moss err handling 2016-02-20 14:25:42 -08:00
Steve Yen
eb315fa500 integrate index/store/moss KV store 2016-02-20 14:25:42 -08:00
Marty Schoch
74a52f94bb prefix,regexp, and fuzzy searchers failed to close fieldDict 2016-02-20 15:41:12 -05:00
Marty Schoch
1af0568036 Merge pull request #340 from MachineShop-IOT/master
Fix masking of error when creating index (minor error reporting fix)
2016-02-18 12:09:23 -05:00
Mark Mindenhall
bae3db1c55 Fix masking of error when creating index 2016-02-17 10:31:27 -07:00
Marty Schoch
1006bf54b4 try to close indexes at end of http handler test 2016-02-09 16:26:03 -05:00
Marty Schoch
208b700e17 add missing build tag guarding cznicb benchmark 2016-02-09 15:57:35 -05:00
Marty Schoch
e523bf757e test slow timer with different way to avoid windows 15ms timer 2016-02-09 15:48:08 -05:00
Marty Schoch
9a1e6e1905 fix some test failures on windows 2016-02-09 13:33:11 -05:00
Marty Schoch
ebb7d2d076 added ability to limit the max number of disjunction clauses
set DisjunctionMaxClauseCount to a non-zero value to enforce
the limit
2016-02-08 17:21:03 -05:00
Marty Schoch
71ba2ea40c Merge pull request #148 from StreamBoat/handle_numerics
Handle numeric fields other than float64 in processProperty
2016-02-05 13:44:27 -05:00
Marty Schoch
40c1a8e835 Merge branch 'slavikm-master2' 2016-02-05 13:10:28 -05:00
Marty Schoch
c07fa47551 added test case to verify boost is working 2016-02-05 13:10:01 -05:00
Marty Schoch
c5fe9da05c Merge branch 'master' of https://github.com/slavikm/bleve into slavikm-master2 2016-02-05 12:47:40 -05:00
Marty Schoch
40c95513b7 add support for including kvstore stats 2016-02-05 12:26:19 -05:00
slavikm
c1ce8910d7 pass the boost value into the term searcher 2016-02-03 14:49:11 -08:00
Marty Schoch
f38e3e1b24 remove temporary error and replace with permanent check 2016-02-03 10:23:49 -05:00
Marty Schoch
c5dea9e882 fix accessing store via Advanced() method which was broken 2016-02-02 11:54:18 -05:00
Marty Schoch
a236737a68 temporary workaround to avoid crashing when an index is not
behaving consistent with the API contracts
2016-02-01 12:31:26 -05:00
Marty Schoch
47ac091581 fix go vet issues 2016-01-29 13:27:22 -05:00
Marty Schoch
d95c9c1063 optionally set MappingJSONStrict to true for strict parsing
in strict parsing mode, unrecognized keys in the mapping will
cause errors.  this is partial at the moment and does not
include the custom analysis components.
2016-01-29 13:18:54 -05:00
Marty Schoch
710d06e974 add support for native C merge operators 2016-01-27 17:51:07 -05:00
Marty Schoch
10e2207179 adding logging for unexplained observed behavior MB-17298
it would appear that a document lookup for an id fails
but that is a document id that was returned as a search hit
since we're using a stable snapshot, this should not happen
2016-01-25 10:45:58 -05:00
Steve Yen
d97e3caf4f fix comment typo 2016-01-22 09:04:24 -08:00
Marty Schoch
96577606c3 honor the enabled=false flag at the top-level of doc mapping
fixes #331
2016-01-21 19:16:16 -05:00