gibheer/bleve
0
0
Fork 0
Code
796 Commits 10 Branches 0 Tags 12 MiB
f5344b191a
Commit Graph

796 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Marty Schoch
f5344b191a Merge pull request #324 from Shugyousha/trivial 
Trivial cleanup
 2016-01-14 10:51:06 -05:00
Silvan Jegen
329d8e00cd Fix error string 2016-01-14 16:42:01 +01:00
Silvan Jegen
d326898f7b Remove unneeded brackets 2016-01-14 16:41:41 +01:00
Marty Schoch
b286466787 Merge branch 'slavikm-master' 2016-01-13 17:13:28 -05:00
Marty Schoch
737a07ddcf update dynamic boolean fields to honor Index/Store Dynamic property 2016-01-13 17:12:40 -05:00
Marty Schoch
2479ddef2e fixed errcheck issues 2016-01-13 17:10:13 -05:00
Marty Schoch
af25e724f6 Merge branch 'master' of https://github.com/slavikm/bleve into slavikm-master 2016-01-13 16:10:59 -05:00
Marty Schoch
81f8509890 add the ability to not store dynamic fields globally 2016-01-12 17:27:11 -05:00
Marty Schoch
b7c03dae1a boolean query defaults to minShould of 0 
fixes #258
 2016-01-12 16:30:10 -05:00
Marty Schoch
847360f1d0 Merge pull request #320 from Shugyousha/gofmt 
Run go fmt ./...
 2016-01-12 16:24:03 -05:00
Silvan Jegen
35ac2b2bee Run go fmt ./... 2016-01-12 22:15:50 +01:00
slavikm
c4f9828b78 fixed typo in comment 2016-01-12 11:31:53 -08:00
Marty Schoch
c3ddf038ab Merge pull request #316 from steveyen/WIP-perf-20160110 
upside-down backindex read concurrency
 2016-01-12 12:53:37 -05:00
Steve Yen
0e72b949b3 upside_down batchRows() takes array of arrays 
In order to spend less time in append(), this change in upside_down
(similar to another recent performance change in firestorm) builds up
an array of arrays as the eventual input to batchRows().
 2016-01-11 18:11:21 -08:00
slavikm
680be52f87 Implemented boolean field support 2016-01-11 17:18:03 -08:00
Steve Yen
7ce7d98cba upside_down merge dictionary deltas before using batch.Merge() 
This change performs more dictionary delta incr/decr math in
batchRows() instead of in the KVStore ExecuteBatch() machinery.
 2016-01-11 16:52:07 -08:00
Steve Yen
94273d5fa9 upside_down process internal rows earlier 
With this change, internal rows are processed while we're waiting for
backIndex rows to be retrieved.
 2016-01-11 16:25:35 -08:00
Steve Yen
bb5cd8f3d6 upside_down merge backIndexRow concurrently 
Previously, the code would gather all the backIndexRows before
processing them.  This change instead merges the backIndexRows
concurrently on the theory that we might as well make progress on
compute & processing tasks while waiting for the rest of the back
index rows to be fetched from the KVStore.
 2016-01-10 18:50:42 -08:00
Steve Yen
c3b5246b0c upside_down track analysis time tighter; and comments 2016-01-10 15:36:54 -08:00
Steve Yen
d3dd40d334 upside_down retrieves backindex concurrently with analysis 
Start backindex reading concurrently with analysi to try to utilize
more I/O bandwidth.

The analysis time vs indexing time stats tracking are also now "off",
since there's now concurrency between those actiivties.

One tradeoff is that the lock area in upside_down Batch() is increased
as part of this change.
 2016-01-10 15:18:28 -08:00
Steve Yen
bff95eef70 firestorm close kvwriter sooner 2016-01-10 15:18:27 -08:00
Marty Schoch
d533b326f6 Merge pull request #315 from steveyen/WIP-perf-20160107 
KVBatch Close() method
 2016-01-08 09:47:37 -05:00
Steve Yen
860de28a28 fix memory leak by closing batches in batchRows() 2016-01-07 17:59:42 -08:00
Steve Yen
70105477cf added Close() method to KVBatch interface 2016-01-07 17:54:21 -08:00
Marty Schoch
2b947e8c14 Merge branch 'steveyen-WIP-perf-20160106' 2016-01-07 15:42:53 -05:00
Marty Schoch
48fcd5a7d5 Merge branch 'WIP-perf-20160106' of https://github.com/steveyen/bleve into steveyen-WIP-perf-20160106 2016-01-07 15:40:29 -05:00
Marty Schoch
665f5c58e1 fix errcheck violation 2016-01-07 11:11:43 -05:00
Marty Schoch
e54db33346 try testing slightly different way 2016-01-07 11:06:18 -05:00
Marty Schoch
cd940cc375 add another check to try to understand test failure on travis 2016-01-07 10:45:20 -05:00
Marty Schoch
155e496121 switch to containerized builds 
switch to go 1.5 (required by errcheck now)
do not install go vet (part of Go now)
 2016-01-07 09:57:23 -05:00
Steve Yen
846912d083 upside_down udc.termVectorsFromTokenFreq rows append optimization 2016-01-07 00:48:34 -08:00
Steve Yen
8b980bd2ef firestorm avoid extra goroutine, similar to upside_down 2016-01-07 00:43:27 -08:00
Steve Yen
fbd0e7bfe9 upside_down backIndexTermEntries precalloc'ed capacity 2016-01-07 00:23:25 -08:00
Steve Yen
4eee8821f9 upside_down storeField/indexField append to provided arrays 
Taking another optimization from firestorm, upside_down's
storeField()/indexField() funcs now also append() to passed-in arrays
rather than always allocating their own arrays.
 2016-01-07 00:13:46 -08:00
Steve Yen
1af2927967 upside_down gets analysis perf rows optimizations from firestorm 2016-01-06 23:53:13 -08:00
Steve Yen
82b8b3468e upside_down analysis converts to docIDBytes once 2016-01-06 23:38:02 -08:00
Steve Yen
d6a997d8c1 firestorm gtreap lookup once per snapshot docID 
Previously, firestorm would lookup docID's in the inFlight gtreap for
every candidate docNum, and this change moves the lookup to outside of
the loop.
 2016-01-06 16:46:15 -08:00
Steve Yen
024848ac91 firestorm valid docNum finding, fixes #310 2016-01-06 16:04:56 -08:00
Steve Yen
7df07f94fa firestorm use the ParseKey() funcs to avoid unneeded value parsing 
With this change, the row allocation also happens only once per loop,
instead of once per item.
 2016-01-06 15:53:12 -08:00
Steve Yen
009d59222a firestorm StoredRow.ParseKey() func 2016-01-06 15:46:26 -08:00
Steve Yen
8389027ae8 firestorm TermFreqRow.ParseKey() func 2016-01-06 15:32:09 -08:00
Marty Schoch
83cd8da394 Merge pull request #307 from steveyen/WIP-perf-20160105 
analyze locations only if includeTermVectors enabled
 2016-01-05 16:04:59 -05:00
Steve Yen
89d17f01ef analyze locations only if includeTermVectors enabled 
With this change, TermLocations are computed and maintained only if
includeTermVectors is enabled, for higher performance.
 2016-01-05 12:46:46 -08:00
Marty Schoch
e5c1af4164 add travis config to run integration tests against firestorm 2016-01-05 13:00:36 -05:00
Marty Schoch
ab67b2f642 Merge pull request #267 from pmezard/doc-dump-methods 
index: document DumpAll, DumpDoc and DumpFields methods
 2016-01-05 09:55:35 -05:00
Marty Schoch
db7363fba1 Merge pull request #305 from steveyen/WIP-perf-20160102 
perf 20160102
 2016-01-05 08:54:47 -05:00
Steve Yen
70b7e73c82 firestorm compensator inFlight.Get() might return nil 2016-01-03 10:21:54 -08:00
Steve Yen
fb8c9a7475 firestorm.Batch() collects [][]IndexRows instead of []IndexRow 
Rather than append() all received rows into a flat []IndexRow during
the result gathering loop, this change instead collects the analysis
result rows into a [][]IndexRow, which avoids extra copying.

As part of this, firestorm batchRows() now takes the [][]IndexRow as
its input.
 2016-01-02 12:30:47 -08:00
Steve Yen
1c5b84911d firestorm DictUpdater NotifyBatch is more async 2016-01-02 12:21:25 -08:00
Steve Yen
b241242465 firestorm.Analyze() preallocs rows, with analyzeField() func 
The new analyzeField() helper func is used for both regular fields and
for composite fields.

With this change, all analysis is done up front, for both regular
fields and composite fields.

After analysis, this change counts up all the row capacity needed and
extends the AnalysisResult.Rows in one shot, as opposed to the
previous approach of dynamically growing the array as needed during
append()'s.

Also, in this change, the TermFreqRow for _id is added first, which
seems more correct.
 2016-01-02 12:21:25 -08:00