Steve Yen
bf318b489b
enable mossStore as configurable lower-level store
...
Also, bumped moss vendor SHA to latest moss with mossStore.
2016-05-26 13:33:22 -07:00
Mark Mindenhall
3aa1d72233
Add compact method to goleveldb store
2016-05-17 16:58:17 -06:00
Marty Schoch
73b514fa4f
do not put +/-Inf or NaN values into the stats map
2016-04-15 13:39:30 -04:00
Marty Schoch
639fb1ab89
remove NativeMergeOperator from core, it requires unsafe
2016-03-24 12:06:43 -04:00
Steve Yen
be2800a8e4
MB-18715 - moss Merge() didn't bump bufUsed correctly
...
And, also allocate more memory for both the partial and full merges.
2016-03-15 17:09:40 -07:00
Steve Yen
c1597842d0
moss lowerLevelUpdate didn't handle batches of size 1
2016-03-11 15:47:23 -08:00
Steve Yen
f1dac8b497
moss defaults to non-nil options.Log
2016-03-09 10:15:11 -08:00
Steve Yen
1d63c55f7c
parse mossLowerLevelMaxBatchSize only when lower-level-store exists
2016-03-09 10:09:15 -08:00
Steve Yen
76b9365928
added moss RegistryCollectionOptions
...
The moss RegistryCollectionOptions allows applications to register
moss-related callback API functions and other advanced feature usage
at process initialization time.
For example, this could be used for moss's OnError(), OnEvent() and
logging callback options.
2016-03-09 09:40:29 -08:00
Marty Schoch
d7292ed891
add support for gathering stats via map for easier consumption
2016-03-07 18:37:46 -05:00
Marty Schoch
147debaa12
expose metrics and moss stats wrapping underlying stats as well
2016-03-04 13:43:39 -05:00
Steve Yen
f6d1bd2c87
moss option MaxPreMergerBatches renamed
2016-03-03 11:18:30 -08:00
Steve Yen
7d67d89a9c
MB-18441 - moss lower-level iterator starts positioned on current
...
The iterator starts off positioned so that Current() is correct, so
invoking Next() right off the bat was incorrect.
2016-03-01 21:45:48 -08:00
Steve Yen
dd1718fa78
index/store/moss uses AllocMerge() instead of Merge()
...
Performance optimization. Before this change, by using Merge()
instead of AllocMerge(), moss's internal batch buf's would be
wastefully, dramatically grown during append()'s to a mis-sized buf.
2016-02-22 11:48:02 -08:00
Steve Yen
ea1a52464d
more index/store/moss err handling
2016-02-20 14:25:42 -08:00
Steve Yen
eb315fa500
integrate index/store/moss KV store
2016-02-20 14:25:42 -08:00
Marty Schoch
40c95513b7
add support for including kvstore stats
2016-02-05 12:26:19 -05:00
Marty Schoch
710d06e974
add support for native C merge operators
2016-01-27 17:51:07 -05:00
Steve Yen
d97e3caf4f
fix comment typo
2016-01-22 09:04:24 -08:00
Steve Yen
d5de1d3da1
metrics implements BatchEx correctly
2016-01-21 11:00:41 -08:00
Marty Schoch
1335eb2a7b
Merge pull request #322 from steveyen/WIP-perf-20160113
...
KVReader.MultiGet and KVWriter.NewBatchEx API's
2016-01-15 14:28:59 -05:00
opennota
8517feb1c6
Fix some typos
2016-01-15 05:46:27 +07:00
Steve Yen
d94ccf2d74
added KVWriter.NewBatchEx() method
2016-01-13 16:19:04 -08:00
Steve Yen
fb048f6c64
added KVReader.MultiGet() method
2016-01-13 15:12:10 -08:00
Steve Yen
70105477cf
added Close() method to KVBatch interface
2016-01-07 17:54:21 -08:00
Marty Schoch
ff11f83842
properly handle errors inside metrics kvstore reporting
2015-11-24 12:52:03 -05:00
Kosov Eugene
45e670b99b
BoltDB wrapper nano optimization which makes code a bit prettier too
2015-11-05 00:27:28 +03:00
Marty Schoch
1a978a4591
fix go vet issues and cleanup reader/iterator
2015-10-26 16:41:58 -04:00
Marty Schoch
f0d282f5f8
add test case for seeing prefix iterators outside of range
...
similar to #256 except for prefix iterators
includes fix for boltdb and gtreap which had incorrect behavior
2015-10-26 16:14:29 -04:00
Patrick Mezard
da72d0c2b9
store_test: deduplicate store initialization
2015-10-20 19:21:01 +02:00
Patrick Mezard
873f483804
gtreap: RangeIterator.Seek should not move before start
2015-10-20 19:12:30 +02:00
Patrick Mezard
5d7628ba3b
boltdb: fix RangeIterator outside of range seeks
...
Two issues:
- Seeking before i.start and iterating returned keys before i.start
- Seeking after the store last key did not invalidate the iterator and
could cause infinite loops.
2015-10-20 19:09:51 +02:00
Patrick Mezard
aada2e7333
store_test: test RangeIterator.Seek on goleveldb
2015-10-20 19:09:38 +02:00
Marty Schoch
f0ee9a3c66
removed commented code and unused functions
2015-10-19 11:13:03 -04:00
Marty Schoch
0f05d1d3ca
Merge branch 'master' into newkvstore
2015-10-09 10:33:41 -04:00
Patrick Mezard
9d5407be13
boltdb: add "nosync" option to force boltdb.DB.NoSync=true
...
Use this option when rebuilding indexes from scratch. In my small case
(~17000 json documents), it reduces indexing from 520s to 250s.
I did not add any test, short of forced indexing termination it only
has performance effects, which are hard to test. And unknown options are
currently ignored.
Issue #240
2015-10-03 14:26:48 +02:00
Marty Schoch
d06b526cbf
more refactoring
2015-09-28 16:50:27 -04:00
Marty Schoch
900f1b4a67
major kvstore interface and impl overhaul
...
clarified the interface contract
2015-09-23 11:25:47 -07:00
Marty Schoch
f81b2be334
major refactor of bleve configuration
...
see #221 for full details
2015-09-16 17:10:59 -04:00
Marty Schoch
7ad7659ce5
add support for using null kvstore outside of bleve internals
2015-09-02 11:50:06 -04:00
Marty Schoch
07d37ca38a
add important rocksdb config options
2015-09-02 11:49:42 -04:00
Marty Schoch
3e60ca24ec
support using end key on forestdb iterator for term freq lookup
...
also additoanl forestdb configs
2015-08-18 16:22:02 -04:00
Marty Schoch
d3dda3d0ea
fixup config parsing and add new options
2015-08-12 13:18:23 -04:00
Marty Schoch
bf8dcae76b
removing build tags
2015-07-28 18:59:10 -04:00
Marty Schoch
17ef48f82a
switching back to the canonical goleveldb repo
2015-07-08 12:21:17 -06:00
Marty Schoch
bf80f4628e
fix bug in curent goleveldb (must copy during iteration)
...
also changed over to mschoch fork of goleveldb (temporary)
the change to my fork is pending some read-only issues described
here: https://github.com/syndtr/goleveldb/issues/111
hopefully we can find a path forward, and get that addressed upstream
2015-07-06 18:00:05 -04:00
Marty Schoch
7be7ecdf8e
fix batch indexing bug, incremented docCount before commit
...
fixes #211
2015-06-08 14:14:05 -04:00
Marty Schoch
a52d3b5c07
put in hack to allow boltdb reader isolation test to pass
...
in boltdb, long readers *MAY* block a writer. in particular if
the write requires additional allocation, it must acquire a lock
already held by the reader. in general this is not a problem
for bleve (though it can affect performance in some cases), but
it is a problem for the reader isolation test. this commit
adds a hack to try and avoid the need for additional allocation
closes #208
2015-05-21 11:39:59 -04:00
Steve Yen
e98ae8ab71
update metrics store to latest kvstore api
2015-04-27 11:01:53 -07:00
Marty Schoch
16f538d7b7
close documents returned by iterator before losing their reference
...
fixes #194
2015-04-24 17:48:21 -04:00
