Steve Yen
5094d2d097
optimize moss PrefixIterator
...
Previously, the PrefixIterator() for moss was implemented by comparing
the prefix bytes on every Next().
With this optimization, the next larger endKeyExclusive is computed at
the iterator's initialization, which allows us to avoid all those
prefix comparisons.
2016-07-21 18:33:34 -07:00
Steve Yen
cbb174b074
optimize moss iterator Next() done/k/v maintenance
2016-07-21 11:10:49 -07:00
Steve Yen
6d7fa0b964
optimize moss iterator checkDone()
2016-07-21 11:10:49 -07:00
Steve Yen
80623f4a8a
MB-20101 - moss KV fix Get() of 0-length vals
...
The moss KV store adapter's Get() implementation was incorrectly
transforming a 0-length val (e.g., []byte{}) into a nil val.
2016-07-15 14:41:30 -07:00
Mark Mindenhall
c3c827aded
Add boltdb config test
2016-06-14 13:36:40 -06:00
Mark Mindenhall
d369bd5c3c
Add bucket fill percent option for boltdb
2016-06-13 18:47:38 -06:00
Marty Schoch
1be5699c54
Merge pull request #381 from MachineShop-IOT/master
...
Compact for boltdb (workaround for #374 )
2016-06-08 00:01:20 -04:00
Steve Yen
4e531ae11b
configurable mossStoreOptions and DeferredSort defaults to true
2016-06-07 17:38:43 -07:00
Mark Mindenhall
09fcc69516
rename defaultBatchSize to defaultCompactBatchSize
2016-06-01 14:25:57 -06:00
Mark Mindenhall
b5a4378a46
Cleanup godoc comments in PR
2016-06-01 13:59:57 -06:00
Mark Mindenhall
fecf7ab5c4
Compact for boltdb (workaround for #374 )
2016-06-01 13:16:43 -06:00
Marty Schoch
92cf2a8974
Merge pull request #376 from MachineShop-IOT/master
...
Remove DictionaryTerm with count 0 during compact (workaround for #374 )
2016-06-01 13:39:30 -04:00
Steve Yen
bf318b489b
enable mossStore as configurable lower-level store
...
Also, bumped moss vendor SHA to latest moss with mossStore.
2016-05-26 13:33:22 -07:00
Mark Mindenhall
04351eb8f1
Move creation of iterator within transaction
2016-05-26 12:29:49 -06:00
Mark Mindenhall
686b20be4f
Remove DictionaryTerm with count 0 during compact (workaround for #374 )
2016-05-26 11:04:53 -06:00
Mark Mindenhall
3aa1d72233
Add compact method to goleveldb store
2016-05-17 16:58:17 -06:00
Marty Schoch
73b514fa4f
do not put +/-Inf or NaN values into the stats map
2016-04-15 13:39:30 -04:00
Marty Schoch
639fb1ab89
remove NativeMergeOperator from core, it requires unsafe
2016-03-24 12:06:43 -04:00
Steve Yen
be2800a8e4
MB-18715 - moss Merge() didn't bump bufUsed correctly
...
And, also allocate more memory for both the partial and full merges.
2016-03-15 17:09:40 -07:00
Steve Yen
c1597842d0
moss lowerLevelUpdate didn't handle batches of size 1
2016-03-11 15:47:23 -08:00
Steve Yen
f1dac8b497
moss defaults to non-nil options.Log
2016-03-09 10:15:11 -08:00
Steve Yen
1d63c55f7c
parse mossLowerLevelMaxBatchSize only when lower-level-store exists
2016-03-09 10:09:15 -08:00
Steve Yen
76b9365928
added moss RegistryCollectionOptions
...
The moss RegistryCollectionOptions allows applications to register
moss-related callback API functions and other advanced feature usage
at process initialization time.
For example, this could be used for moss's OnError(), OnEvent() and
logging callback options.
2016-03-09 09:40:29 -08:00
Marty Schoch
d7292ed891
add support for gathering stats via map for easier consumption
2016-03-07 18:37:46 -05:00
Marty Schoch
147debaa12
expose metrics and moss stats wrapping underlying stats as well
2016-03-04 13:43:39 -05:00
Steve Yen
f6d1bd2c87
moss option MaxPreMergerBatches renamed
2016-03-03 11:18:30 -08:00
Steve Yen
7d67d89a9c
MB-18441 - moss lower-level iterator starts positioned on current
...
The iterator starts off positioned so that Current() is correct, so
invoking Next() right off the bat was incorrect.
2016-03-01 21:45:48 -08:00
Steve Yen
dd1718fa78
index/store/moss uses AllocMerge() instead of Merge()
...
Performance optimization. Before this change, by using Merge()
instead of AllocMerge(), moss's internal batch buf's would be
wastefully, dramatically grown during append()'s to a mis-sized buf.
2016-02-22 11:48:02 -08:00
Steve Yen
ea1a52464d
more index/store/moss err handling
2016-02-20 14:25:42 -08:00
Steve Yen
eb315fa500
integrate index/store/moss KV store
2016-02-20 14:25:42 -08:00
Marty Schoch
40c95513b7
add support for including kvstore stats
2016-02-05 12:26:19 -05:00
Marty Schoch
710d06e974
add support for native C merge operators
2016-01-27 17:51:07 -05:00
Steve Yen
d97e3caf4f
fix comment typo
2016-01-22 09:04:24 -08:00
Steve Yen
d5de1d3da1
metrics implements BatchEx correctly
2016-01-21 11:00:41 -08:00
Marty Schoch
1335eb2a7b
Merge pull request #322 from steveyen/WIP-perf-20160113
...
KVReader.MultiGet and KVWriter.NewBatchEx API's
2016-01-15 14:28:59 -05:00
opennota
8517feb1c6
Fix some typos
2016-01-15 05:46:27 +07:00
Steve Yen
d94ccf2d74
added KVWriter.NewBatchEx() method
2016-01-13 16:19:04 -08:00
Steve Yen
fb048f6c64
added KVReader.MultiGet() method
2016-01-13 15:12:10 -08:00
Steve Yen
70105477cf
added Close() method to KVBatch interface
2016-01-07 17:54:21 -08:00
Marty Schoch
ff11f83842
properly handle errors inside metrics kvstore reporting
2015-11-24 12:52:03 -05:00
Kosov Eugene
45e670b99b
BoltDB wrapper nano optimization which makes code a bit prettier too
2015-11-05 00:27:28 +03:00
Marty Schoch
1a978a4591
fix go vet issues and cleanup reader/iterator
2015-10-26 16:41:58 -04:00
Marty Schoch
f0d282f5f8
add test case for seeing prefix iterators outside of range
...
similar to #256 except for prefix iterators
includes fix for boltdb and gtreap which had incorrect behavior
2015-10-26 16:14:29 -04:00
Patrick Mezard
da72d0c2b9
store_test: deduplicate store initialization
2015-10-20 19:21:01 +02:00
Patrick Mezard
873f483804
gtreap: RangeIterator.Seek should not move before start
2015-10-20 19:12:30 +02:00
Patrick Mezard
5d7628ba3b
boltdb: fix RangeIterator outside of range seeks
...
Two issues:
- Seeking before i.start and iterating returned keys before i.start
- Seeking after the store last key did not invalidate the iterator and
could cause infinite loops.
2015-10-20 19:09:51 +02:00
Patrick Mezard
aada2e7333
store_test: test RangeIterator.Seek on goleveldb
2015-10-20 19:09:38 +02:00
Marty Schoch
f0ee9a3c66
removed commented code and unused functions
2015-10-19 11:13:03 -04:00
Marty Schoch
0f05d1d3ca
Merge branch 'master' into newkvstore
2015-10-09 10:33:41 -04:00
Patrick Mezard
9d5407be13
boltdb: add "nosync" option to force boltdb.DB.NoSync=true
...
Use this option when rebuilding indexes from scratch. In my small case
(~17000 json documents), it reduces indexing from 520s to 250s.
I did not add any test, short of forced indexing termination it only
has performance effects, which are hard to test. And unknown options are
currently ignored.
Issue #240
2015-10-03 14:26:48 +02:00