Steve Yen
dd1718fa78
index/store/moss uses AllocMerge() instead of Merge()
...
Performance optimization. Before this change, by using Merge()
instead of AllocMerge(), moss's internal batch buf's would be
wastefully, dramatically grown during append()'s to a mis-sized buf.
2016-02-22 11:48:02 -08:00
Steve Yen
ea1a52464d
more index/store/moss err handling
2016-02-20 14:25:42 -08:00
Steve Yen
eb315fa500
integrate index/store/moss KV store
2016-02-20 14:25:42 -08:00
Marty Schoch
40c95513b7
add support for including kvstore stats
2016-02-05 12:26:19 -05:00
Marty Schoch
710d06e974
add support for native C merge operators
2016-01-27 17:51:07 -05:00
Steve Yen
d97e3caf4f
fix comment typo
2016-01-22 09:04:24 -08:00
Steve Yen
d5de1d3da1
metrics implements BatchEx correctly
2016-01-21 11:00:41 -08:00
Marty Schoch
1335eb2a7b
Merge pull request #322 from steveyen/WIP-perf-20160113
...
KVReader.MultiGet and KVWriter.NewBatchEx API's
2016-01-15 14:28:59 -05:00
opennota
8517feb1c6
Fix some typos
2016-01-15 05:46:27 +07:00
Steve Yen
d94ccf2d74
added KVWriter.NewBatchEx() method
2016-01-13 16:19:04 -08:00
Steve Yen
fb048f6c64
added KVReader.MultiGet() method
2016-01-13 15:12:10 -08:00
Steve Yen
70105477cf
added Close() method to KVBatch interface
2016-01-07 17:54:21 -08:00
Marty Schoch
ff11f83842
properly handle errors inside metrics kvstore reporting
2015-11-24 12:52:03 -05:00
Kosov Eugene
45e670b99b
BoltDB wrapper nano optimization which makes code a bit prettier too
2015-11-05 00:27:28 +03:00
Marty Schoch
1a978a4591
fix go vet issues and cleanup reader/iterator
2015-10-26 16:41:58 -04:00
Marty Schoch
f0d282f5f8
add test case for seeing prefix iterators outside of range
...
similar to #256 except for prefix iterators
includes fix for boltdb and gtreap which had incorrect behavior
2015-10-26 16:14:29 -04:00
Patrick Mezard
da72d0c2b9
store_test: deduplicate store initialization
2015-10-20 19:21:01 +02:00
Patrick Mezard
873f483804
gtreap: RangeIterator.Seek should not move before start
2015-10-20 19:12:30 +02:00
Patrick Mezard
5d7628ba3b
boltdb: fix RangeIterator outside of range seeks
...
Two issues:
- Seeking before i.start and iterating returned keys before i.start
- Seeking after the store last key did not invalidate the iterator and
could cause infinite loops.
2015-10-20 19:09:51 +02:00
Patrick Mezard
aada2e7333
store_test: test RangeIterator.Seek on goleveldb
2015-10-20 19:09:38 +02:00
Marty Schoch
f0ee9a3c66
removed commented code and unused functions
2015-10-19 11:13:03 -04:00
Marty Schoch
0f05d1d3ca
Merge branch 'master' into newkvstore
2015-10-09 10:33:41 -04:00
Patrick Mezard
9d5407be13
boltdb: add "nosync" option to force boltdb.DB.NoSync=true
...
Use this option when rebuilding indexes from scratch. In my small case
(~17000 json documents), it reduces indexing from 520s to 250s.
I did not add any test, short of forced indexing termination it only
has performance effects, which are hard to test. And unknown options are
currently ignored.
Issue #240
2015-10-03 14:26:48 +02:00
Marty Schoch
d06b526cbf
more refactoring
2015-09-28 16:50:27 -04:00
Marty Schoch
900f1b4a67
major kvstore interface and impl overhaul
...
clarified the interface contract
2015-09-23 11:25:47 -07:00
Marty Schoch
f81b2be334
major refactor of bleve configuration
...
see #221 for full details
2015-09-16 17:10:59 -04:00
Marty Schoch
7ad7659ce5
add support for using null kvstore outside of bleve internals
2015-09-02 11:50:06 -04:00
Marty Schoch
07d37ca38a
add important rocksdb config options
2015-09-02 11:49:42 -04:00
Marty Schoch
3e60ca24ec
support using end key on forestdb iterator for term freq lookup
...
also additoanl forestdb configs
2015-08-18 16:22:02 -04:00
Marty Schoch
d3dda3d0ea
fixup config parsing and add new options
2015-08-12 13:18:23 -04:00
Marty Schoch
bf8dcae76b
removing build tags
2015-07-28 18:59:10 -04:00
Marty Schoch
17ef48f82a
switching back to the canonical goleveldb repo
2015-07-08 12:21:17 -06:00
Marty Schoch
bf80f4628e
fix bug in curent goleveldb (must copy during iteration)
...
also changed over to mschoch fork of goleveldb (temporary)
the change to my fork is pending some read-only issues described
here: https://github.com/syndtr/goleveldb/issues/111
hopefully we can find a path forward, and get that addressed upstream
2015-07-06 18:00:05 -04:00
Marty Schoch
7be7ecdf8e
fix batch indexing bug, incremented docCount before commit
...
fixes #211
2015-06-08 14:14:05 -04:00
Marty Schoch
a52d3b5c07
put in hack to allow boltdb reader isolation test to pass
...
in boltdb, long readers *MAY* block a writer. in particular if
the write requires additional allocation, it must acquire a lock
already held by the reader. in general this is not a problem
for bleve (though it can affect performance in some cases), but
it is a problem for the reader isolation test. this commit
adds a hack to try and avoid the need for additional allocation
closes #208
2015-05-21 11:39:59 -04:00
Steve Yen
e98ae8ab71
update metrics store to latest kvstore api
2015-04-27 11:01:53 -07:00
Marty Schoch
16f538d7b7
close documents returned by iterator before losing their reference
...
fixes #194
2015-04-24 17:48:21 -04:00
Marty Schoch
b54a59139c
change forestdb imports to couchbase not couchbaselabs
2015-04-24 17:35:01 -04:00
Marty Schoch
452fea6a24
adding initial impl of rocksdb kv store
2015-04-24 17:19:44 -04:00
Marty Schoch
a9c07acbfa
refactor of kvstore api to support native merge in rocksdb
...
refactor to share code in emulated batch
refactor to share code in emulated merge
refactor index kvstore benchmarks to share more code
refactor index kvstore benchmarks to be more repeatable
2015-04-24 17:13:50 -04:00
indraniel
a62320a50e
+ fix goleveldb's BytesSafeAfterClose() on reader
...
- it should be set to false
2015-04-10 15:45:22 -05:00
Marty Schoch
5f66bd84c7
fix issues identified by errcheck
2015-04-10 14:59:05 -04:00
indraniel
54ab493b3e
+ correctly copy bytes from the goleveldb store
...
- this is part of a recent bleve KVStore API change.
See the following two google group threads for more details:
* [help adding goleveldb as an alternative Key/Value store for bleve][1]
* [bleve search performance improvement][2]
[1]: https://groups.google.com/forum/#!topic/bleve/aHZ8gmihLiY
[2]: https://groups.google.com/forum/#!topic/bleve/aTyqsSnbhik
2015-04-10 11:25:23 -05:00
indraniel
81bef38cce
Revert "+ make copies of the []bytes returned by goleveldb"
...
This reverts commit cb8c1741289a0f00b30733e0d52d9d81d1199603.
This commit is no longer desired. The KV store API has been changed to
better address this issue.
For more details, see the google group conversation thread at:
https://groups.google.com/forum/#!topic/bleve/aHZ8gmihLiY
2015-04-10 11:12:44 -05:00
indraniel
3a70401835
+ make copies of the []bytes returned by goleveldb
...
- The byte strings returned by goleveldb aren't necessarily safe. See
the following google group thread:
https://groups.google.com/forum/#!topic/bleve/aHZ8gmihLiY
This code change is based on the gist created here:
https://groups.google.com/forum/#!topic/bleve/aHZ8gmihLiY
2015-04-10 11:08:02 -05:00
indraniel
a0a2a61050
+ keep 'get' consistent with levigo implementation
...
- this change keeps the method behavior consistent with the
levigo/leveldb implementation.
- don't issue an err if a key isn't found
2015-04-10 11:08:02 -05:00
indraniel
5e55fa2866
+ keep 'getWithSnapshot' consistent with levigo implementation
...
- this change keeps the method behavior consistent with the
levigo/leveldb implementation.
- the leveldb store_test.go and goleveldb store_test.go are now
identical.
2015-04-10 11:08:02 -05:00
indraniel
caa19e6c36
+ initial stub of goleveldb package
...
- This is a first-pass introduction. Things may not be working
correctly yet.
2015-04-10 11:08:02 -05:00
Marty Schoch
ab24772bf0
fix issues identified by errcheck
...
part of #169
2015-04-07 16:34:29 -04:00
Marty Schoch
56c4a09de1
fix issues identified by errcheck
...
part of #169
2015-04-07 15:39:56 -04:00