gibheer/bleve
0
0
Fork 0
Code
529 Commits 10 Branches 0 Tags 12 MiB
a52d3b5c07
Commit Graph

81 Commits

Author SHA1 Message Date
Marty Schoch
a52d3b5c07 put in hack to allow boltdb reader isolation test to pass 
in boltdb, long readers *MAY* block a writer.  in particular if
the write requires additional allocation, it must acquire a lock
already held by the reader.  in general this is not a problem
for bleve (though it can affect performance in some cases), but
it is a problem for the reader isolation test.  this commit
adds a hack to try and avoid the need for additional allocation
closes #208
 2015-05-21 11:39:59 -04:00
Steve Yen
e98ae8ab71 update metrics store to latest kvstore api 2015-04-27 11:01:53 -07:00
Marty Schoch
16f538d7b7 close documents returned by iterator before losing their reference 
fixes #194
 2015-04-24 17:48:21 -04:00
Marty Schoch
b54a59139c change forestdb imports to couchbase not couchbaselabs 2015-04-24 17:35:01 -04:00
Marty Schoch
452fea6a24 adding initial impl of rocksdb kv store 2015-04-24 17:19:44 -04:00
Marty Schoch
a9c07acbfa refactor of kvstore api to support native merge in rocksdb 
refactor to share code in emulated batch
refactor to share code in emulated merge
refactor index kvstore benchmarks to share more code
refactor index kvstore benchmarks to be more repeatable
 2015-04-24 17:13:50 -04:00
indraniel
a62320a50e + fix goleveldb's BytesSafeAfterClose() on reader 
- it should be set to false
 2015-04-10 15:45:22 -05:00
Marty Schoch
5f66bd84c7 fix issues identified by errcheck 2015-04-10 14:59:05 -04:00
indraniel
54ab493b3e + correctly copy bytes from the goleveldb store 
- this is part of a recent bleve KVStore API change.

    See the following two google group threads for more details:

    * [help adding goleveldb as an alternative Key/Value store for bleve][1]
    * [bleve search performance improvement][2]

    [1]: https://groups.google.com/forum/#!topic/bleve/aHZ8gmihLiY
    [2]: https://groups.google.com/forum/#!topic/bleve/aTyqsSnbhik
 2015-04-10 11:25:23 -05:00
indraniel
81bef38cce Revert "+ make copies of the []bytes returned by goleveldb" 
This reverts commit cb8c1741289a0f00b30733e0d52d9d81d1199603.

This commit is no longer desired. The KV store API has been changed to
better address this issue.

For more details, see the google group conversation thread at:

https://groups.google.com/forum/#!topic/bleve/aHZ8gmihLiY
 2015-04-10 11:12:44 -05:00
indraniel
3a70401835 + make copies of the []bytes returned by goleveldb 
- The byte strings returned by goleveldb aren't necessarily safe.  See
    the following google group thread:

    https://groups.google.com/forum/#!topic/bleve/aHZ8gmihLiY

    This code change is based on the gist created here:

    https://groups.google.com/forum/#!topic/bleve/aHZ8gmihLiY
 2015-04-10 11:08:02 -05:00
indraniel
a0a2a61050 + keep 'get' consistent with levigo implementation 
- this change keeps the method behavior consistent with the
     levigo/leveldb implementation.

   - don't issue an err if a key isn't found
 2015-04-10 11:08:02 -05:00
indraniel
5e55fa2866 + keep 'getWithSnapshot' consistent with levigo implementation 
- this change keeps the method behavior consistent with the
     levigo/leveldb implementation.

   - the leveldb store_test.go and goleveldb store_test.go are now
     identical.
 2015-04-10 11:08:02 -05:00
indraniel
caa19e6c36 + initial stub of goleveldb package 
- This is a first-pass introduction. Things may not be working
    correctly yet.
 2015-04-10 11:08:02 -05:00
Marty Schoch
ab24772bf0 fix issues identified by errcheck 
part of #169
 2015-04-07 16:34:29 -04:00
Marty Schoch
56c4a09de1 fix issues identified by errcheck 
part of #169
 2015-04-07 15:39:56 -04:00
Marty Schoch
93e01a803e fix issues identified by errcheck 
part of #169
 2015-04-07 14:52:00 -04:00
Marty Schoch
dd921d31e3 undoing f92ab131e4 
we now guarantee bytes were copied earlier in the chain
the kv store is NOT responsible for making an additional copy
closes #181
 2015-04-07 11:12:28 -04:00
Marty Schoch
443c0252e0 fix another metrics BytesSafeAfterClose() loop 
closes #184
 2015-04-03 21:17:23 -04:00
Steve Yen
efc39a6857 fix metrics BytesSafeAfterClose() loop 
fixes issue 184
 2015-04-03 16:36:32 -07:00
Marty Schoch
867110e03b major improvements to index row encoding 
improvements uncovered some issues with how k/v data was copied
or not.  to address this, kv abstraction layer now lets impl
specify if the bytes returned are safe to use after a reader
(or writer since writers are also readers) are closed
See index/store/KVReader - BytesSafeAfterClose() bool
false is the safe value if you're not sure
it will cause index impls to copy the data
Some kv impls already have created a copy a the C-api barrier
in which case they can safely return true.

Overall this yields ~25% speedup for searches with leveldb.
It yields ~10% speedup for boltdb.
Returning stored fields is now slower with boltdb, as previously
we were returning unsafe bytes.
 2015-04-03 16:50:48 -04:00
Steve Yen
dbf50b7f29 KVStore gtreap allows only 1 writer at a time 2015-03-26 16:40:18 -07:00
Steve Yen
f92ab131e4 KVStore gtreap implementation copies value bytes 2015-03-26 14:46:37 -07:00
Steve Yen
78453dab7d metrics KVStore now tracks last 100 errors 2015-03-19 18:41:16 -07:00
Steve Yen
38ee9be353 added some batch size 1000 microbenchmarks 2015-01-30 15:58:39 -08:00
Steve Yen
7d6a6aeaa8 single append for inmem KVStore batch 2015-01-29 11:14:08 -08:00
Steve Yen
5a30d36b17 cznicb KVStore uses Put() for faster read-modify-write 2015-01-29 11:02:01 -08:00
Steve Yen
b054cddf76 gtreap KVStore does 1 append for batch Set/Delete 2015-01-29 10:49:39 -08:00
Steve Yen
05d222f490 cznicb KVStore batch uses <2 appends per Set/Delete 2015-01-29 10:22:13 -08:00
Steve Yen
c5c59e61f4 make leveldb faster with non-zero sized batch 2015-01-29 10:20:26 -08:00
Steve Yen
1c1774d4ad throw away data even faster in null KVStore 2015-01-29 10:17:21 -08:00
Steve Yen
782ad94e01 added debug tag for metrics KVStore 2015-01-16 11:18:40 -08:00
Marty Schoch
eebc8e7825 more debuging around forestdb snapshots 2015-01-16 14:18:28 -05:00
Marty Schoch
ba978ea27e improving log messages 2015-01-16 14:07:47 -05:00
Marty Schoch
09fe749913 default to autocompaction for forestdb 2015-01-16 13:35:43 -05:00
Steve Yen
12dc2aff93 add go1.4 build tag to cznicb KVStore 
This is because github.com/cznic/b depends on sync.Pool.
 2015-01-15 15:54:25 -08:00
Steve Yen
11ee0209ad no leading zeros for metrics CSV output 2015-01-15 15:09:53 -08:00
Steve Yen
202191201c added WriteCSV() to metrics KVStore 2015-01-15 14:11:15 -08:00
Steve Yen
9be4e217bc metrics KVStore tracks perf metrics on a wrapped KVStore 2015-01-15 11:42:41 -08:00
Steve Yen
ea0a8657f3 added cznicb in-memory kvstore (no reader isolation) 2015-01-13 17:35:28 -08:00
Marty Schoch
362d240b09 added configurable options to leveldb 2015-01-13 16:24:51 -05:00
Steve Yen
d6e6f655c9 initialize forestdb config if provided 2015-01-13 12:03:24 -08:00
Steve Yen
1fa80ffc40 pass config to forestdb Open() 2015-01-13 11:04:02 -08:00
Steve Yen
3a00a968f2 close levigo's read & write options 2015-01-12 18:42:19 -08:00
Steve Yen
c20726bb93 close levigo.Options when db is closed 2015-01-12 18:42:19 -08:00
Steve Yen
603c3af8bb added gtreap in-memory, copy-on-write KVStore 2015-01-12 11:26:21 -08:00
Steve Yen
ae3600aeea expose forestdb rollback methods 2015-01-06 18:59:02 -08:00
Steve Yen
5467e0a385 forestdb registered name fixed 2015-01-06 17:36:05 -08:00
Marty Schoch
38bdcbeb62 update to new forestdb iterator api 2014-12-27 13:15:14 -08:00
Silvan Jegen
ef18dfe4cd Fix typos in comments and strings 2014-12-18 18:43:12 +01:00