Marty Schoch
452fea6a24
adding initial impl of rocksdb kv store
2015-04-24 17:19:44 -04:00
Marty Schoch
a9c07acbfa
refactor of kvstore api to support native merge in rocksdb
...
refactor to share code in emulated batch
refactor to share code in emulated merge
refactor index kvstore benchmarks to share more code
refactor index kvstore benchmarks to be more repeatable
2015-04-24 17:13:50 -04:00
indraniel
a62320a50e
+ fix goleveldb's BytesSafeAfterClose() on reader
...
- it should be set to false
2015-04-10 15:45:22 -05:00
Marty Schoch
5f66bd84c7
fix issues identified by errcheck
2015-04-10 14:59:05 -04:00
indraniel
54ab493b3e
+ correctly copy bytes from the goleveldb store
...
- this is part of a recent bleve KVStore API change.
See the following two google group threads for more details:
* [help adding goleveldb as an alternative Key/Value store for bleve][1]
* [bleve search performance improvement][2]
[1]: https://groups.google.com/forum/#!topic/bleve/aHZ8gmihLiY
[2]: https://groups.google.com/forum/#!topic/bleve/aTyqsSnbhik
2015-04-10 11:25:23 -05:00
indraniel
81bef38cce
Revert "+ make copies of the []bytes returned by goleveldb"
...
This reverts commit cb8c1741289a0f00b30733e0d52d9d81d1199603.
This commit is no longer desired. The KV store API has been changed to
better address this issue.
For more details, see the google group conversation thread at:
https://groups.google.com/forum/#!topic/bleve/aHZ8gmihLiY
2015-04-10 11:12:44 -05:00
indraniel
3a70401835
+ make copies of the []bytes returned by goleveldb
...
- The byte strings returned by goleveldb aren't necessarily safe. See
the following google group thread:
https://groups.google.com/forum/#!topic/bleve/aHZ8gmihLiY
This code change is based on the gist created here:
https://groups.google.com/forum/#!topic/bleve/aHZ8gmihLiY
2015-04-10 11:08:02 -05:00
indraniel
a0a2a61050
+ keep 'get' consistent with levigo implementation
...
- this change keeps the method behavior consistent with the
levigo/leveldb implementation.
- don't issue an err if a key isn't found
2015-04-10 11:08:02 -05:00
indraniel
5e55fa2866
+ keep 'getWithSnapshot' consistent with levigo implementation
...
- this change keeps the method behavior consistent with the
levigo/leveldb implementation.
- the leveldb store_test.go and goleveldb store_test.go are now
identical.
2015-04-10 11:08:02 -05:00
indraniel
caa19e6c36
+ initial stub of goleveldb package
...
- This is a first-pass introduction. Things may not be working
correctly yet.
2015-04-10 11:08:02 -05:00
Marty Schoch
ab24772bf0
fix issues identified by errcheck
...
part of #169
2015-04-07 16:34:29 -04:00
Marty Schoch
56c4a09de1
fix issues identified by errcheck
...
part of #169
2015-04-07 15:39:56 -04:00
Marty Schoch
93e01a803e
fix issues identified by errcheck
...
part of #169
2015-04-07 14:52:00 -04:00
Marty Schoch
dd921d31e3
undoing f92ab131e4
...
we now guarantee bytes were copied earlier in the chain
the kv store is NOT responsible for making an additional copy
closes #181
2015-04-07 11:12:28 -04:00
Marty Schoch
443c0252e0
fix another metrics BytesSafeAfterClose() loop
...
closes #184
2015-04-03 21:17:23 -04:00
Steve Yen
efc39a6857
fix metrics BytesSafeAfterClose() loop
...
fixes issue 184
2015-04-03 16:36:32 -07:00
Marty Schoch
867110e03b
major improvements to index row encoding
...
improvements uncovered some issues with how k/v data was copied
or not. to address this, kv abstraction layer now lets impl
specify if the bytes returned are safe to use after a reader
(or writer since writers are also readers) are closed
See index/store/KVReader - BytesSafeAfterClose() bool
false is the safe value if you're not sure
it will cause index impls to copy the data
Some kv impls already have created a copy a the C-api barrier
in which case they can safely return true.
Overall this yields ~25% speedup for searches with leveldb.
It yields ~10% speedup for boltdb.
Returning stored fields is now slower with boltdb, as previously
we were returning unsafe bytes.
2015-04-03 16:50:48 -04:00
Steve Yen
dbf50b7f29
KVStore gtreap allows only 1 writer at a time
2015-03-26 16:40:18 -07:00
Steve Yen
f92ab131e4
KVStore gtreap implementation copies value bytes
2015-03-26 14:46:37 -07:00
Steve Yen
78453dab7d
metrics KVStore now tracks last 100 errors
2015-03-19 18:41:16 -07:00
Steve Yen
38ee9be353
added some batch size 1000 microbenchmarks
2015-01-30 15:58:39 -08:00
Steve Yen
7d6a6aeaa8
single append for inmem KVStore batch
2015-01-29 11:14:08 -08:00
Steve Yen
5a30d36b17
cznicb KVStore uses Put() for faster read-modify-write
2015-01-29 11:02:01 -08:00
Steve Yen
b054cddf76
gtreap KVStore does 1 append for batch Set/Delete
2015-01-29 10:49:39 -08:00
Steve Yen
05d222f490
cznicb KVStore batch uses <2 appends per Set/Delete
2015-01-29 10:22:13 -08:00
Steve Yen
c5c59e61f4
make leveldb faster with non-zero sized batch
2015-01-29 10:20:26 -08:00
Steve Yen
1c1774d4ad
throw away data even faster in null KVStore
2015-01-29 10:17:21 -08:00
Steve Yen
782ad94e01
added debug tag for metrics KVStore
2015-01-16 11:18:40 -08:00
Marty Schoch
eebc8e7825
more debuging around forestdb snapshots
2015-01-16 14:18:28 -05:00
Marty Schoch
ba978ea27e
improving log messages
2015-01-16 14:07:47 -05:00
Marty Schoch
09fe749913
default to autocompaction for forestdb
2015-01-16 13:35:43 -05:00
Steve Yen
12dc2aff93
add go1.4 build tag to cznicb KVStore
...
This is because github.com/cznic/b depends on sync.Pool.
2015-01-15 15:54:25 -08:00
Steve Yen
11ee0209ad
no leading zeros for metrics CSV output
2015-01-15 15:09:53 -08:00
Steve Yen
202191201c
added WriteCSV() to metrics KVStore
2015-01-15 14:11:15 -08:00
Steve Yen
9be4e217bc
metrics KVStore tracks perf metrics on a wrapped KVStore
2015-01-15 11:42:41 -08:00
Steve Yen
ea0a8657f3
added cznicb in-memory kvstore (no reader isolation)
2015-01-13 17:35:28 -08:00
Marty Schoch
362d240b09
added configurable options to leveldb
2015-01-13 16:24:51 -05:00
Steve Yen
d6e6f655c9
initialize forestdb config if provided
2015-01-13 12:03:24 -08:00
Steve Yen
1fa80ffc40
pass config to forestdb Open()
2015-01-13 11:04:02 -08:00
Steve Yen
3a00a968f2
close levigo's read & write options
2015-01-12 18:42:19 -08:00
Steve Yen
c20726bb93
close levigo.Options when db is closed
2015-01-12 18:42:19 -08:00
Steve Yen
603c3af8bb
added gtreap in-memory, copy-on-write KVStore
2015-01-12 11:26:21 -08:00
Steve Yen
ae3600aeea
expose forestdb rollback methods
2015-01-06 18:59:02 -08:00
Steve Yen
5467e0a385
forestdb registered name fixed
2015-01-06 17:36:05 -08:00
Marty Schoch
38bdcbeb62
update to new forestdb iterator api
2014-12-27 13:15:14 -08:00
Silvan Jegen
ef18dfe4cd
Fix typos in comments and strings
2014-12-18 18:43:12 +01:00
Silvan Jegen
412049d63c
Remove unneeded import statements
2014-11-29 14:25:24 +01:00
Marty Schoch
6c7237ade9
added test for null kvstore
2014-11-26 15:50:57 -05:00
Marty Schoch
8ad0f64459
upgrade to current forestdb api
2014-11-25 21:52:35 -05:00
Marty Schoch
d5c1f4a9ab
refactored store tests
2014-11-25 21:52:23 -05:00