Marty Schoch
01526e971f
Merge branch 'master' into firestorm
2015-10-28 11:26:01 -04:00
Marty Schoch
1a978a4591
fix go vet issues and cleanup reader/iterator
2015-10-26 16:41:58 -04:00
Marty Schoch
f0d282f5f8
add test case for seeing prefix iterators outside of range
...
similar to #256 except for prefix iterators
includes fix for boltdb and gtreap which had incorrect behavior
2015-10-26 16:14:29 -04:00
Patrick Mezard
5100e00f20
doc: DocIDReader.Advance() is no longer implementation dependent
2015-10-20 20:32:23 +02:00
Patrick Mezard
2fa334fc27
doc: talk about "documents" not "indexed or stored documents"
2015-10-20 20:24:24 +02:00
Patrick Mezard
b174c137fd
doc: document DocIDReader, and some Index bits
2015-10-20 20:24:24 +02:00
Patrick Mezard
da72d0c2b9
store_test: deduplicate store initialization
2015-10-20 19:21:01 +02:00
Patrick Mezard
873f483804
gtreap: RangeIterator.Seek should not move before start
2015-10-20 19:12:30 +02:00
Patrick Mezard
5d7628ba3b
boltdb: fix RangeIterator outside of range seeks
...
Two issues:
- Seeking before i.start and iterating returned keys before i.start
- Seeking after the store last key did not invalidate the iterator and
could cause infinite loops.
2015-10-20 19:09:51 +02:00
Patrick Mezard
aada2e7333
store_test: test RangeIterator.Seek on goleveldb
2015-10-20 19:09:38 +02:00
Marty Schoch
6cc21346dc
fix errcheck issues
2015-10-19 14:27:03 -04:00
Marty Schoch
817c317c90
Merge branch 'master' into newkvstore
2015-10-19 12:04:07 -04:00
Marty Schoch
faceecf87b
make row buffer size constant/configurable
...
also handle case where it is insufficiently sized
2015-10-19 12:03:38 -04:00
Marty Schoch
f0ee9a3c66
removed commented code and unused functions
2015-10-19 11:13:03 -04:00
Marty Schoch
c9471d5739
Merge pull request #244 from kevgs/master
...
reducing allocation count
2015-10-16 15:51:30 -04:00
Marty Schoch
e6d0fc8d95
Merge pull request #247 from pmezard/remove-update-goroutine
...
upside_down: no need for a goroutine to enqueue AnalysisWork
2015-10-16 10:15:55 -04:00
Marty Schoch
4c6bc23043
rewrite to keep using same buffer when possible
2015-10-13 14:04:56 -07:00
Marty Schoch
8de860bf12
2 more places that used old Key()
2015-10-13 12:35:08 -07:00
Marty Schoch
5f594d1acc
Merge branch 'master' into newkvstore
2015-10-12 18:07:04 -07:00
Marty Schoch
08572e4925
move literals outside loop for more predicatble test results
2015-10-12 18:06:38 -07:00
Patrick Mezard
8c928539ee
upside_down: no need for a goroutine to enqueue AnalysisWork
...
It boils down to:
1. client sends some work and a notification channel to a single worker,
then waits.
2. worker processes the work
3. worker sends the result to the client using the notification channel
I do not see any problem with this, even with unbuffered channels.
2015-10-12 10:42:14 +02:00
Marty Schoch
95e06538f3
fix benchmarks for the x kvstores
2015-10-09 11:09:42 -04:00
Marty Schoch
0f05d1d3ca
Merge branch 'master' into newkvstore
2015-10-09 10:33:41 -04:00
Patrick Mezard
aee82f8b49
upside_down: simplify return code in batchRows()
2015-10-09 09:57:12 +02:00
Marty Schoch
e28eb749d7
bump up buffer size
2015-10-06 16:45:38 -04:00
Marty Schoch
71cbb13e07
modify code to reuse buffer for kv generation
2015-10-05 17:49:50 -04:00
Kosov Eugene
a61c350888
reducing allocation count
2015-10-05 22:57:10 +03:00
Patrick Mezard
9d5407be13
boltdb: add "nosync" option to force boltdb.DB.NoSync=true
...
Use this option when rebuilding indexes from scratch. In my small case
(~17000 json documents), it reduces indexing from 520s to 250s.
I did not add any test, short of forced indexing termination it only
has performance effects, which are hard to test. And unknown options are
currently ignored.
Issue #240
2015-10-03 14:26:48 +02:00
Marty Schoch
d06b526cbf
more refactoring
2015-09-28 16:50:27 -04:00
Marty Schoch
66aa1b020a
Merge branch 'master' into firestorm
2015-09-23 11:32:25 -07:00
Marty Schoch
900f1b4a67
major kvstore interface and impl overhaul
...
clarified the interface contract
2015-09-23 11:25:47 -07:00
Marty Schoch
f81b2be334
major refactor of bleve configuration
...
see #221 for full details
2015-09-16 17:10:59 -04:00
Marty Schoch
c308f611cf
skip unnecessary map before slice
...
benchmark old ns/op new ns/op delta
BenchmarkBatch-4 16950972 16377194 -3.38%
benchmark old allocs new allocs delta
BenchmarkBatch-4 136164 136161 -0.00%
benchmark old bytes new bytes delta
BenchmarkBatch-4 7168872 7109691 -0.83%
2015-09-10 08:21:26 -04:00
Marty Schoch
f6f1628b15
avoid doing unnecessary work:
...
benchmark old ns/op new ns/op delta
BenchmarkBatch-4 20738739 17047158 -17.80%
benchmark old allocs new allocs delta
BenchmarkBatch-4 136423 136160 -0.19%
benchmark old bytes new bytes delta
BenchmarkBatch-4 20277781 7168772 -64.65%
2015-09-10 08:19:05 -04:00
Marty Schoch
c8538c835f
Merge branch 'master' into firestorm
2015-09-10 08:14:14 -04:00
Marty Schoch
17c64d37c7
add similar benchmarks from firestorm
2015-09-10 08:13:52 -04:00
Marty Schoch
1e4d637761
adding more benchmarks
2015-09-10 08:01:11 -04:00
Marty Schoch
f74ed6a9ae
Merge remote-tracking branch 'origin' into firestorm
...
cathching up with changes from master
2015-09-02 13:29:03 -04:00
Marty Schoch
dbb93b75a4
refactoring to allow pluggable index encodings
...
this lays the foundation for supporting the new firestorm
indexing scheme. i'm merging these changes ahead of
the rest of the firestorm branch so i can continue
to make changes to the analysis pipeline in parallel
2015-09-02 13:12:08 -04:00
Marty Schoch
7ad7659ce5
add support for using null kvstore outside of bleve internals
2015-09-02 11:50:06 -04:00
Marty Schoch
07d37ca38a
add important rocksdb config options
2015-09-02 11:49:42 -04:00
Marty Schoch
18151862b5
fix go vet issues
2015-08-25 15:13:13 -04:00
Marty Schoch
84811cf5a0
made index type configurable + first version of firestorm
2015-08-25 14:52:42 -04:00
Marty Schoch
3e60ca24ec
support using end key on forestdb iterator for term freq lookup
...
also additoanl forestdb configs
2015-08-18 16:22:02 -04:00
Marty Schoch
ae19d77b04
updated protobuf defs to be valid
2015-08-17 15:37:13 -04:00
Marty Schoch
1187436e46
changed Stored row Values to also use protobuf
2015-08-17 09:48:40 -04:00
Marty Schoch
8d8a05a842
fix more issues
2015-08-14 16:27:00 -04:00
Marty Schoch
e0802a2b39
fixed the worst of the formatting
2015-08-14 16:17:48 -04:00
Marty Schoch
f4df56eb7c
add first draft of firestorm proposal
2015-08-14 16:09:19 -04:00
Marty Schoch
d3dda3d0ea
fixup config parsing and add new options
2015-08-12 13:18:23 -04:00
