0
0
Commit Graph

796 Commits

Author SHA1 Message Date
Marty Schoch
ad0d580587 add option to bleve_create to select storage 2015-07-13 15:19:56 -04:00
Marty Schoch
17ef48f82a switching back to the canonical goleveldb repo 2015-07-08 12:21:17 -06:00
Nimish Gupta
77779b70f4 Allow query string handling which contains only not must
fixes #193
2015-07-08 16:36:14 +05:30
Marty Schoch
bf80f4628e fix bug in curent goleveldb (must copy during iteration)
also changed over to mschoch fork of goleveldb (temporary)

the change to my fork is pending some read-only issues described
here:  https://github.com/syndtr/goleveldb/issues/111

hopefully we can find a path forward, and get that addressed upstream
2015-07-06 18:00:05 -04:00
Marty Schoch
7f0961424d updated tests for <mark></mark> 2015-07-06 18:00:05 -04:00
Marty Schoch
e2223f5121 changed HTML highlighter to use html mark tag 2015-07-06 18:00:05 -04:00
Marty Schoch
65556f45c7 added additional tests for bug #214 2015-07-06 18:00:05 -04:00
Marty Schoch
7be7ecdf8e fix batch indexing bug, incremented docCount before commit
fixes #211
2015-06-08 14:14:05 -04:00
Marty Schoch
2768c2da3c fix previous sloppy fix which hadn't been adequately tested 2015-05-27 19:15:55 -07:00
Marty Schoch
201fb91171 fix up to correctly trim off separator
even though it should never be present
2015-05-27 19:10:12 -07:00
Marty Schoch
a58592ceff fix case where NewBackIndexRowKV returns nil, nil
the logic for reading the docID from the keys
in this row relies on the keys NEVER containing
the byte separator character (0xff), this is OK
as we require that all keys be valid utf-8
however, it turns out that in the case where this
rule was violated, we would panic, because we
return nil, nil and later try to print the doc id
2015-05-27 19:04:57 -07:00
dtynn
59c97ae577 use binary.MaxVarintLen64 2015-05-26 15:35:31 +08:00
Marty Schoch
2af47cea75 fix query string query syntax when term starts with a number
fixes #207
2015-05-21 15:43:13 -04:00
Marty Schoch
165e63ac09 trying to work around errcheck 2015-05-21 15:20:57 -04:00
Marty Schoch
e0887f9113 fix tests which deadlock boltdb due to deferred cleanup
fixes #209
2015-05-21 12:29:31 -04:00
Marty Schoch
a52d3b5c07 put in hack to allow boltdb reader isolation test to pass
in boltdb, long readers *MAY* block a writer.  in particular if
the write requires additional allocation, it must acquire a lock
already held by the reader.  in general this is not a problem
for bleve (though it can affect performance in some cases), but
it is a problem for the reader isolation test.  this commit
adds a hack to try and avoid the need for additional allocation
closes #208
2015-05-21 11:39:59 -04:00
Marty Schoch
5b8c9f2d73 added unit test for bug #207 2015-05-21 07:49:41 -04:00
Steve Yen
9a09689e61 moved bleve-explorer mapping UI to bleve/http/mapping
All this work comes from Marty Schoch's bleve-explorer UI, but was
moved here for increased reusability.
2015-05-18 11:53:25 -07:00
dtynn
b4f7496031 update the index format version number 2015-05-18 15:16:35 +08:00
dtynn
89dc2c22bc update TermVector 2015-05-17 13:07:14 +08:00
Marty Schoch
8f70def63b properly use the stored array positions when loading a document
fixes #205
2015-05-15 15:47:54 -04:00
Marty Schoch
d11f1336f0 fix issues identified by errcheck 2015-05-15 15:14:15 -04:00
Marty Schoch
328bc73ed0 clarify Batch is not threadsafe in docs
in some limited cases we can detect unsafe usage
in these cases, do not trip over ourselves and panic
instead return a strongly typed error upside_down.UnsafeBatchUseDetected
also, introduced Batch.Reset() to allow batch reuse
this is currently still experimental
closes #195
2015-05-15 15:04:52 -04:00
Marty Schoch
580d9013b2 fix registering tokenizers with dependencies
closes #201
2015-05-14 09:50:10 -04:00
Marty Schoch
7b871fde6a add test comparing search that matches everyting with doc count 2015-05-09 14:51:07 -04:00
Marty Schoch
6f28f3e5bd check error returned in test to make errcheck happy 2015-05-08 08:40:46 -04:00
Marty Schoch
57cd67fa88 fix data race on index metadata (docCount)
closes #198
2015-05-08 08:07:20 -04:00
Marty Schoch
57358088ec fix row merging bug
trying to be clever, we reused the memory allocated for the left
operand when doing partial merges
this had been tested to be safe, in general.  however, the
implementation was then written such that we always reused
globally defined operands, this meant that we mutated
the operands which were intended to always represent
+1/-1
this then cascades quickly to making increment/decrement
values much larger/smaller than they should be
related to #197
2015-05-06 11:00:04 -04:00
Marty Schoch
30a0ba1f9b fix bug, dictionary row encoding buffer too small
we incorrectly created a []byte of length 8
but the max for a uvarint is 10
closes #197
2015-05-06 10:04:02 -04:00
Steve Yen
e98ae8ab71 update metrics store to latest kvstore api 2015-04-27 11:01:53 -07:00
Marty Schoch
16f538d7b7 close documents returned by iterator before losing their reference
fixes #194
2015-04-24 17:48:21 -04:00
Marty Schoch
b54a59139c change forestdb imports to couchbase not couchbaselabs 2015-04-24 17:35:01 -04:00
Marty Schoch
ee47d1c21a standardize on including 1000 sized batches 2015-04-24 17:31:34 -04:00
Marty Schoch
00e5412e73 moving goleveldb into main config as it has no build tags 2015-04-24 17:21:35 -04:00
Marty Schoch
452fea6a24 adding initial impl of rocksdb kv store 2015-04-24 17:19:44 -04:00
Marty Schoch
a9c07acbfa refactor of kvstore api to support native merge in rocksdb
refactor to share code in emulated batch
refactor to share code in emulated merge
refactor index kvstore benchmarks to share more code
refactor index kvstore benchmarks to be more repeatable
2015-04-24 17:13:50 -04:00
Marty Schoch
4ef34e1b28 Merge pull request #191 from indraniel/goleveldb-correction
fix goleveldb's BytesSafeAfterClose() on reader
2015-04-10 16:57:32 -04:00
indraniel
a62320a50e + fix goleveldb's BytesSafeAfterClose() on reader
- it should be set to false
2015-04-10 15:45:22 -05:00
Marty Schoch
a2b7740db7 Merge branch 'indraniel-goleveldb' 2015-04-10 15:06:01 -04:00
Marty Schoch
d5dc66313f change variable name conflicting when both LevelDB bencharmks run 2015-04-10 15:03:44 -04:00
Marty Schoch
d5caad4405 changed GoLevelDB benchmark names to be different from LevelDB
this will allow for easier comparision when running both
versions at the same time
2015-04-10 15:00:56 -04:00
Marty Schoch
5f66bd84c7 fix issues identified by errcheck 2015-04-10 14:59:05 -04:00
indraniel
54ab493b3e + correctly copy bytes from the goleveldb store
- this is part of a recent bleve KVStore API change.

    See the following two google group threads for more details:

    * [help adding goleveldb as an alternative Key/Value store for bleve][1]
    * [bleve search performance improvement][2]

    [1]: https://groups.google.com/forum/#!topic/bleve/aHZ8gmihLiY
    [2]: https://groups.google.com/forum/#!topic/bleve/aTyqsSnbhik
2015-04-10 11:25:23 -05:00
indraniel
81bef38cce Revert "+ make copies of the []bytes returned by goleveldb"
This reverts commit cb8c1741289a0f00b30733e0d52d9d81d1199603.

This commit is no longer desired. The KV store API has been changed to
better address this issue.

For more details, see the google group conversation thread at:

https://groups.google.com/forum/#!topic/bleve/aHZ8gmihLiY
2015-04-10 11:12:44 -05:00
indraniel
3a70401835 + make copies of the []bytes returned by goleveldb
- The byte strings returned by goleveldb aren't necessarily safe.  See
    the following google group thread:

    https://groups.google.com/forum/#!topic/bleve/aHZ8gmihLiY

    This code change is based on the gist created here:

    https://groups.google.com/forum/#!topic/bleve/aHZ8gmihLiY
2015-04-10 11:08:02 -05:00
indraniel
8e1c75a1cf + comment out the index function for goleveldb
- don't want to make goleveldb the default kv store yet.
2015-04-10 11:08:02 -05:00
indraniel
cb50368cdb + add a goleveldb config file 2015-04-10 11:08:02 -05:00
indraniel
a88d714778 + add a goleveldb index updside-down benchmark test 2015-04-10 11:08:02 -05:00
indraniel
a0a2a61050 + keep 'get' consistent with levigo implementation
- this change keeps the method behavior consistent with the
     levigo/leveldb implementation.

   - don't issue an err if a key isn't found
2015-04-10 11:08:02 -05:00
indraniel
5e55fa2866 + keep 'getWithSnapshot' consistent with levigo implementation
- this change keeps the method behavior consistent with the
     levigo/leveldb implementation.

   - the leveldb store_test.go and goleveldb store_test.go are now
     identical.
2015-04-10 11:08:02 -05:00