dtynn
89dc2c22bc
update TermVector
2015-05-17 13:07:14 +08:00
Marty Schoch
8f70def63b
properly use the stored array positions when loading a document
...
fixes #205
2015-05-15 15:47:54 -04:00
Marty Schoch
d11f1336f0
fix issues identified by errcheck
2015-05-15 15:14:15 -04:00
Marty Schoch
328bc73ed0
clarify Batch is not threadsafe in docs
...
in some limited cases we can detect unsafe usage
in these cases, do not trip over ourselves and panic
instead return a strongly typed error upside_down.UnsafeBatchUseDetected
also, introduced Batch.Reset() to allow batch reuse
this is currently still experimental
closes #195
2015-05-15 15:04:52 -04:00
Marty Schoch
580d9013b2
fix registering tokenizers with dependencies
...
closes #201
2015-05-14 09:50:10 -04:00
Marty Schoch
7b871fde6a
add test comparing search that matches everyting with doc count
2015-05-09 14:51:07 -04:00
Marty Schoch
6f28f3e5bd
check error returned in test to make errcheck happy
2015-05-08 08:40:46 -04:00
Marty Schoch
57cd67fa88
fix data race on index metadata (docCount)
...
closes #198
2015-05-08 08:07:20 -04:00
Marty Schoch
57358088ec
fix row merging bug
...
trying to be clever, we reused the memory allocated for the left
operand when doing partial merges
this had been tested to be safe, in general. however, the
implementation was then written such that we always reused
globally defined operands, this meant that we mutated
the operands which were intended to always represent
+1/-1
this then cascades quickly to making increment/decrement
values much larger/smaller than they should be
related to #197
2015-05-06 11:00:04 -04:00
Marty Schoch
30a0ba1f9b
fix bug, dictionary row encoding buffer too small
...
we incorrectly created a []byte of length 8
but the max for a uvarint is 10
closes #197
2015-05-06 10:04:02 -04:00
Steve Yen
e98ae8ab71
update metrics store to latest kvstore api
2015-04-27 11:01:53 -07:00
Marty Schoch
16f538d7b7
close documents returned by iterator before losing their reference
...
fixes #194
2015-04-24 17:48:21 -04:00
Marty Schoch
b54a59139c
change forestdb imports to couchbase not couchbaselabs
2015-04-24 17:35:01 -04:00
Marty Schoch
ee47d1c21a
standardize on including 1000 sized batches
2015-04-24 17:31:34 -04:00
Marty Schoch
00e5412e73
moving goleveldb into main config as it has no build tags
2015-04-24 17:21:35 -04:00
Marty Schoch
452fea6a24
adding initial impl of rocksdb kv store
2015-04-24 17:19:44 -04:00
Marty Schoch
a9c07acbfa
refactor of kvstore api to support native merge in rocksdb
...
refactor to share code in emulated batch
refactor to share code in emulated merge
refactor index kvstore benchmarks to share more code
refactor index kvstore benchmarks to be more repeatable
2015-04-24 17:13:50 -04:00
Marty Schoch
4ef34e1b28
Merge pull request #191 from indraniel/goleveldb-correction
...
fix goleveldb's BytesSafeAfterClose() on reader
2015-04-10 16:57:32 -04:00
indraniel
a62320a50e
+ fix goleveldb's BytesSafeAfterClose() on reader
...
- it should be set to false
2015-04-10 15:45:22 -05:00
Marty Schoch
a2b7740db7
Merge branch 'indraniel-goleveldb'
2015-04-10 15:06:01 -04:00
Marty Schoch
d5dc66313f
change variable name conflicting when both LevelDB bencharmks run
2015-04-10 15:03:44 -04:00
Marty Schoch
d5caad4405
changed GoLevelDB benchmark names to be different from LevelDB
...
this will allow for easier comparision when running both
versions at the same time
2015-04-10 15:00:56 -04:00
Marty Schoch
5f66bd84c7
fix issues identified by errcheck
2015-04-10 14:59:05 -04:00
indraniel
54ab493b3e
+ correctly copy bytes from the goleveldb store
...
- this is part of a recent bleve KVStore API change.
See the following two google group threads for more details:
* [help adding goleveldb as an alternative Key/Value store for bleve][1]
* [bleve search performance improvement][2]
[1]: https://groups.google.com/forum/#!topic/bleve/aHZ8gmihLiY
[2]: https://groups.google.com/forum/#!topic/bleve/aTyqsSnbhik
2015-04-10 11:25:23 -05:00
indraniel
81bef38cce
Revert "+ make copies of the []bytes returned by goleveldb"
...
This reverts commit cb8c1741289a0f00b30733e0d52d9d81d1199603.
This commit is no longer desired. The KV store API has been changed to
better address this issue.
For more details, see the google group conversation thread at:
https://groups.google.com/forum/#!topic/bleve/aHZ8gmihLiY
2015-04-10 11:12:44 -05:00
indraniel
3a70401835
+ make copies of the []bytes returned by goleveldb
...
- The byte strings returned by goleveldb aren't necessarily safe. See
the following google group thread:
https://groups.google.com/forum/#!topic/bleve/aHZ8gmihLiY
This code change is based on the gist created here:
https://groups.google.com/forum/#!topic/bleve/aHZ8gmihLiY
2015-04-10 11:08:02 -05:00
indraniel
8e1c75a1cf
+ comment out the index function for goleveldb
...
- don't want to make goleveldb the default kv store yet.
2015-04-10 11:08:02 -05:00
indraniel
cb50368cdb
+ add a goleveldb config file
2015-04-10 11:08:02 -05:00
indraniel
a88d714778
+ add a goleveldb index updside-down benchmark test
2015-04-10 11:08:02 -05:00
indraniel
a0a2a61050
+ keep 'get' consistent with levigo implementation
...
- this change keeps the method behavior consistent with the
levigo/leveldb implementation.
- don't issue an err if a key isn't found
2015-04-10 11:08:02 -05:00
indraniel
5e55fa2866
+ keep 'getWithSnapshot' consistent with levigo implementation
...
- this change keeps the method behavior consistent with the
levigo/leveldb implementation.
- the leveldb store_test.go and goleveldb store_test.go are now
identical.
2015-04-10 11:08:02 -05:00
indraniel
caa19e6c36
+ initial stub of goleveldb package
...
- This is a first-pass introduction. Things may not be working
correctly yet.
2015-04-10 11:08:02 -05:00
Steve Yen
8ed7dc1505
added godoc badge to README
2015-04-09 08:53:29 -07:00
Steve Yen
3f2c90f7c1
moved all badges to the top of README
2015-04-09 08:48:12 -07:00
Steve Yen
86f2b844b4
added config_metrics.go which builds on debug tag
...
When the debug tag is used, the config_metrics.go file
ensures the metrics decorator KV store is imported.
2015-04-08 22:08:04 -07:00
Marty Schoch
0f16eccd6b
new tokenizer that allows you to pre-identify tokens with regexp
...
name "exception"
configure with list of regexp string "exceptions"
these exceptions regexps that match sequences you want treated
as a single token. these sequences are NOT sent to the
underlying tokenizer
configure "tokenizer" is the named tokenizer that should be
used for processing all text regions not matching exceptions
An example configuration with simple patterns to match URLs and
email addresses:
map[string]interface{}{
"type": "exception",
"tokenizer": "unicode",
"exceptions": []interface{}{
`[hH][tT][tT][pP][sS]?://(\S)*`,
`[fF][iI][lL][eE]://(\S)*`,
`[fF][tT][pP]://(\S)*`,
`\S+@\S+`,
}
}
2015-04-08 15:31:58 -04:00
Marty Schoch
056d74901e
fix test to actually work reliably
2015-04-08 11:17:34 -04:00
Marty Schoch
8581e73cef
added String method for Batch
...
also changed Batch methods to pointer receiver
closes #180
2015-04-08 10:41:42 -04:00
Marty Schoch
683c0a7a54
adding errcheck to travis builds
...
closes #169
2015-04-07 18:11:29 -04:00
Marty Schoch
539aeb8dc7
fix errors identified by errcheck
...
part of #169
2015-04-07 18:05:41 -04:00
Marty Schoch
ba6b3c8bb3
fix more issues identified by errcheck
...
part of #169
2015-04-07 16:45:23 -04:00
Marty Schoch
ab24772bf0
fix issues identified by errcheck
...
part of #169
2015-04-07 16:34:29 -04:00
Marty Schoch
56c4a09de1
fix issues identified by errcheck
...
part of #169
2015-04-07 15:39:56 -04:00
Marty Schoch
c8d974048a
fix issues identified by errcheck
...
part of #169
2015-04-07 14:59:35 -04:00
Marty Schoch
93e01a803e
fix issues identified by errcheck
...
part of #169
2015-04-07 14:52:00 -04:00
Marty Schoch
f1ec73e764
fix issues identified by errcheck
...
part of #169
2015-04-07 13:26:54 -04:00
Marty Schoch
56a30a3574
fix issues identified by errcheck
...
part of #169
2015-04-07 13:05:47 -04:00
Marty Schoch
d2e9409413
fix issues identified by errcheck
...
part of #169
2015-04-07 12:04:59 -04:00
Marty Schoch
24729541b5
fix issues identified by errcheck
...
also add bulkindex utility to gitignore
part of #169
2015-04-07 11:42:46 -04:00
Marty Schoch
35a4333bce
fix issues identified by errcheck
...
part of #169
2015-04-07 11:39:01 -04:00