0
0
Fork 0
Commit Graph

714 Commits

Author SHA1 Message Date
Marty Schoch 699c86073a make existing integration tests work with firestorm 2015-12-01 12:29:56 -05:00
Marty Schoch 9777846206 Merge branch 'master' into firestorm 2015-11-30 15:02:46 -05:00
Marty Schoch e472b3e807 add support for a "web" tokenizer/analyzer
The goal of the "web" tokenizer is to recognize web things like
- email addresses
- URLs
- twitter @handles and #hashtags

This implementation uses regexp exceptions.  There will most
likely be endless debate about the regular expressions. These
were chosein as "good enough for now".

There is also a "web" analyzer.  This is just the "standard"
analyzer, but using the "web" tokenizer instead of the "unicode"
one.  NOTE: after processing the exceptions, it still falls back
to the standard "unicode" one.

For many users, you can simply set your mapping's default analyzer
to be "web".

closes #269
2015-11-30 14:27:18 -05:00
Marty Schoch 6d851cfcc2 fix bug in warmup which led to docs being deleted 2015-11-30 10:18:14 -05:00
Marty Schoch aa8d98f5fa include space after prefix in log output 2015-11-30 10:17:48 -05:00
Marty Schoch 68d8742826 correctly prefix internal rows with 'i' and print them in debug 2015-11-30 10:17:15 -05:00
Marty Schoch 17cfe8cff0 Merge branch 'master' into firestorm 2015-11-30 07:25:33 -05:00
Marty Schoch b2ac05c6d0 support metrics through bleve query 2015-11-30 07:24:31 -05:00
Marty Schoch c93de9734e fix issues identified by errcheck 2015-11-24 14:32:33 -05:00
Marty Schoch bbef1980d8 Merge branch 'master' into firestorm 2015-11-24 13:04:36 -05:00
Marty Schoch 808f2c1e43 remove exceptions from errcheck 2015-11-24 12:52:46 -05:00
Marty Schoch ff11f83842 properly handle errors inside metrics kvstore reporting 2015-11-24 12:52:03 -05:00
Marty Schoch a707d44e0b Merge branch 'master' into firestorm 2015-11-24 09:44:47 -05:00
Marty Schoch 3729ac67ea remove bleve mapping from errcheck exclusion list 2015-11-23 12:40:50 -05:00
Steve Yen 2d4cd7a696 go fmt index_text.go 2015-11-23 09:28:09 -08:00
Steve Yen 19c43af6d4 typo in lookupPropertyPathPart() func name 2015-11-23 09:27:22 -08:00
Steve Yen ab3188becb http/mapping subdir replaced by blevesearch/bleve-mapping-ui
The bleve mapping UI is now moved / replaced, and is available at...

   github.com/blevesearch/bleve-mapping-ui
2015-11-23 09:23:52 -08:00
Marty Schoch fcf3980cea set GIT_BRANCH to TRAVIS_BRANCH 2015-11-23 11:10:55 -05:00
Marty Schoch b5af97bc60 try TRAVIS_BRANCH 2015-11-23 11:01:57 -05:00
Marty Schoch 12273c369e try to fix confusing output 2015-11-23 09:39:43 -05:00
Marty Schoch bd22b664c4 print branch name to output 2015-11-23 09:08:40 -05:00
Marty Schoch 89af187fb0 set branch to master to fix goveralls badge 2015-11-23 08:54:47 -05:00
Marty Schoch 16eecd50b4 Merge branch 'MachineShop-IOT-datetimefm' 2015-11-23 08:42:10 -05:00
Marty Schoch 97735ac2b6 set github issue number in testcase name 2015-11-23 08:41:34 -05:00
Mark Mindenhall 17d8391b2f Fixes datetime mapping from JSON, using DateTimeFieldMapping 2015-11-20 19:15:35 -07:00
Marty Schoch a1ce44fcd4 Merge pull request #284 from pmezard/add-indexmapping-fieldanalyzer
mapping_index: add IndexMapping.FieldAnalyzer()
2015-11-19 12:55:27 -05:00
Marty Schoch 46534d79ea Merge pull request #285 from pmezard/phrase-queries-need-term-vectors
mapping_field: document IncludeTermVectors
2015-11-19 12:53:38 -05:00
Patrick Mezard 1591ed1839 mapping_field: document IncludeTermVectors
And mention it is required for phrase and match phrase queries to
succeed.

Fix #280
2015-11-19 15:38:16 +01:00
Patrick Mezard ff41dfec3d mapping_index: add IndexMapping.FieldAnalyzer()
It returns the name of the analyzer used on a field, which can be passed
to IndexMapping.AnalyzeText().

Fix #282
2015-11-19 11:20:32 +01:00
Marty Schoch 7dd52a5463 Merge pull request #279 from pmezard/expose-row-attributes
row: expose TermFrequencyRow term and freq fields
2015-11-17 12:24:16 -05:00
Patrick Mezard e85c9c542e row: expose TermFrequencyRow term and freq fields
Rows content is an implementation detail of bleve index and may change
in the future. That said, they also contains information valuable to
assess the quality of the index or understand its performances. So, as
long as we agree that type asserting rows should only be done if you
know what you are doing and are ready to deal with future changes, I see
no reason to hide the row fields from external packages.

Fix #268
2015-11-17 17:21:26 +01:00
Marty Schoch 236cc156c5 Merge pull request #277 from marksamman/patch-1
Update NumericRangeQuery comments
2015-11-12 22:40:36 +00:00
Mark Samman b3a1e21c12 Update NumericRangeQuery comments 2015-11-12 22:16:10 +01:00
Marty Schoch 7b22332b06 Merge pull request #271 from pmezard/docid-query
query_docid: add DocIDQuery to filter by document identifiers
2015-11-12 15:52:36 +00:00
Marty Schoch bfbc2a6244 Merge pull request #273 from pmezard/document-token-map
token_map: document it along with stop_token_filter
2015-11-12 15:40:40 +00:00
Marty Schoch eee4482964 Merge pull request #272 from kevgs/boltdb_optimize
BoltDB wrapper nano optimization which makes code a bit prettier too
2015-11-12 15:39:37 +00:00
Patrick Mezard ff03874f19 token_map: document it along with stop_token_filter 2015-11-05 14:07:54 +01:00
Kosov Eugene 45e670b99b BoltDB wrapper nano optimization which makes code a bit prettier too 2015-11-05 00:27:28 +03:00
Patrick Mezard 19230b2f8a searcher_docid: catch DocIDReader.Close() possible error 2015-11-04 19:24:01 +01:00
Patrick Mezard ff7234d893 query_docid: add DocIDQuery to filter by document identifiers 2015-11-04 18:41:16 +01:00
Marty Schoch 74c309a7c2 Merge pull request #270 from pmezard/fix-elision-filter
Fix elision filter
2015-11-04 07:36:59 -05:00
Patrick Mezard eb26402924 elision_filter: correctly strip multi-bytes quotation marks 2015-11-04 10:59:10 +01:00
Patrick Mezard bae2079eb2 token_filters: fix typo in right single quotation mark name 2015-11-04 10:29:56 +01:00
Marty Schoch a873c76edc Merge pull request #265 from avsej/simplify-phrase-api
Simplify JSON API for phrase query
2015-11-02 15:42:41 -05:00
Sergey Avseyev 8609e7af08 Simplify JSON API for phrase query
New API looks like this:

    {"query":{"terms":["watered","down"],"field":"desc"}}

instead of

    {"query":{"terms":[{"term":"watered","field":"desc"},{"term":"down","field":"desc"}]}}

So that it eliminats accidental errors by supplying terms with different
fields, or different type of query
2015-11-02 21:08:11 +03:00
Marty Schoch 4791625b9b Merge pull request #262 from pmezard/index-and-tokenizer-doc-and-fix
Index and tokenizer doc and fix
2015-11-02 11:51:21 -05:00
Marty Schoch 30651065e9 fix panic on insufficiently sized buffer
adds test case to reproduce original problem
fixes #264
2015-10-30 18:25:38 -04:00
Patrick Mezard 0579d58263 mapping_test: fix TestMappingWithTokenizerDeps now patterns are required 2015-10-29 20:07:00 +01:00
Marty Schoch 2bd3ef4080 copy relevant k/v pairs before advancing underlying iterator 2015-10-28 12:23:54 -04:00
Marty Schoch d1b07f4909 fix dump methods to properly copy keys and values 2015-10-28 12:06:44 -04:00