0
0
Fork 0
Commit Graph

38 Commits

Author SHA1 Message Date
abhinavdangeti 7e36109b3c MB-28162: Provide API to estimate memory needed to run a search query
This API (unexported) will estimate the amount of memory needed to execute
a search query over an index before the collector begins data collection.

Sample estimates for certain queries:
{Size: 10, BenchmarkUpsidedownSearchOverhead}
                                                           ESTIMATE    BENCHMEM
TermQuery                                                  4616        4796
MatchQuery                                                 5210        5405
DisjunctionQuery (Match queries)                           7700        8447
DisjunctionQuery (Term queries)                            6514        6591
ConjunctionQuery (Match queries)                           7524        8175
Nested disjunction query (disjunction of disjunctions)     10306       10708
…
2018-03-06 13:53:42 -08:00
Sreekanth Sivasankaran 4c256f5669 DocValue Config, new API Changes
-VisitableDocValueFields API for persisted DV field list
-making dv configs overridable at field level
-enabling on the fly/runtime un inverting of doc values
-few UT updates
2018-01-08 10:58:33 +05:30
Steve Yen 87115cbfb7 remove unused Document.Number property 2017-08-24 16:21:26 -07:00
Marty Schoch a16efa5e78 add experimental support for indexing/query geo points
New field type GeoPointField, or "geopoint" in mapping JSON.

Currently structs and maps are considered when a mapping explicitly
marks a field as type "geopoint".  Several variants of "lon", "lng", and "lat"
are looked for in map keys, struct field names, or method names.

New query type GeoBoundingBoxQuery searches for documents which have a
GeoPointField indexed with a value that is inside the specified bounding box.

New query type GeoDistanceQuery searches for documents which have a
GeoPointField indexed with a value that is less than or equal to the
specified distance from the specified location.

New sort by method "geo_distance".  Hits can be sorted by their distance
from the specified location.

New geo utility package with all routines ported from Lucene.

New FilteringSearcher, which wraps an existing Searcher, but filters
all hits with a user-provided callback.
2017-03-24 17:22:21 -07:00
Marty Schoch 2332455bd2 nicer formatting of license header 2016-10-02 10:13:14 -04:00
Marty Schoch c487f29a46 BREAKING CHANGE - rename numeric_util to numeric 2016-09-30 12:36:43 -04:00
Marty Schoch 750e0ac16c change sort field impl to use indexed values not stored values 2016-08-17 09:20:44 -07:00
Marty Schoch 0bb69a9a1c Merge branch 'master' of https://github.com/dtylman/bleve into sort-by-field-try2 2016-08-12 14:23:55 -04:00
Marty Schoch 23a323bc9d add support for numPlainTextBytesIndexed metric 2016-03-05 14:05:08 -05:00
slavikm 680be52f87 Implemented boolean field support 2016-01-11 17:18:03 -08:00
Steve Yen 89d17f01ef analyze locations only if includeTermVectors enabled
With this change, TermLocations are computed and maintained only if
includeTermVectors is enabled, for higher performance.
2016-01-05 12:46:46 -08:00
Marty Schoch 01526e971f Merge branch 'master' into firestorm 2015-10-28 11:26:01 -04:00
Patrick Mezard e2fa3d6351 doc: document Token, TokenFrequencies and Field structs
It helps understanding what is going on in indexing code.
ArrayPositions() was particularly puzzling.
2015-10-09 12:32:44 +02:00
Marty Schoch 37aa5cb027 Merge branch 'master' into firestorm 2015-09-09 09:03:42 -04:00
Marty Schoch d00bc91dc9 minor speed up in token frequency calculations
benchmark               old ns/op     new ns/op     delta
BenchmarkAnalysis-4     1599218       1540991       -3.64%

benchmark               old allocs     new allocs     delta
BenchmarkAnalysis-4     5353           5318           -0.65%

benchmark               old bytes     new bytes     delta
BenchmarkAnalysis-4     370495        362983        -2.03%
2015-09-04 18:57:39 -04:00
Marty Schoch 84811cf5a0 made index type configurable + first version of firestorm 2015-08-25 14:52:42 -04:00
Donald Huang 587bed442d fix invalid fmt string using analysis.Analyzer 2015-08-03 17:53:10 +00:00
Marty Schoch f35e2e42df fix highlighting to work on fields containing arrays
fixes #170
2015-07-31 14:43:12 -04:00
Marty Schoch 3682c25467 update to correctly work with composite fields
also updated search results to return array positions
2015-07-31 11:16:11 -04:00
Marty Schoch 7e3ba85b9d added test and fixed behavior to ensure correct value is stored
optimization introduced last week inadvertently meant we were
not preserving the original byte values of text fields that
were stored
2015-01-19 15:40:18 -05:00
Marty Schoch 0ddfa774ec clean up logging to use package level *log.Logger
by default messages go to ioutil.Discard
2014-12-28 12:14:48 -08:00
Marty Schoch 453d4cf770 change to always return stored fields in UTC 2014-11-26 15:36:34 -05:00
Marty Schoch 8e6c8e5644 continued refactoring of the mapping code
also renamed some constant that didnt follow go convetions
2014-09-03 13:02:10 -04:00
Marty Schoch 7a7eb2e94c add newline between license and package
this avoids cluttering godocs with the license
2014-09-02 10:54:50 -04:00
Marty Schoch 1161361bea rename imports from couchbaselabs to blevesearch 2014-08-28 15:38:57 -04:00
Marty Schoch 41914181ae fix handling of indexing dates which we cannot represent
closes #72
2014-08-19 09:55:26 -04:00
Marty Schoch 082a5b0b03 major change to fields
now can track array positions for field values
stored fields now include this in the key
and the back index now uses protobufs to simplify serialization
closes #73
2014-08-19 08:58:26 -04:00
Marty Schoch 41d4f67ee2 fix storing/retrieving numeric and date fields
also includes new ability to request stored fields be returned with results

closes #55 and closes #56 and closes #58
2014-08-06 13:52:20 -04:00
Marty Schoch 00d6f9700b added support for date range fields and queries
closes #9 and closes #11
2014-08-03 17:19:04 -04:00
Marty Schoch 78465ca686 added initial support for indexing and querying numeric values
closes #8 and closes #10
2014-08-02 19:05:58 -04:00
Marty Schoch dd36f916c4 set token type 2014-07-31 14:10:27 -04:00
Marty Schoch 2968d3538a major refactor, apologies for the large commit
removed analyzers (these are now built as needed through config)
removed html chacter filter (now built as needed through config)
added missing license header
changed constructor signature of filters that cannot return errors
filter constructors that can have errors, now have Must variant which panics
change cdl2 tokenizer into filter (should only see lower-case input)
new top level index api, closes #5
refactored index tests to not rely directly on analyzers
moved query objects to top-level
new top level search api, closes #12
top score collector allows skipping results
index mapping supports _all by default, closes #3 and closes #6
index mapping supports disabled sections, closes #7
new http sub package with reusable http.Handler's, closes #22
2014-07-30 12:30:38 -04:00
Marty Schoch 70a8b03bed added support for composite fields 2014-07-21 17:05:55 -04:00
Marty Schoch d3466f3919 refactored field from struct to interface 2014-07-14 14:47:05 -04:00
Marty Schoch a7107e27f4 added support for document mapping 2014-07-11 14:26:56 -04:00
Marty Schoch 9bebbec267 added support for stored fields and highlighting results 2014-06-26 11:43:13 -04:00
Marty Schoch aa53d0b340 added conjunction term search/query/score 2014-04-24 10:24:19 -06:00
Marty Schoch 3d842dfaf2 initial commit 2014-04-17 16:55:53 -04:00