When performing a MultiSearch, we create child SearchRequests
from the original SearchRequest. In doing so we copy many fields.
But, copying of the SortOrder was incorrect, as this contains
state, and distint SortOrder objects must be used. This change
introduces a Copy() method to the SearchSort interface, and
to the SortOrder types. MultiSearch now creates a new copy of
the SortOrder for each child request.
This is a change in search result behavior in that location
information is no longer provided by default with search results.
Although this looks like a wide-ranging change, it's mostly a
mechanical replacement of the explain bool flag with a new
search.SearcherOptions struct, which holds both the Explain bool flag
and the IncludeTermVectors bool flag.
MultiSearch previously had its own timeout checking. This commit
removes that timeout checking, so that now MultiSearch instead depends
upon the bleve.Index implementations to perform their own context
deadline/timeout checking.
Because deadline/timeout checking is now handled by the bleve.Index
implementations, this change allows applications to provide richer
error and status results during timeouts.
the index mapping contains some relatively messy logic
and the top-level bleve package only cares about a relatively
small portion of this
the motivation for this change is to codify the part that the
top-level bleve package cares about into an interface
then move all the details into its own package
NOTE: the top-level bleve package still has hard dependency on
the actual implementation (for now) because it must deserialize
mappings from JSON and simply assumes it is this one instance.
this is seen as OK for now, and this issue could be revisited
in a future change. moving the logic into a separate package
is seen as a simplification of top-level bleve, even though
we still depend on the one particular implementation.
The DumpXXX() methods were always documented as internal and
unsupported. However, now they are being removed from the
public top-level API. They are still available on the internal
IndexReader, which can be accessed using the Advanced() method.
The DocCount() and DumpXXX() methods on the internal index
have moved to the internal index reader, since they logically
operate on a snapshot of an index.
our implementation uses: golang.org/x/net/context
New method SearchInContext() allows the user to run a search
in the provided context. If that context is cancelled or
exceeds its deadline Bleve will attempt to stop and return
as soon as possible. This is a *best effort* attempt at this
time and may *not* be in a timely manner. If the caller must
return very near the timeout, the call should also be wrapped
in a goroutine.
The IndexAlias implementation is affected in a slightly more
complex way. In order to return partial results when a timeout
occurs on some indexes, the timeout is strictly enforced, and
at the moment this does introduce an additional goroutine.
The Bleve implementation honoring the context is currently
very course-grained. Specifically we check the Done() channel
between each DocumentMatch produced during the search. In the
future we will propogate the context deeper into the internals
of Bleve, and this will allow finer-grained timeout behavior.
the Status section can report on the number of total/fail/success
indexes when querying across multiple indexes through IndexAlias
Further, searching an IndexAlias will now return partial results,
the burden is on the caller to check the number of failed
indexes and decide how to handle this situation.
this introduces disk format v4
now the summary rows for a term are stored in their own
"dictionary row" format, previously the same information
was stored in special term frequency rows
this now allows us to easily iterate all the terms for a field
in sorted order (useful for many other fuzzy data structures)
at the top-level of bleve you can now browse terms within a field
using the following api on the Index interface:
FieldDict(field string) (index.FieldDict, error)
FieldDictRange(field string, startTerm []byte, endTerm []byte) (index.FieldDict, error)
FieldDictPrefix(field string, termPrefix []byte) (index.FieldDict, error)
fixes#127
now created through the index itself
mapping problems reported early at the time
data is added to the batch, previously these
were not reported until the batch was executed
now you can access the underlying index/store implementations
using the Advanced() method. this is intedned for advanced
usage only, and can lead to problems if misused.
also, there is a new method NewUsing(...) which allows callers
of the top-level API to choose which underlying k/v store they
want to use.
more things can return error now
in a couple of places we had to swallow errors because they didn't
fit the existing API. in these case and proactively in a few
others we now return error as well.
also the batch API has been updated to allow performing
set/delete internal within the batch
an IndexAlias allows you easily work with one logical Index
while changing the actual Index its pointing to behind the scenes
Changing which actual Index is backing an IndexAlias can be done
atomically so that your application smoothly transitions from
one Index to another.
A separate use of IndexAlias is allowed when the IndexAlias is
defined to point to multiple Indexes. In this case only the
Search() operation is supported, but the Search will be run
on each of the underlying indexes in parallel, and the results
will be merged.
Marty Schoch