0
0
Fork 0
Commit Graph

8 Commits

Author SHA1 Message Date
Steve Yen 85b4a31e2a scorch zap getField() which panics if the field is unknown 2018-03-20 11:12:18 -07:00
Steve Yen 4af65a7846 scorch zap prealloc buf via estimate from previous interim work 2018-03-14 09:32:14 -07:00
Steve Yen 7578ff7cb8 scorch zap optimize interim's reuse of vellum builders
Since interim structs are now sync.Pool'ed, we can now also hold onto
and reuse the associated vellum builder.
2018-03-14 07:49:28 -07:00
Steve Yen dbfc5e9130 scorch zap reuse interim freq/norm/loc slices 2018-03-12 10:04:11 -07:00
Steve Yen 07901910e2 scorch zap reuse roaring Bitmap in prepareDicts() slice growth
In this change, if the postings/postingsLocs slices need to be grown,
then copy over and reuse any of the preallocated roaring Bitmap's from
the old slice.
2018-03-12 09:19:38 -07:00
Steve Yen c4ceffe584 scorch zap sync Pool for interim data 2018-03-12 09:17:37 -07:00
Steve Yen 531800c479 scorch zap use roaring Add() instead of AddInt()
This change invokes Add() directly as AddInt() is a convenience
wrapper around Add().
2018-03-12 09:17:37 -07:00
Steve Yen e82774ad20 scorch zap AnalysisResultsToSegmentBase()
AnalysisResultsToSegmentBase() allows analysis results to be directly
converted into a zap-encoded SegmentBase, which can then be introduced
onto the root, avoiding the creation of mem.Segment data structures.
This leads to some reduction of garbage memory allocations.

The grouping and sorting and shaping of the postings list information
is taken from the mem.Segment codepaths.

The encoding of stored fields reuses functions from zap's merger,
which has the largest savings of garbage memory avoidance.

And, the encoding of tf/loc chunks, postings & dictionary information
also follows the approach used by zap's merger, which also has some
savings of garbage memory avoidance.

In future changes, the mem.Segment dependencies will be removed from
zap, which should result in a smaller codebase.
2018-03-09 15:22:30 -08:00