0
0
bleve/search
Marty Schoch a4a34cc3b2 topn collector switch approach based on size+skip
we now use the slice store when size+skip <= 10
and use the heap store when size+skip > 10

here are the new perf numbers:

go test -run=xxx -bench=. -benchmem
BenchmarkTop10of0Scores-4            	 1000000	      1150 ns/op	    2304 B/op	      15 allocs/op
BenchmarkTop10of3Scores-4            	 1000000	      1417 ns/op	    2304 B/op	      18 allocs/op
BenchmarkTop10of10Scores-4           	 1000000	      2133 ns/op	    2312 B/op	      25 allocs/op
BenchmarkTop10of25Scores-4           	  500000	      3410 ns/op	    2464 B/op	      26 allocs/op
BenchmarkTop10of50Scores-4           	  300000	      5174 ns/op	    2464 B/op	      26 allocs/op
BenchmarkTop10of10000Scores-4        	    5000	    342955 ns/op	    2488 B/op	      26 allocs/op
BenchmarkTop100of0Scores-4           	  300000	      4796 ns/op	   18320 B/op	      15 allocs/op
BenchmarkTop100of3Scores-4           	  300000	      5160 ns/op	   18352 B/op	      19 allocs/op
BenchmarkTop100of10Scores-4          	  200000	      6354 ns/op	   18408 B/op	      26 allocs/op
BenchmarkTop100of25Scores-4          	  200000	     10023 ns/op	   18568 B/op	      41 allocs/op
BenchmarkTop100of50Scores-4          	  100000	     16821 ns/op	   18832 B/op	      66 allocs/op
BenchmarkTop100of10000Scores-4       	    3000	    508989 ns/op	   19760 B/op	     117 allocs/op
BenchmarkTop1000of10000Scores-4      	    1000	   1814198 ns/op	  184768 B/op	    1017 allocs/op
BenchmarkTop10000of100000Scores-4    	      50	  26623920 ns/op	 1939592 B/op	   19024 allocs/op
BenchmarkTop10of100000Scores-4       	     500	   3730204 ns/op	    2496 B/op	      26 allocs/op
BenchmarkTop100of100000Scores-4      	     300	   4057127 ns/op	   19912 B/op	     117 allocs/op
BenchmarkTop1000of100000Scores-4     	     200	   6390180 ns/op	  186200 B/op	    1017 allocs/op
BenchmarkTop10000of1000000Scores-4   	      20	  82785756 ns/op	 1963897 B/op	   19024 allocs/op
PASS
ok  	github.com/blevesearch/bleve/search/collector	31.537s

Previously with heap:

go test -run=xxx -bench=. -benchmem
BenchmarkTop10of0Scores-4            	 1000000	      1216 ns/op	    2288 B/op	      15 allocs/op
BenchmarkTop10of3Scores-4            	 1000000	      1593 ns/op	    2320 B/op	      19 allocs/op
BenchmarkTop10of10Scores-4           	  500000	      2734 ns/op	    2376 B/op	      26 allocs/op
BenchmarkTop10of25Scores-4           	  300000	      5077 ns/op	    2520 B/op	      27 allocs/op
BenchmarkTop10of50Scores-4           	  200000	      6875 ns/op	    2528 B/op	      27 allocs/op
BenchmarkTop10of10000Scores-4        	    3000	    351210 ns/op	    2552 B/op	      27 allocs/op
BenchmarkTop100of0Scores-4           	  300000	      4846 ns/op	   18304 B/op	      15 allocs/op
BenchmarkTop100of3Scores-4           	  300000	      5357 ns/op	   18336 B/op	      19 allocs/op
BenchmarkTop100of10Scores-4          	  200000	      6462 ns/op	   18392 B/op	      26 allocs/op
BenchmarkTop100of25Scores-4          	  200000	     10012 ns/op	   18552 B/op	      41 allocs/op
BenchmarkTop100of50Scores-4          	  100000	     17089 ns/op	   18816 B/op	      66 allocs/op
BenchmarkTop100of10000Scores-4       	    3000	    528193 ns/op	   19744 B/op	     117 allocs/op
BenchmarkTop1000of10000Scores-4      	    1000	   1859447 ns/op	  184752 B/op	    1017 allocs/op
BenchmarkTop10000of100000Scores-4    	      50	  28005664 ns/op	 1939576 B/op	   19024 allocs/op
BenchmarkTop10of100000Scores-4       	     300	   4120091 ns/op	    2560 B/op	      27 allocs/op
BenchmarkTop100of100000Scores-4      	     300	   4325227 ns/op	   19896 B/op	     117 allocs/op
BenchmarkTop1000of100000Scores-4     	     200	   6799804 ns/op	  186184 B/op	    1017 allocs/op
BenchmarkTop10000of1000000Scores-4   	      20	  88494230 ns/op	 1963881 B/op	   19024 allocs/op
PASS
ok  	github.com/blevesearch/bleve/search/collector	30.198s

Previously with slice:

go test -run=xxx -bench=. -benchmem
BenchmarkTop10of0Scores-4            	 1000000	      1202 ns/op	    2288 B/op	      15 allocs/op
BenchmarkTop10of3Scores-4            	 1000000	      1453 ns/op	    2288 B/op	      18 allocs/op
BenchmarkTop10of10Scores-4           	 1000000	      2162 ns/op	    2296 B/op	      25 allocs/op
BenchmarkTop10of25Scores-4           	  500000	      3420 ns/op	    2448 B/op	      26 allocs/op
BenchmarkTop10of50Scores-4           	  300000	      5336 ns/op	    2448 B/op	      26 allocs/op
BenchmarkTop10of10000Scores-4        	    5000	    356733 ns/op	    2472 B/op	      26 allocs/op
BenchmarkTop100of0Scores-4           	  300000	      4877 ns/op	   18304 B/op	      15 allocs/op
BenchmarkTop100of3Scores-4           	  300000	      5132 ns/op	   18304 B/op	      18 allocs/op
BenchmarkTop100of10Scores-4          	  200000	      5787 ns/op	   18312 B/op	      25 allocs/op
BenchmarkTop100of25Scores-4          	  200000	      8083 ns/op	   18344 B/op	      40 allocs/op
BenchmarkTop100of50Scores-4          	  100000	     14419 ns/op	   18400 B/op	      65 allocs/op
BenchmarkTop100of10000Scores-4       	    2000	    665401 ns/op	   18848 B/op	     116 allocs/op
BenchmarkTop1000of10000Scores-4      	     100	  15417063 ns/op	  176560 B/op	    1016 allocs/op
BenchmarkTop10000of100000Scores-4    	       1	1860011022 ns/op	 1857960 B/op	   19023 allocs/op
BenchmarkTop10of100000Scores-4       	     300	   4099276 ns/op	    2480 B/op	      26 allocs/op
BenchmarkTop100of100000Scores-4      	     300	   4533645 ns/op	   18984 B/op	     116 allocs/op
BenchmarkTop1000of100000Scores-4     	      50	  30519235 ns/op	  178008 B/op	    1016 allocs/op
BenchmarkTop10000of1000000Scores-4   	       1	3483977385 ns/op	 1882072 B/op	   19023 allocs/op
PASS
ok  	github.com/blevesearch/bleve/search/collector	31.666s

It appears that this sucessfully gets the best of both, in these particular benchmark sizes.
2017-04-27 08:57:13 -04:00
..
collector topn collector switch approach based on size+skip 2017-04-27 08:57:13 -04:00
facet reduce garbage created while processing facets 2017-03-02 17:00:46 -05:00
highlight remove use of float64 to represent int things 2017-02-09 20:15:59 -05:00
query introduce new query TermRange 2017-03-31 22:04:00 -04:00
scorer refactor searchers 2017-03-31 17:21:46 -04:00
searcher add option for multi term searcher to skip max disjunction check 2017-04-04 10:46:57 -04:00
collector.go nicer formatting of license header 2016-10-02 10:13:14 -04:00
explanation.go nicer formatting of license header 2016-10-02 10:13:14 -04:00
facets_builder_test.go numeric range facet merging compare range values not pointers 2016-11-03 15:48:46 -04:00
facets_builder.go reduce garbage created while processing facets 2017-03-02 17:00:46 -05:00
levenshtein_test.go removed extra ptr indirection from LevenshteinDistance 2016-10-11 08:49:10 -07:00
levenshtein.go removed extra ptr indirection from LevenshteinDistance 2016-10-11 08:49:10 -07:00
pool_test.go fix misspellings 2016-10-02 12:11:15 -04:00
pool.go DocumentMatchPool hits allocator outside of loop 2017-02-06 14:26:59 -08:00
search.go reduce garbage created while processing facets 2017-03-02 17:00:46 -05:00
sort.go fix race condition in incorrectly shared state in MultiSearch 2017-04-06 17:49:33 -04:00
util_test.go nicer formatting of license header 2016-10-02 10:13:14 -04:00
util.go nicer formatting of license header 2016-10-02 10:13:14 -04:00