This repository was archived by the owner on Aug 13, 2019. It is now read-only.
postings compression exploration#629
Draft
naivewong wants to merge 18 commits into
Draft
Conversation
naivewong
commented
Jun 12, 2019
naivewong
commented
Contributor
- Store the first posting, then store the delta to that (not the delta to the previous element, the delta to the first number only). Find the minimum number of bits required to store the deltas. The bitpack using those many bits. Also be able to do binary search here.
- Use blocks of postings, like 4KB block, store deltas with the previous number. Binary search to the right block and then iterate.
Signed-off-by: naivewong <[email protected]>
Contributor
Author
|
|
||
| startOff := it.offset / (deltaBlockSize * 8) * deltaBlockSize | ||
| num := len(it.bs.bstream) / deltaBlockSize - it.offset / (deltaBlockSize * 8) | ||
| // Do binary search between current position and end. |
Contributor
There was a problem hiding this comment.
Try checking if the next block is too big before doing the search.
Contributor
Author
|
codesome
reviewed
Jun 13, 2019
Signed-off-by: naivewong <[email protected]>
naivewong
force-pushed
the
postings-compression
branch
2 times, most recently
from
June 17, 2019 16:52
ef22dcd to
11f2b1b
Compare
Signed-off-by: naivewong <[email protected]>
naivewong
force-pushed
the
postings-compression
branch
from
June 17, 2019 17:00
11f2b1b to
7cfcf3d
Compare
Signed-off-by: naivewong <[email protected]>
naivewong
force-pushed
the
postings-compression
branch
from
June 18, 2019 16:16
e11da7c to
99ce72a
Compare
Signed-off-by: naivewong <[email protected]>
Signed-off-by: naivewong <[email protected]>
Signed-off-by: naivewong <[email protected]>
Signed-off-by: naivewong <[email protected]>
Signed-off-by: naivewong <[email protected]>
Signed-off-by: naivewong <[email protected]>
naivewong
force-pushed
the
postings-compression
branch
from
June 28, 2019 07:46
d130757 to
b3f2b5e
Compare
Signed-off-by: naivewong <[email protected]>
Signed-off-by: naivewong <[email protected]>
naivewong
force-pushed
the
postings-compression
branch
2 times, most recently
from
July 4, 2019 14:49
c4b6f0f to
c7505ed
Compare
Signed-off-by: naivewong <[email protected]>
naivewong
force-pushed
the
postings-compression
branch
from
July 5, 2019 02:52
c7505ed to
95f3c79
Compare
…comments) Signed-off-by: naivewong <[email protected]>
Signed-off-by: naivewong <[email protected]>
Signed-off-by: naivewong <[email protected]>
Signed-off-by: naivewong <[email protected]>
Signed-off-by: naivewong <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.