Implement error-bounded lossy compression method MacaqueV - #341
Conversation
|
Hi. Although MacaqueTS (according to an ICDE 2026 paper) was already implemented earlier, please let me know if it should also be reflected in the comments. At the moment, only MacaqueV is reflected in this PR. |
There was a problem hiding this comment.
Pull Request Overview
This PR replaces the Gorilla compression model with MacaqueV, an error-bounded lossy compression method for single floating point values. MacaqueV extends Gorilla by adding value preprocessing methods and optimizing flag bits for better compression when used with PMC-Mean and Swing models.
- Replaces all references from Gorilla to MacaqueV across the codebase
- Implements MacaqueV with enhanced value rewriting capabilities using error bounds
- Modifies flag bit encoding by swapping Gorilla's flag bits 0 and 10 for better compression
Reviewed Changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| crates/modelardb_compression/src/types.rs | Updates imports and method calls to use MacaqueV instead of Gorilla |
| crates/modelardb_compression/src/models/mod.rs | Renames module and constants from gorilla to macaque_v, adds lossless compression helper |
| crates/modelardb_compression/src/models/macaque_v.rs | Complete implementation of MacaqueV with value rewriting and optimized flag bits |
| crates/modelardb_compression/src/compression.rs | Updates all references and test cases to use MacaqueV instead of Gorilla |
Comments suppressed due to low confidence (2)
crates/modelardb_compression/src/models/macaque_v.rs:90
- There is a grammatical error in the comment. 'Returns true is compression' should be 'Returns true if compression'.
}
crates/modelardb_compression/src/models/macaque_v.rs:329
- [nitpick] Function name 'rewrite_bits_by_n' is unclear. Consider renaming to 'clear_least_significant_bits' or 'mask_lower_bits' to better describe what the function does.
fn rewrite_bits_by_n(bits_to_rewrite: u32, left_shift_by: i32) -> u32 {
|
@aabduvakhobov thank you very much for the PR. I have asked @CGodiksen to review the code first as he can provide a fresh perspective. In the meantime, please correct the issues found by ast-grep and co-pilot. Be aware that co-pilot is not always correct, but occasionally identifies legitimate issues. Finally, when creating a PR that improves an aspect of ModelarDB instead of simply providing a feature, please include metrics to indicate how big an improvement the changes provide so the trade-off between added code complexity and the resulting improvements can be evaluated. |
skejserjensen
left a comment
There was a problem hiding this comment.
There are still a few comments that require changes to the code.
|
@aabduvakhobov the PR has now been merged. Thank you very much for your contribution. |
This PR implements MacaqueV, a lossy error-bounded single floating point compression method, as a replacement for Gorilla model type. Macaque adds two value preprocessing methods.
First, MacaqueV rewrites a value by 1) replacing it with the previous one within the error bound or 2) rewriting the least mantissa bits of the value within the error bound so that Gorilla uses fewer bits for encoding. Then the value is encoded using the modified version of Gorilla. Gorilla's flag bits 0 and 10 are swapped to achieve better compression when used alongside PMC-Mean and Swing.
Experiments using real-life datasets showed that MacaqueV improves ModelarDB's compression by 1.5-2.5x.