Add v6 .rm format support with multi-color highlights#59
Merged
Conversation
Parse the v6 tagged-block/CRDT scene tree format used by reMarkable firmware 3.0+, enabling `geta` to export annotations from modern documents. V6 PDF-backed pages use DPI-based coordinate scaling (226→72 DPI) instead of the device-pixel mapping used by v3/v5. Stroke line widths are derived from per-point width data rather than the v3/v5 brush-size formula. Reads the firmware 3.0+ `cPages` content layout with CRDT ordering and PDF page redirection, plus a last-resort pageMap fallback discovered from the .rm filenames in the zip. Co-Authored-By: Claude Opus 4.6 <[email protected]>
Highlights on the reMarkable can be yellow, green, pink (and arbitrary RGB on firmware 3.6+ via the optional color_rgba field), but `geta` exports flattened every highlight to yellow because the renderer ignored the per-highlight Color and the parser dropped the optional RGBA tag. - Extend BrushColor with the missing PenColor codes (Yellow=3, Green=4, Pink=5, GreyOverlap=8, HighlightDynamic=9, Green2, Cyan, Magenta, Yellow2) to match rmscene's enum (https://github.com/ricklupton/rmscene). - Read the optional tag-10 BGRA-packed RGBA on glyph blocks into a new Highlight.ColorRGBA field; firmware 3.6+ uses Color=HighlightDynamic with the actual color stored there. - Resolve highlight colors at render time via a small helper that maps the enum to RGB and prefers ColorRGBA when present. Apply the same helper to v5 highlighter strokes so their BrushColor is honored too. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Three new test groups guard against regressions introduced by the v6 parser and color-aware highlight rendering: - TestV3V5NoHighlights: parses the existing test_v3.rm / test_v5.rm fixtures and asserts the new Rm.Highlights field stays empty (a v6-only concept) and layers are still populated. - TestBrushColorPenColorEnum: locks down each BrushColor constant to rmscene's PenColor integer value so the wire format keeps decoding correctly even if constants are reordered later. - TestHighlightRGB*: covers the renderer's color resolution path — explicit color_rgba override (firmware 3.6+), per-enum mapping for every named highlight color, fallback to yellow for unknown codes, and the legacy v3/v5 Black-default brush color landing on a visible grey rather than disappearing. All seven existing TestGenerate* end-to-end PDF render tests (a3/a4/a5/ letter/rm/tmpl/strange) continue to pass, validating that v3/v5 documents still produce the same rendered output after the v6 changes to normalized(), the BrushColor switch, and the highlight loop. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two commits:
getacan export annotations from modern reMarkable documents. Also reads thecPagescontent layout (CRDT ordering + PDF page redirection) and falls back to deriving the page map from.rmfilenames when no other source is available.color_rgbacarried by firmware 3.6+ whencolor == 9), and the renderer maps it to RGB instead of flattening every highlight to yellow.Why v6
The current parser handles v3/v5 only. As of firmware 3.0+ the device writes
.linesfiles in the v6 tagged-block format. Without v6 parsing,getasilently produces PDFs with no annotations from any document edited on a modern device.V6 differs from v3/v5 in several places:
x = page width / 2; positions are in device pixels at 226 DPI and need scaling to 72-DPI PDF points.Widthvalues, not the v3/v5BrushSize × 6 − 10.8formula.pages[]tocPageswith CRDT timestamps and PDF page redirection (redir).Why per-highlight color
The v6 glyph block carries a
PenColorindex (yellow=3, green=4, pink=5, etc., matching rmscene's enum) and, on firmware 3.6+, an optionalcolor_rgbapacked in BGRA at tag 10. The renderer was hardcoded to RGB(1,1,0) so multi-color highlighting on the device was lost on export. The same helper is now used for the v5 highlighter stroke path so itsBrushColoris honored too.Reference
PenColorenum: https://github.com/ricklupton/rmscene/blob/main/src/rmscene/scene_items.pyTest plan
go build ./...cleango test ./...passesgetaon a v3/v5 notebook — verify no regressiongetaon a v6 PDF with multi-color highlights — verify yellow/green/pink each render in their own colorgetaon a v6 notebook with strokes — verify stroke widths and positions look right🤖 Generated with Claude Code