Add preview-with-fade, JPEG support, and white-overlay bake#1
Open
kkhung1991 wants to merge 28 commits into
Open
Add preview-with-fade, JPEG support, and white-overlay bake#1kkhung1991 wants to merge 28 commits into
kkhung1991 wants to merge 28 commits into
Conversation
kkhung1991
commented
May 23, 2026
kkhung1991
commented
- New ImageProcessor native module decodes the source (PNG or JPEG), paints a white overlay with SRC_ATOP so transparency is preserved, and writes a PNG to the app cache. Used at insert time whenever fade > 0 or the source is JPEG (PluginNoteAPI.insertImage is PNG-only).
- StubPackage now registers ImageProcessor instead of returning empty.
- App.tsx: tapping a thumbnail opens a preview screen with the image, a live white-overlay opacity matching the requested fade, a custom PanResponder slider with +/- steppers and 0/25/50/75/90 % presets, and Cancel/Insert buttons. JPEG/JPG added to the accepted extensions.
- Renamed the surfaced plugin from "Embed PNG" to "Embed Image" and bumped to 0.3.0 / versionCode 8.
- New ImageProcessor native module decodes the source (PNG or JPEG), paints a white overlay with SRC_ATOP so transparency is preserved, and writes a PNG to the app cache. Used at insert time whenever fade > 0 or the source is JPEG (PluginNoteAPI.insertImage is PNG-only). - StubPackage now registers ImageProcessor instead of returning empty. - App.tsx: tapping a thumbnail opens a preview screen with the image, a live white-overlay opacity matching the requested fade, a custom PanResponder slider with +/- steppers and 0/25/50/75/90 % presets, and Cancel/Insert buttons. JPEG/JPG added to the accepted extensions. - Renamed the surfaced plugin from "Embed PNG" to "Embed Image" and bumped to 0.3.0 / versionCode 8.
- Folder navigator: replaces the fixed-folder grid. Entries are classified as image (.png/.jpg/.jpeg/.bmp/.gif/.webp) or folder (no extension); folders are tappable and sorted first. Up/Home buttons and a current-path bar. Navigation is clamped to /storage/emulated/0. - Browse… button calls RattaFileSelector.selectFile so users can pick an image from anywhere the host exposes, including WebDAV mounts and the SD card. - ImageProcessor.processForEmbed now accepts brightness (-100..100), contrast (-100..100), gamma (0.5..2.0), and a previewMaxDim that downsamples via BitmapFactory inSampleSize so live previews stay responsive on e-ink. Pixels are processed in a single pass: B/C linear, then a gamma LUT, then the white overlay (alpha-preserving). - New cleanupCache() native method clears prev_*.png/embed_*.png from app cacheDir on plugin open. - Preview screen now has four AdjustRow sliders (each with -/+ steppers and a Reset button) and a debounced auto-bake (~350 ms) that updates the Image source with the downsampled preview PNG. Final Insert bakes at full resolution. - Bumped to 0.4.0 / versionCode 9.
- New Size −/+ buttons in the sort bar adjust grid column count (2..6, default 3). FlatList is re-keyed on column change. - Tile sizing switched from hardcoded flex: 1/3 to a dynamic flexBasis/maxWidth = 100/columns so the grid recomputes per click. - Empty-state hint now spells out that re-tapping Browse re-launches the native picker, which is the only way to force a WebDAV listing refresh (the picker activity is owned by the host, so we can't add a refresh button inside it). - Bumped to 0.4.1 / versionCode 10.
Previously the navigator clamped to /storage/emulated/0, so the in-plugin folder browser couldn't reach the SD card mount. Now: - On open, call FileUtils.getExternalDirPath() and store any returned mounts (filtered to exclude the internal path). - Path bar gains an "Internal" chip and one "SD" chip per external mount (labelled "SD 1", "SD 2" when there are multiples). The chip for the current root is highlighted. - Navigation clamp is now per-root: Up disables when currentDir equals whichever root contains it, and won't escape into the parent /storage/. - Bumped to 0.4.2 / versionCode 11.
…b UI Plugin side ----------- - New macapp/ Python project: Flask HTTP capture server backed by mss (full screen / region / Quartz-enumerated window) plus a Tkinter GUI with LAN-IP/port display, source picker, region-drag overlay, interval slider, start/pause toggle, and a rolling log. - New CaptureScreen: polls /frame at a configurable interval, decodes + bakes via the native module (downloadAndProcess), shows the latest preview, exposes adjustment sliders, an on-screen log (last 12 lines), and Insert + Replace-in-place buttons. - New SettingsScreen: edits and persists host / port / intervalSec via a SharedPreferences-backed key/value pair in the native module. - Browser screen gains Live… and Settings buttons in the header. Element tracking ---------------- - embedTracker uses PluginFileAPI.getLastElement after insertImage to capture the new Picture element's uuid / numInPage / rect, then on Replace re-fetches the element by uuid (so user-applied lasso moves are picked up), deletes it, and re-inserts a Picture element at the same rect/layer via insertElements. Adjustment UI refactor ---------------------- - Old: all four sliders stacked in a scroll view. - New: AdjustmentPanel with tabs (Fade / Brightness / Contrast / Gamma) showing one slider at a time plus ± steppers, Reset, and per-tab preset chips (e.g. 0/25/50/75/100 for Fade). Native module ------------- - downloadAndProcess(url, ...adjustments, previewMaxDim, timeoutMs): HttpURLConnection fetch → cache file → existing bake pipeline → PNG path. Used by the capture loop so streaming is one native round-trip per frame instead of fetch-in-JS + bake. - getConfigValue / setConfigValue: SharedPreferences-backed string KV; Settings serializes its config as JSON under the streamConfig key. - cleanupCache now also clears dl_*.bin temporary downloads. Misc ---- - AndroidManifest gains usesCleartextTraffic=true so the plugin can HTTP-fetch the Mac on the LAN. - App.tsx slimmed down to a router; screens live under src/screens/. - Bumped to 0.5.0 / versionCode 12. Note: Replace-in-place depends on PluginFileAPI.insertElements accepting a synthesised Picture element with type=200 + picture.rect/picturePath. If the host rejects that shape we'll see a "replace failed" log and the Insert path keeps working as a static one-shot.
Crash fix - Drop the FileUtils.getExternalDirPath() probe in Browser.tsx. On the latest Manta beta firmware getCurrentActivity() returns null when the JS bundle first runs, so the native call throws an NPE that escapes the SDK module (its try/catch starts after the throwing line), trips PluginJSExceptionHandler, and tears down the RN bridge before our await even rejects. SD-card and WebDAV access still work through the "Browse..." system picker. Stream adjustments on the Mac side - Server: new /adjust endpoint; brightness/contrast/gamma/fade are baked into each frame with PIL before serving, so the Manta receives pre-adjusted PNGs. - Capture screen: debounce-POSTs adjustments to /adjust, then calls the native bake with identity values so the Supernote side does no per-pixel work. - Native: bake() now short-circuits the per-pixel loop when all adjustments are identity (just decode/resize/encode). Mac region picker - Replaces the fullscreen semi-transparent overlay (which on some multi-monitor setups went black on the wrong display) with an in-window modal: capture the primary monitor once, downscale to ~640px, let the user drag a rectangle on the thumbnail, map back to monitor pixels. Bump 0.5.0 -> 0.5.1 / versionCode 12 -> 13. https://claude.ai/code/session_01FTRgWnGRMVpuvHBVnoRMkh
The Supernote pluginhost process disallows cleartext HTTP (Android's NetworkSecurityPolicy), so JS fetch() to the LAN Mac server fails with "cleartext HTTP traffic to <ip> not permitted". Our own manifest's usesCleartextTraffic="true" doesn't apply because the policy is read from the host's process, not ours. - ImageProcessorModule.rawHttpRequest: HTTP/1.1 over a raw Socket, which doesn't hit NetworkSecurityPolicy at all. Handles Content-Length and chunked Transfer-Encoding responses. - downloadAndProcess now uses rawHttpRequest instead of HttpURLConnection. - New nativeHttp(method, url, bodyJson, timeout) for JS. lanHttp/lanJson wrappers in src/imageProcessor.ts. - Capture screen: /status and /adjust now go through lanHttp/lanJson. Settings "Test connection" too. - StubPackage: best-effort reflection patch of NetworkSecurityPolicy as a defensive layer (cheap, no-op on Android versions where the field shape differs). Bump 0.5.1 -> 0.5.2 / versionCode 13 -> 14. https://claude.ai/code/session_01FTRgWnGRMVpuvHBVnoRMkh
The plugin's JS context is torn down when the view closes, so the embedded image can't auto-update in the background. This patch shortens the refresh loop instead. - Capture screen: bottom-row Close now exits the plugin (back to the note canvas), not back to the file Browser. Header Back still goes to Browser. - Capture screen: new Replace & Close button. One tap, fetches the latest frame, replaces the tracked embed (or inserts on first run), closes the plugin. - New "Refresh Embed" sidebar button (id=2). On tap it opens a tiny status screen that loads the persisted embed track + stream config, fetches the latest frame, calls replaceInPlace, and closes itself. No need to navigate through the file picker. - Embed track persisted via ImageProcessor.setConfigValue so the refresh button works after a fresh plugin launch. Bump 0.5.2 -> 0.5.3 / versionCode 14 -> 15. https://claude.ai/code/session_01FTRgWnGRMVpuvHBVnoRMkh
…fNet Resolution multiplier - New resolution_mul in CaptureState (server-side downscale, 0.1..1.0). - /resolution POST endpoint; included in /status payload. - Settings adds a "Resolution multiplier" text field; Capture screen has a Res − / NN% / + row that pushes /resolution debounced. - StreamConfig.resolutionMul is persisted alongside host/port/interval. Connection-status dot - New useConnStatus hook polls /status every 5s with a 2s timeout. - StatusDot renders solid (connected), outlined (disconnected/unknown). - Browser, Capture, and Preview headers all show the dot. Manta-side source picker - New SourcePicker screen: Full screen / Window list / Region. - Region mode fetches /preview-shot (downscaled primary monitor), the user drags a rectangle on the touchscreen, coords are mapped back to Mac pixels using monitor metadata from /status. - Capture screen's header has a new "Source" button that opens it. - New /preview-shot endpoint serves a one-off downscaled screenshot with monitor headers; /status now also returns the monitor rect. BiRefNet background removal (Mac-side) - New /birefnet POST endpoint: lazy-loads torch + transformers + ZhengPeng7/BiRefNet, runs segmentation, composites onto white (or returns alpha with ?bg=transparent). Returns 503 if torch is not installed (user has to `pip install torch torchvision transformers` separately; requirements.txt has a comment for this). - Native: new nativeHttpPostFile that POSTs a file's bytes and writes the binary response to a cache file (lanHttp would corrupt PNG by decoding as UTF-8). JS wrapper: lanPostFile. - Capture: Remove BG button. Only enabled when paused (live stream would loop forever recalculating). Applies to the still preview; Insert/Replace then embeds the BG-removed frame. - Preview: Remove BG button next to Insert. Sends the gallery image to the Mac, swaps the displayed image with the result, embeds that on Insert. Bump 0.5.3 -> 0.6.0 / versionCode 15 -> 16. https://claude.ai/code/session_01FTRgWnGRMVpuvHBVnoRMkh
…, presets
UI: Windows 95 chrome everywhere
- New theme.ts / Win95.tsx with raised/inset bevels, TitleBar with X
close, MenuBar with drop-down menus, Win95Button, StatusBar.
- VT323 pixel font bundled (android/app/src/main/assets/fonts/).
- Browser, Capture, Preview, Settings, Refresh, SourcePicker all
re-skinned. Folder thumbnails read "[DIR]", logs read "C:\>_".
- StatusDot is now a tiny bezeled LED.
CRT screensaver region picker
- The region mode of SourcePicker now mimics a beige '90s monitor:
rounded plastic frame with stand + knobs + power LED, scanline
overlay, phosphor-green selection rectangle + corner tags ("SOURCE-1",
size readout), flicker animation, "NO SIGNAL" placeholder.
Image filters (#1 dither)
- New dither modes baked into the Mac /adjust pipeline: off, 1-bit
Floyd-Steinberg, 4-level gray (Manta's native range), Atkinson.
- Adjustments type gained `dither: DitherMode`.
- AdjustmentPanel grew a Dither chip row.
Watch folder (#4 dropbox / airdrop magic)
- Mac creates ~/EmbedImage/Drop on startup. New endpoints:
/drop/list, /drop/file?name=X, /drop/consume {name}. Status now
exposes drop_count + watch_folder for badges.
- New DROP.EXE sidebar button (id=3) opens a Win95 inbox that lists
files newest-first with thumbnails. Tap to embed; Mac moves the
file to .consumed/ so it won't show up again.
Lasso to Mac (#5 reverse direction)
- New lasso-toolbar button (id=4, type=2) "Send to Mac". On press,
uses PluginCommAPI.generateLassoPreview to export the lasso as PNG,
POSTs to /sketch?name=…, server stashes it in ~/EmbedImage/Sketches.
Adjustment presets (#9)
- Mac persists named presets in ~/EmbedImage/presets.json.
- /presets GET, POST {name, ...}, DELETE /presets/<name>.
- AdjustmentPanel shows a horizontal preset chip row at the top with
a Save… inline field and long-press-to-delete on existing chips.
- lanHttp / lanJson now also accept DELETE and PUT methods.
Refresh screen
- Now a Win95 progress dialog with a marching-pixel progress bar.
Bump 0.6.0 -> 0.7.0 / versionCode 16 -> 17.
https://claude.ai/code/session_01FTRgWnGRMVpuvHBVnoRMkh
BiRefNet - The published weights ship in fp16; on MPS torchvision transforms produced fp32 input and the model crashed with "Input type (float) and bias type (c10::Half) should be the same". - Force the model to fp32 on load. Also cast input tensor to the model's dtype defensively, in case future revisions ship different precisions. MenuBar - Drop-downs were rendered as siblings of the menu bar inside a normal View; later sibling components (preview frames, status bar, etc.) drew on top of them and clicks needed multiple tries to land. - Render the open dropdown through a transparent <Modal> so it's a top-level overlay above everything. The menu label is measured at open time and the dropdown anchors directly under it. Backdrop dismisses on any outside tap. Bump 0.7.0 -> 0.7.1 / versionCode 17 -> 18. https://claude.ai/code/session_01FTRgWnGRMVpuvHBVnoRMkh
…ures Watchdogs on the headless-ish screens - RefreshScreen: 15s hard ceiling. If any SDK call hangs (the embedded insertAndTrack / replaceInPlace can stall when the note context is off), we set a timeout message and close instead of leaving the spinner running forever. Also publishes per-stage status so the user knows where it got stuck. - SendLasso: 20s watchdog + per-stage status. Lasso export errors are caught explicitly (was bubbling up before). Auto-closes on success after a 1.5s confirm — no need to tap OK. DropInbox robustness - Bumped /drop/list timeout 3s -> 5s. - Thumbnail prefetch loop uses a generation counter so a Refresh tap (or unmount) cleanly invalidates the old run. Each tile updates incrementally as its thumbnail arrives — no big batch at the end that can clobber newer state. - A single failing fetch can't stall the row (per-fetch 5s timeout). Async button registration - registerButton returns a Promise; the sync try/catch around the type=2 lasso button couldn't see async rejection. If the host rejected (older firmware), we ended up with a half-registered button that could confuse the lasso menu. All registerButton calls now have .catch() handlers that log + skip. Bump 0.7.1 -> 0.7.2 / versionCode 18 -> 19. https://claude.ai/code/session_01FTRgWnGRMVpuvHBVnoRMkh
Logcat showed the smoking gun: java.lang.NullPointerException at HashSet.<init>(HashSet.java:118) at PluginButtonAdapter.onBindViewHolder(PluginButtonAdapter.java:195) and at registration time: PluginJSExceptionHandler error: NullPointerException: Attempt to invoke 'java.util.Iterator java.util.List.iterator()' on a null object reference right after our type=2 button got logged with nameMap=null, descMap=null, and no editDataTypes. The host's area-selection adapter does `new HashSet<>(nameMap.keySet())` without a null guard, so the moment the user tapped the lasso "..." menu the whole note app crashed. The earlier registration NPE was the same root cause from a different code path (also missing editDataTypes for the lasso button kind). - Every registerButton call now supplies nameMap and descMap (English for now; can localize later) via a shared helper. - Lasso button additionally supplies editDataTypes covering all six lasso content kinds so the button shows up no matter what was selected. Bump 0.7.2 -> 0.7.3 / versionCode 19 -> 20. https://claude.ai/code/session_01FTRgWnGRMVpuvHBVnoRMkh
Send to Mac - generateLassoPreview was being asked to write to /data/local/tmp/, which is adb-shell-only — the plugin process can't write there so the SDK silently returned no content. Move to the pluginhost's own cache dir (/data/user/0/com.ratta.supernote.pluginhost/cache/), which we know is writable because downloadAndProcess already writes there. Refresh embed - Old order was delete-then-insert. When the insert step failed (any validation hiccup, transient SDK error, etc) the user was left with no embed at all: the delete had already committed. - Try modifyElements first — atomic, matches by uuid+numInPage. If the SDK accepts it, the picture path is swapped in place and the rect is preserved. - Fallback path is insert-then-delete (note the reversed order!). A failed insert now leaves the original embed untouched; a failed delete leaves two pictures, which is still better than zero. Bump 0.7.3 -> 0.7.4 / versionCode 20 -> 21. https://claude.ai/code/session_01FTRgWnGRMVpuvHBVnoRMkh
Logs from 0.7.4 showed modifyElements running cleanly + saveTrailsAsTemp succeeding, but the displayed picture never repainted. The renderer caches the picture bitmap by element identity; a path-only swap via modifyElements isn't enough to invalidate that cache. - Drop modifyElements entirely. Always go through insertElements followed by deleteElements (in that order — the previous bug came from delete-first; an insert failure left the user with nothing). - If insertElements rejects, fall back to PluginNoteAPI.insertImage() which is the same call the first-time embed uses. We lose the rect in that case but the user sees a fresh frame and can re-position. - Heavy console.log around each SDK call. Next time something fails the response shape will be visible in adb logcat instead of having to guess. Bump 0.7.4 -> 0.7.5 / versionCode 21 -> 22. https://claude.ai/code/session_01FTRgWnGRMVpuvHBVnoRMkh
Refresh embed
- Logcat showed insertElements rejecting with code 106 "Invalid API
parameters". A bare {type, pageNum, layerNum, picture} payload isn't
enough — the host validates a bunch of fields a real picture element
carries (maxX, maxY, thickness, status, recognizeResult, ...).
- Fetch the raw existing element via getElements, deep-clone it, then
swap only picture.picturePath. Drop uuid and numInPage so the host
assigns fresh ones. That gives us a payload the host accepts because
it's identical in shape to one it produced itself.
- Drop the insertImage fallback too: observed runs put the new picture
at pageNum=-1, which means the user never sees it. Better to fail
loudly and keep the original embed than silently put it off-page.
SendLasso
- Add console.log around every awaited step (config load,
generateLassoPreview call & result, upload). Last log only showed the
verifyParams line, so the user couldn't tell whether the SDK call
failed or the upload did. Now adb logcat will show exactly which
step stalls.
Bump 0.7.5 -> 0.7.6 / versionCode 22 -> 23.
https://claude.ai/code/session_01FTRgWnGRMVpuvHBVnoRMkh
Cribbed approach from guibor/supernote-endpoint-lasso and Laumss/Inkling
— neither uses generateLassoPreview (its sandbox semantics are flaky).
SendLasso
- Drop generateLassoPreview. Use the same pipeline Inkling/guibor use:
getCurrentFilePath + getCurrentPageNum
getLassoRect
PluginFileAPI.generateNotePng({notePath, page, times:1, pngPath, type:1})
- Upload the rendered page PNG to /sketch with the lasso rect as query
parameters (cropL/T/R/B). Mac crops server-side before saving.
- Per-step console.log so the next adb log shows exactly which call
succeeds or fails.
Refresh embed
- When cloning the existing element, also strip recognizeResult,
status, contoursSrc, angles — those describe the OLD picture's
derived data and confuse the host's validator. Keep maxX/maxY/
thickness/layout fields and the picture sub-object (now with the
new picturePath and the fresh rect).
- Add an insertImage-then-modifyElements-to-move fallback. When
insertElements still rejects, insertImage places a new picture at
the host's default position, then modifyElements moves it to the
original rect. That avoids the previous failure mode where
insertImage worked but the user couldn't see the picture (it
landed off-page).
Server
- /sketch accepts optional cropL/T/R/B query params and crops the
uploaded PNG to that rect with PIL before saving. Bounds are
clamped to the image; out-of-bounds requests just save the whole
page so we never lose data.
Bump 0.7.6 -> 0.7.7 / versionCode 23 -> 24.
https://claude.ai/code/session_01FTRgWnGRMVpuvHBVnoRMkh
Lasso Recognize (Inkling's killer feature)
- New lasso-toolbar button "Recognize" (id=5, type=2).
- src/lassoRecognize.ts ports Inkling's pipeline:
PluginCommAPI.getLassoElements
-> partition into strokes / text boxes / pictures
PluginCommAPI.recognizeElements(strokes, pageSize)
-> recognized text
PluginNoteAPI.insertText(text, rectBelowLastTextBox or belowLasso)
- New RecognizeLasso screen — same Win95 progress dialog as Refresh,
shows per-stage status + closes itself when done. Restricted to
strokes/title/text editDataTypes so it doesn't appear on a pure
picture lasso.
Persistent file logger
- Inkling logs every event to /sdcard/INBOX/localsend-plugin.log so it
survives bridge tear-downs. Same idea here, written to
/sdcard/EmbedImage/log.txt via a new native appendFile method.
- src/util/FileLogger.ts mirrors every entry to console.log AND
enqueues for batch flush to the file. Variadic raw() drop-in for
the existing console.log call-sites.
- Wired into embedTracker.replaceInPlace and SendLasso so the next
refresh/send leaves a complete trace at /sdcard/EmbedImage/log.txt
even if logcat is noisy. `adb pull /sdcard/EmbedImage/log.txt` to
inspect.
Bump 0.7.7 -> 0.8.0 / versionCode 24 -> 25.
https://claude.ai/code/session_01FTRgWnGRMVpuvHBVnoRMkh
Lasso→Mac was failing because generateLassoPreview never reliably wrote
a file. The actual working API (per jpmoo/lassoexport) is:
PluginCommAPI.saveStickerByLasso(stickerPath) // .sticker file
PluginCommAPI.getStickerSize(stickerPath) // {width, height}
PluginCommAPI.generateStickerThumbnail(stickerPath, pngPath, size)
src/lassoExport.ts implements this end-to-end:
- Uses PluginManager.getPluginDirPath() as the writable scratch dir.
- Clears the element cache + any stale .sticker files before saving.
- Derives a base filename from the note name.
- Uploads the resulting PNG to the Mac's /sketch endpoint.
Truly headless button (showType:0)
- The Send to Mac lasso button is now showType:0, so tapping it never
opens a plugin view. The button-press handler in index.js calls
runSendLassoToMac() directly. Feedback comes via Android Toast on
success and NativeUIUtils.showRattaDialog on failure — same pattern
jpmoo uses.
- SendLasso screen and 'sendlasso' route deleted (not needed anymore).
Recognize keeps its progress screen because OCR takes longer and a
spinner is useful.
PNG / JPEG format selection
- StreamConfig gains lassoFormat: 'png' | 'jpg'. Settings has a
two-button toggle next to Resolution multiplier.
- Manta always uploads PNG bytes; ?format=jpg in the query string
tells the Mac to re-encode and rename the file accordingly. PNG
keeps transparency, JPEG is smaller.
Bump 0.8.0 -> 0.8.1 / versionCode 25 -> 26.
https://claude.ai/code/session_01FTRgWnGRMVpuvHBVnoRMkh
Inkling's StitchEditor.tsx is brought in with our Win95 chrome. The gesture math (edge handles, perpendicular shared handles, overlap drag, MIN_VISIBLE clamping) is copied verbatim because it's the meat. Native compositor (Kotlin) - composeStitch in ImageProcessorModule. Decodes both source bitmaps, applies the 0..1 fractional crops, lays them out vertically or horizontally with `overlap` source-pixel overlap, draws bottom layer then top layer onto an ARGB_8888 canvas with a white background, saves as PNG to the requested path. - JS wrapper exported as composeStitch(a, b, params, outPath). Editor screen - src/screens/StitchEditor.tsx — full port. Replaces Inkling's RNFS / @react-native-community/image-editor / opaque-native-compose deps with our ImageProcessor.composeStitch + PluginNoteAPI.insertImage. - On Confirm: composes to /data/user/0/.../cache/stitch_<ts>.png, insertImage()s it, saves the note, closes the plugin view. Full FileLogger trace at /sdcard/EmbedImage/log.txt. User flow - Browser → Tools → "Stitch two images…" flips into stitch-pick mode. First tap on an image becomes image 1; second tap becomes image 2 and routes to the editor with both. Cancel via the same menu item. - Image dimensions are measured (Image.getSize) before constructing the session — the editor's layout math needs width/height up front. Bump 0.8.1 -> 0.9.0 / versionCode 26 -> 27. https://claude.ai/code/session_01FTRgWnGRMVpuvHBVnoRMkh
New lasso-toolbar button "Stitch Layers" (id=6) that opens a layer
picker on the canvas. Same convertElement2Sticker pipeline Inkling
uses for clipSave, but applied per-user-selection of which Supernote
note layers (0..3) to include.
Flow
- surveyLasso():
PluginCommAPI.getLassoRect + .getLassoElements
PluginFileAPI.getLayers
group elements by layerNum -> show count per layer.
- composeLassoLayers({selectedLayerIds, transparentBg}):
filter lasso elements to selected layers
PluginCommAPI.convertElement2Sticker(...)
PluginCommAPI.getStickerSize / .generateStickerThumbnail
if !transparent, ImageProcessor.flattenOntoBg over white
- insertComposed(): PluginNoteAPI.insertImage (known reliable).
UI (LayerStitch screen, Win95 chrome)
- Per-layer rows with ☐/☑ checkboxes + element counts, "(current)"
badge on the active layer, muted style for layers with 0 lassoed
elements.
- Transparent / White background toggle.
- Three actions: Preview (re-renders into the dialog), Insert
(insertImage into current note), To Mac (re-uses /sketch endpoint
with the chosen lassoFormat).
- StatusBar shows running state + total picked elements.
- Errors path: empty lasso → friendly dialog with OK button.
Native flattenOntoBg
- ImageProcessorModule.flattenOntoBg(input, out, r,g,b). Loads the
PNG, draws a solid fill, then drawBitmap on top. Used by the
composer when the user picks the White option.
Bump 0.9.0 -> 0.9.1 / versionCode 27 -> 28.
https://claude.ai/code/session_01FTRgWnGRMVpuvHBVnoRMkh
User report: tapping "Stitch Layers" landed on the default Browser
screen instead of the layer picker. Root cause is the well-known race
on cold-reloaded RN context: the host fires the button event around the
time the JS bundle is reloading, but App.tsx's registerButtonListener
in useEffect only attaches AFTER the first render. By then the SDK's
lastButtonEventMsg replay may or may not still be active, so the first
render shows the wrong screen.
Inkling solves this with a module-level pendingButton:
pendingButton.js — set/check/peek API
index.js — button listener stashes msg.id synchronously
App.tsx — initial-screen useState lazy initializer calls
checkPendingButton() so the FIRST render is correct
Imported the pattern. App.tsx now picks its initial screen based on
the stashed button id; subsequent button presses still flow through
PluginManager.registerButtonListener as before.
Defensive layer listing
- If PluginFileAPI.getLayers fails or returns nothing useful, fall
back to inferring layers from the lassoed elements' layerNum so
the LayerStitch screen always shows something to pick from.
- Always include layer 0 (main) so there's at least one row.
- More detailed FileLogger entries (per-layer element counts) so the
next log shows exactly what the host returned.
Bump 0.9.1 -> 0.9.2 / versionCode 28 -> 29.
https://claude.ai/code/session_01FTRgWnGRMVpuvHBVnoRMkh
Latest user log only shows runs of Send-to-Mac (id=4) and Recognize
(id=5). No Stitch Layers (id=6) attempts, which means either:
(a) the installed snplg is still pre-0.9.1 (button doesn't exist), or
(b) the registration is silently failing on this firmware build.
Switch all registerButton call-sites to a helper that logs success or
failure to /sdcard/EmbedImage/log.txt instead of console.log (which
isn't captured by FileLogger). Also log:
- bundle start with versionName/versionCode (so we can confirm which
build is actually loaded), and
- every onButtonPress event so we see exactly what the lasso menu
dispatches.
Next `adb pull /sdcard/EmbedImage/log.txt` will start with something
like:
[embedimage] bundle start {"version":"0.9.3","code":"30"}
[embedimage] registerButton OK {"id":6,"label":"Stitch Layers",...}
[embedimage] onButtonPress {"id":6,...}
If the registerButton line is missing or has FAIL, we know the host
is rejecting the button payload and can fix it. If the onButtonPress
line is missing, the button just isn't being tapped (probably hidden
in the lasso menu).
Bump 0.9.2 -> 0.9.3 / versionCode 29 -> 30.
https://claude.ai/code/session_01FTRgWnGRMVpuvHBVnoRMkh
The 0.9.3 startup log confirmed all six buttons (incl. Stitch Layers
id=6, type=2) register cleanly:
[embedimage] registerButton OK {"id":6,"label":"Stitch Layers","type":2,"ok":true}
…yet no onButtonPress event for id=6 ever fires. The Manta's lasso
"..." popup has limited visible space; with Sticker + Send to Mac +
Recognize already occupying it, the third plugin button scrolls
off-screen and the user can't reach it.
Add a SIDEBAR (type=1) duplicate registration. The lasso selection
persists when the plugin view opens (Recognize already proves this),
so the user can lasso → open sidebar → tap "Stitch Layers" and the
LayerStitch screen sees the live selection just as if it had been
triggered from the lasso menu. Distinct button id (7) so the two
registrations don't collide; App.tsx routes both 6 and 7 to the same
LayerStitch screen.
The lasso-toolbar registration (id=6) stays in place too — for users
who do scroll the popup or whose firmware shows more buttons.
Bump 0.9.3 -> 0.9.4 / versionCode 30 -> 31.
https://claude.ai/code/session_01FTRgWnGRMVpuvHBVnoRMkh
The 0.9.4 log nailed the actual bug:
23:00:21 bundle start (background context)
23:00:37 onButtonPress {id:7} ← handled here, sets in-memory var
23:00:47 bundle start AGAIN ← fresh UI context for the plugin view
(no [LayerStitch] survey log — App.tsx routed to default Browser)
The host runs index.js TWICE — once as a background context that
registers buttons and the press listener, and again as a fresh UI
context when the plugin view opens. They have disjoint JS module
state, so the in-memory pendingButtonId set during the press is gone
by the time App.tsx mounts in the UI bundle. That's why all our
"opens a view" lasso buttons silently fall through to the default
Browser screen.
Fix: back pendingButton with SharedPreferences via a synchronous
native method.
- ImageProcessorModule.setPendingButton(id) — async write
- ImageProcessorModule.getAndClearPendingButton() — synchronous read
with @ReactMethod(isBlockingSynchronousMethod = true), returns a
String so App.tsx's useState lazy initializer can read it before
the first render frame.
- pendingButton.js now writes via the native setter AND keeps an
in-memory cache for same-context navigation; checkPendingButton
consults SharedPreferences first.
Bump 0.9.4 -> 0.9.5 / versionCode 31 -> 32.
https://claude.ai/code/session_01FTRgWnGRMVpuvHBVnoRMkh
0.9.5 log shows the BG bundle starting and onButtonPress {id:7}
firing, but then nothing — no UI bundle start, no App mount, no
LayerStitch survey. Need to know whether App is mounting (with stale
state) or never mounts at all.
Add three FileLogger entries:
- "App mounted" in App.tsx useEffect
- "App initialScreen { pendingId, chosen }" in the useState lazy init
- "App onButtonPress { id }" inside the registerButtonListener cb
Next adb pull will tell us: bundle started in UI ctx? App mounted?
Routed to layerstitch or browser? pendingButton value was read or
came back null?
Bump 0.9.5 -> 0.9.6 / versionCode 32 -> 33.
https://claude.ai/code/session_01FTRgWnGRMVpuvHBVnoRMkh
User report: 0.9.6 log shows App.initialScreen running and routing to layerstitch correctly, but the screen never visibly shows up (no App.mounted log either). The view either doesn't open or mounts hidden — same routing race we've been chasing. Stop fighting the view lifecycle. Add two headless lasso-toolbar buttons that mirror the showType:0 pattern from jpmoo's lassoexport (the only pattern we KNOW works reliably for this firmware): - "Stitch (transparent)" (id=8): convertElement2Sticker on the lasso, generateStickerThumbnail to PNG, insertImage into the current note, Toast on success / Ratta dialog on failure. - "Stitch (white BG)" (id=9): same but flatten over white via the ImageProcessor.flattenOntoBg native method first. No view, no pendingButton handoff, no App.tsx round trip. Press the button → it works → Toast confirms. The view-based "Stitch Layers" picker stays registered for users who want layer selection (and as a regression target so we can finish debugging the routing later). Implementation lives in src/quickStitch.ts; index.js wires the two button IDs to runQuickStitch(true|false). Bump 0.9.6 -> 0.9.7 / versionCode 33 -> 34. https://claude.ai/code/session_01FTRgWnGRMVpuvHBVnoRMkh
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.