Add wallet-side async payjoin sending - #113
Conversation
Store sender sessions and their rust-payjoin event logs the same way the receiver side stores its own: one row per session, an append-only event table with a unique per-session sequence, and a database-backed JsonSenderSessionPersister so a restart replays the log and resumes the session. Sessions carry the original transaction id, which serves as the fallback handle and as the double-payment guard for retried submissions. TODO in the model: key sessions on the receiver's ephemeral pubkey once the bindings expose PjParam.receiver_pubkey().
PayjoinSenderService starts a session from a BIP 21 URI: it parses the URI through rust-payjoin, funds and signs the original transaction from the store's hot wallet over NBXplorer with authenticated non-witness UTXO data, checks the pending-session double-payment guard, and persists the library's sender state via the session store. The original transaction is not broadcast at this point; it is the session's fallback. PayjoinSenderSessionProcessor advances every pending session one step per five-second tick, replaying the persisted event log first: post the original through an OHTTP relay, poll for the receiver's proposal, sign the proposal with the wallet (SignAll touches only inputs that match the store's derivation scheme, so the receiver's input is never signed here) and broadcast it, or broadcast the original when the library moves the session to its pending-fallback state and close the session through the library so the event log records the handoff. Relay selection iterates the store's configured OHTTP relays and treats an unreachable relay as a transient error for the next tick.
The store-scoped send page takes a BIP 21 URI, starts a sender session through PayjoinSenderService, and lists the store's sender sessions with their status and broadcast transaction. Store settings modification permission gates it, and only hot wallets pass the service's checks. A partial injected into the core onchain-wallet-send zone watches the destination field. When the pasted URI carries a pj endpoint with a fragment, which marks a BIP 77 directory endpoint the core v1 client cannot complete against, it offers the async flow with the URI carried over. The core send flow stays untouched.
Round-trip across fresh store instances, ordered event replay through the database-backed persister, pending filtering, the double-payment guard's lifetime, and terminal-state recording.
One BTCPay instance plays both roles: the merchant store receives through the plugin's receiver sessions, and the payer store pays the invoice through the new async sender session. After StartAsync the two background pollers complete the payjoin unattended. The test asserts the duplicate-submission guard, completion as a payjoin rather than a fallback, the invoice settling, and the receiver's contributed input inside the broadcast transaction.
The receiver's prepare_psbt strips HD keypaths from the proposal, so SignAll had nothing to derive from and finalization failed on the sender's own input. Ask NBXplorer to update the proposal PSBT with the store wallet's metadata first, the same way the core wallet flows do. Found by the end-to-end integration test, which now passes against a live regtest stack and the public payjoin directory.
40b5cbf to
abd47d4
Compare
A cold wallet, a hardware device or a multisig group has no key on the server, so the sender could not use it. BTCPay already has the screen that collects such a signature: PendingTransactionService creates the request, collects the signatures and publishes an event when the transaction is complete. It does not broadcast, which is exactly what payjoin needs. The sender now branches on whether the server holds a key. With a key, nothing changes. Without one, the transaction goes to a pending transaction and the session waits in a new AwaitingSignature status, which the poller skips because there is no library state to drive yet. A listener on the pending-transaction event picks the signed result up. Two rounds go through that path, because the receiver's proposal is a different transaction from the original: the first signature starts the session, and the second broadcasts the payjoin. If the operator never signs the second one, the library moves the session to its fallback and the original goes out instead, so the payment still completes. Two further changes come with it. Coin selection now excludes the outpoints of pending transactions, so a payjoin and a normal send cannot pick the same UTXO. And the repeated-submission guard now keys on the URI: two attempts on one URI do not have to select the same coins, so the transaction id alone did not catch a second submission.
|
Added cold wallet support to the sender. Hardware devices and multisig groups using the same path. |
A self-review against payjoin-cli and the rust-payjoin sender found six problems. This fixes them. The signature arrived only as an in-memory event, so a restart lost it and the session waited for ever. A cancelled or expired pending transaction produced no event at all, with the same result. The poller now sweeps every waiting session on each tick and reads the pending transaction itself, so the event only makes the common case prompt. The fee rate the operator chose never reached the library. It was a hardcoded 1 sat/vB, which is both the floor the receiver's proposal must clear and the size of the fee this sender contributes for the receiver's input, so a receiver could return a proposal that confirms far slower than the operator asked for. payjoin-cli passes the real rate. The session now records it, because the second signing round needs it too. A transaction the network already held was recorded as a failure. Several routes reach one broadcast, and only the first gets an accepting answer. A hot-wallet session reserved no coins, so the next send could spend them and invalidate the payjoin. payjoin-cli locks its inputs; the session now records them and coin selection leaves them alone. There was no way to stop a session. payjoin-cli offers cancel, which broadcasts the original. Stopping now does the same, so it stops the payjoin and not the payment. The last of these exposed a wrong assumption about the state machine: the library closes the sender session as soon as it hands over a valid proposal, before anything is broadcast. A closed session was read as a failure, which would have recorded a successful payjoin as failed after a restart. It now continues the proposal instead. Cancel cannot ask a closed session for its fallback either, so the session keeps its own copy of the signed original.
ValeraFinebits
left a comment
ValeraFinebits
left a comment
There was a problem hiding this comment.
Good feature, but as a separate page I think it will be inconvenient in production. The dedicated page loses coin selection, labels, fee-rate presets, balance, fiat conversion and other.
Core already models this: Paste BIP21 posts the URI to the server, and UIWalletsController.LoadFromBIP21 sets vm.PayJoinBIP21 when the URI carries a pj= endpoint, which drives the v1 payjoin. Async payjoin is the v2 variant of the same thing and belongs in that same flow.
If that isn't reachable from a plugin without a new extension point in core, then I'd suggest scoping this page explicitly as a testing tool rather than a production user flow, it is genuinely handy for making test payments while working on the plugin.
At first, I thought Core had a limitation, but you have mentioned something that I didn't first consider. I assumed there's a limit; let's see. I'm going to try folding it into the main page as you suggest, which was also my preferred shape |
Review feedback: a dedicated send page loses coin selection, labels, fee-rate presets, the balance and the fiat conversion, and core already models this. LoadFromBIP21 sets PayJoinBIP21 whenever a pasted URI carries a pj= endpoint, which drives the v1 payjoin. The v2 variant belongs in the same flow. It reaches from a plugin without a new extension point in core. The onchain-wallet-send extension point sits inside the send form and receives WalletSendModel, so a submit button there posts the whole form to this plugin: outputs, labels, coin selection and fee rate. So sending moves to core's screen. The partial injected there is now server-side Razor rather than JavaScript that read the store id out of the URL, because core resolves the URI server-side already. It also turns core's own payjoin toggle off for a v2 URI, which would otherwise attempt a v1 request against an endpoint that does not answer it. The plugin page keeps what core has no place for: the list of sessions, their state, the link to sign and the control to stop. An async payjoin pays one destination. The library takes its fee contribution from the first output that is not the payee, so a second payee would have the contribution taken from someone else's payment. The send screen allows more than that, so the difference is explained where the operator makes the choice.
ValeraFinebits
left a comment
ValeraFinebits
left a comment
There was a problem hiding this comment.
Requesting changes
Send.cshtml:12, :64 wallet links 403. They build $"{Model.StoreId}-BTC", but WalletId.TryParse requires ^S-… (WalletId.cs:11), so binding fails first. Hit both in a browser, including "Sign the transaction", the only way into the off-server flow. Use new WalletId(...).ToString(), as RedirectAfterStart does.
PayjoinSenderSessionProcessor.cs:386 round two never expires. expiry: null never matches core's sweep (PendingTransactionService.cs:43), and the poller skips AwaitingSignature, so an unsigned proposal waits forever with its coins reserved. Round one passes 7 days, please pass a window here too, and decide what lapsing should do.
PayjoinSenderSessionProcessor.cs:547, PayjoinSenderSignatureHandler.cs:213 the signed fallback is dropped. Both record Failed without touching OriginalTransactionHex. Abandon fires when the operator cancels the pending transaction, which is the same intent as Stop and Stop broadcasts. Deliberate for FailSession?
PayjoinSenderSessionStore.cs:260 the first-round pending transaction is orphaned. Clearing the id drops your handle, and core never retires a Signed row (PendingTransactionService.cs:342) while still excluding its outpoints (UIWalletsController.PSBT.cs:57). If the session later fails, those coins stay blocked permanently. Retire the row before clearing.
PayjoinSenderSignatureListener.cs:61 a cancel can race a collected signature. No status check here, no terminal guard in CompleteSession, and PendingTransactionId survives completion. Core blocks the wide case (PendingTransactionService.cs:182), but events are delivered async, so a signature collected just before Stop can land after it and broadcast the proposal on top of the original. Status check plus clearing the id on completion.
PayjoinWalletSendExtension.cshtml:54 core's v1 field is hidden, not cleared. PayJoinBIP21 is a string rendering as type="text" (WalletSend.cshtml:220), so the checkbox branch never runs. Verified live: the value still submits, so core's own Sign button still fires v1 at a v2 endpoint. toggle.value = '' unconditionally.
PayjoinSenderSessionProcessor.cs:462, PayjoinReceiverRelayTimeoutException : TaskCanceledException matches no catch in ProcessTickAsync, so a stalled relay skips every remaining session that tick.
ReconcileAsync sits outside the per-session guard, so one throwing session stops the rest.
PayjoinSenderSignatureHandler.cs:134 any InvalidOperationException is terminal, so a rejected broadcast discards a fully signed payjoin with no retry.
PayjoinSenderSessionStore.cs:172 core can't see SenderSessions, so an ordinary send can still spend a live session's coins.
Suggestion: reuse IPayjoinReceiverRelayRequestSender
SendThroughRelayAsync re-implements it, and the existing one already handles relay quarantine and rotation, catches the timeout above, disposes the context on every failure path (the copy never disposes it on success at all), and errors out instead of returning null when a store has no relays. It's already generic over the context type and RequestOhttpContext fits as-is. Needs invoiceId renamed and that timeout added to the transient list. Happy to take it.
Tests
Four that turn on timing or an external string: ConcurrentStartSignedSessionLetsExactlyOneWinnerThrough, TwoSubmissionsOfOneUriCreateOneSession, ConcurrentAppendsLeaveTheLogReplayable, BroadcastTreatsCoreMempoolDuplicateAsSuccess.
The first three need real constraint enforcement, the unit project's InMemory provider enforces no unique indexes, so PostgresPayjoinUniqueConstraintViolationDetector never fires and they'd pass against broken code. They can go straight into PayjoinPluginConcurrencyIntegrationTests beside the receiver's equivalents; no new fixture needed. The fourth wants the regtest node instead: broadcast one transaction twice and assert the second call succeeds, so it checks the real Core version's reason string. For the two sequence-conflict tests, pre-seeding the conflicting row is more reliable than racing tasks, the window between reading max(Sequence) and SaveChanges is small.
Two more with no new seams: an end-to-end test through SendFromWallet everything currently calls StartAsync directly, which is how the link bug got through and a sender equivalent of InFlightReceiverSessionSurvivesServerRestartAndCompletesPayjoin.
Both wallet links on the plugin page were built by hand as
"{storeId}-BTC". Core's binder needs the "S-" prefix, so every link
failed, including the only route into the off-server signing flow. The
view model now carries a real WalletId. Every test started a session by
calling the service, so nothing covered the page: two tests now go
through SendFromWallet the way the send screen does.
Round two never expired. It passed no expiry, core's sweep only retires
rows that have one, and the poller skips a session that waits, so an
unsigned proposal held its coins for ever. It gets the same window as
round one.
A session that will never be signed dropped a signed payment. Cancelling
or expiring the transaction recorded a failure, while Stop broadcasts:
the same intent with opposite outcomes. Ending a session now broadcasts
the original whenever it is signed, so the payment still happens.
Round one's pending transaction was orphaned. Clearing the id dropped
the handle to a row core keeps for ever and keeps excluding coins for,
so a session that later failed left those coins blocked with nothing to
release them. The row is retired when the session takes ownership.
A stop could race a signature collected moments earlier. The listener
now re-reads the session, the store keeps the first terminal state and
clears the pending id, so a late signature finds nothing to act on.
Core's payjoin field is a string, so it renders as a text input and the
checkbox branch never ran: hiding it left the value submitting and
core's Sign button still fired v1 at a v2 endpoint. It is cleared.
A relay timeout derives from TaskCanceledException and matched no catch,
so one stalled relay skipped every remaining session that tick. The
sweep ran outside the per-session guard with the same effect. Both are
caught per session now.
A refused broadcast ended the session and threw away a fully signed
payjoin. It has its own exception type and is retried next tick.
|
6fc3cf4 answers the remaining findings. Coin visibility did not need core. Each live session keeps a Signed pending transaction that holds the signed original, so core excludes its coins. The session releases the row when it ends.
I think we found a core bug. Fixed two more faults: the cleared v1 field also emptied the URI our own button posts, and a double submission of one URI could make two sessions. @ValeraFinebits Please take the relay sender. The four named tests are not in this branch. Their subjects now run in the integration project against Postgres. The restart test is skipped, with the same harness limit as the receiver's. |
Taking it. I'll keep this to one commit on top of bfe4742: wire |
ValeraFinebits
left a comment
ValeraFinebits
left a comment
There was a problem hiding this comment.
Requesting changes
Thank you for addressing all findings from my previous review. After reviewing and testing the follow-up changes on a98ef4e, I found the following additional issues. Please add regression coverage for each fix using scenarios like these:
-
Live outpoint uniqueness
PayjoinSenderSessionConfiguration.cs/20260819190647_AddSenderSessions
Enforce outpoint ownership per store at the database level.
Test: create two live sessions in the same store with different BIP21 URIs but the same outpoint. The second must be rejected; after the first becomes terminal, the outpoint should be reusable. -
Second-signature reservation sweep
PayjoinSenderSessionStore.GetPendingSessionsWithCoinReservations
The query excludesAwaitingSignature, so manual fallback activity can be missed during the second signing round.
Test: create a session with a coin reservation, transition it toAwaitingSignature, and verify the reservation sweep still returns it. -
Atomic state transitions
PayjoinSenderSessionStore.AwaitSignature/CompleteSession
Both methods remain read-check-write operations without a conditional update. A late writer can overwrite a terminal state.
Test: force both callers to readPending, letCompleteSessioncommit first, then releaseAwaitSignature; the late transition must fail and must not resurrect the session. Also verify that two concurrent completions produce exactly one winner. -
Proposal pending-transaction cleanup
PayjoinSenderSignatureHandler.BroadcastProposalAsync
A successfully broadcast proposal can leave its signing request looking actionable.
Test: complete the cold-wallet flow with two off-server signatures, wait forCompletedPayjoin, then verify the proposal pending transaction is absent or terminal, notPendingorSigned. -
Permanent broadcast rejection
PayjoinSenderBroadcaster,PayjoinSenderSessionProcessor.cs:133,PayjoinSenderSignatureHandler.cs:279
All broadcast failures are treated as transient, including permanent spent-input rejection.
Test: obtain a valid proposal, spend the receiver’s contributed input first, and process the sender again. It must broadcast the original transaction and finish asCompletedFallback, rather than remainPending. -
Head-of-line blocking
PayjoinSenderSessionProcessor.ProcessTickAsync
Sessions are awaited sequentially, so one relay long-poll blocks every following session.
Test: create two pending sessions and use a coordinating relay sender that blocks the first request. Verify the second request starts before the first is released. -
Store-scoped BIP21 uniqueness
PayjoinSenderSessionConfiguration.cs/20260819190647_AddSenderSessions
The filtered unique index is global instead of store-scoped.
Test: allow two different stores to create live sessions for the same BIP21 URI, while still rejecting a duplicate URI within the same store. -
Cumulative authorization
UIPayjoinSenderController.cs:22and:51
ASP.NET Core combines class- and action-level authorization; the action policy does not replaceCanModifyStoreSettings.
Test: inspect the effective policies forSendFromWallet. They should contain create/sign/broadcast wallet permissions and must not containCanModifyStoreSettings. -
Invalid model submission
UIPayjoinSenderController.SendFromWallet
The action does not checkModelState.IsValid.
Test: submit an otherwise usable model with a binding error and verify that no sender session or pending transaction is created.
Non-blocking: please derive the Status IN (0, 4) filter in PayjoinSenderSessionConfiguration.cs:33 from PayjoinSenderSessionStatus instead of using magic values.
2 participants
Adds wallet-side async payjoin: the store's hot wallet can pay a BIP 77 URI through a persisted sender session that posts, polls, signs, and broadcasts in the background, with the original transaction as the automatic fallback. A partial in the core wallet send screen offers the flow when the destination is a v2 payjoin URI. Built on rust-payjoin's sender state machine and event-log replay.
Draft while the development cycle continues; this PR carries the full history.
Disclosure: co-authored by Claude Code.