fix: resolve bigint timestamps and event upsert race conditions#965
Open
chizzy192 wants to merge 1 commit into
Open
fix: resolve bigint timestamps and event upsert race conditions#965chizzy192 wants to merge 1 commit into
chizzy192 wants to merge 1 commit into
Conversation
## Description This commit resolves two issues: 1. Migrates time-valued fields (startTime, lastUpdateTime, endTime, pausedAt, StreamEvent.timestamp) from Int to BigInt to prevent 2038 overflow (and early overflow on long-duration streams). 2. Prevents unique constraint / write-race collisions between action controllers and the indexer worker on `StreamEvent` records by using `upsert` in controllers, and conditional updates in the worker when a placeholder event exists. ## Type of Change - [x] 🐛 Bug fix (non-breaking change which fixes an issue) - [x] 🧪 Test addition or update ## Related Issues Closes LabsCrypt#830 Closes LabsCrypt#831 ## Changes Made - Modified `backend/prisma/schema.prisma` to change time columns to BigInt. - Added database migration `bigint_timestamps`. - Added global BigInt JSON serializer to `backend/src/app.ts`. - Updated `ClaimableStreamState` in `claimable.service.ts` and updated calculations to support bigint. - Modified withdraw, pause, resume controllers to use `upsert` and pass BigInt values. - Updated indexer worker event handlers to conditionally update ledgerSequence/timestamp on existing placeholder records. - Updated unit and integration tests to mock and expect `upsert`. - Fixed sunset/deprecated route tests. ## Testing - [x] Unit tests added/updated - [x] Integration tests added/updated ### Test Steps 1. Run `DATABASE_URL=postgresql://flowfi:[email protected]:5433/flowfi npm test`
Contributor
|
heads up: main's ci was broken (the Backend CI and Backend Docker Image CI jobs) until the fixes in #969 and #974 just landed, so the red backend/docker checks on this pr are almost certainly stale, they ran against the broken main. please rebase to re-test against the now-green main: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closes #830 closes #831
Description
This commit resolves two issues:
StreamEventrecords by usingupsertin controllers, and conditional updates in the worker when a placeholder event exists.Type of Change
Related Issues
Closes #830
Closes #831
Changes Made
backend/prisma/schema.prismato change time columns to BigInt.bigint_timestamps.backend/src/app.ts.ClaimableStreamStateinclaimable.service.tsand updated calculations to support bigint.upsertand pass BigInt values.upsert.Testing
Test Steps
DATABASE_URL=postgresql://flowfi:[email protected]:5433/flowfi npm testDescription
Type of Change
Related Issues
Closes #
Changes Made
Testing
Test Coverage
Test Steps
Breaking Changes
Breaking Changes:
Migration Guide:
Screenshots/Demo
Checklist
Additional Notes