Deposit Tracking System

An enterprise-grade, highly resilient backend system for tracking financial deposits (e.g., blockchain or crypto deposits). This system guarantees that every deposit is reliably recorded and eventually delivered to an external callback server, even in the event of severe network failures or external downtime.

🚀 The Core Architecture

This project is built around the Transactional Outbox Pattern, which mathematically guarantees no data loss between your database and external webhooks.

                    ┌───────────────────────┐
                    │       Client / API    │
                    │  (Postman / curl /    │
                    │   requests.http)      │
                    └──────────┬───────────┘
                               │
                               ▼
                    ┌───────────────────────┐
                    │   Express API Server  │
                    │ (Validation + Routes) │
                    └──────────┬───────────┘
                               │
                               ▼
                    ┌──────────────────────────────┐
                    │      Service Layer           │
                    │  (Wallet / Deposit Logic)    │
                    └──────────┬───────────────────┘
                               │
                ┌──────────────┴──────────────┐
                ▼                             ▼
     ┌──────────────────────┐     ┌──────────────────────┐
     │   Transactions Table │     │     Outbox Table     │
     │ (Financial Records)  │     │ (Pending Events)     │
     └──────────┬───────────┘     └──────────┬───────────┘
                │                            │
                └──────────────┬─────────────┘
                               ▼
                    ┌───────────────────────┐
                    │    PostgreSQL DB      │
                    └──────────┬───────────┘
                               │
                               ▼
                    ┌───────────────────────┐
                    │   Outbox Worker       │
                    │ (Polling + Retry)     │
                    └──────────┬───────────┘
                               │
                               ▼
                    ┌───────────────────────┐
                    │   Dispatcher Layer    │
                    │ (HTTP Callback Now)   │
                    │ (Kafka-ready Design)  │
                    └──────────┬───────────┘
                               │
                               ▼
                    ┌───────────────────────┐
                    │ External Callback API │
                    │   (Webhook Receiver)  │
                    └───────────────────────┘

1. API Layer: Synchronously accepts deposit webhooks/events.
2. Database Transaction: Atomically saves the financial ledger (Transaction) AND a pending webhook notification (Outbox) in a single PostgreSQL transaction.
3. Background Worker: Asynchronously polls the Outbox table and dispatches HTTP calls.
4. Resilience: If the external webhook server is down, the worker utilizes Exponential Backoff with Jitter to retry later, eventually dead-lettering the event if it exceeds maximum retries.

🛠️ Technology Stack

• Runtime: Node.js, Express.js
• Language: TypeScript
• Database: PostgreSQL (Dockerized)
• ORM: Prisma
• Validation: Zod (Strict schema validation)
• Testing: Jest & Supertest (Isolated database integration tests)
• Logging: Winston (Structured JSON logging for ELK/Datadog)

📦 Project Setup

Prerequisites

• Docker & Docker Compose
• Node.js (v18+)

1. Start the Databases

The docker-compose.yml file provisions two strictly isolated PostgreSQL databases:

• Port 5433: Development Database
• Port 5434: Automated Testing Database

docker-compose up -d

2. Environment Variables

Copy the example environment files to configure both your development and testing ports:

cp .env.example .env
cp .env.test.example .env.test

3. Install & Migrate

Install dependencies and apply the Prisma schema to the development database:

npm install
npx prisma migrate deploy

4. Run the Server

Starts the Express API and the Outbox Polling Worker simultaneously.

npm run dev

The API will be available at http://localhost:3000.

🧪 Testing Strategy

This project features a robust, production-ready integration testing suite.

• Total Isolation: Tests run strictly against the 5434 test container. They will never touch or corrupt your development data.
• Automated Teardown: The test database is wiped globally before execution, ensuring a clean slate.
• Worker Simulation: Network failures are mocked to prove the exponential backoff, retry counting, and dead-letter queueing work flawlessly.

Run the test suite:

npm test

(After running tests, you can connect pgAdmin to localhost:5434 to visually inspect the exact lifecycle of successful, pending, and dead-lettered outbox events).

🔌 API Documentation & Validation Rules

The API is built with strict Zod validation. All bad requests return a structured 400 Bad Request payload detailing exactly which field failed and why.

1. Register a Wallet

POST /api/wallets

Creates a new wallet to track.

• Validation Rules:
  • address: String, Required. Must be exactly starting with 0x, min 10 chars, max 255 chars.

Success Payload (201 Created):

{
  "address": "0xYourWalletAddressHere1234567890"
}

2. Record a Deposit

POST /api/deposits

Records a deposit and triggers the background outbox worker.

• Validation Rules:
  • wallet_address: String, Required. Must start with 0x.
  • transaction_hash: String, Required. Must start with 0x.
  • amount: Number, Required. Must be strictly > 0.

Success Payload (201 Created):

{
  "wallet_address": "0xYourWalletAddressHere1234567890",
  "transaction_hash": "0xUniqueHash123",
  "amount": 1.5
}

• Idempotency (409 Conflict): Sending the exact same transaction_hash twice will yield a 409 Conflict to prevent double-spending.
• Not Found (404 Not Found): Attempting to deposit to a wallet that hasn't been registered yet will fail.

Tip: The requests.http file includes ready to run API requests along with sample responses for easy testing in VS Code.

🔍 Visualizing the Outbox with pgAdmin

Because the system uses Docker, you can visually watch the outbox worker process events in real-time.

1. Connect pgAdmin to localhost:5433 (Username: deposit_user, Password: deposit_pass).
2. Open the deposit_tracking database.
3. Query the outbox table.
4. Send a POST /api/deposits request.
5. Watch the table: You will see the event appear as PENDING, and then near-instantly transition to DONE as the worker picks it up!

🏗️ Clean Architecture & Separation of Concerns

This project is structured using Domain-Driven Design principles, ensuring strict separation of concerns and a highly maintainable codebase:

• Modular Features (src/modules/): Code is grouped by feature (Wallet, Deposit, Outbox) rather than technical type. Each module strictly isolates its Routes, Controllers, Services, and Validation Schemas.
• Business Logic Isolation: The Controllers handle HTTP parsing, while the Services contain pure business and database logic, completely unaware of Express.
• Validation Layer (src/validation/): Zod schemas provide a single source of truth for validation rules, rejecting bad data before it ever touches the business logic.
• Future-Proof Scalability (src/dispatcher/): The external webhook logic is abstracted behind an EventDispatcher interface. Currently implemented as an HttpDispatcher, this abstraction makes it trivial to swap to Kafka, RabbitMQ, or AWS SQS in the future for massive scalability without changing a single line of the core Deposit or Outbox logic.

📂 Project Structure

├── prisma/                 # Database schemas and migrations
├── src/
│   ├── db/                 # Prisma client instantiation
│   ├── dispatcher/         # Webhook dispatchers (Abstracted for Kafka readiness)
│   ├── logger/             # Winston structured logging
│   ├── middleware/         # Global Error Handlers & Request Validation
│   ├── modules/            # Domain Logic (Wallets, Deposits, Outbox)
│   │   ├── wallet/
│   │   └── deposit/
│   ├── validation/         # Zod schemas (Single source of truth)
│   ├── worker/             # Outbox polling and retry engine
│   ├── app.ts              # Server & Worker Bootstrap
│   └── server.ts           # Express App Configuration
├── tests/                  # Jest Integration & E2E Tests
└── requests.http           # VS Code REST Client test cases