Skip to content

UCSB-AMPLab/digitization-toolkit-software

Repository files navigation

Digitization Toolkit

An open-source, modular digitization toolkit designed for low-cost, high-quality scanning using Raspberry Pi Compute Module 4 and 64MP ArduCam cameras. Built for use in low-resource environments and community archives.

Goals

  • Empower small institutions and communities to digitize archival materials.
  • Provide an easy-to-use web interface for image capture, metadata entry, and export.
  • Ensure long-term reproducibility through open standards (BagIt, CSV, TIFF).
  • Prioritize modularity and hardware independence for future-proofing.

Tech Stack

  • Hardware: Raspberry Pi CM4 + CM4IO, Raspberry Pi 5 (dual camera embedded), 2x 64MP ArduCam autofocus cameras, 2 x Cannon EOS Rebel T7 (GPhoto compatible cameras)
  • Backend: Python, FastAPI
  • Frontend: Svelte
  • Database: PostgreSQL

Status

Project currently in development. Kick-off: September 2025 Alpha prototype planned for deployment at SBMAL in June 2026.

Release History

See CHANGELOG.md for version history and release notes.


Quick Start

Development (any machine, no cameras)

# One-command startup (all services in Docker)
./scripts/start-dev.sh

Or manually:

docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile with-backend up

Production (Raspberry Pi with cameras)

# Build the frontend image first (requires internet — do this at home/office):
docker compose build

# Then take it offline and run:
./scripts/start.sh

Or manually:

# Start database and frontend
docker compose up -d

# Start backend with pixi
cd backend && pixi run dev

Access (production): http://localhost:3000 (frontend) | http://localhost:8000/docs (API) Access (dev): http://localhost:5173 (frontend, Vite) | http://localhost:8000/docs (API)

Note: The native backend is required for camera access due to Raspberry Pi-specific libraries (libcamera, picamera2).

📦 Backend Dependency Management

The backend uses pixi for dependency management:

# First time setup
cd backend
pixi install
pixi run setup-camera-link  # Raspberry Pi only

# Start backend
pixi run dev

Production Distribution (SD Card Imaging)

The toolkit is designed to ship as a pre-flashed SD card. The end user only needs to insert the card, power on the Pi, and the application starts automatically — no internet connection required.

Building the golden SD card (requires internet, done once)

# 1. Clone the repository onto a fresh Raspberry Pi OS installation
git clone --recurse-submodules https://github.com/UCSB-AMPLab/digitization-toolkit.git ~/dtk
cd ~/dtk

# 2. Run the one-time provisioning script (internet required here)
#    Builds Docker images, installs pixi env, applies DB migrations
sudo ./scripts/setup.sh

# 3. Install and enable the systemd service (auto-start on boot)
sudo ./scripts/install-service.sh
sudo systemctl start dtk

# 4. Verify the app is running
curl http://localhost:8000/health   # → {"status":"ok"}
curl -I http://localhost:3000       # → HTTP 200

Creating an SD card image

# On the Pi — shut down cleanly
sudo systemctl stop dtk
sudo poweroff

# On a host machine with the SD card inserted (replace sdX with your device)
sudo dd if=/dev/sdX of=dtk-$(date +%Y%m%d).img bs=4M status=progress
# Compress for storage/transfer
xz -T0 dtk-$(date +%Y%m%d).img

Flashing for end users

Use Raspberry Pi Imager (or dd) to write the .img to a new SD card.
The application starts automatically on boot — no configuration needed.

What's self-contained after provisioning

Component Stored at Requires internet after setup?
Frontend (SvelteKit) Docker image in /var/lib/docker/ No
PostgreSQL Docker image in /var/lib/docker/ No
Backend Python env backend/.pixi/ No
camera libraries /usr/lib/python3/dist-packages/ No (system OS)
Application data /var/lib/dtk/ No

Setup

This repository uses Git submodules for the frontend and backend code.
When cloning, make sure to fetch submodules as well:

# Clone with submodules
git clone --recurse-submodules [email protected]:UCSB-AMPLab/digitization-toolkit.git

If you already cloned without --recurse-submodules, you can initialize and update submodules manually:

git submodule update --init --recursive

To pull the latest changes for submodules after updates:

git submodule update --remote --merge

Development Documentation

See the wiki for detailed developer guides and API references.

License

MIT

About

Core repository for the Digitization Toolkit Software

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors