Difftron

Fail the PR when new code isn't tested — in any language, in a few lines of YAML.

Difftron is a language-agnostic delta-coverage gate. It correlates your git diff with a standard coverage report (LCOV, Cobertura, or Go) and holds only the lines you changed to a threshold — so testing debt can't slip in behind a project-wide coverage number that barely moves.

Quickstart (GitHub Action)

Add this to .github/workflows/quality-gate.yml, pointing coverage at the file your test run produces:

name: Quality Gate
on: pull_request
permissions:
  contents: read
  pull-requests: write # lets Difftron post a results comment
jobs:
  delta-coverage:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0 # full history needed to diff against the base branch
      # ...run your tests and emit a coverage file (e.g. coverage/lcov.info)...
      - uses: swantron/difftron@v1
        with:
          coverage: coverage/lcov.info
          threshold: '80'

That's it: the PR fails if changed-line coverage drops below threshold, and a sticky comment shows exactly which new lines are untested. A full copy-paste example lives in examples/quickstart.yml.

Difftron is also a standalone CLI with analyze, ci, and health commands for richer multi-test-type aggregation, baseline tracking, and AI test-gap suggestions — see below.

---

Unlike traditional coverage tools that report on the entire project, Difftron zooms in on the deltas, ensuring that every new line of code is held to a high standard of quality before it hits production. It also provides a holistic view of testing health by aggregating coverage across multiple test types (unit, API, functional) and tracking baseline coverage to prevent false positives.

What Difftron Does

Difftron analyzes code changes and provides comprehensive testing health insights:

1. Delta Coverage Analysis: Analyzes only the code that changed, not the entire codebase
2. Baseline Tracking: Compares current coverage against baseline to detect regressions
3. Multi-Test-Type Aggregation: Combines coverage from unit, API, and functional tests
4. New vs Modified File Detection: Separately tracks coverage for new files vs modified files
5. Holistic Health Reporting: Provides actionable insights and recommendations
6. Regression Prevention: Flags when coverage drops below baseline
7. Test Gap Detection: Identifies when code is only covered by one test type

Key Capabilities

• Accurate Assessment: Only evaluates changed code with baseline context, preventing false positives from untested legacy code
• Comprehensive Coverage: Aggregates coverage across all test types (unit, API, functional, integration, E2E)
• Actionable Insights: Generates clear recommendations with priorities (critical, high, medium, low)
• AI-Friendly Output: Provides structured JSON, Markdown, and text formats for both humans and AI agents
• CI/CD Integration: Ready-to-use templates for GitHub Actions and GitLab CI

The Problem

In large repositories, a developer can introduce 100 lines of untested logic, and the overall project coverage might only drop by 0.01%. This makes it easy for "testing debt" to accumulate unnoticed.

The Solution

Difftron parses the git diff to identify the specific line numbers modified in a PR. It then "intersects" this data with your coverage reports to identify exactly which new lines are untested. Finally, it uses Gemini AI to suggest the missing test cases.

---

Architecture

Component	Responsibility
Hunk Engine	Parses git diff output to map "hunks" to absolute line numbers. Detects new vs modified files.
Coverage Engine	Ingests LCOV/Go coverage files to build an in-memory map of File -> Line -> HitCount. Supports multiple formats.
Health Analyzer	Aggregates coverage across test types, tracks baseline coverage, and generates holistic health reports.
Analyzer	Intersects diff hunks with coverage data, calculates coverage percentages, and identifies gaps.
Risk Engine	Cross-references coverage with Git Churn (frequency of file changes) to flag high-risk gaps. (Planned)
Gemini Integration	Provides "Agentic" test generation by sending uncovered hunks to the AI for boilerplate creation. (Planned)

---

Implementation Highlights (Go)

1. The Hunk-to-Line Mapper

This core logic translates relative diff offsets into absolute line numbers that match your coverage report.

// ParseGitDiff identifies absolute line numbers of added/modified code
func ParseGitDiff(diffOutput string) map[string]map[int]bool {
	changes := make(map[string]map[int]bool)
	scanner := bufio.NewScanner(strings.NewReader(diffOutput))
	var currentFile string
	var currentLine int

	for scanner.Scan() {
		line := scanner.Text()

		if strings.HasPrefix(line, "+++ b/") {
			currentFile = strings.TrimPrefix(line, "+++ b/")
			changes[currentFile] = make(map[int]bool)
			continue
		}

		// Header Example: @@ -10,5 +15,7 @@
		if strings.HasPrefix(line, "@@") {
			parts := strings.Split(line, " ")
			newFilePart := strings.Split(parts[2], ",") // e.g., +15
			startLine, _ := strconv.Atoi(newFilePart[0][1:])
			currentLine = startLine
			continue
		}

		if strings.HasPrefix(line, "+") && !strings.HasPrefix(line, "+++") {
			changes[currentFile][currentLine] = true
			currentLine++
		} else if !strings.HasPrefix(line, "-") {
			currentLine++ // Context line
		}
	}
	return changes
}

2. The LCOV Coverage Parser

A high-performance scanner to ingest standardized coverage data.

// ParseLCOV reads .info files into a File -> Line -> Hits map
func ParseLCOV(filePath string) (map[string]map[int]int, error) {
	file, _ := os.Open(filePath)
	defer file.Close()

	reports := make(map[string]map[int]int)
	var currentFile string
	scanner := bufio.NewScanner(file)

	for scanner.Scan() {
		line := scanner.Text()
		if strings.HasPrefix(line, "SF:") {
			currentFile = line[3:]
			reports[currentFile] = make(map[int]int)
		} else if strings.HasPrefix(line, "DA:") {
			data := strings.Split(line[3:], ",")
			lineNum, _ := strconv.Atoi(data[0])
			hits, _ := strconv.Atoi(data[1])
			reports[currentFile][lineNum] = hits
		}
	}
	return reports, nil
}

---

Key Features

• Universal Language Support: Supports LCOV, Cobertura XML, and Go coverage formats (Go, TS, Java, Python, C++, etc.)
• Automatic Format Detection: Automatically detects coverage format - no need to specify
• Risk Heatmaps: Uses Git Churn data to identify "Hot Spot" files. Low coverage in a high-churn file triggers a CRITICAL alert.
• AI Test Generation: Don't just report the gap—fix it. Difftron generates _test.go or .test.ts snippets for missing paths.
• CI/CD Native: Designed to run as a GitHub Action or GitLab CI Job, posting rich Markdown reports directly to your PR/MR.
• Dogfooding Ready: Use difftron on itself! Built-in support for analyzing your own code changes.

---

Development

Prerequisites

• Go 1.23 or later
• Git (for testing git diff functionality)

Setup

# Clone the repository
git clone <repo-url>
cd difftron

# Initialize Go module (if not already done)
go mod init github.com/swantron/difftron

# Install dependencies
go mod tidy

# Run tests
go test ./...

# Build the CLI
go build -o bin/difftron ./cmd/difftron

# Or use the task runner
go run scripts/task.go build
go run scripts/task.go test

Task Runner

Difftron uses a Go-based task runner instead of a Makefile. This keeps the project 100% Go and avoids external dependencies.

# Build the binary
go run scripts/task.go build

# Run all tests
go run scripts/task.go test

# Run tests with coverage
go run scripts/task.go test-coverage

# Run integration tests with fixtures
go run scripts/task.go test-integration

# Dogfood: Analyze your own changes
go run scripts/task.go dogfood

# Install the binary
go run scripts/task.go install

# Format code
go run scripts/task.go fmt

# Clean build artifacts
go run scripts/task.go clean

# Run the CLI locally
go run scripts/task.go run analyze --coverage coverage.info

Testing

# Run all unit tests
go run scripts/task.go test

# Run integration test with fixtures
go run scripts/task.go test-integration

# Run tests with coverage report
go run scripts/task.go test-coverage

# Or use go test directly
go test ./...
go test -v ./...
go test ./internal/hunk/...

See TESTING.md for comprehensive testing guide.

---

Dogfooding

Difftron can analyze itself! Use it to ensure your own code changes meet coverage standards.

# Analyze your last commit
go run scripts/task.go dogfood

# Analyze specific commit range
go run scripts/task.go dogfood HEAD~5 HEAD

See DOGFOOD.md for detailed dogfooding guide and CI/CD integration.

CI/CD Integration

GitHub Actions

The included .github/workflows/difftron.yml workflow provides:

• Automatic PR comments with coverage analysis
• Gating: Blocks merges if coverage threshold not met
• Configurable threshold: Set via workflow input or PR labels
• Separate thresholds: Different thresholds for new vs modified files

Reusable Template

Use the reusable workflow template in your own workflows:

jobs:
  difftron:
    uses: ./.github/workflows/difftron-template.yml
    with:
      threshold: '80'
      threshold_new: '90'
      threshold_modified: '75'
      use_health_command: 'true'

Configuration

Default threshold: 80% (configurable in workflow file)

Per-PR threshold: Add label coverage-threshold:90 to set 90% threshold

Manual trigger: Set threshold when manually running the workflow

Dogfooding: .github/workflows/dogfood.yml runs difftron on itself

See .github/workflows/README.md for detailed configuration options.

GitLab CI

Difftron integrates seamlessly with GitLab CI/CD pipelines. See GITLAB_CI.md for:

• Complete GitLab CI configuration examples
• Security-friendly artifact distribution options
• Merge request comment integration
• Artifact caching strategies

Quick GitLab CI Example

difftron-analysis:
  stage: test
  image: golang:1.21
  script:
    - go test -coverprofile=coverage.out ./...
    - ./bin/difftron ci --threshold 80 coverage.out
  artifacts:
    reports:
      coverage_report:
        coverage_format: cobertura
        path: coverage.out
  only:
    - merge_requests
    - main

Usage in CI

# Basic usage (auto-detects CI environment)
difftron ci coverage.out

# Explicit refs
difftron ci --base HEAD~1 --head HEAD coverage.out

# Custom threshold
difftron ci --threshold 90 coverage.out

# Holistic health analysis with multiple test types
difftron health \
  --unit-coverage unit-coverage.out \
  --api-coverage api-coverage.info \
  --functional-coverage functional-coverage.info \
  --threshold 80 \
  --threshold-new 90 \
  --threshold-modified 75 \
  --output json > health-report.json

# With baseline comparison
difftron health \
  --unit-coverage unit-coverage.out \
  --baseline-unit-coverage baseline-coverage.out \
  --threshold 80 \
  --output markdown

# Post comment on PR/MR
difftron health \
  --unit-coverage unit-coverage.out \
  --threshold 80 \
  --comment-pr  # For GitHub PRs
  # or
  --comment-mr  # For GitLab MRs

---

Implementation Plan

Phase 1: Foundation (v0.1) - CLI Core

Goal: Build the core CLI with git diff parsing and LCOV coverage analysis.

Tasks:

1. Project Setup

   • Initialize Go module (go mod init)
   • Set up project structure (cmd/, internal/, pkg/)
   • Add CLI framework (Cobra)
   • Create Go-based task runner for common tasks

2. Hunk Engine (internal/hunk/)

   • Implement ParseGitDiff() to extract changed lines
   • Handle unified diff format parsing
   • Support both staged and unstaged diffs
   • Map relative diff positions to absolute line numbers
   • Unit tests with sample diff outputs

3. Coverage Engine (internal/coverage/)

   • Implement ParseLCOV() for .info files
   • Build File -> Line -> HitCount data structure
   • Handle file path normalization (relative vs absolute)
   • Error handling for malformed LCOV files
   • Unit tests with sample LCOV data

4. Core Analysis (internal/analyzer/)

   • Intersect diff hunks with coverage data
   • Calculate coverage percentage for changed lines
   • Identify uncovered lines in diffs
   • Generate coverage report summary

5. CLI Interface (cmd/difftron/)

   • difftron analyze command
   • Flags: --diff, --coverage, --threshold, --output
   • JSON and human-readable output formats
   • Exit codes: 0 (pass), 1 (fail), 2 (error)

Deliverables: Working CLI that can analyze a git diff against LCOV coverage.

---

Phase 2: Risk Scoring (v0.2)

Goal: Add git churn analysis and risk-based prioritization.

Tasks:

1. Git Churn Analysis (internal/churn/)

   • Calculate file change frequency
   • Analyze commit history for hot spots
   • Weight recent changes more heavily
   • Cache churn data for performance

2. Risk Engine (internal/risk/)

   • Combine coverage gaps with churn scores
   • Assign risk levels: LOW, MEDIUM, HIGH, CRITICAL
   • Prioritize uncovered lines in high-churn files
   • Generate risk heatmap report

3. Enhanced Reporting

   • Color-coded output (red/yellow/green)
   • Risk-based sorting of issues
   • File-level and line-level risk scores

Deliverables: Risk-aware coverage analysis with prioritized alerts.

---

Phase 3: CI/CD Integration (v0.3)

Goal: Integrate with GitHub Actions and GitLab CI.

Tasks:

1. GitHub Action (.github/workflows/)

   • Action YAML template
   • Auto-detect coverage files
   • Post PR comments with coverage report
   • Support for pull_request events

2. GitLab CI (.gitlab-ci.yml)

   • CI job template
   • Merge request comment integration
   • Artifact generation

3. Comment Formatting

   • Markdown report generation
   • Collapsible sections for large diffs
   • Links to uncovered lines
   • Summary statistics

Deliverables: Ready-to-use CI/CD templates for GitHub and GitLab.

---

Phase 4: AI Test Generation (v0.4)

Goal: Generate test code suggestions using Gemini AI.

Tasks:

1. Gemini Integration (internal/ai/)

   • Google AI SDK integration
   • Prompt engineering for test generation
   • Context-aware code suggestions
   • Support for multiple languages (Go, TypeScript, Python, etc.)

2. Test Generation Engine

   • Extract uncovered code snippets
   • Generate language-specific test templates
   • Include imports and setup code
   • Format output as code blocks

3. CLI Enhancement

   • difftron generate command
   • --ai-provider flag (default: gemini)
   • --language flag for test generation
   • --output-file for saving generated tests

Deliverables: AI-powered test generation for uncovered code paths.

---

Project Structure

difftron/
├── cmd/
│   └── difftron/
│       └── main.go              # CLI entry point
├── internal/
│   ├── hunk/
│   │   ├── parser.go            # Git diff parsing
│   │   └── parser_test.go
│   ├── coverage/
│   │   ├── lcov.go              # LCOV parser
│   │   ├── gocov.go             # Go coverage parser
│   │   ├── cobertura.go         # Cobertura XML parser
│   │   ├── cobertura_test.go    # Cobertura tests
│   │   └── coverage_test.go
│   ├── analyzer/
│   │   ├── analyzer.go          # Core analysis logic
│   │   └── analyzer_test.go
│   ├── churn/                   # Git churn calculation (v0.2, planned)
│   ├── risk/                    # Risk scoring (v0.2, planned)
│   └── ai/                      # Gemini integration (v0.4, planned)
├── pkg/
│   └── report/
│       └── formatter.go         # Report formatting
├── scripts/
│   ├── task.go                  # Go-based task runner
│   ├── dogfood.sh               # Dogfooding script
│   └── generate-coverage.sh     # Coverage generation helper
├── testdata/
│   └── fixtures/                # Test fixtures
├── .github/
│   └── workflows/
│       └── difftron.yml         # GitHub Action (v0.3)
├── .gitlab-ci.yml               # GitLab CI template (v0.3)
├── go.mod
├── go.sum
├── README.md
├── TESTING.md                   # Testing guide
├── DOGFOOD.md                   # Dogfooding guide
└── LICENSE

---

Quick Start (After v0.1)

# Install
go install github.com/swantron/difftron/cmd/difftron@latest

# Analyze current diff against coverage (auto-detects format)
difftron analyze --coverage coverage.info    # LCOV format
difftron analyze --coverage coverage.xml      # Cobertura XML format
difftron analyze --coverage coverage.out      # Go coverage format

# Analyze specific diff
git diff main...feature-branch | difftron analyze --coverage coverage.info

# Set coverage threshold
difftron analyze --coverage coverage.xml --threshold 80

# Separate thresholds for new vs modified files
difftron analyze --coverage coverage.xml \
  --threshold 80 \
  --threshold-new 90 \
  --threshold-modified 75

# Generate JSON or Markdown report
difftron analyze --coverage coverage.xml --output json > report.json
difftron analyze --coverage coverage.xml --output markdown > report.md

---

Roadmap Summary

• v0.1: CLI Core (Diff + LCOV parsing). COMPLETE
• v0.1+: Health command, line-by-line coverage, separate thresholds, CI templates. COMPLETE
• v0.2: Risk Scoring (Git Churn + Complexity).
• v0.3: Enhanced PR/MR commenter with full API integration.
• v0.4: Gemini-powered Test Generation.

---

Current Status (v0.1+)

Phase 1 is complete! The core CLI functionality is implemented and tested:

Implemented Features:

• Git diff parsing (Hunk Engine) with new/modified file detection
• Multiple coverage formats: LCOV, Cobertura XML, and Go coverage format support
• Line-by-line Go coverage parsing: Parses .out files directly (mode: set/count) for accurate coverage
• Core analysis engine (intersects diffs with coverage)
• Baseline coverage tracking to prevent false positives
• Holistic health analysis with multi-test-type aggregation
• CLI interface with analyze, ci, and health commands
• Text, JSON, and Markdown output formats
• Separate thresholds: Different thresholds for new vs modified files
• Enhanced path normalization: Cross-platform support with repo-root rebasing
• Coverage threshold checking
• Comprehensive test coverage (91%+)
• Dogfooding support (analyze your own code)
• GitHub Actions and GitLab CI templates
• PR/MR commenter support (GitHub/GitLab)

New in Recent Updates:

• Baseline Coverage Tracking: Compares current coverage against baseline to detect regressions
• New vs Modified File Detection: Separately tracks coverage for new files vs modified files
• Holistic Health Reporting: Aggregates coverage across unit, API, and functional tests
• Actionable Insights: Generates prioritized recommendations
• Security-Friendly Distribution: Multiple options for enterprise environments (vendor binary, build from source, internal registry)

Usage Example:

# Analyze current working directory changes
difftron analyze --coverage coverage.info

# Analyze with custom threshold
difftron analyze --coverage coverage.info --threshold 90

# Analyze specific git diff range
difftron analyze --coverage coverage.info --base main --head feature-branch

# JSON output
difftron analyze --coverage coverage.info --output json

# Use a pre-generated diff file
git diff main...feature > diff.patch
difftron analyze --coverage coverage.info --diff diff.patch

# Test with included fixtures
difftron analyze --coverage testdata/fixtures/tronswan-coverage.info --diff testdata/fixtures/sample.diff

# Analyze Cobertura XML format (Python, Java, JavaScript, etc.)
pytest --cov=src --cov-report=xml:cobertura.xml
difftron analyze --coverage cobertura.xml --threshold 80

# Dogfood: Analyze your own changes
go run scripts/task.go dogfood

Documentation

• README.md: This file - overview and quick start
• HEALTH_COMMAND.md: Comprehensive health command documentation
• HOLISTIC_HEALTH.md: Holistic testing health analysis documentation
• BASELINE_COVERAGE.md: Baseline coverage tracking explanation
• GITLAB_CI.md: Complete GitLab CI integration guide with security-friendly artifact distribution
• ARTIFACT_DISTRIBUTION.md: Quick reference for artifact distribution options
• COBERTURA.md: Cobertura XML format support and usage guide
• BUILD.md: Build instructions and version embedding
• TESTING.md: Testing guide and examples
• DOGFOOD.md: Using Difftron on itself
• SECURITY.md: Security policy and best practices for enterprise use
• CI_TROUBLESHOOTING.md: CI/CD troubleshooting guide
• CHANGELOG.md: Version history and changes

Next Steps:

• Implement Risk Engine (git churn analysis) for v0.2
• Add Gemini AI integration for test generation
• Enhanced format detection and error messages
• Enhanced GitLab MR comment integration

---