diff --git a/.gitignore b/.gitignore index a849f92..9bfbe57 100644 --- a/.gitignore +++ b/.gitignore @@ -11,3 +11,4 @@ default.profraw *.dia *.swiftdeps .vscode/ +.build-unchecked/ diff --git a/CHANGELOG.md b/CHANGELOG.md index b04274a..94f2022 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,78 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.9.0] - 2026-06-11 + +### Added +- **Restart-interval support (DRI/RSTm)**: opt-in `Configuration.restartInterval` + (and `jpegls encode --restart-interval N`) writes a DRI segment and emits + RSTm markers every N lines for lossless non-interleaved scans. Per + ITU-T.87, every interval restarts coding as at scan start, so intervals + encode **and** decode in parallel across cores (4096×4096 16-bit: encode + 0.84 s → 0.31 s, decode 0.69 s → 0.24 s wall, ~+0.03 % size) +- `jpegls batch encode`/`batch decode` implemented (they were + "not yet implemented" stubs on top of the existing worker pool); batch + encode output is byte-identical to serial encodes +- Restart-interval test suite (round-trips, marker cycling, DRI parsing, + determinism, multi-component scans, configuration validation) + +### Changed +- **Hot-path performance rewrite** (encoded bitstreams remain byte-identical; + verified against a 44-artifact golden gate and the full conformance suite): + - Bitstream writer: `[UInt8]` backing with a 64-bit accumulator (was + per-byte Foundation `Data` appends with a 32-bit accumulator) + - Bitstream reader: 64-bit window over `[UInt8]` with multi-byte refill and + `leadingZeroBitCount` unary decode (was per-bit reads via `Data` + subscripts) + - Encoder scan loops: per-pixel `Dictionary` neighbour lookups hoisted out; + gradients quantised once per pixel; lossless non-interleaved scans run + over a flat contiguous `UInt16` plane through unsafe buffers + - Decoder scan loops: same flat-plane treatment; decoder gained the + encoder's init-time gradient quantisation table + - Context model: A/B/C/N packed into one record array (one load + one store + per pixel); bias C, Golomb k, and the k = 0 correction come from a single + record read + - Run detection: unrolled exact-equality scan for lossless + - Parser records scan-body byte ranges, removing the decoder's second + full-file marker walk; decoder results skip the redundant O(W·H) + re-validation pass (with the ITU-T.87 C.2.4.1.1 MAXVAL ≤ 2^P−1 check + added at parse time) + - Measured on real radiology DICOM (107 frames, 6 modalities, lossless): + encode 27 → 80 MB/s, decode 41 → 84 MB/s aggregate; synthetic 16-bit + 2048²: encode 37 → 98 MB/s, decode 57 → 104 MB/s +- CLI raw output accumulates in `[UInt8]` instead of per-byte `Data.append` + (33 MB decode-to-raw: 2.0 s → 0.7 s) + +### Fixed +- **Restart-marker streams decode correctly**: the parser and scan extractor + previously treated RST markers (FFD0–FFD7) as scan terminators, truncating + conformant streams at the first restart marker +- Encoder no longer allocates and zeroes a full-frame reconstruction buffer + for lossless scans (it is only read for near-lossless); ~32 MB transient + saved per 2048² scan, ~136 MB for a 17 MP mammography frame +- Robustness fixes from the branch security/correctness review: + - `decode(_:)` rebases `Data` slices with non-zero `startIndex` (slices + previously mis-sliced scan ranges — silent wrong pixels or a trap) + - Encoder rejects sub-sampled component planes up front (previously an + out-of-bounds read in release builds) and preset MAXVAL > 2^P−1 + (previously a trap or an unparseable stream) + - Decoder requires every frame component to have a scan, applies the DRI + in effect at each SOS (T.81 B.2.4.4 per-scan semantics), accepts + interleaved streams whose DRI ≥ height (no actual markers), and rejects + stray RSTm markers in scans without an active restart interval + (previously absorbed silently as entropy data) + - Parser rejects undersized LSE segments and dimension products that + overflow (crafted LSE type-4), both previously uncatchable traps + +### Removed +- **The entire `Platform/` acceleration layer** (Metal, Vulkan, Accelerate, + ARM64/x86-64 SIMD wrappers, ~4,400 lines), plus `JPEGLSBufferPool`, + `JPEGLSCacheFriendlyBuffer`, and `JPEGLSTileProcessor`: profiling showed + none of it was invoked on the codec hot path, and the GPU kernels could not + produce conformant streams (JPEG-LS entropy coding is sequential by + construction). Restart intervals are the supported parallelism mechanism. + Docs and README updated to describe only what ships. + ## [0.8.0] - 2026-05-30 ### Added @@ -319,7 +391,9 @@ See [RELEASE_NOTES_TEMPLATE.md](docs/RELEASE_NOTES_TEMPLATE.md) for the release - **0.6.0** - Memory optimisation (buffer pooling, tile processing) - **0.7.0** - CLI tool (info, verify, encode, decode, batch, completion) - **0.8.0** - Validation & conformance (CharLS, benchmarks, edge cases) +- **0.9.0** - Hot-path performance rewrite, restart-interval parallelism, acceleration-layer removal - **1.0.0** - Planned stable release -[Unreleased]: https://github.com/Raster-Lab/JLSwift/compare/v0.8.0...HEAD +[Unreleased]: https://github.com/Raster-Lab/JLSwift/compare/v0.9.0...HEAD +[0.9.0]: https://github.com/Raster-Lab/JLSwift/releases/tag/v0.9.0 [0.8.0]: https://github.com/Raster-Lab/JLSwift/releases/tag/v0.8.0 diff --git a/Package.swift b/Package.swift index 7c8307f..d6382d9 100644 --- a/Package.swift +++ b/Package.swift @@ -24,9 +24,6 @@ let package = Package( targets: [ .target( name: "JPEGLS", - resources: [ - .process("Platform/Metal/JPEGLSShaders.metal") - ], swiftSettings: [ .swiftLanguageMode(.v6) ] diff --git a/README.md b/README.md index e438b94..9bd12eb 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # JLSwift -A native Swift implementation of **JPEG-LS** (ISO/IEC 14495-1:1999 / ITU-T.87) compression. Usable as a standalone general-purpose codec and optimised for Apple Silicon with hardware acceleration support. Fully compatible with DICOM medical imaging workflows. +A native Swift implementation of **JPEG-LS** (ISO/IEC 14495-1:1999 / ITU-T.87) compression with a heavily optimised pure-Swift hot path. Fully compatible with DICOM medical imaging workflows. [![CI](https://github.com/Raster-Lab/JLSwift/actions/workflows/ci.yml/badge.svg)](https://github.com/Raster-Lab/JLSwift/actions/workflows/ci.yml) @@ -9,7 +9,7 @@ A native Swift implementation of **JPEG-LS** (ISO/IEC 14495-1:1999 / ITU-T.87) c JLSwift is a pure Swift JPEG-LS compression library with no DICOM dependencies. It can be used in any project that requires lossless or near-lossless image compression — no DICOM knowledge is needed. The library is also designed for seamless integration with DICOMkit and other medical-imaging frameworks. Key attributes: - **Type Safety**: Leverages Swift 6.2+ concurrency and type system -- **Performance**: Optimised implementations with support for hardware acceleration +- **Performance**: Flat-buffer scan loops, 64-bit bitstream I/O, packed context records, and optional restart-interval parallelism — measured, not aspirational (see [PERFORMANCE_TUNING.md](docs/PERFORMANCE_TUNING.md)) - **Reliability**: Comprehensive test coverage exceeding 95% for all modules - **DICOM Aware, DICOM Independent**: Full support for DICOM transfer syntaxes; no DICOM runtime dependency @@ -33,7 +33,7 @@ Add JLSwift as a dependency in your `Package.swift`: ```swift dependencies: [ - .package(url: "https://github.com/Raster-Lab/JLSwift.git", from: "0.8.0") + .package(url: "https://github.com/Raster-Lab/JLSwift.git", from: "0.9.0") ] ``` @@ -66,10 +66,7 @@ JPEG-LS is a lossless/near-lossless compression standard specifically designed f | 4.2 | Regular Mode Decoding | ✅ Complete | 96.90% | | 4.3 | Run Mode Decoding | ✅ Complete | 100.00% | | 4.4 | Multi-Component Decoding | ✅ Complete | 92.10% | -| 5.1 | ARM NEON / SIMD Optimisation | ✅ Complete | 100.00% | -| 5.2 | Apple Accelerate Integration | ✅ Complete | 100.00% | -| 5.3 | Metal GPU Acceleration | ✅ Complete | 100.00% | -| 5.4 | Memory Optimisation | ✅ Complete | 100.00% | +| 5.1–5.4 | Platform acceleration layer (NEON/Accelerate/Metal) | 🗑 Removed | — | | 7.3 | CLI Argument Parsing Tests | ✅ Complete | N/A* | | 8.1 | CharLS Reference Integration | ⏳ In Progress | 100.00% | | 8.4 | Edge Cases & Robustness | ✅ Complete | 100.00% | @@ -80,9 +77,7 @@ JPEG-LS is a lossless/near-lossless compression standard specifically designed f | 12.1 | CharLS Decode Interoperability | ⏳ In Progress | — | | 12.2 | CharLS Encode Interoperability | ✅ Complete | 100.00% | | 12.3 | Round-Trip Interoperability (JLSwift) | ⏳ In Progress | 100.00% | -| 13.1 | ARM Neon Optimisation Audit & Enhancement | ✅ Complete | 100.00% | -| 13.2 | Accelerate Framework Deep Integration | ✅ Complete | 100.00% | -| 13.3 | Apple Silicon Memory Architecture Optimisation | ✅ Complete | 100.00% | +| 13.1–13.3 | Platform-specific optimisation layers | 🗑 Removed | — | **Overall Project Coverage: 95.80%** (exceeds 95% threshold) @@ -90,7 +85,7 @@ JPEG-LS is a lossless/near-lossless compression standard specifically designed f *CLI executable target not included in coverage metrics (Swift Package Manager limitation), but validation logic thoroughly tested with 60 comprehensive tests. -**Note**: Coverage may vary slightly by platform due to conditional compilation of platform-specific optimisations (ARM64, Accelerate framework). The reported coverage is measured on Linux x86_64. +**Note**: The platform acceleration layer (ARM NEON wrappers, Accelerate, Metal, Vulkan, x86-64) was removed in 0.9.0: profiling showed none of it was wired into the codec hot path, the GPU kernels could not produce conformant streams (the entropy stage is inherently sequential), and the pure-Swift hot path now outperforms what that layer ever measured. Phase rows above are kept for history. ### Key Features @@ -98,11 +93,9 @@ JPEG-LS is a lossless/near-lossless compression standard specifically designed f |---------|-------------| | **Native Swift** | Pure Swift implementation with no external C dependencies | | **Swift 6.2 Concurrency** | Explicit `.swiftLanguageMode(.v6)` in Package.swift; all shared types are `Sendable`; batch processing uses `withTaskGroup` structured concurrency | -| **Apple Silicon Optimised** | ARM NEON/SIMD acceleration using Swift SIMD types: SIMD8 run-length detection, SIMD8 byte-stuffing scan, and CLZ-based Golomb-Rice parameter computation | -| **Intel x86-64 Optimised** | SSE/AVX SIMD acceleration: SIMD8 run-length detection, SIMD8 byte-stuffing scan, BSR/LZCNT-based Golomb-Rice parameter computation, Intel-tuned cache parameters, and tile-size optimisation | -| **Hardware Acceleration** | Apple Accelerate framework (vDSP) for batch gradient computation, absolute prediction-error accumulation, context-state updates, vImage planar↔interleaved conversion, and vectorised HP1/HP2/HP3 colour transforms | -| **Metal GPU Acceleration** | Optional GPU acceleration for large images (macOS 10.13+, iOS 11+) | -| **Memory Optimised** | Cache-line–aligned context arrays, L1-cache–tuned tile sizes, `UnifiedMemoryBufferPool` for Apple Silicon unified memory, memory-mapped I/O via `mmap`, and prefetch hints for sequential access patterns | +| **Optimised Hot Path** | Flat contiguous pixel planes scanned through unsafe buffers, init-time gradient quantisation tables, 64-bit bitstream reader/writer with `clz` unary decode, packed per-context statistics records, and unrolled exact-equality run scanning | +| **Restart-Interval Parallelism** | Opt-in DRI/RSTm restart markers (`Configuration.restartInterval`): standards-compliant intra-image parallel encode *and* decode for large frames at a ~0.03 % size cost | +| **Memory Optimised** | One UInt16 plane per scan (half the bandwidth of boxed rows), no per-pixel allocations, and lossless encode skips reconstruction tracking entirely | | **DICOM Compatible** | Full support for DICOM transfer syntaxes | | **Multi-Component Support** | Full RGB and greyscale encoding with all interleaving modes | | **Interleaving Modes** | None (separate scans), Line-interleaved, Sample-interleaved | @@ -137,66 +130,32 @@ JPEGLS/ │ ├── NearLosslessEncoder # Near-lossless encoding with NEAR parameter │ ├── MultiComponentEncoder # Multi-component & interleaving orchestration │ └── PixelBuffer # Component-aware pixel access with neighbors -├── Platform/ # Platform-specific optimizations -│ ├── Accelerate/ # Apple Accelerate framework (vDSP batch operations) -│ ├── ARM64/ # Apple Silicon / ARM NEON code -│ └── x86_64/ # x86-64 specific code (removable) -│ ├── X86_64Accelerator # SSE/AVX SIMD accelerator -│ └── IntelMemoryOptimizer # Intel cache/memory optimisation -└── PlatformProtocols # Protocol-based platform abstraction +└── JPEGLS / JPEGLSEncoder / JPEGLSDecoder # Public API surface ``` -### Memory Optimisation Features +### Performance Architecture -JLSwift includes comprehensive memory optimisation features for handling large medical images: +The codec hot path is deliberately plain, fast Swift — measured against real +radiology DICOM data rather than synthetic microbenchmarks: -#### Buffer Pooling (`JPEGLSBufferPool`) -- Thread-safe buffer reuse to reduce allocation overhead -- Supports multiple buffer types (context arrays, pixel data, bitstream) -- Automatic cleanup of expired buffers -- Shared global pool available via `sharedBufferPool` - -#### Tile-Based Processing (`JPEGLSTileProcessor`) -- Divides large images into manageable tiles -- Configurable tile size and overlap for boundary handling -- Memory savings estimation for large images -- Enables processing of images larger than available memory - -#### Cache-Friendly Data Layout (`JPEGLSCacheFriendlyBuffer`) -- Contiguous memory layout in row-major order -- Optimised neighbour access patterns for CPU cache efficiency -- Batch row access for vectorised operations -- Compatible with existing encoder/decoder interfaces - -**Example Usage:** -```swift -// Create a tile processor for a large image -let processor = JPEGLSTileProcessor( - imageWidth: 8192, - imageHeight: 8192, - configuration: TileConfiguration(tileWidth: 512, tileHeight: 512, overlap: 4) -) - -// Calculate tiles with overlap for boundary handling -let tiles = processor.calculateTilesWithOverlap() - -// Estimate memory savings -let savings = processor.estimateMemorySavings(bytesPerPixel: 2) -print("Memory reduction: \(savings * 100)%") - -// Use buffer pooling for context arrays -let contextBuffer = sharedBufferPool.acquire(type: .contextArrays, size: 365) -defer { sharedBufferPool.release(contextBuffer, type: .contextArrays) } -``` +- **Flat scan planes**: each scan encodes/decodes over one contiguous + `UInt16` plane through an unsafe buffer scoped to the whole scan — no + nested-array indirection, bounds checks, or copy-on-write traffic per pixel. +- **64-bit bitstream I/O**: the writer packs bits into a `UInt64` accumulator + over a `[UInt8]` buffer; the reader refills a 64-bit window several bytes at + a time and decodes unary prefixes with `leadingZeroBitCount`. +- **Packed context records**: the 365 per-context statistics (A/B/C/N) live in + one record array — a single load and store per pixel. +- **Restart-interval parallelism**: with `Configuration.restartInterval` set, + every interval is independently codable, so large frames encode and decode + across all cores (see [PERFORMANCE_TUNING.md](docs/PERFORMANCE_TUNING.md)). ### Design Principles -1. **Platform Abstraction**: All platform-specific code behind protocols for clean separation +1. **Measured Performance**: Optimisations must show up in benchmarks on real data and keep encoded output byte-identical (unless a new feature legitimately changes the stream) 2. **Testability**: Every component designed for unit testing with >95% coverage -3. **Performance First**: Optimised for Apple Silicon while maintaining correctness -4. **x86-64 Removability**: Clear compilation boundaries for future x86-64 deprecation -5. **Memory Efficiency**: Buffer pooling, tile-based processing, and cache-friendly layouts for large images -6. **Standards Compliance**: Strict adherence to ISO/IEC 14495-1:1999 / ITU-T.87 +3. **Memory Efficiency**: Flat per-scan planes and allocation-free pixel loops for large images +4. **Standards Compliance**: Strict adherence to ISO/IEC 14495-1:1999 / ITU-T.87 ## Command-Line Tool @@ -730,7 +689,6 @@ JLSwift/ │ │ ├── Core/ # Core types and protocols │ │ ├── Decoder/ # Decoding implementation │ │ ├── Encoder/ # Encoding implementation -│ │ ├── Platform/ # Platform-specific code │ │ └── JPEGLS.swift # Module exports │ └── jpeglscli/ # Command-line tool ├── Tests/ @@ -759,12 +717,10 @@ JLSwift/ | [SERVER_SIDE_EXAMPLES.md](docs/SERVER_SIDE_EXAMPLES.md) | Server-side Swift integration guide (Vapor, Hummingbird, NIO) | | [DICOMKIT_INTEGRATION.md](docs/DICOMKIT_INTEGRATION.md) | DICOMkit integration guide for DICOM imaging workflows | | [PERFORMANCE_TUNING.md](docs/PERFORMANCE_TUNING.md) | Performance optimisation and benchmarking guide | -| [METAL_GPU_ACCELERATION.md](docs/METAL_GPU_ACCELERATION.md) | Metal GPU acceleration guide for large images | | [TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md) | Common issues and solutions | | [VERSIONING.md](docs/VERSIONING.md) | Semantic versioning strategy and release guidelines | | [CHANGELOG.md](CHANGELOG.md) | Complete history of changes and releases | | [MILESTONES.md](docs/MILESTONES.md) | Project milestones and development roadmap | -| [X86_64_REMOVAL_GUIDE.md](docs/X86_64_REMOVAL_GUIDE.md) | Step-by-step guide for removing x86-64 support | | [Copilot Instructions](.github/copilot-instructions.md) | Coding guidelines for contributors | ### API Documentation @@ -779,7 +735,7 @@ swift package generate-documentation ### User Guides - **[Getting Started](docs/GETTING_STARTED.md)**: Installation, quick start, and basic usage examples -- **[Performance Tuning](docs/PERFORMANCE_TUNING.md)**: Hardware acceleration, memory optimisation, and profiling +- **[Performance Tuning](docs/PERFORMANCE_TUNING.md)**: Benchmarking, restart-interval parallelism, and profiling - **[Troubleshooting](docs/TROUBLESHOOTING.md)**: Solutions to common problems and debugging tips ## Contributing diff --git a/Sources/JPEGLS/Core/JPEGLSBitstreamReader.swift b/Sources/JPEGLS/Core/JPEGLSBitstreamReader.swift index 9a917b0..b090460 100644 --- a/Sources/JPEGLS/Core/JPEGLSBitstreamReader.swift +++ b/Sources/JPEGLS/Core/JPEGLSBitstreamReader.swift @@ -9,64 +9,91 @@ import Foundation /// /// Reads bytes and bits from a buffer, handling marker stuffing and /// detecting premature end of stream. +/// +/// Bit-level reads run over a 64-bit window refilled several bytes at a +/// time (applying the ISO 14495-1 §9.1 stuff-bit rule per refilled byte), +/// so per-bit work is a shift and a counter update instead of a byte fetch. +/// Unary prefixes are decoded with `leadingZeroBitCount` over the window. +/// +/// - Important: The eager refill advances the byte position ahead of bit +/// consumption. After any bit-level read, call `resetBitBuffer()` before +/// using the byte-level API (`readByte`, `peekByte`, `readBytes`, +/// `readUInt16`, `findNextMarker`, `currentPosition`, `bytesRemaining`) — +/// it realigns the position to the byte boundary after the last consumed +/// bit. Mixing the two modes without a reset reads ahead of the logical +/// position. public final class JPEGLSBitstreamReader { - private let data: Data + private let bytes: [UInt8] + /// Index of the next byte to load (bit reads) or read directly (byte reads). + /// Bit-level refill advances this eagerly; the unconsumed bits live in + /// `bitBuffer`/`bitsInBuffer`. private var position: Int - private var bitBuffer: UInt32 + /// Bit accumulator. The low `bitsInBuffer` bits below previously-consumed + /// garbage are valid; consumption only decrements `bitsInBuffer` (matching + /// the historical extract-by-shift-and-mask behaviour). + private var bitBuffer: UInt64 private var bitsInBuffer: Int - + /// Byte index where the current buffered bit region began, and the number + /// of bits consumed from it. `resetBitBuffer` replays the stuffing rules + /// over the region to land `position` on the byte boundary following the + /// last consumed bit (the eager refill advances `position` further ahead). + private var bitRegionStart: Int + private var bitsConsumedInRegion: Int + /// Initialize reader with data buffer /// /// - Parameter data: Input data to read from public init(data: Data) { - self.data = data + self.bytes = [UInt8](data) self.position = 0 self.bitBuffer = 0 self.bitsInBuffer = 0 + self.bitRegionStart = 0 + self.bitsConsumedInRegion = 0 } - + /// Current read position in bytes public var currentPosition: Int { return position } - + /// Number of bytes remaining in buffer public var bytesRemaining: Int { - return data.count - position + return bytes.count - position } - + /// Returns true if end of data reached public var isAtEnd: Bool { - return position >= data.count && bitsInBuffer == 0 + return position >= bytes.count && bitsInBuffer == 0 } - + /// Read a single byte from the stream /// /// - Returns: The byte value /// - Throws: `JPEGLSError.prematureEndOfStream` if no data available public func readByte() throws -> UInt8 { - guard position < data.count else { + guard position < bytes.count else { throw JPEGLSError.prematureEndOfStream } - let byte = data[position] + let byte = bytes[position] position += 1 return byte } - + /// Read multiple bytes from the stream /// /// - Parameter count: Number of bytes to read /// - Returns: Data containing the bytes /// - Throws: `JPEGLSError.prematureEndOfStream` if not enough data public func readBytes(_ count: Int) throws -> Data { - guard position + count <= data.count else { + guard count >= 0, position + count <= bytes.count else { throw JPEGLSError.prematureEndOfStream } - let bytes = data[position.. UInt8? { - guard position < data.count else { + guard position < bytes.count else { return nil } - return data[position] + return bytes[position] } - + /// Read a marker (2-byte sequence starting with 0xFF) /// /// - Returns: The marker @@ -96,15 +123,15 @@ public final class JPEGLSBitstreamReader { guard byte1 == JPEGLSMarker.markerPrefix else { throw JPEGLSError.invalidMarker(byte1: byte1, byte2: 0) } - + let byte2 = try readByte() guard let marker = JPEGLSMarker(rawValue: byte2) else { throw JPEGLSError.invalidMarker(byte1: byte1, byte2: byte2) } - + return marker } - + /// Skip to the next marker in the stream /// /// Uses the standard JPEG-LS stuffing rule (ISO 14495-1 §9.1): a byte following 0xFF @@ -127,7 +154,49 @@ public final class JPEGLSBitstreamReader { } throw JPEGLSError.prematureEndOfStream } - + + // MARK: - Bit-level reading + + /// Top up the 64-bit window while at least 16 bits of headroom remain, + /// applying the §9.1 stuffing rule per refilled byte: + /// - 0xFF followed by a byte < 0x80: stuffed pair contributes 8 + 7 bits + /// (the follower's MSB is the discarded stuff bit). + /// - 0xFF followed by a byte ≥ 0x80 (marker) or at end of data: + /// the 0xFF alone contributes 8 bits. + @inline(__always) + private func refill() { + if bitsInBuffer == 0 && bitsConsumedInRegion == 0 { + bitRegionStart = position + } + let count = bytes.count + while bitsInBuffer <= 48 && position < count { + let byte = bytes[position] + if byte == 0xFF && position + 1 < count { + let next = bytes[position + 1] + if next < 0x80 { + // Stuffed pair: 0xFF (8 bits) + stuff bit dropped + 7 data bits. + position += 2 + bitBuffer = (bitBuffer << 15) | (0xFF << 7) | UInt64(next & 0x7F) + bitsInBuffer += 15 + continue + } + // next >= 0x80: marker follows. The FF is added below; decoding + // should finish before any marker bytes are consumed. + } + position += 1 + bitBuffer = (bitBuffer << 8) | UInt64(byte) + bitsInBuffer += 8 + } + } + + /// The valid bits left-aligned at the top of a 64-bit word. + /// Only call with `bitsInBuffer > 0`. + @inline(__always) + private func alignedWindow() -> UInt64 { + let mask: UInt64 = (1 << UInt64(bitsInBuffer)) &- 1 + return (bitBuffer & mask) << UInt64(64 - bitsInBuffer) + } + /// Read bits from the bitstream /// /// Implements JPEG-LS bit-level byte stuffing per ISO 14495-1 §9.1: @@ -149,66 +218,98 @@ public final class JPEGLSBitstreamReader { throw JPEGLSError.internalError(reason: "Invalid bit count: \(count)") } - // Fill buffer if needed - while bitsInBuffer < count { - guard position < data.count else { - break - } - let byte = data[position] - position += 1 - - // Bit-level stuffing per ISO 14495-1 §9.1: - // When a byte of 0xFF is read, peek at the next byte: - // MSB = 0 (value < 0x80): stuffed byte — consume it, add 8 bits (FF) + 7 bits. - // MSB = 1 (value >= 0x80): real marker — don't consume, add FF to buffer only. - if byte == 0xFF && position < data.count { - let next = data[position] - if next < 0x80 { - // Stuffed byte: consume it, contribute 8 bits (FF) + 7 data bits. - position += 1 - bitBuffer = (bitBuffer << 8) | UInt32(byte) - bitsInBuffer += 8 - bitBuffer = (bitBuffer << 7) | UInt32(next & 0x7F) - bitsInBuffer += 7 - continue - } - // next >= 0x80: real marker. FF is added to the buffer below but decoding - // should finish before consuming marker bytes. + if bitsInBuffer < count { + refill() + guard bitsInBuffer >= count else { + throw JPEGLSError.prematureEndOfStream } - - bitBuffer = (bitBuffer << 8) | UInt32(byte) - bitsInBuffer += 8 - } - - guard bitsInBuffer >= count else { - throw JPEGLSError.prematureEndOfStream } // Extract bits from the most-significant valid position. let shift = bitsInBuffer - count let mask: UInt32 = count < 32 ? ((1 << count) - 1) : UInt32.max - let bits = (bitBuffer >> shift) & mask + let bits = UInt32(truncatingIfNeeded: bitBuffer >> UInt64(shift)) & mask bitsInBuffer -= count + bitsConsumedInRegion += count return bits } - + + /// Read a unary prefix: zero or more '0' bits terminated by a single '1' + /// bit, returning the number of zeros. Equivalent to calling + /// `readBits(1)` in a loop until a 1 appears, but counts the zeros with + /// `leadingZeroBitCount` over the buffered window instead of one call + /// per bit. + /// + /// - Returns: The number of '0' bits before the terminating '1' + /// - Throws: `JPEGLSError.prematureEndOfStream` if the stream ends + /// before a '1' bit is found + public func readUnaryCount() throws -> Int { + var count = 0 + while true { + if bitsInBuffer == 0 { + refill() + guard bitsInBuffer > 0 else { + throw JPEGLSError.prematureEndOfStream + } + } + let window = alignedWindow() + let zeros = min(window.leadingZeroBitCount, bitsInBuffer) + if zeros < bitsInBuffer { + // Found the terminating '1': consume the zeros and the 1. + bitsInBuffer -= zeros + 1 + bitsConsumedInRegion += zeros + 1 + return count + zeros + } + // Every buffered bit is 0 — consume them all and keep scanning. + count += bitsInBuffer + bitsConsumedInRegion += bitsInBuffer + bitsInBuffer = 0 + } + } + /// Reset the bit buffer (typically called at scan boundaries) + /// + /// Discards any unconsumed buffered bits and aligns the byte position to + /// the boundary following the last consumed bit, replaying the §9.1 + /// stuffing rule over the buffered region (a stuffed 0xFF pair counts as + /// two bytes carrying 15 bits). public func resetBitBuffer() { + if bitsConsumedInRegion > 0 { + var pos = bitRegionStart + var remaining = bitsConsumedInRegion + let count = bytes.count + while remaining > 0 && pos < count { + let byte = bytes[pos] + if byte == 0xFF && pos + 1 < count && bytes[pos + 1] < 0x80 { + pos += 2 + remaining -= 15 + } else { + pos += 1 + remaining -= 8 + } + } + position = pos + } bitBuffer = 0 bitsInBuffer = 0 + bitRegionStart = position + bitsConsumedInRegion = 0 } - + /// Seek to a specific position in the stream /// /// - Parameter position: Target position in bytes /// - Throws: `JPEGLSError` if position is invalid public func seek(to position: Int) throws { - guard position >= 0 && position <= data.count else { + guard position >= 0 && position <= bytes.count else { throw JPEGLSError.internalError(reason: "Invalid seek position: \(position)") } + bitBuffer = 0 + bitsInBuffer = 0 + bitsConsumedInRegion = 0 self.position = position - resetBitBuffer() + bitRegionStart = position } } diff --git a/Sources/JPEGLS/Core/JPEGLSBitstreamWriter.swift b/Sources/JPEGLS/Core/JPEGLSBitstreamWriter.swift index ea4b486..5cab9e0 100644 --- a/Sources/JPEGLS/Core/JPEGLSBitstreamWriter.swift +++ b/Sources/JPEGLS/Core/JPEGLSBitstreamWriter.swift @@ -8,20 +8,28 @@ import Foundation /// Bitstream writer for JPEG-LS /// /// Writes bytes and bits to a buffer, handling marker stuffing automatically. +/// +/// Internally the writer accumulates bits in a 64-bit buffer and stores bytes +/// in a contiguous `[UInt8]`, converting to `Data` only once in `getData()`. +/// The 64-bit accumulator means a worst-case 7-bit residual plus a full 32-bit +/// `writeBits` call (39 bits, plus transient stuff bits) always fits without +/// overflow — unlike a 32-bit accumulator, which silently drops high bits when +/// `bitsInBuffer + count > 32`. public final class JPEGLSBitstreamWriter { - private var data: Data - private var bitBuffer: UInt32 + private var bytes: [UInt8] + private var bitBuffer: UInt64 private var bitsInBuffer: Int - + /// Initialize writer with optional initial capacity /// /// - Parameter capacity: Initial buffer capacity in bytes public init(capacity: Int = 4096) { - self.data = Data(capacity: capacity) + self.bytes = [] + self.bytes.reserveCapacity(capacity) self.bitBuffer = 0 self.bitsInBuffer = 0 } - + /// Get the written data /// /// - Returns: The complete bitstream data @@ -32,7 +40,7 @@ public final class JPEGLSBitstreamWriter { reason: "Bit buffer not flushed, \(bitsInBuffer) bits remaining" ) } - return data + return Data(bytes) } /// Access the accumulated data without copying via a closure. @@ -55,14 +63,14 @@ public final class JPEGLSBitstreamWriter { reason: "Bit buffer not flushed, \(bitsInBuffer) bits remaining" ) } - return try data.withUnsafeBytes(body) + return try bytes.withUnsafeBytes(body) } /// Current write position in bytes public var currentPosition: Int { - return data.count + return bytes.count } - + /// Write a single byte to the stream (no stuffing). /// /// This method writes raw bytes for structured data (marker segments, headers). @@ -70,45 +78,43 @@ public final class JPEGLSBitstreamWriter { /// /// - Parameter byte: The byte to write public func writeByte(_ byte: UInt8) { - data.append(byte) + bytes.append(byte) } - + /// Write multiple bytes to the stream (no stuffing). /// /// - Parameter bytes: The bytes to write - public func writeBytes(_ bytes: Data) { - data.append(contentsOf: bytes) + public func writeBytes(_ data: Data) { + bytes.append(contentsOf: data) } - + /// Write bytes from a raw buffer pointer without copying, for /// zero-copy bulk transfer of pre-encoded data. /// /// - Parameter buffer: Raw buffer whose bytes are appended verbatim. /// No JPEG-LS bit-stuffing is applied; use only for pre-encoded data. public func writeBytesNoCopy(_ buffer: UnsafeRawBufferPointer) { - data.append(contentsOf: buffer) + bytes.append(contentsOf: buffer) } - + /// Write a 16-bit big-endian value /// /// - Parameter value: The 16-bit value public func writeUInt16(_ value: UInt16) { - let byte1 = UInt8((value >> 8) & 0xFF) - let byte2 = UInt8(value & 0xFF) - data.append(byte1) - data.append(byte2) + bytes.append(UInt8((value >> 8) & 0xFF)) + bytes.append(UInt8(value & 0xFF)) } - + /// Write a marker (2-byte sequence) /// /// Does NOT perform marker stuffing for marker bytes /// /// - Parameter marker: The marker to write public func writeMarker(_ marker: JPEGLSMarker) { - data.append(JPEGLSMarker.markerPrefix) - data.append(marker.rawValue) + bytes.append(JPEGLSMarker.markerPrefix) + bytes.append(marker.rawValue) } - + /// Write bits to the bitstream with JPEG-LS bit-level stuffing. /// /// Accumulates bits in a buffer and flushes complete bytes. Implements bit-level @@ -131,28 +137,32 @@ public final class JPEGLSBitstreamWriter { let mask: UInt32 = count < 32 ? ((1 << count) - 1) : UInt32.max let maskedBits = bits & mask - // Add bits to buffer - bitBuffer = (bitBuffer << count) | maskedBits + // Add bits to buffer. bitsInBuffer never exceeds 7 on entry, so + // 7 + 32 = 39 bits always fit in the 64-bit accumulator. Bits above + // position `bitsInBuffer` are stale garbage from earlier shifts — + // extraction below masks them out; do not assume they are zero. + bitBuffer = (bitBuffer << UInt64(count)) | UInt64(maskedBits) bitsInBuffer += count // Write complete bytes with bit-level stuffing while bitsInBuffer >= 8 { let shift = bitsInBuffer - 8 - let byte = UInt8((bitBuffer >> shift) & 0xFF) - data.append(byte) + let byte = UInt8(truncatingIfNeeded: bitBuffer >> UInt64(shift)) + bytes.append(byte) bitsInBuffer -= 8 // Bit-level stuffing per ISO 14495-1 §9.1: // After emitting a byte of 0xFF, insert a 0 stuff bit at the next bit position. - // The UInt32 buffer already has 0 in unused positions; we clear the specific bit - // at position `bitsInBuffer` (the new MSB of the valid range) to make it 0. + // This clear is REQUIRED: the bit at position `bitsInBuffer` is the + // just-emitted 0xFF's least-significant bit (a 1); clearing it turns + // that position into the 0 stuff bit when bitsInBuffer grows past it. if byte == 0xFF { - bitBuffer &= ~(UInt32(1) << UInt32(bitsInBuffer)) + bitBuffer &= ~(UInt64(1) << UInt64(bitsInBuffer)) bitsInBuffer += 1 } } } - + /// Flush remaining bits in buffer /// /// Pads with zeros to complete the final byte. No stuffing is applied to the @@ -161,13 +171,13 @@ public final class JPEGLSBitstreamWriter { public func flush() { if bitsInBuffer > 0 { let shift = 8 - bitsInBuffer - let byte = UInt8((bitBuffer << shift) & 0xFF) - data.append(byte) + let byte = UInt8(truncatingIfNeeded: (bitBuffer << UInt64(shift)) & 0xFF) + bytes.append(byte) bitBuffer = 0 bitsInBuffer = 0 } } - + /// Reset the bit buffer (typically called at scan boundaries) public func resetBitBuffer() { flush() @@ -176,45 +186,37 @@ public final class JPEGLSBitstreamWriter { /// Write a unary code: n zero bits followed by a single 1 bit. /// /// This is a performance-optimised alternative to calling `writeBits(0, count: 1)` in a loop - /// followed by `writeBits(1, count: 1)`. Writing in batches of up to 24 bits reduces + /// followed by `writeBits(1, count: 1)`. Writing in batches of up to 32 bits reduces /// function-call overhead significantly in the Golomb-Rice coding hot path. /// - /// The batch size is capped at 24 because the internal `bitBuffer` is 32 bits wide and may - /// already hold up to 7 bits from the previous call. Adding 25 bits (24 zeros + 1 terminator) - /// to a 7-bit residual gives exactly 32 bits, which fits without overflow. - /// /// - Parameter n: Number of leading zero bits (must be ≥ 0) public func writeUnaryCode(_ n: Int) { var remaining = n - // Write up to 24 zeros at a time. Combined with a worst-case 7-bit residual in - // the buffer, the total (7 + 24 = 31) safely fits in the 32-bit UInt32 bitBuffer. - while remaining >= 24 { - writeBits(0, count: 24) - remaining -= 24 + while remaining >= 32 { + writeBits(0, count: 32) + remaining -= 32 } - // Write the remaining zeros and the terminating 1 in one call (max count = 24 - // when remaining == 23, giving 7 + 24 = 31 bits total — within UInt32 range). + // Write the remaining zeros and the terminating 1 in one call + // (max count = 32 when remaining == 31). writeBits(1, count: remaining + 1) } /// Write n consecutive 1 bits (used for Golomb run-length continuation codes). /// /// This is a performance-optimised alternative to calling `writeBits(1, count: 1)` in a loop. - /// The batch size is capped at 24 for the same UInt32 overflow reason as `writeUnaryCode`. /// /// - Parameter n: Number of 1 bits to write (must be ≥ 0) public func writeOnes(_ n: Int) { var remaining = n - // (1 << 24) - 1 = 0x00FF_FFFF fits in UInt32 with room to spare. - while remaining >= 24 { - writeBits(0x00FFFFFF, count: 24) - remaining -= 24 + while remaining >= 32 { + writeBits(UInt32.max, count: 32) + remaining -= 32 } if remaining > 0 { writeBits(UInt32((1 << remaining) - 1), count: remaining) } } - + /// Write a marker segment with length field /// /// - Parameters: @@ -222,15 +224,15 @@ public final class JPEGLSBitstreamWriter { /// - payload: The segment payload data public func writeMarkerSegment(marker: JPEGLSMarker, payload: Data) { writeMarker(marker) - + // Length includes the 2 bytes for length field itself let length = UInt16(payload.count + 2) writeUInt16(length) - + // Write payload without stuffing (it's not compressed data) - data.append(payload) + bytes.append(contentsOf: payload) } - + /// Reserve space for a marker segment and return position /// /// Useful for writing segments where length is not known upfront @@ -239,22 +241,20 @@ public final class JPEGLSBitstreamWriter { /// - Returns: Position where length field starts public func beginMarkerSegment(marker: JPEGLSMarker) -> Int { writeMarker(marker) - let lengthPos = data.count + let lengthPos = bytes.count writeUInt16(0) // Placeholder for length return lengthPos } - + /// Finalize a marker segment by updating its length /// /// - Parameter lengthPosition: Position returned by beginMarkerSegment public func endMarkerSegment(lengthPosition: Int) { - let currentPos = data.count + let currentPos = bytes.count let length = UInt16(currentPos - lengthPosition) - + // Update length field - let byte1 = UInt8((length >> 8) & 0xFF) - let byte2 = UInt8(length & 0xFF) - data[lengthPosition] = byte1 - data[lengthPosition + 1] = byte2 + bytes[lengthPosition] = UInt8((length >> 8) & 0xFF) + bytes[lengthPosition + 1] = UInt8(length & 0xFF) } } diff --git a/Sources/JPEGLS/Core/JPEGLSBufferPool.swift b/Sources/JPEGLS/Core/JPEGLSBufferPool.swift deleted file mode 100644 index cc90f72..0000000 --- a/Sources/JPEGLS/Core/JPEGLSBufferPool.swift +++ /dev/null @@ -1,141 +0,0 @@ -/// Buffer pooling system for reducing memory allocations -/// -/// Provides reusable buffer pools for common allocation patterns in JPEG-LS encoding/decoding. -/// Thread-safe implementation suitable for concurrent encoding/decoding operations. -import Foundation - -/// A pool of reusable buffers to reduce allocation overhead -public final class JPEGLSBufferPool: @unchecked Sendable { - /// Buffer type stored in the pool - public enum BufferType: Hashable { - case contextArrays // For context model arrays (A, B, C, N) - case pixelData // For pixel data buffers - case bitstreamData // For bitstream data - case custom(String) // For custom buffer types - } - - /// Pooled buffer wrapper - private struct PooledBuffer { - var data: [Int] - let capacity: Int - var lastUsed: Date - } - - // Thread-safe storage using lock - private let lock = NSLock() - private var pools: [BufferType: [PooledBuffer]] = [:] - private let maxPoolSize: Int - private let bufferLifetime: TimeInterval - - /// Creates a new buffer pool - /// - Parameters: - /// - maxPoolSize: Maximum number of buffers to keep per type (default: 10) - /// - bufferLifetime: Maximum lifetime for cached buffers in seconds (default: 60) - public init(maxPoolSize: Int = 10, bufferLifetime: TimeInterval = 60) { - self.maxPoolSize = maxPoolSize - self.bufferLifetime = bufferLifetime - } - - /// Acquires a buffer from the pool or creates a new one. - /// - /// When a pooled buffer of sufficient capacity is found it is returned - /// after zeroing its contents, avoiding a new heap allocation. New - /// buffers are created only when no suitable pooled buffer exists. - /// - /// - Parameters: - /// - type: The type of buffer to acquire - /// - size: Required buffer size (number of `Int` elements) - /// - Returns: A zero-initialized buffer of at least the requested size. - public func acquire(type: BufferType, size: Int) -> [Int] { - lock.lock() - defer { lock.unlock() } - - // Try to find a suitable buffer in the pool - if var buffers = pools[type] { - // Find the smallest buffer that satisfies the size requirement to - // minimise wasted capacity while still avoiding a new allocation. - if let index = buffers.indices.min(by: { - let ca = buffers[$0].capacity, cb = buffers[$1].capacity - let aOk = ca >= size, bOk = cb >= size - if aOk && bOk { return ca < cb } - return aOk - }), buffers[index].capacity >= size { - var pooledBuffer = buffers.remove(at: index) - pools[type] = buffers - // Zero-fill the live portion and return. - // This reuses the existing heap allocation rather than allocating a new array. - for i in 0.. bufferLifetime - } - if buffers.isEmpty { - pools.removeValue(forKey: type) - } else { - pools[type] = buffers - } - } - } - } - - /// Clears all buffers from the pool - public func clear() { - lock.lock() - defer { lock.unlock() } - pools.removeAll() - } - - /// Returns statistics about the current pool state - public func statistics() -> [BufferType: Int] { - lock.lock() - defer { lock.unlock() } - - return pools.mapValues { $0.count } - } -} - -/// Global shared buffer pool for common use cases -public let sharedBufferPool = JPEGLSBufferPool() diff --git a/Sources/JPEGLS/Core/JPEGLSCacheFriendlyBuffer.swift b/Sources/JPEGLS/Core/JPEGLSCacheFriendlyBuffer.swift deleted file mode 100644 index 28ddc85..0000000 --- a/Sources/JPEGLS/Core/JPEGLSCacheFriendlyBuffer.swift +++ /dev/null @@ -1,215 +0,0 @@ -/// Cache-friendly data layout for JPEG-LS processing -/// -/// Optimizes data structures for better CPU cache utilization during encoding/decoding. -/// Uses row-major layout with contiguous memory and prefetching hints. -import Foundation - -/// Cache-friendly pixel buffer with optimized memory layout -public struct JPEGLSCacheFriendlyBuffer: Sendable { - /// Component data stored in contiguous memory (row-major order) - private let componentData: [UInt8: [Int]] - - /// Image width in pixels - public let width: Int - - /// Image height in pixels - public let height: Int - - /// Number of components - public let componentCount: Int - - /// Cache line size for prefetching (typically 64 bytes on modern CPUs) - private static let cacheLineSize = 64 - - /// Creates a new cache-friendly buffer from 2D pixel data - /// - Parameters: - /// - pixelData: 2D pixel data per component - /// - width: Image width - /// - height: Image height - public init(pixelData: [UInt8: [[Int]]], width: Int, height: Int) { - self.width = width - self.height = height - self.componentCount = pixelData.count - - // Flatten 2D arrays into contiguous 1D arrays for better cache locality - var flattened: [UInt8: [Int]] = [:] - for (componentId, rows) in pixelData { - var flat = [Int]() - flat.reserveCapacity(width * height) - for row in rows { - flat.append(contentsOf: row) - } - flattened[componentId] = flat - } - - self.componentData = flattened - } - - /// Creates a new cache-friendly buffer with contiguous data - /// - Parameters: - /// - contiguousData: Contiguous pixel data per component (row-major order) - /// - width: Image width - /// - height: Image height - public init(contiguousData: [UInt8: [Int]], width: Int, height: Int) { - self.width = width - self.height = height - self.componentCount = contiguousData.count - self.componentData = contiguousData - } - - /// Gets a pixel value at the specified position - /// - /// Uses inline hint for optimal performance in tight encoding/decoding loops. - /// - /// - Parameters: - /// - componentId: Component identifier - /// - row: Row index - /// - column: Column index - /// - Returns: Pixel value or nil if out of bounds - @inline(__always) - public func getPixel(componentId: UInt8, row: Int, column: Int) -> Int? { - guard row >= 0, row < height, column >= 0, column < width else { return nil } - guard let data = componentData[componentId] else { return nil } - - let index = row * width + column - return data[index] - } - - /// Sets a pixel value at the specified position (creates a new buffer) - /// - Parameters: - /// - componentId: Component identifier - /// - row: Row index - /// - column: Column index - /// - value: New pixel value - /// - Returns: New buffer with the updated pixel - public func settingPixel(componentId: UInt8, row: Int, column: Int, value: Int) -> JPEGLSCacheFriendlyBuffer { - guard row >= 0, row < height, column >= 0, column < width else { return self } - - var newData = componentData - if var data = newData[componentId] { - let index = row * width + column - data[index] = value - newData[componentId] = data - } - - return JPEGLSCacheFriendlyBuffer(contiguousData: newData, width: width, height: height) - } - - /// Gets a contiguous row of pixels for cache-efficient processing - /// - /// Optimized for sequential access patterns with better cache locality. - /// - /// - Parameters: - /// - componentId: Component identifier - /// - row: Row index - /// - Returns: Array of pixels in the row or empty array if invalid - @inline(__always) - public func getRow(componentId: UInt8, row: Int) -> [Int] { - guard row >= 0, row < height else { return [] } - guard let data = componentData[componentId] else { return [] } - - let startIndex = row * width - let endIndex = startIndex + width - return Array(data[startIndex.. [Int] { - guard rowStart >= 0, rowEnd <= height, rowStart < rowEnd else { return [] } - guard let data = componentData[componentId] else { return [] } - - let startIndex = rowStart * width - let endIndex = rowEnd * width - return Array(data[startIndex.. (left: Int?, top: Int?, topLeft: Int?, topRight: Int?) { - guard let data = componentData[componentId] else { - return (nil, nil, nil, nil) - } - - // Calculate indices for cache-efficient access - let currentIndex = row * width + column - - let left = column > 0 ? data[currentIndex - 1] : nil - let top = row > 0 ? data[currentIndex - width] : nil - let topLeft = (row > 0 && column > 0) ? data[currentIndex - width - 1] : nil - let topRight = (row > 0 && column < width - 1) ? data[currentIndex - width + 1] : nil - - return (left, top, topLeft, topRight) - } - - /// Converts back to 2D array format for compatibility - /// - Parameter componentId: Component identifier - /// - Returns: 2D pixel array - public func to2DArray(componentId: UInt8) -> [[Int]] { - guard let data = componentData[componentId] else { return [] } - - var result: [[Int]] = [] - result.reserveCapacity(height) - - for row in 0.. [Int] { - return componentData[componentId] ?? [] - } - - /// Gets all component identifiers - public var componentIds: [UInt8] { - return Array(componentData.keys).sorted() - } -} - -/// Memory statistics for profiling -public struct JPEGLSMemoryStatistics: Sendable { - /// Total bytes allocated - public let totalBytes: Int - - /// Peak bytes used - public let peakBytes: Int - - /// Number of allocations - public let allocationCount: Int - - /// Average allocation size - public var averageAllocationSize: Double { - guard allocationCount > 0 else { return 0 } - return Double(totalBytes) / Double(allocationCount) - } - - /// Creates new memory statistics - public init(totalBytes: Int, peakBytes: Int, allocationCount: Int) { - self.totalBytes = totalBytes - self.peakBytes = peakBytes - self.allocationCount = allocationCount - } -} diff --git a/Sources/JPEGLS/Core/JPEGLSContextModel.swift b/Sources/JPEGLS/Core/JPEGLSContextModel.swift index 006044e..c7cf1ef 100644 --- a/Sources/JPEGLS/Core/JPEGLSContextModel.swift +++ b/Sources/JPEGLS/Core/JPEGLSContextModel.swift @@ -21,23 +21,22 @@ public struct JPEGLSContextModel: Sendable { /// Number of run-length contexts public static let runContextCount = 2 - // MARK: - Context State Arrays - - /// Accumulated prediction error sum for each context. - /// Used to compute the bias correction term. - private var contextA: [Int] - - /// Context occurrence counter. - /// Tracks how many times each context has been used. - private var contextB: [Int] - - /// Bias correction value for each context. - /// Represents the accumulated bias in prediction errors. - private var contextC: [Int] - - /// Sample counter for reset operations. - /// When N reaches the RESET value, context statistics are halved. - private var contextN: [Int] + // MARK: - Context State + + /// Per-context adaptive statistics (A = accumulated absolute error, + /// B = bias accumulator, C = bias correction, N = occurrence counter), + /// packed into one record so the per-pixel update is a single array + /// load and store (one bounds check, one copy-on-write uniqueness + /// check, one cache line) instead of up to ten accesses across four + /// parallel arrays. + private struct ContextRecord: Sendable { + var a: Int + var b: Int + var c: Int + var n: Int + } + + private var contexts: [ContextRecord] // MARK: - Run-Length State @@ -74,6 +73,11 @@ public struct JPEGLSContextModel: Sendable { /// A[i] initial value per ITU-T.87 Section 4.3: max(2, floor((RANGE + 32) / 64)) private let aInit: Int + + /// Hoisted per-pixel constants: RESET threshold and the B-update factor + /// (2·NEAR + 1), so the update loop does not reload them per sample. + private let resetThreshold: Int + private let bFactor: Int // MARK: - Initialization @@ -103,12 +107,14 @@ public struct JPEGLSContextModel: Sendable { // Compute A initial value per ITU-T.87 Section 4.3: // A[i] = max(2, floor((RANGE + 32) / 64)) self.aInit = max(2, (range + 32) / 64) - - // Initialize context arrays to default values per ITU-T.87 Section 4.3 - self.contextA = Array(repeating: 0, count: Self.regularContextCount) - self.contextB = Array(repeating: 0, count: Self.regularContextCount) - self.contextC = Array(repeating: 0, count: Self.regularContextCount) - self.contextN = Array(repeating: 1, count: Self.regularContextCount) + self.resetThreshold = parameters.reset + self.bFactor = 2 * near + 1 + + // Initialize context records to default values per ITU-T.87 Section 4.3 + self.contexts = Array( + repeating: ContextRecord(a: 0, b: 0, c: 0, n: 1), + count: Self.regularContextCount + ) // Initialize run-length state self.runInterruptionIndex = Array(repeating: 0, count: Self.runContextCount) @@ -134,10 +140,7 @@ public struct JPEGLSContextModel: Sendable { /// - N[i] = 1 private mutating func initializeContexts() { for i in 0.. (index: Int, sign: Int) { + let sign = computeContextSign(q1: q1, q2: q2, q3: q3) + // Qt = 81 × Q1 + 9 × Q2 + Q3 over sign-normalised gradients + // (ITU-T.87 Section 4.3.1). + let index = 81 * (q1 * sign) + 9 * (q2 * sign) + (q3 * sign) + return (max(0, min(index, Self.regularContextCount - 1)), sign) + } // MARK: - Context State Access @@ -203,7 +227,7 @@ public struct JPEGLSContextModel: Sendable { guard contextIndex >= 0 && contextIndex < Self.regularContextCount else { return 0 } - return contextA[contextIndex] + return contexts[contextIndex].a } /// Get the occurrence counter for a context. @@ -214,7 +238,7 @@ public struct JPEGLSContextModel: Sendable { guard contextIndex >= 0 && contextIndex < Self.regularContextCount else { return 0 } - return contextB[contextIndex] + return contexts[contextIndex].b } /// Get the bias correction for a context. @@ -225,7 +249,7 @@ public struct JPEGLSContextModel: Sendable { guard contextIndex >= 0 && contextIndex < Self.regularContextCount else { return 0 } - return contextC[contextIndex] + return contexts[contextIndex].c } /// Get the reset counter for a context. @@ -236,7 +260,7 @@ public struct JPEGLSContextModel: Sendable { guard contextIndex >= 0 && contextIndex < Self.regularContextCount else { return 1 } - return contextN[contextIndex] + return contexts[contextIndex].n } // MARK: - Context Update @@ -255,41 +279,44 @@ public struct JPEGLSContextModel: Sendable { guard contextIndex >= 0 && contextIndex < Self.regularContextCount else { return } - + + // Load the record once; all updates happen on locals and store back + // in a single write (one bounds + one CoW check per pixel). + var r = contexts[contextIndex] + // Update A (accumulated absolute prediction error) per ITU-T.87 - contextA[contextIndex] += abs(predictionError) - + r.a += abs(predictionError) + // Update B per ITU-T.87 §A.6.2: B[Q] += Errval × (2·NEAR + 1) // The caller passes predictionError = sign × Errval (sign-denormalised), // so sign × predictionError = Errval (sign-normalised error per the standard). let errval = sign * predictionError - let bIncrement = errval * (2 * near + 1) - contextB[contextIndex] += bIncrement - + r.b += errval * bFactor + // Reset when N reaches RESET value per ITU-T.87 Section A.6.2 // Reset check happens BEFORE N is incremented (per standard and CharLS). - if contextN[contextIndex] >= parameters.reset { - contextA[contextIndex] >>= 1 - contextB[contextIndex] >>= 1 + if r.n >= resetThreshold { + r.a >>= 1 + r.b >>= 1 // Use max(..., 1) to ensure N doesn't become zero after the right-shift. - contextN[contextIndex] = max(contextN[contextIndex] >> 1, 1) + r.n = max(r.n >> 1, 1) } - + // Increment N (after reset check, before bias correction) - contextN[contextIndex] += 1 - + r.n += 1 + // Bias correction per ITU-T.87 Section A.6.3 (code segment A.13). // Inner clamping uses max/min instead of nested branches to reduce // branch-predictor pressure in the hot encoding loop. - let b = contextB[contextIndex] - let n = contextN[contextIndex] - if b + n <= 0 { - contextB[contextIndex] = max(b + n, 1 - n) - contextC[contextIndex] = max(contextC[contextIndex] - 1, -128) - } else if b > 0 { - contextB[contextIndex] = min(b - n, 0) - contextC[contextIndex] = min(contextC[contextIndex] + 1, 127) + if r.b + r.n <= 0 { + r.b = max(r.b + r.n, 1 - r.n) + r.c = max(r.c - 1, -128) + } else if r.b > 0 { + r.b = min(r.b - r.n, 0) + r.c = min(r.c + 1, 127) } + + contexts[contextIndex] = r } // MARK: - Golomb Parameter Calculation @@ -306,14 +333,18 @@ public struct JPEGLSContextModel: Sendable { return 0 } - let a = contextA[contextIndex] - let n = contextN[contextIndex] // Use N (occurrence counter), not B + let r = contexts[contextIndex] + return Self.golombParameter(a: r.a, n: r.n) + } + /// Golomb parameter from raw (A, N) statistics: smallest k ≥ 0 such + /// that n << k ≥ a, capped at 16. + @inline(__always) + private static func golombParameter(a: Int, n: Int) -> Int { guard n > 0 else { return 0 } guard a > n else { return 0 } // Fast computation using integer bit widths: - // find smallest k ≥ 0 such that n << k ≥ a. // floor(log2(a)) − floor(log2(n)) gives a lower bound on k; // at most one additional increment is ever needed. let logA = Int.bitWidth - 1 - a.leadingZeroBitCount // floor(log2(a)) @@ -322,6 +353,20 @@ public struct JPEGLSContextModel: Sendable { if n << k < a { k += 1 } return min(k, 16) } + + /// Fetch the full per-pixel regular-mode coding state — bias correction + /// C[Q], Golomb parameter k, and the k = 0 error-correction term — from + /// a single context-record load. Identical results to calling `getC`, + /// `computeGolombParameter`, and `getErrorCorrection` separately. + public func pixelCodingState(contextIndex: Int) -> (biasC: Int, k: Int, errorCorrection: Int) { + guard contextIndex >= 0 && contextIndex < Self.regularContextCount else { + return (0, 0, 0) + } + let r = contexts[contextIndex] + let k = Self.golombParameter(a: r.a, n: r.n) + let errorCorrection = (k == 0 && near == 0 && (2 * r.b + r.n - 1) < 0) ? -1 : 0 + return (r.c, k, errorCorrection) + } /// Compute error correction for k=0 map swap per ITU-T.87 §A.5.2 / CharLS. /// @@ -337,9 +382,8 @@ public struct JPEGLSContextModel: Sendable { public func getErrorCorrection(contextIndex: Int, k: Int) -> Int { guard k == 0 && near == 0 else { return 0 } guard contextIndex >= 0 && contextIndex < Self.regularContextCount else { return 0 } - let b = contextB[contextIndex] - let n = contextN[contextIndex] - return (2 * b + n - 1) < 0 ? -1 : 0 + let r = contexts[contextIndex] + return (2 * r.b + r.n - 1) < 0 ? -1 : 0 } // MARK: - Run-Length Context diff --git a/Sources/JPEGLS/Core/JPEGLSTileProcessor.swift b/Sources/JPEGLS/Core/JPEGLSTileProcessor.swift deleted file mode 100644 index f3065ec..0000000 --- a/Sources/JPEGLS/Core/JPEGLSTileProcessor.swift +++ /dev/null @@ -1,177 +0,0 @@ -/// Tile-based processing for large images to reduce memory footprint -/// -/// Enables processing large images in smaller tiles, reducing peak memory usage -/// while maintaining JPEG-LS encoding/decoding correctness. -import Foundation - -/// Tile boundary information for processing -public struct TileBounds: Equatable, Sendable { - /// Starting row of the tile (inclusive) - public let rowStart: Int - /// Ending row of the tile (exclusive) - public let rowEnd: Int - /// Starting column of the tile (inclusive) - public let columnStart: Int - /// Ending column of the tile (exclusive) - public let columnEnd: Int - - /// Width of the tile in pixels - public var width: Int { columnEnd - columnStart } - - /// Height of the tile in pixels - public var height: Int { rowEnd - rowStart } - - /// Total number of pixels in the tile - public var pixelCount: Int { width * height } - - /// Creates a new tile bounds - public init(rowStart: Int, rowEnd: Int, columnStart: Int, columnEnd: Int) { - self.rowStart = rowStart - self.rowEnd = rowEnd - self.columnStart = columnStart - self.columnEnd = columnEnd - } - - /// Checks if the tile contains a specific position - public func contains(row: Int, column: Int) -> Bool { - row >= rowStart && row < rowEnd && column >= columnStart && column < columnEnd - } -} - -/// Configuration for tile-based processing -public struct TileConfiguration: Sendable { - /// Target tile width in pixels (default: 512) - public let tileWidth: Int - - /// Target tile height in pixels (default: 512) - public let tileHeight: Int - - /// Overlap between adjacent tiles in pixels for boundary handling (default: 4) - public let overlap: Int - - /// Creates a new tile configuration - /// - Parameters: - /// - tileWidth: Target width for each tile - /// - tileHeight: Target height for each tile - /// - overlap: Overlap between tiles in pixels - public init(tileWidth: Int = 512, tileHeight: Int = 512, overlap: Int = 4) { - self.tileWidth = tileWidth - self.tileHeight = tileHeight - self.overlap = overlap - } - - /// Default tile configuration - public static let `default` = TileConfiguration() -} - -/// Manages tile-based processing of large images -public struct JPEGLSTileProcessor: Sendable { - /// Width of the full image in pixels - public let imageWidth: Int - - /// Height of the full image in pixels - public let imageHeight: Int - - /// Tile configuration - public let configuration: TileConfiguration - - /// Creates a new tile processor - /// - Parameters: - /// - imageWidth: Width of the full image - /// - imageHeight: Height of the full image - /// - configuration: Tile processing configuration - public init(imageWidth: Int, imageHeight: Int, configuration: TileConfiguration = .default) { - self.imageWidth = imageWidth - self.imageHeight = imageHeight - self.configuration = configuration - } - - /// Calculates all tile bounds for processing the image - /// - Returns: Array of tile bounds covering the entire image - public func calculateTiles() -> [TileBounds] { - var tiles: [TileBounds] = [] - - let tileWidth = configuration.tileWidth - let tileHeight = configuration.tileHeight - - var rowStart = 0 - while rowStart < imageHeight { - let rowEnd = min(rowStart + tileHeight, imageHeight) - - var columnStart = 0 - while columnStart < imageWidth { - let columnEnd = min(columnStart + tileWidth, imageWidth) - - tiles.append(TileBounds( - rowStart: rowStart, - rowEnd: rowEnd, - columnStart: columnStart, - columnEnd: columnEnd - )) - - columnStart = columnEnd - } - - rowStart = rowEnd - } - - return tiles - } - - /// Calculates tile bounds with overlap for boundary handling - /// - Returns: Array of tile bounds with overlap regions - public func calculateTilesWithOverlap() -> [TileBounds] { - var tiles: [TileBounds] = [] - - let tileWidth = configuration.tileWidth - let tileHeight = configuration.tileHeight - let overlap = configuration.overlap - - var rowStart = 0 - while rowStart < imageHeight { - let rowEnd = min(rowStart + tileHeight, imageHeight) - let rowStartWithOverlap = max(0, rowStart - overlap) - let rowEndWithOverlap = min(imageHeight, rowEnd + overlap) - - var columnStart = 0 - while columnStart < imageWidth { - let columnEnd = min(columnStart + tileWidth, imageWidth) - let columnStartWithOverlap = max(0, columnStart - overlap) - let columnEndWithOverlap = min(imageWidth, columnEnd + overlap) - - tiles.append(TileBounds( - rowStart: rowStartWithOverlap, - rowEnd: rowEndWithOverlap, - columnStart: columnStartWithOverlap, - columnEnd: columnEndWithOverlap - )) - - columnStart = columnEnd - } - - rowStart = rowEnd - } - - return tiles - } - - /// Gets the number of tiles needed to process the image - public func tileCount() -> Int { - let tilesPerRow = (imageWidth + configuration.tileWidth - 1) / configuration.tileWidth - let tilesPerColumn = (imageHeight + configuration.tileHeight - 1) / configuration.tileHeight - return tilesPerRow * tilesPerColumn - } - - /// Estimates memory savings from tile-based processing - /// - Parameter bytesPerPixel: Number of bytes per pixel - /// - Returns: Estimated memory reduction ratio (0.0 to 1.0) - public func estimateMemorySavings(bytesPerPixel: Int) -> Double { - let fullImageMemory = imageWidth * imageHeight * bytesPerPixel - let tileMemory = configuration.tileWidth * configuration.tileHeight * bytesPerPixel - - guard fullImageMemory > 0 else { return 0.0 } - - let savings = 1.0 - (Double(tileMemory) / Double(fullImageMemory)) - return max(0.0, min(1.0, savings)) - } -} diff --git a/Sources/JPEGLS/Core/PlatformProtocols.swift b/Sources/JPEGLS/Core/PlatformProtocols.swift deleted file mode 100644 index ce94ce8..0000000 --- a/Sources/JPEGLS/Core/PlatformProtocols.swift +++ /dev/null @@ -1,182 +0,0 @@ -/// Platform abstraction protocols for JPEG-LS implementation. -/// -/// These protocols define the interface for platform-specific optimizations, -/// allowing the JPEG-LS codec to leverage hardware acceleration on different -/// architectures while maintaining a clean separation of concerns. - -import Foundation - -// MARK: - Platform Accelerator Protocol - -/// Protocol defining platform-specific acceleration capabilities. -/// -/// Implementations of this protocol provide optimized routines for -/// JPEG-LS operations on specific hardware architectures (e.g., ARM64, x86-64). -public protocol PlatformAccelerator: Sendable { - /// The name of the platform (e.g., "ARM64", "x86-64") - static var platformName: String { get } - - /// Returns true if this accelerator is supported on the current hardware. - static var isSupported: Bool { get } - - /// Compute gradients for three neighboring pixels. - /// - /// Gradients are used in JPEG-LS for context determination and prediction. - /// - /// - Parameters: - /// - a: North pixel value - /// - b: West pixel value - /// - c: Northwest pixel value - /// - Returns: A tuple of three gradients (D1, D2, D3) - func computeGradients(a: Int, b: Int, c: Int) -> (d1: Int, d2: Int, d3: Int) - - /// Compute the Median Edge Detector (MED) prediction. - /// - /// The MED predictor is the core prediction algorithm in JPEG-LS. - /// - /// - Parameters: - /// - a: North pixel value - /// - b: West pixel value - /// - c: Northwest pixel value - /// - Returns: The predicted pixel value - func medPredictor(a: Int, b: Int, c: Int) -> Int - - /// Quantize gradients to context indices. - /// - /// Gradient quantization maps continuous gradient values to discrete - /// context bins for context-adaptive coding. - /// - /// - Parameters: - /// - d1: First gradient - /// - d2: Second gradient - /// - d3: Third gradient - /// - t1: Quantization threshold 1 - /// - t2: Quantization threshold 2 - /// - t3: Quantization threshold 3 - /// - Returns: A tuple of three quantized gradient values (Q1, Q2, Q3) - func quantizeGradients(d1: Int, d2: Int, d3: Int, t1: Int, t2: Int, t3: Int) -> (q1: Int, q2: Int, q3: Int) -} - -// MARK: - Default Implementation - -/// Default scalar implementation of platform acceleration. -/// -/// This implementation provides reference algorithms without hardware-specific -/// optimizations. It serves as a fallback and reference for platform-specific -/// implementations. -public struct ScalarAccelerator: PlatformAccelerator { - public static let platformName = "Scalar" - public static let isSupported = true - - /// Initialize a scalar accelerator - public init() {} - - /// Compute gradients for JPEG-LS context modelling - /// - /// Calculates the three gradients used in context determination: - /// - d1 = b - c (horizontal gradient) - /// - d2 = a - c (vertical gradient) - /// - d3 = c - a (diagonal gradient) - /// - /// - Parameters: - /// - a: Left pixel value - /// - b: Top pixel value - /// - c: Top-left pixel value - /// - Returns: Tuple of three gradients (d1, d2, d3) - public func computeGradients(a: Int, b: Int, c: Int) -> (d1: Int, d2: Int, d3: Int) { - let d1 = b - c - let d2 = a - c - let d3 = c - a - return (d1, d2, d3) - } - - /// Compute MED (Median Edge Detector) prediction - /// - /// Implements the non-linear predictor used in JPEG-LS regular mode: - /// - If c >= max(a, b): return min(a, b) - /// - If c <= min(a, b): return max(a, b) - /// - Otherwise: return a + b - c - /// - /// - Parameters: - /// - a: Left pixel value - /// - b: Top pixel value - /// - c: Top-left pixel value - /// - Returns: Predicted pixel value - public func medPredictor(a: Int, b: Int, c: Int) -> Int { - // MED predictor: median of three values - if c >= max(a, b) { - return min(a, b) - } else if c <= min(a, b) { - return max(a, b) - } else { - return a + b - c - } - } - - /// Quantize gradients for context index computation - /// - /// Maps each gradient to a quantized value in range [-4, 4] using - /// threshold parameters T1, T2, T3 per JPEG-LS standard. - /// - /// - Parameters: - /// - d1: First gradient (horizontal) - /// - d2: Second gradient (vertical) - /// - d3: Third gradient (diagonal) - /// - t1: Threshold 1 (smallest) - /// - t2: Threshold 2 (medium) - /// - t3: Threshold 3 (largest) - /// - Returns: Tuple of three quantized gradients (q1, q2, q3) - public func quantizeGradients(d1: Int, d2: Int, d3: Int, t1: Int, t2: Int, t3: Int) -> (q1: Int, q2: Int, q3: Int) { - func quantize(_ d: Int, t1: Int, t2: Int, t3: Int) -> Int { - if d <= -t3 { - return -4 - } else if d <= -t2 { - return -3 - } else if d <= -t1 { - return -2 - } else if d < 0 { - return -1 - } else if d == 0 { - return 0 - } else if d < t1 { - return 1 - } else if d < t2 { - return 2 - } else if d < t3 { - return 3 - } else { - return 4 - } - } - - return (quantize(d1, t1: t1, t2: t2, t3: t3), - quantize(d2, t1: t1, t2: t2, t3: t3), - quantize(d3, t1: t1, t2: t2, t3: t3)) - } -} - -// MARK: - Platform Selection - -/// Selects the optimal platform accelerator for the current hardware. -/// -/// This function returns the most efficient accelerator implementation -/// available on the current system, preferring hardware-accelerated -/// implementations when available. -/// -/// - Returns: An instance of the optimal platform accelerator -public func selectPlatformAccelerator() -> any PlatformAccelerator { - #if arch(arm64) - // Check if ARM64 NEON accelerator is available - if ARM64Accelerator.isSupported { - return ARM64Accelerator() - } - #elseif arch(x86_64) - // Check if x86-64 SIMD accelerator is available - if X86_64Accelerator.isSupported { - return X86_64Accelerator() - } - #endif - - // Fallback to scalar implementation - return ScalarAccelerator() -} diff --git a/Sources/JPEGLS/Decoder/JPEGLSParser.swift b/Sources/JPEGLS/Decoder/JPEGLSParser.swift index f65e31c..228a0b5 100644 --- a/Sources/JPEGLS/Decoder/JPEGLSParser.swift +++ b/Sources/JPEGLS/Decoder/JPEGLSParser.swift @@ -46,7 +46,22 @@ public struct JPEGLSParseResult: Sendable { /// followed by a single byte containing the transform ID. The decoder reads /// this marker and applies the corresponding inverse transform after decoding. public let colorTransformation: JPEGLSColorTransformation - + + /// Byte ranges of each scan's entropy-coded body within the parsed data, + /// one per scan header, in scan order. The parser already walks every + /// scan body to find the terminating marker; recording the offsets here + /// lets the decoder slice the original data directly instead of doing a + /// second full-file marker walk. Empty when the result was constructed + /// without parsing (the decoder then falls back to its own walk). + public let scanDataRanges: [Range] + + /// The restart interval (DRI value) in effect at each scan's SOS marker, + /// one per scan header, in scan order (0 = none). Per T.81 B.2.4.4 a + /// DRI segment between scans applies to following scans only, so a + /// single file-global value cannot represent multi-scan streams that + /// (re)define it. Empty when the result was constructed without parsing. + public let scanRestartIntervals: [Int] + /// Initialize parse result /// /// - Parameters: @@ -58,6 +73,8 @@ public struct JPEGLSParseResult: Sendable { /// - applicationMarkers: Application markers /// - comments: Comment data /// - colorTransformation: Colour transform from APP8 "mrfx" marker (default: .none) + /// - scanDataRanges: Byte ranges of each scan body (default: empty) + /// - scanRestartIntervals: Restart interval in effect at each SOS (default: empty) public init( frameHeader: JPEGLSFrameHeader, scanHeaders: [JPEGLSScanHeader], @@ -66,7 +83,9 @@ public struct JPEGLSParseResult: Sendable { mappingTables: [UInt8: JPEGLSMappingTable] = [:], applicationMarkers: [(marker: JPEGLSMarker, data: Data)] = [], comments: [Data] = [], - colorTransformation: JPEGLSColorTransformation = .none + colorTransformation: JPEGLSColorTransformation = .none, + scanDataRanges: [Range] = [], + scanRestartIntervals: [Int] = [] ) { self.frameHeader = frameHeader self.scanHeaders = scanHeaders @@ -76,6 +95,8 @@ public struct JPEGLSParseResult: Sendable { self.applicationMarkers = applicationMarkers self.comments = comments self.colorTransformation = colorTransformation + self.scanDataRanges = scanDataRanges + self.scanRestartIntervals = scanRestartIntervals } } @@ -116,7 +137,9 @@ public final class JPEGLSParser { var extendedWidth: Int? var extendedHeight: Int? var colorTransformation: JPEGLSColorTransformation = .none - + var scanDataRanges: [Range] = [] + var scanRestartIntervals: [Int] = [] + // Parse marker segments until EOI while !reader.isAtEnd { // Read marker bytes manually to handle unknown markers @@ -175,7 +198,9 @@ public final class JPEGLSParser { mappingTables: mappingTables, applicationMarkers: applicationMarkers, comments: comments, - colorTransformation: colorTransformation + colorTransformation: colorTransformation, + scanDataRanges: scanDataRanges, + scanRestartIntervals: scanRestartIntervals ) case .startOfFrameJPEGLS: @@ -196,28 +221,46 @@ public final class JPEGLSParser { } let scanHeader = try parseScanHeader(frameHeader: frame) scanHeaders.append(scanHeader) - - // Skip scan data until we hit a marker. + // Record the restart interval in effect at this SOS (a DRI + // between scans applies to following scans only, T.81 B.2.4.4). + scanRestartIntervals.append(restartInterval ?? 0) + + // Skip scan data until we hit a marker, recording the body's + // byte range so the decoder can slice it without a second + // full-file walk. // Per ISO 14495-1 §9.1, a byte following 0xFF with MSB = 0 (value < 0x80) // is a stuffed byte; with MSB = 1 (value ≥ 0x80) it is a real marker. + let scanStart = reader.currentPosition + var scanEnd: Int? = nil while !reader.isAtEnd { let byte = try reader.readByte() if byte == JPEGLSMarker.markerPrefix { // Check next byte to determine if it's stuffing or a real marker if let nextByte = reader.peekByte() { + if nextByte >= JPEGLSMarker.restart0.rawValue + && nextByte <= JPEGLSMarker.restart7.rawValue { + // Restart marker (FFD0–FFD7) inside the scan + // body: part of the entropy-coded segment, not + // a scan terminator. Consume and continue. + _ = try reader.readByte() + continue + } if nextByte >= 0x80 { // Real marker — back up to re-read the FF byte in the outer loop try reader.seek(to: reader.currentPosition - 1) + scanEnd = reader.currentPosition break } // nextByte < 0x80: stuffed byte — consume it and continue _ = try reader.readByte() } else { - // End of stream + // End of stream: a trailing lone 0xFF is not scan data + scanEnd = reader.currentPosition - 1 break } } } + scanDataRanges.append(scanStart..<(scanEnd ?? reader.currentPosition)) case .jpegLSExtension: // Parse JPEG-LS extension @@ -426,7 +469,15 @@ public final class JPEGLSParser { extendedHeight: inout Int? ) throws { let length = try reader.readUInt16() - + // Length includes the 2-byte length field and the 1-byte type that + // follows; anything shorter is structurally invalid (and would make + // the skip count below negative). + guard length >= 3 else { + throw JPEGLSError.invalidBitstreamStructure( + reason: "LSE segment length \(length) is shorter than its own header" + ) + } + // Read extension type let extensionTypeByte = try reader.readByte() guard let extensionType = JPEGLSExtensionType(rawValue: extensionTypeByte) else { diff --git a/Sources/JPEGLS/Decoder/JPEGLSRegularModeDecoder.swift b/Sources/JPEGLS/Decoder/JPEGLSRegularModeDecoder.swift index 2fc1ce3..ae5e90a 100644 --- a/Sources/JPEGLS/Decoder/JPEGLSRegularModeDecoder.swift +++ b/Sources/JPEGLS/Decoder/JPEGLSRegularModeDecoder.swift @@ -34,7 +34,13 @@ public struct JPEGLSRegularModeDecoder: Sendable { /// Quantization factor: qbpp = (NEAR == 0) ? 0 : ((NEAR << 1) | 1) private let qbpp: Int - + + /// Gradient quantisation lookup table for the inner range (−T3, T3), + /// indexed by `gradient + gradientTableOffset`. Built in `init` by + /// evaluating the reference branch chain, so it is bit-identical to it. + private let gradientTable: [Int] + private let gradientTableOffset: Int + // MARK: - Initialization /// Initialize regular mode decoder with preset parameters. @@ -61,6 +67,39 @@ public struct JPEGLSRegularModeDecoder: Sendable { } else { self.range = (parameters.maxValue + 2 * near) / qbpp + 1 } + + // Build the gradient quantisation table for the inner range from the + // reference branch chain (values at/beyond ±T3 are handled by the + // early exits in `quantizeGradient`). + let t3 = parameters.threshold3 + self.gradientTableOffset = t3 + var table = [Int](repeating: 0, count: 2 * t3 + 1) + for i in 0.. Int { + if gradient <= -t3 { return -4 } + if gradient <= -t2 { return -3 } + if gradient <= -t1 { return -2 } + if gradient < -near { return -1 } + if gradient <= near { return 0 } + if gradient < t1 { return 1 } + if gradient < t2 { return 2 } + if gradient < t3 { return 3 } + return 4 } // MARK: - Gradient Computation @@ -101,17 +140,11 @@ public struct JPEGLSRegularModeDecoder: Sendable { /// - Parameter gradient: Raw gradient value /// - Returns: Quantized gradient in range [-4, 4] public func quantizeGradient(_ gradient: Int) -> Int { - // Quantization per ITU-T.87 Table A.7 / CharLS quantize_gradient_org. - // Uses strict less-than for upper threshold boundaries. + // Gradients at/beyond ±T3 always map to ±4; the inner range uses the + // pre-computed table (built from the reference branch chain). if gradient <= -parameters.threshold3 { return -4 } - if gradient <= -parameters.threshold2 { return -3 } - if gradient <= -parameters.threshold1 { return -2 } - if gradient < -near { return -1 } - if gradient <= near { return 0 } - if gradient < parameters.threshold1 { return 1 } - if gradient < parameters.threshold2 { return 2 } - if gradient < parameters.threshold3 { return 3 } - return 4 + if gradient >= parameters.threshold3 { return 4 } + return gradientTable[gradient + gradientTableOffset] } // MARK: - MED Prediction @@ -302,21 +335,43 @@ public struct JPEGLSRegularModeDecoder: Sendable { ) -> DecodedPixel { // Step 1: Compute local gradients let (d1, d2, d3) = computeGradients(a: a, b: b, c: c, d: d) - + // Step 2: Quantize gradients let q1 = quantizeGradient(d1) let q2 = quantizeGradient(d2) let q3 = quantizeGradient(d3) - + // Step 3: Compute context index and sign - let contextIndex = context.computeContextIndex(q1: q1, q2: q2, q3: q3) - let sign = context.computeContextSign(q1: q1, q2: q2, q3: q3) - + let (contextIndex, sign) = context.computeContextIndexAndSign(q1: q1, q2: q2, q3: q3) + + return decodePixel( + mappedError: mappedError, a: a, b: b, c: c, + contextIndex: contextIndex, sign: sign, + biasC: context.getC(contextIndex: contextIndex), + errorCorrection: errorCorrection + ) + } + + /// Decode a single pixel in regular mode with a precomputed context. + /// + /// Identical to `decodePixel(mappedError:a:b:c:d:context:errorCorrection:)` + /// from step 4 onward; the caller supplies the context index, sign, and + /// bias correction it already derived (the scan loop fetches the full + /// per-pixel coding state in one context-record load). + public func decodePixel( + mappedError: Int, + a: Int, + b: Int, + c: Int, + contextIndex: Int, + sign: Int, + biasC: Int, + errorCorrection: Int = 0 + ) -> DecodedPixel { // Step 4: Compute MED prediction let basePrediction = computeMEDPrediction(a: a, b: b, c: c) - + // Step 5: Apply bias correction - let biasC = context.getC(contextIndex: contextIndex) let correctedPrediction = applyBiasCorrection( prediction: basePrediction, biasC: biasC, diff --git a/Sources/JPEGLS/Encoder/JPEGLSPixelBuffer.swift b/Sources/JPEGLS/Encoder/JPEGLSPixelBuffer.swift index dda42e3..77de61a 100644 --- a/Sources/JPEGLS/Encoder/JPEGLSPixelBuffer.swift +++ b/Sources/JPEGLS/Encoder/JPEGLSPixelBuffer.swift @@ -116,7 +116,18 @@ public struct MultiComponentImageData: Sendable { self.components = components self.frameHeader = frameHeader } - + + /// Trusted initializer for decoder-produced components. + /// + /// Skips the O(W·H) dimension and range validation of the public + /// initializer: the decode pipeline clamps every sample to [0, MAXVAL] + /// by construction and allocates rows at exact scan dimensions. Internal + /// only — public API always goes through the validating initializer. + internal init(uncheckedComponents components: [ComponentData], frameHeader: JPEGLSFrameHeader) { + self.components = components + self.frameHeader = frameHeader + } + /// Create grayscale image data /// /// ```swift diff --git a/Sources/JPEGLS/Encoder/JPEGLSRegularMode.swift b/Sources/JPEGLS/Encoder/JPEGLSRegularMode.swift index 85f6f6d..51edb97 100644 --- a/Sources/JPEGLS/Encoder/JPEGLSRegularMode.swift +++ b/Sources/JPEGLS/Encoder/JPEGLSRegularMode.swift @@ -373,40 +373,60 @@ public struct JPEGLSRegularMode: Sendable { ) -> EncodedPixel { // Step 1: Compute local gradients let (d1, d2, d3) = computeGradients(a: a, b: b, c: c, d: d) - + // Step 2: Quantize gradients let q1 = quantizeGradient(d1) let q2 = quantizeGradient(d2) let q3 = quantizeGradient(d3) - + // Step 3: Compute context index and sign - let contextIndex = context.computeContextIndex(q1: q1, q2: q2, q3: q3) - let sign = context.computeContextSign(q1: q1, q2: q2, q3: q3) - + let (contextIndex, sign) = context.computeContextIndexAndSign(q1: q1, q2: q2, q3: q3) + + return encodePixel( + actual: actual, a: a, b: b, c: c, + contextIndex: contextIndex, sign: sign, context: context + ) + } + + /// Encode a single pixel in regular mode with a precomputed context. + /// + /// Identical to `encodePixel(actual:a:b:c:d:context:)` from step 4 + /// onward; the caller supplies the context index and sign it already + /// derived from the quantized gradients (the scan loop computes them + /// for the run-mode test, so recomputing here would quantize every + /// gradient twice per pixel). + public func encodePixel( + actual: Int, + a: Int, + b: Int, + c: Int, + contextIndex: Int, + sign: Int, + context: JPEGLSContextModel + ) -> EncodedPixel { + // Steps 5/7/7a inputs: bias C[Q], Golomb k, and the k=0 error + // correction from a single context-record load. + let (biasC, k, errorCorrection) = context.pixelCodingState(contextIndex: contextIndex) + // Step 4: Compute MED prediction let basePrediction = computeMEDPrediction(a: a, b: b, c: c) - + // Step 5: Apply bias correction - let biasC = context.getC(contextIndex: contextIndex) let correctedPrediction = applyBiasCorrection( prediction: basePrediction, biasC: biasC, sign: sign ) - + // Step 6: Compute quantised (near-lossless) or exact (lossless) prediction error let quantisedError = computePredictionError(actual: actual, prediction: correctedPrediction) - + // Step 6a: Apply sign to normalise the error per ITU-T.87 Section 4.3.3. // When the context sign is negative the error is negated so that the encoded // error is always relative to the normalised (positive-sign) context. let error = sign * quantisedError - - // Step 7: Get Golomb parameter k from context (needed for error correction) - let k = context.computeGolombParameter(contextIndex: contextIndex) - + // Step 7a: Apply error correction XOR per ITU-T.87 §A.4.1 - let errorCorrection = context.getErrorCorrection(contextIndex: contextIndex, k: k) let correctedError = error ^ errorCorrection // Step 8: Map to non-negative for Golomb coding diff --git a/Sources/JPEGLS/Encoder/JPEGLSRunMode.swift b/Sources/JPEGLS/Encoder/JPEGLSRunMode.swift index e92fa98..26269c3 100644 --- a/Sources/JPEGLS/Encoder/JPEGLSRunMode.swift +++ b/Sources/JPEGLS/Encoder/JPEGLSRunMode.swift @@ -93,23 +93,38 @@ public struct JPEGLSRunMode: Sendable { startIndex: Int, runValue: Int ) -> Int { - var runLength = 0 let limit = pixels.count - - // Scan ahead to count matching pixels. + // For near-lossless (NEAR > 0) a pixel is part of the run when // |pixel − runValue| ≤ NEAR; for lossless (NEAR = 0) this reduces - // to exact equality. - for i in startIndex.. Int in + var i = startIndex + while i + 4 <= limit { + if (buf[i] != runValue) || (buf[i + 1] != runValue) + || (buf[i + 2] != runValue) || (buf[i + 3] != runValue) { + break + } + i += 4 + } + while i < limit && buf[i] == runValue { + i += 1 + } + return i - startIndex } } - - return runLength + + return pixels.withUnsafeBufferPointer { buf -> Int in + var i = startIndex + while i < limit && abs(buf[i] - runValue) <= near { + i += 1 + } + return i - startIndex + } } // MARK: - J[RUNindex] Mapping diff --git a/Sources/JPEGLS/JPEGLSDecoder.swift b/Sources/JPEGLS/JPEGLSDecoder.swift index cf23029..79536e2 100644 --- a/Sources/JPEGLS/JPEGLSDecoder.swift +++ b/Sources/JPEGLS/JPEGLSDecoder.swift @@ -6,6 +6,38 @@ import Foundation +/// Lock-protected accumulator for parallel restart-interval decoding: +/// each worker stores its interval's decoded plane at its own index, and +/// the first error (if any) wins. +private final class IntervalDecodeResults: @unchecked Sendable { + private let lock = NSLock() + private var regions: [[UInt16]?] + private var firstError: Error? + + init(count: Int) { + self.regions = Array(repeating: nil, count: count) + } + + func set(_ region: [UInt16], at index: Int) { + lock.lock() + regions[index] = region + lock.unlock() + } + + func fail(_ error: Error) { + lock.lock() + if firstError == nil { firstError = error } + lock.unlock() + } + + func finish() throws -> [[UInt16]?] { + lock.lock() + defer { lock.unlock() } + if let firstError { throw firstError } + return regions + } +} + /// High-level JPEG-LS decoder /// /// Decodes JPEG-LS encoded data to multi-component image data per ITU-T.87. @@ -49,10 +81,16 @@ public struct JPEGLSDecoder: Sendable { /// - Returns: Decoded multi-component image data /// - Throws: `JPEGLSError` if decoding fails public func decode(_ data: Data) throws -> MultiComponentImageData { + // Rebase slices: all offset bookkeeping below (parser scan ranges, + // the fallback marker walk) is zero-based, but a Data slice keeps + // its parent's indices — subscripting it with zero-based ranges + // would silently read the wrong bytes or trap. + let data = data.startIndex == 0 ? data : Data(data) + // Parse JPEG-LS structure let parser = JPEGLSParser(data: data) let parseResult = try parser.parse() - + // Get preset parameters (default or custom). The NEAR parameter affects // the default thresholds per ITU-T.87 Table C.2, so it must be supplied. let near = parseResult.scanHeaders.first?.near ?? 0 @@ -60,24 +98,94 @@ public struct JPEGLSDecoder: Sendable { bitsPerSample: parseResult.frameHeader.bitsPerSample, near: near ) - - // Extract scan data from bitstream - let scanDataList = try extractScanData(from: data, parseResult: parseResult) - + + // MAXVAL must satisfy 0 < MAXVAL ≤ 2^P − 1 (ITU-T.87 C.2.4.1.1). + // A larger value would let the decode pipeline's clamps produce + // samples outside the frame's sample range. + let sampleCap = (1 << parseResult.frameHeader.bitsPerSample) - 1 + guard parameters.maxValue <= sampleCap else { + throw JPEGLSError.invalidBitstreamStructure( + reason: "MAXVAL \(parameters.maxValue) exceeds 2^P−1 = \(sampleCap) for \(parseResult.frameHeader.bitsPerSample)-bit samples" + ) + } + + // Reject dimension products that overflow before any buffer sizing + // arithmetic runs on them (a crafted LSE type-4 segment can declare + // axes up to 2^32 − 1 each). + let frameHeader = parseResult.frameHeader + let (_, dimensionOverflow) = frameHeader.width.multipliedReportingOverflow(by: frameHeader.height) + guard !dimensionOverflow else { + throw JPEGLSError.invalidDimensions(width: frameHeader.width, height: frameHeader.height) + } + + // Slice scan data using the ranges the parser recorded during its + // walk; fall back to a marker walk only for externally-constructed + // parse results without ranges. + let scanDataList: [Data] + if parseResult.scanDataRanges.count == parseResult.scanHeaders.count { + scanDataList = parseResult.scanDataRanges.map { Data(data[$0]) } + } else { + scanDataList = try extractScanData(from: data, parseResult: parseResult) + } + // Validate we have the expected number of scans guard scanDataList.count == parseResult.scanHeaders.count else { throw JPEGLSError.invalidBitstreamStructure( reason: "Scan data count (\(scanDataList.count)) doesn't match scan header count (\(parseResult.scanHeaders.count))" ) } - + // Decode based on interleave mode guard let firstScanHeader = parseResult.scanHeaders.first else { throw JPEGLSError.invalidBitstreamStructure(reason: "No scan headers found") } - + var decodedComponents: [MultiComponentImageData.ComponentData] - + + // Per-scan restart intervals: the DRI value in effect at each SOS + // (T.81 B.2.4.4 — a DRI between scans applies to following scans + // only). Fall back to the file-global value for externally-built + // parse results. + let scanRestartIntervals: [Int] + if parseResult.scanRestartIntervals.count == parseResult.scanHeaders.count { + scanRestartIntervals = parseResult.scanRestartIntervals + } else { + scanRestartIntervals = Array( + repeating: parseResult.restartInterval ?? 0, + count: parseResult.scanHeaders.count + ) + } + + // An interval only takes effect when it is shorter than the frame + // (DRI ≥ height produces zero RST markers — the scan body is + // identical to a no-DRI stream). + func restartActive(_ interval: Int) -> Bool { + interval > 0 && interval < frameHeader.height + } + + // Restart intervals (DRI) are supported for non-interleaved scans; + // reject interleaved streams whose scans would actually contain + // restart markers rather than decoding them incorrectly. + if firstScanHeader.interleaveMode != .none, + let interval = scanRestartIntervals.first, restartActive(interval) { + throw JPEGLSError.invalidBitstreamStructure( + reason: "Restart intervals are not supported with \(firstScanHeader.interleaveMode) interleave mode" + ) + } + + // Scans without an active restart interval must not contain RSTm + // markers: the bit reader would otherwise absorb them as entropy + // data and decode garbage silently (e.g. a corrupted stuffed byte). + for (index, scanData) in scanDataList.enumerated() { + let interval = index < scanRestartIntervals.count ? scanRestartIntervals[index] : 0 + let active = restartActive(interval) && firstScanHeader.interleaveMode == .none + if !active && Self.scanBodyContainsRestartMarker(scanData) { + throw JPEGLSError.invalidBitstreamStructure( + reason: "Restart marker found in scan \(index) without an active restart interval" + ) + } + } + switch firstScanHeader.interleaveMode { case .none: // Non-interleaved: one scan per component @@ -86,7 +194,8 @@ public struct JPEGLSDecoder: Sendable { scanHeaders: parseResult.scanHeaders, scanDataList: scanDataList, parameters: parameters, - mappingTables: parseResult.mappingTables + mappingTables: parseResult.mappingTables, + restartIntervals: scanRestartIntervals ) case .line: @@ -124,7 +233,30 @@ public struct JPEGLSDecoder: Sendable { ) } - // Create result + // Every frame component must have been decoded (the parser accepts + // EOI after any number of scans; a short non-interleaved stream + // would otherwise yield fewer components than the frame declares). + guard decodedComponents.count == parseResult.frameHeader.componentCount else { + throw JPEGLSError.invalidBitstreamStructure( + reason: "Decoded \(decodedComponents.count) component(s) but the frame header declares \(parseResult.frameHeader.componentCount)" + ) + } + + // Create result. The decode pipeline clamps every sample to + // [0, MAXVAL ≤ 2^P−1] by construction and builds rows at exact scan + // dimensions, so the O(W·H) re-validation in the public initializer + // adds no information — skip it unless a post-processing step with + // unbounded outputs ran (mapping tables, colour transform) or the + // frame is sub-sampled (which the scan decoders do not handle). + let uniformSampling = parseResult.frameHeader.components.allSatisfy { + $0.horizontalSamplingFactor == 1 && $0.verticalSamplingFactor == 1 + } + if uniformSampling && parseResult.mappingTables.isEmpty && colorTransformation == .none { + return MultiComponentImageData( + uncheckedComponents: decodedComponents, + frameHeader: parseResult.frameHeader + ) + } return try MultiComponentImageData( components: decodedComponents, frameHeader: parseResult.frameHeader @@ -195,11 +327,18 @@ public struct JPEGLSDecoder: Sendable { // Find end of scan data (next real marker, not stuffed byte). // Per ISO 14495-1 §9.1: a byte following 0xFF with MSB = 0 (< 0x80) - // is a stuffed byte; MSB = 1 (≥ 0x80) is a real marker. + // is a stuffed byte; MSB = 1 (≥ 0x80) is a real marker — except + // restart markers (FFD0–FFD7), which are part of the scan body. var scanDataEnd = position while scanDataEnd < data.count - 1 { if data[scanDataEnd] == 0xFF { let nextByte = data[scanDataEnd + 1] + if nextByte >= JPEGLSMarker.restart0.rawValue + && nextByte <= JPEGLSMarker.restart7.rawValue { + // Restart marker inside the scan — keep walking. + scanDataEnd += 2 + continue + } if nextByte >= 0x80 { // Real marker — scan data ends here break @@ -242,22 +381,22 @@ public struct JPEGLSDecoder: Sendable { scanHeaders: [JPEGLSScanHeader], scanDataList: [Data], parameters: JPEGLSPresetParameters, - mappingTables: [UInt8: JPEGLSMappingTable] = [:] + mappingTables: [UInt8: JPEGLSMappingTable] = [:], + restartIntervals: [Int] = [] ) throws -> [MultiComponentImageData.ComponentData] { var components: [MultiComponentImageData.ComponentData] = [] - + for (scanIndex, scanHeader) in scanHeaders.enumerated() { - let scanData = scanDataList[scanIndex] - let reader = JPEGLSBitstreamReader(data: scanData) - - // Decode this component + // Decode this component, with the restart interval in effect at + // this scan's SOS (a DRI between scans applies to later scans only) var pixels = try decodeComponent( - reader: reader, + scanData: scanDataList[scanIndex], width: frameHeader.width, height: frameHeader.height, scanHeader: scanHeader, parameters: parameters, - bitsPerSample: frameHeader.bitsPerSample + bitsPerSample: frameHeader.bitsPerSample, + restartInterval: scanIndex < restartIntervals.count ? restartIntervals[scanIndex] : 0 ) // Apply mapping table lookup if the component references one @@ -430,7 +569,7 @@ public struct JPEGLSDecoder: Sendable { // Note: RUNindex is NOT reset per line. Per ITU-T.87 §A.7.1 and CharLS, // RUNindex persists across scan lines; it is only initialised to 0 at scan start. // Capture and advance edge values per component. - var edgesForThisRow = prevRowEdges + let edgesForThisRow = prevRowEdges for ci in 0.. 0 { prevRowEdges[ci] = componentPixels[ci][row - 1][0] } } @@ -535,12 +674,16 @@ public struct JPEGLSDecoder: Sendable { row: row, col: col, width: frameHeader.width, prevRowEdge: edgesForThisRow[componentIndex] ) + let (d1, d2, d3) = decoder.computeGradients(a: a, b: b, c: c, d: d) let pixel = try decodeSinglePixel( reader: reader, decoder: decoder, runDecoder: runDecoder, context: &context, - a: a, b: b, c: c, d: d, + a: a, b: b, c: c, + q1: decoder.quantizeGradient(d1), + q2: decoder.quantizeGradient(d2), + q3: decoder.quantizeGradient(d3), parameters: parameters, near: scanHeader.near, limit: limit, @@ -571,94 +714,290 @@ public struct JPEGLSDecoder: Sendable { /// Decode a single component (used for non-interleaved mode) private func decodeComponent( - reader: JPEGLSBitstreamReader, + scanData: Data, width: Int, height: Int, scanHeader: JPEGLSScanHeader, parameters: JPEGLSPresetParameters, - bitsPerSample: Int + bitsPerSample: Int, + restartInterval: Int = 0 ) throws -> [[Int]] { - let decoder = try JPEGLSRegularModeDecoder(parameters: parameters, near: scanHeader.near) - let runDecoder = try JPEGLSRunModeDecoder(parameters: parameters, near: scanHeader.near) - var context = try JPEGLSContextModel(parameters: parameters, near: scanHeader.near) let (limit, qbppBits) = computeGolombLimit(parameters: parameters, near: scanHeader.near, bitsPerSample: bitsPerSample) - - // Initialize pixel buffer - var pixels = Array(repeating: Array(repeating: 0, count: width), count: height) - - // Track the left-edge value for boundary Rc at col=0. - // In CharLS this is previous_line[0], which equals the first pixel of - // the row decoded TWO iterations ago (0 for rows 0 and 1). - var prevRowEdge = 0 - - // Decode pixels in raster order - for row in 0.. 0 { - prevRowEdge = pixels[row - 1][0] + let near = scanHeader.near + + if restartInterval > 0 && restartInterval < height { + // Restart intervals: split the scan body at its RSTm markers + // (validating the D0–D7 cycle) and decode the independent + // intervals concurrently — each restarts coding exactly as at + // scan start, so a fresh region decode per interval is correct + // by construction. + let segments = try splitScanDataAtRestartMarkers(scanData) + let intervalCount = (height + restartInterval - 1) / restartInterval + guard segments.count == intervalCount else { + throw JPEGLSError.invalidBitstreamStructure( + reason: "Expected \(intervalCount) restart intervals for \(height) lines, found \(segments.count)" + ) } - var col = 0 - while col < width { - // Get neighbor pixels - let (a, b, c, d) = getNeighbors(pixels: pixels, row: row, col: col, width: width, prevRowEdge: edgeForThisRow) - - // Check for run mode: all quantized gradients are zero - let (d1, d2, d3) = decoder.computeGradients(a: a, b: b, c: c, d: d) - let q1 = decoder.quantizeGradient(d1) - let q2 = decoder.quantizeGradient(d2) - let q3 = decoder.quantizeGradient(d3) - - if q1 == 0 && q2 == 0 && q3 == 0 { - // Run mode: decode run of pixels with value = a (the run value) - let runResult = try decodeRun( - reader: reader, - runDecoder: runDecoder, - context: &context, - runValue: a, - row: row, - col: col, - previousRow: row > 0 ? pixels[row - 1] : nil, - remainingInLine: width - col, - parameters: parameters, - near: scanHeader.near, - limit: limit, - qbppBits: qbppBits - ) - - // Fill in run result - col = fillRunResult( - runResult: runResult, - pixels: &pixels, - row: row, - col: col, - width: width, - runValue: a + let results = IntervalDecodeResults(count: intervalCount) + DispatchQueue.concurrentPerform(iterations: intervalCount) { idx in + do { + let rows = min(restartInterval, height - idx * restartInterval) + let region = try decodeFlatRegion( + reader: JPEGLSBitstreamReader(data: segments[idx]), + rows: rows, width: width, + parameters: parameters, near: near, + limit: limit, qbppBits: qbppBits ) - } else { - // Regular mode - let pixel = try decodeSinglePixel( - reader: reader, - decoder: decoder, - runDecoder: runDecoder, - context: &context, - a: a, b: b, c: c, d: d, - parameters: parameters, - near: scanHeader.near, - limit: limit, - qbppBits: qbppBits + results.set(region, at: idx) + } catch { + results.fail(error) + } + } + let regions = try results.finish() + var pixels: [[Int]] = [] + pixels.reserveCapacity(height) + for (idx, region) in regions.enumerated() { + guard let region else { + throw JPEGLSError.invalidBitstreamStructure( + reason: "Restart interval \(idx) produced no rows" ) - pixels[row][col] = pixel - col += 1 } + pixels.append(contentsOf: widenFlatRows(region, width: width)) } + return pixels } - - return pixels + + let flat = try decodeFlatRegion( + reader: JPEGLSBitstreamReader(data: scanData), + rows: height, width: width, + parameters: parameters, near: near, + limit: limit, qbppBits: qbppBits + ) + return widenFlatRows(flat, width: width) + } + + /// Whether a scan body contains an RSTm marker (0xFF followed by + /// 0xD0–0xD7, skipping stuffed pairs). Used to reject stray restart + /// markers in scans that have no active restart interval. + private static func scanBodyContainsRestartMarker(_ data: Data) -> Bool { + data.withUnsafeBytes { (raw: UnsafeRawBufferPointer) -> Bool in + var i = 0 + while i + 1 < raw.count { + if raw[i] == 0xFF { + let next = raw[i + 1] + if next >= 0xD0 && next <= 0xD7 { + return true + } + if next < 0x80 { + i += 2 // stuffed pair — data + continue + } + } + i += 1 + } + return false + } + } + + /// Split a scan body into its restart-interval segments, removing the + /// RSTm markers and validating that they cycle FFD0–FFD7. Stuffed bytes + /// (0xFF followed by < 0x80) are skipped as data; a lone 0xFF before a + /// non-restart marker byte is treated as data (the encoder's final flush + /// byte may legitimately be 0xFF). + private func splitScanDataAtRestartMarkers(_ data: Data) throws -> [Data] { + let bytes = [UInt8](data) + var segments: [Data] = [] + var segmentStart = 0 + var markerIndex = 0 + var i = 0 + while i + 1 < bytes.count { + if bytes[i] == 0xFF { + let next = bytes[i + 1] + if next >= JPEGLSMarker.restart0.rawValue && next <= JPEGLSMarker.restart7.rawValue { + let expected = JPEGLSMarker.restart0.rawValue + UInt8(markerIndex % 8) + guard next == expected else { + throw JPEGLSError.invalidBitstreamStructure( + reason: "Restart marker out of sequence: expected 0xFF\(String(expected, radix: 16, uppercase: true)), found 0xFF\(String(next, radix: 16, uppercase: true))" + ) + } + segments.append(Data(bytes[segmentStart.. [[Int]] { + let rows = flat.count / width + return flat.withUnsafeBufferPointer { buf in + (0.. [Int] in + let base = row * width + return [Int](unsafeUninitializedCapacity: width) { out, count in + for i in 0.. [UInt16] { + let decoder = try JPEGLSRegularModeDecoder(parameters: parameters, near: near) + let runDecoder = try JPEGLSRunModeDecoder(parameters: parameters, near: near) + var context = try JPEGLSContextModel(parameters: parameters, near: near) + var flat = [UInt16](repeating: 0, count: width * rows) + + try flat.withUnsafeMutableBufferPointer { buf in + // Track the left-edge value for boundary Rc at col=0. + // In CharLS this is previous_line[0], which equals the first pixel + // of the row decoded TWO iterations ago (0 for rows 0 and 1). + var prevRowEdge = 0 + + // Decode pixels in raster order + for row in 0.. 0 { + prevRowEdge = Int(buf[prevBase]) + } + var col = 0 + while col < width { + // Causal neighbours per ITU-T.87 §3.2 (same boundary + // semantics as getNeighbors, over the flat plane). + let a: Int, b: Int, c: Int, d: Int + if row == 0 { + a = col == 0 ? 0 : Int(buf[rowBase + col - 1]) + b = 0; c = 0; d = 0 + } else if col == 0 { + let top = Int(buf[prevBase]) + a = top + b = top + c = edgeForThisRow + d = width > 1 ? Int(buf[prevBase + 1]) : top + } else { + a = Int(buf[rowBase + col - 1]) + b = Int(buf[prevBase + col]) + c = Int(buf[prevBase + col - 1]) + d = col + 1 < width ? Int(buf[prevBase + col + 1]) : b + } + + // Check for run mode: all quantized gradients are zero + let (d1, d2, d3) = decoder.computeGradients(a: a, b: b, c: c, d: d) + let q1 = decoder.quantizeGradient(d1) + let q2 = decoder.quantizeGradient(d2) + let q3 = decoder.quantizeGradient(d3) + + if q1 == 0 && q2 == 0 && q3 == 0 { + // Run mode: decode run of pixels with value = a. + // readRunLength clamps to remainingInLine, so the + // fill below cannot overrun the row. + let remainingInLine = width - col + let runLength = try readRunLength( + reader: reader, + runDecoder: runDecoder, + context: &context, + remainingInLine: remainingInLine + ) + if runLength > 0 { + let rv = UInt16(truncatingIfNeeded: a) + for i in (rowBase + col)..<(rowBase + col + runLength) { + buf[i] = rv + } + col += runLength + } + + if runLength < remainingInLine { + // Interrupted run: decode the interruption sample + // (per ITU-T.87 §A.7.2 / CharLS). + let ra = a + let rb = row > 0 ? Int(buf[prevBase + col]) : 0 + let riType = (abs(ra - rb) <= near) ? 1 : 0 + let k = context.computeRunInterruptionGolombK(riType: riType) + let j = runDecoder.computeJ(runIndex: context.currentRunIndex) + let adjustedLimit = limit - j - 1 + let eMappedErrorValue = try readGolombCode( + reader: reader, k: k, limit: adjustedLimit, qbppBits: qbppBits + ) + let errorValue = context.computeRunInterruptionErrorValue( + temp: eMappedErrorValue + riType, k: k, riType: riType + ) + let sample: Int + if riType == 1 { + sample = runDecoder.reconstructSample(prediction: ra, error: errorValue) + } else { + let signCorrectedError = errorValue * (rb >= ra ? 1 : -1) + sample = runDecoder.reconstructSample(prediction: rb, error: signCorrectedError) + } + context.updateRunInterruptionContext( + errorValue: errorValue, + eMappedErrorValue: eMappedErrorValue, + riType: riType + ) + // Per CharLS, decrement RUNindex AFTER the interruption pixel. + context.decrementRunIndex() + + buf[rowBase + col] = UInt16(truncatingIfNeeded: sample) + col += 1 + } + } else { + // Regular mode + let pixel = try decodeSinglePixel( + reader: reader, + decoder: decoder, + runDecoder: runDecoder, + context: &context, + a: a, b: b, c: c, + q1: q1, q2: q2, q3: q3, + parameters: parameters, + near: near, + limit: limit, + qbppBits: qbppBits + ) + buf[rowBase + col] = UInt16(truncatingIfNeeded: pixel) + col += 1 + } + } + } + } + + return flat } /// Decode a single line for a component (used for line-interleaved mode) @@ -722,13 +1061,14 @@ public struct JPEGLSDecoder: Sendable { decoder: decoder, runDecoder: runDecoder, context: &context, - a: a, b: b, c: c, d: d, + a: a, b: b, c: c, + q1: q1, q2: q2, q3: q3, parameters: parameters, near: scanHeader.near, limit: limit, qbppBits: qbppBits ) - + pixels[row][col] = pixel col += 1 } @@ -773,45 +1113,39 @@ public struct JPEGLSDecoder: Sendable { decoder: JPEGLSRegularModeDecoder, runDecoder: JPEGLSRunModeDecoder, context: inout JPEGLSContextModel, - a: Int, b: Int, c: Int, d: Int, + a: Int, b: Int, c: Int, + q1: Int, q2: Int, q3: Int, parameters: JPEGLSPresetParameters, near: Int, limit: Int, qbppBits: Int ) throws -> Int { - // Compute gradients - let (d1, d2, d3) = decoder.computeGradients(a: a, b: b, c: c, d: d) - - // Quantize gradients - let q1 = decoder.quantizeGradient(d1) - let q2 = decoder.quantizeGradient(d2) - let q3 = decoder.quantizeGradient(d3) - - // Get context - let contextIndex = context.computeContextIndex(q1: q1, q2: q2, q3: q3) - let k = context.computeGolombParameter(contextIndex: contextIndex) - + // Get context, reusing the quantized gradients the scan loop already + // computed for the run-mode test. Bias C[Q], Golomb k, and the k=0 + // error correction come from a single context-record load. + let (contextIndex, sign) = context.computeContextIndexAndSign(q1: q1, q2: q2, q3: q3) + let (biasC, k, errorCorrection) = context.pixelCodingState(contextIndex: contextIndex) + // Read Golomb-Rice encoded error let mappedError = try readGolombCode(reader: reader, k: k, limit: limit, qbppBits: qbppBits) - - // Compute error correction XOR per ITU-T.87 §A.4.1 - let errorCorrection = context.getErrorCorrection(contextIndex: contextIndex, k: k) - + // Decode pixel using decoder let result = decoder.decodePixel( mappedError: mappedError, - a: a, b: b, c: c, d: d, - context: context, + a: a, b: b, c: c, + contextIndex: contextIndex, + sign: sign, + biasC: biasC, errorCorrection: errorCorrection ) - + // Update context context.updateContext( contextIndex: contextIndex, predictionError: result.error, sign: result.sign ) - + return result.sample } @@ -965,14 +1299,7 @@ public struct JPEGLSDecoder: Sendable { ) throws -> Int { let limitThreshold = limit - qbppBits - 1 // Read unary prefix (count zeros until first '1') - var unaryCount = 0 - while true { - let bit = try reader.readBits(1) - if bit == 1 { - break - } - unaryCount += 1 - } + let unaryCount = try reader.readUnaryCount() // Per ITU-T.87 §6.1.2: when unaryCount >= limitThreshold the encoder used // the limited binary code — read qbppBits bits for MErrval − 1. if unaryCount >= limitThreshold { diff --git a/Sources/JPEGLS/JPEGLSEncoder.swift b/Sources/JPEGLS/JPEGLSEncoder.swift index 3795ef0..91db5f3 100644 --- a/Sources/JPEGLS/JPEGLSEncoder.swift +++ b/Sources/JPEGLS/JPEGLSEncoder.swift @@ -29,6 +29,38 @@ import Foundation /// interleaveMode: .none /// ) /// ``` +/// Lock-protected accumulator for parallel restart-interval encoding: +/// each worker stores its interval's bytes at its own index, and the first +/// error (if any) wins. +private final class IntervalEncodeResults: @unchecked Sendable { + private let lock = NSLock() + private var data: [Data?] + private var firstError: Error? + + init(count: Int) { + self.data = Array(repeating: nil, count: count) + } + + func set(_ chunk: Data, at index: Int) { + lock.lock() + data[index] = chunk + lock.unlock() + } + + func fail(_ error: Error) { + lock.lock() + if firstError == nil { firstError = error } + lock.unlock() + } + + func finish() throws -> [Data?] { + lock.lock() + defer { lock.unlock() } + if let firstError { throw firstError } + return data + } +} + public struct JPEGLSEncoder: Sendable { /// Configuration for encoding public struct Configuration: Sendable { @@ -60,6 +92,19 @@ public struct JPEGLSEncoder: Sendable { /// will reference the same mapping table. public let mappingTable: JPEGLSMappingTable? + /// Restart interval in sample lines (0 = no restart markers, the default). + /// + /// When > 0 the encoder writes a DRI marker segment and emits an RSTm + /// marker (cycling FFD0–FFD7) after every `restartInterval` lines of + /// each scan. Per ITU-T.87, the coding state — contexts, run state, + /// bit alignment, and the previous-line prediction — resets at every + /// interval boundary, which makes intervals independently decodable + /// (and lets the encoder process them in parallel) at a small + /// compression-ratio cost. + /// + /// Currently supported for lossless (NEAR = 0), non-interleaved scans. + public let restartInterval: Int + /// Initialize encoding configuration /// /// ```swift @@ -89,23 +134,45 @@ public struct JPEGLSEncoder: Sendable { /// - presetParameters: Optional custom preset parameters (uses defaults if nil) /// - colorTransformation: Colour transform to apply before encoding (default: .none) /// - mappingTable: Optional mapping table for palettised encoding (default: nil) - /// - Throws: `JPEGLSError.invalidNearParameter` if NEAR is out of range + /// - restartInterval: Restart interval in lines (0 = off; lossless non-interleaved only) + /// - Throws: `JPEGLSError.invalidNearParameter` if NEAR is out of range, + /// `JPEGLSError.encodingFailed` if the restart interval is invalid or + /// combined with an unsupported mode public init( near: Int = 0, interleaveMode: JPEGLSInterleaveMode = .none, presetParameters: JPEGLSPresetParameters? = nil, colorTransformation: JPEGLSColorTransformation = .none, - mappingTable: JPEGLSMappingTable? = nil + mappingTable: JPEGLSMappingTable? = nil, + restartInterval: Int = 0 ) throws { guard near >= 0 && near <= 255 else { throw JPEGLSError.invalidNearParameter(near: near) } + guard (0...65535).contains(restartInterval) else { + throw JPEGLSError.encodingFailed( + reason: "Restart interval must be in 0...65535 lines, got \(restartInterval)" + ) + } + if restartInterval > 0 { + guard near == 0 else { + throw JPEGLSError.encodingFailed( + reason: "Restart intervals are currently supported for lossless (NEAR = 0) encoding only" + ) + } + guard interleaveMode == .none else { + throw JPEGLSError.encodingFailed( + reason: "Restart intervals are currently supported for non-interleaved scans only" + ) + } + } self.near = near self.interleaveMode = interleaveMode self.presetParameters = presetParameters self.colorTransformation = colorTransformation self.mappingTable = mappingTable + self.restartInterval = restartInterval } } @@ -168,22 +235,45 @@ public struct JPEGLSEncoder: Sendable { ? try applyForwardColorTransform(imageData, transformation: colorTransformation, maxValue: maxValue) : imageData + // The scan encoders iterate every component plane at the full frame + // dimensions through unsafe buffers; sub-sampled (narrower/shorter) + // planes would read out of bounds, so reject them up front. + let frame = encodingData.frameHeader + for component in encodingData.components { + guard component.pixels.count == frame.height, + component.pixels.allSatisfy({ $0.count == frame.width }) else { + throw JPEGLSError.encodingFailed( + reason: "Sub-sampled component planes are not supported by the encoder (component \(component.id) is not \(frame.width)×\(frame.height))" + ) + } + } + + // Resolve preset parameters (custom or default) and validate + // MAXVAL ≤ 2^P − 1 (ITU-T.87 C.2.4.1.1): LIMIT is derived from the + // frame's bits-per-sample while qbpp comes from MAXVAL, and a larger + // MAXVAL makes the limited-code threshold negative (trapping, or + // emitting a stream no decoder can parse). + let parameters = try configuration.presetParameters ?? JPEGLSPresetParameters.defaultParameters( + bitsPerSample: encodingData.frameHeader.bitsPerSample, + near: configuration.near + ) + let frameSampleCap = (1 << frame.bitsPerSample) - 1 + guard parameters.maxValue <= frameSampleCap else { + throw JPEGLSError.invalidPresetParameters( + reason: "MAXVAL \(parameters.maxValue) exceeds 2^P−1 = \(frameSampleCap) for \(frame.bitsPerSample)-bit samples" + ) + } + // Write LSE type 4 (extended dimensions) before SOF when either dimension > 65535 // per ITU-T.87 §5.1.1.4. - let frame = encodingData.frameHeader if frame.width > 65535 || frame.height > 65535 { writeExtendedDimensions(frame, to: writer) } - + // Write frame header (SOF55) try writeFrameHeader(encodingData.frameHeader, to: writer) - + // Write preset parameters if custom or near-lossless - let parameters = try configuration.presetParameters ?? JPEGLSPresetParameters.defaultParameters( - bitsPerSample: encodingData.frameHeader.bitsPerSample, - near: configuration.near - ) - if configuration.presetParameters != nil || configuration.near > 0 { try writePresetParameters(parameters, to: writer) } @@ -194,6 +284,13 @@ public struct JPEGLSEncoder: Sendable { writeMappingTable(table, to: writer) } + // Write DRI (define restart interval) when restart markers are enabled. + if configuration.restartInterval > 0 { + writer.writeMarker(.defineRestartInterval) + writer.writeUInt16(4) // segment length including the length field + writer.writeUInt16(UInt16(configuration.restartInterval)) + } + // Encode scan(s) based on interleave mode switch configuration.interleaveMode { case .none: @@ -506,13 +603,14 @@ public struct JPEGLSEncoder: Sendable { // Write scan header (SOS) try writeScanHeader(scanHeader, to: writer) - + // Encode scan data try encodeScanData( imageData: imageData, scanHeader: scanHeader, parameters: parameters, - writer: writer + writer: writer, + restartInterval: configuration.restartInterval ) } @@ -557,7 +655,8 @@ public struct JPEGLSEncoder: Sendable { imageData: MultiComponentImageData, scanHeader: JPEGLSScanHeader, parameters: JPEGLSPresetParameters, - writer: JPEGLSBitstreamWriter + writer: JPEGLSBitstreamWriter, + restartInterval: Int = 0 ) throws { // Create pixel buffer let buffer = JPEGLSPixelBuffer(imageData: imageData) @@ -591,7 +690,8 @@ public struct JPEGLSEncoder: Sendable { context: &context, writer: writer, limit: limit, - qbppBits: qbppBits + qbppBits: qbppBits, + restartInterval: restartInterval ) case .line: @@ -655,6 +755,36 @@ public struct JPEGLSEncoder: Sendable { return (limit, qbppBits) } + /// Compute causal neighbours (Ra, Rb, Rc, Rd) for a pixel directly from + /// hoisted row arrays, replicating `JPEGLSPixelBuffer.getNeighbors` + /// boundary semantics (ITU-T.87 §3.2 / CharLS edge handling) without the + /// per-pixel Dictionary lookup that method performs. + /// + /// - `previousRow == nil` means row 0: top/topLeft/topRight are 0. + /// - At column 0: Ra = Rb = top, Rc = prevRowEdge, Rd = top-right (or top + /// when width == 1). + @inline(__always) + private func neighbors( + currentRow: [Int], + previousRow: [Int]?, + col: Int, + width: Int, + prevRowEdge: Int + ) -> (actual: Int, a: Int, b: Int, c: Int, d: Int) { + let actual = currentRow[col] + guard let prev = previousRow else { + return (actual, col == 0 ? 0 : currentRow[col - 1], 0, 0, 0) + } + if col == 0 { + let top = prev[0] + let d = width > 1 ? prev[1] : top + return (actual, top, top, prevRowEdge, d) + } + let b = prev[col] + let d = col + 1 < width ? prev[col + 1] : b + return (actual, currentRow[col - 1], b, prev[col - 1], d) + } + /// Encode non-interleaved scan (component by component) private func encodeNoneInterleaved( buffer: JPEGLSPixelBuffer, @@ -664,7 +794,8 @@ public struct JPEGLSEncoder: Sendable { context: inout JPEGLSContextModel, writer: JPEGLSBitstreamWriter, limit: Int, - qbppBits: Int + qbppBits: Int, + restartInterval: Int = 0 ) throws { guard scanHeader.componentCount == 1 else { throw JPEGLSError.encodingFailed( @@ -674,15 +805,40 @@ public struct JPEGLSEncoder: Sendable { let componentId = scanHeader.components[0].id let near = scanHeader.near - + + // Resolve the component's pixel array once per scan: the component is + // fixed for the whole scan, so the Dictionary lookup must not sit on + // the per-pixel path. + guard let componentPixels = buffer.getComponentPixels(componentId: componentId) else { + throw JPEGLSError.encodingFailed(reason: "Failed to get component pixels") + } + + // Lossless scans take the flat fast path: no reconstructed-value + // tracking is needed, so the whole scan can run over a contiguous + // UInt16 plane. + if near == 0 { + try encodeNoneInterleavedLossless( + componentPixels: componentPixels, + width: buffer.width, + height: buffer.height, + regularMode: regularMode, + runMode: runMode, + context: &context, + writer: writer, + limit: limit, + qbppBits: qbppBits, + restartInterval: restartInterval + ) + return + } + // Track reconstructed values for near-lossless neighbour computation. - // For lossless (NEAR = 0) this array is never read; for near-lossless it - // stores what the decoder will reconstruct so that subsequent pixels use - // the same context as the decoder. - var reconstructed = Array( - repeating: Array(repeating: 0, count: buffer.width), - count: buffer.height - ) + // For lossless (NEAR = 0) this array is never read — every access below + // is guarded by `near > 0` — so skip the full-frame allocation entirely + // (a 2048^2 scan would otherwise allocate and zero 32 MB for nothing). + var reconstructed: [[Int]] = near > 0 + ? Array(repeating: Array(repeating: 0, count: buffer.width), count: buffer.height) + : [] // Encode pixels in raster order with run mode support var prevRowEdge = 0 @@ -690,22 +846,18 @@ public struct JPEGLSEncoder: Sendable { // Note: RUNindex is NOT reset per line. Per ITU-T.87 §A.7.1 and CharLS, // RUNindex persists across scan lines; it is only initialised to 0 at scan start. let edgeForThisRow = prevRowEdge - if row > 0 { - prevRowEdge = buffer.getPixel(componentId: componentId, row: row - 1, column: 0) ?? 0 + let currentRow = componentPixels[row] + let previousRow: [Int]? = row > 0 ? componentPixels[row - 1] : nil + if let previousRow { + prevRowEdge = previousRow[0] } var col = 0 while col < buffer.width { - guard let neighbors = buffer.getNeighbors( - componentId: componentId, - row: row, - column: col, - prevRowEdge: edgeForThisRow - ) else { - throw JPEGLSError.encodingFailed( - reason: "Failed to get neighbors for pixel at (\(row), \(col))" - ) - } - + let neighbors = self.neighbors( + currentRow: currentRow, previousRow: previousRow, + col: col, width: buffer.width, prevRowEdge: edgeForThisRow + ) + // Use reconstructed neighbours for near-lossless; originals for lossless. let (a, b, c, d): (Int, Int, Int, Int) if near > 0 { @@ -714,7 +866,7 @@ public struct JPEGLSEncoder: Sendable { width: buffer.width, height: buffer.height ) } else { - (a, b, c, d) = (neighbors.left, neighbors.top, neighbors.topLeft, neighbors.topRight) + (a, b, c, d) = (neighbors.a, neighbors.b, neighbors.c, neighbors.d) } // Check for run mode: all quantized gradients are zero @@ -727,12 +879,8 @@ public struct JPEGLSEncoder: Sendable { // Run mode: scan ahead for matching pixels. // The run value is the reconstructed left neighbour (a). let runValue = a - guard let componentPixels = buffer.getComponentPixels(componentId: componentId) else { - throw JPEGLSError.encodingFailed(reason: "Failed to get component pixels") - } - let linePixels = componentPixels[row] let runLength = runMode.detectRunLength( - pixels: linePixels, + pixels: currentRow, startIndex: col, runValue: runValue ) @@ -768,25 +916,12 @@ public struct JPEGLSEncoder: Sendable { // Encode the interruption pixel let interruptionCol = col + actualRunLength if interruptionCol < buffer.width { - guard let interruptionNeighbors = buffer.getNeighbors( - componentId: componentId, - row: row, - column: interruptionCol, - prevRowEdge: edgeForThisRow - ) else { - throw JPEGLSError.encodingFailed( - reason: "Failed to get neighbors for interruption pixel at (\(row), \(interruptionCol))" - ) - } - + let interruptionActual = currentRow[interruptionCol] + // Compute Rb at the interruption position let encRb: Int - if row > 0 { - if let compPixels = buffer.getComponentPixels(componentId: componentId) { - encRb = near > 0 ? reconstructed[row - 1][interruptionCol] : compPixels[row - 1][interruptionCol] - } else { - encRb = 0 - } + if let previousRow { + encRb = near > 0 ? reconstructed[row - 1][interruptionCol] : previousRow[interruptionCol] } else { encRb = 0 } @@ -796,7 +931,7 @@ public struct JPEGLSEncoder: Sendable { // The decoder also uses finalRunIndex at this point. context.setRunIndex(finalRunIndex) let rv = writeRunInterruptionBits( - interruptionValue: interruptionNeighbors.actual, + interruptionValue: interruptionActual, runValue: runValue, rb: encRb, near: near, @@ -838,7 +973,8 @@ public struct JPEGLSEncoder: Sendable { // Regular mode let rv = encodePixel( actual: neighbors.actual, - a: a, b: b, c: c, d: d, + a: a, b: b, c: c, + q1: q1, q2: q2, q3: q3, regularMode: regularMode, context: &context, writer: writer, @@ -854,6 +990,242 @@ public struct JPEGLSEncoder: Sendable { } } + /// Lossless (NEAR = 0) non-interleaved scan over a flat UInt16 plane. + /// + /// Identical coding decisions to the general path — same neighbours, + /// gradients, run detection, and bit output — but the pixels live in one + /// contiguous buffer accessed through an unsafe pointer scoped over the + /// whole scan: no nested-array indirection, no per-access bounds checks, + /// and the run scan compares against the row directly. Input samples are + /// validated to [0, MAXVAL ≤ 2^16 − 1] by MultiComponentImageData. + private func encodeNoneInterleavedLossless( + componentPixels: [[Int]], + width: Int, + height: Int, + regularMode: JPEGLSRegularMode, + runMode: JPEGLSRunMode, + context: inout JPEGLSContextModel, + writer: JPEGLSBitstreamWriter, + limit: Int, + qbppBits: Int, + restartInterval: Int = 0 + ) throws { + // Flatten once per scan. + var flat = [UInt16](repeating: 0, count: width * height) + flat.withUnsafeMutableBufferPointer { out in + for row in 0.. 0 && restartInterval < height { + // Restart intervals: every interval restarts coding exactly as at + // scan start (fresh contexts, run state, bit alignment, zero + // previous line), so the intervals are independent and can encode + // in parallel into per-interval buffers concatenated with RSTm + // markers (cycling FFD0–FFD7) between them. + let chunkCount = (height + restartInterval - 1) / restartInterval + let presetParameters = regularMode.presetParameters + let plane = flat + let results = IntervalEncodeResults(count: chunkCount) + DispatchQueue.concurrentPerform(iterations: chunkCount) { idx in + do { + let lo = idx * restartInterval + let hi = min(lo + restartInterval, height) + var chunkContext = try JPEGLSContextModel( + parameters: presetParameters, near: 0 + ) + let chunkWriter = JPEGLSBitstreamWriter( + capacity: (hi - lo) * width * 2 + 64 + ) + encodeFlatRowsLossless( + flat: plane, rowRange: lo.., + width: Int, + regularMode: JPEGLSRegularMode, + runMode: JPEGLSRunMode, + context: inout JPEGLSContextModel, + writer: JPEGLSBitstreamWriter, + limit: Int, + qbppBits: Int + ) { + flat.withUnsafeBufferPointer { buf in + var prevRowEdge = 0 + let firstRow = rowRange.lowerBound + for row in rowRange { + // Note: RUNindex is NOT reset per line. Per ITU-T.87 §A.7.1 and CharLS, + // RUNindex persists across scan lines; it is only initialised to 0 at scan start. + let rowBase = row * width + let prevBase = rowBase - width + let edgeForThisRow = prevRowEdge + if row > firstRow { + prevRowEdge = Int(buf[prevBase]) + } + var col = 0 + while col < width { + // Causal neighbours per ITU-T.87 §3.2 (same boundary + // semantics as the general path). The first row of the + // range uses row-0 semantics (zero previous line). + let actual = Int(buf[rowBase + col]) + let a: Int, b: Int, c: Int, d: Int + if row == firstRow { + a = col == 0 ? 0 : Int(buf[rowBase + col - 1]) + b = 0; c = 0; d = 0 + } else if col == 0 { + let top = Int(buf[prevBase]) + a = top + b = top + c = edgeForThisRow + d = width > 1 ? Int(buf[prevBase + 1]) : top + } else { + a = Int(buf[rowBase + col - 1]) + b = Int(buf[prevBase + col]) + c = Int(buf[prevBase + col - 1]) + d = col + 1 < width ? Int(buf[prevBase + col + 1]) : b + } + + // Check for run mode: all quantized gradients are zero + let (d1, d2, d3) = regularMode.computeGradients(a: a, b: b, c: c, d: d) + let q1 = regularMode.quantizeGradient(d1) + let q2 = regularMode.quantizeGradient(d2) + let q3 = regularMode.quantizeGradient(d3) + + if q1 == 0 && q2 == 0 && q3 == 0 { + // Run mode: scan the rest of the row for the run value + // (exact equality — lossless) with a 4-way unrolled test. + let runValue = a + let rv16 = UInt16(truncatingIfNeeded: runValue) + let rowEnd = rowBase + width + var i = rowBase + col + while i + 4 <= rowEnd { + if buf[i] != rv16 || buf[i + 1] != rv16 + || buf[i + 2] != rv16 || buf[i + 3] != rv16 { + break + } + i += 4 + } + while i < rowEnd && buf[i] == rv16 { + i += 1 + } + let actualRunLength = i - (rowBase + col) + let remainingInLine = width - col + + // Encode run length + let encoded = runMode.encodeRunLength( + runLength: actualRunLength, + runIndex: context.currentRunIndex + ) + + // Write continuation bits (1s) + writer.writeOnes(encoded.continuationBits) + + // Compute finalRunIndex now so it can be used for the interruption + // pixel's adjustedLimit (matching the decoder, which uses the + // post-continuation run index when computing J for the limit). + let finalRunIndex = min(encoded.runIndex + encoded.continuationBits, 31) + + if actualRunLength < remainingInLine { + // Run was interrupted — write termination and remainder. + writeRunTermination(encoded: encoded, writer: writer) + + let interruptionCol = col + actualRunLength + let interruptionActual = Int(buf[rowBase + interruptionCol]) + let encRb = row > firstRow ? Int(buf[prevBase + interruptionCol]) : 0 + + // Per ITU-T.87 / CharLS: use finalRunIndex (post-continuation) + // for J when computing adjustedLimit in the interruption pixel. + context.setRunIndex(finalRunIndex) + _ = writeRunInterruptionBits( + interruptionValue: interruptionActual, + runValue: runValue, + rb: encRb, + near: 0, + context: &context, + regularMode: regularMode, + runMode: runMode, + writer: writer, + limit: limit, + qbppBits: qbppBits + ) + // Decrement RUNindex after the interruption pixel, matching + // the decoder which calls decrementRunIndex() at this point. + context.setRunIndex(max(finalRunIndex - 1, 0)) + col = interruptionCol + 1 + } else { + // Run reaches end of line: write one '1' bit for a + // partial last block; nothing for an exact fill + // (per ITU-T.87 §A.7.1). + if encoded.remainder > 0 { + writer.writeBits(1, count: 1) + } + col += actualRunLength + context.setRunIndex(finalRunIndex) + } + } else { + // Regular mode + _ = encodePixel( + actual: actual, + a: a, b: b, c: c, + q1: q1, q2: q2, q3: q3, + regularMode: regularMode, + context: &context, + writer: writer, + limit: limit, + qbppBits: qbppBits + ) + col += 1 + } + } + } + } + } + /// Encode line-interleaved scan private func encodeLineInterleaved( buffer: JPEGLSPixelBuffer, @@ -872,6 +1244,15 @@ public struct JPEGLSEncoder: Sendable { // Per-component RUNindex per CharLS: each component line preserves its own run index. var componentRunIndex: [UInt8: Int] = [:] for component in scanHeader.components { componentRunIndex[component.id] = 0 } + // Resolve each component's pixel array once per scan so the Dictionary + // lookup never sits on the per-pixel path. + var componentPixelsById: [UInt8: [[Int]]] = [:] + for component in scanHeader.components { + guard let pixels = buffer.getComponentPixels(componentId: component.id) else { + throw JPEGLSError.encodingFailed(reason: "Failed to get component pixels") + } + componentPixelsById[component.id] = pixels + } // Track reconstructed values per component for near-lossless neighbour computation. var reconstructedPerComponent: [UInt8: [[Int]]] = [:] if near > 0 { @@ -888,23 +1269,20 @@ public struct JPEGLSEncoder: Sendable { for component in scanHeader.components { // Restore this component's run index context.setRunIndex(componentRunIndex[component.id] ?? 0) + let componentPixels = componentPixelsById[component.id]! + let currentRow = componentPixels[row] + let previousRow: [Int]? = row > 0 ? componentPixels[row - 1] : nil let edgeForThisRow = prevRowEdges[component.id] ?? 0 - if row > 0 { - prevRowEdges[component.id] = buffer.getPixel(componentId: component.id, row: row - 1, column: 0) ?? 0 + if let previousRow { + prevRowEdges[component.id] = previousRow[0] } var col = 0 while col < buffer.width { - guard let neighbors = buffer.getNeighbors( - componentId: component.id, - row: row, - column: col, - prevRowEdge: edgeForThisRow - ) else { - throw JPEGLSError.encodingFailed( - reason: "Failed to get neighbors for pixel at (\(row), \(col))" - ) - } - + let neighbors = self.neighbors( + currentRow: currentRow, previousRow: previousRow, + col: col, width: buffer.width, prevRowEdge: edgeForThisRow + ) + // Use reconstructed neighbours for near-lossless; originals for lossless. let (a, b, c, d): (Int, Int, Int, Int) if near > 0, let recArray = reconstructedPerComponent[component.id] { @@ -913,7 +1291,7 @@ public struct JPEGLSEncoder: Sendable { width: buffer.width, height: buffer.height ) } else { - (a, b, c, d) = (neighbors.left, neighbors.top, neighbors.topLeft, neighbors.topRight) + (a, b, c, d) = (neighbors.a, neighbors.b, neighbors.c, neighbors.d) } // Check for run mode @@ -925,12 +1303,8 @@ public struct JPEGLSEncoder: Sendable { if q1 == 0 && q2 == 0 && q3 == 0 { // Run mode: the run value is the reconstructed left neighbour. let runValue = a - guard let componentPixels = buffer.getComponentPixels(componentId: component.id) else { - throw JPEGLSError.encodingFailed(reason: "Failed to get component pixels") - } - let linePixels = componentPixels[row] let runLength = runMode.detectRunLength( - pixels: linePixels, + pixels: currentRow, startIndex: col, runValue: runValue ) @@ -959,34 +1333,23 @@ public struct JPEGLSEncoder: Sendable { let interruptionCol = col + actualRunLength if interruptionCol < buffer.width { - guard let interruptionNeighbors = buffer.getNeighbors( - componentId: component.id, - row: row, - column: interruptionCol, - prevRowEdge: edgeForThisRow - ) else { - throw JPEGLSError.encodingFailed( - reason: "Failed to get neighbors for interruption pixel" - ) - } - + let interruptionActual = currentRow[interruptionCol] + // Compute Rb at the interruption position (use reconstructed for near-lossless) let encRb2: Int - if row > 0 { + if let previousRow { if near > 0, let recArray = reconstructedPerComponent[component.id] { encRb2 = recArray[row - 1][interruptionCol] - } else if let compPixels = buffer.getComponentPixels(componentId: component.id) { - encRb2 = compPixels[row - 1][interruptionCol] } else { - encRb2 = 0 + encRb2 = previousRow[interruptionCol] } } else { encRb2 = 0 } - + context.setRunIndex(finalRunIndex) let rv = writeRunInterruptionBits( - interruptionValue: interruptionNeighbors.actual, + interruptionValue: interruptionActual, runValue: runValue, rb: encRb2, near: near, @@ -1022,7 +1385,7 @@ public struct JPEGLSEncoder: Sendable { a: a, b: b, c: c, - d: d, + q1: q1, q2: q2, q3: q3, regularMode: regularMode, context: &context, writer: writer, @@ -1060,6 +1423,18 @@ public struct JPEGLSEncoder: Sendable { let near = scanHeader.near let components = scanHeader.components + // Resolve every component's pixel array once per scan, aligned with + // the `components` ordering, so the Dictionary lookup never sits on + // the per-pixel path. + var componentPixelArrays: [[[Int]]] = [] + componentPixelArrays.reserveCapacity(components.count) + for component in components { + guard let pixels = buffer.getComponentPixels(componentId: component.id) else { + throw JPEGLSError.encodingFailed(reason: "Failed to get component pixels") + } + componentPixelArrays.append(pixels) + } + // Track left-edge values per component for boundary Rc at col=0. var prevRowEdges: [UInt8: Int] = [:] for component in components { prevRowEdges[component.id] = 0 } @@ -1079,11 +1454,13 @@ public struct JPEGLSEncoder: Sendable { for row in 0.. 0 { - prevRowEdges[component.id] = buffer.getPixel(componentId: component.id, row: row - 1, column: 0) ?? 0 + let currentRows = componentPixelArrays.map { $0[row] } + let previousRows: [[Int]]? = row > 0 ? componentPixelArrays.map { $0[row - 1] } : nil + var edgesForThisRow = [Int](repeating: 0, count: components.count) + for (cIdx, component) in components.enumerated() { + edgesForThisRow[cIdx] = prevRowEdges[component.id] ?? 0 + if let previousRows { + prevRowEdges[component.id] = previousRows[cIdx][0] } } var col = 0 @@ -1091,7 +1468,7 @@ public struct JPEGLSEncoder: Sendable { // Check if ALL components have zero quantised gradients at (row, col) // using reconstructed neighbours for near-lossless. var allGradientsZero = true - for component in components { + for (cIdx, component) in components.enumerated() { let compA: Int let compB: Int let compC: Int @@ -1102,15 +1479,11 @@ public struct JPEGLSEncoder: Sendable { width: buffer.width, height: buffer.height ) } else { - guard let neighbors = buffer.getNeighbors( - componentId: component.id, row: row, column: col, - prevRowEdge: edgesForThisRow[component.id] ?? 0 - ) else { - throw JPEGLSError.encodingFailed( - reason: "Failed to get neighbors for pixel at (\(row), \(col))" - ) - } - (compA, compB, compC, compD) = (neighbors.left, neighbors.top, neighbors.topLeft, neighbors.topRight) + let n = self.neighbors( + currentRow: currentRows[cIdx], previousRow: previousRows?[cIdx], + col: col, width: buffer.width, prevRowEdge: edgesForThisRow[cIdx] + ) + (compA, compB, compC, compD) = (n.a, n.b, n.c, n.d) } let (d1, d2, d3) = regularMode.computeGradients(a: compA, b: compB, c: compC, d: compD) if regularMode.quantizeGradient(d1) != 0 || @@ -1126,8 +1499,7 @@ public struct JPEGLSEncoder: Sendable { // The run continues while every component's pixel equals its run value. // The run value is the reconstructed left neighbour of each component. var runValue: [Int] = [] - var componentLinePixels: [[Int]] = [] - for component in components { + for (cIdx, component) in components.enumerated() { let rv: Int if near > 0, let recArray = reconstructedPerComponent[component.id] { let (a, _, _, _) = computeReconstructedNeighbors( @@ -1136,22 +1508,15 @@ public struct JPEGLSEncoder: Sendable { ) rv = a } else { - guard let neighbors = buffer.getNeighbors( - componentId: component.id, row: row, column: col, - prevRowEdge: edgesForThisRow[component.id] ?? 0 - ) else { - throw JPEGLSError.encodingFailed( - reason: "Failed to get neighbors for run at (\(row), \(col))" - ) - } - rv = neighbors.left + let n = self.neighbors( + currentRow: currentRows[cIdx], previousRow: previousRows?[cIdx], + col: col, width: buffer.width, prevRowEdge: edgesForThisRow[cIdx] + ) + rv = n.a } runValue.append(rv) - guard let allPixels = buffer.getComponentPixels(componentId: component.id) else { - throw JPEGLSError.encodingFailed(reason: "Failed to get component pixels") - } - componentLinePixels.append(allPixels[row]) } + let componentLinePixels = currentRows // Detect run: the minimum run length across all components let remainingInLine = buffer.width - col @@ -1193,30 +1558,20 @@ public struct JPEGLSEncoder: Sendable { if interruptionCol < buffer.width { context.setRunIndex(finalRunIndex) for (cIdx, component) in components.enumerated() { - guard let intNeighbors = buffer.getNeighbors( - componentId: component.id, - row: row, column: interruptionCol, - prevRowEdge: edgesForThisRow[component.id] ?? 0 - ) else { - throw JPEGLSError.encodingFailed( - reason: "Failed to get interruption neighbors" - ) - } + let interruptionActual = currentRows[cIdx][interruptionCol] // Compute Rb at the interruption position (use reconstructed for near-lossless) let encRb3: Int - if row > 0 { + if let previousRows { if near > 0, let recArray = reconstructedPerComponent[component.id] { encRb3 = recArray[row - 1][interruptionCol] - } else if let compPixels = buffer.getComponentPixels(componentId: component.id) { - encRb3 = compPixels[row - 1][interruptionCol] } else { - encRb3 = 0 + encRb3 = previousRows[cIdx][interruptionCol] } } else { encRb3 = 0 } let rv3 = writeRunInterruptionBits( - interruptionValue: intNeighbors.actual, + interruptionValue: interruptionActual, runValue: runValue[cIdx], rb: encRb3, near: near, @@ -1249,7 +1604,7 @@ public struct JPEGLSEncoder: Sendable { } } else { // Regular mode: encode each component at (row, col) - for component in components { + for (cIdx, component) in components.enumerated() { let compA: Int let compB: Int let compC: Int @@ -1260,25 +1615,24 @@ public struct JPEGLSEncoder: Sendable { from: recArray, row: row, col: col, width: buffer.width, height: buffer.height ) - actual = buffer.getPixel(componentId: component.id, row: row, column: col) ?? 0 + actual = currentRows[cIdx][col] } else { - guard let neighbors = buffer.getNeighbors( - componentId: component.id, row: row, column: col, - prevRowEdge: edgesForThisRow[component.id] ?? 0 - ) else { - throw JPEGLSError.encodingFailed( - reason: "Failed to get neighbors for pixel at (\(row), \(col))" - ) - } - (compA, compB, compC, compD) = (neighbors.left, neighbors.top, neighbors.topLeft, neighbors.topRight) - actual = neighbors.actual + let n = self.neighbors( + currentRow: currentRows[cIdx], previousRow: previousRows?[cIdx], + col: col, width: buffer.width, prevRowEdge: edgesForThisRow[cIdx] + ) + (compA, compB, compC, compD) = (n.a, n.b, n.c, n.d) + actual = n.actual } + let (sd1, sd2, sd3) = regularMode.computeGradients(a: compA, b: compB, c: compC, d: compD) let rv = encodePixel( actual: actual, a: compA, b: compB, c: compC, - d: compD, + q1: regularMode.quantizeGradient(sd1), + q2: regularMode.quantizeGradient(sd2), + q3: regularMode.quantizeGradient(sd3), regularMode: regularMode, context: &context, writer: writer, @@ -1349,20 +1703,25 @@ public struct JPEGLSEncoder: Sendable { a: Int, b: Int, c: Int, - d: Int, + q1: Int, + q2: Int, + q3: Int, regularMode: JPEGLSRegularMode, context: inout JPEGLSContextModel, writer: JPEGLSBitstreamWriter, limit: Int, qbppBits: Int ) -> Int { - // Regular mode encoding + // Regular mode encoding, reusing the quantized gradients the scan + // loop already computed for the run-mode test. + let (contextIndex, sign) = context.computeContextIndexAndSign(q1: q1, q2: q2, q3: q3) let encodedPixel = regularMode.encodePixel( actual: actual, a: a, b: b, c: c, - d: d, + contextIndex: contextIndex, + sign: sign, context: context ) diff --git a/Sources/JPEGLS/Platform/ARM64/ARM64Accelerator.swift b/Sources/JPEGLS/Platform/ARM64/ARM64Accelerator.swift deleted file mode 100644 index 4196464..0000000 --- a/Sources/JPEGLS/Platform/ARM64/ARM64Accelerator.swift +++ /dev/null @@ -1,326 +0,0 @@ -/// ARM64-specific acceleration using NEON SIMD instructions. -/// -/// This implementation leverages ARM NEON intrinsics for vectorized operations -/// on Apple Silicon and other ARM64 processors. Swift's SIMD types compile to -/// efficient NEON instructions on ARM64 hardware. -/// -/// **Note**: This file is conditionally compiled only on ARM64 architectures. - -#if arch(arm64) - -import Foundation - -/// ARM64 NEON-accelerated implementation of platform acceleration. -/// -/// Provides hardware-accelerated gradient computation, prediction, and -/// quantization optimized for Apple Silicon (M1, M2, M3) and ARM64 processors. -/// -/// The implementation uses Swift's SIMD types which compile to native NEON -/// instructions for maximum performance on ARM64 hardware. -public struct ARM64Accelerator: PlatformAccelerator { - public static let platformName = "ARM64" - - /// Always returns true on ARM64 architectures. - public static var isSupported: Bool { - return true - } - - /// Initialize an ARM64 NEON accelerator - public init() {} - - // MARK: - NEON-Optimized Gradient Computation - - /// Compute local gradients using NEON SIMD operations. - /// - /// This implementation uses vectorized subtraction to compute all three - /// gradients in parallel using NEON instructions: - /// - D1 = b - c (horizontal gradient) - /// - D2 = a - c (vertical gradient) - /// - D3 = c - a (diagonal gradient) - /// - /// - Parameters: - /// - a: North pixel value - /// - b: West pixel value - /// - c: Northwest pixel value - /// - Returns: A tuple of three gradients (d1, d2, d3) - public func computeGradients(a: Int, b: Int, c: Int) -> (d1: Int, d2: Int, d3: Int) { - // Pack values into SIMD vector for parallel computation - // Vector layout: [a, b, c, 0] - let values = SIMD4(Int32(a), Int32(b), Int32(c), 0) - - // Create subtraction operands using SIMD shuffles - // For D1 = b - c: operand1 = [b, x, x, x], operand2 = [c, x, x, x] - // For D2 = a - c: operand1 = [x, a, x, x], operand2 = [x, c, x, x] - // For D3 = c - a: operand1 = [x, x, c, x], operand2 = [x, x, a, x] - let operand1 = SIMD4(values[1], values[0], values[2], 0) // [b, a, c, 0] - let operand2 = SIMD4(values[2], values[2], values[0], 0) // [c, c, a, 0] - - // Vectorized subtraction using NEON - let gradients = operand1 &- operand2 // [b-c, a-c, c-a, 0] - - return (Int(gradients[0]), Int(gradients[1]), Int(gradients[2])) - } - - // MARK: - NEON-Optimized Run-Length Detection - - /// Detect run length using SIMD8 comparisons on ARM64. - /// - /// Scans ahead from `startIndex` in the `pixels` array, counting - /// consecutive elements equal to `runValue`. Uses SIMD8 vectorised - /// comparison to process 8 pixels per iteration, leveraging NEON - /// comparison instructions for maximum throughput. - /// - /// - Parameters: - /// - pixels: Array of pixel values to scan - /// - startIndex: Starting index for the scan - /// - runValue: The pixel value that constitutes a run - /// - maxLength: Maximum run length to detect - /// - Returns: Length of the run starting at `startIndex` - public func detectRunLength( - in pixels: [Int32], - startIndex: Int, - runValue: Int32, - maxLength: Int - ) -> Int { - let limit = min(pixels.count - startIndex, maxLength) - guard limit > 0 else { return 0 } - - var runLength = 0 - let vectorSize = 8 - let runVec = SIMD8(repeating: runValue) - - // Process 8 pixels at a time using NEON comparisons - while runLength + vectorSize <= limit { - let idx = startIndex + runLength - let chunk = SIMD8( - pixels[idx], pixels[idx + 1], pixels[idx + 2], pixels[idx + 3], - pixels[idx + 4], pixels[idx + 5], pixels[idx + 6], pixels[idx + 7] - ) - let matches = chunk .== runVec - - // Find first mismatch within the vector - for j in 0.. [Int] { - var positions: [Int] = [] - let count = data.count - let vectorSize = 8 - let ffVec = SIMD8(repeating: 0xFF) - - var i = 0 - // Process 8 bytes at a time using NEON - while i + vectorSize <= count { - let chunk = SIMD8( - data[i], data[i + 1], data[i + 2], data[i + 3], - data[i + 4], data[i + 5], data[i + 6], data[i + 7] - ) - let mask = chunk .== ffVec - - if mask != SIMDMask(repeating: false) { - for j in 0.. Int { - guard n > 0 else { return 0 } - guard a > 0 else { return 0 } - - // Iterative Golomb-Rice k calculation: find smallest k where 2^k * n >= a - // Use CLZ to provide an O(1) starting estimate, then walk upward once. - let aN = max(1, a / n) - // floor(log2(aN)) via CLZ: bit_width - 1 - leadingZeroBitCount - let log2Estimate = max(0, (UInt64.bitWidth - 1 - UInt64(aN).leadingZeroBitCount)) - - // Start just below the estimate and advance until the condition is met. - var k = max(0, log2Estimate > 0 ? log2Estimate - 1 : 0) - while k < 31 && (n << k) < a { - k += 1 - } - - // Clamp to the documented [0, 31] range. When the CLZ-based estimate - // already starts above 31 (pathological a/n), the loop above is skipped, - // so the clamp here is what enforces the upper bound (matches X86_64Accelerator). - return min(k, 31) - } - - // MARK: - NEON-Optimized MED Predictor - - /// Compute MED (Median Edge Detector) prediction using NEON operations. - /// - /// The MED predictor uses vectorized min/max operations available in NEON - /// to efficiently compute the prediction value: - /// - If c >= max(a, b): return min(a, b) - /// - If c <= min(a, b): return max(a, b) - /// - Otherwise: return a + b - c - /// - /// - Parameters: - /// - a: North pixel value - /// - b: West pixel value - /// - c: Northwest pixel value - /// - Returns: The predicted pixel value - public func medPredictor(a: Int, b: Int, c: Int) -> Int { - // Use SIMD for parallel min/max operations - let vec = SIMD4(Int32(a), Int32(b), Int32(c), 0) - - // Compute min(a, b) and max(a, b) using NEON min/max instructions - let minAB = min(vec[0], vec[1]) - let maxAB = max(vec[0], vec[1]) - - // MED predictor logic using NEON comparison operations - if vec[2] >= maxAB { - // c >= max(a, b) → return min(a, b) - return Int(minAB) - } else if vec[2] <= minAB { - // c <= min(a, b) → return max(a, b) - return Int(maxAB) - } else { - // Otherwise → return a + b - c - // Use SIMD addition and subtraction - let sum = vec[0] &+ vec[1] // a + b - let result = sum &- vec[2] // (a + b) - c - return Int(result) - } - } - - // MARK: - NEON-Optimized Gradient Quantization - - /// Quantize gradients using NEON SIMD comparison operations. - /// - /// This implementation uses vectorized comparisons to process all three - /// gradients in parallel, leveraging NEON's comparison and select operations - /// for maximum throughput. - /// - /// The quantization maps gradient values to discrete levels [-4, 4] based - /// on threshold parameters (t1, t2, t3) per ITU-T.87 Section 4.3.1. - /// - /// - Parameters: - /// - d1: First gradient - /// - d2: Second gradient - /// - d3: Third gradient - /// - t1: Quantization threshold 1 - /// - t2: Quantization threshold 2 - /// - t3: Quantization threshold 3 - /// - Returns: A tuple of three quantized gradient values (q1, q2, q3) - public func quantizeGradients(d1: Int, d2: Int, d3: Int, t1: Int, t2: Int, t3: Int) -> (q1: Int, q2: Int, q3: Int) { - // Quantize each gradient using ITU-T.87 compliant logic - // While we use SIMD for loading gradients, the actual quantization - // uses scalar operations due to the branching nature of the algorithm - let q1 = quantizeSingleGradient(gradient: d1, t1: t1, t2: t2, t3: t3) - let q2 = quantizeSingleGradient(gradient: d2, t1: t1, t2: t2, t3: t3) - let q3 = quantizeSingleGradient(gradient: d3, t1: t1, t2: t2, t3: t3) - - return (q1, q2, q3) - } - - /// Quantize a single gradient value using NEON-friendly logic. - /// - /// This helper function implements the ITU-T.87 quantization algorithm - /// with comparisons that can be optimized by the compiler to NEON instructions. - /// - /// Quantization per ITU-T.87 Section 4.3.1: - /// - Q = -4 if d <= -t3 - /// - Q = -3 if -t3 < d <= -t2 - /// - Q = -2 if -t2 < d <= -t1 - /// - Q = -1 if -t1 < d < 0 - /// - Q = 0 if d == 0 - /// - Q = 1 if 0 < d < t1 - /// - Q = 2 if t1 <= d < t2 - /// - Q = 3 if t2 <= d < t3 - /// - Q = 4 if t3 <= d - /// - /// - Parameters: - /// - gradient: Raw gradient value - /// - t1: Threshold 1 - /// - t2: Threshold 2 - /// - t3: Threshold 3 - /// - Returns: Quantized gradient value in range [-4, 4] - @inline(__always) - private func quantizeSingleGradient( - gradient: Int, - t1: Int, - t2: Int, - t3: Int - ) -> Int { - // Quantization using signed comparisons per ITU-T.87 - // The compiler will optimise these to NEON comparison instructions - - if gradient <= -t3 { - return -4 - } else if gradient <= -t2 { - return -3 - } else if gradient <= -t1 { - return -2 - } else if gradient < 0 { - return -1 - } else if gradient == 0 { - return 0 - } else if gradient < t1 { - return 1 - } else if gradient < t2 { - return 2 - } else if gradient < t3 { - return 3 - } else { - return 4 - } - } -} - -#endif diff --git a/Sources/JPEGLS/Platform/ARM64/AppleSiliconMemoryOptimizer.swift b/Sources/JPEGLS/Platform/ARM64/AppleSiliconMemoryOptimizer.swift deleted file mode 100644 index 1f2bda3..0000000 --- a/Sources/JPEGLS/Platform/ARM64/AppleSiliconMemoryOptimizer.swift +++ /dev/null @@ -1,279 +0,0 @@ -/// Apple Silicon memory architecture optimisation for JPEG-LS. -/// -/// Provides cache-hierarchy-aware data layouts, unified-memory buffer pooling, -/// memory-mapped I/O helpers, and L1/L2 tile-size tuning tailored for -/// A-series and M-series Apple Silicon processors. -/// -/// **Note**: This file is conditionally compiled only on ARM64 architectures -/// so that the x86-64 code path remains cleanly separable. - -#if arch(arm64) - -import Foundation - -// MARK: - Apple Silicon Cache Parameters - -/// Cache and memory-architecture parameters for Apple Silicon processors. -/// -/// Values are approximate medians for M-series chips; the tile-tuning -/// helpers below use these constants to compute optimal tile sizes. -public enum AppleSiliconCacheParameters { - /// L1 data-cache size per performance core (bytes) — typical M-series value. - public static let l1DataCacheSize: Int = 128 * 1024 // 128 KiB - - /// L2 shared cache size per cluster (bytes) — typical M-series value. - public static let l2CacheSize: Int = 16 * 1024 * 1024 // 16 MiB - - /// L3 / System Level Cache size (bytes) — typical M2 Pro / M3 Pro value. - public static let l3CacheSize: Int = 32 * 1024 * 1024 // 32 MiB (conservative) - - /// CPU cache-line size in bytes (ARM64 standard). - public static let cacheLineSize: Int = 64 - - /// Optimal JPEG-LS context array alignment for cache-line boundaries. - public static let contextArrayAlignment: Int = 64 - - /// Maximum single-strip tile height (rows) recommended for L1 fit with 3-component 8-bit data. - public static let recommendedStripHeight: Int = 16 -} - -// MARK: - Tile Size Tuning - -/// Compute the optimal tile dimensions for JPEG-LS encoding on Apple Silicon. -/// -/// Selects tile width and height so that the working set for one tile -/// (three rows of context neighbours + one output row) fits within the -/// L1 data cache of an Apple Silicon performance core. -/// -/// ```swift -/// let (tw, th) = optimalTileSize(imageWidth: 3840, imageHeight: 2160, bytesPerSample: 2) -/// // tw ≈ 512, th ≈ 16 (fits within 128 KiB L1 cache) -/// ``` -/// -/// - Parameters: -/// - imageWidth: Full image width in pixels -/// - imageHeight: Full image height in pixels -/// - bytesPerSample: Bytes per sample (1 for 8-bit, 2 for 16-bit) -/// - componentCount: Number of image components (1 = greyscale, 3 = RGB) -/// - Returns: A tuple `(tileWidth, tileHeight)` optimised for Apple Silicon L1 cache -public func optimalTileSize( - imageWidth: Int, - imageHeight: Int, - bytesPerSample: Int = 1, - componentCount: Int = 1 -) -> (tileWidth: Int, tileHeight: Int) { - // Budget: L1 cache; reserve 25% for stack/code, use 75% for pixel data - let budget = (AppleSiliconCacheParameters.l1DataCacheSize * 3) / 4 - - // Working set per row = width * bytesPerSample * componentCount - // We need ~4 rows in cache at once (current row + 3 context rows) - let rowSize = imageWidth * bytesPerSample * componentCount - let rowsInBudget = max(1, budget / max(1, rowSize)) - let tileHeight = min(imageHeight, max(1, rowsInBudget / 4)) - - // Tile width: round up to the nearest cache-line boundary in samples, - // then clamp to the actual image width. - let samplesPerCacheLine = AppleSiliconCacheParameters.cacheLineSize / max(1, bytesPerSample) - let alignedWidth = ((imageWidth + samplesPerCacheLine - 1) / samplesPerCacheLine) * samplesPerCacheLine - let tileWidth = min(imageWidth, alignedWidth) - - return (tileWidth, tileHeight) -} - -// MARK: - Cache-Line Aligned Buffer Allocation - -/// Allocate a cache-line–aligned integer buffer for JPEG-LS context arrays. -/// -/// Context arrays (A, B, C, N) are accessed with stride equal to one entry -/// per quantised context (up to 365 contexts). Alignment to cache-line -/// boundaries prevents false-sharing on Apple Silicon's multi-cluster design. -/// -/// - Parameter count: Number of elements to allocate -/// - Returns: A zero-initialised `[Int]` of the requested size -/// -/// - Note: Swift arrays are heap-allocated and typically 16-byte aligned. -/// This helper pads `count` to the next multiple of the cache-line stride -/// in `Int` units so that adjacent arrays in a struct avoid cache aliasing. -public func allocateCacheAlignedContextArray(count: Int) -> [Int] { - let alignment = AppleSiliconCacheParameters.cacheLineSize / MemoryLayout.stride - let alignedCount = ((count + alignment - 1) / alignment) * alignment - return [Int](repeating: 0, count: alignedCount) -} - -// MARK: - Memory-Mapped I/O - -/// Open a file for memory-mapped read-only access on Apple platforms. -/// -/// Memory-mapped I/O avoids copying file data into user-space buffers on -/// Apple Silicon's unified memory architecture; the OS kernel maps file -/// pages directly into the process address space using the underlying -/// `mmap(2)` system call. -/// -/// ```swift -/// let data = try memoryMappedData(at: url) -/// // Use `data` as a normal Data value — pages are faulted in on demand -/// ``` -/// -/// - Parameter url: URL of the file to map -/// - Returns: A `Data` value backed by a memory mapping of the file -/// - Throws: `CocoaError` if the file cannot be opened or mapped -public func memoryMappedData(at url: URL) throws -> Data { - return try Data(contentsOf: url, options: .mappedIfSafe) -} - -/// Write data to a file using memory-mapped I/O on Apple platforms. -/// -/// Writes the provided `Data` to `url`. For large files on Apple Silicon -/// unified memory, `mappedIfSafe` avoids redundant copies on the write path. -/// -/// - Parameters: -/// - data: Data to write -/// - url: Destination file URL -/// - Throws: `CocoaError` if the write fails -public func writeMemoryMapped(_ data: Data, to url: URL) throws { - try data.write(to: url, options: .atomic) -} - -// MARK: - Unified Memory Buffer Pool - -/// A buffer pool optimised for Apple Silicon's unified CPU/GPU memory. -/// -/// On Apple Silicon, the CPU and GPU share the same physical memory, so -/// Metal `storageModeShared` buffers are accessible without any copy -/// between host and device. This pool manages a set of pre-allocated -/// reusable `Data` buffers of fixed sizes, reducing allocation pressure -/// during encode/decode loops. -/// -/// ```swift -/// let pool = UnifiedMemoryBufferPool(bufferSize: 512 * 1024, poolCapacity: 4) -/// let buf = pool.acquire() -/// // ... use buf for one encode tile ... -/// pool.release(buf) -/// ``` -public final class UnifiedMemoryBufferPool: @unchecked Sendable { - private let bufferSize: Int - private let poolCapacity: Int - private var available: [Data] = [] - private let lock = NSLock() - - /// Create a new unified-memory buffer pool. - /// - /// - Parameters: - /// - bufferSize: Size of each buffer in bytes - /// - poolCapacity: Maximum number of buffers held in the pool - public init(bufferSize: Int, poolCapacity: Int = 4) { - self.bufferSize = bufferSize - self.poolCapacity = poolCapacity - } - - /// Acquire a buffer from the pool, allocating a new one if empty. - /// - /// - Returns: A `Data` value of `bufferSize` bytes (zeroed on first allocation; - /// contents undefined on subsequent reuse) - public func acquire() -> Data { - lock.lock() - defer { lock.unlock() } - if !available.isEmpty { - return available.removeLast() - } - return Data(count: bufferSize) - } - - /// Return a buffer to the pool for reuse. - /// - /// Buffers exceeding `poolCapacity` are silently discarded to bound - /// peak memory usage. - /// - /// - Parameter buffer: Previously acquired buffer - public func release(_ buffer: Data) { - lock.lock() - defer { lock.unlock() } - if available.count < poolCapacity { - available.append(buffer) - } - } - - /// Pre-warm the pool by allocating `poolCapacity` buffers eagerly. - /// - /// Call this once at startup to avoid allocation latency on the first - /// batch of encode/decode tiles. - public func prewarm() { - lock.lock() - defer { lock.unlock() } - while available.count < poolCapacity { - available.append(Data(count: bufferSize)) - } - } - - /// The number of buffers currently available in the pool. - public var availableCount: Int { - lock.lock() - defer { lock.unlock() } - return available.count - } -} - -// MARK: - Prefetch Hints - -/// Issue a software prefetch hint for the given memory region. -/// -/// On ARM64, the compiler is free to lower this to `PRFM PLDL1KEEP` -/// instructions when inlining. The function hint guides the hardware -/// prefetcher for predictable sequential access patterns during -/// row-by-row JPEG-LS encoding/decoding. -/// -/// - Parameters: -/// - array: Array whose data should be prefetched -/// - startIndex: First element index to prefetch -/// - count: Number of elements to prefetch (one cache line covers 8 Int values on ARM64) -@inline(__always) -public func prefetchContextArray(_ array: [Int], startIndex: Int, count: Int) { - let end = min(startIndex + count, array.count) - guard startIndex < end else { return } - - // Touch the first element of every cache line in the range. - // On ARM64 one cache line holds 64 / MemoryLayout.stride = 8 Int values. - let stride = AppleSiliconCacheParameters.cacheLineSize / MemoryLayout.stride - var i = startIndex - while i < end { - _ = array[i] - i += stride - } -} - -// MARK: - Hardware-Specific Tuning Parameters - -/// Recommended tuning parameters for JPEG-LS on Apple Silicon. -/// -/// These constants document the rationale behind the chosen defaults and -/// serve as a single source of truth for any future benchmark-driven tuning. -public enum AppleSiliconTuningParameters { - /// Recommended RESET threshold for context adaptation on Apple Silicon. - /// - /// A higher RESET value keeps context statistics longer before halving, - /// which is beneficial on Apple Silicon's large register file since the - /// extra arithmetic is cheaper than a cache miss into a cold context. - public static let recommendedReset: Int = 64 - - /// Minimum pixel count for which GPU (Metal) acceleration is beneficial. - /// - /// Below this threshold, CPU processing avoids the fixed overhead of - /// Metal command-buffer encoding and submission. - public static let metalGpuThreshold: Int = 1024 - - /// Strip height (rows) for tiled encoding on Apple Silicon performance cores. - /// - /// Each strip processes `stripHeight` rows of the image at a time. This - /// value is chosen so that the three-row context window (current row plus - /// two predecessor rows) fits comfortably in the L1 data cache. - public static let stripHeight: Int = AppleSiliconCacheParameters.recommendedStripHeight - - /// Context array pre-allocation count (rounded to cache-line boundary). - /// - /// JPEG-LS uses 365 regular contexts + 2 run-interruption contexts. - /// Pre-allocating 384 entries (6 × 64) aligns the end of each context - /// array to a cache-line boundary on ARM64. - public static let contextArrayCount: Int = 384 -} - -#endif // arch(arm64) diff --git a/Sources/JPEGLS/Platform/Accelerate/AccelerateFrameworkAccelerator.swift b/Sources/JPEGLS/Platform/Accelerate/AccelerateFrameworkAccelerator.swift deleted file mode 100644 index 5db6441..0000000 --- a/Sources/JPEGLS/Platform/Accelerate/AccelerateFrameworkAccelerator.swift +++ /dev/null @@ -1,697 +0,0 @@ -/// Apple Accelerate framework-based acceleration using vDSP. -/// -/// This implementation leverages the Apple Accelerate framework for vectorized -/// batch operations on image data. The Accelerate framework provides highly -/// optimized implementations of common signal processing and mathematical -/// operations. -/// -/// **Note**: This file is conditionally compiled only on Apple platforms where -/// the Accelerate framework is available (macOS, iOS, tvOS, watchOS). - -#if canImport(Accelerate) - -import Foundation -import Accelerate - -/// Accelerate framework-based implementation for batch operations. -/// -/// Provides hardware-accelerated batch gradient computation, statistical -/// analysis, and histogram operations optimized for Apple platforms using -/// the vDSP (vector Digital Signal Processing) library. -/// -/// Unlike the ARM64Accelerator which optimizes single-pixel operations, -/// this accelerator focuses on batch operations across multiple pixels -/// to leverage vDSP's highly optimized array processing capabilities. -public struct AccelerateFrameworkAccelerator: Sendable { - public static let platformName = "Accelerate" - - /// Returns true if the Accelerate framework is available. - public static var isSupported: Bool { - return true - } - - /// Initialize an Accelerate framework accelerator - public init() {} - - // MARK: - Batch Gradient Computation - - /// Compute gradients for a batch of pixels using vDSP operations. - /// - /// This function uses Accelerate's vectorized subtraction operations to - /// compute gradients for multiple pixels simultaneously, providing - /// significant performance improvements for large image regions. - /// - /// For each pixel position i: - /// - D1[i] = b[i] - c[i] (horizontal gradient) - /// - D2[i] = a[i] - c[i] (vertical gradient) - /// - D3[i] = c[i] - a[i] (diagonal gradient) - /// - /// - Parameters: - /// - a: Array of north pixel values - /// - b: Array of west pixel values - /// - c: Array of northwest pixel values - /// - Returns: A tuple of three arrays containing the computed gradients (d1, d2, d3) - /// - Precondition: All arrays must have the same length - public func computeGradientsBatch( - a: [Int], - b: [Int], - c: [Int] - ) -> (d1: [Int], d2: [Int], d3: [Int]) { - precondition(a.count == b.count && b.count == c.count, "Arrays must have same length") - - let count = a.count - guard count > 0 else { - return ([], [], []) - } - - // Convert Int arrays to Float for vDSP operations - let aFloat = a.map { Float($0) } - let bFloat = b.map { Float($0) } - let cFloat = c.map { Float($0) } - - var d1Float = [Float](repeating: 0, count: count) - var d2Float = [Float](repeating: 0, count: count) - var d3Float = [Float](repeating: 0, count: count) - - // D1 = b - c (vectorized subtraction) - vDSP_vsub(cFloat, 1, bFloat, 1, &d1Float, 1, vDSP_Length(count)) - - // D2 = a - c (vectorized subtraction) - vDSP_vsub(cFloat, 1, aFloat, 1, &d2Float, 1, vDSP_Length(count)) - - // D3 = c - a (vectorized subtraction) - vDSP_vsub(aFloat, 1, cFloat, 1, &d3Float, 1, vDSP_Length(count)) - - // Convert back to Int - let d1 = d1Float.map { Int($0) } - let d2 = d2Float.map { Int($0) } - let d3 = d3Float.map { Int($0) } - - return (d1, d2, d3) - } - - // MARK: - Statistical Analysis - - /// Compute histogram of pixel values using Accelerate. - /// - /// This function uses vDSP to efficiently compute a histogram of pixel - /// values, which can be useful for analyzing image characteristics and - /// parameter tuning. - /// - /// - Parameters: - /// - pixels: Array of pixel values - /// - binCount: Number of histogram bins - /// - minValue: Minimum value for histogram range - /// - maxValue: Maximum value for histogram range - /// - Returns: Array of histogram bin counts - public func computeHistogram( - pixels: [Int], - binCount: Int, - minValue: Int, - maxValue: Int - ) -> [Int] { - guard !pixels.isEmpty && binCount > 0 && minValue < maxValue else { - return Array(repeating: 0, count: binCount) - } - - // Convert to Float for vDSP operations - let pixelsFloat = pixels.map { Float($0) } - - // Create histogram bins - var histogram = [vDSP_Length](repeating: 0, count: binCount) - - // Compute histogram using vDSP - // Note: vDSP_vhist is available but requires specific setup - // For now, use a manual binning approach optimized with vDSP operations - let range = Float(maxValue - minValue) - let binWidth = range / Float(binCount) - - for value in pixelsFloat { - let normalizedValue = value - Float(minValue) - if normalizedValue >= 0 && normalizedValue <= range { - let binIndex = min(Int(normalizedValue / binWidth), binCount - 1) - histogram[binIndex] += 1 - } - } - - return histogram.map { Int($0) } - } - - /// Compute mean value of an array using vDSP. - /// - /// - Parameter values: Array of values - /// - Returns: The mean value - public func computeMean(values: [Int]) -> Double { - guard !values.isEmpty else { - return 0.0 - } - - let valuesFloat = values.map { Float($0) } - var mean: Float = 0 - - vDSP_meanv(valuesFloat, 1, &mean, vDSP_Length(values.count)) - - return Double(mean) - } - - /// Compute variance of an array using vDSP. - /// - /// - Parameter values: Array of values - /// - Returns: The variance value - public func computeVariance(values: [Int]) -> Double { - guard values.count > 1 else { - return 0.0 - } - - let valuesFloat = values.map { Float($0) } - var mean: Float = 0 - var variance: Float = 0 - - // Compute mean - vDSP_meanv(valuesFloat, 1, &mean, vDSP_Length(values.count)) - - // Compute variance: sum of (x - mean)^2 / (n - 1) - var differences = [Float](repeating: 0, count: values.count) - var meanArray = [Float](repeating: mean, count: values.count) - - // differences = values - mean - vDSP_vsub(meanArray, 1, valuesFloat, 1, &differences, 1, vDSP_Length(values.count)) - - // square the differences - var squaredDifferences = [Float](repeating: 0, count: values.count) - vDSP_vsq(differences, 1, &squaredDifferences, 1, vDSP_Length(values.count)) - - // sum the squared differences - var sum: Float = 0 - vDSP_sve(squaredDifferences, 1, &sum, vDSP_Length(values.count)) - - // divide by (n - 1) for sample variance - variance = sum / Float(values.count - 1) - - return Double(variance) - } - - /// Compute standard deviation of an array using vDSP. - /// - /// - Parameter values: Array of values - /// - Returns: The standard deviation value - public func computeStandardDeviation(values: [Int]) -> Double { - return sqrt(computeVariance(values: values)) - } - - /// Compute minimum and maximum values using vDSP. - /// - /// - Parameter values: Array of values - /// - Returns: A tuple containing (min, max) - public func computeMinMax(values: [Int]) -> (min: Int, max: Int) { - guard !values.isEmpty else { - return (0, 0) - } - - let valuesFloat = values.map { Float($0) } - var min: Float = 0 - var max: Float = 0 - - vDSP_minv(valuesFloat, 1, &min, vDSP_Length(values.count)) - vDSP_maxv(valuesFloat, 1, &max, vDSP_Length(values.count)) - - return (Int(min), Int(max)) - } - - // MARK: - Batch Vector Operations - - /// Add two arrays element-wise using vDSP. - /// - /// - Parameters: - /// - a: First array - /// - b: Second array - /// - Returns: Array containing element-wise sum - /// - Precondition: Arrays must have the same length - public func addArrays(a: [Int], b: [Int]) -> [Int] { - precondition(a.count == b.count, "Arrays must have same length") - - guard !a.isEmpty else { - return [] - } - - let aFloat = a.map { Float($0) } - let bFloat = b.map { Float($0) } - var result = [Float](repeating: 0, count: a.count) - - vDSP_vadd(aFloat, 1, bFloat, 1, &result, 1, vDSP_Length(a.count)) - - return result.map { Int($0) } - } - - /// Subtract two arrays element-wise using vDSP. - /// - /// - Parameters: - /// - a: First array - /// - b: Second array (subtracted from first) - /// - Returns: Array containing element-wise difference (a - b) - /// - Precondition: Arrays must have the same length - public func subtractArrays(a: [Int], b: [Int]) -> [Int] { - precondition(a.count == b.count, "Arrays must have same length") - - guard !a.isEmpty else { - return [] - } - - let aFloat = a.map { Float($0) } - let bFloat = b.map { Float($0) } - var result = [Float](repeating: 0, count: a.count) - - vDSP_vsub(bFloat, 1, aFloat, 1, &result, 1, vDSP_Length(a.count)) - - return result.map { Int($0) } - } - - /// Multiply array by a scalar using vDSP. - /// - /// - Parameters: - /// - array: Input array - /// - scalar: Scalar multiplier - /// - Returns: Array with each element multiplied by scalar - public func multiplyByScalar(array: [Int], scalar: Int) -> [Int] { - guard !array.isEmpty else { - return [] - } - - let arrayFloat = array.map { Float($0) } - var scalarFloat = Float(scalar) - var result = [Float](repeating: 0, count: array.count) - - vDSP_vsmul(arrayFloat, 1, &scalarFloat, &result, 1, vDSP_Length(array.count)) - - return result.map { Int($0) } - } - - // MARK: - vDSP-Accelerated Prediction Error Computation - - /// Compute prediction errors for a batch of pixels using vDSP. - /// - /// For each pixel i, computes: error[i] = actual[i] - predicted[i] - /// - /// Uses `vDSP_vsub` for vectorised subtraction over the entire batch, - /// avoiding per-element overhead for large images. - /// - /// - Parameters: - /// - actual: Array of actual pixel values - /// - predicted: Array of predicted pixel values - /// - Returns: Array of prediction errors - /// - Precondition: Both arrays must have the same length - public func computePredictionErrors(actual: [Int], predicted: [Int]) -> [Int] { - return subtractArrays(a: actual, b: predicted) - } - - /// Compute absolute prediction errors for a batch using vDSP. - /// - /// For each pixel i, computes: absError[i] = |actual[i] - predicted[i]| - /// - /// Uses `vDSP_vsub` followed by `vDSP_vabs` to vectorise both steps. - /// - /// - Parameters: - /// - actual: Array of actual pixel values - /// - predicted: Array of predicted pixel values - /// - Returns: Array of absolute prediction errors - /// - Precondition: Both arrays must have the same length - public func computeAbsolutePredictionErrors(actual: [Int], predicted: [Int]) -> [Int] { - precondition(actual.count == predicted.count, "Arrays must have same length") - - let count = actual.count - guard count > 0 else { return [] } - - let actualFloat = actual.map { Float($0) } - let predictedFloat = predicted.map { Float($0) } - - var errorFloat = [Float](repeating: 0, count: count) - var absErrorFloat = [Float](repeating: 0, count: count) - - // errors = actual - predicted - vDSP_vsub(predictedFloat, 1, actualFloat, 1, &errorFloat, 1, vDSP_Length(count)) - - // absErrors = |errors| - vDSP_vabs(errorFloat, 1, &absErrorFloat, 1, vDSP_Length(count)) - - return absErrorFloat.map { Int($0) } - } - - // MARK: - vDSP-Accelerated Context State Updates - - /// Batch-update context accumulator A using vDSP absolute-value sum. - /// - /// Accumulates |error| into each context's A value. Where contexts are - /// sparse (many different contexts per image line), the scatter step is - /// sequential; the absolute-value computation over all errors is vectorised - /// using `vDSP_vabs`. - /// - /// - Parameters: - /// - aArray: Context accumulator array A (modified in place) - /// - errors: Signed prediction errors for each pixel - /// - contextIndices: Context index for each pixel (parallel to `errors`) - public func updateAccumulatorA( - aArray: inout [Int], - errors: [Int], - contextIndices: [Int] - ) { - precondition(errors.count == contextIndices.count, "Arrays must have same length") - - guard !errors.isEmpty else { return } - - // Compute absolute values directly using Swift integer abs to avoid - // Float conversion overhead and floating-point rounding artefacts. - for i in 0.. [UInt8] { - let componentCount = planes.count - guard componentCount > 0, width > 0, height > 0 else { return [] } - - let pixelCount = width * height - guard planes.allSatisfy({ $0.count == pixelCount }) else { - preconditionFailure("Each plane must contain exactly width × height bytes") - } - - if componentCount == 1 { - return planes[0] - } - - var interleaved = [UInt8](repeating: 0, count: pixelCount * componentCount) - - for p in 0.. [[UInt8]] { - guard componentCount > 0, width > 0, height > 0 else { return [] } - - let pixelCount = width * height - guard interleaved.count == pixelCount * componentCount else { - preconditionFailure("Interleaved buffer size must equal width × height × componentCount") - } - - if componentCount == 1 { - return [interleaved] - } - - var planes = [[UInt8]](repeating: [UInt8](repeating: 0, count: pixelCount), count: componentCount) - - for i in 0.. (r: [Int], g: [Int], b: [Int]) { - precondition(r.count == g.count && g.count == b.count, "Arrays must have same length") - - let count = r.count - guard count > 0 else { return ([], [], []) } - - let rFloat = r.map { Float($0) } - let gFloat = g.map { Float($0) } - let bFloat = b.map { Float($0) } - - var rPrime = [Float](repeating: 0, count: count) - var bPrime = [Float](repeating: 0, count: count) - - // R′ = R − G - vDSP_vsub(gFloat, 1, rFloat, 1, &rPrime, 1, vDSP_Length(count)) - // B′ = B − G - vDSP_vsub(gFloat, 1, bFloat, 1, &bPrime, 1, vDSP_Length(count)) - - return (rPrime.map { Int($0) }, g, bPrime.map { Int($0) }) - } - - /// Apply HP1 inverse colour transform to a batch of transformed pixels using vDSP. - /// - /// HP1 inverse transform: - /// - G = G′ - /// - R = R′ + G′ - /// - B = B′ + G′ - /// - /// - Parameters: - /// - rPrime: Transformed red component values - /// - gPrime: Transformed green component values (unchanged) - /// - bPrime: Transformed blue component values - /// - Returns: Recovered (r, g, b) components as integer arrays - /// - Precondition: All arrays must have the same length - public func applyHP1Inverse(rPrime: [Int], gPrime: [Int], bPrime: [Int]) -> (r: [Int], g: [Int], b: [Int]) { - precondition(rPrime.count == gPrime.count && gPrime.count == bPrime.count, "Arrays must have same length") - - let count = rPrime.count - guard count > 0 else { return ([], [], []) } - - let rPrimeFloat = rPrime.map { Float($0) } - let gPrimeFloat = gPrime.map { Float($0) } - let bPrimeFloat = bPrime.map { Float($0) } - - var r = [Float](repeating: 0, count: count) - var b = [Float](repeating: 0, count: count) - - // R = R′ + G′ - vDSP_vadd(rPrimeFloat, 1, gPrimeFloat, 1, &r, 1, vDSP_Length(count)) - // B = B′ + G′ - vDSP_vadd(bPrimeFloat, 1, gPrimeFloat, 1, &b, 1, vDSP_Length(count)) - - return (r.map { Int($0) }, gPrime, b.map { Int($0) }) - } - - /// Apply HP2 forward colour transform to a batch of RGB pixels using vDSP. - /// - /// HP2 forward transform (lossless, reversible): - /// - G′ = G - /// - R′ = R − G - /// - B′ = B − ((R + G) >> 1) - /// - /// Note: The arithmetic right-shift step is computed per-element after the - /// vDSP vectorised addition, since integer right-shift cannot be expressed - /// directly as a vDSP primitive. - /// - /// - Parameters: - /// - r: Red component values - /// - g: Green component values - /// - b: Blue component values - /// - Returns: Transformed (r′, g′, b′) components as integer arrays - /// - Precondition: All arrays must have the same length - public func applyHP2Forward(r: [Int], g: [Int], b: [Int]) -> (r: [Int], g: [Int], b: [Int]) { - precondition(r.count == g.count && g.count == b.count, "Arrays must have same length") - - let count = r.count - guard count > 0 else { return ([], [], []) } - - let rFloat = r.map { Float($0) } - let gFloat = g.map { Float($0) } - let bFloat = b.map { Float($0) } - - var rPrime = [Float](repeating: 0, count: count) - var rPlusG = [Float](repeating: 0, count: count) - - // R′ = R − G - vDSP_vsub(gFloat, 1, rFloat, 1, &rPrime, 1, vDSP_Length(count)) - - // R + G (for the B′ formula) - vDSP_vadd(rFloat, 1, gFloat, 1, &rPlusG, 1, vDSP_Length(count)) - - // B′ = B − ((R + G) >> 1) — integer arithmetic shift - let bPrime = zip(bFloat, rPlusG).map { bVal, rgVal in - Int(bVal) - (Int(rgVal) >> 1) - } - - return (rPrime.map { Int($0) }, g, bPrime) - } - - /// Apply HP2 inverse colour transform to a batch of transformed pixels using vDSP. - /// - /// HP2 inverse transform: - /// - G = G′ - /// - R = R′ + G′ - /// - B = B′ + ((R + G) >> 1) - /// - /// - Parameters: - /// - rPrime: Transformed red component values - /// - gPrime: Transformed green component values (unchanged) - /// - bPrime: Transformed blue component values - /// - Returns: Recovered (r, g, b) components as integer arrays - /// - Precondition: All arrays must have the same length - public func applyHP2Inverse(rPrime: [Int], gPrime: [Int], bPrime: [Int]) -> (r: [Int], g: [Int], b: [Int]) { - precondition(rPrime.count == gPrime.count && gPrime.count == bPrime.count, "Arrays must have same length") - - let count = rPrime.count - guard count > 0 else { return ([], [], []) } - - let rPrimeFloat = rPrime.map { Float($0) } - let gPrimeFloat = gPrime.map { Float($0) } - - var r = [Float](repeating: 0, count: count) - - // R = R′ + G′ - vDSP_vadd(rPrimeFloat, 1, gPrimeFloat, 1, &r, 1, vDSP_Length(count)) - - let rInt = r.map { Int($0) } - - // B = B′ + ((R + G) >> 1) - let b = zip(bPrime, zip(rInt, gPrime)).map { bVal, rg in - bVal + ((rg.0 + rg.1) >> 1) - } - - return (rInt, gPrime, b) - } - - /// Apply HP3 forward colour transform to a batch of RGB pixels using vDSP. - /// - /// HP3 forward transform (lossless, reversible): - /// - B′ = B - /// - R′ = R − B - /// - G′ = G − ((R + B) >> 1) - /// - /// - Parameters: - /// - r: Red component values - /// - g: Green component values - /// - b: Blue component values - /// - Returns: Transformed (r′, g′, b′) components as integer arrays - /// - Precondition: All arrays must have the same length - public func applyHP3Forward(r: [Int], g: [Int], b: [Int]) -> (r: [Int], g: [Int], b: [Int]) { - precondition(r.count == g.count && g.count == b.count, "Arrays must have same length") - - let count = r.count - guard count > 0 else { return ([], [], []) } - - let rFloat = r.map { Float($0) } - let gFloat = g.map { Float($0) } - let bFloat = b.map { Float($0) } - - var rPrime = [Float](repeating: 0, count: count) - var rPlusB = [Float](repeating: 0, count: count) - - // R′ = R − B - vDSP_vsub(bFloat, 1, rFloat, 1, &rPrime, 1, vDSP_Length(count)) - - // R + B - vDSP_vadd(rFloat, 1, bFloat, 1, &rPlusB, 1, vDSP_Length(count)) - - // G′ = G − ((R + B) >> 1) - let gPrime = zip(gFloat, rPlusB).map { gVal, rbVal in - Int(gVal) - (Int(rbVal) >> 1) - } - - return (rPrime.map { Int($0) }, gPrime, b) - } - - /// Apply HP3 inverse colour transform to a batch of transformed pixels using vDSP. - /// - /// HP3 inverse transform: - /// - B = B′ - /// - R = R′ + B′ - /// - G = G′ + ((R + B) >> 1) - /// - /// - Parameters: - /// - rPrime: Transformed red component values - /// - gPrime: Transformed green component values - /// - bPrime: Transformed blue component values (unchanged) - /// - Returns: Recovered (r, g, b) components as integer arrays - /// - Precondition: All arrays must have the same length - public func applyHP3Inverse(rPrime: [Int], gPrime: [Int], bPrime: [Int]) -> (r: [Int], g: [Int], b: [Int]) { - precondition(rPrime.count == gPrime.count && gPrime.count == bPrime.count, "Arrays must have same length") - - let count = rPrime.count - guard count > 0 else { return ([], [], []) } - - let rPrimeFloat = rPrime.map { Float($0) } - let bPrimeFloat = bPrime.map { Float($0) } - - var r = [Float](repeating: 0, count: count) - - // R = R′ + B′ - vDSP_vadd(rPrimeFloat, 1, bPrimeFloat, 1, &r, 1, vDSP_Length(count)) - - let rInt = r.map { Int($0) } - - // G = G′ + ((R + B) >> 1) - let g = zip(gPrime, zip(rInt, bPrime)).map { gVal, rb in - gVal + ((rb.0 + rb.1) >> 1) - } - - return (rInt, g, bPrime) - } -} - -#endif diff --git a/Sources/JPEGLS/Platform/Metal/JPEGLSShaders.metal b/Sources/JPEGLS/Platform/Metal/JPEGLSShaders.metal deleted file mode 100644 index dfb354a..0000000 --- a/Sources/JPEGLS/Platform/Metal/JPEGLSShaders.metal +++ /dev/null @@ -1,434 +0,0 @@ -// Metal compute shaders for JPEG-LS GPU acceleration. -// -// These shaders implement GPU-accelerated gradient computation, -// MED prediction, colour space transformation, and gradient -// quantisation for JPEG-LS encoding. They are designed to process -// large batches of pixels in parallel on the GPU. -// -// Metal Shading Language (MSL) version 2.0+ - -#include -using namespace metal; - -/// Compute gradients for a batch of pixels. -/// -/// For each pixel position i, computes: -/// - d1[i] = b[i] - c[i] (horizontal gradient) -/// - d2[i] = a[i] - c[i] (vertical gradient) -/// - d3[i] = c[i] - a[i] (diagonal gradient) -/// -/// Thread layout: 1D with one thread per pixel -kernel void compute_gradients( - constant int* a [[buffer(0)]], // North pixel values - constant int* b [[buffer(1)]], // West pixel values - constant int* c [[buffer(2)]], // Northwest pixel values - device int* d1 [[buffer(3)]], // Output: horizontal gradients - device int* d2 [[buffer(4)]], // Output: vertical gradients - device int* d3 [[buffer(5)]], // Output: diagonal gradients - constant uint& count [[buffer(6)]], // Number of elements - uint gid [[thread_position_in_grid]] -) { - // Bounds check - if (gid >= count) { - return; - } - - // Load pixel values - int av = a[gid]; - int bv = b[gid]; - int cv = c[gid]; - - // Compute gradients - d1[gid] = bv - cv; // Horizontal gradient - d2[gid] = av - cv; // Vertical gradient - d3[gid] = cv - av; // Diagonal gradient -} - -/// Compute MED (Median Edge Detector) predictions for a batch of pixels. -/// -/// Implements the JPEG-LS MED predictor: -/// - If c >= max(a, b): return min(a, b) -/// - If c <= min(a, b): return max(a, b) -/// - Otherwise: return a + b - c -/// -/// Thread layout: 1D with one thread per pixel -kernel void compute_med_prediction( - constant int* a [[buffer(0)]], // North pixel values - constant int* b [[buffer(1)]], // West pixel values - constant int* c [[buffer(2)]], // Northwest pixel values - device int* pred [[buffer(3)]], // Output: predicted values - constant uint& count [[buffer(4)]], // Number of elements - uint gid [[thread_position_in_grid]] -) { - // Bounds check - if (gid >= count) { - return; - } - - // Load pixel values - int av = a[gid]; - int bv = b[gid]; - int cv = c[gid]; - - // Compute min and max of a and b - int minAB = min(av, bv); - int maxAB = max(av, bv); - - // MED predictor logic - int prediction; - if (cv >= maxAB) { - // c >= max(a, b) → return min(a, b) - prediction = minAB; - } else if (cv <= minAB) { - // c <= min(a, b) → return max(a, b) - prediction = maxAB; - } else { - // Otherwise → return a + b - c - prediction = av + bv - cv; - } - - pred[gid] = prediction; -} - -// MARK: - Gradient Quantisation - -/// Quantise a single gradient value to a context index in [-4, 4]. -/// -/// Applies the JPEG-LS threshold quantisation mapping: -/// d <= -t3 → -4, d <= -t2 → -3, d <= -t1 → -2, d < 0 → -1, -/// d == 0 → 0, d < t1 → 1, d < t2 → 2, d < t3 → 3, else → 4 -static inline int quantise_gradient(int d, int t1, int t2, int t3) { - if (d <= -t3) return -4; - if (d <= -t2) return -3; - if (d <= -t1) return -2; - if (d < 0) return -1; - if (d == 0) return 0; - if (d < t1) return 1; - if (d < t2) return 2; - if (d < t3) return 3; - return 4; -} - -/// Quantise a gradient value with NEAR-lossless awareness (ITU-T.87 §4.3.1). -/// -/// Extends the basic quantisation to support the NEAR parameter: -/// d < -near → -1, -near <= d <= near → 0, d < t1 → 1, … -/// When near == 0 this is identical to `quantise_gradient`. -static inline int quantise_gradient_near(int d, int t1, int t2, int t3, int near) { - if (d <= -t3) return -4; - if (d <= -t2) return -3; - if (d <= -t1) return -2; - if (d < -near) return -1; - if (d <= near) return 0; - if (d < t1) return 1; - if (d < t2) return 2; - if (d < t3) return 3; - return 4; -} - -/// Compute the MED (Median Edge Detector) prediction from three neighbours. -/// -/// - a: north (top) pixel -/// - b: west (left) pixel -/// - c: northwest (top-left) pixel -static inline int med_predict(int a, int b, int c) { - int minAB = min(a, b); - int maxAB = max(a, b); - if (c >= maxAB) return minAB; - if (c <= minAB) return maxAB; - return a + b - c; -} - -/// Quantise a batch of gradients to context indices using JPEG-LS thresholds. -/// -/// For each element i, applies threshold quantisation to d1, d2, and d3, -/// mapping each gradient to a value in [-4, 4]. -/// -/// Thread layout: 1D with one thread per pixel -kernel void compute_quantize_gradients( - constant int* d1 [[buffer(0)]], // Input: first gradients - constant int* d2 [[buffer(1)]], // Input: second gradients - constant int* d3 [[buffer(2)]], // Input: third gradients - device int* q1 [[buffer(3)]], // Output: first quantised gradients - device int* q2 [[buffer(4)]], // Output: second quantised gradients - device int* q3 [[buffer(5)]], // Output: third quantised gradients - constant uint& count [[buffer(6)]], // Number of elements - constant int& t1 [[buffer(7)]], // Quantisation threshold 1 - constant int& t2 [[buffer(8)]], // Quantisation threshold 2 - constant int& t3 [[buffer(9)]], // Quantisation threshold 3 - uint gid [[thread_position_in_grid]] -) { - if (gid >= count) { - return; - } - q1[gid] = quantise_gradient(d1[gid], t1, t2, t3); - q2[gid] = quantise_gradient(d2[gid], t1, t2, t3); - q3[gid] = quantise_gradient(d3[gid], t1, t2, t3); -} - -// MARK: - Colour Space Transformations - -/// Apply HP1 forward colour transform to a batch of RGB pixels. -/// -/// HP1 forward transform (lossless, reversible): -/// R′ = R − G -/// G′ = G -/// B′ = B − G -/// -/// Thread layout: 1D with one thread per pixel -kernel void compute_colour_transform_hp1_forward( - constant int* r [[buffer(0)]], // Input: red component - constant int* g [[buffer(1)]], // Input: green component - constant int* b [[buffer(2)]], // Input: blue component - device int* rPrime [[buffer(3)]], // Output: transformed red - device int* gPrime [[buffer(4)]], // Output: transformed green (= G) - device int* bPrime [[buffer(5)]], // Output: transformed blue - constant uint& count [[buffer(6)]], // Number of pixels - uint gid [[thread_position_in_grid]] -) { - if (gid >= count) { - return; - } - rPrime[gid] = r[gid] - g[gid]; - gPrime[gid] = g[gid]; - bPrime[gid] = b[gid] - g[gid]; -} - -/// Apply HP1 inverse colour transform to a batch of transformed pixels. -/// -/// HP1 inverse transform: -/// R = R′ + G′ -/// G = G′ -/// B = B′ + G′ -/// -/// Thread layout: 1D with one thread per pixel -kernel void compute_colour_transform_hp1_inverse( - constant int* rPrime [[buffer(0)]], // Input: transformed red - constant int* gPrime [[buffer(1)]], // Input: transformed green (= G) - constant int* bPrime [[buffer(2)]], // Input: transformed blue - device int* r [[buffer(3)]], // Output: recovered red - device int* g [[buffer(4)]], // Output: recovered green - device int* b [[buffer(5)]], // Output: recovered blue - constant uint& count [[buffer(6)]], // Number of pixels - uint gid [[thread_position_in_grid]] -) { - if (gid >= count) { - return; - } - r[gid] = rPrime[gid] + gPrime[gid]; - g[gid] = gPrime[gid]; - b[gid] = bPrime[gid] + gPrime[gid]; -} - -/// Apply HP2 forward colour transform to a batch of RGB pixels. -/// -/// HP2 forward transform (lossless, reversible): -/// R′ = R − G -/// G′ = G -/// B′ = B − ((R + G) >> 1) -/// -/// The arithmetic right-shift (>> 1) performs floor division by 2. -/// -/// Thread layout: 1D with one thread per pixel -kernel void compute_colour_transform_hp2_forward( - constant int* r [[buffer(0)]], - constant int* g [[buffer(1)]], - constant int* b [[buffer(2)]], - device int* rPrime [[buffer(3)]], - device int* gPrime [[buffer(4)]], - device int* bPrime [[buffer(5)]], - constant uint& count [[buffer(6)]], - uint gid [[thread_position_in_grid]] -) { - if (gid >= count) { - return; - } - int rv = r[gid]; - int gv = g[gid]; - int bv = b[gid]; - rPrime[gid] = rv - gv; - gPrime[gid] = gv; - bPrime[gid] = bv - ((rv + gv) >> 1); -} - -/// Apply HP2 inverse colour transform to a batch of transformed pixels. -/// -/// HP2 inverse transform: -/// R = R′ + G′ -/// G = G′ -/// B = B′ + ((R + G) >> 1) -/// -/// Thread layout: 1D with one thread per pixel -kernel void compute_colour_transform_hp2_inverse( - constant int* rPrime [[buffer(0)]], - constant int* gPrime [[buffer(1)]], - constant int* bPrime [[buffer(2)]], - device int* r [[buffer(3)]], - device int* g [[buffer(4)]], - device int* b [[buffer(5)]], - constant uint& count [[buffer(6)]], - uint gid [[thread_position_in_grid]] -) { - if (gid >= count) { - return; - } - int rv = rPrime[gid] + gPrime[gid]; // R = R′ + G′ - int gv = gPrime[gid]; - r[gid] = rv; - g[gid] = gv; - b[gid] = bPrime[gid] + ((rv + gv) >> 1); -} - -/// Apply HP3 forward colour transform to a batch of RGB pixels. -/// -/// HP3 forward transform (lossless, reversible): -/// B′ = B -/// R′ = R − B -/// G′ = G − ((R + B) >> 1) -/// -/// Thread layout: 1D with one thread per pixel -kernel void compute_colour_transform_hp3_forward( - constant int* r [[buffer(0)]], - constant int* g [[buffer(1)]], - constant int* b [[buffer(2)]], - device int* rPrime [[buffer(3)]], - device int* gPrime [[buffer(4)]], - device int* bPrime [[buffer(5)]], - constant uint& count [[buffer(6)]], - uint gid [[thread_position_in_grid]] -) { - if (gid >= count) { - return; - } - int rv = r[gid]; - int gv = g[gid]; - int bv = b[gid]; - rPrime[gid] = rv - bv; - gPrime[gid] = gv - ((rv + bv) >> 1); - bPrime[gid] = bv; -} - -/// Apply HP3 inverse colour transform to a batch of transformed pixels. -/// -/// HP3 inverse transform: -/// B = B′ -/// R = R′ + B′ -/// G = G′ + ((R + B) >> 1) -/// -/// Thread layout: 1D with one thread per pixel -kernel void compute_colour_transform_hp3_inverse( - constant int* rPrime [[buffer(0)]], - constant int* gPrime [[buffer(1)]], - constant int* bPrime [[buffer(2)]], - device int* r [[buffer(3)]], - device int* g [[buffer(4)]], - device int* b [[buffer(5)]], - constant uint& count [[buffer(6)]], - uint gid [[thread_position_in_grid]] -) { - if (gid >= count) { - return; - } - int bv = bPrime[gid]; - int rv = rPrime[gid] + bv; // R = R′ + B′ - r[gid] = rv; - g[gid] = gPrime[gid] + ((rv + bv) >> 1); - b[gid] = bv; -} - -// MARK: - Full Encoding Preprocessing Pipeline - -/// Combined encoding preprocessing shader. -/// -/// Performs the complete JPEG-LS encoding preprocessing for a batch of pixels -/// in a single GPU pass, replacing three separate dispatch calls -/// (compute_gradients + compute_med_prediction + compute_quantize_gradients) -/// with one. This reduces dispatch overhead and improves data locality. -/// -/// For each pixel position i, given neighbours a (north), b (west), -/// c (northwest) and the current pixel x, the shader outputs: -/// - prediction[i] = MED(a, b, c) prediction value -/// - predError[i] = x[i] − prediction[i] (raw prediction error) -/// - q1[i], q2[i], q3[i] = quantised gradients (context indices in [−4, 4]) -/// -/// The quantisation uses the NEAR-aware formula: -/// d < −NEAR → −1, −NEAR ≤ d ≤ NEAR → 0 -/// which reduces to the standard lossless mapping when NEAR = 0. -/// -/// Thread layout: 1D with one thread per pixel -kernel void compute_encoding_pipeline( - constant int* a [[buffer(0)]], // North pixel values - constant int* b [[buffer(1)]], // West pixel values - constant int* c [[buffer(2)]], // Northwest pixel values - constant int* x [[buffer(3)]], // Current pixel values - device int* prediction [[buffer(4)]], // Output: MED predictions - device int* predError [[buffer(5)]], // Output: raw prediction errors - device int* q1 [[buffer(6)]], // Output: quantised gradient 1 - device int* q2 [[buffer(7)]], // Output: quantised gradient 2 - device int* q3 [[buffer(8)]], // Output: quantised gradient 3 - constant uint& count [[buffer(9)]], // Number of elements - constant int& near [[buffer(10)]], // NEAR parameter (0 = lossless) - constant int& t1 [[buffer(11)]], // Quantisation threshold 1 - constant int& t2 [[buffer(12)]], // Quantisation threshold 2 - constant int& t3 [[buffer(13)]], // Quantisation threshold 3 - uint gid [[thread_position_in_grid]] -) { - if (gid >= count) { - return; - } - - int av = a[gid]; - int bv = b[gid]; - int cv = c[gid]; - int xv = x[gid]; - - // Gradients (convention: a=north, b=west, c=northwest) - int d1 = bv - cv; - int d2 = av - cv; - int d3 = cv - av; - - // Quantise gradients with NEAR-aware formula - q1[gid] = quantise_gradient_near(d1, t1, t2, t3, near); - q2[gid] = quantise_gradient_near(d2, t1, t2, t3, near); - q3[gid] = quantise_gradient_near(d3, t1, t2, t3, near); - - // MED prediction - int px = med_predict(av, bv, cv); - prediction[gid] = px; - - // Raw prediction error - predError[gid] = xv - px; -} - -// MARK: - Full Decoding Reconstruction Pipeline - -/// Combined decoding reconstruction shader. -/// -/// Performs the JPEG-LS decoding reconstruction for a batch of pixels in a -/// single GPU pass. Given the MED neighbours (a, b, c) and the mapped -/// prediction error for each pixel (output from Golomb-Rice entropy decoding -/// on the CPU), reconstructs the original pixel values. -/// -/// For each pixel position i: -/// reconstructed[i] = MED(a[i], b[i], c[i]) + errval[i] -/// -/// The entropy-decoded error values must already be de-mapped (signed) -/// before being passed to this shader. -/// -/// Thread layout: 1D with one thread per pixel -kernel void compute_decoding_pipeline( - constant int* a [[buffer(0)]], // North pixel (already decoded) - constant int* b [[buffer(1)]], // West pixel (already decoded) - constant int* c [[buffer(2)]], // Northwest pixel (already decoded) - constant int* errval [[buffer(3)]], // Mapped prediction error - device int* reconstructed [[buffer(4)]], // Output: reconstructed pixel values - constant uint& count [[buffer(5)]], // Number of elements - uint gid [[thread_position_in_grid]] -) { - if (gid >= count) { - return; - } - - int px = med_predict(a[gid], b[gid], c[gid]); - reconstructed[gid] = px + errval[gid]; -} diff --git a/Sources/JPEGLS/Platform/Metal/MetalAccelerator.swift b/Sources/JPEGLS/Platform/Metal/MetalAccelerator.swift deleted file mode 100644 index 096dd0e..0000000 --- a/Sources/JPEGLS/Platform/Metal/MetalAccelerator.swift +++ /dev/null @@ -1,882 +0,0 @@ -/// Metal GPU-based acceleration for JPEG-LS operations. -/// -/// This implementation leverages Apple's Metal framework to perform GPU-accelerated -/// encoding operations for large images. Metal compute shaders provide massive -/// parallelism for pixel-level operations, making them ideal for processing large -/// medical images and high-resolution data. -/// -/// **Note**: This file is conditionally compiled only on Apple platforms where -/// Metal is available (macOS 10.13+, iOS 11+). -/// -/// **Architecture**: -/// - GPU is used for batch operations on large image tiles -/// - CPU fallback is used for small images where GPU overhead exceeds benefits -/// - Automatic workload distribution between GPU and CPU based on image size -/// - Efficient memory transfer using shared Metal buffers - -#if canImport(Metal) - -import Foundation -import Metal - -/// Metal GPU-accelerated implementation for JPEG-LS operations. -/// -/// Provides GPU-accelerated batch gradient computation, prediction, colour space -/// transformation, and context operations optimised for large images on Apple -/// platforms with Metal support. -/// -/// The implementation uses Metal compute shaders to process multiple pixels -/// in parallel on the GPU, significantly improving performance for large images -/// while falling back to CPU for small images where GPU overhead is not justified. -public final class MetalAccelerator: @unchecked Sendable { - public static let platformName = "Metal" - - /// The Metal device used for GPU operations. - private let device: MTLDevice - - /// The command queue for submitting GPU work. - private let commandQueue: MTLCommandQueue - - /// The compute pipeline state for gradient computation. - private let gradientPipelineState: MTLComputePipelineState - - /// The compute pipeline state for MED prediction. - private let predictionPipelineState: MTLComputePipelineState - - /// The compute pipeline state for gradient quantisation. - private let quantizeGradientsPipelineState: MTLComputePipelineState - - /// The compute pipeline state for HP1 forward colour transform. - private let colourTransformHP1ForwardPipelineState: MTLComputePipelineState - - /// The compute pipeline state for HP1 inverse colour transform. - private let colourTransformHP1InversePipelineState: MTLComputePipelineState - - /// The compute pipeline state for HP2 forward colour transform. - private let colourTransformHP2ForwardPipelineState: MTLComputePipelineState - - /// The compute pipeline state for HP2 inverse colour transform. - private let colourTransformHP2InversePipelineState: MTLComputePipelineState - - /// The compute pipeline state for HP3 forward colour transform. - private let colourTransformHP3ForwardPipelineState: MTLComputePipelineState - - /// The compute pipeline state for HP3 inverse colour transform. - private let colourTransformHP3InversePipelineState: MTLComputePipelineState - - /// The compute pipeline state for the combined encoding preprocessing pipeline. - private let encodingPipelineState: MTLComputePipelineState - - /// The compute pipeline state for the combined decoding reconstruction pipeline. - private let decodingPipelineState: MTLComputePipelineState - - /// Minimum number of pixels to use GPU (below this, use CPU fallback). - /// This threshold is determined empirically based on GPU overhead vs. benefit. - public static let gpuThreshold = 1024 - - /// Returns true if Metal is available on this device. - public static var isSupported: Bool { - return MTLCreateSystemDefaultDevice() != nil - } - - /// Initialize a Metal GPU accelerator. - /// - /// - Throws: `MetalAcceleratorError` if Metal initialization fails - public init() throws { - guard let device = MTLCreateSystemDefaultDevice() else { - throw MetalAcceleratorError.metalNotAvailable - } - - guard let queue = device.makeCommandQueue() else { - throw MetalAcceleratorError.commandQueueCreationFailed - } - - self.device = device - self.commandQueue = queue - - // Create compute pipeline states for all shaders - do { - let library = try Self.loadShaderLibrary(device: device) - - self.gradientPipelineState = try Self.makePipelineState( - library: library, device: device, functionName: "compute_gradients") - self.predictionPipelineState = try Self.makePipelineState( - library: library, device: device, functionName: "compute_med_prediction") - self.quantizeGradientsPipelineState = try Self.makePipelineState( - library: library, device: device, functionName: "compute_quantize_gradients") - self.colourTransformHP1ForwardPipelineState = try Self.makePipelineState( - library: library, device: device, functionName: "compute_colour_transform_hp1_forward") - self.colourTransformHP1InversePipelineState = try Self.makePipelineState( - library: library, device: device, functionName: "compute_colour_transform_hp1_inverse") - self.colourTransformHP2ForwardPipelineState = try Self.makePipelineState( - library: library, device: device, functionName: "compute_colour_transform_hp2_forward") - self.colourTransformHP2InversePipelineState = try Self.makePipelineState( - library: library, device: device, functionName: "compute_colour_transform_hp2_inverse") - self.colourTransformHP3ForwardPipelineState = try Self.makePipelineState( - library: library, device: device, functionName: "compute_colour_transform_hp3_forward") - self.colourTransformHP3InversePipelineState = try Self.makePipelineState( - library: library, device: device, functionName: "compute_colour_transform_hp3_inverse") - self.encodingPipelineState = try Self.makePipelineState( - library: library, device: device, functionName: "compute_encoding_pipeline") - self.decodingPipelineState = try Self.makePipelineState( - library: library, device: device, functionName: "compute_decoding_pipeline") - - } catch let e as MetalAcceleratorError { - throw e - } catch { - throw MetalAcceleratorError.pipelineCreationFailed(error) - } - } - - // MARK: - Private Helpers - - /// Load the Metal shader library for `JPEGLSShaders.metal`. - /// - /// Tries the precompiled `default.metallib` first — this is produced when the - /// package is built by Xcode's build system. Under Swift Package Manager on the - /// command line (`swift build`/`swift test`), `.process` only *copies* the - /// `.metal` file into the resource bundle without compiling it, so no - /// `default.metallib` exists and `makeDefaultLibrary(bundle:)` fails. In that - /// case we locate the bundled `.metal` source and compile it at runtime, so the - /// GPU path works in both environments. - private static func loadShaderLibrary(device: MTLDevice) throws -> MTLLibrary { - // Fast path: precompiled default library (Xcode-built bundles). - if let library = try? device.makeDefaultLibrary(bundle: .module) { - return library - } - - // Fallback: compile the bundled shader source at runtime (SPM CLI builds). - guard let sourceURL = Bundle.module.url( - forResource: "JPEGLSShaders", withExtension: "metal" - ) else { - throw MetalAcceleratorError.libraryCreationFailed( - "JPEGLSShaders.metal not found in module bundle and no default.metallib present") - } - - let source: String - do { - source = try String(contentsOf: sourceURL, encoding: .utf8) - } catch { - throw MetalAcceleratorError.libraryCreationFailed( - "Failed to read JPEGLSShaders.metal: \(error)") - } - - do { - return try device.makeLibrary(source: source, options: nil) - } catch { - throw MetalAcceleratorError.libraryCreationFailed( - "Runtime compilation of JPEGLSShaders.metal failed: \(error)") - } - } - - /// Create a compute pipeline state for the named shader function. - private static func makePipelineState( - library: MTLLibrary, - device: MTLDevice, - functionName: String - ) throws -> MTLComputePipelineState { - guard let function = library.makeFunction(name: functionName) else { - throw MetalAcceleratorError.shaderFunctionNotFound(functionName) - } - return try device.makeComputePipelineState(function: function) - } - - /// Execute a 1-D compute dispatch and wait for completion. - /// - /// - Parameters: - /// - pipelineState: The compute pipeline state to use. - /// - count: Number of elements (threads) to dispatch. - /// - configure: Closure that sets buffers/bytes on the encoder before dispatch. - /// - Throws: `MetalAcceleratorError` if the command buffer fails. - private func dispatch1D( - pipelineState: MTLComputePipelineState, - count: Int, - configure: (MTLComputeCommandEncoder) -> Void - ) throws { - guard let commandBuffer = commandQueue.makeCommandBuffer(), - let encoder = commandBuffer.makeComputeCommandEncoder() else { - throw MetalAcceleratorError.commandBufferCreationFailed - } - - encoder.setComputePipelineState(pipelineState) - configure(encoder) - - let threadGroupWidth = min(pipelineState.maxTotalThreadsPerThreadgroup, count) - let threadGroupSize = MTLSize(width: threadGroupWidth, height: 1, depth: 1) - let threadGroups = MTLSize( - width: (count + threadGroupWidth - 1) / threadGroupWidth, - height: 1, depth: 1) - - encoder.dispatchThreadgroups(threadGroups, threadsPerThreadgroup: threadGroupSize) - encoder.endEncoding() - - commandBuffer.commit() - commandBuffer.waitUntilCompleted() - - if commandBuffer.status == .error { - throw MetalAcceleratorError.commandBufferExecutionFailed - } - } - - /// Create a read-only Metal buffer from a Swift array. - private func makeReadBuffer(_ array: [T]) throws -> MTLBuffer { - let size = array.count * MemoryLayout.stride - guard let buf = device.makeBuffer(bytes: array, length: size, options: .storageModeShared) else { - throw MetalAcceleratorError.bufferCreationFailed - } - return buf - } - - /// Create a write-only Metal buffer of the given element count. - private func makeWriteBuffer(count: Int, type: T.Type) throws -> MTLBuffer { - let size = count * MemoryLayout.stride - guard let buf = device.makeBuffer(length: size, options: .storageModeShared) else { - throw MetalAcceleratorError.bufferCreationFailed - } - return buf - } - - /// Read the contents of a Metal buffer as a Swift array. - private func readBuffer(_ buffer: MTLBuffer, count: Int, type: T.Type) -> [T] { - let pointer = buffer.contents().bindMemory(to: T.self, capacity: count) - return Array(UnsafeBufferPointer(start: pointer, count: count)) - } - - // MARK: - Batch Gradient Computation - - /// Compute gradients for a batch of pixels using Metal GPU. - /// - /// This function dispatches compute shaders to the GPU to compute gradients - /// for multiple pixels in parallel. For small batches (< gpuThreshold pixels), - /// falls back to CPU computation to avoid GPU overhead. - /// - /// For each pixel position i: - /// - D1[i] = b[i] - c[i] (horizontal gradient) - /// - D2[i] = a[i] - c[i] (vertical gradient) - /// - D3[i] = c[i] - a[i] (diagonal gradient) - /// - /// - Parameters: - /// - a: Array of north pixel values - /// - b: Array of west pixel values - /// - c: Array of northwest pixel values - /// - Returns: A tuple of three arrays containing the computed gradients (d1, d2, d3) - /// - Throws: `MetalAcceleratorError` if GPU operations fail - /// - Precondition: All arrays must have the same length - public func computeGradientsBatch( - a: [Int32], - b: [Int32], - c: [Int32] - ) throws -> (d1: [Int32], d2: [Int32], d3: [Int32]) { - precondition(a.count == b.count && b.count == c.count, "Arrays must have same length") - - let count = a.count - guard count > 0 else { - return ([], [], []) - } - - // Use CPU fallback for small batches - if count < Self.gpuThreshold { - return computeGradientsBatchCPU(a: a, b: b, c: c) - } - - let aBuffer = try makeReadBuffer(a) - let bBuffer = try makeReadBuffer(b) - let cBuffer = try makeReadBuffer(c) - let d1Buffer = try makeWriteBuffer(count: count, type: Int32.self) - let d2Buffer = try makeWriteBuffer(count: count, type: Int32.self) - let d3Buffer = try makeWriteBuffer(count: count, type: Int32.self) - - var elementCount = UInt32(count) - try dispatch1D(pipelineState: gradientPipelineState, count: count) { encoder in - encoder.setBuffer(aBuffer, offset: 0, index: 0) - encoder.setBuffer(bBuffer, offset: 0, index: 1) - encoder.setBuffer(cBuffer, offset: 0, index: 2) - encoder.setBuffer(d1Buffer, offset: 0, index: 3) - encoder.setBuffer(d2Buffer, offset: 0, index: 4) - encoder.setBuffer(d3Buffer, offset: 0, index: 5) - encoder.setBytes(&elementCount, length: MemoryLayout.size, index: 6) - } - - return ( - readBuffer(d1Buffer, count: count, type: Int32.self), - readBuffer(d2Buffer, count: count, type: Int32.self), - readBuffer(d3Buffer, count: count, type: Int32.self) - ) - } - - // MARK: - Batch MED Prediction - - /// Compute MED predictions for a batch of pixels using Metal GPU. - /// - /// This function dispatches compute shaders to the GPU to compute MED - /// predictions for multiple pixels in parallel. Falls back to CPU for - /// small batches. - /// - /// - Parameters: - /// - a: Array of north pixel values - /// - b: Array of west pixel values - /// - c: Array of northwest pixel values - /// - Returns: Array of predicted pixel values - /// - Throws: `MetalAcceleratorError` if GPU operations fail - /// - Precondition: All arrays must have the same length - public func computeMEDPredictionBatch( - a: [Int32], - b: [Int32], - c: [Int32] - ) throws -> [Int32] { - precondition(a.count == b.count && b.count == c.count, "Arrays must have same length") - - let count = a.count - guard count > 0 else { - return [] - } - - // Use CPU fallback for small batches - if count < Self.gpuThreshold { - return computeMEDPredictionBatchCPU(a: a, b: b, c: c) - } - - let aBuffer = try makeReadBuffer(a) - let bBuffer = try makeReadBuffer(b) - let cBuffer = try makeReadBuffer(c) - let predBuffer = try makeWriteBuffer(count: count, type: Int32.self) - - var elementCount = UInt32(count) - try dispatch1D(pipelineState: predictionPipelineState, count: count) { encoder in - encoder.setBuffer(aBuffer, offset: 0, index: 0) - encoder.setBuffer(bBuffer, offset: 0, index: 1) - encoder.setBuffer(cBuffer, offset: 0, index: 2) - encoder.setBuffer(predBuffer, offset: 0, index: 3) - encoder.setBytes(&elementCount, length: MemoryLayout.size, index: 4) - } - - return readBuffer(predBuffer, count: count, type: Int32.self) - } - - // MARK: - Batch Gradient Quantisation - - /// Quantise a batch of gradients to context indices using Metal GPU. - /// - /// Maps each gradient to a context index in the range [-4, 4] using the - /// JPEG-LS threshold parameters T1, T2, T3. Falls back to CPU for small batches. - /// - /// - Parameters: - /// - d1: First gradient array (horizontal) - /// - d2: Second gradient array (vertical) - /// - d3: Third gradient array (diagonal) - /// - t1: Quantisation threshold 1 - /// - t2: Quantisation threshold 2 - /// - t3: Quantisation threshold 3 - /// - Returns: A tuple of three arrays containing the quantised gradients (q1, q2, q3) - /// - Throws: `MetalAcceleratorError` if GPU operations fail - /// - Precondition: All gradient arrays must have the same length - public func quantizeGradientsBatch( - d1: [Int32], d2: [Int32], d3: [Int32], - t1: Int32, t2: Int32, t3: Int32 - ) throws -> (q1: [Int32], q2: [Int32], q3: [Int32]) { - precondition(d1.count == d2.count && d2.count == d3.count, "Arrays must have same length") - - let count = d1.count - guard count > 0 else { - return ([], [], []) - } - - if count < Self.gpuThreshold { - return quantizeGradientsBatchCPU(d1: d1, d2: d2, d3: d3, t1: t1, t2: t2, t3: t3) - } - - let d1Buffer = try makeReadBuffer(d1) - let d2Buffer = try makeReadBuffer(d2) - let d3Buffer = try makeReadBuffer(d3) - let q1Buffer = try makeWriteBuffer(count: count, type: Int32.self) - let q2Buffer = try makeWriteBuffer(count: count, type: Int32.self) - let q3Buffer = try makeWriteBuffer(count: count, type: Int32.self) - - var elementCount = UInt32(count) - var t1v = t1, t2v = t2, t3v = t3 - try dispatch1D(pipelineState: quantizeGradientsPipelineState, count: count) { encoder in - encoder.setBuffer(d1Buffer, offset: 0, index: 0) - encoder.setBuffer(d2Buffer, offset: 0, index: 1) - encoder.setBuffer(d3Buffer, offset: 0, index: 2) - encoder.setBuffer(q1Buffer, offset: 0, index: 3) - encoder.setBuffer(q2Buffer, offset: 0, index: 4) - encoder.setBuffer(q3Buffer, offset: 0, index: 5) - encoder.setBytes(&elementCount, length: MemoryLayout.size, index: 6) - encoder.setBytes(&t1v, length: MemoryLayout.size, index: 7) - encoder.setBytes(&t2v, length: MemoryLayout.size, index: 8) - encoder.setBytes(&t3v, length: MemoryLayout.size, index: 9) - } - - return ( - readBuffer(q1Buffer, count: count, type: Int32.self), - readBuffer(q2Buffer, count: count, type: Int32.self), - readBuffer(q3Buffer, count: count, type: Int32.self) - ) - } - - // MARK: - Batch Colour Space Transformation - - /// Apply a colour space transformation to a batch of RGB pixels using Metal GPU. - /// - /// Dispatches the appropriate HP forward transform (HP1, HP2, or HP3) to the GPU. - /// Falls back to CPU for small batches or when the transform is `.none`. - /// - /// - Parameters: - /// - transform: The colour transformation to apply - /// - r: Red component array - /// - g: Green component array - /// - b: Blue component array - /// - Returns: Transformed (r′, g′, b′) components as `Int32` arrays - /// - Throws: `MetalAcceleratorError` if GPU operations fail - /// - Precondition: All arrays must have the same length - public func applyColourTransformForwardBatch( - transform: JPEGLSColorTransformation, - r: [Int32], g: [Int32], b: [Int32] - ) throws -> (r: [Int32], g: [Int32], b: [Int32]) { - precondition(r.count == g.count && g.count == b.count, "Arrays must have same length") - - let count = r.count - guard count > 0 else { return ([], [], []) } - - if transform == .none { - return (r, g, b) - } - - if count < Self.gpuThreshold { - return applyColourTransformCPU(transform: transform, r: r, g: g, b: b, forward: true) - } - - let pipelineState: MTLComputePipelineState - switch transform { - case .hp1: pipelineState = colourTransformHP1ForwardPipelineState - case .hp2: pipelineState = colourTransformHP2ForwardPipelineState - case .hp3: pipelineState = colourTransformHP3ForwardPipelineState - case .none: return (r, g, b) // unreachable — handled above - } - - return try dispatchColourTransform( - pipelineState: pipelineState, count: count, r: r, g: g, b: b) - } - - /// Apply the inverse colour space transformation to a batch of pixels using Metal GPU. - /// - /// Dispatches the appropriate HP inverse transform (HP1, HP2, or HP3) to the GPU. - /// Falls back to CPU for small batches or when the transform is `.none`. - /// - /// - Parameters: - /// - transform: The colour transformation to invert - /// - r: Transformed red component array (R′) - /// - g: Transformed green component array (G′) - /// - b: Transformed blue component array (B′) - /// - Returns: Recovered (r, g, b) components as `Int32` arrays - /// - Throws: `MetalAcceleratorError` if GPU operations fail - /// - Precondition: All arrays must have the same length - public func applyColourTransformInverseBatch( - transform: JPEGLSColorTransformation, - r: [Int32], g: [Int32], b: [Int32] - ) throws -> (r: [Int32], g: [Int32], b: [Int32]) { - precondition(r.count == g.count && g.count == b.count, "Arrays must have same length") - - let count = r.count - guard count > 0 else { return ([], [], []) } - - if transform == .none { - return (r, g, b) - } - - if count < Self.gpuThreshold { - return applyColourTransformCPU(transform: transform, r: r, g: g, b: b, forward: false) - } - - let pipelineState: MTLComputePipelineState - switch transform { - case .hp1: pipelineState = colourTransformHP1InversePipelineState - case .hp2: pipelineState = colourTransformHP2InversePipelineState - case .hp3: pipelineState = colourTransformHP3InversePipelineState - case .none: return (r, g, b) // unreachable — handled above - } - - return try dispatchColourTransform( - pipelineState: pipelineState, count: count, r: r, g: g, b: b) - } - - /// Dispatch a colour transform shader (forward or inverse) and return the result. - private func dispatchColourTransform( - pipelineState: MTLComputePipelineState, - count: Int, - r: [Int32], g: [Int32], b: [Int32] - ) throws -> (r: [Int32], g: [Int32], b: [Int32]) { - let rInBuffer = try makeReadBuffer(r) - let gInBuffer = try makeReadBuffer(g) - let bInBuffer = try makeReadBuffer(b) - let rOutBuffer = try makeWriteBuffer(count: count, type: Int32.self) - let gOutBuffer = try makeWriteBuffer(count: count, type: Int32.self) - let bOutBuffer = try makeWriteBuffer(count: count, type: Int32.self) - - var elementCount = UInt32(count) - try dispatch1D(pipelineState: pipelineState, count: count) { encoder in - encoder.setBuffer(rInBuffer, offset: 0, index: 0) - encoder.setBuffer(gInBuffer, offset: 0, index: 1) - encoder.setBuffer(bInBuffer, offset: 0, index: 2) - encoder.setBuffer(rOutBuffer, offset: 0, index: 3) - encoder.setBuffer(gOutBuffer, offset: 0, index: 4) - encoder.setBuffer(bOutBuffer, offset: 0, index: 5) - encoder.setBytes(&elementCount, length: MemoryLayout.size, index: 6) - } - - return ( - readBuffer(rOutBuffer, count: count, type: Int32.self), - readBuffer(gOutBuffer, count: count, type: Int32.self), - readBuffer(bOutBuffer, count: count, type: Int32.self) - ) - } - - // MARK: - CPU Fallback Implementations - - /// CPU fallback for gradient computation (used for small batches). - private func computeGradientsBatchCPU( - a: [Int32], - b: [Int32], - c: [Int32] - ) -> (d1: [Int32], d2: [Int32], d3: [Int32]) { - let count = a.count - var d1 = [Int32](repeating: 0, count: count) - var d2 = [Int32](repeating: 0, count: count) - var d3 = [Int32](repeating: 0, count: count) - - for i in 0.. [Int32] { - let count = a.count - var predictions = [Int32](repeating: 0, count: count) - - for i in 0..= maxAB { - predictions[i] = minAB - } else if cv <= minAB { - predictions[i] = maxAB - } else { - predictions[i] = av + bv - cv - } - } - - return predictions - } - - /// CPU fallback for gradient quantisation (used for small batches). - private func quantizeGradientsBatchCPU( - d1: [Int32], d2: [Int32], d3: [Int32], - t1: Int32, t2: Int32, t3: Int32 - ) -> (q1: [Int32], q2: [Int32], q3: [Int32]) { - func quantise(_ d: Int32) -> Int32 { - if d <= -t3 { return -4 } - if d <= -t2 { return -3 } - if d <= -t1 { return -2 } - if d < 0 { return -1 } - if d == 0 { return 0 } - if d < t1 { return 1 } - if d < t2 { return 2 } - if d < t3 { return 3 } - return 4 - } - let count = d1.count - var q1 = [Int32](repeating: 0, count: count) - var q2 = [Int32](repeating: 0, count: count) - var q3 = [Int32](repeating: 0, count: count) - for i in 0.. (r: [Int32], g: [Int32], b: [Int32]) { - let count = r.count - var outR = [Int32](repeating: 0, count: count) - var outG = [Int32](repeating: 0, count: count) - var outB = [Int32](repeating: 0, count: count) - - for i in 0.. (prediction: [Int32], predError: [Int32], q1: [Int32], q2: [Int32], q3: [Int32]) { - func quantiseNear(_ d: Int32) -> Int32 { - if d <= -t3 { return -4 } - if d <= -t2 { return -3 } - if d <= -t1 { return -2 } - if d < -near { return -1 } - if d <= near { return 0 } - if d < t1 { return 1 } - if d < t2 { return 2 } - if d < t3 { return 3 } - return 4 - } - func medPredict(_ av: Int32, _ bv: Int32, _ cv: Int32) -> Int32 { - let minAB = min(av, bv) - let maxAB = max(av, bv) - if cv >= maxAB { return minAB } - if cv <= minAB { return maxAB } - return av + bv - cv - } - let count = a.count - var prediction = [Int32](repeating: 0, count: count) - var predError = [Int32](repeating: 0, count: count) - var q1 = [Int32](repeating: 0, count: count) - var q2 = [Int32](repeating: 0, count: count) - var q3 = [Int32](repeating: 0, count: count) - for i in 0.. [Int32] { - let count = a.count - var reconstructed = [Int32](repeating: 0, count: count) - for i in 0..= maxAB { px = minAB } - else if cv <= minAB { px = maxAB } - else { px = av + bv - cv } - reconstructed[i] = px + errval[i] - } - return reconstructed - } -} - -// MARK: - Encoding / Decoding Pipeline - -extension MetalAccelerator { - - /// Run the combined encoding preprocessing pipeline for a batch of pixels. - /// - /// Performs gradient computation, NEAR-aware gradient quantisation, MED - /// prediction, and prediction-error computation in a single GPU dispatch - /// (or CPU fallback for small batches). This replaces three separate - /// `computeGradientsBatch` + `computeMEDPredictionBatch` + - /// `quantizeGradientsBatch` calls with one, reducing GPU dispatch overhead. - /// - /// - Parameters: - /// - a: North neighbour pixel values. - /// - b: West neighbour pixel values. - /// - c: Northwest neighbour pixel values. - /// - x: Current pixel values. - /// - near: NEAR parameter (0 = lossless, 1–255 = near-lossless). - /// - t1: Quantisation threshold 1. - /// - t2: Quantisation threshold 2. - /// - t3: Quantisation threshold 3. - /// - Returns: A tuple containing MED predictions, raw prediction errors, - /// and quantised gradients (q1, q2, q3). - /// - Throws: `MetalAcceleratorError` if GPU operations fail. - /// - Precondition: All input arrays must have the same length. - public func computeEncodingPipelineBatch( - a: [Int32], b: [Int32], c: [Int32], x: [Int32], - near: Int32 = 0, t1: Int32, t2: Int32, t3: Int32 - ) throws -> (prediction: [Int32], predError: [Int32], - q1: [Int32], q2: [Int32], q3: [Int32]) { - precondition( - a.count == b.count && b.count == c.count && c.count == x.count, - "Arrays must have same length") - let count = a.count - guard count > 0 else { return ([], [], [], [], []) } - - if count < Self.gpuThreshold { - return computeEncodingPipelineCPU( - a: a, b: b, c: c, x: x, near: near, t1: t1, t2: t2, t3: t3) - } - - let aBuffer = try makeReadBuffer(a) - let bBuffer = try makeReadBuffer(b) - let cBuffer = try makeReadBuffer(c) - let xBuffer = try makeReadBuffer(x) - let predBuffer = try makeWriteBuffer(count: count, type: Int32.self) - let predErrorBuffer = try makeWriteBuffer(count: count, type: Int32.self) - let q1Buffer = try makeWriteBuffer(count: count, type: Int32.self) - let q2Buffer = try makeWriteBuffer(count: count, type: Int32.self) - let q3Buffer = try makeWriteBuffer(count: count, type: Int32.self) - - var elementCount = UInt32(count) - var nearVal = near, t1v = t1, t2v = t2, t3v = t3 - try dispatch1D(pipelineState: encodingPipelineState, count: count) { encoder in - encoder.setBuffer(aBuffer, offset: 0, index: 0) - encoder.setBuffer(bBuffer, offset: 0, index: 1) - encoder.setBuffer(cBuffer, offset: 0, index: 2) - encoder.setBuffer(xBuffer, offset: 0, index: 3) - encoder.setBuffer(predBuffer, offset: 0, index: 4) - encoder.setBuffer(predErrorBuffer, offset: 0, index: 5) - encoder.setBuffer(q1Buffer, offset: 0, index: 6) - encoder.setBuffer(q2Buffer, offset: 0, index: 7) - encoder.setBuffer(q3Buffer, offset: 0, index: 8) - encoder.setBytes(&elementCount, length: MemoryLayout.size, index: 9) - encoder.setBytes(&nearVal, length: MemoryLayout.size, index: 10) - encoder.setBytes(&t1v, length: MemoryLayout.size, index: 11) - encoder.setBytes(&t2v, length: MemoryLayout.size, index: 12) - encoder.setBytes(&t3v, length: MemoryLayout.size, index: 13) - } - - return ( - readBuffer(predBuffer, count: count, type: Int32.self), - readBuffer(predErrorBuffer, count: count, type: Int32.self), - readBuffer(q1Buffer, count: count, type: Int32.self), - readBuffer(q2Buffer, count: count, type: Int32.self), - readBuffer(q3Buffer, count: count, type: Int32.self) - ) - } - - /// Run the combined decoding reconstruction pipeline for a batch of pixels. - /// - /// Given the already-entropy-decoded (and de-mapped) prediction errors and - /// the MED neighbours for each pixel, reconstructs the original pixel values - /// in a single GPU dispatch (or CPU fallback for small batches). - /// - /// - Parameters: - /// - a: North neighbour pixel values (already reconstructed). - /// - b: West neighbour pixel values (already reconstructed). - /// - c: Northwest neighbour pixel values (already reconstructed). - /// - errval: Signed prediction errors from the entropy decoder. - /// - Returns: Reconstructed pixel values. - /// - Throws: `MetalAcceleratorError` if GPU operations fail. - /// - Precondition: All input arrays must have the same length. - public func computeDecodingPipelineBatch( - a: [Int32], b: [Int32], c: [Int32], errval: [Int32] - ) throws -> [Int32] { - precondition( - a.count == b.count && b.count == c.count && c.count == errval.count, - "Arrays must have same length") - let count = a.count - guard count > 0 else { return [] } - - if count < Self.gpuThreshold { - return computeDecodingPipelineCPU(a: a, b: b, c: c, errval: errval) - } - - let aBuffer = try makeReadBuffer(a) - let bBuffer = try makeReadBuffer(b) - let cBuffer = try makeReadBuffer(c) - let errvalBuffer = try makeReadBuffer(errval) - let reconstructedBuffer = try makeWriteBuffer(count: count, type: Int32.self) - - var elementCount = UInt32(count) - try dispatch1D(pipelineState: decodingPipelineState, count: count) { encoder in - encoder.setBuffer(aBuffer, offset: 0, index: 0) - encoder.setBuffer(bBuffer, offset: 0, index: 1) - encoder.setBuffer(cBuffer, offset: 0, index: 2) - encoder.setBuffer(errvalBuffer, offset: 0, index: 3) - encoder.setBuffer(reconstructedBuffer, offset: 0, index: 4) - encoder.setBytes(&elementCount, length: MemoryLayout.size, index: 5) - } - - return readBuffer(reconstructedBuffer, count: count, type: Int32.self) - } -} - -// MARK: - Error Types - -/// Errors that can occur during Metal GPU acceleration. -public enum MetalAcceleratorError: Error, CustomStringConvertible { - /// Metal is not available on this device. - case metalNotAvailable - - /// Failed to create Metal command queue. - case commandQueueCreationFailed - - /// Failed to find shader function. - case shaderFunctionNotFound(String) - - /// Failed to load or compile the Metal shader library. - case libraryCreationFailed(String) - - /// Failed to create compute pipeline state. - case pipelineCreationFailed(Error) - - /// Failed to create Metal buffer. - case bufferCreationFailed - - /// Failed to create command buffer. - case commandBufferCreationFailed - - /// Command buffer execution failed. - case commandBufferExecutionFailed - - public var description: String { - switch self { - case .metalNotAvailable: - return "Metal is not available on this device" - case .commandQueueCreationFailed: - return "Failed to create Metal command queue" - case .shaderFunctionNotFound(let name): - return "Shader function '\(name)' not found in Metal library" - case .libraryCreationFailed(let reason): - return "Failed to load Metal shader library: \(reason)" - case .pipelineCreationFailed(let error): - return "Failed to create compute pipeline state: \(error)" - case .bufferCreationFailed: - return "Failed to create Metal buffer" - case .commandBufferCreationFailed: - return "Failed to create command buffer" - case .commandBufferExecutionFailed: - return "Command buffer execution failed" - } - } -} - -#endif // canImport(Metal) diff --git a/Sources/JPEGLS/Platform/Vulkan/VulkanAccelerator.swift b/Sources/JPEGLS/Platform/Vulkan/VulkanAccelerator.swift deleted file mode 100644 index c2315ac..0000000 --- a/Sources/JPEGLS/Platform/Vulkan/VulkanAccelerator.swift +++ /dev/null @@ -1,468 +0,0 @@ -/// Vulkan GPU compute accelerator for JPEG-LS operations. -/// -/// This file implements the Vulkan compute backend for JPEG-LS acceleration. -/// The design mirrors the Metal accelerator API so that application code can -/// swap between Metal (Apple) and Vulkan (Linux/Windows) backends with minimal -/// changes. -/// -/// **Current Status** (Phase 15.2): -/// GPU compute kernels require the Vulkan SDK and SPIR-V shader binaries. -/// Because no Vulkan Swift package is present in the project dependencies, the -/// GPU execution path is gated behind `#if canImport(VulkanSwift)` and is not -/// yet compiled. All public methods unconditionally use the CPU fallback path, -/// which produces bit-exact results identical to those that would be produced -/// by a GPU implementation. -/// -/// **Architecture**: -/// - `VulkanAccelerator.isSupported` returns `true` only when a real GPU device -/// is available via `selectBestVulkanDevice()`. -/// - All batch methods check `count >= gpuThreshold` before dispatching to GPU; -/// small batches always use the CPU path. -/// - The CPU fallback implements the same algorithms as the GPU shaders, -/// guaranteeing bit-exact cross-path results. -/// -/// **SPIR-V Shaders** (planned, not yet compiled): -/// The following compute shader entry points are planned: -/// - `compute_gradients` — gradient computation (D1, D2, D3) -/// - `compute_med_prediction` — MED predictor -/// - `compute_quantize_gradients` — threshold-based gradient quantisation -/// - `compute_colour_transform_hp1_forward` / `_inverse` -/// - `compute_colour_transform_hp2_forward` / `_inverse` -/// - `compute_colour_transform_hp3_forward` / `_inverse` - -import Foundation - -// MARK: - Errors - -/// Errors that can occur during Vulkan GPU acceleration. -public enum VulkanAcceleratorError: Error, CustomStringConvertible, Equatable, Sendable { - /// No Vulkan-capable GPU was found on this system. - case noGPUDeviceAvailable - /// GPU execution failed (e.g. device lost). - case commandExecutionFailed - /// A required SPIR-V shader could not be loaded or compiled. - case shaderLoadFailed(String) - /// Memory allocation on the GPU failed. - case bufferAllocationFailed - /// Input arrays have mismatched lengths. - case inputLengthMismatch - - public var description: String { - switch self { - case .noGPUDeviceAvailable: - return "No Vulkan-capable GPU device is available on this system" - case .commandExecutionFailed: - return "Vulkan command buffer execution failed" - case .shaderLoadFailed(let name): - return "Failed to load SPIR-V shader '\(name)'" - case .bufferAllocationFailed: - return "Failed to allocate Vulkan GPU buffer" - case .inputLengthMismatch: - return "Input arrays must have the same length" - } - } -} - -// MARK: - Accelerator - -/// Vulkan GPU-accelerated implementation for JPEG-LS operations. -/// -/// Provides the same batch operations as `MetalAccelerator` (gradient -/// computation, MED prediction, gradient quantisation, and colour space -/// transformation) using Vulkan compute shaders on Linux/Windows, with -/// automatic CPU fallback when no GPU is available. -/// -/// All methods are safe to call on any platform; they route to the CPU -/// fallback when the Vulkan SDK is not present or no GPU is found. -public final class VulkanAccelerator: @unchecked Sendable { - - public static let platformName = "Vulkan" - - /// Returns `true` when a Vulkan-capable GPU device is found. - /// - /// On platforms without the Vulkan SDK this always returns `false`. - public static var isSupported: Bool { - selectBestVulkanDevice() != nil - } - - /// Minimum batch size (in pixels) before routing work to the GPU. - /// - /// Batches smaller than this threshold are always processed on the CPU - /// to avoid the fixed overhead of GPU command submission. - public static let gpuThreshold = 1024 - - /// The selected Vulkan device (nil when no GPU is available). - public let device: VulkanDevice? - - /// Initialise a Vulkan accelerator. - /// - /// Selects the best available GPU via `selectBestVulkanDevice()`. - /// Initialisation succeeds even when no GPU is found; in that case all - /// operations use the CPU fallback. - public init() { - self.device = selectBestVulkanDevice() - } - - // MARK: - Batch Gradient Computation - - /// Compute gradients for a batch of pixels. - /// - /// For each element i: - /// - D1[i] = b[i] − c[i] (horizontal gradient) - /// - D2[i] = a[i] − c[i] (vertical gradient) - /// - D3[i] = c[i] − a[i] (diagonal gradient) - /// - /// Uses GPU compute when `count >= gpuThreshold` and a device is available; - /// otherwise uses the CPU fallback. - /// - /// - Precondition: All input arrays must have the same length. - public func computeGradientsBatch( - a: [Int32], b: [Int32], c: [Int32] - ) -> (d1: [Int32], d2: [Int32], d3: [Int32]) { - precondition(a.count == b.count && b.count == c.count, "Arrays must have same length") - let count = a.count - guard count > 0 else { return ([], [], []) } - - // GPU path (when Vulkan SDK is available and device is present) - // #if canImport(VulkanSwift) - // if count >= Self.gpuThreshold, device?.isGPUDevice == true { - // return dispatchGradientGPU(a: a, b: b, c: c) - // } - // #endif - - return computeGradientsCPU(a: a, b: b, c: c) - } - - // MARK: - Batch MED Prediction - - /// Compute MED predictions for a batch of pixels. - /// - /// Implements the JPEG-LS MED predictor: - /// - c >= max(a, b) → min(a, b) - /// - c <= min(a, b) → max(a, b) - /// - otherwise → a + b − c - /// - /// - Precondition: All input arrays must have the same length. - public func computeMEDPredictionBatch( - a: [Int32], b: [Int32], c: [Int32] - ) -> [Int32] { - precondition(a.count == b.count && b.count == c.count, "Arrays must have same length") - let count = a.count - guard count > 0 else { return [] } - - // GPU path (when Vulkan SDK is available) - // #if canImport(VulkanSwift) - // if count >= Self.gpuThreshold, device?.isGPUDevice == true { - // return dispatchMEDPredictionGPU(a: a, b: b, c: c) - // } - // #endif - - return computeMEDPredictionCPU(a: a, b: b, c: c) - } - - // MARK: - Batch Gradient Quantisation - - /// Quantise a batch of gradients to context indices in [−4, 4]. - /// - /// Applies the JPEG-LS threshold quantisation mapping using parameters T1, T2, T3. - /// - /// - Precondition: All gradient arrays must have the same length. - public func quantizeGradientsBatch( - d1: [Int32], d2: [Int32], d3: [Int32], - t1: Int32, t2: Int32, t3: Int32 - ) -> (q1: [Int32], q2: [Int32], q3: [Int32]) { - precondition(d1.count == d2.count && d2.count == d3.count, "Arrays must have same length") - let count = d1.count - guard count > 0 else { return ([], [], []) } - - // GPU path (when Vulkan SDK is available) - // #if canImport(VulkanSwift) - // if count >= Self.gpuThreshold, device?.isGPUDevice == true { - // return dispatchQuantizeGPU(d1: d1, d2: d2, d3: d3, t1: t1, t2: t2, t3: t3) - // } - // #endif - - return quantizeGradientsCPU(d1: d1, d2: d2, d3: d3, t1: t1, t2: t2, t3: t3) - } - - // MARK: - Batch Colour Space Transformation - - /// Apply a forward colour space transformation to a batch of RGB pixels. - /// - /// Supports HP1, HP2, HP3, and `.none` (identity). - /// - /// - Precondition: All arrays must have the same length. - public func applyColourTransformForwardBatch( - transform: JPEGLSColorTransformation, - r: [Int32], g: [Int32], b: [Int32] - ) -> (r: [Int32], g: [Int32], b: [Int32]) { - precondition(r.count == g.count && g.count == b.count, "Arrays must have same length") - let count = r.count - guard count > 0 else { return ([], [], []) } - if transform == .none { return (r, g, b) } - - // GPU path (when Vulkan SDK is available) - // #if canImport(VulkanSwift) - // if count >= Self.gpuThreshold, device?.isGPUDevice == true { - // return dispatchColourTransformGPU(transform: transform, r: r, g: g, b: b, forward: true) - // } - // #endif - - return applyColourTransformCPU(transform: transform, r: r, g: g, b: b, forward: true) - } - - /// Apply the inverse colour space transformation to a batch of RGB pixels. - /// - /// Supports HP1, HP2, HP3, and `.none` (identity). - /// - /// - Precondition: All arrays must have the same length. - public func applyColourTransformInverseBatch( - transform: JPEGLSColorTransformation, - r: [Int32], g: [Int32], b: [Int32] - ) -> (r: [Int32], g: [Int32], b: [Int32]) { - precondition(r.count == g.count && g.count == b.count, "Arrays must have same length") - let count = r.count - guard count > 0 else { return ([], [], []) } - if transform == .none { return (r, g, b) } - - // GPU path (when Vulkan SDK is available) - // #if canImport(VulkanSwift) - // if count >= Self.gpuThreshold, device?.isGPUDevice == true { - // return dispatchColourTransformGPU(transform: transform, r: r, g: g, b: b, forward: false) - // } - // #endif - - return applyColourTransformCPU(transform: transform, r: r, g: g, b: b, forward: false) - } - - // MARK: - CPU Fallback Implementations - - private func computeGradientsCPU( - a: [Int32], b: [Int32], c: [Int32] - ) -> (d1: [Int32], d2: [Int32], d3: [Int32]) { - let count = a.count - var d1 = [Int32](repeating: 0, count: count) - var d2 = [Int32](repeating: 0, count: count) - var d3 = [Int32](repeating: 0, count: count) - for i in 0.. [Int32] { - let count = a.count - var predictions = [Int32](repeating: 0, count: count) - for i in 0..= maxAB { - predictions[i] = minAB - } else if cv <= minAB { - predictions[i] = maxAB - } else { - predictions[i] = av + bv - cv - } - } - return predictions - } - - private func quantizeGradientsCPU( - d1: [Int32], d2: [Int32], d3: [Int32], - t1: Int32, t2: Int32, t3: Int32 - ) -> (q1: [Int32], q2: [Int32], q3: [Int32]) { - func quantise(_ d: Int32) -> Int32 { - if d <= -t3 { return -4 } - if d <= -t2 { return -3 } - if d <= -t1 { return -2 } - if d < 0 { return -1 } - if d == 0 { return 0 } - if d < t1 { return 1 } - if d < t2 { return 2 } - if d < t3 { return 3 } - return 4 - } - let count = d1.count - var q1 = [Int32](repeating: 0, count: count) - var q2 = [Int32](repeating: 0, count: count) - var q3 = [Int32](repeating: 0, count: count) - for i in 0.. (r: [Int32], g: [Int32], b: [Int32]) { - let count = r.count - var outR = [Int32](repeating: 0, count: count) - var outG = [Int32](repeating: 0, count: count) - var outB = [Int32](repeating: 0, count: count) - for i in 0.. (prediction: [Int32], predError: [Int32], - q1: [Int32], q2: [Int32], q3: [Int32]) { - guard a.count == b.count && b.count == c.count && c.count == x.count else { - throw VulkanAcceleratorError.inputLengthMismatch - } - let count = a.count - guard count > 0 else { return ([], [], [], [], []) } - - // GPU path (when Vulkan SDK is available and device is present) - // #if canImport(VulkanSwift) - // if count >= Self.gpuThreshold, device?.isGPUDevice == true { - // return try dispatchEncodingPipelineGPU(a:b:c:x:near:t1:t2:t3:) - // } - // #endif - - return computeEncodingPipelineCPU( - a: a, b: b, c: c, x: x, near: near, t1: t1, t2: t2, t3: t3) - } - - /// Run the combined decoding reconstruction pipeline for a batch of pixels. - /// - /// Given the already-entropy-decoded (and de-mapped) prediction errors and - /// the MED neighbours for each pixel, reconstructs the original pixel values - /// in a single pass. Mirrors `MetalAccelerator.computeDecodingPipelineBatch`. - /// - /// - Parameters: - /// - a: West neighbour pixel values (already reconstructed). - /// - b: North neighbour pixel values (already reconstructed). - /// - c: North-west neighbour pixel values (already reconstructed). - /// - errval: Signed prediction errors from the entropy decoder. - /// - Returns: Reconstructed pixel values. - /// - Throws: `VulkanAcceleratorError` if input arrays have mismatched lengths. - /// - Precondition: All input arrays must have the same length. - public func computeDecodingPipelineBatch( - a: [Int32], b: [Int32], c: [Int32], errval: [Int32] - ) throws -> [Int32] { - guard a.count == b.count && b.count == c.count && c.count == errval.count else { - throw VulkanAcceleratorError.inputLengthMismatch - } - let count = a.count - guard count > 0 else { return [] } - - // GPU path (when Vulkan SDK is available and device is present) - // #if canImport(VulkanSwift) - // if count >= Self.gpuThreshold, device?.isGPUDevice == true { - // return try dispatchDecodingPipelineGPU(a:b:c:errval:) - // } - // #endif - - return computeDecodingPipelineCPU(a: a, b: b, c: c, errval: errval) - } - - // MARK: - CPU Implementations for Pipeline - - private func computeEncodingPipelineCPU( - a: [Int32], b: [Int32], c: [Int32], x: [Int32], - near: Int32, t1: Int32, t2: Int32, t3: Int32 - ) -> (prediction: [Int32], predError: [Int32], q1: [Int32], q2: [Int32], q3: [Int32]) { - func quantiseNear(_ d: Int32) -> Int32 { - if d <= -t3 { return -4 } - if d <= -t2 { return -3 } - if d <= -t1 { return -2 } - if d < -near { return -1 } - if d <= near { return 0 } - if d < t1 { return 1 } - if d < t2 { return 2 } - if d < t3 { return 3 } - return 4 - } - func medPredict(_ av: Int32, _ bv: Int32, _ cv: Int32) -> Int32 { - let minAB = min(av, bv) - let maxAB = max(av, bv) - if cv >= maxAB { return minAB } - if cv <= minAB { return maxAB } - return av + bv - cv - } - let count = a.count - var prediction = [Int32](repeating: 0, count: count) - var predError = [Int32](repeating: 0, count: count) - var q1 = [Int32](repeating: 0, count: count) - var q2 = [Int32](repeating: 0, count: count) - var q3 = [Int32](repeating: 0, count: count) - for i in 0.. [Int32] { - let count = a.count - var reconstructed = [Int32](repeating: 0, count: count) - for i in 0..= maxAB { px = minAB } - else if cv <= minAB { px = maxAB } - else { px = av + bv - cv } - reconstructed[i] = px + errval[i] - } - return reconstructed - } -} diff --git a/Sources/JPEGLS/Platform/Vulkan/VulkanCommandBuffer.swift b/Sources/JPEGLS/Platform/Vulkan/VulkanCommandBuffer.swift deleted file mode 100644 index f9c2912..0000000 --- a/Sources/JPEGLS/Platform/Vulkan/VulkanCommandBuffer.swift +++ /dev/null @@ -1,206 +0,0 @@ -/// Vulkan command buffer recording and submission architecture. -/// -/// This file provides CPU-side types that represent Vulkan command recording -/// and submission. When the Vulkan SDK (`VulkanSwift`) is integrated, these -/// types will delegate to the actual Vulkan API calls: -/// `vkBeginCommandBuffer`, `vkCmdBindPipeline`, `vkCmdDispatch`, -/// `vkEndCommandBuffer`, and `vkQueueSubmit`. -/// -/// In the current CPU-fallback implementation the recorded commands are -/// stored in a Swift array. This lets the full dispatch path be tested and -/// the command-recording API be stabilised before the GPU execution path -/// is connected to real Vulkan calls. - -import Foundation - -// MARK: - Compute Command - -/// A single recorded Vulkan compute command. -/// -/// Each `VulkanComputeCommand` wraps one of the operations a compute -/// command buffer can record: pipeline binding, buffer binding, -/// push-constants upload, or workgroup dispatch. -public struct VulkanComputeCommand: Sendable { - - /// The kind of operation this command represents. - public enum Operation: Sendable { - /// Bind a named compute pipeline (shader entry point). - case bindPipeline(name: String) - /// Bind a `VulkanBuffer` to a specific descriptor-set binding slot. - case bindBuffer(buffer: VulkanBuffer, binding: UInt32) - /// Upload raw bytes to the shader's push-constant range. - case pushConstants(data: [UInt8]) - /// Dispatch a compute workgroup grid of `(x, y, z)` groups. - case dispatch(x: UInt32, y: UInt32, z: UInt32) - } - - /// The operation to execute when this command is submitted. - public let operation: Operation -} - -// MARK: - Command Buffer - -/// Records a sequence of Vulkan compute commands for later submission. -/// -/// In a real Vulkan integration this type wraps a `VkCommandBuffer` handle. -/// Here it stores recorded commands in an array so that: -/// - The full encode/decode command-recording path can be exercised in tests. -/// - The API is defined before a GPU device is available in the project. -/// - CPU-fallback execution can iterate `recordedCommands` to perform the -/// equivalent work on the CPU. -/// -/// - Important: `VulkanCommandBuffer` is marked `@unchecked Sendable` to -/// match the Vulkan command-buffer ownership model: a command buffer is -/// owned by one thread at a time (recording, then submitting). Concurrent -/// access from multiple tasks or threads is not supported and requires -/// external synchronisation. -/// -/// ## Recording Pattern -/// -/// ```swift -/// let pool = VulkanCommandPool() -/// let cmdBuf = pool.allocate() -/// cmdBuf.begin() -/// cmdBuf.bindPipeline(name: "compute_gradients") -/// cmdBuf.bindBuffer(inputBuf, binding: 0) -/// cmdBuf.bindBuffer(outputBuf, binding: 1) -/// cmdBuf.dispatch(x: UInt32((pixelCount + 63) / 64)) -/// cmdBuf.end() -/// // … submit cmdBuf to a VulkanAccelerator or execute CPU fallback … -/// ``` -public final class VulkanCommandBuffer: @unchecked Sendable { - - private var commands: [VulkanComputeCommand] = [] - - /// Whether the command buffer is currently open for recording. - /// - /// `begin()` sets this to `true`; `end()` sets it to `false`. - /// Recording methods (`bindPipeline`, `bindBuffer`, `dispatch`, etc.) - /// are silently ignored when `isRecording` is `false`. - public private(set) var isRecording = false - - /// All commands recorded between the last `begin()` and the most - /// recent `end()` call (or recorded so far if `end()` has not been - /// called yet). - public var recordedCommands: [VulkanComputeCommand] { commands } - - /// The number of commands recorded in this buffer. - public var commandCount: Int { commands.count } - - // MARK: Recording API - - /// Open the command buffer for recording, discarding any prior commands. - /// - /// Must be called before `bindPipeline`, `bindBuffer`, `pushConstants`, - /// or `dispatch`. - public func begin() { - commands.removeAll() - isRecording = true - } - - /// Record a pipeline bind command. - /// - /// - Parameter name: Logical name of the compute pipeline (corresponds - /// to the shader function name, e.g. `"compute_gradients"`). - public func bindPipeline(name: String) { - guard isRecording else { return } - commands.append(VulkanComputeCommand(operation: .bindPipeline(name: name))) - } - - /// Record a buffer bind command. - /// - /// - Parameters: - /// - buffer: The `VulkanBuffer` to bind. - /// - binding: Descriptor-set binding index as declared in the GLSL/HLSL shader. - public func bindBuffer(_ buffer: VulkanBuffer, binding: UInt32) { - guard isRecording else { return } - commands.append(VulkanComputeCommand( - operation: .bindBuffer(buffer: buffer, binding: binding))) - } - - /// Record a push-constants upload. - /// - /// Push constants allow small amounts of data (e.g. scalar parameters - /// like `NEAR` or threshold values) to be uploaded to the shader without - /// creating a separate uniform buffer. - /// - /// - Parameter data: Raw bytes to push to the shader's push-constant range. - public func pushConstants(data: [UInt8]) { - guard isRecording else { return } - commands.append(VulkanComputeCommand(operation: .pushConstants(data: data))) - } - - /// Record a compute dispatch. - /// - /// Launches `x * y * z` workgroups. Each workgroup processes a fixed - /// number of elements as declared by the `local_size_*` qualifier in - /// the GLSL shader. - /// - /// - Parameters: - /// - x: Number of workgroups in the X dimension. - /// - y: Number of workgroups in the Y dimension (default 1). - /// - z: Number of workgroups in the Z dimension (default 1). - public func dispatch(x: UInt32, y: UInt32 = 1, z: UInt32 = 1) { - guard isRecording else { return } - commands.append(VulkanComputeCommand(operation: .dispatch(x: x, y: y, z: z))) - } - - /// Close the command buffer, making it ready for submission. - /// - /// After `end()`, `isRecording` is `false` and `recordedCommands` - /// contains the full command sequence for this recording session. - public func end() { - isRecording = false - } -} - -// MARK: - Command Pool - -/// Allocates and manages `VulkanCommandBuffer` instances. -/// -/// In a real Vulkan integration this type wraps a `VkCommandPool` handle. -/// The pool provides efficient allocation of command buffers from a -/// pre-allocated block of GPU memory, and supports bulk reset to reuse -/// that block for a new frame or dispatch cycle. -/// -/// - Important: `VulkanCommandPool` is marked `@unchecked Sendable` to -/// match the Vulkan pool ownership model. Concurrent calls to `allocate()` -/// or `reset()` are not thread-safe; use the pool from a single task or -/// thread, or provide external synchronisation. -/// -/// ## Example -/// -/// ```swift -/// let pool = VulkanCommandPool() -/// let cmdBuf = pool.allocate() -/// cmdBuf.begin() -/// // … record commands … -/// cmdBuf.end() -/// pool.reset() // reuse the pool for the next frame -/// ``` -public final class VulkanCommandPool: @unchecked Sendable { - - private var buffers: [VulkanCommandBuffer] = [] - - /// Allocate a new, empty command buffer from this pool. - /// - /// - Returns: A fresh `VulkanCommandBuffer` that is not yet recording. - /// Call `begin()` before recording any commands into it. - public func allocate() -> VulkanCommandBuffer { - let buffer = VulkanCommandBuffer() - buffers.append(buffer) - return buffer - } - - /// Reset all command buffers allocated from this pool. - /// - /// Discards all previously allocated command buffers. After `reset()`, - /// use `allocate()` to obtain fresh command buffers. In a real Vulkan - /// implementation this corresponds to `vkResetCommandPool`. - public func reset() { - buffers.removeAll() - } - - /// The number of command buffers currently allocated from this pool. - public var allocatedCount: Int { buffers.count } -} diff --git a/Sources/JPEGLS/Platform/Vulkan/VulkanDevice.swift b/Sources/JPEGLS/Platform/Vulkan/VulkanDevice.swift deleted file mode 100644 index 7c4627e..0000000 --- a/Sources/JPEGLS/Platform/Vulkan/VulkanDevice.swift +++ /dev/null @@ -1,108 +0,0 @@ -/// Vulkan device information and capability detection for JPEG-LS GPU compute. -/// -/// This file provides device discovery and capability reporting for the Vulkan -/// compute backend. When the Vulkan SDK is not present, the implementation -/// reports no GPU devices and routes all work through the CPU fallback path. -/// -/// **Platform Support**: -/// - Linux and Windows with Vulkan 1.1+ SDK: GPU compute available -/// - macOS / iOS / tvOS / watchOS: Not supported (use Metal instead) -/// - All other platforms without Vulkan SDK: CPU fallback only -/// -/// **Conditional Compilation**: -/// Actual Vulkan device enumeration is gated behind `#if canImport(VulkanSwift)`. -/// The CPU-always-available type `VulkanDevice` is unconditionally compiled -/// to allow the rest of the library to reference it on all platforms. - -import Foundation - -// MARK: - Vulkan Device Type - -/// Represents a single Vulkan-capable GPU device. -/// -/// On platforms where the Vulkan SDK is not available, this type is still -/// usable but `isGPUDevice` will always return `false`, and all GPU -/// operations will be handled by the CPU fallback path. -public struct VulkanDevice: Sendable, CustomStringConvertible { - - /// The device name reported by Vulkan (or a CPU fallback description). - public let name: String - - /// The Vulkan API version supported by this device (0 when no Vulkan SDK). - public let apiVersion: UInt32 - - /// The kind of device. - public let deviceType: VulkanDeviceType - - /// Returns `true` if this entry represents a real GPU device (not a CPU fallback stub). - public var isGPUDevice: Bool { - deviceType == .discreteGPU || deviceType == .integratedGPU || deviceType == .virtualGPU - } - - public var description: String { - "VulkanDevice(name: \"\(name)\", type: \(deviceType), apiVersion: \(apiVersion))" - } -} - -// MARK: - Device Type - -/// The physical device type as reported by Vulkan. -public enum VulkanDeviceType: Sendable, CustomStringConvertible { - /// A discrete (dedicated) GPU. - case discreteGPU - /// An integrated GPU sharing system memory. - case integratedGPU - /// A virtual GPU inside a virtualisation environment. - case virtualGPU - /// A CPU-based Vulkan implementation (e.g. lavapipe/SwiftShader). - case cpu - /// Device type could not be determined. - case other - - public var description: String { - switch self { - case .discreteGPU: return "Discrete GPU" - case .integratedGPU: return "Integrated GPU" - case .virtualGPU: return "Virtual GPU" - case .cpu: return "CPU" - case .other: return "Other" - } - } -} - -// MARK: - Device Enumeration - -/// Enumerate Vulkan-capable physical devices on the current system. -/// -/// Returns an empty array on platforms without the Vulkan SDK, allowing callers -/// to use the CPU fallback path transparently. -/// -/// - Returns: Array of available `VulkanDevice` instances (empty when no Vulkan SDK). -public func enumerateVulkanDevices() -> [VulkanDevice] { - // When the Vulkan SDK / VulkanSwift package is available, real device - // enumeration would go here. For now, return an empty array so that - // VulkanAccelerator always uses its CPU fallback path. - // - // #if canImport(VulkanSwift) - // import VulkanSwift - // … vkEnumeratePhysicalDevices logic … - // #endif - return [] -} - -/// Select the best available Vulkan device for compute work. -/// -/// Prefers discrete GPUs over integrated GPUs over virtual GPUs over CPU devices. -/// Returns `nil` when no suitable device is available (no Vulkan SDK or no GPU). -/// -/// - Returns: The best `VulkanDevice`, or `nil` if none is available. -public func selectBestVulkanDevice() -> VulkanDevice? { - let devices = enumerateVulkanDevices() - let priority: [VulkanDeviceType] = [.discreteGPU, .integratedGPU, .virtualGPU, .cpu] - for preferred in priority { - if let device = devices.first(where: { $0.deviceType == preferred }) { - return device - } - } - return nil -} diff --git a/Sources/JPEGLS/Platform/Vulkan/VulkanMemory.swift b/Sources/JPEGLS/Platform/Vulkan/VulkanMemory.swift deleted file mode 100644 index 6599933..0000000 --- a/Sources/JPEGLS/Platform/Vulkan/VulkanMemory.swift +++ /dev/null @@ -1,194 +0,0 @@ -/// Vulkan memory management types for JPEG-LS GPU compute. -/// -/// This file provides CPU-side buffer and memory-pool abstractions that -/// mirror the Vulkan memory model. When the Vulkan SDK (`VulkanSwift`) is -/// integrated, these types will wrap `VkBuffer` + `VkDeviceMemory` objects. -/// For now they provide a CPU-backed implementation that ensures the API -/// and data-layout used by `VulkanAccelerator` are correct for future GPU -/// integration and that all data paths can be exercised in tests today. - -import Foundation - -// MARK: - Buffer Usage Flags - -/// Flags indicating the intended usage of a Vulkan buffer. -/// -/// Mirror the `VkBufferUsageFlags` bitmask from the Vulkan specification. -/// Multiple flags may be combined to describe a buffer used in more than -/// one way (for example, a buffer that acts as both a transfer source and -/// a compute storage buffer). -public struct VulkanBufferUsage: OptionSet, Sendable { - public let rawValue: UInt32 - public init(rawValue: UInt32) { self.rawValue = rawValue } - - /// Buffer used as a storage buffer in compute shaders (`VK_BUFFER_USAGE_STORAGE_BUFFER_BIT`). - public static let storageBuffer = VulkanBufferUsage(rawValue: 1 << 0) - - /// Buffer used as a uniform buffer (constant data) in shaders (`VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT`). - public static let uniformBuffer = VulkanBufferUsage(rawValue: 1 << 1) - - /// Buffer acts as the source of a transfer operation (`VK_BUFFER_USAGE_TRANSFER_SRC_BIT`). - public static let transferSrc = VulkanBufferUsage(rawValue: 1 << 2) - - /// Buffer acts as the destination of a transfer operation (`VK_BUFFER_USAGE_TRANSFER_DST_BIT`). - public static let transferDst = VulkanBufferUsage(rawValue: 1 << 3) -} - -// MARK: - VulkanBuffer - -/// A CPU-backed representation of a Vulkan buffer allocation. -/// -/// On platforms with the Vulkan SDK this type would wrap a `VkBuffer` handle -/// together with its backing `VkDeviceMemory`. In the current CPU-fallback -/// implementation it manages a contiguous `[UInt8]` storage block, ensuring -/// the host-visible data layout matches what a GPU buffer would hold. -/// -/// Use `VulkanMemoryPool.allocate(size:usage:)` to create buffers in batch; -/// use `VulkanBuffer.init(size:usage:)` to create standalone buffers. -/// -/// - Important: `VulkanBuffer` is marked `@unchecked Sendable` to align with -/// the ownership model used by GPU command-buffer APIs: a buffer is owned -/// by one producer at a time (typically the CPU during host–device transfer, -/// or the GPU during shader execution). Callers must not access a buffer -/// concurrently from multiple Swift tasks without external synchronisation. -public final class VulkanBuffer: @unchecked Sendable { - - /// The size of this buffer in bytes. - public let size: Int - - /// The intended usage flags for this buffer. - public let usage: VulkanBufferUsage - - /// CPU-side storage backing this buffer. - private var storage: [UInt8] - - /// Initialise a new buffer of the given size. - /// - /// - Parameters: - /// - size: Buffer size in bytes. Must be greater than zero. - /// - usage: Intended Vulkan buffer usage flags. - /// - Throws: `VulkanAcceleratorError.bufferAllocationFailed` if `size` is zero. - public init(size: Int, usage: VulkanBufferUsage) throws { - guard size > 0 else { throw VulkanAcceleratorError.bufferAllocationFailed } - self.size = size - self.usage = usage - self.storage = [UInt8](repeating: 0, count: size) - } - - // MARK: Host-Accessible Data Transfer - - /// Write a typed array into the buffer starting at offset 0. - /// - /// This mirrors a host-visible mapped Vulkan buffer write (or a staging - /// buffer `memcpy`) and is used to transfer data from the CPU to the - /// (currently simulated) GPU. - /// - /// - Parameter array: Elements to write. Their packed byte representation - /// must fit within `size` bytes. - /// - Precondition: `array.count * MemoryLayout.stride <= size`. - public func write(_ array: [T]) { - let byteCount = array.count * MemoryLayout.stride - precondition(byteCount <= size, "Array does not fit in buffer") - array.withUnsafeBytes { src in - storage.withUnsafeMutableBytes { dst in - dst.copyMemory(from: src) - } - } - } - - /// Read typed elements from the buffer starting at offset 0. - /// - /// This mirrors a host-visible mapped Vulkan buffer read (or a readback - /// staging transfer) and is used to transfer results from the (currently - /// simulated) GPU back to the CPU. - /// - /// - Parameters: - /// - count: Number of elements to read. - /// - type: Element type. - /// - Returns: Array of `count` elements read from the start of the buffer. - /// - Precondition: `count * MemoryLayout.stride <= size`. - public func read(count: Int, type: T.Type) -> [T] { - let byteCount = count * MemoryLayout.stride - precondition(byteCount <= size, "Read range exceeds buffer size") - return storage.withUnsafeBytes { ptr in - Array(ptr.bindMemory(to: T.self).prefix(count)) - } - } -} - -// MARK: - VulkanMemoryPool - -/// A pool-based allocator for `VulkanBuffer` instances. -/// -/// `VulkanMemoryPool` manages a set of `VulkanBuffer` allocations backed by -/// a pre-declared capacity, mirroring the pattern used in Vulkan applications -/// where a large `VkDeviceMemory` block is sub-allocated into individual -/// buffers. This avoids the overhead of a separate `vkAllocateMemory` call -/// for every buffer. -/// -/// Call `allocate(size:usage:)` to obtain buffers from the pool, and -/// `reset()` to release all allocations and reclaim the capacity. -/// -/// - Important: `VulkanMemoryPool` is marked `@unchecked Sendable` to match -/// the single-owner GPU memory model. The pool must only be accessed from -/// one thread or Swift task at a time; concurrent calls to `allocate()` or -/// `reset()` are not thread-safe and require external synchronisation if -/// used across concurrency boundaries. -/// -/// ## Example -/// ```swift -/// let pool = VulkanMemoryPool(maxPoolSize: 64 * 1024 * 1024) // 64 MB -/// let inputBuf = try pool.allocate(size: pixels * 4, usage: .transferSrc) -/// let outputBuf = try pool.allocate(size: pixels * 4, usage: .storageBuffer) -/// // … fill inputBuf, dispatch shader, read outputBuf … -/// pool.reset() // free all allocations for reuse -/// ``` -public final class VulkanMemoryPool: @unchecked Sendable { - - /// Maximum total bytes available for allocation from this pool. - public let maxPoolSize: Int - - /// Total bytes currently allocated from this pool. - public private(set) var totalAllocated: Int = 0 - - private var buffers: [VulkanBuffer] = [] - - /// Initialise a memory pool with the given capacity. - /// - /// - Parameter maxPoolSize: Maximum total bytes available for allocation. - public init(maxPoolSize: Int) { - self.maxPoolSize = maxPoolSize - } - - /// Allocate a new buffer from the pool. - /// - /// - Parameters: - /// - size: Required buffer size in bytes. - /// - usage: Intended Vulkan buffer usage flags. - /// - Returns: A new `VulkanBuffer` of the requested size. - /// - Throws: `VulkanAcceleratorError.bufferAllocationFailed` when the - /// remaining pool capacity is insufficient. - public func allocate(size: Int, usage: VulkanBufferUsage) throws -> VulkanBuffer { - guard totalAllocated + size <= maxPoolSize else { - throw VulkanAcceleratorError.bufferAllocationFailed - } - let buffer = try VulkanBuffer(size: size, usage: usage) - buffers.append(buffer) - totalAllocated += size - return buffer - } - - /// Release all allocations and reset the pool to empty. - /// - /// After calling `reset()`, `totalAllocated` returns to zero and all - /// previously returned `VulkanBuffer` objects must be discarded. In a - /// real Vulkan implementation this corresponds to `vkResetDescriptorPool` - /// or re-using the backing `VkDeviceMemory` block. - public func reset() { - buffers.removeAll() - totalAllocated = 0 - } - - /// The number of buffers currently alive in this pool. - public var bufferCount: Int { buffers.count } -} diff --git a/Sources/JPEGLS/Platform/x86_64/IntelMemoryOptimizer.swift b/Sources/JPEGLS/Platform/x86_64/IntelMemoryOptimizer.swift deleted file mode 100644 index 1e5c924..0000000 --- a/Sources/JPEGLS/Platform/x86_64/IntelMemoryOptimizer.swift +++ /dev/null @@ -1,269 +0,0 @@ -/// Intel x86-64 memory architecture optimisation for JPEG-LS. -/// -/// Provides cache-hierarchy-aware data layouts, buffer pooling, -/// memory-mapped I/O helpers, prefetch hints, and L1/L2 tile-size -/// tuning tailored for Intel Core and Xeon processors. -/// -/// **Note**: This file is conditionally compiled only on x86-64 -/// architectures so that the ARM64 code path remains cleanly separable. - -#if arch(x86_64) - -import Foundation - -// MARK: - Intel Cache Parameters - -/// Cache and memory-architecture parameters for Intel x86-64 processors. -/// -/// Values are conservative estimates suitable for a broad range of Intel -/// Core (i3/i5/i7/i9) and Xeon processors. The tile-tuning helpers -/// below use these constants to compute optimal tile sizes. -public enum IntelCacheParameters { - /// L1 data-cache size per core (bytes) — typical Intel Core value. - public static let l1DataCacheSize: Int = 32 * 1024 // 32 KiB - - /// L2 cache size per core (bytes) — typical Intel Core value. - public static let l2CacheSize: Int = 256 * 1024 // 256 KiB - - /// L3 / Last-Level Cache size (bytes) — conservative estimate. - public static let l3CacheSize: Int = 8 * 1024 * 1024 // 8 MiB - - /// CPU cache-line size in bytes (x86-64 standard). - public static let cacheLineSize: Int = 64 - - /// Optimal JPEG-LS context array alignment for cache-line boundaries. - public static let contextArrayAlignment: Int = 64 - - /// Maximum single-strip tile height (rows) recommended for L1 fit - /// with 3-component 8-bit data on Intel processors. - public static let recommendedStripHeight: Int = 8 -} - -// MARK: - Tile Size Tuning - -/// Compute the optimal tile dimensions for JPEG-LS encoding on Intel x86-64. -/// -/// Selects tile width and height so that the working set for one tile -/// (three rows of context neighbours + one output row) fits within the -/// L1 data cache of a typical Intel Core processor. -/// -/// ```swift -/// let (tw, th) = intelOptimalTileSize(imageWidth: 3840, imageHeight: 2160, bytesPerSample: 2) -/// // tw ≈ 512, th ≈ 4 (fits within 32 KiB L1 cache) -/// ``` -/// -/// - Parameters: -/// - imageWidth: Full image width in pixels -/// - imageHeight: Full image height in pixels -/// - bytesPerSample: Bytes per sample (1 for 8-bit, 2 for 16-bit) -/// - componentCount: Number of image components (1 = greyscale, 3 = RGB) -/// - Returns: A tuple `(tileWidth, tileHeight)` optimised for Intel L1 cache -public func intelOptimalTileSize( - imageWidth: Int, - imageHeight: Int, - bytesPerSample: Int = 1, - componentCount: Int = 1 -) -> (tileWidth: Int, tileHeight: Int) { - // Budget: L1 cache; reserve 25% for stack/code, use 75% for pixel data - let budget = (IntelCacheParameters.l1DataCacheSize * 3) / 4 - - // Working set per row = width * bytesPerSample * componentCount - // We need ~4 rows in cache at once (current row + 3 context rows) - let rowSize = imageWidth * bytesPerSample * componentCount - let rowsInBudget = max(1, budget / max(1, rowSize)) - let tileHeight = min(imageHeight, max(1, rowsInBudget / 4)) - - // Tile width: round up to the nearest cache-line boundary in samples, - // then clamp to the actual image width. - let samplesPerCacheLine = IntelCacheParameters.cacheLineSize / max(1, bytesPerSample) - let alignedWidth = ((imageWidth + samplesPerCacheLine - 1) / samplesPerCacheLine) * samplesPerCacheLine - let tileWidth = min(imageWidth, alignedWidth) - - return (tileWidth, tileHeight) -} - -// MARK: - Cache-Line Aligned Buffer Allocation - -/// Allocate a cache-line–aligned integer buffer for JPEG-LS context arrays. -/// -/// Context arrays (A, B, C, N) are accessed with stride equal to one entry -/// per quantised context (up to 365 contexts). Alignment to cache-line -/// boundaries prevents false-sharing on multi-core Intel processors. -/// -/// - Parameter count: Number of elements to allocate -/// - Returns: A zero-initialised `[Int]` of the requested size -/// -/// - Note: Swift arrays are heap-allocated and typically 16-byte aligned. -/// This helper pads `count` to the next multiple of the cache-line stride -/// in `Int` units so that adjacent arrays in a struct avoid cache aliasing. -public func intelAllocateCacheAlignedContextArray(count: Int) -> [Int] { - let alignment = IntelCacheParameters.cacheLineSize / MemoryLayout.stride - let alignedCount = ((count + alignment - 1) / alignment) * alignment - return [Int](repeating: 0, count: alignedCount) -} - -// MARK: - Memory-Mapped I/O - -/// Open a file for memory-mapped read-only access on Intel x86-64. -/// -/// Memory-mapped I/O avoids copying file data into user-space buffers; -/// the OS kernel maps file pages directly into the process address space -/// using the underlying `mmap(2)` system call. -/// -/// ```swift -/// let data = try intelMemoryMappedData(at: url) -/// // Use `data` as a normal Data value — pages are faulted in on demand -/// ``` -/// -/// - Parameter url: URL of the file to map -/// - Returns: A `Data` value backed by a memory mapping of the file -/// - Throws: An error if the file cannot be opened or mapped -public func intelMemoryMappedData(at url: URL) throws -> Data { - return try Data(contentsOf: url, options: .mappedIfSafe) -} - -/// Write data to a file on Intel x86-64. -/// -/// Writes the provided `Data` to `url` atomically to avoid partial writes. -/// -/// - Parameters: -/// - data: Data to write -/// - url: Destination file URL -/// - Throws: An error if the write fails -public func intelWriteMemoryMapped(_ data: Data, to url: URL) throws { - try data.write(to: url, options: .atomic) -} - -// MARK: - Buffer Pool - -/// A buffer pool optimised for Intel x86-64 JPEG-LS processing. -/// -/// This pool manages a set of pre-allocated reusable `Data` buffers of -/// fixed sizes, reducing allocation pressure during encode/decode loops. -/// -/// ```swift -/// let pool = IntelBufferPool(bufferSize: 256 * 1024, poolCapacity: 4) -/// let buf = pool.acquire() -/// // ... use buf for one encode tile ... -/// pool.release(buf) -/// ``` -public final class IntelBufferPool: @unchecked Sendable { - private let bufferSize: Int - private let poolCapacity: Int - private var available: [Data] = [] - private let lock = NSLock() - - /// Create a new buffer pool. - /// - /// - Parameters: - /// - bufferSize: Size of each buffer in bytes - /// - poolCapacity: Maximum number of buffers held in the pool - public init(bufferSize: Int, poolCapacity: Int = 4) { - self.bufferSize = bufferSize - self.poolCapacity = poolCapacity - } - - /// Acquire a buffer from the pool, allocating a new one if empty. - /// - /// - Returns: A `Data` value of `bufferSize` bytes (zeroed on first allocation; - /// contents undefined on subsequent reuse) - public func acquire() -> Data { - lock.lock() - defer { lock.unlock() } - if !available.isEmpty { - return available.removeLast() - } - return Data(count: bufferSize) - } - - /// Return a buffer to the pool for reuse. - /// - /// Buffers exceeding `poolCapacity` are silently discarded to bound - /// peak memory usage. - /// - /// - Parameter buffer: Previously acquired buffer - public func release(_ buffer: Data) { - lock.lock() - defer { lock.unlock() } - if available.count < poolCapacity { - available.append(buffer) - } - } - - /// Pre-warm the pool by allocating `poolCapacity` buffers eagerly. - /// - /// Call this once at startup to avoid allocation latency on the first - /// batch of encode/decode tiles. - public func prewarm() { - lock.lock() - defer { lock.unlock() } - while available.count < poolCapacity { - available.append(Data(count: bufferSize)) - } - } - - /// The number of buffers currently available in the pool. - public var availableCount: Int { - lock.lock() - defer { lock.unlock() } - return available.count - } -} - -// MARK: - Prefetch Hints - -/// Issue a software prefetch hint for the given memory region. -/// -/// On x86-64, the compiler may lower this to PREFETCHT0 instructions -/// when inlining. The function hint guides the hardware prefetcher for -/// predictable sequential access patterns during row-by-row JPEG-LS -/// encoding/decoding. -/// -/// - Parameters: -/// - array: Array whose data should be prefetched -/// - startIndex: First element index to prefetch -/// - count: Number of elements to prefetch (one cache line covers 8 Int values on x86-64) -@inline(__always) -public func intelPrefetchContextArray(_ array: [Int], startIndex: Int, count: Int) { - let end = min(startIndex + count, array.count) - guard startIndex < end else { return } - - // Touch the first element of every cache line in the range. - // On x86-64 one cache line holds 64 / MemoryLayout.stride = 8 Int values. - let stride = IntelCacheParameters.cacheLineSize / MemoryLayout.stride - var i = startIndex - while i < end { - _ = array[i] - i += stride - } -} - -// MARK: - Hardware-Specific Tuning Parameters - -/// Recommended tuning parameters for JPEG-LS on Intel x86-64. -/// -/// These constants document the rationale behind the chosen defaults and -/// serve as a single source of truth for any future benchmark-driven tuning. -public enum IntelTuningParameters { - /// Recommended RESET threshold for context adaptation on Intel x86-64. - /// - /// Intel processors have smaller register files than ARM64, so keeping - /// RESET moderate balances context accuracy against cache pressure. - public static let recommendedReset: Int = 64 - - /// Strip height (rows) for tiled encoding on Intel x86-64. - /// - /// Each strip processes `stripHeight` rows of the image at a time. - /// This value is chosen so that the three-row context window fits - /// comfortably in the Intel L1 data cache (32 KiB). - public static let stripHeight: Int = IntelCacheParameters.recommendedStripHeight - - /// Context array pre-allocation count (rounded to cache-line boundary). - /// - /// JPEG-LS uses 365 regular contexts + 2 run-interruption contexts. - /// Pre-allocating 384 entries (6 × 64) aligns the end of each context - /// array to a cache-line boundary on x86-64. - public static let contextArrayCount: Int = 384 -} - -#endif // arch(x86_64) diff --git a/Sources/JPEGLS/Platform/x86_64/X86_64Accelerator.swift b/Sources/JPEGLS/Platform/x86_64/X86_64Accelerator.swift deleted file mode 100644 index 19f2da0..0000000 --- a/Sources/JPEGLS/Platform/x86_64/X86_64Accelerator.swift +++ /dev/null @@ -1,320 +0,0 @@ -/// x86-64-specific acceleration using SSE/AVX SIMD instructions. -/// -/// This implementation provides optimised routines for Intel processors using -/// Swift's SIMD types which compile to efficient SSE/AVX instructions. -/// Phase 14.1 adds Golomb-Rice parameter computation (BSR-based), -/// SIMD8 run-length detection, and SIMD8 byte stuffing scanning to -/// bring the x86-64 accelerator to parity with the ARM64 accelerator. -/// -/// **Important**: This module is designed for future removal as part of -/// the project's focus on Apple Silicon. All x86-64 code is isolated in -/// this file and conditionally compiled to facilitate clean removal. -/// -/// **Note**: This file is conditionally compiled only on x86-64 architectures. - -#if arch(x86_64) - -import Foundation - -/// x86-64 SIMD-accelerated implementation of platform acceleration. -/// -/// Provides hardware-accelerated gradient computation, prediction, -/// quantisation, Golomb-Rice parameter estimation, run-length detection, -/// and byte stuffing scanning optimised for Intel x86-64 processors. -/// The implementation uses Swift's SIMD types which compile to native -/// SSE/AVX instructions. -/// -/// **Removal Notice**: This implementation is planned for deprecation -/// when ARM64 becomes the sole supported platform. -public struct X86_64Accelerator: PlatformAccelerator { - public static let platformName = "x86-64" - - /// Always returns true on x86-64 architectures. - public static var isSupported: Bool { - return true - } - - /// Initialise an x86-64 SSE/AVX accelerator. - public init() {} - - // MARK: - SSE/AVX-Optimised Gradient Computation - - /// Compute local gradients using SSE/AVX SIMD operations. - /// - /// This implementation uses vectorised subtraction to compute all three - /// gradients in parallel using SSE/AVX instructions: - /// - D1 = b - c (horizontal gradient) - /// - D2 = a - c (vertical gradient) - /// - D3 = c - a (diagonal gradient) - /// - /// - Parameters: - /// - a: North pixel value - /// - b: West pixel value - /// - c: Northwest pixel value - /// - Returns: A tuple of three gradients (d1, d2, d3) - public func computeGradients(a: Int, b: Int, c: Int) -> (d1: Int, d2: Int, d3: Int) { - // Pack values into SIMD vector for parallel computation - // Vector layout: [a, b, c, 0] - let values = SIMD4(Int32(a), Int32(b), Int32(c), 0) - - // Create subtraction operands using SIMD shuffles - let operand1 = SIMD4(values[1], values[0], values[2], 0) // [b, a, c, 0] - let operand2 = SIMD4(values[2], values[2], values[0], 0) // [c, c, a, 0] - - // Vectorised subtraction using SSE/AVX - let gradients = operand1 &- operand2 // [b-c, a-c, c-a, 0] - - return (Int(gradients[0]), Int(gradients[1]), Int(gradients[2])) - } - - // MARK: - SSE/AVX-Optimised Run-Length Detection - - /// Detect run length using SIMD8 comparisons on x86-64. - /// - /// Scans ahead from `startIndex` in the `pixels` array, counting - /// consecutive elements equal to `runValue`. Uses SIMD8 vectorised - /// comparison to process 8 pixels per iteration, leveraging SSE - /// comparison instructions for maximum throughput. - /// - /// - Parameters: - /// - pixels: Array of pixel values to scan - /// - startIndex: Starting index for the scan - /// - runValue: The pixel value that constitutes a run - /// - maxLength: Maximum run length to detect - /// - Returns: Length of the run starting at `startIndex` - public func detectRunLength( - in pixels: [Int32], - startIndex: Int, - runValue: Int32, - maxLength: Int - ) -> Int { - let limit = min(pixels.count - startIndex, maxLength) - guard limit > 0 else { return 0 } - - var runLength = 0 - let vectorSize = 8 - let runVec = SIMD8(repeating: runValue) - - // Process 8 pixels at a time using SSE comparisons - while runLength + vectorSize <= limit { - let idx = startIndex + runLength - let chunk = SIMD8( - pixels[idx], pixels[idx + 1], pixels[idx + 2], pixels[idx + 3], - pixels[idx + 4], pixels[idx + 5], pixels[idx + 6], pixels[idx + 7] - ) - let matches = chunk .== runVec - - // Find first mismatch within the vector - for j in 0.. [Int] { - var positions: [Int] = [] - let count = data.count - let vectorSize = 8 - let ffVec = SIMD8(repeating: 0xFF) - - var i = 0 - // Process 8 bytes at a time using SSE - while i + vectorSize <= count { - let chunk = SIMD8( - data[i], data[i + 1], data[i + 2], data[i + 3], - data[i + 4], data[i + 5], data[i + 6], data[i + 7] - ) - let mask = chunk .== ffVec - - for j in 0.. Int { - guard n > 0 else { return 0 } - guard a > 0 else { return 0 } - - // floor(log2(a/n)) via leading-zero count (BSR/LZCNT on x86-64) - let aN = max(1, a / n) - let log2Estimate = max(0, (UInt64.bitWidth - 1 - UInt64(aN).leadingZeroBitCount)) - - // Start just below the estimate and advance until the condition is met - var k = max(0, log2Estimate > 0 ? log2Estimate - 1 : 0) - while k < 31 && (n << k) < a { - k += 1 - } - - return min(k, 31) - } - - // MARK: - SSE/AVX-Optimised MED Predictor - - /// Compute MED (Median Edge Detector) prediction using SSE/AVX operations. - /// - /// The MED predictor uses vectorised min/max operations available in SSE/AVX - /// to efficiently compute the prediction value: - /// - If c >= max(a, b): return min(a, b) - /// - If c <= min(a, b): return max(a, b) - /// - Otherwise: return a + b - c - /// - /// - Parameters: - /// - a: North pixel value - /// - b: West pixel value - /// - c: Northwest pixel value - /// - Returns: The predicted pixel value - public func medPredictor(a: Int, b: Int, c: Int) -> Int { - // Use SIMD for parallel min/max operations - let vec = SIMD4(Int32(a), Int32(b), Int32(c), 0) - - // Compute min(a, b) and max(a, b) using SSE/AVX min/max instructions - let minAB = min(vec[0], vec[1]) - let maxAB = max(vec[0], vec[1]) - - // MED predictor logic using SSE/AVX comparison operations - if vec[2] >= maxAB { - // c >= max(a, b) → return min(a, b) - return Int(minAB) - } else if vec[2] <= minAB { - // c <= min(a, b) → return max(a, b) - return Int(maxAB) - } else { - // Otherwise → return a + b - c - // Use SIMD addition and subtraction - let sum = vec[0] &+ vec[1] // a + b - let result = sum &- vec[2] // (a + b) - c - return Int(result) - } - } - - // MARK: - SSE/AVX-Optimised Gradient Quantisation - - /// Quantise gradients using SSE/AVX SIMD comparison operations. - /// - /// This implementation uses vectorised comparisons to process all three - /// gradients in parallel, leveraging SSE/AVX comparison and select operations - /// for maximum throughput. - /// - /// The quantisation maps gradient values to discrete levels [-4, 4] based - /// on threshold parameters (t1, t2, t3) per ITU-T.87 Section 4.3.1. - /// - /// - Parameters: - /// - d1: First gradient - /// - d2: Second gradient - /// - d3: Third gradient - /// - t1: Quantisation threshold 1 - /// - t2: Quantisation threshold 2 - /// - t3: Quantisation threshold 3 - /// - Returns: A tuple of three quantised gradient values (q1, q2, q3) - public func quantizeGradients(d1: Int, d2: Int, d3: Int, t1: Int, t2: Int, t3: Int) -> (q1: Int, q2: Int, q3: Int) { - let q1 = quantizeSingleGradient(gradient: d1, t1: t1, t2: t2, t3: t3) - let q2 = quantizeSingleGradient(gradient: d2, t1: t1, t2: t2, t3: t3) - let q3 = quantizeSingleGradient(gradient: d3, t1: t1, t2: t2, t3: t3) - - return (q1, q2, q3) - } - - /// Quantise a single gradient value using SSE/AVX-friendly logic. - /// - /// This helper function implements the ITU-T.87 quantisation algorithm - /// with comparisons that can be optimised by the compiler to SSE/AVX - /// instructions. - /// - /// Quantisation per ITU-T.87 Section 4.3.1: - /// - Q = -4 if d <= -t3 - /// - Q = -3 if -t3 < d <= -t2 - /// - Q = -2 if -t2 < d <= -t1 - /// - Q = -1 if -t1 < d < 0 - /// - Q = 0 if d == 0 - /// - Q = 1 if 0 < d < t1 - /// - Q = 2 if t1 <= d < t2 - /// - Q = 3 if t2 <= d < t3 - /// - Q = 4 if t3 <= d - /// - /// - Parameters: - /// - gradient: Raw gradient value - /// - t1: Threshold 1 - /// - t2: Threshold 2 - /// - t3: Threshold 3 - /// - Returns: Quantised gradient value in range [-4, 4] - @inline(__always) - private func quantizeSingleGradient( - gradient: Int, - t1: Int, - t2: Int, - t3: Int - ) -> Int { - if gradient <= -t3 { - return -4 - } else if gradient <= -t2 { - return -3 - } else if gradient <= -t1 { - return -2 - } else if gradient < 0 { - return -1 - } else if gradient == 0 { - return 0 - } else if gradient < t1 { - return 1 - } else if gradient < t2 { - return 2 - } else if gradient < t3 { - return 3 - } else { - return 4 - } - } -} - -#endif diff --git a/Sources/jpeglscli/BatchCommand.swift b/Sources/jpeglscli/BatchCommand.swift index 92dead6..1a93cbe 100644 --- a/Sources/jpeglscli/BatchCommand.swift +++ b/Sources/jpeglscli/BatchCommand.swift @@ -18,30 +18,21 @@ struct Batch: ParsableCommand { @Argument(help: "Operation to perform: encode, decode, info, verify") var operation: String - @Argument(help: "Input glob pattern (e.g., '*.jls', 'images/*.raw') or directory path") + @Argument(help: "Input glob pattern (e.g., '*.jls', 'scans/*.pgm') or directory path") var inputPattern: String - + @Option(name: .shortAndLong, help: "Output directory for processed files") var outputDir: String? - + // MARK: - Encoding Options - - @Option(name: .shortAndLong, help: "Image width in pixels (required for encode)") - var width: Int? - - @Option(name: .shortAndLong, help: "Image height in pixels (required for encode)") - var height: Int? - - @Option(name: .shortAndLong, help: "Bits per sample, 2-16 (default: 8)") - var bitsPerSample: Int = 8 - - @Option(name: .shortAndLong, help: "Number of components - 1 (greyscale) or 3 (RGB) (default: 1)") - var components: Int = 1 - + // + // Image geometry and bit depth are auto-detected from the input files + // (PGM/PPM, PNG, or TIFF); raw pixel input is not supported in batch mode. + @Option(help: "NEAR parameter, 0=lossless, 1-255=lossy (default: 0)") var near: Int = 0 - - @Option(help: "Interleave mode: none, line, sample (default: none)") + + @Option(help: "Interleave mode for colour inputs: none, line, sample (default: none)") var interleave: String = "none" @Option( @@ -90,23 +81,19 @@ struct Batch: ParsableCommand { throw ValidationError("Cannot specify both --verbose and --quiet") } - // Validate encode-specific requirements + // Validate encode-specific requirements. Image geometry and bit depth + // are auto-detected from the input files (PGM/PPM/PNG/TIFF), so only + // the coding parameters need validation here. if operation.lowercased() == "encode" { - guard let width = width, width > 0 else { - throw ValidationError("--width is required and must be positive for encode operation") - } - guard let height = height, height > 0 else { - throw ValidationError("--height is required and must be positive for encode operation") - } - guard (2...16).contains(bitsPerSample) else { - throw ValidationError("--bits-per-sample must be between 2 and 16") - } - guard components == 1 || components == 3 else { - throw ValidationError("--components must be 1 (greyscale) or 3 (RGB)") - } guard (0...255).contains(near) else { throw ValidationError("--near must be between 0 and 255") } + guard ["none", "line", "sample"].contains(interleave.lowercased()) else { + throw ValidationError("--interleave must be one of: none, line, sample") + } + guard ["none", "hp1", "hp2", "hp3"].contains(colorTransform.lowercased()) else { + throw ValidationError("--color-transform must be one of: none, hp1, hp2, hp3") + } } // Validate output directory for encode/decode operations @@ -130,10 +117,6 @@ struct Batch: ParsableCommand { inputPattern: inputPattern, outputDir: outputDir, encodeOptions: EncodeOptions( - width: width ?? 0, - height: height ?? 0, - bitsPerSample: bitsPerSample, - components: components, near: near, interleave: interleave, colorTransform: colorTransform @@ -153,10 +136,6 @@ struct Batch: ParsableCommand { // MARK: - Encode Options struct EncodeOptions: Sendable { - let width: Int - let height: Int - let bitsPerSample: Int - let components: Int let near: Int let interleave: String let colorTransform: String @@ -417,16 +396,106 @@ struct BatchProcessor: Sendable { return (outputDir as NSString).appendingPathComponent(outputFilename) } + /// Lossless-encode one image file (PGM/PPM, PNG, or TIFF — auto-detected) + /// to JPEG-LS, mirroring `jpegls encode` defaults. `JPEGLSEncoder` is a + /// stateless Sendable struct, so concurrent per-file use from the worker + /// pool is safe. private func processEncode(input: String, output: String) throws { - // Placeholder for encode implementation - // TODO: Integrate with actual encoder when bitstream writer is complete - throw ValidationError("Encode operation requires bitstream writer integration (not yet implemented)") + let inputData = try Data(contentsOf: URL(fileURLWithPath: input)) + + let componentPixels: [[[Int]]] + let bitsPerSample: Int + if PNGSupport.isPNG(inputData) { + let png = try PNGSupport.decode(inputData) + componentPixels = png.componentPixels + bitsPerSample = png.bitDepth + } else if TIFFSupport.isTIFF(inputData) { + let tiff = try TIFFSupport.decode(inputData) + componentPixels = tiff.componentPixels + bitsPerSample = tiff.bitsPerSample + } else if inputData.count >= 2, + inputData[inputData.startIndex] == UInt8(ascii: "P"), + inputData[inputData.startIndex + 1] == UInt8(ascii: "5") + || inputData[inputData.startIndex + 1] == UInt8(ascii: "6") { + let pnm = try PNMSupport.parse(inputData) + componentPixels = pnm.componentPixels + var bits = 1 + while (1 << bits) - 1 < pnm.maxVal { bits += 1 } + bitsPerSample = max(2, bits) + } else { + throw ValidationError( + "Unsupported input format for batch encode: \(input) (PGM/PPM, PNG, or TIFF required)" + ) + } + + let imageData: MultiComponentImageData + switch componentPixels.count { + case 1: + imageData = try MultiComponentImageData.grayscale( + pixels: componentPixels[0], bitsPerSample: bitsPerSample + ) + case 3: + imageData = try MultiComponentImageData.rgb( + redPixels: componentPixels[0], + greenPixels: componentPixels[1], + bluePixels: componentPixels[2], + bitsPerSample: bitsPerSample + ) + default: + throw ValidationError("Batch encode requires 1 or 3 components; got \(componentPixels.count)") + } + + let interleaveMode: JPEGLSInterleaveMode + switch encodeOptions.interleave.lowercased() { + case "line": interleaveMode = .line + case "sample": interleaveMode = .sample + default: interleaveMode = .none + } + let colorTransformation: JPEGLSColorTransformation + switch encodeOptions.colorTransform.lowercased() { + case "hp1": colorTransformation = .hp1 + case "hp2": colorTransformation = .hp2 + case "hp3": colorTransformation = .hp3 + default: colorTransformation = .none + } + // Greyscale inputs always use a single non-interleaved scan; the + // interleave/colour-transform options only apply to colour inputs. + let isColour = componentPixels.count == 3 + let config = try JPEGLSEncoder.Configuration( + near: encodeOptions.near, + interleaveMode: isColour ? interleaveMode : .none, + colorTransformation: isColour ? colorTransformation : .none + ) + let encoded = try JPEGLSEncoder().encode(imageData, configuration: config) + try encoded.write(to: URL(fileURLWithPath: output)) } - + + /// Decode one JPEG-LS file to packed raw samples (8-bit bytes, or 16-bit + /// big-endian words), matching `jpegls decode --format raw`. private func processDecode(input: String, output: String) throws { - // Placeholder for decode implementation - // TODO: Integrate with actual decoder when bitstream reader is complete - throw ValidationError("Decode operation requires bitstream reader integration (not yet implemented)") + let inputData = try Data(contentsOf: URL(fileURLWithPath: input)) + let imageData = try JPEGLSDecoder().decode(inputData) + + let wide = imageData.frameHeader.bitsPerSample > 8 + var bytes = [UInt8]() + bytes.reserveCapacity( + imageData.frameHeader.width * imageData.frameHeader.height + * imageData.components.count * (wide ? 2 : 1) + ) + for component in imageData.components { + for row in component.pixels { + for pixel in row { + if wide { + let value = UInt16(clamping: pixel) + bytes.append(UInt8((value >> 8) & 0xFF)) + bytes.append(UInt8(value & 0xFF)) + } else { + bytes.append(UInt8(clamping: pixel)) + } + } + } + } + try Data(bytes).write(to: URL(fileURLWithPath: output)) } private func processInfo(input: String) throws { diff --git a/Sources/jpeglscli/DecodeCommand.swift b/Sources/jpeglscli/DecodeCommand.swift index 20d3a38..bdd18e3 100644 --- a/Sources/jpeglscli/DecodeCommand.swift +++ b/Sources/jpeglscli/DecodeCommand.swift @@ -95,24 +95,30 @@ extension JPEGLSCLITool { // Write output based on format switch format.lowercased() { case "raw": - // Write raw pixel data - var outputData = Data() + // Write raw pixel data. Accumulate in [UInt8] and convert to + // Data once — per-byte Data.append is ~70x slower. + let wide = imageData.frameHeader.bitsPerSample > 8 + var bytes = [UInt8]() + bytes.reserveCapacity( + imageData.components.reduce(0) { $0 + $1.pixels.count * ($1.pixels.first?.count ?? 0) } + * (wide ? 2 : 1) + ) for component in imageData.components { for row in component.pixels { for pixel in row { - // Write pixel value in appropriate byte size - if imageData.frameHeader.bitsPerSample <= 8 { - outputData.append(UInt8(clamping: pixel)) - } else { + if wide { // Write as 16-bit big-endian let value = UInt16(clamping: pixel) - outputData.append(UInt8((value >> 8) & 0xFF)) - outputData.append(UInt8(value & 0xFF)) + bytes.append(UInt8((value >> 8) & 0xFF)) + bytes.append(UInt8(value & 0xFF)) + } else { + bytes.append(UInt8(clamping: pixel)) } } } } - + let outputData = Data(bytes) + try outputData.write(to: URL(fileURLWithPath: output)) if !quiet { diff --git a/Sources/jpeglscli/EncodeCommand.swift b/Sources/jpeglscli/EncodeCommand.swift index d43ea57..dd885fc 100644 --- a/Sources/jpeglscli/EncodeCommand.swift +++ b/Sources/jpeglscli/EncodeCommand.swift @@ -62,6 +62,9 @@ extension JPEGLSCLITool { @Option(name: .long, help: "NEAR parameter for near-lossless encoding (0=lossless, 1-255=lossy, default: 0)") var near: Int = 0 + + @Option(name: .long, help: "Restart interval in lines (0=off, default). Emits DRI + RSTm markers so intervals decode independently; lossless non-interleaved only") + var restartInterval: Int = 0 @Option(name: .long, help: "Interleave mode: none, line, sample (default: none)") var interleave: String = "none" @@ -303,7 +306,8 @@ extension JPEGLSCLITool { interleaveMode: actualInterleaveMode, presetParameters: resolvedPresetParameters, colorTransformation: colorTransformValue, - mappingTable: resolvedMappingTable + mappingTable: resolvedMappingTable, + restartInterval: restartInterval ) if verbose { diff --git a/Tests/JPEGLSTests/ARM64AcceleratorPhase13Tests.swift b/Tests/JPEGLSTests/ARM64AcceleratorPhase13Tests.swift deleted file mode 100644 index 8d418df..0000000 --- a/Tests/JPEGLSTests/ARM64AcceleratorPhase13Tests.swift +++ /dev/null @@ -1,205 +0,0 @@ -/// Tests for the Phase 13.1 ARM64 Neon enhancements. -/// -/// These tests cover the Golomb-Rice parameter computation, run-length -/// detection, and byte stuffing detection added to ARM64Accelerator as -/// part of Milestone 13 Phase 13.1. -/// -/// All tests are compiled and run only on ARM64 architectures where -/// the ARM64Accelerator is available. - -#if arch(arm64) - -import Testing -import Foundation -@testable import JPEGLS - -@Suite("ARM64 Accelerator Phase 13.1 Tests") -struct ARM64AcceleratorPhase13Tests { - - // MARK: - Golomb-Rice Parameter Computation - - @Test("Golomb-Rice parameter is zero when a is zero") - func golombRiceParamZeroA() { - let acc = ARM64Accelerator() - #expect(acc.computeGolombRiceParameter(a: 0, n: 64) == 0) - } - - @Test("Golomb-Rice parameter is zero when n is zero") - func golombRiceParamZeroN() { - let acc = ARM64Accelerator() - #expect(acc.computeGolombRiceParameter(a: 100, n: 0) == 0) - } - - @Test("Golomb-Rice parameter is zero when a <= n (small error accumulator)") - func golombRiceParamSmallA() { - let acc = ARM64Accelerator() - // When a <= n, k should be 0 (threshold n*1 >= a) - #expect(acc.computeGolombRiceParameter(a: 10, n: 64) == 0) - } - - @Test("Golomb-Rice parameter increases with larger a relative to n") - func golombRiceParamIncreases() { - let acc = ARM64Accelerator() - let k1 = acc.computeGolombRiceParameter(a: 64, n: 64) // a/n = 1 - let k2 = acc.computeGolombRiceParameter(a: 128, n: 64) // a/n = 2 - let k3 = acc.computeGolombRiceParameter(a: 512, n: 64) // a/n = 8 - // k should be non-decreasing as a grows - #expect(k1 <= k2) - #expect(k2 <= k3) - } - - @Test("Golomb-Rice parameter satisfies 2^k * n >= a") - func golombRiceParamSatisfiesCondition() { - let acc = ARM64Accelerator() - let n = 64 - for a in [64, 128, 256, 512, 1024, 2048] { - let k = acc.computeGolombRiceParameter(a: a, n: n) - // Primary condition: 2^k * n >= a - #expect((n << k) >= a, "k=\(k) for a=\(a) n=\(n): 2^k*n should be >= a") - // Minimality: if k > 0, then 2^(k-1) * n < a - if k > 0 { - #expect((n << (k - 1)) < a, "k=\(k) should be minimal for a=\(a) n=\(n)") - } - } - } - - @Test("Golomb-Rice parameter is bounded within [0, 31]") - func golombRiceParamBounded() { - let acc = ARM64Accelerator() - let k = acc.computeGolombRiceParameter(a: Int.max / 2, n: 1) - #expect(k >= 0) - #expect(k <= 31) - } - - // MARK: - Run-Length Detection - - @Test("Run-length detection returns 0 for empty slice") - func runLengthEmpty() { - let acc = ARM64Accelerator() - #expect(acc.detectRunLength(in: [], startIndex: 0, runValue: 0, maxLength: 100) == 0) - } - - @Test("Run-length detection with maxLength 0 returns 0") - func runLengthMaxLengthZero() { - let acc = ARM64Accelerator() - let pixels: [Int32] = [10, 10, 10] - #expect(acc.detectRunLength(in: pixels, startIndex: 0, runValue: 10, maxLength: 0) == 0) - } - - @Test("Run-length detection counts full run of equal pixels") - func runLengthFullRun() { - let acc = ARM64Accelerator() - let pixels: [Int32] = [5, 5, 5, 5, 5] - let length = acc.detectRunLength(in: pixels, startIndex: 0, runValue: 5, maxLength: 100) - #expect(length == 5) - } - - @Test("Run-length detection stops at first mismatch") - func runLengthStopsAtMismatch() { - let acc = ARM64Accelerator() - let pixels: [Int32] = [10, 10, 10, 20, 10] - let length = acc.detectRunLength(in: pixels, startIndex: 0, runValue: 10, maxLength: 100) - #expect(length == 3) - } - - @Test("Run-length detection respects maxLength") - func runLengthRespectMaxLength() { - let acc = ARM64Accelerator() - let pixels: [Int32] = [7, 7, 7, 7, 7, 7, 7, 7] - let length = acc.detectRunLength(in: pixels, startIndex: 0, runValue: 7, maxLength: 4) - #expect(length == 4) - } - - @Test("Run-length detection can start from non-zero index") - func runLengthStartIndex() { - let acc = ARM64Accelerator() - let pixels: [Int32] = [1, 2, 3, 3, 3, 3, 4] - let length = acc.detectRunLength(in: pixels, startIndex: 2, runValue: 3, maxLength: 100) - #expect(length == 4) - } - - @Test("Run-length detection with single matching element") - func runLengthSingleMatch() { - let acc = ARM64Accelerator() - let pixels: [Int32] = [99] - #expect(acc.detectRunLength(in: pixels, startIndex: 0, runValue: 99, maxLength: 100) == 1) - } - - @Test("Run-length detection returns 0 when first element mismatches") - func runLengthFirstMismatch() { - let acc = ARM64Accelerator() - let pixels: [Int32] = [5, 5, 5] - #expect(acc.detectRunLength(in: pixels, startIndex: 0, runValue: 9, maxLength: 100) == 0) - } - - @Test("Run-length detection across SIMD vector boundary") - func runLengthAcrossVectorBoundary() { - let acc = ARM64Accelerator() - // 8 matching + 2 more = run of 10, crossing the 8-element SIMD boundary - let pixels = [Int32](repeating: 42, count: 10) + [Int32](repeating: 0, count: 5) - let length = acc.detectRunLength(in: pixels, startIndex: 0, runValue: 42, maxLength: 100) - #expect(length == 10) - } - - // MARK: - Byte Stuffing Detection - - @Test("Byte stuffing detection returns empty for non-0xFF data") - func byteStuffingNone() { - let acc = ARM64Accelerator() - let data: [UInt8] = [0x00, 0x01, 0x7F, 0xFE, 0x80, 0x55] - #expect(acc.detectByteStuffingPositions(in: data).isEmpty) - } - - @Test("Byte stuffing detection finds single 0xFF") - func byteStuffingSingle() { - let acc = ARM64Accelerator() - let data: [UInt8] = [0x00, 0xFF, 0x01] - let positions = acc.detectByteStuffingPositions(in: data) - #expect(positions == [1]) - } - - @Test("Byte stuffing detection finds multiple 0xFF bytes") - func byteStuffingMultiple() { - let acc = ARM64Accelerator() - let data: [UInt8] = [0xFF, 0x00, 0xFF, 0x7F, 0xFF] - let positions = acc.detectByteStuffingPositions(in: data) - #expect(positions == [0, 2, 4]) - } - - @Test("Byte stuffing detection handles empty data") - func byteStuffingEmpty() { - let acc = ARM64Accelerator() - #expect(acc.detectByteStuffingPositions(in: []).isEmpty) - } - - @Test("Byte stuffing detection handles all-0xFF data") - func byteStuffingAllFF() { - let acc = ARM64Accelerator() - let data = [UInt8](repeating: 0xFF, count: 16) - let positions = acc.detectByteStuffingPositions(in: data) - #expect(positions == Array(0..<16)) - } - - @Test("Byte stuffing detection crosses SIMD boundary") - func byteStuffingCrossesVectorBoundary() { - let acc = ARM64Accelerator() - // 0xFF at index 7 (last of first SIMD chunk) and 8 (first of second) - var data = [UInt8](repeating: 0x00, count: 16) - data[7] = 0xFF - data[8] = 0xFF - let positions = acc.detectByteStuffingPositions(in: data) - #expect(positions == [7, 8]) - } - - @Test("Byte stuffing detection in tail (count not multiple of 8)") - func byteStuffingInTail() { - let acc = ARM64Accelerator() - // 9 bytes: 0xFF is at index 8 (the tail byte after the first SIMD chunk) - var data = [UInt8](repeating: 0x00, count: 9) - data[8] = 0xFF - let positions = acc.detectByteStuffingPositions(in: data) - #expect(positions == [8]) - } -} - -#endif // arch(arm64) diff --git a/Tests/JPEGLSTests/AccelerateFrameworkAcceleratorTests.swift b/Tests/JPEGLSTests/AccelerateFrameworkAcceleratorTests.swift deleted file mode 100644 index ca8978e..0000000 --- a/Tests/JPEGLSTests/AccelerateFrameworkAcceleratorTests.swift +++ /dev/null @@ -1,490 +0,0 @@ -import Testing -import Foundation -@testable import JPEGLS - -#if canImport(Accelerate) - -@Suite("Accelerate Framework Accelerator Tests") -struct AccelerateFrameworkAcceleratorTests { - // MARK: - Platform Info Tests - - @Test("AccelerateFrameworkAccelerator platformName is correct") - func acceleratePlatformName() { - #expect(AccelerateFrameworkAccelerator.platformName == "Accelerate") - } - - @Test("AccelerateFrameworkAccelerator is supported when Accelerate is available") - func accelerateIsSupported() { - #expect(AccelerateFrameworkAccelerator.isSupported == true) - } - - @Test("AccelerateFrameworkAccelerator initialization") - func accelerateInitialization() { - let accelerator = AccelerateFrameworkAccelerator() - // If it initializes without crashing, the test passes - #expect(AccelerateFrameworkAccelerator.isSupported) - } - - // MARK: - Batch Gradient Computation Tests - - @Test("Batch gradient computation with simple values") - func batchGradientsSimple() { - let accelerator = AccelerateFrameworkAccelerator() - - let a = [10, 20, 30] - let b = [20, 30, 40] - let c = [15, 25, 35] - - let result = accelerator.computeGradientsBatch(a: a, b: b, c: c) - - // D1 = b - c - #expect(result.d1 == [5, 5, 5]) - - // D2 = a - c - #expect(result.d2 == [-5, -5, -5]) - - // D3 = c - a - #expect(result.d3 == [5, 5, 5]) - } - - @Test("Batch gradient computation with varying values") - func batchGradientsVarying() { - let accelerator = AccelerateFrameworkAccelerator() - - let a = [10, 50, 100] - let b = [20, 40, 90] - let c = [15, 45, 95] - - let result = accelerator.computeGradientsBatch(a: a, b: b, c: c) - - // D1 = b - c: [20-15, 40-45, 90-95] = [5, -5, -5] - #expect(result.d1 == [5, -5, -5]) - - // D2 = a - c: [10-15, 50-45, 100-95] = [-5, 5, 5] - #expect(result.d2 == [-5, 5, 5]) - - // D3 = c - a: [15-10, 45-50, 95-100] = [5, -5, -5] - #expect(result.d3 == [5, -5, -5]) - } - - @Test("Batch gradient computation with zero gradients") - func batchGradientsZero() { - let accelerator = AccelerateFrameworkAccelerator() - - let a = [10, 10, 10] - let b = [10, 10, 10] - let c = [10, 10, 10] - - let result = accelerator.computeGradientsBatch(a: a, b: b, c: c) - - #expect(result.d1 == [0, 0, 0]) - #expect(result.d2 == [0, 0, 0]) - #expect(result.d3 == [0, 0, 0]) - } - - @Test("Batch gradient computation with empty arrays") - func batchGradientsEmpty() { - let accelerator = AccelerateFrameworkAccelerator() - - let result = accelerator.computeGradientsBatch(a: [], b: [], c: []) - - #expect(result.d1.isEmpty) - #expect(result.d2.isEmpty) - #expect(result.d3.isEmpty) - } - - @Test("Batch gradient computation with single element") - func batchGradientsSingle() { - let accelerator = AccelerateFrameworkAccelerator() - - let result = accelerator.computeGradientsBatch(a: [10], b: [20], c: [15]) - - #expect(result.d1 == [5]) - #expect(result.d2 == [-5]) - #expect(result.d3 == [5]) - } - - @Test("Batch gradient computation with large arrays") - func batchGradientsLarge() { - let accelerator = AccelerateFrameworkAccelerator() - - let count = 1000 - let a = Array(repeating: 100, count: count) - let b = Array(repeating: 150, count: count) - let c = Array(repeating: 120, count: count) - - let result = accelerator.computeGradientsBatch(a: a, b: b, c: c) - - #expect(result.d1.count == count) - #expect(result.d2.count == count) - #expect(result.d3.count == count) - - // D1 = b - c = 150 - 120 = 30 - #expect(result.d1.allSatisfy { $0 == 30 }) - - // D2 = a - c = 100 - 120 = -20 - #expect(result.d2.allSatisfy { $0 == -20 }) - - // D3 = c - a = 120 - 100 = 20 - #expect(result.d3.allSatisfy { $0 == 20 }) - } - - // MARK: - Statistical Analysis Tests - - @Test("Compute mean of values") - func computeMean() { - let accelerator = AccelerateFrameworkAccelerator() - - let values = [10, 20, 30, 40, 50] - let mean = accelerator.computeMean(values: values) - - #expect(abs(mean - 30.0) < 0.001) - } - - @Test("Compute mean with empty array") - func computeMeanEmpty() { - let accelerator = AccelerateFrameworkAccelerator() - - let mean = accelerator.computeMean(values: []) - #expect(mean == 0.0) - } - - @Test("Compute mean with single value") - func computeMeanSingle() { - let accelerator = AccelerateFrameworkAccelerator() - - let mean = accelerator.computeMean(values: [42]) - #expect(abs(mean - 42.0) < 0.001) - } - - @Test("Compute variance of values") - func computeVariance() { - let accelerator = AccelerateFrameworkAccelerator() - - let values = [10, 20, 30, 40, 50] - let variance = accelerator.computeVariance(values: values) - - // Expected variance: ((10-30)^2 + (20-30)^2 + (30-30)^2 + (40-30)^2 + (50-30)^2) / 4 - // = (400 + 100 + 0 + 100 + 400) / 4 = 1000 / 4 = 250 - #expect(abs(variance - 250.0) < 0.001) - } - - @Test("Compute variance with two values") - func computeVarianceTwoValues() { - let accelerator = AccelerateFrameworkAccelerator() - - let values = [10, 20] - let variance = accelerator.computeVariance(values: values) - - // Expected variance: ((10-15)^2 + (20-15)^2) / 1 = (25 + 25) / 1 = 50 - #expect(abs(variance - 50.0) < 0.001) - } - - @Test("Compute variance with single value returns zero") - func computeVarianceSingle() { - let accelerator = AccelerateFrameworkAccelerator() - - let variance = accelerator.computeVariance(values: [42]) - #expect(variance == 0.0) - } - - @Test("Compute standard deviation") - func computeStdDev() { - let accelerator = AccelerateFrameworkAccelerator() - - let values = [10, 20, 30, 40, 50] - let stdDev = accelerator.computeStandardDeviation(values: values) - - // Expected std dev: sqrt(250) ≈ 15.811 - #expect(abs(stdDev - 15.811) < 0.01) - } - - @Test("Compute min and max values") - func computeMinMax() { - let accelerator = AccelerateFrameworkAccelerator() - - let values = [30, 10, 50, 20, 40] - let (min, max) = accelerator.computeMinMax(values: values) - - #expect(min == 10) - #expect(max == 50) - } - - @Test("Compute min and max with single value") - func computeMinMaxSingle() { - let accelerator = AccelerateFrameworkAccelerator() - - let (min, max) = accelerator.computeMinMax(values: [42]) - - #expect(min == 42) - #expect(max == 42) - } - - @Test("Compute min and max with empty array") - func computeMinMaxEmpty() { - let accelerator = AccelerateFrameworkAccelerator() - - let (min, max) = accelerator.computeMinMax(values: []) - - #expect(min == 0) - #expect(max == 0) - } - - // MARK: - Histogram Tests - - @Test("Compute histogram with uniform distribution") - func histogramUniform() { - let accelerator = AccelerateFrameworkAccelerator() - - // Values 0-9, each appearing once - let pixels = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] - let histogram = accelerator.computeHistogram( - pixels: pixels, - binCount: 10, - minValue: 0, - maxValue: 10 - ) - - #expect(histogram.count == 10) - // Each bin should have 1 value (approximately uniform) - #expect(histogram.allSatisfy { $0 == 1 }) - } - - @Test("Compute histogram with concentrated values") - func histogramConcentrated() { - let accelerator = AccelerateFrameworkAccelerator() - - // All values in middle range - let pixels = [50, 51, 52, 53, 54, 55] - let histogram = accelerator.computeHistogram( - pixels: pixels, - binCount: 10, - minValue: 0, - maxValue: 100 - ) - - #expect(histogram.count == 10) - // Most bins should be empty except bin 5 (50-59 range) - #expect(histogram[5] == 6) - } - - @Test("Compute histogram with empty pixels") - func histogramEmpty() { - let accelerator = AccelerateFrameworkAccelerator() - - let histogram = accelerator.computeHistogram( - pixels: [], - binCount: 10, - minValue: 0, - maxValue: 100 - ) - - #expect(histogram.count == 10) - #expect(histogram.allSatisfy { $0 == 0 }) - } - - @Test("Compute histogram with out-of-range values") - func histogramOutOfRange() { - let accelerator = AccelerateFrameworkAccelerator() - - let pixels = [-10, 0, 50, 100, 150] - let histogram = accelerator.computeHistogram( - pixels: pixels, - binCount: 10, - minValue: 0, - maxValue: 100 - ) - - #expect(histogram.count == 10) - // Only values 0, 50, 100 should be counted - let totalCount = histogram.reduce(0, +) - #expect(totalCount == 3) - } - - @Test("Compute histogram with single bin") - func histogramSingleBin() { - let accelerator = AccelerateFrameworkAccelerator() - - let pixels = [0, 50, 100, 25, 75] - let histogram = accelerator.computeHistogram( - pixels: pixels, - binCount: 1, - minValue: 0, - maxValue: 100 - ) - - #expect(histogram.count == 1) - #expect(histogram[0] == 5) // All values in one bin - } - - // MARK: - Batch Vector Operations Tests - - @Test("Add two arrays element-wise") - func addArrays() { - let accelerator = AccelerateFrameworkAccelerator() - - let a = [10, 20, 30] - let b = [5, 10, 15] - let result = accelerator.addArrays(a: a, b: b) - - #expect(result == [15, 30, 45]) - } - - @Test("Add arrays with zero values") - func addArraysZero() { - let accelerator = AccelerateFrameworkAccelerator() - - let a = [10, 20, 30] - let b = [0, 0, 0] - let result = accelerator.addArrays(a: a, b: b) - - #expect(result == [10, 20, 30]) - } - - @Test("Add empty arrays") - func addArraysEmpty() { - let accelerator = AccelerateFrameworkAccelerator() - - let result = accelerator.addArrays(a: [], b: []) - - #expect(result.isEmpty) - } - - @Test("Subtract two arrays element-wise") - func subtractArrays() { - let accelerator = AccelerateFrameworkAccelerator() - - let a = [30, 40, 50] - let b = [10, 20, 30] - let result = accelerator.subtractArrays(a: a, b: b) - - #expect(result == [20, 20, 20]) - } - - @Test("Subtract arrays resulting in negative values") - func subtractArraysNegative() { - let accelerator = AccelerateFrameworkAccelerator() - - let a = [10, 20, 30] - let b = [20, 30, 40] - let result = accelerator.subtractArrays(a: a, b: b) - - #expect(result == [-10, -10, -10]) - } - - @Test("Subtract arrays with zero result") - func subtractArraysZero() { - let accelerator = AccelerateFrameworkAccelerator() - - let a = [10, 20, 30] - let b = [10, 20, 30] - let result = accelerator.subtractArrays(a: a, b: b) - - #expect(result == [0, 0, 0]) - } - - @Test("Multiply array by positive scalar") - func multiplyByScalarPositive() { - let accelerator = AccelerateFrameworkAccelerator() - - let array = [10, 20, 30] - let result = accelerator.multiplyByScalar(array: array, scalar: 3) - - #expect(result == [30, 60, 90]) - } - - @Test("Multiply array by zero scalar") - func multiplyByScalarZero() { - let accelerator = AccelerateFrameworkAccelerator() - - let array = [10, 20, 30] - let result = accelerator.multiplyByScalar(array: array, scalar: 0) - - #expect(result == [0, 0, 0]) - } - - @Test("Multiply array by negative scalar") - func multiplyByScalarNegative() { - let accelerator = AccelerateFrameworkAccelerator() - - let array = [10, 20, 30] - let result = accelerator.multiplyByScalar(array: array, scalar: -2) - - #expect(result == [-20, -40, -60]) - } - - @Test("Multiply empty array by scalar") - func multiplyByScalarEmpty() { - let accelerator = AccelerateFrameworkAccelerator() - - let result = accelerator.multiplyByScalar(array: [], scalar: 5) - - #expect(result.isEmpty) - } - - // MARK: - Integration Tests - - @Test("Batch gradients match scalar implementation") - func batchGradientsMatchScalar() { - let accelerateAccelerator = AccelerateFrameworkAccelerator() - let scalarAccelerator = ScalarAccelerator() - - let a = [10, 50, 100, 25, 75] - let b = [20, 40, 90, 30, 80] - let c = [15, 45, 95, 27, 77] - - let batchResult = accelerateAccelerator.computeGradientsBatch(a: a, b: b, c: c) - - // Compare with scalar implementation - for i in 0.. 0) - } - - @Test("Vector operations maintain array length") - func vectorOperationsLength() { - let accelerator = AccelerateFrameworkAccelerator() - - let count = 100 - let a = Array(0..> 1) = 50 − 150 = −100 - #expect(rP == [100]) - #expect(gP == [100]) - #expect(bP == [-100]) - } - - @Test("applyHP2Inverse recovers original values") - func hp2Inverse() { - let acc = AccelerateFrameworkAccelerator() - let r = [200, 100, 50], g = [100, 200, 150], b = [50, 75, 200] - let (rP, gP, bP) = acc.applyHP2Forward(r: r, g: g, b: b) - let (rRec, gRec, bRec) = acc.applyHP2Inverse(rPrime: rP, gPrime: gP, bPrime: bP) - #expect(rRec == r) - #expect(gRec == g) - #expect(bRec == b) - } - - @Test("applyHP2Forward and Inverse are consistent with scalar transform") - func hp2BatchConsistency() { - let acc = AccelerateFrameworkAccelerator() - let r = [10, 200, 0, 128] - let g = [20, 100, 255, 64] - let b = [30, 150, 100, 32] - - let (rP, gP, bP) = acc.applyHP2Forward(r: r, g: g, b: b) - - for i in 0..> 1), "HP2 B′ mismatch at index \(i)") - } - } - - @Test("applyHP2 with empty arrays returns empty") - func hp2Empty() { - let acc = AccelerateFrameworkAccelerator() - let (rP, gP, bP) = acc.applyHP2Forward(r: [], g: [], b: []) - #expect(rP.isEmpty && gP.isEmpty && bP.isEmpty) - } - - // MARK: - HP3 Colour Transform - - @Test("applyHP3Forward transforms correctly") - func hp3Forward() { - let acc = AccelerateFrameworkAccelerator() - let r = [200], g = [100], b = [50] - let (rP, gP, bP) = acc.applyHP3Forward(r: r, g: g, b: b) - // R′ = R−B = 150 - // G′ = G − ((R+B) >> 1) = 100 − 125 = −25 - // B′ = B = 50 (unchanged) - #expect(rP == [150]) - #expect(gP == [-25]) - #expect(bP == [50]) - } - - @Test("applyHP3Inverse recovers original values") - func hp3Inverse() { - let acc = AccelerateFrameworkAccelerator() - let r = [200, 100, 50], g = [100, 200, 150], b = [50, 75, 200] - let (rP, gP, bP) = acc.applyHP3Forward(r: r, g: g, b: b) - let (rRec, gRec, bRec) = acc.applyHP3Inverse(rPrime: rP, gPrime: gP, bPrime: bP) - #expect(rRec == r) - #expect(gRec == g) - #expect(bRec == b) - } - - @Test("applyHP3Forward and Inverse are consistent with scalar transform") - func hp3BatchConsistency() { - let acc = AccelerateFrameworkAccelerator() - let r = [10, 200, 0, 128] - let g = [20, 100, 255, 64] - let b = [30, 150, 100, 32] - - let (rP, gP, bP) = acc.applyHP3Forward(r: r, g: g, b: b) - - for i in 0..> 1), "HP3 G′ mismatch at index \(i)") - #expect(bP[i] == b[i], "HP3 B′ should be unchanged at index \(i)") - } - } - - @Test("applyHP3 with empty arrays returns empty") - func hp3Empty() { - let acc = AccelerateFrameworkAccelerator() - let (rP, gP, bP) = acc.applyHP3Forward(r: [], g: [], b: []) - #expect(rP.isEmpty && gP.isEmpty && bP.isEmpty) - } - - // MARK: - Consistency with JPEGLSColorTransformation - - @Test("HP1 batch transform matches JPEGLSColorTransformation scalar for single pixel") - func hp1ConsistencyWithScalar() throws { - let acc = AccelerateFrameworkAccelerator() - let r = [200], g = [100], b = [50] - let (rP, gP, bP) = acc.applyHP1Forward(r: r, g: g, b: b) - - let scalarResult = try JPEGLSColorTransformation.hp1.transformForward([r[0], g[0], b[0]]) - #expect(rP[0] == scalarResult[0]) - #expect(gP[0] == scalarResult[1]) - #expect(bP[0] == scalarResult[2]) - } - - @Test("HP2 batch transform matches JPEGLSColorTransformation scalar for single pixel") - func hp2ConsistencyWithScalar() throws { - let acc = AccelerateFrameworkAccelerator() - let r = [200], g = [100], b = [50] - let (rP, gP, bP) = acc.applyHP2Forward(r: r, g: g, b: b) - - let scalarResult = try JPEGLSColorTransformation.hp2.transformForward([r[0], g[0], b[0]]) - #expect(rP[0] == scalarResult[0]) - #expect(gP[0] == scalarResult[1]) - #expect(bP[0] == scalarResult[2]) - } - - @Test("HP3 batch transform matches JPEGLSColorTransformation scalar for single pixel") - func hp3ConsistencyWithScalar() throws { - let acc = AccelerateFrameworkAccelerator() - let r = [200], g = [100], b = [50] - let (rP, gP, bP) = acc.applyHP3Forward(r: r, g: g, b: b) - - let scalarResult = try JPEGLSColorTransformation.hp3.transformForward([r[0], g[0], b[0]]) - #expect(rP[0] == scalarResult[0]) - #expect(gP[0] == scalarResult[1]) - #expect(bP[0] == scalarResult[2]) - } -} - -#endif // canImport(Accelerate) diff --git a/Tests/JPEGLSTests/AppleSiliconMemoryOptimizerTests.swift b/Tests/JPEGLSTests/AppleSiliconMemoryOptimizerTests.swift deleted file mode 100644 index 380100d..0000000 --- a/Tests/JPEGLSTests/AppleSiliconMemoryOptimizerTests.swift +++ /dev/null @@ -1,219 +0,0 @@ -/// Tests for Phase 13.3 Apple Silicon memory architecture optimisation. -/// -/// These tests verify the tile-size tuning, cache-aligned buffer allocation, -/// unified-memory buffer pool, and prefetch helpers added in -/// `AppleSiliconMemoryOptimizer.swift` as part of Milestone 13 Phase 13.3. -/// -/// All tests are compiled and run only on ARM64 architectures. - -#if arch(arm64) - -import Testing -import Foundation -@testable import JPEGLS - -@Suite("Apple Silicon Memory Optimizer Tests") -struct AppleSiliconMemoryOptimizerTests { - - // MARK: - Cache Parameters - - @Test("L1 cache size is a positive power of two") - func l1CacheSizePositive() { - let size = AppleSiliconCacheParameters.l1DataCacheSize - #expect(size > 0) - #expect((size & (size - 1)) == 0, "L1 cache size should be a power of two") - } - - @Test("Cache line size is 64 bytes on ARM64") - func cacheLineSizeIs64() { - #expect(AppleSiliconCacheParameters.cacheLineSize == 64) - } - - @Test("L2 cache is larger than L1") - func l2LargerThanL1() { - #expect(AppleSiliconCacheParameters.l2CacheSize > AppleSiliconCacheParameters.l1DataCacheSize) - } - - @Test("L3 cache is larger than L2") - func l3LargerThanL2() { - #expect(AppleSiliconCacheParameters.l3CacheSize > AppleSiliconCacheParameters.l2CacheSize) - } - - // MARK: - Tile Size Tuning - - @Test("optimalTileSize returns positive dimensions") - func tileSizePositive() { - let (tw, th) = optimalTileSize(imageWidth: 1920, imageHeight: 1080, bytesPerSample: 1) - #expect(tw > 0) - #expect(th > 0) - } - - @Test("optimalTileSize width does not exceed image width") - func tileSizeWidthBounded() { - let (tw, _) = optimalTileSize(imageWidth: 100, imageHeight: 100, bytesPerSample: 1) - #expect(tw <= 100) - } - - @Test("optimalTileSize height does not exceed image height") - func tileSizeHeightBounded() { - let (_, th) = optimalTileSize(imageWidth: 1920, imageHeight: 4, bytesPerSample: 1) - #expect(th <= 4) - } - - @Test("optimalTileSize for 1×1 image returns 1×1") - func tileSizeOneByOne() { - let (tw, th) = optimalTileSize(imageWidth: 1, imageHeight: 1, bytesPerSample: 1) - #expect(tw == 1) - #expect(th == 1) - } - - @Test("optimalTileSize tile width is aligned to cache-line boundary in samples") - func tileSizeAligned() { - let bytesPerSample = 1 - let samplesPerCacheLine = AppleSiliconCacheParameters.cacheLineSize / bytesPerSample - let (tw, _) = optimalTileSize(imageWidth: 3840, imageHeight: 2160, bytesPerSample: bytesPerSample) - #expect(tw % samplesPerCacheLine == 0 || tw <= samplesPerCacheLine, - "Tile width should be aligned to cache-line boundary") - } - - @Test("optimalTileSize for 16-bit samples returns smaller tile height than 8-bit") - func tileSizeSmallerFor16Bit() { - let (_, th8) = optimalTileSize(imageWidth: 3840, imageHeight: 2160, bytesPerSample: 1) - let (_, th16) = optimalTileSize(imageWidth: 3840, imageHeight: 2160, bytesPerSample: 2) - // 16-bit rows are twice as wide, so fewer rows fit in cache - #expect(th16 <= th8) - } - - // MARK: - Cache-Aligned Buffer Allocation - - @Test("allocateCacheAlignedContextArray returns correct minimum size") - func cacheAlignedMinSize() { - let count = 365 - let buf = allocateCacheAlignedContextArray(count: count) - #expect(buf.count >= count) - } - - @Test("allocateCacheAlignedContextArray size is a multiple of cache-line stride") - func cacheAlignedMultiple() { - let stride = AppleSiliconCacheParameters.cacheLineSize / MemoryLayout.stride - for count in [1, 100, 365, 384, 1000] { - let buf = allocateCacheAlignedContextArray(count: count) - #expect(buf.count % stride == 0, - "Allocated count \(buf.count) should be a multiple of \(stride) for count \(count)") - } - } - - @Test("allocateCacheAlignedContextArray is zero-initialised") - func cacheAlignedZeroInit() { - let buf = allocateCacheAlignedContextArray(count: 100) - #expect(buf.allSatisfy { $0 == 0 }) - } - - // MARK: - Unified Memory Buffer Pool - - @Test("UnifiedMemoryBufferPool acquire returns correct buffer size") - func poolAcquireSize() { - let pool = UnifiedMemoryBufferPool(bufferSize: 1024, poolCapacity: 2) - let buf = pool.acquire() - #expect(buf.count == 1024) - } - - @Test("UnifiedMemoryBufferPool release and re-acquire reuses buffer") - func poolReuseBuffer() { - let pool = UnifiedMemoryBufferPool(bufferSize: 512, poolCapacity: 2) - let buf = pool.acquire() - pool.release(buf) - #expect(pool.availableCount == 1) - let buf2 = pool.acquire() - #expect(buf2.count == 512) - #expect(pool.availableCount == 0) - } - - @Test("UnifiedMemoryBufferPool respects poolCapacity limit") - func poolCapacityLimit() { - let pool = UnifiedMemoryBufferPool(bufferSize: 128, poolCapacity: 2) - pool.release(Data(count: 128)) - pool.release(Data(count: 128)) - pool.release(Data(count: 128)) // Third release should be discarded - #expect(pool.availableCount == 2) - } - - @Test("UnifiedMemoryBufferPool prewarm fills pool to capacity") - func poolPrewarm() { - let capacity = 3 - let pool = UnifiedMemoryBufferPool(bufferSize: 64, poolCapacity: capacity) - pool.prewarm() - #expect(pool.availableCount == capacity) - } - - @Test("UnifiedMemoryBufferPool allocates new buffer when empty") - func poolAllocatesNewBuffer() { - let pool = UnifiedMemoryBufferPool(bufferSize: 256, poolCapacity: 2) - // Pool is empty; acquire should still return a valid buffer - let buf = pool.acquire() - #expect(buf.count == 256) - #expect(pool.availableCount == 0) - } - - // MARK: - Memory-Mapped I/O - - @Test("memoryMappedData reads a file correctly") - func memoryMappedReadWrite() throws { - // Write a temporary file and read it back via memory mapping - let tmp = URL(fileURLWithPath: NSTemporaryDirectory()) - .appendingPathComponent("jlswift_mmap_test_\(Int.random(in: 0.. 0) - #expect(AppleSiliconTuningParameters.metalGpuThreshold > 0) - #expect(AppleSiliconTuningParameters.stripHeight > 0) - #expect(AppleSiliconTuningParameters.contextArrayCount > 0) - } - - @Test("AppleSiliconTuningParameters contextArrayCount >= 367") - func tuningContextArrayCount() { - // Must cover at least 365 regular contexts + 2 run-interruption contexts - #expect(AppleSiliconTuningParameters.contextArrayCount >= 367) - } - - @Test("AppleSiliconTuningParameters contextArrayCount is aligned to cache-line boundary") - func tuningContextArrayCountAligned() { - let stride = AppleSiliconCacheParameters.cacheLineSize / MemoryLayout.stride - #expect(AppleSiliconTuningParameters.contextArrayCount % stride == 0) - } - - // MARK: - Prefetch Hints - - @Test("prefetchContextArray does not crash with valid index") - func prefetchValidIndex() { - let arr = [Int](repeating: 42, count: 100) - // Just verify it doesn't crash - prefetchContextArray(arr, startIndex: 0, count: 64) - prefetchContextArray(arr, startIndex: 50, count: 64) - } - - @Test("prefetchContextArray does not crash with out-of-bounds start") - func prefetchOutOfBoundsStart() { - let arr = [Int](repeating: 0, count: 10) - // Should be a no-op, not a crash - prefetchContextArray(arr, startIndex: 100, count: 10) - } - - @Test("prefetchContextArray does not crash with empty array") - func prefetchEmptyArray() { - prefetchContextArray([], startIndex: 0, count: 10) - } -} - -#endif // arch(arm64) diff --git a/Tests/JPEGLSTests/EdgeCasesTests.swift b/Tests/JPEGLSTests/EdgeCasesTests.swift index 86249e1..a7f819f 100644 --- a/Tests/JPEGLSTests/EdgeCasesTests.swift +++ b/Tests/JPEGLSTests/EdgeCasesTests.swift @@ -404,131 +404,4 @@ struct EdgeCasesTests { } } - // MARK: - Buffer Pool Edge Cases - - @Test("Buffer pool with zero-size buffer") - func testBufferPoolZeroSize() { - let pool = JPEGLSBufferPool() - let buffer = pool.acquire(type: .contextArrays, size: 0) - #expect(buffer.isEmpty) - } - - @Test("Buffer pool with very large buffer") - func testBufferPoolVeryLargeBuffer() { - let pool = JPEGLSBufferPool() - let size = 10_000_000 // 10 million elements - let buffer = pool.acquire(type: .pixelData, size: size) - #expect(buffer.count == size) - pool.release(buffer, type: .pixelData) - } - - @Test("Buffer pool cleanup") - func testBufferPoolCleanup() { - let pool = JPEGLSBufferPool(maxPoolSize: 2, bufferLifetime: 0.1) - - // Acquire and release buffers - let buffer1 = pool.acquire(type: .contextArrays, size: 100) - pool.release(buffer1, type: .contextArrays) - - let buffer2 = pool.acquire(type: .contextArrays, size: 200) - pool.release(buffer2, type: .contextArrays) - - // Cleanup should work without errors - pool.cleanup() - } - - // MARK: - Tile Processor Edge Cases - - @Test("Tile processor with single tile (tile larger than image)") - func testTileProcessorSingleTile() { - let processor = JPEGLSTileProcessor( - imageWidth: 100, - imageHeight: 100, - configuration: TileConfiguration(tileWidth: 200, tileHeight: 200, overlap: 0) - ) - let tiles = processor.calculateTiles() - #expect(tiles.count == 1) - } - - @Test("Tile processor with minimum image size (1x1)") - func testTileProcessorMinimumImageSize() { - let processor = JPEGLSTileProcessor( - imageWidth: 1, - imageHeight: 1, - configuration: TileConfiguration(tileWidth: 1, tileHeight: 1, overlap: 0) - ) - let tiles = processor.calculateTiles() - #expect(tiles.count == 1) - #expect(tiles[0].width == 1) - #expect(tiles[0].height == 1) - } - - @Test("Tile processor with maximum overlap") - func testTileProcessorMaximumOverlap() { - let processor = JPEGLSTileProcessor( - imageWidth: 100, - imageHeight: 100, - configuration: TileConfiguration(tileWidth: 50, tileHeight: 50, overlap: 25) - ) - let tiles = processor.calculateTilesWithOverlap() - // With large overlap, there should be more tiles - #expect(tiles.count >= 4) - } - - @Test("Tile processor memory savings calculation") - func testTileProcessorMemorySavings() { - let processor = JPEGLSTileProcessor( - imageWidth: 4096, - imageHeight: 4096, - configuration: TileConfiguration(tileWidth: 512, tileHeight: 512, overlap: 0) - ) - let savings = processor.estimateMemorySavings(bytesPerPixel: 2) - // Should have positive memory savings - #expect(savings > 0) - #expect(savings < 1.0) // Can't save more than 100% - } - - // MARK: - Cache-Friendly Buffer Edge Cases - - @Test("Cache-friendly buffer with single pixel") - func testCacheFriendlyBufferSinglePixel() { - let pixelData: [UInt8: [[Int]]] = [ - 1: [[42]] - ] - let buffer = JPEGLSCacheFriendlyBuffer(pixelData: pixelData, width: 1, height: 1) - let pixel = buffer.getPixel(componentId: 1, row: 0, column: 0) - #expect(pixel == 42) - } - - @Test("Cache-friendly buffer boundary access") - func testCacheFriendlyBufferBoundaryAccess() { - let pixelData: [UInt8: [[Int]]] = [ - 1: (0..<10).map { y in - (0..<10).map { x in - x + y * 10 - } - } - ] - let buffer = JPEGLSCacheFriendlyBuffer(pixelData: pixelData, width: 10, height: 10) - - // Test corners - #expect(buffer.getPixel(componentId: 1, row: 0, column: 0) == 0) - #expect(buffer.getPixel(componentId: 1, row: 0, column: 9) == 9) - #expect(buffer.getPixel(componentId: 1, row: 9, column: 0) == 90) - #expect(buffer.getPixel(componentId: 1, row: 9, column: 9) == 99) - } - - @Test("Cache-friendly buffer with multiple components") - func testCacheFriendlyBufferMultipleComponents() { - let pixelData: [UInt8: [[Int]]] = [ - 1: [[100]], - 2: [[200]], - 3: [[300]] - ] - let buffer = JPEGLSCacheFriendlyBuffer(pixelData: pixelData, width: 1, height: 1) - - #expect(buffer.getPixel(componentId: 1, row: 0, column: 0) == 100) - #expect(buffer.getPixel(componentId: 2, row: 0, column: 0) == 200) - #expect(buffer.getPixel(componentId: 3, row: 0, column: 0) == 300) - } } diff --git a/Tests/JPEGLSTests/GPUComputePhase15ExtendedTests.swift b/Tests/JPEGLSTests/GPUComputePhase15ExtendedTests.swift deleted file mode 100644 index a5017db..0000000 --- a/Tests/JPEGLSTests/GPUComputePhase15ExtendedTests.swift +++ /dev/null @@ -1,695 +0,0 @@ -/// Extended Phase 15.3 GPU Compute tests. -/// -/// Validates GPU compute operations (via Vulkan CPU fallback on all platforms) -/// across the full configuration matrix required by Milestone 15: -/// -/// - All image sizes (tiny, small, medium, large, above GPU threshold) -/// - All relevant bit depths (8-bit, 12-bit, 16-bit pixel value ranges) -/// - Greyscale (single component) and RGB (three component) configurations -/// - Near-lossless encoding modes (NEAR > 0) -/// -/// Metal variants are exercised on Apple platforms only (skipped elsewhere). - -import Testing -@testable import JPEGLS - -// MARK: - Image-Size Matrix Tests - -@Suite("GPU Pipeline — Image Size Matrix (Vulkan CPU Fallback)") -struct GPUImageSizeMatrixTests { - - let accelerator = VulkanAccelerator() - - // MARK: Gradient computation — all sizes - - @Test("Gradients: tiny batch (1 pixel)") - func testGradients1Pixel() { - let (d1, d2, d3) = accelerator.computeGradientsBatch( - a: [100], b: [150], c: [80]) - #expect(d1 == [70]) // 150 - 80 - #expect(d2 == [20]) // 100 - 80 - #expect(d3 == [-20]) // 80 - 100 - } - - @Test("Gradients: small batch (16 pixels)") - func testGradients16Pixels() { - let count = 16 - let a = [Int32](repeating: 200, count: count) - let b = [Int32](repeating: 100, count: count) - let c = [Int32](repeating: 50, count: count) - let (d1, d2, d3) = accelerator.computeGradientsBatch(a: a, b: b, c: c) - #expect(d1.allSatisfy { $0 == 50 }) // 100 - 50 - #expect(d2.allSatisfy { $0 == 150 }) // 200 - 50 - #expect(d3.allSatisfy { $0 == -150 }) // 50 - 200 - } - - @Test("Gradients: medium batch (256 pixels)") - func testGradients256Pixels() { - let count = 256 - let a = [Int32](repeating: 128, count: count) - let b = [Int32](repeating: 200, count: count) - let c = [Int32](repeating: 100, count: count) - let (d1, d2, d3) = accelerator.computeGradientsBatch(a: a, b: b, c: c) - #expect(d1.allSatisfy { $0 == 100 }) - #expect(d2.allSatisfy { $0 == 28 }) - #expect(d3.allSatisfy { $0 == -28 }) - } - - @Test("Gradients: batch just below GPU threshold") - func testGradientsBelowThreshold() { - let count = VulkanAccelerator.gpuThreshold - 1 - let a = [Int32](repeating: 50, count: count) - let b = [Int32](repeating: 100, count: count) - let c = [Int32](repeating: 25, count: count) - let (d1, d2, d3) = accelerator.computeGradientsBatch(a: a, b: b, c: c) - #expect(d1.count == count) - #expect(d1.allSatisfy { $0 == 75 }) - #expect(d2.allSatisfy { $0 == 25 }) - #expect(d3.allSatisfy { $0 == -25 }) - } - - @Test("Gradients: batch at GPU threshold") - func testGradientsAtThreshold() { - let count = VulkanAccelerator.gpuThreshold - let a = [Int32](repeating: 30, count: count) - let b = [Int32](repeating: 60, count: count) - let c = [Int32](repeating: 15, count: count) - let (d1, d2, d3) = accelerator.computeGradientsBatch(a: a, b: b, c: c) - #expect(d1.allSatisfy { $0 == 45 }) - #expect(d2.allSatisfy { $0 == 15 }) - #expect(d3.allSatisfy { $0 == -15 }) - } - - @Test("Gradients: batch well above GPU threshold (4× threshold)") - func testGradientsLargeBatch() { - let count = VulkanAccelerator.gpuThreshold * 4 - var a = [Int32](repeating: 0, count: count) - var b = [Int32](repeating: 0, count: count) - var c = [Int32](repeating: 0, count: count) - for i in 0..= max, c <= min, c > max, between - let pred = accelerator.computeMEDPredictionBatch(a: a, b: b, c: c) - // Case 0: c(255) >= max(0,255)=255 → min(0,255)=0 - #expect(pred[0] == 0) - // Case 1: c(0) <= min(255,0)=0 → max(255,0)=255 - #expect(pred[1] == 255) - // Case 2: c(255) >= max(128,200)=200 → min(128,200)=128 - #expect(pred[2] == 128) - // Case 3: c(80) is between min(100,50)=50 and max(100,50)=100 → a+b-c=70 - #expect(pred[3] == 70) - } - - @Test("Gradient quantisation: 8-bit, standard JPEG-LS thresholds") - func testQuantize8BitStandardThresholds() { - // Default 8-bit JPEG-LS thresholds: T1=3, T2=7, T3=21 - let t1: Int32 = 3, t2: Int32 = 7, t3: Int32 = 21 - let gradients: [Int32] = [-30, -21, -7, -3, -1, 0, 1, 3, 7, 21, 30] - let expected: [Int32] = [ -4, -4, -3, -2, -1, 0, 1, 2, 3, 4, 4] - let (q1, _, _) = accelerator.quantizeGradientsBatch( - d1: gradients, d2: gradients, d3: gradients, t1: t1, t2: t2, t3: t3) - #expect(q1 == expected) - } - - // MARK: 12-bit pixel values (0–4095) - - @Test("Gradients: 12-bit pixel range") - func testGradients12Bit() { - let count = 64 - var a = [Int32](repeating: 0, count: count) - var b = [Int32](repeating: 0, count: count) - var c = [Int32](repeating: 0, count: count) - for i in 0..= max(n, w) { expected = min(n, w) } - else if nwVal <= min(n, w) { expected = max(n, w) } - else { expected = n + w - nwVal } - #expect(pred[i] == expected) - } - } - - // MARK: Three components (RGB) - - @Test("RGB (3-component): forward and inverse HP1 round-trip") - func testRGBHP1RoundTrip() { - // Simulate a 4-pixel RGB image - let r: [Int32] = [200, 150, 100, 50] - let g: [Int32] = [100, 120, 80, 30] - let b: [Int32] = [ 50, 60, 200, 180] - - let (rp, gp, bp) = accelerator.applyColourTransformForwardBatch( - transform: .hp1, r: r, g: g, b: b) - let (rr, gr, br) = accelerator.applyColourTransformInverseBatch( - transform: .hp1, r: rp, g: gp, b: bp) - - #expect(rr == r) - #expect(gr == g) - #expect(br == b) - } - - @Test("RGB (3-component): forward and inverse HP2 round-trip") - func testRGBHP2RoundTrip() { - let r: [Int32] = [200, 150, 100, 50] - let g: [Int32] = [100, 120, 80, 30] - let b: [Int32] = [ 50, 60, 200, 180] - - let (rp, gp, bp) = accelerator.applyColourTransformForwardBatch( - transform: .hp2, r: r, g: g, b: b) - let (rr, gr, br) = accelerator.applyColourTransformInverseBatch( - transform: .hp2, r: rp, g: gp, b: bp) - - #expect(rr == r) - #expect(gr == g) - #expect(br == b) - } - - @Test("RGB (3-component): forward and inverse HP3 round-trip") - func testRGBHP3RoundTrip() { - let r: [Int32] = [200, 150, 100, 50] - let g: [Int32] = [100, 120, 80, 30] - let b: [Int32] = [ 50, 60, 200, 180] - - let (rp, gp, bp) = accelerator.applyColourTransformForwardBatch( - transform: .hp3, r: r, g: g, b: b) - let (rr, gr, br) = accelerator.applyColourTransformInverseBatch( - transform: .hp3, r: rp, g: gp, b: bp) - - #expect(rr == r) - #expect(gr == g) - #expect(br == b) - } - - @Test("RGB (3-component): large batch HP2 round-trip (above threshold)") - func testRGBHP2LargeBatchRoundTrip() { - let count = VulkanAccelerator.gpuThreshold * 2 - var r = [Int32](repeating: 0, count: count) - var g = [Int32](repeating: 0, count: count) - var b = [Int32](repeating: 0, count: count) - for i in 0.. 0 - - @Test("Quantisation: NEAR=0 matches standard lossless boundary") - func testQuantizeNear0() { - // With NEAR=0, d=0 → 0, d=-1 → -1, d=1 → 1 (standard lossless) - let t1: Int32 = 3, t2: Int32 = 7, t3: Int32 = 21 - let d: [Int32] = [-1, 0, 1] - let expected: [Int32] = [-1, 0, 1] - let (q1, _, _) = accelerator.quantizeGradientsBatch( - d1: d, d2: d, d3: d, t1: t1, t2: t2, t3: t3) - #expect(q1 == expected) - } - - @Test("Vulkan: MED prediction unchanged by NEAR parameter (predicts same as lossless)") - func testMEDPredictionIndependentOfNear() { - // MED prediction does not depend on NEAR — it always predicts the same value. - // This test verifies that the GPU pipeline's prediction matches CPU. - let a: [Int32] = [100, 200, 150] - let b: [Int32] = [110, 190, 160] - let c: [Int32] = [105, 205, 145] - - let pred = accelerator.computeMEDPredictionBatch(a: a, b: b, c: c) - - // Manually compute expected values - for i in 0..= max(av, bv) { expected = min(av, bv) } - else if cv <= min(av, bv) { expected = max(av, bv) } - else { expected = av + bv - cv } - #expect(pred[i] == expected) - } - } - - @Test("Vulkan: quantiseGradients NEAR=3 — values within NEAR range → 0 when T1 > NEAR") - func testQuantizeNear3SmallValues() { - // When T1 > NEAR, values in [-NEAR, NEAR] map to 0. - // Use T1=5, T2=9, T3=25 (so T1 > NEAR=3) to demonstrate the NEAR range. - let near = 3 - let t1 = 5, t2 = 9, t3 = 25 - for d in -near...near { - // NEAR-aware quantisation: values in [-near, near] → 0 when T1 > near - let result: Int - if d <= -t3 { result = -4 } - else if d <= -t2 { result = -3 } - else if d <= -t1 { result = -2 } - else if d < -near { result = -1 } - else if d <= near { result = 0 } - else if d < t1 { result = 1 } - else if d < t2 { result = 2 } - else if d < t3 { result = 3 } - else { result = 4 } - #expect(result == 0, "d=\(d) with NEAR=\(near) and T1=\(t1) should map to 0") - } - } - - @Test("Vulkan: quantiseGradients — T1=NEAR boundary: d=-T1 maps to -2, not 0") - func testQuantizeNearEqualT1Boundary() { - // When NEAR == T1, the threshold check `d <= -T1` takes precedence over - // the NEAR check. So d = -T1 maps to -2, not 0. - let near = 3 - let t1 = 3, t2 = 7, t3 = 21 - - // d = -3 = -T1: hits `d <= -T1` first → maps to -2 - let dAtNegT1 = -t1 - let result: Int - if dAtNegT1 <= -t3 { result = -4 } - else if dAtNegT1 <= -t2 { result = -3 } - else if dAtNegT1 <= -t1 { result = -2 } - else if dAtNegT1 < -near { result = -1 } - else if dAtNegT1 <= near { result = 0 } - else { result = 1 } - #expect(result == -2, "d=\(dAtNegT1) with T1=\(t1) should map to -2 (threshold takes precedence)") - - // d = -2 (strictly inside -T1): hits NEAR check → maps to 0 - let dInsideNear = -2 - let result2: Int - if dInsideNear <= -t3 { result2 = -4 } - else if dInsideNear <= -t2 { result2 = -3 } - else if dInsideNear <= -t1 { result2 = -2 } - else if dInsideNear < -near { result2 = -1 } - else if dInsideNear <= near { result2 = 0 } - else { result2 = 1 } - #expect(result2 == 0, "d=\(dInsideNear) with NEAR=\(near) should map to 0") - } - - @Test("Vulkan: large-batch gradient quantisation is consistent across sizes") - func testQuantizeConsistencyAcrossSizes() { - let t1: Int32 = 3, t2: Int32 = 7, t3: Int32 = 21 - // Compare small-batch vs large-batch quantisation for same data - let smallCount = VulkanAccelerator.gpuThreshold / 2 - let largeCount = VulkanAccelerator.gpuThreshold * 2 - let pattern: [Int32] = [-25, -8, -4, -2, -1, 0, 1, 2, 4, 8, 25] - - // Build arrays by repeating the pattern - let buildArray = { (n: Int) -> [Int32] in - (0.. Int32 in - let err = Int(orig - prediction[i]) - // Round toward zero (floor division toward zero) - return Int32(err >= 0 ? err / qbpp : -((-err) / qbpp)) - } - // Reconstruct: prediction + quantisedError * qbpp - let reconstructed = quantisedErrors.enumerated().map { i, qe -> Int32 in - prediction[i] + qe * Int32(qbpp) - } - // Verify |reconstructed - original| <= NEAR - for i in 0.. NEAR=\(near)") - } - } -} - -// MARK: - Metal Encoding/Decoding Pipeline Tests (Apple platforms only) - -#if canImport(Metal) - -@Suite("Metal Encoding/Decoding Pipeline Tests") -struct MetalEncodingDecodingPipelineTests { - - // MARK: Encoding pipeline — small batch (CPU fallback) - - @Test("Metal: encodingPipeline — small batch lossless (NEAR=0)") - func testEncodingPipelineSmallBatchLossless() throws { - guard MetalAccelerator.isSupported else { return } - let accelerator = try MetalAccelerator() - let a: [Int32] = [100, 200, 50] - let b: [Int32] = [110, 190, 60] - let c: [Int32] = [ 95, 195, 55] - let x: [Int32] = [105, 185, 65] - let (pred, err, _, _, _) = try accelerator.computeEncodingPipelineBatch( - a: a, b: b, c: c, x: x, near: 0, t1: 3, t2: 7, t3: 21) - // Verify prediction + error = x - for i in 0..= -4 && q1[i] <= 4) - #expect(q2[i] >= -4 && q2[i] <= 4) - #expect(q3[i] >= -4 && q3[i] <= 4) - } - } - - @Test("Metal: encodingPipeline — NEAR=3 gradients near zero → q=0") - func testEncodingPipelineNear3GradientsNearZero() throws { - guard MetalAccelerator.isSupported else { return } - let accelerator = try MetalAccelerator() - // b[i]-c[i] = 1, a[i]-c[i] = 0, c[i]-a[i] = 0 → all within NEAR=3 → q=0 - // d1 = b - c = 11 - 10 = 1 (≤ NEAR=3) - // d2 = a - c = 10 - 10 = 0 (≤ NEAR=3) - // d3 = c - a = 10 - 10 = 0 (≤ NEAR=3) - let a: [Int32] = [10] - let b: [Int32] = [11] - let c: [Int32] = [10] // a-c=0, b-c=1, c-a=0 (all ≤ NEAR=3) - let x: [Int32] = [12] - let (_, _, q1, q2, q3) = try accelerator.computeEncodingPipelineBatch( - a: a, b: b, c: c, x: x, near: 3, t1: 3, t2: 7, t3: 21) - // d1=b-c=1 (≤ NEAR=3) → 0, d2=a-c=0 (≤ NEAR) → 0, d3=c-a=0 (≤ NEAR) → 0 - #expect(q1 == [0]) - #expect(q2 == [0]) - #expect(q3 == [0]) - } - - @Test("Metal: encodingPipeline — quantised gradients match NEAR=0 for large gradients") - func testEncodingPipelineQuantisedGradientsLarge() throws { - guard MetalAccelerator.isSupported else { return } - let accelerator = try MetalAccelerator() - // With NEAR=0, the encoding pipeline should produce same quantisation - // as the standalone quantizeGradientsBatch call. - let t1: Int32 = 3, t2: Int32 = 7, t3: Int32 = 21 - let count = 9 - // Use neighbours where d1 = b-c spans all 9 quantisation buckets - let gradientValues: [Int32] = [-30, -10, -5, -1, 0, 1, 5, 10, 30] - // a=c, b=c+gradient (so b-c = gradient, a-c=0, c-a=0) - let c: [Int32] = [Int32](repeating: 50, count: count) - let a: [Int32] = [Int32](repeating: 50, count: count) - let b: [Int32] = gradientValues.map { c[0] + $0 } - let x: [Int32] = [Int32](repeating: 50, count: count) - - let (_, _, q1, _, _) = try accelerator.computeEncodingPipelineBatch( - a: a, b: b, c: c, x: x, near: 0, t1: t1, t2: t2, t3: t3) - let expected: [Int32] = [-4, -3, -2, -1, 0, 1, 2, 3, 4] - #expect(q1 == expected) - } - - @Test("Metal: encodingPipeline — empty arrays") - func testEncodingPipelineEmpty() throws { - guard MetalAccelerator.isSupported else { return } - let accelerator = try MetalAccelerator() - let (pred, err, q1, q2, q3) = try accelerator.computeEncodingPipelineBatch( - a: [], b: [], c: [], x: [], near: 0, t1: 3, t2: 7, t3: 21) - #expect(pred.isEmpty && err.isEmpty && q1.isEmpty && q2.isEmpty && q3.isEmpty) - } - - // MARK: Decoding pipeline — small batch (CPU fallback) - - @Test("Metal: decodingPipeline — reconstructs pixels from errors (small batch)") - func testDecodingPipelineSmallBatch() throws { - guard MetalAccelerator.isSupported else { return } - let accelerator = try MetalAccelerator() - let a: [Int32] = [100, 200, 50] - let b: [Int32] = [110, 190, 60] - let c: [Int32] = [ 95, 195, 55] - let errval: [Int32] = [ 5, -5, 10] - let reconstructed = try accelerator.computeDecodingPipelineBatch( - a: a, b: b, c: c, errval: errval) - // Each reconstructed[i] = MED(a[i],b[i],c[i]) + errval[i] - for i in 0..= max(av, bv) { px = min(av, bv) } - else if cv <= min(av, bv) { px = max(av, bv) } - else { px = av + bv - cv } - #expect(reconstructed[i] == px + errval[i]) - } - } - - @Test("Metal: decodingPipeline — empty arrays") - func testDecodingPipelineEmpty() throws { - guard MetalAccelerator.isSupported else { return } - let accelerator = try MetalAccelerator() - let result = try accelerator.computeDecodingPipelineBatch( - a: [], b: [], c: [], errval: []) - #expect(result.isEmpty) - } - - // MARK: Encode → Decode round-trip - - @Test("Metal: encode → decode round-trip (lossless, small batch)") - func testEncodeDecodeLosslessRoundTripSmall() throws { - guard MetalAccelerator.isSupported else { return } - let accelerator = try MetalAccelerator() - let a: [Int32] = [100, 200, 50, 150] - let b: [Int32] = [110, 190, 60, 140] - let c: [Int32] = [ 95, 195, 55, 145] - let x: [Int32] = [105, 188, 62, 148] - - let (_, err, _, _, _) = try accelerator.computeEncodingPipelineBatch( - a: a, b: b, c: c, x: x, near: 0, t1: 3, t2: 7, t3: 21) - let reconstructed = try accelerator.computeDecodingPipelineBatch( - a: a, b: b, c: c, errval: err) - #expect(reconstructed == x, "Lossless encode-decode must reconstruct exactly") - } - - @Test("Metal: encode → decode round-trip (lossless, large batch, GPU path)") - func testEncodeDecodeLosslessRoundTripLarge() throws { - guard MetalAccelerator.isSupported else { return } - let accelerator = try MetalAccelerator() - let count = MetalAccelerator.gpuThreshold * 2 - var a = [Int32](repeating: 0, count: count) - var b = [Int32](repeating: 0, count: count) - var c = [Int32](repeating: 0, count: count) - var x = [Int32](repeating: 0, count: count) - for i in 0..= max(a,b) → min(a,b)") - func testMEDCaseHigh() { - let a: [Int32] = [10] - let b: [Int32] = [15] - let c: [Int32] = [20] // c >= max - let pred = accelerator.computeMEDPredictionBatch(a: a, b: b, c: c) - #expect(pred == [10]) - } - - @Test("Vulkan: computeMEDPredictionBatch — c <= min(a,b) → max(a,b)") - func testMEDCaseLow() { - let a: [Int32] = [15] - let b: [Int32] = [20] - let c: [Int32] = [5] // c <= min - let pred = accelerator.computeMEDPredictionBatch(a: a, b: b, c: c) - #expect(pred == [20]) - } - - @Test("Vulkan: computeMEDPredictionBatch — middle case → a + b - c") - func testMEDCaseMiddle() { - let a: [Int32] = [10] - let b: [Int32] = [20] - let c: [Int32] = [15] // min < c < max - let pred = accelerator.computeMEDPredictionBatch(a: a, b: b, c: c) - #expect(pred == [15]) - } - - @Test("Vulkan: computeMEDPredictionBatch empty arrays") - func testMEDEmpty() { - let pred = accelerator.computeMEDPredictionBatch(a: [], b: [], c: []) - #expect(pred.isEmpty) - } - - // MARK: Gradient quantisation - - @Test("Vulkan: quantizeGradientsBatch — boundary values map correctly") - func testQuantizeGradientsBasic() { - // t1=3, t2=7, t3=21 — typical JPEG-LS default-like values - let t1: Int32 = 3, t2: Int32 = 7, t3: Int32 = 21 - let d: [Int32] = [-30, -10, -5, -1, 0, 1, 5, 10, 30] - let expected: [Int32] = [-4, -3, -2, -1, 0, 1, 2, 3, 4] - let (q1, _, _) = accelerator.quantizeGradientsBatch( - d1: d, d2: d, d3: d, t1: t1, t2: t2, t3: t3) - #expect(q1 == expected) - } - - @Test("Vulkan: quantizeGradientsBatch — three channels quantised independently") - func testQuantizeGradientsThreeChannels() { - let t1: Int32 = 2, t2: Int32 = 4, t3: Int32 = 8 - let d1: [Int32] = [0, 1, 3] // 0→0, 1→1 (d-t2), -9→-4 (d≤-t3) - let d3: [Int32] = [2, 5, 10] // 2→2 (t1≤d>1) = 50 - 150 = -100 - let r: [Int32] = [200] - let g: [Int32] = [100] - let b: [Int32] = [ 50] - let (rp, gp, bp) = accelerator.applyColourTransformForwardBatch( - transform: .hp2, r: r, g: g, b: b) - #expect(rp == [100]) - #expect(gp == [100]) - #expect(bp == [-100]) - } - - @Test("Vulkan: HP2 inverse round-trips correctly") - func testHP2RoundTrip() { - let r: [Int32] = [200, 100, 30] - let g: [Int32] = [100, 50, 20] - let b: [Int32] = [ 50, 150, 10] - let (rp, gp, bp) = accelerator.applyColourTransformForwardBatch( - transform: .hp2, r: r, g: g, b: b) - let (rr, gr, br) = accelerator.applyColourTransformInverseBatch( - transform: .hp2, r: rp, g: gp, b: bp) - #expect(rr == r) - #expect(gr == g) - #expect(br == b) - } - - // MARK: Colour transforms — HP3 - - @Test("Vulkan: HP3 forward transform is correct") - func testHP3Forward() { - // R=200, G=100, B=50 - // B′ = 50 - // R′ = 200-50 = 150 - // G′ = 100 - ((200+50)>>1) = 100 - 125 = -25 - let r: [Int32] = [200] - let g: [Int32] = [100] - let b: [Int32] = [ 50] - let (rp, gp, bp) = accelerator.applyColourTransformForwardBatch( - transform: .hp3, r: r, g: g, b: b) - #expect(rp == [150]) - #expect(gp == [-25]) - #expect(bp == [50]) - } - - @Test("Vulkan: HP3 inverse round-trips correctly") - func testHP3RoundTrip() { - let r: [Int32] = [200, 100, 30] - let g: [Int32] = [100, 50, 20] - let b: [Int32] = [ 50, 150, 10] - let (rp, gp, bp) = accelerator.applyColourTransformForwardBatch( - transform: .hp3, r: r, g: g, b: b) - let (rr, gr, br) = accelerator.applyColourTransformInverseBatch( - transform: .hp3, r: rp, g: gp, b: bp) - #expect(rr == r) - #expect(gr == g) - #expect(br == b) - } - - // MARK: Identity transform - - @Test("Vulkan: .none transform returns input unchanged") - func testIdentityTransform() { - let r: [Int32] = [10, 20, 30] - let g: [Int32] = [40, 50, 60] - let b: [Int32] = [70, 80, 90] - let (rp, gp, bp) = accelerator.applyColourTransformForwardBatch( - transform: .none, r: r, g: g, b: b) - #expect(rp == r && gp == g && bp == b) - } - - // MARK: Encoding pipeline - - @Test("Vulkan: computeEncodingPipelineBatch — lossless prediction+error=x") - func testEncodingPipelineLossless() throws { - let a: [Int32] = [100, 200, 50] - let b: [Int32] = [110, 190, 60] - let c: [Int32] = [ 95, 195, 55] - let x: [Int32] = [105, 185, 65] - let (pred, err, _, _, _) = try accelerator.computeEncodingPipelineBatch( - a: a, b: b, c: c, x: x, near: 0, t1: 3, t2: 7, t3: 21) - for i in 0..= -4 && q1[i] <= 4) - #expect(q2[i] >= -4 && q2[i] <= 4) - #expect(q3[i] >= -4 && q3[i] <= 4) - } - } - - @Test("Vulkan: computeEncodingPipelineBatch — empty arrays") - func testEncodingPipelineEmpty() throws { - let (pred, err, q1, q2, q3) = try accelerator.computeEncodingPipelineBatch( - a: [], b: [], c: [], x: [], near: 0, t1: 3, t2: 7, t3: 21) - #expect(pred.isEmpty && err.isEmpty && q1.isEmpty && q2.isEmpty && q3.isEmpty) - } - - @Test("Vulkan: computeEncodingPipelineBatch — NEAR=3 gradients within near → q=0") - func testEncodingPipelineNear3ZeroGradients() throws { - // d1 = b-c = 1 (≤ NEAR=3), d2 = a-c = 0, d3 = c-a = 0 → all q=0 - let a: [Int32] = [10] - let b: [Int32] = [11] - let c: [Int32] = [10] - let x: [Int32] = [12] - let (_, _, q1, q2, q3) = try accelerator.computeEncodingPipelineBatch( - a: a, b: b, c: c, x: x, near: 3, t1: 3, t2: 7, t3: 21) - #expect(q1 == [0]) - #expect(q2 == [0]) - #expect(q3 == [0]) - } - - @Test("Vulkan: computeEncodingPipelineBatch — mismatched array lengths throws") - func testEncodingPipelineMismatchedLengths() { - #expect(throws: VulkanAcceleratorError.inputLengthMismatch) { - try accelerator.computeEncodingPipelineBatch( - a: [1, 2], b: [3], c: [4, 5], x: [6, 7], - near: 0, t1: 3, t2: 7, t3: 21) - } - } - - @Test("Vulkan: computeEncodingPipelineBatch matches standalone gradient+MED+quantise") - func testEncodingPipelineMatchesStandaloneOps() throws { - let count = 32 - var a = [Int32](repeating: 0, count: count) - var b = [Int32](repeating: 0, count: count) - var c = [Int32](repeating: 0, count: count) - var x = [Int32](repeating: 0, count: count) - for i in 0..= max(av, bv) { px = min(av, bv) } - else if cv <= min(av, bv) { px = max(av, bv) } - else { px = av + bv - cv } - #expect(reconstructed[i] == px + errval[i]) - } - } - - @Test("Vulkan: computeDecodingPipelineBatch — empty arrays") - func testDecodingPipelineEmpty() throws { - let result = try accelerator.computeDecodingPipelineBatch( - a: [], b: [], c: [], errval: []) - #expect(result.isEmpty) - } - - @Test("Vulkan: computeDecodingPipelineBatch — mismatched array lengths throws") - func testDecodingPipelineMismatchedLengths() { - #expect(throws: VulkanAcceleratorError.inputLengthMismatch) { - try accelerator.computeDecodingPipelineBatch( - a: [1, 2], b: [3, 4], c: [5], errval: [6, 7]) - } - } - - // MARK: Encode → Decode round-trip - - @Test("Vulkan: encode→decode round-trip (lossless, small batch)") - func testEncodeDecodeLosslessRoundTripSmall() throws { - let a: [Int32] = [100, 200, 50, 150] - let b: [Int32] = [110, 190, 60, 140] - let c: [Int32] = [ 95, 195, 55, 145] - let x: [Int32] = [105, 188, 62, 148] - - let (_, err, _, _, _) = try accelerator.computeEncodingPipelineBatch( - a: a, b: b, c: c, x: x, near: 0, t1: 3, t2: 7, t3: 21) - let reconstructed = try accelerator.computeDecodingPipelineBatch( - a: a, b: b, c: c, errval: err) - #expect(reconstructed == x, "Lossless encode→decode must reconstruct exactly") - } - - @Test("Vulkan: encode→decode round-trip (lossless, large batch, above threshold)") - func testEncodeDecodeLosslessRoundTripLarge() throws { - let count = VulkanAccelerator.gpuThreshold * 2 - var a = [Int32](repeating: 0, count: count) - var b = [Int32](repeating: 0, count: count) - var c = [Int32](repeating: 0, count: count) - var x = [Int32](repeating: 0, count: count) - for i in 0.. -t3) - let d3 = [Int32](repeating: 0, count: count) // maps to 0 - let (q1, q2, q3) = try accelerator.quantizeGradientsBatch( - d1: d1, d2: d2, d3: d3, t1: t1, t2: t2, t3: t3) - #expect(q1.allSatisfy { $0 == 2 }) - #expect(q2.allSatisfy { $0 == -3 }) - #expect(q3.allSatisfy { $0 == 0 }) - } - - @Test("Metal: quantizeGradientsBatch — empty arrays") - func testQuantizeGradientsEmpty() throws { - guard MetalAccelerator.isSupported else { return } - let accelerator = try MetalAccelerator() - let (q1, q2, q3) = try accelerator.quantizeGradientsBatch( - d1: [], d2: [], d3: [], t1: 3, t2: 7, t3: 21) - #expect(q1.isEmpty && q2.isEmpty && q3.isEmpty) - } - - @Test("Metal: quantizeGradientsBatch matches Vulkan CPU fallback") - func testQuantizeGradientsBitExact() throws { - guard MetalAccelerator.isSupported else { return } - let metalAcc = try MetalAccelerator() - let vulkanAcc = VulkanAccelerator() - let t1: Int32 = 3, t2: Int32 = 7, t3: Int32 = 21 - let count = MetalAccelerator.gpuThreshold * 2 - var d1 = [Int32](repeating: 0, count: count) - var d2 = [Int32](repeating: 0, count: count) - var d3 = [Int32](repeating: 0, count: count) - for i in 0..= 1) - #expect(th >= 1) - #expect(tw <= 128) - #expect(th <= 128) - } - - @Test("Tile size fits within L1 budget for large RGB 16-bit image") - func tileSizeLargeRGB16() { - let (tw, th) = intelOptimalTileSize( - imageWidth: 3840, imageHeight: 2160, - bytesPerSample: 2, componentCount: 3 - ) - #expect(tw >= 1) - #expect(th >= 1) - #expect(tw <= 3840) - #expect(th <= 2160) - } - - @Test("Tile width is cache-line aligned") - func tileWidthCacheLineAligned() { - let (tw, _) = intelOptimalTileSize(imageWidth: 1000, imageHeight: 500) - let samplesPerCacheLine = IntelCacheParameters.cacheLineSize / 1 // 1 byte per sample - #expect(tw % samplesPerCacheLine == 0 || tw == 1000) - } - - @Test("Tile height never exceeds image height") - func tileHeightBound() { - let (_, th) = intelOptimalTileSize(imageWidth: 16, imageHeight: 4) - #expect(th <= 4) - } - - // MARK: - Cache-Aligned Buffer Allocation - - @Test("Allocated context array has at least requested count") - func cacheAlignedArraySize() { - let arr = intelAllocateCacheAlignedContextArray(count: 365) - #expect(arr.count >= 365) - } - - @Test("Allocated context array count is multiple of cache-line stride") - func cacheAlignedArrayAlignment() { - let arr = intelAllocateCacheAlignedContextArray(count: 365) - let stride = IntelCacheParameters.cacheLineSize / MemoryLayout.stride - #expect(arr.count % stride == 0) - } - - @Test("Allocated context array is zero-initialised") - func cacheAlignedArrayZeroed() { - let arr = intelAllocateCacheAlignedContextArray(count: 100) - #expect(arr.allSatisfy { $0 == 0 }) - } - - // MARK: - Buffer Pool - - @Test("Buffer pool acquire returns correct-size buffer") - func poolAcquireSize() { - let pool = IntelBufferPool(bufferSize: 1024) - let buf = pool.acquire() - #expect(buf.count == 1024) - } - - @Test("Buffer pool release and reuse") - func poolReleaseReuse() { - let pool = IntelBufferPool(bufferSize: 512, poolCapacity: 2) - let buf1 = pool.acquire() - pool.release(buf1) - #expect(pool.availableCount == 1) - let _ = pool.acquire() - #expect(pool.availableCount == 0) - } - - @Test("Buffer pool discards buffers beyond capacity") - func poolCapacityLimit() { - let pool = IntelBufferPool(bufferSize: 256, poolCapacity: 2) - let b1 = pool.acquire() - let b2 = pool.acquire() - let b3 = pool.acquire() - pool.release(b1) - pool.release(b2) - pool.release(b3) // exceeds capacity - #expect(pool.availableCount == 2) - } - - @Test("Buffer pool prewarm fills to capacity") - func poolPrewarm() { - let pool = IntelBufferPool(bufferSize: 128, poolCapacity: 4) - pool.prewarm() - #expect(pool.availableCount == 4) - } - - // MARK: - Prefetch Hint - - @Test("Prefetch does not crash on valid range") - func prefetchValid() { - let arr = [Int](repeating: 1, count: 128) - // Should not crash - intelPrefetchContextArray(arr, startIndex: 0, count: 128) - } - - @Test("Prefetch handles empty range gracefully") - func prefetchEmptyRange() { - let arr = [Int](repeating: 0, count: 10) - // startIndex == end → no-op - intelPrefetchContextArray(arr, startIndex: 10, count: 0) - } - - // MARK: - Tuning Parameters - - @Test("Intel tuning parameters have expected values") - func tuningParameterValues() { - #expect(IntelTuningParameters.recommendedReset == 64) - #expect(IntelTuningParameters.stripHeight == IntelCacheParameters.recommendedStripHeight) - #expect(IntelTuningParameters.contextArrayCount == 384) - } - - // MARK: - Memory-Mapped I/O - - @Test("Memory-mapped read of temp file succeeds") - func memoryMappedRead() throws { - let tempDir = FileManager.default.temporaryDirectory - let url = tempDir.appendingPathComponent("intel_mmap_test.bin") - let testData = Data([0x01, 0x02, 0x03, 0x04]) - try testData.write(to: url) - defer { try? FileManager.default.removeItem(at: url) } - - let mapped = try intelMemoryMappedData(at: url) - #expect(mapped == testData) - } - - @Test("Memory-mapped write of temp file succeeds") - func memoryMappedWrite() throws { - let tempDir = FileManager.default.temporaryDirectory - let url = tempDir.appendingPathComponent("intel_mmap_write_test.bin") - let testData = Data([0xAA, 0xBB, 0xCC]) - try intelWriteMemoryMapped(testData, to: url) - defer { try? FileManager.default.removeItem(at: url) } - - let read = try Data(contentsOf: url) - #expect(read == testData) - } -} - -#endif // arch(x86_64) diff --git a/Tests/JPEGLSTests/JPEGLSBufferPoolTests.swift b/Tests/JPEGLSTests/JPEGLSBufferPoolTests.swift deleted file mode 100644 index e5e5013..0000000 --- a/Tests/JPEGLSTests/JPEGLSBufferPoolTests.swift +++ /dev/null @@ -1,156 +0,0 @@ -/// Tests for JPEG-LS buffer pool -import Testing -@testable import JPEGLS - -@Suite("JPEG-LS Buffer Pool Tests") -struct JPEGLSBufferPoolTests { - - @Test("Buffer pool acquires new buffer") - func testAcquireNewBuffer() { - let pool = JPEGLSBufferPool() - let buffer = pool.acquire(type: .contextArrays, size: 100) - - #expect(buffer.count == 100) - #expect(buffer.allSatisfy { $0 == 0 }) - } - - @Test("Buffer pool reuses released buffers") - func testReuseBuffer() { - let pool = JPEGLSBufferPool() - - // Acquire and release a buffer - let buffer1 = pool.acquire(type: .contextArrays, size: 100) - pool.release(buffer1, type: .contextArrays) - - // Acquire again - should get from pool - let buffer2 = pool.acquire(type: .contextArrays, size: 100) - #expect(buffer2.count == 100) - - // Check statistics - let stats = pool.statistics() - #expect(stats[.contextArrays] == 0) // Should be checked out - } - - @Test("Buffer pool handles multiple types") - func testMultipleTypes() { - let pool = JPEGLSBufferPool() - - let contextBuffer = pool.acquire(type: .contextArrays, size: 365) - let pixelBuffer = pool.acquire(type: .pixelData, size: 1000) - - #expect(contextBuffer.count == 365) - #expect(pixelBuffer.count == 1000) - - pool.release(contextBuffer, type: .contextArrays) - pool.release(pixelBuffer, type: .pixelData) - - let stats = pool.statistics() - #expect(stats[.contextArrays] == 1) - #expect(stats[.pixelData] == 1) - } - - @Test("Buffer pool respects max pool size") - func testMaxPoolSize() { - let pool = JPEGLSBufferPool(maxPoolSize: 2) - - // Create and release 3 buffers directly - let buffer1 = Array(repeating: 0, count: 100) - let buffer2 = Array(repeating: 0, count: 100) - let buffer3 = Array(repeating: 0, count: 100) - - pool.release(buffer1, type: .contextArrays) - pool.release(buffer2, type: .contextArrays) - pool.release(buffer3, type: .contextArrays) - - let stats = pool.statistics() - #expect(stats[.contextArrays] == 2) // Should only keep 2 - } - - @Test("Buffer pool clears all buffers") - func testClear() { - let pool = JPEGLSBufferPool() - - let buffer = pool.acquire(type: .contextArrays, size: 100) - pool.release(buffer, type: .contextArrays) - - pool.clear() - - let stats = pool.statistics() - #expect(stats.isEmpty) - } - - @Test("Buffer pool does not store empty buffers") - func testEmptyBufferNotStored() { - let pool = JPEGLSBufferPool() - - let emptyBuffer: [Int] = [] - pool.release(emptyBuffer, type: .contextArrays) - - let stats = pool.statistics() - #expect(stats[.contextArrays] == nil) - } - - @Test("Buffer pool handles custom types") - func testCustomType() { - let pool = JPEGLSBufferPool() - let customType = JPEGLSBufferPool.BufferType.custom("testBuffer") - - let buffer = pool.acquire(type: customType, size: 50) - #expect(buffer.count == 50) - - pool.release(buffer, type: customType) - let stats = pool.statistics() - #expect(stats[customType] == 1) - } - - @Test("Buffer pool cleanup removes expired buffers") - func testCleanup() async { - let pool = JPEGLSBufferPool(bufferLifetime: 0.1) // 100ms lifetime - - let buffer = pool.acquire(type: .contextArrays, size: 100) - pool.release(buffer, type: .contextArrays) - - // Wait for buffer to expire - try? await Task.sleep(nanoseconds: 200_000_000) // 200ms - - pool.cleanup() - - let stats = pool.statistics() - #expect(stats[.contextArrays] == nil) - } - - @Test("Shared buffer pool is accessible") - func testSharedPool() { - let buffer = sharedBufferPool.acquire(type: .pixelData, size: 100) - #expect(buffer.count == 100) - - sharedBufferPool.release(buffer, type: .pixelData) - sharedBufferPool.clear() // Clean up for other tests - } - - @Test("Buffer pool handles large allocations") - func testLargeAllocation() { - let pool = JPEGLSBufferPool() - let largeBuffer = pool.acquire(type: .pixelData, size: 1_000_000) - - #expect(largeBuffer.count == 1_000_000) - } - - @Test("Buffer pool thread safety") - func testThreadSafety() async { - let pool = JPEGLSBufferPool() - - // Create multiple concurrent tasks - await withTaskGroup(of: Void.self) { group in - for _ in 0..<10 { - group.addTask { - let buffer = pool.acquire(type: .pixelData, size: 100) - pool.release(buffer, type: .pixelData) - } - } - } - - // Should not crash - #expect(true) - } -} diff --git a/Tests/JPEGLSTests/JPEGLSCacheFriendlyBufferTests.swift b/Tests/JPEGLSTests/JPEGLSCacheFriendlyBufferTests.swift deleted file mode 100644 index 8e06b11..0000000 --- a/Tests/JPEGLSTests/JPEGLSCacheFriendlyBufferTests.swift +++ /dev/null @@ -1,268 +0,0 @@ -/// Tests for cache-friendly buffer -import Testing -@testable import JPEGLS - -@Suite("JPEG-LS Cache-Friendly Buffer Tests") -struct JPEGLSCacheFriendlyBufferTests { - - @Test("Create buffer from 2D data") - func testCreateFrom2DData() { - let pixelData: [UInt8: [[Int]]] = [ - 0: [ - [1, 2, 3], - [4, 5, 6], - [7, 8, 9] - ] - ] - - let buffer = JPEGLSCacheFriendlyBuffer(pixelData: pixelData, width: 3, height: 3) - - #expect(buffer.width == 3) - #expect(buffer.height == 3) - #expect(buffer.componentCount == 1) - } - - @Test("Get pixel from buffer") - func testGetPixel() { - let pixelData: [UInt8: [[Int]]] = [ - 0: [ - [10, 20, 30], - [40, 50, 60] - ] - ] - - let buffer = JPEGLSCacheFriendlyBuffer(pixelData: pixelData, width: 3, height: 2) - - #expect(buffer.getPixel(componentId: 0, row: 0, column: 0) == 10) - #expect(buffer.getPixel(componentId: 0, row: 0, column: 2) == 30) - #expect(buffer.getPixel(componentId: 0, row: 1, column: 1) == 50) - } - - @Test("Get pixel out of bounds") - func testGetPixelOutOfBounds() { - let pixelData: [UInt8: [[Int]]] = [ - 0: [[1, 2], [3, 4]] - ] - - let buffer = JPEGLSCacheFriendlyBuffer(pixelData: pixelData, width: 2, height: 2) - - #expect(buffer.getPixel(componentId: 0, row: -1, column: 0) == nil) - #expect(buffer.getPixel(componentId: 0, row: 0, column: 5) == nil) - #expect(buffer.getPixel(componentId: 0, row: 10, column: 0) == nil) - } - - @Test("Set pixel creates new buffer") - func testSetPixel() { - let pixelData: [UInt8: [[Int]]] = [ - 0: [[1, 2], [3, 4]] - ] - - let buffer = JPEGLSCacheFriendlyBuffer(pixelData: pixelData, width: 2, height: 2) - let newBuffer = buffer.settingPixel(componentId: 0, row: 1, column: 1, value: 99) - - #expect(buffer.getPixel(componentId: 0, row: 1, column: 1) == 4) - #expect(newBuffer.getPixel(componentId: 0, row: 1, column: 1) == 99) - } - - @Test("Get row from buffer") - func testGetRow() { - let pixelData: [UInt8: [[Int]]] = [ - 0: [ - [10, 20, 30], - [40, 50, 60], - [70, 80, 90] - ] - ] - - let buffer = JPEGLSCacheFriendlyBuffer(pixelData: pixelData, width: 3, height: 3) - let row1 = buffer.getRow(componentId: 0, row: 1) - - #expect(row1 == [40, 50, 60]) - } - - @Test("Get multiple rows") - func testGetRows() { - let pixelData: [UInt8: [[Int]]] = [ - 0: [ - [1, 2], - [3, 4], - [5, 6] - ] - ] - - let buffer = JPEGLSCacheFriendlyBuffer(pixelData: pixelData, width: 2, height: 3) - let rows = buffer.getRows(componentId: 0, rowStart: 0, rowEnd: 2) - - #expect(rows == [1, 2, 3, 4]) - } - - @Test("Get neighbors") - func testGetNeighbors() { - let pixelData: [UInt8: [[Int]]] = [ - 0: [ - [1, 2, 3], - [4, 5, 6], - [7, 8, 9] - ] - ] - - let buffer = JPEGLSCacheFriendlyBuffer(pixelData: pixelData, width: 3, height: 3) - - // Center pixel - let neighbors = buffer.getNeighbors(componentId: 0, row: 1, column: 1) - #expect(neighbors.left == 4) - #expect(neighbors.top == 2) - #expect(neighbors.topLeft == 1) - #expect(neighbors.topRight == 3) - } - - @Test("Get neighbors at edges") - func testGetNeighborsAtEdges() { - let pixelData: [UInt8: [[Int]]] = [ - 0: [ - [1, 2, 3], - [4, 5, 6] - ] - ] - - let buffer = JPEGLSCacheFriendlyBuffer(pixelData: pixelData, width: 3, height: 2) - - // Top-left corner - let topLeft = buffer.getNeighbors(componentId: 0, row: 0, column: 0) - #expect(topLeft.left == nil) - #expect(topLeft.top == nil) - #expect(topLeft.topLeft == nil) - #expect(topLeft.topRight == nil) - - // Top edge - let topEdge = buffer.getNeighbors(componentId: 0, row: 0, column: 1) - #expect(topEdge.left == 1) - #expect(topEdge.top == nil) - #expect(topEdge.topLeft == nil) - #expect(topEdge.topRight == nil) - - // Bottom-right corner - let bottomRight = buffer.getNeighbors(componentId: 0, row: 1, column: 2) - #expect(bottomRight.left == 5) - #expect(bottomRight.top == 3) - #expect(bottomRight.topLeft == 2) - #expect(bottomRight.topRight == nil) - } - - @Test("Convert to 2D array") - func testTo2DArray() { - let pixelData: [UInt8: [[Int]]] = [ - 0: [ - [1, 2, 3], - [4, 5, 6] - ] - ] - - let buffer = JPEGLSCacheFriendlyBuffer(pixelData: pixelData, width: 3, height: 2) - let result = buffer.to2DArray(componentId: 0) - - #expect(result == [[1, 2, 3], [4, 5, 6]]) - } - - @Test("Get contiguous data") - func testGetContiguousData() { - let pixelData: [UInt8: [[Int]]] = [ - 0: [ - [1, 2], - [3, 4] - ] - ] - - let buffer = JPEGLSCacheFriendlyBuffer(pixelData: pixelData, width: 2, height: 2) - let data = buffer.getContiguousData(componentId: 0) - - #expect(data == [1, 2, 3, 4]) - } - - @Test("Multiple components") - func testMultipleComponents() { - let pixelData: [UInt8: [[Int]]] = [ - 0: [[1, 2], [3, 4]], - 1: [[5, 6], [7, 8]], - 2: [[9, 10], [11, 12]] - ] - - let buffer = JPEGLSCacheFriendlyBuffer(pixelData: pixelData, width: 2, height: 2) - - #expect(buffer.componentCount == 3) - #expect(buffer.componentIds.sorted() == [0, 1, 2]) - - #expect(buffer.getPixel(componentId: 0, row: 0, column: 0) == 1) - #expect(buffer.getPixel(componentId: 1, row: 0, column: 0) == 5) - #expect(buffer.getPixel(componentId: 2, row: 0, column: 0) == 9) - } - - @Test("Large buffer performance") - func testLargeBuffer() { - // Create a large buffer to test performance - var rows: [[Int]] = [] - for row in 0..<1024 { - var rowData: [Int] = [] - for col in 0..<1024 { - rowData.append(row * 1024 + col) - } - rows.append(rowData) - } - - let pixelData: [UInt8: [[Int]]] = [0: rows] - let buffer = JPEGLSCacheFriendlyBuffer(pixelData: pixelData, width: 1024, height: 1024) - - // Test random access - let pixel = buffer.getPixel(componentId: 0, row: 500, column: 750) - #expect(pixel == 500 * 1024 + 750) - - // Test row access - let row = buffer.getRow(componentId: 0, row: 100) - #expect(row.count == 1024) - #expect(row[0] == 100 * 1024) - } - - @Test("Cache-friendly neighbor access pattern") - func testCacheFriendlyNeighborAccess() { - let pixelData: [UInt8: [[Int]]] = [ - 0: [ - [10, 20, 30, 40], - [50, 60, 70, 80], - [90, 100, 110, 120] - ] - ] - - let buffer = JPEGLSCacheFriendlyBuffer(pixelData: pixelData, width: 4, height: 3) - - // Simulate processing a scanline (cache-friendly) - for col in 0..<4 { - let neighbors = buffer.getNeighbors(componentId: 0, row: 1, column: col) - - // Verify correct neighbors - if col == 0 { - #expect(neighbors.left == nil) - #expect(neighbors.top == 10) - } else { - #expect(neighbors.left != nil) - #expect(neighbors.top != nil) - } - } - } - - @Test("Memory statistics initialization") - func testMemoryStatistics() { - let stats = JPEGLSMemoryStatistics(totalBytes: 1000, peakBytes: 500, allocationCount: 10) - - #expect(stats.totalBytes == 1000) - #expect(stats.peakBytes == 500) - #expect(stats.allocationCount == 10) - #expect(stats.averageAllocationSize == 100.0) - } - - @Test("Memory statistics with zero allocations") - func testMemoryStatisticsZeroAllocations() { - let stats = JPEGLSMemoryStatistics(totalBytes: 0, peakBytes: 0, allocationCount: 0) - - #expect(stats.averageAllocationSize == 0.0) - } -} diff --git a/Tests/JPEGLSTests/JPEGLSPerformanceBenchmarks.swift b/Tests/JPEGLSTests/JPEGLSPerformanceBenchmarks.swift index 9f2aea7..cceff95 100644 --- a/Tests/JPEGLSTests/JPEGLSPerformanceBenchmarks.swift +++ b/Tests/JPEGLSTests/JPEGLSPerformanceBenchmarks.swift @@ -535,22 +535,25 @@ struct JPEGLSPerformanceBenchmarks { scanHeader: scanHeader ) - _ = try encoder.encodeScan(buffer: buffer) - + let encoded = try encoder.encodeScan(buffer: buffer) + let peakMemory = getCurrentMemoryUsage() let memoryUsedMB = Double(peakMemory - initialMemory) / (1024 * 1024) - + let width = imageData.frameHeader.width let height = imageData.frameHeader.height let bitsPerSample = imageData.frameHeader.bitsPerSample let imageDataSizeMB = Double(width * height * bitsPerSample / 8) / (1024 * 1024) - + print("Memory usage during 2048x2048 8-bit grayscale encoding:") print(" Image size: \(String(format: "%.2f", imageDataSizeMB)) MB") print(" Memory used: \(String(format: "%.2f", memoryUsedMB)) MB") print(" Memory ratio: \(String(format: "%.2f", memoryUsedMB / imageDataSizeMB))x") - - #expect(memoryUsedMB > 0) + + // The RSS delta is diagnostic only: with tests running in parallel + // and the encoder reusing allocator pages, the delta can legitimately + // be zero or negative, so asserting `> 0` is flaky by construction. + #expect(encoded.pixelsEncoded == width * height) } // MARK: - Helper Methods diff --git a/Tests/JPEGLSTests/JPEGLSRestartIntervalTests.swift b/Tests/JPEGLSTests/JPEGLSRestartIntervalTests.swift new file mode 100644 index 0000000..e65aaf0 --- /dev/null +++ b/Tests/JPEGLSTests/JPEGLSRestartIntervalTests.swift @@ -0,0 +1,169 @@ +/// Tests for DRI/RSTm restart-interval support: encoder emission, decoder +/// consumption, bitstream structure, and configuration validation. + +import Foundation +import Testing +@testable import JPEGLS + +@Suite("Restart interval (DRI/RSTm) support") +struct JPEGLSRestartIntervalTests { + + // MARK: - Helpers + + private func makeGradientPixels(width: Int, height: Int, maxValue: Int) -> [[Int]] { + (0.. [[Int]] { + (0.. [UInt8] { + var found: [UInt8] = [] + let bytes = [UInt8](data) + var i = 0 + while i < bytes.count - 1 { + if bytes[i] == 0xFF && (0xD0...0xD7).contains(bytes[i + 1]) { + found.append(bytes[i + 1]) + i += 2 + } else { + i += 1 + } + } + return found + } + + private func roundTrip( + pixels: [[Int]], bitsPerSample: Int, restartInterval: Int + ) throws -> (encoded: Data, decoded: [[Int]]) { + let imageData = try MultiComponentImageData.grayscale( + pixels: pixels, bitsPerSample: bitsPerSample + ) + let config = try JPEGLSEncoder.Configuration(restartInterval: restartInterval) + let encoded = try JPEGLSEncoder().encode(imageData, configuration: config) + let decoded = try JPEGLSDecoder().decode(encoded) + return (encoded, decoded.components[0].pixels) + } + + // MARK: - Round-trip + + @Test("8-bit lossless round-trip with restart interval") + func roundTrip8Bit() throws { + let pixels = makeGradientPixels(width: 100, height: 64, maxValue: 255) + let (encoded, decoded) = try roundTrip(pixels: pixels, bitsPerSample: 8, restartInterval: 16) + #expect(decoded == pixels) + // 64 lines / 16 per interval = 4 intervals -> 3 RST markers D0,D1,D2. + #expect(countRestartMarkers(encoded) == [0xD0, 0xD1, 0xD2]) + } + + @Test("16-bit lossless round-trip with restart interval") + func roundTrip16Bit() throws { + let pixels = makeGradientPixels(width: 80, height: 50, maxValue: 4095) + let (encoded, decoded) = try roundTrip(pixels: pixels, bitsPerSample: 12, restartInterval: 8) + #expect(decoded == pixels) + // ceil(50 / 8) = 7 intervals -> 6 RST markers. + #expect(countRestartMarkers(encoded).count == 6) + } + + @Test("Run-heavy image round-trips across interval boundaries") + func roundTripRunHeavy() throws { + let pixels = makeRunHeavyPixels(width: 64, height: 48, value: 200) + let (_, decoded) = try roundTrip(pixels: pixels, bitsPerSample: 8, restartInterval: 7) + #expect(decoded == pixels) + } + + @Test("Marker index cycles through D0–D7 for many intervals") + func markerCycling() throws { + let pixels = makeGradientPixels(width: 16, height: 40, maxValue: 255) + let (encoded, decoded) = try roundTrip(pixels: pixels, bitsPerSample: 8, restartInterval: 2) + #expect(decoded == pixels) + // 20 intervals -> 19 markers cycling D0..D7,D0.. + let markers = countRestartMarkers(encoded) + #expect(markers.count == 19) + for (i, marker) in markers.enumerated() { + #expect(marker == 0xD0 + UInt8(i % 8)) + } + } + + @Test("Restart interval >= height emits DRI but no RST markers") + func intervalLargerThanImage() throws { + let pixels = makeGradientPixels(width: 32, height: 16, maxValue: 255) + let (encoded, decoded) = try roundTrip(pixels: pixels, bitsPerSample: 8, restartInterval: 64) + #expect(decoded == pixels) + #expect(countRestartMarkers(encoded).isEmpty) + } + + @Test("Restart encoding is deterministic (parallel intervals)") + func deterministicOutput() throws { + let pixels = makeGradientPixels(width: 128, height: 96, maxValue: 1023) + let (first, _) = try roundTrip(pixels: pixels, bitsPerSample: 10, restartInterval: 8) + let (second, _) = try roundTrip(pixels: pixels, bitsPerSample: 10, restartInterval: 8) + #expect(first == second) + } + + @Test("Multi-component non-interleaved scans each honour the restart interval") + func multiComponentNonInterleaved() throws { + let red = makeGradientPixels(width: 40, height: 32, maxValue: 255) + let green = makeRunHeavyPixels(width: 40, height: 32, value: 99) + let blue = makeGradientPixels(width: 40, height: 32, maxValue: 200) + let imageData = try MultiComponentImageData.rgb( + redPixels: red, greenPixels: green, bluePixels: blue, bitsPerSample: 8 + ) + let config = try JPEGLSEncoder.Configuration( + interleaveMode: .none, restartInterval: 8 + ) + let encoded = try JPEGLSEncoder().encode(imageData, configuration: config) + let decoded = try JPEGLSDecoder().decode(encoded) + #expect(decoded.components[0].pixels == red) + #expect(decoded.components[1].pixels == green) + #expect(decoded.components[2].pixels == blue) + // 3 scans x (32/8 - 1) = 9 RST markers in total. + #expect(countRestartMarkers(encoded).count == 9) + } + + @Test("DRI marker segment is present and carries the interval") + func driSegment() throws { + let pixels = makeGradientPixels(width: 32, height: 32, maxValue: 255) + let (encoded, _) = try roundTrip(pixels: pixels, bitsPerSample: 8, restartInterval: 5) + let parser = JPEGLSParser(data: encoded) + let result = try parser.parse() + #expect(result.restartInterval == 5) + } + + // MARK: - Validation + + @Test("Restart interval with near-lossless throws") + func nearLosslessRejected() { + #expect(throws: JPEGLSError.self) { + _ = try JPEGLSEncoder.Configuration(near: 2, restartInterval: 8) + } + } + + @Test("Restart interval with interleaved mode throws") + func interleavedRejected() { + #expect(throws: JPEGLSError.self) { + _ = try JPEGLSEncoder.Configuration(interleaveMode: .line, restartInterval: 8) + } + #expect(throws: JPEGLSError.self) { + _ = try JPEGLSEncoder.Configuration(interleaveMode: .sample, restartInterval: 8) + } + } + + @Test("Out-of-range restart interval throws") + func outOfRangeRejected() { + #expect(throws: JPEGLSError.self) { + _ = try JPEGLSEncoder.Configuration(restartInterval: -1) + } + #expect(throws: JPEGLSError.self) { + _ = try JPEGLSEncoder.Configuration(restartInterval: 65536) + } + } +} diff --git a/Tests/JPEGLSTests/JPEGLSRobustnessRegressionTests.swift b/Tests/JPEGLSTests/JPEGLSRobustnessRegressionTests.swift new file mode 100644 index 0000000..927c2c8 --- /dev/null +++ b/Tests/JPEGLSTests/JPEGLSRobustnessRegressionTests.swift @@ -0,0 +1,246 @@ +/// Regression tests for issues found by the hot-path-optimizations branch +/// review: Data-slice handling, malformed/hostile streams, and encoder +/// configuration validation. Every malformed input must throw a +/// `JPEGLSError` — never crash, and never decode silently to garbage. + +import Foundation +import Testing +@testable import JPEGLS + +@Suite("Robustness regressions (branch review)") +struct JPEGLSRobustnessRegressionTests { + + // MARK: - Helpers + + private func encodeSample( + width: Int = 24, height: Int = 20, restartInterval: Int = 0 + ) throws -> (encoded: Data, pixels: [[Int]]) { + let pixels = (0.. Data { + var bytes = [UInt8](data) + var i = 0 + while i + 1 < bytes.count { + if bytes[i] == 0xFF && bytes[i + 1] == 0xDD { + bytes.removeSubrange(i..= height still decodes") + func interleavedStreamWithInactiveDRI() throws { + // A DRI larger than the frame height produces zero RST markers; the + // scan body is identical to a no-DRI stream and must not be rejected. + let size = 8 + let plane = (0..= height 8) right after SOI. + var bytes = [UInt8](encoded) + let dri: [UInt8] = [0xFF, 0xDD, 0x00, 0x04, 0x27, 0x0F] // 0x270F = 9999 + bytes.insert(contentsOf: dri, at: 2) + let decoded = try JPEGLSDecoder().decode(Data(bytes)) + #expect(decoded.components[0].pixels == plane) + } + + @Test("Per-scan DRI: interval defined after the first scan applies to later scans only") + func driAfterFirstScan() throws { + // Encode two single-component streams — one plain, one with restart — + // and splice them into a 2-component non-interleaved stream where the + // DRI appears between scan 1 (no restart) and scan 2 (restart). + let size = 16 + let plane = (0.. 0.9) - #expect(savings < 1.0) - } - - @Test("Estimate memory savings for single tile") - func testEstimateMemorySavingsSingleTile() { - let processor = JPEGLSTileProcessor( - imageWidth: 256, - imageHeight: 256, - configuration: TileConfiguration(tileWidth: 512, tileHeight: 512, overlap: 0) - ) - - let savings = processor.estimateMemorySavings(bytesPerPixel: 2) - - // Image fits in single tile, no savings - #expect(savings == 0.0) - } - - @Test("Large image tiling") - func testLargeImageTiling() { - let processor = JPEGLSTileProcessor( - imageWidth: 8192, - imageHeight: 8192, - configuration: TileConfiguration(tileWidth: 512, tileHeight: 512, overlap: 0) - ) - - let tiles = processor.calculateTiles() - let expectedTiles = 16 * 16 // 16x16 grid - - #expect(tiles.count == expectedTiles) - - // Verify each tile is correctly sized - for tile in tiles { - #expect(tile.width <= 512) - #expect(tile.height <= 512) - } - } - - @Test("Non-square tiles") - func testNonSquareTiles() { - let processor = JPEGLSTileProcessor( - imageWidth: 2048, - imageHeight: 1024, - configuration: TileConfiguration(tileWidth: 512, tileHeight: 256, overlap: 0) - ) - - let tiles = processor.calculateTiles() - - // Should be 4x4 grid - #expect(tiles.count == 16) - } - - @Test("Tile bounds equality") - func testTileBoundsEquality() { - let bounds1 = TileBounds(rowStart: 0, rowEnd: 100, columnStart: 0, columnEnd: 100) - let bounds2 = TileBounds(rowStart: 0, rowEnd: 100, columnStart: 0, columnEnd: 100) - let bounds3 = TileBounds(rowStart: 0, rowEnd: 100, columnStart: 0, columnEnd: 50) - - #expect(bounds1 == bounds2) - #expect(bounds1 != bounds3) - } -} diff --git a/Tests/JPEGLSTests/MetalAcceleratorTests.swift b/Tests/JPEGLSTests/MetalAcceleratorTests.swift deleted file mode 100644 index 95fc117..0000000 --- a/Tests/JPEGLSTests/MetalAcceleratorTests.swift +++ /dev/null @@ -1,401 +0,0 @@ -/// Tests for Metal GPU acceleration. -/// -/// These tests verify the correctness and performance of Metal GPU-accelerated -/// operations for JPEG-LS encoding, ensuring bit-exact results compared to -/// CPU implementations. - -#if canImport(Metal) - -import Testing -@testable import JPEGLS - -@Suite("Metal GPU Acceleration Tests") -struct MetalAcceleratorTests { - - // MARK: - Initialization Tests - - @Test("Metal accelerator initializes on supported devices") - func testInitialization() throws { - #expect(MetalAccelerator.isSupported, "Metal should be available for testing") - - let accelerator = try MetalAccelerator() - #expect(MetalAccelerator.platformName == "Metal") - } - - @Test("Metal accelerator reports correct support status") - func testSupportStatus() { - // Metal support depends on hardware availability - // This test just verifies the check doesn't crash - let isSupported = MetalAccelerator.isSupported - #expect(isSupported == true || isSupported == false) - } - - // MARK: - Gradient Computation Tests - - @Test("Compute gradients for single pixel batch") - func testSinglePixelGradients() throws { - guard MetalAccelerator.isSupported else { return } - - let accelerator = try MetalAccelerator() - - // Test with simple values - let a: [Int32] = [10] - let b: [Int32] = [15] - let c: [Int32] = [5] - - let (d1, d2, d3) = try accelerator.computeGradientsBatch(a: a, b: b, c: c) - - // Expected gradients: - // d1 = b - c = 15 - 5 = 10 - // d2 = a - c = 10 - 5 = 5 - // d3 = c - a = 5 - 10 = -5 - #expect(d1 == [10]) - #expect(d2 == [5]) - #expect(d3 == [-5]) - } - - @Test("Compute gradients for small batch (CPU fallback)") - func testSmallBatchGradients() throws { - guard MetalAccelerator.isSupported else { return } - - let accelerator = try MetalAccelerator() - - // Use batch smaller than GPU threshold to test CPU fallback - let count = MetalAccelerator.gpuThreshold / 2 - let a = [Int32](repeating: 10, count: count) - let b = [Int32](repeating: 15, count: count) - let c = [Int32](repeating: 5, count: count) - - let (d1, d2, d3) = try accelerator.computeGradientsBatch(a: a, b: b, c: c) - - // All gradients should be identical - #expect(d1.allSatisfy { $0 == 10 }) - #expect(d2.allSatisfy { $0 == 5 }) - #expect(d3.allSatisfy { $0 == -5 }) - } - - @Test("Compute gradients for large batch (GPU)") - func testLargeBatchGradients() throws { - guard MetalAccelerator.isSupported else { return } - - let accelerator = try MetalAccelerator() - - // Use batch larger than GPU threshold to test GPU execution - let count = MetalAccelerator.gpuThreshold * 2 - var a = [Int32](repeating: 0, count: count) - var b = [Int32](repeating: 0, count: count) - var c = [Int32](repeating: 0, count: count) - - // Fill with test pattern - for i in 0..= max(a, b) → should return min(a, b) - let a1: [Int32] = [10] - let b1: [Int32] = [15] - let c1: [Int32] = [20] - let pred1 = try accelerator.computeMEDPredictionBatch(a: a1, b: b1, c: c1) - #expect(pred1 == [10], "c >= max(a,b) should return min(a,b)") - - // Test case: c <= min(a, b) → should return max(a, b) - let a2: [Int32] = [15] - let b2: [Int32] = [20] - let c2: [Int32] = [10] - let pred2 = try accelerator.computeMEDPredictionBatch(a: a2, b: b2, c: c2) - #expect(pred2 == [20], "c <= min(a,b) should return max(a,b)") - - // Test case: min(a,b) < c < max(a,b) → should return a + b - c - let a3: [Int32] = [10] - let b3: [Int32] = [20] - let c3: [Int32] = [15] - let pred3 = try accelerator.computeMEDPredictionBatch(a: a3, b: b3, c: c3) - #expect(pred3 == [15], "Should return a+b-c = 10+20-15 = 15") - } - - @Test("Compute MED prediction for small batch (CPU fallback)") - func testSmallBatchMEDPrediction() throws { - guard MetalAccelerator.isSupported else { return } - - let accelerator = try MetalAccelerator() - - // Create small batch for CPU fallback - let count = MetalAccelerator.gpuThreshold / 2 - let a = [Int32](repeating: 10, count: count) - let b = [Int32](repeating: 20, count: count) - let c = [Int32](repeating: 15, count: count) - - let predictions = try accelerator.computeMEDPredictionBatch(a: a, b: b, c: c) - - // All predictions should be a + b - c = 15 - #expect(predictions.allSatisfy { $0 == 15 }) - } - - @Test("Compute MED prediction for large batch (GPU)") - func testLargeBatchMEDPrediction() throws { - guard MetalAccelerator.isSupported else { return } - - let accelerator = try MetalAccelerator() - - // Create large batch for GPU execution - let count = MetalAccelerator.gpuThreshold * 2 - var a = [Int32](repeating: 0, count: count) - var b = [Int32](repeating: 0, count: count) - var c = [Int32](repeating: 0, count: count) - - // Fill with test pattern covering all MED cases - for i in 0..= max(a, b) - a[i] = 10 - b[i] = 15 - c[i] = 20 - } else if idx == 1 { - // Case: c <= min(a, b) - a[i] = 15 - b[i] = 20 - c[i] = 10 - } else { - // Case: min < c < max - a[i] = 10 - b[i] = 20 - c[i] = 15 - } - } - - let predictions = try accelerator.computeMEDPredictionBatch(a: a, b: b, c: c) - - // Verify predictions match expected values - for i in 0..= maxAB { - expected = minAB - } else if cv <= minAB { - expected = maxAB - } else { - expected = av + bv - cv - } - - #expect(predictions[i] == expected, "Prediction at index \(i) should match expected value") - } - } - - @Test("Compute MED prediction with equal pixel values") - func testMEDPredictionWithEqualValues() throws { - guard MetalAccelerator.isSupported else { return } - - let accelerator = try MetalAccelerator() - - // When all pixels are equal, prediction should equal the pixel value - let a: [Int32] = [100, 100, 100] - let b: [Int32] = [100, 100, 100] - let c: [Int32] = [100, 100, 100] - - let predictions = try accelerator.computeMEDPredictionBatch(a: a, b: b, c: c) - - #expect(predictions == [100, 100, 100]) - } - - @Test("Compute MED prediction with zero values") - func testMEDPredictionWithZeroValues() throws { - guard MetalAccelerator.isSupported else { return } - - let accelerator = try MetalAccelerator() - - let a: [Int32] = [0, 0, 0] - let b: [Int32] = [0, 0, 0] - let c: [Int32] = [0, 0, 0] - - let predictions = try accelerator.computeMEDPredictionBatch(a: a, b: b, c: c) - - #expect(predictions == [0, 0, 0]) - } - - @Test("Compute MED prediction with empty arrays") - func testMEDPredictionWithEmptyArrays() throws { - guard MetalAccelerator.isSupported else { return } - - let accelerator = try MetalAccelerator() - - let a: [Int32] = [] - let b: [Int32] = [] - let c: [Int32] = [] - - let predictions = try accelerator.computeMEDPredictionBatch(a: a, b: b, c: c) - - #expect(predictions.isEmpty) - } - - // MARK: - Bit-Exact Comparison Tests - - @Test("Metal gradients match scalar implementation") - func testGradientsBitExactMatch() throws { - guard MetalAccelerator.isSupported else { return } - - let metalAccelerator = try MetalAccelerator() - let scalarAccelerator = ScalarAccelerator() - - // Test with random values - let count = MetalAccelerator.gpuThreshold * 2 - var a = [Int32](repeating: 0, count: count) - var b = [Int32](repeating: 0, count: count) - var c = [Int32](repeating: 0, count: count) - - for i in 0.. Void) rethrows -> (min: TimeInterval, max: TimeInterval, avg: TimeInterval) { - var times: [TimeInterval] = [] - - for _ in 0.. (a: [Int32], b: [Int32], c: [Int32]) { - var a = [Int32](repeating: 0, count: count) - var b = [Int32](repeating: 0, count: count) - var c = [Int32](repeating: 0, count: count) - - for i in 0...size) / (1024 * 1024) - - print(""" - - Metal Gradients 2048×2048 (Large Image): - Image: 2048×2048 (\(count) pixels) - Data size: \(String(format: "%.2f", dataSize)) MB - Iterations: \(Self.benchmarkIterations) - Average time: \(String(format: "%.2f", avgTime * 1000)) ms - Min time: \(String(format: "%.2f", minTime * 1000)) ms - Max time: \(String(format: "%.2f", maxTime * 1000)) ms - Throughput: \(String(format: "%.2f", throughput)) Mpixels/s - Note: Should show GPU benefit - """) - } - - @Test("Benchmark: Metal gradient computation - 4096×4096 (very large)") - func benchmarkGradients4096x4096() throws { - guard MetalAccelerator.isSupported else { return } - - let accelerator = try MetalAccelerator() - let count = 4096 * 4096 // 16,777,216 pixels - let (a, b, c) = generateTestData(count: count) - - // Warmup - for _ in 0...size) / (1024 * 1024) - - print(""" - - Metal Gradients 4096×4096 (Very Large Image): - Image: 4096×4096 (\(count) pixels) - Data size: \(String(format: "%.2f", dataSize)) MB - Iterations: \(Self.benchmarkIterations) - Average time: \(String(format: "%.2f", avgTime * 1000)) ms - Min time: \(String(format: "%.2f", minTime * 1000)) ms - Max time: \(String(format: "%.2f", maxTime * 1000)) ms - Throughput: \(String(format: "%.2f", throughput)) Mpixels/s - Note: Should show maximum GPU benefit - """) - } - - // MARK: - MED Prediction Benchmarks - - @Test("Benchmark: Metal MED prediction - 1024×1024") - func benchmarkMEDPrediction1024x1024() throws { - guard MetalAccelerator.isSupported else { return } - - let accelerator = try MetalAccelerator() - let count = 1024 * 1024 - let (a, b, c) = generateTestData(count: count) - - // Warmup - for _ in 0..= 64) - #expect(buf2[0..<64].allSatisfy { $0 == 0 }, "Pooled buffer not zeroed on re-acquire") - } - - /// Verify that pool returns a buffer at least as large as requested when - /// only a larger buffer is pooled. - @Test("BufferPool: larger pooled buffer satisfies smaller request") - func testLargerPooledBufferSatisfiesSmallerRequest() { - let pool = JPEGLSBufferPool(maxPoolSize: 4) - let big = pool.acquire(type: .bitstreamData, size: 128) - pool.release(big, type: .bitstreamData) - let small = pool.acquire(type: .bitstreamData, size: 32) - #expect(small.count >= 32, "Re-used buffer should satisfy smaller request") - } - - /// Verify that a fresh buffer is allocated when the pool is empty. - @Test("BufferPool: allocates new buffer when pool is empty") - func testAllocatesNewWhenEmpty() { - let pool = JPEGLSBufferPool(maxPoolSize: 4) - let buf = pool.acquire(type: .contextArrays, size: 365) - #expect(buf.count == 365) - #expect(buf.allSatisfy { $0 == 0 }) - } - - /// Verify that the pool does not overflow beyond maxPoolSize. - @Test("BufferPool: pool respects maxPoolSize") - func testPoolRespectsMaxSize() { - let pool = JPEGLSBufferPool(maxPoolSize: 3) - for _ in 0..<5 { - let b = pool.acquire(type: .pixelData, size: 10) - pool.release(b, type: .pixelData) - } - let stats = pool.statistics() - #expect((stats[.pixelData] ?? 0) <= 3, "Pool exceeded maxPoolSize") - } -} - // MARK: - Phase 16.3: Zero-Copy Bitstream Writer @Suite("Phase 16.3: Zero-Copy Bitstream Writer Paths") diff --git a/Tests/JPEGLSTests/PlatformBenchmarks.swift b/Tests/JPEGLSTests/PlatformBenchmarks.swift deleted file mode 100644 index dceadc8..0000000 --- a/Tests/JPEGLSTests/PlatformBenchmarks.swift +++ /dev/null @@ -1,228 +0,0 @@ -import Testing -import Foundation -@testable import JPEGLS - -/// Benchmarks for comparing scalar vs SIMD platform implementations. -/// -/// These benchmarks measure the performance improvement of SIMD-optimized -/// implementations compared to the scalar reference implementation. -@Suite("Platform Benchmarks") -struct PlatformBenchmarks { - /// Sample size for benchmark iterations - private static let benchmarkIterations = 10_000 - - // MARK: - Gradient Computation Benchmarks - - @Test("Benchmark: Gradient computation performance") - func benchmarkGradientComputation() { - let scalar = ScalarAccelerator() - let simd = selectPlatformAccelerator() - - // Generate test data - let testCases = (0..= max(a, b)") - func scalarMEDCase1() { - let accelerator = ScalarAccelerator() - let result = accelerator.medPredictor(a: 10, b: 20, c: 30) - #expect(result == 10) // min(a, b) = min(10, 20) = 10 - } - - @Test("ScalarAccelerator MED predictor with c <= min(a, b)") - func scalarMEDCase2() { - let accelerator = ScalarAccelerator() - let result = accelerator.medPredictor(a: 20, b: 30, c: 10) - #expect(result == 30) // max(a, b) = max(20, 30) = 30 - } - - @Test("ScalarAccelerator MED predictor with c between a and b") - func scalarMEDCase3() { - let accelerator = ScalarAccelerator() - let result = accelerator.medPredictor(a: 10, b: 30, c: 20) - #expect(result == 20) // a + b - c = 10 + 30 - 20 = 20 - } - - @Test("ScalarAccelerator MED predictor edge cases") - func scalarMEDEdgeCases() { - let accelerator = ScalarAccelerator() - - // All equal - #expect(accelerator.medPredictor(a: 10, b: 10, c: 10) == 10) - - // Zero values - #expect(accelerator.medPredictor(a: 0, b: 0, c: 0) == 0) - } - - @Test("ScalarAccelerator quantizes gradients correctly - positive values") - func scalarQuantizePositive() { - let accelerator = ScalarAccelerator() - let t1 = 3, t2 = 7, t3 = 21 - - // Test d = 0 - let result0 = accelerator.quantizeGradients(d1: 0, d2: 0, d3: 0, t1: t1, t2: t2, t3: t3) - #expect(result0.q1 == 0) - #expect(result0.q2 == 0) - #expect(result0.q3 == 0) - - // Test 0 < d < t1 - let result1 = accelerator.quantizeGradients(d1: 1, d2: 2, d3: 2, t1: t1, t2: t2, t3: t3) - #expect(result1.q1 == 1) - #expect(result1.q2 == 1) - #expect(result1.q3 == 1) - - // Test t1 <= d < t2 - let result2 = accelerator.quantizeGradients(d1: 3, d2: 5, d3: 6, t1: t1, t2: t2, t3: t3) - #expect(result2.q1 == 2) - #expect(result2.q2 == 2) - #expect(result2.q3 == 2) - - // Test t2 <= d < t3 - let result3 = accelerator.quantizeGradients(d1: 7, d2: 10, d3: 20, t1: t1, t2: t2, t3: t3) - #expect(result3.q1 == 3) - #expect(result3.q2 == 3) - #expect(result3.q3 == 3) - - // Test d >= t3 - let result4 = accelerator.quantizeGradients(d1: 21, d2: 50, d3: 100, t1: t1, t2: t2, t3: t3) - #expect(result4.q1 == 4) - #expect(result4.q2 == 4) - #expect(result4.q3 == 4) - } - - @Test("ScalarAccelerator quantizes gradients correctly - negative values") - func scalarQuantizeNegative() { - let accelerator = ScalarAccelerator() - let t1 = 3, t2 = 7, t3 = 21 - - // Test -t1 < d < 0 - let result1 = accelerator.quantizeGradients(d1: -1, d2: -2, d3: -2, t1: t1, t2: t2, t3: t3) - #expect(result1.q1 == -1) - #expect(result1.q2 == -1) - #expect(result1.q3 == -1) - - // Test -t2 < d <= -t1 - let result2 = accelerator.quantizeGradients(d1: -3, d2: -5, d3: -6, t1: t1, t2: t2, t3: t3) - #expect(result2.q1 == -2) - #expect(result2.q2 == -2) - #expect(result2.q3 == -2) - - // Test -t3 < d <= -t2 - let result3 = accelerator.quantizeGradients(d1: -7, d2: -10, d3: -20, t1: t1, t2: t2, t3: t3) - #expect(result3.q1 == -3) - #expect(result3.q2 == -3) - #expect(result3.q3 == -3) - - // Test d <= -t3 - let result4 = accelerator.quantizeGradients(d1: -21, d2: -50, d3: -100, t1: t1, t2: t2, t3: t3) - #expect(result4.q1 == -4) - #expect(result4.q2 == -4) - #expect(result4.q3 == -4) - } - - @Test("ScalarAccelerator quantizes gradients with different values") - func scalarQuantizeMixed() { - let accelerator = ScalarAccelerator() - let t1 = 3, t2 = 7, t3 = 21 - - let result = accelerator.quantizeGradients(d1: -10, d2: 0, d3: 10, t1: t1, t2: t2, t3: t3) - #expect(result.q1 == -3) // -10 is in range -t3 < d <= -t2 - #expect(result.q2 == 0) // 0 maps to 0 - #expect(result.q3 == 3) // 10 is in range t2 <= d < t3 - } - - // MARK: - Platform Selection Tests - - @Test("selectPlatformAccelerator returns a valid accelerator") - func platformSelection() { - let accelerator = selectPlatformAccelerator() - - // Verify that the accelerator works by testing basic operations - let gradients = accelerator.computeGradients(a: 10, b: 20, c: 15) - #expect(gradients.d1 == 5) - #expect(gradients.d2 == -5) - #expect(gradients.d3 == 5) - - let prediction = accelerator.medPredictor(a: 10, b: 30, c: 20) - #expect(prediction == 20) - } - - @Test("selectPlatformAccelerator returns correct platform for architecture") - func platformSelectionCorrectType() { - let accelerator = selectPlatformAccelerator() - - #if arch(arm64) - // On ARM64, should get ARM64Accelerator - #expect(type(of: accelerator) is ARM64Accelerator.Type || type(of: accelerator) is ScalarAccelerator.Type) - #elseif arch(x86_64) - // On x86_64, should get X86_64Accelerator - #expect(type(of: accelerator) is X86_64Accelerator.Type || type(of: accelerator) is ScalarAccelerator.Type) - #else - // On other architectures, should get ScalarAccelerator - #expect(type(of: accelerator) is ScalarAccelerator.Type) - #endif - } - - // MARK: - ARM64Accelerator Tests (conditionally compiled) - - #if arch(arm64) - @Test("ARM64Accelerator platformName is correct") - func arm64PlatformName() { - #expect(ARM64Accelerator.platformName == "ARM64") - } - - @Test("ARM64Accelerator is supported on ARM64") - func arm64IsSupported() { - #expect(ARM64Accelerator.isSupported == true) - } - - @Test("ARM64Accelerator produces correct results") - func arm64Results() { - let accelerator = ARM64Accelerator() - - // Test gradients - let gradients = accelerator.computeGradients(a: 10, b: 20, c: 15) - #expect(gradients.d1 == 5) - #expect(gradients.d2 == -5) - #expect(gradients.d3 == 5) - - // Test MED predictor - let prediction = accelerator.medPredictor(a: 10, b: 30, c: 20) - #expect(prediction == 20) - - // Test quantization - let quant = accelerator.quantizeGradients(d1: 5, d2: 0, d3: -10, t1: 3, t2: 7, t3: 21) - #expect(quant.q1 == 2) - #expect(quant.q2 == 0) - #expect(quant.q3 == -3) - } - #endif - - // MARK: - X86_64Accelerator Tests (conditionally compiled) - - #if arch(x86_64) - @Test("X86_64Accelerator platformName is correct") - func x86_64PlatformName() { - #expect(X86_64Accelerator.platformName == "x86-64") - } - - @Test("X86_64Accelerator is supported on x86-64") - func x86_64IsSupported() { - #expect(X86_64Accelerator.isSupported == true) - } - - @Test("X86_64Accelerator produces correct results") - func x86_64Results() { - let accelerator = X86_64Accelerator() - - // Test gradients - let gradients = accelerator.computeGradients(a: 10, b: 20, c: 15) - #expect(gradients.d1 == 5) - #expect(gradients.d2 == -5) - #expect(gradients.d3 == 5) - - // Test MED predictor - all cases - // Case 1: c >= max(a, b) - let pred1 = accelerator.medPredictor(a: 10, b: 20, c: 30) - #expect(pred1 == 10) - - // Case 2: c <= min(a, b) - let pred2 = accelerator.medPredictor(a: 20, b: 30, c: 10) - #expect(pred2 == 30) - - // Case 3: c between a and b - let pred3 = accelerator.medPredictor(a: 10, b: 30, c: 20) - #expect(pred3 == 20) - - // Test quantization - all quantization levels - // Positive quantization levels - let quant1 = accelerator.quantizeGradients(d1: 0, d2: 2, d3: 5, t1: 3, t2: 7, t3: 21) - #expect(quant1.q1 == 0) // d == 0 - #expect(quant1.q2 == 1) // 0 < d < t1 - #expect(quant1.q3 == 2) // t1 <= d < t2 - - let quant2 = accelerator.quantizeGradients(d1: 10, d2: 25, d3: 1, t1: 3, t2: 7, t3: 21) - #expect(quant2.q1 == 3) // t2 <= d < t3 - #expect(quant2.q2 == 4) // d >= t3 - #expect(quant2.q3 == 1) // 0 < d < t1 - - // Negative quantization levels - let quant3 = accelerator.quantizeGradients(d1: -1, d2: -5, d3: -10, t1: 3, t2: 7, t3: 21) - #expect(quant3.q1 == -1) // -t1 < d < 0 - #expect(quant3.q2 == -2) // -t2 < d <= -t1 - #expect(quant3.q3 == -3) // -t3 < d <= -t2 - - let quant4 = accelerator.quantizeGradients(d1: -25, d2: -21, d3: -2, t1: 3, t2: 7, t3: 21) - #expect(quant4.q1 == -4) // d <= -t3 - #expect(quant4.q2 == -4) // d <= -t3 - #expect(quant4.q3 == -1) // -t1 < d < 0 - } - #endif - - // MARK: - Additional Edge Case Tests - - @Test("ScalarAccelerator gradient computation with large values") - func scalarGradientsLargeValues() { - let accelerator = ScalarAccelerator() - let result = accelerator.computeGradients(a: 1000, b: 2000, c: 1500) - #expect(result.d1 == 500) - #expect(result.d2 == -500) - #expect(result.d3 == 500) - } - - @Test("ScalarAccelerator MED predictor with all same values") - func scalarMEDAllSame() { - let accelerator = ScalarAccelerator() - let result = accelerator.medPredictor(a: 100, b: 100, c: 100) - #expect(result == 100) - } - - @Test("ScalarAccelerator MED predictor boundary between cases") - func scalarMEDBoundary() { - let accelerator = ScalarAccelerator() - - // Test case where c == max(a, b) - let result1 = accelerator.medPredictor(a: 10, b: 20, c: 20) - #expect(result1 == 10) // c >= max(a,b), so min(a,b) - - // Test case where c == min(a, b) - let result2 = accelerator.medPredictor(a: 20, b: 30, c: 20) - #expect(result2 == 30) // c <= min(a,b), so max(a,b) = 30 - } - - @Test("ScalarAccelerator quantization at boundaries") - func scalarQuantizeBoundaries() { - let accelerator = ScalarAccelerator() - let t1 = 3, t2 = 7, t3 = 21 - - // Test exactly at boundaries - let resultT1 = accelerator.quantizeGradients(d1: 3, d2: 3, d3: 3, t1: t1, t2: t2, t3: t3) - #expect(resultT1.q1 == 2) // t1 <= d < t2 - - let resultT2 = accelerator.quantizeGradients(d1: 7, d2: 7, d3: 7, t1: t1, t2: t2, t3: t3) - #expect(resultT2.q1 == 3) // t2 <= d < t3 - - let resultT3 = accelerator.quantizeGradients(d1: 21, d2: 21, d3: 21, t1: t1, t2: t2, t3: t3) - #expect(resultT3.q1 == 4) // d >= t3 - - // Negative boundaries - let resultNegT1 = accelerator.quantizeGradients(d1: -3, d2: -3, d3: -3, t1: t1, t2: t2, t3: t3) - #expect(resultNegT1.q1 == -2) // -t2 < d <= -t1 - - let resultNegT2 = accelerator.quantizeGradients(d1: -7, d2: -7, d3: -7, t1: t1, t2: t2, t3: t3) - #expect(resultNegT2.q1 == -3) // -t3 < d <= -t2 - - let resultNegT3 = accelerator.quantizeGradients(d1: -21, d2: -21, d3: -21, t1: t1, t2: t2, t3: t3) - #expect(resultNegT3.q1 == -4) // d <= -t3 - } - - @Test("ScalarAccelerator quantization with different thresholds") - func scalarQuantizeDifferentThresholds() { - let accelerator = ScalarAccelerator() - - // Test with different threshold values - let result1 = accelerator.quantizeGradients(d1: 5, d2: 5, d3: 5, t1: 10, t2: 20, t3: 30) - #expect(result1.q1 == 1) // 0 < d < t1 - - let result2 = accelerator.quantizeGradients(d1: 15, d2: 15, d3: 15, t1: 10, t2: 20, t3: 30) - #expect(result2.q1 == 2) // t1 <= d < t2 - - let result3 = accelerator.quantizeGradients(d1: 25, d2: 25, d3: 25, t1: 10, t2: 20, t3: 30) - #expect(result3.q1 == 3) // t2 <= d < t3 - } - - @Test("Platform accelerator type conformance") - func platformAcceleratorConformance() { - // Test that ScalarAccelerator conforms to PlatformAccelerator - let scalar: any PlatformAccelerator = ScalarAccelerator() - #expect(scalar.computeGradients(a: 1, b: 2, c: 3).d1 == -1) - #expect(scalar.medPredictor(a: 1, b: 2, c: 3) == 1) // c <= min(a,b), so max(a,b) = 2... wait let me check - - // Test through protocol - let selected = selectPlatformAccelerator() - let gradients = selected.computeGradients(a: 5, b: 10, c: 8) - #expect(gradients.d1 == 2) // b - c = 10 - 8 - #expect(gradients.d2 == -3) // a - c = 5 - 8 - #expect(gradients.d3 == 3) // c - a = 8 - 5 - } - - @Test("ScalarAccelerator initialization") - func scalarInitialization() { - // Test that we can create multiple instances - let acc1 = ScalarAccelerator() - let acc2 = ScalarAccelerator() - - // Both should work identically - let result1 = acc1.computeGradients(a: 10, b: 20, c: 15) - let result2 = acc2.computeGradients(a: 10, b: 20, c: 15) - - #expect(result1.d1 == result2.d1) - #expect(result1.d2 == result2.d2) - #expect(result1.d3 == result2.d3) - } -} diff --git a/Tests/JPEGLSTests/VulkanMemoryCommandBufferTests.swift b/Tests/JPEGLSTests/VulkanMemoryCommandBufferTests.swift deleted file mode 100644 index fc70faf..0000000 --- a/Tests/JPEGLSTests/VulkanMemoryCommandBufferTests.swift +++ /dev/null @@ -1,415 +0,0 @@ -/// Tests for Vulkan memory management and command buffer types (Phase 15.2). -/// -/// These tests validate the CPU-side Vulkan abstraction types introduced in -/// Milestone 15.2: -/// - `VulkanBuffer`: typed read/write host-accessible buffer -/// - `VulkanMemoryPool`: pool-based buffer allocator -/// - `VulkanCommandBuffer`: command recording and introspection -/// - `VulkanCommandPool`: command buffer lifecycle management - -import Testing -@testable import JPEGLS - -// MARK: - VulkanBuffer Tests - -@Suite("VulkanBuffer Tests") -struct VulkanBufferTests { - - // MARK: Initialization - - @Test("VulkanBuffer initialises with correct size and usage") - func testInitSizeAndUsage() throws { - let buf = try VulkanBuffer(size: 1024, usage: .storageBuffer) - #expect(buf.size == 1024) - #expect(buf.usage == .storageBuffer) - } - - @Test("VulkanBuffer throws on zero size") - func testInitZeroSizeThrows() { - #expect(throws: (any Error).self) { - _ = try VulkanBuffer(size: 0, usage: .storageBuffer) - } - } - - @Test("VulkanBuffer combined usage flags are preserved") - func testInitCombinedUsage() throws { - let usage: VulkanBufferUsage = [.storageBuffer, .transferDst] - let buf = try VulkanBuffer(size: 256, usage: usage) - #expect(buf.usage.contains(.storageBuffer)) - #expect(buf.usage.contains(.transferDst)) - #expect(!buf.usage.contains(.uniformBuffer)) - } - - // MARK: Write / Read round-trips - - @Test("VulkanBuffer write+read round-trips Int32 array") - func testWriteReadInt32() throws { - let data: [Int32] = [10, 20, 30, 40, 50] - let buf = try VulkanBuffer( - size: data.count * MemoryLayout.stride, usage: .storageBuffer) - buf.write(data) - let result = buf.read(count: data.count, type: Int32.self) - #expect(result == data) - } - - @Test("VulkanBuffer write+read round-trips UInt8 array") - func testWriteReadUInt8() throws { - let data: [UInt8] = [0, 1, 127, 128, 255] - let buf = try VulkanBuffer(size: data.count, usage: .transferSrc) - buf.write(data) - let result = buf.read(count: data.count, type: UInt8.self) - #expect(result == data) - } - - @Test("VulkanBuffer write+read round-trips large Int32 array") - func testWriteReadLargeArray() throws { - let count = 65536 - let data = (0...stride, usage: .storageBuffer) - buf.write(data) - let result = buf.read(count: count, type: Int32.self) - #expect(result == data) - } - - @Test("VulkanBuffer second write overwrites first") - func testWriteOverwrite() throws { - let buf = try VulkanBuffer( - size: 4 * MemoryLayout.stride, usage: .storageBuffer) - buf.write([Int32](repeating: 0, count: 4)) - buf.write([Int32](repeating: 99, count: 4)) - let result = buf.read(count: 4, type: Int32.self) - #expect(result == [99, 99, 99, 99]) - } -} - -// MARK: - VulkanBufferUsage Tests - -@Suite("VulkanBufferUsage Tests") -struct VulkanBufferUsageTests { - - @Test("Individual flags have distinct raw values") - func testDistinctRawValues() { - let flags: [VulkanBufferUsage] = [ - .storageBuffer, .uniformBuffer, .transferSrc, .transferDst - ] - let rawValues = Set(flags.map { $0.rawValue }) - #expect(rawValues.count == flags.count) - } - - @Test("OptionSet union and intersection work correctly") - func testOptionSetOperations() { - let combined: VulkanBufferUsage = [.storageBuffer, .transferSrc] - #expect(combined.contains(.storageBuffer)) - #expect(combined.contains(.transferSrc)) - #expect(!combined.contains(.uniformBuffer)) - #expect(!combined.contains(.transferDst)) - } -} - -// MARK: - VulkanMemoryPool Tests - -@Suite("VulkanMemoryPool Tests") -struct VulkanMemoryPoolTests { - - // MARK: Initialization - - @Test("VulkanMemoryPool initialises with correct capacity") - func testInit() { - let pool = VulkanMemoryPool(maxPoolSize: 1024 * 1024) - #expect(pool.maxPoolSize == 1024 * 1024) - #expect(pool.totalAllocated == 0) - #expect(pool.bufferCount == 0) - } - - // MARK: Allocation - - @Test("VulkanMemoryPool allocates buffer and tracks usage") - func testAllocateSingleBuffer() throws { - let pool = VulkanMemoryPool(maxPoolSize: 1024) - let buf = try pool.allocate(size: 256, usage: .storageBuffer) - #expect(buf.size == 256) - #expect(pool.totalAllocated == 256) - #expect(pool.bufferCount == 1) - } - - @Test("VulkanMemoryPool allocates multiple buffers") - func testAllocateMultipleBuffers() throws { - let pool = VulkanMemoryPool(maxPoolSize: 4096) - _ = try pool.allocate(size: 1024, usage: .storageBuffer) - _ = try pool.allocate(size: 512, usage: .uniformBuffer) - _ = try pool.allocate(size: 256, usage: .transferSrc) - #expect(pool.totalAllocated == 1792) - #expect(pool.bufferCount == 3) - } - - @Test("VulkanMemoryPool throws when capacity is exceeded") - func testAllocateExceedsCapacity() throws { - let pool = VulkanMemoryPool(maxPoolSize: 512) - _ = try pool.allocate(size: 256, usage: .storageBuffer) - #expect(throws: (any Error).self) { - _ = try pool.allocate(size: 512, usage: .storageBuffer) - } - } - - @Test("VulkanMemoryPool allows allocations up to exact capacity") - func testAllocateExactCapacity() throws { - let pool = VulkanMemoryPool(maxPoolSize: 1024) - _ = try pool.allocate(size: 512, usage: .storageBuffer) - _ = try pool.allocate(size: 512, usage: .transferDst) - #expect(pool.totalAllocated == 1024) - } - - // MARK: Reset - - @Test("VulkanMemoryPool.reset() returns pool to empty state") - func testReset() throws { - let pool = VulkanMemoryPool(maxPoolSize: 2048) - _ = try pool.allocate(size: 512, usage: .storageBuffer) - _ = try pool.allocate(size: 512, usage: .transferSrc) - pool.reset() - #expect(pool.totalAllocated == 0) - #expect(pool.bufferCount == 0) - } - - @Test("VulkanMemoryPool can be reused after reset") - func testReuseAfterReset() throws { - let pool = VulkanMemoryPool(maxPoolSize: 512) - _ = try pool.allocate(size: 512, usage: .storageBuffer) - pool.reset() - let buf = try pool.allocate(size: 512, usage: .uniformBuffer) - #expect(buf.size == 512) - #expect(pool.totalAllocated == 512) - } - - // MARK: Data integrity - - @Test("VulkanMemoryPool: allocated buffers hold correct data") - func testAllocatedBufferDataIntegrity() throws { - let pool = VulkanMemoryPool(maxPoolSize: 4096) - let count = 256 - let data: [Int32] = (0...stride, usage: .storageBuffer) - buf.write(data) - let result = buf.read(count: count, type: Int32.self) - #expect(result == data) - } -} - -// MARK: - VulkanCommandBuffer Tests - -@Suite("VulkanCommandBuffer Tests") -struct VulkanCommandBufferTests { - - // MARK: Initial state - - @Test("VulkanCommandBuffer starts not recording with no commands") - func testInitialState() { - let buf = VulkanCommandBuffer() - #expect(!buf.isRecording) - #expect(buf.commandCount == 0) - #expect(buf.recordedCommands.isEmpty) - } - - // MARK: Recording lifecycle - - @Test("VulkanCommandBuffer begin() opens recording") - func testBeginOpensRecording() { - let buf = VulkanCommandBuffer() - buf.begin() - #expect(buf.isRecording) - } - - @Test("VulkanCommandBuffer end() closes recording") - func testEndClosesRecording() { - let buf = VulkanCommandBuffer() - buf.begin() - buf.end() - #expect(!buf.isRecording) - } - - @Test("VulkanCommandBuffer begin() clears prior commands") - func testBeginClearsPriorCommands() { - let buf = VulkanCommandBuffer() - buf.begin() - buf.dispatch(x: 16) - buf.end() - #expect(buf.commandCount == 1) - buf.begin() // second recording session - #expect(buf.commandCount == 0) - } - - // MARK: Recording commands - - @Test("VulkanCommandBuffer records bindPipeline") - func testRecordBindPipeline() { - let buf = VulkanCommandBuffer() - buf.begin() - buf.bindPipeline(name: "compute_gradients") - buf.end() - #expect(buf.commandCount == 1) - if case .bindPipeline(let name) = buf.recordedCommands[0].operation { - #expect(name == "compute_gradients") - } else { - Issue.record("Expected bindPipeline command") - } - } - - @Test("VulkanCommandBuffer records bindBuffer with binding index") - func testRecordBindBuffer() throws { - let buf = VulkanCommandBuffer() - let buffer = try VulkanBuffer(size: 256, usage: .storageBuffer) - buf.begin() - buf.bindBuffer(buffer, binding: 2) - buf.end() - #expect(buf.commandCount == 1) - if case .bindBuffer(_, let binding) = buf.recordedCommands[0].operation { - #expect(binding == 2) - } else { - Issue.record("Expected bindBuffer command") - } - } - - @Test("VulkanCommandBuffer records pushConstants") - func testRecordPushConstants() { - let buf = VulkanCommandBuffer() - let data: [UInt8] = [1, 0, 0, 0] // Int32(1) in little-endian - buf.begin() - buf.pushConstants(data: data) - buf.end() - #expect(buf.commandCount == 1) - if case .pushConstants(let d) = buf.recordedCommands[0].operation { - #expect(d == data) - } else { - Issue.record("Expected pushConstants command") - } - } - - @Test("VulkanCommandBuffer records dispatch with default y=1, z=1") - func testRecordDispatchDefaults() { - let buf = VulkanCommandBuffer() - buf.begin() - buf.dispatch(x: 64) - buf.end() - if case .dispatch(let x, let y, let z) = buf.recordedCommands[0].operation { - #expect(x == 64) - #expect(y == 1) - #expect(z == 1) - } else { - Issue.record("Expected dispatch command") - } - } - - @Test("VulkanCommandBuffer records dispatch with explicit y and z") - func testRecordDispatchExplicit() { - let buf = VulkanCommandBuffer() - buf.begin() - buf.dispatch(x: 8, y: 4, z: 2) - buf.end() - if case .dispatch(let x, let y, let z) = buf.recordedCommands[0].operation { - #expect(x == 8) - #expect(y == 4) - #expect(z == 2) - } else { - Issue.record("Expected dispatch command") - } - } - - @Test("VulkanCommandBuffer records a complete compute sequence") - func testRecordFullComputeSequence() throws { - let buf = VulkanCommandBuffer() - let inputBuf = try VulkanBuffer(size: 256, usage: .storageBuffer) - let outputBuf = try VulkanBuffer(size: 256, usage: .storageBuffer) - - buf.begin() - buf.bindPipeline(name: "compute_quantize_gradients") - buf.bindBuffer(inputBuf, binding: 0) - buf.bindBuffer(outputBuf, binding: 1) - buf.pushConstants(data: [9, 0, 0, 0]) // count = 9, as Int32 little-endian - buf.dispatch(x: 1) - buf.end() - - #expect(buf.commandCount == 5) - #expect(!buf.isRecording) - } - - // MARK: Commands ignored when not recording - - @Test("VulkanCommandBuffer ignores commands when not recording") - func testCommandsIgnoredWhenNotRecording() throws { - let buf = VulkanCommandBuffer() - let buffer = try VulkanBuffer(size: 64, usage: .storageBuffer) - // These should all be silently ignored - buf.bindPipeline(name: "ignored") - buf.bindBuffer(buffer, binding: 0) - buf.pushConstants(data: [0]) - buf.dispatch(x: 1) - #expect(buf.commandCount == 0) - } -} - -// MARK: - VulkanCommandPool Tests - -@Suite("VulkanCommandPool Tests") -struct VulkanCommandPoolTests { - - @Test("VulkanCommandPool starts empty") - func testInitEmpty() { - let pool = VulkanCommandPool() - #expect(pool.allocatedCount == 0) - } - - @Test("VulkanCommandPool allocates command buffers") - func testAllocate() { - let pool = VulkanCommandPool() - let buf1 = pool.allocate() - let buf2 = pool.allocate() - #expect(pool.allocatedCount == 2) - // Each allocation returns a distinct object - #expect(buf1 !== buf2) - } - - @Test("VulkanCommandPool reset() releases all buffers") - func testReset() { - let pool = VulkanCommandPool() - _ = pool.allocate() - _ = pool.allocate() - _ = pool.allocate() - pool.reset() - #expect(pool.allocatedCount == 0) - } - - @Test("VulkanCommandPool can be reused after reset") - func testReuseAfterReset() { - let pool = VulkanCommandPool() - _ = pool.allocate() - pool.reset() - let buf = pool.allocate() - #expect(pool.allocatedCount == 1) - // The returned buffer should be usable - buf.begin() - buf.dispatch(x: 4) - buf.end() - #expect(buf.commandCount == 1) - } - - @Test("VulkanCommandPool allocated buffers are independently usable") - func testAllocatedBuffersAreIndependent() { - let pool = VulkanCommandPool() - let buf1 = pool.allocate() - let buf2 = pool.allocate() - - buf1.begin() - buf1.dispatch(x: 8) - buf1.end() - - buf2.begin() - buf2.bindPipeline(name: "test") - buf2.dispatch(x: 16) - buf2.end() - - #expect(buf1.commandCount == 1) - #expect(buf2.commandCount == 2) - } -} diff --git a/Tests/JPEGLSTests/VulkanPerformanceBenchmarks.swift b/Tests/JPEGLSTests/VulkanPerformanceBenchmarks.swift deleted file mode 100644 index e1c27b2..0000000 --- a/Tests/JPEGLSTests/VulkanPerformanceBenchmarks.swift +++ /dev/null @@ -1,415 +0,0 @@ -/// Vulkan GPU compute performance benchmarks (Phase 15.3). -/// -/// These benchmarks measure the performance of `VulkanAccelerator` operations -/// via the CPU-fallback path, which executes the same algorithms that the -/// Vulkan GPU path will use. The results document CPU-path performance -/// characteristics and serve as a baseline for future GPU comparison once -/// the Vulkan SDK integration is complete. -/// -/// All benchmarks exercise the full configuration matrix: -/// - Multiple image sizes (small, medium, large, very large) -/// - All accelerated operations (gradients, MED prediction, quantisation, -/// colour transforms) -/// -/// These tests run on all platforms (no Vulkan SDK required). - -import Testing -@testable import JPEGLS -import Foundation - -@Suite("Vulkan CPU-Fallback Performance Benchmarks") -struct VulkanPerformanceBenchmarks { - - // MARK: - Configuration - - private static let warmupIterations = 2 - private static let benchmarkIterations = 5 - - // MARK: - Helpers - - private func measure( - iterations: Int = VulkanPerformanceBenchmarks.benchmarkIterations, - _ block: () -> Void - ) -> (min: TimeInterval, max: TimeInterval, avg: TimeInterval) { - var times: [TimeInterval] = [] - for _ in 0.. (a: [Int32], b: [Int32], c: [Int32]) { - var a = [Int32](repeating: 0, count: count) - var b = [Int32](repeating: 0, count: count) - var c = [Int32](repeating: 0, count: count) - for i in 0...size) / (1024 * 1024) - print(""" - - Vulkan Gradients 2048×2048: - Pixels: \(count) - Data: \(String(format: "%.1f", dataSize)) MB - Avg time: \(String(format: "%.3f", avgTime * 1000)) ms - Min/Max: \(String(format: "%.3f", mnTime * 1000)) / \(String(format: "%.3f", mxTime * 1000)) ms - Throughput: \(String(format: "%.2f", throughput)) Mpix/s (CPU fallback) - """) - } - - // MARK: - MED Prediction Benchmarks - - @Test("Benchmark: Vulkan MED prediction — 512×512") - func benchmarkMEDPrediction512x512() { - let accelerator = VulkanAccelerator() - let count = 512 * 512 - let (a, b, c) = makePixelData(count: count) - for _ in 0...size) / (1024 * 1024) - print(""" - - Vulkan HP3 Colour Transform (forward) 2048×2048: - Pixels: \(count) - Data: \(String(format: "%.1f", dataSize)) MB - Avg time: \(String(format: "%.3f", avgTime * 1000)) ms - Min/Max: \(String(format: "%.3f", mnTime * 1000)) / \(String(format: "%.3f", mxTime * 1000)) ms - Throughput: \(String(format: "%.2f", throughput)) Mpix/s (CPU fallback) - """) - } - - // MARK: - Memory Management Benchmarks - - @Test("Benchmark: VulkanMemoryPool allocation and reset") - func benchmarkMemoryPoolAllocationReset() { - let (mnTime, mxTime, avgTime) = measure { - let pool = VulkanMemoryPool(maxPoolSize: 64 * 1024 * 1024) - // Simulate allocating buffers for a 512×512 3-channel operation - let pixelCount = 512 * 512 - let byteCount = pixelCount * MemoryLayout.stride - _ = try? pool.allocate(size: byteCount, usage: .storageBuffer) // a - _ = try? pool.allocate(size: byteCount, usage: .storageBuffer) // b - _ = try? pool.allocate(size: byteCount, usage: .storageBuffer) // c - _ = try? pool.allocate(size: byteCount, usage: .storageBuffer) // output - pool.reset() - } - print(""" - - VulkanMemoryPool alloc+reset (4 × 512×512 buffers): - Avg time: \(String(format: "%.3f", avgTime * 1000)) ms - Min/Max: \(String(format: "%.3f", mnTime * 1000)) / \(String(format: "%.3f", mxTime * 1000)) ms - """) - } - - @Test("Benchmark: VulkanBuffer write and read round-trip") - func benchmarkBufferWriteReadRoundTrip() { - let count = 512 * 512 - let pixels = (0...stride, usage: .storageBuffer) - buf.write(pixels) - _ = buf.read(count: count, type: Int32.self) - } - print(""" - - VulkanBuffer write+read (512×512 Int32): - Elements: \(count) - Avg time: \(String(format: "%.3f", avgTime * 1000)) ms - Min/Max: \(String(format: "%.3f", mnTime * 1000)) / \(String(format: "%.3f", mxTime * 1000)) ms - """) - // Correctness check - let buf = try! VulkanBuffer( - size: count * MemoryLayout.stride, usage: .transferSrc) - buf.write(pixels) - let result = buf.read(count: count, type: Int32.self) - #expect(result == pixels) - } - - // MARK: - Encoding/Decoding Pipeline Benchmarks - - @Test("Benchmark: Vulkan encoding pipeline — 512×512") - func benchmarkEncodingPipeline512x512() { - let accelerator = VulkanAccelerator() - let count = 512 * 512 - let (a, b, c) = makePixelData(count: count) - let x = (0...size) / (1024 * 1024) - print(""" - - Vulkan Encoding Pipeline 2048×2048: - Pixels: \(count) - Data: \(String(format: "%.1f", dataSize)) MB - Avg time: \(String(format: "%.3f", avgTime * 1000)) ms - Min/Max: \(String(format: "%.3f", mnTime * 1000)) / \(String(format: "%.3f", mxTime * 1000)) ms - Throughput: \(String(format: "%.2f", throughput)) Mpix/s (CPU fallback) - """) - } -} diff --git a/Tests/JPEGLSTests/X86_64AcceleratorPhase14Tests.swift b/Tests/JPEGLSTests/X86_64AcceleratorPhase14Tests.swift deleted file mode 100644 index f0b9a47..0000000 --- a/Tests/JPEGLSTests/X86_64AcceleratorPhase14Tests.swift +++ /dev/null @@ -1,205 +0,0 @@ -/// Tests for the Phase 14.1 x86-64 SSE/AVX enhancements. -/// -/// These tests cover the Golomb-Rice parameter computation, run-length -/// detection, and byte stuffing detection added to X86_64Accelerator as -/// part of Milestone 14 Phase 14.1. -/// -/// All tests are compiled and run only on x86-64 architectures where -/// the X86_64Accelerator is available. - -#if arch(x86_64) - -import Testing -import Foundation -@testable import JPEGLS - -@Suite("X86_64 Accelerator Phase 14.1 Tests") -struct X86_64AcceleratorPhase14Tests { - - // MARK: - Golomb-Rice Parameter Computation - - @Test("Golomb-Rice parameter is zero when a is zero") - func golombRiceParamZeroA() { - let acc = X86_64Accelerator() - #expect(acc.computeGolombRiceParameter(a: 0, n: 64) == 0) - } - - @Test("Golomb-Rice parameter is zero when n is zero") - func golombRiceParamZeroN() { - let acc = X86_64Accelerator() - #expect(acc.computeGolombRiceParameter(a: 100, n: 0) == 0) - } - - @Test("Golomb-Rice parameter is zero when a <= n (small error accumulator)") - func golombRiceParamSmallA() { - let acc = X86_64Accelerator() - // When a <= n, k should be 0 (threshold n*1 >= a) - #expect(acc.computeGolombRiceParameter(a: 10, n: 64) == 0) - } - - @Test("Golomb-Rice parameter increases with larger a relative to n") - func golombRiceParamIncreases() { - let acc = X86_64Accelerator() - let k1 = acc.computeGolombRiceParameter(a: 64, n: 64) // a/n = 1 - let k2 = acc.computeGolombRiceParameter(a: 128, n: 64) // a/n = 2 - let k3 = acc.computeGolombRiceParameter(a: 512, n: 64) // a/n = 8 - // k should be non-decreasing as a grows - #expect(k1 <= k2) - #expect(k2 <= k3) - } - - @Test("Golomb-Rice parameter satisfies 2^k * n >= a") - func golombRiceParamSatisfiesCondition() { - let acc = X86_64Accelerator() - let n = 64 - for a in [64, 128, 256, 512, 1024, 2048] { - let k = acc.computeGolombRiceParameter(a: a, n: n) - // Primary condition: 2^k * n >= a - #expect((n << k) >= a, "k=\(k) for a=\(a) n=\(n): 2^k*n should be >= a") - // Minimality: if k > 0, then 2^(k-1) * n < a - if k > 0 { - #expect((n << (k - 1)) < a, "k=\(k) should be minimal for a=\(a) n=\(n)") - } - } - } - - @Test("Golomb-Rice parameter is bounded within [0, 31]") - func golombRiceParamBounded() { - let acc = X86_64Accelerator() - let k = acc.computeGolombRiceParameter(a: Int.max / 2, n: 1) - #expect(k >= 0) - #expect(k <= 31) - } - - // MARK: - Run-Length Detection - - @Test("Run-length detection returns 0 for empty slice") - func runLengthEmpty() { - let acc = X86_64Accelerator() - #expect(acc.detectRunLength(in: [], startIndex: 0, runValue: 0, maxLength: 100) == 0) - } - - @Test("Run-length detection with maxLength 0 returns 0") - func runLengthMaxLengthZero() { - let acc = X86_64Accelerator() - let pixels: [Int32] = [10, 10, 10] - #expect(acc.detectRunLength(in: pixels, startIndex: 0, runValue: 10, maxLength: 0) == 0) - } - - @Test("Run-length detection counts full run of equal pixels") - func runLengthFullRun() { - let acc = X86_64Accelerator() - let pixels: [Int32] = [5, 5, 5, 5, 5] - let length = acc.detectRunLength(in: pixels, startIndex: 0, runValue: 5, maxLength: 100) - #expect(length == 5) - } - - @Test("Run-length detection stops at first mismatch") - func runLengthStopsAtMismatch() { - let acc = X86_64Accelerator() - let pixels: [Int32] = [10, 10, 10, 20, 10] - let length = acc.detectRunLength(in: pixels, startIndex: 0, runValue: 10, maxLength: 100) - #expect(length == 3) - } - - @Test("Run-length detection respects maxLength") - func runLengthRespectMaxLength() { - let acc = X86_64Accelerator() - let pixels: [Int32] = [7, 7, 7, 7, 7, 7, 7, 7] - let length = acc.detectRunLength(in: pixels, startIndex: 0, runValue: 7, maxLength: 4) - #expect(length == 4) - } - - @Test("Run-length detection can start from non-zero index") - func runLengthStartIndex() { - let acc = X86_64Accelerator() - let pixels: [Int32] = [1, 2, 3, 3, 3, 3, 4] - let length = acc.detectRunLength(in: pixels, startIndex: 2, runValue: 3, maxLength: 100) - #expect(length == 4) - } - - @Test("Run-length detection with single matching element") - func runLengthSingleMatch() { - let acc = X86_64Accelerator() - let pixels: [Int32] = [99] - #expect(acc.detectRunLength(in: pixels, startIndex: 0, runValue: 99, maxLength: 100) == 1) - } - - @Test("Run-length detection returns 0 when first element mismatches") - func runLengthFirstMismatch() { - let acc = X86_64Accelerator() - let pixels: [Int32] = [5, 5, 5] - #expect(acc.detectRunLength(in: pixels, startIndex: 0, runValue: 9, maxLength: 100) == 0) - } - - @Test("Run-length detection across SIMD vector boundary") - func runLengthAcrossVectorBoundary() { - let acc = X86_64Accelerator() - // 8 matching + 2 more = run of 10, crossing the 8-element SIMD boundary - let pixels = [Int32](repeating: 42, count: 10) + [Int32](repeating: 0, count: 5) - let length = acc.detectRunLength(in: pixels, startIndex: 0, runValue: 42, maxLength: 100) - #expect(length == 10) - } - - // MARK: - Byte Stuffing Detection - - @Test("Byte stuffing detection returns empty for non-0xFF data") - func byteStuffingNone() { - let acc = X86_64Accelerator() - let data: [UInt8] = [0x00, 0x01, 0x7F, 0xFE, 0x80, 0x55] - #expect(acc.detectByteStuffingPositions(in: data).isEmpty) - } - - @Test("Byte stuffing detection finds single 0xFF") - func byteStuffingSingle() { - let acc = X86_64Accelerator() - let data: [UInt8] = [0x00, 0xFF, 0x01] - let positions = acc.detectByteStuffingPositions(in: data) - #expect(positions == [1]) - } - - @Test("Byte stuffing detection finds multiple 0xFF bytes") - func byteStuffingMultiple() { - let acc = X86_64Accelerator() - let data: [UInt8] = [0xFF, 0x00, 0xFF, 0x7F, 0xFF] - let positions = acc.detectByteStuffingPositions(in: data) - #expect(positions == [0, 2, 4]) - } - - @Test("Byte stuffing detection handles empty data") - func byteStuffingEmpty() { - let acc = X86_64Accelerator() - #expect(acc.detectByteStuffingPositions(in: []).isEmpty) - } - - @Test("Byte stuffing detection handles all-0xFF data") - func byteStuffingAllFF() { - let acc = X86_64Accelerator() - let data = [UInt8](repeating: 0xFF, count: 16) - let positions = acc.detectByteStuffingPositions(in: data) - #expect(positions == Array(0..<16)) - } - - @Test("Byte stuffing detection crosses SIMD boundary") - func byteStuffingCrossesVectorBoundary() { - let acc = X86_64Accelerator() - // 0xFF at index 7 (last of first SIMD chunk) and 8 (first of second) - var data = [UInt8](repeating: 0x00, count: 16) - data[7] = 0xFF - data[8] = 0xFF - let positions = acc.detectByteStuffingPositions(in: data) - #expect(positions == [7, 8]) - } - - @Test("Byte stuffing detection in tail (count not multiple of 8)") - func byteStuffingInTail() { - let acc = X86_64Accelerator() - // 9 bytes: 0xFF is at index 8 (the tail byte after the first SIMD chunk) - var data = [UInt8](repeating: 0x00, count: 9) - data[8] = 0xFF - let positions = acc.detectByteStuffingPositions(in: data) - #expect(positions == [8]) - } -} - -#endif // arch(x86_64) diff --git a/docs/DICOMKIT_INTEGRATION.md b/docs/DICOMKIT_INTEGRATION.md index ffcb434..dd072a0 100644 --- a/docs/DICOMKIT_INTEGRATION.md +++ b/docs/DICOMKIT_INTEGRATION.md @@ -26,7 +26,7 @@ This guide demonstrates how to integrate JLSwift JPEG-LS compression into DICOM - [Transcoding Pipeline](#transcoding-pipeline) - [Performance Considerations](#performance-considerations) - [Buffer Pooling for Batch Processing](#buffer-pooling-for-batch-processing) - - [Tile-Based Processing for Large Images](#tile-based-processing-for-large-images) + - [Restart-Interval Parallelism for Large Images](#restart-interval-parallelism-for-large-images) - [Memory Management](#memory-management) - [Error Handling](#error-handling) - [Testing DICOM Integration](#testing-dicom-integration) @@ -44,7 +44,7 @@ JLSwift implements the full JPEG-LS standard (ISO/IEC 14495-1:1999 / ITU-T.87) i - **No C dependencies** — simplifies deployment and auditing - **Apple Silicon optimised** — ARM NEON/SIMD acceleration -- **Memory efficient** — buffer pooling and tile-based processing for large images +- **Memory efficient** — internal buffer pooling; restart-interval parallelism for large images - **Standards compliant** — full support for all JPEG-LS interleaving modes and colour transforms ## Prerequisites @@ -447,7 +447,7 @@ import JPEGLS /// Encode CR/DX pixel data with JPEG-LS /// /// CR/DX images are high-resolution (e.g., 3000x3000) with 10-14 bits stored. -/// Tile-based processing is recommended for large images. +/// Restart-interval parallelism is recommended for large images. func encodeCRImage( pixelData: [[Int]], rows: Int, @@ -695,36 +695,35 @@ func processDICOMSeries( } ``` -### Tile-Based Processing for Large Images +### Restart-Interval Parallelism for Large Images -Use tile-based processing for very large images (e.g., digital pathology): +For very large frames (e.g., digital pathology), parallelise a single image +across cores with restart markers (ITU-T.87 DRI/RSTm). There is no tiling +API — the codec decodes each scan into one flat pixel plane; restart +intervals are the parallelism and error-resilience mechanism: ```swift import JPEGLS -/// Process a large DICOM image using tile-based approach +/// Encode a large DICOM frame with restart-interval parallelism /// -/// Tile-based processing reduces peak memory usage by encoding -/// the image in smaller tiles. -func processLargeDICOMImage( - rows: Int, - columns: Int, - bytesPerPixel: Int -) -> (tiles: [TileRegion], memorySavings: Double) { - let processor = JPEGLSTileProcessor( - imageWidth: columns, - imageHeight: rows, - configuration: TileConfiguration( - tileWidth: 512, - tileHeight: 512, - overlap: 4 // Overlap for boundary handling - ) +/// The encoder writes a DRI marker and emits RSTm markers every +/// `restartInterval` lines, making the intervals independently codable +/// so they can be processed in parallel. Supported for lossless +/// (NEAR = 0), non-interleaved scans. +func encodeLargeDICOMFrame( + pixelData: [[Int]], + bitsStored: Int +) throws -> Data { + let imageData = try MultiComponentImageData.grayscale( + pixels: pixelData, + bitsPerSample: bitsStored ) - let tiles = processor.calculateTilesWithOverlap() - let savings = processor.estimateMemorySavings(bytesPerPixel: bytesPerPixel) - - return (tiles, savings) + // Large frames: parallelise a single image across cores with restart markers + let config = try JPEGLSEncoder.Configuration(restartInterval: 256) + return try JPEGLSEncoder().encode(imageData, configuration: config) + // Decoding splits at the RST markers automatically and decodes intervals concurrently. } ``` @@ -739,28 +738,17 @@ import JPEGLS /// /// 1. Buffer pooling is handled internally by the encoder /// 2. Process frames sequentially to limit peak memory -/// 3. Use tile-based processing for large single images +/// 3. Use restart intervals to parallelise large single frames /// 4. Release image data promptly after encoding +/// +/// Note: the codec decodes each scan into one flat UInt16 pixel plane, so +/// peak memory scales with rows × columns × components. func processWithMemoryEfficiency( pixelData: [[Int]], rows: Int, columns: Int, bitsStored: Int ) throws -> Data { - // Use cache-friendly buffer for better CPU performance - let cacheBuffer = JPEGLSCacheFriendlyBuffer( - width: columns, - height: rows, - initialValue: 0 - ) - - // Populate cache-friendly buffer - for row in 0..= 1024 * 1024 { - let metalAccelerator = try? MetalAccelerator() - // Use metalAccelerator for tile processing -} -#endif -``` - -### Error Handling - -```swift -do { - let accelerator = try MetalAccelerator() - let results = try accelerator.computeGradientsBatch(a: a, b: b, c: c) -} catch MetalAcceleratorError.metalNotAvailable { - print("Metal not supported on this device") -} catch MetalAcceleratorError.commandBufferExecutionFailed { - print("GPU execution failed, falling back to CPU") -} catch { - print("Unexpected error: \(error)") -} -``` - -## Performance Characteristics - -### GPU Advantages - -- **Massive Parallelism**: Process thousands of pixels simultaneously -- **Vectorised Operations**: SIMD operations on pixel data -- **Memory Bandwidth**: High-bandwidth GPU memory for large datasets - -### GPU Overhead - -- **Buffer Creation**: Allocating GPU buffers for input/output -- **Data Transfer**: CPU ↔ GPU memory transfers -- **Synchronization**: Waiting for GPU command completion - -### Optimal Use Cases - -✅ **Good for GPU:** -- Large images (2048×2048 or larger) -- Batch processing of multiple images -- High-resolution medical imaging (4K, 8K) -- Video frame processing - -❌ **Better on CPU:** -- Small images (< 512×512) -- Single-pixel operations -- Real-time interactive editing -- Memory-constrained environments - -## Platform Support - -### Requirements - -- **macOS 10.13+** (High Sierra or later) -- **iOS 11+** -- **tvOS 11+** -- **Mac with GPU**: Apple Silicon (M1/M2/M3) or Intel Mac with discrete GPU - -### Conditional Compilation - -Metal code is conditionally compiled only on supported platforms: - -```swift -#if canImport(Metal) -// Metal GPU code -#endif -``` - -This ensures the library builds on Linux and other non-Apple platforms without Metal. - -## Implementation Details - -### Compute Shaders - -The Metal shaders implement nine operations across three categories: - -**Gradient Computation and Prediction:** - -1. **Gradient Computation** (`compute_gradients`) -2. **MED Prediction** (`compute_med_prediction`) -3. **Gradient Quantisation** (`compute_quantize_gradients`) — maps each gradient to [-4, 4] using T1/T2/T3 thresholds - -**Colour Space Transformations:** - -4. **HP1 Forward** (`compute_colour_transform_hp1_forward`) — R′=R−G, G′=G, B′=B−G -5. **HP1 Inverse** (`compute_colour_transform_hp1_inverse`) — R=R′+G′, G=G′, B=B′+G′ -6. **HP2 Forward** (`compute_colour_transform_hp2_forward`) — R′=R−G, G′=G, B′=B−((R+G)>>1) -7. **HP2 Inverse** (`compute_colour_transform_hp2_inverse`) -8. **HP3 Forward** (`compute_colour_transform_hp3_forward`) — B′=B, R′=R−B, G′=G−((R+B)>>1) -9. **HP3 Inverse** (`compute_colour_transform_hp3_inverse`) - -### Thread Group Configuration - -The accelerator dynamically calculates optimal thread group sizes based on: -- Pipeline state maximum threads per threadgroup -- Batch size -- GPU capabilities - -```swift -let threadGroupSize = MTLSize( - width: min(pipelineState.maxTotalThreadsPerThreadgroup, count), - height: 1, - depth: 1 -) -``` - -### Memory Management - -- **Shared Memory Mode**: Uses `.storageModeShared` for zero-copy access on Apple Silicon -- **Unified Memory**: Leverages Apple Silicon unified memory architecture -- **Automatic Buffer Management**: Buffers are automatically released after use - -## Benchmarking - -### Measuring Performance - -```swift -let accelerator = try MetalAccelerator() -let startTime = Date() - -let (d1, d2, d3) = try accelerator.computeGradientsBatch(a: a, b: b, c: c) - -let elapsed = Date().timeIntervalSince(startTime) -let throughput = Double(a.count) / elapsed / 1_000_000.0 // Mpixels/s -print("Throughput: \(throughput) Mpixels/s") -``` - -### Expected Performance (Apple Silicon M1) - -| Image Size | Pixels | GPU Throughput | CPU Throughput | Speedup | -|-----------|--------|----------------|----------------|---------| -| 512×512 | 262K | ~50 Mpixels/s | ~40 Mpixels/s | 1.25× | -| 1024×1024 | 1M | ~200 Mpixels/s | ~50 Mpixels/s | 4× | -| 2048×2048 | 4M | ~500 Mpixels/s | ~50 Mpixels/s | 10× | -| 4096×4096 | 16M | ~800 Mpixels/s | ~50 Mpixels/s | 16× | - -*Note: Actual performance varies by device, image content, and system load.* - -## Testing - -The Metal accelerator includes comprehensive tests: - -```bash -# Run Metal-specific tests (macOS/iOS only) -swift test --filter MetalAcceleratorTests - -# Run Phase 15 GPU compute tests -swift test --filter MetalPhase15Tests -``` - -Tests verify: -- ✅ Initialisation and device detection -- ✅ Gradient computation correctness -- ✅ MED prediction correctness -- ✅ Gradient quantisation correctness (Phase 15.1) -- ✅ HP1 forward and inverse colour transforms (Phase 15.1) -- ✅ HP2 forward and inverse colour transforms (Phase 15.1) -- ✅ HP3 forward and inverse colour transforms (Phase 15.1) -- ✅ CPU fallback for small batches -- ✅ GPU execution for large batches -- ✅ Bit-exact match between Metal GPU and Vulkan CPU fallback -- ✅ Round-trip correctness (forward → inverse restores original) -- ✅ Edge cases and boundary values -- ✅ Error handling - -## Future Enhancements - -Potential improvements for future versions: - -1. **Persistent Command Buffers**: Reuse command buffers for repeated operations -2. **Async Execution**: Non-blocking GPU operations with completion handlers -3. **Multi-GPU Support**: Distribute work across multiple GPUs -4. **Metal Performance Shaders**: Leverage MPS for additional optimisations -5. **Context Quantisation**: GPU-accelerated context computation -6. **Full Pipeline**: End-to-end encoding on GPU - -## Troubleshooting - -### Metal Not Available - -**Problem**: `MetalAcceleratorError.metalNotAvailable` - -**Solutions**: -- Verify running on macOS 10.13+ or iOS 11+ -- Check that device has GPU (use `MTLCreateSystemDefaultDevice()`) -- For VMs or CI: Metal may not be available in virtualized environments - -### Command Buffer Execution Failed - -**Problem**: `MetalAcceleratorError.commandBufferExecutionFailed` - -**Solutions**: -- Check GPU memory availability -- Reduce batch size if hitting memory limits -- Verify shader compilation succeeded -- Check system console for GPU errors - -### Performance Not Improving - -**Problem**: GPU slower than CPU - -**Possible causes**: -- Batch size too small (< 1024 pixels) - GPU overhead dominates -- System under heavy GPU load -- Thermal throttling on mobile devices -- Data transfer overhead for repeated small operations - -**Solutions**: -- Increase batch size (process larger tiles) -- Batch multiple operations together -- Profile with Instruments to identify bottlenecks -- Consider CPU-only mode for small images - -## References - -- **Metal Programming Guide**: https://developer.apple.com/metal/ -- **Metal Shading Language Specification**: https://developer.apple.com/metal/Metal-Shading-Language-Specification.pdf -- **JPEG-LS Standard**: ISO/IEC 14495-1:1999 / ITU-T.87 -- **Apple Silicon Performance**: https://developer.apple.com/documentation/apple-silicon - -## Contributing - -When contributing to Metal GPU acceleration: - -1. Ensure bit-exact results match CPU implementations -2. Add comprehensive tests for new operations -3. Profile performance on various image sizes -4. Update documentation with benchmarks -5. Test on both Intel and Apple Silicon Macs -6. Verify conditional compilation works on non-Apple platforms diff --git a/docs/MILESTONES.md b/docs/MILESTONES.md index 3b2f041..2a7b04e 100644 --- a/docs/MILESTONES.md +++ b/docs/MILESTONES.md @@ -220,7 +220,7 @@ Native Swift implementation of JPEG-LS (ISO/IEC 14495-1:1999 / ITU-T.87) compres **Implementation Details:** - Created `JPEGLSBufferPool` for reusable buffer management with thread-safe operations -- Implemented `JPEGLSTileProcessor` for dividing large images into manageable tiles +- Implemented `JPEGLSTileProcessor` for dividing large images into manageable tiles (removed in 0.9.0 — superseded by restart-interval parallelism) - Developed `JPEGLSCacheFriendlyBuffer` with contiguous memory layout for better cache performance - All implementations include comprehensive test suites with 49 total tests - Buffer pooling reduces allocation overhead for context arrays and pixel data diff --git a/docs/OPTIMIZATION_PLAN.md b/docs/OPTIMIZATION_PLAN.md new file mode 100644 index 0000000..e708662 --- /dev/null +++ b/docs/OPTIMIZATION_PLAN.md @@ -0,0 +1,163 @@ +# JLSwift Optimization Plan + +Synthesis of six verified analysis lenses + measured baseline. Scope: pure-Swift JPEG-LS codec (`Sources/JPEGLS`), lossless correctness non-negotiable. All file:line references verified against HEAD (`bcf00eb`). + +--- + +## 1. The performance story + +**Where time goes today.** Baseline (release, Apple Silicon): real DICOM encode **26.4 MB/s**, decode **40.1 MB/s** aggregate (CT 21.8/30.8, MG 33.1/53.1); synthetic 16-bit 2048² encode 37.3 MB/s, decode 58.0 MB/s. CharLS-class C++ does 200–400+ MB/s single-threaded — a 5–10x gap. + +The profile (8,482 samples, 150-iteration 16-bit roundtrip; full report was at `/tmp/jlswift-sample.txt`) shows the gap is **Swift mechanics, not JPEG-LS math**: + +| Cost | Share | Root cause | +|---|---|---| +| `Array._checkSubscript` | ~15% | `[[Int]]` pixel storage, double bounds checks everywhere | +| Encoder loop body | ~16% | includes per-pixel `getNeighbors` w/ Dictionary lookup + boundary branches | +| Decoder loop body | ~12% | same pattern, plus per-pixel `[[Int]]` writes | +| `updateContext` + CoW checks | ~15% | 4 parallel `[Int]` arrays, uniqueness check per store | +| `quantizeGradient` (decoder, outlined) | ~6.5% | 8-branch chain called 3x/pixel; encoder has a LUT, decoder doesn't | +| `Hasher._hash` + `find` | ~6.5% | `Dictionary` subscript **per pixel** in the encoder | +| Bitstream read/write leaves | ~6.6% | per-byte Foundation `Data` append/subscript, per-bit unary reads | + +Corroboration: a measured `-Ounchecked` A/B gave **+65–85% encode, +35–48% decode** — i.e., checks alone are a third to half the runtime. A writer microbench measured `Data.append(UInt8)` at **38.9 ns/byte vs 0.53 ns** for `[UInt8]` (~70x), against a total per-pixel budget of ~20–120 ns. + +**Realistic end state.** Phase 1 (days): ~1.5–2x → real-DICOM encode ~40–55 MB/s. Phase 2 (structural, 1–2 weeks): cumulative 3–6x → **100–200+ MB/s single-threaded**, i.e., low end of CharLS-class. Phase 3 adds multicore wall-clock wins (restart-interval stripes, batch parallelism), not single-thread MB/s. The core scanline loop is inherently sequential (causal prediction + adaptive Golomb + run mode); nothing below violates that. + +**Invariant for all phases:** every change in Phases 1–2 is representational — it must produce **byte-identical encoded streams and pixel-identical decodes**. Gate every merge on the golden-hash check + bench-dicom lossless round-trip (§6). + +--- + +## 2. Phase 1 — Quick wins (days each, ordered by impact-per-effort) + +### W1.1 Bitstream writer: `[UInt8]` backing + 64-bit accumulator +*Merges: encoder/memory/swiftperf writer findings.* +- **Change:** `Sources/JPEGLS/Core/JPEGLSBitstreamWriter.swift:12-14, 125-154, 187-216`. Replace `Data` + `UInt32 bitBuffer` with a pre-reserved `[UInt8]` (capacity already estimated at `JPEGLSEncoder.swift:148-152`) and a `UInt64` accumulator so a whole Golomb code (unary prefix + k-bit remainder) packs in one call. Flush 8 bytes at a time with a single no-0xFF-byte word test, falling back to byte-wise stuffing when 0xFF is present. Convert to `Data` once in `getData()`. +- **Safety:** ISO 14495-1 §9.1 stuffing depends only on each emitted byte being 0xFF (current rule at line 149), so bulk flush gated on the no-0xFF test is bit-exact — verified bit-identical in the microbench. Preserve `endMarkerSegment`'s by-index patch (lines 250-259). +- **Impact:** measured 5.4x ([UInt8] swap alone) to 8.5x (full 64-bit) on the writer path; end-to-end ~1.3–2x encode on poorly-compressing modalities (CT/DX/PX at 2–3.5:1, ~0.5–1 output byte/pixel), single digits at 25:1. The backing-store swap alone captures most of the win if time is short. +- **Verify:** golden encode hashes identical; bench-dicom round-trip; synthetic 16-bit encode MB/s. + +### W1.2 Hoist the encoder's per-pixel Dictionary lookup +*Merges: encoder/memory/swiftperf getNeighbors-dictionary findings (minimal version; full neighbor-carrying is W2.2).* +- **Change:** `JPEGLSEncoder.swift:698, 730, 771, 785, 897, 1105, 1139` + `Encoder/JPEGLSPixelBuffer.swift:275-338`. Resolve `componentPixels[componentId]` **once per scan** before the row loop (mirror the decoder, whose private `getNeighbors(pixels: [[Int]], ...)` at `JPEGLSDecoder.swift:1076` already takes the array directly); hoist the current/previous row arrays once per line; kill the per-run `getComponentPixels` lookups at 730/785. +- **Safety:** pure mechanical hoist — identical values read in identical order; component is fixed for the scan. +- **Impact:** removes the ~6.5% Hasher+find cost plus per-pixel ARC retain/release on the returned `[[Int]]`; est. 10–15% encode. +- **Verify:** golden hashes; bench-dicom. + +### W1.3 Encoder: quantize gradients once per pixel +- **Change:** `JPEGLSEncoder.swift:721-724` (and 920-923, 1115-1118) compute q1/q2/q3 for the run-entry test; `Encoder/JPEGLSRegularMode.swift:375-384` recomputes them. Add an `encodePixel` overload taking precomputed (q1,q2,q3); have `computeContextIndex` return (index, sign) in one call (`Core/JPEGLSContextModel.swift:161-177`). For near==0, the run-entry test is exactly `a==b && b==c && b==d`. +- **Safety:** identical inputs → identical quantized values; the lossless shortcut equivalence is verified from the table construction (`JPEGLSRegularMode.swift:100-113`). +- **Impact:** release-disassembly confirmed **six** gradient-LUT lookup sequences per pixel (each with an outlined bounds-check call and SIMD register spill/reload) where three suffice; est. 5–15% encode. +- **Verify:** golden hashes; synthetic 16-bit encode. + +### W1.4 Decoder: gradient LUT + single computation pass-down +*Merges: decoder triple-quantization + decoder-LUT findings. Calibrated expectation — see Do-NOT #4.* +- **Change:** Port the encoder's init-time `gradientTable` (`Encoder/JPEGLSRegularMode.swift:94-114, 157-163`) into `Decoder/JPEGLSRegularModeDecoder.swift:103-115`, matching the strict-vs-inclusive Table A.7 boundary semantics (verified line-by-line bit-identical by the verifier). Pass q1/q2/q3 (or contextIndex+sign) from the scan-loop run-test (`JPEGLSDecoder.swift:611-614`) into `decodeSinglePixel` (:783-792) and a slimmed `decodePixel` (`JPEGLSRegularModeDecoder.swift:304-313`). +- **Safety:** pure functions of the same (a,b,c,d) and immutable thresholds; LUT semantics verified bit-identical. +- **Impact:** the profile shows `quantizeGradient` as an outlined call at ~6.5% of total; disassembly shows the compiler already CSEs the source-level 9x down to 3 calls — so expect ~5–10% decode, not more. +- **Verify:** golden decoded-pixel hashes against existing `.jls` fixtures; bench-dicom. + +### W1.5 Run-length scan: per-line hoist + word-compare for near==0 +- **Change:** `Encoder/JPEGLSRunMode.swift:91-113` scans `[Int]` element-wise with `abs()`. For near==0 scan via `withUnsafeBufferPointer` with plain `!=` / 64-bit word compares against the run value; keep `abs()` only for near>0. Hoist the row slice once per line instead of per run entry (`JPEGLSEncoder.swift:730-738`, 928-936, ~1150-1165). +- **Safety:** `detectRunLength` is pure counting; near==0 is exact equality; buffer immutable during scan. Scan must still extend to true end-of-line (`JPEGLSRunMode.swift:79-84`). +- **Impact:** ~2–6x on the scan itself while rows are still `[Int]` (full SIMD lands with W2.2); end-to-end ~10–20% on background-heavy modalities (MG, collimated CT), ~0 on dense ones. +- **Verify:** golden hashes; bench-dicom per-modality table (watch MG/MR). + +### W1.6 Free-wins bundle (each ≤ half a day) +1. **Gate the dead `reconstructed` allocation on `near > 0`** — `JPEGLSEncoder.swift:678-685` allocates+zeroes a full H×W `[[Int]]` (32 MB per 2048² scan, ~136 MB per 17 MP MG frame) that is never read when near==0; the line-interleaved path already guards the identical allocation at :876-879. One-line fix. +2. **Trusted decode-result init** — `JPEGLSDecoder.swift:128-131` runs a full O(W·H) validation pass via `MultiComponentImageData.init` (`Encoder/JPEGLSPixelBuffer.swift:96-113`); decoder output is clamped by construction (`JPEGLSRegularModeDecoder.swift:270`, `JPEGLSRunModeDecoder.swift:188, 272`). Add `init(uncheckedComponents:frameHeader:)` for the decoder only. **Required guards:** add an O(1) parse-time check `MAXVAL <= (1<= remainingInLine` (:1027 — a run can end without a terminating 0 bit). +- **Impact:** today every unary bit is a function call with refill check + Data-subscript byte fetch; this is the canonical decoder optimization. Est. 1.5–2.5x decode combined with W1.6.3. +- **Verify:** golden decoded-pixel hashes on existing `.jls` fixtures including the corpus's one real JPEG-LS DICOM frame (decode-conformance fixture); bench-dicom. + +### W2.2 Flat pixel storage + carried-neighbor scan loops (the keystone) +*Merges: encoder getNeighbors-restructure, decoder getNeighbors/CoW-rows, three `[[Int]]`-storage findings, fillRunResult branch.* +- **Change:** Keep the public `[[Int]]` API; convert once per scan at `encodeScanData` (`JPEGLSEncoder.swift:556-624`) / `decodeComponent` (`JPEGLSDecoder.swift:573-662`) entry to a flat `ContiguousArray` (UInt8 for bps≤8) per component plane, and run the entire scan over `withUnsafeBufferPointer` regions with two line pointers (previousLine/currentLine). Carry neighbors in locals — per pixel: `c=b; b=d; a=justCoded; d=prevLine[col+2]` — handling row==0/col==0 once per line via an edge-padded previous-line buffer. Replicate exactly: encoder boundary semantics at `JPEGLSPixelBuffer.swift:292-328` (row 0 → b=c=d=0; col 0 → Ra=Rb=top, Rc=prevRowEdge; last col → Rd=Rb) and decoder `prevRowEdge` (`JPEGLSDecoder.swift:592-604, 1073-1074`: Rc at col 0 = row r−2 first pixel). Fill runs with `initialize(repeating:)` (run length already clamped to `remainingInLine` at :1042, so the per-element branch at :752-756 is provably dead). Convert back to `[[Int]]` once when building `ComponentData`. Samples validated to [0, maxval≤65535] at `JPEGLSPixelBuffer.swift:104-113`, so UInt16 is lossless. +- **Safety:** purely representational — identical a/b/c/d values feed unchanged gradient/context/Golomb logic → bit-exact. Note `Core/JPEGLSCacheFriendlyBuffer.swift` is *not* a drop-in (flat `[Int]` behind a Dictionary); build fresh. +- **Impact:** eliminates the ~15% bounds-check line, the per-row hidden CoW copies and per-store uniqueness checks (~7%), the residual nested-array indirection, and 4x cache footprint; reduces neighbor fetch from 4–5 random 2D reads to one load. Verifiers' estimate: 1.5–3x decode, comparable encode (on top of Phase 1). Also unlocks true SIMD run scanning (upgrade W1.5 to `SIMD16/32` equality + mask-based first-false afterward) and makes run fills memset-cheap. +- **Verify:** golden hashes (encode bytes + decode pixels); bench-dicom full gate; re-check the `-O` vs `-Ounchecked` gap (W1.7) — it should mostly vanish. + +### W2.3 Packed context records +- **Change:** `Core/JPEGLSContextModel.swift:28-46, 202-343`. Replace the four parallel `[Int]` arrays with one array of a packed struct (A, B, C, **N all Int32** — Int16 N is unsafe because RESET is user-settable); drop the four redundant `0..<365` guards (sole producer clamps at :176; all 7 producers verified to flow through it); fuse getC/computeGolombParameter/getErrorCorrection/updateContext into one read-modify-write per pixel under `withUnsafeMutableBufferPointer` scoped over the scan; hoist `2*near+1` (:266) and `parameters.reset` (:271) into stored lets. +- **Safety:** A bounded by RESET·MAXVAL ≈ 4.2M fits Int32; C clamped to [−128,127] at :288/291; storage-only change, arithmetic untouched. Disassembly confirmed `updateContext` survives as an outlined function with 4+ uniqueness-check runtime calls and outlined bounds checks per pixel — real, uneliminated cost. +- **Impact:** `updateContext` is ~8% + a share of the ~7% CoW line; est. 5–15% both sides. +- **Verify:** golden hashes; round-trip tests including custom-RESET/16-bit edge cases. + +--- + +## 4. Phase 3 — Larger bets + +### W3.1 Wire up batch encode/decode (dead feature, easiest multicore win) +- `Sources/jpeglscli/BatchCommand.swift:420-430`: `processEncode`/`processDecode` unconditionally throw "not yet implemented" while the semaphore-throttled pool (:318-354) is real. Wire them to `JPEGLSEncoder`/`JPEGLSDecoder` exactly as `EncodeCommand.swift:313` / `DecodeCommand.swift:83` do. Both codecs are stateless `Sendable` structs with no global mutable state — concurrent per-file use is safe and trivially bit-exact (identical code path). Yields ~Ncores aggregate for multi-file radiology series. Effort: medium. + +### W3.2 Restart-interval (DRI/RSTm) intra-frame parallelism +- The only standards-compliant way to parallelize a single large scan (17 MP MG frames). Implement per T.87: encoder writes DRI, emits RSTm every N lines with full context + run-index + bit-buffer reset; intervals encode in parallel into per-interval buffers and concatenate; decoder indexes RST markers (cheap byte scan) and decodes intervals concurrently. Today the parser stores `restartInterval` but nothing consumes it, and `extractScanData` truncates at the first RST marker — a conformant restart stream currently **fails to decode**; fix that first regardless. `Core/JPEGLSTileProcessor.swift` is dead code — delete it or rebind it to this stripe partitioning. +- **Caveats:** changes the bitstream (small ratio cost), so the bit-identical gate does not apply — gate on lossless round-trip + CharLS interop fixtures (restart handling is a classic cross-implementation bug area). Make it an opt-in encode flag, default off. Near-linear multicore on big frames; does **not** close the single-thread gap. Effort: large. Do this only after Phase 2, when single-thread is respectable. + +### W3.3 Acceleration-layer disposition + docs honesty +- The entire `Sources/JPEGLS/Platform/` layer (11 files, 4,365 lines) plus `JPEGLSBufferPool` and `JPEGLSCacheFriendlyBuffer` have **zero production call sites** — the baseline is pure scalar Swift. Delete Platform/Vulkan, Platform/Metal, Platform/x86_64 (removal guide exists: `docs/X86_64_REMOVAL_GUIDE.md`), Platform/Accelerate and their ~4,750 lines of tests; the one salvageable *idea* (SIMD run scan) is re-implemented properly in W2.2's follow-up, not transplanted (the existing version takes `[Int32]`, builds vectors from bounds-checked subscripts, and resolves matches lane-by-lane). +- Rewrite `docs/PERFORMANCE_TUNING.md` (advertises automatic accelerator selection with "~2–3x" speedups, nonexistent APIs `computeBatchGradients`/`computeStatistics`, a non-compiling TaskGroup example), `README.md:101-104`, and `docs/METAL_GPU_ACCELERATION.md` around what actually runs. Zero MB/s change; the value is stopping future sessions from optimizing a layer that never executes. + +### W3.4 Parallel multi-component `.none` scans (low priority) +- Scans are context-isolated and byte-aligned (fresh context per `encodeScanData`, `writer.flush()` per scan; stuffing carries no state across flush), so per-component parallel encode + ordered splice is provably byte-identical, and the decoder's pre-split `scanDataList` parallelizes trivially. But the DICOM corpus is effectively all single-component grayscale → zero benefit there. Only do this if planar RGB workloads materialize. + +--- + +## 5. Do NOT do (re-litigated and rejected — leave these alone) + +1. **GPU acceleration (Metal/Vulkan), in any form.** Vulkan's GPU path is commented-out pseudocode over a hardcoded-empty device list. Metal's encode kernel computes raw `x − MED`, which is *not* the value that gets Golomb-coded (bias correction C[Q] is applied sequentially *before* the error), and its decode kernel is logically circular — its inputs are already-decoded neighbors that cannot exist before the answer. The sequential entropy stage cannot be GPU-ified. Delete (W3.3), don't fix. +2. **Wiring the `PlatformAccelerator` protocol into the codec.** Per-pixel existential dispatch, and the "SIMD" implementations are packing-overhead wrappers that execute more instructions than the scalar code. The CLZ Golomb trick it contains is already in production (`JPEGLSContextModel.swift:319-322`). +3. **A Traits-generic "lossless specialization" redesign.** The code already hand-specializes every near==0 arm; the residue is ~4 loop-invariant predicted compares per pixel. The only real waste found was the unconditional `reconstructed` allocation — fixed by the one-line guard in W1.6.1. +4. **Expecting a big decoder win from de-duplicating the source-level 9x gradient quantization.** Release disassembly showed the optimizer fully inlines decodeSinglePixel/decodePixel and CSEs the duplication down to 3 `quantizeGradient` calls + 1 `computeContextIndex` per pixel. The recoverable cost is the outlined call + branch chain (W1.4, ~5–10%), no more. (The *encoder's* 6-vs-3 duplication is disassembly-confirmed real — that's W1.3.) +5. **Parallelizing bench-dicom.** Its serial timed region is the measurement instrument; parallel encode would invalidate the per-modality MB/s metric. Multi-file parallelism belongs in BatchCommand (W3.1). +6. **`unsafeFlags`/`-Ounchecked` in Package.swift.** Breaks the package as a versioned dependency, and `-Ounchecked` turns overflow traps into UB (recent real overflow bug: `db25f17`). Opt-in diagnostic build only (W1.7). +7. **Build-flag hunting beyond default `-O`.** SwiftPM release already does WMO; cross-module opt is irrelevant (hot loops are one module); Swift has no mature PGO. The structural fixes deliver the same wins safely. + +--- + +## 6. Measurement protocol (this machine) + +**Build (git `safe.bareRepository=explicit` workaround required):** +```sh +GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=safe.bareRepository GIT_CONFIG_VALUE_0=all \ + swift build -c release +# binary: .build/arm64-apple-macosx/release/jpegls +``` + +**Bit-exactness gate (run after EVERY Phase 1–2 change; this is the merge blocker):** +1. Before starting work, with the baseline binary: encode the two synthetic references and ~10 fixed DICOM frames (one per modality from the local corpus copy below) to `.jls`; store SHA-256 of each encoded file and of each decoded pixel dump in a `golden/` checksum file. +2. After each change: re-encode/re-decode the same inputs; **all hashes must match**. Exception: W3.2 (restart markers) legitimately changes bytes — gate it on lossless round-trip + CharLS interop instead. +3. Full test suite: `GIT_CONFIG_COUNT=1 ... swift test` (same prefix). + +**Synthetic CPU-truth benchmark (no I/O noise, matches the profiled baseline):** +```sh +.build/arm64-apple-macosx/release/jpegls benchmark --size 2048 --bits-per-sample 16 \ + --iterations 10 --warmup 3 --json # baseline: enc 214.61 ms / dec 137.83 ms +.build/arm64-apple-macosx/release/jpegls benchmark --size 2048 --bits-per-sample 8 \ + --iterations 10 --warmup 3 --json # baseline: enc 30.80 ms / dec 31.55 ms +``` +Trust the **16-bit** number; the 8-bit gradient image compresses 59:1 (run-mode heavy) and flatters run-path changes while hiding regular-mode regressions. Report means; run on a quiet machine on AC power; alternate old/new binaries A/B within one session to cancel thermal drift. + +**Real-DICOM end-to-end:** the corpus lives on Google Drive CloudStorage (`/Users/raster/Library/CloudStorage/GoogleDrive-…/Radiology DICOM Data`, ~30k files, top-level folder = modality). To kill sync/download noise, **copy a fixed subset locally once** (e.g., the first ~20 uncompressed Implicit-VR files per modality to `~/dicom-bench/`, preserving the modality folder structure), then: +```sh +.build/arm64-apple-macosx/release/jpegls bench-dicom ~/dicom-bench --limit 20 --near 0 --json +``` +Baseline to beat: ALL 26.4 MB/s encode / 40.1 MB/s decode / 3.01:1; per-modality CT 21.8/30.8, DX 19.7/28.4, MG 33.1/53.1, MR 35.7/49.7, PX 16.2/23.2, XA 21.0/29.7; 107/107 frames lossless. **Regression gate: every frame still round-trips losslessly (zero mismatches).** US is expected to skip (non-grayscale/encapsulated). Throughput columns are codec-time-only; treat as approximate but comparable run-to-run on local files. + +**Profiling between changes:** start a long run (e.g., a 150-iteration 16-bit roundtrip via `benchmark --iterations 150`), then `sample 10` (1 ms interval); diff top frames against the baseline attribution in §1. After each Phase 2 item, also re-measure the `-O` vs `-Ounchecked` gap (W1.7) — remaining gap ≈ remaining bounds/CoW headroom. + +**Discipline:** one work item per measurement cycle; record (commit, synthetic 16-bit enc/dec ms, bench-dicom ALL row, golden-hash pass/fail) in a running table so wins compose honestly and regressions are attributable. diff --git a/docs/PERFORMANCE_TUNING.md b/docs/PERFORMANCE_TUNING.md index d58c864..e9bb87e 100644 --- a/docs/PERFORMANCE_TUNING.md +++ b/docs/PERFORMANCE_TUNING.md @@ -1,532 +1,152 @@ -# Performance Tuning Guide - -Optimise JPEG-LS encoding and decoding performance with JLSwift. - -## Table of Contents - -- [Overview](#overview) -- [Hardware Acceleration](#hardware-acceleration) -- [Memory Optimisation](#memory-optimisation) -- [Encoding Optimisation](#encoding-optimisation) -- [Decoding Optimisation](#decoding-optimisation) -- [Profiling and Benchmarking](#profiling-and-benchmarking) -- [Best Practices](#best-practices) - -## Overview - -JLSwift is designed for high performance on Apple Silicon while maintaining compatibility with x86-64. Performance characteristics vary significantly based on hardware, image characteristics, and encoding parameters. - -### Performance Factors - -| Factor | Impact | Optimisation | -|--------|---------|--------------| -| **Hardware** | High | Use ARM64 on Apple Silicon | -| **Image Size** | High | Consider tile-based processing for large images | -| **Bit Depth** | Medium | Higher bit depths require more processing | -| **Interleaving** | Medium | Sample-interleaved is fastest for RGB | -| **NEAR Parameter** | Low | Near-lossless slightly faster than lossless | -| **Image Content** | Medium | Flat regions compress faster (run mode) | - -## Hardware Acceleration - -### Platform Selection - -JLSwift automatically selects the best accelerator for your platform: +# Performance Tuning + +How to get the most out of JLSwift, how to benchmark it honestly, and what the +codec does under the hood. Everything in this document describes shipping +code; measured numbers come from real radiology DICOM data (CT/DX/MG/MR/PX/XA) +and the built-in synthetic benchmark on Apple Silicon. + +## TL;DR + +- The codec is fast by default — there is nothing to enable for single-image + encode/decode. +- For **large frames** (e.g. 17 MP mammography), set + `Configuration.restartInterval` to parallelise a single image across cores. +- For **many files**, use `jpegls batch` (or your own task pool — the encoder + and decoder are `Sendable` value types, safe to use concurrently). +- Always benchmark release builds: `swift build -c release`. + +## What makes the hot path fast + +These are the structural properties of the codec, useful to know when +profiling an integration: + +1. **Flat scan planes.** Each scan converts to one contiguous `UInt16` plane + and the whole scan loop runs over an unsafe buffer — no nested-array + indirection, per-access bounds checks, or copy-on-write traffic per pixel, + and half the memory bandwidth of boxed `[[Int]]` rows. +2. **64-bit bitstream I/O.** The writer packs bits into a `UInt64` accumulator + over a pre-reserved `[UInt8]`; the reader refills a 64-bit window several + bytes at a time (applying the ISO 14495-1 §9.1 stuff-bit rule per byte) and + decodes Golomb unary prefixes with `leadingZeroBitCount` instead of one + call per bit. +3. **Init-time gradient tables.** Gradient quantisation (ITU-T.87 Table A.7) + is a table lookup built once per scan, on both the encode and decode side, + and each pixel's gradients are quantised exactly once. +4. **Packed context records.** The 365 per-context adaptation statistics + (A/B/C/N) live in a single record array: one load and one store per pixel, + with bias correction, Golomb-k, and the k = 0 error-correction term all + derived from one record read. +5. **Run scanning.** Lossless run detection is an exact-equality scan over the + row, 4-way unrolled, with no per-element `abs()`. + +The public API is unchanged by all of this: pixels in and out are `[[Int]]`, +and encoded streams are byte-identical to previous releases (verified by a +golden-bitstream gate during development). + +## Restart-interval parallelism (single large image) + +JPEG-LS entropy coding is inherently sequential — each pixel's coding state +depends on every pixel before it — so a single scan cannot be parallelised +without help from the bitstream. Restart markers (DRI/RSTm, ITU-T.87 §C.2.5) +are the standards-compliant way to provide that help: at every interval +boundary the coding state resets exactly as at scan start, which makes the +intervals independently codable. ```swift -import JPEGLS +// Encode a large frame with one restart interval every 256 lines. +let config = try JPEGLSEncoder.Configuration(restartInterval: 256) +let encoded = try JPEGLSEncoder().encode(imageData, configuration: config) -let accelerator = selectPlatformAccelerator() -print("Using: \(type(of: accelerator).platformName)") +// Decoding needs no configuration: the DRI segment is in the stream, and the +// decoder splits at the RST markers and decodes the intervals concurrently. +let decoded = try JPEGLSDecoder().decode(encoded) ``` -**Platform Priority:** -1. **ARM64**: Fastest on Apple Silicon (M1/M2/M3) -2. **x86-64**: Optimised for Intel processors -3. **Scalar**: Fallback for all platforms - -### ARM64 / Apple Silicon (Best Performance) - -- **NEON SIMD**: Vectorised gradient computation and prediction -- **Hardware**: M1, M2, M3, ARM64 processors -- **Speedup**: ~2-3x over scalar implementation - -**Optimisation Tips:** -- Build with `-c release` for full optimisation -- Use Swift 6.2+ for best SIMD codegen -- Run on Apple Silicon devices for maximum benefit +CLI equivalent: ```bash -# Build optimized for Apple Silicon -swift build -c release --arch arm64 +jpegls encode huge.pgm huge.jls --restart-interval 256 ``` -### x86-64 / Intel (Good Performance) - -- **SSE/AVX**: Vectorised operations on Intel processors -- **Hardware**: Intel Core, Xeon processors -- **Speedup**: ~1.5-2x over scalar implementation - -```bash -# Build optimized for x86-64 -swift build -c release --arch x86_64 -``` +Measured on a 4096×4096 16-bit image (Apple Silicon, interval 256, wall +clock including file I/O): encode 0.84 s → 0.31 s, decode 0.69 s → 0.24 s, +at a size cost of about **+0.03 %**. -### Accelerate Framework (Batch Operations) +Notes and trade-offs: -For batch processing on Apple platforms: +- Each interval restarts the adaptive contexts, so compression ratio drops + slightly; the cost shrinks as the interval grows. Intervals of 64–512 lines + are a good range for multi-megapixel frames. +- Currently supported for lossless (NEAR = 0), non-interleaved scans — the + DICOM grayscale case. The configuration initializer rejects unsupported + combinations rather than producing a non-parallel stream silently. +- Streams with restart markers are valid JPEG-LS and decode in any conformant + decoder; conversely JLSwift decodes restart streams produced by other + encoders (intervals are validated to cycle FFD0–FFD7). +- A side benefit is error resilience: a corrupted interval cannot corrupt the + decode of subsequent intervals. -```swift -import JPEGLS - -#if canImport(Accelerate) -let accelerateAccel = AccelerateFrameworkAccelerator() - -// Batch gradient computation for multiple pixels -let (d1Array, d2Array, d3Array) = accelerateAccel.computeBatchGradients( - a: leftPixels, - b: topPixels, - c: topLeftPixels -) - -// Statistical analysis -let stats = accelerateAccel.computeStatistics(pixelValues) -print("Mean: \(stats.mean), StdDev: \(stats.standardDeviation)") -#endif -``` - -**When to Use:** -- Processing large image regions (>1000 pixels) -- Preprocessing or statistical analysis -- Histogram computation - -**When Not to Use:** -- Single-pixel operations (overhead outweighs benefits) -- Tight encoding/decoding loops - -## Memory Optimisation - -### Tile-Based Processing - -For large images, tile-based processing reduces memory footprint: - -```swift -import JPEGLS - -// Configure tile processor -let processor = JPEGLSTileProcessor( - imageWidth: 8192, - imageHeight: 8192, - configuration: TileConfiguration( - tileWidth: 512, // Adjust based on available memory - tileHeight: 512, - overlap: 4 // For boundary continuity - ) -) - -// Estimate memory savings -let bytesPerPixel = 2 // 16-bit image -let savings = processor.estimateMemorySavings(bytesPerPixel: bytesPerPixel) -print("Memory reduction: \(Int(savings * 100))%") - -// Calculate tiles -let tiles = processor.calculateTilesWithOverlap() - -// Process tiles sequentially or in parallel -for tile in tiles { - // Load only this tile's data - let tileData = loadTileData(tile) - - // Process tile - processTile(tileData) -} -``` - -**Tile Size Guidelines:** -- **Small tiles (256×256)**: Lower memory, more overhead -- **Medium tiles (512×512)**: Good balance ✓ Recommended -- **Large tiles (1024×1024)**: Higher memory, less overhead - -**Memory Savings:** -- 8192×8192 image with 512×512 tiles: ~97% memory reduction -- 4096×4096 image with 512×512 tiles: ~94% memory reduction - -### Buffer Pooling - -Reuse buffers to reduce allocation overhead: - -```swift -import JPEGLS - -// Use global shared pool -let contextBuffer = sharedBufferPool.acquire( - type: .contextArrays, - size: 365 -) -defer { - sharedBufferPool.release(contextBuffer, type: .contextArrays) -} - -// Or create a custom pool -let customPool = JPEGLSBufferPool() -let pixelBuffer = customPool.acquire(type: .pixelData, size: width * height) -defer { - customPool.release(pixelBuffer, type: .pixelData) -} -``` - -**Buffer Types:** -- `.contextArrays`: 365 Int arrays for context states -- `.pixelData`: Large pixel data buffers -- `.bitstreamData`: Encoded bitstream buffers - -**Performance Impact:** -- First allocation: Standard speed -- Reused allocations: ~5-10x faster -- Best for: Encoding/decoding many images in sequence - -### Cache-Friendly Data Layout - -Use contiguous memory for better cache locality: - -```swift -import JPEGLS - -// Convert 2D arrays to cache-friendly format -let cacheFriendlyBuffer = JPEGLSCacheFriendlyBuffer( - pixelData: [ - 1: pixels // Component 1 (grayscale or red) - ], - width: width, - height: height -) - -// Access patterns optimized for CPU cache -let row = cacheFriendlyBuffer.getRow(componentId: 1, row: rowIndex) -let rows = cacheFriendlyBuffer.getRows(componentId: 1, rowStart: 0, rowEnd: 10) -``` - -**Benefits:** -- ~10-20% faster neighbour access -- Better prefetching from memory -- Reduced cache misses in tight loops - -## Encoding Optimisation - -### Interleaving Mode Selection - -Choose interleaving based on image type: - -```swift -// Greyscale: Always use .none -let greyscaleConfig = try JPEGLSEncoder.Configuration( - near: 0, - interleaveMode: .none // Required for single component -) - -// RGB: Use .sample for best performance -let rgbConfig = try JPEGLSEncoder.Configuration( - near: 0, - interleaveMode: .sample // Best cache locality -) - -// Alternative: Line-interleaved (slightly slower) -let lineConfig = try JPEGLSEncoder.Configuration( - near: 0, - interleaveMode: .line -) -``` - -**Performance Comparison:** -- Sample-interleaved: Fastest (best cache locality) ✓ -- Line-interleaved: ~5-10% slower -- None (separate scans): ~10-15% slower - -### Near-Lossless vs Lossless - -Near-lossless encoding can be slightly faster: - -```swift -// Lossless (NEAR=0) -let losslessData = try JPEGLSEncoder().encode(imageData) - -// Near-lossless (NEAR=3) -let config = try JPEGLSEncoder.Configuration(near: 3) // Allows ±3 error -let nearLosslessData = try JPEGLSEncoder().encode(imageData, configuration: config) -``` - -**Performance Impact:** -- Near-lossless: ~5-10% faster -- Compression ratio: ~10-30% better -- Use when: Perfect reconstruction not required - -### Image Content Characteristics - -Different content types compress at different speeds: - -| Content Type | Relative Speed | Why | -|--------------|----------------|-----| -| Flat regions | Fastest (100%) | Run mode dominates | -| Gradients | Medium (70%) | Regular mode with smooth transitions | -| High-frequency | Slowest (50%) | Regular mode with many context switches | -| Medical images | Medium (65%) | Mix of flat and textured regions | - -**Optimisation:** -- Pre-process images to increase flat regions (lossy only) -- Consider tiling to isolate different content types -- Use profiling to identify bottlenecks - -## Decoding Optimisation - -### Parser Optimisation - -The parser reads and validates JPEG-LS file structure: - -```swift -import JPEGLS +## Batch throughput (many files) -// Parse file -let data = try Data(contentsOf: fileURL) -let parser = JPEGLSParser(data: data) -let result = try parser.parse() - -// Cache parsed results for multiple operations -let frameHeader = result.frameHeader -let scanHeaders = result.scanHeaders -let presetParams = result.presetParameters -``` - -**Tips:** -- Parse once, decode multiple times if needed -- Validate file structure before decoding -- Cache frame and scan headers - -### Bitstream Reading - -Efficient bitstream reading is critical: - -```swift -import JPEGLS - -let reader = JPEGLSBitstreamReader(data: encodedData) - -// Reset bit buffer at scan boundaries -reader.resetBitBuffer() - -// Seek to specific positions when needed -try reader.seek(to: scanDataOffset) -``` - -**Performance Tips:** -- Minimise bit buffer resets -- Use seek() sparingly (resets bit buffer) -- Read in larger chunks when possible - -## Profiling and Benchmarking - -### Built-in Benchmarks - -Run comprehensive benchmarks: +`jpegls batch` runs encode/decode/info/verify over a glob or directory with a +worker pool sized to the machine: ```bash -# Run all performance benchmarks -swift test --filter JPEGLSPerformanceBenchmarks - -# Run specific benchmark -swift test --filter "benchmarkEncode512x512" +jpegls batch encode "scans/*.pgm" --output-dir encoded/ --parallelism 8 +jpegls batch decode "encoded/*.jls" --output-dir decoded/ ``` -**Benchmark Categories:** -1. **Encoding by size**: 256×256 to 4096×4096 -2. **Encoding by bit depth**: 8-bit, 12-bit, 16-bit -3. **Encoding by component**: Greyscale, RGB -4. **Near-lossless**: NEAR=3, NEAR=10 -5. **Interleaving modes**: none, line, sample -6. **Content types**: flat, gradient, medical-like +Batch encode output is byte-identical to serial `jpegls encode` of the same +files. In library code, the same effect is one `withTaskGroup` away — +`JPEGLSEncoder` and `JPEGLSDecoder` are stateless `Sendable` structs, so one +instance per task or a shared instance are both safe. -### Custom Benchmarking +## Benchmarking -```swift -import JPEGLS -import Foundation - -// Measure encoding time -let startTime = Date() - -let encoder = JPEGLSEncoder() -let jpegLSData = try encoder.encode(imageData) - -let elapsed = Date().timeIntervalSince(startTime) - -// Calculate throughput -let pixelCount = imageData.frameHeader.width * imageData.frameHeader.height -let throughputPixels = Double(pixelCount) / elapsed / 1_000_000.0 // Mpixels/s -let throughputBytes = Double(pixelCount) / elapsed / 1_000_000.0 // MB/s - -print("Encoded \(pixelCount) pixels in \(elapsed) seconds") -print("Throughput: \(throughputPixels) Mpixels/s, \(throughputBytes) MB/s") -``` - -### Profiling Tools +Use the built-in benchmark for CPU-bound numbers without file-I/O noise: -**macOS / Xcode:** ```bash -# Use Instruments for detailed profiling -xcodebuild -scheme JPEGLS -configuration Release -# Open in Instruments: Time Profiler, Allocations, System Trace -``` - -**Linux:** -```bash -# Use perf for CPU profiling -swift build -c release -perf record --call-graph=dwarf .build/release/YourApp -perf report -``` +# 16-bit synthetic benchmark (the trustworthy one — see note below) +jpegls benchmark --size 2048 --bits-per-sample 16 --iterations 10 --warmup 3 --json -### Platform Benchmarks - -Compare performance across accelerators: - -```bash -# Run platform benchmark tests -swift test --filter PlatformBenchmarks +# Real-data round-trip over a DICOM corpus, grouped by modality +jpegls bench-dicom /path/to/corpus --limit 20 ``` -**Expected Results (relative to scalar):** -- ARM64: 2-3x faster -- x86-64: 1.5-2x faster -- Accelerate (batch): 3-5x faster for large batches +Caveats that will save you from misleading numbers: -## Best Practices +- **Prefer the 16-bit synthetic benchmark.** The 8-bit gradient image + compresses ~59:1 and spends most of its time in run mode, which flatters + run-path changes and hides regular-mode regressions. Real medical data sits + around 2–6:1. +- `bench-dicom` measures codec time only, but reads files inside the loop — + run it from a local disk, not cloud-synced storage. +- Benchmark release builds on AC power, and A/B alternate binaries within one + session to cancel thermal drift. -### Build Configuration +## Profiling -Always use release builds for production: +On macOS, `sample` against a long benchmark run gives a quick hot-function +picture: ```bash -# Release build with optimizations -swift build -c release - -# Debug build for development (slower) -swift build -c debug -``` - -**Optimisation flags:** -- `-c release`: Full optimisations, no debug symbols -- `-c debug`: No optimisations, full debug info - -### Concurrency - -Process multiple images in parallel: - -```swift -import JPEGLS -import Foundation - -// Process images concurrently -await withTaskGroup(of: Data.self) { group in - for imageData in imageBatch { - group.addTask { - let encoder = JPEGLSEncoder() - return try encoder.encode(imageData) - } - } - - for await encoded in group { - print("Encoded \(encoded.count) bytes") - } -} -``` - -**Scaling:** -- CPU-bound: Use processor count parallel tasks -- I/O-bound: Use 2-4x processor count -- Memory-limited: Use fewer parallel tasks - -### Image Size Guidelines - -| Image Size | Processing Strategy | Memory | Speed | -|------------|-------------------|---------|-------| -| < 512×512 | Direct encoding | Low | Fastest | -| 512-2048 | Direct or tiled | Medium | Fast | -| 2048-4096 | Tiled recommended | Medium | Medium | -| > 4096 | Tiled required | High | Slower | - -### Memory Usage Estimates - -| Image | Uncompressed | Tiled (512×512) | Savings | -|-------|--------------|-----------------|---------| -| 2048×2048, 8-bit | 4 MB | 256 KB | 94% | -| 4096×4096, 8-bit | 16 MB | 256 KB | 98% | -| 8192×8192, 16-bit | 128 MB | 512 KB | 99.6% | - -### Monitoring Performance - -Track key metrics: - -```swift -import JPEGLS - -// Monitor encoding statistics -let statistics = try encoder.encodeScan(buffer: buffer) - -print("Pixels encoded: \(statistics.pixelsEncoded)") -print("Components: \(statistics.componentCount)") -print("Interleave mode: \(statistics.interleaveMode)") - -// Calculate compression ratio (when bitstream I/O available) -// let uncompressedSize = width * height * bytesPerSample -// let compressedSize = encodedData.count -// let ratio = Double(uncompressedSize) / Double(compressedSize) -``` - -### Common Pitfalls - -❌ **Avoid:** -- Debug builds in production -- Processing large images without tiling -- Ignoring platform-specific optimisations -- Frequent buffer allocations without pooling - -✅ **Prefer:** -- Release builds with full optimisations -- Tile-based processing for large images -- Using `selectPlatformAccelerator()` for automatic optimisation -- Buffer pooling for repeated operations -- Cache-friendly data layouts for neighbour access - -## Summary - -### Quick Wins - -1. **Use release builds**: 2-5x faster than debug -2. **Run on Apple Silicon**: 2-3x faster with ARM64 -3. **Use tile-based processing**: 90%+ memory savings for large images -4. **Enable buffer pooling**: 5-10x faster allocations -5. **Choose sample-interleaving**: Fastest for RGB images - -### Advanced Optimisations - -1. Cache-friendly buffers for neighbour access -2. Accelerate framework for batch operations -3. Parallel processing for multiple images -4. Content-aware tile sizing -5. Custom profiling and benchmarking - -### Measurement - -Before optimising, always: -1. Profile your specific workload -2. Run benchmarks on target hardware -3. Measure memory usage patterns -4. Validate compression ratios -5. Test with representative images - ---- - -For questions or performance issues, please [open an issue](https://github.com/Raster-Lab/JLSwift/issues) with: -- Hardware details (CPU, memory) -- Image characteristics (size, bit depth, content type) -- Benchmark results -- Profiling data if available +jpegls benchmark --size 2048 --bits-per-sample 16 --iterations 200 & +sample $! 10 -file /tmp/jls-profile.txt +``` + +For allocation work, Instruments' Allocations template on the same invocation +shows per-scan transients; the codec performs no per-pixel allocations, so +anything hot there is in the integration layer (e.g. converting pixel +formats). + +## A note on GPU / SIMD acceleration layers + +Earlier releases shipped a `Platform/` layer (Metal, Vulkan, Accelerate, +ARM64/x86-64 SIMD wrappers) advertised as accelerating the codec. Profiling +during the 0.9 optimisation effort showed none of it was invoked on the +encode/decode hot path, and the GPU kernels could not produce conformant +streams: JPEG-LS bias correction and context adaptation make the value being +entropy-coded depend on all previously coded pixels, which is exactly what a +data-parallel kernel cannot see. The layer was removed in favour of the +measured CPU optimisations above; restart intervals are the supported (and +standards-compliant) parallelism mechanism. diff --git a/docs/README.md b/docs/README.md index cd48b8f..ee65b7a 100644 --- a/docs/README.md +++ b/docs/README.md @@ -14,11 +14,9 @@ Documentation for the JLSwift JPEG-LS codec. For the project overview, see the - [SERVER_SIDE_EXAMPLES.md](SERVER_SIDE_EXAMPLES.md) — server-side Swift usage - [DICOMKIT_INTEGRATION.md](DICOMKIT_INTEGRATION.md) — DICOM/medical-imaging integration -## Performance & acceleration -- [PERFORMANCE_TUNING.md](PERFORMANCE_TUNING.md) — tuning guide -- [METAL_GPU_ACCELERATION.md](METAL_GPU_ACCELERATION.md) — Metal GPU path -- [VULKAN_GPU_ACCELERATION.md](VULKAN_GPU_ACCELERATION.md) — Vulkan GPU path -- [X86_64_REMOVAL_GUIDE.md](X86_64_REMOVAL_GUIDE.md) — Apple-only consolidation notes +## Performance +- [PERFORMANCE_TUNING.md](PERFORMANCE_TUNING.md) — benchmarking, restart-interval parallelism, profiling +- [OPTIMIZATION_PLAN.md](OPTIMIZATION_PLAN.md) — the measured analysis behind the 0.9 hot-path rewrite ## Standards & internals - [CONFORMANCE_MATRIX.md](CONFORMANCE_MATRIX.md) — ITU-T.87 conformance mapping diff --git a/docs/RELEASE_NOTES_TEMPLATE.md b/docs/RELEASE_NOTES_TEMPLATE.md index 09a0c75..5e5367c 100644 --- a/docs/RELEASE_NOTES_TEMPLATE.md +++ b/docs/RELEASE_NOTES_TEMPLATE.md @@ -201,7 +201,8 @@ let encoder = JPEGLSEncoder() import JPEGLS // Example showing how to use the performance optimization -let processor = JPEGLSTileProcessor(/* ... */) +let config = try JPEGLSEncoder.Configuration(restartInterval: 256) +let encoded = try JPEGLSEncoder().encode(imageData, configuration: config) // ... code example ``` diff --git a/docs/SERVER_SIDE_EXAMPLES.md b/docs/SERVER_SIDE_EXAMPLES.md index daa6eed..a47d4f5 100644 --- a/docs/SERVER_SIDE_EXAMPLES.md +++ b/docs/SERVER_SIDE_EXAMPLES.md @@ -9,7 +9,7 @@ This guide demonstrates how to integrate JLSwift JPEG-LS compression into server - [Vapor Framework Examples](#vapor-framework-examples) - [REST API for JPEG-LS Conversion](#rest-api-for-jpeg-ls-conversion) - [Medical Imaging Upload Service](#medical-imaging-upload-service) - - [Streaming Large File Encoder](#streaming-large-file-encoder) + - [Large File Encoder](#large-file-encoder) - [Batch Processing API](#batch-processing-api) - [Hummingbird Framework Examples](#hummingbird-framework-examples) - [Simple JPEG-LS API Service](#simple-jpeg-ls-api-service) @@ -41,7 +41,7 @@ JLSwift is well-suited for server-side Swift applications that need to: Key benefits for server-side use: - **Pure Swift**: No C dependencies, easier deployment -- **Memory Efficient**: Buffer pooling and tile-based processing +- **Memory Efficient**: Internal buffer pooling; restart-interval parallelism for large frames - **Performance**: Hardware acceleration on Apple Silicon servers - **Concurrent**: Safe to use across multiple concurrent requests - **Standards Compliant**: Full JPEG-LS (ISO/IEC 14495-1:1999) support @@ -380,26 +380,22 @@ func validateMedicalImageParameters(_ metadata: MedicalImageMetadata) throws { } ``` -### Streaming Large File Encoder +### Large File Encoder -Process large files with streaming to minimise memory usage: +Encode large images using restart-interval parallelism (ITU-T.87 DRI/RSTm). +There is no tiling API — the codec works on one flat pixel plane per scan; +restart intervals parallelise a single image across cores and add error +resilience. Supported for lossless (NEAR = 0), non-interleaved scans: ```swift import Vapor import JPEGLS import NIOCore -func configureStreamingRoutes(_ app: Application) throws { +func configureLargeImageRoutes(_ app: Application) throws { - // POST /api/stream/encode - Stream-encode large image - app.on(.POST, "api", "stream", "encode", body: .stream) { req async throws -> Response in - // Use tile-based processing for large images - let tileConfig = TileConfiguration( - tileWidth: 512, - tileHeight: 512, - overlap: 4 - ) - + // POST /api/large/encode - Encode large image with restart-interval parallelism + app.on(.POST, "api", "large", "encode", body: .collect(maxSize: "512mb")) { req async throws -> Response in // Parse dimensions from query parameters guard let width = req.query[Int.self, at: "width"], let height = req.query[Int.self, at: "height"], @@ -407,37 +403,27 @@ func configureStreamingRoutes(_ app: Application) throws { throw Abort(.badRequest, reason: "Missing dimensions") } - let processor = JPEGLSTileProcessor( - imageWidth: width, - imageHeight: height, - configuration: tileConfig - ) + // Load pixel data from the request body + // (simplified - actual implementation would parse raw samples from req.body) + let pixels = try loadPixels(from: req, width: width, height: height) - let tiles = processor.calculateTilesWithOverlap() + let imageData = try MultiComponentImageData.grayscale( + pixels: pixels, + bitsPerSample: bitsPerSample + ) - // Calculate memory savings - let savings = processor.estimateMemorySavings(bytesPerPixel: (bitsPerSample + 7) / 8) + // Large frames: parallelise a single image across cores with restart markers + let config = try JPEGLSEncoder.Configuration(restartInterval: 256) + let encoded = try JPEGLSEncoder().encode(imageData, configuration: config) + // Decoding splits at the RST markers automatically and decodes intervals concurrently. - req.logger.info("Processing large image with \(tiles.count) tiles", metadata: [ - "memory_savings": .string("\(Int(savings * 100))%"), - "tile_size": .string("\(tileConfig.tileWidth)x\(tileConfig.tileHeight)") + req.logger.info("Encoded large image", metadata: [ + "pixels": .string("\(width * height)"), + "restart_interval_lines": .string("256"), + "compressed_bytes": .string("\(encoded.count)") ]) - // Stream processing - var totalPixelsEncoded = 0 - - for (index, tile) in tiles.enumerated() { - req.logger.debug("Processing tile \(index + 1)/\(tiles.count)") - - // Process tile (simplified - actual implementation would read from stream) - totalPixelsEncoded += tile.width * tile.height - } - - return Response(status: .ok, headers: [ - "X-Tiles-Processed": "\(tiles.count)", - "X-Pixels-Encoded": "\(totalPixelsEncoded)", - "X-Memory-Savings": "\(Int(savings * 100))%" - ]) + return Response(status: .ok, body: .init(data: encoded)) } } ``` @@ -1242,7 +1228,10 @@ struct EncodeResult { ### Memory-Efficient Streaming -Process large files with minimal memory footprint: +Stream large uploads with backpressure, then encode the assembled frame. Note +that the codec encodes and decodes whole frames (one flat pixel plane per +scan) — there is no tiling API. For large frames, restart-interval +parallelism is the mechanism for spreading the work across cores: ```swift import Vapor @@ -1251,24 +1240,14 @@ import JPEGLS func configureStreamingProcessing(_ app: Application) { app.on(.POST, "api", "stream", "process", body: .stream) { req async throws -> Response in - let tileConfig = TileConfiguration(tileWidth: 512, tileHeight: 512, overlap: 4) - + var buffer = ByteBuffer() var bytesProcessed: Int64 = 0 - var tilesProcessed = 0 - // Stream processing with backpressure + // Stream collection with backpressure for try await chunk in req.body { - // Process chunk bytesProcessed += Int64(chunk.readableBytes) - - // Use cache-friendly buffer for better performance - let cacheBuffer = JPEGLSCacheFriendlyBuffer( - width: tileConfig.tileWidth, - height: tileConfig.tileHeight, - componentCount: 1 - ) - - tilesProcessed += 1 + var chunk = chunk + buffer.writeBuffer(&chunk) // Apply backpressure if memory usage is high if bytesProcessed > 100_000_000 { // 100MB @@ -1276,15 +1255,25 @@ func configureStreamingProcessing(_ app: Application) { } } + // Parse raw samples into pixel rows + // (simplified - parse `buffer` according to your upload format) + let pixels = try parsePixelRows(buffer) + let imageData = try MultiComponentImageData.grayscale( + pixels: pixels, + bitsPerSample: 16 + ) + + // Large frames: parallelise a single image across cores with restart markers + let config = try JPEGLSEncoder.Configuration(restartInterval: 256) + let encoded = try JPEGLSEncoder().encode(imageData, configuration: config) + // Decoding splits at the RST markers automatically and decodes intervals concurrently. + req.logger.info("Streaming processing complete", metadata: [ "bytes_processed": "\(bytesProcessed)", - "tiles_processed": "\(tilesProcessed)" + "compressed_bytes": "\(encoded.count)" ]) - return Response(status: .ok, headers: [ - "X-Bytes-Processed": "\(bytesProcessed)", - "X-Tiles-Processed": "\(tilesProcessed)" - ]) + return Response(status: .ok, body: .init(data: encoded)) } } ``` @@ -1602,7 +1591,7 @@ func configureMonitoring(_ app: Application) { This guide provides comprehensive examples for integrating JLSwift into server-side Swift applications. Key takeaways: 1. **Use appropriate frameworks**: Vapor for full-featured apps, Hummingbird for lightweight services, NIO for custom protocols -2. **Optimise for performance**: Buffer pooling, tile-based processing, worker threads +2. **Optimise for performance**: Restart-interval parallelism for large frames, worker threads 3. **Handle errors gracefully**: Comprehensive error handling and validation 4. **Secure your API**: Authentication, rate limiting, input validation 5. **Monitor and log**: Health checks, structured logging, metrics diff --git a/docs/SWIFTUI_EXAMPLES.md b/docs/SWIFTUI_EXAMPLES.md index 053370a..b2ab7c1 100644 --- a/docs/SWIFTUI_EXAMPLES.md +++ b/docs/SWIFTUI_EXAMPLES.md @@ -17,7 +17,7 @@ This guide demonstrates how to integrate JLSwift JPEG-LS compression into SwiftU - [Performance Optimisation](#performance-optimisation) - [Caching Decoded Images](#caching-decoded-images) - [Background Decoding](#background-decoding) - - [Memory-Efficient Tile Loading](#memory-efficient-tile-loading) + - [Loading Very Large Images](#loading-very-large-images) - [Error Handling](#error-handling) - [Platform Differences](#platform-differences) @@ -1036,90 +1036,55 @@ struct BackgroundDecodingImageView: View { } ``` -### Memory-Efficient Tile Loading +### Loading Very Large Images -For very large medical images, use tile-based loading: +For very large medical images, decode off the main actor and rely on +restart-interval parallelism. The codec decodes the full frame as one flat +pixel plane per scan — there is no tile or region decoding API — but images +encoded with restart markers (ITU-T.87 DRI/RSTm) are split at the RST +markers and decoded concurrently across cores, automatically: ```swift import SwiftUI import JPEGLS -struct TiledImageView: View { +struct LargeImageView: View { let imageURL: URL - let tileSize: Int = 512 - @State private var tiles: [TileInfo] = [] - - struct TileInfo: Identifiable { - let id = UUID() - let rect: CGRect - var image: Image? - } + @State private var image: Image? var body: some View { - GeometryReader { geometry in - Canvas { context, size in - for tile in tiles { - if let image = tile.image { - // Draw tile at its position - context.draw(image, in: tile.rect) - } - } - } - .onAppear { - loadTiles() + Group { + if let image { + image + .resizable() + .scaledToFit() + } else { + ProgressView("Decoding…") } } - } - - private func loadTiles() { - Task { + .task { do { - // Load file metadata - let data = try Data(contentsOf: imageURL) - let parser = JPEGLSParser(data: data) - let parseResult = try parser.parse() - - let width = Int(parseResult.frameHeader.width) - let height = Int(parseResult.frameHeader.height) - - // Calculate tile layout - let tileProcessor = JPEGLSTileProcessor( - imageWidth: width, - imageHeight: height, - configuration: TileConfiguration( - tileWidth: tileSize, - tileHeight: tileSize, - overlap: 0 - ) - ) - - let tileRects = tileProcessor.calculateTiles() - - // Create tile info - await MainActor.run { - tiles = tileRects.map { rect in - TileInfo(rect: CGRect( - x: CGFloat(rect.x), - y: CGFloat(rect.y), - width: CGFloat(rect.width), - height: CGFloat(rect.height) - )) - } - } - - // Load tiles progressively - for index in tiles.indices { - // In a real implementation, you would decode only the tile region - // This is a simplified example - try await Task.sleep(nanoseconds: 100_000_000) // Simulate load - } + // Decode on a background task. When the file contains + // restart markers, the decoder splits at the RST markers + // and decodes the intervals concurrently. + let cgImage = try JPEGLSImageLoader.loadCGImage(from: imageURL) + #if os(macOS) + image = Image(nsImage: NSImage(cgImage: cgImage, size: .zero)) + #else + image = Image(uiImage: UIImage(cgImage: cgImage)) + #endif } catch { - print("Failed to load tiles: \(error)") + print("Failed to load image: \(error)") } } } } + +// When producing large frames, encode with restart markers so that +// decoding (and encoding) can parallelise across cores: +// let config = try JPEGLSEncoder.Configuration(restartInterval: 256) +// let encoded = try JPEGLSEncoder().encode(imageData, configuration: config) ``` ## Error Handling diff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.md index 500e243..bed5bad 100644 --- a/docs/TROUBLESHOOTING.md +++ b/docs/TROUBLESHOOTING.md @@ -51,15 +51,14 @@ swift --version **Problem**: `error: no such module 'Accelerate'` on non-Apple platforms. **Solution**: -The Accelerate framework is Apple-only. The library will fall back to scalar or x86-64 accelerators on Linux/Windows: +The Accelerate framework is Apple-only. JLSwift imports it conditionally and +falls back to portable code on Linux — no configuration is needed. If the +error comes from your own code, guard the import: ```swift // Conditional import - safe on all platforms #if canImport(Accelerate) import Accelerate -let accel = AccelerateFrameworkAccelerator() -#else -let accel = selectPlatformAccelerator() // Uses ARM64 or x86-64 or scalar #endif ``` @@ -265,15 +264,12 @@ swift build swift build -c release ``` -2. **Check hardware acceleration**: +2. **Parallelise large frames with restart intervals**: ```swift -let accelerator = selectPlatformAccelerator() -print("Using: \(type(of: accelerator).platformName)") - -// Should be: -// - "ARM64" on Apple Silicon -// - "x86-64" on Intel -// - "Scalar" as fallback +// Large frames: parallelise a single image across cores with restart markers +let config = try JPEGLSEncoder.Configuration(restartInterval: 256) +let encoded = try JPEGLSEncoder().encode(imageData, configuration: config) +// Decoding splits at the RST markers automatically and decodes intervals concurrently. ``` 3. **Use appropriate interleaving**: @@ -323,55 +319,39 @@ swift test -c release --filter JPEGLSPerformanceBenchmarks **Problem**: Encoding/decoding large images causes memory exhaustion. -**Solution**: Use tile-based processing: +**Solution**: The codec decodes each scan into a single flat `UInt16` pixel +plane — there is no tiling API, so peak memory scales with +width × height × components (roughly 2 bytes per sample for the decoded +output, plus the compressed input). If a frame is too large for available +memory, split it into separate JPEG-LS images at the application level. + +For throughput (not memory) on large frames, use restart-interval +parallelism (ITU-T.87 DRI/RSTm markers): ```swift import JPEGLS -// Instead of loading entire image -// let allPixels = loadEntireImage() // ✗ - -// Load and process in tiles ✓ -let processor = JPEGLSTileProcessor( - imageWidth: 8192, - imageHeight: 8192, - configuration: TileConfiguration( - tileWidth: 512, - tileHeight: 512, - overlap: 4 - ) -) - -let tiles = processor.calculateTilesWithOverlap() - -for tile in tiles { - let tilePixels = loadTileData(tile) - processTile(tilePixels) -} +// Large frames: parallelise a single image across cores with restart markers +let config = try JPEGLSEncoder.Configuration(restartInterval: 256) +let encoded = try JPEGLSEncoder().encode(imageData, configuration: config) +// Decoding splits at the RST markers automatically and decodes intervals concurrently. ``` -**Memory savings**: -- 4096×4096 with 512×512 tiles: 94% reduction -- 8192×8192 with 512×512 tiles: 97% reduction - ### Memory Leaks **Problem**: Memory usage grows over time when processing many images. -**Solution**: Use buffer pooling: +**Solution**: Internal working buffers are pooled and reused automatically by +the codec — no manual buffer management is needed. If memory still grows +across a batch loop on Apple platforms, wrap each iteration in an +autorelease pool: ```swift -// Reuse buffers instead of allocating new ones for imageData in imageBatch { - let buffer = sharedBufferPool.acquire( - type: .contextArrays, - size: 365 - ) - defer { - sharedBufferPool.release(buffer, type: .contextArrays) + try autoreleasepool { + let encoded = try JPEGLSEncoder().encode(imageData, configuration: config) + // Process encoded data... } - - // Use buffer for encoding/decoding } ``` @@ -381,7 +361,7 @@ for imageData in imageBatch { **Solution**: -1. **Pre-allocate buffers**: +**Pre-allocate buffers**: ```swift // Pre-allocate with correct capacity var pixels = [[Int]]() @@ -394,16 +374,6 @@ for _ in 0.. [[Int]] { - // Simulate loading a specific tile from a large image - // In practice, this would read from file or memory-mapped data + // Load the full frame (test pattern here; read from file in practice) var pixels: [[Int]] = [] - - for row in y.. **Status**: Phase 15.2 — Swift architecture implemented with CPU fallback. GPU execution via Vulkan compute requires the Vulkan SDK and SPIR-V shader binaries, which are not yet bundled with the project. All operations currently use the CPU fallback path, producing bit-exact results that will match the future GPU implementation. - -## Design Goals - -The Vulkan GPU acceleration is designed to mirror the Metal pipeline with these principles: - -1. **Cross-Vendor Compatibility**: Supports NVIDIA, AMD, Intel, and ARM Mali GPUs via Vulkan 1.1+ -2. **Bit-Exact Results**: GPU and CPU implementations produce identical results -3. **Conditional Compilation**: GPU code gated behind `#if canImport(VulkanSwift)` — CPU fallback always available -4. **CPU Fallback**: Automatic fallback when Vulkan is unavailable or unsupported -5. **Platform Independence**: Shared algorithm logic between Metal and Vulkan pipelines - -## Current Architecture - -``` -Platform/Vulkan/ -├── VulkanAccelerator.swift # Swift API with CPU fallback (implemented) -├── VulkanDevice.swift # Device selection and capability detection (implemented) -├── VulkanMemory.swift # VulkanBuffer + VulkanMemoryPool (implemented) -└── VulkanCommandBuffer.swift # VulkanCommandBuffer + VulkanCommandPool (implemented) -``` - -**Planned (requires Vulkan SDK):** - -``` -Platform/Vulkan/Shaders/ -├── jpegls_gradients.spv # SPIR-V gradient + MED prediction shader -├── jpegls_quantize.spv # SPIR-V gradient quantisation shader -├── jpegls_colour_hp1.spv # SPIR-V HP1 colour transform shader -├── jpegls_colour_hp2.spv # SPIR-V HP2 colour transform shader -└── jpegls_colour_hp3.spv # SPIR-V HP3 colour transform shader -``` - -### Memory Management Architecture (`VulkanMemory.swift`) - -`VulkanBuffer` and `VulkanMemoryPool` implement the Vulkan memory model on the CPU side. -When the Vulkan SDK is integrated, these types will wrap `VkBuffer` + `VkDeviceMemory`. - -```swift -// Pool-based buffer allocation (matches Vulkan sub-allocation pattern) -let pool = VulkanMemoryPool(maxPoolSize: 64 * 1024 * 1024) // 64 MB - -let inputBuf = try pool.allocate(size: pixelCount * 4, usage: .storageBuffer) -let outputBuf = try pool.allocate(size: pixelCount * 4, usage: .storageBuffer) - -inputBuf.write(pixels) // host → device transfer -// … GPU dispatch … -let result = outputBuf.read(count: pixelCount, type: Int32.self) // device → host - -pool.reset() // free all allocations for reuse -``` - -### Command Buffer Architecture (`VulkanCommandBuffer.swift`) - -`VulkanCommandBuffer` and `VulkanCommandPool` implement the Vulkan command recording -model. When the Vulkan SDK is integrated, these will delegate to `vkCmdBindPipeline`, -`vkCmdDispatch`, etc. - -```swift -let cmdPool = VulkanCommandPool() -let cmdBuf = cmdPool.allocate() - -cmdBuf.begin() -cmdBuf.bindPipeline(name: "compute_gradients") -cmdBuf.bindBuffer(inputBuf, binding: 0) -cmdBuf.bindBuffer(outputBuf, binding: 1) -cmdBuf.dispatch(x: UInt32((pixelCount + 63) / 64)) -cmdBuf.end() - -cmdPool.reset() // reset for next frame -``` - -## GPU vs CPU Decision - -The Vulkan accelerator uses the same threshold-based decision as the Metal implementation: - -- **Small images** (< 1024 pixels): Use CPU fallback — GPU overhead exceeds benefit -- **Large images** (≥ 1024 pixels): Use GPU compute — parallelism outweighs transfer cost -- **Batch processing**: GPU preferred for batches of 8+ images regardless of size - -## Usage - -### Basic Usage - -```swift -import JPEGLS - -// VulkanAccelerator is available on all platforms (no import guard needed). -// isSupported reflects whether a real Vulkan GPU is found. -let accelerator = VulkanAccelerator() - -// Check for GPU availability -if VulkanAccelerator.isSupported { - print("Vulkan GPU compute available: \(accelerator.device?.name ?? "unknown")") -} else { - print("No Vulkan GPU found — using CPU fallback") -} - -// Compute gradients (GPU when available, CPU fallback otherwise) -let a: [Int32] = // ... north pixel values -let b: [Int32] = // ... west pixel values -let c: [Int32] = // ... northwest pixel values - -let (d1, d2, d3) = accelerator.computeGradientsBatch(a: a, b: b, c: c) -let predictions = accelerator.computeMEDPredictionBatch(a: a, b: b, c: c) - -// Quantise gradients to context indices -let (q1, q2, q3) = accelerator.quantizeGradientsBatch( - d1: d1, d2: d2, d3: d3, t1: 3, t2: 7, t3: 21) - -// Apply HP1 colour transform -let (rPrime, gPrime, bPrime) = accelerator.applyColourTransformForwardBatch( - transform: .hp1, r: rPixels, g: gPixels, b: bPixels) -``` - -### Device Selection - -```swift -import JPEGLS - -// List available Vulkan devices (returns [] when no SDK present) -let devices = enumerateVulkanDevices() -for device in devices { - print("\(device.name): \(device.deviceType)") -} - -// Select best device -if let best = selectBestVulkanDevice() { - print("Selected: \(best.name)") -} -``` - -## Prerequisites (for GPU Execution) - -To enable real GPU acceleration, the following are required: - -1. **Vulkan Runtime**: Vulkan 1.1 or later installed - - Linux: Install via package manager (`apt install libvulkan-dev`) - - Windows: Install LunarG Vulkan SDK from vulkan.lunarg.com -2. **Vulkan-capable GPU**: Any GPU with Vulkan compute support (NVIDIA, AMD, Intel, ARM Mali) -3. **VulkanSwift package**: A Swift package wrapping the Vulkan API (to be added as a dependency) - -```bash -# Linux: Install Vulkan development libraries -sudo apt install libvulkan-dev vulkan-tools - -# Verify Vulkan installation -vulkaninfo --summary -``` - -## Supported Platforms - -| Platform | Vulkan Support | GPU Acceleration | -|-----------|---------------|-----------------| -| Linux x86-64 | ✅ Planned | Vulkan compute | -| Linux ARM64 | ✅ Planned | Vulkan compute | -| Windows x86-64 | ✅ Planned | Vulkan compute | -| macOS | ❌ Not planned | Use Metal instead | -| iOS / iPadOS | ❌ Not planned | Use Metal instead | - -## Comparison with Metal - -| Feature | Metal (Apple) | Vulkan (Linux/Windows) | -|---------|--------------|----------------------| -| Status | ✅ Implemented | 📋 Planned | -| Platforms | macOS, iOS, tvOS | Linux, Windows | -| API Style | High-level Swift | Low-level C / Swift wrapper | -| Shader Language | MSL | GLSL → SPIR-V | -| Unified Memory | ✅ Apple Silicon | ❌ Discrete GPU only | -| Setup Complexity | Low | Medium | - -## Performance Characteristics (CPU Fallback) - -The CPU-fallback path provides a baseline for comparison with future GPU implementation. -Benchmarks measured on an x86-64 Linux build (single-threaded): - -| Operation | 64×64 | 512×512 | 2048×2048 | -|-----------|-------|---------|-----------| -| Gradient computation | < 0.1 ms | ~4 ms | ~65 ms | -| MED prediction | < 0.1 ms | ~4 ms | ~65 ms | -| Gradient quantisation | < 0.1 ms | ~3 ms | ~50 ms | -| HP1 colour transform (forward) | < 0.1 ms | ~3 ms | ~50 ms | - -Run `swift test` to see current measurements in `VulkanPerformanceBenchmarks`. - -## Performance Targets (GPU) - -Once Vulkan GPU execution is integrated, expected speedups over the CPU path: - -- **Gradient computation**: 4–8× speedup (large images) -- **MED prediction**: 3–6× speedup -- **Context quantisation**: 2–4× speedup -- **End-to-end encoding**: 2–3× speedup for images ≥ 1 MP - -## Development Roadmap - -See [MILESTONES.md](MILESTONES.md) **Phase 15.2** for the full implementation plan: - -- [x] Design Vulkan compute pipeline architecture -- [x] Implement SPIR-V shaders for gradient computation and MED prediction -- [x] Implement Vulkan memory management and buffer allocation (`VulkanMemory.swift`) -- [x] Implement Vulkan command buffer recording and submission (`VulkanCommandBuffer.swift`) -- [x] Implement CPU fallback for systems without Vulkan -- [ ] Integrate Vulkan SDK and compile SPIR-V shaders -- [ ] Benchmark against CPU-only on Linux -- [x] Verify bit-exact results against CPU implementation - -## Related Documentation - -- [METAL_GPU_ACCELERATION.md](METAL_GPU_ACCELERATION.md) — Metal GPU acceleration (Apple platforms, implemented) -- [PERFORMANCE_TUNING.md](PERFORMANCE_TUNING.md) — General performance optimisation guide -- [MILESTONES.md](MILESTONES.md) — Development roadmap and status - ---- - -**Version**: 1.0 (Draft) -**Last Updated**: 2026-02-28 -**Status**: Planned — implementation scheduled for Milestone 15 diff --git a/docs/X86_64_REMOVAL_GUIDE.md b/docs/X86_64_REMOVAL_GUIDE.md deleted file mode 100644 index 3105359..0000000 --- a/docs/X86_64_REMOVAL_GUIDE.md +++ /dev/null @@ -1,425 +0,0 @@ -# x86-64 Removal Guide - -This document provides a comprehensive guide for removing x86-64 support from JLSwift when Apple Silicon becomes the sole supported platform. - -## Overview - -The x86-64 implementation in JLSwift is designed for **clean removal**. All x86-64-specific code is: -- Isolated in dedicated modules with clear boundaries -- Conditionally compiled using `#if arch(x86_64)` directives -- Tested independently with cross-platform compatibility verification -- Documented for future deprecation - -This guide outlines the exact steps required to remove x86-64 support while maintaining the ARM64/Apple Silicon implementation. - -## x86-64 Code Inventory - -### Files to Remove - -The following files contain x86-64-specific code and should be removed entirely: - -1. **`Sources/JPEGLS/Platform/x86_64/X86_64Accelerator.swift`** - - Primary x86-64 SIMD accelerator implementation - - Contains SSE/AVX-optimised gradient computation, MED prediction, quantisation, - Golomb-Rice parameter computation, run-length detection, and byte stuffing scanning - - **Action**: Delete entire file - -2. **`Sources/JPEGLS/Platform/x86_64/IntelMemoryOptimizer.swift`** - - Intel cache-hierarchy parameters, tile-size tuning, cache-aligned buffer allocation, - `IntelBufferPool`, prefetch hints, memory-mapped I/O helpers, and tuning parameters - - **Action**: Delete entire file and directory - -### Files to Modify - -The following files contain conditional compilation for x86-64 and require targeted modifications: - -#### 1. `Sources/JPEGLS/Core/PlatformProtocols.swift` - -**Lines to Remove**: 135-139 - -```swift - #elseif arch(x86_64) - // Check if x86-64 SIMD accelerator is available - if X86_64Accelerator.isSupported { - return X86_64Accelerator() - } -``` - -**Before**: -```swift -public func selectPlatformAccelerator() -> any PlatformAccelerator { - #if arch(arm64) - // Check if ARM64 NEON accelerator is available - if ARM64Accelerator.isSupported { - return ARM64Accelerator() - } - #elseif arch(x86_64) - // Check if x86-64 SIMD accelerator is available - if X86_64Accelerator.isSupported { - return X86_64Accelerator() - } - #endif - - // Fallback to scalar implementation - return ScalarAccelerator() -} -``` - -**After**: -```swift -public func selectPlatformAccelerator() -> any PlatformAccelerator { - #if arch(arm64) - // Check if ARM64 NEON accelerator is available - if ARM64Accelerator.isSupported { - return ARM64Accelerator() - } - #endif - - // Fallback to scalar implementation - return ScalarAccelerator() -} -``` - -#### 2. `Tests/JPEGLSTests/PlatformProtocolsTests.swift` - -**Lines to Remove**: Test cases specific to x86-64 (search for `#if arch(x86_64)`) - -**x86-64 Test Cases to Remove**: -- `x86_64PlatformName()` test -- `x86_64IsSupported()` test -- `x86_64Results()` test -- Any `#elseif arch(x86_64)` branches in platform selection tests - -**Example Before**: -```swift - #elseif arch(x86_64) - // On x86_64, should get X86_64Accelerator -``` - -**Example After**: Delete the entire `#elseif arch(x86_64)` branch - -#### 3. `Tests/JPEGLSTests/PlatformBenchmarks.swift` - -**Lines to Remove**: x86-64 benchmark branches - -**Example Before**: -```swift - #elseif arch(x86_64) - // On x86_64, should get X86_64Accelerator - print("Running on x86_64 with SSE/AVX acceleration") -``` - -**Example After**: Delete the entire `#elseif arch(x86_64)` branch - -#### 4. `Tests/JPEGLSTests/X86_64AcceleratorPhase14Tests.swift` - -**Action**: Delete entire file. Contains Phase 14.1 tests for Golomb-Rice, run-length, and byte stuffing on x86-64. - -#### 5. `Tests/JPEGLSTests/IntelMemoryOptimizerTests.swift` - -**Action**: Delete entire file. Contains Phase 14.2 tests for Intel cache parameters, tile sizing, buffer pooling, and memory-mapped I/O. - -### Documentation Files to Update - -After removal, update these documentation files: - -1. **`README.md`** - - Remove references to x86-64 support - - Update "Hardware Targets" section to show ARM64 only - - Remove x86-64 from architecture overview - - Update platform requirements - -2. **`MILESTONES.md`** - - Mark Milestone 6 as "Deprecated/Removed" - - Update summary table - - Remove x86-64 from dependencies and hardware targets sections - -3. **`.github/copilot-instructions.md`** (if applicable) - - Remove x86-64 build instructions - - Update platform requirements - -## Step-by-Step Removal Process - -Follow these steps in order to cleanly remove x86-64 support: - -### Step 1: Verify Current State - -Before making any changes, verify the current project state: - -```bash -# Run all tests to establish baseline -swift test - -# Verify code coverage -swift test --enable-code-coverage - -# List all x86-64 files -find . -path "*/x86_64/*" -o -name "*x86_64*" -o -name "*X86_64*" - -# Search for x86_64 references -grep -r "x86_64" --include="*.swift" Sources/ Tests/ -grep -r "x86-64" --include="*.md" . -``` - -### Step 2: Remove x86-64 Implementation Files - -```bash -# Delete x86-64 accelerator directory -rm -rf Sources/JPEGLS/Platform/x86_64/ - -# Verify removal -ls -la Sources/JPEGLS/Platform/ -``` - -### Step 3: Update Platform Protocols - -Edit `Sources/JPEGLS/Core/PlatformProtocols.swift`: - -1. Open the file -2. Locate the `selectPlatformAccelerator()` function (around line 129) -3. Remove lines 135-139 (the `#elseif arch(x86_64)` block) -4. Verify the function now only contains ARM64 check and scalar fallback -5. Save the file - -### Step 4: Update Test Files - -#### Update PlatformProtocolsTests.swift - -```bash -# Open the file -vim Tests/JPEGLSTests/PlatformProtocolsTests.swift -``` - -Remove: -- All `#if arch(x86_64)` test cases -- All `#elseif arch(x86_64)` branches in platform selection tests - -#### Update PlatformBenchmarks.swift - -```bash -# Open the file -vim Tests/JPEGLSTests/PlatformBenchmarks.swift -``` - -Remove: -- All `#elseif arch(x86_64)` benchmark branches - -### Step 5: Verify Compilation and Tests - -After making code changes, verify everything still works: - -```bash -# Clean build -swift package clean - -# Build project -swift build - -# Run tests -swift test - -# Verify coverage is still >95% -swift test --enable-code-coverage -``` - -Expected result: All tests should pass with coverage remaining above 95%. - -### Step 6: Update Documentation - -Update the following documentation files to reflect x86-64 removal: - -#### README.md - -**Section: Requirements** -- Change "Platforms: Linux, macOS 12+ (Monterey), iOS 15+" to "Platforms: macOS 12+ (Monterey), iOS 15+ (Apple Silicon)" -- Remove any Linux x86-64 mentions - -**Section: Architecture Overview** -- Remove `x86_64/` from the directory tree -- Update platform section - -**Section: Hardware Targets** (if present) -- Remove "Secondary: x86-64 (Intel Macs, Linux)" line - -#### MILESTONES.md - -**Milestone 6 Section**: -- Change status from "Planned" or "Complete" to "Deprecated - Removed" -- Add removal date and version - -**Summary Table**: -- Mark Milestone 6 as "Removed ❌" or update status appropriately - -**Architecture Principles**: -- Remove "x86-64 Removability" principle (it's no longer needed) - -**Hardware Targets**: -- Remove "Secondary: x86-64" line - -### Step 7: Update Package.swift (If Needed) - -Check if `Package.swift` has any x86-64-specific configurations: - -```bash -grep -i "x86" Package.swift -``` - -If there are any x86-64-specific settings, remove them. - -### Step 8: Final Verification - -Perform a final comprehensive check: - -```bash -# Search for any remaining x86-64 references -grep -r "x86_64\|x86-64\|X86_64" --include="*.swift" --include="*.md" . - -# Verify no broken references -swift build - -# Run full test suite -swift test - -# Generate and review coverage report -swift test --enable-code-coverage -``` - -### Step 9: Commit Changes - -Once everything is verified: - -```bash -# Review changes -git status -git diff - -# Stage changes -git add -A - -# Commit with descriptive message -git commit -m "Remove x86-64 support - Apple Silicon only - -- Removed Platform/x86_64/X86_64Accelerator.swift -- Updated PlatformProtocols.swift to remove x86-64 selection -- Removed x86-64 test cases from PlatformProtocolsTests -- Removed x86-64 benchmark cases from PlatformBenchmarks -- Updated documentation (README.md, MILESTONES.md) -- Maintained >95% test coverage -- All tests passing on ARM64 with scalar fallback - -Project now targets Apple Silicon exclusively." - -# Push changes -git push -``` - -## Impact Assessment - -### What Remains After Removal - -After x86-64 removal, the project will: -- ✅ Fully support Apple Silicon (ARM64) with NEON acceleration -- ✅ Maintain scalar fallback for any non-ARM64 platforms -- ✅ Retain >95% test coverage -- ✅ Keep all JPEG-LS encoding/decoding functionality -- ✅ Preserve Accelerate framework integration -- ✅ Maintain clean platform abstraction architecture - -### What Is Lost - -After removal: -- ❌ Native x86-64 SSE/AVX SIMD optimisations -- ❌ Intel Mac hardware acceleration -- ❌ Linux x86-64 performance optimisations -- ❌ Cross-platform benchmarking capabilities - -On non-ARM64 platforms, the `ScalarAccelerator` will be used, which: -- Provides correct, reference implementation -- Has lower performance than SIMD-optimised versions -- Is fully tested and maintains correctness - -### Performance Considerations - -On Intel Macs after removal: -- The project will fall back to `ScalarAccelerator` -- Performance will be slower than with `X86_64Accelerator` -- Correctness is maintained (bit-exact results) -- Consider Rosetta 2 as an alternative for Intel Macs if ARM64 binary is used - -## Verification Checklist - -Before considering the removal complete, verify: - -- [ ] `Sources/JPEGLS/Platform/x86_64/` directory deleted -- [ ] All `#elseif arch(x86_64)` conditionals removed from source -- [ ] All x86-64 test cases removed -- [ ] `swift build` succeeds with no warnings -- [ ] `swift test` passes all tests -- [ ] Code coverage remains >95% -- [ ] No references to "x86_64" or "x86-64" in code (except comments/docs noting removal) -- [ ] README.md updated -- [ ] MILESTONES.md updated -- [ ] This removal guide marked as completed -- [ ] Changes committed and pushed -- [ ] CI/CD pipeline passes - -## Rollback Procedure - -If removal needs to be reverted: - -```bash -# Revert the commit -git revert - -# Or restore from git history -git checkout -- Sources/JPEGLS/Platform/x86_64/ -git checkout -- Sources/JPEGLS/Core/PlatformProtocols.swift -git checkout -- Tests/JPEGLSTests/PlatformProtocolsTests.swift -git checkout -- Tests/JPEGLSTests/PlatformBenchmarks.swift - -# Rebuild and test -swift build -swift test -``` - -## Alternative: Deprecation Without Removal - -If complete removal is premature, consider deprecation instead: - -1. Add deprecation warnings to `X86_64Accelerator`: - ```swift - @available(*, deprecated, message: "x86-64 support is deprecated and will be removed in a future version") - public struct X86_64Accelerator: PlatformAccelerator { - // ... - } - ``` - -2. Update documentation to note deprecation timeline - -3. Continue maintaining x86-64 code until a specific version - -4. Follow this guide for complete removal in the chosen future version - -## Questions or Issues - -If issues arise during removal: - -1. Verify all tests pass before making changes -2. Make changes incrementally, testing after each step -3. Keep git history clean with atomic commits -4. Document any unexpected issues or deviations from this guide -5. Update this guide with lessons learned - -## Completion - -Once removal is complete, consider: -- Archiving this guide (move to `docs/archive/` or delete) -- Creating a tag for the "last x86-64 supported version" -- Updating release notes to inform users of the change -- Providing migration guidance for Intel Mac users - ---- - -**Version**: 2.0 -**Last Updated**: 2026-03-01 -**Status**: Ready for use when x86-64 deprecation is scheduled