From 39cdb79fcae04155c4cc524b153c6faac765ffd7 Mon Sep 17 00:00:00 2001 From: Vincent Herbst Date: Tue, 28 Jul 2026 14:58:58 +0200 Subject: [PATCH 1/2] feat(subtitles): keep text subtitle styling and placement to the host Re #233, for everything the source layer can actually deliver. The styling was never lost in transit. libavcodec converts every text subtitle format into an ASS event line before the engine sees it, so the markup was on AVSubtitleRect.ass the whole time: SRT through ff_htmlmarkup_to_ass, WebVTT through its own tag table, teletext because the decoder already runs txt_format=ass. What discarded it was ours, cleanASSBody stripping every {...} block and the colour parser handling \c and documenting that it ignored the rest. So this is one parser, not one per format. coloredRuns becomes styledRuns, and SRT, WebVTT, teletext and ASS light up together. The teletext positioning that was arriving and being dropped comes with it. SubtitleTextRun gains isBold, isItalic, isUnderlined, isStruckThrough, fontName and fontSize; SubtitleCue gains placement, an ASS numpad alignment plus an optional anchor normalized like SubtitleImage.position is. New initializer parameters are defaulted, so no call site changes. \pos normalizes against a declared PlayResX/Y when the ASS header has one, otherwise libavcodec's 384x288 default, which is what the lines it synthesises use. Unconditional, no gate. An unstyled cue still arrives as .text carrying the same string, so a host handling only that case sees nothing new, and .richText was already reachable through teletext. The PiP compositor was flattening rich text to a joined string in one white font, so the styling would have stopped at the PiP window. It now builds a per-run attributed line (colour, face, ASS-relative size, real CoreText bold and italic traits, underline, and a drawn rule for strikeout since CoreText has no attribute for it) and honours placement. Not covered, and not coverable here: WebVTT cue settings never arrive, libavcodec does not convert them and webvttdec.c says so in its file header. VTT inline styling works; VTT positioning needs an upstream patch. SRT positioning needs AV_PKT_DATA_SUBTITLE_POSITION side data that most files lack. The #107 teletext blank-line fold moved out of the run parser into teletextBody, where it belongs: only libzvbi's row joining produces those, and a blank line in an ASS or SRT cue can be deliberate. 1210 tests green, strict-concurrency=complete clean, tvOS and iOS Simulator BUILD SUCCEEDED. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01At1agC2qU2Y2MX2BKLdT4N --- CHANGELOG.md | 9 + README.md | 13 +- .../Decoder/EmbeddedSubtitleDecoder.swift | 51 +++- .../Decoder/SubtitleDecoder.swift | 44 +++- Sources/AetherEngine/PlayerState.swift | 63 ++++- .../Renderer/SubtitleFrameCompositor.swift | 185 ++++++++++++-- .../Subtitles/SubtitleRectText.swift | 201 ++++++++++++--- .../Issue233ASSOverrideStyleTests.swift | 228 ++++++++++++++++++ .../Issue233CompositorStylingTests.swift | 110 +++++++++ .../TeletextColourRunsTests.swift | 58 ++--- 10 files changed, 857 insertions(+), 105 deletions(-) create mode 100644 Tests/AetherEngineTests/Issue233ASSOverrideStyleTests.swift create mode 100644 Tests/AetherEngineTests/Issue233CompositorStylingTests.swift diff --git a/CHANGELOG.md b/CHANGELOG.md index 0ffcb2e9..4c01f47e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,15 @@ the public-API contract. ## [Unreleased] +### Added + +- **Text subtitle styling reaches the host instead of being stripped: bold, italic, underline, strikeout, colour, font face and size, plus the placement a cue asks for.** `SubtitleTextRun` gained `isBold`, `isItalic`, `isUnderlined`, `isStruckThrough`, `fontName` and `fontSize` alongside the colour it already carried, and `SubtitleCue` gained `placement` (`SubtitleTextPlacement`: an ASS numpad alignment and an optional anchor normalized to [0, 1] the same way `SubtitleImage.position` is). Both additions are source-compatible: the new initializer parameters are defaulted, so existing call sites are untouched. This needed no per-format parsing, because libavcodec converts every text subtitle format into an ASS event line before the engine sees it and the markup was arriving on `AVSubtitleRect.ass` the whole time. SRT goes through `ff_htmlmarkup_to_ass`, which turns `///` into `{\b1}/{\i1}/{\u1}/{\s1}`, `` into `{\c&HBBGGRR&}`, `` into `{\fs}` and `` into `{\fn}`; WebVTT maps its own inline tags to the same overrides; dvb_teletext already decoded with `txt_format=ass`, which is why its colours alone survived. What discarded the rest was on this side: `cleanASSBody` stripped every `{...}` block with a regex, and the colour parser handled `\c`/`\1c` and documented that it ignored everything else. That parser is now a full override parser, so SRT, WebVTT, teletext and ASS all light up through one code path, and the teletext positioning that was arriving and being dropped comes with it. Lookalike tags are left alone rather than half-parsed (`\be` and `\bord` are not `\b`, `\iclip` is not `\i`, `\shad` is not `\s`, `\fscx` and `\fsp` are not `\fs`), `\r` resets the accumulated state, and `\an`/`\pos` are lifted out to the cue instead of splitting a run. `\pos` is normalized against the play resolution the line actually uses: a real ASS script's declared `PlayResX/Y` when its header provides one, otherwise libavcodec's 384x288 default, which is what the synthesised lines use. No option gates any of this. An unstyled cue still arrives as `.text` with the same string as before, so a host that only handles that case sees no change, and `.richText` was already a case every subtitle-rendering host had to handle for teletext. +- **The software-path PiP compositor renders that styling instead of flattening it.** It previously reduced a rich-text cue to `runs.map(\.text).joined()` and drew every cue in one white font stacked from the bottom, so styling would have stopped at the PiP window. It now builds a per-run attributed line (per-run colour, font face, ASS-relative size, real CoreText bold and italic traits, underline, and a manually drawn rule for strikeout, which CoreText has no attribute for) and honours a cue's placement, mapping the ASS numpad alignment to a corner inset by the layout margin, or anchoring the block on an explicit `\pos`. Cues without placement keep the existing bottom-up stack. + +### Notes + +- WebVTT cue settings (`line`, `position`, `align`, `size`, `vertical`) still do not arrive: libavcodec does not convert them, as `webvttdec.c` states in its own file header. WebVTT bold, italic and underline work; WebVTT positioning needs an upstream change. SRT positioning arrives only when the demuxer supplies `AV_PKT_DATA_SUBTITLE_POSITION` side data, which most `.srt` files do not carry, though a leading `{\anN}` in the text survives libavcodec's conversion. + ## [5.25.2] - 2026-07-28 ([release notes](https://github.com/superuser404notfound/AetherEngine/releases/tag/5.25.2)) diff --git a/README.md b/README.md index 49dc66b1..0088f6e8 100644 --- a/README.md +++ b/README.md @@ -133,7 +133,18 @@ player.selectAudioTrack(index: trackID) player.subtitleTracks // [TrackInfo], text + bitmap + external, one list player.selectSubtitleTrack(index: streamID) player.clearSubtitle() -player.$subtitleCues // [SubtitleCue]: .text(String), .richText([SubtitleTextRun]) (coloured teletext), or .image(SubtitleImage) +player.$subtitleCues // [SubtitleCue]: .text(String), .richText([SubtitleTextRun]), or .image(SubtitleImage) + +// #233: styled text arrives as .richText. A run carries colour, bold, italic, underline, +// strikeout, font face and an ASS-relative font size; the cue carries the placement it asks +// for (numpad alignment plus an optional [0, 1] anchor). This covers SRT, WebVTT, teletext +// and ASS alike, because libavcodec converts them all to ASS event lines before the engine +// sees them. A cue with no styling still arrives as .text, so handling only that case keeps +// working. WebVTT cue settings are the exception: libavcodec does not convert them, so VTT +// positioning does not arrive (its inline bold/italic/underline does). +for cue in player.subtitleCues { + if case .richText(let runs) = cue.body { render(runs, at: cue.placement) } +} // External subtitle files are first-class tracks (#88): they appear in subtitleTracks // (isExternal == true, synthetic id) and select through the same call as embedded streams. diff --git a/Sources/AetherEngine/Decoder/EmbeddedSubtitleDecoder.swift b/Sources/AetherEngine/Decoder/EmbeddedSubtitleDecoder.swift index 969bfeb8..8a560ed0 100644 --- a/Sources/AetherEngine/Decoder/EmbeddedSubtitleDecoder.swift +++ b/Sources/AetherEngine/Decoder/EmbeddedSubtitleDecoder.swift @@ -50,6 +50,11 @@ final class EmbeddedSubtitleDecoder { /// #107: explicit teletext page override (nil = libzvbi `subtitle` auto-detect). private let teletextPage: Int? + /// #233: coordinate space `\pos` is expressed in. A real ASS track declares its own PlayResX/Y + /// in the header libavcodec hands over as extradata; every event line libavcodec synthesises + /// for SRT, WebVTT or teletext uses the default 384x288 instead. + private let assPlayRes: CGSize + /// Open the subtitle decoder for `stream`. Returns `nil` if the codec couldn't be opened. init?(stream: UnsafeMutablePointer, sourceVideoWidth: Int32, sourceVideoHeight: Int32, preserveASSMarkup: Bool = false, teletextPage: Int? = nil) { guard let codecpar = stream.pointee.codecpar, @@ -92,6 +97,14 @@ final class EmbeddedSubtitleDecoder { && (id == AV_CODEC_ID_ASS || id == AV_CODEC_ID_SSA) self.teletextPage = teletextPage + var playRes = SubtitleRectText.defaultASSPlayRes + if let extra = codecpar.pointee.extradata, codecpar.pointee.extradata_size > 0 { + let header = String(decoding: UnsafeBufferPointer( + start: extra, count: Int(codecpar.pointee.extradata_size)), as: UTF8.self) + if let declared = SubtitleRectText.playRes(fromASSHeader: header) { playRes = declared } + } + self.assPlayRes = playRes + // Some demuxers default to AVDISCARD_DEFAULT and swallow packets; force NONE so everything reaches av_read_frame. stream.pointee.discard = AVDISCARD_NONE } @@ -185,17 +198,33 @@ final class EmbeddedSubtitleDecoder { var bodies: [SubtitleCue.Body] = [] var textLines: [String] = [] + var placement: SubtitleTextPlacement? if sub.num_rects > 0, let rects = sub.rects { for i in 0.. 0 { + var assPlayRes = SubtitleRectText.defaultASSPlayRes + if let extradata = codecpar.pointee.extradata, codecpar.pointee.extradata_size > 0 { let bytes = Data(bytes: extradata, count: Int(codecpar.pointee.extradata_size)) // Strip NUL bytes: extradata is often NUL-terminated; libass parses C-string-style and a NUL hides everything after it. - assHeader = String(data: bytes, encoding: .utf8)? + let header = String(data: bytes, encoding: .utf8)? .replacingOccurrences(of: "\0", with: "") + if keepMarkup { assHeader = header } + // #233: a real ASS script declares the space its \pos coordinates live in; without one + // the line came from libavcodec's own conversion and uses the 384x288 default. + if let header, let declared = SubtitleRectText.playRes(fromASSHeader: header) { + assPlayRes = declared + } } guard let codec = avcodec_find_decoder(codecpar.pointee.codec_id) else { @@ -244,6 +249,8 @@ enum SubtitleDecoder { var lines: [String] = [] var images: [SubtitleImage] = [] + var styledBodies: [SubtitleCue.Body] = [] + var placement: SubtitleTextPlacement? if sub.num_rects > 0, let rects = sub.rects { for i in 0.. startTime { + for body in styledBodies { + cues.append(SubtitleCue( + id: nextID, + startTime: startTime, + endTime: endTime, + body: body, + placement: placement + )) + nextID += 1 + } + } if endTime > startTime { for image in images { pendingImageCueIndices.append(cues.count) diff --git a/Sources/AetherEngine/PlayerState.swift b/Sources/AetherEngine/PlayerState.swift index b2b00cbf..566e2e1e 100644 --- a/Sources/AetherEngine/PlayerState.swift +++ b/Sources/AetherEngine/PlayerState.swift @@ -621,11 +621,62 @@ public struct SubtitleColor: Sendable, Equatable { public init(r: UInt8, g: UInt8, b: UInt8) { self.r = r; self.g = g; self.b = b } } -/// One contiguous same-colour span of a rich-text cue. +/// One contiguous same-styling span of a rich-text cue. +/// +/// #233: libavcodec converts every text subtitle format to an ASS event line before the engine +/// sees it, so SRT (``, ``), WebVTT (`//`), teletext and ASS +/// itself all arrive carrying the same override tags and populate the same fields here. A run +/// with no attribute set is plain text; those cues stay `.text` rather than becoming `.richText`. public struct SubtitleTextRun: Sendable, Equatable { public let text: String public let color: SubtitleColor? - public init(text: String, color: SubtitleColor?) { self.text = text; self.color = color } + public let isBold: Bool + public let isItalic: Bool + public let isUnderlined: Bool + public let isStruckThrough: Bool + /// Face requested by `\fn` (SRT ``); nil means the host's default. + public let fontName: String? + /// Size requested by `\fs` (SRT ``), in ASS play-resolution points, so it is a + /// relative hint rather than a pixel size; nil means the host's default. + public let fontSize: Int? + + public init(text: String, color: SubtitleColor?, + isBold: Bool = false, isItalic: Bool = false, + isUnderlined: Bool = false, isStruckThrough: Bool = false, + fontName: String? = nil, fontSize: Int? = nil) { + self.text = text + self.color = color + self.isBold = isBold + self.isItalic = isItalic + self.isUnderlined = isUnderlined + self.isStruckThrough = isStruckThrough + self.fontName = fontName + self.fontSize = fontSize + } + + /// True when the run asks for anything beyond plain text. Drives the `.text` / `.richText` + /// choice, so an unstyled track keeps the body it has always had. + public var isStyled: Bool { + color != nil || isBold || isItalic || isUnderlined || isStruckThrough + || fontName != nil || fontSize != nil + } +} + +/// Where a text cue asks to be drawn, from the ASS `\an` / `\pos` overrides (#233). +/// +/// Bitmap cues carry their own geometry on `SubtitleImage`; this is the text equivalent and is nil +/// for the overwhelming majority of cues, which simply want the host's default placement. +public struct SubtitleTextPlacement: Sendable, Equatable { + /// ASS numpad alignment from `\an`: 1 bottom-left through 9 top-right, 5 centred. + public let alignment: Int? + /// Anchor from `\pos`, normalized to [0, 1] against the source video frame the same way + /// `SubtitleImage.position` is, with y measured from the top. + public let position: CGPoint? + + public init(alignment: Int?, position: CGPoint?) { + self.alignment = alignment + self.position = position + } } /// Decoded subtitle cue (start/end in container seconds). Payload is plain text (SubRip / ASS / SSA / WebVTT / mov_text), coloured rich text (teletext / ASS colour tags), or a rendered bitmap (PGS / DVB / HDMV) with position normalized against the source video frame. @@ -636,6 +687,10 @@ public struct SubtitleCue: Identifiable, Sendable { public let startTime: Double public let endTime: Double public let body: Body + /// #233: placement the source asked for, from ASS `\an` / `\pos`. nil means the host places the + /// cue itself, which is the case for nearly every cue. Text cues only; a bitmap cue carries its + /// geometry on `SubtitleImage`. + public let placement: SubtitleTextPlacement? public enum Body: Sendable { case text(String) @@ -643,11 +698,13 @@ public struct SubtitleCue: Identifiable, Sendable { case richText([SubtitleTextRun]) } - public init(id: Int, startTime: Double, endTime: Double, body: Body) { + public init(id: Int, startTime: Double, endTime: Double, body: Body, + placement: SubtitleTextPlacement? = nil) { self.id = id self.startTime = startTime self.endTime = endTime self.body = body + self.placement = placement } /// Plain text for text and rich-text cues (rich runs concatenated); nil for bitmap cues. diff --git a/Sources/AetherEngine/Renderer/SubtitleFrameCompositor.swift b/Sources/AetherEngine/Renderer/SubtitleFrameCompositor.swift index 129414b1..086c86f8 100644 --- a/Sources/AetherEngine/Renderer/SubtitleFrameCompositor.swift +++ b/Sources/AetherEngine/Renderer/SubtitleFrameCompositor.swift @@ -143,38 +143,177 @@ final class SubtitleFrameCompositor: @unchecked Sendable { let rect = CGRect(x: rectTopLeft.minX, y: frameHeight - rectTopLeft.maxY, width: rectTopLeft.width, height: rectTopLeft.height) ctx.draw(image.cgImage, in: rect) case .text(let text): - drawTextBlock(text, in: ctx, layout: layout, frameWidth: frameWidth, baselineFromBottom: &textBaselineFromBottom) + drawTextBlock([SubtitleTextRun(text: text, color: nil)], placement: cue.placement, + in: ctx, layout: layout, frameWidth: frameWidth, + frameHeight: frameHeight, baselineFromBottom: &textBaselineFromBottom) case .richText(let runs): - let flat = runs.map(\.text).joined() - drawTextBlock(flat, in: ctx, layout: layout, frameWidth: frameWidth, baselineFromBottom: &textBaselineFromBottom) + drawTextBlock(runs, placement: cue.placement, + in: ctx, layout: layout, frameWidth: frameWidth, + frameHeight: frameHeight, baselineFromBottom: &textBaselineFromBottom) } } return ctx.makeImage() } - private nonisolated static func drawTextBlock(_ text: String, in ctx: CGContext, layout: TextLayout, frameWidth: CGFloat, baselineFromBottom: inout CGFloat) { - let trimmed = text.trimmingCharacters(in: .whitespacesAndNewlines) - guard !trimmed.isEmpty else { return } - let font = CTFontCreateWithName("HelveticaNeue-Medium" as CFString, layout.fontSize, nil) - for line in trimmed.split(separator: "\n").reversed() { - let attributes: [CFString: Any] = [ - kCTFontAttributeName: font, - kCTForegroundColorAttributeName: CGColor(red: 1, green: 1, blue: 1, alpha: 1) - ] - guard let attributed = CFAttributedStringCreate(kCFAllocatorDefault, String(line) as CFString, attributes as CFDictionary) else { continue } + /// Split a run sequence at its line breaks, keeping each run's styling. A run spanning a break + /// contributes to both lines. + nonisolated static func lines(from runs: [SubtitleTextRun]) -> [[SubtitleTextRun]] { + var out: [[SubtitleTextRun]] = [[]] + for run in runs { + let parts = run.text.components(separatedBy: "\n") + for (i, part) in parts.enumerated() { + if i > 0 { out.append([]) } + if !part.isEmpty { out[out.count - 1].append(run.withText(part)) } + } + } + return out.filter { !$0.isEmpty } + } + + /// Font for a run. ASS `\fs` is relative to libavcodec's default style size + /// (`ASS_DEFAULT_FONT_SIZE` = 16), so it scales the layout's base size rather than naming + /// pixels, which keeps a cue proportionate at any frame height. + nonisolated static func font(for run: SubtitleTextRun, baseSize: CGFloat) -> CTFont { + let size = run.fontSize.map { baseSize * CGFloat($0) / 16.0 } ?? baseSize + let name = run.fontName ?? "HelveticaNeue-Medium" + let base = CTFontCreateWithName(name as CFString, size, nil) + var traits: CTFontSymbolicTraits = [] + if run.isBold { traits.insert(.traitBold) } + if run.isItalic { traits.insert(.traitItalic) } + guard !traits.isEmpty else { return base } + return CTFontCreateCopyWithSymbolicTraits(base, size, nil, traits, traits) ?? base + } + + /// Bottom-left origin for a text block under an explicit placement (#233). + /// + /// ASS numpad alignment: 1 to 3 bottom, 4 to 6 middle, 7 to 9 top, left to right within each + /// row. With a `\pos` the point is the anchor and the alignment says which part of the block + /// sits on it; without one the alignment picks a corner inset by `margin`. A position with no + /// alignment takes the ASS default of 2 (bottom centre). Normalized positions measure y from + /// the top, the context draws bottom-up, so y flips here. + nonisolated static func blockOrigin(placement: SubtitleTextPlacement, blockSize: CGSize, + frameWidth: CGFloat, frameHeight: CGFloat, + margin: CGFloat) -> CGPoint { + let an = placement.alignment ?? 2 + let column = (an - 1) % 3 // 0 left, 1 centre, 2 right + let row = (an - 1) / 3 // 0 bottom, 1 middle, 2 top + + guard let position = placement.position else { + let x: CGFloat = column == 0 ? margin + : column == 1 ? (frameWidth - blockSize.width) / 2 + : frameWidth - blockSize.width - margin + let y: CGFloat = row == 0 ? margin + : row == 1 ? (frameHeight - blockSize.height) / 2 + : frameHeight - blockSize.height - margin + return CGPoint(x: x, y: y) + } + + let anchorX = position.x * frameWidth + let anchorY = frameHeight - position.y * frameHeight + let x: CGFloat = column == 0 ? anchorX + : column == 1 ? anchorX - blockSize.width / 2 + : anchorX - blockSize.width + let y: CGFloat = row == 0 ? anchorY + : row == 1 ? anchorY - blockSize.height / 2 + : anchorY - blockSize.height + return CGPoint(x: x, y: y) + } + + private nonisolated static func drawTextBlock(_ runs: [SubtitleTextRun], + placement: SubtitleTextPlacement?, + in ctx: CGContext, layout: TextLayout, + frameWidth: CGFloat, frameHeight: CGFloat, + baselineFromBottom: inout CGFloat) { + let split = lines(from: runs) + guard !split.isEmpty else { return } + let pad = layout.fontSize * 0.4 + + // Build every line first: a placed block needs its own measured size before it can be put + // anywhere, and the default stack needs the same measurements anyway. + let built: [(line: CTLine, runs: [SubtitleTextRun], bounds: CGRect)] = split.compactMap { + guard let attributed = attributedLine($0, layout: layout) else { return nil } let ctLine = CTLineCreateWithAttributedString(attributed) - let bounds = CTLineGetBoundsWithOptions(ctLine, .useOpticalBounds) - let pad = layout.fontSize * 0.4 - let boxWidth = min(bounds.width + pad * 2, layout.maxTextWidth) - let boxHeight = bounds.height + pad - let boxX = (frameWidth - boxWidth) / 2 - let boxY = baselineFromBottom - pad / 2 + return (ctLine, $0, CTLineGetBoundsWithOptions(ctLine, .useOpticalBounds)) + } + guard !built.isEmpty else { return } + + let boxWidth = min((built.map(\.bounds.width).max() ?? 0) + pad * 2, layout.maxTextWidth) + let lineHeights = built.map { $0.bounds.height + pad } + + var originX = (frameWidth - boxWidth) / 2 + var cursorY = baselineFromBottom + if let placement { + let blockHeight = lineHeights.reduce(0, +) + layout.fontSize * 0.2 * CGFloat(built.count - 1) + let origin = blockOrigin(placement: placement, + blockSize: CGSize(width: boxWidth, height: blockHeight), + frameWidth: frameWidth, frameHeight: frameHeight, + margin: layout.bottomMargin) + originX = origin.x + cursorY = origin.y + } + // Bottom-up, so the last line is drawn first and reads top-down on screen. + for (index, entry) in built.enumerated().reversed() { + let boxHeight = lineHeights[index] ctx.setFillColor(CGColor(red: 0, green: 0, blue: 0, alpha: 0.6)) - ctx.fill(CGRect(x: boxX, y: boxY, width: boxWidth, height: boxHeight).insetBy(dx: -2, dy: -2)) - ctx.textPosition = CGPoint(x: boxX + pad, y: baselineFromBottom + pad / 2 - bounds.minY) - CTLineDraw(ctLine, ctx) - baselineFromBottom += boxHeight + layout.fontSize * 0.2 + ctx.fill(CGRect(x: originX, y: cursorY - pad / 2, width: boxWidth, height: boxHeight) + .insetBy(dx: -2, dy: -2)) + let textX = originX + pad + let baseline = cursorY + pad / 2 - entry.bounds.minY + ctx.textPosition = CGPoint(x: textX, y: baseline) + CTLineDraw(entry.line, ctx) + drawStrikethrough(entry.runs, on: entry.line, in: ctx, at: CGPoint(x: textX, y: baseline), + layout: layout) + cursorY += boxHeight + layout.fontSize * 0.2 + } + if placement == nil { baselineFromBottom = cursorY } + } + + private nonisolated static func attributedLine(_ runs: [SubtitleTextRun], + layout: TextLayout) -> CFAttributedString? { + guard let attributed = CFAttributedStringCreateMutable(kCFAllocatorDefault, 0) else { return nil } + CFAttributedStringBeginEditing(attributed) + for run in runs { + var attributes: [CFString: Any] = [ + kCTFontAttributeName: font(for: run, baseSize: layout.fontSize), + kCTForegroundColorAttributeName: run.color.map { + CGColor(red: CGFloat($0.r) / 255, green: CGFloat($0.g) / 255, + blue: CGFloat($0.b) / 255, alpha: 1) + } ?? CGColor(red: 1, green: 1, blue: 1, alpha: 1) + ] + if run.isUnderlined { + attributes[kCTUnderlineStyleAttributeName] = CTUnderlineStyle.single.rawValue + } + guard let piece = CFAttributedStringCreate( + kCFAllocatorDefault, run.text as CFString, attributes as CFDictionary) else { continue } + CFAttributedStringReplaceAttributedString( + attributed, CFRange(location: CFAttributedStringGetLength(attributed), length: 0), piece) + } + CFAttributedStringEndEditing(attributed) + return CFAttributedStringGetLength(attributed) > 0 ? attributed : nil + } + + /// CoreText has no strikethrough attribute (unlike AppKit's NSAttributedString), so struck runs + /// get a rule drawn over their own glyph range. + private nonisolated static func drawStrikethrough(_ runs: [SubtitleTextRun], on line: CTLine, + in ctx: CGContext, at origin: CGPoint, + layout: TextLayout) { + var index = 0 + for run in runs { + let length = run.text.utf16.count + defer { index += length } + guard run.isStruckThrough, length > 0 else { continue } + let startX = CTLineGetOffsetForStringIndex(line, index, nil) + let endX = CTLineGetOffsetForStringIndex(line, index + length, nil) + let size = CTFontGetSize(font(for: run, baseSize: layout.fontSize)) + ctx.setStrokeColor(run.color.map { + CGColor(red: CGFloat($0.r) / 255, green: CGFloat($0.g) / 255, + blue: CGFloat($0.b) / 255, alpha: 1) + } ?? CGColor(red: 1, green: 1, blue: 1, alpha: 1)) + ctx.setLineWidth(max(1, size * 0.06)) + let y = origin.y + size * 0.28 + ctx.move(to: CGPoint(x: origin.x + startX, y: y)) + ctx.addLine(to: CGPoint(x: origin.x + endX, y: y)) + ctx.strokePath() } } diff --git a/Sources/AetherEngine/Subtitles/SubtitleRectText.swift b/Sources/AetherEngine/Subtitles/SubtitleRectText.swift index 26e49a75..f41ca0b3 100644 --- a/Sources/AetherEngine/Subtitles/SubtitleRectText.swift +++ b/Sources/AetherEngine/Subtitles/SubtitleRectText.swift @@ -1,6 +1,16 @@ +import CoreGraphics import Foundation import Libavcodec +extension SubtitleTextRun { + /// Same styling, different text. Keeps the trim and fold passes from restating every attribute. + func withText(_ newText: String) -> SubtitleTextRun { + SubtitleTextRun(text: newText, color: color, isBold: isBold, isItalic: isItalic, + isUnderlined: isUnderlined, isStruckThrough: isStruckThrough, + fontName: fontName, fontSize: fontSize) + } +} + /// Plain-text extraction from FFmpeg subtitle rects, shared by `SubtitleDecoder` (sidecar) and `EmbeddedSubtitleDecoder` (in-container) so ASS parsing fixes live in one place. enum SubtitleRectText { @@ -57,27 +67,73 @@ enum SubtitleRectText { return trimmed.isEmpty ? nil : trimmed } - /// Parse an ASS event line's Text body into coloured runs (#107 teletext colour). Tracks the - /// current foreground from `\c&Hbbggrr&` / `\1c&Hbbggrr&` (ASS colour is BGR); a bare `\c`/`\1c` - /// or unparseable value resets to nil (page default). Applies `\N`/`\n` -> newline, `\h` -> space. - /// All other override tags are ignored. Adjacent equal-colour runs are collapsed. nil when nothing - /// displayable remains. - static func coloredRuns(fromASSEventLine line: String) -> [SubtitleTextRun]? { + /// Default ASS play resolution (`ASS_DEFAULT_PLAYRESX/Y`, libavcodec/ass.h). Every event line + /// libavcodec synthesises for SRT, WebVTT and teletext positions against this space, so it is + /// the right frame of reference unless a real ASS header declares its own. + static let defaultASSPlayRes = CGSize(width: 384, height: 288) + + /// Play resolution declared by an ASS `[Script Info]` header, or nil when it declares none. + static func playRes(fromASSHeader header: String) -> CGSize? { + func value(_ key: String) -> Double? { + for line in header.split(separator: "\n", omittingEmptySubsequences: false) { + let l = line.trimmingCharacters(in: .whitespaces) + guard l.lowercased().hasPrefix(key.lowercased() + ":") else { continue } + return Double(l.dropFirst(key.count + 1).trimmingCharacters(in: .whitespaces)) + } + return nil + } + guard let x = value("PlayResX"), let y = value("PlayResY"), x > 0, y > 0 else { return nil } + return CGSize(width: x, height: y) + } + + /// Accumulated inline ASS override state while walking an event line. + private struct RunStyle: Equatable { + var color: SubtitleColor? + var isBold = false + var isItalic = false + var isUnderlined = false + var isStruckThrough = false + var fontName: String? + var fontSize: Int? + + func run(_ text: String) -> SubtitleTextRun { + SubtitleTextRun(text: text, color: color, isBold: isBold, isItalic: isItalic, + isUnderlined: isUnderlined, isStruckThrough: isStruckThrough, + fontName: fontName, fontSize: fontSize) + } + } + + /// Parse an ASS event line's Text body into styled runs plus the cue-level placement it asks + /// for (#233). Every text subtitle format reaches the engine through libavcodec as one of these + /// lines, so this single parser serves SRT (`ff_htmlmarkup_to_ass`), WebVTT, dvb_teletext + /// (`txt_format=ass`) and ASS itself. + /// + /// Inline state comes from `\c`/`\1c` (BGR; a bare tag or unparseable value resets to the page + /// default), `\b`, `\i`, `\u`, `\s`, `\fn`, `\fs` and `\r`. Cue-level `\an` and `\pos` are + /// lifted out into the placement, with `\pos` normalized against `playRes`. Applies `\N`/`\n` + /// -> newline and `\h` -> space. Tags that merely look like these (`\be`, `\bord`, `\iclip`, + /// `\shad`, `\fscx`, `\fsp`) are left alone. Adjacent runs of equal styling are collapsed. + /// nil when nothing displayable remains. + static func styledRuns(fromASSEventLine line: String, + playRes: CGSize = SubtitleRectText.defaultASSPlayRes) + -> (runs: [SubtitleTextRun], placement: SubtitleTextPlacement?)? { var body = line if body.hasPrefix("Dialogue: ") { body.removeFirst("Dialogue: ".count) } let parts = body.split(separator: ",", maxSplits: 8, omittingEmptySubsequences: false) let text: String = (parts.count == 9 && Int(parts[0]) != nil) ? String(parts[8]) : body - var runs: [SubtitleTextRun] = [] + var pieces: [(style: RunStyle, text: String)] = [] var current = "" - var color: SubtitleColor? = nil + var style = RunStyle() + var alignment: Int? + var position: CGPoint? func flush() { guard !current.isEmpty else { return } - if let last = runs.last, last.color == color { - runs[runs.count - 1] = SubtitleTextRun(text: last.text + current, color: color) + if let last = pieces.indices.last, pieces[last].style == style { + pieces[last].text += current } else { - runs.append(SubtitleTextRun(text: current, color: color)) + pieces.append((style, current)) } current = "" } @@ -92,14 +148,14 @@ enum SubtitleRectText { if n == "h" { current += " "; i += 2; continue } } if c == "{" { - // Override block: colour changes start a new run. var j = i + 1 - var tag = "" - while j < chars.count, chars[j] != "}" { tag.append(chars[j]); j += 1 } - if let newColor = parseColorTag(tag) { - flush() - color = newColor // nil means reset - } + var block = "" + while j < chars.count, chars[j] != "}" { block.append(chars[j]); j += 1 } + var next = style + applyOverrides(block, to: &next, alignment: &alignment, + position: &position, playRes: playRes) + // The text collected so far belongs to the style in force before this block. + if next != style { flush(); style = next } i = (j < chars.count) ? j + 1 : j continue } @@ -108,6 +164,72 @@ enum SubtitleRectText { } flush() + var runs = pieces.map { $0.style.run($0.text) } + let placement = (alignment == nil && position == nil) + ? nil : SubtitleTextPlacement(alignment: alignment, position: position) + guard let trimmed = edgeTrimmed(runs) else { return nil } + runs = trimmed + return (runs, placement) + } + + /// Apply one override block's tags. Inline attributes mutate `style`; `\an` and `\pos` are + /// cue-level and are lifted out instead, so they never split a run. + private static func applyOverrides(_ block: String, to style: inout RunStyle, + alignment: inout Int?, position: inout CGPoint?, + playRes: CGSize) { + for tag in block.split(separator: "\\").map(String.init) { + if tag == "r" || (tag.hasPrefix("r") && !tag.hasPrefix("rnd")) { + style = RunStyle() + } else if let color = parseColorTag("\\" + tag) { + style.color = color // nil means reset + } else if let v = intValue(tag, after: "b") { + style.isBold = v != 0 + } else if let v = intValue(tag, after: "i") { + style.isItalic = v != 0 + } else if let v = intValue(tag, after: "u") { + style.isUnderlined = v != 0 + } else if let v = intValue(tag, after: "s") { + style.isStruckThrough = v != 0 + } else if let v = intValue(tag, after: "an"), (1...9).contains(v) { + alignment = v + } else if tag.hasPrefix("fn") { + let name = String(tag.dropFirst(2)) + style.fontName = name.isEmpty ? nil : name + } else if tag.hasPrefix("fs"), tag.dropFirst(2).allSatisfy(\.isNumber) { + style.fontSize = Int(tag.dropFirst(2)) + } else if let p = parsePositionTag(tag, playRes: playRes) { + position = p + } + } + } + + /// `tag` as `prefix` followed by digits and nothing else, so `\b1` parses while `\bord2`, + /// `\be1`, `\iclip(...)` and `\shad2` do not. + private static func intValue(_ tag: String, after prefix: String) -> Int? { + guard tag.hasPrefix(prefix) else { return nil } + let rest = tag.dropFirst(prefix.count) + guard !rest.isEmpty, rest.allSatisfy(\.isNumber) else { return nil } + return Int(rest) + } + + /// `pos(x,y)` normalized against the play resolution, the same [0, 1] convention + /// `SubtitleImage.position` uses. + private static func parsePositionTag(_ tag: String, playRes: CGSize) -> CGPoint? { + guard tag.hasPrefix("pos("), tag.hasSuffix(")"), + playRes.width > 0, playRes.height > 0 else { return nil } + let inner = tag.dropFirst(4).dropLast() + let parts = inner.split(separator: ",").map { + Double($0.trimmingCharacters(in: .whitespaces)) + } + guard parts.count == 2, let x = parts[0], let y = parts[1] else { return nil } + return CGPoint(x: x / playRes.width, y: y / playRes.height) + } + + /// Trim whitespace and newlines across the edges of a run sequence, so a styled cue matches + /// what the plain path produces. libzvbi teletext ass can prefix a row-positioning newline that + /// would otherwise render as a blank line ONLY on styled cues (#107). Interior blank lines are + /// NOT folded here: that is teletext-specific and lives in `teletextBody`. + private static func edgeTrimmed(_ runs: [SubtitleTextRun]) -> [SubtitleTextRun]? { var cleaned = runs.filter { !$0.text.isEmpty } // Edge-trim leading/trailing whitespace and newlines across the run sequence so a coloured // cue matches the plain path (teletextBody flattens + trims the .text case). libzvbi @@ -117,17 +239,16 @@ enum SubtitleRectText { while let first = cleaned.first { let d = String(first.text.drop(while: { $0 == " " || $0 == "\t" || $0 == "\n" })) if d.isEmpty { cleaned.removeFirst(); continue } - cleaned[0] = SubtitleTextRun(text: d, color: first.color) + cleaned[0] = first.withText(d) break } while let last = cleaned.last { var s = last.text while let c = s.last, c == " " || c == "\t" || c == "\n" { s.removeLast() } if s.isEmpty { cleaned.removeLast(); continue } - cleaned[cleaned.count - 1] = SubtitleTextRun(text: s, color: last.color) + cleaned[cleaned.count - 1] = last.withText(s) break } - cleaned = collapseInteriorBlankLines(cleaned) guard !cleaned.isEmpty else { return nil } return cleaned } @@ -141,31 +262,43 @@ enum SubtitleRectText { /// boundary between adjacent runs is folded too. private static func collapseInteriorBlankLines(_ runs: [SubtitleTextRun]) -> [SubtitleTextRun] { var folded = runs.map { run in - SubtitleTextRun( - text: run.text.replacingOccurrences( - of: #"\n(?:[ \t]*\n)+"#, with: "\n", options: .regularExpression), - color: run.color - ) + run.withText(run.text.replacingOccurrences( + of: #"\n(?:[ \t]*\n)+"#, with: "\n", options: .regularExpression)) } var i = 0 while i < folded.count - 1 { if folded[i].text.hasSuffix("\n") { var next = folded[i + 1].text while next.hasPrefix("\n") { next.removeFirst() } - folded[i + 1] = SubtitleTextRun(text: next, color: folded[i + 1].color) + folded[i + 1] = folded[i + 1].withText(next) } i += 1 } return folded.filter { !$0.text.isEmpty } } - /// Body for a teletext rect's ASS line: `.richText` when any run is coloured, `.text` (flattened) - /// when none is, nil when empty. Keeps the all-white page on the existing plain-text path (#107). - static func teletextBody(fromASSEventLine line: String) -> SubtitleCue.Body? { - guard let runs = coloredRuns(fromASSEventLine: line) else { return nil } - if runs.contains(where: { $0.color != nil }) { - return .richText(runs) - } + /// Body + placement for any text rect's ASS event line (#233): `.richText` when a run asks for + /// styling, `.text` (flattened) when none does, nil when empty. An unstyled track therefore + /// keeps the exact body it produced before, so a host handling only `.text` sees no change. + static func styledBody(fromASSEventLine line: String, + playRes: CGSize = SubtitleRectText.defaultASSPlayRes) + -> (body: SubtitleCue.Body, placement: SubtitleTextPlacement?)? { + guard let parsed = styledRuns(fromASSEventLine: line, playRes: playRes) else { return nil } + return body(for: parsed.runs).map { ($0, parsed.placement) } + } + + /// Teletext variant (#107): identical, plus the interior blank-line fold libzvbi's row joining + /// requires. That fold is deliberately teletext-only, since a blank line in an ASS or SRT cue + /// can be intentional. + static func teletextBody(fromASSEventLine line: String, + playRes: CGSize = SubtitleRectText.defaultASSPlayRes) + -> (body: SubtitleCue.Body, placement: SubtitleTextPlacement?)? { + guard let parsed = styledRuns(fromASSEventLine: line, playRes: playRes) else { return nil } + return body(for: collapseInteriorBlankLines(parsed.runs)).map { ($0, parsed.placement) } + } + + private static func body(for runs: [SubtitleTextRun]) -> SubtitleCue.Body? { + if runs.contains(where: \.isStyled) { return .richText(runs) } let plain = runs.map(\.text).joined().trimmingCharacters(in: .whitespacesAndNewlines) return plain.isEmpty ? nil : .text(plain) } diff --git a/Tests/AetherEngineTests/Issue233ASSOverrideStyleTests.swift b/Tests/AetherEngineTests/Issue233ASSOverrideStyleTests.swift new file mode 100644 index 00000000..3e97e444 --- /dev/null +++ b/Tests/AetherEngineTests/Issue233ASSOverrideStyleTests.swift @@ -0,0 +1,228 @@ +import Testing +import Foundation +@testable import AetherEngine + +/// #233: text subtitle styling was discarded at the parser, not at the source. +/// +/// libavcodec converts every text subtitle format into an ASS event line before the engine sees +/// it, so the markup is already present on `AVSubtitleRect.ass`: SRT through +/// `ff_htmlmarkup_to_ass` (`` to `{\b1}`, `` to `{\c&HBBGGRR&}`, `` to +/// `{\fn}`, `` to `{\fs}`), WebVTT through its own tag table (`//`), and +/// dvb_teletext because the decoder already runs with `txt_format=ass`. What threw it away was +/// ours: `cleanASSBody` stripped every `{...}` block, and `coloredRuns` handled `\c`/`\1c` and +/// documented that "All other override tags are ignored". +/// +/// So there is one parser here, not one per format, and SRT, WebVTT, teletext and ASS all reach +/// the host styled through it. +struct Issue233ASSOverrideStyleTests { + + private func runs(_ line: String) -> [SubtitleTextRun] { + SubtitleRectText.styledRuns(fromASSEventLine: line)?.runs ?? [] + } + + private func placement(_ line: String) -> SubtitleTextPlacement? { + SubtitleRectText.styledRuns(fromASSEventLine: line)?.placement + } + + // MARK: - Inline attributes + + @Test("bold, italic, underline and strikeout are carried on the run") + func binaryAttributes() { + let got = runs(#"0,0,Default,,0,0,0,,{\b1}bold{\b0} {\i1}it{\i0} {\u1}un{\u0} {\s1}st{\s0}"#) + #expect(got.first(where: { $0.text == "bold" })?.isBold == true) + #expect(got.first(where: { $0.text == "it" })?.isItalic == true) + #expect(got.first(where: { $0.text == "un" })?.isUnderlined == true) + #expect(got.first(where: { $0.text == "st" })?.isStruckThrough == true) + } + + @Test("an attribute turned off does not leak into later runs") + func attributeTurnsOff() { + let got = runs(#"0,0,Default,,0,0,0,,{\b1}loud{\b0} quiet"#) + #expect(got.count == 2) + #expect(got[0].isBold == true) + #expect(got[1].isBold == false) + #expect(got[1].text == " quiet") + } + + @Test("several tags in one override block all apply") + func multipleTagsInOneBlock() { + let got = runs(#"0,0,Default,,0,0,0,,{\i1\b1}both"#) + #expect(got.count == 1) + #expect(got[0].isBold == true) + #expect(got[0].isItalic == true) + } + + @Test("a non-zero ass weight counts as bold") + func assWeightIsBold() { + #expect(runs(#"0,0,Default,,0,0,0,,{\b700}heavy"#).first?.isBold == true) + #expect(runs(#"0,0,Default,,0,0,0,,{\b0}plain"#).first?.isBold == false) + } + + @Test("font name and size are carried and reset by a bare tag") + func fontNameAndSize() { + let got = runs(#"0,0,Default,,0,0,0,,{\fnArial}{\fs40}big{\fn}{\fs}back"#) + #expect(got.first(where: { $0.text == "big" })?.fontName == "Arial") + #expect(got.first(where: { $0.text == "big" })?.fontSize == 40) + #expect(got.first(where: { $0.text == "back" })?.fontName == nil) + #expect(got.first(where: { $0.text == "back" })?.fontSize == nil) + } + + @Test("a font name containing a space survives") + func fontNameWithSpace() { + #expect(runs(#"0,0,Default,,0,0,0,,{\fnArial Black}x"#).first?.fontName == "Arial Black") + } + + /// `\be`, `\bord`, `\iclip`, `\shad`, `\fscx` and `\fsp` all start with a letter this parser + /// cares about and none of them is the tag it looks like. + @Test("lookalike tags do not set the attribute they resemble") + func lookalikeTagsAreIgnored() { + #expect(runs(#"0,0,Default,,0,0,0,,{\be1}x"#).first?.isBold == false) + #expect(runs(#"0,0,Default,,0,0,0,,{\bord2}x"#).first?.isBold == false) + #expect(runs(#"0,0,Default,,0,0,0,,{\iclip(1,2,3,4)}x"#).first?.isItalic == false) + #expect(runs(#"0,0,Default,,0,0,0,,{\shad2}x"#).first?.isStruckThrough == false) + #expect(runs(#"0,0,Default,,0,0,0,,{\fscx200}x"#).first?.fontSize == nil) + #expect(runs(#"0,0,Default,,0,0,0,,{\fsp3}x"#).first?.fontSize == nil) + } + + @Test("a style reset clears every accumulated attribute") + func styleResetClearsAttributes() { + let got = runs(#"0,0,Default,,0,0,0,,{\b1\i1\fnArial}styled{\r}plain"#) + let plain = got.first(where: { $0.text == "plain" }) + #expect(plain?.isBold == false) + #expect(plain?.isItalic == false) + #expect(plain?.fontName == nil) + } + + // MARK: - Colour, the behaviour that already existed + + @Test("colour still parses and still splits runs") + func colourRegression() { + let got = runs(#"0,0,Default,,0,0,0,,{\c&H0000FF&}red{\c&H00FF00&}green"#) + #expect(got.count == 2) + #expect(got[0].color == SubtitleColor(r: 255, g: 0, b: 0)) + #expect(got[1].color == SubtitleColor(r: 0, g: 255, b: 0)) + } + + @Test("colour and a binary attribute coexist on one run") + func colourAndAttribute() { + let got = runs(#"0,0,Default,,0,0,0,,{\c&H0000FF&\i1}both"#) + #expect(got.count == 1) + #expect(got[0].color == SubtitleColor(r: 255, g: 0, b: 0)) + #expect(got[0].isItalic == true) + } + + // MARK: - Placement + + @Test("an alignment tag lands on the cue, not the run") + func alignmentIsCueLevel() { + #expect(placement(#"0,0,Default,,0,0,0,,{\an8}top"#)?.alignment == 8) + #expect(placement(#"0,0,Default,,0,0,0,,plain"#) == nil) + } + + /// A libavcodec-generated line positions against the default ASS play resolution, 384x288 + /// (`ASS_DEFAULT_PLAYRESX/Y`), so the centre of that space is the centre of the frame. + @Test("a position tag normalizes against the play resolution") + func positionNormalizes() { + let p = placement(#"0,0,Default,,0,0,0,,{\pos(192,144)}mid"#) + #expect(p?.position?.x == 0.5) + #expect(p?.position?.y == 0.5) + } + + @Test("a position is normalized against a declared play resolution when one is given") + func positionUsesDeclaredPlayRes() { + let parsed = SubtitleRectText.styledRuns( + fromASSEventLine: #"0,0,Default,,0,0,0,,{\pos(960,540)}mid"#, + playRes: CGSize(width: 1920, height: 1080)) + #expect(parsed?.placement?.position?.x == 0.5) + #expect(parsed?.placement?.position?.y == 0.5) + } + + @Test("play resolution is read from an ass header when it declares one") + func playResFromHeader() { + let header = """ + [Script Info] + ScriptType: v4.00+ + PlayResX: 1920 + PlayResY: 1080 + """ + #expect(SubtitleRectText.playRes(fromASSHeader: header) == CGSize(width: 1920, height: 1080)) + #expect(SubtitleRectText.playRes(fromASSHeader: "[Script Info]\nScriptType: v4.00+") == nil) + } + + // MARK: - The formats this actually serves + + /// What `ff_htmlmarkup_to_ass` emits for `ciao rosso`. + @Test("an srt line converted by libavcodec reaches the host styled") + func srtShapedLine() { + let got = runs(#"0,0,Default,,0,0,0,,{\i1}ciao{\i0} {\c&H0000FF&}rosso"#) + #expect(got.first(where: { $0.text == "ciao" })?.isItalic == true) + #expect(got.first(where: { $0.text == "rosso" })?.color == SubtitleColor(r: 255, g: 0, b: 0)) + } + + /// WebVTT maps only its inline tags; cue settings never arrive (webvttdec `@todo`). + @Test("a webvtt line converted by libavcodec reaches the host styled") + func webvttShapedLine() { + let got = runs(#"0,0,Default,,0,0,0,,{\b1}loud{\b0} then {\i1}soft"#) + #expect(got.first(where: { $0.text == "loud" })?.isBold == true) + #expect(got.first(where: { $0.text == "soft" })?.isItalic == true) + } + + // MARK: - Body selection + + @Test("an unstyled line stays plain text so nothing regresses for text-only hosts") + func unstyledStaysPlainText() { + guard case .text(let s) = SubtitleRectText.styledBody(fromASSEventLine: "0,0,Default,,0,0,0,,just words")?.body else { + Issue.record("expected .text") + return + } + #expect(s == "just words") + } + + @Test("a styled line becomes rich text") + func styledBecomesRichText() { + guard case .richText(let r) = SubtitleRectText.styledBody(fromASSEventLine: #"0,0,Default,,0,0,0,,{\i1}words"#)?.body else { + Issue.record("expected .richText") + return + } + #expect(r.count == 1) + #expect(r[0].isItalic == true) + } + + @Test("escapes and line breaks survive the styled path") + func escapesSurvive() { + let got = runs(#"0,0,Default,,0,0,0,,{\i1}one\Ntwo\hthree"#) + #expect(got.map(\.text).joined() == "one\ntwo three") + } + + // MARK: - Teletext behaviour that must not regress + + /// #107: libzvbi joins teletext rows with `\N`, so a caption on non-adjacent rows arrives as + /// `line1\n\nline2`. That fold is teletext-specific and must not apply to ASS or SRT, where a + /// blank line can be deliberate. + @Test("teletext still folds interior blank lines") + func teletextFoldsBlankLines() { + guard case .text(let s) = SubtitleRectText.teletextBody(fromASSEventLine: #"0,0,Default,,0,0,0,,line1\N\Nline2"#)?.body else { + Issue.record("expected .text") + return + } + #expect(s == "line1\nline2") + } + + @Test("the general path keeps a deliberate blank line") + func generalPathKeepsBlankLine() { + guard case .text(let s) = SubtitleRectText.styledBody(fromASSEventLine: #"0,0,Default,,0,0,0,,line1\N\Nline2"#)?.body else { + Issue.record("expected .text") + return + } + #expect(s == "line1\n\nline2") + } + + @Test("teletext colour still produces rich text") + func teletextColourRegression() { + guard case .richText(let r) = SubtitleRectText.teletextBody(fromASSEventLine: #"0,0,Default,,0,0,0,,{\c&H00FFFF&}yellow"#)?.body else { + Issue.record("expected .richText") + return + } + #expect(r[0].color == SubtitleColor(r: 255, g: 255, b: 0)) + } +} diff --git a/Tests/AetherEngineTests/Issue233CompositorStylingTests.swift b/Tests/AetherEngineTests/Issue233CompositorStylingTests.swift new file mode 100644 index 00000000..b7b173fa --- /dev/null +++ b/Tests/AetherEngineTests/Issue233CompositorStylingTests.swift @@ -0,0 +1,110 @@ +import Testing +import CoreGraphics +import CoreText +import Foundation +@testable import AetherEngine + +/// #233: the SW-PiP compositor flattened `.richText` to `runs.map(\.text).joined()` and drew every +/// cue in one white font at the bottom, so the styling the parser now preserves would have stopped +/// at the PiP window. It renders per-run attributes and honours the cue's placement instead. +struct Issue233CompositorStylingTests { + + private let frame = CGSize(width: 1920, height: 1080) + private let block = CGSize(width: 400, height: 100) + + private func origin(_ alignment: Int?, _ position: CGPoint?) -> CGPoint { + SubtitleFrameCompositor.blockOrigin( + placement: SubtitleTextPlacement(alignment: alignment, position: position), + blockSize: block, frameWidth: frame.width, frameHeight: frame.height, + margin: 64) + } + + // MARK: - Alignment without an explicit position + + @Test("alignment 1 pins the block to the bottom left inside the margin") + func bottomLeft() { + #expect(origin(1, nil) == CGPoint(x: 64, y: 64)) + } + + @Test("alignment 3 pins the block to the bottom right inside the margin") + func bottomRight() { + #expect(origin(3, nil) == CGPoint(x: 1920 - 400 - 64, y: 64)) + } + + @Test("alignment 8 pins the block to the top, centred") + func topCentre() { + #expect(origin(8, nil) == CGPoint(x: (1920 - 400) / 2, y: 1080 - 100 - 64)) + } + + @Test("alignment 5 centres the block on both axes") + func middleCentre() { + #expect(origin(5, nil) == CGPoint(x: (1920 - 400) / 2, y: (1080 - 100) / 2)) + } + + // MARK: - Explicit position + + /// `\pos` names the anchor point, and the alignment says which part of the block sits on it. + /// Position is normalized with y measured from the top; the context draws bottom-up. + @Test("a centred position anchors the block centre on the point") + func positionWithCentreAlignment() { + let got = origin(5, CGPoint(x: 0.5, y: 0.5)) + #expect(got == CGPoint(x: 960 - 200, y: 540 - 50)) + } + + @Test("a bottom-left alignment puts the block corner on the point") + func positionWithCornerAlignment() { + let got = origin(1, CGPoint(x: 0.25, y: 0.75)) + #expect(got == CGPoint(x: 480, y: 270)) + } + + @Test("a position with no alignment falls back to the ass default of bottom centre") + func positionDefaultsToBottomCentre() { + #expect(origin(nil, CGPoint(x: 0.5, y: 0.5)) == origin(2, CGPoint(x: 0.5, y: 0.5))) + } + + // MARK: - Rendering + + @Test("a styled cue still produces an overlay image") + func rendersStyledCue() { + let runs = [ + SubtitleTextRun(text: "bold ", color: SubtitleColor(r: 255, g: 0, b: 0), isBold: true), + SubtitleTextRun(text: "italic", color: nil, isItalic: true, fontName: "Helvetica", fontSize: 32), + ] + let cue = SubtitleCue(id: 1, startTime: 0, endTime: 5, body: .richText(runs)) + #expect(SubtitleFrameCompositor.renderOverlay(for: [cue], frameWidth: 640, frameHeight: 360) != nil) + } + + @Test("a placed cue still produces an overlay image") + func rendersPlacedCue() { + let cue = SubtitleCue( + id: 1, startTime: 0, endTime: 5, body: .text("top"), + placement: SubtitleTextPlacement(alignment: 8, position: nil)) + #expect(SubtitleFrameCompositor.renderOverlay(for: [cue], frameWidth: 640, frameHeight: 360) != nil) + } + + /// Bold and italic have to reach CoreText as real traits, not be silently dropped. + @Test("bold and italic resolve to a font carrying those traits") + func fontTraits() { + let plain = SubtitleFrameCompositor.font( + for: SubtitleTextRun(text: "x", color: nil), baseSize: 40) + let bold = SubtitleFrameCompositor.font( + for: SubtitleTextRun(text: "x", color: nil, isBold: true), baseSize: 40) + let italic = SubtitleFrameCompositor.font( + for: SubtitleTextRun(text: "x", color: nil, isItalic: true), baseSize: 40) + #expect(CTFontGetSymbolicTraits(plain).contains(CTFontSymbolicTraits.traitBold) == false) + #expect(CTFontGetSymbolicTraits(bold).contains(CTFontSymbolicTraits.traitBold)) + #expect(CTFontGetSymbolicTraits(italic).contains(CTFontSymbolicTraits.traitItalic)) + } + + /// ASS `\fs` is relative to libavcodec's default style size of 16, so it scales the base size + /// rather than naming pixels. + @Test("an ass font size scales the base size relative to the ass default") + func fontSizeScales() { + let doubled = SubtitleFrameCompositor.font( + for: SubtitleTextRun(text: "x", color: nil, fontSize: 32), baseSize: 40) + #expect(CTFontGetSize(doubled) == 80) + let unset = SubtitleFrameCompositor.font( + for: SubtitleTextRun(text: "x", color: nil), baseSize: 40) + #expect(CTFontGetSize(unset) == 40) + } +} diff --git a/Tests/AetherEngineTests/TeletextColourRunsTests.swift b/Tests/AetherEngineTests/TeletextColourRunsTests.swift index 94c949fc..1afaf84a 100644 --- a/Tests/AetherEngineTests/TeletextColourRunsTests.swift +++ b/Tests/AetherEngineTests/TeletextColourRunsTests.swift @@ -1,20 +1,26 @@ import Testing @testable import AetherEngine +/// #107 teletext colour, kept green through the #233 generalization: the colour parser became a +/// full ASS override parser (`coloredRuns` -> `styledRuns`) and the interior blank-line fold moved +/// out of the run parser into `teletextBody`, where it belongs, since only libzvbi's row joining +/// produces those blank lines. struct TeletextColourRunsTests { + private func runs(_ line: String) -> [SubtitleTextRun]? { + SubtitleRectText.styledRuns(fromASSEventLine: line)?.runs + } + // ASS colour is &Hbbggrr& (BGR). &H00FFFF& = R=255 G=255 B=0 (yellow). @Test("parses BGR hex into RGB runs") func bgrToRgb() { let line = "0,0,Default,,0,0,0,,{\\c&H00FFFF&}Yellow" - let runs = SubtitleRectText.coloredRuns(fromASSEventLine: line) - #expect(runs == [SubtitleTextRun(text: "Yellow", color: SubtitleColor(r: 255, g: 255, b: 0))]) + #expect(runs(line) == [SubtitleTextRun(text: "Yellow", color: SubtitleColor(r: 255, g: 255, b: 0))]) } @Test("splits into multiple runs at colour changes, collapses equal-colour neighbours") func multiRun() { let line = "0,0,Default,,0,0,0,,{\\c&H0000FF&}Red {\\c&HFFFFFF&}White" - let runs = SubtitleRectText.coloredRuns(fromASSEventLine: line) - #expect(runs == [ + #expect(runs(line) == [ SubtitleTextRun(text: "Red ", color: SubtitleColor(r: 255, g: 0, b: 0)), SubtitleTextRun(text: "White", color: SubtitleColor(r: 255, g: 255, b: 255)), ]) @@ -23,8 +29,7 @@ struct TeletextColourRunsTests { @Test("bare \\c resets to default (nil colour)") func resetColour() { let line = "0,0,Default,,0,0,0,,{\\c&H0000FF&}Red{\\c}plain" - let runs = SubtitleRectText.coloredRuns(fromASSEventLine: line) - #expect(runs == [ + #expect(runs(line) == [ SubtitleTextRun(text: "Red", color: SubtitleColor(r: 255, g: 0, b: 0)), SubtitleTextRun(text: "plain", color: nil), ]) @@ -33,41 +38,36 @@ struct TeletextColourRunsTests { @Test("applies \\N newline and \\h hard space escapes") func escapes() { let line = "0,0,Default,,0,0,0,,line1\\Nline2\\hend" - let runs = SubtitleRectText.coloredRuns(fromASSEventLine: line) - #expect(runs == [SubtitleTextRun(text: "line1\nline2 end", color: nil)]) + #expect(runs(line) == [SubtitleTextRun(text: "line1\nline2 end", color: nil)]) } @Test("uncoloured line yields a single nil-colour run") func noColour() { - let runs = SubtitleRectText.coloredRuns(fromASSEventLine: "0,0,Default,,0,0,0,,just text") - #expect(runs == [SubtitleTextRun(text: "just text", color: nil)]) + #expect(runs("0,0,Default,,0,0,0,,just text") == [SubtitleTextRun(text: "just text", color: nil)]) } @Test("teletextBody returns richText when coloured, text when not, nil when empty") func bodySelection() { - if case .richText? = SubtitleRectText.teletextBody(fromASSEventLine: "0,0,D,,0,0,0,,{\\c&H0000FF&}Red") {} else { Issue.record("expected richText") } - if case .text(let s)? = SubtitleRectText.teletextBody(fromASSEventLine: "0,0,D,,0,0,0,,plain") { #expect(s == "plain") } else { Issue.record("expected text") } + if case .richText? = SubtitleRectText.teletextBody(fromASSEventLine: "0,0,D,,0,0,0,,{\\c&H0000FF&}Red")?.body {} else { Issue.record("expected richText") } + if case .text(let s)? = SubtitleRectText.teletextBody(fromASSEventLine: "0,0,D,,0,0,0,,plain")?.body { #expect(s == "plain") } else { Issue.record("expected text") } #expect(SubtitleRectText.teletextBody(fromASSEventLine: "0,0,D,,0,0,0,,{\\c&H0&}") == nil) } @Test("non-event line with too few fields is cleaned as-is") func nonEventLine() { - let runs = SubtitleRectText.coloredRuns(fromASSEventLine: "plain, with, commas") - #expect(runs == [SubtitleTextRun(text: "plain, with, commas", color: nil)]) + #expect(runs("plain, with, commas") == [SubtitleTextRun(text: "plain, with, commas", color: nil)]) } @Test("adjacent same-colour runs collapse into one") func collapseSameColour() { let line = "0,0,Default,,0,0,0,,{\\c&H0000FF&}A{\\c&H0000FF&}B" - let runs = SubtitleRectText.coloredRuns(fromASSEventLine: line) - #expect(runs == [SubtitleTextRun(text: "AB", color: SubtitleColor(r: 255, g: 0, b: 0))]) + #expect(runs(line) == [SubtitleTextRun(text: "AB", color: SubtitleColor(r: 255, g: 0, b: 0))]) } @Test("non-colour override tags like \\clip are ignored, not treated as a reset") func clipTagIgnored() { let line = "0,0,Default,,0,0,0,,{\\c&H0000FF&}Red{\\clip(1,2,3,4)}still" - let runs = SubtitleRectText.coloredRuns(fromASSEventLine: line) - #expect(runs == [SubtitleTextRun(text: "Redstill", color: SubtitleColor(r: 255, g: 0, b: 0))]) + #expect(runs(line) == [SubtitleTextRun(text: "Redstill", color: SubtitleColor(r: 255, g: 0, b: 0))]) } @Test("leading/trailing newlines are edge-trimmed across coloured runs (no blank line)") @@ -75,8 +75,7 @@ struct TeletextColourRunsTests { // libzvbi teletext ass can prefix a row-positioning newline; a coloured cue must not // render a blank line the plain path already trims. Interior line breaks are kept. let line = "0,0,Default,,0,0,0,,\\N{\\c&H0000FF&}Red\\NWhite\\N" - let runs = SubtitleRectText.coloredRuns(fromASSEventLine: line) - #expect(runs == [SubtitleTextRun(text: "Red\nWhite", color: SubtitleColor(r: 255, g: 0, b: 0))]) + #expect(runs(line) == [SubtitleTextRun(text: "Red\nWhite", color: SubtitleColor(r: 255, g: 0, b: 0))]) } @Test("interior blank line from a skipped teletext row collapses to a single break (#107)") @@ -85,14 +84,7 @@ struct TeletextColourRunsTests { // row between them, used only for vertical placement) arrives as line1\N\Nline2 and would // render a blank line the broadcaster never intended. It must read as two adjacent lines. let line = "0,0,Default,,0,0,0,,Can you tell someone\\N\\Nthey're not a good singer?" - let runs = SubtitleRectText.coloredRuns(fromASSEventLine: line) - #expect(runs == [SubtitleTextRun(text: "Can you tell someone\nthey're not a good singer?", color: nil)]) - } - - @Test("teletextBody flattens the collapsed blank line on the plain-text path (#107)") - func teletextBodyCollapsesInteriorBlankLine() { - let line = "0,0,Default,,0,0,0,,Can you tell someone\\N\\Nthey're not a good singer?" - if case .text(let s)? = SubtitleRectText.teletextBody(fromASSEventLine: line) { + if case .text(let s)? = SubtitleRectText.teletextBody(fromASSEventLine: line)?.body { #expect(s == "Can you tell someone\nthey're not a good singer?") } else { Issue.record("expected text body") @@ -102,14 +94,16 @@ struct TeletextColourRunsTests { @Test("multiple skipped rows collapse to a single break, colours preserved (#107)") func collapsesMultipleBlankRowsColoured() { let line = "0,0,Default,,0,0,0,,{\\c&H00FFFF&}Line A\\N\\N\\NLine B" - let runs = SubtitleRectText.coloredRuns(fromASSEventLine: line) - #expect(runs == [SubtitleTextRun(text: "Line A\nLine B", color: SubtitleColor(r: 255, g: 255, b: 0))]) + if case .richText(let r)? = SubtitleRectText.teletextBody(fromASSEventLine: line)?.body { + #expect(r == [SubtitleTextRun(text: "Line A\nLine B", color: SubtitleColor(r: 255, g: 255, b: 0))]) + } else { + Issue.record("expected richText body") + } } @Test("adjacent teletext rows keep their single line break (no over-collapse)") func keepsAdjacentRowBreak() { let line = "0,0,Default,,0,0,0,,First line\\NSecond line" - let runs = SubtitleRectText.coloredRuns(fromASSEventLine: line) - #expect(runs == [SubtitleTextRun(text: "First line\nSecond line", color: nil)]) + #expect(runs(line) == [SubtitleTextRun(text: "First line\nSecond line", color: nil)]) } } From 1d1b2face800d077b75f30fd7fb1501f5503be07 Mon Sep 17 00:00:00 2001 From: Vincent Herbst Date: Tue, 28 Jul 2026 15:06:37 +0200 Subject: [PATCH 2/2] test(subtitles): update the sidecar default-path contract for #233 SidecarASSMarkupTests is an XCTest suite, so its failure did not appear in the Swift Testing summary I checked locally and CI caught it instead. The assertion was correct for the old behaviour: the default sidecar path flattened {\i1}Hello{\i0} world to plain text. That is the behaviour #233 deliberately changes, and this test is a useful end-to-end proof of it, since the fixture also declares PlayResX/Y and so exercises the header play-resolution parsing. It now asserts the new contract: still header-less, still no raw event lines, but a styled cue arrives as .richText with the italic resolved and an unstyled one stays .text. The flattened SubtitleCue.text is asserted unchanged, which is what a host reading plain text actually sees. XCTest 392 tests 0 failures, Swift Testing 1210 tests green. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01At1agC2qU2Y2MX2BKLdT4N --- .../SidecarASSMarkupTests.swift | 22 +++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/Tests/AetherEngineTests/SidecarASSMarkupTests.swift b/Tests/AetherEngineTests/SidecarASSMarkupTests.swift index 2c75ca09..042e1ceb 100644 --- a/Tests/AetherEngineTests/SidecarASSMarkupTests.swift +++ b/Tests/AetherEngineTests/SidecarASSMarkupTests.swift @@ -54,7 +54,11 @@ final class SidecarASSMarkupTests: XCTestCase { XCTAssertTrue(builder.script().contains("{\\i1}Hello{\\i0} world")) } - func testDefaultPathStaysPlainTextAndHeaderless() async throws { + /// #233 changed half of this contract. The default path is still header-less and still hands + /// over no raw event lines, but it no longer flattens the markup away: a styled cue arrives as + /// `.richText` with the attributes resolved, and only an unstyled one stays `.text`. The + /// flattened string a host reads through `SubtitleCue.text` is unchanged either way. + func testDefaultPathResolvesStylingAndStaysHeaderless() async throws { let url = try writeTempASS() defer { try? FileManager.default.removeItem(at: url) } @@ -62,9 +66,19 @@ final class SidecarASSMarkupTests: XCTestCase { XCTAssertNil(result.assHeader, "no header when markup preservation is off") XCTAssertEqual(result.cues.count, 2) - guard case let .text(first) = result.cues[0].body else { - return XCTFail("expected text body") + + guard case let .richText(runs) = result.cues[0].body else { + return XCTFail("expected a styled body for {\\i1}Hello{\\i0} world") + } + XCTAssertEqual(runs.map(\.text), ["Hello", " world"]) + XCTAssertTrue(runs[0].isItalic, "the \\i1 run must reach the host italic") + XCTAssertFalse(runs[1].isItalic, "\\i0 must end the italic run") + XCTAssertEqual(result.cues[0].text, "Hello world", "the flattened accessor is unchanged") + + // No override tags, so nothing to resolve and the body stays exactly what it was. + guard case let .text(second) = result.cues[1].body else { + return XCTFail("expected a plain body for an unstyled line") } - XCTAssertEqual(first, "Hello world") + XCTAssertEqual(second, "Second line") } }