Skip to content

Add true HDR output, RT shadows, and PCSS shadows for vulkan#7554

Open
The-E wants to merge 18 commits into
scp-fs2open:masterfrom
The-E:true-hdr
Open

Add true HDR output, RT shadows, and PCSS shadows for vulkan#7554
The-E wants to merge 18 commits into
scp-fs2open:masterfrom
The-E:true-hdr

Conversation

@The-E

@The-E The-E commented Jun 30, 2026

Copy link
Copy Markdown
Member

Note that this is based on @notimaginative's PR (#7553) and should only be merged afterwards.
This also adapts the work done in @BMagnu's Shadow Overhaul PR (#7529), which is therefore another path dependency.

Summary

Adds optional HDR10 (PQ / ST.2084 + BT.2020) swap chain output to the Vulkan renderer, with paper-white / peak-luminance controls and an in-game calibration screen. When disabled (the default) or when running on a non-HDR display / the OpenGL renderer, behavior is unchanged.

This PR has since grown to also bring the Vulkan renderer's shadow rendering up to par with the shadow-rendering overhaul merged in from lafiel/shadow_overhaul_2 (dynamic cascade counts, PCF/PCSS, cockpit shadow cascades, and an optional raytraced-shadow path), since both landed on this branch together. Unlike the HDR work, the shadow overhaul itself affects both the OpenGL and Vulkan backends.

What's included

Swap chain & metadata (VulkanRendererSetup.cpp)

  • Enables VK_EXT_swapchain_colorspace (instance) and VK_EXT_hdr_metadata (device, when available).
  • When HDR is requested, negotiates an A2B10G10R10 swap chain with the HDR10_ST2084 color space, falling back cleanly to SDR/sRGB otherwise.
  • Submits VkHdrMetadataEXT (paper white / peak luminance, BT.2020 primaries) and re-applies it on swap chain recreation (resize / fullscreen toggle).

Frame composition refactor (VulkanRenderer.cpp, VulkanRendererLoop.cpp)

  • Introduces an intermediate RGBA16F composition buffer at window resolution. All rendering (scene, UI, ImGui) now targets this buffer.
  • A final encode pass converts the composition buffer to the swap chain: SDR passthrough when HDR is off, HDR10 PQ encode when on.

Tonemap / output shaders (tonemapping-f.sdr, gamma.sdr)

  • New hdr_mode uniform: 0 = existing SDR tonemap, 1 = HDR scene tonemap (exposure + headroom clamp relative to paper white, stored as extended sRGB in the fp16 composition buffer), 2 = HDR10 output encode (linearize, scale to nits, BT.709 -> BT.2020, PQ encode).
  • LDR intermediate targets (Scene_ldr, Scene_luminance) widen to fp16 when HDR is active so highlights above paper white survive.

Post-process anti-aliasing (VulkanPostProcessingSMAA.cpp, VulkanPostProcessingLDR.cpp, smaa-*.sdr, fxaapre-f.sdr)

  • Ported SMAA to Vulkan. It previously had no Vulkan implementation at all — Graphics.AAMode listed the SMAA presets unconditionally, but selecting one while running Vulkan silently produced zero anti-aliasing with no warning. All three passes (edge detection, blending-weight calculation, neighborhood blending) are now implemented, plus the area/search lookup texture uploads; the shared SMAA.sdr algorithm body needed no changes, only thin per-backend wrapper shaders.
  • FXAA and SMAA both now work correctly while HDR10 output is active, instead of being unconditionally disabled there. Their fixed luma thresholds assume [0,1] LDR input, which breaks once Scene_ldr carries extended-range values; fixed by tonemapping a second, properly-compressed proxy buffer for edge/luma detection only, while the actual blended output still reads the real extended-range colors — so anti-aliasing works without sacrificing HDR headroom.

Options (2d.cpp / 2d.h)

  • Graphics.HDR (bool, requires restart), Graphics.HDRPaperWhite (nits, live), Graphics.HDRPeakLuminance (nits, live), persisted via the options system.
  • Gr_hdr_output_active reflects whether the renderer actually negotiated an HDR10 swap chain (distinct from the request flag).

Calibration screen (ingame_options_ui.cpp / .h)

  • New "HDR Calibration" entry in the SCP Options menu.
  • Status banner (active / enabled-but-inactive / disabled), live paper-white and peak-luminance sliders bound to the persistent options, a paper-white-relative grayscale ramp, and R/G/B/white primary patches for sanity-checking the BT.709 -> BT.2020 mapping.

OpenGL (gropenglpostprocessing.cpp)

  • New tonemap UBO fields are explicitly zero-initialized so the OpenGL backend's behavior is unchanged.

HDR framebuffer readback (1c3f8b7f9)

  • Adds an fp16 -> BGRA8 conversion path for framebuffer readback, so screenshots and other systems that assume 8-bit output keep working when HDR display output is enabled.

Vulkan raytraced shadows — TLAS/BLAS (63f8fa059)

  • New VulkanRaytracingManager: BLAS caching keyed to model lifecycle, per-frame TLAS build/rebuild (VulkanRaytracingTlas.cpp, VulkanRaytracingBlas.cpp) with dynamic capacity growth.
  • Vulkan buffer/device-address plumbing needed for acceleration structures (VulkanBuffer, VulkanMemory).
  • Descriptor manager support for binding the TLAS and RT-shadow resources, plus a capability query for raytracing support (gr_vulkan.cpp).

Shadow rendering overhaul, OpenGL side (merged from shadow_overhaul_2)

  • Extracted shadow rendering into its own shader/render pass and consolidated render-queue code — model_draw_list and the new shadow_render_list now share a common render_queue<Derived, DrawEntryT> CRTP base (code/graphics/render_queue.h).
  • Switched from VSM to hardware PCF + PCSS with proper depth textures; smoothness exposed as a mod-table option.
  • Shadow cascade count is now dynamic (mod-table configurable) instead of a hardcoded 4, and cockpit geometry gets its own dedicated shadow cascades instead of fighting with the main scene's.
  • Assorted correctness fixes: depth-clamped shadow geometry to avoid artifacts, FOV multiplier fix, shadow cascade override handling, inter-frame shadow buffer clearing.

Vulkan shadow parity port (f35864fda)

  • shader_get_shadow_cascade_defines() shared between both backends' shader compilers so cascade-count defines stay in sync.
  • New VulkanDrawManager::renderShadowDraw(): builds the SDR_TYPE_SHADOW_MAP_GEN pipeline, binds ShadowMapData/ShadowCascadeParams UBOs, and does a single instanced draw across all active cascades, replacing the old "spawn N instances" hack.
  • VulkanShadowMap rewritten: dropped the VSM color target entirely, dynamic layer count (Num_shadow_cascades + Num_cockpit_shadow_cascades), and a depth-compare sampler for hardware PCF via sampler2DArrayShadow.
  • New descriptor bindings (GlobalBinding::ShadowCascadeParams, MaterialBinding::ShadowMapData), and VulkanDeletionQueue::queueAccelerationStructure() so TLAS rebuilds defer destruction of the old acceleration structure instead of destroying it synchronously out from under an in-flight command buffer.
  • Shaders unified across both backends (single transformToShadowMap(), matching sampler2DArrayShadow usage, matching vertex/fragment interface blocks, shared shadowCascadeParams UBO layout).
  • Fixed a startup crash (shadow_cascade_params_init() needed to run eagerly, not lazily, since shadows_start_render() indexes Shadow_frustums before the lazy init point is reached) and a validation error from building the shadow TLAS inside an active render pass.
  • Fixed a Vulkan-only regression where destroyed/blown-off submodels (e.g. a ship's turret rotator after its subsystem is destroyed) stayed visible, stuck in place with an identity transform, instead of being hidden — the shader computed the "clip this submodel" flag but the code that acted on it was accidentally wrapped in #ifndef VULKAN.

Logging

  • Vulkan logging calls unified on mprintf/scoped nprintf categories for consistency with the rest of the codebase, plus a new in-game log filter UI for toggling categories and browsing session logs.

Fixes picked up along the way

  • Fixed a Vulkan startup crash: gr_uniform_buffer_managers_init() was only ever called from the OpenGL init path, so UniformBufferManager stayed null under Vulkan and the first draw needing a uniform buffer (title screen rendering) dereferenced it (SIGSEGV).
  • Fixed all four Mac CI configs failing with ccache: No such file or directory — the workflow hardcodes the Intel Homebrew ccache path (/usr/local/bin/ccache), which doesn't exist on Apple Silicon-hosted GitHub runners (Homebrew installs under /opt/homebrew there). configure_cmake.sh now falls back to a PATH-resolved ccache when the configured path isn't executable.

Notes / limitations

  • The HDR10 output path itself is Vulkan only; OpenGL is intentionally untouched there. The shadow-rendering overhaul affects both backends, with this PR bringing Vulkan up to parity with the OpenGL-side changes.
  • HDR requires a restart to take effect (swap chain format negotiation).
  • ImGui patches cannot exceed paper white, so peak luminance is best tuned in-game by raising it until bright highlights/explosions stop clipping.

@The-E The-E requested review from asarium and z64555 as code owners June 30, 2026 17:45

@BMagnu BMagnu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check out SDR_FLAG_TONEMAPPING_LINEAR_OUT.
It's basically poor-man's HDR output in the OpenGL pass, currently only used for the OpenXR pass, since the headset swapchain seems to do its own tonemap, or at the very least expects non-SDR input (it's highly possible that proper HDR input is in fact the correct thing to forward here, instead of truly linear data).
This should very likely be merged into this work, even if it means supporting OpenGL to some degree, rather than exist as a weird, second, half-baked HDR pass. Especially since proper handling across not only gameplay but also menus will fix #7181.
I'm not yet fully familiar with most of the Vulkan PR, but I assume that before this change, there is no handling of Cmdline_window_res, which this new buffer could then do in the future as well? I also assume that this PR is going to be the proper place to mirror #7484 for the SDL3 upgrade. If so, we should at least prepare / design this in a way that makes it easy to retrofit.

@The-E The-E changed the title Add true HDR output Add true HDR output, RT shadows, and PCSS shadows for vulkan Jul 4, 2026
BMagnu and others added 5 commits July 5, 2026 13:38
…adows

Squashed snapshot of lafiel/shadow_overhaul_2's continuation work
(d3922aa..010f9d2), which has no separate upstream PR of its own.
Squashed snapshot of the Vulkan backend as vendored into this branch
(ce2d1e8..6ce5394, authored by Mara van der Laan with build/CI
fixes by Taylor Richards). This work has its own upstream PR scp-fs2open#7553
(notimaginative:vulkan-pr-new) and is not otherwise original to true-hdr.
Squashed snapshot of true-hdr's own integration work: true HDR
framebuffer output, wiring the shadow overhaul into the Vulkan
backend (raytraced + cascaded shadow maps), logging refactors, log
filter UI, and CI/build fixes.
The-E and others added 10 commits July 5, 2026 15:13
CI hardcodes /usr/local/bin/ccache, but macOS runners on Apple Silicon
hosts install Homebrew (and ccache) under /opt/homebrew instead,
causing all four Mac configs to fail with "No such file or directory".
pixel_swizzle.{h,cpp}: collapse `namespace graphics { namespace util {`
into `namespace graphics::util {` (modernize-concat-nested-namespaces).

VulkanState.h/VulkanDraw.h: getClearColor()/setClearColor() only touch
gr_screen and other globals, no instance state, so make them static
(readability-convert-member-functions-to-static) and drop the
top-level const on getClearColor()'s return type
(readability-const-return-type).
@The-E The-E added graphics A feature or issue related to graphics (2d and 3d) vulkan Issues and Features related to the vulkan render backend labels Jul 5, 2026
claude and others added 3 commits July 5, 2026 20:36
RT shadows previously only shadowed the first directional light in the
scene, even though multi-sun missions push more than one directional
light into the engine. The TLAS raytraced shadows query against is
already light-agnostic, so this extends both the deferred lighting
pass and the forward MODEL shader to shadow every directional light
(capped by a new Max Raytraced Shadow Lights option, configurable via
the in-game Options menu or the Lab's Scene rendering options).

Cascaded shadow maps are untouched: they're built for a single light
only, so the deferred pass still limits itself to one shadowed
directional light whenever CSM (not raytracing) is the active method.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_016DA6JHJ3twz2oHGQcZ1Udc
… in Vulkan backend

- Consolidate shader flag setup logic in VulkanPostProcessingLighting to simplify configuration.
- Fix dead field usage in VulkanDraw by ensuring correct image view selection for environment maps.
- Add mip chain regeneration in VulkanRenderer::endRenderTarget to ensure correct mip levels for cubemaps.
- Add mip chain generation to VulkanDraw for cubemaps with multiple mip levels.
- Fix potential issue with uninitialized mip levels above level 0 in deferred-f.sdr sampling paths.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

graphics A feature or issue related to graphics (2d and 3d) vulkan Issues and Features related to the vulkan render backend

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants