Building a Raw Vulkan HDR Compositor in Rust

“Enable HDR” sounds like a display switch. In a compositor, it is an end-to-end data contract. A client buffer arrives with a format and a color meaning; composition needs a linear, high-precision working space; the final image needs the right gamut and transfer function; and the kernel must scan out a matching format with matching metadata.

When any one of those statements is merely implied, the monitor can enter HDR mode while the picture is still wrong. That was the failure I needed to make observable in FocalDesk. The higher-level rendering paths were productive for normal desktop composition, but they did not expose enough of the HDR chain to isolate whether a bad result came from import, sampling, blending, encoding, synchronization, or presentation.

The first approaches produced HDR signals—not trustworthy HDR images.

My first attempts kept more of the established rendering abstraction in place. They could negotiate an HDR-capable output and make the display report an HDR mode, but the visual result exposed a deeper problem: signaling HDR does not prove that every pixel was interpreted and transformed correctly before scanout.

Trying to compensate at the end of the pipeline was equally fragile. A final shader can apply a transfer function, but it cannot repair content that was decoded with the wrong assumptions or blended while still non-linear. It can also make one test image look plausible while breaking ordinary sRGB UI, wide-gamut P3 content, or video.

The color bugs were disagreements between stages.

The investigation reduced the problem to explicit questions at each boundary. Is the incoming value encoded or linear? Which primaries give those numbers meaning? Has content already been mapped? Which precision does composition preserve? What transfer function does the display expect? The pipeline needed one answer for every buffer, not a global guess.

Three distinctions mattered. Transfer functions are not gamuts: decoding sRGB and encoding PQ are different operations from converting color primaries. Blending belongs in linear light, not directly in encoded sRGB or PQ values. And DRM color metadata describes the signal sent to the display; it does not perform the pixel conversion on the compositor’s behalf.

The breakthrough was treating color as typed data all the way through the renderer—not as three floating-point channels whose meaning could be inferred later.

Raw Ash made every hidden decision an owned decision.

I implemented a dedicated Vulkan renderer directly with Ash. Client DMA-BUFs are imported as Vulkan images, sampled according to their source meaning, and composed into an FP16 intermediate target. The extra precision keeps the working image from being prematurely constrained to the final 10-bit signal and gives SDR and HDR content room to coexist during composition.

The output pass converts the composed image to the target output color space, applies the PQ encoding expected by the HDR10 signal, and writes a 10-bit scanout buffer. GBM provides buffers the display engine can use; atomic KMS commits the framebuffer and output state together, including the connector’s HDR metadata and color-space properties.

Explicit synchronization completes the ownership model. GPU rendering and KMS presentation cannot race over the same buffer, and reuse waits on the correct completion boundary. That matters for correctness even when a synchronization bug presents as an intermittent visual artifact rather than an obvious crash.

Mixed SDR and HDR outputs made a better systems test.

FocalDesk runs the path with an HDR display and an SDR display active at the same time. That prevents “HDR mode” from becoming one global desktop assumption. Each output needs its own target transform, format, metadata, and presentation state while the compositor continues to manage one coherent session.

I exercised ordinary sRGB interface content, Display P3 material, and HDR video across that configuration. Browser diagnostics reported HDR support and 30-bit output, providing an application-level check alongside the DRM and compositor state.

The current evidence is visual hardware validation, not formal calibration.

The working configuration is Fedora 44 on an NVIDIA GeForce RTX 4090 with the NVIDIA 595.91.07 driver. It produces a stable 10-bit BT.2020/PQ path, preserves usable sRGB and P3 content, plays HDR video, and operates mixed HDR/SDR outputs. Those results validate the integration across userspace, the driver, DRM/KMS, and real displays.

The next validation step is an instrumented ArgyllCMS run. FocalDesk can now drive configured SDR reference white, a 10% HDR peak window, and a sustained full-frame HDR field, read luminance and chromaticity throughspotread, enforce tolerances, and save a report. That workflow is implemented and software-tested, but no meter result has been recorded yet. I will publish the display, meter, correction, targets, and measured values before describing the output as colorimeter-verified.

The low-level path was valuable because it made failure legible.

Raw Vulkan is not automatically the right renderer for every compositor. It creates substantial responsibility: memory types, image layouts, descriptors, command buffers, queues, fences, semaphores, buffer lifetime, format negotiation, and error recovery all become application concerns.

Here, that responsibility was the point. Owning Ash → GBM → atomic KMS created a single inspectable chain from a client’s DMA-BUF to photons on a physical display. It turned “the color looks wrong” from an observation into a set of testable contracts—and produced an HDR implementation whose behavior I can explain, instrument, and continue to validate.