Add HDR10 output support to the DX11 renderer

[deggua]

Changes:

• Duplicated libraries/x64, moved the original library files to libraries/x64.old, the new libraries are in libraries/x64. This contains the latest DirectX libraries from the Windows 10 SDK (version 10.0.22000.0\um\x64). Obviously committing binaries like this isn't a good idea (for security and practicality) but this was the simplest way to get the DX11 renderer to build using the new interfaces (specifically CreateDXGIFactory2) required for HDR support, and there were already old DX SDK library binaries in the repo. I'm in favor of removing these if there's a better way of providing libraries from the Windows SDK (there probably is but I'm not familiar with VS configuration and how it prioritizes library search directories, or how to do that in a way that works outside my machine)
• Added HDR parameters to the console variables and global shader constants. ps_r4_hdr_whitepoint_nits should be set to the monitor's HDR whitepoint (max brightness). ps_r4_hdr_ui_nits is the UI brightness in nits and should be set below the whitepoint at a level that is comfortable for the user. ps_r4_hdr_pda isn't a console variable, but a global I'm using as a flag to determine when the PDA's UI is being rendered, this is to avoid applying HDR tonemapping twice since it is also rendered in the world as a texture. ps_r4_hdr_on controls whether HDR is enabled or not (requires game restart so the render targets are created in the correct format)
• In order for the renderer to support HDR output the swapchain and DX 11 version had to be updated to use the dxgi11_4/d3d11_4 interfaces, HDR also requires using a different swapchain flip effect, this also has performance benefits for borderless and windowed modes (see: https://learn.microsoft.com/en-us/windows/win32/direct3ddxgi/for-best-performance--use-dxgi-flip-model ) making them perform the same as exclusive fullscreen. There are probably other performance improvements available, one flag I haven't turned on is D3D11_CREATE_DEVICE_SINGLETHREADED which should improve performance provided the renderer is singlethreaded (I think it is, but not sure if someone wants to comment on that)
• I had to rework some of the GUID symbols in the xrNetEngine because their symbol names collide with symbols in the Windows SDK's dxguid.lib which would cause linker errors when building

Issues:

• Built in screenshots won't work in HDR mode because the backbuffer is a different texture format than the code expects, I put a TODO in because even if it was fixed there are other issues that make them useless (output format would require HDR support)
• The PDA global is a shitty hack, maybe there's a better way that someone could suggest?
• HDR should probably be toggleable from the Modded Exes section of the menu, haven't figured out how to do that yet
• I'm not sure the way I've integrated the HDR option is the best (just a global console var), it seems like there are options that e.g. SSFX uses that are in RImplementation, I probably need to look into how all that works
• To get the UI to be HDR tonemapped, I had to modify the stub_default.ps shader to apply HDR tonemapping. I'm not sure if this breaks anything else that uses this shader (visually I haven't noticed any issues). A better solution would be to modify the logic the game uses for selecting the shader for UI and have it pick something else, but it's not clear how best to do that or if it is even required
  • I realized I can just add a hud_default.s and a hud_cursor.s to specify the pixel shaders as separate files

Requirements:

• For the HDR output to be tonemapped correctly into the ST2084 colorspace the monitor expects, the shaders here are required: https://github.com/deggua/xray-hdr10-shaders
• I've provided shaders for Anomaly and GAMMA (GAMMA is really just SSS's combine_2_naa.ps shader modified to apply the HDR tonemapping)

Any feedback is appreciated. If you want to direct message me my Discord name is Antagonist in the GAMMA discord (username: @antag0n1st)

[deggua]

Some other issues I've noticed in testing:

• I'm not sure the HDR tonemapping shader code is correct (some colors seem a little off), I got the code from here: https://panoskarabelas.com/blog/posts/hdr_in_under_10_minutes/ which seems to be a transform from Rec.709 to Rec.2020 then to ST2084. I believe the game renders in sRGB (i.e. not linear) colorspace so I apply the inverse gamma correction to put it into linear color, but I'm not sure that the game is in Rec.709 at that point so the color transform may not be correct
  • After more reading I think it is correct
• Setting r4_hdr_whitepoint_nits to the max screen brightness (at least on my screen that claims to support peak 1K nits) seems to blow out the image too much, setting it to a value somewhat below the screen's claimed whitepoint (800 nits in my case) looks much more reasonable, the HDR tonemapping code might be at fault, or the game is too bright to begin with, etc. Not really sure, but tweaking r__gamma might help.
  • This is an issue with some monitor's HDR modes, the VESA certified ones should not have issues with blowout or auto-brightness limiting
• Seems like there might be more compatibility issues with SSFX/SSS, enabling the Force Volumetric option in SSS then turning on the flashlight results in a way too bright volumetric flashlight cone (even for day time). I'm not sure why this is, but basically any shader that renders directly to the backbuffer (or at least doesn't go through the combine_2_naa.ps shader) needs HDR tonemapping to be applied to its output for it to look correct. Or it could be because the render targets have much higher ranges and don't become saturated to white (e.g. like they would for R8G8B8A8). I'd have to look more into how SSS's volumetric lighting works
  • This seems to happen if Force Volumetric was enabled then turned off without reloading the map
• The postprocess.ps shader likely needs more corrections. I corrected the color tinting (in a shitty way mind you, doing the conversion in the pixel shader on what is basically a uniform seems like a waste unless the shader is optimized to evaluate that only once). But other effects that are applied in postprocessing seem a little off. For example, the game does some kind of desaturation with the gray vertex color and applies noise and brightness that probably assumes sRGB/linear color space. This results in noticeable darkening when postprocessing is applied (e.g. near an anomaly). Technically speaking not hard to fix, but it's not entirely clear what the best way to do so is. Probably keeping everything in sRGB/linear and writing to an intermediate render target instead of the backbuffer, applying all the image corrections/modifications, etc. then a final pass that applies the HDR tonemapping when going from the internal (sRGB/linear) render target to the backbuffer is the right way to do it but is fairly involved
  • The issues I had with the postprocess shader not looking right were resolved by changing my HDR mode from Peak 1000 to HDR 400
• PDA can seem too bright without the UI correction, might need to use a scalar to darken the PDA in sRGB/linear when rendering its UI
  • Also solved by using certified HDR 400 mode

[LVutner]

It does not make sense to me.

You did not removed tonemapping curves from other shaders.
So in fact, you store LDR values in FP16 rendertargets (post combine1, sky, translucents), then you tonemap it again (combine2).

[deggua]

It does not make sense to me.

You did not removed tonemapping curves from other shaders. So in fact, you store LDR values in FP16 rendertargets (post combine1, sky, translucents), then you tonemap it again (combine2).

It's not color or gamma correct in quite a few places then. I'll look at fixing the ones you mentioned and see if I can find any other shaders with issues. Anything tonemapped from HDR won't look right for sure and is probably why some things look blown out or too dark (imo sky is too bright currently and translucents are too dark).

[rruss263]

Forgive my large amount of ignorance, i've actually done very little work in the code side of things, but considering that a lot of emphasis is placed on HDR in this conversation, I wanted to posit that the HBAO, HDAO, and SSAO might be better replaced with a type of GI with Path/Raytracing if at all possible, allowing for a better color palette and more authentic lighting. considering that, would it be possible to add a D3D11on12 API, then add Path tracing in that manner? Or use DXVX to pull the Vulkan API and ray trace from there? I know that SSRTGI is basically created from implied depth, but integrated PT/RT uses both screen space and the actual engine positions and so on to ray trace. If you had access to DX12 or Vulkan HDR10 should also be easier to implement overall due to the way that the API handles it.

https://github.com/microsoft/D3D11On12
https://github.com/NVIDIAGameWorks/RTX-Path-Tracing
https://learn.microsoft.com/en-us/windows/win32/direct3d12/porting-from-direct3d-11-to-direct3d-12
https://github.com/microsoft/DirectXTK12/wiki/Using-HDR-rendering

[deggua]

It does not make sense to me.

You did not removed tonemapping curves from other shaders. So in fact, you store LDR values in FP16 rendertargets (post combine1, sky, translucents), then you tonemap it again (combine2).

Is it as simple as modifying the tonemap function in common_functions.h to not perform tonemapping? I did that, and it doesn't look wrong, not sure if it's right either though. I'm confused about what the high output arg is from that function, is it used for bloom? Not scaling the input by 1/def_hdr for high results in a very dark image. I'm guessing the output of tonemap should be gamma corrected, looking at the tonemap function in KennShade's Enhanced Shaders mod, it looks like the input is sRGB and the output for low is sRGB, so I'm still applying inverse gamma correction in sRGBToHDR10 since the input image in combine_2 is (I believe) in sRGB (which must be why r__gamma == 1 looks correct for me in SDR). There are other places in the code where tonemap shows up that I'm less certain of, e.g. some parameters to shaders say the tonemapping is preapplied to the color, s_tonemap, some vertex colors, etc. There's also r2_tnmp_* but that only seems to be applied to combine_2 and is off if r2_tnmp_onoff == 0.0, though I should probably force disable that in combine_2 if HDR10 is enabled.

Lens flares and the sun (even with lens flares off) still look really blown out, I'll see if I can figure out what's happening there.
I think ^ and some other issues I've noticed were a result of my monitor configuration. The peak 1000 nits mode has some issues. The HDR 400 mode doesn't seem to have the sky blowout issue (although lens flares are still huge, but I need to compare to SDR) or some of the darkening/weird issues with postprocess.ps effects.

I believe the above change to the tonemap function also fixed the issue I mentioned about SSS's Force Volumetric setting blowing out the flashlight's brightness. I compared the SDR lens flares to HDR and they're definitely more intense, probably needs tuning.

[deggua]

I updated the shaders associated with this PR. These should remove SDR tonemapping when HDR is enabled (which should not be applied as LVutner pointed out), they also attenuate the sun and lens flare effects since those seemed too bright, and I found a better way to apply HDR tonemapping to the HUD that should avoid applying HDR tonemapping to other effects that use the stub_default pixel shader.

[themrdemonized]

I'll check it out later, thanks. Let me know if your changes are final and ready to go

[deggua]

I'll check it out later, thanks. Let me know if your changes are final and ready to go

There's some cleanup to be done in dx10HW.cpp, I think I'm missing a _RELEASE/->Release in a few places where QueryInterface is called, I also need to test the single threaded flag (I assume the renderer is single threaded so it should remove a little CPU overhead). I need to add the HDR toggle and sliders for the nits options to the Modded Exes menu, ideally in a way that doesn't apply HDR until restart and that notifies the user that a restart is required (I looked at some other PRs so I think I understand how to add options now). Should I commit the vanilla Anomaly shaders to this repo's gamedata as well? Without them the HDR won't display correctly.

[themrdemonized]

yea, you should put all shader files that you changed and are requried into gamedata/gamedata/shaders/r3.

[deggua]

yea, you should put all shader files that you changed and are requried into gamedata/gamedata/shaders/r3.

I think everything is ready. My editor or my git config might have messed up the text encoding on some of the files with Cyrillic characters, so you might want to check that. I think I fixed it for rus/ui_mm_modded_exes.xml (the HDR strings are machine translations, feel free to fix those if they're bad) but some of the comments in .cpp/.h files might be messed up still.

I also added a colorspace option since most HDR games let you choose that.

[themrdemonized]

Is that how it is supposed to look like? I have OLED display with HDR support, supposedly with 1000 nit brightness, so i set whitepoint_nits to 1000 in the console. I enabled HDR in Windows setting.
I took phone photo cause i cant do printscreen, it looks very bad

[themrdemonized]

Alright, i was running shaders mod before testing, on an absolute clean install it looks much better. I lowered nits to 600.

So it raises a mod compatibility problem. The previous screenshot had i think Beef's NVG enabled, and it overwrites shader files from modded exes install. I guess someone has to update the mods to have correct HDR support. I can issue a warning for that case.

[deggua]

Is that how it is supposed to look like? I have OLED display with HDR support, supposedly with 1000 nit brightness, so i set whitepoint_nits to 1000 in the console. I enabled HDR in Windows setting. I took phone photo cause i cant do printscreen, it looks very bad

I have an OLED as well, the 1000 nits mode w/ r4_hdr_whitepoint_nits = 1000 looked very blown out (sky especially), if it has a HDR 400 mode try that w/ r4_hdr_whitepoint_ntis = 400. In the 1000 nits mode I had to lower the whitepoint to 800 before it looked reasonable, but there were still weird problems with auto-brightness limiting darkening the whole screen. Also try changing the color space to DCI-P3 or Rec.2020 if it's too saturated. There is some shader incompatibility, I put the patch required for GAMMA (SSS + Enhanced Shaders + Beef's NVGs) here: https://github.com/deggua/xray-hdr10-shaders

I've been playing GAMMA for the last few days with it on and it looks good imo, although a little dark by default, but tweaking gamma and brightness fixed that.

Also, if you want to take HDR screenshots, the best way (if you have nvidia at least) I've found is to use the Nvidia overlay and screenshot with Alt+F1. This will save it as JPEG XR in HDR, viewing JPEG XR files in HDR requires an HDR image viewer, this one works for me: https://13thsymphony.github.io/hdrimageviewer/

[sundaylee91]

ever since your HDR10 update to modded exes, all macOS and linux users are unable to play anymore....linux players can turn off some mods like SSS to continue play as I heard, macOS is totally unable to play

[themrdemonized]

ever since your HDR10 update to modded exes, all macOS and linux users are unable to play anymore....linux players can turn off some mods like SSS to continue play as I heard, macOS is totally unable to play

Please read the latest message there #95