How to get the best stream quality with an NVIDIA GPU (NVENC)
Having toyed around in the streaming world for 10+ years and always trying to maximize stream quality given the resolution, frame rate, and bit rate restrictions, I’m happy to say that I’ve arrived at settings that will be powering my stream from now on.
NVIDIA released the NVENC SDK version 10 some time ago and with it came new NVENC tuning options, allowing us to squeeze every inch of quality out of the NVENC encoder. You can read more about the changes here – https://developer.nvidia.com/blog/introducing-video-codec-sdk-10-presets/
Note: Starting with OBS Studio 28.1 Beta, the improved NVIDIA encoding settings are built into OBS Studio. The guide below is no longer required.
The trick to getting the best quality is by replacing the FFMpeg version that OBS uses. As of OBS Studio 27.1.3 an older version of FFMpeg is used that is not capable of using the new NVIDIA SDK changes, so we need to manually update it.
- OBS Studio 27.1.3
- StreamFX 11.0
- NVIDIA RTX 20-series GPU or newer with the Turing encoder (7th gen)
- More details: https://developer.nvidia.com/video-encode-and-decode-gpu-support-matrix-new
Set up steps
Install OBS Studio 64bit
Once OBS is installed, install StreamFX – https://streamfx.xaymar.com/
Keep OBS Studio closed
Locate where OBS Studio is installed, which by default is
Simply right-click on the OBS Studio shortcut and select “open file location”
bin and then
Look for a file called avcodec-58.dll. Is it there? Good.
Download FFmpeg version 4.4 shared GPLv3 from BtbN. The file should be called
Extract the zip file.
Go into the folder and copy all the .dll files.
Then, go back to the OBS Studio 64bit folder and paste the new FFMpeg files. You will be asked if you want replace the existing files – Make sure to select yes!
Now open OBS Studio and open the settings
Click on the Output settings and set the mode to advanced
From the encoder dropdown select NVENC with FFmpeg
Now, here are my recommended settings. These are mostly pulled from Xaymar’s excellent encoding guide found here.
- Preset: High Quality
- Profile: High
- Level: Automatic
- Mode: Constant bitrate
- Two Pass: Enabled
- Look Ahead: 32 frames
- Adaptive I-frames: Enabled
- Adaptive B-frames: enabled
- Target bit rate: Depends on your resolution and frame rate. Refer to Xaymat’s high quality streaming guide here.
- Buffer size: Needs to be 2x the target bit rate
- Minimum QP: -1
- Maxikmum QP: -1
- Spatial Adative Quantization: Enabled
- Spatial Adaptive Quantization Strength: -1 (The default is 15)
- Temporal Adaptive Quantization: Enabled
- Maximum B-frames: 4
- B-frame reference mode: Use only Middle B-frames as reference
- Zero Latency: Off
- Weighted prediction: off
- Non-reference P-frames: Enabled
- Reference Frames: -1
Now here is where we get into what makes the magic happen: Custom command line parameters. Enter these parameters:
-preset=p7 -tune=hq -multipass=2
That’s it. Now you are streaming with the absolute best quality NVENC has to offer.