Support Of Nvidia RTX 50 Series / 5090 – Win11

Hello All

I’ve been here since November 2024 and I started on a RTX 2070 super with 8gb VRAM, so I was well (over) due for an upgrade, I had issues with the GPU in January 2025. As the GPU market is a little crazy and new stock are low because of Nvidia’s appalling 50 series launch, I bought the cheapest GPU possible the RTX 4060 8gb VRAM (whoo..9% performance boost over the 2070 super) I least I had a GPU that worked.

As my AI interest grew I knew I needed a better GPU with more VRAM, so to cut a long story short I now have a RTX 5090. I was warned by Andrew prior to having the 5090 that the 50 series GPU’s are having compatibility issues with AI software and add-ons, due to driver issues and these getting updated in various  applications and add-ons.  I used the logic: Well I’ve waited this long for a better GPU with higher VRAM, that I can wait a little long for everything else to  catchup, when it does I’ll be ready.

After searching on this form I could see nothing that related to information on the RTX 50 Series and issues around them, and any solutions or applications that currently don’t work because of driver issues and updates. That is the reason I started this thread to try and show some light on the RTX 50 series what can be done and what can’t be done with the GPU, and as things change what starts working.

I use ComfyUI desktop  (Dot Exe version) and Window 11 and every thing installed  around that installation to allow ComfyUI to function. Latest Nvidia drivers, Cuda Toolkit,  Python, Pytorch and Github.

Next I had Google’s Gemini help, mainly to stop me waffling and typing, so I hope this is useful.

I asked Gemini this:

Can you list me all the things that need to be done on a Windows 11 using a RTX 50 series GPU like my 5090, that enables the GPU to function properly. For AI tasks like LLM’s and Ollama openwebUI, AI generative tasks like A1111, Forge, Invoke, ComfyUI and nodes, plus other apps that use AI for image and video generation. To make these work or don’t work and what are the latest drivers and app versions like Python, Pytorch, Cuda etc.

The Output:

>>>> Start <<<<

Essential Foundations for RTX 50 Series (and Newer NVIDIA GPUs) on Windows 11
The RTX 50 series, being a new architecture, will rely heavily on the very latest software to fully leverage its capabilities. The core challenge often revolves around CUDA compute capability support within the various AI frameworks.

I. Core System & Driver Setup

Windows 11:

Keep it Updated: Ensure your Windows 11 installation is fully up to date with the latest security patches and feature updates. This often includes crucial system-level optimisations and bug fixes.
NVIDIA GPU Drivers:

Always the Latest: This is paramount. For an RTX 5090, you must use the very latest Game Ready Driver or Studio Driver directly from NVIDIA’s official website. These drivers contain the necessary low-level support for new GPU architectures and their CUDA versions.
Clean Installation: If you encounter persistent issues, consider performing a clean installation of the drivers using a tool like Display Driver Uninstaller (DDU) in Safe Mode, followed by a fresh install of the latest NVIDIA drivers.
Driver Version Implies CUDA Support: The NVIDIA driver version dictates the maximum CUDA Toolkit version you can use. Newer drivers inherently support newer CUDA versions. For your 5090, the latest drivers will support CUDA 12.x.
CUDA Toolkit:

Purpose: This is NVIDIA’s parallel computing platform and programming model that allows software to use the GPU for general-purpose processing. AI frameworks rely on this.
Version for RTX 50 Series: For the RTX 50 series (and generally the Ada Lovelace architecture of RTX 40 series onwards), you absolutely must use CUDA Toolkit version 12.x or newer. Older CUDA 11.x toolkits will not properly support the compute capabilities of these newer GPUs. Always download the latest stable CUDA Toolkit directly from the NVIDIA Developer website.
Installation: Follow the official NVIDIA installation guide carefully. Ensure it integrates correctly with your system’s PATH environment variables.
cuDNN:

Purpose: CUDA Deep Neural Network library. This is a GPU-accelerated library of primitives for deep learning. It’s essential for performance in AI tasks.
Version Compatibility: You must download a cuDNN version that is compatible with your installed CUDA Toolkit version.
Installation: After downloading from NVIDIA (requires a developer account), you typically copy the cuDNN files into the corresponding directories within your CUDA Toolkit installation.

II. Core AI Frameworks & Libraries

Python:

Recommended Versions: Python 3.10.x, 3.11.x, or 3.12.x are generally the most widely supported by AI applications. Some applications might specifically recommend one over the others, so always check their documentation. Python 3.10 is a common sweet spot, while 3.11 and 3.12 offer performance improvements.
Virtual Environments (Crucial): Always, always, always use virtual environments (venv is built-in, conda is also popular). This prevents conflicts between the Python packages required by different AI applications. For example:PowerShell

python -m venv my_ai_env
.\my_ai_env\Scripts\activate # On Windows PowerShell

<c/ode>

Then install all packages within this active environment.

PyTorch:

The Most Critical Compatibility Point: This is where many users with new NVIDIA GPUs face issues. PyTorch is the predominant deep learning framework for generative AI applications.
RTX 50 Series Requirement: Stable PyTorch releases often lag behind new hardware support. For the RTX 5090 (and its underlying compute capability), you will likely need to use a PyTorch nightly build specifically compiled for CUDA 12.8 (or the very latest CUDA 12.x version).
Installation Command Example (for cu128):PowerShell

<

pre>pip install --no-cache-dir --upgrade torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128

Always check the official PyTorch website (pytorch.org/get-started/locally/) and select "Nightly" and the appropriate CUDA version (cu12x) for the most up-to-date command.
xformers:

Purpose: A highly optimised library for attention mechanisms, crucial for speed and VRAM efficiency in many generative AI models (like Stable Diffusion).
Compatibility: Must be built for your specific PyTorch and CUDA versions. Often, specific versions like 0.0.26.post1 are recommended, sometimes installed directly from source or specific wheels.
bitsandbytes:

Purpose: Essential for running large language models (LLMs) efficiently by enabling 4-bit or 8-bit quantization, significantly reducing VRAM usage.
Compatibility: Also requires a compatible CUDA version.
Hugging Face transformers and accelerate:

Purpose: Core libraries for working with pre-trained models, especially LLMs. accelerate helps distribute computations across multiple GPUs or use techniques like mixed-precision.
Installation: Typically via pip.
III. Application-Specific Notes (A1111, ComfyUI, Ollama, etc.)

Most popular AI applications (A1111, Forge, InvokeAI, ComfyUI) often come with their own installation scripts that attempt to set up a Python virtual environment and install dependencies.

Override Defaults (if necessary): For new GPUs, you might need to manually edit their setup scripts (.bat files on Windows) to force the use of the correct PyTorch nightly build or to pass specific arguments (e.g., --xformers, or flags related to new compute capabilities if the application supports them).
A1111/Forge/InvokeAI/ComfyUI: These are generally PyTorch-based. Ensure their requirements.txt (or similar) are satisfied and that your base PyTorch setup is correct. They will heavily benefit from xformers.
Ollama: This is a self-contained application for running LLMs locally. It often comes with its own bundled CUDA dependencies but will still benefit from having the latest NVIDIA GPU drivers installed on your system.
Open WebUI: This is a front-end that interacts with Ollama or other LLM backends. Its dependencies are mainly Python-based and less about direct GPU interaction; it relies on the backend to handle the GPU.
Other Image/Video Generation Apps: The principles remain the same: ensure your drivers, CUDA Toolkit, and the AI framework (usually PyTorch) are all up-to-date and compatible with your 5090.
Common Pitfalls & What Makes Them "Not Work"
Driver/CUDA/PyTorch Mismatch: This is the #1 cause of issues. If your PyTorch isn't compiled for your CUDA Toolkit version, or your CUDA Toolkit isn't recent enough for your 5090, things will fail with errors like "no GPU found", "CUDA out of memory", or "compute capability mismatch".
Outdated Software: Relying on older stable releases of PyTorch, xformers, or other libraries that predate the RTX 50 series.
sm_120 (or similar new architecture) Warnings/Errors: If you see messages referencing sm_120 or similar high compute capabilities, it's a strong indicator that your PyTorch (or another CUDA-dependent library) is too old to fully recognise or utilise your new GPU. The PyTorch nightly cu128 build (or similar) is the solution.
Installation Conflicts: Installing packages directly into your system-wide Python or mixing pip and conda environments. Always use virtual environments.
Firewall/Antivirus: Can block model downloads or local web UI access. Ensure your firewall allows the necessary connections (e.g., port 7860 for Gradio UIs).
Insufficient Disk Space: AI models are massive. Ensure you have hundreds of gigabytes (or even terabytes) of free space for models, outputs, and intermediate files.
VRAM Management (even with 5090): While the 5090 has a massive VRAM capacity, extremely large models, very high resolutions, or complex workflows can still lead to "CUDA out of memory" errors. Quantisation (e.g., 4-bit with bitsandbytes) and optimisations (xformers) are key.
Latest Versions (General Guidance, as of June 2025)
NVIDIA Drivers: Always download the absolute latest Game Ready or Studio Driver package directly from NVIDIA's official website.
CUDA Toolkit: The latest stable 12.x version (e.g., CUDA 12.5, 12.6, 12.7, 12.8). Check NVIDIA's developer site.
PyTorch:For RTX 50 Series, you almost certainly need the PyTorch Nightly build compiled for the latest CUDA 12.x (e.g., cu128 for CUDA 12.8). Do not rely on pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121 as that's for older stable CUDA.
Python: Generally, Python 3.10.x, 3.11.x, or 3.12.x. Check specific application requirements.
xformers: Always try to get the latest version compatible with your PyTorch and CUDA setup. Often installed directly from specific wheel files or GitHub.
bitsandbytes: Latest version available via pip.
Applications (A1111, ComfyUI, etc.): Regularly perform git pull on their repositories to get the latest application-level updates and bug fixes.
By focusing on keeping your NVIDIA drivers, CUDA Toolkit, and crucially, your PyTorch installation at the bleeding edge (specifically the nightly builds for CUDA 12.x compatibility), you'll be well-prepared to make the most of your RTX 5090 for AI tasks.

Sources:

Certainly, here are the primary sources for the information provided regarding setting up your Windows 11 system with an RTX 50 series GPU for AI tasks:

NVIDIA Official Resources:

NVIDIA Driver Downloads: For the latest Game Ready or Studio Drivers. You would visit the official NVIDIA website and navigate to their driver download section:NVIDIA Driver Downloads
NVIDIA CUDA Toolkit: For downloading the essential CUDA Toolkit that enables GPU computing. You'll find it on the NVIDIA Developer website: NVIDIA CUDA Toolkit Downloads
NVIDIA cuDNN: For the Deep Neural Network library, which requires an NVIDIA Developer Program account to download: NVIDIA cuDNN Downloads
NVIDIA Developer Documentation: For detailed information on GPU architectures, compute capabilities, and best practices.
PyTorch Official Website:

This is the definitive source for PyTorch installation instructions, including stable and nightly builds for various CUDA versions: PyTorch Get Started Locally (Ensure you select "Nightly" and the appropriate CUDA version for your 5090, e.g., cu128).
Python Official Website:

For downloading the Python interpreter: Python Downloads
Hugging Face Documentation:

For information on libraries like transformers and accelerate, which are widely used for LLMs and other large models: Hugging Face Transformers Documentation
Hugging Face Accelerate Documentation
Specific AI Application GitHub Repositories/Documentation:

For applications like Automatic1111, ComfyUI, InvokeAI, Forge, Ollama, and Open WebUI, their respective GitHub repositories or official documentation pages are the best places for detailed installation steps, specific Python/dependency versions, and troubleshooting unique to each application. You would typically find these by searching for the application name on GitHub.
PyPI (Python Package Index):

While you often install xformers, bitsandbytes, etc., as part of an application's requirements or from specific wheel files, PyPI is the central repository for most Python packages:PyPI (pypi.org) (You can search for specific packages here).
These sources provide the official, most up-to-date, and authoritative information for configuring your system for the best AI performance.

>>>> End <<<<

As this post is so long I'll reply to this for issues that don't work, fixed or updates for the 50 series cards, as it will be a shame if I didn't share, and to help others in the same situation. Hope this thread / post helps people.

All the best