Support cudnn9 for cuda 12

[Jian Chen (jchen351)]

Description

Adding Cudnn9 for cuda 12

Motivation and Context

Enable CuDnn for Cuda 12 on 1.18.1 release

[snnn (snnn)]

Did you copy cudnn files and zlib.dll to this folder?

[Jian Chen (jchen351)]

I copied zlibwapi.dll from v12.2/bin to v12.3/bin. Cudnn files are in both v12.2 and v12.3's bin, lib and include folders

[snnn (snnn)]

Good. Thank you!

[snnn (snnn)]

Please note this should be considered as a breaking change as the newly built packages will require CUDNN 9 at run time instead of CUDNN 8. It will impact how our customers do their deployments. We must document it well in our release notes.

[Jian Chen (jchen351)]

Please note this should be considered as a breaking change as the newly built packages will require CUDNN 9 at run time instead of CUDNN 8. It will impact how our customers do their deployments. We must document it well in our release notes.

Sophie Schoenmeyer (@sophies927) FYI

[George Wu (jywu-mysoft)]

Please note this should be considered as a breaking change as the newly built packages will require CUDNN 9 at run time instead of CUDNN 8. It will impact how our customers do their deployments. We must document it well in our release notes.

Agreed. Let's document it.
but people who are on CUDA 12.x will be using CuDNN 9.x, and not 8.
9.1.0 was already released 2 months ago.
and the last version of CuDNN 8.x was published last year.

[Sophie Schoenmeyer (sophies927)]

Please note this should be considered as a breaking change as the newly built packages will require CUDNN 9 at run time instead of CUDNN 8. It will impact how our customers do their deployments. We must document it well in our release notes.

Agreed. Let's document it. but people who are on CUDA 12.x will be using CuDNN 9.x, and not 8. 9.1.0 was already released 2 months ago. and the last version of CuDNN 8.x was published last year.

I will be sure to include in the 1.18.1 patch release notes.

[snnn (snnn)]

And we need to update the doc in PR #20914 once this package is published via a release.

[George Wu (jywu-mysoft)]

https://aiinfra.visualstudio.com/Lotus/_build/results?buildId=478755&view=logs&j=d46bfb7e-8a17-564a-8f0d-20f6340ae35e&t=71906b08-1304-567a-61c5-75eca5271cb8&l=13
seems like the gpu driver cannot support cuda 12.3 ? need to update it?

[Jian Chen (jchen351)]

https://aiinfra.visualstudio.com/Lotus/_build/results?buildId=478755&view=logs&j=d46bfb7e-8a17-564a-8f0d-20f6340ae35e&t=71906b08-1304-567a-61c5-75eca5271cb8&l=13 seems like the gpu driver cannot support cuda 12.3 ? need to update it?

I am working on it.=

[Jian Chen (jchen351)]

Current issue is the grid drivers provided by Azure has cuda 12.2 with nVidia driver version 535.154.05. We need to find a way to install the cuda 12.3 driver that support A-10 machines from Azure.

[Jian Chen (jchen351)]

+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.154.05 Driver Version: 535.154.05 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA A10-4Q On | 00000002:00:00.0 Off | 0 |
| N/A N/A P8 N/A / N/A | 0MiB / 4096MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+----------------------+----------------------+

[Jian Chen (jchen351)]

Using #20925 instead of this PR.

[George Wu (jywu-mysoft)]

Using #20925 instead of this PR.

you'll add windows related changes to the other one?
or that one can be merged first and then this one later?