ENH: Remove ITKCudaCommon from the source tree by AlexyPellegrini · Pull Request #492 · RTKConsortium/RTK

AlexyPellegrini · 2022-05-30T12:57:23Z

Now builds with the remote module CudaCommon of ITK. Builds both as a remote module of ITK and as a standalone library.
This MR replaces https://github.com/SimonRit/RTK/pull/427

AlexyPellegrini · 2022-05-30T13:24:38Z

@LucasGandel

SimonRit · 2022-05-31T08:26:55Z

Thanks @AlexyPellegrini! I think that the CI must be updated to account for the ITKCudaCommon dependency. See this comment to know how to do it:
InsightSoftwareConsortium/ITKModuleTemplate#123 (comment)

SimonRit · 2022-06-03T13:52:50Z

FYI, my plan is to finalize SimonRit/ITKCudaCommon#15. I'll update then #495. I will ask you then to rebase your PR on it... It's taking longer than expected because I have uncovered some issues by setting up the CI, sorry for the delay.

SimonRit · 2022-06-08T15:34:04Z

        cd ITK-build
        call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvars64.bat"
-        cmake -DCMAKE_C_COMPILER:FILEPATH="${{ matrix.c-compiler }}" -DBUILD_SHARED_LIBS:BOOL=ON -DCMAKE_CXX_COMPILER="${{ matrix.cxx-compiler }}" -DCMAKE_BUILD_TYPE:STRING=${{ matrix.cmake-build-type }} -DBUILD_TESTING:BOOL=OFF -GNinja ../ITK
+        cmake -DCMAKE_C_COMPILER:FILEPATH="${{ matrix.c-compiler }}" -DBUILD_SHARED_LIBS:BOOL=ON -DCMAKE_CXX_COMPILER="${{ matrix.cxx-compiler }}" -DCMAKE_BUILD_TYPE:STRING=${{ matrix.cmake-build-type }} -DModule_CudaCommon:BOOL=ON -DBUILD_TESTING:BOOL=OFF -GNinja ../ITK


There seems to be a problem wtith itkCudaSquareImage.cu see the log. Maybe you should try to compile the master version of ITKCudaCommon by setting Module_CudaCommon_GIT_TAG here?

LucasGandel · 2022-06-28T11:29:59Z

@SimonRit The windows cuda python packages seems to work well. However there is an issue with docker for the linux cuda python packages (see here). Do you have any clue about what is happening with the linux self-hosted runner?

SimonRit · 2022-07-04T05:36:13Z

@SimonRit The windows cuda python packages seems to work well. However there is an issue with docker for the linux cuda python packages (see here). Do you have any clue about what is happening with the linux self-hosted runner?

Sorry, the docker service was no longer running. It should be fixed now, I have scheduled a new run or all jobs. However, shouldn't this trigger a job failure? I'm not sure to understand why it doesn't.

SimonRit · 2022-07-05T09:55:56Z

The compilation is still struggling due to missing CudaCommonExport.h, see here. This file should be generated when compiling CudaCommon, any clue why it can't find it? I'm not sure I understand this manual operation but for sure CudaCommonExport.h is not there but in the binary directory.

AlexyPellegrini · 2022-07-06T11:32:09Z

To be honest, we copied it from ITKUltrasound's source code, all remote module are build this way with this copy of their include at the end. Maybe we need to also copy that generated config header to that include directory to make it accessible, or maybe it is not the good way of doing this and we should find a way to build it without any copy.
Can you check the path of the directory that contains the config header ?

SimonRit · 2022-07-07T14:25:45Z

I can't find the file... I can find ./_work/RTK/RTK/_skbuild/linux-x86_64-3.7/cmake-build/include/RTKExport.h but _work/RTK/RTK/ITKCudaCommon/_skbuild is empty. I'm lost...

SimonRit · 2022-07-07T14:29:55Z

As far as I can see, the remote modules BSplineGradient, HigherOrderAccurateGradient, SplitComponents, Strain, from which Ultrasound depends, do not need this generated file (grep Export in their source repo was empty).

AlexyPellegrini · 2022-07-07T15:31:04Z

I found a typo in what I wrote, and it was one of the include dir copy so I could make sense, can you try to launch build-linux-cuda-python-packages (37) once again ?

SimonRit · 2022-07-12T10:05:14Z

FYI, the problem with CudaCommonExport.h is still here. During the build, the file is in ./_work/RTK/RTK/ITKCudaCommon/_skbuild/linux-x86_64-3.7/cmake-build/include/CudaCommonExport.h but it seems to be cleaned automatically. I guess it should be installed here and I'm not sure why it is not?

SimonRit · 2022-08-17T21:53:06Z

I'd like to quickly do a release now that #507 is merged (see InsightSoftwareConsortium/ITK#3528). Do you think we should finalize this PR before? What is its status?

LucasGandel · 2022-08-18T06:35:55Z

I'd like to quickly do a release now that #507 is merged (see InsightSoftwareConsortium/ITK#3528). Do you think we should finalize this PR before? What is its status?

Yes I will rebase this on master and it should be ready

LucasGandel · 2022-08-26T07:37:08Z

@SimonRit This should finally pass once InsightSoftwareConsortium/ITKPythonPackage#200 is merged and new ITKPythonBuilds are released

SimonRit · 2022-09-07T12:32:58Z

@SimonRit This should finally pass once InsightSoftwareConsortium/ITKPythonPackage#200 is merged and new ITKPythonBuilds are released

Following @thewtex' comment, I have relaunched the checks. But it will likely fail for manylinux2014...

SimonRit · 2022-10-12T07:23:33Z

Can you rebase on master to retrigger the tests based on #510?

LucasGandel · 2022-10-12T12:54:04Z

Can you rebase on master to retrigger the tests based on #510?

Done! Thanks for the update!

Please make sure to use "squash and merge" button if everything goes well.

LucasGandel · 2022-10-18T06:54:43Z

@SimonRit It looks like the pipelines are passing 🥳 🎉 We should test the produced wheels and merge if everything is fine

SimonRit · 2022-11-21T07:22:51Z

Can you please rebase on master? I have updated the CI infrastructure for compability with the latest ITK builds, see 7cda37a.

SimonRit · 2022-11-24T07:48:45Z

I have tested the Linux 3.10 CI package (from ITKCudaCommon and this PR) and it works! There are a bunch of warnings at execution to address, see below. I also think that there is a missing install requirement for itk-cudacommon-cuda116 in setup.py?

(rtk_cudacommon_split) srit@linux-b3vn:~/src/rtk/rtk/examples/FirstReconstruction> LD_LIBRARY_PATH=/home/srit/Downloads/cuda116/lib64:$LD_LIBRARY_PATH python ~/src/rtk/rtk/examples/FirstReconstruction/FirstCudaReconstruction.py a.mha a.xml
Template itk::ImageSource<itk::CudaImage<float,2>>
 already defined as <class 'itk.itkImageSourceCudaCommonPython.itkImageSourceCIF2'>
 is redefined as {cl}
Warning: template already defined 'itk::ImageSource<itk::CudaImage<float,2>>'
Template itk::ImageSource<itk::CudaImage<float,3>>
 already defined as <class 'itk.itkImageSourceCudaCommonPython.itkImageSourceCIF3'>
 is redefined as {cl}
Warning: template already defined 'itk::ImageSource<itk::CudaImage<float,3>>'
Template itk::ImageSource<itk::CudaImage<float,4>>
 already defined as <class 'itk.itkImageSourceCudaCommonPython.itkImageSourceCIF4'>
 is redefined as {cl}
Warning: template already defined 'itk::ImageSource<itk::CudaImage<float,4>>'
Template itk::ImageSource<itk::CudaImage<itk::Vector<float,2>,2>>
 already defined as <class 'itk.itkImageSourceCudaCommonPython.itkImageSourceCIVF22'>
 is redefined as {cl}
Warning: template already defined 'itk::ImageSource<itk::CudaImage<itk::Vector<float,2>,2>>'
Template itk::ImageSource<itk::CudaImage<itk::Vector<float,2>,3>>
 already defined as <class 'itk.itkImageSourceCudaCommonPython.itkImageSourceCIVF23'>
 is redefined as {cl}
Warning: template already defined 'itk::ImageSource<itk::CudaImage<itk::Vector<float,2>,3>>'
Template itk::ImageSource<itk::CudaImage<itk::Vector<float,2>,4>>
 already defined as <class 'itk.itkImageSourceCudaCommonPython.itkImageSourceCIVF24'>
 is redefined as {cl}
Warning: template already defined 'itk::ImageSource<itk::CudaImage<itk::Vector<float,2>,4>>'
Template itk::ImageSource<itk::CudaImage<itk::Vector<float,3>,2>>
 already defined as <class 'itk.itkImageSourceCudaCommonPython.itkImageSourceCIVF32'>
 is redefined as {cl}
Warning: template already defined 'itk::ImageSource<itk::CudaImage<itk::Vector<float,3>,2>>'
Template itk::ImageSource<itk::CudaImage<itk::Vector<float,3>,3>>
 already defined as <class 'itk.itkImageSourceCudaCommonPython.itkImageSourceCIVF33'>
 is redefined as {cl}
Warning: template already defined 'itk::ImageSource<itk::CudaImage<itk::Vector<float,3>,3>>'
Template itk::ImageSource<itk::CudaImage<itk::Vector<float,3>,4>>
 already defined as <class 'itk.itkImageSourceCudaCommonPython.itkImageSourceCIVF34'>
 is redefined as {cl}
Warning: template already defined 'itk::ImageSource<itk::CudaImage<itk::Vector<float,3>,4>>'
Template itk::ImageSource<itk::CudaImage<itk::Vector<float,4>,2>>
 already defined as <class 'itk.itkImageSourceCudaCommonPython.itkImageSourceCIVF42'>
 is redefined as {cl}
Warning: template already defined 'itk::ImageSource<itk::CudaImage<itk::Vector<float,4>,2>>'
Template itk::ImageSource<itk::CudaImage<itk::Vector<float,4>,3>>
 already defined as <class 'itk.itkImageSourceCudaCommonPython.itkImageSourceCIVF43'>
 is redefined as {cl}
Warning: template already defined 'itk::ImageSource<itk::CudaImage<itk::Vector<float,4>,3>>'
Template itk::ImageSource<itk::CudaImage<itk::Vector<float,4>,4>>
 already defined as <class 'itk.itkImageSourceCudaCommonPython.itkImageSourceCIVF44'>
 is redefined as {cl}
Warning: template already defined 'itk::ImageSource<itk::CudaImage<itk::Vector<float,4>,4>>'
Warning: Unknown parameter 'itk::CudaImage<itk::Vector<float,5>,4>' in template 'itk::ImageSource'
Warning: Unknown parameter 'itk::CudaImage<itk::Vector<float,5>,4>' in template 'itk::ImageToImageFilter'
Warning: Unknown parameter 'itk::CudaImage<itk::Vector<float,5>,4>' in template 'itk::CudaImageToImageFilter'
Warning: Unknown parameter 'itk::CudaImage<itk::Vector<float,5>,4>' in template 'itk::InPlaceImageFilter'
Warning: template already defined 'itk::InPlaceImageFilter<itk::CudaImage<itk::Vector<float,5>,4>,itk::CudaImage<itk::Vector<float,5>,4>>'
Warning: Unknown parameter 'itk::CudaImage<itk::Vector<float,5>,4>' in template 'itk::CudaInPlaceImageFilter'
Reconstructing...
Writing output image...
Done!

Now builds with the remote module CudaCommon of ITK. Builds both as a remote module of ITK and as a standalone library.

SimonRit · 2023-03-14T08:29:11Z

I have been working on this PR and inadvertently modified @AlexyPellegrini's branch, sorry. The reference PRs are now RTKConsortium/ITKCudaCommon#20 and #528.

AlexyPellegrini force-pushed the external-cuda-common branch 2 times, most recently from 522aed4 to 58efe1d Compare June 3, 2022 07:22

LucasGandel self-requested a review June 3, 2022 07:40

LucasGandel mentioned this pull request Jun 3, 2022

Add self-hosted CI for Windows python package with Cuda #495

Merged

AlexyPellegrini force-pushed the external-cuda-common branch from cacea8a to 8077364 Compare June 3, 2022 15:01

LucasGandel reviewed Jun 3, 2022

View reviewed changes

Comment thread .github/workflows/build-test-package.yml Outdated

LucasGandel reviewed Jun 3, 2022

View reviewed changes

Comment thread .github/workflows/build-test-package.yml Outdated

LucasGandel reviewed Jun 3, 2022

View reviewed changes

Comment thread .github/workflows/build-test-package.yml Outdated

SimonRit reviewed Jun 8, 2022

View reviewed changes

AlexyPellegrini force-pushed the external-cuda-common branch from db039dd to d559d31 Compare June 16, 2022 12:30

LucasGandel force-pushed the external-cuda-common branch from 4a8351c to da0d1f7 Compare August 18, 2022 07:01

SimonRit mentioned this pull request Aug 18, 2022

Turn ITKCudaCommon into a remote module #427

Closed

1 task

LucasGandel force-pushed the external-cuda-common branch from 54d39a9 to 195888b Compare August 18, 2022 08:31

LucasGandel force-pushed the external-cuda-common branch from 195888b to dcd19c0 Compare August 26, 2022 07:35

LucasGandel force-pushed the external-cuda-common branch from dcd19c0 to 1ffe8bb Compare October 12, 2022 12:51

LucasGandel force-pushed the external-cuda-common branch from cbef5d7 to b37c5e5 Compare November 23, 2022 11:11

SimonRit mentioned this pull request Nov 29, 2022

Investigate accommodating ITK remote module dependencies InsightSoftwareConsortium/ITKRemoteModuleBuildTestPackageAction#11

Closed

SimonRit mentioned this pull request Jan 6, 2023

Pass option to setup.py and cmake on Windows, clean-up and avoid unnecessary downloads InsightSoftwareConsortium/ITKPythonPackage#248

Merged

SimonRit mentioned this pull request Jan 27, 2023

External cuda common #528

Merged

SimonRit force-pushed the external-cuda-common branch 5 times, most recently from 62792b0 to 8ae0735 Compare February 3, 2023 11:22

SimonRit force-pushed the external-cuda-common branch 8 times, most recently from efb54dc to 7fd1fab Compare March 13, 2023 15:28

AlexyPellegrini and others added 2 commits March 14, 2023 06:46

ENH: Remove ITKCudaCommon from the source tree

4c035d6

Now builds with the remote module CudaCommon of ITK. Builds both as a remote module of ITK and as a standalone library.

ENH: Add CudaCommon dependency to Cuda wheels

0f67b3c

SimonRit force-pushed the external-cuda-common branch from 7fd1fab to 0f67b3c Compare March 14, 2023 05:48

SimonRit closed this Mar 14, 2023

Conversation

AlexyPellegrini commented May 30, 2022

Uh oh!

AlexyPellegrini commented May 30, 2022

Uh oh!

SimonRit commented May 31, 2022

Uh oh!

SimonRit commented Jun 3, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SimonRit Jun 8, 2022

Choose a reason for hiding this comment

Uh oh!

LucasGandel commented Jun 28, 2022

Uh oh!

SimonRit commented Jul 4, 2022

Uh oh!

SimonRit commented Jul 5, 2022

Uh oh!

AlexyPellegrini commented Jul 6, 2022

Uh oh!

SimonRit commented Jul 7, 2022

Uh oh!

SimonRit commented Jul 7, 2022

Uh oh!

AlexyPellegrini commented Jul 7, 2022

Uh oh!

SimonRit commented Jul 12, 2022

Uh oh!

SimonRit commented Aug 17, 2022

Uh oh!

LucasGandel commented Aug 18, 2022

Uh oh!

LucasGandel commented Aug 26, 2022

Uh oh!

SimonRit commented Sep 7, 2022

Uh oh!

SimonRit commented Oct 12, 2022

Uh oh!

LucasGandel commented Oct 12, 2022

Uh oh!

LucasGandel commented Oct 18, 2022

Uh oh!

SimonRit commented Nov 21, 2022

Uh oh!

SimonRit commented Nov 24, 2022

Uh oh!

SimonRit commented Mar 14, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants