Move dirac-install to management repo

[fstagni]

From discussions under #4858, and to decouple from it:
see discussions starting in #4858 (comment)

[andresailer]

Also used here: https://github.com/DIRACGrid/management/blob/451bf9faa52c7deed9f0c47fbc786821fcfae8e8/dirac-distribution/Dockerfile#L45

[fstagni]

To be sorted out with high priority as now dirac-install is in way too many locations: #4880

[fstagni]

Trying to collect more info and use cases to sort out this little mess.

Right now, dirac-install.py is in 3 locations:

• DIRAC repo (integration branch):
  • src/DIRAC/Core/scripts/dirac-install.py [1]
  • Core/scripts/dirac-install.py [2]
• Pilot repo:
  • Pilot/dirac-install.py [3]

and we want to have it only in one location, in management repo (location [4]).

[1] is referenced in:

• DIRAC/.github/workflows/diracinstall.yml
• DIRAC/.github/workflows/pilotWrapper.yml
• DIRAC/src/DIRAC/WorkloadManagementSystem/Utilities/test/Test_PilotWrapper.py
• DIRAC/container/Dockerfile
• DIRAC/docs/source/AdministratorGuide/Tutorials/basicTutoSetup.rst
• DIRAC/docs/source/AdministratorGuide/Tutorials/basicTutoSetup.sh
• DIRAC/docs/source/DeveloperGuide/WebAppDIRAC/InstallWebAppDIRAC/index.rst
• DIRAC/docs/source/UserGuide/GettingStarted/InstallingClient/index.rst
• DIRAC/tests/Integration/WorkloadManagementSystem/Test_GenerateAndExecutePilotWrapper.py
• DIRAC/tests/Jenkins/utilities.sh

[2] is referenced in:

• https://github.com/DIRACGrid/management/blob/451bf9faa52c7deed9f0c47fbc786821fcfae8e8/dirac-distribution/Dockerfile#L45
• DIRAC/src/DIRAC/Core/scripts/install_site.sh (to be checked)
• DIRAC/src/DIRAC/Resources/Computing/SingularityComputingElement.py (indirectly: it's expected to be found in the locally installed DIRAC version)
• DIRAC/src/DIRAC/WorkloadManagementSystem/Agent/SiteDirector.py (indirectly: it's expected to be found in the locally installed DIRAC version)
• DIRAC/src/DIRAC/WorkloadManagementSystem/Utilities/PilotCStoJSONSynchronizer.py (as a fallback, and indirectly: it's expected to be found in the locally installed DIRAC version)
• DIRAC/docs/source/UserGuide/Tutorials/ClientInstallation/index.rst
• DIRAC/virtualmachine/Vagrantfile
• VMDIRAC/VMDIRAC/WorkloadManagementSystem/Bootstrap/vm-bootstrap-functions
• DIRACOS/docs/60_useDIRACOS.md

[3] is referenced in:

• DIRAC/src/DIRAC/TransformationSystem/scripts/dirac-production-runjoblocal.py
• DIRAC/src/DIRAC/WorkloadManagementSystem/Utilities/PilotCStoJSONSynchronizer.py (with a fallback)
• Pilot/tests/CI/utilities.sh

(all DIRAC locations above are for the integration branch)

As for DIRAC extensions (add here just for convenience), from what I know:
[2] is referenced in:

• ILCDIRAC CIs
• LHCbDIRAC: should be seen after update to new structure (MR ready)

My attack strategy would be:

for v7r2:

• put dirac-install.py in [4]
• move all references to [1] to [4]
• move all references to [3] to [4]
• remove it from [1]
• remove it from SingularityComputingElement
• remove it from dirac_production_runjoblocal.py
• remove it from all the documentation
• remove it from install_site.sh
• cleanup tests/Jenkins scripts
• remove it from [3]
• move all references to [2] to [4]

Then, we can't remove it from [2] until also extensions pick up [4]. We can target v7r3 for that, even though the release is not very important.

Comments?

[fstagni]

Linking DIRACGrid/management#10

[chaen]

so, clearly, duplicating 42 times dirac-install was not a brilliant idea.

This commit of mine is not anymore in integration in any of the dirac-install.py.
a7cdbd9

And at the very moment, they are not even the same

(dev-dirac) [chaen@pceplbc04 DIRAC]$ diff src/DIRAC/Core/scripts/dirac-install.py Core/scripts/dirac-install.py 
2116c2116
<           'export X509_VOMSES=%s' %
---
>           'export X509_VOMSES=${X509_VOMSES:-%s}' %
2462c2462
<           'export X509_VOMSES=%s' %
---
>           'export X509_VOMSES=${X509_VOMSES:-%s}' %
2661,2663d2660
<                                  "scripts", "dirac_deploy_scripts.py")
<     if not os.path.isfile(ddeLocation):
<       ddeLocation = os.path.join(cliParams.targetPath, "DIRAC", "src", "DIRAC", "Core",

So, how do we do concretely now ?