Remove the py extension from all the scripts

[chaen]

Could we do that ? It does not harm, and would ease a lot the distribution with standard tools (setuptool or others)

[phicharp]

You mean the scripts "in place" in github?
I would much more prefer that the scripts are real python modules, i.e. with a valid name (replacing - with _) as it would avoid to have to create a small script which imports the actual engine, such that SW can be shared between scripts...
But if we go your route proposed here, I would "impose" this schema, i.e. almost no code in the script but the definition of Script switches (or not ;-)

[chaen]

Either or, I am fine. I am happy if we make the scripts modules with entry point, so we can even use proper setuptools features:
http://setuptools.readthedocs.io/en/latest/setuptools.html#automatic-script-creation

[atsareg]

I do not think that there is an urgent need to refactor scripts contents leaving only swicthes in these modules. This is a lot of work and should be done all at once, I do not think that there will be a hero to undertake this change, not to mention a lot of debugging of inevitable errors. Removing .py extensions will make the scripts "non-python" modules, so breaking another requirement for code checking tools. So, I do not see advantages of dropping the .py extensions

[chaen]

That is the whole point: they are not python modules, they are scripts :-) And as I said, it really ease the move to standard packaging tools.
I agree with you that changing all the scripts would require a hero :-) So I am not in favour of doing it in one go either.

[phicharp]

I am a bit confused as I was proposing to change the names of commands with underscores to make them valid python modules... and you seem to agree:
#3531

We seem being trying to do the same thing in many different ways...
In LHCbDirac I have changed gen_scripts already with 2 changes
(MR is https://gitlab.cern.ch/charpent/LHCbDIRAC/commit/903518d825d6b7b0b81a162cee04e6c735f4d9df)

1. Replaces "_" with "-" when creating the command in the scripts directory
2. Use a symlink rather than a copy as a copy is not necessary (symlink must be relative of course).

I believe it doesn't harm to take both approached:

1. As a first mesure, rename scripts with _ instead of -
2. Shrink command scripts to almost nothing and move the code in a module in the Client subdirectory. In doing this the .py can be removed or not, this is irrelevant...

My 2 cents

[chaen]

About the content we agree.
About the *.py it is relevant because of the link I gave. These tools will use the same name as your script. SO if your script is called a_b-c.py, that will be it's name when being distributed. The idea with such tools is that gen_scripts is not needed anymore

[phicharp]

OK, understood...

[atsareg]

Am I understanding correctly that ALL the commands must be listed in the setup.py ? So, it should be updated with each new command or dropping a command, no automatic traversal in search of commands is possible ? Also, the DIRAC command wrapper makes necessary environment settings which are necessary for example to run commands on multiple platforms with a single installation, e.g. shared via AFS or CVMFS. Are you suggesting that these code should go into the shrinked command scripts ?

[chaen]

The scripts are discovered : https://github.com/DIRACGrid/DIRAC/blob/integration/setup.py#L33

Sorry I am not familiar with the wrapper you talk about, we use the scripts directly in LHCb. What is it exactly ?

[phicharp]

I think this wrapper is a very old story that Ricardo implemented ages ago and that has gone for long, no?
from the gen_scripts that I saw, the script file is just copied to a file without the .py
In LHCb instead of copying we create a symlink (as of v9r0, but I have been doing it in my own installation for years).

[atsareg]

You can see the command wrapper details right inside the dirac-deploy-scripts command. It is still in use. This is how commands are installed with a vanilla dirac-install.

[phicharp]

Oups, sorry, my bad!
I must have overlooked that... Always looked to me as an overkill, as setting up PYTHONPATH, PATH and LD_LIBRARY_PATH is not a big deal...

[chaen]

Do I understand correctly that this script sets up a few environment variable (PYTHONPATH, LD_LIBRARY_PATH, PATH, X509_CERT_DIR, and DCOMMANDS_PPID + DYLD_LIBRRY_PATH that I did not know about :-) ) before calling the actual script ? If so, I must confess I do not exactly know how, but I am convinced that standard tools give us such a possibility.