Merge epoch2 and epoch1 - third part (CPPProcess and HelAmps)

[valassi]

This is the third (final?) PR for issue #139: the idea is to make sure that epoch1 has the same code as, or at least includes relevant code from, epoch2. This is because all my next PRs (vectorization, heterogeneous) are based on epoch1.

This is the followup to PR #140 and #149.

I keep it as WIP for the moment.

[valassi]

I have finally completed this PR #151. Now epoch1 and epoch2 (before vectorization) are strictly identical. I will then go on and develop on top of epoch1 (vectorization and more), while keeping epoch2 as a pre-vectorization reference.

This is a summary of what this contains:

In general

• proceed iteratively, with detailed code/performance comparisons at each step
• merge into the vectorization ("klas2ep12") branch frequently to prepare for that eventually
• first use the same function names and formatting to allow side-by-side comparisons, then modify both a lot
• going on, I almost rewrote from scratch the XXX and FFV functions starting from those of epoch2
• eventually, epoch1 and epoch2 are now IDENTICAL (except for the "epoch_process_id" tag)

Changes in file directory structure

• HelAmps_sm.h/cc : epoch2 comment out code but keep files as they were initially, also copy them as-is in epoch1
• CPPProcess.cc : epoch2 embedded epoch2 HelAmps_sm.cc initially as-is, then started modifying the contents in parallel to epoch1
• performance scripts, headers etc: try to move everything to Subprocesses and add links in PSigma, as in epoch2
  (WARNING: not yet done for epoch1 runTest.cc, where I have big changes for vectorization; temporarely moved it also for epoch2)

Changes in file formatting, cosmetics, minor content issues

• CPPProcess.cc : epoch2 (and epoch1) add mgDebug calls (plus minor fixese when already there) and returns on void

Code cleanup potentially affecting performance

• CPPProcess.cc : epoch2 "clean up" the code, also to simplify vectorization later on
  (e.g. define variables only when used, and initialise them immediately; use c++11 zero initialization...)

Changes in XXX functions

• CPPProcess.cc : epoch1 rename imzxxxM0, ixzxxxM0, oxzxxxM0 (remove M0 as in epoch2)
• CPPProcess.cc : epoch1 add (commented out) ixxxxx, ipzxxx, vxxxxx, sxxxxx, oxxxxx, opzxxx, omzxxx as-is from epoch2
• CPPProcess.cc : epoch1/2, rewrite ASSUMPTIONS comment using the explanation given in epoch2
• CPPProcess.cc : epoch2 interface change, replace 'const int&' by 'const int' (nhel and nsf)
• CPPProcess.cc : epoch2 interface change, replace 'const fptype&' by 'const fptype' (e.g. masses)
• CPPProcess.cc : epoch1/2 interface change, replace 'cxtype fi[6]' by 'cxtype* fi'
• CPPProcess.cc : epoch1/2 move the definition of nwf=5 and nw6=6 from mgOnGpuConfig.h to CPPProcess.cc
• CPPProcess.cc : epoch1 interface change, replace fis/fos by fi/fo as in epoch2
  (generally try to use Olivier's epoch2 naming/structure, eventually rewrite them from scratch starting from epoch2)
• CPPProcess.cc : epoch1 use opzxxx instead of oxzxxx as in epoch2!
• CPPProcess.cc : epoch1 eventually rewrite all xxx functions from scratch starting from epoch2 modified as above
• CPPProcess.cc : epoch1/2 ensure ixxxxx/oxxxxx are ok, test these instead of the mass=0 functions, functionally ok but slower
• CPPProcess.cc : epoch1/2 uncomment all xxx functions and ensure they build (including ipzxxx, vxxxxx, sxxxxx, omzxxx)
• CPPProcess.cc : epoch1/2 replace p0123 by pvec0123 in all xxx functions consistently
• CPPProcess.cc : epoch1/2 remove two build warnings from sxxxxx (unused function)
• CPPProcess.cc : epoch1/2 remove two builds warnings in vxxxxx (ternary operator), WARNING must be cross checked!
• CPPProcess.cc : epoch1/2 (eventually) add START/END LOOP and extra brace in all xxx functions
  (the extra brace is because later on in some cases fptype_v is handled by expliciting loop on ieppV without SIMD)

Changes in FFV functions

• CPPProcess.cc : epoch1 add (initially commented out, later uncommented) FFV2_0, FFV2_3, FFV4_0, FFV4_3 as in epoch2
• CPPProcess.cc : epoch1/2 eventually ensure all FFV functions build including FFV2_0, FFV2_3, FFV4_0, FFV4_3
• CPPProcess.cc : epoch1 use GPU constant memory for couplings as in epoch2
  (WARNING: this increases registers without affecting throughput, from a performance view point hardcoding them is better,
  but from a usability view point these "constants" must be read from user configuration files - hence do use constant memory)
• CPPProcess.cc : epoch2 use fptype (was hardcoded double) for masses/couplings as in epoch1
• CPPProcess.cc : epoch1 rewrite all FFV functions from scratch starting from epoch2 (with few modifications as above)
  (this includes the FFV1_0, FFV1P0_3, FFV2_4_0 and FFV2_4_3 that I had previously modified in epoch1 - I discard those!)
• CPPProcess.cc : epoch1/2 gain 1% by inverting order of two statements in FFV2_4_0 (?!)
• CPPProcess.cc : epoch1/2 gain ~1% by inverting order of two statements in FFV2_4_3 (?!)
• CPPProcess.cc : epoch1/2 replace ".real()" by "cxreal()", more portable (cucomplex) and makes vectorization easier

Changes in sigmakin or calculate_wavefunction

• CPPProcess.cc : epoch1 helicity filtering bug fix ME!=MELast (as in epoch2) instead of ME>MELast
• CPPProcess.cc : epoch1 add using namespace MG5_sm in calculate_wavefunction, as in epoch2
• CPPProcess.cc : epoch1 use amp[1] instead of amp[2] as in epoch2, add to running sum after each FFV call
  (WARNING! this decreases epoch1 C++ performance by ~5% to the same level as epoch2, but is useful
  for "2 to many" processes as it probably saves some registers; I was not yet monitoring registers)
• CPPProcess.cc : epoch1/2 use "jamp -= amp0" instead of "jamp += (-amp0)"
  (surprisingly, this gains back some of the ppereformance previously lost in epoch1)
• CPPProcess.cc : epoch2 add OMP multi-threading as in epoch1
• CPPProcess.cc : epoch2 use fptype (was hardcoded double) for colors as in epoch1
• CPPProcess.cc : epoch1 use opzxxx instead of oxzxxx as in epoch2!

Changes in CPPProcess other than XXX, FFV or formatting

• CPPProcess.cc : epoch1 remove duplicate code defining tHel in the same way for c++ and cuda
• CPPProcess.cc : epoch2 remove 'static' from 'const int tHel" as in epoch1
• CPPProcess.cc : epoch2 clean up class structure (fix warnings) and copy to epoch1

Printouts and performance tools

• check.cc : epoch1/2, add Epoch/Process/Language printout (using epoch_process_id header)
• throughput12.sh : epoch1/2 add script to compare ep2/ep1 performances as the code changes
• throughput12.sh : epoch1/2 repeat c++ test also with full OMP threads (if -omp is specified)
• throughput12.sh : dump sigmaKin register usage from ncu
• profile.sh : epoch1 a few fixes for ncu

TODO EVENTUALLY (after vectorization: add to the running list from previous PRs)

• remove XXX and FFV functions from CPPProcess, move them only to HelAmps
• move runTests.cc to Subprocesses and add a link in the PSigma directory

[valassi]

My CURRENT BASELINE PERFORMANCE (before vectorization) after this PR is described in 1c25007

-------------------------------------------------------------------------
Process                     = EPOCH1_EEMUMU_CPP
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MatrixElems] (3) = ( 1.133317e+06                 )  sec^-1
MeanMatrixElemValue         = ( 1.372152e-02 +- 3.269516e-06 )  GeV^0
TOTAL       :     8.050711 sec
real    0m8.079s
-------------------------------------------------------------------------
Process                     = EPOCH1_EEMUMU_CUDA
EvtsPerSec[MatrixElems] (3) = ( 6.852279e+08                 )  sec^-1
MeanMatrixElemValue         = ( 1.372152e-02 +- 3.269516e-06 )  GeV^0
TOTAL       :     1.233023 sec
real    0m1.552s
==PROF== Profiling "_ZN5gProc8sigmaKinEPKdPd": launch__registers_per_thread 164
-------------------------------------------------------------------------
Process                     = EPOCH2_EEMUMU_CPP
OMP threads / `nproc --all` = 1 / 4
EvtsPerSec[MatrixElems] (3) = ( 1.132827e+06                 )  sec^-1
MeanMatrixElemValue         = ( 1.372152e-02 +- 3.269516e-06 )  GeV^0
TOTAL       :     8.059035 sec
real    0m8.086s
-------------------------------------------------------------------------
Process                     = EPOCH2_EEMUMU_CUDA
EvtsPerSec[MatrixElems] (3) = ( 6.870531e+08                 )  sec^-1
MeanMatrixElemValue         = ( 1.372152e-02 +- 3.269516e-06 )  GeV^0
TOTAL       :     1.177079 sec
real    0m1.485s
==PROF== Profiling "_ZN5gProc8sigmaKinEPKdPd": launch__registers_per_thread 164
-------------------------------------------------------------------------