[v7r1] Make M2Crypto the default over pyGSI by chaen · Pull Request #4461 · DIRACGrid/DIRAC

chaen · 2020-02-20T15:12:55Z

First part of #4272

BEGINRELEASENOTES
*Core
CHANGE: use M2Crypto by default
ENDRELEASENOTES

chaen · 2020-02-20T16:11:24Z

@fstagni @atsareg there is a bad non resolved merged. I am fixing it here

chaen · 2020-02-20T16:45:30Z

@atsareg @TaykYoku The ProxyDB tests are failing

 test_connectDB (__main__.testDB)
Try to connect to the ProxyDB ... ok
test_getRemoveProxy (__main__.testDB)
Testing get, store proxy ... NOTE: __storeProxyOld function is deprecated (Only for DIRAC v6).
NOTE:   Used at /home/dirac/ServerInstallDIR/DIRAC/FrameworkSystem/DB/ProxyDB.py:352
FAIL
test_getUsers (__main__.testDB)
Try to get users from DB ... ok
test_purgeExpiredProxies (__main__.testDB)
Try to purge expired proxies ... ok

======================================================================
FAIL: test_getRemoveProxy (__main__.testDB)
Testing get, store proxy
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dirac/ServerInstallDIR/DIRAC/tests/Integration/Framework/Test_ProxyDB.py", line 451, in test_getRemoveProxy
    ': %s' % result.get('Message') or 'Error message is absent.'))
AssertionError: Must be ended with error: None

----------------------------------------------------------------------

Can you please check ? Thanks

chrisburr · 2020-02-27T09:23:22Z

@atsareg @TaykYoku this is still needs fixing before v7r1 can be released

TaykYoku · 2020-02-27T09:59:13Z

@chaen @chrisburr yes, this fixing in a progress

TaykYoku · 2020-02-27T14:32:08Z

@chaen @chrisburr fixed in #4469

chaen · 2020-03-10T09:44:39Z

Now I see two issues:

One is the known one with the proxy that @TaykYoku is looking at

 ======================================================================
ERROR: test_getRemoveProxy (__main__.testDB)
Testing get, store proxy
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dirac/ServerInstallDIR/DIRAC/tests/Integration/Framework/Test_ProxyDB.py", line 540, in test_getRemoveProxy
    result = db.getVOMSProxy(dn, group, time, role)
  File "/home/dirac/ServerInstallDIR/DIRAC/FrameworkSystem/DB/ProxyDB.py", line 903, in getVOMSProxy
    attrs = vomsMgr.getVOMSAttributes(chain).get('Value') or ['']
  File "/home/dirac/ServerInstallDIR/DIRAC/Core/Security/VOMS.py", line 39, in getVOMSAttributes
    result = self.getVOMSProxyInfo(proxy, "all")
  File "/home/dirac/ServerInstallDIR/DIRAC/Core/Security/VOMS.py", line 131, in getVOMSProxyInfo
    res = proxyDict['chain'].getVOMSData()
  File "/home/dirac/ServerInstallDIR/DIRAC/Core/Security/m2crypto/X509Chain.py", line 580, in getVOMSData
    res = cert.getVOMSData()
  File "/home/dirac/ServerInstallDIR/DIRAC/Core/Utilities/Decorators.py", line 185, in innerFunc
    return meth(*args, **kwargs)
  File "/home/dirac/ServerInstallDIR/DIRAC/Core/Security/m2crypto/X509Certificate.py", line 382, in getVOMSData
    vomsExt = asn1_utils.decodeVOMSExtension(self.__certObj)
  File "/home/dirac/ServerInstallDIR/DIRAC/Core/Security/m2crypto/asn1_utils.py", line 227, in decodeVOMSExtension
    attrValStr = _decodeASN1String(rdnNameAttr['value'])
  File "/home/dirac/ServerInstallDIR/DIRAC/Core/Security/m2crypto/asn1_utils.py", line 119, in _decodeASN1String
    raise PyAsn1Error("Could not find a correct decoding type")
PyAsn1Error: Could not find a correct decoding type

And the other one is with the TaskQueueDB

2020-03-10 09:22:20 UTC WorkloadManagement/JobMonitoring ERROR: Exception while calling initialization function
Traceback (most recent call last):
  File "/home/dirac/ServerInstallDIR/DIRAC/Core/DISET/private/Service.py", line 133, in initialize
    result = initFunc(dict(self._serviceInfoDict))
  File "/home/dirac/ServerInstallDIR/DIRAC/WorkloadManagementSystem/Service/JobMonitoringHandler.py", line 43, in initializeJobMonitoringHandler
    gTaskQueueDB = TaskQueueDB()
  File "/home/dirac/ServerInstallDIR/DIRAC/WorkloadManagementSystem/DB/TaskQueueDB.py", line 67, in __init__
    raise Exception("Can't create tables: %s" % result['Message'])
Exception: Can't create tables: MySQL Error ( 1131 : Recursive Foreign Keys in tq_Jobs, tq_TQToSites, tq_TQToGridMiddlewares)
2020-03-10 09:22:20 UTC WorkloadManagement/JobMonitoring ERROR: Exception while calling initialization function: Can't create tables: MySQL Error ( 1131 : Recursive Foreign Keys in tq_Jobs, tq_TQToSites, tq_TQToGridMiddlewares)

fstagni · 2020-03-10T10:03:34Z

Is the TaskQueueDB issue that you showed only for some specific MySQL version?

chaen · 2020-03-10T10:07:31Z

It only shows up in (8,slc6,yes)

fstagni · 2020-03-10T11:38:12Z

I doubt that the yes (M2Crypto) flag there means something. If it does, I am not sure we want to debug it...

TaykYoku · 2020-03-10T12:08:01Z

Is this test running with patch #4469 or not?

TaykYoku · 2020-03-10T12:25:11Z

I already see that test running without #4469 patch.
PyAsn1Error: Could not find a correct decoding type should be resolved in #4469.

chaen · 2020-03-10T12:28:00Z

I believe yes is for the ThreadPool, not M2Crypto

chaen · 2020-03-10T12:29:25Z

#4469 hides the problem. The fact of having user_1 instead of user should not be a problem. But let's discuss it on that PR

TaykYoku · 2020-03-10T12:34:05Z

@chaen can you describe your opinion in #4469 to start discuss, thanks.

chaen · 2020-03-13T08:42:41Z

@fstagni again the expected checks are not the correct one

chaen · 2020-03-16T10:07:13Z

@fstagni can you please correct the expected checks, and also look at the TaskQueueDB error ?

fstagni · 2020-03-16T10:22:32Z

I have fixed the expected checks. I will have a look at the TaskQueueDB thing later.

chaen · 2020-03-23T18:02:12Z

@fstagni any news ?

fstagni · 2020-03-23T18:08:05Z

Sorry, I was busy with other stuff and this slipped a bit below. I will try tomorrow morning.

fstagni · 2020-03-24T09:35:57Z

The TaskQueueDB errors that you mention are not appearing in the last tests run.

chaen · 2020-03-24T10:15:54Z

you mean I should just rebase that one and re-run ?

fstagni · 2020-03-24T11:16:32Z

I mean that this:

2020-03-10 09:22:20 UTC WorkloadManagement/JobMonitoring ERROR: Exception while calling initialization function
Traceback (most recent call last):
  File "/home/dirac/ServerInstallDIR/DIRAC/Core/DISET/private/Service.py", line 133, in initialize
    result = initFunc(dict(self._serviceInfoDict))
  File "/home/dirac/ServerInstallDIR/DIRAC/WorkloadManagementSystem/Service/JobMonitoringHandler.py", line 43, in initializeJobMonitoringHandler
    gTaskQueueDB = TaskQueueDB()
  File "/home/dirac/ServerInstallDIR/DIRAC/WorkloadManagementSystem/DB/TaskQueueDB.py", line 67, in __init__
    raise Exception("Can't create tables: %s" % result['Message'])
Exception: Can't create tables: MySQL Error ( 1131 : Recursive Foreign Keys in tq_Jobs, tq_TQToSites, tq_TQToGridMiddlewares)
2020-03-10 09:22:20 UTC WorkloadManagement/JobMonitoring ERROR: Exception while calling initialization function: Can't create tables: MySQL Error ( 1131 : Recursive Foreign Keys in tq_Jobs, tq_TQToSites, tq_TQToGridMiddlewares)

that you showed above does not appear anywhere in the last tests run.

chaen force-pushed the rel-v7r1_FEAT_defaultM2Crypto branch from 96a1029 to f5e4f3b Compare March 2, 2020 15:09

chrisburr mentioned this pull request Mar 5, 2020

[v7r2] Remove pyGSI #4481

Merged

TaykYoku mentioned this pull request Mar 5, 2020

[v7r1] Fix proxyDB test #4469

Merged

chaen force-pushed the rel-v7r1_FEAT_defaultM2Crypto branch from f5e4f3b to 81df8ea Compare March 10, 2020 09:12

chaen force-pushed the rel-v7r1_FEAT_defaultM2Crypto branch from 81df8ea to 7174be5 Compare March 12, 2020 10:24

chaen force-pushed the rel-v7r1_FEAT_defaultM2Crypto branch from 57850b4 to 74a9ae1 Compare March 24, 2020 13:23

chaen force-pushed the rel-v7r1_FEAT_defaultM2Crypto branch from 74a9ae1 to db1f395 Compare April 9, 2020 08:07

Make M2Crypto the default over pyGSI

d8dfdd8

chaen force-pushed the rel-v7r1_FEAT_defaultM2Crypto branch from db1f395 to d8dfdd8 Compare April 16, 2020 13:56

chaen marked this pull request as ready for review April 16, 2020 14:26

chaen requested review from andresailer, atsareg and fstagni as code owners April 16, 2020 14:26

chaen changed the title ~~[WIP][v7r1] Make M2Crypto the default over pyGSI~~ [v7r1] Make M2Crypto the default over pyGSI Apr 16, 2020

andresailer approved these changes Apr 16, 2020

View reviewed changes

fstagni approved these changes Apr 16, 2020

View reviewed changes

atsareg merged commit d70aca3 into DIRACGrid:rel-v7r1 Apr 19, 2020

chaen deleted the rel-v7r1_FEAT_defaultM2Crypto branch August 11, 2022 12:57

Uh oh!

Conversation

chaen commented Feb 20, 2020

Uh oh!

chaen commented Feb 20, 2020

Uh oh!

chaen commented Feb 20, 2020

Uh oh!

chrisburr commented Feb 27, 2020

Uh oh!

TaykYoku commented Feb 27, 2020

Uh oh!

TaykYoku commented Feb 27, 2020

Uh oh!

chaen commented Mar 10, 2020

Uh oh!

fstagni commented Mar 10, 2020

Uh oh!

chaen commented Mar 10, 2020

Uh oh!

fstagni commented Mar 10, 2020

Uh oh!

TaykYoku commented Mar 10, 2020

Uh oh!

TaykYoku commented Mar 10, 2020

Uh oh!

chaen commented Mar 10, 2020

Uh oh!

chaen commented Mar 10, 2020

Uh oh!

TaykYoku commented Mar 10, 2020

Uh oh!

chaen commented Mar 13, 2020

Uh oh!

chaen commented Mar 16, 2020

Uh oh!

fstagni commented Mar 16, 2020

Uh oh!

chaen commented Mar 23, 2020

Uh oh!

fstagni commented Mar 23, 2020

Uh oh!

fstagni commented Mar 24, 2020

Uh oh!

chaen commented Mar 24, 2020

Uh oh!

fstagni commented Mar 24, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants