Skip to content

[v7r3][WIP] OAuth2/OIDC AuthN/AuthZ mechanism (Full version)#4650

Closed
TaykYoku wants to merge 105 commits into
DIRACGrid:integrationfrom
TaykYoku:FullOAuth_v7r2
Closed

[v7r3][WIP] OAuth2/OIDC AuthN/AuthZ mechanism (Full version)#4650
TaykYoku wants to merge 105 commits into
DIRACGrid:integrationfrom
TaykYoku:FullOAuth_v7r2

Conversation

@TaykYoku

@TaykYoku TaykYoku commented Jun 17, 2020

Copy link
Copy Markdown
Contributor

This PR introduces the AuthN/AuthZ mechanism to DIRAC based on the use of Identity Provider
services using OAuth2/OIDC protocols.

Few remarks about the implementation.

The user AuthN/AuthZ paradigm changes. Before we collected all the necessary user information
in the CS Registry and then used it to identify and authorize client requests. Now we can use
external identity providers dynamically. Each time a user is starting a DIRAC session, the
authentication can be delegated to an external identity provider. If authentication is successful,
then actual user profile information is cached in a dynamic session object which is created
for each active user. This session is used for further identification of user requests. Therefore,
DIRAC CS Registry becomes just one of the possible sources of the user profile information. Other
sources are (Federated) Identity Providers, e.g. Check-In, Indigo AIM, Google, etc, VOMS.

The use of X509 certificates is no more the only way for user identification. Therefore,
certificate DN's are no more the primary user identifiers. Now there can be also user IDs
from various identity providers. Within DIRAC, user name becomes the main, single and unique
user identifier independent on various user identities from different providers. As a consequence,
all the CS options with user identities as values, e.g. pilot user, should be expressed in terms
of user names rather than DNs. For backward compatibility current options in a form of DN are
still accepted but should be replaced eventually.

There can be now DIRAC users that do not have usual personal certificates and will identified
via OAuth2/OIDC mechanism. However, in order to ensure the work of the DISET protocol still based
on the use of X509 certificate proxies, the us of Proxy Provider services is enabled. These
services can create a certificate proxy on demand to be used by users in CLI interfaces and
by DIRAC services for performing operation on the users' behalf. Current solution is DIRAC Proxy
Provider using DIRAC CA certificates to generate user proxies (for the DIRAC internal use only).
Solution using RCauth Proxy Provider service is tested and will be available in later PRs.
Users having usual X509 certificates will continue to use those in a usual way.

Identity Providers and Proxy Providers are added as new types of Resources.

Although this PR introduces new functionalities, the current mechanisms of X509 based user
authentication are still maintained. Installations and users using old good X509 certificates
will continue to work without any changes.

It also presents the "core part" of the WebAppDIRAC extension for implementing some of the OAuth2 AuthN/AuthZ flow required REST API endpoints.

BEGINRELEASENOTES

NEW: Multiple changes to provide OAuth2/OIDC user AuthN/AuthZ and OAuth session management.
CHANGE: multiple changes to use username instead of DN where applicable.
Changes affect the following Systems: Transformation, Configuration, DataManagement, Framework, RequestManagement, WorkloadManagement. Also tests, Core, Interfaces, Resources.

*ConfigurationSystem
NEW: Registry - use cached data from the AuthManager and ProxyManager clients, modify methods in new logic context
FIX: Resources - split difficult method
FIX: Utilities - fix method name and path
NEW: Add REST API

*Core
NEW: Move the WebApp "core part" to the DIRAC
NEW: Add DB version
NEW: DictCache - add getDict method
CHANGE: AuthManager - split authorization logic, fix test
CHANGE: RequestHandler - fill credDict by AuthManager methods
CHANGE: align with the PR changes, use ID as IdP user ID in DISET transport flow

*FrameworkSystem
NEW: Add AuthManager service, DB, client and client with caching IdP information data, REST API
NEW: ProxyManager - split client part to parts with VOMS information cache data and simple client, add REST API
CHANGE: ProxyManager - modify to use user/group in requests
NEW: dirac-proxy-init - able to use authentication flow through the Identity providers
NEW: halo - new class to use spinners for waiting process, for ex. waiting authentication

*RequestManagementSystem
CHANGE: Request - add owner parameter, use AuthManager methods
CHANGE: align with the PR changes

*WorkloadManagementSystem
NEW: Add User/pilotUser parameter
CHANGE: align with the PR changes

*TransformationSystem
CHANGE: align with the PR changes

*DataManagementSystem
CHANGE: align with the PR changes

*Resources
NEW: add OAuth identity and proxy providers
CHANGE: align with the PR changes

*Interfaces
CHANGE: align with the PR changes

*tests
CHANGE: align with the PR changes

ENDRELEASENOTES

@chrisburr chrisburr marked this pull request as draft June 25, 2020 08:39
@TaykYoku TaykYoku closed this Jun 29, 2020
@TaykYoku TaykYoku reopened this Jun 29, 2020
@TaykYoku TaykYoku closed this Jun 29, 2020
@TaykYoku TaykYoku reopened this Jun 29, 2020
@TaykYoku TaykYoku closed this Jun 29, 2020
@TaykYoku TaykYoku reopened this Jun 29, 2020
@TaykYoku TaykYoku closed this Jun 29, 2020
@TaykYoku TaykYoku reopened this Jun 29, 2020
@TaykYoku TaykYoku closed this Jun 29, 2020
@TaykYoku TaykYoku reopened this Jun 29, 2020
@TaykYoku TaykYoku closed this Jun 30, 2020
@TaykYoku TaykYoku reopened this Jun 30, 2020
@TaykYoku TaykYoku marked this pull request as ready for review July 11, 2020 14:22
@TaykYoku TaykYoku changed the title [WIP] OAuth2/OIDC AuthN/AuthZ mechanism (Full version) [v7r2] OAuth2/OIDC AuthN/AuthZ mechanism (Full version) Jul 11, 2020

@fstagni fstagni left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't look at everything. This is just a first round of review.

Comment thread ConfigurationSystem/Client/Helpers/Registry.py Outdated
Comment thread ConfigurationSystem/Client/Helpers/Registry.py
Comment thread ConfigurationSystem/Client/Helpers/Registry.py Outdated
Comment thread ConfigurationSystem/Client/Helpers/Registry.py Outdated
Comment thread ConfigurationSystem/Client/Helpers/Registry.py
Comment thread FrameworkSystem/Client/ProxyManagerData.py Outdated
Comment thread FrameworkSystem/Client/ProxyManagerData.py Outdated
Comment thread FrameworkSystem/Utilities/halo.py
The starting definition of this class is as follows::

from WebAppDIRAC.Lib.WebHandler import WebHandler
from DIRAC.Core.Web.WebHandler import WebHandler

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like these changes. Not for this release, at least.

SESSION_FINISHING = "finishing"


class AuthDB(DB):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no integration test of this class...?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in progress

Comment thread FrameworkSystem/Utilities/OAuth2.py Outdated
Comment thread ConfigurationSystem/Client/Helpers/Registry.py Outdated
Comment thread FrameworkSystem/Utilities/OAuth2.py Outdated
@TaykYoku TaykYoku changed the title [v7r2] OAuth2/OIDC AuthN/AuthZ mechanism (Full version) [v7r2][WIP] OAuth2/OIDC AuthN/AuthZ mechanism (Full version) Sep 14, 2020

@andresailer andresailer left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here are a number of comments from minor to maybe major.
Some of them were written quite some time ago, so they might no longer apply.
Quite a lot of work to understand.

Comment thread ConfigurationSystem/Client/Helpers/Registry.py
Comment thread ConfigurationSystem/Client/Helpers/Registry.py Outdated
Comment thread ConfigurationSystem/Client/Helpers/Resources.py
Comment thread ConfigurationSystem/Client/Helpers/Resources.py
Comment thread ConfigurationSystem/Client/Helpers/Resources.py
Comment thread Resources/IdProvider/OAuth2IdProvider.py
Comment thread Resources/IdProvider/OAuth2IdProvider.py
Comment thread Resources/IdProvider/OAuth2IdProvider.py
Comment thread Resources/IdProvider/OAuth2IdProvider.py
Comment thread FrameworkSystem/scripts/dirac-proxy-init.py
@fstagni fstagni changed the title [v7r2][WIP] OAuth2/OIDC AuthN/AuthZ mechanism (Full version) [v7r3][WIP] OAuth2/OIDC AuthN/AuthZ mechanism (Full version) Jan 19, 2021
@TaykYoku TaykYoku mentioned this pull request Mar 18, 2021
@TaykYoku

Copy link
Copy Markdown
Contributor Author

This version is obsolete with the release of a v7.2.0, based on it a new one #5045 is created, all comments are taken into account.

@TaykYoku TaykYoku closed this Mar 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants