What RESTful endpoints are used for the new OAuth2 mechanism?

[chrisburr]

I'm splitting this into a dedicated issue as the main PR (#4650) is going to be too busy already.

The things I would like to know for each exposed URL pattern are:

• What resource methods are supported? (i.e. GET/POST/PUT/...)
• What form of response is expected?
• What are the valid return codes and what do they mean?

Once this is stable it should be included in the documentation (or better yet, auto generated).

[TaykYoku]

Since the main of this implementation of PR #4650 is OAuth2 mechanism, only the necessary REST was provided so as not to burden the PR. At the moment, a tornado framework from the WebAppDIRAC project is used to implement REST endpoints, it was changed for this purpose.

• The main parent method for handlers providing the REST interface is tornado.web.Requesthandler. accordingly, all methods supported by it can be applied. Specifically, only GET method are described in this PR, because they are sufficient for the purposes.
• The expected response is a http response with json data. Json data - encoded request result(S_OK/S_ERROR) into a JSON string using encode method from DIRAC.Core.Utilities.JEncode.
• Return codes is the standart HTTP status codes.

I'll provide docs. Also the handlers contain some docs.

Also IIUC #4650 will implemented after #4677, so some part of code will be modify, I think, to use as a basis the changes in #4677.

[chrisburr]

Sorry but that's not the answer to my question. For example if I look at part of the GitLab API docs I can see all the URLs, which verbs are supported, what parameters are available and what response I should expect.

Looking at your slides I guess at the very least there is something like:

GET /auth/<IdP>?<options>

• URL variables:
  • IdP a string from xxxx/yyyy/IdProviders in the CS
• URL parameters:
  • I'm not sure what options might be on this endpoint. It should be clear what they're called, if they're optional, what values they accept and what they do.
• Responses:
  • 200 OK with body {"username": "cburr"}
  • 302 Found redirect to the IdP authorisation page

[TaykYoku]

All is well. I will prepare the relevant documentation for present APIs. Let's say in the section UserGuide/RESTAPIs(if it's good place for it). And I'll update documentations in the handlers code.

[chrisburr]

As it's hard to change once released, I think it would be nice to summarise them here first so we can think about if there is any room for improvements.

[TaykYoku]

Description of APIs in ConfigurationSystem/Utilities/ConfigurationHandler.py, FrameworkSystem/Utilities/AuthHandler.py, FrameworkSystem/Utilities/ProxyHandler.py updated. Others documentations in progress.

[TaykYoku]

Current RESTful endpoints for users authentication:

GET /auth/<IdP>?<options> -- this request for initialization authentication flow

• IdP -- is a Identity provider name(any section in Resources/IdProviders) registred in DIRAC that need to use for authentication.

Options:

• email -- send the authentication URL to the email(optional)

Response in the json format with the next keys:

• Status -- session status, required values: "needToAuth", "authed", "failed", "prepared", "in progress", "redirect", "finishing".
• URL -- link for authentication, generated by choosed identity provider. Returned if status is "needToAuth". This URL for
• Session -- current session ID, returned if status is "needToAuth".
• UserName -- user name, returned if status is "ready".

GET /auth/<session> -- redirect to identity provider authentication endpoint to enable the user to authenticate

• session -- session ID

Response depended from redirected endpoint

GET /auth/<session>/status -- retrieve session with status and describe

• session -- session ID

Response in the json format with the next keys:

• Comment -- comment
• Status -- session status, required statuses: "prepared", "in progress", "failed"
• Session -- session ID
• ID -- identity provider user identity number
• Provider -- identity provider name

GET /auth/redirect -- redirect endpoint to catch response from authentication flow of identity provider

Response in the html format

[chrisburr]

I think the **GET** /conf/<key>?<options> endpoint in #4650 currently exposes the entire CS to any unauthenticated person while access to the current CS always requires credentials. Have I understood this correctly?

If so, was this change discussed at any point? I can see benefits to it being public (e.g. dirac-configure would no longer require a proxy) but also potential issues (e.g. publicly exposing personally identifiable information).

[TaykYoku]

for the first synchronization with the server I need some public information, for this purpose I use REST because as far as I understand, even for access to new "Tornado CS" regardless of what information is requested, authorization is expected.
This endpoint may need to be upgraded, but overall there is some checking of user credentionals before requesting the Registry and group statuses information.

Actually, I thought about it, and .. IMHO its possible to add to diraccfg something like sections/options permissions and ownership, as it is organized for files/directories in Linux, for example. That is, for example, now when creating a section, a comment wrapped in special characters is added, you can also add the owner or group of the section, etc. This will allow more flexible use of the configuration and dynamicly application of access control for everyone, and as if at first glance it is not very difficult to do. How do you think this opinion is worth to discussing?

[chaen]

I need to look more in depth about the first part.

But for the second part about the permissions, I think it is a bad idea. The CS is one of the main bottleneck of DIRAC and I think one of the first place where it would be good to look at what the outside world is doing (random example: Consul). So I am not really in favor of adding yet an extra complexity in it.

[fstagni]

As of now the CS is the place where we dump ANY information on a running DIRAC infrastructure. Part of what's there is worthy protecting, but other parts can be public (part of it is ALREADY exposed publicly, as we need it for Pilot3 to work).

IIUC you need to bootstrap. What's the minimal amount of info that you need "possibly public"?

[TaykYoku]

For the first authorization request, need to know the public client_id and URL of the authorization endpoint.
But I think I can easily do without requesting this information, because it can be written in the configuration when installing the DIRAC client, so only two RESTful endpoints are needed.

proxy endpoint

RESTful endpoints to user proxy management to retrieve personal proxy.

GET LOCATION/proxy

Parameters:
+----------------+--------+-------------------------------------------+---------------------------------------+
| **name**       | **in** | **description**                           | **example**                           |
+----------------+--------+-------------------------------------------+---------------------------------------+
| Authorization  | header | Provide access token                      | Bearer jkagfbfd3r4ubf887gqduyqwogasd8 |
+----------------+--------+-------------------------------------------+---------------------------------------+
| lifetime       | query  | requested proxy live time in seconds      | 3600                                  |
|                |        | by default 6 hours (optional)             |                                       |
+----------------+--------+-------------------------------------------+---------------------------------------+
| voms           | query  | to get user proxy with VOMS extension,    | true                                  |
|                |        | by default false (optional)                |                                       |
+----------------+--------+-------------------------------------------+---------------------------------------+

Request example::

  GET LOCATION/proxy
  Authorization: Bearer <access_token>

Response::

  HTTP/1.1 200 OK

authorization server endpoints, provided by authlib and meets the RFC

Well known endpoint, specified by
`RFC8414 <https://tools.ietf.org/html/rfc8414#section-3>`_

Request examples::

  GET: LOCATION/.well-known/openid-configuration
  GET: LOCATION/.well-known/oauth-authorization-server

Responce::

  HTTP/1.1 200 OK
  Content-Type: application/json

  {
    "registration_endpoint": "https://domain/DIRAC/auth/register",
    "userinfo_endpoint": "https://domain/DIRAC/auth/userinfo",
    "jwks_uri": "https://domain/DIRAC/auth/jwk",
    "code_challenge_methods_supported": [
      "S256"
    ],
    "grant_types_supported": [
      "authorization_code",
      "code",
      "urn:ietf:params:oauth:grant-type:device_code",
      "implicit",
      "refresh_token"
    ],
    "token_endpoint": "https://domain/DIRAC/auth/token",
    "response_types_supported": [
      "code",
      "device",
      "id_token token",
      "id_token",
      "token"
    ],
    "authorization_endpoint": "https://domain/DIRAC/auth/authorization",
    "issuer": "https://domain/DIRAC/auth"
  }

JWKs endpoint

Request example::

  GET LOCATION/jwk

Response::

  HTTP/1.1 200 OK
  Content-Type: application/json

  {
    "keys": [
      {
        "e": "AQAB",
        "kty": "RSA",
        "n": "3Vv5h5...X3Y7k"
      }
    ]
  }

The UserInfo endpoint can be used to retrieve identity information about a user,
see `spec <https://openid.net/specs/openid-connect-core-1_0.html#UserInfo>`_

GET LOCATION/userinfo

Parameters:
+---------------+--------+---------------------------------+--------------------------------------------------+
| **name**      | **in** | **description**                 | **example**                                      |
+---------------+--------+---------------------------------+--------------------------------------------------+
| Authorization | header | Provide access token            | Bearer jkagfbfd3r4ubf887gqduyqwogasd87           |
+---------------+--------+---------------------------------+--------------------------------------------------+

Request example::

  GET LOCATION/userinfo
  Authorization: Bearer <access_token>

Response::

  HTTP/1.1 200 OK
  Content-Type: application/json

  {
    "sub": "248289761001",
    "name": "Bob Smith",
    "given_name": "Bob",
    "family_name": "Smith",
    "group": [
      "dirac_user",
      "dirac_admin"
    ]

The Client Registration Endpoint, specified by
`RFC7591 <https://tools.ietf.org/html/rfc7591#section-3.1>`_

POST LOCATION/register?data..

Parameters:
+----------------+--------+-------------------------------------------+---------------------------------------+
| **name**       | **in** | **description**                           | **example**                           |
+----------------+--------+-------------------------------------------+---------------------------------------+
| Authorization  | header | Provide access token                      | Bearer jkagfbfd3r4ubf887gqduyqwogasd8 |
+----------------+--------+-------------------------------------------+---------------------------------------+
| grant_types    | data   | list of grant types, more supported       | ["authorization_code","refresh_token"]|
|                |        | more supported grant types in *grants     |                                       |
+----------------+--------+-------------------------------------------+---------------------------------------+
| scope          | data   | list of scoupes separated by a space      | something                             |
+----------------+--------+-------------------------------------------+---------------------------------------+
| response_types | data   | list of returned responses                | ["token","id_token token","code"]     |
+----------------+--------+-------------------------------------------+---------------------------------------+
| redirect_uris  | data   | Redirection URI to which the response will| ['https://dirac.egi.eu/redirect']     |
|                |        | be sent.                                  |                                       |
+----------------+--------+-------------------------------------------+---------------------------------------+

*:mod:`grants <DIRAC.FrameworkSystem.private.authorization.grants>`

The device authorization endpoint can be used to request device and user codes.
This endpoint is used to start the device flow authorization process and user code verification.

POST LOCATION/device/<user code>?<query>

Parameters:
+----------------+--------+-------------------------------------------+---------------------------------------+
| **name**       | **in** | **description**                           | **example**                           |
+----------------+--------+-------------------------------------------+---------------------------------------+
| user code      | path   | in the last step to confirm recived user  | WE8R-WEN9                             |
|                |        | code put it as path parameter (optional)  |                                       |
|                |        | It's possible to add it interactively.    |                                       |
+----------------+--------+-------------------------------------------+---------------------------------------+
| client_id      | query  | The public client ID                      | 3f6eNw0E6JGq1VuzRkpWUL9XTxhL86efZw    |
+----------------+--------+-------------------------------------------+---------------------------------------+
| scope          | query  | list of scoupes separated by a space, to  | g:dirac_user                          |
|                |        | add a group you must add "g:" before the  |                                       |
|                |        | group name                                |                                       |
+----------------+--------+-------------------------------------------+---------------------------------------+
| provider       | query  | identity provider to autorize (optional)  | CheckIn                               |
|                |        | It's possible to add it interactively.    |                                       |
+----------------+--------+-------------------------------------------+---------------------------------------+


User code confirmation::

  GET LOCATION/device/<UserCode>

  Parameters:
    UserCode - recived user code (optional, it's possible to add it interactively)

Request example, to initialize a Device authentication flow::

  POST LOCATION/device?client_id=3f1DAj8z6eNw0E6JGq1Vu6efZwyV&scope=g:dirac_admin&provider=CheckIn_dev

Response::

  HTTP/1.1 200 OK
  Content-Type: application/json

  {
    "device_code": "TglwLiow0HUwowjB9aHH5HqH3bZKP9d420LkNhCEuR",
    "verification_uri": "https://marosvn32.in2p3.fr/DIRAC/auth/device",
    "interval": 5,
    "expires_in": 1800,
    "verification_uri_complete": "https://marosvn32.in2p3.fr/DIRAC/auth/device/WSRL-HJMR",
    "user_code": "WSRL-HJMR"
  }

Request example, to confirm the user code::

  POST LOCATION/device/WSRL-HJMR

Response::

  HTTP/1.1 200 OK

Redirect endpoint.
After a user successfully authorizes an application, the authorization server will redirect
the user back to the application with either an authorization code or access token in the URL.
The full URL of this endpoint must be registered in the identity provider.

Read more in `oauth.com <https://www.oauth.com/oauth2-servers/redirect-uris/>`_.
Specified by `RFC6749 <https://tools.ietf.org/html/rfc6749#section-3.1.2>`_.

GET LOCATION/redirect

The token endpoint, the description of the parameters will differ depending on the selected grant_type

POST LOCATION/token

Parameters:
+----------------+--------+-------------------------------------+---------------------------------------------+
| **name**       | **in** | **description**                     | **example**                                 |
+----------------+--------+-------------------------------------+---------------------------------------------+
| grant_type     | query  | what grant type to use, more        | urn:ietf:params:oauth:grant-type:device_code|
|                |        | supported grant types in *grants    |                                             |
+----------------+--------+-------------------------------------+---------------------------------------------+
| client_id      | query  | The public client ID                | 3f1DAj8z6eNw0E6JGq1VuzRkpWUL9XTxhL86efZw    |
+----------------+--------+-------------------------------------+---------------------------------------------+
| device_code    | query  | device code                         | uW5xL4hr2tqwBPKL5d0JO9Fcc67gLqhJsNqYTSp     |
+----------------+--------+-------------------------------------+---------------------------------------------+

*:mod:`grants <DIRAC.FrameworkSystem.private.authorization.grants>`

Request example::

  POST LOCATION/token?client_id=L86..yV&grant_type=urn:ietf:params:oauth:grant-type:device_code&device_code=uW5

Response::

  HTTP/1.1 400 OK
  Content-Type: application/json

  {
    "error": "authorization_pending"
  }

Authorization endpoint, the description of the parameters will differ depending on the selected grant_type

GET: LOCATION/authorization/<provider>

Parameters:
+----------------+--------+-------------------------------------------+---------------------------------------+
| **name**       | **in** | **description**                           | **example**                           |
+----------------+--------+-------------------------------------------+---------------------------------------+
| response_type  | query  | informs of the desired grant type         | code                                  |
+----------------+--------+-------------------------------------------+---------------------------------------+
| client_id      | query  | The client ID                             | 3f6eNw0E6JGq1VuzRkpWUL9XTxhL86efZw    |
+----------------+--------+-------------------------------------------+---------------------------------------+
| scope          | query  | list of scoupes separated by a space, to  | g:dirac_user                          |
|                |        | add a group you must add "g:" before the  |                                       |
|                |        | group name                                |                                       |
+----------------+--------+-------------------------------------------+---------------------------------------+
| provider       | query  | identity provider to autorize (optional)  | CheckIn                               |
|                |        | It's possible to add it interactively.    |                                       |

[TaykYoku]

proxy endpoint is deleted, the proxy will be returned instead of the token upon successful authorization, if the proxy scope present in request.