(for PilotLogging) find the VO from the proxy (and from the token)

[fstagni]

This issue is a follow-up of discussions happening in the thread started at #6208 (comment) where we asked how to find a VO from an existing proxy and/or token.

Few alternative options were considered:

1. use off-the-shelf tools like openssl but this not seem possible (?) for VOMS proxies
2. use voms-proxy-info, but this might not be available
3. implement a (REST?) API that, given a pilotStamp, returns the VO.

Option 3 might be the way to go.

[andresailer]

Maybe I missed this by not paying attention during the discussion, but why not add another environment variable besides the DIRAC_PILOT_STAMP (#6405) to add the VO that pilot is for?

[martynia]

We, could, if we can pass any environment variables at all. This is a question to @fstagni on the lhcb HLT use case. Is it possible to pass an environment variables in this case? Otherwise, if we use a SiteDirector , we could add another variable, I think. But if we continue to add more variables and parameters we might end up in a similar situation we have now, when a pilot is configured by a quite large set of command line arguments.

[andresailer]

I think the LHCB HLT will know that it will be LHCB VO, there is no need to query things for Single VO instances.

DIRAC_PILOT_STAMP is going to be an environment variable in most cases in #6405 (I can't tell for SSHComputingElement at first glance, maybe missing more)

If the VO is the thing to use further configuration according to pilot.json, it seems more direct to me to send the VO along instead of asking a service for the VO given a pilot stamp.

[martynia]

Don't forget and even in a single VO case we need DIRAC_PILOT_STAMP, otherwise we will be unable to link a pilot log file or a future pilo log MQ entry to a pilot stamp which uniquely identifies a pilot.

[chrisburr]

This is a question to @fstagni on the lhcb HLT use case. Is it possible to pass an environment variables in this case?

Yes, it's launched by a shell script so we can set whatever we need.

[chaen]

For what it's worth (car surely be improved)

proxyFile=/tmp/x509up_u56212
vomsOffset=$(openssl x509 -in ${proxyFile} -outform der | openssl asn1parse -inform der -i | grep -A 1 '1.3.6.1.4.1.8005.100.100.5' | tail -n 1| awk -F ':' {'print $1'})
fqansOffset=$(openssl x509 -in ${proxyFile} -outform der | openssl asn1parse -inform der -i -strparse ${vomsOffset}  | grep -B 1 '1.3.6.1.4.1.8005.100.100.4' | head -n 1 | awk -F ':' {'print $1'})
openssl x509 -in ${proxyFile} -outform der | openssl asn1parse -inform der -i -strparse ${vomsOffset}  -strparse ${fqansOffset} | grep Role | head -n 1  | sed -E 's@.*:/([a-zA-Z0-9]+)/.*@\1@g'

results in

lhcb

[chaen]

as rightly noticed by @chrisburr , if there are further delegation of proxy (typically in a job), you need to loop that on the inner certificates

[chrisburr]

This needs downgrading to be Python 2.7 compatible but this should work:

import re
from base64 import b16decode
from subprocess import check_output
from typing import Optional

VOMS_FQANS_OID = b"1.3.6.1.4.1.8005.100.100.4"
VOMS_EXTENSION_OID = b"1.3.6.1.4.1.8005.100.100.5"
RE_OPENSSL_ANS1_FORMAT = re.compile(rb"^\s*\d+:d=(\d+)\s+hl=")


def parse_asn1(data: bytes) -> list[bytes]:
    cmd = ["openssl", "asn1parse", "-inform", "der"]
    return check_output(cmd, input=data).split(b"\n")


def find_extension(oid: bytes, lines: bytes) -> Optional[int]:
    for i, line in enumerate(lines):
        if oid in line:
            return i


def get_vo(proxy_data: bytes) -> str:
    chain = re.findall(rb"-----BEGIN CERTIFICATE-----\n.+?\n-----END CERTIFICATE-----", proxy_data, flags=re.DOTALL)
    for cert in chain:
        cert_info = parse_asn1(check_output(["openssl", "x509", "-outform", "der"], input=cert))
        # Look for the VOMS extension
        idx_voms_line = find_extension(VOMS_EXTENSION_OID, cert_info)
        if idx_voms_line is None:
            continue
        voms_extension = parse_asn1(b16decode(cert_info[idx_voms_line+1].split(b":")[-1]))
        # Look for the attribute names
        idx_fqans = find_extension(VOMS_FQANS_OID, voms_extension)
        initial_depth, = map(int, RE_OPENSSL_ANS1_FORMAT.match(voms_extension[idx_fqans-1]).groups())
        for line in voms_extension[idx_fqans:]:
            depth, = map(int, RE_OPENSSL_ANS1_FORMAT.match(line).groups())
            if depth <= initial_depth:
                break
            # Look for a role, if it exists the VO is the first element
            if match := re.search(rb"OCTET STRING\s+:/([a-zA-Z0-9]+)/Role=", line):
                return match.groups()[0].decode()
    raise NotImplementedError("Something went very wrong")


with open("/tmp/x509up_u1000", "rb") as fp:
    print(get_vo(fp.read()))