Skip to content

[9.0] fix: get DiracX token #8300

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: integration
Choose a base branch
from
Open

[9.0] fix: get DiracX token #8300

wants to merge 2 commits into from

Conversation

arrabito
Copy link
Contributor

*Core
FIX: Fix regular expression that matches DiracX token

@arrabito
Copy link
Contributor Author

This fix is needed to fix the DiracX token extraction by the JobAgent.
We found that the regular expression was not matching the correct block and was giving the following error:


2025-09-11T14:07:32,044796Z WorkloadManagement/JobAgent/WorkloadManagement/JobAgent ERROR: Exception occurred when submitting JobID: 4
Traceback (most recent call last):
  File "/ctadirac/3.0.0a5/Linux-x86_64/lib/python3.11/site-packages/DIRAC/WorkloadManagementSystem/Agent/JobAgent.py", line 647, in _submitJob
    result = self.computingElement.submitJob(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/ctadirac/3.0.0a5/Linux-x86_64/lib/python3.11/site-packages/DIRAC/Resources/Computing/InProcessComputingElement.py", line 41, in submitJob
    result = self.writeProxyToFile(proxy)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/ctadirac/3.0.0a5/Linux-x86_64/lib/python3.11/site-packages/DIRAC/Resources/Computing/ComputingElement.py", line 297, in writeProxyToFile
    result = getProxyInfoAsString(proxyLocation)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/ctadirac/3.0.0a5/Linux-x86_64/lib/python3.11/site-packages/DIRAC/Core/Security/ProxyInfo.py", line 87, in getProxyInfoAsString
    retVal = getProxyInfo(proxyLoc, disableVOMS)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/ctadirac/3.0.0a5/Linux-x86_64/lib/python3.11/site-packages/DIRAC/Core/Security/ProxyInfo.py", line 78, in getProxyInfo
    infoDict["hasDiracxToken"] = bool(diracxTokenFromPEM(proxyLocation))
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/ctadirac/3.0.0a5/Linux-x86_64/lib/python3.11/site-packages/DIRAC/Core/Security/DiracX.py", line 80, in diracxTokenFromPEM
    return json.loads(base64.b64decode(match).decode("utf-8"))
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd0 in position 991: invalid continuation byte

It seems that the problem arises when there is more than one block :

-----BEGIN DIRACX-----
-----END DIRACX-----

In this case the regular expression was matching everything between the first -----BEGIN DIRACX-----
and the last -----END DIRACX-----

With this fix the correct block between the first -----BEGIN DIRACX----- and the first -----END DIRACX----- is matched.

@arrabito arrabito force-pushed the fix_get_token branch 2 times, most recently from b2ffe70 to 75e027b Compare September 16, 2025 14:16
@chaen
Copy link
Contributor

chaen commented Sep 17, 2025

I am not sure that the regular expression needs fixing, your error is really strange.
Also, I fail to see a scenario where there would be multiple DIRACX extension in a PEM file. Do you know how that happened ?

@arrabito
Copy link
Contributor Author

@chaen in my tests the fix is needed.

I have to further check but it seems to me that the following sequence of calls ends up with a pem file with 2 tokens which causes the issue I've reported:

JobAgent._submitJob(proxyChain, …)
   ….
   # Pass proxy to the CE, writing it to a temporary file to ensure the DiracX token is included
    paylodProxy = writeChainToTemporaryFile(proxyChain)
    (here the first token is added to payloadProxy)
    InProcessComputingElement.submitJob(payloadProxy, ....)
     .... 
	  pem_file = writeProxyToFile(payloadProxy)
    (here the second token is added to the pem file)

@chaen
Copy link
Contributor

chaen commented Sep 24, 2025

@arrabito following private discussions, can you test #8312 ?

@chaen chaen mentioned this pull request Sep 24, 2025
Draft
@arrabito
Copy link
Contributor Author

@chaen I confirm that the fix works.
Just I've noticed that the code that removes any existing DiracX token:

        # Remove any existing DiracX token there would be
        new_pem = re.sub(RE_DIRACX_PEM, "", pem)
        new_pem += token_pem

        Path(pemPath).write_text(new_pem)

adds an empty line in the pem certificate, like:

..........
-----END CERTIFICATE-----

-----BEGIN DIRACX-----
...........

However it does not seem to create any problem.

@chaen
Copy link
Contributor

chaen commented Sep 25, 2025

I noticed that too and came to the same conclusion as you. Handling that empty line is surprisingly annoying :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@fstagni fstagni Awaiting requested review from fstagni fstagni is a code owner

@atsareg atsareg Awaiting requested review from atsareg atsareg is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@arrabito @chaen