Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xrootd connections through dask not working #1343

Open
sihyunjeon opened this issue Nov 22, 2024 · 0 comments
Open

xrootd connections through dask not working #1343

sihyunjeon opened this issue Nov 22, 2024 · 0 comments
Labels
bug (unverified) The problem described would be a bug, but needs to be triaged

Comments

@sihyunjeon
Copy link

sihyunjeon commented Nov 22, 2024

Hello,

I am using uproot through CMS purdue analysis facility (uproot is 5.5.0).

I am facing errors when trying to get access to files through xrootd but only through when it's through uproot.dask.
Below are 4 test cases i had and only the 2nd case fails, which I understood that xcache was that allows some caching of accessed data through xrootd. The ones without xcache is using global redirector to access files.

Based on the fact that everything else worked fine

  1. I don't think it's my proxy problem (all other tests should've failed as well)
  2. Unlike the error message I pasted below suggests, I don't think it's different environment issue (otherwise dask with global redirector should've failed).

Error during deserialization of the task graph. This frequently occurs if the Scheduler and Client have different environments. For more information, see https://docs.dask.org/en/stable/deployment-considerations.html#consistent-software-environments

import uproot
try: 
    test = uproot.dask("root://cms-xrd-global.cern.ch//store/mc/Run3Summer23NanoAODv12/QCD-4Jets_HT-400to600_TuneCP5_13p6TeV_madgraphMLM-pythia8/NANOAODSIM/130X_mcRun3_2023_realistic_v14-v3/60000/c5b8293b-65fd-4a14-a25b-2073f9bf6ac7.root:Events")
    print (test["run"].compute())
except Exception as e:
    print ("test1 failed", e)
try: 
    test = uproot.dask("root://cms-xcache.rcac.purdue.edu:1094//store/mc/Run3Summer23NanoAODv12/QCD-4Jets_HT-400to600_TuneCP5_13p6TeV_madgraphMLM-pythia8/NANOAODSIM/130X_mcRun3_2023_realistic_v14-v3/60000/c5b8293b-65fd-4a14-a25b-2073f9bf6ac7.root:Events")
    print (test["run"].compute())
except Exception as e:
    print ("test2 failed", e)
try: 
    test = uproot.open("root://cms-xrd-global.cern.ch//store/mc/Run3Summer23NanoAODv12/QCD-4Jets_HT-400to600_TuneCP5_13p6TeV_madgraphMLM-pythia8/NANOAODSIM/130X_mcRun3_2023_realistic_v14-v3/60000/c5b8293b-65fd-4a14-a25b-2073f9bf6ac7.root")["Events"]
    print (test["run"].array())
except Exception as e:
    print ("test3 failed", e)
try: 
    test = uproot.open("root://cms-xcache.rcac.purdue.edu:1094//store/mc/Run3Summer23NanoAODv12/QCD-4Jets_HT-400to600_TuneCP5_13p6TeV_madgraphMLM-pythia8/NANOAODSIM/130X_mcRun3_2023_realistic_v14-v3/60000/c5b8293b-65fd-4a14-a25b-2073f9bf6ac7.root")["Events"]
    print (test["run"].array())
except Exception as e:
    print ("test4 failed", e)

As I wrote above, only the 2nd case fails when it tries to use dask and access through xcache. The other three cases, dask with global redirector, directly opening xcache, directly opening global redirector, all works fine

It will be great if you can provide some help with this.

@sihyunjeon sihyunjeon added the bug (unverified) The problem described would be a bug, but needs to be triaged label Nov 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug (unverified) The problem described would be a bug, but needs to be triaged
Projects
None yet
Development

No branches or pull requests

1 participant