[ntpl] Allow XRootD redirection with EOS files

[vepadulano]

Fixes #20213

With these changes I can see the redirection happening when reading an RNTuple on EOS from lxplus (going from ntpl001_staff.root to root://eoshome-v.cern.ch//eos/user/v/vpadulan/ntpl001_staff.root

Breakpoint 1, ROOT::Internal::RRawFile::Create (url=..., options=...) at /tmp/vpadulan/root/io/io/src/RRawFile.cxx:88       
88         std::string transport = GetTransport(url);                                                                       
(gdb) n  
89         if (transport == "file") {
(gdb) p url                                                                                                                 
$1 = {_M_len = 18, _M_str = 0x4030a3 "ntpl001_staff.root"}  
[...]
103                    if (auto baseName = fileNameFromUrl.substr(fileNameFromUrl.find_last_of("/") + 1);
(gdb) 
105                       return Create(xurl, options);
(gdb) s
[...]
Breakpoint 1, ROOT::Internal::RRawFile::Create (url=..., options=...) at /tmp/vpadulan/root/io/io/src/RRawFile.cxx:88       
88         std::string transport = GetTransport(url);                                                                       
(gdb) n                                                                                                                     
89         if (transport == "file") {                                                                                       
(gdb) 
113        if (transport == "http" || transport == "https" ||
(gdb) p url
$8 = {_M_len = 64, _M_str = 0x136a790 "root://eoshome-v.cern.ch//eos/user/v/vpadulan/ntpl001_staff.root"}
(gdb) p transport
$9 = {static npos = 18446744073709551615, 
  _M_dataplus = {<std::allocator<char>> = {<std::__new_allocator<char>> = {<No data fields>}, <No data fields>}, 
    _M_p = 0x7fffffffa230 "root"}, _M_string_length = 4, {
    _M_local_buf = "root\000\177\000\000\220\242\377\377\377\177\000", _M_allocated_capacity = 139639930187634}}
(gdb) 

There might be more places where the redirection needs to happen though.

[jblomer]

Thanks for addressing this!

[github-actions]

Test Results

    23 files      23 suites   3d 13h 13m 8s ⏱️
 3 872 tests  3 848 ✅ 0 💤 24 ❌
78 787 runs  78 763 ✅ 0 💤 24 ❌

For more details on these failures, see this check.

Results for commit d447fee.

♻️ This comment has been updated with latest results.

[vepadulano]

To the best of my understanding, this PR is simply moving the code that used to live separately in TFile::Open and RLoopManager::ChangeSpec extracting it into a common place. I have been testing these changes on EOS. Both TFile::Open and RNTupleReader::Open work flawlessly. For some reason, with the redirection happening in RDataFrame to store also the redirected URL as a sample ID, I see this error

terminate called after throwing an instance of 'std::runtime_error'
  what():  Full sample identifier 'root://eoshome-v.cern.ch//eos/user/v/vpadulan/rdf_sample.root/events' cannot be found in the available samples.

which is the same that triggered #17544 . What's even more worrying is that I don't see the error on every test execution, but sometimes the test also runs without failing. Before merging this PR I need to understand why that happens. It's possible the problem is still there somehow also without the code changes of this PR.

[jblomer]

In principle looks good to me! We should think about how to test it. Extended attributes are file system dependent, so we may need to add some code just for testing, e.g. something that lets us control what GetXAttrVal returns under unit tests.

[vepadulano]

Regarding the issues with the eos.url.xroot attribute being wrong:

• Wrong URL returned for eos.url.xroot attribute cern-eos/eos#41 describes the issue I remembered about the base name sometimes not being populated. I remember this still happening last year, can't say for sure at this point, but at some point it was there.
• URL for eos.url.xrot attribute wrongfully injects double forward slash in directory name cern-eos/eos#50 describes a new regression that introduces double forward slash in the XRootD URL, thus breaking all versions of ROOT relying on this redirection

I would wait for the latter issue to also be confirmed fix before moving on with this PR, FYI @amadio.

[jblomer]

@vepadulano it looks like this PR can advance

[vepadulano]

The two EOS issues I had mentioned previously seem to be fixed, although I am now looking at another failure that is happening. This is the ROOT reproducer

#include <ROOT/RDataFrame.hxx>
#include <ROOT/RNTupleReader.hxx>
#include <ROOT/RSnapshotOptions.hxx>

void write(const char *dataset, const char *path, const char *type) {
  ROOT::RDataFrame df{5};
  ROOT::RDF::RSnapshotOptions opts;
  if (strcmp(type, "ttree") == 0)
    opts.fOutputFormat = ROOT::RDF::ESnapshotOutputFormat::kTTree;
  else
    opts.fOutputFormat = ROOT::RDF::ESnapshotOutputFormat::kRNTuple;
  df.Define("x", []() { return 42; }).Snapshot(dataset, path, "", opts);
}

void test_rntuplereader_open() {
  write("events", "anotherfile.root", "rntuple");
  write("events", "rntuplereader_myfile.root", "rntuple");
  auto r = ROOT::RNTupleReader::Open("events", "rntuplereader_myfile.root");
}

int main() {
  test_rntuplereader_open();
}

The logic is fairly simple:

• Write a file named "anotherfile.root" with an RNTuple inside called "events"
• Write a file named "rntuplereader_myfile.root" with an RNTuple inside called "events"
• Open the second file with RNTupleReader

It looks like writing two files one after another is causing initially some corruption in the file being read, then on subsequent runs of the same application shows an error message about not having permission to write to the first file

[vpadulan@lxplus9104 /eos/user/v/vpadulan/xrootd_redirection_tests]$./xrootd_redirection_rntuple.out 
terminate called after throwing an instance of 'ROOT::RException'
  what():  invalid read (expected bytes: 4, read: 0)
At:
  ROOT::Internal::RMiniFileReader::TryReadBuffer(void*, size_t, uint64_t)::<lambda(std::size_t, std::size_t)> [/tmp/vpadulan/rootproject/root/tree/ntuple/src/RMiniFile.cxx:895]

Aborted (core dumped)
[vpadulan@lxplus9104 /eos/user/v/vpadulan/xrootd_redirection_tests]$./xrootd_redirection_rntuple.out 
terminate called after throwing an instance of 'ROOT::RException'
  what():  The file 'root://eoshome-v.cern.ch//eos/user/v/vpadulan/xrootd_redirection_tests/anotherfile.root' given to RNTupleParallelWriter is not writable. Open it with 'UPDATE' or 'RECREATE' if you want to write into it.
At:
  static std::unique_ptr<ROOT::RNTupleParallelWriter> ROOT::RNTupleParallelWriter::Append(std::unique_ptr<ROOT::RNTupleModel>, std::string_view, TDirectory&, const ROOT::RNTupleWriteOptions&) [/tmp/vpadulan/rootproject/root/tree/ntuple/src/RNTupleParallelWriter.cxx:196]

Aborted (core dumped)
[vpadulan@lxplus9104 /eos/user/v/vpadulan/xrootd_redirection_tests]$./xrootd_redirection_rntuple.out 
terminate called after throwing an instance of 'ROOT::RException'
  what():  The file 'root://eoshome-v.cern.ch//eos/user/v/vpadulan/xrootd_redirection_tests/anotherfile.root' given to RNTupleParallelWriter is not writable. Open it with 'UPDATE' or 'RECREATE' if you want to write into it.
At:
  static std::unique_ptr<ROOT::RNTupleParallelWriter> ROOT::RNTupleParallelWriter::Append(std::unique_ptr<ROOT::RNTupleModel>, std::string_view, TDirectory&, const ROOT::RNTupleWriteOptions&) [/tmp/vpadulan/rootproject/root/tree/ntuple/src/RNTupleParallelWriter.cxx:196]

Aborted (core dumped)

I am at the moment trying to see if I can create an EOS-only reproducer without ROOT.

[vepadulano]

The error from the parallel writer about the file not being writable is due to TNetXNGFile not properly sanitising the input mode option "RECREATE_WITHOUT_GLOBALREGISTRATION" which is then parsed to the appropriate XRootD file open mode. Since it's not recognised, the file is opened in read only mode

root/net/netxng/src/TNetXNGFile.cxx

Line 180 in 63097c0

if (ParseOpenMode(mode, fOption, fMode, kTRUE)<0) {

. The patch is straightforward in this case.

The other issue about the invalid read from RMiniFileReader is still unsolved.

[vepadulano]

The invalid read is caused by not closing the output TFile in UntypedSnapshotRNTupleHelper::Finalize

root/tree/dataframe/src/RDFSnapshotHelpers.cxx

Line 967 in 63097c0

void ROOT::Internal::RDF::UntypedSnapshotRNTupleHelper::Finalize()

.