Windows Error in GHA

[hmgaudecker]

Finally getting close to getting the econ-project-templates run smoothly on Windows, thanks to updated plotly/kaleido.

However, GHA fails with the error below ("ValueError: path is on mount 'C:', start on mount 'D:'"). I have no clue why that would occur, but I'd think it would need to be addressed on the pytask-latex side of things. Apologies for my nearly fully deprecated Windows-knowledge, if that is not the case!

──────────────────────────── Start pytask session ─────────────────────────────
Platform: win32 -- Python 3.13.2, pytask 0.5.2, pluggy 1.5.0
Root: D:\\a\\econ-project-templates\\econ-project-templates
Configuration: 
D:\\a\\econ-project-templates\\econ-project-templates\\pyproject.toml
Plugins: pytask_latex-0.4.2, pytask_parallel-0.5.0
Collected 11 tasks.

┌───────────────────────────────────────────────────────────────────┬─────────┐
│ Task                                                              │ Outcome │
├───────────────────────────────────────────────────────────────────┼─────────┤
│ task_analysis_template.py::task_fit_model                         │ .       │
│ task_analysis_template.py::task_predict[highest_qualification]    │ .       │
│ task_analysis_template.py::task_predict[marital_status]           │ .       │
│ task_data_management_template.py::task_clean_stats4schools_smokin │ .       │
│ g_data                                                            │         │
│ task_documents.py::task_compile_document[paper]                   │ F       │
│ task_documents.py::task_compile_document[presentation]            │ F       │
│ task_final_template.py::task_create_results_table                 │ .       │
│ task_final_template.py::task_plot_results_by_age[highest_qualific │ .       │
│ ation]                                                            │         │
│ task_final_template.py::task_plot_results_by_age[marital_status]  │ .       │
└───────────────────────────────────────────────────────────────────┴─────────┘

────────────────────────────────── Failures ───────────────────────────────────

───────── Task task_documents.py::task_compile_document[paper] failed ─────────

┌───────────────────── Traceback (most recent call last) ─────────────────────┐
│ D:\\a\\econ-project-templates\\econ-project-templates\\.pixi\\envs\\default\\Lib\\s │
│ ite-packages\\pytask_latex\\collect.py:76 in compile_latex_document           │
│                                                                             │
│    73 │   """                                                               │
│    74 │   try:                                                              │
│    75 │   │   for step in _compilation_steps:                               │
│ >  76 │   │   │   step(path_to_tex=_path_to_tex, path_to_document=_path_to_ │
│    77 │   except CalledProcessError as e:                                   │
│    78 │   │   msg = f"Compilation step {step.__name__} failed."             │
│    79 │   │   raise RuntimeError(msg) from e                                │
│                                                                             │
│ D:\\a\\econ-project-templates\\econ-project-templates\\.pixi\\envs\\default\\Lib\\s │
│ ite-packages\\pytask_latex\\compilation_steps.py:41 in run_latexmk            │
│                                                                             │
│   38 │   def run_latexmk(path_to_tex: Path, path_to_document: Path) -> None │
│   39 │   │   job_name_opt = [f"--jobname={path_to_document.stem}"]          │
│   40 │   │   out_dir_opt = [                                                │
│ > 41 │   │   │   f"--output-directory={relative_to(path_to_tex, path_to_doc │
│    42 │   │   ]                                                              │
│    43 │   │   cmd = ["latexmk", *options, *job_name_opt, *out_dir_opt, path_ │
│    44 │   │   subprocess.run(cmd, check=True)  # noqa: S603                  │
│                                                                             │
│ D:\\a\\econ-project-templates\\econ-project-templates\\.pixi\\envs\\default\\Lib\\s │
│ ite-packages\\pytask_latex\\path.py:36 in relative_to                         │
│                                                                             │
│   33 │   '../../bld/docs'                                                   │
│   34 │                                                                      │
│   35 │   """                                                                │
│ > 36 │   return Path(os.path.relpath(relative_to_, path.parent)).as_posix() │
│    37 │                                                                     │
│ in relpath:796                                                              │
└─────────────────────────────────────────────────────────────────────────────┘
ValueError: path is on mount 'C:', start on mount 'D:'

───── Task task_documents.py::task_compile_document[presentation] failed ──────

┌───────────────────── Traceback (most recent call last) ─────────────────────┐
│ D:\\a\\econ-project-templates\\econ-project-templates\\.pixi\\envs\\default\\Lib\\s │
│ ite-packages\\pytask_latex\\collect.py:76 in compile_latex_document           │
│                                                                             │
│    73 │   """                                                               │
│    74 │   try:                                                              │
│    75 │   │   for step in _compilation_steps:                               │
│ >  76 │   │   │   step(path_to_tex=_path_to_tex, path_to_document=_path_to_ │
│    77 │   except CalledProcessError as e:                                   │
│    78 │   │   msg = f"Compilation step {step.__name__} failed."             │
│    79 │   │   raise RuntimeError(msg) from e                                │
│                                                                             │
│ D:\\a\\econ-project-templates\\econ-project-templates\\.pixi\\envs\\default\\Lib\\s │
│ ite-packages\\pytask_latex\\compilation_steps.py:41 in run_latexmk            │
│                                                                             │
│   38 │   def run_latexmk(path_to_tex: Path, path_to_document: Path) -> None │
│   39 │   │   job_name_opt = [f"--jobname={path_to_document.stem}"]          │
│   40 │   │   out_dir_opt = [                                                │
│ > 41 │   │   │   f"--output-directory={relative_to(path_to_tex, path_to_doc │
│    42 │   │   ]                                                              │
│    43 │   │   cmd = ["latexmk", *options, *job_name_opt, *out_dir_opt, path_ │
│    44 │   │   subprocess.run(cmd, check=True)  # noqa: S603                  │
│                                                                             │
│ D:\\a\\econ-project-templates\\econ-project-templates\\.pixi\\envs\\default\\Lib\\s │
│ ite-packages\\pytask_latex\\path.py:36 in relative_to                         │
│                                                                             │
│   33 │   '../../bld/docs'                                                   │
│   34 │                                                                      │
│   35 │   """                                                                │
│ > 36 │   return Path(os.path.relpath(relative_to_, path.parent)).as_posix() │
│    37 │                                                                     │
│ in relpath:796                                                              │
└─────────────────────────────────────────────────────────────────────────────┘
ValueError: path is on mount 'C:', start on mount 'D:'

───────────────────────────────────────────────────────────────────────────────
┌─────────────────── Summary ────────────────────┐
│  11  Collected tasks                           │
│  7   Succeeded                        (63.6%)  │
│  2   Skipped because previous failed  (18.2%)  │
│  2   Failed                           (18.2%)  │
└────────────────────────────────────────────────┘
─────────────────────────── Failed in 8.31 seconds ────────────────────────────

[tobiasraabe]

I believe the function relative_to and in there os.path.relpath cannot handle the case if the second path is also absolute. When both paths are from different mounts, the function fails. I hope a fix would be as simple as checking whether the second path is absolute. If so, just take it and don't compute a relative path. Could you try that?

[hmgaudecker]

This is weird... I made all paths absolute in the config at some point because there were errors otherwise -- no idea when then was though.

SRC = Path(__file__).parent.resolve()
ROOT = SRC.joinpath("..", "..").resolve()

BLD = ROOT.joinpath("bld").resolve()

DOCUMENTS = ROOT.joinpath("documents").resolve()

which is what I am using for pytask-latex:

@pytask.mark.latex(
    script=DOCUMENTS / f"{document}.tex",
    document=BLD / "documents" / f"{document}.pdf",
    compilation_steps=cs.latexmk(
        options=("--pdf", "--interaction=nonstopmode", "--synctex=1", "--cd"),
    ),
)
@pytask.task(id=document)
def task_compile_document():
    """Compile the document specified in the latex decorator."""

So you are saying I should do something like script=(DOCUMENTS / f"{document}.tex").relative_to(BLD / "documents") ? (apologies I only looked at the above output, not the pytask-latex source code, just in between things)

[tobiasraabe]

To me it looks like BLD and DOCUMENTS are paths on different mounts, C: and D:, and when relative_to tries to create a relative path between them, it fails.

I actually wanted to propose something really audacious like this

import pytask_latex.path


def new_relative_to(path: Path, relative_to_: Path) -> str:
    if relative_to_.is_absolute():
        return relative_to_.as_posix()
    return Path(os.path.relpath(relative_to_, path.parent)).as_posix()


pytask_latex.path.relative_to = new_relative_to

somewhere in your code.

I do not remember why I wanted the path for --output-directory always to be relative. Maybe I mixed it up with --jobname? I found this explanation here, which states that an absolute path for --output-directory is fine.

[hmgaudecker]

Thanks!

To me it looks like BLD and DOCUMENTS are paths on different mounts, C: and D:, and when relative_to tries to create a relative path between them, it fails.

That makes sense -- probably GHA separates the package being tested from anything that is written locally.

Unless I did something wrong in the above commit, the fix did not help, I am afraid (in between all the uni-codes, there still is ValueError: path is on mount 'C:', start on mount 'D:'...

[tobiasraabe]

If you or someone wants to take a stab at the issue, you are welcome. It should be as easy as creating a task on one mount and creating the document on another mount. I won't be able to tackle this issue before my upcoming vacation.

[hmgaudecker]

I am afraid the workflow of writing locally and then seeing whether GHA fails would be a bit cumbersome for me as I don't have access to a Windows box. But there is no rush at all, after all the Windows pipeline never ran so far on GHA due to Kaleido, so no change in behaviour. Enjoy your vacation!

[hmgaudecker]

Many thanks!!!! Works perfectly.