Can't control the render order on GitHub Actions ubuntu-latest

[eculler]

Bug description

I am having this problem as well on GitHub Actions with the ubuntu-latest runner, despite:

• listing files individually in project/render and
• Numbering the files so that alphanumeric order should be the correct one

The render runs as expected on my system (MacOS Sonoma 14.5).

I need the notebooks to render in order, because they use jupyter storemagic to carry variables over from one to the next.

Steps to reproduce

My project has the following in _quarto.yml:

project:
  render:
    - pages/03-git-github/02-github-collaboration/02-pull-requests.qmd
    - notebooks/01-climate/climate-31.qmd
    - notebooks/01-climate/climate-32.qmd
    - notebooks/01-climate/climate-33.qmd
    - notebooks/01-climate/climate-34.qmd

Expected behavior

I expect the files to be run in the order they appear in the render list, or the order they appear in the table of contents, or in alphabetic order. As it is, the run order seems to be entirely unpredictable and depends on the system.

Actual behavior

the actual run order in my project on GitHub Actions ubuntu-latest runner is:
- notebooks/01-climate/climate-31.qmd
- pages/03-git-github/02-github-collaboration/02-pull-requests.qmd
- notebooks/01-climate/climate-33.qmd
- notebooks/01-climate/climate-32.qmd
- notebooks/01-climate/climate-34.qmd

Your environment

• GitHub Actions ubuntu-latest runner
• quarto installed with quarto-actions/setupv2 in an activated conda environment

Quarto check output

Run quarto check

Quarto 1.4.557
[✓] Checking versions of quarto binary dependencies...
Pandoc version 3.1.12: OK
Dart Sass version 1.58.3: OK
Deno version 1.37.2: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
Version: 1.4.557
Path: /usr/share/miniconda/envs/learning-portal/bin

[✓] Checking tools....................OK
TinyTeX: (not installed)
Chromium: (not installed)

[✓] Checking LaTeX....................OK
Tex: (not detected)
(|) Checking basic markdown render....
[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
Version: 3.10.14 (Conda)
Path: /usr/share/miniconda/envs/learning-portal/bin/python
Jupyter: 5.7.2
Kernels: ir, python3
(|) Checking Jupyter engine render....
[✓] Checking Jupyter engine render....OK
(|) Checking R installation...........
[✓] Checking R installation...........OK
Version: 4.3.3
Path: /usr/share/miniconda/envs/learning-portal/lib/R
LibPaths:
- /usr/share/miniconda/envs/learning-portal/lib/R/library
knitr: 1.48
rmarkdown: 2.27
(|) Checking Knitr engine render......
[✓] Checking Knitr engine render......OK

[mcanouil]

• Control order of rendering #6944

Tracked:

• Epic: Execution Planner #6518

[allefeld]

This is a duplicate of #10490.

@mcanouil: Yes, the official support will come with the epic execution planner, but previously listing files explicitly in project / render was a workaround. This workaround now stopped working.

[mcanouil]

I know but a workaround is a workaround.
There was/is/will be no guarantee with workarounds.

Relying on workarounds is never something to be done.

Edit: as there is a brief mention in the documentation, it's possible that the "workaround" was at some some point considered a feature which would make this a regression from previous versions.

To be clear #10490 is the duplicate not the
Other way around.

[allefeld]

The problem is that I need to be able to control the order for an existing project, and the execution planner is still in the medium to far future.

Is it a legitimate solution in the current situation (to avoid the term "workaround") to render from a shell script, where the script calls quarto render <file> for each file in sequence?
Does quarto render for a project do something different than quarto render <file> for all the render targets in the project?

[mcanouil]

To be clear, I never said it's not issue or not a bug.
I am only saying that all this comes in a more global implementation.
Now, whether or not it's a bug, that the "documented workaround" stopped working, I cannot say... thus the fact that the issue is opened and not assigned to anyone or any milestone.

There are no arguing about the need to be able to have ordered rendering.

[allefeld]

I understand.

Could you please comment on my approach to control rendering from a shell script? Should that work?

[cscheid]

Just to repeat from a slightly more official source, Quarto doesn't promise control of rendering order. The robust solution is to use a pre-render script to produce all necessary inputs before the files are rendered. We understand that this isn't ideal, but we truly can't fix this right now given the rest of our workload.

[allefeld]

@cscheid, I'm not asking you to completely overturn your work plan or take on additional work, but for a bit of practical advice for a concrete problem I need to solve.

Regarding producing the necessary inputs before – the thing is that the code which creates those inputs is in the form of qmd files, which can't be run from pre-render scripts (I think?).

What I'm currently doing is not have a pre-render script called from Quarto, but a render script calling Quarto, for all qmd files separately in order. So instead of

quarto render <folder>

I do

quarto render <folder>/<file1.qmd>
quarto render <folder>/<file2.qmd>
…

That seems to be working okay, except that I don't get sidebar contents in the main index.md rendering, but that may be an unrelated issue.

Could you please comment whether that is a reasonable approach, or whether there are any drawbacks to it?

[cscheid]

That seems to be working okay, except that I don't get sidebar contents in the main index.md rendering, but that may be an unrelated issue.

It's hard to have a discussion without a reprex around it. Can you create one first?

[allefeld]

I'll investigate the sidebar issue, and if it points to a problem with Quarto I'll create an issue incl. reprex.

But my question is about the general behavior of quarto render. Say I have a Quarto project with the following structure:

folder/
folder/quarto.yml
folder/index.md
folder/file1.qmd
folder/file2.qmd

can I expect the result of

quarto render folder/index.md
quarto render folder/file1.qmd
quarto render folder/file2.qmd

to be identical to the result of

quarto render folder

(apart of course from enforcing order)?

If yes, I then I have a usable workaround to ensure order of rendering.

[willgryan]

Just to repeat from a slightly more official source, Quarto doesn't promise control of rendering order. The robust solution is to use a pre-render script to produce all necessary inputs before the files are rendered. We understand that this isn't ideal, but we truly can't fix this right now given the rest of our workload.

A report generation system that can't control the order of the report it renders?

[mcanouil]

@willgryan what is the purpose of your comment exactly as it brings nothing to the discussion?! Please, be mindful in your comments.

Warning

As a member of the Quarto community, you are expected to follow our Code of Conduct, which outlines the standards of behavior and communication that we value. The Code of Conduct applies to all online interactions within the Quarto project, and it is enforced by the Quarto team. By participating in the Quarto community, you agree to abide by the Code of Conduct and respect the rights and dignity of others. You can find the Code of Conduct at https://github.com/quarto-dev/quarto-cli?tab=coc-ov-file#readme.

Thank you for your cooperation and contribution. 🙏

[willgryan]

@willgryan what is the purpose of your comment exactly as it brings nothing to the discussion?! Please, be mindful in your comments.

Warning

As a member of the Quarto community, you are expected to follow our Code of Conduct, which outlines the standards of behavior and communication that we value. The Code of Conduct applies to all online interactions within the Quarto project, and it is enforced by the Quarto team. By participating in the Quarto community, you agree to abide by the Code of Conduct and respect the rights and dignity of others. You can find the Code of Conduct at https://github.com/quarto-dev/quarto-cli?tab=coc-ov-file#readme.

Thank you for your cooperation and contribution. 🙏

Hi 🙂 Thanks for the prompt reply now on this pressing issue! To be more clear, I'm asking what is the meaning of promising control of rendering order? How is rendering order currently determined? In what way can rendering order be controlled?

The documentation still lists this as the method to control rendering order, and doesn't indicate it was ever a work-around. In fact, it is considered part of "Project Basics."

(Note: I decline the Warning, as it was given imprudently and hastily based on contextual misunderstanding, evidenced in later proceedings of these comments)

[AliSajid]

I want to emphasize that the precise ordering is not something we have thought about in a vacuum. It is something found on the official website.

This is a link to the section. The relevant section is paragraph after the numbered list. Here is the archived link. I'm also copying the relevant section with emphasis mine.

If you don’t want to render all of the target documents in a project, or you wish to control the order of rendering more precisely, you can add a project: render: [files] entry to your project metadata

@mcanouil

I know but a workaround is a workaround. There was/is/will be no guarantee with workarounds.

Relying on workarounds is never something to be done.

Edit: as there is a brief mention in the documentation, it's possible that the "workaround" was at some some point considered a feature which would make this a regression from previous versions.

I'm sorry, but this comes across as disingenuous. Even if there were an offhand mention in the documentation, the @quarto-dev team would be responsible for it. However, this is an explicit declaration of a way to use quarto.

I know but a workaround is a workaround.
There was/is/will be no guarantee with workarounds.

This, again, is disingenuous.

If we cannot trust official documentation on the project's website, what should we do? This feature has been broken for quite some time since the first release of v1.5.0 was in January. The documentation should have been updated to reflect that.

I ran into this problem last week and I have lost valuable time trying to figure out what went wrong.

@cscheid I would like to ask which official source are you quoting, please.

Just to repeat from a slightly more official source, Quarto doesn't promise control of rendering order. The robust solution is to use a pre-render script to produce all necessary inputs before the files are rendered. We understand that this isn't ideal, but we truly can't fix this right now given the rest of our workload.

[mcanouil]

@AliSajid when quoting, please try to avoid quoting twice the same thing while making a judgment and don't ignore the exact thing you are saying, i.e., that it was in the documentation.

Edit: as there is a brief mention in the documentation, it's possible that the "workaround" was at some some point considered a feature which would make this a regression from previous versions.

Clearly this considers it might be a "regression" thus a bug.

At no point, the request to have the ability to control render order was considered a weird idea or an invalid use case or this issue would not even be open anymore.

I would kindly ask you to avoid making judgments and assumptions here as it does not serve any positive purpose. Thank you!

[AliSajid]

Thank you for acknowledging that this is a bug. I'll go ahead and open a separate bug report.

[mcanouil]

Why do you want to open a new issue?

I acknowledged that this "might" be a bug. Only the developers know what they intended or not.

[willgryan]

Render target control is a documented feature for over 2 years: quarto-dev/quarto-web@936e9b9

[cderv]

Thanks @AliSajid for your input.

I suggest we work on this problem based on your new issue containing clear instructions on how to reproduce and observe the regression. We'll track down what happens with regard to this render list specification order. Then, we'll be able to tell more and take action.

[cscheid]

(Note: I decline the Warning, as it was given imprudently and hastily based on contextual misunderstanding, evidenced in later proceedings of these comments)

@willgryan I'm the main maintainer here, and I'm responsible for both enforcing the code of conduct and delegating the responsibility for doing so. "Declining a warning" is not a thing, and @mcanouil's warning was entirely justified based on your comments.

Let me be clear: participation on these forums is a privilege. Do read the code of conduct and adjust your behavior accordingly in future interactions.

[cscheid]

This is closed by #11062 and will be in 1.5 and 1.6 releases soon.

[allefeld]

Brilliant, thanks!