Generated .qmd file is ignored on first build

[BradyAJohnston]

Bug description

I am unsure whether this would be even the expected behaviour, so feel free to flag it as not.

I am generating documentation for a website. I have a .qmd file that is generated from a pre-render script that generates the file. As the file initially doesn't exist, quarto ignores it even if I explicitly list it in the contents.

When it is then run a second time, it will then include it. Even though it is regenerated, because it was there initially it picks it up and renders the newly generated one.

Steps to reproduce

A reproducible example is these three files:

_quarto.yml

project:
  type: website
  output-dir: _build
  pre-render: script.py

website:
  title: "Test"
  sidebar:
    contents:
      - index.qmd
      - script.qmd

index.qmd

# Testing!

This is some text.

Lots of text goes in here.

script.py

with open("script.qmd", "w") as file:
    file.write('# This is generated\n')
    file.write('By a python script, none the less!')

Running the quarto build or quarto render prints a message that it successfully runs the script.py, but despite the script.qmd now existing, it isn't included in the rendered website as it didn't exist when it was first generated.

Expected behavior

That the generated .qmd file would be included as it is listed directly in the _quarto.yml.

Actual behavior

It is not included if it doesn't exist when first calling quarto render.

Your environment

• M2 Max MacOS Sonoma 14.0

Quarto check output

quarto check
Quarto 1.4.449
[✓] Checking versions of quarto binary dependencies...
      Pandoc version 3.1.8: OK
      Dart Sass version 1.55.0: OK
      Deno version 1.33.4: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
      Version: 1.4.449
      Path: /Users/brady/.local/share/qvm/versions/v1.4.449/bin

[✓] Checking tools....................OK
      TinyTeX: v2023.09
      Chromium: (not installed)

[✓] Checking LaTeX....................OK
      Using: TinyTex
      Path: /Users/brady/Library/TinyTeX/bin/universal-darwin
      Version: 2023

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
      Version: 3.11.4 (Conda)
      Path: /opt/homebrew/Caskroom/miniconda/base/bin/python
      Jupyter: (None)

      Jupyter is not available in this Python installation.
      Install with conda install jupyter

[✓] Checking R installation...........OK
      Version: 4.3.1
      Path: /opt/homebrew/Cellar/r/4.3.1/lib/R
      LibPaths:
        - /opt/homebrew/lib/R/4.3/site-library
        - /opt/homebrew/Cellar/r/4.3.1/lib/R/library
      knitr: 1.44
      rmarkdown: 2.25

[✓] Checking Knitr engine render......OK

[mcanouil]

It's expected.
That's because (for now) Quarto retrieves the resources needed to render first, thus anything generated in pre-render is unknown to it.

This might change in 1.5.

• Epic: Execution Planner #6518

[BradyAJohnston]

Thanks for the clarification. I'll just ensure I run the render script before I call quarto and it'll all workout OK.

[mcanouil]

Note that once the file exists, each render should get the updated information, i.e., no need to render twice every time.
You can see this as an initialisation step.

[cscheid]

@BradyAJohnston This is an unfortunate consequences of how projects currently resolve files. One ugly hack that some folks have used in the past is to have your pre-render script fail if it created any new files (maybe with a message like "please run again"). Then, the next run can detect that no files need creating, and succeed without doing anything.

We understand that's pretty annoying and we'll fix this, but it won't happen for a while.

[BradyAJohnston]

Not so bad, currently it's just for documentation generated through GitHub actions, so I can just ensure the render script is called and generates the .qmd beforehand. Thanks for all the info and of course for all of the great work on Quarto.

[cscheid]

@dragonstyle This has been fixed by your recent changes, right? Do I have it correct that projects with a pre-render script that alters the render list will now see the correct render list?

[dragonstyle]

Correct- the 1.5 builds should be reloading the project input list and context after the pre-render script completes!