[help] tar_quarto_rep() with multiple workers with crew::crew_controller_local
#230
Replies: 1 comment 2 replies
-
|
I can reproduce this with just Quarto. Report: If I run 2 render commands concurrently: quarto render report.qmd --output 1.pdf -P value=1
quarto render report.qmd --output 2.pdf -P value=2one of them errors: [typst]: Compiling report.typ to report.pdf...error: file not found (searched at reprex/report.typ)Seems this is a known issue in Quarto: quarto-dev/quarto-cli#4730 |
Beta Was this translation helpful? Give feedback.
2 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Help
Description
Hi all
I'm trying to use
tar_quarto_rep()to produce parameterised reports. I have multiple thousands of reports to produce, so would like to run in parallel.My experiments have found that using
batches = 1will run without causing an error, but runs linearly through the reports on a single worker.Leaving
batchesas its default appears to create a batch for each individual output. This almost always seems to create an error if you are using multiple workers withcrew_controller_local.Am I misunderstanding something? My gut says this is something about the system refusing to use a
.qmdfile that is already being used to avoid unintentionally giving an output file with content forxthe name fory.Below is a reprex to show what I mean.
Adding
batches = 1totar_quarto_rep()enables it to run through.(I just updated the three packages and it is complaining about R 4.5.0 vs 4.5.1 - that's all the warnings are. I can't imagine that will affect anything, but I'm not removing them from the output just in case)
My current workaround is to create multiple separate targets using subsets of the parameter data frames, and a separate target that updates the reference qmds for each of them whenever the base-qmd (or its child documents) changes.
library(targets)Created on 2025-11-23 with reprex v2.1.1
Beta Was this translation helpful? Give feedback.
All reactions