Replies: 1 comment 7 replies
-
|
I agree that there is room to make it easier to create tar_option_with(
tar_target(file, "data.csv"),
format = "file"
)
tar_option_with(
tar_target(data, read_csv(file, col_types = cols()) |> filter(!is.na(Ozone))),
deployment = "main"
)This could lead to a DSL like: #| format: file
#| deployment: main
file <- "data.csv"
#| deployment: main
data <- read_csv(file, col_types = cols()) |>
filter(!is.na(Ozone))
#| storage: worker
model <- lm(Ozone ~ Temp, data) |>
coefficients()
#| deployment: main
plot <- ggplot(data) +
geom_point(aes(x = Temp, y = Ozone)) +
geom_abline(intercept = model[1], slope = model[2]) +
theme_gray(24) |
Beta Was this translation helpful? Give feedback.
-
|
|
Beta Was this translation helpful? Give feedback.
-
Understood! Idly thinking about how, it, tar_target captured its call, modifying the call might be easier than modifying the object. Regarding the DSL:
|
Beta Was this translation helpful? Give feedback.
-
To keep it simple, I was thinking all top-level assignments would be interpreted as targets. As you say, a |
Beta Was this translation helpful? Give feedback.
-
|
The DSL is turning into ropensci/tarchetypes#226 (comment), which has a different set of goals. For modifying targets, I will add a new tar_option_set(format = "qs")
target_data <- tar_target(data, get_data())
tar_option_unset("format") # Sets `format` back to the default.
target_model <- tar_target(model, fit_model(data)) |
Beta Was this translation helpful? Give feedback.
-
|
Added in 029befe. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Help
Description
It would be useful to provide some mechanism for modifying targets in code, allowing a target to be defined, then changed within the pipeline definition. This would allow for separation of pipeline logic and storage / deployment settings. A basic proposal would be a
tar_modify()function that allows re-definition oftar_target()'s options, like this:I am thinking primarily of the idea of "nested reproducibility", where one provides simple and low-dependency code surrounded by layers guaranteeing increasing levels of reproducibility (e.g script < targets < renv < container). Thanks to
tarchetypes::tar_assign(), one can write a targets pipeline as generic, running R code, which can then be put into a targets file, like so:script.R, is fully functional R code:
_targets.R can then convert script.R into a pipeline (using the example code from #1309 (comment)):
However, what's lacking here is the ability to set target-specific options, such as branching, deployment, storage format, etc, separately from the script. In the example, "data.csv" is not defined as a "file"-type target. Putting
file <- tar_modify(file, format = "file")in the_targets.Rfile would enable this flow.The applications are a bit broader than this specific approach. In my own work, it would also enable separation of conditional logic I have around cues, deployment, etc which are controlled by environmental variables, from the pipeline logic, making for a cleaner and more interpretable
_targets.Rfile.Beta Was this translation helpful? Give feedback.
All reactions