Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql escape for 'non-translated' function in window context #1526

Open
talegari opened this issue Jul 27, 2024 · 0 comments
Open

sql escape for 'non-translated' function in window context #1526

talegari opened this issue Jul 27, 2024 · 0 comments

Comments

@talegari
Copy link

In the context of the following issue: #1527

Something like this does not work at the moment (generated sql does ignores window options) for spark sql backend:

dbplyr::lazy_frame(rep_df) |> 
  group_by(user_id) %>%
  dbplyr::window_order(dates) %>%
  dbplyr::window_frame(-Inf, -1) %>%
  mutate(list_amount = sql("collect_list(amount)")) %>% 
  dbplyr::sql_render()

In general, would it be better to generate the sql one would expect by replacing whatever that it there in the sql call.

In this case, it is:

SELECT
`df`.*,
collect_list(`amount`) OVER (PARTITION BY `user_id` ORDER BY `dates` ROWS BETWEEN UNBOUNDED PRECEDING AND 1 PRECEDING) AS `list_amount`
FROM `df`

Why is this required: There is no way to generate this sql chunk via dbplyr and thereby breaks one's workflow. Its practically impossible to cover all "translations" that some backend offers. Would it make sense to create a meaningful "escape hatch"?

Personally, for a serious dbplyr user like me, I will be forced to switch to some other tool say pyspark (which I do not want to) for day-to-day work or do some monkey patching with sdf_sql with handwritten sql (I choose dbplyr for convenience and elegance).

@DavisVaughan DavisVaughan transferred this issue from tidyverse/dplyr Jul 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant