Unit test optimization toggles from LazyFrame #405
Conversation
|
Currently the unit test is doing nothing because there is no way to extract optimization toggles after setting them in functions like |
|
Hi @Sicheng-Pan thx for looking into this :) how about setting optstate on a lazyframe with internal functions and then check if actually set with this new getter ? |
@sorhawell Sounds good to me. I've already exported this getter function to R and I will write an internal setter as well. May be we could also make the setter public and allow the users to keep existing optimization settings in functions like |
|
How about having some global option defining if optstate should be set via an external setter or vi collect etc? Then the option could be used in collect. We should let default behavior match py-polars when there is no clear need otherwise. Some edge casesnto consider: What if two lazyframes are joined, what shouldhappen to optstate? How does collect know if a lazyframe settings should not be altered, because a user has already set them? |
tests/testthat/test-lazy.R
Outdated
| @@ -766,3 +766,18 @@ test_that("unnest", { | |||
| to_data_frame() | |||
| ) | |||
| }) | |||
|
|
|||
| test_that("opt_toggles", { | |||
| lf = pl$LazyFrame(mtcars)$select(pl$col("mpg") * 0.42) | |||
There was a problem hiding this comment.
try add some more unit tests for sink_parquet etc to confirm no errors
There was a problem hiding this comment.
I've added a few more tests.
R/lazyframe__lazy.R
Outdated
| self |> | ||
| .pr$LazyFrame$optimization_toggle( | ||
| if (isTRUE(inherit_optimization)) { | ||
| self |
There was a problem hiding this comment.
this self is not being fetched
R/lazyframe__lazy.R
Outdated
| if (isTRUE(inherit_optimization)) { | ||
| self | ||
| } else { | ||
| self$set_optimization_toggle( |
There was a problem hiding this comment.
any error in set_optimization_toggle will be raised and escaped the final unwrap
There was a problem hiding this comment.
consider using .pr$LazyFrame$set_opt... or wrap entire block in result() to catch any errors
There was a problem hiding this comment.
thx I will review it soon
#325
Update nix flake and write the unit test for LazyFrame optimization toggles.