Etienne Bacher
@etiennebacher.bsky.social
PhD in economics from LISER, Luxembourg, now looking for research software engineer or data science positions. Mostly here to talk about #rstats
https://github.com/etiennebacher
https://github.com/etiennebacher
I think we're in the same boat. I want to promote the use of targets, but my colleagues dont necessarily want to spend the time learning it, which I can also understand. And even if I can use it in projects where I control the tech, it just makes it harder to share with others later if needed
November 6, 2025 at 7:44 PM
I think we're in the same boat. I want to promote the use of targets, but my colleagues dont necessarily want to spend the time learning it, which I can also understand. And even if I can use it in projects where I control the tech, it just makes it harder to share with others later if needed
Depends also on your colleagues I guess, it adds some complexity and not everyone wants to learn it
November 6, 2025 at 4:32 PM
Depends also on your colleagues I guess, it adds some complexity and not everyone wants to learn it
It can be useful to translate some code into another syntax. For instance, in tidypolars, I translate the standard R syntax to the polars syntax. The function translate() is called recursively to translate a function call, then the arguments of this call, etc.
github.com/etiennebache...
github.com/etiennebache...
tidypolars/R/utils-expr.R at main · etiennebacher/tidypolars
More efficient tidyverse code, using polars in the background - etiennebacher/tidypolars
github.com
November 6, 2025 at 12:43 PM
It can be useful to translate some code into another syntax. For instance, in tidypolars, I translate the standard R syntax to the polars syntax. The function translate() is called recursively to translate a function call, then the arguments of this call, etc.
github.com/etiennebache...
github.com/etiennebache...
For polars, I don't know how you measure the memory usage, but if you use the numbers reported by bench::mark() then those are wrong because R cannot measure the memory used by the Rust code that runs in the background
October 28, 2025 at 12:30 PM
For polars, I don't know how you measure the memory usage, but if you use the numbers reported by bench::mark() then those are wrong because R cannot measure the memory used by the Rust code that runs in the background
If you just want to please lintr, I'd use `# nolint: line_length_linter` on this line instead (or in the lintr config file)
October 23, 2025 at 7:46 AM
If you just want to please lintr, I'd use `# nolint: line_length_linter` on this line instead (or in the lintr config file)
Just FYI, I don't know if it's on purpose or not, but the date of the post is 28 August so it doesn't appear as the latest post on the blog homepage
October 20, 2025 at 4:21 PM
Just FYI, I don't know if it's on purpose or not, but the date of the post is 28 August so it doesn't appear as the latest post on the blog homepage
Just FYI, I reported this and they said this will be fixed: github.com/posit-dev/po...
Assistant Lands in Console · posit-dev positron · Discussion #9275
Assistant and Console are two of Positron's defining features, and with the 2025.09 release they integrate even deeper to streamline your coding experience. Assistant is now ready to help with any ...
github.com
October 12, 2025 at 2:58 PM
Just FYI, I reported this and they said this will be fixed: github.com/posit-dev/po...
You may be interested in constructive: cynkra.github.io/constructive/
Same objective but produces more readable code
Same objective but produces more readable code
Display Idiomatic Code to Construct Most R Objects
Prints code that can be used to recreate R objects. In a sense it is similar to base::dput() or base::deparse() but constructive strives to use idiomatic constructors.
cynkra.github.io
October 11, 2025 at 7:51 AM
You may be interested in constructive: cynkra.github.io/constructive/
Same objective but produces more readable code
Same objective but produces more readable code
I might have something for you in a few weeks
October 7, 2025 at 6:00 PM
I might have something for you in a few weeks
I remember seeing {debugme} being used in several r-lib packages as well (I've never used it however):
github.com/r-lib/debugme
github.com/r-lib/debugme
GitHub - r-lib/debugme: Easy and efficient debugging for R packages
Easy and efficient debugging for R packages. Contribute to r-lib/debugme development by creating an account on GitHub.
github.com
September 25, 2025 at 5:37 AM
I remember seeing {debugme} being used in several r-lib packages as well (I've never used it however):
github.com/r-lib/debugme
github.com/r-lib/debugme
Other features include:
* improved support for lubridate functions
* experimental support for fallback to R when a function cannot be
translated to polars
* performance improvements when handling thousands of columns
and more.
Full changelog: www.tidypolars.etiennebacher.com/news/
(2/2)
* improved support for lubridate functions
* experimental support for fallback to R when a function cannot be
translated to polars
* performance improvements when handling thousands of columns
and more.
Full changelog: www.tidypolars.etiennebacher.com/news/
(2/2)
Changelog
www.tidypolars.etiennebacher.com
July 24, 2025 at 6:27 AM
Other features include:
* improved support for lubridate functions
* experimental support for fallback to R when a function cannot be
translated to polars
* performance improvements when handling thousands of columns
and more.
Full changelog: www.tidypolars.etiennebacher.com/news/
(2/2)
* improved support for lubridate functions
* experimental support for fallback to R when a function cannot be
translated to polars
* performance improvements when handling thousands of columns
and more.
Full changelog: www.tidypolars.etiennebacher.com/news/
(2/2)
Yes, there's a learning curve for sure.
I have also developed tidypolars to keep using tidyverse code while using polars in the background: tidypolars.etiennebacher.com
This takes care of most dataframe-level functions, and there's a number of other functions supported.
I have also developed tidypolars to keep using tidyverse code while using polars in the background: tidypolars.etiennebacher.com
This takes care of most dataframe-level functions, and there's a number of other functions supported.
More Efficient Tidyverse Code, Using Polars in the Background
Polars is a cross-language tool for manipulating very large data. However, one drawback is that the R implementation has a syntax that will look odd to many R users who are not used to Python syntax. ...
tidypolars.etiennebacher.com
July 23, 2025 at 4:44 PM
Yes, there's a learning curve for sure.
I have also developed tidypolars to keep using tidyverse code while using polars in the background: tidypolars.etiennebacher.com
This takes care of most dataframe-level functions, and there's a number of other functions supported.
I have also developed tidypolars to keep using tidyverse code while using polars in the background: tidypolars.etiennebacher.com
This takes care of most dataframe-level functions, and there's a number of other functions supported.
Would love to but looks unlikely. The upstream polars library uses very recent Rust versions which are not available on CRAN machines, and they update the rust version they use quite frequently.
R-universe / R-multiverse is the best way to get polars binaries for now.
R-universe / R-multiverse is the best way to get polars binaries for now.
July 23, 2025 at 6:33 AM
Would love to but looks unlikely. The upstream polars library uses very recent Rust versions which are not available on CRAN machines, and they update the rust version they use quite frequently.
R-universe / R-multiverse is the best way to get polars binaries for now.
R-universe / R-multiverse is the best way to get polars binaries for now.
polars can be used via webR, thanks to the work of @gws.phd both on webR and on the Rust component of polars.
It also uses savvy, developed by @yutannihilation-en.bsky.social, to connect Rust and R.
It also uses savvy, developed by @yutannihilation-en.bsky.social, to connect Rust and R.
July 23, 2025 at 6:01 AM
polars can be used via webR, thanks to the work of @gws.phd both on webR and on the Rust component of polars.
It also uses savvy, developed by @yutannihilation-en.bsky.social, to connect Rust and R.
It also uses savvy, developed by @yutannihilation-en.bsky.social, to connect Rust and R.
polars 1.0.0 contains many breaking changes because internals have changed or because we better match the function and argument names provided in python polars.
We encourage you to refer to the python polars documentation if you use some functions or arguments that don't exist anymore.
We encourage you to refer to the python polars documentation if you use some functions or arguments that don't exist anymore.
July 23, 2025 at 6:01 AM
polars 1.0.0 contains many breaking changes because internals have changed or because we better match the function and argument names provided in python polars.
We encourage you to refer to the python polars documentation if you use some functions or arguments that don't exist anymore.
We encourage you to refer to the python polars documentation if you use some functions or arguments that don't exist anymore.
The way to handle custom R functions has been revamped and no longer relies on map_elements() or map_batches(). Instead, if your code cannot be implemented using polars syntax, we encourage the use of purrr and mirai.
A new vignette explains this in more details: pola-rs.github.io/r-polars/vig...
A new vignette explains this in more details: pola-rs.github.io/r-polars/vig...
Using custom functions - Polars R Package
pola-rs.github.io
July 23, 2025 at 6:01 AM
The way to handle custom R functions has been revamped and no longer relies on map_elements() or map_batches(). Instead, if your code cannot be implemented using polars syntax, we encourage the use of purrr and mirai.
A new vignette explains this in more details: pola-rs.github.io/r-polars/vig...
A new vignette explains this in more details: pola-rs.github.io/r-polars/vig...
Catching up with the python implementation means that we now have access to the new streaming engine with collect(engine = "streaming").
This is in many cases faster than the default engine and allows processing data that is larger than RAM. It will also continue to improve in future releases.
This is in many cases faster than the default engine and allows processing data that is larger than RAM. It will also continue to improve in future releases.
July 23, 2025 at 6:01 AM
Catching up with the python implementation means that we now have access to the new streaming engine with collect(engine = "streaming").
This is in many cases faster than the default engine and allows processing data that is larger than RAM. It will also continue to improve in future releases.
This is in many cases faster than the default engine and allows processing data that is larger than RAM. It will also continue to improve in future releases.
polars now uses rlang to take advantages of dynamic dots, better checks, and clearer error messages.
Compare for instance the old (polars0) and new (polars) errors when a column name doesn't exist:
Compare for instance the old (polars0) and new (polars) errors when a column name doesn't exist:
![A picture of code that shows an error message with the old version of polars.
> library(polars0)
> pl$DataFrame(x = "a")$select(pl$col("a")$str$pad_end(5, "multiple_chars"))
Error:
! Execution halted with the following contexts
0: In R: in $select()
0: During function call [pl$DataFrame(x = "a")$select(pl$col("a")$str$pad_end(5, "multiple_chars"))]
1: The argument [fillchar] caused an error
2: Expected a value of type [char]
3: Got value [Rvalue: ["multiple_chars"], Rsexp: Strings, Rclass: ["character"]]](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:gmnxrds63czhwmmzn5kdfjxa/bafkreiez3pgpgudscw6giipxbh3snxuc7mikehuputwx4w6uaqlsr3e5mi@jpeg)
July 23, 2025 at 6:01 AM
polars now uses rlang to take advantages of dynamic dots, better checks, and clearer error messages.
Compare for instance the old (polars0) and new (polars) errors when a column name doesn't exist:
Compare for instance the old (polars0) and new (polars) errors when a column name doesn't exist:
You could also be interested in tidypolars, especially for (but not limited to) parquet datasets.
(Disclaimer: I'm the author)
Same as duckplyr, that depends on whether the functions you use are translated.
Major drawback is that it's not on CRAN: tidypolars.etiennebacher.com
(Disclaimer: I'm the author)
Same as duckplyr, that depends on whether the functions you use are translated.
Major drawback is that it's not on CRAN: tidypolars.etiennebacher.com
More Efficient Tidyverse Code, Using Polars in the Background
Polars is a cross-language tool for manipulating very large data. However, one drawback is that the R implementation has a syntax that will look odd to many R users who are not used to Python syntax. ...
tidypolars.etiennebacher.com
July 7, 2025 at 3:25 PM
You could also be interested in tidypolars, especially for (but not limited to) parquet datasets.
(Disclaimer: I'm the author)
Same as duckplyr, that depends on whether the functions you use are translated.
Major drawback is that it's not on CRAN: tidypolars.etiennebacher.com
(Disclaimer: I'm the author)
Same as duckplyr, that depends on whether the functions you use are translated.
Major drawback is that it's not on CRAN: tidypolars.etiennebacher.com
Wow that's surprisingly low to me
July 7, 2025 at 9:26 AM
Wow that's surprisingly low to me
Is this about dependencies mentioned in DESCRIPTION or also recursive ones?
July 7, 2025 at 9:05 AM
Is this about dependencies mentioned in DESCRIPTION or also recursive ones?