furrr 0.2.0

  2020/10/19

  Davis Vaughan

We’re stoked to announce the release of furrr 0.2.0. furrr is a bridge between purrr‘s family of mapping functions and future‘s parallel processing capabilities. It attempts to make mapping in parallel as seamless as possible.

You can install it from CRAN with:

install.packages("furrr")

This blog post will highlight a few of the key changes since the last release of furrr, which was over two years ago!

This release of furrr is also a complete rewrite of the original version. This should make furrr more maintainable going forward, and fixed a ton of minor bugs from the original release. You can see a full list of those changes in the release notes.

library(furrr)

Hex sticker

Perhaps most importantly, furrr now has a hex sticker! A big thanks to Dan Kuhn for creating this furrry little guy.

New furrr hex sticker.

future_walk()

furrr now includes a parallel version of purrr::walk(). This was a highly requested addition, and allows you to call .f for its side effects, like for rendering output to the screen or for saving files to disk.

plan(multisession, workers = 2)

future_walk(1:5, ~print(paste0("Iteration: ", .x)))

#> [1] "Iteration: 1"
#> [1] "Iteration: 2"
#> [1] "Iteration: 3"
#> [1] "Iteration: 4"
#> [1] "Iteration: 5"

Vignettes

There are a whopping 5 new vignettes detailing several frequently asked questions about furrr.

Progress bar update

The above vignette regarding progress bars deserves a special mention. Henrik Bengtsson (the author of the future and globals packages, which furrr would be nothing without) recently introduced a new package for generalized progress updates, progressr. It has been integrated with future in such a way that it can relay near real-time progress updates from sequential, multisession, and even cluster futures (meaning that even remote connections can return live updates). This integration automatically extends to furrr, and looks a little like this:

library(progressr)

plan(multisession, workers = 2)

x <- replicate(n = 10, runif(20), simplify = FALSE)

with_progress({
  p <- progressor(steps = length(x))
  
  result <- future_map(x, ~{
    p()
    Sys.sleep(.2)
    sum(.x)
  })
})
#> |=====================                                               |  20%

progressr is a relatively new package, and its API isn’t perfectly compatible with furrr and tidyverse workflows yet, but I’d encourage you to read the previously mentioned vignette about progress notifications with progressr to learn more. In the future, furrr will likely have an even tighter integration with progressr to make this even easier.

Along those lines, once furrr and progressr become more tightly integrated, the .progress bar of furrr will be removed. It has not been deprecated yet, but I would encourage you to go ahead and switch to using progressr, if possible. To be completely honest, the progress bar in furrr is a bit of a hack, and progressr provides a much more robust solution.

Options

The future_options() helper has been renamed in favor of furrr_options(). This change was made to free up this function name in case the future package requires it. Additionally, furrr_options() has a number of new arguments, including one that allows you to further tweak how furrr “chunks” your input, chunk_size. If you are curious about this, read the new vignette on chunking.

Acknowledgements

We’re very thankful to the 81 contributions that went into this release. In particular, a huge thanks to Henrik Bengtsson for his work on the future and globals packages that power furrr.

@aaronpeikert, @adrfantini, @agilebean, @al-obrien, @alexhallam, @andrjohns, @aornugent, @Ax3man, @BChukwuSmith, @burchill, @cipherz, @cwickham, @data-al, @datawookie, @dhicks, @draben, @edavidaja, @edgBR, @EdJeeOnGitHub, @edvardoss, @gadenbuie, @Gomesdrg, @GShotwell, @hadley, @HanjoStudy, @HenrikBengtsson, @ignacio82, @Ilia-Kosenkov, @ivanhigueram, @JanLauGe, @jmlondon, @joethorley, @jschelbert, @julou, @jzadra, @kendonB, @khvorov45, @kimip24, @kkmann, @klahrich, @kurt1984, @leungi, @lrnv, @marcosci, @MatthieuStigler, @mattocci27, @mattwarkentin, @mikekaminsky, @mikkeltp, @mikldk, @MokeEire, @mpickard-niu, @naglemi, @nick-youngblut, @philerooski, @picousse, @Plebejer, @PMassicotte, @qpmnguyen, @randomgambit, @rcarboni, @rlbarter, @roman-tremmel, @rossellhayes, @sefabey, @sheffe, @ShixiangWang, @skalyan91, @snp, @solomonsg, @solunsteve, @srvanderplas, @statist-bhfz, @timvink, @tklebel, @vincentarelbundock, @vrontosc, @wenjia2018, @wjchulme, @xiaodaigh, and @yonicd.

Contents