tune version 2.0.0

  2025/11/05

  parallel processing, postprocessing, tidymodels

  Max Kuhn, Simon Couch, Emil Hvitfeldt, Hannah Frick

We’re very chuffed to announce the release of tune 2.0.0. tune is a package that can be used to resample models and/or optimize their tuning parameters

You can install it from CRAN with:

install.packages("tune")

This blog post will describe the two major updates to the package. You can see a full list of changes in the release notes.

Those two big improvements to the package: new parallel processing features and postprocessing.

Using future or mirai for parallel processing

Historically, we’ve used the foreach package to run calculations in parallel. Sadly, that package is no longer under active development. We’ve been progressively moving away from it, and as of this version, it is deprecated. In its place, we’ve added functionality for the future and mirai packages.

Previously, you would load a foreach parallel backend package, such as doParallel, doMC, or doFuture, and then register it. For example:

library(doParallel)
cl <- makePSOCKcluster()
registerDoParallel(cl)

Instead, you can use the future package via:

library(future)
plan("multisession")

or the mirai package by using

library(mirai)
daemons(num_cores)

Each of these is configurable to run in various ways, such as on remote servers.

tidymodels.org and the tune pkgdown site have more information to help users switch away from foreach.

Tuning your postprocessor

A postprocessor is an operation that modifies model predictions. For example, if your classifier can separate classes but its probability estimates are not accurate enough, you can add a calibrator operation that can attempt to adjust those probability estimates. Another good example is for binary classifiers, where the default threshold for classifying a prediction as an event can be adjusted based on its corresponding probability estimate.

Currently, we’ve enabled postprocessing using the tailor package. The operations that are currently available:

  • adjust_numeric_calibration(): Estimate and apply a calibration model for regression problems.
  • adjust_numeric_range(): Truncate the range of predictions.
  • adjust_probability_calibration(): Estimate and apply a calibration model for classification problems.
  • adjust_probability_threshold(): Covert binary class probabilities to hard class predictions using different thresholds.
  • adjust_equivocal_zone(): Decline to predict a sample if its strongest class probability is low.
  • adjust_predictions_custom(): A general mutate()-like adjustment.

If the operations have arguments, these can be tuned in the same way as the preprocessors (e.g., a recipe) or the supervised model. For example, let’s tune the probability threshold for a random forest classifier.

We’ll simulate some data with a class imbalance:

library(tidymodels)

set.seed(296)
sim_data <- sim_classification(2000, intercept = -12)
sim_data |> count(class)

## # A tibble: 2 × 2
##   class       n
##   <fct>   <int>
## 1 class_1   234
## 2 class_2  1766

We’ll resampling them via 10-fold cross-validation:

sim_rs <- vfold_cv(sim_data, strata = class)

We define a tailor object that tags the class probability threshold for optimization:

tlr_spec <- 
  tailor() |> 
  adjust_probability_threshold(threshold = tune())

We also specify a random forest that uses its default tuning parameters:

rf_spec <- rand_forest(mode = "classification")
rf_thrsh_wflow <- workflow(class ~ ., rf_spec, tlr_spec)
rf_thrsh_wflow

## ══ Workflow ════════════════════════════════════════════════════════════
## Preprocessor: Formula
## Model: rand_forest()
## Postprocessor: tailor
## 
## ── Preprocessor ────────────────────────────────────────────────────────
## class ~ .
## 
## ── Model ───────────────────────────────────────────────────────────────
## Random Forest Model Specification (classification)
## 
## Computational engine: ranger 
## 
## 
## ── Postprocessor ───────────────────────────────────────────────────────

## 

## ── tailor ──────────────────────────────────────────────────────────────

## A binary postprocessor with 1 adjustment:

## 

## • Adjust probability threshold to optimized value.

## NA
## NA
## NA

With a class imbalance, the default 50% threshold yields high specificity but low sensitivity. When we alter the threshold, those numbers will change, and we can select the best trade-off for our application. Let’s tune the workflow:

cls_mtr <- metric_set(roc_auc, sensitivity, specificity)

# To run all resamples in parallel:
mirai::daemons(10)

set.seed(985)
rf_thrsh_res <- 
  rf_thrsh_wflow |> 
  tune_grid(
    resamples = sim_rs,
    grid = tibble(threshold = seq(0, 0.6, by = 0.01)),
    metrics = cls_mtr
  )

Let’s visualize the results:

autoplot(rf_thrsh_res) + lims(y = 0:1)

plot of chunk autoplot

We can see that we can improve sensitivity by reducing the threshold. The rate of decay in specificity is slow compared to the gain in sensitivity until thresholds less than 10% are used. The Brier score is constant over the threshold since it only uses the estimated class probabilities, which are unaffected by the threshold.

We’ve taken great pains to avoid redundant calculations. In this example, for each resample, a single random forest model is trained, and then the postprocessing grid is evaluated. This conditional execution strategy is used to fit the fewest possible preprocessors, models, and postprocessors.

For this classification example, recent updates to the desirability2 package can enable you to jointly find the best sensitivity/specificity trade-off using the threshold parameter and model calibration/separation using other parameters.

We’ll add more examples and tutorials to tidymodels.org to showcase what we can do with postprocessing.

What’s next

This had been a race towards posit::conf(2025). Our focus had to be on the two big features for this release (since we taught workshops that use them). There are a few other relatively minor issues to address as the year closes.

One is to swap the package that we currently use for Gaussian Processes in Bayesian optimization from the GPfit package to the GauPro package. The former is not actively supported, and the latter has a few features that we’d love to have. Specifically, better kernel methods for non-numeric tuning parameters (e.g., the type of activation function used in neural networks). Hopefully, we’ll have another planned release before the end of the year.

Another near-future development goal is to have comprehensive integration for quantile regression models. We’ve added a few parsnip engines already and will expand the support in yardstick and tune.

Acknowledgements

We’d like to thanks everyone who contributed since the previous version: @3styleJam, @Diyar0D, @EmilHvitfeldt, @hfrick, @MatthieuStigler, @MattJEM, @mthulin, @tjburch, and @topepo.

Contents