Over the summer I had the pleasure of working with Jim Hester on the cpp11 package as a tidyverse summer intern. The cpp11 package is a header-only R package that helps R package developers handle R objects with C++ code. Its goals and syntax are similar to the excellent Rcpp package. During most of my internship, I worked on triaging issues, fixing bugs, and adding new features to cpp11. Near the end of the summer, I got to work with Jenny Bryan on converting readxl to using cpp11. Jim has written a great post about converting packages from Rcpp to cpp11, which I heavily referenced during the process. But there were still some challenges I encountered. To help others going through a similar process, I wanted to review the workflows and tools we used to make this easier.
The entire process took about a week. Using both Rcpp and cpp11 at the same time in a package is okay for short term work like this, so I didn’t feel rushed. To get started, I followed the initial set-up steps outlined in Jim’s article and then recompiled, to confirm it was successful. Next, I needed to include either the cpp11/R.hpp header or the macros R_NO_REMAP and STRICT_R_HEADERS. I found it easier to include the macros since the header file needs to come before any headers that use Rcpp.
Step-by-step integration
After I’d set-up my environment it was time to start converting the files. Initially, I made the mistake of selecting the most interconnected file in the readxl package. Since this file was central to the package, I would have had to convert the entire package over to cpp11 before the compilation errors or test failures would stop. After Jenny and Jim helped me select a more approachable file, the process was more incremental and I successfully maintained my sanity 🙌.
When I was finished with a file, I would recompile the package and fix the compilation errors. To fix errors, I sometimes had to edit other functions in other files, but I only edited files enough to fix the failures. If I did too much it was difficult for me to know what was causing the failures. Then I’d run the tests for the readxl package. Thankfully, Jenny had great test coverage for the readxl package prior to this project, so we didn’t need to write more tests.
Another incremental approach I used was to convert one function in a file, and then recompile and run the tests. This approach worked best for larger files. Once everything was passing, I’d commit and push to the PR so that my gracious reviewers could review my changes in stages, rather than in one big batch. This also allowed me to benefit from continuous integration, i.e. my changes were checked on a greater variety of R versions and operating systems, thanks to the checks we run via GitHub Actions. It’s a win-win!
The nitty-gritty
For the nitty-gritty details of converting the code, I definitely took advantage of the table in Jim’s post! That comparison table is great for converting between Rcpp and cpp11 classes. The table also provided me with information on whether a class was readable, writable or both. This new feature in cpp11 is great, since writable vectors are costly because the data must be fully copied, so using readable where appropriate is a good idea. When I was unsure of whether an object should be readable or writable, I would make it readable and then recompile to see if I was correct.
Almost done
When I had finished converting all the objects to cpp11, I removed the macros R_NO_REMAP and STRICT_R_HEADERS and any stray #include "Rcpp.h" directives. Finally, to check for any other updates that might be required, such as in the DESCRIPTION file, I also ran devtools::check(). And that’s it!
I had so much fun working with the tidyverse team. And a big thank you to Jim for all the support over the summer and to Jenny for their help on readxl! If you’re looking for more examples of updating packages to using cpp11, Jim has also gone through the process of converting readr to using cpp11. RStudio is a great place to look for summer internship opportunities. They had a variety of opportunities this summer and I’d encourage anyone looking for summer internships to apply for 2022!