parallelRemake

The parallelRemake package is a helper add-on for remake, a Makefile-like reproducible build system for R. If you haven’t done so already, go learn remake! Also learn GNU make, and then recall that make -j 4 runs a Makefile while distributing the rules over four parallel processes. This mode of parallelism is the whole point of parallelRemake. With parallelRemake, you can write an overarching Makefile for a remake project to run remake targets in parallel. This distributed parallelism is extremely helpful for large clusters that use the Slurm job scheduler, for example, as explained in this post.

Installation

To install the development version, get the devtools package and run

devtools::install_github("wlandau/parallelRemake", build = TRUE)

If you specify a tag, you can install a GitHub release.

devtools::install_github("wlandau/parallelRemake@v0.0.2", build = TRUE)

Rtools for Windows users

The example and tests sometimes use system("make") and similar commands. So if you’re using the Windows operating system, you will need to install the Rtools package.

Tutorial

The online package vignette has a complete tutorial. You can load the compiled version from an R session.

vignette("parallelRemake")

Help and troubleshooting

Use the help_parallelRemake() function to obtain a collection of helpful links. For troubleshooting, please refer to TROUBLESHOOTING.md on the GitHub page for instructions.

Acknowledgements

This package stands on the shoulders of Rich FitzJohn’s remake package. Also thanks to Daniel Falster for the idea that made parallelRemake practical and clean.