An Introduction to <em>fakemake</em>

Why Mock the Unix Make Utility?

There are many build systems, and even more uses for build systems (see [Powers], sections 11.10 and 11.11).

I have been using the unix make utility when developing R packages since 2012. But sometimes I get caught on a machine where make is not available and where I am not entitled to install it
[This is a nice example of what restrictive software policies are good for: you end up with a buggy imitation like fakemake instead of the well established original. You should not regulate software installations for programmers, unless you take away their interpreters/compilers.]

This is why I wrote fakemake: to build an R package conditionally on the modification times of (file) dependencies without having to rely on external software. If you have any proper build system at hand: stick to it, do not use fakemake.

withr and knitr

Throughout this vignette I use R's temporary directory, often by using withr::with_dir(tempdir(), …). Because this is a vignette and the codes are examples. In real life, we would skip the temporary directory stuff.

This vignette is built using knitr, which itself uses sink(). As sink() is central to fakemake for redirecting output to files in the make chain, I have to disable some of knitr's output here and there. Don’t worry, it’s just because knitr and fakemake both want to use sink() exclusively and it only affects vignettes built with knitr.

Makelists

A makelist is fakemake's representation of a Makefile. It’s just a list of lists. Look at the minimal makelist provided by fakemake:

str(fakemake::provide_make_list("minimal", clean_sink = TRUE))

## List of 4
##  $ :List of 3
##   ..$ target       : chr "all.Rout"
##   ..$ prerequisites: chr [1:2] "a1.Rout" "a2.Rout"
##   ..$ code         : chr "print(\"all\")"
##  $ :List of 2
##   ..$ target: chr "a2.Rout"
##   ..$ code  : chr "print(\"a2\")"
##  $ :List of 3
##   ..$ target       : chr "a1.Rout"
##   ..$ prerequisites: chr "b1.Rout"
##   ..$ code         : chr "print(\"a1\")"
##  $ :List of 2
##   ..$ target: chr "b1.Rout"
##   ..$ code  : chr "print(\"b1\")"

Each sublist represents a Makefile’s target rule and has several items: at least a target and either code or prerequisites, possibly both. This makelist would still be a Makefile’s valid representation if target rule #3 with target "a1.Rout" had no (or an empty) code item.

Other possible target rule entries are:

alias: An alias to target that would be easier to remember and/or type (we will come back to his later).
sink: By default, all output of the code item is dumped (well, with R's sink function) into a file with the name given by target. If target should be created by the code you will want to redirect the output into sink (we will come back to his later).
.PHONY: If set to TRUE, the target is rebuilt unconditionally every time the target is hit (trying to mock GNU make’s .PHONY-extension) (we will come back to his later).

Using fakemake

Suppose we would have a minimal makelist:

ml <- fakemake::provide_make_list("minimal", clean_sink = TRUE)

We can visualize the makelist (giving the root is optional, in this case it just makes a neater plot):

fakemake::visualize(ml, root = "all.Rout")

Figure 1. A Makelist

Building and Rebuilding

Now build the "all.Rout" target:

withr::with_dir(tempdir(), print(fakemake::make("all.Rout", ml)))

## [1] "b1.Rout"  "a1.Rout"  "a2.Rout"  "all.Rout"

We can see the files created:

show_file_mtime <- function(files = list.files(tempdir(), full.names = TRUE,
                                               pattern = "^.*\\.Rout")) {
    return(file.info(files)["mtime"])
}
show_file_mtime()

##                                        mtime
## /tmp/RtmpK8kWr5/a1.Rout  2022-03-31 14:16:33
## /tmp/RtmpK8kWr5/a2.Rout  2022-03-31 14:16:33
## /tmp/RtmpK8kWr5/all.Rout 2022-03-31 14:16:33
## /tmp/RtmpK8kWr5/b1.Rout  2022-03-31 14:16:33

If we wait for a second and rerun the build process, we get:

# ensure the modification time would change if the files were recreated
Sys.sleep(1)
withr::with_dir(tempdir(), print(fakemake::make("all.Rout", ml)))

## NULL

show_file_mtime()

##                                        mtime
## /tmp/RtmpK8kWr5/a1.Rout  2022-03-31 14:16:33
## /tmp/RtmpK8kWr5/a2.Rout  2022-03-31 14:16:33
## /tmp/RtmpK8kWr5/all.Rout 2022-03-31 14:16:33
## /tmp/RtmpK8kWr5/b1.Rout  2022-03-31 14:16:33

Nothing changed. Good. Now, we change one file down the build chain:

fakemake::touch(file.path(tempdir(), "b1.Rout"))
withr::with_dir(tempdir(), print(fakemake::make("all.Rout", ml)))

## [1] "a1.Rout"  "all.Rout"

show_file_mtime()

##                                        mtime
## /tmp/RtmpK8kWr5/a1.Rout  2022-03-31 14:16:34
## /tmp/RtmpK8kWr5/a2.Rout  2022-03-31 14:16:33
## /tmp/RtmpK8kWr5/all.Rout 2022-03-31 14:16:34
## /tmp/RtmpK8kWr5/b1.Rout  2022-03-31 14:16:34

Since a1.Rout depends on b1.Rout and all.Rout depends on a1.Rout, these targets get rebuilt while a2.Rout stays untouched.

Had we touched a1.Rout, b1.Rout would not have been rebuilt:

fakemake::touch(file.path(tempdir(), "a1.Rout"))
withr::with_dir(tempdir(), print(fakemake::make("all.Rout", ml)))

## [1] "all.Rout"

show_file_mtime()

##                                        mtime
## /tmp/RtmpK8kWr5/a1.Rout  2022-03-31 14:16:35
## /tmp/RtmpK8kWr5/a2.Rout  2022-03-31 14:16:33
## /tmp/RtmpK8kWr5/all.Rout 2022-03-31 14:16:35
## /tmp/RtmpK8kWr5/b1.Rout  2022-03-31 14:16:34

Forcing the Build

If you set the force option, you can force the target and all its prerequisites down the build chain to be built:

withr::with_dir(tempdir(), print(fakemake::make("all.Rout", ml, force = TRUE)))

## [1] "b1.Rout"  "a1.Rout"  "a2.Rout"  "all.Rout"

If you want to force the target itself, but not all its prerequisites, set recursive = FALSE:

withr::with_dir(tempdir(), print(fakemake::make("all.Rout", ml, force = TRUE,
                                                recursive = FALSE)))

## [1] "all.Rout"

Faking the Build

If you don’t actually want to run the recipes but would rather like to know what would happen if you ran the build chain (this mocks GNU make’s -n option), you can set dry_run = TRUE:

file.remove(dir(tempdir(), pattern = ".*\\.Rout", full.names = TRUE))

## [1] TRUE TRUE TRUE TRUE

withr::with_dir(tempdir(), print(fakemake::make("all.Rout", ml, dry_run = TRUE)))

## [1] "b1.Rout"  "a1.Rout"  "a2.Rout"  "all.Rout"

Note that no files have been created:

dir(tempdir(), pattern = ".*\\.Rout")

## character(0)

So we recreate them now:

withr::with_dir(tempdir(), print(fakemake::make("all.Rout", ml)))

## [1] "b1.Rout"  "a1.Rout"  "a2.Rout"  "all.Rout"

dir(tempdir(), pattern = ".*\\.Rout")

## [1] "a1.Rout"  "a2.Rout"  "all.Rout" "b1.Rout"

Using Aliases

If you find a target rule’s target too hard to type, you can use an alias:

i <- which(sapply(ml, "[[", "target") == "all.Rout")
ml[[i]]["alias"] <- "all"
withr::with_dir(tempdir(), print(fakemake::make("all", ml, force = TRUE)))

## [1] "b1.Rout"  "a1.Rout"  "a2.Rout"  "all.Rout"

This is pointless here, but _target_s might be files down a directory tree like log/roxygen2.Rout when building R packages: you might want to alias that target to roxygen

Diverting Output / Programmatically Creating a Target Rule’s Target

Target rule b1 dumps its output to b1.Rout:

cat(readLines(file.path(tempdir(), "b1.Rout")), sep = "\n")

## [1] "b1"

Suppose it would programmatically create the target:

i <- which(sapply(ml, "[[", "target") == "b1.Rout")
ml[[i]]["code"]  <- paste(ml[[i]]["code"],
                      "cat('hello, world\n', file = \"b1.Rout\")",
                      "print(\"foobar\")",
                      sep = ";")
withr::with_dir(tempdir(), print(fakemake::make("b1.Rout", ml, force = TRUE)))

## [1] "b1.Rout"

cat(readLines(file.path(tempdir(), "b1.Rout")), sep = "\n")

## hello, wo[1] "foobar"

You end up with a broken target file, so you need to add a *sink*:

ml[[i]]["sink"] <- "b1.txt"
withr::with_dir(tempdir(), print(fakemake::make("b1.Rout", ml, force = TRUE)))

## [1] "b1.Rout"

Now you get what you wanted:

cat(readLines(file.path(tempdir(), "b1.Rout")), sep = "\n")

## hello, world

cat(readLines(file.path(tempdir(), "b1.txt")), sep = "\n")

## [1] "b1"
## [1] "foobar"

We need sinks when the target’s code creates the target, for example when it builds a package’s tarball: we would want to get the output of building the tarball to be written to a file the path of which we specify via the target’s sink.

No Code Targets

Rule a1 has code

i <- which(sapply(ml, "[[", "target") == "a1.Rout")
ml[[i]]["code"]

## $code
## [1] "print(\"a1\")"

that prints "a1" into "a1.Rout":

cat(readLines(file.path(tempdir(), "a1.Rout")), sep = "\n")

## [1] "a1"

If we remove that code and its output file and rerun

ml[[i]]["code"]  <- NULL
withr::with_dir(tempdir(), print(fakemake::make("a1.Rout", ml, force = TRUE)))

## [1] "b1.Rout" "a1.Rout"

the file is still created (note that target rule b1 down the make chain is run since we did not set recursive = FALSE ) but empty:

file.size(file.path(tempdir(), "a1.Rout"))

## [1] 0

Phony Targets

As you have seen, you can temporarily force a build. You may set a target to be .PHONY which forces it (but not its prerequisites) to be built:

ml[[i]][".PHONY"]  <- TRUE
withr::with_dir(tempdir(), print(fakemake::make("a1.Rout", ml)))

## [1] "a1.Rout"

Building Packages

See vignette Building R Packages Using fakemake.

References

[Powers] Shelley Powers, Jerry Peek, Tim OReilly and Mike Loudikes. Unix Power Tools, O’Reilly & Associates, ISBN: 0-596-00330-7, 2002.

An Introduction to fakemake