Makepp Build Check Methods

How makepp decides to rebuild files

• Build check methods included in the distribution

A: architecture_independent,  E: exact_match,  I: ignore_action,  O: only_action,  T: target_newer

Makepp stores a variety of information about how any given file was built the last time. This information includes the build command, the architecture, and the signatures of all the file's dependencies. (All the stored information is in the subdirectory .makepp of each directory.) If any of this information has changed, makepp usually decides to rebuild the file. The build check method is what controls makepp's decision to rebuild. It decides which information to look at, and which to ignore.

Makepp usually picks the correct build check method automatically. However, you can change the signature method for an individual rule by using :build_check modifier on the rule, or for all rules in a makefile by using the build_check statement, or for all makefiles at once using the -m or --build-check-method command line option.

The data used to decide about a rebuild or a repository or build cache import is stored in the internal build info file. You can display it with makeppinfo, mppi. Below each method gives an example of how to see its keys.

Build check methods included in the distribution

At present, there are five build check methods included in the distribution:

exact_match

This method uses the modification dates on the file as signatures. It rebuilds the targets unless all of the following conditions are true:

• The signature of each dependency is the same as it was on the last build.

• The signature of each target is the same as it was on the last build (i.e., no one has messed with the targets since makepp built them).

• The build command has not changed.

• The machine architecture (or what Perl thinks it is) has not changed.

exact_match is the default method unless you are rebuilding a Makefile (see below). This is a highly reliable way of ensuring correct builds, and is almost always what you want. However, it does have a few side effects that may be surprising:

• If you've been compiling with the traditional make, and then switch to makepp, everything is recompiled the first time you run makepp.

• If you damage makepp's information about what happened on the last build (e.g., you delete the subdirectory .makepp, or don't copy it when you copy everything else), then a rebuild is triggered.

• If you replace a file with an older version, a rebuild is triggered. This is normally what you want, but it might be surprising.

• If you modify a file outside of the control of makepp (e.g., you run the compilation command yourself), then makepp will rebuild the file next time. (If you want to avoid this, check out the --dont-build command line option.)

• Architecture-independent files are rebuilt when you switch to a different architecture. This is usually not a problem, because they often don't take long to build. The reason why all files are tagged with the architecture, instead of just binary files, is that often times even ASCII files are architecture-dependent. For example, output from the Solaris lex program won't compile on Linux (or at least this used to be true the last time I tried it).

Concretely, a file will not be rebuilt, or can be fetched from repository or build cache, if the following command output stays the same, i.e. matches the signatures of the dependencies:

mppi -k'COMMAND ARCH SORTED_DEPS DEP_SIGS ENV_{DEP,VAL}S' file

architecture_independent

The architecture_independent method is the same as exact_match except that it does not check the architecture. This can be useful for architecture-independent files, that don't need to be rebuilt when you switch to a different architecture. For example, you probably don't need to rerun bison on Solaris if you already ran it on Linux.

The architecture_independent method is best used by specifying it using the :build_check architecture_independent modifier to the each rule that produces architecture independent files. Makepp by default never assumes any files are architecture independent, because even .c files can be architecture dependent. For example, the output of Solaris lex will not compile under Linux, or at least it wouldn't last time I tried. So you must manually specify this build check method for any files which are truly architecture-independent.

Concretely, a file will not be rebuilt, or can be fetched from repository or build cache, if the following command output stays the same, i.e. matches the signatures of the dependencies:

mppi -k'COMMAND SORTED_DEPS DEP_SIGS ENV_{DEP,VAL}S' file

ignore_action

The ignore_action method is the same as exact_match except that it does not check the action string (the command). Sometimes a command can change and you don't want to force a rebuild.

For example, you might want to explicitly put a date into your command to log when the build was done, but you don't want to force a rebuild every time the command is executed. For example,

BUILD_DATE := $(shell date)
my_program : $(MODULES).o 
    $(CXX) $(inputs) -DBUILD_DATE="\"$(BUILD_DATE)\"" date_stamp.c -o $(output)

This will compile date_stamp.c with the last build date stamp, but won't force a recompile when the date changes. Unfortunately, if something else about the link command changes (e.g., you change linker options), it also won't trigger a rebuild.

This is also useful in conjunction with the $(changed_inputs) or $? variable for actions that merely update a target, rather than rebuilding it from scratch. For example, you could update a .a file like this:

libmine.a : *.o : build_check ignore_action
    $(AR) ru $(output) $(changed_inputs)

This will still mostly work if you forget to specify the : build_check ignore_action. However, suppose that none of the .o files have changed. The command will now be ar ru libmine.a which is probably different from what it was last time (e.g., ar ru libmine.a buggy_module.o), so makepp will run the command. In this case, the command won't do anything except waste time.

Building .a files like this is discouraged, because it can leave stale .o files inside the archive. If you delete a source file, the .o file is still inside the .a file, and this can lead to incorrect builds. It's better to build a .a file like this:

libmine.a : *.o
    &rm $(output)
    $(AR) ru $(output) $(inputs)

Concretely, a file will not be rebuilt, or can be fetched from repository or build cache, if the following command output stays the same, i.e. matches the signatures of the dependencies:

mppi -k'ARCH SORTED_DEPS DEP_SIGS ENV_{DEP,VAL}S' file

target_newer

The target_newer method looks only at the file date. If any dependency is more recent than the target, the target is rebuilt. This is the algorithm that the traditional Unix make utility uses.

The target_newer method isn't as safe as the exact_match method because it won't trigger a rebuild if you change the build command, or if you replace a file with an older version. Sometimes also it can get confused if clocks are not properly synchronized. For example, if a file somehow gets a date of June 4, 2048, then between now and 2048, every file that depends on that file will be rebuilt even though the file doesn't change. Also switching to a different architecture won't trigger a rebuild. It prevents fetching a rule's target from a build cache, because there is no unique signature that can be associated to the endless set of pairs fulfilling the relationship newer than.

But there are a few cases where you may want to use the target_newer method:

• When it is reasonable for a user to build a file outside of the control of makepp. Perhaps the most common example are the commands that generate the makefile itself, i.e., the autoconfigure procedure. Users commonly issue the configure command manually, but makefiles often have a way to update themselves automatically. In this case, we don't want to force the makefile to rebuild itself if the user typed the command in manually, so the target_newer method is more appropriate than the exact_match method. In fact, if makepp is trying to build a makefile, it makes target_newer the default method until it has finished building the makefile.

• When it is reasonable for a user to modify a file after makepp has built it. For example, if a file does not exist, you may want to copy it from a central location, or check it out from a repository; but the user should be allowed to modify it. If you use the default exact_match build check method, makepp will detect that the user has changed the file and so it will force a fresh copy from the central location or a fresh checkout, wiping out the user's changes.

If you need to manually check the timestamps, see makeppinfo examples for how to get the path of each dependency.

only_action

The very specific only_action method will only execute the action if the command string differs from the last time it was executed. For example,

$(ROOT)/include/%.h : %.h
    &ln -fr $(input) $(output)

publishes a file, but does not repeat this when the file changes. Note that the &ln command is builtin and thus cheap, but makepp still has to fork off and monitor a process to perform the whole action. So if you have lots of files to publish, there is still a benefit. Actually we did not specify the method, because, when the target is a symbolic link, this build check gets used automatically. You only need to specify it for other commands that depend solely on the command (which usually contains the names of the inputs):

%.list : %.x : build_check only_action
    &echo $(inputs) -o $(output)

Concretely, a file will not be rebuilt, or can be fetched from repository or build cache, if the following command output stays the same, i.e. matches the signatures of the dependencies:

mppi -kCOMMAND file

Other build check methods are possible. You can write your own build check method by creating a module Mpp::BuildCheck::MyMethod. Read the documentation in Mpp/BuildCheck.pm in the makepp distribution. Most likely, you will want your build check method to inherit from Mpp::BuildCheck::exact_match, so read its documentation too.

It's more commonly useful modify the signature mechanism than to modify the build check mechanism directly. Before you change the build check mechanism, see if your problem is better served by changing signatures (see the makepp_signatures manpage for details).

Here are some reasons why a custom build check method might be useful:

• If you want makepp to ignore part of the command. For example, if you have commands in your makefile like this:

  x.o : x.c
      ssh $(REMOTE_MACHINE) cc $< -o $@

  you might want makepp not to force a rebuild if $(REMOTE_MACHINE) changes. You could modify the exact_match method so it knows about ssh commands and ignores the machine name. Check :dispatch for another way to achieve that.