How makepp finds include files and other hidden dependencies
Makepp can guess additional dependencies or targets for certain commands that it knows something about. This is especially important for C/C++ compilation, where it is too error-prone to list manually all of the include files that a given source file depends on. By looking at the compilation command and the source files themselves, makepp is able to determine accurately which object files need to be rebuilt when some include file changes.
Makepp looks at the first word of your command line.
If it recognizes it, it uses the scanner corresponding to
that first word. Specifically, it isolates the first word and looks it
up in its table; if nothing is found, it strips off the directory
information and looks it up again. (This means that you can specify the
path to your compiler and makepp will still recognize it.) Currently,
makepp recognizes most C/C++ compiler names. It also recognizes
commands invoking sh and libtool; for these commands, it skips to
the first word in the command that's not an option and looks it up in
the table again.
If makepp thinks it's compiling a C/C++ program but can't find a scanner, it will give a warning message to let you know. This usually means that you buried your compiler command too deeply in the action for makepp to find it. For example, I have seen rules like this:
%.o: %.c
@echo Compiling $< now
@gcc -c $< $(CFLAGS) -o $@
The first word of the action here is echo, for which therer is no
scanner, so makepp will not scan for include files in this case.
The C/C++ scanner is activated by a command beginning with a compile
program that makepp knows about. (I've included every compiler I've
ever heard of, but if I missed your compiler, you can tell makepp about
it by adding an entry to the %Makesubs::scanners array (see
Makesubs.pm in the distribution). Also send me email at
holt-makepp@gholt.net so I can include it in subsequent releases.
It looks at the command for -Idir options specifying the include path
or -Ldir options specifying the link path. It then scans any source
files for #include directives, and also looks at the command line to
see if there are any source files or libraries mentioned which are not
listed as dependencies. It recognizes these by their extension.
This scanner gives a warning message if files included with #include
"file.h" are not found in the include path, or in the directory
containing the file which is #including, or in /usr/include. No
warning is given if a file included with #include <file.h>
is not found. makepp assumes it is in some system include directory
that the compiler knows about, and that files in system include
directories won't change.
In addition, files in /usr/include, /usr/local/include,
/usr/X11R6/include, and any other directory which is not writable are
not scanned to see what they include. Makepp assumes that these files
won't change. (If you're running as root, the writability test is
performed with the UID and GID of the directory you ran makepp from.
This is so compiling a program as an ordinary user and then doing
makepp install as root won't cause extra directories to be scanned.)
This is a fairly simple-minded scanner. It will get confused if you do things like this:
#ifdef INCLUDE_THIS #include "this.h" #endif
because it doesn't know about preprocessor conditionals. This is
usually harmless; it might cause additional extra files to be labelled
as dependencies (occasionally causing unnecessary rebuilds), or else it
might cause makepp to warn that the include file was not found. You can
either ignore the warning messages, or put an empty file this.h out
there to shut makepp up.
Libtool is a very clever compilation system that greatly simplifies
making shared libraries by hiding all the system-dependent details away
in a shell script. The only difficulty is that the library binary files
are not actually stored in the same directory as the output
file--libtool actuall creates a subdirectory, .libs, which contains
the real files. This is ordinarily not a problem, but makepp has to
know where the real binaries are if it is to link them in from a
repository. At the moment, libtool libraries (.la files) are not
linked in from repositories; they are always rebuilt if needed. Also,
makepp at the moment is not able to use the dependency information that
is stored inside the .la file itself. This will hopefully change
soon.
Swig (Simplified Wrapper and Interface Generator, http://www.swig.org) is a program that converts a C/C++ header file into the wrapper functions needed to make your code callable from a variety of other languages, such as perl, python, tcl, C#, ruby, ocaml, and probably some others that I don't know about.
Makepp understands and parses the swig command line, looking for -I and
-l options. It also knows how to scan swig's interface definition files
(.i files) looking for %include, %import, and also #include
if -includeall is in effect.
Makepp recognizes the following command words and skips over them
appropriately in in its search for the correct scanner:
condor_compile, echo ignore_error libtool, noecho
purify, sh.
The :quickscan and :smartscan rule options, if applicable, affect the way that files are scanned.
In :quickscan mode (the default), all include directives are assumed active. This allows for very efficient scanning.
In :smartscan mode, an attempt is made to interpret macros and expressions so that inactive include directives are ignored. For example, the executable produced by compiling the following C program ought not to depend on foo.h:
#if 0
# include "foo.h"
#endif
int main() { return 0; }
In the future, :smartscan might become the default.
Writing your own scanner is somewhat involved, but it is possible. There are 3 ways to hook into this:
If scanner foo is specified in a rule option, and there is a subroutine
called parser_foo defined in the makefile's package, then that subroutine
must return an object of type ActionParser, which will be used to
determine a command parser for every command.
If scanner foo is specified in a rule option, and there is a subroutine
called scanner_foo defined in the makefile's package, then that subroutine
must either do all of the scanning work (which is deprecated), or return
an object of type CommandParser which will be used to parse every command.
If no scanner is specified in a rule option, then an object of the
ActionParser base class is used.
That object determines the scanner based on the first word of each command.
The base name is sought as a key in %scanners hash in the makefile's package,
which is seeded with all of the default scanners.
This hash can be extended using the register_scanner and
register_command_parser statements.
The value from this hash lookup is a coderef that either does the scanning
work (which is deprecated), or returns an object of type CommandParser.
In most cases, objects of type CommandParser should instantiate at least
one object of type Scanner.
The Scanner base class takes care of the distinction between quickscan
and smartscan.
Note that the behavior of Scanner can be markedly affected by this
distinction, but that should be transparent to the derived class if it
is well-formed.
New derived Scanner classes ought to be tested in both modes.
If you want to specify a particular CommandParser class for a rule, then
you can specify a scanner with a rule option, and arrange for the parser_*
subroutine to return an object of type ActionParser::Specific.
For more details, refer to the respective class documentation.
For examples, see CommandParser::Gcc and CommandParser::Vcs.
If the all of the scanner's important side effects are effected through calls
to methods of the CommandParser base class, then those side effects can
be cached in the build info file, so that they can be played back by a
subsequent invocation of makepp without doing all of the costly scanning work.
This can save quite a bit of time, especially in smartscan mode.
If the scanner has other important side effects, then it should call the
Rule object's mark_scaninfo_uncacheable method.
Otherwise, the scanner info retrieved from the build info may be inaccurate,
causing the build result possibly to be incorrect.
This method is called automatically when a value from the %scanners hash does
not return an object of type CommandParser, or when the scanner is specified
with a rule option, no parser_* subroutine is defined and the scanner_*
routine does not return an object of type CommandParser.
Cached scan info is invalidated using criteria similar to those used for determining when the target is out of date. Similarly, it can be retrieved from a repository using criteria similar to those used for determining when a target can be linked in from a repository.
You can force makepp to ignore the cached scanner info with the
--force-rescan option.
This is useful when a broken scanner may have caused incorrect scanner info
to be cached.
Often you will have just one or few files which contain dependency information. You don't want to write this into a makefile reduntantly (since redundancy later often leads to inconsistencies when one update gets forgotten). But you also don't want to write a Scanner? As a workaround you can generate an include file on the fly. For example Qt has .qrc files which can look like:
<RCC>
<qresource prefix="...">
<file>abc</file>
<file>xyz</file>
...
If you adhere to the above layout, you can transform the relevant lines into a makepp include file, which gets automatically created by being included.
%.qrc.makepp: %.qrc
&grep 's!<RCC>\n!$(stem).cc:! || s! *<file>! ! && s!</file>\n!!' $(input) -o $(output)
include $(wildcard *.qrc) # .makepp is appended automatically