Makepp Release Notes

Major changes in each version of makepp

• Version 2.1 (under development)
• Version 2.0 (March 2012)
• Version 1.40 (December 2004)
• Version 1.19 (July 2003)
• Version 1.18 (January 2002)
• Version 1.10 (February 2001)
• Version 1.05
• Version 0.99 (January 2001)
• Version 0.95
• Version 0.90

Version 2.1 (under development)

• Warning and error messages switched to GNU style.

• Pattern rules can pick up phony dependencies (gmake compatibility), rule option :no-phony prevents that.

• Builtin &template mini-language documentation rewritten. It now has an @include(filename)@ macro.

• C Scanner now takes into account compiler specific path variables like C_INCLUDE_PATH or LIB.

• Parens now nest as in GNU make or Shell: $(name ...()...)

• New environment variable MAKEPP_DEBUG.

• Installation defaults for html-doc and man have changed slightly.

• The new option --loop is the successor to --stop-after-loading, which it enhances to get the head start for every edit compile cycle in an endless loop.

• The include statements and the functions find-first-upwards and find-upwards will no longer search higher than a RootMakeppfile, if one is present.

• The gmake style of putting the rule to make an include file after the include statement is not recommended, but now supported. Thus the option --defer-include has been removed.

• All pre-2.0 features which issued deprecated-warnings are eliminated, as are $Mpp::Makefile::legacy_functions and $MAKEPP_INSTALL_OLD_MODULES.

• You can use the new makeppinfo, mppi option --unremembered to spot no longer needed files.

• You can use the new makeppreplay, mppr option --sed to replay a command differently, e.g. to stop after preprocessing or to add debugging options to the linker.

• There was an undocumented (and slightly buggy) feature whereby making .makepp/log unwritable would suppress logging. Now instead it falls back to --verbose, so you must pass --no-log if you want that. This option and a few others were also already available for makeppreplay, mppr, but not documented.

• At least Perl 5.8 is required.

Version 2.0 (March 2012)

The items are roughly ordered by increasing age, so you need to read only the first parts if you've been using snapshots newer than 1.40.

• The signature statement no longer overrides the method found by command parsers — now you must provide the override keyword if you want that. There are new signature methods xml and xml-space. The c_compilation_md5 signature can now also be invoked as C (because signatures pertain to files, not actions). It (and its subclasses) can now easily be extended to other suffixes as in C.ipp,tpp meaning that besides the built in suffixes it will also apply to files ending in .ipp or .tpp. Thanks to this makeppreplay and embedded SQL/C now works more reliably. Beware: this is not understood by older versions. So don't call an older mpp on things built with this version, which includes from a repository you built in.

• The keyword global can precede the build_cache, build_check and signature statements and the define statement which now also allows export. Therefore s_ subs now get a 4th parameter, a hash reference with the encountered keywords as keys.

• Makefiles are now parsed with the same priority as in gmake. Hence include : or include = are now statements (no space: still rule or assignment.) This means that perl { qualified::name } no longer mysteriously disapears (it used to be a useless rule.) Now variable names may contain whitespace. This means that &preprocess --assignment will treat just about any equal sign as an assignment (replace them with $E from the funny assignment E==.)

• If you have a federated build cache over several disks, and some of them are preferred, you must edit their build_cache_options.pl and change PREFERRED to xPREFERRED. (This reflects a general change in names where we now prefix by 'x' all xATTRIBUTES which are are true iff they exist).

• The VPATH variable and vpath statement are now emulated.

• Makefile functions (sub f_...) may now get their 1st parameter as a string reference. In that case you are responsible for expanding your arguments. Use the accessor functions &arg or args described in makepp extensions. Until you get round to updating your functions, you can turn this off by setting $Mpp::Makefile::legacy_functions = 1 either in your makefile (one per build system is enough, as it's a Perl variable) or patch it into your new makepp installation. If you want your updated functions to work with both your old and this new makepp, use ref $_[0] ? &arg : $_[0] or ref $_[0] ? args(...) : $_[0] in the transition phase.

  Within $(call), the special variables $0, $1, ..., $(11), ... are now expanded like any other make variable. This causes slight differences to the previous work around. The result is closer to gmake, at least if makepp_simple_concatenation=1 is set.

• CMake generated makefiles no longer lead to deep recursion.

• New scanner esql_compilation for embedded SQL C, now covers all databases. You must supply the rules, however.

• New option --hybrid-recursive-make (follow each option link, to see shorter forms) as a smarter alternative to --traditional-recursive-make.

  New option --stop-after-loading gives makepp a headstart while you're still editing.

  New options --rm-stale, --sandbox and --dont-read, for sandbox control when running multiple concurrent (possibly distributed) makepp commands.

  Makepp will now also look for options in files called .makepprc. The option --args-file is now consistently available on all commands.

  The environment variable MAKEPP_CASE_SENSITIVE_FILENAMES supercedes the options --case-sensitive-filenames and --no-case-sensitive-filenames.

  Removed --keep-repository-links option, the behaviour of which is now the default.

  All utilities now also query an environment variable for presetting options. Each one is called like the utility in upper case, with FLAGS added, e.g. $MAKEPPCLEANFLAGS or $MAKEPPLOGFLAGS.

  Short command line options may now be grouped in the Unix way, so -k -j 4 may be given as -kj4. In all long options the dash between words may now consistently be omitted or replaced with an underscore, so that --no-log can be --nolog or --no_log. Unknown options now cause an error.

• Originally makeppclean -l meant --leave-src-info, but now the options have been extended and this one changed to -k|--keep-src-info.

• Scanning terminology has been cleaned up (and the page rewritten) to finally match a redesign that happened a few years ago. There are now 3 separate terms, the lexer (which users don't need to worry about), the (command) parsers and the (file) scanners.

  For advanced users: The new interface consists of the register_parser or register_command_parser statements, the :parser rule option and the p_* parser factory functions which get aliased to their parser class as factory. The misnamed register_scanner statement, :scanner rule option and :scanner_* or :parser_* functions are deprecated.

• New variable MAKEPP_VERSION.

• All internal classes have been moved to the new package Mpp:: so as to abide by CPAN rules, and to avoid potential collision with any module you might load into your makefiles. This is transparent to casual users of makepp.

  In case you did Perl programming for your makefiles, and you made use of some internals this would break your build. Therefore there is a temporary backward compatibility feature, to be removed in the future, which you can activate during installation. The new environment variable $MAKEPP_INSTALL_OLD_MODULES is checked for a list of old modules you want created as wrappers around the new ones. Additionally if you have makefiles you can't quickly change, which rely on these things being available without a use statement, you must prefix those modules with a +, to get them preloaded:

  MAKEPP_INSTALL_OLD_MODULES='+Glob Rule +MakeEvent'

• New makeppreplay, mppr utility to repeat some of makepp's actions very fast. New makepplog, mppl utility to see the log data readably, needed due to a changed file format. New makeppgraph, mppg utility to graphically analyze dependencies, includes and partially the reasons for a rebuild. New makeppinfo, mppi utility to see somewhat cryptically what makepp knows about some file.

• Ported to IBM z/OS Unix System Services. Only smart recursive make doesn't work.

• Makepp has become noticeably faster.

• Abolish the undocumented fancy renaming of only '.' to '_dot_' in variable and function names.

• New variable $/ for portable directory separator. Lots of Windows fixes, including -j (parallel builds) for Cygwin and MinGW and smart recursive builds on Cygwin.

• Also install abbreviations consisting of 'mpp' plus the first letter of every following word, e.g. 'mppc' for makeppclean.

• New `:build_check only_action' for commands that don't depend on the contents of their dependencies, like symlink creation, where it is used automatically.

• Removed --norc-substitution and --percent-subdirs and $(rc_substitution) and percent_subdirs. They are now to be given anywhere from target specific assignment to command line or environment vars makepp_simple_concatenation and makepp_percent_subdirs.

• New action syntax &perl_function 'arg 1' arg2 ... and &external-perl-script 'arg 1' arg2 ... New Perl function run.

  There are the following builtin commands: &chmod, &cp, &cut, &echo, &expr, &grep, &install, &ln, &mkdir, &mv, &perl, &preprocess, &printf, &rm, &sed, &sort, &template, &touch, &uninstall, &uniq and &yes, which can replace Unix commands of the same name, and more or less also the following: awk, chgrp, chown, head, m4, rmdir, tail and tr. They are also available stand-alone from the Shell. They can also be used as functions, e.g. $(&cat file), or as statements, or standalone.

  Note that, unlike earlier CVS versions, &cut -f, &grep -v and &sort -r now behave as in Unix. Note that in earlier CVS versions of &template @@ was processed before @, but now lines are consistently handled front to back.

• Added a global statement for sharing variables across makefiles. The assignment variant of export now works like a normal assignment, so you may have to change to :=. Added the override modifier to assignments. The define var := statement now optionally allows specifying the kind of assignment. And the forms with immediate evaluation retain the newlines in $(shell ...) or $(&command). There are new assignment operators &= for prepending, and ;= which is a = when set, but automatically turns into a := when first used.

• A makefile is now also found if it is called Makeppfile.mk.

• There are two new possible filenames for makefiles: RootMakeppfile or equivalently RootMakeppfile.mk. The presence of either of these gives your build tree a formal root, accessible through the new variable $(ROOT). The advantage is that this file is always loaded first, if present, allowing you to more easily create a build system where makepp can be called from anywhere, without telling it which makefile to start at.

  The root of the file system is then automatically marked for --dont-build, so that makepp doesn't go messing into other directories you include or use libs from, just because they happen to have a Makefile or sources.

  Usually this means that the root of your build system gets marked for --do-build. If, however, you say --do-build for something under your build system root, which doesn't inherit --dont-build, then instead your build system root gets marked for --dont-build.

• Removed command makeppclient since we never managed to let builds start significantly faster. The option --stop-after-loading is more beneficial.

• Interface definition files for SWIG (.i files) are now scanned for includes. Makepp now understands swig invocations. (SWIG stands for Simplified Wrapper and Interface Generator. It automatically generates all the wrapper functions to link your C or C++ code to a variety of other languages such as Perl, Python, Tcl, Ruby, OCaml, C#, etc. See http://www.swig.org/.)

• GNU Emacs 22 now has a standard makefile-makepp-mode, which is the default when visiting Makeppfile.

• $[VARIABLE] or $[function ...] is evaluated when reading a makefile line, so the variable may contain makepp syntax.

• $( ...) is now always a list in rc-style substitution, so that -I$( $(DIRLIST)) will leave no lonely option when DIRLIST is empty.

• You can now double-paren functions and multi-line lists, allowing things like $((perl if( $a < 5 ) { ... })).

• New c_compilation_md5 signature, which also allows adding whitespace where there was none and inversely. It also ignores whitespace and comments after the last token. This is useful for preventing a useless rebuild if your VC adds lines at a $Log$ tag when checking in.

• Implement $? exactly as GNU make does. New long name $(changed_inputs) for it.

• Implement $(error ...) and $(warning ...) as in GNU make.

• New method : build_check ignore_action to ignore changes to the action string.

• New statements ifperl, ifmakeperl, iftrue, ifntrue, ifsys and ifnsys.

• Conditionals ifxxx may now be grouped with and and or. When written on the same line after else, they create a branch of the same statement, rather than requiring nesting.

• Added support for dependencies on environment variables, using the :env rule option.

• Various signal handling fixes.

• New command makeppclean that efficiently removes generated files without loading makefiles.

• Ported to MinGW.

• New build caches, to cache files that are identical. This means that if you change a file and revert, then you can have makepp drop back to the immediately preceding .o file without rebuilding. Or you can share builds of identical files between separate source trees. Newly added grouping of build caches for big setups, possibly spanning several machines.

• At least Perl 5.6 is required.

Version 1.40 (December 2004)

Thanks to Anders Johnson and Daniel Pfeiffer for major contributions of code to this release.

• Too many bug fixes to list individually. Probably the most salient fixes are to make it work significantly more reliably on Cygwin, but there were also fixes to variable expansion, scanning, repositories, etc.

• Rewritten command parser and file scanner architecture so that it is more easily extensible. Makepp now supports Verilog (a language used for chip design) in addition to C++ and Fortran. It should be relatively straightforward to support additional languages. (Anders Johnson)

• New command makeppclient that lets builds start faster. (Daniel Pfeiffer)

• If you have Perl 5.6.0 or higher, HTML documentation now comes with the new working camel logo and syntax highlighting in the examples. (Daniel Pfeiffer)

• Numerous corrections and improvements to the documentation. (Mostly Anders Johnson)

• Support for GNU make's define statement to define multi-line variable values.

• $(PWD) and $(CURDIR) now work as in GNU make.

• New --keep-repository-links option to prevent makepp from deleting all the soft links it creates when making repositories.

• New --assume-old, --assume-new, and --dont-build options, and support for the -n option.

• Support for double colon rules has slightly improved, so that we can handle makefiles produced by MakeMaker without much trouble.

• Added syntax for performing Perl code as a statement and in rules perl { ... } or makeperl { ... }. Added functions for evaluating Perl statements $(perl ... ) or $(makeperl ... ). Added statement makesub { ... }. (Daniel Pfeiffer)

• Short options can now be directly followed by argument as in -j4. Documented options --jobs, --keep-going, --makefile, --what-if, --assume-new, --new-file, --assume-old & --old-file are now really accepted. (Daniel Pfeiffer)

Version 1.19 (July 2003)

Special thanks to Matthew Lovell and Chris van Engelen for lots of suggestions and tracking down problems in the code.

• Documentation was reorganized so that man pages as well as HTML pages can be produced, and a cookbook/FAQ was added (see the makepp_cookbook manpage).

• A “configure” script was added so installation is more like other software products from the user point of view.

• The $(origin ) function from GNU make is now supported.

• Target-specific variables are now supported as in GNU make, except that they do not propagate their values to dependencies.

• New functions $(find_upwards ) and $(relative_filename ) (contributed by Matthew Lovell) and $(relative_to ).

• In compilation commands, -I dir and -L dir are now supported and work just like -Idir and -Ldir.

• Recompilation of C files will now occur if a multi-line comment was inserted, or if the line numbering changed in any way. Previously it ignored newlines in computing the checksum, which meant that a change that affected debugger info might not force a recompilation.

• A bug in $(shell ) which caused it to return a null string occasionally (especially when the system was heavily loaded) was fixed.

• Unreadable files or directories suppress importing from repositories but are not matched by wildcards.

• A few other minor bugs were fixed.

Version 1.18 (January 2002)

The most important change was support for the Cygwin build environment. You can now run makepp with the Cygwin version of Perl; I do not think it will work properly with the native Windows version of Perl yet.

A few other bug fixes went into this release.

Version 1.10 (February 2001)

The most important change in this version is that makepp can accept a vastly larger number of makefiles without any command line options because of some changes to the implementation of recursive make. There are a few minor improvements in the GNU make compatibility, and a slight improvement in memory usage.

There are several user visible changes:

• c_compilation_md5 is now the default signature method. This means that by default, makepp won't recompile C/C++ modules if only whitespace or comments have changed.

• A new signature method md5 has been added, which runs an MD5 checksum on the file's contents. This is not enabled by default; makepp still uses its original method (exact_match) for any files other than C/C++ source files.

Because of these changes, makepp will recompile everything the first time you run it.

Version 1.05

In addition to bug-fixes, this version has one user-visible change. The --norc-substitution command line option was introduced to allow compatible handling of whitespace in makefiles.

Version 0.99 (January 2001)

In addition to numerous bug fixes, this version has several user-visible changes:

• Multiple targets for a rule are now treated in a way which is more compatible with old makefiles. Makepp has a heuristic algorithm for guessing whether the rule is supposed to build all targets at once or whether the rule needs to be invoked multiple times. I do not think this will break any existing makefiles, but it should allow makepp to work with many more makefiles designed for Unix make.

• The --traditional-recursive-make option can be used for legacy makefiles which use recursive invocations of make in a way that didn't work with makepp's default implementation.

• Repositories now work with libtool.

• Variable settings are now allowed with the load_makefile statement and with recursive make.

Version 0.95

This version has several user-visible changes:

• A tutorial on writing makefiles for makepp has been added.

• GNU make style conditionals (ifeq/ifneq/ifdef/ifndef) are now supported, as is the $(if ) function.

• By default, the % wildcard now matches only files within a directory; %.c is now equivalent to *.c, not **/*.c. The reason for this change was that rules almost never need to use the more complicated wildcard, and it often caused unnecessary directories to be searched. You can get the old behavior by specifying --percent-subdirs on the command line. (I'm curious how people feel about this change. Please let me know if you have good reasons for it being one way or the other.)

• By default, makefiles from any directory that contains a dependency, or that is searched by a wildcard, are loaded automatically. Usually this means you don't need any load_makefile statements at all. If this causes you problems, you can turn off implicit loading of makefiles by adding --no-implicit-load to the command line. (I'm curious whether people like or dislike implicit loading of makefiles.)

• A target may now be declared phony on the same line that defines the target by using the new $(phony ) function, like this:

  $(phony all): program_1 program_2

  The $(phony ) function simply returns its arguments, but marks them as phony targets. You can still use the older syntax that looks like this:

  all: program_1 program_2
  .PHONY: all 

  The $(phony ) function is an attempt to improve the readability of makefiles. I'd welcome other suggestions, as I'm still not entirely happy with the syntax.

Version 0.90

In order to support features like parallel make, most of the internals had to be reorganized or rewritten. The result is much cleaner and hopefully more reliable.

Bugs too numerous to mention have been fixed. In order to help ensure reliability, a test suite has been developed. It doesn't test absolutely everything yet, but it does test most things, and I hope to make it more extensive in the future. You can run it by typing makepp test in the makepp distribution directory. If an unmodified makepp fails the test suite, please let me know so I can fix it.

There are many new features:

• Repositories are now supported.

• It is now possible to specify different methods for calculating and comparing file signatures. For example, you can use an MD5 checksum ignoring comments and whitespace, or you can require merely that the target be newer than the dependencies (the method that the traditional make uses).

• Makefiles are only rebuilt if they are older than their dependencies; a different signature method is used in this special case.

• Parallel builds are now supported, though this is still an experimental feature. See the -j option.

• It is now possible to write your own functions. See the sub statement and the makepp_extending manpage for details.

• Filenames with characters like colon or space are now supported with a new quoting syntax.

• Synonymous, less cryptic names for automatic variables have been introduced to encourage more readable makefiles.

• Makepp now remembers the architecture of the machine you built on, and rebuilds if the architecture is different.

• Directories can now be targets or dependencies; a previous restriction that all directories had to exist before the start of the build has been lifted.

• Makepp now writes a log file called .makepp_log explaining why it rebuilt everything. This is extremely useful for debugging.

• The usual -k option for continuing to build even when an error occurs is now supported.

• The documentation has been reorganized and extended.

There are also (unfortunately) a few incompatibilities with previous versions:

• The file format for storing information about the last build has changed. Thus makepp will insist on rebuilding everything the first time you run the new version.

• load_makefile no longer supports targets or variable settings. It now supports loading a list of makefiles rather than just one, so the -F switch is now unnecessary (and no longer documented).

• Recursive make now ignores variable settings on the command line. This is necessary to load makefiles in a consistent way.

• $(INFERRED_OBJS) is no longer supported (since it was not a well-designed interface anyway). Use the newer $(infer_objects) function instead.

• $_ is no longer supported. Use $(foreach) instead.

• A few seldom used GNU make options such as -s, -n, and -q are no longer supported due to internal architecture changes. -n will probably be supported again in future releases.

• A man page is no longer provided since the documentation is no longer written in the Perl pod format. Use the HTML documentation instead.

• The automatic clean target is no longer supported. A better way to do it is with $(only_targets).