Module Manager Facility

Module Manager Facility

The Module Manager Facility provides a lightweight and easy to use mechanism for compiling and loading module files. The facility keeps track of the dependencies between modules and the modules that have been compiled and loaded. The Module Manager Facility was designed to be easy to use, yet it is powerful enough to manage substantial software projects, each operating on multiple hardware platforms and Common Lisp implementations. Other, more complex, defsystem packages, such as (ASDF), are available, but their complexity makes defining and maintaining correct system definitions difficult.

The :module-manager module is automatically loaded by the GBBopen <install-directory>/startup.lisp file (by the module-manager-loader.lisp file located in the module-manager subdirectory). If a gbbopen-init.lisp file (source or compiled) is present in the user's “home” directory, it is loaded immediately following the loading of the Module Manager Facility. A personal <homedir>/gbbopen-init.lisp file is very useful for defining GBBopen parameters and other personalizations (see the GBBopen start-up description).

Key concepts

The Module Manager Facility incorporates several important concepts:

• Dependencies are maintained among modules, rather than among individual files. Defining compilation and loading dependencies at the file level (and maintaining those dependencies as applications and libraries evolve) is both complex and error prone. Instead, each Module Manager module definition specifies a total ordering of any other modules that it requires (directly) as well as a total ordering of the individual files that comprise the module. (It is an error for a file to be associated with more than one module.)
• A total ordering of all the individual files that are required and used by a module is generated from the module definitions. The Module Manager Facility expands a module's specified required modules recursively to obtain a fully expanded sequence of all the modules that are required (directly or indirectly) by the module. The individual files associated with each module in this fully expanded sequence of required modules define the sequence of all the individual files that must be possibly compiled and/or loaded as part of compiling or loading the module.
• Recompilation and reloading dependencies are specified as options associated with individual files. For example, if the option :forces-recompile is specified for a file and that file needs to be recompiled, all files later in the total file sequence need to be recompiled and reloaded. This simple, “somewhat aggressive” recompilation mechanism replaces complex definitions of pairwise dependencies between individual files (and the risk of missed and no-longer-appropriate dependencies), but at the cost of some unneeded recompilation. It also simplifies handling dependencies that exist between some files in a requiring module and files in ones of that module's required modules when each module is supported by a different developer/maintainer. A developer only needs to consider whether a file includes forms (such as macro definitions and compiler transforms) that may effect the compilation and/or loading of downstream files (whether or not those files are known to the developer). With today's fast Common Lisp compilers, some unnecessary recompilation is a small price to pay for easier and more likely to be correct dependency specifications.
• The fully expanded required module sequences of all modules (and therefore the compilation and loading sequence of files for all modules) must be consistent. This requirement is checked by the Module Manager and ensures that the ordering of file compilation and loading will be maintained no matter what module (or modules) is being loaded. Ordering consistency is important in avoiding unnecessary recompilation of unchanged files if different modules were allowed to specify their sequence of overlapping required modules differently. Although required-module ordering consistency is a very strong constraint—especially when independently development modules use common libraries as required modules—in practice, existing required-module orderings quickly eliminate most “otherwise arbitrary” required-module orderings. Where two required modules are not constrained as to their pairwise ordering, the convention of specifying the module that is earlier alphabetically first is recommended.
• Separate compiled-file directory trees are created for each Common Lisp implementation and platform. These compiled-file directories mirror the directory structure of the original source-file directory tree of a system. This separation makes packaging and distributing compiled files for one or more platforms (with or without the source tree) simple by including only the desired directory trees.
• Freezing a release (and providing patches later, if needed) are important phases of the release life cycle of some applications. Support for freezing and patches that are in keeping with the above concepts is provided explicitly by Module Manager operators.

Stand-alone usage

To bootstrap the Module Manager Facility stand-alone (separate from the GBBopen Project software tree), do the following:

• create a “root” directory to contain the Module Manager software tree
  (for example: $ mkdir my-tree)
• create the Module Manager portion of the source tree in the newly created “root” directory
  (for example: $ cd my-tree ; mkdir -p source/module-manager)
• copy the module-manager-loader.lisp,   module-manager.lisp,   and module-manager-user.lisp files into the source/module-manager directory
• start your Common Lisp implementation and then load the module-manager-loader.lisp file
     > (load "my-tree/source/module-manager/module-manager-loader")
• compile the :module-manager and :module-manager-user modules:
     > (module-manager:compile-module :module-manager-user :create-dirs :propagate)

Entities

• *automatically-create-missing-directories*
• *autorun-modules*
• *patches-only*
• compile-module
• continue-patch
• define-module
• define-relative-directory
• define-root-directory
• describe-module
• describe-patches
• finish-patch
• get-directory
• get-patch-description
• get-root-directory
• load-module
• load-module-file
• module-directories
• module-loaded-p
• patch
• patch-loaded-p
• show-defined-directories
• start-patch
• with-module-redefinitions

Module Manager Facility