Making ConnectionsTopMultiple WalkersCreating a GBBopen ApplicationGoTo Top

Creating a GBBopen Application

GBBopen's Module Manager Facility provides mechanisms that make it easy to define and use your own GBBopen applications.


This exercise shows you how to:


Prerequisites

Step 1: Create your personal gbbopen-modules directory

Create a directory named gbbopen-modules in your “homedir” directory. For example:

  [~]$ mkdir gbbopen-modules
  [~]$
This is a special directory that is read by used by GBBopen to find applications when GBBopen is started using <install-dir>/initiate.lisp, as described in Steps 1 and 2 of the Enhancing Your Development Environment exercise.

Step 2: Create a module-definition file for the random-walk application

Recall that you created a directory to hold the random-walk application in Step 1 of Working Within a File exercise. I used these shell commands to create my directories:

  [~]$ mkdir tutorial
  [~]$ cd tutorial
  [~/tutorial]$ mkdir source
  [~/tutorial]$
Then you created the tutorial-example.lisp file in this source subdirectory. We said that we would explain why we created the source directory in a later exercise. Well, later has arrived.

Each GBBopen application is packaged in a directory that contains:

You already have the source directory and the tutorial-example.lisp source file. Next, we create the modules.lisp file for the application. (We will create a commands.lisp file for the random-walk application in Step 5.)

Use your Common Lisp editor to create a new file named modules.lisp in the tutorial directory (just as you created the tutorial-example.lisp file in Step 2 of Working Within a File exercise. Note that this file is not in the source subdirectory, but in the tutorial directory that contains the source subdirectory.

Type the following two forms into the new modules.lisp file:

  (in-package :module-manager-user)

  (define-module :tutorial
    (:requires :agenda-shell-user)
    (:files "tutorial-example"))
and then save the file.

Recall that the in-package form specifies the Common Lisp package that is made current when the file is compiled or loaded. A modules.lisp file should always specify the :module-manager-user package as the first form in the file.

The second form defines our application module, which we will name :tutorial. The :requires subform specifies that the :agenda-shell-user module must be compiled (if necessary) and then loaded before our :tutorial module. The :files subform specified the files that comprise the module. In our case, there is one file: tutorial-example.lisp. We leave off the .lisp file extension, as the Module Manager will add the appropriate source or compiled file extension for us.

Step 3: Add the random-walk application to your personal gbbopen-modules directory

The gbbopen-modules directory in your “homedir” is expected to consist of directories each containing an individual GBBopen application. We could place the random-walk application directly in the gbbopen-modules directory by moving the tutorial directory there. However, it is generally more convenient to use a symbolic link to point to the actual application directory. For example, an application can be provided to a number of users by creating a symbolic link to the application directory in each user's gbbopen-modules directory.

Unless you are running Windows, add the random-walk application to your gbbopen-applications by creating a symbolic link. For example:

  [~]$ cd ~/gbbopen-modules/
  [~/gbbopen-modules]$ ln -s ~/tutorial .
  [~/gbbopen-modules]$

Windows users

Instead of creating a symbolic link, GBBopen also supports a special “pseudo symbolic-link” file that can be used with Windows. This is simply a text file with the file extension .sym that contains the target directory path as the sole line in the file. For example, you could create the file tutorial.sym in your gbbopen-modules directory with:

  C:\tutorial\
as the sole line in the file.

Step 4: Try the :tutorial module definition

Let's try out our module definition. Exit Common Lisp and start a fresh Common Lisp session. If you have set up your environment according to the Enhancing Your Development Environment exercise, the following files should be loaded:

     ...
  ;; Loading <homedir>/shared-init.lisp
  ;;   Loading <install-dir>/initiate.lisp
  ;; GBBopen is installed in <install-dir>
  ;; Your "home" directory is <homedir>
  ;;     Loading <install-dir>/extended-repl.lisp
  ;;     Loading <install-dir>/commands.lisp
  ;;     Loading <install-dir>/gbbopen-modules-directory.lisp
  ;; No shared module command definitions were found in <install-dir>/gbbopen-modules/.
  ;; No personal module command definitions were found in <homedir>/gbbopen-modules/.
 cl-user>

Note that some basic GBBopen initialization files have been loaded for us as well as GBBopen's command definitions and any command definitions for applications linked from our <homedir>/gbbopen-modules/ directory. No module definitions have been defined yet, and GBBopen itself (or even the Module Manager Facility) were not loaded by <install-dir>/initiate.lisp.

Now, instead of loading the :agenda-shell-user module, let's load only the :module-manager-user module:

  cl-user> :module-manager-user
  ;; Loading <install-dir>/startup.lisp
     ...
  ;;     Loading <install-dir>/<platform-dir>/module-manager/module-manager-user.fasl
  ;;   Loading <install-dir>/modules.lisp
  ;; No shared module definitions were found in <install-dir>/gbbopen-modules/.
  ;; Loading personal module definitions from <homedir>/gbbopen-modules/...
  ;;     Loading <homedir>/tutorial/modules.lisp
  module-manager-user>
Note that when the Module Manager was loaded as part of loading the :module-manager-user module, the module definitions for our personal GBBopen modules were loaded automatically. (In this case, the <homedir>/tutorial/modules.lisp file.)

Now that we have defined our :tutorial module, we can use the compile-module REPL command, :cm, to compile (if needed) and load it. Before doing so, however, let's explore what is happening when :tutorial is being compiled by instructing the Module Manager not to create new compiled-file directories automatically (its default behavior). Enter the following in the REPL:

  module-manager-user> (setf *automatically-create-missing-directories* nil)
  nil
  module-manager-user>>

Now compile and load the :tutorial module:

  module-manager-user> :cm :tutorial
  ;;   Loading <install-dir>/<platform-dir>/tools/preamble.fasl
     ...
  ;;   Loading .../gbbopen/control-shells/agenda-shell-user.fasl

  Error: Directory <homedir>/tutorial/<platform-dir>/ 
         in module :tutorial does not exist.

  Restart actions (select using :c n):
    0: Create this directory.
    1: Create this directory and any future missing directories.
  module-manager-user>>
The :requires in our :tutorial module definition causes the :agenda-shell-user module (and before that, all its required modules) to be loaded for us. Then the Module Manager signals a continuable error, telling us that the directory to hold the compiled application files for :tutorial does not exist. Compiled files are put in a Common Lisp and platform-specific subdirectory, <platform-dir>, in our tutorial directory that mirrors the source directory. This organization makes it easy to use the application with a number of Common Lisp implementations and on a file system shared with a number of different hosts and operating systems.

By default, the Module Manager would have created the missing <homedir>/tutorial/<platform-dir>/ directory for us automatically (and continued compiling our :tutorial module), but we disabled automatic directory creation by setting /textbf*automatically-create-missing-directories* to nil. We still could have avoided this continuable error by providing the :create-dirs option to the :cm command:

  module-manager-user> :cm :tutorial :create-dirs
to allow the Module Manager to create the <platform-dir> subdirectory automatically for us. Since we did not do this, we can still continue from the error:
  Restart actions (select using :c n):
    0: Create this directory.
    1: Create this directory and any future missing directories.
  module-manager-user>> :c 0
  ;; Compiling file <homedir>/tutorial/source/tutorial-example.lisp
  ;; Loading <homedir>/tutorial/<platform-dir>/tutorial-example.fasl
  module-manager-user>

At this point, we've compiled and loaded our :tutorial application module.

Step 5: Create a command-definition file for the random-walk application

It is convenient to define a REPL command to compile and load you application (and any required GBBopen modules).

Use your Common Lisp editor to create a new file named commands.lisp in the tutorial directory. Type the following two forms into the new commands.lisp file:

  (in-package :common-lisp-user)

  (define-repl-command :tutorial (&rest options)
    "Compile and load the Random-Walk Tutorial Application Module"
    (startup-module :tutorial options :gbbopen-user))
and then save the file. A commands.lisp file should always specify the :common-lisp-user package as the first form in the file.

The define-repl-command form adds a REPL command, named :tutorial, to the set of handy REPL commands. The startup-module call does all the work associated with executing the command. The first argument to startup-module specifies that the :tutorial module will be compiled (if necessary) and then loaded by the Module Manager when the :tutorial command is issued. The second, options, argument passes any options given with the command to a compile-module call that is performed by startup-module. The third argument, :gbbopen-user is optional and specifies that the REPL's current package should be changed to :gbbopen-user after the :tutorial module is loaded.

Step 6: Try the :tutorial command

Let's try our command definition. Exit Common Lisp and start a fresh Common Lisp session. If you have set up your environment according to the Enhancing Your Development Environment exercise, the following files should be loaded:

     ...
  ;; Loading <homedir>/shared-init.lisp
  ;;   Loading <install-dir>/initiate.lisp
  ;; GBBopen is installed in <install-dir>
  ;; Your "home" directory is <homedir>
  ;;     Loading <install-dir>/extended-repl.lisp
  ;;     Loading <install-dir>/commands.lisp
  ;;     Loading <install-dir>/gbbopen-modules-directory.lisp
  ;; No shared module command definitions were found in <install-dir>/gbbopen-modules/.
  ;; Loading personal module command definitions from <homedir>/gbbopen-modules/...
  ;;     Loading <homedir>/gbbopen-modules/tutorial/commands.lisp
  cl-user>
Note that the commands.lisp file from the tutorial directory has been loaded by <install-dir>/initiate.lisp.

Now, we can compile and load the :tutorial module by simply issuing the :tutorial REPL command:

  cl-user> :tutorial
  ;; Loading <install-dir>/startup.lisp
     ...
  ;; Loading <homedir>/tutorial/<platform-dir>/tutorial-example.fasl
  gbbopen-user>
With the command definition in place, we are able to compile and load our random-walk application by issuing a single command, :tutorial.

Note a potential continuable error due to a missing <platform-dir> subdirectory can always be avoided by providing the :create-dirs option to the :tutorial command:

  gbbopen-user> :tutorial :create-dirs
to allow the Module Manager to create the <platform-dir> subdirectory automatically for us. However, since we created <platform-dir> in Step 4, we did not need to specify the :create-dirs option again in this step.

Step 7: Controlling automatic creation of missing subdirectories

If you prefer, you can specify that the Module Manager not create missing <platform-dir> directories and subdirectories automatically but, instead, signal an continuable error if a directory is missing (and the :create-dirs compile-module option was not specified). As we have seen, this behavior is controlled by the value of the symbol *automatically-create-missing-directories*, which is in the :common-lisp-user package and is set to t by default. If you would like to turn-off automatic directory creation by the Module Manager, add the following form to your shared-init.lisp file (in your “homedir” directory):

  (defparameter *automatically-create-missing-directories* nil)

I prefer to have the Module Manager generate the continuable error if it needs to create a <platform-dir> directory and I didn't specify :create-dirs when compiling a new module, so I set *automatically-create-missing-directories* to t in my shared-init.lisp file.

Step 8: Create and use an application-specific package

We have been developing our random-walk application in GBBopen's :gbbopen-user package. The :gbbopen-user package is convenient, and we could continue using it. However, if we develop multiple GBBopen applications in the :gbbopen-user package and load several of them at the same time, symbol-name clashes could occur.

To eliminate this possibility, we can create our own package for the random-walk application. First, let's determine what packages are being used by GBBopen's :gbbopen-user package. Evaluate the following:

  gbbopen-user> (package-use-list :gbbopen-user)
  (#<package PORTABLE-THREADS> #<package AGENDA-SHELL>
   #<package MODULE-MANAGER> #<package COMMON-LISP>
   #<package GBBOPEN-TOOLS> #<package GBBOPEN>)
  gbbopen-user>

Change tutorial-example.lisp

We want our new :tutorial package to use the same packages that the :agenda-shell-user package used. Edit your tutorial-example.lisp file and replace the :gbbopen-user package specification:

  (in-package :gbbopen-user)
with the following:
  (eval-when (:compile-toplevel :load-toplevel :execute)
    (unless (find-package :tutorial)
      (defpackage :tutorial
        (:use :common-lisp :module-manager :gbbopen-tools :gbbopen 
              :portable-threads :agenda-shell))))

  (in-package :tutorial)
and save the file.

Note the use of eval-when in the first form above. Normally, top-level forms in a file are not evaluated at compile time. In this case, however, we want to define the :tutorial package when needed, whether the file is being compiled or loaded. The eval-when special operator with the three situations (:compile-toplevel, :load-toplevel, and :execute) provides this behavior to the forms that it contains. Such eval-when forms are a standard Common Lisp idiom for compile-time and load-time evaluation.

An application feature

In my applications, I also add a feature to Common Lisp's *features* list to indicate that the application has been fully loaded. To do this, add the following at the end of your tutorial-example.lisp file:

  (pushnew :tutorial *features*)
and save the file.

Change commands.lisp

Next, edit your commands.lisp file and delete the :gbbopen-user package-name argument to startup-module:

  (define-repl-command :tutorial (&rest options)
    "Compile and load the Random-Walk Tutorial Application Module"
    (startup-module :tutorial options :gbbopen-user))
and add the package-name :tutorial in its place:
  (define-repl-command :tutorial (&rest options)
    "Compile and load the Random-Walk Tutorial Application Module"
    (startup-module :tutorial options :tutorial))
Save the file.

Change modules.lisp

Finally, we no longer need the :gbbopen-user package that is created by requiring the :agenda-shell-user module. Let's take a closer look at the :tutorial module that we defined:

  gbbopen-user> (describe-module :tutorial)
  Module :tutorial (loaded)
    Requires: (:agenda-shell-user)
    Fully expanded requires: (:module-manager :module-manager-user :portable-threads
                              :gbbopen-tools :gbbopen-core :polling-functions
                              :queue :agenda-shell :os-interface :gbbopen-user
                              :agenda-shell-user)
    Source directory: <homedir>/tutorial/source/
    Compiled directory: <homedir>/<platform-dir>/
    Forces recompile date: None
    Files:   Mar 24 06:02  tutorial-example
  gbbopen-user>
Although we only specified that the :agenda-shell-user module was required, our :tutorial module implicitly requires a number of packages that are required by the :agenda-shell-user module and its implicitly required packages. These are shown as the “Fully expanded requires” value.

If we look at the details of the :agenda-shell-user module we see:

  gbbopen-user> (describe-module :agenda-shell-user)
  Module :agenda-shell-user (loaded)
    Requires: (:agenda-shell :gbbopen-user)
    Fully expanded requires: (:module-manager :module-manager-user :portable-threads
                              :gbbopen-tools :gbbopen-core :polling-functions
                              :queue :agenda-shell :os-interface :gbbopen-user)
    Source directory: <install-dir>/source/gbbopen/control-shells/
    Compiled directory: <install-dir>/<platform-dir>/gbbopen/control-shells/
    Forces recompile date: None
    Files:   Mar 23 12:27  agenda-shell-user 
  gbbopen-user>
Note that the :agenda-shell-user module requires two modules: :agenda-shell and :gbbopen-user. We can eliminate the loading of the :gbbopen-user module by editing our modules.lisp file and delete :agenda-shell-user in the :requires option in our :tutorial module definition:
  (in-package :module-manager-user)

  (define-module :tutorial
    (:requires :agenda-shell-user)
    (:files "tutorial-example"))
and replace it with :agenda-shell:
  (in-package :module-manager-user)

  (define-module :tutorial
    (:requires :agenda-shell)
    (:files "tutorial-example"))
Save the file.

Step 9: Verify your changes

Let's make sure that everything is still working. Exit Common Lisp and start a fresh Common Lisp session. Next enter the :tutorial REPL command:

  cl-user> :tutorial
  ;; Loading <install-dir>/startup.lisp
     ...
  ;; Loading <homedir>/tutorial/<platform-dir>/tutorial-example.fasl
  tutorial>
Note that we are now in our newly defined :tutorial package. We should still be able to run the random-walk application:
  tutorial> (start-control-shell)
  ;; Control shell 1 started

  Walked off the world:  (56, 38).

  The random walk:
  1 (0 0)
  2 (-1 -1)
  3 (-8 -10)
  4 (0 -2)
  5 (-5 2)
  6 (3 11)
  7 (8 5)
  8 (12 2)
  9 (3 12)
  10 (10 4)
     ...
  55 (50 40)
  56 (42 47)
  57 (47 41)
  ;; Explicit :stop issued by KS print-walk-ks
  ;; Control shell 1 exited: 60 cycles completed
  ;; Run time: 0.01 seconds
  ;; Elapsed time: 0 seconds
  :stop
  gbbopen-user>

Installation-wide sharing

GBBopen also has an <install-directory>/shared-gbbopen-modules directory. As with our personal gbbopen-modules directory, this shared-gbbopen-modules directory is assumed to contain symbolic links (or “pseudo-symbolic-link” files on Windows) to individual GBBopen module directory trees.

This is the recommended mechanism for installation-wide managing and sharing of modules and applications, and if we wanted to share our random-walk application to everyone using our GBBopen installation, we could create our symbolic link (or “pseudo-symbolic-link” file) in the shared-gbbopen-modules directory.

Step 10: Add an “autorun” action

Suppose we want the random-walk application to run automatically when it is loaded. You could simply add:

  (start-control-shell)
as a top-level form at the end of your tutorial-example.lisp file. The problem with this is that sometimes you may want to compile and load the application without running it.

GBBopen's Module Manager Facility supports a convention that makes it easy to conditionalize load-time action execution via the value of *autorun-modules*. Normally, *autorun-modules* will be true, but it can be set to nil when a module is loaded with the :noautorun option.

Add the following at the end of your tutorial-example.lisp file:

  (when *autorun-modules* 
    (format t "~{~&~s~%~}" (multiple-value-list (start-control-shell))))
and save the file. We could have simply called start-control-shell when *autorun-modules* is true, but then we would not be able to see what values are returned by the Agenda Shell. The format form above prints each returned value on a separate output line.

By convention, the “autorun” form is placed at the very end of the file, immediately after the form to add :tutorial to Common Lisp's *features*. This is so that the :tutorial feature will be present during the “autorun” execution and thereafter—even if an error occurs when executing the “autorun” form.

Step 11: Try it out

Enter the :tutorial REPL command. The modified tutorial-example.lisp file should compile and load, followed by a random walk:

  tutorial> :tutorial
  ;; Compiling <homedir>/tutorial/source/tutorial-example.lisp
  ;; Loading <homedir>/tutorial/<platform-dir>/tutorial-example.fasl
  ;; Control shell 1 started

  Walked off the world: (11, -51).

  The random walk:
  1 (0 0)
  2 (6 7)
  3 (6 3)
  4 (3 -4)
  5 (10 -13)
     ...
  21 (-5 -46)
  22 (-9 -39)
  23 (1 -33)
  24 (8 -41)
  ;; Explicit :stop issued by KS print-walk-ks
  ;; Control shell 1 exited: 27 cycles completed
  ;; Run time: 0.01 seconds
  ;; Elapsed time: 0 seconds
  :stop
  tutorial>

Let's try it again:

  tutorial> :tutorial
  tutorial>
This time, nothing happened. Why?

Since no source files were modified, the Module Manager knows that the latest compiled files for the :tutorial module and its required modules have all been loaded. So, because the tutorial-example file is not loaded, its “autorun” conditional form is not evaluated.

We can tell the Module Manager to always reload the tutorial-example file by editing our modules.lisp file and adding the :reload file option to the :tutorial module definition:

  (define-module :tutorial
    (:requires :agenda-shell)
    (:files ("tutorial-example" :reload)))
Note that once a file has one or more options, the file name and its options are enclosed in parentheses.

Save the modified modules.lisp file. Now, if you try the :tutorial command, the tutorial-example file will always be loaded and its “autorun” form evaluated:

  tutorial> :tutorial
  ;; Loading <homedir>/tutorial/modules.lisp
  ;; Loading <homedir>/tutorial/<platform-dir>/tutorial-example.fasl
  ;; Control shell 1 started
     ...
  tutorial>
The Module Manager noticed our updated modules.lisp file and loaded it, redefining the :tutorial module definition, and then followed our :reload specification.

Let's try the :tutorial command one more time, just to be certain that :reload is happening when no files have been updated:

  tutorial> :tutorial
  ;; Loading <homedir>/tutorial/<platform-dir>/tutorial-example.fasl
  ;; Control shell 1 started
     ...
  tutorial>

Module Manager propagation

The Module Manager's compile-module function also accepts a :reload option, so we might be tempted to simply add that option when we specify our :tutorial command:

  tutorial> :tutorial :reload
  ;;   Loading <install-dir>/<platform-dir>/module-manager/module-manager.fasl
  ;;     Loading
  <install-dir>/<platform-dir>/module-manager/module-manager-user.fasl
     ...
  ;; Control shell 1 started
     ...
  tutorial>
however, this would also reload all the files of very module required by the :tutorial module, as the startup-module function that we used in defining our :tutorial REPL command always adds a :propagate option to the options that we provide to the command. We could override (cancel) this propagation behavior by adding the :nopropagate option when we specify our :tutorial command:
  tutorial> :tutorial :reload :nopropagate
  ;; Loading <homedir>/tutorial/<platform-dir>/tutorial-example.fasl
  ;; Control shell 1 started
     ...
  tutorial>
which would eliminate reloading of all but the files in the :tutorial module. Since we only have one file (at the moment) specified in our :tutorial module definition, this behavior is equivalent, but less convenient, than specifying the :reload file option to our tutorial-example file in the module definition. If we had a number of files, however, we would probably only want the last one reloaded every time.

Module Manager memories

The Module Manager also remembers the last module (and any provided options, such as :create-dirs) that was specified to a :cm (textbfcompile-module) or :lm (load-module) command. The startup-module function that we used in defining our :tutorial REPL command performs an implicit :cm command for us, so we could have alternatively typed the :cm REPL command rather than :tutorial once we have issued the first :tutorial REPL command:

  tutorial> :cm
  ;; :cm :tutorial :propagate
  ;; Loading <homedir>/tutorial/<platform-dir>/tutorial-example.fasl
  ;; Control shell 1 started
     ...
  tutorial>
Just the thing for lazy typists like me! Note that the :cm command echos the full (implicitly completed) command with the remembered module name and any remembered options.

Step 12: Add a second source file

Reloading the entire tutorial-application file in order to evaluate our “autorun” form is still a bit heavy handed. There are two ways to improve this situation, and both involve placing the “autorun” form in a separate file. Use your Common Lisp editor to create a new file named autorun.lisp in the source subdirectory of the tutorial directory. Type (or copy) the following two forms into the new autorun.lisp file:

  (in-package :tutorial)

  (when *autorun-modules*
    (format t "~{~&~s~%~}" (multiple-value-list (start-control-shell))))
and save the file.

Next, edit the tutorial-example.lisp file and remove the “autorun” form from the end of the file:

  (when *autorun-modules* 
    (format t "~{~&~s~%~}" (multiple-value-list (start-control-shell))))
Save the file.

We could define a second random-walk application module, say :run-tutorial, in our modules.lisp file that contains the new autorun.lisp file and requires our current :tutorial module. This definition would look like:

  (define-module :run-tutorial
    (:requires :tutorial
    (:files ("autorun" :reload))))
However, we can avoid creating a new module by simply adding the autorun file to our current :tutorial module definition.

Edit the modules.lisp file in the tutorial directory and remove the :reload file option from the :tutorial-example file specification:

  (define-module :tutorial
    (:requires :agenda-shell)
    (:files ("tutorial-example" :reload)))
and then add a new line for the autorun file with the :reload file option:
  (define-module :tutorial
    (:requires :agenda-shell)
    (:files "tutorial-example"
            ("autorun" :reload)))
and save the file.

Step 13: One last check

Let's double-check that everything is working:

  tutorial> :cm
  ;; :cm :tutorial :propagate
  ;; Loading <homedir>/tutorial/modules.lisp
  ;; Compiling <homedir>/tutorial/source/tutorial-example.lisp
  ;; Loading <homedir>/tutorial/<platform-dir>/tutorial-example.fasl
  ;; Compiling <homedir>/tutorial/source/autorun.lisp
  ;; Loading <homedir>/tutorial/<platform-dir>/autorun.fasl
  ;; Control shell 1 started
     ...
  tutorial>
The modified files are compiled and loaded and the Agenda Shell is invoked.

Let's try it one last time, just to be sure that the application runs when no files have been modified:

  tutorial> :cm
  ;; :cm :tutorial :propagate
  ;; Loading <homedir>/tutorial/<platform-dir>/autorun.fasl
  ;; Control shell 1 started
     ...
  tutorial>

Congratulations! It's time to move to the next exercise.


The GBBopen Project


Making ConnectionsTopMultiple WalkersCreating a GBBopen ApplicationGoTo Top