WHAT IS GPT The Grid Packaging Tools are a collection of modules and scripts which are used to manage component based distributions and installations. The modules and scripts are written in Perl. A typical component called a GPT package consists of a binary or source distribution for the component plus an XML based definition file that tells GPT how to manage the component. In theory, GPT should run anywhere Perl itself works. Documentation for specific architectures will come with later releases. SPECIFICS OF GPT-4 GPT-4 is a complete rewrite designed to generalize the deployment features which older releases implemented. Previous version of GPT had features that specifically applied to the Globus Toolkit Version 2.x. This release is designed to be more independent of specific Grid components so that the code base will remain more stable. The features changes are summarized as follows: More descriptive XML. GPT now allows packages to advertise multiple features with a set of name-version pairs called aliases. GPT also supports more ways to specify version requirements. GPT dependencies have a new 'composite' attribute which allows a set of packages to be deployed as a single entity. Deployment Stages. GPT now is aware of a set of script hooks called deployment stages that allow packages to run scripts during the deployement process. GPT executes the following stages automatically: PreInstall - Runs before a binary package is installed. Install - Runs after a binary package is installed. Uninstall - Runs before the package is removed. PostUninstall - Runs after the package is removed. In addition, GPT provides the script gpt-run which allows other stages to be run on a set of packages. A stage label can be any valid XML attribute value. Deployment Actions. GPT encodes the execution of deployment scripts in a flexible set of XML structures, which can be used to taylor the invocation of the scripts. Here is a brief summary of the structures available: Action - XML structure which encodes the script invocation. Sequence - Contains a list of Actions, Checks and a working directory these need to be executed in. A Sequence can also contain other sequences as well as references to action definitions. DeploymentStage - A Sequence that is executed when a stage is 'called' A stage can be any arbitrary label although GPT reserves some stage names as was mentioned in the previous section. The structure includes a set of variable references which GPT uses for variable substitutions. DeploymentStages can also 'own' their own files which GPT will deploy automatically. The PreInstall, and PostUninstall stages are special examples of this. When the PreInstall stage is executed, GPT will install the files 'owned' by the stage before executing it. With the PostUninstall stage, GPT will not remove the files owned by the stage until after it is executed. ActionDefinition - A Sequence that is installed by a package so that it can be used by the deployment scripts of other packages. This allows common deployment tasks to be scripted in one place rather than replicated across a set of package data. ActionDefinitions have variable references and file lists just like the DeploymentStages. ActionReference - A namespace/name pair that tells GPT to execute an ActionDefinition deployed by another package. ActionFunction - A namespace/name pair that tells GPT to run a GPT action extension. Check - An Action whose return value GPT uses to decide whether to execute other actions (the implementation of this is still fuzzy). Variables. GPT provides a means of passing information to deployment scripts with variables. Variable values can be set from the command line, stored in an installation or GPT's configuration settings, set by a bundle definition, or set as part of the package data. Variables values are substituted in deployment scripts using references which look like 'NameSpace::Name'. Substitutions are done on any deployment action as well as some directory references (i.e. the working directory for an action sequence). Locations. Previous GPT releases had scripts that made implicit and arbitrary assumptions about where packages are located. GPT-4 scripts can be run on any combination of installations, bundles, and/or package directories. The only remaining resriction is that only one 'target' (from which packages are added or removed) can be specified at a time. GPT also allows installations to be 'linked' so that dependency trees can span several installation locations. Finally GPT can be configured to recognize environmental variables which contain installation locations (GLOBUS_LOCATION is no longer required). Extensions. The GPT distribution is now a set of components which have extensions designed in. The core GPT-4 release is actually a smaller footprint than previous releases because it no longer requires any perl modules written in 'C'. However, the feature set of GPT can now be modified by installing other packages that uses GPT extensions. The following extensions are supported by GPT: Flavors - Tells GPT what a valid flavor label is. XML - Tells GPT how to convert XML into GPT's internal data representation. Logging - Tells GPT how to output errors, script output and its own messages. Configuration Parameters - Adds switches to the GPT scripts. Configuration Probes - Adds tests to gpt-config -probe. Inputs - Extends the types of GPT stuff (i.e packages, bundles, installations) GPT can scan in as script input. Package and PkgSet Operations - extends how GPT adds and removes packages. Archive - extends the way GPT packs and unpacks packages and bundles Action Functions - Adds functions which execute internal GPT stuff when called by an action sequence. For example, the function 'VarSync::reload' tells GPT to read into memory the variable cache from an installation. The function 'Setup::finish' tells GPT to mark setup aliases as done in the installation variable cache. Dependency Sorting. GPT now sorts packages in dependency order when running deployments. Flavor Deployment. Previous releases of GPT had a complicated set of rules when deploying multiple flavors of a package to a single location. GPT-4 has replaced this with a package data attribute called FlavorDeploy which packagers can use to choose one of the following behaviours: CoExist - Multiple flavors of the same package can be deployed in the same location. CoDeploy - Multiple flavors of the same package are deployed by copying the files into unique backup locations and setting a current default flavor for the package in the installation variable cache. Conflict - Multiple flavors of the same package are not allowed to be installed into the same location. File Ownership. GPT-4 has expanded file ownership by packages in the following ways: GPT now tracks ownership of directories within an installation so that empty directories are removed when the last file is removed. GPT now has a codeploy feature which creates backups of files and swaps these files around when flavors are changed. GPT now allows packages to own non-relocatable files. So for example files located in /etc can now be part of a GPT package. GPT now allows packages to add user and temporary file entries to the installation filecache. Temporary files are then removed when their package is removed. User files are copied to a *.bak location when a package is removed. OVERVIEW OF THE SCRIPTS Here is the current set of scripts for GPT-4. use the switch '-man' to get a detailed description of each script. gpt-install - The script to use when installing packages. In the next release this script will also support installing source packages. gpt-uninstall - The script to use when removing packages. gpt-query - Script which can be used to extract package data from installations, bundles, and package directories. gpt-verify - Script which runs various consistancy checks on package data. gpt-diff - Script which compares the versions of two collection of packages (i.e installations, package directories, or bundles). gpt-run - Script which runs a deployment stage on a set of packages. gpt-vars - Scripts which sets and gets GPT variables. gpt-config - Script which configures GPT itself. gpt-default-flavor - Script used to list and swap the flavors of co-deployed packages. CONVENTION CHANGES Input packages, bundles need to be referred to the absolute path or use './'. Examples: /Users/mbletzin/gpt-test/BundleTest-1.2.gpt - good absolute path ./BundleTest-1.2.gpt - good starts with './' ../gp-test/BundleTest-1.2.gpt - bad BundleTest-1.2.gpt - bad ~/Install/gpt4/testinstall - bad $HOME/Install/gpt4/testinstall - ok if shell expands $HOME Bundles and packages have now use the *.gpt extension. GPT data files use the *.xml extensions. The flavor 'noflavor' and the package type 'pgm' are considered default by GPT nd so are never displayed to the user. For example, package 'foo-noflavor-pgm' is displayed as 'foo'. WHY IS THIS AN ALPHA All testing for this release has been internal. GPT-4 does not deploy source packages yet. A GPT extension needs to be written to support the flavor building of the Globus Toolkit that was in the previous releases. The code needs to be optimized. The dependency sorting makes the code slower than previous releases. Perl has some profiling and benchmarking tools available that we have not used yet. We will probably also provide a 'BulkInstall' bundle feature that will allow a large set of packages to be installed as one archive rather than a collection of individual archives. File installation needs more work. Perl has no architecture independent way to determine and set file permission and ownership. Because of this GPT is ignoring these for now. No documentation is available yet BACKWARDS COMPATIBLITY This release supports GPT-3.x installations, binary packages, and binary bundles. Source packages and bundles will not be supported because the old build instructions has too many hidden behaviours that are not worth replicating. GPT-2.x is not supported at all. GPT-4 scripts will read in 3.x installations, bundles and packages. gpt-install will convert 3.x bundles and packages into GPT-4 XML during installation. gpt-install will convert a 3.x installation into a GPT-4 installation when something is installed or removed. GPT-4 will not create any 3.x XML. To run the GPT-3.x postinstall, execute the following: gpt-run -deptype=setups stage=PostInstall