GreenSocs Package Management Manual

Introduction

The GreenSocs Package Manager, or GSPM for short, is a tool to easy the instalation and execution of programs and libraries that chose to use it. It was done mainlly focused on the Eletronic System Level (ESL) area, but in reality can deal with any kind of software.

One of the many problems GSPM tries to solve is to handle the install locations of the libraries. Each installed part have the chance to set it's own path and other attributes. So other related tools, (like makefiles, compilers, GUIs, etc) can access this information easilly.

Other important point that GSPM allows is the instalation of concurrent versions of the same tool. The system is based on envirionment variables, so even the same user can use different versions at the same time, but in different shell instances. The loaded version for the current shell can change dynamically at any time.

GSPM was projected to allow any non-administrator user to set its own world, which is a folder that can be located anywhere in the computer and contains the installed packages. This system does not touch any files outside this folder. There can be any number of worlds in the same machine. So a group of users can chose to install the most used packages in a globally accessible world while each single user installs the packages she uses, or beta versions, in her private (or again, shared) world.

The foundation of GSPM is based on Debian tools (http://www.debian.org) to manipulate packages and dependencies, but can run on non-Debian systems too, and on Environment Modules (http://modules.sf.net) to dynamically set variables allowing the load and switch of packages.

GSPM is free open source software and can be downloaded from http://www.greensocs.com. Installation is hassle-free, just untar the downloaded tarball in any directory you want, like:

  1. tar xzf gspm_2007.03-1*.tar.gz

Now we are ready to use GSPM!

Using GSPM

There are two important concepts when using GSPM: enter the world and load packages, as can be seen in the following sections.

Enter the world

One feature of GSPM is the ability to have many worlds installed in the same machine. Then the concept of entering the world. This is necessary to select the default world to use, which is the directory where the new packages will be installed. Every time you open a new shell and want to work with GSPM, you need to enter the world. The command to enter the world is to source a script into the current running shell, which will set some environment variables and shell functions. The directory for new instalations will be set to the same as the sourced script location:

  1. source gspm/gspm.bashrc

As the name implies, this script will only work for Bash shells. We believe it is the most used shell. If you need support for other shell, please contact greensocs@greensocs.com">greensocs@greensocs.com.

Load packages

After we enter the world, we have many packages to chose from and load. There can be many versions of the same package in the same world, so, another concept for GSPM is to load a package. This concept is important for two reasons. One is to switch on/off the use of a package, other is to dynamically select the version of the package you want.

For example, when a code instrumentation library is loaded, then code that is aware of this library will be instrumented automatically, without any other user intervention like compiler flags. For this to work, users will have to use the GreenSocs Compiler Wrapper gsc, that controls the options to the GNU Compiler gcc.

We load packages with the gs-use shell function. Just list as arguments the name of the packages you want to load. A package will be installed the first time it is needed, so, only in the first time, there will be a delay to download and install the required package version. The command to load packages is:

  1. gs-use gsc systemc

There are some packages that is important to have more then one

version installed concurrently, for example, libraries and beta versions of programs. In these cases, we can select the version to load using a slash and the version after the package name. To load an older SystemC, we use:

  1. gs-use systemc/2.1

Important: Which version is loaded when only the package name is given? The answer to this question depends on two cases. If there is already one or more version with this name installed, then the last one in the alphabetically sorted list of versions will be loaded. In the other case, where there is no previous instalation of this package, a default version, which is normally the highest stable version of the package, will be installed and loaded.

Unloading packages

The command gs-use can also be used to unload packages. Use one of the options -r or --remove or --unload. For now, only one package can be unloaded at a time. Example:

  1. gs-use -r systemc

Updating packages

The GSPM system never update packages by itself. The user must explicitly issue a command for that. This is important so there can be no surprises.

The first thing to do is to update the package index files, then the system will get to know the existence of new and updated packages. The command is:

  1. gs-use --update

Now we can list and have the chance to upgrade all the packages in the GSPM system. Don't worry, the next command will list the possible upgrades but will ask before going any further. If you want to upgrade just some of the packages, answer no after reading the list of possible upgrades.

  1. gs-use --upgrade

Now we know the packages that can be upgraded. To upgrade selected packages, use the following command. For example, upgrading the gsc package: NOTE: use the Debian package name, which is different from the GreenSocs package description, please see [TODO] for details

  1. gs-apt-get install gsc

Uninstalling packages

When we don't need a packages anymore, we can uninstall it from the GSPM system using the following command:

  1. gs-use --uninstall systemc/2.1

Creating a new package

This section is about taking an existing source directory, which already has its own means of compiling to binaries, and creating a source package that is compatible with GSPM conventions.

Note that package developers handles source packages, but they can choose if their source packages will be available for users or not. In the last case, only the binary packages will be available and the intellectual property will be preserved.

The next few sections will guide the developer from the current source code to a GSPM-compliant source package.

Step 1: Preparation

The GSPM system uses some Debian tools to control the packages. Also some, but not all, the Debian rules are used. The changes were done to allow easier transformation of source code into packages and to allow more then one version of a package to be installed concurrently, with dynamic switching between them.

To start the process, chose a name for the new package. There are some restrictions for the name. It must be lower case letters or numbers, but can contain the symbols hyphen ('-') and plus ('+'). Note that space, underscore ('_') and dot ('.') are forbidden!

Also chose the version number. It must start with a number, but can contain letters and the symbols hyphen ('-'), plus ('+') and dot ('.'). If that piece of software is not numbered with versions like X.Y.Z, but with some kind of date, feel free to use that date as the version number, prepended with a "0.0." (just in case in the future there is a nice version like 1.0). So, if the release or snapshot date was 19th of December, 1998, you can use the version string of 0.0.19981219. This text will refer to the package name as package and the package version as version. You should start packaging with a completely clean (pristine) source directory, or simply with freshly unpacked sources. Then take these steps:

  1. Rename the source directory to package-version.
  2. Create a tarball with a special name:
tar czf package_version.orig.tar.gz package-version

Step 2: Make the application portable

We start from the point that the tool has a working installation process and runs correctly outside GSPM. This section will help you to verify if the tool is portable and, if not, guide to convert it to be portable.

We say an application is portable if the directory which it is installed can be freely moved to a new location. Note that portability is a good characteristic even without considering GSPM. For example, to be able to install a tool into a memory key from one machine and mount/run it in another computer in possibly a new location.

A simple test to try this requirement is to compile the package in one directory, move this directory to other location and try to run it from this new place. If the tool does not run correctly, then the developer that is packing it for GSPM will have the work to make the tool portable.

An application is portable if each and every reference to it's installation directory is relative to the current location (instead of the hard-coded directory where the tool was installed). This means that some calculation must be done to archive the actual path, for example apending an environment variable value in front of this relative location.

Now let's make the application (or library) compatible with GSPM. The GSPM system allows only one version of each package to be loaded at a time (for each shell). When one package is loaded, an environment variable is set with the absolute path where the tool resides. This variable is named by the tool name, but with all letters capitalized and hyphens converted to underscores (example: package gs-example will have the variable GS_EXAMPLE set to the installation path when loaded).

The work now is to find and replace each absolute reference to the installation directory in the source code to use this environment variable instead. An example code in C language to get the installation path is bellow.

  1. #include <stdlib.h>
  2.  
  3. int main()
  4. {
  5.    char *instpath;
  6.  
  7.    /* Get a pointer to the variable value in the environment */
  8.    instpath = getenv("GS_EXAMPLE");
  9.  
  10.    /* Do something... */
  11. }

Don't forget to also change the references that are done from configuration files. If there is an absolute path inside a configuration file, the part of the code that reads this file should be changed to use the environment variable too.

Also note that the configuration files are commonly located inside a subdirectory of the installation directory. For a portable application, this is essential. Please, verify if this is true for your application, or change the source code in a way to make this condition true.

Congratulations! Your program is now portable! It is always good to test the portability outside the GSPM world. Install your program, move it's directory and manually set the environment variable to the new location. If it runs ok, we are ready for the next section.

Step 3: Create Debian files

The Debian tools requires that some files are present in order to operate. These files are located in a directory called debian inside the source root directory. The 3 files that must exist in this directory are named: changelog, control and rules.

The changelog file

It is always good to have a file to log the changes in the project. But this changelog that is inside the debian directory is used to log only the changes occured when adapting the original software to be used with the GSPM. It is required because the package version number is taken from it.

The format of this file is a series of entries like the one bellow.

  1. <package> (<version>-<debversion>) <distribution>; urgency=<urgency>
  2.         [optional blank line(s)]
  3.    * change details
  4.      more change details
  5.         [optional blank line(s)]
  6.    * even more change details
  7.         [optional blank line(s)]
  8.  -- <maintainer name> <<email address>>[two spaces]  <date>

It is important to note some restrictions of this format:

  1. The first "title" line with the package name should start at the left hand margin; the "trailer" line with the maintainer and date details should be preceded by exactly one space. The maintainer details and the date must be separated by exactly two spaces.
  2. The change details may in fact be any series of lines starting with at least two spaces, but conventionally each change starts with an asterisk and a separating space and continuation lines are indented so as to bring them in line with the start of the text above. Blank lines may be used here to separate groups of changes, if desired.
  3. The date should be in RFC822 format. Use the command gs-822-date to show the current date in the correct format.
  4. The <debversion> normally has the value 1. It is incremented only in the case of errors in a previous trial to package.
  5. The <distribution> can be stable, unstable or testing.

The changelog for our gs-example package in version 1.0 would be:

  1. gs-example (1.0-1) stable; urgency=low
  2.  
  3.   * First release
  4.   * It has many features like this and this
  5.  
  6.  -- GreenSocs <greensocs@greensocs.com>  Thu 31 May 2007 19:36:26 -0300

The control file

This file is necessary to satisfy Debian rules. The format is very simple. All the lines have the form Variable: value. The first part is about the source package that will be generated. The second part is separated from the first one by a blank line an is about the binary package.

A realistic example for the SystemC package is shown bellow:

  1. Source: systemc
  2. Section: devel
  3. Priority: optional
  4. Maintainer: GreenSocs <greensocs@greensocs.com>
  5. Standards-Version: 3.6.1
  6.  
  7. Package: systemc
  8. Architecture: any
  9. Description: C++ class library to describe and simulate hardware
  10.  SystemC is a C++ class library that helps to describe new hardware
  11.  and simulate it.

More details on the format:

  1. Keep unchanged the Standards-Version field.
  2. The Section field is optional. The value can be an arbitrary alphanumerical word, but we preffer to follow the Debian rules an use one of these: admin, base, comm, contrib, devel, doc, editors, electronics, embedded, games, gnome, graphics, hamradio, interpreters, kde, libs, libdevel, mail, math, misc, net, news, non-free, oldlibs, otherosfs, perl, python, science, shells, sound, tex, text, utils, web, x11
  3. The Priority field, if present, should have the value optional. Unless for core GSPM packages.
  4. The Architecture field is mandatory, and normally has the value any, so it is substituted by the architecture where the binary package is generated.
  5. The Description field is optional. It is composed by a 1-line description in the first line and by a longer description in the next lines. The long description must be indented by exactly one space for all lines.
The rules script

This is the script used to build the binary package from this source package. As a script, it must have the executable attribute set (chmod +x debian/rules). It is really a Makefile, but it is different from other ordinary Debian rules file. For GSPM, this file is composed only by variable definitions. This variables configures a system Makefile that is able to compile (when necessary) and build the binary package.

The full rules file the is actually used for the SystemC source package is bellow:

  1. #!/usr/bin/make -f
  2.  
  3. # Package properties
  4. PACKAGE = systemc
  5. VENDOR = OSCI
  6. VARIATION = gcc$(GCC_VERSION)
  7. VERSIONED = 1
  8.  
  9. # Dependencies
  10. BUILD_DEPENDS = gcc
  11. LOAD_DEPENDS = gcc/$(GCC_VERSION)
  12.  
  13. # Other
  14. CFLAGS = -O2
  15.  
  16. # Modulefile
  17. define MODULEFILE_ENVIRONMENT
  18.    prepend-path CPLUS_INCLUDE_PATH "$$GSROOT${PREFIX}/include"
  19.    prepend-path LIBRARY_PATH       "$$GSROOT${PREFIX}/lib"
  20.    prepend-path LD_LIBRARY_PATH    "$$GSROOT${PREFIX}/lib"
  21.    prepend-path LIBRARIES          "-l${PACKAGE}"
  22. endef
  23.  
  24. # Call the real Makefile
  25. include $(MAKEFILE_PACKAGES)

Note that this file doen't have the package version. The version is taken from the changelog file. So it is expected that this file doesn't change between different versions of the same software.

In this case, there is nothing to tell if the package needs to be compiled or not. But the system makefile, called at the last line, is intelligent enough to find out when the package supports the default GNU installation with Autotools. It does that by searching the files configure and Makefile.in in the package directory. In this case, it runs a default configure; make; make install procedure.

When a package needs to be compiled by a non-default procedure, you can set a variable called BUILD_COMMAND. This variable is passed verbatim to the shell when building the package in the user system. Please, don't quote it. Just write after the igual sign ("=") what you would write into a shell window. You have just one line of arbitrary size, but many commands can be issued by separating them with ; or &&. Have in mind that you should use only standard tools to create your package.

The most used variables in this file are listed bellow. For information on other variables see [TODO].

Variable Description
PACKAGE The package name must be the same as the one in the debian/control file.
VENDOR The vendor of the package.
VARIATION When there can be more then one way to compile the package, use the VARIATION to identify among them. The SystemC compiled library, which is dependent on the GCC version that compiled it, for example, includes this information in the VARIATION. It could also include a -pthreads when building with pthreads support.
VERSIONED Set VERSIONED to 1 to be able to install more then one version of the same package in the same GSPM world. This should be used only for libraries. Use it wisely, because the installation of newer versions will not substitute the old ones and it can easily take too much unecessary disk space. If you want the users to test beta versions in the same world, consider doing a new package called package-beta instead.
BUILD_DEPENDS A list of #pkgdesc package descriptions that are required to be loaded in order to build the package correctly. Normally gcc will be on the list to compile the package.
LOAD_DEPENDS A list of #pkgdesc package descriptions that are required to be loaded together with this package.
EXTRA_CONFIGURE_OPTIONS Everything in this variable is expanded once by make and feed as additional configure options.
MODULEFILE_ENVIRONMENT Everything in this multi-line variable is expanded once by make and copied to the #modulefile modulefile. This are a list of environment variables that will be changed when this package gets loaded.

You can check #rules_variables all the variables that are recognized in the debian/rules file.

Step 4: Packing the new source package

All the work is done for creating the source package. Now the only step left is to pack the directory. Use the following commands:

 # Exit from the package source directory

 cd ..

 # Create the source package. The first argument is the package

 # source directory, the second is the tarball created in step 1.
 gs-dpkg-source -b ''package''-''version'' ''package''_''version''.orig.tar.gz
The three generated files that appear in the command output composes the source package. Contact Marcus (bartho@greensocs.com">bartho@greensocs.com) if you want your package to be included in the main GSPM repository.

Advanced use of GSPM

Topics in this section are for the users that want to go beyond the basics of GSPM.

(unfinished)

Direct manipulation of packages

We can access the installed packages and respective versions by using the Debian sub-system:

  1. gs-apt-cache dumpavail | less

And search by a regular expression:

  1. gs-apt-get search
  2.  
  3. gs-apt-cache show apt
  4.  
  5. gs-dpkg-query -l gscore

Appendix A: Concepts used in this manual

Concept Description
Anchor(modulefile) modulefile TODO
Anchor(pkgdesc) package descriptions TODO

Anchor(rules_variables)

Appendix B: All variables recognized in the debian/rules file

TODO