Goto Chapter: Top 1 2 3 4 5 6 7 8 9 10 Ind
 [Top of Book]  [Contents]   [Previous Chapter]   [Next Chapter] 

2 Preparing GAP Releases
 2.1 Committing changes for the next minor or major release
 2.2 Wrapping Archives for a Release or Update
 2.3 Releasing a new version

2 Preparing GAP Releases

This chapter documents various tasks for preparing a major, minor or an update GAP release. To clarify the terminology, releasing 4.X.Y+1 after 4.X.Y is called a "minor release" and releasing 4.X+1.Y after 4.X.Z is called a "major release". An "update release" means releasing the latest officially distributed version of the core GAP system with new and updated packages (thus, the version number of the core GAP system stays the same).

The wrapping and publication processes are essentially the same for all three kinds of releases with some minor differences. For example, there is some more work to set up new paths on the ftp server for the major release, and there is no need to update main GAP manuals for the update release.

2.1 Committing changes for the next minor or major release

2.1-1 Adding changes to the release branch

Changes that should appear in the next release should first appear in the the stable branch for the next minor release or in the master branch for the next major release. Normally they may first be developed and documented in feature branches, then tested and only after that merged into the appropriate release branch.

2.1-2 Document your changes

The dev/Updates directory is used to describe changes intended for the next release. It contains the README file explaining in more details how to document changes, and two templates: template-long with explanatory comments and template-short without them. To document a change, copy one of these templates to an appropriately named new file and fill in the details. This should be done in the same branch where the change itself has been developed in order to keep changes and their documentation together and facilitate tracking those changes which are ready for the release.

During the release preparation, files with these descriptions will be used to compose the overview of changes introduced in the release. When they include test code, it will be used to extend the test file tst/bugfix.tst.

Documenting changes, do not describe the wrong behaviour in present tense like in:

Rather, write it like in these examples:

Otherwise, it will sound as if we knowingly introduced a new bug.

Please assume that the text from your description will be pasted verbatim into the overview of changes for the release announcement (only cross-references and other needed GAPDoc markup will be added).

After the release, descriptions of those changes from dev/Updates which made their way into the release should be moved to a new directory with a name containing the release version (e.g. dev/Updates4.5.7) using git mv command to keep them under version control. The dev/Updates directory in the branch from which the release has been made must be reset to contain only the README file and templates (and possibly descriptions of changes that has been added there after the release but before such ressetting will take place).

WARNING: The content of the dev/Updates directory may look different dependently on the branch! For example, in the integration branch it may list changes ready to both next minor and major releases, as well as changes merged into the integration branch for testing. It is important to update your clone of the repository to the proper branch (normally, stable) before resetting dev/Updates.

2.2 Wrapping Archives for a Release or Update

2.2-1 What you need to start

Here is an overview about the hard- and software needed for the wrapping process (at least with the currently available scripts and tools).

The following descriptions will explain what has to be done in the various steps, and will mention some tools which are available in the dev/DistributionUpdate/dist45 directory of the GAP git repository (the name dist45 reflects that these tools were revised for the GAP 4.5 release; we are continuing to use them to produce all subseqent releases).

2.2-2 Before the actual wrapping

Here we describe preparational steps for wrapping a minor or major release. Since an update release has no changes in the core GAP system, to wrap an update release you may proceed with steps described in Section 2.2-3.

Run tests on release branch.

Before thinking about wrapping apply all the standard tests described in Chapter 6 to a checked out git release branch and make sure there are no problems. (But also do the tests again later, starting with the archives which will be actually delivered to the users!)

Edit variables in setvar* files.

For each of the three release types, some basic variables for the scripts used below are set in the files setvarmajor, setvarminor and setvarupdate in the dev/DistributionUpdate/dist45 directory. Follow instructions from that files to adjust the variables as required. Normally you will need to change only one or two lines with version numbers and do not need to edit anything else, except when setting up release wrapping on a new machine. In that case, choose a directory for the wrapping which can hold at least several GB of data and specify it as DISTROOT. Find the path to the directory with the merged package archive (we assume that it is available locally) and specify it as MERGEDPKGLOC.

Extract test code from Update file.

Changes from previous version(s) are accumulated in individual files in the dev/Updates directory, as explained in 2.1-2.

Test code contained there may be tested regularly in the period between releases using the testupdates target from the GAP test suite (see Chapter  6) to ensure that it works in various settings.

The test code may be copied into tst/bugfix.tst at any stage, as soon as it is clear that the relevant change will be a part of the next release. At latest, it should appear in the version of tst/bugfix.tst which will be actually tested in the release candidate and delivered to the users (exceptions may be made for the test code which may be checked only interactively or requies a very specific testing environment).

Usually, copying of the test code to tst/bugfix.tst may happen gradually in several iterations if there will be several wrapping attempts; it can be left out while the wrapped archives are not yet expected to become release candidates and were just produced as a part of nightly/weekly tests.

Remark: formerly it was possible to use the file Update.g from dev/DistributionUpdate/maindist to extract test code, e.g. by calling

d := UpdateData("../../Update");;
l := FixesAndOthersData(d);;
PrintTestLines("guck.tst", l);

to extract it to the file guck.tst. Now Update.g needs to be adjusted since test codes are now distributed across multiple files. The testupdates target from the GAP test suite extracts tests from these files, but it does not save them to a file, testing them "on the fly".

Prepare an overview of changes for the CHANGES.md file

If regular nightly/weekly tests go well and you hope to wrap a suitable release candidate, you may start to prepare an overview of changes that will appear in the next release in CHANGES.md, combining it from descriptions provided in files from the dev/Updates directory.

You may postpone this step until the very release, or perform it in several iterations, keeping track on new entries that will appear in the dev/Updates directory. You may group various entries together and re-order them so that entries which are likely to be interesting for more GAP users are closer to the top of each category. While converting plain text to markdown, add cross-references and other markup as appropriate. Ideally, you need only to add the markup, and the text of descriptions need not further editing, but please evaluate it critically. At least you may need to change tenses or reorder the text to better fit surrounding items. Also, do not forget to describe new packages added for the redistribution or packages with essential changes in their new versions.

Remark: formerly it was possible to use the file Util.g to extract descriptions from the dev/Update file and roughly sort them into several categories, e.g. by calling

d := UpdateData("../../Update");;
l := FixesAndOthersData(d);;
PrintDescriptionLines("guck.mixer", l);

to extract it to the file guck.mixer. Now Util.g needs to be adjusted since test codes are now distributed across multiple files, and we need to produce GAPDoc markup and not Mixer.

2.2-3 Wrapping the archives for the GAP source distribution

Now we explain how to wrap the archives of the GAP source distribution (note that the GAP binaries for Windows are not a part of the GAP source distribution, and they should added to the -win.zip archive at a later step).

The GAP source distribution will be produced in the form of .tar.gz, .tar.bz2 and .zip archives, and also in the -nobin-win.zip archive whose text files have Windows-stile line breaks.

To wrap the archives, change to the dev/DistributionUpdate/dist45/ directory of the stable branch of the GAP git repository, check that you have properly set up version numbers in setvar* file(s) as explained in 2.2-2 and then call one of the following:

make minor
make major
make update

dependently on the type of the release that you are going to wrap.

We will now explain in more details which steps will be performed. If not specified, all files are assumed to be from the same dev/DistributionUpdate/dist45/ directory.

First, after calling make with the appropriate target, the corresponding setvar* file will be copied into the setvar file (which may be removed with make clean if needed). Then the script ./doit will be called. This script contains only calls to other scripts, arranged in three stages with clean interfaces between them (so, theoretically, each step may be performed on a different machine):

We will now describe in some more details what each of the called scripts is doing (you don't normally need to call them directly except while debugging; calling them out of order is not guaranteed to work!).

./setvar

sets various environment variables (paths, version numbers, etc.).

./checkoutgit

creates a new clone of the GAP git repository. Dependently on the release (minor, major or update) it will set the working directory to the stable branch, master branch or to the revision taggest as the last minor release respectively. It will also record the revision on which the working directory is based and the time of cloning which will be subsequently used in the timestamp.

./classifyfiles

classifies files into text and binary files to be included in the main archive or in the tools archive, or not shipped at all, using the Python script classifyfiles.py and files patternscolor.txt and patternstextbinary.txt which specify classification rules.

./zipreleasebranch

wraps files from the core GAP system that are marked for the redistribution into the release branch archive and the tools archive

./zipmetainfo

wraps the meta-information archive, containing the .servar file, details about the checkout time and various lists of files to be included. This script completes stage 1.

./unpackreleasebranch

unpacks all three archives produced at two preceding steps: the release branch archive, the tools archive and the meta-information archive (at this step, all files and subdirectories that are not redistributed are already excluded).

./updateversioninfo

inserts version number, release date and other depending information (so these adjustments will never appear in the repository; instead, after the release we will find the parent revision used to wrap the release and will tag it with an appropriate tag).

Version numbers must be adjusted in the following files:

./fixpermissions

gives directories 755-permissions and files 644-permissions, and also gives 755-permissions to a few explicitly listed there executable files (so it must be adjusted if new executable files are added to the GAP distribution).

./zipgapcore

produces the etc/tools.tar.gz archive, also adding to it the gapmacrodoc.pdf document, and then prepares the archive of the GAP core system.

./updatemetainfo

updates the meta-information archive, containing the .servar file, details about the checkout time and various lists of files to be included. This script completes stage 2.

./unpackgapcore

unpacks the GAP core system archive and the meta-information archive produced by previous two steps (at this step, the core GAP system is exactly in the same state as will be included in the distribution, except hat manuals are yet to be built).

./unpackpackages

finds in the location specified in MERGEDPKGLOC and unpacks the latest merged packages archive and its corresponding metainformation archive (with, in particular, list of text/binary files).

./checkpermissions

checks for potentially problematic file permissions, i.e.:

These situations may occur in some packages, since we adjust all permissions in the core GAP system, but we do not modify permissions chosen by package authors in case they deliberately use non-standard settings.

./makedoc

builds the main GAP manuals (the packages archives were unpacked on the previous step to ensure that cross-references to package manuals will be resolved).

./addmanualfiles

adds lists of text and binary GAP documentation files to the lists of files to be included in the distribution (thus, despite the previous step builds GAP binaries needed to run GAP to build its manuals, our relying on classifyfiles.py and explicit control what has to be included ensures that nothing produced by that build will appear in the release, except of the manuals themselves).

./zipgapsourcedistro

produces final GAP source distribution archives in all four formats: .tar.gz, .tar.bz2, .zip archives and -nobin-win.zip with Windows-style line breaks.

./finalisemetainfo

produces the final version of the meta-information archive.

Now archives with GAP source are ready for testing on as many different systems as possible. Recall the test suites described in Chapter 6.

2.2-4 Compiling on Windows with Cygwin

To produce the -win.zip archive with binaries for GAP and some packages, you need the -nobin-win.zip archive with Windows-style line breaks from the GAP source distribution produced as described above, and a Windows computer with Cygwin (https://www.cygwin.com/) installed.

Unpack the archive, start Cygwin shell, change to the GAP root directory created after unpacking and enter

./configure
make cygwin

This will build GAP executable for Windows. After that, you may also build packages with kernel modules which seem to work under Windows. As in GAP 4.6, these are the following: Browse, cvec, EDIM, IO and orb. Each of them supports the standard ./configure ; make building procedure.

Note that there is a nasty problem with respect to dynamically linked libraries used by cygwin under Windows: There seems to be no working version bookkeeping with DLLs. That is, GAP might not be working on Windows systems where another version than the version against which the GAP executable was linked is installed (at least as long as a process is running that uses the other version of cygwin1.dll). However, we do not see a clean solution to this problem.

2.3 Releasing a new version

After all archives are wrapped and checked and no more errors are found, one has to do the following things to properly publish the release:

  1. Install the brand new version yourself using one of the produced GAP source distribution archives (one needs to have this during the next few steps). The archive will already have documentation created and will contain latest released versions of all currently redistributed packages. DO NOT USE any versions of packages from other sources!

  2. Copy .tar.bz2 .tar.gz, .zip archives, -win.zip and -noreadline-win.zip archives (with Windows binaries), and also .exe Windows installer to the ftp server, distributed into directories by the archive type. Also, place the archive of the core GAP system without packages into ftp://ftp.gap-system.org/pub/gap/gap4core/ for developers.

    From this point, the release is considered to be published, since the archives are now publicly available.

  3. Prepare individual archives for the same versions of GAP packages that were included in the release and copy them to the ftp server. The procedure should be explained in more details in Chapter 8 (while this chapter is empty, see the DistributionUpdate/PackageUpdate directory from the https://github.com/gap-system/gap-distribution repository for more details). In brief, you should change to the DistributionUpdate/PackageUpdate directory of the mentioned repository and do the following:

  4. Using some clone of the GAP website repository, do further editing as necessary:

  5. Copy the complete doc directory (of your brand new installation) to the public website Manuals/doc so that manual links will work.

  6. In the GAP repository, tag what has been released with v4.X.Y where X is the major release number and Y is the minor release number (e.g. v4.6.3). The tag command has the format

         git tag TAGNAME REVISION
         

    and the revision that should be tagged should be determined from the release wrapping log. Normally this should be the tip of the stable branch at the moment of wrapping the release (not at the moment of tagging!).

  7. Update dev/Updates directory:

  8. Edit variables in dev/DistributionUpdate/dist45/setvar* files, setting the numbers as appropriate for wrapping the next release candidate.

  9. Test download links and proofread web pages.

  10. Announce the new version in the forum (for the major release, it is preferable to wait with the announcement until it will be picked up by all alternative distributions as well).

 [Top of Book]  [Contents]   [Previous Chapter]   [Next Chapter] 
Goto Chapter: Top 1 2 3 4 5 6 7 8 9 10 Ind

generated by GAPDoc2HTML