This chapter documents various tasks for preparing a major, minor or an update GAP release. To clarify the terminology, releasing 4.X.Y+1 after 4.X.Y is called a "minor release" and releasing 4.X+1.Y after 4.X.Z is called a "major release". An "update release" means releasing the latest officially distributed version of the core GAP system with new and updated packages (thus, the version number of the core GAP system stays the same).
The wrapping and publication processes are essentially the same for all three kinds of releases with some minor differences. For example, there is some more work to set up new paths on the ftp server for the major release, and there is no need to update main GAP manuals for the update release.
Changes that should appear in the next release should first appear in the the stable
branch for the next minor release or in the master
branch for the next major release. Normally they may first be developed and documented in feature branches, then tested and only after that merged into the appropriate release branch.
The dev/Updates
directory is used to describe changes intended for the next release. It contains the README
file explaining in more details how to document changes, and two templates: template-long
with explanatory comments and template-short
without them. To document a change, copy one of these templates to an appropriately named new file and fill in the details. This should be done in the same branch where the change itself has been developed in order to keep changes and their documentation together and facilitate tracking those changes which are ready for the release.
During the release preparation, files with these descriptions will be used to compose the overview of changes introduced in the release. When they include test code, it will be used to extend the test file tst/bugfix.tst
.
Documenting changes, do not describe the wrong behaviour in present tense like in:
"Function XYZ
returns the wrong result when computing ABC
."
Rather, write it like in these examples:
"Function XYZ
returned the wrong result when computing ABC
."
"Fixed an error in function XYZ
which returned the wrong result when computing ABC
."
"Function XYZ
now returns the correct result when computing ABC
."
Otherwise, it will sound as if we knowingly introduced a new bug.
Please assume that the text from your description will be pasted verbatim into the overview of changes for the release announcement (only cross-references and other needed GAPDoc markup will be added).
After the release, descriptions of those changes from dev/Updates
which made their way into the release should be moved to a new directory with a name containing the release version (e.g. dev/Updates4.5.7
) using git mv
command to keep them under version control. The dev/Updates
directory in the branch from which the release has been made must be reset to contain only the README
file and templates (and possibly descriptions of changes that has been added there after the release but before such ressetting will take place).
WARNING: The content of the dev/Updates
directory may look different dependently on the branch! For example, in the integration branch it may list changes ready to both next minor and major releases, as well as changes merged into the integration branch for testing. It is important to update your clone of the repository to the proper branch (normally, stable
) before resetting dev/Updates
.
Here is an overview about the hard- and software needed for the wrapping process (at least with the currently available scripts and tools).
a UNIX/Linux machine
a disc partition with at least several GB of space
the standard sh
shell and basic UNIX utilities like cp
, mv
or cmp
git
a TeX installation (which can process big files)
a C-compiler and make
tar
, gzip
, bzip2
, zip
, and python
(for repacking into the various formats)
an installed GAP
persons who can provide Windows binaries of the GAP kernel
The following descriptions will explain what has to be done in the various steps, and will mention some tools which are available in the dev/DistributionUpdate/dist45
directory of the GAP git repository (the name dist45
reflects that these tools were revised for the GAP 4.5 release; we are continuing to use them to produce all subseqent releases).
Here we describe preparational steps for wrapping a minor or major release. Since an update release has no changes in the core GAP system, to wrap an update release you may proceed with steps described in Section 2.2-3.
Before thinking about wrapping apply all the standard tests described in Chapter 6 to a checked out git release branch and make sure there are no problems. (But also do the tests again later, starting with the archives which will be actually delivered to the users!)
setvar*
files.For each of the three release types, some basic variables for the scripts used below are set in the files setvarmajor
, setvarminor
and setvarupdate
in the dev/DistributionUpdate/dist45
directory. Follow instructions from that files to adjust the variables as required. Normally you will need to change only one or two lines with version numbers and do not need to edit anything else, except when setting up release wrapping on a new machine. In that case, choose a directory for the wrapping which can hold at least several GB of data and specify it as DISTROOT
. Find the path to the directory with the merged package archive (we assume that it is available locally) and specify it as MERGEDPKGLOC
.
Update
file.Changes from previous version(s) are accumulated in individual files in the dev/Updates
directory, as explained in 2.1-2.
Test code contained there may be tested regularly in the period between releases using the testupdates
target from the GAP test suite (see Chapter 6) to ensure that it works in various settings.
The test code may be copied into tst/bugfix.tst
at any stage, as soon as it is clear that the relevant change will be a part of the next release. At latest, it should appear in the version of tst/bugfix.tst
which will be actually tested in the release candidate and delivered to the users (exceptions may be made for the test code which may be checked only interactively or requies a very specific testing environment).
Usually, copying of the test code to tst/bugfix.tst
may happen gradually in several iterations if there will be several wrapping attempts; it can be left out while the wrapped archives are not yet expected to become release candidates and were just produced as a part of nightly/weekly tests.
Remark: formerly it was possible to use the file Update.g
from dev/DistributionUpdate/maindist
to extract test code, e.g. by calling
d := UpdateData("../../Update");; l := FixesAndOthersData(d);; PrintTestLines("guck.tst", l);
to extract it to the file guck.tst
. Now Update.g
needs to be adjusted since test codes are now distributed across multiple files. The testupdates
target from the GAP test suite extracts tests from these files, but it does not save them to a file, testing them "on the fly".
CHANGES.md
fileIf regular nightly/weekly tests go well and you hope to wrap a suitable release candidate, you may start to prepare an overview of changes that will appear in the next release in CHANGES.md
, combining it from descriptions provided in files from the dev/Updates
directory.
You may postpone this step until the very release, or perform it in several iterations, keeping track on new entries that will appear in the dev/Updates
directory. You may group various entries together and re-order them so that entries which are likely to be interesting for more GAP users are closer to the top of each category. While converting plain text to markdown, add cross-references and other markup as appropriate. Ideally, you need only to add the markup, and the text of descriptions need not further editing, but please evaluate it critically. At least you may need to change tenses or reorder the text to better fit surrounding items. Also, do not forget to describe new packages added for the redistribution or packages with essential changes in their new versions.
Remark: formerly it was possible to use the file Util.g
to extract descriptions from the dev/Update
file and roughly sort them into several categories, e.g. by calling
d := UpdateData("../../Update");; l := FixesAndOthersData(d);; PrintDescriptionLines("guck.mixer", l);
to extract it to the file guck.mixer
. Now Util.g
needs to be adjusted since test codes are now distributed across multiple files, and we need to produce GAPDoc markup and not Mixer.
Now we explain how to wrap the archives of the GAP source distribution (note that the GAP binaries for Windows are not a part of the GAP source distribution, and they should added to the -win.zip
archive at a later step).
The GAP source distribution will be produced in the form of .tar.gz
, .tar.bz2
and .zip
archives, and also in the -nobin-win.zip
archive whose text files have Windows-stile line breaks.
To wrap the archives, change to the dev/DistributionUpdate/dist45/
directory of the stable
branch of the GAP git repository, check that you have properly set up version numbers in setvar*
file(s) as explained in 2.2-2 and then call one of the following:
make minor make major make update
dependently on the type of the release that you are going to wrap.
We will now explain in more details which steps will be performed. If not specified, all files are assumed to be from the same dev/DistributionUpdate/dist45/
directory.
First, after calling make
with the appropriate target, the corresponding setvar*
file will be copied into the setvar
file (which may be removed with make clean
if needed). Then the script ./doit
will be called. This script contains only calls to other scripts, arranged in three stages with clean interfaces between them (so, theoretically, each step may be performed on a different machine):
checkout and archive the release branch;
make necessary preparations in the GAP core;
merge the GAP core with packages.
We will now describe in some more details what each of the called scripts is doing (you don't normally need to call them directly except while debugging; calling them out of order is not guaranteed to work!).
./setvar
sets various environment variables (paths, version numbers, etc.).
./checkoutgit
creates a new clone of the GAP git repository. Dependently on the release (minor, major or update) it will set the working directory to the stable
branch, master
branch or to the revision taggest as the last minor release respectively. It will also record the revision on which the working directory is based and the time of cloning which will be subsequently used in the timestamp.
./classifyfiles
classifies files into text and binary files to be included in the main archive or in the tools archive, or not shipped at all, using the Python script classifyfiles.py
and files patternscolor.txt
and patternstextbinary.txt
which specify classification rules.
./zipreleasebranch
wraps files from the core GAP system that are marked for the redistribution into the release branch archive and the tools archive
./zipmetainfo
wraps the meta-information archive, containing the .servar
file, details about the checkout time and various lists of files to be included. This script completes stage 1.
./unpackreleasebranch
unpacks all three archives produced at two preceding steps: the release branch archive, the tools archive and the meta-information archive (at this step, all files and subdirectories that are not redistributed are already excluded).
./updateversioninfo
inserts version number, release date and other depending information (so these adjustments will never appear in the repository; instead, after the release we will find the parent revision used to wrap the release and will tag it with an appropriate tag).
Version numbers must be adjusted in the following files:
configure.ac
, look at the top of the file
./fixpermissions
gives directories 755
-permissions and files 644
-permissions, and also gives 755
-permissions to a few explicitly listed there executable files (so it must be adjusted if new executable files are added to the GAP distribution).
./zipgapcore
produces the etc/tools.tar.gz
archive, also adding to it the gapmacrodoc.pdf
document, and then prepares the archive of the GAP core system.
./updatemetainfo
updates the meta-information archive, containing the .servar
file, details about the checkout time and various lists of files to be included. This script completes stage 2.
./unpackgapcore
unpacks the GAP core system archive and the meta-information archive produced by previous two steps (at this step, the core GAP system is exactly in the same state as will be included in the distribution, except hat manuals are yet to be built).
./unpackpackages
finds in the location specified in MERGEDPKGLOC
and unpacks the latest merged packages archive and its corresponding metainformation archive (with, in particular, list of text/binary files).
./checkpermissions
checks for potentially problematic file permissions, i.e.:
directories not having 755 permissions;
files not having 644 or 755 permissions;
files with 755 permissions, not recognized as executables by file
.
These situations may occur in some packages, since we adjust all permissions in the core GAP system, but we do not modify permissions chosen by package authors in case they deliberately use non-standard settings.
./makedoc
builds the main GAP manuals (the packages archives were unpacked on the previous step to ensure that cross-references to package manuals will be resolved).
./addmanualfiles
adds lists of text and binary GAP documentation files to the lists of files to be included in the distribution (thus, despite the previous step builds GAP binaries needed to run GAP to build its manuals, our relying on classifyfiles.py
and explicit control what has to be included ensures that nothing produced by that build will appear in the release, except of the manuals themselves).
./zipgapsourcedistro
produces final GAP source distribution archives in all four formats: .tar.gz
, .tar.bz2
, .zip
archives and -nobin-win.zip
with Windows-style line breaks.
./finalisemetainfo
produces the final version of the meta-information archive.
Now archives with GAP source are ready for testing on as many different systems as possible. Recall the test suites described in Chapter 6.
To produce the -win.zip
archive with binaries for GAP and some packages, you need the -nobin-win.zip
archive with Windows-style line breaks from the GAP source distribution produced as described above, and a Windows computer with Cygwin (https://www.cygwin.com/) installed.
Unpack the archive, start Cygwin shell, change to the GAP root directory created after unpacking and enter
./configure make cygwin
This will build GAP executable for Windows. After that, you may also build packages with kernel modules which seem to work under Windows. As in GAP 4.6, these are the following: Browse, cvec, EDIM, IO and orb. Each of them supports the standard ./configure ; make
building procedure.
Note that there is a nasty problem with respect to dynamically linked libraries used by cygwin under Windows: There seems to be no working version bookkeeping with DLLs. That is, GAP might not be working on Windows systems where another version than the version against which the GAP executable was linked is installed (at least as long as a process is running that uses the other version of cygwin1.dll
). However, we do not see a clean solution to this problem.
After all archives are wrapped and checked and no more errors are found, one has to do the following things to properly publish the release:
Install the brand new version yourself using one of the produced GAP source distribution archives (one needs to have this during the next few steps). The archive will already have documentation created and will contain latest released versions of all currently redistributed packages. DO NOT USE any versions of packages from other sources!
Copy .tar.bz2
.tar.gz
, .zip
archives, -win.zip
and -noreadline-win.zip
archives (with Windows binaries), and also .exe
Windows installer to the ftp server, distributed into directories by the archive type. Also, place the archive of the core GAP system without packages into ftp://ftp.gap-system.org/pub/gap/gap4core/
for developers.
From this point, the release is considered to be published, since the archives are now publicly available.
Prepare individual archives for the same versions of GAP packages that were included in the release and copy them to the ftp server. The procedure should be explained in more details in Chapter 8 (while this chapter is empty, see the DistributionUpdate/PackageUpdate
directory from the https://github.com/gap-system/gap-distribution repository for more details). In brief, you should change to the DistributionUpdate/PackageUpdate
directory of the mentioned repository and do the following:
mark those versions of packages which are included in the release as "stable" using ./markStableRevisions
or ./markAllLatestStable
scripts
mark all stable versions of packages with the bookmark denoting the release, e.g.
./markAllStableWithTimestamp gap4r5p6_2012_11_04-18_46
create individual archives for stable versions of GAP packages with ./mergePackages all stable
update package manuals for the website with ./updatePackageDocs
generate Mixer code for packages section of the website with ./writePackageWebInfos
copy package archives to the ftp server with ./CopyToFtpServer
(this script will perform a dry run first, so you will be able to check if the list is correct before copying)
update the local clone of the GAP website repository to the latest revision, then copy there package manuals and autogenerated Mixer code using ./CopyToWWW2
(this script will also perform a dry run first, so you will be able to check if everything is right before copying)
change to the local clone of the GAP website repository. Copy manuals of updated packages to the public website. Revise, commit and push changes in the files kept under the version control. Some files for packages which are just added for the redistribution may have to be placed under version control.
Since now the autogenerated data are pushed into the central repository, mosts of the website editing steps listed below may be done on another machine different from the one used for checking for package updates and wrapping package archives.
Using some clone of the GAP website repository, do further editing as necessary:
In case of a minor or major release, read the file dev/LinksOfAllHelpSections.g
from the stable
branch of the GAP git repository using your brand new installation of the GAP version to be released (remember to start it with -r
option to avoid loading your private packages). This will create the file AllLinksOfAllHelpSections.data
which is needed to create links from the webpages into the manual. Now update lib/AllLinksOfAllHelpSections.data
with this new version.
Change lib/config
to update version numbers, release date and time, etc.
For a minor or major release of GAP 4.X.Y, add the file Releases/4.X.Y.mixer
under the version control. You may use a similar file for the previous release as a template. Links and names of the archives (version number) and file sizes have to be adjusted manually. For the specification of all packages included in the distribition, see the file Packages/pkgstaticlist
. Update the symlink Releases/index.html
to point to the new Releases/4.X.Y.html
file (you may need to run Mixer to create it first). Add a new entry to the Releases/tree
file.
For an update release, you need only to continue the existing page for the latest release. You may also use previous section on that page as a template, and find detauls about packages in the file Packages/pkgstaticlist
. You do not need to list all changed packages in this case: listing only new and updated packages will suffice.
If there is a new package added for the redistribution, you should edit Packages/packages.mixer
and Packages/tree
to add links to its overview page from the list of packages and from the left navigation panel.
Change Download/index.mixer
and Download/upgrade.mixer
to update version numbers, the GAP banner etc.
For a major release only, change WWW2/Download/Updates/index.mixer
: add new entry for the new release with a link to the Changes manual.
Update package installation scripts Download/InstPackages.sh
and Download/InstPackages32.sh
, if there were any changes.
Build a local copy of the website using Mixer, check that it works, then commit and push all changes to the central repository. If need be (e.g. for major releases with more structural changes), you may use the testing website to check it there first. See 9 for further instructions.
Copy the complete doc
directory (of your brand new installation) to the public website Manuals/doc
so that manual links will work.
In the GAP repository, tag what has been released with v4.X.Y
where X
is the major release number and Y
is the minor release number (e.g. v4.6.3
). The tag
command has the format
git tag TAGNAME REVISION
and the revision that should be tagged should be determined from the release wrapping log. Normally this should be the tip of the stable
branch at the moment of wrapping the release (not at the moment of tagging!).
Update dev/Updates
directory:
Create dev/Update4.X.Y
directory and add it to the version control.
Move all files with documented changes from dev/Updates
directory to dev/Update4.X.Y
directory using git mv
command, so that the only files left in dev/Updates
directory should be README
and two templates (important to do this in the stable
branch, since other branches may have more documented changes).
Commit and push changes.
Edit variables in dev/DistributionUpdate/dist45/setvar*
files, setting the numbers as appropriate for wrapping the next release candidate.
Test download links and proofread web pages.
Announce the new version in the forum (for the major release, it is preferable to wait with the announcement until it will be picked up by all alternative distributions as well).
generated by GAPDoc2HTML