[[!meta title="Baserock release process"]]

Baserock release process
========================

These are the steps to go through to make a Baserock release.  The release
process is a work in progress, so expect it to require a lot of improvement
over time. Also, further tool support is needed.

Releases tend to involve uploading lots of data. As a rule of thumb, you should
aim to be uploading the release images and artifacts the night before the date
that the release is due.

Prerequisites for making a release
----------------------------------

> **Note**: Typically, only Baserock maintainers have enough access to complete
> this process.

You will need:

* Push access to the `git.baserock.org` Trove.
* Write access to the backing store for `download.baserock.org`.
* A fresh `x86_64` (or `x86_32` at a push) VM to coordinate the release.
  This VM should have enough space to coordinate the release artifacts.
  In the past, 30G or more was required.  Please refer to other guides
  for setting up such a VM.

If you are doing a GENIVI release, you will also need:

* An `x86` system capable of emulating ARM's `versatile-pb` platform.

You will also need access to:

* A local Trove synced with `git.baserock.org`.
* An `x86_64` distributed build infrastructure.
* An `x86_32` distributed build infrastructure.
* An `armv7lhf` distributed build infrastructure.
* Configuration for the above suitable for use with your coordination VM.

Configuration
-------------

Ensure that `morph.conf` on your coordination VM is configured to use your
internal Trove by setting `trove-id` and `trove-host` as appropriate.

Deliverables
------------

See `release.morph` in `definitions.git` for the systems we release.
In addition to the system images and other files produced by
`release.morph`, the release requires manually producing or updating
the following:

* Release notes, on `wiki.baserock.org`.
* Updated symlinks to "current" images and tarballs

For a GENIVI release, the following artifacts are produced:

* GENIVI baseline manifest and license manifest
* GENIVI release notes
* An updated version of the `run-built-arm-image.sh` script in
  `baserock:baserock/genivi-initial-setup`

Release steps
-------------

You should do the release in a clean instance of the previous Baserock
devel system release, preferably on x86 (for speed). This makes it
possible for others to reproduce the release images you created.

* Create a new branch (`baserock/release/baserock-X.Y`) in
  `definitions.git`. This is the "release branch", and any changes
  made during the release will happen there, avoiding the need to
  freeze master while the release is happening.
* Get the branch mirrored to your local Trove (push to
  `git.baserock.org`, promote on your local Trove to get it lorried
  quickly).
* Run `scripts/release-build` to produce release artifacts. This
  script builds all the artifacts needed for the release (using
  distbuild, so they end up on your local Trove), and creates the
  release artifacts (all the images and tar archives etc).
    - See the `scripts/release-build.test.conf` sample configuration
      file and use that as a basis for your own.
* Run `scripts/release-upload` to publish release artifacts.
  This copies build artifacts from your local Trove to
  `git.baserock.org`, and other files to `download.baserock.org`.
  After this script has finished successfully, Baserock users can
  start using the new release.
    - See the `scripts/release-upload.test.conf` sample configuration
      file and use that as a basis for your own.
* Manually create the "current" symlinks pointing at the new release.
  (FIXME: The upload script should do this automatically, but that
  code hasn't been written yet.)
* Tag the HEAD of the release branch with the new release
  (`baserock-X.Y`). Push the release branch, including tags (`git push
  --tags`).
* Prepare or update release notes, wikis, and websites.
    - Drafting these can start early on, while building and uploading
      are running.

The upload script assumes that you have appropriate credentials to upload files to the
servers using rsync. If a password is required, rsync will prompt for your
password at that stage of the script. If you rely on SSH agent forwarding to
provide access, and are leaving the release script running overnight, remember
that the machine running your SSH agent must **also** remain running overnight!

GENIVI Baseline release steps
-----------------------------

For the GENIVI Baseline systems there are several extra steps. Beware: these
scripts are slow and can take several hours!

1. Generate a manifest for each baseline system, using the `morph generate-manifest` command:

        morph generate-manifest /src/cache/artifacts/*.genivi-baseline-system-armv7lhf-versatile-rootfs > \
            /src/release/genivi-baseline-system-armv7lhf-versatile-manifest.txt
        morph generate-manifest /src/cache/artifacts/*.genivi-baseline-system-x86_64-generic-rootfs > \
            /src/release/genivi-baseline-system-x86_64-generic-manifest.txt

   > **NOTE**: You will need the rootfs artifacts because while
   > generate-manifest claims to operate from disk images, the code is not
   > in-sync with our current raw disk generator

2. Generate a license manifest for each baseline system, using the
   `license-check` script in definitions.git:

        scripts/licensecheck.sh genivi-baseline-system-armv7lhf-versatile > \
            /src/release/genivi-baseline-system-armv7lhf-versatile-license-manifest.txt
        scripts/licensecheck.sh genivi-baseline-system-x86_64-generic > \
            /src/release/genivi-baseline-system-x86_64-generic-license-manifest.txt

   > **NOTE**: You will need the release workspace to run the licensecheck
       script

3. Extract the kernel zImage file from the ARMv7lhf system, by loopback
   mounting it and copying the file out. (FIXME: there is an unmerged patch in
   morph.git branch baserock/richardipsum/write_kernel to make `morph deploy`
   handle this automatically).

4. Write GENIVI release notes. These should summarise the changes relevant to
   developers of systems based on the Baserock GENIVI Baseline system, and link
   to the Baserock release notes. See
   <http://projects.genivi.org/GENIVI_Baselines/genivi-baserock/blog> for
   examples.

You'll need to upload these extra files to the download.baserock.org server
manually; the script doesn't take care of this yet (see 'future plans', below).


Announcing the release
----------------------

Once the `release-upload` script completes successfully:

1. Update all pages on `wiki.baserock.org` that refer to the previous release
   number to the new release number.
2. Send release notes to baserock-announce@baserock.org and
   baserock-dev@baserock.org.

Announcing the GENIVI release
-----------------------------

1. Send announcement email to genivi-dev@mail.genivi.org and
   genivi-baserock@lists.genivi.org
2. Post release notes into blog at
   <http://projects.genivi.org/GENIVI_Baselines/genivi-baserock>.
   Currently Jonathan Maw (jonathanmaw), Pedro Alvarez (pedroalvaraz), Sam
   Thursfield (ssam) and Rob Taylor (robtaylor) have admin access to do this.
   WARNING: the 'preview' renderer will show you something quite different
   to the actual rendering after you've pressed 'post'.
3. Review wiki.baserock.org's GENIVI related how-tos.

Future goals
------------

Checksum everything we release, to guard against corruption at all stages of
the release process.

Consider the bit-for-bit reproducibility of the release images, and how we can
make it easy for those who want to reproduce the Baserock release images that
are produced as part of this process.

Allow greater flexibility in what we release. Some of us think that
Baserock's release process should, in future, be as follows: the build-system
image is released every week (this image is capable of building any other
Baserock image), other systems can optionally be released at the same time
according either to schedule or need. For example, we can continue to release
the GENIVI Baseline demo system every 6 weeks, following the GENIVI Baseline
release process. We can release other example systems when they have received
significant changes to merit a new release.

Automate GENIVI releases. Ideally, this should be done in a generic way,
treating the GENIVI Baseline system as if it was one of many example Baserock
systems which each had their own release requirements.

Ensure that this script is useful for projects derived from Baserock, as well
as for the Baserock project itself.