dh-kpatches implementation

- The two Parts
  - Packaging-time work
    - The `dh_installkpatches` helper
    - Patch processing
    - `kpatch:depends` substvar
  - Kernel-build-time work
    - Apply script
    - Unpatch script
    - Common stuff
- Reading a kpatches control file
  - Generic (v1) data structures
  - Handling v0 definitions
  - Fields of kpatch files
    - From text fields to data structures
      - Generic alternative fields
      - Per-operation fields
    - Field-handling library
      - Field inheritance
      - Recording a patchfile field for later use
- Patch processing
- Index of chunks

# The two Parts

The dh-kpatches system is made of two parts. The first one takes patch descriptions, and produces kernel-patch packages; the second one is (currently) contained in those patch packages, and handles application of the patches to a kernel source tree, as well as their removal.

The application/removal scripts are (currently) generated from templates by the `dh_installkpatches` script for each kernel-patch package.

Currently, the patch descriptions are completely parsed by `dh_installkpatches`, and the apply/unpatch scripts are specific to each package, and can only act on the patches `dh_installkpatches` taught them about.

## Packaging-time work

### The `dh_installkpatches` helper

This is the debhelper-based script that will cause our packages to install patches in the One True Way. Its base structure is that of a standard debhelper script.

    <dh_installkpatches.in>=
    #!/usr/bin/perl -w
    #
    # dh_installkpatches $Revision: 1.17.1.4.1.24.1.5 $
    #
    # Reads debian/$package.kpatches[.foo], installs all files necessary
    # to have make-kpkg use the kernel patches described in there.
    #
    # (c) 2000-2003 Yann Dirson <dirson@debian.org>
    # Some parts based on code from Adam Di Carlo and Joey Hess

    use strict;
    use Debian::Debhelper::Dh_Lib;
    init();

    <definitions for the core kpatch system>

    PACKAGE: foreach my $package (@{$dh{DOPACKAGES}}) {
      my $tmp = tmpdir($package);
      my $ext = pkgext($package);

      # There are two filename formats, the usual
      # plus an extended format (debian/package.*).

      opendir(DEB,"debian/") || error("can't read debian directory: $!");
      # If this is the main package, we need to handle unprefixed filenames.
      # For all packages, we must support both the usual filename format plus
      # that format with a period an something appended.
      my $regexp="\Q$package\E\.";
      if ($package eq $dh{MAINPACKAGE}) {
        $regexp="(|$regexp)";
      }
      my @files = grep { /^${regexp}kpatches(\..*)?$/ } readdir(DEB);
      closedir(DEB);

      # next package if there are no patches in there
      next PACKAGE if $#files < 0;

      <process binary package $package>
    }

### Patch processing

For each binary package, all `kpatches` files are processed to generate one `apply` script with a single `diff` operation (and its corresponding `unpatch` script).

    <process binary package $package>= (<-U)
    <assert kpatch:depends is referenced in control file>

    foreach my $file (@files) {
      my %patchinfo = read_control_file ("debian/$file");

      #   use Data::Dumper;
      #   print Dumper (%patchinfo);

      my $patchid = $patchinfo{general}->{'patch-id'};

      # transformation of the ID to be acceptable as part of an envvar's name
      $patchinfo{general}->{'clean-patch-id'} = $patchinfo{general}->{'patch-id'};
      $patchinfo{general}->{'clean-patch-id'} =~ s/-/_/g;

      # protect pipes and dquotes for sed command-line
      $patchinfo{general}->{'patch-name'} =~ s,([|\"]),\\$1,g;

      <generate apply/unpatch scripts for given $package and kpatches $file>
    }

    <set kpatch:Depends substvar>

### `kpatch:depends` substvar

A number of packages are needed by kernel-patch packages:

bash 2.x or better
  because apply scripts use arrays
patch
  because we (currently) only apply patches made of diff files
grep-dctrl
  is used to register the version of the kernel-patch package into the kernel-image package

    <definitions for the core kpatch system>= (<-U) [D->]
    my $pkgdeps = "bash (>= 2.0), patch, grep-dctrl";

The substvar code was derived from similar functionnality in dh_perl v3.4.1. For idempotency, we first remove anything this program might have previously added to the substvars file.

    <set kpatch:Depends substvar>= (<-U)
    if (-e "debian/${ext}substvars") {
      complex_doit("grep -v ^kpatch:Depends= debian/${ext}substvars > debian/${ext}substvars.new || true");
      doit("mv", "debian/${ext}substvars.new","debian/${ext}substvars");
    }

    complex_doit("echo 'kpatch:Depends=$pkgdeps' >> debian/${ext}substvars");

We also make sure the package uses our substvar, and abort if not.

    <assert kpatch:depends is referenced in control file>= (<-U)
    die 'debian/control must make package ' . $package . ' depend on ${kpatch:Depends}'
      if system ("dpkg-gencontrol -p$package -Pdebian -O -T/dev/null -Vkpatch:Depends=KPATCHISUSED |"
               . "grep -q '^Depends: .*KPATCHISUSED'") != 0;

## Kernel-build-time work

For each v0 kpatches file, we produce an apply and an unpatch script, necessary to work properly with make-kpkg. Those are currently produced from templates.

### Apply script

    <apply.tmpl>=
    #! /bin/bash
    set -e

    <Assert this is a kernel tree>
    <Apply.tmpl constants>

    DEPENDS=(#DEPENDS#)

    KVERSIONS=(#KVERSIONS#)
    PATCHFILES=(#PATCHFILES#)
    DEBPATCHFILES=(#DEBPATCHFILES#)
    STRIPLEVELS=(#STRIPLEVELS#)

    <Make apply idempotent>
    <Get current kernel version>
    <Select alternative to apply, as $IDX>

    echo >&2 "START applying #PATCHID# patch (#PATCHNAME#)"

    <Handle dependencies>

    <Assert patch applies, using dry-run>
    <Apply the patch>

    <Store necessary info for unpatching>
    <Create script to have this patch registered in doc-dir>

    echo >&2 "END applying #PATCHID# patch"

    <Get current kernel version>= (<-U)
    VERSION=$(grep ^VERSION Makefile 2>/dev/null | \
            sed -e 's/[^0-9]*\([0-9]*\)/\1/')
    PATCHLEVEL=$( grep ^PATCHLEVEL Makefile 2>/dev/null | \
            sed -e 's/[^0-9]*\([0-9]*\)/\1/')
    SUBLEVEL=$(grep ^SUBLEVEL Makefile 2>/dev/null | \
            sed -e 's/[^0-9]*\([0-9]*\)/\1/')
    EXTRAVERSION=$(grep ^EXTRAVERSION Makefile | head -1 2>/dev/null | \
            sed -e 's/EXTRAVERSION =[       ]*\([^  ]*\)$/\1/')
    KERNELBRANCHLET=${VERSION}.${PATCHLEVEL}.${SUBLEVEL}
    KERNELRELEASE=${KERNELBRANCHLET}${EXTRAVERSION}

The candidates patch version for application to our kernel (eg. 2.6.11.11) is the first of:

1. the version passed through the KPATCH_* mechanism
2. the version exactly matching the kernel (eg. 2.6.11.11)
3. the version of the "branchlet" used for short stable branches in the 2.6 branch (eg. 2.6.11)
4. the `all` version

    <Select alternative to apply, as $IDX>= (<-U)
    IDX=

    declare -i i=${#PATCHFILES[*]}-1
    while [ $i -ge 0 ]
    do
        v=${KVERSIONS[$i]}
        if [ -n "$KPATCH_#CLEANPATCHID#" -a "$v" = "$KPATCH_#CLEANPATCHID#" \
             -o "$v" = "$KERNELRELEASE"
             -o "$v" = "$KERNELBRANCHLET"
             -o "$v" = all ]
        then
            IDX=$i
        fi
        i=i-1
    done

    <Assert we have an alternative to apply>
    <Record parameters for applying alternative $IDX>

If the KPATCH_* mechanism was used, we have a special error message listing the requested version.

    <Assert we have an alternative to apply>= (<-U)
    if [ -n "$KPATCH_#CLEANPATCHID#" -a ${KVERSIONS[$IDX]} != "$KPATCH_#CLEANPATCHID#" ]
    then
        echo >&2 "Requested kernel version \`$KPATCH_#CLEANPATCHID#' not found for patch #PATCHID#"
        exit 1
    elif [ -z "$IDX" ]
    then
        echo >&2 "No \"#PATCHNAME#\" patch found for kernel version $KERNELRELEASE"
        exit 1
    fi

Of special notice is the "debian" pseudo-flavour. Before the kernel-patch-debian era, we had to check for README.Debian. Now we must check for version.Debian.

    <Record parameters for applying alternative $IDX>= (<-U)
    KVERSION=${KVERSIONS[$IDX]}
    STRIPLEVEL=${STRIPLEVELS[$IDX]}

    if [ "${DEBPATCHFILES[$IDX]}" != '' -a \
        \( -r version.Debian -o -r README.Debian \) ]
    then
        PATCHFILE=${DEBPATCHFILES[$IDX]}
    else
        PATCHFILE=${PATCHFILES[$IDX]}
    fi

    <Handle dependencies>= (<-U)
    <Check for dependencies>
    <Apply dependencies>

    <Check for dependencies>= (<-U)
    NEEDED_DEPS=
    for dep in ${DEPENDS[$IDX]}
    do
        if [ -x ${TOPPATCHDIR}/${ARCHITECTURE}/${KERNELBRANCHLET}/apply/$dep ]
        then
            NEEDED_DEPS="${ARCHITECTURE}/${KERNELBRANCHLET}/apply/$dep $NEEDED_DEPS"
        elif [ -x ${TOPPATCHDIR}/all/${KERNELBRANCHLET}/apply/$dep ]
        then
            NEEDED_DEPS="all/${KERNELBRANCHLET}/apply/$dep $NEEDED_DEPS"
        elif [ -x ${TOPPATCHDIR}/${ARCHITECTURE}/apply/$dep ]
        then
            NEEDED_DEPS="${ARCHITECTURE}/apply/$dep $NEEDED_DEPS"
        elif [ -x ${TOPPATCHDIR}/all/apply/$dep ]
        then
            NEEDED_DEPS="all/apply/$dep $NEEDED_DEPS"
        else
            echo >&2 "ERROR: Patch dependency \`$dep' not found - aborting"
            echo >&2 "END applying #PATCHID# patch"
            exit 1
        fi
    done

    <Apply dependencies>= (<-U)
    if [ "$NEEDED_DEPS" ]
    then
        echo >&2 "Ensuring the following patches are applied first: $NEEDED_DEPS"
        for apply in ${NEEDED_DEPS}
        do
            dep=$(basename $apply)
            ${TOPPATCHDIR}/$apply

            # check something was applied
            if [ ! -f debian/APPLIED_${ARCHITECTURE}_$dep -a \
                 ! -f debian/APPLIED_all_$dep ]
            then
                echo >&2 "ERROR: patch dependency did not left a patch stamp (version mismatch ?) - aborting"
                echo >&2 "END applying #PATCHID# patch"
                exit 1
            fi
        done
        UNPATCHDEPS=$(echo ${NEEDED_DEPS} | sed s,/apply/,/unpatch/,g)
    fi

    <Assert patch applies, using dry-run>= (<-U)
    echo >&2 "Testing whether \"#PATCHNAME#\" patch for $KVERSION applies (dry run):"
    if ! [ -r $PATCHFILE ]
    then
        echo >&2 "\"#PATCHNAME#\" patch for $KVERSION not found"
        exit 1
    elif ! $DECOMPRESSOR $PATCHFILE |
            patch --force --dry-run $PATCH_OPTIONS -p$STRIPLEVEL
    then
        echo >&2 "\"#PATCHNAME#\" patch for $KVERSION does not apply cleanly"
        exit 1
    fi

We do not use --force on second run, there should be no need for it. If something requires interaction, it is likely there is a bug somewhere, better catch it.

After applying the diff, we remove any empty files, so that files "removed" by the diff are really removed. We make an exception for ./debian/ contents, and for files named APPLIED*, or else some (buggy) stamp files may be caught too.

    <Apply the patch>= (<-U)
    if ! $DECOMPRESSOR $PATCHFILE |
            patch $PATCH_OPTIONS -p$STRIPLEVEL
    then
        # This should never happen, thanks to the dry-run
        echo >&2 "ASSERTION FAILED - \"#PATCHNAME#\" patch for $KVERSION failed"
        echo >&2 "END applying #PATCHID# patch"
        exit 1
    fi
    echo >&2 "\"#PATCHNAME#\" patch for $KVERSION succeeded"

    <Remove empty files>

All information necessary for unpatching is stored in a `debian/APPLIED_*` file, for use by the `unpatch` script.

    <Store necessary info for unpatching>= (<-U)
    mkdir -p debian
    cat > 'debian/APPLIED_#PATCHARCH#_#PATCHID#' <<EOF
    PATCHFILE='$PATCHFILE'
    STRIPLEVEL='$STRIPLEVEL'
    DEPENDS='$UNPATCHDEPS'
    EOF

    <Create script to have this patch registered in doc-dir>= (<-U)
    mkdir -p debian/image.d
    PKGNAME=`dpkg -S $PATCHFILE | cut -d: -f1`
    PKGVERS=`grep-dctrl -n -P $PKGNAME -s Version -X /var/lib/dpkg/status`
    cat > 'debian/image.d/register-#PATCHID#' <<EOF
    #!/bin/sh

    # This scripts documents the "#PATCHNAME#" kernel patch into the
    # kernel-image package, as being applied to the kernel.

    docdir=\${IMAGE_TOP}/usr/share/doc/kernel-image-\${version}

    mkdir -p \${docdir}

    (
        printf '#PATCHNAME# (#PATCHID#)${KPATCH_#CLEANPATCHID#:+ for kernel ${KPATCH_#CLEANPATCHID#}},'
        echo ' from package $PKGNAME, version $PKGVERS'
    ) >> \${docdir}/applied-patches
    EOF
    chmod +x 'debian/image.d/register-#PATCHID#'

    <Apply.tmpl constants>= (<-U)
    TOPPATCHDIR=/usr/src/kernel-patches
    ARCHITECTURE=`dpkg --print-installation-architecture`
    DECOMPRESSOR="zcat -f"
    PATCH_OPTIONS="--ignore-whitespace --silent"
    # This is informational only, used by lskpatches
    DHPKPATCHES_VERSION=#DHKPVERS#

We consider it a success if the patch is already applied.

FIXME: we should make sure that the currently-applied patch is the one that would have been applied.

    <Make apply idempotent>= (<-U)
    [ -f debian/APPLIED_${ARCHITECTURE}_#PATCHID# -o \
      -f debian/APPLIED_all_#PATCHID# ] && exit 0

### Unpatch script

    <unpatch.tmpl>=
    #! /bin/bash
    set -e

    # This "unpatch" script was automatically generated by
    # dh_installkpatches from unpatch.tmpl $Revision: 1.17.1.4.1.24.1.5 $

    <Assert this is a kernel tree>

    ARCHITECTURE=`dpkg --print-installation-architecture`
    PATCHID=#PATCHID#
    PATCHARCH=#PATCHARCH#

    TOPPATCHDIR=/usr/src/kernel-patches
    DECOMPRESSOR="zcat -f"

    STAMP=debian/APPLIED_${PATCHARCH}_$PATCHID

    PATCH_OPTIONS="--ignore-whitespace --silent"

    [ -f $STAMP ] || exit 0

    <Assert stamp file is not empty>
    <Read contents from stamp file>
    <Assert diff file is still there>
    <Do not unpatch if there are still dependants>

    <Remove the patch>

    <Assert stamp file is not empty>= (<-U)
    if [ ! -s $STAMP ]
    then
        cat >&2 <<EOF
    ERROR: "#PATCHNAME#" patch was applied using old mechanism.
    The relevant patch file may even not be on your system any more.
    I strongly suggest you remove this kernel tree and unpack a clean one.
    EOF
        exit 1
    fi

    <Read contents from stamp file>= (<-U)
    if [ $(echo $(wc -l < $STAMP) ) != 1 ]
    then
        . $STAMP
    else
        # old format
        APPLY_INFO=$(cat $STAMP)
        PATCHFILE=$(echo ${APPLY_INFO} | cut -d# -f1)
        STRIPLEVEL=$(echo ${APPLY_INFO} | cut -d# -f2)
    fi

    <Assert diff file is still there>= (<-U)
    if [ ! -r $PATCHFILE ]
    then
        cat >&2 <<EOF
    ERROR: applied "#PATCHNAME#" patch file could not be found.
    Presumably the package containing it was removed or upgraded.
    I strongly suggest you remove this kernel tree and unpack a clean one.
    EOF
        exit 1
    fi

    <Remove those patches we depended on>= (U->)
    if [ -n "$DEPENDS" ]
    then
        echo >&2 "Also un-applying the following patches: $DEPENDS"
        for dep in $DEPENDS
        do
            if [ -x ${TOPPATCHDIR}/$dep ]
            then
                ${TOPPATCHDIR}/$dep
            else
                echo >&2 "Could not unpatch dependency: $dep - stopping here."
                echo >&2 "END unpatching #PATCHID# patch"
                exit 1
            fi
        done
    fi

    <Do not unpatch if there are still dependants>= (<-U)
    if grep -l "^DEPENDS=.*[' ]#PATCHARCH#/unpatch/#PATCHID#[' ]" debian/APPLIED_* >/dev/null 2>/dev/null
    then
        # There are patches depending on this one, to be removed before.
        # the dep will be removed by the last of those other patches.
        echo >&2 "NOT unpatching \"#PATCHNAME#\" since other patches still rely on it"
        exit 0
    fi

    <Remove the patch>= (<-U)
    echo >&2 "START unpatching #PATCHID# patch (#PATCHNAME#)"

    $DECOMPRESSOR $PATCHFILE | patch -R -p$STRIPLEVEL $PATCH_OPTIONS
    rm -f debian/APPLIED_${PATCHARCH}_$PATCHID debian/image.d/register-#PATCHID#

    <Remove those patches we depended on>
    <Remove empty files>
    rmdir -p debian/image.d || true

    echo >&2 "END unpatching #PATCHID# patch"

### Common stuff

    <Assert this is a kernel tree>= (<-U <-U)
    if ! [ -d kernel -a -d Documentation ]
    then
        echo >&2 "Not in kernel top level directory. Exiting"
        exit 1
    fi

    <Remove empty files>= (<-U <-U)
    echo >&2 "Removing empty files:"
    # make an exception for ./debian, or else the stamp files will go too.
    find . -path ./debian -prune -o \
           -type f -size 0 ! -name 'APPLIED*' -exec rm {} \; -print
    echo >&2 "Done."

# Reading a kpatches control file

When reading a kpatches control file, we store its contents according a data model suitable for the most recent file-format, and will map data extracted from old file-formats into the modern one.

## Generic (v1) data structures

The v1 data structures are a hash of general fields taken from the initial stance (eg. `PatchName`), a hash of default values also taken from the initial stance, and a sequence of alternative versions of the patch, each of which containing a sequence of operations to apply to the kernel tree:

    <initialise %patchinfo with skeleton structure>= (U->)
    my %patchinfo = ('general' => {},
                     'defaults' => {},
                     'alternatives' => []);

We provide a single entry-point to the controlfile reader, which will act in different ways according to the format-version.

If we have a v0 header, we migrate the relevant contents of the v0 header into the default-values section ot the `%patchinfo` structure, before going on reading.

    <definitions for the core kpatch system>+= (<-U) [<-D->]

    # RE pattern matching any kernel version (marks branch as $1 and version as $2)
    my $kversion_pattern = qr/^(\d+\.\d+)\.(\d+(?:\.\d+)?)$/;

    # Irregular kernel versions by branch, and their predecessor and successors.
    # (Key is the predecessor, value is an array ref;
    #  contains the irregular version itself and the successor.)
    my %irregular_kversions = (
      # "2.6" => {
      #   "8" => [ "8.1", "9" ]
      # }
    );

    # Gives back an array containing all the kernel versions
    # from $branch.".".$start to $branch.".".$end (inclusive).
    # Handles some irregular versions correctly, but depends on
    # the monotonic numeric relation of successive versions.
    sub iterate_over_kversions {
      my ($branch, $start, $end) = @_;
      my @ret;

      my $v = $start;
      while ($v <= $end) {
        push @ret, $branch . "." . $v;
        if (exists $irregular_kversions{$branch}->{$v}) {
          if ($irregular_kversions{$branch}->{$v}[0] <= $end) {
            push @ret, $branch . "." . $irregular_kversions{$branch}->{$v}[0];
            $v = $irregular_kversions{$branch}->{$v}[1];
          } else {
            last;
          }
        } else {
          $v++;
        }
      }

      return @ret;
    }

    <definitions for the core kpatch system>+= (<-U) [<-D->]
    sub read_control_file {
      my ($file) = @_;

      <initialise %patchinfo with skeleton structure>

      open (IN, $file) or die "cannot open $file: $!";

      # read control-file header
      read_control_file_section ($patchinfo{general});

      if ((!defined $patchinfo{general}->{'kpatch-format-version'}) or
          ($patchinfo{general}->{'kpatch-format-version'} == 0)) {
        $patchinfo{defaults} = control_file_v0_header_to_v1_defaults ($patchinfo{general});
        <read v0 file content>

        # } elsif ($patchinfo{general}->{'kpatch-format-version'} == 1) {
        # Revision 1
        # Eh, not yet :)
      } else {
        die "Unsupported Kpatch-format-version: \`" .
          $patchinfo{general}->{'kpatch-format-version'} . "'";
      }

      close IN;

      validate (\%patchinfo);
      return %patchinfo;
    }

Once a `%patchinfo` structure has been filled, we can run some checks on it, to make as sure as possible that it is correct.

FIXME: this should do much more tests !

    <definitions for the core kpatch system>+= (<-U) [<-D->]
    sub validate {
      my ($patchinfo) = @_;

      die "Patch-Id can only contain alphanumerics, hyphens, and underscores"
        if $patchinfo->{general}->{'patch-id'} =~ /[^\w-]/;

      foreach my $alternative (@{$patchinfo->{alternatives}}) {
        foreach my $operation (@{$alternative->{operations}}) {
          die "Diff file does not exist: " . $operation->{'diff-file'}
            if ($operation->{format} eq 'diff') and ! -r $operation->{'diff-file'};
          die "Diff file changes EXTRAVERSION: " . $operation->{'diff-file'}
            if 0 == system ('grep -q "^-EXTRAVERSION\>" ' . $operation->{'diff-file'});
        }
      }
    }

Reading a control-file section is something quite generic. Maybe there is a generic API for doing that now ?

This function slightly adapted from code in install-docs from Adam Di Carlo, and this is the main reason why it is not split into chunks.

    <definitions for the core kpatch system>+= (<-U) [<-D->]
    sub read_control_file_section {
      my ($pfields) = @_;

      my $alreadyreadsomething = 0;
      my ($key,$value);
      while (<IN>) {
        chomp;

        # empty line?
        if (/^\s*$/o) {
          if ($alreadyreadsomething) {
            last;
          } else {
            next;
          }
        }

        $alreadyreadsomething = 1;

        if (/^(\S+)\s*:\s*(.*)$/) {
          # first line of a new field

          ($key,$value) = (lc $1,$2);
          #print STDERR "$key -> $value\n";
          if (exists $pfields->{$key}) {
            warn "warning: $key: overwriting previous setting of control field";
          }
          $pfields->{$key} = $value;

        } elsif (/^\s+(\S.*)$/) {
          # additional line in a multi-line field
          $value = $1;
          defined($key) or die "syntax error in control file: no field specified";
          #print STDERR "$key -> $value (continued)\n";
          $pfields->{$key} .= "\n$value";

        } else {
          die "syntax error in control file: $_";
        }
      }

      return $alreadyreadsomething;
    }

## Handling v0 definitions

A v0 kpatches file is just a list of alternatives, each composed of only one `diff` operation. We have to convert them into proper structures so that the v1-based processing works as expected.

    <read v0 file content>= (<-U)
    my $cfs = {};
    while (read_control_file_section ($cfs)) {
      push (@{$patchinfo{alternatives}},
            control_file_v0_section_to_alternative($cfs));
      $cfs = {};
    }

The function that does the conversion simply maps known fields into their final location.

    <definitions for the core kpatch system>+= (<-U) [<-D->]
    sub control_file_v0_section_to_alternative {
      my ($cfs) = @_;

      # FIXME: should also process general section, and convert default
      # values - probably in a similar function

      return {
              conditions => {
                             'kernel-version' => $cfs->{'kernel-version'},
                             'architecture' => $cfs->{architecture},
                            },
              depends => $cfs->{depends},
              defaults => {},
              operations => [
                             {
                              'format' => 'diff',
                              'diff-file' => $cfs->{'patch-file'},
                              'debian-diff-file' => $cfs->{'debian-patch-file'},
                              'path-strip-level' => $cfs->{'path-strip-level'},
                             }
                            ],
             };
    }

The v0 header differs from the v1 header, in that "default" entries are implicit, and for a small well-defined set of fields. When reading a v0 header, we have to explicit them, and strip them from the general fields.

    <definitions for the core kpatch system>+= (<-U) [<-D->]
    sub control_file_v0_header_to_v1_defaults {
      my ($header) = @_;
      my %defaults;

      foreach my $key (keys %{$header}) {
        if (! grep { $key eq $_ } ('patch-name', 'patch-id')) {
          $defaults{$key} = $header->{$key};
          delete $header->{$key};
        }
      }

      return \%defaults;
    }

## Fields of kpatch files

There are 2 types of fields, denoted by constants defined the fields library. Some are mandatory, some are optional. The optional ones can all be given a default value, but mandatory ones cannot use this inheritance mechanism -- hence the `$FIELD_INHERITS` name.

Due to this inheritance mechanism, the final value of a field has to be computed according to given rules for inheritance and defaulting.

### From text fields to data structures

#### Generic alternative fields

We record all archs mentionned in the kpatches file, so that we know for which archs we have to generate the apply/unpatch scripts.

    <compute and split 'architecture' field>= (U->)
    {
     my @archfield =
     split (/, */, field_value ($patchinfo{general}, $alternative->{conditions},
                                'architecture', $FIELD_INHERITS, 'all',
                                \%patchinfo));
     $alternative->{conditions}->{architecture} = \@archfield;

     foreach my $arch (@archfield) {
       push @archs, $arch unless grep { $_ eq $arch } @archs;
     }
    }

The kernel version can be specified either as a single version or as a range.

FIXME: how a is value "all" handled ? It seems filtered out by the test below...

    <compute 'kernel-version' value or range>= (U->)
    {
      my $kversion = field_value ($patchinfo{general}, $alternative->{conditions},
                                  'kernel-version', $FIELD_MANDATORY);
      # parse "2.4.5 - 2.4.7" and "2.5.4 -" syntaxes

      my @kv = split (/\s+/, $kversion);
      if ($#kv > 0) {
        # FIXME: validity check is really too strict, but we need a
        # good kversion comparison algorithm to attempt any better
        # (ie. "-pre" and "-test" at least are special)
        $kv[0] =~ $kversion_pattern or die "Malformed kernel version: `$kv[0]'";

        my ($branch, $first) = ($1, $2);
        die "Malformed kernel-version range \`$kversion'"
          unless ($kv[1] eq '-') && ($#kv <= 2);
        if ($#kv == 1) {
          die "Unbounded ranges not supported yet: \`$kversion'";
          $kversion = [ $branch, $first ];
        } else {
          $kv[2] =~ $kversion_pattern or die "Malformed kernel version: `$kv[2]'";
          die "Cross-branch ranges are not allowed: `$kversion'"
            unless $1 == $branch;
          die "Reverse-ordered range: `$kversion'" if $2 < $first;
          $kversion = [ $branch, $first, $2 ];
        }
      } else {
        $kv[0] =~ $kversion_pattern or die "Malformed kernel version: `$kv[0]'";
      }

      $alternative->{conditions}->{'kernel-version'} = $kversion;
    }

`Depends` is a comma- and space-separated list, and will be used in the bash apply script to initialize an array, so must be made space-separated only.

    <compute 'depends' field>= (U->)
    $alternative->{depends} = field_value ($patchinfo{general}, $alternative,
                                           'depends', $FIELD_INHERITS, "",
                                           \%patchinfo);
    $alternative->{depends} =~ s/, */ /g;

#### Per-operation fields

This is simplest of all fields, nothing special is done apart from memorizing it.

    <compute 'path-strip-level' field for $op>= (U->)
    $op->{'path-strip-level'} = field_value ($patchinfo{general}, $op,
                                             'path-strip-level', $FIELD_INHERITS, 1,
                                             $alternative, \%patchinfo);

Since the diff files may be compressed, we are only recording here their location in the source tree, and we are delaying the recording of their final filename until they are installed and maybe compressed. This will be done as we `<install the diff files>`.

    <compute (debian-)diff-file fields for $op>= (U->)
    $op->{'diff-file'} = field_value ($patchinfo{general}, $op,
                                      'diff-file', $FIELD_MANDATORY);
    $op->{'debian-diff-file'} = field_value ($patchinfo{general}, $op,
                                             'debian-diff-file', $FIELD_INHERITS);

### Field-handling library

#### Field inheritance

    <definitions for the core kpatch system>+= (<-U) [<-D->]
    my $FIELD_MANDATORY = 0;
    my $FIELD_INHERITS = 1;

`$inherits` should have one of the values defined above. `$default` can be `undef` when there is no default value for the field.

    <definitions for the core kpatch system>+= (<-U) [<-D->]
    sub field_value {
      my ($general, $hash, $name, $inherits, $default, @defaultlists) = @_;

      my $value = $hash->{$name};
      if ($inherits == $FIELD_MANDATORY) {
        die "Patchfile info lacks $name field" unless defined $value;
      }
      # first go through explicit default values
      foreach my $defaultlist (@defaultlists) {
        $value = $defaultlist->{defaults}->{$name} unless defined $value;
      }
      # then use hardcoded default as a fallback
      if (defined $default) {
        $value = $default unless defined $value;
      }

      return $value;
    }

#### Recording a patchfile field for later use

We use hashes to store sequences of values for each arch. These sequences of values will be directly used to produce the apply/unpatch scripts from their templates.

    <definitions for the core kpatch system>+= (<-U) [<-D]
    # records a field value for a given patchfile on a given arch
    sub record_patchfile_field {
      my ($hash, $arch, $value) = @_;

      if (defined $hash->{$arch}) {
        $hash->{$arch} .= " $value";
      } else {
        $hash->{$arch} = "$value";
      }
    }

# Patch processing

In a single `apply` script, all alternatives are handled, and the one to use will be determined at runtime.

We currently support only one operation per alternative (v0 format), and a great deal of this restriction lies in this part, so I will not spend to much time structuring this part, until things are properly implemented.

To get proper v1 support we will put everything including kpatches file parsing into a generic apply script, and have patch packages only symlink to this one.

    <generate apply/unpatch scripts for given $package and kpatches $file>= (<-U)
    my %kversions=();
    my %patchfiles=();
    my %debpatchfiles=();
    my %striplevels=();
    my %depends=();
    my @archs=();

    # put the right files in the right places
    foreach my $alternative (@{$patchinfo{alternatives}}) {
      my $op = $alternative->{operations}->[0];
      <compute values for $alternative>
      <install the diff files>
      <record the $alternative and single $op>
    }
    <output apply/unpatch scripts>

For each alternative, we compute the value to use for each field, possibly using a default value if none was provided, and in some cases splitting into an array. The final value is stored in the place of the original one.

Diff files thus identified are then installed, and entries for the diff files are then recorded for use when generating the apply/unpatch scripts for each of the named architectures.

There is no particular ordering required between the computation of the various fields, but to help moving towards v1 format, we group them by first handling those specific to the alternative itself, and then those specific to the single diff operation in this alternative.

    <compute values for $alternative>= (<-U)
    <compute 'depends' field>
    <compute and split 'architecture' field>
    <compute 'kernel-version' value or range>

    <compute 'path-strip-level' field for $op>
    <compute (debian-)diff-file fields for $op>

All diff files are now located under a single `/usr/src/kernel-patches/diffs/` tree.

    <install the diff files>= (<-U)
    my $srcdir = "/usr/src/kernel-patches/diffs/$patchid";
    doit ("mkdir",  "-p", "$tmp$srcdir") unless -d "$tmp$srcdir";

    $op->{'installed-diff-file'} = "$srcdir/" . basename($op->{'diff-file'});
    doit ("cp", $op->{'diff-file'}, "$tmp$op->{'installed-diff-file'}");
    doit ("gzip", "-9fq", "$tmp$op->{'installed-diff-file'}");
    $op->{'installed-diff-file'} = "$op->{'installed-diff-file'}.gz"
      if -r "$tmp$op->{'installed-diff-file'}.gz";

    if (defined $op->{'debian-diff-file'}) {
      $op->{'installed-debian-diff-file'} = "$srcdir/" . basename($op->{'debian-diff-file'});
      doit ("cp", $op->{'debian-diff-file'}, "$tmp$op->{'installed-debian-diff-file'}");
      doit ("gzip", "-9fq", "$tmp$op->{'installed-debian-diff-file'}");
      $op->{'installed-debian-diff-file'} = "$op->{'installed-debian-diff-file'}.gz"
        if -r "$tmp$op->{'installed-debian-diff-file'}.gz";
    } else {
      $op->{'installed-debian-diff-file'} = '';
    }

Version ranges are currently emulated, by duplicating the entry for each integer kernel revision in the range.

This is a temporary behaviour, which does not allow pre-release kernel versions to be seen as part of the range. Adequate version-comparison function and data-structures will be used instead, when time permits.

    <record the $alternative and single $op>= (<-U)
    foreach my $arch (@{$alternative->{conditions}->{architecture}}) {
      if ((ref $alternative->{conditions}->{'kernel-version'}) eq 'ARRAY') {
        my @kernel_version = @{$alternative->{conditions}->{'kernel-version'}};
        foreach my $version (iterate_over_kversions($kernel_version[0], $kernel_version[1], $kernel_version[2])) {
          record_patchfile_field (\%kversions, $arch, $version);
          record_patchfile_field (\%striplevels, $arch, $op->{'path-strip-level'});
          record_patchfile_field (\%depends, $arch, '"' . $alternative->{depends} . '"');
          record_patchfile_field (\%patchfiles, $arch, '"' . $op->{'installed-diff-file'} . '"');
          record_patchfile_field (\%debpatchfiles, $arch, '"' . $op->{'installed-debian-diff-file'} . '"');
        }
      } else {
        record_patchfile_field (\%kversions, $arch,
                                $alternative->{conditions}->{'kernel-version'});
        record_patchfile_field (\%striplevels, $arch, $op->{'path-strip-level'});
        record_patchfile_field (\%depends, $arch, '"' . $alternative->{depends} . '"');
        record_patchfile_field (\%patchfiles, $arch, '"' . $op->{'installed-diff-file'} . '"');
        record_patchfile_field (\%debpatchfiles, $arch, '"' . $op->{'installed-debian-diff-file'} . '"');
      }
    }

`apply` and `unpatch` are currently generated from templates, by substituting in there the values computed from the patch definition expressed in the `kpatches` file.

    <output apply/unpatch scripts>= (<-U)
    foreach my $arch (@archs) {
      my $pdir = "/usr/src/kernel-patches/$arch";
      foreach my $script ('apply', 'unpatch') {
        doit ("mkdir", "-p", "$tmp$pdir/$script");
        complex_doit ('sed < @TMPLDIR@/' . "$script.tmpl >$tmp$pdir/$script/$patchid" .
                      ' -e \'s/#PATCHID#/' . $patchinfo{general}->{'patch-id'} . '/g\'' .
                      ' -e \'s/#CLEANPATCHID#/' . $patchinfo{general}->{'clean-patch-id'} . '/g\'' .
                      ' -e \'s|#PATCHNAME#|' . $patchinfo{general}->{'patch-name'} . '|g\'' .
                      " -e 's/#DHKPVERS#/@DHKPVERS@/g'" .
                      " -e 's,#TMPLDIR#,@TMPLDIR@,g'" .
                      " -e 's/#DEPENDS#/$depends{$arch}/g'" .
                      " -e 's/#KVERSIONS#/$kversions{$arch}/g'" .
                      " -e 's|#PATCHFILES#|$patchfiles{$arch}|g'" .
                      " -e 's|#DEBPATCHFILES#|$debpatchfiles{$arch}|g'" .
                      " -e 's/#PATCHARCH#/$arch/g'" .
                      " -e 's/#STRIPLEVELS#/$striplevels{$arch}/g'"
                     );
        doit ("chmod", "0755", "$tmp$pdir/$script/$patchid");

        doit ("mkdir", "-p", "$tmp/usr/share/doc/$package");
        doit ('cp',
              '@TMPLDIR@/README-kernelpatch.Debian',
              "$tmp/usr/share/doc/$package/");
      }
    }

# Index of chunks

- <Apply dependencies>: U1, D2
- <Apply the patch>: U1, D2
- <apply.tmpl>: D1
- <Apply.tmpl constants>: U1, D2
- <Assert diff file is still there>: U1, D2
- <assert kpatch:depends is referenced in control file>: U1, D2
- <Assert patch applies, using dry-run>: U1, D2
- <Assert stamp file is not empty>: U1, D2
- <Assert this is a kernel tree>: U1, U2, D3
- <Assert we have an alternative to apply>: U1, D2
- <Check for dependencies>: U1, D2
- <compute 'depends' field>: D1, U2
- <compute 'kernel-version' value or range>: D1, U2
- <compute 'path-strip-level' field for $op>: D1, U2
- <compute (debian-)diff-file fields for $op>: D1, U2
- <compute and split 'architecture' field>: D1, U2
- <compute values for $alternative>: U1, D2
- <Create script to have this patch registered in doc-dir>: U1, D2
- <definitions for the core kpatch system>: U1, D2, D3, D4, D5, D6, D7, D8, D9, D10, D11
- <dh_installkpatches.in>: D1
- <Do not unpatch if there are still dependants>: U1, D2
- <generate apply/unpatch scripts for given $package and kpatches $file>: U1, D2
- <Get current kernel version>: U1, D2
- <Handle dependencies>: U1, D2
- <initialise %patchinfo with skeleton structure>: D1, U2
- <install the diff files>: U1, D2
- <Make apply idempotent>: U1, D2
- <output apply/unpatch scripts>: U1, D2
- <process binary package $package>: U1, D2
- <Read contents from stamp file>: U1, D2
- <read v0 file content>: U1, D2
- <Record parameters for applying alternative $IDX>: U1, D2
- <record the $alternative and single $op>: U1, D2
- <Remove empty files>: U1, U2, D3
- <Remove the patch>: U1, D2
- <Remove those patches we depended on>: D1, U2
- <Select alternative to apply, as $IDX>: U1, D2
- <set kpatch:Depends substvar>: U1, D2
- <Store necessary info for unpatching>: U1, D2
- <unpatch.tmpl>: D1

*