Taking control over Debian and its package repositories

October 29, 2013 – 2:45 pm by Adnan Hodzic

When we talk about Debian we must talk in the superlative. One of the reasons why Google and International Space Station are choosing Debian as their default Linux distribution is because it has (by far) the biggest package collection. At the time of writing this document, there are 61801 packages in Debian Sid (Unstable/Development distribution).

But as with many things in life, your greatest asset can be also your biggest liability—unless you take things under control. As an example, people usually complain how package versions in Debian “Stable” are too old, and they are spot on right ignorant. The author of this document has never used Debian “Stable” outside of production and has solely relied on some of the ingenious mechanisms provided by Debian, which when properly configured can provide you with unlimited possibilities.

APT Pinning

Pinning allows you to install and run package versions from other (Testing/Unstable/Experimental) Debian branches without having to upgrade the whole distribution to that particular branch.

Example 1:

You are running Debian 7.2 (Wheezy), and you want latest “libjmagick6-java” version (i.e: 6.6.9), however you only see the version which is present in Stable repository (6.2.6). When you look for the package on Debian packages, you can see that the version you want is present in Testing/Unstable.

Example 2:

You are running Debian 7.2 (Wheezy) and would want the latest GNOME Shell from Unstable/Experimental. Generally, you would want to turn to Backports. However, if the package you are looking for is not there, you might be out of luck. That is where APT Pinning steps up.

Building blocks

First you specify all the desired branches in your /etc/apt/source.list file. i.e:

# wheezy
deb http://ftp.nl.debian.org/debian/ wheezy main contrib non-free
deb-src http://ftp.nl.debian.org/debian/ wheezy main contrib non-free

# wheezy Security
deb http://security.debian.org/ wheezy/updates main contrib non-free
deb-src http://security.debian.org/ wheezy/updates main contrib non-free

# wheezy-updates, previously known as 'volatile'
deb http://ftp.nl.debian.org/debian/ wheezy-updates main contrib non-free
deb-src http://ftp.nl.debian.org/debian/ wheezy-updates main contrib non-free

# wheezy-backports
deb http://ftp.nl.debian.org/debian/ wheezy-backports main contrib non-free
deb-src http://ftp.nl.debian.org/debian/ wheezy-backports main contrib non-free

# jessie
deb http://ftp.nl.debian.org/debian/ jessie main contrib non-free
deb-src http://ftp.nl.debian.org/debian/ jessie main contrib non-free

# jessie Security
deb http://security.debian.org/ jessie/updates main contrib non-free
deb-src http://security.debian.org/ jessie/updates main contrib non-free

# sid
deb http://ftp.nl.debian.org/debian/ sid main contrib non-free
deb-src http://ftp.nl.debian.org/debian/ sid main contrib non-free

# experimental
deb http://ftp.nl.debian.org/debian/ experimental main contrib non-free
deb-src http://ftp.nl.debian.org/debian/ experimental main contrib non-free

Then you’ll want to create a preferences file in /etc/apt/preferences i.e:

Package: *
Pin: release a=testing
Pin-Priority: 600

Package: *
Pin: release a=stable
Pin-Priority: 300

Package: *
Pin: release a=unstable
Pin-Priority: 200

Package: *
Pin: release a=experimental
Pin-Priority: 100

In this case, if we run lsb_release -a we will get the following information about the version of Debian that we are running:

Distributor ID: Debian
Description: Debian GNU/Linux testing (jessie) 
Release: testing 
Codename: jessie

So what just happened here?

  • In source.list file you declare which repositories you want to be available at your disposal
  • in preferences file you define which branch you want to use based on the priority you gave to certain branch/es.

Rule of thumb: the higher the number it is, the higher the priority that branch will have. In this case “Jessie/Testing” is our base distribution, and when it comes to package installation, you prefer branches in following order:

  1. Jessie/Testing
  2. Stable/Wheezy
  3. Sid/Unstable
  4. Experimental

So if you were about to install “libjmagick6-java” version from Sid/Unstable what would happen?

First you need to specify: apt-get -t sid install libjmagick6-java
The -t(–target-release) option specifies which branch you want to install your package from (i.e: -t sid/testing/experimental). So in this case, libjmagick6-java package binary will be pulled and installed from Sid. In the case where this package has dependencies, they will first be fetched from “Jessie/Testing” since that is the branch that has the highest priority. In the case where that specific package version is not available in this branch, APT will look for the same version in the branch which has the next highest priority, and so on, until the specified version is found in one of them.

Dissecting the preferences file

From what you have read so far, you will be able to install packages from other branches. However, with one single typo you will end up running Unstable system and you will not have a clue what happened.

Therefore it might help to explain some of the widely used options in preferences as well its definitions.

Package: *
Pin: release a=testing
Pin-Priority: 600

Package field:

Asterisk (*), represents all packages names. Using wildcards will not work, thus specifying Package: ebuddy-* will not work.

Pin field:

Most widely used options are “Pin: release a=branch” or “Pin: release o=ebuddy“. Although another great example of using Pin field is specifying repository origin, i.e:

Pin: origin http://apt.ebuddy-office.net/ebuddy

A great example of using “Package” with combination of “Pin” field is specifying a particular package name. In other words, you want to keep a particular package at a particular version (not depending on which branch that version is).

Package: ebuddy-xms-cli 
Pin: version 0.5.34 
Pin-Priority: 900

Pin-Priority field:

As previously mentioned, it is as simple as the higher the number is, the higher priority a particular branch has. However, there are caveats as well as additional features worth mentioning. From my personal experience, the highest priority number you should go with is <=1000. As in case you use 1001 for “Pin-Priority” it will enable downgrade disregarding any prior package priority. Going negative on other side will disable package version upgrade. Thus, if you give a package Pin-Priority of -10, any further updates of that package will be disabled. As previously noted, stay in rage of 100 to 900.

An important note regarding preferences file, you will get more desired/accurate results by using branches name stable/testing/unstable rather than codenames i.e: wheezy/jessie/sid

Different ways of being able to use apt-get -t option:

  • apt-get -t branch install package_name
  • apt-get install -t branch package_name
  • apt-get install package_name/branch

Setting restrictions/pinning on repositories?

One of the most obvious examples is Backports repository, which once added is by default pinned to 100. So in order to install something from backports, you usually need to specify apt-get -t wheezy-backports install packaga_name. Otherwise even though the package name might have a higher version, it will not be made an installation candidate (which is the default behaviour in Debian).

backports example

Package installation candidate with Backports enabled

So you might want to mimic this behaviour on your repositories. In our case we do this for repositories with ebuddy-* packages. The sole reason we do this is because we don’t want ebuddy-* packages to be upgraded during the distribution upgrade process, etc.

Adding “NotAutomatic: yes” field to your Release file will do the trick, however you won’t be able to just edit the current Release file within the repository you want to make these changes to. But there’s a hack which will enable you to do so, in this case it’s using the sed (unix stream editor) utility.

Running sed -i ‘/^Date: / aNotAutomatic: yes’ Release in directory where Release file is located will do the trick.

"NotAutomatic: yes" added to "Release" file

“NotAutomatic: yes” added to “Release” file

Since all of this seems like a lot of manual labor, our “update-deb” script which adds/removes packages to apt-ftparchive has been updated with this additional feature.

Thus adding “NotAutomatic: yes” field to your Release file is as simple as running “update-deb.sh -i -u -p all

update-deb -u flag

ButAutomaticUpgrades: yes

This is another field you might be interested in using in combination with “NotAutomatic: yes”. Since adding the previously noted field might have strict implications on your repository, you might want to have all updates from that repository automatically installed. This field will enable you to do so.

Summary

Using options described in this document will help you extend APT functionality to the maximum. I am certain that I have overlooked some of the options, as, in general, extending this goes into the same category as asking a submarine captain how deep his submarine can go. Not even he knows until he tries it.

Happy hacking.

  • What do “a” and “o” stand for in the following command: “Pin: release a=branch” or “Pin: release o=ebuddy“?
    Is there any “man” entry to read about available preferences file fields (related to the packaging) and explanation for Pin, Package, Pin-Priority fields?

  • “a” stands for branch, i.e: a=stable/a=testing/etc …

    “o” stands for suite, i.e: o=ebuddy/o=custom/o=debian/etc …

    AFAIK, there’s no man page on preferences file as it’s completely optional.

  • Interesting, brAnch (a?), o – for suite, even unexplainable ))))

  • Moray

    man apt_preferences

  • Also there is a preferences.d directory where you probably can put multiple preferences files per use case.

  • Ok, so according to man apt_preferences,
    a – is for “archive”
    o – is for “origin”
    Also there are c – for component, l – for label, and n – for codename.
    Also according to man apt_preferences, Package field actually supports regex, as an example from the man:

    Package: gnome* /kde/
    Pin: release n=experimental
    Pin-Priority: 500

  • Pingback: Taking control over Debian and its package repositories « eBuddy Tech Blog()

  • Eugen Martynov

    I think the email ahodzic@ebuddy.com is not working anymore

  • Thanks for the heads up, I’ll have the contact info changed.

    In meantime if you need to contact me via email use: adnan AT hodzic.org

  • Pingback: FoolControl – Phear the penguin » Blog Archive » Juniper/Pulse Secure VPN on Linux (2015 edition)()