Create Your Own Debian Mirror With Debmirror

We will walk through these steps to get the local Debian mirror up and running:

  1. make sure you have enough space on your harddrive
  2. install debmirror, configure a cronjob to sync data
  3. make your mirror available to your clients (via apache or nfs)
  4. keep an eye on your cronjob - from time to time your script may have trouble to sync
  5. configure sources.list to use your local mirror

1. Space Requirements

You will need some space to host the mirror.

My local mirror hosts the distributions for wheezy, wheezy-updates and jessie with architectures i386 and amd64.

This setup currently (2015-11) consumes ~140 GB.

2. Installation

The Basics

My current setup runs on wheezy.
Parts of this setup date back to the days when lenny was your most recent choice. So be aware of that.

First, install the debmirror and debian-keyring package:

apt-get install debmirror debian-keyring

Decide where to store your mirror:

mkdir /srv/mirror

Add a user who will run the mirror script:

groupadd mirror
useradd -d /srv/mirror -c "Debmirror" -g mirror mirror

Change permissions:

chown -R mirror.mirror /srv/mirror

Handle GPG Keys

This is the most tricky part.

If something breaks that's usually because some keys
have changed/vanished or are missing.

Import gpg keys.
If you are missing a key for some reason you can find the key in question at http://keyserver.ubuntu.com. Search for “Debian Archive Automatic Signing Key ” and pick the ID.

# get mirror
su - mirror
 
# 1st import the following:
gpg --no-default-keyring --keyring trustedkeys.gpg --import /usr/share/keyrings/debian-archive-keyring.gpg
 
# check the key list, it should give you at least:
gpg --list-keys --keyring trustedkeys.gpg
 
pub   1024D/2D230C5F 2006-01-03 [expired: 2007-02-07]
uid                  Debian Archive Automatic Signing Key (2006) <ftpmaster@debian.org>
 
pub   1024D/6070D3A1 2006-11-20 [expires: 2009-07-01]
uid                  Debian Archive Automatic Signing Key (4.0/etch) <ftpmaster@debian.org>
 
pub   1024D/ADB11277 2006-09-17
uid                  Etch Stable Release Key <debian-release@lists.debian.org>

If you are missing e.g. ID 6070D3A1 then you can try the following:

:!: Warning: gpg uses trustdb.gpg as the default keyring; debmirror however expects the keys to reside in trustedkeys.gpg !

gpg --no-default-keyring --keyring trustedkeys.gpg --keyserver keyserver.ubuntu.com --recv-keys B98321F9

:!: From time to time keys will be updated or removed. Install the current debian-archive-keyring.deb and run

gpg  --no-default-keyring --keyring trustedkeys.gpg --import /usr/share/keyrings/debian-archive-keyring.gpg 

gpg --list-keys --keyring trustedkeys.gpg
pub   1024D/2D230C5F 2006-01-03 [expired: 2007-02-07]
uid                  Debian Archive Automatic Signing Key (2006) <ftpmaster@debian.org>

pub   1024D/6070D3A1 2006-11-20 [expires: 2009-07-01]
uid                  Debian Archive Automatic Signing Key (4.0/etch) <ftpmaster@debian.org>

pub   1024D/ADB11277 2006-09-17
uid                  Etch Stable Release Key <debian-release@lists.debian.org>

pub   1024D/BBE55AB3 2007-03-31 [expires: 2010-03-30]
uid                  Debian-Volatile Archive Automatic Signing Key (4.0/etch)
sub   2048g/36CA98F3 2007-03-31 [expires: 2010-03-30]

pub   1024D/F42584E6 2008-04-06 [expires: 2012-05-15]
uid                  Lenny Stable Release Key <debian-release@lists.debian.org>

pub   4096R/55BE302B 2009-01-27 [expires: 2012-12-31]
uid                  Debian Archive Automatic Signing Key (5.0/lenny) <ftpmaster@debian.org>

pub   2048R/6D849617 2009-01-24 [expires: 2013-01-23]
uid                  Debian-Volatile Archive Automatic Signing Key (5.0/lenny)

This list is current for 2009-05-04.

New keys for jessie (2015-05):

$ gpg --no-default-keyring --keyring trustedkeys.gpg --keyserver keyserver.ubuntu.com --recv-keys 2B90D010
gpg: fordere Schlüssel 2B90D010 von hkp-Server keyserver.ubuntu.com an
gpg: Schlüssel 2B90D010: Öffentlicher Schlüssel "Debian Archive Automatic Signing Key (8/jessie) <ftpmaster@debian.org>" importiert
gpg: öff. Schlüssel des uneingeschränkt vertrautem Schlüssel 9A5CBC06 nicht gefunden
gpg: öff. Schlüssel des uneingeschränkt vertrautem Schlüssel 76CE518A nicht gefunden
gpg: 3 marginal-needed, 1 complete-needed, PGP Vertrauensmodell
gpg: Tiefe: 0  gültig:   2  unterschrieben:   0  Vertrauen: 0-, 0q, 0n, 0m, 0f, 2u
gpg: Anzahl insgesamt bearbeiteter Schlüssel: 1
gpg:                              importiert: 1  (RSA: 1)

$ gpg --no-default-keyring --keyring trustedkeys.gpg --keyserver keyserver.ubuntu.com --recv-keys 518E17E1
gpg: fordere Schlüssel 518E17E1 von hkp-Server keyserver.ubuntu.com an
gpg: Schlüssel 518E17E1: Öffentlicher Schlüssel "Jessie Stable Release Key <debian-release@lists.debian.org>" importiert
gpg: öff. Schlüssel des uneingeschränkt vertrautem Schlüssel 9A5CBC06 nicht gefunden
gpg: öff. Schlüssel des uneingeschränkt vertrautem Schlüssel 76CE518A nicht gefunden
gpg: 3 marginal-needed, 1 complete-needed, PGP Vertrauensmodell
gpg: Tiefe: 0  gültig:   2  unterschrieben:   0  Vertrauen: 0-, 0q, 0n, 0m, 0f, 2u
gpg: Anzahl insgesamt bearbeiteter Schlüssel: 1
gpg:                              importiert: 1  (RSA: 1)

:!: NOTE:

If you do not have ~/.gnupg/trustedkeys.gpg then you can try to copy the complete archive keyring file (as user 'mirror'):

cp /usr/share/keyrings/debian-archive-keyring.gpg ~/.gnupg/trustedkeys.gpg 

To list which keys are in ~/.gnupg/trustedkeys.gpg use this command:

gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --list-key 

Script & Cronjob

Create a script to be run by cron as root.
The script below will mirror wheezy and jessie.

#!/bin/bash
 
# sourcehost: choose a mirror in your proximity!
HOST=ftp2.de.debian.org;
 
# destination directory
DEST=/srv/mirror/debian
 
# Debian version(s) to mirror
DIST=wheezy,wheezy-updates,jessie
 
# architecture
ARCH=i386,amd64
 
# -----------------------------------------
# check whether we're online:
 
ping $HOST -c 4 -i 3 >/dev/null 2>&1
 
if [ "$?" -eq 0 ]; then
 # yes, we're online :-)
 logger -t mirror[$$] updating Debian mirror
 
 su mirror -c \
 "debmirror ${DEST} \
 --nosource \
 --host=${HOST} \
 --root=/debian \
 --dist=${DIST} \
 --section=main,contrib,non-free,main/debian-installer \
 --arch=${ARCH} \
 --passive --cleanup \
 $VERBOSE"
 
 logger -t mirror[$$] finished updating Debian mirror
fi

Edit /etc/cron.d/local-debmirror:

# debmirror
38 04 * * 1-5 root /root/scripts/mirror

3. Make the Mirror Available

The easiest way would be to set up a webserver (e.g. apache) to serve the data to your clients.

Your mirror directory tree should look like:

tree -d -L 2 /srv/mirror/debian
/srv/mirror/debian
├── dists
│   ├── jessie
│   ├── oldstable -> wheezy
│   ├── oldstable-updates -> wheezy-updates
│   ├── stable -> jessie
│   ├── wheezy
│   └── wheezy-updates
├── pool
│   ├── contrib
│   ├── main
│   └── non-free
└── project
    └── trace

Decide how you want to access the mirror - debmirror.example.com sounds great, doesn't it? Don't forget to update your DNS!

Create a virtual host configuration like so:

/etc/apache2/sites-available/debmirror
<VirtualHost *:80>
        ServerName debmirror.example.com
        ServerAdmin webmaster@localhost

        DocumentRoot /srv/mirror
        <Directory />
                Options FollowSymLinks
                AllowOverride None
        </Directory>
        <Directory /srv/mirror>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride All
                Order allow,deny
                allow from all
        </Directory>

        ErrorLog /var/log/apache2/debmirror.example.com-error.log

        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn

        CustomLog /var/log/apache2/debmirror.example.com-access.log combined
        ServerSignature On

</VirtualHost>

To tune your landing page on debmirror.example.com, you can pimp your directory index.

And finally enable the shiny new site:

a2ensite debmirror
/etc/init.d/apache force-reload

4. Cronjob Monitoring

The least you can do is to make sure you recieve any output your cronjob generates by mail.

So check whether your /etc/aliases is setup properly to redirect any mail to your mirror user to your preferred mailaccount:

aliases
# /etc/aliases
root: localuser, you@example.com
[...]
mirror: you@example.com
[...]

Run newaliases and verify you receive mail sent to the mirror user. If you are not directly connected to the internet and sitting behind a NAT router, you will have to tune your mailserver configuration.

5. Configure Your Clients

Now we are done on the server side. It is time to actually use the local mirror. All we have to do is to enable the mirror in /etc/apt/sources.list - just replace the existing upstream resource with something like that:

sources.list
[...]

# Wheezy:
deb http://debmirror.example.com/debian/ wheezy main contrib non-free

[...]

Now you can happily apt-get update and with your next apt-get install whatever the selected package(s) will be retrieved from your local mirror.

Anything you're missing?
Feel free to leave your comments below - or contact me via Twitter: @T_Baecker
I'll do my best to improve this guide.

debian/debmirror.txt · Last modified: 2015-12 by tb
Driven by DokuWiki Recent changes RSS feed Valid CSS Valid XHTML 1.0 ipv6 ready