Create your own Debian mirror with debmirror

Create your own Debian mirror with debmirror

We will walk through these steps to get the local Debian mirror up and running:

  1. make sure you have enough space on your harddrive
  2. install debmirror, configure a cronjob to sync data
  3. make your mirror available to your clients (via apache or nfs)
  4. keep an eye on your cronjob – from time to time your script may have trouble to sync
  5. configure sources.list to use your local mirror

1. Space requirements

You will need some space to host the mirror.

My local mirror hosts the distributions for jessie and stretch with architecture amd64.

This setup currently (2017-11) consumes ~113 GB.

2. Installation

The basics

My debmirror runs on jessie. But that’s not really relevant here.

First, install the debmirror and debian-keyring package:

apt-get install debmirror debian-keyring

Decide where to store your mirror:

mkdir /srv/mirror

Add a user who will run the mirror script:

groupadd mirror
useradd -d /srv/mirror -c "Debmirror" -g mirror mirror

Change permissions:

chown -R mirror.mirror /srv/mirror

Handle gpg keys

gpg uses trustdb.gpg as the default keyring; debmirror however expects the keys to reside in trustedkeys.gpg !

If something breaks that’s usually because some keys have been added to the official keyring and are missing locally.

Import gpg keys.
The easiest thing is to import the official debian-archive-keyring for debmirror.

# become mirror
su - mirror
 
# 1st import the following:
gpg --no-default-keyring --keyring trustedkeys.gpg --import /usr/share/keyrings/debian-archive-keyring.gpg
 
# check the key list, it should give you something like this:
gpg --list-keys --keyring trustedkeys.gpg
 
/srv/mirror/.gnupg/trustedkeys.gpg
----------------------------------
 
pub   4096R/473041FA 2010-08-27 [verfällt: 2018-03-05]
uid                  Debian Archive Automatic Signing Key (6.0/squeeze) <ftpmaster@debian.org>
 
pub   4096R/B98321F9 2010-08-07 [verfallen: 2017-08-05]
uid                  Squeeze Stable Release Key <debian-release@lists.debian.org>
 
pub   4096R/65FFB764 2012-05-08 [verfällt: 2019-05-07]
uid                  Wheezy Stable Release Key <debian-release@lists.debian.org>
 
pub   4096R/46925553 2012-04-27 [verfällt: 2020-04-25]
uid                  Debian Archive Automatic Signing Key (7.0/wheezy) <ftpmaster@debian.org>
 
pub   4096R/2B90D010 2014-11-21 [verfällt: 2022-11-19]
uid                  Debian Archive Automatic Signing Key (8/jessie) <ftpmaster@debian.org>
 
pub   4096R/518E17E1 2013-08-17 [verfällt: 2021-08-15]
uid                  Jessie Stable Release Key <debian-release@lists.debian.org>
 
pub   4096R/C857C906 2014-11-21 [verfällt: 2022-11-19]
uid                  Debian Security Archive Automatic Signing Key (8/jessie) <ftpmaster@debian.org>
 
pub   4096R/1A7B6500 2017-05-20 [verfällt: 2025-05-18]
uid                  Debian Stable Release Key (9/stretch) <debian-release@lists.debian.org>
 
pub   4096R/F66AEC98 2017-05-22 [verfällt: 2025-05-20]
uid                  Debian Archive Automatic Signing Key (9/stretch) <ftpmaster@debian.org>
sub   4096R/B7D453EC 2017-05-22 [verfällt: 2025-05-20]
 
pub   4096R/8AE22BA9 2017-05-22 [verfällt: 2025-05-20]
uid                  Debian Security Archive Automatic Signing Key (9/stretch) <ftpmaster@debian.org>
sub   4096R/331F7F50 2017-05-22 [verfällt: 2025-05-20]

From time to time keys will be updated or removed. We just need to sync keys from debian-archive-keyring. gpg will automatically skip duplicates and add what is missing. This step is vitally important when you add a new distribution like stretch.

# update the keyring, same procedure as install:
gpg --no-default-keyring --keyring trustedkeys.gpg --import /usr/share/keyrings/debian-archive-keyring.gpg 
 
# and verify what's in the keyring:
gpg --list-keys --keyring trustedkeys.gpg

NOTE:

If you do not have ~/.gnupg/trustedkeys.gpg then you can also copy the complete archive keyring file (as user ‘mirror’):

cp /usr/share/keyrings/debian-archive-keyring.gpg ~/.gnupg/trustedkeys.gpg 

Script & Cronjob

Create a script to be run by cron as root.
The script below will mirror jessie and stretch.

#!/bin/bash
 
# sourcehost: choose a mirror in your proximity!
HOST=ftp2.de.debian.org;
 
# destination directory
DEST=/srv/mirror/debian
 
# Debian version(s) to mirror
DIST=jessie,stretch
 
# architecture
ARCH=amd64
 
# log timestamp
logger -t mirror[$$] updating Debian mirror
 
su mirror -c \
 "debmirror ${DEST} \
 --nosource \
 --host=${HOST} \
 --root=/debian \
 --dist=${DIST} \
 --section=main,contrib,non-free,main/debian-installer \
 --i18n \
 --arch=${ARCH} \
 --passive --cleanup \
 $VERBOSE"
 
logger -t mirror[$$] finished updating Debian mirror

Edit /etc/cron.d/local-debmirror:

# debmirror
38 04 * * 1-5 root /root/scripts/mirror

3. Make the mirror available

The easiest way would be to set up a webserver (e.g. apache) to serve the data to your clients.

Your mirror directory tree should look like:

tree -d -L 2 /srv/mirror/debian
/srv/mirror/debian
├── dists
│   ├── jessie
│   ├── oldstable -> jessie
│   ├── stable -> stretch
│   └── stretch
├── pool
│   ├── contrib
│   ├── main
│   └── non-free
└── project
    └── trace

Decide how you want to access the mirror – debmirror.example.com sounds great, doesn’t it? Don’t forget to update your DNS!

Create a virtual host configuration like so: /etc/apache2/sites-available/debmirror

<VirtualHost *:80>
        ServerName debmirror.example.com
        ServerAdmin webmaster@localhost

        DocumentRoot /srv/mirror
        <Directory />
                Options FollowSymLinks
                AllowOverride None
        </Directory>
        <Directory /srv/mirror>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride All
                Order allow,deny
                allow from all
        </Directory>

        ErrorLog /var/log/apache2/debmirror.example.com-error.log

        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn

        CustomLog /var/log/apache2/debmirror.example.com-access.log combined
        ServerSignature On

</VirtualHost>

To tune your landing page on debmirror.example.com, you can pimp your directory index.

And finally enable the shiny new site:

a2ensite debmirror
/etc/init.d/apache force-reload

4. Cronjob monitoring

The least you can do is to make sure you recieve any output your cronjob generates by mail.

So check whether your /etc/aliases is setup properly to redirect any mail to your mirror user to your preferred mailaccount: aliases

# /etc/aliases
root: localuser, you@example.com
[...]
mirror: you@example.com
[...]

Run newaliases and verify you receive mail sent to the mirror user. If you are not directly connected to the internet and sitting behind a NAT router, you will have to tune your mailserver configuration.

5. Configure your clients

Now we are done on the server side. It is time to actually use the local mirror. All we have to do is to enable the mirror in /etc/apt/sources.list – just replace the existing upstream resource with something like that: sources.list

[...]

# Jessie:
deb http://debmirror.example.com/debian/ jessie main contrib non-free

[...]

Now you can happily apt-get update and with your next apt-get install whatever the selected package(s) will be retrieved from your local mirror.

One thought on “Create your own Debian mirror with debmirror

  1. Brilliant mate, thanks so much for this.
    I was pulling my hair out trying to get this working!

Leave a Reply

Your email address will not be published. Required fields are marked *