How To Create Repository In Ubuntu

Create your own custom and authenticated APT repository

Internet is full of resources on how to create an APT local repository and how to expose it via HTTP. Unfortunately, APT changed quite a lot since these tutorials and howtos have been written: most of them do not work out of the box. Examples of changes are SHA1 removal, which broke many many repos, and turning on authenticated repositories by default since APT 1.1.

This howto works with Ubuntu 16.04.4 (latest LTS at the time of writing) and uses SHA256 signatures. It will help setting up GPG keys, draft a script to update your repo and provide many crunchy details about APT internals.

APT Repository Layout

The ultimate goal of the repository is to be exposed via HTTP (using apache httpd), so the best place we can store the repo is inside the DocumentRoot of httpd, which defaults to /var/www/html/. We'll use a subfolder here, namely /var/www/html/repo.

The listing below shows the organisation inside this repository:

          /var/www/html/repo/
|-- amd64/           ← where arch=amd64 *.deb packages will go
|   |-- package1-version1.deb
|   └-- ... more deb files here ...
|-- InRelease        ← "in-file-signed"                         Release file (see below)
|-- KEY.gpg          ← gpg key signing the repository and packages
|-- Packages         ← description of repo's packages
|-- Packages.gz      ← gzip'ed version of the Package file above
|-- Release          ← metadata (file listing and checksum) of the current distribution (i.e. the current repo in our            case)
|-- Release.gpg      ← gpg signature of the file above

If you want more details, the complete repository format specification can be found here: https://wiki.debian.org/DebianRepository/Format

Setup GPG Key

The first step, also a one-off step, is to create a GPG key. This is rather simple, but you might be surprised that default gpg key still uses SHA1 and some extra work is needed to avoid this.

The following section assumes gnupg is installed.

Gnupg configuration

The~/.gnupg/gpg.conf file needs some tweaking:

          mkdir             ~/.gnupg
echo "cert-digest-algo SHA256" >> ~/.gnupg/gpg.conf
echo "digest-algo SHA256" >> ~/.gnupg/gpg.conf

Prompt-less generation of a GPG key

This part explains how to generate a GPG key without any prompted question. (But please keep in mind that this is a bad idea.)

The following snippet assumes that $KEYNAME and $EMAIL variables are set.

          KEYNAME=dpkg1
EMAIL=${KEYNAME}@mydomain.ext

Generate a configuration file used to generate the key.

          cat > $KEYNAME.batch <<EOF
            %echo Generating a standard key
            Key-Type: RSA
            Key-Length: 4096
            Subkey-Length: 4096
            Name-Real: ${KEYNAME}
            Name-Email: ${EMAIL}
            Expire-Date: 0
            %pubring ${KEYNAME}.pub
            %secring ${KEYNAME}.key
            # Do a commit here, so that we can later print "done" :-)
            %commit
            %echo done
EOF

Generate the key. Two files will be created by the following line, respectively ${KEYNAME}.pub and ${KEYNAME}.key:

          gpg --batch --gen-key $KEYNAME.batch

Now you can import the key, using the following commands:

          gpg --no-default-keyring --secret-keyring ${KEYNAME}.key --keyring ${KEYNAME}.pub --list-secret-keys          gpg --import ${KEYNAME}.key

Retrieving the key id

The next step is to retrieve the ID of your key to be able to export it in the right format. The following command will allow you to retrieve the ID of the key:

          gpg ${KEYNAME}.key

The key id is the hexstring after 4096R/, i.e. EAE1F4AA in the example below:

          sec 4096R/EAE1F4AA 2018-03-13 dpkg1 <dpkg1@mydomain.ext>

Export the key and store it within the APT repo to be imported by your repo users:

          gpg --output /var/www/html/repo/KEY.gpg --armor --export $KEYID

You're all good, you have a key referred as ${KEYNAME} created, exported and usable.

For the curious, more details about GPG and GPG migration are provided by the ubuntu Security team here: https://wiki.ubuntu.com/SecurityTeam/GPGMigration

APT Local Repository

The next task in our journey is to create a local APT repository. For this we'll need to install a bunch of packages and generate the repo-specific files like Packages and Release.

Setup local repo

Let's install a bunch of packages:

          apt install apache2 dpkg-dev dpkg-sig

The apache2 package should have created /var/www/html/, and we'll create our dedicated directory where we'll store the packages and metadata:

          mkdir -p /var/www/html/repo/

This was easy. Next, create the metadata files.

Let's move to our repository and assume all the later commands are run from this working directory:

          cd /var/www/html/repo/

Packages

There are tools to generate this Packages, mostly apt-ftparchive and dpkg-scanpackages. We'll use here apt-ftparchive:

          apt-ftparchive --arch amd64 packages amd64 > Packages

A gzip'ed version of the file can also be provided to save a bit of bandwidth (please ensure -k is used, i.e. both Packages and Packages.gz are present is the repository):

          gzip -k -f Packages

Release

For building the Release file, we'll use again apt-ftparchive:

          apt-ftparchive release . > Release

Now we can sign the Release file, in two version: a stand-alone Release.gpg file and a InRelease which embbeded the signature in the file.

The first rm statement is to ensure prompt-less command if the file already exists, i.e. this can be used in a script run every time you add a new deb packages in your repo.

Generate Release.gpg file

          rm -fr Release.gpg; gpg --default-key ${KEYNAME} -abs -o Release.gpg Release

Generate InRelease file

          rm -fr InRelease; gpg --default-key ${KEYNAME} --clearsign -o InRelease Release

Done! You know have a fully authenticated APT repository, which is exposed through HTTP. You're no longer forced to use [trusted=yes] in your apt source list!

Using your APT repository from another server

Now you have a working and authenticated APT repository exposed through HTTP, you can configure apt-clients to use your repository via the following set of configuration.

We assume here that the HOST variable is set, for instance:

          HOST=1.2.3.4          echo "deb http://${HOST}/repo/ /" > /etc/apt/sources.list.d/internal.list

And importing the GPG key

          wget -q -O - http://${HOST}/repo/KEY.gpg | apt-key add -

Try it, i.e. running apt update should not throw any error:

          apt update

Bonus: Signing a deb package

With the GPG setup detailed above, signing a deb package is straightforward. Still assuming the variable ${KEYNAME} is set accordingly:

          dpkg-sig -k ${KEYNAME} --sign repo my-package.deb

Your deb file is now signed. You can verify this using

          dpkg-sig -c my-package.deb