How I Created Local Package Repositories

With Oracle Solaris 11 and Oracle Linux 6

by Suzanne Zorn, November 2011

Suzanne Zorn explains how she simplified software maintenance by setting up a local software repository for Oracle Linux 6 and Oracle Solaris 11.

Introduction

Both Oracle Linux and Oracle Solaris 11 use software repositories to manage installation and updates of the operating system software. In this article, I show how to set up local repositories for Oracle Linux 6 and Oracle Solaris 11 by downloading and configuring the repository on a server and then configuring clients to access the newly created local repository.

What did you hate most about upgrading Oracle Solaris 10 (and earlier) software? Was it manually reconciling the package dependencies? Tracking down software that's not currently installed but is needed? Manually downloading patches? The risk and uncertainty of each upgrade? No argument: Software patching and upgrades were a major headache with earlier Oracle Solaris releases. And these headaches and risks were exacerbated by the proliferation of Oracle Solaris Zones and the increased number of OS instances to maintain.

Oracle Linux avoids these patching and upgrade hassles with the RPM package management system and repositories (storage locations) for software packages. The yum updater, an automatic updater and package installer/remover for RPM systems, simplifies software maintenance by automatically computing package dependencies. This automated approach is easier and less error-prone than manually updating system software, especially on large groups of machines.

Oracle Solaris 11 uses a similar approach with its new Image Packaging System (IPS). IPS also uses repositories with automated retrieval of patches and automated dependency resolution. And because IPS is integrated with the ZFS snapshot and clone mechanisms, you can upgrade a live system and reboot into the new ZFS boot environment with minimal downtime. Bottom line: no more patching with Oracle Solaris 11, plus automated software upgrades with minimal system downtime.

Local Software Repositories

Oracle maintains public repositories of the Oracle Linux and Oracle Solaris 11 operating system software. You can create a local copy of these repositories within your corporate firewall for increased security, better update performance, and administrative convenience. Your corporate policy might dictate no Internet updates. Or, you might be looking to conserve bandwidth and time by downloading the files only once for multiple machines. Your client updates will be faster using a local software repository, since the systems can access the files locally rather than retrieving them from the Internet. These local repositories can be mirrored for availability and efficient access in large environments.

Figure 1 shows an Oracle Solaris 11 environment that uses IPS and a mirrored local network repository. Multiple client systems on the network can use this local repository to install and update their system software.

Oracle Linux also supports the creation of local repositories, using RPM and the yum updater in place of IPS. The remainder of this article shows how to set up local repositories for both Oracle Solaris 11 and Oracle Linux.

Figure 1. Local software repositories can simplify and speed up software updates.

Setting Up a Local Repository on Oracle Solaris 11

You can access the Oracle Solaris 11 software from My Oracle Support (which requires a support contract), from the Oracle Solaris distribution media, or from the public Oracle Solaris 11 repository. The Oracle public repository is a free and convenient way to install Oracle Solaris 11 across the Internet. Keep in mind, however, that the public repository is not updated with security updates and bug fixes. The best way to maintain up-to-date and secure systems is with a paid support subscription.

Oracle Solaris 11 uses ZFS boot environments and the ZFS clone feature to perform updates on live systems. (This is similar in concept to Live Upgrade in Oracle Solaris 10). When updating the system software, Oracle Solaris 11 first clones the boot environment and applies changes to the clone. By default, the clone is automatically activated on reboot. If a problem occurs, you can easily roll back to the previous boot environment image.

When you set up a local repository on Oracle Solaris 11, you have two basic choices for copying the repository:

  • You can use the repository file from media or from the Oracle Solaris 11 download site.
  • You can retrieve a repository from the Internet.

You also have two options for providing access to your local repository:

  • Clients can retrieve the packages from a local Web server.
  • Clients can retrieve the packages from a local directory.

In either case, you will need to configure each client to use the local repository that you create.

In this article, I compare two different methods for creating the local repository on Oracle Solaris 11:

  • In the first example, the repository is copied from the public Oracle Solaris download site (or from media) and served using the Service Management Facility (SMF).
  • In the second example, the repository is retrieved from the Oracle Solaris support repository and served from a local file system.

Note: Your IPS repository server must be running the Oracle Solaris 11 operating system, and it must have 15 gigabytes of free space to create a local repository. See Copying and Creating Oracle Solaris 11 Package Repositories for more details.

The biggest difference in the two scenarios I tried was the client access to the local repository: HTTP access versus file-based access. Setting up file-based access was easier. But, if you need to provide HTTP access, using SMF (rather than starting pkg.depotd directly) gives you the advantage of automatic service restart.

Serving a Local Repository Using SMF


$ su - root
# zfs create -o compression=on rpool/export/repo2010_11
# zfs list
NAME                       USED  AVAIL  REFER  MOUNTPOINT
rpool                     14.7G   168G    94K  /rpool
rpool/export/repo2010_11    31K   168G    31K  /export/repo2010_11
...


# unzip sol-11-repo-full-iso-a.zip
# unzip unzip sol-11-repo-full-iso-b.zip
# cat sol-11-repo-full.iso-a sol-11-repo-full.iso-b > sol-11-repo-full.iso
# ls /export/repo2010_11
sol-11-repo-full.iso
# zfs list
NAME                       USED  AVAIL  REFER  MOUNTPOINT
rpool                     18.7G   164G    94K  /rpool
rpool/export/repo2010_11  3.93G   164G  3.93G  /export/repo2010_11
...


# df-k /export/repo2010_11
Filesystem                1K-blocks     Used  Available  Use%  Mounted on
rpool/export/repo2010_11  176548586  9103073  167445513    6%  /export/repo2010_11


# umount /mnt
# lofiadm
Block Device  File                                                 Options
/dev/lofi/1   /export/repo2010_11/sol-11-exp-201011-repo-full.iso  -
# lofiadm -d /dev/lofi/1
# lofiadm
Block Device  File                                         Options
  1. First, create a ZFS file system to hold the repository. Using a separate ZFS file system helps you achieve better performance, separate file system characteristics (such as compression or atime), and directly snapshot and recover file systems.

    Best practice: If your system will host more than one IPS repository, make each repository a separate ZFS snapshot so that you can roll back and recover each repository separately.

    a. Use zpool list to view your current ZFS pools. Use zfs list to view your current ZFS data sets:

    
$ zpool list
NAME    SIZE  ALLOC  FREE  CAP  DEDUP  HEALTH  ALTROOT
rpool   186G  12.8G  173G   6%  1.00x  ONLINE  -
$ zfs list
NAME    USED  AVAIL  REFER  MOUNTPOINT
rpool  14.7G   186G    93K  /rpool
...

    b. As root, create a ZFS file system for the package repository in the root pool and set compression to on:

  2. Next, make a local copy of the IPS repository file. This example shows how to copy from a file that is on media or available on the public Oracle Solaris download site.

    a. Download the Oracle Solaris 11 IPS package repository .iso file from the same location where you downloaded the system installation image, or locate the repository DVD in the media packet. The repository, approximately 5 GB in size, is in two files that must be concatenated together.

    b. Copy the repository files to the file system you created in the last step, uncompress the files, and concatenate them into one:

  3. Make the contents of the repository file available to the pkg.depotd(1M) server, using the lofiadm command to mount the ISO image. (Be sure to use a fully qualified path name for the .iso file.) 
    
# lofiadm -a /export/repo2010_11/sol-11-repo-full.iso
/dev/lofi/1
# lofiadm
Block Device  File                                                Options
/dev/lofi/1   /export/repo2010_11/sol-11-exp-201011-repo-full.iso  -
# mount -F hsfs /dev/lofi/1 /mnt
  4. Copy the repository files to the ZFS file system you created. This will increase the performance of repository accesses and avoid the need to remount the .iso image each time the system restarts. (Be sure to use /mnt/repo, not /mnt/repo/, so that you copy the repo directory and not just the files and subdirectories in the repo directory.)

    # rsync -aP /mnt/repo /export/repo2010_11

    You can use the df command to confirm the copy:

  5. Once the files are copied, unmount the image and deallocate the block device:
  6. Now that your local copy is ready, use the svccfg command to configure the repository server service, specifying the location of your local repository and setting readonly to true: 
    
# svccfg -s application/pkg/server setprop pkg/inst_root=/export/repo2010_11/repo
# svccfg -s application/pkg/server setprop pkg/readonly=true

    If you prefer, you can use the following command to open a vi session where you can edit all properties at once:

    # svccfg -s pkg/server editprop

  7. Start the pkg.depotd repository service: 
    
# svcadm refresh application/pkg/server
# svcadm enable application/pkg/server

    You can check that the repository server is working by opening a browser window on the //localhost location. You should see a page very similar to the http://pkg.oracle.com/solaris/release page.

  8. Once the local repository is set up on your server, you need to configure your client systems to get packages from the local repository. The default preferred publisher for Oracle Solaris 11 systems is solaris and the default origin for that publisher is http://pkg.oracle.com/solaris/release. You need to reset the publisher origin for each client.

    Execute the following command on each client, replacing host with your server's host name:

    # pkg set-publisher -G '*' -g /net/host/export/repo2010_11/ solaris

    -G '*' removes all existing origins for the solaris publisher. And -g adds the URI of the newly created local repository as the new origin.

    Now, the clients will use the local repository for software updates. For example, you can use the Package Manager GUI or the pkg update command to update all system packages to the newest version.

File System-Based Local Repository

The previous section described how to copy the repository from media and use SMF to serve a local repository. This section shows an alternative method of retrieving the repository from its default location on the Internet and serving the packages from a directory on your local file system.

Note: This example copies the repository from the Oracle Solaris 11 support repository (registration required). To use the support repository, you will need to obtain a key and certificate and install them on your system.

  1. Create a ZFS file system (see Step 1 in the previous section).
  2. Create the required repository infrastructure using the pkgrepo command:

    # pkgrepo create /export/repo2010_11

  3. Now, copy the repository using the pkgrecv command. This example shows copying from the Oracle Solaris 11 support repository. Replace path-to-key-file and path-to-cert-file with your locations for the key and certification files.

    Note: Copying the repository can take an hour or two to complete. Wait until you see the Completed output line and the process completes some finish work and displays your system prompt before continuing.

    
# pkgrecv -s //pkg.oracle.com/solaris/support/ -d \ /export/repo2010_11 --key path-to-key-file --cert path-to-cert-file '*'

Processing packages for publisher solaris ...
Creating Plan
Retrieving and evaluating 3941 package(s)...
PROCESS                       ITEMS      GET (MB)       SEND (MB)
developer/build/cmake      446/3941  332.1/4589.7  1000.2/14511.8
...
Completed                 3941/3941 4589.7/4589.7 14511.8/14511.8
  4. Once the repository is copied onto your local system, use the pkgrepo command to build a package index. This will enable clients to search for packages in the local repository.

    # pkgrepo -s /export/repo2010_11 refresh

    Repository refresh initiated.

  5. Once the local repository is set up on your server, you need to configure your client systems to get packages from the local repository. The default preferred publisher for Oracle Solaris 11 systems is solaris and the default origin for that publisher is http://pkg.oracle.com/solaris/release. You need to reset the publisher origin for each client.

    Execute the following command on each client, replacing host with your server's host name:

    # pkg set-publisher -G '*' -g /net/host/export/repo2010_11/ solaris

    -G '*' removes all existing origins for the solaris publisher. And -g adds the URI of the newly created local repository as the new origin.

Setting Up a Local Repository on Oracle Linux

You can access the Oracle Linux software from the Oracle Unbreakable Linux Network (ULN, which requires a support contract), from the Oracle Linux distribution media, or from the Oracle public yum repository. The Oracle public yum repository is a free and convenient way to install Oracle Linux across the Internet. Keep in mind, however, that while the Oracle public yum repository is updated for every minor release (for example, from 6.1 to 6.2), it is not updated with security updates and bug fixes. The best way to maintain up-to-date and secure systems is with a ULN subscription.

This section provides procedures for setting up a local yum repository on Oracle Linux. This example shows how to download the latest packages from ULN and make them available through a local yum server. You'll need to first set up the local repository on your server, and then configure your clients to use your newly created local repository.

Server Setup

This example uses a system running Oracle Linux Release 6 for x86_64, a 64-bit version of Oracle Linux.

Table 1. Required Channels

OS ARCHITECTURE REQUIRED CHANNELS
Oracle Linux 6 i386 ol6_i386_latest, ol6_i386_addons
Oracle Linux 6 x86_64 ol6_x86_64_latest, ol6_x86_64_addons

Note: The addons channel is needed to install the uln-yum-proxy package, which in turn will install packages needed to create yum repositories.

Next, check all channels for which you wish to create yum repositories, and click subscribe. For example, Table 2 shows the channels that are also available for Oracle Linux 6.

Table 2. Available Channels for Oracle Linux 6

I386 CHANNEL X86_64 CHANNEL DESCRIPTION
ol6_i386_oracle ol6_x86_64_oracle Oracle Software for Oracle Linux 6
ol6_ga_i386_base ol6_ga_x86_64_base Oracle Linux 6 GA base packages
ol6_u1_i386_base ol6_u1_x86_64_base Oracle Linux 6 Update 1 base package
ol6_ga_i386_patch ol6_ga_x86_64_patch Oracle Linux 6 GA patches
ol6_u1_i386_patch ol6_u1_x86_64_patch Oracle Linux 6 Update 1 patches

In addition to these channels for Oracle Linux 6, there are other channels for Oracle Linux 4, Oracle Linux 5, and Oracle VM 2.1, 2.2, and 3.0.

Table 3. Space Requirements

CHANNEL BINARIES ONLY BINARIES AND SOURCE
el*/ol*_latest 3 to 10 G 5 to 15 G
el*/ol*_addons 600 M 1 G
el*/ol*_oracle 1 G N/A
el*/ol*_base 3 G 5.5 G
el*/ol*_patch 1 G 2 G 

# chkconfig httpd on
# /etc/init.d/httpd start

Note: You'll want to update your local repository regularly by running the script 167283.sh. You can set the script to run automatically once a day by creating a link to it from the /etc/cron.daily/ directory.

  1. Register the machine on ULN:

    # uln_register

    You will need to enter a user name, a password, and a valid customer support identifier (CSI).

  2. Subscribe this machine to all desired channels:

    a. Open up a Web browser and go to ULN.

    b. Log in to ULN using the user name and password that were used to register this machine.

    c. Click the Systems tab on the upper right corner.

    d. Find the registered machine on the list and click it.

    e. Click the EDIT button.

    f. Select the Yum Server checkbox, and click the Apply Changes button.

    g. Subscribe to channels. You'll need to select required channels for this server, plus any channels for which you wish to create yum repositories.

    First, make sure you register the machine to the correct latest channel and addon channel. Required channels depend on the version and architecture of the OS that is running on this server machine, as shown in Table 1.

  3. Create a yum repository base directory in /var/www/html. This directory will house the repository. Make sure that the yum repository owner has read/write permission to this directory.

    # mkdir -p /var/www/html/yum

    Approximate space requirements for these channels are shown in Table 3.

  4. Now that the repository is created, it's time to populate it.

    a. Download the script 167283.sh from the OTN article Yum Repository Setup. Edit the repository locations in this script if they are different from your system.

    b. Run the script 167283.sh. This will create yum repositories for the registered channels. If you want to download the binary RPMs only, run the script with no options:

    # sh 167283.sh

    If you want to download both the binary and source RPMs, run the script with the src option:

    # sh 167283.sh src

  5. Start the Web server:
  6. Update the rhnplugin.conf file to disable all channels that are not applicable to this local machine. For example, if this system is running Oracle Linux 6, disable all other OS channels (Oracle Enterprise Linux 4, Oracle Linux 5, and so on). This will prevent a local yum update from trying to pull packages from those channels.

    a. Use the following command to display a list of the repositories, and make note of the repo ID of those that are not needed:

    
# yum repolist all
repo id                 repo name                            status
el4_x86_64_addons       Enterprise Linux 4 Add ons (x86_64)  enabled
el4_x86_64_latest       Enterprise Linux 4 Latest (x86_64)   enabled
...

    b. Using a text editor, open the file /etc/yum/pluginconf.d/rhnplugin.conf and add stanzas such as the following to disable unwanted channels:

    
[el4_x86_64_addons]
enabled = 0

[el4_x86_64_latest]
enabled = 0

Client Setup

After the server is set up, you need to modify your clients to use the local repository that you just created. The following steps show how to configure your client systems.


[ol6_latest]
name=Oracle Linux $releasever - $basearch - latest
baseurl=//repository_server/yum/OracleLinux/OL6/latest/$basearch/
gpgcheck=1
enabled=1

[ol6_addons]
name=Oracle Linux $releasever - $basearch - addons
baseurl=//repository_server/yum/OracleLinux/OL6/addons/$basearch/
gpgcheck=1
enabled=0

[ol6_oracle]
name=Oracle Linux $releasever - $basearch - oracle
baseurl=//repository_server/yum/OracleLinux/OL6/oracle/$basearch/
gpgcheck=1
enabled=0

[ol6_ga_base]
name=Oracle Linux $releasever GA - $basearch - base
baseurl=//repository_server/yum/OracleLinux/OL6/0/base/$basearch/
gpgcheck=1
enabled=0

[ol6_u1_base]
name=Oracle Linux $releasever U1 - $basearch - base
baseurl=//repository_server/yum/OracleLinux/OL6/1/base/$basearch/
gpgcheck=1
enabled=0

[ol6_ga_patch]
name=Oracle Linux $releasever GA - $basearch - patch
baseurl=//repository_server/yum/OracleLinux/OL6/0/patch/$basearch/
gpgcheck=1
enabled=0

[ol6_u1_patch]
name=Oracle Linux $releasever U1 - $basearch - patch
baseurl=//repository_server/yum/OracleLinux/OL6/1/patch/$basearch/
gpgcheck=1
enabled=0
  1. Import the GPG key. This key is used to verify the signature of the Oracle Linux RPM packages:

    # rpm --import /usr/share/rhn/RPM-GPG-KEY

    Note: Instead of importing the GPG key, you can add the GPG key to the yum .repo file:

    gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY

  2. To create the .repo files and point them to the local server, use a text editor to replace the /etc/yum.repos.d/ULN-Base.repo file with the text shown in Listing 1. You'll need to change repository_server to be the host name or IP address of the server that is hosting the local yum repository. Note that only the latest channel is enabled by default. To enable a channel, replace enabled=0 with enabled=1.

    Note: This example assumes an Oracle Linux 6 client ;

    Listing 1: Replacement Text for .repo File

  3. 3. You're now ready to use yum to install and update files from your local repository, for example:

    # yum update

Final Thoughts

You can set up and access local repositories for Oracle Linux (using the yum updater) and Oracle Solaris 11 (using the Image Packaging System [IPS]). Creating a local repository within a corporate firewall provides security and version control. Setting up a local repository can also improve performance for client updates, since files can be accessed locally rather than across the Internet.

Resources

The following resources are available for Oracle Solaris 11 and IPS:

The following resources are available for Oracle Linux: