GRUB and the Solaris 10 1/06 OS The New Bootloader for x86 Platforms

Shudong Zhou and Jan Setje-Eiler, December 2005

This paper describes the boot architecture for the Solaris 10 1/06 Operating System on x86 platforms, as well as the user experience. Please note: Some of the details in this paper may change without notice. Refer to man pages and Solaris OS administration documents for supported interfaces.

1. Introduction to the Boot Process

When an x86-based system powers on, the BIOS initializes the CPU, memory, and platform hardware. Upon completion, the BIOS loads the initial bootstrap software (that is, the bootloader) from the configured boot device and hands control over to the bootloader. The Solaris 10 3/05 OS and earlier releases use a Sun-developed bootloader that includes an interactive shell and a menu-driven device configuration assistant based on realmode drivers.

Starting with the Solaris 10 1/06 release, the open source GRUB or GNU GRand Unified Bootloader (see reference 1) is used as the bootloader. The initial delivery is based on GRUB version 0.95 and will be updated as newer versions become available. The Solaris kernel is fully compliant with the Multiboot Specification (reference 2); hence, the Solaris OS can be booted via any bootloader implementing the Multiboot Specification.

The switch to GRUB brings several benefits to Solaris customers.

With GRUB it is very easy to specify kernel and boot options in the boot menu.
For end users, booting and installing from USB DVD drives is now supported.
It is easier for the Solaris OS to coexist with other operating systems on the same machine. In particular, it is possible for the Solaris OS to share the same GRUB bootloader with Linux.
Deploying the Solaris OS via the network is also simplified, particularly in the area of DHCP server setup. Vendor-specific options are no longer required in the DHCP server setup.
Developers no longer need to deal with realmode drivers, which were part of the bootloader required for previous Solaris releases.
For IHVs, it is now possible to deliver drivers at install time via CD/DVD in addition to floppies.

Finally, by adopting a bootloader developed by the open source community, Sun's customers can leverage the considerable GRUB experience gained within that community.

2. Booting the Solaris OS With GRUB

Once GRUB gains control, it displays a menu on the console asking the user to choose an OS instance to boot. The user may pick a menu item, modify a menu item using the built-in editor, or manually load an OS kernel in command mode. To boot the Solaris OS, GRUB must load a boot_archive file and a "multiboot" program. The boot archive is a ramdisk image containing Solaris kernel modules and data. GRUB simply puts it in memory without any interpretation. The multiboot program is an ELF executable with a Multiboot Specification-compliant header. Once loading is complete, GRUB hands control over to the multiboot program. GRUB itself then becomes inactive, and its memory is reclaimed.

The multiboot program is responsible for assembling core kernel modules in memory by reading the boot_archive, and passing boot-related information (as specified in the Multiboot Specification) to the kernel. Note that the multiboot program goes hand-in-hand with the boot_archive file. You cannot mix and match multiboot and boot_archive information from different releases or OS instances.

Once the kernel gains control, it will initialize CPU, memory, and I/O devices, and it will mount the root file system on the device as specified by the bootpath property with a file system type as specified by the property fstype. Properties can be set in /boot/solaris/bootenv.rc via the eeprom(1M) command or in the GRUB command line via the GRUB menu or shell. If the properties are not specified, the root file system defaults to UFS on /devices/ramdisk:a, which is the case when booting the install miniroot.

3. Installation

The Solaris OS may be installed from CD, DVD, and net install servers. The Solaris 10 1/06 release differs from the Solaris 10 3/05 release in several ways:

Minimum memory requirement: The system must have 256MB of main memory to boot the install miniroot. Systems with insufficient memory will get a message from GRUB: "Selected item can not fit in memory".
USB drive support: Installation from CD/DVD drives connected via USB interfaces is fully supported.
Net install: The standard procedure for setting up net install images remains the same. Clients are assumed to boot via the Preboot eXecution Environment (PXE) mechanism. Clients not capable of PXE boot can use a GRUB floppy (see Appendix B).

When booting the install miniroot, a GRUB menu is displayed. A user may interactively edit boot options (see section 4.2). After GRUB loads the Solaris OS, the following install menu is displayed:

Solaris Interactive (default)
Custom JumpStart
Solaris Interactive Text (Desktop session)
Solaris Interactive Text (Console session)
Apply driver updates
Single user shell

The Device Configuration Assistant and associated interactive shell, which users are familiar with from the Solaris 10 3/05 OS and earlier, are no longer present. Users wishing to add drivers required during install (for example, host adapter drivers) should choose option 5 and supply an ITU (Install Time Update) floppy or CD/DVD.

Option 6 is available for system recovery. It provides quick access to a root prompt without going through system identification. This option is identical to booting a Solaris Failsafe session (see section 4.4).

4. Managing the Boot Subsystem

4.1 BIOS

It is generally a good idea to update the BIOS firmware to the latest revision before installing the Solaris OS. This is typically accomplished by visiting the support page for the vendor that manufactured the computer.

Compared to the Solaris 10 3/05 release, the Solaris 10 1/06 OS uses a different subset of BIOS features. In particular, the kernel makes use of more information from the Advanced Configuration and Power Management Interface (ACPI) table, using the parser from Intel's ACPI CA software.

On systems that do not conform to BIOS 2.0 specifications, the syslog may contain messages related to parsing the ACPI table, such as:

ACPI-0725: *** Warning:
Type override - [4s] had invalid type (DEB_[\200IODB

Such messages are harmless and do not impact normal system operation. If ACPI errors prevent normal system boot, the user can disable the ACPI parser by setting acpi-user-options to 2 (see eeprom(1M)) in the kernel line of the GRUB menu:

kernel .. -B ...,acpi-user-options=2

In this case, the system assumes a set of standard ISA devices is present, including a keyboard, a mouse, two serial ports, and a parallel port.

4.2 Boot Options

To boot the Solaris OS, a user may specify the kernel to load, options to be passed to the kernel (see kernel(1M)), and a list of property names and values to customize system behaviors (see eeprom(1M)). At Solaris installation time, a set of default values are chosen for the system and stored in /boot/solaris/bootenv.rc. Users may change the settings by editing the GRUB menu or modifying the bootenv.rc file indirectly via the eeprom(1M) command.

Specifying kernel name and kernel options via eeprom requires setting the boot-file property. To boot the 32-bit kernel in verbose mode, run the following command:

# eeprom boot-file="kernel/unix -v"

To specify the same thing on the GRUB menu, modify the kernel command of the GRUB menu from:

kernel /platform/i86pc/multiboot

to:

kernel /platform/i86pc/multiboot kernel/unix -v

See kernel(1M) for additional boot arguments that the Solaris kernel accepts.

Properties other than boot-file can be specified on the GRUB kernel command line with this syntax:

kernel /platform/i86pc/multiboot -B prop1=val1[,prop2=val2...]

To configure the serial console on ttya (com1), set the console property to ttya:

kernel /platform/i86pc/multiboot -B console=ttya

If the property value contains commas, the value should be quoted. The following GRUB command sets the Solaris console to ttya in high speed.

kernel /platform/i86pc/multiboot -B console=ttya,ttya-mode="115200,8,n,1,-"

In short, specifying "-B foo=bar" in the GRUB menu is equivalent to running "eeprom foo=bar". The -B option in GRUB is primarily for temporary overrides. Permanent settings should be specified via eeprom(1M) so that they are preserved by the Solaris upgrade process.

4.3 Boot Archive

The boot archive refers to the file platform/i86pc/boot_archive. It is a collection of core kernel modules and configuration files packed in either UFS or ISOFS format. At boot time, GRUB loads the boot archive into system memory. The kernel can now initialize itself from data and text in the boot archive without performing I/O to the root device. Once the kernel gains sufficient I/O capability, it will mount the root file system on the real root device as specified by the bootpath property. At this point, the boot archive loaded by GRUB is discarded from memory.

The content of the boot archive is specified in /boot/solaris/filelist.ramdisk. Upon system shutdown, the system checks for updates to the root file system and updates the boot archive when necessary. The system may manually update the boot archive prior to system shutdown by running the bootadm(1M) command.

4.4 The Failsafe Menu Entry

New to the Solaris 10 1/06 OS is a file, /boot/x86.miniroot-safe, containing a bootable, standalone Solaris image. This file can be loaded by choosing the Solaris failsafe entry from the GRUB menu. This is for the convenience of system administrators when the normal entry fails to boot.

Suppose you add a new package containing a faulty driver, and the system panics at boot time. Upon reboot, you can pick the Solaris failsafe menu entry. While in the failsafe session, mount the root file system on /a and run pkgrm -R to remove the faulty package. Once this is complete, you can reboot to the normal Solaris entry to resume system operation.

The file /boot/x86.miniroot-safe can also be copied to portable media, such as a USB stick, as a recovery tool.

4.5 Keeping the System Bootable

To ensure that the system remains bootable, the GRUB boot blocks, the GRUB menu, and the boot archive must be up-to-date.

The GRUB boot blocks reside in the Solaris partition. If the boot blocks become corrupt, they should be reinstalled using the installgrub(1M) command. Note that installboot(1M) and fmthard(1M) cannot be used to write GRUB boot blocks.

The GRUB menu resides in /boot/grub/menu.lst (or /stubboot/boot/grub/menu.lst if a Solaris boot partition is used). The menu is maintained with the bootadm(1M) update-menu subcommand. Because GRUB names disks by the BIOS disk number, a change in BIOS boot device configuration may render GRUB menu entries invalid in some cases. Running bootadm update-menu will create the correct menu entry in such cases.

The boot archive must be updated as the root file system is modified. In cases of system failure (power failure or kernel panic) immediately following a kernel file update, the boot archive may be out of sync with the root file system. In such cases, the system/boot-archive service, managed through the Solaris Service Manager (see svcadm(1M) for example), will fail on the next reboot. Solaris will print a message telling the user that it is still possible to boot and clear the event that triggered the error, but it is safer to reboot the system, select failsafe session, and update the boot archive while in the failsafe session.

5. Multiboot Setup

If other bootable partitions exist, the Solaris install program attempts to add GRUB entries for them. At the time of this writing, Microsoft Windows and diagnostic partitions are recognized. Users may add other operating systems by editing the GRUB menu directly.

Suppose you have a system with the Solaris OS, Linux, and Windows laid out on the same disk as follows:

fdisk partition 0:      Windows
        fdisk partition 1:      Linux
        fdisk partition 2:
                slice 0         Solaris 9
                slice 3         Solaris 10 1/06

The GRUB menu should look like:

#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
title Solaris 10 1/06
        root (hd0,2,d)
        kernel /platform/i86pc/multiboot
        module /platform/i86pc/boot_archive
#---------------------END BOOTADM--------------------
title Solaris 9
        root (hd0,2,a)
        chainloader +1
        makeactive
title Linux
        root (hd0,1)
        kernel <from Linux's GRUB menu...>
        initrd <from Linux's GRUB menu...>
title Windows
        root (hd0,0)
        chainloader +1

Note that the Solaris fdisk partition must be the active partition. Do not put "makeactive" under the Windows menu. Doing so will make the system boot Windows all the time.

GRUB as obtained from sources other than Sun does not currently recognize Solaris on-disk VTOC and UFS formats. Sun has submitted changes to the GRUB project to support this; until they have been integrated, only the Solaris GRUB will work. If Linux installed GRUB on the master boot block, you will not be able to get to the Solaris OS even if you make the Solaris partition the active partition. In this case, you can chainload from the Linux GRUB by modifying the menu on Linux. Alternatively, you can replace the master boot sector with the Solaris GRUB in the above example, by using the installgrub(1M) command:

installgrub -m /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c0t2d0s3

Before the Solaris VTOC and UFS implementation is propagated to the standard GRUB release, only the Solaris version of GRUB will work.

6. Netboot and Diskless Clients

Booting from a network typically requires a DHCP server configured for PXE clients and a boot server providing TFTP service. If there is no PXE/DHCP server available (or the BIOS does not contain PXE code), it is possible to load GRUB from floppy, CD-ROM, or a local disk, configure the network manually in the GRUB menu, and boot the Solaris OS from a file server. See Appendix B for additional details.

Setting up the DHCP server is a site-specific task and is left to the local network administrator.

To boot the Solaris OS, the DHCP server must respond to the BIOS PXE request with the IP address of the file server and the name of the boot file (pxegrub). The boot server is responsible for serving the pxegrub binary, the GRUB menu, the multiboot program, and a boot archive. Standard tools are provided for setting up the boot server.

6.1 Net Install Server

The add_install_client(1M) command is used to set up the net install for install clients. For Solaris 10 1/06 and later releases, add_install_client(1M) should always be invoked with the -d option for PXE/DHCP boot, even if a GRUB floppy is used to bootstrap to the network.

After running add_install_client, you should see /tftpboot/01<eth_addr> as a link to pxegrub and /tftpboot/menu.lst.01<eth_addr> containing a GRUB menu entry. The <eth_addr> is the Ethernet address of the client network interface in uppercase. In addition, the /boot directory of the net image is loopback mounted under /tftpboot to make multiboot and x86.miniroot accessible via tftp.

6.2. Diskless Clients

The smdiskless(1M) command is used to set up diskless clients. In earlier releases, smdiskless(1M) sets up the root and usr file systems and exports to clients over NFS. However, users must manually configure the /tftpboot area in order to boot clients. This manual step has been removed in the Solaris 10 1/06 OS. The smdiskless(1M) command automatically invokes a script in /usr/sadm/lib/wbem/config_tftp to set up the /tftpboot area for PXE boot.

After running smdiskless, you should see /tftpboot/01<eth_addr> as a link to pxegrub and /tftpboot/menu.lst.01<eth_addr> containing a GRUB menu entry. The <eth_addr> is the Ethernet address of the client network interface in uppercase. In addition, /export/root/<hostname>/boot is loopback mounted under /tftpboot to make the client's multiboot and boot archive accessible via tftp.

The boot archive of diskless clients is automatically updated upon shutdown of the client. If the client's boot archive is out of date when the client is down, the archive can be updated from the server side with the following:

# bootadm update-archive -R /export/root/<hostname>

This works on both SPARC and x86 platform-based servers, serving x86 clients.

7. Legacy Issues

7.1 The Solaris x86 Boot Partition

For x86 systems, the Solaris boot partition is a very small (about 10MB) primary fdisk partition in PCFS (FAT) format. It was introduced in the Solaris 2.6 release, primarily because BIOS firmware at that time could not access disk blocks beyond 1024 cylinders. It was necessary to put boot code in a small area near the beginning of the disk. The boot partition allowed the Solaris OS to be installed on a non-boot disk with minimal space requirements on the boot disk.

With modern systems, there is no longer a need for a boot partition. From the Solaris 10 1/06 OS onward, users no longer have the option to create a boot partition. When upgrading from an old release, an existing x86 boot partition may be retained to keep the system bootable without manual intervention. In such cases, the x86 boot partition is mounted at /stubboot and contains GRUB boot blocks and a menu file /stubboot/boot/grub/menu.lst. The user can eliminate the x86 boot partition by deleting the /stubboot entry from /etc/vfstab, fixing up /boot/grub/menu.lst, and reconfiguring BIOS to boot from the Solaris disk partition.

7.2 Realmode Drivers

Prior to the Solaris 10 1/06 OS, realmode drivers were run as part of the boot process. It is possible that some existing kernel drivers implicitly depend on a corresponding realmode driver to place hardware in a specific state. Since realmode drivers are no longer part of the GRUB-based boot, such a kernel driver may not function properly. If you run into such a case, please report the problem to the driver vendor so that the kernel driver will be fixed.

Appendix A. GRUB Basics

This appendix describes the basic concepts of GRUB. Some of the details in this appendix may change without notice. Refer to GRUB documentation for more detailed information.

A.1 Device Naming

The primary purpose of a bootloader is to load data from a storage device to system memory. To work with GRUB, it is necessary to understand how GRUB names devices.

The floppy disk is named as:

(fd0) -- first, second floppy

GRUB can only reference a single network interface:

(nd) -- network interface

And this is almost always the interface the BIOS probed and configured via DHCP. It is also possible to configure a network interface by booting GRUB from floppy or other local media (see Appendix B).

Hard disk names start with hd and a number, where 0 maps to BIOS disk 0x80 (first disk enumerated by the BIOS), 1 maps to 0x81, and so on.

(hd0) -- first bios disk (also the BIOS boot disk 0x81)
(hd1) -- second bios disk (BIOS disk 0x81)
(hd0,1) -- first bios disk, second primary partition
(hd0,0,a) -- first bios disk, first fdisk partition, Solaris/BSD slice 0 and 1

Be aware that BIOS disk numbering may change when BIOS configuration is modified. Hence, the GRUB menu may become invalid if you change the BIOS boot disk order or modify the disk configuration. To handle boot issues related to disk renumbering in BIOS, it's essential to know the disk naming conventions.

A.2 Functional Components

The functional GRUB components include stage1, stage2, and menu.lst. stage1 is installed on the first sector of the Solaris fdisk partition and can be optionally installed on the master boot sector. stage2 is installed in a reserved area in the Solaris partition. The menu.lst is located in /boot/grub and read by GRUB stage2.

Note that you cannot use dd(1M) to write stage1 and stage2 to disk because stage1 must be told where stage2 is located on the disk. The installgrub(1M) command is the supported way to install GRUB boot blocks.

The GRUB shell, although present in /boot/grub/bin/grub, is not yet supported, and some of the commands will not function on the Solaris OS due to operating system limitations in making BIOS calls.

A.3 The GRUB Menu

The default menu file is /boot/grub/menu.lst. Manual edits to menu.lst take effect on the next reboot. A typical GRUB menu looks like:

default 0
timeout 10
serial --unit=0 --speed=9600
terminal serial
#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
title Solaris
  root (hd0,0,a)
  kernel /platform/i86pc/multiboot -B console=ttya
  module /platform/i86pc/boot_archive
#---------------------END BOOTADM--------------------
#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
title Solaris failsafe
  root (hd0,0,a)
  kernel /boot/multiboot -B console=ttya -s
  module /boot/x86.miniroot.safe
#---------------------END BOOTADM--------------------

The "default" line specifies which item to boot if the timer runs out. The "timeout" line specifies the number of seconds to wait for user input before booting the default entry. If timeout is -1, user input is required. The serial and terminal commands together switch GRUB output to the serial port specified in the serial --unit tip line. The root command specifies from which disk, partition, and slice to load files. GRUB automatically detects the file system type. The kernel command must be followed by the multiboot program. The string after multiboot is passed to the Solaris OS without interpretation.

To explicitly specify a 64-bit kernel, modify the GRUB kernel command to:

kernel /platform/i86pc/multiboot kernel/amd64/unix

Likewise, the following command boots the 32-bit kernel:

kernel /platform/i86pc/multiboot kernel/unix

A.4 GRUB Command Mode

You can boot the OS by typing commands at the GRUB prompt. To get the GRUB prompt, hit 'c' at the GRUB menu. Type 'help' for a list of available commands. Hitting Esc returns to the menu. You can also hit 'e' to edit an existing menu item. When done with editing, hit 'b' to boot the system with the modified GRUB entry.

Appendix B. Network Deployment

This appendix describes the best practices of deploying the Solaris OS on x86 platforms via the network. Some of the details in this appendix may change without notice. Refer to man pages and Solaris administration documents for supported interfaces.

B.1 Overview of Solaris Network Deployment

The idea behind deploying the Solaris OS over the network is to place Solaris OS images in a centralized location. Any machine connected to the network may boot and install the release of choice with little or no involvement of the administrator. The installation process can even be done in a hands-off manner via Solaris JumpStart software.

Booting from the network typically requires a DHCP server configured for PXE clients and a file server providing tftp service. The DHCP server is responsible for supplying the information necessary for the client to configure its network interface. The DHCP response also contains the IP address of the file server and the boot file name (pxegrub). The file server is responsible for serving the pxegrub binary, the GRUB menu, the multiboot program, and a boot archive. By default, the GRUB menu file is menu.lst.01<eth_addr>. If this file does not exist, pxegrub falls back to DHCP option 150 (if specified) or /tftpboot/boot/grub/menu.lst.

Booting the Solaris OS using PXE works as follows:

User configures BIOS to boot from a network interface.
BIOS sends a DHCP request.
DHCP server replies with server address and the name of the boot file.
BIOS downloads pxegrub from server via tftp and executes pxegrub.
System downloads a GRUB menu file via tftp and shows menu entries.
User selects menu entry and starts loading the Solaris OS.

In a typical deployment scenario, pxegrub downloads the multiboot program and the x86.miniroot file, which is decompressed into an in-memory miniroot. Once the miniroot is running, it mounts the Solaris image (as specified in the install_media property on the GRUB menu) on /cdrom, and the installation process starts.

In the following sections, we will elaborate on the DHCP server setup, client requirements, and several different deployment scenarios.

B.2 The DHCP Service

When a client sends a DHCP request, the server is informed of the client's ID (typically Ethernet address), the "class" of the request, and the subnet the client resides in. The Solaris DHCP server forms the response-based macros matching the client request:

A class macro
A network macro
An IP macro
A client macro

The class macro is named by a "class string" embedded in the DHCP request. On x86 platforms, BIOS always makes a DHCP request with the class PXEClient:Arch:00000:UNDI:002001. If a macro by this name is defined in the DHCP server configuration, then macro content is sent to x86 clients in step (c) of section B.1.

The network macro is named by the IP address of the subnet. If a macro 129.146.87.0 is defined on the DHCP server, the macro content is sent to all clients on that subnet, regardless of the class of the request. If an option is defined in both the class macro and the network macro, the network macro takes precedence.

The IP macro is named by an IP address. It is rarely used in the context of booting the Solaris OS on x86 platforms.

The client macro is named by the client's type (01 for ether) and MAC address in uppercase letters. For a client with Ethernet address 0:0:39:fc:f2:ef, the corresponding macro name is 01000039FCF2EF.

Suppose we have a client in subnet 129.146.87.0 with Ethernet address 0:0:39:fc:f2:ef making a DHCP request of class PXEClient.

The DHCP server has matching macros:

- PXEClient
BootSrvA:  129.146.87.194
BootFile:  pxegrub
- 129.146.87.0
Router:    129.146.87.1
NISdmain:  sunsoft.eng.sun.com
- 01000039FCF2EF
BootFile:  01000039FCF2EF

The actual DHCP response will be as follows:

BootSrvA:  129.146.87.194
BootFile:  01000039FCF2EF
Router:    129.146.87.1
NISdmain:  sunsoft.eng.sun.com

Note that the BootFile in the client macro overrides the BootFile in the class macro.

B.3. The Simplest Netboot Setup

For the bare minimum, you need a server and a client. The server should have a running DHCP server and a Solaris distribution image available. The client machine is connected to the same subnet as the server capable of supporting a PXE boot request. Here are steps for server setup, assuming the server's IP address is 192.168.0.123 and the client's Ethernet address is 0:0:39:fc:f2:ef.

a. Go to the Tools directory of the Solaris distribution image and run:

# add_install_client -d -e 0:0:39:fc:f2:ef i86pc

b. Create a client macro name 01000039FCF2EF for DHCP service with:

BootSrvA:  192.168.0.123
BootFile:  01000039FCF2EF

You can do this either from the dhcpmgr GUI or from command-line interfaces.

Now you can boot the client and install the Solaris OS.

B.4. Net Install Without PXE/DHCP

It is possible to load GRUB from floppy, CD-ROM, or a local disk, configure the network manually in GRUB, and download multiboot and boot_archive from the file server. You may want to do this for a number of reasons:

The client's BIOS does not support PXE.
The network card does not have UNDI ROM.
There is no DHCP service on the network.
You can automate install/reboot sequences without reconfiguring boot device order in the BIOS.

The standard GRUB stage2 installed with the Solaris OS contains netboot support. You may use the functionality by hitting 'c' in the GRUB menu to go into command mode. Here is a working example:

grub> ifconfig --address=192.168.0.02 --gateway=192.168.0.1 
   --mask=255.255.255.0 --server=192.168.123.110
grub> root (nd)
grub> kernel /multiboot.I86PC.Solaris_10-1 kernel/unix -B
   install_media=192.168.123.110:/export/setje/boot_74L2
grub> module /x86.miniroot
grub> boot

Of course, the ifconfig command must be specified with values appropriate for your local network environment. You can also add a menu entry with the above commands to the menu.lst file on disk or floppy.

Make sure you edit /boot/grub/menu.lst on the floppy before using it. If your network card contains UNDI ROM, simply pick the UNDI-based floppy image. Otherwise, you should pick the floppy image with the matching Solaris driver name (bge, e1000g, and so on). If you have a special GRUB stage2 and wish to build your own floppy, here are the instructions:

<format floppy>
# mount -F pcfs /dev/diskette /mnt (you may have to disable vold prior to doing this)
# cd /boot/grub
# /sbin/installgrub stage1 stage2 /dev/rdiskette
stage1 written to boot sector on floppy
first 2 sectors of stage2 written on floppy

Edit /mnt/boot/grub/menu.lst to contain the following entry:

title Solaris
  ifconfig --address=192.168.0.2 --gateway=192.168.0.1 
     --mask=255.255.255.0 --server=192.168.123.110
  root (nd)
  kernel /multiboot.I86PC.Solaris_11-1 kernel/unix -B
     install_media=192.168.123.110:/export/setje/boot_74L2
  module /x86.miniroot

B.5. Managing Multiple Solaris Images

In the presence of multiple Solaris images, you need a way to pick which release or build to install for each client. Here are some possible ways to do this.

Keep DHCP server unchanged, and run add_install_client on the boot server.
Keep the boot server unchanged, and modify the DHCP client macro (via web interface) to pick the release/build.
Keep both boot server and DHCP server unchanged, and the client picks the release/build from GRUB menu.

Case (a) is the standard way of managing the boot server. The network admin does a one-time setup of DHCP service for each client. After that, the client user logs in the boot server and runs add_install_client. If you have a working DHCP setup for Solaris 10 FCS clients, the same setup works with pxegrub based boot.

Case (b) requires the use of DHCP option 150 to supply the GRUB menu file. When the user selects a release/build for a client, the DHCP configuration is modified to set option 150 to the menu file containing the selected release/build. Note that the menu file for each release/build must be created a priori on the boot server. If the selected release/build does not use pxegrub to boot, the old Solaris setting is configured.

Case (c) works only for releases or builds based on pxegrub. As a new image is added on the server side, a menu entry is appended. When a client boots, the new menu entry is available for selection. Unfortunately, pxegrub cannot "chainload" the old Solaris netboot program (called nbp). To make client selection work across nbp and pxegrub, you may configure pxelinux on the server, which can load either nbp or pxegrub. A detailed discussion of pxelinux is beyond the scope of this paper.

B.6. Automated Deployment Sequence

Automated deployment often performs the following sequence:

Boot net and install the Solaris OS on a disk via Solaris JumpStart software.
Reboot to disk and run a preconfigured set of tasks.

At the end of the deployment, the machine sits idle, waiting for user action.

In most cases, the initial netboot can be initiated manually by pressing F12 (or some other key sequence, depending on the platform). However, some systems do not support BIOS serial console redirection, and requiring the user to be at the physical console is unreasonable. Also, some BIOSes provides a very short time window for requesting netboot, making it a poor user experience. One workaround is to boot from floppy to net, then rewrite the floppy to boot to disk. This method is no longer viable as newer platforms no longer ship with floppy drives.

We recommend loading GRUB from disk. Add the following menu entry in /boot/grub/menu.lst to boot to the network for JumpStart and make it the default boot entry through this command:

# bootadm set-menu default=<menu_entry_number>

where <menu_entry_number> is the menu item (numbering from 0) of the option you add below:

title Solaris Jumpstart
        dhcp
        root (nd)
        kernel .../multiboot kernel/unix - install -B console=...
        module ../x86.miniroot

Once the system is installed, the default boot =. entry is modified automatically to boot from disk. This way, the whole deployment sequence can be initiated via a power-cycle.

B.7. Trouble-Shooting

Now, what happens if this doesn't work? Here are some tips for trouble-shooting.

a. Client boots straight to disk without attempting netboot.

This usually means the client PXE ROM is disabled or that the BIOS is configured to boot from the disk before the network. The fix is to go into BIOS and modify the configuration. Details vary from one machine to another.

b. Client attempts to boot net, but there is no response.

Perform the usual checking of network cables, LED lights (if any) on the interface card, and so on. Run snoop on the server:

# snoop -v [-d <interface>] 0:0:39:fc:f2:ef

where <interface> is something like bge0. The -d option is needed only if the server has more than one network interface card. Here is what you should see:

DHCPDISCOVER (client -> server)
DHCPOFFER (server -> client)
DHCPREQUEST (client -> server)
DHCPACK (server -> client)

If there is no DHCPDISCOVER message, then the client is not transmitting, usually due to a bad cable or interface card.

If the DHCPDISCOVER packet is seen, but no DHCPOFFER message, it means the server is not responding. Look at the DHCPDISCOVER message to verify that the Ethernet address matches the macro name in the DHCP server. If the server uses class macros, make sure the class string matches. The class string in the BIOS DHCP request is always the following:

PXEClient:Arch:00000:UNDI:002001

c. Client displays TFTP timeout message.

This can happen when tftp service is not set up properly. Check the server to verify that tftp is enabled (in.tftpd is running) and that the file /tftpboot/01000039FCF2EF exists and is linked to a pxegrub binary. The add_install_client step is supposed to set up both the tftp file and the tftp service.

d. Client hangs in the middle of downloading x86.miniroot.

The TFTP protocol can transfer up to 65K blocks. Some tftp servers (for example, those running the Solaris 8 OS) only support small block sizes (512), resulting in a maximum file size of 32MB, less than the size of x86.miniroot (about 45MB). Upgrading the tftp server will solve the problem.

If the server is running the Solaris 8 OS, you may copy /usr/sbin/in.tftpd from the Solaris 9 OS, until the TFTP patch for the Solaris 8 OS becomes available.

e. Client finishes downloading x86.miniroot and then hangs.

Most likely, the console output went somewhere else. Make sure the console variable is set correctly on the GRUB menu line. Setting input-device/output-device also works. See section 4.2.

f. Client displays "No ethernet card found" message.

This may happen when booting GRUB from local media and loading the Solaris OS from the network. It is an indication that the GRUB cannot find any network interface using the PCI BIOS32 service.

g. Client displays "No PnPBios found" message.

This may happen when booting GRUB from local media and loading the Solaris OS from the network. In this case, GRUB located the network interface, but failed to locate the PnP Bios service to install UNDI ROM from the network interface card.

References

GNU GRUB web page (http://www.gnu.org/software/grub/grub.html)
Multiboot Specification web page on the GNU site (http://www.gnu.org/software/grub/manual/multiboot/multiboot.html)