Tips for Securing an Oracle Linux Environment

by Ginny Henningsen with James Morris and Lenz Grimmer
Published September 2012

An overview of the software, network, and system monitoring recommendations for maintaining a secure Oracle Linux environment, plus an overview of the tools to help you do it.

Part 1- Tips for Hardening an Oracle Linux Server
Part 2- Tips for Securing an Oracle Linux Environment

Introduction

Oracle Linux is designed and configured to be "secure by default." However, today's IT infrastructures reside on widely distributed networks that are commonly linked to the internet, providing intruders with readily available entry points to corporate systems and data. This article explores several administrative tools and practices that can reduce vulnerabilities and minimize risk for your Oracle Linux environment.

This is the second article in a series. An earlier article, "Tips for Hardening an Oracle Linux Server," describes general steps administrators can take to fortify Oracle Linux configurations during initial installation. The first article suggests steps for reducing the attack surface, such as removing unused software packages and minimizing network services. After Oracle Linux is configured at startup, the goal is then to preserve the integrity of the system over the rest of its lifespan.

In this second article, we'll review procedures and frequently used tools that administrators can use to safeguard Oracle Linux systems, monitor potential threats, and detect compromise in the event it does occur.

Beyond system hardening, there are several key operational steps administrators can take to protect and strengthen Oracle Linux systems:

Maintain physical system security
Monitor system performance, logs, and audit trails on a routine basis
Use security and management tools to scan for signs of compromise
Apply software and security updates promptly
Review procedures and policies at regular intervals

This article discusses each of these strategies and highlights related tools available for Oracle Linux systems. The strategies discussed here are options to consider rather than definitive rules to apply—the procedures you implement, of course, should adhere to your corporate security and operational policies.

Physical Security

While it might seem somewhat obvious, physical system security is an important prerequisite. Keeping systems in a locked data center and limiting access to authorized and trained personnel is an important first step. (And let's not minimize the value of proper administrative training—human error is often the root cause of a security breach. Be sure to take advantage of the available Oracle Linux coursework and certification options at http://education.oracle.com.)

Beyond the risk of theft and data compromise, physical security is critical because it prevents an unauthorized user from possibly modifying the system BIOS, altering the boot device, and booting from an alternate medium. For this reason, if you have a system that is not in a locked data center, you might want to password-protect the BIOS or boot loader. (Consult the system manufacturer's documentation to learn how to set a BIOS password. Instructions for setting a GRUB boot loader password are available in the security guide.) As a general rule, however, setting a BIOS or boot loader password can make rebooting difficult since you must manually enter the password at every reboot.

As described in the first article, administrators should first log in to a system using a regular user account to perform privileged administrative tasks so that their actions can be properly attributed to them in logs and audit files. Of course, all system and user accounts should be protected by strong passwords, and administrators should enforce stringent password policies. Pluggable Authentication Modules (PAM) in Oracle Linux make it easier to enforce strong user authentication and password policies, including rules enforcing password complexity, length, age, and expiration. Using multifactor identification techniques (such as card keys, biometric identification, and so on) in addition to strong passwords can add an additional level of identification control if needed.

Software, Network, and System Monitoring

Regular systematic reviews of software, the network, system logs, and performance metrics can help to detect intrusion attempts and reveal whether a system has been compromised. Of course, collecting data is useful only if you routinely review it. System logs and performance metrics can sometimes expose anomalies that require further investigation and can ultimately reveal an intrusion attempt or a compromised system.

System Logging

Logging can capture important events from a variety of system programs, including the kernel, system services, and running applications. Best practice for logging involves capturing security-relevant events, maintaining the integrity of log files, and (perhaps most importantly) reviewing the collected log data.

Oracle Linux 6 uses the multithreaded system log daemon rsyslog, which replaces the sysklogd daemon in Oracle Linux 5 and earlier releases. Although both logging services are similar, rsyslog provides enhanced filtering and relies on TCP rather than the UDP protocol. Administrators can fine-tune logging parameters in the respective configuration files, either /etc/rsyslog.conf or /etc/syslog.conf.

An intruder who has gained access to a system will sometimes modify the local log files to escape detection and cover up unauthorized access. For this reason, it is good practice to direct log files to a dedicated remote system for collection and analysis. This is easily done in the logging configuration file—simply add @hostname to forward events to a centralized log server.

Wherever log files reside, check that there is adequate space to support ongoing event collection. Logs are rotated by cron and logrotate, but you might need to adjust the rotation frequency to meet space and site requirements.

If you do not use a commercially available log-monitoring tool, Oracle Linux includes logwatch, a highly customizable log analysis system written in Perl that parses system logs according to customizable rules and creates detailed reports based on user-specified criteria. You can define keywords or patterns that, when matched, will trigger a specific action. Using logwatch simplifies the task of reviewing logs, saving valuable administrative time. For more information, see the man pages for logwatch(8).

Process Accounting

Process accounting allows you to log user activity, giving you visibility into what commands were executed at what times. The package psacct contains the psacct service as well as several GNU utilities for monitoring process activities:

ac: Displays statistics about how long users have been logged on
lastcomm: Displays information about previously executed commands
accton: Turns process accounting on or off
sa: Summarizes information about previously executed commands

As with any logging activity, be sure to configure the file system destination with adequate space when accounting is used. The default process accounting file is /var/account/pact.

System Management Tools

System management tools are useful for monitoring system performance and troubleshooting problems. They are particularly valuable in identifying ill-behaved processes that intentionally or inadvertently hijack system resources, eating up available memory or CPU cycles and causing in a Denial of Service (DoS) attack. There are many options for tracking resource utilization and monitoring performance: command line tools, the GNOME System Monitor GUI, and enterprise monitoring tools.

Command line tools include the traditional UNIX commands ps, top, free, vmstat, and iostat. When you are responsible for watching system performance metrics on multiple servers across an enterprise, server lifecycle management tools are more effective and easier to use. All Oracle Linux support subscriptions include free access to Oracle Enterprise Manager 12c and Oracle Enterprise Manager Ops Center 12c, allowing you to manage thousands of servers from a single browser-based interface. In addition to performance monitoring, these server lifecycle management tools support bare-metal provisioning and configuration management, and they can automate the data collection, analysis, and auditing procedures needed for effectively managing systems across an enterprise data center.

Software Integrity Checking

As a part of routine system and security administration, it's a good idea to check the integrity of installed system files. For a quick first assessment, the RPM package manager can verify the integrity of installed system files. The commands rpm -V <package> or rpm -Vf <filename> verify files in an installed package, making comparisons to package metadata in the RPM database. The verify operation compares file size, MD5 checksum, permissions, type, owner, and group and displays any discrepancies.

Installing, enabling, and periodically running a file integrity checker can also reveal whether a system has been compromised. For Oracle Linux, there are several available open source (as well as commercial) file integrity checking tools, including AIDE (Advanced Intrusion Detection Environment) and Tripwire.

AIDE and Tripwire are intrusion detection systems that, when initially configured, scan file systems and record cryptographic hashes of each file in a database. It is recommended that you then move the database to a read-only medium to avoid tampering. On subsequent file system checks, the tool compares the stored checksums to current files, alerting you if there have been any changes. (This is similar functionality to that of BART, the Basic Audit Reporting Tool for the Oracle Solaris operating environment.) For more information, see the AIDE or Tripwire Websites.

Network Intrusion Prevention and Detection

Since networks are a primary entry point into IT systems, network intrusion prevention and detection tools can help avert or uncover a security breach. The previous article describes important networking considerations, such as minimizing unused network services (since these can provide unauthorized points of entry) and configuring TCP wrappers and IPtables.

In addition to taking these precautions, there are open source and commercial tools for performing packet logging and analysis. For example, tcpdump and Snort are network sniffer, logging, and analysis tools that operate on a dedicated host or network segment. They are designed to capture TCP traffic and analyze it for suspicious usage patterns, such as those that typically occur with port scans or network DoS attacks. There are also comprehensive open source and commercial network intrusion and detection systems designed to help simplify log analysis and reporting. The open source-based Sguil, for example, incorporates tcpdump and Snort but adds in functionality from the Wireshark protocol analyzer, an underlying MySQL database, and other tools.

Auditing and Audit Monitoring

Like logging, auditing collects data that must be analyzed and reviewed to identify unauthorized activity. However, auditing differs from system logging in that it occurs at the kernel level. As a result, auditing typically collects more detail than logging, but audited events can often be uninteresting and insignificant—sifting through audit trails to find events of interest can often prove challenging.

By default, auditing captures specific events such as system logins, account modifications, and the actions of sudo. Auditing can also be configured to capture detailed system call activity or modifications to certain files. The configuration file (/etc/audit/auditd.conf) defines the data retention policy, the audit volume's maximum capacity (and the action to take if that capacity is exceeded), and local or remote audit trail locations. The kernel audit system daemon (auditd) records configured events, including the event type, a time stamp, the associated user ID, and success or failure of the system call.

The audit rules file (/etc/audit.rules) determines what events are audited and it is typically configured to match security policy. Adding -e 2 as the last rule in the file makes the audit configuration unchangeable without a reboot. Examples of audit rules are provided in /usr/share/doc/<audit-version>/stig.rules.

Some governmental agencies have stringent auditing requirements that generate large audit volumes—in some cases, security policy even stipulates system shutdown when events cannot be recorded because audit volumes have exceeded capacity. As a general rule, audit trails should be directed to separate rotating file systems to enable spillover and facilitate backups.

From the command line, you can use the auditctl command to add or delete rules. For example, this command audits permission changes to the /etc/shadow file:

# auditctl -w /etc/shadow -p war -k shadow-file

The command inserts a watch (-w) using the permissions filter specified by -p war (w for write, a for append, r for read). The -k argument sets a filter key that uniquely identifies the audit records produced by the watch, making it easier to search the resulting audit logs.

The tools for reviewing collected audit data are ausearch and aureport. To examine records with the shadow-file key (set in the previous example), you would enter the following:

# ausearch -k shadow-file

The aureport command generates summaries of audit record data. If you set up auditing on Oracle Linux systems, it's a good idea to set up cron jobs that run aureport periodically to generate reports of interest. For example, you could run the following command in a daily cron job to generate a report that shows every login:

# aureport -l -i -ts yesterday -te today

Update and Patch Management

In addition to closely watching system activity through system performance monitoring, logging, and auditing, effective security practice relies on keeping system software up to date. For this reason, it's essential to apply system security updates as soon as they are published. It's strongly recommended that you register every IT system with an update management infrastructure—for Oracle Linux systems, the Unbreakable Linux Network (ULN) tracks system software release levels, automatically advising you as soon as critical updates become available. Updates and errata are also available at no charge from the Oracle public yum repository.

Updating the kernel or core system libraries typically requires a system reboot. In mission-critical enterprise or cloud environments, this can delay the installation of crucial updates until systems are rebooted during a scheduled maintenance window. As a result, systems that support critical business applications might be running while they are unprotected from known vulnerabilities. To counter this problem, Oracle Linux Premier Support includes access to Ksplice, an innovative technology that allows administrators to apply security updates, patches, and critical bug fixes without rebooting. Ksplice improves the security, reliability, and availability of Oracle Linux by enabling zero downtime updates, helping to keep systems up to date without downtime or service disruption. To get started using Ksplice, see the "Oracle Ksplice User's Guide." Oracle is also offering a free 30-day trial of Ksplice for Red Hat Enterprise Linux customers (see OTN's Kplice Technology page).

Final Thoughts

While Oracle Linux is known for its robust, built-in security features, proper operations can help to optimize system security and protect data assets. This article provides some guidelines and operational practices to consider.

Beyond hardening the operating system (discussed in the first article) and performing comprehensive administrative procedures (highlighted in this article), a comprehensive security architecture and governance framework is critical for ensuring the integrity and availability of systems and data. Such a framework includes documented security policies, routine system monitoring and management procedures, and periodic security evaluations. It's important to periodically review administrative practices and end-to-end security precautions to determine what additional safety measures you can take to protect your Oracle Linux and other enterprise IT systems.

About the Authors

Ginny Henningsen has worked for the last 15 years as a freelance writer developing technical collateral and documentation for high-tech companies. Prior to that, Ginny worked for Sun Microsystems, Inc. as a Systems Engineer in King of Prussia, PA and Milwaukee, WI. Ginny has a BA from Carnegie-Mellon University and a MSCS from Villanova University.

James Morris is the Linux kernel security subsystem maintainer. He is the author of sVirt (virtualization security), multi-category security (MCS), and the kernel cryptographic API, and he has contributed to the SELinux, Netfilter, and IPsec projects. He leads the mainline Linux kernel team for Oracle and is based in Sydney, Australia.

Lenz Grimmer is a member of the Oracle Linux product management team. He has been involved in Linux and Open Source Software since 1995.