Oracle Berkeley DB Storage Layer

Oracle Berkeley DB is an small and efficient software library, providing the basic building blocks for any data storage solution. Incorporating Berkeley DB into your software application allows you to quickly and easily store application data in relational tables using SQL, as XML documents accessible using XQuery, as persistent graphs of Java objects, or as opaque unstructured data in key/value pairs accessible using simple get/put or cursor positioning operations.

The information below describes the features provided by the Oracle Berkeley DB storage engine. You will notice configuration options—no two application scenarios are identical, and we do not presume to know which trade-offs are best for your application. We leave those decisions up to you.

Services

• Data Access

  The most commonly used data access storage option in Berkeley DB is B-trees. Berkeley DB B-trees fundamentally store keys in sorted order and a value associated with that key. Internal pages within the B-tree contain a sorted list of keys that point to the data pages, where the data is stored. Berkeley DB B-trees are the basic method of storage for the Key/Value API, as well as all of the higher level APIs. For example, the XML API provides XQuery access to your XML data, which is stored in document containers with automatically maintained application-defined indices. Those document containers and indices are actually Berkeley DB B-trees, providing highly performant, concurrent, reliable, scalable indexed access to the underlying data. Likewise, regardless of whether you are using the Java Direct Persistence Layer API or the SQL API, the application data is being stored in Berkeley DB B-trees. There are some applications for which B-trees may not be optimal. Some applications require the use of Queues (for FIFO or LIFO data access) or Hash tables (for very large data sets where there is no inherent ordering ). For these applications, Berkeley DB offers alternate data access methods, Queue and Hash, available via the key/value API.

  XML content lives in document containers. Berkeley DB supports arbitrary indexes on your XML content and provides XQuery to efficiently access the portions of the documents you need. Relational content, such as tables, rows and indexes are also BTREEs. When Berkeley DB stores relational, graph, or hierarchical data it uses these indexes to project and select data from within those BTREEs.

  Most customers find BTREEs the most useful and versatile access method, so we recommend starting there, creating a custom comparison function to optimize the organization of your keys, and then evaluate the resulting performance.

• Locking

  Oracle Berkeley DB manages concurrent access to data using a highly advanced and configurable locking system. Without locking it is impossible to maintain consistency in a concurrent storage system. When configured for concurrent or transactional data access, Berkeley DB automatically manages all the locks based on the nature of the various operations in progress. For the most part, as an application developer, locking is something that once configured simply works. There is no need for your program to acquire and release locks, Berkeley DB does that for you.

  Multi-version concurrency control (MVCC) is especially useful in highly concurrent and contentious applications. MVCC prevents readers from blocking writers by keeping snapshots of pages involved in the transaction. Applications that use MVCC typically configure a larger cache to account or the increased memory used for snapshots.

• Transactions, Write-ahead Logging, and Recovery

  Berkeley DB provides highly configurable, field tested transactions and database recovery. Transactions exhibit the properties of atomicity, consistency, isolation, and durability (ACID). A write-ahead transaction log maintains undo/redo information for all actions protected by a transaction (including file system operations).

  Berkeley DB allows the application programmer to fine tune the transactional behavior (isolation and durability) of Berkeley DB on a per-transaction basis. Berkeley DB supports Serializable (default), Read Committed and Read Uncommitted isolation levels. You can choose to change the isolation level for any given transaction (or set a default for all transactions). If you want to ensure that reads performed for a given transaction won’t ever block writes then you can enable multi-version concurrency control (MVCC), also known as snapshot isolation.

  Like isolation, transaction durability is flexible on a per transaction basis under the application’s control. You can specify that a transaction is not considered durable until it has been fully written to the storage media, written to the operating system, or simply written into application buffers. The first case will allow recovery even when the operating system unexpectedly halts (power outage, systems failure, etc.). The second will survive application failure because the data is in the operating system’s buffers even after the application exits. The third option still guarantees that the database is transactionally consistent although some recent transactions may be rolled back because the application or system failure occurred before the transaction data was completely written to the log.

Tools

• SQL Developer

  Oracle SQL Developer is a free and supported graphical tool for database development. With SQL Developer, you can browse database objects, run SQL statements and SQL scripts, and edit and debug schema objects. Using the provided ODBC driver and the Oracle Database Gateway for ODBC (DG4ODBC) it is possible to use SQL Developer with Oracle Berkeley DB 11g for SQL databases.

• JDeveloper

  Oracle JDeveloper is a free integrated development environment that simplifies the development of Java-based SOA applications and user interfaces with support for the full development life cycle. JDeveloper now has support for Berkeley DB XML. You can browse Berkeley DB XML containers, execute XQuery statements, and edit XML content.

• Eclipse

  Oracle Berkeley Database 11g can be used in conjunction with the Eclipse IDE. The easiest way to store Java objects in a database is to use the Direct Persistence Layer (DPL). To speed development and ensure proper use of the DPL API you can use the Direct Persistence Layer Assistant for Eclipse. This plug-in performs validation of DPL annotations in Java source code. Each time you save changes to Java source code, the evaluator analyzes annotations (@Entity, @Persistent, etc.) and reports any errors or warnings that it can detect, in a similar way to how the IDE reports Java compilation errors. This helps developers find issues early in the development cycle reducing the time required to build the application.

• SQLite Tools, Add-ons, Extensions

  Because Berkeley DB’s SQL support is API compatible with SQLite most if not all tools, add-ons, extension and other application development tools that work with SQLite should also be able to work with Berkeley DB as long as the tool uses the Berkeley DB library.

• Apache Maven

  Object-oriented programming, especially when using Java, has finally delivered on the promise of reuse. One tool that helps programmers combine a multitude of components into a single solution is Apache Maven. Oracle Berkeley DB 11g is available via Maven to ease integration into your complex projects.

Platforms

• Solaris, Linux, Mac OS/X, BSD

  Berkeley DB relies on a standard set of POSIX Operating System interfaces. It should work on most UNIX-like platforms with little or no modifications required. Oracle Berkeley DB ships as a standard component of most UNIX operating systems. Commonly you’ll find the installed library in /usr/lib called libdb.[so,a] and the header file in /usr/include/db.h. Check your system to see if it’s already there. Most UNIX operating systems also have automatic software installers (RPM on RedHat, APT on Debian and Ubuntu systems, and the BSD PORTS system to name a few) to install Berkeley DB automatically on your system. Of course you can always download the latest version and build the library from source. The default install location is /usr/local/BerkeleyDB[version]. Building the library allows you the most control over what features are enabled allowing you to tailor the services to your particular use case.

• Microsoft Windows

  Oracle Berkeley DB 11g is easily integrated into Microsoft Windows applications. A pre-built Berkeley DB dynamic link library (DLL) is part of the Windows installer package. The Berkeley DB source code includes support for building the library using Microsoft Visual Studio. Berkeley DB includes support for the .NET runtime and has a C# language API. It is a managed interface available to all Common Language Runtime (CLR) applications. C++ applications will benefit from the support for persistent Standard Template Library (STL) classes. Berkeley DB also supports Microsoft Windows Embedded, Windows CE, and Windows Mobile platforms making it an ideal database for Windows applications regardless of where they are deployed.

• Android, iPhone OS, VxWorks, QNX

  Oracle Berkeley DB 11g is well suited to mobile consumer devices. Applications deployed to consumer devices need a database that is small and capable. Berkeley DB supports many such consumer device platforms and has been deployed in millions of mobile phones, mobile communications infrastructure, medical monitoring devices, and other industry specific mobile devices. These devices contain sensitive and important information and, due to the simple fact that most are battery powered, they can stop operating at unforeseen and inopportune times. The new requirements in these mobile devices are data integrity after power loss, data encryption to prevent unauthorized access, data synchronization, and flash storage optimizations to compress data and reduce power usage.

• Compilers

  Sun Studio, Intel Pro*C/C++, GNU Compiler Collection (GCC), Microsoft Visual Studio, Low Level Virtual Machine (LLVM) C-language (Clang), and most other standards-compliant compilers with POSIX libraries.

• Platform Portability

  Berkeley DB was designed to be highly portable across operating systems, chip sets, and compilers. Because of this careful design it often “just works” in the target platform environmnet even in cases where the Oracle Berkeley DB team may never have tested it. The Berkeley DB Reference Guide includes a Porting Guide, we are always happy to work with customers and partners who would like to port Berkeley DB to other platforms.