OCI File Storage
What is Oracle Cloud Infrastructure (OCI) File Storage?
OCI File Storage is a fully managed, network-attached storage system that offers high scalability, high durability, and high availability for your data in any Oracle Cloud Infrastructure availability domain. OCI File Storage supports the Network File System (NFS) version 3 protocol with Network Lock Manager (NLM) for the locking mechanism to provide POSIX semantics. This enterprise-grade file storage service scales up to meet your storage needs and can be accessed concurrently by thousands of compute instances. You can start with a file system that contains only a few kilobytes of data and scale to 8 exabytes of data without any up-front provisioning. Moreover, your data and metadata are protected with encryption at rest by default, and you have the option to enable encryption in transit as well. Additionally, snapshot capabilities give copy-on-write local replication of your data, and File Storage replication provides replication of file systems across availability domains or across regions.
When should I use Oracle File Storage?
You should use File Storage when:
- Your workload requires durable and highly available shared storage with file semantics.
- You need service elasticity in the cloud and scale-out performance for shared access.
- You want your data to persist beyond your instance life.
File Storage provides consistency of traditional NFS files, operational flexibility, remove storage administrative and maintenance tasks with simple pay-per-capacity pricing. As soon as you delete your data, you don't pay for that.
What use cases does File Storage support?
File Storage supports a broad range of use cases, including the following ones:
- Oracle applications:
- Shared file storage for enterprise lift-and-shift applications for optimal disk consumption and ease of deployments
- Structured and unstructured data:
- User and application data storage, including images, videos, Internet of Things (IoT), and log files
- Target storage for backup and archive utilities such as Oracle Recovery Manager
- Container-based applications:
- Persistent storage for Docker and Kubernetes environments
- Big data and analytics:
- Storage for both source and value-added data
Can my current application take advantage of File Storage?
Any application that uses NFS version 3 works with minimal to no modification with File Storage. This includes Oracle applications and solutions such as Oracle E-Business Suite, and any enterprise applications that needs scale-out access or storage space.
What are the core components of File Storage?
- File systems are for storing and organizing your shared data, and its metadata. Metadata contains information about your files such as name, directory, and permissions.
- Mount targets are highly-available NFS endpoints in your subnet of choice, used to access your file systems. Your NFS clients mount to file systems through mount targets in order to be able to read and write data. You can access multiple file systems behind one mount target.
- Exports control which file systems are available through a given mount target. The information stored in an export includes the file system ID, the export path, and NFS export options.
- NFS Export Options are a set of parameters that enables a more granular access control on a per file system basis, when connecting to one mount target.
- Export Path is specified when a file system is associated with a mount target.
To learn more about these components, watch the short video on File Storage components.
What are the default settings in Oracle File Storage?
By default, File Storage lets you create 100 file systems and 2 mount targets in every availability domain of your choice. Each file system can grow to 8 exabytes of data. You can create 10,000 snapshots of every file system. For a list of default settings and instructions for requesting an increase, see Service Limits.
What interface does Oracle File Storage support?
File Storage supports NFS version 3, including a file locking mechanism that uses Network Lock Manager (NLM) protocol. File Storage works with common NFS version 3 clients, including those on Linux, Solaris, and Windows, as well as Oracle's Direct NFS driver. For instructions on how mount from Unix-style and Windows systems, see Overview File Storage.
What is the durability and availability of my data stored in OCI File Storage?
our data is replicated for durability within each availability domain in a highly available infrastructure that implements industry-leading data protection techniques and best practices. OCI File Storage service is designed to provide 99.999999999% (eleven 9s) annual durability for file systems. We recommend making regular backups of file-system snapshots or using File Storage replication to protect against the failure of an availability domain or region.
How does File Storage support snapshots?
File Storage service lets you create snapshots by using the web-based Console, command line interface (CLI), or REST APIs, or from the file system itself by creating a directory in the file system's .snapshot directory. Snapshots provide a read-only, space-efficient, point-in-time view of your entire file system. File Storage snapshots employ copy-on-write and consume storage space only when data changes. You are billed only for the storage space that your snapshots use for the delta of your changed data.
Getting started
How do I get started with File Storage?
For an overview of File Storage and its concepts, see the File Storage documentation. You can create a file system and mount to it with only few clicks in the web-based Console. You can also use the REST APIs , CLI and Terraform to create file systems and mount targets.
After creating a file system and a mount target, access your file system from your compute instances. Log in to an instance to install the NFS client and mount to your mount target. For information about installing the NFS client, see the following mounting instructions for each instance type:
Unix-style Instances:
Windows Instances:
How do I access File Storage?
File Storage is physically located in every Oracle Cloud Infrastructure availability domain—you can access it from anywhere! To access it within a region, you need only to ensure that your security rules are correctly configured to allow NFS traffic. For more information, see Configuring VCN Security List Rules for File Storage. For on-premises connection to your file system, you also need FastConnect or a virtual private network (VPN). Accessing a file system from an instance in a different Oracle Cloud Infrastructure availability domain incurs a small performance penalty because of the network latency that exists between availability domains.
How do I remove file locks from a host that is no longer available?
For information about removing locks from a file system, see the File Storage documentation.
How do I use OCI File Storage snapshots for backup?
Use File Storage replication to replicate the snapshots to another availability domain or region. You can also use rsync, tar, or any third-party tool that supports NFS version 3 to copy your data to another Oracle Cloud Infrastructure availability domain or region, to Oracle Cloud Infrastructure Object Storage, or to your on-premises storage.
How do I migrate my existing data to File Storage?
Within a region, you can use standard tools like scp, rsync, or SSHFS to move data. Because File Storage can be accessed from multiple compute instances concurrently, you can improve copying speeds with parallel uploads. If you want to bring data from outside of a region, use a VPN or a FastConnect to mount to your file system from your on-premises data center. For additional options, please visit Data Transfer Service and Storage Gateway.
Access and security
How do I manage security and access control for my file systems?
File Storage provides several ways for you to ensure that your data remains secure. Use these methods together to restrict access to your file systems.
- Oracle Cloud Infrastructure policies : Create policies to control what users can do within Oracle Cloud Infrastructure, such as creating a VCN and its security rules, file systems, mount targets, and export options.
- Network security lists : Create security lists to control which IP addresses and ports can access your mount targets.
- NFS export options : NFS export options apply access control to network security lists and NFS version 3 UNIX authentication. You can use NFS export options to limit access levels by IP addresses or CIDR blocks that connect to multiple file systems through exports of an associated mount target. As a result, access can be restricted so that each client’s file system is completely inaccessible and invisible to the others, which provides security for multitenant or managed-hosted environments. Moreover, you can set permissions for read-only, read/write, or root-squash for your file systems.
- NFS version 3 UNIX security model support : NFS version 3 manages security with standard UNIX-style read/write/execute permissions, based on user and group IDs. We verify the UNIX security model for authentication each time files are accessed.
- NFS access with Kerberos and Lightweight Directory Access Protocol (LDAP): Integrate with industry standard identity management systems using Kerberos for strong authentication and centralized management of users.
For more information about how different types of security work together in your file system, see About Security.
What is a high performance mount target?
A high performance mount target (HPMT) scales the throughput for each mount target up to 80 Gb/sec. You can add multiple such HPMTs to increase the aggregate throughput of a single file system. HPMTs are engineered to deliver exceptional speed, making them ideal to use with GPUs for AI/ML training and checkpointing.
Can I limit access to my file systems?
You can use NFS export options on export paths to limit access. Export paths are specified when a file system is associated with a mount target. The export path uniquely identifies the file system within the mount target, letting you associate up to 100 file systems behind a single mount target. The export path is appended to the mount target IP address, and used to mount (logically attach) to the file system. The export path exists solely as a way to distinguish one file system from another within a single mount target. For more information, see Paths in File Systems.
After you create a file system, set security options on your export paths for granular access control. For example, you can limit root user access, require connection from a privileged port, or completely deny access to some clients. For more information about access control lists with NFS export options, see Working with NFS Export Options.
What encryption does File Storage offer?
File Storage uses AES 256 for encryption at rest for newly created file systems, with unique Oracle-provided encryption keys for each file system. You also have the option to encrypt all of your file systems using the keys that you own, managed by the Key Management service. Additionally, File Storage provides the option for configuring encryption in transit for all data and metadata. Data encryption in transit uses Transport Layer Security (TLS) 1.2 to encrypt data sent between your clients and your filesystems.
Performance
How do I achieve the best performance with File Storage?
To optimize the performance of File Storage, consider the following guidelines:
- File Storage performance increases with parallelism. Increase concurrency by using multiple threads, multiple clients, and multiple mount targets. In particular, scalability will be greatest when clients and threads are accessing independent portions of the file system.
- Use tools to run file operations in parallel. The File Storage engineering team has developed parallel tar and untar (puntar), parallel copy (parcp), and parallel remove (parrm) tools. These tools are available in the fss-parallel-tools package in Oracle Linux.
- The available bandwidth to a file system can significantly impact its performance. In Oracle Cloud Infrastructure, larger instances (with more CPUs) are entitled to more network bandwidth. File Storage performance is best with Oracle bare metal instances or large VM shapes.
- To minimize latency, clients, mount targets, and file systems should be all in the same availability domain.
- For best performance, don't set any mount options such as rsize or wsize when mounting the file system. In the absence of these options, the system automatically negotiates optimal window sizes. To learn more, visit Mounting File Systems.
- Due to the limitations of OCI's VNICs, each mount target is limited to about 600 MB/s of read or write traffic. If you have bandwidth-heavy workloads, consider spreading your workload across multiple mount targets once your file system exceeds 10 TB.
Disaster recovery and business continuity
What OCI File Storage features are available for disaster recovery?
You can replicate File Storage to another availability domain and/or another region. File Storage replication allows up to three replication sessions per file system.
What other options are available to help with data mobility?
Snapshots provide a consistent, point-in-time view of your file system, and you can take as many snapshots as you need. You can turn any snapshot into an independent live read-write file system instantaneously by cloning snapshots. With File Storage replication, you can create clones across availability domains or across regions by cloning replicated snapshots on the target side.
More questions
If you have more questions, contact Oracle File Storage Product Management at filestorage_grp@oracle.com.