Lowering Storage and File-serving Costs with Network Attached Storage (NAS) Clusters
By Michael Callahan
Profound changes in enterprise storage and file serving are causing alarm among IT managers. IT managers must tackle one of their most vexing problems: how to get a grip on the rapid expansion of data and the demands it places on file serving and storage capacity. The data center is forced to store more data and keep it accessible longer to support business objectives and meet regulatory and compliance guidelines. This places an enormous strain on IT organizations in the areas of manageability, scalability and availability of their file-serving resources. An emerging file-serving alternative, called NAS clustering, is enabling IT managers to gain control over the constant growth, management and cost of file serving.
Today, IT managers find themselves with a proliferation of file servers of various sizes and types. In this environment, the concern is that the total cost of ownership (TCO) is spiraling out of control. In particular, budget constraints put pressure on administrative overhead or operating expense (OPEX). The danger is that, as storage is added, these expenses can grow disproportionately.
Thus, as data growth and increasing business demand press on administration overhead, IT managers face the following three key challenges:
- Manageability. Larger numbers of clients have driven many IT organizations to deploy multiple UNIX, Windows, or NAS servers, introducing management complexity and overhead in provisioning storage, backing up the data routinely, and performing maintenance and security upgrades.
- Scalability. Some users have turned to high-end servers and NAS appliances to simplify the provisioning of file services to a large number of clients or to support heavy workloads. However, this comes at a steep cost and, in some cases, this high-end hardware still represents a performance bottleneck between clients and data.
- Availability. As NAS has increasingly come to be used for mission-critical tasks, it has been necessary to purchase hardware in pairs to provide high availability, introducing yet more expense.
Clearly, circumstances cannot continue this way. A new approach must be used to gain control of the situation. What is needed is a way to provide scalable, highly available file services using inexpensive, industry-standard servers that can be added incrementally as demands require, while retaining the management simplicity of a single server and a single pool of exported file systems. This approach is NAS clustering.
NAS clustering has recently emerged to enable a scalable, fault-tolerant, file-serving cluster for thousands of users, Web clients or computing nodes. An NAS cluster offers a single point of management and backup for file services. It consolidates multiple management points onto a single unified file service utility. It also gives the IT manager the ability to scale capacity and performance with off-the-shelf components. For IT managers using expensive NAS appliances or UNIX file servers, NAS clustering can free them from proprietary approaches that create bottlenecks and do not scale easily or cost- effectively.
Introducing NAS Clusters
An NAS cluster implementation for file serving allows an IT manager to leverage the economics of industry-standard servers and storage and then couple those components into a single high-performance NAS system. At its core, the NAS cluster integrates Network File System (NFS) or Common Internet File System (CIFS) protocol functionality with a true “symmetric” cluster file system (CFS), high-availability services, and cluster and storage management capabilities. The NAS cluster approach can aggregate the performance of low-cost Linux- or Windows-based servers for modularly scalable, fault-tolerant file serving across a SAN. Shared data and management capabilities ease IT administration by enabling servers and storage to be managed as one.
At the heart of the NAS cluster is CFS technology that enables each cost-effective server to read and write to a shared pool of storage. An NAS cluster of x86-based servers running Linux or Windows changes the economics of NAS deployment. A cluster of inexpensive servers can now function as a single, easily managed, incrementally scalable “pay-as-you-grow,” highly available NAS appliance, without the cost of high-end servers or NAS devices.
The NAS cluster solves the problems posed by conventional NAS approaches. For example, the cluster can be treated as a single NAS server with a single pool of storage and a single set of exported file systems. There is no need to create redundant copies of data, or to divide clients manually among different servers. Additional servers and storage can be added to the cluster to meet growing workloads. Also, in the event of failure, servers in the cluster automatically take over each other’s clients immediately, with no interruption of service. Dynamic online addition of servers and storage means that no scheduled downtime is required for capacity upgrades.
The Economics of NAS Clusters
NAS clusters represent a different approach compared to traditional, fully integrated server, storage, and software products used in most data centers today. NAS clusters, when combined with industry-standard servers and storage and a mainstream operating system, such as Linux or Windows, enable a 6:1 price/performance advantage compared to traditional network attached storage.
Today, the most popular way to achieve highly available NAS is to deploy a cluster of two independent NAS appliances linked together for failover. However, an NAS cluster based on standard servers with a CFS can do much better. Based on benchmark data, a representative enterprise-class NAS failover cluster can deliver approximately 250 megabytes per second (MB/second) of highly available throughput performance—based on sustained sequential writes to a single file system. The list price of a representative enterprise-class NAS cluster consisting of two NAS appliances, 12.6 terabytes (TB) of storage, and licenses for the NFS protocol and failover clustering software would be $476,000 1. A two-node NAS cluster using standard servers provides equivalent performance, but only costs $79,242. In short, an NAS cluster is 82 percent less expensive than a comparable enterprise NAS appliance solution.
1 Source: Taneja Group
NAS Clustering Combines Key Technologies
An NAS cluster solution comprises several relatively new but proven software technologies:
- A high-performance, symmetric cluster file system (CFS). A CFS enables industry-standard servers to share concurrent read/write access to file systems resident in a SAN (Figure 1). The CFS provides very high throughput for the most demanding workloads, with direct disk access speeds and complete cache consistency among servers in the cluster.
- High-availability functionality. Protects against server, network and storage interconnect failures. It provides automatic restart of servers and services, immediate failover of clients to surviving nodes and support for Multi-Path I/O (MPIO).
- Integrated cluster-wide management. A single console provides an overview of server status. Cluster operations can be reconfigured either from the console or using a command line tool from any member of the cluster.
Figure 1. With a NAS cluster, all servers in the cluster have direct, simultaneous access to file systems in the SAN.
Figure 2 shows a cluster of industry-standard servers running on NAS cluster software, sharing access to a common set of file systems that they export via NFS or CIFS to clients. A client can access the exported file systems via any of the servers in the cluster. Also, multiple clients who have mounted the file systems via different servers can use the same files simultaneously, receiving the same results as if they were using a single server. Thus, clients can be spread among servers using DNS round robin. Alternately, a hardware load balancer can be used.
The NAS cluster’s high-availability features, along with support for NFS or CIFS failover, ensure that, in the event of a failure, clients will transparently be moved to a surviving cluster node. Thus, the cluster as a whole acts, essentially, as a single, highly scalable, highly available virtual NFS or CIFS server, but one that is cost-effectively built from industry-standard components.
The Challenge of Scalable, Manageable NAS
As the capacity of file servers, whether they be NAS appliances, or UNIX or Windows servers, is exceeded for particular applications, IT managers have been forced to choose between either upgrading to larger file servers or NAS appliances or adding more of them, perhaps replicating the same data among them. These choices have serious drawbacks:
- Larger servers/NAS appliances are more expensive per increment of serving capacity, and in some cases still remain bottlenecks on overall performance.
- More file servers create additional points of server and storage management.
- Fragmentation of workloads across more servers and storage decreases capacity utilization.
- Replicated data wastes storage capacity on copies of data and generates new data management challenges associated with keeping data copies consistent.
- High availability becomes increasingly expensive, requiring duplication of large amounts of expensive hardware.
The NAS Performance Challenge—Cost-Effective Scalability
A UNIX or Windows file server or NAS appliance represents a potential performance bottleneck for mission-critical, data-intensive applications. In NFS or CIFS, file system operations performed by clients generate network requests that must be received by the server, interpreted, executed by the server on the client’s behalf—perhaps requiring disk I/O—and then replied to. Thus, a large number of NFS or CIFS clients, or even a small number of clients with heavy demands, can saturate the NFS server’s network interfaces, CPU or storage connections, causing performance to degrade to an unacceptable level. These NFS or CIFS performance and scalability issues are particularly acute in oil and gas, biosciences, video production, and credit card data processing applications, where there is a need to process many single files in parallel or large data sets in sequence.
Figure 3 illustrates the file server bottleneck created by a traditional NAS configuration. In a high-performance NAS cluster configuration, each individual server contributes incremental resources to the cluster, including network bandwidth for client requests and replies, CPU capacity for file system processing, and I/O bandwidth to storage. No individual server’s capacity—in any of these dimensions—imposes a bottleneck on system performance. As needs grow, a cluster can be scaled from as little as a pair of dual-processor servers to a set of 16 servers each with four or eight CPUs, multiple gigabit Ethernet and Fibre Channel connections—all without interrupting client service and without requiring any migration of data.
In a recent benchmark, it was demonstrated that a nine-node cluster could deliver over one gigabyte per second of I/O throughput. In fact, based on the findings of that benchmark, a 10-node NAS cluster showed performance 470 percent faster than a traditional high-end NAS appliance. NAS clusters also exhibit near linear scaling. One deployment of an NAS cluster doubled performance with a doubling of the cluster size (100 percent scalability) up to eight nodes.
Simplifying NAS Deployments—Improving Manageability
Traditionally, multiple NAS appliances, Windows or UNIX servers have required administrators to divide data and clients among them, perhaps replicate data among servers, and monitor and provision free space separately on each server or appliance. In an NAS cluster environment, none of this is required. In an NAS cluster, all servers in the cluster can export a single set of file systems simultaneously. There is no need to divide data among separate servers or to replicate data among servers. Similarly, since any server can service any client’s requests, there is no need to assign clients manually to individual servers. Clients can be balanced among servers automatically using DNS round robin or a hardware load-balancer. In either case, no manual intervention is required. Free space is shared among all servers, simplifying storage provisioning.
Affordable High Availability
NAS clusters include built-in, high-availability functionality that permits the cluster to detect and respond to failures of network interfaces, storage interfaces, servers and operating systems within the cluster. The NAS cluster builds on this infrastructure to provide transparent failover of NFS or CIFS sessions.
A single server can provide failover protection for the entire cluster. If a server fails, the CFS will automatically reassign the failed server’s IP address to the standby server, allowing affected clients to continue making NFS or CIFS requests without any interruption of service. In fact, typically a client will have no indication that a server failure has occurred.
Because a single machine can provide failover protection for the entire cluster, it is not necessary to buy hardware in pairs to achieve high-availability. For example, in Figure 3, one server could provide failover protection for the other nine, giving just a 10 percent overhead for availability, as opposed to 50 percent in a typical, paired configuration.
In addition to protecting clients from failures, the CFS also enhances availability by reducing the need for scheduled downtime. Online addition of servers and storage is supported, making it possible to grow from a three-node cluster supporting just a few hundred gigabytes of storage, to a 10-node cluster hosting many terabytes, with no interruption of service for system upgrades, and with no need to reconfigure or restart any clients.
Summary
The NAS cluster provides a unique solution that addresses the scalability, availability and manageability of file serving in the enterprise. NAS cluster technology addresses the scalability and performance bottleneck posed by the most demanding, data-intensive applications. In addition, NAS clusters can be utilized to consolidate numerous, disparate NAS appliances or file servers onto a single centrally managed NAS cluster that provides inherent high availability and a clear growth path.
Michael Callahan is chief technical officer of PolyServe, Inc. PolyServe makes shared data clustering software that simplifies management, improves availability and enhances scalability by allowing industry-standard servers and blades using Linux or Windows to share concurrent access to data on a SAN.
Previously, Michael ran Advanced Development at Ask Jeeves and worked for the Roda Group, an early-stage venture capital group in Berkeley, California. He also worked as a mathematician in the areas of symplectic geometry and topology and differential geometry, and conducted research in scientific visualization, distributed systems, file systems, and computer vision. Previous technical work includes a port of the Coda distributed file system to Windows and networking code contributed to the Linux kernel.
Michael was a Rhodes Scholar, Jowett Senior Scholar, and Junior Research Fellow in Mathematics at Oxford University. He has a BA summa cum laude in social studies from Harvard University.