Cluster computing is not a new area of computing. It is, however, evident that there is a growing interest in it usage in all areas where applications have traditionally used parallel or distributed computing platforms. The mounting interest has been fuelled in part by the availability of powerful microprocessors and high-speed networks as off-the-shelf commodity components as well as in part by the rapidly maturing software components available to support high performance and high availability applications |
-
What is a Cluster?
A cluster is a type of parallel or distributed processing system, which consists of a collection of interconnected stand-alone/complete computers cooperatively working together as a single, integrated computing resource. (more) |
Key Benefits of Clustering
- System availability (HA)
- Hardware Fault Tolerance
- OS and application reliability
- Scalability
- High Performance
Why Linux Cluster?
- Cost Effective and Easy configurability
- Fast technical development with open source
- Many references in various fields
Types of Clustering
- High Performance Clusters - low price supercomputing(Beowulf project)
- High Availability Clusters - high available and fault tolerant system(HA project)
- Bulk Storage Clusters - stored data sharing and service
- Web/Internet Clusters - load balancing(LVS project)
Cluster
Requirements
- SSI(Single System Image) seen as single system to end user
- File System Requirements global file system with NFS
- Cluster Management Software need to manage as single system
- High Speed Interconnection Network Gigabit / Myrinet / SAN
Major Issues in Cluster Design
- Size Scalability (physical & application)
- Enhanced Availability (failure management)
- Single System Image (look-and-feel of one system)
- Fast Communication (networks & protocols)
- Load Balancing (CPU, Net, Memory, Disk)
- Security and Encryption (clusters of clusters)
- Distributed Environment (Social issues)
- Manageability (admin. And control)
- Programmability (simple API if required)
- Applicability (cluster-aware and non-aware app.)
How to Build Linux Cluster?
Research Projects - Cluster Computing
- Beowulf (CalTech and NASA) - USA
- CCS (Computing Centre Software) - Paderborn, Germany
- Condor - Wisconsin State University, USA
- DQS (Distributed Queuing System) - Florida State University, US.
- EASY - Argonne National Lab, USA
- HPVM -(High Performance Virtual Machine),UIUC&now UCSB,US
- far - University of Liverpool, UK
- Gardens - Queensland University of Technology, Australia
- MOSIX - Hebrew University of Jerusalem, Israel
- MPI (MPI Forum, MPICH is one of the popular implementations)
- NOW (Network of Workstations) - Berkeley, USA
- NIMROD - Monash University, Australia
- NetSolve - University of Tennessee, USA
- PBS (Portable Batch System) - NASA Ames and LLNL, USA
- PVM - Oak Ridge National Lab./UTK/Emory, USA
Linux Cluster Products
- Beowulf
- Linux Networx Evolocity
- SteelEye LifeKeeper
- Linux Virtual Server
- MOSIX
- Red Hat Piranha
- TurboCluster EnFuzion
Presentation Slides
- Clustering Technology Overview : Clustering with Linux
- Cluster Tutorial
- How to Build Linux Cluster
- Survey on Linux Clustering - Yoon-Jung OH
International Forum
- IEEE Task Force on Cluster Computing(TFCC)
Related Links
- The Linux Virtual Server project (LVS includes Network Address Translation, tunneling, and direct routing). LVS uses Linux-HA "heartbeat" code.
- Red Hat High Availability Server project
- Piranha - Load-balanced Web and FTP Clusters
- Beowulf project at CESDIS
- Beowulf Tutorial : Building a Beowulf System
- Beowulf Installation and Administration HOWTO
- Linux Parallel Processing
- High Availability RAID from Software RAID HOWTO
- Mosix (scalable computing clusters, including process migration)
- The CODA distributed, fault-tolerant filesystem
- The Linux Network Address Translation project
- The Linux Scalability Project. The primary goal of this research is to improve the scalability and robustness of the Linux operating system to support greater network server workloads more reliably. We are specifically interested in single-system scalability, performance, and reliability of network server infrastructure products running on Linux, such as LDAP directory servers, IMAP electronic mail servers, and web servers, among others.
- OSCAR Open Source Cluster Application Resources. The OSCAR is a snapshot of the best known methods for building, programming, and using clusters. It will hopefully bring some form of uniformity to clusters, foster commercial versions of the cluster software, and make clusters more broadly acceptable.
- Beowulf (Linux supercomputing clusters) home page
- Scyld Computing. The original Beowulf team providing second generation Beowulf Clustering.
- The Linux/IR project
- Linux Enterprise Computing overview
- A High-Availability Cluster Checklist A Linux Journal Article by Tim Burke
- A High-Availability Cluster for Linux A Linux Journal Article by Phil Lewis
- Linux Journal #55 (November, 1998) article: High Availability Linux Web Servers by Aaron Gowatch - aarong@divinia.com
- MontaVista HA white paper. by John Mehaffey <mehaf@mvista.com>
- Linux in depth(http://www.linux-mag.com/2000-10/clustering_01.html)
- Top Cluster List site