Blog

Study Guide

Disclaimer:
Educational Use Only:
The website links provided are to be used strictly for educational and knowledge-gaining purposes only.
No Misuse:
Do not misuse the links or the information contained therein for any commercial, illegal, or unethical activities.
Respect for Websites and Businesses:
Respect the intellectual property and business interests of the websites and their owners. Do not attempt to alter, copy, or redistribute content without proper authorization.
Acknowledge Value of Knowledge:
Recognize and value the knowledge and resources shared by these websites. Use the information responsibly and ethically.
No Endorsement:
The provision of these links does not imply endorsement of the linked sites, their content, or their products and services. Users are responsible for evaluating the suitability and accuracy of the information.
User Responsibility:
By accessing these links, you agree to use the information at your own risk and to comply with all applicable laws and website terms of use.

 

Introduction

Fencing is a foundational concept in server technology, especially in high-availability (HA) clusters and enterprise environments. It refers to the process of isolating a malfunctioning node or server to protect shared resources and maintain data integrity. Fencing ensures that when a node fails or behaves unpredictably, it cannot disrupt the operation of the rest of the system or corrupt critical data1514. This script explores fencing in depth, covering definitions, key concepts, technical overviews, types, operational principles, real-world applications, advantages, disadvantages, and future outlook.

 

Definitions and Explanations

Fencing (Computing):
Fencing is the process of isolating a node in a computer cluster or restricting its access to shared resources when it appears to be malfunctioning114. The primary goal is to ensure that only healthy nodes can access and modify shared resources, thereby preventing data corruption and maintaining system stability.

Key Terms:

  • Node: An individual server or computer participating in a cluster.
  • Cluster: A group of interconnected computers working together to provide redundancy and high availability.
  • STONITH: “Shoot The Other Node In The Head”—a fencing technique that forcibly powers off or disables a malfunctioning node1512.
  • I/O Fencing: Input/Output fencing; a mechanism to prevent a node from accessing shared storage, typically by revoking its access rights13.
  • Split Brain: A scenario where cluster nodes lose communication but continue to operate independently, risking data corruption112.

Analogy:
Imagine a bank vault with multiple managers (nodes) who can access the vault (shared storage). If one manager starts acting suspiciously (malfunctioning), the system (cluster) must ensure that this manager cannot access the vault until it is certain they are trustworthy again. Fencing is the digital equivalent of revoking that manager’s access badge.

 

Key Concepts, Theories, and Principles

Why Fencing is Necessary:

  • Data Integrity: Prevents simultaneous access to shared resources by malfunctioning nodes, avoiding data corruption114.
  • Cluster Stability: Ensures that the failure of one node does not cascade into system-wide issues512.
  • Split Brain Prevention: Fencing is critical for preventing split brain, where two or more nodes believe they are the primary resource controller, leading to conflicting actions11213.

Principles of Fencing:

  1. Isolation: The failed node must be completely isolated from shared resources.
  2. Automated Response: Fencing should be automated to ensure rapid response to failures.
  3. Fail-Safe: The system must assume the worst-case scenario and fence nodes when in doubt112.
  4. Recovery: After fencing, recovery processes can safely restore services using healthy nodes.

Mnemonic:
Remember FENCE:

  • Fault isolation
  • Ensure data integrity
  • Node removal
  • Cluster stability
  • Eliminate split brain

 

Technical Overview

Fencing is implemented through a combination of software agents and hardware devices. The process typically involves:

  1. Detection: Cluster management software detects a node failure.
  2. Decision: The system determines the appropriate fencing method.
  3. Execution: A fencing agent communicates with a fencing device to isolate the node.
  4. Confirmation: Once fencing is complete, recovery actions proceed.

Cluster Infrastructure Components:

  • Cluster Manager: Orchestrates fencing and recovery actions.
  • Fencing Agent: Software component that interfaces with hardware (e.g., power controllers, network switches) to execute fencing912.
  • Fencing Device: The hardware or virtual mechanism that enforces isolation (e.g., power switch, storage controller).

Diagram Example:

text

[Cluster Manager] –(detects failure)–> [Fencing Agent] –(commands)–> [Fencing Device] –(isolates)–> [Failed Node]

 

Types of Fencing

Fencing Type

Description

Example Devices/Mechanisms

Power Fencing

Powers off the failed node

IPMI, HP ILO, DRAC, Power Distribution Units (PDUs)451012

Storage/I/O Fencing

Blocks node’s access to shared storage

SCSI-3 Persistent Reservations, Fibre Channel Switches71013

Network Fencing

Cuts network access for the node

Network switches, firewalls

Virtual Machine Fencing

Isolates or powers off a VM in cloud environments

Cloud provider APIs, AWS CLI scripts5

Fabric Fencing

Disables access to a specific resource (e.g., disk, network)

GNBD, Fibre Channel fencing1410

STONITH (Shoot The Other Node In The Head):
The most direct fencing method, STONITH forcibly powers off or disables the node, ensuring it cannot interfere with the cluster1512.

Persistent Reservation Fencing:
Uses storage device features (like SCSI-3 PR) to block access from the failed node by removing its reservation key710.

 

Characteristics

  • Automated and Deterministic: Fencing actions are automated and follow strict rules to ensure consistency112.
  • Transparent to Applications: Applications running on healthy nodes are largely unaffected by fencing actions.
  • Vendor and Platform Support: Supported by major server and storage vendors, as well as open-source cluster management tools (e.g., Pacemaker, Red Hat Cluster Suite)212.
  • Scalable: Works in small two-node clusters and large enterprise environments.

Table: Fencing Device Properties

Property

Description

Host Mapping

Maps hostnames to device ports

Device List

Lists machines controlled by the device

Location

Defines which machines are controlled by the device

Advanced Options

For specific device features (e.g., delay, retries)

 

Applications and Use Cases

  • High-Availability Clusters: Ensures continuous service by isolating failed nodes and allowing others to take over resources514.
  • Database Clusters: Prevents split brain and data corruption in clustered databases like PostgreSQL, MySQL, Oracle RAC510.
  • Virtualization and Cloud: Isolates malfunctioning VMs or hosts in cloud platforms (e.g., AWS, Azure)5.
  • Enterprise Data Centers: Protects shared storage in SAN/NAS environments13.
  • Kubernetes Clusters: Maintains storage consistency in containerized environments8.

Case Study: Fujitsu Enterprise Postgres HA Cluster
In a Fujitsu Enterprise Postgres cluster, fencing is implemented using IPMI for on-premises servers and AWS CLI scripts for cloud VMs. If the primary server becomes unreachable, a script fences it off before failing over to a standby server, preventing split brain and data loss5.

 

Operational Principles

  1. Failure Detection: Cluster monitors nodes for health and responsiveness.
  2. Fencing Triggered: If a node is unresponsive or erratic, fencing is initiated.
  3. Resource Reclamation: Once fencing is confirmed, resources (e.g., storage, IP addresses) are reassigned to healthy nodes.
  4. Recovery: The failed node is either repaired and rejoined or replaced.

Example: SCSI-3 Persistent Reservation Fencing

  • Each node registers a unique key with the storage device.
  • On failure, the key is removed, blocking the node’s access to the disk710.

 

Advantages

  • Ensures Data Integrity: Prevents simultaneous or unauthorized access to shared resources113.
  • Minimizes Downtime: Automated fencing enables rapid failover and recovery5.
  • Prevents Split Brain: Critical for avoiding data corruption in clustered environments1213.
  • Vendor Support: Widely supported across hardware and software platforms212.

 

Disadvantages

  • Complex Configuration: Requires careful setup and testing to avoid accidental outages2.
  • Hardware Dependency: Some methods depend on specific hardware features (e.g., IPMI, SCSI-3 PR)510.
  • Potential for False Positives: Network glitches or misconfigurations can trigger unnecessary fencing.
  • Service Disruption: Fencing forcibly disables nodes, which may impact co-hosted services if not properly isolated5.

 

Examples and Illustrations

Example 1: Power Fencing with IPMI

  • A cluster node becomes unresponsive.
  • The cluster manager uses IPMI to power off the node’s server.
  • The remaining nodes continue operating, and the failed node is safely isolated5.

Example 2: Storage Fencing with SCSI-3 PR

  • Multiple nodes access a shared SAN.
  • Each node holds a reservation key.
  • Upon failure, the cluster removes the failed node’s key, blocking its access to the storage710.

Example 3: Virtual Machine Fencing in AWS

  • A database VM becomes unresponsive.
  • A fencing script uses AWS CLI to stop the VM, isolating it from the cluster5.

 

Charts and Tables

Table: Fencing Methods Comparison

Method

Hardware Required

Isolation Level

Use Case Example

Pros

Cons

Power Fencing

Yes (IPMI, PDU)

Node

Physical server clusters

Simple, effective

Requires hardware support

Storage Fencing

Yes (SAN, SCSI)

Storage

Database clusters

Granular, storage-focused

Complex setup

Network Fencing

Yes (Switches)

Network

Multi-tenant clusters

Flexible

May not fully isolate node

VM Fencing

No (Cloud API)

VM

Cloud environments

No hardware needed

Cloud API limitations

 

Future Outlook and Relevance

  • Smart Fencing: Integration with AI/ML to detect failures and automate fencing decisions more intelligently.
  • Cloud-Native Fencing: Enhanced support for containerized and virtualized environments, including Kubernetes and hybrid clouds8.
  • Virtual Fencing: Increased adoption of software-based fencing in serverless and edge computing scenarios.
  • Security Integration: Combining fencing with advanced security protocols to protect against insider threats and cyberattacks36.

Market Trends:
The demand for fencing, both physical and digital, is growing with the proliferation of data centers, cloud computing, and critical infrastructure requiring robust protection6.

 

Concise Summaries

  • Fencing is essential for isolating malfunctioning nodes in server clusters, ensuring data integrity and system stability.
  • STONITH and I/O fencing are primary methods, implemented through hardware (IPMI, PDUs) or software (storage reservations, cloud scripts).
  • Fencing prevents split brain, minimizes downtime, and is widely supported across platforms.
  • Proper configuration and understanding of fencing methods are crucial to avoid unintended service disruptions.
  • The future of fencing includes smarter, software-driven solutions for increasingly complex and virtualized environments.

 

Conclusion

Fencing is a critical safeguard in server technology, particularly in high-availability and clustered environments. By isolating malfunctioning nodes, fencing protects shared resources, prevents data corruption, and ensures the continuity of essential services. As server environments evolve toward greater virtualization and cloud adoption, fencing mechanisms will continue to adapt, integrating smarter automation and broader compatibility to meet the demands of modern IT infrastructure.

 

Recommended Further Reading and Resources

  • Red Hat High Availability Cluster Documentation: In-depth guides on fencing configuration and best practices24914.
  • ClusterLabs Pacemaker Documentation: Detailed explanations and examples of fencing in open-source clusters12.
  • IBM Informix and GPFS Documentation: Technical details on I/O fencing and storage protection10.
  • YouTube: InfoScale 8.0 and I/O Fencing in Kubernetes: Video demonstration of fencing in containerized environments8.
  • AWS Documentation: Guides on using AWS CLI for VM fencing in cloud environments.
  • DevX Glossary: Input/Output Fencing: Clear definitions and real-world scenarios13.

 

Mnemonic Recap:
Remember FENCE—Fault isolation, Ensure data integrity, Node removal, Cluster stability, Eliminate split brain.

 

  1. https://en.wikipedia.org/wiki/Fencing_(computing)
  2. https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/
    8/html/configuring_and_managing_high_availability_clusters/assembly_
    configuring-fencing-configuring-and-managing-high-availability-clusters
  3. https://www.makefencing.com.au/post/fencing-and-parameter-solutions
  4. https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/
    4/html/cluster_suite_overview/s2-fencing-overview-cso
  5. https://www.postgresql.fastware.com/pzone/2024-07-implementing-
    fencing-in-fujitsu-enterprise
  6. https://www.businesswire.com/news/home/20250115967580/en/Fencing-Market-Outlook-Report-2024-2029-New-Technologies-like-Solar-powered-Electrical-Fencing-Virtual-Fencing-and-Smart-Fencing-are-Increasing-the-Adoption-of-Fencing-Usage—ResearchAndMarkets.com
  7. https://documentation.ubuntu.com/server/explanation/high-availability/pacemaker-fence-agents/
  8. https://www.youtube.com/watch?v=snGmbGFkE7I
  9. https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/6/
    html/high_availability_add-on_overview/ch-fencing
  10. https://www.ibm.com/docs/en/informix-servers/12.10.0?topic=clusters-io-fencing-shared-file-systems
  11. https://www.youtube.com/watch?v=ZnhJ91-5y1Q
  12. https://clusterlabs.org/projects/pacemaker/doc/2.1/Pacemaker_
    Explained/html/fencing.html
  13. https://www.devx.com/terms/inputoutput-fencing/
  14. https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/
    7/html/high_availability_add-on_overview/s1-fencing-haao
  15. https://pve.proxmox.com/wiki/Fencing
  16. https://www.techtarget.com/searchitoperations/definition/STONITH-
    Shoot-The-Other-Node-In-The-Head
  17. https://www.wildlifesafetysolutions.com.au/our-products
  18. https://documentation.suse.com/sle-ha/15-SP4/html/SLE-HA-all/cha-ha-fencing.html
  19. https://www.externalworksindex.co.uk/Docs/1683802456616/preview/The-Essential-Guide-To-Data-Centre-Security-Fencing.pdf
  20. https://www.pir.sa.gov.au/__data/assets/pdf_file/0006/466035/Case_
    Study_Virtual_Fencing.pdf
  21. https://www.sciencedirect.com/science/article/pii/S1871141323002196
  22. https://www.magonlinelibrary.com/doi/10.12968/live.2023.28.5.227
  23. https://www.publish.csiro.au/an/pdf/AN20525
  24. https://www.sciencedirect.com/science/article/pii/S0006320724002982
  25. https://www.veritas.com/support/en_US/article.100021261
  26. https://www.reddit.com/r/Fencing/comments/8s0ylb/does_anyone_
    have_a_good_diagram_or_mnemonic_of/
  27. https://www.veritas.com/support/en_US/doc/79757062-166315475-0/v45911703-166315475
  28. https://www.veritas.com/support/en_US/doc/79561893-149454824-0/v45013039-149454824
  29. https://www.hashstudioz.com/blog/passive-to-proactive-secure-your-perimeter-with-smart-fencing/
  30. https://fencingtime.s3.amazonaws.com/FencingTimeServerSystem
    Guide.pdf
  31. https://au.tru-test.com/products/fence-monitoring/fence-monitoring
  32. https://documentation.suse.com/fr-fr/sle-ha/12-SP5/html/SLE-HA-all/
    cha-ha-fencing.html

Leave a Reply

Your email address will not be published. Required fields are marked *

error: Content is protected !!