VMware vSAN Cheatsheet: Hyperconverged Infrastructure Storage

Overview

VMware vSAN is a software-defined storage solution that integrates directly with VMware vSphere to create a distributed storage layer. Modern vSAN (8.x) supports two architectures: the legacy Original Storage Architecture (OSA) and the newer Express Storage Architecture (ESA) optimized for NVMe-based all-flash clusters.

Core Concepts

• Hyperconverged Infrastructure (HCI)
• Distributed Storage Architecture
• Software-Defined Storage
• Storage Policy-Based Management (SPBM)
• Original Storage Architecture (OSA) vs Express Storage Architecture (ESA)
• vSAN Max (disaggregated storage cluster)

Key Components

1. vSAN Cluster

   • Minimum 3 hosts (2-node with witness supported for ROBO/Edge)
   • ESXi hosts with local NVMe/SSD/HDD devices
   • vSAN ESA requires all-NVMe certified hardware (vSAN ReadyNode for ESA)

2. Storage Pool / Disk Groups

   • OSA: Cache tier (SSD/NVMe) + Capacity tier (HDD/SSD), up to 5 disk groups per host
   • ESA: Single flat storage pool of NVMe devices — no cache/capacity split

3. Storage Policies (SPBM)

   • Define availability, performance, and capacity characteristics
   • Auto-Policy Management in ESA tunes policies to cluster topology
   • RAID-5/6 with mirrored performance leg in ESA (efficiency of EC, performance of mirroring)

Installation Prerequisites

• vSphere 7.0 U3 or later (vSphere 8.0 U3+ recommended; vSphere 9.0 supported)
• vSAN 8.x for ESA; vSAN ESA requires certified ReadyNodes
• ESXi hosts on the vSAN HCL / VCG
• Minimum hardware (per host):
  • 32 GB RAM (more for ESA / large clusters)
  • Supported NVMe/SSD/HDD configurations per architecture
  • 25 GbE network recommended for ESA (10 GbE minimum for OSA all-flash)
  • Dedicated vSAN VMkernel adapter

Configuration Workflow

1
2
3
4
5
6
7
8
9
# vSAN Deployment Steps
1. Validate hardware against the VMware Compatibility Guide (HCL)
2. Configure vSAN VMkernel networking (vDS recommended, NIOC enabled)
3. Enable vSAN on the vSphere cluster (choose OSA or ESA)
4. Claim disks (disk groups for OSA / storage pool for ESA)
5. Assign / customize SPBM storage policies
6. Deploy workloads to the vSAN datastore
7. Monitor with Skyline Health and vSAN performance service

Key CLI Commands

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# vSAN management (ESXi shell)
esxcli vsan network list
esxcli vsan storage list
esxcli vsan cluster get
esxcli vsan health cluster list
esxcli vsan debug disk summary

# ESA-specific
esxcli vsan storagepool list
esxcli vsan storagepool device list

# RVC (Ruby vSphere Console) for cluster-wide ops
vsan.cluster_info
vsan.disks_stats
vsan.resync_dashboard
vsan.health.health_summary

# PowerCLI
Get-VsanClusterConfiguration
Get-VsanDisk
Get-VsanSpaceUsage
Test-VsanClusterHealth

Storage Policies

• Failure Tolerance Method (FTT / FTM)
  • RAID-0 (no redundancy)
  • RAID-1 (mirroring)
  • RAID-5/6 erasure coding
  • ESA RAID-5/6 with adaptive write path — full performance without all-flash penalty
• Performance & Capacity Rules
  • Number of failures to tolerate (PFTT/SFTT for stretched clusters)
  • Object Space Reservation
  • Number of Disk Stripes per Object
  • IOPS limit per object
  • Checksum and data-at-rest/in-transit encryption flags
• Auto-Policy Management (ESA): cluster recommends optimal default policy

Security Best Practices

1. Enable data-at-rest encryption (DARE) and/or data-in-transit encryption
2. Use a supported Native Key Provider (vSphere Native Key Provider) or external KMS (KMIP)
3. Isolate vSAN traffic on a dedicated VLAN; enable jumbo frames (MTU 9000) end to end
4. Apply lockdown mode and role-based access control (RBAC) via vCenter
5. Patch ESXi, vCenter, and firmware/drivers to versions on the VCG
6. Monitor with Skyline Health and vSAN Health Service

Troubleshooting Techniques

• Run Skyline Health checks in vCenter (replaces the legacy vSAN Observer)
• Validate physical network and vmkping with the vSAN MTU
• Review vSAN Performance Service metrics (cluster, host, VM, backend)
• Check resync/rebalance progress with the Resync Objects view
• Verify storage policy compliance for all objects
• Use vsantop on ESXi for live I/O telemetry
• Collect support bundles via vm-support / vSAN support assistant

Performance Optimization

• Prefer vSAN ESA on all-NVMe ReadyNodes for new deployments
• Use 25/100 GbE networking with NIOC reservations for ESA
• Right-size storage policies (avoid over-mirroring; use RAID-5/6 where appropriate)
• Keep cluster slack space ≥ recommended threshold (Skyline Health flags this)
• Enable cluster-level compression (ESA compresses by default; deduplication is OSA-only)
• Distribute workloads evenly; allow proactive rebalance

Common Use Cases

1. Software-Defined Datacenter / Private Cloud (VCF)
   • vSAN is the principal storage for VMware Cloud Foundation
2. Remote and Branch Offices (ROBO) / Edge
   • 2-node clusters with a witness appliance
3. Virtual Desktop Infrastructure (VDI)
   • Predictable IOPS with all-flash / ESA
4. Cloud-Native and Modern Apps
   • vSAN Direct and CNS (Cloud Native Storage) for Kubernetes / Tanzu / vSphere with Tanzu
5. Disaggregated HCI
   • vSAN Max provides petabyte-scale shared vSAN storage to remote compute clusters

Licensing

Following the Broadcom acquisition, vSAN is primarily consumed through subscription bundles:

• VMware Cloud Foundation (VCF) — includes vSAN entitlement (TiB per core)
• VMware vSphere Foundation (VVF) — includes a base vSAN TiB allotment; add-on capacity available
• vSAN add-on — additional per-TiB capacity for VVF/VCF
• vSAN Max — separately licensed per TiB for disaggregated storage clusters

Legacy perpetual editions (vSAN Standard / Advanced / Enterprise / Enterprise Plus) are no longer sold for new deployments.

Scaling Strategies

• Scale-out: add hosts to grow capacity and performance linearly
• Scale-up: add NVMe devices to existing hosts (storage pool in ESA, disk groups in OSA)
• Non-disruptive upgrades via vSphere Lifecycle Manager (vLCM) cluster images
• OSA → ESA migration via rolling host evacuation and reclaim (no in-place conversion)
• Stretched clusters across two sites with a witness for site-level HA

Resource Recommendations

• Official VMware (Broadcom) vSAN documentation portal
• vSAN Design and Sizing Guide (current edition)
• vSAN ESA Architecture and Deployment Guide
• VMware Compatibility Guide (VCG) for vSAN ReadyNodes
• Broadcom Support Knowledge Base and community forums

Recommended Learning Path

1. Virtualization and vSphere fundamentals
2. Storage architecture and SPBM concepts
3. vSAN OSA vs ESA deep dive
4. Hands-on labs (VMware Hands-on Labs / HoL)
5. Certification: VCP-VCF and VCAP-VCF Deploy/Design tracks (current Broadcom certification path)