A new VAAI primitive, using the SCSI UNMAP command, enables an ESXi host to inform the storage array that
space can be reclaimed that previously had been occupied by a virtual machine that has been migrated to
another datastore or deleted. This enables an array to accurately report space consumption of a thin-provisioned
datastore and enables users to monitor and correctly forecast new storage requirements.
The mechanism for conducting a space reclamation operation has changed somewhat since the primitive was
introduced in vSphere 5.0. Then, the operation was automatic: When a virtual machine was migrated from a
datastore or deleted, the UNMAP was called immediately and space was reclaimed on the array. There were
some issues with this approach, however, primarily regarding performance and an array’s ability to reclaim the
space in an optimal time frame. For this reason, the UNMAP operation is now a manual process. Because it is the
only manually initiated primitive, the process to reclaim dead space will be discussed in detail.
To reclaim dead space from a thin-provisioned datastore in vSphere 5.1, first verify that the storage array is
capable of processing the SCSI UNMAP command. With esxcli, users can display device-specific details
regarding Thin Provisioning and VAAI.
# esxcli storage core device list –d naa.xxx
naa.xxx
Display Name: NETAPP Fibre Channel Disk (naa.xxx)
Has Settable Display Name: true
Size: 51200
Device Type: Direct-Access
Multipath Plugin: NMP
Devfs Path: /vmfs/devices/disks/naa.xxx
Vendor: NETAPP
Model: LUN
Revision: 8020
SCSI Level: 4
Is Pseudo: false
Status: on
Is RDM Capable: true
Is Local: false
Is Removable: false
Is SSD: false
Is Offline: false
Is Perennially Reserved: false
Thin Provisioning Status: yes
Attached Filters: VAAI_FILTER
VAAI Status: supported
Other UIDs: vml.xxx
The device is indeed thin provisioned and supports VAAI. Next, users can display the VAAI primitives supported
by the array for that device, as well as whether the array supports the UNMAP primitive for dead space
reclamation (referred to as the Delete Status). Another esxcli command is used for this step, as is shown in
the following:
# esxcli storage core device vaai status get -d naa.xxx
naa.xxx
VAAI Plugin Name: VMW_VAAIP_NETAPP
ATS Status: supported
Clone Status: supported
Zero Status: supported
Delete Status: supported
The device displays Delete Status: supported to indicate that it is capable of sending SCSI UNMAP commands
to the array when a space reclamation operation is requested.
If a Storage vMotion operation is initiated and a virtual machine is moved from a source datastore to a
destination datastore, the following actions will reclaim that space: When the Storage vMotion operation has
completed, the vSphere Client will report that the VMFS5 volume now has much more free space. The issue is
that when the amount of free space is checked on the thin-provisioned LUN backing this VMFS5 volume on the
storage array, there is unused and stranded space. In this example, a NetApp FAS array is providing the
back-end storage. Using a lun show CLI command on this NetApp array, we see in the following that 8.8GB of
space is still being consumed, even though there are now no virtual machines or any other files on this datastore.
lun show -v /vol/vol2/thin-lun
/vol/vol2/thin-lun 50g (53687091200) (r/w, online, mapped)
Serial#: xxx
Share: none
Space Reservation: disabled
Multiprotocol Type: vmware
Maps: unmap=51 issi=51
Occupied Size: 8.8g (9473908736)
Creation Time: Tue Apr 24 15:16:52 BST 2012
Cluster Shared Volume Information: 0x0
This is the crux of the issue that we are trying to solve with the VAAI UNMAP primitive. To run the UNMAP
command, change the directory to the root of the VMFS volume that you want to reclaim space from. The
following command is run:
vmkfstools –y <% of free space to unmap>
In this example, we attempted a reclamation of 60 percent of free space. The vmkfstools -y command displays
the following:
Attempting to reclaim 60% of free capacity 48.8 GB (29.3 GB) on VMFS-5 file system
‘source-datastore’ with max file size 64 TB.
Create file .vmfsBalloontsWt8w of size 29.3 GB to reclaim free blocks.
Done.
vmkfstools -y created a balloon file of 29.3GB, which is 60 percent of the free capacity (48.8GB). This
tempora****alloon file is equal to the size of the space to be unmapped/reclaimed.
NOTE: Caution – If a percentage value in the high 90s or 100 is specified, the tempora****alloon file that is
created during the reclamation operation might fill up the VMFS volume. Any growth of current VMDK files due
to running virtual machines writing to their disks or the creation of new files, such as snapshots, might fail due
to unavailable space. Care should be taken when calculating the amount of free space to reclaim.
Finally, query the status of the thin-provisioned LUN on the storage array. It should show a difference in
Occupied Size, as shown in the following:
lun show -v /vol/vol2/thin-lun
/vol/vol2/thin-lun 50g (53687091200) (r/w, online, mapped)
Serial#: xxx
Share: none
Space Reservation: disabled
Multiprotocol Type: vmware
Maps: unmap=51 issi=51
Occupied Size: 76.3m (79966208)
Creation Time: Tue Apr 24 15:16:52 BST 2012
Cluster Shared Volume Information: 0x0
The dead space successfully has been reclaimed for the thin-provisioned datastore; the array now can correctly
report the space usage.
Disabling the UNMAP primitive does not affect any of the other Thin Provisioning primitives such as
Thin Provisioning Stun and the space threshold warning. All primitives are orthogonal.
