DiskTuna VSS Safe-mode
DiskTuna features a ‘VSS safe mode’. But that’s just a name that I made up. Many defraggers offer a ‘VSS aware’ or ‘VSS compatible’ option. VSS-aware defragmentation will always make defragmentation and optimization less effective. But what is it? What does it do? What is VSS anyway?
Windows Volume Snapshot Service (VSS)
VSS is short for ‘Volume Shadow copy Service’. It is also referred to as ‘Volume Snapshot Service’. It is a Windows service that requires NTFS, and it provides features to Windows or other applications. They can use the service to for example create consistent backups, disk images and ‘restore points’. If you’d normally create a disk image from a volume that Windows and other applications are reading from and writing to, so some files are constantly changing, then there is a good chance a disk image will be inconsistent.
Another example of what can be done with the service is the ‘previous versions’ feature in Windows that allows you to go back to a previous version of a file (even after it was deleted) and the Windows system restore points that allow you to go back to a certain point in time after for example the installation of new software messed up Windows.
Uh oh. No previous versions for this folder
So, VSS keeps track of changes to a NTFS volume at the block level and records those changes. But what does this have to do with ‘defragging’?
VSS and defragmenting a volume
In Microsoft’s own words:
The System Shadow Copy provider uses a copy-on-write mechanism that operates at a 16-KB block level. This is independent of the file system’s cluster allocation unit size. If the file system’s cluster size is smaller than 16 KB, the System Shadow Copy provider cannot easily determine that disk defragmentation I/O is different from typical write I/O, and performs a copy-on-write operation. This might cause the Shadow Copy storage area to grow very quickly. If the storage area reaches its user-defined limit, the oldest shadow copies are deleted first.
In short a file move done by defragmentation software triggers VSS to add the changes to the file system to a snapshot. If defragmentation software moves a lot of files, and large files, this causes snapshots to grow. As there is a limit to the space that can be used for snapshots, VSS will start deleting older snapshots in order to make room for the newer, growing snapshots. You can watch those deletions in the Windows event viewer.
Notification of a ‘snapshot’ or shadow copy being deleted
To circumvent those snapshot deletions, defragmentation software authors had to come up with something, as those snapshots growing, and old snapshots being deleted is an undesired side effect of defragmentation.
Circumventing shadow copy deletion in defragmentation software
Defragmentation software authors didn’t only come up with all kinds of different marketing terms, they also came up with different solutions to the problem of the growing and as a result of that, deleted snapshots. Trying different defraggers I discovered a few, and I can come up with some more on my own:
- Disable VSS. I’m not sure this is wise. Other software may be relying on VSS to do for example continuous backups. Disable VSS is what Defraggler does I think. It does not sound like a real solution to me.
- Skip files that aren’t that fragmented. So, accepts a few fragments per file. It’s arbitrary really, let’s defrag only if the file is more than n fragments. Make n whatever you want.
- Skip files larger than nn MB. I think the Windows built-in defragger does this (64 MB if I remember correctly). And a few others as well. The less you move, the less ends up being ‘snap-shotted’.
But wait a minute! Microsoft recognizes the problem, and they give some sort of work around! So, why not just do that? We do. This is exactly how DiskTuna handles moving files when VSS safe mode is enabled:
Minimizing interactions between defragmentation and shadow copies
When possible, move data in blocks aligned relative to each other in 16-kilobyte (KB) increments. This reduces copy-on-write overhead when shadow copies are enabled, because shadow copy space is increased and performance is reduced when the following conditions occur:
- The move request block size is less than or equal to 16 KB.
- The move delta is not in increments of 16 KB.
Move delta is the number of bytes between the start of the source block and the start of the target block. In other words, a block starting at offset X (on-disk) can be moved to a starting offset Y if the absolute value of X minus Y is an even multiple of 16 KB. So, assuming 4-KB clusters, a move from cluster 3 to cluster 27 will be optimized, but a move from cluster 18 to cluster 24 will not. Note that mod(3,4) = 3 = mod(27,4). Mod 4 is chosen because four clusters at 4 KB each is equivalent to 16 KB. Therefore, a volume formatted to a 16-KB cluster size will result in all move files being optimized.
Regarding VSS safe-mode: DiskTuna will not detect ‘shadow copies’ so you will have to explicitly select this option if you have shadow copies enabled. If this option is enabled DiskTuna will treat the selected volume as if it were formatted with a 16 KB cluster size. Microsoft recommends 16 KB clusters for volume on which you have shadow copies running and that will defragged (yes indeed, what were they thinking). This will reduce interaction between defragmentation and shadow copies not prevent it (according to Microsoft Developer Network documentation). This is not a limitation of Disktuna, but a result of the shadow copy inability to recognize typical defragmentation disk I/O. As a result shadow copies may perform a copy-on- write operation causing the shadow copy area to grow. If this area continues to grow beyond the user defined limit shadow copies may be deleted (oldest first).
To ‘simulate’ 16 KB clusters and DiskTuna will make sure the move-delta of a block of clusters is an increment of 16 KB and Disktuna will not move files <= 16 KB as the shadow copy will not be able to recognize it as typical defrag file I/O. Implication of this are that small files will not be de-fragmented. Also, if the move delta between the start cluster of the file to be moved and the start cluster of the free gap is not an increment of 16 KB Disktuna may decide to ‘shift’ a few clusters towards the end of the free gap so that the move delta does become an increment of 16 KB. As a result a small gap of free space is created which could have been avoided if we did not have to take shadow copies into account.