“Can my data be recovered using file recovery software if ..” , or, “can I recover my data after ..” is the most common type of pre-sale question I get. In this article I focus on Windows file systems (FAT(32) and NTFS).
Note: In this article we will assume the disk is physically intact: It spins up, the system detects the disk and it does not produce unusual noises. If in doubt, check the disk with a S.M.A.R.T. diagnostic utility.
Whether data can be recovered using file recovery software depends upon what exactly is wrong inside a volume. It also depends on the file system format (FAT, NTFS etc.) on the original volume. If the data loss was preceded by a specific action that can explain the data loss, the answer can be straight forward. Or not.
In general a chain of disk structures eventually points to your data (files):
Partition Records – created when partitioning a disk
Points to specific areas on the disk. Stores start location and size (in sectors).
If your hard disk is in one big chunk, it means there is one partition.
Can data be recovered if damaged: yes
Boot Record or Sector – created when formatting a partition
Stores vital information about the size of the file system and cluster size.
Also points to vital file system structures within the volume.
Can data be recovered if damaged: yes
File System Structures – created when formatting a partition
In case of FAT: FAT (file allocation table), Root Directory -> individual directories
In case of NTFS: MFT (Master File Table), Indexes
Can data be recovered if damaged: depends
Individual Files
Can data be recovered if deleted: depends
Conventional hard disk drive (HDD) or Solid State Drive (SSD)
Data can be recovered because the assumption is that pointers to data are removed or corrupt while the actual data is still present. On a HDD, in general, this true in a lot of situations. However, on SSD drives data may actually be removed after certain operations. If data is deleted the operating system (such as Windows) notifies the SSD. The SSD in turn ‘cleans up’ the area where the data was stored after which it can no longer be recovered. This clean up is required because of the nature of how a SSD works.
So, when you delete a file, or multiple files, or format a volume, the data may be really gone from the SSD. On a conventional hard disk this data probably would have been recoverable.
However, in case of corruption of file system structures the situation for SSD and HDD is the same. If for example a drive suddenly disappears or the file system changes from NTFS to RAW due to unknown reasons, Windows will not notify the SSD of a more or less planned deletion.
File Recovery software making sense of a disk
The key to recovering intact data is ‘the volume’ or ‘logical drive’. Disks are partitioned. Each partition can contain a file system which is created when a partition is formatted. A Partition formatted with a file system is what I call a volume or logical drive. A volume typically contains structures to keep track of files and folders and their properties such as size, creation date and location It is within this file system that the data is organized. To address specific locations within the volume, it is divided into clusters. A cluster is typically one or more sectors in size.
We can see that the earlier in the chain the damage occurs, the more dramatic the data loss appears. Corruption in the partition records will make the entire physical disk to appear empty. Where corruption in an individual file record only has consequences for one file.
File recovery software scans for the structures where file names and properties are stored. To determine the location of the actual file data it has to work out the start of the volume and the cluster size. It can then use the information in file system structures, the file records, to work out the exact location of the actual file data on the disk.
How file recovery software sees data on a disk
I’ll post a few screenshots of a disk viewer. In each screenshot, on the left size the raw data is shown. On the right side, the software interprets this raw data as a file system structure. Most of these structures are recognizable by what we call a signature or magic bytes. It is these bytes file recovery software will be looking for while scanning a disk.
First up: The partition table containing 4 partition records.
Apart from that the MBR contains executable code used to boot the system.
screenshot 1.
One of the partition records points to this sector. It is a NTFS boot record
or boot sector. A boot record contains important data about the
file system and if this data is missing or corrupt, Disk Management
will display a RAW file system. Apart from that the boot record
contains executable code used to boot the system.
screenshot 2.
The boot record contains the value for the first cluster of the MFT and
also a value for ‘sectors per cluster‘. Both are needed to
find the MFT. We can also see the MFT is self describing: It
contains ‘runlists’ which determine the exact clusters allocated
to the MFT. As far as the MFT is concerned, the MFT is a
file like any other file.
screenshot 3.
The boot record also describes the size for one MFT or File Record.
With this information we can work our way through the MFT
and compile a list of all files.
screenshot 4.
Examples of common user mistakes leading to data loss
Can data be recovered after deleting a file
If you delete a file on a FAT(32) volume, the filename’s first character is replaced by HEX value E5h and the FAT entries for the file are set to zero (available). Often we can regenerate the file name using the entry for the file’s long file name and we can determine the start cluster. We can not use the FAT to determine following clusters so fragmented files will most likely be corrupt.
On NTFS the file record for the file is only ‘flagged’ not in use while for the rest it remains untouched. As long as we have the record and the clusters that store the file’s actual data are not used again, we can recover intact files. See screenshot 4.
For both FAT and NTFS goes that the system can use the file system structures and their clusters to store new data. And once that happens we can no longer recover the file. When you need to recover a deleted file, do it as quickly as possible. Don’t waste time trying different undelete programs!
Can data be recovered after (Quick) formatting a volume by mistake
We can determine from the above that during formatting certain file system structures are created. What we also know is that over time a file system tends to become fragmented. This goes for both FAT based and NTFS file systems.
On an accidentally formatted FAT based file system fragmented files can’t be recovered. This is due to the file allocation tables are ‘zeroed’ during the format operation. The FAT tracks which exact clusters are allocated to any specific file.
Formatting also creates a boot sector and a root directory. For recovery we have individual directories at our disposal. If the original file system was FAT(32), we can recover the majority of the files. However we can not guarantee their integrity as we do not know the degree of fragmentation in advance.
On NTFS however the MFT that keep tracks of clusters allocated to a file is largely kept intact. So in case the original volume was formatted was NTFS, we can say with a high degree of certainty that the data we recover will be intact.
Can data be recovered after a partition was accidentally deleted
Partitions are recorded outside the actual file system(s) on a disk. A partition record typically stores the start, size and file system of a partition (or logical volume once the partition is formatted). Deleting a partition zeroes this record which means the entire file system remains intact whether it is a FAT type file system or NTFS.
Partition deleted and then re-created in an attempt to undo damage done
Depending somewhat on the software you use to recreate the partition, the file system remains largely intact. Some of them may write sector size byte patterns to the partition at regular intervals that can corrupt a small portion of the file system structures and data. File structures and data overwritten by this pattern will be slightly corrupt.
Can data be recovered after incorrectly disconnecting an external disk or memory card
It is very common cause for a RAW file system. It basically means that the OS can not identify the file system. Boot record corruption will cause it. But I also see RAW file systems while the boot record contains valid information. If damage is limits itself to the boot record, it’s almost 100% certain that all data will be intact.
Possible (error) messages are:
You need to format the disk in drive before you can use it. Do you want to format it?
This volume does not contain a recognized file system. Please make sure that all required file system drivers are loaded and that the volume is not corrupted.
chkdsk can not be run on the drive
It is however impossible to guarantee anything as it is unknown what structures and to what extend they are corrupt. If the system can not determine the file system because 60% of the MFT was deleted then the majority of the data can not be saved.
This type of recovery also highly depends on the quality of the software you use! For example, without a valid boot record the software has to guess the cluster size. This is vital: assume we find for that for a particular file the start cluster is 1000. The number of sectors per cluster (a cluster is the unit a file system uses to determine the location of a file) determines where we have to look for the file. If the software gets this value wrong it may present you an intact looking directory tree, while all files will be corrupt after saving them.
A deleted RAW partition
I see cases like this people, I am not making this up. As we have seen deleting a partition in itself isn’t catastrophic. A RAW file system can be but doesn’t have to. However, it is in these more complex scenarios that many file recovery tools will prove to be useless. Simply put, as they rely solely on a boot sector to determine the start of a partition (apart from cluster size you will need that reference), which they fail to find, the can not do a file system recovery. So, in itself no reason to not try, but you will have to use the right software which I will come to later.
Tables – Data Recover-ability.
The below tables attempt to give an overview of the effect that damage to certain structures has on recover-ability. Note however that these are conservative estimations. We will assume that the file recovery software is able to determine the correct cluster size and the start of the volume. If not, the tables become entirely invalid.
Table 1 – FAT based file systems
| (damaged) FAT file system structures: | Prognosis | |||
|---|---|---|---|---|
| Partition table | Boot record | FAT | Root directory | |
| OK | OK | OK | OK | Good |
| OK | OK | OK | Damaged | Good |
| OK | OK | Damaged | Irrelevant | Fair* |
| OK | Damaged | OK | Irrelevant | Good |
| OK | Damaged | Damaged | Irrelevant | Fair* |
| Damaged | OK | OK | OK | Good |
| Damaged | OK | Damaged | Irrelevant | Fair* |
| Damaged | Damaged | Damaged | Irrelevant | Poor |
From table we can tell that the condition of the FAT is the most determining factor. In short, using directory entries we can determine file names and start clusters for files. Without an intact FAT we need to assume that the file is not fragmented as we have no way of determining the exact cluster chain. If it actually is, the file we rescue will be corrupt.
Table 2 – NTFS
| (damaged) NTFS file system structures | Prognosis | ||
|---|---|---|---|
| Partition table | Boot record | MFT | |
| OK | OK | OK | Good |
| OK | OK | Damaged | Fair* |
| OK | Damaged | OK | Good |
| OK | Damaged | Damaged | Fair* |
| Damaged | OK | OK | Good |
| Damaged | OK | Damaged | Fair* |
| Damaged | Damaged | OK | Good |
| Damaged | Damaged | Damaged | Fair* |
For NTFS the MFT is the most important file system structure. A few corrupt MFT entries mean we will not be able to recover a couple of files. If 5 out of 10 MFT records is corrupt then it becomes a different story.
Conclusion – Can data be recovered after …
Although it is possible to make fairly certain estimates whether data that was lost due to know mistakes can be saved or not, there will always be a degree of uncertainty. Most data recovery software will or should be able to deal with a deleted partition or a reformatted disk.
It’s the more complex scenarios where you may have a harder time finding a tool to be successful. Many will run into a problem when for example a FAT32 volume was later formatted NTFS. They will fail to understand that they will have to utilize their FAT procedures.
Also, combined damage, such as a RAW drive that was later deleted proves to be too difficult for most software. The below examples, were able to recover from more complex scenarios.