Repair and recovery of digital photos

By | November 29, 2020

[automatically translated. Original language Dutch]

DiskTuna has extensive experience with generic data recovery but specializes in recovering photos from small flash-based data carriers (SD cards, USB thumb drives, etc.). DiskTuna is also one of the few (worldwide!) That is able to repair corrupt digital photos. You can read all about it on this page.

You can think of a few things about photo recovery:

  • Restoration of old analogue photos
  • Recovering deleted digital photos
  • Repairing corrupt digital photos

DiskTuna only deals with the second and third options: Recovering deleted photos and recovering corrupted digital photos. DiskTuna is limited to data recovery from small flash-based data carriers (memory cards and USB flash drives).

Recovery of digital photos

Recover deleted photos

When recovering photos, it is important to distinguish between different causes of data loss:

  • Physical cause, or damage to and wear and tear on the data carrier
  • Logical cause, or corruption in file system metadata

If data loss is due to a physical cause, ‘do it yourself’ is often unwise and impossible. But even with a logical cause, errors are lurking, so be careful.

Physical cause

You can think of something as simple as a broken USB plug or a blown fuse, but also the NAND chip itself. Although it is generally assumed that flash drives are very robust, there are things that can and will go wrong. An advantage with regard to repairing damage is that, in contrast to repairing classic disks, no so-called clean-room is required.

No cleanroom is required for repairing flash-based disks

No cleanroom is required for repair of flash based disks: Requirements are multimeter, microscope, solder station, re-work station, NAND reader.

Broken components, blown fuses, etc.

Things like a broken USB plug can usually be repaired, but the problem is often that components are very small. Requires precision, experience and a microscope. Do it yourself attempts often lead to damage. Repair of so-called monoliths is excluded. Monoliths are chips on which all components are located in a chip.

Bad sectors, freezing USB drives and SD cards…

DeepSpar data recovery hardware

Like conventional disks, flash-based “disks” can develop bad sectors. The problem with small flash drives is that they are almost always addressed via USB and that USB is very bad at handling errors. This complicates the recovery of data from such drives and can sometimes only be done with professional data recovery hardware. I use hardware from the Canadian firm DeepSpar and a self-developed hardware / software combination for these cases (see video).

Controller broken

While it is common with normal hard disks to exchange broken components with parts of a working disk, this is often not possible with flash drives. Not because this is fundamentally impossible, but because it is almost always impossible to find a donor. Often the only option is chip-off or directly dumping NAND memory using a NAND reader and specialized software. This software must then imitate the task of the controller. Given the wide variety of controllers and firmware revisions, this is sometimes impossible.

Worn and corrupt NAND memory

It may sound strange but NAND memory ‘wears out’ and certainly does not have eternal life. I often use the analogy with a battery: Just like a battery, NAND can lose memory data just like a battery loses its charge. It is also the case that with repeated use the capacity to retain charge (data) decreases. At best, when this occurs, it will affect individual files. Depending on the firmware, many flash drives (USB sticks and memory cards) switch to a read-only mode when the maximum number of erase / write cycles has been reached. In principle, you can still access the data, but you cannot save new files.

However, it also happens that file system metadata (usually the file allocation table) gets corrupted or, in the worst case, the NAND’s ‘translation table’ itself. The latter can also be damaged when the flash drive abruptly loses contact with the connected device. As a result, the controller cannot update the translation tables within the NAND. The next time the firmware hangs or switches in a kind of safe mode: An ID of the drive can be determined but the capacity is 0 or only a few dozen MB’s.

When this translation table is corrupted, only recovery with a special reader and specialized software is still possible. The same problems apply as in the previous point (controller broken). The raw NAND dump will have to be transformed into a logical entity from where files can be rescued by means of file recovery software.

This task is normally up to the controller and includes de-scrambling, ECC correction, RR correction, and the correct linking of data blocks (which are actually randomly distributed for wear-leveling and bad block management). On top of that, the fact is that a chip can consist of multiple parts that are managed in a RAID-like setup. With multiple chips this becomes an even more complicated RAID!

Logical cause

As with any disk, the file system can ‘lose sight of a file’. This could be by deleting a file, formatting a disk or by corruption of the file system itself.

Since we limit ourselves to small flash-based data carriers, we are almost automatically talking about the FAT file system or some variant thereof; FAT32 and exFAT (FAT64). The central and most important structure is the file allocation table (FAT) as it is the only way to link individual clusters to a file.

Unfortunately, many actions such as deleting photos or formatting the SD card, for example, will ‘reset’ this FAT. When this happens, it is really only possible to recover files that have not been fragmented as long as the file data has not been overwritten.

DiskTuna can often recover fragmented photos, either manually or with software developed by DiskTuna together with a Russian developer. Even files returned corrupted by reputable labs! A requirement is that such a lab makes the logical drive image of the data carrier available (not the NAND dump). Even if you tried to retrieve photos yourself but only find corrupt files, it may be worth sending the SD card to me. If I don’t find anything, you don’t pay anything.

I can often recover fragmented videos intact (MP4 + MOV).

Repairing corrupt digital photos

DiskTuna is one of the few in the world that ventures to recover corrupt digital photos. This has to do with the complexity of the subject and the fact that we are talking about a small market. Repairing digital photos is labor-intensive and expensive, and few are willing to pay for it. In an extreme case, repairing a single photo could be as expensive as recovering all photos and videos from an entire SD card.

Damage to an individual file can include:

  1. file header damage
  2. Damage to the payload, the actual image data
  3. a combination

For examples I assume a JPEG file, still the most commonly used format for digital photography.

Damage to the file header

The header is the key to the image information. It contains pointers to file structures and data, as well as important tables required for decoding image data. The information in these tables depends on the camera, photo resolution and photo quality. It also includes tables necessary to extract compressed image data. DiskTuna will always ask for an intact photo taken with the same camera with identical settings as much as possible.

Damaged image data

This is the most labor-intensive to recover. As mentioned, image data is compressed and therefore there is no one-to-one relationship between the raw data as we find it in a file and the image. For example, a blue sky contains a lot of data that can be compressed well. Assume that this blue sky takes up 40% of the photo, because of compression it could be that the raw data only makes up 25% of the file. That means that the pixels at 40% within the photo do not correlate with the raw data that we find 40% within the file.

Different stages of repair of a photo with bit errors

Different stages of repair of a photo with bit errors

Furthermore, each piece of information builds on the foregoing, meaning that a single corrupt bit affects all of the data that follows. AI aside, it takes a human to assess the effect of corruption on the image. Corrupt raw image data is ‘patched’; deleted. The consequences of each patch action must be assessed, this is time consuming and labor intensive, even using the tools that DiskTuna has developed.

A combination, damage to header and image data

It is of course possible that damage to the file is not limited to the header. It must then be examined per file whether repair is possible. A ‘spectacular’ example are photos that are the victims of the DJVU ransomware: The entire header and part of the image data are usually encrypted. With an intact reference file, DiskTuna can partially repair this kind of damage!

Where does the corruption actually come from?

This is an important question. Corruption in the file system or even the underlying layer (the NAND memory) will often show itself as individual corrupt photos, while repairing in such a case makes no sense. If the FAT for example, does not refer to the appropriate clusters while the directory entry for the file is intact, looks the picture corrupt. In reality, the image data may reside on the data carrier but the correct reference is missing. Experience is involved in assessing this situation. In many cases where I am asked to repair one or more corrupt photos, this is impossible while recovery is possible.

All photos appear corrupt. Actual cause is damage to the FAT

All photos appear corrupt. Actual cause is damage to the FAT

To repair a file with internal damage we only need the file, where the damage is due to file system damage, we work with the entire data carrier or an image file thereof.

Leave a Reply

Your email address will not be published. Required fields are marked *