Often I am asked to repair JPEGs that can be opened but don’t display correctly. Many of those look okay for some part, but turn into a grey area at some point. Repairing these files (or recovering them as we’ll see) isn’t always straight forward and sometimes requires some detective work.
Half grey photos
In general this means:
- part of the image data is missing or
- all data is there but part of image data is not processed by the photo viewer/editor due to corruption.
Part of image data missing.
Often seen after data recovery from corrupt memory cards due to file fragmentation. As this image data simply isn’t there it can not be repaired either.
In the case below however, the files were not the result of a failed recovery but of inconsistencies in the file system. As I will demonstrate, all file data was actually written to the memory card, but for some reason the file system contained erroneous information about the files.
Part of image data not processed.
The photo partially displaying means the header of the JPEG is intact. Almost always the footer is intact as well.
This is mostly caused by corruption in the image data. Depending on the amount of corrupt data, this can often be repaired by editing image data using utilities I developed and a hex editor. In short the process is:
- Get rid of invalid markers that stops the viewer from processing the entire image file. This results in the viewer rendering the entire, although corrupt picture.
- Get rid of corrupt data. This is a process of trial and error. Using the info from the first step (invalid marker data) I delete data from the image stream and view the results. I do this until I get a more or less normal picture, where colors look correct.
- Stuff bytes to account for deleted data. This is necessary to correctly align the picture.
- Use an inpaint tool and/or photo editor to clean up the picture.
The corrupt JPEG that wasn’t corrupt
Recently I received a bunch of images that were partially okay, partially grey. They were not the result of being recovered using file recovery software. I examined the files and none of my ‘tricks’ worked. Then I noticed something obvious that I initially missed. The files were so darned small. Also, they are all without a JPEG end marker (FF D9) in the hex editor.
When sorting the files by size it showed that the smaller the file, the larger the grey area. These files weren’t corrupt, they simple were incomplete. I can not repair that, no one can. However that does not mean all is lost yet.
Working with my customer, I had him make an image of the original memory card, and upload it. An image file is a sector by sector copy of the card. Thus it contains the entire file system and everything stored on it.
Not the picture is corrupt, the file system is …
I loaded the card image into a tool that can interpret file systems, FAT32 in this case. It confirmed the too small file sizes (according to the file system). So, for example a 4 MB (estimated by comparing against same resolution images) was only 2 MB or even smaller.
As I was able to extract previews from all files (JPEG) that all show the correct image, I assumed that at one point the sensor did capture all data, the full frame. I decided that our only chance was to see if we can discover the full images on the card without relying on the file system.
The method to do this is carving: Detect images based on their characteristics rather than using the file system. Initially I use simple header/footer carving as it is faster than advanced carving.
Success! For whatever reason, the full image was written to the card but incorrectly entered in the file system. This is encouraging enough to now process the SD Card image again using advanced carving (using JpegDigger) to retrieve as many files as possible.