Often I am asked to repair JPEGs that can be opened but don’t display correctly. Many of those look okay for some part, but turn into a grey area at some point. Repairing these files (or recovering them as we’ll see) isn’t always straight forward and sometimes requires some detective work.
Half grey photos
In general this means:
- part of the image data is missing or
- all data is there but part of image data is not processed by the photo viewer/editor due to corruption.
Part of image data missing.
Often seen after data recovery from corrupt memory cards due to file fragmentation. As this image data simply isn’t there it can not be repaired either.
In the case below however, the files were not the result of a failed recovery but of inconsistencies in the file system. As I will demonstrate, all file data was actually written to the memory card, but for some reason the file system contained erroneous information about the files.
Part of image data not processed.
The photo partially displaying means the header of the JPEG is intact. Almost always the footer is intact as well.
This is mostly caused by corruption in the image data. Depending on the amount of corrupt data, this can often be repaired by editing image data using utilities I developed and a hex editor. In short the process is:
- Get rid of invalid markers that stops the viewer from processing the entire image file. This results in the viewer rendering the entire, although corrupt picture.
- Extract JPEG preview data so I have an idea what the complete photo looks like (using JpegDigger).
- Get rid of corrupt data. This is a process of trial and error. Using the info from the first step (invalid marker data) I delete data from the image stream and view the results. I do this until I get a more or less normal picture, where colors look correct.
- Stuff bytes to account for deleted data. This is necessary to correctly align the picture.
- Use an inpaint tool and photo editor to clean up the picture.
The corrupt JPEG that wasn’t corrupt
Recently I received a bunch of images that were partially okay, partially grey. They were not the result of being recovered using file recovery software. I examined the files and none of my ‘tricks’ worked. Then I noticed something obvious that I initially missed. The files were so darned small. Also, they are all without a JPEG end marker (FF D9) in the hex editor.
When sorting the files by size it showed that the smaller the file, the larger the grey area. These files weren’t corrupt, they simple were incomplete. I can not repair that, no one can. However that does not mean all is lost yet.
Working with my customer, I had him make an image of the original memory card, and upload it. An image file is a sector by sector copy of the card. Thus it contains the entire file system and everything stored on it.
Not the picture is corrupt, the file system is …
I loaded the card image into a tool that can interpret file systems, FAT32 in this case. It confirmed the too small file sizes (according to the file system). So, for example a 4 MB (estimated by comparing against same resolution images) was only 2 MB or even smaller.
As I was able to extract previews from all files (JPEG) that all show the correct image, I assumed that at one point the sensor did capture all data, the full frame. I decided that our only chance was to see if we can discover the full images on the card without relying on the file system.
The method to do this is carving: Detect images based on their characteristics rather than using the file system. Initially I use simple header/footer carving as it is faster than advanced carving.
Success! For whatever reason, the full image was written to the card but incorrectly entered in the file system. This is encouraging enough to now process the SD Card image again using advanced carving (using JPEG Recovery LAB) to retrieve as many files as possible.
JPEG Recovery LAB gets the complete picture!
Recovery LAB is a unique tool for fragmented photo and video recovery from memory cards. $39.95
- It recovers fragmented files using “advanced carving”
- Gets complete photos where other get half
- Photo formats supported: JPEG, CR2 (Canon raw), NEF (Nikon raw)
- Video formats: MP4, MOV, AVI, MPEG-2(AVCHD, MTS)
- Free to try with real previews, WYSIWYG!
- Designed for memory cards.
Before downloading and using JPEG Recovery LAB consider the following:
- JPEG Recovery LAB does not process individual photos or videos! In other words, if ‘myimage.jpg’ is corrupt then there is no use in scanning this file only. Instead try recovering the file again from the original memory card.
- If you need to recover digital photos from a NTFS formatted hard drive, you may first want to try a generic file recovery software like ReclaiMe. File fragmentation is less of a issue when recovering files from NTFS unlike FAT32 or exFAT formatted memory cards.
- Be prepared that it will take a long time. JPEG Recovery LAB will grab all available resources leaving very little for you. It’s best to leave the PC untouched for a few hours and let JPEG Recovery LAB do its thing. If you need the PC for work, limit the number of cores JPEG Recovery LAB can use (settings).
- JPEG Recovery LAB requires and internet connection to communicate with our server. More information on this.
- Also, by using the software you indicate to agree with the license agreement.
- Requires 64 bit Windows 7 or better, 64 bit with .NET 4.5. NET 4.5 is normally installed on Windows 8 or better. For Windows 7 you can download it here.
- Minimum amount of memory installed, 8 GB. Hint: If you barely meet system requirements memory wise, consider limiting the number of cores (number of parallel threads) to 2 for example.
- When looking for deleted files only tick ‘Only search unallocated space’. By deleted, I mean files that you selected and then deleted using Windows or the camera itself.
More information on JPEG Recovery LAB.