In this blogpost we’ll look at:
- A good JPEG as point of reference
- JPEG header corruption and how to repair
- Header + part of image data damaged and repair
- Corruption in JPEG image data resulting in visual anomalies
- JPEGs beyond repair that no one likes to hear about
First things first, what is repair and what is recovery?
Corrupt JPEG file Repair and JPEG File Recovery are often mixed up.
- Recovery: Meaning getting lost or deleted files back from for example a memory card. Maybe the card itself (the file system) is corrupt or the card was accidentally formatted. Key is you do not see the actual files listed anywhere.
- Repair: Repair existing JPEG files that can not be opened in image viewing or image editing software, or that appear to be corrupt when viewing them (colors off, shifted pixels etc.). You can see the file in the Windows Explorer but there is something wrong with the file itself.
We speak of recovery when you need to get access to files that are no longer visible in the file system. They were deleted, wiped from a formatted drive or otherwise lost (RAW file system for example).
For these situations you typically use Photo or Digital Image Recovery software. This is ‘specialized’ software. But you might as well use standard file recovery or undelete software. Photo Recovery tools are often (but not always) either ‘dumbed-down’ generic file recovery tools or RAW scanners. The latter are also known as ‘carvers’ or ‘file carving software’.
By ‘dumbed down generic file recovery or undelete tools’ I mean that it is a generic file recovery tool, however the manufacturer simply limits the software so it only recovers certain file types, such as digital images. A RAW scanner can have it’s uses, specially in case of photo recovery if the file system is too corrupt or if the file system is unreliable due to fox example a corrupt file allocation table.
It is confusing that many companies selling recovery software will use terms like ‘recover corrupt JPEG files’, suggesting that their software can somehow fix corrupt JPEG and other files. In general a corrupt file is still corrupt after recovery. Imagine you’re car is getting stolen, it needs to be recovered. If the car is damaged due to a traffic accident it needs repairs.
First, a warning. You may have found this page while searching for a solution to repair corrupt JPEG files. There are many pages that address this issue, but often they offer incorrect or plain dumb advice that may even damage the photos beyond repair. In my opinion any page or guide disqualifies itself if it suggests things like running chkdsk /f, or SFC (system file checker) or reinstalling drivers to repair corrupt JPEGs. Avoid such pages.
Half the work is diagnosing the issue. I’ll go over all common scenarios. We’ll see if we can diagnose the issue and if we can repair the JPEG.
A good JPEG
A good JPEG is when header and actual image data is intact. A JPEG can be divided in a header + the payload, so the encoded and compressed image data. Some argue there is no JPEG header as the JPEG is basically a linked list of sections all contributing and all important to decode the image. Each section starts with a JPEG marker.
Depending on where damage occurs we may observe different issues with the file.
JPEG header corruption.
As markers preceding the image data are a relatively small portion of the file, it is common that in case of corruption the entire section is damaged. Windows Photo Viewer will not open the file at all and instead show an error (‘It looks like we don’t support this file format‘).
Easiest way to repair this is by using a ‘reference file’. A reference file is another JPEG shot by the same camera with settings matching as closely as possible (resolution and quality). You can do this using a hex editor: You basically copy image data + EOI marker from corrupt file and overwrite image data + EOI marker of the reference. Or take everything from SOI marker up to SOS marker from reference file and paste that over this section in corrupt file.
No SOS detected
If you tried header repair and JPEG-Repair reports no SOS reported then damage extends into the image data.
We see this often in JPEG damaged by ransomware but it can happen for a number of other reasons too. An example is incorrectly recovered photos as demonstrated in the video below. These files may still be repairable but repair is done file by file using the JPEG-Repair Patch tool. It too requires a reference file. In short we glue header onto the corrupt file and then weed corrupt data. Due to how JPEG is encoded we then need to realign the image and correct color.
Visual corruption: Corruption in JPEG image data
Corruption in image data is more common than one might think. There’s a good chance that if you go through the JPEGs on your hard drive, a few of them can be opened but look distorted. JPEGs are prone to silent corruption. Another common cause for this type of issue is errors during transfer (from memory card to PC for example), bad media or other hardware issues.
We can open such files but immediately spot something isn’t right. Due to how JPEGs are encoded a single bit flipped bit can cause visual corruption but often it is larger areas that are corrupt. It can also cause what’s commonly known as ‘grey boxes’: Image decodes up to a certain point, the rest of the image is grey (or another solid color). In this case corruption coincidentally cause a byte value into something the JPEG standard considers a JPEG marker. The image viewer will stop decoding image data at that point.
JPEGs are divided in blocks of pixels, often 8 x 8 pixels. We call a block an MCU (Minimum Coded Unit). To reduce data that needs to be stored, certain values that are kept for an MCU are deltas: They do not store the actual value but the difference relative to the previous MCU. result is that is corruption affects brightness in one MCU, all blocks following are affected. In this video we see result of 2 corrupt areas and how you can use JPEG-Repair to fix the JPEG:
JPEGs beyond repair
Can all JPEGs be repaired? Unfortunately the answer to that is no. From the above we can tell we can repair files that contain actual image data. As long as we have a reference file we can repair the file. Visual corruption can be repaired up to a degree. If we have corruption throughout the file, we can remove that and if these areas are small we can repair them using for example a content aware filling technique.
For example we start with:
This case is somewhat what is maximally achievable (click for large):
Image 1 is result of the ‘technical repair’: corrupt data was removed, image data realigned and colors repaired. This was all done using JPEG-Repair. Image 2 is result of technical repair and using thumbnail data to fill blank areas. Image 3 is the somewhat more creative solution using photo editing techniques such as clone-stamping. No matter the method, the quality of the image suffers.
Of course if all image data is lost, nothing can be repaired or rescued. We often see these JPEGs are completely filled with zeros or another repeating byte pattern.