JpegDigger user manual

Make sure you always use the latest version!

JpegDigger JPEG Carver

JpegDigger is a light-weight, specialistic tool that can help you recover deleted or otherwise lost photos. It was designed with memory cards in mind, but works on hard drives too.

Virus Total: No engines detected this file

Version: 2.4.44

 

JpegDigger is fairly simple and easy to use.

  1. Select a drive – it needs to have a drive letter assigned
  2. Select a destination drive – make sure this is a different drive than the source drive! You can skip this and JpegDigger will prompt you for a destination drive as soon as you ask it to save a file.
  3. Set maximum file size – Check typical size of a JPEG produced by your camera
  4. Set some options – see example below
  5. Click Scan 

With Autosave set JpegDigger will now scan the selected media and recover all JPEGs. File names are automatically generated based on the LBA sector number on which they were found: ‘Sector.JPG’. This will work for recovering deleted files from a healthy memory card or drive or re-formatted cards (formatted with same device). Use overrides if no files are detected on a corrupt card.

This video shows a scenario where a SD Card originally formatted in-camera was accidentally formatted in Windows:

Selecting a drive or disk image

JpegDigger list all drives Windows assigns a drive letter to. Simply select the drive letter of the memory card.

If need to scan a sector-by-sector RAW disk image rather than the original drive itself you can use OFSMount (free download) to mount the disk image and assign it a drive letter.

Do create a disk image also called dd image or sector by sector disk copy you can for example use this for Windows or this for OSX. In general creating a disk image is recommended if the memory card is unstable (due to for example bad sectors).

If you receive a message that the file system parameters can not be determined, use Override.

Override

You normally do not need this option. However if results are not as expected like no or only a few files are detected) you may want to change these values. Setting start value to 0 and blocksize to 1 should detect recoverable JPEGs but will make operation slower.

These tables may help setting these values, taking the memory card size into account: https://www.disktuna.com/default-cluster-sizes-for-fat-exfat-and-ntfs/

Although JpegDigger ignores most file system structures, an idea about where the data area starts and the cluster (block) size is convenient as it will improve speed and lower the occurrence of false positives: As files start at cluster boundaries, knowing the cluster size limits the number of places to look for JPEGs. With a 4K cluster size the number of places to check is 8 times less than having to look at each and every cluster. Also, bitfragment carving needs the cluster-size and may fail if this value is incorrect. This is why JpegDigger will try to examine the boot sector of the selected drive and determine these values.

Click ‘Help me determine values‘ in the override window to have JpegDigger try detect correct values. This may take from a few seconds up to a few minutes. JpegDigger can not detect the file system size so you need to select that manually from the drop-down list.

Note: On a NTFS drive setting start to 0 (zero) and blocksize to 8 is virtually always correct when dealing with volumes up to 32 TB.

Options:

Set destination:

Here you specify where recovered files will be saved. If you do not set a destination JpegDigger will prompt you to select one when you ask it to save the first file. Do not save files to the same drive you are recovering from!

JpegDigger assigns file names automatically based on sector address of the first sector of a JPEG it detected. So if it detects a JPEG at sector 104856 the file name that is assigned is 104856.JPG. If a file with that name already exists (after previous attempts) it will simple overwrite it. As only one JPEG can be detected at any given start sector this is not a problem unless you make multiple recovery attempts to try for example different options.

Maximum file size: 

Adjust to the largest JPEGs your camera produces. Setting this value higher will make operation slower so just selecting highest value ‘to be on the safe side’ is a bad idea.

To determine this setting review the file size of some photos shot with the same camera.

Minimum resolution:

Specially when scanning a hard disk this is useful to skip small files that are bound to be present due to web browsers caching loads of small images.

Example: You shot photos using a NIKON D3000 then typically JPEGs produced are 3872 x 2592 and need to recover them from your hard drive. Set minimum resolution to 3 MP (3008 x 2000) to skip non relevant smaller JPEGs.

Also: If trying to recover JPEGs from RAW photos (TIFF based), set minimum resolution or the tool may recover lower resolution JPEGs. Some RAW photo types contain a medium resolution preview which may be recovered if it meets minimum resolution settings. I have seen NEF photos with 160×120, 1620×1080 and 7360×4912 JPEGs embedded for example. JpegDigger default setting would recover the 1620×1080 variant. If setting is set to 3.0 MP the high resolution JPEGs will be recovered.

JPEG from RAW is tested so far with NEF (Nikon), CR2 (Canon), ORF (Olympus), RW2 (Panasonic-Lumix), ARW (Sony) and DNG (from a Leica Q2) but should work on all TIFF based RAW photos.

Autosave:

As files are detected they are automatically saved. So this allows hands-free operation. An added advantage is that access to the media from which data is recovered from is restricted to a minimum. Essentially JpegDigger accesses media in a sequential mode from start of the media till the end.

Only free space

Enable this option to scan free space only. Useful for recovering deleted files only. This feature depends on the Windows defrag API returning a correct volume bitmap structure and therefor is only reliable on an intact and consistent file system.

* Advanced options *

Skip corrupted files: 

Part of the problem of file carving is that is bound to produce corrupted files.

First of all simple header/footer carvers produce many false positives. JpegDigger does not suffer from this problem as it uses more advanced carving where it actually parses JPEG structures.

Second category is JPEG files simply being corrupt. JpegDigger can to a degree detect corruption: It can not tell if a JPEG contains an intact and the correct image, however it can detect ‘structural’ errors.

Red LED will flash when file is skipped (hardly noticeable though due to speed)

Attempt file repair:

JpegDigger can often repair corrupt JPEGs to a degree where they can at least be opened/viewed after having been recovered.

Always try with this feature disabled first!*

Green LED will flash when repair is attempted.

Bit-fragment gap carver:

JpegDigger includes routines to detect non contiguous files and to attempt reassembly of such files. In general only enable this option if an initial recovery without this option produces incomplete files or images mixed with parts of different images. Enabling this option automatically invokes the file repair routines.

Bitfragment gap caring relies on a correctly determined cluster size as it is assumed fragments begin and end at cluster boundaries.

Always try with this feature disabled first!*

Blue LED will flash when defrag is triggered and attempted.

Ignore EOI:

The times you need to check this option is virtually zero. In other words, don’t do it.

Always try with this feature disabled first!*

*These features ‘force’ JpegDigger to deal with potentially corrupted data. This means it has to evaluate data and estimate whether to keep it or not. First of all this takes time, so it will slow down the process. Second big thing is that things can go wrong: potentially good data is discarded or even worse, corrupted (not on the source media of course) causing unexpected behavior.

Examples:

Recover JPEGs deleted from a SD Card and select files manually, because you only need a few deleted files:

  1. Select a drive 
  2. Select a destination drive
  3. Tick option free space only
  4. Click Scan

Now use the list on the right to review photos. Selecting one will dig up the preview. Put a check-mark in the box for the files you want to save. Click Save. if you didn’t select a destination drive yet, you will be prompted to select one now. If JpegDigger fails to detect the deleted files, try with free space only disabled.

Recover JPEGs from RAW photos that are incorrectly recovered by fully RAW capable software:

  1. Select a drive 
  2. Select a destination drive
  3. Set minimum resolution to 3.0 MP (to skip lower res embedded JPEGs)
  4. Click Scan

This feature can be useful if software capable of recovering RAW files, recovers corrupted RAW photos. Example: JpegDigger can recover 8369 x 5584 JPEGs where other software (PhotoRec in this case) produced corrupted .DNG photos produced by a Leica Q2. Note that this was not due to a limitation in PhotoRec but due to actual DNG files being corrupt/fragmented.

Recover ALL JPEGs from a formatted memory card unattended:

  1. Select a drive 
  2. Select a destination drive
  3. Review / Set maximum file size
  4. Set Autosave option
  5. Click Scan

JpegDigger will scan the media and save files automatically.

Recover ALL JPEGs from a formatted hard drive unattended:

It is assumed you need to recover ‘serious’ photos taken with a digital camera, not cached thumbnails etc..

  1. Select a drive 
  2. Select a destination drive
  3. Review / Set maximum file size – Set minimum resolution 
  4. Set Autosave option – Skip corrupted files
  5. Click Scan

JpegDigger will scan the media and save files automatically. To determine the minimum resolution review the camera user guide or compare to other photos show with the same camera.

Recover JPEGs from a formatted memory card that were incorrectly recovered:

  1. Select a drive 
  2. Select a destination drive
  3. Review / Set maximum file size
  4. Set Bitfragment option
  5. Click Scan

The filenames of the corrupted files correspond with the sector number in the list on the right side of the JpegDigger window. If the preview shows a better image than the previously recovered one, put a check mark in the check box and click ‘Save’. The new file will overwrite the previously recovered file if the same destination folder is selected.

Same file without and with repair option

Same file without and with repair option

Even though the file may not look perfect, consider it can possibly be further improved using JPEG Repair.

What is a ‘carver’? Advantages and disadvantages of carving.

JpegDigger is a so called carver, also called RAW recovery tool. It largely ignores the file system. As things like a directory structure and file names are maintained by a file system, these things are not recovered by JpegDigger.

Memory cards are formatted with a FAT file system. Either FAT16, FAT32 or exFAT. Directory structure and file names are kept in directories. A typical entry looks like this (Simplified):

myphoto1.jpeg Size Date and time Attributes start cluster
mytext.txt Size Date and time Attributes start cluster

To get all clusters allocated to the file, the driver reads the file allocation tables (FAT). Let’s assume Start Cluster value 131. The file system driver now reads FAT entry 131 where it finds the value 132. This means the next cluster allocated to the file is 132. The entry in the FAT then points to the next cluster and so on until a special end of file value is reached. By following this chain of clusters the file system driver can determine all clusters allocated to any specific file.

Now we see some strange humps or gaps: FAT entry 134 points to cluster 138. Entry 139 points to 139. So it skips a few cluster which means the file is no stored in contiguous clusters. It is fragmented.

132 [entry no: 131] 133 [entry no: 132] 134 [entry no: 133] 138 [entry no: 134] 136 [entry no: 135]
137 [entry no: 136] 144 [entry no: 137] 139 [entry no: 138] 140 [entry no: 139] 141 [entry no: 140]
142 [entry no: 141] 143 [entry no: 142] EOF [entry no: 143] 145 [entry no: 144] EOF [entry no: 145]

After myphoto1.jpg gets deleted the FAT looks like, while the directory entry for the file remains largely intact as well as the actual file content. It remains this way until the directory entry and clusters once allocated to myphoto.jpg are assigned to a new file.

If the memory card was re-formatted instead then ALL file allocation table entries would read zero.

0 [entry no: 131] 0 [entry no: 132] 0 [entry no: 133] 0 [entry no: 134] 136 [entry no: 135]
137 [entry no: 136] 144 [entry no: 137] 0 [entry no: 138] 0 [entry no: 139] 0 [entry no: 140]
0 [entry no: 141] 0 [entry no: 142] 0 [entry no: 143] 145 [entry no: 144] EOF [entry no: 145]

A typical JPEG file looks like this (if we’d look at the actual clusters in which the data is stored):

How recovery software recovers lost and deleted JPEGs:

To recover a deleted or lost file typical undelete, file recovery and photo recovery software will:

Locate the directory entry > Look up file size & start cluster > open new file (including original filename) > dump data from startcluster + file size > close file. From this you can tell that fragmented files will not be recovered intact.

Typical file carver ignores structures like directories.

Instead it scans media for FF D8 magic number to locate start of JPEG > keep scanning until FF D9 is reached and dump everything in between to a new file (header / footer carving). Another event that can trigger the end of a file is it detects FF D8 (start of a new file) or a certain threshold is reached (like a preset maximum file size).

A couple of problems and challenges arise:

Fragmented files can not be recovered. Also, all JPEGs start with the byte pair FF D8, but that does not mean that this byte combination is exclusive to JPEG: Possible result is ‘false positives’.

Also, as JPEGs can embed a preview or thumbnail JPEG (with all characteristics of a full JPEG) the routine needs to decide if it indeed detected a new JPEG or a thumbnail.

Carving the JpegDigger way:

JpegDigger also scan the media for magic numbers to identify the start of a JPEG. It then fires up it’s JPEG parser: Rather than keep scanning until it finds the possible end of file, it actually interprets JPEG markers. As long as this can be done without errors the likeliness it found a JPEG increases. Also, embedded JPEGs are not a problem as they are part of the normal JPEG structure and can be recognized as such.

It also allows it to detect structural errors in the encoded and compressed image data and to detect the file ending prematurely. And, with the appropriate options set it can attempt repair and/or finding the cluster where the file continues.

With regards to bitfragment gap carving:

This is a fast but simple implementation and far less advanced than the defragmentation routines in JPEG Recovery LAB. it partially depends on the FAT file system and will be less effective on NTFS formatted hard drives.

Bitfragment carving disabled Bitfragment carving enabled

fragmented photo recovered without bitfragment gap carver enabled

Same photo with bitfragment gap carver enabled