This a small tale about how a relatively ‘obscure’ and rare file format can be close to unrecoverable using existing tools.
To refresh our minds we can basically recover files in two ways:
- File system based. This is always the preferred method as the file system allows us to determine the precise location of the file and in addition gives us important attributes like the filename and (depending on overall condition of file system meta data we can retrieve) the folder structure.
- Signature based. Also referred to as RAW scan or file carving. If for whatever reason a file is completely detached from the file system we may be able to find a file if we know what the file data is supposed to look like. An example of this is the JPEG file which always starts with byte pair FF D8 and ends with FF D9.
Recovering .BRAW files from 80TB RAID array.
Then to the case at hand:
- Client (of other lab) exclusively interested in Blackmagic RAW video footage.
- Since this footage is on a 80TB RAID array I presume other lab correctly solved RAID parameters.
- However file system parameters can not be solved for some reason.
- Using various tools, such as R-Studio, UFS Explorer (both using custom scan signatures) and PhotoRec RAW scans have been attempted but they yield no intact and playable video files. IOW the recovered .BRAW files are corrupt. In utter desperation even crap tools like DiskDrill were tried.
I examine a few of the incorrectly recovered files and it appears we’re dealing with a QuickTime type container although Blackmagic added it’s custom atoms. MP4 Inspector confirms my analysis, if I feed it an intact .BRAW file it happily parses the file’s atom structure. All corrupt, recovered files lack various vital atoms and in addition it turns out after some experimenting the BRAW video player is quite picky. It refuses to play video with the smallest of deviations from it’s standard (but not all).
We can parse the .BRAW file as if it were .mp4, .mov, etc.
Correctly detecting end of file
Anyway, what it comes down to is the file recovery tools correctly detect the start of the file (that starts with a ‘wide’ atom) but all prematurely finish the file. Both the pro-grade tools (R-Studio, UFS Explorer) detect the file using a custom signature but the end signature fails. I notify the support of R-Studio (https://forum.r-tt.com/viewtopic.php?t=11328).
What it comes down to is we need a custom end signature, the problem is, that there is no such unique signature. The one we can use appears throughout the file (so not unique) and so we need a more complex set of rules where we tick several AND, OR, AND NOT type conditions to determine the correct end of the file.
some of my notes
The only tool I am aware of that allows for such conditions is DMDE. After several signature iterations I can correctly carve, intact and playable .BRAW files. For me the job ends here as I do not have access to the RAID array containing the lost data. Creation of the custom signature was the service provided to another lab. Although relatively simple, such jobs do consume a fair amount of time.
Future solution?
Although the solution works, it is actually quite clumsy. Due to limitations of the tools we need and excessive set of AND, OR, AND NOT rules to correctly determine the file end. And it might very well be, the custom signature I made will only work with files produced by this particular camera. Since we know we’re basically dealing with a QuickTime container it would be far more convenient and precise if we could apply our knowledge of the container format which is well documented. I have already documented some pseudo code for this and if time permits I’ll create a tool for more convenient recovery of .BRAW files.