The importance of cross-validation in digital forensics cannot be overstated. Given the complexity and diversity of digital devices and data, it is not uncommon for different methods to produce different results. This can be due to a range of factors such as the type of data, the software used, or the expertise of the examiner. Without cross-validation, it can be challenging to determine which method is more accurate, which can lead to incorrect conclusions or even legal challenges. Nobody needs those. 

In my own work, I have found that it is not rare for different software tools to yield different results in several types of data analysis. I have seen USB Detective (USBD) find external device activity that other tools missed completely, and USBD’s verbose output allowed me to manually substantiate its output and prove its accuracy. Oxygen Forensic Detective (OFD) and Cellebrite’s Physical Analyzer (PA) can differ radically in what they parse and present from the same extracted dataset. It is not a lock that one tool is inherently better than another—this could be the case, but one shouldn’t assume this because of one instance—so why does this happen? It happens because software is made by humans and humans make mistakes. Bugs slip into versions that get pushed out to users, or maybe it’s plain old user error. Whatever the cause, we should do our best to catch, understand, and solve these discrepancies before we form our conclusions, much less formally present them to the client and beyond. 

I saw the Cellebrite-Oxygen discrepancy play out in court because the opposing examiner did not cross-validate. He put his faith completely in the accuracy and reliability of Physical Analyzer, then took the stand and testified that the phone at issue was missing all its contacts and that thousands of text messages had no sender or recipient data. With a show of righteous indignation, the attorney who hired him asked the court to sanction my party for spoliation. 

Then I took the stand and explained how I had loaded the same data, which the opposing examiner had collected and provided to me, into Oxygen Forensic Detective, which showed 450+ contacts and exactly one problematic text message, out of 83,000+. You can guess how the hearing unfolded after that. 

Cross-validation can be performed in several ways, depending on the nature of the investigation and the available resources. Different tools. Different examiners; trusted peer review is always a great idea. The goal is to compare the results obtained from each method or by each examiner and identify any discrepancies or inconsistencies, especially when you have results critical to the case or results that are unexpected or odd. 

Speaking of odd, just this week I encountered something I have never seen before. A client asked me to clarify two recordings from a Ring doorbell camera. (Because this was an audio/video clarification procedure and not a forensic examination, cross-validation would not normally apply in its classic sense, but stay tuned!) I used Amped FIVE for the video and a mix of Adobe Audition and iZotope RX 10 Advanced for the audio. 

To my surprise, one of the recordings showed a different duration in iZotope than the same recording did everywhere else, including Audition, Premiere Pro, and the duration column in Directory Opus (the greatest file manager on the planet), the latter of course being pulled from the file’s own metadata. And it wasn’t just some metadata quirk; when the audio track was played in iZotope, it had an additional second of someone speaking at the end of the recording, content that played nowhere else, content beyond the end of the video track. The result was that when I extracted the audio track in iZotope and married it back to the video track in Premiere Pro (after the video had been clarified in Amped FIVE), the video played, with audio, and when the video ended, another second or so of audio played. In this instance, that content was not critical to the events at hand, but what if it had been? The lesson to be learned is clear: cross-validation can reveal unusual characteristics in the evidence itself. 

In addition to ensuring the accuracy and reliability of the findings, cross-validation can also help to identify potential errors or biases in the investigation process. By comparing the results obtained from different methods, examiners can identify areas where further investigation or clarification may be needed. This can help to improve the overall quality and credibility of the investigation. And the investigator. 

In conclusion, cross-validation is a critical technique in digital forensics that can help to ensure the accuracy and reliability of our findings. By comparing the results obtained from different methods, we can identify discrepancies or inconsistencies and improve the overall quality of our investigations. Plainly put, it is essential that we digital forensic examiners incorporate cross-validation into our investigative processes whenever feasible to ensure the best possible results.