Reducing the number of documents to review during an e-discovery project is a high priority for both attorneys and their clients. One commonly used technique is to remove files from a document set that are known to belong to certain software programs. This process is called “Known File Filtering” and is often referred to as “de-NISTing” since it uses a list of file hashes created by the National Institute of Standards and Technology (NIST).
The “NIST” list is actually a database called the National Software Reference Library (NSRL). This list contains information about software, including “hash” values which uniquely identify the data within a file, regardless of its name, date of creation or location. If two files contain identical data they will also have identical hash values.
The NSRL database represents a collection of categorized file information for software of all kinds. It organizes programs into groups, such as word processing software, system files, gaming programs, etc. This is different than several earlier file collections of common computer file information (like HashKeeper), the NSRL does not make a distinction between “good” and “bad” files and does not contain lists of contraband data, such as child pornography.
One of the key features of the NSRL is that anyone can submit software for review and inclusion in the list which has helped keep the list up to date. In fact, many e-discovery and digital forensics software companies have included the NSRL in their products to assist with culling out irrelevant data in the early stages of investigation.
The NSRL currently contains approximately 53,000,000 file entries and a new list is released every month to address software updates and newly available programs.
Using a complete list and the most current version is a very important step to properly reduce the amount of data that must be reviewed, since some software is not automatically updated and does not contain the complete list.
De-NISTing is a very helpful part of ESI processing but is not a “silver bullet” for reducing e-discovery document sets. There are certain files or programs in the list that may actually be relevant depending on the scope of a particular case. For example, remote access software programs have legitimate IT functions but can also be utilized for nefarious purposes. In situations where misuse of this type of program is suspected, it is critical to communicate this information to an e-discovery or digital forensics vendor to ensure that key information is not inadvertently excluded. Likewise, commercially available data wiping software (such as Evidence Eliminator or Disk Redactor) is certainly present in the NSRL but may be a pivotal part of an investigation involving data deletion.
