Serves NSRL RDS hashes
The National Institute of Standards and Technology (NIST) maintains the National Software Reference Library (NSRL) — a giant compendium of software contributed by vendors. It’s not a library in the sense that you can check things out from it, though: it’s more a library that you can check to see whether a given file already exists. All the system files from Windows 7 are in the NSRL, as is the latest releases of Firefox and Opera and Chrome, Winamp and…
It's large, really large: over forty million distinct hashes.
The Reference Data Set (RDS) is a list of hashes for all of the files maintained within the NSRL. It’s over forty million hashes, each one corresponding to a known piece of software. This isn’t to say everything listed is known good, known safe, or anything like that — just that it’s known.
Forensic investigators, first responders and technical support staff often have a needle-in-a-haystack problem: of all the files on a given storage medium they are probably only interested in a handful. A good way to begin is by finding out what things may be ignored. The odds are excellent that a file present in the RDS is of no real interest to the investigator.
It keeps track of 40 million hash values in an in-memory dataset and allows users to query that set at extremely high volume. This allows an investigator using an NSRL tool (such as
nsrllookup) to winnow through large numbers of files in a very short period of time.
I did — Rob Hansen, or rjhansen on GitHub. Feel free to email me.