Using Bulk Extractor for Quick OSINT Wins
Early this week, Archive.org hosted a dump of a SQL databasehacked from a neo nazi forum online known as Iron March at https://archive.org/details/iron_march_201911.While there were some .CSV files, there was also an 750MB SQL database file. Withsome massaging, SQL databases can be queried for the data they contain.Sometimes all you’re looking for is a quick and dirty list of selectors andthis data dump seemed like the perfect opportunity to do a quick write-up onusing Bulk Extractor for OSINT.
Bulk extractor is an open source tool that can be downloadedfrom https://github.com/simsong/bulk_extractor.I first learned about it in a digital forensics class years ago and I’ve been afan ever since. It’s designed to quickly chew threw a pile of data and extractthe selectors (IP addresses, email addresses, phone numbers etc.) containedwithin that data. I’ve run it on hard drives, forensics image files, databasefiles, folders full of different file types, memory dumps from mobile phonesetc. It’s easy to use and can produce amazingly quick results.
Usually I will point Bulk extractor at the directory full offiles but for this example, I’m pointing it solely at the 750MB SQL database.
I left the options on default regarding what types of entities to extract and how much machine resources should be used to search for them. Bulk Extractor took about a minute and a half to scan the 750MB database file.
You can either view the results from the Bulk ExtractorViewer GUI:
Or view the textfiles created for each of the selectortypes:
Bulk extractor creates histogram files which are extremely useful. Instead of a file full duplicate email addresses, you can view the emails and the number of occurrences within the data. quickly.
If what you’re looking for is a professionally done visualreport, bulk extractor may not be your favorite tool. But if time as short and you’re asked toproduce selectors “yesterday”, Bulk Extractor is often a perfect solution. It’sa tool that I always rave about to forensics professionals and I recently realizedthat I need to share it with more OSINT analysts and researchers as well.