Using Bulk Extractor for Quick OSINT Wins

Early this week, Archive.org hosted a dump of a SQL databasehacked from a neo nazi forum online known as Iron March at https://archive.org/details/iron_march_201911.While there were some .CSV files, there was also an 750MB SQL database file. Withsome massaging, SQL databases can be queried for the data they contain.Sometimes all you’re looking for is a quick and dirty list of selectors andthis data dump seemed like the perfect opportunity to do a quick write-up onusing Bulk Extractor for OSINT.

Bulk extractor is an open source tool that can be downloadedfrom https://github.com/simsong/bulk_extractor.I first learned about it in a digital forensics class years ago and I’ve been afan ever since. It’s designed to quickly chew threw a pile of data and extractthe selectors (IP addresses, email addresses, phone numbers etc.) containedwithin that data. I’ve run it on hard drives, forensics image files, databasefiles, folders full of different file types, memory dumps from mobile phonesetc. It’s easy to use and can produce amazingly quick results.

Usually I will point Bulk extractor at the directory full offiles but for this example, I’m pointing it solely at the 750MB SQL database.

I left the options on default regarding what types of entities to extract and how much machine resources should be used to search for them. Bulk Extractor took about a minute and a half to scan the 750MB database file.

You can either view the results from the Bulk ExtractorViewer GUI:

Or view the textfiles created for each of the selectortypes:

Bulk extractor creates histogram files which are extremely useful. Instead of a file full duplicate email addresses, you can view the emails and the number of occurrences within the data. quickly.

If what you’re looking for is a professionally done visualreport, bulk extractor may not be your favorite tool.  But if time as short and you’re asked toproduce selectors “yesterday”, Bulk Extractor is often a perfect solution. It’sa tool that I always rave about to forensics professionals and I recently realizedthat I need to share it with more OSINT analysts and researchers as well.

Comments

Post a Comment

Popular posts from this blog

SANS Index How To Guide with Pictures

Image
I got some great advice recently on creating an index for SANS exams and I wanted to write a blog post to share it with others. I took the SANS FOR 508 Computer Forensics course in 2008. It was way over my head but I had a great time and learned a ton. A few months ago I finally decided to go for my GCFA certification. I had four year old material from a course that had been completely revamped and no index. I passed the exam with a score in the 80s but it was a grueling experience. I had to rush on the last part of the exam and never felt comfortable. A few months after my GCFA exam I got an opportunity to attend a SANS SEC 504 class. I really wanted to prepare for my GCIH exam the right way so while I was at the conference I asked several individuals how they prepared their index. Most people told me that their indexes were 8-10 pages. A lot of these people had more SANS certs than I have friends so their methods obviously worked for them. My class had a teaching assistant (also SANS
Read more

Introducing FaviconLocator: The Eazy Button to Searching by Favicon

Image
  Favicons (short for favorite icons) are the cute little pixelated images that appear next to the site name in web browser tabs, bookmarks, etc. In the image below we can see the iconic GitHub logo on their site and the KFC logo on a bucket of chicken on their site. Originally, favicons were designed to add a touch of professionalism and branding, but for anyone who is like me and has over a dozen tabs open at any time, favicons are the only thing displayed and how I navigate tabs. Most of us rely on favicons on a daily basis but many never think of them as a tool we can use in OSINT and CTI investigations. That’s what we’re doing to talk about here as well as introduce a new tool. In addition to branding and aiding in navigation between tabs, favicons can serve as unique identifiers for websites. These unique identifiers can help us: Trace the online presence of organizations and discover obscure digital assets Map the online infrastructure of potential threats Potentially de-ano
Read more

Automating Domain Squatting Detection with DNSTwist and Python

Image
  There’s a good chance that, at some point, you’ve received a spam email with a link that looked close to the name of a popular domain but was just a little off. Payapl.com instead of Paypal.com or similar. Domain squatting is a malicious activity where attackers register domain names similar to legitimate ones. Attackers may use these domains to deceive users into believing they are visiting a legitimate website, which can lead to phishing and other attacks. DNSTwist (available here: https://github.com/elceef/dnstwist ) is a popular tool that helps identify domain names that are similar to a given domain name. It generates a list of potential domain names by applying various fuzzing techniques to the given domain name and then checks if these domains are registered. Check out the list of Bank Of America copycat domains: DNSTwist is an amazing tool that should likely be a part of every organization's Cyber Threat Intelligence monitoring efforts, and I wanted to automate it
Read more