Today we'll be talking about robots and dorks because at Number Four of the Open Source Intelligence Top Ten (aka OSINT Top Ten) is Search Engine Exploitation. What do robots and dorks have to do with exploiting search engines, you ask? Read on...

Search engine exploitation is the malicious act of performing targeted queries to uncover anything and everything about an organization.

Commonly known as "dorks," these queries can reveal infrastructure technologies, applications, services, confidential/proprietary/sensitive data, and even IoT entities associated with your organization.

It is imperative to know what indexed information search engine queries could broadcast or even uncovered about your organization. Within everything and everyone having a digital presence, it is not that difficult to find an organization's technical infrastructure from the network, servers, and even security appliances/devices through queries revealing error messages, log files, application files, and even portals/login pages.

Bad actors can also facilitate attacks and exploits through footholds and vulnerabilities uncovered by such simple search engine queries. Folders and files containing sensitive information (e.g., passwords, usernames, customer data, supplier data, partner data, credit card data, etc.) can be accessed online without much of a hurdle as well.

Another critical aspect of controlling your exposure or susceptibility to search engine exploits is appropriately maintaining and managing your robots.txt file.

The robots.txt file is a file that is honored by all major web crawlers, indexers, and archive services. This file's proper maintenance and calibration can help shield yourself from unintended over-exposure and most search engine exploits. The robots.txt file can, and should, be fairly explicit on what to limit as not to interfere with SEO and other marketing tactics.

Next on the OSINT Top Ten is Number Three - Sensitive Code, and we’re not talking about code that likes to listen to Dashboard Confessional or Fall Out Boy.