Reconnaissance

https://nmap.org/  - The SYN scan (-sS option) is the default and most popular scan option

Shodan - Search Engine for the Internet of Everything

Robot.txt File - Web Robots (also known as Web Wanderers, Crawlers, or Spiders), are programs that traverse the Web automatically. Search engines such as Google use them to index the web content, spammers use them to scan for email addresses, and they have many other uses.



Standard networking command-line tools;

dig, nslookup, and whois to gather public information about a target network from DNS registries. The nslookup and whois tools are available on both Windows, UNIX, and Linux platforms, and dig (domain information groper) is available on UNIX and Linux systems.