cyber-security-resources/osint

Open-source Intelligence (OSINT)

Open-source intelligence (OSINT) is data collected from open source and publicly available sources. The following are a few OSINT resources and references:

Passive Recon Tools:

• AMass
• Exiftool
• ExtractMetadata
• Findsubdomains
• FOCA
• IntelTechniques
• Maltego
• Recon-NG
• Scrapy
• Screaming Frog
• Shodan
• SpiderFoot
• theHarvester
• Visual SEO Studio
• Web Data Extractor
• Xenu
• ParamSpider

Open Source Threat Intelligence

• GOSINT - a project used for collecting, processing, and exporting high quality indicators of compromise (IOCs). GOSINT allows a security analyst to collect and standardize structured and unstructured threat intelligence.
• Awesome Threat Intelligence - A curated list of awesome Threat Intelligence resources. This is a great resource and I try to contribute to it.

Website Exploration and "Google Hacking"

• censys : https://censys.io
• Certficate Search: https://crt.sh/
• ExifTool: https://www.sno.phy.queensu.ca/~phil/exiftool
• Google Hacking Database (GHDB): https://www.exploit-db.com/google-hacking-database
• Google Transparency Report: https://transparencyreport.google.com/https/certificates
• Huge TLS/SSL certificate DB with advanced search: https://certdb.com
• netcraft: https://searchdns.netcraft.com
• SiteDigger: http://www.mcafee.com/us/downloads/free-tools/sitedigger.aspx
• Spyse: https://spyse.com

Data Breach Query Tools

• BaseQuery: https://github.com/g666gle/BaseQuery
• Buster: https://github.com/sham00n/buster
• h8mail: https://github.com/khast3x/h8mail
• LeakLooker: https://github.com/woj-ciech/LeakLooker
• PwnDB: https://github.com/davidtavarez/pwndb
• Scavenger: https://github.com/rndinfosecguy/Scavenger
• WhatBreach: https://github.com/Ekultek/WhatBreach

IP address and DNS Lookup Tools

• bgp
• Bgpview
• DataSploit (IP Address Modules)
• Domain Dossier
• Domaintoipconverter
• Googleapps Dig
• Hurricane Electric BGP Toolkit
• ICANN Whois
• Massdns
• Mxtoolbox
• Ultratools ipv6Info
• Viewdns
• Umbrella (OpenDNS) Popularity List

Social Media

• A tool to scrape LinkedIn: https://github.com/dchrastil/TTSL
• cree.py http://ilektrojohn.github.com/creepy

Whois

WHOIS information is based upon a tree hierarchy. ICANN (IANA) is the authoritative registry for all of the TLDs and is a great starting point for all manual WHOIS queries.

• ICANN: http://www.icann.org
• IANA: http://www.iana.com
• NRO: http://www.nro.net
• AFRINIC: http://www.afrinic.net
• APNIC: http://www.apnic.net
• ARIN: http://ws.arin.net
• LACNIC: http://www.lacnic.net
• RIPE: http://www.ripe.net

BGP looking glasses

• BGP4: http://www.bgp4.as/looking-glasses
• BPG6: http://lg.he.net/

DNS

• dnsenum - http://code.google.com/p/dnsenum
• dnsmap: http://code.google.com/p/dnsmap
• dnsrecon: http://www.darkoperator.com/tools-and-scripts
• dnstracer: http://www.mavetju.org/unix/dnstracer.php
• dnswalk: http://sourceforge.net/projects/dnswalk

The OSINT Framework

• OSINT Framework

Dark Web OSINT Tools

Dark Web Search Engine Tools

• Ahmia Search Engine and their GitHub repo
• DarkSearch and their GitHub repo
• Katana
• OnionSearch
• Search Engines for Academic Research
• DarkDump

Tools to Obtain Information of .onion Links

• H-Indexer
• Hunchly
• Tor66 Fresh Onions

Tools to scan onion links

• Onioff
• Onion-nmap
• Onionscan

Tools to Crawl Dark Web Data

• TorBot
• TorCrawl
• OnionIngestor

Other Great Intelligence Gathering Sources and Tools

• Resources from Pentest-standard.org - http://www.pentest-standard.org/index.php/PTES_Technical_Guidelines#Intelligence_Gathering

Active Recon

• Tons of references to scanners and vulnerability management software for active reconnaissance - http://www.pentest-standard.org/index.php/PTES_Technical_Guidelines#Vulnerability_Analysis