cyber-security-resources/osint

Open-source Intelligence (OSINT)

Open-source intelligence (OSINT) is data collected from open source and publicly available sources. The following are a few OSINT resources and references:

Passive Recon Tools:

• AMass
• Deepinfo (commercial tool)
• Exiftool
• ExtractMetadata
• Findsubdomains
• FOCA
• IntelTechniques
• Maltego
• Recon-NG
• Scrapy
• Screaming Frog
• Shodan
• SpiderFoot
• theHarvester
• Visual SEO Studio
• Web Data Extractor
• Xenu
• ParamSpider

Open Source Threat Intelligence

• Awesome Threat Intelligence - A curated list of awesome Threat Intelligence resources. This is a great resource and I try to contribute to it.

OSINT Source Highlights

Website	Description
shodan.io	Server
google.com	Dorks
wigle.net	WiFi Networks
grep.app	Codes Search
app.binaryedge	Threat Intelligence
onyphe.io	Server
viz.greynoise.io	Threat Intelligence
censys.io	Server
hunter.io	Email Addresses
fofa.info	Threat Intelligence
zoomeye.org	Threat Intelligence
leakix.net	Threat Intelligence
intelx.io	OSINT
app.netlas.io	Attack Surface
searchcode.com	Codes Search
urlscan.io	Threat Intelligence
publicwww.com	Codes Search
fullhunt.io	Attack Surface
socradar.io	Threat Intelligence
binaryedge.io	Attack Surface
ivre.rocks	Server
crt.sh	Certificate Search
vulners.com	Vulnerabilities
pulsedive.com	Threat Intelligence

Website Exploration and "Google Hacking"

• censys : https://censys.io
• Certficate Search: https://crt.sh/
• ExifTool: https://www.sno.phy.queensu.ca/~phil/exiftool
• Google Hacking Database (GHDB): https://www.exploit-db.com/google-hacking-database
• Google Transparency Report: https://transparencyreport.google.com/https/certificates
• Huge TLS/SSL certificate DB with advanced search: https://certdb.com
• netcraft: https://searchdns.netcraft.com
• SiteDigger: http://www.mcafee.com/us/downloads/free-tools/sitedigger.aspx
• Spyse: https://spyse.com

Data Breach Query Tools

• BaseQuery: https://github.com/g666gle/BaseQuery
• Buster: https://github.com/sham00n/buster
• h8mail: https://github.com/khast3x/h8mail
• LeakLooker: https://github.com/woj-ciech/LeakLooker
• PwnDB: https://github.com/davidtavarez/pwndb
• Scavenger: https://github.com/rndinfosecguy/Scavenger
• WhatBreach: https://github.com/Ekultek/WhatBreach

IP address and DNS Lookup Tools

• bgp
• Bgpview
• DataSploit (IP Address Modules)
• Domain Dossier
• Domaintoipconverter
• Googleapps Dig
• Hurricane Electric BGP Toolkit
• ICANN Whois
• Massdns
• Mxtoolbox
• Ultratools ipv6Info
• Viewdns
• Umbrella (OpenDNS) Popularity List

Social Media

• A tool to scrape LinkedIn
• cree.py

Acquisitions and

• OCCRP Aleph - The global archive of research material for investigative reporting.

Whois

WHOIS information is based upon a tree hierarchy. ICANN (IANA) is the authoritative registry for all of the TLDs and is a great starting point for all manual WHOIS queries.

• ICANN: http://www.icann.org
• IANA: http://www.iana.com
• NRO: http://www.nro.net
• AFRINIC: http://www.afrinic.net
• APNIC: http://www.apnic.net
• ARIN: http://ws.arin.net
• LACNIC: http://www.lacnic.net
• RIPE: http://www.ripe.net

BGP looking glasses

• BGP4: http://www.bgp4.as/looking-glasses
• BPG6: http://lg.he.net/

DNS

• dnsenum - https://code.google.com/p/dnsenum
• dnsmap: https://code.google.com/p/dnsmap
• dnsrecon: https://www.darkoperator.com/tools-and-scripts
• dnstracer: https://www.mavetju.org/unix/dnstracer.php
• dnswalk: https://sourceforge.net/projects/dnswalk

The OSINT Framework

• OSINT Framework

Dark Web OSINT Tools

Dark Web Search Engine Tools

• Ahmia Search Engine and their GitHub repo
• DarkSearch and their GitHub repo
• Katana
• OnionSearch
• Search Engines for Academic Research
• DarkDump

Tools to Obtain Information of .onion Links

• H-Indexer
• Hunchly
• Tor66 Fresh Onions

Tools to scan onion links

• Onioff
• Onion-nmap
• Onionscan

Tools to Crawl Dark Web Data

• TorBot
• TorCrawl
• OnionIngestor

Other Great Intelligence Gathering Sources and Tools

• Resources from Pentest-standard.org - http://www.pentest-standard.org/index.php/PTES_Technical_Guidelines#Intelligence_Gathering

Active Recon

• Tons of references to scanners and vulnerability management software for active reconnaissance - http://www.pentest-standard.org/index.php/PTES_Technical_Guidelines#Vulnerability_Analysis