Git-Mirrors/sec-pentesting-toolkit
1
0
Fork 0
mirror of https://github.com/autistic-symposium/sec-pentesting-toolkit.git synced 2025-04-28 11:36:08 -04:00
Code Issues Packages Projects Releases Wiki Activity

urllib2 readme

Browse Source
This commit is contained in:
Mari Wahl 2014-12-30 13:31:43 -05:00
parent ec815dd94a
commit c5796321ce
1 changed files with 8 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
Web_Security/urllib2/README.md
View File

@ -1,8 +1,8 @@
# Hacking the Web with Python's urllib2 (by bt3) # Hacking the Web with Python's urllib2 (by bt3)
Python's [urllib2](https://docs.python.org/2/library/urllib2.html) library is the tool to interact with web services, with several functions and classes to help handling URLs. **urllib2** is written in the top of [httplib](https://docs.python.org/2/library/httplib.html) library (which defines classes to implement the client side of HTTP and HTTPs). In turn, **httplib** uses the [socket](http://bt3gl.github.io/black-hat-python-networking-the-socket-module.html) library. Python's [urllib2](https://docs.python.org/2/library/urllib2.html) library is **the tool** to interact with web services, with several functions and classes to help handling URLs. **urllib2** is written in the top of [httplib](https://docs.python.org/2/library/httplib.html) library (which defines classes to implement the client side of HTTP and HTTPs). In turn, **httplib** uses the [socket](http://bt3gl.github.io/black-hat-python-networking-the-socket-module.html) library.
In this post I [introduce urllib2](#intro) and then I work on two problems: [mapping webapp from their installation files](#map) and [brute-forcing webapp content to find hidden resources](#brute1). In this post I [introduce urllib2](#intro) and then I work on two problems: [mapping webapps from their installation files](#map) and [brute-forcing the contents of webapps to find hidden resources](#brute1).
----- -----
@ -10,7 +10,7 @@ In this post I [introduce urllib2](#intro) and then I work on two problems: [map
## <a name="intro"></a>urllib2 101 ## <a name="intro"></a>urllib2 101
The easiest way to start is with the **urlopen** method, which returns an object similar to a **file** in Python (plus three more methods: **geturl**, for the URL of the resource; **info**, for meta-information; and **getcode**, for HTTP status code). The easiest way to start is by taking a look at the **urlopen** method, which returns an object similar to a **file** in Python (plus three more methods: **geturl**, for the URL of the resource; **info**, for meta-information; and **getcode**, for HTTP status code).
@ -106,7 +106,7 @@ response = urllib2.urlopen(req)
print response.read() print response.read()
``` ```
That's one of the differences between **urllib2** and **urllib**: the former can accept a Request object to set the headers for a URL request, while the last accepts only a URL. That's one of the differences between **urllib2** and **urllib**: the former can accept a **Request object** to set the headers for a URL request, while the last accepts only a URL.
@ -212,7 +212,7 @@ pagehandle = urllib2.urlopen(top_url)
### Error Handling ### Error Handling
**urllib2** has methods to handle URL errors: **urllib2** has also methods for handling URL errors:
```python ```python
>>> request = urllib2.Request('http://www.false_server.com') >>> request = urllib2.Request('http://www.false_server.com')
@ -223,7 +223,7 @@ pagehandle = urllib2.urlopen(top_url)
(4, 'getaddrinfo failed') (4, 'getaddrinfo failed')
``` ```
Every HTTP response from the server contains a numeric [status code](http://en.wikipedia.org/wiki/List_of_HTTP_status_codes). The default handlers handle some of these responses and for those it can't handle, **urlopen** raises an **HTTPError**, which is a subclass of **URLError**. Every HTTP response from the server contains a numeric [status code](http://en.wikipedia.org/wiki/List_of_HTTP_status_codes). The default handlers take care some of these responses and for the others, **urlopen** raises an **HTTPError** (which is a subclass of **URLError**).
### Other Available Methods ### Other Available Methods
@ -261,7 +261,7 @@ Other common functions are **urljoin** and **urlsplit**.
--- ---
## <a name="map"></a>Mapping Webapp from their Installation Packages ## <a name="map"></a>Mapping Webapps from their Installation Packages
[Content management systems](http://en.wikipedia.org/wiki/Content_management_system) are platforms to make it easy to start blogs or simple websites. They are common in shared host environments. However, when all of the security procedures are not followed, they can be a easy target for attackers to gain access to the server. [Content management systems](http://en.wikipedia.org/wiki/Content_management_system) are platforms to make it easy to start blogs or simple websites. They are common in shared host environments. However, when all of the security procedures are not followed, they can be a easy target for attackers to gain access to the server.
@ -400,7 +400,7 @@ In all of these results we are able to find some nice results including XML and
----- -----
## <a name="brute1"></a>Brute-Forcing Webapp Content ## <a name="brute1"></a>Brute-Forcing the Contents of Webapps
In general we are not aware about the structure of files that are accessible in a web server (we don't have the webapp available like in the previous example). Usually we can deploy a spider, like in the [Burp suite](http://portswigger.net/burp/), to crawl the target and find them. However this might not be able to find sensitive files such as, for example, development/configuration files and debugging scripts. In general we are not aware about the structure of files that are accessible in a web server (we don't have the webapp available like in the previous example). Usually we can deploy a spider, like in the [Burp suite](http://portswigger.net/burp/), to crawl the target and find them. However this might not be able to find sensitive files such as, for example, development/configuration files and debugging scripts.