Git-Mirrors/brozzler
1
0
Fork 0
mirror of https://github.com/internetarchive/brozzler.git synced 2025-02-24 08:39:59 -05:00
Code Issues Actions Packages Projects Releases Wiki Activity

fix bug preventing brozzler-new-site from working, add note about brozzler-new-site in readme

Browse Source
This commit is contained in:
Noah Levitt 2016-06-29 18:41:45 -05:00
parent 479713e25b
commit 7431ae0eb1
3 changed files with 11 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
README.rst
View File

@ -70,6 +70,14 @@ must be specified, everything else is optional.
scope: scope:
surt: http://(org,example, surt: http://(org,example,
Submit a Site to Crawl Without Configuring a Job
------------------------------------------------
::
brozzler-new-site --proxy=localhost:8000 --enable-warcprox-features \
--time-limit=600 http://example.com/
Brozzler Web Console Brozzler Web Console
-------------------- --------------------

2
brozzler/frontier.py
View File

@ -233,6 +233,8 @@ class RethinkDbFrontier:
yield brozzler.Job(**result) yield brozzler.Job(**result)
def job(self, id): def job(self, id):
if id is None:
return None
result = self.r.table("jobs").get(id).run() result = self.r.table("jobs").get(id).run()
if result: if result:
return brozzler.Job(**result) return brozzler.Job(**result)

2
setup.py
View File

@ -21,7 +21,7 @@ import setuptools
setuptools.setup( setuptools.setup(
name='brozzler', name='brozzler',
version='1.1.dev32', version='1.1.dev33',
description='Distributed web crawling with browsers', description='Distributed web crawling with browsers',
url='https://github.com/internetarchive/brozzler', url='https://github.com/internetarchive/brozzler',
author='Noah Levitt', author='Noah Levitt',