Git-Mirrors/brozzler
1
0
Fork 0
mirror of https://github.com/internetarchive/brozzler.git synced 2025-05-02 06:36:20 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

rewrite frontier.scope_and_schedule_outlinks() to use batch rethinkdb queries, because we have witnessed the method running for hours(!)

Browse source
This commit is contained in:
Noah Levitt 2017-05-26 13:24:14 -07:00
parent d904daea9c
commit bdc0badec3
2 changed files with 87 additions and 44 deletions
Download patch file Download diff file Expand all files Collapse all files

2
setup.py
View file

@ -32,7 +32,7 @@ def find_package_data(package):
setuptools.setup(
name='brozzler',
version='1.1b11.dev250',
version='1.1b11.dev251',
description='Distributed web crawling with browsers',
url='https://github.com/internetarchive/brozzler',
author='Noah Levitt',