Git-Mirrors/brozzler
1
0
Fork 0
mirror of https://github.com/internetarchive/brozzler.git synced 2025-05-02 14:46:18 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

always save outlinks info on rethinkdb page object, get rid of 'remember_outlinks' option, to keep config simple, and because it's not a very expensive thing

Browse source
This commit is contained in:
Noah Levitt 2017-03-17 10:04:10 -07:00
parent 701f7654a8
commit 0685c77d01
4 changed files with 11 additions and 16 deletions
Download patch file Download diff file Expand all files Collapse all files

3
tests/test_frontier.py
View file

@ -252,8 +252,7 @@ def test_field_defaults():
def test_scope_and_schedule_outlinks():
rr = doublethink.Rethinker('localhost', db='ignoreme')
frontier = brozzler.RethinkDbFrontier(rr)
site = brozzler.Site(rr, {
'seed':'http://example.com/', 'remember_outlinks':True})
site = brozzler.Site(rr, {'seed':'http://example.com/'})
parent_page = brozzler.Page(rr, {
'hops_from_seed': 1, 'url': 'http://example.com/whatever'})
outlinks = [