Git-Mirrors/brozzler
1
0
Fork 0
mirror of https://github.com/internetarchive/brozzler.git synced 2025-05-03 07:05:12 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Merge branch 'master' into qa

Browse source 
* master:
  lowercase readme.rst
  explain brozzler use of warcprox_meta
  update README copyright date
  bump dev version after PR #102
  these ssurts are strings too
  fix bad copy/paste
  ssurts are strings now
  travis-ci install warcprox from github
  incorporate urlcanon fix
  update warcprox dependency to include recent fixes
  backward compatibility for old scope["surt"]
  missed a spot where is_permitted_by_robots needs monkeying
  handle new chrome cookie db schema
  describe scope rule conditions
  more explication of scoping
  update docs to match new seed ssurt behavior
  ok seriously tests
  fix more tests for new approach sans scope['surt']
  s/max_hops_off_surt/max_hops_off/
  new test of max_hops_off
  rename page.hops_off_surt to page.hops_off
  doublethink had a bug fix
  tests for new approach without scope['surt']
  tests for new approach without of scope['surt']
  WIP add an accept rule instead of modifying surt
  WIP some words on scoping
  WIP starting to flesh out "scoping" section
  WIP some explanation of automatic login
  WIP documentation!
This commit is contained in:
Noah Levitt 2018-06-01 16:46:32 -07:00
commit 5c34bd3119
14 changed files with 699 additions and 257 deletions
Download patch file Download diff file Expand all files Collapse all files

12
setup.py
View file

@ -2,7 +2,7 @@
'''
setup.py - brozzler setup script
Copyright (C) 2014-2017 Internet Archive
Copyright (C) 2014-2018 Internet Archive
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
@ -32,12 +32,12 @@ def find_package_data(package):
setuptools.setup(
name='brozzler',
version='1.1b13.dev288',
version='1.1b13.dev290',
description='Distributed web crawling with browsers',
url='https://github.com/internetarchive/brozzler',
author='Noah Levitt',
author_email='nlevitt@archive.org',
long_description=open('README.rst', mode='rb').read().decode('UTF-8'),
long_description=open('readme.rst', mode='rb').read().decode('UTF-8'),
license='Apache License 2.0',
packages=['brozzler', 'brozzler.dashboard'],
package_data={
@ -69,8 +69,8 @@ setuptools.setup(
'requests',
'websocket-client!=0.39.0',
'pillow==3.3.0',
'urlcanon>=0.1.dev16',
'doublethink>=0.2.0.dev81',
'urlcanon>=0.1.dev23',
'doublethink>=0.2.0.dev88',
'rethinkdb>=2.3,<2.4',
'cerberus==1.0.1',
'jinja2',
@ -79,7 +79,7 @@ setuptools.setup(
extras_require={
'dashboard': ['flask>=0.11', 'gunicorn'],
'easy': [
'warcprox>=2.4b1.dev145',
'warcprox>=2.4b2.dev173',
'pywb<2',
'flask>=0.11',
'gunicorn'