brozzler/setup.py

import setuptools
import glob

setuptools.setup(name='brozzler',
        version='1.1.dev2',
        description='Distributed web crawling with browsers',
        url='https://github.com/nlevitt/brozzler',
        author='Noah Levitt',
        author_email='nlevitt@archive.org',
        long_description=open('README.rst').read(),
        license='Apache License 2.0',
        packages=['brozzler'],
        package_data={'brozzler': ['behaviors.d/*.js*', 'behaviors.yaml']},
        scripts=glob.glob('bin/*'),
        install_requires=[
            'PyYAML',
            'youtube-dl',
            'reppy',
            'requests',
            'websocket-client',
            'pillow',
            'surt>=0.3b2',
            'rethinkstuff',
            ],
        zip_safe=False,
        classifiers=[
            'Development Status :: 3 - Alpha',
            'Environment :: Console',
            'License :: OSI Approved :: Apache Software License',
            'Programming Language :: Python :: 3.4',
            'Topic :: Internet :: WWW/HTTP',
            'Topic :: System :: Archiving',
        ])
refactor umbra.py into controller.py and browser.py, improve class names 2014-05-20 02:42:40 -07:00			`import setuptools`
improve helper utilities 2014-05-20 16:44:13 -07:00			`import glob`
First commit of umbra 2014-01-21 06:41:46 +00:00
update readme, s/umbra/brozzler/ in most places, delete non-brozzler stuff 2015-07-13 17:09:39 -07:00			`setuptools.setup(name='brozzler',`
if youtube-dl fetches pages or makes HEAD requests, look at the responses to determine if the page is html and therefore needs to be browsed; if it doesn't need to be browsed, check if youtube-dl has already fetched it (GET request to final bounce of redirect chain that returned a 200); if not, simply fetch it 2016-04-06 17:50:48 -07:00			`version='1.1.dev2',`
update readme, s/umbra/brozzler/ in most places, delete non-brozzler stuff 2015-07-13 17:09:39 -07:00			`description='Distributed web crawling with browsers',`
			`url='https://github.com/nlevitt/brozzler',`
			`author='Noah Levitt',`
			`author_email='nlevitt@archive.org',`
remove "dev" from version number and switch README to rst 2015-09-23 22:35:26 +00:00			`long_description=open('README.rst').read(),`
Inccognito messes with currently running chromium instances, disable it 2014-01-23 18:26:20 -05:00			`license='Apache License 2.0',`
update readme, s/umbra/brozzler/ in most places, delete non-brozzler stuff 2015-07-13 17:09:39 -07:00			`packages=['brozzler'],`
giving up on using git revision in version number :( latest issue is when installing a package that calls git to compute a version number, but cwd is some other git project, you get the wrong thing 2015-09-24 00:17:33 +00:00			`package_data={'brozzler': ['behaviors.d/.js', 'behaviors.yaml']},`
fancy --version that includes git branch and timestamp of last commit if available 2014-05-29 20:43:00 -07:00			`scripts=glob.glob('bin/*'),`
goodbye requirements.txt now that we have devpi 2015-09-23 00:49:20 +00:00			`install_requires=[`
			`'PyYAML',`
			`'youtube-dl',`
			`'reppy',`
			`'requests',`
			`'websocket-client',`
			`'pillow',`
loosen surt req 2016-03-02 00:16:58 +00:00			`'surt>=0.3b2',`
goodbye requirements.txt now that we have devpi 2015-09-23 00:49:20 +00:00			`'rethinkstuff',`
			`],`
First commit of umbra 2014-01-21 06:41:46 +00:00			`zip_safe=False,`
			`classifiers=[`
update readme, s/umbra/brozzler/ in most places, delete non-brozzler stuff 2015-07-13 17:09:39 -07:00			`'Development Status :: 3 - Alpha',`
First commit of umbra 2014-01-21 06:41:46 +00:00			`'Environment :: Console',`
Update setup.py, get rid of unused dependency 2014-01-23 16:18:13 -05:00			`'License :: OSI Approved :: Apache Software License',`
basic of site/seed crawling with scoping 2015-07-09 18:36:07 -07:00			`'Programming Language :: Python :: 3.4',`
update readme, s/umbra/brozzler/ in most places, delete non-brozzler stuff 2015-07-13 17:09:39 -07:00			`'Topic :: Internet :: WWW/HTTP',`
First commit of umbra 2014-01-21 06:41:46 +00:00			`'Topic :: System :: Archiving',`
			`])`