528 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Hunter Stern	8d225b8859	More debugging.	2014-12-19 15:13:02 -08:00
Hunter Stern	5304f2909d	Less verbose logging.	2014-12-19 14:35:11 -08:00
Hunter Stern	ae60205648	Fix for https://webarchive.jira.com/browse/ARI-4150	2014-12-19 14:17:50 -08:00
Hunter Stern	cf88b9968c	Merge branch 'master' of github.com:internetarchive/umbra	2014-12-12 15:59:25 -08:00
Noah Levitt	1108ef9362	Merge pull request #33 from adam-miller/ARI-4016 ... ARI-4016 - Support: embedded videos on marquette.edu	2014-11-21 15:10:53 -08:00
Adam Miller	7f8e6802de	Implementing suggestions in pull request.	2014-11-07 15:56:05 -08:00
vonrosen	8e6859ef56	Merge pull request #35 from nlevitt/amqp-socket-error ... properly handle socket.error from amqp conn.drain_events (was previously...	2014-11-03 12:09:27 -08:00
Noah Levitt	9053279b4e	change default routing key to "urls"	2014-11-03 11:54:59 -08:00
Noah Levitt	ab86426475	properly handle socket.error from amqp conn.drain_events (was previously diagnosed as error starting browser)	2014-11-03 11:54:10 -08:00
Noah Levitt	f40bd39e1a	Merge pull request #34 from dhamaniasad/patch-1 ... Update README.md	2014-10-30 19:04:24 -07:00
Asad Dhamani	9231cc2b5c	Update README.md	2014-10-31 07:02:49 +05:30
Asad Dhamani	e264f09c27	Update README.md	2014-10-29 12:42:43 +05:30
Hunter Stern	52bb02cbbe	Merge branch 'master' of github.com:internetarchive/umbra	2014-10-16 20:09:42 +00:00
vonrosen	01ed5a7d4d	Merge pull request #28 from internetarchive/ari-3940 ... Ari 3940 - prioritize scrolling all the way to the bottom	2014-10-09 21:21:02 +00:00
Adam Miller	bdf3e73062	Wait until big image is loaded before clicking to next image.	2014-10-03 14:17:07 -07:00
Hunter Stern	1ee45053c5	Even more formatting changes.	2014-09-22 14:22:52 -07:00
Hunter Stern	6af3455dbf	Improve formatting.	2014-09-22 14:21:00 -07:00
Hunter Stern	0edef7be6b	Merge remote-tracking branch 'internetarchive/master' into ari-3774	2014-09-22 14:12:59 -07:00
Adam Miller	916f1b990e	Cleanup instagram timeout and state handling	2014-09-17 16:26:53 -07:00
Adam Miller	eb3ea95b87	Cleanup timeout logic	2014-09-17 15:26:13 -07:00
Adam Miller	5a3c8e9a05	ARI-4016 - Support: embedded videos on marquette.edu	2014-09-15 11:06:33 -07:00
Hunter Stern	a2ea2501db	More soundcloud changes.	2014-09-12 16:07:32 -07:00
Hunter Stern	e320654d1e	Allow selector to detect https and http soundcloud widget.	2014-09-12 09:56:41 -07:00
Adam Miller	7afdd7b50b	Added behavior for instagram to scroll past two pages, and click to enlarge images.	2014-09-02 17:02:30 -07:00
Noah Levitt	9052fd8569	add license section	2014-09-02 16:11:49 -07:00
Noah Levitt	51d6b1a4e2	apache license	2014-09-02 16:10:00 -07:00
Hunter Stern	eb8c9faf89	Merge remote-tracking branch 'internetarchive/master'	2014-08-28 10:56:27 -07:00
Adam Miller	ce2957269f	Merge pull request #31 from nlevitt/drain-republish ... new utility queue-json, and another change to help with draining from and republishing to amqp	2014-08-26 16:58:21 -07:00
Noah Levitt	2ab767eaa9	make drain-queue output actual json instead of python dict syntax	2014-08-26 23:46:00 +00:00
Noah Levitt	fe1d9e01eb	utility queue-json to publish an arbitrary json blob to amqp	2014-08-26 23:45:42 +00:00
Hunter Stern	0e7fd93967	Merge remote-tracking branch 'internetarchive/master' into ari-3774	2014-08-26 15:12:13 -07:00
vonrosen	bbba344886	Merge pull request #29 from nlevitt/handle-bad-message ... reject (discard) bad messages	2014-08-20 08:21:29 -07:00
Noah Levitt	c886b57d3a	reject (discard) bad messages	2014-08-19 18:51:43 -07:00
Hunter Stern	b110a57938	Merge remote-tracking branch 'internetarchive/master'	2014-08-14 15:26:15 -07:00
Noah Levitt	9d90b5830a	facebook - scroll all the to the bottom before scrolling back up to click more stuff	2014-08-01 16:53:13 -07:00
Noah Levitt	dd9ef50484	suppress logging of umbraBehaviorFinished() message which is sent a lot	2014-08-01 16:22:45 -07:00
Hunter Stern	6a5d1e2266	Disable web security in chromium so iframes on different domains can be accessed by behavior javascript.	2014-07-24 16:46:06 -07:00
Hunter Stern	80f3a4a067	Enhancement to allow embedded soundcloud audio files to be detected	2014-07-24 16:44:05 -07:00
Hunter Stern	e7e82aa913	Merge branch 'master' of github.com:vonrosen/umbra into vonrosenmaster	2014-07-23 11:09:27 -07:00
Adam Miller	8e44e18053	Merge pull request #26 from nlevitt/dev ... stability!	2014-07-21 13:18:24 -07:00
Noah Levitt	ae838af25d	set amqp prefetch count to the number of urls we can handle at a time, i.e. max_active_browsers (with prefetch=1 umbra was only browsing one url at a time, after quickly burning through urls already on the queue when started)	2014-07-02 10:30:51 -07:00
Noah Levitt	6306c16698	kill -HUP to immediately close and reopen amqp consumer connection	2014-06-23 17:18:27 -07:00
Noah Levitt	02c054c284	do not wait forever for zombie websocket threads (this change should also reveal how we get these sometimes)	2014-06-20 18:13:45 -07:00
Noah Levitt	9b32f9a3d1	ugh, it was better with the default width, in spite of the ridiculous behavior.script	2014-06-20 14:40:12 -07:00
Noah Levitt	2cf69bdaff	seriously, don't try to wrap any lines, pprint	2014-06-20 14:37:33 -07:00
Noah Levitt	c6fa00812c	when dumping state on SIGQUIT, build the whole string before printing to avoid stuff getting intermingled with other logging and stuff	2014-06-20 14:33:01 -07:00
Noah Levitt	ead46d5716	more elaborate dumping of state on SIGQUIT to replace faulthandler	2014-06-20 14:05:33 -07:00
Noah Levitt	ebb14ff889	get rid of chrome_wait straggler	2014-06-18 17:31:28 -07:00
Noah Levitt	17ef9d9f28	close and reopen the amqp consumer connection only every 2.5 hours instead of every 15 minutes, because now that we have to wait for all browsers to close when we do the reconnection, it slows us down a lot	2014-06-18 14:58:44 -07:00
Noah Levitt	025db91dea	get rid of --browser-wait and --routing-key in favor of sensible defaults, some other tweaks	2014-06-11 10:58:08 -07:00