44 Commits

Author	SHA1	Message	Date
Hunter Stern	5ccc535f51	More changes	2015-09-16 09:23:13 -07:00
Hunter Stern	3467670900	More changes for handling psu24 site	2015-09-15 18:03:08 -07:00
Hunter Stern	ea41653c44	Pulled in changes from https://github.com/nlevitt/umbra/tree/aitfive-451-alt	2015-09-15 11:53:53 -07:00
Noah Levitt	b30cc2d68b	simpler implementation for https://github.com/internetarchive/umbra/pull/42/files	2015-09-14 17:57:01 -07:00
Noah Levitt	73bbd87d5d	merge in latest from master and adjust config as needed	2015-02-02 14:52:56 -08:00
Noah Levitt	776a6dac68	Merge branch 'master' into simple-behaviors	2015-02-02 14:49:34 -08:00
Noah Levitt	48b8754f40	Merge branch 'master' into simple-behaviors	2015-02-02 14:48:26 -08:00
Noah Levitt	db759f1066	Merge pull request #32 from adam-miller/ARI-3904 ... ARI-3904 Instagram behavior to scroll past two pages, and click to enla...	2015-02-02 14:47:44 -08:00
Adam Miller	ce47461656	Making scrolling and image loading more tolerant of slow loading.	2015-01-30 16:55:53 -08:00
Noah Levitt	9e5900c61f	ARI-3956 simple behavior for usask.ca slideshows (which also required enhancing the simple behavior logic)	2015-01-27 16:03:58 -08:00
Noah Levitt	c5c642a990	support for simple behavior that clicks on elements matching configured css selector; and one such behavior for acalog sites ARI-3775	2015-01-26 16:58:12 -08:00
Noah Levitt	0647df1ab9	behaviors.yaml to configure behaviors, in preparation for "simple" behavior support	2015-01-26 16:01:53 -08:00
Hunter Stern	91f9788eb2	Add iframe css path to target id for soundcloud buttons.	2015-01-21 16:28:29 -08:00
Hunter Stern	e9451f88d8	Merge branch 'master' of github.com:internetarchive/umbra into ari-3774	2015-01-21 16:21:13 -08:00
Noah Levitt	cdcef934e7	rewrite instagram behavior to be more like a state machine; update css selectors for current instagram; refactor as a sort of singleton class for cleaner namespacing	2015-01-16 13:21:12 -08:00
Noah Levitt	ddc7064585	Merge branch 'master' into ARI-3904	2015-01-15 18:37:28 -08:00
Hunter Stern	5ea12fd053	More refinements.	2014-12-19 15:52:13 -08:00
Hunter Stern	8d225b8859	More debugging.	2014-12-19 15:13:02 -08:00
Hunter Stern	5304f2909d	Less verbose logging.	2014-12-19 14:35:11 -08:00
Hunter Stern	ae60205648	Fix for https://webarchive.jira.com/browse/ARI-4150	2014-12-19 14:17:50 -08:00
Noah Levitt	1108ef9362	Merge pull request #33 from adam-miller/ARI-4016 ... ARI-4016 - Support: embedded videos on marquette.edu	2014-11-21 15:10:53 -08:00
Adam Miller	7f8e6802de	Implementing suggestions in pull request.	2014-11-07 15:56:05 -08:00
Adam Miller	bdf3e73062	Wait until big image is loaded before clicking to next image.	2014-10-03 14:17:07 -07:00
Hunter Stern	1ee45053c5	Even more formatting changes.	2014-09-22 14:22:52 -07:00
Hunter Stern	6af3455dbf	Improve formatting.	2014-09-22 14:21:00 -07:00
Adam Miller	916f1b990e	Cleanup instagram timeout and state handling	2014-09-17 16:26:53 -07:00
Adam Miller	eb3ea95b87	Cleanup timeout logic	2014-09-17 15:26:13 -07:00
Adam Miller	5a3c8e9a05	ARI-4016 - Support: embedded videos on marquette.edu	2014-09-15 11:06:33 -07:00
Hunter Stern	a2ea2501db	More soundcloud changes.	2014-09-12 16:07:32 -07:00
Hunter Stern	e320654d1e	Allow selector to detect https and http soundcloud widget.	2014-09-12 09:56:41 -07:00
Adam Miller	7afdd7b50b	Added behavior for instagram to scroll past two pages, and click to enlarge images.	2014-09-02 17:02:30 -07:00
Noah Levitt	9d90b5830a	facebook - scroll all the to the bottom before scrolling back up to click more stuff	2014-08-01 16:53:13 -07:00
Hunter Stern	80f3a4a067	Enhancement to allow embedded soundcloud audio files to be detected	2014-07-24 16:44:05 -07:00
Noah Levitt	dd2d36328f	scroll up faster on facebook	2014-06-04 12:34:20 -07:00
Noah Levitt	93b16f28b9	improve facebook behavior: when we expect a "close" button to appear, wait for it before moving on to other actions; and when we discover a missed click target above, scroll back up to click on it	2014-05-05 18:39:16 -07:00
Noah Levitt	a62a07e6b7	change magic first line of behavior js files to a commented-out json blob, which should include the fields 'url_regex' and 'request_idle_timeout_sec'; behavior.is_finished() incorporates the custom idle timeout into its check; also rename variables in behavior scripts with umbra/UMBRA_ prefix to sort of namespace them; and add "finished" logic to facebook and vimeo behaviors (flickr needs work to support it)	2014-05-05 11:58:55 -07:00
Noah Levitt	2a9633ad77	Bunch of improvements, most importantly a default fallback behavior script which scrolls to the bottom of the page, and rearchitecting some stuff so that the behavior script can have some say on when it's finished with the page. Also some doc comments.	2014-05-04 21:33:13 -07:00
Noah Levitt	be9115fd11	to address ARI-3771 "Lasalle Facebook last scrolldown doesn't work", scroll by 200 pixels each time instead of 100 on facebook, which avoids hitting the 15 second idle timeout in my tests; also detect when unclicked targets are above the screen/viewport and not below and log it as such, instead of trying to continue scrolling down	2014-04-04 12:16:00 -07:00
Eldon	427b74ebfc	Check to see if the object has a click method before calling it	2014-03-10 14:58:16 -04:00
Noah Levitt	5637e7111f	use *rel=["theater"] to click on photos and videos that won't navigate to a new page; don't click on comments links for now, since it might interfere with other stuff; more verbose logging of click targets	2014-03-07 19:37:43 -08:00
Noah Levitt	9cb9172a4d	behavior for vimeo - click on <video> elements	2014-03-06 18:24:12 -08:00
Noah Levitt	9848c41d5f	make regexes the same that crawlman puts in crawler-beans.cxml	2014-03-06 18:23:31 -08:00
Eldon	393df3f16e	Update behaviors for facebook theater	2014-03-05 23:44:52 -05:00
Eldon	f2f78d2ced	Convert from one big json file, to js files with a regex as a comment at the top.	2014-03-05 23:19:09 -05:00