Noah Levitt
|
55fad80553
|
UmbraWorker.send_to_chrome() - central place to send message to chrome via websocket
|
2014-05-05 12:26:39 -07:00 |
|
Noah Levitt
|
a62a07e6b7
|
change magic first line of behavior js files to a commented-out json blob, which should include the fields 'url_regex' and 'request_idle_timeout_sec'; behavior.is_finished() incorporates the custom idle timeout into its check; also rename variables in behavior scripts with umbra/UMBRA_ prefix to sort of namespace them; and add "finished" logic to facebook and vimeo behaviors (flickr needs work to support it)
|
2014-05-05 11:58:55 -07:00 |
|
Noah Levitt
|
2a9633ad77
|
Bunch of improvements, most importantly a default fallback behavior script which scrolls to the bottom of the page, and rearchitecting some stuff so that the behavior script can have some say on when it's finished with the page. Also some doc comments.
|
2014-05-04 21:33:13 -07:00 |
|
Noah Levitt
|
8679ee0ea7
|
disable google analytics by setting a breakpoint in www.google-analytics.com/analytics.js and replacing the content of that script when the breakpoint is hit
|
2014-05-02 18:30:28 -07:00 |
|
Adam Miller
|
9cf20f195c
|
Removing first run ui checks
|
2014-05-02 17:37:10 -07:00 |
|
Noah Levitt
|
89e41e7c82
|
remove exception raised for testing
|
2014-04-07 11:45:54 -07:00 |
|
Noah Levitt
|
aacb886b62
|
ARI-3814 try to recover from rabbitmq communication problems
|
2014-04-07 11:45:12 -07:00 |
|
Eldon
|
4e72cbae58
|
Merge pull request #18 from nlevitt/ari-3771
to address ARI-3771 "Lasalle Facebook last scrolldown doesn't work", scr...
|
2014-04-04 16:04:38 -04:00 |
|
Noah Levitt
|
be9115fd11
|
to address ARI-3771 "Lasalle Facebook last scrolldown doesn't work", scroll by 200 pixels each time instead of 100 on facebook, which avoids hitting the 15 second idle timeout in my tests; also detect when unclicked targets are above the screen/viewport and not below and log it as such, instead of trying to continue scrolling down
|
2014-04-04 12:16:00 -07:00 |
|
Noah Levitt
|
da975bc586
|
thread dump on SIGQUIT a la java
|
2014-04-03 21:19:08 -07:00 |
|
Eldon
|
427b74ebfc
|
Check to see if the object has a click method before calling it
|
2014-03-10 14:58:16 -04:00 |
|
Noah Levitt
|
3fd792fddb
|
lengthen timeouts and improve timeout handling; log js console messages from browser
|
2014-03-07 19:39:27 -08:00 |
|
Noah Levitt
|
5637e7111f
|
use *rel=["theater"] to click on photos and videos that won't navigate to a new page; don't click on comments links for now, since it might interfere with other stuff; more verbose logging of click targets
|
2014-03-07 19:37:43 -08:00 |
|
Noah Levitt
|
5a7a24083f
|
simplify checking for *.js
|
2014-03-07 11:29:43 -08:00 |
|
Noah Levitt
|
a30b5d8dd2
|
only reset idle timer on Network.requestWillBeSent instead of all events (otherwise long-running videos keep the browser open unnecessarily)
|
2014-03-06 18:35:04 -08:00 |
|
Noah Levitt
|
9d9014c864
|
start the hard stop timer
|
2014-03-06 18:32:30 -08:00 |
|
Noah Levitt
|
52db581a3c
|
restore logging
|
2014-03-06 18:25:46 -08:00 |
|
Noah Levitt
|
12d66982d1
|
only load behaviors files named like *.js (avoids vim .swp files and stuff); tweak logging
|
2014-03-06 18:25:35 -08:00 |
|
Noah Levitt
|
9cb9172a4d
|
behavior for vimeo - click on <video> elements
|
2014-03-06 18:24:12 -08:00 |
|
Noah Levitt
|
9848c41d5f
|
make regexes the same that crawlman puts in crawler-beans.cxml
|
2014-03-06 18:23:31 -08:00 |
|
Eldon
|
393df3f16e
|
Update behaviors for facebook theater
|
2014-03-05 23:44:52 -05:00 |
|
Eldon
|
f2f78d2ced
|
Convert from one big json file, to js files with a regex as a comment at the top.
|
2014-03-05 23:19:09 -05:00 |
|
Noah Levitt
|
b763d6550f
|
remove unused function
|
2014-02-25 14:26:10 -08:00 |
|
Noah Levitt
|
11da122ec2
|
remove old commented out line of code
|
2014-02-18 13:20:18 -08:00 |
|
Noah Levitt
|
b96d8856d4
|
create temp dir for user profile rather than rely on --temp-profile
|
2014-02-14 19:45:16 -08:00 |
|
Noah Levitt
|
b4846e1063
|
scrolldown seems to get everything for flickr and facebook at the moment
|
2014-02-14 17:57:04 -08:00 |
|
Noah Levitt
|
28282641f2
|
add a little logging
|
2014-02-14 15:18:10 -08:00 |
|
Noah Levitt
|
2368688fbe
|
Merge remote-tracking branch 'eldondev/master' into nlevitt-master (add behaviors)
Conflicts:
umbra/umbra.py
|
2014-02-14 15:10:23 -08:00 |
|
Noah Levitt
|
3389c5a66d
|
remove some extraneous debug logging
|
2014-02-13 18:36:08 -08:00 |
|
Eldon
|
fe15932c26
|
Click on photos in gallery behavior
|
2014-02-13 13:37:08 -05:00 |
|
Eldon
|
af01fcbcfe
|
Add more flickr behavior
|
2014-02-13 13:32:34 -05:00 |
|
Eldon
|
445288d5e7
|
First few behaviors
|
2014-02-13 13:10:19 -05:00 |
|
Noah Levitt
|
f69edd5a87
|
handle multiple clients, browsers
|
2014-02-13 01:59:09 -08:00 |
|
Eldon
|
bdf00cc515
|
Refactor to pull Chrome execution inside of umbra, simplify some things
|
2014-02-12 19:31:03 -05:00 |
|
Noah Levitt
|
f9d56d3071
|
formatting change only - indent with 4 spaces
|
2014-02-10 20:45:18 -08:00 |
|
Noah Levitt
|
02fbe725cb
|
cache parent url metadata and send back via amqp with child urls
|
2014-02-10 20:40:06 -08:00 |
|
Eldon
|
8afe7d90a2
|
Replace js evaluation with direct page navigation, add default for dump_queue
|
2014-01-28 00:10:31 -05:00 |
|
Noah Levitt
|
e9f6408881
|
improved browser ownership locking
|
2014-01-27 19:24:28 -08:00 |
|
Noah Levitt
|
ae2c3668a9
|
get rid of multiple window handling, add more logging, clean shutdown on ctrl+c, other tweaks
|
2014-01-27 19:15:31 -08:00 |
|
Eldon
|
bd0183058d
|
Inccognito messes with currently running chromium instances, disable it
|
2014-01-23 18:26:20 -05:00 |
|
Eldon
|
0c25bbb4c3
|
Raise exception if we are never able to connect to the browser
|
2014-01-23 16:28:52 -05:00 |
|
Eldon
|
ca3d6d5999
|
Raise an exception if no browser starts
|
2014-01-23 16:24:53 -05:00 |
|
Eldon
|
4852fbf29f
|
Update setup.py, get rid of unused dependency
|
2014-01-23 16:18:13 -05:00 |
|
Eldon
|
fdfaf496a9
|
Incognito mode prevents local caching
|
2014-01-23 19:25:55 +00:00 |
|
Eldon
|
09b0060137
|
Add additional checks for weird chrome behavior, serialize requests
|
2014-01-23 18:39:17 +00:00 |
|
Eldon
|
6dc20e660f
|
Remove debugging output, improve support scripts
|
2014-01-22 18:41:00 +00:00 |
|
Eldon
|
4e38a142d4
|
Some refactor/testing and utility scripts
|
2014-01-22 18:03:02 +00:00 |
|
Eldon
|
e0a2917b81
|
Add amqp connectivity
|
2014-01-22 07:38:03 +00:00 |
|
Eldon
|
5a6749777b
|
It appears to be possible for the debug port to open without any tabs, so we should handle that.
|
2014-01-22 02:38:51 +00:00 |
|
Eldon
|
428d6cb7da
|
Rework executable script so that it uses a main
|
2014-01-22 02:30:12 +00:00 |
|