Adam Miller
|
9cf20f195c
|
Removing first run ui checks
|
2014-05-02 17:37:10 -07:00 |
|
Noah Levitt
|
89e41e7c82
|
remove exception raised for testing
|
2014-04-07 11:45:54 -07:00 |
|
Noah Levitt
|
aacb886b62
|
ARI-3814 try to recover from rabbitmq communication problems
|
2014-04-07 11:45:12 -07:00 |
|
Eldon
|
4e72cbae58
|
Merge pull request #18 from nlevitt/ari-3771
to address ARI-3771 "Lasalle Facebook last scrolldown doesn't work", scr...
|
2014-04-04 16:04:38 -04:00 |
|
Noah Levitt
|
be9115fd11
|
to address ARI-3771 "Lasalle Facebook last scrolldown doesn't work", scroll by 200 pixels each time instead of 100 on facebook, which avoids hitting the 15 second idle timeout in my tests; also detect when unclicked targets are above the screen/viewport and not below and log it as such, instead of trying to continue scrolling down
|
2014-04-04 12:16:00 -07:00 |
|
Noah Levitt
|
da975bc586
|
thread dump on SIGQUIT a la java
|
2014-04-03 21:19:08 -07:00 |
|
Eldon
|
427b74ebfc
|
Check to see if the object has a click method before calling it
|
2014-03-10 14:58:16 -04:00 |
|
Noah Levitt
|
3fd792fddb
|
lengthen timeouts and improve timeout handling; log js console messages from browser
|
2014-03-07 19:39:27 -08:00 |
|
Noah Levitt
|
5637e7111f
|
use *rel=["theater"] to click on photos and videos that won't navigate to a new page; don't click on comments links for now, since it might interfere with other stuff; more verbose logging of click targets
|
2014-03-07 19:37:43 -08:00 |
|
Noah Levitt
|
5a7a24083f
|
simplify checking for *.js
|
2014-03-07 11:29:43 -08:00 |
|
Noah Levitt
|
a30b5d8dd2
|
only reset idle timer on Network.requestWillBeSent instead of all events (otherwise long-running videos keep the browser open unnecessarily)
|
2014-03-06 18:35:04 -08:00 |
|
Noah Levitt
|
9d9014c864
|
start the hard stop timer
|
2014-03-06 18:32:30 -08:00 |
|
Noah Levitt
|
52db581a3c
|
restore logging
|
2014-03-06 18:25:46 -08:00 |
|
Noah Levitt
|
12d66982d1
|
only load behaviors files named like *.js (avoids vim .swp files and stuff); tweak logging
|
2014-03-06 18:25:35 -08:00 |
|
Noah Levitt
|
9cb9172a4d
|
behavior for vimeo - click on <video> elements
|
2014-03-06 18:24:12 -08:00 |
|
Noah Levitt
|
9848c41d5f
|
make regexes the same that crawlman puts in crawler-beans.cxml
|
2014-03-06 18:23:31 -08:00 |
|
Eldon
|
393df3f16e
|
Update behaviors for facebook theater
|
2014-03-05 23:44:52 -05:00 |
|
Eldon
|
f2f78d2ced
|
Convert from one big json file, to js files with a regex as a comment at the top.
|
2014-03-05 23:19:09 -05:00 |
|
Noah Levitt
|
b763d6550f
|
remove unused function
|
2014-02-25 14:26:10 -08:00 |
|
Noah Levitt
|
11da122ec2
|
remove old commented out line of code
|
2014-02-18 13:20:18 -08:00 |
|
Noah Levitt
|
b96d8856d4
|
create temp dir for user profile rather than rely on --temp-profile
|
2014-02-14 19:45:16 -08:00 |
|
Noah Levitt
|
b4846e1063
|
scrolldown seems to get everything for flickr and facebook at the moment
|
2014-02-14 17:57:04 -08:00 |
|
Noah Levitt
|
28282641f2
|
add a little logging
|
2014-02-14 15:18:10 -08:00 |
|
Noah Levitt
|
2368688fbe
|
Merge remote-tracking branch 'eldondev/master' into nlevitt-master (add behaviors)
Conflicts:
umbra/umbra.py
|
2014-02-14 15:10:23 -08:00 |
|
Noah Levitt
|
3389c5a66d
|
remove some extraneous debug logging
|
2014-02-13 18:36:08 -08:00 |
|
Eldon
|
fe15932c26
|
Click on photos in gallery behavior
|
2014-02-13 13:37:08 -05:00 |
|
Eldon
|
af01fcbcfe
|
Add more flickr behavior
|
2014-02-13 13:32:34 -05:00 |
|
Eldon
|
445288d5e7
|
First few behaviors
|
2014-02-13 13:10:19 -05:00 |
|
Noah Levitt
|
f69edd5a87
|
handle multiple clients, browsers
|
2014-02-13 01:59:09 -08:00 |
|
Eldon
|
bdf00cc515
|
Refactor to pull Chrome execution inside of umbra, simplify some things
|
2014-02-12 19:31:03 -05:00 |
|
Noah Levitt
|
f9d56d3071
|
formatting change only - indent with 4 spaces
|
2014-02-10 20:45:18 -08:00 |
|
Noah Levitt
|
02fbe725cb
|
cache parent url metadata and send back via amqp with child urls
|
2014-02-10 20:40:06 -08:00 |
|
Eldon
|
8afe7d90a2
|
Replace js evaluation with direct page navigation, add default for dump_queue
|
2014-01-28 00:10:31 -05:00 |
|
Noah Levitt
|
e9f6408881
|
improved browser ownership locking
|
2014-01-27 19:24:28 -08:00 |
|
Noah Levitt
|
ae2c3668a9
|
get rid of multiple window handling, add more logging, clean shutdown on ctrl+c, other tweaks
|
2014-01-27 19:15:31 -08:00 |
|
Eldon
|
bd0183058d
|
Inccognito messes with currently running chromium instances, disable it
|
2014-01-23 18:26:20 -05:00 |
|
Eldon
|
0c25bbb4c3
|
Raise exception if we are never able to connect to the browser
|
2014-01-23 16:28:52 -05:00 |
|
Eldon
|
ca3d6d5999
|
Raise an exception if no browser starts
|
2014-01-23 16:24:53 -05:00 |
|
Eldon
|
4852fbf29f
|
Update setup.py, get rid of unused dependency
|
2014-01-23 16:18:13 -05:00 |
|
Eldon
|
fdfaf496a9
|
Incognito mode prevents local caching
|
2014-01-23 19:25:55 +00:00 |
|
Eldon
|
09b0060137
|
Add additional checks for weird chrome behavior, serialize requests
|
2014-01-23 18:39:17 +00:00 |
|
Eldon
|
6dc20e660f
|
Remove debugging output, improve support scripts
|
2014-01-22 18:41:00 +00:00 |
|
Eldon
|
4e38a142d4
|
Some refactor/testing and utility scripts
|
2014-01-22 18:03:02 +00:00 |
|
Eldon
|
e0a2917b81
|
Add amqp connectivity
|
2014-01-22 07:38:03 +00:00 |
|
Eldon
|
5a6749777b
|
It appears to be possible for the debug port to open without any tabs, so we should handle that.
|
2014-01-22 02:38:51 +00:00 |
|
Eldon
|
428d6cb7da
|
Rework executable script so that it uses a main
|
2014-01-22 02:30:12 +00:00 |
|
Eldon
|
7b219ab011
|
Fix parameter passing and work with chromiums wrapper stuff
|
2014-01-22 02:22:16 +00:00 |
|
Eldon
|
db9eee5f2b
|
Should be full python 3 now
|
2014-01-22 01:32:41 +00:00 |
|
Eldon
|
5900e90e7d
|
Moving to python3
|
2014-01-21 22:53:28 +00:00 |
|
Eldon
|
f90b4db690
|
Add additional option to control invoked executable and wait time
|
2014-01-21 22:47:49 +00:00 |
|