brozzler

Git-Mirrors/brozzler

Fork 0

mirror of https://github.com/internetarchive/brozzler.git synced 2025-04-20 23:56:34 -04:00

Commit Graph

Select branches

Hide Pull Requests

ARI-5259

ARI-5294

ARI-5407

ARI-5409

ARI-5460

ARI-5517

ARI-5671

ARI-5747

Thorium-support

adam/add_new_page_cli

adam/behavior_check_refactor

adam/disable-chrome-model-downloads

adam/log-behavior-status

adam/log_long_brozzle_sessions

adam/new_claim_sites_query

avdempsey/optional-doublethink-ytdlp

avdempsey/use-black

behavior-refactor

bigger-window

blocks-shrink

bmiller/proxy_select

brofurb

browserless

brozzler_ia

bump-version-to-1.5.23

chrome130fixes

copyright-2018

fix-travis

galgeek-patch-1

galgeek-patch-2

galgeek-patch-3

galgeek-websocket-url-timeout

gmiller/2950-skip-ytdlp

gretchen/readme-reqs-update

limit_recaptures

master

metrics_plus_proxy_retries

moma-behavior

noYoutubeDash

optimizes-rethinkdb-load-query

pageDown

proxyrack

pypi_1dot6dot2

qa

qa-20191008

qa-20250416

refactor-browsing

simpledo

skipDashManifest

skip_youtube_dl

stealth-too

travis

travis-debugging

wt-376-yt-user-page-fix

x11-support

ydl-second

yt-dlp

yt-dlp-format-fix

yt-dlp-log

yt-dlp-mp4-again

yt-dlp-skip-live

yt-dlp-timeout

yt-dlp-too

#1

#10

#10

#100

#101

#102

#103

#107

#108

#109

#110

#111

#113

#114

#114

#115

#116

#117

#118

#119

#12

#120

#121

#122

#127

#129

#13

#130

#131

#132

#133

#134

#135

#136

#137

#138

#14

#14

#141

#142

#143

#144

#145

#146

#147

#148

#149

#15

#150

#151

#152

#153

#154

#155

#157

#158

#16

#160

#162

#163

#164

#164

#165

#166

#167

#168

#169

#17

#170

#171

#172

#173

#174

#175

#176

#177

#178

#179

#18

#180

#181

#182

#183

#184

#186

#187

#188

#189

#19

#190

#191

#192

#193

#194

#195

#196

#197

#199

#2

#20

#200

#202

#203

#204

#205

#206

#207

#208

#209

#21

#210

#210

#211

#212

#216

#218

#218

#219

#219

#22

#22

#220

#220

#222

#223

#225

#229

#229

#23

#232

#232

#233

#237

#238

#239

#24

#240

#241

#242

#243

#244

#245

#246

#247

#248

#249

#25

#252

#252

#253

#254

#255

#255

#257

#258

#259

#26

#260

#261

#262

#263

#264

#265

#266

#268

#269

#27

#270

#271

#273

#274

#276

#277

#278

#279

#279

#280

#281

#282

#283

#284

#285

#286

#287

#288

#288

#289

#29

#29

#290

#290

#291

#293

#294

#295

#296

#297

#298

#299

#3

#30

#300

#301

#302

#303

#305

#306

#307

#308

#310

#310

#311

#312

#313

#313

#314

#315

#316

#318

#319

#319

#320

#321

#322

#323

#324

#325

#326

#327

#328

#329

#331

#332

#333

#334

#335

#336

#337

#338

#339

#34

#340

#341

#342

#343

#344

#345

#346

#347

#348

#348

#36

#38

#39

#4

#40

#41

#42

#44

#46

#47

#48

#49

#49

#5

#50

#51

#52

#53

#54

#55

#56

#56

#57

#58

#59

#6

#60

#61

#62

#63

#64

#65

#66

#67

#68

#69

#7

#70

#71

#72

#73

#74

#75

#77

#78

#81

#81

#82

#83

#84

#84

#85

#87

#88

#89

#9

#90

#92

#93

#93

#94

#95

#96

#97

#98

#98

#99

1.16.10

1.1b10

1.1b11

1.1b12

1.1b2

1.1b3

1.1b4

1.1b5

1.1b6

1.1b7

1.1b8

1.1b9

1.2

1.3

1.4

1.5.6

a2ac3a0374 logging.warn is deprecated and replaced by logging.warning Vangelis Banos 2019-05-14 12:10:59 +00:00
ee8ef23f0c fix mistake in job-conf.rst Noah Levitt 2019-04-30 10:49:48 -07:00
411b3f266a bump version after merge Noah Levitt 2019-04-09 22:07:51 +00:00
d4386491df

Merge pull request #151 from nlevitt/no-cerberus-normalize Noah Levitt 2019-04-09 15:06:17 -07:00
5385232b40 don't attempt cerberus normalization Noah Levitt 2019-04-09 01:45:37 -07:00
8dfd92cf7f fix this utility Noah Levitt 2019-04-09 01:44:14 -07:00
433b201b52 use logging.warning() to quiet py37 warnings Noah Levitt 2019-04-09 01:43:38 -07:00
dfd9d9ecdd omfg Noah Levitt 2019-04-04 17:22:15 -07:00
fd0fe811e9 so little output from chromium-browser :( Noah Levitt 2019-04-04 16:09:21 -07:00
55541be9e9 let's see chromium output inside brozzler-worker Noah Levitt 2019-04-04 15:11:24 -07:00
58d1d1c429 chromium-browser with no args isn't dying at start Noah Levitt 2019-04-04 14:38:29 -07:00
473e891fb4 not sure if --disable-extensions did something Noah Levitt 2019-04-04 13:34:45 -07:00
6d145c87c8 chromium-browser --disable-extensions ? Noah Levitt 2019-04-04 13:24:12 -07:00
0d46d8ce19 still trying to figure out what's up with chromium Noah Levitt 2019-04-04 13:15:17 -07:00
45ac12117a maybe Xvnc.log will tell us something Noah Levitt 2019-04-04 13:09:02 -07:00
8303fd3ab3 guessing DISPLAY was the issue here Noah Levitt 2019-04-04 12:50:50 -07:00
899794f2da debug what's going on with chromium in travis Noah Levitt 2019-04-04 12:38:46 -07:00
9459ed40d0 fix typo Noah Levitt 2019-04-04 12:38:41 -07:00
68ce9eac76 debugging travis-ci is a slow process Noah Levitt 2019-04-02 13:05:36 -07:00
85c6ac0ab2 fix next travis-ci problem Noah Levitt 2019-04-02 12:05:08 -07:00
06e072a716 update some dependencies Noah Levitt 2019-04-02 17:58:35 +00:00
8b6e5cbfb9 new option brozzler-purge --finished-before=... Noah Levitt 2019-04-02 17:58:13 +00:00
9c658cddf7 fix a couple of svc definitions Noah Levitt 2019-03-24 16:06:36 -07:00
48bb03418f daemontools Noah Levitt 2019-03-23 00:26:39 -07:00
18b4a26db6 porting ansible config to xenial Noah Levitt 2019-03-22 23:50:46 -07:00
19522aff85 adjusting ansible config for xenial Noah Levitt 2019-03-19 16:37:13 -07:00
d4f8bc768f trying to make this work with xenial for travis Noah Levitt 2019-03-18 16:38:23 -07:00
f2a9908395 travis only has py 3.7 for xenial Noah Levitt 2019-03-18 16:20:54 -07:00
d729c8d0d5 use yaml.safe_load() Noah Levitt 2019-03-18 15:49:44 -07:00
6f5f090c33 test py 3.7 Noah Levitt 2019-03-18 15:49:03 -07:00
ef981706f4 fix rethinkdb dependency version Noah Levitt 2019-03-18 15:08:36 -07:00
c686fc7443 Merge branch 'master' into qa Noah Levitt 2019-03-14 20:06:12 +00:00
61274ae994 peg to working doublethink Noah Levitt 2019-03-14 20:03:17 +00:00
7d5bb4b5d4

Merge pull request #148 from vbanos/disk-cache Noah Levitt 2019-02-12 14:39:49 -08:00
9c48a6fa11 Use disk cache params only on Chrome.start Vangelis Banos 2019-02-12 20:59:08 +00:00
adeca823dd Remove stale comment Vangelis Banos 2019-02-12 07:21:44 +00:00
31e611771e Improve disk cache options Vangelis Banos 2019-02-07 07:42:45 +00:00
c288c9ae98 Add disk cache options to Chrome Vangelis Banos 2019-02-06 16:22:10 +00:00
809ea3885f

Merge pull request #147 from galgeek/bye_simpleclicks Noah Levitt 2019-01-14 13:48:48 -08:00
f6ffb4acea update (C) Barbara Miller 2019-01-10 16:11:24 -08:00
46f64fb5e3 Merge branch 'bye_simpleclicks' into qa Barbara Miller 2019-01-10 16:08:10 -08:00
9001156b54 rm simpleclicks.js.j2 mouseovers.js.j2 Barbara Miller 2019-01-10 15:58:38 -08:00
770ea6de1e no more simpleclicks/mouseovers Barbara Miller 2019-01-10 15:54:47 -08:00
2aade83017 Merge branch 'instaInterval' into qa Barbara Miller 2018-12-21 17:13:42 -08:00
db5df9cdd4 interval 2000 Barbara Miller 2018-12-21 17:13:22 -08:00
e1ceb87ca2

Merge pull request #146 from nlevitt/https-redirect Barbara Miller 2018-12-21 15:26:04 -08:00
a74f46dc53 least surprise on http/https seed redirects Noah Levitt 2018-12-21 15:17:31 -08:00
8b93c078b7 Merge branch 'instaInterval' into qa Barbara Miller 2018-12-21 14:41:27 -08:00
93e5769428 instagram interval 1000 Barbara Miller 2018-12-21 14:41:04 -08:00
6b8e597a43 bump version after merge Noah Levitt 2018-12-20 11:30:49 -08:00
0a08c01461

Merge pull request #145 from galgeek/no-skipIframes Noah Levitt 2018-12-20 11:30:28 -08:00
bf8bbfba27 Merge branch 'no-skipIframes' into qa Barbara Miller 2018-12-20 11:25:54 -08:00
047b46bc4e back out now unnecessary updates Barbara Miller 2018-12-20 11:25:06 -08:00
d8f97e7b3f no current need for skipIframes with new try/catch Barbara Miller 2018-12-20 11:24:30 -08:00
034f7938c4 catch common exception in default behavior Noah Levitt 2018-12-20 10:46:05 -08:00
2cd64811b3 bump version after merge Noah Levitt 2018-12-17 15:10:26 -08:00
d8c9dd2ff4

Merge pull request #144 from galgeek/umbraBehavior18q4 Noah Levitt 2018-12-17 15:09:52 -08:00
921261df39 Merge branch 'umbraBehavior18q4' into qa Barbara Miller 2018-12-17 15:05:46 -08:00
4a0d95277f update umbraBehavior Barbara Miller 2018-12-17 15:04:36 -08:00
cbd6f0f90a Merge branch 'insta18q4' into qa Barbara Miller 2018-12-13 17:29:36 -08:00
425d44bf4a updates for jina2 Barbara Miller 2018-12-13 15:13:03 -08:00
6c21a9f773 iframe option and other instagram updates Barbara Miller 2018-12-09 14:25:59 -08:00
15870e6010 avoid IndexError Noah Levitt 2018-12-13 15:49:38 -08:00
b577fe3c36 log browser uncaught exceptions at debug level Noah Levitt 2018-12-13 15:45:35 -08:00
c50e9637ae Merge branch 'insta18q4' into qa Barbara Miller 2018-12-09 14:26:38 -08:00
cb0c0f51ef iframe option and other instagram updates Barbara Miller 2018-12-09 14:25:59 -08:00
b447063099 Merge branch 'master' into qa Noah Levitt 2018-11-29 14:52:32 -08:00
ebcc063fe2 bump version after merge Noah Levitt 2018-11-29 14:52:11 -08:00
898756690f

Merge pull request #142 from nlevitt/service-worker jkafader 2018-11-29 13:42:59 -08:00
9c27e829aa

Merge pull request #136 from nlevitt/revert-time-limit jkafader 2018-11-29 12:29:35 -08:00
983ed7bc60 Merge branch 'service-worker' into qa Noah Levitt 2018-11-27 16:07:35 -08:00
db62402be8 fix tests Noah Levitt 2018-11-27 14:35:00 -08:00
b204e9aec1 Merge branch 'service-worker' into qa Barbara Miller 2018-11-27 12:58:47 -08:00
f63947cfe9 fetch service worker script with proper headers Noah Levitt 2018-11-27 12:35:33 -08:00
bc6a2f4b95 handle http auth (#138) Barbara Miller 2018-11-16 15:10:30 -08:00
574af7846e bump version after merge Noah Levitt 2018-11-16 15:10:46 -08:00
e2b2542d4a handle http auth (#138) Barbara Miller 2018-11-16 15:10:30 -08:00
a3ab6ebaaa inline server 401 code Noah Levitt 2018-11-15 11:34:33 -08:00
a885ac1f1c test_page_interstitial_exception only Barbara Miller 2018-11-14 16:53:01 -08:00
05fab8b909 change time limit enforcement Noah Levitt 2018-11-12 16:21:38 -08:00
9db7744f2c Merge branch 'master' into qa Noah Levitt 2018-11-01 15:57:52 -07:00
15610fa990 fail quickly if browser dies at startup Noah Levitt 2018-11-01 15:57:03 -07:00
27ba877932 Merge branch 'master' into qa Noah Levitt 2018-10-29 17:45:09 -07:00
1073431f76 handle exceptions extracting links Noah Levitt 2018-10-29 17:42:25 -07:00
68b4d8ba52 test_page_interstitial_exception_live Barbara Miller 2018-10-25 09:42:48 -07:00
ddd61bdeeb test better? Barbara Miller 2018-10-24 21:35:10 -07:00
5349b3c40e auth request? Barbara Miller 2018-10-24 17:39:25 -07:00
74551f77ab add 401 url Barbara Miller 2018-10-24 17:06:28 -07:00
2f5deab2c6 fix rebase error Barbara Miller 2018-10-24 16:59:42 -07:00
ef6e615c7f page.interstitial exception test Barbara Miller 2018-10-17 17:23:50 -07:00
d6730cba8b return outlinks = [] Barbara Miller 2018-09-27 16:22:12 -07:00
462418ec99 simplest exception ok Barbara Miller 2018-09-27 16:19:08 -07:00
0455e0d393 except log and return [] Barbara Miller 2018-09-24 17:33:38 -07:00
914d72ba60 tidier, better exception handling? Barbara Miller 2018-09-21 17:46:19 -07:00
732b1b8c2f expect exception PageInterstitialShown Barbara Miller 2018-09-20 17:23:49 -07:00
6350e4ee03 raise exception PageInterstitialShown Barbara Miller 2018-09-20 17:22:52 -07:00
165a34787f add exception PageIntersitialShown Barbara Miller 2018-09-20 17:21:49 -07:00
cc08554878 Browsing Exception for Page.interstitialShown Barbara Miller 2018-09-20 15:30:12 -07:00
af85f28908 fix reported chromium crash by removing argument Noah Levitt 2018-10-22 14:28:31 -07:00
f3f9505657 Merge branch 'pageInterstitialShown' into qa Barbara Miller 2018-10-17 17:24:39 -07:00