brozzler

Git-Mirrors/brozzler

Fork 0

mirror of https://github.com/internetarchive/brozzler.git synced 2025-04-20 23:56:34 -04:00

Commit Graph

Select branches

Hide Pull Requests

ARI-5259

ARI-5294

ARI-5407

ARI-5409

ARI-5460

ARI-5517

ARI-5671

ARI-5747

Thorium-support

adam/add_new_page_cli

adam/behavior_check_refactor

adam/disable-chrome-model-downloads

adam/log-behavior-status

adam/log_long_brozzle_sessions

adam/new_claim_sites_query

avdempsey/optional-doublethink-ytdlp

avdempsey/use-black

behavior-refactor

bigger-window

blocks-shrink

bmiller/proxy_select

brofurb

browserless

brozzler_ia

bump-version-to-1.5.23

chrome130fixes

copyright-2018

fix-travis

galgeek-patch-1

galgeek-patch-2

galgeek-patch-3

galgeek-websocket-url-timeout

gmiller/2950-skip-ytdlp

gretchen/readme-reqs-update

limit_recaptures

master

metrics_plus_proxy_retries

moma-behavior

noYoutubeDash

optimizes-rethinkdb-load-query

pageDown

proxyrack

pypi_1dot6dot2

qa

qa-20191008

qa-20250416

refactor-browsing

simpledo

skipDashManifest

skip_youtube_dl

stealth-too

travis

travis-debugging

wt-376-yt-user-page-fix

x11-support

ydl-second

yt-dlp

yt-dlp-format-fix

yt-dlp-log

yt-dlp-mp4-again

yt-dlp-skip-live

yt-dlp-timeout

yt-dlp-too

#1

#10

#10

#100

#101

#102

#103

#107

#108

#109

#110

#111

#113

#114

#114

#115

#116

#117

#118

#119

#12

#120

#121

#122

#127

#129

#13

#130

#131

#132

#133

#134

#135

#136

#137

#138

#14

#14

#141

#142

#143

#144

#145

#146

#147

#148

#149

#15

#150

#151

#152

#153

#154

#155

#157

#158

#16

#160

#162

#163

#164

#164

#165

#166

#167

#168

#169

#17

#170

#171

#172

#173

#174

#175

#176

#177

#178

#179

#18

#180

#181

#182

#183

#184

#186

#187

#188

#189

#19

#190

#191

#192

#193

#194

#195

#196

#197

#199

#2

#20

#200

#202

#203

#204

#205

#206

#207

#208

#209

#21

#210

#210

#211

#212

#216

#218

#218

#219

#219

#22

#22

#220

#220

#222

#223

#225

#229

#229

#23

#232

#232

#233

#237

#238

#239

#24

#240

#241

#242

#243

#244

#245

#246

#247

#248

#249

#25

#252

#252

#253

#254

#255

#255

#257

#258

#259

#26

#260

#261

#262

#263

#264

#265

#266

#268

#269

#27

#270

#271

#273

#274

#276

#277

#278

#279

#279

#280

#281

#282

#283

#284

#285

#286

#287

#288

#288

#289

#29

#29

#290

#290

#291

#293

#294

#295

#296

#297

#298

#299

#3

#30

#300

#301

#302

#303

#305

#306

#307

#308

#310

#310

#311

#312

#313

#313

#314

#315

#316

#318

#319

#319

#320

#321

#322

#323

#324

#325

#326

#327

#328

#329

#331

#332

#333

#334

#335

#336

#337

#338

#339

#34

#340

#341

#342

#343

#344

#345

#346

#347

#348

#348

#36

#38

#39

#4

#40

#41

#42

#44

#46

#47

#48

#49

#49

#5

#50

#51

#52

#53

#54

#55

#56

#56

#57

#58

#59

#6

#60

#61

#62

#63

#64

#65

#66

#67

#68

#69

#7

#70

#71

#72

#73

#74

#75

#77

#78

#81

#81

#82

#83

#84

#84

#85

#87

#88

#89

#9

#90

#92

#93

#93

#94

#95

#96

#97

#98

#98

#99

1.16.10

1.1b10

1.1b11

1.1b12

1.1b2

1.1b3

1.1b4

1.1b5

1.1b6

1.1b7

1.1b8

1.1b9

1.2

1.3

1.4

1.5.6

15db75e825 try saving only Brotli conifg Barbara Miller 2016-09-30 10:55:13 -07:00
cf82d266bd disable SDCH? Barbara Miller 2016-09-30 11:17:30 -07:00
84947c1148 make data_dir Barbara Miller 2016-09-30 09:30:39 -07:00
abc7c0bfca simplify to troubleshoot Barbara Miller 2016-09-30 00:18:41 -07:00
8b63555f73 better variable name Barbara Miller 2016-09-29 23:58:51 -07:00
022a4a60ca read/write chromium local state Barbara Miller 2016-09-29 23:13:15 -07:00
b5f2077491 clean up Browser dirs and add flags_location Barbara Miller 2016-09-29 20:58:10 -07:00
0b9518bc5c read/write chromium local state Barbara Miller 2016-09-29 23:13:15 -07:00
016367bc46 clean up Browser dirs and add flags_location Barbara Miller 2016-09-29 20:58:10 -07:00
a8430baf1b better still facebook login logging Barbara Miller 2016-09-16 13:39:00 -07:00
76d8516898 better facebook login logging Barbara Miller 2016-09-15 15:13:51 -07:00
c9a0e1ed24 facebook login troubleshooting Barbara Miller 2016-09-14 20:24:01 -07:00
d398d2d006 disable downloads & kccfed behavior Barbara Miller 2016-06-25 15:05:37 -07:00
73454d2ac7 clickGetPDFs for kansascityfed Barbara Miller 2016-05-08 19:55:50 -07:00
d7bd19a7f6 kcfed test Barbara Miller 2016-05-27 22:20:36 -07:00
3c5c6a714a clickGetPDFs.template Barbara Miller 2016-05-21 19:27:05 -07:00
3d0c6b7e72 huffpo slideshow custom behavior Barbara Miller 2016-05-17 17:40:41 -07:00
a147ed1940 multiclicks behavior Barbara Miller 2016-05-09 22:50:40 -07:00
0d5274ac64 rm old WIP item Barbara Miller 2016-10-12 14:10:28 -07:00
d82b40be68 fix typo Noah Levitt 2016-10-06 18:20:41 -07:00
c736f2e703 Merge pull request #19 from ato/schema-file-fix Noah Levitt 2016-10-05 09:02:18 -07:00
0fe2ef9387 Ensure job_schema.yaml is installed by pip Alex Osborne 2016-10-05 15:45:11 +11:00
59a15d7f5c bump dev version after merging pull requests Noah Levitt 2016-10-04 14:43:43 -07:00
f5e138c7eb Merge pull request #18 from ato/user-agent Noah Levitt 2016-10-04 14:39:50 -07:00
692c1c48e1 Merge pull request #17 from ato/validate-conf Noah Levitt 2016-10-04 14:13:47 -07:00
eafc65938b Enforce at least one seed Alex Osborne 2016-10-05 04:31:22 +11:00
743b5a4347 Add user_agent option Alex Osborne 2016-10-05 04:25:09 +11:00
5ac67fe513 Validate job conf against a Cerberus schema Alex Osborne 2016-10-04 21:16:16 +11:00
15633be612 finish vagrant-brozzler-new-job.py Noah Levitt 2016-10-03 18:17:35 -07:00
5bdf916a78 Merge pull request #16 from galgeek/logCaptchaHref Noah Levitt 2016-10-03 18:12:07 -07:00
c857647198 log location.href always Barbara Miller 2016-09-29 13:21:36 -07:00
2f6ac120e3 log facebook captcha Barbara Miller 2016-09-28 22:05:13 -07:00
de5c520ad7 pin psutil version, too Noah Levitt 2016-10-03 17:12:00 -07:00
f220707aa4 pin pillow to version 3.3.0, primarily because we have a wheel in devpi for that version Noah Levitt 2016-10-03 17:06:58 -07:00
df676223aa Merge branch 'master' of github.com:internetarchive/brozzler Noah Levitt 2016-10-03 15:03:22 -07:00
0e096dd4e4 don't try to read the browser's cookie database if the browser hasn't been started (which can happen if the page is simply fetched rather than browsed because it's not html) Noah Levitt 2016-10-03 15:03:08 -07:00
6de75717a4 Merge pull request #15 from ato/headless-flash Noah Levitt 2016-10-02 13:12:29 -07:00
a31ee2bcef Document how to use Flash with Headless Chromium Alex Osborne 2016-10-02 23:40:41 +11:00
64cde55ba9 Noah's ExtraHTTPHeaders encoding fix Barbara Miller 2016-09-30 17:18:53 -07:00
87a4d93a1c Merge pull request #13 from ato/headless Noah Levitt 2016-09-30 16:09:59 -07:00
4af9225256 Merge branch 'extraHeaders' into qa Barbara Miller 2016-09-30 14:54:18 -07:00
ec85687a70 fixing mis-copy Barbara Miller 2016-09-30 14:53:25 -07:00
46c48ef070 fixing mis-copy Barbara Miller 2016-09-30 14:53:25 -07:00
0b84abc4ec just test gzip,deflate after all Barbara Miller 2016-09-30 14:08:00 -07:00
8accfcb8d9 test identity encoding Barbara Miller 2016-09-30 12:36:10 -07:00
707f0a133f test extra headers Barbara Miller 2016-09-30 12:34:05 -07:00
043aebd9f0 correct local_state Barbara Miller 2016-09-30 11:20:54 -07:00
1e2884db17 try saving only Brotli conifg Barbara Miller 2016-09-30 10:55:13 -07:00
6cda2814ea disable SDCH? Barbara Miller 2016-09-30 11:17:30 -07:00
f0035f610e Merge branch 'disableBrotli' into qa Barbara Miller 2016-09-30 09:35:23 -07:00
a7937193b3 make data_dir Barbara Miller 2016-09-30 09:30:39 -07:00
80883c9784 Merge branch 'master' into qa Noah Levitt 2016-09-30 08:51:17 -07:00
659d46afdd simplify to troubleshoot Barbara Miller 2016-09-30 00:18:41 -07:00
006ab21e3a simplify to troubleshoot Barbara Miller 2016-09-30 00:18:41 -07:00
cd670e8d01 better variable name Barbara Miller 2016-09-29 23:58:51 -07:00
261fdf8a33 better variable name Barbara Miller 2016-09-29 23:58:51 -07:00
84753c87af read/write chromium local state Barbara Miller 2016-09-29 23:13:15 -07:00
efa214964e clean up Browser dirs and add flags_location Barbara Miller 2016-09-29 20:58:10 -07:00
1b7b6b2c0e read/write chromium local state Barbara Miller 2016-09-29 23:13:15 -07:00
e5c55969e6 clean up Browser dirs and add flags_location Barbara Miller 2016-09-29 20:58:10 -07:00
bfd4c1f8c6 document a bunch of job settings Noah Levitt 2016-09-29 16:15:44 -07:00
3e23345ef4 log location.href always Barbara Miller 2016-09-29 13:21:36 -07:00
8c9a9c5666 starting on documenting job configuration Noah Levitt 2016-09-29 12:03:16 -07:00
0e5762c547 log facebook captcha Barbara Miller 2016-09-28 22:05:13 -07:00
9062cb7b5b s/Chrome/Chromium/ Alex Osborne 2016-09-29 08:03:27 +10:00
a4f908b22a Add notes on Headless Chromium Alex Osborne 2016-09-29 07:55:39 +10:00
2462efc4ed replace vagrant-brozzler-new-site with python script that fills in default options and passes through others Noah Levitt 2016-09-22 01:47:23 +01:00
cc9517cb45 add missing rethinkdb config file to ansible config Noah Levitt 2016-09-22 01:45:28 +01:00
10d17ce7c3 Merge pull request #12 from galgeek/logFBlog Noah Levitt 2016-09-19 21:18:17 -07:00
253122d061 new script runs brozzler-new-site queues a new site to brozzle on the vagrant brozzler deployment Noah Levitt 2016-09-16 16:35:44 -07:00
8f44eac2f3 better logs for facebook logins Barbara Miller 2016-09-16 15:42:19 -07:00
82cc6af366 better still facebook login logging Barbara Miller 2016-09-16 13:39:00 -07:00
98ea743489 better facebook login logging Barbara Miller 2016-09-15 15:13:51 -07:00
07353bcc1f facebook login troubleshooting Barbara Miller 2016-09-14 20:24:01 -07:00
38af0f347b working on including pywb in vagrant environment (not finished) Noah Levitt 2016-09-14 17:08:00 -07:00
c864499a64 starting to create a framework for testing Noah Levitt 2016-09-14 17:06:49 -07:00
be27b4e16e header comment tweak Noah Levitt 2016-09-14 17:04:32 -07:00
85073ab82b new prog "brozzler-wayback" runs monkey-patched pywb Noah Levitt 2016-09-14 17:04:01 -07:00
caadb2beff Merge branch 'master' into qa Noah Levitt 2016-08-29 09:59:55 -07:00
1c5c9417d2 avoid "Uncaught TypeError: Cannot read property 'querySelectorAll' of undefined" from outlinks script Noah Levitt 2016-08-25 13:10:30 -07:00
ed7e01210d little readme fix Noah Levitt 2016-08-12 17:02:41 -07:00
197a571033 Py3 update to README Mat Kelly 2016-08-11 15:08:48 -04:00
c9bc9fb67d for vagrant, static ansible inventory file, add brozzler-webconsole Noah Levitt 2016-08-10 18:41:23 -07:00
f671cf4f11 add info to display of jobless sites in brozzler-webconsole; fix creation of "least_hops" index on the rethinkdb table "pages" Noah Levitt 2016-08-09 11:24:58 -07:00
74b229cfb0 add arguments --webconsole-address --webconsole-port --pywb-address and change default ports Noah Levitt 2016-08-09 10:43:52 -07:00
94a8e70226 list jobless sites on brozzler-webconsole front page Noah Levitt 2016-08-08 17:44:41 -07:00
4fa1571bc5 run brozzler-webconsole inside brozzler-easy Noah Levitt 2016-08-08 17:43:38 -07:00
531b26aabb add section about brozzler-easy to the readme Noah Levitt 2016-08-05 18:28:30 -07:00
c04bf85f4e add --help to brozzler-webconsole Noah Levitt 2016-08-05 18:19:15 -07:00
10c90431e6 Merge branch 'master' into qa Noah Levitt 2016-08-05 17:13:09 -07:00
ba6b342e28 fix exception happening now that we have binary data in rethinkdb (the cookie db) "TypeError: <binary, 7168 bytes, '53 51 4c 69 74 65...'> is not JSON serializable" Noah Levitt 2016-08-05 17:12:22 -07:00
a211cc0514 dev version number again Noah Levitt 2016-08-04 17:34:58 -07:00
ae63369c3c another version for pypi 1.1b5 Noah Levitt 2016-08-04 17:33:47 -07:00
20f9934dd9 avoid "Uncaught RangeError: Maximum call stack size exceeded" compiling outlinks Noah Levitt 2016-08-04 17:33:06 -07:00
7734399a22 back to a dev version number Noah Levitt 2016-08-04 16:00:42 -07:00
57c0d84fbd bump version to 1.1b4 for pypi upload 1.1b4 Noah Levitt 2016-08-04 15:55:56 -07:00
e62055d7d6 logging tweak Noah Levitt 2016-08-04 15:54:05 -07:00
65d97caa9a install brozzler.webconsole package Noah Levitt 2016-07-29 12:56:10 -05:00
feff7a8d05 Merge branch 'master' into qa Noah Levitt 2016-07-28 20:32:33 -05:00
cfc18e6845 add docstring to _chain_chrome_messages, remove debug logging, tweak name of websock thread Noah Levitt 2016-07-28 20:29:11 -05:00