Git-Mirrors/brozzler
1
0
Fork 0
mirror of https://github.com/internetarchive/brozzler.git synced 2025-04-20 23:56:34 -04:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

work around vaguely familiar bug

Browse Source 
--- Logging error ---
Traceback (most recent call last):
  File "/opt/brozzler-ve3/lib/python3.7/site-packages/rethinkdb/errors.py", line 219, in __iter__
    for sub in next(itr):
StopIteration
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/opt/python/3.7.1/lib/python3.7/logging/__init__.py", line 1034, in emit
    msg = self.format(record)
  File "/opt/python/3.7.1/lib/python3.7/logging/__init__.py", line 880, in format
    return fmt.format(record)
  File "/opt/python/3.7.1/lib/python3.7/logging/__init__.py", line 619, in format
    record.message = record.getMessage()
  File "/opt/python/3.7.1/lib/python3.7/logging/__init__.py", line 380, in getMessage
    msg = msg % self.args
  File "/opt/brozzler-ve3/lib/python3.7/site-packages/doublethink/rethinker.py", line 40, in __repr__
    return '<RethinkerWrapper{}>'.format(repr(self.wrapped))
  File "/opt/brozzler-ve3/lib/python3.7/site-packages/rethinkdb/ast.py", line 130, in __repr__
    return "<RqlQuery instance: %s >" % str(self)
  File "/opt/brozzler-ve3/lib/python3.7/site-packages/rethinkdb/ast.py", line 127, in __str__
    return qp.print_query()
  File "/opt/brozzler-ve3/lib/python3.7/site-packages/rethinkdb/errors.py", line 175, in print_query
    return ''.join(self.compose_term(self.root))
  File "/opt/brozzler-ve3/lib/python3.7/site-packages/rethinkdb/errors.py", line 224, in __iter__
    for sub in token:
  File "/opt/brozzler-ve3/lib/python3.7/site-packages/rethinkdb/errors.py", line 219, in __iter__
    for sub in next(itr):
  File "/opt/brozzler-ve3/lib/python3.7/site-packages/rethinkdb/errors.py", line 224, in __iter__
    for sub in token:
  File "/opt/brozzler-ve3/lib/python3.7/site-packages/rethinkdb/errors.py", line 219, in __iter__
    for sub in next(itr):
  File "/opt/brozzler-ve3/lib/python3.7/site-packages/rethinkdb/errors.py", line 224, in __iter__
    for sub in token:
  File "/opt/brozzler-ve3/lib/python3.7/site-packages/rethinkdb/errors.py", line 224, in __iter__
    for sub in token:
  File "/opt/brozzler-ve3/lib/python3.7/site-packages/rethinkdb/errors.py", line 224, in __iter__
    for sub in token:
  [Previous line repeated 1 more time]
RuntimeError: generator raised StopIteration
Call stack:
  File "/opt/python/3.7.1/lib/python3.7/threading.py", line 885, in _bootstrap
    self._bootstrap_inner()
  File "/opt/python/3.7.1/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/opt/python/3.7.1/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/brozzler-ve3/lib/python3.7/site-packages/brozzler/worker.py", line 416, in _brozzle_site_thread_target
    self.brozzle_site(browser, site)
  File "/opt/brozzler-ve3/lib/python3.7/site-packages/brozzler/worker.py", line 378, in brozzle_site
    site, page, outlinks)
  File "/opt/brozzler-ve3/lib/python3.7/site-packages/brozzler/frontier.py", line 399, in scope_and_schedule_outlinks
    self.logger.trace('running query: %r', reql)
  File "/opt/brozzler-ve3/lib/python3.7/site-packages/brozzler/__init__.py", line 68, in _logger_trace
    self._log(logging.TRACE, msg, args, **kwargs)
  File "/opt/python/3.7.1/lib/python3.7/logging/__init__.py", line 1519, in _log
    self.handle(record)
  File "/opt/python/3.7.1/lib/python3.7/logging/__init__.py", line 1529, in handle
    self.callHandlers(record)
  File "/opt/python/3.7.1/lib/python3.7/logging/__init__.py", line 1591, in callHandlers
    hdlr.handle(record)
  File "/opt/brozzler-ve3/lib/python3.7/site-packages/brozzler/__init__.py", line 87, in _logging_handler_handle
    self.emit(record)
Unable to print the message and arguments - possible formatting error.
Use the traceback above to help find the error.
This commit is contained in:
Noah Levitt 2019-10-17 00:31:14 -07:00
parent 0a9213e0cc
commit c64f45728d
1 changed files with 3 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
brozzler/frontier.py
View File

@ -396,7 +396,9 @@ class RethinkDbFrontier:
self.logger.debug(
'inserting/replacing batch of %s pages', len(batch))
reql = self.rr.table('pages').insert(batch, conflict='replace')
self.logger.trace('running query: %r', reql)
self.logger.trace(
'running query self.rr.table("pages").insert(%r, '
'conflict="replace")', batch)
result = reql.run()
except Exception as e:
self.logger.error(