distutils was removed beginning in Python 3.12, but it's used at
runtime by rethinkdb 2.4.9. setuptools provides a copy of distutils,
so we should make sure to install it when we're on Python 3.12 or
newer until we're able to upgrade to a version of rethinkdb that
no longer needs it.
See: https://www.python.org/downloads/release/python-3120/
* Makefile improvements + pre-commit hook
* update make target in CI
* fix CI more
* .gitignore update
* couple more Makefile refinements
* make target-version explicit on ruff import sorting
* __init__.py: rework imports
Although doublethink is an optional dependency to allow brozzler to be
used as a library without it, in practice we had some mandatory import
statements that prevented brozzler from being imported without it.
This fixes that by gating off some of the imports and exports.
If doublethink is available, brozzler works as it is now. But if it
isn't, we make a few changes:
* brozzler.worker, brozzler.cli and brozzler.model reexports are
disabled
* One brozzler.cli function, which is used outside brozzler's own cli,
has been moved into brozzler's __init__.py. For compatibility, it's
reexported from brozzler.cli.
* Make tz-aware datetime of the epoch with stdlib
* Only import yt-dlp if we're using it
* ydl: never try if extra missing
* cli: use worker's yt-dlp check
---------
Co-authored-by: Alex Dempsey <avdempsey@archive.org>
This adds a new commandline flag allowing the worker ID to be specified.
If present, it will be added to the global context so that it will be
included in every logging statement.
Previously, we only had some indirect values to tie logging statements
to specific workers, so this should make it easier to follow.
This adds a commandline option which enables log level prefixing.
These prefixes enable log level-based filtering in journalctl when
present so long as logs are going to the journal, and
`SyslogLevelPrefix=` is set to `true` (which it is by default).
For documentation: https://manpages.debian.org/testing/libsystemd-dev/sd-daemon.3.en.html
There are a few minor changes here compared to black; it flagged
unnecessary string concatenations, and has slightly different
opinions on line length.
This adds two CI runs: a quick one that happens for every pull
request and merge to master, and a longer one that happens daily.
This also adds a new installation group to setup.py because the
`easy` group isn't currently installable, and some of the dependencies
specified there need to be present for the tests to run.
This ports the logging from `logging` to `structlog`. This updates
all of the logger instantiations along with all of the places
`logging` was called. Data that was being inlined into log statements
has been broken out so that it's now structured arguments to the
log statements instead.
On macOS, we can find Chrome even if it's installed in a non-default
path by querying `mdfind`. This is the CLI entrypoint to Spotlight,
and we can use it to look up applications using their unique bundle
identifiers.
If `mdfind` fails to find anything, this falls back to the hardcoded
paths. This should ensure this still works if Spotlight indexing is
off, but Chrome is in the default path.
