Right now we expect these YouTube tests to fail for reasons unrelated to
yt-dlp. We still want to try them, but we won't count them towards
capture failures.
This runs every time we get a new yt-dlp version - we test to see if
this script is able to download at least 3/5 out of a set of videos
we've defined. If it succeeds, we go ahead and automatically merge
the new yt-dlp version into the qa branch so that we can test
further.
This adds a tag-to-release Actions config based around uv.
This is triggered by pushing a tag with a new version; it will
automatically kick off this job, which will publish the new
version to PyPI on completion. We can push that tag from a PR
or directly to master.
At the moment, this doesn't do anything to automatically create
a GitHub release from the tag; we can do that manually for now,
but if we're interested I can add something to automatically
generate the release too.
We don't need to provide a token to uv to publish; instead, we
just need to configure the repo for PyPI access using this:
https://docs.pypi.org/trusted-publishers/adding-a-publisher/
distutils was removed beginning in Python 3.12, but it's used at
runtime by rethinkdb 2.4.9. setuptools provides a copy of distutils,
so we should make sure to install it when we're on Python 3.12 or
newer until we're able to upgrade to a version of rethinkdb that
no longer needs it.
See: https://www.python.org/downloads/release/python-3120/
* Makefile improvements + pre-commit hook
* update make target in CI
* fix CI more
* .gitignore update
* couple more Makefile refinements
* make target-version explicit on ruff import sorting
* __init__.py: rework imports
Although doublethink is an optional dependency to allow brozzler to be
used as a library without it, in practice we had some mandatory import
statements that prevented brozzler from being imported without it.
This fixes that by gating off some of the imports and exports.
If doublethink is available, brozzler works as it is now. But if it
isn't, we make a few changes:
* brozzler.worker, brozzler.cli and brozzler.model reexports are
disabled
* One brozzler.cli function, which is used outside brozzler's own cli,
has been moved into brozzler's __init__.py. For compatibility, it's
reexported from brozzler.cli.
* Make tz-aware datetime of the epoch with stdlib
* Only import yt-dlp if we're using it
* ydl: never try if extra missing
* cli: use worker's yt-dlp check
---------
Co-authored-by: Alex Dempsey <avdempsey@archive.org>
This adds a new commandline flag allowing the worker ID to be specified.
If present, it will be added to the global context so that it will be
included in every logging statement.
Previously, we only had some indirect values to tie logging statements
to specific workers, so this should make it easier to follow.
This adds a commandline option which enables log level prefixing.
These prefixes enable log level-based filtering in journalctl when
present so long as logs are going to the journal, and
`SyslogLevelPrefix=` is set to `true` (which it is by default).
For documentation: https://manpages.debian.org/testing/libsystemd-dev/sd-daemon.3.en.html
