Awesome-Mirrors/awesome-web-archiving
1
0
Fork 0
mirror of https://github.com/iipc/awesome-web-archiving.git synced 2025-03-13 04:36:30 -04:00
Code Issues Packages Projects Releases Wiki Activity

Added warcdb

Browse Source
This commit is contained in:
Ed Summers 2023-10-16 12:11:08 -04:00
parent 4b12cc7b32
commit f7e3eb61a3
No known key found for this signature in database
GPG Key ID: D012FF557AFFF80A
1 changed files with 1 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
README.md
View File

@ -148,6 +148,7 @@ This list of tools and software is intended to briefly describe some of the most
* [tikalinkextract](https://github.com/httpreserve/tikalinkextract) - Extract hyperlinks as a seed for web archiving from folders of document types that can be parsed by Apache Tika (Golang, Apache Tika Server). *(In Development)*
* [wasapi-downloader](https://github.com/sul-dlss/wasapi-downloader) - Java command line application to download crawls from WASAPI. *(Stable)*
* [Warchaeology](https://nlnwa.github.io/warchaeology/) - Warchaeology is a collection of tools for inspecting, manipulating, deduplicating and validating WARC-files. *Stable*
* [warcdb](https://github.com/florents-Tselai/warcdb) - A command line utility (Python) for importing WARC files into a SQLite database. *(Stable)*
* [warcdedupe](https://gitlab.com/taricorp/warcdedupe) - WARC deduplication tool (and WARC library) written in Rust. (In Development)
* [WarcPartitioner](https://github.com/helgeho/WarcPartitioner) - Partition (W)ARC Files by MIME Type and Year. *(Stable)*
* [warcrefs](https://github.com/arcalex/warcrefs) - Web archive deduplication tools. *Stable*