From 1484996cd5669a013b1027f95432bb5fa468c536 Mon Sep 17 00:00:00 2001 From: Greg Lindahl Date: Tue, 4 Jul 2023 12:14:41 +0000 Subject: [PATCH] add cc analysis --- README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/README.md b/README.md index ff484e0..e83d012 100644 --- a/README.md +++ b/README.md @@ -171,6 +171,9 @@ This list of tools and software is intended to briefly describe some of the most * [ArchiveSpark](https://github.com/helgeho/ArchiveSpark) - An Apache Spark framework (not only) for Web Archives that enables easy data processing, extraction as well as derivation. *(Stable)* * [Archives Unleashed Notebooks](https://github.com/archivesunleashed/notebooks) - Notebooks for working with web archives with the Archives Unleashed Toolkit, and derivatives generated by the Archives Unleashed Toolkit. *(Stable)* * [Archives Unleashed Toolkit](https://github.com/archivesunleashed/aut) - Archives Unleashed Toolkit (AUT) is an open-source platform for analyzing web archives with Apache Spark. *(Stable)* +* [Common Crawl Columnar Index](https://commoncrawl.org/tag/columnar-index/) - SQL-queryable index, with CDX info plus language classification. *(Stable)* +* [Common Crawl Web Graph](https://commoncrawl.org/category/web-graph/) - A host or domain-level graph of the web, with ranking information. *(Stable)* +* [Common Crawl Jupyter notebooks](https://github.com/commoncrawl/cc-notebooks) - A collection of notebooks using Common Crawl's various datasets. *(Stable)* * [Tweet Archvies Unleashed Toolkit](https://github.com/archivesunleashed/twut) - An open-source toolkit for analyzing line-oriented JSON Twitter archives with Apache Spark. *(In Development)* ### Quality Assurance