From 5d41023b2bd08af28cbdf0bba17cddea38d9ab04 Mon Sep 17 00:00:00 2001 From: Greg Lindahl Date: Tue, 4 Jul 2023 09:54:21 -0700 Subject: [PATCH] add cc analysis (#138) Co-authored-by: Greg Lindahl Co-authored-by: Nick Ruest --- README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/README.md b/README.md index 7f2aab9..edd9ff5 100644 --- a/README.md +++ b/README.md @@ -172,6 +172,9 @@ This list of tools and software is intended to briefly describe some of the most * [ArchiveSpark](https://github.com/helgeho/ArchiveSpark) - An Apache Spark framework (not only) for Web Archives that enables easy data processing, extraction as well as derivation. *(Stable)* * [Archives Unleashed Notebooks](https://github.com/archivesunleashed/notebooks) - Notebooks for working with web archives with the Archives Unleashed Toolkit, and derivatives generated by the Archives Unleashed Toolkit. *(Stable)* * [Archives Unleashed Toolkit](https://github.com/archivesunleashed/aut) - Archives Unleashed Toolkit (AUT) is an open-source platform for analyzing web archives with Apache Spark. *(Stable)* +* [Common Crawl Columnar Index](https://commoncrawl.org/tag/columnar-index/) - SQL-queryable index, with CDX info plus language classification. *(Stable)* +* [Common Crawl Web Graph](https://commoncrawl.org/category/web-graph/) - A host or domain-level graph of the web, with ranking information. *(Stable)* +* [Common Crawl Jupyter notebooks](https://github.com/commoncrawl/cc-notebooks) - A collection of notebooks using Common Crawl's various datasets. *(Stable)* * [Tweet Archvies Unleashed Toolkit](https://github.com/archivesunleashed/twut) - An open-source toolkit for analyzing line-oriented JSON Twitter archives with Apache Spark. *(In Development)* * [Web Data Commons](http://webdatacommons.org/) - Structured data extracted from Common Crawl. *(Stable)*