Add duplicut, C Tool to remove duplicates, without changing the order, and without getting OOM on huge wordlists.

duplicut is a C tool, highly optimized for a single task:
Removing duplicate entries from a wordlist, without changing the order, and without getting OOM on huge wordlists whose size exceeds available memory.

It's trivial to remove duplicates by sorting, but duplicut is the only tool capable of removing them without changing the order, to assist the creation of statictically optimized wordlists for password cracking purposes.
This commit is contained in:
nil0x42 2019-10-13 20:15:10 +00:00 committed by fabacab
parent aabcdb14fe
commit bd4e52fab6

View File

@ -303,6 +303,7 @@ See also [DEF CON Suggested Reading](https://www.defcon.org/html/links/book-list
* [BruteForce Wallet](https://github.com/glv2/bruteforce-wallet) - Find the password of an encrypted wallet file (i.e. `wallet.dat`).
* [CeWL](https://digi.ninja/projects/cewl.php) - Generates custom wordlists by spidering a target's website and collecting unique words.
* [duplicut](https://github.com/nil0x42/duplicut) - Quickly remove duplicates, without changing the order, and without getting OOM on huge wordlists.
* [GoCrack](https://github.com/fireeye/gocrack) - Management Web frontend for distributed password cracking sessions using hashcat (or other supported tools) written in Go.
* [Hashcat](http://hashcat.net/hashcat/) - The more fast hash cracker.
* [hate_crack](https://github.com/trustedsec/hate_crack) - Tool for automating cracking methodologies through Hashcat.