CLI tool that fast checks if your bundle contains multiple versions of the same package, only by looking in package.json.

Detecting near-duplicate videos by aggregating features from intermediate CNN layers

Find similar audio files easily

A collection of publicly available bug reports

Advanced similarity and duplicate source code proof of concept for our research efforts.

Quickly remove duplicates, without changing the order, and without getting OOM on huge wordlists.

Advanced similarity and duplicate source code at scale.

Advanced Duplicate File Finder for Python

Duplicates finder for various source code formats.

🚀 Cloc & duplicate code checker tool

Wanna get DRY? Static analysis tool for detecting repeat code.

Vidupe is a program that can find duplicate and similar video files. V1.211 released on 2019-09-18, Windows exe here:

Fast Near-Duplicate Image Search and Delete using pHash, t-SNE and KDTree.

File Checksum Integrity Verifier & Duplicate File Finder written in C++ Qt

Duplicate file finder written in Nim

CLI utility to find near duplicate images and remove all but the best copy of it.

OpenStaticAnalyzer is a source code analyzer tool, which can perform deep static analysis of the source code of complex systems.

Engine for analysis of Siegfried export files and DROID CSV. The tool has three purposes, break the export into its components and store them within a SQLite database; create additional columns to augment the output where useful; and query the SQLite database, outputting results in a readable form useful for analysis by researchers and archivists within digital preservation departments in memory institutions. The tool will find duplicates, unidentified files, blacklisted objects, character encoding issues, and more.

A command-line tool which automate the deletion of duplicate files based on their hash or perceptual-hash.

Java DSL for (online) deduplication

Findm is a python script to find duplicate file copies in a given directory.

Command line utility to remove exact duplicate files.

Find, remove and avoid duplicates with dugu: The Duplicates Guru

Find duplicates and similar images in a folder

A node command line tool which can be used to find duplicate images in a directory even if the images differ in resolution, contrast, brightness or color.

Perceptual GIF hashing for easily finding near-duplicate GIFs.

Duplicate frame detector/decimator filter for AviSynth+

A simple Python 2 code for duplicate image detection, using deep CNNs as backbone.

🐻 The decluttering deduplicator

Python tool to help you knockout duplicate entries from multiple files and generate the final output