cancel
Showing results for 
Search instead for 
Did you mean: 

De-duplicating music collection.

Community Veteran
Posts: 4,758
Thanks: 1,048
Fixes: 27
Registered: 16-10-2014

De-duplicating music collection.

I wrote some code to de-duplicate my music collection so I can consolidate it in a single mount point and out of one of the directories that contained 4004 MP3s I've successfully found 32 true duplicates, so I'm quite pleased with that and done without using tags or file names.  Grin
3 REPLIES
Community Veteran
Posts: 13,921
Thanks: 514
Fixes: 7
Registered: 01-08-2007

Re: De-duplicating music collection.

Checksums and hashes are incredibly handy aren't they!
I need a new signature... i'm bored of the old one!
Moderator
Moderator
Posts: 16,541
Thanks: 1,792
Fixes: 125
Registered: 06-04-2007

Re: De-duplicating music collection.

Wish I could write my own code Tongue
I use Auslogics Duplicate File Finder. It's quite fast and does a good job.

Forum Moderator and Customer
Courage is resistance to fear, mastery of fear, not absence of fear - Mark Twain
He who feared he would not succeed sat still

Community Veteran
Posts: 4,758
Thanks: 1,048
Fixes: 27
Registered: 16-10-2014

Re: De-duplicating music collection.

Quote from: 7up
Checksums and hashes are incredibly handy aren't they!

Well it's a variation on that using Echonest as a hash of the file can't be used for obvious reasons.
I create a hash of the fingerprint created from the scan as this is much smaller than the fingerprint, and store it and the tag details in MongoDB. Then using an aggregate function I get a list of the duplicates. This database is currently maintained by my own media streamer daemon and its web UI but I still need toincorporate this feature in to the streamer daemon.