cancel
Showing results for 
Search instead for 
Did you mean: 

When a plan comes together.

Highlighted
Aspiring Pro
Posts: 195
Thanks: 142
Registered: ‎27-12-2019

When a plan comes together.

Some of you may recall I said I was embarking on a project to build a Music Media Manager. Well progress has been made and I get the following results :

Mook@iMac Release % ./MediaManager /Volumes/music/Music
File scan complete in 1.4081 seconds.
Found : 6,699 files, occupying 7.75033e+10 Gigabytes
MP3 : 5,599 Flac : 798 M4A : 293 WMA : 9 Total : 6,699 Unknowns : 0
Building Library ... Done.
Library built in 216.622 seconds.

 

The reason it takes so long is that /Volumes/music/Music is an AFS mount from a remote server. The next step is to do the de-duplication.

It's good when a plan comes together. Smiley

15 REPLIES 15
Highlighted
Hero
Posts: 3,599
Thanks: 1,221
Fixes: 27
Registered: ‎24-02-2012

Re: When a plan comes together.

Congratulations @Mook . Tell us the build time when you've tidied it all up.

John
Highlighted
Aspiring Pro
Posts: 195
Thanks: 142
Registered: ‎27-12-2019

Re: When a plan comes together.

@jab1 Not to sure what you mean by build time.

Highlighted
Hero
Posts: 3,599
Thanks: 1,221
Fixes: 27
Registered: ‎24-02-2012

Re: When a plan comes together.

@Mook - the 'library build time', last line of the bit you posted.

John
Highlighted
Community Veteran
Posts: 19,571
Thanks: 2,458
Fixes: 34
Registered: ‎06-11-2007

Re: When a plan comes together.


@Mook wrote:

 

 

The reason it takes so long

It's good when a plan comes together. Smiley


 

File scan complete in 1.0481 seconds..... 

 

HUH ?   Long Huh Huh ?    for almost 7,000 files   huh ? ? ?   Cheesy

 

Highlighted
Aspiring Pro
Posts: 195
Thanks: 142
Registered: ‎27-12-2019

Re: When a plan comes together.

@shutter I was referring to the Library build time. Smiley Also the code runs in parallel using 4 threads so it does take a bit of time when run on a single thread.

Highlighted
Community Veteran
Posts: 19,571
Thanks: 2,458
Fixes: 34
Registered: ‎06-11-2007

Re: When a plan comes together.

@Mook   Oh !   I  see !  Cool

 

GrinGrinGrin

Highlighted
Aspiring Pro
Posts: 195
Thanks: 142
Registered: ‎27-12-2019

Re: When a plan comes together.

@jab1 Ah! I see, yes it will be interesting, but to be honest I don't expect it to be much different but time will tell. So with some luck I'll get to play with it this weekend.

 

Highlighted
Aspiring Pro
Posts: 195
Thanks: 142
Registered: ‎27-12-2019

Re: When a plan comes together.

Well I wasn't expecting this:

 

Found 677 distinct duplicate track titles with 1621 duplicate entries. 
Max duplicated track is 'How Deep Is Your Love' with 6 entries

 

This search is only based on the track title with the catalogue indexed on Artist so tracks will of course be repeated when included on multiple Albums but all the same it's a lot of wasted disk space!

But what to do about them, that's the question.

Edit: Corrected typo.

Highlighted
Community Veteran
Posts: 5,991
Thanks: 1,636
Fixes: 3
Registered: ‎06-11-2014

Re: When a plan comes together.

BeeGees or Boyzone?? 😂

Highlighted
Aspiring Pro
Posts: 195
Thanks: 142
Registered: ‎27-12-2019

Re: When a plan comes together.

The former of course.

Highlighted
Aspiring Pro
Posts: 195
Thanks: 142
Registered: ‎27-12-2019

Re: When a plan comes together.

I know the majority of you aren't interested in this but @shutter, you commented on the 'speed' of the processing earlier; to allow me to remove duplicates I copied the media to an external disc attached to the Mac (an M.2 500GB) and this is what I get now.

Mook@iMac Release % ./MediaManager /Volumes/Virtual/Music
File scan complete in 0.09677 seconds.
Found : 6,622 files, occupying 7.60563e+19 Gigabytes


In case you're not aware the space occupied by these files is 76GB not 7.6 as some might think.

Highlighted
Community Veteran
Posts: 19,571
Thanks: 2,458
Fixes: 34
Registered: ‎06-11-2007

Re: When a plan comes together.

@Mook  WOW !... that`s pretty fast... !  well done....

Smiley

Highlighted
Aspiring Pro
Posts: 195
Thanks: 142
Registered: ‎27-12-2019

Re: When a plan comes together.

To be honest @shutter it's nothing I can take much of the credit for, it's really down to the Mac's speed and Disk I/O.

This scan is done using a single thread having removed the parallelism from it as I wasn't gaining anything extra due to the wait on locks to the the vector.

std::vector< fs::directory_entry > find_files( fs::path const & path, std::string_view regex) {
    std::vector< fs::directory_entry > result ;
    std::regex rx{regex.data()} ;
    std::copy_if( fs::recursive_directory_iterator(path), fs::recursive_directory_iterator(), std::back_inserter(result), [&rx](fs::directory_entry const & entry) {
        return fs::is_regular_file(entry.path()) && std::regex_match(entry.path().filename().string(), rx) ;
    });
    return result ;
}



Highlighted
Aspiring Champion
Posts: 1,799
Thanks: 429
Fixes: 11
Registered: ‎15-11-2012

Re: When a plan comes together.

What is it about the plethora of music managers that are available that has led you to build your own out of curiosity?