How to organize and enjoy your digital music library

Preface:

The methods described below are an amalgamation of articles I’ve read online, advice I have gotten from friends, and plenty of good old trial and error.  So, if you think I copied you, I probably did.  Thanks for the good idea. 

Also, it goes without saying but I’ll say it anyway.  Back up your music.  Back up your everything.  Back up to at least two locations.  One external drive in your house and in case your house burns down, you’ll want an online back up too.  I like CrashPlan but there are a million options to choose from.

This article is long and describes some techniques that are a bit tricky.  If something is unclear (or wrong), please let me know and I will do my best to clarify.

Introduction:

At a social event the other night, I made the rather controversial statement that I find itunes to be the best software for organizing music.  It was immediately clear that I needed to change the subject to avoid a severe verbal confrontation.  I think with enough time I could have gotten my point across but I just didn’t want to get into that rather lengthy discussion.  Instead I’ve written it all down here.  As a bonus, since I am about to begin preparation of my annual SXSW mixes and I thought it could be of interest to some out there how I actually go about boiling down 1000+ tracks to the 10 or so 30 minute mixes.

Before I get into the whole process though, I should say a couple things about the selection of itunes as my software of choice.  First off, I never ever ever ever buy music from the Itunes store.  Why would you pay the same or more for music that is copyright protected when you can get mp3s of the same or better quality on Bandcamp, Beatport, Amazon, Juno, etc…?  You shouldn’t.  Simple.  Second, I never use itunes for podcasts.  I almost solely use my phone for podcasts and frankly nothing can touch Downcast.  Downcast is hands down the best podcast app available.  If you listen to podcasts just pay the couple bucks and get it.

Most importantly, I really tried to find an alternative to itunes.  I went so far as to get an Android phone and go through the hassle of fully customizing my own Foobar setup.  After all the setup, I absolutely loved Foobar.  You can customize it however you want and it runs super light on your computer.  It opens so fast it’s ridiculous.  In the end though, syncing with my phone was the problem. I tried all sorts of crazy scripting to try and get Foobar and the many different android apps to sync with all the features I wanted.  As much as I hate to admit it the synchronizing provided by the iPhone and Itunes are simply superior and that finally lead me to, with much hesitation, return my android and go back to the iphone and itunes.  The syncing, specifically of song ratings, is really the main issues and as you’ll see, for me at least, it is a critical component.

Some people might say why not just use Spotify or some other streaming service.  Then you don’t need to organize anything.  Probably true but not all music is on those services.  More importantly though, I’m a collector and I want my collection.

Okay, here we go.  This is going to be long.  We start off easy and get progressively more and more OCD. 

Step 1: Let itunes organize your library

I know I know, you don’t want to do that because you say itunes puts your music in some really weird directory that you can’t find it.  That’s dumb.  Go to Preferences\Advanced and select the folder you want you’re music to go to.  For reasons I’ll get into later, I suggest leaving it as the default (\user\Music\iTunes\iTunes Media). 

Now click the box to “Keep iTunes Media folder organized”.  This places all the tracks in folders by artist and then subdivides into subfolders by album.  The key here is that if you change the artist of an mp3 in itunes, it will move the actual file to the folder of the corrected artist and album.  This means you never have to go in and change file names and move files around yourself.  I don’t know if anyone does that anymore but back in the days of Winamp and Napster a decade or more ago I remember spending hours renaming and sorting files.  Never again.

Next, click the box “Copy files to Itunes media fold when adding to library”.  This is super helpful.  Whenever you download some random track from the internet and you save it to some random place on your computer like the Downloads folder or the Desktop no need to try and cut and paste it into the right subfolder of \Music\.  Just drag it into itunes.  If you can play the track in itunes, then it has been copied to the appropriate subdirectory and is actually playing from that location.  You can then just delete the version on your desktop.

Now drag all the music from all the disparate directories on your computer into itunes and itunes will automatically copy everything to the main music directory you set above.  You can go delete the files from all those random directories now.  Before you empty the trash though spot check a few tracks to make sure you got everything.

Step 2: Get rid of songs whose original file cannot be found

When you deleted all those files that were in locations all over your computer, itunes didn’t get the memo.  That means there may be songs in itunes that will appear to have multiple copies.  Invariably the first one you click will be the link to the deleted version and you will get an error that the file can’t be found and then a little exclamation point will appear next to the file.  Wouldn’t it be nice if you could just get rid of all of those at once?  You can but you’ll need hacker status.

Make a playlist and call it “Real Songs” and drag your entire library into it.  Only the songs that actually have files will be added to that playlist and now all the songs missing files will have exclamation marks next to them.  Next, make a smart playlist called “Bad Links” and setup the criterion to be all songs whose “Playlist” “is not” “Real Songs”.  You will now have a playlist of all the songs with bad links.  Select all the songs in the “Bad Links” playlist and Right Click “Uncheck Selection”.

Now go back to the full library and click the Check column to sort by checked and unchecked songs.  All the unchecked songs should be the songs with no file behind them.  You can select them all and delete them in one go.  Pretty cool huh?

Step 3: Get rid of duplicates

Once you have collected all your files into the same library and removed all the bad links, you are still going to have a bunch of duplicates.  Itunes has a great way to find them.  Depending on your version of itunes you’ll want to go to either the File or View menu and find “Show Duplicates Items”.  Don’t click it.  If you did, just go back and select “Display all items”.  We’ll use that later but for now it’s way too broad.  Instead, hold down the Shift key and go back to that same menu.  Now you will see “Show Exact Duplicate Items”.  Click that.

Before you start deleting things, make sure the Bit Rate column is visible.  Now, carefully go through these files and delete the actual duplicates, keeping the one with the highest bit rate.  You need to double check to make sure each one is actually a duplicate and not just a really similar song but the “Show Exact Duplicate Items” does a pretty good job.  When you delete the song from your library, make sure you also send the file to the trash so you don’t waste hard drive space on unused files.

Once you have gone through this and gotten rid of all the exact duplicates, if you want you can go back through and use the “Show Duplicate Items”.  Sometimes this can help you find duplicate files that are titled exactly the same thing but it casts a pretty broad net as you’ll see so be careful. 

Step 4: Genre your music

Okay so now we’re getting pretty advanced here but you want your music organized right?  This is how you get it done.  Go through all your songs and give them a genre.  This is hard for two reasons.  First, you probably have a lot of music and almost none of it has genres that make sense.  That’s okay.  This just takes time.  So just do a little everyday and while you’re at it correct artist, track name, and album title if need be.

The second reason it’s hard is because , “What the hell genre is this song?”  Right.  The best music is typically great because it transcends specific labels.  The way to think about this is in terms of big boxes.  Pick a few really broad genres.  I use the following: Acoustic, Electronica, Hip Hop, Jazz, and Rock.  Assign one of these to each and every song in your library.  And if there is some more specific genre that a certain song definitely fits in you can just put a backslash and add the other labels you’d like.  For example, I have a ton of drum and bass so all those tracks are genred “Electronica \ DnB” or all my bluegrass is “Acoustic \ Bluegrass” or Bad Religion and NOFX are “Rock \ Punk”. You get the idea.

Since you are only going with five or so main genres, it will be really easy check your progress if you use the column browser.  The column browser is a highly recommended featured that I became obsessed with after my time with Foobar.  You can turn it on in the View menu.  It makes selecting and playing different segments of your library so easy.  I recommend using the columns Genre, Groupings, Artists, and Albums. The reason for the Groupings column will be explained in the later on.

Does this run into problems? Sure.  For example what about Bela Fleck?  He’s definitely got some bluegrass but he’s not always acoustic.  All of an artist or album doesn’t need to be in the same genre but just for the record, I have most of Bela Fleck as “Jazz \ Bluegrass”.  That way he comes up when I want any type of Bluegrass or only Jazz and not all the other acoustic stuff.  Just pick something and go with it.  We’re gonna give all the songs star ratings next so you’ll always be able to find the best songs.

Step 5: Rate your music

Wait, did he just say we going to rate to every single song?  Yes I did.  It’s going to take time. I am still not done but I am making progress.  Remember your music library isn’t going anywhere.  It will actually outlive you.  So think of this as a long term project and something you will pass on to your children. Just put a little work in each day and long before you are actually done, you will start seeing how much better it makes your listening experience.

Now while this may sound painful, I’ve got a system going that actually makes it pretty fun.  You are going to listen to so many good songs that you totally forgot about.  And even better, if you put your whole library on shuffle you’re never going to hear any of the songs you really hate or any of those stupid interludes they used to put all over hip hop albums.  It’s awesome. 

First off, you need to decide on your criteria for rating songs.  This is tough.  The most important thing is to be consistent so once you pick a method you stick with it.  I think I have a pretty good system worked out but go with whatever you think is best.  You have five star ratings to choose from but I only really use three for songs that I want to hear.  The breakdown is as follows

1 Star – There is some problem with the file that needs fixing

2 Stars – I never want to hear this song play ever

3 Stars – I only want to hear this song if I am listening to this specific album

4 Stars – I want to hear this song when I play this genre

5 Stars – I want to hear this song

Once you have a system set up, it’s time to go to town and rate away.  To make rating song as much like standard listening I recommend the use of Smart Playlists.  Make a Playlist Folder called “Unrated”.  In that folder make one Smart Playlists that is all the songs with no stars, and then another one for songs with no stars and “Acoustic” in the genre, and another for “Hip Hop” and so on.  This way if you want to listen to some hip hop, you can just pull up your “Unrated Hip Hop” Smart Playlist, put it on shuffle and listen away.  One note, I recommend disabling the Live Update feature in the Smart Playlists.  If you don’t, as soon as you click a rating the song disappears from the playlists.  That’s all well and good unless of course you clicked the wrong rating.  Then it’s a pain to go find the track and correct it.  (I used to just use itunes DJ for this which was awesome but because it was awesome Apple deleted it)

To really be able to rate all your music however you are likely going to need to do some rating when you are away from your computer.  This is where the syncing with your phone comes in.  Make a playlist folder called “For Phone”.  You can now make the same Smart Playlists as in the Unrated folder but here you may want to limit the number of songs in each playlist since you have limited space on your phone.

Depending on the version of itunes you are using, you may start seeing songs with hollow or unfilled stars.  So you know, you did not rate these songs.  This is a really annoying problem with itunes where it assigns an album rating (different than song rating) based on the average of the rated songs on the album and then applies that average rating as the song rating to all the unrated songs on the album as hollow stars.  The problem is Smart Playlists don’t distinguish between filled and hollow stars so tracks you have not rated will appear as rated in your Smart Playlists.  This is a pain and if you search the web there are millions of people complaining about this.  My suggestion, since you are going to rate all the songs eventually anyway just listen to the tracks with hollow stars and rate them.  Your rating always over rules the computed average.  Sometimes I think Apple makes things difficult on purpose.  

Step 6: Save your work

You have now rated tons of songs and you want to make sure you’re able to back that information up.  The problem with the itunes star rating is that it is not written to the file tag itself.  Instead it is saved in the itunes library file.  You can obviously just back up the library file but who knows if Apple is going to change something later that screws this up and you lose all your hard work.  Or what if you want to add the track to a library on a different computer but you want to keep the rating?

There is an easy fix.  Most of the other tag fields that you can modify in itunes are actually written to the individual file tag itself.  We are going to use Grouping because it is written to the file tag and can be viewed in the Column Browser (star rating can’t for some stupid reason).  The most efficient way to do this is to set up a Smart Playlist of songs that have not had the Grouping applied.  Make a Smart Playlist and call it “Needs Grouping”.  Define one criterion to select only songs with star rating between 1 and 5 so unrated songs are left out.  Define a second criterion to select only songs with the Grouping blank. 

You can perform two operations with this playlist.  First, sort it by star rating.  Now highlight all the 1 and 2 star items and right click and “Uncheck Selection”.  When files are unchecked they will only play if you double click them.  In other words, when you hit shuffle on your entire library these tracks will never play so no more annoying hip hop interludes or Chipmunk songs interrupting you music listening experience.  Some people ask why I don’t just delete these tracks.  Hard disk space is cheap and I’m a collector so as long as it’s organized, why not keep it?

The second operation is to apply the Grouping.  Highlight all the 5 star (filled star) songs.  Right click Get Info and agree that you want to edit info more multiple items.  In the Grouping field type “5 Stars” and click okay.  Repeat for 4, 3, 2, and 1 Star tracks.  Now the rating is saved to the individual file tag.  As for what to do with the hollow stars (computed rating based on album average), if you just rate them yourself the problem goes away.  You’re planning on rating all the songs eventually so no time like the present.

Step 7: Enjoy the fruits of you labor with Smart Playlists

The whole point of this process is to allow you to more easily enjoy the massive music collection.  Smart Playlists are a great way to do this.  I encourage you to try all different combinations involving things like “Date Added” or “Play count” but for starters here are a couple I would recommend.  First, you want a Smart Playlist of all the 5 Star tracks.  Maybe you make another one for all the 4 and 5 Star tracks.  You can also make some good genre mixes by making Smart Playlists that have all the 4+ Star tracks for a specific genre.  Remember that was our criterion for 4 Stars?  I also have versions of these specifically for my iphone where I have limited the number of songs per list due to the space limitation.

You are going to want to set these up early so you can watch them grow as you continue the task of applying genre and rating.  As you organize more of you library, these Smart Playlists will get better and motivate you to keep going.  So, get to work and get to enjoying your entire music library.

Step 8: Take your library with you

So you love your new organized library and you want to be able to play it when you’re over at a buddy’s house.  Sure you can bring some songs on a USB stick but you want the whole genre, star rating, smart playlist thing you’ve got going when you open itunes on your home computer.  If you have a portable external hard drive you can do this.

You will need to copy the whole Itunes directory (not just the subfolder with your music files) to the external drive.  You can do a simple drag and drop operation but when transferring 100’s of gigs it seems like something always goes wrong.  If you’re on a pc I recommend using robocopy.  It logs everything so you know that everything has copies and can be restarted if for some reason the transfer stops.  It is a command line operation so open a command prompt and type in the command below substituting the directories in <> for your setup.   You can check the internet for the meaning of each of the tags but it’s what you want.

robocopy /E /tee /Z /R:3 /W:10 /xo /log: Music_to_USB.txt

With everything on your external drive, head over to your buddy’s place.  Plug in your drive and while you open itunes, hold down the shift key.  A dialog box will pop up asking you to “Choose the itunes library”.  Click choose and navigate to your itunes folder d:\music\itunes\ for example and select the itunes library file “iTunes Library.itl”.  It will take a while for itunes to load but when it does it will be the exact setup you have at home.

Bonus: How I prepare the SXSW mixes

First I download the unofficial torrent file containing all the songs which are posted as artist samples on the official SXSW website. (www.sxswtorrent.com)  If you listen to my podcast, you know I am all about buying the music and supporting the artists so why get all these songs free?  Simply because these artists are all unknown and I will never hear of them without this.  If I like this one song that I get for free, you can be sure I will seek out and purchase their full album.

Next I make a new playlist folder called “SXSW 2013” for example and make 6 smart playlists.  Each playlist has two criteria.  Each song must have the album title “SXSW 2013 Showcasing Artists”.  This is the album title for all the songs in the torrent.  Next each playlist respectively requires a specific star rating 0, 1, 2, 3, 4, or 5.  This is the system by which I make the first cut and simultaneously do a rough sorting of the tracks.  The playlist with 0 stars is obviously unrated.  1 star tracks will be trash.  2 star tracks are acoustic, 3 star tracks are electronica, 4 star tracks are hip hop, and 5 star tracks are rock.  Like I said, it is a rough sorting.  With that set up, I load the 0 star playlist on my phone and listen to every single song sorting them it into one of these 5 categories as I go.

If you start having problems with hollow stars kicking unrated song out of your Smart Playlist for Unrated songs, just select all the hollow star tracks (unrated) and change the album title.  Just add a 1 on the end or something.  Then there will be no album average.  You will need to go back and adjust the Unrated Smart Playlist too.  I know I know.  It’s dumb.   I think itunes 11 may have fixed this problem but they got rid of DJ so…  This may be a pain but it works.

With all the songs actually rated (no hollow stars!) and in one of the five starred playlists, I now apply a genre.  All the songs in the trash playlists are labeled trash, acoustic labeled acoustic, and so on.  With the genres defined, I clear all the star ratings and adjust the smart playlists to require instead of a star rating, the specific genre.  Now I can focus on each subset.  I listen to all the songs in each genre and split them up into five more groupings by star rating.  The definition of these subgroups gets a little tricky.  Usually a couple are obvious like under hip hop I have a crunk subset, a back pack rap set,  maybe a dj set but that leaves two more undefined.  This is the feel part of the process.  I do still usually reserve 1 star in each of these sets as the cutting room floor.

After that, there is just some tweeking to make sure mixes are around the same length and that the order of tracks in a playlist sounds right.  It’s a lot of work but it’s probably my favorite musical project of the year.  With the playlists finalized, I mix the tracks together in one audio file, I use Illustrator to make the track listings, and this year I may actually go through and use Garage Band to place chapters so you can skip songs.  Maybe.