Now normally I am an album guy all the way (which is one of the reasons I haven't purchased an iPod yet), but I definitely think there is room in any good CD collection for rarities compilations.
It's different to "Greatest Hits" or "Best Of" releases which essentially become redundant when you end up liking the band and buying all of their albums. Not only does it become redundant because you end up having multiple copies of the same track, but it can deter from the enjoyment of the proper albums when you get them, because you already know so many songs from them.
With classic British band The Jam, I made the mistake of buying two (!) compilation albums before buying any of their albums. The first one I got was Beat Surrender, a sub-par compilation of pretty average Jam tracks with a few exceptions like the brilliant A town called malice and That's entertainment. I should have gone for one of their albums at this point, but instead I stupidly decided to go for another compilation, The Jam Collection. This was a much better compilation, but by the time I got around to purchasing their best album All mod cons, I already had six out of the twelve tracks on it! Considering one of the remaining six tracks was a Kinks cover (David Watts), it definitely detracted from my initial enjoyment of the album. It's like seeing bits of a movie in a hotel room at different times, and then trying to watch it all from start to finish. Why not just watch it from start to finish up front?
But, I digress. I don't see B-Sides and Rarities compilations as having the same issues as best-of compilations. Firstly, there's generally the lack of redundancy. Even if there are album tracks on it, they are generally scarce and usually they give you a different version than the album track (e.g. a demo or acoustic version). Listening to the first disc of the Nick Cave set this morning, there were acoustic versions of two of his classic cuts - Deanna and The mercy seat. I have both of these on the Best of Nick Cave & The Bad Seeds CD, but it's great hearing these different versions of the songs. Whether they are works in progress, or just alternate versions, they give you a window into the artistic process that the artist goes through when recording music.
I think Nick Cave is one of those artists who suits the "rarities" compilation format really well. His music has such a rustic and shambolic feel to it, and hearing bits and pieces on a compilation like this is a great way to experience his music. There's also an alternate version of The moon is in the gutter from his debut album From her to eternity. I always loved this song when listening to that album even though I wasn't a huge fan of the album as a whole. But hearing the alternate version on the rarities compilation made me start to appreciate the song again.
Finally, here are some of my favourite "B-side and rarities" compilations:
Oasis - The Masterplan
Quite honestly, this B-Sides compilation has some of Oasis' best ever songs on it. Their B-Sides are better than many band's A-Sides. And the trilogy of Listen up, Rockin' chair and Half the world away could be the best set of three consecutive songs that I know on almost any album.
Suede - Sci-fi Lullabies
In hindsight, disc two is pretty shite (with a few exceptions). But disc one has many of the best Suede songs ever released. Some of my personal favourites are To the birds, and The living dead and the best of the bunch -- the utterly beautiful My dark star.
The Smiths - Louder than bombs
Crowded House - Afterglow
Beatles - Past Masters Volume One & Two
Manic Street Preachers - Lipstick Traces
[A secret history of]
The Stone Roses - Turns into stone