Personally, I think a shelf full of colorful books is one of the most inviting objects in a room, the first thing which catches my eye. I recently read an article from the Wall Street Journal about decorating with books. The focus of the article was based on recent interior decorating trends of turning the books around on the shelf so you just see the pages, backwards books.
If you don't like a rainbow of color for you book shelf, an option is to find books with spines all the same color. Or you could choose 2 different colors and add other vintage goodies as accents. It doesn't need to be the whole shelf or several shelves. A grouping of 3 or 4 books makes a beautiful decorative display. Stack them, tie a piece of jute around them, add some greenery, whatever makes your idea of a warm inviting display.