I made a powered speaker from an old toolbox, along with my DMG is the cheapest music setup ever (or is it?...)
Details of the project are at the flickr link (below), but at the heart of it is a $8 amplifier from Adafruit which works great for these super cheap powered speaker setups. You can get the speakers for like $20 at Target or Amazon. Not super loud but has better base than most mp3 amp/speakers I've had/tried and with a little soldering you can make the thing yourself. I use a rechargable battery but I tested it using 3 AA batteries and it works fine. The amplifier is a class D amp but sounds great and is very efficient.
-Arduinerd