If you have a computer then you don't need much more to make simple recordings. A good audio interface is the most important component (I have an M-Audio FastTrack Ultra which is a terrific mid-range unit).
If you have a PC you can pick up Reaper for $40 and have a studio-grade DAW at your disposal. If you have a Mac you have GarageBand which is free. The trick is amplification. If you are using a physical amp, then you have to mic it, which is a bit of an art in itself. But even that's not too hard.
I prefer going the all virtual route. No physical amps for me anymore. I either use AmpliTube as a DAW plugin, or feed the output of my Axe-FX into the audio interface and let it do all the work. Either way I get perfect tone and I don't have to worry about crappy room acoustics, bad mic placement, volume compromises, etc.
Unfortunately, I have perfect gear, and now the weakest link is my playing ability. No more blaming the equipment and no more excuses...