Hello again Jeremy, well it depend where you are from... Canada,US...
If you want to buy online, there is Digikey or Amazon or eBay, otherwise, you can easily find the switch and resistor in an electronic repair shop or in a RadioShack if you are in the States. Remember that the push button switch is a momentary switch, and not an ON / OFF switch.
And for the pen eraser... any office supply stores should carry them.
Hope that help...
Cheers
Dan