As a person who primarily modifies hardware for fun (and profit), I find I use my unmodified equipment just as much, sometimes more.
A Game Boy Color is probably your best starting point; you can get one with LSDj and a flash cart for under $100 USD.
This, of course, doesn't preclude the wonderful option that is an LSDj license and a copy of BGB. You can get that set up going for the price of your generosity and gratitude to Johan, expressed in donation form.