You can drop in a $4 boost converter and won't have to worry about battery quality/charge.
Are you sure? The GBP (like all GBs) has a switching voltage converter built-in. If the voltage cut-off of Nintendo's converter is set too high, it may stop batteries from working before they're fully discharged, and then a boost converter would help. But I thought the problem with the EMS cart was that it's using excessive power, so that it both drains the battery faster, and is using more power than a typical AAA battery can deliver when partially discharged due to ESR. In that case, no boost converter in the world would help.
In other news, I had a spare Li-ion 3.7V, 810mAh/2.997Wh which I jumpered up to the Pocket's terminals and it worked better than I expected. Is there a reason I can't/shouldn't put a single Li-ion AAA size cell with a dummy cell in the back?
That should work, better than using AAA batteries in fact. One thing to keep in mind: If it doesn't have protection circuitry built-in, you may want to be careful not to over-discharge it, or accidentally short it.