The switching supply doesn't isolate the batteries from the socket, the socket has a contact inside which isolates the battery when a plug is inserted (plus a diode to prevent the battery supply bleeding back into the socket)
Yes, you can add a LiPo, a charging circuit, an undervoltage protection circuit and you'll be good to go! Except you'll be running the GameBoy from 4.2 down to 3.7volts which may cause more switching noise, contrast flicker etc... I think there was a test done with the gameboy's output voltage vs input voltage, Nitro2K01 did it AFAIK.
You shouldn't run a LiPo down to its protection circuit cutoff (usually 3.0-3.1volts) this reduces its life a LOT. A custom circuit that isolates at 3.6 is a far better idea though will add complexity.
Alternately, buy a 'battery bank' for charging a mobile phone and use that directly. Pull it apart to use its PCB and if the larger LiPo cell won't fit, replace it with one from hobbyking that will. That has your charging circuit, a boost converter to generate 5v (you can modify it to output 6v easily), undervoltage protection and at the current draw the GB operates at, will be very quiet electrically.